Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1 | |
| 2 | :mod:`weakref` --- Weak references |
| 3 | ================================== |
| 4 | |
| 5 | .. module:: weakref |
| 6 | :synopsis: Support for weak references and weak dictionaries. |
| 7 | .. moduleauthor:: Fred L. Drake, Jr. <fdrake@acm.org> |
| 8 | .. moduleauthor:: Neil Schemenauer <nas@arctrix.com> |
| 9 | .. moduleauthor:: Martin von Lรถwis <martin@loewis.home.cs.tu-berlin.de> |
| 10 | .. sectionauthor:: Fred L. Drake, Jr. <fdrake@acm.org> |
| 11 | |
| 12 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 13 | The :mod:`weakref` module allows the Python programmer to create :dfn:`weak |
| 14 | references` to objects. |
| 15 | |
| 16 | .. % When making changes to the examples in this file, be sure to update |
| 17 | .. % Lib/test/test_weakref.py::libreftest too! |
| 18 | |
| 19 | In the following, the term :dfn:`referent` means the object which is referred to |
| 20 | by a weak reference. |
| 21 | |
| 22 | A weak reference to an object is not enough to keep the object alive: when the |
| 23 | only remaining references to a referent are weak references, garbage collection |
| 24 | is free to destroy the referent and reuse its memory for something else. A |
| 25 | primary use for weak references is to implement caches or mappings holding large |
| 26 | objects, where it's desired that a large object not be kept alive solely because |
| 27 | it appears in a cache or mapping. For example, if you have a number of large |
| 28 | binary image objects, you may wish to associate a name with each. If you used a |
| 29 | Python dictionary to map names to images, or images to names, the image objects |
| 30 | would remain alive just because they appeared as values or keys in the |
| 31 | dictionaries. The :class:`WeakKeyDictionary` and :class:`WeakValueDictionary` |
| 32 | classes supplied by the :mod:`weakref` module are an alternative, using weak |
| 33 | references to construct mappings that don't keep objects alive solely because |
| 34 | they appear in the mapping objects. If, for example, an image object is a value |
| 35 | in a :class:`WeakValueDictionary`, then when the last remaining references to |
| 36 | that image object are the weak references held by weak mappings, garbage |
| 37 | collection can reclaim the object, and its corresponding entries in weak |
| 38 | mappings are simply deleted. |
| 39 | |
| 40 | :class:`WeakKeyDictionary` and :class:`WeakValueDictionary` use weak references |
| 41 | in their implementation, setting up callback functions on the weak references |
| 42 | that notify the weak dictionaries when a key or value has been reclaimed by |
| 43 | garbage collection. Most programs should find that using one of these weak |
| 44 | dictionary types is all they need -- it's not usually necessary to create your |
| 45 | own weak references directly. The low-level machinery used by the weak |
| 46 | dictionary implementations is exposed by the :mod:`weakref` module for the |
| 47 | benefit of advanced uses. |
| 48 | |
| 49 | Not all objects can be weakly referenced; those objects which can include class |
| 50 | instances, functions written in Python (but not in C), methods (both bound and |
| 51 | unbound), sets, frozensets, file objects, generators, type objects, DBcursor |
| 52 | objects from the :mod:`bsddb` module, sockets, arrays, deques, and regular |
| 53 | expression pattern objects. |
| 54 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 55 | Several builtin types such as :class:`list` and :class:`dict` do not directly |
| 56 | support weak references but can add support through subclassing:: |
| 57 | |
| 58 | class Dict(dict): |
| 59 | pass |
| 60 | |
| 61 | obj = Dict(red=1, green=2, blue=3) # this object is weak referencable |
| 62 | |
| 63 | Extension types can easily be made to support weak references; see |
| 64 | :ref:`weakref-support`. |
| 65 | |
| 66 | |
| 67 | .. class:: ref(object[, callback]) |
| 68 | |
| 69 | Return a weak reference to *object*. The original object can be retrieved by |
| 70 | calling the reference object if the referent is still alive; if the referent is |
| 71 | no longer alive, calling the reference object will cause :const:`None` to be |
| 72 | returned. If *callback* is provided and not :const:`None`, and the returned |
| 73 | weakref object is still alive, the callback will be called when the object is |
| 74 | about to be finalized; the weak reference object will be passed as the only |
| 75 | parameter to the callback; the referent will no longer be available. |
| 76 | |
| 77 | It is allowable for many weak references to be constructed for the same object. |
| 78 | Callbacks registered for each weak reference will be called from the most |
| 79 | recently registered callback to the oldest registered callback. |
| 80 | |
| 81 | Exceptions raised by the callback will be noted on the standard error output, |
| 82 | but cannot be propagated; they are handled in exactly the same way as exceptions |
| 83 | raised from an object's :meth:`__del__` method. |
| 84 | |
| 85 | Weak references are hashable if the *object* is hashable. They will maintain |
| 86 | their hash value even after the *object* was deleted. If :func:`hash` is called |
| 87 | the first time only after the *object* was deleted, the call will raise |
| 88 | :exc:`TypeError`. |
| 89 | |
| 90 | Weak references support tests for equality, but not ordering. If the referents |
| 91 | are still alive, two references have the same equality relationship as their |
| 92 | referents (regardless of the *callback*). If either referent has been deleted, |
| 93 | the references are equal only if the reference objects are the same object. |
| 94 | |
Georg Brandl | 55ac8f0 | 2007-09-01 13:51:09 +0000 | [diff] [blame] | 95 | This is a subclassable type rather than a factory function. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 96 | |
| 97 | |
| 98 | .. function:: proxy(object[, callback]) |
| 99 | |
| 100 | Return a proxy to *object* which uses a weak reference. This supports use of |
| 101 | the proxy in most contexts instead of requiring the explicit dereferencing used |
| 102 | with weak reference objects. The returned object will have a type of either |
| 103 | ``ProxyType`` or ``CallableProxyType``, depending on whether *object* is |
| 104 | callable. Proxy objects are not hashable regardless of the referent; this |
| 105 | avoids a number of problems related to their fundamentally mutable nature, and |
| 106 | prevent their use as dictionary keys. *callback* is the same as the parameter |
| 107 | of the same name to the :func:`ref` function. |
| 108 | |
| 109 | |
| 110 | .. function:: getweakrefcount(object) |
| 111 | |
| 112 | Return the number of weak references and proxies which refer to *object*. |
| 113 | |
| 114 | |
| 115 | .. function:: getweakrefs(object) |
| 116 | |
| 117 | Return a list of all weak reference and proxy objects which refer to *object*. |
| 118 | |
| 119 | |
| 120 | .. class:: WeakKeyDictionary([dict]) |
| 121 | |
| 122 | Mapping class that references keys weakly. Entries in the dictionary will be |
| 123 | discarded when there is no longer a strong reference to the key. This can be |
| 124 | used to associate additional data with an object owned by other parts of an |
| 125 | application without adding attributes to those objects. This can be especially |
| 126 | useful with objects that override attribute accesses. |
| 127 | |
| 128 | .. note:: |
| 129 | |
| 130 | Caution: Because a :class:`WeakKeyDictionary` is built on top of a Python |
| 131 | dictionary, it must not change size when iterating over it. This can be |
| 132 | difficult to ensure for a :class:`WeakKeyDictionary` because actions performed |
| 133 | by the program during iteration may cause items in the dictionary to vanish "by |
| 134 | magic" (as a side effect of garbage collection). |
| 135 | |
| 136 | :class:`WeakKeyDictionary` objects have the following additional methods. These |
| 137 | expose the internal references directly. The references are not guaranteed to |
| 138 | be "live" at the time they are used, so the result of calling the references |
| 139 | needs to be checked before being used. This can be used to avoid creating |
| 140 | references that will cause the garbage collector to keep the keys around longer |
| 141 | than needed. |
| 142 | |
| 143 | |
| 144 | .. method:: WeakKeyDictionary.iterkeyrefs() |
| 145 | |
| 146 | Return an iterator that yields the weak references to the keys. |
| 147 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 148 | |
| 149 | .. method:: WeakKeyDictionary.keyrefs() |
| 150 | |
| 151 | Return a list of weak references to the keys. |
| 152 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 153 | |
| 154 | .. class:: WeakValueDictionary([dict]) |
| 155 | |
| 156 | Mapping class that references values weakly. Entries in the dictionary will be |
| 157 | discarded when no strong reference to the value exists any more. |
| 158 | |
| 159 | .. note:: |
| 160 | |
| 161 | Caution: Because a :class:`WeakValueDictionary` is built on top of a Python |
| 162 | dictionary, it must not change size when iterating over it. This can be |
| 163 | difficult to ensure for a :class:`WeakValueDictionary` because actions performed |
| 164 | by the program during iteration may cause items in the dictionary to vanish "by |
| 165 | magic" (as a side effect of garbage collection). |
| 166 | |
| 167 | :class:`WeakValueDictionary` objects have the following additional methods. |
| 168 | These method have the same issues as the :meth:`iterkeyrefs` and :meth:`keyrefs` |
| 169 | methods of :class:`WeakKeyDictionary` objects. |
| 170 | |
| 171 | |
| 172 | .. method:: WeakValueDictionary.itervaluerefs() |
| 173 | |
| 174 | Return an iterator that yields the weak references to the values. |
| 175 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 176 | |
| 177 | .. method:: WeakValueDictionary.valuerefs() |
| 178 | |
| 179 | Return a list of weak references to the values. |
| 180 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 181 | |
| 182 | .. data:: ReferenceType |
| 183 | |
| 184 | The type object for weak references objects. |
| 185 | |
| 186 | |
| 187 | .. data:: ProxyType |
| 188 | |
| 189 | The type object for proxies of objects which are not callable. |
| 190 | |
| 191 | |
| 192 | .. data:: CallableProxyType |
| 193 | |
| 194 | The type object for proxies of callable objects. |
| 195 | |
| 196 | |
| 197 | .. data:: ProxyTypes |
| 198 | |
| 199 | Sequence containing all the type objects for proxies. This can make it simpler |
| 200 | to test if an object is a proxy without being dependent on naming both proxy |
| 201 | types. |
| 202 | |
| 203 | |
| 204 | .. exception:: ReferenceError |
| 205 | |
| 206 | Exception raised when a proxy object is used but the underlying object has been |
| 207 | collected. This is the same as the standard :exc:`ReferenceError` exception. |
| 208 | |
| 209 | |
| 210 | .. seealso:: |
| 211 | |
| 212 | :pep:`0205` - Weak References |
| 213 | The proposal and rationale for this feature, including links to earlier |
| 214 | implementations and information about similar features in other languages. |
| 215 | |
| 216 | |
| 217 | .. _weakref-objects: |
| 218 | |
| 219 | Weak Reference Objects |
| 220 | ---------------------- |
| 221 | |
| 222 | Weak reference objects have no attributes or methods, but do allow the referent |
| 223 | to be obtained, if it still exists, by calling it:: |
| 224 | |
| 225 | >>> import weakref |
| 226 | >>> class Object: |
| 227 | ... pass |
| 228 | ... |
| 229 | >>> o = Object() |
| 230 | >>> r = weakref.ref(o) |
| 231 | >>> o2 = r() |
| 232 | >>> o is o2 |
| 233 | True |
| 234 | |
| 235 | If the referent no longer exists, calling the reference object returns |
| 236 | :const:`None`:: |
| 237 | |
| 238 | >>> del o, o2 |
| 239 | >>> print r() |
| 240 | None |
| 241 | |
| 242 | Testing that a weak reference object is still live should be done using the |
| 243 | expression ``ref() is not None``. Normally, application code that needs to use |
| 244 | a reference object should follow this pattern:: |
| 245 | |
| 246 | # r is a weak reference object |
| 247 | o = r() |
| 248 | if o is None: |
| 249 | # referent has been garbage collected |
| 250 | print "Object has been deallocated; can't frobnicate." |
| 251 | else: |
| 252 | print "Object is still live!" |
| 253 | o.do_something_useful() |
| 254 | |
| 255 | Using a separate test for "liveness" creates race conditions in threaded |
| 256 | applications; another thread can cause a weak reference to become invalidated |
| 257 | before the weak reference is called; the idiom shown above is safe in threaded |
| 258 | applications as well as single-threaded applications. |
| 259 | |
| 260 | Specialized versions of :class:`ref` objects can be created through subclassing. |
| 261 | This is used in the implementation of the :class:`WeakValueDictionary` to reduce |
| 262 | the memory overhead for each entry in the mapping. This may be most useful to |
| 263 | associate additional information with a reference, but could also be used to |
| 264 | insert additional processing on calls to retrieve the referent. |
| 265 | |
| 266 | This example shows how a subclass of :class:`ref` can be used to store |
| 267 | additional information about an object and affect the value that's returned when |
| 268 | the referent is accessed:: |
| 269 | |
| 270 | import weakref |
| 271 | |
| 272 | class ExtendedRef(weakref.ref): |
| 273 | def __init__(self, ob, callback=None, **annotations): |
| 274 | super(ExtendedRef, self).__init__(ob, callback) |
| 275 | self.__counter = 0 |
| 276 | for k, v in annotations.iteritems(): |
| 277 | setattr(self, k, v) |
| 278 | |
| 279 | def __call__(self): |
| 280 | """Return a pair containing the referent and the number of |
| 281 | times the reference has been called. |
| 282 | """ |
| 283 | ob = super(ExtendedRef, self).__call__() |
| 284 | if ob is not None: |
| 285 | self.__counter += 1 |
| 286 | ob = (ob, self.__counter) |
| 287 | return ob |
| 288 | |
| 289 | |
| 290 | .. _weakref-example: |
| 291 | |
| 292 | Example |
| 293 | ------- |
| 294 | |
| 295 | This simple example shows how an application can use objects IDs to retrieve |
| 296 | objects that it has seen before. The IDs of the objects can then be used in |
| 297 | other data structures without forcing the objects to remain alive, but the |
| 298 | objects can still be retrieved by ID if they do. |
| 299 | |
| 300 | .. % Example contributed by Tim Peters. |
| 301 | |
| 302 | :: |
| 303 | |
| 304 | import weakref |
| 305 | |
| 306 | _id2obj_dict = weakref.WeakValueDictionary() |
| 307 | |
| 308 | def remember(obj): |
| 309 | oid = id(obj) |
| 310 | _id2obj_dict[oid] = obj |
| 311 | return oid |
| 312 | |
| 313 | def id2obj(oid): |
| 314 | return _id2obj_dict[oid] |
| 315 | |