Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1 | :mod:`weakref` --- Weak references |
| 2 | ================================== |
| 3 | |
| 4 | .. module:: weakref |
| 5 | :synopsis: Support for weak references and weak dictionaries. |
| 6 | .. moduleauthor:: Fred L. Drake, Jr. <fdrake@acm.org> |
| 7 | .. moduleauthor:: Neil Schemenauer <nas@arctrix.com> |
| 8 | .. moduleauthor:: Martin von Lรถwis <martin@loewis.home.cs.tu-berlin.de> |
| 9 | .. sectionauthor:: Fred L. Drake, Jr. <fdrake@acm.org> |
| 10 | |
Raymond Hettinger | 469271d | 2011-01-27 20:38:46 +0000 | [diff] [blame] | 11 | **Source code:** :source:`Lib/weakref.py` |
| 12 | |
| 13 | -------------- |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 14 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 15 | The :mod:`weakref` module allows the Python programmer to create :dfn:`weak |
| 16 | references` to objects. |
| 17 | |
Christian Heimes | 5b5e81c | 2007-12-31 16:14:33 +0000 | [diff] [blame] | 18 | .. When making changes to the examples in this file, be sure to update |
| 19 | Lib/test/test_weakref.py::libreftest too! |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 20 | |
| 21 | In the following, the term :dfn:`referent` means the object which is referred to |
| 22 | by a weak reference. |
| 23 | |
| 24 | A weak reference to an object is not enough to keep the object alive: when the |
Christian Heimes | d8654cf | 2007-12-02 15:22:16 +0000 | [diff] [blame] | 25 | only remaining references to a referent are weak references, |
| 26 | :term:`garbage collection` is free to destroy the referent and reuse its memory |
Antoine Pitrou | 9439f04 | 2012-08-21 00:07:07 +0200 | [diff] [blame] | 27 | for something else. However, until the object is actually destroyed the weak |
| 28 | reference may return the object even if there are no strong references to it. |
| 29 | |
| 30 | A primary use for weak references is to implement caches or |
Christian Heimes | d8654cf | 2007-12-02 15:22:16 +0000 | [diff] [blame] | 31 | mappings holding large objects, where it's desired that a large object not be |
Christian Heimes | fe337bf | 2008-03-23 21:54:12 +0000 | [diff] [blame] | 32 | kept alive solely because it appears in a cache or mapping. |
| 33 | |
| 34 | For example, if you have a number of large binary image objects, you may wish to |
| 35 | associate a name with each. If you used a Python dictionary to map names to |
| 36 | images, or images to names, the image objects would remain alive just because |
| 37 | they appeared as values or keys in the dictionaries. The |
| 38 | :class:`WeakKeyDictionary` and :class:`WeakValueDictionary` classes supplied by |
| 39 | the :mod:`weakref` module are an alternative, using weak references to construct |
| 40 | mappings that don't keep objects alive solely because they appear in the mapping |
| 41 | objects. If, for example, an image object is a value in a |
| 42 | :class:`WeakValueDictionary`, then when the last remaining references to that |
| 43 | image object are the weak references held by weak mappings, garbage collection |
| 44 | can reclaim the object, and its corresponding entries in weak mappings are |
| 45 | simply deleted. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 46 | |
| 47 | :class:`WeakKeyDictionary` and :class:`WeakValueDictionary` use weak references |
| 48 | in their implementation, setting up callback functions on the weak references |
| 49 | that notify the weak dictionaries when a key or value has been reclaimed by |
Georg Brandl | 3b8cb17 | 2007-10-23 06:26:46 +0000 | [diff] [blame] | 50 | garbage collection. :class:`WeakSet` implements the :class:`set` interface, |
| 51 | but keeps weak references to its elements, just like a |
| 52 | :class:`WeakKeyDictionary` does. |
| 53 | |
Richard Oudkerk | 7a3dae0 | 2013-05-05 23:05:00 +0100 | [diff] [blame] | 54 | :class:`finalize` provides a straight forward way to register a |
| 55 | cleanup function to be called when an object is garbage collected. |
| 56 | This is simpler to use than setting up a callback function on a raw |
Nick Coghlan | be57ab8 | 2013-09-22 21:26:30 +1000 | [diff] [blame] | 57 | weak reference, since the module automatically ensures that the finalizer |
| 58 | remains alive until the object is collected. |
Richard Oudkerk | 7a3dae0 | 2013-05-05 23:05:00 +0100 | [diff] [blame] | 59 | |
| 60 | Most programs should find that using one of these weak container types |
| 61 | or :class:`finalize` is all they need -- it's not usually necessary to |
| 62 | create your own weak references directly. The low-level machinery is |
| 63 | exposed by the :mod:`weakref` module for the benefit of advanced uses. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 64 | |
| 65 | Not all objects can be weakly referenced; those objects which can include class |
Georg Brandl | 2e0b755 | 2007-11-27 12:43:08 +0000 | [diff] [blame] | 66 | instances, functions written in Python (but not in C), instance methods, sets, |
Antoine Pitrou | 11cb961 | 2010-09-15 11:11:28 +0000 | [diff] [blame] | 67 | frozensets, some :term:`file objects <file object>`, :term:`generator`\s, type |
| 68 | objects, sockets, arrays, deques, regular expression pattern objects, and code |
| 69 | objects. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 70 | |
Benjamin Peterson | bec4d57 | 2009-10-10 01:16:07 +0000 | [diff] [blame] | 71 | .. versionchanged:: 3.2 |
Collin Winter | 4222e9c | 2010-03-18 22:46:40 +0000 | [diff] [blame] | 72 | Added support for thread.lock, threading.Lock, and code objects. |
Benjamin Peterson | bec4d57 | 2009-10-10 01:16:07 +0000 | [diff] [blame] | 73 | |
Georg Brandl | 22b3431 | 2009-07-26 14:54:51 +0000 | [diff] [blame] | 74 | Several built-in types such as :class:`list` and :class:`dict` do not directly |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 75 | support weak references but can add support through subclassing:: |
| 76 | |
| 77 | class Dict(dict): |
| 78 | pass |
| 79 | |
Christian Heimes | c3f30c4 | 2008-02-22 16:37:40 +0000 | [diff] [blame] | 80 | obj = Dict(red=1, green=2, blue=3) # this object is weak referenceable |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 81 | |
Benjamin Peterson | 905982b | 2010-05-08 15:26:30 +0000 | [diff] [blame] | 82 | Other built-in types such as :class:`tuple` and :class:`int` do not support weak |
| 83 | references even when subclassed (This is an implementation detail and may be |
| 84 | different across various Python implementations.). |
Georg Brandl | ff8c1e5 | 2009-10-21 07:17:48 +0000 | [diff] [blame] | 85 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 86 | Extension types can easily be made to support weak references; see |
| 87 | :ref:`weakref-support`. |
| 88 | |
| 89 | |
| 90 | .. class:: ref(object[, callback]) |
| 91 | |
| 92 | Return a weak reference to *object*. The original object can be retrieved by |
| 93 | calling the reference object if the referent is still alive; if the referent is |
| 94 | no longer alive, calling the reference object will cause :const:`None` to be |
| 95 | returned. If *callback* is provided and not :const:`None`, and the returned |
| 96 | weakref object is still alive, the callback will be called when the object is |
| 97 | about to be finalized; the weak reference object will be passed as the only |
| 98 | parameter to the callback; the referent will no longer be available. |
| 99 | |
| 100 | It is allowable for many weak references to be constructed for the same object. |
| 101 | Callbacks registered for each weak reference will be called from the most |
| 102 | recently registered callback to the oldest registered callback. |
| 103 | |
| 104 | Exceptions raised by the callback will be noted on the standard error output, |
| 105 | but cannot be propagated; they are handled in exactly the same way as exceptions |
| 106 | raised from an object's :meth:`__del__` method. |
| 107 | |
Georg Brandl | 7f01a13 | 2009-09-16 15:58:14 +0000 | [diff] [blame] | 108 | Weak references are :term:`hashable` if the *object* is hashable. They will |
| 109 | maintain their hash value even after the *object* was deleted. If |
| 110 | :func:`hash` is called the first time only after the *object* was deleted, |
| 111 | the call will raise :exc:`TypeError`. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 112 | |
| 113 | Weak references support tests for equality, but not ordering. If the referents |
| 114 | are still alive, two references have the same equality relationship as their |
| 115 | referents (regardless of the *callback*). If either referent has been deleted, |
| 116 | the references are equal only if the reference objects are the same object. |
| 117 | |
Georg Brandl | 55ac8f0 | 2007-09-01 13:51:09 +0000 | [diff] [blame] | 118 | This is a subclassable type rather than a factory function. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 119 | |
Mark Dickinson | 556e94b | 2013-04-13 15:45:44 +0100 | [diff] [blame] | 120 | .. attribute:: __callback__ |
| 121 | |
| 122 | This read-only attribute returns the callback currently associated to the |
| 123 | weakref. If there is no callback or if the referent of the weakref is |
| 124 | no longer alive then this attribute will have value ``None``. |
| 125 | |
Richard Oudkerk | 7a3dae0 | 2013-05-05 23:05:00 +0100 | [diff] [blame] | 126 | .. versionchanged:: 3.4 |
Mark Dickinson | 9b6fdf8 | 2013-04-13 16:09:18 +0100 | [diff] [blame] | 127 | Added the :attr:`__callback__` attribute. |
Mark Dickinson | 556e94b | 2013-04-13 15:45:44 +0100 | [diff] [blame] | 128 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 129 | |
| 130 | .. function:: proxy(object[, callback]) |
| 131 | |
| 132 | Return a proxy to *object* which uses a weak reference. This supports use of |
| 133 | the proxy in most contexts instead of requiring the explicit dereferencing used |
| 134 | with weak reference objects. The returned object will have a type of either |
| 135 | ``ProxyType`` or ``CallableProxyType``, depending on whether *object* is |
Guido van Rossum | 2cc30da | 2007-11-02 23:46:40 +0000 | [diff] [blame] | 136 | callable. Proxy objects are not :term:`hashable` regardless of the referent; this |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 137 | avoids a number of problems related to their fundamentally mutable nature, and |
| 138 | prevent their use as dictionary keys. *callback* is the same as the parameter |
| 139 | of the same name to the :func:`ref` function. |
| 140 | |
| 141 | |
| 142 | .. function:: getweakrefcount(object) |
| 143 | |
| 144 | Return the number of weak references and proxies which refer to *object*. |
| 145 | |
| 146 | |
| 147 | .. function:: getweakrefs(object) |
| 148 | |
| 149 | Return a list of all weak reference and proxy objects which refer to *object*. |
| 150 | |
| 151 | |
| 152 | .. class:: WeakKeyDictionary([dict]) |
| 153 | |
| 154 | Mapping class that references keys weakly. Entries in the dictionary will be |
| 155 | discarded when there is no longer a strong reference to the key. This can be |
| 156 | used to associate additional data with an object owned by other parts of an |
| 157 | application without adding attributes to those objects. This can be especially |
| 158 | useful with objects that override attribute accesses. |
| 159 | |
| 160 | .. note:: |
| 161 | |
Christian Heimes | fe337bf | 2008-03-23 21:54:12 +0000 | [diff] [blame] | 162 | Caution: Because a :class:`WeakKeyDictionary` is built on top of a Python |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 163 | dictionary, it must not change size when iterating over it. This can be |
Christian Heimes | fe337bf | 2008-03-23 21:54:12 +0000 | [diff] [blame] | 164 | difficult to ensure for a :class:`WeakKeyDictionary` because actions |
| 165 | performed by the program during iteration may cause items in the |
| 166 | dictionary to vanish "by magic" (as a side effect of garbage collection). |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 167 | |
| 168 | :class:`WeakKeyDictionary` objects have the following additional methods. These |
| 169 | expose the internal references directly. The references are not guaranteed to |
| 170 | be "live" at the time they are used, so the result of calling the references |
| 171 | needs to be checked before being used. This can be used to avoid creating |
| 172 | references that will cause the garbage collector to keep the keys around longer |
| 173 | than needed. |
| 174 | |
| 175 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 176 | .. method:: WeakKeyDictionary.keyrefs() |
| 177 | |
Antoine Pitrou | c1baa60 | 2010-01-08 17:54:23 +0000 | [diff] [blame] | 178 | Return an iterable of the weak references to the keys. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 179 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 180 | |
| 181 | .. class:: WeakValueDictionary([dict]) |
| 182 | |
| 183 | Mapping class that references values weakly. Entries in the dictionary will be |
| 184 | discarded when no strong reference to the value exists any more. |
| 185 | |
| 186 | .. note:: |
| 187 | |
| 188 | Caution: Because a :class:`WeakValueDictionary` is built on top of a Python |
| 189 | dictionary, it must not change size when iterating over it. This can be |
| 190 | difficult to ensure for a :class:`WeakValueDictionary` because actions performed |
| 191 | by the program during iteration may cause items in the dictionary to vanish "by |
| 192 | magic" (as a side effect of garbage collection). |
| 193 | |
| 194 | :class:`WeakValueDictionary` objects have the following additional methods. |
Barry Warsaw | ecaab83 | 2008-09-04 01:42:51 +0000 | [diff] [blame] | 195 | These method have the same issues as the and :meth:`keyrefs` method of |
| 196 | :class:`WeakKeyDictionary` objects. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 197 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 198 | |
| 199 | .. method:: WeakValueDictionary.valuerefs() |
| 200 | |
Antoine Pitrou | c1baa60 | 2010-01-08 17:54:23 +0000 | [diff] [blame] | 201 | Return an iterable of the weak references to the values. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 202 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 203 | |
Georg Brandl | 3b8cb17 | 2007-10-23 06:26:46 +0000 | [diff] [blame] | 204 | .. class:: WeakSet([elements]) |
| 205 | |
| 206 | Set class that keeps weak references to its elements. An element will be |
| 207 | discarded when no strong reference to it exists any more. |
| 208 | |
| 209 | |
Antoine Pitrou | c3afba1 | 2012-11-17 18:57:38 +0100 | [diff] [blame] | 210 | .. class:: WeakMethod(method) |
| 211 | |
| 212 | A custom :class:`ref` subclass which simulates a weak reference to a bound |
| 213 | method (i.e., a method defined on a class and looked up on an instance). |
| 214 | Since a bound method is ephemeral, a standard weak reference cannot keep |
| 215 | hold of it. :class:`WeakMethod` has special code to recreate the bound |
| 216 | method until either the object or the original function dies:: |
| 217 | |
| 218 | >>> class C: |
| 219 | ... def method(self): |
| 220 | ... print("method called!") |
| 221 | ... |
| 222 | >>> c = C() |
| 223 | >>> r = weakref.ref(c.method) |
| 224 | >>> r() |
| 225 | >>> r = weakref.WeakMethod(c.method) |
| 226 | >>> r() |
| 227 | <bound method C.method of <__main__.C object at 0x7fc859830220>> |
| 228 | >>> r()() |
| 229 | method called! |
| 230 | >>> del c |
| 231 | >>> gc.collect() |
| 232 | 0 |
| 233 | >>> r() |
| 234 | >>> |
| 235 | |
| 236 | .. versionadded:: 3.4 |
| 237 | |
Richard Oudkerk | 7a3dae0 | 2013-05-05 23:05:00 +0100 | [diff] [blame] | 238 | .. class:: finalize(obj, func, *args, **kwargs) |
| 239 | |
| 240 | Return a callable finalizer object which will be called when *obj* |
R David Murray | a101bdb | 2014-01-06 16:32:05 -0500 | [diff] [blame^] | 241 | is garbage collected. Unlike an ordinary weak reference, a finalizer |
Nick Coghlan | be57ab8 | 2013-09-22 21:26:30 +1000 | [diff] [blame] | 242 | will always survive until the reference object is collected, greatly |
| 243 | simplifying lifecycle management. |
| 244 | |
| 245 | A finalizer is considered *alive* until it is called (either explicitly |
| 246 | or at garbage collection), and after that it is *dead*. Calling a live |
| 247 | finalizer returns the result of evaluating ``func(*arg, **kwargs)``, |
| 248 | whereas calling a dead finalizer returns :const:`None`. |
Richard Oudkerk | 7a3dae0 | 2013-05-05 23:05:00 +0100 | [diff] [blame] | 249 | |
| 250 | Exceptions raised by finalizer callbacks during garbage collection |
| 251 | will be shown on the standard error output, but cannot be |
| 252 | propagated. They are handled in the same way as exceptions raised |
| 253 | from an object's :meth:`__del__` method or a weak reference's |
| 254 | callback. |
| 255 | |
| 256 | When the program exits, each remaining live finalizer is called |
| 257 | unless its :attr:`atexit` attribute has been set to false. They |
| 258 | are called in reverse order of creation. |
| 259 | |
| 260 | A finalizer will never invoke its callback during the later part of |
| 261 | the interpreter shutdown when module globals are liable to have |
| 262 | been replaced by :const:`None`. |
| 263 | |
| 264 | .. method:: __call__() |
| 265 | |
| 266 | If *self* is alive then mark it as dead and return the result of |
| 267 | calling ``func(*args, **kwargs)``. If *self* is dead then return |
| 268 | :const:`None`. |
| 269 | |
| 270 | .. method:: detach() |
| 271 | |
| 272 | If *self* is alive then mark it as dead and return the tuple |
| 273 | ``(obj, func, args, kwargs)``. If *self* is dead then return |
| 274 | :const:`None`. |
| 275 | |
| 276 | .. method:: peek() |
| 277 | |
| 278 | If *self* is alive then return the tuple ``(obj, func, args, |
| 279 | kwargs)``. If *self* is dead then return :const:`None`. |
| 280 | |
| 281 | .. attribute:: alive |
| 282 | |
| 283 | Property which is true if the finalizer is alive, false otherwise. |
| 284 | |
| 285 | .. attribute:: atexit |
| 286 | |
| 287 | A writable boolean property which by default is true. When the |
| 288 | program exits, it calls all remaining live finalizers for which |
| 289 | :attr:`.atexit` is true. They are called in reverse order of |
| 290 | creation. |
| 291 | |
| 292 | .. note:: |
| 293 | |
| 294 | It is important to ensure that *func*, *args* and *kwargs* do |
| 295 | not own any references to *obj*, either directly or indirectly, |
| 296 | since otherwise *obj* will never be garbage collected. In |
| 297 | particular, *func* should not be a bound method of *obj*. |
| 298 | |
| 299 | .. versionadded:: 3.4 |
| 300 | |
Antoine Pitrou | c3afba1 | 2012-11-17 18:57:38 +0100 | [diff] [blame] | 301 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 302 | .. data:: ReferenceType |
| 303 | |
| 304 | The type object for weak references objects. |
| 305 | |
| 306 | |
| 307 | .. data:: ProxyType |
| 308 | |
| 309 | The type object for proxies of objects which are not callable. |
| 310 | |
| 311 | |
| 312 | .. data:: CallableProxyType |
| 313 | |
| 314 | The type object for proxies of callable objects. |
| 315 | |
| 316 | |
| 317 | .. data:: ProxyTypes |
| 318 | |
| 319 | Sequence containing all the type objects for proxies. This can make it simpler |
| 320 | to test if an object is a proxy without being dependent on naming both proxy |
| 321 | types. |
| 322 | |
| 323 | |
| 324 | .. exception:: ReferenceError |
| 325 | |
| 326 | Exception raised when a proxy object is used but the underlying object has been |
| 327 | collected. This is the same as the standard :exc:`ReferenceError` exception. |
| 328 | |
| 329 | |
| 330 | .. seealso:: |
| 331 | |
| 332 | :pep:`0205` - Weak References |
| 333 | The proposal and rationale for this feature, including links to earlier |
| 334 | implementations and information about similar features in other languages. |
| 335 | |
| 336 | |
| 337 | .. _weakref-objects: |
| 338 | |
| 339 | Weak Reference Objects |
| 340 | ---------------------- |
| 341 | |
Mark Dickinson | 556e94b | 2013-04-13 15:45:44 +0100 | [diff] [blame] | 342 | Weak reference objects have no methods and no attributes besides |
| 343 | :attr:`ref.__callback__`. A weak reference object allows the referent to be |
| 344 | obtained, if it still exists, by calling it: |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 345 | |
| 346 | >>> import weakref |
| 347 | >>> class Object: |
| 348 | ... pass |
| 349 | ... |
| 350 | >>> o = Object() |
| 351 | >>> r = weakref.ref(o) |
| 352 | >>> o2 = r() |
| 353 | >>> o is o2 |
| 354 | True |
| 355 | |
| 356 | If the referent no longer exists, calling the reference object returns |
Christian Heimes | fe337bf | 2008-03-23 21:54:12 +0000 | [diff] [blame] | 357 | :const:`None`: |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 358 | |
| 359 | >>> del o, o2 |
Collin Winter | c79461b | 2007-09-01 23:34:30 +0000 | [diff] [blame] | 360 | >>> print(r()) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 361 | None |
| 362 | |
| 363 | Testing that a weak reference object is still live should be done using the |
| 364 | expression ``ref() is not None``. Normally, application code that needs to use |
| 365 | a reference object should follow this pattern:: |
| 366 | |
| 367 | # r is a weak reference object |
| 368 | o = r() |
| 369 | if o is None: |
| 370 | # referent has been garbage collected |
Collin Winter | c79461b | 2007-09-01 23:34:30 +0000 | [diff] [blame] | 371 | print("Object has been deallocated; can't frobnicate.") |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 372 | else: |
Collin Winter | c79461b | 2007-09-01 23:34:30 +0000 | [diff] [blame] | 373 | print("Object is still live!") |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 374 | o.do_something_useful() |
| 375 | |
| 376 | Using a separate test for "liveness" creates race conditions in threaded |
| 377 | applications; another thread can cause a weak reference to become invalidated |
| 378 | before the weak reference is called; the idiom shown above is safe in threaded |
| 379 | applications as well as single-threaded applications. |
| 380 | |
| 381 | Specialized versions of :class:`ref` objects can be created through subclassing. |
| 382 | This is used in the implementation of the :class:`WeakValueDictionary` to reduce |
| 383 | the memory overhead for each entry in the mapping. This may be most useful to |
| 384 | associate additional information with a reference, but could also be used to |
| 385 | insert additional processing on calls to retrieve the referent. |
| 386 | |
| 387 | This example shows how a subclass of :class:`ref` can be used to store |
| 388 | additional information about an object and affect the value that's returned when |
| 389 | the referent is accessed:: |
| 390 | |
| 391 | import weakref |
| 392 | |
| 393 | class ExtendedRef(weakref.ref): |
| 394 | def __init__(self, ob, callback=None, **annotations): |
| 395 | super(ExtendedRef, self).__init__(ob, callback) |
| 396 | self.__counter = 0 |
Barry Warsaw | ecaab83 | 2008-09-04 01:42:51 +0000 | [diff] [blame] | 397 | for k, v in annotations.items(): |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 398 | setattr(self, k, v) |
| 399 | |
| 400 | def __call__(self): |
| 401 | """Return a pair containing the referent and the number of |
| 402 | times the reference has been called. |
| 403 | """ |
| 404 | ob = super(ExtendedRef, self).__call__() |
| 405 | if ob is not None: |
| 406 | self.__counter += 1 |
| 407 | ob = (ob, self.__counter) |
| 408 | return ob |
| 409 | |
| 410 | |
| 411 | .. _weakref-example: |
| 412 | |
| 413 | Example |
| 414 | ------- |
| 415 | |
| 416 | This simple example shows how an application can use objects IDs to retrieve |
| 417 | objects that it has seen before. The IDs of the objects can then be used in |
| 418 | other data structures without forcing the objects to remain alive, but the |
| 419 | objects can still be retrieved by ID if they do. |
| 420 | |
Christian Heimes | 5b5e81c | 2007-12-31 16:14:33 +0000 | [diff] [blame] | 421 | .. Example contributed by Tim Peters. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 422 | |
| 423 | :: |
| 424 | |
| 425 | import weakref |
| 426 | |
| 427 | _id2obj_dict = weakref.WeakValueDictionary() |
| 428 | |
| 429 | def remember(obj): |
| 430 | oid = id(obj) |
| 431 | _id2obj_dict[oid] = obj |
| 432 | return oid |
| 433 | |
| 434 | def id2obj(oid): |
| 435 | return _id2obj_dict[oid] |
| 436 | |
Richard Oudkerk | 7a3dae0 | 2013-05-05 23:05:00 +0100 | [diff] [blame] | 437 | |
| 438 | .. _finalize-examples: |
| 439 | |
| 440 | Finalizer Objects |
| 441 | ----------------- |
| 442 | |
Nick Coghlan | be57ab8 | 2013-09-22 21:26:30 +1000 | [diff] [blame] | 443 | The main benefit of using :class:`finalize` is that it makes it simple |
| 444 | to register a callback without needing to preserve the returned finalizer |
| 445 | object. For instance |
Richard Oudkerk | 7a3dae0 | 2013-05-05 23:05:00 +0100 | [diff] [blame] | 446 | |
| 447 | >>> import weakref |
| 448 | >>> class Object: |
| 449 | ... pass |
| 450 | ... |
| 451 | >>> kenny = Object() |
| 452 | >>> weakref.finalize(kenny, print, "You killed Kenny!") #doctest:+ELLIPSIS |
| 453 | <finalize object at ...; for 'Object' at ...> |
| 454 | >>> del kenny |
| 455 | You killed Kenny! |
| 456 | |
| 457 | The finalizer can be called directly as well. However the finalizer |
| 458 | will invoke the callback at most once. |
| 459 | |
| 460 | >>> def callback(x, y, z): |
| 461 | ... print("CALLBACK") |
| 462 | ... return x + y + z |
| 463 | ... |
| 464 | >>> obj = Object() |
| 465 | >>> f = weakref.finalize(obj, callback, 1, 2, z=3) |
| 466 | >>> assert f.alive |
| 467 | >>> assert f() == 6 |
| 468 | CALLBACK |
| 469 | >>> assert not f.alive |
| 470 | >>> f() # callback not called because finalizer dead |
| 471 | >>> del obj # callback not called because finalizer dead |
| 472 | |
| 473 | You can unregister a finalizer using its :meth:`~finalize.detach` |
| 474 | method. This kills the finalizer and returns the arguments passed to |
| 475 | the constructor when it was created. |
| 476 | |
| 477 | >>> obj = Object() |
| 478 | >>> f = weakref.finalize(obj, callback, 1, 2, z=3) |
| 479 | >>> f.detach() #doctest:+ELLIPSIS |
| 480 | (<__main__.Object object ...>, <function callback ...>, (1, 2), {'z': 3}) |
| 481 | >>> newobj, func, args, kwargs = _ |
| 482 | >>> assert not f.alive |
| 483 | >>> assert newobj is obj |
| 484 | >>> assert func(*args, **kwargs) == 6 |
| 485 | CALLBACK |
| 486 | |
| 487 | Unless you set the :attr:`~finalize.atexit` attribute to |
Nick Coghlan | be57ab8 | 2013-09-22 21:26:30 +1000 | [diff] [blame] | 488 | :const:`False`, a finalizer will be called when the program exits if it |
Richard Oudkerk | 7a3dae0 | 2013-05-05 23:05:00 +0100 | [diff] [blame] | 489 | is still alive. For instance |
| 490 | |
| 491 | >>> obj = Object() |
| 492 | >>> weakref.finalize(obj, print, "obj dead or exiting") #doctest:+ELLIPSIS |
| 493 | <finalize object at ...; for 'Object' at ...> |
| 494 | >>> exit() #doctest:+SKIP |
| 495 | obj dead or exiting |
| 496 | |
| 497 | |
| 498 | Comparing finalizers with :meth:`__del__` methods |
| 499 | ------------------------------------------------- |
| 500 | |
| 501 | Suppose we want to create a class whose instances represent temporary |
| 502 | directories. The directories should be deleted with their contents |
| 503 | when the first of the following events occurs: |
| 504 | |
| 505 | * the object is garbage collected, |
| 506 | * the object's :meth:`remove` method is called, or |
| 507 | * the program exits. |
| 508 | |
| 509 | We might try to implement the class using a :meth:`__del__` method as |
| 510 | follows:: |
| 511 | |
| 512 | class TempDir: |
| 513 | def __init__(self): |
| 514 | self.name = tempfile.mkdtemp() |
| 515 | |
| 516 | def remove(self): |
| 517 | if self.name is not None: |
| 518 | shutil.rmtree(self.name) |
| 519 | self.name = None |
| 520 | |
| 521 | @property |
| 522 | def removed(self): |
| 523 | return self.name is None |
| 524 | |
| 525 | def __del__(self): |
| 526 | self.remove() |
| 527 | |
Nick Coghlan | be57ab8 | 2013-09-22 21:26:30 +1000 | [diff] [blame] | 528 | Starting with Python 3.4, :meth:`__del__` methods no longer prevent |
| 529 | reference cycles from being garbage collected, and module globals are |
| 530 | no longer forced to :const:`None` during interpreter shutdown. So this |
| 531 | code should work without any issues on CPython. |
Richard Oudkerk | 7a3dae0 | 2013-05-05 23:05:00 +0100 | [diff] [blame] | 532 | |
Nick Coghlan | be57ab8 | 2013-09-22 21:26:30 +1000 | [diff] [blame] | 533 | However, handling of :meth:`__del__` methods is notoriously implementation |
Nick Coghlan | 4c7fe6a | 2013-09-22 21:32:12 +1000 | [diff] [blame] | 534 | specific, since it depends on internal details of the interpreter's garbage |
| 535 | collector implementation. |
Nick Coghlan | be57ab8 | 2013-09-22 21:26:30 +1000 | [diff] [blame] | 536 | |
| 537 | A more robust alternative can be to define a finalizer which only references |
| 538 | the specific functions and objects that it needs, rather than having access |
| 539 | to the full state of the object:: |
Richard Oudkerk | 7a3dae0 | 2013-05-05 23:05:00 +0100 | [diff] [blame] | 540 | |
| 541 | class TempDir: |
| 542 | def __init__(self): |
| 543 | self.name = tempfile.mkdtemp() |
| 544 | self._finalizer = weakref.finalize(self, shutil.rmtree, self.name) |
| 545 | |
| 546 | def remove(self): |
| 547 | self._finalizer() |
| 548 | |
| 549 | @property |
| 550 | def removed(self): |
| 551 | return not self._finalizer.alive |
| 552 | |
Nick Coghlan | be57ab8 | 2013-09-22 21:26:30 +1000 | [diff] [blame] | 553 | Defined like this, our finalizer only receives a reference to the details |
| 554 | it needs to clean up the directory appropriately. If the object never gets |
| 555 | garbage collected the finalizer will still be called at exit. |
| 556 | |
| 557 | The other advantage of weakref based finalizers is that they can be used to |
| 558 | register finalizers for classes where the definition is controlled by a |
| 559 | third party, such as running code when a module is unloaded:: |
| 560 | |
| 561 | import weakref, sys |
| 562 | def unloading_module(): |
| 563 | # implicit reference to the module globals from the function body |
| 564 | weakref.finalize(sys.modules[__name__], unloading_module) |
| 565 | |
Richard Oudkerk | 7a3dae0 | 2013-05-05 23:05:00 +0100 | [diff] [blame] | 566 | |
| 567 | .. note:: |
| 568 | |
| 569 | If you create a finalizer object in a daemonic thread just as the |
| 570 | the program exits then there is the possibility that the finalizer |
| 571 | does not get called at exit. However, in a daemonic thread |
| 572 | :func:`atexit.register`, ``try: ... finally: ...`` and ``with: ...`` |
| 573 | do not guarantee that cleanup occurs either. |