Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1 | :mod:`weakref` --- Weak references |
| 2 | ================================== |
| 3 | |
| 4 | .. module:: weakref |
| 5 | :synopsis: Support for weak references and weak dictionaries. |
Terry Jan Reedy | fa089b9 | 2016-06-11 15:02:54 -0400 | [diff] [blame] | 6 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 7 | .. moduleauthor:: Fred L. Drake, Jr. <fdrake@acm.org> |
| 8 | .. moduleauthor:: Neil Schemenauer <nas@arctrix.com> |
| 9 | .. moduleauthor:: Martin von Lรถwis <martin@loewis.home.cs.tu-berlin.de> |
| 10 | .. sectionauthor:: Fred L. Drake, Jr. <fdrake@acm.org> |
| 11 | |
Raymond Hettinger | 469271d | 2011-01-27 20:38:46 +0000 | [diff] [blame] | 12 | **Source code:** :source:`Lib/weakref.py` |
| 13 | |
| 14 | -------------- |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 15 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 16 | The :mod:`weakref` module allows the Python programmer to create :dfn:`weak |
| 17 | references` to objects. |
| 18 | |
Christian Heimes | 5b5e81c | 2007-12-31 16:14:33 +0000 | [diff] [blame] | 19 | .. When making changes to the examples in this file, be sure to update |
| 20 | Lib/test/test_weakref.py::libreftest too! |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 21 | |
| 22 | In the following, the term :dfn:`referent` means the object which is referred to |
| 23 | by a weak reference. |
| 24 | |
| 25 | A weak reference to an object is not enough to keep the object alive: when the |
Christian Heimes | d8654cf | 2007-12-02 15:22:16 +0000 | [diff] [blame] | 26 | only remaining references to a referent are weak references, |
| 27 | :term:`garbage collection` is free to destroy the referent and reuse its memory |
Antoine Pitrou | 9439f04 | 2012-08-21 00:07:07 +0200 | [diff] [blame] | 28 | for something else. However, until the object is actually destroyed the weak |
| 29 | reference may return the object even if there are no strong references to it. |
| 30 | |
| 31 | A primary use for weak references is to implement caches or |
Christian Heimes | d8654cf | 2007-12-02 15:22:16 +0000 | [diff] [blame] | 32 | mappings holding large objects, where it's desired that a large object not be |
Christian Heimes | fe337bf | 2008-03-23 21:54:12 +0000 | [diff] [blame] | 33 | kept alive solely because it appears in a cache or mapping. |
| 34 | |
| 35 | For example, if you have a number of large binary image objects, you may wish to |
| 36 | associate a name with each. If you used a Python dictionary to map names to |
| 37 | images, or images to names, the image objects would remain alive just because |
| 38 | they appeared as values or keys in the dictionaries. The |
| 39 | :class:`WeakKeyDictionary` and :class:`WeakValueDictionary` classes supplied by |
| 40 | the :mod:`weakref` module are an alternative, using weak references to construct |
| 41 | mappings that don't keep objects alive solely because they appear in the mapping |
| 42 | objects. If, for example, an image object is a value in a |
| 43 | :class:`WeakValueDictionary`, then when the last remaining references to that |
| 44 | image object are the weak references held by weak mappings, garbage collection |
| 45 | can reclaim the object, and its corresponding entries in weak mappings are |
| 46 | simply deleted. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 47 | |
| 48 | :class:`WeakKeyDictionary` and :class:`WeakValueDictionary` use weak references |
| 49 | in their implementation, setting up callback functions on the weak references |
| 50 | that notify the weak dictionaries when a key or value has been reclaimed by |
Georg Brandl | 3b8cb17 | 2007-10-23 06:26:46 +0000 | [diff] [blame] | 51 | garbage collection. :class:`WeakSet` implements the :class:`set` interface, |
| 52 | but keeps weak references to its elements, just like a |
| 53 | :class:`WeakKeyDictionary` does. |
| 54 | |
Richard Oudkerk | 7a3dae0 | 2013-05-05 23:05:00 +0100 | [diff] [blame] | 55 | :class:`finalize` provides a straight forward way to register a |
| 56 | cleanup function to be called when an object is garbage collected. |
| 57 | This is simpler to use than setting up a callback function on a raw |
Nick Coghlan | be57ab8 | 2013-09-22 21:26:30 +1000 | [diff] [blame] | 58 | weak reference, since the module automatically ensures that the finalizer |
| 59 | remains alive until the object is collected. |
Richard Oudkerk | 7a3dae0 | 2013-05-05 23:05:00 +0100 | [diff] [blame] | 60 | |
| 61 | Most programs should find that using one of these weak container types |
| 62 | or :class:`finalize` is all they need -- it's not usually necessary to |
| 63 | create your own weak references directly. The low-level machinery is |
| 64 | exposed by the :mod:`weakref` module for the benefit of advanced uses. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 65 | |
| 66 | Not all objects can be weakly referenced; those objects which can include class |
Georg Brandl | 2e0b755 | 2007-11-27 12:43:08 +0000 | [diff] [blame] | 67 | instances, functions written in Python (but not in C), instance methods, sets, |
Antoine Pitrou | 11cb961 | 2010-09-15 11:11:28 +0000 | [diff] [blame] | 68 | frozensets, some :term:`file objects <file object>`, :term:`generator`\s, type |
| 69 | objects, sockets, arrays, deques, regular expression pattern objects, and code |
| 70 | objects. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 71 | |
Benjamin Peterson | bec4d57 | 2009-10-10 01:16:07 +0000 | [diff] [blame] | 72 | .. versionchanged:: 3.2 |
Collin Winter | 4222e9c | 2010-03-18 22:46:40 +0000 | [diff] [blame] | 73 | Added support for thread.lock, threading.Lock, and code objects. |
Benjamin Peterson | bec4d57 | 2009-10-10 01:16:07 +0000 | [diff] [blame] | 74 | |
Georg Brandl | 22b3431 | 2009-07-26 14:54:51 +0000 | [diff] [blame] | 75 | Several built-in types such as :class:`list` and :class:`dict` do not directly |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 76 | support weak references but can add support through subclassing:: |
| 77 | |
| 78 | class Dict(dict): |
| 79 | pass |
| 80 | |
Christian Heimes | c3f30c4 | 2008-02-22 16:37:40 +0000 | [diff] [blame] | 81 | obj = Dict(red=1, green=2, blue=3) # this object is weak referenceable |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 82 | |
Benjamin Peterson | 905982b | 2010-05-08 15:26:30 +0000 | [diff] [blame] | 83 | Other built-in types such as :class:`tuple` and :class:`int` do not support weak |
| 84 | references even when subclassed (This is an implementation detail and may be |
| 85 | different across various Python implementations.). |
Georg Brandl | ff8c1e5 | 2009-10-21 07:17:48 +0000 | [diff] [blame] | 86 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 87 | Extension types can easily be made to support weak references; see |
| 88 | :ref:`weakref-support`. |
| 89 | |
| 90 | |
| 91 | .. class:: ref(object[, callback]) |
| 92 | |
| 93 | Return a weak reference to *object*. The original object can be retrieved by |
| 94 | calling the reference object if the referent is still alive; if the referent is |
| 95 | no longer alive, calling the reference object will cause :const:`None` to be |
| 96 | returned. If *callback* is provided and not :const:`None`, and the returned |
| 97 | weakref object is still alive, the callback will be called when the object is |
| 98 | about to be finalized; the weak reference object will be passed as the only |
| 99 | parameter to the callback; the referent will no longer be available. |
| 100 | |
| 101 | It is allowable for many weak references to be constructed for the same object. |
| 102 | Callbacks registered for each weak reference will be called from the most |
| 103 | recently registered callback to the oldest registered callback. |
| 104 | |
| 105 | Exceptions raised by the callback will be noted on the standard error output, |
| 106 | but cannot be propagated; they are handled in exactly the same way as exceptions |
| 107 | raised from an object's :meth:`__del__` method. |
| 108 | |
Georg Brandl | 7f01a13 | 2009-09-16 15:58:14 +0000 | [diff] [blame] | 109 | Weak references are :term:`hashable` if the *object* is hashable. They will |
| 110 | maintain their hash value even after the *object* was deleted. If |
| 111 | :func:`hash` is called the first time only after the *object* was deleted, |
| 112 | the call will raise :exc:`TypeError`. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 113 | |
| 114 | Weak references support tests for equality, but not ordering. If the referents |
| 115 | are still alive, two references have the same equality relationship as their |
| 116 | referents (regardless of the *callback*). If either referent has been deleted, |
| 117 | the references are equal only if the reference objects are the same object. |
| 118 | |
Georg Brandl | 55ac8f0 | 2007-09-01 13:51:09 +0000 | [diff] [blame] | 119 | This is a subclassable type rather than a factory function. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 120 | |
Mark Dickinson | 556e94b | 2013-04-13 15:45:44 +0100 | [diff] [blame] | 121 | .. attribute:: __callback__ |
| 122 | |
| 123 | This read-only attribute returns the callback currently associated to the |
| 124 | weakref. If there is no callback or if the referent of the weakref is |
| 125 | no longer alive then this attribute will have value ``None``. |
| 126 | |
Richard Oudkerk | 7a3dae0 | 2013-05-05 23:05:00 +0100 | [diff] [blame] | 127 | .. versionchanged:: 3.4 |
Mark Dickinson | 9b6fdf8 | 2013-04-13 16:09:18 +0100 | [diff] [blame] | 128 | Added the :attr:`__callback__` attribute. |
Mark Dickinson | 556e94b | 2013-04-13 15:45:44 +0100 | [diff] [blame] | 129 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 130 | |
| 131 | .. function:: proxy(object[, callback]) |
| 132 | |
| 133 | Return a proxy to *object* which uses a weak reference. This supports use of |
| 134 | the proxy in most contexts instead of requiring the explicit dereferencing used |
| 135 | with weak reference objects. The returned object will have a type of either |
| 136 | ``ProxyType`` or ``CallableProxyType``, depending on whether *object* is |
Guido van Rossum | 2cc30da | 2007-11-02 23:46:40 +0000 | [diff] [blame] | 137 | callable. Proxy objects are not :term:`hashable` regardless of the referent; this |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 138 | avoids a number of problems related to their fundamentally mutable nature, and |
| 139 | prevent their use as dictionary keys. *callback* is the same as the parameter |
| 140 | of the same name to the :func:`ref` function. |
| 141 | |
| 142 | |
| 143 | .. function:: getweakrefcount(object) |
| 144 | |
| 145 | Return the number of weak references and proxies which refer to *object*. |
| 146 | |
| 147 | |
| 148 | .. function:: getweakrefs(object) |
| 149 | |
| 150 | Return a list of all weak reference and proxy objects which refer to *object*. |
| 151 | |
| 152 | |
| 153 | .. class:: WeakKeyDictionary([dict]) |
| 154 | |
| 155 | Mapping class that references keys weakly. Entries in the dictionary will be |
| 156 | discarded when there is no longer a strong reference to the key. This can be |
| 157 | used to associate additional data with an object owned by other parts of an |
| 158 | application without adding attributes to those objects. This can be especially |
| 159 | useful with objects that override attribute accesses. |
| 160 | |
| 161 | .. note:: |
| 162 | |
Christian Heimes | fe337bf | 2008-03-23 21:54:12 +0000 | [diff] [blame] | 163 | Caution: Because a :class:`WeakKeyDictionary` is built on top of a Python |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 164 | dictionary, it must not change size when iterating over it. This can be |
Christian Heimes | fe337bf | 2008-03-23 21:54:12 +0000 | [diff] [blame] | 165 | difficult to ensure for a :class:`WeakKeyDictionary` because actions |
| 166 | performed by the program during iteration may cause items in the |
| 167 | dictionary to vanish "by magic" (as a side effect of garbage collection). |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 168 | |
Mariatta | 3110a37 | 2017-02-12 08:17:50 -0800 | [diff] [blame] | 169 | :class:`WeakKeyDictionary` objects have an additional method that |
| 170 | exposes the internal references directly. The references are not guaranteed to |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 171 | be "live" at the time they are used, so the result of calling the references |
| 172 | needs to be checked before being used. This can be used to avoid creating |
| 173 | references that will cause the garbage collector to keep the keys around longer |
| 174 | than needed. |
| 175 | |
| 176 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 177 | .. method:: WeakKeyDictionary.keyrefs() |
| 178 | |
Antoine Pitrou | c1baa60 | 2010-01-08 17:54:23 +0000 | [diff] [blame] | 179 | Return an iterable of the weak references to the keys. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 180 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 181 | |
| 182 | .. class:: WeakValueDictionary([dict]) |
| 183 | |
| 184 | Mapping class that references values weakly. Entries in the dictionary will be |
| 185 | discarded when no strong reference to the value exists any more. |
| 186 | |
| 187 | .. note:: |
| 188 | |
| 189 | Caution: Because a :class:`WeakValueDictionary` is built on top of a Python |
| 190 | dictionary, it must not change size when iterating over it. This can be |
| 191 | difficult to ensure for a :class:`WeakValueDictionary` because actions performed |
| 192 | by the program during iteration may cause items in the dictionary to vanish "by |
| 193 | magic" (as a side effect of garbage collection). |
| 194 | |
Mariatta | 3110a37 | 2017-02-12 08:17:50 -0800 | [diff] [blame] | 195 | :class:`WeakValueDictionary` objects have an additional method that has the |
| 196 | same issues as the :meth:`keyrefs` method of :class:`WeakKeyDictionary` |
| 197 | objects. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 198 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 199 | |
| 200 | .. method:: WeakValueDictionary.valuerefs() |
| 201 | |
Antoine Pitrou | c1baa60 | 2010-01-08 17:54:23 +0000 | [diff] [blame] | 202 | Return an iterable of the weak references to the values. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 203 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 204 | |
Georg Brandl | 3b8cb17 | 2007-10-23 06:26:46 +0000 | [diff] [blame] | 205 | .. class:: WeakSet([elements]) |
| 206 | |
| 207 | Set class that keeps weak references to its elements. An element will be |
| 208 | discarded when no strong reference to it exists any more. |
| 209 | |
| 210 | |
Antoine Pitrou | c3afba1 | 2012-11-17 18:57:38 +0100 | [diff] [blame] | 211 | .. class:: WeakMethod(method) |
| 212 | |
| 213 | A custom :class:`ref` subclass which simulates a weak reference to a bound |
| 214 | method (i.e., a method defined on a class and looked up on an instance). |
| 215 | Since a bound method is ephemeral, a standard weak reference cannot keep |
| 216 | hold of it. :class:`WeakMethod` has special code to recreate the bound |
| 217 | method until either the object or the original function dies:: |
| 218 | |
| 219 | >>> class C: |
| 220 | ... def method(self): |
| 221 | ... print("method called!") |
| 222 | ... |
| 223 | >>> c = C() |
| 224 | >>> r = weakref.ref(c.method) |
| 225 | >>> r() |
| 226 | >>> r = weakref.WeakMethod(c.method) |
| 227 | >>> r() |
| 228 | <bound method C.method of <__main__.C object at 0x7fc859830220>> |
| 229 | >>> r()() |
| 230 | method called! |
| 231 | >>> del c |
| 232 | >>> gc.collect() |
| 233 | 0 |
| 234 | >>> r() |
| 235 | >>> |
| 236 | |
| 237 | .. versionadded:: 3.4 |
| 238 | |
Richard Oudkerk | 7a3dae0 | 2013-05-05 23:05:00 +0100 | [diff] [blame] | 239 | .. class:: finalize(obj, func, *args, **kwargs) |
| 240 | |
| 241 | Return a callable finalizer object which will be called when *obj* |
R David Murray | a101bdb | 2014-01-06 16:32:05 -0500 | [diff] [blame] | 242 | is garbage collected. Unlike an ordinary weak reference, a finalizer |
Nick Coghlan | be57ab8 | 2013-09-22 21:26:30 +1000 | [diff] [blame] | 243 | will always survive until the reference object is collected, greatly |
| 244 | simplifying lifecycle management. |
| 245 | |
| 246 | A finalizer is considered *alive* until it is called (either explicitly |
| 247 | or at garbage collection), and after that it is *dead*. Calling a live |
| 248 | finalizer returns the result of evaluating ``func(*arg, **kwargs)``, |
| 249 | whereas calling a dead finalizer returns :const:`None`. |
Richard Oudkerk | 7a3dae0 | 2013-05-05 23:05:00 +0100 | [diff] [blame] | 250 | |
| 251 | Exceptions raised by finalizer callbacks during garbage collection |
| 252 | will be shown on the standard error output, but cannot be |
| 253 | propagated. They are handled in the same way as exceptions raised |
| 254 | from an object's :meth:`__del__` method or a weak reference's |
| 255 | callback. |
| 256 | |
| 257 | When the program exits, each remaining live finalizer is called |
| 258 | unless its :attr:`atexit` attribute has been set to false. They |
| 259 | are called in reverse order of creation. |
| 260 | |
| 261 | A finalizer will never invoke its callback during the later part of |
Antoine Pitrou | 5db1bb8 | 2014-12-07 01:28:27 +0100 | [diff] [blame] | 262 | the :term:`interpreter shutdown` when module globals are liable to have |
Richard Oudkerk | 7a3dae0 | 2013-05-05 23:05:00 +0100 | [diff] [blame] | 263 | been replaced by :const:`None`. |
| 264 | |
| 265 | .. method:: __call__() |
| 266 | |
| 267 | If *self* is alive then mark it as dead and return the result of |
| 268 | calling ``func(*args, **kwargs)``. If *self* is dead then return |
| 269 | :const:`None`. |
| 270 | |
| 271 | .. method:: detach() |
| 272 | |
| 273 | If *self* is alive then mark it as dead and return the tuple |
| 274 | ``(obj, func, args, kwargs)``. If *self* is dead then return |
| 275 | :const:`None`. |
| 276 | |
| 277 | .. method:: peek() |
| 278 | |
| 279 | If *self* is alive then return the tuple ``(obj, func, args, |
| 280 | kwargs)``. If *self* is dead then return :const:`None`. |
| 281 | |
| 282 | .. attribute:: alive |
| 283 | |
| 284 | Property which is true if the finalizer is alive, false otherwise. |
| 285 | |
| 286 | .. attribute:: atexit |
| 287 | |
| 288 | A writable boolean property which by default is true. When the |
| 289 | program exits, it calls all remaining live finalizers for which |
| 290 | :attr:`.atexit` is true. They are called in reverse order of |
| 291 | creation. |
| 292 | |
| 293 | .. note:: |
| 294 | |
| 295 | It is important to ensure that *func*, *args* and *kwargs* do |
| 296 | not own any references to *obj*, either directly or indirectly, |
| 297 | since otherwise *obj* will never be garbage collected. In |
| 298 | particular, *func* should not be a bound method of *obj*. |
| 299 | |
| 300 | .. versionadded:: 3.4 |
| 301 | |
Antoine Pitrou | c3afba1 | 2012-11-17 18:57:38 +0100 | [diff] [blame] | 302 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 303 | .. data:: ReferenceType |
| 304 | |
| 305 | The type object for weak references objects. |
| 306 | |
| 307 | |
| 308 | .. data:: ProxyType |
| 309 | |
| 310 | The type object for proxies of objects which are not callable. |
| 311 | |
| 312 | |
| 313 | .. data:: CallableProxyType |
| 314 | |
| 315 | The type object for proxies of callable objects. |
| 316 | |
| 317 | |
| 318 | .. data:: ProxyTypes |
| 319 | |
| 320 | Sequence containing all the type objects for proxies. This can make it simpler |
| 321 | to test if an object is a proxy without being dependent on naming both proxy |
| 322 | types. |
| 323 | |
| 324 | |
| 325 | .. exception:: ReferenceError |
| 326 | |
| 327 | Exception raised when a proxy object is used but the underlying object has been |
| 328 | collected. This is the same as the standard :exc:`ReferenceError` exception. |
| 329 | |
| 330 | |
| 331 | .. seealso:: |
| 332 | |
Serhiy Storchaka | e4ba872 | 2016-03-31 15:30:54 +0300 | [diff] [blame] | 333 | :pep:`205` - Weak References |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 334 | The proposal and rationale for this feature, including links to earlier |
| 335 | implementations and information about similar features in other languages. |
| 336 | |
| 337 | |
| 338 | .. _weakref-objects: |
| 339 | |
| 340 | Weak Reference Objects |
| 341 | ---------------------- |
| 342 | |
Mark Dickinson | 556e94b | 2013-04-13 15:45:44 +0100 | [diff] [blame] | 343 | Weak reference objects have no methods and no attributes besides |
| 344 | :attr:`ref.__callback__`. A weak reference object allows the referent to be |
| 345 | obtained, if it still exists, by calling it: |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 346 | |
| 347 | >>> import weakref |
| 348 | >>> class Object: |
| 349 | ... pass |
| 350 | ... |
| 351 | >>> o = Object() |
| 352 | >>> r = weakref.ref(o) |
| 353 | >>> o2 = r() |
| 354 | >>> o is o2 |
| 355 | True |
| 356 | |
| 357 | If the referent no longer exists, calling the reference object returns |
Christian Heimes | fe337bf | 2008-03-23 21:54:12 +0000 | [diff] [blame] | 358 | :const:`None`: |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 359 | |
| 360 | >>> del o, o2 |
Collin Winter | c79461b | 2007-09-01 23:34:30 +0000 | [diff] [blame] | 361 | >>> print(r()) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 362 | None |
| 363 | |
| 364 | Testing that a weak reference object is still live should be done using the |
| 365 | expression ``ref() is not None``. Normally, application code that needs to use |
| 366 | a reference object should follow this pattern:: |
| 367 | |
| 368 | # r is a weak reference object |
| 369 | o = r() |
| 370 | if o is None: |
| 371 | # referent has been garbage collected |
Collin Winter | c79461b | 2007-09-01 23:34:30 +0000 | [diff] [blame] | 372 | print("Object has been deallocated; can't frobnicate.") |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 373 | else: |
Collin Winter | c79461b | 2007-09-01 23:34:30 +0000 | [diff] [blame] | 374 | print("Object is still live!") |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 375 | o.do_something_useful() |
| 376 | |
| 377 | Using a separate test for "liveness" creates race conditions in threaded |
| 378 | applications; another thread can cause a weak reference to become invalidated |
| 379 | before the weak reference is called; the idiom shown above is safe in threaded |
| 380 | applications as well as single-threaded applications. |
| 381 | |
| 382 | Specialized versions of :class:`ref` objects can be created through subclassing. |
| 383 | This is used in the implementation of the :class:`WeakValueDictionary` to reduce |
| 384 | the memory overhead for each entry in the mapping. This may be most useful to |
| 385 | associate additional information with a reference, but could also be used to |
| 386 | insert additional processing on calls to retrieve the referent. |
| 387 | |
| 388 | This example shows how a subclass of :class:`ref` can be used to store |
| 389 | additional information about an object and affect the value that's returned when |
| 390 | the referent is accessed:: |
| 391 | |
| 392 | import weakref |
| 393 | |
| 394 | class ExtendedRef(weakref.ref): |
| 395 | def __init__(self, ob, callback=None, **annotations): |
| 396 | super(ExtendedRef, self).__init__(ob, callback) |
| 397 | self.__counter = 0 |
Barry Warsaw | ecaab83 | 2008-09-04 01:42:51 +0000 | [diff] [blame] | 398 | for k, v in annotations.items(): |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 399 | setattr(self, k, v) |
| 400 | |
| 401 | def __call__(self): |
| 402 | """Return a pair containing the referent and the number of |
| 403 | times the reference has been called. |
| 404 | """ |
| 405 | ob = super(ExtendedRef, self).__call__() |
| 406 | if ob is not None: |
| 407 | self.__counter += 1 |
| 408 | ob = (ob, self.__counter) |
| 409 | return ob |
| 410 | |
| 411 | |
| 412 | .. _weakref-example: |
| 413 | |
| 414 | Example |
| 415 | ------- |
| 416 | |
Martin Panter | 0f0eac4 | 2016-09-07 11:04:41 +0000 | [diff] [blame] | 417 | This simple example shows how an application can use object IDs to retrieve |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 418 | objects that it has seen before. The IDs of the objects can then be used in |
| 419 | other data structures without forcing the objects to remain alive, but the |
| 420 | objects can still be retrieved by ID if they do. |
| 421 | |
Christian Heimes | 5b5e81c | 2007-12-31 16:14:33 +0000 | [diff] [blame] | 422 | .. Example contributed by Tim Peters. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 423 | |
| 424 | :: |
| 425 | |
| 426 | import weakref |
| 427 | |
| 428 | _id2obj_dict = weakref.WeakValueDictionary() |
| 429 | |
| 430 | def remember(obj): |
| 431 | oid = id(obj) |
| 432 | _id2obj_dict[oid] = obj |
| 433 | return oid |
| 434 | |
| 435 | def id2obj(oid): |
| 436 | return _id2obj_dict[oid] |
| 437 | |
Richard Oudkerk | 7a3dae0 | 2013-05-05 23:05:00 +0100 | [diff] [blame] | 438 | |
| 439 | .. _finalize-examples: |
| 440 | |
| 441 | Finalizer Objects |
| 442 | ----------------- |
| 443 | |
Nick Coghlan | be57ab8 | 2013-09-22 21:26:30 +1000 | [diff] [blame] | 444 | The main benefit of using :class:`finalize` is that it makes it simple |
| 445 | to register a callback without needing to preserve the returned finalizer |
| 446 | object. For instance |
Richard Oudkerk | 7a3dae0 | 2013-05-05 23:05:00 +0100 | [diff] [blame] | 447 | |
| 448 | >>> import weakref |
| 449 | >>> class Object: |
| 450 | ... pass |
| 451 | ... |
| 452 | >>> kenny = Object() |
| 453 | >>> weakref.finalize(kenny, print, "You killed Kenny!") #doctest:+ELLIPSIS |
| 454 | <finalize object at ...; for 'Object' at ...> |
| 455 | >>> del kenny |
| 456 | You killed Kenny! |
| 457 | |
| 458 | The finalizer can be called directly as well. However the finalizer |
| 459 | will invoke the callback at most once. |
| 460 | |
| 461 | >>> def callback(x, y, z): |
| 462 | ... print("CALLBACK") |
| 463 | ... return x + y + z |
| 464 | ... |
| 465 | >>> obj = Object() |
| 466 | >>> f = weakref.finalize(obj, callback, 1, 2, z=3) |
| 467 | >>> assert f.alive |
| 468 | >>> assert f() == 6 |
| 469 | CALLBACK |
| 470 | >>> assert not f.alive |
| 471 | >>> f() # callback not called because finalizer dead |
| 472 | >>> del obj # callback not called because finalizer dead |
| 473 | |
| 474 | You can unregister a finalizer using its :meth:`~finalize.detach` |
| 475 | method. This kills the finalizer and returns the arguments passed to |
| 476 | the constructor when it was created. |
| 477 | |
| 478 | >>> obj = Object() |
| 479 | >>> f = weakref.finalize(obj, callback, 1, 2, z=3) |
| 480 | >>> f.detach() #doctest:+ELLIPSIS |
Marco Buttu | 7b2491a | 2017-04-13 16:17:59 +0200 | [diff] [blame] | 481 | (<...Object object ...>, <function callback ...>, (1, 2), {'z': 3}) |
Richard Oudkerk | 7a3dae0 | 2013-05-05 23:05:00 +0100 | [diff] [blame] | 482 | >>> newobj, func, args, kwargs = _ |
| 483 | >>> assert not f.alive |
| 484 | >>> assert newobj is obj |
| 485 | >>> assert func(*args, **kwargs) == 6 |
| 486 | CALLBACK |
| 487 | |
| 488 | Unless you set the :attr:`~finalize.atexit` attribute to |
Nick Coghlan | be57ab8 | 2013-09-22 21:26:30 +1000 | [diff] [blame] | 489 | :const:`False`, a finalizer will be called when the program exits if it |
Richard Oudkerk | 7a3dae0 | 2013-05-05 23:05:00 +0100 | [diff] [blame] | 490 | is still alive. For instance |
| 491 | |
| 492 | >>> obj = Object() |
| 493 | >>> weakref.finalize(obj, print, "obj dead or exiting") #doctest:+ELLIPSIS |
| 494 | <finalize object at ...; for 'Object' at ...> |
| 495 | >>> exit() #doctest:+SKIP |
| 496 | obj dead or exiting |
| 497 | |
| 498 | |
| 499 | Comparing finalizers with :meth:`__del__` methods |
| 500 | ------------------------------------------------- |
| 501 | |
| 502 | Suppose we want to create a class whose instances represent temporary |
| 503 | directories. The directories should be deleted with their contents |
| 504 | when the first of the following events occurs: |
| 505 | |
| 506 | * the object is garbage collected, |
| 507 | * the object's :meth:`remove` method is called, or |
| 508 | * the program exits. |
| 509 | |
| 510 | We might try to implement the class using a :meth:`__del__` method as |
| 511 | follows:: |
| 512 | |
| 513 | class TempDir: |
| 514 | def __init__(self): |
| 515 | self.name = tempfile.mkdtemp() |
| 516 | |
| 517 | def remove(self): |
| 518 | if self.name is not None: |
| 519 | shutil.rmtree(self.name) |
| 520 | self.name = None |
| 521 | |
| 522 | @property |
| 523 | def removed(self): |
| 524 | return self.name is None |
| 525 | |
| 526 | def __del__(self): |
| 527 | self.remove() |
| 528 | |
Nick Coghlan | be57ab8 | 2013-09-22 21:26:30 +1000 | [diff] [blame] | 529 | Starting with Python 3.4, :meth:`__del__` methods no longer prevent |
| 530 | reference cycles from being garbage collected, and module globals are |
Antoine Pitrou | 5db1bb8 | 2014-12-07 01:28:27 +0100 | [diff] [blame] | 531 | no longer forced to :const:`None` during :term:`interpreter shutdown`. |
| 532 | So this code should work without any issues on CPython. |
Richard Oudkerk | 7a3dae0 | 2013-05-05 23:05:00 +0100 | [diff] [blame] | 533 | |
Nick Coghlan | be57ab8 | 2013-09-22 21:26:30 +1000 | [diff] [blame] | 534 | However, handling of :meth:`__del__` methods is notoriously implementation |
Nick Coghlan | 4c7fe6a | 2013-09-22 21:32:12 +1000 | [diff] [blame] | 535 | specific, since it depends on internal details of the interpreter's garbage |
| 536 | collector implementation. |
Nick Coghlan | be57ab8 | 2013-09-22 21:26:30 +1000 | [diff] [blame] | 537 | |
| 538 | A more robust alternative can be to define a finalizer which only references |
| 539 | the specific functions and objects that it needs, rather than having access |
| 540 | to the full state of the object:: |
Richard Oudkerk | 7a3dae0 | 2013-05-05 23:05:00 +0100 | [diff] [blame] | 541 | |
| 542 | class TempDir: |
| 543 | def __init__(self): |
| 544 | self.name = tempfile.mkdtemp() |
| 545 | self._finalizer = weakref.finalize(self, shutil.rmtree, self.name) |
| 546 | |
| 547 | def remove(self): |
| 548 | self._finalizer() |
| 549 | |
| 550 | @property |
| 551 | def removed(self): |
| 552 | return not self._finalizer.alive |
| 553 | |
Nick Coghlan | be57ab8 | 2013-09-22 21:26:30 +1000 | [diff] [blame] | 554 | Defined like this, our finalizer only receives a reference to the details |
| 555 | it needs to clean up the directory appropriately. If the object never gets |
| 556 | garbage collected the finalizer will still be called at exit. |
| 557 | |
| 558 | The other advantage of weakref based finalizers is that they can be used to |
| 559 | register finalizers for classes where the definition is controlled by a |
| 560 | third party, such as running code when a module is unloaded:: |
| 561 | |
| 562 | import weakref, sys |
| 563 | def unloading_module(): |
| 564 | # implicit reference to the module globals from the function body |
| 565 | weakref.finalize(sys.modules[__name__], unloading_module) |
| 566 | |
Richard Oudkerk | 7a3dae0 | 2013-05-05 23:05:00 +0100 | [diff] [blame] | 567 | |
| 568 | .. note:: |
| 569 | |
Donald Stufft | 8b852f1 | 2014-05-20 12:58:38 -0400 | [diff] [blame] | 570 | If you create a finalizer object in a daemonic thread just as the program |
| 571 | exits then there is the possibility that the finalizer |
Richard Oudkerk | 7a3dae0 | 2013-05-05 23:05:00 +0100 | [diff] [blame] | 572 | does not get called at exit. However, in a daemonic thread |
| 573 | :func:`atexit.register`, ``try: ... finally: ...`` and ``with: ...`` |
| 574 | do not guarantee that cleanup occurs either. |