Fred Drake | ebcf6a8 | 2001-02-01 05:20:20 +0000 | [diff] [blame] | 1 | \section{\module{weakref} --- |
| 2 | Weak references} |
| 3 | |
| 4 | \declaremodule{extension}{weakref} |
Fred Drake | eedf985 | 2001-04-11 19:17:11 +0000 | [diff] [blame] | 5 | \modulesynopsis{Support for weak references and weak dictionaries.} |
Fred Drake | ebcf6a8 | 2001-02-01 05:20:20 +0000 | [diff] [blame] | 6 | \moduleauthor{Fred L. Drake, Jr.}{fdrake@acm.org} |
Martin v. Löwis | 5e16333 | 2001-02-27 18:36:56 +0000 | [diff] [blame] | 7 | \moduleauthor{Neil Schemenauer}{nas@arctrix.com} |
Fred Drake | 0c20904 | 2001-06-29 16:25:07 +0000 | [diff] [blame] | 8 | \moduleauthor{Martin von L\"owis}{martin@loewis.home.cs.tu-berlin.de} |
Fred Drake | ebcf6a8 | 2001-02-01 05:20:20 +0000 | [diff] [blame] | 9 | \sectionauthor{Fred L. Drake, Jr.}{fdrake@acm.org} |
| 10 | |
| 11 | \versionadded{2.1} |
| 12 | |
| 13 | |
| 14 | The \module{weakref} module allows the Python programmer to create |
| 15 | \dfn{weak references} to objects. |
| 16 | |
Fred Drake | 7408da5 | 2001-10-26 17:40:22 +0000 | [diff] [blame] | 17 | In the discussion which follows, the term \dfn{referent} means the |
| 18 | object which is referred to by a weak reference. |
| 19 | |
Fred Drake | ebcf6a8 | 2001-02-01 05:20:20 +0000 | [diff] [blame] | 20 | XXX --- need to say more here! |
| 21 | |
| 22 | Not all objects can be weakly referenced; those objects which do |
Fred Drake | 5e0dfac | 2001-03-23 04:36:02 +0000 | [diff] [blame] | 23 | include class instances, functions written in Python (but not in C), |
| 24 | and methods (both bound and unbound). Extension types can easily |
Fred Drake | ebcf6a8 | 2001-02-01 05:20:20 +0000 | [diff] [blame] | 25 | be made to support weak references; see section \ref{weakref-extension}, |
| 26 | ``Weak References in Extension Types,'' for more information. |
| 27 | |
Fred Drake | ebcf6a8 | 2001-02-01 05:20:20 +0000 | [diff] [blame] | 28 | |
| 29 | \begin{funcdesc}{ref}{object\optional{, callback}} |
Fred Drake | 7408da5 | 2001-10-26 17:40:22 +0000 | [diff] [blame] | 30 | Return a weak reference to \var{object}. The original object can be |
| 31 | retrieved by calling the reference object if the referent is still |
| 32 | alive; if the referent is no longer alive, calling the reference |
| 33 | object will cause \code{None} to be returned. If \var{callback} is |
Fred Drake | ebcf6a8 | 2001-02-01 05:20:20 +0000 | [diff] [blame] | 34 | provided, it will be called when the object is about to be |
| 35 | finalized; the weak reference object will be passed as the only |
| 36 | parameter to the callback; the referent will no longer be available. |
Fred Drake | ebcf6a8 | 2001-02-01 05:20:20 +0000 | [diff] [blame] | 37 | |
| 38 | It is allowable for many weak references to be constructed for the |
| 39 | same object. Callbacks registered for each weak reference will be |
| 40 | called from the most recently registered callback to the oldest |
| 41 | registered callback. |
| 42 | |
| 43 | Exceptions raised by the callback will be noted on the standard |
Andrew M. Kuchling | e7d7e6c | 2001-02-14 02:39:11 +0000 | [diff] [blame] | 44 | error output, but cannot be propagated; they are handled in exactly |
Fred Drake | ebcf6a8 | 2001-02-01 05:20:20 +0000 | [diff] [blame] | 45 | the same way as exceptions raised from an object's |
| 46 | \method{__del__()} method. |
Martin v. Löwis | 5e16333 | 2001-02-27 18:36:56 +0000 | [diff] [blame] | 47 | |
| 48 | Weak references are hashable if the \var{object} is hashable. They |
| 49 | will maintain their hash value even after the \var{object} was |
| 50 | deleted. If \function{hash()} is called the first time only after |
| 51 | the \var{object} was deleted, the call will raise |
| 52 | \exception{TypeError}. |
| 53 | |
Fred Drake | 3a2c462 | 2001-10-26 03:00:39 +0000 | [diff] [blame] | 54 | Weak references support tests for equality, but not ordering. If |
Fred Drake | 7408da5 | 2001-10-26 17:40:22 +0000 | [diff] [blame] | 55 | the referents are still alive, two references have the same |
Fred Drake | b03d0cc | 2001-11-26 21:39:40 +0000 | [diff] [blame] | 56 | equality relationship as their referents (regardless of the |
Fred Drake | 7408da5 | 2001-10-26 17:40:22 +0000 | [diff] [blame] | 57 | \var{callback}). If either referent has been deleted, the |
| 58 | references are equal only if the reference objects are the same |
| 59 | object. |
Fred Drake | ebcf6a8 | 2001-02-01 05:20:20 +0000 | [diff] [blame] | 60 | \end{funcdesc} |
| 61 | |
Fred Drake | ebcf6a8 | 2001-02-01 05:20:20 +0000 | [diff] [blame] | 62 | \begin{funcdesc}{proxy}{object\optional{, callback}} |
| 63 | Return a proxy to \var{object} which uses a weak reference. This |
| 64 | supports use of the proxy in most contexts instead of requiring the |
| 65 | explicit dereferencing used with weak reference objects. The |
| 66 | returned object will have a type of either \code{ProxyType} or |
| 67 | \code{CallableProxyType}, depending on whether \var{object} is |
| 68 | callable. Proxy objects are not hashable regardless of the |
| 69 | referent; this avoids a number of problems related to their |
| 70 | fundamentally mutable nature, and prevent their use as dictionary |
Fred Drake | e7ec1ef | 2001-05-10 17:22:17 +0000 | [diff] [blame] | 71 | keys. \var{callback} is the same as the parameter of the same name |
Fred Drake | ebcf6a8 | 2001-02-01 05:20:20 +0000 | [diff] [blame] | 72 | to the \function{ref()} function. |
| 73 | \end{funcdesc} |
| 74 | |
| 75 | \begin{funcdesc}{getweakrefcount}{object} |
| 76 | Return the number of weak references and proxies which refer to |
| 77 | \var{object}. |
| 78 | \end{funcdesc} |
| 79 | |
| 80 | \begin{funcdesc}{getweakrefs}{object} |
| 81 | Return a list of all weak reference and proxy objects which refer to |
| 82 | \var{object}. |
| 83 | \end{funcdesc} |
| 84 | |
Martin v. Löwis | 5e16333 | 2001-02-27 18:36:56 +0000 | [diff] [blame] | 85 | \begin{classdesc}{WeakKeyDictionary}{\optional{dict}} |
Fred Drake | ac154a1 | 2001-04-10 19:57:58 +0000 | [diff] [blame] | 86 | Mapping class that references keys weakly. Entries in the |
| 87 | dictionary will be discarded when there is no longer a strong |
| 88 | reference to the key. This can be used to associate additional data |
| 89 | with an object owned by other parts of an application without adding |
| 90 | attributes to those objects. This can be especially useful with |
| 91 | objects that override attribute accesses. |
Martin v. Löwis | 5e16333 | 2001-02-27 18:36:56 +0000 | [diff] [blame] | 92 | \end{classdesc} |
| 93 | |
| 94 | \begin{classdesc}{WeakValueDictionary}{\optional{dict}} |
Fred Drake | ac154a1 | 2001-04-10 19:57:58 +0000 | [diff] [blame] | 95 | Mapping class that references values weakly. Entries in the |
| 96 | dictionary will be discarded when no strong reference to the value |
Fred Drake | 7408da5 | 2001-10-26 17:40:22 +0000 | [diff] [blame] | 97 | exists any more. |
Fred Drake | ebcf6a8 | 2001-02-01 05:20:20 +0000 | [diff] [blame] | 98 | \end{classdesc} |
| 99 | |
| 100 | \begin{datadesc}{ReferenceType} |
| 101 | The type object for weak references objects. |
| 102 | \end{datadesc} |
| 103 | |
| 104 | \begin{datadesc}{ProxyType} |
| 105 | The type object for proxies of objects which are not callable. |
| 106 | \end{datadesc} |
| 107 | |
| 108 | \begin{datadesc}{CallableProxyType} |
| 109 | The type object for proxies of callable objects. |
| 110 | \end{datadesc} |
| 111 | |
| 112 | \begin{datadesc}{ProxyTypes} |
| 113 | Sequence containing all the type objects for proxies. This can make |
| 114 | it simpler to test if an object is a proxy without being dependent |
| 115 | on naming both proxy types. |
| 116 | \end{datadesc} |
| 117 | |
Fred Drake | ac154a1 | 2001-04-10 19:57:58 +0000 | [diff] [blame] | 118 | \begin{excdesc}{ReferenceError} |
| 119 | Exception raised when a proxy object is used but the underlying |
Fred Drake | 8c2c3d3 | 2001-10-06 06:10:54 +0000 | [diff] [blame] | 120 | object has been collected. This is the same as the standard |
| 121 | \exception{ReferenceError} exception. |
Fred Drake | ac154a1 | 2001-04-10 19:57:58 +0000 | [diff] [blame] | 122 | \end{excdesc} |
| 123 | |
Fred Drake | ebcf6a8 | 2001-02-01 05:20:20 +0000 | [diff] [blame] | 124 | |
| 125 | \begin{seealso} |
| 126 | \seepep{0205}{Weak References}{The proposal and rationale for this |
| 127 | feature, including links to earlier implementations |
| 128 | and information about similar features in other |
| 129 | languages.} |
| 130 | \end{seealso} |
| 131 | |
| 132 | |
| 133 | \subsection{Weak Reference Objects |
| 134 | \label{weakref-objects}} |
| 135 | |
| 136 | Weak reference objects have no attributes or methods, but do allow the |
| 137 | referent to be obtained, if it still exists, by calling it: |
| 138 | |
| 139 | \begin{verbatim} |
| 140 | >>> import weakref |
| 141 | >>> class Object: |
| 142 | ... pass |
| 143 | ... |
| 144 | >>> o = Object() |
| 145 | >>> r = weakref.ref(o) |
| 146 | >>> o2 = r() |
| 147 | >>> o is o2 |
| 148 | 1 |
| 149 | \end{verbatim} |
| 150 | |
| 151 | If the referent no longer exists, calling the reference object returns |
| 152 | \code{None}: |
| 153 | |
| 154 | \begin{verbatim} |
| 155 | >>> del o, o2 |
| 156 | >>> print r() |
| 157 | None |
| 158 | \end{verbatim} |
| 159 | |
| 160 | Testing that a weak reference object is still live should be done |
Fred Drake | 5d54879 | 2001-08-03 03:50:28 +0000 | [diff] [blame] | 161 | using the expression \code{\var{ref}() is not None}. Normally, |
Fred Drake | ebcf6a8 | 2001-02-01 05:20:20 +0000 | [diff] [blame] | 162 | application code that needs to use a reference object should follow |
| 163 | this pattern: |
| 164 | |
| 165 | \begin{verbatim} |
Fred Drake | 7408da5 | 2001-10-26 17:40:22 +0000 | [diff] [blame] | 166 | # r is a weak reference object |
| 167 | o = r() |
Fred Drake | ebcf6a8 | 2001-02-01 05:20:20 +0000 | [diff] [blame] | 168 | if o is None: |
| 169 | # referent has been garbage collected |
| 170 | print "Object has been allocated; can't frobnicate." |
| 171 | else: |
| 172 | print "Object is still live!" |
| 173 | o.do_something_useful() |
| 174 | \end{verbatim} |
| 175 | |
| 176 | Using a separate test for ``liveness'' creates race conditions in |
| 177 | threaded applications; another thread can cause a weak reference to |
Fred Drake | 7408da5 | 2001-10-26 17:40:22 +0000 | [diff] [blame] | 178 | become invalidated before the weak reference is called; the |
Fred Drake | ebcf6a8 | 2001-02-01 05:20:20 +0000 | [diff] [blame] | 179 | idiom shown above is safe in threaded applications as well as |
| 180 | single-threaded applications. |
| 181 | |
| 182 | |
Fred Drake | cb83988 | 2001-03-28 21:15:41 +0000 | [diff] [blame] | 183 | \subsection{Example \label{weakref-example}} |
| 184 | |
| 185 | This simple example shows how an application can use objects IDs to |
| 186 | retrieve objects that it has seen before. The IDs of the objects can |
| 187 | then be used in other data structures without forcing the objects to |
| 188 | remain alive, but the objects can still be retrieved by ID if they |
| 189 | do. |
| 190 | |
| 191 | % Example contributed by Tim Peters <tim_one@msn.com>. |
| 192 | \begin{verbatim} |
| 193 | import weakref |
| 194 | |
Fred Drake | ac154a1 | 2001-04-10 19:57:58 +0000 | [diff] [blame] | 195 | _id2obj_dict = weakref.WeakValueDictionary() |
Fred Drake | cb83988 | 2001-03-28 21:15:41 +0000 | [diff] [blame] | 196 | |
| 197 | def remember(obj): |
Fred Drake | 7408da5 | 2001-10-26 17:40:22 +0000 | [diff] [blame] | 198 | oid = id(obj) |
| 199 | _id2obj_dict[oid] = obj |
| 200 | return oid |
Fred Drake | cb83988 | 2001-03-28 21:15:41 +0000 | [diff] [blame] | 201 | |
Fred Drake | 7408da5 | 2001-10-26 17:40:22 +0000 | [diff] [blame] | 202 | def id2obj(oid): |
| 203 | return _id2obj_dict[oid] |
Fred Drake | cb83988 | 2001-03-28 21:15:41 +0000 | [diff] [blame] | 204 | \end{verbatim} |
| 205 | |
| 206 | |
Fred Drake | ebcf6a8 | 2001-02-01 05:20:20 +0000 | [diff] [blame] | 207 | \subsection{Weak References in Extension Types |
| 208 | \label{weakref-extension}} |
| 209 | |
| 210 | One of the goals of the implementation is to allow any type to |
| 211 | participate in the weak reference mechanism without incurring the |
| 212 | overhead on those objects which do not benefit by weak referencing |
| 213 | (such as numbers). |
| 214 | |
| 215 | For an object to be weakly referencable, the extension must include a |
Fred Drake | 7408da5 | 2001-10-26 17:40:22 +0000 | [diff] [blame] | 216 | \ctype{PyObject*} field in the instance structure for the use of the |
Fred Drake | 5e0dfac | 2001-03-23 04:36:02 +0000 | [diff] [blame] | 217 | weak reference mechanism; it must be initialized to \NULL{} by the |
| 218 | object's constructor. It must also set the \member{tp_weaklistoffset} |
Fred Drake | ebcf6a8 | 2001-02-01 05:20:20 +0000 | [diff] [blame] | 219 | field of the corresponding type object to the offset of the field. |
Raymond Hettinger | 22c001b | 2002-08-07 16:18:54 +0000 | [diff] [blame^] | 220 | Also, it needs to add \constant{Py_TPFLAGS_HAVE_WEAKREFS} to the |
| 221 | tp_flags slot. For example, the instance type is defined with the |
| 222 | following structure: |
Fred Drake | ebcf6a8 | 2001-02-01 05:20:20 +0000 | [diff] [blame] | 223 | |
| 224 | \begin{verbatim} |
| 225 | typedef struct { |
| 226 | PyObject_HEAD |
| 227 | PyClassObject *in_class; /* The class object */ |
Martin v. Löwis | 5e16333 | 2001-02-27 18:36:56 +0000 | [diff] [blame] | 228 | PyObject *in_dict; /* A dictionary */ |
| 229 | PyObject *in_weakreflist; /* List of weak references */ |
Fred Drake | ebcf6a8 | 2001-02-01 05:20:20 +0000 | [diff] [blame] | 230 | } PyInstanceObject; |
| 231 | \end{verbatim} |
| 232 | |
| 233 | The statically-declared type object for instances is defined this way: |
| 234 | |
| 235 | \begin{verbatim} |
| 236 | PyTypeObject PyInstance_Type = { |
| 237 | PyObject_HEAD_INIT(&PyType_Type) |
| 238 | 0, |
Guido van Rossum | 1464839 | 2001-12-08 18:02:58 +0000 | [diff] [blame] | 239 | "module.instance", |
Fred Drake | ebcf6a8 | 2001-02-01 05:20:20 +0000 | [diff] [blame] | 240 | |
Fred Drake | f66cb5d | 2001-06-22 17:20:29 +0000 | [diff] [blame] | 241 | /* Lots of stuff omitted for brevity... */ |
Fred Drake | ebcf6a8 | 2001-02-01 05:20:20 +0000 | [diff] [blame] | 242 | |
Raymond Hettinger | 22c001b | 2002-08-07 16:18:54 +0000 | [diff] [blame^] | 243 | Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_WEAKREFS /* tp_flags */ |
| 244 | 0, /* tp_doc */ |
| 245 | 0, /* tp_traverse */ |
| 246 | 0, /* tp_clear */ |
| 247 | 0, /* tp_richcompare */ |
| 248 | offsetof(PyInstanceObject, in_weakreflist), /* tp_weaklistoffset */ |
Fred Drake | ebcf6a8 | 2001-02-01 05:20:20 +0000 | [diff] [blame] | 249 | }; |
| 250 | \end{verbatim} |
| 251 | |
Raymond Hettinger | 22c001b | 2002-08-07 16:18:54 +0000 | [diff] [blame^] | 252 | The type constructor is responsible for initializing the weak reference |
| 253 | list to \NULL: |
| 254 | |
| 255 | \begin{verbatim} |
| 256 | static PyObject * |
| 257 | instance_new() { |
| 258 | /* Other initialization stuff omitted for brevity */ |
| 259 | |
| 260 | self->in_weakreflist = NULL; |
| 261 | |
| 262 | return (PyObject *) self; |
| 263 | } |
| 264 | \end{verbatim} |
| 265 | |
Fred Drake | ebcf6a8 | 2001-02-01 05:20:20 +0000 | [diff] [blame] | 266 | The only further addition is that the destructor needs to call the |
Fred Drake | f66cb5d | 2001-06-22 17:20:29 +0000 | [diff] [blame] | 267 | weak reference manager to clear any weak references. This should be |
Fred Drake | 7408da5 | 2001-10-26 17:40:22 +0000 | [diff] [blame] | 268 | done before any other parts of the destruction have occurred, but is |
| 269 | only required if the weak reference list is non-\NULL: |
Fred Drake | ebcf6a8 | 2001-02-01 05:20:20 +0000 | [diff] [blame] | 270 | |
| 271 | \begin{verbatim} |
| 272 | static void |
| 273 | instance_dealloc(PyInstanceObject *inst) |
| 274 | { |
Fred Drake | 7408da5 | 2001-10-26 17:40:22 +0000 | [diff] [blame] | 275 | /* Allocate temporaries if needed, but do not begin |
Fred Drake | f66cb5d | 2001-06-22 17:20:29 +0000 | [diff] [blame] | 276 | destruction just yet. |
Fred Drake | ebcf6a8 | 2001-02-01 05:20:20 +0000 | [diff] [blame] | 277 | */ |
| 278 | |
Fred Drake | 7408da5 | 2001-10-26 17:40:22 +0000 | [diff] [blame] | 279 | if (inst->in_weakreflist != NULL) |
| 280 | PyObject_ClearWeakRefs((PyObject *) inst); |
Fred Drake | ebcf6a8 | 2001-02-01 05:20:20 +0000 | [diff] [blame] | 281 | |
Fred Drake | de3d060 | 2001-10-26 11:27:54 +0000 | [diff] [blame] | 282 | /* Proceed with object destruction normally. */ |
Fred Drake | ebcf6a8 | 2001-02-01 05:20:20 +0000 | [diff] [blame] | 283 | } |
| 284 | \end{verbatim} |