Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 1 | .. highlightlang:: c |
| 2 | |
| 3 | .. _supporting-cycle-detection: |
| 4 | |
| 5 | Supporting Cyclic Garbage Collection |
| 6 | ==================================== |
| 7 | |
| 8 | Python's support for detecting and collecting garbage which involves circular |
| 9 | references requires support from object types which are "containers" for other |
| 10 | objects which may also be containers. Types which do not store references to |
| 11 | other objects, or which only store references to atomic types (such as numbers |
Jeroen Ruigrok van der Werven | bd87552 | 2009-04-26 21:06:15 +0000 | [diff] [blame] | 12 | or strings), do not need to provide any explicit support for garbage |
| 13 | collection. |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 14 | |
Antoine Pitrou | 39668f5 | 2013-08-01 21:12:45 +0200 | [diff] [blame] | 15 | To create a container type, the :c:member:`~PyTypeObject.tp_flags` field of the type object must |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 16 | include the :const:`Py_TPFLAGS_HAVE_GC` and provide an implementation of the |
Antoine Pitrou | 39668f5 | 2013-08-01 21:12:45 +0200 | [diff] [blame] | 17 | :c:member:`~PyTypeObject.tp_traverse` handler. If instances of the type are mutable, a |
| 18 | :c:member:`~PyTypeObject.tp_clear` implementation must also be provided. |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 19 | |
| 20 | |
| 21 | .. data:: Py_TPFLAGS_HAVE_GC |
Georg Brandl | c31b0f7 | 2008-01-20 11:00:14 +0000 | [diff] [blame] | 22 | :noindex: |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 23 | |
Jeroen Ruigrok van der Werven | bd87552 | 2009-04-26 21:06:15 +0000 | [diff] [blame] | 24 | Objects with a type with this flag set must conform with the rules |
| 25 | documented here. For convenience these objects will be referred to as |
| 26 | container objects. |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 27 | |
| 28 | Constructors for container types must conform to two rules: |
| 29 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 30 | #. The memory for the object must be allocated using :c:func:`PyObject_GC_New` |
| 31 | or :c:func:`PyObject_GC_NewVar`. |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 32 | |
| 33 | #. Once all the fields which may contain references to other containers are |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 34 | initialized, it must call :c:func:`PyObject_GC_Track`. |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 35 | |
| 36 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 37 | .. c:function:: TYPE* PyObject_GC_New(TYPE, PyTypeObject *type) |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 38 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 39 | Analogous to :c:func:`PyObject_New` but for container objects with the |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 40 | :const:`Py_TPFLAGS_HAVE_GC` flag set. |
| 41 | |
| 42 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 43 | .. c:function:: TYPE* PyObject_GC_NewVar(TYPE, PyTypeObject *type, Py_ssize_t size) |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 44 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 45 | Analogous to :c:func:`PyObject_NewVar` but for container objects with the |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 46 | :const:`Py_TPFLAGS_HAVE_GC` flag set. |
| 47 | |
Jeroen Ruigrok van der Werven | 47a7d70 | 2009-04-27 05:43:17 +0000 | [diff] [blame] | 48 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 49 | .. c:function:: TYPE* PyObject_GC_Resize(TYPE, PyVarObject *op, Py_ssize_t newsize) |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 50 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 51 | Resize an object allocated by :c:func:`PyObject_NewVar`. Returns the |
Jeroen Ruigrok van der Werven | bd87552 | 2009-04-26 21:06:15 +0000 | [diff] [blame] | 52 | resized object or *NULL* on failure. |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 53 | |
| 54 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 55 | .. c:function:: void PyObject_GC_Track(PyObject *op) |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 56 | |
Jeroen Ruigrok van der Werven | bd87552 | 2009-04-26 21:06:15 +0000 | [diff] [blame] | 57 | Adds the object *op* to the set of container objects tracked by the |
| 58 | collector. The collector can run at unexpected times so objects must be |
| 59 | valid while being tracked. This should be called once all the fields |
Antoine Pitrou | 39668f5 | 2013-08-01 21:12:45 +0200 | [diff] [blame] | 60 | followed by the :c:member:`~PyTypeObject.tp_traverse` handler become valid, usually near the |
Jeroen Ruigrok van der Werven | bd87552 | 2009-04-26 21:06:15 +0000 | [diff] [blame] | 61 | end of the constructor. |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 62 | |
| 63 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 64 | .. c:function:: void _PyObject_GC_TRACK(PyObject *op) |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 65 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 66 | A macro version of :c:func:`PyObject_GC_Track`. It should not be used for |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 67 | extension modules. |
| 68 | |
| 69 | Similarly, the deallocator for the object must conform to a similar pair of |
| 70 | rules: |
| 71 | |
| 72 | #. Before fields which refer to other containers are invalidated, |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 73 | :c:func:`PyObject_GC_UnTrack` must be called. |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 74 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 75 | #. The object's memory must be deallocated using :c:func:`PyObject_GC_Del`. |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 76 | |
| 77 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 78 | .. c:function:: void PyObject_GC_Del(void *op) |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 79 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 80 | Releases memory allocated to an object using :c:func:`PyObject_GC_New` or |
| 81 | :c:func:`PyObject_GC_NewVar`. |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 82 | |
| 83 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 84 | .. c:function:: void PyObject_GC_UnTrack(void *op) |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 85 | |
| 86 | Remove the object *op* from the set of container objects tracked by the |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 87 | collector. Note that :c:func:`PyObject_GC_Track` can be called again on |
Jeroen Ruigrok van der Werven | bd87552 | 2009-04-26 21:06:15 +0000 | [diff] [blame] | 88 | this object to add it back to the set of tracked objects. The deallocator |
Antoine Pitrou | 39668f5 | 2013-08-01 21:12:45 +0200 | [diff] [blame] | 89 | (:c:member:`~PyTypeObject.tp_dealloc` handler) should call this for the object before any of |
| 90 | the fields used by the :c:member:`~PyTypeObject.tp_traverse` handler become invalid. |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 91 | |
| 92 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 93 | .. c:function:: void _PyObject_GC_UNTRACK(PyObject *op) |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 94 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 95 | A macro version of :c:func:`PyObject_GC_UnTrack`. It should not be used for |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 96 | extension modules. |
| 97 | |
Antoine Pitrou | 39668f5 | 2013-08-01 21:12:45 +0200 | [diff] [blame] | 98 | The :c:member:`~PyTypeObject.tp_traverse` handler accepts a function parameter of this type: |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 99 | |
| 100 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 101 | .. c:type:: int (*visitproc)(PyObject *object, void *arg) |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 102 | |
Antoine Pitrou | 39668f5 | 2013-08-01 21:12:45 +0200 | [diff] [blame] | 103 | Type of the visitor function passed to the :c:member:`~PyTypeObject.tp_traverse` handler. |
Jeroen Ruigrok van der Werven | bd87552 | 2009-04-26 21:06:15 +0000 | [diff] [blame] | 104 | The function should be called with an object to traverse as *object* and |
Antoine Pitrou | 39668f5 | 2013-08-01 21:12:45 +0200 | [diff] [blame] | 105 | the third parameter to the :c:member:`~PyTypeObject.tp_traverse` handler as *arg*. The |
Jeroen Ruigrok van der Werven | bd87552 | 2009-04-26 21:06:15 +0000 | [diff] [blame] | 106 | Python core uses several visitor functions to implement cyclic garbage |
| 107 | detection; it's not expected that users will need to write their own |
| 108 | visitor functions. |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 109 | |
Antoine Pitrou | 39668f5 | 2013-08-01 21:12:45 +0200 | [diff] [blame] | 110 | The :c:member:`~PyTypeObject.tp_traverse` handler must have the following type: |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 111 | |
| 112 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 113 | .. c:type:: int (*traverseproc)(PyObject *self, visitproc visit, void *arg) |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 114 | |
| 115 | Traversal function for a container object. Implementations must call the |
| 116 | *visit* function for each object directly contained by *self*, with the |
Jeroen Ruigrok van der Werven | bd87552 | 2009-04-26 21:06:15 +0000 | [diff] [blame] | 117 | parameters to *visit* being the contained object and the *arg* value passed |
| 118 | to the handler. The *visit* function must not be called with a *NULL* |
| 119 | object argument. If *visit* returns a non-zero value that value should be |
| 120 | returned immediately. |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 121 | |
Antoine Pitrou | 39668f5 | 2013-08-01 21:12:45 +0200 | [diff] [blame] | 122 | To simplify writing :c:member:`~PyTypeObject.tp_traverse` handlers, a :c:func:`Py_VISIT` macro is |
| 123 | provided. In order to use this macro, the :c:member:`~PyTypeObject.tp_traverse` implementation |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 124 | must name its arguments exactly *visit* and *arg*: |
| 125 | |
| 126 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 127 | .. c:function:: void Py_VISIT(PyObject *o) |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 128 | |
Jeroen Ruigrok van der Werven | bd87552 | 2009-04-26 21:06:15 +0000 | [diff] [blame] | 129 | Call the *visit* callback, with arguments *o* and *arg*. If *visit* returns |
Antoine Pitrou | 39668f5 | 2013-08-01 21:12:45 +0200 | [diff] [blame] | 130 | a non-zero value, then return it. Using this macro, :c:member:`~PyTypeObject.tp_traverse` |
Jeroen Ruigrok van der Werven | bd87552 | 2009-04-26 21:06:15 +0000 | [diff] [blame] | 131 | handlers look like:: |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 132 | |
| 133 | static int |
| 134 | my_traverse(Noddy *self, visitproc visit, void *arg) |
| 135 | { |
| 136 | Py_VISIT(self->foo); |
| 137 | Py_VISIT(self->bar); |
| 138 | return 0; |
| 139 | } |
| 140 | |
Antoine Pitrou | 39668f5 | 2013-08-01 21:12:45 +0200 | [diff] [blame] | 141 | The :c:member:`~PyTypeObject.tp_clear` handler must be of the :c:type:`inquiry` type, or *NULL* |
Jeroen Ruigrok van der Werven | bd87552 | 2009-04-26 21:06:15 +0000 | [diff] [blame] | 142 | if the object is immutable. |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 143 | |
| 144 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 145 | .. c:type:: int (*inquiry)(PyObject *self) |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 146 | |
Jeroen Ruigrok van der Werven | bd87552 | 2009-04-26 21:06:15 +0000 | [diff] [blame] | 147 | Drop references that may have created reference cycles. Immutable objects |
| 148 | do not have to define this method since they can never directly create |
| 149 | reference cycles. Note that the object must still be valid after calling |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 150 | this method (don't just call :c:func:`Py_DECREF` on a reference). The |
Jeroen Ruigrok van der Werven | bd87552 | 2009-04-26 21:06:15 +0000 | [diff] [blame] | 151 | collector will call this method if it detects that this object is involved |
| 152 | in a reference cycle. |