Issue #18112: PEP 442 implementation (safe object finalization).
diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst
index ea1a0ad..ea787de 100644
--- a/Doc/c-api/typeobj.rst
+++ b/Doc/c-api/typeobj.rst
@@ -465,6 +465,14 @@
:const:`Py_TPFLAGS_HAVE_VERSION_TAG`.
+ .. data:: Py_TPFLAGS_HAVE_FINALIZE
+
+ This bit is set when the :attr:`tp_finalize` slot is present in the
+ type structure.
+
+ .. versionadded:: 3.4
+
+
.. c:member:: char* PyTypeObject.tp_doc
An optional pointer to a NUL-terminated C string giving the docstring for this
@@ -968,6 +976,47 @@
This field is not inherited; it is calculated fresh by :c:func:`PyType_Ready`.
+.. c:member:: destructor PyTypeObject.tp_finalize
+
+ An optional pointer to an instance finalization function. Its signature is
+ :c:type:`destructor`::
+
+ void tp_finalize(PyObject *)
+
+ If :attr:`tp_finalize` is set, the interpreter calls it once when
+ finalizing an instance. It is called either from the garbage
+ collector (if the instance is part of an isolated reference cycle) or
+ just before the object is deallocated. Either way, it is guaranteed
+ to be called before attempting to break reference cycles, ensuring
+ that it finds the object in a sane state.
+
+ :attr:`tp_finalize` should not mutate the current exception status;
+ therefore, a recommended way to write a non-trivial finalizer is::
+
+ static void
+ local_finalize(PyObject *self)
+ {
+ PyObject *error_type, *error_value, *error_traceback;
+
+ /* Save the current exception, if any. */
+ PyErr_Fetch(&error_type, &error_value, &error_traceback);
+
+ /* ... */
+
+ /* Restore the saved exception. */
+ PyErr_Restore(error_type, error_value, error_traceback);
+ }
+
+ For this field to be taken into account (even through inheritance),
+ you must also set the :const:`Py_TPFLAGS_HAVE_FINALIZE` flags bit.
+
+ This field is inherited by subtypes.
+
+ .. versionadded:: 3.4
+
+ .. seealso:: "Safe object finalization" (:pep:`442`)
+
+
.. c:member:: PyObject* PyTypeObject.tp_cache
Unused. Not inherited. Internal use only.
diff --git a/Doc/extending/newtypes.rst b/Doc/extending/newtypes.rst
index cb20bce..d5ee877 100644
--- a/Doc/extending/newtypes.rst
+++ b/Doc/extending/newtypes.rst
@@ -157,7 +157,8 @@
Py_TPFLAGS_DEFAULT, /* tp_flags */
All types should include this constant in their flags. It enables all of the
-members defined by the current version of Python.
+members defined until at least Python 3.3. If you need further members,
+you will need to OR the corresponding flags.
We provide a doc string for the type in :attr:`tp_doc`. ::
@@ -928,8 +929,9 @@
This function is called when the reference count of the instance of your type is
reduced to zero and the Python interpreter wants to reclaim it. If your type
-has memory to free or other clean-up to perform, put it here. The object itself
-needs to be freed here as well. Here is an example of this function::
+has memory to free or other clean-up to perform, you can put it here. The
+object itself needs to be freed here as well. Here is an example of this
+function::
static void
newdatatype_dealloc(newdatatypeobject * obj)
@@ -981,6 +983,22 @@
Py_TYPE(obj)->tp_free((PyObject*)self);
}
+.. note::
+ There are limitations to what you can safely do in a deallocator function.
+ First, if your type supports garbage collection (using :attr:`tp_traverse`
+ and/or :attr:`tp_clear`), some of the object's members can have been
+ cleared or finalized by the time :attr:`tp_dealloc` is called. Second, in
+ :attr:`tp_dealloc`, your object is in an unstable state: its reference
+ count is equal to zero. Any call to a non-trivial object or API (as in the
+ example above) might end up calling :attr:`tp_dealloc` again, causing a
+ double free and a crash.
+
+ Starting with Python 3.4, it is recommended not to put any complex
+ finalization code in :attr:`tp_dealloc`, and instead use the new
+ :c:member:`~PyTypeObject.tp_finalize` type method.
+
+ .. seealso::
+ :pep:`442` explains the new finalization scheme.
.. index::
single: string; object representation
diff --git a/Doc/includes/typestruct.h b/Doc/includes/typestruct.h
index 32647c0..fcb846a 100644
--- a/Doc/includes/typestruct.h
+++ b/Doc/includes/typestruct.h
@@ -70,4 +70,11 @@
PyObject *tp_subclasses;
PyObject *tp_weaklist;
+ destructor tp_del;
+
+ /* Type attribute cache version tag. Added in version 2.6 */
+ unsigned int tp_version_tag;
+
+ destructor tp_finalize;
+
} PyTypeObject;
diff --git a/Doc/library/gc.rst b/Doc/library/gc.rst
index 95df2f8..c578690 100644
--- a/Doc/library/gc.rst
+++ b/Doc/library/gc.rst
@@ -176,24 +176,13 @@
.. data:: garbage
- A list of objects which the collector found to be unreachable but could not be
- freed (uncollectable objects). By default, this list contains only objects with
- :meth:`__del__` methods. Objects that have :meth:`__del__` methods and are
- part of a reference cycle cause the entire reference cycle to be uncollectable,
- including objects not necessarily in the cycle but reachable only from it.
- Python doesn't collect such cycles automatically because, in general, it isn't
- possible for Python to guess a safe order in which to run the :meth:`__del__`
- methods. If you know a safe order, you can force the issue by examining the
- *garbage* list, and explicitly breaking cycles due to your objects within the
- list. Note that these objects are kept alive even so by virtue of being in the
- *garbage* list, so they should be removed from *garbage* too. For example,
- after breaking cycles, do ``del gc.garbage[:]`` to empty the list. It's
- generally better to avoid the issue by not creating cycles containing objects
- with :meth:`__del__` methods, and *garbage* can be examined in that case to
- verify that no such cycles are being created.
+ A list of objects which the collector found to be unreachable but could
+ not be freed (uncollectable objects). Starting with Python 3.4, this
+ list should be empty most of the time, except when using instances of
+ C extension types with a non-NULL ``tp_del`` slot.
- If :const:`DEBUG_SAVEALL` is set, then all unreachable objects will be added
- to this list rather than freed.
+ If :const:`DEBUG_SAVEALL` is set, then all unreachable objects will be
+ added to this list rather than freed.
.. versionchanged:: 3.2
If this list is non-empty at interpreter shutdown, a
@@ -201,6 +190,10 @@
:const:`DEBUG_UNCOLLECTABLE` is set, in addition all uncollectable objects
are printed.
+ .. versionchanged:: 3.4
+ Following :pep:`442`, objects with a :meth:`__del__` method don't end
+ up in :attr:`gc.garbage` anymore.
+
.. data:: callbacks
A list of callbacks that will be invoked by the garbage collector before and
diff --git a/Doc/library/weakref.rst b/Doc/library/weakref.rst
index 5b5e460..818a6fa 100644
--- a/Doc/library/weakref.rst
+++ b/Doc/library/weakref.rst
@@ -529,22 +529,13 @@
def __del__(self):
self.remove()
-This solution has a couple of serious problems:
+This solution has a serious problem: the :meth:`__del__` method may be
+called at shutdown after the :mod:`shutil` module has been cleaned up,
+in which case :attr:`shutil.rmtree` will have been replaced by :const:`None`.
+This will cause the :meth:`__del__` method to fail and the directory
+will not be removed.
-* There is no guarantee that the object will be garbage collected
- before the program exists, so the directory might be left. This is
- because reference cycles containing an object with a :meth:`__del__`
- method can never be collected. And even if the :class:`TempDir`
- object is not itself part of a reference cycle, it may still be kept
- alive by some unkown uncollectable reference cycle.
-
-* The :meth:`__del__` method may be called at shutdown after the
- :mod:`shutil` module has been cleaned up, in which case
- :attr:`shutil.rmtree` will have been replaced by :const:`None`.
- This will cause the :meth:`__del__` method to fail and the directory
- will not be removed.
-
-Using finalizers we can avoid these problems::
+Using finalizers we can avoid this problem::
class TempDir:
def __init__(self):
diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst
index 1a48c1f..95028c2 100644
--- a/Doc/reference/datamodel.rst
+++ b/Doc/reference/datamodel.rst
@@ -1120,12 +1120,10 @@
``sys.last_traceback`` keeps the stack frame alive). The first situation
can only be remedied by explicitly breaking the cycles; the latter two
situations can be resolved by storing ``None`` in ``sys.last_traceback``.
- Circular references which are garbage are detected when the option cycle
- detector is enabled (it's on by default), but can only be cleaned up if
- there are no Python- level :meth:`__del__` methods involved. Refer to the
- documentation for the :mod:`gc` module for more information about how
- :meth:`__del__` methods are handled by the cycle detector, particularly
- the description of the ``garbage`` value.
+ Circular references which are garbage are detected and cleaned up when
+ the cyclic garbage collector is enabled (it's on by default). Refer to the
+ documentation for the :mod:`gc` module for more information about this
+ topic.
.. warning::
diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst
index b5be568..17cec3f 100644
--- a/Doc/whatsnew/3.4.rst
+++ b/Doc/whatsnew/3.4.rst
@@ -119,6 +119,21 @@
Python memory allocators.
+.. _pep-442:
+
+PEP 442: Safe object finalization
+=================================
+
+This PEP removes the current limitations and quirks of object finalization.
+With it, objects with :meth:`__del__` methods, as well as generators
+with :keyword:`finally` clauses, can be finalized when they are part of a
+reference cycle.
+
+.. seealso::
+
+ :pep:`442` - Safe object finalization
+ PEP written and implemented by Antoine Pitrou
+
Other Language Changes
======================