bpo-41229: Update docs for explicit aclose()-required cases and add contextlib.aclosing() method (GH-21545)
This is a PR to:
* Add `contextlib.aclosing` which ia analogous to `contextlib.closing` but for async-generators with an explicit test case for [bpo-41229]()
* Update the docs to describe when we need explicit `aclose()` invocation.
which are motivated by the following issues, articles, and examples:
* [bpo-41229]()
* https://github.com/njsmith/async_generator
* https://vorpus.org/blog/some-thoughts-on-asynchronous-api-design-in-a-post-asyncawait-world/#cleanup-in-generators-and-async-generators
* https://www.python.org/dev/peps/pep-0533/
* https://github.com/achimnol/aiotools/blob/ef7bf0cea7af/src/aiotools/context.py#L152
Particuarly regarding [PEP-533](https://www.python.org/dev/peps/pep-0533/), its acceptance (`__aiterclose__()`) would make this little addition of `contextlib.aclosing()` unnecessary for most use cases, but until then this could serve as a good counterpart and analogy to `contextlib.closing()`. The same applies for `contextlib.closing` with `__iterclose__()`.
Also, still there are other use cases, e.g., when working with non-generator objects with `aclose()` methods.
diff --git a/Doc/library/contextlib.rst b/Doc/library/contextlib.rst
index 0aa4ad7..e42f5a9 100644
--- a/Doc/library/contextlib.rst
+++ b/Doc/library/contextlib.rst
@@ -154,6 +154,39 @@
``page.close()`` will be called when the :keyword:`with` block is exited.
+.. class:: aclosing(thing)
+
+ Return an async context manager that calls the ``aclose()`` method of *thing*
+ upon completion of the block. This is basically equivalent to::
+
+ from contextlib import asynccontextmanager
+
+ @asynccontextmanager
+ async def aclosing(thing):
+ try:
+ yield thing
+ finally:
+ await thing.aclose()
+
+ Significantly, ``aclosing()`` supports deterministic cleanup of async
+ generators when they happen to exit early by :keyword:`break` or an
+ exception. For example::
+
+ from contextlib import aclosing
+
+ async with aclosing(my_generator()) as values:
+ async for value in values:
+ if value == 42:
+ break
+
+ This pattern ensures that the generator's async exit code is executed in
+ the same context as its iterations (so that exceptions and context
+ variables work as expected, and the exit code isn't run after the
+ lifetime of some task it depends on).
+
+ .. versionadded:: 3.10
+
+
.. _simplifying-support-for-single-optional-context-managers:
.. function:: nullcontext(enter_result=None)
diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst
index 512aa5a..8ac6264 100644
--- a/Doc/reference/expressions.rst
+++ b/Doc/reference/expressions.rst
@@ -643,6 +643,16 @@
:meth:`~agen.asend` is used, then the result will be the value passed in to
that method.
+If an asynchronous generator happens to exit early by :keyword:`break`, the caller
+task being cancelled, or other exceptions, the generator's async cleanup code
+will run and possibly raise exceptions or access context variables in an
+unexpected context--perhaps after the lifetime of tasks it depends, or
+during the event loop shutdown when the async-generator garbage collection hook
+is called.
+To prevent this, the caller must explicitly close the async generator by calling
+:meth:`~agen.aclose` method to finalize the generator and ultimately detach it
+from the event loop.
+
In an asynchronous generator function, yield expressions are allowed anywhere
in a :keyword:`try` construct. However, if an asynchronous generator is not
resumed before it is finalized (by reaching a zero reference count or by
@@ -654,9 +664,9 @@
coroutine object, thus allowing any pending :keyword:`!finally` clauses
to execute.
-To take care of finalization, an event loop should define
-a *finalizer* function which takes an asynchronous generator-iterator
-and presumably calls :meth:`~agen.aclose` and executes the coroutine.
+To take care of finalization upon event loop termination, an event loop should
+define a *finalizer* function which takes an asynchronous generator-iterator and
+presumably calls :meth:`~agen.aclose` and executes the coroutine.
This *finalizer* may be registered by calling :func:`sys.set_asyncgen_hooks`.
When first iterated over, an asynchronous generator-iterator will store the
registered *finalizer* to be called upon finalization. For a reference example