Victor Stinner | db39a0d | 2014-01-16 18:58:01 +0100 | [diff] [blame] | 1 | .. currentmodule:: asyncio |
| 2 | |
Victor Stinner | 0f3e6bc | 2014-02-19 23:15:02 +0100 | [diff] [blame] | 3 | .. _asyncio-dev: |
| 4 | |
Victor Stinner | db39a0d | 2014-01-16 18:58:01 +0100 | [diff] [blame] | 5 | Develop with asyncio |
| 6 | ==================== |
| 7 | |
| 8 | Asynchronous programming is different than classical "sequential" programming. |
Eli Bendersky | 679688e | 2014-01-20 08:13:31 -0800 | [diff] [blame] | 9 | This page lists common traps and explains how to avoid them. |
Victor Stinner | db39a0d | 2014-01-16 18:58:01 +0100 | [diff] [blame] | 10 | |
| 11 | |
Victor Stinner | 62511fd | 2014-06-23 00:36:11 +0200 | [diff] [blame] | 12 | .. _asyncio-debug-mode: |
| 13 | |
| 14 | Debug mode of asyncio |
| 15 | --------------------- |
| 16 | |
Andrew Svetlov | 4919907 | 2015-09-24 14:32:39 +0300 | [diff] [blame] | 17 | The implementation of :mod:`asyncio` has been written for performance. |
| 18 | In order to ease the development of asynchronous code, you may wish to |
| 19 | enable *debug mode*. |
Victor Stinner | d71dcbb | 2014-08-25 17:04:12 +0200 | [diff] [blame] | 20 | |
Andrew Svetlov | 4919907 | 2015-09-24 14:32:39 +0300 | [diff] [blame] | 21 | To enable all debug checks for an application: |
Victor Stinner | d71dcbb | 2014-08-25 17:04:12 +0200 | [diff] [blame] | 22 | |
Victor Stinner | 6a1b004 | 2015-02-04 16:14:33 +0100 | [diff] [blame] | 23 | * Enable the asyncio debug mode globally by setting the environment variable |
Guido van Rossum | f68afd8 | 2016-08-08 09:41:21 -0700 | [diff] [blame] | 24 | :envvar:`PYTHONASYNCIODEBUG` to ``1``, or by calling :meth:`AbstractEventLoop.set_debug`. |
Victor Stinner | 6a1b004 | 2015-02-04 16:14:33 +0100 | [diff] [blame] | 25 | * Set the log level of the :ref:`asyncio logger <asyncio-logger>` to |
| 26 | :py:data:`logging.DEBUG`. For example, call |
| 27 | ``logging.basicConfig(level=logging.DEBUG)`` at startup. |
| 28 | * Configure the :mod:`warnings` module to display :exc:`ResourceWarning` |
| 29 | warnings. For example, use the ``-Wdefault`` command line option of Python to |
| 30 | display them. |
| 31 | |
| 32 | Examples debug checks: |
Victor Stinner | 62511fd | 2014-06-23 00:36:11 +0200 | [diff] [blame] | 33 | |
| 34 | * Log :ref:`coroutines defined but never "yielded from" |
| 35 | <asyncio-coroutine-not-scheduled>` |
Guido van Rossum | f68afd8 | 2016-08-08 09:41:21 -0700 | [diff] [blame] | 36 | * :meth:`~AbstractEventLoop.call_soon` and :meth:`~AbstractEventLoop.call_at` methods |
Victor Stinner | 62511fd | 2014-06-23 00:36:11 +0200 | [diff] [blame] | 37 | raise an exception if they are called from the wrong thread. |
| 38 | * Log the execution time of the selector |
| 39 | * Log callbacks taking more than 100 ms to be executed. The |
Guido van Rossum | f68afd8 | 2016-08-08 09:41:21 -0700 | [diff] [blame] | 40 | :attr:`AbstractEventLoop.slow_callback_duration` attribute is the minimum |
Victor Stinner | 62511fd | 2014-06-23 00:36:11 +0200 | [diff] [blame] | 41 | duration in seconds of "slow" callbacks. |
Victor Stinner | 6a1b004 | 2015-02-04 16:14:33 +0100 | [diff] [blame] | 42 | * :exc:`ResourceWarning` warnings are emitted when transports and event loops |
| 43 | are :ref:`not closed explicitly <asyncio-close-transports>`. |
Victor Stinner | 62511fd | 2014-06-23 00:36:11 +0200 | [diff] [blame] | 44 | |
| 45 | .. seealso:: |
| 46 | |
Guido van Rossum | f68afd8 | 2016-08-08 09:41:21 -0700 | [diff] [blame] | 47 | The :meth:`AbstractEventLoop.set_debug` method and the :ref:`asyncio logger |
Victor Stinner | 62511fd | 2014-06-23 00:36:11 +0200 | [diff] [blame] | 48 | <asyncio-logger>`. |
| 49 | |
| 50 | |
Victor Stinner | 1077dee | 2015-01-30 00:55:58 +0100 | [diff] [blame] | 51 | Cancellation |
| 52 | ------------ |
| 53 | |
| 54 | Cancellation of tasks is not common in classic programming. In asynchronous |
| 55 | programming, not only it is something common, but you have to prepare your |
| 56 | code to handle it. |
| 57 | |
| 58 | Futures and tasks can be cancelled explicitly with their :meth:`Future.cancel` |
| 59 | method. The :func:`wait_for` function cancels the waited task when the timeout |
| 60 | occurs. There are many other cases where a task can be cancelled indirectly. |
| 61 | |
| 62 | Don't call :meth:`~Future.set_result` or :meth:`~Future.set_exception` method |
| 63 | of :class:`Future` if the future is cancelled: it would fail with an exception. |
| 64 | For example, write:: |
| 65 | |
| 66 | if not fut.cancelled(): |
| 67 | fut.set_result('done') |
| 68 | |
| 69 | Don't schedule directly a call to the :meth:`~Future.set_result` or the |
| 70 | :meth:`~Future.set_exception` method of a future with |
Guido van Rossum | f68afd8 | 2016-08-08 09:41:21 -0700 | [diff] [blame] | 71 | :meth:`AbstractEventLoop.call_soon`: the future can be cancelled before its method |
Victor Stinner | 1077dee | 2015-01-30 00:55:58 +0100 | [diff] [blame] | 72 | is called. |
| 73 | |
| 74 | If you wait for a future, you should check early if the future was cancelled to |
| 75 | avoid useless operations. Example:: |
| 76 | |
| 77 | @coroutine |
| 78 | def slow_operation(fut): |
| 79 | if fut.cancelled(): |
| 80 | return |
| 81 | # ... slow computation ... |
| 82 | yield from fut |
| 83 | # ... |
| 84 | |
| 85 | The :func:`shield` function can also be used to ignore cancellation. |
| 86 | |
| 87 | |
Victor Stinner | 606ab03 | 2014-02-01 03:18:58 +0100 | [diff] [blame] | 88 | .. _asyncio-multithreading: |
| 89 | |
| 90 | Concurrency and multithreading |
| 91 | ------------------------------ |
| 92 | |
| 93 | An event loop runs in a thread and executes all callbacks and tasks in the same |
Victor Stinner | 86516d9 | 2014-02-18 09:22:00 +0100 | [diff] [blame] | 94 | thread. While a task is running in the event loop, no other task is running in |
Victor Stinner | 5cb84ed | 2014-02-04 18:18:27 +0100 | [diff] [blame] | 95 | the same thread. But when the task uses ``yield from``, the task is suspended |
| 96 | and the event loop executes the next task. |
Victor Stinner | 606ab03 | 2014-02-01 03:18:58 +0100 | [diff] [blame] | 97 | |
Victor Stinner | 5cb84ed | 2014-02-04 18:18:27 +0100 | [diff] [blame] | 98 | To schedule a callback from a different thread, the |
Guido van Rossum | f68afd8 | 2016-08-08 09:41:21 -0700 | [diff] [blame] | 99 | :meth:`AbstractEventLoop.call_soon_threadsafe` method should be used. Example:: |
Victor Stinner | 5cb84ed | 2014-02-04 18:18:27 +0100 | [diff] [blame] | 100 | |
Guido van Rossum | 601953b | 2015-10-05 16:20:00 -0700 | [diff] [blame] | 101 | loop.call_soon_threadsafe(callback, *args) |
Victor Stinner | 606ab03 | 2014-02-01 03:18:58 +0100 | [diff] [blame] | 102 | |
Victor Stinner | 790202d | 2014-02-07 19:03:05 +0100 | [diff] [blame] | 103 | Most asyncio objects are not thread safe. You should only worry if you access |
| 104 | objects outside the event loop. For example, to cancel a future, don't call |
| 105 | directly its :meth:`Future.cancel` method, but:: |
| 106 | |
| 107 | loop.call_soon_threadsafe(fut.cancel) |
| 108 | |
Victor Stinner | 606ab03 | 2014-02-01 03:18:58 +0100 | [diff] [blame] | 109 | To handle signals and to execute subprocesses, the event loop must be run in |
| 110 | the main thread. |
| 111 | |
Guido van Rossum | 601953b | 2015-10-05 16:20:00 -0700 | [diff] [blame] | 112 | To schedule a coroutine object from a different thread, the |
| 113 | :func:`run_coroutine_threadsafe` function should be used. It returns a |
| 114 | :class:`concurrent.futures.Future` to access the result:: |
| 115 | |
| 116 | future = asyncio.run_coroutine_threadsafe(coro_func(), loop) |
| 117 | result = future.result(timeout) # Wait for the result with a timeout |
| 118 | |
Guido van Rossum | f68afd8 | 2016-08-08 09:41:21 -0700 | [diff] [blame] | 119 | The :meth:`AbstractEventLoop.run_in_executor` method can be used with a thread pool |
Victor Stinner | 606ab03 | 2014-02-01 03:18:58 +0100 | [diff] [blame] | 120 | executor to execute a callback in different thread to not block the thread of |
| 121 | the event loop. |
| 122 | |
| 123 | .. seealso:: |
| 124 | |
Zachary Ware | 5819cfa | 2015-01-06 00:40:43 -0600 | [diff] [blame] | 125 | The :ref:`Synchronization primitives <asyncio-sync>` section describes ways |
| 126 | to synchronize tasks. |
Victor Stinner | 606ab03 | 2014-02-01 03:18:58 +0100 | [diff] [blame] | 127 | |
Victor Stinner | 399c59d | 2015-01-09 01:32:02 +0100 | [diff] [blame] | 128 | The :ref:`Subprocess and threads <asyncio-subprocess-threads>` section lists |
| 129 | asyncio limitations to run subprocesses from different threads. |
| 130 | |
| 131 | |
| 132 | |
Victor Stinner | 606ab03 | 2014-02-01 03:18:58 +0100 | [diff] [blame] | 133 | |
Victor Stinner | 45b27ed | 2014-02-01 02:36:43 +0100 | [diff] [blame] | 134 | .. _asyncio-handle-blocking: |
| 135 | |
Eli Bendersky | b73c833 | 2014-02-09 06:07:47 -0800 | [diff] [blame] | 136 | Handle blocking functions correctly |
Victor Stinner | db39a0d | 2014-01-16 18:58:01 +0100 | [diff] [blame] | 137 | ----------------------------------- |
| 138 | |
| 139 | Blocking functions should not be called directly. For example, if a function |
| 140 | blocks for 1 second, other tasks are delayed by 1 second which can have an |
| 141 | important impact on reactivity. |
| 142 | |
| 143 | For networking and subprocesses, the :mod:`asyncio` module provides high-level |
Victor Stinner | 9592edb | 2014-02-02 15:03:02 +0100 | [diff] [blame] | 144 | APIs like :ref:`protocols <asyncio-protocol>`. |
Victor Stinner | db39a0d | 2014-01-16 18:58:01 +0100 | [diff] [blame] | 145 | |
| 146 | An executor can be used to run a task in a different thread or even in a |
| 147 | different process, to not block the thread of the event loop. See the |
Guido van Rossum | f68afd8 | 2016-08-08 09:41:21 -0700 | [diff] [blame] | 148 | :meth:`AbstractEventLoop.run_in_executor` method. |
Victor Stinner | db39a0d | 2014-01-16 18:58:01 +0100 | [diff] [blame] | 149 | |
Victor Stinner | 45b27ed | 2014-02-01 02:36:43 +0100 | [diff] [blame] | 150 | .. seealso:: |
| 151 | |
| 152 | The :ref:`Delayed calls <asyncio-delayed-calls>` section details how the |
| 153 | event loop handles time. |
| 154 | |
Victor Stinner | db39a0d | 2014-01-16 18:58:01 +0100 | [diff] [blame] | 155 | |
| 156 | .. _asyncio-logger: |
| 157 | |
Victor Stinner | 45b27ed | 2014-02-01 02:36:43 +0100 | [diff] [blame] | 158 | Logging |
| 159 | ------- |
Victor Stinner | db39a0d | 2014-01-16 18:58:01 +0100 | [diff] [blame] | 160 | |
Victor Stinner | 45b27ed | 2014-02-01 02:36:43 +0100 | [diff] [blame] | 161 | The :mod:`asyncio` module logs information with the :mod:`logging` module in |
| 162 | the logger ``'asyncio'``. |
Victor Stinner | db39a0d | 2014-01-16 18:58:01 +0100 | [diff] [blame] | 163 | |
Guido van Rossum | ba29a4f | 2016-10-13 13:56:40 -0700 | [diff] [blame] | 164 | The default log level for the :mod:`asyncio` module is :py:data:`logging.INFO`. |
| 165 | For those not wanting such verbosity from :mod:`asyncio` the log level can |
| 166 | be changed. For example, to change the level to :py:data:`logging.WARNING`: |
| 167 | |
| 168 | .. code-block:: none |
| 169 | |
| 170 | logging.getLogger('asyncio').setLevel(logging.WARNING) |
| 171 | |
Victor Stinner | db39a0d | 2014-01-16 18:58:01 +0100 | [diff] [blame] | 172 | |
| 173 | .. _asyncio-coroutine-not-scheduled: |
| 174 | |
| 175 | Detect coroutine objects never scheduled |
| 176 | ---------------------------------------- |
| 177 | |
Victor Stinner | 530ef2f | 2014-07-08 12:39:10 +0200 | [diff] [blame] | 178 | When a coroutine function is called and its result is not passed to |
Guido van Rossum | f68afd8 | 2016-08-08 09:41:21 -0700 | [diff] [blame] | 179 | :func:`ensure_future` or to the :meth:`AbstractEventLoop.create_task` method, |
Yury Selivanov | 04356e1 | 2015-06-30 22:13:22 -0400 | [diff] [blame] | 180 | the execution of the coroutine object will never be scheduled which is |
| 181 | probably a bug. :ref:`Enable the debug mode of asyncio <asyncio-debug-mode>` |
| 182 | to :ref:`log a warning <asyncio-logger>` to detect it. |
Victor Stinner | db39a0d | 2014-01-16 18:58:01 +0100 | [diff] [blame] | 183 | |
| 184 | Example with the bug:: |
| 185 | |
| 186 | import asyncio |
Victor Stinner | db39a0d | 2014-01-16 18:58:01 +0100 | [diff] [blame] | 187 | |
| 188 | @asyncio.coroutine |
| 189 | def test(): |
| 190 | print("never scheduled") |
| 191 | |
| 192 | test() |
| 193 | |
| 194 | Output in debug mode:: |
| 195 | |
Victor Stinner | 530ef2f | 2014-07-08 12:39:10 +0200 | [diff] [blame] | 196 | Coroutine test() at test.py:3 was never yielded from |
| 197 | Coroutine object created at (most recent call last): |
| 198 | File "test.py", line 7, in <module> |
| 199 | test() |
Victor Stinner | db39a0d | 2014-01-16 18:58:01 +0100 | [diff] [blame] | 200 | |
Yury Selivanov | 04356e1 | 2015-06-30 22:13:22 -0400 | [diff] [blame] | 201 | The fix is to call the :func:`ensure_future` function or the |
Guido van Rossum | f68afd8 | 2016-08-08 09:41:21 -0700 | [diff] [blame] | 202 | :meth:`AbstractEventLoop.create_task` method with the coroutine object. |
Victor Stinner | 530ef2f | 2014-07-08 12:39:10 +0200 | [diff] [blame] | 203 | |
| 204 | .. seealso:: |
| 205 | |
| 206 | :ref:`Pending task destroyed <asyncio-pending-task-destroyed>`. |
Victor Stinner | db39a0d | 2014-01-16 18:58:01 +0100 | [diff] [blame] | 207 | |
| 208 | |
Victor Stinner | 530ef2f | 2014-07-08 12:39:10 +0200 | [diff] [blame] | 209 | Detect exceptions never consumed |
| 210 | -------------------------------- |
Victor Stinner | db39a0d | 2014-01-16 18:58:01 +0100 | [diff] [blame] | 211 | |
| 212 | Python usually calls :func:`sys.displayhook` on unhandled exceptions. If |
Victor Stinner | 530ef2f | 2014-07-08 12:39:10 +0200 | [diff] [blame] | 213 | :meth:`Future.set_exception` is called, but the exception is never consumed, |
Zachary Ware | 5819cfa | 2015-01-06 00:40:43 -0600 | [diff] [blame] | 214 | :func:`sys.displayhook` is not called. Instead, :ref:`a log is emitted |
Victor Stinner | 530ef2f | 2014-07-08 12:39:10 +0200 | [diff] [blame] | 215 | <asyncio-logger>` when the future is deleted by the garbage collector, with the |
| 216 | traceback where the exception was raised. |
Victor Stinner | db39a0d | 2014-01-16 18:58:01 +0100 | [diff] [blame] | 217 | |
| 218 | Example of unhandled exception:: |
| 219 | |
| 220 | import asyncio |
| 221 | |
| 222 | @asyncio.coroutine |
| 223 | def bug(): |
| 224 | raise Exception("not consumed") |
| 225 | |
| 226 | loop = asyncio.get_event_loop() |
Yury Selivanov | d7e19bb | 2015-05-11 16:33:41 -0400 | [diff] [blame] | 227 | asyncio.ensure_future(bug()) |
Victor Stinner | db39a0d | 2014-01-16 18:58:01 +0100 | [diff] [blame] | 228 | loop.run_forever() |
Victor Stinner | b8064a8 | 2015-02-23 11:41:56 +0100 | [diff] [blame] | 229 | loop.close() |
Victor Stinner | db39a0d | 2014-01-16 18:58:01 +0100 | [diff] [blame] | 230 | |
| 231 | Output:: |
| 232 | |
Victor Stinner | 530ef2f | 2014-07-08 12:39:10 +0200 | [diff] [blame] | 233 | Task exception was never retrieved |
Victor Stinner | ab1c853 | 2014-10-12 21:37:16 +0200 | [diff] [blame] | 234 | future: <Task finished coro=<coro() done, defined at asyncio/coroutines.py:139> exception=Exception('not consumed',)> |
Victor Stinner | db39a0d | 2014-01-16 18:58:01 +0100 | [diff] [blame] | 235 | Traceback (most recent call last): |
Victor Stinner | ab1c853 | 2014-10-12 21:37:16 +0200 | [diff] [blame] | 236 | File "asyncio/tasks.py", line 237, in _step |
Victor Stinner | db39a0d | 2014-01-16 18:58:01 +0100 | [diff] [blame] | 237 | result = next(coro) |
Victor Stinner | 530ef2f | 2014-07-08 12:39:10 +0200 | [diff] [blame] | 238 | File "asyncio/coroutines.py", line 141, in coro |
Victor Stinner | db39a0d | 2014-01-16 18:58:01 +0100 | [diff] [blame] | 239 | res = func(*args, **kw) |
Victor Stinner | ab1c853 | 2014-10-12 21:37:16 +0200 | [diff] [blame] | 240 | File "test.py", line 5, in bug |
Victor Stinner | db39a0d | 2014-01-16 18:58:01 +0100 | [diff] [blame] | 241 | raise Exception("not consumed") |
| 242 | Exception: not consumed |
| 243 | |
Victor Stinner | 530ef2f | 2014-07-08 12:39:10 +0200 | [diff] [blame] | 244 | :ref:`Enable the debug mode of asyncio <asyncio-debug-mode>` to get the |
Victor Stinner | ab1c853 | 2014-10-12 21:37:16 +0200 | [diff] [blame] | 245 | traceback where the task was created. Output in debug mode:: |
| 246 | |
| 247 | Task exception was never retrieved |
| 248 | future: <Task finished coro=<bug() done, defined at test.py:3> exception=Exception('not consumed',) created at test.py:8> |
| 249 | source_traceback: Object created at (most recent call last): |
| 250 | File "test.py", line 8, in <module> |
Yury Selivanov | d7e19bb | 2015-05-11 16:33:41 -0400 | [diff] [blame] | 251 | asyncio.ensure_future(bug()) |
Victor Stinner | ab1c853 | 2014-10-12 21:37:16 +0200 | [diff] [blame] | 252 | Traceback (most recent call last): |
| 253 | File "asyncio/tasks.py", line 237, in _step |
| 254 | result = next(coro) |
| 255 | File "asyncio/coroutines.py", line 79, in __next__ |
| 256 | return next(self.gen) |
| 257 | File "asyncio/coroutines.py", line 141, in coro |
| 258 | res = func(*args, **kw) |
| 259 | File "test.py", line 5, in bug |
| 260 | raise Exception("not consumed") |
| 261 | Exception: not consumed |
Victor Stinner | 530ef2f | 2014-07-08 12:39:10 +0200 | [diff] [blame] | 262 | |
Zachary Ware | 5819cfa | 2015-01-06 00:40:43 -0600 | [diff] [blame] | 263 | There are different options to fix this issue. The first option is to chain the |
Victor Stinner | db39a0d | 2014-01-16 18:58:01 +0100 | [diff] [blame] | 264 | coroutine in another coroutine and use classic try/except:: |
| 265 | |
| 266 | @asyncio.coroutine |
| 267 | def handle_exception(): |
| 268 | try: |
| 269 | yield from bug() |
| 270 | except Exception: |
| 271 | print("exception consumed") |
| 272 | |
| 273 | loop = asyncio.get_event_loop() |
Yury Selivanov | d7e19bb | 2015-05-11 16:33:41 -0400 | [diff] [blame] | 274 | asyncio.ensure_future(handle_exception()) |
Victor Stinner | db39a0d | 2014-01-16 18:58:01 +0100 | [diff] [blame] | 275 | loop.run_forever() |
Victor Stinner | b8064a8 | 2015-02-23 11:41:56 +0100 | [diff] [blame] | 276 | loop.close() |
Victor Stinner | db39a0d | 2014-01-16 18:58:01 +0100 | [diff] [blame] | 277 | |
Guido van Rossum | f68afd8 | 2016-08-08 09:41:21 -0700 | [diff] [blame] | 278 | Another option is to use the :meth:`AbstractEventLoop.run_until_complete` |
Victor Stinner | db39a0d | 2014-01-16 18:58:01 +0100 | [diff] [blame] | 279 | function:: |
| 280 | |
Yury Selivanov | d7e19bb | 2015-05-11 16:33:41 -0400 | [diff] [blame] | 281 | task = asyncio.ensure_future(bug()) |
Victor Stinner | db39a0d | 2014-01-16 18:58:01 +0100 | [diff] [blame] | 282 | try: |
| 283 | loop.run_until_complete(task) |
| 284 | except Exception: |
| 285 | print("exception consumed") |
| 286 | |
Zachary Ware | 5819cfa | 2015-01-06 00:40:43 -0600 | [diff] [blame] | 287 | .. seealso:: |
| 288 | |
| 289 | The :meth:`Future.exception` method. |
Victor Stinner | db39a0d | 2014-01-16 18:58:01 +0100 | [diff] [blame] | 290 | |
| 291 | |
Zachary Ware | 5819cfa | 2015-01-06 00:40:43 -0600 | [diff] [blame] | 292 | Chain coroutines correctly |
Victor Stinner | db39a0d | 2014-01-16 18:58:01 +0100 | [diff] [blame] | 293 | -------------------------- |
| 294 | |
| 295 | When a coroutine function calls other coroutine functions and tasks, they |
Eli Bendersky | 679688e | 2014-01-20 08:13:31 -0800 | [diff] [blame] | 296 | should be chained explicitly with ``yield from``. Otherwise, the execution is |
| 297 | not guaranteed to be sequential. |
Victor Stinner | db39a0d | 2014-01-16 18:58:01 +0100 | [diff] [blame] | 298 | |
Eli Bendersky | 679688e | 2014-01-20 08:13:31 -0800 | [diff] [blame] | 299 | Example with different bugs using :func:`asyncio.sleep` to simulate slow |
| 300 | operations:: |
Victor Stinner | db39a0d | 2014-01-16 18:58:01 +0100 | [diff] [blame] | 301 | |
| 302 | import asyncio |
| 303 | |
| 304 | @asyncio.coroutine |
| 305 | def create(): |
| 306 | yield from asyncio.sleep(3.0) |
| 307 | print("(1) create file") |
| 308 | |
| 309 | @asyncio.coroutine |
| 310 | def write(): |
| 311 | yield from asyncio.sleep(1.0) |
| 312 | print("(2) write into file") |
| 313 | |
| 314 | @asyncio.coroutine |
| 315 | def close(): |
| 316 | print("(3) close file") |
| 317 | |
| 318 | @asyncio.coroutine |
| 319 | def test(): |
Yury Selivanov | d7e19bb | 2015-05-11 16:33:41 -0400 | [diff] [blame] | 320 | asyncio.ensure_future(create()) |
| 321 | asyncio.ensure_future(write()) |
| 322 | asyncio.ensure_future(close()) |
Victor Stinner | db39a0d | 2014-01-16 18:58:01 +0100 | [diff] [blame] | 323 | yield from asyncio.sleep(2.0) |
| 324 | loop.stop() |
| 325 | |
| 326 | loop = asyncio.get_event_loop() |
Yury Selivanov | d7e19bb | 2015-05-11 16:33:41 -0400 | [diff] [blame] | 327 | asyncio.ensure_future(test()) |
Victor Stinner | db39a0d | 2014-01-16 18:58:01 +0100 | [diff] [blame] | 328 | loop.run_forever() |
| 329 | print("Pending tasks at exit: %s" % asyncio.Task.all_tasks(loop)) |
Victor Stinner | f40c663 | 2014-01-28 23:32:40 +0100 | [diff] [blame] | 330 | loop.close() |
Victor Stinner | db39a0d | 2014-01-16 18:58:01 +0100 | [diff] [blame] | 331 | |
Martin Panter | 1050d2d | 2016-07-26 11:18:21 +0200 | [diff] [blame] | 332 | Expected output: |
| 333 | |
| 334 | .. code-block:: none |
Victor Stinner | db39a0d | 2014-01-16 18:58:01 +0100 | [diff] [blame] | 335 | |
| 336 | (1) create file |
| 337 | (2) write into file |
| 338 | (3) close file |
| 339 | Pending tasks at exit: set() |
| 340 | |
Martin Panter | 1050d2d | 2016-07-26 11:18:21 +0200 | [diff] [blame] | 341 | Actual output: |
| 342 | |
| 343 | .. code-block:: none |
Victor Stinner | db39a0d | 2014-01-16 18:58:01 +0100 | [diff] [blame] | 344 | |
| 345 | (3) close file |
| 346 | (2) write into file |
Victor Stinner | 530ef2f | 2014-07-08 12:39:10 +0200 | [diff] [blame] | 347 | Pending tasks at exit: {<Task pending create() at test.py:7 wait_for=<Future pending cb=[Task._wakeup()]>>} |
| 348 | Task was destroyed but it is pending! |
| 349 | task: <Task pending create() done at test.py:5 wait_for=<Future pending cb=[Task._wakeup()]>> |
Victor Stinner | db39a0d | 2014-01-16 18:58:01 +0100 | [diff] [blame] | 350 | |
| 351 | The loop stopped before the ``create()`` finished, ``close()`` has been called |
| 352 | before ``write()``, whereas coroutine functions were called in this order: |
| 353 | ``create()``, ``write()``, ``close()``. |
| 354 | |
| 355 | To fix the example, tasks must be marked with ``yield from``:: |
| 356 | |
| 357 | @asyncio.coroutine |
| 358 | def test(): |
Yury Selivanov | d7e19bb | 2015-05-11 16:33:41 -0400 | [diff] [blame] | 359 | yield from asyncio.ensure_future(create()) |
| 360 | yield from asyncio.ensure_future(write()) |
| 361 | yield from asyncio.ensure_future(close()) |
Victor Stinner | db39a0d | 2014-01-16 18:58:01 +0100 | [diff] [blame] | 362 | yield from asyncio.sleep(2.0) |
| 363 | loop.stop() |
| 364 | |
Yury Selivanov | d7e19bb | 2015-05-11 16:33:41 -0400 | [diff] [blame] | 365 | Or without ``asyncio.ensure_future()``:: |
Victor Stinner | db39a0d | 2014-01-16 18:58:01 +0100 | [diff] [blame] | 366 | |
| 367 | @asyncio.coroutine |
| 368 | def test(): |
| 369 | yield from create() |
| 370 | yield from write() |
| 371 | yield from close() |
| 372 | yield from asyncio.sleep(2.0) |
| 373 | loop.stop() |
| 374 | |
Victor Stinner | 530ef2f | 2014-07-08 12:39:10 +0200 | [diff] [blame] | 375 | |
| 376 | .. _asyncio-pending-task-destroyed: |
| 377 | |
| 378 | Pending task destroyed |
| 379 | ---------------------- |
| 380 | |
| 381 | If a pending task is destroyed, the execution of its wrapped :ref:`coroutine |
| 382 | <coroutine>` did not complete. It is probably a bug and so a warning is logged. |
| 383 | |
Martin Panter | 1050d2d | 2016-07-26 11:18:21 +0200 | [diff] [blame] | 384 | Example of log: |
| 385 | |
| 386 | .. code-block:: none |
Victor Stinner | 530ef2f | 2014-07-08 12:39:10 +0200 | [diff] [blame] | 387 | |
| 388 | Task was destroyed but it is pending! |
Victor Stinner | ab1c853 | 2014-10-12 21:37:16 +0200 | [diff] [blame] | 389 | task: <Task pending coro=<kill_me() done, defined at test.py:5> wait_for=<Future pending cb=[Task._wakeup()]>> |
Victor Stinner | 530ef2f | 2014-07-08 12:39:10 +0200 | [diff] [blame] | 390 | |
| 391 | :ref:`Enable the debug mode of asyncio <asyncio-debug-mode>` to get the |
Martin Panter | 1050d2d | 2016-07-26 11:18:21 +0200 | [diff] [blame] | 392 | traceback where the task was created. Example of log in debug mode: |
| 393 | |
| 394 | .. code-block:: none |
Victor Stinner | ab1c853 | 2014-10-12 21:37:16 +0200 | [diff] [blame] | 395 | |
| 396 | Task was destroyed but it is pending! |
| 397 | source_traceback: Object created at (most recent call last): |
| 398 | File "test.py", line 15, in <module> |
Yury Selivanov | d7e19bb | 2015-05-11 16:33:41 -0400 | [diff] [blame] | 399 | task = asyncio.ensure_future(coro, loop=loop) |
Victor Stinner | ab1c853 | 2014-10-12 21:37:16 +0200 | [diff] [blame] | 400 | task: <Task pending coro=<kill_me() done, defined at test.py:5> wait_for=<Future pending cb=[Task._wakeup()] created at test.py:7> created at test.py:15> |
| 401 | |
Victor Stinner | 530ef2f | 2014-07-08 12:39:10 +0200 | [diff] [blame] | 402 | |
| 403 | .. seealso:: |
| 404 | |
| 405 | :ref:`Detect coroutine objects never scheduled <asyncio-coroutine-not-scheduled>`. |
| 406 | |
Victor Stinner | 6a1b004 | 2015-02-04 16:14:33 +0100 | [diff] [blame] | 407 | .. _asyncio-close-transports: |
Victor Stinner | 188f2c0 | 2015-01-30 01:35:14 +0100 | [diff] [blame] | 408 | |
Victor Stinner | 6a1b004 | 2015-02-04 16:14:33 +0100 | [diff] [blame] | 409 | Close transports and event loops |
| 410 | -------------------------------- |
Victor Stinner | 188f2c0 | 2015-01-30 01:35:14 +0100 | [diff] [blame] | 411 | |
| 412 | When a transport is no more needed, call its ``close()`` method to release |
Victor Stinner | 6a1b004 | 2015-02-04 16:14:33 +0100 | [diff] [blame] | 413 | resources. Event loops must also be closed explicitly. |
Victor Stinner | 188f2c0 | 2015-01-30 01:35:14 +0100 | [diff] [blame] | 414 | |
Victor Stinner | 6a1b004 | 2015-02-04 16:14:33 +0100 | [diff] [blame] | 415 | If a transport or an event loop is not closed explicitly, a |
| 416 | :exc:`ResourceWarning` warning will be emitted in its destructor. By default, |
| 417 | :exc:`ResourceWarning` warnings are ignored. The :ref:`Debug mode of asyncio |
| 418 | <asyncio-debug-mode>` section explains how to display them. |