R David Murray | 6a14381 | 2013-12-20 14:37:39 -0500 | [diff] [blame] | 1 | .. currentmodule:: asyncio |
Victor Stinner | ea3183f | 2013-12-03 01:08:00 +0100 | [diff] [blame] | 2 | |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 3 | |
| 4 | ==================== |
| 5 | Coroutines and Tasks |
Victor Stinner | ea3183f | 2013-12-03 01:08:00 +0100 | [diff] [blame] | 6 | ==================== |
| 7 | |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 8 | This section outlines high-level asyncio APIs to work with coroutines |
| 9 | and Tasks. |
lf | 627d2c8 | 2017-07-25 17:03:51 -0600 | [diff] [blame] | 10 | |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 11 | .. contents:: |
| 12 | :depth: 1 |
| 13 | :local: |
| 14 | |
lf | 627d2c8 | 2017-07-25 17:03:51 -0600 | [diff] [blame] | 15 | |
Victor Stinner | ea3183f | 2013-12-03 01:08:00 +0100 | [diff] [blame] | 16 | .. _coroutine: |
| 17 | |
| 18 | Coroutines |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 19 | ========== |
Victor Stinner | ea3183f | 2013-12-03 01:08:00 +0100 | [diff] [blame] | 20 | |
Kyle Stanley | f900064 | 2019-10-10 19:18:46 -0400 | [diff] [blame] | 21 | :term:`Coroutines <coroutine>` declared with the async/await syntax is the |
| 22 | preferred way of writing asyncio applications. For example, the following |
| 23 | snippet of code (requires Python 3.7+) prints "hello", waits 1 second, |
Yury Selivanov | b042cf1 | 2018-09-18 02:47:54 -0400 | [diff] [blame] | 24 | and then prints "world":: |
Victor Stinner | ea3183f | 2013-12-03 01:08:00 +0100 | [diff] [blame] | 25 | |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 26 | >>> import asyncio |
Victor Stinner | ea3183f | 2013-12-03 01:08:00 +0100 | [diff] [blame] | 27 | |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 28 | >>> async def main(): |
| 29 | ... print('hello') |
| 30 | ... await asyncio.sleep(1) |
| 31 | ... print('world') |
Victor Stinner | ea3183f | 2013-12-03 01:08:00 +0100 | [diff] [blame] | 32 | |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 33 | >>> asyncio.run(main()) |
| 34 | hello |
| 35 | world |
Victor Stinner | ea3183f | 2013-12-03 01:08:00 +0100 | [diff] [blame] | 36 | |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 37 | Note that simply calling a coroutine will not schedule it to |
| 38 | be executed:: |
Victor Stinner | ea3183f | 2013-12-03 01:08:00 +0100 | [diff] [blame] | 39 | |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 40 | >>> main() |
| 41 | <coroutine object main at 0x1053bb7c8> |
Victor Stinner | ea3183f | 2013-12-03 01:08:00 +0100 | [diff] [blame] | 42 | |
Boštjan Mejak | 1d5bdef | 2019-05-19 11:01:36 +0200 | [diff] [blame] | 43 | To actually run a coroutine, asyncio provides three main mechanisms: |
Victor Stinner | ea3183f | 2013-12-03 01:08:00 +0100 | [diff] [blame] | 44 | |
Elvis Pranskevichus | 1fa2ec4 | 2018-09-17 19:16:44 -0400 | [diff] [blame] | 45 | * The :func:`asyncio.run` function to run the top-level |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 46 | entry point "main()" function (see the above example.) |
Victor Stinner | ea3183f | 2013-12-03 01:08:00 +0100 | [diff] [blame] | 47 | |
Elvis Pranskevichus | 1fa2ec4 | 2018-09-17 19:16:44 -0400 | [diff] [blame] | 48 | * Awaiting on a coroutine. The following snippet of code will |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 49 | print "hello" after waiting for 1 second, and then print "world" |
| 50 | after waiting for *another* 2 seconds:: |
Victor Stinner | ea3183f | 2013-12-03 01:08:00 +0100 | [diff] [blame] | 51 | |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 52 | import asyncio |
| 53 | import time |
Victor Stinner | ea3183f | 2013-12-03 01:08:00 +0100 | [diff] [blame] | 54 | |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 55 | async def say_after(delay, what): |
| 56 | await asyncio.sleep(delay) |
| 57 | print(what) |
| 58 | |
| 59 | async def main(): |
Mariatta | 9f43fbb | 2018-10-24 15:37:12 -0700 | [diff] [blame] | 60 | print(f"started at {time.strftime('%X')}") |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 61 | |
| 62 | await say_after(1, 'hello') |
| 63 | await say_after(2, 'world') |
| 64 | |
Mariatta | 9f43fbb | 2018-10-24 15:37:12 -0700 | [diff] [blame] | 65 | print(f"finished at {time.strftime('%X')}") |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 66 | |
| 67 | asyncio.run(main()) |
| 68 | |
| 69 | Expected output:: |
| 70 | |
| 71 | started at 17:13:52 |
| 72 | hello |
| 73 | world |
| 74 | finished at 17:13:55 |
| 75 | |
Elvis Pranskevichus | 1fa2ec4 | 2018-09-17 19:16:44 -0400 | [diff] [blame] | 76 | * The :func:`asyncio.create_task` function to run coroutines |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 77 | concurrently as asyncio :class:`Tasks <Task>`. |
| 78 | |
Danny Hermes | 7bfbda4 | 2018-09-17 21:49:21 -0700 | [diff] [blame] | 79 | Let's modify the above example and run two ``say_after`` coroutines |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 80 | *concurrently*:: |
| 81 | |
| 82 | async def main(): |
| 83 | task1 = asyncio.create_task( |
| 84 | say_after(1, 'hello')) |
| 85 | |
| 86 | task2 = asyncio.create_task( |
| 87 | say_after(2, 'world')) |
| 88 | |
Mariatta | 9f43fbb | 2018-10-24 15:37:12 -0700 | [diff] [blame] | 89 | print(f"started at {time.strftime('%X')}") |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 90 | |
| 91 | # Wait until both tasks are completed (should take |
| 92 | # around 2 seconds.) |
| 93 | await task1 |
| 94 | await task2 |
| 95 | |
Mariatta | 9f43fbb | 2018-10-24 15:37:12 -0700 | [diff] [blame] | 96 | print(f"finished at {time.strftime('%X')}") |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 97 | |
| 98 | Note that expected output now shows that the snippet runs |
| 99 | 1 second faster than before:: |
| 100 | |
| 101 | started at 17:14:32 |
| 102 | hello |
| 103 | world |
| 104 | finished at 17:14:34 |
| 105 | |
Yury Selivanov | 4715039 | 2018-09-18 17:55:44 -0400 | [diff] [blame] | 106 | |
| 107 | .. _asyncio-awaitables: |
| 108 | |
| 109 | Awaitables |
| 110 | ========== |
| 111 | |
Yury Selivanov | e247b46 | 2018-09-20 12:43:59 -0400 | [diff] [blame] | 112 | We say that an object is an **awaitable** object if it can be used |
| 113 | in an :keyword:`await` expression. Many asyncio APIs are designed to |
| 114 | accept awaitables. |
| 115 | |
| 116 | There are three main types of *awaitable* objects: |
| 117 | **coroutines**, **Tasks**, and **Futures**. |
Yury Selivanov | 4715039 | 2018-09-18 17:55:44 -0400 | [diff] [blame] | 118 | |
| 119 | |
Yury Selivanov | e247b46 | 2018-09-20 12:43:59 -0400 | [diff] [blame] | 120 | .. rubric:: Coroutines |
Yury Selivanov | 4715039 | 2018-09-18 17:55:44 -0400 | [diff] [blame] | 121 | |
Yury Selivanov | e247b46 | 2018-09-20 12:43:59 -0400 | [diff] [blame] | 122 | Python coroutines are *awaitables* and therefore can be awaited from |
| 123 | other coroutines:: |
| 124 | |
| 125 | import asyncio |
Yury Selivanov | 4715039 | 2018-09-18 17:55:44 -0400 | [diff] [blame] | 126 | |
| 127 | async def nested(): |
| 128 | return 42 |
| 129 | |
| 130 | async def main(): |
Yury Selivanov | e247b46 | 2018-09-20 12:43:59 -0400 | [diff] [blame] | 131 | # Nothing happens if we just call "nested()". |
Yury Selivanov | db1a80e | 2018-09-21 16:23:15 -0400 | [diff] [blame] | 132 | # A coroutine object is created but not awaited, |
| 133 | # so it *won't run at all*. |
Yury Selivanov | e247b46 | 2018-09-20 12:43:59 -0400 | [diff] [blame] | 134 | nested() |
| 135 | |
| 136 | # Let's do it differently now and await it: |
| 137 | print(await nested()) # will print "42". |
| 138 | |
| 139 | asyncio.run(main()) |
| 140 | |
| 141 | .. important:: |
| 142 | |
| 143 | In this documentation the term "coroutine" can be used for |
| 144 | two closely related concepts: |
| 145 | |
| 146 | * a *coroutine function*: an :keyword:`async def` function; |
| 147 | |
| 148 | * a *coroutine object*: an object returned by calling a |
| 149 | *coroutine function*. |
| 150 | |
| 151 | asyncio also supports legacy :ref:`generator-based |
| 152 | <asyncio_generator_based_coro>` coroutines. |
| 153 | |
| 154 | |
| 155 | .. rubric:: Tasks |
Yury Selivanov | 4715039 | 2018-09-18 17:55:44 -0400 | [diff] [blame] | 156 | |
| 157 | *Tasks* are used to schedule coroutines *concurrently*. |
Yury Selivanov | 4715039 | 2018-09-18 17:55:44 -0400 | [diff] [blame] | 158 | |
Yury Selivanov | e247b46 | 2018-09-20 12:43:59 -0400 | [diff] [blame] | 159 | When a coroutine is wrapped into a *Task* with functions like |
| 160 | :func:`asyncio.create_task` the coroutine is automatically |
| 161 | scheduled to run soon:: |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 162 | |
Yury Selivanov | e247b46 | 2018-09-20 12:43:59 -0400 | [diff] [blame] | 163 | import asyncio |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 164 | |
Yury Selivanov | e247b46 | 2018-09-20 12:43:59 -0400 | [diff] [blame] | 165 | async def nested(): |
| 166 | return 42 |
| 167 | |
| 168 | async def main(): |
| 169 | # Schedule nested() to run soon concurrently |
| 170 | # with "main()". |
| 171 | task = asyncio.create_task(nested()) |
| 172 | |
| 173 | # "task" can now be used to cancel "nested()", or |
| 174 | # can simply be awaited to wait until it is complete: |
| 175 | await task |
| 176 | |
| 177 | asyncio.run(main()) |
Victor Stinner | 337e03f | 2014-08-11 01:11:13 +0200 | [diff] [blame] | 178 | |
Victor Stinner | ea3183f | 2013-12-03 01:08:00 +0100 | [diff] [blame] | 179 | |
Yury Selivanov | 4715039 | 2018-09-18 17:55:44 -0400 | [diff] [blame] | 180 | .. rubric:: Futures |
| 181 | |
Yury Selivanov | e247b46 | 2018-09-20 12:43:59 -0400 | [diff] [blame] | 182 | A :class:`Future` is a special **low-level** awaitable object that |
| 183 | represents an **eventual result** of an asynchronous operation. |
Yury Selivanov | 4715039 | 2018-09-18 17:55:44 -0400 | [diff] [blame] | 184 | |
Yury Selivanov | e247b46 | 2018-09-20 12:43:59 -0400 | [diff] [blame] | 185 | When a Future object is *awaited* it means that the coroutine will |
| 186 | wait until the Future is resolved in some other place. |
| 187 | |
Yury Selivanov | 4715039 | 2018-09-18 17:55:44 -0400 | [diff] [blame] | 188 | Future objects in asyncio are needed to allow callback-based code |
| 189 | to be used with async/await. |
| 190 | |
Yury Selivanov | e247b46 | 2018-09-20 12:43:59 -0400 | [diff] [blame] | 191 | Normally **there is no need** to create Future objects at the |
Yury Selivanov | 4715039 | 2018-09-18 17:55:44 -0400 | [diff] [blame] | 192 | application level code. |
| 193 | |
| 194 | Future objects, sometimes exposed by libraries and some asyncio |
Yury Selivanov | e247b46 | 2018-09-20 12:43:59 -0400 | [diff] [blame] | 195 | APIs, can be awaited:: |
Yury Selivanov | 4715039 | 2018-09-18 17:55:44 -0400 | [diff] [blame] | 196 | |
| 197 | async def main(): |
| 198 | await function_that_returns_a_future_object() |
| 199 | |
| 200 | # this is also valid: |
| 201 | await asyncio.gather( |
| 202 | function_that_returns_a_future_object(), |
| 203 | some_python_coroutine() |
| 204 | ) |
| 205 | |
Yury Selivanov | e247b46 | 2018-09-20 12:43:59 -0400 | [diff] [blame] | 206 | A good example of a low-level function that returns a Future object |
| 207 | is :meth:`loop.run_in_executor`. |
| 208 | |
Yury Selivanov | 4715039 | 2018-09-18 17:55:44 -0400 | [diff] [blame] | 209 | |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 210 | Running an asyncio Program |
| 211 | ========================== |
Victor Stinner | ea3183f | 2013-12-03 01:08:00 +0100 | [diff] [blame] | 212 | |
Elvis Pranskevichus | 63536bd | 2018-05-19 23:15:06 -0400 | [diff] [blame] | 213 | .. function:: run(coro, \*, debug=False) |
Yury Selivanov | 02a0a19 | 2017-12-14 09:42:21 -0500 | [diff] [blame] | 214 | |
Kyle Stanley | e407013 | 2019-09-30 20:12:21 -0400 | [diff] [blame] | 215 | Execute the :term:`coroutine` *coro* and return the result. |
| 216 | |
Yury Selivanov | 02a0a19 | 2017-12-14 09:42:21 -0500 | [diff] [blame] | 217 | This function runs the passed coroutine, taking care of |
Kyle Stanley | 9fdc64c | 2019-09-19 08:47:22 -0400 | [diff] [blame] | 218 | managing the asyncio event loop, *finalizing asynchronous |
| 219 | generators*, and closing the threadpool. |
Yury Selivanov | 02a0a19 | 2017-12-14 09:42:21 -0500 | [diff] [blame] | 220 | |
| 221 | This function cannot be called when another asyncio event loop is |
| 222 | running in the same thread. |
| 223 | |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 224 | If *debug* is ``True``, the event loop will be run in debug mode. |
Yury Selivanov | 02a0a19 | 2017-12-14 09:42:21 -0500 | [diff] [blame] | 225 | |
| 226 | This function always creates a new event loop and closes it at |
| 227 | the end. It should be used as a main entry point for asyncio |
| 228 | programs, and should ideally only be called once. |
| 229 | |
Emmanuel Arias | 17deb16 | 2019-09-25 05:53:49 -0300 | [diff] [blame] | 230 | Example:: |
| 231 | |
| 232 | async def main(): |
| 233 | await asyncio.sleep(1) |
| 234 | print('hello') |
| 235 | |
| 236 | asyncio.run(main()) |
| 237 | |
Yury Selivanov | 02a0a19 | 2017-12-14 09:42:21 -0500 | [diff] [blame] | 238 | .. versionadded:: 3.7 |
| 239 | |
Kyle Stanley | 9fdc64c | 2019-09-19 08:47:22 -0400 | [diff] [blame] | 240 | .. versionchanged:: 3.9 |
| 241 | Updated to use :meth:`loop.shutdown_default_executor`. |
Yury Selivanov | 02a0a19 | 2017-12-14 09:42:21 -0500 | [diff] [blame] | 242 | |
Kyle Stanley | f900064 | 2019-10-10 19:18:46 -0400 | [diff] [blame] | 243 | .. note:: |
| 244 | The source code for ``asyncio.run()`` can be found in |
| 245 | :source:`Lib/asyncio/runners.py`. |
| 246 | |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 247 | Creating Tasks |
| 248 | ============== |
Victor Stinner | ea3183f | 2013-12-03 01:08:00 +0100 | [diff] [blame] | 249 | |
Alex Grönholm | cca4eec | 2018-08-09 00:06:47 +0300 | [diff] [blame] | 250 | .. function:: create_task(coro, \*, name=None) |
Andrew Svetlov | f74ef45 | 2017-12-15 07:04:38 +0200 | [diff] [blame] | 251 | |
Yury Selivanov | e247b46 | 2018-09-20 12:43:59 -0400 | [diff] [blame] | 252 | Wrap the *coro* :ref:`coroutine <coroutine>` into a :class:`Task` |
| 253 | and schedule its execution. Return the Task object. |
Alex Grönholm | cca4eec | 2018-08-09 00:06:47 +0300 | [diff] [blame] | 254 | |
| 255 | If *name* is not ``None``, it is set as the name of the task using |
| 256 | :meth:`Task.set_name`. |
Andrew Svetlov | f74ef45 | 2017-12-15 07:04:38 +0200 | [diff] [blame] | 257 | |
Elvis Pranskevichus | 1fa2ec4 | 2018-09-17 19:16:44 -0400 | [diff] [blame] | 258 | The task is executed in the loop returned by :func:`get_running_loop`, |
Andrew Svetlov | f74ef45 | 2017-12-15 07:04:38 +0200 | [diff] [blame] | 259 | :exc:`RuntimeError` is raised if there is no running loop in |
| 260 | current thread. |
| 261 | |
Yury Selivanov | 4715039 | 2018-09-18 17:55:44 -0400 | [diff] [blame] | 262 | This function has been **added in Python 3.7**. Prior to |
| 263 | Python 3.7, the low-level :func:`asyncio.ensure_future` function |
| 264 | can be used instead:: |
| 265 | |
| 266 | async def coro(): |
| 267 | ... |
| 268 | |
| 269 | # In Python 3.7+ |
| 270 | task = asyncio.create_task(coro()) |
| 271 | ... |
| 272 | |
| 273 | # This works in all Python versions but is less readable |
| 274 | task = asyncio.ensure_future(coro()) |
| 275 | ... |
| 276 | |
Andrew Svetlov | f74ef45 | 2017-12-15 07:04:38 +0200 | [diff] [blame] | 277 | .. versionadded:: 3.7 |
| 278 | |
Alex Grönholm | cca4eec | 2018-08-09 00:06:47 +0300 | [diff] [blame] | 279 | .. versionchanged:: 3.8 |
| 280 | Added the ``name`` parameter. |
| 281 | |
Victor Stinner | ea3183f | 2013-12-03 01:08:00 +0100 | [diff] [blame] | 282 | |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 283 | Sleeping |
| 284 | ======== |
Andrew Svetlov | f124016 | 2016-01-11 14:40:35 +0200 | [diff] [blame] | 285 | |
Victor Stinner | bdd574d | 2015-02-12 22:49:18 +0100 | [diff] [blame] | 286 | .. coroutinefunction:: sleep(delay, result=None, \*, loop=None) |
Victor Stinner | ea3183f | 2013-12-03 01:08:00 +0100 | [diff] [blame] | 287 | |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 288 | Block for *delay* seconds. |
| 289 | |
| 290 | If *result* is provided, it is returned to the caller |
Eli Bendersky | 2d26af8 | 2014-01-20 06:59:23 -0800 | [diff] [blame] | 291 | when the coroutine completes. |
Victor Stinner | ea3183f | 2013-12-03 01:08:00 +0100 | [diff] [blame] | 292 | |
Hrvoje Nikšić | cd602b8 | 2018-10-01 12:09:38 +0200 | [diff] [blame] | 293 | ``sleep()`` always suspends the current task, allowing other tasks |
| 294 | to run. |
| 295 | |
Matthias Bussonnier | d0ebf13 | 2019-05-20 23:20:10 -0700 | [diff] [blame] | 296 | .. deprecated-removed:: 3.8 3.10 |
| 297 | The *loop* parameter. |
Yury Selivanov | 4715039 | 2018-09-18 17:55:44 -0400 | [diff] [blame] | 298 | |
Yury Selivanov | 7372c3b | 2018-09-14 15:11:24 -0700 | [diff] [blame] | 299 | .. _asyncio_example_sleep: |
Victor Stinner | 45b27ed | 2014-02-01 02:36:43 +0100 | [diff] [blame] | 300 | |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 301 | Example of coroutine displaying the current date every second |
Elvis Pranskevichus | 1fa2ec4 | 2018-09-17 19:16:44 -0400 | [diff] [blame] | 302 | for 5 seconds:: |
Victor Stinner | bdd574d | 2015-02-12 22:49:18 +0100 | [diff] [blame] | 303 | |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 304 | import asyncio |
| 305 | import datetime |
Victor Stinner | ea3183f | 2013-12-03 01:08:00 +0100 | [diff] [blame] | 306 | |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 307 | async def display_date(): |
| 308 | loop = asyncio.get_running_loop() |
| 309 | end_time = loop.time() + 5.0 |
| 310 | while True: |
| 311 | print(datetime.datetime.now()) |
| 312 | if (loop.time() + 1.0) >= end_time: |
| 313 | break |
| 314 | await asyncio.sleep(1) |
| 315 | |
| 316 | asyncio.run(display_date()) |
| 317 | |
| 318 | |
| 319 | Running Tasks Concurrently |
| 320 | ========================== |
| 321 | |
Yury Selivanov | e247b46 | 2018-09-20 12:43:59 -0400 | [diff] [blame] | 322 | .. awaitablefunction:: gather(\*aws, loop=None, return_exceptions=False) |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 323 | |
Yury Selivanov | e247b46 | 2018-09-20 12:43:59 -0400 | [diff] [blame] | 324 | Run :ref:`awaitable objects <asyncio-awaitables>` in the *aws* |
Yury Selivanov | 4715039 | 2018-09-18 17:55:44 -0400 | [diff] [blame] | 325 | sequence *concurrently*. |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 326 | |
Yury Selivanov | e247b46 | 2018-09-20 12:43:59 -0400 | [diff] [blame] | 327 | If any awaitable in *aws* is a coroutine, it is automatically |
Yury Selivanov | 4715039 | 2018-09-18 17:55:44 -0400 | [diff] [blame] | 328 | scheduled as a Task. |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 329 | |
Yury Selivanov | 4715039 | 2018-09-18 17:55:44 -0400 | [diff] [blame] | 330 | If all awaitables are completed successfully, the result is an |
| 331 | aggregate list of returned values. The order of result values |
Yury Selivanov | e247b46 | 2018-09-20 12:43:59 -0400 | [diff] [blame] | 332 | corresponds to the order of awaitables in *aws*. |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 333 | |
Yury Selivanov | db1a80e | 2018-09-21 16:23:15 -0400 | [diff] [blame] | 334 | If *return_exceptions* is ``False`` (default), the first |
| 335 | raised exception is immediately propagated to the task that |
| 336 | awaits on ``gather()``. Other awaitables in the *aws* sequence |
| 337 | **won't be cancelled** and will continue to run. |
| 338 | |
Yury Selivanov | 4715039 | 2018-09-18 17:55:44 -0400 | [diff] [blame] | 339 | If *return_exceptions* is ``True``, exceptions are treated the |
| 340 | same as successful results, and aggregated in the result list. |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 341 | |
Yury Selivanov | db1a80e | 2018-09-21 16:23:15 -0400 | [diff] [blame] | 342 | If ``gather()`` is *cancelled*, all submitted awaitables |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 343 | (that have not completed yet) are also *cancelled*. |
| 344 | |
Yury Selivanov | e247b46 | 2018-09-20 12:43:59 -0400 | [diff] [blame] | 345 | If any Task or Future from the *aws* sequence is *cancelled*, it is |
Yury Selivanov | 4715039 | 2018-09-18 17:55:44 -0400 | [diff] [blame] | 346 | treated as if it raised :exc:`CancelledError` -- the ``gather()`` |
| 347 | call is **not** cancelled in this case. This is to prevent the |
| 348 | cancellation of one submitted Task/Future to cause other |
| 349 | Tasks/Futures to be cancelled. |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 350 | |
Andrew Svetlov | a488879 | 2019-09-12 15:40:40 +0300 | [diff] [blame] | 351 | .. deprecated-removed:: 3.8 3.10 |
| 352 | The *loop* parameter. |
| 353 | |
Yury Selivanov | 7372c3b | 2018-09-14 15:11:24 -0700 | [diff] [blame] | 354 | .. _asyncio_example_gather: |
| 355 | |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 356 | Example:: |
| 357 | |
| 358 | import asyncio |
| 359 | |
| 360 | async def factorial(name, number): |
| 361 | f = 1 |
| 362 | for i in range(2, number + 1): |
| 363 | print(f"Task {name}: Compute factorial({i})...") |
| 364 | await asyncio.sleep(1) |
| 365 | f *= i |
| 366 | print(f"Task {name}: factorial({number}) = {f}") |
| 367 | |
| 368 | async def main(): |
Yury Selivanov | 4715039 | 2018-09-18 17:55:44 -0400 | [diff] [blame] | 369 | # Schedule three calls *concurrently*: |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 370 | await asyncio.gather( |
| 371 | factorial("A", 2), |
| 372 | factorial("B", 3), |
| 373 | factorial("C", 4), |
Miguel Ángel García | 9c53fa6 | 2018-09-18 08:01:26 +0200 | [diff] [blame] | 374 | ) |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 375 | |
| 376 | asyncio.run(main()) |
| 377 | |
| 378 | # Expected output: |
| 379 | # |
| 380 | # Task A: Compute factorial(2)... |
| 381 | # Task B: Compute factorial(2)... |
| 382 | # Task C: Compute factorial(2)... |
| 383 | # Task A: factorial(2) = 2 |
| 384 | # Task B: Compute factorial(3)... |
| 385 | # Task C: Compute factorial(3)... |
| 386 | # Task B: factorial(3) = 6 |
| 387 | # Task C: Compute factorial(4)... |
| 388 | # Task C: factorial(4) = 24 |
| 389 | |
Yury Selivanov | 4715039 | 2018-09-18 17:55:44 -0400 | [diff] [blame] | 390 | .. versionchanged:: 3.7 |
| 391 | If the *gather* itself is cancelled, the cancellation is |
| 392 | propagated regardless of *return_exceptions*. |
| 393 | |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 394 | |
Yury Selivanov | db1a80e | 2018-09-21 16:23:15 -0400 | [diff] [blame] | 395 | Shielding From Cancellation |
| 396 | =========================== |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 397 | |
Yury Selivanov | e247b46 | 2018-09-20 12:43:59 -0400 | [diff] [blame] | 398 | .. awaitablefunction:: shield(aw, \*, loop=None) |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 399 | |
Yury Selivanov | 4715039 | 2018-09-18 17:55:44 -0400 | [diff] [blame] | 400 | Protect an :ref:`awaitable object <asyncio-awaitables>` |
| 401 | from being :meth:`cancelled <Task.cancel>`. |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 402 | |
Yury Selivanov | db1a80e | 2018-09-21 16:23:15 -0400 | [diff] [blame] | 403 | If *aw* is a coroutine it is automatically scheduled as a Task. |
Victor Stinner | ea3183f | 2013-12-03 01:08:00 +0100 | [diff] [blame] | 404 | |
| 405 | The statement:: |
| 406 | |
Andrew Svetlov | 8874342 | 2017-12-11 17:35:49 +0200 | [diff] [blame] | 407 | res = await shield(something()) |
Victor Stinner | ea3183f | 2013-12-03 01:08:00 +0100 | [diff] [blame] | 408 | |
Elvis Pranskevichus | 1fa2ec4 | 2018-09-17 19:16:44 -0400 | [diff] [blame] | 409 | is equivalent to:: |
Victor Stinner | ea3183f | 2013-12-03 01:08:00 +0100 | [diff] [blame] | 410 | |
Andrew Svetlov | 8874342 | 2017-12-11 17:35:49 +0200 | [diff] [blame] | 411 | res = await something() |
Victor Stinner | ea3183f | 2013-12-03 01:08:00 +0100 | [diff] [blame] | 412 | |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 413 | *except* that if the coroutine containing it is cancelled, the |
| 414 | Task running in ``something()`` is not cancelled. From the point |
| 415 | of view of ``something()``, the cancellation did not happen. |
| 416 | Although its caller is still cancelled, so the "await" expression |
Elvis Pranskevichus | 1fa2ec4 | 2018-09-17 19:16:44 -0400 | [diff] [blame] | 417 | still raises a :exc:`CancelledError`. |
Victor Stinner | ea3183f | 2013-12-03 01:08:00 +0100 | [diff] [blame] | 418 | |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 419 | If ``something()`` is cancelled by other means (i.e. from within |
| 420 | itself) that would also cancel ``shield()``. |
| 421 | |
| 422 | If it is desired to completely ignore cancellation (not recommended) |
| 423 | the ``shield()`` function should be combined with a try/except |
| 424 | clause, as follows:: |
Victor Stinner | ea3183f | 2013-12-03 01:08:00 +0100 | [diff] [blame] | 425 | |
| 426 | try: |
Andrew Svetlov | 8874342 | 2017-12-11 17:35:49 +0200 | [diff] [blame] | 427 | res = await shield(something()) |
Victor Stinner | ea3183f | 2013-12-03 01:08:00 +0100 | [diff] [blame] | 428 | except CancelledError: |
| 429 | res = None |
| 430 | |
Andrew Svetlov | a488879 | 2019-09-12 15:40:40 +0300 | [diff] [blame] | 431 | .. deprecated-removed:: 3.8 3.10 |
| 432 | The *loop* parameter. |
| 433 | |
Yury Selivanov | 950204d | 2016-05-16 16:23:00 -0400 | [diff] [blame] | 434 | |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 435 | Timeouts |
| 436 | ======== |
| 437 | |
Yury Selivanov | e247b46 | 2018-09-20 12:43:59 -0400 | [diff] [blame] | 438 | .. coroutinefunction:: wait_for(aw, timeout, \*, loop=None) |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 439 | |
Yury Selivanov | e247b46 | 2018-09-20 12:43:59 -0400 | [diff] [blame] | 440 | Wait for the *aw* :ref:`awaitable <asyncio-awaitables>` |
Yury Selivanov | 4715039 | 2018-09-18 17:55:44 -0400 | [diff] [blame] | 441 | to complete with a timeout. |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 442 | |
Yury Selivanov | e247b46 | 2018-09-20 12:43:59 -0400 | [diff] [blame] | 443 | If *aw* is a coroutine it is automatically scheduled as a Task. |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 444 | |
| 445 | *timeout* can either be ``None`` or a float or int number of seconds |
| 446 | to wait for. If *timeout* is ``None``, block until the future |
| 447 | completes. |
| 448 | |
| 449 | If a timeout occurs, it cancels the task and raises |
| 450 | :exc:`asyncio.TimeoutError`. |
| 451 | |
Yury Selivanov | 4715039 | 2018-09-18 17:55:44 -0400 | [diff] [blame] | 452 | To avoid the task :meth:`cancellation <Task.cancel>`, |
| 453 | wrap it in :func:`shield`. |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 454 | |
| 455 | The function will wait until the future is actually cancelled, |
romasku | 382a563 | 2020-05-15 23:12:05 +0300 | [diff] [blame] | 456 | so the total wait time may exceed the *timeout*. If an exception |
| 457 | happens during cancellation, it is propagated. |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 458 | |
Yury Selivanov | e247b46 | 2018-09-20 12:43:59 -0400 | [diff] [blame] | 459 | If the wait is cancelled, the future *aw* is also cancelled. |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 460 | |
Matthias Bussonnier | d0ebf13 | 2019-05-20 23:20:10 -0700 | [diff] [blame] | 461 | .. deprecated-removed:: 3.8 3.10 |
| 462 | The *loop* parameter. |
Yury Selivanov | 4715039 | 2018-09-18 17:55:44 -0400 | [diff] [blame] | 463 | |
Yury Selivanov | 7372c3b | 2018-09-14 15:11:24 -0700 | [diff] [blame] | 464 | .. _asyncio_example_waitfor: |
| 465 | |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 466 | Example:: |
| 467 | |
| 468 | async def eternity(): |
| 469 | # Sleep for one hour |
| 470 | await asyncio.sleep(3600) |
| 471 | print('yay!') |
| 472 | |
| 473 | async def main(): |
| 474 | # Wait for at most 1 second |
| 475 | try: |
| 476 | await asyncio.wait_for(eternity(), timeout=1.0) |
| 477 | except asyncio.TimeoutError: |
| 478 | print('timeout!') |
| 479 | |
| 480 | asyncio.run(main()) |
| 481 | |
| 482 | # Expected output: |
| 483 | # |
| 484 | # timeout! |
| 485 | |
| 486 | .. versionchanged:: 3.7 |
Yury Selivanov | e247b46 | 2018-09-20 12:43:59 -0400 | [diff] [blame] | 487 | When *aw* is cancelled due to a timeout, ``wait_for`` waits |
| 488 | for *aw* to be cancelled. Previously, it raised |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 489 | :exc:`asyncio.TimeoutError` immediately. |
| 490 | |
| 491 | |
| 492 | Waiting Primitives |
| 493 | ================== |
| 494 | |
Yury Selivanov | e247b46 | 2018-09-20 12:43:59 -0400 | [diff] [blame] | 495 | .. coroutinefunction:: wait(aws, \*, loop=None, timeout=None,\ |
Andrew Svetlov | f124016 | 2016-01-11 14:40:35 +0200 | [diff] [blame] | 496 | return_when=ALL_COMPLETED) |
Victor Stinner | ea3183f | 2013-12-03 01:08:00 +0100 | [diff] [blame] | 497 | |
Yury Selivanov | e247b46 | 2018-09-20 12:43:59 -0400 | [diff] [blame] | 498 | Run :ref:`awaitable objects <asyncio-awaitables>` in the *aws* |
Yury Selivanov | 996859a | 2018-09-25 14:51:21 -0400 | [diff] [blame] | 499 | set concurrently and block until the condition specified |
Yury Selivanov | 4715039 | 2018-09-18 17:55:44 -0400 | [diff] [blame] | 500 | by *return_when*. |
Victor Stinner | ea3183f | 2013-12-03 01:08:00 +0100 | [diff] [blame] | 501 | |
Joel Rosdahl | 9d74658 | 2020-05-04 23:56:00 +0200 | [diff] [blame] | 502 | The *aws* set must not be empty. |
| 503 | |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 504 | Returns two sets of Tasks/Futures: ``(done, pending)``. |
Victor Stinner | ea3183f | 2013-12-03 01:08:00 +0100 | [diff] [blame] | 505 | |
Yury Selivanov | 996859a | 2018-09-25 14:51:21 -0400 | [diff] [blame] | 506 | Usage:: |
| 507 | |
| 508 | done, pending = await asyncio.wait(aws) |
| 509 | |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 510 | *timeout* (a float or int), if specified, can be used to control |
| 511 | the maximum number of seconds to wait before returning. |
| 512 | |
| 513 | Note that this function does not raise :exc:`asyncio.TimeoutError`. |
Elvis Pranskevichus | 1fa2ec4 | 2018-09-17 19:16:44 -0400 | [diff] [blame] | 514 | Futures or Tasks that aren't done when the timeout occurs are simply |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 515 | returned in the second set. |
| 516 | |
| 517 | *return_when* indicates when this function should return. It must |
| 518 | be one of the following constants: |
Victor Stinner | ea3183f | 2013-12-03 01:08:00 +0100 | [diff] [blame] | 519 | |
| 520 | .. tabularcolumns:: |l|L| |
| 521 | |
| 522 | +-----------------------------+----------------------------------------+ |
| 523 | | Constant | Description | |
| 524 | +=============================+========================================+ |
| 525 | | :const:`FIRST_COMPLETED` | The function will return when any | |
| 526 | | | future finishes or is cancelled. | |
| 527 | +-----------------------------+----------------------------------------+ |
| 528 | | :const:`FIRST_EXCEPTION` | The function will return when any | |
| 529 | | | future finishes by raising an | |
| 530 | | | exception. If no future raises an | |
| 531 | | | exception then it is equivalent to | |
| 532 | | | :const:`ALL_COMPLETED`. | |
| 533 | +-----------------------------+----------------------------------------+ |
| 534 | | :const:`ALL_COMPLETED` | The function will return when all | |
| 535 | | | futures finish or are cancelled. | |
| 536 | +-----------------------------+----------------------------------------+ |
| 537 | |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 538 | Unlike :func:`~asyncio.wait_for`, ``wait()`` does not cancel the |
| 539 | futures when a timeout occurs. |
Victor Stinner | ea3183f | 2013-12-03 01:08:00 +0100 | [diff] [blame] | 540 | |
Andrew Svetlov | a488879 | 2019-09-12 15:40:40 +0300 | [diff] [blame] | 541 | .. deprecated:: 3.8 |
| 542 | |
| 543 | If any awaitable in *aws* is a coroutine, it is automatically |
| 544 | scheduled as a Task. Passing coroutines objects to |
| 545 | ``wait()`` directly is deprecated as it leads to |
| 546 | :ref:`confusing behavior <asyncio_example_wait_coroutine>`. |
| 547 | |
| 548 | .. deprecated-removed:: 3.8 3.10 |
| 549 | |
| 550 | The *loop* parameter. |
| 551 | |
Yury Selivanov | 996859a | 2018-09-25 14:51:21 -0400 | [diff] [blame] | 552 | .. _asyncio_example_wait_coroutine: |
| 553 | .. note:: |
Victor Stinner | ea3183f | 2013-12-03 01:08:00 +0100 | [diff] [blame] | 554 | |
Yury Selivanov | 996859a | 2018-09-25 14:51:21 -0400 | [diff] [blame] | 555 | ``wait()`` schedules coroutines as Tasks automatically and later |
| 556 | returns those implicitly created Task objects in ``(done, pending)`` |
| 557 | sets. Therefore the following code won't work as expected:: |
| 558 | |
| 559 | async def foo(): |
| 560 | return 42 |
| 561 | |
| 562 | coro = foo() |
| 563 | done, pending = await asyncio.wait({coro}) |
| 564 | |
| 565 | if coro in done: |
| 566 | # This branch will never be run! |
| 567 | |
| 568 | Here is how the above snippet can be fixed:: |
| 569 | |
| 570 | async def foo(): |
| 571 | return 42 |
| 572 | |
| 573 | task = asyncio.create_task(foo()) |
| 574 | done, pending = await asyncio.wait({task}) |
| 575 | |
| 576 | if task in done: |
| 577 | # Everything will work as expected now. |
| 578 | |
jack1142 | de92769 | 2020-05-13 20:55:12 +0200 | [diff] [blame] | 579 | .. deprecated-removed:: 3.8 3.11 |
Matthias Bussonnier | d0ebf13 | 2019-05-20 23:20:10 -0700 | [diff] [blame] | 580 | |
Yury Selivanov | 996859a | 2018-09-25 14:51:21 -0400 | [diff] [blame] | 581 | Passing coroutine objects to ``wait()`` directly is |
| 582 | deprecated. |
Victor Stinner | ea3183f | 2013-12-03 01:08:00 +0100 | [diff] [blame] | 583 | |
Victor Stinner | ea3183f | 2013-12-03 01:08:00 +0100 | [diff] [blame] | 584 | |
Yury Selivanov | e247b46 | 2018-09-20 12:43:59 -0400 | [diff] [blame] | 585 | .. function:: as_completed(aws, \*, loop=None, timeout=None) |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 586 | |
Yury Selivanov | e247b46 | 2018-09-20 12:43:59 -0400 | [diff] [blame] | 587 | Run :ref:`awaitable objects <asyncio-awaitables>` in the *aws* |
Miss Islington (bot) | 9181e2e | 2020-05-23 16:23:53 -0700 | [diff] [blame] | 588 | set concurrently. Return an iterator of coroutines. |
| 589 | Each coroutine returned can be awaited to get the earliest next |
| 590 | result from the set of the remaining awaitables. |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 591 | |
| 592 | Raises :exc:`asyncio.TimeoutError` if the timeout occurs before |
| 593 | all Futures are done. |
| 594 | |
Andrew Svetlov | a488879 | 2019-09-12 15:40:40 +0300 | [diff] [blame] | 595 | .. deprecated-removed:: 3.8 3.10 |
| 596 | The *loop* parameter. |
| 597 | |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 598 | Example:: |
| 599 | |
Miss Islington (bot) | 9181e2e | 2020-05-23 16:23:53 -0700 | [diff] [blame] | 600 | for coro in as_completed(aws): |
| 601 | earliest_result = await coro |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 602 | # ... |
Victor Stinner | ea3183f | 2013-12-03 01:08:00 +0100 | [diff] [blame] | 603 | |
Victor Stinner | 3e09e32 | 2013-12-03 01:22:06 +0100 | [diff] [blame] | 604 | |
Miss Islington (bot) | e299130 | 2020-05-19 03:03:25 -0700 | [diff] [blame] | 605 | Running in Threads |
| 606 | ================== |
| 607 | |
| 608 | .. coroutinefunction:: to_thread(func, /, \*args, \*\*kwargs) |
| 609 | |
| 610 | Asynchronously run function *func* in a separate thread. |
| 611 | |
| 612 | Any \*args and \*\*kwargs supplied for this function are directly passed |
Miss Islington (bot) | 3e65054 | 2020-05-20 22:38:00 -0700 | [diff] [blame] | 613 | to *func*. Also, the current :class:`contextvars.Context` is propogated, |
| 614 | allowing context variables from the event loop thread to be accessed in the |
| 615 | separate thread. |
Miss Islington (bot) | e299130 | 2020-05-19 03:03:25 -0700 | [diff] [blame] | 616 | |
Miss Islington (bot) | cdb015b | 2020-05-31 00:26:20 -0700 | [diff] [blame^] | 617 | Return a coroutine that can be awaited to get the eventual result of *func*. |
Miss Islington (bot) | e299130 | 2020-05-19 03:03:25 -0700 | [diff] [blame] | 618 | |
| 619 | This coroutine function is primarily intended to be used for executing |
| 620 | IO-bound functions/methods that would otherwise block the event loop if |
| 621 | they were ran in the main thread. For example:: |
| 622 | |
| 623 | def blocking_io(): |
| 624 | print(f"start blocking_io at {time.strftime('%X')}") |
| 625 | # Note that time.sleep() can be replaced with any blocking |
| 626 | # IO-bound operation, such as file operations. |
| 627 | time.sleep(1) |
| 628 | print(f"blocking_io complete at {time.strftime('%X')}") |
| 629 | |
| 630 | async def main(): |
| 631 | print(f"started main at {time.strftime('%X')}") |
| 632 | |
| 633 | await asyncio.gather( |
| 634 | asyncio.to_thread(blocking_io), |
| 635 | asyncio.sleep(1)) |
| 636 | |
| 637 | print(f"finished main at {time.strftime('%X')}") |
| 638 | |
| 639 | |
| 640 | asyncio.run(main()) |
| 641 | |
| 642 | # Expected output: |
| 643 | # |
| 644 | # started main at 19:50:53 |
| 645 | # start blocking_io at 19:50:53 |
| 646 | # blocking_io complete at 19:50:54 |
| 647 | # finished main at 19:50:54 |
| 648 | |
| 649 | Directly calling `blocking_io()` in any coroutine would block the event loop |
| 650 | for its duration, resulting in an additional 1 second of run time. Instead, |
| 651 | by using `asyncio.to_thread()`, we can run it in a separate thread without |
| 652 | blocking the event loop. |
| 653 | |
| 654 | .. note:: |
| 655 | |
| 656 | Due to the :term:`GIL`, `asyncio.to_thread()` can typically only be used |
| 657 | to make IO-bound functions non-blocking. However, for extension modules |
| 658 | that release the GIL or alternative Python implementations that don't |
| 659 | have one, `asyncio.to_thread()` can also be used for CPU-bound functions. |
| 660 | |
Miss Islington (bot) | 3e65054 | 2020-05-20 22:38:00 -0700 | [diff] [blame] | 661 | .. versionadded:: 3.9 |
| 662 | |
Miss Islington (bot) | e299130 | 2020-05-19 03:03:25 -0700 | [diff] [blame] | 663 | |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 664 | Scheduling From Other Threads |
| 665 | ============================= |
Victor Stinner | 1ad5afc | 2014-01-30 00:18:50 +0100 | [diff] [blame] | 666 | |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 667 | .. function:: run_coroutine_threadsafe(coro, loop) |
Victor Stinner | 1ad5afc | 2014-01-30 00:18:50 +0100 | [diff] [blame] | 668 | |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 669 | Submit a coroutine to the given event loop. Thread-safe. |
Victor Stinner | 1ad5afc | 2014-01-30 00:18:50 +0100 | [diff] [blame] | 670 | |
Yury Selivanov | 4715039 | 2018-09-18 17:55:44 -0400 | [diff] [blame] | 671 | Return a :class:`concurrent.futures.Future` to wait for the result |
| 672 | from another OS thread. |
Victor Stinner | 1ad5afc | 2014-01-30 00:18:50 +0100 | [diff] [blame] | 673 | |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 674 | This function is meant to be called from a different OS thread |
| 675 | than the one where the event loop is running. Example:: |
Victor Stinner | 72dcb0a | 2015-04-03 17:08:19 +0200 | [diff] [blame] | 676 | |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 677 | # Create a coroutine |
| 678 | coro = asyncio.sleep(1, result=3) |
Yury Selivanov | 37f15bc | 2014-02-20 16:20:44 -0500 | [diff] [blame] | 679 | |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 680 | # Submit the coroutine to a given loop |
| 681 | future = asyncio.run_coroutine_threadsafe(coro, loop) |
Victor Stinner | 1ad5afc | 2014-01-30 00:18:50 +0100 | [diff] [blame] | 682 | |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 683 | # Wait for the result with an optional timeout argument |
| 684 | assert future.result(timeout) == 3 |
| 685 | |
| 686 | If an exception is raised in the coroutine, the returned Future |
| 687 | will be notified. It can also be used to cancel the task in |
| 688 | the event loop:: |
| 689 | |
| 690 | try: |
| 691 | result = future.result(timeout) |
| 692 | except asyncio.TimeoutError: |
| 693 | print('The coroutine took too long, cancelling the task...') |
| 694 | future.cancel() |
| 695 | except Exception as exc: |
Mariatta | 9f43fbb | 2018-10-24 15:37:12 -0700 | [diff] [blame] | 696 | print(f'The coroutine raised an exception: {exc!r}') |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 697 | else: |
Mariatta | 9f43fbb | 2018-10-24 15:37:12 -0700 | [diff] [blame] | 698 | print(f'The coroutine returned: {result!r}') |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 699 | |
| 700 | See the :ref:`concurrency and multithreading <asyncio-multithreading>` |
| 701 | section of the documentation. |
| 702 | |
Vaibhav Gupta | 3a81076 | 2018-12-26 20:17:17 +0530 | [diff] [blame] | 703 | Unlike other asyncio functions this function requires the *loop* |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 704 | argument to be passed explicitly. |
| 705 | |
| 706 | .. versionadded:: 3.5.1 |
| 707 | |
| 708 | |
| 709 | Introspection |
| 710 | ============= |
| 711 | |
| 712 | |
| 713 | .. function:: current_task(loop=None) |
| 714 | |
| 715 | Return the currently running :class:`Task` instance, or ``None`` if |
| 716 | no task is running. |
| 717 | |
| 718 | If *loop* is ``None`` :func:`get_running_loop` is used to get |
| 719 | the current loop. |
| 720 | |
| 721 | .. versionadded:: 3.7 |
| 722 | |
| 723 | |
| 724 | .. function:: all_tasks(loop=None) |
| 725 | |
| 726 | Return a set of not yet finished :class:`Task` objects run by |
| 727 | the loop. |
| 728 | |
| 729 | If *loop* is ``None``, :func:`get_running_loop` is used for getting |
| 730 | current loop. |
| 731 | |
| 732 | .. versionadded:: 3.7 |
| 733 | |
| 734 | |
| 735 | Task Object |
| 736 | =========== |
| 737 | |
| 738 | .. class:: Task(coro, \*, loop=None, name=None) |
| 739 | |
Yury Selivanov | db1a80e | 2018-09-21 16:23:15 -0400 | [diff] [blame] | 740 | A :class:`Future-like <Future>` object that runs a Python |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 741 | :ref:`coroutine <coroutine>`. Not thread-safe. |
| 742 | |
| 743 | Tasks are used to run coroutines in event loops. |
| 744 | If a coroutine awaits on a Future, the Task suspends |
| 745 | the execution of the coroutine and waits for the completion |
| 746 | of the Future. When the Future is *done*, the execution of |
| 747 | the wrapped coroutine resumes. |
| 748 | |
| 749 | Event loops use cooperative scheduling: an event loop runs |
| 750 | one Task at a time. While a Task awaits for the completion of a |
| 751 | Future, the event loop runs other Tasks, callbacks, or performs |
| 752 | IO operations. |
| 753 | |
| 754 | Use the high-level :func:`asyncio.create_task` function to create |
Elvis Pranskevichus | 1fa2ec4 | 2018-09-17 19:16:44 -0400 | [diff] [blame] | 755 | Tasks, or the low-level :meth:`loop.create_task` or |
| 756 | :func:`ensure_future` functions. Manual instantiation of Tasks |
| 757 | is discouraged. |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 758 | |
| 759 | To cancel a running Task use the :meth:`cancel` method. Calling it |
| 760 | will cause the Task to throw a :exc:`CancelledError` exception into |
| 761 | the wrapped coroutine. If a coroutine is awaiting on a Future |
| 762 | object during cancellation, the Future object will be cancelled. |
| 763 | |
| 764 | :meth:`cancelled` can be used to check if the Task was cancelled. |
| 765 | The method returns ``True`` if the wrapped coroutine did not |
| 766 | suppress the :exc:`CancelledError` exception and was actually |
| 767 | cancelled. |
| 768 | |
| 769 | :class:`asyncio.Task` inherits from :class:`Future` all of its |
| 770 | APIs except :meth:`Future.set_result` and |
| 771 | :meth:`Future.set_exception`. |
| 772 | |
| 773 | Tasks support the :mod:`contextvars` module. When a Task |
| 774 | is created it copies the current context and later runs its |
| 775 | coroutine in the copied context. |
Elvis Pranskevichus | e2b340a | 2018-05-29 17:31:01 -0400 | [diff] [blame] | 776 | |
| 777 | .. versionchanged:: 3.7 |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 778 | Added support for the :mod:`contextvars` module. |
| 779 | |
| 780 | .. versionchanged:: 3.8 |
| 781 | Added the ``name`` parameter. |
| 782 | |
Andrew Svetlov | a488879 | 2019-09-12 15:40:40 +0300 | [diff] [blame] | 783 | .. deprecated-removed:: 3.8 3.10 |
| 784 | The *loop* parameter. |
| 785 | |
Chris Jerdonek | 1ce5841 | 2020-05-15 16:55:50 -0700 | [diff] [blame] | 786 | .. method:: cancel(msg=None) |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 787 | |
| 788 | Request the Task to be cancelled. |
| 789 | |
| 790 | This arranges for a :exc:`CancelledError` exception to be thrown |
| 791 | into the wrapped coroutine on the next cycle of the event loop. |
| 792 | |
| 793 | The coroutine then has a chance to clean up or even deny the |
| 794 | request by suppressing the exception with a :keyword:`try` ... |
| 795 | ... ``except CancelledError`` ... :keyword:`finally` block. |
| 796 | Therefore, unlike :meth:`Future.cancel`, :meth:`Task.cancel` does |
| 797 | not guarantee that the Task will be cancelled, although |
| 798 | suppressing cancellation completely is not common and is actively |
| 799 | discouraged. |
| 800 | |
Chris Jerdonek | 1ce5841 | 2020-05-15 16:55:50 -0700 | [diff] [blame] | 801 | .. versionchanged:: 3.9 |
| 802 | Added the ``msg`` parameter. |
| 803 | |
Yury Selivanov | 7372c3b | 2018-09-14 15:11:24 -0700 | [diff] [blame] | 804 | .. _asyncio_example_task_cancel: |
| 805 | |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 806 | The following example illustrates how coroutines can intercept |
| 807 | the cancellation request:: |
| 808 | |
| 809 | async def cancel_me(): |
| 810 | print('cancel_me(): before sleep') |
| 811 | |
| 812 | try: |
| 813 | # Wait for 1 hour |
| 814 | await asyncio.sleep(3600) |
| 815 | except asyncio.CancelledError: |
| 816 | print('cancel_me(): cancel sleep') |
| 817 | raise |
| 818 | finally: |
| 819 | print('cancel_me(): after sleep') |
| 820 | |
| 821 | async def main(): |
| 822 | # Create a "cancel_me" Task |
| 823 | task = asyncio.create_task(cancel_me()) |
| 824 | |
| 825 | # Wait for 1 second |
| 826 | await asyncio.sleep(1) |
| 827 | |
| 828 | task.cancel() |
| 829 | try: |
| 830 | await task |
| 831 | except asyncio.CancelledError: |
| 832 | print("main(): cancel_me is cancelled now") |
| 833 | |
| 834 | asyncio.run(main()) |
| 835 | |
| 836 | # Expected output: |
| 837 | # |
| 838 | # cancel_me(): before sleep |
| 839 | # cancel_me(): cancel sleep |
| 840 | # cancel_me(): after sleep |
| 841 | # main(): cancel_me is cancelled now |
| 842 | |
| 843 | .. method:: cancelled() |
| 844 | |
| 845 | Return ``True`` if the Task is *cancelled*. |
| 846 | |
| 847 | The Task is *cancelled* when the cancellation was requested with |
| 848 | :meth:`cancel` and the wrapped coroutine propagated the |
| 849 | :exc:`CancelledError` exception thrown into it. |
| 850 | |
| 851 | .. method:: done() |
| 852 | |
| 853 | Return ``True`` if the Task is *done*. |
| 854 | |
| 855 | A Task is *done* when the wrapped coroutine either returned |
| 856 | a value, raised an exception, or the Task was cancelled. |
| 857 | |
Yury Selivanov | e247b46 | 2018-09-20 12:43:59 -0400 | [diff] [blame] | 858 | .. method:: result() |
| 859 | |
| 860 | Return the result of the Task. |
| 861 | |
| 862 | If the Task is *done*, the result of the wrapped coroutine |
| 863 | is returned (or if the coroutine raised an exception, that |
| 864 | exception is re-raised.) |
| 865 | |
| 866 | If the Task has been *cancelled*, this method raises |
| 867 | a :exc:`CancelledError` exception. |
| 868 | |
| 869 | If the Task's result isn't yet available, this method raises |
| 870 | a :exc:`InvalidStateError` exception. |
| 871 | |
| 872 | .. method:: exception() |
| 873 | |
| 874 | Return the exception of the Task. |
| 875 | |
| 876 | If the wrapped coroutine raised an exception that exception |
| 877 | is returned. If the wrapped coroutine returned normally |
| 878 | this method returns ``None``. |
| 879 | |
| 880 | If the Task has been *cancelled*, this method raises a |
| 881 | :exc:`CancelledError` exception. |
| 882 | |
| 883 | If the Task isn't *done* yet, this method raises an |
| 884 | :exc:`InvalidStateError` exception. |
| 885 | |
| 886 | .. method:: add_done_callback(callback, *, context=None) |
| 887 | |
| 888 | Add a callback to be run when the Task is *done*. |
| 889 | |
| 890 | This method should only be used in low-level callback-based code. |
| 891 | |
| 892 | See the documentation of :meth:`Future.add_done_callback` |
| 893 | for more details. |
| 894 | |
| 895 | .. method:: remove_done_callback(callback) |
| 896 | |
| 897 | Remove *callback* from the callbacks list. |
| 898 | |
| 899 | This method should only be used in low-level callback-based code. |
| 900 | |
| 901 | See the documentation of :meth:`Future.remove_done_callback` |
| 902 | for more details. |
| 903 | |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 904 | .. method:: get_stack(\*, limit=None) |
| 905 | |
| 906 | Return the list of stack frames for this Task. |
| 907 | |
| 908 | If the wrapped coroutine is not done, this returns the stack |
| 909 | where it is suspended. If the coroutine has completed |
| 910 | successfully or was cancelled, this returns an empty list. |
| 911 | If the coroutine was terminated by an exception, this returns |
| 912 | the list of traceback frames. |
| 913 | |
| 914 | The frames are always ordered from oldest to newest. |
| 915 | |
| 916 | Only one stack frame is returned for a suspended coroutine. |
| 917 | |
| 918 | The optional *limit* argument sets the maximum number of frames |
| 919 | to return; by default all available frames are returned. |
| 920 | The ordering of the returned list differs depending on whether |
| 921 | a stack or a traceback is returned: the newest frames of a |
| 922 | stack are returned, but the oldest frames of a traceback are |
| 923 | returned. (This matches the behavior of the traceback module.) |
| 924 | |
| 925 | .. method:: print_stack(\*, limit=None, file=None) |
| 926 | |
| 927 | Print the stack or traceback for this Task. |
| 928 | |
| 929 | This produces output similar to that of the traceback module |
| 930 | for the frames retrieved by :meth:`get_stack`. |
| 931 | |
| 932 | The *limit* argument is passed to :meth:`get_stack` directly. |
| 933 | |
| 934 | The *file* argument is an I/O stream to which the output |
| 935 | is written; by default output is written to :data:`sys.stderr`. |
| 936 | |
Alex Grönholm | 98ef920 | 2019-05-30 18:30:09 +0300 | [diff] [blame] | 937 | .. method:: get_coro() |
| 938 | |
| 939 | Return the coroutine object wrapped by the :class:`Task`. |
| 940 | |
| 941 | .. versionadded:: 3.8 |
| 942 | |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 943 | .. method:: get_name() |
| 944 | |
| 945 | Return the name of the Task. |
| 946 | |
| 947 | If no name has been explicitly assigned to the Task, the default |
| 948 | asyncio Task implementation generates a default name during |
| 949 | instantiation. |
| 950 | |
| 951 | .. versionadded:: 3.8 |
| 952 | |
| 953 | .. method:: set_name(value) |
| 954 | |
| 955 | Set the name of the Task. |
| 956 | |
| 957 | The *value* argument can be any object, which is then |
| 958 | converted to a string. |
| 959 | |
| 960 | In the default Task implementation, the name will be visible |
| 961 | in the :func:`repr` output of a task object. |
| 962 | |
| 963 | .. versionadded:: 3.8 |
| 964 | |
| 965 | .. classmethod:: all_tasks(loop=None) |
| 966 | |
| 967 | Return a set of all tasks for an event loop. |
| 968 | |
| 969 | By default all tasks for the current event loop are returned. |
| 970 | If *loop* is ``None``, the :func:`get_event_loop` function |
| 971 | is used to get the current loop. |
| 972 | |
Matthias Bussonnier | d0ebf13 | 2019-05-20 23:20:10 -0700 | [diff] [blame] | 973 | .. deprecated-removed:: 3.7 3.9 |
| 974 | |
| 975 | Do not call this as a task method. Use the :func:`asyncio.all_tasks` |
| 976 | function instead. |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 977 | |
| 978 | .. classmethod:: current_task(loop=None) |
| 979 | |
| 980 | Return the currently running task or ``None``. |
| 981 | |
| 982 | If *loop* is ``None``, the :func:`get_event_loop` function |
| 983 | is used to get the current loop. |
| 984 | |
Matthias Bussonnier | d0ebf13 | 2019-05-20 23:20:10 -0700 | [diff] [blame] | 985 | .. deprecated-removed:: 3.7 3.9 |
| 986 | |
| 987 | Do not call this as a task method. Use the |
| 988 | :func:`asyncio.current_task` function instead. |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 989 | |
| 990 | |
| 991 | .. _asyncio_generator_based_coro: |
| 992 | |
| 993 | Generator-based Coroutines |
| 994 | ========================== |
| 995 | |
| 996 | .. note:: |
| 997 | |
| 998 | Support for generator-based coroutines is **deprecated** and |
Yury Selivanov | fad6af2 | 2018-09-25 17:44:52 -0400 | [diff] [blame] | 999 | is scheduled for removal in Python 3.10. |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 1000 | |
| 1001 | Generator-based coroutines predate async/await syntax. They are |
Elvis Pranskevichus | 1fa2ec4 | 2018-09-17 19:16:44 -0400 | [diff] [blame] | 1002 | Python generators that use ``yield from`` expressions to await |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 1003 | on Futures and other coroutines. |
| 1004 | |
| 1005 | Generator-based coroutines should be decorated with |
| 1006 | :func:`@asyncio.coroutine <asyncio.coroutine>`, although this is not |
| 1007 | enforced. |
| 1008 | |
| 1009 | |
| 1010 | .. decorator:: coroutine |
| 1011 | |
| 1012 | Decorator to mark generator-based coroutines. |
| 1013 | |
| 1014 | This decorator enables legacy generator-based coroutines to be |
| 1015 | compatible with async/await code:: |
| 1016 | |
| 1017 | @asyncio.coroutine |
| 1018 | def old_style_coroutine(): |
| 1019 | yield from asyncio.sleep(1) |
| 1020 | |
| 1021 | async def main(): |
| 1022 | await old_style_coroutine() |
| 1023 | |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 1024 | This decorator should not be used for :keyword:`async def` |
| 1025 | coroutines. |
| 1026 | |
Andrew Svetlov | 68b34a7 | 2019-05-16 17:52:10 +0300 | [diff] [blame] | 1027 | .. deprecated-removed:: 3.8 3.10 |
| 1028 | |
| 1029 | Use :keyword:`async def` instead. |
| 1030 | |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 1031 | .. function:: iscoroutine(obj) |
| 1032 | |
| 1033 | Return ``True`` if *obj* is a :ref:`coroutine object <coroutine>`. |
| 1034 | |
| 1035 | This method is different from :func:`inspect.iscoroutine` because |
Yury Selivanov | 59ee5b1 | 2018-09-27 15:48:30 -0400 | [diff] [blame] | 1036 | it returns ``True`` for generator-based coroutines. |
Yury Selivanov | 3faaa88 | 2018-09-14 13:32:07 -0700 | [diff] [blame] | 1037 | |
| 1038 | .. function:: iscoroutinefunction(func) |
| 1039 | |
| 1040 | Return ``True`` if *func* is a :ref:`coroutine function |
| 1041 | <coroutine>`. |
| 1042 | |
| 1043 | This method is different from :func:`inspect.iscoroutinefunction` |
| 1044 | because it returns ``True`` for generator-based coroutine functions |
| 1045 | decorated with :func:`@coroutine <coroutine>`. |