blob: 1ca1b4a06d7e8801c0b723d5fe947f2af8d92fc1 [file] [log] [blame]
R David Murray6a143812013-12-20 14:37:39 -05001.. currentmodule:: asyncio
Victor Stinnerea3183f2013-12-03 01:08:00 +01002
Yury Selivanov3faaa882018-09-14 13:32:07 -07003
4====================
5Coroutines and Tasks
Victor Stinnerea3183f2013-12-03 01:08:00 +01006====================
7
Yury Selivanov3faaa882018-09-14 13:32:07 -07008This section outlines high-level asyncio APIs to work with coroutines
9and Tasks.
lf627d2c82017-07-25 17:03:51 -060010
Yury Selivanov3faaa882018-09-14 13:32:07 -070011.. contents::
12 :depth: 1
13 :local:
14
lf627d2c82017-07-25 17:03:51 -060015
Victor Stinnerea3183f2013-12-03 01:08:00 +010016.. _coroutine:
17
18Coroutines
Yury Selivanov3faaa882018-09-14 13:32:07 -070019==========
Victor Stinnerea3183f2013-12-03 01:08:00 +010020
Kyle Stanleyf9000642019-10-10 19:18:46 -040021:term:`Coroutines <coroutine>` declared with the async/await syntax is the
22preferred way of writing asyncio applications. For example, the following
23snippet of code (requires Python 3.7+) prints "hello", waits 1 second,
Yury Selivanovb042cf12018-09-18 02:47:54 -040024and then prints "world"::
Victor Stinnerea3183f2013-12-03 01:08:00 +010025
Yury Selivanov3faaa882018-09-14 13:32:07 -070026 >>> import asyncio
Victor Stinnerea3183f2013-12-03 01:08:00 +010027
Yury Selivanov3faaa882018-09-14 13:32:07 -070028 >>> async def main():
29 ... print('hello')
30 ... await asyncio.sleep(1)
31 ... print('world')
Victor Stinnerea3183f2013-12-03 01:08:00 +010032
Yury Selivanov3faaa882018-09-14 13:32:07 -070033 >>> asyncio.run(main())
34 hello
35 world
Victor Stinnerea3183f2013-12-03 01:08:00 +010036
Yury Selivanov3faaa882018-09-14 13:32:07 -070037Note that simply calling a coroutine will not schedule it to
38be executed::
Victor Stinnerea3183f2013-12-03 01:08:00 +010039
Yury Selivanov3faaa882018-09-14 13:32:07 -070040 >>> main()
41 <coroutine object main at 0x1053bb7c8>
Victor Stinnerea3183f2013-12-03 01:08:00 +010042
Boštjan Mejak1d5bdef2019-05-19 11:01:36 +020043To actually run a coroutine, asyncio provides three main mechanisms:
Victor Stinnerea3183f2013-12-03 01:08:00 +010044
Elvis Pranskevichus1fa2ec42018-09-17 19:16:44 -040045* The :func:`asyncio.run` function to run the top-level
Yury Selivanov3faaa882018-09-14 13:32:07 -070046 entry point "main()" function (see the above example.)
Victor Stinnerea3183f2013-12-03 01:08:00 +010047
Elvis Pranskevichus1fa2ec42018-09-17 19:16:44 -040048* Awaiting on a coroutine. The following snippet of code will
Yury Selivanov3faaa882018-09-14 13:32:07 -070049 print "hello" after waiting for 1 second, and then print "world"
50 after waiting for *another* 2 seconds::
Victor Stinnerea3183f2013-12-03 01:08:00 +010051
Yury Selivanov3faaa882018-09-14 13:32:07 -070052 import asyncio
53 import time
Victor Stinnerea3183f2013-12-03 01:08:00 +010054
Yury Selivanov3faaa882018-09-14 13:32:07 -070055 async def say_after(delay, what):
56 await asyncio.sleep(delay)
57 print(what)
58
59 async def main():
Mariatta9f43fbb2018-10-24 15:37:12 -070060 print(f"started at {time.strftime('%X')}")
Yury Selivanov3faaa882018-09-14 13:32:07 -070061
62 await say_after(1, 'hello')
63 await say_after(2, 'world')
64
Mariatta9f43fbb2018-10-24 15:37:12 -070065 print(f"finished at {time.strftime('%X')}")
Yury Selivanov3faaa882018-09-14 13:32:07 -070066
67 asyncio.run(main())
68
69 Expected output::
70
71 started at 17:13:52
72 hello
73 world
74 finished at 17:13:55
75
Elvis Pranskevichus1fa2ec42018-09-17 19:16:44 -040076* The :func:`asyncio.create_task` function to run coroutines
Yury Selivanov3faaa882018-09-14 13:32:07 -070077 concurrently as asyncio :class:`Tasks <Task>`.
78
Danny Hermes7bfbda42018-09-17 21:49:21 -070079 Let's modify the above example and run two ``say_after`` coroutines
Yury Selivanov3faaa882018-09-14 13:32:07 -070080 *concurrently*::
81
82 async def main():
83 task1 = asyncio.create_task(
84 say_after(1, 'hello'))
85
86 task2 = asyncio.create_task(
87 say_after(2, 'world'))
88
Mariatta9f43fbb2018-10-24 15:37:12 -070089 print(f"started at {time.strftime('%X')}")
Yury Selivanov3faaa882018-09-14 13:32:07 -070090
91 # Wait until both tasks are completed (should take
92 # around 2 seconds.)
93 await task1
94 await task2
95
Mariatta9f43fbb2018-10-24 15:37:12 -070096 print(f"finished at {time.strftime('%X')}")
Yury Selivanov3faaa882018-09-14 13:32:07 -070097
98 Note that expected output now shows that the snippet runs
99 1 second faster than before::
100
101 started at 17:14:32
102 hello
103 world
104 finished at 17:14:34
105
Yury Selivanov47150392018-09-18 17:55:44 -0400106
107.. _asyncio-awaitables:
108
109Awaitables
110==========
111
Yury Selivanove247b462018-09-20 12:43:59 -0400112We say that an object is an **awaitable** object if it can be used
113in an :keyword:`await` expression. Many asyncio APIs are designed to
114accept awaitables.
115
116There are three main types of *awaitable* objects:
117**coroutines**, **Tasks**, and **Futures**.
Yury Selivanov47150392018-09-18 17:55:44 -0400118
119
Yury Selivanove247b462018-09-20 12:43:59 -0400120.. rubric:: Coroutines
Yury Selivanov47150392018-09-18 17:55:44 -0400121
Yury Selivanove247b462018-09-20 12:43:59 -0400122Python coroutines are *awaitables* and therefore can be awaited from
123other coroutines::
124
125 import asyncio
Yury Selivanov47150392018-09-18 17:55:44 -0400126
127 async def nested():
128 return 42
129
130 async def main():
Yury Selivanove247b462018-09-20 12:43:59 -0400131 # Nothing happens if we just call "nested()".
Yury Selivanovdb1a80e2018-09-21 16:23:15 -0400132 # A coroutine object is created but not awaited,
133 # so it *won't run at all*.
Yury Selivanove247b462018-09-20 12:43:59 -0400134 nested()
135
136 # Let's do it differently now and await it:
137 print(await nested()) # will print "42".
138
139 asyncio.run(main())
140
141.. important::
142
143 In this documentation the term "coroutine" can be used for
144 two closely related concepts:
145
146 * a *coroutine function*: an :keyword:`async def` function;
147
148 * a *coroutine object*: an object returned by calling a
149 *coroutine function*.
150
151asyncio also supports legacy :ref:`generator-based
152<asyncio_generator_based_coro>` coroutines.
153
154
155.. rubric:: Tasks
Yury Selivanov47150392018-09-18 17:55:44 -0400156
157*Tasks* are used to schedule coroutines *concurrently*.
Yury Selivanov47150392018-09-18 17:55:44 -0400158
Yury Selivanove247b462018-09-20 12:43:59 -0400159When a coroutine is wrapped into a *Task* with functions like
160:func:`asyncio.create_task` the coroutine is automatically
161scheduled to run soon::
Yury Selivanov3faaa882018-09-14 13:32:07 -0700162
Yury Selivanove247b462018-09-20 12:43:59 -0400163 import asyncio
Yury Selivanov3faaa882018-09-14 13:32:07 -0700164
Yury Selivanove247b462018-09-20 12:43:59 -0400165 async def nested():
166 return 42
167
168 async def main():
169 # Schedule nested() to run soon concurrently
170 # with "main()".
171 task = asyncio.create_task(nested())
172
173 # "task" can now be used to cancel "nested()", or
174 # can simply be awaited to wait until it is complete:
175 await task
176
177 asyncio.run(main())
Victor Stinner337e03f2014-08-11 01:11:13 +0200178
Victor Stinnerea3183f2013-12-03 01:08:00 +0100179
Yury Selivanov47150392018-09-18 17:55:44 -0400180.. rubric:: Futures
181
Yury Selivanove247b462018-09-20 12:43:59 -0400182A :class:`Future` is a special **low-level** awaitable object that
183represents an **eventual result** of an asynchronous operation.
Yury Selivanov47150392018-09-18 17:55:44 -0400184
Yury Selivanove247b462018-09-20 12:43:59 -0400185When a Future object is *awaited* it means that the coroutine will
186wait until the Future is resolved in some other place.
187
Yury Selivanov47150392018-09-18 17:55:44 -0400188Future objects in asyncio are needed to allow callback-based code
189to be used with async/await.
190
Yury Selivanove247b462018-09-20 12:43:59 -0400191Normally **there is no need** to create Future objects at the
Yury Selivanov47150392018-09-18 17:55:44 -0400192application level code.
193
194Future objects, sometimes exposed by libraries and some asyncio
Yury Selivanove247b462018-09-20 12:43:59 -0400195APIs, can be awaited::
Yury Selivanov47150392018-09-18 17:55:44 -0400196
197 async def main():
198 await function_that_returns_a_future_object()
199
200 # this is also valid:
201 await asyncio.gather(
202 function_that_returns_a_future_object(),
203 some_python_coroutine()
204 )
205
Yury Selivanove247b462018-09-20 12:43:59 -0400206A good example of a low-level function that returns a Future object
207is :meth:`loop.run_in_executor`.
208
Yury Selivanov47150392018-09-18 17:55:44 -0400209
Yury Selivanov3faaa882018-09-14 13:32:07 -0700210Running an asyncio Program
211==========================
Victor Stinnerea3183f2013-12-03 01:08:00 +0100212
Andre Delfinodcc997c2020-12-16 22:37:28 -0300213.. function:: run(coro, *, debug=False)
Yury Selivanov02a0a192017-12-14 09:42:21 -0500214
Kyle Stanleye4070132019-09-30 20:12:21 -0400215 Execute the :term:`coroutine` *coro* and return the result.
216
Yury Selivanov02a0a192017-12-14 09:42:21 -0500217 This function runs the passed coroutine, taking care of
Kyle Stanley9fdc64c2019-09-19 08:47:22 -0400218 managing the asyncio event loop, *finalizing asynchronous
219 generators*, and closing the threadpool.
Yury Selivanov02a0a192017-12-14 09:42:21 -0500220
221 This function cannot be called when another asyncio event loop is
222 running in the same thread.
223
Yury Selivanov3faaa882018-09-14 13:32:07 -0700224 If *debug* is ``True``, the event loop will be run in debug mode.
Yury Selivanov02a0a192017-12-14 09:42:21 -0500225
226 This function always creates a new event loop and closes it at
227 the end. It should be used as a main entry point for asyncio
228 programs, and should ideally only be called once.
229
Emmanuel Arias17deb162019-09-25 05:53:49 -0300230 Example::
231
232 async def main():
233 await asyncio.sleep(1)
234 print('hello')
235
236 asyncio.run(main())
237
Yury Selivanov02a0a192017-12-14 09:42:21 -0500238 .. versionadded:: 3.7
239
Kyle Stanley9fdc64c2019-09-19 08:47:22 -0400240 .. versionchanged:: 3.9
241 Updated to use :meth:`loop.shutdown_default_executor`.
Yury Selivanov02a0a192017-12-14 09:42:21 -0500242
Kyle Stanleyf9000642019-10-10 19:18:46 -0400243 .. note::
244 The source code for ``asyncio.run()`` can be found in
245 :source:`Lib/asyncio/runners.py`.
246
Yury Selivanov3faaa882018-09-14 13:32:07 -0700247Creating Tasks
248==============
Victor Stinnerea3183f2013-12-03 01:08:00 +0100249
Andre Delfinodcc997c2020-12-16 22:37:28 -0300250.. function:: create_task(coro, *, name=None)
Andrew Svetlovf74ef452017-12-15 07:04:38 +0200251
Yury Selivanove247b462018-09-20 12:43:59 -0400252 Wrap the *coro* :ref:`coroutine <coroutine>` into a :class:`Task`
253 and schedule its execution. Return the Task object.
Alex Grönholmcca4eec2018-08-09 00:06:47 +0300254
255 If *name* is not ``None``, it is set as the name of the task using
256 :meth:`Task.set_name`.
Andrew Svetlovf74ef452017-12-15 07:04:38 +0200257
Elvis Pranskevichus1fa2ec42018-09-17 19:16:44 -0400258 The task is executed in the loop returned by :func:`get_running_loop`,
Andrew Svetlovf74ef452017-12-15 07:04:38 +0200259 :exc:`RuntimeError` is raised if there is no running loop in
260 current thread.
261
Yury Selivanov47150392018-09-18 17:55:44 -0400262 This function has been **added in Python 3.7**. Prior to
263 Python 3.7, the low-level :func:`asyncio.ensure_future` function
264 can be used instead::
265
266 async def coro():
267 ...
268
269 # In Python 3.7+
270 task = asyncio.create_task(coro())
271 ...
272
273 # This works in all Python versions but is less readable
274 task = asyncio.ensure_future(coro())
275 ...
276
Andrew Svetlovf74ef452017-12-15 07:04:38 +0200277 .. versionadded:: 3.7
278
Alex Grönholmcca4eec2018-08-09 00:06:47 +0300279 .. versionchanged:: 3.8
280 Added the ``name`` parameter.
281
Victor Stinnerea3183f2013-12-03 01:08:00 +0100282
Yury Selivanov3faaa882018-09-14 13:32:07 -0700283Sleeping
284========
Andrew Svetlovf1240162016-01-11 14:40:35 +0200285
Yurii Karabas86150d32020-11-29 14:50:57 +0200286.. coroutinefunction:: sleep(delay, result=None)
Victor Stinnerea3183f2013-12-03 01:08:00 +0100287
Yury Selivanov3faaa882018-09-14 13:32:07 -0700288 Block for *delay* seconds.
289
290 If *result* is provided, it is returned to the caller
Eli Bendersky2d26af82014-01-20 06:59:23 -0800291 when the coroutine completes.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100292
Hrvoje Nikšićcd602b82018-10-01 12:09:38 +0200293 ``sleep()`` always suspends the current task, allowing other tasks
294 to run.
295
Simon Willison5c301452021-01-06 18:03:18 -0800296 Setting the delay to 0 provides an optimized path to allow other
297 tasks to run. This can be used by long-running functions to avoid
298 blocking the event loop for the full duration of the function call.
299
Yury Selivanov7372c3b2018-09-14 15:11:24 -0700300 .. _asyncio_example_sleep:
Victor Stinner45b27ed2014-02-01 02:36:43 +0100301
Yury Selivanov3faaa882018-09-14 13:32:07 -0700302 Example of coroutine displaying the current date every second
Elvis Pranskevichus1fa2ec42018-09-17 19:16:44 -0400303 for 5 seconds::
Victor Stinnerbdd574d2015-02-12 22:49:18 +0100304
Yury Selivanov3faaa882018-09-14 13:32:07 -0700305 import asyncio
306 import datetime
Victor Stinnerea3183f2013-12-03 01:08:00 +0100307
Yury Selivanov3faaa882018-09-14 13:32:07 -0700308 async def display_date():
309 loop = asyncio.get_running_loop()
310 end_time = loop.time() + 5.0
311 while True:
312 print(datetime.datetime.now())
313 if (loop.time() + 1.0) >= end_time:
314 break
315 await asyncio.sleep(1)
316
317 asyncio.run(display_date())
318
319
320Running Tasks Concurrently
321==========================
322
Andre Delfinodcc997c2020-12-16 22:37:28 -0300323.. awaitablefunction:: gather(*aws, return_exceptions=False)
Yury Selivanov3faaa882018-09-14 13:32:07 -0700324
Yury Selivanove247b462018-09-20 12:43:59 -0400325 Run :ref:`awaitable objects <asyncio-awaitables>` in the *aws*
Yury Selivanov47150392018-09-18 17:55:44 -0400326 sequence *concurrently*.
Yury Selivanov3faaa882018-09-14 13:32:07 -0700327
Yury Selivanove247b462018-09-20 12:43:59 -0400328 If any awaitable in *aws* is a coroutine, it is automatically
Yury Selivanov47150392018-09-18 17:55:44 -0400329 scheduled as a Task.
Yury Selivanov3faaa882018-09-14 13:32:07 -0700330
Yury Selivanov47150392018-09-18 17:55:44 -0400331 If all awaitables are completed successfully, the result is an
332 aggregate list of returned values. The order of result values
Yury Selivanove247b462018-09-20 12:43:59 -0400333 corresponds to the order of awaitables in *aws*.
Yury Selivanov3faaa882018-09-14 13:32:07 -0700334
Yury Selivanovdb1a80e2018-09-21 16:23:15 -0400335 If *return_exceptions* is ``False`` (default), the first
336 raised exception is immediately propagated to the task that
337 awaits on ``gather()``. Other awaitables in the *aws* sequence
338 **won't be cancelled** and will continue to run.
339
Yury Selivanov47150392018-09-18 17:55:44 -0400340 If *return_exceptions* is ``True``, exceptions are treated the
341 same as successful results, and aggregated in the result list.
Yury Selivanov3faaa882018-09-14 13:32:07 -0700342
Yury Selivanovdb1a80e2018-09-21 16:23:15 -0400343 If ``gather()`` is *cancelled*, all submitted awaitables
Yury Selivanov3faaa882018-09-14 13:32:07 -0700344 (that have not completed yet) are also *cancelled*.
345
Yury Selivanove247b462018-09-20 12:43:59 -0400346 If any Task or Future from the *aws* sequence is *cancelled*, it is
Yury Selivanov47150392018-09-18 17:55:44 -0400347 treated as if it raised :exc:`CancelledError` -- the ``gather()``
348 call is **not** cancelled in this case. This is to prevent the
349 cancellation of one submitted Task/Future to cause other
350 Tasks/Futures to be cancelled.
Yury Selivanov3faaa882018-09-14 13:32:07 -0700351
Yury Selivanov7372c3b2018-09-14 15:11:24 -0700352 .. _asyncio_example_gather:
353
Yury Selivanov3faaa882018-09-14 13:32:07 -0700354 Example::
355
356 import asyncio
357
358 async def factorial(name, number):
359 f = 1
360 for i in range(2, number + 1):
361 print(f"Task {name}: Compute factorial({i})...")
362 await asyncio.sleep(1)
363 f *= i
364 print(f"Task {name}: factorial({number}) = {f}")
365
366 async def main():
Yury Selivanov47150392018-09-18 17:55:44 -0400367 # Schedule three calls *concurrently*:
Yury Selivanov3faaa882018-09-14 13:32:07 -0700368 await asyncio.gather(
369 factorial("A", 2),
370 factorial("B", 3),
371 factorial("C", 4),
Miguel Ángel García9c53fa62018-09-18 08:01:26 +0200372 )
Yury Selivanov3faaa882018-09-14 13:32:07 -0700373
374 asyncio.run(main())
375
376 # Expected output:
377 #
378 # Task A: Compute factorial(2)...
379 # Task B: Compute factorial(2)...
380 # Task C: Compute factorial(2)...
381 # Task A: factorial(2) = 2
382 # Task B: Compute factorial(3)...
383 # Task C: Compute factorial(3)...
384 # Task B: factorial(3) = 6
385 # Task C: Compute factorial(4)...
386 # Task C: factorial(4) = 24
387
Vinay Sharmad42528a2020-07-20 14:12:57 +0530388 .. note::
389 If *return_exceptions* is False, cancelling gather() after it
390 has been marked done won't cancel any submitted awaitables.
391 For instance, gather can be marked done after propagating an
392 exception to the caller, therefore, calling ``gather.cancel()``
393 after catching an exception (raised by one of the awaitables) from
394 gather won't cancel any other awaitables.
395
Yury Selivanov47150392018-09-18 17:55:44 -0400396 .. versionchanged:: 3.7
397 If the *gather* itself is cancelled, the cancellation is
398 propagated regardless of *return_exceptions*.
399
Yury Selivanov3faaa882018-09-14 13:32:07 -0700400
Yury Selivanovdb1a80e2018-09-21 16:23:15 -0400401Shielding From Cancellation
402===========================
Yury Selivanov3faaa882018-09-14 13:32:07 -0700403
Yurii Karabas86150d32020-11-29 14:50:57 +0200404.. awaitablefunction:: shield(aw)
Yury Selivanov3faaa882018-09-14 13:32:07 -0700405
Yury Selivanov47150392018-09-18 17:55:44 -0400406 Protect an :ref:`awaitable object <asyncio-awaitables>`
407 from being :meth:`cancelled <Task.cancel>`.
Yury Selivanov3faaa882018-09-14 13:32:07 -0700408
Yury Selivanovdb1a80e2018-09-21 16:23:15 -0400409 If *aw* is a coroutine it is automatically scheduled as a Task.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100410
411 The statement::
412
Andrew Svetlov88743422017-12-11 17:35:49 +0200413 res = await shield(something())
Victor Stinnerea3183f2013-12-03 01:08:00 +0100414
Elvis Pranskevichus1fa2ec42018-09-17 19:16:44 -0400415 is equivalent to::
Victor Stinnerea3183f2013-12-03 01:08:00 +0100416
Andrew Svetlov88743422017-12-11 17:35:49 +0200417 res = await something()
Victor Stinnerea3183f2013-12-03 01:08:00 +0100418
Yury Selivanov3faaa882018-09-14 13:32:07 -0700419 *except* that if the coroutine containing it is cancelled, the
420 Task running in ``something()`` is not cancelled. From the point
421 of view of ``something()``, the cancellation did not happen.
422 Although its caller is still cancelled, so the "await" expression
Elvis Pranskevichus1fa2ec42018-09-17 19:16:44 -0400423 still raises a :exc:`CancelledError`.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100424
Yury Selivanov3faaa882018-09-14 13:32:07 -0700425 If ``something()`` is cancelled by other means (i.e. from within
426 itself) that would also cancel ``shield()``.
427
428 If it is desired to completely ignore cancellation (not recommended)
429 the ``shield()`` function should be combined with a try/except
430 clause, as follows::
Victor Stinnerea3183f2013-12-03 01:08:00 +0100431
432 try:
Andrew Svetlov88743422017-12-11 17:35:49 +0200433 res = await shield(something())
Victor Stinnerea3183f2013-12-03 01:08:00 +0100434 except CancelledError:
435 res = None
436
Yury Selivanov950204d2016-05-16 16:23:00 -0400437
Yury Selivanov3faaa882018-09-14 13:32:07 -0700438Timeouts
439========
440
Yurii Karabas86150d32020-11-29 14:50:57 +0200441.. coroutinefunction:: wait_for(aw, timeout)
Yury Selivanov3faaa882018-09-14 13:32:07 -0700442
Yury Selivanove247b462018-09-20 12:43:59 -0400443 Wait for the *aw* :ref:`awaitable <asyncio-awaitables>`
Yury Selivanov47150392018-09-18 17:55:44 -0400444 to complete with a timeout.
Yury Selivanov3faaa882018-09-14 13:32:07 -0700445
Yury Selivanove247b462018-09-20 12:43:59 -0400446 If *aw* is a coroutine it is automatically scheduled as a Task.
Yury Selivanov3faaa882018-09-14 13:32:07 -0700447
448 *timeout* can either be ``None`` or a float or int number of seconds
449 to wait for. If *timeout* is ``None``, block until the future
450 completes.
451
452 If a timeout occurs, it cancels the task and raises
453 :exc:`asyncio.TimeoutError`.
454
Yury Selivanov47150392018-09-18 17:55:44 -0400455 To avoid the task :meth:`cancellation <Task.cancel>`,
456 wrap it in :func:`shield`.
Yury Selivanov3faaa882018-09-14 13:32:07 -0700457
458 The function will wait until the future is actually cancelled,
romasku382a5632020-05-15 23:12:05 +0300459 so the total wait time may exceed the *timeout*. If an exception
460 happens during cancellation, it is propagated.
Yury Selivanov3faaa882018-09-14 13:32:07 -0700461
Yury Selivanove247b462018-09-20 12:43:59 -0400462 If the wait is cancelled, the future *aw* is also cancelled.
Yury Selivanov3faaa882018-09-14 13:32:07 -0700463
Yury Selivanov7372c3b2018-09-14 15:11:24 -0700464 .. _asyncio_example_waitfor:
465
Yury Selivanov3faaa882018-09-14 13:32:07 -0700466 Example::
467
468 async def eternity():
469 # Sleep for one hour
470 await asyncio.sleep(3600)
471 print('yay!')
472
473 async def main():
474 # Wait for at most 1 second
475 try:
476 await asyncio.wait_for(eternity(), timeout=1.0)
477 except asyncio.TimeoutError:
478 print('timeout!')
479
480 asyncio.run(main())
481
482 # Expected output:
483 #
484 # timeout!
485
486 .. versionchanged:: 3.7
Yury Selivanove247b462018-09-20 12:43:59 -0400487 When *aw* is cancelled due to a timeout, ``wait_for`` waits
488 for *aw* to be cancelled. Previously, it raised
Yury Selivanov3faaa882018-09-14 13:32:07 -0700489 :exc:`asyncio.TimeoutError` immediately.
490
491
492Waiting Primitives
493==================
494
Andre Delfinodcc997c2020-12-16 22:37:28 -0300495.. coroutinefunction:: wait(aws, *, timeout=None, return_when=ALL_COMPLETED)
Victor Stinnerea3183f2013-12-03 01:08:00 +0100496
Yury Selivanove247b462018-09-20 12:43:59 -0400497 Run :ref:`awaitable objects <asyncio-awaitables>` in the *aws*
Jakub Stasiak3d86d092020-11-02 11:56:35 +0100498 iterable concurrently and block until the condition specified
Yury Selivanov47150392018-09-18 17:55:44 -0400499 by *return_when*.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100500
Jakub Stasiak3d86d092020-11-02 11:56:35 +0100501 The *aws* iterable must not be empty.
Joel Rosdahl9d746582020-05-04 23:56:00 +0200502
Yury Selivanov3faaa882018-09-14 13:32:07 -0700503 Returns two sets of Tasks/Futures: ``(done, pending)``.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100504
Yury Selivanov996859a2018-09-25 14:51:21 -0400505 Usage::
506
507 done, pending = await asyncio.wait(aws)
508
Yury Selivanov3faaa882018-09-14 13:32:07 -0700509 *timeout* (a float or int), if specified, can be used to control
510 the maximum number of seconds to wait before returning.
511
512 Note that this function does not raise :exc:`asyncio.TimeoutError`.
Elvis Pranskevichus1fa2ec42018-09-17 19:16:44 -0400513 Futures or Tasks that aren't done when the timeout occurs are simply
Yury Selivanov3faaa882018-09-14 13:32:07 -0700514 returned in the second set.
515
516 *return_when* indicates when this function should return. It must
517 be one of the following constants:
Victor Stinnerea3183f2013-12-03 01:08:00 +0100518
519 .. tabularcolumns:: |l|L|
520
521 +-----------------------------+----------------------------------------+
522 | Constant | Description |
523 +=============================+========================================+
524 | :const:`FIRST_COMPLETED` | The function will return when any |
525 | | future finishes or is cancelled. |
526 +-----------------------------+----------------------------------------+
527 | :const:`FIRST_EXCEPTION` | The function will return when any |
528 | | future finishes by raising an |
529 | | exception. If no future raises an |
530 | | exception then it is equivalent to |
531 | | :const:`ALL_COMPLETED`. |
532 +-----------------------------+----------------------------------------+
533 | :const:`ALL_COMPLETED` | The function will return when all |
534 | | futures finish or are cancelled. |
535 +-----------------------------+----------------------------------------+
536
Yury Selivanov3faaa882018-09-14 13:32:07 -0700537 Unlike :func:`~asyncio.wait_for`, ``wait()`` does not cancel the
538 futures when a timeout occurs.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100539
Andrew Svetlova4888792019-09-12 15:40:40 +0300540 .. deprecated:: 3.8
541
542 If any awaitable in *aws* is a coroutine, it is automatically
543 scheduled as a Task. Passing coroutines objects to
544 ``wait()`` directly is deprecated as it leads to
545 :ref:`confusing behavior <asyncio_example_wait_coroutine>`.
546
Yury Selivanov996859a2018-09-25 14:51:21 -0400547 .. _asyncio_example_wait_coroutine:
548 .. note::
Victor Stinnerea3183f2013-12-03 01:08:00 +0100549
Yury Selivanov996859a2018-09-25 14:51:21 -0400550 ``wait()`` schedules coroutines as Tasks automatically and later
551 returns those implicitly created Task objects in ``(done, pending)``
552 sets. Therefore the following code won't work as expected::
553
554 async def foo():
555 return 42
556
557 coro = foo()
558 done, pending = await asyncio.wait({coro})
559
560 if coro in done:
561 # This branch will never be run!
562
563 Here is how the above snippet can be fixed::
564
565 async def foo():
566 return 42
567
568 task = asyncio.create_task(foo())
569 done, pending = await asyncio.wait({task})
570
571 if task in done:
572 # Everything will work as expected now.
573
jack1142de927692020-05-13 20:55:12 +0200574 .. deprecated-removed:: 3.8 3.11
Matthias Bussonnierd0ebf132019-05-20 23:20:10 -0700575
Yury Selivanov996859a2018-09-25 14:51:21 -0400576 Passing coroutine objects to ``wait()`` directly is
577 deprecated.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100578
Victor Stinnerea3183f2013-12-03 01:08:00 +0100579
Andre Delfinodcc997c2020-12-16 22:37:28 -0300580.. function:: as_completed(aws, *, timeout=None)
Yury Selivanov3faaa882018-09-14 13:32:07 -0700581
Yury Selivanove247b462018-09-20 12:43:59 -0400582 Run :ref:`awaitable objects <asyncio-awaitables>` in the *aws*
Jakub Stasiak3d86d092020-11-02 11:56:35 +0100583 iterable concurrently. Return an iterator of coroutines.
Bar Harel13206b52020-05-24 02:14:31 +0300584 Each coroutine returned can be awaited to get the earliest next
Jakub Stasiak3d86d092020-11-02 11:56:35 +0100585 result from the iterable of the remaining awaitables.
Yury Selivanov3faaa882018-09-14 13:32:07 -0700586
587 Raises :exc:`asyncio.TimeoutError` if the timeout occurs before
588 all Futures are done.
589
590 Example::
591
Bar Harel13206b52020-05-24 02:14:31 +0300592 for coro in as_completed(aws):
593 earliest_result = await coro
Yury Selivanov3faaa882018-09-14 13:32:07 -0700594 # ...
Victor Stinnerea3183f2013-12-03 01:08:00 +0100595
Victor Stinner3e09e322013-12-03 01:22:06 +0100596
Kyle Stanleycc2bbc22020-05-18 23:03:28 -0400597Running in Threads
598==================
599
Andre Delfinodcc997c2020-12-16 22:37:28 -0300600.. coroutinefunction:: to_thread(func, /, *args, **kwargs)
Kyle Stanleycc2bbc22020-05-18 23:03:28 -0400601
602 Asynchronously run function *func* in a separate thread.
603
604 Any \*args and \*\*kwargs supplied for this function are directly passed
Jesús Cea989af252020-11-24 00:56:30 +0100605 to *func*. Also, the current :class:`contextvars.Context` is propagated,
Kyle Stanley0f562632020-05-21 01:20:43 -0400606 allowing context variables from the event loop thread to be accessed in the
607 separate thread.
Kyle Stanleycc2bbc22020-05-18 23:03:28 -0400608
Kyle Stanley2b201362020-05-31 03:07:04 -0400609 Return a coroutine that can be awaited to get the eventual result of *func*.
Kyle Stanleycc2bbc22020-05-18 23:03:28 -0400610
611 This coroutine function is primarily intended to be used for executing
612 IO-bound functions/methods that would otherwise block the event loop if
613 they were ran in the main thread. For example::
614
615 def blocking_io():
616 print(f"start blocking_io at {time.strftime('%X')}")
617 # Note that time.sleep() can be replaced with any blocking
618 # IO-bound operation, such as file operations.
619 time.sleep(1)
620 print(f"blocking_io complete at {time.strftime('%X')}")
621
622 async def main():
623 print(f"started main at {time.strftime('%X')}")
624
625 await asyncio.gather(
626 asyncio.to_thread(blocking_io),
627 asyncio.sleep(1))
628
629 print(f"finished main at {time.strftime('%X')}")
630
631
632 asyncio.run(main())
633
634 # Expected output:
635 #
636 # started main at 19:50:53
637 # start blocking_io at 19:50:53
638 # blocking_io complete at 19:50:54
639 # finished main at 19:50:54
640
641 Directly calling `blocking_io()` in any coroutine would block the event loop
642 for its duration, resulting in an additional 1 second of run time. Instead,
643 by using `asyncio.to_thread()`, we can run it in a separate thread without
644 blocking the event loop.
645
646 .. note::
647
648 Due to the :term:`GIL`, `asyncio.to_thread()` can typically only be used
649 to make IO-bound functions non-blocking. However, for extension modules
650 that release the GIL or alternative Python implementations that don't
651 have one, `asyncio.to_thread()` can also be used for CPU-bound functions.
652
Kyle Stanley0f562632020-05-21 01:20:43 -0400653 .. versionadded:: 3.9
654
Kyle Stanleycc2bbc22020-05-18 23:03:28 -0400655
Yury Selivanov3faaa882018-09-14 13:32:07 -0700656Scheduling From Other Threads
657=============================
Victor Stinner1ad5afc2014-01-30 00:18:50 +0100658
Yury Selivanov3faaa882018-09-14 13:32:07 -0700659.. function:: run_coroutine_threadsafe(coro, loop)
Victor Stinner1ad5afc2014-01-30 00:18:50 +0100660
Yury Selivanov3faaa882018-09-14 13:32:07 -0700661 Submit a coroutine to the given event loop. Thread-safe.
Victor Stinner1ad5afc2014-01-30 00:18:50 +0100662
Yury Selivanov47150392018-09-18 17:55:44 -0400663 Return a :class:`concurrent.futures.Future` to wait for the result
664 from another OS thread.
Victor Stinner1ad5afc2014-01-30 00:18:50 +0100665
Yury Selivanov3faaa882018-09-14 13:32:07 -0700666 This function is meant to be called from a different OS thread
667 than the one where the event loop is running. Example::
Victor Stinner72dcb0a2015-04-03 17:08:19 +0200668
Yury Selivanov3faaa882018-09-14 13:32:07 -0700669 # Create a coroutine
670 coro = asyncio.sleep(1, result=3)
Yury Selivanov37f15bc2014-02-20 16:20:44 -0500671
Yury Selivanov3faaa882018-09-14 13:32:07 -0700672 # Submit the coroutine to a given loop
673 future = asyncio.run_coroutine_threadsafe(coro, loop)
Victor Stinner1ad5afc2014-01-30 00:18:50 +0100674
Yury Selivanov3faaa882018-09-14 13:32:07 -0700675 # Wait for the result with an optional timeout argument
676 assert future.result(timeout) == 3
677
678 If an exception is raised in the coroutine, the returned Future
679 will be notified. It can also be used to cancel the task in
680 the event loop::
681
682 try:
683 result = future.result(timeout)
684 except asyncio.TimeoutError:
685 print('The coroutine took too long, cancelling the task...')
686 future.cancel()
687 except Exception as exc:
Mariatta9f43fbb2018-10-24 15:37:12 -0700688 print(f'The coroutine raised an exception: {exc!r}')
Yury Selivanov3faaa882018-09-14 13:32:07 -0700689 else:
Mariatta9f43fbb2018-10-24 15:37:12 -0700690 print(f'The coroutine returned: {result!r}')
Yury Selivanov3faaa882018-09-14 13:32:07 -0700691
692 See the :ref:`concurrency and multithreading <asyncio-multithreading>`
693 section of the documentation.
694
Vaibhav Gupta3a810762018-12-26 20:17:17 +0530695 Unlike other asyncio functions this function requires the *loop*
Yury Selivanov3faaa882018-09-14 13:32:07 -0700696 argument to be passed explicitly.
697
698 .. versionadded:: 3.5.1
699
700
701Introspection
702=============
703
704
705.. function:: current_task(loop=None)
706
707 Return the currently running :class:`Task` instance, or ``None`` if
708 no task is running.
709
710 If *loop* is ``None`` :func:`get_running_loop` is used to get
711 the current loop.
712
713 .. versionadded:: 3.7
714
715
716.. function:: all_tasks(loop=None)
717
718 Return a set of not yet finished :class:`Task` objects run by
719 the loop.
720
721 If *loop* is ``None``, :func:`get_running_loop` is used for getting
722 current loop.
723
724 .. versionadded:: 3.7
725
726
727Task Object
728===========
729
Andre Delfinodcc997c2020-12-16 22:37:28 -0300730.. class:: Task(coro, *, loop=None, name=None)
Yury Selivanov3faaa882018-09-14 13:32:07 -0700731
Yury Selivanovdb1a80e2018-09-21 16:23:15 -0400732 A :class:`Future-like <Future>` object that runs a Python
Yury Selivanov3faaa882018-09-14 13:32:07 -0700733 :ref:`coroutine <coroutine>`. Not thread-safe.
734
735 Tasks are used to run coroutines in event loops.
736 If a coroutine awaits on a Future, the Task suspends
737 the execution of the coroutine and waits for the completion
738 of the Future. When the Future is *done*, the execution of
739 the wrapped coroutine resumes.
740
741 Event loops use cooperative scheduling: an event loop runs
742 one Task at a time. While a Task awaits for the completion of a
743 Future, the event loop runs other Tasks, callbacks, or performs
744 IO operations.
745
746 Use the high-level :func:`asyncio.create_task` function to create
Elvis Pranskevichus1fa2ec42018-09-17 19:16:44 -0400747 Tasks, or the low-level :meth:`loop.create_task` or
748 :func:`ensure_future` functions. Manual instantiation of Tasks
749 is discouraged.
Yury Selivanov3faaa882018-09-14 13:32:07 -0700750
751 To cancel a running Task use the :meth:`cancel` method. Calling it
752 will cause the Task to throw a :exc:`CancelledError` exception into
753 the wrapped coroutine. If a coroutine is awaiting on a Future
754 object during cancellation, the Future object will be cancelled.
755
756 :meth:`cancelled` can be used to check if the Task was cancelled.
757 The method returns ``True`` if the wrapped coroutine did not
758 suppress the :exc:`CancelledError` exception and was actually
759 cancelled.
760
761 :class:`asyncio.Task` inherits from :class:`Future` all of its
762 APIs except :meth:`Future.set_result` and
763 :meth:`Future.set_exception`.
764
765 Tasks support the :mod:`contextvars` module. When a Task
766 is created it copies the current context and later runs its
767 coroutine in the copied context.
Elvis Pranskevichuse2b340a2018-05-29 17:31:01 -0400768
769 .. versionchanged:: 3.7
Yury Selivanov3faaa882018-09-14 13:32:07 -0700770 Added support for the :mod:`contextvars` module.
771
772 .. versionchanged:: 3.8
773 Added the ``name`` parameter.
774
Andrew Svetlova4888792019-09-12 15:40:40 +0300775 .. deprecated-removed:: 3.8 3.10
776 The *loop* parameter.
777
Chris Jerdonek1ce58412020-05-15 16:55:50 -0700778 .. method:: cancel(msg=None)
Yury Selivanov3faaa882018-09-14 13:32:07 -0700779
780 Request the Task to be cancelled.
781
782 This arranges for a :exc:`CancelledError` exception to be thrown
783 into the wrapped coroutine on the next cycle of the event loop.
784
785 The coroutine then has a chance to clean up or even deny the
786 request by suppressing the exception with a :keyword:`try` ...
787 ... ``except CancelledError`` ... :keyword:`finally` block.
788 Therefore, unlike :meth:`Future.cancel`, :meth:`Task.cancel` does
789 not guarantee that the Task will be cancelled, although
790 suppressing cancellation completely is not common and is actively
791 discouraged.
792
Chris Jerdonek1ce58412020-05-15 16:55:50 -0700793 .. versionchanged:: 3.9
794 Added the ``msg`` parameter.
795
Yury Selivanov7372c3b2018-09-14 15:11:24 -0700796 .. _asyncio_example_task_cancel:
797
Yury Selivanov3faaa882018-09-14 13:32:07 -0700798 The following example illustrates how coroutines can intercept
799 the cancellation request::
800
801 async def cancel_me():
802 print('cancel_me(): before sleep')
803
804 try:
805 # Wait for 1 hour
806 await asyncio.sleep(3600)
807 except asyncio.CancelledError:
808 print('cancel_me(): cancel sleep')
809 raise
810 finally:
811 print('cancel_me(): after sleep')
812
813 async def main():
814 # Create a "cancel_me" Task
815 task = asyncio.create_task(cancel_me())
816
817 # Wait for 1 second
818 await asyncio.sleep(1)
819
820 task.cancel()
821 try:
822 await task
823 except asyncio.CancelledError:
824 print("main(): cancel_me is cancelled now")
825
826 asyncio.run(main())
827
828 # Expected output:
829 #
830 # cancel_me(): before sleep
831 # cancel_me(): cancel sleep
832 # cancel_me(): after sleep
833 # main(): cancel_me is cancelled now
834
835 .. method:: cancelled()
836
837 Return ``True`` if the Task is *cancelled*.
838
839 The Task is *cancelled* when the cancellation was requested with
840 :meth:`cancel` and the wrapped coroutine propagated the
841 :exc:`CancelledError` exception thrown into it.
842
843 .. method:: done()
844
845 Return ``True`` if the Task is *done*.
846
847 A Task is *done* when the wrapped coroutine either returned
848 a value, raised an exception, or the Task was cancelled.
849
Yury Selivanove247b462018-09-20 12:43:59 -0400850 .. method:: result()
851
852 Return the result of the Task.
853
854 If the Task is *done*, the result of the wrapped coroutine
855 is returned (or if the coroutine raised an exception, that
856 exception is re-raised.)
857
858 If the Task has been *cancelled*, this method raises
859 a :exc:`CancelledError` exception.
860
861 If the Task's result isn't yet available, this method raises
862 a :exc:`InvalidStateError` exception.
863
864 .. method:: exception()
865
866 Return the exception of the Task.
867
868 If the wrapped coroutine raised an exception that exception
869 is returned. If the wrapped coroutine returned normally
870 this method returns ``None``.
871
872 If the Task has been *cancelled*, this method raises a
873 :exc:`CancelledError` exception.
874
875 If the Task isn't *done* yet, this method raises an
876 :exc:`InvalidStateError` exception.
877
878 .. method:: add_done_callback(callback, *, context=None)
879
880 Add a callback to be run when the Task is *done*.
881
882 This method should only be used in low-level callback-based code.
883
884 See the documentation of :meth:`Future.add_done_callback`
885 for more details.
886
887 .. method:: remove_done_callback(callback)
888
889 Remove *callback* from the callbacks list.
890
891 This method should only be used in low-level callback-based code.
892
893 See the documentation of :meth:`Future.remove_done_callback`
894 for more details.
895
Andre Delfinodcc997c2020-12-16 22:37:28 -0300896 .. method:: get_stack(*, limit=None)
Yury Selivanov3faaa882018-09-14 13:32:07 -0700897
898 Return the list of stack frames for this Task.
899
900 If the wrapped coroutine is not done, this returns the stack
901 where it is suspended. If the coroutine has completed
902 successfully or was cancelled, this returns an empty list.
903 If the coroutine was terminated by an exception, this returns
904 the list of traceback frames.
905
906 The frames are always ordered from oldest to newest.
907
908 Only one stack frame is returned for a suspended coroutine.
909
910 The optional *limit* argument sets the maximum number of frames
911 to return; by default all available frames are returned.
912 The ordering of the returned list differs depending on whether
913 a stack or a traceback is returned: the newest frames of a
914 stack are returned, but the oldest frames of a traceback are
915 returned. (This matches the behavior of the traceback module.)
916
Andre Delfinodcc997c2020-12-16 22:37:28 -0300917 .. method:: print_stack(*, limit=None, file=None)
Yury Selivanov3faaa882018-09-14 13:32:07 -0700918
919 Print the stack or traceback for this Task.
920
921 This produces output similar to that of the traceback module
922 for the frames retrieved by :meth:`get_stack`.
923
924 The *limit* argument is passed to :meth:`get_stack` directly.
925
926 The *file* argument is an I/O stream to which the output
927 is written; by default output is written to :data:`sys.stderr`.
928
Alex Grönholm98ef9202019-05-30 18:30:09 +0300929 .. method:: get_coro()
930
931 Return the coroutine object wrapped by the :class:`Task`.
932
933 .. versionadded:: 3.8
934
Yury Selivanov3faaa882018-09-14 13:32:07 -0700935 .. method:: get_name()
936
937 Return the name of the Task.
938
939 If no name has been explicitly assigned to the Task, the default
940 asyncio Task implementation generates a default name during
941 instantiation.
942
943 .. versionadded:: 3.8
944
945 .. method:: set_name(value)
946
947 Set the name of the Task.
948
949 The *value* argument can be any object, which is then
950 converted to a string.
951
952 In the default Task implementation, the name will be visible
953 in the :func:`repr` output of a task object.
954
955 .. versionadded:: 3.8
956
Yury Selivanov3faaa882018-09-14 13:32:07 -0700957
958.. _asyncio_generator_based_coro:
959
960Generator-based Coroutines
961==========================
962
963.. note::
964
965 Support for generator-based coroutines is **deprecated** and
Yury Selivanovfad6af22018-09-25 17:44:52 -0400966 is scheduled for removal in Python 3.10.
Yury Selivanov3faaa882018-09-14 13:32:07 -0700967
968Generator-based coroutines predate async/await syntax. They are
Elvis Pranskevichus1fa2ec42018-09-17 19:16:44 -0400969Python generators that use ``yield from`` expressions to await
Yury Selivanov3faaa882018-09-14 13:32:07 -0700970on Futures and other coroutines.
971
972Generator-based coroutines should be decorated with
973:func:`@asyncio.coroutine <asyncio.coroutine>`, although this is not
974enforced.
975
976
977.. decorator:: coroutine
978
979 Decorator to mark generator-based coroutines.
980
981 This decorator enables legacy generator-based coroutines to be
982 compatible with async/await code::
983
984 @asyncio.coroutine
985 def old_style_coroutine():
986 yield from asyncio.sleep(1)
987
988 async def main():
989 await old_style_coroutine()
990
Yury Selivanov3faaa882018-09-14 13:32:07 -0700991 This decorator should not be used for :keyword:`async def`
992 coroutines.
993
Andrew Svetlov68b34a72019-05-16 17:52:10 +0300994 .. deprecated-removed:: 3.8 3.10
995
996 Use :keyword:`async def` instead.
997
Yury Selivanov3faaa882018-09-14 13:32:07 -0700998.. function:: iscoroutine(obj)
999
1000 Return ``True`` if *obj* is a :ref:`coroutine object <coroutine>`.
1001
1002 This method is different from :func:`inspect.iscoroutine` because
Yury Selivanov59ee5b12018-09-27 15:48:30 -04001003 it returns ``True`` for generator-based coroutines.
Yury Selivanov3faaa882018-09-14 13:32:07 -07001004
1005.. function:: iscoroutinefunction(func)
1006
1007 Return ``True`` if *func* is a :ref:`coroutine function
1008 <coroutine>`.
1009
1010 This method is different from :func:`inspect.iscoroutinefunction`
1011 because it returns ``True`` for generator-based coroutine functions
1012 decorated with :func:`@coroutine <coroutine>`.