blob: 21824ca537f77fc983b2564d39ccca572d7efb6e [file] [log] [blame]
R David Murray6a143812013-12-20 14:37:39 -05001.. currentmodule:: asyncio
Victor Stinnerea3183f2013-12-03 01:08:00 +01002
Yury Selivanov3faaa882018-09-14 13:32:07 -07003
4====================
5Coroutines and Tasks
Victor Stinnerea3183f2013-12-03 01:08:00 +01006====================
7
Yury Selivanov3faaa882018-09-14 13:32:07 -07008This section outlines high-level asyncio APIs to work with coroutines
9and Tasks.
lf627d2c82017-07-25 17:03:51 -060010
Yury Selivanov3faaa882018-09-14 13:32:07 -070011.. contents::
12 :depth: 1
13 :local:
14
lf627d2c82017-07-25 17:03:51 -060015
Victor Stinnerea3183f2013-12-03 01:08:00 +010016.. _coroutine:
17
18Coroutines
Yury Selivanov3faaa882018-09-14 13:32:07 -070019==========
Victor Stinnerea3183f2013-12-03 01:08:00 +010020
Kyle Stanleyf9000642019-10-10 19:18:46 -040021:term:`Coroutines <coroutine>` declared with the async/await syntax is the
22preferred way of writing asyncio applications. For example, the following
23snippet of code (requires Python 3.7+) prints "hello", waits 1 second,
Yury Selivanovb042cf12018-09-18 02:47:54 -040024and then prints "world"::
Victor Stinnerea3183f2013-12-03 01:08:00 +010025
Yury Selivanov3faaa882018-09-14 13:32:07 -070026 >>> import asyncio
Victor Stinnerea3183f2013-12-03 01:08:00 +010027
Yury Selivanov3faaa882018-09-14 13:32:07 -070028 >>> async def main():
29 ... print('hello')
30 ... await asyncio.sleep(1)
31 ... print('world')
Victor Stinnerea3183f2013-12-03 01:08:00 +010032
Yury Selivanov3faaa882018-09-14 13:32:07 -070033 >>> asyncio.run(main())
34 hello
35 world
Victor Stinnerea3183f2013-12-03 01:08:00 +010036
Yury Selivanov3faaa882018-09-14 13:32:07 -070037Note that simply calling a coroutine will not schedule it to
38be executed::
Victor Stinnerea3183f2013-12-03 01:08:00 +010039
Yury Selivanov3faaa882018-09-14 13:32:07 -070040 >>> main()
41 <coroutine object main at 0x1053bb7c8>
Victor Stinnerea3183f2013-12-03 01:08:00 +010042
Boštjan Mejak1d5bdef2019-05-19 11:01:36 +020043To actually run a coroutine, asyncio provides three main mechanisms:
Victor Stinnerea3183f2013-12-03 01:08:00 +010044
Elvis Pranskevichus1fa2ec42018-09-17 19:16:44 -040045* The :func:`asyncio.run` function to run the top-level
Yury Selivanov3faaa882018-09-14 13:32:07 -070046 entry point "main()" function (see the above example.)
Victor Stinnerea3183f2013-12-03 01:08:00 +010047
Elvis Pranskevichus1fa2ec42018-09-17 19:16:44 -040048* Awaiting on a coroutine. The following snippet of code will
Yury Selivanov3faaa882018-09-14 13:32:07 -070049 print "hello" after waiting for 1 second, and then print "world"
50 after waiting for *another* 2 seconds::
Victor Stinnerea3183f2013-12-03 01:08:00 +010051
Yury Selivanov3faaa882018-09-14 13:32:07 -070052 import asyncio
53 import time
Victor Stinnerea3183f2013-12-03 01:08:00 +010054
Yury Selivanov3faaa882018-09-14 13:32:07 -070055 async def say_after(delay, what):
56 await asyncio.sleep(delay)
57 print(what)
58
59 async def main():
Mariatta9f43fbb2018-10-24 15:37:12 -070060 print(f"started at {time.strftime('%X')}")
Yury Selivanov3faaa882018-09-14 13:32:07 -070061
62 await say_after(1, 'hello')
63 await say_after(2, 'world')
64
Mariatta9f43fbb2018-10-24 15:37:12 -070065 print(f"finished at {time.strftime('%X')}")
Yury Selivanov3faaa882018-09-14 13:32:07 -070066
67 asyncio.run(main())
68
69 Expected output::
70
71 started at 17:13:52
72 hello
73 world
74 finished at 17:13:55
75
Elvis Pranskevichus1fa2ec42018-09-17 19:16:44 -040076* The :func:`asyncio.create_task` function to run coroutines
Yury Selivanov3faaa882018-09-14 13:32:07 -070077 concurrently as asyncio :class:`Tasks <Task>`.
78
Danny Hermes7bfbda42018-09-17 21:49:21 -070079 Let's modify the above example and run two ``say_after`` coroutines
Yury Selivanov3faaa882018-09-14 13:32:07 -070080 *concurrently*::
81
82 async def main():
83 task1 = asyncio.create_task(
84 say_after(1, 'hello'))
85
86 task2 = asyncio.create_task(
87 say_after(2, 'world'))
88
Mariatta9f43fbb2018-10-24 15:37:12 -070089 print(f"started at {time.strftime('%X')}")
Yury Selivanov3faaa882018-09-14 13:32:07 -070090
91 # Wait until both tasks are completed (should take
92 # around 2 seconds.)
93 await task1
94 await task2
95
Mariatta9f43fbb2018-10-24 15:37:12 -070096 print(f"finished at {time.strftime('%X')}")
Yury Selivanov3faaa882018-09-14 13:32:07 -070097
98 Note that expected output now shows that the snippet runs
99 1 second faster than before::
100
101 started at 17:14:32
102 hello
103 world
104 finished at 17:14:34
105
Yury Selivanov47150392018-09-18 17:55:44 -0400106
107.. _asyncio-awaitables:
108
109Awaitables
110==========
111
Yury Selivanove247b462018-09-20 12:43:59 -0400112We say that an object is an **awaitable** object if it can be used
113in an :keyword:`await` expression. Many asyncio APIs are designed to
114accept awaitables.
115
116There are three main types of *awaitable* objects:
117**coroutines**, **Tasks**, and **Futures**.
Yury Selivanov47150392018-09-18 17:55:44 -0400118
119
Yury Selivanove247b462018-09-20 12:43:59 -0400120.. rubric:: Coroutines
Yury Selivanov47150392018-09-18 17:55:44 -0400121
Yury Selivanove247b462018-09-20 12:43:59 -0400122Python coroutines are *awaitables* and therefore can be awaited from
123other coroutines::
124
125 import asyncio
Yury Selivanov47150392018-09-18 17:55:44 -0400126
127 async def nested():
128 return 42
129
130 async def main():
Yury Selivanove247b462018-09-20 12:43:59 -0400131 # Nothing happens if we just call "nested()".
Yury Selivanovdb1a80e2018-09-21 16:23:15 -0400132 # A coroutine object is created but not awaited,
133 # so it *won't run at all*.
Yury Selivanove247b462018-09-20 12:43:59 -0400134 nested()
135
136 # Let's do it differently now and await it:
137 print(await nested()) # will print "42".
138
139 asyncio.run(main())
140
141.. important::
142
143 In this documentation the term "coroutine" can be used for
144 two closely related concepts:
145
146 * a *coroutine function*: an :keyword:`async def` function;
147
148 * a *coroutine object*: an object returned by calling a
149 *coroutine function*.
150
151asyncio also supports legacy :ref:`generator-based
152<asyncio_generator_based_coro>` coroutines.
153
154
155.. rubric:: Tasks
Yury Selivanov47150392018-09-18 17:55:44 -0400156
157*Tasks* are used to schedule coroutines *concurrently*.
Yury Selivanov47150392018-09-18 17:55:44 -0400158
Yury Selivanove247b462018-09-20 12:43:59 -0400159When a coroutine is wrapped into a *Task* with functions like
160:func:`asyncio.create_task` the coroutine is automatically
161scheduled to run soon::
Yury Selivanov3faaa882018-09-14 13:32:07 -0700162
Yury Selivanove247b462018-09-20 12:43:59 -0400163 import asyncio
Yury Selivanov3faaa882018-09-14 13:32:07 -0700164
Yury Selivanove247b462018-09-20 12:43:59 -0400165 async def nested():
166 return 42
167
168 async def main():
169 # Schedule nested() to run soon concurrently
170 # with "main()".
171 task = asyncio.create_task(nested())
172
173 # "task" can now be used to cancel "nested()", or
174 # can simply be awaited to wait until it is complete:
175 await task
176
177 asyncio.run(main())
Victor Stinner337e03f2014-08-11 01:11:13 +0200178
Victor Stinnerea3183f2013-12-03 01:08:00 +0100179
Yury Selivanov47150392018-09-18 17:55:44 -0400180.. rubric:: Futures
181
Yury Selivanove247b462018-09-20 12:43:59 -0400182A :class:`Future` is a special **low-level** awaitable object that
183represents an **eventual result** of an asynchronous operation.
Yury Selivanov47150392018-09-18 17:55:44 -0400184
Yury Selivanove247b462018-09-20 12:43:59 -0400185When a Future object is *awaited* it means that the coroutine will
186wait until the Future is resolved in some other place.
187
Yury Selivanov47150392018-09-18 17:55:44 -0400188Future objects in asyncio are needed to allow callback-based code
189to be used with async/await.
190
Yury Selivanove247b462018-09-20 12:43:59 -0400191Normally **there is no need** to create Future objects at the
Yury Selivanov47150392018-09-18 17:55:44 -0400192application level code.
193
194Future objects, sometimes exposed by libraries and some asyncio
Yury Selivanove247b462018-09-20 12:43:59 -0400195APIs, can be awaited::
Yury Selivanov47150392018-09-18 17:55:44 -0400196
197 async def main():
198 await function_that_returns_a_future_object()
199
200 # this is also valid:
201 await asyncio.gather(
202 function_that_returns_a_future_object(),
203 some_python_coroutine()
204 )
205
Yury Selivanove247b462018-09-20 12:43:59 -0400206A good example of a low-level function that returns a Future object
207is :meth:`loop.run_in_executor`.
208
Yury Selivanov47150392018-09-18 17:55:44 -0400209
Yury Selivanov3faaa882018-09-14 13:32:07 -0700210Running an asyncio Program
211==========================
Victor Stinnerea3183f2013-12-03 01:08:00 +0100212
Elvis Pranskevichus63536bd2018-05-19 23:15:06 -0400213.. function:: run(coro, \*, debug=False)
Yury Selivanov02a0a192017-12-14 09:42:21 -0500214
Kyle Stanleye4070132019-09-30 20:12:21 -0400215 Execute the :term:`coroutine` *coro* and return the result.
216
Yury Selivanov02a0a192017-12-14 09:42:21 -0500217 This function runs the passed coroutine, taking care of
Kyle Stanley9fdc64c2019-09-19 08:47:22 -0400218 managing the asyncio event loop, *finalizing asynchronous
219 generators*, and closing the threadpool.
Yury Selivanov02a0a192017-12-14 09:42:21 -0500220
221 This function cannot be called when another asyncio event loop is
222 running in the same thread.
223
Yury Selivanov3faaa882018-09-14 13:32:07 -0700224 If *debug* is ``True``, the event loop will be run in debug mode.
Yury Selivanov02a0a192017-12-14 09:42:21 -0500225
226 This function always creates a new event loop and closes it at
227 the end. It should be used as a main entry point for asyncio
228 programs, and should ideally only be called once.
229
Emmanuel Arias17deb162019-09-25 05:53:49 -0300230 Example::
231
232 async def main():
233 await asyncio.sleep(1)
234 print('hello')
235
236 asyncio.run(main())
237
Yury Selivanov02a0a192017-12-14 09:42:21 -0500238 .. versionadded:: 3.7
239
Kyle Stanley9fdc64c2019-09-19 08:47:22 -0400240 .. versionchanged:: 3.9
241 Updated to use :meth:`loop.shutdown_default_executor`.
Yury Selivanov02a0a192017-12-14 09:42:21 -0500242
Kyle Stanleyf9000642019-10-10 19:18:46 -0400243 .. note::
244 The source code for ``asyncio.run()`` can be found in
245 :source:`Lib/asyncio/runners.py`.
246
Yury Selivanov3faaa882018-09-14 13:32:07 -0700247Creating Tasks
248==============
Victor Stinnerea3183f2013-12-03 01:08:00 +0100249
Alex Grönholmcca4eec2018-08-09 00:06:47 +0300250.. function:: create_task(coro, \*, name=None)
Andrew Svetlovf74ef452017-12-15 07:04:38 +0200251
Yury Selivanove247b462018-09-20 12:43:59 -0400252 Wrap the *coro* :ref:`coroutine <coroutine>` into a :class:`Task`
253 and schedule its execution. Return the Task object.
Alex Grönholmcca4eec2018-08-09 00:06:47 +0300254
255 If *name* is not ``None``, it is set as the name of the task using
256 :meth:`Task.set_name`.
Andrew Svetlovf74ef452017-12-15 07:04:38 +0200257
Elvis Pranskevichus1fa2ec42018-09-17 19:16:44 -0400258 The task is executed in the loop returned by :func:`get_running_loop`,
Andrew Svetlovf74ef452017-12-15 07:04:38 +0200259 :exc:`RuntimeError` is raised if there is no running loop in
260 current thread.
261
Yury Selivanov47150392018-09-18 17:55:44 -0400262 This function has been **added in Python 3.7**. Prior to
263 Python 3.7, the low-level :func:`asyncio.ensure_future` function
264 can be used instead::
265
266 async def coro():
267 ...
268
269 # In Python 3.7+
270 task = asyncio.create_task(coro())
271 ...
272
273 # This works in all Python versions but is less readable
274 task = asyncio.ensure_future(coro())
275 ...
276
Andrew Svetlovf74ef452017-12-15 07:04:38 +0200277 .. versionadded:: 3.7
278
Alex Grönholmcca4eec2018-08-09 00:06:47 +0300279 .. versionchanged:: 3.8
280 Added the ``name`` parameter.
281
Victor Stinnerea3183f2013-12-03 01:08:00 +0100282
Yury Selivanov3faaa882018-09-14 13:32:07 -0700283Sleeping
284========
Andrew Svetlovf1240162016-01-11 14:40:35 +0200285
Victor Stinnerbdd574d2015-02-12 22:49:18 +0100286.. coroutinefunction:: sleep(delay, result=None, \*, loop=None)
Victor Stinnerea3183f2013-12-03 01:08:00 +0100287
Yury Selivanov3faaa882018-09-14 13:32:07 -0700288 Block for *delay* seconds.
289
290 If *result* is provided, it is returned to the caller
Eli Bendersky2d26af82014-01-20 06:59:23 -0800291 when the coroutine completes.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100292
Hrvoje Nikšićcd602b82018-10-01 12:09:38 +0200293 ``sleep()`` always suspends the current task, allowing other tasks
294 to run.
295
Matthias Bussonnierd0ebf132019-05-20 23:20:10 -0700296 .. deprecated-removed:: 3.8 3.10
297 The *loop* parameter.
Yury Selivanov47150392018-09-18 17:55:44 -0400298
Yury Selivanov7372c3b2018-09-14 15:11:24 -0700299 .. _asyncio_example_sleep:
Victor Stinner45b27ed2014-02-01 02:36:43 +0100300
Yury Selivanov3faaa882018-09-14 13:32:07 -0700301 Example of coroutine displaying the current date every second
Elvis Pranskevichus1fa2ec42018-09-17 19:16:44 -0400302 for 5 seconds::
Victor Stinnerbdd574d2015-02-12 22:49:18 +0100303
Yury Selivanov3faaa882018-09-14 13:32:07 -0700304 import asyncio
305 import datetime
Victor Stinnerea3183f2013-12-03 01:08:00 +0100306
Yury Selivanov3faaa882018-09-14 13:32:07 -0700307 async def display_date():
308 loop = asyncio.get_running_loop()
309 end_time = loop.time() + 5.0
310 while True:
311 print(datetime.datetime.now())
312 if (loop.time() + 1.0) >= end_time:
313 break
314 await asyncio.sleep(1)
315
316 asyncio.run(display_date())
317
318
319Running Tasks Concurrently
320==========================
321
Yury Selivanove247b462018-09-20 12:43:59 -0400322.. awaitablefunction:: gather(\*aws, loop=None, return_exceptions=False)
Yury Selivanov3faaa882018-09-14 13:32:07 -0700323
Yury Selivanove247b462018-09-20 12:43:59 -0400324 Run :ref:`awaitable objects <asyncio-awaitables>` in the *aws*
Yury Selivanov47150392018-09-18 17:55:44 -0400325 sequence *concurrently*.
Yury Selivanov3faaa882018-09-14 13:32:07 -0700326
Yury Selivanove247b462018-09-20 12:43:59 -0400327 If any awaitable in *aws* is a coroutine, it is automatically
Yury Selivanov47150392018-09-18 17:55:44 -0400328 scheduled as a Task.
Yury Selivanov3faaa882018-09-14 13:32:07 -0700329
Yury Selivanov47150392018-09-18 17:55:44 -0400330 If all awaitables are completed successfully, the result is an
331 aggregate list of returned values. The order of result values
Yury Selivanove247b462018-09-20 12:43:59 -0400332 corresponds to the order of awaitables in *aws*.
Yury Selivanov3faaa882018-09-14 13:32:07 -0700333
Yury Selivanovdb1a80e2018-09-21 16:23:15 -0400334 If *return_exceptions* is ``False`` (default), the first
335 raised exception is immediately propagated to the task that
336 awaits on ``gather()``. Other awaitables in the *aws* sequence
337 **won't be cancelled** and will continue to run.
338
Yury Selivanov47150392018-09-18 17:55:44 -0400339 If *return_exceptions* is ``True``, exceptions are treated the
340 same as successful results, and aggregated in the result list.
Yury Selivanov3faaa882018-09-14 13:32:07 -0700341
Yury Selivanovdb1a80e2018-09-21 16:23:15 -0400342 If ``gather()`` is *cancelled*, all submitted awaitables
Yury Selivanov3faaa882018-09-14 13:32:07 -0700343 (that have not completed yet) are also *cancelled*.
344
Yury Selivanove247b462018-09-20 12:43:59 -0400345 If any Task or Future from the *aws* sequence is *cancelled*, it is
Yury Selivanov47150392018-09-18 17:55:44 -0400346 treated as if it raised :exc:`CancelledError` -- the ``gather()``
347 call is **not** cancelled in this case. This is to prevent the
348 cancellation of one submitted Task/Future to cause other
349 Tasks/Futures to be cancelled.
Yury Selivanov3faaa882018-09-14 13:32:07 -0700350
Andrew Svetlova4888792019-09-12 15:40:40 +0300351 .. deprecated-removed:: 3.8 3.10
352 The *loop* parameter.
353
Yury Selivanov7372c3b2018-09-14 15:11:24 -0700354 .. _asyncio_example_gather:
355
Yury Selivanov3faaa882018-09-14 13:32:07 -0700356 Example::
357
358 import asyncio
359
360 async def factorial(name, number):
361 f = 1
362 for i in range(2, number + 1):
363 print(f"Task {name}: Compute factorial({i})...")
364 await asyncio.sleep(1)
365 f *= i
366 print(f"Task {name}: factorial({number}) = {f}")
367
368 async def main():
Yury Selivanov47150392018-09-18 17:55:44 -0400369 # Schedule three calls *concurrently*:
Yury Selivanov3faaa882018-09-14 13:32:07 -0700370 await asyncio.gather(
371 factorial("A", 2),
372 factorial("B", 3),
373 factorial("C", 4),
Miguel Ángel García9c53fa62018-09-18 08:01:26 +0200374 )
Yury Selivanov3faaa882018-09-14 13:32:07 -0700375
376 asyncio.run(main())
377
378 # Expected output:
379 #
380 # Task A: Compute factorial(2)...
381 # Task B: Compute factorial(2)...
382 # Task C: Compute factorial(2)...
383 # Task A: factorial(2) = 2
384 # Task B: Compute factorial(3)...
385 # Task C: Compute factorial(3)...
386 # Task B: factorial(3) = 6
387 # Task C: Compute factorial(4)...
388 # Task C: factorial(4) = 24
389
Yury Selivanov47150392018-09-18 17:55:44 -0400390 .. versionchanged:: 3.7
391 If the *gather* itself is cancelled, the cancellation is
392 propagated regardless of *return_exceptions*.
393
Yury Selivanov3faaa882018-09-14 13:32:07 -0700394
Yury Selivanovdb1a80e2018-09-21 16:23:15 -0400395Shielding From Cancellation
396===========================
Yury Selivanov3faaa882018-09-14 13:32:07 -0700397
Yury Selivanove247b462018-09-20 12:43:59 -0400398.. awaitablefunction:: shield(aw, \*, loop=None)
Yury Selivanov3faaa882018-09-14 13:32:07 -0700399
Yury Selivanov47150392018-09-18 17:55:44 -0400400 Protect an :ref:`awaitable object <asyncio-awaitables>`
401 from being :meth:`cancelled <Task.cancel>`.
Yury Selivanov3faaa882018-09-14 13:32:07 -0700402
Yury Selivanovdb1a80e2018-09-21 16:23:15 -0400403 If *aw* is a coroutine it is automatically scheduled as a Task.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100404
405 The statement::
406
Andrew Svetlov88743422017-12-11 17:35:49 +0200407 res = await shield(something())
Victor Stinnerea3183f2013-12-03 01:08:00 +0100408
Elvis Pranskevichus1fa2ec42018-09-17 19:16:44 -0400409 is equivalent to::
Victor Stinnerea3183f2013-12-03 01:08:00 +0100410
Andrew Svetlov88743422017-12-11 17:35:49 +0200411 res = await something()
Victor Stinnerea3183f2013-12-03 01:08:00 +0100412
Yury Selivanov3faaa882018-09-14 13:32:07 -0700413 *except* that if the coroutine containing it is cancelled, the
414 Task running in ``something()`` is not cancelled. From the point
415 of view of ``something()``, the cancellation did not happen.
416 Although its caller is still cancelled, so the "await" expression
Elvis Pranskevichus1fa2ec42018-09-17 19:16:44 -0400417 still raises a :exc:`CancelledError`.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100418
Yury Selivanov3faaa882018-09-14 13:32:07 -0700419 If ``something()`` is cancelled by other means (i.e. from within
420 itself) that would also cancel ``shield()``.
421
422 If it is desired to completely ignore cancellation (not recommended)
423 the ``shield()`` function should be combined with a try/except
424 clause, as follows::
Victor Stinnerea3183f2013-12-03 01:08:00 +0100425
426 try:
Andrew Svetlov88743422017-12-11 17:35:49 +0200427 res = await shield(something())
Victor Stinnerea3183f2013-12-03 01:08:00 +0100428 except CancelledError:
429 res = None
430
Andrew Svetlova4888792019-09-12 15:40:40 +0300431 .. deprecated-removed:: 3.8 3.10
432 The *loop* parameter.
433
Yury Selivanov950204d2016-05-16 16:23:00 -0400434
Yury Selivanov3faaa882018-09-14 13:32:07 -0700435Timeouts
436========
437
Yury Selivanove247b462018-09-20 12:43:59 -0400438.. coroutinefunction:: wait_for(aw, timeout, \*, loop=None)
Yury Selivanov3faaa882018-09-14 13:32:07 -0700439
Yury Selivanove247b462018-09-20 12:43:59 -0400440 Wait for the *aw* :ref:`awaitable <asyncio-awaitables>`
Yury Selivanov47150392018-09-18 17:55:44 -0400441 to complete with a timeout.
Yury Selivanov3faaa882018-09-14 13:32:07 -0700442
Yury Selivanove247b462018-09-20 12:43:59 -0400443 If *aw* is a coroutine it is automatically scheduled as a Task.
Yury Selivanov3faaa882018-09-14 13:32:07 -0700444
445 *timeout* can either be ``None`` or a float or int number of seconds
446 to wait for. If *timeout* is ``None``, block until the future
447 completes.
448
449 If a timeout occurs, it cancels the task and raises
450 :exc:`asyncio.TimeoutError`.
451
Yury Selivanov47150392018-09-18 17:55:44 -0400452 To avoid the task :meth:`cancellation <Task.cancel>`,
453 wrap it in :func:`shield`.
Yury Selivanov3faaa882018-09-14 13:32:07 -0700454
455 The function will wait until the future is actually cancelled,
romasku382a5632020-05-15 23:12:05 +0300456 so the total wait time may exceed the *timeout*. If an exception
457 happens during cancellation, it is propagated.
Yury Selivanov3faaa882018-09-14 13:32:07 -0700458
Yury Selivanove247b462018-09-20 12:43:59 -0400459 If the wait is cancelled, the future *aw* is also cancelled.
Yury Selivanov3faaa882018-09-14 13:32:07 -0700460
Matthias Bussonnierd0ebf132019-05-20 23:20:10 -0700461 .. deprecated-removed:: 3.8 3.10
462 The *loop* parameter.
Yury Selivanov47150392018-09-18 17:55:44 -0400463
Yury Selivanov7372c3b2018-09-14 15:11:24 -0700464 .. _asyncio_example_waitfor:
465
Yury Selivanov3faaa882018-09-14 13:32:07 -0700466 Example::
467
468 async def eternity():
469 # Sleep for one hour
470 await asyncio.sleep(3600)
471 print('yay!')
472
473 async def main():
474 # Wait for at most 1 second
475 try:
476 await asyncio.wait_for(eternity(), timeout=1.0)
477 except asyncio.TimeoutError:
478 print('timeout!')
479
480 asyncio.run(main())
481
482 # Expected output:
483 #
484 # timeout!
485
486 .. versionchanged:: 3.7
Yury Selivanove247b462018-09-20 12:43:59 -0400487 When *aw* is cancelled due to a timeout, ``wait_for`` waits
488 for *aw* to be cancelled. Previously, it raised
Yury Selivanov3faaa882018-09-14 13:32:07 -0700489 :exc:`asyncio.TimeoutError` immediately.
490
491
492Waiting Primitives
493==================
494
Yury Selivanove247b462018-09-20 12:43:59 -0400495.. coroutinefunction:: wait(aws, \*, loop=None, timeout=None,\
Andrew Svetlovf1240162016-01-11 14:40:35 +0200496 return_when=ALL_COMPLETED)
Victor Stinnerea3183f2013-12-03 01:08:00 +0100497
Yury Selivanove247b462018-09-20 12:43:59 -0400498 Run :ref:`awaitable objects <asyncio-awaitables>` in the *aws*
Yury Selivanov996859a2018-09-25 14:51:21 -0400499 set concurrently and block until the condition specified
Yury Selivanov47150392018-09-18 17:55:44 -0400500 by *return_when*.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100501
Joel Rosdahl9d746582020-05-04 23:56:00 +0200502 The *aws* set must not be empty.
503
Yury Selivanov3faaa882018-09-14 13:32:07 -0700504 Returns two sets of Tasks/Futures: ``(done, pending)``.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100505
Yury Selivanov996859a2018-09-25 14:51:21 -0400506 Usage::
507
508 done, pending = await asyncio.wait(aws)
509
Yury Selivanov3faaa882018-09-14 13:32:07 -0700510 *timeout* (a float or int), if specified, can be used to control
511 the maximum number of seconds to wait before returning.
512
513 Note that this function does not raise :exc:`asyncio.TimeoutError`.
Elvis Pranskevichus1fa2ec42018-09-17 19:16:44 -0400514 Futures or Tasks that aren't done when the timeout occurs are simply
Yury Selivanov3faaa882018-09-14 13:32:07 -0700515 returned in the second set.
516
517 *return_when* indicates when this function should return. It must
518 be one of the following constants:
Victor Stinnerea3183f2013-12-03 01:08:00 +0100519
520 .. tabularcolumns:: |l|L|
521
522 +-----------------------------+----------------------------------------+
523 | Constant | Description |
524 +=============================+========================================+
525 | :const:`FIRST_COMPLETED` | The function will return when any |
526 | | future finishes or is cancelled. |
527 +-----------------------------+----------------------------------------+
528 | :const:`FIRST_EXCEPTION` | The function will return when any |
529 | | future finishes by raising an |
530 | | exception. If no future raises an |
531 | | exception then it is equivalent to |
532 | | :const:`ALL_COMPLETED`. |
533 +-----------------------------+----------------------------------------+
534 | :const:`ALL_COMPLETED` | The function will return when all |
535 | | futures finish or are cancelled. |
536 +-----------------------------+----------------------------------------+
537
Yury Selivanov3faaa882018-09-14 13:32:07 -0700538 Unlike :func:`~asyncio.wait_for`, ``wait()`` does not cancel the
539 futures when a timeout occurs.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100540
Andrew Svetlova4888792019-09-12 15:40:40 +0300541 .. deprecated:: 3.8
542
543 If any awaitable in *aws* is a coroutine, it is automatically
544 scheduled as a Task. Passing coroutines objects to
545 ``wait()`` directly is deprecated as it leads to
546 :ref:`confusing behavior <asyncio_example_wait_coroutine>`.
547
548 .. deprecated-removed:: 3.8 3.10
549
550 The *loop* parameter.
551
Yury Selivanov996859a2018-09-25 14:51:21 -0400552 .. _asyncio_example_wait_coroutine:
553 .. note::
Victor Stinnerea3183f2013-12-03 01:08:00 +0100554
Yury Selivanov996859a2018-09-25 14:51:21 -0400555 ``wait()`` schedules coroutines as Tasks automatically and later
556 returns those implicitly created Task objects in ``(done, pending)``
557 sets. Therefore the following code won't work as expected::
558
559 async def foo():
560 return 42
561
562 coro = foo()
563 done, pending = await asyncio.wait({coro})
564
565 if coro in done:
566 # This branch will never be run!
567
568 Here is how the above snippet can be fixed::
569
570 async def foo():
571 return 42
572
573 task = asyncio.create_task(foo())
574 done, pending = await asyncio.wait({task})
575
576 if task in done:
577 # Everything will work as expected now.
578
jack1142de927692020-05-13 20:55:12 +0200579 .. deprecated-removed:: 3.8 3.11
Matthias Bussonnierd0ebf132019-05-20 23:20:10 -0700580
Yury Selivanov996859a2018-09-25 14:51:21 -0400581 Passing coroutine objects to ``wait()`` directly is
582 deprecated.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100583
Victor Stinnerea3183f2013-12-03 01:08:00 +0100584
Yury Selivanove247b462018-09-20 12:43:59 -0400585.. function:: as_completed(aws, \*, loop=None, timeout=None)
Yury Selivanov3faaa882018-09-14 13:32:07 -0700586
Yury Selivanove247b462018-09-20 12:43:59 -0400587 Run :ref:`awaitable objects <asyncio-awaitables>` in the *aws*
Miss Islington (bot)9181e2e2020-05-23 16:23:53 -0700588 set concurrently. Return an iterator of coroutines.
589 Each coroutine returned can be awaited to get the earliest next
590 result from the set of the remaining awaitables.
Yury Selivanov3faaa882018-09-14 13:32:07 -0700591
592 Raises :exc:`asyncio.TimeoutError` if the timeout occurs before
593 all Futures are done.
594
Andrew Svetlova4888792019-09-12 15:40:40 +0300595 .. deprecated-removed:: 3.8 3.10
596 The *loop* parameter.
597
Yury Selivanov3faaa882018-09-14 13:32:07 -0700598 Example::
599
Miss Islington (bot)9181e2e2020-05-23 16:23:53 -0700600 for coro in as_completed(aws):
601 earliest_result = await coro
Yury Selivanov3faaa882018-09-14 13:32:07 -0700602 # ...
Victor Stinnerea3183f2013-12-03 01:08:00 +0100603
Victor Stinner3e09e322013-12-03 01:22:06 +0100604
Miss Islington (bot)e2991302020-05-19 03:03:25 -0700605Running in Threads
606==================
607
608.. coroutinefunction:: to_thread(func, /, \*args, \*\*kwargs)
609
610 Asynchronously run function *func* in a separate thread.
611
612 Any \*args and \*\*kwargs supplied for this function are directly passed
Miss Islington (bot)3e650542020-05-20 22:38:00 -0700613 to *func*. Also, the current :class:`contextvars.Context` is propogated,
614 allowing context variables from the event loop thread to be accessed in the
615 separate thread.
Miss Islington (bot)e2991302020-05-19 03:03:25 -0700616
Miss Islington (bot)cdb015b2020-05-31 00:26:20 -0700617 Return a coroutine that can be awaited to get the eventual result of *func*.
Miss Islington (bot)e2991302020-05-19 03:03:25 -0700618
619 This coroutine function is primarily intended to be used for executing
620 IO-bound functions/methods that would otherwise block the event loop if
621 they were ran in the main thread. For example::
622
623 def blocking_io():
624 print(f"start blocking_io at {time.strftime('%X')}")
625 # Note that time.sleep() can be replaced with any blocking
626 # IO-bound operation, such as file operations.
627 time.sleep(1)
628 print(f"blocking_io complete at {time.strftime('%X')}")
629
630 async def main():
631 print(f"started main at {time.strftime('%X')}")
632
633 await asyncio.gather(
634 asyncio.to_thread(blocking_io),
635 asyncio.sleep(1))
636
637 print(f"finished main at {time.strftime('%X')}")
638
639
640 asyncio.run(main())
641
642 # Expected output:
643 #
644 # started main at 19:50:53
645 # start blocking_io at 19:50:53
646 # blocking_io complete at 19:50:54
647 # finished main at 19:50:54
648
649 Directly calling `blocking_io()` in any coroutine would block the event loop
650 for its duration, resulting in an additional 1 second of run time. Instead,
651 by using `asyncio.to_thread()`, we can run it in a separate thread without
652 blocking the event loop.
653
654 .. note::
655
656 Due to the :term:`GIL`, `asyncio.to_thread()` can typically only be used
657 to make IO-bound functions non-blocking. However, for extension modules
658 that release the GIL or alternative Python implementations that don't
659 have one, `asyncio.to_thread()` can also be used for CPU-bound functions.
660
Miss Islington (bot)3e650542020-05-20 22:38:00 -0700661 .. versionadded:: 3.9
662
Miss Islington (bot)e2991302020-05-19 03:03:25 -0700663
Yury Selivanov3faaa882018-09-14 13:32:07 -0700664Scheduling From Other Threads
665=============================
Victor Stinner1ad5afc2014-01-30 00:18:50 +0100666
Yury Selivanov3faaa882018-09-14 13:32:07 -0700667.. function:: run_coroutine_threadsafe(coro, loop)
Victor Stinner1ad5afc2014-01-30 00:18:50 +0100668
Yury Selivanov3faaa882018-09-14 13:32:07 -0700669 Submit a coroutine to the given event loop. Thread-safe.
Victor Stinner1ad5afc2014-01-30 00:18:50 +0100670
Yury Selivanov47150392018-09-18 17:55:44 -0400671 Return a :class:`concurrent.futures.Future` to wait for the result
672 from another OS thread.
Victor Stinner1ad5afc2014-01-30 00:18:50 +0100673
Yury Selivanov3faaa882018-09-14 13:32:07 -0700674 This function is meant to be called from a different OS thread
675 than the one where the event loop is running. Example::
Victor Stinner72dcb0a2015-04-03 17:08:19 +0200676
Yury Selivanov3faaa882018-09-14 13:32:07 -0700677 # Create a coroutine
678 coro = asyncio.sleep(1, result=3)
Yury Selivanov37f15bc2014-02-20 16:20:44 -0500679
Yury Selivanov3faaa882018-09-14 13:32:07 -0700680 # Submit the coroutine to a given loop
681 future = asyncio.run_coroutine_threadsafe(coro, loop)
Victor Stinner1ad5afc2014-01-30 00:18:50 +0100682
Yury Selivanov3faaa882018-09-14 13:32:07 -0700683 # Wait for the result with an optional timeout argument
684 assert future.result(timeout) == 3
685
686 If an exception is raised in the coroutine, the returned Future
687 will be notified. It can also be used to cancel the task in
688 the event loop::
689
690 try:
691 result = future.result(timeout)
692 except asyncio.TimeoutError:
693 print('The coroutine took too long, cancelling the task...')
694 future.cancel()
695 except Exception as exc:
Mariatta9f43fbb2018-10-24 15:37:12 -0700696 print(f'The coroutine raised an exception: {exc!r}')
Yury Selivanov3faaa882018-09-14 13:32:07 -0700697 else:
Mariatta9f43fbb2018-10-24 15:37:12 -0700698 print(f'The coroutine returned: {result!r}')
Yury Selivanov3faaa882018-09-14 13:32:07 -0700699
700 See the :ref:`concurrency and multithreading <asyncio-multithreading>`
701 section of the documentation.
702
Vaibhav Gupta3a810762018-12-26 20:17:17 +0530703 Unlike other asyncio functions this function requires the *loop*
Yury Selivanov3faaa882018-09-14 13:32:07 -0700704 argument to be passed explicitly.
705
706 .. versionadded:: 3.5.1
707
708
709Introspection
710=============
711
712
713.. function:: current_task(loop=None)
714
715 Return the currently running :class:`Task` instance, or ``None`` if
716 no task is running.
717
718 If *loop* is ``None`` :func:`get_running_loop` is used to get
719 the current loop.
720
721 .. versionadded:: 3.7
722
723
724.. function:: all_tasks(loop=None)
725
726 Return a set of not yet finished :class:`Task` objects run by
727 the loop.
728
729 If *loop* is ``None``, :func:`get_running_loop` is used for getting
730 current loop.
731
732 .. versionadded:: 3.7
733
734
735Task Object
736===========
737
738.. class:: Task(coro, \*, loop=None, name=None)
739
Yury Selivanovdb1a80e2018-09-21 16:23:15 -0400740 A :class:`Future-like <Future>` object that runs a Python
Yury Selivanov3faaa882018-09-14 13:32:07 -0700741 :ref:`coroutine <coroutine>`. Not thread-safe.
742
743 Tasks are used to run coroutines in event loops.
744 If a coroutine awaits on a Future, the Task suspends
745 the execution of the coroutine and waits for the completion
746 of the Future. When the Future is *done*, the execution of
747 the wrapped coroutine resumes.
748
749 Event loops use cooperative scheduling: an event loop runs
750 one Task at a time. While a Task awaits for the completion of a
751 Future, the event loop runs other Tasks, callbacks, or performs
752 IO operations.
753
754 Use the high-level :func:`asyncio.create_task` function to create
Elvis Pranskevichus1fa2ec42018-09-17 19:16:44 -0400755 Tasks, or the low-level :meth:`loop.create_task` or
756 :func:`ensure_future` functions. Manual instantiation of Tasks
757 is discouraged.
Yury Selivanov3faaa882018-09-14 13:32:07 -0700758
759 To cancel a running Task use the :meth:`cancel` method. Calling it
760 will cause the Task to throw a :exc:`CancelledError` exception into
761 the wrapped coroutine. If a coroutine is awaiting on a Future
762 object during cancellation, the Future object will be cancelled.
763
764 :meth:`cancelled` can be used to check if the Task was cancelled.
765 The method returns ``True`` if the wrapped coroutine did not
766 suppress the :exc:`CancelledError` exception and was actually
767 cancelled.
768
769 :class:`asyncio.Task` inherits from :class:`Future` all of its
770 APIs except :meth:`Future.set_result` and
771 :meth:`Future.set_exception`.
772
773 Tasks support the :mod:`contextvars` module. When a Task
774 is created it copies the current context and later runs its
775 coroutine in the copied context.
Elvis Pranskevichuse2b340a2018-05-29 17:31:01 -0400776
777 .. versionchanged:: 3.7
Yury Selivanov3faaa882018-09-14 13:32:07 -0700778 Added support for the :mod:`contextvars` module.
779
780 .. versionchanged:: 3.8
781 Added the ``name`` parameter.
782
Andrew Svetlova4888792019-09-12 15:40:40 +0300783 .. deprecated-removed:: 3.8 3.10
784 The *loop* parameter.
785
Chris Jerdonek1ce58412020-05-15 16:55:50 -0700786 .. method:: cancel(msg=None)
Yury Selivanov3faaa882018-09-14 13:32:07 -0700787
788 Request the Task to be cancelled.
789
790 This arranges for a :exc:`CancelledError` exception to be thrown
791 into the wrapped coroutine on the next cycle of the event loop.
792
793 The coroutine then has a chance to clean up or even deny the
794 request by suppressing the exception with a :keyword:`try` ...
795 ... ``except CancelledError`` ... :keyword:`finally` block.
796 Therefore, unlike :meth:`Future.cancel`, :meth:`Task.cancel` does
797 not guarantee that the Task will be cancelled, although
798 suppressing cancellation completely is not common and is actively
799 discouraged.
800
Chris Jerdonek1ce58412020-05-15 16:55:50 -0700801 .. versionchanged:: 3.9
802 Added the ``msg`` parameter.
803
Yury Selivanov7372c3b2018-09-14 15:11:24 -0700804 .. _asyncio_example_task_cancel:
805
Yury Selivanov3faaa882018-09-14 13:32:07 -0700806 The following example illustrates how coroutines can intercept
807 the cancellation request::
808
809 async def cancel_me():
810 print('cancel_me(): before sleep')
811
812 try:
813 # Wait for 1 hour
814 await asyncio.sleep(3600)
815 except asyncio.CancelledError:
816 print('cancel_me(): cancel sleep')
817 raise
818 finally:
819 print('cancel_me(): after sleep')
820
821 async def main():
822 # Create a "cancel_me" Task
823 task = asyncio.create_task(cancel_me())
824
825 # Wait for 1 second
826 await asyncio.sleep(1)
827
828 task.cancel()
829 try:
830 await task
831 except asyncio.CancelledError:
832 print("main(): cancel_me is cancelled now")
833
834 asyncio.run(main())
835
836 # Expected output:
837 #
838 # cancel_me(): before sleep
839 # cancel_me(): cancel sleep
840 # cancel_me(): after sleep
841 # main(): cancel_me is cancelled now
842
843 .. method:: cancelled()
844
845 Return ``True`` if the Task is *cancelled*.
846
847 The Task is *cancelled* when the cancellation was requested with
848 :meth:`cancel` and the wrapped coroutine propagated the
849 :exc:`CancelledError` exception thrown into it.
850
851 .. method:: done()
852
853 Return ``True`` if the Task is *done*.
854
855 A Task is *done* when the wrapped coroutine either returned
856 a value, raised an exception, or the Task was cancelled.
857
Yury Selivanove247b462018-09-20 12:43:59 -0400858 .. method:: result()
859
860 Return the result of the Task.
861
862 If the Task is *done*, the result of the wrapped coroutine
863 is returned (or if the coroutine raised an exception, that
864 exception is re-raised.)
865
866 If the Task has been *cancelled*, this method raises
867 a :exc:`CancelledError` exception.
868
869 If the Task's result isn't yet available, this method raises
870 a :exc:`InvalidStateError` exception.
871
872 .. method:: exception()
873
874 Return the exception of the Task.
875
876 If the wrapped coroutine raised an exception that exception
877 is returned. If the wrapped coroutine returned normally
878 this method returns ``None``.
879
880 If the Task has been *cancelled*, this method raises a
881 :exc:`CancelledError` exception.
882
883 If the Task isn't *done* yet, this method raises an
884 :exc:`InvalidStateError` exception.
885
886 .. method:: add_done_callback(callback, *, context=None)
887
888 Add a callback to be run when the Task is *done*.
889
890 This method should only be used in low-level callback-based code.
891
892 See the documentation of :meth:`Future.add_done_callback`
893 for more details.
894
895 .. method:: remove_done_callback(callback)
896
897 Remove *callback* from the callbacks list.
898
899 This method should only be used in low-level callback-based code.
900
901 See the documentation of :meth:`Future.remove_done_callback`
902 for more details.
903
Yury Selivanov3faaa882018-09-14 13:32:07 -0700904 .. method:: get_stack(\*, limit=None)
905
906 Return the list of stack frames for this Task.
907
908 If the wrapped coroutine is not done, this returns the stack
909 where it is suspended. If the coroutine has completed
910 successfully or was cancelled, this returns an empty list.
911 If the coroutine was terminated by an exception, this returns
912 the list of traceback frames.
913
914 The frames are always ordered from oldest to newest.
915
916 Only one stack frame is returned for a suspended coroutine.
917
918 The optional *limit* argument sets the maximum number of frames
919 to return; by default all available frames are returned.
920 The ordering of the returned list differs depending on whether
921 a stack or a traceback is returned: the newest frames of a
922 stack are returned, but the oldest frames of a traceback are
923 returned. (This matches the behavior of the traceback module.)
924
925 .. method:: print_stack(\*, limit=None, file=None)
926
927 Print the stack or traceback for this Task.
928
929 This produces output similar to that of the traceback module
930 for the frames retrieved by :meth:`get_stack`.
931
932 The *limit* argument is passed to :meth:`get_stack` directly.
933
934 The *file* argument is an I/O stream to which the output
935 is written; by default output is written to :data:`sys.stderr`.
936
Alex Grönholm98ef9202019-05-30 18:30:09 +0300937 .. method:: get_coro()
938
939 Return the coroutine object wrapped by the :class:`Task`.
940
941 .. versionadded:: 3.8
942
Yury Selivanov3faaa882018-09-14 13:32:07 -0700943 .. method:: get_name()
944
945 Return the name of the Task.
946
947 If no name has been explicitly assigned to the Task, the default
948 asyncio Task implementation generates a default name during
949 instantiation.
950
951 .. versionadded:: 3.8
952
953 .. method:: set_name(value)
954
955 Set the name of the Task.
956
957 The *value* argument can be any object, which is then
958 converted to a string.
959
960 In the default Task implementation, the name will be visible
961 in the :func:`repr` output of a task object.
962
963 .. versionadded:: 3.8
964
965 .. classmethod:: all_tasks(loop=None)
966
967 Return a set of all tasks for an event loop.
968
969 By default all tasks for the current event loop are returned.
970 If *loop* is ``None``, the :func:`get_event_loop` function
971 is used to get the current loop.
972
Matthias Bussonnierd0ebf132019-05-20 23:20:10 -0700973 .. deprecated-removed:: 3.7 3.9
974
975 Do not call this as a task method. Use the :func:`asyncio.all_tasks`
976 function instead.
Yury Selivanov3faaa882018-09-14 13:32:07 -0700977
978 .. classmethod:: current_task(loop=None)
979
980 Return the currently running task or ``None``.
981
982 If *loop* is ``None``, the :func:`get_event_loop` function
983 is used to get the current loop.
984
Matthias Bussonnierd0ebf132019-05-20 23:20:10 -0700985 .. deprecated-removed:: 3.7 3.9
986
987 Do not call this as a task method. Use the
988 :func:`asyncio.current_task` function instead.
Yury Selivanov3faaa882018-09-14 13:32:07 -0700989
990
991.. _asyncio_generator_based_coro:
992
993Generator-based Coroutines
994==========================
995
996.. note::
997
998 Support for generator-based coroutines is **deprecated** and
Yury Selivanovfad6af22018-09-25 17:44:52 -0400999 is scheduled for removal in Python 3.10.
Yury Selivanov3faaa882018-09-14 13:32:07 -07001000
1001Generator-based coroutines predate async/await syntax. They are
Elvis Pranskevichus1fa2ec42018-09-17 19:16:44 -04001002Python generators that use ``yield from`` expressions to await
Yury Selivanov3faaa882018-09-14 13:32:07 -07001003on Futures and other coroutines.
1004
1005Generator-based coroutines should be decorated with
1006:func:`@asyncio.coroutine <asyncio.coroutine>`, although this is not
1007enforced.
1008
1009
1010.. decorator:: coroutine
1011
1012 Decorator to mark generator-based coroutines.
1013
1014 This decorator enables legacy generator-based coroutines to be
1015 compatible with async/await code::
1016
1017 @asyncio.coroutine
1018 def old_style_coroutine():
1019 yield from asyncio.sleep(1)
1020
1021 async def main():
1022 await old_style_coroutine()
1023
Yury Selivanov3faaa882018-09-14 13:32:07 -07001024 This decorator should not be used for :keyword:`async def`
1025 coroutines.
1026
Andrew Svetlov68b34a72019-05-16 17:52:10 +03001027 .. deprecated-removed:: 3.8 3.10
1028
1029 Use :keyword:`async def` instead.
1030
Yury Selivanov3faaa882018-09-14 13:32:07 -07001031.. function:: iscoroutine(obj)
1032
1033 Return ``True`` if *obj* is a :ref:`coroutine object <coroutine>`.
1034
1035 This method is different from :func:`inspect.iscoroutine` because
Yury Selivanov59ee5b12018-09-27 15:48:30 -04001036 it returns ``True`` for generator-based coroutines.
Yury Selivanov3faaa882018-09-14 13:32:07 -07001037
1038.. function:: iscoroutinefunction(func)
1039
1040 Return ``True`` if *func* is a :ref:`coroutine function
1041 <coroutine>`.
1042
1043 This method is different from :func:`inspect.iscoroutinefunction`
1044 because it returns ``True`` for generator-based coroutine functions
1045 decorated with :func:`@coroutine <coroutine>`.