blob: 9f55a3534a5e0c8842261730c7a499b387260419 [file] [log] [blame]
R David Murray6a143812013-12-20 14:37:39 -05001.. currentmodule:: asyncio
Victor Stinnerea3183f2013-12-03 01:08:00 +01002
Yury Selivanov3faaa882018-09-14 13:32:07 -07003
4====================
5Coroutines and Tasks
Victor Stinnerea3183f2013-12-03 01:08:00 +01006====================
7
Yury Selivanov3faaa882018-09-14 13:32:07 -07008This section outlines high-level asyncio APIs to work with coroutines
9and Tasks.
lf627d2c82017-07-25 17:03:51 -060010
Yury Selivanov3faaa882018-09-14 13:32:07 -070011.. contents::
12 :depth: 1
13 :local:
14
lf627d2c82017-07-25 17:03:51 -060015
Victor Stinnerea3183f2013-12-03 01:08:00 +010016.. _coroutine:
17
18Coroutines
Yury Selivanov3faaa882018-09-14 13:32:07 -070019==========
Victor Stinnerea3183f2013-12-03 01:08:00 +010020
Yury Selivanov3faaa882018-09-14 13:32:07 -070021Coroutines declared with async/await syntax is the preferred way of
22writing asyncio applications. For example, the following snippet
Yury Selivanovb042cf12018-09-18 02:47:54 -040023of code (requires Python 3.7+) prints "hello", waits 1 second,
24and then prints "world"::
Victor Stinnerea3183f2013-12-03 01:08:00 +010025
Yury Selivanov3faaa882018-09-14 13:32:07 -070026 >>> import asyncio
Victor Stinnerea3183f2013-12-03 01:08:00 +010027
Yury Selivanov3faaa882018-09-14 13:32:07 -070028 >>> async def main():
29 ... print('hello')
30 ... await asyncio.sleep(1)
31 ... print('world')
Victor Stinnerea3183f2013-12-03 01:08:00 +010032
Yury Selivanov3faaa882018-09-14 13:32:07 -070033 >>> asyncio.run(main())
34 hello
35 world
Victor Stinnerea3183f2013-12-03 01:08:00 +010036
Yury Selivanov3faaa882018-09-14 13:32:07 -070037Note that simply calling a coroutine will not schedule it to
38be executed::
Victor Stinnerea3183f2013-12-03 01:08:00 +010039
Yury Selivanov3faaa882018-09-14 13:32:07 -070040 >>> main()
41 <coroutine object main at 0x1053bb7c8>
Victor Stinnerea3183f2013-12-03 01:08:00 +010042
Boštjan Mejak1d5bdef2019-05-19 11:01:36 +020043To actually run a coroutine, asyncio provides three main mechanisms:
Victor Stinnerea3183f2013-12-03 01:08:00 +010044
Elvis Pranskevichus1fa2ec42018-09-17 19:16:44 -040045* The :func:`asyncio.run` function to run the top-level
Yury Selivanov3faaa882018-09-14 13:32:07 -070046 entry point "main()" function (see the above example.)
Victor Stinnerea3183f2013-12-03 01:08:00 +010047
Elvis Pranskevichus1fa2ec42018-09-17 19:16:44 -040048* Awaiting on a coroutine. The following snippet of code will
Yury Selivanov3faaa882018-09-14 13:32:07 -070049 print "hello" after waiting for 1 second, and then print "world"
50 after waiting for *another* 2 seconds::
Victor Stinnerea3183f2013-12-03 01:08:00 +010051
Yury Selivanov3faaa882018-09-14 13:32:07 -070052 import asyncio
53 import time
Victor Stinnerea3183f2013-12-03 01:08:00 +010054
Yury Selivanov3faaa882018-09-14 13:32:07 -070055 async def say_after(delay, what):
56 await asyncio.sleep(delay)
57 print(what)
58
59 async def main():
Mariatta9f43fbb2018-10-24 15:37:12 -070060 print(f"started at {time.strftime('%X')}")
Yury Selivanov3faaa882018-09-14 13:32:07 -070061
62 await say_after(1, 'hello')
63 await say_after(2, 'world')
64
Mariatta9f43fbb2018-10-24 15:37:12 -070065 print(f"finished at {time.strftime('%X')}")
Yury Selivanov3faaa882018-09-14 13:32:07 -070066
67 asyncio.run(main())
68
69 Expected output::
70
71 started at 17:13:52
72 hello
73 world
74 finished at 17:13:55
75
Elvis Pranskevichus1fa2ec42018-09-17 19:16:44 -040076* The :func:`asyncio.create_task` function to run coroutines
Yury Selivanov3faaa882018-09-14 13:32:07 -070077 concurrently as asyncio :class:`Tasks <Task>`.
78
Danny Hermes7bfbda42018-09-17 21:49:21 -070079 Let's modify the above example and run two ``say_after`` coroutines
Yury Selivanov3faaa882018-09-14 13:32:07 -070080 *concurrently*::
81
82 async def main():
83 task1 = asyncio.create_task(
84 say_after(1, 'hello'))
85
86 task2 = asyncio.create_task(
87 say_after(2, 'world'))
88
Mariatta9f43fbb2018-10-24 15:37:12 -070089 print(f"started at {time.strftime('%X')}")
Yury Selivanov3faaa882018-09-14 13:32:07 -070090
91 # Wait until both tasks are completed (should take
92 # around 2 seconds.)
93 await task1
94 await task2
95
Mariatta9f43fbb2018-10-24 15:37:12 -070096 print(f"finished at {time.strftime('%X')}")
Yury Selivanov3faaa882018-09-14 13:32:07 -070097
98 Note that expected output now shows that the snippet runs
99 1 second faster than before::
100
101 started at 17:14:32
102 hello
103 world
104 finished at 17:14:34
105
Yury Selivanov47150392018-09-18 17:55:44 -0400106
107.. _asyncio-awaitables:
108
109Awaitables
110==========
111
Yury Selivanove247b462018-09-20 12:43:59 -0400112We say that an object is an **awaitable** object if it can be used
113in an :keyword:`await` expression. Many asyncio APIs are designed to
114accept awaitables.
115
116There are three main types of *awaitable* objects:
117**coroutines**, **Tasks**, and **Futures**.
Yury Selivanov47150392018-09-18 17:55:44 -0400118
119
Yury Selivanove247b462018-09-20 12:43:59 -0400120.. rubric:: Coroutines
Yury Selivanov47150392018-09-18 17:55:44 -0400121
Yury Selivanove247b462018-09-20 12:43:59 -0400122Python coroutines are *awaitables* and therefore can be awaited from
123other coroutines::
124
125 import asyncio
Yury Selivanov47150392018-09-18 17:55:44 -0400126
127 async def nested():
128 return 42
129
130 async def main():
Yury Selivanove247b462018-09-20 12:43:59 -0400131 # Nothing happens if we just call "nested()".
Yury Selivanovdb1a80e2018-09-21 16:23:15 -0400132 # A coroutine object is created but not awaited,
133 # so it *won't run at all*.
Yury Selivanove247b462018-09-20 12:43:59 -0400134 nested()
135
136 # Let's do it differently now and await it:
137 print(await nested()) # will print "42".
138
139 asyncio.run(main())
140
141.. important::
142
143 In this documentation the term "coroutine" can be used for
144 two closely related concepts:
145
146 * a *coroutine function*: an :keyword:`async def` function;
147
148 * a *coroutine object*: an object returned by calling a
149 *coroutine function*.
150
151asyncio also supports legacy :ref:`generator-based
152<asyncio_generator_based_coro>` coroutines.
153
154
155.. rubric:: Tasks
Yury Selivanov47150392018-09-18 17:55:44 -0400156
157*Tasks* are used to schedule coroutines *concurrently*.
Yury Selivanov47150392018-09-18 17:55:44 -0400158
Yury Selivanove247b462018-09-20 12:43:59 -0400159When a coroutine is wrapped into a *Task* with functions like
160:func:`asyncio.create_task` the coroutine is automatically
161scheduled to run soon::
Yury Selivanov3faaa882018-09-14 13:32:07 -0700162
Yury Selivanove247b462018-09-20 12:43:59 -0400163 import asyncio
Yury Selivanov3faaa882018-09-14 13:32:07 -0700164
Yury Selivanove247b462018-09-20 12:43:59 -0400165 async def nested():
166 return 42
167
168 async def main():
169 # Schedule nested() to run soon concurrently
170 # with "main()".
171 task = asyncio.create_task(nested())
172
173 # "task" can now be used to cancel "nested()", or
174 # can simply be awaited to wait until it is complete:
175 await task
176
177 asyncio.run(main())
Victor Stinner337e03f2014-08-11 01:11:13 +0200178
Victor Stinnerea3183f2013-12-03 01:08:00 +0100179
Yury Selivanov47150392018-09-18 17:55:44 -0400180.. rubric:: Futures
181
Yury Selivanove247b462018-09-20 12:43:59 -0400182A :class:`Future` is a special **low-level** awaitable object that
183represents an **eventual result** of an asynchronous operation.
Yury Selivanov47150392018-09-18 17:55:44 -0400184
Yury Selivanove247b462018-09-20 12:43:59 -0400185When a Future object is *awaited* it means that the coroutine will
186wait until the Future is resolved in some other place.
187
Yury Selivanov47150392018-09-18 17:55:44 -0400188Future objects in asyncio are needed to allow callback-based code
189to be used with async/await.
190
Yury Selivanove247b462018-09-20 12:43:59 -0400191Normally **there is no need** to create Future objects at the
Yury Selivanov47150392018-09-18 17:55:44 -0400192application level code.
193
194Future objects, sometimes exposed by libraries and some asyncio
Yury Selivanove247b462018-09-20 12:43:59 -0400195APIs, can be awaited::
Yury Selivanov47150392018-09-18 17:55:44 -0400196
197 async def main():
198 await function_that_returns_a_future_object()
199
200 # this is also valid:
201 await asyncio.gather(
202 function_that_returns_a_future_object(),
203 some_python_coroutine()
204 )
205
Yury Selivanove247b462018-09-20 12:43:59 -0400206A good example of a low-level function that returns a Future object
207is :meth:`loop.run_in_executor`.
208
Yury Selivanov47150392018-09-18 17:55:44 -0400209
Yury Selivanov3faaa882018-09-14 13:32:07 -0700210Running an asyncio Program
211==========================
Victor Stinnerea3183f2013-12-03 01:08:00 +0100212
Elvis Pranskevichus63536bd2018-05-19 23:15:06 -0400213.. function:: run(coro, \*, debug=False)
Yury Selivanov02a0a192017-12-14 09:42:21 -0500214
215 This function runs the passed coroutine, taking care of
Kyle Stanley9fdc64c2019-09-19 08:47:22 -0400216 managing the asyncio event loop, *finalizing asynchronous
217 generators*, and closing the threadpool.
Yury Selivanov02a0a192017-12-14 09:42:21 -0500218
219 This function cannot be called when another asyncio event loop is
220 running in the same thread.
221
Yury Selivanov3faaa882018-09-14 13:32:07 -0700222 If *debug* is ``True``, the event loop will be run in debug mode.
Yury Selivanov02a0a192017-12-14 09:42:21 -0500223
224 This function always creates a new event loop and closes it at
225 the end. It should be used as a main entry point for asyncio
226 programs, and should ideally only be called once.
227
Emmanuel Arias17deb162019-09-25 05:53:49 -0300228 Return a result of *coro* execution, or raise a :exc:`RuntimeError`
229 if ``asyncio.run()`` is called from a running event loop, or a
230 :exc:`ValueError` if *coro* is not a courutine.
231
232 Example::
233
234 async def main():
235 await asyncio.sleep(1)
236 print('hello')
237
238 asyncio.run(main())
239
Yury Selivanov02a0a192017-12-14 09:42:21 -0500240 .. versionadded:: 3.7
241
Kyle Stanley9fdc64c2019-09-19 08:47:22 -0400242 .. versionchanged:: 3.9
243 Updated to use :meth:`loop.shutdown_default_executor`.
Yury Selivanov02a0a192017-12-14 09:42:21 -0500244
Yury Selivanov3faaa882018-09-14 13:32:07 -0700245Creating Tasks
246==============
Victor Stinnerea3183f2013-12-03 01:08:00 +0100247
Alex Grönholmcca4eec2018-08-09 00:06:47 +0300248.. function:: create_task(coro, \*, name=None)
Andrew Svetlovf74ef452017-12-15 07:04:38 +0200249
Yury Selivanove247b462018-09-20 12:43:59 -0400250 Wrap the *coro* :ref:`coroutine <coroutine>` into a :class:`Task`
251 and schedule its execution. Return the Task object.
Alex Grönholmcca4eec2018-08-09 00:06:47 +0300252
253 If *name* is not ``None``, it is set as the name of the task using
254 :meth:`Task.set_name`.
Andrew Svetlovf74ef452017-12-15 07:04:38 +0200255
Elvis Pranskevichus1fa2ec42018-09-17 19:16:44 -0400256 The task is executed in the loop returned by :func:`get_running_loop`,
Andrew Svetlovf74ef452017-12-15 07:04:38 +0200257 :exc:`RuntimeError` is raised if there is no running loop in
258 current thread.
259
Yury Selivanov47150392018-09-18 17:55:44 -0400260 This function has been **added in Python 3.7**. Prior to
261 Python 3.7, the low-level :func:`asyncio.ensure_future` function
262 can be used instead::
263
264 async def coro():
265 ...
266
267 # In Python 3.7+
268 task = asyncio.create_task(coro())
269 ...
270
271 # This works in all Python versions but is less readable
272 task = asyncio.ensure_future(coro())
273 ...
274
Andrew Svetlovf74ef452017-12-15 07:04:38 +0200275 .. versionadded:: 3.7
276
Alex Grönholmcca4eec2018-08-09 00:06:47 +0300277 .. versionchanged:: 3.8
278 Added the ``name`` parameter.
279
Victor Stinnerea3183f2013-12-03 01:08:00 +0100280
Yury Selivanov3faaa882018-09-14 13:32:07 -0700281Sleeping
282========
Andrew Svetlovf1240162016-01-11 14:40:35 +0200283
Victor Stinnerbdd574d2015-02-12 22:49:18 +0100284.. coroutinefunction:: sleep(delay, result=None, \*, loop=None)
Victor Stinnerea3183f2013-12-03 01:08:00 +0100285
Yury Selivanov3faaa882018-09-14 13:32:07 -0700286 Block for *delay* seconds.
287
288 If *result* is provided, it is returned to the caller
Eli Bendersky2d26af82014-01-20 06:59:23 -0800289 when the coroutine completes.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100290
Hrvoje Nikšićcd602b82018-10-01 12:09:38 +0200291 ``sleep()`` always suspends the current task, allowing other tasks
292 to run.
293
Matthias Bussonnierd0ebf132019-05-20 23:20:10 -0700294 .. deprecated-removed:: 3.8 3.10
295 The *loop* parameter.
Yury Selivanov47150392018-09-18 17:55:44 -0400296
Yury Selivanov7372c3b2018-09-14 15:11:24 -0700297 .. _asyncio_example_sleep:
Victor Stinner45b27ed2014-02-01 02:36:43 +0100298
Yury Selivanov3faaa882018-09-14 13:32:07 -0700299 Example of coroutine displaying the current date every second
Elvis Pranskevichus1fa2ec42018-09-17 19:16:44 -0400300 for 5 seconds::
Victor Stinnerbdd574d2015-02-12 22:49:18 +0100301
Yury Selivanov3faaa882018-09-14 13:32:07 -0700302 import asyncio
303 import datetime
Victor Stinnerea3183f2013-12-03 01:08:00 +0100304
Yury Selivanov3faaa882018-09-14 13:32:07 -0700305 async def display_date():
306 loop = asyncio.get_running_loop()
307 end_time = loop.time() + 5.0
308 while True:
309 print(datetime.datetime.now())
310 if (loop.time() + 1.0) >= end_time:
311 break
312 await asyncio.sleep(1)
313
314 asyncio.run(display_date())
315
316
317Running Tasks Concurrently
318==========================
319
Yury Selivanove247b462018-09-20 12:43:59 -0400320.. awaitablefunction:: gather(\*aws, loop=None, return_exceptions=False)
Yury Selivanov3faaa882018-09-14 13:32:07 -0700321
Yury Selivanove247b462018-09-20 12:43:59 -0400322 Run :ref:`awaitable objects <asyncio-awaitables>` in the *aws*
Yury Selivanov47150392018-09-18 17:55:44 -0400323 sequence *concurrently*.
Yury Selivanov3faaa882018-09-14 13:32:07 -0700324
Yury Selivanove247b462018-09-20 12:43:59 -0400325 If any awaitable in *aws* is a coroutine, it is automatically
Yury Selivanov47150392018-09-18 17:55:44 -0400326 scheduled as a Task.
Yury Selivanov3faaa882018-09-14 13:32:07 -0700327
Yury Selivanov47150392018-09-18 17:55:44 -0400328 If all awaitables are completed successfully, the result is an
329 aggregate list of returned values. The order of result values
Yury Selivanove247b462018-09-20 12:43:59 -0400330 corresponds to the order of awaitables in *aws*.
Yury Selivanov3faaa882018-09-14 13:32:07 -0700331
Yury Selivanovdb1a80e2018-09-21 16:23:15 -0400332 If *return_exceptions* is ``False`` (default), the first
333 raised exception is immediately propagated to the task that
334 awaits on ``gather()``. Other awaitables in the *aws* sequence
335 **won't be cancelled** and will continue to run.
336
Yury Selivanov47150392018-09-18 17:55:44 -0400337 If *return_exceptions* is ``True``, exceptions are treated the
338 same as successful results, and aggregated in the result list.
Yury Selivanov3faaa882018-09-14 13:32:07 -0700339
Yury Selivanovdb1a80e2018-09-21 16:23:15 -0400340 If ``gather()`` is *cancelled*, all submitted awaitables
Yury Selivanov3faaa882018-09-14 13:32:07 -0700341 (that have not completed yet) are also *cancelled*.
342
Yury Selivanove247b462018-09-20 12:43:59 -0400343 If any Task or Future from the *aws* sequence is *cancelled*, it is
Yury Selivanov47150392018-09-18 17:55:44 -0400344 treated as if it raised :exc:`CancelledError` -- the ``gather()``
345 call is **not** cancelled in this case. This is to prevent the
346 cancellation of one submitted Task/Future to cause other
347 Tasks/Futures to be cancelled.
Yury Selivanov3faaa882018-09-14 13:32:07 -0700348
Andrew Svetlova4888792019-09-12 15:40:40 +0300349 .. deprecated-removed:: 3.8 3.10
350 The *loop* parameter.
351
Yury Selivanov7372c3b2018-09-14 15:11:24 -0700352 .. _asyncio_example_gather:
353
Yury Selivanov3faaa882018-09-14 13:32:07 -0700354 Example::
355
356 import asyncio
357
358 async def factorial(name, number):
359 f = 1
360 for i in range(2, number + 1):
361 print(f"Task {name}: Compute factorial({i})...")
362 await asyncio.sleep(1)
363 f *= i
364 print(f"Task {name}: factorial({number}) = {f}")
365
366 async def main():
Yury Selivanov47150392018-09-18 17:55:44 -0400367 # Schedule three calls *concurrently*:
Yury Selivanov3faaa882018-09-14 13:32:07 -0700368 await asyncio.gather(
369 factorial("A", 2),
370 factorial("B", 3),
371 factorial("C", 4),
Miguel Ángel García9c53fa62018-09-18 08:01:26 +0200372 )
Yury Selivanov3faaa882018-09-14 13:32:07 -0700373
374 asyncio.run(main())
375
376 # Expected output:
377 #
378 # Task A: Compute factorial(2)...
379 # Task B: Compute factorial(2)...
380 # Task C: Compute factorial(2)...
381 # Task A: factorial(2) = 2
382 # Task B: Compute factorial(3)...
383 # Task C: Compute factorial(3)...
384 # Task B: factorial(3) = 6
385 # Task C: Compute factorial(4)...
386 # Task C: factorial(4) = 24
387
Yury Selivanov47150392018-09-18 17:55:44 -0400388 .. versionchanged:: 3.7
389 If the *gather* itself is cancelled, the cancellation is
390 propagated regardless of *return_exceptions*.
391
Yury Selivanov3faaa882018-09-14 13:32:07 -0700392
Yury Selivanovdb1a80e2018-09-21 16:23:15 -0400393Shielding From Cancellation
394===========================
Yury Selivanov3faaa882018-09-14 13:32:07 -0700395
Yury Selivanove247b462018-09-20 12:43:59 -0400396.. awaitablefunction:: shield(aw, \*, loop=None)
Yury Selivanov3faaa882018-09-14 13:32:07 -0700397
Yury Selivanov47150392018-09-18 17:55:44 -0400398 Protect an :ref:`awaitable object <asyncio-awaitables>`
399 from being :meth:`cancelled <Task.cancel>`.
Yury Selivanov3faaa882018-09-14 13:32:07 -0700400
Yury Selivanovdb1a80e2018-09-21 16:23:15 -0400401 If *aw* is a coroutine it is automatically scheduled as a Task.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100402
403 The statement::
404
Andrew Svetlov88743422017-12-11 17:35:49 +0200405 res = await shield(something())
Victor Stinnerea3183f2013-12-03 01:08:00 +0100406
Elvis Pranskevichus1fa2ec42018-09-17 19:16:44 -0400407 is equivalent to::
Victor Stinnerea3183f2013-12-03 01:08:00 +0100408
Andrew Svetlov88743422017-12-11 17:35:49 +0200409 res = await something()
Victor Stinnerea3183f2013-12-03 01:08:00 +0100410
Yury Selivanov3faaa882018-09-14 13:32:07 -0700411 *except* that if the coroutine containing it is cancelled, the
412 Task running in ``something()`` is not cancelled. From the point
413 of view of ``something()``, the cancellation did not happen.
414 Although its caller is still cancelled, so the "await" expression
Elvis Pranskevichus1fa2ec42018-09-17 19:16:44 -0400415 still raises a :exc:`CancelledError`.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100416
Yury Selivanov3faaa882018-09-14 13:32:07 -0700417 If ``something()`` is cancelled by other means (i.e. from within
418 itself) that would also cancel ``shield()``.
419
420 If it is desired to completely ignore cancellation (not recommended)
421 the ``shield()`` function should be combined with a try/except
422 clause, as follows::
Victor Stinnerea3183f2013-12-03 01:08:00 +0100423
424 try:
Andrew Svetlov88743422017-12-11 17:35:49 +0200425 res = await shield(something())
Victor Stinnerea3183f2013-12-03 01:08:00 +0100426 except CancelledError:
427 res = None
428
Andrew Svetlova4888792019-09-12 15:40:40 +0300429 .. deprecated-removed:: 3.8 3.10
430 The *loop* parameter.
431
Yury Selivanov950204d2016-05-16 16:23:00 -0400432
Yury Selivanov3faaa882018-09-14 13:32:07 -0700433Timeouts
434========
435
Yury Selivanove247b462018-09-20 12:43:59 -0400436.. coroutinefunction:: wait_for(aw, timeout, \*, loop=None)
Yury Selivanov3faaa882018-09-14 13:32:07 -0700437
Yury Selivanove247b462018-09-20 12:43:59 -0400438 Wait for the *aw* :ref:`awaitable <asyncio-awaitables>`
Yury Selivanov47150392018-09-18 17:55:44 -0400439 to complete with a timeout.
Yury Selivanov3faaa882018-09-14 13:32:07 -0700440
Yury Selivanove247b462018-09-20 12:43:59 -0400441 If *aw* is a coroutine it is automatically scheduled as a Task.
Yury Selivanov3faaa882018-09-14 13:32:07 -0700442
443 *timeout* can either be ``None`` or a float or int number of seconds
444 to wait for. If *timeout* is ``None``, block until the future
445 completes.
446
447 If a timeout occurs, it cancels the task and raises
448 :exc:`asyncio.TimeoutError`.
449
Yury Selivanov47150392018-09-18 17:55:44 -0400450 To avoid the task :meth:`cancellation <Task.cancel>`,
451 wrap it in :func:`shield`.
Yury Selivanov3faaa882018-09-14 13:32:07 -0700452
453 The function will wait until the future is actually cancelled,
454 so the total wait time may exceed the *timeout*.
455
Yury Selivanove247b462018-09-20 12:43:59 -0400456 If the wait is cancelled, the future *aw* is also cancelled.
Yury Selivanov3faaa882018-09-14 13:32:07 -0700457
Matthias Bussonnierd0ebf132019-05-20 23:20:10 -0700458 .. deprecated-removed:: 3.8 3.10
459 The *loop* parameter.
Yury Selivanov47150392018-09-18 17:55:44 -0400460
Yury Selivanov7372c3b2018-09-14 15:11:24 -0700461 .. _asyncio_example_waitfor:
462
Yury Selivanov3faaa882018-09-14 13:32:07 -0700463 Example::
464
465 async def eternity():
466 # Sleep for one hour
467 await asyncio.sleep(3600)
468 print('yay!')
469
470 async def main():
471 # Wait for at most 1 second
472 try:
473 await asyncio.wait_for(eternity(), timeout=1.0)
474 except asyncio.TimeoutError:
475 print('timeout!')
476
477 asyncio.run(main())
478
479 # Expected output:
480 #
481 # timeout!
482
483 .. versionchanged:: 3.7
Yury Selivanove247b462018-09-20 12:43:59 -0400484 When *aw* is cancelled due to a timeout, ``wait_for`` waits
485 for *aw* to be cancelled. Previously, it raised
Yury Selivanov3faaa882018-09-14 13:32:07 -0700486 :exc:`asyncio.TimeoutError` immediately.
487
488
489Waiting Primitives
490==================
491
Yury Selivanove247b462018-09-20 12:43:59 -0400492.. coroutinefunction:: wait(aws, \*, loop=None, timeout=None,\
Andrew Svetlovf1240162016-01-11 14:40:35 +0200493 return_when=ALL_COMPLETED)
Victor Stinnerea3183f2013-12-03 01:08:00 +0100494
Yury Selivanove247b462018-09-20 12:43:59 -0400495 Run :ref:`awaitable objects <asyncio-awaitables>` in the *aws*
Yury Selivanov996859a2018-09-25 14:51:21 -0400496 set concurrently and block until the condition specified
Yury Selivanov47150392018-09-18 17:55:44 -0400497 by *return_when*.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100498
Yury Selivanov3faaa882018-09-14 13:32:07 -0700499 Returns two sets of Tasks/Futures: ``(done, pending)``.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100500
Yury Selivanov996859a2018-09-25 14:51:21 -0400501 Usage::
502
503 done, pending = await asyncio.wait(aws)
504
Yury Selivanov3faaa882018-09-14 13:32:07 -0700505 *timeout* (a float or int), if specified, can be used to control
506 the maximum number of seconds to wait before returning.
507
508 Note that this function does not raise :exc:`asyncio.TimeoutError`.
Elvis Pranskevichus1fa2ec42018-09-17 19:16:44 -0400509 Futures or Tasks that aren't done when the timeout occurs are simply
Yury Selivanov3faaa882018-09-14 13:32:07 -0700510 returned in the second set.
511
512 *return_when* indicates when this function should return. It must
513 be one of the following constants:
Victor Stinnerea3183f2013-12-03 01:08:00 +0100514
515 .. tabularcolumns:: |l|L|
516
517 +-----------------------------+----------------------------------------+
518 | Constant | Description |
519 +=============================+========================================+
520 | :const:`FIRST_COMPLETED` | The function will return when any |
521 | | future finishes or is cancelled. |
522 +-----------------------------+----------------------------------------+
523 | :const:`FIRST_EXCEPTION` | The function will return when any |
524 | | future finishes by raising an |
525 | | exception. If no future raises an |
526 | | exception then it is equivalent to |
527 | | :const:`ALL_COMPLETED`. |
528 +-----------------------------+----------------------------------------+
529 | :const:`ALL_COMPLETED` | The function will return when all |
530 | | futures finish or are cancelled. |
531 +-----------------------------+----------------------------------------+
532
Yury Selivanov3faaa882018-09-14 13:32:07 -0700533 Unlike :func:`~asyncio.wait_for`, ``wait()`` does not cancel the
534 futures when a timeout occurs.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100535
Andrew Svetlova4888792019-09-12 15:40:40 +0300536 .. deprecated:: 3.8
537
538 If any awaitable in *aws* is a coroutine, it is automatically
539 scheduled as a Task. Passing coroutines objects to
540 ``wait()`` directly is deprecated as it leads to
541 :ref:`confusing behavior <asyncio_example_wait_coroutine>`.
542
543 .. deprecated-removed:: 3.8 3.10
544
545 The *loop* parameter.
546
Yury Selivanov996859a2018-09-25 14:51:21 -0400547 .. _asyncio_example_wait_coroutine:
548 .. note::
Victor Stinnerea3183f2013-12-03 01:08:00 +0100549
Yury Selivanov996859a2018-09-25 14:51:21 -0400550 ``wait()`` schedules coroutines as Tasks automatically and later
551 returns those implicitly created Task objects in ``(done, pending)``
552 sets. Therefore the following code won't work as expected::
553
554 async def foo():
555 return 42
556
557 coro = foo()
558 done, pending = await asyncio.wait({coro})
559
560 if coro in done:
561 # This branch will never be run!
562
563 Here is how the above snippet can be fixed::
564
565 async def foo():
566 return 42
567
568 task = asyncio.create_task(foo())
569 done, pending = await asyncio.wait({task})
570
571 if task in done:
572 # Everything will work as expected now.
573
Matthias Bussonnierd0ebf132019-05-20 23:20:10 -0700574 .. deprecated:: 3.8
575
Yury Selivanov996859a2018-09-25 14:51:21 -0400576 Passing coroutine objects to ``wait()`` directly is
577 deprecated.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100578
Victor Stinnerea3183f2013-12-03 01:08:00 +0100579
Yury Selivanove247b462018-09-20 12:43:59 -0400580.. function:: as_completed(aws, \*, loop=None, timeout=None)
Yury Selivanov3faaa882018-09-14 13:32:07 -0700581
Yury Selivanove247b462018-09-20 12:43:59 -0400582 Run :ref:`awaitable objects <asyncio-awaitables>` in the *aws*
Yury Selivanov47150392018-09-18 17:55:44 -0400583 set concurrently. Return an iterator of :class:`Future` objects.
584 Each Future object returned represents the earliest result
585 from the set of the remaining awaitables.
Yury Selivanov3faaa882018-09-14 13:32:07 -0700586
587 Raises :exc:`asyncio.TimeoutError` if the timeout occurs before
588 all Futures are done.
589
Andrew Svetlova4888792019-09-12 15:40:40 +0300590 .. deprecated-removed:: 3.8 3.10
591 The *loop* parameter.
592
Yury Selivanov3faaa882018-09-14 13:32:07 -0700593 Example::
594
Yury Selivanove247b462018-09-20 12:43:59 -0400595 for f in as_completed(aws):
Yury Selivanov47150392018-09-18 17:55:44 -0400596 earliest_result = await f
Yury Selivanov3faaa882018-09-14 13:32:07 -0700597 # ...
Victor Stinnerea3183f2013-12-03 01:08:00 +0100598
Victor Stinner3e09e322013-12-03 01:22:06 +0100599
Yury Selivanov3faaa882018-09-14 13:32:07 -0700600Scheduling From Other Threads
601=============================
Victor Stinner1ad5afc2014-01-30 00:18:50 +0100602
Yury Selivanov3faaa882018-09-14 13:32:07 -0700603.. function:: run_coroutine_threadsafe(coro, loop)
Victor Stinner1ad5afc2014-01-30 00:18:50 +0100604
Yury Selivanov3faaa882018-09-14 13:32:07 -0700605 Submit a coroutine to the given event loop. Thread-safe.
Victor Stinner1ad5afc2014-01-30 00:18:50 +0100606
Yury Selivanov47150392018-09-18 17:55:44 -0400607 Return a :class:`concurrent.futures.Future` to wait for the result
608 from another OS thread.
Victor Stinner1ad5afc2014-01-30 00:18:50 +0100609
Yury Selivanov3faaa882018-09-14 13:32:07 -0700610 This function is meant to be called from a different OS thread
611 than the one where the event loop is running. Example::
Victor Stinner72dcb0a2015-04-03 17:08:19 +0200612
Yury Selivanov3faaa882018-09-14 13:32:07 -0700613 # Create a coroutine
614 coro = asyncio.sleep(1, result=3)
Yury Selivanov37f15bc2014-02-20 16:20:44 -0500615
Yury Selivanov3faaa882018-09-14 13:32:07 -0700616 # Submit the coroutine to a given loop
617 future = asyncio.run_coroutine_threadsafe(coro, loop)
Victor Stinner1ad5afc2014-01-30 00:18:50 +0100618
Yury Selivanov3faaa882018-09-14 13:32:07 -0700619 # Wait for the result with an optional timeout argument
620 assert future.result(timeout) == 3
621
622 If an exception is raised in the coroutine, the returned Future
623 will be notified. It can also be used to cancel the task in
624 the event loop::
625
626 try:
627 result = future.result(timeout)
628 except asyncio.TimeoutError:
629 print('The coroutine took too long, cancelling the task...')
630 future.cancel()
631 except Exception as exc:
Mariatta9f43fbb2018-10-24 15:37:12 -0700632 print(f'The coroutine raised an exception: {exc!r}')
Yury Selivanov3faaa882018-09-14 13:32:07 -0700633 else:
Mariatta9f43fbb2018-10-24 15:37:12 -0700634 print(f'The coroutine returned: {result!r}')
Yury Selivanov3faaa882018-09-14 13:32:07 -0700635
636 See the :ref:`concurrency and multithreading <asyncio-multithreading>`
637 section of the documentation.
638
Vaibhav Gupta3a810762018-12-26 20:17:17 +0530639 Unlike other asyncio functions this function requires the *loop*
Yury Selivanov3faaa882018-09-14 13:32:07 -0700640 argument to be passed explicitly.
641
642 .. versionadded:: 3.5.1
643
644
645Introspection
646=============
647
648
649.. function:: current_task(loop=None)
650
651 Return the currently running :class:`Task` instance, or ``None`` if
652 no task is running.
653
654 If *loop* is ``None`` :func:`get_running_loop` is used to get
655 the current loop.
656
657 .. versionadded:: 3.7
658
659
660.. function:: all_tasks(loop=None)
661
662 Return a set of not yet finished :class:`Task` objects run by
663 the loop.
664
665 If *loop* is ``None``, :func:`get_running_loop` is used for getting
666 current loop.
667
668 .. versionadded:: 3.7
669
670
671Task Object
672===========
673
674.. class:: Task(coro, \*, loop=None, name=None)
675
Yury Selivanovdb1a80e2018-09-21 16:23:15 -0400676 A :class:`Future-like <Future>` object that runs a Python
Yury Selivanov3faaa882018-09-14 13:32:07 -0700677 :ref:`coroutine <coroutine>`. Not thread-safe.
678
679 Tasks are used to run coroutines in event loops.
680 If a coroutine awaits on a Future, the Task suspends
681 the execution of the coroutine and waits for the completion
682 of the Future. When the Future is *done*, the execution of
683 the wrapped coroutine resumes.
684
685 Event loops use cooperative scheduling: an event loop runs
686 one Task at a time. While a Task awaits for the completion of a
687 Future, the event loop runs other Tasks, callbacks, or performs
688 IO operations.
689
690 Use the high-level :func:`asyncio.create_task` function to create
Elvis Pranskevichus1fa2ec42018-09-17 19:16:44 -0400691 Tasks, or the low-level :meth:`loop.create_task` or
692 :func:`ensure_future` functions. Manual instantiation of Tasks
693 is discouraged.
Yury Selivanov3faaa882018-09-14 13:32:07 -0700694
695 To cancel a running Task use the :meth:`cancel` method. Calling it
696 will cause the Task to throw a :exc:`CancelledError` exception into
697 the wrapped coroutine. If a coroutine is awaiting on a Future
698 object during cancellation, the Future object will be cancelled.
699
700 :meth:`cancelled` can be used to check if the Task was cancelled.
701 The method returns ``True`` if the wrapped coroutine did not
702 suppress the :exc:`CancelledError` exception and was actually
703 cancelled.
704
705 :class:`asyncio.Task` inherits from :class:`Future` all of its
706 APIs except :meth:`Future.set_result` and
707 :meth:`Future.set_exception`.
708
709 Tasks support the :mod:`contextvars` module. When a Task
710 is created it copies the current context and later runs its
711 coroutine in the copied context.
Elvis Pranskevichuse2b340a2018-05-29 17:31:01 -0400712
713 .. versionchanged:: 3.7
Yury Selivanov3faaa882018-09-14 13:32:07 -0700714 Added support for the :mod:`contextvars` module.
715
716 .. versionchanged:: 3.8
717 Added the ``name`` parameter.
718
Andrew Svetlova4888792019-09-12 15:40:40 +0300719 .. deprecated-removed:: 3.8 3.10
720 The *loop* parameter.
721
Yury Selivanov3faaa882018-09-14 13:32:07 -0700722 .. method:: cancel()
723
724 Request the Task to be cancelled.
725
726 This arranges for a :exc:`CancelledError` exception to be thrown
727 into the wrapped coroutine on the next cycle of the event loop.
728
729 The coroutine then has a chance to clean up or even deny the
730 request by suppressing the exception with a :keyword:`try` ...
731 ... ``except CancelledError`` ... :keyword:`finally` block.
732 Therefore, unlike :meth:`Future.cancel`, :meth:`Task.cancel` does
733 not guarantee that the Task will be cancelled, although
734 suppressing cancellation completely is not common and is actively
735 discouraged.
736
Yury Selivanov7372c3b2018-09-14 15:11:24 -0700737 .. _asyncio_example_task_cancel:
738
Yury Selivanov3faaa882018-09-14 13:32:07 -0700739 The following example illustrates how coroutines can intercept
740 the cancellation request::
741
742 async def cancel_me():
743 print('cancel_me(): before sleep')
744
745 try:
746 # Wait for 1 hour
747 await asyncio.sleep(3600)
748 except asyncio.CancelledError:
749 print('cancel_me(): cancel sleep')
750 raise
751 finally:
752 print('cancel_me(): after sleep')
753
754 async def main():
755 # Create a "cancel_me" Task
756 task = asyncio.create_task(cancel_me())
757
758 # Wait for 1 second
759 await asyncio.sleep(1)
760
761 task.cancel()
762 try:
763 await task
764 except asyncio.CancelledError:
765 print("main(): cancel_me is cancelled now")
766
767 asyncio.run(main())
768
769 # Expected output:
770 #
771 # cancel_me(): before sleep
772 # cancel_me(): cancel sleep
773 # cancel_me(): after sleep
774 # main(): cancel_me is cancelled now
775
776 .. method:: cancelled()
777
778 Return ``True`` if the Task is *cancelled*.
779
780 The Task is *cancelled* when the cancellation was requested with
781 :meth:`cancel` and the wrapped coroutine propagated the
782 :exc:`CancelledError` exception thrown into it.
783
784 .. method:: done()
785
786 Return ``True`` if the Task is *done*.
787
788 A Task is *done* when the wrapped coroutine either returned
789 a value, raised an exception, or the Task was cancelled.
790
Yury Selivanove247b462018-09-20 12:43:59 -0400791 .. method:: result()
792
793 Return the result of the Task.
794
795 If the Task is *done*, the result of the wrapped coroutine
796 is returned (or if the coroutine raised an exception, that
797 exception is re-raised.)
798
799 If the Task has been *cancelled*, this method raises
800 a :exc:`CancelledError` exception.
801
802 If the Task's result isn't yet available, this method raises
803 a :exc:`InvalidStateError` exception.
804
805 .. method:: exception()
806
807 Return the exception of the Task.
808
809 If the wrapped coroutine raised an exception that exception
810 is returned. If the wrapped coroutine returned normally
811 this method returns ``None``.
812
813 If the Task has been *cancelled*, this method raises a
814 :exc:`CancelledError` exception.
815
816 If the Task isn't *done* yet, this method raises an
817 :exc:`InvalidStateError` exception.
818
819 .. method:: add_done_callback(callback, *, context=None)
820
821 Add a callback to be run when the Task is *done*.
822
823 This method should only be used in low-level callback-based code.
824
825 See the documentation of :meth:`Future.add_done_callback`
826 for more details.
827
828 .. method:: remove_done_callback(callback)
829
830 Remove *callback* from the callbacks list.
831
832 This method should only be used in low-level callback-based code.
833
834 See the documentation of :meth:`Future.remove_done_callback`
835 for more details.
836
Yury Selivanov3faaa882018-09-14 13:32:07 -0700837 .. method:: get_stack(\*, limit=None)
838
839 Return the list of stack frames for this Task.
840
841 If the wrapped coroutine is not done, this returns the stack
842 where it is suspended. If the coroutine has completed
843 successfully or was cancelled, this returns an empty list.
844 If the coroutine was terminated by an exception, this returns
845 the list of traceback frames.
846
847 The frames are always ordered from oldest to newest.
848
849 Only one stack frame is returned for a suspended coroutine.
850
851 The optional *limit* argument sets the maximum number of frames
852 to return; by default all available frames are returned.
853 The ordering of the returned list differs depending on whether
854 a stack or a traceback is returned: the newest frames of a
855 stack are returned, but the oldest frames of a traceback are
856 returned. (This matches the behavior of the traceback module.)
857
858 .. method:: print_stack(\*, limit=None, file=None)
859
860 Print the stack or traceback for this Task.
861
862 This produces output similar to that of the traceback module
863 for the frames retrieved by :meth:`get_stack`.
864
865 The *limit* argument is passed to :meth:`get_stack` directly.
866
867 The *file* argument is an I/O stream to which the output
868 is written; by default output is written to :data:`sys.stderr`.
869
Alex Grönholm98ef9202019-05-30 18:30:09 +0300870 .. method:: get_coro()
871
872 Return the coroutine object wrapped by the :class:`Task`.
873
874 .. versionadded:: 3.8
875
Yury Selivanov3faaa882018-09-14 13:32:07 -0700876 .. method:: get_name()
877
878 Return the name of the Task.
879
880 If no name has been explicitly assigned to the Task, the default
881 asyncio Task implementation generates a default name during
882 instantiation.
883
884 .. versionadded:: 3.8
885
886 .. method:: set_name(value)
887
888 Set the name of the Task.
889
890 The *value* argument can be any object, which is then
891 converted to a string.
892
893 In the default Task implementation, the name will be visible
894 in the :func:`repr` output of a task object.
895
896 .. versionadded:: 3.8
897
898 .. classmethod:: all_tasks(loop=None)
899
900 Return a set of all tasks for an event loop.
901
902 By default all tasks for the current event loop are returned.
903 If *loop* is ``None``, the :func:`get_event_loop` function
904 is used to get the current loop.
905
Matthias Bussonnierd0ebf132019-05-20 23:20:10 -0700906 .. deprecated-removed:: 3.7 3.9
907
908 Do not call this as a task method. Use the :func:`asyncio.all_tasks`
909 function instead.
Yury Selivanov3faaa882018-09-14 13:32:07 -0700910
911 .. classmethod:: current_task(loop=None)
912
913 Return the currently running task or ``None``.
914
915 If *loop* is ``None``, the :func:`get_event_loop` function
916 is used to get the current loop.
917
Matthias Bussonnierd0ebf132019-05-20 23:20:10 -0700918 .. deprecated-removed:: 3.7 3.9
919
920 Do not call this as a task method. Use the
921 :func:`asyncio.current_task` function instead.
Yury Selivanov3faaa882018-09-14 13:32:07 -0700922
923
924.. _asyncio_generator_based_coro:
925
926Generator-based Coroutines
927==========================
928
929.. note::
930
931 Support for generator-based coroutines is **deprecated** and
Yury Selivanovfad6af22018-09-25 17:44:52 -0400932 is scheduled for removal in Python 3.10.
Yury Selivanov3faaa882018-09-14 13:32:07 -0700933
934Generator-based coroutines predate async/await syntax. They are
Elvis Pranskevichus1fa2ec42018-09-17 19:16:44 -0400935Python generators that use ``yield from`` expressions to await
Yury Selivanov3faaa882018-09-14 13:32:07 -0700936on Futures and other coroutines.
937
938Generator-based coroutines should be decorated with
939:func:`@asyncio.coroutine <asyncio.coroutine>`, although this is not
940enforced.
941
942
943.. decorator:: coroutine
944
945 Decorator to mark generator-based coroutines.
946
947 This decorator enables legacy generator-based coroutines to be
948 compatible with async/await code::
949
950 @asyncio.coroutine
951 def old_style_coroutine():
952 yield from asyncio.sleep(1)
953
954 async def main():
955 await old_style_coroutine()
956
Yury Selivanov3faaa882018-09-14 13:32:07 -0700957 This decorator should not be used for :keyword:`async def`
958 coroutines.
959
Andrew Svetlov68b34a72019-05-16 17:52:10 +0300960 .. deprecated-removed:: 3.8 3.10
961
962 Use :keyword:`async def` instead.
963
Yury Selivanov3faaa882018-09-14 13:32:07 -0700964.. function:: iscoroutine(obj)
965
966 Return ``True`` if *obj* is a :ref:`coroutine object <coroutine>`.
967
968 This method is different from :func:`inspect.iscoroutine` because
Yury Selivanov59ee5b12018-09-27 15:48:30 -0400969 it returns ``True`` for generator-based coroutines.
Yury Selivanov3faaa882018-09-14 13:32:07 -0700970
971.. function:: iscoroutinefunction(func)
972
973 Return ``True`` if *func* is a :ref:`coroutine function
974 <coroutine>`.
975
976 This method is different from :func:`inspect.iscoroutinefunction`
977 because it returns ``True`` for generator-based coroutine functions
978 decorated with :func:`@coroutine <coroutine>`.