blob: 84a52cb2d575710591308d1768a8f3c88209a806 [file] [log] [blame]
R David Murray6a143812013-12-20 14:37:39 -05001.. currentmodule:: asyncio
Yury Selivanov8be876e2018-09-11 17:10:37 -07002
Victor Stinner606ab032014-02-01 03:18:58 +01003.. _asyncio-sync:
Victor Stinnerea3183f2013-12-03 01:08:00 +01004
Yury Selivanov8be876e2018-09-11 17:10:37 -07005==========================
6Synchronization Primitives
Victor Stinnerea3183f2013-12-03 01:08:00 +01007==========================
8
Kyle Stanleyf9000642019-10-10 19:18:46 -04009**Source code:** :source:`Lib/asyncio/locks.py`
10
11-----------------------------------------------
12
Yury Selivanov8be876e2018-09-11 17:10:37 -070013asyncio synchronization primitives are designed to be similar to
14those of the :mod:`threading` module with two important caveats:
lf627d2c82017-07-25 17:03:51 -060015
Yury Selivanov8be876e2018-09-11 17:10:37 -070016* asyncio primitives are not thread-safe, therefore they should not
Elvis Pranskevichus1fa2ec42018-09-17 19:16:44 -040017 be used for OS thread synchronization (use :mod:`threading` for
Yury Selivanov8be876e2018-09-11 17:10:37 -070018 that);
19
Elvis Pranskevichus1fa2ec42018-09-17 19:16:44 -040020* methods of these synchronization primitives do not accept the *timeout*
Yury Selivanov8be876e2018-09-11 17:10:37 -070021 argument; use the :func:`asyncio.wait_for` function to perform
22 operations with timeouts.
23
Fredrik Averpil3e986de2019-04-21 01:06:38 +020024asyncio has the following basic synchronization primitives:
Victor Stinner7a55b882015-01-30 00:37:04 +010025
26* :class:`Lock`
27* :class:`Event`
28* :class:`Condition`
29* :class:`Semaphore`
30* :class:`BoundedSemaphore`
31
Yury Selivanov8be876e2018-09-11 17:10:37 -070032
33---------
Victor Stinner7a55b882015-01-30 00:37:04 +010034
Victor Stinnerea3183f2013-12-03 01:08:00 +010035
Victor Stinner8c462c52014-01-24 18:11:43 +010036Lock
Yury Selivanov8be876e2018-09-11 17:10:37 -070037====
Victor Stinner8c462c52014-01-24 18:11:43 +010038
Victor Stinnerea3183f2013-12-03 01:08:00 +010039.. class:: Lock(\*, loop=None)
40
Yury Selivanov8be876e2018-09-11 17:10:37 -070041 Implements a mutex lock for asyncio tasks. Not thread-safe.
Victor Stinnerea3183f2013-12-03 01:08:00 +010042
Yury Selivanov8be876e2018-09-11 17:10:37 -070043 An asyncio lock can be used to guarantee exclusive access to a
44 shared resource.
Victor Stinnerea3183f2013-12-03 01:08:00 +010045
Yury Selivanov8be876e2018-09-11 17:10:37 -070046 The preferred way to use a Lock is an :keyword:`async with`
47 statement::
Victor Stinnerea3183f2013-12-03 01:08:00 +010048
Yury Selivanov8be876e2018-09-11 17:10:37 -070049 lock = asyncio.Lock()
Victor Stinnerea3183f2013-12-03 01:08:00 +010050
Yury Selivanov8be876e2018-09-11 17:10:37 -070051 # ... later
52 async with lock:
53 # access shared state
Victor Stinnerea3183f2013-12-03 01:08:00 +010054
Yury Selivanov8be876e2018-09-11 17:10:37 -070055 which is equivalent to::
Victor Stinnerea3183f2013-12-03 01:08:00 +010056
Yury Selivanov8be876e2018-09-11 17:10:37 -070057 lock = asyncio.Lock()
Victor Stinner7b2c3c62015-02-26 10:39:16 +010058
Yury Selivanov8be876e2018-09-11 17:10:37 -070059 # ... later
60 await lock.acquire()
61 try:
62 # access shared state
63 finally:
64 lock.release()
Victor Stinnerea3183f2013-12-03 01:08:00 +010065
Emmanuel Arias537877d2019-09-10 07:55:07 -030066 .. deprecated-removed:: 3.8 3.10
67 The *loop* parameter.
68
Victor Stinnerbdd574d2015-02-12 22:49:18 +010069 .. coroutinemethod:: acquire()
Victor Stinnerea3183f2013-12-03 01:08:00 +010070
Yury Selivanov8be876e2018-09-11 17:10:37 -070071 Acquire the lock.
Victor Stinnerea3183f2013-12-03 01:08:00 +010072
Yury Selivanov8be876e2018-09-11 17:10:37 -070073 This method waits until the lock is *unlocked*, sets it to
74 *locked* and returns ``True``.
Victor Stinnerea3183f2013-12-03 01:08:00 +010075
Hrvoje Nikšić34f4f5e2019-05-29 19:08:17 +020076 When more than one coroutine is blocked in :meth:`acquire`
77 waiting for the lock to be unlocked, only one coroutine
78 eventually proceeds.
79
80 Acquiring a lock is *fair*: the coroutine that proceeds will be
81 the first coroutine that started waiting on the lock.
82
Victor Stinnerea3183f2013-12-03 01:08:00 +010083 .. method:: release()
84
Yury Selivanov8be876e2018-09-11 17:10:37 -070085 Release the lock.
Victor Stinnerea3183f2013-12-03 01:08:00 +010086
Yury Selivanov8be876e2018-09-11 17:10:37 -070087 When the lock is *locked*, reset it to *unlocked* and return.
Victor Stinnerea3183f2013-12-03 01:08:00 +010088
Carol Willing4e824e92018-09-13 18:28:19 -070089 If the lock is *unlocked*, a :exc:`RuntimeError` is raised.
Victor Stinnerea3183f2013-12-03 01:08:00 +010090
Yury Selivanov8be876e2018-09-11 17:10:37 -070091 .. method:: locked()
92
93 Return ``True`` if the lock is *locked*.
Victor Stinnerea3183f2013-12-03 01:08:00 +010094
95
Victor Stinner8c462c52014-01-24 18:11:43 +010096Event
Yury Selivanov8be876e2018-09-11 17:10:37 -070097=====
Victor Stinner8c462c52014-01-24 18:11:43 +010098
Victor Stinnerea3183f2013-12-03 01:08:00 +010099.. class:: Event(\*, loop=None)
100
Yury Selivanov8be876e2018-09-11 17:10:37 -0700101 An event object. Not thread-safe.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100102
Yury Selivanov8be876e2018-09-11 17:10:37 -0700103 An asyncio event can be used to notify multiple asyncio tasks
104 that some event has happened.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100105
Yury Selivanov8be876e2018-09-11 17:10:37 -0700106 An Event object manages an internal flag that can be set to *true*
107 with the :meth:`set` method and reset to *false* with the
108 :meth:`clear` method. The :meth:`wait` method blocks until the
109 flag is set to *true*. The flag is set to *false* initially.
Victor Stinner7b2c3c62015-02-26 10:39:16 +0100110
Emmanuel Arias537877d2019-09-10 07:55:07 -0300111
112 .. deprecated-removed:: 3.8 3.10
113 The *loop* parameter.
114
Yury Selivanov7372c3b2018-09-14 15:11:24 -0700115 .. _asyncio_example_sync_event:
116
Yury Selivanov8be876e2018-09-11 17:10:37 -0700117 Example::
Victor Stinnerea3183f2013-12-03 01:08:00 +0100118
Yury Selivanov8be876e2018-09-11 17:10:37 -0700119 async def waiter(event):
Carol Willing4e824e92018-09-13 18:28:19 -0700120 print('waiting for it ...')
Yury Selivanov8be876e2018-09-11 17:10:37 -0700121 await event.wait()
122 print('... got it!')
Victor Stinnerea3183f2013-12-03 01:08:00 +0100123
Yury Selivanov8be876e2018-09-11 17:10:37 -0700124 async def main():
125 # Create an Event object.
126 event = asyncio.Event()
Victor Stinnerea3183f2013-12-03 01:08:00 +0100127
Yury Selivanov8be876e2018-09-11 17:10:37 -0700128 # Spawn a Task to wait until 'event' is set.
129 waiter_task = asyncio.create_task(waiter(event))
Victor Stinnerea3183f2013-12-03 01:08:00 +0100130
Yury Selivanov8be876e2018-09-11 17:10:37 -0700131 # Sleep for 1 second and set the event.
132 await asyncio.sleep(1)
133 event.set()
Victor Stinnerea3183f2013-12-03 01:08:00 +0100134
Yury Selivanov8be876e2018-09-11 17:10:37 -0700135 # Wait until the waiter task is finished.
136 await waiter_task
137
138 asyncio.run(main())
Victor Stinnerea3183f2013-12-03 01:08:00 +0100139
Victor Stinnerbdd574d2015-02-12 22:49:18 +0100140 .. coroutinemethod:: wait()
Victor Stinnerea3183f2013-12-03 01:08:00 +0100141
Yury Selivanov8be876e2018-09-11 17:10:37 -0700142 Wait until the event is set.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100143
Yury Selivanov8be876e2018-09-11 17:10:37 -0700144 If the event is set, return ``True`` immediately.
145 Otherwise block until another task calls :meth:`set`.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100146
Yury Selivanov8be876e2018-09-11 17:10:37 -0700147 .. method:: set()
148
149 Set the event.
150
151 All tasks waiting for event to be set will be immediately
152 awakened.
153
154 .. method:: clear()
155
156 Clear (unset) the event.
157
158 Tasks awaiting on :meth:`wait` will now block until the
159 :meth:`set` method is called again.
160
161 .. method:: is_set()
162
163 Return ``True`` if the event is set.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100164
165
Victor Stinner8c462c52014-01-24 18:11:43 +0100166Condition
Yury Selivanov8be876e2018-09-11 17:10:37 -0700167=========
Victor Stinner8c462c52014-01-24 18:11:43 +0100168
Andrew Svetlovf200ce62014-07-26 19:50:37 +0300169.. class:: Condition(lock=None, \*, loop=None)
Victor Stinnerea3183f2013-12-03 01:08:00 +0100170
Yury Selivanov8be876e2018-09-11 17:10:37 -0700171 A Condition object. Not thread-safe.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100172
Yury Selivanov8be876e2018-09-11 17:10:37 -0700173 An asyncio condition primitive can be used by a task to wait for
Elvis Pranskevichus1fa2ec42018-09-17 19:16:44 -0400174 some event to happen and then get exclusive access to a shared
Yury Selivanov8be876e2018-09-11 17:10:37 -0700175 resource.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100176
Yury Selivanov8be876e2018-09-11 17:10:37 -0700177 In essence, a Condition object combines the functionality
Elvis Pranskevichus1fa2ec42018-09-17 19:16:44 -0400178 of an :class:`Event` and a :class:`Lock`. It is possible to have
179 multiple Condition objects share one Lock, which allows coordinating
Yury Selivanov8be876e2018-09-11 17:10:37 -0700180 exclusive access to a shared resource between different tasks
181 interested in particular states of that shared resource.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100182
Yury Selivanov8be876e2018-09-11 17:10:37 -0700183 The optional *lock* argument must be a :class:`Lock` object or
184 ``None``. In the latter case a new Lock object is created
185 automatically.
Andrew Svetlov28d8d142017-12-09 20:00:05 +0200186
Emmanuel Arias537877d2019-09-10 07:55:07 -0300187
188 .. deprecated-removed:: 3.8 3.10
189 The *loop* parameter.
190
Yury Selivanov8be876e2018-09-11 17:10:37 -0700191 The preferred way to use a Condition is an :keyword:`async with`
192 statement::
193
194 cond = asyncio.Condition()
195
196 # ... later
197 async with cond:
198 await cond.wait()
199
200 which is equivalent to::
201
202 cond = asyncio.Condition()
203
204 # ... later
Kevin Mai-Husan Chiad73ac0e2019-02-14 10:39:25 +0800205 await cond.acquire()
Yury Selivanov8be876e2018-09-11 17:10:37 -0700206 try:
207 await cond.wait()
208 finally:
Kevin Mai-Husan Chiad73ac0e2019-02-14 10:39:25 +0800209 cond.release()
Victor Stinner7b2c3c62015-02-26 10:39:16 +0100210
Victor Stinnerbdd574d2015-02-12 22:49:18 +0100211 .. coroutinemethod:: acquire()
Larry Hastings3732ed22014-03-15 21:13:56 -0700212
213 Acquire the underlying lock.
214
Yury Selivanov8be876e2018-09-11 17:10:37 -0700215 This method waits until the underlying lock is *unlocked*,
216 sets it to *locked* and returns ``True``.
Larry Hastings3732ed22014-03-15 21:13:56 -0700217
Victor Stinnerea3183f2013-12-03 01:08:00 +0100218 .. method:: notify(n=1)
219
Yury Selivanov8be876e2018-09-11 17:10:37 -0700220 Wake up at most *n* tasks (1 by default) waiting on this
221 condition. The method is no-op if no tasks are waiting.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100222
Yury Selivanov8be876e2018-09-11 17:10:37 -0700223 The lock must be acquired before this method is called and
224 released shortly after. If called with an *unlocked* lock
225 a :exc:`RuntimeError` error is raised.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100226
Larry Hastings3732ed22014-03-15 21:13:56 -0700227 .. method:: locked()
228
229 Return ``True`` if the underlying lock is acquired.
230
Victor Stinnerea3183f2013-12-03 01:08:00 +0100231 .. method:: notify_all()
232
Yury Selivanov8be876e2018-09-11 17:10:37 -0700233 Wake up all tasks waiting on this condition.
234
235 This method acts like :meth:`notify`, but wakes up all waiting
236 tasks.
237
238 The lock must be acquired before this method is called and
239 released shortly after. If called with an *unlocked* lock
240 a :exc:`RuntimeError` error is raised.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100241
Larry Hastings3732ed22014-03-15 21:13:56 -0700242 .. method:: release()
243
244 Release the underlying lock.
245
Yury Selivanov8be876e2018-09-11 17:10:37 -0700246 When invoked on an unlocked lock, a :exc:`RuntimeError` is
247 raised.
Larry Hastings3732ed22014-03-15 21:13:56 -0700248
Victor Stinnerbdd574d2015-02-12 22:49:18 +0100249 .. coroutinemethod:: wait()
Victor Stinnerea3183f2013-12-03 01:08:00 +0100250
251 Wait until notified.
252
Yury Selivanov8be876e2018-09-11 17:10:37 -0700253 If the calling task has not acquired the lock when this method is
Victor Stinnerea3183f2013-12-03 01:08:00 +0100254 called, a :exc:`RuntimeError` is raised.
255
Yury Selivanov8be876e2018-09-11 17:10:37 -0700256 This method releases the underlying lock, and then blocks until
257 it is awakened by a :meth:`notify` or :meth:`notify_all` call.
258 Once awakened, the Condition re-acquires its lock and this method
259 returns ``True``.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100260
Victor Stinnerbdd574d2015-02-12 22:49:18 +0100261 .. coroutinemethod:: wait_for(predicate)
Victor Stinnerea3183f2013-12-03 01:08:00 +0100262
Yury Selivanov8be876e2018-09-11 17:10:37 -0700263 Wait until a predicate becomes *true*.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100264
Yury Selivanov8be876e2018-09-11 17:10:37 -0700265 The predicate must be a callable which result will be
266 interpreted as a boolean value. The final value is the
267 return value.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100268
269
Victor Stinner8c462c52014-01-24 18:11:43 +0100270Semaphore
Yury Selivanov8be876e2018-09-11 17:10:37 -0700271=========
Victor Stinner8c462c52014-01-24 18:11:43 +0100272
Victor Stinnerea3183f2013-12-03 01:08:00 +0100273.. class:: Semaphore(value=1, \*, loop=None)
274
Yury Selivanov8be876e2018-09-11 17:10:37 -0700275 A Semaphore object. Not thread-safe.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100276
277 A semaphore manages an internal counter which is decremented by each
Yury Selivanov8be876e2018-09-11 17:10:37 -0700278 :meth:`acquire` call and incremented by each :meth:`release` call.
279 The counter can never go below zero; when :meth:`acquire` finds
280 that it is zero, it blocks, waiting until some task calls
281 :meth:`release`.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100282
Yury Selivanov8be876e2018-09-11 17:10:37 -0700283 The optional *value* argument gives the initial value for the
284 internal counter (``1`` by default). If the given value is
285 less than ``0`` a :exc:`ValueError` is raised.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100286
Emmanuel Arias537877d2019-09-10 07:55:07 -0300287
288 .. deprecated-removed:: 3.8 3.10
289 The *loop* parameter.
290
Yury Selivanov8be876e2018-09-11 17:10:37 -0700291 The preferred way to use a Semaphore is an :keyword:`async with`
292 statement::
Andrew Svetlov28d8d142017-12-09 20:00:05 +0200293
Yury Selivanov8be876e2018-09-11 17:10:37 -0700294 sem = asyncio.Semaphore(10)
295
296 # ... later
297 async with sem:
298 # work with shared resource
299
300 which is equivalent to::
301
302 sem = asyncio.Semaphore(10)
303
304 # ... later
305 await sem.acquire()
306 try:
307 # work with shared resource
308 finally:
309 sem.release()
Victor Stinner7b2c3c62015-02-26 10:39:16 +0100310
Victor Stinnerbdd574d2015-02-12 22:49:18 +0100311 .. coroutinemethod:: acquire()
Victor Stinnerea3183f2013-12-03 01:08:00 +0100312
313 Acquire a semaphore.
314
Yury Selivanov8be876e2018-09-11 17:10:37 -0700315 If the internal counter is greater than zero, decrement
Elvis Pranskevichus1fa2ec42018-09-17 19:16:44 -0400316 it by one and return ``True`` immediately. If it is zero, wait
Yury Selivanov8be876e2018-09-11 17:10:37 -0700317 until a :meth:`release` is called and return ``True``.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100318
319 .. method:: locked()
320
321 Returns ``True`` if semaphore can not be acquired immediately.
322
Andrew Svetlovb79e0122015-05-08 14:13:41 +0300323 .. method:: release()
Victor Stinnerea3183f2013-12-03 01:08:00 +0100324
Yury Selivanov8be876e2018-09-11 17:10:37 -0700325 Release a semaphore, incrementing the internal counter by one.
326 Can wake up a task waiting to acquire the semaphore.
327
328 Unlike :class:`BoundedSemaphore`, :class:`Semaphore` allows
Elvis Pranskevichus1fa2ec42018-09-17 19:16:44 -0400329 making more ``release()`` calls than ``acquire()`` calls.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100330
331
Victor Stinner8c462c52014-01-24 18:11:43 +0100332BoundedSemaphore
Yury Selivanov8be876e2018-09-11 17:10:37 -0700333================
Victor Stinner8c462c52014-01-24 18:11:43 +0100334
Victor Stinnerea3183f2013-12-03 01:08:00 +0100335.. class:: BoundedSemaphore(value=1, \*, loop=None)
336
Yury Selivanov8be876e2018-09-11 17:10:37 -0700337 A bounded semaphore object. Not thread-safe.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100338
Yury Selivanov8be876e2018-09-11 17:10:37 -0700339 Bounded Semaphore is a version of :class:`Semaphore` that raises
340 a :exc:`ValueError` in :meth:`~Semaphore.release` if it
341 increases the internal counter above the initial *value*.
Andrew Svetlov28d8d142017-12-09 20:00:05 +0200342
343
Emmanuel Arias537877d2019-09-10 07:55:07 -0300344 .. deprecated-removed:: 3.8 3.10
345 The *loop* parameter.
346
Yury Selivanov8be876e2018-09-11 17:10:37 -0700347---------
Andrew Svetlov28d8d142017-12-09 20:00:05 +0200348
Andrew Svetlov28d8d142017-12-09 20:00:05 +0200349
Andrew Svetlov90d9ba62020-02-01 13:12:52 +0200350.. versionchanged:: 3.9
Andrew Svetlov28d8d142017-12-09 20:00:05 +0200351
Yury Selivanov8be876e2018-09-11 17:10:37 -0700352 Acquiring a lock using ``await lock`` or ``yield from lock`` and/or
Andrew Svetlov28d8d142017-12-09 20:00:05 +0200353 :keyword:`with` statement (``with await lock``, ``with (yield from
Andrew Svetlov90d9ba62020-02-01 13:12:52 +0200354 lock)``) was removed. Use ``async with lock`` instead.