blob: cc8c29dc3b6bc5fc97b94c92d709cc7bf5d9a095 [file] [log] [blame]
R David Murray6a143812013-12-20 14:37:39 -05001.. currentmodule:: asyncio
Yury Selivanov8be876e2018-09-11 17:10:37 -07002
Victor Stinner606ab032014-02-01 03:18:58 +01003.. _asyncio-sync:
Victor Stinnerea3183f2013-12-03 01:08:00 +01004
Yury Selivanov8be876e2018-09-11 17:10:37 -07005==========================
6Synchronization Primitives
Victor Stinnerea3183f2013-12-03 01:08:00 +01007==========================
8
Yury Selivanov8be876e2018-09-11 17:10:37 -07009asyncio synchronization primitives are designed to be similar to
10those of the :mod:`threading` module with two important caveats:
lf627d2c82017-07-25 17:03:51 -060011
Yury Selivanov8be876e2018-09-11 17:10:37 -070012* asyncio primitives are not thread-safe, therefore they should not
Elvis Pranskevichus1fa2ec42018-09-17 19:16:44 -040013 be used for OS thread synchronization (use :mod:`threading` for
Yury Selivanov8be876e2018-09-11 17:10:37 -070014 that);
15
Elvis Pranskevichus1fa2ec42018-09-17 19:16:44 -040016* methods of these synchronization primitives do not accept the *timeout*
Yury Selivanov8be876e2018-09-11 17:10:37 -070017 argument; use the :func:`asyncio.wait_for` function to perform
18 operations with timeouts.
19
Fredrik Averpil3e986de2019-04-21 01:06:38 +020020asyncio has the following basic synchronization primitives:
Victor Stinner7a55b882015-01-30 00:37:04 +010021
22* :class:`Lock`
23* :class:`Event`
24* :class:`Condition`
25* :class:`Semaphore`
26* :class:`BoundedSemaphore`
27
Yury Selivanov8be876e2018-09-11 17:10:37 -070028
29---------
Victor Stinner7a55b882015-01-30 00:37:04 +010030
Victor Stinnerea3183f2013-12-03 01:08:00 +010031
Victor Stinner8c462c52014-01-24 18:11:43 +010032Lock
Yury Selivanov8be876e2018-09-11 17:10:37 -070033====
Victor Stinner8c462c52014-01-24 18:11:43 +010034
Victor Stinnerea3183f2013-12-03 01:08:00 +010035.. class:: Lock(\*, loop=None)
36
Yury Selivanov8be876e2018-09-11 17:10:37 -070037 Implements a mutex lock for asyncio tasks. Not thread-safe.
Victor Stinnerea3183f2013-12-03 01:08:00 +010038
Yury Selivanov8be876e2018-09-11 17:10:37 -070039 An asyncio lock can be used to guarantee exclusive access to a
40 shared resource.
Victor Stinnerea3183f2013-12-03 01:08:00 +010041
Yury Selivanov8be876e2018-09-11 17:10:37 -070042 The preferred way to use a Lock is an :keyword:`async with`
43 statement::
Victor Stinnerea3183f2013-12-03 01:08:00 +010044
Yury Selivanov8be876e2018-09-11 17:10:37 -070045 lock = asyncio.Lock()
Victor Stinnerea3183f2013-12-03 01:08:00 +010046
Yury Selivanov8be876e2018-09-11 17:10:37 -070047 # ... later
48 async with lock:
49 # access shared state
Victor Stinnerea3183f2013-12-03 01:08:00 +010050
Yury Selivanov8be876e2018-09-11 17:10:37 -070051 which is equivalent to::
Victor Stinnerea3183f2013-12-03 01:08:00 +010052
Yury Selivanov8be876e2018-09-11 17:10:37 -070053 lock = asyncio.Lock()
Victor Stinner7b2c3c62015-02-26 10:39:16 +010054
Yury Selivanov8be876e2018-09-11 17:10:37 -070055 # ... later
56 await lock.acquire()
57 try:
58 # access shared state
59 finally:
60 lock.release()
Victor Stinnerea3183f2013-12-03 01:08:00 +010061
Emmanuel Arias537877d2019-09-10 07:55:07 -030062 .. deprecated-removed:: 3.8 3.10
63 The *loop* parameter.
64
Victor Stinnerbdd574d2015-02-12 22:49:18 +010065 .. coroutinemethod:: acquire()
Victor Stinnerea3183f2013-12-03 01:08:00 +010066
Yury Selivanov8be876e2018-09-11 17:10:37 -070067 Acquire the lock.
Victor Stinnerea3183f2013-12-03 01:08:00 +010068
Yury Selivanov8be876e2018-09-11 17:10:37 -070069 This method waits until the lock is *unlocked*, sets it to
70 *locked* and returns ``True``.
Victor Stinnerea3183f2013-12-03 01:08:00 +010071
Hrvoje Nikšić34f4f5e2019-05-29 19:08:17 +020072 When more than one coroutine is blocked in :meth:`acquire`
73 waiting for the lock to be unlocked, only one coroutine
74 eventually proceeds.
75
76 Acquiring a lock is *fair*: the coroutine that proceeds will be
77 the first coroutine that started waiting on the lock.
78
Victor Stinnerea3183f2013-12-03 01:08:00 +010079 .. method:: release()
80
Yury Selivanov8be876e2018-09-11 17:10:37 -070081 Release the lock.
Victor Stinnerea3183f2013-12-03 01:08:00 +010082
Yury Selivanov8be876e2018-09-11 17:10:37 -070083 When the lock is *locked*, reset it to *unlocked* and return.
Victor Stinnerea3183f2013-12-03 01:08:00 +010084
Carol Willing4e824e92018-09-13 18:28:19 -070085 If the lock is *unlocked*, a :exc:`RuntimeError` is raised.
Victor Stinnerea3183f2013-12-03 01:08:00 +010086
Yury Selivanov8be876e2018-09-11 17:10:37 -070087 .. method:: locked()
88
89 Return ``True`` if the lock is *locked*.
Victor Stinnerea3183f2013-12-03 01:08:00 +010090
91
Victor Stinner8c462c52014-01-24 18:11:43 +010092Event
Yury Selivanov8be876e2018-09-11 17:10:37 -070093=====
Victor Stinner8c462c52014-01-24 18:11:43 +010094
Victor Stinnerea3183f2013-12-03 01:08:00 +010095.. class:: Event(\*, loop=None)
96
Yury Selivanov8be876e2018-09-11 17:10:37 -070097 An event object. Not thread-safe.
Victor Stinnerea3183f2013-12-03 01:08:00 +010098
Yury Selivanov8be876e2018-09-11 17:10:37 -070099 An asyncio event can be used to notify multiple asyncio tasks
100 that some event has happened.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100101
Yury Selivanov8be876e2018-09-11 17:10:37 -0700102 An Event object manages an internal flag that can be set to *true*
103 with the :meth:`set` method and reset to *false* with the
104 :meth:`clear` method. The :meth:`wait` method blocks until the
105 flag is set to *true*. The flag is set to *false* initially.
Victor Stinner7b2c3c62015-02-26 10:39:16 +0100106
Emmanuel Arias537877d2019-09-10 07:55:07 -0300107
108 .. deprecated-removed:: 3.8 3.10
109 The *loop* parameter.
110
Yury Selivanov7372c3b2018-09-14 15:11:24 -0700111 .. _asyncio_example_sync_event:
112
Yury Selivanov8be876e2018-09-11 17:10:37 -0700113 Example::
Victor Stinnerea3183f2013-12-03 01:08:00 +0100114
Yury Selivanov8be876e2018-09-11 17:10:37 -0700115 async def waiter(event):
Carol Willing4e824e92018-09-13 18:28:19 -0700116 print('waiting for it ...')
Yury Selivanov8be876e2018-09-11 17:10:37 -0700117 await event.wait()
118 print('... got it!')
Victor Stinnerea3183f2013-12-03 01:08:00 +0100119
Yury Selivanov8be876e2018-09-11 17:10:37 -0700120 async def main():
121 # Create an Event object.
122 event = asyncio.Event()
Victor Stinnerea3183f2013-12-03 01:08:00 +0100123
Yury Selivanov8be876e2018-09-11 17:10:37 -0700124 # Spawn a Task to wait until 'event' is set.
125 waiter_task = asyncio.create_task(waiter(event))
Victor Stinnerea3183f2013-12-03 01:08:00 +0100126
Yury Selivanov8be876e2018-09-11 17:10:37 -0700127 # Sleep for 1 second and set the event.
128 await asyncio.sleep(1)
129 event.set()
Victor Stinnerea3183f2013-12-03 01:08:00 +0100130
Yury Selivanov8be876e2018-09-11 17:10:37 -0700131 # Wait until the waiter task is finished.
132 await waiter_task
133
134 asyncio.run(main())
Victor Stinnerea3183f2013-12-03 01:08:00 +0100135
Victor Stinnerbdd574d2015-02-12 22:49:18 +0100136 .. coroutinemethod:: wait()
Victor Stinnerea3183f2013-12-03 01:08:00 +0100137
Yury Selivanov8be876e2018-09-11 17:10:37 -0700138 Wait until the event is set.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100139
Yury Selivanov8be876e2018-09-11 17:10:37 -0700140 If the event is set, return ``True`` immediately.
141 Otherwise block until another task calls :meth:`set`.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100142
Yury Selivanov8be876e2018-09-11 17:10:37 -0700143 .. method:: set()
144
145 Set the event.
146
147 All tasks waiting for event to be set will be immediately
148 awakened.
149
150 .. method:: clear()
151
152 Clear (unset) the event.
153
154 Tasks awaiting on :meth:`wait` will now block until the
155 :meth:`set` method is called again.
156
157 .. method:: is_set()
158
159 Return ``True`` if the event is set.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100160
161
Victor Stinner8c462c52014-01-24 18:11:43 +0100162Condition
Yury Selivanov8be876e2018-09-11 17:10:37 -0700163=========
Victor Stinner8c462c52014-01-24 18:11:43 +0100164
Andrew Svetlovf200ce62014-07-26 19:50:37 +0300165.. class:: Condition(lock=None, \*, loop=None)
Victor Stinnerea3183f2013-12-03 01:08:00 +0100166
Yury Selivanov8be876e2018-09-11 17:10:37 -0700167 A Condition object. Not thread-safe.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100168
Yury Selivanov8be876e2018-09-11 17:10:37 -0700169 An asyncio condition primitive can be used by a task to wait for
Elvis Pranskevichus1fa2ec42018-09-17 19:16:44 -0400170 some event to happen and then get exclusive access to a shared
Yury Selivanov8be876e2018-09-11 17:10:37 -0700171 resource.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100172
Yury Selivanov8be876e2018-09-11 17:10:37 -0700173 In essence, a Condition object combines the functionality
Elvis Pranskevichus1fa2ec42018-09-17 19:16:44 -0400174 of an :class:`Event` and a :class:`Lock`. It is possible to have
175 multiple Condition objects share one Lock, which allows coordinating
Yury Selivanov8be876e2018-09-11 17:10:37 -0700176 exclusive access to a shared resource between different tasks
177 interested in particular states of that shared resource.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100178
Yury Selivanov8be876e2018-09-11 17:10:37 -0700179 The optional *lock* argument must be a :class:`Lock` object or
180 ``None``. In the latter case a new Lock object is created
181 automatically.
Andrew Svetlov28d8d142017-12-09 20:00:05 +0200182
Emmanuel Arias537877d2019-09-10 07:55:07 -0300183
184 .. deprecated-removed:: 3.8 3.10
185 The *loop* parameter.
186
Yury Selivanov8be876e2018-09-11 17:10:37 -0700187 The preferred way to use a Condition is an :keyword:`async with`
188 statement::
189
190 cond = asyncio.Condition()
191
192 # ... later
193 async with cond:
194 await cond.wait()
195
196 which is equivalent to::
197
198 cond = asyncio.Condition()
199
200 # ... later
Kevin Mai-Husan Chiad73ac0e2019-02-14 10:39:25 +0800201 await cond.acquire()
Yury Selivanov8be876e2018-09-11 17:10:37 -0700202 try:
203 await cond.wait()
204 finally:
Kevin Mai-Husan Chiad73ac0e2019-02-14 10:39:25 +0800205 cond.release()
Victor Stinner7b2c3c62015-02-26 10:39:16 +0100206
Victor Stinnerbdd574d2015-02-12 22:49:18 +0100207 .. coroutinemethod:: acquire()
Larry Hastings3732ed22014-03-15 21:13:56 -0700208
209 Acquire the underlying lock.
210
Yury Selivanov8be876e2018-09-11 17:10:37 -0700211 This method waits until the underlying lock is *unlocked*,
212 sets it to *locked* and returns ``True``.
Larry Hastings3732ed22014-03-15 21:13:56 -0700213
Victor Stinnerea3183f2013-12-03 01:08:00 +0100214 .. method:: notify(n=1)
215
Yury Selivanov8be876e2018-09-11 17:10:37 -0700216 Wake up at most *n* tasks (1 by default) waiting on this
217 condition. The method is no-op if no tasks are waiting.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100218
Yury Selivanov8be876e2018-09-11 17:10:37 -0700219 The lock must be acquired before this method is called and
220 released shortly after. If called with an *unlocked* lock
221 a :exc:`RuntimeError` error is raised.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100222
Larry Hastings3732ed22014-03-15 21:13:56 -0700223 .. method:: locked()
224
225 Return ``True`` if the underlying lock is acquired.
226
Victor Stinnerea3183f2013-12-03 01:08:00 +0100227 .. method:: notify_all()
228
Yury Selivanov8be876e2018-09-11 17:10:37 -0700229 Wake up all tasks waiting on this condition.
230
231 This method acts like :meth:`notify`, but wakes up all waiting
232 tasks.
233
234 The lock must be acquired before this method is called and
235 released shortly after. If called with an *unlocked* lock
236 a :exc:`RuntimeError` error is raised.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100237
Larry Hastings3732ed22014-03-15 21:13:56 -0700238 .. method:: release()
239
240 Release the underlying lock.
241
Yury Selivanov8be876e2018-09-11 17:10:37 -0700242 When invoked on an unlocked lock, a :exc:`RuntimeError` is
243 raised.
Larry Hastings3732ed22014-03-15 21:13:56 -0700244
Victor Stinnerbdd574d2015-02-12 22:49:18 +0100245 .. coroutinemethod:: wait()
Victor Stinnerea3183f2013-12-03 01:08:00 +0100246
247 Wait until notified.
248
Yury Selivanov8be876e2018-09-11 17:10:37 -0700249 If the calling task has not acquired the lock when this method is
Victor Stinnerea3183f2013-12-03 01:08:00 +0100250 called, a :exc:`RuntimeError` is raised.
251
Yury Selivanov8be876e2018-09-11 17:10:37 -0700252 This method releases the underlying lock, and then blocks until
253 it is awakened by a :meth:`notify` or :meth:`notify_all` call.
254 Once awakened, the Condition re-acquires its lock and this method
255 returns ``True``.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100256
Victor Stinnerbdd574d2015-02-12 22:49:18 +0100257 .. coroutinemethod:: wait_for(predicate)
Victor Stinnerea3183f2013-12-03 01:08:00 +0100258
Yury Selivanov8be876e2018-09-11 17:10:37 -0700259 Wait until a predicate becomes *true*.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100260
Yury Selivanov8be876e2018-09-11 17:10:37 -0700261 The predicate must be a callable which result will be
262 interpreted as a boolean value. The final value is the
263 return value.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100264
265
Victor Stinner8c462c52014-01-24 18:11:43 +0100266Semaphore
Yury Selivanov8be876e2018-09-11 17:10:37 -0700267=========
Victor Stinner8c462c52014-01-24 18:11:43 +0100268
Victor Stinnerea3183f2013-12-03 01:08:00 +0100269.. class:: Semaphore(value=1, \*, loop=None)
270
Yury Selivanov8be876e2018-09-11 17:10:37 -0700271 A Semaphore object. Not thread-safe.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100272
273 A semaphore manages an internal counter which is decremented by each
Yury Selivanov8be876e2018-09-11 17:10:37 -0700274 :meth:`acquire` call and incremented by each :meth:`release` call.
275 The counter can never go below zero; when :meth:`acquire` finds
276 that it is zero, it blocks, waiting until some task calls
277 :meth:`release`.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100278
Yury Selivanov8be876e2018-09-11 17:10:37 -0700279 The optional *value* argument gives the initial value for the
280 internal counter (``1`` by default). If the given value is
281 less than ``0`` a :exc:`ValueError` is raised.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100282
Emmanuel Arias537877d2019-09-10 07:55:07 -0300283
284 .. deprecated-removed:: 3.8 3.10
285 The *loop* parameter.
286
Yury Selivanov8be876e2018-09-11 17:10:37 -0700287 The preferred way to use a Semaphore is an :keyword:`async with`
288 statement::
Andrew Svetlov28d8d142017-12-09 20:00:05 +0200289
Yury Selivanov8be876e2018-09-11 17:10:37 -0700290 sem = asyncio.Semaphore(10)
291
292 # ... later
293 async with sem:
294 # work with shared resource
295
296 which is equivalent to::
297
298 sem = asyncio.Semaphore(10)
299
300 # ... later
301 await sem.acquire()
302 try:
303 # work with shared resource
304 finally:
305 sem.release()
Victor Stinner7b2c3c62015-02-26 10:39:16 +0100306
Victor Stinnerbdd574d2015-02-12 22:49:18 +0100307 .. coroutinemethod:: acquire()
Victor Stinnerea3183f2013-12-03 01:08:00 +0100308
309 Acquire a semaphore.
310
Yury Selivanov8be876e2018-09-11 17:10:37 -0700311 If the internal counter is greater than zero, decrement
Elvis Pranskevichus1fa2ec42018-09-17 19:16:44 -0400312 it by one and return ``True`` immediately. If it is zero, wait
Yury Selivanov8be876e2018-09-11 17:10:37 -0700313 until a :meth:`release` is called and return ``True``.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100314
315 .. method:: locked()
316
317 Returns ``True`` if semaphore can not be acquired immediately.
318
Andrew Svetlovb79e0122015-05-08 14:13:41 +0300319 .. method:: release()
Victor Stinnerea3183f2013-12-03 01:08:00 +0100320
Yury Selivanov8be876e2018-09-11 17:10:37 -0700321 Release a semaphore, incrementing the internal counter by one.
322 Can wake up a task waiting to acquire the semaphore.
323
324 Unlike :class:`BoundedSemaphore`, :class:`Semaphore` allows
Elvis Pranskevichus1fa2ec42018-09-17 19:16:44 -0400325 making more ``release()`` calls than ``acquire()`` calls.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100326
327
Victor Stinner8c462c52014-01-24 18:11:43 +0100328BoundedSemaphore
Yury Selivanov8be876e2018-09-11 17:10:37 -0700329================
Victor Stinner8c462c52014-01-24 18:11:43 +0100330
Victor Stinnerea3183f2013-12-03 01:08:00 +0100331.. class:: BoundedSemaphore(value=1, \*, loop=None)
332
Yury Selivanov8be876e2018-09-11 17:10:37 -0700333 A bounded semaphore object. Not thread-safe.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100334
Yury Selivanov8be876e2018-09-11 17:10:37 -0700335 Bounded Semaphore is a version of :class:`Semaphore` that raises
336 a :exc:`ValueError` in :meth:`~Semaphore.release` if it
337 increases the internal counter above the initial *value*.
Andrew Svetlov28d8d142017-12-09 20:00:05 +0200338
339
Emmanuel Arias537877d2019-09-10 07:55:07 -0300340 .. deprecated-removed:: 3.8 3.10
341 The *loop* parameter.
342
Yury Selivanov8be876e2018-09-11 17:10:37 -0700343---------
Andrew Svetlov28d8d142017-12-09 20:00:05 +0200344
Andrew Svetlov28d8d142017-12-09 20:00:05 +0200345
346.. deprecated:: 3.7
347
Yury Selivanov8be876e2018-09-11 17:10:37 -0700348 Acquiring a lock using ``await lock`` or ``yield from lock`` and/or
Andrew Svetlov28d8d142017-12-09 20:00:05 +0200349 :keyword:`with` statement (``with await lock``, ``with (yield from
Yury Selivanov8be876e2018-09-11 17:10:37 -0700350 lock)``) is deprecated. Use ``async with lock`` instead.