blob: 79f6b02d85e2e535a411f908ee6d6f8bf965b28c [file] [log] [blame]
R David Murray6a143812013-12-20 14:37:39 -05001.. currentmodule:: asyncio
Yury Selivanov8be876e2018-09-11 17:10:37 -07002
Victor Stinner606ab032014-02-01 03:18:58 +01003.. _asyncio-sync:
Victor Stinnerea3183f2013-12-03 01:08:00 +01004
Yury Selivanov8be876e2018-09-11 17:10:37 -07005==========================
6Synchronization Primitives
Victor Stinnerea3183f2013-12-03 01:08:00 +01007==========================
8
Yury Selivanov8be876e2018-09-11 17:10:37 -07009asyncio synchronization primitives are designed to be similar to
10those of the :mod:`threading` module with two important caveats:
lf627d2c82017-07-25 17:03:51 -060011
Yury Selivanov8be876e2018-09-11 17:10:37 -070012* asyncio primitives are not thread-safe, therefore they should not
Elvis Pranskevichus1fa2ec42018-09-17 19:16:44 -040013 be used for OS thread synchronization (use :mod:`threading` for
Yury Selivanov8be876e2018-09-11 17:10:37 -070014 that);
15
Elvis Pranskevichus1fa2ec42018-09-17 19:16:44 -040016* methods of these synchronization primitives do not accept the *timeout*
Yury Selivanov8be876e2018-09-11 17:10:37 -070017 argument; use the :func:`asyncio.wait_for` function to perform
18 operations with timeouts.
19
Fredrik Averpil3e986de2019-04-21 01:06:38 +020020asyncio has the following basic synchronization primitives:
Victor Stinner7a55b882015-01-30 00:37:04 +010021
22* :class:`Lock`
23* :class:`Event`
24* :class:`Condition`
25* :class:`Semaphore`
26* :class:`BoundedSemaphore`
27
Yury Selivanov8be876e2018-09-11 17:10:37 -070028
29---------
Victor Stinner7a55b882015-01-30 00:37:04 +010030
Victor Stinnerea3183f2013-12-03 01:08:00 +010031
Victor Stinner8c462c52014-01-24 18:11:43 +010032Lock
Yury Selivanov8be876e2018-09-11 17:10:37 -070033====
Victor Stinner8c462c52014-01-24 18:11:43 +010034
Victor Stinnerea3183f2013-12-03 01:08:00 +010035.. class:: Lock(\*, loop=None)
36
Yury Selivanov8be876e2018-09-11 17:10:37 -070037 Implements a mutex lock for asyncio tasks. Not thread-safe.
Victor Stinnerea3183f2013-12-03 01:08:00 +010038
Yury Selivanov8be876e2018-09-11 17:10:37 -070039 An asyncio lock can be used to guarantee exclusive access to a
40 shared resource.
Victor Stinnerea3183f2013-12-03 01:08:00 +010041
Yury Selivanov8be876e2018-09-11 17:10:37 -070042 The preferred way to use a Lock is an :keyword:`async with`
43 statement::
Victor Stinnerea3183f2013-12-03 01:08:00 +010044
Yury Selivanov8be876e2018-09-11 17:10:37 -070045 lock = asyncio.Lock()
Victor Stinnerea3183f2013-12-03 01:08:00 +010046
Yury Selivanov8be876e2018-09-11 17:10:37 -070047 # ... later
48 async with lock:
49 # access shared state
Victor Stinnerea3183f2013-12-03 01:08:00 +010050
Yury Selivanov8be876e2018-09-11 17:10:37 -070051 which is equivalent to::
Victor Stinnerea3183f2013-12-03 01:08:00 +010052
Yury Selivanov8be876e2018-09-11 17:10:37 -070053 lock = asyncio.Lock()
Victor Stinner7b2c3c62015-02-26 10:39:16 +010054
Yury Selivanov8be876e2018-09-11 17:10:37 -070055 # ... later
56 await lock.acquire()
57 try:
58 # access shared state
59 finally:
60 lock.release()
Victor Stinnerea3183f2013-12-03 01:08:00 +010061
Victor Stinnerbdd574d2015-02-12 22:49:18 +010062 .. coroutinemethod:: acquire()
Victor Stinnerea3183f2013-12-03 01:08:00 +010063
Yury Selivanov8be876e2018-09-11 17:10:37 -070064 Acquire the lock.
Victor Stinnerea3183f2013-12-03 01:08:00 +010065
Yury Selivanov8be876e2018-09-11 17:10:37 -070066 This method waits until the lock is *unlocked*, sets it to
67 *locked* and returns ``True``.
Victor Stinnerea3183f2013-12-03 01:08:00 +010068
Hrvoje Nikšić34f4f5e2019-05-29 19:08:17 +020069 When more than one coroutine is blocked in :meth:`acquire`
70 waiting for the lock to be unlocked, only one coroutine
71 eventually proceeds.
72
73 Acquiring a lock is *fair*: the coroutine that proceeds will be
74 the first coroutine that started waiting on the lock.
75
Victor Stinnerea3183f2013-12-03 01:08:00 +010076 .. method:: release()
77
Yury Selivanov8be876e2018-09-11 17:10:37 -070078 Release the lock.
Victor Stinnerea3183f2013-12-03 01:08:00 +010079
Yury Selivanov8be876e2018-09-11 17:10:37 -070080 When the lock is *locked*, reset it to *unlocked* and return.
Victor Stinnerea3183f2013-12-03 01:08:00 +010081
Carol Willing4e824e92018-09-13 18:28:19 -070082 If the lock is *unlocked*, a :exc:`RuntimeError` is raised.
Victor Stinnerea3183f2013-12-03 01:08:00 +010083
Yury Selivanov8be876e2018-09-11 17:10:37 -070084 .. method:: locked()
85
86 Return ``True`` if the lock is *locked*.
Victor Stinnerea3183f2013-12-03 01:08:00 +010087
88
Victor Stinner8c462c52014-01-24 18:11:43 +010089Event
Yury Selivanov8be876e2018-09-11 17:10:37 -070090=====
Victor Stinner8c462c52014-01-24 18:11:43 +010091
Victor Stinnerea3183f2013-12-03 01:08:00 +010092.. class:: Event(\*, loop=None)
93
Yury Selivanov8be876e2018-09-11 17:10:37 -070094 An event object. Not thread-safe.
Victor Stinnerea3183f2013-12-03 01:08:00 +010095
Yury Selivanov8be876e2018-09-11 17:10:37 -070096 An asyncio event can be used to notify multiple asyncio tasks
97 that some event has happened.
Victor Stinnerea3183f2013-12-03 01:08:00 +010098
Yury Selivanov8be876e2018-09-11 17:10:37 -070099 An Event object manages an internal flag that can be set to *true*
100 with the :meth:`set` method and reset to *false* with the
101 :meth:`clear` method. The :meth:`wait` method blocks until the
102 flag is set to *true*. The flag is set to *false* initially.
Victor Stinner7b2c3c62015-02-26 10:39:16 +0100103
Yury Selivanov7372c3b2018-09-14 15:11:24 -0700104 .. _asyncio_example_sync_event:
105
Yury Selivanov8be876e2018-09-11 17:10:37 -0700106 Example::
Victor Stinnerea3183f2013-12-03 01:08:00 +0100107
Yury Selivanov8be876e2018-09-11 17:10:37 -0700108 async def waiter(event):
Carol Willing4e824e92018-09-13 18:28:19 -0700109 print('waiting for it ...')
Yury Selivanov8be876e2018-09-11 17:10:37 -0700110 await event.wait()
111 print('... got it!')
Victor Stinnerea3183f2013-12-03 01:08:00 +0100112
Yury Selivanov8be876e2018-09-11 17:10:37 -0700113 async def main():
114 # Create an Event object.
115 event = asyncio.Event()
Victor Stinnerea3183f2013-12-03 01:08:00 +0100116
Yury Selivanov8be876e2018-09-11 17:10:37 -0700117 # Spawn a Task to wait until 'event' is set.
118 waiter_task = asyncio.create_task(waiter(event))
Victor Stinnerea3183f2013-12-03 01:08:00 +0100119
Yury Selivanov8be876e2018-09-11 17:10:37 -0700120 # Sleep for 1 second and set the event.
121 await asyncio.sleep(1)
122 event.set()
Victor Stinnerea3183f2013-12-03 01:08:00 +0100123
Yury Selivanov8be876e2018-09-11 17:10:37 -0700124 # Wait until the waiter task is finished.
125 await waiter_task
126
127 asyncio.run(main())
Victor Stinnerea3183f2013-12-03 01:08:00 +0100128
Victor Stinnerbdd574d2015-02-12 22:49:18 +0100129 .. coroutinemethod:: wait()
Victor Stinnerea3183f2013-12-03 01:08:00 +0100130
Yury Selivanov8be876e2018-09-11 17:10:37 -0700131 Wait until the event is set.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100132
Yury Selivanov8be876e2018-09-11 17:10:37 -0700133 If the event is set, return ``True`` immediately.
134 Otherwise block until another task calls :meth:`set`.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100135
Yury Selivanov8be876e2018-09-11 17:10:37 -0700136 .. method:: set()
137
138 Set the event.
139
140 All tasks waiting for event to be set will be immediately
141 awakened.
142
143 .. method:: clear()
144
145 Clear (unset) the event.
146
147 Tasks awaiting on :meth:`wait` will now block until the
148 :meth:`set` method is called again.
149
150 .. method:: is_set()
151
152 Return ``True`` if the event is set.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100153
154
Victor Stinner8c462c52014-01-24 18:11:43 +0100155Condition
Yury Selivanov8be876e2018-09-11 17:10:37 -0700156=========
Victor Stinner8c462c52014-01-24 18:11:43 +0100157
Andrew Svetlovf200ce62014-07-26 19:50:37 +0300158.. class:: Condition(lock=None, \*, loop=None)
Victor Stinnerea3183f2013-12-03 01:08:00 +0100159
Yury Selivanov8be876e2018-09-11 17:10:37 -0700160 A Condition object. Not thread-safe.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100161
Yury Selivanov8be876e2018-09-11 17:10:37 -0700162 An asyncio condition primitive can be used by a task to wait for
Elvis Pranskevichus1fa2ec42018-09-17 19:16:44 -0400163 some event to happen and then get exclusive access to a shared
Yury Selivanov8be876e2018-09-11 17:10:37 -0700164 resource.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100165
Yury Selivanov8be876e2018-09-11 17:10:37 -0700166 In essence, a Condition object combines the functionality
Elvis Pranskevichus1fa2ec42018-09-17 19:16:44 -0400167 of an :class:`Event` and a :class:`Lock`. It is possible to have
168 multiple Condition objects share one Lock, which allows coordinating
Yury Selivanov8be876e2018-09-11 17:10:37 -0700169 exclusive access to a shared resource between different tasks
170 interested in particular states of that shared resource.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100171
Yury Selivanov8be876e2018-09-11 17:10:37 -0700172 The optional *lock* argument must be a :class:`Lock` object or
173 ``None``. In the latter case a new Lock object is created
174 automatically.
Andrew Svetlov28d8d142017-12-09 20:00:05 +0200175
Yury Selivanov8be876e2018-09-11 17:10:37 -0700176 The preferred way to use a Condition is an :keyword:`async with`
177 statement::
178
179 cond = asyncio.Condition()
180
181 # ... later
182 async with cond:
183 await cond.wait()
184
185 which is equivalent to::
186
187 cond = asyncio.Condition()
188
189 # ... later
Kevin Mai-Husan Chiad73ac0e2019-02-14 10:39:25 +0800190 await cond.acquire()
Yury Selivanov8be876e2018-09-11 17:10:37 -0700191 try:
192 await cond.wait()
193 finally:
Kevin Mai-Husan Chiad73ac0e2019-02-14 10:39:25 +0800194 cond.release()
Victor Stinner7b2c3c62015-02-26 10:39:16 +0100195
Victor Stinnerbdd574d2015-02-12 22:49:18 +0100196 .. coroutinemethod:: acquire()
Larry Hastings3732ed22014-03-15 21:13:56 -0700197
198 Acquire the underlying lock.
199
Yury Selivanov8be876e2018-09-11 17:10:37 -0700200 This method waits until the underlying lock is *unlocked*,
201 sets it to *locked* and returns ``True``.
Larry Hastings3732ed22014-03-15 21:13:56 -0700202
Victor Stinnerea3183f2013-12-03 01:08:00 +0100203 .. method:: notify(n=1)
204
Yury Selivanov8be876e2018-09-11 17:10:37 -0700205 Wake up at most *n* tasks (1 by default) waiting on this
206 condition. The method is no-op if no tasks are waiting.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100207
Yury Selivanov8be876e2018-09-11 17:10:37 -0700208 The lock must be acquired before this method is called and
209 released shortly after. If called with an *unlocked* lock
210 a :exc:`RuntimeError` error is raised.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100211
Larry Hastings3732ed22014-03-15 21:13:56 -0700212 .. method:: locked()
213
214 Return ``True`` if the underlying lock is acquired.
215
Victor Stinnerea3183f2013-12-03 01:08:00 +0100216 .. method:: notify_all()
217
Yury Selivanov8be876e2018-09-11 17:10:37 -0700218 Wake up all tasks waiting on this condition.
219
220 This method acts like :meth:`notify`, but wakes up all waiting
221 tasks.
222
223 The lock must be acquired before this method is called and
224 released shortly after. If called with an *unlocked* lock
225 a :exc:`RuntimeError` error is raised.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100226
Larry Hastings3732ed22014-03-15 21:13:56 -0700227 .. method:: release()
228
229 Release the underlying lock.
230
Yury Selivanov8be876e2018-09-11 17:10:37 -0700231 When invoked on an unlocked lock, a :exc:`RuntimeError` is
232 raised.
Larry Hastings3732ed22014-03-15 21:13:56 -0700233
Victor Stinnerbdd574d2015-02-12 22:49:18 +0100234 .. coroutinemethod:: wait()
Victor Stinnerea3183f2013-12-03 01:08:00 +0100235
236 Wait until notified.
237
Yury Selivanov8be876e2018-09-11 17:10:37 -0700238 If the calling task has not acquired the lock when this method is
Victor Stinnerea3183f2013-12-03 01:08:00 +0100239 called, a :exc:`RuntimeError` is raised.
240
Yury Selivanov8be876e2018-09-11 17:10:37 -0700241 This method releases the underlying lock, and then blocks until
242 it is awakened by a :meth:`notify` or :meth:`notify_all` call.
243 Once awakened, the Condition re-acquires its lock and this method
244 returns ``True``.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100245
Victor Stinnerbdd574d2015-02-12 22:49:18 +0100246 .. coroutinemethod:: wait_for(predicate)
Victor Stinnerea3183f2013-12-03 01:08:00 +0100247
Yury Selivanov8be876e2018-09-11 17:10:37 -0700248 Wait until a predicate becomes *true*.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100249
Yury Selivanov8be876e2018-09-11 17:10:37 -0700250 The predicate must be a callable which result will be
251 interpreted as a boolean value. The final value is the
252 return value.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100253
254
Victor Stinner8c462c52014-01-24 18:11:43 +0100255Semaphore
Yury Selivanov8be876e2018-09-11 17:10:37 -0700256=========
Victor Stinner8c462c52014-01-24 18:11:43 +0100257
Victor Stinnerea3183f2013-12-03 01:08:00 +0100258.. class:: Semaphore(value=1, \*, loop=None)
259
Yury Selivanov8be876e2018-09-11 17:10:37 -0700260 A Semaphore object. Not thread-safe.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100261
262 A semaphore manages an internal counter which is decremented by each
Yury Selivanov8be876e2018-09-11 17:10:37 -0700263 :meth:`acquire` call and incremented by each :meth:`release` call.
264 The counter can never go below zero; when :meth:`acquire` finds
265 that it is zero, it blocks, waiting until some task calls
266 :meth:`release`.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100267
Yury Selivanov8be876e2018-09-11 17:10:37 -0700268 The optional *value* argument gives the initial value for the
269 internal counter (``1`` by default). If the given value is
270 less than ``0`` a :exc:`ValueError` is raised.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100271
Yury Selivanov8be876e2018-09-11 17:10:37 -0700272 The preferred way to use a Semaphore is an :keyword:`async with`
273 statement::
Andrew Svetlov28d8d142017-12-09 20:00:05 +0200274
Yury Selivanov8be876e2018-09-11 17:10:37 -0700275 sem = asyncio.Semaphore(10)
276
277 # ... later
278 async with sem:
279 # work with shared resource
280
281 which is equivalent to::
282
283 sem = asyncio.Semaphore(10)
284
285 # ... later
286 await sem.acquire()
287 try:
288 # work with shared resource
289 finally:
290 sem.release()
Victor Stinner7b2c3c62015-02-26 10:39:16 +0100291
Victor Stinnerbdd574d2015-02-12 22:49:18 +0100292 .. coroutinemethod:: acquire()
Victor Stinnerea3183f2013-12-03 01:08:00 +0100293
294 Acquire a semaphore.
295
Yury Selivanov8be876e2018-09-11 17:10:37 -0700296 If the internal counter is greater than zero, decrement
Elvis Pranskevichus1fa2ec42018-09-17 19:16:44 -0400297 it by one and return ``True`` immediately. If it is zero, wait
Yury Selivanov8be876e2018-09-11 17:10:37 -0700298 until a :meth:`release` is called and return ``True``.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100299
300 .. method:: locked()
301
302 Returns ``True`` if semaphore can not be acquired immediately.
303
Andrew Svetlovb79e0122015-05-08 14:13:41 +0300304 .. method:: release()
Victor Stinnerea3183f2013-12-03 01:08:00 +0100305
Yury Selivanov8be876e2018-09-11 17:10:37 -0700306 Release a semaphore, incrementing the internal counter by one.
307 Can wake up a task waiting to acquire the semaphore.
308
309 Unlike :class:`BoundedSemaphore`, :class:`Semaphore` allows
Elvis Pranskevichus1fa2ec42018-09-17 19:16:44 -0400310 making more ``release()`` calls than ``acquire()`` calls.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100311
312
Victor Stinner8c462c52014-01-24 18:11:43 +0100313BoundedSemaphore
Yury Selivanov8be876e2018-09-11 17:10:37 -0700314================
Victor Stinner8c462c52014-01-24 18:11:43 +0100315
Victor Stinnerea3183f2013-12-03 01:08:00 +0100316.. class:: BoundedSemaphore(value=1, \*, loop=None)
317
Yury Selivanov8be876e2018-09-11 17:10:37 -0700318 A bounded semaphore object. Not thread-safe.
Victor Stinnerea3183f2013-12-03 01:08:00 +0100319
Yury Selivanov8be876e2018-09-11 17:10:37 -0700320 Bounded Semaphore is a version of :class:`Semaphore` that raises
321 a :exc:`ValueError` in :meth:`~Semaphore.release` if it
322 increases the internal counter above the initial *value*.
Andrew Svetlov28d8d142017-12-09 20:00:05 +0200323
324
Yury Selivanov8be876e2018-09-11 17:10:37 -0700325---------
Andrew Svetlov28d8d142017-12-09 20:00:05 +0200326
Andrew Svetlov28d8d142017-12-09 20:00:05 +0200327
328.. deprecated:: 3.7
329
Yury Selivanov8be876e2018-09-11 17:10:37 -0700330 Acquiring a lock using ``await lock`` or ``yield from lock`` and/or
Andrew Svetlov28d8d142017-12-09 20:00:05 +0200331 :keyword:`with` statement (``with await lock``, ``with (yield from
Yury Selivanov8be876e2018-09-11 17:10:37 -0700332 lock)``) is deprecated. Use ``async with lock`` instead.