blob: 690735469d0bf688ba4c85c18bebc3b2ce314669 [file] [log] [blame]
Antoine Pitrou64a467d2010-12-12 20:34:49 +00001:mod:`threading` --- Thread-based parallelism
2=============================================
Georg Brandl116aa622007-08-15 14:28:22 +00003
4.. module:: threading
Antoine Pitrou64a467d2010-12-12 20:34:49 +00005 :synopsis: Thread-based parallelism.
Georg Brandl116aa622007-08-15 14:28:22 +00006
Raymond Hettinger10480942011-01-10 03:26:08 +00007**Source code:** :source:`Lib/threading.py`
Georg Brandl116aa622007-08-15 14:28:22 +00008
Raymond Hettinger4f707fd2011-01-10 19:54:11 +00009--------------
10
Georg Brandl2067bfd2008-05-25 13:05:15 +000011This module constructs higher-level threading interfaces on top of the lower
12level :mod:`_thread` module. See also the :mod:`queue` module.
Georg Brandl116aa622007-08-15 14:28:22 +000013
Antoine Pitroub43c4ca2017-09-18 22:04:20 +020014.. versionchanged:: 3.7
15 This module used to be optional, it is now always available.
Georg Brandl116aa622007-08-15 14:28:22 +000016
Benjamin Peterson8bdd5452008-08-18 22:38:41 +000017.. note::
18
Benjamin Petersonb3085c92008-09-01 23:09:31 +000019 While they are not listed below, the ``camelCase`` names used for some
20 methods and functions in this module in the Python 2.x series are still
21 supported by this module.
Benjamin Peterson8bdd5452008-08-18 22:38:41 +000022
Antoine Pitrou00342812011-01-06 16:31:28 +000023
R David Murrayef4d2862012-10-06 14:35:35 -040024This module defines the following functions:
Georg Brandl116aa622007-08-15 14:28:22 +000025
26
Benjamin Peterson672b8032008-06-11 19:14:14 +000027.. function:: active_count()
Georg Brandl116aa622007-08-15 14:28:22 +000028
29 Return the number of :class:`Thread` objects currently alive. The returned
Benjamin Peterson4ac9ce42009-10-04 14:49:41 +000030 count is equal to the length of the list returned by :func:`.enumerate`.
Georg Brandl116aa622007-08-15 14:28:22 +000031
32
Benjamin Peterson672b8032008-06-11 19:14:14 +000033.. function:: current_thread()
Georg Brandl116aa622007-08-15 14:28:22 +000034
35 Return the current :class:`Thread` object, corresponding to the caller's thread
36 of control. If the caller's thread of control was not created through the
37 :mod:`threading` module, a dummy thread object with limited functionality is
38 returned.
39
40
Victor Stinnercd590a72019-05-28 00:39:52 +020041.. function:: excepthook(args, /)
42
43 Handle uncaught exception raised by :func:`Thread.run`.
44
45 The *args* argument has the following attributes:
46
47 * *exc_type*: Exception type.
48 * *exc_value*: Exception value, can be ``None``.
49 * *exc_traceback*: Exception traceback, can be ``None``.
50 * *thread*: Thread which raised the exception, can be ``None``.
51
52 If *exc_type* is :exc:`SystemExit`, the exception is silently ignored.
53 Otherwise, the exception is printed out on :data:`sys.stderr`.
54
55 If this function raises an exception, :func:`sys.excepthook` is called to
56 handle it.
57
58 :func:`threading.excepthook` can be overridden to control how uncaught
59 exceptions raised by :func:`Thread.run` are handled.
60
Victor Stinner212646c2019-06-14 18:03:22 +020061 Storing *exc_value* using a custom hook can create a reference cycle. It
62 should be cleared explicitly to break the reference cycle when the
63 exception is no longer needed.
64
Daniel Baskal91528f42019-10-22 23:37:47 +030065 Storing *thread* using a custom hook can resurrect it if it is set to an
66 object which is being finalized. Avoid storing *thread* after the custom
Victor Stinner212646c2019-06-14 18:03:22 +020067 hook completes to avoid resurrecting objects.
68
Victor Stinnercd590a72019-05-28 00:39:52 +020069 .. seealso::
70 :func:`sys.excepthook` handles uncaught exceptions.
71
72 .. versionadded:: 3.8
73
Mario Corchero750c5ab2020-11-12 18:27:44 +010074.. data:: __excepthook__
75
76 Holds the original value of :func:`threading.excepthook`. It is saved so that the
77 original value can be restored in case they happen to get replaced with
78 broken or alternative objects.
79
80 .. versionadded:: 3.10
Victor Stinnercd590a72019-05-28 00:39:52 +020081
Victor Stinner2a129742011-05-30 23:02:52 +020082.. function:: get_ident()
83
84 Return the 'thread identifier' of the current thread. This is a nonzero
85 integer. Its value has no direct meaning; it is intended as a magic cookie
86 to be used e.g. to index a dictionary of thread-specific data. Thread
87 identifiers may be recycled when a thread exits and another thread is
88 created.
89
90 .. versionadded:: 3.3
91
92
Jake Teslerb121f632019-05-22 08:43:17 -070093.. function:: get_native_id()
94
95 Return the native integral Thread ID of the current thread assigned by the kernel.
96 This is a non-negative integer.
97 Its value may be used to uniquely identify this particular thread system-wide
98 (until the thread terminates, after which the value may be recycled by the OS).
99
Michael Feltd0eeb932019-06-14 00:34:46 +0200100 .. availability:: Windows, FreeBSD, Linux, macOS, OpenBSD, NetBSD, AIX.
Jake Teslerb121f632019-05-22 08:43:17 -0700101
102 .. versionadded:: 3.8
103
104
Georg Brandl116aa622007-08-15 14:28:22 +0000105.. function:: enumerate()
106
Benjamin Peterson672b8032008-06-11 19:14:14 +0000107 Return a list of all :class:`Thread` objects currently alive. The list
108 includes daemonic threads, dummy thread objects created by
109 :func:`current_thread`, and the main thread. It excludes terminated threads
110 and threads that have not yet been started.
Georg Brandl116aa622007-08-15 14:28:22 +0000111
112
Andrew Svetlov58b5c5a2013-09-04 07:01:07 +0300113.. function:: main_thread()
114
115 Return the main :class:`Thread` object. In normal conditions, the
116 main thread is the thread from which the Python interpreter was
117 started.
118
119 .. versionadded:: 3.4
120
121
Georg Brandl116aa622007-08-15 14:28:22 +0000122.. function:: settrace(func)
123
124 .. index:: single: trace function
125
126 Set a trace function for all threads started from the :mod:`threading` module.
127 The *func* will be passed to :func:`sys.settrace` for each thread, before its
Serhiy Storchaka9e0ae532013-08-24 00:23:38 +0300128 :meth:`~Thread.run` method is called.
Georg Brandl116aa622007-08-15 14:28:22 +0000129
Georg Brandl116aa622007-08-15 14:28:22 +0000130
Mario Corchero0001a1b2020-11-04 10:27:43 +0100131.. function:: gettrace()
132
133 .. index::
134 single: trace function
135 single: debugger
136
137 Get the trace function as set by :func:`settrace`.
138
139 .. versionadded:: 3.10
140
141
Georg Brandl116aa622007-08-15 14:28:22 +0000142.. function:: setprofile(func)
143
144 .. index:: single: profile function
145
146 Set a profile function for all threads started from the :mod:`threading` module.
147 The *func* will be passed to :func:`sys.setprofile` for each thread, before its
Serhiy Storchaka9e0ae532013-08-24 00:23:38 +0300148 :meth:`~Thread.run` method is called.
Georg Brandl116aa622007-08-15 14:28:22 +0000149
Georg Brandl116aa622007-08-15 14:28:22 +0000150
Mario Corchero0001a1b2020-11-04 10:27:43 +0100151.. function:: getprofile()
152
153 .. index:: single: profile function
154
155 Get the profiler function as set by :func:`setprofile`.
156
157 .. versionadded:: 3.10
158
159
Georg Brandl116aa622007-08-15 14:28:22 +0000160.. function:: stack_size([size])
161
162 Return the thread stack size used when creating new threads. The optional
163 *size* argument specifies the stack size to be used for subsequently created
164 threads, and must be 0 (use platform or configured default) or a positive
Martin Panter31e7f502015-08-31 03:15:52 +0000165 integer value of at least 32,768 (32 KiB). If *size* is not specified,
166 0 is used. If changing the thread stack size is
Georg Brandl9a13b432012-04-05 09:53:04 +0200167 unsupported, a :exc:`RuntimeError` is raised. If the specified stack size is
Serhiy Storchakaf8def282013-02-16 17:29:56 +0200168 invalid, a :exc:`ValueError` is raised and the stack size is unmodified. 32 KiB
Georg Brandl116aa622007-08-15 14:28:22 +0000169 is currently the minimum supported stack size value to guarantee sufficient
170 stack space for the interpreter itself. Note that some platforms may have
171 particular restrictions on values for the stack size, such as requiring a
Serhiy Storchakaf8def282013-02-16 17:29:56 +0200172 minimum stack size > 32 KiB or requiring allocation in multiples of the system
Georg Brandl116aa622007-08-15 14:28:22 +0000173 memory page size - platform documentation should be referred to for more
Serhiy Storchakaf8def282013-02-16 17:29:56 +0200174 information (4 KiB pages are common; using multiples of 4096 for the stack size is
Georg Brandl116aa622007-08-15 14:28:22 +0000175 the suggested approach in the absence of more specific information).
Cheryl Sabella2d6097d2018-10-12 10:55:20 -0400176
177 .. availability:: Windows, systems with POSIX threads.
Georg Brandl116aa622007-08-15 14:28:22 +0000178
Georg Brandl116aa622007-08-15 14:28:22 +0000179
Antoine Pitrou7c3e5772010-04-14 15:44:10 +0000180This module also defines the following constant:
181
182.. data:: TIMEOUT_MAX
183
184 The maximum value allowed for the *timeout* parameter of blocking functions
185 (:meth:`Lock.acquire`, :meth:`RLock.acquire`, :meth:`Condition.wait`, etc.).
Georg Brandl6faee4e2010-09-21 14:48:28 +0000186 Specifying a timeout greater than this value will raise an
Antoine Pitrou7c3e5772010-04-14 15:44:10 +0000187 :exc:`OverflowError`.
188
Antoine Pitrouadbc0092010-04-19 14:05:51 +0000189 .. versionadded:: 3.2
Antoine Pitrou7c3e5772010-04-14 15:44:10 +0000190
Georg Brandl67b21b72010-08-17 15:07:14 +0000191
R David Murrayef4d2862012-10-06 14:35:35 -0400192This module defines a number of classes, which are detailed in the sections
193below.
Georg Brandl116aa622007-08-15 14:28:22 +0000194
195The design of this module is loosely based on Java's threading model. However,
196where Java makes locks and condition variables basic behavior of every object,
197they are separate objects in Python. Python's :class:`Thread` class supports a
198subset of the behavior of Java's Thread class; currently, there are no
199priorities, no thread groups, and threads cannot be destroyed, stopped,
200suspended, resumed, or interrupted. The static methods of Java's Thread class,
201when implemented, are mapped to module-level functions.
202
203All of the methods described below are executed atomically.
204
205
R David Murrayef4d2862012-10-06 14:35:35 -0400206Thread-Local Data
207-----------------
208
209Thread-local data is data whose values are thread specific. To manage
210thread-local data, just create an instance of :class:`local` (or a
211subclass) and store attributes on it::
212
213 mydata = threading.local()
214 mydata.x = 1
215
216The instance's values will be different for separate threads.
217
218
219.. class:: local()
220
221 A class that represents thread-local data.
222
223 For more details and extensive examples, see the documentation string of the
224 :mod:`_threading_local` module.
225
226
Georg Brandla971c652008-11-07 09:39:56 +0000227.. _thread-objects:
228
229Thread Objects
230--------------
231
R David Murrayef4d2862012-10-06 14:35:35 -0400232The :class:`Thread` class represents an activity that is run in a separate
233thread of control. There are two ways to specify the activity: by passing a
234callable object to the constructor, or by overriding the :meth:`~Thread.run`
235method in a subclass. No other methods (except for the constructor) should be
236overridden in a subclass. In other words, *only* override the
237:meth:`~Thread.__init__` and :meth:`~Thread.run` methods of this class.
Georg Brandla971c652008-11-07 09:39:56 +0000238
239Once a thread object is created, its activity must be started by calling the
Antoine Pitrou2c9f1042012-04-10 22:35:53 +0200240thread's :meth:`~Thread.start` method. This invokes the :meth:`~Thread.run`
241method in a separate thread of control.
Georg Brandla971c652008-11-07 09:39:56 +0000242
243Once the thread's activity is started, the thread is considered 'alive'. It
Antoine Pitrou2c9f1042012-04-10 22:35:53 +0200244stops being alive when its :meth:`~Thread.run` method terminates -- either
245normally, or by raising an unhandled exception. The :meth:`~Thread.is_alive`
246method tests whether the thread is alive.
Georg Brandla971c652008-11-07 09:39:56 +0000247
Antoine Pitrou2c9f1042012-04-10 22:35:53 +0200248Other threads can call a thread's :meth:`~Thread.join` method. This blocks
249the calling thread until the thread whose :meth:`~Thread.join` method is
250called is terminated.
Georg Brandla971c652008-11-07 09:39:56 +0000251
252A thread has a name. The name can be passed to the constructor, and read or
Antoine Pitrou2c9f1042012-04-10 22:35:53 +0200253changed through the :attr:`~Thread.name` attribute.
Georg Brandla971c652008-11-07 09:39:56 +0000254
Victor Stinnercd590a72019-05-28 00:39:52 +0200255If the :meth:`~Thread.run` method raises an exception,
256:func:`threading.excepthook` is called to handle it. By default,
257:func:`threading.excepthook` ignores silently :exc:`SystemExit`.
258
Georg Brandla971c652008-11-07 09:39:56 +0000259A thread can be flagged as a "daemon thread". The significance of this flag is
260that the entire Python program exits when only daemon threads are left. The
261initial value is inherited from the creating thread. The flag can be set
Antoine Pitrou61d85ba2012-04-10 22:51:26 +0200262through the :attr:`~Thread.daemon` property or the *daemon* constructor
263argument.
Georg Brandla971c652008-11-07 09:39:56 +0000264
Antoine Pitrou38b82542013-02-15 21:27:18 +0100265.. note::
266 Daemon threads are abruptly stopped at shutdown. Their resources (such
267 as open files, database transactions, etc.) may not be released properly.
268 If you want your threads to stop gracefully, make them non-daemonic and
269 use a suitable signalling mechanism such as an :class:`Event`.
270
Georg Brandla971c652008-11-07 09:39:56 +0000271There is a "main thread" object; this corresponds to the initial thread of
272control in the Python program. It is not a daemon thread.
273
274There is the possibility that "dummy thread objects" are created. These are
275thread objects corresponding to "alien threads", which are threads of control
276started outside the threading module, such as directly from C code. Dummy
277thread objects have limited functionality; they are always considered alive and
Antoine Pitrou2c9f1042012-04-10 22:35:53 +0200278daemonic, and cannot be :meth:`~Thread.join`\ ed. They are never deleted,
279since it is impossible to detect the termination of alien threads.
Georg Brandla971c652008-11-07 09:39:56 +0000280
281
Ezio Melotti8b616112012-09-08 20:49:18 +0300282.. class:: Thread(group=None, target=None, name=None, args=(), kwargs={}, *, \
283 daemon=None)
Georg Brandla971c652008-11-07 09:39:56 +0000284
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000285 This constructor should always be called with keyword arguments. Arguments
286 are:
Georg Brandla971c652008-11-07 09:39:56 +0000287
288 *group* should be ``None``; reserved for future extension when a
289 :class:`ThreadGroup` class is implemented.
290
291 *target* is the callable object to be invoked by the :meth:`run` method.
292 Defaults to ``None``, meaning nothing is called.
293
Victor Stinner98c16c92020-09-23 23:21:19 +0200294 *name* is the thread name. By default, a unique name is constructed
295 of the form "Thread-*N*" where *N* is a small decimal number,
296 or "Thread-*N* (target)" where "target" is ``target.__name__`` if the
297 *target* argument is specified.
Georg Brandla971c652008-11-07 09:39:56 +0000298
299 *args* is the argument tuple for the target invocation. Defaults to ``()``.
300
301 *kwargs* is a dictionary of keyword arguments for the target invocation.
302 Defaults to ``{}``.
303
Antoine Pitrou0bd4deb2011-02-25 22:07:43 +0000304 If not ``None``, *daemon* explicitly sets whether the thread is daemonic.
305 If ``None`` (the default), the daemonic property is inherited from the
306 current thread.
307
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000308 If the subclass overrides the constructor, it must make sure to invoke the
309 base class constructor (``Thread.__init__()``) before doing anything else to
310 the thread.
Georg Brandla971c652008-11-07 09:39:56 +0000311
Victor Stinner98c16c92020-09-23 23:21:19 +0200312 .. versionchanged:: 3.10
313 Use the *target* name if *name* argument is omitted.
314
Antoine Pitrou0bd4deb2011-02-25 22:07:43 +0000315 .. versionchanged:: 3.3
316 Added the *daemon* argument.
317
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000318 .. method:: start()
Georg Brandla971c652008-11-07 09:39:56 +0000319
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000320 Start the thread's activity.
Georg Brandla971c652008-11-07 09:39:56 +0000321
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000322 It must be called at most once per thread object. It arranges for the
Antoine Pitrou2c9f1042012-04-10 22:35:53 +0200323 object's :meth:`~Thread.run` method to be invoked in a separate thread
324 of control.
Georg Brandla971c652008-11-07 09:39:56 +0000325
Brian Curtinbd0c8972011-01-31 19:35:02 +0000326 This method will raise a :exc:`RuntimeError` if called more than once
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000327 on the same thread object.
Georg Brandla971c652008-11-07 09:39:56 +0000328
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000329 .. method:: run()
Georg Brandla971c652008-11-07 09:39:56 +0000330
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000331 Method representing the thread's activity.
Georg Brandla971c652008-11-07 09:39:56 +0000332
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000333 You may override this method in a subclass. The standard :meth:`run`
334 method invokes the callable object passed to the object's constructor as
Mathieu Dupuy29d018a2019-04-23 15:01:09 +0200335 the *target* argument, if any, with positional and keyword arguments taken
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000336 from the *args* and *kwargs* arguments, respectively.
Georg Brandla971c652008-11-07 09:39:56 +0000337
Georg Brandl7f01a132009-09-16 15:58:14 +0000338 .. method:: join(timeout=None)
Georg Brandla971c652008-11-07 09:39:56 +0000339
Antoine Pitrou2c9f1042012-04-10 22:35:53 +0200340 Wait until the thread terminates. This blocks the calling thread until
341 the thread whose :meth:`~Thread.join` method is called terminates -- either
Martin Panter972e04e2016-12-24 07:28:26 +0000342 normally or through an unhandled exception -- or until the optional
Antoine Pitrou2c9f1042012-04-10 22:35:53 +0200343 timeout occurs.
Georg Brandla971c652008-11-07 09:39:56 +0000344
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000345 When the *timeout* argument is present and not ``None``, it should be a
346 floating point number specifying a timeout for the operation in seconds
Antoine Pitrou2c9f1042012-04-10 22:35:53 +0200347 (or fractions thereof). As :meth:`~Thread.join` always returns ``None``,
348 you must call :meth:`~Thread.is_alive` after :meth:`~Thread.join` to
349 decide whether a timeout happened -- if the thread is still alive, the
350 :meth:`~Thread.join` call timed out.
Georg Brandla971c652008-11-07 09:39:56 +0000351
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000352 When the *timeout* argument is not present or ``None``, the operation will
353 block until the thread terminates.
Georg Brandla971c652008-11-07 09:39:56 +0000354
Antoine Pitrou2c9f1042012-04-10 22:35:53 +0200355 A thread can be :meth:`~Thread.join`\ ed many times.
Georg Brandla971c652008-11-07 09:39:56 +0000356
Antoine Pitrou2c9f1042012-04-10 22:35:53 +0200357 :meth:`~Thread.join` raises a :exc:`RuntimeError` if an attempt is made
358 to join the current thread as that would cause a deadlock. It is also
359 an error to :meth:`~Thread.join` a thread before it has been started
360 and attempts to do so raise the same exception.
Georg Brandla971c652008-11-07 09:39:56 +0000361
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000362 .. attribute:: name
Georg Brandla971c652008-11-07 09:39:56 +0000363
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000364 A string used for identification purposes only. It has no semantics.
365 Multiple threads may be given the same name. The initial name is set by
366 the constructor.
Georg Brandla971c652008-11-07 09:39:56 +0000367
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000368 .. method:: getName()
369 setName()
Georg Brandla971c652008-11-07 09:39:56 +0000370
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000371 Old getter/setter API for :attr:`~Thread.name`; use it directly as a
372 property instead.
Georg Brandla971c652008-11-07 09:39:56 +0000373
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000374 .. attribute:: ident
Georg Brandla971c652008-11-07 09:39:56 +0000375
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000376 The 'thread identifier' of this thread or ``None`` if the thread has not
Benjamin Peterson236329e2017-09-26 23:13:15 -0700377 been started. This is a nonzero integer. See the :func:`get_ident`
378 function. Thread identifiers may be recycled when a thread exits and
379 another thread is created. The identifier is available even after the
380 thread has exited.
Georg Brandla971c652008-11-07 09:39:56 +0000381
Jake Teslerb121f632019-05-22 08:43:17 -0700382 .. attribute:: native_id
383
Antoinea6ac2392020-06-13 00:50:18 +0200384 The Thread ID (``TID``) of this thread, as assigned by the OS (kernel).
Jake Teslerb121f632019-05-22 08:43:17 -0700385 This is a non-negative integer, or ``None`` if the thread has not
386 been started. See the :func:`get_native_id` function.
Antoinea6ac2392020-06-13 00:50:18 +0200387 This value may be used to uniquely identify this particular thread
388 system-wide (until the thread terminates, after which the value
389 may be recycled by the OS).
Jake Teslerb121f632019-05-22 08:43:17 -0700390
391 .. note::
392
393 Similar to Process IDs, Thread IDs are only valid (guaranteed unique
394 system-wide) from the time the thread is created until the thread
395 has been terminated.
396
Jake Tesler84846b02019-07-30 14:41:46 -0700397 .. availability:: Requires :func:`get_native_id` function.
Jake Teslerb121f632019-05-22 08:43:17 -0700398
399 .. versionadded:: 3.8
400
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000401 .. method:: is_alive()
Georg Brandla971c652008-11-07 09:39:56 +0000402
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000403 Return whether the thread is alive.
Georg Brandl770b0be2009-01-02 20:10:05 +0000404
Antoine Pitrou2c9f1042012-04-10 22:35:53 +0200405 This method returns ``True`` just before the :meth:`~Thread.run` method
406 starts until just after the :meth:`~Thread.run` method terminates. The
407 module function :func:`.enumerate` returns a list of all alive threads.
Georg Brandl770b0be2009-01-02 20:10:05 +0000408
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000409 .. attribute:: daemon
Georg Brandl770b0be2009-01-02 20:10:05 +0000410
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000411 A boolean value indicating whether this thread is a daemon thread (True)
Antoine Pitrou2c9f1042012-04-10 22:35:53 +0200412 or not (False). This must be set before :meth:`~Thread.start` is called,
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000413 otherwise :exc:`RuntimeError` is raised. Its initial value is inherited
414 from the creating thread; the main thread is not a daemon thread and
Antoine Pitrou2c9f1042012-04-10 22:35:53 +0200415 therefore all threads created in the main thread default to
416 :attr:`~Thread.daemon` = ``False``.
Georg Brandla971c652008-11-07 09:39:56 +0000417
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000418 The entire Python program exits when no alive non-daemon threads are left.
Georg Brandla971c652008-11-07 09:39:56 +0000419
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000420 .. method:: isDaemon()
421 setDaemon()
Georg Brandla971c652008-11-07 09:39:56 +0000422
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000423 Old getter/setter API for :attr:`~Thread.daemon`; use it directly as a
424 property instead.
Georg Brandl770b0be2009-01-02 20:10:05 +0000425
426
Antoine Pitroud6d17c52011-02-28 22:04:51 +0000427.. impl-detail::
428
Victor Stinnerbb0b0852020-08-14 12:20:05 +0200429 In CPython, due to the :term:`Global Interpreter Lock
430 <global interpreter lock>`, only one thread
Antoine Pitroud6d17c52011-02-28 22:04:51 +0000431 can execute Python code at once (even though certain performance-oriented
432 libraries might overcome this limitation).
Ezio Melotti6d043fc2013-01-18 19:58:47 +0200433 If you want your application to make better use of the computational
Antoine Pitroud6d17c52011-02-28 22:04:51 +0000434 resources of multi-core machines, you are advised to use
435 :mod:`multiprocessing` or :class:`concurrent.futures.ProcessPoolExecutor`.
436 However, threading is still an appropriate model if you want to run
437 multiple I/O-bound tasks simultaneously.
438
439
Georg Brandl116aa622007-08-15 14:28:22 +0000440.. _lock-objects:
441
442Lock Objects
443------------
444
445A primitive lock is a synchronization primitive that is not owned by a
446particular thread when locked. In Python, it is currently the lowest level
Georg Brandl2067bfd2008-05-25 13:05:15 +0000447synchronization primitive available, implemented directly by the :mod:`_thread`
Georg Brandl116aa622007-08-15 14:28:22 +0000448extension module.
449
450A primitive lock is in one of two states, "locked" or "unlocked". It is created
Antoine Pitrou2c9f1042012-04-10 22:35:53 +0200451in the unlocked state. It has two basic methods, :meth:`~Lock.acquire` and
452:meth:`~Lock.release`. When the state is unlocked, :meth:`~Lock.acquire`
453changes the state to locked and returns immediately. When the state is locked,
454:meth:`~Lock.acquire` blocks until a call to :meth:`~Lock.release` in another
455thread changes it to unlocked, then the :meth:`~Lock.acquire` call resets it
456to locked and returns. The :meth:`~Lock.release` method should only be
457called in the locked state; it changes the state to unlocked and returns
458immediately. If an attempt is made to release an unlocked lock, a
459:exc:`RuntimeError` will be raised.
Georg Brandl116aa622007-08-15 14:28:22 +0000460
Serhiy Storchaka14867992014-09-10 23:43:41 +0300461Locks also support the :ref:`context management protocol <with-locks>`.
Antoine Pitroub96a3542012-04-10 22:47:55 +0200462
Antoine Pitrou2c9f1042012-04-10 22:35:53 +0200463When more than one thread is blocked in :meth:`~Lock.acquire` waiting for the
464state to turn to unlocked, only one thread proceeds when a :meth:`~Lock.release`
465call resets the state to unlocked; which one of the waiting threads proceeds
466is not defined, and may vary across implementations.
Georg Brandl116aa622007-08-15 14:28:22 +0000467
468All methods are executed atomically.
469
470
R David Murrayef4d2862012-10-06 14:35:35 -0400471.. class:: Lock()
Georg Brandl116aa622007-08-15 14:28:22 +0000472
R David Murrayef4d2862012-10-06 14:35:35 -0400473 The class implementing primitive lock objects. Once a thread has acquired a
474 lock, subsequent attempts to acquire it block, until it is released; any
475 thread may release it.
Georg Brandl116aa622007-08-15 14:28:22 +0000476
csabella56ddfd22017-05-31 20:14:19 -0400477 Note that ``Lock`` is actually a factory function which returns an instance
478 of the most efficient version of the concrete Lock class that is supported
479 by the platform.
Antoine Pitrou810023d2010-12-15 22:59:16 +0000480
Georg Brandl67b21b72010-08-17 15:07:14 +0000481
R David Murrayef4d2862012-10-06 14:35:35 -0400482 .. method:: acquire(blocking=True, timeout=-1)
Georg Brandl116aa622007-08-15 14:28:22 +0000483
R David Murrayef4d2862012-10-06 14:35:35 -0400484 Acquire a lock, blocking or non-blocking.
Georg Brandl116aa622007-08-15 14:28:22 +0000485
R David Murrayef4d2862012-10-06 14:35:35 -0400486 When invoked with the *blocking* argument set to ``True`` (the default),
487 block until the lock is unlocked, then set it to locked and return ``True``.
Georg Brandl116aa622007-08-15 14:28:22 +0000488
R David Murrayef4d2862012-10-06 14:35:35 -0400489 When invoked with the *blocking* argument set to ``False``, do not block.
490 If a call with *blocking* set to ``True`` would block, return ``False``
491 immediately; otherwise, set the lock to locked and return ``True``.
Georg Brandl116aa622007-08-15 14:28:22 +0000492
R David Murrayef4d2862012-10-06 14:35:35 -0400493 When invoked with the floating-point *timeout* argument set to a positive
494 value, block for at most the number of seconds specified by *timeout*
Georg Brandlb19ef182013-10-06 10:48:08 +0200495 and as long as the lock cannot be acquired. A *timeout* argument of ``-1``
R David Murrayef4d2862012-10-06 14:35:35 -0400496 specifies an unbounded wait. It is forbidden to specify a *timeout*
497 when *blocking* is false.
498
499 The return value is ``True`` if the lock is acquired successfully,
500 ``False`` if not (for example if the *timeout* expired).
501
502 .. versionchanged:: 3.2
503 The *timeout* parameter is new.
504
505 .. versionchanged:: 3.2
Benjamin Peterson5b10d512018-09-12 13:48:03 -0700506 Lock acquisition can now be interrupted by signals on POSIX if the
507 underlying threading implementation supports it.
R David Murrayef4d2862012-10-06 14:35:35 -0400508
509
510 .. method:: release()
511
512 Release a lock. This can be called from any thread, not only the thread
513 which has acquired the lock.
514
515 When the lock is locked, reset it to unlocked, and return. If any other threads
516 are blocked waiting for the lock to become unlocked, allow exactly one of them
517 to proceed.
518
519 When invoked on an unlocked lock, a :exc:`RuntimeError` is raised.
520
521 There is no return value.
Georg Brandl116aa622007-08-15 14:28:22 +0000522
idomicfdafa1d2019-12-01 15:07:39 -0500523 .. method:: locked()
Grant Jenksef8844f2020-01-17 14:54:44 -0800524
idomicfdafa1d2019-12-01 15:07:39 -0500525 Return true if the lock is acquired.
526
527
Georg Brandl116aa622007-08-15 14:28:22 +0000528
529.. _rlock-objects:
530
531RLock Objects
532-------------
533
534A reentrant lock is a synchronization primitive that may be acquired multiple
535times by the same thread. Internally, it uses the concepts of "owning thread"
536and "recursion level" in addition to the locked/unlocked state used by primitive
537locks. In the locked state, some thread owns the lock; in the unlocked state,
538no thread owns it.
539
Antoine Pitrou2c9f1042012-04-10 22:35:53 +0200540To lock the lock, a thread calls its :meth:`~RLock.acquire` method; this
541returns once the thread owns the lock. To unlock the lock, a thread calls
542its :meth:`~Lock.release` method. :meth:`~Lock.acquire`/:meth:`~Lock.release`
543call pairs may be nested; only the final :meth:`~Lock.release` (the
544:meth:`~Lock.release` of the outermost pair) resets the lock to unlocked and
545allows another thread blocked in :meth:`~Lock.acquire` to proceed.
Georg Brandl116aa622007-08-15 14:28:22 +0000546
Serhiy Storchaka14867992014-09-10 23:43:41 +0300547Reentrant locks also support the :ref:`context management protocol <with-locks>`.
Georg Brandl116aa622007-08-15 14:28:22 +0000548
549
R David Murrayef4d2862012-10-06 14:35:35 -0400550.. class:: RLock()
Georg Brandl116aa622007-08-15 14:28:22 +0000551
R David Murrayef4d2862012-10-06 14:35:35 -0400552 This class implements reentrant lock objects. A reentrant lock must be
553 released by the thread that acquired it. Once a thread has acquired a
554 reentrant lock, the same thread may acquire it again without blocking; the
555 thread must release it once for each time it has acquired it.
Georg Brandl116aa622007-08-15 14:28:22 +0000556
R David Murrayef4d2862012-10-06 14:35:35 -0400557 Note that ``RLock`` is actually a factory function which returns an instance
558 of the most efficient version of the concrete RLock class that is supported
559 by the platform.
Antoine Pitrouadbc0092010-04-19 14:05:51 +0000560
Georg Brandl116aa622007-08-15 14:28:22 +0000561
R David Murrayef4d2862012-10-06 14:35:35 -0400562 .. method:: acquire(blocking=True, timeout=-1)
Georg Brandl116aa622007-08-15 14:28:22 +0000563
R David Murrayef4d2862012-10-06 14:35:35 -0400564 Acquire a lock, blocking or non-blocking.
Georg Brandl116aa622007-08-15 14:28:22 +0000565
R David Murrayef4d2862012-10-06 14:35:35 -0400566 When invoked without arguments: if this thread already owns the lock, increment
567 the recursion level by one, and return immediately. Otherwise, if another
568 thread owns the lock, block until the lock is unlocked. Once the lock is
569 unlocked (not owned by any thread), then grab ownership, set the recursion level
570 to one, and return. If more than one thread is blocked waiting until the lock
571 is unlocked, only one at a time will be able to grab ownership of the lock.
572 There is no return value in this case.
Georg Brandl116aa622007-08-15 14:28:22 +0000573
R David Murrayef4d2862012-10-06 14:35:35 -0400574 When invoked with the *blocking* argument set to true, do the same thing as when
Serhiy Storchaka138ccbb2019-11-12 16:57:03 +0200575 called without arguments, and return ``True``.
R David Murrayef4d2862012-10-06 14:35:35 -0400576
577 When invoked with the *blocking* argument set to false, do not block. If a call
Serhiy Storchaka138ccbb2019-11-12 16:57:03 +0200578 without an argument would block, return ``False`` immediately; otherwise, do the
579 same thing as when called without arguments, and return ``True``.
R David Murrayef4d2862012-10-06 14:35:35 -0400580
581 When invoked with the floating-point *timeout* argument set to a positive
582 value, block for at most the number of seconds specified by *timeout*
Serhiy Storchaka138ccbb2019-11-12 16:57:03 +0200583 and as long as the lock cannot be acquired. Return ``True`` if the lock has
R David Murrayef4d2862012-10-06 14:35:35 -0400584 been acquired, false if the timeout has elapsed.
585
586 .. versionchanged:: 3.2
587 The *timeout* parameter is new.
588
589
590 .. method:: release()
591
592 Release a lock, decrementing the recursion level. If after the decrement it is
593 zero, reset the lock to unlocked (not owned by any thread), and if any other
594 threads are blocked waiting for the lock to become unlocked, allow exactly one
595 of them to proceed. If after the decrement the recursion level is still
596 nonzero, the lock remains locked and owned by the calling thread.
597
598 Only call this method when the calling thread owns the lock. A
599 :exc:`RuntimeError` is raised if this method is called when the lock is
600 unlocked.
601
602 There is no return value.
Georg Brandl116aa622007-08-15 14:28:22 +0000603
604
605.. _condition-objects:
606
607Condition Objects
608-----------------
609
610A condition variable is always associated with some kind of lock; this can be
Antoine Pitrou126aef72012-04-10 22:24:05 +0200611passed in or one will be created by default. Passing one in is useful when
612several condition variables must share the same lock. The lock is part of
613the condition object: you don't have to track it separately.
Georg Brandl116aa622007-08-15 14:28:22 +0000614
Serhiy Storchaka14867992014-09-10 23:43:41 +0300615A condition variable obeys the :ref:`context management protocol <with-locks>`:
Antoine Pitroub96a3542012-04-10 22:47:55 +0200616using the ``with`` statement acquires the associated lock for the duration of
617the enclosed block. The :meth:`~Condition.acquire` and
618:meth:`~Condition.release` methods also call the corresponding methods of
619the associated lock.
Georg Brandl116aa622007-08-15 14:28:22 +0000620
Antoine Pitrou126aef72012-04-10 22:24:05 +0200621Other methods must be called with the associated lock held. The
622:meth:`~Condition.wait` method releases the lock, and then blocks until
623another thread awakens it by calling :meth:`~Condition.notify` or
624:meth:`~Condition.notify_all`. Once awakened, :meth:`~Condition.wait`
625re-acquires the lock and returns. It is also possible to specify a timeout.
Georg Brandl116aa622007-08-15 14:28:22 +0000626
Antoine Pitrou126aef72012-04-10 22:24:05 +0200627The :meth:`~Condition.notify` method wakes up one of the threads waiting for
628the condition variable, if any are waiting. The :meth:`~Condition.notify_all`
629method wakes up all threads waiting for the condition variable.
Georg Brandl116aa622007-08-15 14:28:22 +0000630
Antoine Pitrou126aef72012-04-10 22:24:05 +0200631Note: the :meth:`~Condition.notify` and :meth:`~Condition.notify_all` methods
632don't release the lock; this means that the thread or threads awakened will
633not return from their :meth:`~Condition.wait` call immediately, but only when
634the thread that called :meth:`~Condition.notify` or :meth:`~Condition.notify_all`
635finally relinquishes ownership of the lock.
Georg Brandl116aa622007-08-15 14:28:22 +0000636
Antoine Pitrou126aef72012-04-10 22:24:05 +0200637The typical programming style using condition variables uses the lock to
Georg Brandl116aa622007-08-15 14:28:22 +0000638synchronize access to some shared state; threads that are interested in a
Antoine Pitrou126aef72012-04-10 22:24:05 +0200639particular change of state call :meth:`~Condition.wait` repeatedly until they
640see the desired state, while threads that modify the state call
641:meth:`~Condition.notify` or :meth:`~Condition.notify_all` when they change
642the state in such a way that it could possibly be a desired state for one
643of the waiters. For example, the following code is a generic
644producer-consumer situation with unlimited buffer capacity::
Georg Brandl116aa622007-08-15 14:28:22 +0000645
646 # Consume one item
Antoine Pitrou126aef72012-04-10 22:24:05 +0200647 with cv:
648 while not an_item_is_available():
649 cv.wait()
650 get_an_available_item()
Georg Brandl116aa622007-08-15 14:28:22 +0000651
652 # Produce one item
Antoine Pitrou126aef72012-04-10 22:24:05 +0200653 with cv:
654 make_an_item_available()
Antoine Pitrouf6cd9b22012-04-11 19:37:56 +0200655 cv.notify()
Georg Brandl116aa622007-08-15 14:28:22 +0000656
Antoine Pitrou126aef72012-04-10 22:24:05 +0200657The ``while`` loop checking for the application's condition is necessary
658because :meth:`~Condition.wait` can return after an arbitrary long time,
Antoine Pitrouf6cd9b22012-04-11 19:37:56 +0200659and the condition which prompted the :meth:`~Condition.notify` call may
660no longer hold true. This is inherent to multi-threaded programming. The
661:meth:`~Condition.wait_for` method can be used to automate the condition
662checking, and eases the computation of timeouts::
Georg Brandl116aa622007-08-15 14:28:22 +0000663
Antoine Pitrou126aef72012-04-10 22:24:05 +0200664 # Consume an item
665 with cv:
666 cv.wait_for(an_item_is_available)
667 get_an_available_item()
Kristján Valur Jónsson63315202010-11-18 12:46:39 +0000668
Antoine Pitrou126aef72012-04-10 22:24:05 +0200669To choose between :meth:`~Condition.notify` and :meth:`~Condition.notify_all`,
670consider whether one state change can be interesting for only one or several
671waiting threads. E.g. in a typical producer-consumer situation, adding one
672item to the buffer only needs to wake up one consumer thread.
673
674
Georg Brandl7f01a132009-09-16 15:58:14 +0000675.. class:: Condition(lock=None)
Georg Brandl116aa622007-08-15 14:28:22 +0000676
R David Murrayef4d2862012-10-06 14:35:35 -0400677 This class implements condition variable objects. A condition variable
678 allows one or more threads to wait until they are notified by another thread.
679
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000680 If the *lock* argument is given and not ``None``, it must be a :class:`Lock`
681 or :class:`RLock` object, and it is used as the underlying lock. Otherwise,
682 a new :class:`RLock` object is created and used as the underlying lock.
Georg Brandl116aa622007-08-15 14:28:22 +0000683
R David Murrayef4d2862012-10-06 14:35:35 -0400684 .. versionchanged:: 3.3
685 changed from a factory function to a class.
686
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000687 .. method:: acquire(*args)
Georg Brandl116aa622007-08-15 14:28:22 +0000688
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000689 Acquire the underlying lock. This method calls the corresponding method on
690 the underlying lock; the return value is whatever that method returns.
Georg Brandl116aa622007-08-15 14:28:22 +0000691
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000692 .. method:: release()
Georg Brandl116aa622007-08-15 14:28:22 +0000693
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000694 Release the underlying lock. This method calls the corresponding method on
695 the underlying lock; there is no return value.
Georg Brandl116aa622007-08-15 14:28:22 +0000696
Georg Brandl7f01a132009-09-16 15:58:14 +0000697 .. method:: wait(timeout=None)
Georg Brandl116aa622007-08-15 14:28:22 +0000698
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000699 Wait until notified or until a timeout occurs. If the calling thread has
700 not acquired the lock when this method is called, a :exc:`RuntimeError` is
701 raised.
Georg Brandl116aa622007-08-15 14:28:22 +0000702
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000703 This method releases the underlying lock, and then blocks until it is
704 awakened by a :meth:`notify` or :meth:`notify_all` call for the same
705 condition variable in another thread, or until the optional timeout
706 occurs. Once awakened or timed out, it re-acquires the lock and returns.
Georg Brandl116aa622007-08-15 14:28:22 +0000707
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000708 When the *timeout* argument is present and not ``None``, it should be a
709 floating point number specifying a timeout for the operation in seconds
710 (or fractions thereof).
Georg Brandl116aa622007-08-15 14:28:22 +0000711
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000712 When the underlying lock is an :class:`RLock`, it is not released using
713 its :meth:`release` method, since this may not actually unlock the lock
714 when it was acquired multiple times recursively. Instead, an internal
715 interface of the :class:`RLock` class is used, which really unlocks it
716 even when it has been recursively acquired several times. Another internal
717 interface is then used to restore the recursion level when the lock is
718 reacquired.
Georg Brandl116aa622007-08-15 14:28:22 +0000719
Georg Brandlb9a43912010-10-28 09:03:20 +0000720 The return value is ``True`` unless a given *timeout* expired, in which
721 case it is ``False``.
722
723 .. versionchanged:: 3.2
724 Previously, the method always returned ``None``.
725
Kristján Valur Jónsson63315202010-11-18 12:46:39 +0000726 .. method:: wait_for(predicate, timeout=None)
727
Serhiy Storchaka4adf01c2016-10-19 18:30:05 +0300728 Wait until a condition evaluates to true. *predicate* should be a
Kristján Valur Jónsson63315202010-11-18 12:46:39 +0000729 callable which result will be interpreted as a boolean value.
730 A *timeout* may be provided giving the maximum time to wait.
731
732 This utility method may call :meth:`wait` repeatedly until the predicate
733 is satisfied, or until a timeout occurs. The return value is
734 the last return value of the predicate and will evaluate to
735 ``False`` if the method timed out.
736
737 Ignoring the timeout feature, calling this method is roughly equivalent to
738 writing::
739
740 while not predicate():
741 cv.wait()
742
743 Therefore, the same rules apply as with :meth:`wait`: The lock must be
Senthil Kumaranb4760ef2015-06-14 17:35:37 -0700744 held when called and is re-acquired on return. The predicate is evaluated
Kristján Valur Jónsson63315202010-11-18 12:46:39 +0000745 with the lock held.
746
Kristján Valur Jónsson63315202010-11-18 12:46:39 +0000747 .. versionadded:: 3.2
748
Eli Benderskyd44af822011-11-12 20:44:25 +0200749 .. method:: notify(n=1)
Georg Brandl116aa622007-08-15 14:28:22 +0000750
Eli Benderskyd44af822011-11-12 20:44:25 +0200751 By default, wake up one thread waiting on this condition, if any. If the
752 calling thread has not acquired the lock when this method is called, a
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000753 :exc:`RuntimeError` is raised.
Georg Brandl116aa622007-08-15 14:28:22 +0000754
Eli Benderskyd44af822011-11-12 20:44:25 +0200755 This method wakes up at most *n* of the threads waiting for the condition
756 variable; it is a no-op if no threads are waiting.
Georg Brandl116aa622007-08-15 14:28:22 +0000757
Eli Benderskyd44af822011-11-12 20:44:25 +0200758 The current implementation wakes up exactly *n* threads, if at least *n*
759 threads are waiting. However, it's not safe to rely on this behavior.
760 A future, optimized implementation may occasionally wake up more than
761 *n* threads.
Georg Brandl116aa622007-08-15 14:28:22 +0000762
Eli Benderskyd44af822011-11-12 20:44:25 +0200763 Note: an awakened thread does not actually return from its :meth:`wait`
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000764 call until it can reacquire the lock. Since :meth:`notify` does not
765 release the lock, its caller should.
Georg Brandl116aa622007-08-15 14:28:22 +0000766
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000767 .. method:: notify_all()
Georg Brandl116aa622007-08-15 14:28:22 +0000768
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000769 Wake up all threads waiting on this condition. This method acts like
770 :meth:`notify`, but wakes up all waiting threads instead of one. If the
771 calling thread has not acquired the lock when this method is called, a
772 :exc:`RuntimeError` is raised.
Georg Brandl116aa622007-08-15 14:28:22 +0000773
774
775.. _semaphore-objects:
776
777Semaphore Objects
778-----------------
779
780This is one of the oldest synchronization primitives in the history of computer
781science, invented by the early Dutch computer scientist Edsger W. Dijkstra (he
Antoine Pitrou2c9f1042012-04-10 22:35:53 +0200782used the names ``P()`` and ``V()`` instead of :meth:`~Semaphore.acquire` and
783:meth:`~Semaphore.release`).
Georg Brandl116aa622007-08-15 14:28:22 +0000784
785A semaphore manages an internal counter which is decremented by each
Antoine Pitrou2c9f1042012-04-10 22:35:53 +0200786:meth:`~Semaphore.acquire` call and incremented by each :meth:`~Semaphore.release`
787call. The counter can never go below zero; when :meth:`~Semaphore.acquire`
788finds that it is zero, it blocks, waiting until some other thread calls
789:meth:`~Semaphore.release`.
Georg Brandl116aa622007-08-15 14:28:22 +0000790
Serhiy Storchaka14867992014-09-10 23:43:41 +0300791Semaphores also support the :ref:`context management protocol <with-locks>`.
Georg Brandl116aa622007-08-15 14:28:22 +0000792
793
Georg Brandl7f01a132009-09-16 15:58:14 +0000794.. class:: Semaphore(value=1)
Georg Brandl116aa622007-08-15 14:28:22 +0000795
Garrett Berga0374dd2017-12-07 11:04:26 -0700796 This class implements semaphore objects. A semaphore manages an atomic
797 counter representing the number of :meth:`release` calls minus the number of
R David Murrayef4d2862012-10-06 14:35:35 -0400798 :meth:`acquire` calls, plus an initial value. The :meth:`acquire` method
799 blocks if necessary until it can return without making the counter negative.
800 If not given, *value* defaults to 1.
801
Georg Brandl116aa622007-08-15 14:28:22 +0000802 The optional argument gives the initial *value* for the internal counter; it
803 defaults to ``1``. If the *value* given is less than 0, :exc:`ValueError` is
804 raised.
805
R David Murrayef4d2862012-10-06 14:35:35 -0400806 .. versionchanged:: 3.3
807 changed from a factory function to a class.
808
Antoine Pitrou0454af92010-04-17 23:51:58 +0000809 .. method:: acquire(blocking=True, timeout=None)
Georg Brandl116aa622007-08-15 14:28:22 +0000810
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000811 Acquire a semaphore.
Georg Brandl116aa622007-08-15 14:28:22 +0000812
Garrett Berga0374dd2017-12-07 11:04:26 -0700813 When invoked without arguments:
814
815 * If the internal counter is larger than zero on entry, decrement it by
Serhiy Storchaka138ccbb2019-11-12 16:57:03 +0200816 one and return ``True`` immediately.
Garrett Berga0374dd2017-12-07 11:04:26 -0700817 * If the internal counter is zero on entry, block until awoken by a call to
818 :meth:`~Semaphore.release`. Once awoken (and the counter is greater
Serhiy Storchaka138ccbb2019-11-12 16:57:03 +0200819 than 0), decrement the counter by 1 and return ``True``. Exactly one
Garrett Berga0374dd2017-12-07 11:04:26 -0700820 thread will be awoken by each call to :meth:`~Semaphore.release`. The
821 order in which threads are awoken should not be relied on.
Georg Brandl116aa622007-08-15 14:28:22 +0000822
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000823 When invoked with *blocking* set to false, do not block. If a call
Serhiy Storchaka138ccbb2019-11-12 16:57:03 +0200824 without an argument would block, return ``False`` immediately; otherwise, do
825 the same thing as when called without arguments, and return ``True``.
Antoine Pitrou0454af92010-04-17 23:51:58 +0000826
Serhiy Storchakaecf41da2016-10-19 16:29:26 +0300827 When invoked with a *timeout* other than ``None``, it will block for at
Antoine Pitrou0454af92010-04-17 23:51:58 +0000828 most *timeout* seconds. If acquire does not complete successfully in
Serhiy Storchaka138ccbb2019-11-12 16:57:03 +0200829 that interval, return ``False``. Return ``True`` otherwise.
Antoine Pitrou0454af92010-04-17 23:51:58 +0000830
831 .. versionchanged:: 3.2
832 The *timeout* parameter is new.
Georg Brandl116aa622007-08-15 14:28:22 +0000833
Raymond Hettinger35f63012019-08-29 01:45:19 -0700834 .. method:: release(n=1)
Georg Brandl116aa622007-08-15 14:28:22 +0000835
Raymond Hettinger35f63012019-08-29 01:45:19 -0700836 Release a semaphore, incrementing the internal counter by *n*. When it
837 was zero on entry and other threads are waiting for it to become larger
838 than zero again, wake up *n* of those threads.
839
840 .. versionchanged:: 3.9
841 Added the *n* parameter to release multiple waiting threads at once.
Georg Brandl116aa622007-08-15 14:28:22 +0000842
843
R David Murrayef4d2862012-10-06 14:35:35 -0400844.. class:: BoundedSemaphore(value=1)
845
846 Class implementing bounded semaphore objects. A bounded semaphore checks to
847 make sure its current value doesn't exceed its initial value. If it does,
848 :exc:`ValueError` is raised. In most situations semaphores are used to guard
849 resources with limited capacity. If the semaphore is released too many times
850 it's a sign of a bug. If not given, *value* defaults to 1.
851
852 .. versionchanged:: 3.3
853 changed from a factory function to a class.
854
855
Georg Brandl116aa622007-08-15 14:28:22 +0000856.. _semaphore-examples:
857
858:class:`Semaphore` Example
859^^^^^^^^^^^^^^^^^^^^^^^^^^
860
861Semaphores are often used to guard resources with limited capacity, for example,
Georg Brandla5724762011-01-06 19:28:18 +0000862a database server. In any situation where the size of the resource is fixed,
863you should use a bounded semaphore. Before spawning any worker threads, your
864main thread would initialize the semaphore::
Georg Brandl116aa622007-08-15 14:28:22 +0000865
866 maxconnections = 5
R David Murrayef4d2862012-10-06 14:35:35 -0400867 # ...
Georg Brandl116aa622007-08-15 14:28:22 +0000868 pool_sema = BoundedSemaphore(value=maxconnections)
869
870Once spawned, worker threads call the semaphore's acquire and release methods
871when they need to connect to the server::
872
Antoine Pitroub96a3542012-04-10 22:47:55 +0200873 with pool_sema:
874 conn = connectdb()
875 try:
R David Murrayef4d2862012-10-06 14:35:35 -0400876 # ... use connection ...
Antoine Pitroub96a3542012-04-10 22:47:55 +0200877 finally:
878 conn.close()
Georg Brandl116aa622007-08-15 14:28:22 +0000879
880The use of a bounded semaphore reduces the chance that a programming error which
881causes the semaphore to be released more than it's acquired will go undetected.
882
883
884.. _event-objects:
885
886Event Objects
887-------------
888
889This is one of the simplest mechanisms for communication between threads: one
890thread signals an event and other threads wait for it.
891
892An event object manages an internal flag that can be set to true with the
Antoine Pitrou2c9f1042012-04-10 22:35:53 +0200893:meth:`~Event.set` method and reset to false with the :meth:`~Event.clear`
894method. The :meth:`~Event.wait` method blocks until the flag is true.
Georg Brandl116aa622007-08-15 14:28:22 +0000895
896
897.. class:: Event()
898
R David Murrayef4d2862012-10-06 14:35:35 -0400899 Class implementing event objects. An event manages a flag that can be set to
900 true with the :meth:`~Event.set` method and reset to false with the
901 :meth:`clear` method. The :meth:`wait` method blocks until the flag is true.
902 The flag is initially false.
903
904 .. versionchanged:: 3.3
905 changed from a factory function to a class.
Georg Brandl116aa622007-08-15 14:28:22 +0000906
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000907 .. method:: is_set()
Georg Brandl116aa622007-08-15 14:28:22 +0000908
Serhiy Storchaka138ccbb2019-11-12 16:57:03 +0200909 Return ``True`` if and only if the internal flag is true.
Georg Brandl116aa622007-08-15 14:28:22 +0000910
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000911 .. method:: set()
Georg Brandl116aa622007-08-15 14:28:22 +0000912
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000913 Set the internal flag to true. All threads waiting for it to become true
914 are awakened. Threads that call :meth:`wait` once the flag is true will
915 not block at all.
Georg Brandl116aa622007-08-15 14:28:22 +0000916
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000917 .. method:: clear()
Georg Brandl116aa622007-08-15 14:28:22 +0000918
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000919 Reset the internal flag to false. Subsequently, threads calling
Georg Brandl502d9a52009-07-26 15:02:41 +0000920 :meth:`wait` will block until :meth:`.set` is called to set the internal
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000921 flag to true again.
Georg Brandl116aa622007-08-15 14:28:22 +0000922
Georg Brandl7f01a132009-09-16 15:58:14 +0000923 .. method:: wait(timeout=None)
Georg Brandl116aa622007-08-15 14:28:22 +0000924
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000925 Block until the internal flag is true. If the internal flag is true on
926 entry, return immediately. Otherwise, block until another thread calls
Antoine Pitrou2c9f1042012-04-10 22:35:53 +0200927 :meth:`.set` to set the flag to true, or until the optional timeout occurs.
Georg Brandl116aa622007-08-15 14:28:22 +0000928
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000929 When the timeout argument is present and not ``None``, it should be a
930 floating point number specifying a timeout for the operation in seconds
931 (or fractions thereof).
Georg Brandl116aa622007-08-15 14:28:22 +0000932
Serhiy Storchaka138ccbb2019-11-12 16:57:03 +0200933 This method returns ``True`` if and only if the internal flag has been set to
Charles-François Natalided03482012-01-07 18:24:56 +0100934 true, either before the wait call or after the wait starts, so it will
935 always return ``True`` except if a timeout is given and the operation
936 times out.
Georg Brandl116aa622007-08-15 14:28:22 +0000937
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000938 .. versionchanged:: 3.1
939 Previously, the method always returned ``None``.
Benjamin Petersond23f8222009-04-05 19:13:16 +0000940
Georg Brandl116aa622007-08-15 14:28:22 +0000941
Georg Brandl116aa622007-08-15 14:28:22 +0000942.. _timer-objects:
943
944Timer Objects
945-------------
946
947This class represents an action that should be run only after a certain amount
948of time has passed --- a timer. :class:`Timer` is a subclass of :class:`Thread`
949and as such also functions as an example of creating custom threads.
950
Serhiy Storchaka9e0ae532013-08-24 00:23:38 +0300951Timers are started, as with threads, by calling their :meth:`~Timer.start`
952method. The timer can be stopped (before its action has begun) by calling the
953:meth:`~Timer.cancel` method. The interval the timer will wait before
954executing its action may not be exactly the same as the interval specified by
955the user.
Georg Brandl116aa622007-08-15 14:28:22 +0000956
957For example::
958
959 def hello():
Collin Winterc79461b2007-09-01 23:34:30 +0000960 print("hello, world")
Georg Brandl116aa622007-08-15 14:28:22 +0000961
962 t = Timer(30.0, hello)
Serhiy Storchakadba90392016-05-10 12:01:23 +0300963 t.start() # after 30 seconds, "hello, world" will be printed
Georg Brandl116aa622007-08-15 14:28:22 +0000964
965
R David Murray19aeb432013-03-30 17:19:38 -0400966.. class:: Timer(interval, function, args=None, kwargs=None)
Georg Brandl116aa622007-08-15 14:28:22 +0000967
968 Create a timer that will run *function* with arguments *args* and keyword
969 arguments *kwargs*, after *interval* seconds have passed.
Serhiy Storchakaecf41da2016-10-19 16:29:26 +0300970 If *args* is ``None`` (the default) then an empty list will be used.
971 If *kwargs* is ``None`` (the default) then an empty dict will be used.
Georg Brandl116aa622007-08-15 14:28:22 +0000972
R David Murrayef4d2862012-10-06 14:35:35 -0400973 .. versionchanged:: 3.3
974 changed from a factory function to a class.
975
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000976 .. method:: cancel()
Georg Brandl116aa622007-08-15 14:28:22 +0000977
Georg Brandl7a72b3a2009-07-26 14:48:09 +0000978 Stop the timer, and cancel the execution of the timer's action. This will
979 only work if the timer is still in its waiting stage.
Georg Brandl116aa622007-08-15 14:28:22 +0000980
981
Kristján Valur Jónsson3be00032010-10-28 09:43:10 +0000982Barrier Objects
983---------------
984
Georg Brandl5bc16862010-10-28 13:07:50 +0000985.. versionadded:: 3.2
986
987This class provides a simple synchronization primitive for use by a fixed number
988of threads that need to wait for each other. Each of the threads tries to pass
Antoine Pitrou2c9f1042012-04-10 22:35:53 +0200989the barrier by calling the :meth:`~Barrier.wait` method and will block until
Saurabh Chaturvedi143be362017-08-15 00:24:53 +0530990all of the threads have made their :meth:`~Barrier.wait` calls. At this point,
991the threads are released simultaneously.
Kristján Valur Jónsson3be00032010-10-28 09:43:10 +0000992
993The barrier can be reused any number of times for the same number of threads.
994
995As an example, here is a simple way to synchronize a client and server thread::
996
997 b = Barrier(2, timeout=5)
Georg Brandl5bc16862010-10-28 13:07:50 +0000998
999 def server():
Kristján Valur Jónsson3be00032010-10-28 09:43:10 +00001000 start_server()
1001 b.wait()
1002 while True:
1003 connection = accept_connection()
1004 process_server_connection(connection)
1005
Georg Brandl5bc16862010-10-28 13:07:50 +00001006 def client():
Kristján Valur Jónsson3be00032010-10-28 09:43:10 +00001007 b.wait()
1008 while True:
Georg Brandl5bc16862010-10-28 13:07:50 +00001009 connection = make_connection()
1010 process_client_connection(connection)
1011
Kristján Valur Jónsson3be00032010-10-28 09:43:10 +00001012
1013.. class:: Barrier(parties, action=None, timeout=None)
1014
Georg Brandl5bc16862010-10-28 13:07:50 +00001015 Create a barrier object for *parties* number of threads. An *action*, when
1016 provided, is a callable to be called by one of the threads when they are
1017 released. *timeout* is the default timeout value if none is specified for
1018 the :meth:`wait` method.
Kristján Valur Jónsson3be00032010-10-28 09:43:10 +00001019
1020 .. method:: wait(timeout=None)
1021
1022 Pass the barrier. When all the threads party to the barrier have called
Georg Brandl5bc16862010-10-28 13:07:50 +00001023 this function, they are all released simultaneously. If a *timeout* is
Ezio Melottie130a522011-10-19 10:58:56 +03001024 provided, it is used in preference to any that was supplied to the class
Georg Brandl5bc16862010-10-28 13:07:50 +00001025 constructor.
Kristján Valur Jónsson3be00032010-10-28 09:43:10 +00001026
Georg Brandl5bc16862010-10-28 13:07:50 +00001027 The return value is an integer in the range 0 to *parties* -- 1, different
Raymond Hettinger5cee47f2011-01-11 19:59:46 +00001028 for each thread. This can be used to select a thread to do some special
Georg Brandl5bc16862010-10-28 13:07:50 +00001029 housekeeping, e.g.::
Kristján Valur Jónsson3be00032010-10-28 09:43:10 +00001030
1031 i = barrier.wait()
1032 if i == 0:
Georg Brandl5bc16862010-10-28 13:07:50 +00001033 # Only one thread needs to print this
1034 print("passed the barrier")
Kristján Valur Jónsson3be00032010-10-28 09:43:10 +00001035
Georg Brandl5bc16862010-10-28 13:07:50 +00001036 If an *action* was provided to the constructor, one of the threads will
1037 have called it prior to being released. Should this call raise an error,
1038 the barrier is put into the broken state.
Kristján Valur Jónsson3be00032010-10-28 09:43:10 +00001039
1040 If the call times out, the barrier is put into the broken state.
1041
1042 This method may raise a :class:`BrokenBarrierError` exception if the
Georg Brandl5bc16862010-10-28 13:07:50 +00001043 barrier is broken or reset while a thread is waiting.
Kristján Valur Jónsson3be00032010-10-28 09:43:10 +00001044
1045 .. method:: reset()
1046
Georg Brandl5bc16862010-10-28 13:07:50 +00001047 Return the barrier to the default, empty state. Any threads waiting on it
1048 will receive the :class:`BrokenBarrierError` exception.
Kristján Valur Jónsson3be00032010-10-28 09:43:10 +00001049
Géry Ogam51a860e2019-05-18 00:44:57 +02001050 Note that using this function may require some external
Georg Brandl5bc16862010-10-28 13:07:50 +00001051 synchronization if there are other threads whose state is unknown. If a
1052 barrier is broken it may be better to just leave it and create a new one.
Kristján Valur Jónsson3be00032010-10-28 09:43:10 +00001053
1054 .. method:: abort()
1055
1056 Put the barrier into a broken state. This causes any active or future
Georg Brandl5bc16862010-10-28 13:07:50 +00001057 calls to :meth:`wait` to fail with the :class:`BrokenBarrierError`. Use
Géry Ogam51a860e2019-05-18 00:44:57 +02001058 this for example if one of the threads needs to abort, to avoid deadlocking the
Georg Brandl5bc16862010-10-28 13:07:50 +00001059 application.
Kristján Valur Jónsson3be00032010-10-28 09:43:10 +00001060
1061 It may be preferable to simply create the barrier with a sensible
Georg Brandl5bc16862010-10-28 13:07:50 +00001062 *timeout* value to automatically guard against one of the threads going
1063 awry.
Kristján Valur Jónsson3be00032010-10-28 09:43:10 +00001064
1065 .. attribute:: parties
1066
1067 The number of threads required to pass the barrier.
1068
1069 .. attribute:: n_waiting
1070
1071 The number of threads currently waiting in the barrier.
1072
1073 .. attribute:: broken
1074
1075 A boolean that is ``True`` if the barrier is in the broken state.
1076
Kristján Valur Jónsson3be00032010-10-28 09:43:10 +00001077
Georg Brandl5bc16862010-10-28 13:07:50 +00001078.. exception:: BrokenBarrierError
Kristján Valur Jónsson3be00032010-10-28 09:43:10 +00001079
Georg Brandl5bc16862010-10-28 13:07:50 +00001080 This exception, a subclass of :exc:`RuntimeError`, is raised when the
1081 :class:`Barrier` object is reset or broken.
Kristján Valur Jónsson3be00032010-10-28 09:43:10 +00001082
1083
Georg Brandl116aa622007-08-15 14:28:22 +00001084.. _with-locks:
1085
Serhiy Storchaka2b57c432018-12-19 08:09:46 +02001086Using locks, conditions, and semaphores in the :keyword:`!with` statement
1087-------------------------------------------------------------------------
Georg Brandl116aa622007-08-15 14:28:22 +00001088
1089All of the objects provided by this module that have :meth:`acquire` and
1090:meth:`release` methods can be used as context managers for a :keyword:`with`
Antoine Pitroub96a3542012-04-10 22:47:55 +02001091statement. The :meth:`acquire` method will be called when the block is
1092entered, and :meth:`release` will be called when the block is exited. Hence,
1093the following snippet::
1094
1095 with some_lock:
1096 # do something...
1097
1098is equivalent to::
1099
1100 some_lock.acquire()
1101 try:
1102 # do something...
1103 finally:
1104 some_lock.release()
Georg Brandl116aa622007-08-15 14:28:22 +00001105
1106Currently, :class:`Lock`, :class:`RLock`, :class:`Condition`,
1107:class:`Semaphore`, and :class:`BoundedSemaphore` objects may be used as
Antoine Pitroub96a3542012-04-10 22:47:55 +02001108:keyword:`with` statement context managers.