blob: 3a812eb21471a0d8267ce1611f7cf995ddefa98a [file] [log] [blame]
Georg Brandl116aa622007-08-15 14:28:22 +00001
2.. _datamodel:
3
4**********
5Data model
6**********
7
8
9.. _objects:
10
11Objects, values and types
12=========================
13
14.. index::
15 single: object
16 single: data
17
18:dfn:`Objects` are Python's abstraction for data. All data in a Python program
19is represented by objects or by relations between objects. (In a sense, and in
Mathieu Dupuyc49016e2020-03-30 23:28:25 +020020conformance to Von Neumann's model of a "stored program computer", code is also
Georg Brandl116aa622007-08-15 14:28:22 +000021represented by objects.)
22
23.. index::
24 builtin: id
25 builtin: type
26 single: identity of an object
27 single: value of an object
28 single: type of an object
29 single: mutable object
30 single: immutable object
31
Georg Brandl85eb8c12007-08-31 16:33:38 +000032.. XXX it *is* now possible in some cases to change an object's
33 type, under certain controlled conditions
34
Georg Brandl116aa622007-08-15 14:28:22 +000035Every object has an identity, a type and a value. An object's *identity* never
36changes once it has been created; you may think of it as the object's address in
37memory. The ':keyword:`is`' operator compares the identity of two objects; the
Nick Coghlan337b2bf2012-05-20 18:30:49 +100038:func:`id` function returns an integer representing its identity.
39
40.. impl-detail::
41
42 For CPython, ``id(x)`` is the memory address where ``x`` is stored.
43
Georg Brandl116aa622007-08-15 14:28:22 +000044An object's type determines the operations that the object supports (e.g., "does
45it have a length?") and also defines the possible values for objects of that
46type. The :func:`type` function returns an object's type (which is an object
Nick Coghlan337b2bf2012-05-20 18:30:49 +100047itself). Like its identity, an object's :dfn:`type` is also unchangeable.
48[#]_
49
50The *value* of some objects can change. Objects whose value can
Georg Brandl116aa622007-08-15 14:28:22 +000051change are said to be *mutable*; objects whose value is unchangeable once they
52are created are called *immutable*. (The value of an immutable container object
53that contains a reference to a mutable object can change when the latter's value
54is changed; however the container is still considered immutable, because the
55collection of objects it contains cannot be changed. So, immutability is not
56strictly the same as having an unchangeable value, it is more subtle.) An
57object's mutability is determined by its type; for instance, numbers, strings
58and tuples are immutable, while dictionaries and lists are mutable.
59
60.. index::
61 single: garbage collection
62 single: reference counting
63 single: unreachable object
64
65Objects are never explicitly destroyed; however, when they become unreachable
66they may be garbage-collected. An implementation is allowed to postpone garbage
67collection or omit it altogether --- it is a matter of implementation quality
68how garbage collection is implemented, as long as no objects are collected that
Georg Brandl495f7b52009-10-27 15:28:25 +000069are still reachable.
70
71.. impl-detail::
72
73 CPython currently uses a reference-counting scheme with (optional) delayed
74 detection of cyclically linked garbage, which collects most objects as soon
75 as they become unreachable, but is not guaranteed to collect garbage
76 containing circular references. See the documentation of the :mod:`gc`
77 module for information on controlling the collection of cyclic garbage.
78 Other implementations act differently and CPython may change.
Gregory P. Smithc5425472011-03-10 11:28:50 -080079 Do not depend on immediate finalization of objects when they become
Raymond Hettingeraa7886d2014-05-26 22:20:37 -070080 unreachable (so you should always close files explicitly).
Georg Brandl116aa622007-08-15 14:28:22 +000081
82Note that the use of the implementation's tracing or debugging facilities may
83keep objects alive that would normally be collectable. Also note that catching
84an exception with a ':keyword:`try`...\ :keyword:`except`' statement may keep
85objects alive.
86
87Some objects contain references to "external" resources such as open files or
88windows. It is understood that these resources are freed when the object is
89garbage-collected, but since garbage collection is not guaranteed to happen,
90such objects also provide an explicit way to release the external resource,
91usually a :meth:`close` method. Programs are strongly recommended to explicitly
92close such objects. The ':keyword:`try`...\ :keyword:`finally`' statement
Nick Coghlan3a5d7e32008-08-31 12:40:14 +000093and the ':keyword:`with`' statement provide convenient ways to do this.
Georg Brandl116aa622007-08-15 14:28:22 +000094
95.. index:: single: container
96
97Some objects contain references to other objects; these are called *containers*.
98Examples of containers are tuples, lists and dictionaries. The references are
99part of a container's value. In most cases, when we talk about the value of a
100container, we imply the values, not the identities of the contained objects;
101however, when we talk about the mutability of a container, only the identities
102of the immediately contained objects are implied. So, if an immutable container
103(like a tuple) contains a reference to a mutable object, its value changes if
104that mutable object is changed.
105
106Types affect almost all aspects of object behavior. Even the importance of
107object identity is affected in some sense: for immutable types, operations that
108compute new values may actually return a reference to any existing object with
109the same type and value, while for mutable objects this is not allowed. E.g.,
110after ``a = 1; b = 1``, ``a`` and ``b`` may or may not refer to the same object
111with the value one, depending on the implementation, but after ``c = []; d =
112[]``, ``c`` and ``d`` are guaranteed to refer to two different, unique, newly
113created empty lists. (Note that ``c = d = []`` assigns the same object to both
114``c`` and ``d``.)
115
116
117.. _types:
118
119The standard type hierarchy
120===========================
121
122.. index::
123 single: type
124 pair: data; type
125 pair: type; hierarchy
126 pair: extension; module
127 pair: C; language
128
129Below is a list of the types that are built into Python. Extension modules
130(written in C, Java, or other languages, depending on the implementation) can
131define additional types. Future versions of Python may add types to the type
Nick Coghlan3a5d7e32008-08-31 12:40:14 +0000132hierarchy (e.g., rational numbers, efficiently stored arrays of integers, etc.),
133although such additions will often be provided via the standard library instead.
Georg Brandl116aa622007-08-15 14:28:22 +0000134
135.. index::
136 single: attribute
137 pair: special; attribute
138 triple: generic; special; attribute
139
140Some of the type descriptions below contain a paragraph listing 'special
141attributes.' These are attributes that provide access to the implementation and
142are not intended for general use. Their definition may change in the future.
143
144None
145 .. index:: object: None
146
147 This type has a single value. There is a single object with this value. This
148 object is accessed through the built-in name ``None``. It is used to signify the
149 absence of a value in many situations, e.g., it is returned from functions that
150 don't explicitly return anything. Its truth value is false.
151
152NotImplemented
153 .. index:: object: NotImplemented
154
155 This type has a single value. There is a single object with this value. This
156 object is accessed through the built-in name ``NotImplemented``. Numeric methods
Ethan Furmanb0049432014-11-26 21:15:35 -0800157 and rich comparison methods should return this value if they do not implement the
Georg Brandl116aa622007-08-15 14:28:22 +0000158 operation for the operands provided. (The interpreter will then try the
MojoVampire469325c2020-03-03 18:50:17 +0000159 reflected operation, or some other fallback, depending on the operator.) It
160 should not be evaluated in a boolean context.
Georg Brandl116aa622007-08-15 14:28:22 +0000161
Ethan Furmanb0049432014-11-26 21:15:35 -0800162 See
163 :ref:`implementing-the-arithmetic-operations`
164 for more details.
165
MojoVampire469325c2020-03-03 18:50:17 +0000166 .. versionchanged:: 3.9
167 Evaluating ``NotImplemented`` in a boolean context is deprecated. While
168 it currently evaluates as true, it will emit a :exc:`DeprecationWarning`.
169 It will raise a :exc:`TypeError` in a future version of Python.
170
Ethan Furmanb0049432014-11-26 21:15:35 -0800171
Georg Brandl116aa622007-08-15 14:28:22 +0000172Ellipsis
Serhiy Storchakaddb961d2018-10-26 09:00:49 +0300173 .. index::
174 object: Ellipsis
175 single: ...; ellipsis literal
Georg Brandl116aa622007-08-15 14:28:22 +0000176
177 This type has a single value. There is a single object with this value. This
178 object is accessed through the literal ``...`` or the built-in name
179 ``Ellipsis``. Its truth value is true.
180
Christian Heimes072c0f12008-01-03 23:01:04 +0000181:class:`numbers.Number`
Georg Brandl116aa622007-08-15 14:28:22 +0000182 .. index:: object: numeric
183
184 These are created by numeric literals and returned as results by arithmetic
185 operators and arithmetic built-in functions. Numeric objects are immutable;
186 once created their value never changes. Python numbers are of course strongly
187 related to mathematical numbers, but subject to the limitations of numerical
188 representation in computers.
189
kpincf8b1ccd2020-10-21 15:34:15 -0500190 The string representations of the numeric classes, computed by
kpincc60394c2020-10-21 12:13:50 -0500191 :meth:`__repr__` and :meth:`__str__`, have the following
192 properties:
193
194 * They are valid numeric literals which, when passed to their
195 class constructor, produce an object having the value of the
196 original numeric.
197
198 * The representation is in base 10, when possible.
199
200 * Leading zeros, possibly excepting a single zero before a
201 decimal point, are not shown.
202
203 * Trailing zeros, possibly excepting a single zero after a
204 decimal point, are not shown.
205
206 * A sign is shown only when the number is negative.
207
Georg Brandl116aa622007-08-15 14:28:22 +0000208 Python distinguishes between integers, floating point numbers, and complex
209 numbers:
210
Christian Heimes072c0f12008-01-03 23:01:04 +0000211 :class:`numbers.Integral`
Georg Brandl116aa622007-08-15 14:28:22 +0000212 .. index:: object: integer
213
214 These represent elements from the mathematical set of integers (positive and
215 negative).
216
Georg Brandl59d69162008-01-07 09:27:36 +0000217 There are two types of integers:
Georg Brandl116aa622007-08-15 14:28:22 +0000218
Nick Coghlan3a5d7e32008-08-31 12:40:14 +0000219 Integers (:class:`int`)
Georg Brandl116aa622007-08-15 14:28:22 +0000220 These represent numbers in an unlimited range, subject to available (virtual)
221 memory only. For the purpose of shift and mask operations, a binary
222 representation is assumed, and negative numbers are represented in a variant of
223 2's complement which gives the illusion of an infinite string of sign bits
224 extending to the left.
225
Nick Coghlan3a5d7e32008-08-31 12:40:14 +0000226 Booleans (:class:`bool`)
Georg Brandl116aa622007-08-15 14:28:22 +0000227 .. index::
228 object: Boolean
229 single: False
230 single: True
231
232 These represent the truth values False and True. The two objects representing
Serhiy Storchakafbc1c262013-11-29 12:17:13 +0200233 the values ``False`` and ``True`` are the only Boolean objects. The Boolean type is a
Georg Brandl95817b32008-05-11 14:30:18 +0000234 subtype of the integer type, and Boolean values behave like the values 0 and 1,
Georg Brandl116aa622007-08-15 14:28:22 +0000235 respectively, in almost all contexts, the exception being that when converted to
236 a string, the strings ``"False"`` or ``"True"`` are returned, respectively.
237
238 .. index:: pair: integer; representation
239
240 The rules for integer representation are intended to give the most meaningful
Georg Brandlbb74a782008-05-11 10:53:16 +0000241 interpretation of shift and mask operations involving negative integers.
Georg Brandl116aa622007-08-15 14:28:22 +0000242
Christian Heimes072c0f12008-01-03 23:01:04 +0000243 :class:`numbers.Real` (:class:`float`)
Georg Brandl116aa622007-08-15 14:28:22 +0000244 .. index::
245 object: floating point
246 pair: floating point; number
247 pair: C; language
248 pair: Java; language
249
250 These represent machine-level double precision floating point numbers. You are
251 at the mercy of the underlying machine architecture (and C or Java
252 implementation) for the accepted range and handling of overflow. Python does not
253 support single-precision floating point numbers; the savings in processor and
Terry Jan Reedyb6271f22014-09-30 19:07:49 -0400254 memory usage that are usually the reason for using these are dwarfed by the
Georg Brandl116aa622007-08-15 14:28:22 +0000255 overhead of using objects in Python, so there is no reason to complicate the
256 language with two kinds of floating point numbers.
257
Nick Coghlan3a5d7e32008-08-31 12:40:14 +0000258 :class:`numbers.Complex` (:class:`complex`)
Georg Brandl116aa622007-08-15 14:28:22 +0000259 .. index::
260 object: complex
261 pair: complex; number
262
263 These represent complex numbers as a pair of machine-level double precision
264 floating point numbers. The same caveats apply as for floating point numbers.
265 The real and imaginary parts of a complex number ``z`` can be retrieved through
266 the read-only attributes ``z.real`` and ``z.imag``.
267
Georg Brandl116aa622007-08-15 14:28:22 +0000268Sequences
269 .. index::
270 builtin: len
271 object: sequence
272 single: index operation
273 single: item selection
274 single: subscription
275
276 These represent finite ordered sets indexed by non-negative numbers. The
277 built-in function :func:`len` returns the number of items of a sequence. When
278 the length of a sequence is *n*, the index set contains the numbers 0, 1,
279 ..., *n*-1. Item *i* of sequence *a* is selected by ``a[i]``.
280
281 .. index:: single: slicing
282
283 Sequences also support slicing: ``a[i:j]`` selects all items with index *k* such
284 that *i* ``<=`` *k* ``<`` *j*. When used as an expression, a slice is a
285 sequence of the same type. This implies that the index set is renumbered so
286 that it starts at 0.
287
Georg Brandl116aa622007-08-15 14:28:22 +0000288 Some sequences also support "extended slicing" with a third "step" parameter:
289 ``a[i:j:k]`` selects all items of *a* with index *x* where ``x = i + n*k``, *n*
290 ``>=`` ``0`` and *i* ``<=`` *x* ``<`` *j*.
291
292 Sequences are distinguished according to their mutability:
293
294 Immutable sequences
295 .. index::
296 object: immutable sequence
297 object: immutable
298
299 An object of an immutable sequence type cannot change once it is created. (If
300 the object contains references to other objects, these other objects may be
301 mutable and may be changed; however, the collection of objects directly
302 referenced by an immutable object cannot change.)
303
304 The following types are immutable sequences:
305
Chris Jerdonekbb4e9412012-11-28 01:38:40 -0800306 .. index::
307 single: string; immutable sequences
308
Georg Brandl116aa622007-08-15 14:28:22 +0000309 Strings
310 .. index::
311 builtin: chr
312 builtin: ord
Georg Brandl116aa622007-08-15 14:28:22 +0000313 single: character
314 single: integer
315 single: Unicode
316
Nick Coghlan14627862014-06-07 23:21:14 +1000317 A string is a sequence of values that represent Unicode code points.
318 All the code points in the range ``U+0000 - U+10FFFF`` can be
319 represented in a string. Python doesn't have a :c:type:`char` type;
320 instead, every code point in the string is represented as a string
321 object with length ``1``. The built-in function :func:`ord`
322 converts a code point from its string form to an integer in the
323 range ``0 - 10FFFF``; :func:`chr` converts an integer in the range
324 ``0 - 10FFFF`` to the corresponding length ``1`` string object.
Ezio Melottif4d76e62011-10-25 09:23:42 +0300325 :meth:`str.encode` can be used to convert a :class:`str` to
Nick Coghlan14627862014-06-07 23:21:14 +1000326 :class:`bytes` using the given text encoding, and
327 :meth:`bytes.decode` can be used to achieve the opposite.
Georg Brandl116aa622007-08-15 14:28:22 +0000328
329 Tuples
330 .. index::
331 object: tuple
332 pair: singleton; tuple
333 pair: empty; tuple
334
Georg Brandldcc56f82007-08-31 16:41:12 +0000335 The items of a tuple are arbitrary Python objects. Tuples of two or
336 more items are formed by comma-separated lists of expressions. A tuple
337 of one item (a 'singleton') can be formed by affixing a comma to an
338 expression (an expression by itself does not create a tuple, since
339 parentheses must be usable for grouping of expressions). An empty
340 tuple can be formed by an empty pair of parentheses.
Georg Brandl116aa622007-08-15 14:28:22 +0000341
Nick Coghlan3a5d7e32008-08-31 12:40:14 +0000342 Bytes
343 .. index:: bytes, byte
344
345 A bytes object is an immutable array. The items are 8-bit bytes,
346 represented by integers in the range 0 <= x < 256. Bytes literals
csabellac6db4812017-04-26 01:47:01 -0400347 (like ``b'abc'``) and the built-in :func:`bytes()` constructor
348 can be used to create bytes objects. Also, bytes objects can be
349 decoded to strings via the :meth:`~bytes.decode` method.
Nick Coghlan3a5d7e32008-08-31 12:40:14 +0000350
Georg Brandl116aa622007-08-15 14:28:22 +0000351 Mutable sequences
352 .. index::
353 object: mutable sequence
354 object: mutable
355 pair: assignment; statement
Georg Brandl116aa622007-08-15 14:28:22 +0000356 single: subscription
357 single: slicing
358
359 Mutable sequences can be changed after they are created. The subscription and
360 slicing notations can be used as the target of assignment and :keyword:`del`
361 (delete) statements.
362
Benjamin Petersonb58dda72009-01-18 22:27:04 +0000363 There are currently two intrinsic mutable sequence types:
Georg Brandl116aa622007-08-15 14:28:22 +0000364
365 Lists
366 .. index:: object: list
367
Georg Brandldcc56f82007-08-31 16:41:12 +0000368 The items of a list are arbitrary Python objects. Lists are formed by
369 placing a comma-separated list of expressions in square brackets. (Note
370 that there are no special cases needed to form lists of length 0 or 1.)
371
Nick Coghlan3a5d7e32008-08-31 12:40:14 +0000372 Byte Arrays
373 .. index:: bytearray
Georg Brandldcc56f82007-08-31 16:41:12 +0000374
Nick Coghlan3a5d7e32008-08-31 12:40:14 +0000375 A bytearray object is a mutable array. They are created by the built-in
csabellac6db4812017-04-26 01:47:01 -0400376 :func:`bytearray` constructor. Aside from being mutable
377 (and hence unhashable), byte arrays otherwise provide the same interface
378 and functionality as immutable :class:`bytes` objects.
Georg Brandl116aa622007-08-15 14:28:22 +0000379
380 .. index:: module: array
381
Georg Brandldcc56f82007-08-31 16:41:12 +0000382 The extension module :mod:`array` provides an additional example of a
Nick Coghlan3a5d7e32008-08-31 12:40:14 +0000383 mutable sequence type, as does the :mod:`collections` module.
Georg Brandl116aa622007-08-15 14:28:22 +0000384
Georg Brandl116aa622007-08-15 14:28:22 +0000385Set types
386 .. index::
387 builtin: len
388 object: set type
389
390 These represent unordered, finite sets of unique, immutable objects. As such,
391 they cannot be indexed by any subscript. However, they can be iterated over, and
392 the built-in function :func:`len` returns the number of items in a set. Common
393 uses for sets are fast membership testing, removing duplicates from a sequence,
394 and computing mathematical operations such as intersection, union, difference,
395 and symmetric difference.
396
397 For set elements, the same immutability rules apply as for dictionary keys. Note
398 that numeric types obey the normal rules for numeric comparison: if two numbers
399 compare equal (e.g., ``1`` and ``1.0``), only one of them can be contained in a
400 set.
401
402 There are currently two intrinsic set types:
403
404 Sets
405 .. index:: object: set
406
407 These represent a mutable set. They are created by the built-in :func:`set`
408 constructor and can be modified afterwards by several methods, such as
Serhiy Storchaka0d196ed2013-10-09 14:02:31 +0300409 :meth:`~set.add`.
Georg Brandl116aa622007-08-15 14:28:22 +0000410
411 Frozen sets
412 .. index:: object: frozenset
413
Guido van Rossum2cc30da2007-11-02 23:46:40 +0000414 These represent an immutable set. They are created by the built-in
415 :func:`frozenset` constructor. As a frozenset is immutable and
416 :term:`hashable`, it can be used again as an element of another set, or as
417 a dictionary key.
Georg Brandl116aa622007-08-15 14:28:22 +0000418
Georg Brandl116aa622007-08-15 14:28:22 +0000419Mappings
420 .. index::
421 builtin: len
422 single: subscription
423 object: mapping
424
425 These represent finite sets of objects indexed by arbitrary index sets. The
426 subscript notation ``a[k]`` selects the item indexed by ``k`` from the mapping
427 ``a``; this can be used in expressions and as the target of assignments or
428 :keyword:`del` statements. The built-in function :func:`len` returns the number
429 of items in a mapping.
430
431 There is currently a single intrinsic mapping type:
432
433 Dictionaries
434 .. index:: object: dictionary
435
436 These represent finite sets of objects indexed by nearly arbitrary values. The
437 only types of values not acceptable as keys are values containing lists or
438 dictionaries or other mutable types that are compared by value rather than by
439 object identity, the reason being that the efficient implementation of
440 dictionaries requires a key's hash value to remain constant. Numeric types used
441 for keys obey the normal rules for numeric comparison: if two numbers compare
442 equal (e.g., ``1`` and ``1.0``) then they can be used interchangeably to index
443 the same dictionary entry.
444
Lahfa Samy59c644e2020-03-26 15:54:04 +0100445 Dictionaries preserve insertion order, meaning that keys will be produced
446 in the same order they were added sequentially over the dictionary.
447 Replacing an existing key does not change the order, however removing a key
448 and re-inserting it will add it to the end instead of keeping its old place.
449
Georg Brandl116aa622007-08-15 14:28:22 +0000450 Dictionaries are mutable; they can be created by the ``{...}`` notation (see
451 section :ref:`dict`).
452
453 .. index::
Georg Brandl0a7ac7d2008-05-26 10:29:35 +0000454 module: dbm.ndbm
455 module: dbm.gnu
Georg Brandl116aa622007-08-15 14:28:22 +0000456
Benjamin Peterson9a46cab2008-09-08 02:49:30 +0000457 The extension modules :mod:`dbm.ndbm` and :mod:`dbm.gnu` provide
458 additional examples of mapping types, as does the :mod:`collections`
Nick Coghlan3a5d7e32008-08-31 12:40:14 +0000459 module.
Georg Brandl116aa622007-08-15 14:28:22 +0000460
Lahfa Samy59c644e2020-03-26 15:54:04 +0100461 .. versionchanged:: 3.7
462 Dictionaries did not preserve insertion order in versions of Python before 3.6.
463 In CPython 3.6, insertion order was preserved, but it was considered
464 an implementation detail at that time rather than a language guarantee.
465
Georg Brandl116aa622007-08-15 14:28:22 +0000466Callable types
467 .. index::
468 object: callable
469 pair: function; call
470 single: invocation
471 pair: function; argument
472
473 These are the types to which the function call operation (see section
474 :ref:`calls`) can be applied:
475
476 User-defined functions
477 .. index::
478 pair: user-defined; function
479 object: function
480 object: user-defined function
481
482 A user-defined function object is created by a function definition (see
483 section :ref:`function`). It should be called with an argument list
484 containing the same number of items as the function's formal parameter
485 list.
486
487 Special attributes:
488
Georg Brandl44ea77b2013-03-28 13:28:44 +0100489 .. tabularcolumns:: |l|L|l|
490
Martin Panterbae5d812016-06-18 03:57:31 +0000491 .. index::
492 single: __doc__ (function attribute)
493 single: __name__ (function attribute)
494 single: __module__ (function attribute)
495 single: __dict__ (function attribute)
496 single: __defaults__ (function attribute)
497 single: __closure__ (function attribute)
498 single: __code__ (function attribute)
499 single: __globals__ (function attribute)
500 single: __annotations__ (function attribute)
501 single: __kwdefaults__ (function attribute)
502 pair: global; namespace
503
Georg Brandl116aa622007-08-15 14:28:22 +0000504 +-------------------------+-------------------------------+-----------+
505 | Attribute | Meaning | |
506 +=========================+===============================+===========+
507 | :attr:`__doc__` | The function's documentation | Writable |
508 | | string, or ``None`` if | |
Ethan Furmanf87f5152014-10-17 22:25:22 -0700509 | | unavailable; not inherited by | |
Jules Lasne (jlasne)1fc5bf22019-03-27 11:10:33 +0100510 | | subclasses. | |
Georg Brandl116aa622007-08-15 14:28:22 +0000511 +-------------------------+-------------------------------+-----------+
Jules Lasne (jlasne)1fc5bf22019-03-27 11:10:33 +0100512 | :attr:`~definition.\ | The function's name. | Writable |
Martin Panterbae5d812016-06-18 03:57:31 +0000513 | __name__` | | |
Georg Brandl116aa622007-08-15 14:28:22 +0000514 +-------------------------+-------------------------------+-----------+
Martin Panterbae5d812016-06-18 03:57:31 +0000515 | :attr:`~definition.\ | The function's | Writable |
Jules Lasne (jlasne)1fc5bf22019-03-27 11:10:33 +0100516 | __qualname__` | :term:`qualified name`. | |
Antoine Pitrou86a36b52011-11-25 18:56:07 +0100517 | | | |
518 | | .. versionadded:: 3.3 | |
519 +-------------------------+-------------------------------+-----------+
Georg Brandl116aa622007-08-15 14:28:22 +0000520 | :attr:`__module__` | The name of the module the | Writable |
521 | | function was defined in, or | |
522 | | ``None`` if unavailable. | |
523 +-------------------------+-------------------------------+-----------+
524 | :attr:`__defaults__` | A tuple containing default | Writable |
525 | | argument values for those | |
526 | | arguments that have defaults, | |
527 | | or ``None`` if no arguments | |
Jules Lasne (jlasne)1fc5bf22019-03-27 11:10:33 +0100528 | | have a default value. | |
Georg Brandl116aa622007-08-15 14:28:22 +0000529 +-------------------------+-------------------------------+-----------+
530 | :attr:`__code__` | The code object representing | Writable |
531 | | the compiled function body. | |
532 +-------------------------+-------------------------------+-----------+
533 | :attr:`__globals__` | A reference to the dictionary | Read-only |
534 | | that holds the function's | |
535 | | global variables --- the | |
536 | | global namespace of the | |
537 | | module in which the function | |
538 | | was defined. | |
539 +-------------------------+-------------------------------+-----------+
Martin Panterbae5d812016-06-18 03:57:31 +0000540 | :attr:`~object.__dict__`| The namespace supporting | Writable |
Georg Brandl116aa622007-08-15 14:28:22 +0000541 | | arbitrary function | |
542 | | attributes. | |
543 +-------------------------+-------------------------------+-----------+
544 | :attr:`__closure__` | ``None`` or a tuple of cells | Read-only |
545 | | that contain bindings for the | |
546 | | function's free variables. | |
Lisa Roach64505a12017-06-08 04:43:26 -0700547 | | See below for information on | |
548 | | the ``cell_contents`` | |
549 | | attribute. | |
Georg Brandl116aa622007-08-15 14:28:22 +0000550 +-------------------------+-------------------------------+-----------+
551 | :attr:`__annotations__` | A dict containing annotations | Writable |
552 | | of parameters. The keys of | |
553 | | the dict are the parameter | |
Benjamin Peterson002033e2014-01-02 16:47:50 -0600554 | | names, and ``'return'`` for | |
Georg Brandl116aa622007-08-15 14:28:22 +0000555 | | the return annotation, if | |
556 | | provided. | |
557 +-------------------------+-------------------------------+-----------+
558 | :attr:`__kwdefaults__` | A dict containing defaults | Writable |
559 | | for keyword-only parameters. | |
560 +-------------------------+-------------------------------+-----------+
561
562 Most of the attributes labelled "Writable" check the type of the assigned value.
563
Georg Brandl116aa622007-08-15 14:28:22 +0000564 Function objects also support getting and setting arbitrary attributes, which
565 can be used, for example, to attach metadata to functions. Regular attribute
566 dot-notation is used to get and set such attributes. *Note that the current
567 implementation only supports function attributes on user-defined functions.
568 Function attributes on built-in functions may be supported in the future.*
569
Lisa Roach64505a12017-06-08 04:43:26 -0700570 A cell object has the attribute ``cell_contents``. This can be used to get
571 the value of the cell, as well as set the value.
572
Georg Brandl116aa622007-08-15 14:28:22 +0000573 Additional information about a function's definition can be retrieved from its
Pierre Glaserdf8d2cd2019-02-07 20:36:48 +0100574 code object; see the description of internal types below. The
575 :data:`cell <types.CellType>` type can be accessed in the :mod:`types`
576 module.
Georg Brandl116aa622007-08-15 14:28:22 +0000577
Georg Brandl2e0b7552007-11-27 12:43:08 +0000578 Instance methods
Georg Brandl116aa622007-08-15 14:28:22 +0000579 .. index::
580 object: method
581 object: user-defined method
582 pair: user-defined; method
583
Georg Brandl2e0b7552007-11-27 12:43:08 +0000584 An instance method object combines a class, a class instance and any
585 callable object (normally a user-defined function).
586
587 .. index::
588 single: __func__ (method attribute)
589 single: __self__ (method attribute)
590 single: __doc__ (method attribute)
591 single: __name__ (method attribute)
592 single: __module__ (method attribute)
Georg Brandl116aa622007-08-15 14:28:22 +0000593
Christian Heimesff737952007-11-27 10:40:20 +0000594 Special read-only attributes: :attr:`__self__` is the class instance object,
595 :attr:`__func__` is the function object; :attr:`__doc__` is the method's
Martin Panterbae5d812016-06-18 03:57:31 +0000596 documentation (same as ``__func__.__doc__``); :attr:`~definition.__name__` is the
Christian Heimesff737952007-11-27 10:40:20 +0000597 method name (same as ``__func__.__name__``); :attr:`__module__` is the
598 name of the module the method was defined in, or ``None`` if unavailable.
Georg Brandl116aa622007-08-15 14:28:22 +0000599
Georg Brandl116aa622007-08-15 14:28:22 +0000600 Methods also support accessing (but not setting) the arbitrary function
601 attributes on the underlying function object.
602
Georg Brandl2e0b7552007-11-27 12:43:08 +0000603 User-defined method objects may be created when getting an attribute of a
604 class (perhaps via an instance of that class), if that attribute is a
605 user-defined function object or a class method object.
Nick Coghlan3a5d7e32008-08-31 12:40:14 +0000606
Georg Brandl2e0b7552007-11-27 12:43:08 +0000607 When an instance method object is created by retrieving a user-defined
608 function object from a class via one of its instances, its
609 :attr:`__self__` attribute is the instance, and the method object is said
610 to be bound. The new method's :attr:`__func__` attribute is the original
611 function object.
Georg Brandl116aa622007-08-15 14:28:22 +0000612
Georg Brandl2e0b7552007-11-27 12:43:08 +0000613 When an instance method object is created by retrieving a class method
614 object from a class or instance, its :attr:`__self__` attribute is the
615 class itself, and its :attr:`__func__` attribute is the function object
616 underlying the class method.
Georg Brandl116aa622007-08-15 14:28:22 +0000617
Georg Brandl2e0b7552007-11-27 12:43:08 +0000618 When an instance method object is called, the underlying function
619 (:attr:`__func__`) is called, inserting the class instance
620 (:attr:`__self__`) in front of the argument list. For instance, when
621 :class:`C` is a class which contains a definition for a function
622 :meth:`f`, and ``x`` is an instance of :class:`C`, calling ``x.f(1)`` is
623 equivalent to calling ``C.f(x, 1)``.
Georg Brandl116aa622007-08-15 14:28:22 +0000624
Georg Brandl2e0b7552007-11-27 12:43:08 +0000625 When an instance method object is derived from a class method object, the
626 "class instance" stored in :attr:`__self__` will actually be the class
627 itself, so that calling either ``x.f(1)`` or ``C.f(1)`` is equivalent to
628 calling ``f(C,1)`` where ``f`` is the underlying function.
Georg Brandl116aa622007-08-15 14:28:22 +0000629
Georg Brandl2e0b7552007-11-27 12:43:08 +0000630 Note that the transformation from function object to instance method
631 object happens each time the attribute is retrieved from the instance. In
632 some cases, a fruitful optimization is to assign the attribute to a local
633 variable and call that local variable. Also notice that this
634 transformation only happens for user-defined functions; other callable
635 objects (and all non-callable objects) are retrieved without
636 transformation. It is also important to note that user-defined functions
637 which are attributes of a class instance are not converted to bound
638 methods; this *only* happens when the function is an attribute of the
639 class.
Georg Brandl116aa622007-08-15 14:28:22 +0000640
641 Generator functions
642 .. index::
643 single: generator; function
644 single: generator; iterator
645
646 A function or method which uses the :keyword:`yield` statement (see section
Nick Coghlan3a5d7e32008-08-31 12:40:14 +0000647 :ref:`yield`) is called a :dfn:`generator function`. Such a function, when
648 called, always returns an iterator object which can be used to execute the
Serhiy Storchaka0d196ed2013-10-09 14:02:31 +0300649 body of the function: calling the iterator's :meth:`iterator.__next__`
Ezio Melotti7fa82222012-10-12 13:42:08 +0300650 method will cause the function to execute until it provides a value
Serhiy Storchaka2b57c432018-12-19 08:09:46 +0200651 using the :keyword:`!yield` statement. When the function executes a
Georg Brandl116aa622007-08-15 14:28:22 +0000652 :keyword:`return` statement or falls off the end, a :exc:`StopIteration`
653 exception is raised and the iterator will have reached the end of the set of
654 values to be returned.
655
Yury Selivanovf3e40fa2015-05-21 11:50:30 -0400656 Coroutine functions
657 .. index::
658 single: coroutine; function
659
660 A function or method which is defined using :keyword:`async def` is called
661 a :dfn:`coroutine function`. Such a function, when called, returns a
662 :term:`coroutine` object. It may contain :keyword:`await` expressions,
663 as well as :keyword:`async with` and :keyword:`async for` statements. See
Yury Selivanov66f88282015-06-24 11:04:15 -0400664 also the :ref:`coroutine-objects` section.
Yury Selivanovf3e40fa2015-05-21 11:50:30 -0400665
Yury Selivanov03660042016-12-15 17:36:05 -0500666 Asynchronous generator functions
667 .. index::
668 single: asynchronous generator; function
669 single: asynchronous generator; asynchronous iterator
670
671 A function or method which is defined using :keyword:`async def` and
672 which uses the :keyword:`yield` statement is called a
673 :dfn:`asynchronous generator function`. Such a function, when called,
674 returns an asynchronous iterator object which can be used in an
675 :keyword:`async for` statement to execute the body of the function.
676
677 Calling the asynchronous iterator's :meth:`aiterator.__anext__` method
678 will return an :term:`awaitable` which when awaited
679 will execute until it provides a value using the :keyword:`yield`
680 expression. When the function executes an empty :keyword:`return`
681 statement or falls off the end, a :exc:`StopAsyncIteration` exception
682 is raised and the asynchronous iterator will have reached the end of
683 the set of values to be yielded.
684
Georg Brandl116aa622007-08-15 14:28:22 +0000685 Built-in functions
686 .. index::
687 object: built-in function
688 object: function
689 pair: C; language
690
691 A built-in function object is a wrapper around a C function. Examples of
692 built-in functions are :func:`len` and :func:`math.sin` (:mod:`math` is a
693 standard built-in module). The number and type of the arguments are
694 determined by the C function. Special read-only attributes:
695 :attr:`__doc__` is the function's documentation string, or ``None`` if
Martin Panterbae5d812016-06-18 03:57:31 +0000696 unavailable; :attr:`~definition.__name__` is the function's name; :attr:`__self__` is
Georg Brandl116aa622007-08-15 14:28:22 +0000697 set to ``None`` (but see the next item); :attr:`__module__` is the name of
698 the module the function was defined in or ``None`` if unavailable.
699
700 Built-in methods
701 .. index::
702 object: built-in method
703 object: method
704 pair: built-in; method
705
706 This is really a different disguise of a built-in function, this time containing
707 an object passed to the C function as an implicit extra argument. An example of
708 a built-in method is ``alist.append()``, assuming *alist* is a list object. In
709 this case, the special read-only attribute :attr:`__self__` is set to the object
Éric Araujoc9562f32010-12-26 02:18:49 +0000710 denoted by *alist*.
Georg Brandl116aa622007-08-15 14:28:22 +0000711
Georg Brandl85eb8c12007-08-31 16:33:38 +0000712 Classes
713 Classes are callable. These objects normally act as factories for new
714 instances of themselves, but variations are possible for class types that
715 override :meth:`__new__`. The arguments of the call are passed to
716 :meth:`__new__` and, in the typical case, to :meth:`__init__` to
717 initialize the new instance.
Georg Brandl116aa622007-08-15 14:28:22 +0000718
Georg Brandl85eb8c12007-08-31 16:33:38 +0000719 Class Instances
720 Instances of arbitrary classes can be made callable by defining a
721 :meth:`__call__` method in their class.
Georg Brandl116aa622007-08-15 14:28:22 +0000722
Georg Brandl116aa622007-08-15 14:28:22 +0000723
724Modules
725 .. index::
726 statement: import
727 object: module
728
Barry Warsawd7d21942012-07-29 16:36:17 -0400729 Modules are a basic organizational unit of Python code, and are created by
Barry Warsawdadebab2012-07-31 16:03:09 -0400730 the :ref:`import system <importsystem>` as invoked either by the
Serhiy Storchaka2b57c432018-12-19 08:09:46 +0200731 :keyword:`import` statement, or by calling
Barry Warsawdadebab2012-07-31 16:03:09 -0400732 functions such as :func:`importlib.import_module` and built-in
733 :func:`__import__`. A module object has a namespace implemented by a
734 dictionary object (this is the dictionary referenced by the ``__globals__``
735 attribute of functions defined in the module). Attribute references are
736 translated to lookups in this dictionary, e.g., ``m.x`` is equivalent to
737 ``m.__dict__["x"]``. A module object does not contain the code object used
738 to initialize the module (since it isn't needed once the initialization is
739 done).
Georg Brandl116aa622007-08-15 14:28:22 +0000740
Barry Warsawd7d21942012-07-29 16:36:17 -0400741 Attribute assignment updates the module's namespace dictionary, e.g.,
742 ``m.x = 1`` is equivalent to ``m.__dict__["x"] = 1``.
Georg Brandl116aa622007-08-15 14:28:22 +0000743
Yury Selivanovf8cb8a12016-09-08 20:50:03 -0700744 .. index::
745 single: __name__ (module attribute)
746 single: __doc__ (module attribute)
747 single: __file__ (module attribute)
748 single: __annotations__ (module attribute)
749 pair: module; namespace
750
751 Predefined (writable) attributes: :attr:`__name__` is the module's name;
752 :attr:`__doc__` is the module's documentation string, or ``None`` if
753 unavailable; :attr:`__annotations__` (optional) is a dictionary containing
754 :term:`variable annotations <variable annotation>` collected during module
755 body execution; :attr:`__file__` is the pathname of the file from which the
756 module was loaded, if it was loaded from a file. The :attr:`__file__`
757 attribute may be missing for certain types of modules, such as C modules
758 that are statically linked into the interpreter; for extension modules
759 loaded dynamically from a shared library, it is the pathname of the shared
760 library file.
761
Georg Brandl116aa622007-08-15 14:28:22 +0000762 .. index:: single: __dict__ (module attribute)
763
Yury Selivanovf8cb8a12016-09-08 20:50:03 -0700764 Special read-only attribute: :attr:`~object.__dict__` is the module's
765 namespace as a dictionary object.
Georg Brandl116aa622007-08-15 14:28:22 +0000766
Benjamin Peterson5c4bfc42010-10-12 22:57:59 +0000767 .. impl-detail::
768
769 Because of the way CPython clears module dictionaries, the module
770 dictionary will be cleared when the module falls out of scope even if the
771 dictionary still has live references. To avoid this, copy the dictionary
772 or keep the module around while using its dictionary directly.
773
Georg Brandl85eb8c12007-08-31 16:33:38 +0000774Custom classes
Georg Brandl5dbb84a2009-09-02 20:31:26 +0000775 Custom class types are typically created by class definitions (see section
Nick Coghlan3a5d7e32008-08-31 12:40:14 +0000776 :ref:`class`). A class has a namespace implemented by a dictionary object.
777 Class attribute references are translated to lookups in this dictionary, e.g.,
778 ``C.x`` is translated to ``C.__dict__["x"]`` (although there are a number of
779 hooks which allow for other means of locating attributes). When the attribute
780 name is not found there, the attribute search continues in the base classes.
781 This search of the base classes uses the C3 method resolution order which
782 behaves correctly even in the presence of 'diamond' inheritance structures
783 where there are multiple inheritance paths leading back to a common ancestor.
784 Additional details on the C3 MRO used by Python can be found in the
785 documentation accompanying the 2.3 release at
Georg Brandle73778c2014-10-29 08:36:35 +0100786 https://www.python.org/download/releases/2.3/mro/.
Georg Brandl116aa622007-08-15 14:28:22 +0000787
Nick Coghlan3a5d7e32008-08-31 12:40:14 +0000788 .. XXX: Could we add that MRO doc as an appendix to the language ref?
Georg Brandl85eb8c12007-08-31 16:33:38 +0000789
Georg Brandl116aa622007-08-15 14:28:22 +0000790 .. index::
791 object: class
792 object: class instance
793 object: instance
794 pair: class object; call
795 single: container
796 object: dictionary
797 pair: class; attribute
798
799 When a class attribute reference (for class :class:`C`, say) would yield a
Georg Brandl2e0b7552007-11-27 12:43:08 +0000800 class method object, it is transformed into an instance method object whose
Zach Mitchell00818c82018-06-02 10:29:47 -0400801 :attr:`__self__` attribute is :class:`C`. When it would yield a static
Georg Brandl2e0b7552007-11-27 12:43:08 +0000802 method object, it is transformed into the object wrapped by the static method
803 object. See section :ref:`descriptors` for another way in which attributes
804 retrieved from a class may differ from those actually contained in its
Martin Panterbae5d812016-06-18 03:57:31 +0000805 :attr:`~object.__dict__`.
Georg Brandl116aa622007-08-15 14:28:22 +0000806
807 .. index:: triple: class; attribute; assignment
808
809 Class attribute assignments update the class's dictionary, never the dictionary
810 of a base class.
811
812 .. index:: pair: class object; call
813
814 A class object can be called (see above) to yield a class instance (see below).
815
816 .. index::
817 single: __name__ (class attribute)
818 single: __module__ (class attribute)
819 single: __dict__ (class attribute)
820 single: __bases__ (class attribute)
821 single: __doc__ (class attribute)
Yury Selivanovf8cb8a12016-09-08 20:50:03 -0700822 single: __annotations__ (class attribute)
Georg Brandl116aa622007-08-15 14:28:22 +0000823
Martin Panterbae5d812016-06-18 03:57:31 +0000824 Special attributes: :attr:`~definition.__name__` is the class name; :attr:`__module__` is
825 the module name in which the class was defined; :attr:`~object.__dict__` is the
Serhiy Storchaka0d196ed2013-10-09 14:02:31 +0300826 dictionary containing the class's namespace; :attr:`~class.__bases__` is a
Berker Peksag7b4e5512017-01-03 03:34:15 +0300827 tuple containing the base classes, in the order of their occurrence in the
Berker Peksag406c2522017-01-03 03:35:49 +0300828 base class list; :attr:`__doc__` is the class's documentation string,
829 or ``None`` if undefined; :attr:`__annotations__` (optional) is a dictionary
830 containing :term:`variable annotations <variable annotation>` collected during
Yury Selivanovf8cb8a12016-09-08 20:50:03 -0700831 class body execution.
Georg Brandl116aa622007-08-15 14:28:22 +0000832
833Class instances
834 .. index::
835 object: class instance
836 object: instance
837 pair: class; instance
838 pair: class instance; attribute
839
Georg Brandl2e0b7552007-11-27 12:43:08 +0000840 A class instance is created by calling a class object (see above). A class
841 instance has a namespace implemented as a dictionary which is the first place
842 in which attribute references are searched. When an attribute is not found
843 there, and the instance's class has an attribute by that name, the search
844 continues with the class attributes. If a class attribute is found that is a
845 user-defined function object, it is transformed into an instance method
846 object whose :attr:`__self__` attribute is the instance. Static method and
847 class method objects are also transformed; see above under "Classes". See
848 section :ref:`descriptors` for another way in which attributes of a class
849 retrieved via its instances may differ from the objects actually stored in
Martin Panterbae5d812016-06-18 03:57:31 +0000850 the class's :attr:`~object.__dict__`. If no class attribute is found, and the
Georg Brandl2e0b7552007-11-27 12:43:08 +0000851 object's class has a :meth:`__getattr__` method, that is called to satisfy
852 the lookup.
Georg Brandl116aa622007-08-15 14:28:22 +0000853
854 .. index:: triple: class instance; attribute; assignment
855
856 Attribute assignments and deletions update the instance's dictionary, never a
857 class's dictionary. If the class has a :meth:`__setattr__` or
858 :meth:`__delattr__` method, this is called instead of updating the instance
859 dictionary directly.
860
861 .. index::
862 object: numeric
863 object: sequence
864 object: mapping
865
866 Class instances can pretend to be numbers, sequences, or mappings if they have
867 methods with certain special names. See section :ref:`specialnames`.
868
869 .. index::
870 single: __dict__ (instance attribute)
871 single: __class__ (instance attribute)
872
Serhiy Storchaka0d196ed2013-10-09 14:02:31 +0300873 Special attributes: :attr:`~object.__dict__` is the attribute dictionary;
874 :attr:`~instance.__class__` is the instance's class.
Georg Brandl116aa622007-08-15 14:28:22 +0000875
Antoine Pitrou4adb2882010-01-04 18:50:53 +0000876I/O objects (also known as file objects)
Georg Brandl116aa622007-08-15 14:28:22 +0000877 .. index::
Georg Brandl116aa622007-08-15 14:28:22 +0000878 builtin: open
Antoine Pitrou4adb2882010-01-04 18:50:53 +0000879 module: io
Georg Brandl116aa622007-08-15 14:28:22 +0000880 single: popen() (in module os)
881 single: makefile() (socket method)
882 single: sys.stdin
883 single: sys.stdout
884 single: sys.stderr
885 single: stdio
886 single: stdin (in module sys)
887 single: stdout (in module sys)
888 single: stderr (in module sys)
889
Antoine Pitrou0b65b0f2010-09-15 09:58:26 +0000890 A :term:`file object` represents an open file. Various shortcuts are
891 available to create file objects: the :func:`open` built-in function, and
Serhiy Storchaka0d196ed2013-10-09 14:02:31 +0300892 also :func:`os.popen`, :func:`os.fdopen`, and the
893 :meth:`~socket.socket.makefile` method of socket objects (and perhaps by
894 other functions or methods provided by extension modules).
Antoine Pitrou4adb2882010-01-04 18:50:53 +0000895
896 The objects ``sys.stdin``, ``sys.stdout`` and ``sys.stderr`` are
897 initialized to file objects corresponding to the interpreter's standard
898 input, output and error streams; they are all open in text mode and
899 therefore follow the interface defined by the :class:`io.TextIOBase`
900 abstract class.
Georg Brandl116aa622007-08-15 14:28:22 +0000901
902Internal types
903 .. index::
904 single: internal type
905 single: types, internal
906
907 A few types used internally by the interpreter are exposed to the user. Their
908 definitions may change with future versions of the interpreter, but they are
909 mentioned here for completeness.
910
Tommy Beadlee9b84032016-06-02 19:26:51 -0400911 .. index:: bytecode, object; code, code object
Georg Brandl116aa622007-08-15 14:28:22 +0000912
Tommy Beadlee9b84032016-06-02 19:26:51 -0400913 Code objects
Georg Brandl9afde1c2007-11-01 20:32:30 +0000914 Code objects represent *byte-compiled* executable Python code, or :term:`bytecode`.
Georg Brandl116aa622007-08-15 14:28:22 +0000915 The difference between a code object and a function object is that the function
916 object contains an explicit reference to the function's globals (the module in
917 which it was defined), while a code object contains no context; also the default
918 argument values are stored in the function object, not in the code object
919 (because they represent values calculated at run-time). Unlike function
920 objects, code objects are immutable and contain no references (directly or
921 indirectly) to mutable objects.
922
Senthil Kumaran7cafd262010-10-02 03:16:04 +0000923 .. index::
924 single: co_argcount (code object attribute)
Pablo Galindoed222a72019-05-31 12:13:04 +0100925 single: co_posonlyargcount (code object attribute)
926 single: co_kwonlyargcount (code object attribute)
Senthil Kumaran7cafd262010-10-02 03:16:04 +0000927 single: co_code (code object attribute)
928 single: co_consts (code object attribute)
929 single: co_filename (code object attribute)
930 single: co_firstlineno (code object attribute)
931 single: co_flags (code object attribute)
932 single: co_lnotab (code object attribute)
933 single: co_name (code object attribute)
934 single: co_names (code object attribute)
935 single: co_nlocals (code object attribute)
936 single: co_stacksize (code object attribute)
937 single: co_varnames (code object attribute)
938 single: co_cellvars (code object attribute)
939 single: co_freevars (code object attribute)
940
Georg Brandl116aa622007-08-15 14:28:22 +0000941 Special read-only attributes: :attr:`co_name` gives the function name;
Pablo Galindocd74e662019-06-01 18:08:04 +0100942 :attr:`co_argcount` is the total number of positional arguments
943 (including positional-only arguments and arguments with default values);
944 :attr:`co_posonlyargcount` is the number of positional-only arguments
945 (including arguments with default values); :attr:`co_kwonlyargcount` is
946 the number of keyword-only arguments (including arguments with default
947 values); :attr:`co_nlocals` is the number of local variables used by the
948 function (including arguments); :attr:`co_varnames` is a tuple containing
949 the names of the local variables (starting with the argument names);
950 :attr:`co_cellvars` is a tuple containing the names of local variables
Pablo Galindoed222a72019-05-31 12:13:04 +0100951 that are referenced by nested functions; :attr:`co_freevars` is a tuple
Pablo Galindocd74e662019-06-01 18:08:04 +0100952 containing the names of free variables; :attr:`co_code` is a string
953 representing the sequence of bytecode instructions; :attr:`co_consts` is
954 a tuple containing the literals used by the bytecode; :attr:`co_names` is
955 a tuple containing the names used by the bytecode; :attr:`co_filename` is
956 the filename from which the code was compiled; :attr:`co_firstlineno` is
957 the first line number of the function; :attr:`co_lnotab` is a string
958 encoding the mapping from bytecode offsets to line numbers (for details
959 see the source code of the interpreter); :attr:`co_stacksize` is the
Batuhan TaÅŸkayad5872722019-12-16 01:02:47 +0300960 required stack size; :attr:`co_flags` is an integer encoding a number
961 of flags for the interpreter.
Georg Brandl116aa622007-08-15 14:28:22 +0000962
Georg Brandl116aa622007-08-15 14:28:22 +0000963 .. index:: object: generator
964
965 The following flag bits are defined for :attr:`co_flags`: bit ``0x04`` is set if
966 the function uses the ``*arguments`` syntax to accept an arbitrary number of
967 positional arguments; bit ``0x08`` is set if the function uses the
968 ``**keywords`` syntax to accept arbitrary keyword arguments; bit ``0x20`` is set
969 if the function is a generator.
970
971 Future feature declarations (``from __future__ import division``) also use bits
972 in :attr:`co_flags` to indicate whether a code object was compiled with a
973 particular feature enabled: bit ``0x2000`` is set if the function was compiled
974 with future division enabled; bits ``0x10`` and ``0x1000`` were used in earlier
975 versions of Python.
976
977 Other bits in :attr:`co_flags` are reserved for internal use.
978
979 .. index:: single: documentation string
980
981 If a code object represents a function, the first item in :attr:`co_consts` is
982 the documentation string of the function, or ``None`` if undefined.
983
Georg Brandla6053b42009-09-01 08:11:14 +0000984 .. _frame-objects:
985
Georg Brandl116aa622007-08-15 14:28:22 +0000986 Frame objects
987 .. index:: object: frame
988
989 Frame objects represent execution frames. They may occur in traceback objects
Nick Coghlanaec75322018-02-13 18:10:58 +1000990 (see below), and are also passed to registered trace functions.
Georg Brandl116aa622007-08-15 14:28:22 +0000991
992 .. index::
993 single: f_back (frame attribute)
994 single: f_code (frame attribute)
995 single: f_globals (frame attribute)
996 single: f_locals (frame attribute)
997 single: f_lasti (frame attribute)
998 single: f_builtins (frame attribute)
999
1000 Special read-only attributes: :attr:`f_back` is to the previous stack frame
1001 (towards the caller), or ``None`` if this is the bottom stack frame;
1002 :attr:`f_code` is the code object being executed in this frame; :attr:`f_locals`
1003 is the dictionary used to look up local variables; :attr:`f_globals` is used for
1004 global variables; :attr:`f_builtins` is used for built-in (intrinsic) names;
1005 :attr:`f_lasti` gives the precise instruction (this is an index into the
1006 bytecode string of the code object).
1007
Ryan Hileman9a2c2a92021-04-29 16:15:55 -07001008 Accessing ``f_code`` raises an :ref:`auditing event <auditing>`
1009 ``object.__getattr__`` with arguments ``obj`` and ``"f_code"``.
1010
Georg Brandl116aa622007-08-15 14:28:22 +00001011 .. index::
1012 single: f_trace (frame attribute)
Nick Coghlan5a851672017-09-08 10:14:16 +10001013 single: f_trace_lines (frame attribute)
1014 single: f_trace_opcodes (frame attribute)
Georg Brandl116aa622007-08-15 14:28:22 +00001015 single: f_lineno (frame attribute)
1016
1017 Special writable attributes: :attr:`f_trace`, if not ``None``, is a function
Nick Coghlan5a851672017-09-08 10:14:16 +10001018 called for various events during code execution (this is used by the debugger).
1019 Normally an event is triggered for each new source line - this can be
1020 disabled by setting :attr:`f_trace_lines` to :const:`False`.
1021
1022 Implementations *may* allow per-opcode events to be requested by setting
1023 :attr:`f_trace_opcodes` to :const:`True`. Note that this may lead to
1024 undefined interpreter behaviour if exceptions raised by the trace
1025 function escape to the function being traced.
1026
Benjamin Petersoneec3d712008-06-11 15:59:43 +00001027 :attr:`f_lineno` is the current line number of the frame --- writing to this
1028 from within a trace function jumps to the given line (only for the bottom-most
1029 frame). A debugger can implement a Jump command (aka Set Next Statement)
1030 by writing to f_lineno.
Georg Brandl116aa622007-08-15 14:28:22 +00001031
Antoine Pitrou58720d62013-08-05 23:26:40 +02001032 Frame objects support one method:
1033
1034 .. method:: frame.clear()
1035
1036 This method clears all references to local variables held by the
1037 frame. Also, if the frame belonged to a generator, the generator
1038 is finalized. This helps break reference cycles involving frame
1039 objects (for example when catching an exception and storing its
1040 traceback for later use).
1041
1042 :exc:`RuntimeError` is raised if the frame is currently executing.
1043
1044 .. versionadded:: 3.4
1045
Nick Coghlanaec75322018-02-13 18:10:58 +10001046 .. _traceback-objects:
1047
Georg Brandl116aa622007-08-15 14:28:22 +00001048 Traceback objects
1049 .. index::
1050 object: traceback
1051 pair: stack; trace
1052 pair: exception; handler
1053 pair: execution; stack
1054 single: exc_info (in module sys)
Georg Brandl116aa622007-08-15 14:28:22 +00001055 single: last_traceback (in module sys)
1056 single: sys.exc_info
1057 single: sys.last_traceback
1058
1059 Traceback objects represent a stack trace of an exception. A traceback object
Nick Coghlanaec75322018-02-13 18:10:58 +10001060 is implicitly created when an exception occurs, and may also be explicitly
1061 created by calling :class:`types.TracebackType`.
1062
1063 For implicitly created tracebacks, when the search for an exception handler
Georg Brandl116aa622007-08-15 14:28:22 +00001064 unwinds the execution stack, at each unwound level a traceback object is
1065 inserted in front of the current traceback. When an exception handler is
1066 entered, the stack trace is made available to the program. (See section
1067 :ref:`try`.) It is accessible as the third item of the
Nick Coghlanaec75322018-02-13 18:10:58 +10001068 tuple returned by ``sys.exc_info()``, and as the ``__traceback__`` attribute
1069 of the caught exception.
1070
1071 When the program contains no suitable
Georg Brandl116aa622007-08-15 14:28:22 +00001072 handler, the stack trace is written (nicely formatted) to the standard error
1073 stream; if the interpreter is interactive, it is also made available to the user
1074 as ``sys.last_traceback``.
1075
Nick Coghlanaec75322018-02-13 18:10:58 +10001076 For explicitly created tracebacks, it is up to the creator of the traceback
1077 to determine how the ``tb_next`` attributes should be linked to form a
1078 full stack trace.
1079
Georg Brandl116aa622007-08-15 14:28:22 +00001080 .. index::
Georg Brandl116aa622007-08-15 14:28:22 +00001081 single: tb_frame (traceback attribute)
1082 single: tb_lineno (traceback attribute)
1083 single: tb_lasti (traceback attribute)
1084 statement: try
1085
Nick Coghlanaec75322018-02-13 18:10:58 +10001086 Special read-only attributes:
1087 :attr:`tb_frame` points to the execution frame of the current level;
1088 :attr:`tb_lineno` gives the line number where the exception occurred;
1089 :attr:`tb_lasti` indicates the precise instruction.
1090 The line number and last instruction in the traceback may differ from the
1091 line number of its frame object if the exception occurred in a
1092 :keyword:`try` statement with no matching except clause or with a
1093 finally clause.
1094
Ryan Hileman9a2c2a92021-04-29 16:15:55 -07001095 Accessing ``tb_frame`` raises an :ref:`auditing event <auditing>`
1096 ``object.__getattr__`` with arguments ``obj`` and ``"tb_frame"``.
1097
Nick Coghlanaec75322018-02-13 18:10:58 +10001098 .. index::
1099 single: tb_next (traceback attribute)
1100
1101 Special writable attribute: :attr:`tb_next` is the next level in the stack
1102 trace (towards the frame where the exception occurred), or ``None`` if
1103 there is no next level.
1104
Berker Peksag3c1b5902018-08-06 23:52:49 +03001105 .. versionchanged:: 3.7
1106 Traceback objects can now be explicitly instantiated from Python code,
1107 and the ``tb_next`` attribute of existing instances can be updated.
Georg Brandl116aa622007-08-15 14:28:22 +00001108
1109 Slice objects
1110 .. index:: builtin: slice
1111
Georg Brandlcb8ecb12007-09-04 06:35:14 +00001112 Slice objects are used to represent slices for :meth:`__getitem__`
1113 methods. They are also created by the built-in :func:`slice` function.
Georg Brandl116aa622007-08-15 14:28:22 +00001114
1115 .. index::
1116 single: start (slice object attribute)
1117 single: stop (slice object attribute)
1118 single: step (slice object attribute)
1119
Serhiy Storchaka0d196ed2013-10-09 14:02:31 +03001120 Special read-only attributes: :attr:`~slice.start` is the lower bound;
1121 :attr:`~slice.stop` is the upper bound; :attr:`~slice.step` is the step
1122 value; each is ``None`` if omitted. These attributes can have any type.
Georg Brandl116aa622007-08-15 14:28:22 +00001123
1124 Slice objects support one method:
1125
Georg Brandl116aa622007-08-15 14:28:22 +00001126 .. method:: slice.indices(self, length)
1127
Georg Brandlcb8ecb12007-09-04 06:35:14 +00001128 This method takes a single integer argument *length* and computes
1129 information about the slice that the slice object would describe if
1130 applied to a sequence of *length* items. It returns a tuple of three
1131 integers; respectively these are the *start* and *stop* indices and the
1132 *step* or stride length of the slice. Missing or out-of-bounds indices
1133 are handled in a manner consistent with regular slices.
Georg Brandl116aa622007-08-15 14:28:22 +00001134
Georg Brandl116aa622007-08-15 14:28:22 +00001135 Static method objects
1136 Static method objects provide a way of defeating the transformation of function
1137 objects to method objects described above. A static method object is a wrapper
1138 around any other object, usually a user-defined method object. When a static
1139 method object is retrieved from a class or a class instance, the object actually
1140 returned is the wrapped object, which is not subject to any further
Victor Stinner553ee272021-04-12 00:21:22 +02001141 transformation. Static method objects are also callable. Static method
1142 objects are created by the built-in :func:`staticmethod` constructor.
Georg Brandl116aa622007-08-15 14:28:22 +00001143
1144 Class method objects
1145 A class method object, like a static method object, is a wrapper around another
1146 object that alters the way in which that object is retrieved from classes and
1147 class instances. The behaviour of class method objects upon such retrieval is
1148 described above, under "User-defined methods". Class method objects are created
1149 by the built-in :func:`classmethod` constructor.
1150
Georg Brandl116aa622007-08-15 14:28:22 +00001151
Georg Brandl116aa622007-08-15 14:28:22 +00001152.. _specialnames:
1153
1154Special method names
1155====================
1156
1157.. index::
1158 pair: operator; overloading
1159 single: __getitem__() (mapping object method)
1160
1161A class can implement certain operations that are invoked by special syntax
1162(such as arithmetic operations or subscripting and slicing) by defining methods
1163with special names. This is Python's approach to :dfn:`operator overloading`,
1164allowing classes to define their own behavior with respect to language
1165operators. For instance, if a class defines a method named :meth:`__getitem__`,
Nick Coghlan3a5d7e32008-08-31 12:40:14 +00001166and ``x`` is an instance of this class, then ``x[i]`` is roughly equivalent
1167to ``type(x).__getitem__(x, i)``. Except where mentioned, attempts to execute an
1168operation raise an exception when no appropriate method is defined (typically
1169:exc:`AttributeError` or :exc:`TypeError`).
Georg Brandl65ea9bd2007-09-05 13:36:27 +00001170
Guido van Rossum97c1adf2016-08-18 09:22:23 -07001171Setting a special method to ``None`` indicates that the corresponding
1172operation is not available. For example, if a class sets
1173:meth:`__iter__` to ``None``, the class is not iterable, so calling
1174:func:`iter` on its instances will raise a :exc:`TypeError` (without
1175falling back to :meth:`__getitem__`). [#]_
1176
Georg Brandl116aa622007-08-15 14:28:22 +00001177When implementing a class that emulates any built-in type, it is important that
1178the emulation only be implemented to the degree that it makes sense for the
1179object being modelled. For example, some sequences may work well with retrieval
1180of individual elements, but extracting a slice may not make sense. (One example
Serhiy Storchaka0d196ed2013-10-09 14:02:31 +03001181of this is the :class:`~xml.dom.NodeList` interface in the W3C's Document
1182Object Model.)
Georg Brandl116aa622007-08-15 14:28:22 +00001183
1184
1185.. _customization:
1186
1187Basic customization
1188-------------------
1189
Georg Brandl116aa622007-08-15 14:28:22 +00001190.. method:: object.__new__(cls[, ...])
1191
Georg Brandlaf265f42008-12-07 15:06:20 +00001192 .. index:: pair: subclassing; immutable types
1193
Georg Brandl116aa622007-08-15 14:28:22 +00001194 Called to create a new instance of class *cls*. :meth:`__new__` is a static
1195 method (special-cased so you need not declare it as such) that takes the class
1196 of which an instance was requested as its first argument. The remaining
1197 arguments are those passed to the object constructor expression (the call to the
1198 class). The return value of :meth:`__new__` should be the new object instance
1199 (usually an instance of *cls*).
1200
1201 Typical implementations create a new instance of the class by invoking the
csabella12b1c182017-05-14 20:42:00 -07001202 superclass's :meth:`__new__` method using ``super().__new__(cls[, ...])``
1203 with appropriate arguments and then modifying the newly-created instance
1204 as necessary before returning it.
Georg Brandl116aa622007-08-15 14:28:22 +00001205
Joannah Nanjekye6b16d932019-08-26 03:53:11 -03001206 If :meth:`__new__` is invoked during object construction and it returns an
1207 instance or subclass of *cls*, then the new instance’s :meth:`__init__` method
1208 will be invoked like ``__init__(self[, ...])``, where *self* is the new instance
1209 and the remaining arguments are the same as were passed to the object constructor.
Georg Brandl116aa622007-08-15 14:28:22 +00001210
1211 If :meth:`__new__` does not return an instance of *cls*, then the new instance's
1212 :meth:`__init__` method will not be invoked.
1213
1214 :meth:`__new__` is intended mainly to allow subclasses of immutable types (like
Christian Heimes790c8232008-01-07 21:14:23 +00001215 int, str, or tuple) to customize instance creation. It is also commonly
1216 overridden in custom metaclasses in order to customize class creation.
Georg Brandl116aa622007-08-15 14:28:22 +00001217
1218
1219.. method:: object.__init__(self[, ...])
1220
1221 .. index:: pair: class; constructor
1222
Ethan Furman119479f2015-01-14 21:56:10 -08001223 Called after the instance has been created (by :meth:`__new__`), but before
1224 it is returned to the caller. The arguments are those passed to the
1225 class constructor expression. If a base class has an :meth:`__init__`
1226 method, the derived class's :meth:`__init__` method, if any, must explicitly
1227 call it to ensure proper initialization of the base class part of the
csabella12b1c182017-05-14 20:42:00 -07001228 instance; for example: ``super().__init__([args...])``.
Ethan Furman119479f2015-01-14 21:56:10 -08001229
1230 Because :meth:`__new__` and :meth:`__init__` work together in constructing
Raymond Hettinger7ea386e2016-08-25 21:11:50 -07001231 objects (:meth:`__new__` to create it, and :meth:`__init__` to customize it),
Ethan Furman119479f2015-01-14 21:56:10 -08001232 no non-``None`` value may be returned by :meth:`__init__`; doing so will
1233 cause a :exc:`TypeError` to be raised at runtime.
Georg Brandl116aa622007-08-15 14:28:22 +00001234
1235
1236.. method:: object.__del__(self)
1237
1238 .. index::
1239 single: destructor
Antoine Pitrou4b965932017-12-19 19:48:45 +01001240 single: finalizer
Georg Brandl116aa622007-08-15 14:28:22 +00001241 statement: del
1242
1243 Called when the instance is about to be destroyed. This is also called a
Antoine Pitrou4b965932017-12-19 19:48:45 +01001244 finalizer or (improperly) a destructor. If a base class has a
1245 :meth:`__del__` method, the derived class's :meth:`__del__` method,
1246 if any, must explicitly call it to ensure proper deletion of the base
1247 class part of the instance.
1248
1249 It is possible (though not recommended!) for the :meth:`__del__` method
1250 to postpone destruction of the instance by creating a new reference to
1251 it. This is called object *resurrection*. It is implementation-dependent
1252 whether :meth:`__del__` is called a second time when a resurrected object
1253 is about to be destroyed; the current :term:`CPython` implementation
1254 only calls it once.
1255
1256 It is not guaranteed that :meth:`__del__` methods are called for objects
1257 that still exist when the interpreter exits.
Georg Brandl116aa622007-08-15 14:28:22 +00001258
1259 .. note::
1260
1261 ``del x`` doesn't directly call ``x.__del__()`` --- the former decrements
1262 the reference count for ``x`` by one, and the latter is only called when
Antoine Pitrou4b965932017-12-19 19:48:45 +01001263 ``x``'s reference count reaches zero.
1264
1265 .. impl-detail::
1266 It is possible for a reference cycle to prevent the reference count
1267 of an object from going to zero. In this case, the cycle will be
1268 later detected and deleted by the :term:`cyclic garbage collector
1269 <garbage collection>`. A common cause of reference cycles is when
1270 an exception has been caught in a local variable. The frame's
1271 locals then reference the exception, which references its own
1272 traceback, which references the locals of all frames caught in the
1273 traceback.
1274
1275 .. seealso::
1276 Documentation for the :mod:`gc` module.
Georg Brandl116aa622007-08-15 14:28:22 +00001277
1278 .. warning::
1279
1280 Due to the precarious circumstances under which :meth:`__del__` methods are
1281 invoked, exceptions that occur during their execution are ignored, and a warning
Antoine Pitrou4b965932017-12-19 19:48:45 +01001282 is printed to ``sys.stderr`` instead. In particular:
Georg Brandl116aa622007-08-15 14:28:22 +00001283
Antoine Pitrou4b965932017-12-19 19:48:45 +01001284 * :meth:`__del__` can be invoked when arbitrary code is being executed,
1285 including from any arbitrary thread. If :meth:`__del__` needs to take
1286 a lock or invoke any other blocking resource, it may deadlock as
1287 the resource may already be taken by the code that gets interrupted
1288 to execute :meth:`__del__`.
Chris Jerdonek5fae0e52012-11-20 17:45:51 -08001289
Antoine Pitrou4b965932017-12-19 19:48:45 +01001290 * :meth:`__del__` can be executed during interpreter shutdown. As a
1291 consequence, the global variables it needs to access (including other
1292 modules) may already have been deleted or set to ``None``. Python
1293 guarantees that globals whose name begins with a single underscore
1294 are deleted from their module before other globals are deleted; if
1295 no other references to such globals exist, this may help in assuring
1296 that imported modules are still available at the time when the
1297 :meth:`__del__` method is called.
1298
1299
1300 .. index::
1301 single: repr() (built-in function); __repr__() (object method)
Georg Brandl116aa622007-08-15 14:28:22 +00001302
1303.. method:: object.__repr__(self)
1304
Benjamin Peterson1c9313f2008-10-12 12:51:12 +00001305 Called by the :func:`repr` built-in function to compute the "official" string
1306 representation of an object. If at all possible, this should look like a
1307 valid Python expression that could be used to recreate an object with the
1308 same value (given an appropriate environment). If this is not possible, a
1309 string of the form ``<...some useful description...>`` should be returned.
1310 The return value must be a string object. If a class defines :meth:`__repr__`
1311 but not :meth:`__str__`, then :meth:`__repr__` is also used when an
1312 "informal" string representation of instances of that class is required.
Georg Brandl116aa622007-08-15 14:28:22 +00001313
Georg Brandl116aa622007-08-15 14:28:22 +00001314 This is typically used for debugging, so it is important that the representation
1315 is information-rich and unambiguous.
1316
Chris Jerdonek5fae0e52012-11-20 17:45:51 -08001317 .. index::
1318 single: string; __str__() (object method)
1319 single: format() (built-in function); __str__() (object method)
1320 single: print() (built-in function); __str__() (object method)
1321
Georg Brandl116aa622007-08-15 14:28:22 +00001322
1323.. method:: object.__str__(self)
1324
Chris Jerdonek5fae0e52012-11-20 17:45:51 -08001325 Called by :func:`str(object) <str>` and the built-in functions
1326 :func:`format` and :func:`print` to compute the "informal" or nicely
1327 printable string representation of an object. The return value must be a
1328 :ref:`string <textseq>` object.
Georg Brandl116aa622007-08-15 14:28:22 +00001329
Chris Jerdonek5fae0e52012-11-20 17:45:51 -08001330 This method differs from :meth:`object.__repr__` in that there is no
1331 expectation that :meth:`__str__` return a valid Python expression: a more
1332 convenient or concise representation can be used.
1333
1334 The default implementation defined by the built-in type :class:`object`
1335 calls :meth:`object.__repr__`.
Georg Brandl116aa622007-08-15 14:28:22 +00001336
Georg Brandldcc56f82007-08-31 16:41:12 +00001337 .. XXX what about subclasses of string?
1338
Georg Brandl116aa622007-08-15 14:28:22 +00001339
Benjamin Peterson1fafc1a2011-10-25 00:03:51 -04001340.. method:: object.__bytes__(self)
1341
1342 .. index:: builtin: bytes
1343
csabellac6db4812017-04-26 01:47:01 -04001344 Called by :ref:`bytes <func-bytes>` to compute a byte-string representation
1345 of an object. This should return a :class:`bytes` object.
Benjamin Peterson1fafc1a2011-10-25 00:03:51 -04001346
Chris Jerdonekbb4e9412012-11-28 01:38:40 -08001347 .. index::
1348 single: string; __format__() (object method)
1349 pair: string; conversion
1350 builtin: print
1351
Benjamin Peterson1fafc1a2011-10-25 00:03:51 -04001352
Georg Brandl4b491312007-08-31 09:22:56 +00001353.. method:: object.__format__(self, format_spec)
1354
Martin Panterbc1ee462016-02-13 00:41:37 +00001355 Called by the :func:`format` built-in function,
1356 and by extension, evaluation of :ref:`formatted string literals
1357 <f-strings>` and the :meth:`str.format` method, to produce a "formatted"
Catherine Alvarado5e98f052019-05-04 17:54:35 -04001358 string representation of an object. The *format_spec* argument is
Georg Brandl4b491312007-08-31 09:22:56 +00001359 a string that contains a description of the formatting options desired.
Catherine Alvarado5e98f052019-05-04 17:54:35 -04001360 The interpretation of the *format_spec* argument is up to the type
Georg Brandl4b491312007-08-31 09:22:56 +00001361 implementing :meth:`__format__`, however most classes will either
1362 delegate formatting to one of the built-in types, or use a similar
1363 formatting option syntax.
Georg Brandl48310cd2009-01-03 21:18:54 +00001364
Georg Brandl4b491312007-08-31 09:22:56 +00001365 See :ref:`formatspec` for a description of the standard formatting syntax.
1366
1367 The return value must be a string object.
1368
R David Murrayd630e792014-02-11 18:34:22 -05001369 .. versionchanged:: 3.4
1370 The __format__ method of ``object`` itself raises a :exc:`TypeError`
1371 if passed any non-empty string.
1372
Serhiy Storchaka7e19dbc2017-05-13 12:40:52 +03001373 .. versionchanged:: 3.7
1374 ``object.__format__(x, '')`` is now equivalent to ``str(x)`` rather
Heshy Roskesef337122020-04-25 21:57:09 -04001375 than ``format(str(x), '')``.
Serhiy Storchaka7e19dbc2017-05-13 12:40:52 +03001376
Georg Brandl4b491312007-08-31 09:22:56 +00001377
Georg Brandl33413cb2009-03-31 19:06:37 +00001378.. _richcmpfuncs:
Georg Brandl116aa622007-08-15 14:28:22 +00001379.. method:: object.__lt__(self, other)
1380 object.__le__(self, other)
1381 object.__eq__(self, other)
1382 object.__ne__(self, other)
1383 object.__gt__(self, other)
1384 object.__ge__(self, other)
1385
Guido van Rossum2cc30da2007-11-02 23:46:40 +00001386 .. index::
1387 single: comparisons
1388
Georg Brandl05f5ab72008-09-24 09:11:47 +00001389 These are the so-called "rich comparison" methods. The correspondence between
Georg Brandl116aa622007-08-15 14:28:22 +00001390 operator symbols and method names is as follows: ``x<y`` calls ``x.__lt__(y)``,
1391 ``x<=y`` calls ``x.__le__(y)``, ``x==y`` calls ``x.__eq__(y)``, ``x!=y`` calls
1392 ``x.__ne__(y)``, ``x>y`` calls ``x.__gt__(y)``, and ``x>=y`` calls
1393 ``x.__ge__(y)``.
1394
1395 A rich comparison method may return the singleton ``NotImplemented`` if it does
1396 not implement the operation for a given pair of arguments. By convention,
1397 ``False`` and ``True`` are returned for a successful comparison. However, these
1398 methods can return any value, so if the comparison operator is used in a Boolean
1399 context (e.g., in the condition of an ``if`` statement), Python will call
1400 :func:`bool` on the value to determine if the result is true or false.
1401
Brett Cannon3c69f0c2020-10-21 16:24:38 -07001402 By default, ``object`` implements :meth:`__eq__` by using ``is``, returning
1403 ``NotImplemented`` in the case of a false comparison:
1404 ``True if x is y else NotImplemented``. For :meth:`__ne__`, by default it
1405 delegates to :meth:`__eq__` and inverts the result unless it is
1406 ``NotImplemented``. There are no other implied relationships among the
1407 comparison operators or default implementations; for example, the truth of
1408 ``(x<y or x==y)`` does not imply ``x<=y``. To automatically generate ordering
1409 operations from a single root operation, see :func:`functools.total_ordering`.
Robert Collinsd84b29f2015-08-07 10:22:54 +12001410
1411 See the paragraph on :meth:`__hash__` for
Guido van Rossum2cc30da2007-11-02 23:46:40 +00001412 some important notes on creating :term:`hashable` objects which support
1413 custom comparison operations and are usable as dictionary keys.
Georg Brandl116aa622007-08-15 14:28:22 +00001414
Guido van Rossum2cc30da2007-11-02 23:46:40 +00001415 There are no swapped-argument versions of these methods (to be used when the
1416 left argument does not support the operation but the right argument does);
1417 rather, :meth:`__lt__` and :meth:`__gt__` are each other's reflection,
Georg Brandl116aa622007-08-15 14:28:22 +00001418 :meth:`__le__` and :meth:`__ge__` are each other's reflection, and
1419 :meth:`__eq__` and :meth:`__ne__` are their own reflection.
Robert Collinsd84b29f2015-08-07 10:22:54 +12001420 If the operands are of different types, and right operand's type is
1421 a direct or indirect subclass of the left operand's type,
1422 the reflected method of the right operand has priority, otherwise
1423 the left operand's method has priority. Virtual subclassing is
1424 not considered.
Georg Brandl116aa622007-08-15 14:28:22 +00001425
Georg Brandl116aa622007-08-15 14:28:22 +00001426.. method:: object.__hash__(self)
1427
1428 .. index::
1429 object: dictionary
1430 builtin: hash
1431
Benjamin Peterson6cadba72008-11-19 22:38:29 +00001432 Called by built-in function :func:`hash` and for operations on members of
1433 hashed collections including :class:`set`, :class:`frozenset`, and
Victor Stinner509476b2016-12-19 13:09:28 +01001434 :class:`dict`. :meth:`__hash__` should return an integer. The only required
1435 property is that objects which compare equal have the same hash value; it is
1436 advised to mix together the hash values of the components of the object that
1437 also play a part in comparison of objects by packing them into a tuple and
1438 hashing the tuple. Example::
1439
1440 def __hash__(self):
1441 return hash((self.name, self.nick, self.color))
Barry Warsaw224a5992013-07-15 14:47:29 -04001442
1443 .. note::
1444
1445 :func:`hash` truncates the value returned from an object's custom
1446 :meth:`__hash__` method to the size of a :c:type:`Py_ssize_t`. This is
1447 typically 8 bytes on 64-bit builds and 4 bytes on 32-bit builds. If an
1448 object's :meth:`__hash__` must interoperate on builds of different bit
1449 sizes, be sure to check the width on all supported builds. An easy way
1450 to do this is with
Martin Panterd21e0b52015-10-10 10:36:22 +00001451 ``python -c "import sys; print(sys.hash_info.width)"``.
Georg Brandl116aa622007-08-15 14:28:22 +00001452
Georg Brandl05f5ab72008-09-24 09:11:47 +00001453 If a class does not define an :meth:`__eq__` method it should not define a
1454 :meth:`__hash__` operation either; if it defines :meth:`__eq__` but not
Benjamin Peterson6cadba72008-11-19 22:38:29 +00001455 :meth:`__hash__`, its instances will not be usable as items in hashable
1456 collections. If a class defines mutable objects and implements an
1457 :meth:`__eq__` method, it should not implement :meth:`__hash__`, since the
1458 implementation of hashable collections requires that a key's hash value is
1459 immutable (if the object's hash value changes, it will be in the wrong hash
1460 bucket).
1461
Georg Brandl05f5ab72008-09-24 09:11:47 +00001462 User-defined classes have :meth:`__eq__` and :meth:`__hash__` methods
Nick Coghlan73c96db2008-08-31 13:21:24 +00001463 by default; with them, all objects compare unequal (except with themselves)
Nick Coghlan337b2bf2012-05-20 18:30:49 +10001464 and ``x.__hash__()`` returns an appropriate value such that ``x == y``
1465 implies both that ``x is y`` and ``hash(x) == hash(y)``.
1466
R David Murrayd8bbde32012-09-11 13:01:43 -04001467 A class that overrides :meth:`__eq__` and does not define :meth:`__hash__`
1468 will have its :meth:`__hash__` implicitly set to ``None``. When the
1469 :meth:`__hash__` method of a class is ``None``, instances of the class will
1470 raise an appropriate :exc:`TypeError` when a program attempts to retrieve
1471 their hash value, and will also be correctly identified as unhashable when
Serhiy Storchaka2e576f52017-04-24 09:05:00 +03001472 checking ``isinstance(obj, collections.abc.Hashable)``.
Nick Coghlan73c96db2008-08-31 13:21:24 +00001473
Georg Brandlae2dbe22009-03-13 19:04:40 +00001474 If a class that overrides :meth:`__eq__` needs to retain the implementation
Georg Brandl05f5ab72008-09-24 09:11:47 +00001475 of :meth:`__hash__` from a parent class, the interpreter must be told this
R David Murrayd8bbde32012-09-11 13:01:43 -04001476 explicitly by setting ``__hash__ = <ParentClass>.__hash__``.
1477
1478 If a class that does not override :meth:`__eq__` wishes to suppress hash
1479 support, it should include ``__hash__ = None`` in the class definition.
1480 A class which defines its own :meth:`__hash__` that explicitly raises
1481 a :exc:`TypeError` would be incorrectly identified as hashable by
Serhiy Storchaka2e576f52017-04-24 09:05:00 +03001482 an ``isinstance(obj, collections.abc.Hashable)`` call.
Georg Brandl05f5ab72008-09-24 09:11:47 +00001483
Benjamin Petersonc9f54cf2012-02-21 16:08:05 -05001484
1485 .. note::
1486
Serhiy Storchakae9c90aa2019-08-24 12:49:27 +03001487 By default, the :meth:`__hash__` values of str and bytes objects are
1488 "salted" with an unpredictable random value. Although they
Benjamin Petersonc9f54cf2012-02-21 16:08:05 -05001489 remain constant within an individual Python process, they are not
1490 predictable between repeated invocations of Python.
1491
1492 This is intended to provide protection against a denial-of-service caused
1493 by carefully-chosen inputs that exploit the worst case performance of a
1494 dict insertion, O(n^2) complexity. See
1495 http://www.ocert.org/advisories/ocert-2011-003.html for details.
1496
wim glenna48e0eb2018-11-16 05:58:19 -06001497 Changing hash values affects the iteration order of sets.
1498 Python has never made guarantees about this ordering
Antoine Pitrouc86e8d92012-08-01 14:53:22 +02001499 (and it typically varies between 32-bit and 64-bit builds).
Benjamin Petersonc9f54cf2012-02-21 16:08:05 -05001500
1501 See also :envvar:`PYTHONHASHSEED`.
1502
1503 .. versionchanged:: 3.3
1504 Hash randomization is enabled by default.
Georg Brandl2daf6ae2012-02-20 19:54:16 +01001505
Georg Brandl116aa622007-08-15 14:28:22 +00001506
1507.. method:: object.__bool__(self)
Georg Brandl1aeaadd2008-09-06 17:42:52 +00001508
Georg Brandl116aa622007-08-15 14:28:22 +00001509 .. index:: single: __len__() (mapping object method)
1510
Benjamin Petersonf07d0022009-03-21 17:31:58 +00001511 Called to implement truth value testing and the built-in operation
Amaury Forgeot d'Arc097cd072009-07-07 00:43:08 +00001512 ``bool()``; should return ``False`` or ``True``. When this method is not
1513 defined, :meth:`__len__` is called, if it is defined, and the object is
1514 considered true if its result is nonzero. If a class defines neither
1515 :meth:`__len__` nor :meth:`__bool__`, all its instances are considered
1516 true.
Georg Brandl116aa622007-08-15 14:28:22 +00001517
1518
Georg Brandl116aa622007-08-15 14:28:22 +00001519.. _attribute-access:
1520
1521Customizing attribute access
1522----------------------------
1523
1524The following methods can be defined to customize the meaning of attribute
1525access (use of, assignment to, or deletion of ``x.name``) for class instances.
1526
Georg Brandl85eb8c12007-08-31 16:33:38 +00001527.. XXX explain how descriptors interfere here!
1528
Georg Brandl116aa622007-08-15 14:28:22 +00001529
1530.. method:: object.__getattr__(self, name)
1531
Cheryl Sabellad1f31812018-02-04 21:03:22 -05001532 Called when the default attribute access fails with an :exc:`AttributeError`
1533 (either :meth:`__getattribute__` raises an :exc:`AttributeError` because
1534 *name* is not an instance attribute or an attribute in the class tree
1535 for ``self``; or :meth:`__get__` of a *name* property raises
1536 :exc:`AttributeError`). This method should either return the (computed)
1537 attribute value or raise an :exc:`AttributeError` exception.
Georg Brandl116aa622007-08-15 14:28:22 +00001538
Georg Brandl116aa622007-08-15 14:28:22 +00001539 Note that if the attribute is found through the normal mechanism,
1540 :meth:`__getattr__` is not called. (This is an intentional asymmetry between
1541 :meth:`__getattr__` and :meth:`__setattr__`.) This is done both for efficiency
Nick Coghlan3a5d7e32008-08-31 12:40:14 +00001542 reasons and because otherwise :meth:`__getattr__` would have no way to access
Georg Brandl116aa622007-08-15 14:28:22 +00001543 other attributes of the instance. Note that at least for instance variables,
1544 you can fake total control by not inserting any values in the instance attribute
1545 dictionary (but instead inserting them in another object). See the
Georg Brandl85eb8c12007-08-31 16:33:38 +00001546 :meth:`__getattribute__` method below for a way to actually get total control
1547 over attribute access.
Georg Brandl116aa622007-08-15 14:28:22 +00001548
1549
1550.. method:: object.__getattribute__(self, name)
1551
1552 Called unconditionally to implement attribute accesses for instances of the
1553 class. If the class also defines :meth:`__getattr__`, the latter will not be
1554 called unless :meth:`__getattribute__` either calls it explicitly or raises an
1555 :exc:`AttributeError`. This method should return the (computed) attribute value
1556 or raise an :exc:`AttributeError` exception. In order to avoid infinite
1557 recursion in this method, its implementation should always call the base class
1558 method with the same name to access any attributes it needs, for example,
1559 ``object.__getattribute__(self, name)``.
1560
Nick Coghlan3a5d7e32008-08-31 12:40:14 +00001561 .. note::
1562
1563 This method may still be bypassed when looking up special methods as the
Georg Brandl22b34312009-07-26 14:54:51 +00001564 result of implicit invocation via language syntax or built-in functions.
Nick Coghlan3a5d7e32008-08-31 12:40:14 +00001565 See :ref:`special-lookup`.
1566
Saiyang Gou3f7e9902020-10-20 12:23:15 -07001567 .. audit-event:: object.__getattr__ obj,name object.__getattribute__
1568
1569 For certain sensitive attribute accesses, raises an
1570 :ref:`auditing event <auditing>` ``object.__getattr__`` with arguments
1571 ``obj`` and ``name``.
1572
Georg Brandl116aa622007-08-15 14:28:22 +00001573
Georg Brandl85eb8c12007-08-31 16:33:38 +00001574.. method:: object.__setattr__(self, name, value)
1575
1576 Called when an attribute assignment is attempted. This is called instead of
1577 the normal mechanism (i.e. store the value in the instance dictionary).
1578 *name* is the attribute name, *value* is the value to be assigned to it.
1579
1580 If :meth:`__setattr__` wants to assign to an instance attribute, it should
1581 call the base class method with the same name, for example,
1582 ``object.__setattr__(self, name, value)``.
1583
Saiyang Gou3f7e9902020-10-20 12:23:15 -07001584 .. audit-event:: object.__setattr__ obj,name,value object.__setattr__
1585
1586 For certain sensitive attribute assignments, raises an
1587 :ref:`auditing event <auditing>` ``object.__setattr__`` with arguments
1588 ``obj``, ``name``, ``value``.
1589
Georg Brandl85eb8c12007-08-31 16:33:38 +00001590
1591.. method:: object.__delattr__(self, name)
1592
1593 Like :meth:`__setattr__` but for attribute deletion instead of assignment. This
1594 should only be implemented if ``del obj.name`` is meaningful for the object.
1595
Saiyang Gou3f7e9902020-10-20 12:23:15 -07001596 .. audit-event:: object.__delattr__ obj,name object.__delattr__
1597
1598 For certain sensitive attribute deletions, raises an
1599 :ref:`auditing event <auditing>` ``object.__delattr__`` with arguments
1600 ``obj`` and ``name``.
1601
Georg Brandl85eb8c12007-08-31 16:33:38 +00001602
Benjamin Peterson1cef37c2008-07-02 14:44:54 +00001603.. method:: object.__dir__(self)
1604
Benjamin Peterson3bbb7222011-06-11 16:12:08 -05001605 Called when :func:`dir` is called on the object. A sequence must be
1606 returned. :func:`dir` converts the returned sequence to a list and sorts it.
Benjamin Peterson1cef37c2008-07-02 14:44:54 +00001607
1608
Ivan Levkivskyi5364b5c2017-12-14 11:59:44 +01001609Customizing module attribute access
1610^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1611
1612.. index::
1613 single: __getattr__ (module attribute)
1614 single: __dir__ (module attribute)
1615 single: __class__ (module attribute)
1616
1617Special names ``__getattr__`` and ``__dir__`` can be also used to customize
1618access to module attributes. The ``__getattr__`` function at the module level
1619should accept one argument which is the name of an attribute and return the
1620computed value or raise an :exc:`AttributeError`. If an attribute is
1621not found on a module object through the normal lookup, i.e.
1622:meth:`object.__getattribute__`, then ``__getattr__`` is searched in
1623the module ``__dict__`` before raising an :exc:`AttributeError`. If found,
1624it is called with the attribute name and the result is returned.
1625
alclarks4544e782019-11-17 22:00:43 +00001626The ``__dir__`` function should accept no arguments, and return a sequence of
Ivan Levkivskyi5364b5c2017-12-14 11:59:44 +01001627strings that represents the names accessible on module. If present, this
1628function overrides the standard :func:`dir` search on a module.
1629
1630For a more fine grained customization of the module behavior (setting
1631attributes, properties, etc.), one can set the ``__class__`` attribute of
1632a module object to a subclass of :class:`types.ModuleType`. For example::
1633
1634 import sys
1635 from types import ModuleType
1636
1637 class VerboseModule(ModuleType):
1638 def __repr__(self):
1639 return f'Verbose {self.__name__}'
1640
1641 def __setattr__(self, attr, value):
1642 print(f'Setting {attr}...')
Denis Osipov0bee3c32018-11-06 06:53:21 +05001643 super().__setattr__(attr, value)
Ivan Levkivskyi5364b5c2017-12-14 11:59:44 +01001644
1645 sys.modules[__name__].__class__ = VerboseModule
1646
1647.. note::
1648 Defining module ``__getattr__`` and setting module ``__class__`` only
1649 affect lookups made using the attribute access syntax -- directly accessing
1650 the module globals (whether by code within the module, or via a reference
1651 to the module's globals dictionary) is unaffected.
1652
Cheryl Sabella85527cf2018-01-26 21:40:52 -05001653.. versionchanged:: 3.5
1654 ``__class__`` module attribute is now writable.
1655
1656.. versionadded:: 3.7
1657 ``__getattr__`` and ``__dir__`` module attributes.
1658
1659.. seealso::
1660
1661 :pep:`562` - Module __getattr__ and __dir__
1662 Describes the ``__getattr__`` and ``__dir__`` functions on modules.
1663
Ivan Levkivskyi5364b5c2017-12-14 11:59:44 +01001664
Georg Brandl116aa622007-08-15 14:28:22 +00001665.. _descriptors:
1666
1667Implementing Descriptors
1668^^^^^^^^^^^^^^^^^^^^^^^^
1669
1670The following methods only apply when an instance of the class containing the
Raymond Hettinger3b654be2011-03-22 16:27:02 -07001671method (a so-called *descriptor* class) appears in an *owner* class (the
1672descriptor must be in either the owner's class dictionary or in the class
1673dictionary for one of its parents). In the examples below, "the attribute"
1674refers to the attribute whose name is the key of the property in the owner
Martin Panterbae5d812016-06-18 03:57:31 +00001675class' :attr:`~object.__dict__`.
Georg Brandl116aa622007-08-15 14:28:22 +00001676
1677
Raymond Hettinger0dac68f2019-08-29 01:27:42 -07001678.. method:: object.__get__(self, instance, owner=None)
Georg Brandl116aa622007-08-15 14:28:22 +00001679
Raymond Hettinger0dac68f2019-08-29 01:27:42 -07001680 Called to get the attribute of the owner class (class attribute access) or
1681 of an instance of that class (instance attribute access). The optional
1682 *owner* argument is the owner class, while *instance* is the instance that
1683 the attribute was accessed through, or ``None`` when the attribute is
1684 accessed through the *owner*.
Georg Brandl116aa622007-08-15 14:28:22 +00001685
Raymond Hettinger0dac68f2019-08-29 01:27:42 -07001686 This method should return the computed attribute value or raise an
1687 :exc:`AttributeError` exception.
1688
1689 :PEP:`252` specifies that :meth:`__get__` is callable with one or two
1690 arguments. Python's own built-in descriptors support this specification;
1691 however, it is likely that some third-party tools have descriptors
1692 that require both arguments. Python's own :meth:`__getattribute__`
1693 implementation always passes in both arguments whether they are required
1694 or not.
Georg Brandl116aa622007-08-15 14:28:22 +00001695
1696.. method:: object.__set__(self, instance, value)
1697
1698 Called to set the attribute on an instance *instance* of the owner class to a
1699 new value, *value*.
1700
Raymond Hettinger0dac68f2019-08-29 01:27:42 -07001701 Note, adding :meth:`__set__` or :meth:`__delete__` changes the kind of
1702 descriptor to a "data descriptor". See :ref:`descriptor-invocation` for
1703 more details.
Georg Brandl116aa622007-08-15 14:28:22 +00001704
1705.. method:: object.__delete__(self, instance)
1706
1707 Called to delete the attribute on an instance *instance* of the owner class.
1708
1709
Nick Coghland78448e2016-07-30 16:26:03 +10001710.. method:: object.__set_name__(self, owner, name)
1711
1712 Called at the time the owning class *owner* is created. The
1713 descriptor has been assigned to *name*.
1714
Florian Dahlitz1bddf892019-11-27 09:46:40 +01001715 .. note::
Berker Peksag01d17192016-07-30 14:06:15 +03001716
Tal Einat02519f72019-11-28 07:22:09 +02001717 :meth:`__set_name__` is only called implicitly as part of the
1718 :class:`type` constructor, so it will need to be called explicitly with
1719 the appropriate parameters when a descriptor is added to a class after
1720 initial creation::
Florian Dahlitz1bddf892019-11-27 09:46:40 +01001721
Tal Einat02519f72019-11-28 07:22:09 +02001722 class A:
1723 pass
Florian Dahlitz1bddf892019-11-27 09:46:40 +01001724 descr = custom_descriptor()
Tal Einat02519f72019-11-28 07:22:09 +02001725 A.attr = descr
1726 descr.__set_name__(A, 'attr')
Florian Dahlitz1bddf892019-11-27 09:46:40 +01001727
1728 See :ref:`class-object-creation` for more details.
1729
1730 .. versionadded:: 3.6
Nick Coghland78448e2016-07-30 16:26:03 +10001731
Yury Selivanov490a72e2014-04-08 14:01:12 -04001732The attribute :attr:`__objclass__` is interpreted by the :mod:`inspect` module
1733as specifying the class where this object was defined (setting this
1734appropriately can assist in runtime introspection of dynamic class attributes).
1735For callables, it may indicate that an instance of the given type (or a
1736subclass) is expected or required as the first positional argument (for example,
1737CPython sets this attribute for unbound methods that are implemented in C).
Yury Selivanovb9aa8cb2014-04-08 12:04:04 -04001738
1739
Georg Brandl116aa622007-08-15 14:28:22 +00001740.. _descriptor-invocation:
1741
1742Invoking Descriptors
1743^^^^^^^^^^^^^^^^^^^^
1744
1745In general, a descriptor is an object attribute with "binding behavior", one
1746whose attribute access has been overridden by methods in the descriptor
1747protocol: :meth:`__get__`, :meth:`__set__`, and :meth:`__delete__`. If any of
1748those methods are defined for an object, it is said to be a descriptor.
1749
1750The default behavior for attribute access is to get, set, or delete the
1751attribute from an object's dictionary. For instance, ``a.x`` has a lookup chain
1752starting with ``a.__dict__['x']``, then ``type(a).__dict__['x']``, and
1753continuing through the base classes of ``type(a)`` excluding metaclasses.
1754
1755However, if the looked-up value is an object defining one of the descriptor
1756methods, then Python may override the default behavior and invoke the descriptor
1757method instead. Where this occurs in the precedence chain depends on which
Georg Brandl23e8db52008-04-07 19:17:06 +00001758descriptor methods were defined and how they were called.
Georg Brandl116aa622007-08-15 14:28:22 +00001759
1760The starting point for descriptor invocation is a binding, ``a.x``. How the
1761arguments are assembled depends on ``a``:
1762
1763Direct Call
1764 The simplest and least common call is when user code directly invokes a
1765 descriptor method: ``x.__get__(a)``.
1766
1767Instance Binding
Georg Brandl85eb8c12007-08-31 16:33:38 +00001768 If binding to an object instance, ``a.x`` is transformed into the call:
Georg Brandl116aa622007-08-15 14:28:22 +00001769 ``type(a).__dict__['x'].__get__(a, type(a))``.
1770
1771Class Binding
Georg Brandl85eb8c12007-08-31 16:33:38 +00001772 If binding to a class, ``A.x`` is transformed into the call:
Georg Brandl116aa622007-08-15 14:28:22 +00001773 ``A.__dict__['x'].__get__(None, A)``.
1774
1775Super Binding
csabella12b1c182017-05-14 20:42:00 -07001776 If ``a`` is an instance of :class:`super`, then the binding ``super(B, obj).m()``
1777 searches ``obj.__class__.__mro__`` for the base class ``A``
Georg Brandl116aa622007-08-15 14:28:22 +00001778 immediately preceding ``B`` and then invokes the descriptor with the call:
Raymond Hettingerb199b222011-03-22 15:28:45 -07001779 ``A.__dict__['m'].__get__(obj, obj.__class__)``.
Georg Brandl116aa622007-08-15 14:28:22 +00001780
Géry Ogama1a5e3c2021-04-25 23:13:19 +02001781For instance bindings, the precedence of descriptor invocation depends on
Benjamin Peterson5e55b3e2010-02-03 02:35:45 +00001782which descriptor methods are defined. A descriptor can define any combination
1783of :meth:`__get__`, :meth:`__set__` and :meth:`__delete__`. If it does not
1784define :meth:`__get__`, then accessing the attribute will return the descriptor
1785object itself unless there is a value in the object's instance dictionary. If
1786the descriptor defines :meth:`__set__` and/or :meth:`__delete__`, it is a data
1787descriptor; if it defines neither, it is a non-data descriptor. Normally, data
1788descriptors define both :meth:`__get__` and :meth:`__set__`, while non-data
1789descriptors have just the :meth:`__get__` method. Data descriptors with
Géry Ogam4c155f72019-10-29 08:04:01 +01001790:meth:`__get__` and :meth:`__set__` (and/or :meth:`__delete__`) defined always override a redefinition in an
Georg Brandl116aa622007-08-15 14:28:22 +00001791instance dictionary. In contrast, non-data descriptors can be overridden by
Benjamin Peterson5e55b3e2010-02-03 02:35:45 +00001792instances.
Georg Brandl116aa622007-08-15 14:28:22 +00001793
1794Python methods (including :func:`staticmethod` and :func:`classmethod`) are
1795implemented as non-data descriptors. Accordingly, instances can redefine and
1796override methods. This allows individual instances to acquire behaviors that
1797differ from other instances of the same class.
1798
1799The :func:`property` function is implemented as a data descriptor. Accordingly,
1800instances cannot override the behavior of a property.
1801
1802
1803.. _slots:
1804
1805__slots__
1806^^^^^^^^^
1807
Aaron Hall, MBA2b44e302017-05-25 22:33:26 -07001808*__slots__* allow us to explicitly declare data members (like
1809properties) and deny the creation of *__dict__* and *__weakref__*
1810(unless explicitly declared in *__slots__* or available in a parent.)
Georg Brandl116aa622007-08-15 14:28:22 +00001811
Aaron Hall, MBA2b44e302017-05-25 22:33:26 -07001812The space saved over using *__dict__* can be significant.
Raymond Hettinger74638842019-02-21 03:34:04 -08001813Attribute lookup speed can be significantly improved as well.
Georg Brandl116aa622007-08-15 14:28:22 +00001814
Georg Brandl85eb8c12007-08-31 16:33:38 +00001815.. data:: object.__slots__
Georg Brandl116aa622007-08-15 14:28:22 +00001816
Georg Brandl85eb8c12007-08-31 16:33:38 +00001817 This class variable can be assigned a string, iterable, or sequence of
Georg Brandla4c8c472014-10-31 10:38:49 +01001818 strings with variable names used by instances. *__slots__* reserves space
1819 for the declared variables and prevents the automatic creation of *__dict__*
1820 and *__weakref__* for each instance.
Georg Brandl116aa622007-08-15 14:28:22 +00001821
Georg Brandl116aa622007-08-15 14:28:22 +00001822
1823Notes on using *__slots__*
Georg Brandl16174572007-09-01 12:38:06 +00001824""""""""""""""""""""""""""
Georg Brandl116aa622007-08-15 14:28:22 +00001825
Aaron Hall, MBA2b44e302017-05-25 22:33:26 -07001826* When inheriting from a class without *__slots__*, the *__dict__* and
1827 *__weakref__* attribute of the instances will always be accessible.
Georg Brandl3dbca812008-07-23 16:10:53 +00001828
Georg Brandl116aa622007-08-15 14:28:22 +00001829* Without a *__dict__* variable, instances cannot be assigned new variables not
1830 listed in the *__slots__* definition. Attempts to assign to an unlisted
1831 variable name raises :exc:`AttributeError`. If dynamic assignment of new
Georg Brandl85eb8c12007-08-31 16:33:38 +00001832 variables is desired, then add ``'__dict__'`` to the sequence of strings in
1833 the *__slots__* declaration.
Georg Brandl116aa622007-08-15 14:28:22 +00001834
Georg Brandl116aa622007-08-15 14:28:22 +00001835* Without a *__weakref__* variable for each instance, classes defining
1836 *__slots__* do not support weak references to its instances. If weak reference
1837 support is needed, then add ``'__weakref__'`` to the sequence of strings in the
1838 *__slots__* declaration.
1839
Georg Brandl116aa622007-08-15 14:28:22 +00001840* *__slots__* are implemented at the class level by creating descriptors
1841 (:ref:`descriptors`) for each variable name. As a result, class attributes
1842 cannot be used to set default values for instance variables defined by
1843 *__slots__*; otherwise, the class attribute would overwrite the descriptor
1844 assignment.
1845
Aaron Hall, MBA2b44e302017-05-25 22:33:26 -07001846* The action of a *__slots__* declaration is not limited to the class
1847 where it is defined. *__slots__* declared in parents are available in
1848 child classes. However, child subclasses will get a *__dict__* and
1849 *__weakref__* unless they also define *__slots__* (which should only
1850 contain names of any *additional* slots).
Georg Brandl495f7b52009-10-27 15:28:25 +00001851
Georg Brandl116aa622007-08-15 14:28:22 +00001852* If a class defines a slot also defined in a base class, the instance variable
1853 defined by the base class slot is inaccessible (except by retrieving its
1854 descriptor directly from the base class). This renders the meaning of the
1855 program undefined. In the future, a check may be added to prevent this.
1856
Benjamin Peterson1a6e0d02008-10-25 15:49:17 +00001857* Nonempty *__slots__* does not work for classes derived from "variable-length"
Zachary Ware340a6922013-12-31 12:09:26 -06001858 built-in types such as :class:`int`, :class:`bytes` and :class:`tuple`.
Georg Brandl116aa622007-08-15 14:28:22 +00001859
1860* Any non-string iterable may be assigned to *__slots__*. Mappings may also be
1861 used; however, in the future, special meaning may be assigned to the values
1862 corresponding to each key.
1863
1864* *__class__* assignment works only if both classes have the same *__slots__*.
1865
Aaron Hall, MBA2b44e302017-05-25 22:33:26 -07001866* Multiple inheritance with multiple slotted parent classes can be used,
1867 but only one parent is allowed to have attributes created by slots
1868 (the other bases must have empty slot layouts) - violations raise
1869 :exc:`TypeError`.
Georg Brandl116aa622007-08-15 14:28:22 +00001870
alclarks4544e782019-11-17 22:00:43 +00001871* If an iterator is used for *__slots__* then a descriptor is created for each
1872 of the iterator's values. However, the *__slots__* attribute will be an empty
1873 iterator.
1874
Nick Coghland78448e2016-07-30 16:26:03 +10001875.. _class-customization:
Georg Brandl116aa622007-08-15 14:28:22 +00001876
1877Customizing class creation
1878--------------------------
1879
Nick Coghland78448e2016-07-30 16:26:03 +10001880Whenever a class inherits from another class, *__init_subclass__* is
1881called on that class. This way, it is possible to write classes which
1882change the behavior of subclasses. This is closely related to class
1883decorators, but where class decorators only affect the specific class they're
1884applied to, ``__init_subclass__`` solely applies to future subclasses of the
1885class defining the method.
1886
1887.. classmethod:: object.__init_subclass__(cls)
Berker Peksag01d17192016-07-30 14:06:15 +03001888
Nick Coghland78448e2016-07-30 16:26:03 +10001889 This method is called whenever the containing class is subclassed.
1890 *cls* is then the new subclass. If defined as a normal instance method,
1891 this method is implicitly converted to a class method.
1892
1893 Keyword arguments which are given to a new class are passed to
1894 the parent's class ``__init_subclass__``. For compatibility with
1895 other classes using ``__init_subclass__``, one should take out the
1896 needed keyword arguments and pass the others over to the base
1897 class, as in::
1898
1899 class Philosopher:
Serhiy Storchaka70c5f2a2019-06-01 11:38:24 +03001900 def __init_subclass__(cls, /, default_name, **kwargs):
Nick Coghland78448e2016-07-30 16:26:03 +10001901 super().__init_subclass__(**kwargs)
1902 cls.default_name = default_name
1903
1904 class AustralianPhilosopher(Philosopher, default_name="Bruce"):
1905 pass
1906
1907 The default implementation ``object.__init_subclass__`` does
1908 nothing, but raises an error if it is called with any arguments.
1909
Nick Coghlan607e1c42016-07-31 12:42:49 +10001910 .. note::
1911
1912 The metaclass hint ``metaclass`` is consumed by the rest of the type
1913 machinery, and is never passed to ``__init_subclass__`` implementations.
1914 The actual metaclass (rather than the explicit hint) can be accessed as
1915 ``type(cls)``.
1916
Berker Peksag01d17192016-07-30 14:06:15 +03001917 .. versionadded:: 3.6
1918
Nick Coghland78448e2016-07-30 16:26:03 +10001919
1920.. _metaclasses:
1921
1922Metaclasses
1923^^^^^^^^^^^
1924
Nick Coghlan19d24672016-12-05 16:47:55 +10001925.. index::
Serhiy Storchakaddb961d2018-10-26 09:00:49 +03001926 single: metaclass
1927 builtin: type
Serhiy Storchaka913876d2018-10-28 13:41:26 +02001928 single: = (equals); class definition
Nick Coghlan19d24672016-12-05 16:47:55 +10001929
Nick Coghlan7fc570a2012-05-20 02:34:13 +10001930By default, classes are constructed using :func:`type`. The class body is
1931executed in a new namespace and the class name is bound locally to the
1932result of ``type(name, bases, namespace)``.
Georg Brandl116aa622007-08-15 14:28:22 +00001933
Raymond Hettinger7ea386e2016-08-25 21:11:50 -07001934The class creation process can be customized by passing the ``metaclass``
Nick Coghlan7fc570a2012-05-20 02:34:13 +10001935keyword argument in the class definition line, or by inheriting from an
1936existing class that included such an argument. In the following example,
1937both ``MyClass`` and ``MySubclass`` are instances of ``Meta``::
Georg Brandl116aa622007-08-15 14:28:22 +00001938
Nick Coghlan7fc570a2012-05-20 02:34:13 +10001939 class Meta(type):
1940 pass
Georg Brandl116aa622007-08-15 14:28:22 +00001941
Nick Coghlan7fc570a2012-05-20 02:34:13 +10001942 class MyClass(metaclass=Meta):
1943 pass
Georg Brandl116aa622007-08-15 14:28:22 +00001944
Nick Coghlan7fc570a2012-05-20 02:34:13 +10001945 class MySubclass(MyClass):
1946 pass
Christian Heimes790c8232008-01-07 21:14:23 +00001947
Nick Coghlan7fc570a2012-05-20 02:34:13 +10001948Any other keyword arguments that are specified in the class definition are
1949passed through to all metaclass operations described below.
Christian Heimes790c8232008-01-07 21:14:23 +00001950
Nick Coghlan7fc570a2012-05-20 02:34:13 +10001951When a class definition is executed, the following steps occur:
Christian Heimes790c8232008-01-07 21:14:23 +00001952
Jules Lasne (jlasne)1fc5bf22019-03-27 11:10:33 +01001953* MRO entries are resolved;
1954* the appropriate metaclass is determined;
1955* the class namespace is prepared;
1956* the class body is executed;
1957* the class object is created.
Georg Brandl116aa622007-08-15 14:28:22 +00001958
Ivan Levkivskyibd5f9652018-05-08 19:38:41 +01001959
1960Resolving MRO entries
1961^^^^^^^^^^^^^^^^^^^^^
1962
1963If a base that appears in class definition is not an instance of :class:`type`,
1964then an ``__mro_entries__`` method is searched on it. If found, it is called
1965with the original bases tuple. This method must return a tuple of classes that
1966will be used instead of this base. The tuple may be empty, in such case
1967the original base is ignored.
1968
1969.. seealso::
1970
1971 :pep:`560` - Core support for typing module and generic types
1972
1973
Nick Coghlan7fc570a2012-05-20 02:34:13 +10001974Determining the appropriate metaclass
1975^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Nick Coghlan19d24672016-12-05 16:47:55 +10001976.. index::
1977 single: metaclass hint
Georg Brandl116aa622007-08-15 14:28:22 +00001978
Nick Coghlan7fc570a2012-05-20 02:34:13 +10001979The appropriate metaclass for a class definition is determined as follows:
Georg Brandl116aa622007-08-15 14:28:22 +00001980
Jules Lasne (jlasne)1fc5bf22019-03-27 11:10:33 +01001981* if no bases and no explicit metaclass are given, then :func:`type` is used;
Nick Coghlan7fc570a2012-05-20 02:34:13 +10001982* if an explicit metaclass is given and it is *not* an instance of
Jules Lasne (jlasne)1fc5bf22019-03-27 11:10:33 +01001983 :func:`type`, then it is used directly as the metaclass;
Nick Coghlan7fc570a2012-05-20 02:34:13 +10001984* if an instance of :func:`type` is given as the explicit metaclass, or
Jules Lasne (jlasne)1fc5bf22019-03-27 11:10:33 +01001985 bases are defined, then the most derived metaclass is used.
Georg Brandl116aa622007-08-15 14:28:22 +00001986
Nick Coghlan7fc570a2012-05-20 02:34:13 +10001987The most derived metaclass is selected from the explicitly specified
1988metaclass (if any) and the metaclasses (i.e. ``type(cls)``) of all specified
1989base classes. The most derived metaclass is one which is a subtype of *all*
1990of these candidate metaclasses. If none of the candidate metaclasses meets
1991that criterion, then the class definition will fail with ``TypeError``.
1992
1993
R David Murrayaf7d2c42014-02-12 13:00:36 -05001994.. _prepare:
1995
Nick Coghlan7fc570a2012-05-20 02:34:13 +10001996Preparing the class namespace
1997^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1998
Nick Coghlan19d24672016-12-05 16:47:55 +10001999.. index::
2000 single: __prepare__ (metaclass method)
2001
Nick Coghlan7fc570a2012-05-20 02:34:13 +10002002Once the appropriate metaclass has been identified, then the class namespace
2003is prepared. If the metaclass has a ``__prepare__`` attribute, it is called
2004as ``namespace = metaclass.__prepare__(name, bases, **kwds)`` (where the
alclarks7de61742020-01-25 18:49:58 +00002005additional keyword arguments, if any, come from the class definition). The
ananthan-123fbe2e0b2020-02-22 23:26:02 +05302006``__prepare__`` method should be implemented as a :func:`classmethod`. The
2007namespace returned by ``__prepare__`` is passed in to ``__new__``, but when
2008the final class object is created the namespace is copied into a new ``dict``.
Nick Coghlan7fc570a2012-05-20 02:34:13 +10002009
2010If the metaclass has no ``__prepare__`` attribute, then the class namespace
Caleb Donovicke59334e2020-03-06 10:20:48 -08002011is initialised as an empty ordered mapping.
Eric Snow92a6c172016-09-05 14:50:11 -07002012
Nick Coghlan7fc570a2012-05-20 02:34:13 +10002013.. seealso::
2014
2015 :pep:`3115` - Metaclasses in Python 3000
2016 Introduced the ``__prepare__`` namespace hook
2017
2018
2019Executing the class body
2020^^^^^^^^^^^^^^^^^^^^^^^^
2021
Nick Coghlan19d24672016-12-05 16:47:55 +10002022.. index::
2023 single: class; body
2024
Nick Coghlan7fc570a2012-05-20 02:34:13 +10002025The class body is executed (approximately) as
2026``exec(body, globals(), namespace)``. The key difference from a normal
2027call to :func:`exec` is that lexical scoping allows the class body (including
2028any methods) to reference names from the current and outer scopes when the
2029class definition occurs inside a function.
2030
2031However, even when the class definition occurs inside the function, methods
2032defined inside the class still cannot see names defined at the class scope.
2033Class variables must be accessed through the first parameter of instance or
Nick Coghlan19d24672016-12-05 16:47:55 +10002034class methods, or through the implicit lexically scoped ``__class__`` reference
2035described in the next section.
Nick Coghlan7fc570a2012-05-20 02:34:13 +10002036
Nick Coghlan19d24672016-12-05 16:47:55 +10002037.. _class-object-creation:
Nick Coghlan7fc570a2012-05-20 02:34:13 +10002038
2039Creating the class object
2040^^^^^^^^^^^^^^^^^^^^^^^^^
2041
Nick Coghlan19d24672016-12-05 16:47:55 +10002042.. index::
2043 single: __class__ (method cell)
2044 single: __classcell__ (class namespace entry)
2045
2046
Nick Coghlan7fc570a2012-05-20 02:34:13 +10002047Once the class namespace has been populated by executing the class body,
2048the class object is created by calling
2049``metaclass(name, bases, namespace, **kwds)`` (the additional keywords
Nick Coghlan78770f02012-05-20 18:15:11 +10002050passed here are the same as those passed to ``__prepare__``).
Nick Coghlan7fc570a2012-05-20 02:34:13 +10002051
2052This class object is the one that will be referenced by the zero-argument
2053form of :func:`super`. ``__class__`` is an implicit closure reference
2054created by the compiler if any methods in a class body refer to either
2055``__class__`` or ``super``. This allows the zero argument form of
2056:func:`super` to correctly identify the class being defined based on
2057lexical scoping, while the class or instance that was used to make the
2058current call is identified based on the first argument passed to the method.
2059
Nick Coghlan19d24672016-12-05 16:47:55 +10002060.. impl-detail::
2061
2062 In CPython 3.6 and later, the ``__class__`` cell is passed to the metaclass
2063 as a ``__classcell__`` entry in the class namespace. If present, this must
2064 be propagated up to the ``type.__new__`` call in order for the class to be
2065 initialised correctly.
Serhiy Storchakaf5e7b192018-05-20 08:48:12 +03002066 Failing to do so will result in a :exc:`RuntimeError` in Python 3.8.
Nick Coghlan19d24672016-12-05 16:47:55 +10002067
2068When using the default metaclass :class:`type`, or any metaclass that ultimately
2069calls ``type.__new__``, the following additional customisation steps are
2070invoked after creating the class object:
2071
2072* first, ``type.__new__`` collects all of the descriptors in the class
2073 namespace that define a :meth:`~object.__set_name__` method;
2074* second, all of these ``__set_name__`` methods are called with the class
Jules Lasne (jlasne)1fc5bf22019-03-27 11:10:33 +01002075 being defined and the assigned name of that particular descriptor;
Nick Coghlan19d24672016-12-05 16:47:55 +10002076* finally, the :meth:`~object.__init_subclass__` hook is called on the
2077 immediate parent of the new class in its method resolution order.
2078
Nick Coghlanb2674752012-05-20 19:36:40 +10002079After the class object is created, it is passed to the class decorators
2080included in the class definition (if any) and the resulting object is bound
2081in the local namespace as the defined class.
Nick Coghlan7fc570a2012-05-20 02:34:13 +10002082
R David Murraydd4fcf52016-06-02 20:05:43 -04002083When a new class is created by ``type.__new__``, the object provided as the
Eric Snow68f4dd82016-09-09 11:22:14 -07002084namespace parameter is copied to a new ordered mapping and the original
2085object is discarded. The new copy is wrapped in a read-only proxy, which
2086becomes the :attr:`~object.__dict__` attribute of the class object.
R David Murraydd4fcf52016-06-02 20:05:43 -04002087
Nick Coghlan7fc570a2012-05-20 02:34:13 +10002088.. seealso::
2089
2090 :pep:`3135` - New super
2091 Describes the implicit ``__class__`` closure reference
2092
2093
Andrés Delfinoc2ccac72018-11-16 08:41:55 -03002094Uses for metaclasses
2095^^^^^^^^^^^^^^^^^^^^
Georg Brandl116aa622007-08-15 14:28:22 +00002096
2097The potential uses for metaclasses are boundless. Some ideas that have been
Louie Lub8d1b9d2017-09-13 12:44:14 +08002098explored include enum, logging, interface checking, automatic delegation,
2099automatic property creation, proxies, frameworks, and automatic resource
Georg Brandl116aa622007-08-15 14:28:22 +00002100locking/synchronization.
2101
2102
Georg Brandl8569e582010-05-19 20:57:08 +00002103Customizing instance and subclass checks
2104----------------------------------------
2105
2106The following methods are used to override the default behavior of the
2107:func:`isinstance` and :func:`issubclass` built-in functions.
2108
2109In particular, the metaclass :class:`abc.ABCMeta` implements these methods in
2110order to allow the addition of Abstract Base Classes (ABCs) as "virtual base
Benjamin Petersond7c3ed52010-06-27 22:32:30 +00002111classes" to any class or type (including built-in types), including other
Georg Brandl8569e582010-05-19 20:57:08 +00002112ABCs.
2113
2114.. method:: class.__instancecheck__(self, instance)
2115
2116 Return true if *instance* should be considered a (direct or indirect)
2117 instance of *class*. If defined, called to implement ``isinstance(instance,
2118 class)``.
2119
2120
2121.. method:: class.__subclasscheck__(self, subclass)
2122
2123 Return true if *subclass* should be considered a (direct or indirect)
2124 subclass of *class*. If defined, called to implement ``issubclass(subclass,
2125 class)``.
2126
2127
2128Note that these methods are looked up on the type (metaclass) of a class. They
2129cannot be defined as class methods in the actual class. This is consistent with
Benjamin Petersond7c3ed52010-06-27 22:32:30 +00002130the lookup of special methods that are called on instances, only in this
Georg Brandl8569e582010-05-19 20:57:08 +00002131case the instance is itself a class.
2132
2133.. seealso::
2134
2135 :pep:`3119` - Introducing Abstract Base Classes
2136 Includes the specification for customizing :func:`isinstance` and
Serhiy Storchaka0d196ed2013-10-09 14:02:31 +03002137 :func:`issubclass` behavior through :meth:`~class.__instancecheck__` and
2138 :meth:`~class.__subclasscheck__`, with motivation for this functionality
2139 in the context of adding Abstract Base Classes (see the :mod:`abc`
2140 module) to the language.
Georg Brandl8569e582010-05-19 20:57:08 +00002141
2142
Ivan Levkivskyibd5f9652018-05-08 19:38:41 +01002143Emulating generic types
2144-----------------------
2145
2146One can implement the generic class syntax as specified by :pep:`484`
Jules Lasne (jlasne)1fc5bf22019-03-27 11:10:33 +01002147(for example ``List[int]``) by defining a special method:
Ivan Levkivskyibd5f9652018-05-08 19:38:41 +01002148
2149.. classmethod:: object.__class_getitem__(cls, key)
2150
2151 Return an object representing the specialization of a generic class
2152 by type arguments found in *key*.
2153
2154This method is looked up on the class object itself, and when defined in
2155the class body, this method is implicitly a class method. Note, this
2156mechanism is primarily reserved for use with static type hints, other usage
2157is discouraged.
2158
2159.. seealso::
2160
2161 :pep:`560` - Core support for typing module and generic types
2162
2163
Georg Brandl116aa622007-08-15 14:28:22 +00002164.. _callable-types:
2165
2166Emulating callable objects
2167--------------------------
2168
2169
2170.. method:: object.__call__(self[, args...])
2171
2172 .. index:: pair: call; instance
2173
2174 Called when the instance is "called" as a function; if this method is defined,
Andre Delfino95f710c2020-10-27 13:18:57 -03002175 ``x(arg1, arg2, ...)`` roughly translates to ``type(x).__call__(x, arg1, ...)``.
Georg Brandl116aa622007-08-15 14:28:22 +00002176
2177
2178.. _sequence-types:
2179
2180Emulating container types
2181-------------------------
2182
2183The following methods can be defined to implement container objects. Containers
2184usually are sequences (such as lists or tuples) or mappings (like dictionaries),
2185but can represent other containers as well. The first set of methods is used
2186either to emulate a sequence or to emulate a mapping; the difference is that for
2187a sequence, the allowable keys should be the integers *k* for which ``0 <= k <
2188N`` where *N* is the length of the sequence, or slice objects, which define a
Georg Brandlcb8ecb12007-09-04 06:35:14 +00002189range of items. It is also recommended that mappings provide the methods
Georg Brandlc7723722008-05-26 17:47:11 +00002190:meth:`keys`, :meth:`values`, :meth:`items`, :meth:`get`, :meth:`clear`,
Serhiy Storchaka0d196ed2013-10-09 14:02:31 +03002191:meth:`setdefault`, :meth:`pop`, :meth:`popitem`, :meth:`!copy`, and
Georg Brandlcb8ecb12007-09-04 06:35:14 +00002192:meth:`update` behaving similar to those for Python's standard dictionary
Serhiy Storchaka2e576f52017-04-24 09:05:00 +03002193objects. The :mod:`collections.abc` module provides a
Serhiy Storchaka0d196ed2013-10-09 14:02:31 +03002194:class:`~collections.abc.MutableMapping`
Georg Brandlc7723722008-05-26 17:47:11 +00002195abstract base class to help create those methods from a base set of
2196:meth:`__getitem__`, :meth:`__setitem__`, :meth:`__delitem__`, and :meth:`keys`.
2197Mutable sequences should provide methods :meth:`append`, :meth:`count`,
2198:meth:`index`, :meth:`extend`, :meth:`insert`, :meth:`pop`, :meth:`remove`,
2199:meth:`reverse` and :meth:`sort`, like Python standard list objects. Finally,
2200sequence types should implement addition (meaning concatenation) and
2201multiplication (meaning repetition) by defining the methods :meth:`__add__`,
2202:meth:`__radd__`, :meth:`__iadd__`, :meth:`__mul__`, :meth:`__rmul__` and
2203:meth:`__imul__` described below; they should not define other numerical
2204operators. It is recommended that both mappings and sequences implement the
2205:meth:`__contains__` method to allow efficient use of the ``in`` operator; for
2206mappings, ``in`` should search the mapping's keys; for sequences, it should
2207search through the values. It is further recommended that both mappings and
2208sequences implement the :meth:`__iter__` method to allow efficient iteration
alclarks4544e782019-11-17 22:00:43 +00002209through the container; for mappings, :meth:`__iter__` should iterate
2210through the object's keys; for sequences, it should iterate through the values.
Georg Brandl116aa622007-08-15 14:28:22 +00002211
2212.. method:: object.__len__(self)
2213
2214 .. index::
2215 builtin: len
2216 single: __bool__() (object method)
2217
2218 Called to implement the built-in function :func:`len`. Should return the length
2219 of the object, an integer ``>=`` 0. Also, an object that doesn't define a
2220 :meth:`__bool__` method and whose :meth:`__len__` method returns zero is
2221 considered to be false in a Boolean context.
2222
Serhiy Storchaka85157cd2017-04-23 08:37:58 +03002223 .. impl-detail::
2224
2225 In CPython, the length is required to be at most :attr:`sys.maxsize`.
2226 If the length is larger than :attr:`!sys.maxsize` some features (such as
2227 :func:`len`) may raise :exc:`OverflowError`. To prevent raising
2228 :exc:`!OverflowError` by truth value testing, an object must define a
2229 :meth:`__bool__` method.
2230
Georg Brandl116aa622007-08-15 14:28:22 +00002231
Armin Ronacher74b38b12012-10-07 10:29:32 +02002232.. method:: object.__length_hint__(self)
2233
Ezio Melottie12dc282012-10-07 12:09:36 +03002234 Called to implement :func:`operator.length_hint`. Should return an estimated
Armin Ronacher74b38b12012-10-07 10:29:32 +02002235 length for the object (which may be greater or less than the actual length).
Jeroen Demeyer009ef292019-09-10 16:01:13 +02002236 The length must be an integer ``>=`` 0. The return value may also be
Steve Dowera39a4c72019-09-10 15:25:12 +01002237 :const:`NotImplemented`, which is treated the same as if the
2238 ``__length_hint__`` method didn't exist at all. This method is purely an
2239 optimization and is never required for correctness.
Armin Ronacher74b38b12012-10-07 10:29:32 +02002240
2241 .. versionadded:: 3.4
2242
Serhiy Storchaka85157cd2017-04-23 08:37:58 +03002243
Andre Delfino1ce853f2018-12-05 16:42:44 -03002244.. index:: object: slice
2245
Georg Brandlcb8ecb12007-09-04 06:35:14 +00002246.. note::
2247
2248 Slicing is done exclusively with the following three methods. A call like ::
2249
2250 a[1:2] = b
2251
2252 is translated to ::
2253
2254 a[slice(1, 2, None)] = b
2255
2256 and so forth. Missing slice items are always filled in with ``None``.
2257
2258
Georg Brandl116aa622007-08-15 14:28:22 +00002259.. method:: object.__getitem__(self, key)
2260
Georg Brandl116aa622007-08-15 14:28:22 +00002261 Called to implement evaluation of ``self[key]``. For sequence types, the
2262 accepted keys should be integers and slice objects. Note that the special
2263 interpretation of negative indexes (if the class wishes to emulate a sequence
2264 type) is up to the :meth:`__getitem__` method. If *key* is of an inappropriate
2265 type, :exc:`TypeError` may be raised; if of a value outside the set of indexes
2266 for the sequence (after any special interpretation of negative values),
2267 :exc:`IndexError` should be raised. For mapping types, if *key* is missing (not
2268 in the container), :exc:`KeyError` should be raised.
2269
2270 .. note::
2271
2272 :keyword:`for` loops expect that an :exc:`IndexError` will be raised for illegal
2273 indexes to allow proper detection of the end of the sequence.
2274
2275
2276.. method:: object.__setitem__(self, key, value)
2277
2278 Called to implement assignment to ``self[key]``. Same note as for
2279 :meth:`__getitem__`. This should only be implemented for mappings if the
2280 objects support changes to the values for keys, or if new keys can be added, or
2281 for sequences if elements can be replaced. The same exceptions should be raised
2282 for improper *key* values as for the :meth:`__getitem__` method.
2283
2284
2285.. method:: object.__delitem__(self, key)
2286
2287 Called to implement deletion of ``self[key]``. Same note as for
2288 :meth:`__getitem__`. This should only be implemented for mappings if the
2289 objects support removal of keys, or for sequences if elements can be removed
2290 from the sequence. The same exceptions should be raised for improper *key*
2291 values as for the :meth:`__getitem__` method.
2292
2293
Andre Delfino1ce853f2018-12-05 16:42:44 -03002294.. method:: object.__missing__(self, key)
2295
2296 Called by :class:`dict`\ .\ :meth:`__getitem__` to implement ``self[key]`` for dict subclasses
2297 when key is not in the dictionary.
2298
2299
Georg Brandl116aa622007-08-15 14:28:22 +00002300.. method:: object.__iter__(self)
2301
2302 This method is called when an iterator is required for a container. This method
2303 should return a new iterator object that can iterate over all the objects in the
R David Murrayc9f5f2d2014-12-10 09:51:01 -05002304 container. For mappings, it should iterate over the keys of the container.
Georg Brandl116aa622007-08-15 14:28:22 +00002305
2306 Iterator objects also need to implement this method; they are required to return
2307 themselves. For more information on iterator objects, see :ref:`typeiter`.
2308
Christian Heimes7f044312008-01-06 17:05:40 +00002309
2310.. method:: object.__reversed__(self)
2311
Georg Brandl22b34312009-07-26 14:54:51 +00002312 Called (if present) by the :func:`reversed` built-in to implement
Christian Heimes7f044312008-01-06 17:05:40 +00002313 reverse iteration. It should return a new iterator object that iterates
2314 over all the objects in the container in reverse order.
2315
Georg Brandl8a1e4c42009-05-25 21:13:36 +00002316 If the :meth:`__reversed__` method is not provided, the :func:`reversed`
Georg Brandl22b34312009-07-26 14:54:51 +00002317 built-in will fall back to using the sequence protocol (:meth:`__len__` and
Georg Brandl8a1e4c42009-05-25 21:13:36 +00002318 :meth:`__getitem__`). Objects that support the sequence protocol should
2319 only provide :meth:`__reversed__` if they can provide an implementation
2320 that is more efficient than the one provided by :func:`reversed`.
Christian Heimes7f044312008-01-06 17:05:40 +00002321
2322
Georg Brandl116aa622007-08-15 14:28:22 +00002323The membership test operators (:keyword:`in` and :keyword:`not in`) are normally
alclarks4544e782019-11-17 22:00:43 +00002324implemented as an iteration through a container. However, container objects can
Georg Brandl116aa622007-08-15 14:28:22 +00002325supply the following special method with a more efficient implementation, which
alclarks4544e782019-11-17 22:00:43 +00002326also does not require the object be iterable.
Georg Brandl116aa622007-08-15 14:28:22 +00002327
Georg Brandl116aa622007-08-15 14:28:22 +00002328.. method:: object.__contains__(self, item)
2329
Georg Brandl495f7b52009-10-27 15:28:25 +00002330 Called to implement membership test operators. Should return true if *item*
2331 is in *self*, false otherwise. For mapping objects, this should consider the
2332 keys of the mapping rather than the values or the key-item pairs.
2333
2334 For objects that don't define :meth:`__contains__`, the membership test first
2335 tries iteration via :meth:`__iter__`, then the old sequence iteration
2336 protocol via :meth:`__getitem__`, see :ref:`this section in the language
2337 reference <membership-test-details>`.
Georg Brandl116aa622007-08-15 14:28:22 +00002338
2339
Georg Brandl116aa622007-08-15 14:28:22 +00002340.. _numeric-types:
2341
2342Emulating numeric types
2343-----------------------
2344
2345The following methods can be defined to emulate numeric objects. Methods
2346corresponding to operations that are not supported by the particular kind of
2347number implemented (e.g., bitwise operations for non-integral numbers) should be
2348left undefined.
2349
2350
2351.. method:: object.__add__(self, other)
2352 object.__sub__(self, other)
2353 object.__mul__(self, other)
Benjamin Petersond51374e2014-04-09 23:55:56 -04002354 object.__matmul__(self, other)
Georg Brandlae55dc02008-09-06 17:43:49 +00002355 object.__truediv__(self, other)
Georg Brandl116aa622007-08-15 14:28:22 +00002356 object.__floordiv__(self, other)
2357 object.__mod__(self, other)
2358 object.__divmod__(self, other)
2359 object.__pow__(self, other[, modulo])
2360 object.__lshift__(self, other)
2361 object.__rshift__(self, other)
2362 object.__and__(self, other)
2363 object.__xor__(self, other)
2364 object.__or__(self, other)
2365
2366 .. index::
2367 builtin: divmod
2368 builtin: pow
2369 builtin: pow
2370
Benjamin Petersond51374e2014-04-09 23:55:56 -04002371 These methods are called to implement the binary arithmetic operations
2372 (``+``, ``-``, ``*``, ``@``, ``/``, ``//``, ``%``, :func:`divmod`,
2373 :func:`pow`, ``**``, ``<<``, ``>>``, ``&``, ``^``, ``|``). For instance, to
2374 evaluate the expression ``x + y``, where *x* is an instance of a class that
2375 has an :meth:`__add__` method, ``x.__add__(y)`` is called. The
2376 :meth:`__divmod__` method should be the equivalent to using
2377 :meth:`__floordiv__` and :meth:`__mod__`; it should not be related to
2378 :meth:`__truediv__`. Note that :meth:`__pow__` should be defined to accept
2379 an optional third argument if the ternary version of the built-in :func:`pow`
2380 function is to be supported.
Georg Brandl116aa622007-08-15 14:28:22 +00002381
2382 If one of those methods does not support the operation with the supplied
2383 arguments, it should return ``NotImplemented``.
2384
2385
Georg Brandl116aa622007-08-15 14:28:22 +00002386.. method:: object.__radd__(self, other)
2387 object.__rsub__(self, other)
2388 object.__rmul__(self, other)
Benjamin Petersond51374e2014-04-09 23:55:56 -04002389 object.__rmatmul__(self, other)
Georg Brandl116aa622007-08-15 14:28:22 +00002390 object.__rtruediv__(self, other)
2391 object.__rfloordiv__(self, other)
2392 object.__rmod__(self, other)
2393 object.__rdivmod__(self, other)
HongWeipengabc0c4f2020-01-05 16:20:29 -06002394 object.__rpow__(self, other[, modulo])
Georg Brandl116aa622007-08-15 14:28:22 +00002395 object.__rlshift__(self, other)
2396 object.__rrshift__(self, other)
2397 object.__rand__(self, other)
2398 object.__rxor__(self, other)
2399 object.__ror__(self, other)
2400
2401 .. index::
2402 builtin: divmod
2403 builtin: pow
2404
Benjamin Petersond51374e2014-04-09 23:55:56 -04002405 These methods are called to implement the binary arithmetic operations
2406 (``+``, ``-``, ``*``, ``@``, ``/``, ``//``, ``%``, :func:`divmod`,
2407 :func:`pow`, ``**``, ``<<``, ``>>``, ``&``, ``^``, ``|``) with reflected
2408 (swapped) operands. These functions are only called if the left operand does
Guido van Rossum97c1adf2016-08-18 09:22:23 -07002409 not support the corresponding operation [#]_ and the operands are of different
Benjamin Petersond51374e2014-04-09 23:55:56 -04002410 types. [#]_ For instance, to evaluate the expression ``x - y``, where *y* is
2411 an instance of a class that has an :meth:`__rsub__` method, ``y.__rsub__(x)``
2412 is called if ``x.__sub__(y)`` returns *NotImplemented*.
Georg Brandl116aa622007-08-15 14:28:22 +00002413
2414 .. index:: builtin: pow
2415
2416 Note that ternary :func:`pow` will not try calling :meth:`__rpow__` (the
2417 coercion rules would become too complicated).
2418
2419 .. note::
2420
Brett Cannond02d8242020-10-05 09:42:21 -07002421 If the right operand's type is a subclass of the left operand's type and
2422 that subclass provides a different implementation of the reflected method
2423 for the operation, this method will be called before the left operand's
2424 non-reflected method. This behavior allows subclasses to override their
2425 ancestors' operations.
Georg Brandl116aa622007-08-15 14:28:22 +00002426
2427
2428.. method:: object.__iadd__(self, other)
2429 object.__isub__(self, other)
2430 object.__imul__(self, other)
Benjamin Petersond51374e2014-04-09 23:55:56 -04002431 object.__imatmul__(self, other)
Georg Brandl116aa622007-08-15 14:28:22 +00002432 object.__itruediv__(self, other)
2433 object.__ifloordiv__(self, other)
2434 object.__imod__(self, other)
2435 object.__ipow__(self, other[, modulo])
2436 object.__ilshift__(self, other)
2437 object.__irshift__(self, other)
2438 object.__iand__(self, other)
2439 object.__ixor__(self, other)
2440 object.__ior__(self, other)
2441
Benjamin Petersonb58dda72009-01-18 22:27:04 +00002442 These methods are called to implement the augmented arithmetic assignments
Benjamin Petersond51374e2014-04-09 23:55:56 -04002443 (``+=``, ``-=``, ``*=``, ``@=``, ``/=``, ``//=``, ``%=``, ``**=``, ``<<=``,
2444 ``>>=``, ``&=``, ``^=``, ``|=``). These methods should attempt to do the
2445 operation in-place (modifying *self*) and return the result (which could be,
2446 but does not have to be, *self*). If a specific method is not defined, the
2447 augmented assignment falls back to the normal methods. For instance, if *x*
2448 is an instance of a class with an :meth:`__iadd__` method, ``x += y`` is
2449 equivalent to ``x = x.__iadd__(y)`` . Otherwise, ``x.__add__(y)`` and
2450 ``y.__radd__(x)`` are considered, as with the evaluation of ``x + y``. In
2451 certain situations, augmented assignment can result in unexpected errors (see
2452 :ref:`faq-augmented-assignment-tuple-error`), but this behavior is in fact
2453 part of the data model.
Georg Brandl116aa622007-08-15 14:28:22 +00002454
2455
2456.. method:: object.__neg__(self)
2457 object.__pos__(self)
2458 object.__abs__(self)
2459 object.__invert__(self)
2460
2461 .. index:: builtin: abs
2462
2463 Called to implement the unary arithmetic operations (``-``, ``+``, :func:`abs`
2464 and ``~``).
2465
2466
2467.. method:: object.__complex__(self)
2468 object.__int__(self)
Georg Brandl116aa622007-08-15 14:28:22 +00002469 object.__float__(self)
2470
2471 .. index::
2472 builtin: complex
2473 builtin: int
Georg Brandl116aa622007-08-15 14:28:22 +00002474 builtin: float
2475
Mark Summerfield9557f602008-07-01 14:42:30 +00002476 Called to implement the built-in functions :func:`complex`,
Serhiy Storchaka496431f2018-03-10 17:18:32 +02002477 :func:`int` and :func:`float`. Should return a value
Mark Summerfield9557f602008-07-01 14:42:30 +00002478 of the appropriate type.
Georg Brandl116aa622007-08-15 14:28:22 +00002479
2480
2481.. method:: object.__index__(self)
2482
Ethan Furmandf3ed242014-01-05 06:50:30 -08002483 Called to implement :func:`operator.index`, and whenever Python needs to
2484 losslessly convert the numeric object to an integer object (such as in
2485 slicing, or in the built-in :func:`bin`, :func:`hex` and :func:`oct`
2486 functions). Presence of this method indicates that the numeric object is
2487 an integer type. Must return an integer.
2488
Serhiy Storchakabdbad712019-06-02 00:05:48 +03002489 If :meth:`__int__`, :meth:`__float__` and :meth:`__complex__` are not
2490 defined then corresponding built-in functions :func:`int`, :func:`float`
2491 and :func:`complex` fall back to :meth:`__index__`.
Georg Brandl116aa622007-08-15 14:28:22 +00002492
Georg Brandl116aa622007-08-15 14:28:22 +00002493
Serhiy Storchaka496431f2018-03-10 17:18:32 +02002494.. method:: object.__round__(self, [,ndigits])
2495 object.__trunc__(self)
2496 object.__floor__(self)
2497 object.__ceil__(self)
2498
2499 .. index:: builtin: round
2500
2501 Called to implement the built-in function :func:`round` and :mod:`math`
2502 functions :func:`~math.trunc`, :func:`~math.floor` and :func:`~math.ceil`.
2503 Unless *ndigits* is passed to :meth:`!__round__` all these methods should
2504 return the value of the object truncated to an :class:`~numbers.Integral`
2505 (typically an :class:`int`).
2506
2507 If :meth:`__int__` is not defined then the built-in function :func:`int`
2508 falls back to :meth:`__trunc__`.
2509
2510
Georg Brandl116aa622007-08-15 14:28:22 +00002511.. _context-managers:
2512
2513With Statement Context Managers
2514-------------------------------
2515
Georg Brandl116aa622007-08-15 14:28:22 +00002516A :dfn:`context manager` is an object that defines the runtime context to be
2517established when executing a :keyword:`with` statement. The context manager
2518handles the entry into, and the exit from, the desired runtime context for the
2519execution of the block of code. Context managers are normally invoked using the
Serhiy Storchaka2b57c432018-12-19 08:09:46 +02002520:keyword:`!with` statement (described in section :ref:`with`), but can also be
Georg Brandl116aa622007-08-15 14:28:22 +00002521used by directly invoking their methods.
2522
2523.. index::
2524 statement: with
2525 single: context manager
2526
2527Typical uses of context managers include saving and restoring various kinds of
2528global state, locking and unlocking resources, closing opened files, etc.
2529
2530For more information on context managers, see :ref:`typecontextmanager`.
2531
2532
2533.. method:: object.__enter__(self)
2534
2535 Enter the runtime context related to this object. The :keyword:`with` statement
2536 will bind this method's return value to the target(s) specified in the
Serhiy Storchaka2b57c432018-12-19 08:09:46 +02002537 :keyword:`!as` clause of the statement, if any.
Georg Brandl116aa622007-08-15 14:28:22 +00002538
2539
2540.. method:: object.__exit__(self, exc_type, exc_value, traceback)
2541
2542 Exit the runtime context related to this object. The parameters describe the
2543 exception that caused the context to be exited. If the context was exited
2544 without an exception, all three arguments will be :const:`None`.
2545
2546 If an exception is supplied, and the method wishes to suppress the exception
2547 (i.e., prevent it from being propagated), it should return a true value.
2548 Otherwise, the exception will be processed normally upon exit from this method.
2549
2550 Note that :meth:`__exit__` methods should not reraise the passed-in exception;
2551 this is the caller's responsibility.
2552
2553
2554.. seealso::
2555
Serhiy Storchakae4ba8722016-03-31 15:30:54 +03002556 :pep:`343` - The "with" statement
Georg Brandl116aa622007-08-15 14:28:22 +00002557 The specification, background, and examples for the Python :keyword:`with`
2558 statement.
2559
Nick Coghlan3a5d7e32008-08-31 12:40:14 +00002560
Daniel F Moisseta22bca62021-03-01 04:08:38 +00002561.. _class-pattern-matching:
2562
2563Customizing positional arguments in class pattern matching
2564----------------------------------------------------------
2565
2566When using a class name in a pattern, positional arguments in the pattern are not
2567allowed by default, i.e. ``case MyClass(x, y)`` is typically invalid without special
2568support in ``MyClass``. To be able to use that kind of patterns, the class needs to
2569define a *__match_args__* attribute.
2570
2571.. data:: object.__match_args__
2572
Ken Jin5143fd12021-04-07 00:03:00 +08002573 This class variable can be assigned a tuple of strings. When this class is
Daniel F Moisseta22bca62021-03-01 04:08:38 +00002574 used in a class pattern with positional arguments, each positional argument will
2575 be converted into a keyword argument, using the corresponding value in
2576 *__match_args__* as the keyword. The absence of this attribute is equivalent to
2577 setting it to ``()``.
2578
2579For example, if ``MyClass.__match_args__`` is ``("left", "center", "right")`` that means
2580that ``case MyClass(x, y)`` is equivalent to ``case MyClass(left=x, center=y)``. Note
2581that the number of arguments in the pattern must be smaller than or equal to the number
2582of elements in *__match_args__*; if it is larger, the pattern match attempt will raise
2583a :exc:`TypeError`.
2584
2585.. versionadded:: 3.10
2586
2587.. seealso::
2588
2589 :pep:`634` - Structural Pattern Matching
2590 The specification for the Python ``match`` statement.
2591
2592
Nick Coghlan3a5d7e32008-08-31 12:40:14 +00002593.. _special-lookup:
2594
2595Special method lookup
2596---------------------
2597
2598For custom classes, implicit invocations of special methods are only guaranteed
2599to work correctly if defined on an object's type, not in the object's instance
2600dictionary. That behaviour is the reason why the following code raises an
2601exception::
2602
Éric Araujo28053fb2010-11-22 03:09:19 +00002603 >>> class C:
Nick Coghlan3a5d7e32008-08-31 12:40:14 +00002604 ... pass
2605 ...
2606 >>> c = C()
2607 >>> c.__len__ = lambda: 5
2608 >>> len(c)
2609 Traceback (most recent call last):
2610 File "<stdin>", line 1, in <module>
2611 TypeError: object of type 'C' has no len()
2612
2613The rationale behind this behaviour lies with a number of special methods such
2614as :meth:`__hash__` and :meth:`__repr__` that are implemented by all objects,
2615including type objects. If the implicit lookup of these methods used the
2616conventional lookup process, they would fail when invoked on the type object
2617itself::
2618
2619 >>> 1 .__hash__() == hash(1)
2620 True
2621 >>> int.__hash__() == hash(int)
2622 Traceback (most recent call last):
2623 File "<stdin>", line 1, in <module>
2624 TypeError: descriptor '__hash__' of 'int' object needs an argument
2625
2626Incorrectly attempting to invoke an unbound method of a class in this way is
2627sometimes referred to as 'metaclass confusion', and is avoided by bypassing
2628the instance when looking up special methods::
2629
2630 >>> type(1).__hash__(1) == hash(1)
2631 True
2632 >>> type(int).__hash__(int) == hash(int)
2633 True
2634
2635In addition to bypassing any instance attributes in the interest of
Georg Brandlaf265f42008-12-07 15:06:20 +00002636correctness, implicit special method lookup generally also bypasses the
Nick Coghlan3a5d7e32008-08-31 12:40:14 +00002637:meth:`__getattribute__` method even of the object's metaclass::
2638
2639 >>> class Meta(type):
Berker Peksag770319d2015-04-11 14:59:30 +03002640 ... def __getattribute__(*args):
2641 ... print("Metaclass getattribute invoked")
2642 ... return type.__getattribute__(*args)
Nick Coghlan3a5d7e32008-08-31 12:40:14 +00002643 ...
Benjamin Petersone348d1a2008-10-19 21:29:05 +00002644 >>> class C(object, metaclass=Meta):
Nick Coghlan3a5d7e32008-08-31 12:40:14 +00002645 ... def __len__(self):
2646 ... return 10
2647 ... def __getattribute__(*args):
Benjamin Peterson64106fb2008-10-29 20:35:35 +00002648 ... print("Class getattribute invoked")
Nick Coghlan3a5d7e32008-08-31 12:40:14 +00002649 ... return object.__getattribute__(*args)
2650 ...
2651 >>> c = C()
2652 >>> c.__len__() # Explicit lookup via instance
2653 Class getattribute invoked
2654 10
2655 >>> type(c).__len__(c) # Explicit lookup via type
2656 Metaclass getattribute invoked
2657 10
2658 >>> len(c) # Implicit lookup
2659 10
2660
2661Bypassing the :meth:`__getattribute__` machinery in this fashion
2662provides significant scope for speed optimisations within the
2663interpreter, at the cost of some flexibility in the handling of
2664special methods (the special method *must* be set on the class
2665object itself in order to be consistently invoked by the interpreter).
2666
2667
Yury Selivanov66f88282015-06-24 11:04:15 -04002668.. index::
2669 single: coroutine
Yury Selivanovf3e40fa2015-05-21 11:50:30 -04002670
2671Coroutines
2672==========
2673
Yury Selivanovf3e40fa2015-05-21 11:50:30 -04002674
2675Awaitable Objects
2676-----------------
2677
Yury Selivanov66f88282015-06-24 11:04:15 -04002678An :term:`awaitable` object generally implements an :meth:`__await__` method.
Victor Stinnerbb0b0852020-08-14 12:20:05 +02002679:term:`Coroutine objects <coroutine>` returned from :keyword:`async def` functions
Yury Selivanov66f88282015-06-24 11:04:15 -04002680are awaitable.
Yury Selivanovf3e40fa2015-05-21 11:50:30 -04002681
Yury Selivanov66f88282015-06-24 11:04:15 -04002682.. note::
Yury Selivanovf3e40fa2015-05-21 11:50:30 -04002683
Yury Selivanov66f88282015-06-24 11:04:15 -04002684 The :term:`generator iterator` objects returned from generators
2685 decorated with :func:`types.coroutine` or :func:`asyncio.coroutine`
2686 are also awaitable, but they do not implement :meth:`__await__`.
Yury Selivanovf3e40fa2015-05-21 11:50:30 -04002687
2688.. method:: object.__await__(self)
2689
2690 Must return an :term:`iterator`. Should be used to implement
2691 :term:`awaitable` objects. For instance, :class:`asyncio.Future` implements
2692 this method to be compatible with the :keyword:`await` expression.
2693
2694.. versionadded:: 3.5
2695
2696.. seealso:: :pep:`492` for additional information about awaitable objects.
2697
2698
Yury Selivanov66f88282015-06-24 11:04:15 -04002699.. _coroutine-objects:
2700
2701Coroutine Objects
2702-----------------
2703
Victor Stinnerbb0b0852020-08-14 12:20:05 +02002704:term:`Coroutine objects <coroutine>` are :term:`awaitable` objects.
Yury Selivanov66f88282015-06-24 11:04:15 -04002705A coroutine's execution can be controlled by calling :meth:`__await__` and
2706iterating over the result. When the coroutine has finished executing and
2707returns, the iterator raises :exc:`StopIteration`, and the exception's
2708:attr:`~StopIteration.value` attribute holds the return value. If the
2709coroutine raises an exception, it is propagated by the iterator. Coroutines
2710should not directly raise unhandled :exc:`StopIteration` exceptions.
2711
2712Coroutines also have the methods listed below, which are analogous to
2713those of generators (see :ref:`generator-methods`). However, unlike
2714generators, coroutines do not directly support iteration.
2715
Yury Selivanov77c96812016-02-13 17:59:05 -05002716.. versionchanged:: 3.5.2
2717 It is a :exc:`RuntimeError` to await on a coroutine more than once.
2718
2719
Yury Selivanov66f88282015-06-24 11:04:15 -04002720.. method:: coroutine.send(value)
2721
2722 Starts or resumes execution of the coroutine. If *value* is ``None``,
2723 this is equivalent to advancing the iterator returned by
2724 :meth:`__await__`. If *value* is not ``None``, this method delegates
2725 to the :meth:`~generator.send` method of the iterator that caused
2726 the coroutine to suspend. The result (return value,
2727 :exc:`StopIteration`, or other exception) is the same as when
2728 iterating over the :meth:`__await__` return value, described above.
2729
2730.. method:: coroutine.throw(type[, value[, traceback]])
2731
2732 Raises the specified exception in the coroutine. This method delegates
2733 to the :meth:`~generator.throw` method of the iterator that caused
2734 the coroutine to suspend, if it has such a method. Otherwise,
2735 the exception is raised at the suspension point. The result
2736 (return value, :exc:`StopIteration`, or other exception) is the same as
2737 when iterating over the :meth:`__await__` return value, described
2738 above. If the exception is not caught in the coroutine, it propagates
2739 back to the caller.
2740
2741.. method:: coroutine.close()
2742
2743 Causes the coroutine to clean itself up and exit. If the coroutine
2744 is suspended, this method first delegates to the :meth:`~generator.close`
2745 method of the iterator that caused the coroutine to suspend, if it
2746 has such a method. Then it raises :exc:`GeneratorExit` at the
2747 suspension point, causing the coroutine to immediately clean itself up.
2748 Finally, the coroutine is marked as having finished executing, even if
2749 it was never started.
2750
2751 Coroutine objects are automatically closed using the above process when
2752 they are about to be destroyed.
2753
Yury Selivanova6f6edb2016-06-09 15:08:31 -04002754.. _async-iterators:
Yury Selivanov66f88282015-06-24 11:04:15 -04002755
Yury Selivanovf3e40fa2015-05-21 11:50:30 -04002756Asynchronous Iterators
2757----------------------
2758
Yury Selivanovfaa135a2017-10-06 02:08:57 -04002759An *asynchronous iterator* can call asynchronous code in
2760its ``__anext__`` method.
Yury Selivanovf3e40fa2015-05-21 11:50:30 -04002761
Martin Panterd2ad5712015-11-02 04:20:33 +00002762Asynchronous iterators can be used in an :keyword:`async for` statement.
Yury Selivanovf3e40fa2015-05-21 11:50:30 -04002763
2764.. method:: object.__aiter__(self)
2765
Yury Selivanova6f6edb2016-06-09 15:08:31 -04002766 Must return an *asynchronous iterator* object.
Yury Selivanovf3e40fa2015-05-21 11:50:30 -04002767
2768.. method:: object.__anext__(self)
2769
2770 Must return an *awaitable* resulting in a next value of the iterator. Should
2771 raise a :exc:`StopAsyncIteration` error when the iteration is over.
2772
2773An example of an asynchronous iterable object::
2774
2775 class Reader:
2776 async def readline(self):
2777 ...
2778
Yury Selivanova6f6edb2016-06-09 15:08:31 -04002779 def __aiter__(self):
Yury Selivanovf3e40fa2015-05-21 11:50:30 -04002780 return self
2781
2782 async def __anext__(self):
2783 val = await self.readline()
2784 if val == b'':
2785 raise StopAsyncIteration
2786 return val
2787
2788.. versionadded:: 3.5
2789
Yury Selivanovfaa135a2017-10-06 02:08:57 -04002790.. versionchanged:: 3.7
2791 Prior to Python 3.7, ``__aiter__`` could return an *awaitable*
2792 that would resolve to an
2793 :term:`asynchronous iterator <asynchronous iterator>`.
Yury Selivanova6f6edb2016-06-09 15:08:31 -04002794
Yury Selivanovfaa135a2017-10-06 02:08:57 -04002795 Starting with Python 3.7, ``__aiter__`` must return an
2796 asynchronous iterator object. Returning anything else
2797 will result in a :exc:`TypeError` error.
Yury Selivanova6f6edb2016-06-09 15:08:31 -04002798
Yury Selivanovf3e40fa2015-05-21 11:50:30 -04002799
Jelle Zijlstra2e624692017-04-30 18:25:58 -07002800.. _async-context-managers:
2801
Yury Selivanovf3e40fa2015-05-21 11:50:30 -04002802Asynchronous Context Managers
2803-----------------------------
2804
2805An *asynchronous context manager* is a *context manager* that is able to
2806suspend execution in its ``__aenter__`` and ``__aexit__`` methods.
2807
Martin Panterd2ad5712015-11-02 04:20:33 +00002808Asynchronous context managers can be used in an :keyword:`async with` statement.
Yury Selivanovf3e40fa2015-05-21 11:50:30 -04002809
2810.. method:: object.__aenter__(self)
2811
Andre Delfinoa8a79ca2019-05-03 12:08:10 -03002812 Semantically similar to :meth:`__enter__`, the only
2813 difference being that it must return an *awaitable*.
Yury Selivanovf3e40fa2015-05-21 11:50:30 -04002814
2815.. method:: object.__aexit__(self, exc_type, exc_value, traceback)
2816
Andre Delfinoa8a79ca2019-05-03 12:08:10 -03002817 Semantically similar to :meth:`__exit__`, the only
2818 difference being that it must return an *awaitable*.
Yury Selivanovf3e40fa2015-05-21 11:50:30 -04002819
2820An example of an asynchronous context manager class::
2821
2822 class AsyncContextManager:
2823 async def __aenter__(self):
2824 await log('entering context')
2825
2826 async def __aexit__(self, exc_type, exc, tb):
2827 await log('exiting context')
2828
2829.. versionadded:: 3.5
2830
2831
Georg Brandl116aa622007-08-15 14:28:22 +00002832.. rubric:: Footnotes
2833
Nick Coghlan3a5d7e32008-08-31 12:40:14 +00002834.. [#] It *is* possible in some cases to change an object's type, under certain
2835 controlled conditions. It generally isn't a good idea though, since it can
2836 lead to some very strange behaviour if it is handled incorrectly.
2837
Guido van Rossum97c1adf2016-08-18 09:22:23 -07002838.. [#] The :meth:`__hash__`, :meth:`__iter__`, :meth:`__reversed__`, and
2839 :meth:`__contains__` methods have special handling for this; others
2840 will still raise a :exc:`TypeError`, but may do so by relying on
2841 the behavior that ``None`` is not callable.
2842
2843.. [#] "Does not support" here means that the class has no such method, or
2844 the method returns ``NotImplemented``. Do not set the method to
2845 ``None`` if you want to force fallback to the right operand's reflected
Martin Panter28540182016-11-21 04:10:45 +00002846 method—that will instead have the opposite effect of explicitly
Guido van Rossum97c1adf2016-08-18 09:22:23 -07002847 *blocking* such fallback.
2848
Brett Cannond02d8242020-10-05 09:42:21 -07002849.. [#] For operands of the same type, it is assumed that if the non-reflected
2850 method -- such as :meth:`__add__` -- fails then the overall operation is not
2851 supported, which is why the reflected method is not called.