blob: dfe5eb6857c0f5368418af36546573a704700071 [file] [log] [blame]
Georg Brandl116aa622007-08-15 14:28:22 +00001
2.. _datamodel:
3
4**********
5Data model
6**********
7
8
9.. _objects:
10
11Objects, values and types
12=========================
13
14.. index::
15 single: object
16 single: data
17
18:dfn:`Objects` are Python's abstraction for data. All data in a Python program
19is represented by objects or by relations between objects. (In a sense, and in
Mathieu Dupuyc49016e2020-03-30 23:28:25 +020020conformance to Von Neumann's model of a "stored program computer", code is also
Georg Brandl116aa622007-08-15 14:28:22 +000021represented by objects.)
22
23.. index::
24 builtin: id
25 builtin: type
26 single: identity of an object
27 single: value of an object
28 single: type of an object
29 single: mutable object
30 single: immutable object
31
Georg Brandl85eb8c12007-08-31 16:33:38 +000032.. XXX it *is* now possible in some cases to change an object's
33 type, under certain controlled conditions
34
Georg Brandl116aa622007-08-15 14:28:22 +000035Every object has an identity, a type and a value. An object's *identity* never
36changes once it has been created; you may think of it as the object's address in
37memory. The ':keyword:`is`' operator compares the identity of two objects; the
Nick Coghlan337b2bf2012-05-20 18:30:49 +100038:func:`id` function returns an integer representing its identity.
39
40.. impl-detail::
41
42 For CPython, ``id(x)`` is the memory address where ``x`` is stored.
43
Georg Brandl116aa622007-08-15 14:28:22 +000044An object's type determines the operations that the object supports (e.g., "does
45it have a length?") and also defines the possible values for objects of that
46type. The :func:`type` function returns an object's type (which is an object
Nick Coghlan337b2bf2012-05-20 18:30:49 +100047itself). Like its identity, an object's :dfn:`type` is also unchangeable.
48[#]_
49
50The *value* of some objects can change. Objects whose value can
Georg Brandl116aa622007-08-15 14:28:22 +000051change are said to be *mutable*; objects whose value is unchangeable once they
52are created are called *immutable*. (The value of an immutable container object
53that contains a reference to a mutable object can change when the latter's value
54is changed; however the container is still considered immutable, because the
55collection of objects it contains cannot be changed. So, immutability is not
56strictly the same as having an unchangeable value, it is more subtle.) An
57object's mutability is determined by its type; for instance, numbers, strings
58and tuples are immutable, while dictionaries and lists are mutable.
59
60.. index::
61 single: garbage collection
62 single: reference counting
63 single: unreachable object
64
65Objects are never explicitly destroyed; however, when they become unreachable
66they may be garbage-collected. An implementation is allowed to postpone garbage
67collection or omit it altogether --- it is a matter of implementation quality
68how garbage collection is implemented, as long as no objects are collected that
Georg Brandl495f7b52009-10-27 15:28:25 +000069are still reachable.
70
71.. impl-detail::
72
73 CPython currently uses a reference-counting scheme with (optional) delayed
74 detection of cyclically linked garbage, which collects most objects as soon
75 as they become unreachable, but is not guaranteed to collect garbage
76 containing circular references. See the documentation of the :mod:`gc`
77 module for information on controlling the collection of cyclic garbage.
78 Other implementations act differently and CPython may change.
Gregory P. Smithc5425472011-03-10 11:28:50 -080079 Do not depend on immediate finalization of objects when they become
Raymond Hettingeraa7886d2014-05-26 22:20:37 -070080 unreachable (so you should always close files explicitly).
Georg Brandl116aa622007-08-15 14:28:22 +000081
82Note that the use of the implementation's tracing or debugging facilities may
83keep objects alive that would normally be collectable. Also note that catching
84an exception with a ':keyword:`try`...\ :keyword:`except`' statement may keep
85objects alive.
86
87Some objects contain references to "external" resources such as open files or
88windows. It is understood that these resources are freed when the object is
89garbage-collected, but since garbage collection is not guaranteed to happen,
90such objects also provide an explicit way to release the external resource,
91usually a :meth:`close` method. Programs are strongly recommended to explicitly
92close such objects. The ':keyword:`try`...\ :keyword:`finally`' statement
Nick Coghlan3a5d7e32008-08-31 12:40:14 +000093and the ':keyword:`with`' statement provide convenient ways to do this.
Georg Brandl116aa622007-08-15 14:28:22 +000094
95.. index:: single: container
96
97Some objects contain references to other objects; these are called *containers*.
98Examples of containers are tuples, lists and dictionaries. The references are
99part of a container's value. In most cases, when we talk about the value of a
100container, we imply the values, not the identities of the contained objects;
101however, when we talk about the mutability of a container, only the identities
102of the immediately contained objects are implied. So, if an immutable container
103(like a tuple) contains a reference to a mutable object, its value changes if
104that mutable object is changed.
105
106Types affect almost all aspects of object behavior. Even the importance of
107object identity is affected in some sense: for immutable types, operations that
108compute new values may actually return a reference to any existing object with
109the same type and value, while for mutable objects this is not allowed. E.g.,
110after ``a = 1; b = 1``, ``a`` and ``b`` may or may not refer to the same object
111with the value one, depending on the implementation, but after ``c = []; d =
112[]``, ``c`` and ``d`` are guaranteed to refer to two different, unique, newly
113created empty lists. (Note that ``c = d = []`` assigns the same object to both
114``c`` and ``d``.)
115
116
117.. _types:
118
119The standard type hierarchy
120===========================
121
122.. index::
123 single: type
124 pair: data; type
125 pair: type; hierarchy
126 pair: extension; module
127 pair: C; language
128
129Below is a list of the types that are built into Python. Extension modules
130(written in C, Java, or other languages, depending on the implementation) can
131define additional types. Future versions of Python may add types to the type
Nick Coghlan3a5d7e32008-08-31 12:40:14 +0000132hierarchy (e.g., rational numbers, efficiently stored arrays of integers, etc.),
133although such additions will often be provided via the standard library instead.
Georg Brandl116aa622007-08-15 14:28:22 +0000134
135.. index::
136 single: attribute
137 pair: special; attribute
138 triple: generic; special; attribute
139
140Some of the type descriptions below contain a paragraph listing 'special
141attributes.' These are attributes that provide access to the implementation and
142are not intended for general use. Their definition may change in the future.
143
144None
145 .. index:: object: None
146
147 This type has a single value. There is a single object with this value. This
148 object is accessed through the built-in name ``None``. It is used to signify the
149 absence of a value in many situations, e.g., it is returned from functions that
150 don't explicitly return anything. Its truth value is false.
151
152NotImplemented
153 .. index:: object: NotImplemented
154
155 This type has a single value. There is a single object with this value. This
156 object is accessed through the built-in name ``NotImplemented``. Numeric methods
Ethan Furmanb0049432014-11-26 21:15:35 -0800157 and rich comparison methods should return this value if they do not implement the
Georg Brandl116aa622007-08-15 14:28:22 +0000158 operation for the operands provided. (The interpreter will then try the
MojoVampire469325c2020-03-03 18:50:17 +0000159 reflected operation, or some other fallback, depending on the operator.) It
160 should not be evaluated in a boolean context.
Georg Brandl116aa622007-08-15 14:28:22 +0000161
Ethan Furmanb0049432014-11-26 21:15:35 -0800162 See
163 :ref:`implementing-the-arithmetic-operations`
164 for more details.
165
MojoVampire469325c2020-03-03 18:50:17 +0000166 .. versionchanged:: 3.9
167 Evaluating ``NotImplemented`` in a boolean context is deprecated. While
168 it currently evaluates as true, it will emit a :exc:`DeprecationWarning`.
169 It will raise a :exc:`TypeError` in a future version of Python.
170
Ethan Furmanb0049432014-11-26 21:15:35 -0800171
Georg Brandl116aa622007-08-15 14:28:22 +0000172Ellipsis
Serhiy Storchakaddb961d2018-10-26 09:00:49 +0300173 .. index::
174 object: Ellipsis
175 single: ...; ellipsis literal
Georg Brandl116aa622007-08-15 14:28:22 +0000176
177 This type has a single value. There is a single object with this value. This
178 object is accessed through the literal ``...`` or the built-in name
179 ``Ellipsis``. Its truth value is true.
180
Christian Heimes072c0f12008-01-03 23:01:04 +0000181:class:`numbers.Number`
Georg Brandl116aa622007-08-15 14:28:22 +0000182 .. index:: object: numeric
183
184 These are created by numeric literals and returned as results by arithmetic
185 operators and arithmetic built-in functions. Numeric objects are immutable;
186 once created their value never changes. Python numbers are of course strongly
187 related to mathematical numbers, but subject to the limitations of numerical
188 representation in computers.
189
kpincf8b1ccd2020-10-21 15:34:15 -0500190 The string representations of the numeric classes, computed by
kpincc60394c2020-10-21 12:13:50 -0500191 :meth:`__repr__` and :meth:`__str__`, have the following
192 properties:
193
194 * They are valid numeric literals which, when passed to their
195 class constructor, produce an object having the value of the
196 original numeric.
197
198 * The representation is in base 10, when possible.
199
200 * Leading zeros, possibly excepting a single zero before a
201 decimal point, are not shown.
202
203 * Trailing zeros, possibly excepting a single zero after a
204 decimal point, are not shown.
205
206 * A sign is shown only when the number is negative.
207
Georg Brandl116aa622007-08-15 14:28:22 +0000208 Python distinguishes between integers, floating point numbers, and complex
209 numbers:
210
Christian Heimes072c0f12008-01-03 23:01:04 +0000211 :class:`numbers.Integral`
Georg Brandl116aa622007-08-15 14:28:22 +0000212 .. index:: object: integer
213
214 These represent elements from the mathematical set of integers (positive and
215 negative).
216
Georg Brandl59d69162008-01-07 09:27:36 +0000217 There are two types of integers:
Georg Brandl116aa622007-08-15 14:28:22 +0000218
Nick Coghlan3a5d7e32008-08-31 12:40:14 +0000219 Integers (:class:`int`)
Georg Brandl116aa622007-08-15 14:28:22 +0000220 These represent numbers in an unlimited range, subject to available (virtual)
221 memory only. For the purpose of shift and mask operations, a binary
222 representation is assumed, and negative numbers are represented in a variant of
223 2's complement which gives the illusion of an infinite string of sign bits
224 extending to the left.
225
Nick Coghlan3a5d7e32008-08-31 12:40:14 +0000226 Booleans (:class:`bool`)
Georg Brandl116aa622007-08-15 14:28:22 +0000227 .. index::
228 object: Boolean
229 single: False
230 single: True
231
232 These represent the truth values False and True. The two objects representing
Serhiy Storchakafbc1c262013-11-29 12:17:13 +0200233 the values ``False`` and ``True`` are the only Boolean objects. The Boolean type is a
Georg Brandl95817b32008-05-11 14:30:18 +0000234 subtype of the integer type, and Boolean values behave like the values 0 and 1,
Georg Brandl116aa622007-08-15 14:28:22 +0000235 respectively, in almost all contexts, the exception being that when converted to
236 a string, the strings ``"False"`` or ``"True"`` are returned, respectively.
237
238 .. index:: pair: integer; representation
239
240 The rules for integer representation are intended to give the most meaningful
Georg Brandlbb74a782008-05-11 10:53:16 +0000241 interpretation of shift and mask operations involving negative integers.
Georg Brandl116aa622007-08-15 14:28:22 +0000242
Christian Heimes072c0f12008-01-03 23:01:04 +0000243 :class:`numbers.Real` (:class:`float`)
Georg Brandl116aa622007-08-15 14:28:22 +0000244 .. index::
245 object: floating point
246 pair: floating point; number
247 pair: C; language
248 pair: Java; language
249
250 These represent machine-level double precision floating point numbers. You are
251 at the mercy of the underlying machine architecture (and C or Java
252 implementation) for the accepted range and handling of overflow. Python does not
253 support single-precision floating point numbers; the savings in processor and
Terry Jan Reedyb6271f22014-09-30 19:07:49 -0400254 memory usage that are usually the reason for using these are dwarfed by the
Georg Brandl116aa622007-08-15 14:28:22 +0000255 overhead of using objects in Python, so there is no reason to complicate the
256 language with two kinds of floating point numbers.
257
Nick Coghlan3a5d7e32008-08-31 12:40:14 +0000258 :class:`numbers.Complex` (:class:`complex`)
Georg Brandl116aa622007-08-15 14:28:22 +0000259 .. index::
260 object: complex
261 pair: complex; number
262
263 These represent complex numbers as a pair of machine-level double precision
264 floating point numbers. The same caveats apply as for floating point numbers.
265 The real and imaginary parts of a complex number ``z`` can be retrieved through
266 the read-only attributes ``z.real`` and ``z.imag``.
267
Georg Brandl116aa622007-08-15 14:28:22 +0000268Sequences
269 .. index::
270 builtin: len
271 object: sequence
272 single: index operation
273 single: item selection
274 single: subscription
275
276 These represent finite ordered sets indexed by non-negative numbers. The
277 built-in function :func:`len` returns the number of items of a sequence. When
278 the length of a sequence is *n*, the index set contains the numbers 0, 1,
279 ..., *n*-1. Item *i* of sequence *a* is selected by ``a[i]``.
280
281 .. index:: single: slicing
282
283 Sequences also support slicing: ``a[i:j]`` selects all items with index *k* such
284 that *i* ``<=`` *k* ``<`` *j*. When used as an expression, a slice is a
285 sequence of the same type. This implies that the index set is renumbered so
286 that it starts at 0.
287
Georg Brandl116aa622007-08-15 14:28:22 +0000288 Some sequences also support "extended slicing" with a third "step" parameter:
289 ``a[i:j:k]`` selects all items of *a* with index *x* where ``x = i + n*k``, *n*
290 ``>=`` ``0`` and *i* ``<=`` *x* ``<`` *j*.
291
292 Sequences are distinguished according to their mutability:
293
294 Immutable sequences
295 .. index::
296 object: immutable sequence
297 object: immutable
298
299 An object of an immutable sequence type cannot change once it is created. (If
300 the object contains references to other objects, these other objects may be
301 mutable and may be changed; however, the collection of objects directly
302 referenced by an immutable object cannot change.)
303
304 The following types are immutable sequences:
305
Chris Jerdonekbb4e9412012-11-28 01:38:40 -0800306 .. index::
307 single: string; immutable sequences
308
Georg Brandl116aa622007-08-15 14:28:22 +0000309 Strings
310 .. index::
311 builtin: chr
312 builtin: ord
Georg Brandl116aa622007-08-15 14:28:22 +0000313 single: character
314 single: integer
315 single: Unicode
316
Nick Coghlan14627862014-06-07 23:21:14 +1000317 A string is a sequence of values that represent Unicode code points.
318 All the code points in the range ``U+0000 - U+10FFFF`` can be
319 represented in a string. Python doesn't have a :c:type:`char` type;
320 instead, every code point in the string is represented as a string
321 object with length ``1``. The built-in function :func:`ord`
322 converts a code point from its string form to an integer in the
323 range ``0 - 10FFFF``; :func:`chr` converts an integer in the range
324 ``0 - 10FFFF`` to the corresponding length ``1`` string object.
Ezio Melottif4d76e62011-10-25 09:23:42 +0300325 :meth:`str.encode` can be used to convert a :class:`str` to
Nick Coghlan14627862014-06-07 23:21:14 +1000326 :class:`bytes` using the given text encoding, and
327 :meth:`bytes.decode` can be used to achieve the opposite.
Georg Brandl116aa622007-08-15 14:28:22 +0000328
329 Tuples
330 .. index::
331 object: tuple
332 pair: singleton; tuple
333 pair: empty; tuple
334
Georg Brandldcc56f82007-08-31 16:41:12 +0000335 The items of a tuple are arbitrary Python objects. Tuples of two or
336 more items are formed by comma-separated lists of expressions. A tuple
337 of one item (a 'singleton') can be formed by affixing a comma to an
338 expression (an expression by itself does not create a tuple, since
339 parentheses must be usable for grouping of expressions). An empty
340 tuple can be formed by an empty pair of parentheses.
Georg Brandl116aa622007-08-15 14:28:22 +0000341
Nick Coghlan3a5d7e32008-08-31 12:40:14 +0000342 Bytes
343 .. index:: bytes, byte
344
345 A bytes object is an immutable array. The items are 8-bit bytes,
346 represented by integers in the range 0 <= x < 256. Bytes literals
csabellac6db4812017-04-26 01:47:01 -0400347 (like ``b'abc'``) and the built-in :func:`bytes()` constructor
348 can be used to create bytes objects. Also, bytes objects can be
349 decoded to strings via the :meth:`~bytes.decode` method.
Nick Coghlan3a5d7e32008-08-31 12:40:14 +0000350
Georg Brandl116aa622007-08-15 14:28:22 +0000351 Mutable sequences
352 .. index::
353 object: mutable sequence
354 object: mutable
355 pair: assignment; statement
Georg Brandl116aa622007-08-15 14:28:22 +0000356 single: subscription
357 single: slicing
358
359 Mutable sequences can be changed after they are created. The subscription and
360 slicing notations can be used as the target of assignment and :keyword:`del`
361 (delete) statements.
362
Benjamin Petersonb58dda72009-01-18 22:27:04 +0000363 There are currently two intrinsic mutable sequence types:
Georg Brandl116aa622007-08-15 14:28:22 +0000364
365 Lists
366 .. index:: object: list
367
Georg Brandldcc56f82007-08-31 16:41:12 +0000368 The items of a list are arbitrary Python objects. Lists are formed by
369 placing a comma-separated list of expressions in square brackets. (Note
370 that there are no special cases needed to form lists of length 0 or 1.)
371
Nick Coghlan3a5d7e32008-08-31 12:40:14 +0000372 Byte Arrays
373 .. index:: bytearray
Georg Brandldcc56f82007-08-31 16:41:12 +0000374
Nick Coghlan3a5d7e32008-08-31 12:40:14 +0000375 A bytearray object is a mutable array. They are created by the built-in
csabellac6db4812017-04-26 01:47:01 -0400376 :func:`bytearray` constructor. Aside from being mutable
377 (and hence unhashable), byte arrays otherwise provide the same interface
378 and functionality as immutable :class:`bytes` objects.
Georg Brandl116aa622007-08-15 14:28:22 +0000379
380 .. index:: module: array
381
Georg Brandldcc56f82007-08-31 16:41:12 +0000382 The extension module :mod:`array` provides an additional example of a
Nick Coghlan3a5d7e32008-08-31 12:40:14 +0000383 mutable sequence type, as does the :mod:`collections` module.
Georg Brandl116aa622007-08-15 14:28:22 +0000384
Georg Brandl116aa622007-08-15 14:28:22 +0000385Set types
386 .. index::
387 builtin: len
388 object: set type
389
390 These represent unordered, finite sets of unique, immutable objects. As such,
391 they cannot be indexed by any subscript. However, they can be iterated over, and
392 the built-in function :func:`len` returns the number of items in a set. Common
393 uses for sets are fast membership testing, removing duplicates from a sequence,
394 and computing mathematical operations such as intersection, union, difference,
395 and symmetric difference.
396
397 For set elements, the same immutability rules apply as for dictionary keys. Note
398 that numeric types obey the normal rules for numeric comparison: if two numbers
399 compare equal (e.g., ``1`` and ``1.0``), only one of them can be contained in a
400 set.
401
402 There are currently two intrinsic set types:
403
404 Sets
405 .. index:: object: set
406
407 These represent a mutable set. They are created by the built-in :func:`set`
408 constructor and can be modified afterwards by several methods, such as
Serhiy Storchaka0d196ed2013-10-09 14:02:31 +0300409 :meth:`~set.add`.
Georg Brandl116aa622007-08-15 14:28:22 +0000410
411 Frozen sets
412 .. index:: object: frozenset
413
Guido van Rossum2cc30da2007-11-02 23:46:40 +0000414 These represent an immutable set. They are created by the built-in
415 :func:`frozenset` constructor. As a frozenset is immutable and
416 :term:`hashable`, it can be used again as an element of another set, or as
417 a dictionary key.
Georg Brandl116aa622007-08-15 14:28:22 +0000418
Georg Brandl116aa622007-08-15 14:28:22 +0000419Mappings
420 .. index::
421 builtin: len
422 single: subscription
423 object: mapping
424
425 These represent finite sets of objects indexed by arbitrary index sets. The
426 subscript notation ``a[k]`` selects the item indexed by ``k`` from the mapping
427 ``a``; this can be used in expressions and as the target of assignments or
428 :keyword:`del` statements. The built-in function :func:`len` returns the number
429 of items in a mapping.
430
431 There is currently a single intrinsic mapping type:
432
433 Dictionaries
434 .. index:: object: dictionary
435
436 These represent finite sets of objects indexed by nearly arbitrary values. The
437 only types of values not acceptable as keys are values containing lists or
438 dictionaries or other mutable types that are compared by value rather than by
439 object identity, the reason being that the efficient implementation of
440 dictionaries requires a key's hash value to remain constant. Numeric types used
441 for keys obey the normal rules for numeric comparison: if two numbers compare
442 equal (e.g., ``1`` and ``1.0``) then they can be used interchangeably to index
443 the same dictionary entry.
444
Lahfa Samy59c644e2020-03-26 15:54:04 +0100445 Dictionaries preserve insertion order, meaning that keys will be produced
446 in the same order they were added sequentially over the dictionary.
447 Replacing an existing key does not change the order, however removing a key
448 and re-inserting it will add it to the end instead of keeping its old place.
449
Georg Brandl116aa622007-08-15 14:28:22 +0000450 Dictionaries are mutable; they can be created by the ``{...}`` notation (see
451 section :ref:`dict`).
452
453 .. index::
Georg Brandl0a7ac7d2008-05-26 10:29:35 +0000454 module: dbm.ndbm
455 module: dbm.gnu
Georg Brandl116aa622007-08-15 14:28:22 +0000456
Benjamin Peterson9a46cab2008-09-08 02:49:30 +0000457 The extension modules :mod:`dbm.ndbm` and :mod:`dbm.gnu` provide
458 additional examples of mapping types, as does the :mod:`collections`
Nick Coghlan3a5d7e32008-08-31 12:40:14 +0000459 module.
Georg Brandl116aa622007-08-15 14:28:22 +0000460
Lahfa Samy59c644e2020-03-26 15:54:04 +0100461 .. versionchanged:: 3.7
462 Dictionaries did not preserve insertion order in versions of Python before 3.6.
463 In CPython 3.6, insertion order was preserved, but it was considered
464 an implementation detail at that time rather than a language guarantee.
465
Georg Brandl116aa622007-08-15 14:28:22 +0000466Callable types
467 .. index::
468 object: callable
469 pair: function; call
470 single: invocation
471 pair: function; argument
472
473 These are the types to which the function call operation (see section
474 :ref:`calls`) can be applied:
475
476 User-defined functions
477 .. index::
478 pair: user-defined; function
479 object: function
480 object: user-defined function
481
482 A user-defined function object is created by a function definition (see
483 section :ref:`function`). It should be called with an argument list
484 containing the same number of items as the function's formal parameter
485 list.
486
487 Special attributes:
488
Georg Brandl44ea77b2013-03-28 13:28:44 +0100489 .. tabularcolumns:: |l|L|l|
490
Martin Panterbae5d812016-06-18 03:57:31 +0000491 .. index::
492 single: __doc__ (function attribute)
493 single: __name__ (function attribute)
494 single: __module__ (function attribute)
495 single: __dict__ (function attribute)
496 single: __defaults__ (function attribute)
497 single: __closure__ (function attribute)
498 single: __code__ (function attribute)
499 single: __globals__ (function attribute)
500 single: __annotations__ (function attribute)
501 single: __kwdefaults__ (function attribute)
502 pair: global; namespace
503
Georg Brandl116aa622007-08-15 14:28:22 +0000504 +-------------------------+-------------------------------+-----------+
505 | Attribute | Meaning | |
506 +=========================+===============================+===========+
507 | :attr:`__doc__` | The function's documentation | Writable |
508 | | string, or ``None`` if | |
Ethan Furmanf87f5152014-10-17 22:25:22 -0700509 | | unavailable; not inherited by | |
Jules Lasne (jlasne)1fc5bf22019-03-27 11:10:33 +0100510 | | subclasses. | |
Georg Brandl116aa622007-08-15 14:28:22 +0000511 +-------------------------+-------------------------------+-----------+
Jules Lasne (jlasne)1fc5bf22019-03-27 11:10:33 +0100512 | :attr:`~definition.\ | The function's name. | Writable |
Martin Panterbae5d812016-06-18 03:57:31 +0000513 | __name__` | | |
Georg Brandl116aa622007-08-15 14:28:22 +0000514 +-------------------------+-------------------------------+-----------+
Martin Panterbae5d812016-06-18 03:57:31 +0000515 | :attr:`~definition.\ | The function's | Writable |
Jules Lasne (jlasne)1fc5bf22019-03-27 11:10:33 +0100516 | __qualname__` | :term:`qualified name`. | |
Antoine Pitrou86a36b52011-11-25 18:56:07 +0100517 | | | |
518 | | .. versionadded:: 3.3 | |
519 +-------------------------+-------------------------------+-----------+
Georg Brandl116aa622007-08-15 14:28:22 +0000520 | :attr:`__module__` | The name of the module the | Writable |
521 | | function was defined in, or | |
522 | | ``None`` if unavailable. | |
523 +-------------------------+-------------------------------+-----------+
524 | :attr:`__defaults__` | A tuple containing default | Writable |
525 | | argument values for those | |
526 | | arguments that have defaults, | |
527 | | or ``None`` if no arguments | |
Jules Lasne (jlasne)1fc5bf22019-03-27 11:10:33 +0100528 | | have a default value. | |
Georg Brandl116aa622007-08-15 14:28:22 +0000529 +-------------------------+-------------------------------+-----------+
530 | :attr:`__code__` | The code object representing | Writable |
531 | | the compiled function body. | |
532 +-------------------------+-------------------------------+-----------+
533 | :attr:`__globals__` | A reference to the dictionary | Read-only |
534 | | that holds the function's | |
535 | | global variables --- the | |
536 | | global namespace of the | |
537 | | module in which the function | |
538 | | was defined. | |
539 +-------------------------+-------------------------------+-----------+
Martin Panterbae5d812016-06-18 03:57:31 +0000540 | :attr:`~object.__dict__`| The namespace supporting | Writable |
Georg Brandl116aa622007-08-15 14:28:22 +0000541 | | arbitrary function | |
542 | | attributes. | |
543 +-------------------------+-------------------------------+-----------+
544 | :attr:`__closure__` | ``None`` or a tuple of cells | Read-only |
545 | | that contain bindings for the | |
546 | | function's free variables. | |
Lisa Roach64505a12017-06-08 04:43:26 -0700547 | | See below for information on | |
548 | | the ``cell_contents`` | |
549 | | attribute. | |
Georg Brandl116aa622007-08-15 14:28:22 +0000550 +-------------------------+-------------------------------+-----------+
551 | :attr:`__annotations__` | A dict containing annotations | Writable |
552 | | of parameters. The keys of | |
553 | | the dict are the parameter | |
Benjamin Peterson002033e2014-01-02 16:47:50 -0600554 | | names, and ``'return'`` for | |
Georg Brandl116aa622007-08-15 14:28:22 +0000555 | | the return annotation, if | |
556 | | provided. | |
557 +-------------------------+-------------------------------+-----------+
558 | :attr:`__kwdefaults__` | A dict containing defaults | Writable |
559 | | for keyword-only parameters. | |
560 +-------------------------+-------------------------------+-----------+
561
562 Most of the attributes labelled "Writable" check the type of the assigned value.
563
Georg Brandl116aa622007-08-15 14:28:22 +0000564 Function objects also support getting and setting arbitrary attributes, which
565 can be used, for example, to attach metadata to functions. Regular attribute
566 dot-notation is used to get and set such attributes. *Note that the current
567 implementation only supports function attributes on user-defined functions.
568 Function attributes on built-in functions may be supported in the future.*
569
Lisa Roach64505a12017-06-08 04:43:26 -0700570 A cell object has the attribute ``cell_contents``. This can be used to get
571 the value of the cell, as well as set the value.
572
Georg Brandl116aa622007-08-15 14:28:22 +0000573 Additional information about a function's definition can be retrieved from its
Pierre Glaserdf8d2cd2019-02-07 20:36:48 +0100574 code object; see the description of internal types below. The
575 :data:`cell <types.CellType>` type can be accessed in the :mod:`types`
576 module.
Georg Brandl116aa622007-08-15 14:28:22 +0000577
Georg Brandl2e0b7552007-11-27 12:43:08 +0000578 Instance methods
Georg Brandl116aa622007-08-15 14:28:22 +0000579 .. index::
580 object: method
581 object: user-defined method
582 pair: user-defined; method
583
Georg Brandl2e0b7552007-11-27 12:43:08 +0000584 An instance method object combines a class, a class instance and any
585 callable object (normally a user-defined function).
586
587 .. index::
588 single: __func__ (method attribute)
589 single: __self__ (method attribute)
590 single: __doc__ (method attribute)
591 single: __name__ (method attribute)
592 single: __module__ (method attribute)
Georg Brandl116aa622007-08-15 14:28:22 +0000593
Christian Heimesff737952007-11-27 10:40:20 +0000594 Special read-only attributes: :attr:`__self__` is the class instance object,
595 :attr:`__func__` is the function object; :attr:`__doc__` is the method's
Martin Panterbae5d812016-06-18 03:57:31 +0000596 documentation (same as ``__func__.__doc__``); :attr:`~definition.__name__` is the
Christian Heimesff737952007-11-27 10:40:20 +0000597 method name (same as ``__func__.__name__``); :attr:`__module__` is the
598 name of the module the method was defined in, or ``None`` if unavailable.
Georg Brandl116aa622007-08-15 14:28:22 +0000599
Georg Brandl116aa622007-08-15 14:28:22 +0000600 Methods also support accessing (but not setting) the arbitrary function
601 attributes on the underlying function object.
602
Georg Brandl2e0b7552007-11-27 12:43:08 +0000603 User-defined method objects may be created when getting an attribute of a
604 class (perhaps via an instance of that class), if that attribute is a
605 user-defined function object or a class method object.
Nick Coghlan3a5d7e32008-08-31 12:40:14 +0000606
Georg Brandl2e0b7552007-11-27 12:43:08 +0000607 When an instance method object is created by retrieving a user-defined
608 function object from a class via one of its instances, its
609 :attr:`__self__` attribute is the instance, and the method object is said
610 to be bound. The new method's :attr:`__func__` attribute is the original
611 function object.
Georg Brandl116aa622007-08-15 14:28:22 +0000612
Georg Brandl2e0b7552007-11-27 12:43:08 +0000613 When an instance method object is created by retrieving a class method
614 object from a class or instance, its :attr:`__self__` attribute is the
615 class itself, and its :attr:`__func__` attribute is the function object
616 underlying the class method.
Georg Brandl116aa622007-08-15 14:28:22 +0000617
Georg Brandl2e0b7552007-11-27 12:43:08 +0000618 When an instance method object is called, the underlying function
619 (:attr:`__func__`) is called, inserting the class instance
620 (:attr:`__self__`) in front of the argument list. For instance, when
621 :class:`C` is a class which contains a definition for a function
622 :meth:`f`, and ``x`` is an instance of :class:`C`, calling ``x.f(1)`` is
623 equivalent to calling ``C.f(x, 1)``.
Georg Brandl116aa622007-08-15 14:28:22 +0000624
Georg Brandl2e0b7552007-11-27 12:43:08 +0000625 When an instance method object is derived from a class method object, the
626 "class instance" stored in :attr:`__self__` will actually be the class
627 itself, so that calling either ``x.f(1)`` or ``C.f(1)`` is equivalent to
628 calling ``f(C,1)`` where ``f`` is the underlying function.
Georg Brandl116aa622007-08-15 14:28:22 +0000629
Georg Brandl2e0b7552007-11-27 12:43:08 +0000630 Note that the transformation from function object to instance method
631 object happens each time the attribute is retrieved from the instance. In
632 some cases, a fruitful optimization is to assign the attribute to a local
633 variable and call that local variable. Also notice that this
634 transformation only happens for user-defined functions; other callable
635 objects (and all non-callable objects) are retrieved without
636 transformation. It is also important to note that user-defined functions
637 which are attributes of a class instance are not converted to bound
638 methods; this *only* happens when the function is an attribute of the
639 class.
Georg Brandl116aa622007-08-15 14:28:22 +0000640
641 Generator functions
642 .. index::
643 single: generator; function
644 single: generator; iterator
645
646 A function or method which uses the :keyword:`yield` statement (see section
Nick Coghlan3a5d7e32008-08-31 12:40:14 +0000647 :ref:`yield`) is called a :dfn:`generator function`. Such a function, when
648 called, always returns an iterator object which can be used to execute the
Serhiy Storchaka0d196ed2013-10-09 14:02:31 +0300649 body of the function: calling the iterator's :meth:`iterator.__next__`
Ezio Melotti7fa82222012-10-12 13:42:08 +0300650 method will cause the function to execute until it provides a value
Serhiy Storchaka2b57c432018-12-19 08:09:46 +0200651 using the :keyword:`!yield` statement. When the function executes a
Georg Brandl116aa622007-08-15 14:28:22 +0000652 :keyword:`return` statement or falls off the end, a :exc:`StopIteration`
653 exception is raised and the iterator will have reached the end of the set of
654 values to be returned.
655
Yury Selivanovf3e40fa2015-05-21 11:50:30 -0400656 Coroutine functions
657 .. index::
658 single: coroutine; function
659
660 A function or method which is defined using :keyword:`async def` is called
661 a :dfn:`coroutine function`. Such a function, when called, returns a
662 :term:`coroutine` object. It may contain :keyword:`await` expressions,
663 as well as :keyword:`async with` and :keyword:`async for` statements. See
Yury Selivanov66f88282015-06-24 11:04:15 -0400664 also the :ref:`coroutine-objects` section.
Yury Selivanovf3e40fa2015-05-21 11:50:30 -0400665
Yury Selivanov03660042016-12-15 17:36:05 -0500666 Asynchronous generator functions
667 .. index::
668 single: asynchronous generator; function
669 single: asynchronous generator; asynchronous iterator
670
671 A function or method which is defined using :keyword:`async def` and
672 which uses the :keyword:`yield` statement is called a
673 :dfn:`asynchronous generator function`. Such a function, when called,
674 returns an asynchronous iterator object which can be used in an
675 :keyword:`async for` statement to execute the body of the function.
676
677 Calling the asynchronous iterator's :meth:`aiterator.__anext__` method
678 will return an :term:`awaitable` which when awaited
679 will execute until it provides a value using the :keyword:`yield`
680 expression. When the function executes an empty :keyword:`return`
681 statement or falls off the end, a :exc:`StopAsyncIteration` exception
682 is raised and the asynchronous iterator will have reached the end of
683 the set of values to be yielded.
684
Georg Brandl116aa622007-08-15 14:28:22 +0000685 Built-in functions
686 .. index::
687 object: built-in function
688 object: function
689 pair: C; language
690
691 A built-in function object is a wrapper around a C function. Examples of
692 built-in functions are :func:`len` and :func:`math.sin` (:mod:`math` is a
693 standard built-in module). The number and type of the arguments are
694 determined by the C function. Special read-only attributes:
695 :attr:`__doc__` is the function's documentation string, or ``None`` if
Martin Panterbae5d812016-06-18 03:57:31 +0000696 unavailable; :attr:`~definition.__name__` is the function's name; :attr:`__self__` is
Georg Brandl116aa622007-08-15 14:28:22 +0000697 set to ``None`` (but see the next item); :attr:`__module__` is the name of
698 the module the function was defined in or ``None`` if unavailable.
699
700 Built-in methods
701 .. index::
702 object: built-in method
703 object: method
704 pair: built-in; method
705
706 This is really a different disguise of a built-in function, this time containing
707 an object passed to the C function as an implicit extra argument. An example of
708 a built-in method is ``alist.append()``, assuming *alist* is a list object. In
709 this case, the special read-only attribute :attr:`__self__` is set to the object
Éric Araujoc9562f32010-12-26 02:18:49 +0000710 denoted by *alist*.
Georg Brandl116aa622007-08-15 14:28:22 +0000711
Georg Brandl85eb8c12007-08-31 16:33:38 +0000712 Classes
713 Classes are callable. These objects normally act as factories for new
714 instances of themselves, but variations are possible for class types that
715 override :meth:`__new__`. The arguments of the call are passed to
716 :meth:`__new__` and, in the typical case, to :meth:`__init__` to
717 initialize the new instance.
Georg Brandl116aa622007-08-15 14:28:22 +0000718
Georg Brandl85eb8c12007-08-31 16:33:38 +0000719 Class Instances
720 Instances of arbitrary classes can be made callable by defining a
721 :meth:`__call__` method in their class.
Georg Brandl116aa622007-08-15 14:28:22 +0000722
Georg Brandl116aa622007-08-15 14:28:22 +0000723
724Modules
725 .. index::
726 statement: import
727 object: module
728
Barry Warsawd7d21942012-07-29 16:36:17 -0400729 Modules are a basic organizational unit of Python code, and are created by
Barry Warsawdadebab2012-07-31 16:03:09 -0400730 the :ref:`import system <importsystem>` as invoked either by the
Serhiy Storchaka2b57c432018-12-19 08:09:46 +0200731 :keyword:`import` statement, or by calling
Barry Warsawdadebab2012-07-31 16:03:09 -0400732 functions such as :func:`importlib.import_module` and built-in
733 :func:`__import__`. A module object has a namespace implemented by a
734 dictionary object (this is the dictionary referenced by the ``__globals__``
735 attribute of functions defined in the module). Attribute references are
736 translated to lookups in this dictionary, e.g., ``m.x`` is equivalent to
737 ``m.__dict__["x"]``. A module object does not contain the code object used
738 to initialize the module (since it isn't needed once the initialization is
739 done).
Georg Brandl116aa622007-08-15 14:28:22 +0000740
Barry Warsawd7d21942012-07-29 16:36:17 -0400741 Attribute assignment updates the module's namespace dictionary, e.g.,
742 ``m.x = 1`` is equivalent to ``m.__dict__["x"] = 1``.
Georg Brandl116aa622007-08-15 14:28:22 +0000743
Yury Selivanovf8cb8a12016-09-08 20:50:03 -0700744 .. index::
745 single: __name__ (module attribute)
746 single: __doc__ (module attribute)
747 single: __file__ (module attribute)
748 single: __annotations__ (module attribute)
749 pair: module; namespace
750
751 Predefined (writable) attributes: :attr:`__name__` is the module's name;
752 :attr:`__doc__` is the module's documentation string, or ``None`` if
753 unavailable; :attr:`__annotations__` (optional) is a dictionary containing
754 :term:`variable annotations <variable annotation>` collected during module
755 body execution; :attr:`__file__` is the pathname of the file from which the
756 module was loaded, if it was loaded from a file. The :attr:`__file__`
757 attribute may be missing for certain types of modules, such as C modules
758 that are statically linked into the interpreter; for extension modules
759 loaded dynamically from a shared library, it is the pathname of the shared
760 library file.
761
Georg Brandl116aa622007-08-15 14:28:22 +0000762 .. index:: single: __dict__ (module attribute)
763
Yury Selivanovf8cb8a12016-09-08 20:50:03 -0700764 Special read-only attribute: :attr:`~object.__dict__` is the module's
765 namespace as a dictionary object.
Georg Brandl116aa622007-08-15 14:28:22 +0000766
Benjamin Peterson5c4bfc42010-10-12 22:57:59 +0000767 .. impl-detail::
768
769 Because of the way CPython clears module dictionaries, the module
770 dictionary will be cleared when the module falls out of scope even if the
771 dictionary still has live references. To avoid this, copy the dictionary
772 or keep the module around while using its dictionary directly.
773
Georg Brandl85eb8c12007-08-31 16:33:38 +0000774Custom classes
Georg Brandl5dbb84a2009-09-02 20:31:26 +0000775 Custom class types are typically created by class definitions (see section
Nick Coghlan3a5d7e32008-08-31 12:40:14 +0000776 :ref:`class`). A class has a namespace implemented by a dictionary object.
777 Class attribute references are translated to lookups in this dictionary, e.g.,
778 ``C.x`` is translated to ``C.__dict__["x"]`` (although there are a number of
779 hooks which allow for other means of locating attributes). When the attribute
780 name is not found there, the attribute search continues in the base classes.
781 This search of the base classes uses the C3 method resolution order which
782 behaves correctly even in the presence of 'diamond' inheritance structures
783 where there are multiple inheritance paths leading back to a common ancestor.
784 Additional details on the C3 MRO used by Python can be found in the
785 documentation accompanying the 2.3 release at
Georg Brandle73778c2014-10-29 08:36:35 +0100786 https://www.python.org/download/releases/2.3/mro/.
Georg Brandl116aa622007-08-15 14:28:22 +0000787
Nick Coghlan3a5d7e32008-08-31 12:40:14 +0000788 .. XXX: Could we add that MRO doc as an appendix to the language ref?
Georg Brandl85eb8c12007-08-31 16:33:38 +0000789
Georg Brandl116aa622007-08-15 14:28:22 +0000790 .. index::
791 object: class
792 object: class instance
793 object: instance
794 pair: class object; call
795 single: container
796 object: dictionary
797 pair: class; attribute
798
799 When a class attribute reference (for class :class:`C`, say) would yield a
Georg Brandl2e0b7552007-11-27 12:43:08 +0000800 class method object, it is transformed into an instance method object whose
Zach Mitchell00818c82018-06-02 10:29:47 -0400801 :attr:`__self__` attribute is :class:`C`. When it would yield a static
Georg Brandl2e0b7552007-11-27 12:43:08 +0000802 method object, it is transformed into the object wrapped by the static method
803 object. See section :ref:`descriptors` for another way in which attributes
804 retrieved from a class may differ from those actually contained in its
Martin Panterbae5d812016-06-18 03:57:31 +0000805 :attr:`~object.__dict__`.
Georg Brandl116aa622007-08-15 14:28:22 +0000806
807 .. index:: triple: class; attribute; assignment
808
809 Class attribute assignments update the class's dictionary, never the dictionary
810 of a base class.
811
812 .. index:: pair: class object; call
813
814 A class object can be called (see above) to yield a class instance (see below).
815
816 .. index::
817 single: __name__ (class attribute)
818 single: __module__ (class attribute)
819 single: __dict__ (class attribute)
820 single: __bases__ (class attribute)
821 single: __doc__ (class attribute)
Yury Selivanovf8cb8a12016-09-08 20:50:03 -0700822 single: __annotations__ (class attribute)
Georg Brandl116aa622007-08-15 14:28:22 +0000823
Martin Panterbae5d812016-06-18 03:57:31 +0000824 Special attributes: :attr:`~definition.__name__` is the class name; :attr:`__module__` is
825 the module name in which the class was defined; :attr:`~object.__dict__` is the
Serhiy Storchaka0d196ed2013-10-09 14:02:31 +0300826 dictionary containing the class's namespace; :attr:`~class.__bases__` is a
Berker Peksag7b4e5512017-01-03 03:34:15 +0300827 tuple containing the base classes, in the order of their occurrence in the
Berker Peksag406c2522017-01-03 03:35:49 +0300828 base class list; :attr:`__doc__` is the class's documentation string,
829 or ``None`` if undefined; :attr:`__annotations__` (optional) is a dictionary
830 containing :term:`variable annotations <variable annotation>` collected during
Yury Selivanovf8cb8a12016-09-08 20:50:03 -0700831 class body execution.
Georg Brandl116aa622007-08-15 14:28:22 +0000832
833Class instances
834 .. index::
835 object: class instance
836 object: instance
837 pair: class; instance
838 pair: class instance; attribute
839
Georg Brandl2e0b7552007-11-27 12:43:08 +0000840 A class instance is created by calling a class object (see above). A class
841 instance has a namespace implemented as a dictionary which is the first place
842 in which attribute references are searched. When an attribute is not found
843 there, and the instance's class has an attribute by that name, the search
844 continues with the class attributes. If a class attribute is found that is a
845 user-defined function object, it is transformed into an instance method
846 object whose :attr:`__self__` attribute is the instance. Static method and
847 class method objects are also transformed; see above under "Classes". See
848 section :ref:`descriptors` for another way in which attributes of a class
849 retrieved via its instances may differ from the objects actually stored in
Martin Panterbae5d812016-06-18 03:57:31 +0000850 the class's :attr:`~object.__dict__`. If no class attribute is found, and the
Georg Brandl2e0b7552007-11-27 12:43:08 +0000851 object's class has a :meth:`__getattr__` method, that is called to satisfy
852 the lookup.
Georg Brandl116aa622007-08-15 14:28:22 +0000853
854 .. index:: triple: class instance; attribute; assignment
855
856 Attribute assignments and deletions update the instance's dictionary, never a
857 class's dictionary. If the class has a :meth:`__setattr__` or
858 :meth:`__delattr__` method, this is called instead of updating the instance
859 dictionary directly.
860
861 .. index::
862 object: numeric
863 object: sequence
864 object: mapping
865
866 Class instances can pretend to be numbers, sequences, or mappings if they have
867 methods with certain special names. See section :ref:`specialnames`.
868
869 .. index::
870 single: __dict__ (instance attribute)
871 single: __class__ (instance attribute)
872
Serhiy Storchaka0d196ed2013-10-09 14:02:31 +0300873 Special attributes: :attr:`~object.__dict__` is the attribute dictionary;
874 :attr:`~instance.__class__` is the instance's class.
Georg Brandl116aa622007-08-15 14:28:22 +0000875
Antoine Pitrou4adb2882010-01-04 18:50:53 +0000876I/O objects (also known as file objects)
Georg Brandl116aa622007-08-15 14:28:22 +0000877 .. index::
Georg Brandl116aa622007-08-15 14:28:22 +0000878 builtin: open
Antoine Pitrou4adb2882010-01-04 18:50:53 +0000879 module: io
Georg Brandl116aa622007-08-15 14:28:22 +0000880 single: popen() (in module os)
881 single: makefile() (socket method)
882 single: sys.stdin
883 single: sys.stdout
884 single: sys.stderr
885 single: stdio
886 single: stdin (in module sys)
887 single: stdout (in module sys)
888 single: stderr (in module sys)
889
Antoine Pitrou0b65b0f2010-09-15 09:58:26 +0000890 A :term:`file object` represents an open file. Various shortcuts are
891 available to create file objects: the :func:`open` built-in function, and
Serhiy Storchaka0d196ed2013-10-09 14:02:31 +0300892 also :func:`os.popen`, :func:`os.fdopen`, and the
893 :meth:`~socket.socket.makefile` method of socket objects (and perhaps by
894 other functions or methods provided by extension modules).
Antoine Pitrou4adb2882010-01-04 18:50:53 +0000895
896 The objects ``sys.stdin``, ``sys.stdout`` and ``sys.stderr`` are
897 initialized to file objects corresponding to the interpreter's standard
898 input, output and error streams; they are all open in text mode and
899 therefore follow the interface defined by the :class:`io.TextIOBase`
900 abstract class.
Georg Brandl116aa622007-08-15 14:28:22 +0000901
902Internal types
903 .. index::
904 single: internal type
905 single: types, internal
906
907 A few types used internally by the interpreter are exposed to the user. Their
908 definitions may change with future versions of the interpreter, but they are
909 mentioned here for completeness.
910
Tommy Beadlee9b84032016-06-02 19:26:51 -0400911 .. index:: bytecode, object; code, code object
Georg Brandl116aa622007-08-15 14:28:22 +0000912
Tommy Beadlee9b84032016-06-02 19:26:51 -0400913 Code objects
Georg Brandl9afde1c2007-11-01 20:32:30 +0000914 Code objects represent *byte-compiled* executable Python code, or :term:`bytecode`.
Georg Brandl116aa622007-08-15 14:28:22 +0000915 The difference between a code object and a function object is that the function
916 object contains an explicit reference to the function's globals (the module in
917 which it was defined), while a code object contains no context; also the default
918 argument values are stored in the function object, not in the code object
919 (because they represent values calculated at run-time). Unlike function
920 objects, code objects are immutable and contain no references (directly or
921 indirectly) to mutable objects.
922
Senthil Kumaran7cafd262010-10-02 03:16:04 +0000923 .. index::
924 single: co_argcount (code object attribute)
Pablo Galindoed222a72019-05-31 12:13:04 +0100925 single: co_posonlyargcount (code object attribute)
926 single: co_kwonlyargcount (code object attribute)
Senthil Kumaran7cafd262010-10-02 03:16:04 +0000927 single: co_code (code object attribute)
928 single: co_consts (code object attribute)
929 single: co_filename (code object attribute)
930 single: co_firstlineno (code object attribute)
931 single: co_flags (code object attribute)
932 single: co_lnotab (code object attribute)
933 single: co_name (code object attribute)
934 single: co_names (code object attribute)
935 single: co_nlocals (code object attribute)
936 single: co_stacksize (code object attribute)
937 single: co_varnames (code object attribute)
938 single: co_cellvars (code object attribute)
939 single: co_freevars (code object attribute)
940
Georg Brandl116aa622007-08-15 14:28:22 +0000941 Special read-only attributes: :attr:`co_name` gives the function name;
Pablo Galindocd74e662019-06-01 18:08:04 +0100942 :attr:`co_argcount` is the total number of positional arguments
943 (including positional-only arguments and arguments with default values);
944 :attr:`co_posonlyargcount` is the number of positional-only arguments
945 (including arguments with default values); :attr:`co_kwonlyargcount` is
946 the number of keyword-only arguments (including arguments with default
947 values); :attr:`co_nlocals` is the number of local variables used by the
948 function (including arguments); :attr:`co_varnames` is a tuple containing
949 the names of the local variables (starting with the argument names);
950 :attr:`co_cellvars` is a tuple containing the names of local variables
Pablo Galindoed222a72019-05-31 12:13:04 +0100951 that are referenced by nested functions; :attr:`co_freevars` is a tuple
Pablo Galindocd74e662019-06-01 18:08:04 +0100952 containing the names of free variables; :attr:`co_code` is a string
953 representing the sequence of bytecode instructions; :attr:`co_consts` is
954 a tuple containing the literals used by the bytecode; :attr:`co_names` is
955 a tuple containing the names used by the bytecode; :attr:`co_filename` is
956 the filename from which the code was compiled; :attr:`co_firstlineno` is
957 the first line number of the function; :attr:`co_lnotab` is a string
958 encoding the mapping from bytecode offsets to line numbers (for details
959 see the source code of the interpreter); :attr:`co_stacksize` is the
Batuhan TaÅŸkayad5872722019-12-16 01:02:47 +0300960 required stack size; :attr:`co_flags` is an integer encoding a number
961 of flags for the interpreter.
Georg Brandl116aa622007-08-15 14:28:22 +0000962
Georg Brandl116aa622007-08-15 14:28:22 +0000963 .. index:: object: generator
964
965 The following flag bits are defined for :attr:`co_flags`: bit ``0x04`` is set if
966 the function uses the ``*arguments`` syntax to accept an arbitrary number of
967 positional arguments; bit ``0x08`` is set if the function uses the
968 ``**keywords`` syntax to accept arbitrary keyword arguments; bit ``0x20`` is set
969 if the function is a generator.
970
971 Future feature declarations (``from __future__ import division``) also use bits
972 in :attr:`co_flags` to indicate whether a code object was compiled with a
973 particular feature enabled: bit ``0x2000`` is set if the function was compiled
974 with future division enabled; bits ``0x10`` and ``0x1000`` were used in earlier
975 versions of Python.
976
977 Other bits in :attr:`co_flags` are reserved for internal use.
978
979 .. index:: single: documentation string
980
981 If a code object represents a function, the first item in :attr:`co_consts` is
982 the documentation string of the function, or ``None`` if undefined.
983
Georg Brandla6053b42009-09-01 08:11:14 +0000984 .. _frame-objects:
985
Georg Brandl116aa622007-08-15 14:28:22 +0000986 Frame objects
987 .. index:: object: frame
988
989 Frame objects represent execution frames. They may occur in traceback objects
Nick Coghlanaec75322018-02-13 18:10:58 +1000990 (see below), and are also passed to registered trace functions.
Georg Brandl116aa622007-08-15 14:28:22 +0000991
992 .. index::
993 single: f_back (frame attribute)
994 single: f_code (frame attribute)
995 single: f_globals (frame attribute)
996 single: f_locals (frame attribute)
997 single: f_lasti (frame attribute)
998 single: f_builtins (frame attribute)
999
1000 Special read-only attributes: :attr:`f_back` is to the previous stack frame
1001 (towards the caller), or ``None`` if this is the bottom stack frame;
1002 :attr:`f_code` is the code object being executed in this frame; :attr:`f_locals`
1003 is the dictionary used to look up local variables; :attr:`f_globals` is used for
1004 global variables; :attr:`f_builtins` is used for built-in (intrinsic) names;
1005 :attr:`f_lasti` gives the precise instruction (this is an index into the
1006 bytecode string of the code object).
1007
1008 .. index::
1009 single: f_trace (frame attribute)
Nick Coghlan5a851672017-09-08 10:14:16 +10001010 single: f_trace_lines (frame attribute)
1011 single: f_trace_opcodes (frame attribute)
Georg Brandl116aa622007-08-15 14:28:22 +00001012 single: f_lineno (frame attribute)
1013
1014 Special writable attributes: :attr:`f_trace`, if not ``None``, is a function
Nick Coghlan5a851672017-09-08 10:14:16 +10001015 called for various events during code execution (this is used by the debugger).
1016 Normally an event is triggered for each new source line - this can be
1017 disabled by setting :attr:`f_trace_lines` to :const:`False`.
1018
1019 Implementations *may* allow per-opcode events to be requested by setting
1020 :attr:`f_trace_opcodes` to :const:`True`. Note that this may lead to
1021 undefined interpreter behaviour if exceptions raised by the trace
1022 function escape to the function being traced.
1023
Benjamin Petersoneec3d712008-06-11 15:59:43 +00001024 :attr:`f_lineno` is the current line number of the frame --- writing to this
1025 from within a trace function jumps to the given line (only for the bottom-most
1026 frame). A debugger can implement a Jump command (aka Set Next Statement)
1027 by writing to f_lineno.
Georg Brandl116aa622007-08-15 14:28:22 +00001028
Antoine Pitrou58720d62013-08-05 23:26:40 +02001029 Frame objects support one method:
1030
1031 .. method:: frame.clear()
1032
1033 This method clears all references to local variables held by the
1034 frame. Also, if the frame belonged to a generator, the generator
1035 is finalized. This helps break reference cycles involving frame
1036 objects (for example when catching an exception and storing its
1037 traceback for later use).
1038
1039 :exc:`RuntimeError` is raised if the frame is currently executing.
1040
1041 .. versionadded:: 3.4
1042
Nick Coghlanaec75322018-02-13 18:10:58 +10001043 .. _traceback-objects:
1044
Georg Brandl116aa622007-08-15 14:28:22 +00001045 Traceback objects
1046 .. index::
1047 object: traceback
1048 pair: stack; trace
1049 pair: exception; handler
1050 pair: execution; stack
1051 single: exc_info (in module sys)
Georg Brandl116aa622007-08-15 14:28:22 +00001052 single: last_traceback (in module sys)
1053 single: sys.exc_info
1054 single: sys.last_traceback
1055
1056 Traceback objects represent a stack trace of an exception. A traceback object
Nick Coghlanaec75322018-02-13 18:10:58 +10001057 is implicitly created when an exception occurs, and may also be explicitly
1058 created by calling :class:`types.TracebackType`.
1059
1060 For implicitly created tracebacks, when the search for an exception handler
Georg Brandl116aa622007-08-15 14:28:22 +00001061 unwinds the execution stack, at each unwound level a traceback object is
1062 inserted in front of the current traceback. When an exception handler is
1063 entered, the stack trace is made available to the program. (See section
1064 :ref:`try`.) It is accessible as the third item of the
Nick Coghlanaec75322018-02-13 18:10:58 +10001065 tuple returned by ``sys.exc_info()``, and as the ``__traceback__`` attribute
1066 of the caught exception.
1067
1068 When the program contains no suitable
Georg Brandl116aa622007-08-15 14:28:22 +00001069 handler, the stack trace is written (nicely formatted) to the standard error
1070 stream; if the interpreter is interactive, it is also made available to the user
1071 as ``sys.last_traceback``.
1072
Nick Coghlanaec75322018-02-13 18:10:58 +10001073 For explicitly created tracebacks, it is up to the creator of the traceback
1074 to determine how the ``tb_next`` attributes should be linked to form a
1075 full stack trace.
1076
Georg Brandl116aa622007-08-15 14:28:22 +00001077 .. index::
Georg Brandl116aa622007-08-15 14:28:22 +00001078 single: tb_frame (traceback attribute)
1079 single: tb_lineno (traceback attribute)
1080 single: tb_lasti (traceback attribute)
1081 statement: try
1082
Nick Coghlanaec75322018-02-13 18:10:58 +10001083 Special read-only attributes:
1084 :attr:`tb_frame` points to the execution frame of the current level;
1085 :attr:`tb_lineno` gives the line number where the exception occurred;
1086 :attr:`tb_lasti` indicates the precise instruction.
1087 The line number and last instruction in the traceback may differ from the
1088 line number of its frame object if the exception occurred in a
1089 :keyword:`try` statement with no matching except clause or with a
1090 finally clause.
1091
1092 .. index::
1093 single: tb_next (traceback attribute)
1094
1095 Special writable attribute: :attr:`tb_next` is the next level in the stack
1096 trace (towards the frame where the exception occurred), or ``None`` if
1097 there is no next level.
1098
Berker Peksag3c1b5902018-08-06 23:52:49 +03001099 .. versionchanged:: 3.7
1100 Traceback objects can now be explicitly instantiated from Python code,
1101 and the ``tb_next`` attribute of existing instances can be updated.
Georg Brandl116aa622007-08-15 14:28:22 +00001102
1103 Slice objects
1104 .. index:: builtin: slice
1105
Georg Brandlcb8ecb12007-09-04 06:35:14 +00001106 Slice objects are used to represent slices for :meth:`__getitem__`
1107 methods. They are also created by the built-in :func:`slice` function.
Georg Brandl116aa622007-08-15 14:28:22 +00001108
1109 .. index::
1110 single: start (slice object attribute)
1111 single: stop (slice object attribute)
1112 single: step (slice object attribute)
1113
Serhiy Storchaka0d196ed2013-10-09 14:02:31 +03001114 Special read-only attributes: :attr:`~slice.start` is the lower bound;
1115 :attr:`~slice.stop` is the upper bound; :attr:`~slice.step` is the step
1116 value; each is ``None`` if omitted. These attributes can have any type.
Georg Brandl116aa622007-08-15 14:28:22 +00001117
1118 Slice objects support one method:
1119
Georg Brandl116aa622007-08-15 14:28:22 +00001120 .. method:: slice.indices(self, length)
1121
Georg Brandlcb8ecb12007-09-04 06:35:14 +00001122 This method takes a single integer argument *length* and computes
1123 information about the slice that the slice object would describe if
1124 applied to a sequence of *length* items. It returns a tuple of three
1125 integers; respectively these are the *start* and *stop* indices and the
1126 *step* or stride length of the slice. Missing or out-of-bounds indices
1127 are handled in a manner consistent with regular slices.
Georg Brandl116aa622007-08-15 14:28:22 +00001128
Georg Brandl116aa622007-08-15 14:28:22 +00001129 Static method objects
1130 Static method objects provide a way of defeating the transformation of function
1131 objects to method objects described above. A static method object is a wrapper
1132 around any other object, usually a user-defined method object. When a static
1133 method object is retrieved from a class or a class instance, the object actually
1134 returned is the wrapped object, which is not subject to any further
1135 transformation. Static method objects are not themselves callable, although the
1136 objects they wrap usually are. Static method objects are created by the built-in
1137 :func:`staticmethod` constructor.
1138
1139 Class method objects
1140 A class method object, like a static method object, is a wrapper around another
1141 object that alters the way in which that object is retrieved from classes and
1142 class instances. The behaviour of class method objects upon such retrieval is
1143 described above, under "User-defined methods". Class method objects are created
1144 by the built-in :func:`classmethod` constructor.
1145
Georg Brandl116aa622007-08-15 14:28:22 +00001146
Georg Brandl116aa622007-08-15 14:28:22 +00001147.. _specialnames:
1148
1149Special method names
1150====================
1151
1152.. index::
1153 pair: operator; overloading
1154 single: __getitem__() (mapping object method)
1155
1156A class can implement certain operations that are invoked by special syntax
1157(such as arithmetic operations or subscripting and slicing) by defining methods
1158with special names. This is Python's approach to :dfn:`operator overloading`,
1159allowing classes to define their own behavior with respect to language
1160operators. For instance, if a class defines a method named :meth:`__getitem__`,
Nick Coghlan3a5d7e32008-08-31 12:40:14 +00001161and ``x`` is an instance of this class, then ``x[i]`` is roughly equivalent
1162to ``type(x).__getitem__(x, i)``. Except where mentioned, attempts to execute an
1163operation raise an exception when no appropriate method is defined (typically
1164:exc:`AttributeError` or :exc:`TypeError`).
Georg Brandl65ea9bd2007-09-05 13:36:27 +00001165
Guido van Rossum97c1adf2016-08-18 09:22:23 -07001166Setting a special method to ``None`` indicates that the corresponding
1167operation is not available. For example, if a class sets
1168:meth:`__iter__` to ``None``, the class is not iterable, so calling
1169:func:`iter` on its instances will raise a :exc:`TypeError` (without
1170falling back to :meth:`__getitem__`). [#]_
1171
Georg Brandl116aa622007-08-15 14:28:22 +00001172When implementing a class that emulates any built-in type, it is important that
1173the emulation only be implemented to the degree that it makes sense for the
1174object being modelled. For example, some sequences may work well with retrieval
1175of individual elements, but extracting a slice may not make sense. (One example
Serhiy Storchaka0d196ed2013-10-09 14:02:31 +03001176of this is the :class:`~xml.dom.NodeList` interface in the W3C's Document
1177Object Model.)
Georg Brandl116aa622007-08-15 14:28:22 +00001178
1179
1180.. _customization:
1181
1182Basic customization
1183-------------------
1184
Georg Brandl116aa622007-08-15 14:28:22 +00001185.. method:: object.__new__(cls[, ...])
1186
Georg Brandlaf265f42008-12-07 15:06:20 +00001187 .. index:: pair: subclassing; immutable types
1188
Georg Brandl116aa622007-08-15 14:28:22 +00001189 Called to create a new instance of class *cls*. :meth:`__new__` is a static
1190 method (special-cased so you need not declare it as such) that takes the class
1191 of which an instance was requested as its first argument. The remaining
1192 arguments are those passed to the object constructor expression (the call to the
1193 class). The return value of :meth:`__new__` should be the new object instance
1194 (usually an instance of *cls*).
1195
1196 Typical implementations create a new instance of the class by invoking the
csabella12b1c182017-05-14 20:42:00 -07001197 superclass's :meth:`__new__` method using ``super().__new__(cls[, ...])``
1198 with appropriate arguments and then modifying the newly-created instance
1199 as necessary before returning it.
Georg Brandl116aa622007-08-15 14:28:22 +00001200
Joannah Nanjekye6b16d932019-08-26 03:53:11 -03001201 If :meth:`__new__` is invoked during object construction and it returns an
1202 instance or subclass of *cls*, then the new instance’s :meth:`__init__` method
1203 will be invoked like ``__init__(self[, ...])``, where *self* is the new instance
1204 and the remaining arguments are the same as were passed to the object constructor.
Georg Brandl116aa622007-08-15 14:28:22 +00001205
1206 If :meth:`__new__` does not return an instance of *cls*, then the new instance's
1207 :meth:`__init__` method will not be invoked.
1208
1209 :meth:`__new__` is intended mainly to allow subclasses of immutable types (like
Christian Heimes790c8232008-01-07 21:14:23 +00001210 int, str, or tuple) to customize instance creation. It is also commonly
1211 overridden in custom metaclasses in order to customize class creation.
Georg Brandl116aa622007-08-15 14:28:22 +00001212
1213
1214.. method:: object.__init__(self[, ...])
1215
1216 .. index:: pair: class; constructor
1217
Ethan Furman119479f2015-01-14 21:56:10 -08001218 Called after the instance has been created (by :meth:`__new__`), but before
1219 it is returned to the caller. The arguments are those passed to the
1220 class constructor expression. If a base class has an :meth:`__init__`
1221 method, the derived class's :meth:`__init__` method, if any, must explicitly
1222 call it to ensure proper initialization of the base class part of the
csabella12b1c182017-05-14 20:42:00 -07001223 instance; for example: ``super().__init__([args...])``.
Ethan Furman119479f2015-01-14 21:56:10 -08001224
1225 Because :meth:`__new__` and :meth:`__init__` work together in constructing
Raymond Hettinger7ea386e2016-08-25 21:11:50 -07001226 objects (:meth:`__new__` to create it, and :meth:`__init__` to customize it),
Ethan Furman119479f2015-01-14 21:56:10 -08001227 no non-``None`` value may be returned by :meth:`__init__`; doing so will
1228 cause a :exc:`TypeError` to be raised at runtime.
Georg Brandl116aa622007-08-15 14:28:22 +00001229
1230
1231.. method:: object.__del__(self)
1232
1233 .. index::
1234 single: destructor
Antoine Pitrou4b965932017-12-19 19:48:45 +01001235 single: finalizer
Georg Brandl116aa622007-08-15 14:28:22 +00001236 statement: del
1237
1238 Called when the instance is about to be destroyed. This is also called a
Antoine Pitrou4b965932017-12-19 19:48:45 +01001239 finalizer or (improperly) a destructor. If a base class has a
1240 :meth:`__del__` method, the derived class's :meth:`__del__` method,
1241 if any, must explicitly call it to ensure proper deletion of the base
1242 class part of the instance.
1243
1244 It is possible (though not recommended!) for the :meth:`__del__` method
1245 to postpone destruction of the instance by creating a new reference to
1246 it. This is called object *resurrection*. It is implementation-dependent
1247 whether :meth:`__del__` is called a second time when a resurrected object
1248 is about to be destroyed; the current :term:`CPython` implementation
1249 only calls it once.
1250
1251 It is not guaranteed that :meth:`__del__` methods are called for objects
1252 that still exist when the interpreter exits.
Georg Brandl116aa622007-08-15 14:28:22 +00001253
1254 .. note::
1255
1256 ``del x`` doesn't directly call ``x.__del__()`` --- the former decrements
1257 the reference count for ``x`` by one, and the latter is only called when
Antoine Pitrou4b965932017-12-19 19:48:45 +01001258 ``x``'s reference count reaches zero.
1259
1260 .. impl-detail::
1261 It is possible for a reference cycle to prevent the reference count
1262 of an object from going to zero. In this case, the cycle will be
1263 later detected and deleted by the :term:`cyclic garbage collector
1264 <garbage collection>`. A common cause of reference cycles is when
1265 an exception has been caught in a local variable. The frame's
1266 locals then reference the exception, which references its own
1267 traceback, which references the locals of all frames caught in the
1268 traceback.
1269
1270 .. seealso::
1271 Documentation for the :mod:`gc` module.
Georg Brandl116aa622007-08-15 14:28:22 +00001272
1273 .. warning::
1274
1275 Due to the precarious circumstances under which :meth:`__del__` methods are
1276 invoked, exceptions that occur during their execution are ignored, and a warning
Antoine Pitrou4b965932017-12-19 19:48:45 +01001277 is printed to ``sys.stderr`` instead. In particular:
Georg Brandl116aa622007-08-15 14:28:22 +00001278
Antoine Pitrou4b965932017-12-19 19:48:45 +01001279 * :meth:`__del__` can be invoked when arbitrary code is being executed,
1280 including from any arbitrary thread. If :meth:`__del__` needs to take
1281 a lock or invoke any other blocking resource, it may deadlock as
1282 the resource may already be taken by the code that gets interrupted
1283 to execute :meth:`__del__`.
Chris Jerdonek5fae0e52012-11-20 17:45:51 -08001284
Antoine Pitrou4b965932017-12-19 19:48:45 +01001285 * :meth:`__del__` can be executed during interpreter shutdown. As a
1286 consequence, the global variables it needs to access (including other
1287 modules) may already have been deleted or set to ``None``. Python
1288 guarantees that globals whose name begins with a single underscore
1289 are deleted from their module before other globals are deleted; if
1290 no other references to such globals exist, this may help in assuring
1291 that imported modules are still available at the time when the
1292 :meth:`__del__` method is called.
1293
1294
1295 .. index::
1296 single: repr() (built-in function); __repr__() (object method)
Georg Brandl116aa622007-08-15 14:28:22 +00001297
1298.. method:: object.__repr__(self)
1299
Benjamin Peterson1c9313f2008-10-12 12:51:12 +00001300 Called by the :func:`repr` built-in function to compute the "official" string
1301 representation of an object. If at all possible, this should look like a
1302 valid Python expression that could be used to recreate an object with the
1303 same value (given an appropriate environment). If this is not possible, a
1304 string of the form ``<...some useful description...>`` should be returned.
1305 The return value must be a string object. If a class defines :meth:`__repr__`
1306 but not :meth:`__str__`, then :meth:`__repr__` is also used when an
1307 "informal" string representation of instances of that class is required.
Georg Brandl116aa622007-08-15 14:28:22 +00001308
Georg Brandl116aa622007-08-15 14:28:22 +00001309 This is typically used for debugging, so it is important that the representation
1310 is information-rich and unambiguous.
1311
Chris Jerdonek5fae0e52012-11-20 17:45:51 -08001312 .. index::
1313 single: string; __str__() (object method)
1314 single: format() (built-in function); __str__() (object method)
1315 single: print() (built-in function); __str__() (object method)
1316
Georg Brandl116aa622007-08-15 14:28:22 +00001317
1318.. method:: object.__str__(self)
1319
Chris Jerdonek5fae0e52012-11-20 17:45:51 -08001320 Called by :func:`str(object) <str>` and the built-in functions
1321 :func:`format` and :func:`print` to compute the "informal" or nicely
1322 printable string representation of an object. The return value must be a
1323 :ref:`string <textseq>` object.
Georg Brandl116aa622007-08-15 14:28:22 +00001324
Chris Jerdonek5fae0e52012-11-20 17:45:51 -08001325 This method differs from :meth:`object.__repr__` in that there is no
1326 expectation that :meth:`__str__` return a valid Python expression: a more
1327 convenient or concise representation can be used.
1328
1329 The default implementation defined by the built-in type :class:`object`
1330 calls :meth:`object.__repr__`.
Georg Brandl116aa622007-08-15 14:28:22 +00001331
Georg Brandldcc56f82007-08-31 16:41:12 +00001332 .. XXX what about subclasses of string?
1333
Georg Brandl116aa622007-08-15 14:28:22 +00001334
Benjamin Peterson1fafc1a2011-10-25 00:03:51 -04001335.. method:: object.__bytes__(self)
1336
1337 .. index:: builtin: bytes
1338
csabellac6db4812017-04-26 01:47:01 -04001339 Called by :ref:`bytes <func-bytes>` to compute a byte-string representation
1340 of an object. This should return a :class:`bytes` object.
Benjamin Peterson1fafc1a2011-10-25 00:03:51 -04001341
Chris Jerdonekbb4e9412012-11-28 01:38:40 -08001342 .. index::
1343 single: string; __format__() (object method)
1344 pair: string; conversion
1345 builtin: print
1346
Benjamin Peterson1fafc1a2011-10-25 00:03:51 -04001347
Georg Brandl4b491312007-08-31 09:22:56 +00001348.. method:: object.__format__(self, format_spec)
1349
Martin Panterbc1ee462016-02-13 00:41:37 +00001350 Called by the :func:`format` built-in function,
1351 and by extension, evaluation of :ref:`formatted string literals
1352 <f-strings>` and the :meth:`str.format` method, to produce a "formatted"
Catherine Alvarado5e98f052019-05-04 17:54:35 -04001353 string representation of an object. The *format_spec* argument is
Georg Brandl4b491312007-08-31 09:22:56 +00001354 a string that contains a description of the formatting options desired.
Catherine Alvarado5e98f052019-05-04 17:54:35 -04001355 The interpretation of the *format_spec* argument is up to the type
Georg Brandl4b491312007-08-31 09:22:56 +00001356 implementing :meth:`__format__`, however most classes will either
1357 delegate formatting to one of the built-in types, or use a similar
1358 formatting option syntax.
Georg Brandl48310cd2009-01-03 21:18:54 +00001359
Georg Brandl4b491312007-08-31 09:22:56 +00001360 See :ref:`formatspec` for a description of the standard formatting syntax.
1361
1362 The return value must be a string object.
1363
R David Murrayd630e792014-02-11 18:34:22 -05001364 .. versionchanged:: 3.4
1365 The __format__ method of ``object`` itself raises a :exc:`TypeError`
1366 if passed any non-empty string.
1367
Serhiy Storchaka7e19dbc2017-05-13 12:40:52 +03001368 .. versionchanged:: 3.7
1369 ``object.__format__(x, '')`` is now equivalent to ``str(x)`` rather
Heshy Roskesef337122020-04-25 21:57:09 -04001370 than ``format(str(x), '')``.
Serhiy Storchaka7e19dbc2017-05-13 12:40:52 +03001371
Georg Brandl4b491312007-08-31 09:22:56 +00001372
Georg Brandl33413cb2009-03-31 19:06:37 +00001373.. _richcmpfuncs:
Georg Brandl116aa622007-08-15 14:28:22 +00001374.. method:: object.__lt__(self, other)
1375 object.__le__(self, other)
1376 object.__eq__(self, other)
1377 object.__ne__(self, other)
1378 object.__gt__(self, other)
1379 object.__ge__(self, other)
1380
Guido van Rossum2cc30da2007-11-02 23:46:40 +00001381 .. index::
1382 single: comparisons
1383
Georg Brandl05f5ab72008-09-24 09:11:47 +00001384 These are the so-called "rich comparison" methods. The correspondence between
Georg Brandl116aa622007-08-15 14:28:22 +00001385 operator symbols and method names is as follows: ``x<y`` calls ``x.__lt__(y)``,
1386 ``x<=y`` calls ``x.__le__(y)``, ``x==y`` calls ``x.__eq__(y)``, ``x!=y`` calls
1387 ``x.__ne__(y)``, ``x>y`` calls ``x.__gt__(y)``, and ``x>=y`` calls
1388 ``x.__ge__(y)``.
1389
1390 A rich comparison method may return the singleton ``NotImplemented`` if it does
1391 not implement the operation for a given pair of arguments. By convention,
1392 ``False`` and ``True`` are returned for a successful comparison. However, these
1393 methods can return any value, so if the comparison operator is used in a Boolean
1394 context (e.g., in the condition of an ``if`` statement), Python will call
1395 :func:`bool` on the value to determine if the result is true or false.
1396
Brett Cannon3c69f0c2020-10-21 16:24:38 -07001397 By default, ``object`` implements :meth:`__eq__` by using ``is``, returning
1398 ``NotImplemented`` in the case of a false comparison:
1399 ``True if x is y else NotImplemented``. For :meth:`__ne__`, by default it
1400 delegates to :meth:`__eq__` and inverts the result unless it is
1401 ``NotImplemented``. There are no other implied relationships among the
1402 comparison operators or default implementations; for example, the truth of
1403 ``(x<y or x==y)`` does not imply ``x<=y``. To automatically generate ordering
1404 operations from a single root operation, see :func:`functools.total_ordering`.
Robert Collinsd84b29f2015-08-07 10:22:54 +12001405
1406 See the paragraph on :meth:`__hash__` for
Guido van Rossum2cc30da2007-11-02 23:46:40 +00001407 some important notes on creating :term:`hashable` objects which support
1408 custom comparison operations and are usable as dictionary keys.
Georg Brandl116aa622007-08-15 14:28:22 +00001409
Guido van Rossum2cc30da2007-11-02 23:46:40 +00001410 There are no swapped-argument versions of these methods (to be used when the
1411 left argument does not support the operation but the right argument does);
1412 rather, :meth:`__lt__` and :meth:`__gt__` are each other's reflection,
Georg Brandl116aa622007-08-15 14:28:22 +00001413 :meth:`__le__` and :meth:`__ge__` are each other's reflection, and
1414 :meth:`__eq__` and :meth:`__ne__` are their own reflection.
Robert Collinsd84b29f2015-08-07 10:22:54 +12001415 If the operands are of different types, and right operand's type is
1416 a direct or indirect subclass of the left operand's type,
1417 the reflected method of the right operand has priority, otherwise
1418 the left operand's method has priority. Virtual subclassing is
1419 not considered.
Georg Brandl116aa622007-08-15 14:28:22 +00001420
Georg Brandl116aa622007-08-15 14:28:22 +00001421.. method:: object.__hash__(self)
1422
1423 .. index::
1424 object: dictionary
1425 builtin: hash
1426
Benjamin Peterson6cadba72008-11-19 22:38:29 +00001427 Called by built-in function :func:`hash` and for operations on members of
1428 hashed collections including :class:`set`, :class:`frozenset`, and
Victor Stinner509476b2016-12-19 13:09:28 +01001429 :class:`dict`. :meth:`__hash__` should return an integer. The only required
1430 property is that objects which compare equal have the same hash value; it is
1431 advised to mix together the hash values of the components of the object that
1432 also play a part in comparison of objects by packing them into a tuple and
1433 hashing the tuple. Example::
1434
1435 def __hash__(self):
1436 return hash((self.name, self.nick, self.color))
Barry Warsaw224a5992013-07-15 14:47:29 -04001437
1438 .. note::
1439
1440 :func:`hash` truncates the value returned from an object's custom
1441 :meth:`__hash__` method to the size of a :c:type:`Py_ssize_t`. This is
1442 typically 8 bytes on 64-bit builds and 4 bytes on 32-bit builds. If an
1443 object's :meth:`__hash__` must interoperate on builds of different bit
1444 sizes, be sure to check the width on all supported builds. An easy way
1445 to do this is with
Martin Panterd21e0b52015-10-10 10:36:22 +00001446 ``python -c "import sys; print(sys.hash_info.width)"``.
Georg Brandl116aa622007-08-15 14:28:22 +00001447
Georg Brandl05f5ab72008-09-24 09:11:47 +00001448 If a class does not define an :meth:`__eq__` method it should not define a
1449 :meth:`__hash__` operation either; if it defines :meth:`__eq__` but not
Benjamin Peterson6cadba72008-11-19 22:38:29 +00001450 :meth:`__hash__`, its instances will not be usable as items in hashable
1451 collections. If a class defines mutable objects and implements an
1452 :meth:`__eq__` method, it should not implement :meth:`__hash__`, since the
1453 implementation of hashable collections requires that a key's hash value is
1454 immutable (if the object's hash value changes, it will be in the wrong hash
1455 bucket).
1456
Georg Brandl05f5ab72008-09-24 09:11:47 +00001457 User-defined classes have :meth:`__eq__` and :meth:`__hash__` methods
Nick Coghlan73c96db2008-08-31 13:21:24 +00001458 by default; with them, all objects compare unequal (except with themselves)
Nick Coghlan337b2bf2012-05-20 18:30:49 +10001459 and ``x.__hash__()`` returns an appropriate value such that ``x == y``
1460 implies both that ``x is y`` and ``hash(x) == hash(y)``.
1461
R David Murrayd8bbde32012-09-11 13:01:43 -04001462 A class that overrides :meth:`__eq__` and does not define :meth:`__hash__`
1463 will have its :meth:`__hash__` implicitly set to ``None``. When the
1464 :meth:`__hash__` method of a class is ``None``, instances of the class will
1465 raise an appropriate :exc:`TypeError` when a program attempts to retrieve
1466 their hash value, and will also be correctly identified as unhashable when
Serhiy Storchaka2e576f52017-04-24 09:05:00 +03001467 checking ``isinstance(obj, collections.abc.Hashable)``.
Nick Coghlan73c96db2008-08-31 13:21:24 +00001468
Georg Brandlae2dbe22009-03-13 19:04:40 +00001469 If a class that overrides :meth:`__eq__` needs to retain the implementation
Georg Brandl05f5ab72008-09-24 09:11:47 +00001470 of :meth:`__hash__` from a parent class, the interpreter must be told this
R David Murrayd8bbde32012-09-11 13:01:43 -04001471 explicitly by setting ``__hash__ = <ParentClass>.__hash__``.
1472
1473 If a class that does not override :meth:`__eq__` wishes to suppress hash
1474 support, it should include ``__hash__ = None`` in the class definition.
1475 A class which defines its own :meth:`__hash__` that explicitly raises
1476 a :exc:`TypeError` would be incorrectly identified as hashable by
Serhiy Storchaka2e576f52017-04-24 09:05:00 +03001477 an ``isinstance(obj, collections.abc.Hashable)`` call.
Georg Brandl05f5ab72008-09-24 09:11:47 +00001478
Benjamin Petersonc9f54cf2012-02-21 16:08:05 -05001479
1480 .. note::
1481
Serhiy Storchakae9c90aa2019-08-24 12:49:27 +03001482 By default, the :meth:`__hash__` values of str and bytes objects are
1483 "salted" with an unpredictable random value. Although they
Benjamin Petersonc9f54cf2012-02-21 16:08:05 -05001484 remain constant within an individual Python process, they are not
1485 predictable between repeated invocations of Python.
1486
1487 This is intended to provide protection against a denial-of-service caused
1488 by carefully-chosen inputs that exploit the worst case performance of a
1489 dict insertion, O(n^2) complexity. See
1490 http://www.ocert.org/advisories/ocert-2011-003.html for details.
1491
wim glenna48e0eb2018-11-16 05:58:19 -06001492 Changing hash values affects the iteration order of sets.
1493 Python has never made guarantees about this ordering
Antoine Pitrouc86e8d92012-08-01 14:53:22 +02001494 (and it typically varies between 32-bit and 64-bit builds).
Benjamin Petersonc9f54cf2012-02-21 16:08:05 -05001495
1496 See also :envvar:`PYTHONHASHSEED`.
1497
1498 .. versionchanged:: 3.3
1499 Hash randomization is enabled by default.
Georg Brandl2daf6ae2012-02-20 19:54:16 +01001500
Georg Brandl116aa622007-08-15 14:28:22 +00001501
1502.. method:: object.__bool__(self)
Georg Brandl1aeaadd2008-09-06 17:42:52 +00001503
Georg Brandl116aa622007-08-15 14:28:22 +00001504 .. index:: single: __len__() (mapping object method)
1505
Benjamin Petersonf07d0022009-03-21 17:31:58 +00001506 Called to implement truth value testing and the built-in operation
Amaury Forgeot d'Arc097cd072009-07-07 00:43:08 +00001507 ``bool()``; should return ``False`` or ``True``. When this method is not
1508 defined, :meth:`__len__` is called, if it is defined, and the object is
1509 considered true if its result is nonzero. If a class defines neither
1510 :meth:`__len__` nor :meth:`__bool__`, all its instances are considered
1511 true.
Georg Brandl116aa622007-08-15 14:28:22 +00001512
1513
Georg Brandl116aa622007-08-15 14:28:22 +00001514.. _attribute-access:
1515
1516Customizing attribute access
1517----------------------------
1518
1519The following methods can be defined to customize the meaning of attribute
1520access (use of, assignment to, or deletion of ``x.name``) for class instances.
1521
Georg Brandl85eb8c12007-08-31 16:33:38 +00001522.. XXX explain how descriptors interfere here!
1523
Georg Brandl116aa622007-08-15 14:28:22 +00001524
1525.. method:: object.__getattr__(self, name)
1526
Cheryl Sabellad1f31812018-02-04 21:03:22 -05001527 Called when the default attribute access fails with an :exc:`AttributeError`
1528 (either :meth:`__getattribute__` raises an :exc:`AttributeError` because
1529 *name* is not an instance attribute or an attribute in the class tree
1530 for ``self``; or :meth:`__get__` of a *name* property raises
1531 :exc:`AttributeError`). This method should either return the (computed)
1532 attribute value or raise an :exc:`AttributeError` exception.
Georg Brandl116aa622007-08-15 14:28:22 +00001533
Georg Brandl116aa622007-08-15 14:28:22 +00001534 Note that if the attribute is found through the normal mechanism,
1535 :meth:`__getattr__` is not called. (This is an intentional asymmetry between
1536 :meth:`__getattr__` and :meth:`__setattr__`.) This is done both for efficiency
Nick Coghlan3a5d7e32008-08-31 12:40:14 +00001537 reasons and because otherwise :meth:`__getattr__` would have no way to access
Georg Brandl116aa622007-08-15 14:28:22 +00001538 other attributes of the instance. Note that at least for instance variables,
1539 you can fake total control by not inserting any values in the instance attribute
1540 dictionary (but instead inserting them in another object). See the
Georg Brandl85eb8c12007-08-31 16:33:38 +00001541 :meth:`__getattribute__` method below for a way to actually get total control
1542 over attribute access.
Georg Brandl116aa622007-08-15 14:28:22 +00001543
1544
1545.. method:: object.__getattribute__(self, name)
1546
1547 Called unconditionally to implement attribute accesses for instances of the
1548 class. If the class also defines :meth:`__getattr__`, the latter will not be
1549 called unless :meth:`__getattribute__` either calls it explicitly or raises an
1550 :exc:`AttributeError`. This method should return the (computed) attribute value
1551 or raise an :exc:`AttributeError` exception. In order to avoid infinite
1552 recursion in this method, its implementation should always call the base class
1553 method with the same name to access any attributes it needs, for example,
1554 ``object.__getattribute__(self, name)``.
1555
Nick Coghlan3a5d7e32008-08-31 12:40:14 +00001556 .. note::
1557
1558 This method may still be bypassed when looking up special methods as the
Georg Brandl22b34312009-07-26 14:54:51 +00001559 result of implicit invocation via language syntax or built-in functions.
Nick Coghlan3a5d7e32008-08-31 12:40:14 +00001560 See :ref:`special-lookup`.
1561
Saiyang Gou3f7e9902020-10-20 12:23:15 -07001562 .. audit-event:: object.__getattr__ obj,name object.__getattribute__
1563
1564 For certain sensitive attribute accesses, raises an
1565 :ref:`auditing event <auditing>` ``object.__getattr__`` with arguments
1566 ``obj`` and ``name``.
1567
Georg Brandl116aa622007-08-15 14:28:22 +00001568
Georg Brandl85eb8c12007-08-31 16:33:38 +00001569.. method:: object.__setattr__(self, name, value)
1570
1571 Called when an attribute assignment is attempted. This is called instead of
1572 the normal mechanism (i.e. store the value in the instance dictionary).
1573 *name* is the attribute name, *value* is the value to be assigned to it.
1574
1575 If :meth:`__setattr__` wants to assign to an instance attribute, it should
1576 call the base class method with the same name, for example,
1577 ``object.__setattr__(self, name, value)``.
1578
Saiyang Gou3f7e9902020-10-20 12:23:15 -07001579 .. audit-event:: object.__setattr__ obj,name,value object.__setattr__
1580
1581 For certain sensitive attribute assignments, raises an
1582 :ref:`auditing event <auditing>` ``object.__setattr__`` with arguments
1583 ``obj``, ``name``, ``value``.
1584
Georg Brandl85eb8c12007-08-31 16:33:38 +00001585
1586.. method:: object.__delattr__(self, name)
1587
1588 Like :meth:`__setattr__` but for attribute deletion instead of assignment. This
1589 should only be implemented if ``del obj.name`` is meaningful for the object.
1590
Saiyang Gou3f7e9902020-10-20 12:23:15 -07001591 .. audit-event:: object.__delattr__ obj,name object.__delattr__
1592
1593 For certain sensitive attribute deletions, raises an
1594 :ref:`auditing event <auditing>` ``object.__delattr__`` with arguments
1595 ``obj`` and ``name``.
1596
Georg Brandl85eb8c12007-08-31 16:33:38 +00001597
Benjamin Peterson1cef37c2008-07-02 14:44:54 +00001598.. method:: object.__dir__(self)
1599
Benjamin Peterson3bbb7222011-06-11 16:12:08 -05001600 Called when :func:`dir` is called on the object. A sequence must be
1601 returned. :func:`dir` converts the returned sequence to a list and sorts it.
Benjamin Peterson1cef37c2008-07-02 14:44:54 +00001602
1603
Ivan Levkivskyi5364b5c2017-12-14 11:59:44 +01001604Customizing module attribute access
1605^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1606
1607.. index::
1608 single: __getattr__ (module attribute)
1609 single: __dir__ (module attribute)
1610 single: __class__ (module attribute)
1611
1612Special names ``__getattr__`` and ``__dir__`` can be also used to customize
1613access to module attributes. The ``__getattr__`` function at the module level
1614should accept one argument which is the name of an attribute and return the
1615computed value or raise an :exc:`AttributeError`. If an attribute is
1616not found on a module object through the normal lookup, i.e.
1617:meth:`object.__getattribute__`, then ``__getattr__`` is searched in
1618the module ``__dict__`` before raising an :exc:`AttributeError`. If found,
1619it is called with the attribute name and the result is returned.
1620
alclarks4544e782019-11-17 22:00:43 +00001621The ``__dir__`` function should accept no arguments, and return a sequence of
Ivan Levkivskyi5364b5c2017-12-14 11:59:44 +01001622strings that represents the names accessible on module. If present, this
1623function overrides the standard :func:`dir` search on a module.
1624
1625For a more fine grained customization of the module behavior (setting
1626attributes, properties, etc.), one can set the ``__class__`` attribute of
1627a module object to a subclass of :class:`types.ModuleType`. For example::
1628
1629 import sys
1630 from types import ModuleType
1631
1632 class VerboseModule(ModuleType):
1633 def __repr__(self):
1634 return f'Verbose {self.__name__}'
1635
1636 def __setattr__(self, attr, value):
1637 print(f'Setting {attr}...')
Denis Osipov0bee3c32018-11-06 06:53:21 +05001638 super().__setattr__(attr, value)
Ivan Levkivskyi5364b5c2017-12-14 11:59:44 +01001639
1640 sys.modules[__name__].__class__ = VerboseModule
1641
1642.. note::
1643 Defining module ``__getattr__`` and setting module ``__class__`` only
1644 affect lookups made using the attribute access syntax -- directly accessing
1645 the module globals (whether by code within the module, or via a reference
1646 to the module's globals dictionary) is unaffected.
1647
Cheryl Sabella85527cf2018-01-26 21:40:52 -05001648.. versionchanged:: 3.5
1649 ``__class__`` module attribute is now writable.
1650
1651.. versionadded:: 3.7
1652 ``__getattr__`` and ``__dir__`` module attributes.
1653
1654.. seealso::
1655
1656 :pep:`562` - Module __getattr__ and __dir__
1657 Describes the ``__getattr__`` and ``__dir__`` functions on modules.
1658
Ivan Levkivskyi5364b5c2017-12-14 11:59:44 +01001659
Georg Brandl116aa622007-08-15 14:28:22 +00001660.. _descriptors:
1661
1662Implementing Descriptors
1663^^^^^^^^^^^^^^^^^^^^^^^^
1664
1665The following methods only apply when an instance of the class containing the
Raymond Hettinger3b654be2011-03-22 16:27:02 -07001666method (a so-called *descriptor* class) appears in an *owner* class (the
1667descriptor must be in either the owner's class dictionary or in the class
1668dictionary for one of its parents). In the examples below, "the attribute"
1669refers to the attribute whose name is the key of the property in the owner
Martin Panterbae5d812016-06-18 03:57:31 +00001670class' :attr:`~object.__dict__`.
Georg Brandl116aa622007-08-15 14:28:22 +00001671
1672
Raymond Hettinger0dac68f2019-08-29 01:27:42 -07001673.. method:: object.__get__(self, instance, owner=None)
Georg Brandl116aa622007-08-15 14:28:22 +00001674
Raymond Hettinger0dac68f2019-08-29 01:27:42 -07001675 Called to get the attribute of the owner class (class attribute access) or
1676 of an instance of that class (instance attribute access). The optional
1677 *owner* argument is the owner class, while *instance* is the instance that
1678 the attribute was accessed through, or ``None`` when the attribute is
1679 accessed through the *owner*.
Georg Brandl116aa622007-08-15 14:28:22 +00001680
Raymond Hettinger0dac68f2019-08-29 01:27:42 -07001681 This method should return the computed attribute value or raise an
1682 :exc:`AttributeError` exception.
1683
1684 :PEP:`252` specifies that :meth:`__get__` is callable with one or two
1685 arguments. Python's own built-in descriptors support this specification;
1686 however, it is likely that some third-party tools have descriptors
1687 that require both arguments. Python's own :meth:`__getattribute__`
1688 implementation always passes in both arguments whether they are required
1689 or not.
Georg Brandl116aa622007-08-15 14:28:22 +00001690
1691.. method:: object.__set__(self, instance, value)
1692
1693 Called to set the attribute on an instance *instance* of the owner class to a
1694 new value, *value*.
1695
Raymond Hettinger0dac68f2019-08-29 01:27:42 -07001696 Note, adding :meth:`__set__` or :meth:`__delete__` changes the kind of
1697 descriptor to a "data descriptor". See :ref:`descriptor-invocation` for
1698 more details.
Georg Brandl116aa622007-08-15 14:28:22 +00001699
1700.. method:: object.__delete__(self, instance)
1701
1702 Called to delete the attribute on an instance *instance* of the owner class.
1703
1704
Nick Coghland78448e2016-07-30 16:26:03 +10001705.. method:: object.__set_name__(self, owner, name)
1706
1707 Called at the time the owning class *owner* is created. The
1708 descriptor has been assigned to *name*.
1709
Florian Dahlitz1bddf892019-11-27 09:46:40 +01001710 .. note::
Berker Peksag01d17192016-07-30 14:06:15 +03001711
Tal Einat02519f72019-11-28 07:22:09 +02001712 :meth:`__set_name__` is only called implicitly as part of the
1713 :class:`type` constructor, so it will need to be called explicitly with
1714 the appropriate parameters when a descriptor is added to a class after
1715 initial creation::
Florian Dahlitz1bddf892019-11-27 09:46:40 +01001716
Tal Einat02519f72019-11-28 07:22:09 +02001717 class A:
1718 pass
Florian Dahlitz1bddf892019-11-27 09:46:40 +01001719 descr = custom_descriptor()
Tal Einat02519f72019-11-28 07:22:09 +02001720 A.attr = descr
1721 descr.__set_name__(A, 'attr')
Florian Dahlitz1bddf892019-11-27 09:46:40 +01001722
1723 See :ref:`class-object-creation` for more details.
1724
1725 .. versionadded:: 3.6
Nick Coghland78448e2016-07-30 16:26:03 +10001726
Yury Selivanov490a72e2014-04-08 14:01:12 -04001727The attribute :attr:`__objclass__` is interpreted by the :mod:`inspect` module
1728as specifying the class where this object was defined (setting this
1729appropriately can assist in runtime introspection of dynamic class attributes).
1730For callables, it may indicate that an instance of the given type (or a
1731subclass) is expected or required as the first positional argument (for example,
1732CPython sets this attribute for unbound methods that are implemented in C).
Yury Selivanovb9aa8cb2014-04-08 12:04:04 -04001733
1734
Georg Brandl116aa622007-08-15 14:28:22 +00001735.. _descriptor-invocation:
1736
1737Invoking Descriptors
1738^^^^^^^^^^^^^^^^^^^^
1739
1740In general, a descriptor is an object attribute with "binding behavior", one
1741whose attribute access has been overridden by methods in the descriptor
1742protocol: :meth:`__get__`, :meth:`__set__`, and :meth:`__delete__`. If any of
1743those methods are defined for an object, it is said to be a descriptor.
1744
1745The default behavior for attribute access is to get, set, or delete the
1746attribute from an object's dictionary. For instance, ``a.x`` has a lookup chain
1747starting with ``a.__dict__['x']``, then ``type(a).__dict__['x']``, and
1748continuing through the base classes of ``type(a)`` excluding metaclasses.
1749
1750However, if the looked-up value is an object defining one of the descriptor
1751methods, then Python may override the default behavior and invoke the descriptor
1752method instead. Where this occurs in the precedence chain depends on which
Georg Brandl23e8db52008-04-07 19:17:06 +00001753descriptor methods were defined and how they were called.
Georg Brandl116aa622007-08-15 14:28:22 +00001754
1755The starting point for descriptor invocation is a binding, ``a.x``. How the
1756arguments are assembled depends on ``a``:
1757
1758Direct Call
1759 The simplest and least common call is when user code directly invokes a
1760 descriptor method: ``x.__get__(a)``.
1761
1762Instance Binding
Georg Brandl85eb8c12007-08-31 16:33:38 +00001763 If binding to an object instance, ``a.x`` is transformed into the call:
Georg Brandl116aa622007-08-15 14:28:22 +00001764 ``type(a).__dict__['x'].__get__(a, type(a))``.
1765
1766Class Binding
Georg Brandl85eb8c12007-08-31 16:33:38 +00001767 If binding to a class, ``A.x`` is transformed into the call:
Georg Brandl116aa622007-08-15 14:28:22 +00001768 ``A.__dict__['x'].__get__(None, A)``.
1769
1770Super Binding
csabella12b1c182017-05-14 20:42:00 -07001771 If ``a`` is an instance of :class:`super`, then the binding ``super(B, obj).m()``
1772 searches ``obj.__class__.__mro__`` for the base class ``A``
Georg Brandl116aa622007-08-15 14:28:22 +00001773 immediately preceding ``B`` and then invokes the descriptor with the call:
Raymond Hettingerb199b222011-03-22 15:28:45 -07001774 ``A.__dict__['m'].__get__(obj, obj.__class__)``.
Georg Brandl116aa622007-08-15 14:28:22 +00001775
1776For instance bindings, the precedence of descriptor invocation depends on the
Benjamin Peterson5e55b3e2010-02-03 02:35:45 +00001777which descriptor methods are defined. A descriptor can define any combination
1778of :meth:`__get__`, :meth:`__set__` and :meth:`__delete__`. If it does not
1779define :meth:`__get__`, then accessing the attribute will return the descriptor
1780object itself unless there is a value in the object's instance dictionary. If
1781the descriptor defines :meth:`__set__` and/or :meth:`__delete__`, it is a data
1782descriptor; if it defines neither, it is a non-data descriptor. Normally, data
1783descriptors define both :meth:`__get__` and :meth:`__set__`, while non-data
1784descriptors have just the :meth:`__get__` method. Data descriptors with
Géry Ogam4c155f72019-10-29 08:04:01 +01001785:meth:`__get__` and :meth:`__set__` (and/or :meth:`__delete__`) defined always override a redefinition in an
Georg Brandl116aa622007-08-15 14:28:22 +00001786instance dictionary. In contrast, non-data descriptors can be overridden by
Benjamin Peterson5e55b3e2010-02-03 02:35:45 +00001787instances.
Georg Brandl116aa622007-08-15 14:28:22 +00001788
1789Python methods (including :func:`staticmethod` and :func:`classmethod`) are
1790implemented as non-data descriptors. Accordingly, instances can redefine and
1791override methods. This allows individual instances to acquire behaviors that
1792differ from other instances of the same class.
1793
1794The :func:`property` function is implemented as a data descriptor. Accordingly,
1795instances cannot override the behavior of a property.
1796
1797
1798.. _slots:
1799
1800__slots__
1801^^^^^^^^^
1802
Aaron Hall, MBA2b44e302017-05-25 22:33:26 -07001803*__slots__* allow us to explicitly declare data members (like
1804properties) and deny the creation of *__dict__* and *__weakref__*
1805(unless explicitly declared in *__slots__* or available in a parent.)
Georg Brandl116aa622007-08-15 14:28:22 +00001806
Aaron Hall, MBA2b44e302017-05-25 22:33:26 -07001807The space saved over using *__dict__* can be significant.
Raymond Hettinger74638842019-02-21 03:34:04 -08001808Attribute lookup speed can be significantly improved as well.
Georg Brandl116aa622007-08-15 14:28:22 +00001809
Georg Brandl85eb8c12007-08-31 16:33:38 +00001810.. data:: object.__slots__
Georg Brandl116aa622007-08-15 14:28:22 +00001811
Georg Brandl85eb8c12007-08-31 16:33:38 +00001812 This class variable can be assigned a string, iterable, or sequence of
Georg Brandla4c8c472014-10-31 10:38:49 +01001813 strings with variable names used by instances. *__slots__* reserves space
1814 for the declared variables and prevents the automatic creation of *__dict__*
1815 and *__weakref__* for each instance.
Georg Brandl116aa622007-08-15 14:28:22 +00001816
Georg Brandl116aa622007-08-15 14:28:22 +00001817
1818Notes on using *__slots__*
Georg Brandl16174572007-09-01 12:38:06 +00001819""""""""""""""""""""""""""
Georg Brandl116aa622007-08-15 14:28:22 +00001820
Aaron Hall, MBA2b44e302017-05-25 22:33:26 -07001821* When inheriting from a class without *__slots__*, the *__dict__* and
1822 *__weakref__* attribute of the instances will always be accessible.
Georg Brandl3dbca812008-07-23 16:10:53 +00001823
Georg Brandl116aa622007-08-15 14:28:22 +00001824* Without a *__dict__* variable, instances cannot be assigned new variables not
1825 listed in the *__slots__* definition. Attempts to assign to an unlisted
1826 variable name raises :exc:`AttributeError`. If dynamic assignment of new
Georg Brandl85eb8c12007-08-31 16:33:38 +00001827 variables is desired, then add ``'__dict__'`` to the sequence of strings in
1828 the *__slots__* declaration.
Georg Brandl116aa622007-08-15 14:28:22 +00001829
Georg Brandl116aa622007-08-15 14:28:22 +00001830* Without a *__weakref__* variable for each instance, classes defining
1831 *__slots__* do not support weak references to its instances. If weak reference
1832 support is needed, then add ``'__weakref__'`` to the sequence of strings in the
1833 *__slots__* declaration.
1834
Georg Brandl116aa622007-08-15 14:28:22 +00001835* *__slots__* are implemented at the class level by creating descriptors
1836 (:ref:`descriptors`) for each variable name. As a result, class attributes
1837 cannot be used to set default values for instance variables defined by
1838 *__slots__*; otherwise, the class attribute would overwrite the descriptor
1839 assignment.
1840
Aaron Hall, MBA2b44e302017-05-25 22:33:26 -07001841* The action of a *__slots__* declaration is not limited to the class
1842 where it is defined. *__slots__* declared in parents are available in
1843 child classes. However, child subclasses will get a *__dict__* and
1844 *__weakref__* unless they also define *__slots__* (which should only
1845 contain names of any *additional* slots).
Georg Brandl495f7b52009-10-27 15:28:25 +00001846
Georg Brandl116aa622007-08-15 14:28:22 +00001847* If a class defines a slot also defined in a base class, the instance variable
1848 defined by the base class slot is inaccessible (except by retrieving its
1849 descriptor directly from the base class). This renders the meaning of the
1850 program undefined. In the future, a check may be added to prevent this.
1851
Benjamin Peterson1a6e0d02008-10-25 15:49:17 +00001852* Nonempty *__slots__* does not work for classes derived from "variable-length"
Zachary Ware340a6922013-12-31 12:09:26 -06001853 built-in types such as :class:`int`, :class:`bytes` and :class:`tuple`.
Georg Brandl116aa622007-08-15 14:28:22 +00001854
1855* Any non-string iterable may be assigned to *__slots__*. Mappings may also be
1856 used; however, in the future, special meaning may be assigned to the values
1857 corresponding to each key.
1858
1859* *__class__* assignment works only if both classes have the same *__slots__*.
1860
Aaron Hall, MBA2b44e302017-05-25 22:33:26 -07001861* Multiple inheritance with multiple slotted parent classes can be used,
1862 but only one parent is allowed to have attributes created by slots
1863 (the other bases must have empty slot layouts) - violations raise
1864 :exc:`TypeError`.
Georg Brandl116aa622007-08-15 14:28:22 +00001865
alclarks4544e782019-11-17 22:00:43 +00001866* If an iterator is used for *__slots__* then a descriptor is created for each
1867 of the iterator's values. However, the *__slots__* attribute will be an empty
1868 iterator.
1869
Nick Coghland78448e2016-07-30 16:26:03 +10001870.. _class-customization:
Georg Brandl116aa622007-08-15 14:28:22 +00001871
1872Customizing class creation
1873--------------------------
1874
Nick Coghland78448e2016-07-30 16:26:03 +10001875Whenever a class inherits from another class, *__init_subclass__* is
1876called on that class. This way, it is possible to write classes which
1877change the behavior of subclasses. This is closely related to class
1878decorators, but where class decorators only affect the specific class they're
1879applied to, ``__init_subclass__`` solely applies to future subclasses of the
1880class defining the method.
1881
1882.. classmethod:: object.__init_subclass__(cls)
Berker Peksag01d17192016-07-30 14:06:15 +03001883
Nick Coghland78448e2016-07-30 16:26:03 +10001884 This method is called whenever the containing class is subclassed.
1885 *cls* is then the new subclass. If defined as a normal instance method,
1886 this method is implicitly converted to a class method.
1887
1888 Keyword arguments which are given to a new class are passed to
1889 the parent's class ``__init_subclass__``. For compatibility with
1890 other classes using ``__init_subclass__``, one should take out the
1891 needed keyword arguments and pass the others over to the base
1892 class, as in::
1893
1894 class Philosopher:
Serhiy Storchaka70c5f2a2019-06-01 11:38:24 +03001895 def __init_subclass__(cls, /, default_name, **kwargs):
Nick Coghland78448e2016-07-30 16:26:03 +10001896 super().__init_subclass__(**kwargs)
1897 cls.default_name = default_name
1898
1899 class AustralianPhilosopher(Philosopher, default_name="Bruce"):
1900 pass
1901
1902 The default implementation ``object.__init_subclass__`` does
1903 nothing, but raises an error if it is called with any arguments.
1904
Nick Coghlan607e1c42016-07-31 12:42:49 +10001905 .. note::
1906
1907 The metaclass hint ``metaclass`` is consumed by the rest of the type
1908 machinery, and is never passed to ``__init_subclass__`` implementations.
1909 The actual metaclass (rather than the explicit hint) can be accessed as
1910 ``type(cls)``.
1911
Berker Peksag01d17192016-07-30 14:06:15 +03001912 .. versionadded:: 3.6
1913
Nick Coghland78448e2016-07-30 16:26:03 +10001914
1915.. _metaclasses:
1916
1917Metaclasses
1918^^^^^^^^^^^
1919
Nick Coghlan19d24672016-12-05 16:47:55 +10001920.. index::
Serhiy Storchakaddb961d2018-10-26 09:00:49 +03001921 single: metaclass
1922 builtin: type
Serhiy Storchaka913876d2018-10-28 13:41:26 +02001923 single: = (equals); class definition
Nick Coghlan19d24672016-12-05 16:47:55 +10001924
Nick Coghlan7fc570a2012-05-20 02:34:13 +10001925By default, classes are constructed using :func:`type`. The class body is
1926executed in a new namespace and the class name is bound locally to the
1927result of ``type(name, bases, namespace)``.
Georg Brandl116aa622007-08-15 14:28:22 +00001928
Raymond Hettinger7ea386e2016-08-25 21:11:50 -07001929The class creation process can be customized by passing the ``metaclass``
Nick Coghlan7fc570a2012-05-20 02:34:13 +10001930keyword argument in the class definition line, or by inheriting from an
1931existing class that included such an argument. In the following example,
1932both ``MyClass`` and ``MySubclass`` are instances of ``Meta``::
Georg Brandl116aa622007-08-15 14:28:22 +00001933
Nick Coghlan7fc570a2012-05-20 02:34:13 +10001934 class Meta(type):
1935 pass
Georg Brandl116aa622007-08-15 14:28:22 +00001936
Nick Coghlan7fc570a2012-05-20 02:34:13 +10001937 class MyClass(metaclass=Meta):
1938 pass
Georg Brandl116aa622007-08-15 14:28:22 +00001939
Nick Coghlan7fc570a2012-05-20 02:34:13 +10001940 class MySubclass(MyClass):
1941 pass
Christian Heimes790c8232008-01-07 21:14:23 +00001942
Nick Coghlan7fc570a2012-05-20 02:34:13 +10001943Any other keyword arguments that are specified in the class definition are
1944passed through to all metaclass operations described below.
Christian Heimes790c8232008-01-07 21:14:23 +00001945
Nick Coghlan7fc570a2012-05-20 02:34:13 +10001946When a class definition is executed, the following steps occur:
Christian Heimes790c8232008-01-07 21:14:23 +00001947
Jules Lasne (jlasne)1fc5bf22019-03-27 11:10:33 +01001948* MRO entries are resolved;
1949* the appropriate metaclass is determined;
1950* the class namespace is prepared;
1951* the class body is executed;
1952* the class object is created.
Georg Brandl116aa622007-08-15 14:28:22 +00001953
Ivan Levkivskyibd5f9652018-05-08 19:38:41 +01001954
1955Resolving MRO entries
1956^^^^^^^^^^^^^^^^^^^^^
1957
1958If a base that appears in class definition is not an instance of :class:`type`,
1959then an ``__mro_entries__`` method is searched on it. If found, it is called
1960with the original bases tuple. This method must return a tuple of classes that
1961will be used instead of this base. The tuple may be empty, in such case
1962the original base is ignored.
1963
1964.. seealso::
1965
1966 :pep:`560` - Core support for typing module and generic types
1967
1968
Nick Coghlan7fc570a2012-05-20 02:34:13 +10001969Determining the appropriate metaclass
1970^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Nick Coghlan19d24672016-12-05 16:47:55 +10001971.. index::
1972 single: metaclass hint
Georg Brandl116aa622007-08-15 14:28:22 +00001973
Nick Coghlan7fc570a2012-05-20 02:34:13 +10001974The appropriate metaclass for a class definition is determined as follows:
Georg Brandl116aa622007-08-15 14:28:22 +00001975
Jules Lasne (jlasne)1fc5bf22019-03-27 11:10:33 +01001976* if no bases and no explicit metaclass are given, then :func:`type` is used;
Nick Coghlan7fc570a2012-05-20 02:34:13 +10001977* if an explicit metaclass is given and it is *not* an instance of
Jules Lasne (jlasne)1fc5bf22019-03-27 11:10:33 +01001978 :func:`type`, then it is used directly as the metaclass;
Nick Coghlan7fc570a2012-05-20 02:34:13 +10001979* if an instance of :func:`type` is given as the explicit metaclass, or
Jules Lasne (jlasne)1fc5bf22019-03-27 11:10:33 +01001980 bases are defined, then the most derived metaclass is used.
Georg Brandl116aa622007-08-15 14:28:22 +00001981
Nick Coghlan7fc570a2012-05-20 02:34:13 +10001982The most derived metaclass is selected from the explicitly specified
1983metaclass (if any) and the metaclasses (i.e. ``type(cls)``) of all specified
1984base classes. The most derived metaclass is one which is a subtype of *all*
1985of these candidate metaclasses. If none of the candidate metaclasses meets
1986that criterion, then the class definition will fail with ``TypeError``.
1987
1988
R David Murrayaf7d2c42014-02-12 13:00:36 -05001989.. _prepare:
1990
Nick Coghlan7fc570a2012-05-20 02:34:13 +10001991Preparing the class namespace
1992^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1993
Nick Coghlan19d24672016-12-05 16:47:55 +10001994.. index::
1995 single: __prepare__ (metaclass method)
1996
Nick Coghlan7fc570a2012-05-20 02:34:13 +10001997Once the appropriate metaclass has been identified, then the class namespace
1998is prepared. If the metaclass has a ``__prepare__`` attribute, it is called
1999as ``namespace = metaclass.__prepare__(name, bases, **kwds)`` (where the
alclarks7de61742020-01-25 18:49:58 +00002000additional keyword arguments, if any, come from the class definition). The
ananthan-123fbe2e0b2020-02-22 23:26:02 +05302001``__prepare__`` method should be implemented as a :func:`classmethod`. The
2002namespace returned by ``__prepare__`` is passed in to ``__new__``, but when
2003the final class object is created the namespace is copied into a new ``dict``.
Nick Coghlan7fc570a2012-05-20 02:34:13 +10002004
2005If the metaclass has no ``__prepare__`` attribute, then the class namespace
Caleb Donovicke59334e2020-03-06 10:20:48 -08002006is initialised as an empty ordered mapping.
Eric Snow92a6c172016-09-05 14:50:11 -07002007
Nick Coghlan7fc570a2012-05-20 02:34:13 +10002008.. seealso::
2009
2010 :pep:`3115` - Metaclasses in Python 3000
2011 Introduced the ``__prepare__`` namespace hook
2012
2013
2014Executing the class body
2015^^^^^^^^^^^^^^^^^^^^^^^^
2016
Nick Coghlan19d24672016-12-05 16:47:55 +10002017.. index::
2018 single: class; body
2019
Nick Coghlan7fc570a2012-05-20 02:34:13 +10002020The class body is executed (approximately) as
2021``exec(body, globals(), namespace)``. The key difference from a normal
2022call to :func:`exec` is that lexical scoping allows the class body (including
2023any methods) to reference names from the current and outer scopes when the
2024class definition occurs inside a function.
2025
2026However, even when the class definition occurs inside the function, methods
2027defined inside the class still cannot see names defined at the class scope.
2028Class variables must be accessed through the first parameter of instance or
Nick Coghlan19d24672016-12-05 16:47:55 +10002029class methods, or through the implicit lexically scoped ``__class__`` reference
2030described in the next section.
Nick Coghlan7fc570a2012-05-20 02:34:13 +10002031
Nick Coghlan19d24672016-12-05 16:47:55 +10002032.. _class-object-creation:
Nick Coghlan7fc570a2012-05-20 02:34:13 +10002033
2034Creating the class object
2035^^^^^^^^^^^^^^^^^^^^^^^^^
2036
Nick Coghlan19d24672016-12-05 16:47:55 +10002037.. index::
2038 single: __class__ (method cell)
2039 single: __classcell__ (class namespace entry)
2040
2041
Nick Coghlan7fc570a2012-05-20 02:34:13 +10002042Once the class namespace has been populated by executing the class body,
2043the class object is created by calling
2044``metaclass(name, bases, namespace, **kwds)`` (the additional keywords
Nick Coghlan78770f02012-05-20 18:15:11 +10002045passed here are the same as those passed to ``__prepare__``).
Nick Coghlan7fc570a2012-05-20 02:34:13 +10002046
2047This class object is the one that will be referenced by the zero-argument
2048form of :func:`super`. ``__class__`` is an implicit closure reference
2049created by the compiler if any methods in a class body refer to either
2050``__class__`` or ``super``. This allows the zero argument form of
2051:func:`super` to correctly identify the class being defined based on
2052lexical scoping, while the class or instance that was used to make the
2053current call is identified based on the first argument passed to the method.
2054
Nick Coghlan19d24672016-12-05 16:47:55 +10002055.. impl-detail::
2056
2057 In CPython 3.6 and later, the ``__class__`` cell is passed to the metaclass
2058 as a ``__classcell__`` entry in the class namespace. If present, this must
2059 be propagated up to the ``type.__new__`` call in order for the class to be
2060 initialised correctly.
Serhiy Storchakaf5e7b192018-05-20 08:48:12 +03002061 Failing to do so will result in a :exc:`RuntimeError` in Python 3.8.
Nick Coghlan19d24672016-12-05 16:47:55 +10002062
2063When using the default metaclass :class:`type`, or any metaclass that ultimately
2064calls ``type.__new__``, the following additional customisation steps are
2065invoked after creating the class object:
2066
2067* first, ``type.__new__`` collects all of the descriptors in the class
2068 namespace that define a :meth:`~object.__set_name__` method;
2069* second, all of these ``__set_name__`` methods are called with the class
Jules Lasne (jlasne)1fc5bf22019-03-27 11:10:33 +01002070 being defined and the assigned name of that particular descriptor;
Nick Coghlan19d24672016-12-05 16:47:55 +10002071* finally, the :meth:`~object.__init_subclass__` hook is called on the
2072 immediate parent of the new class in its method resolution order.
2073
Nick Coghlanb2674752012-05-20 19:36:40 +10002074After the class object is created, it is passed to the class decorators
2075included in the class definition (if any) and the resulting object is bound
2076in the local namespace as the defined class.
Nick Coghlan7fc570a2012-05-20 02:34:13 +10002077
R David Murraydd4fcf52016-06-02 20:05:43 -04002078When a new class is created by ``type.__new__``, the object provided as the
Eric Snow68f4dd82016-09-09 11:22:14 -07002079namespace parameter is copied to a new ordered mapping and the original
2080object is discarded. The new copy is wrapped in a read-only proxy, which
2081becomes the :attr:`~object.__dict__` attribute of the class object.
R David Murraydd4fcf52016-06-02 20:05:43 -04002082
Nick Coghlan7fc570a2012-05-20 02:34:13 +10002083.. seealso::
2084
2085 :pep:`3135` - New super
2086 Describes the implicit ``__class__`` closure reference
2087
2088
Andrés Delfinoc2ccac72018-11-16 08:41:55 -03002089Uses for metaclasses
2090^^^^^^^^^^^^^^^^^^^^
Georg Brandl116aa622007-08-15 14:28:22 +00002091
2092The potential uses for metaclasses are boundless. Some ideas that have been
Louie Lub8d1b9d2017-09-13 12:44:14 +08002093explored include enum, logging, interface checking, automatic delegation,
2094automatic property creation, proxies, frameworks, and automatic resource
Georg Brandl116aa622007-08-15 14:28:22 +00002095locking/synchronization.
2096
2097
Georg Brandl8569e582010-05-19 20:57:08 +00002098Customizing instance and subclass checks
2099----------------------------------------
2100
2101The following methods are used to override the default behavior of the
2102:func:`isinstance` and :func:`issubclass` built-in functions.
2103
2104In particular, the metaclass :class:`abc.ABCMeta` implements these methods in
2105order to allow the addition of Abstract Base Classes (ABCs) as "virtual base
Benjamin Petersond7c3ed52010-06-27 22:32:30 +00002106classes" to any class or type (including built-in types), including other
Georg Brandl8569e582010-05-19 20:57:08 +00002107ABCs.
2108
2109.. method:: class.__instancecheck__(self, instance)
2110
2111 Return true if *instance* should be considered a (direct or indirect)
2112 instance of *class*. If defined, called to implement ``isinstance(instance,
2113 class)``.
2114
2115
2116.. method:: class.__subclasscheck__(self, subclass)
2117
2118 Return true if *subclass* should be considered a (direct or indirect)
2119 subclass of *class*. If defined, called to implement ``issubclass(subclass,
2120 class)``.
2121
2122
2123Note that these methods are looked up on the type (metaclass) of a class. They
2124cannot be defined as class methods in the actual class. This is consistent with
Benjamin Petersond7c3ed52010-06-27 22:32:30 +00002125the lookup of special methods that are called on instances, only in this
Georg Brandl8569e582010-05-19 20:57:08 +00002126case the instance is itself a class.
2127
2128.. seealso::
2129
2130 :pep:`3119` - Introducing Abstract Base Classes
2131 Includes the specification for customizing :func:`isinstance` and
Serhiy Storchaka0d196ed2013-10-09 14:02:31 +03002132 :func:`issubclass` behavior through :meth:`~class.__instancecheck__` and
2133 :meth:`~class.__subclasscheck__`, with motivation for this functionality
2134 in the context of adding Abstract Base Classes (see the :mod:`abc`
2135 module) to the language.
Georg Brandl8569e582010-05-19 20:57:08 +00002136
2137
Ivan Levkivskyibd5f9652018-05-08 19:38:41 +01002138Emulating generic types
2139-----------------------
2140
2141One can implement the generic class syntax as specified by :pep:`484`
Jules Lasne (jlasne)1fc5bf22019-03-27 11:10:33 +01002142(for example ``List[int]``) by defining a special method:
Ivan Levkivskyibd5f9652018-05-08 19:38:41 +01002143
2144.. classmethod:: object.__class_getitem__(cls, key)
2145
2146 Return an object representing the specialization of a generic class
2147 by type arguments found in *key*.
2148
2149This method is looked up on the class object itself, and when defined in
2150the class body, this method is implicitly a class method. Note, this
2151mechanism is primarily reserved for use with static type hints, other usage
2152is discouraged.
2153
2154.. seealso::
2155
2156 :pep:`560` - Core support for typing module and generic types
2157
2158
Georg Brandl116aa622007-08-15 14:28:22 +00002159.. _callable-types:
2160
2161Emulating callable objects
2162--------------------------
2163
2164
2165.. method:: object.__call__(self[, args...])
2166
2167 .. index:: pair: call; instance
2168
2169 Called when the instance is "called" as a function; if this method is defined,
Andre Delfino95f710c2020-10-27 13:18:57 -03002170 ``x(arg1, arg2, ...)`` roughly translates to ``type(x).__call__(x, arg1, ...)``.
Georg Brandl116aa622007-08-15 14:28:22 +00002171
2172
2173.. _sequence-types:
2174
2175Emulating container types
2176-------------------------
2177
2178The following methods can be defined to implement container objects. Containers
2179usually are sequences (such as lists or tuples) or mappings (like dictionaries),
2180but can represent other containers as well. The first set of methods is used
2181either to emulate a sequence or to emulate a mapping; the difference is that for
2182a sequence, the allowable keys should be the integers *k* for which ``0 <= k <
2183N`` where *N* is the length of the sequence, or slice objects, which define a
Georg Brandlcb8ecb12007-09-04 06:35:14 +00002184range of items. It is also recommended that mappings provide the methods
Georg Brandlc7723722008-05-26 17:47:11 +00002185:meth:`keys`, :meth:`values`, :meth:`items`, :meth:`get`, :meth:`clear`,
Serhiy Storchaka0d196ed2013-10-09 14:02:31 +03002186:meth:`setdefault`, :meth:`pop`, :meth:`popitem`, :meth:`!copy`, and
Georg Brandlcb8ecb12007-09-04 06:35:14 +00002187:meth:`update` behaving similar to those for Python's standard dictionary
Serhiy Storchaka2e576f52017-04-24 09:05:00 +03002188objects. The :mod:`collections.abc` module provides a
Serhiy Storchaka0d196ed2013-10-09 14:02:31 +03002189:class:`~collections.abc.MutableMapping`
Georg Brandlc7723722008-05-26 17:47:11 +00002190abstract base class to help create those methods from a base set of
2191:meth:`__getitem__`, :meth:`__setitem__`, :meth:`__delitem__`, and :meth:`keys`.
2192Mutable sequences should provide methods :meth:`append`, :meth:`count`,
2193:meth:`index`, :meth:`extend`, :meth:`insert`, :meth:`pop`, :meth:`remove`,
2194:meth:`reverse` and :meth:`sort`, like Python standard list objects. Finally,
2195sequence types should implement addition (meaning concatenation) and
2196multiplication (meaning repetition) by defining the methods :meth:`__add__`,
2197:meth:`__radd__`, :meth:`__iadd__`, :meth:`__mul__`, :meth:`__rmul__` and
2198:meth:`__imul__` described below; they should not define other numerical
2199operators. It is recommended that both mappings and sequences implement the
2200:meth:`__contains__` method to allow efficient use of the ``in`` operator; for
2201mappings, ``in`` should search the mapping's keys; for sequences, it should
2202search through the values. It is further recommended that both mappings and
2203sequences implement the :meth:`__iter__` method to allow efficient iteration
alclarks4544e782019-11-17 22:00:43 +00002204through the container; for mappings, :meth:`__iter__` should iterate
2205through the object's keys; for sequences, it should iterate through the values.
Georg Brandl116aa622007-08-15 14:28:22 +00002206
2207.. method:: object.__len__(self)
2208
2209 .. index::
2210 builtin: len
2211 single: __bool__() (object method)
2212
2213 Called to implement the built-in function :func:`len`. Should return the length
2214 of the object, an integer ``>=`` 0. Also, an object that doesn't define a
2215 :meth:`__bool__` method and whose :meth:`__len__` method returns zero is
2216 considered to be false in a Boolean context.
2217
Serhiy Storchaka85157cd2017-04-23 08:37:58 +03002218 .. impl-detail::
2219
2220 In CPython, the length is required to be at most :attr:`sys.maxsize`.
2221 If the length is larger than :attr:`!sys.maxsize` some features (such as
2222 :func:`len`) may raise :exc:`OverflowError`. To prevent raising
2223 :exc:`!OverflowError` by truth value testing, an object must define a
2224 :meth:`__bool__` method.
2225
Georg Brandl116aa622007-08-15 14:28:22 +00002226
Armin Ronacher74b38b12012-10-07 10:29:32 +02002227.. method:: object.__length_hint__(self)
2228
Ezio Melottie12dc282012-10-07 12:09:36 +03002229 Called to implement :func:`operator.length_hint`. Should return an estimated
Armin Ronacher74b38b12012-10-07 10:29:32 +02002230 length for the object (which may be greater or less than the actual length).
Jeroen Demeyer009ef292019-09-10 16:01:13 +02002231 The length must be an integer ``>=`` 0. The return value may also be
Steve Dowera39a4c72019-09-10 15:25:12 +01002232 :const:`NotImplemented`, which is treated the same as if the
2233 ``__length_hint__`` method didn't exist at all. This method is purely an
2234 optimization and is never required for correctness.
Armin Ronacher74b38b12012-10-07 10:29:32 +02002235
2236 .. versionadded:: 3.4
2237
Serhiy Storchaka85157cd2017-04-23 08:37:58 +03002238
Andre Delfino1ce853f2018-12-05 16:42:44 -03002239.. index:: object: slice
2240
Georg Brandlcb8ecb12007-09-04 06:35:14 +00002241.. note::
2242
2243 Slicing is done exclusively with the following three methods. A call like ::
2244
2245 a[1:2] = b
2246
2247 is translated to ::
2248
2249 a[slice(1, 2, None)] = b
2250
2251 and so forth. Missing slice items are always filled in with ``None``.
2252
2253
Georg Brandl116aa622007-08-15 14:28:22 +00002254.. method:: object.__getitem__(self, key)
2255
Georg Brandl116aa622007-08-15 14:28:22 +00002256 Called to implement evaluation of ``self[key]``. For sequence types, the
2257 accepted keys should be integers and slice objects. Note that the special
2258 interpretation of negative indexes (if the class wishes to emulate a sequence
2259 type) is up to the :meth:`__getitem__` method. If *key* is of an inappropriate
2260 type, :exc:`TypeError` may be raised; if of a value outside the set of indexes
2261 for the sequence (after any special interpretation of negative values),
2262 :exc:`IndexError` should be raised. For mapping types, if *key* is missing (not
2263 in the container), :exc:`KeyError` should be raised.
2264
2265 .. note::
2266
2267 :keyword:`for` loops expect that an :exc:`IndexError` will be raised for illegal
2268 indexes to allow proper detection of the end of the sequence.
2269
2270
2271.. method:: object.__setitem__(self, key, value)
2272
2273 Called to implement assignment to ``self[key]``. Same note as for
2274 :meth:`__getitem__`. This should only be implemented for mappings if the
2275 objects support changes to the values for keys, or if new keys can be added, or
2276 for sequences if elements can be replaced. The same exceptions should be raised
2277 for improper *key* values as for the :meth:`__getitem__` method.
2278
2279
2280.. method:: object.__delitem__(self, key)
2281
2282 Called to implement deletion of ``self[key]``. Same note as for
2283 :meth:`__getitem__`. This should only be implemented for mappings if the
2284 objects support removal of keys, or for sequences if elements can be removed
2285 from the sequence. The same exceptions should be raised for improper *key*
2286 values as for the :meth:`__getitem__` method.
2287
2288
Andre Delfino1ce853f2018-12-05 16:42:44 -03002289.. method:: object.__missing__(self, key)
2290
2291 Called by :class:`dict`\ .\ :meth:`__getitem__` to implement ``self[key]`` for dict subclasses
2292 when key is not in the dictionary.
2293
2294
Georg Brandl116aa622007-08-15 14:28:22 +00002295.. method:: object.__iter__(self)
2296
2297 This method is called when an iterator is required for a container. This method
2298 should return a new iterator object that can iterate over all the objects in the
R David Murrayc9f5f2d2014-12-10 09:51:01 -05002299 container. For mappings, it should iterate over the keys of the container.
Georg Brandl116aa622007-08-15 14:28:22 +00002300
2301 Iterator objects also need to implement this method; they are required to return
2302 themselves. For more information on iterator objects, see :ref:`typeiter`.
2303
Christian Heimes7f044312008-01-06 17:05:40 +00002304
2305.. method:: object.__reversed__(self)
2306
Georg Brandl22b34312009-07-26 14:54:51 +00002307 Called (if present) by the :func:`reversed` built-in to implement
Christian Heimes7f044312008-01-06 17:05:40 +00002308 reverse iteration. It should return a new iterator object that iterates
2309 over all the objects in the container in reverse order.
2310
Georg Brandl8a1e4c42009-05-25 21:13:36 +00002311 If the :meth:`__reversed__` method is not provided, the :func:`reversed`
Georg Brandl22b34312009-07-26 14:54:51 +00002312 built-in will fall back to using the sequence protocol (:meth:`__len__` and
Georg Brandl8a1e4c42009-05-25 21:13:36 +00002313 :meth:`__getitem__`). Objects that support the sequence protocol should
2314 only provide :meth:`__reversed__` if they can provide an implementation
2315 that is more efficient than the one provided by :func:`reversed`.
Christian Heimes7f044312008-01-06 17:05:40 +00002316
2317
Georg Brandl116aa622007-08-15 14:28:22 +00002318The membership test operators (:keyword:`in` and :keyword:`not in`) are normally
alclarks4544e782019-11-17 22:00:43 +00002319implemented as an iteration through a container. However, container objects can
Georg Brandl116aa622007-08-15 14:28:22 +00002320supply the following special method with a more efficient implementation, which
alclarks4544e782019-11-17 22:00:43 +00002321also does not require the object be iterable.
Georg Brandl116aa622007-08-15 14:28:22 +00002322
Georg Brandl116aa622007-08-15 14:28:22 +00002323.. method:: object.__contains__(self, item)
2324
Georg Brandl495f7b52009-10-27 15:28:25 +00002325 Called to implement membership test operators. Should return true if *item*
2326 is in *self*, false otherwise. For mapping objects, this should consider the
2327 keys of the mapping rather than the values or the key-item pairs.
2328
2329 For objects that don't define :meth:`__contains__`, the membership test first
2330 tries iteration via :meth:`__iter__`, then the old sequence iteration
2331 protocol via :meth:`__getitem__`, see :ref:`this section in the language
2332 reference <membership-test-details>`.
Georg Brandl116aa622007-08-15 14:28:22 +00002333
2334
Georg Brandl116aa622007-08-15 14:28:22 +00002335.. _numeric-types:
2336
2337Emulating numeric types
2338-----------------------
2339
2340The following methods can be defined to emulate numeric objects. Methods
2341corresponding to operations that are not supported by the particular kind of
2342number implemented (e.g., bitwise operations for non-integral numbers) should be
2343left undefined.
2344
2345
2346.. method:: object.__add__(self, other)
2347 object.__sub__(self, other)
2348 object.__mul__(self, other)
Benjamin Petersond51374e2014-04-09 23:55:56 -04002349 object.__matmul__(self, other)
Georg Brandlae55dc02008-09-06 17:43:49 +00002350 object.__truediv__(self, other)
Georg Brandl116aa622007-08-15 14:28:22 +00002351 object.__floordiv__(self, other)
2352 object.__mod__(self, other)
2353 object.__divmod__(self, other)
2354 object.__pow__(self, other[, modulo])
2355 object.__lshift__(self, other)
2356 object.__rshift__(self, other)
2357 object.__and__(self, other)
2358 object.__xor__(self, other)
2359 object.__or__(self, other)
2360
2361 .. index::
2362 builtin: divmod
2363 builtin: pow
2364 builtin: pow
2365
Benjamin Petersond51374e2014-04-09 23:55:56 -04002366 These methods are called to implement the binary arithmetic operations
2367 (``+``, ``-``, ``*``, ``@``, ``/``, ``//``, ``%``, :func:`divmod`,
2368 :func:`pow`, ``**``, ``<<``, ``>>``, ``&``, ``^``, ``|``). For instance, to
2369 evaluate the expression ``x + y``, where *x* is an instance of a class that
2370 has an :meth:`__add__` method, ``x.__add__(y)`` is called. The
2371 :meth:`__divmod__` method should be the equivalent to using
2372 :meth:`__floordiv__` and :meth:`__mod__`; it should not be related to
2373 :meth:`__truediv__`. Note that :meth:`__pow__` should be defined to accept
2374 an optional third argument if the ternary version of the built-in :func:`pow`
2375 function is to be supported.
Georg Brandl116aa622007-08-15 14:28:22 +00002376
2377 If one of those methods does not support the operation with the supplied
2378 arguments, it should return ``NotImplemented``.
2379
2380
Georg Brandl116aa622007-08-15 14:28:22 +00002381.. method:: object.__radd__(self, other)
2382 object.__rsub__(self, other)
2383 object.__rmul__(self, other)
Benjamin Petersond51374e2014-04-09 23:55:56 -04002384 object.__rmatmul__(self, other)
Georg Brandl116aa622007-08-15 14:28:22 +00002385 object.__rtruediv__(self, other)
2386 object.__rfloordiv__(self, other)
2387 object.__rmod__(self, other)
2388 object.__rdivmod__(self, other)
HongWeipengabc0c4f2020-01-05 16:20:29 -06002389 object.__rpow__(self, other[, modulo])
Georg Brandl116aa622007-08-15 14:28:22 +00002390 object.__rlshift__(self, other)
2391 object.__rrshift__(self, other)
2392 object.__rand__(self, other)
2393 object.__rxor__(self, other)
2394 object.__ror__(self, other)
2395
2396 .. index::
2397 builtin: divmod
2398 builtin: pow
2399
Benjamin Petersond51374e2014-04-09 23:55:56 -04002400 These methods are called to implement the binary arithmetic operations
2401 (``+``, ``-``, ``*``, ``@``, ``/``, ``//``, ``%``, :func:`divmod`,
2402 :func:`pow`, ``**``, ``<<``, ``>>``, ``&``, ``^``, ``|``) with reflected
2403 (swapped) operands. These functions are only called if the left operand does
Guido van Rossum97c1adf2016-08-18 09:22:23 -07002404 not support the corresponding operation [#]_ and the operands are of different
Benjamin Petersond51374e2014-04-09 23:55:56 -04002405 types. [#]_ For instance, to evaluate the expression ``x - y``, where *y* is
2406 an instance of a class that has an :meth:`__rsub__` method, ``y.__rsub__(x)``
2407 is called if ``x.__sub__(y)`` returns *NotImplemented*.
Georg Brandl116aa622007-08-15 14:28:22 +00002408
2409 .. index:: builtin: pow
2410
2411 Note that ternary :func:`pow` will not try calling :meth:`__rpow__` (the
2412 coercion rules would become too complicated).
2413
2414 .. note::
2415
Brett Cannond02d8242020-10-05 09:42:21 -07002416 If the right operand's type is a subclass of the left operand's type and
2417 that subclass provides a different implementation of the reflected method
2418 for the operation, this method will be called before the left operand's
2419 non-reflected method. This behavior allows subclasses to override their
2420 ancestors' operations.
Georg Brandl116aa622007-08-15 14:28:22 +00002421
2422
2423.. method:: object.__iadd__(self, other)
2424 object.__isub__(self, other)
2425 object.__imul__(self, other)
Benjamin Petersond51374e2014-04-09 23:55:56 -04002426 object.__imatmul__(self, other)
Georg Brandl116aa622007-08-15 14:28:22 +00002427 object.__itruediv__(self, other)
2428 object.__ifloordiv__(self, other)
2429 object.__imod__(self, other)
2430 object.__ipow__(self, other[, modulo])
2431 object.__ilshift__(self, other)
2432 object.__irshift__(self, other)
2433 object.__iand__(self, other)
2434 object.__ixor__(self, other)
2435 object.__ior__(self, other)
2436
Benjamin Petersonb58dda72009-01-18 22:27:04 +00002437 These methods are called to implement the augmented arithmetic assignments
Benjamin Petersond51374e2014-04-09 23:55:56 -04002438 (``+=``, ``-=``, ``*=``, ``@=``, ``/=``, ``//=``, ``%=``, ``**=``, ``<<=``,
2439 ``>>=``, ``&=``, ``^=``, ``|=``). These methods should attempt to do the
2440 operation in-place (modifying *self*) and return the result (which could be,
2441 but does not have to be, *self*). If a specific method is not defined, the
2442 augmented assignment falls back to the normal methods. For instance, if *x*
2443 is an instance of a class with an :meth:`__iadd__` method, ``x += y`` is
2444 equivalent to ``x = x.__iadd__(y)`` . Otherwise, ``x.__add__(y)`` and
2445 ``y.__radd__(x)`` are considered, as with the evaluation of ``x + y``. In
2446 certain situations, augmented assignment can result in unexpected errors (see
2447 :ref:`faq-augmented-assignment-tuple-error`), but this behavior is in fact
2448 part of the data model.
Georg Brandl116aa622007-08-15 14:28:22 +00002449
2450
2451.. method:: object.__neg__(self)
2452 object.__pos__(self)
2453 object.__abs__(self)
2454 object.__invert__(self)
2455
2456 .. index:: builtin: abs
2457
2458 Called to implement the unary arithmetic operations (``-``, ``+``, :func:`abs`
2459 and ``~``).
2460
2461
2462.. method:: object.__complex__(self)
2463 object.__int__(self)
Georg Brandl116aa622007-08-15 14:28:22 +00002464 object.__float__(self)
2465
2466 .. index::
2467 builtin: complex
2468 builtin: int
Georg Brandl116aa622007-08-15 14:28:22 +00002469 builtin: float
2470
Mark Summerfield9557f602008-07-01 14:42:30 +00002471 Called to implement the built-in functions :func:`complex`,
Serhiy Storchaka496431f2018-03-10 17:18:32 +02002472 :func:`int` and :func:`float`. Should return a value
Mark Summerfield9557f602008-07-01 14:42:30 +00002473 of the appropriate type.
Georg Brandl116aa622007-08-15 14:28:22 +00002474
2475
2476.. method:: object.__index__(self)
2477
Ethan Furmandf3ed242014-01-05 06:50:30 -08002478 Called to implement :func:`operator.index`, and whenever Python needs to
2479 losslessly convert the numeric object to an integer object (such as in
2480 slicing, or in the built-in :func:`bin`, :func:`hex` and :func:`oct`
2481 functions). Presence of this method indicates that the numeric object is
2482 an integer type. Must return an integer.
2483
Serhiy Storchakabdbad712019-06-02 00:05:48 +03002484 If :meth:`__int__`, :meth:`__float__` and :meth:`__complex__` are not
2485 defined then corresponding built-in functions :func:`int`, :func:`float`
2486 and :func:`complex` fall back to :meth:`__index__`.
Georg Brandl116aa622007-08-15 14:28:22 +00002487
Georg Brandl116aa622007-08-15 14:28:22 +00002488
Serhiy Storchaka496431f2018-03-10 17:18:32 +02002489.. method:: object.__round__(self, [,ndigits])
2490 object.__trunc__(self)
2491 object.__floor__(self)
2492 object.__ceil__(self)
2493
2494 .. index:: builtin: round
2495
2496 Called to implement the built-in function :func:`round` and :mod:`math`
2497 functions :func:`~math.trunc`, :func:`~math.floor` and :func:`~math.ceil`.
2498 Unless *ndigits* is passed to :meth:`!__round__` all these methods should
2499 return the value of the object truncated to an :class:`~numbers.Integral`
2500 (typically an :class:`int`).
2501
2502 If :meth:`__int__` is not defined then the built-in function :func:`int`
2503 falls back to :meth:`__trunc__`.
2504
2505
Georg Brandl116aa622007-08-15 14:28:22 +00002506.. _context-managers:
2507
2508With Statement Context Managers
2509-------------------------------
2510
Georg Brandl116aa622007-08-15 14:28:22 +00002511A :dfn:`context manager` is an object that defines the runtime context to be
2512established when executing a :keyword:`with` statement. The context manager
2513handles the entry into, and the exit from, the desired runtime context for the
2514execution of the block of code. Context managers are normally invoked using the
Serhiy Storchaka2b57c432018-12-19 08:09:46 +02002515:keyword:`!with` statement (described in section :ref:`with`), but can also be
Georg Brandl116aa622007-08-15 14:28:22 +00002516used by directly invoking their methods.
2517
2518.. index::
2519 statement: with
2520 single: context manager
2521
2522Typical uses of context managers include saving and restoring various kinds of
2523global state, locking and unlocking resources, closing opened files, etc.
2524
2525For more information on context managers, see :ref:`typecontextmanager`.
2526
2527
2528.. method:: object.__enter__(self)
2529
2530 Enter the runtime context related to this object. The :keyword:`with` statement
2531 will bind this method's return value to the target(s) specified in the
Serhiy Storchaka2b57c432018-12-19 08:09:46 +02002532 :keyword:`!as` clause of the statement, if any.
Georg Brandl116aa622007-08-15 14:28:22 +00002533
2534
2535.. method:: object.__exit__(self, exc_type, exc_value, traceback)
2536
2537 Exit the runtime context related to this object. The parameters describe the
2538 exception that caused the context to be exited. If the context was exited
2539 without an exception, all three arguments will be :const:`None`.
2540
2541 If an exception is supplied, and the method wishes to suppress the exception
2542 (i.e., prevent it from being propagated), it should return a true value.
2543 Otherwise, the exception will be processed normally upon exit from this method.
2544
2545 Note that :meth:`__exit__` methods should not reraise the passed-in exception;
2546 this is the caller's responsibility.
2547
2548
2549.. seealso::
2550
Serhiy Storchakae4ba8722016-03-31 15:30:54 +03002551 :pep:`343` - The "with" statement
Georg Brandl116aa622007-08-15 14:28:22 +00002552 The specification, background, and examples for the Python :keyword:`with`
2553 statement.
2554
Nick Coghlan3a5d7e32008-08-31 12:40:14 +00002555
Daniel F Moisseta22bca62021-03-01 04:08:38 +00002556.. _class-pattern-matching:
2557
2558Customizing positional arguments in class pattern matching
2559----------------------------------------------------------
2560
2561When using a class name in a pattern, positional arguments in the pattern are not
2562allowed by default, i.e. ``case MyClass(x, y)`` is typically invalid without special
2563support in ``MyClass``. To be able to use that kind of patterns, the class needs to
2564define a *__match_args__* attribute.
2565
2566.. data:: object.__match_args__
2567
2568 This class variable can be assigned a tuple or list of strings. When this class is
2569 used in a class pattern with positional arguments, each positional argument will
2570 be converted into a keyword argument, using the corresponding value in
2571 *__match_args__* as the keyword. The absence of this attribute is equivalent to
2572 setting it to ``()``.
2573
2574For example, if ``MyClass.__match_args__`` is ``("left", "center", "right")`` that means
2575that ``case MyClass(x, y)`` is equivalent to ``case MyClass(left=x, center=y)``. Note
2576that the number of arguments in the pattern must be smaller than or equal to the number
2577of elements in *__match_args__*; if it is larger, the pattern match attempt will raise
2578a :exc:`TypeError`.
2579
2580.. versionadded:: 3.10
2581
2582.. seealso::
2583
2584 :pep:`634` - Structural Pattern Matching
2585 The specification for the Python ``match`` statement.
2586
2587
Nick Coghlan3a5d7e32008-08-31 12:40:14 +00002588.. _special-lookup:
2589
2590Special method lookup
2591---------------------
2592
2593For custom classes, implicit invocations of special methods are only guaranteed
2594to work correctly if defined on an object's type, not in the object's instance
2595dictionary. That behaviour is the reason why the following code raises an
2596exception::
2597
Éric Araujo28053fb2010-11-22 03:09:19 +00002598 >>> class C:
Nick Coghlan3a5d7e32008-08-31 12:40:14 +00002599 ... pass
2600 ...
2601 >>> c = C()
2602 >>> c.__len__ = lambda: 5
2603 >>> len(c)
2604 Traceback (most recent call last):
2605 File "<stdin>", line 1, in <module>
2606 TypeError: object of type 'C' has no len()
2607
2608The rationale behind this behaviour lies with a number of special methods such
2609as :meth:`__hash__` and :meth:`__repr__` that are implemented by all objects,
2610including type objects. If the implicit lookup of these methods used the
2611conventional lookup process, they would fail when invoked on the type object
2612itself::
2613
2614 >>> 1 .__hash__() == hash(1)
2615 True
2616 >>> int.__hash__() == hash(int)
2617 Traceback (most recent call last):
2618 File "<stdin>", line 1, in <module>
2619 TypeError: descriptor '__hash__' of 'int' object needs an argument
2620
2621Incorrectly attempting to invoke an unbound method of a class in this way is
2622sometimes referred to as 'metaclass confusion', and is avoided by bypassing
2623the instance when looking up special methods::
2624
2625 >>> type(1).__hash__(1) == hash(1)
2626 True
2627 >>> type(int).__hash__(int) == hash(int)
2628 True
2629
2630In addition to bypassing any instance attributes in the interest of
Georg Brandlaf265f42008-12-07 15:06:20 +00002631correctness, implicit special method lookup generally also bypasses the
Nick Coghlan3a5d7e32008-08-31 12:40:14 +00002632:meth:`__getattribute__` method even of the object's metaclass::
2633
2634 >>> class Meta(type):
Berker Peksag770319d2015-04-11 14:59:30 +03002635 ... def __getattribute__(*args):
2636 ... print("Metaclass getattribute invoked")
2637 ... return type.__getattribute__(*args)
Nick Coghlan3a5d7e32008-08-31 12:40:14 +00002638 ...
Benjamin Petersone348d1a2008-10-19 21:29:05 +00002639 >>> class C(object, metaclass=Meta):
Nick Coghlan3a5d7e32008-08-31 12:40:14 +00002640 ... def __len__(self):
2641 ... return 10
2642 ... def __getattribute__(*args):
Benjamin Peterson64106fb2008-10-29 20:35:35 +00002643 ... print("Class getattribute invoked")
Nick Coghlan3a5d7e32008-08-31 12:40:14 +00002644 ... return object.__getattribute__(*args)
2645 ...
2646 >>> c = C()
2647 >>> c.__len__() # Explicit lookup via instance
2648 Class getattribute invoked
2649 10
2650 >>> type(c).__len__(c) # Explicit lookup via type
2651 Metaclass getattribute invoked
2652 10
2653 >>> len(c) # Implicit lookup
2654 10
2655
2656Bypassing the :meth:`__getattribute__` machinery in this fashion
2657provides significant scope for speed optimisations within the
2658interpreter, at the cost of some flexibility in the handling of
2659special methods (the special method *must* be set on the class
2660object itself in order to be consistently invoked by the interpreter).
2661
2662
Yury Selivanov66f88282015-06-24 11:04:15 -04002663.. index::
2664 single: coroutine
Yury Selivanovf3e40fa2015-05-21 11:50:30 -04002665
2666Coroutines
2667==========
2668
Yury Selivanovf3e40fa2015-05-21 11:50:30 -04002669
2670Awaitable Objects
2671-----------------
2672
Yury Selivanov66f88282015-06-24 11:04:15 -04002673An :term:`awaitable` object generally implements an :meth:`__await__` method.
Victor Stinnerbb0b0852020-08-14 12:20:05 +02002674:term:`Coroutine objects <coroutine>` returned from :keyword:`async def` functions
Yury Selivanov66f88282015-06-24 11:04:15 -04002675are awaitable.
Yury Selivanovf3e40fa2015-05-21 11:50:30 -04002676
Yury Selivanov66f88282015-06-24 11:04:15 -04002677.. note::
Yury Selivanovf3e40fa2015-05-21 11:50:30 -04002678
Yury Selivanov66f88282015-06-24 11:04:15 -04002679 The :term:`generator iterator` objects returned from generators
2680 decorated with :func:`types.coroutine` or :func:`asyncio.coroutine`
2681 are also awaitable, but they do not implement :meth:`__await__`.
Yury Selivanovf3e40fa2015-05-21 11:50:30 -04002682
2683.. method:: object.__await__(self)
2684
2685 Must return an :term:`iterator`. Should be used to implement
2686 :term:`awaitable` objects. For instance, :class:`asyncio.Future` implements
2687 this method to be compatible with the :keyword:`await` expression.
2688
2689.. versionadded:: 3.5
2690
2691.. seealso:: :pep:`492` for additional information about awaitable objects.
2692
2693
Yury Selivanov66f88282015-06-24 11:04:15 -04002694.. _coroutine-objects:
2695
2696Coroutine Objects
2697-----------------
2698
Victor Stinnerbb0b0852020-08-14 12:20:05 +02002699:term:`Coroutine objects <coroutine>` are :term:`awaitable` objects.
Yury Selivanov66f88282015-06-24 11:04:15 -04002700A coroutine's execution can be controlled by calling :meth:`__await__` and
2701iterating over the result. When the coroutine has finished executing and
2702returns, the iterator raises :exc:`StopIteration`, and the exception's
2703:attr:`~StopIteration.value` attribute holds the return value. If the
2704coroutine raises an exception, it is propagated by the iterator. Coroutines
2705should not directly raise unhandled :exc:`StopIteration` exceptions.
2706
2707Coroutines also have the methods listed below, which are analogous to
2708those of generators (see :ref:`generator-methods`). However, unlike
2709generators, coroutines do not directly support iteration.
2710
Yury Selivanov77c96812016-02-13 17:59:05 -05002711.. versionchanged:: 3.5.2
2712 It is a :exc:`RuntimeError` to await on a coroutine more than once.
2713
2714
Yury Selivanov66f88282015-06-24 11:04:15 -04002715.. method:: coroutine.send(value)
2716
2717 Starts or resumes execution of the coroutine. If *value* is ``None``,
2718 this is equivalent to advancing the iterator returned by
2719 :meth:`__await__`. If *value* is not ``None``, this method delegates
2720 to the :meth:`~generator.send` method of the iterator that caused
2721 the coroutine to suspend. The result (return value,
2722 :exc:`StopIteration`, or other exception) is the same as when
2723 iterating over the :meth:`__await__` return value, described above.
2724
2725.. method:: coroutine.throw(type[, value[, traceback]])
2726
2727 Raises the specified exception in the coroutine. This method delegates
2728 to the :meth:`~generator.throw` method of the iterator that caused
2729 the coroutine to suspend, if it has such a method. Otherwise,
2730 the exception is raised at the suspension point. The result
2731 (return value, :exc:`StopIteration`, or other exception) is the same as
2732 when iterating over the :meth:`__await__` return value, described
2733 above. If the exception is not caught in the coroutine, it propagates
2734 back to the caller.
2735
2736.. method:: coroutine.close()
2737
2738 Causes the coroutine to clean itself up and exit. If the coroutine
2739 is suspended, this method first delegates to the :meth:`~generator.close`
2740 method of the iterator that caused the coroutine to suspend, if it
2741 has such a method. Then it raises :exc:`GeneratorExit` at the
2742 suspension point, causing the coroutine to immediately clean itself up.
2743 Finally, the coroutine is marked as having finished executing, even if
2744 it was never started.
2745
2746 Coroutine objects are automatically closed using the above process when
2747 they are about to be destroyed.
2748
Yury Selivanova6f6edb2016-06-09 15:08:31 -04002749.. _async-iterators:
Yury Selivanov66f88282015-06-24 11:04:15 -04002750
Yury Selivanovf3e40fa2015-05-21 11:50:30 -04002751Asynchronous Iterators
2752----------------------
2753
Yury Selivanovfaa135a2017-10-06 02:08:57 -04002754An *asynchronous iterator* can call asynchronous code in
2755its ``__anext__`` method.
Yury Selivanovf3e40fa2015-05-21 11:50:30 -04002756
Martin Panterd2ad5712015-11-02 04:20:33 +00002757Asynchronous iterators can be used in an :keyword:`async for` statement.
Yury Selivanovf3e40fa2015-05-21 11:50:30 -04002758
2759.. method:: object.__aiter__(self)
2760
Yury Selivanova6f6edb2016-06-09 15:08:31 -04002761 Must return an *asynchronous iterator* object.
Yury Selivanovf3e40fa2015-05-21 11:50:30 -04002762
2763.. method:: object.__anext__(self)
2764
2765 Must return an *awaitable* resulting in a next value of the iterator. Should
2766 raise a :exc:`StopAsyncIteration` error when the iteration is over.
2767
2768An example of an asynchronous iterable object::
2769
2770 class Reader:
2771 async def readline(self):
2772 ...
2773
Yury Selivanova6f6edb2016-06-09 15:08:31 -04002774 def __aiter__(self):
Yury Selivanovf3e40fa2015-05-21 11:50:30 -04002775 return self
2776
2777 async def __anext__(self):
2778 val = await self.readline()
2779 if val == b'':
2780 raise StopAsyncIteration
2781 return val
2782
2783.. versionadded:: 3.5
2784
Yury Selivanovfaa135a2017-10-06 02:08:57 -04002785.. versionchanged:: 3.7
2786 Prior to Python 3.7, ``__aiter__`` could return an *awaitable*
2787 that would resolve to an
2788 :term:`asynchronous iterator <asynchronous iterator>`.
Yury Selivanova6f6edb2016-06-09 15:08:31 -04002789
Yury Selivanovfaa135a2017-10-06 02:08:57 -04002790 Starting with Python 3.7, ``__aiter__`` must return an
2791 asynchronous iterator object. Returning anything else
2792 will result in a :exc:`TypeError` error.
Yury Selivanova6f6edb2016-06-09 15:08:31 -04002793
Yury Selivanovf3e40fa2015-05-21 11:50:30 -04002794
Jelle Zijlstra2e624692017-04-30 18:25:58 -07002795.. _async-context-managers:
2796
Yury Selivanovf3e40fa2015-05-21 11:50:30 -04002797Asynchronous Context Managers
2798-----------------------------
2799
2800An *asynchronous context manager* is a *context manager* that is able to
2801suspend execution in its ``__aenter__`` and ``__aexit__`` methods.
2802
Martin Panterd2ad5712015-11-02 04:20:33 +00002803Asynchronous context managers can be used in an :keyword:`async with` statement.
Yury Selivanovf3e40fa2015-05-21 11:50:30 -04002804
2805.. method:: object.__aenter__(self)
2806
Andre Delfinoa8a79ca2019-05-03 12:08:10 -03002807 Semantically similar to :meth:`__enter__`, the only
2808 difference being that it must return an *awaitable*.
Yury Selivanovf3e40fa2015-05-21 11:50:30 -04002809
2810.. method:: object.__aexit__(self, exc_type, exc_value, traceback)
2811
Andre Delfinoa8a79ca2019-05-03 12:08:10 -03002812 Semantically similar to :meth:`__exit__`, the only
2813 difference being that it must return an *awaitable*.
Yury Selivanovf3e40fa2015-05-21 11:50:30 -04002814
2815An example of an asynchronous context manager class::
2816
2817 class AsyncContextManager:
2818 async def __aenter__(self):
2819 await log('entering context')
2820
2821 async def __aexit__(self, exc_type, exc, tb):
2822 await log('exiting context')
2823
2824.. versionadded:: 3.5
2825
2826
Georg Brandl116aa622007-08-15 14:28:22 +00002827.. rubric:: Footnotes
2828
Nick Coghlan3a5d7e32008-08-31 12:40:14 +00002829.. [#] It *is* possible in some cases to change an object's type, under certain
2830 controlled conditions. It generally isn't a good idea though, since it can
2831 lead to some very strange behaviour if it is handled incorrectly.
2832
Guido van Rossum97c1adf2016-08-18 09:22:23 -07002833.. [#] The :meth:`__hash__`, :meth:`__iter__`, :meth:`__reversed__`, and
2834 :meth:`__contains__` methods have special handling for this; others
2835 will still raise a :exc:`TypeError`, but may do so by relying on
2836 the behavior that ``None`` is not callable.
2837
2838.. [#] "Does not support" here means that the class has no such method, or
2839 the method returns ``NotImplemented``. Do not set the method to
2840 ``None`` if you want to force fallback to the right operand's reflected
Martin Panter28540182016-11-21 04:10:45 +00002841 method—that will instead have the opposite effect of explicitly
Guido van Rossum97c1adf2016-08-18 09:22:23 -07002842 *blocking* such fallback.
2843
Brett Cannond02d8242020-10-05 09:42:21 -07002844.. [#] For operands of the same type, it is assumed that if the non-reflected
2845 method -- such as :meth:`__add__` -- fails then the overall operation is not
2846 supported, which is why the reflected method is not called.