blob: c09a578d255cd3c6187c0fe9b2e8dad9f1b4ddbc [file] [log] [blame]
Georg Brandl116aa622007-08-15 14:28:22 +00001
2.. _datamodel:
3
4**********
5Data model
6**********
7
8
9.. _objects:
10
11Objects, values and types
12=========================
13
14.. index::
15 single: object
16 single: data
17
18:dfn:`Objects` are Python's abstraction for data. All data in a Python program
19is represented by objects or by relations between objects. (In a sense, and in
20conformance to Von Neumann's model of a "stored program computer," code is also
21represented by objects.)
22
23.. index::
24 builtin: id
25 builtin: type
26 single: identity of an object
27 single: value of an object
28 single: type of an object
29 single: mutable object
30 single: immutable object
31
Georg Brandl85eb8c12007-08-31 16:33:38 +000032.. XXX it *is* now possible in some cases to change an object's
33 type, under certain controlled conditions
34
Georg Brandl116aa622007-08-15 14:28:22 +000035Every object has an identity, a type and a value. An object's *identity* never
36changes once it has been created; you may think of it as the object's address in
37memory. The ':keyword:`is`' operator compares the identity of two objects; the
Nick Coghlan337b2bf2012-05-20 18:30:49 +100038:func:`id` function returns an integer representing its identity.
39
40.. impl-detail::
41
42 For CPython, ``id(x)`` is the memory address where ``x`` is stored.
43
Georg Brandl116aa622007-08-15 14:28:22 +000044An object's type determines the operations that the object supports (e.g., "does
45it have a length?") and also defines the possible values for objects of that
46type. The :func:`type` function returns an object's type (which is an object
Nick Coghlan337b2bf2012-05-20 18:30:49 +100047itself). Like its identity, an object's :dfn:`type` is also unchangeable.
48[#]_
49
50The *value* of some objects can change. Objects whose value can
Georg Brandl116aa622007-08-15 14:28:22 +000051change are said to be *mutable*; objects whose value is unchangeable once they
52are created are called *immutable*. (The value of an immutable container object
53that contains a reference to a mutable object can change when the latter's value
54is changed; however the container is still considered immutable, because the
55collection of objects it contains cannot be changed. So, immutability is not
56strictly the same as having an unchangeable value, it is more subtle.) An
57object's mutability is determined by its type; for instance, numbers, strings
58and tuples are immutable, while dictionaries and lists are mutable.
59
60.. index::
61 single: garbage collection
62 single: reference counting
63 single: unreachable object
64
65Objects are never explicitly destroyed; however, when they become unreachable
66they may be garbage-collected. An implementation is allowed to postpone garbage
67collection or omit it altogether --- it is a matter of implementation quality
68how garbage collection is implemented, as long as no objects are collected that
Georg Brandl495f7b52009-10-27 15:28:25 +000069are still reachable.
70
71.. impl-detail::
72
73 CPython currently uses a reference-counting scheme with (optional) delayed
74 detection of cyclically linked garbage, which collects most objects as soon
75 as they become unreachable, but is not guaranteed to collect garbage
76 containing circular references. See the documentation of the :mod:`gc`
77 module for information on controlling the collection of cyclic garbage.
78 Other implementations act differently and CPython may change.
Gregory P. Smithc5425472011-03-10 11:28:50 -080079 Do not depend on immediate finalization of objects when they become
80 unreachable (ex: always close files).
Georg Brandl116aa622007-08-15 14:28:22 +000081
82Note that the use of the implementation's tracing or debugging facilities may
83keep objects alive that would normally be collectable. Also note that catching
84an exception with a ':keyword:`try`...\ :keyword:`except`' statement may keep
85objects alive.
86
87Some objects contain references to "external" resources such as open files or
88windows. It is understood that these resources are freed when the object is
89garbage-collected, but since garbage collection is not guaranteed to happen,
90such objects also provide an explicit way to release the external resource,
91usually a :meth:`close` method. Programs are strongly recommended to explicitly
92close such objects. The ':keyword:`try`...\ :keyword:`finally`' statement
Nick Coghlan3a5d7e32008-08-31 12:40:14 +000093and the ':keyword:`with`' statement provide convenient ways to do this.
Georg Brandl116aa622007-08-15 14:28:22 +000094
95.. index:: single: container
96
97Some objects contain references to other objects; these are called *containers*.
98Examples of containers are tuples, lists and dictionaries. The references are
99part of a container's value. In most cases, when we talk about the value of a
100container, we imply the values, not the identities of the contained objects;
101however, when we talk about the mutability of a container, only the identities
102of the immediately contained objects are implied. So, if an immutable container
103(like a tuple) contains a reference to a mutable object, its value changes if
104that mutable object is changed.
105
106Types affect almost all aspects of object behavior. Even the importance of
107object identity is affected in some sense: for immutable types, operations that
108compute new values may actually return a reference to any existing object with
109the same type and value, while for mutable objects this is not allowed. E.g.,
110after ``a = 1; b = 1``, ``a`` and ``b`` may or may not refer to the same object
111with the value one, depending on the implementation, but after ``c = []; d =
112[]``, ``c`` and ``d`` are guaranteed to refer to two different, unique, newly
113created empty lists. (Note that ``c = d = []`` assigns the same object to both
114``c`` and ``d``.)
115
116
117.. _types:
118
119The standard type hierarchy
120===========================
121
122.. index::
123 single: type
124 pair: data; type
125 pair: type; hierarchy
126 pair: extension; module
127 pair: C; language
128
129Below is a list of the types that are built into Python. Extension modules
130(written in C, Java, or other languages, depending on the implementation) can
131define additional types. Future versions of Python may add types to the type
Nick Coghlan3a5d7e32008-08-31 12:40:14 +0000132hierarchy (e.g., rational numbers, efficiently stored arrays of integers, etc.),
133although such additions will often be provided via the standard library instead.
Georg Brandl116aa622007-08-15 14:28:22 +0000134
135.. index::
136 single: attribute
137 pair: special; attribute
138 triple: generic; special; attribute
139
140Some of the type descriptions below contain a paragraph listing 'special
141attributes.' These are attributes that provide access to the implementation and
142are not intended for general use. Their definition may change in the future.
143
144None
145 .. index:: object: None
146
147 This type has a single value. There is a single object with this value. This
148 object is accessed through the built-in name ``None``. It is used to signify the
149 absence of a value in many situations, e.g., it is returned from functions that
150 don't explicitly return anything. Its truth value is false.
151
152NotImplemented
153 .. index:: object: NotImplemented
154
155 This type has a single value. There is a single object with this value. This
156 object is accessed through the built-in name ``NotImplemented``. Numeric methods
157 and rich comparison methods may return this value if they do not implement the
158 operation for the operands provided. (The interpreter will then try the
159 reflected operation, or some other fallback, depending on the operator.) Its
160 truth value is true.
161
162Ellipsis
163 .. index:: object: Ellipsis
164
165 This type has a single value. There is a single object with this value. This
166 object is accessed through the literal ``...`` or the built-in name
167 ``Ellipsis``. Its truth value is true.
168
Christian Heimes072c0f12008-01-03 23:01:04 +0000169:class:`numbers.Number`
Georg Brandl116aa622007-08-15 14:28:22 +0000170 .. index:: object: numeric
171
172 These are created by numeric literals and returned as results by arithmetic
173 operators and arithmetic built-in functions. Numeric objects are immutable;
174 once created their value never changes. Python numbers are of course strongly
175 related to mathematical numbers, but subject to the limitations of numerical
176 representation in computers.
177
178 Python distinguishes between integers, floating point numbers, and complex
179 numbers:
180
Christian Heimes072c0f12008-01-03 23:01:04 +0000181 :class:`numbers.Integral`
Georg Brandl116aa622007-08-15 14:28:22 +0000182 .. index:: object: integer
183
184 These represent elements from the mathematical set of integers (positive and
185 negative).
186
Georg Brandl59d69162008-01-07 09:27:36 +0000187 There are two types of integers:
Georg Brandl116aa622007-08-15 14:28:22 +0000188
Nick Coghlan3a5d7e32008-08-31 12:40:14 +0000189 Integers (:class:`int`)
Georg Brandl116aa622007-08-15 14:28:22 +0000190
Georg Brandl116aa622007-08-15 14:28:22 +0000191 These represent numbers in an unlimited range, subject to available (virtual)
192 memory only. For the purpose of shift and mask operations, a binary
193 representation is assumed, and negative numbers are represented in a variant of
194 2's complement which gives the illusion of an infinite string of sign bits
195 extending to the left.
196
Nick Coghlan3a5d7e32008-08-31 12:40:14 +0000197 Booleans (:class:`bool`)
Georg Brandl116aa622007-08-15 14:28:22 +0000198 .. index::
199 object: Boolean
200 single: False
201 single: True
202
203 These represent the truth values False and True. The two objects representing
Serhiy Storchakafbc1c262013-11-29 12:17:13 +0200204 the values ``False`` and ``True`` are the only Boolean objects. The Boolean type is a
Georg Brandl95817b32008-05-11 14:30:18 +0000205 subtype of the integer type, and Boolean values behave like the values 0 and 1,
Georg Brandl116aa622007-08-15 14:28:22 +0000206 respectively, in almost all contexts, the exception being that when converted to
207 a string, the strings ``"False"`` or ``"True"`` are returned, respectively.
208
209 .. index:: pair: integer; representation
210
211 The rules for integer representation are intended to give the most meaningful
Georg Brandlbb74a782008-05-11 10:53:16 +0000212 interpretation of shift and mask operations involving negative integers.
Georg Brandl116aa622007-08-15 14:28:22 +0000213
Christian Heimes072c0f12008-01-03 23:01:04 +0000214 :class:`numbers.Real` (:class:`float`)
Georg Brandl116aa622007-08-15 14:28:22 +0000215 .. index::
216 object: floating point
217 pair: floating point; number
218 pair: C; language
219 pair: Java; language
220
221 These represent machine-level double precision floating point numbers. You are
222 at the mercy of the underlying machine architecture (and C or Java
223 implementation) for the accepted range and handling of overflow. Python does not
224 support single-precision floating point numbers; the savings in processor and
225 memory usage that are usually the reason for using these is dwarfed by the
226 overhead of using objects in Python, so there is no reason to complicate the
227 language with two kinds of floating point numbers.
228
Nick Coghlan3a5d7e32008-08-31 12:40:14 +0000229 :class:`numbers.Complex` (:class:`complex`)
Georg Brandl116aa622007-08-15 14:28:22 +0000230 .. index::
231 object: complex
232 pair: complex; number
233
234 These represent complex numbers as a pair of machine-level double precision
235 floating point numbers. The same caveats apply as for floating point numbers.
236 The real and imaginary parts of a complex number ``z`` can be retrieved through
237 the read-only attributes ``z.real`` and ``z.imag``.
238
Georg Brandl116aa622007-08-15 14:28:22 +0000239Sequences
240 .. index::
241 builtin: len
242 object: sequence
243 single: index operation
244 single: item selection
245 single: subscription
246
247 These represent finite ordered sets indexed by non-negative numbers. The
248 built-in function :func:`len` returns the number of items of a sequence. When
249 the length of a sequence is *n*, the index set contains the numbers 0, 1,
250 ..., *n*-1. Item *i* of sequence *a* is selected by ``a[i]``.
251
252 .. index:: single: slicing
253
254 Sequences also support slicing: ``a[i:j]`` selects all items with index *k* such
255 that *i* ``<=`` *k* ``<`` *j*. When used as an expression, a slice is a
256 sequence of the same type. This implies that the index set is renumbered so
257 that it starts at 0.
258
Georg Brandl116aa622007-08-15 14:28:22 +0000259 Some sequences also support "extended slicing" with a third "step" parameter:
260 ``a[i:j:k]`` selects all items of *a* with index *x* where ``x = i + n*k``, *n*
261 ``>=`` ``0`` and *i* ``<=`` *x* ``<`` *j*.
262
263 Sequences are distinguished according to their mutability:
264
265 Immutable sequences
266 .. index::
267 object: immutable sequence
268 object: immutable
269
270 An object of an immutable sequence type cannot change once it is created. (If
271 the object contains references to other objects, these other objects may be
272 mutable and may be changed; however, the collection of objects directly
273 referenced by an immutable object cannot change.)
274
275 The following types are immutable sequences:
276
Chris Jerdonekbb4e9412012-11-28 01:38:40 -0800277 .. index::
278 single: string; immutable sequences
279
Georg Brandl116aa622007-08-15 14:28:22 +0000280 Strings
281 .. index::
282 builtin: chr
283 builtin: ord
Georg Brandl116aa622007-08-15 14:28:22 +0000284 single: character
285 single: integer
286 single: Unicode
287
Ezio Melottif4d76e62011-10-25 09:23:42 +0300288 A string is a sequence of values that represent Unicode codepoints.
289 All the codepoints in range ``U+0000 - U+10FFFF`` can be represented
290 in a string. Python doesn't have a :c:type:`chr` type, and
Ezio Melottif7f0a662011-10-25 17:22:22 +0300291 every character in the string is represented as a string object
292 with length ``1``. The built-in function :func:`ord` converts a
293 character to its codepoint (as an integer); :func:`chr` converts
Ezio Melottif4d76e62011-10-25 09:23:42 +0300294 an integer in range ``0 - 10FFFF`` to the corresponding character.
295 :meth:`str.encode` can be used to convert a :class:`str` to
296 :class:`bytes` using the given encoding, and :meth:`bytes.decode` can
297 be used to achieve the opposite.
Georg Brandl116aa622007-08-15 14:28:22 +0000298
299 Tuples
300 .. index::
301 object: tuple
302 pair: singleton; tuple
303 pair: empty; tuple
304
Georg Brandldcc56f82007-08-31 16:41:12 +0000305 The items of a tuple are arbitrary Python objects. Tuples of two or
306 more items are formed by comma-separated lists of expressions. A tuple
307 of one item (a 'singleton') can be formed by affixing a comma to an
308 expression (an expression by itself does not create a tuple, since
309 parentheses must be usable for grouping of expressions). An empty
310 tuple can be formed by an empty pair of parentheses.
Georg Brandl116aa622007-08-15 14:28:22 +0000311
Nick Coghlan3a5d7e32008-08-31 12:40:14 +0000312 Bytes
313 .. index:: bytes, byte
314
315 A bytes object is an immutable array. The items are 8-bit bytes,
316 represented by integers in the range 0 <= x < 256. Bytes literals
Andrew Svetlovf5320352012-10-02 18:39:25 +0300317 (like ``b'abc'``) and the built-in function :func:`bytes` can be used to
Nick Coghlan3a5d7e32008-08-31 12:40:14 +0000318 construct bytes objects. Also, bytes objects can be decoded to strings
Serhiy Storchaka0d196ed2013-10-09 14:02:31 +0300319 via the :meth:`~bytes.decode` method.
Nick Coghlan3a5d7e32008-08-31 12:40:14 +0000320
Georg Brandl116aa622007-08-15 14:28:22 +0000321 Mutable sequences
322 .. index::
323 object: mutable sequence
324 object: mutable
325 pair: assignment; statement
326 single: delete
327 statement: del
328 single: subscription
329 single: slicing
330
331 Mutable sequences can be changed after they are created. The subscription and
332 slicing notations can be used as the target of assignment and :keyword:`del`
333 (delete) statements.
334
Benjamin Petersonb58dda72009-01-18 22:27:04 +0000335 There are currently two intrinsic mutable sequence types:
Georg Brandl116aa622007-08-15 14:28:22 +0000336
337 Lists
338 .. index:: object: list
339
Georg Brandldcc56f82007-08-31 16:41:12 +0000340 The items of a list are arbitrary Python objects. Lists are formed by
341 placing a comma-separated list of expressions in square brackets. (Note
342 that there are no special cases needed to form lists of length 0 or 1.)
343
Nick Coghlan3a5d7e32008-08-31 12:40:14 +0000344 Byte Arrays
345 .. index:: bytearray
Georg Brandldcc56f82007-08-31 16:41:12 +0000346
Nick Coghlan3a5d7e32008-08-31 12:40:14 +0000347 A bytearray object is a mutable array. They are created by the built-in
348 :func:`bytearray` constructor. Aside from being mutable (and hence
349 unhashable), byte arrays otherwise provide the same interface and
350 functionality as immutable bytes objects.
Georg Brandl116aa622007-08-15 14:28:22 +0000351
352 .. index:: module: array
353
Georg Brandldcc56f82007-08-31 16:41:12 +0000354 The extension module :mod:`array` provides an additional example of a
Nick Coghlan3a5d7e32008-08-31 12:40:14 +0000355 mutable sequence type, as does the :mod:`collections` module.
Georg Brandl116aa622007-08-15 14:28:22 +0000356
Georg Brandl116aa622007-08-15 14:28:22 +0000357Set types
358 .. index::
359 builtin: len
360 object: set type
361
362 These represent unordered, finite sets of unique, immutable objects. As such,
363 they cannot be indexed by any subscript. However, they can be iterated over, and
364 the built-in function :func:`len` returns the number of items in a set. Common
365 uses for sets are fast membership testing, removing duplicates from a sequence,
366 and computing mathematical operations such as intersection, union, difference,
367 and symmetric difference.
368
369 For set elements, the same immutability rules apply as for dictionary keys. Note
370 that numeric types obey the normal rules for numeric comparison: if two numbers
371 compare equal (e.g., ``1`` and ``1.0``), only one of them can be contained in a
372 set.
373
374 There are currently two intrinsic set types:
375
376 Sets
377 .. index:: object: set
378
379 These represent a mutable set. They are created by the built-in :func:`set`
380 constructor and can be modified afterwards by several methods, such as
Serhiy Storchaka0d196ed2013-10-09 14:02:31 +0300381 :meth:`~set.add`.
Georg Brandl116aa622007-08-15 14:28:22 +0000382
383 Frozen sets
384 .. index:: object: frozenset
385
Guido van Rossum2cc30da2007-11-02 23:46:40 +0000386 These represent an immutable set. They are created by the built-in
387 :func:`frozenset` constructor. As a frozenset is immutable and
388 :term:`hashable`, it can be used again as an element of another set, or as
389 a dictionary key.
Georg Brandl116aa622007-08-15 14:28:22 +0000390
Georg Brandl116aa622007-08-15 14:28:22 +0000391Mappings
392 .. index::
393 builtin: len
394 single: subscription
395 object: mapping
396
397 These represent finite sets of objects indexed by arbitrary index sets. The
398 subscript notation ``a[k]`` selects the item indexed by ``k`` from the mapping
399 ``a``; this can be used in expressions and as the target of assignments or
400 :keyword:`del` statements. The built-in function :func:`len` returns the number
401 of items in a mapping.
402
403 There is currently a single intrinsic mapping type:
404
405 Dictionaries
406 .. index:: object: dictionary
407
408 These represent finite sets of objects indexed by nearly arbitrary values. The
409 only types of values not acceptable as keys are values containing lists or
410 dictionaries or other mutable types that are compared by value rather than by
411 object identity, the reason being that the efficient implementation of
412 dictionaries requires a key's hash value to remain constant. Numeric types used
413 for keys obey the normal rules for numeric comparison: if two numbers compare
414 equal (e.g., ``1`` and ``1.0``) then they can be used interchangeably to index
415 the same dictionary entry.
416
417 Dictionaries are mutable; they can be created by the ``{...}`` notation (see
418 section :ref:`dict`).
419
420 .. index::
Georg Brandl0a7ac7d2008-05-26 10:29:35 +0000421 module: dbm.ndbm
422 module: dbm.gnu
Georg Brandl116aa622007-08-15 14:28:22 +0000423
Benjamin Peterson9a46cab2008-09-08 02:49:30 +0000424 The extension modules :mod:`dbm.ndbm` and :mod:`dbm.gnu` provide
425 additional examples of mapping types, as does the :mod:`collections`
Nick Coghlan3a5d7e32008-08-31 12:40:14 +0000426 module.
Georg Brandl116aa622007-08-15 14:28:22 +0000427
Georg Brandl116aa622007-08-15 14:28:22 +0000428Callable types
429 .. index::
430 object: callable
431 pair: function; call
432 single: invocation
433 pair: function; argument
434
435 These are the types to which the function call operation (see section
436 :ref:`calls`) can be applied:
437
438 User-defined functions
439 .. index::
440 pair: user-defined; function
441 object: function
442 object: user-defined function
443
444 A user-defined function object is created by a function definition (see
445 section :ref:`function`). It should be called with an argument list
446 containing the same number of items as the function's formal parameter
447 list.
448
449 Special attributes:
450
Georg Brandl44ea77b2013-03-28 13:28:44 +0100451 .. tabularcolumns:: |l|L|l|
452
Georg Brandl116aa622007-08-15 14:28:22 +0000453 +-------------------------+-------------------------------+-----------+
454 | Attribute | Meaning | |
455 +=========================+===============================+===========+
456 | :attr:`__doc__` | The function's documentation | Writable |
457 | | string, or ``None`` if | |
458 | | unavailable | |
459 +-------------------------+-------------------------------+-----------+
460 | :attr:`__name__` | The function's name | Writable |
461 +-------------------------+-------------------------------+-----------+
Antoine Pitrou86a36b52011-11-25 18:56:07 +0100462 | :attr:`__qualname__` | The function's | Writable |
463 | | :term:`qualified name` | |
464 | | | |
465 | | .. versionadded:: 3.3 | |
466 +-------------------------+-------------------------------+-----------+
Georg Brandl116aa622007-08-15 14:28:22 +0000467 | :attr:`__module__` | The name of the module the | Writable |
468 | | function was defined in, or | |
469 | | ``None`` if unavailable. | |
470 +-------------------------+-------------------------------+-----------+
471 | :attr:`__defaults__` | A tuple containing default | Writable |
472 | | argument values for those | |
473 | | arguments that have defaults, | |
474 | | or ``None`` if no arguments | |
475 | | have a default value | |
476 +-------------------------+-------------------------------+-----------+
477 | :attr:`__code__` | The code object representing | Writable |
478 | | the compiled function body. | |
479 +-------------------------+-------------------------------+-----------+
480 | :attr:`__globals__` | A reference to the dictionary | Read-only |
481 | | that holds the function's | |
482 | | global variables --- the | |
483 | | global namespace of the | |
484 | | module in which the function | |
485 | | was defined. | |
486 +-------------------------+-------------------------------+-----------+
487 | :attr:`__dict__` | The namespace supporting | Writable |
488 | | arbitrary function | |
489 | | attributes. | |
490 +-------------------------+-------------------------------+-----------+
491 | :attr:`__closure__` | ``None`` or a tuple of cells | Read-only |
492 | | that contain bindings for the | |
493 | | function's free variables. | |
494 +-------------------------+-------------------------------+-----------+
495 | :attr:`__annotations__` | A dict containing annotations | Writable |
496 | | of parameters. The keys of | |
497 | | the dict are the parameter | |
498 | | names, or ``'return'`` for | |
499 | | the return annotation, if | |
500 | | provided. | |
501 +-------------------------+-------------------------------+-----------+
502 | :attr:`__kwdefaults__` | A dict containing defaults | Writable |
503 | | for keyword-only parameters. | |
504 +-------------------------+-------------------------------+-----------+
505
506 Most of the attributes labelled "Writable" check the type of the assigned value.
507
Georg Brandl116aa622007-08-15 14:28:22 +0000508 Function objects also support getting and setting arbitrary attributes, which
509 can be used, for example, to attach metadata to functions. Regular attribute
510 dot-notation is used to get and set such attributes. *Note that the current
511 implementation only supports function attributes on user-defined functions.
512 Function attributes on built-in functions may be supported in the future.*
513
514 Additional information about a function's definition can be retrieved from its
515 code object; see the description of internal types below.
516
517 .. index::
518 single: __doc__ (function attribute)
519 single: __name__ (function attribute)
520 single: __module__ (function attribute)
521 single: __dict__ (function attribute)
522 single: __defaults__ (function attribute)
523 single: __closure__ (function attribute)
524 single: __code__ (function attribute)
525 single: __globals__ (function attribute)
526 single: __annotations__ (function attribute)
527 single: __kwdefaults__ (function attribute)
528 pair: global; namespace
529
Georg Brandl2e0b7552007-11-27 12:43:08 +0000530 Instance methods
Georg Brandl116aa622007-08-15 14:28:22 +0000531 .. index::
532 object: method
533 object: user-defined method
534 pair: user-defined; method
535
Georg Brandl2e0b7552007-11-27 12:43:08 +0000536 An instance method object combines a class, a class instance and any
537 callable object (normally a user-defined function).
538
539 .. index::
540 single: __func__ (method attribute)
541 single: __self__ (method attribute)
542 single: __doc__ (method attribute)
543 single: __name__ (method attribute)
544 single: __module__ (method attribute)
Georg Brandl116aa622007-08-15 14:28:22 +0000545
Christian Heimesff737952007-11-27 10:40:20 +0000546 Special read-only attributes: :attr:`__self__` is the class instance object,
547 :attr:`__func__` is the function object; :attr:`__doc__` is the method's
548 documentation (same as ``__func__.__doc__``); :attr:`__name__` is the
549 method name (same as ``__func__.__name__``); :attr:`__module__` is the
550 name of the module the method was defined in, or ``None`` if unavailable.
Georg Brandl116aa622007-08-15 14:28:22 +0000551
Georg Brandl116aa622007-08-15 14:28:22 +0000552 Methods also support accessing (but not setting) the arbitrary function
553 attributes on the underlying function object.
554
Georg Brandl2e0b7552007-11-27 12:43:08 +0000555 User-defined method objects may be created when getting an attribute of a
556 class (perhaps via an instance of that class), if that attribute is a
557 user-defined function object or a class method object.
Nick Coghlan3a5d7e32008-08-31 12:40:14 +0000558
Georg Brandl2e0b7552007-11-27 12:43:08 +0000559 When an instance method object is created by retrieving a user-defined
560 function object from a class via one of its instances, its
561 :attr:`__self__` attribute is the instance, and the method object is said
562 to be bound. The new method's :attr:`__func__` attribute is the original
563 function object.
Georg Brandl116aa622007-08-15 14:28:22 +0000564
Georg Brandl2e0b7552007-11-27 12:43:08 +0000565 When a user-defined method object is created by retrieving another method
566 object from a class or instance, the behaviour is the same as for a
567 function object, except that the :attr:`__func__` attribute of the new
568 instance is not the original method object but its :attr:`__func__`
569 attribute.
Georg Brandl116aa622007-08-15 14:28:22 +0000570
Georg Brandl2e0b7552007-11-27 12:43:08 +0000571 When an instance method object is created by retrieving a class method
572 object from a class or instance, its :attr:`__self__` attribute is the
573 class itself, and its :attr:`__func__` attribute is the function object
574 underlying the class method.
Georg Brandl116aa622007-08-15 14:28:22 +0000575
Georg Brandl2e0b7552007-11-27 12:43:08 +0000576 When an instance method object is called, the underlying function
577 (:attr:`__func__`) is called, inserting the class instance
578 (:attr:`__self__`) in front of the argument list. For instance, when
579 :class:`C` is a class which contains a definition for a function
580 :meth:`f`, and ``x`` is an instance of :class:`C`, calling ``x.f(1)`` is
581 equivalent to calling ``C.f(x, 1)``.
Georg Brandl116aa622007-08-15 14:28:22 +0000582
Georg Brandl2e0b7552007-11-27 12:43:08 +0000583 When an instance method object is derived from a class method object, the
584 "class instance" stored in :attr:`__self__` will actually be the class
585 itself, so that calling either ``x.f(1)`` or ``C.f(1)`` is equivalent to
586 calling ``f(C,1)`` where ``f`` is the underlying function.
Georg Brandl116aa622007-08-15 14:28:22 +0000587
Georg Brandl2e0b7552007-11-27 12:43:08 +0000588 Note that the transformation from function object to instance method
589 object happens each time the attribute is retrieved from the instance. In
590 some cases, a fruitful optimization is to assign the attribute to a local
591 variable and call that local variable. Also notice that this
592 transformation only happens for user-defined functions; other callable
593 objects (and all non-callable objects) are retrieved without
594 transformation. It is also important to note that user-defined functions
595 which are attributes of a class instance are not converted to bound
596 methods; this *only* happens when the function is an attribute of the
597 class.
Georg Brandl116aa622007-08-15 14:28:22 +0000598
599 Generator functions
600 .. index::
601 single: generator; function
602 single: generator; iterator
603
604 A function or method which uses the :keyword:`yield` statement (see section
Nick Coghlan3a5d7e32008-08-31 12:40:14 +0000605 :ref:`yield`) is called a :dfn:`generator function`. Such a function, when
606 called, always returns an iterator object which can be used to execute the
Serhiy Storchaka0d196ed2013-10-09 14:02:31 +0300607 body of the function: calling the iterator's :meth:`iterator.__next__`
Ezio Melotti7fa82222012-10-12 13:42:08 +0300608 method will cause the function to execute until it provides a value
609 using the :keyword:`yield` statement. When the function executes a
Georg Brandl116aa622007-08-15 14:28:22 +0000610 :keyword:`return` statement or falls off the end, a :exc:`StopIteration`
611 exception is raised and the iterator will have reached the end of the set of
612 values to be returned.
613
614 Built-in functions
615 .. index::
616 object: built-in function
617 object: function
618 pair: C; language
619
620 A built-in function object is a wrapper around a C function. Examples of
621 built-in functions are :func:`len` and :func:`math.sin` (:mod:`math` is a
622 standard built-in module). The number and type of the arguments are
623 determined by the C function. Special read-only attributes:
624 :attr:`__doc__` is the function's documentation string, or ``None`` if
625 unavailable; :attr:`__name__` is the function's name; :attr:`__self__` is
626 set to ``None`` (but see the next item); :attr:`__module__` is the name of
627 the module the function was defined in or ``None`` if unavailable.
628
629 Built-in methods
630 .. index::
631 object: built-in method
632 object: method
633 pair: built-in; method
634
635 This is really a different disguise of a built-in function, this time containing
636 an object passed to the C function as an implicit extra argument. An example of
637 a built-in method is ``alist.append()``, assuming *alist* is a list object. In
638 this case, the special read-only attribute :attr:`__self__` is set to the object
Éric Araujoc9562f32010-12-26 02:18:49 +0000639 denoted by *alist*.
Georg Brandl116aa622007-08-15 14:28:22 +0000640
Georg Brandl85eb8c12007-08-31 16:33:38 +0000641 Classes
642 Classes are callable. These objects normally act as factories for new
643 instances of themselves, but variations are possible for class types that
644 override :meth:`__new__`. The arguments of the call are passed to
645 :meth:`__new__` and, in the typical case, to :meth:`__init__` to
646 initialize the new instance.
Georg Brandl116aa622007-08-15 14:28:22 +0000647
Georg Brandl85eb8c12007-08-31 16:33:38 +0000648 Class Instances
649 Instances of arbitrary classes can be made callable by defining a
650 :meth:`__call__` method in their class.
Georg Brandl116aa622007-08-15 14:28:22 +0000651
Georg Brandl116aa622007-08-15 14:28:22 +0000652
653Modules
654 .. index::
655 statement: import
656 object: module
657
Barry Warsawd7d21942012-07-29 16:36:17 -0400658 Modules are a basic organizational unit of Python code, and are created by
Barry Warsawdadebab2012-07-31 16:03:09 -0400659 the :ref:`import system <importsystem>` as invoked either by the
660 :keyword:`import` statement (see :keyword:`import`), or by calling
661 functions such as :func:`importlib.import_module` and built-in
662 :func:`__import__`. A module object has a namespace implemented by a
663 dictionary object (this is the dictionary referenced by the ``__globals__``
664 attribute of functions defined in the module). Attribute references are
665 translated to lookups in this dictionary, e.g., ``m.x`` is equivalent to
666 ``m.__dict__["x"]``. A module object does not contain the code object used
667 to initialize the module (since it isn't needed once the initialization is
668 done).
Georg Brandl116aa622007-08-15 14:28:22 +0000669
Barry Warsawd7d21942012-07-29 16:36:17 -0400670 Attribute assignment updates the module's namespace dictionary, e.g.,
671 ``m.x = 1`` is equivalent to ``m.__dict__["x"] = 1``.
Georg Brandl116aa622007-08-15 14:28:22 +0000672
673 .. index:: single: __dict__ (module attribute)
674
675 Special read-only attribute: :attr:`__dict__` is the module's namespace as a
676 dictionary object.
677
Benjamin Peterson5c4bfc42010-10-12 22:57:59 +0000678 .. impl-detail::
679
680 Because of the way CPython clears module dictionaries, the module
681 dictionary will be cleared when the module falls out of scope even if the
682 dictionary still has live references. To avoid this, copy the dictionary
683 or keep the module around while using its dictionary directly.
684
Georg Brandl116aa622007-08-15 14:28:22 +0000685 .. index::
686 single: __name__ (module attribute)
687 single: __doc__ (module attribute)
688 single: __file__ (module attribute)
689 pair: module; namespace
690
691 Predefined (writable) attributes: :attr:`__name__` is the module's name;
692 :attr:`__doc__` is the module's documentation string, or ``None`` if
Barry Warsawd7d21942012-07-29 16:36:17 -0400693 unavailable; :attr:`__file__` is the pathname of the file from which the
694 module was loaded, if it was loaded from a file. The :attr:`__file__`
695 attribute may be missing for certain types of modules, such as C modules
696 that are statically linked into the interpreter; for extension modules
697 loaded dynamically from a shared library, it is the pathname of the shared
698 library file.
Georg Brandl116aa622007-08-15 14:28:22 +0000699
Georg Brandl85eb8c12007-08-31 16:33:38 +0000700Custom classes
Georg Brandl5dbb84a2009-09-02 20:31:26 +0000701 Custom class types are typically created by class definitions (see section
Nick Coghlan3a5d7e32008-08-31 12:40:14 +0000702 :ref:`class`). A class has a namespace implemented by a dictionary object.
703 Class attribute references are translated to lookups in this dictionary, e.g.,
704 ``C.x`` is translated to ``C.__dict__["x"]`` (although there are a number of
705 hooks which allow for other means of locating attributes). When the attribute
706 name is not found there, the attribute search continues in the base classes.
707 This search of the base classes uses the C3 method resolution order which
708 behaves correctly even in the presence of 'diamond' inheritance structures
709 where there are multiple inheritance paths leading back to a common ancestor.
710 Additional details on the C3 MRO used by Python can be found in the
711 documentation accompanying the 2.3 release at
712 http://www.python.org/download/releases/2.3/mro/.
Georg Brandl116aa622007-08-15 14:28:22 +0000713
Nick Coghlan3a5d7e32008-08-31 12:40:14 +0000714 .. XXX: Could we add that MRO doc as an appendix to the language ref?
Georg Brandl85eb8c12007-08-31 16:33:38 +0000715
Georg Brandl116aa622007-08-15 14:28:22 +0000716 .. index::
717 object: class
718 object: class instance
719 object: instance
720 pair: class object; call
721 single: container
722 object: dictionary
723 pair: class; attribute
724
725 When a class attribute reference (for class :class:`C`, say) would yield a
Georg Brandl2e0b7552007-11-27 12:43:08 +0000726 class method object, it is transformed into an instance method object whose
727 :attr:`__self__` attributes is :class:`C`. When it would yield a static
728 method object, it is transformed into the object wrapped by the static method
729 object. See section :ref:`descriptors` for another way in which attributes
730 retrieved from a class may differ from those actually contained in its
731 :attr:`__dict__`.
Georg Brandl116aa622007-08-15 14:28:22 +0000732
733 .. index:: triple: class; attribute; assignment
734
735 Class attribute assignments update the class's dictionary, never the dictionary
736 of a base class.
737
738 .. index:: pair: class object; call
739
740 A class object can be called (see above) to yield a class instance (see below).
741
742 .. index::
743 single: __name__ (class attribute)
744 single: __module__ (class attribute)
745 single: __dict__ (class attribute)
746 single: __bases__ (class attribute)
747 single: __doc__ (class attribute)
748
749 Special attributes: :attr:`__name__` is the class name; :attr:`__module__` is
750 the module name in which the class was defined; :attr:`__dict__` is the
Serhiy Storchaka0d196ed2013-10-09 14:02:31 +0300751 dictionary containing the class's namespace; :attr:`~class.__bases__` is a
752 tuple (possibly empty or a singleton) containing the base classes, in the
753 order of their occurrence in the base class list; :attr:`__doc__` is the
754 class's documentation string, or None if undefined.
Georg Brandl116aa622007-08-15 14:28:22 +0000755
756Class instances
757 .. index::
758 object: class instance
759 object: instance
760 pair: class; instance
761 pair: class instance; attribute
762
Georg Brandl2e0b7552007-11-27 12:43:08 +0000763 A class instance is created by calling a class object (see above). A class
764 instance has a namespace implemented as a dictionary which is the first place
765 in which attribute references are searched. When an attribute is not found
766 there, and the instance's class has an attribute by that name, the search
767 continues with the class attributes. If a class attribute is found that is a
768 user-defined function object, it is transformed into an instance method
769 object whose :attr:`__self__` attribute is the instance. Static method and
770 class method objects are also transformed; see above under "Classes". See
771 section :ref:`descriptors` for another way in which attributes of a class
772 retrieved via its instances may differ from the objects actually stored in
773 the class's :attr:`__dict__`. If no class attribute is found, and the
774 object's class has a :meth:`__getattr__` method, that is called to satisfy
775 the lookup.
Georg Brandl116aa622007-08-15 14:28:22 +0000776
777 .. index:: triple: class instance; attribute; assignment
778
779 Attribute assignments and deletions update the instance's dictionary, never a
780 class's dictionary. If the class has a :meth:`__setattr__` or
781 :meth:`__delattr__` method, this is called instead of updating the instance
782 dictionary directly.
783
784 .. index::
785 object: numeric
786 object: sequence
787 object: mapping
788
789 Class instances can pretend to be numbers, sequences, or mappings if they have
790 methods with certain special names. See section :ref:`specialnames`.
791
792 .. index::
793 single: __dict__ (instance attribute)
794 single: __class__ (instance attribute)
795
Serhiy Storchaka0d196ed2013-10-09 14:02:31 +0300796 Special attributes: :attr:`~object.__dict__` is the attribute dictionary;
797 :attr:`~instance.__class__` is the instance's class.
Georg Brandl116aa622007-08-15 14:28:22 +0000798
Antoine Pitrou4adb2882010-01-04 18:50:53 +0000799I/O objects (also known as file objects)
Georg Brandl116aa622007-08-15 14:28:22 +0000800 .. index::
Georg Brandl116aa622007-08-15 14:28:22 +0000801 builtin: open
Antoine Pitrou4adb2882010-01-04 18:50:53 +0000802 module: io
Georg Brandl116aa622007-08-15 14:28:22 +0000803 single: popen() (in module os)
804 single: makefile() (socket method)
805 single: sys.stdin
806 single: sys.stdout
807 single: sys.stderr
808 single: stdio
809 single: stdin (in module sys)
810 single: stdout (in module sys)
811 single: stderr (in module sys)
812
Antoine Pitrou0b65b0f2010-09-15 09:58:26 +0000813 A :term:`file object` represents an open file. Various shortcuts are
814 available to create file objects: the :func:`open` built-in function, and
Serhiy Storchaka0d196ed2013-10-09 14:02:31 +0300815 also :func:`os.popen`, :func:`os.fdopen`, and the
816 :meth:`~socket.socket.makefile` method of socket objects (and perhaps by
817 other functions or methods provided by extension modules).
Antoine Pitrou4adb2882010-01-04 18:50:53 +0000818
819 The objects ``sys.stdin``, ``sys.stdout`` and ``sys.stderr`` are
820 initialized to file objects corresponding to the interpreter's standard
821 input, output and error streams; they are all open in text mode and
822 therefore follow the interface defined by the :class:`io.TextIOBase`
823 abstract class.
Georg Brandl116aa622007-08-15 14:28:22 +0000824
825Internal types
826 .. index::
827 single: internal type
828 single: types, internal
829
830 A few types used internally by the interpreter are exposed to the user. Their
831 definitions may change with future versions of the interpreter, but they are
832 mentioned here for completeness.
833
834 Code objects
835 .. index::
836 single: bytecode
837 object: code
838
Georg Brandl9afde1c2007-11-01 20:32:30 +0000839 Code objects represent *byte-compiled* executable Python code, or :term:`bytecode`.
Georg Brandl116aa622007-08-15 14:28:22 +0000840 The difference between a code object and a function object is that the function
841 object contains an explicit reference to the function's globals (the module in
842 which it was defined), while a code object contains no context; also the default
843 argument values are stored in the function object, not in the code object
844 (because they represent values calculated at run-time). Unlike function
845 objects, code objects are immutable and contain no references (directly or
846 indirectly) to mutable objects.
847
Senthil Kumaran7cafd262010-10-02 03:16:04 +0000848 .. index::
849 single: co_argcount (code object attribute)
850 single: co_code (code object attribute)
851 single: co_consts (code object attribute)
852 single: co_filename (code object attribute)
853 single: co_firstlineno (code object attribute)
854 single: co_flags (code object attribute)
855 single: co_lnotab (code object attribute)
856 single: co_name (code object attribute)
857 single: co_names (code object attribute)
858 single: co_nlocals (code object attribute)
859 single: co_stacksize (code object attribute)
860 single: co_varnames (code object attribute)
861 single: co_cellvars (code object attribute)
862 single: co_freevars (code object attribute)
863
Georg Brandl116aa622007-08-15 14:28:22 +0000864 Special read-only attributes: :attr:`co_name` gives the function name;
865 :attr:`co_argcount` is the number of positional arguments (including arguments
866 with default values); :attr:`co_nlocals` is the number of local variables used
867 by the function (including arguments); :attr:`co_varnames` is a tuple containing
868 the names of the local variables (starting with the argument names);
869 :attr:`co_cellvars` is a tuple containing the names of local variables that are
870 referenced by nested functions; :attr:`co_freevars` is a tuple containing the
871 names of free variables; :attr:`co_code` is a string representing the sequence
872 of bytecode instructions; :attr:`co_consts` is a tuple containing the literals
873 used by the bytecode; :attr:`co_names` is a tuple containing the names used by
874 the bytecode; :attr:`co_filename` is the filename from which the code was
875 compiled; :attr:`co_firstlineno` is the first line number of the function;
Georg Brandl9afde1c2007-11-01 20:32:30 +0000876 :attr:`co_lnotab` is a string encoding the mapping from bytecode offsets to
Georg Brandl116aa622007-08-15 14:28:22 +0000877 line numbers (for details see the source code of the interpreter);
878 :attr:`co_stacksize` is the required stack size (including local variables);
879 :attr:`co_flags` is an integer encoding a number of flags for the interpreter.
880
Georg Brandl116aa622007-08-15 14:28:22 +0000881 .. index:: object: generator
882
883 The following flag bits are defined for :attr:`co_flags`: bit ``0x04`` is set if
884 the function uses the ``*arguments`` syntax to accept an arbitrary number of
885 positional arguments; bit ``0x08`` is set if the function uses the
886 ``**keywords`` syntax to accept arbitrary keyword arguments; bit ``0x20`` is set
887 if the function is a generator.
888
889 Future feature declarations (``from __future__ import division``) also use bits
890 in :attr:`co_flags` to indicate whether a code object was compiled with a
891 particular feature enabled: bit ``0x2000`` is set if the function was compiled
892 with future division enabled; bits ``0x10`` and ``0x1000`` were used in earlier
893 versions of Python.
894
895 Other bits in :attr:`co_flags` are reserved for internal use.
896
897 .. index:: single: documentation string
898
899 If a code object represents a function, the first item in :attr:`co_consts` is
900 the documentation string of the function, or ``None`` if undefined.
901
Georg Brandla6053b42009-09-01 08:11:14 +0000902 .. _frame-objects:
903
Georg Brandl116aa622007-08-15 14:28:22 +0000904 Frame objects
905 .. index:: object: frame
906
907 Frame objects represent execution frames. They may occur in traceback objects
908 (see below).
909
910 .. index::
911 single: f_back (frame attribute)
912 single: f_code (frame attribute)
913 single: f_globals (frame attribute)
914 single: f_locals (frame attribute)
915 single: f_lasti (frame attribute)
916 single: f_builtins (frame attribute)
917
918 Special read-only attributes: :attr:`f_back` is to the previous stack frame
919 (towards the caller), or ``None`` if this is the bottom stack frame;
920 :attr:`f_code` is the code object being executed in this frame; :attr:`f_locals`
921 is the dictionary used to look up local variables; :attr:`f_globals` is used for
922 global variables; :attr:`f_builtins` is used for built-in (intrinsic) names;
923 :attr:`f_lasti` gives the precise instruction (this is an index into the
924 bytecode string of the code object).
925
926 .. index::
927 single: f_trace (frame attribute)
Georg Brandl116aa622007-08-15 14:28:22 +0000928 single: f_lineno (frame attribute)
929
930 Special writable attributes: :attr:`f_trace`, if not ``None``, is a function
931 called at the start of each source code line (this is used by the debugger);
Benjamin Petersoneec3d712008-06-11 15:59:43 +0000932 :attr:`f_lineno` is the current line number of the frame --- writing to this
933 from within a trace function jumps to the given line (only for the bottom-most
934 frame). A debugger can implement a Jump command (aka Set Next Statement)
935 by writing to f_lineno.
Georg Brandl116aa622007-08-15 14:28:22 +0000936
Antoine Pitrou58720d62013-08-05 23:26:40 +0200937 Frame objects support one method:
938
939 .. method:: frame.clear()
940
941 This method clears all references to local variables held by the
942 frame. Also, if the frame belonged to a generator, the generator
943 is finalized. This helps break reference cycles involving frame
944 objects (for example when catching an exception and storing its
945 traceback for later use).
946
947 :exc:`RuntimeError` is raised if the frame is currently executing.
948
949 .. versionadded:: 3.4
950
Georg Brandl116aa622007-08-15 14:28:22 +0000951 Traceback objects
952 .. index::
953 object: traceback
954 pair: stack; trace
955 pair: exception; handler
956 pair: execution; stack
957 single: exc_info (in module sys)
Georg Brandl116aa622007-08-15 14:28:22 +0000958 single: last_traceback (in module sys)
959 single: sys.exc_info
960 single: sys.last_traceback
961
962 Traceback objects represent a stack trace of an exception. A traceback object
963 is created when an exception occurs. When the search for an exception handler
964 unwinds the execution stack, at each unwound level a traceback object is
965 inserted in front of the current traceback. When an exception handler is
966 entered, the stack trace is made available to the program. (See section
967 :ref:`try`.) It is accessible as the third item of the
968 tuple returned by ``sys.exc_info()``. When the program contains no suitable
969 handler, the stack trace is written (nicely formatted) to the standard error
970 stream; if the interpreter is interactive, it is also made available to the user
971 as ``sys.last_traceback``.
972
973 .. index::
974 single: tb_next (traceback attribute)
975 single: tb_frame (traceback attribute)
976 single: tb_lineno (traceback attribute)
977 single: tb_lasti (traceback attribute)
978 statement: try
979
980 Special read-only attributes: :attr:`tb_next` is the next level in the stack
981 trace (towards the frame where the exception occurred), or ``None`` if there is
982 no next level; :attr:`tb_frame` points to the execution frame of the current
983 level; :attr:`tb_lineno` gives the line number where the exception occurred;
984 :attr:`tb_lasti` indicates the precise instruction. The line number and last
985 instruction in the traceback may differ from the line number of its frame object
986 if the exception occurred in a :keyword:`try` statement with no matching except
987 clause or with a finally clause.
988
989 Slice objects
990 .. index:: builtin: slice
991
Georg Brandlcb8ecb12007-09-04 06:35:14 +0000992 Slice objects are used to represent slices for :meth:`__getitem__`
993 methods. They are also created by the built-in :func:`slice` function.
Georg Brandl116aa622007-08-15 14:28:22 +0000994
995 .. index::
996 single: start (slice object attribute)
997 single: stop (slice object attribute)
998 single: step (slice object attribute)
999
Serhiy Storchaka0d196ed2013-10-09 14:02:31 +03001000 Special read-only attributes: :attr:`~slice.start` is the lower bound;
1001 :attr:`~slice.stop` is the upper bound; :attr:`~slice.step` is the step
1002 value; each is ``None`` if omitted. These attributes can have any type.
Georg Brandl116aa622007-08-15 14:28:22 +00001003
1004 Slice objects support one method:
1005
Georg Brandl116aa622007-08-15 14:28:22 +00001006 .. method:: slice.indices(self, length)
1007
Georg Brandlcb8ecb12007-09-04 06:35:14 +00001008 This method takes a single integer argument *length* and computes
1009 information about the slice that the slice object would describe if
1010 applied to a sequence of *length* items. It returns a tuple of three
1011 integers; respectively these are the *start* and *stop* indices and the
1012 *step* or stride length of the slice. Missing or out-of-bounds indices
1013 are handled in a manner consistent with regular slices.
Georg Brandl116aa622007-08-15 14:28:22 +00001014
Georg Brandl116aa622007-08-15 14:28:22 +00001015 Static method objects
1016 Static method objects provide a way of defeating the transformation of function
1017 objects to method objects described above. A static method object is a wrapper
1018 around any other object, usually a user-defined method object. When a static
1019 method object is retrieved from a class or a class instance, the object actually
1020 returned is the wrapped object, which is not subject to any further
1021 transformation. Static method objects are not themselves callable, although the
1022 objects they wrap usually are. Static method objects are created by the built-in
1023 :func:`staticmethod` constructor.
1024
1025 Class method objects
1026 A class method object, like a static method object, is a wrapper around another
1027 object that alters the way in which that object is retrieved from classes and
1028 class instances. The behaviour of class method objects upon such retrieval is
1029 described above, under "User-defined methods". Class method objects are created
1030 by the built-in :func:`classmethod` constructor.
1031
Georg Brandl116aa622007-08-15 14:28:22 +00001032
Georg Brandl116aa622007-08-15 14:28:22 +00001033.. _specialnames:
1034
1035Special method names
1036====================
1037
1038.. index::
1039 pair: operator; overloading
1040 single: __getitem__() (mapping object method)
1041
1042A class can implement certain operations that are invoked by special syntax
1043(such as arithmetic operations or subscripting and slicing) by defining methods
1044with special names. This is Python's approach to :dfn:`operator overloading`,
1045allowing classes to define their own behavior with respect to language
1046operators. For instance, if a class defines a method named :meth:`__getitem__`,
Nick Coghlan3a5d7e32008-08-31 12:40:14 +00001047and ``x`` is an instance of this class, then ``x[i]`` is roughly equivalent
1048to ``type(x).__getitem__(x, i)``. Except where mentioned, attempts to execute an
1049operation raise an exception when no appropriate method is defined (typically
1050:exc:`AttributeError` or :exc:`TypeError`).
Georg Brandl65ea9bd2007-09-05 13:36:27 +00001051
Georg Brandl116aa622007-08-15 14:28:22 +00001052When implementing a class that emulates any built-in type, it is important that
1053the emulation only be implemented to the degree that it makes sense for the
1054object being modelled. For example, some sequences may work well with retrieval
1055of individual elements, but extracting a slice may not make sense. (One example
Serhiy Storchaka0d196ed2013-10-09 14:02:31 +03001056of this is the :class:`~xml.dom.NodeList` interface in the W3C's Document
1057Object Model.)
Georg Brandl116aa622007-08-15 14:28:22 +00001058
1059
1060.. _customization:
1061
1062Basic customization
1063-------------------
1064
Georg Brandl116aa622007-08-15 14:28:22 +00001065.. method:: object.__new__(cls[, ...])
1066
Georg Brandlaf265f42008-12-07 15:06:20 +00001067 .. index:: pair: subclassing; immutable types
1068
Georg Brandl116aa622007-08-15 14:28:22 +00001069 Called to create a new instance of class *cls*. :meth:`__new__` is a static
1070 method (special-cased so you need not declare it as such) that takes the class
1071 of which an instance was requested as its first argument. The remaining
1072 arguments are those passed to the object constructor expression (the call to the
1073 class). The return value of :meth:`__new__` should be the new object instance
1074 (usually an instance of *cls*).
1075
1076 Typical implementations create a new instance of the class by invoking the
1077 superclass's :meth:`__new__` method using ``super(currentclass,
1078 cls).__new__(cls[, ...])`` with appropriate arguments and then modifying the
1079 newly-created instance as necessary before returning it.
1080
1081 If :meth:`__new__` returns an instance of *cls*, then the new instance's
1082 :meth:`__init__` method will be invoked like ``__init__(self[, ...])``, where
1083 *self* is the new instance and the remaining arguments are the same as were
1084 passed to :meth:`__new__`.
1085
1086 If :meth:`__new__` does not return an instance of *cls*, then the new instance's
1087 :meth:`__init__` method will not be invoked.
1088
1089 :meth:`__new__` is intended mainly to allow subclasses of immutable types (like
Christian Heimes790c8232008-01-07 21:14:23 +00001090 int, str, or tuple) to customize instance creation. It is also commonly
1091 overridden in custom metaclasses in order to customize class creation.
Georg Brandl116aa622007-08-15 14:28:22 +00001092
1093
1094.. method:: object.__init__(self[, ...])
1095
1096 .. index:: pair: class; constructor
1097
1098 Called when the instance is created. The arguments are those passed to the
1099 class constructor expression. If a base class has an :meth:`__init__` method,
1100 the derived class's :meth:`__init__` method, if any, must explicitly call it to
1101 ensure proper initialization of the base class part of the instance; for
1102 example: ``BaseClass.__init__(self, [args...])``. As a special constraint on
1103 constructors, no value may be returned; doing so will cause a :exc:`TypeError`
1104 to be raised at runtime.
1105
1106
1107.. method:: object.__del__(self)
1108
1109 .. index::
1110 single: destructor
1111 statement: del
1112
1113 Called when the instance is about to be destroyed. This is also called a
1114 destructor. If a base class has a :meth:`__del__` method, the derived class's
1115 :meth:`__del__` method, if any, must explicitly call it to ensure proper
1116 deletion of the base class part of the instance. Note that it is possible
1117 (though not recommended!) for the :meth:`__del__` method to postpone destruction
1118 of the instance by creating a new reference to it. It may then be called at a
1119 later time when this new reference is deleted. It is not guaranteed that
1120 :meth:`__del__` methods are called for objects that still exist when the
1121 interpreter exits.
1122
1123 .. note::
1124
1125 ``del x`` doesn't directly call ``x.__del__()`` --- the former decrements
1126 the reference count for ``x`` by one, and the latter is only called when
1127 ``x``'s reference count reaches zero. Some common situations that may
1128 prevent the reference count of an object from going to zero include:
1129 circular references between objects (e.g., a doubly-linked list or a tree
1130 data structure with parent and child pointers); a reference to the object
1131 on the stack frame of a function that caught an exception (the traceback
1132 stored in ``sys.exc_info()[2]`` keeps the stack frame alive); or a
1133 reference to the object on the stack frame that raised an unhandled
1134 exception in interactive mode (the traceback stored in
1135 ``sys.last_traceback`` keeps the stack frame alive). The first situation
1136 can only be remedied by explicitly breaking the cycles; the latter two
1137 situations can be resolved by storing ``None`` in ``sys.last_traceback``.
Antoine Pitrou796564c2013-07-30 19:59:21 +02001138 Circular references which are garbage are detected and cleaned up when
1139 the cyclic garbage collector is enabled (it's on by default). Refer to the
1140 documentation for the :mod:`gc` module for more information about this
1141 topic.
Georg Brandl116aa622007-08-15 14:28:22 +00001142
1143 .. warning::
1144
1145 Due to the precarious circumstances under which :meth:`__del__` methods are
1146 invoked, exceptions that occur during their execution are ignored, and a warning
1147 is printed to ``sys.stderr`` instead. Also, when :meth:`__del__` is invoked in
1148 response to a module being deleted (e.g., when execution of the program is
1149 done), other globals referenced by the :meth:`__del__` method may already have
Brett Cannone1327f72009-01-29 04:10:21 +00001150 been deleted or in the process of being torn down (e.g. the import
1151 machinery shutting down). For this reason, :meth:`__del__` methods
1152 should do the absolute
Georg Brandl116aa622007-08-15 14:28:22 +00001153 minimum needed to maintain external invariants. Starting with version 1.5,
1154 Python guarantees that globals whose name begins with a single underscore are
1155 deleted from their module before other globals are deleted; if no other
1156 references to such globals exist, this may help in assuring that imported
1157 modules are still available at the time when the :meth:`__del__` method is
1158 called.
1159
Chris Jerdonek5fae0e52012-11-20 17:45:51 -08001160 .. index::
1161 single: repr() (built-in function); __repr__() (object method)
1162
Georg Brandl116aa622007-08-15 14:28:22 +00001163
1164.. method:: object.__repr__(self)
1165
Benjamin Peterson1c9313f2008-10-12 12:51:12 +00001166 Called by the :func:`repr` built-in function to compute the "official" string
1167 representation of an object. If at all possible, this should look like a
1168 valid Python expression that could be used to recreate an object with the
1169 same value (given an appropriate environment). If this is not possible, a
1170 string of the form ``<...some useful description...>`` should be returned.
1171 The return value must be a string object. If a class defines :meth:`__repr__`
1172 but not :meth:`__str__`, then :meth:`__repr__` is also used when an
1173 "informal" string representation of instances of that class is required.
Georg Brandl116aa622007-08-15 14:28:22 +00001174
Georg Brandl116aa622007-08-15 14:28:22 +00001175 This is typically used for debugging, so it is important that the representation
1176 is information-rich and unambiguous.
1177
Chris Jerdonek5fae0e52012-11-20 17:45:51 -08001178 .. index::
1179 single: string; __str__() (object method)
1180 single: format() (built-in function); __str__() (object method)
1181 single: print() (built-in function); __str__() (object method)
1182
Georg Brandl116aa622007-08-15 14:28:22 +00001183
1184.. method:: object.__str__(self)
1185
Chris Jerdonek5fae0e52012-11-20 17:45:51 -08001186 Called by :func:`str(object) <str>` and the built-in functions
1187 :func:`format` and :func:`print` to compute the "informal" or nicely
1188 printable string representation of an object. The return value must be a
1189 :ref:`string <textseq>` object.
Georg Brandl116aa622007-08-15 14:28:22 +00001190
Chris Jerdonek5fae0e52012-11-20 17:45:51 -08001191 This method differs from :meth:`object.__repr__` in that there is no
1192 expectation that :meth:`__str__` return a valid Python expression: a more
1193 convenient or concise representation can be used.
1194
1195 The default implementation defined by the built-in type :class:`object`
1196 calls :meth:`object.__repr__`.
Georg Brandl116aa622007-08-15 14:28:22 +00001197
Georg Brandldcc56f82007-08-31 16:41:12 +00001198 .. XXX what about subclasses of string?
1199
Georg Brandl116aa622007-08-15 14:28:22 +00001200
Benjamin Peterson1fafc1a2011-10-25 00:03:51 -04001201.. method:: object.__bytes__(self)
1202
1203 .. index:: builtin: bytes
1204
1205 Called by :func:`bytes` to compute a byte-string representation of an
1206 object. This should return a ``bytes`` object.
1207
Chris Jerdonekbb4e9412012-11-28 01:38:40 -08001208 .. index::
1209 single: string; __format__() (object method)
1210 pair: string; conversion
1211 builtin: print
1212
Benjamin Peterson1fafc1a2011-10-25 00:03:51 -04001213
Georg Brandl4b491312007-08-31 09:22:56 +00001214.. method:: object.__format__(self, format_spec)
1215
Georg Brandl4b491312007-08-31 09:22:56 +00001216 Called by the :func:`format` built-in function (and by extension, the
Chris Jerdonekaf947242012-10-11 18:47:54 -07001217 :meth:`str.format` method of class :class:`str`) to produce a "formatted"
Georg Brandl4b491312007-08-31 09:22:56 +00001218 string representation of an object. The ``format_spec`` argument is
1219 a string that contains a description of the formatting options desired.
1220 The interpretation of the ``format_spec`` argument is up to the type
1221 implementing :meth:`__format__`, however most classes will either
1222 delegate formatting to one of the built-in types, or use a similar
1223 formatting option syntax.
Georg Brandl48310cd2009-01-03 21:18:54 +00001224
Georg Brandl4b491312007-08-31 09:22:56 +00001225 See :ref:`formatspec` for a description of the standard formatting syntax.
1226
1227 The return value must be a string object.
1228
1229
Georg Brandl33413cb2009-03-31 19:06:37 +00001230.. _richcmpfuncs:
Georg Brandl116aa622007-08-15 14:28:22 +00001231.. method:: object.__lt__(self, other)
1232 object.__le__(self, other)
1233 object.__eq__(self, other)
1234 object.__ne__(self, other)
1235 object.__gt__(self, other)
1236 object.__ge__(self, other)
1237
Guido van Rossum2cc30da2007-11-02 23:46:40 +00001238 .. index::
1239 single: comparisons
1240
Georg Brandl05f5ab72008-09-24 09:11:47 +00001241 These are the so-called "rich comparison" methods. The correspondence between
Georg Brandl116aa622007-08-15 14:28:22 +00001242 operator symbols and method names is as follows: ``x<y`` calls ``x.__lt__(y)``,
1243 ``x<=y`` calls ``x.__le__(y)``, ``x==y`` calls ``x.__eq__(y)``, ``x!=y`` calls
1244 ``x.__ne__(y)``, ``x>y`` calls ``x.__gt__(y)``, and ``x>=y`` calls
1245 ``x.__ge__(y)``.
1246
1247 A rich comparison method may return the singleton ``NotImplemented`` if it does
1248 not implement the operation for a given pair of arguments. By convention,
1249 ``False`` and ``True`` are returned for a successful comparison. However, these
1250 methods can return any value, so if the comparison operator is used in a Boolean
1251 context (e.g., in the condition of an ``if`` statement), Python will call
1252 :func:`bool` on the value to determine if the result is true or false.
1253
Guido van Rossum2cc30da2007-11-02 23:46:40 +00001254 There are no implied relationships among the comparison operators. The truth
1255 of ``x==y`` does not imply that ``x!=y`` is false. Accordingly, when
1256 defining :meth:`__eq__`, one should also define :meth:`__ne__` so that the
1257 operators will behave as expected. See the paragraph on :meth:`__hash__` for
1258 some important notes on creating :term:`hashable` objects which support
1259 custom comparison operations and are usable as dictionary keys.
Georg Brandl116aa622007-08-15 14:28:22 +00001260
Guido van Rossum2cc30da2007-11-02 23:46:40 +00001261 There are no swapped-argument versions of these methods (to be used when the
1262 left argument does not support the operation but the right argument does);
1263 rather, :meth:`__lt__` and :meth:`__gt__` are each other's reflection,
Georg Brandl116aa622007-08-15 14:28:22 +00001264 :meth:`__le__` and :meth:`__ge__` are each other's reflection, and
1265 :meth:`__eq__` and :meth:`__ne__` are their own reflection.
1266
1267 Arguments to rich comparison methods are never coerced.
1268
Raymond Hettinger6c4b4b22009-03-12 00:25:29 +00001269 To automatically generate ordering operations from a single root operation,
Raymond Hettingerc50846a2010-04-05 18:56:31 +00001270 see :func:`functools.total_ordering`.
Georg Brandl116aa622007-08-15 14:28:22 +00001271
Georg Brandl116aa622007-08-15 14:28:22 +00001272.. method:: object.__hash__(self)
1273
1274 .. index::
1275 object: dictionary
1276 builtin: hash
1277
Benjamin Peterson6cadba72008-11-19 22:38:29 +00001278 Called by built-in function :func:`hash` and for operations on members of
1279 hashed collections including :class:`set`, :class:`frozenset`, and
Barry Warsaw224a5992013-07-15 14:47:29 -04001280 :class:`dict`. :meth:`__hash__` should return an integer. The only
1281 required property is that objects which compare equal have the same hash
1282 value; it is advised to somehow mix together (e.g. using exclusive or) the
1283 hash values for the components of the object that also play a part in
1284 comparison of objects.
1285
1286 .. note::
1287
1288 :func:`hash` truncates the value returned from an object's custom
1289 :meth:`__hash__` method to the size of a :c:type:`Py_ssize_t`. This is
1290 typically 8 bytes on 64-bit builds and 4 bytes on 32-bit builds. If an
1291 object's :meth:`__hash__` must interoperate on builds of different bit
1292 sizes, be sure to check the width on all supported builds. An easy way
1293 to do this is with
1294 ``python -c "import sys; print(sys.hash_info.width)"``
Georg Brandl116aa622007-08-15 14:28:22 +00001295
Georg Brandl05f5ab72008-09-24 09:11:47 +00001296 If a class does not define an :meth:`__eq__` method it should not define a
1297 :meth:`__hash__` operation either; if it defines :meth:`__eq__` but not
Benjamin Peterson6cadba72008-11-19 22:38:29 +00001298 :meth:`__hash__`, its instances will not be usable as items in hashable
1299 collections. If a class defines mutable objects and implements an
1300 :meth:`__eq__` method, it should not implement :meth:`__hash__`, since the
1301 implementation of hashable collections requires that a key's hash value is
1302 immutable (if the object's hash value changes, it will be in the wrong hash
1303 bucket).
1304
Georg Brandl05f5ab72008-09-24 09:11:47 +00001305 User-defined classes have :meth:`__eq__` and :meth:`__hash__` methods
Nick Coghlan73c96db2008-08-31 13:21:24 +00001306 by default; with them, all objects compare unequal (except with themselves)
Nick Coghlan337b2bf2012-05-20 18:30:49 +10001307 and ``x.__hash__()`` returns an appropriate value such that ``x == y``
1308 implies both that ``x is y`` and ``hash(x) == hash(y)``.
1309
R David Murrayd8bbde32012-09-11 13:01:43 -04001310 A class that overrides :meth:`__eq__` and does not define :meth:`__hash__`
1311 will have its :meth:`__hash__` implicitly set to ``None``. When the
1312 :meth:`__hash__` method of a class is ``None``, instances of the class will
1313 raise an appropriate :exc:`TypeError` when a program attempts to retrieve
1314 their hash value, and will also be correctly identified as unhashable when
1315 checking ``isinstance(obj, collections.Hashable``).
Nick Coghlan73c96db2008-08-31 13:21:24 +00001316
Georg Brandlae2dbe22009-03-13 19:04:40 +00001317 If a class that overrides :meth:`__eq__` needs to retain the implementation
Georg Brandl05f5ab72008-09-24 09:11:47 +00001318 of :meth:`__hash__` from a parent class, the interpreter must be told this
R David Murrayd8bbde32012-09-11 13:01:43 -04001319 explicitly by setting ``__hash__ = <ParentClass>.__hash__``.
1320
1321 If a class that does not override :meth:`__eq__` wishes to suppress hash
1322 support, it should include ``__hash__ = None`` in the class definition.
1323 A class which defines its own :meth:`__hash__` that explicitly raises
1324 a :exc:`TypeError` would be incorrectly identified as hashable by
1325 an ``isinstance(obj, collections.Hashable)`` call.
Georg Brandl05f5ab72008-09-24 09:11:47 +00001326
Benjamin Petersonc9f54cf2012-02-21 16:08:05 -05001327
1328 .. note::
1329
Antoine Pitrouc86e8d92012-08-01 14:53:22 +02001330 By default, the :meth:`__hash__` values of str, bytes and datetime
Benjamin Petersonc9f54cf2012-02-21 16:08:05 -05001331 objects are "salted" with an unpredictable random value. Although they
1332 remain constant within an individual Python process, they are not
1333 predictable between repeated invocations of Python.
1334
1335 This is intended to provide protection against a denial-of-service caused
1336 by carefully-chosen inputs that exploit the worst case performance of a
1337 dict insertion, O(n^2) complexity. See
1338 http://www.ocert.org/advisories/ocert-2011-003.html for details.
1339
Antoine Pitrouc86e8d92012-08-01 14:53:22 +02001340 Changing hash values affects the iteration order of dicts, sets and
1341 other mappings. Python has never made guarantees about this ordering
1342 (and it typically varies between 32-bit and 64-bit builds).
Benjamin Petersonc9f54cf2012-02-21 16:08:05 -05001343
1344 See also :envvar:`PYTHONHASHSEED`.
1345
1346 .. versionchanged:: 3.3
1347 Hash randomization is enabled by default.
Georg Brandl2daf6ae2012-02-20 19:54:16 +01001348
Georg Brandl116aa622007-08-15 14:28:22 +00001349
1350.. method:: object.__bool__(self)
Georg Brandl1aeaadd2008-09-06 17:42:52 +00001351
Georg Brandl116aa622007-08-15 14:28:22 +00001352 .. index:: single: __len__() (mapping object method)
1353
Benjamin Petersonf07d0022009-03-21 17:31:58 +00001354 Called to implement truth value testing and the built-in operation
Amaury Forgeot d'Arc097cd072009-07-07 00:43:08 +00001355 ``bool()``; should return ``False`` or ``True``. When this method is not
1356 defined, :meth:`__len__` is called, if it is defined, and the object is
1357 considered true if its result is nonzero. If a class defines neither
1358 :meth:`__len__` nor :meth:`__bool__`, all its instances are considered
1359 true.
Georg Brandl116aa622007-08-15 14:28:22 +00001360
1361
Georg Brandl116aa622007-08-15 14:28:22 +00001362.. _attribute-access:
1363
1364Customizing attribute access
1365----------------------------
1366
1367The following methods can be defined to customize the meaning of attribute
1368access (use of, assignment to, or deletion of ``x.name``) for class instances.
1369
Georg Brandl85eb8c12007-08-31 16:33:38 +00001370.. XXX explain how descriptors interfere here!
1371
Georg Brandl116aa622007-08-15 14:28:22 +00001372
1373.. method:: object.__getattr__(self, name)
1374
1375 Called when an attribute lookup has not found the attribute in the usual places
1376 (i.e. it is not an instance attribute nor is it found in the class tree for
1377 ``self``). ``name`` is the attribute name. This method should return the
1378 (computed) attribute value or raise an :exc:`AttributeError` exception.
1379
Georg Brandl116aa622007-08-15 14:28:22 +00001380 Note that if the attribute is found through the normal mechanism,
1381 :meth:`__getattr__` is not called. (This is an intentional asymmetry between
1382 :meth:`__getattr__` and :meth:`__setattr__`.) This is done both for efficiency
Nick Coghlan3a5d7e32008-08-31 12:40:14 +00001383 reasons and because otherwise :meth:`__getattr__` would have no way to access
Georg Brandl116aa622007-08-15 14:28:22 +00001384 other attributes of the instance. Note that at least for instance variables,
1385 you can fake total control by not inserting any values in the instance attribute
1386 dictionary (but instead inserting them in another object). See the
Georg Brandl85eb8c12007-08-31 16:33:38 +00001387 :meth:`__getattribute__` method below for a way to actually get total control
1388 over attribute access.
Georg Brandl116aa622007-08-15 14:28:22 +00001389
1390
1391.. method:: object.__getattribute__(self, name)
1392
1393 Called unconditionally to implement attribute accesses for instances of the
1394 class. If the class also defines :meth:`__getattr__`, the latter will not be
1395 called unless :meth:`__getattribute__` either calls it explicitly or raises an
1396 :exc:`AttributeError`. This method should return the (computed) attribute value
1397 or raise an :exc:`AttributeError` exception. In order to avoid infinite
1398 recursion in this method, its implementation should always call the base class
1399 method with the same name to access any attributes it needs, for example,
1400 ``object.__getattribute__(self, name)``.
1401
Nick Coghlan3a5d7e32008-08-31 12:40:14 +00001402 .. note::
1403
1404 This method may still be bypassed when looking up special methods as the
Georg Brandl22b34312009-07-26 14:54:51 +00001405 result of implicit invocation via language syntax or built-in functions.
Nick Coghlan3a5d7e32008-08-31 12:40:14 +00001406 See :ref:`special-lookup`.
1407
Georg Brandl116aa622007-08-15 14:28:22 +00001408
Georg Brandl85eb8c12007-08-31 16:33:38 +00001409.. method:: object.__setattr__(self, name, value)
1410
1411 Called when an attribute assignment is attempted. This is called instead of
1412 the normal mechanism (i.e. store the value in the instance dictionary).
1413 *name* is the attribute name, *value* is the value to be assigned to it.
1414
1415 If :meth:`__setattr__` wants to assign to an instance attribute, it should
1416 call the base class method with the same name, for example,
1417 ``object.__setattr__(self, name, value)``.
1418
1419
1420.. method:: object.__delattr__(self, name)
1421
1422 Like :meth:`__setattr__` but for attribute deletion instead of assignment. This
1423 should only be implemented if ``del obj.name`` is meaningful for the object.
1424
1425
Benjamin Peterson1cef37c2008-07-02 14:44:54 +00001426.. method:: object.__dir__(self)
1427
Benjamin Peterson3bbb7222011-06-11 16:12:08 -05001428 Called when :func:`dir` is called on the object. A sequence must be
1429 returned. :func:`dir` converts the returned sequence to a list and sorts it.
Benjamin Peterson1cef37c2008-07-02 14:44:54 +00001430
1431
Georg Brandl116aa622007-08-15 14:28:22 +00001432.. _descriptors:
1433
1434Implementing Descriptors
1435^^^^^^^^^^^^^^^^^^^^^^^^
1436
1437The following methods only apply when an instance of the class containing the
Raymond Hettinger3b654be2011-03-22 16:27:02 -07001438method (a so-called *descriptor* class) appears in an *owner* class (the
1439descriptor must be in either the owner's class dictionary or in the class
1440dictionary for one of its parents). In the examples below, "the attribute"
1441refers to the attribute whose name is the key of the property in the owner
1442class' :attr:`__dict__`.
Georg Brandl116aa622007-08-15 14:28:22 +00001443
1444
1445.. method:: object.__get__(self, instance, owner)
1446
1447 Called to get the attribute of the owner class (class attribute access) or of an
1448 instance of that class (instance attribute access). *owner* is always the owner
1449 class, while *instance* is the instance that the attribute was accessed through,
1450 or ``None`` when the attribute is accessed through the *owner*. This method
1451 should return the (computed) attribute value or raise an :exc:`AttributeError`
1452 exception.
1453
1454
1455.. method:: object.__set__(self, instance, value)
1456
1457 Called to set the attribute on an instance *instance* of the owner class to a
1458 new value, *value*.
1459
1460
1461.. method:: object.__delete__(self, instance)
1462
1463 Called to delete the attribute on an instance *instance* of the owner class.
1464
1465
1466.. _descriptor-invocation:
1467
1468Invoking Descriptors
1469^^^^^^^^^^^^^^^^^^^^
1470
1471In general, a descriptor is an object attribute with "binding behavior", one
1472whose attribute access has been overridden by methods in the descriptor
1473protocol: :meth:`__get__`, :meth:`__set__`, and :meth:`__delete__`. If any of
1474those methods are defined for an object, it is said to be a descriptor.
1475
1476The default behavior for attribute access is to get, set, or delete the
1477attribute from an object's dictionary. For instance, ``a.x`` has a lookup chain
1478starting with ``a.__dict__['x']``, then ``type(a).__dict__['x']``, and
1479continuing through the base classes of ``type(a)`` excluding metaclasses.
1480
1481However, if the looked-up value is an object defining one of the descriptor
1482methods, then Python may override the default behavior and invoke the descriptor
1483method instead. Where this occurs in the precedence chain depends on which
Georg Brandl23e8db52008-04-07 19:17:06 +00001484descriptor methods were defined and how they were called.
Georg Brandl116aa622007-08-15 14:28:22 +00001485
1486The starting point for descriptor invocation is a binding, ``a.x``. How the
1487arguments are assembled depends on ``a``:
1488
1489Direct Call
1490 The simplest and least common call is when user code directly invokes a
1491 descriptor method: ``x.__get__(a)``.
1492
1493Instance Binding
Georg Brandl85eb8c12007-08-31 16:33:38 +00001494 If binding to an object instance, ``a.x`` is transformed into the call:
Georg Brandl116aa622007-08-15 14:28:22 +00001495 ``type(a).__dict__['x'].__get__(a, type(a))``.
1496
1497Class Binding
Georg Brandl85eb8c12007-08-31 16:33:38 +00001498 If binding to a class, ``A.x`` is transformed into the call:
Georg Brandl116aa622007-08-15 14:28:22 +00001499 ``A.__dict__['x'].__get__(None, A)``.
1500
1501Super Binding
1502 If ``a`` is an instance of :class:`super`, then the binding ``super(B,
1503 obj).m()`` searches ``obj.__class__.__mro__`` for the base class ``A``
1504 immediately preceding ``B`` and then invokes the descriptor with the call:
Raymond Hettingerb199b222011-03-22 15:28:45 -07001505 ``A.__dict__['m'].__get__(obj, obj.__class__)``.
Georg Brandl116aa622007-08-15 14:28:22 +00001506
1507For instance bindings, the precedence of descriptor invocation depends on the
Benjamin Peterson5e55b3e2010-02-03 02:35:45 +00001508which descriptor methods are defined. A descriptor can define any combination
1509of :meth:`__get__`, :meth:`__set__` and :meth:`__delete__`. If it does not
1510define :meth:`__get__`, then accessing the attribute will return the descriptor
1511object itself unless there is a value in the object's instance dictionary. If
1512the descriptor defines :meth:`__set__` and/or :meth:`__delete__`, it is a data
1513descriptor; if it defines neither, it is a non-data descriptor. Normally, data
1514descriptors define both :meth:`__get__` and :meth:`__set__`, while non-data
1515descriptors have just the :meth:`__get__` method. Data descriptors with
1516:meth:`__set__` and :meth:`__get__` defined always override a redefinition in an
Georg Brandl116aa622007-08-15 14:28:22 +00001517instance dictionary. In contrast, non-data descriptors can be overridden by
Benjamin Peterson5e55b3e2010-02-03 02:35:45 +00001518instances.
Georg Brandl116aa622007-08-15 14:28:22 +00001519
1520Python methods (including :func:`staticmethod` and :func:`classmethod`) are
1521implemented as non-data descriptors. Accordingly, instances can redefine and
1522override methods. This allows individual instances to acquire behaviors that
1523differ from other instances of the same class.
1524
1525The :func:`property` function is implemented as a data descriptor. Accordingly,
1526instances cannot override the behavior of a property.
1527
1528
1529.. _slots:
1530
1531__slots__
1532^^^^^^^^^
1533
Georg Brandl85eb8c12007-08-31 16:33:38 +00001534By default, instances of classes have a dictionary for attribute storage. This
1535wastes space for objects having very few instance variables. The space
1536consumption can become acute when creating large numbers of instances.
Georg Brandl116aa622007-08-15 14:28:22 +00001537
Georg Brandl85eb8c12007-08-31 16:33:38 +00001538The default can be overridden by defining *__slots__* in a class definition.
1539The *__slots__* declaration takes a sequence of instance variables and reserves
1540just enough space in each instance to hold a value for each variable. Space is
1541saved because *__dict__* is not created for each instance.
Georg Brandl116aa622007-08-15 14:28:22 +00001542
1543
Georg Brandl85eb8c12007-08-31 16:33:38 +00001544.. data:: object.__slots__
Georg Brandl116aa622007-08-15 14:28:22 +00001545
Georg Brandl85eb8c12007-08-31 16:33:38 +00001546 This class variable can be assigned a string, iterable, or sequence of
Georg Brandl23e8db52008-04-07 19:17:06 +00001547 strings with variable names used by instances. If defined in a
Georg Brandl85eb8c12007-08-31 16:33:38 +00001548 class, *__slots__* reserves space for the declared variables and prevents the
1549 automatic creation of *__dict__* and *__weakref__* for each instance.
Georg Brandl116aa622007-08-15 14:28:22 +00001550
Georg Brandl116aa622007-08-15 14:28:22 +00001551
1552Notes on using *__slots__*
Georg Brandl16174572007-09-01 12:38:06 +00001553""""""""""""""""""""""""""
Georg Brandl116aa622007-08-15 14:28:22 +00001554
Georg Brandl3dbca812008-07-23 16:10:53 +00001555* When inheriting from a class without *__slots__*, the *__dict__* attribute of
1556 that class will always be accessible, so a *__slots__* definition in the
1557 subclass is meaningless.
1558
Georg Brandl116aa622007-08-15 14:28:22 +00001559* Without a *__dict__* variable, instances cannot be assigned new variables not
1560 listed in the *__slots__* definition. Attempts to assign to an unlisted
1561 variable name raises :exc:`AttributeError`. If dynamic assignment of new
Georg Brandl85eb8c12007-08-31 16:33:38 +00001562 variables is desired, then add ``'__dict__'`` to the sequence of strings in
1563 the *__slots__* declaration.
Georg Brandl116aa622007-08-15 14:28:22 +00001564
Georg Brandl116aa622007-08-15 14:28:22 +00001565* Without a *__weakref__* variable for each instance, classes defining
1566 *__slots__* do not support weak references to its instances. If weak reference
1567 support is needed, then add ``'__weakref__'`` to the sequence of strings in the
1568 *__slots__* declaration.
1569
Georg Brandl116aa622007-08-15 14:28:22 +00001570* *__slots__* are implemented at the class level by creating descriptors
1571 (:ref:`descriptors`) for each variable name. As a result, class attributes
1572 cannot be used to set default values for instance variables defined by
1573 *__slots__*; otherwise, the class attribute would overwrite the descriptor
1574 assignment.
1575
Georg Brandl495f7b52009-10-27 15:28:25 +00001576* The action of a *__slots__* declaration is limited to the class where it is
1577 defined. As a result, subclasses will have a *__dict__* unless they also define
1578 *__slots__* (which must only contain names of any *additional* slots).
1579
Georg Brandl116aa622007-08-15 14:28:22 +00001580* If a class defines a slot also defined in a base class, the instance variable
1581 defined by the base class slot is inaccessible (except by retrieving its
1582 descriptor directly from the base class). This renders the meaning of the
1583 program undefined. In the future, a check may be added to prevent this.
1584
Benjamin Peterson1a6e0d02008-10-25 15:49:17 +00001585* Nonempty *__slots__* does not work for classes derived from "variable-length"
Zachary Ware340a6922013-12-31 12:09:26 -06001586 built-in types such as :class:`int`, :class:`bytes` and :class:`tuple`.
Georg Brandl116aa622007-08-15 14:28:22 +00001587
1588* Any non-string iterable may be assigned to *__slots__*. Mappings may also be
1589 used; however, in the future, special meaning may be assigned to the values
1590 corresponding to each key.
1591
1592* *__class__* assignment works only if both classes have the same *__slots__*.
1593
Georg Brandl116aa622007-08-15 14:28:22 +00001594
1595.. _metaclasses:
1596
1597Customizing class creation
1598--------------------------
1599
Nick Coghlan7fc570a2012-05-20 02:34:13 +10001600By default, classes are constructed using :func:`type`. The class body is
1601executed in a new namespace and the class name is bound locally to the
1602result of ``type(name, bases, namespace)``.
Georg Brandl116aa622007-08-15 14:28:22 +00001603
Nick Coghlan7fc570a2012-05-20 02:34:13 +10001604The class creation process can be customised by passing the ``metaclass``
1605keyword argument in the class definition line, or by inheriting from an
1606existing class that included such an argument. In the following example,
1607both ``MyClass`` and ``MySubclass`` are instances of ``Meta``::
Georg Brandl116aa622007-08-15 14:28:22 +00001608
Nick Coghlan7fc570a2012-05-20 02:34:13 +10001609 class Meta(type):
1610 pass
Georg Brandl116aa622007-08-15 14:28:22 +00001611
Nick Coghlan7fc570a2012-05-20 02:34:13 +10001612 class MyClass(metaclass=Meta):
1613 pass
Georg Brandl116aa622007-08-15 14:28:22 +00001614
Nick Coghlan7fc570a2012-05-20 02:34:13 +10001615 class MySubclass(MyClass):
1616 pass
Christian Heimes790c8232008-01-07 21:14:23 +00001617
Nick Coghlan7fc570a2012-05-20 02:34:13 +10001618Any other keyword arguments that are specified in the class definition are
1619passed through to all metaclass operations described below.
Christian Heimes790c8232008-01-07 21:14:23 +00001620
Nick Coghlan7fc570a2012-05-20 02:34:13 +10001621When a class definition is executed, the following steps occur:
Christian Heimes790c8232008-01-07 21:14:23 +00001622
Nick Coghlan7fc570a2012-05-20 02:34:13 +10001623* the appropriate metaclass is determined
1624* the class namespace is prepared
1625* the class body is executed
1626* the class object is created
Georg Brandl116aa622007-08-15 14:28:22 +00001627
Nick Coghlan7fc570a2012-05-20 02:34:13 +10001628Determining the appropriate metaclass
1629^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Georg Brandl116aa622007-08-15 14:28:22 +00001630
Nick Coghlan7fc570a2012-05-20 02:34:13 +10001631The appropriate metaclass for a class definition is determined as follows:
Georg Brandl116aa622007-08-15 14:28:22 +00001632
Nick Coghlan7fc570a2012-05-20 02:34:13 +10001633* if no bases and no explicit metaclass are given, then :func:`type` is used
1634* if an explicit metaclass is given and it is *not* an instance of
1635 :func:`type`, then it is used directly as the metaclass
1636* if an instance of :func:`type` is given as the explicit metaclass, or
1637 bases are defined, then the most derived metaclass is used
Georg Brandl116aa622007-08-15 14:28:22 +00001638
Nick Coghlan7fc570a2012-05-20 02:34:13 +10001639The most derived metaclass is selected from the explicitly specified
1640metaclass (if any) and the metaclasses (i.e. ``type(cls)``) of all specified
1641base classes. The most derived metaclass is one which is a subtype of *all*
1642of these candidate metaclasses. If none of the candidate metaclasses meets
1643that criterion, then the class definition will fail with ``TypeError``.
1644
1645
1646Preparing the class namespace
1647^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1648
1649Once the appropriate metaclass has been identified, then the class namespace
1650is prepared. If the metaclass has a ``__prepare__`` attribute, it is called
1651as ``namespace = metaclass.__prepare__(name, bases, **kwds)`` (where the
1652additional keyword arguments, if any, come from the class definition).
1653
1654If the metaclass has no ``__prepare__`` attribute, then the class namespace
1655is initialised as an empty :func:`dict` instance.
1656
1657.. seealso::
1658
1659 :pep:`3115` - Metaclasses in Python 3000
1660 Introduced the ``__prepare__`` namespace hook
1661
1662
1663Executing the class body
1664^^^^^^^^^^^^^^^^^^^^^^^^
1665
1666The class body is executed (approximately) as
1667``exec(body, globals(), namespace)``. The key difference from a normal
1668call to :func:`exec` is that lexical scoping allows the class body (including
1669any methods) to reference names from the current and outer scopes when the
1670class definition occurs inside a function.
1671
1672However, even when the class definition occurs inside the function, methods
1673defined inside the class still cannot see names defined at the class scope.
1674Class variables must be accessed through the first parameter of instance or
1675class methods, and cannot be accessed at all from static methods.
1676
1677
1678Creating the class object
1679^^^^^^^^^^^^^^^^^^^^^^^^^
1680
1681Once the class namespace has been populated by executing the class body,
1682the class object is created by calling
1683``metaclass(name, bases, namespace, **kwds)`` (the additional keywords
Nick Coghlan78770f02012-05-20 18:15:11 +10001684passed here are the same as those passed to ``__prepare__``).
Nick Coghlan7fc570a2012-05-20 02:34:13 +10001685
1686This class object is the one that will be referenced by the zero-argument
1687form of :func:`super`. ``__class__`` is an implicit closure reference
1688created by the compiler if any methods in a class body refer to either
1689``__class__`` or ``super``. This allows the zero argument form of
1690:func:`super` to correctly identify the class being defined based on
1691lexical scoping, while the class or instance that was used to make the
1692current call is identified based on the first argument passed to the method.
1693
Nick Coghlanb2674752012-05-20 19:36:40 +10001694After the class object is created, it is passed to the class decorators
1695included in the class definition (if any) and the resulting object is bound
1696in the local namespace as the defined class.
Nick Coghlan7fc570a2012-05-20 02:34:13 +10001697
1698.. seealso::
1699
1700 :pep:`3135` - New super
1701 Describes the implicit ``__class__`` closure reference
1702
1703
1704Metaclass example
1705^^^^^^^^^^^^^^^^^
Georg Brandl116aa622007-08-15 14:28:22 +00001706
1707The potential uses for metaclasses are boundless. Some ideas that have been
Nick Coghlan7fc570a2012-05-20 02:34:13 +10001708explored include logging, interface checking, automatic delegation, automatic
Georg Brandl116aa622007-08-15 14:28:22 +00001709property creation, proxies, frameworks, and automatic resource
1710locking/synchronization.
1711
Raymond Hettinger15efcb62009-04-07 02:09:15 +00001712Here is an example of a metaclass that uses an :class:`collections.OrderedDict`
1713to remember the order that class members were defined::
Raymond Hettinger958e3682009-04-07 02:08:23 +00001714
1715 class OrderedClass(type):
1716
1717 @classmethod
1718 def __prepare__(metacls, name, bases, **kwds):
1719 return collections.OrderedDict()
1720
Nick Coghlan7fc570a2012-05-20 02:34:13 +10001721 def __new__(cls, name, bases, namespace, **kwds):
1722 result = type.__new__(cls, name, bases, dict(namespace))
1723 result.members = tuple(namespace)
Raymond Hettinger958e3682009-04-07 02:08:23 +00001724 return result
1725
1726 class A(metaclass=OrderedClass):
1727 def one(self): pass
1728 def two(self): pass
1729 def three(self): pass
1730 def four(self): pass
1731
1732 >>> A.members
1733 ('__module__', 'one', 'two', 'three', 'four')
1734
Raymond Hettingerc4faeea2009-04-07 02:31:14 +00001735When the class definition for *A* gets executed, the process begins with
1736calling the metaclass's :meth:`__prepare__` method which returns an empty
Raymond Hettinger958e3682009-04-07 02:08:23 +00001737:class:`collections.OrderedDict`. That mapping records the methods and
1738attributes of *A* as they are defined within the body of the class statement.
Raymond Hettingerc4faeea2009-04-07 02:31:14 +00001739Once those definitions are executed, the ordered dictionary is fully populated
Hirokazu Yamamotoae9eb5c2009-04-26 03:34:06 +00001740and the metaclass's :meth:`__new__` method gets invoked. That method builds
Raymond Hettingerc4faeea2009-04-07 02:31:14 +00001741the new type and it saves the ordered dictionary keys in an attribute
Fred Drake11c49a52010-11-13 04:24:26 +00001742called ``members``.
Raymond Hettinger958e3682009-04-07 02:08:23 +00001743
Georg Brandl116aa622007-08-15 14:28:22 +00001744
Georg Brandl8569e582010-05-19 20:57:08 +00001745Customizing instance and subclass checks
1746----------------------------------------
1747
1748The following methods are used to override the default behavior of the
1749:func:`isinstance` and :func:`issubclass` built-in functions.
1750
1751In particular, the metaclass :class:`abc.ABCMeta` implements these methods in
1752order to allow the addition of Abstract Base Classes (ABCs) as "virtual base
Benjamin Petersond7c3ed52010-06-27 22:32:30 +00001753classes" to any class or type (including built-in types), including other
Georg Brandl8569e582010-05-19 20:57:08 +00001754ABCs.
1755
1756.. method:: class.__instancecheck__(self, instance)
1757
1758 Return true if *instance* should be considered a (direct or indirect)
1759 instance of *class*. If defined, called to implement ``isinstance(instance,
1760 class)``.
1761
1762
1763.. method:: class.__subclasscheck__(self, subclass)
1764
1765 Return true if *subclass* should be considered a (direct or indirect)
1766 subclass of *class*. If defined, called to implement ``issubclass(subclass,
1767 class)``.
1768
1769
1770Note that these methods are looked up on the type (metaclass) of a class. They
1771cannot be defined as class methods in the actual class. This is consistent with
Benjamin Petersond7c3ed52010-06-27 22:32:30 +00001772the lookup of special methods that are called on instances, only in this
Georg Brandl8569e582010-05-19 20:57:08 +00001773case the instance is itself a class.
1774
1775.. seealso::
1776
1777 :pep:`3119` - Introducing Abstract Base Classes
1778 Includes the specification for customizing :func:`isinstance` and
Serhiy Storchaka0d196ed2013-10-09 14:02:31 +03001779 :func:`issubclass` behavior through :meth:`~class.__instancecheck__` and
1780 :meth:`~class.__subclasscheck__`, with motivation for this functionality
1781 in the context of adding Abstract Base Classes (see the :mod:`abc`
1782 module) to the language.
Georg Brandl8569e582010-05-19 20:57:08 +00001783
1784
Georg Brandl116aa622007-08-15 14:28:22 +00001785.. _callable-types:
1786
1787Emulating callable objects
1788--------------------------
1789
1790
1791.. method:: object.__call__(self[, args...])
1792
1793 .. index:: pair: call; instance
1794
1795 Called when the instance is "called" as a function; if this method is defined,
1796 ``x(arg1, arg2, ...)`` is a shorthand for ``x.__call__(arg1, arg2, ...)``.
1797
1798
1799.. _sequence-types:
1800
1801Emulating container types
1802-------------------------
1803
1804The following methods can be defined to implement container objects. Containers
1805usually are sequences (such as lists or tuples) or mappings (like dictionaries),
1806but can represent other containers as well. The first set of methods is used
1807either to emulate a sequence or to emulate a mapping; the difference is that for
1808a sequence, the allowable keys should be the integers *k* for which ``0 <= k <
1809N`` where *N* is the length of the sequence, or slice objects, which define a
Georg Brandlcb8ecb12007-09-04 06:35:14 +00001810range of items. It is also recommended that mappings provide the methods
Georg Brandlc7723722008-05-26 17:47:11 +00001811:meth:`keys`, :meth:`values`, :meth:`items`, :meth:`get`, :meth:`clear`,
Serhiy Storchaka0d196ed2013-10-09 14:02:31 +03001812:meth:`setdefault`, :meth:`pop`, :meth:`popitem`, :meth:`!copy`, and
Georg Brandlcb8ecb12007-09-04 06:35:14 +00001813:meth:`update` behaving similar to those for Python's standard dictionary
Serhiy Storchaka0d196ed2013-10-09 14:02:31 +03001814objects. The :mod:`collections` module provides a
1815:class:`~collections.abc.MutableMapping`
Georg Brandlc7723722008-05-26 17:47:11 +00001816abstract base class to help create those methods from a base set of
1817:meth:`__getitem__`, :meth:`__setitem__`, :meth:`__delitem__`, and :meth:`keys`.
1818Mutable sequences should provide methods :meth:`append`, :meth:`count`,
1819:meth:`index`, :meth:`extend`, :meth:`insert`, :meth:`pop`, :meth:`remove`,
1820:meth:`reverse` and :meth:`sort`, like Python standard list objects. Finally,
1821sequence types should implement addition (meaning concatenation) and
1822multiplication (meaning repetition) by defining the methods :meth:`__add__`,
1823:meth:`__radd__`, :meth:`__iadd__`, :meth:`__mul__`, :meth:`__rmul__` and
1824:meth:`__imul__` described below; they should not define other numerical
1825operators. It is recommended that both mappings and sequences implement the
1826:meth:`__contains__` method to allow efficient use of the ``in`` operator; for
1827mappings, ``in`` should search the mapping's keys; for sequences, it should
1828search through the values. It is further recommended that both mappings and
1829sequences implement the :meth:`__iter__` method to allow efficient iteration
1830through the container; for mappings, :meth:`__iter__` should be the same as
Fred Drake2e748782007-09-04 17:33:11 +00001831:meth:`keys`; for sequences, it should iterate through the values.
Georg Brandl116aa622007-08-15 14:28:22 +00001832
1833.. method:: object.__len__(self)
1834
1835 .. index::
1836 builtin: len
1837 single: __bool__() (object method)
1838
1839 Called to implement the built-in function :func:`len`. Should return the length
1840 of the object, an integer ``>=`` 0. Also, an object that doesn't define a
1841 :meth:`__bool__` method and whose :meth:`__len__` method returns zero is
1842 considered to be false in a Boolean context.
1843
1844
Armin Ronacher74b38b12012-10-07 10:29:32 +02001845.. method:: object.__length_hint__(self)
1846
Ezio Melottie12dc282012-10-07 12:09:36 +03001847 Called to implement :func:`operator.length_hint`. Should return an estimated
Armin Ronacher74b38b12012-10-07 10:29:32 +02001848 length for the object (which may be greater or less than the actual length).
1849 The length must be an integer ``>=`` 0. This method is purely an
1850 optimization and is never required for correctness.
1851
1852 .. versionadded:: 3.4
1853
Georg Brandlcb8ecb12007-09-04 06:35:14 +00001854.. note::
1855
1856 Slicing is done exclusively with the following three methods. A call like ::
1857
1858 a[1:2] = b
1859
1860 is translated to ::
1861
1862 a[slice(1, 2, None)] = b
1863
1864 and so forth. Missing slice items are always filled in with ``None``.
1865
1866
Georg Brandl116aa622007-08-15 14:28:22 +00001867.. method:: object.__getitem__(self, key)
1868
1869 .. index:: object: slice
1870
1871 Called to implement evaluation of ``self[key]``. For sequence types, the
1872 accepted keys should be integers and slice objects. Note that the special
1873 interpretation of negative indexes (if the class wishes to emulate a sequence
1874 type) is up to the :meth:`__getitem__` method. If *key* is of an inappropriate
1875 type, :exc:`TypeError` may be raised; if of a value outside the set of indexes
1876 for the sequence (after any special interpretation of negative values),
1877 :exc:`IndexError` should be raised. For mapping types, if *key* is missing (not
1878 in the container), :exc:`KeyError` should be raised.
1879
1880 .. note::
1881
1882 :keyword:`for` loops expect that an :exc:`IndexError` will be raised for illegal
1883 indexes to allow proper detection of the end of the sequence.
1884
1885
1886.. method:: object.__setitem__(self, key, value)
1887
1888 Called to implement assignment to ``self[key]``. Same note as for
1889 :meth:`__getitem__`. This should only be implemented for mappings if the
1890 objects support changes to the values for keys, or if new keys can be added, or
1891 for sequences if elements can be replaced. The same exceptions should be raised
1892 for improper *key* values as for the :meth:`__getitem__` method.
1893
1894
1895.. method:: object.__delitem__(self, key)
1896
1897 Called to implement deletion of ``self[key]``. Same note as for
1898 :meth:`__getitem__`. This should only be implemented for mappings if the
1899 objects support removal of keys, or for sequences if elements can be removed
1900 from the sequence. The same exceptions should be raised for improper *key*
1901 values as for the :meth:`__getitem__` method.
1902
1903
1904.. method:: object.__iter__(self)
1905
1906 This method is called when an iterator is required for a container. This method
1907 should return a new iterator object that can iterate over all the objects in the
1908 container. For mappings, it should iterate over the keys of the container, and
Fred Drake2e748782007-09-04 17:33:11 +00001909 should also be made available as the method :meth:`keys`.
Georg Brandl116aa622007-08-15 14:28:22 +00001910
1911 Iterator objects also need to implement this method; they are required to return
1912 themselves. For more information on iterator objects, see :ref:`typeiter`.
1913
Christian Heimes7f044312008-01-06 17:05:40 +00001914
1915.. method:: object.__reversed__(self)
1916
Georg Brandl22b34312009-07-26 14:54:51 +00001917 Called (if present) by the :func:`reversed` built-in to implement
Christian Heimes7f044312008-01-06 17:05:40 +00001918 reverse iteration. It should return a new iterator object that iterates
1919 over all the objects in the container in reverse order.
1920
Georg Brandl8a1e4c42009-05-25 21:13:36 +00001921 If the :meth:`__reversed__` method is not provided, the :func:`reversed`
Georg Brandl22b34312009-07-26 14:54:51 +00001922 built-in will fall back to using the sequence protocol (:meth:`__len__` and
Georg Brandl8a1e4c42009-05-25 21:13:36 +00001923 :meth:`__getitem__`). Objects that support the sequence protocol should
1924 only provide :meth:`__reversed__` if they can provide an implementation
1925 that is more efficient than the one provided by :func:`reversed`.
Christian Heimes7f044312008-01-06 17:05:40 +00001926
1927
Georg Brandl116aa622007-08-15 14:28:22 +00001928The membership test operators (:keyword:`in` and :keyword:`not in`) are normally
1929implemented as an iteration through a sequence. However, container objects can
1930supply the following special method with a more efficient implementation, which
1931also does not require the object be a sequence.
1932
Georg Brandl116aa622007-08-15 14:28:22 +00001933.. method:: object.__contains__(self, item)
1934
Georg Brandl495f7b52009-10-27 15:28:25 +00001935 Called to implement membership test operators. Should return true if *item*
1936 is in *self*, false otherwise. For mapping objects, this should consider the
1937 keys of the mapping rather than the values or the key-item pairs.
1938
1939 For objects that don't define :meth:`__contains__`, the membership test first
1940 tries iteration via :meth:`__iter__`, then the old sequence iteration
1941 protocol via :meth:`__getitem__`, see :ref:`this section in the language
1942 reference <membership-test-details>`.
Georg Brandl116aa622007-08-15 14:28:22 +00001943
1944
Georg Brandl116aa622007-08-15 14:28:22 +00001945.. _numeric-types:
1946
1947Emulating numeric types
1948-----------------------
1949
1950The following methods can be defined to emulate numeric objects. Methods
1951corresponding to operations that are not supported by the particular kind of
1952number implemented (e.g., bitwise operations for non-integral numbers) should be
1953left undefined.
1954
1955
1956.. method:: object.__add__(self, other)
1957 object.__sub__(self, other)
1958 object.__mul__(self, other)
Georg Brandlae55dc02008-09-06 17:43:49 +00001959 object.__truediv__(self, other)
Georg Brandl116aa622007-08-15 14:28:22 +00001960 object.__floordiv__(self, other)
1961 object.__mod__(self, other)
1962 object.__divmod__(self, other)
1963 object.__pow__(self, other[, modulo])
1964 object.__lshift__(self, other)
1965 object.__rshift__(self, other)
1966 object.__and__(self, other)
1967 object.__xor__(self, other)
1968 object.__or__(self, other)
1969
1970 .. index::
1971 builtin: divmod
1972 builtin: pow
1973 builtin: pow
1974
1975 These methods are called to implement the binary arithmetic operations (``+``,
Georg Brandlae55dc02008-09-06 17:43:49 +00001976 ``-``, ``*``, ``/``, ``//``, ``%``, :func:`divmod`, :func:`pow`, ``**``, ``<<``,
Georg Brandl116aa622007-08-15 14:28:22 +00001977 ``>>``, ``&``, ``^``, ``|``). For instance, to evaluate the expression
Brett Cannon3a954da2008-08-14 05:59:39 +00001978 ``x + y``, where *x* is an instance of a class that has an :meth:`__add__`
Georg Brandl116aa622007-08-15 14:28:22 +00001979 method, ``x.__add__(y)`` is called. The :meth:`__divmod__` method should be the
1980 equivalent to using :meth:`__floordiv__` and :meth:`__mod__`; it should not be
Georg Brandlae55dc02008-09-06 17:43:49 +00001981 related to :meth:`__truediv__`. Note that :meth:`__pow__` should be defined
1982 to accept an optional third argument if the ternary version of the built-in
1983 :func:`pow` function is to be supported.
Georg Brandl116aa622007-08-15 14:28:22 +00001984
1985 If one of those methods does not support the operation with the supplied
1986 arguments, it should return ``NotImplemented``.
1987
1988
Georg Brandl116aa622007-08-15 14:28:22 +00001989.. method:: object.__radd__(self, other)
1990 object.__rsub__(self, other)
1991 object.__rmul__(self, other)
Georg Brandl116aa622007-08-15 14:28:22 +00001992 object.__rtruediv__(self, other)
1993 object.__rfloordiv__(self, other)
1994 object.__rmod__(self, other)
1995 object.__rdivmod__(self, other)
1996 object.__rpow__(self, other)
1997 object.__rlshift__(self, other)
1998 object.__rrshift__(self, other)
1999 object.__rand__(self, other)
2000 object.__rxor__(self, other)
2001 object.__ror__(self, other)
2002
2003 .. index::
2004 builtin: divmod
2005 builtin: pow
2006
2007 These methods are called to implement the binary arithmetic operations (``+``,
Georg Brandlae55dc02008-09-06 17:43:49 +00002008 ``-``, ``*``, ``/``, ``//``, ``%``, :func:`divmod`, :func:`pow`, ``**``,
2009 ``<<``, ``>>``, ``&``, ``^``, ``|``) with reflected (swapped) operands.
2010 These functions are only called if the left operand does not support the
2011 corresponding operation and the operands are of different types. [#]_ For
2012 instance, to evaluate the expression ``x - y``, where *y* is an instance of
2013 a class that has an :meth:`__rsub__` method, ``y.__rsub__(x)`` is called if
2014 ``x.__sub__(y)`` returns *NotImplemented*.
Georg Brandl116aa622007-08-15 14:28:22 +00002015
2016 .. index:: builtin: pow
2017
2018 Note that ternary :func:`pow` will not try calling :meth:`__rpow__` (the
2019 coercion rules would become too complicated).
2020
2021 .. note::
2022
2023 If the right operand's type is a subclass of the left operand's type and that
2024 subclass provides the reflected method for the operation, this method will be
2025 called before the left operand's non-reflected method. This behavior allows
2026 subclasses to override their ancestors' operations.
2027
2028
2029.. method:: object.__iadd__(self, other)
2030 object.__isub__(self, other)
2031 object.__imul__(self, other)
Georg Brandl116aa622007-08-15 14:28:22 +00002032 object.__itruediv__(self, other)
2033 object.__ifloordiv__(self, other)
2034 object.__imod__(self, other)
2035 object.__ipow__(self, other[, modulo])
2036 object.__ilshift__(self, other)
2037 object.__irshift__(self, other)
2038 object.__iand__(self, other)
2039 object.__ixor__(self, other)
2040 object.__ior__(self, other)
2041
Benjamin Petersonb58dda72009-01-18 22:27:04 +00002042 These methods are called to implement the augmented arithmetic assignments
Georg Brandl116aa622007-08-15 14:28:22 +00002043 (``+=``, ``-=``, ``*=``, ``/=``, ``//=``, ``%=``, ``**=``, ``<<=``, ``>>=``,
2044 ``&=``, ``^=``, ``|=``). These methods should attempt to do the operation
2045 in-place (modifying *self*) and return the result (which could be, but does
2046 not have to be, *self*). If a specific method is not defined, the augmented
Benjamin Petersonb58dda72009-01-18 22:27:04 +00002047 assignment falls back to the normal methods. For instance, to execute the
2048 statement ``x += y``, where *x* is an instance of a class that has an
Georg Brandl116aa622007-08-15 14:28:22 +00002049 :meth:`__iadd__` method, ``x.__iadd__(y)`` is called. If *x* is an instance
2050 of a class that does not define a :meth:`__iadd__` method, ``x.__add__(y)``
Brett Cannon3a954da2008-08-14 05:59:39 +00002051 and ``y.__radd__(x)`` are considered, as with the evaluation of ``x + y``.
Georg Brandl116aa622007-08-15 14:28:22 +00002052
2053
2054.. method:: object.__neg__(self)
2055 object.__pos__(self)
2056 object.__abs__(self)
2057 object.__invert__(self)
2058
2059 .. index:: builtin: abs
2060
2061 Called to implement the unary arithmetic operations (``-``, ``+``, :func:`abs`
2062 and ``~``).
2063
2064
2065.. method:: object.__complex__(self)
2066 object.__int__(self)
Georg Brandl116aa622007-08-15 14:28:22 +00002067 object.__float__(self)
Mark Summerfield9557f602008-07-01 14:42:30 +00002068 object.__round__(self, [,n])
Georg Brandl116aa622007-08-15 14:28:22 +00002069
2070 .. index::
2071 builtin: complex
2072 builtin: int
Georg Brandl116aa622007-08-15 14:28:22 +00002073 builtin: float
Mark Summerfield9557f602008-07-01 14:42:30 +00002074 builtin: round
Georg Brandl116aa622007-08-15 14:28:22 +00002075
Mark Summerfield9557f602008-07-01 14:42:30 +00002076 Called to implement the built-in functions :func:`complex`,
2077 :func:`int`, :func:`float` and :func:`round`. Should return a value
2078 of the appropriate type.
Georg Brandl116aa622007-08-15 14:28:22 +00002079
2080
2081.. method:: object.__index__(self)
2082
2083 Called to implement :func:`operator.index`. Also called whenever Python needs
2084 an integer object (such as in slicing, or in the built-in :func:`bin`,
Georg Brandl5c106642007-11-29 17:41:05 +00002085 :func:`hex` and :func:`oct` functions). Must return an integer.
Georg Brandl116aa622007-08-15 14:28:22 +00002086
Georg Brandl116aa622007-08-15 14:28:22 +00002087
2088.. _context-managers:
2089
2090With Statement Context Managers
2091-------------------------------
2092
Georg Brandl116aa622007-08-15 14:28:22 +00002093A :dfn:`context manager` is an object that defines the runtime context to be
2094established when executing a :keyword:`with` statement. The context manager
2095handles the entry into, and the exit from, the desired runtime context for the
2096execution of the block of code. Context managers are normally invoked using the
2097:keyword:`with` statement (described in section :ref:`with`), but can also be
2098used by directly invoking their methods.
2099
2100.. index::
2101 statement: with
2102 single: context manager
2103
2104Typical uses of context managers include saving and restoring various kinds of
2105global state, locking and unlocking resources, closing opened files, etc.
2106
2107For more information on context managers, see :ref:`typecontextmanager`.
2108
2109
2110.. method:: object.__enter__(self)
2111
2112 Enter the runtime context related to this object. The :keyword:`with` statement
2113 will bind this method's return value to the target(s) specified in the
2114 :keyword:`as` clause of the statement, if any.
2115
2116
2117.. method:: object.__exit__(self, exc_type, exc_value, traceback)
2118
2119 Exit the runtime context related to this object. The parameters describe the
2120 exception that caused the context to be exited. If the context was exited
2121 without an exception, all three arguments will be :const:`None`.
2122
2123 If an exception is supplied, and the method wishes to suppress the exception
2124 (i.e., prevent it from being propagated), it should return a true value.
2125 Otherwise, the exception will be processed normally upon exit from this method.
2126
2127 Note that :meth:`__exit__` methods should not reraise the passed-in exception;
2128 this is the caller's responsibility.
2129
2130
2131.. seealso::
2132
2133 :pep:`0343` - The "with" statement
2134 The specification, background, and examples for the Python :keyword:`with`
2135 statement.
2136
Nick Coghlan3a5d7e32008-08-31 12:40:14 +00002137
2138.. _special-lookup:
2139
2140Special method lookup
2141---------------------
2142
2143For custom classes, implicit invocations of special methods are only guaranteed
2144to work correctly if defined on an object's type, not in the object's instance
2145dictionary. That behaviour is the reason why the following code raises an
2146exception::
2147
Éric Araujo28053fb2010-11-22 03:09:19 +00002148 >>> class C:
Nick Coghlan3a5d7e32008-08-31 12:40:14 +00002149 ... pass
2150 ...
2151 >>> c = C()
2152 >>> c.__len__ = lambda: 5
2153 >>> len(c)
2154 Traceback (most recent call last):
2155 File "<stdin>", line 1, in <module>
2156 TypeError: object of type 'C' has no len()
2157
2158The rationale behind this behaviour lies with a number of special methods such
2159as :meth:`__hash__` and :meth:`__repr__` that are implemented by all objects,
2160including type objects. If the implicit lookup of these methods used the
2161conventional lookup process, they would fail when invoked on the type object
2162itself::
2163
2164 >>> 1 .__hash__() == hash(1)
2165 True
2166 >>> int.__hash__() == hash(int)
2167 Traceback (most recent call last):
2168 File "<stdin>", line 1, in <module>
2169 TypeError: descriptor '__hash__' of 'int' object needs an argument
2170
2171Incorrectly attempting to invoke an unbound method of a class in this way is
2172sometimes referred to as 'metaclass confusion', and is avoided by bypassing
2173the instance when looking up special methods::
2174
2175 >>> type(1).__hash__(1) == hash(1)
2176 True
2177 >>> type(int).__hash__(int) == hash(int)
2178 True
2179
2180In addition to bypassing any instance attributes in the interest of
Georg Brandlaf265f42008-12-07 15:06:20 +00002181correctness, implicit special method lookup generally also bypasses the
Nick Coghlan3a5d7e32008-08-31 12:40:14 +00002182:meth:`__getattribute__` method even of the object's metaclass::
2183
2184 >>> class Meta(type):
2185 ... def __getattribute__(*args):
Benjamin Peterson64106fb2008-10-29 20:35:35 +00002186 ... print("Metaclass getattribute invoked")
Nick Coghlan3a5d7e32008-08-31 12:40:14 +00002187 ... return type.__getattribute__(*args)
2188 ...
Benjamin Petersone348d1a2008-10-19 21:29:05 +00002189 >>> class C(object, metaclass=Meta):
Nick Coghlan3a5d7e32008-08-31 12:40:14 +00002190 ... def __len__(self):
2191 ... return 10
2192 ... def __getattribute__(*args):
Benjamin Peterson64106fb2008-10-29 20:35:35 +00002193 ... print("Class getattribute invoked")
Nick Coghlan3a5d7e32008-08-31 12:40:14 +00002194 ... return object.__getattribute__(*args)
2195 ...
2196 >>> c = C()
2197 >>> c.__len__() # Explicit lookup via instance
2198 Class getattribute invoked
2199 10
2200 >>> type(c).__len__(c) # Explicit lookup via type
2201 Metaclass getattribute invoked
2202 10
2203 >>> len(c) # Implicit lookup
2204 10
2205
2206Bypassing the :meth:`__getattribute__` machinery in this fashion
2207provides significant scope for speed optimisations within the
2208interpreter, at the cost of some flexibility in the handling of
2209special methods (the special method *must* be set on the class
2210object itself in order to be consistently invoked by the interpreter).
2211
2212
Georg Brandl116aa622007-08-15 14:28:22 +00002213.. rubric:: Footnotes
2214
Nick Coghlan3a5d7e32008-08-31 12:40:14 +00002215.. [#] It *is* possible in some cases to change an object's type, under certain
2216 controlled conditions. It generally isn't a good idea though, since it can
2217 lead to some very strange behaviour if it is handled incorrectly.
2218
Georg Brandl116aa622007-08-15 14:28:22 +00002219.. [#] For operands of the same type, it is assumed that if the non-reflected method
2220 (such as :meth:`__add__`) fails the operation is not supported, which is why the
2221 reflected method is not called.