blob: 0ab5f115c8f567bf1d9f3f4071e15777c44f5c57 [file] [log] [blame]
Georg Brandl116aa622007-08-15 14:28:22 +00001
2.. _datamodel:
3
4**********
5Data model
6**********
7
8
9.. _objects:
10
11Objects, values and types
12=========================
13
14.. index::
15 single: object
16 single: data
17
18:dfn:`Objects` are Python's abstraction for data. All data in a Python program
19is represented by objects or by relations between objects. (In a sense, and in
20conformance to Von Neumann's model of a "stored program computer," code is also
21represented by objects.)
22
23.. index::
24 builtin: id
25 builtin: type
26 single: identity of an object
27 single: value of an object
28 single: type of an object
29 single: mutable object
30 single: immutable object
31
Georg Brandl85eb8c12007-08-31 16:33:38 +000032.. XXX it *is* now possible in some cases to change an object's
33 type, under certain controlled conditions
34
Georg Brandl116aa622007-08-15 14:28:22 +000035Every object has an identity, a type and a value. An object's *identity* never
36changes once it has been created; you may think of it as the object's address in
37memory. The ':keyword:`is`' operator compares the identity of two objects; the
38:func:`id` function returns an integer representing its identity (currently
Georg Brandl85eb8c12007-08-31 16:33:38 +000039implemented as its address). An object's :dfn:`type` is also unchangeable.
Georg Brandl116aa622007-08-15 14:28:22 +000040An object's type determines the operations that the object supports (e.g., "does
41it have a length?") and also defines the possible values for objects of that
42type. The :func:`type` function returns an object's type (which is an object
43itself). The *value* of some objects can change. Objects whose value can
44change are said to be *mutable*; objects whose value is unchangeable once they
45are created are called *immutable*. (The value of an immutable container object
46that contains a reference to a mutable object can change when the latter's value
47is changed; however the container is still considered immutable, because the
48collection of objects it contains cannot be changed. So, immutability is not
49strictly the same as having an unchangeable value, it is more subtle.) An
50object's mutability is determined by its type; for instance, numbers, strings
51and tuples are immutable, while dictionaries and lists are mutable.
52
53.. index::
54 single: garbage collection
55 single: reference counting
56 single: unreachable object
57
58Objects are never explicitly destroyed; however, when they become unreachable
59they may be garbage-collected. An implementation is allowed to postpone garbage
60collection or omit it altogether --- it is a matter of implementation quality
61how garbage collection is implemented, as long as no objects are collected that
62are still reachable. (Implementation note: the current implementation uses a
63reference-counting scheme with (optional) delayed detection of cyclically linked
64garbage, which collects most objects as soon as they become unreachable, but is
65not guaranteed to collect garbage containing circular references. See the
66documentation of the :mod:`gc` module for information on controlling the
67collection of cyclic garbage.)
68
69Note that the use of the implementation's tracing or debugging facilities may
70keep objects alive that would normally be collectable. Also note that catching
71an exception with a ':keyword:`try`...\ :keyword:`except`' statement may keep
72objects alive.
73
74Some objects contain references to "external" resources such as open files or
75windows. It is understood that these resources are freed when the object is
76garbage-collected, but since garbage collection is not guaranteed to happen,
77such objects also provide an explicit way to release the external resource,
78usually a :meth:`close` method. Programs are strongly recommended to explicitly
79close such objects. The ':keyword:`try`...\ :keyword:`finally`' statement
80provides a convenient way to do this.
81
82.. index:: single: container
83
84Some objects contain references to other objects; these are called *containers*.
85Examples of containers are tuples, lists and dictionaries. The references are
86part of a container's value. In most cases, when we talk about the value of a
87container, we imply the values, not the identities of the contained objects;
88however, when we talk about the mutability of a container, only the identities
89of the immediately contained objects are implied. So, if an immutable container
90(like a tuple) contains a reference to a mutable object, its value changes if
91that mutable object is changed.
92
93Types affect almost all aspects of object behavior. Even the importance of
94object identity is affected in some sense: for immutable types, operations that
95compute new values may actually return a reference to any existing object with
96the same type and value, while for mutable objects this is not allowed. E.g.,
97after ``a = 1; b = 1``, ``a`` and ``b`` may or may not refer to the same object
98with the value one, depending on the implementation, but after ``c = []; d =
99[]``, ``c`` and ``d`` are guaranteed to refer to two different, unique, newly
100created empty lists. (Note that ``c = d = []`` assigns the same object to both
101``c`` and ``d``.)
102
103
104.. _types:
105
106The standard type hierarchy
107===========================
108
109.. index::
110 single: type
111 pair: data; type
112 pair: type; hierarchy
113 pair: extension; module
114 pair: C; language
115
116Below is a list of the types that are built into Python. Extension modules
117(written in C, Java, or other languages, depending on the implementation) can
118define additional types. Future versions of Python may add types to the type
119hierarchy (e.g., rational numbers, efficiently stored arrays of integers, etc.).
120
121.. index::
122 single: attribute
123 pair: special; attribute
124 triple: generic; special; attribute
125
126Some of the type descriptions below contain a paragraph listing 'special
127attributes.' These are attributes that provide access to the implementation and
128are not intended for general use. Their definition may change in the future.
129
130None
131 .. index:: object: None
132
133 This type has a single value. There is a single object with this value. This
134 object is accessed through the built-in name ``None``. It is used to signify the
135 absence of a value in many situations, e.g., it is returned from functions that
136 don't explicitly return anything. Its truth value is false.
137
138NotImplemented
139 .. index:: object: NotImplemented
140
141 This type has a single value. There is a single object with this value. This
142 object is accessed through the built-in name ``NotImplemented``. Numeric methods
143 and rich comparison methods may return this value if they do not implement the
144 operation for the operands provided. (The interpreter will then try the
145 reflected operation, or some other fallback, depending on the operator.) Its
146 truth value is true.
147
148Ellipsis
149 .. index:: object: Ellipsis
150
151 This type has a single value. There is a single object with this value. This
152 object is accessed through the literal ``...`` or the built-in name
153 ``Ellipsis``. Its truth value is true.
154
Christian Heimes072c0f12008-01-03 23:01:04 +0000155:class:`numbers.Number`
Georg Brandl116aa622007-08-15 14:28:22 +0000156 .. index:: object: numeric
157
158 These are created by numeric literals and returned as results by arithmetic
159 operators and arithmetic built-in functions. Numeric objects are immutable;
160 once created their value never changes. Python numbers are of course strongly
161 related to mathematical numbers, but subject to the limitations of numerical
162 representation in computers.
163
164 Python distinguishes between integers, floating point numbers, and complex
165 numbers:
166
Christian Heimes072c0f12008-01-03 23:01:04 +0000167 :class:`numbers.Integral`
Georg Brandl116aa622007-08-15 14:28:22 +0000168 .. index:: object: integer
169
170 These represent elements from the mathematical set of integers (positive and
171 negative).
172
Georg Brandl59d69162008-01-07 09:27:36 +0000173 There are two types of integers:
Georg Brandl116aa622007-08-15 14:28:22 +0000174
175 Plain integers
176 .. index::
177 object: plain integer
178 single: OverflowError (built-in exception)
179
Georg Brandl116aa622007-08-15 14:28:22 +0000180 These represent numbers in an unlimited range, subject to available (virtual)
181 memory only. For the purpose of shift and mask operations, a binary
182 representation is assumed, and negative numbers are represented in a variant of
183 2's complement which gives the illusion of an infinite string of sign bits
184 extending to the left.
185
186 Booleans
187 .. index::
188 object: Boolean
189 single: False
190 single: True
191
192 These represent the truth values False and True. The two objects representing
193 the values False and True are the only Boolean objects. The Boolean type is a
194 subtype of plain integers, and Boolean values behave like the values 0 and 1,
195 respectively, in almost all contexts, the exception being that when converted to
196 a string, the strings ``"False"`` or ``"True"`` are returned, respectively.
197
198 .. index:: pair: integer; representation
199
200 The rules for integer representation are intended to give the most meaningful
Georg Brandlba956ae2007-11-29 17:24:34 +0000201 interpretation of shift and mask operations involving negative integers. Any
Georg Brandl116aa622007-08-15 14:28:22 +0000202 operation except left shift, if it yields a result in the plain integer domain
Georg Brandlba956ae2007-11-29 17:24:34 +0000203 without causing overflow, will yield the same result when using mixed operands.
Georg Brandl116aa622007-08-15 14:28:22 +0000204
Christian Heimes072c0f12008-01-03 23:01:04 +0000205 :class:`numbers.Real` (:class:`float`)
Georg Brandl116aa622007-08-15 14:28:22 +0000206 .. index::
207 object: floating point
208 pair: floating point; number
209 pair: C; language
210 pair: Java; language
211
212 These represent machine-level double precision floating point numbers. You are
213 at the mercy of the underlying machine architecture (and C or Java
214 implementation) for the accepted range and handling of overflow. Python does not
215 support single-precision floating point numbers; the savings in processor and
216 memory usage that are usually the reason for using these is dwarfed by the
217 overhead of using objects in Python, so there is no reason to complicate the
218 language with two kinds of floating point numbers.
219
Christian Heimes072c0f12008-01-03 23:01:04 +0000220 :class:`numbers.Complex`
Georg Brandl116aa622007-08-15 14:28:22 +0000221 .. index::
222 object: complex
223 pair: complex; number
224
225 These represent complex numbers as a pair of machine-level double precision
226 floating point numbers. The same caveats apply as for floating point numbers.
227 The real and imaginary parts of a complex number ``z`` can be retrieved through
228 the read-only attributes ``z.real`` and ``z.imag``.
229
Georg Brandl116aa622007-08-15 14:28:22 +0000230Sequences
231 .. index::
232 builtin: len
233 object: sequence
234 single: index operation
235 single: item selection
236 single: subscription
237
238 These represent finite ordered sets indexed by non-negative numbers. The
239 built-in function :func:`len` returns the number of items of a sequence. When
240 the length of a sequence is *n*, the index set contains the numbers 0, 1,
241 ..., *n*-1. Item *i* of sequence *a* is selected by ``a[i]``.
242
243 .. index:: single: slicing
244
245 Sequences also support slicing: ``a[i:j]`` selects all items with index *k* such
246 that *i* ``<=`` *k* ``<`` *j*. When used as an expression, a slice is a
247 sequence of the same type. This implies that the index set is renumbered so
248 that it starts at 0.
249
Georg Brandl116aa622007-08-15 14:28:22 +0000250 Some sequences also support "extended slicing" with a third "step" parameter:
251 ``a[i:j:k]`` selects all items of *a* with index *x* where ``x = i + n*k``, *n*
252 ``>=`` ``0`` and *i* ``<=`` *x* ``<`` *j*.
253
254 Sequences are distinguished according to their mutability:
255
256 Immutable sequences
257 .. index::
258 object: immutable sequence
259 object: immutable
260
261 An object of an immutable sequence type cannot change once it is created. (If
262 the object contains references to other objects, these other objects may be
263 mutable and may be changed; however, the collection of objects directly
264 referenced by an immutable object cannot change.)
265
266 The following types are immutable sequences:
267
268 Strings
269 .. index::
270 builtin: chr
271 builtin: ord
Georg Brandldcc56f82007-08-31 16:41:12 +0000272 builtin: str
Georg Brandl116aa622007-08-15 14:28:22 +0000273 single: character
274 single: integer
275 single: Unicode
276
Georg Brandldcc56f82007-08-31 16:41:12 +0000277 The items of a string object are Unicode code units. A Unicode code
278 unit is represented by a string object of one item and can hold either
279 a 16-bit or 32-bit value representing a Unicode ordinal (the maximum
280 value for the ordinal is given in ``sys.maxunicode``, and depends on
281 how Python is configured at compile time). Surrogate pairs may be
282 present in the Unicode object, and will be reported as two separate
283 items. The built-in functions :func:`chr` and :func:`ord` convert
284 between code units and nonnegative integers representing the Unicode
285 ordinals as defined in the Unicode Standard 3.0. Conversion from and to
286 other encodings are possible through the string method :meth:`encode`.
Georg Brandl116aa622007-08-15 14:28:22 +0000287
288 Tuples
289 .. index::
290 object: tuple
291 pair: singleton; tuple
292 pair: empty; tuple
293
Georg Brandldcc56f82007-08-31 16:41:12 +0000294 The items of a tuple are arbitrary Python objects. Tuples of two or
295 more items are formed by comma-separated lists of expressions. A tuple
296 of one item (a 'singleton') can be formed by affixing a comma to an
297 expression (an expression by itself does not create a tuple, since
298 parentheses must be usable for grouping of expressions). An empty
299 tuple can be formed by an empty pair of parentheses.
Georg Brandl116aa622007-08-15 14:28:22 +0000300
Georg Brandl116aa622007-08-15 14:28:22 +0000301 Mutable sequences
302 .. index::
303 object: mutable sequence
304 object: mutable
305 pair: assignment; statement
306 single: delete
307 statement: del
308 single: subscription
309 single: slicing
310
311 Mutable sequences can be changed after they are created. The subscription and
312 slicing notations can be used as the target of assignment and :keyword:`del`
313 (delete) statements.
314
315 There is currently a single intrinsic mutable sequence type:
316
317 Lists
318 .. index:: object: list
319
Georg Brandldcc56f82007-08-31 16:41:12 +0000320 The items of a list are arbitrary Python objects. Lists are formed by
321 placing a comma-separated list of expressions in square brackets. (Note
322 that there are no special cases needed to form lists of length 0 or 1.)
323
324 Bytes
325 .. index:: bytes, byte
326
327 A bytes object is a mutable array. The items are 8-bit bytes,
328 represented by integers in the range 0 <= x < 256. Bytes literals
329 (like ``b'abc'`` and the built-in function :func:`bytes` can be used to
330 construct bytes objects. Also, bytes objects can be decoded to strings
331 via the :meth:`decode` method.
Georg Brandl116aa622007-08-15 14:28:22 +0000332
333 .. index:: module: array
334
Georg Brandldcc56f82007-08-31 16:41:12 +0000335 The extension module :mod:`array` provides an additional example of a
336 mutable sequence type.
Georg Brandl116aa622007-08-15 14:28:22 +0000337
Georg Brandl116aa622007-08-15 14:28:22 +0000338Set types
339 .. index::
340 builtin: len
341 object: set type
342
343 These represent unordered, finite sets of unique, immutable objects. As such,
344 they cannot be indexed by any subscript. However, they can be iterated over, and
345 the built-in function :func:`len` returns the number of items in a set. Common
346 uses for sets are fast membership testing, removing duplicates from a sequence,
347 and computing mathematical operations such as intersection, union, difference,
348 and symmetric difference.
349
350 For set elements, the same immutability rules apply as for dictionary keys. Note
351 that numeric types obey the normal rules for numeric comparison: if two numbers
352 compare equal (e.g., ``1`` and ``1.0``), only one of them can be contained in a
353 set.
354
355 There are currently two intrinsic set types:
356
357 Sets
358 .. index:: object: set
359
360 These represent a mutable set. They are created by the built-in :func:`set`
361 constructor and can be modified afterwards by several methods, such as
362 :meth:`add`.
363
364 Frozen sets
365 .. index:: object: frozenset
366
Guido van Rossum2cc30da2007-11-02 23:46:40 +0000367 These represent an immutable set. They are created by the built-in
368 :func:`frozenset` constructor. As a frozenset is immutable and
369 :term:`hashable`, it can be used again as an element of another set, or as
370 a dictionary key.
Georg Brandl116aa622007-08-15 14:28:22 +0000371
Georg Brandl116aa622007-08-15 14:28:22 +0000372Mappings
373 .. index::
374 builtin: len
375 single: subscription
376 object: mapping
377
378 These represent finite sets of objects indexed by arbitrary index sets. The
379 subscript notation ``a[k]`` selects the item indexed by ``k`` from the mapping
380 ``a``; this can be used in expressions and as the target of assignments or
381 :keyword:`del` statements. The built-in function :func:`len` returns the number
382 of items in a mapping.
383
384 There is currently a single intrinsic mapping type:
385
386 Dictionaries
387 .. index:: object: dictionary
388
389 These represent finite sets of objects indexed by nearly arbitrary values. The
390 only types of values not acceptable as keys are values containing lists or
391 dictionaries or other mutable types that are compared by value rather than by
392 object identity, the reason being that the efficient implementation of
393 dictionaries requires a key's hash value to remain constant. Numeric types used
394 for keys obey the normal rules for numeric comparison: if two numbers compare
395 equal (e.g., ``1`` and ``1.0``) then they can be used interchangeably to index
396 the same dictionary entry.
397
398 Dictionaries are mutable; they can be created by the ``{...}`` notation (see
399 section :ref:`dict`).
400
401 .. index::
402 module: dbm
403 module: gdbm
404 module: bsddb
405
406 The extension modules :mod:`dbm`, :mod:`gdbm`, and :mod:`bsddb` provide
407 additional examples of mapping types.
408
Georg Brandl116aa622007-08-15 14:28:22 +0000409Callable types
410 .. index::
411 object: callable
412 pair: function; call
413 single: invocation
414 pair: function; argument
415
416 These are the types to which the function call operation (see section
417 :ref:`calls`) can be applied:
418
419 User-defined functions
420 .. index::
421 pair: user-defined; function
422 object: function
423 object: user-defined function
424
425 A user-defined function object is created by a function definition (see
426 section :ref:`function`). It should be called with an argument list
427 containing the same number of items as the function's formal parameter
428 list.
429
430 Special attributes:
431
432 +-------------------------+-------------------------------+-----------+
433 | Attribute | Meaning | |
434 +=========================+===============================+===========+
435 | :attr:`__doc__` | The function's documentation | Writable |
436 | | string, or ``None`` if | |
437 | | unavailable | |
438 +-------------------------+-------------------------------+-----------+
439 | :attr:`__name__` | The function's name | Writable |
440 +-------------------------+-------------------------------+-----------+
441 | :attr:`__module__` | The name of the module the | Writable |
442 | | function was defined in, or | |
443 | | ``None`` if unavailable. | |
444 +-------------------------+-------------------------------+-----------+
445 | :attr:`__defaults__` | A tuple containing default | Writable |
446 | | argument values for those | |
447 | | arguments that have defaults, | |
448 | | or ``None`` if no arguments | |
449 | | have a default value | |
450 +-------------------------+-------------------------------+-----------+
451 | :attr:`__code__` | The code object representing | Writable |
452 | | the compiled function body. | |
453 +-------------------------+-------------------------------+-----------+
454 | :attr:`__globals__` | A reference to the dictionary | Read-only |
455 | | that holds the function's | |
456 | | global variables --- the | |
457 | | global namespace of the | |
458 | | module in which the function | |
459 | | was defined. | |
460 +-------------------------+-------------------------------+-----------+
461 | :attr:`__dict__` | The namespace supporting | Writable |
462 | | arbitrary function | |
463 | | attributes. | |
464 +-------------------------+-------------------------------+-----------+
465 | :attr:`__closure__` | ``None`` or a tuple of cells | Read-only |
466 | | that contain bindings for the | |
467 | | function's free variables. | |
468 +-------------------------+-------------------------------+-----------+
469 | :attr:`__annotations__` | A dict containing annotations | Writable |
470 | | of parameters. The keys of | |
471 | | the dict are the parameter | |
472 | | names, or ``'return'`` for | |
473 | | the return annotation, if | |
474 | | provided. | |
475 +-------------------------+-------------------------------+-----------+
476 | :attr:`__kwdefaults__` | A dict containing defaults | Writable |
477 | | for keyword-only parameters. | |
478 +-------------------------+-------------------------------+-----------+
479
480 Most of the attributes labelled "Writable" check the type of the assigned value.
481
Georg Brandl116aa622007-08-15 14:28:22 +0000482 Function objects also support getting and setting arbitrary attributes, which
483 can be used, for example, to attach metadata to functions. Regular attribute
484 dot-notation is used to get and set such attributes. *Note that the current
485 implementation only supports function attributes on user-defined functions.
486 Function attributes on built-in functions may be supported in the future.*
487
488 Additional information about a function's definition can be retrieved from its
489 code object; see the description of internal types below.
490
491 .. index::
492 single: __doc__ (function attribute)
493 single: __name__ (function attribute)
494 single: __module__ (function attribute)
495 single: __dict__ (function attribute)
496 single: __defaults__ (function attribute)
497 single: __closure__ (function attribute)
498 single: __code__ (function attribute)
499 single: __globals__ (function attribute)
500 single: __annotations__ (function attribute)
501 single: __kwdefaults__ (function attribute)
502 pair: global; namespace
503
Georg Brandl2e0b7552007-11-27 12:43:08 +0000504 Instance methods
Georg Brandl116aa622007-08-15 14:28:22 +0000505 .. index::
506 object: method
507 object: user-defined method
508 pair: user-defined; method
509
Georg Brandl2e0b7552007-11-27 12:43:08 +0000510 An instance method object combines a class, a class instance and any
511 callable object (normally a user-defined function).
512
513 .. index::
514 single: __func__ (method attribute)
515 single: __self__ (method attribute)
516 single: __doc__ (method attribute)
517 single: __name__ (method attribute)
518 single: __module__ (method attribute)
Georg Brandl116aa622007-08-15 14:28:22 +0000519
Christian Heimesff737952007-11-27 10:40:20 +0000520 Special read-only attributes: :attr:`__self__` is the class instance object,
521 :attr:`__func__` is the function object; :attr:`__doc__` is the method's
522 documentation (same as ``__func__.__doc__``); :attr:`__name__` is the
523 method name (same as ``__func__.__name__``); :attr:`__module__` is the
524 name of the module the method was defined in, or ``None`` if unavailable.
Georg Brandl116aa622007-08-15 14:28:22 +0000525
Georg Brandl116aa622007-08-15 14:28:22 +0000526 Methods also support accessing (but not setting) the arbitrary function
527 attributes on the underlying function object.
528
Georg Brandl2e0b7552007-11-27 12:43:08 +0000529 User-defined method objects may be created when getting an attribute of a
530 class (perhaps via an instance of that class), if that attribute is a
531 user-defined function object or a class method object.
532
533 When an instance method object is created by retrieving a user-defined
534 function object from a class via one of its instances, its
535 :attr:`__self__` attribute is the instance, and the method object is said
536 to be bound. The new method's :attr:`__func__` attribute is the original
537 function object.
Georg Brandl116aa622007-08-15 14:28:22 +0000538
Georg Brandl2e0b7552007-11-27 12:43:08 +0000539 When a user-defined method object is created by retrieving another method
540 object from a class or instance, the behaviour is the same as for a
541 function object, except that the :attr:`__func__` attribute of the new
542 instance is not the original method object but its :attr:`__func__`
543 attribute.
Georg Brandl116aa622007-08-15 14:28:22 +0000544
Georg Brandl2e0b7552007-11-27 12:43:08 +0000545 When an instance method object is created by retrieving a class method
546 object from a class or instance, its :attr:`__self__` attribute is the
547 class itself, and its :attr:`__func__` attribute is the function object
548 underlying the class method.
Georg Brandl116aa622007-08-15 14:28:22 +0000549
Georg Brandl2e0b7552007-11-27 12:43:08 +0000550 When an instance method object is called, the underlying function
551 (:attr:`__func__`) is called, inserting the class instance
552 (:attr:`__self__`) in front of the argument list. For instance, when
553 :class:`C` is a class which contains a definition for a function
554 :meth:`f`, and ``x`` is an instance of :class:`C`, calling ``x.f(1)`` is
555 equivalent to calling ``C.f(x, 1)``.
Georg Brandl116aa622007-08-15 14:28:22 +0000556
Georg Brandl2e0b7552007-11-27 12:43:08 +0000557 When an instance method object is derived from a class method object, the
558 "class instance" stored in :attr:`__self__` will actually be the class
559 itself, so that calling either ``x.f(1)`` or ``C.f(1)`` is equivalent to
560 calling ``f(C,1)`` where ``f`` is the underlying function.
Georg Brandl116aa622007-08-15 14:28:22 +0000561
Georg Brandl2e0b7552007-11-27 12:43:08 +0000562 Note that the transformation from function object to instance method
563 object happens each time the attribute is retrieved from the instance. In
564 some cases, a fruitful optimization is to assign the attribute to a local
565 variable and call that local variable. Also notice that this
566 transformation only happens for user-defined functions; other callable
567 objects (and all non-callable objects) are retrieved without
568 transformation. It is also important to note that user-defined functions
569 which are attributes of a class instance are not converted to bound
570 methods; this *only* happens when the function is an attribute of the
571 class.
Georg Brandl116aa622007-08-15 14:28:22 +0000572
573 Generator functions
574 .. index::
575 single: generator; function
576 single: generator; iterator
577
578 A function or method which uses the :keyword:`yield` statement (see section
579 :ref:`yield`) is called a :dfn:`generator
580 function`. Such a function, when called, always returns an iterator object
581 which can be used to execute the body of the function: calling the iterator's
582 :meth:`__next__` method will cause the function to execute until it provides a
583 value using the :keyword:`yield` statement. When the function executes a
584 :keyword:`return` statement or falls off the end, a :exc:`StopIteration`
585 exception is raised and the iterator will have reached the end of the set of
586 values to be returned.
587
588 Built-in functions
589 .. index::
590 object: built-in function
591 object: function
592 pair: C; language
593
594 A built-in function object is a wrapper around a C function. Examples of
595 built-in functions are :func:`len` and :func:`math.sin` (:mod:`math` is a
596 standard built-in module). The number and type of the arguments are
597 determined by the C function. Special read-only attributes:
598 :attr:`__doc__` is the function's documentation string, or ``None`` if
599 unavailable; :attr:`__name__` is the function's name; :attr:`__self__` is
600 set to ``None`` (but see the next item); :attr:`__module__` is the name of
601 the module the function was defined in or ``None`` if unavailable.
602
603 Built-in methods
604 .. index::
605 object: built-in method
606 object: method
607 pair: built-in; method
608
609 This is really a different disguise of a built-in function, this time containing
610 an object passed to the C function as an implicit extra argument. An example of
611 a built-in method is ``alist.append()``, assuming *alist* is a list object. In
612 this case, the special read-only attribute :attr:`__self__` is set to the object
613 denoted by *list*.
614
Georg Brandl85eb8c12007-08-31 16:33:38 +0000615 Classes
616 Classes are callable. These objects normally act as factories for new
617 instances of themselves, but variations are possible for class types that
618 override :meth:`__new__`. The arguments of the call are passed to
619 :meth:`__new__` and, in the typical case, to :meth:`__init__` to
620 initialize the new instance.
Georg Brandl116aa622007-08-15 14:28:22 +0000621
Georg Brandl85eb8c12007-08-31 16:33:38 +0000622 Class Instances
623 Instances of arbitrary classes can be made callable by defining a
624 :meth:`__call__` method in their class.
Georg Brandl116aa622007-08-15 14:28:22 +0000625
Georg Brandl116aa622007-08-15 14:28:22 +0000626
627Modules
628 .. index::
629 statement: import
630 object: module
631
632 Modules are imported by the :keyword:`import` statement (see section
633 :ref:`import`). A module object has a
634 namespace implemented by a dictionary object (this is the dictionary referenced
635 by the __globals__ attribute of functions defined in the module). Attribute
636 references are translated to lookups in this dictionary, e.g., ``m.x`` is
637 equivalent to ``m.__dict__["x"]``. A module object does not contain the code
638 object used to initialize the module (since it isn't needed once the
639 initialization is done).
640
Georg Brandl116aa622007-08-15 14:28:22 +0000641 Attribute assignment updates the module's namespace dictionary, e.g., ``m.x =
642 1`` is equivalent to ``m.__dict__["x"] = 1``.
643
644 .. index:: single: __dict__ (module attribute)
645
646 Special read-only attribute: :attr:`__dict__` is the module's namespace as a
647 dictionary object.
648
649 .. index::
650 single: __name__ (module attribute)
651 single: __doc__ (module attribute)
652 single: __file__ (module attribute)
653 pair: module; namespace
654
655 Predefined (writable) attributes: :attr:`__name__` is the module's name;
656 :attr:`__doc__` is the module's documentation string, or ``None`` if
657 unavailable; :attr:`__file__` is the pathname of the file from which the module
658 was loaded, if it was loaded from a file. The :attr:`__file__` attribute is not
659 present for C modules that are statically linked into the interpreter; for
660 extension modules loaded dynamically from a shared library, it is the pathname
661 of the shared library file.
662
Georg Brandl85eb8c12007-08-31 16:33:38 +0000663.. XXX "Classes" and "Instances" is outdated!
664 see http://www.python.org/doc/newstyle.html for newstyle information
665
666Custom classes
Georg Brandl116aa622007-08-15 14:28:22 +0000667 Class objects are created by class definitions (see section :ref:`class`). A
668 class has a namespace implemented by a dictionary object. Class attribute
669 references are translated to lookups in this dictionary, e.g., ``C.x`` is
670 translated to ``C.__dict__["x"]``. When the attribute name is not found
671 there, the attribute search continues in the base classes. The search is
672 depth-first, left-to-right in the order of occurrence in the base class list.
673
Georg Brandl85eb8c12007-08-31 16:33:38 +0000674 .. XXX document descriptors and new MRO
675
Georg Brandl116aa622007-08-15 14:28:22 +0000676 .. index::
677 object: class
678 object: class instance
679 object: instance
680 pair: class object; call
681 single: container
682 object: dictionary
683 pair: class; attribute
684
685 When a class attribute reference (for class :class:`C`, say) would yield a
Georg Brandl2e0b7552007-11-27 12:43:08 +0000686 class method object, it is transformed into an instance method object whose
687 :attr:`__self__` attributes is :class:`C`. When it would yield a static
688 method object, it is transformed into the object wrapped by the static method
689 object. See section :ref:`descriptors` for another way in which attributes
690 retrieved from a class may differ from those actually contained in its
691 :attr:`__dict__`.
Georg Brandl116aa622007-08-15 14:28:22 +0000692
693 .. index:: triple: class; attribute; assignment
694
695 Class attribute assignments update the class's dictionary, never the dictionary
696 of a base class.
697
698 .. index:: pair: class object; call
699
700 A class object can be called (see above) to yield a class instance (see below).
701
702 .. index::
703 single: __name__ (class attribute)
704 single: __module__ (class attribute)
705 single: __dict__ (class attribute)
706 single: __bases__ (class attribute)
707 single: __doc__ (class attribute)
708
709 Special attributes: :attr:`__name__` is the class name; :attr:`__module__` is
710 the module name in which the class was defined; :attr:`__dict__` is the
711 dictionary containing the class's namespace; :attr:`__bases__` is a tuple
712 (possibly empty or a singleton) containing the base classes, in the order of
713 their occurrence in the base class list; :attr:`__doc__` is the class's
714 documentation string, or None if undefined.
715
716Class instances
717 .. index::
718 object: class instance
719 object: instance
720 pair: class; instance
721 pair: class instance; attribute
722
Georg Brandl2e0b7552007-11-27 12:43:08 +0000723 A class instance is created by calling a class object (see above). A class
724 instance has a namespace implemented as a dictionary which is the first place
725 in which attribute references are searched. When an attribute is not found
726 there, and the instance's class has an attribute by that name, the search
727 continues with the class attributes. If a class attribute is found that is a
728 user-defined function object, it is transformed into an instance method
729 object whose :attr:`__self__` attribute is the instance. Static method and
730 class method objects are also transformed; see above under "Classes". See
731 section :ref:`descriptors` for another way in which attributes of a class
732 retrieved via its instances may differ from the objects actually stored in
733 the class's :attr:`__dict__`. If no class attribute is found, and the
734 object's class has a :meth:`__getattr__` method, that is called to satisfy
735 the lookup.
Georg Brandl116aa622007-08-15 14:28:22 +0000736
737 .. index:: triple: class instance; attribute; assignment
738
739 Attribute assignments and deletions update the instance's dictionary, never a
740 class's dictionary. If the class has a :meth:`__setattr__` or
741 :meth:`__delattr__` method, this is called instead of updating the instance
742 dictionary directly.
743
744 .. index::
745 object: numeric
746 object: sequence
747 object: mapping
748
749 Class instances can pretend to be numbers, sequences, or mappings if they have
750 methods with certain special names. See section :ref:`specialnames`.
751
752 .. index::
753 single: __dict__ (instance attribute)
754 single: __class__ (instance attribute)
755
756 Special attributes: :attr:`__dict__` is the attribute dictionary;
757 :attr:`__class__` is the instance's class.
758
759Files
760 .. index::
761 object: file
762 builtin: open
763 single: popen() (in module os)
764 single: makefile() (socket method)
765 single: sys.stdin
766 single: sys.stdout
767 single: sys.stderr
768 single: stdio
769 single: stdin (in module sys)
770 single: stdout (in module sys)
771 single: stderr (in module sys)
772
773 A file object represents an open file. File objects are created by the
774 :func:`open` built-in function, and also by :func:`os.popen`,
775 :func:`os.fdopen`, and the :meth:`makefile` method of socket objects (and
776 perhaps by other functions or methods provided by extension modules). The
777 objects ``sys.stdin``, ``sys.stdout`` and ``sys.stderr`` are initialized to
778 file objects corresponding to the interpreter's standard input, output and
779 error streams. See :ref:`bltin-file-objects` for complete documentation of
780 file objects.
781
782Internal types
783 .. index::
784 single: internal type
785 single: types, internal
786
787 A few types used internally by the interpreter are exposed to the user. Their
788 definitions may change with future versions of the interpreter, but they are
789 mentioned here for completeness.
790
791 Code objects
792 .. index::
793 single: bytecode
794 object: code
795
Georg Brandl9afde1c2007-11-01 20:32:30 +0000796 Code objects represent *byte-compiled* executable Python code, or :term:`bytecode`.
Georg Brandl116aa622007-08-15 14:28:22 +0000797 The difference between a code object and a function object is that the function
798 object contains an explicit reference to the function's globals (the module in
799 which it was defined), while a code object contains no context; also the default
800 argument values are stored in the function object, not in the code object
801 (because they represent values calculated at run-time). Unlike function
802 objects, code objects are immutable and contain no references (directly or
803 indirectly) to mutable objects.
804
805 Special read-only attributes: :attr:`co_name` gives the function name;
806 :attr:`co_argcount` is the number of positional arguments (including arguments
807 with default values); :attr:`co_nlocals` is the number of local variables used
808 by the function (including arguments); :attr:`co_varnames` is a tuple containing
809 the names of the local variables (starting with the argument names);
810 :attr:`co_cellvars` is a tuple containing the names of local variables that are
811 referenced by nested functions; :attr:`co_freevars` is a tuple containing the
812 names of free variables; :attr:`co_code` is a string representing the sequence
813 of bytecode instructions; :attr:`co_consts` is a tuple containing the literals
814 used by the bytecode; :attr:`co_names` is a tuple containing the names used by
815 the bytecode; :attr:`co_filename` is the filename from which the code was
816 compiled; :attr:`co_firstlineno` is the first line number of the function;
Georg Brandl9afde1c2007-11-01 20:32:30 +0000817 :attr:`co_lnotab` is a string encoding the mapping from bytecode offsets to
Georg Brandl116aa622007-08-15 14:28:22 +0000818 line numbers (for details see the source code of the interpreter);
819 :attr:`co_stacksize` is the required stack size (including local variables);
820 :attr:`co_flags` is an integer encoding a number of flags for the interpreter.
821
822 .. index::
823 single: co_argcount (code object attribute)
824 single: co_code (code object attribute)
825 single: co_consts (code object attribute)
826 single: co_filename (code object attribute)
827 single: co_firstlineno (code object attribute)
828 single: co_flags (code object attribute)
829 single: co_lnotab (code object attribute)
830 single: co_name (code object attribute)
831 single: co_names (code object attribute)
832 single: co_nlocals (code object attribute)
833 single: co_stacksize (code object attribute)
834 single: co_varnames (code object attribute)
835 single: co_cellvars (code object attribute)
836 single: co_freevars (code object attribute)
837
838 .. index:: object: generator
839
840 The following flag bits are defined for :attr:`co_flags`: bit ``0x04`` is set if
841 the function uses the ``*arguments`` syntax to accept an arbitrary number of
842 positional arguments; bit ``0x08`` is set if the function uses the
843 ``**keywords`` syntax to accept arbitrary keyword arguments; bit ``0x20`` is set
844 if the function is a generator.
845
846 Future feature declarations (``from __future__ import division``) also use bits
847 in :attr:`co_flags` to indicate whether a code object was compiled with a
848 particular feature enabled: bit ``0x2000`` is set if the function was compiled
849 with future division enabled; bits ``0x10`` and ``0x1000`` were used in earlier
850 versions of Python.
851
852 Other bits in :attr:`co_flags` are reserved for internal use.
853
854 .. index:: single: documentation string
855
856 If a code object represents a function, the first item in :attr:`co_consts` is
857 the documentation string of the function, or ``None`` if undefined.
858
859 Frame objects
860 .. index:: object: frame
861
862 Frame objects represent execution frames. They may occur in traceback objects
863 (see below).
864
865 .. index::
866 single: f_back (frame attribute)
867 single: f_code (frame attribute)
868 single: f_globals (frame attribute)
869 single: f_locals (frame attribute)
870 single: f_lasti (frame attribute)
871 single: f_builtins (frame attribute)
872
873 Special read-only attributes: :attr:`f_back` is to the previous stack frame
874 (towards the caller), or ``None`` if this is the bottom stack frame;
875 :attr:`f_code` is the code object being executed in this frame; :attr:`f_locals`
876 is the dictionary used to look up local variables; :attr:`f_globals` is used for
877 global variables; :attr:`f_builtins` is used for built-in (intrinsic) names;
878 :attr:`f_lasti` gives the precise instruction (this is an index into the
879 bytecode string of the code object).
880
881 .. index::
882 single: f_trace (frame attribute)
883 single: f_exc_type (frame attribute)
884 single: f_exc_value (frame attribute)
885 single: f_exc_traceback (frame attribute)
886 single: f_lineno (frame attribute)
887
888 Special writable attributes: :attr:`f_trace`, if not ``None``, is a function
889 called at the start of each source code line (this is used by the debugger);
890 :attr:`f_exc_type`, :attr:`f_exc_value`, :attr:`f_exc_traceback` represent the
891 last exception raised in the parent frame provided another exception was ever
892 raised in the current frame (in all other cases they are None); :attr:`f_lineno`
893 is the current line number of the frame --- writing to this from within a trace
894 function jumps to the given line (only for the bottom-most frame). A debugger
895 can implement a Jump command (aka Set Next Statement) by writing to f_lineno.
896
897 Traceback objects
898 .. index::
899 object: traceback
900 pair: stack; trace
901 pair: exception; handler
902 pair: execution; stack
903 single: exc_info (in module sys)
Georg Brandl116aa622007-08-15 14:28:22 +0000904 single: last_traceback (in module sys)
905 single: sys.exc_info
906 single: sys.last_traceback
907
908 Traceback objects represent a stack trace of an exception. A traceback object
909 is created when an exception occurs. When the search for an exception handler
910 unwinds the execution stack, at each unwound level a traceback object is
911 inserted in front of the current traceback. When an exception handler is
912 entered, the stack trace is made available to the program. (See section
913 :ref:`try`.) It is accessible as the third item of the
914 tuple returned by ``sys.exc_info()``. When the program contains no suitable
915 handler, the stack trace is written (nicely formatted) to the standard error
916 stream; if the interpreter is interactive, it is also made available to the user
917 as ``sys.last_traceback``.
918
919 .. index::
920 single: tb_next (traceback attribute)
921 single: tb_frame (traceback attribute)
922 single: tb_lineno (traceback attribute)
923 single: tb_lasti (traceback attribute)
924 statement: try
925
926 Special read-only attributes: :attr:`tb_next` is the next level in the stack
927 trace (towards the frame where the exception occurred), or ``None`` if there is
928 no next level; :attr:`tb_frame` points to the execution frame of the current
929 level; :attr:`tb_lineno` gives the line number where the exception occurred;
930 :attr:`tb_lasti` indicates the precise instruction. The line number and last
931 instruction in the traceback may differ from the line number of its frame object
932 if the exception occurred in a :keyword:`try` statement with no matching except
933 clause or with a finally clause.
934
935 Slice objects
936 .. index:: builtin: slice
937
Georg Brandlcb8ecb12007-09-04 06:35:14 +0000938 Slice objects are used to represent slices for :meth:`__getitem__`
939 methods. They are also created by the built-in :func:`slice` function.
Georg Brandl116aa622007-08-15 14:28:22 +0000940
941 .. index::
942 single: start (slice object attribute)
943 single: stop (slice object attribute)
944 single: step (slice object attribute)
945
946 Special read-only attributes: :attr:`start` is the lower bound; :attr:`stop` is
947 the upper bound; :attr:`step` is the step value; each is ``None`` if omitted.
948 These attributes can have any type.
949
950 Slice objects support one method:
951
Georg Brandl116aa622007-08-15 14:28:22 +0000952 .. method:: slice.indices(self, length)
953
Georg Brandlcb8ecb12007-09-04 06:35:14 +0000954 This method takes a single integer argument *length* and computes
955 information about the slice that the slice object would describe if
956 applied to a sequence of *length* items. It returns a tuple of three
957 integers; respectively these are the *start* and *stop* indices and the
958 *step* or stride length of the slice. Missing or out-of-bounds indices
959 are handled in a manner consistent with regular slices.
Georg Brandl116aa622007-08-15 14:28:22 +0000960
Georg Brandl116aa622007-08-15 14:28:22 +0000961 Static method objects
962 Static method objects provide a way of defeating the transformation of function
963 objects to method objects described above. A static method object is a wrapper
964 around any other object, usually a user-defined method object. When a static
965 method object is retrieved from a class or a class instance, the object actually
966 returned is the wrapped object, which is not subject to any further
967 transformation. Static method objects are not themselves callable, although the
968 objects they wrap usually are. Static method objects are created by the built-in
969 :func:`staticmethod` constructor.
970
971 Class method objects
972 A class method object, like a static method object, is a wrapper around another
973 object that alters the way in which that object is retrieved from classes and
974 class instances. The behaviour of class method objects upon such retrieval is
975 described above, under "User-defined methods". Class method objects are created
976 by the built-in :func:`classmethod` constructor.
977
Georg Brandl116aa622007-08-15 14:28:22 +0000978
Georg Brandl9afde1c2007-11-01 20:32:30 +0000979.. _newstyle:
Georg Brandl116aa622007-08-15 14:28:22 +0000980
Christian Heimes5b5e81c2007-12-31 16:14:33 +0000981New-style and classic classes
982=============================
983
984Classes and instances come in two flavors: old-style or classic, and new-style.
985
986Up to Python 2.1, old-style classes were the only flavour available to the user.
987The concept of (old-style) class is unrelated to the concept of type: if *x* is
988an instance of an old-style class, then ``x.__class__`` designates the class of
989*x*, but ``type(x)`` is always ``<type 'instance'>``. This reflects the fact
990that all old-style instances, independently of their class, are implemented with
991a single built-in type, called ``instance``.
992
993New-style classes were introduced in Python 2.2 to unify classes and types. A
Christian Heimes292d3512008-02-03 16:51:08 +0000994new-style class is neither more nor less than a user-defined type. If *x* is an
Christian Heimes5b5e81c2007-12-31 16:14:33 +0000995instance of a new-style class, then ``type(x)`` is the same as ``x.__class__``.
996
997The major motivation for introducing new-style classes is to provide a unified
998object model with a full meta-model. It also has a number of immediate
999benefits, like the ability to subclass most built-in types, or the introduction
1000of "descriptors", which enable computed properties.
1001
1002For compatibility reasons, classes are still old-style by default. New-style
1003classes are created by specifying another new-style class (i.e. a type) as a
1004parent class, or the "top-level type" :class:`object` if no other parent is
1005needed. The behaviour of new-style classes differs from that of old-style
1006classes in a number of important details in addition to what :func:`type`
1007returns. Some of these changes are fundamental to the new object model, like
1008the way special methods are invoked. Others are "fixes" that could not be
1009implemented before for compatibility concerns, like the method resolution order
1010in case of multiple inheritance.
1011
1012This manual is not up-to-date with respect to new-style classes. For now,
Christian Heimesfaf2f632008-01-06 16:59:19 +00001013please see http://www.python.org/doc/newstyle/ for more information.
Christian Heimes5b5e81c2007-12-31 16:14:33 +00001014
Georg Brandl8de8e032008-01-07 09:29:34 +00001015.. XXX remove old style classes from docs
Christian Heimes5b5e81c2007-12-31 16:14:33 +00001016
1017
Georg Brandl116aa622007-08-15 14:28:22 +00001018.. _specialnames:
1019
1020Special method names
1021====================
1022
1023.. index::
1024 pair: operator; overloading
1025 single: __getitem__() (mapping object method)
1026
1027A class can implement certain operations that are invoked by special syntax
1028(such as arithmetic operations or subscripting and slicing) by defining methods
1029with special names. This is Python's approach to :dfn:`operator overloading`,
1030allowing classes to define their own behavior with respect to language
1031operators. For instance, if a class defines a method named :meth:`__getitem__`,
Georg Brandl85eb8c12007-08-31 16:33:38 +00001032and ``x`` is an instance of this class, then ``x[i]`` is equivalent to
Georg Brandl116aa622007-08-15 14:28:22 +00001033``x.__getitem__(i)``. Except where mentioned, attempts to execute an operation
1034raise an exception when no appropriate method is defined.
1035
Georg Brandl85eb8c12007-08-31 16:33:38 +00001036.. XXX above translation is not correct for new-style classes!
1037
Georg Brandl65ea9bd2007-09-05 13:36:27 +00001038Special methods are only guaranteed to work if defined in an object's class, not
1039in the object's instance dictionary. That explains why this won't work::
1040
1041 >>> class C:
1042 ... pass
1043 ...
1044 >>> c = C()
1045 >>> c.__len__ = lambda: 5
1046 >>> len(c)
1047 Traceback (most recent call last):
1048 File "<stdin>", line 1, in <module>
1049 TypeError: object of type 'C' has no len()
1050
1051
Georg Brandl116aa622007-08-15 14:28:22 +00001052When implementing a class that emulates any built-in type, it is important that
1053the emulation only be implemented to the degree that it makes sense for the
1054object being modelled. For example, some sequences may work well with retrieval
1055of individual elements, but extracting a slice may not make sense. (One example
1056of this is the :class:`NodeList` interface in the W3C's Document Object Model.)
1057
1058
1059.. _customization:
1060
1061Basic customization
1062-------------------
1063
1064
1065.. method:: object.__new__(cls[, ...])
1066
1067 Called to create a new instance of class *cls*. :meth:`__new__` is a static
1068 method (special-cased so you need not declare it as such) that takes the class
1069 of which an instance was requested as its first argument. The remaining
1070 arguments are those passed to the object constructor expression (the call to the
1071 class). The return value of :meth:`__new__` should be the new object instance
1072 (usually an instance of *cls*).
1073
1074 Typical implementations create a new instance of the class by invoking the
1075 superclass's :meth:`__new__` method using ``super(currentclass,
1076 cls).__new__(cls[, ...])`` with appropriate arguments and then modifying the
1077 newly-created instance as necessary before returning it.
1078
1079 If :meth:`__new__` returns an instance of *cls*, then the new instance's
1080 :meth:`__init__` method will be invoked like ``__init__(self[, ...])``, where
1081 *self* is the new instance and the remaining arguments are the same as were
1082 passed to :meth:`__new__`.
1083
1084 If :meth:`__new__` does not return an instance of *cls*, then the new instance's
1085 :meth:`__init__` method will not be invoked.
1086
1087 :meth:`__new__` is intended mainly to allow subclasses of immutable types (like
Christian Heimes790c8232008-01-07 21:14:23 +00001088 int, str, or tuple) to customize instance creation. It is also commonly
1089 overridden in custom metaclasses in order to customize class creation.
Georg Brandl116aa622007-08-15 14:28:22 +00001090
1091
1092.. method:: object.__init__(self[, ...])
1093
1094 .. index:: pair: class; constructor
1095
1096 Called when the instance is created. The arguments are those passed to the
1097 class constructor expression. If a base class has an :meth:`__init__` method,
1098 the derived class's :meth:`__init__` method, if any, must explicitly call it to
1099 ensure proper initialization of the base class part of the instance; for
1100 example: ``BaseClass.__init__(self, [args...])``. As a special constraint on
1101 constructors, no value may be returned; doing so will cause a :exc:`TypeError`
1102 to be raised at runtime.
1103
1104
1105.. method:: object.__del__(self)
1106
1107 .. index::
1108 single: destructor
1109 statement: del
1110
1111 Called when the instance is about to be destroyed. This is also called a
1112 destructor. If a base class has a :meth:`__del__` method, the derived class's
1113 :meth:`__del__` method, if any, must explicitly call it to ensure proper
1114 deletion of the base class part of the instance. Note that it is possible
1115 (though not recommended!) for the :meth:`__del__` method to postpone destruction
1116 of the instance by creating a new reference to it. It may then be called at a
1117 later time when this new reference is deleted. It is not guaranteed that
1118 :meth:`__del__` methods are called for objects that still exist when the
1119 interpreter exits.
1120
1121 .. note::
1122
1123 ``del x`` doesn't directly call ``x.__del__()`` --- the former decrements
1124 the reference count for ``x`` by one, and the latter is only called when
1125 ``x``'s reference count reaches zero. Some common situations that may
1126 prevent the reference count of an object from going to zero include:
1127 circular references between objects (e.g., a doubly-linked list or a tree
1128 data structure with parent and child pointers); a reference to the object
1129 on the stack frame of a function that caught an exception (the traceback
1130 stored in ``sys.exc_info()[2]`` keeps the stack frame alive); or a
1131 reference to the object on the stack frame that raised an unhandled
1132 exception in interactive mode (the traceback stored in
1133 ``sys.last_traceback`` keeps the stack frame alive). The first situation
1134 can only be remedied by explicitly breaking the cycles; the latter two
1135 situations can be resolved by storing ``None`` in ``sys.last_traceback``.
1136 Circular references which are garbage are detected when the option cycle
1137 detector is enabled (it's on by default), but can only be cleaned up if
1138 there are no Python- level :meth:`__del__` methods involved. Refer to the
1139 documentation for the :mod:`gc` module for more information about how
1140 :meth:`__del__` methods are handled by the cycle detector, particularly
1141 the description of the ``garbage`` value.
1142
1143 .. warning::
1144
1145 Due to the precarious circumstances under which :meth:`__del__` methods are
1146 invoked, exceptions that occur during their execution are ignored, and a warning
1147 is printed to ``sys.stderr`` instead. Also, when :meth:`__del__` is invoked in
1148 response to a module being deleted (e.g., when execution of the program is
1149 done), other globals referenced by the :meth:`__del__` method may already have
1150 been deleted. For this reason, :meth:`__del__` methods should do the absolute
1151 minimum needed to maintain external invariants. Starting with version 1.5,
1152 Python guarantees that globals whose name begins with a single underscore are
1153 deleted from their module before other globals are deleted; if no other
1154 references to such globals exist, this may help in assuring that imported
1155 modules are still available at the time when the :meth:`__del__` method is
1156 called.
1157
1158
1159.. method:: object.__repr__(self)
1160
1161 .. index:: builtin: repr
1162
1163 Called by the :func:`repr` built-in function and by string conversions (reverse
1164 quotes) to compute the "official" string representation of an object. If at all
1165 possible, this should look like a valid Python expression that could be used to
1166 recreate an object with the same value (given an appropriate environment). If
1167 this is not possible, a string of the form ``<...some useful description...>``
1168 should be returned. The return value must be a string object. If a class
1169 defines :meth:`__repr__` but not :meth:`__str__`, then :meth:`__repr__` is also
1170 used when an "informal" string representation of instances of that class is
1171 required.
1172
Georg Brandl116aa622007-08-15 14:28:22 +00001173 This is typically used for debugging, so it is important that the representation
1174 is information-rich and unambiguous.
1175
1176
1177.. method:: object.__str__(self)
1178
1179 .. index::
1180 builtin: str
Georg Brandl4b491312007-08-31 09:22:56 +00001181 builtin: print
Georg Brandl116aa622007-08-15 14:28:22 +00001182
Georg Brandldcc56f82007-08-31 16:41:12 +00001183 Called by the :func:`str` built-in function and by the :func:`print` function
1184 to compute the "informal" string representation of an object. This differs
1185 from :meth:`__repr__` in that it does not have to be a valid Python
Georg Brandl116aa622007-08-15 14:28:22 +00001186 expression: a more convenient or concise representation may be used instead.
1187 The return value must be a string object.
1188
Georg Brandldcc56f82007-08-31 16:41:12 +00001189 .. XXX what about subclasses of string?
1190
Georg Brandl116aa622007-08-15 14:28:22 +00001191
Georg Brandl4b491312007-08-31 09:22:56 +00001192.. method:: object.__format__(self, format_spec)
1193
1194 .. index::
1195 pair: string; conversion
1196 builtin: str
1197 builtin: print
1198
1199 Called by the :func:`format` built-in function (and by extension, the
1200 :meth:`format` method of class :class:`str`) to produce a "formatted"
1201 string representation of an object. The ``format_spec`` argument is
1202 a string that contains a description of the formatting options desired.
1203 The interpretation of the ``format_spec`` argument is up to the type
1204 implementing :meth:`__format__`, however most classes will either
1205 delegate formatting to one of the built-in types, or use a similar
1206 formatting option syntax.
1207
1208 See :ref:`formatspec` for a description of the standard formatting syntax.
1209
1210 The return value must be a string object.
1211
1212
Georg Brandl116aa622007-08-15 14:28:22 +00001213.. method:: object.__lt__(self, other)
1214 object.__le__(self, other)
1215 object.__eq__(self, other)
1216 object.__ne__(self, other)
1217 object.__gt__(self, other)
1218 object.__ge__(self, other)
1219
Guido van Rossum2cc30da2007-11-02 23:46:40 +00001220 .. index::
1221 single: comparisons
1222
Georg Brandl116aa622007-08-15 14:28:22 +00001223 These are the so-called "rich comparison" methods, and are called for comparison
1224 operators in preference to :meth:`__cmp__` below. The correspondence between
1225 operator symbols and method names is as follows: ``x<y`` calls ``x.__lt__(y)``,
1226 ``x<=y`` calls ``x.__le__(y)``, ``x==y`` calls ``x.__eq__(y)``, ``x!=y`` calls
1227 ``x.__ne__(y)``, ``x>y`` calls ``x.__gt__(y)``, and ``x>=y`` calls
1228 ``x.__ge__(y)``.
1229
1230 A rich comparison method may return the singleton ``NotImplemented`` if it does
1231 not implement the operation for a given pair of arguments. By convention,
1232 ``False`` and ``True`` are returned for a successful comparison. However, these
1233 methods can return any value, so if the comparison operator is used in a Boolean
1234 context (e.g., in the condition of an ``if`` statement), Python will call
1235 :func:`bool` on the value to determine if the result is true or false.
1236
Guido van Rossum2cc30da2007-11-02 23:46:40 +00001237 There are no implied relationships among the comparison operators. The truth
1238 of ``x==y`` does not imply that ``x!=y`` is false. Accordingly, when
1239 defining :meth:`__eq__`, one should also define :meth:`__ne__` so that the
1240 operators will behave as expected. See the paragraph on :meth:`__hash__` for
1241 some important notes on creating :term:`hashable` objects which support
1242 custom comparison operations and are usable as dictionary keys.
Georg Brandl116aa622007-08-15 14:28:22 +00001243
Guido van Rossum2cc30da2007-11-02 23:46:40 +00001244 There are no swapped-argument versions of these methods (to be used when the
1245 left argument does not support the operation but the right argument does);
1246 rather, :meth:`__lt__` and :meth:`__gt__` are each other's reflection,
Georg Brandl116aa622007-08-15 14:28:22 +00001247 :meth:`__le__` and :meth:`__ge__` are each other's reflection, and
1248 :meth:`__eq__` and :meth:`__ne__` are their own reflection.
1249
1250 Arguments to rich comparison methods are never coerced.
1251
1252
1253.. method:: object.__cmp__(self, other)
1254
1255 .. index::
1256 builtin: cmp
1257 single: comparisons
1258
Guido van Rossum2cc30da2007-11-02 23:46:40 +00001259 Called by comparison operations if rich comparison (see above) is not
1260 defined. Should return a negative integer if ``self < other``, zero if
1261 ``self == other``, a positive integer if ``self > other``. If no
1262 :meth:`__cmp__`, :meth:`__eq__` or :meth:`__ne__` operation is defined, class
1263 instances are compared by object identity ("address"). See also the
1264 description of :meth:`__hash__` for some important notes on creating
1265 :term:`hashable` objects which support custom comparison operations and are
Georg Brandldb629672007-11-03 08:44:43 +00001266 usable as dictionary keys.
Georg Brandl116aa622007-08-15 14:28:22 +00001267
1268
Georg Brandl116aa622007-08-15 14:28:22 +00001269.. method:: object.__hash__(self)
1270
1271 .. index::
1272 object: dictionary
1273 builtin: hash
Georg Brandl16174572007-09-01 12:38:06 +00001274 single: __cmp__() (object method)
Georg Brandl116aa622007-08-15 14:28:22 +00001275
Guido van Rossum2cc30da2007-11-02 23:46:40 +00001276 Called for the key object for dictionary operations, and by the built-in
1277 function :func:`hash`. Should return an integer usable as a hash value
Georg Brandl116aa622007-08-15 14:28:22 +00001278 for dictionary operations. The only required property is that objects which
1279 compare equal have the same hash value; it is advised to somehow mix together
1280 (e.g., using exclusive or) the hash values for the components of the object that
Guido van Rossum2cc30da2007-11-02 23:46:40 +00001281 also play a part in comparison of objects.
Georg Brandl116aa622007-08-15 14:28:22 +00001282
Georg Brandldb629672007-11-03 08:44:43 +00001283 If a class does not define a :meth:`__cmp__` or :meth:`__eq__` method it
1284 should not define a :meth:`__hash__` operation either; if it defines
1285 :meth:`__cmp__` or :meth:`__eq__` but not :meth:`__hash__`, its instances
1286 will not be usable as dictionary keys. If a class defines mutable objects
1287 and implements a :meth:`__cmp__` or :meth:`__eq__` method, it should not
1288 implement :meth:`__hash__`, since the dictionary implementation requires that
1289 a key's hash value is immutable (if the object's hash value changes, it will
1290 be in the wrong hash bucket).
1291
1292 User-defined classes have :meth:`__cmp__` and :meth:`__hash__` methods
1293 by default; with them, all objects compare unequal and ``x.__hash__()``
1294 returns ``id(x)``.
Georg Brandl116aa622007-08-15 14:28:22 +00001295
1296
1297.. method:: object.__bool__(self)
1298
1299 .. index:: single: __len__() (mapping object method)
1300
1301 Called to implement truth value testing, and the built-in operation ``bool()``;
1302 should return ``False`` or ``True``. When this method is not defined,
1303 :meth:`__len__` is called, if it is defined (see below) and ``True`` is returned
1304 when the length is not zero. If a class defines neither :meth:`__len__` nor
1305 :meth:`__bool__`, all its instances are considered true.
1306
1307
Georg Brandl116aa622007-08-15 14:28:22 +00001308.. _attribute-access:
1309
1310Customizing attribute access
1311----------------------------
1312
1313The following methods can be defined to customize the meaning of attribute
1314access (use of, assignment to, or deletion of ``x.name``) for class instances.
1315
Georg Brandl85eb8c12007-08-31 16:33:38 +00001316.. XXX explain how descriptors interfere here!
1317
Georg Brandl116aa622007-08-15 14:28:22 +00001318
1319.. method:: object.__getattr__(self, name)
1320
1321 Called when an attribute lookup has not found the attribute in the usual places
1322 (i.e. it is not an instance attribute nor is it found in the class tree for
1323 ``self``). ``name`` is the attribute name. This method should return the
1324 (computed) attribute value or raise an :exc:`AttributeError` exception.
1325
Georg Brandl116aa622007-08-15 14:28:22 +00001326 Note that if the attribute is found through the normal mechanism,
1327 :meth:`__getattr__` is not called. (This is an intentional asymmetry between
1328 :meth:`__getattr__` and :meth:`__setattr__`.) This is done both for efficiency
1329 reasons and because otherwise :meth:`__setattr__` would have no way to access
1330 other attributes of the instance. Note that at least for instance variables,
1331 you can fake total control by not inserting any values in the instance attribute
1332 dictionary (but instead inserting them in another object). See the
Georg Brandl85eb8c12007-08-31 16:33:38 +00001333 :meth:`__getattribute__` method below for a way to actually get total control
1334 over attribute access.
Georg Brandl116aa622007-08-15 14:28:22 +00001335
1336
1337.. method:: object.__getattribute__(self, name)
1338
1339 Called unconditionally to implement attribute accesses for instances of the
1340 class. If the class also defines :meth:`__getattr__`, the latter will not be
1341 called unless :meth:`__getattribute__` either calls it explicitly or raises an
1342 :exc:`AttributeError`. This method should return the (computed) attribute value
1343 or raise an :exc:`AttributeError` exception. In order to avoid infinite
1344 recursion in this method, its implementation should always call the base class
1345 method with the same name to access any attributes it needs, for example,
1346 ``object.__getattribute__(self, name)``.
1347
1348
Georg Brandl85eb8c12007-08-31 16:33:38 +00001349.. method:: object.__setattr__(self, name, value)
1350
1351 Called when an attribute assignment is attempted. This is called instead of
1352 the normal mechanism (i.e. store the value in the instance dictionary).
1353 *name* is the attribute name, *value* is the value to be assigned to it.
1354
1355 If :meth:`__setattr__` wants to assign to an instance attribute, it should
1356 call the base class method with the same name, for example,
1357 ``object.__setattr__(self, name, value)``.
1358
1359
1360.. method:: object.__delattr__(self, name)
1361
1362 Like :meth:`__setattr__` but for attribute deletion instead of assignment. This
1363 should only be implemented if ``del obj.name`` is meaningful for the object.
1364
1365
Georg Brandl116aa622007-08-15 14:28:22 +00001366.. _descriptors:
1367
1368Implementing Descriptors
1369^^^^^^^^^^^^^^^^^^^^^^^^
1370
1371The following methods only apply when an instance of the class containing the
1372method (a so-called *descriptor* class) appears in the class dictionary of
Georg Brandl85eb8c12007-08-31 16:33:38 +00001373another class, known as the *owner* class. In the examples below, "the
Georg Brandl116aa622007-08-15 14:28:22 +00001374attribute" refers to the attribute whose name is the key of the property in the
Georg Brandl85eb8c12007-08-31 16:33:38 +00001375owner class' :attr:`__dict__`.
Georg Brandl116aa622007-08-15 14:28:22 +00001376
1377
1378.. method:: object.__get__(self, instance, owner)
1379
1380 Called to get the attribute of the owner class (class attribute access) or of an
1381 instance of that class (instance attribute access). *owner* is always the owner
1382 class, while *instance* is the instance that the attribute was accessed through,
1383 or ``None`` when the attribute is accessed through the *owner*. This method
1384 should return the (computed) attribute value or raise an :exc:`AttributeError`
1385 exception.
1386
1387
1388.. method:: object.__set__(self, instance, value)
1389
1390 Called to set the attribute on an instance *instance* of the owner class to a
1391 new value, *value*.
1392
1393
1394.. method:: object.__delete__(self, instance)
1395
1396 Called to delete the attribute on an instance *instance* of the owner class.
1397
1398
1399.. _descriptor-invocation:
1400
1401Invoking Descriptors
1402^^^^^^^^^^^^^^^^^^^^
1403
1404In general, a descriptor is an object attribute with "binding behavior", one
1405whose attribute access has been overridden by methods in the descriptor
1406protocol: :meth:`__get__`, :meth:`__set__`, and :meth:`__delete__`. If any of
1407those methods are defined for an object, it is said to be a descriptor.
1408
1409The default behavior for attribute access is to get, set, or delete the
1410attribute from an object's dictionary. For instance, ``a.x`` has a lookup chain
1411starting with ``a.__dict__['x']``, then ``type(a).__dict__['x']``, and
1412continuing through the base classes of ``type(a)`` excluding metaclasses.
1413
1414However, if the looked-up value is an object defining one of the descriptor
1415methods, then Python may override the default behavior and invoke the descriptor
1416method instead. Where this occurs in the precedence chain depends on which
1417descriptor methods were defined and how they were called. Note that descriptors
1418are only invoked for new style objects or classes (ones that subclass
1419:class:`object()` or :class:`type()`).
1420
1421The starting point for descriptor invocation is a binding, ``a.x``. How the
1422arguments are assembled depends on ``a``:
1423
1424Direct Call
1425 The simplest and least common call is when user code directly invokes a
1426 descriptor method: ``x.__get__(a)``.
1427
1428Instance Binding
Georg Brandl85eb8c12007-08-31 16:33:38 +00001429 If binding to an object instance, ``a.x`` is transformed into the call:
Georg Brandl116aa622007-08-15 14:28:22 +00001430 ``type(a).__dict__['x'].__get__(a, type(a))``.
1431
1432Class Binding
Georg Brandl85eb8c12007-08-31 16:33:38 +00001433 If binding to a class, ``A.x`` is transformed into the call:
Georg Brandl116aa622007-08-15 14:28:22 +00001434 ``A.__dict__['x'].__get__(None, A)``.
1435
1436Super Binding
1437 If ``a`` is an instance of :class:`super`, then the binding ``super(B,
1438 obj).m()`` searches ``obj.__class__.__mro__`` for the base class ``A``
1439 immediately preceding ``B`` and then invokes the descriptor with the call:
1440 ``A.__dict__['m'].__get__(obj, A)``.
1441
1442For instance bindings, the precedence of descriptor invocation depends on the
Guido van Rossum04110fb2007-08-24 16:32:05 +00001443which descriptor methods are defined. Normally, data descriptors define both
1444:meth:`__get__` and :meth:`__set__`, while non-data descriptors have just the
Georg Brandl116aa622007-08-15 14:28:22 +00001445:meth:`__get__` method. Data descriptors always override a redefinition in an
1446instance dictionary. In contrast, non-data descriptors can be overridden by
Guido van Rossum04110fb2007-08-24 16:32:05 +00001447instances. [#]_
Georg Brandl116aa622007-08-15 14:28:22 +00001448
1449Python methods (including :func:`staticmethod` and :func:`classmethod`) are
1450implemented as non-data descriptors. Accordingly, instances can redefine and
1451override methods. This allows individual instances to acquire behaviors that
1452differ from other instances of the same class.
1453
1454The :func:`property` function is implemented as a data descriptor. Accordingly,
1455instances cannot override the behavior of a property.
1456
1457
1458.. _slots:
1459
1460__slots__
1461^^^^^^^^^
1462
Georg Brandl85eb8c12007-08-31 16:33:38 +00001463By default, instances of classes have a dictionary for attribute storage. This
1464wastes space for objects having very few instance variables. The space
1465consumption can become acute when creating large numbers of instances.
Georg Brandl116aa622007-08-15 14:28:22 +00001466
Georg Brandl85eb8c12007-08-31 16:33:38 +00001467The default can be overridden by defining *__slots__* in a class definition.
1468The *__slots__* declaration takes a sequence of instance variables and reserves
1469just enough space in each instance to hold a value for each variable. Space is
1470saved because *__dict__* is not created for each instance.
Georg Brandl116aa622007-08-15 14:28:22 +00001471
1472
Georg Brandl85eb8c12007-08-31 16:33:38 +00001473.. data:: object.__slots__
Georg Brandl116aa622007-08-15 14:28:22 +00001474
Georg Brandl85eb8c12007-08-31 16:33:38 +00001475 This class variable can be assigned a string, iterable, or sequence of
1476 strings with variable names used by instances. If defined in a new-style
1477 class, *__slots__* reserves space for the declared variables and prevents the
1478 automatic creation of *__dict__* and *__weakref__* for each instance.
Georg Brandl116aa622007-08-15 14:28:22 +00001479
Georg Brandl116aa622007-08-15 14:28:22 +00001480
1481Notes on using *__slots__*
Georg Brandl16174572007-09-01 12:38:06 +00001482""""""""""""""""""""""""""
Georg Brandl116aa622007-08-15 14:28:22 +00001483
1484* Without a *__dict__* variable, instances cannot be assigned new variables not
1485 listed in the *__slots__* definition. Attempts to assign to an unlisted
1486 variable name raises :exc:`AttributeError`. If dynamic assignment of new
Georg Brandl85eb8c12007-08-31 16:33:38 +00001487 variables is desired, then add ``'__dict__'`` to the sequence of strings in
1488 the *__slots__* declaration.
Georg Brandl116aa622007-08-15 14:28:22 +00001489
Georg Brandl116aa622007-08-15 14:28:22 +00001490* Without a *__weakref__* variable for each instance, classes defining
1491 *__slots__* do not support weak references to its instances. If weak reference
1492 support is needed, then add ``'__weakref__'`` to the sequence of strings in the
1493 *__slots__* declaration.
1494
Georg Brandl116aa622007-08-15 14:28:22 +00001495* *__slots__* are implemented at the class level by creating descriptors
1496 (:ref:`descriptors`) for each variable name. As a result, class attributes
1497 cannot be used to set default values for instance variables defined by
1498 *__slots__*; otherwise, the class attribute would overwrite the descriptor
1499 assignment.
1500
1501* If a class defines a slot also defined in a base class, the instance variable
1502 defined by the base class slot is inaccessible (except by retrieving its
1503 descriptor directly from the base class). This renders the meaning of the
1504 program undefined. In the future, a check may be added to prevent this.
1505
1506* The action of a *__slots__* declaration is limited to the class where it is
1507 defined. As a result, subclasses will have a *__dict__* unless they also define
1508 *__slots__*.
1509
1510* *__slots__* do not work for classes derived from "variable-length" built-in
Georg Brandl5c106642007-11-29 17:41:05 +00001511 types such as :class:`int`, :class:`str` and :class:`tuple`.
Georg Brandl116aa622007-08-15 14:28:22 +00001512
1513* Any non-string iterable may be assigned to *__slots__*. Mappings may also be
1514 used; however, in the future, special meaning may be assigned to the values
1515 corresponding to each key.
1516
1517* *__class__* assignment works only if both classes have the same *__slots__*.
1518
Georg Brandl116aa622007-08-15 14:28:22 +00001519
1520.. _metaclasses:
1521
1522Customizing class creation
1523--------------------------
1524
Georg Brandl85eb8c12007-08-31 16:33:38 +00001525By default, classes are constructed using :func:`type`. A class definition is
1526read into a separate namespace and the value of class name is bound to the
1527result of ``type(name, bases, dict)``.
Georg Brandl116aa622007-08-15 14:28:22 +00001528
1529When the class definition is read, if *__metaclass__* is defined then the
Christian Heimes790c8232008-01-07 21:14:23 +00001530callable assigned to it will be called instead of :func:`type`. This allows
Georg Brandl116aa622007-08-15 14:28:22 +00001531classes or functions to be written which monitor or alter the class creation
1532process:
1533
1534* Modifying the class dictionary prior to the class being created.
1535
1536* Returning an instance of another class -- essentially performing the role of a
1537 factory function.
1538
Christian Heimes790c8232008-01-07 21:14:23 +00001539These steps will have to be performed in the metaclass's :meth:`__new__` method
1540-- :meth:`type.__new__` can then be called from this method to create a class
1541with different properties. This example adds a new element to the class
1542dictionary before creating the class::
1543
1544 class metacls(type):
1545 def __new__(mcs, name, bases, dict):
1546 dict['foo'] = 'metacls was here'
1547 return type.__new__(mcs, name, bases, dict)
1548
1549You can of course also override other class methods (or add new methods); for
1550example defining a custom :meth:`__call__` method in the metaclass allows custom
1551behavior when the class is called, e.g. not always creating a new instance.
1552
1553
Georg Brandl116aa622007-08-15 14:28:22 +00001554.. data:: __metaclass__
1555
1556 This variable can be any callable accepting arguments for ``name``, ``bases``,
1557 and ``dict``. Upon class creation, the callable is used instead of the built-in
1558 :func:`type`.
1559
Georg Brandl116aa622007-08-15 14:28:22 +00001560The appropriate metaclass is determined by the following precedence rules:
1561
1562* If ``dict['__metaclass__']`` exists, it is used.
1563
1564* Otherwise, if there is at least one base class, its metaclass is used (this
1565 looks for a *__class__* attribute first and if not found, uses its type).
1566
1567* Otherwise, if a global variable named __metaclass__ exists, it is used.
1568
Georg Brandl85eb8c12007-08-31 16:33:38 +00001569* Otherwise, the default metaclass (:class:`type`) is used.
Georg Brandl116aa622007-08-15 14:28:22 +00001570
1571The potential uses for metaclasses are boundless. Some ideas that have been
1572explored including logging, interface checking, automatic delegation, automatic
1573property creation, proxies, frameworks, and automatic resource
1574locking/synchronization.
1575
1576
1577.. _callable-types:
1578
1579Emulating callable objects
1580--------------------------
1581
1582
1583.. method:: object.__call__(self[, args...])
1584
1585 .. index:: pair: call; instance
1586
1587 Called when the instance is "called" as a function; if this method is defined,
1588 ``x(arg1, arg2, ...)`` is a shorthand for ``x.__call__(arg1, arg2, ...)``.
1589
1590
1591.. _sequence-types:
1592
1593Emulating container types
1594-------------------------
1595
1596The following methods can be defined to implement container objects. Containers
1597usually are sequences (such as lists or tuples) or mappings (like dictionaries),
1598but can represent other containers as well. The first set of methods is used
1599either to emulate a sequence or to emulate a mapping; the difference is that for
1600a sequence, the allowable keys should be the integers *k* for which ``0 <= k <
1601N`` where *N* is the length of the sequence, or slice objects, which define a
Georg Brandlcb8ecb12007-09-04 06:35:14 +00001602range of items. It is also recommended that mappings provide the methods
Collin Winter19ab2bd2007-09-10 00:20:46 +00001603:meth:`keys`, :meth:`values`, :meth:`items`, :meth:`get`,
Fred Drake2e748782007-09-04 17:33:11 +00001604:meth:`clear`, :meth:`setdefault`,
1605:meth:`pop`, :meth:`popitem`, :meth:`copy`, and
Georg Brandlcb8ecb12007-09-04 06:35:14 +00001606:meth:`update` behaving similar to those for Python's standard dictionary
1607objects. The :mod:`UserDict` module provides a :class:`DictMixin` class to help
1608create those methods from a base set of :meth:`__getitem__`,
1609:meth:`__setitem__`, :meth:`__delitem__`, and :meth:`keys`. Mutable sequences
1610should provide methods :meth:`append`, :meth:`count`, :meth:`index`,
1611:meth:`extend`, :meth:`insert`, :meth:`pop`, :meth:`remove`, :meth:`reverse` and
1612:meth:`sort`, like Python standard list objects. Finally, sequence types should
1613implement addition (meaning concatenation) and multiplication (meaning
1614repetition) by defining the methods :meth:`__add__`, :meth:`__radd__`,
1615:meth:`__iadd__`, :meth:`__mul__`, :meth:`__rmul__` and :meth:`__imul__`
1616described below; they should not define other numerical operators. It is
1617recommended that both mappings and sequences implement the :meth:`__contains__`
1618method to allow efficient use of the ``in`` operator; for mappings, ``in``
Collin Winter19ab2bd2007-09-10 00:20:46 +00001619should search the mapping's keys; for sequences, it should search
1620through the values. It is further recommended that both mappings and sequences
Georg Brandlcb8ecb12007-09-04 06:35:14 +00001621implement the :meth:`__iter__` method to allow efficient iteration through the
1622container; for mappings, :meth:`__iter__` should be the same as
Fred Drake2e748782007-09-04 17:33:11 +00001623:meth:`keys`; for sequences, it should iterate through the values.
Georg Brandl116aa622007-08-15 14:28:22 +00001624
1625.. method:: object.__len__(self)
1626
1627 .. index::
1628 builtin: len
1629 single: __bool__() (object method)
1630
1631 Called to implement the built-in function :func:`len`. Should return the length
1632 of the object, an integer ``>=`` 0. Also, an object that doesn't define a
1633 :meth:`__bool__` method and whose :meth:`__len__` method returns zero is
1634 considered to be false in a Boolean context.
1635
1636
Georg Brandlcb8ecb12007-09-04 06:35:14 +00001637.. note::
1638
1639 Slicing is done exclusively with the following three methods. A call like ::
1640
1641 a[1:2] = b
1642
1643 is translated to ::
1644
1645 a[slice(1, 2, None)] = b
1646
1647 and so forth. Missing slice items are always filled in with ``None``.
1648
1649
Georg Brandl116aa622007-08-15 14:28:22 +00001650.. method:: object.__getitem__(self, key)
1651
1652 .. index:: object: slice
1653
1654 Called to implement evaluation of ``self[key]``. For sequence types, the
1655 accepted keys should be integers and slice objects. Note that the special
1656 interpretation of negative indexes (if the class wishes to emulate a sequence
1657 type) is up to the :meth:`__getitem__` method. If *key* is of an inappropriate
1658 type, :exc:`TypeError` may be raised; if of a value outside the set of indexes
1659 for the sequence (after any special interpretation of negative values),
1660 :exc:`IndexError` should be raised. For mapping types, if *key* is missing (not
1661 in the container), :exc:`KeyError` should be raised.
1662
1663 .. note::
1664
1665 :keyword:`for` loops expect that an :exc:`IndexError` will be raised for illegal
1666 indexes to allow proper detection of the end of the sequence.
1667
1668
1669.. method:: object.__setitem__(self, key, value)
1670
1671 Called to implement assignment to ``self[key]``. Same note as for
1672 :meth:`__getitem__`. This should only be implemented for mappings if the
1673 objects support changes to the values for keys, or if new keys can be added, or
1674 for sequences if elements can be replaced. The same exceptions should be raised
1675 for improper *key* values as for the :meth:`__getitem__` method.
1676
1677
1678.. method:: object.__delitem__(self, key)
1679
1680 Called to implement deletion of ``self[key]``. Same note as for
1681 :meth:`__getitem__`. This should only be implemented for mappings if the
1682 objects support removal of keys, or for sequences if elements can be removed
1683 from the sequence. The same exceptions should be raised for improper *key*
1684 values as for the :meth:`__getitem__` method.
1685
1686
1687.. method:: object.__iter__(self)
1688
1689 This method is called when an iterator is required for a container. This method
1690 should return a new iterator object that can iterate over all the objects in the
1691 container. For mappings, it should iterate over the keys of the container, and
Fred Drake2e748782007-09-04 17:33:11 +00001692 should also be made available as the method :meth:`keys`.
Georg Brandl116aa622007-08-15 14:28:22 +00001693
1694 Iterator objects also need to implement this method; they are required to return
1695 themselves. For more information on iterator objects, see :ref:`typeiter`.
1696
Christian Heimes7f044312008-01-06 17:05:40 +00001697
1698.. method:: object.__reversed__(self)
1699
1700 Called (if present) by the :func:`reversed` builtin to implement
1701 reverse iteration. It should return a new iterator object that iterates
1702 over all the objects in the container in reverse order.
1703
1704 If the :meth:`__reversed__` method is not provided, the
1705 :func:`reversed` builtin will fall back to using the sequence protocol
1706 (:meth:`__len__` and :meth:`__getitem__`). Objects should normally
1707 only provide :meth:`__reversed__` if they do not support the sequence
1708 protocol and an efficient implementation of reverse iteration is possible.
1709
1710
Georg Brandl116aa622007-08-15 14:28:22 +00001711The membership test operators (:keyword:`in` and :keyword:`not in`) are normally
1712implemented as an iteration through a sequence. However, container objects can
1713supply the following special method with a more efficient implementation, which
1714also does not require the object be a sequence.
1715
1716
1717.. method:: object.__contains__(self, item)
1718
1719 Called to implement membership test operators. Should return true if *item* is
1720 in *self*, false otherwise. For mapping objects, this should consider the keys
1721 of the mapping rather than the values or the key-item pairs.
1722
1723
Georg Brandl116aa622007-08-15 14:28:22 +00001724.. _numeric-types:
1725
1726Emulating numeric types
1727-----------------------
1728
1729The following methods can be defined to emulate numeric objects. Methods
1730corresponding to operations that are not supported by the particular kind of
1731number implemented (e.g., bitwise operations for non-integral numbers) should be
1732left undefined.
1733
1734
1735.. method:: object.__add__(self, other)
1736 object.__sub__(self, other)
1737 object.__mul__(self, other)
1738 object.__floordiv__(self, other)
1739 object.__mod__(self, other)
1740 object.__divmod__(self, other)
1741 object.__pow__(self, other[, modulo])
1742 object.__lshift__(self, other)
1743 object.__rshift__(self, other)
1744 object.__and__(self, other)
1745 object.__xor__(self, other)
1746 object.__or__(self, other)
1747
1748 .. index::
1749 builtin: divmod
1750 builtin: pow
1751 builtin: pow
1752
1753 These methods are called to implement the binary arithmetic operations (``+``,
1754 ``-``, ``*``, ``//``, ``%``, :func:`divmod`, :func:`pow`, ``**``, ``<<``,
1755 ``>>``, ``&``, ``^``, ``|``). For instance, to evaluate the expression
1756 *x*``+``*y*, where *x* is an instance of a class that has an :meth:`__add__`
1757 method, ``x.__add__(y)`` is called. The :meth:`__divmod__` method should be the
1758 equivalent to using :meth:`__floordiv__` and :meth:`__mod__`; it should not be
1759 related to :meth:`__truediv__` (described below). Note that :meth:`__pow__`
1760 should be defined to accept an optional third argument if the ternary version of
1761 the built-in :func:`pow` function is to be supported.
1762
1763 If one of those methods does not support the operation with the supplied
1764 arguments, it should return ``NotImplemented``.
1765
1766
1767.. method:: object.__div__(self, other)
1768 object.__truediv__(self, other)
1769
1770 The division operator (``/``) is implemented by these methods. The
1771 :meth:`__truediv__` method is used when ``__future__.division`` is in effect,
1772 otherwise :meth:`__div__` is used. If only one of these two methods is defined,
1773 the object will not support division in the alternate context; :exc:`TypeError`
1774 will be raised instead.
1775
1776
1777.. method:: object.__radd__(self, other)
1778 object.__rsub__(self, other)
1779 object.__rmul__(self, other)
1780 object.__rdiv__(self, other)
1781 object.__rtruediv__(self, other)
1782 object.__rfloordiv__(self, other)
1783 object.__rmod__(self, other)
1784 object.__rdivmod__(self, other)
1785 object.__rpow__(self, other)
1786 object.__rlshift__(self, other)
1787 object.__rrshift__(self, other)
1788 object.__rand__(self, other)
1789 object.__rxor__(self, other)
1790 object.__ror__(self, other)
1791
1792 .. index::
1793 builtin: divmod
1794 builtin: pow
1795
1796 These methods are called to implement the binary arithmetic operations (``+``,
1797 ``-``, ``*``, ``/``, ``%``, :func:`divmod`, :func:`pow`, ``**``, ``<<``, ``>>``,
1798 ``&``, ``^``, ``|``) with reflected (swapped) operands. These functions are
1799 only called if the left operand does not support the corresponding operation and
1800 the operands are of different types. [#]_ For instance, to evaluate the
1801 expression *x*``-``*y*, where *y* is an instance of a class that has an
1802 :meth:`__rsub__` method, ``y.__rsub__(x)`` is called if ``x.__sub__(y)`` returns
1803 *NotImplemented*.
1804
1805 .. index:: builtin: pow
1806
1807 Note that ternary :func:`pow` will not try calling :meth:`__rpow__` (the
1808 coercion rules would become too complicated).
1809
1810 .. note::
1811
1812 If the right operand's type is a subclass of the left operand's type and that
1813 subclass provides the reflected method for the operation, this method will be
1814 called before the left operand's non-reflected method. This behavior allows
1815 subclasses to override their ancestors' operations.
1816
1817
1818.. method:: object.__iadd__(self, other)
1819 object.__isub__(self, other)
1820 object.__imul__(self, other)
1821 object.__idiv__(self, other)
1822 object.__itruediv__(self, other)
1823 object.__ifloordiv__(self, other)
1824 object.__imod__(self, other)
1825 object.__ipow__(self, other[, modulo])
1826 object.__ilshift__(self, other)
1827 object.__irshift__(self, other)
1828 object.__iand__(self, other)
1829 object.__ixor__(self, other)
1830 object.__ior__(self, other)
1831
1832 These methods are called to implement the augmented arithmetic operations
1833 (``+=``, ``-=``, ``*=``, ``/=``, ``//=``, ``%=``, ``**=``, ``<<=``, ``>>=``,
1834 ``&=``, ``^=``, ``|=``). These methods should attempt to do the operation
1835 in-place (modifying *self*) and return the result (which could be, but does
1836 not have to be, *self*). If a specific method is not defined, the augmented
1837 operation falls back to the normal methods. For instance, to evaluate the
1838 expression *x*``+=``*y*, where *x* is an instance of a class that has an
1839 :meth:`__iadd__` method, ``x.__iadd__(y)`` is called. If *x* is an instance
1840 of a class that does not define a :meth:`__iadd__` method, ``x.__add__(y)``
1841 and ``y.__radd__(x)`` are considered, as with the evaluation of *x*``+``*y*.
1842
1843
1844.. method:: object.__neg__(self)
1845 object.__pos__(self)
1846 object.__abs__(self)
1847 object.__invert__(self)
1848
1849 .. index:: builtin: abs
1850
1851 Called to implement the unary arithmetic operations (``-``, ``+``, :func:`abs`
1852 and ``~``).
1853
1854
1855.. method:: object.__complex__(self)
1856 object.__int__(self)
Georg Brandl116aa622007-08-15 14:28:22 +00001857 object.__float__(self)
1858
1859 .. index::
1860 builtin: complex
1861 builtin: int
Georg Brandl116aa622007-08-15 14:28:22 +00001862 builtin: float
1863
Georg Brandl5c106642007-11-29 17:41:05 +00001864 Called to implement the built-in functions :func:`complex`, :func:`int`
1865 and :func:`float`. Should return a value of the appropriate type.
Georg Brandl116aa622007-08-15 14:28:22 +00001866
1867
1868.. method:: object.__index__(self)
1869
1870 Called to implement :func:`operator.index`. Also called whenever Python needs
1871 an integer object (such as in slicing, or in the built-in :func:`bin`,
Georg Brandl5c106642007-11-29 17:41:05 +00001872 :func:`hex` and :func:`oct` functions). Must return an integer.
Georg Brandl116aa622007-08-15 14:28:22 +00001873
Georg Brandl116aa622007-08-15 14:28:22 +00001874
1875.. _context-managers:
1876
1877With Statement Context Managers
1878-------------------------------
1879
Georg Brandl116aa622007-08-15 14:28:22 +00001880A :dfn:`context manager` is an object that defines the runtime context to be
1881established when executing a :keyword:`with` statement. The context manager
1882handles the entry into, and the exit from, the desired runtime context for the
1883execution of the block of code. Context managers are normally invoked using the
1884:keyword:`with` statement (described in section :ref:`with`), but can also be
1885used by directly invoking their methods.
1886
1887.. index::
1888 statement: with
1889 single: context manager
1890
1891Typical uses of context managers include saving and restoring various kinds of
1892global state, locking and unlocking resources, closing opened files, etc.
1893
1894For more information on context managers, see :ref:`typecontextmanager`.
1895
1896
1897.. method:: object.__enter__(self)
1898
1899 Enter the runtime context related to this object. The :keyword:`with` statement
1900 will bind this method's return value to the target(s) specified in the
1901 :keyword:`as` clause of the statement, if any.
1902
1903
1904.. method:: object.__exit__(self, exc_type, exc_value, traceback)
1905
1906 Exit the runtime context related to this object. The parameters describe the
1907 exception that caused the context to be exited. If the context was exited
1908 without an exception, all three arguments will be :const:`None`.
1909
1910 If an exception is supplied, and the method wishes to suppress the exception
1911 (i.e., prevent it from being propagated), it should return a true value.
1912 Otherwise, the exception will be processed normally upon exit from this method.
1913
1914 Note that :meth:`__exit__` methods should not reraise the passed-in exception;
1915 this is the caller's responsibility.
1916
1917
1918.. seealso::
1919
1920 :pep:`0343` - The "with" statement
1921 The specification, background, and examples for the Python :keyword:`with`
1922 statement.
1923
1924.. rubric:: Footnotes
1925
Christian Heimesfaf2f632008-01-06 16:59:19 +00001926.. [#] Since Python 2.2, a gradual merging of types and classes has been started that
1927 makes this and a few other assertions made in this manual not 100% accurate and
1928 complete: for example, it *is* now possible in some cases to change an object's
1929 type, under certain controlled conditions. Until this manual undergoes
1930 extensive revision, it must now be taken as authoritative only regarding
1931 "classic classes", that are still the default, for compatibility purposes, in
1932 Python 2.2 and 2.3. For more information, see
1933 http://www.python.org/doc/newstyle/.
1934
1935.. [#] This, and other statements, are only roughly true for instances of new-style
1936 classes.
1937
Guido van Rossum04110fb2007-08-24 16:32:05 +00001938.. [#] A descriptor can define any combination of :meth:`__get__`,
1939 :meth:`__set__` and :meth:`__delete__`. If it does not define :meth:`__get__`,
1940 then accessing the attribute even on an instance will return the descriptor
1941 object itself. If the descriptor defines :meth:`__set__` and/or
1942 :meth:`__delete__`, it is a data descriptor; if it defines neither, it is a
1943 non-data descriptor.
1944
Georg Brandl116aa622007-08-15 14:28:22 +00001945.. [#] For operands of the same type, it is assumed that if the non-reflected method
1946 (such as :meth:`__add__`) fails the operation is not supported, which is why the
1947 reflected method is not called.
1948