blob: 7966a2ea9a05f70651072c615a0e15495071b810 [file] [log] [blame]
Georg Brandl116aa622007-08-15 14:28:22 +00001
Raymond Hettinger53dbe392008-02-12 20:03:09 +00002:mod:`collections` --- Container datatypes
3==========================================
Georg Brandl116aa622007-08-15 14:28:22 +00004
5.. module:: collections
Raymond Hettinger53dbe392008-02-12 20:03:09 +00006 :synopsis: Container datatypes
Georg Brandl116aa622007-08-15 14:28:22 +00007.. moduleauthor:: Raymond Hettinger <python@rcn.com>
8.. sectionauthor:: Raymond Hettinger <python@rcn.com>
9
Christian Heimesfe337bf2008-03-23 21:54:12 +000010.. testsetup:: *
11
12 from collections import *
13 import itertools
14 __name__ = '<doctest>'
Georg Brandl116aa622007-08-15 14:28:22 +000015
Georg Brandl116aa622007-08-15 14:28:22 +000016This module implements high-performance container datatypes. Currently,
17there are two datatypes, :class:`deque` and :class:`defaultdict`, and
Mark Summerfield71316b02008-02-14 16:28:00 +000018one datatype factory function, :func:`namedtuple`. This module also
19provides the :class:`UserDict` and :class:`UserList` classes which may
20be useful when inheriting directly from :class:`dict` or
21:class:`list` isn't convenient.
Christian Heimes0bd4e112008-02-12 22:59:25 +000022
Raymond Hettingerebcee3f2008-02-06 19:54:00 +000023The specialized containers provided in this module provide alternatives
Christian Heimesfe337bf2008-03-23 21:54:12 +000024to Python's general purpose built-in containers, :class:`dict`,
Raymond Hettingerebcee3f2008-02-06 19:54:00 +000025:class:`list`, :class:`set`, and :class:`tuple`.
Raymond Hettingerebcee3f2008-02-06 19:54:00 +000026Besides the containers provided here, the optional :mod:`bsddb`
Christian Heimesfe337bf2008-03-23 21:54:12 +000027module offers the ability to create in-memory or file based ordered
Raymond Hettingerebcee3f2008-02-06 19:54:00 +000028dictionaries with string keys using the :meth:`bsddb.btopen` method.
Georg Brandl116aa622007-08-15 14:28:22 +000029
Mark Summerfield08898b42007-09-05 08:43:04 +000030In addition to containers, the collections module provides some ABCs
Christian Heimesfe337bf2008-03-23 21:54:12 +000031(abstract base classes) that can be used to test whether a class
Raymond Hettingerebcee3f2008-02-06 19:54:00 +000032provides a particular interface, for example, is it hashable or
Mark Summerfield71316b02008-02-14 16:28:00 +000033a mapping, and some of them can also be used as mixin classes.
Raymond Hettingerebcee3f2008-02-06 19:54:00 +000034
35ABCs - abstract base classes
36----------------------------
37
38The collections module offers the following ABCs:
Mark Summerfield08898b42007-09-05 08:43:04 +000039
Raymond Hettinger409fb2c2008-02-09 02:17:06 +000040========================= ==================== ====================== ====================================================
41ABC Inherits Abstract Methods Mixin Methods
42========================= ==================== ====================== ====================================================
43:class:`Container` ``__contains__``
44:class:`Hashable` ``__hash__``
45:class:`Iterable` ``__iter__``
46:class:`Iterator` :class:`Iterable` ``__next__`` ``__iter__``
47:class:`Sized` ``__len__``
48
49:class:`Mapping` :class:`Sized`, ``__getitem__``, ``__contains__``, ``keys``, ``items``, ``values``,
50 :class:`Iterable`, ``__len__``. and ``get``, ``__eq__``, and ``__ne__``
51 :class:`Container` ``__iter__``
52
53:class:`MutableMapping` :class:`Mapping` ``__getitem__`` Inherited Mapping methods and
54 ``__setitem__``, ``pop``, ``popitem``, ``clear``, ``update``,
55 ``__delitem__``, and ``setdefault``
56 ``__iter__``, and
57 ``__len__``
58
59:class:`Sequence` :class:`Sized`, ``__getitem__`` ``__contains__``. ``__iter__``, ``__reversed__``.
60 :class:`Iterable`, and ``__len__`` ``index``, and ``count``
61 :class:`Container`
62
63:class:`MutableSequnce` :class:`Sequence` ``__getitem__`` Inherited Sequence methods and
64 ``__delitem__``, ``append``, ``reverse``, ``extend``, ``pop``,
65 ``insert``, ``remove``, and ``__iadd__``
66 and ``__len__``
67
Raymond Hettinger0dbdab22008-02-09 03:48:16 +000068:class:`Set` :class:`Sized`, ``__len__``, ``__le__``, ``__lt__``, ``__eq__``, ``__ne__``,
Raymond Hettinger409fb2c2008-02-09 02:17:06 +000069 :class:`Iterable`, ``__iter__``, and ``__gt__``, ``__ge__``, ``__and__``, ``__or__``
70 :class:`Container` ``__contains__`` ``__sub__``, ``__xor__``, and ``isdisjoint``
71
72:class:`MutableSet` :class:`Set` ``add`` and Inherited Set methods and
73 ``discard`` ``clear``, ``pop``, ``remove``, ``__ior__``,
74 ``__iand__``, ``__ixor__``, and ``__isub__``
75========================= ==================== ====================== ====================================================
Mark Summerfield08898b42007-09-05 08:43:04 +000076
Mark Summerfield08898b42007-09-05 08:43:04 +000077These ABCs allow us to ask classes or instances if they provide
78particular functionality, for example::
79
Mark Summerfield08898b42007-09-05 08:43:04 +000080 size = None
Raymond Hettingerebcee3f2008-02-06 19:54:00 +000081 if isinstance(myvar, collections.Sized):
Mark Summerfield08898b42007-09-05 08:43:04 +000082 size = len(myvar)
83
Raymond Hettingerebcee3f2008-02-06 19:54:00 +000084Several of the ABCs are also useful as mixins that make it easier to develop
85classes supporting container APIs. For example, to write a class supporting
86the full :class:`Set` API, it only necessary to supply the three underlying
87abstract methods: :meth:`__contains__`, :meth:`__iter__`, and :meth:`__len__`.
88The ABC supplies the remaining methods such as :meth:`__and__` and
89:meth:`isdisjoint` ::
90
91 class ListBasedSet(collections.Set):
Raymond Hettingerc1b6a4a2008-02-08 23:46:23 +000092 ''' Alternate set implementation favoring space over speed
93 and not requiring the set elements to be hashable. '''
Raymond Hettingerebcee3f2008-02-06 19:54:00 +000094 def __init__(self, iterable):
Raymond Hettingerc1b6a4a2008-02-08 23:46:23 +000095 self.elements = lst = []
96 for value in iterable:
97 if value not in lst:
98 lst.append(value)
Raymond Hettingerebcee3f2008-02-06 19:54:00 +000099 def __iter__(self):
100 return iter(self.elements)
101 def __contains__(self, value):
102 return value in self.elements
103 def __len__(self):
104 return len(self.elements)
105
106 s1 = ListBasedSet('abcdef')
107 s2 = ListBasedSet('defghi')
108 overlap = s1 & s2 # The __and__() method is supported automatically
109
Raymond Hettinger7aebb642008-02-09 03:25:08 +0000110Notes on using :class:`Set` and :class:`MutableSet` as a mixin:
111
Christian Heimesfe337bf2008-03-23 21:54:12 +0000112(1)
Raymond Hettinger7aebb642008-02-09 03:25:08 +0000113 Since some set operations create new sets, the default mixin methods need
Christian Heimesfe337bf2008-03-23 21:54:12 +0000114 a way to create new instances from an iterable. The class constructor is
115 assumed to have a signature in the form ``ClassName(iterable)``.
Mark Summerfield71316b02008-02-14 16:28:00 +0000116 That assumption is factored-out to a single internal classmethod called
Raymond Hettinger7aebb642008-02-09 03:25:08 +0000117 :meth:`_from_iterable` which calls ``cls(iterable)`` to produce a new set.
118 If the :class:`Set` mixin is being used in a class with a different
Christian Heimesfe337bf2008-03-23 21:54:12 +0000119 constructor signature, you will need to override :meth:`from_iterable`
120 with a classmethod that can construct new instances from
Raymond Hettinger7aebb642008-02-09 03:25:08 +0000121 an iterable argument.
122
123(2)
124 To override the comparisons (presumably for speed, as the
125 semantics are fixed), redefine :meth:`__le__` and
126 then the other operations will automatically follow suit.
Raymond Hettingerebcee3f2008-02-06 19:54:00 +0000127
Raymond Hettinger0dbdab22008-02-09 03:48:16 +0000128(3)
129 The :class:`Set` mixin provides a :meth:`_hash` method to compute a hash value
130 for the set; however, :meth:`__hash__` is not defined because not all sets
131 are hashable or immutable. To add set hashabilty using mixins,
132 inherit from both :meth:`Set` and :meth:`Hashable`, then define
133 ``__hash__ = Set._hash``.
134
Mark Summerfield08898b42007-09-05 08:43:04 +0000135(For more about ABCs, see the :mod:`abc` module and :pep:`3119`.)
136
137
Georg Brandl116aa622007-08-15 14:28:22 +0000138.. _deque-objects:
139
140:class:`deque` objects
141----------------------
142
143
Georg Brandl9afde1c2007-11-01 20:32:30 +0000144.. class:: deque([iterable[, maxlen]])
Georg Brandl116aa622007-08-15 14:28:22 +0000145
146 Returns a new deque object initialized left-to-right (using :meth:`append`) with
147 data from *iterable*. If *iterable* is not specified, the new deque is empty.
148
149 Deques are a generalization of stacks and queues (the name is pronounced "deck"
150 and is short for "double-ended queue"). Deques support thread-safe, memory
151 efficient appends and pops from either side of the deque with approximately the
152 same O(1) performance in either direction.
153
154 Though :class:`list` objects support similar operations, they are optimized for
155 fast fixed-length operations and incur O(n) memory movement costs for
156 ``pop(0)`` and ``insert(0, v)`` operations which change both the size and
157 position of the underlying data representation.
158
Georg Brandl116aa622007-08-15 14:28:22 +0000159
Georg Brandl9afde1c2007-11-01 20:32:30 +0000160 If *maxlen* is not specified or is *None*, deques may grow to an
161 arbitrary length. Otherwise, the deque is bounded to the specified maximum
162 length. Once a bounded length deque is full, when new items are added, a
163 corresponding number of items are discarded from the opposite end. Bounded
164 length deques provide functionality similar to the ``tail`` filter in
165 Unix. They are also useful for tracking transactions and other pools of data
166 where only the most recent activity is of interest.
167
Georg Brandl9afde1c2007-11-01 20:32:30 +0000168
Georg Brandl116aa622007-08-15 14:28:22 +0000169Deque objects support the following methods:
170
Georg Brandl116aa622007-08-15 14:28:22 +0000171.. method:: deque.append(x)
172
173 Add *x* to the right side of the deque.
174
175
176.. method:: deque.appendleft(x)
177
178 Add *x* to the left side of the deque.
179
180
181.. method:: deque.clear()
182
183 Remove all elements from the deque leaving it with length 0.
184
185
186.. method:: deque.extend(iterable)
187
188 Extend the right side of the deque by appending elements from the iterable
189 argument.
190
191
192.. method:: deque.extendleft(iterable)
193
194 Extend the left side of the deque by appending elements from *iterable*. Note,
195 the series of left appends results in reversing the order of elements in the
196 iterable argument.
197
198
199.. method:: deque.pop()
200
201 Remove and return an element from the right side of the deque. If no elements
202 are present, raises an :exc:`IndexError`.
203
204
205.. method:: deque.popleft()
206
207 Remove and return an element from the left side of the deque. If no elements are
208 present, raises an :exc:`IndexError`.
209
210
211.. method:: deque.remove(value)
212
213 Removed the first occurrence of *value*. If not found, raises a
214 :exc:`ValueError`.
215
Georg Brandl116aa622007-08-15 14:28:22 +0000216
217.. method:: deque.rotate(n)
218
219 Rotate the deque *n* steps to the right. If *n* is negative, rotate to the
220 left. Rotating one step to the right is equivalent to:
221 ``d.appendleft(d.pop())``.
222
223In addition to the above, deques support iteration, pickling, ``len(d)``,
224``reversed(d)``, ``copy.copy(d)``, ``copy.deepcopy(d)``, membership testing with
225the :keyword:`in` operator, and subscript references such as ``d[-1]``.
226
Christian Heimesfe337bf2008-03-23 21:54:12 +0000227Example:
228
229.. doctest::
Georg Brandl116aa622007-08-15 14:28:22 +0000230
231 >>> from collections import deque
232 >>> d = deque('ghi') # make a new deque with three items
233 >>> for elem in d: # iterate over the deque's elements
Christian Heimesfe337bf2008-03-23 21:54:12 +0000234 ... print elem.upper()
Georg Brandl116aa622007-08-15 14:28:22 +0000235 G
236 H
237 I
238
239 >>> d.append('j') # add a new entry to the right side
240 >>> d.appendleft('f') # add a new entry to the left side
241 >>> d # show the representation of the deque
242 deque(['f', 'g', 'h', 'i', 'j'])
243
244 >>> d.pop() # return and remove the rightmost item
245 'j'
246 >>> d.popleft() # return and remove the leftmost item
247 'f'
248 >>> list(d) # list the contents of the deque
249 ['g', 'h', 'i']
250 >>> d[0] # peek at leftmost item
251 'g'
252 >>> d[-1] # peek at rightmost item
253 'i'
254
255 >>> list(reversed(d)) # list the contents of a deque in reverse
256 ['i', 'h', 'g']
257 >>> 'h' in d # search the deque
258 True
259 >>> d.extend('jkl') # add multiple elements at once
260 >>> d
261 deque(['g', 'h', 'i', 'j', 'k', 'l'])
262 >>> d.rotate(1) # right rotation
263 >>> d
264 deque(['l', 'g', 'h', 'i', 'j', 'k'])
265 >>> d.rotate(-1) # left rotation
266 >>> d
267 deque(['g', 'h', 'i', 'j', 'k', 'l'])
268
269 >>> deque(reversed(d)) # make a new deque in reverse order
270 deque(['l', 'k', 'j', 'i', 'h', 'g'])
271 >>> d.clear() # empty the deque
272 >>> d.pop() # cannot pop from an empty deque
273 Traceback (most recent call last):
274 File "<pyshell#6>", line 1, in -toplevel-
275 d.pop()
276 IndexError: pop from an empty deque
277
278 >>> d.extendleft('abc') # extendleft() reverses the input order
279 >>> d
280 deque(['c', 'b', 'a'])
281
282
283.. _deque-recipes:
284
Georg Brandl9afde1c2007-11-01 20:32:30 +0000285:class:`deque` Recipes
286^^^^^^^^^^^^^^^^^^^^^^
Georg Brandl116aa622007-08-15 14:28:22 +0000287
288This section shows various approaches to working with deques.
289
290The :meth:`rotate` method provides a way to implement :class:`deque` slicing and
291deletion. For example, a pure python implementation of ``del d[n]`` relies on
292the :meth:`rotate` method to position elements to be popped::
293
294 def delete_nth(d, n):
295 d.rotate(-n)
296 d.popleft()
297 d.rotate(n)
298
299To implement :class:`deque` slicing, use a similar approach applying
300:meth:`rotate` to bring a target element to the left side of the deque. Remove
301old entries with :meth:`popleft`, add new entries with :meth:`extend`, and then
302reverse the rotation.
Georg Brandl116aa622007-08-15 14:28:22 +0000303With minor variations on that approach, it is easy to implement Forth style
304stack manipulations such as ``dup``, ``drop``, ``swap``, ``over``, ``pick``,
305``rot``, and ``roll``.
306
Georg Brandl116aa622007-08-15 14:28:22 +0000307Multi-pass data reduction algorithms can be succinctly expressed and efficiently
308coded by extracting elements with multiple calls to :meth:`popleft`, applying
Georg Brandl9afde1c2007-11-01 20:32:30 +0000309a reduction function, and calling :meth:`append` to add the result back to the
310deque.
Georg Brandl116aa622007-08-15 14:28:22 +0000311
312For example, building a balanced binary tree of nested lists entails reducing
Christian Heimesfe337bf2008-03-23 21:54:12 +0000313two adjacent nodes into one by grouping them in a list:
Georg Brandl116aa622007-08-15 14:28:22 +0000314
315 >>> def maketree(iterable):
316 ... d = deque(iterable)
317 ... while len(d) > 1:
318 ... pair = [d.popleft(), d.popleft()]
319 ... d.append(pair)
320 ... return list(d)
321 ...
Georg Brandl6911e3c2007-09-04 07:15:32 +0000322 >>> print(maketree('abcdefgh'))
Georg Brandl116aa622007-08-15 14:28:22 +0000323 [[[['a', 'b'], ['c', 'd']], [['e', 'f'], ['g', 'h']]]]
324
Georg Brandl9afde1c2007-11-01 20:32:30 +0000325Bounded length deques provide functionality similar to the ``tail`` filter
326in Unix::
Georg Brandl116aa622007-08-15 14:28:22 +0000327
Georg Brandl9afde1c2007-11-01 20:32:30 +0000328 def tail(filename, n=10):
329 'Return the last n lines of a file'
330 return deque(open(filename), n)
Georg Brandl116aa622007-08-15 14:28:22 +0000331
332.. _defaultdict-objects:
333
334:class:`defaultdict` objects
335----------------------------
336
337
338.. class:: defaultdict([default_factory[, ...]])
339
340 Returns a new dictionary-like object. :class:`defaultdict` is a subclass of the
341 builtin :class:`dict` class. It overrides one method and adds one writable
342 instance variable. The remaining functionality is the same as for the
343 :class:`dict` class and is not documented here.
344
345 The first argument provides the initial value for the :attr:`default_factory`
346 attribute; it defaults to ``None``. All remaining arguments are treated the same
347 as if they were passed to the :class:`dict` constructor, including keyword
348 arguments.
349
Georg Brandl116aa622007-08-15 14:28:22 +0000350
351:class:`defaultdict` objects support the following method in addition to the
352standard :class:`dict` operations:
353
Georg Brandl116aa622007-08-15 14:28:22 +0000354.. method:: defaultdict.__missing__(key)
355
356 If the :attr:`default_factory` attribute is ``None``, this raises an
357 :exc:`KeyError` exception with the *key* as argument.
358
359 If :attr:`default_factory` is not ``None``, it is called without arguments to
360 provide a default value for the given *key*, this value is inserted in the
361 dictionary for the *key*, and returned.
362
363 If calling :attr:`default_factory` raises an exception this exception is
364 propagated unchanged.
365
366 This method is called by the :meth:`__getitem__` method of the :class:`dict`
367 class when the requested key is not found; whatever it returns or raises is then
368 returned or raised by :meth:`__getitem__`.
369
370:class:`defaultdict` objects support the following instance variable:
371
372
373.. attribute:: defaultdict.default_factory
374
375 This attribute is used by the :meth:`__missing__` method; it is initialized from
376 the first argument to the constructor, if present, or to ``None``, if absent.
377
378
379.. _defaultdict-examples:
380
381:class:`defaultdict` Examples
382^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
383
384Using :class:`list` as the :attr:`default_factory`, it is easy to group a
Christian Heimesfe337bf2008-03-23 21:54:12 +0000385sequence of key-value pairs into a dictionary of lists:
Georg Brandl116aa622007-08-15 14:28:22 +0000386
387 >>> s = [('yellow', 1), ('blue', 2), ('yellow', 3), ('blue', 4), ('red', 1)]
388 >>> d = defaultdict(list)
389 >>> for k, v in s:
390 ... d[k].append(v)
391 ...
392 >>> d.items()
393 [('blue', [2, 4]), ('red', [1]), ('yellow', [1, 3])]
394
395When each key is encountered for the first time, it is not already in the
396mapping; so an entry is automatically created using the :attr:`default_factory`
397function which returns an empty :class:`list`. The :meth:`list.append`
398operation then attaches the value to the new list. When keys are encountered
399again, the look-up proceeds normally (returning the list for that key) and the
400:meth:`list.append` operation adds another value to the list. This technique is
Christian Heimesfe337bf2008-03-23 21:54:12 +0000401simpler and faster than an equivalent technique using :meth:`dict.setdefault`:
Georg Brandl116aa622007-08-15 14:28:22 +0000402
403 >>> d = {}
404 >>> for k, v in s:
405 ... d.setdefault(k, []).append(v)
406 ...
407 >>> d.items()
408 [('blue', [2, 4]), ('red', [1]), ('yellow', [1, 3])]
409
410Setting the :attr:`default_factory` to :class:`int` makes the
411:class:`defaultdict` useful for counting (like a bag or multiset in other
Christian Heimesfe337bf2008-03-23 21:54:12 +0000412languages):
Georg Brandl116aa622007-08-15 14:28:22 +0000413
414 >>> s = 'mississippi'
415 >>> d = defaultdict(int)
416 >>> for k in s:
417 ... d[k] += 1
418 ...
419 >>> d.items()
420 [('i', 4), ('p', 2), ('s', 4), ('m', 1)]
421
422When a letter is first encountered, it is missing from the mapping, so the
423:attr:`default_factory` function calls :func:`int` to supply a default count of
424zero. The increment operation then builds up the count for each letter.
425
426The function :func:`int` which always returns zero is just a special case of
427constant functions. A faster and more flexible way to create constant functions
428is to use a lambda function which can supply any constant value (not just
Christian Heimesfe337bf2008-03-23 21:54:12 +0000429zero):
Georg Brandl116aa622007-08-15 14:28:22 +0000430
431 >>> def constant_factory(value):
432 ... return lambda: value
433 >>> d = defaultdict(constant_factory('<missing>'))
434 >>> d.update(name='John', action='ran')
435 >>> '%(name)s %(action)s to %(object)s' % d
436 'John ran to <missing>'
437
438Setting the :attr:`default_factory` to :class:`set` makes the
Christian Heimesfe337bf2008-03-23 21:54:12 +0000439:class:`defaultdict` useful for building a dictionary of sets:
Georg Brandl116aa622007-08-15 14:28:22 +0000440
441 >>> s = [('red', 1), ('blue', 2), ('red', 3), ('blue', 4), ('red', 1), ('blue', 4)]
442 >>> d = defaultdict(set)
443 >>> for k, v in s:
444 ... d[k].add(v)
445 ...
446 >>> d.items()
447 [('blue', set([2, 4])), ('red', set([1, 3]))]
448
449
450.. _named-tuple-factory:
451
Guido van Rossum3d392eb2007-11-16 00:35:22 +0000452:func:`namedtuple` Factory Function for Tuples with Named Fields
Christian Heimes790c8232008-01-07 21:14:23 +0000453----------------------------------------------------------------
Georg Brandl116aa622007-08-15 14:28:22 +0000454
Thomas Wouters1b7f8912007-09-19 03:06:30 +0000455Named tuples assign meaning to each position in a tuple and allow for more readable,
456self-documenting code. They can be used wherever regular tuples are used, and
457they add the ability to access fields by name instead of position index.
Georg Brandl116aa622007-08-15 14:28:22 +0000458
Guido van Rossum3d392eb2007-11-16 00:35:22 +0000459.. function:: namedtuple(typename, fieldnames, [verbose])
Georg Brandl116aa622007-08-15 14:28:22 +0000460
461 Returns a new tuple subclass named *typename*. The new subclass is used to
Christian Heimesc3f30c42008-02-22 16:37:40 +0000462 create tuple-like objects that have fields accessible by attribute lookup as
Georg Brandl116aa622007-08-15 14:28:22 +0000463 well as being indexable and iterable. Instances of the subclass also have a
464 helpful docstring (with typename and fieldnames) and a helpful :meth:`__repr__`
465 method which lists the tuple contents in a ``name=value`` format.
466
Georg Brandl9afde1c2007-11-01 20:32:30 +0000467 The *fieldnames* are a single string with each fieldname separated by whitespace
Christian Heimes25bb7832008-01-11 16:17:00 +0000468 and/or commas, for example ``'x y'`` or ``'x, y'``. Alternatively, *fieldnames*
469 can be a sequence of strings such as ``['x', 'y']``.
Georg Brandl9afde1c2007-11-01 20:32:30 +0000470
471 Any valid Python identifier may be used for a fieldname except for names
Christian Heimes0449f632007-12-15 01:27:15 +0000472 starting with an underscore. Valid identifiers consist of letters, digits,
473 and underscores but do not start with a digit or underscore and cannot be
Georg Brandlf6945182008-02-01 11:56:49 +0000474 a :mod:`keyword` such as *class*, *for*, *return*, *global*, *pass*,
Georg Brandl9afde1c2007-11-01 20:32:30 +0000475 or *raise*.
Georg Brandl116aa622007-08-15 14:28:22 +0000476
Christian Heimes25bb7832008-01-11 16:17:00 +0000477 If *verbose* is true, the class definition is printed just before being built.
Georg Brandl116aa622007-08-15 14:28:22 +0000478
Georg Brandl9afde1c2007-11-01 20:32:30 +0000479 Named tuple instances do not have per-instance dictionaries, so they are
Thomas Wouters8ce81f72007-09-20 18:22:40 +0000480 lightweight and require no more memory than regular tuples.
Georg Brandl116aa622007-08-15 14:28:22 +0000481
Christian Heimesfe337bf2008-03-23 21:54:12 +0000482Example:
483
484.. doctest::
485 :options: +NORMALIZE_WHITESPACE
Georg Brandl116aa622007-08-15 14:28:22 +0000486
Guido van Rossum3d392eb2007-11-16 00:35:22 +0000487 >>> Point = namedtuple('Point', 'x y', verbose=True)
Thomas Wouters1b7f8912007-09-19 03:06:30 +0000488 class Point(tuple):
489 'Point(x, y)'
Christian Heimesfe337bf2008-03-23 21:54:12 +0000490 <BLANKLINE>
Thomas Wouters1b7f8912007-09-19 03:06:30 +0000491 __slots__ = ()
Christian Heimesfe337bf2008-03-23 21:54:12 +0000492 <BLANKLINE>
Christian Heimesfaf2f632008-01-06 16:59:19 +0000493 _fields = ('x', 'y')
Christian Heimesfe337bf2008-03-23 21:54:12 +0000494 <BLANKLINE>
Thomas Wouters1b7f8912007-09-19 03:06:30 +0000495 def __new__(cls, x, y):
496 return tuple.__new__(cls, (x, y))
Christian Heimesfe337bf2008-03-23 21:54:12 +0000497 <BLANKLINE>
Christian Heimesfaf2f632008-01-06 16:59:19 +0000498 @classmethod
Christian Heimesfe337bf2008-03-23 21:54:12 +0000499 def _make(cls, iterable, new=tuple.__new__, len=len):
Christian Heimesfaf2f632008-01-06 16:59:19 +0000500 'Make a new Point object from a sequence or iterable'
Christian Heimesfe337bf2008-03-23 21:54:12 +0000501 result = new(cls, iterable)
Christian Heimesfaf2f632008-01-06 16:59:19 +0000502 if len(result) != 2:
503 raise TypeError('Expected 2 arguments, got %d' % len(result))
504 return result
Christian Heimesfe337bf2008-03-23 21:54:12 +0000505 <BLANKLINE>
Thomas Wouters1b7f8912007-09-19 03:06:30 +0000506 def __repr__(self):
507 return 'Point(x=%r, y=%r)' % self
Christian Heimesfe337bf2008-03-23 21:54:12 +0000508 <BLANKLINE>
Christian Heimes99170a52007-12-19 02:07:34 +0000509 def _asdict(t):
Christian Heimes0449f632007-12-15 01:27:15 +0000510 'Return a new dict which maps field names to their values'
Christian Heimes99170a52007-12-19 02:07:34 +0000511 return {'x': t[0], 'y': t[1]}
Christian Heimesfe337bf2008-03-23 21:54:12 +0000512 <BLANKLINE>
Christian Heimes0449f632007-12-15 01:27:15 +0000513 def _replace(self, **kwds):
Guido van Rossum3d392eb2007-11-16 00:35:22 +0000514 'Return a new Point object replacing specified fields with new values'
Christian Heimesfaf2f632008-01-06 16:59:19 +0000515 result = self._make(map(kwds.pop, ('x', 'y'), self))
516 if kwds:
517 raise ValueError('Got unexpected field names: %r' % kwds.keys())
518 return result
Christian Heimesfe337bf2008-03-23 21:54:12 +0000519 <BLANKLINE>
Thomas Wouters1b7f8912007-09-19 03:06:30 +0000520 x = property(itemgetter(0))
521 y = property(itemgetter(1))
Georg Brandl116aa622007-08-15 14:28:22 +0000522
Thomas Wouters1b7f8912007-09-19 03:06:30 +0000523 >>> p = Point(11, y=22) # instantiate with positional or keyword arguments
Christian Heimes99170a52007-12-19 02:07:34 +0000524 >>> p[0] + p[1] # indexable like the plain tuple (11, 22)
Thomas Wouters1b7f8912007-09-19 03:06:30 +0000525 33
526 >>> x, y = p # unpack like a regular tuple
527 >>> x, y
528 (11, 22)
Christian Heimesc3f30c42008-02-22 16:37:40 +0000529 >>> p.x + p.y # fields also accessible by name
Thomas Wouters1b7f8912007-09-19 03:06:30 +0000530 33
531 >>> p # readable __repr__ with a name=value style
532 Point(x=11, y=22)
Georg Brandl116aa622007-08-15 14:28:22 +0000533
Thomas Wouters1b7f8912007-09-19 03:06:30 +0000534Named tuples are especially useful for assigning field names to result tuples returned
535by the :mod:`csv` or :mod:`sqlite3` modules::
536
Guido van Rossum3d392eb2007-11-16 00:35:22 +0000537 EmployeeRecord = namedtuple('EmployeeRecord', 'name, age, title, department, paygrade')
Georg Brandl9afde1c2007-11-01 20:32:30 +0000538
Thomas Wouters1b7f8912007-09-19 03:06:30 +0000539 import csv
Christian Heimesfaf2f632008-01-06 16:59:19 +0000540 for emp in map(EmployeeRecord._make, csv.reader(open("employees.csv", "rb"))):
Thomas Wouters1b7f8912007-09-19 03:06:30 +0000541 print(emp.name, emp.title)
542
Georg Brandl9afde1c2007-11-01 20:32:30 +0000543 import sqlite3
544 conn = sqlite3.connect('/companydata')
545 cursor = conn.cursor()
546 cursor.execute('SELECT name, age, title, department, paygrade FROM employees')
Christian Heimesfaf2f632008-01-06 16:59:19 +0000547 for emp in map(EmployeeRecord._make, cursor.fetchall()):
Christian Heimes00412232008-01-10 16:02:19 +0000548 print(emp.name, emp.title)
Georg Brandl9afde1c2007-11-01 20:32:30 +0000549
Christian Heimes99170a52007-12-19 02:07:34 +0000550In addition to the methods inherited from tuples, named tuples support
Christian Heimes2380ac72008-01-09 00:17:24 +0000551three additional methods and one attribute. To prevent conflicts with
552field names, the method and attribute names start with an underscore.
Christian Heimes99170a52007-12-19 02:07:34 +0000553
Christian Heimes790c8232008-01-07 21:14:23 +0000554.. method:: somenamedtuple._make(iterable)
Christian Heimes99170a52007-12-19 02:07:34 +0000555
Christian Heimesfaf2f632008-01-06 16:59:19 +0000556 Class method that makes a new instance from an existing sequence or iterable.
Christian Heimes99170a52007-12-19 02:07:34 +0000557
Christian Heimesfe337bf2008-03-23 21:54:12 +0000558.. doctest::
Thomas Wouters1b7f8912007-09-19 03:06:30 +0000559
Christian Heimesfaf2f632008-01-06 16:59:19 +0000560 >>> t = [11, 22]
561 >>> Point._make(t)
562 Point(x=11, y=22)
Thomas Wouters1b7f8912007-09-19 03:06:30 +0000563
Christian Heimes790c8232008-01-07 21:14:23 +0000564.. method:: somenamedtuple._asdict()
Georg Brandl9afde1c2007-11-01 20:32:30 +0000565
Christian Heimesfe337bf2008-03-23 21:54:12 +0000566 Return a new dict which maps field names to their corresponding values::
Georg Brandl9afde1c2007-11-01 20:32:30 +0000567
Christian Heimes0449f632007-12-15 01:27:15 +0000568 >>> p._asdict()
Georg Brandl9afde1c2007-11-01 20:32:30 +0000569 {'x': 11, 'y': 22}
Christian Heimesfe337bf2008-03-23 21:54:12 +0000570
Christian Heimes790c8232008-01-07 21:14:23 +0000571.. method:: somenamedtuple._replace(kwargs)
Thomas Wouters1b7f8912007-09-19 03:06:30 +0000572
Christian Heimesfe337bf2008-03-23 21:54:12 +0000573 Return a new instance of the named tuple replacing specified fields with new
574 values:
Thomas Wouters8ce81f72007-09-20 18:22:40 +0000575
576::
Thomas Wouters1b7f8912007-09-19 03:06:30 +0000577
578 >>> p = Point(x=11, y=22)
Christian Heimes0449f632007-12-15 01:27:15 +0000579 >>> p._replace(x=33)
Thomas Wouters1b7f8912007-09-19 03:06:30 +0000580 Point(x=33, y=22)
581
Guido van Rossum3d392eb2007-11-16 00:35:22 +0000582 >>> for partnum, record in inventory.items():
Christian Heimes454f37b2008-01-10 00:10:02 +0000583 ... inventory[partnum] = record._replace(price=newprices[partnum], timestamp=time.now())
Thomas Wouters1b7f8912007-09-19 03:06:30 +0000584
Christian Heimes790c8232008-01-07 21:14:23 +0000585.. attribute:: somenamedtuple._fields
Thomas Wouters1b7f8912007-09-19 03:06:30 +0000586
Christian Heimes2380ac72008-01-09 00:17:24 +0000587 Tuple of strings listing the field names. Useful for introspection
Georg Brandl9afde1c2007-11-01 20:32:30 +0000588 and for creating new named tuple types from existing named tuples.
Thomas Wouters8ce81f72007-09-20 18:22:40 +0000589
Christian Heimesfe337bf2008-03-23 21:54:12 +0000590.. doctest::
Thomas Wouters1b7f8912007-09-19 03:06:30 +0000591
Christian Heimes0449f632007-12-15 01:27:15 +0000592 >>> p._fields # view the field names
Thomas Wouters1b7f8912007-09-19 03:06:30 +0000593 ('x', 'y')
Thomas Wouters1b7f8912007-09-19 03:06:30 +0000594
Guido van Rossum3d392eb2007-11-16 00:35:22 +0000595 >>> Color = namedtuple('Color', 'red green blue')
Christian Heimes0449f632007-12-15 01:27:15 +0000596 >>> Pixel = namedtuple('Pixel', Point._fields + Color._fields)
Thomas Wouters1b7f8912007-09-19 03:06:30 +0000597 >>> Pixel(11, 22, 128, 255, 0)
Christian Heimes454f37b2008-01-10 00:10:02 +0000598 Pixel(x=11, y=22, red=128, green=255, blue=0)
Georg Brandl116aa622007-08-15 14:28:22 +0000599
Christian Heimes0449f632007-12-15 01:27:15 +0000600To retrieve a field whose name is stored in a string, use the :func:`getattr`
Christian Heimesfe337bf2008-03-23 21:54:12 +0000601function:
Christian Heimes0449f632007-12-15 01:27:15 +0000602
603 >>> getattr(p, 'x')
604 11
605
Christian Heimesfe337bf2008-03-23 21:54:12 +0000606To convert a dictionary to a named tuple, use the double-star-operator [#]_:
Christian Heimes99170a52007-12-19 02:07:34 +0000607
608 >>> d = {'x': 11, 'y': 22}
609 >>> Point(**d)
610 Point(x=11, y=22)
611
Guido van Rossum3d392eb2007-11-16 00:35:22 +0000612Since a named tuple is a regular Python class, it is easy to add or change
Christian Heimes043d6f62008-01-07 17:19:16 +0000613functionality with a subclass. Here is how to add a calculated field and
Christian Heimesfe337bf2008-03-23 21:54:12 +0000614a fixed-width print format:
Guido van Rossum3d392eb2007-11-16 00:35:22 +0000615
Christian Heimes043d6f62008-01-07 17:19:16 +0000616 >>> class Point(namedtuple('Point', 'x y')):
Christian Heimes25bb7832008-01-11 16:17:00 +0000617 ... __slots__ = ()
Christian Heimes454f37b2008-01-10 00:10:02 +0000618 ... @property
619 ... def hypot(self):
620 ... return (self.x ** 2 + self.y ** 2) ** 0.5
621 ... def __str__(self):
Christian Heimes25bb7832008-01-11 16:17:00 +0000622 ... return 'Point: x=%6.3f y=%6.3f hypot=%6.3f' % (self.x, self.y, self.hypot)
Guido van Rossum3d392eb2007-11-16 00:35:22 +0000623
Christian Heimes25bb7832008-01-11 16:17:00 +0000624 >>> for p in Point(3, 4), Point(14, 5/7.):
Christian Heimes00412232008-01-10 16:02:19 +0000625 ... print(p)
Christian Heimes25bb7832008-01-11 16:17:00 +0000626 Point: x= 3.000 y= 4.000 hypot= 5.000
627 Point: x=14.000 y= 0.714 hypot=14.018
Christian Heimes043d6f62008-01-07 17:19:16 +0000628
Christian Heimesaf98da12008-01-27 15:18:18 +0000629The subclass shown above sets ``__slots__`` to an empty tuple. This keeps
Christian Heimes679db4a2008-01-18 09:56:22 +0000630keep memory requirements low by preventing the creation of instance dictionaries.
631
Christian Heimes2380ac72008-01-09 00:17:24 +0000632
633Subclassing is not useful for adding new, stored fields. Instead, simply
Christian Heimesfe337bf2008-03-23 21:54:12 +0000634create a new named tuple type from the :attr:`_fields` attribute:
Christian Heimes2380ac72008-01-09 00:17:24 +0000635
Christian Heimes25bb7832008-01-11 16:17:00 +0000636 >>> Point3D = namedtuple('Point3D', Point._fields + ('z',))
Christian Heimes2380ac72008-01-09 00:17:24 +0000637
638Default values can be implemented by using :meth:`_replace` to
Christian Heimesfe337bf2008-03-23 21:54:12 +0000639customize a prototype instance:
Guido van Rossum3d392eb2007-11-16 00:35:22 +0000640
641 >>> Account = namedtuple('Account', 'owner balance transaction_count')
Christian Heimes587c2bf2008-01-19 16:21:02 +0000642 >>> default_account = Account('<owner name>', 0.0, 0)
643 >>> johns_account = default_account._replace(owner='John')
Guido van Rossum3d392eb2007-11-16 00:35:22 +0000644
Thomas Wouters47b49bf2007-08-30 22:15:33 +0000645.. rubric:: Footnotes
646
Christian Heimes99170a52007-12-19 02:07:34 +0000647.. [#] For information on the double-star-operator see
Thomas Wouters47b49bf2007-08-30 22:15:33 +0000648 :ref:`tut-unpacking-arguments` and :ref:`calls`.
Raymond Hettingere4c96ad2008-02-06 01:23:58 +0000649
650
651
652:class:`UserDict` objects
Mark Summerfield8f2d0062008-02-06 13:30:44 +0000653-------------------------
Raymond Hettingere4c96ad2008-02-06 01:23:58 +0000654
655The class, :class:`UserDict` acts as a wrapper around dictionary objects.
656The need for this class has been partially supplanted by the ability to
657subclass directly from :class:`dict`; however, this class can be easier
658to work with because the underlying dictionary is accessible as an
659attribute.
660
661.. class:: UserDict([initialdata])
662
663 Class that simulates a dictionary. The instance's contents are kept in a
664 regular dictionary, which is accessible via the :attr:`data` attribute of
665 :class:`UserDict` instances. If *initialdata* is provided, :attr:`data` is
666 initialized with its contents; note that a reference to *initialdata* will not
667 be kept, allowing it be used for other purposes.
668
669In addition to supporting the methods and operations of mappings,
Raymond Hettingerebcee3f2008-02-06 19:54:00 +0000670:class:`UserDict` instances provide the following attribute:
Raymond Hettingere4c96ad2008-02-06 01:23:58 +0000671
672.. attribute:: UserDict.data
673
674 A real dictionary used to store the contents of the :class:`UserDict` class.
Raymond Hettinger53dbe392008-02-12 20:03:09 +0000675
676
677
678:class:`UserList` objects
679-------------------------
680
681This class acts as a wrapper around list objects. It is a useful base class
682for your own list-like classes which can inherit from them and override
683existing methods or add new ones. In this way, one can add new behaviors to
684lists.
685
686The need for this class has been partially supplanted by the ability to
687subclass directly from :class:`list`; however, this class can be easier
688to work with because the underlying list is accessible as an attribute.
689
690.. class:: UserList([list])
691
692 Class that simulates a list. The instance's contents are kept in a regular
693 list, which is accessible via the :attr:`data` attribute of :class:`UserList`
694 instances. The instance's contents are initially set to a copy of *list*,
695 defaulting to the empty list ``[]``. *list* can be any iterable, for
696 example a real Python list or a :class:`UserList` object.
697
698In addition to supporting the methods and operations of mutable sequences,
699:class:`UserList` instances provide the following attribute:
700
701.. attribute:: UserList.data
702
703 A real :class:`list` object used to store the contents of the
704 :class:`UserList` class.
705
706**Subclassing requirements:** Subclasses of :class:`UserList` are expect to
707offer a constructor which can be called with either no arguments or one
708argument. List operations which return a new sequence attempt to create an
709instance of the actual implementation class. To do so, it assumes that the
710constructor can be called with a single parameter, which is a sequence object
711used as a data source.
712
713If a derived class does not wish to comply with this requirement, all of the
714special methods supported by this class will need to be overridden; please
715consult the sources for information about the methods which need to be provided
716in that case.
Raymond Hettingerb3a65f82008-02-21 22:11:37 +0000717
718:class:`UserString` objects
Christian Heimesc3f30c42008-02-22 16:37:40 +0000719---------------------------
Raymond Hettingerb3a65f82008-02-21 22:11:37 +0000720
721The class, :class:`UserString` acts as a wrapper around string objects.
722The need for this class has been partially supplanted by the ability to
723subclass directly from :class:`str`; however, this class can be easier
724to work with because the underlying string is accessible as an
725attribute.
726
727.. class:: UserString([sequence])
728
729 Class that simulates a string or a Unicode string object. The instance's
730 content is kept in a regular string object, which is accessible via the
731 :attr:`data` attribute of :class:`UserString` instances. The instance's
732 contents are initially set to a copy of *sequence*. The *sequence* can
733 be an instance of :class:`bytes`, :class:`str`, :class:`UserString` (or a
734 subclass) or an arbitrary sequence which can be converted into a string using
735 the built-in :func:`str` function.