blob: 1d6687d4de28f2d412758cb20401ff1757426cff [file] [log] [blame]
Georg Brandl116aa622007-08-15 14:28:22 +00001
Raymond Hettinger53dbe392008-02-12 20:03:09 +00002:mod:`collections` --- Container datatypes
3==========================================
Georg Brandl116aa622007-08-15 14:28:22 +00004
5.. module:: collections
Raymond Hettinger53dbe392008-02-12 20:03:09 +00006 :synopsis: Container datatypes
Georg Brandl116aa622007-08-15 14:28:22 +00007.. moduleauthor:: Raymond Hettinger <python@rcn.com>
8.. sectionauthor:: Raymond Hettinger <python@rcn.com>
9
10
Georg Brandl116aa622007-08-15 14:28:22 +000011This module implements high-performance container datatypes. Currently,
12there are two datatypes, :class:`deque` and :class:`defaultdict`, and
Mark Summerfield71316b02008-02-14 16:28:00 +000013one datatype factory function, :func:`namedtuple`. This module also
14provides the :class:`UserDict` and :class:`UserList` classes which may
15be useful when inheriting directly from :class:`dict` or
16:class:`list` isn't convenient.
Christian Heimes0bd4e112008-02-12 22:59:25 +000017
Raymond Hettingerebcee3f2008-02-06 19:54:00 +000018The specialized containers provided in this module provide alternatives
19to Python's general purpose built-in containers, :class:`dict`,
20:class:`list`, :class:`set`, and :class:`tuple`.
Raymond Hettingerebcee3f2008-02-06 19:54:00 +000021Besides the containers provided here, the optional :mod:`bsddb`
22module offers the ability to create in-memory or file based ordered
23dictionaries with string keys using the :meth:`bsddb.btopen` method.
Georg Brandl116aa622007-08-15 14:28:22 +000024
Mark Summerfield08898b42007-09-05 08:43:04 +000025In addition to containers, the collections module provides some ABCs
Mark Summerfield71316b02008-02-14 16:28:00 +000026(abstract base classes). These can be used to test whether a class
Raymond Hettingerebcee3f2008-02-06 19:54:00 +000027provides a particular interface, for example, is it hashable or
Mark Summerfield71316b02008-02-14 16:28:00 +000028a mapping, and some of them can also be used as mixin classes.
Raymond Hettingerebcee3f2008-02-06 19:54:00 +000029
30ABCs - abstract base classes
31----------------------------
32
33The collections module offers the following ABCs:
Mark Summerfield08898b42007-09-05 08:43:04 +000034
Raymond Hettinger409fb2c2008-02-09 02:17:06 +000035========================= ==================== ====================== ====================================================
36ABC Inherits Abstract Methods Mixin Methods
37========================= ==================== ====================== ====================================================
38:class:`Container` ``__contains__``
39:class:`Hashable` ``__hash__``
40:class:`Iterable` ``__iter__``
41:class:`Iterator` :class:`Iterable` ``__next__`` ``__iter__``
42:class:`Sized` ``__len__``
43
44:class:`Mapping` :class:`Sized`, ``__getitem__``, ``__contains__``, ``keys``, ``items``, ``values``,
45 :class:`Iterable`, ``__len__``. and ``get``, ``__eq__``, and ``__ne__``
46 :class:`Container` ``__iter__``
47
48:class:`MutableMapping` :class:`Mapping` ``__getitem__`` Inherited Mapping methods and
49 ``__setitem__``, ``pop``, ``popitem``, ``clear``, ``update``,
50 ``__delitem__``, and ``setdefault``
51 ``__iter__``, and
52 ``__len__``
53
54:class:`Sequence` :class:`Sized`, ``__getitem__`` ``__contains__``. ``__iter__``, ``__reversed__``.
55 :class:`Iterable`, and ``__len__`` ``index``, and ``count``
56 :class:`Container`
57
58:class:`MutableSequnce` :class:`Sequence` ``__getitem__`` Inherited Sequence methods and
59 ``__delitem__``, ``append``, ``reverse``, ``extend``, ``pop``,
60 ``insert``, ``remove``, and ``__iadd__``
61 and ``__len__``
62
Raymond Hettinger0dbdab22008-02-09 03:48:16 +000063:class:`Set` :class:`Sized`, ``__len__``, ``__le__``, ``__lt__``, ``__eq__``, ``__ne__``,
Raymond Hettinger409fb2c2008-02-09 02:17:06 +000064 :class:`Iterable`, ``__iter__``, and ``__gt__``, ``__ge__``, ``__and__``, ``__or__``
65 :class:`Container` ``__contains__`` ``__sub__``, ``__xor__``, and ``isdisjoint``
66
67:class:`MutableSet` :class:`Set` ``add`` and Inherited Set methods and
68 ``discard`` ``clear``, ``pop``, ``remove``, ``__ior__``,
69 ``__iand__``, ``__ixor__``, and ``__isub__``
70========================= ==================== ====================== ====================================================
Mark Summerfield08898b42007-09-05 08:43:04 +000071
Mark Summerfield08898b42007-09-05 08:43:04 +000072These ABCs allow us to ask classes or instances if they provide
73particular functionality, for example::
74
Mark Summerfield08898b42007-09-05 08:43:04 +000075 size = None
Raymond Hettingerebcee3f2008-02-06 19:54:00 +000076 if isinstance(myvar, collections.Sized):
Mark Summerfield08898b42007-09-05 08:43:04 +000077 size = len(myvar)
78
Raymond Hettingerebcee3f2008-02-06 19:54:00 +000079Several of the ABCs are also useful as mixins that make it easier to develop
80classes supporting container APIs. For example, to write a class supporting
81the full :class:`Set` API, it only necessary to supply the three underlying
82abstract methods: :meth:`__contains__`, :meth:`__iter__`, and :meth:`__len__`.
83The ABC supplies the remaining methods such as :meth:`__and__` and
84:meth:`isdisjoint` ::
85
86 class ListBasedSet(collections.Set):
Raymond Hettingerc1b6a4a2008-02-08 23:46:23 +000087 ''' Alternate set implementation favoring space over speed
88 and not requiring the set elements to be hashable. '''
Raymond Hettingerebcee3f2008-02-06 19:54:00 +000089 def __init__(self, iterable):
Raymond Hettingerc1b6a4a2008-02-08 23:46:23 +000090 self.elements = lst = []
91 for value in iterable:
92 if value not in lst:
93 lst.append(value)
Raymond Hettingerebcee3f2008-02-06 19:54:00 +000094 def __iter__(self):
95 return iter(self.elements)
96 def __contains__(self, value):
97 return value in self.elements
98 def __len__(self):
99 return len(self.elements)
100
101 s1 = ListBasedSet('abcdef')
102 s2 = ListBasedSet('defghi')
103 overlap = s1 & s2 # The __and__() method is supported automatically
104
Raymond Hettinger7aebb642008-02-09 03:25:08 +0000105Notes on using :class:`Set` and :class:`MutableSet` as a mixin:
106
107(1)
108 Since some set operations create new sets, the default mixin methods need
109 a way to create new instances from an iterable. The class constructor is
110 assumed to have a signature in the form ``ClassName(iterable)``.
Mark Summerfield71316b02008-02-14 16:28:00 +0000111 That assumption is factored-out to a single internal classmethod called
Raymond Hettinger7aebb642008-02-09 03:25:08 +0000112 :meth:`_from_iterable` which calls ``cls(iterable)`` to produce a new set.
113 If the :class:`Set` mixin is being used in a class with a different
114 constructor signature, you will need to override :meth:`from_iterable`
115 with a classmethod that can construct new instances from
116 an iterable argument.
117
118(2)
119 To override the comparisons (presumably for speed, as the
120 semantics are fixed), redefine :meth:`__le__` and
121 then the other operations will automatically follow suit.
Raymond Hettingerebcee3f2008-02-06 19:54:00 +0000122
Raymond Hettinger0dbdab22008-02-09 03:48:16 +0000123(3)
124 The :class:`Set` mixin provides a :meth:`_hash` method to compute a hash value
125 for the set; however, :meth:`__hash__` is not defined because not all sets
126 are hashable or immutable. To add set hashabilty using mixins,
127 inherit from both :meth:`Set` and :meth:`Hashable`, then define
128 ``__hash__ = Set._hash``.
129
Mark Summerfield08898b42007-09-05 08:43:04 +0000130(For more about ABCs, see the :mod:`abc` module and :pep:`3119`.)
131
132
Georg Brandl116aa622007-08-15 14:28:22 +0000133.. _deque-objects:
134
135:class:`deque` objects
136----------------------
137
138
Georg Brandl9afde1c2007-11-01 20:32:30 +0000139.. class:: deque([iterable[, maxlen]])
Georg Brandl116aa622007-08-15 14:28:22 +0000140
141 Returns a new deque object initialized left-to-right (using :meth:`append`) with
142 data from *iterable*. If *iterable* is not specified, the new deque is empty.
143
144 Deques are a generalization of stacks and queues (the name is pronounced "deck"
145 and is short for "double-ended queue"). Deques support thread-safe, memory
146 efficient appends and pops from either side of the deque with approximately the
147 same O(1) performance in either direction.
148
149 Though :class:`list` objects support similar operations, they are optimized for
150 fast fixed-length operations and incur O(n) memory movement costs for
151 ``pop(0)`` and ``insert(0, v)`` operations which change both the size and
152 position of the underlying data representation.
153
Georg Brandl116aa622007-08-15 14:28:22 +0000154
Georg Brandl9afde1c2007-11-01 20:32:30 +0000155 If *maxlen* is not specified or is *None*, deques may grow to an
156 arbitrary length. Otherwise, the deque is bounded to the specified maximum
157 length. Once a bounded length deque is full, when new items are added, a
158 corresponding number of items are discarded from the opposite end. Bounded
159 length deques provide functionality similar to the ``tail`` filter in
160 Unix. They are also useful for tracking transactions and other pools of data
161 where only the most recent activity is of interest.
162
Georg Brandl9afde1c2007-11-01 20:32:30 +0000163
Georg Brandl116aa622007-08-15 14:28:22 +0000164Deque objects support the following methods:
165
Georg Brandl116aa622007-08-15 14:28:22 +0000166.. method:: deque.append(x)
167
168 Add *x* to the right side of the deque.
169
170
171.. method:: deque.appendleft(x)
172
173 Add *x* to the left side of the deque.
174
175
176.. method:: deque.clear()
177
178 Remove all elements from the deque leaving it with length 0.
179
180
181.. method:: deque.extend(iterable)
182
183 Extend the right side of the deque by appending elements from the iterable
184 argument.
185
186
187.. method:: deque.extendleft(iterable)
188
189 Extend the left side of the deque by appending elements from *iterable*. Note,
190 the series of left appends results in reversing the order of elements in the
191 iterable argument.
192
193
194.. method:: deque.pop()
195
196 Remove and return an element from the right side of the deque. If no elements
197 are present, raises an :exc:`IndexError`.
198
199
200.. method:: deque.popleft()
201
202 Remove and return an element from the left side of the deque. If no elements are
203 present, raises an :exc:`IndexError`.
204
205
206.. method:: deque.remove(value)
207
208 Removed the first occurrence of *value*. If not found, raises a
209 :exc:`ValueError`.
210
Georg Brandl116aa622007-08-15 14:28:22 +0000211
212.. method:: deque.rotate(n)
213
214 Rotate the deque *n* steps to the right. If *n* is negative, rotate to the
215 left. Rotating one step to the right is equivalent to:
216 ``d.appendleft(d.pop())``.
217
218In addition to the above, deques support iteration, pickling, ``len(d)``,
219``reversed(d)``, ``copy.copy(d)``, ``copy.deepcopy(d)``, membership testing with
220the :keyword:`in` operator, and subscript references such as ``d[-1]``.
221
222Example::
223
224 >>> from collections import deque
225 >>> d = deque('ghi') # make a new deque with three items
226 >>> for elem in d: # iterate over the deque's elements
Georg Brandl6911e3c2007-09-04 07:15:32 +0000227 ... print(elem.upper())
Georg Brandl116aa622007-08-15 14:28:22 +0000228 G
229 H
230 I
231
232 >>> d.append('j') # add a new entry to the right side
233 >>> d.appendleft('f') # add a new entry to the left side
234 >>> d # show the representation of the deque
235 deque(['f', 'g', 'h', 'i', 'j'])
236
237 >>> d.pop() # return and remove the rightmost item
238 'j'
239 >>> d.popleft() # return and remove the leftmost item
240 'f'
241 >>> list(d) # list the contents of the deque
242 ['g', 'h', 'i']
243 >>> d[0] # peek at leftmost item
244 'g'
245 >>> d[-1] # peek at rightmost item
246 'i'
247
248 >>> list(reversed(d)) # list the contents of a deque in reverse
249 ['i', 'h', 'g']
250 >>> 'h' in d # search the deque
251 True
252 >>> d.extend('jkl') # add multiple elements at once
253 >>> d
254 deque(['g', 'h', 'i', 'j', 'k', 'l'])
255 >>> d.rotate(1) # right rotation
256 >>> d
257 deque(['l', 'g', 'h', 'i', 'j', 'k'])
258 >>> d.rotate(-1) # left rotation
259 >>> d
260 deque(['g', 'h', 'i', 'j', 'k', 'l'])
261
262 >>> deque(reversed(d)) # make a new deque in reverse order
263 deque(['l', 'k', 'j', 'i', 'h', 'g'])
264 >>> d.clear() # empty the deque
265 >>> d.pop() # cannot pop from an empty deque
266 Traceback (most recent call last):
267 File "<pyshell#6>", line 1, in -toplevel-
268 d.pop()
269 IndexError: pop from an empty deque
270
271 >>> d.extendleft('abc') # extendleft() reverses the input order
272 >>> d
273 deque(['c', 'b', 'a'])
274
275
276.. _deque-recipes:
277
Georg Brandl9afde1c2007-11-01 20:32:30 +0000278:class:`deque` Recipes
279^^^^^^^^^^^^^^^^^^^^^^
Georg Brandl116aa622007-08-15 14:28:22 +0000280
281This section shows various approaches to working with deques.
282
283The :meth:`rotate` method provides a way to implement :class:`deque` slicing and
284deletion. For example, a pure python implementation of ``del d[n]`` relies on
285the :meth:`rotate` method to position elements to be popped::
286
287 def delete_nth(d, n):
288 d.rotate(-n)
289 d.popleft()
290 d.rotate(n)
291
292To implement :class:`deque` slicing, use a similar approach applying
293:meth:`rotate` to bring a target element to the left side of the deque. Remove
294old entries with :meth:`popleft`, add new entries with :meth:`extend`, and then
295reverse the rotation.
Georg Brandl116aa622007-08-15 14:28:22 +0000296With minor variations on that approach, it is easy to implement Forth style
297stack manipulations such as ``dup``, ``drop``, ``swap``, ``over``, ``pick``,
298``rot``, and ``roll``.
299
Georg Brandl116aa622007-08-15 14:28:22 +0000300Multi-pass data reduction algorithms can be succinctly expressed and efficiently
301coded by extracting elements with multiple calls to :meth:`popleft`, applying
Georg Brandl9afde1c2007-11-01 20:32:30 +0000302a reduction function, and calling :meth:`append` to add the result back to the
303deque.
Georg Brandl116aa622007-08-15 14:28:22 +0000304
305For example, building a balanced binary tree of nested lists entails reducing
306two adjacent nodes into one by grouping them in a list::
307
308 >>> def maketree(iterable):
309 ... d = deque(iterable)
310 ... while len(d) > 1:
311 ... pair = [d.popleft(), d.popleft()]
312 ... d.append(pair)
313 ... return list(d)
314 ...
Georg Brandl6911e3c2007-09-04 07:15:32 +0000315 >>> print(maketree('abcdefgh'))
Georg Brandl116aa622007-08-15 14:28:22 +0000316 [[[['a', 'b'], ['c', 'd']], [['e', 'f'], ['g', 'h']]]]
317
Georg Brandl9afde1c2007-11-01 20:32:30 +0000318Bounded length deques provide functionality similar to the ``tail`` filter
319in Unix::
Georg Brandl116aa622007-08-15 14:28:22 +0000320
Georg Brandl9afde1c2007-11-01 20:32:30 +0000321 def tail(filename, n=10):
322 'Return the last n lines of a file'
323 return deque(open(filename), n)
Georg Brandl116aa622007-08-15 14:28:22 +0000324
325.. _defaultdict-objects:
326
327:class:`defaultdict` objects
328----------------------------
329
330
331.. class:: defaultdict([default_factory[, ...]])
332
333 Returns a new dictionary-like object. :class:`defaultdict` is a subclass of the
334 builtin :class:`dict` class. It overrides one method and adds one writable
335 instance variable. The remaining functionality is the same as for the
336 :class:`dict` class and is not documented here.
337
338 The first argument provides the initial value for the :attr:`default_factory`
339 attribute; it defaults to ``None``. All remaining arguments are treated the same
340 as if they were passed to the :class:`dict` constructor, including keyword
341 arguments.
342
Georg Brandl116aa622007-08-15 14:28:22 +0000343
344:class:`defaultdict` objects support the following method in addition to the
345standard :class:`dict` operations:
346
Georg Brandl116aa622007-08-15 14:28:22 +0000347.. method:: defaultdict.__missing__(key)
348
349 If the :attr:`default_factory` attribute is ``None``, this raises an
350 :exc:`KeyError` exception with the *key* as argument.
351
352 If :attr:`default_factory` is not ``None``, it is called without arguments to
353 provide a default value for the given *key*, this value is inserted in the
354 dictionary for the *key*, and returned.
355
356 If calling :attr:`default_factory` raises an exception this exception is
357 propagated unchanged.
358
359 This method is called by the :meth:`__getitem__` method of the :class:`dict`
360 class when the requested key is not found; whatever it returns or raises is then
361 returned or raised by :meth:`__getitem__`.
362
363:class:`defaultdict` objects support the following instance variable:
364
365
366.. attribute:: defaultdict.default_factory
367
368 This attribute is used by the :meth:`__missing__` method; it is initialized from
369 the first argument to the constructor, if present, or to ``None``, if absent.
370
371
372.. _defaultdict-examples:
373
374:class:`defaultdict` Examples
375^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
376
377Using :class:`list` as the :attr:`default_factory`, it is easy to group a
378sequence of key-value pairs into a dictionary of lists::
379
380 >>> s = [('yellow', 1), ('blue', 2), ('yellow', 3), ('blue', 4), ('red', 1)]
381 >>> d = defaultdict(list)
382 >>> for k, v in s:
383 ... d[k].append(v)
384 ...
385 >>> d.items()
386 [('blue', [2, 4]), ('red', [1]), ('yellow', [1, 3])]
387
388When each key is encountered for the first time, it is not already in the
389mapping; so an entry is automatically created using the :attr:`default_factory`
390function which returns an empty :class:`list`. The :meth:`list.append`
391operation then attaches the value to the new list. When keys are encountered
392again, the look-up proceeds normally (returning the list for that key) and the
393:meth:`list.append` operation adds another value to the list. This technique is
394simpler and faster than an equivalent technique using :meth:`dict.setdefault`::
395
396 >>> d = {}
397 >>> for k, v in s:
398 ... d.setdefault(k, []).append(v)
399 ...
400 >>> d.items()
401 [('blue', [2, 4]), ('red', [1]), ('yellow', [1, 3])]
402
403Setting the :attr:`default_factory` to :class:`int` makes the
404:class:`defaultdict` useful for counting (like a bag or multiset in other
405languages)::
406
407 >>> s = 'mississippi'
408 >>> d = defaultdict(int)
409 >>> for k in s:
410 ... d[k] += 1
411 ...
412 >>> d.items()
413 [('i', 4), ('p', 2), ('s', 4), ('m', 1)]
414
415When a letter is first encountered, it is missing from the mapping, so the
416:attr:`default_factory` function calls :func:`int` to supply a default count of
417zero. The increment operation then builds up the count for each letter.
418
419The function :func:`int` which always returns zero is just a special case of
420constant functions. A faster and more flexible way to create constant functions
421is to use a lambda function which can supply any constant value (not just
422zero)::
423
424 >>> def constant_factory(value):
425 ... return lambda: value
426 >>> d = defaultdict(constant_factory('<missing>'))
427 >>> d.update(name='John', action='ran')
428 >>> '%(name)s %(action)s to %(object)s' % d
429 'John ran to <missing>'
430
431Setting the :attr:`default_factory` to :class:`set` makes the
432:class:`defaultdict` useful for building a dictionary of sets::
433
434 >>> s = [('red', 1), ('blue', 2), ('red', 3), ('blue', 4), ('red', 1), ('blue', 4)]
435 >>> d = defaultdict(set)
436 >>> for k, v in s:
437 ... d[k].add(v)
438 ...
439 >>> d.items()
440 [('blue', set([2, 4])), ('red', set([1, 3]))]
441
442
443.. _named-tuple-factory:
444
Guido van Rossum3d392eb2007-11-16 00:35:22 +0000445:func:`namedtuple` Factory Function for Tuples with Named Fields
Christian Heimes790c8232008-01-07 21:14:23 +0000446----------------------------------------------------------------
Georg Brandl116aa622007-08-15 14:28:22 +0000447
Thomas Wouters1b7f8912007-09-19 03:06:30 +0000448Named tuples assign meaning to each position in a tuple and allow for more readable,
449self-documenting code. They can be used wherever regular tuples are used, and
450they add the ability to access fields by name instead of position index.
Georg Brandl116aa622007-08-15 14:28:22 +0000451
Guido van Rossum3d392eb2007-11-16 00:35:22 +0000452.. function:: namedtuple(typename, fieldnames, [verbose])
Georg Brandl116aa622007-08-15 14:28:22 +0000453
454 Returns a new tuple subclass named *typename*. The new subclass is used to
455 create tuple-like objects that have fields accessable by attribute lookup as
456 well as being indexable and iterable. Instances of the subclass also have a
457 helpful docstring (with typename and fieldnames) and a helpful :meth:`__repr__`
458 method which lists the tuple contents in a ``name=value`` format.
459
Georg Brandl9afde1c2007-11-01 20:32:30 +0000460 The *fieldnames* are a single string with each fieldname separated by whitespace
Christian Heimes25bb7832008-01-11 16:17:00 +0000461 and/or commas, for example ``'x y'`` or ``'x, y'``. Alternatively, *fieldnames*
462 can be a sequence of strings such as ``['x', 'y']``.
Georg Brandl9afde1c2007-11-01 20:32:30 +0000463
464 Any valid Python identifier may be used for a fieldname except for names
Christian Heimes0449f632007-12-15 01:27:15 +0000465 starting with an underscore. Valid identifiers consist of letters, digits,
466 and underscores but do not start with a digit or underscore and cannot be
Georg Brandlf6945182008-02-01 11:56:49 +0000467 a :mod:`keyword` such as *class*, *for*, *return*, *global*, *pass*,
Georg Brandl9afde1c2007-11-01 20:32:30 +0000468 or *raise*.
Georg Brandl116aa622007-08-15 14:28:22 +0000469
Christian Heimes25bb7832008-01-11 16:17:00 +0000470 If *verbose* is true, the class definition is printed just before being built.
Georg Brandl116aa622007-08-15 14:28:22 +0000471
Georg Brandl9afde1c2007-11-01 20:32:30 +0000472 Named tuple instances do not have per-instance dictionaries, so they are
Thomas Wouters8ce81f72007-09-20 18:22:40 +0000473 lightweight and require no more memory than regular tuples.
Georg Brandl116aa622007-08-15 14:28:22 +0000474
Thomas Wouters1b7f8912007-09-19 03:06:30 +0000475Example::
Georg Brandl116aa622007-08-15 14:28:22 +0000476
Guido van Rossum3d392eb2007-11-16 00:35:22 +0000477 >>> Point = namedtuple('Point', 'x y', verbose=True)
Thomas Wouters1b7f8912007-09-19 03:06:30 +0000478 class Point(tuple):
479 'Point(x, y)'
Christian Heimes0449f632007-12-15 01:27:15 +0000480
Thomas Wouters1b7f8912007-09-19 03:06:30 +0000481 __slots__ = ()
Christian Heimes0449f632007-12-15 01:27:15 +0000482
Christian Heimesfaf2f632008-01-06 16:59:19 +0000483 _fields = ('x', 'y')
484
Thomas Wouters1b7f8912007-09-19 03:06:30 +0000485 def __new__(cls, x, y):
486 return tuple.__new__(cls, (x, y))
Christian Heimes0449f632007-12-15 01:27:15 +0000487
Christian Heimesfaf2f632008-01-06 16:59:19 +0000488 @classmethod
489 def _make(cls, iterable):
490 'Make a new Point object from a sequence or iterable'
491 result = tuple.__new__(cls, iterable)
492 if len(result) != 2:
493 raise TypeError('Expected 2 arguments, got %d' % len(result))
494 return result
Christian Heimes99170a52007-12-19 02:07:34 +0000495
Thomas Wouters1b7f8912007-09-19 03:06:30 +0000496 def __repr__(self):
497 return 'Point(x=%r, y=%r)' % self
Christian Heimes0449f632007-12-15 01:27:15 +0000498
Christian Heimes99170a52007-12-19 02:07:34 +0000499 def _asdict(t):
Christian Heimes0449f632007-12-15 01:27:15 +0000500 'Return a new dict which maps field names to their values'
Christian Heimes99170a52007-12-19 02:07:34 +0000501 return {'x': t[0], 'y': t[1]}
Christian Heimes0449f632007-12-15 01:27:15 +0000502
503 def _replace(self, **kwds):
Guido van Rossum3d392eb2007-11-16 00:35:22 +0000504 'Return a new Point object replacing specified fields with new values'
Christian Heimesfaf2f632008-01-06 16:59:19 +0000505 result = self._make(map(kwds.pop, ('x', 'y'), self))
506 if kwds:
507 raise ValueError('Got unexpected field names: %r' % kwds.keys())
508 return result
Christian Heimes0449f632007-12-15 01:27:15 +0000509
Thomas Wouters1b7f8912007-09-19 03:06:30 +0000510 x = property(itemgetter(0))
511 y = property(itemgetter(1))
Georg Brandl116aa622007-08-15 14:28:22 +0000512
Thomas Wouters1b7f8912007-09-19 03:06:30 +0000513 >>> p = Point(11, y=22) # instantiate with positional or keyword arguments
Christian Heimes99170a52007-12-19 02:07:34 +0000514 >>> p[0] + p[1] # indexable like the plain tuple (11, 22)
Thomas Wouters1b7f8912007-09-19 03:06:30 +0000515 33
516 >>> x, y = p # unpack like a regular tuple
517 >>> x, y
518 (11, 22)
519 >>> p.x + p.y # fields also accessable by name
520 33
521 >>> p # readable __repr__ with a name=value style
522 Point(x=11, y=22)
Georg Brandl116aa622007-08-15 14:28:22 +0000523
Thomas Wouters1b7f8912007-09-19 03:06:30 +0000524Named tuples are especially useful for assigning field names to result tuples returned
525by the :mod:`csv` or :mod:`sqlite3` modules::
526
Guido van Rossum3d392eb2007-11-16 00:35:22 +0000527 EmployeeRecord = namedtuple('EmployeeRecord', 'name, age, title, department, paygrade')
Georg Brandl9afde1c2007-11-01 20:32:30 +0000528
Thomas Wouters1b7f8912007-09-19 03:06:30 +0000529 import csv
Christian Heimesfaf2f632008-01-06 16:59:19 +0000530 for emp in map(EmployeeRecord._make, csv.reader(open("employees.csv", "rb"))):
Thomas Wouters1b7f8912007-09-19 03:06:30 +0000531 print(emp.name, emp.title)
532
Georg Brandl9afde1c2007-11-01 20:32:30 +0000533 import sqlite3
534 conn = sqlite3.connect('/companydata')
535 cursor = conn.cursor()
536 cursor.execute('SELECT name, age, title, department, paygrade FROM employees')
Christian Heimesfaf2f632008-01-06 16:59:19 +0000537 for emp in map(EmployeeRecord._make, cursor.fetchall()):
Christian Heimes00412232008-01-10 16:02:19 +0000538 print(emp.name, emp.title)
Georg Brandl9afde1c2007-11-01 20:32:30 +0000539
Christian Heimes99170a52007-12-19 02:07:34 +0000540In addition to the methods inherited from tuples, named tuples support
Christian Heimes2380ac72008-01-09 00:17:24 +0000541three additional methods and one attribute. To prevent conflicts with
542field names, the method and attribute names start with an underscore.
Christian Heimes99170a52007-12-19 02:07:34 +0000543
Christian Heimes790c8232008-01-07 21:14:23 +0000544.. method:: somenamedtuple._make(iterable)
Christian Heimes99170a52007-12-19 02:07:34 +0000545
Christian Heimesfaf2f632008-01-06 16:59:19 +0000546 Class method that makes a new instance from an existing sequence or iterable.
Christian Heimes99170a52007-12-19 02:07:34 +0000547
548::
Thomas Wouters1b7f8912007-09-19 03:06:30 +0000549
Christian Heimesfaf2f632008-01-06 16:59:19 +0000550 >>> t = [11, 22]
551 >>> Point._make(t)
552 Point(x=11, y=22)
Thomas Wouters1b7f8912007-09-19 03:06:30 +0000553
Christian Heimes790c8232008-01-07 21:14:23 +0000554.. method:: somenamedtuple._asdict()
Georg Brandl9afde1c2007-11-01 20:32:30 +0000555
556 Return a new dict which maps field names to their corresponding values:
557
558::
559
Christian Heimes0449f632007-12-15 01:27:15 +0000560 >>> p._asdict()
Georg Brandl9afde1c2007-11-01 20:32:30 +0000561 {'x': 11, 'y': 22}
562
Christian Heimes790c8232008-01-07 21:14:23 +0000563.. method:: somenamedtuple._replace(kwargs)
Thomas Wouters1b7f8912007-09-19 03:06:30 +0000564
Guido van Rossum3d392eb2007-11-16 00:35:22 +0000565 Return a new instance of the named tuple replacing specified fields with new values:
Thomas Wouters8ce81f72007-09-20 18:22:40 +0000566
567::
Thomas Wouters1b7f8912007-09-19 03:06:30 +0000568
569 >>> p = Point(x=11, y=22)
Christian Heimes0449f632007-12-15 01:27:15 +0000570 >>> p._replace(x=33)
Thomas Wouters1b7f8912007-09-19 03:06:30 +0000571 Point(x=33, y=22)
572
Guido van Rossum3d392eb2007-11-16 00:35:22 +0000573 >>> for partnum, record in inventory.items():
Christian Heimes454f37b2008-01-10 00:10:02 +0000574 ... inventory[partnum] = record._replace(price=newprices[partnum], timestamp=time.now())
Thomas Wouters1b7f8912007-09-19 03:06:30 +0000575
Christian Heimes790c8232008-01-07 21:14:23 +0000576.. attribute:: somenamedtuple._fields
Thomas Wouters1b7f8912007-09-19 03:06:30 +0000577
Christian Heimes2380ac72008-01-09 00:17:24 +0000578 Tuple of strings listing the field names. Useful for introspection
Georg Brandl9afde1c2007-11-01 20:32:30 +0000579 and for creating new named tuple types from existing named tuples.
Thomas Wouters8ce81f72007-09-20 18:22:40 +0000580
581::
Thomas Wouters1b7f8912007-09-19 03:06:30 +0000582
Christian Heimes0449f632007-12-15 01:27:15 +0000583 >>> p._fields # view the field names
Thomas Wouters1b7f8912007-09-19 03:06:30 +0000584 ('x', 'y')
Thomas Wouters1b7f8912007-09-19 03:06:30 +0000585
Guido van Rossum3d392eb2007-11-16 00:35:22 +0000586 >>> Color = namedtuple('Color', 'red green blue')
Christian Heimes0449f632007-12-15 01:27:15 +0000587 >>> Pixel = namedtuple('Pixel', Point._fields + Color._fields)
Thomas Wouters1b7f8912007-09-19 03:06:30 +0000588 >>> Pixel(11, 22, 128, 255, 0)
Christian Heimes454f37b2008-01-10 00:10:02 +0000589 Pixel(x=11, y=22, red=128, green=255, blue=0)
Georg Brandl116aa622007-08-15 14:28:22 +0000590
Christian Heimes0449f632007-12-15 01:27:15 +0000591To retrieve a field whose name is stored in a string, use the :func:`getattr`
Christian Heimes790c8232008-01-07 21:14:23 +0000592function::
Christian Heimes0449f632007-12-15 01:27:15 +0000593
594 >>> getattr(p, 'x')
595 11
596
Christian Heimes25bb7832008-01-11 16:17:00 +0000597To convert a dictionary to a named tuple, use the double-star-operator [#]_::
Christian Heimes99170a52007-12-19 02:07:34 +0000598
599 >>> d = {'x': 11, 'y': 22}
600 >>> Point(**d)
601 Point(x=11, y=22)
602
Guido van Rossum3d392eb2007-11-16 00:35:22 +0000603Since a named tuple is a regular Python class, it is easy to add or change
Christian Heimes043d6f62008-01-07 17:19:16 +0000604functionality with a subclass. Here is how to add a calculated field and
605a fixed-width print format::
Guido van Rossum3d392eb2007-11-16 00:35:22 +0000606
Christian Heimes043d6f62008-01-07 17:19:16 +0000607 >>> class Point(namedtuple('Point', 'x y')):
Christian Heimes25bb7832008-01-11 16:17:00 +0000608 ... __slots__ = ()
Christian Heimes454f37b2008-01-10 00:10:02 +0000609 ... @property
610 ... def hypot(self):
611 ... return (self.x ** 2 + self.y ** 2) ** 0.5
612 ... def __str__(self):
Christian Heimes25bb7832008-01-11 16:17:00 +0000613 ... return 'Point: x=%6.3f y=%6.3f hypot=%6.3f' % (self.x, self.y, self.hypot)
Guido van Rossum3d392eb2007-11-16 00:35:22 +0000614
Christian Heimes25bb7832008-01-11 16:17:00 +0000615 >>> for p in Point(3, 4), Point(14, 5/7.):
Christian Heimes00412232008-01-10 16:02:19 +0000616 ... print(p)
Christian Heimes790c8232008-01-07 21:14:23 +0000617
Christian Heimes25bb7832008-01-11 16:17:00 +0000618 Point: x= 3.000 y= 4.000 hypot= 5.000
619 Point: x=14.000 y= 0.714 hypot=14.018
Christian Heimes043d6f62008-01-07 17:19:16 +0000620
Christian Heimesaf98da12008-01-27 15:18:18 +0000621The subclass shown above sets ``__slots__`` to an empty tuple. This keeps
Christian Heimes679db4a2008-01-18 09:56:22 +0000622keep memory requirements low by preventing the creation of instance dictionaries.
623
Christian Heimes2380ac72008-01-09 00:17:24 +0000624
625Subclassing is not useful for adding new, stored fields. Instead, simply
626create a new named tuple type from the :attr:`_fields` attribute::
627
Christian Heimes25bb7832008-01-11 16:17:00 +0000628 >>> Point3D = namedtuple('Point3D', Point._fields + ('z',))
Christian Heimes2380ac72008-01-09 00:17:24 +0000629
630Default values can be implemented by using :meth:`_replace` to
Christian Heimes790c8232008-01-07 21:14:23 +0000631customize a prototype instance::
Guido van Rossum3d392eb2007-11-16 00:35:22 +0000632
633 >>> Account = namedtuple('Account', 'owner balance transaction_count')
Christian Heimes587c2bf2008-01-19 16:21:02 +0000634 >>> default_account = Account('<owner name>', 0.0, 0)
635 >>> johns_account = default_account._replace(owner='John')
Guido van Rossum3d392eb2007-11-16 00:35:22 +0000636
Thomas Wouters47b49bf2007-08-30 22:15:33 +0000637.. rubric:: Footnotes
638
Christian Heimes99170a52007-12-19 02:07:34 +0000639.. [#] For information on the double-star-operator see
Thomas Wouters47b49bf2007-08-30 22:15:33 +0000640 :ref:`tut-unpacking-arguments` and :ref:`calls`.
Raymond Hettingere4c96ad2008-02-06 01:23:58 +0000641
642
643
644:class:`UserDict` objects
Mark Summerfield8f2d0062008-02-06 13:30:44 +0000645-------------------------
Raymond Hettingere4c96ad2008-02-06 01:23:58 +0000646
647The class, :class:`UserDict` acts as a wrapper around dictionary objects.
648The need for this class has been partially supplanted by the ability to
649subclass directly from :class:`dict`; however, this class can be easier
650to work with because the underlying dictionary is accessible as an
651attribute.
652
653.. class:: UserDict([initialdata])
654
655 Class that simulates a dictionary. The instance's contents are kept in a
656 regular dictionary, which is accessible via the :attr:`data` attribute of
657 :class:`UserDict` instances. If *initialdata* is provided, :attr:`data` is
658 initialized with its contents; note that a reference to *initialdata* will not
659 be kept, allowing it be used for other purposes.
660
661In addition to supporting the methods and operations of mappings,
Raymond Hettingerebcee3f2008-02-06 19:54:00 +0000662:class:`UserDict` instances provide the following attribute:
Raymond Hettingere4c96ad2008-02-06 01:23:58 +0000663
664.. attribute:: UserDict.data
665
666 A real dictionary used to store the contents of the :class:`UserDict` class.
Raymond Hettinger53dbe392008-02-12 20:03:09 +0000667
668
669
670:class:`UserList` objects
671-------------------------
672
673This class acts as a wrapper around list objects. It is a useful base class
674for your own list-like classes which can inherit from them and override
675existing methods or add new ones. In this way, one can add new behaviors to
676lists.
677
678The need for this class has been partially supplanted by the ability to
679subclass directly from :class:`list`; however, this class can be easier
680to work with because the underlying list is accessible as an attribute.
681
682.. class:: UserList([list])
683
684 Class that simulates a list. The instance's contents are kept in a regular
685 list, which is accessible via the :attr:`data` attribute of :class:`UserList`
686 instances. The instance's contents are initially set to a copy of *list*,
687 defaulting to the empty list ``[]``. *list* can be any iterable, for
688 example a real Python list or a :class:`UserList` object.
689
690In addition to supporting the methods and operations of mutable sequences,
691:class:`UserList` instances provide the following attribute:
692
693.. attribute:: UserList.data
694
695 A real :class:`list` object used to store the contents of the
696 :class:`UserList` class.
697
698**Subclassing requirements:** Subclasses of :class:`UserList` are expect to
699offer a constructor which can be called with either no arguments or one
700argument. List operations which return a new sequence attempt to create an
701instance of the actual implementation class. To do so, it assumes that the
702constructor can be called with a single parameter, which is a sequence object
703used as a data source.
704
705If a derived class does not wish to comply with this requirement, all of the
706special methods supported by this class will need to be overridden; please
707consult the sources for information about the methods which need to be provided
708in that case.
Raymond Hettingerb3a65f82008-02-21 22:11:37 +0000709
710:class:`UserString` objects
711-------------------------
712
713The class, :class:`UserString` acts as a wrapper around string objects.
714The need for this class has been partially supplanted by the ability to
715subclass directly from :class:`str`; however, this class can be easier
716to work with because the underlying string is accessible as an
717attribute.
718
719.. class:: UserString([sequence])
720
721 Class that simulates a string or a Unicode string object. The instance's
722 content is kept in a regular string object, which is accessible via the
723 :attr:`data` attribute of :class:`UserString` instances. The instance's
724 contents are initially set to a copy of *sequence*. The *sequence* can
725 be an instance of :class:`bytes`, :class:`str`, :class:`UserString` (or a
726 subclass) or an arbitrary sequence which can be converted into a string using
727 the built-in :func:`str` function.