blob: a44eb85b27dbab6712f6dc4199a76625426f1290 [file] [log] [blame]
Georg Brandl6c89a792012-01-25 22:36:25 +01001:mod:`functools` --- Higher-order functions and operations on callable objects
Georg Brandl116aa622007-08-15 14:28:22 +00002==============================================================================
3
4.. module:: functools
Georg Brandl6c89a792012-01-25 22:36:25 +01005 :synopsis: Higher-order functions and operations on callable objects.
Terry Jan Reedyfa089b92016-06-11 15:02:54 -04006
Georg Brandl116aa622007-08-15 14:28:22 +00007.. moduleauthor:: Peter Harris <scav@blueyonder.co.uk>
8.. moduleauthor:: Raymond Hettinger <python@rcn.com>
9.. moduleauthor:: Nick Coghlan <ncoghlan@gmail.com>
Łukasz Langa6f692512013-06-05 12:20:24 +020010.. moduleauthor:: Łukasz Langa <lukasz@langa.pl>
Pablo Galindo99e6c262020-01-23 15:29:52 +000011.. moduleauthor:: Pablo Galindo <pablogsal@gmail.com>
Georg Brandl116aa622007-08-15 14:28:22 +000012.. sectionauthor:: Peter Harris <scav@blueyonder.co.uk>
13
Raymond Hettinger05ce0792011-01-10 21:16:07 +000014**Source code:** :source:`Lib/functools.py`
15
Pablo Galindo99e6c262020-01-23 15:29:52 +000016.. testsetup:: default
17
18 import functools
19 from functools import *
20
Raymond Hettinger05ce0792011-01-10 21:16:07 +000021--------------
Georg Brandl116aa622007-08-15 14:28:22 +000022
Georg Brandl116aa622007-08-15 14:28:22 +000023The :mod:`functools` module is for higher-order functions: functions that act on
24or return other functions. In general, any callable object can be treated as a
25function for the purposes of this module.
26
Thomas Woutersed03b412007-08-28 21:37:11 +000027The :mod:`functools` module defines the following functions:
28
Raymond Hettinger21cdb712020-05-11 17:00:53 -070029.. decorator:: cache(user_function)
30
31 Simple lightweight unbounded function cache. Sometimes called
32 `"memoize" <https://en.wikipedia.org/wiki/Memoization>`_.
33
34 Returns the same as ``lru_cache(maxsize=None)``, creating a thin
35 wrapper around a dictionary lookup for the function arguments. Because it
36 never needs to evict old values, this is smaller and faster than
37 :func:`lru_cache()` with a size limit.
38
39 For example::
40
41 @cache
42 def factorial(n):
43 return n * factorial(n-1) if n else 1
44
45 >>> factorial(10) # no previously cached result, makes 11 recursive calls
46 3628800
47 >>> factorial(5) # just looks up cached value result
48 120
49 >>> factorial(12) # makes two new recursive calls, the other 10 are cached
50 479001600
51
52 .. versionadded:: 3.9
53
54
Carl Meyerd658dea2018-08-28 01:11:56 -060055.. decorator:: cached_property(func)
56
57 Transform a method of a class into a property whose value is computed once
58 and then cached as a normal attribute for the life of the instance. Similar
59 to :func:`property`, with the addition of caching. Useful for expensive
60 computed properties of instances that are otherwise effectively immutable.
61
62 Example::
63
64 class DataSet:
65 def __init__(self, sequence_of_numbers):
66 self._data = sequence_of_numbers
67
68 @cached_property
69 def stdev(self):
70 return statistics.stdev(self._data)
71
72 @cached_property
73 def variance(self):
74 return statistics.variance(self._data)
75
76 .. versionadded:: 3.8
77
78 .. note::
79
80 This decorator requires that the ``__dict__`` attribute on each instance
81 be a mutable mapping. This means it will not work with some types, such as
82 metaclasses (since the ``__dict__`` attributes on type instances are
83 read-only proxies for the class namespace), and those that specify
84 ``__slots__`` without including ``__dict__`` as one of the defined slots
85 (as such classes don't provide a ``__dict__`` attribute at all).
86
87
Éric Araujob10089e2010-11-18 14:22:08 +000088.. function:: cmp_to_key(func)
Raymond Hettingerc50846a2010-04-05 18:56:31 +000089
Raymond Hettinger86e9b6b2014-11-09 17:20:56 -080090 Transform an old-style comparison function to a :term:`key function`. Used
91 with tools that accept key functions (such as :func:`sorted`, :func:`min`,
Benjamin Petersoncca65312010-08-09 02:13:10 +000092 :func:`max`, :func:`heapq.nlargest`, :func:`heapq.nsmallest`,
93 :func:`itertools.groupby`). This function is primarily used as a transition
Ezio Melotti9ecb6be2012-01-16 08:28:54 +020094 tool for programs being converted from Python 2 which supported the use of
Benjamin Petersoncca65312010-08-09 02:13:10 +000095 comparison functions.
Raymond Hettingerc50846a2010-04-05 18:56:31 +000096
Georg Brandl6c89a792012-01-25 22:36:25 +010097 A comparison function is any callable that accept two arguments, compares them,
Benjamin Petersoncca65312010-08-09 02:13:10 +000098 and returns a negative number for less-than, zero for equality, or a positive
99 number for greater-than. A key function is a callable that accepts one
Raymond Hettinger86e9b6b2014-11-09 17:20:56 -0800100 argument and returns another value to be used as the sort key.
Raymond Hettingerc50846a2010-04-05 18:56:31 +0000101
Benjamin Petersoncca65312010-08-09 02:13:10 +0000102 Example::
Raymond Hettingerc50846a2010-04-05 18:56:31 +0000103
Benjamin Petersoncca65312010-08-09 02:13:10 +0000104 sorted(iterable, key=cmp_to_key(locale.strcoll)) # locale-aware sort order
Raymond Hettingerc50846a2010-04-05 18:56:31 +0000105
Raymond Hettinger86e9b6b2014-11-09 17:20:56 -0800106 For sorting examples and a brief sorting tutorial, see :ref:`sortinghowto`.
107
Raymond Hettingerc50846a2010-04-05 18:56:31 +0000108 .. versionadded:: 3.2
109
Georg Brandl67b21b72010-08-17 15:07:14 +0000110
Raymond Hettingerb8218682019-05-26 11:27:35 -0700111.. decorator:: lru_cache(user_function)
112 lru_cache(maxsize=128, typed=False)
Georg Brandl2e7346a2010-07-31 18:09:23 +0000113
114 Decorator to wrap a function with a memoizing callable that saves up to the
115 *maxsize* most recent calls. It can save time when an expensive or I/O bound
116 function is periodically called with the same arguments.
117
Raymond Hettinger7496b412010-11-30 19:15:45 +0000118 Since a dictionary is used to cache results, the positional and keyword
119 arguments to the function must be hashable.
Georg Brandl2e7346a2010-07-31 18:09:23 +0000120
Raymond Hettinger902bcd92018-09-14 00:53:20 -0700121 Distinct argument patterns may be considered to be distinct calls with
122 separate cache entries. For example, `f(a=1, b=2)` and `f(b=2, a=1)`
123 differ in their keyword argument order and may have two separate cache
124 entries.
125
Raymond Hettingerb8218682019-05-26 11:27:35 -0700126 If *user_function* is specified, it must be a callable. This allows the
127 *lru_cache* decorator to be applied directly to a user function, leaving
128 the *maxsize* at its default value of 128::
129
130 @lru_cache
131 def count_vowels(sentence):
132 sentence = sentence.casefold()
133 return sum(sentence.count(vowel) for vowel in 'aeiou')
134
Serhiy Storchakaecf41da2016-10-19 16:29:26 +0300135 If *maxsize* is set to ``None``, the LRU feature is disabled and the cache can
Raymond Hettingerad9eaea2020-05-03 16:45:13 -0700136 grow without bound.
Raymond Hettingerc79fb0e2010-12-01 03:45:41 +0000137
Serhiy Storchaka4adf01c2016-10-19 18:30:05 +0300138 If *typed* is set to true, function arguments of different types will be
Raymond Hettingercd9fdfd2011-10-20 08:57:45 -0700139 cached separately. For example, ``f(3)`` and ``f(3.0)`` will be treated
140 as distinct calls with distinct results.
141
Manjusaka051ff522019-11-12 15:30:18 +0800142 The wrapped function is instrumented with a :func:`cache_parameters`
143 function that returns a new :class:`dict` showing the values for *maxsize*
144 and *typed*. This is for information purposes only. Mutating the values
145 has no effect.
146
Raymond Hettinger7496b412010-11-30 19:15:45 +0000147 To help measure the effectiveness of the cache and tune the *maxsize*
148 parameter, the wrapped function is instrumented with a :func:`cache_info`
149 function that returns a :term:`named tuple` showing *hits*, *misses*,
Raymond Hettingerc79fb0e2010-12-01 03:45:41 +0000150 *maxsize* and *currsize*. In a multi-threaded environment, the hits
151 and misses are approximate.
Nick Coghlan234515a2010-11-30 06:19:46 +0000152
Raymond Hettinger7496b412010-11-30 19:15:45 +0000153 The decorator also provides a :func:`cache_clear` function for clearing or
154 invalidating the cache.
Georg Brandl2e7346a2010-07-31 18:09:23 +0000155
Raymond Hettinger3fccfcb2010-08-17 19:19:29 +0000156 The original underlying function is accessible through the
Raymond Hettinger7496b412010-11-30 19:15:45 +0000157 :attr:`__wrapped__` attribute. This is useful for introspection, for
158 bypassing the cache, or for rewrapping the function with a different cache.
Nick Coghlan98876832010-08-17 06:17:18 +0000159
Raymond Hettingercc038582010-11-30 20:02:57 +0000160 An `LRU (least recently used) cache
Allen Guo3d542112020-05-12 18:54:18 -0400161 <https://en.wikipedia.org/wiki/Cache_replacement_policies#Least_recently_used_(LRU)>`_
162 works best when the most recent calls are the best predictors of upcoming
163 calls (for example, the most popular articles on a news server tend to
164 change each day). The cache's size limit assures that the cache does not
165 grow without bound on long-running processes such as web servers.
Raymond Hettinger7496b412010-11-30 19:15:45 +0000166
Raymond Hettingerf0e0f202018-11-25 16:24:52 -0800167 In general, the LRU cache should only be used when you want to reuse
168 previously computed values. Accordingly, it doesn't make sense to cache
169 functions with side-effects, functions that need to create distinct mutable
170 objects on each call, or impure functions such as time() or random().
171
Raymond Hettingercc038582010-11-30 20:02:57 +0000172 Example of an LRU cache for static web content::
Raymond Hettinger7496b412010-11-30 19:15:45 +0000173
Raymond Hettinger17328e42013-04-06 20:27:33 -0700174 @lru_cache(maxsize=32)
Raymond Hettinger7496b412010-11-30 19:15:45 +0000175 def get_pep(num):
176 'Retrieve text of a Python Enhancement Proposal'
177 resource = 'http://www.python.org/dev/peps/pep-%04d/' % num
178 try:
179 with urllib.request.urlopen(resource) as s:
180 return s.read()
181 except urllib.error.HTTPError:
182 return 'Not Found'
183
184 >>> for n in 8, 290, 308, 320, 8, 218, 320, 279, 289, 320, 9991:
185 ... pep = get_pep(n)
186 ... print(n, len(pep))
187
Raymond Hettinger17328e42013-04-06 20:27:33 -0700188 >>> get_pep.cache_info()
189 CacheInfo(hits=3, misses=8, maxsize=32, currsize=8)
Georg Brandl2e7346a2010-07-31 18:09:23 +0000190
Raymond Hettingerc79fb0e2010-12-01 03:45:41 +0000191 Example of efficiently computing
Georg Brandl5d941342016-02-26 19:37:12 +0100192 `Fibonacci numbers <https://en.wikipedia.org/wiki/Fibonacci_number>`_
Raymond Hettingerc79fb0e2010-12-01 03:45:41 +0000193 using a cache to implement a
Georg Brandl5d941342016-02-26 19:37:12 +0100194 `dynamic programming <https://en.wikipedia.org/wiki/Dynamic_programming>`_
Raymond Hettingerc79fb0e2010-12-01 03:45:41 +0000195 technique::
196
197 @lru_cache(maxsize=None)
198 def fib(n):
199 if n < 2:
200 return n
201 return fib(n-1) + fib(n-2)
202
Raymond Hettinger17328e42013-04-06 20:27:33 -0700203 >>> [fib(n) for n in range(16)]
Raymond Hettingerc79fb0e2010-12-01 03:45:41 +0000204 [0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, 377, 610]
205
Raymond Hettinger17328e42013-04-06 20:27:33 -0700206 >>> fib.cache_info()
Raymond Hettingerc79fb0e2010-12-01 03:45:41 +0000207 CacheInfo(hits=28, misses=16, maxsize=None, currsize=16)
208
Georg Brandl2e7346a2010-07-31 18:09:23 +0000209 .. versionadded:: 3.2
210
Raymond Hettingercd9fdfd2011-10-20 08:57:45 -0700211 .. versionchanged:: 3.3
212 Added the *typed* option.
213
Raymond Hettingerb8218682019-05-26 11:27:35 -0700214 .. versionchanged:: 3.8
215 Added the *user_function* option.
216
Manjusaka051ff522019-11-12 15:30:18 +0800217 .. versionadded:: 3.9
218 Added the function :func:`cache_parameters`
219
Georg Brandl8a1caa22010-07-29 16:01:11 +0000220.. decorator:: total_ordering
Raymond Hettingerc50846a2010-04-05 18:56:31 +0000221
222 Given a class defining one or more rich comparison ordering methods, this
Benjamin Peterson08bf91c2010-04-11 16:12:57 +0000223 class decorator supplies the rest. This simplifies the effort involved
Raymond Hettingerc50846a2010-04-05 18:56:31 +0000224 in specifying all of the possible rich comparison operations:
225
226 The class must define one of :meth:`__lt__`, :meth:`__le__`,
227 :meth:`__gt__`, or :meth:`__ge__`.
228 In addition, the class should supply an :meth:`__eq__` method.
229
230 For example::
231
232 @total_ordering
233 class Student:
Nick Coghlanf05d9812013-10-02 00:02:03 +1000234 def _is_valid_operand(self, other):
235 return (hasattr(other, "lastname") and
236 hasattr(other, "firstname"))
Raymond Hettingerc50846a2010-04-05 18:56:31 +0000237 def __eq__(self, other):
Nick Coghlanf05d9812013-10-02 00:02:03 +1000238 if not self._is_valid_operand(other):
239 return NotImplemented
Raymond Hettingerc50846a2010-04-05 18:56:31 +0000240 return ((self.lastname.lower(), self.firstname.lower()) ==
241 (other.lastname.lower(), other.firstname.lower()))
242 def __lt__(self, other):
Nick Coghlanf05d9812013-10-02 00:02:03 +1000243 if not self._is_valid_operand(other):
244 return NotImplemented
Raymond Hettingerc50846a2010-04-05 18:56:31 +0000245 return ((self.lastname.lower(), self.firstname.lower()) <
246 (other.lastname.lower(), other.firstname.lower()))
247
Nick Coghlanf05d9812013-10-02 00:02:03 +1000248 .. note::
249
250 While this decorator makes it easy to create well behaved totally
251 ordered types, it *does* come at the cost of slower execution and
252 more complex stack traces for the derived comparison methods. If
253 performance benchmarking indicates this is a bottleneck for a given
254 application, implementing all six rich comparison methods instead is
255 likely to provide an easy speed boost.
256
Raymond Hettingerc50846a2010-04-05 18:56:31 +0000257 .. versionadded:: 3.2
258
Nick Coghlanf05d9812013-10-02 00:02:03 +1000259 .. versionchanged:: 3.4
260 Returning NotImplemented from the underlying comparison function for
261 unrecognised types is now supported.
Georg Brandl67b21b72010-08-17 15:07:14 +0000262
Serhiy Storchaka2085bd02019-06-01 11:00:15 +0300263.. function:: partial(func, /, *args, **keywords)
Georg Brandl116aa622007-08-15 14:28:22 +0000264
Andrei Petre83a07652018-10-22 23:11:20 -0700265 Return a new :ref:`partial object<partial-objects>` which when called
266 will behave like *func* called with the positional arguments *args*
267 and keyword arguments *keywords*. If more arguments are supplied to the
268 call, they are appended to *args*. If additional keyword arguments are
269 supplied, they extend and override *keywords*.
Georg Brandl116aa622007-08-15 14:28:22 +0000270 Roughly equivalent to::
271
Serhiy Storchaka2085bd02019-06-01 11:00:15 +0300272 def partial(func, /, *args, **keywords):
Georg Brandl116aa622007-08-15 14:28:22 +0000273 def newfunc(*fargs, **fkeywords):
Sergey Fedoseevb981fec2018-10-20 02:42:07 +0500274 newkeywords = {**keywords, **fkeywords}
Martin Panter0c0da482016-06-12 01:46:50 +0000275 return func(*args, *fargs, **newkeywords)
Georg Brandl116aa622007-08-15 14:28:22 +0000276 newfunc.func = func
277 newfunc.args = args
278 newfunc.keywords = keywords
279 return newfunc
280
281 The :func:`partial` is used for partial function application which "freezes"
282 some portion of a function's arguments and/or keywords resulting in a new object
283 with a simplified signature. For example, :func:`partial` can be used to create
284 a callable that behaves like the :func:`int` function where the *base* argument
Christian Heimesfe337bf2008-03-23 21:54:12 +0000285 defaults to two:
Georg Brandl116aa622007-08-15 14:28:22 +0000286
Christian Heimesfe337bf2008-03-23 21:54:12 +0000287 >>> from functools import partial
Georg Brandl116aa622007-08-15 14:28:22 +0000288 >>> basetwo = partial(int, base=2)
289 >>> basetwo.__doc__ = 'Convert base 2 string to an int.'
290 >>> basetwo('10010')
291 18
292
293
Serhiy Storchaka70c5f2a2019-06-01 11:38:24 +0300294.. class:: partialmethod(func, /, *args, **keywords)
Nick Coghlanf4cb48a2013-11-03 16:41:46 +1000295
296 Return a new :class:`partialmethod` descriptor which behaves
297 like :class:`partial` except that it is designed to be used as a method
298 definition rather than being directly callable.
299
300 *func* must be a :term:`descriptor` or a callable (objects which are both,
301 like normal functions, are handled as descriptors).
302
303 When *func* is a descriptor (such as a normal Python function,
304 :func:`classmethod`, :func:`staticmethod`, :func:`abstractmethod` or
305 another instance of :class:`partialmethod`), calls to ``__get__`` are
306 delegated to the underlying descriptor, and an appropriate
Andrei Petre83a07652018-10-22 23:11:20 -0700307 :ref:`partial object<partial-objects>` returned as the result.
Nick Coghlanf4cb48a2013-11-03 16:41:46 +1000308
309 When *func* is a non-descriptor callable, an appropriate bound method is
310 created dynamically. This behaves like a normal Python function when
311 used as a method: the *self* argument will be inserted as the first
312 positional argument, even before the *args* and *keywords* supplied to
313 the :class:`partialmethod` constructor.
314
315 Example::
316
Serhiy Storchakae042a452019-06-10 13:35:52 +0300317 >>> class Cell:
Benjamin Peterson3a434032014-03-30 15:07:09 -0400318 ... def __init__(self):
319 ... self._alive = False
Nick Coghlanf4cb48a2013-11-03 16:41:46 +1000320 ... @property
321 ... def alive(self):
322 ... return self._alive
323 ... def set_state(self, state):
324 ... self._alive = bool(state)
Nick Coghlan3daaf5f2013-11-04 23:32:16 +1000325 ... set_alive = partialmethod(set_state, True)
326 ... set_dead = partialmethod(set_state, False)
Nick Coghlanf4cb48a2013-11-03 16:41:46 +1000327 ...
328 >>> c = Cell()
329 >>> c.alive
330 False
331 >>> c.set_alive()
332 >>> c.alive
333 True
334
335 .. versionadded:: 3.4
336
337
Georg Brandl58f9e4f2008-04-19 22:18:33 +0000338.. function:: reduce(function, iterable[, initializer])
Georg Brandl116aa622007-08-15 14:28:22 +0000339
Brendan Jurd9df10022018-10-01 16:52:10 +1000340 Apply *function* of two arguments cumulatively to the items of *iterable*, from
341 left to right, so as to reduce the iterable to a single value. For example,
Georg Brandl116aa622007-08-15 14:28:22 +0000342 ``reduce(lambda x, y: x+y, [1, 2, 3, 4, 5])`` calculates ``((((1+2)+3)+4)+5)``.
343 The left argument, *x*, is the accumulated value and the right argument, *y*, is
Brendan Jurd9df10022018-10-01 16:52:10 +1000344 the update value from the *iterable*. If the optional *initializer* is present,
345 it is placed before the items of the iterable in the calculation, and serves as
346 a default when the iterable is empty. If *initializer* is not given and
347 *iterable* contains only one item, the first item is returned.
Georg Brandl116aa622007-08-15 14:28:22 +0000348
Raymond Hettinger558dcf32014-12-16 18:16:57 -0800349 Roughly equivalent to::
Raymond Hettinger64801682013-10-12 16:04:17 -0700350
351 def reduce(function, iterable, initializer=None):
352 it = iter(iterable)
353 if initializer is None:
354 value = next(it)
355 else:
356 value = initializer
357 for element in it:
358 value = function(value, element)
359 return value
360
Gerrit Hollbd81cbd2018-07-04 23:26:32 +0100361 See :func:`itertools.accumulate` for an iterator that yields all intermediate
362 values.
Georg Brandl116aa622007-08-15 14:28:22 +0000363
Daisuke Miyakawa0e61e672017-10-12 23:39:43 +0900364.. decorator:: singledispatch
Łukasz Langa6f692512013-06-05 12:20:24 +0200365
Daisuke Miyakawa0e61e672017-10-12 23:39:43 +0900366 Transform a function into a :term:`single-dispatch <single
Łukasz Langafdcf2b72013-06-07 22:54:03 +0200367 dispatch>` :term:`generic function`.
Łukasz Langa6f692512013-06-05 12:20:24 +0200368
369 To define a generic function, decorate it with the ``@singledispatch``
370 decorator. Note that the dispatch happens on the type of the first argument,
371 create your function accordingly::
372
373 >>> from functools import singledispatch
374 >>> @singledispatch
375 ... def fun(arg, verbose=False):
376 ... if verbose:
377 ... print("Let me just say,", end=" ")
378 ... print(arg)
379
380 To add overloaded implementations to the function, use the :func:`register`
Łukasz Langae5697532017-12-11 13:56:31 -0800381 attribute of the generic function. It is a decorator. For functions
382 annotated with types, the decorator will infer the type of the first
383 argument automatically::
Łukasz Langa6f692512013-06-05 12:20:24 +0200384
Łukasz Langae5697532017-12-11 13:56:31 -0800385 >>> @fun.register
386 ... def _(arg: int, verbose=False):
Łukasz Langa6f692512013-06-05 12:20:24 +0200387 ... if verbose:
388 ... print("Strength in numbers, eh?", end=" ")
389 ... print(arg)
390 ...
Łukasz Langae5697532017-12-11 13:56:31 -0800391 >>> @fun.register
392 ... def _(arg: list, verbose=False):
Łukasz Langa6f692512013-06-05 12:20:24 +0200393 ... if verbose:
394 ... print("Enumerate this:")
395 ... for i, elem in enumerate(arg):
396 ... print(i, elem)
397
Łukasz Langae5697532017-12-11 13:56:31 -0800398 For code which doesn't use type annotations, the appropriate type
399 argument can be passed explicitly to the decorator itself::
400
401 >>> @fun.register(complex)
402 ... def _(arg, verbose=False):
403 ... if verbose:
404 ... print("Better than complicated.", end=" ")
405 ... print(arg.real, arg.imag)
406 ...
407
408
Łukasz Langa6f692512013-06-05 12:20:24 +0200409 To enable registering lambdas and pre-existing functions, the
410 :func:`register` attribute can be used in a functional form::
411
412 >>> def nothing(arg, verbose=False):
413 ... print("Nothing.")
414 ...
415 >>> fun.register(type(None), nothing)
416
417 The :func:`register` attribute returns the undecorated function which
418 enables decorator stacking, pickling, as well as creating unit tests for
419 each variant independently::
420
421 >>> @fun.register(float)
422 ... @fun.register(Decimal)
423 ... def fun_num(arg, verbose=False):
424 ... if verbose:
425 ... print("Half of your number:", end=" ")
426 ... print(arg / 2)
427 ...
428 >>> fun_num is fun
429 False
430
431 When called, the generic function dispatches on the type of the first
432 argument::
433
434 >>> fun("Hello, world.")
435 Hello, world.
436 >>> fun("test.", verbose=True)
437 Let me just say, test.
438 >>> fun(42, verbose=True)
439 Strength in numbers, eh? 42
440 >>> fun(['spam', 'spam', 'eggs', 'spam'], verbose=True)
441 Enumerate this:
442 0 spam
443 1 spam
444 2 eggs
445 3 spam
446 >>> fun(None)
447 Nothing.
448 >>> fun(1.23)
449 0.615
450
451 Where there is no registered implementation for a specific type, its
452 method resolution order is used to find a more generic implementation.
453 The original function decorated with ``@singledispatch`` is registered
454 for the base ``object`` type, which means it is used if no better
455 implementation is found.
456
Batuhan Taşkaya24555ce2019-11-19 11:16:46 +0300457 If an implementation registered to :term:`abstract base class`, virtual
458 subclasses will be dispatched to that implementation::
459
460 >>> from collections.abc import Mapping
461 >>> @fun.register
462 ... def _(arg: Mapping, verbose=False):
463 ... if verbose:
464 ... print("Keys & Values")
465 ... for key, value in arg.items():
466 ... print(key, "=>", value)
467 ...
468 >>> fun({"a": "b"})
469 a => b
470
Łukasz Langa6f692512013-06-05 12:20:24 +0200471 To check which implementation will the generic function choose for
472 a given type, use the ``dispatch()`` attribute::
473
474 >>> fun.dispatch(float)
475 <function fun_num at 0x1035a2840>
476 >>> fun.dispatch(dict) # note: default implementation
477 <function fun at 0x103fe0000>
478
479 To access all registered implementations, use the read-only ``registry``
480 attribute::
481
482 >>> fun.registry.keys()
483 dict_keys([<class 'NoneType'>, <class 'int'>, <class 'object'>,
484 <class 'decimal.Decimal'>, <class 'list'>,
485 <class 'float'>])
486 >>> fun.registry[float]
487 <function fun_num at 0x1035a2840>
488 >>> fun.registry[object]
489 <function fun at 0x103fe0000>
490
491 .. versionadded:: 3.4
492
Łukasz Langae5697532017-12-11 13:56:31 -0800493 .. versionchanged:: 3.7
494 The :func:`register` attribute supports using type annotations.
495
Łukasz Langa6f692512013-06-05 12:20:24 +0200496
Ethan Smithc6512752018-05-26 16:38:33 -0400497.. class:: singledispatchmethod(func)
498
499 Transform a method into a :term:`single-dispatch <single
500 dispatch>` :term:`generic function`.
501
502 To define a generic method, decorate it with the ``@singledispatchmethod``
503 decorator. Note that the dispatch happens on the type of the first non-self
504 or non-cls argument, create your function accordingly::
505
506 class Negator:
507 @singledispatchmethod
508 def neg(self, arg):
509 raise NotImplementedError("Cannot negate a")
510
511 @neg.register
512 def _(self, arg: int):
513 return -arg
514
515 @neg.register
516 def _(self, arg: bool):
517 return not arg
518
519 ``@singledispatchmethod`` supports nesting with other decorators such as
520 ``@classmethod``. Note that to allow for ``dispatcher.register``,
521 ``singledispatchmethod`` must be the *outer most* decorator. Here is the
522 ``Negator`` class with the ``neg`` methods being class bound::
523
524 class Negator:
525 @singledispatchmethod
526 @classmethod
527 def neg(cls, arg):
528 raise NotImplementedError("Cannot negate a")
529
530 @neg.register
531 @classmethod
532 def _(cls, arg: int):
533 return -arg
534
535 @neg.register
536 @classmethod
537 def _(cls, arg: bool):
538 return not arg
539
540 The same pattern can be used for other similar decorators: ``staticmethod``,
541 ``abstractmethod``, and others.
542
Inada Naokibc284f02019-03-27 18:15:17 +0900543 .. versionadded:: 3.8
544
545
Pablo Galindo99e6c262020-01-23 15:29:52 +0000546.. class:: TopologicalSorter(graph=None)
547
548 Provides functionality to topologically sort a graph of hashable nodes.
549
Pablo Galindo65ecc392020-01-23 21:01:50 +0000550 A topological order is a linear ordering of the vertices in a graph such that
551 for every directed edge u -> v from vertex u to vertex v, vertex u comes
552 before vertex v in the ordering. For instance, the vertices of the graph may
553 represent tasks to be performed, and the edges may represent constraints that
554 one task must be performed before another; in this example, a topological
555 ordering is just a valid sequence for the tasks. A complete topological
556 ordering is possible if and only if the graph has no directed cycles, that
557 is, if it is a directed acyclic graph.
Pablo Galindo99e6c262020-01-23 15:29:52 +0000558
Pablo Galindo65ecc392020-01-23 21:01:50 +0000559 If the optional *graph* argument is provided it must be a dictionary
560 representing a directed acyclic graph where the keys are nodes and the values
561 are iterables of all predecessors of that node in the graph (the nodes that
562 have edges that point to the value in the key). Additional nodes can be added
563 to the graph using the :meth:`~TopologicalSorter.add` method.
Pablo Galindo99e6c262020-01-23 15:29:52 +0000564
Pablo Galindo65ecc392020-01-23 21:01:50 +0000565 In the general case, the steps required to perform the sorting of a given
566 graph are as follows:
Pablo Galindo99e6c262020-01-23 15:29:52 +0000567
Pablo Galindo65ecc392020-01-23 21:01:50 +0000568 * Create an instance of the :class:`TopologicalSorter` with an optional
569 initial graph.
Pablo Galindo99e6c262020-01-23 15:29:52 +0000570 * Add additional nodes to the graph.
571 * Call :meth:`~TopologicalSorter.prepare` on the graph.
Pablo Galindo65ecc392020-01-23 21:01:50 +0000572 * While :meth:`~TopologicalSorter.is_active` is ``True``, iterate over
573 the nodes returned by :meth:`~TopologicalSorter.get_ready` and
574 process them. Call :meth:`~TopologicalSorter.done` on each node as it
575 finishes processing.
Pablo Galindo99e6c262020-01-23 15:29:52 +0000576
577 In case just an immediate sorting of the nodes in the graph is required and
Pablo Galindo65ecc392020-01-23 21:01:50 +0000578 no parallelism is involved, the convenience method
579 :meth:`TopologicalSorter.static_order` can be used directly:
Pablo Galindo99e6c262020-01-23 15:29:52 +0000580
581 .. doctest::
582
Pablo Galindo65ecc392020-01-23 21:01:50 +0000583 >>> graph = {"D": {"B", "C"}, "C": {"A"}, "B": {"A"}}
Pablo Galindo99e6c262020-01-23 15:29:52 +0000584 >>> ts = TopologicalSorter(graph)
Pablo Galindo65ecc392020-01-23 21:01:50 +0000585 >>> tuple(ts.static_order())
586 ('A', 'C', 'B', 'D')
Pablo Galindo99e6c262020-01-23 15:29:52 +0000587
Pablo Galindo65ecc392020-01-23 21:01:50 +0000588 The class is designed to easily support parallel processing of the nodes as
589 they become ready. For instance::
Pablo Galindo99e6c262020-01-23 15:29:52 +0000590
591 topological_sorter = TopologicalSorter()
592
593 # Add nodes to 'topological_sorter'...
594
595 topological_sorter.prepare()
596 while topological_sorter.is_active():
597 for node in topological_sorter.get_ready():
598 # Worker threads or processes take nodes to work on off the
599 # 'task_queue' queue.
600 task_queue.put(node)
601
602 # When the work for a node is done, workers put the node in
603 # 'finalized_tasks_queue' so we can get more nodes to work on.
604 # The definition of 'is_active()' guarantees that, at this point, at
605 # least one node has been placed on 'task_queue' that hasn't yet
606 # been passed to 'done()', so this blocking 'get()' must (eventually)
607 # succeed. After calling 'done()', we loop back to call 'get_ready()'
608 # again, so put newly freed nodes on 'task_queue' as soon as
609 # logically possible.
610 node = finalized_tasks_queue.get()
611 topological_sorter.done(node)
612
613 .. method:: add(node, *predecessors)
614
Pablo Galindo65ecc392020-01-23 21:01:50 +0000615 Add a new node and its predecessors to the graph. Both the *node* and all
616 elements in *predecessors* must be hashable.
Pablo Galindo99e6c262020-01-23 15:29:52 +0000617
Pablo Galindo65ecc392020-01-23 21:01:50 +0000618 If called multiple times with the same node argument, the set of
619 dependencies will be the union of all dependencies passed in.
Pablo Galindo99e6c262020-01-23 15:29:52 +0000620
621 It is possible to add a node with no dependencies (*predecessors* is not
622 provided) or to provide a dependency twice. If a node that has not been
Pablo Galindo65ecc392020-01-23 21:01:50 +0000623 provided before is included among *predecessors* it will be automatically
624 added to the graph with no predecessors of its own.
Pablo Galindo99e6c262020-01-23 15:29:52 +0000625
626 Raises :exc:`ValueError` if called after :meth:`~TopologicalSorter.prepare`.
627
628 .. method:: prepare()
629
Pablo Galindo65ecc392020-01-23 21:01:50 +0000630 Mark the graph as finished and check for cycles in the graph. If any cycle
631 is detected, :exc:`CycleError` will be raised, but
632 :meth:`~TopologicalSorter.get_ready` can still be used to obtain as many
633 nodes as possible until cycles block more progress. After a call to this
634 function, the graph cannot be modified, and therefore no more nodes can be
635 added using :meth:`~TopologicalSorter.add`.
Pablo Galindo99e6c262020-01-23 15:29:52 +0000636
637 .. method:: is_active()
638
Pablo Galindo65ecc392020-01-23 21:01:50 +0000639 Returns ``True`` if more progress can be made and ``False`` otherwise.
640 Progress can be made if cycles do not block the resolution and either
641 there are still nodes ready that haven't yet been returned by
Pablo Galindo99e6c262020-01-23 15:29:52 +0000642 :meth:`TopologicalSorter.get_ready` or the number of nodes marked
Pablo Galindo65ecc392020-01-23 21:01:50 +0000643 :meth:`TopologicalSorter.done` is less than the number that have been
644 returned by :meth:`TopologicalSorter.get_ready`.
Pablo Galindo99e6c262020-01-23 15:29:52 +0000645
Pablo Galindo65ecc392020-01-23 21:01:50 +0000646 The :meth:`~TopologicalSorter.__bool__` method of this class defers to
647 this function, so instead of::
Pablo Galindo99e6c262020-01-23 15:29:52 +0000648
649 if ts.is_active():
650 ...
651
652 if possible to simply do::
653
654 if ts:
655 ...
656
Pablo Galindo65ecc392020-01-23 21:01:50 +0000657 Raises :exc:`ValueError` if called without calling
658 :meth:`~TopologicalSorter.prepare` previously.
Pablo Galindo99e6c262020-01-23 15:29:52 +0000659
660 .. method:: done(*nodes)
661
662 Marks a set of nodes returned by :meth:`TopologicalSorter.get_ready` as
Pablo Galindo65ecc392020-01-23 21:01:50 +0000663 processed, unblocking any successor of each node in *nodes* for being
664 returned in the future by a call to :meth:`TopologicalSorter.get_ready`.
Pablo Galindo99e6c262020-01-23 15:29:52 +0000665
666 Raises :exc:`ValueError` if any node in *nodes* has already been marked as
Pablo Galindo65ecc392020-01-23 21:01:50 +0000667 processed by a previous call to this method or if a node was not added to
668 the graph by using :meth:`TopologicalSorter.add`, if called without
669 calling :meth:`~TopologicalSorter.prepare` or if node has not yet been
670 returned by :meth:`~TopologicalSorter.get_ready`.
Pablo Galindo99e6c262020-01-23 15:29:52 +0000671
672 .. method:: get_ready()
673
Pablo Galindo65ecc392020-01-23 21:01:50 +0000674 Returns a ``tuple`` with all the nodes that are ready. Initially it
675 returns all nodes with no predecessors, and once those are marked as
676 processed by calling :meth:`TopologicalSorter.done`, further calls will
677 return all new nodes that have all their predecessors already processed.
678 Once no more progress can be made, empty tuples are returned.
Pablo Galindo99e6c262020-01-23 15:29:52 +0000679
680 Raises :exc:`ValueError` if called without calling
681 :meth:`~TopologicalSorter.prepare` previously.
682
683 .. method:: static_order()
684
685 Returns an iterable of nodes in a topological order. Using this method
686 does not require to call :meth:`TopologicalSorter.prepare` or
687 :meth:`TopologicalSorter.done`. This method is equivalent to::
688
689 def static_order(self):
690 self.prepare()
691 while self.is_active():
692 node_group = self.get_ready()
693 yield from node_group
694 self.done(*node_group)
695
696 The particular order that is returned may depend on the specific order in
697 which the items were inserted in the graph. For example:
698
699 .. doctest::
700
701 >>> ts = TopologicalSorter()
702 >>> ts.add(3, 2, 1)
703 >>> ts.add(1, 0)
704 >>> print([*ts.static_order()])
705 [2, 0, 1, 3]
706
707 >>> ts2 = TopologicalSorter()
708 >>> ts2.add(1, 0)
709 >>> ts2.add(3, 2, 1)
710 >>> print([*ts2.static_order()])
711 [0, 2, 1, 3]
712
Pablo Galindo65ecc392020-01-23 21:01:50 +0000713 This is due to the fact that "0" and "2" are in the same level in the
714 graph (they would have been returned in the same call to
715 :meth:`~TopologicalSorter.get_ready`) and the order between them is
716 determined by the order of insertion.
Pablo Galindo99e6c262020-01-23 15:29:52 +0000717
718
719 If any cycle is detected, :exc:`CycleError` will be raised.
720
721 .. versionadded:: 3.9
722
723
Georg Brandl036490d2009-05-17 13:00:36 +0000724.. function:: update_wrapper(wrapper, wrapped, assigned=WRAPPER_ASSIGNMENTS, updated=WRAPPER_UPDATES)
Georg Brandl116aa622007-08-15 14:28:22 +0000725
726 Update a *wrapper* function to look like the *wrapped* function. The optional
727 arguments are tuples to specify which attributes of the original function are
728 assigned directly to the matching attributes on the wrapper function and which
729 attributes of the wrapper function are updated with the corresponding attributes
730 from the original function. The default values for these arguments are the
Berker Peksag472233e2016-04-18 21:20:50 +0300731 module level constants ``WRAPPER_ASSIGNMENTS`` (which assigns to the wrapper
732 function's ``__module__``, ``__name__``, ``__qualname__``, ``__annotations__``
733 and ``__doc__``, the documentation string) and ``WRAPPER_UPDATES`` (which
734 updates the wrapper function's ``__dict__``, i.e. the instance dictionary).
Georg Brandl116aa622007-08-15 14:28:22 +0000735
Nick Coghlan98876832010-08-17 06:17:18 +0000736 To allow access to the original function for introspection and other purposes
737 (e.g. bypassing a caching decorator such as :func:`lru_cache`), this function
Nick Coghlan24c05bc2013-07-15 21:13:08 +1000738 automatically adds a ``__wrapped__`` attribute to the wrapper that refers to
739 the function being wrapped.
Nick Coghlan98876832010-08-17 06:17:18 +0000740
Christian Heimesd8654cf2007-12-02 15:22:16 +0000741 The main intended use for this function is in :term:`decorator` functions which
742 wrap the decorated function and return the wrapper. If the wrapper function is
743 not updated, the metadata of the returned function will reflect the wrapper
Georg Brandl116aa622007-08-15 14:28:22 +0000744 definition rather than the original function definition, which is typically less
745 than helpful.
746
Nick Coghlan98876832010-08-17 06:17:18 +0000747 :func:`update_wrapper` may be used with callables other than functions. Any
748 attributes named in *assigned* or *updated* that are missing from the object
749 being wrapped are ignored (i.e. this function will not attempt to set them
750 on the wrapper function). :exc:`AttributeError` is still raised if the
751 wrapper function itself is missing any attributes named in *updated*.
752
753 .. versionadded:: 3.2
Georg Brandl9e257012010-08-17 14:11:59 +0000754 Automatic addition of the ``__wrapped__`` attribute.
Nick Coghlan98876832010-08-17 06:17:18 +0000755
756 .. versionadded:: 3.2
Georg Brandl9e257012010-08-17 14:11:59 +0000757 Copying of the ``__annotations__`` attribute by default.
Nick Coghlan98876832010-08-17 06:17:18 +0000758
759 .. versionchanged:: 3.2
Georg Brandl9e257012010-08-17 14:11:59 +0000760 Missing attributes no longer trigger an :exc:`AttributeError`.
761
Nick Coghlan24c05bc2013-07-15 21:13:08 +1000762 .. versionchanged:: 3.4
763 The ``__wrapped__`` attribute now always refers to the wrapped
764 function, even if that function defined a ``__wrapped__`` attribute.
765 (see :issue:`17482`)
766
Georg Brandl116aa622007-08-15 14:28:22 +0000767
Georg Brandl8a1caa22010-07-29 16:01:11 +0000768.. decorator:: wraps(wrapped, assigned=WRAPPER_ASSIGNMENTS, updated=WRAPPER_UPDATES)
Georg Brandl116aa622007-08-15 14:28:22 +0000769
Ezio Melotti67f6d5f2014-08-05 08:14:28 +0300770 This is a convenience function for invoking :func:`update_wrapper` as a
771 function decorator when defining a wrapper function. It is equivalent to
772 ``partial(update_wrapper, wrapped=wrapped, assigned=assigned, updated=updated)``.
773 For example::
Georg Brandl116aa622007-08-15 14:28:22 +0000774
Christian Heimesfe337bf2008-03-23 21:54:12 +0000775 >>> from functools import wraps
Georg Brandl116aa622007-08-15 14:28:22 +0000776 >>> def my_decorator(f):
777 ... @wraps(f)
778 ... def wrapper(*args, **kwds):
Georg Brandl6911e3c2007-09-04 07:15:32 +0000779 ... print('Calling decorated function')
Georg Brandl116aa622007-08-15 14:28:22 +0000780 ... return f(*args, **kwds)
781 ... return wrapper
782 ...
783 >>> @my_decorator
784 ... def example():
785 ... """Docstring"""
Georg Brandl6911e3c2007-09-04 07:15:32 +0000786 ... print('Called example function')
Georg Brandl116aa622007-08-15 14:28:22 +0000787 ...
788 >>> example()
789 Calling decorated function
790 Called example function
791 >>> example.__name__
792 'example'
793 >>> example.__doc__
794 'Docstring'
795
796 Without the use of this decorator factory, the name of the example function
797 would have been ``'wrapper'``, and the docstring of the original :func:`example`
798 would have been lost.
799
800
801.. _partial-objects:
802
803:class:`partial` Objects
804------------------------
805
806:class:`partial` objects are callable objects created by :func:`partial`. They
807have three read-only attributes:
808
809
810.. attribute:: partial.func
811
812 A callable object or function. Calls to the :class:`partial` object will be
813 forwarded to :attr:`func` with new arguments and keywords.
814
815
816.. attribute:: partial.args
817
818 The leftmost positional arguments that will be prepended to the positional
819 arguments provided to a :class:`partial` object call.
820
821
822.. attribute:: partial.keywords
823
824 The keyword arguments that will be supplied when the :class:`partial` object is
825 called.
826
827:class:`partial` objects are like :class:`function` objects in that they are
828callable, weak referencable, and can have attributes. There are some important
Martin Panterbae5d812016-06-18 03:57:31 +0000829differences. For instance, the :attr:`~definition.__name__` and :attr:`__doc__` attributes
Georg Brandl116aa622007-08-15 14:28:22 +0000830are not created automatically. Also, :class:`partial` objects defined in
831classes behave like static methods and do not transform into bound methods
832during instance attribute look-up.
Pablo Galindo99e6c262020-01-23 15:29:52 +0000833
834
835Exceptions
836----------
837The :mod:`functools` module defines the following exception classes:
838
839.. exception:: CycleError
840
841 Subclass of :exc:`ValueError` raised by :meth:`TopologicalSorter.prepare` if cycles exist
842 in the working graph. If multiple cycles exist, only one undefined choice among them will
843 be reported and included in the exception.
844
845 The detected cycle can be accessed via the second element in the :attr:`~CycleError.args`
846 attribute of the exception instance and consists in a list of nodes, such that each node is,
847 in the graph, an immediate predecessor of the next node in the list. In the reported list,
848 the first and the last node will be the same, to make it clear that it is cyclic.