Georg Brandl | 6c89a79 | 2012-01-25 22:36:25 +0100 | [diff] [blame] | 1 | :mod:`functools` --- Higher-order functions and operations on callable objects |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 2 | ============================================================================== |
| 3 | |
| 4 | .. module:: functools |
Georg Brandl | 6c89a79 | 2012-01-25 22:36:25 +0100 | [diff] [blame] | 5 | :synopsis: Higher-order functions and operations on callable objects. |
Terry Jan Reedy | fa089b9 | 2016-06-11 15:02:54 -0400 | [diff] [blame] | 6 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 7 | .. moduleauthor:: Peter Harris <scav@blueyonder.co.uk> |
| 8 | .. moduleauthor:: Raymond Hettinger <python@rcn.com> |
| 9 | .. moduleauthor:: Nick Coghlan <ncoghlan@gmail.com> |
Łukasz Langa | 6f69251 | 2013-06-05 12:20:24 +0200 | [diff] [blame] | 10 | .. moduleauthor:: Łukasz Langa <lukasz@langa.pl> |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 11 | .. sectionauthor:: Peter Harris <scav@blueyonder.co.uk> |
| 12 | |
Raymond Hettinger | 05ce079 | 2011-01-10 21:16:07 +0000 | [diff] [blame] | 13 | **Source code:** :source:`Lib/functools.py` |
| 14 | |
| 15 | -------------- |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 16 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 17 | The :mod:`functools` module is for higher-order functions: functions that act on |
| 18 | or return other functions. In general, any callable object can be treated as a |
| 19 | function for the purposes of this module. |
| 20 | |
Thomas Wouters | ed03b41 | 2007-08-28 21:37:11 +0000 | [diff] [blame] | 21 | The :mod:`functools` module defines the following functions: |
| 22 | |
Éric Araujo | b10089e | 2010-11-18 14:22:08 +0000 | [diff] [blame] | 23 | .. function:: cmp_to_key(func) |
Raymond Hettinger | c50846a | 2010-04-05 18:56:31 +0000 | [diff] [blame] | 24 | |
Raymond Hettinger | 86e9b6b | 2014-11-09 17:20:56 -0800 | [diff] [blame] | 25 | Transform an old-style comparison function to a :term:`key function`. Used |
| 26 | with tools that accept key functions (such as :func:`sorted`, :func:`min`, |
Benjamin Peterson | cca6531 | 2010-08-09 02:13:10 +0000 | [diff] [blame] | 27 | :func:`max`, :func:`heapq.nlargest`, :func:`heapq.nsmallest`, |
| 28 | :func:`itertools.groupby`). This function is primarily used as a transition |
Ezio Melotti | 9ecb6be | 2012-01-16 08:28:54 +0200 | [diff] [blame] | 29 | tool for programs being converted from Python 2 which supported the use of |
Benjamin Peterson | cca6531 | 2010-08-09 02:13:10 +0000 | [diff] [blame] | 30 | comparison functions. |
Raymond Hettinger | c50846a | 2010-04-05 18:56:31 +0000 | [diff] [blame] | 31 | |
Georg Brandl | 6c89a79 | 2012-01-25 22:36:25 +0100 | [diff] [blame] | 32 | A comparison function is any callable that accept two arguments, compares them, |
Benjamin Peterson | cca6531 | 2010-08-09 02:13:10 +0000 | [diff] [blame] | 33 | and returns a negative number for less-than, zero for equality, or a positive |
| 34 | number for greater-than. A key function is a callable that accepts one |
Raymond Hettinger | 86e9b6b | 2014-11-09 17:20:56 -0800 | [diff] [blame] | 35 | argument and returns another value to be used as the sort key. |
Raymond Hettinger | c50846a | 2010-04-05 18:56:31 +0000 | [diff] [blame] | 36 | |
Benjamin Peterson | cca6531 | 2010-08-09 02:13:10 +0000 | [diff] [blame] | 37 | Example:: |
Raymond Hettinger | c50846a | 2010-04-05 18:56:31 +0000 | [diff] [blame] | 38 | |
Benjamin Peterson | cca6531 | 2010-08-09 02:13:10 +0000 | [diff] [blame] | 39 | sorted(iterable, key=cmp_to_key(locale.strcoll)) # locale-aware sort order |
Raymond Hettinger | c50846a | 2010-04-05 18:56:31 +0000 | [diff] [blame] | 40 | |
Raymond Hettinger | 86e9b6b | 2014-11-09 17:20:56 -0800 | [diff] [blame] | 41 | For sorting examples and a brief sorting tutorial, see :ref:`sortinghowto`. |
| 42 | |
Raymond Hettinger | c50846a | 2010-04-05 18:56:31 +0000 | [diff] [blame] | 43 | .. versionadded:: 3.2 |
| 44 | |
Georg Brandl | 67b21b7 | 2010-08-17 15:07:14 +0000 | [diff] [blame] | 45 | |
Raymond Hettinger | 010ce32 | 2012-05-19 21:20:48 -0700 | [diff] [blame] | 46 | .. decorator:: lru_cache(maxsize=128, typed=False) |
Georg Brandl | 2e7346a | 2010-07-31 18:09:23 +0000 | [diff] [blame] | 47 | |
| 48 | Decorator to wrap a function with a memoizing callable that saves up to the |
| 49 | *maxsize* most recent calls. It can save time when an expensive or I/O bound |
| 50 | function is periodically called with the same arguments. |
| 51 | |
Raymond Hettinger | 7496b41 | 2010-11-30 19:15:45 +0000 | [diff] [blame] | 52 | Since a dictionary is used to cache results, the positional and keyword |
| 53 | arguments to the function must be hashable. |
Georg Brandl | 2e7346a | 2010-07-31 18:09:23 +0000 | [diff] [blame] | 54 | |
Serhiy Storchaka | ecf41da | 2016-10-19 16:29:26 +0300 | [diff] [blame] | 55 | If *maxsize* is set to ``None``, the LRU feature is disabled and the cache can |
Raymond Hettinger | 7d74eff | 2012-06-04 00:32:15 -0700 | [diff] [blame] | 56 | grow without bound. The LRU feature performs best when *maxsize* is a |
| 57 | power-of-two. |
Raymond Hettinger | c79fb0e | 2010-12-01 03:45:41 +0000 | [diff] [blame] | 58 | |
Serhiy Storchaka | 4adf01c | 2016-10-19 18:30:05 +0300 | [diff] [blame] | 59 | If *typed* is set to true, function arguments of different types will be |
Raymond Hettinger | cd9fdfd | 2011-10-20 08:57:45 -0700 | [diff] [blame] | 60 | cached separately. For example, ``f(3)`` and ``f(3.0)`` will be treated |
| 61 | as distinct calls with distinct results. |
| 62 | |
Raymond Hettinger | 7496b41 | 2010-11-30 19:15:45 +0000 | [diff] [blame] | 63 | To help measure the effectiveness of the cache and tune the *maxsize* |
| 64 | parameter, the wrapped function is instrumented with a :func:`cache_info` |
| 65 | function that returns a :term:`named tuple` showing *hits*, *misses*, |
Raymond Hettinger | c79fb0e | 2010-12-01 03:45:41 +0000 | [diff] [blame] | 66 | *maxsize* and *currsize*. In a multi-threaded environment, the hits |
| 67 | and misses are approximate. |
Nick Coghlan | 234515a | 2010-11-30 06:19:46 +0000 | [diff] [blame] | 68 | |
Raymond Hettinger | 7496b41 | 2010-11-30 19:15:45 +0000 | [diff] [blame] | 69 | The decorator also provides a :func:`cache_clear` function for clearing or |
| 70 | invalidating the cache. |
Georg Brandl | 2e7346a | 2010-07-31 18:09:23 +0000 | [diff] [blame] | 71 | |
Raymond Hettinger | 3fccfcb | 2010-08-17 19:19:29 +0000 | [diff] [blame] | 72 | The original underlying function is accessible through the |
Raymond Hettinger | 7496b41 | 2010-11-30 19:15:45 +0000 | [diff] [blame] | 73 | :attr:`__wrapped__` attribute. This is useful for introspection, for |
| 74 | bypassing the cache, or for rewrapping the function with a different cache. |
Nick Coghlan | 9887683 | 2010-08-17 06:17:18 +0000 | [diff] [blame] | 75 | |
Raymond Hettinger | cc03858 | 2010-11-30 20:02:57 +0000 | [diff] [blame] | 76 | An `LRU (least recently used) cache |
Georg Brandl | 5d94134 | 2016-02-26 19:37:12 +0100 | [diff] [blame] | 77 | <https://en.wikipedia.org/wiki/Cache_algorithms#Examples>`_ works |
Raymond Hettinger | cd9fdfd | 2011-10-20 08:57:45 -0700 | [diff] [blame] | 78 | best when the most recent calls are the best predictors of upcoming calls (for |
| 79 | example, the most popular articles on a news server tend to change each day). |
Raymond Hettinger | 7496b41 | 2010-11-30 19:15:45 +0000 | [diff] [blame] | 80 | The cache's size limit assures that the cache does not grow without bound on |
| 81 | long-running processes such as web servers. |
| 82 | |
Raymond Hettinger | cc03858 | 2010-11-30 20:02:57 +0000 | [diff] [blame] | 83 | Example of an LRU cache for static web content:: |
Raymond Hettinger | 7496b41 | 2010-11-30 19:15:45 +0000 | [diff] [blame] | 84 | |
Raymond Hettinger | 17328e4 | 2013-04-06 20:27:33 -0700 | [diff] [blame] | 85 | @lru_cache(maxsize=32) |
Raymond Hettinger | 7496b41 | 2010-11-30 19:15:45 +0000 | [diff] [blame] | 86 | def get_pep(num): |
| 87 | 'Retrieve text of a Python Enhancement Proposal' |
| 88 | resource = 'http://www.python.org/dev/peps/pep-%04d/' % num |
| 89 | try: |
| 90 | with urllib.request.urlopen(resource) as s: |
| 91 | return s.read() |
| 92 | except urllib.error.HTTPError: |
| 93 | return 'Not Found' |
| 94 | |
| 95 | >>> for n in 8, 290, 308, 320, 8, 218, 320, 279, 289, 320, 9991: |
| 96 | ... pep = get_pep(n) |
| 97 | ... print(n, len(pep)) |
| 98 | |
Raymond Hettinger | 17328e4 | 2013-04-06 20:27:33 -0700 | [diff] [blame] | 99 | >>> get_pep.cache_info() |
| 100 | CacheInfo(hits=3, misses=8, maxsize=32, currsize=8) |
Georg Brandl | 2e7346a | 2010-07-31 18:09:23 +0000 | [diff] [blame] | 101 | |
Raymond Hettinger | c79fb0e | 2010-12-01 03:45:41 +0000 | [diff] [blame] | 102 | Example of efficiently computing |
Georg Brandl | 5d94134 | 2016-02-26 19:37:12 +0100 | [diff] [blame] | 103 | `Fibonacci numbers <https://en.wikipedia.org/wiki/Fibonacci_number>`_ |
Raymond Hettinger | c79fb0e | 2010-12-01 03:45:41 +0000 | [diff] [blame] | 104 | using a cache to implement a |
Georg Brandl | 5d94134 | 2016-02-26 19:37:12 +0100 | [diff] [blame] | 105 | `dynamic programming <https://en.wikipedia.org/wiki/Dynamic_programming>`_ |
Raymond Hettinger | c79fb0e | 2010-12-01 03:45:41 +0000 | [diff] [blame] | 106 | technique:: |
| 107 | |
| 108 | @lru_cache(maxsize=None) |
| 109 | def fib(n): |
| 110 | if n < 2: |
| 111 | return n |
| 112 | return fib(n-1) + fib(n-2) |
| 113 | |
Raymond Hettinger | 17328e4 | 2013-04-06 20:27:33 -0700 | [diff] [blame] | 114 | >>> [fib(n) for n in range(16)] |
Raymond Hettinger | c79fb0e | 2010-12-01 03:45:41 +0000 | [diff] [blame] | 115 | [0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, 377, 610] |
| 116 | |
Raymond Hettinger | 17328e4 | 2013-04-06 20:27:33 -0700 | [diff] [blame] | 117 | >>> fib.cache_info() |
Raymond Hettinger | c79fb0e | 2010-12-01 03:45:41 +0000 | [diff] [blame] | 118 | CacheInfo(hits=28, misses=16, maxsize=None, currsize=16) |
| 119 | |
Georg Brandl | 2e7346a | 2010-07-31 18:09:23 +0000 | [diff] [blame] | 120 | .. versionadded:: 3.2 |
| 121 | |
Raymond Hettinger | cd9fdfd | 2011-10-20 08:57:45 -0700 | [diff] [blame] | 122 | .. versionchanged:: 3.3 |
| 123 | Added the *typed* option. |
| 124 | |
Georg Brandl | 8a1caa2 | 2010-07-29 16:01:11 +0000 | [diff] [blame] | 125 | .. decorator:: total_ordering |
Raymond Hettinger | c50846a | 2010-04-05 18:56:31 +0000 | [diff] [blame] | 126 | |
| 127 | Given a class defining one or more rich comparison ordering methods, this |
Benjamin Peterson | 08bf91c | 2010-04-11 16:12:57 +0000 | [diff] [blame] | 128 | class decorator supplies the rest. This simplifies the effort involved |
Raymond Hettinger | c50846a | 2010-04-05 18:56:31 +0000 | [diff] [blame] | 129 | in specifying all of the possible rich comparison operations: |
| 130 | |
| 131 | The class must define one of :meth:`__lt__`, :meth:`__le__`, |
| 132 | :meth:`__gt__`, or :meth:`__ge__`. |
| 133 | In addition, the class should supply an :meth:`__eq__` method. |
| 134 | |
| 135 | For example:: |
| 136 | |
| 137 | @total_ordering |
| 138 | class Student: |
Nick Coghlan | f05d981 | 2013-10-02 00:02:03 +1000 | [diff] [blame] | 139 | def _is_valid_operand(self, other): |
| 140 | return (hasattr(other, "lastname") and |
| 141 | hasattr(other, "firstname")) |
Raymond Hettinger | c50846a | 2010-04-05 18:56:31 +0000 | [diff] [blame] | 142 | def __eq__(self, other): |
Nick Coghlan | f05d981 | 2013-10-02 00:02:03 +1000 | [diff] [blame] | 143 | if not self._is_valid_operand(other): |
| 144 | return NotImplemented |
Raymond Hettinger | c50846a | 2010-04-05 18:56:31 +0000 | [diff] [blame] | 145 | return ((self.lastname.lower(), self.firstname.lower()) == |
| 146 | (other.lastname.lower(), other.firstname.lower())) |
| 147 | def __lt__(self, other): |
Nick Coghlan | f05d981 | 2013-10-02 00:02:03 +1000 | [diff] [blame] | 148 | if not self._is_valid_operand(other): |
| 149 | return NotImplemented |
Raymond Hettinger | c50846a | 2010-04-05 18:56:31 +0000 | [diff] [blame] | 150 | return ((self.lastname.lower(), self.firstname.lower()) < |
| 151 | (other.lastname.lower(), other.firstname.lower())) |
| 152 | |
Nick Coghlan | f05d981 | 2013-10-02 00:02:03 +1000 | [diff] [blame] | 153 | .. note:: |
| 154 | |
| 155 | While this decorator makes it easy to create well behaved totally |
| 156 | ordered types, it *does* come at the cost of slower execution and |
| 157 | more complex stack traces for the derived comparison methods. If |
| 158 | performance benchmarking indicates this is a bottleneck for a given |
| 159 | application, implementing all six rich comparison methods instead is |
| 160 | likely to provide an easy speed boost. |
| 161 | |
Raymond Hettinger | c50846a | 2010-04-05 18:56:31 +0000 | [diff] [blame] | 162 | .. versionadded:: 3.2 |
| 163 | |
Nick Coghlan | f05d981 | 2013-10-02 00:02:03 +1000 | [diff] [blame] | 164 | .. versionchanged:: 3.4 |
| 165 | Returning NotImplemented from the underlying comparison function for |
| 166 | unrecognised types is now supported. |
Georg Brandl | 67b21b7 | 2010-08-17 15:07:14 +0000 | [diff] [blame] | 167 | |
Georg Brandl | 036490d | 2009-05-17 13:00:36 +0000 | [diff] [blame] | 168 | .. function:: partial(func, *args, **keywords) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 169 | |
| 170 | Return a new :class:`partial` object which when called will behave like *func* |
| 171 | called with the positional arguments *args* and keyword arguments *keywords*. If |
| 172 | more arguments are supplied to the call, they are appended to *args*. If |
| 173 | additional keyword arguments are supplied, they extend and override *keywords*. |
| 174 | Roughly equivalent to:: |
| 175 | |
| 176 | def partial(func, *args, **keywords): |
| 177 | def newfunc(*fargs, **fkeywords): |
| 178 | newkeywords = keywords.copy() |
| 179 | newkeywords.update(fkeywords) |
Martin Panter | 0c0da48 | 2016-06-12 01:46:50 +0000 | [diff] [blame] | 180 | return func(*args, *fargs, **newkeywords) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 181 | newfunc.func = func |
| 182 | newfunc.args = args |
| 183 | newfunc.keywords = keywords |
| 184 | return newfunc |
| 185 | |
| 186 | The :func:`partial` is used for partial function application which "freezes" |
| 187 | some portion of a function's arguments and/or keywords resulting in a new object |
| 188 | with a simplified signature. For example, :func:`partial` can be used to create |
| 189 | a callable that behaves like the :func:`int` function where the *base* argument |
Christian Heimes | fe337bf | 2008-03-23 21:54:12 +0000 | [diff] [blame] | 190 | defaults to two: |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 191 | |
Christian Heimes | fe337bf | 2008-03-23 21:54:12 +0000 | [diff] [blame] | 192 | >>> from functools import partial |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 193 | >>> basetwo = partial(int, base=2) |
| 194 | >>> basetwo.__doc__ = 'Convert base 2 string to an int.' |
| 195 | >>> basetwo('10010') |
| 196 | 18 |
| 197 | |
| 198 | |
Nick Coghlan | f4cb48a | 2013-11-03 16:41:46 +1000 | [diff] [blame] | 199 | .. class:: partialmethod(func, *args, **keywords) |
| 200 | |
| 201 | Return a new :class:`partialmethod` descriptor which behaves |
| 202 | like :class:`partial` except that it is designed to be used as a method |
| 203 | definition rather than being directly callable. |
| 204 | |
| 205 | *func* must be a :term:`descriptor` or a callable (objects which are both, |
| 206 | like normal functions, are handled as descriptors). |
| 207 | |
| 208 | When *func* is a descriptor (such as a normal Python function, |
| 209 | :func:`classmethod`, :func:`staticmethod`, :func:`abstractmethod` or |
| 210 | another instance of :class:`partialmethod`), calls to ``__get__`` are |
| 211 | delegated to the underlying descriptor, and an appropriate |
| 212 | :class:`partial` object returned as the result. |
| 213 | |
| 214 | When *func* is a non-descriptor callable, an appropriate bound method is |
| 215 | created dynamically. This behaves like a normal Python function when |
| 216 | used as a method: the *self* argument will be inserted as the first |
| 217 | positional argument, even before the *args* and *keywords* supplied to |
| 218 | the :class:`partialmethod` constructor. |
| 219 | |
| 220 | Example:: |
| 221 | |
| 222 | >>> class Cell(object): |
Benjamin Peterson | 3a43403 | 2014-03-30 15:07:09 -0400 | [diff] [blame] | 223 | ... def __init__(self): |
| 224 | ... self._alive = False |
Nick Coghlan | f4cb48a | 2013-11-03 16:41:46 +1000 | [diff] [blame] | 225 | ... @property |
| 226 | ... def alive(self): |
| 227 | ... return self._alive |
| 228 | ... def set_state(self, state): |
| 229 | ... self._alive = bool(state) |
Nick Coghlan | 3daaf5f | 2013-11-04 23:32:16 +1000 | [diff] [blame] | 230 | ... set_alive = partialmethod(set_state, True) |
| 231 | ... set_dead = partialmethod(set_state, False) |
Nick Coghlan | f4cb48a | 2013-11-03 16:41:46 +1000 | [diff] [blame] | 232 | ... |
| 233 | >>> c = Cell() |
| 234 | >>> c.alive |
| 235 | False |
| 236 | >>> c.set_alive() |
| 237 | >>> c.alive |
| 238 | True |
| 239 | |
| 240 | .. versionadded:: 3.4 |
| 241 | |
| 242 | |
Georg Brandl | 58f9e4f | 2008-04-19 22:18:33 +0000 | [diff] [blame] | 243 | .. function:: reduce(function, iterable[, initializer]) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 244 | |
| 245 | Apply *function* of two arguments cumulatively to the items of *sequence*, from |
| 246 | left to right, so as to reduce the sequence to a single value. For example, |
| 247 | ``reduce(lambda x, y: x+y, [1, 2, 3, 4, 5])`` calculates ``((((1+2)+3)+4)+5)``. |
| 248 | The left argument, *x*, is the accumulated value and the right argument, *y*, is |
| 249 | the update value from the *sequence*. If the optional *initializer* is present, |
| 250 | it is placed before the items of the sequence in the calculation, and serves as |
| 251 | a default when the sequence is empty. If *initializer* is not given and |
| 252 | *sequence* contains only one item, the first item is returned. |
| 253 | |
Raymond Hettinger | 558dcf3 | 2014-12-16 18:16:57 -0800 | [diff] [blame] | 254 | Roughly equivalent to:: |
Raymond Hettinger | 6480168 | 2013-10-12 16:04:17 -0700 | [diff] [blame] | 255 | |
| 256 | def reduce(function, iterable, initializer=None): |
| 257 | it = iter(iterable) |
| 258 | if initializer is None: |
| 259 | value = next(it) |
| 260 | else: |
| 261 | value = initializer |
| 262 | for element in it: |
| 263 | value = function(value, element) |
| 264 | return value |
| 265 | |
Gerrit Holl | bd81cbd | 2018-07-04 23:26:32 +0100 | [diff] [blame] | 266 | See :func:`itertools.accumulate` for an iterator that yields all intermediate |
| 267 | values. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 268 | |
Daisuke Miyakawa | 0e61e67 | 2017-10-12 23:39:43 +0900 | [diff] [blame] | 269 | .. decorator:: singledispatch |
Łukasz Langa | 6f69251 | 2013-06-05 12:20:24 +0200 | [diff] [blame] | 270 | |
Daisuke Miyakawa | 0e61e67 | 2017-10-12 23:39:43 +0900 | [diff] [blame] | 271 | Transform a function into a :term:`single-dispatch <single |
Łukasz Langa | fdcf2b7 | 2013-06-07 22:54:03 +0200 | [diff] [blame] | 272 | dispatch>` :term:`generic function`. |
Łukasz Langa | 6f69251 | 2013-06-05 12:20:24 +0200 | [diff] [blame] | 273 | |
| 274 | To define a generic function, decorate it with the ``@singledispatch`` |
| 275 | decorator. Note that the dispatch happens on the type of the first argument, |
| 276 | create your function accordingly:: |
| 277 | |
| 278 | >>> from functools import singledispatch |
| 279 | >>> @singledispatch |
| 280 | ... def fun(arg, verbose=False): |
| 281 | ... if verbose: |
| 282 | ... print("Let me just say,", end=" ") |
| 283 | ... print(arg) |
| 284 | |
| 285 | To add overloaded implementations to the function, use the :func:`register` |
Łukasz Langa | e569753 | 2017-12-11 13:56:31 -0800 | [diff] [blame] | 286 | attribute of the generic function. It is a decorator. For functions |
| 287 | annotated with types, the decorator will infer the type of the first |
| 288 | argument automatically:: |
Łukasz Langa | 6f69251 | 2013-06-05 12:20:24 +0200 | [diff] [blame] | 289 | |
Łukasz Langa | e569753 | 2017-12-11 13:56:31 -0800 | [diff] [blame] | 290 | >>> @fun.register |
| 291 | ... def _(arg: int, verbose=False): |
Łukasz Langa | 6f69251 | 2013-06-05 12:20:24 +0200 | [diff] [blame] | 292 | ... if verbose: |
| 293 | ... print("Strength in numbers, eh?", end=" ") |
| 294 | ... print(arg) |
| 295 | ... |
Łukasz Langa | e569753 | 2017-12-11 13:56:31 -0800 | [diff] [blame] | 296 | >>> @fun.register |
| 297 | ... def _(arg: list, verbose=False): |
Łukasz Langa | 6f69251 | 2013-06-05 12:20:24 +0200 | [diff] [blame] | 298 | ... if verbose: |
| 299 | ... print("Enumerate this:") |
| 300 | ... for i, elem in enumerate(arg): |
| 301 | ... print(i, elem) |
| 302 | |
Łukasz Langa | e569753 | 2017-12-11 13:56:31 -0800 | [diff] [blame] | 303 | For code which doesn't use type annotations, the appropriate type |
| 304 | argument can be passed explicitly to the decorator itself:: |
| 305 | |
| 306 | >>> @fun.register(complex) |
| 307 | ... def _(arg, verbose=False): |
| 308 | ... if verbose: |
| 309 | ... print("Better than complicated.", end=" ") |
| 310 | ... print(arg.real, arg.imag) |
| 311 | ... |
| 312 | |
| 313 | |
Łukasz Langa | 6f69251 | 2013-06-05 12:20:24 +0200 | [diff] [blame] | 314 | To enable registering lambdas and pre-existing functions, the |
| 315 | :func:`register` attribute can be used in a functional form:: |
| 316 | |
| 317 | >>> def nothing(arg, verbose=False): |
| 318 | ... print("Nothing.") |
| 319 | ... |
| 320 | >>> fun.register(type(None), nothing) |
| 321 | |
| 322 | The :func:`register` attribute returns the undecorated function which |
| 323 | enables decorator stacking, pickling, as well as creating unit tests for |
| 324 | each variant independently:: |
| 325 | |
| 326 | >>> @fun.register(float) |
| 327 | ... @fun.register(Decimal) |
| 328 | ... def fun_num(arg, verbose=False): |
| 329 | ... if verbose: |
| 330 | ... print("Half of your number:", end=" ") |
| 331 | ... print(arg / 2) |
| 332 | ... |
| 333 | >>> fun_num is fun |
| 334 | False |
| 335 | |
| 336 | When called, the generic function dispatches on the type of the first |
| 337 | argument:: |
| 338 | |
| 339 | >>> fun("Hello, world.") |
| 340 | Hello, world. |
| 341 | >>> fun("test.", verbose=True) |
| 342 | Let me just say, test. |
| 343 | >>> fun(42, verbose=True) |
| 344 | Strength in numbers, eh? 42 |
| 345 | >>> fun(['spam', 'spam', 'eggs', 'spam'], verbose=True) |
| 346 | Enumerate this: |
| 347 | 0 spam |
| 348 | 1 spam |
| 349 | 2 eggs |
| 350 | 3 spam |
| 351 | >>> fun(None) |
| 352 | Nothing. |
| 353 | >>> fun(1.23) |
| 354 | 0.615 |
| 355 | |
| 356 | Where there is no registered implementation for a specific type, its |
| 357 | method resolution order is used to find a more generic implementation. |
| 358 | The original function decorated with ``@singledispatch`` is registered |
| 359 | for the base ``object`` type, which means it is used if no better |
| 360 | implementation is found. |
| 361 | |
| 362 | To check which implementation will the generic function choose for |
| 363 | a given type, use the ``dispatch()`` attribute:: |
| 364 | |
| 365 | >>> fun.dispatch(float) |
| 366 | <function fun_num at 0x1035a2840> |
| 367 | >>> fun.dispatch(dict) # note: default implementation |
| 368 | <function fun at 0x103fe0000> |
| 369 | |
| 370 | To access all registered implementations, use the read-only ``registry`` |
| 371 | attribute:: |
| 372 | |
| 373 | >>> fun.registry.keys() |
| 374 | dict_keys([<class 'NoneType'>, <class 'int'>, <class 'object'>, |
| 375 | <class 'decimal.Decimal'>, <class 'list'>, |
| 376 | <class 'float'>]) |
| 377 | >>> fun.registry[float] |
| 378 | <function fun_num at 0x1035a2840> |
| 379 | >>> fun.registry[object] |
| 380 | <function fun at 0x103fe0000> |
| 381 | |
| 382 | .. versionadded:: 3.4 |
| 383 | |
Łukasz Langa | e569753 | 2017-12-11 13:56:31 -0800 | [diff] [blame] | 384 | .. versionchanged:: 3.7 |
| 385 | The :func:`register` attribute supports using type annotations. |
| 386 | |
Łukasz Langa | 6f69251 | 2013-06-05 12:20:24 +0200 | [diff] [blame] | 387 | |
Ethan Smith | c651275 | 2018-05-26 16:38:33 -0400 | [diff] [blame] | 388 | .. class:: singledispatchmethod(func) |
| 389 | |
| 390 | Transform a method into a :term:`single-dispatch <single |
| 391 | dispatch>` :term:`generic function`. |
| 392 | |
| 393 | To define a generic method, decorate it with the ``@singledispatchmethod`` |
| 394 | decorator. Note that the dispatch happens on the type of the first non-self |
| 395 | or non-cls argument, create your function accordingly:: |
| 396 | |
| 397 | class Negator: |
| 398 | @singledispatchmethod |
| 399 | def neg(self, arg): |
| 400 | raise NotImplementedError("Cannot negate a") |
| 401 | |
| 402 | @neg.register |
| 403 | def _(self, arg: int): |
| 404 | return -arg |
| 405 | |
| 406 | @neg.register |
| 407 | def _(self, arg: bool): |
| 408 | return not arg |
| 409 | |
| 410 | ``@singledispatchmethod`` supports nesting with other decorators such as |
| 411 | ``@classmethod``. Note that to allow for ``dispatcher.register``, |
| 412 | ``singledispatchmethod`` must be the *outer most* decorator. Here is the |
| 413 | ``Negator`` class with the ``neg`` methods being class bound:: |
| 414 | |
| 415 | class Negator: |
| 416 | @singledispatchmethod |
| 417 | @classmethod |
| 418 | def neg(cls, arg): |
| 419 | raise NotImplementedError("Cannot negate a") |
| 420 | |
| 421 | @neg.register |
| 422 | @classmethod |
| 423 | def _(cls, arg: int): |
| 424 | return -arg |
| 425 | |
| 426 | @neg.register |
| 427 | @classmethod |
| 428 | def _(cls, arg: bool): |
| 429 | return not arg |
| 430 | |
| 431 | The same pattern can be used for other similar decorators: ``staticmethod``, |
| 432 | ``abstractmethod``, and others. |
| 433 | |
Georg Brandl | 036490d | 2009-05-17 13:00:36 +0000 | [diff] [blame] | 434 | .. function:: update_wrapper(wrapper, wrapped, assigned=WRAPPER_ASSIGNMENTS, updated=WRAPPER_UPDATES) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 435 | |
| 436 | Update a *wrapper* function to look like the *wrapped* function. The optional |
| 437 | arguments are tuples to specify which attributes of the original function are |
| 438 | assigned directly to the matching attributes on the wrapper function and which |
| 439 | attributes of the wrapper function are updated with the corresponding attributes |
| 440 | from the original function. The default values for these arguments are the |
Berker Peksag | 472233e | 2016-04-18 21:20:50 +0300 | [diff] [blame] | 441 | module level constants ``WRAPPER_ASSIGNMENTS`` (which assigns to the wrapper |
| 442 | function's ``__module__``, ``__name__``, ``__qualname__``, ``__annotations__`` |
| 443 | and ``__doc__``, the documentation string) and ``WRAPPER_UPDATES`` (which |
| 444 | updates the wrapper function's ``__dict__``, i.e. the instance dictionary). |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 445 | |
Nick Coghlan | 9887683 | 2010-08-17 06:17:18 +0000 | [diff] [blame] | 446 | To allow access to the original function for introspection and other purposes |
| 447 | (e.g. bypassing a caching decorator such as :func:`lru_cache`), this function |
Nick Coghlan | 24c05bc | 2013-07-15 21:13:08 +1000 | [diff] [blame] | 448 | automatically adds a ``__wrapped__`` attribute to the wrapper that refers to |
| 449 | the function being wrapped. |
Nick Coghlan | 9887683 | 2010-08-17 06:17:18 +0000 | [diff] [blame] | 450 | |
Christian Heimes | d8654cf | 2007-12-02 15:22:16 +0000 | [diff] [blame] | 451 | The main intended use for this function is in :term:`decorator` functions which |
| 452 | wrap the decorated function and return the wrapper. If the wrapper function is |
| 453 | not updated, the metadata of the returned function will reflect the wrapper |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 454 | definition rather than the original function definition, which is typically less |
| 455 | than helpful. |
| 456 | |
Nick Coghlan | 9887683 | 2010-08-17 06:17:18 +0000 | [diff] [blame] | 457 | :func:`update_wrapper` may be used with callables other than functions. Any |
| 458 | attributes named in *assigned* or *updated* that are missing from the object |
| 459 | being wrapped are ignored (i.e. this function will not attempt to set them |
| 460 | on the wrapper function). :exc:`AttributeError` is still raised if the |
| 461 | wrapper function itself is missing any attributes named in *updated*. |
| 462 | |
| 463 | .. versionadded:: 3.2 |
Georg Brandl | 9e25701 | 2010-08-17 14:11:59 +0000 | [diff] [blame] | 464 | Automatic addition of the ``__wrapped__`` attribute. |
Nick Coghlan | 9887683 | 2010-08-17 06:17:18 +0000 | [diff] [blame] | 465 | |
| 466 | .. versionadded:: 3.2 |
Georg Brandl | 9e25701 | 2010-08-17 14:11:59 +0000 | [diff] [blame] | 467 | Copying of the ``__annotations__`` attribute by default. |
Nick Coghlan | 9887683 | 2010-08-17 06:17:18 +0000 | [diff] [blame] | 468 | |
| 469 | .. versionchanged:: 3.2 |
Georg Brandl | 9e25701 | 2010-08-17 14:11:59 +0000 | [diff] [blame] | 470 | Missing attributes no longer trigger an :exc:`AttributeError`. |
| 471 | |
Nick Coghlan | 24c05bc | 2013-07-15 21:13:08 +1000 | [diff] [blame] | 472 | .. versionchanged:: 3.4 |
| 473 | The ``__wrapped__`` attribute now always refers to the wrapped |
| 474 | function, even if that function defined a ``__wrapped__`` attribute. |
| 475 | (see :issue:`17482`) |
| 476 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 477 | |
Georg Brandl | 8a1caa2 | 2010-07-29 16:01:11 +0000 | [diff] [blame] | 478 | .. decorator:: wraps(wrapped, assigned=WRAPPER_ASSIGNMENTS, updated=WRAPPER_UPDATES) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 479 | |
Ezio Melotti | 67f6d5f | 2014-08-05 08:14:28 +0300 | [diff] [blame] | 480 | This is a convenience function for invoking :func:`update_wrapper` as a |
| 481 | function decorator when defining a wrapper function. It is equivalent to |
| 482 | ``partial(update_wrapper, wrapped=wrapped, assigned=assigned, updated=updated)``. |
| 483 | For example:: |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 484 | |
Christian Heimes | fe337bf | 2008-03-23 21:54:12 +0000 | [diff] [blame] | 485 | >>> from functools import wraps |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 486 | >>> def my_decorator(f): |
| 487 | ... @wraps(f) |
| 488 | ... def wrapper(*args, **kwds): |
Georg Brandl | 6911e3c | 2007-09-04 07:15:32 +0000 | [diff] [blame] | 489 | ... print('Calling decorated function') |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 490 | ... return f(*args, **kwds) |
| 491 | ... return wrapper |
| 492 | ... |
| 493 | >>> @my_decorator |
| 494 | ... def example(): |
| 495 | ... """Docstring""" |
Georg Brandl | 6911e3c | 2007-09-04 07:15:32 +0000 | [diff] [blame] | 496 | ... print('Called example function') |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 497 | ... |
| 498 | >>> example() |
| 499 | Calling decorated function |
| 500 | Called example function |
| 501 | >>> example.__name__ |
| 502 | 'example' |
| 503 | >>> example.__doc__ |
| 504 | 'Docstring' |
| 505 | |
| 506 | Without the use of this decorator factory, the name of the example function |
| 507 | would have been ``'wrapper'``, and the docstring of the original :func:`example` |
| 508 | would have been lost. |
| 509 | |
| 510 | |
| 511 | .. _partial-objects: |
| 512 | |
| 513 | :class:`partial` Objects |
| 514 | ------------------------ |
| 515 | |
| 516 | :class:`partial` objects are callable objects created by :func:`partial`. They |
| 517 | have three read-only attributes: |
| 518 | |
| 519 | |
| 520 | .. attribute:: partial.func |
| 521 | |
| 522 | A callable object or function. Calls to the :class:`partial` object will be |
| 523 | forwarded to :attr:`func` with new arguments and keywords. |
| 524 | |
| 525 | |
| 526 | .. attribute:: partial.args |
| 527 | |
| 528 | The leftmost positional arguments that will be prepended to the positional |
| 529 | arguments provided to a :class:`partial` object call. |
| 530 | |
| 531 | |
| 532 | .. attribute:: partial.keywords |
| 533 | |
| 534 | The keyword arguments that will be supplied when the :class:`partial` object is |
| 535 | called. |
| 536 | |
| 537 | :class:`partial` objects are like :class:`function` objects in that they are |
| 538 | callable, weak referencable, and can have attributes. There are some important |
Martin Panter | bae5d81 | 2016-06-18 03:57:31 +0000 | [diff] [blame] | 539 | differences. For instance, the :attr:`~definition.__name__` and :attr:`__doc__` attributes |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 540 | are not created automatically. Also, :class:`partial` objects defined in |
| 541 | classes behave like static methods and do not transform into bound methods |
| 542 | during instance attribute look-up. |