blob: 4e9fad30d31c4f1efb83aa0724c393585258ab08 [file] [log] [blame]
Raymond Hettinger8d3d7312020-10-23 12:55:39 -07001.. _descriptorhowto:
2
Georg Brandl45cceeb2010-05-19 21:39:51 +00003======================
4Descriptor HowTo Guide
5======================
6
7:Author: Raymond Hettinger
8:Contact: <python at rcn dot com>
9
10.. Contents::
11
Raymond Hettinger8d3d7312020-10-23 12:55:39 -070012
13:term:`Descriptors <descriptor>` let objects customize attribute lookup,
14storage, and deletion.
15
16This HowTo guide has three major sections:
17
181) The "primer" gives a basic overview, moving gently from simple examples,
19 adding one feature at a time. It is a great place to start.
20
212) The second section shows a complete, practical descriptor example. If you
22 already know the basics, start there.
23
243) The third section provides a more technical tutorial that goes into the
25 detailed mechanics of how descriptors work. Most people don't need this
26 level of detail.
27
28
29Primer
30^^^^^^
31
32In this primer, we start with most basic possible example and then we'll add
33new capabilities one by one.
34
35
36Simple example: A descriptor that returns a constant
37----------------------------------------------------
38
39The :class:`Ten` class is a descriptor that always returns the constant ``10``::
40
41
42 class Ten:
43 def __get__(self, obj, objtype=None):
44 return 10
45
46To use the descriptor, it must be stored as a class variable in another class::
47
48 class A:
49 x = 5 # Regular class attribute
50 y = Ten() # Descriptor
51
52An interactive session shows the difference between normal attribute lookup
53and descriptor lookup::
54
55 >>> a = A() # Make an instance of class A
56 >>> a.x # Normal attribute lookup
57 5
58 >>> a.y # Descriptor lookup
59 10
60
61In the ``a.x`` attribute lookup, the dot operator finds the value ``5`` stored
62in the class dictionary. In the ``a.y`` descriptor lookup, the dot operator
63calls the descriptor's :meth:`__get__()` method. That method returns ``10``.
64Note that the value ``10`` is not stored in either the class dictionary or the
65instance dictionary. Instead, the value ``10`` is computed on demand.
66
67This example shows how a simple descriptor works, but it isn't very useful.
68For retrieving constants, normal attribute lookup would be better.
69
70In the next section, we'll create something more useful, a dynamic lookup.
71
72
73Dynamic lookups
74---------------
75
76Interesting descriptors typically run computations instead of doing lookups::
77
78
79 import os
80
81 class DirectorySize:
82
83 def __get__(self, obj, objtype=None):
84 return len(os.listdir(obj.dirname))
85
86 class Directory:
87
88 size = DirectorySize() # Descriptor
89
90 def __init__(self, dirname):
91 self.dirname = dirname # Regular instance attribute
92
93An interactive session shows that the lookup is dynamic — it computes
94different, updated answers each time::
95
96 >>> g = Directory('games')
97 >>> s = Directory('songs')
98 >>> g.size # The games directory has three files
99 3
100 >>> os.system('touch games/newfile') # Add a fourth file to the directory
101 0
102 >>> g.size
103 4
104 >>> s.size # The songs directory has twenty files
105 20
106
107Besides showing how descriptors can run computations, this example also
108reveals the purpose of the parameters to :meth:`__get__`. The *self*
109parameter is *size*, an instance of *DirectorySize*. The *obj* parameter is
110either *g* or *s*, an instance of *Directory*. It is *obj* parameter that
111lets the :meth:`__get__` method learn the target directory. The *objtype*
112parameter is the class *Directory*.
113
114
115Managed attributes
116------------------
117
118A popular use for descriptors is managing access to instance data. The
119descriptor is assigned to a public attribute in the class dictionary while the
120actual data is stored as a private attribute in the instance dictionary. The
121descriptor's :meth:`__get__` and :meth:`__set__` methods are triggered when
122the public attribute is accessed.
123
124In the following example, *age* is the public attribute and *_age* is the
125private attribute. When the public attribute is accessed, the descriptor logs
126the lookup or update::
127
128 import logging
129
130 logging.basicConfig(level=logging.INFO)
131
132 class LoggedAgeAccess:
133
134 def __get__(self, obj, objtype=None):
135 value = obj._age
136 logging.info('Accessing %r giving %r', 'age', value)
137 return value
138
139 def __set__(self, obj, value):
140 logging.info('Updating %r to %r', 'age', value)
141 obj._age = value
142
143 class Person:
144
145 age = LoggedAgeAccess() # Descriptor
146
147 def __init__(self, name, age):
148 self.name = name # Regular instance attribute
149 self.age = age # Calls the descriptor
150
151 def birthday(self):
152 self.age += 1 # Calls both __get__() and __set__()
153
154
155An interactive session shows that all access to the managed attribute *age* is
156logged, but that the regular attribute *name* is not logged::
157
158 >>> mary = Person('Mary M', 30) # The initial age update is logged
159 INFO:root:Updating 'age' to 30
160 >>> dave = Person('David D', 40)
161 INFO:root:Updating 'age' to 40
162
163 >>> vars(mary) # The actual data is in a private attribute
164 {'name': 'Mary M', '_age': 30}
165 >>> vars(dave)
166 {'name': 'David D', '_age': 40}
167
168 >>> mary.age # Access the data and log the lookup
169 INFO:root:Accessing 'age' giving 30
170 30
171 >>> mary.birthday() # Updates are logged as well
172 INFO:root:Accessing 'age' giving 30
173 INFO:root:Updating 'age' to 31
174
175 >>> dave.name # Regular attribute lookup isn't logged
176 'David D'
177 >>> dave.age # Only the managed attribute is logged
178 INFO:root:Accessing 'age' giving 40
179 40
180
181One major issue with this example is the private name *_age* is hardwired in
182the *LoggedAgeAccess* class. That means that each instance can only have one
183logged attribute and that its name is unchangeable. In the next example,
184we'll fix that problem.
185
186
187Customized Names
188----------------
189
190When a class uses descriptors, it can inform each descriptor about what
191variable name was used.
192
193In this example, the :class:`Person` class has two descriptor instances,
194*name* and *age*. When the :class:`Person` class is defined, it makes a
195callback to :meth:`__set_name__` in *LoggedAccess* so that the field names can
196be recorded, giving each descriptor its own *public_name* and *private_name*::
197
198 import logging
199
200 logging.basicConfig(level=logging.INFO)
201
202 class LoggedAccess:
203
204 def __set_name__(self, owner, name):
205 self.public_name = name
206 self.private_name = f'_{name}'
207
208 def __get__(self, obj, objtype=None):
209 value = getattr(obj, self.private_name)
210 logging.info('Accessing %r giving %r', self.public_name, value)
211 return value
212
213 def __set__(self, obj, value):
214 logging.info('Updating %r to %r', self.public_name, value)
215 setattr(obj, self.private_name, value)
216
217 class Person:
218
219 name = LoggedAccess() # First descriptor
220 age = LoggedAccess() # Second descriptor
221
222 def __init__(self, name, age):
223 self.name = name # Calls the first descriptor
224 self.age = age # Calls the second descriptor
225
226 def birthday(self):
227 self.age += 1
228
229An interactive session shows that the :class:`Person` class has called
230:meth:`__set_name__` so that the field names would be recorded. Here
231we call :func:`vars` to lookup the descriptor without triggering it::
232
233 >>> vars(vars(Person)['name'])
234 {'public_name': 'name', 'private_name': '_name'}
235 >>> vars(vars(Person)['age'])
236 {'public_name': 'age', 'private_name': '_age'}
237
238The new class now logs access to both *name* and *age*::
239
240 >>> pete = Person('Peter P', 10)
241 INFO:root:Updating 'name' to 'Peter P'
242 INFO:root:Updating 'age' to 10
243 >>> kate = Person('Catherine C', 20)
244 INFO:root:Updating 'name' to 'Catherine C'
245 INFO:root:Updating 'age' to 20
246
247The two *Person* instances contain only the private names::
248
249 >>> vars(pete)
250 {'_name': 'Peter P', '_age': 10}
251 >>> vars(kate)
252 {'_name': 'Catherine C', '_age': 20}
253
254
255Closing thoughts
256----------------
257
258A :term:`descriptor` is what we call any object that defines :meth:`__get__`,
259:meth:`__set__`, or :meth:`__delete__`.
260
261Descriptors get invoked by the dot operator during attribute lookup. If a
262descriptor is accessed indirectly with ``vars(some_class)[descriptor_name]``,
263the descriptor instance is returned without invoking it.
264
265Descriptors only work when used as class variables. When put in instances,
266they have no effect.
267
268The main motivation for descriptors is to provide a hook allowing objects
269stored in class variables to control what happens during dotted lookup.
270
271Traditionally, the calling class controls what happens during lookup.
272Descriptors invert that relationship and allow the data being looked-up to
273have a say in the matter.
274
275Descriptors are used throughout the language. It is how functions turn into
276bound methods. Common tools like :func:`classmethod`, :func:`staticmethod`,
277:func:`property`, and :func:`functools.cached_property` are all implemented as
278descriptors.
279
280
281Complete Practical Example
282^^^^^^^^^^^^^^^^^^^^^^^^^^
283
284In this example, we create a practical and powerful tool for locating
285notoriously hard to find data corruption bugs.
286
287
288Validator class
289---------------
290
291A validator is a descriptor for managed attribute access. Prior to storing
292any data, it verifies that the new value meets various type and range
293restrictions. If those restrictions aren't met, it raises an exception to
294prevents data corruption at its source.
295
296This :class:`Validator` class is both an :term:`abstract base class` and a
297managed attribute descriptor::
298
299 from abc import ABC, abstractmethod
300
301 class Validator(ABC):
302
303 def __set_name__(self, owner, name):
304 self.private_name = f'_{name}'
305
306 def __get__(self, obj, objtype=None):
307 return getattr(obj, self.private_name)
308
309 def __set__(self, obj, value):
310 self.validate(value)
311 setattr(obj, self.private_name, value)
312
313 @abstractmethod
314 def validate(self, value):
315 pass
316
317Custom validators need to subclass from :class:`Validator` and supply a
318:meth:`validate` method to test various restrictions as needed.
319
320
321Custom validators
322-----------------
323
324Here are three practical data validation utilities:
325
3261) :class:`OneOf` verifies that a value is one of a restricted set of options.
327
3282) :class:`Number` verifies that a value is either an :class:`int` or
329 :class:`float`. Optionally, it verifies that a value is between a given
330 minimum or maximum.
331
3323) :class:`String` verifies that a value is a :class:`str`. Optionally, it
333 validates a given minimum or maximum length. Optionally, it can test for
334 another predicate as well.
335
336::
337
338 class OneOf(Validator):
339
340 def __init__(self, *options):
341 self.options = set(options)
342
343 def validate(self, value):
344 if value not in self.options:
345 raise ValueError(f'Expected {value!r} to be one of {self.options!r}')
346
347 class Number(Validator):
348
349 def __init__(self, minvalue=None, maxvalue=None):
350 self.minvalue = minvalue
351 self.maxvalue = maxvalue
352
353 def validate(self, value):
354 if not isinstance(value, (int, float)):
355 raise TypeError(f'Expected {value!r} to be an int or float')
356 if self.minvalue is not None and value < self.minvalue:
357 raise ValueError(
358 f'Expected {value!r} to be at least {self.minvalue!r}'
359 )
360 if self.maxvalue is not None and value > self.maxvalue:
361 raise ValueError(
362 f'Expected {value!r} to be no more than {self.maxvalue!r}'
363 )
364
365 class String(Validator):
366
367 def __init__(self, minsize=None, maxsize=None, predicate=None):
368 self.minsize = minsize
369 self.maxsize = maxsize
370 self.predicate = predicate
371
372 def validate(self, value):
373 if not isinstance(value, str):
374 raise TypeError(f'Expected {value!r} to be an str')
375 if self.minsize is not None and len(value) < self.minsize:
376 raise ValueError(
377 f'Expected {value!r} to be no smaller than {self.minsize!r}'
378 )
379 if self.maxsize is not None and len(value) > self.maxsize:
380 raise ValueError(
381 f'Expected {value!r} to be no bigger than {self.maxsize!r}'
382 )
383 if self.predicate is not None and not self.predicate(value):
384 raise ValueError(
385 f'Expected {self.predicate} to be true for {value!r}'
386 )
387
388
389Practical use
390-------------
391
392Here's how the data validators can be used in a real class::
393
394 class Component:
395
396 name = String(minsize=3, maxsize=10, predicate=str.isupper)
397 kind = OneOf('plastic', 'metal')
398 quantity = Number(minvalue=0)
399
400 def __init__(self, name, kind, quantity):
401 self.name = name
402 self.kind = kind
403 self.quantity = quantity
404
405The descriptors prevent invalid instances from being created::
406
407 Component('WIDGET', 'metal', 5) # Allowed.
408 Component('Widget', 'metal', 5) # Blocked: 'Widget' is not all uppercase
409 Component('WIDGET', 'metle', 5) # Blocked: 'metle' is misspelled
410 Component('WIDGET', 'metal', -5) # Blocked: -5 is negative
411 Component('WIDGET', 'metal', 'V') # Blocked: 'V' isn't a number
412
413
414Technical Tutorial
415^^^^^^^^^^^^^^^^^^
416
417What follows is a more technical tutorial for the mechanics and details of how
418descriptors work.
419
420
Georg Brandl45cceeb2010-05-19 21:39:51 +0000421Abstract
422--------
423
424Defines descriptors, summarizes the protocol, and shows how descriptors are
Andrés Delfino271818f2018-09-14 14:13:09 -0300425called. Examines a custom descriptor and several built-in Python descriptors
Georg Brandl45cceeb2010-05-19 21:39:51 +0000426including functions, properties, static methods, and class methods. Shows how
427each works by giving a pure Python equivalent and a sample application.
428
429Learning about descriptors not only provides access to a larger toolset, it
430creates a deeper understanding of how Python works and an appreciation for the
431elegance of its design.
432
433
434Definition and Introduction
435---------------------------
436
437In general, a descriptor is an object attribute with "binding behavior", one
438whose attribute access has been overridden by methods in the descriptor
439protocol. Those methods are :meth:`__get__`, :meth:`__set__`, and
440:meth:`__delete__`. If any of those methods are defined for an object, it is
441said to be a descriptor.
442
443The default behavior for attribute access is to get, set, or delete the
444attribute from an object's dictionary. For instance, ``a.x`` has a lookup chain
445starting with ``a.__dict__['x']``, then ``type(a).__dict__['x']``, and
446continuing through the base classes of ``type(a)`` excluding metaclasses. If the
447looked-up value is an object defining one of the descriptor methods, then Python
448may override the default behavior and invoke the descriptor method instead.
449Where this occurs in the precedence chain depends on which descriptor methods
Florent Xiclunaaa6c1d22011-12-12 18:54:29 +0100450were defined.
Georg Brandl45cceeb2010-05-19 21:39:51 +0000451
452Descriptors are a powerful, general purpose protocol. They are the mechanism
Raymond Hettinger8d3d7312020-10-23 12:55:39 -0700453behind properties, methods, static methods, class methods, and
454:func:`super()`. They are used throughout Python itself. Descriptors
455simplify the underlying C code and offer a flexible set of new tools for
456everyday Python programs.
Georg Brandl45cceeb2010-05-19 21:39:51 +0000457
458
459Descriptor Protocol
460-------------------
461
NotAFile28ea4c22018-09-10 23:35:38 +0200462``descr.__get__(self, obj, type=None) -> value``
Georg Brandl45cceeb2010-05-19 21:39:51 +0000463
NotAFile28ea4c22018-09-10 23:35:38 +0200464``descr.__set__(self, obj, value) -> None``
Georg Brandl45cceeb2010-05-19 21:39:51 +0000465
NotAFile28ea4c22018-09-10 23:35:38 +0200466``descr.__delete__(self, obj) -> None``
Georg Brandl45cceeb2010-05-19 21:39:51 +0000467
468That is all there is to it. Define any of these methods and an object is
469considered a descriptor and can override default behavior upon being looked up
470as an attribute.
471
Aaron Hall, MBA4054b172018-05-20 19:46:42 -0400472If an object defines :meth:`__set__` or :meth:`__delete__`, it is considered
Georg Brandl45cceeb2010-05-19 21:39:51 +0000473a data descriptor. Descriptors that only define :meth:`__get__` are called
474non-data descriptors (they are typically used for methods but other uses are
475possible).
476
477Data and non-data descriptors differ in how overrides are calculated with
478respect to entries in an instance's dictionary. If an instance's dictionary
479has an entry with the same name as a data descriptor, the data descriptor
480takes precedence. If an instance's dictionary has an entry with the same
481name as a non-data descriptor, the dictionary entry takes precedence.
482
483To make a read-only data descriptor, define both :meth:`__get__` and
484:meth:`__set__` with the :meth:`__set__` raising an :exc:`AttributeError` when
485called. Defining the :meth:`__set__` method with an exception raising
486placeholder is enough to make it a data descriptor.
487
488
489Invoking Descriptors
490--------------------
491
492A descriptor can be called directly by its method name. For example,
493``d.__get__(obj)``.
494
495Alternatively, it is more common for a descriptor to be invoked automatically
496upon attribute access. For example, ``obj.d`` looks up ``d`` in the dictionary
497of ``obj``. If ``d`` defines the method :meth:`__get__`, then ``d.__get__(obj)``
498is invoked according to the precedence rules listed below.
499
500The details of invocation depend on whether ``obj`` is an object or a class.
Georg Brandl45cceeb2010-05-19 21:39:51 +0000501
502For objects, the machinery is in :meth:`object.__getattribute__` which
503transforms ``b.x`` into ``type(b).__dict__['x'].__get__(b, type(b))``. The
504implementation works through a precedence chain that gives data descriptors
505priority over instance variables, instance variables priority over non-data
Benjamin Peterson57fb11b2014-10-06 21:10:25 -0400506descriptors, and assigns lowest priority to :meth:`__getattr__` if provided.
507The full C implementation can be found in :c:func:`PyObject_GenericGetAttr()` in
508:source:`Objects/object.c`.
Georg Brandl45cceeb2010-05-19 21:39:51 +0000509
510For classes, the machinery is in :meth:`type.__getattribute__` which transforms
511``B.x`` into ``B.__dict__['x'].__get__(None, B)``. In pure Python, it looks
512like::
513
514 def __getattribute__(self, key):
515 "Emulate type_getattro() in Objects/typeobject.c"
516 v = object.__getattribute__(self, key)
517 if hasattr(v, '__get__'):
Serhiy Storchakadba90392016-05-10 12:01:23 +0300518 return v.__get__(None, self)
Georg Brandl45cceeb2010-05-19 21:39:51 +0000519 return v
520
521The important points to remember are:
522
523* descriptors are invoked by the :meth:`__getattribute__` method
524* overriding :meth:`__getattribute__` prevents automatic descriptor calls
Georg Brandl45cceeb2010-05-19 21:39:51 +0000525* :meth:`object.__getattribute__` and :meth:`type.__getattribute__` make
526 different calls to :meth:`__get__`.
527* data descriptors always override instance dictionaries.
528* non-data descriptors may be overridden by instance dictionaries.
529
530The object returned by ``super()`` also has a custom :meth:`__getattribute__`
Raymond Hettinger03acba62019-08-28 22:59:43 -0700531method for invoking descriptors. The attribute lookup ``super(B, obj).m`` searches
Georg Brandl45cceeb2010-05-19 21:39:51 +0000532``obj.__class__.__mro__`` for the base class ``A`` immediately following ``B``
Benjamin Peterson910a6652013-10-18 12:57:55 -0400533and then returns ``A.__dict__['m'].__get__(obj, B)``. If not a descriptor,
Georg Brandl45cceeb2010-05-19 21:39:51 +0000534``m`` is returned unchanged. If not in the dictionary, ``m`` reverts to a
535search using :meth:`object.__getattribute__`.
536
Florent Xiclunaaa6c1d22011-12-12 18:54:29 +0100537The implementation details are in :c:func:`super_getattro()` in
Benjamin Peterson57fb11b2014-10-06 21:10:25 -0400538:source:`Objects/typeobject.c`. and a pure Python equivalent can be found in
539`Guido's Tutorial`_.
Georg Brandl45cceeb2010-05-19 21:39:51 +0000540
Georg Brandl9bdcb3b2014-10-29 09:37:43 +0100541.. _`Guido's Tutorial`: https://www.python.org/download/releases/2.2.3/descrintro/#cooperation
Georg Brandl45cceeb2010-05-19 21:39:51 +0000542
543The details above show that the mechanism for descriptors is embedded in the
544:meth:`__getattribute__()` methods for :class:`object`, :class:`type`, and
545:func:`super`. Classes inherit this machinery when they derive from
Raymond Hettinger8d3d7312020-10-23 12:55:39 -0700546:class:`object` or if they have a metaclass providing similar functionality.
Georg Brandl45cceeb2010-05-19 21:39:51 +0000547Likewise, classes can turn-off descriptor invocation by overriding
548:meth:`__getattribute__()`.
549
550
Raymond Hettinger8d3d7312020-10-23 12:55:39 -0700551Automatic Name Notification
552---------------------------
553
554Sometimes it is desirable for a descriptor to know what class variable name it
555was assigned to. When a new class is created, the :class:`type` metaclass
556scans the dictionary of the new class. If any of the entries are descriptors
557and if they define :meth:`__set_name__`, that method is called with two
558arguments. The *owner* is the class where the descriptor is used, the *name*
559is class variable the descriptor was assigned to.
560
561The implementation details are in :c:func:`type_new()` and
562:c:func:`set_names()` in :source:`Objects/typeobject.c`.
563
564Since the update logic is in :meth:`type.__new__`, notifications only take
565place at the time of class creation. If descriptors are added to the class
566afterwards, :meth:`__set_name__` will need to be called manually.
567
568
Georg Brandl45cceeb2010-05-19 21:39:51 +0000569Descriptor Example
570------------------
571
572The following code creates a class whose objects are data descriptors which
573print a message for each get or set. Overriding :meth:`__getattribute__` is
574alternate approach that could do this for every attribute. However, this
575descriptor is useful for monitoring just a few chosen attributes::
576
Serhiy Storchakae042a452019-06-10 13:35:52 +0300577 class RevealAccess:
Georg Brandl45cceeb2010-05-19 21:39:51 +0000578 """A data descriptor that sets and returns values
579 normally and prints a message logging their access.
580 """
581
582 def __init__(self, initval=None, name='var'):
583 self.val = initval
584 self.name = name
585
Raymond Hettinger8d3d7312020-10-23 12:55:39 -0700586 def __get__(self, obj, objtype=None):
Georg Brandl45cceeb2010-05-19 21:39:51 +0000587 print('Retrieving', self.name)
588 return self.val
589
590 def __set__(self, obj, val):
591 print('Updating', self.name)
592 self.val = val
593
Raymond Hettinger8d3d7312020-10-23 12:55:39 -0700594 class B:
595 x = RevealAccess(10, 'var "x"')
596 y = 5
597
598 >>> m = B()
Georg Brandl45cceeb2010-05-19 21:39:51 +0000599 >>> m.x
600 Retrieving var "x"
601 10
602 >>> m.x = 20
603 Updating var "x"
604 >>> m.x
605 Retrieving var "x"
606 20
607 >>> m.y
608 5
609
610The protocol is simple and offers exciting possibilities. Several use cases are
611so common that they have been packaged into individual function calls.
Raymond Hettinger0d4497b2017-09-25 01:05:49 -0700612Properties, bound methods, static methods, and class methods are all
Georg Brandl45cceeb2010-05-19 21:39:51 +0000613based on the descriptor protocol.
614
615
616Properties
617----------
618
619Calling :func:`property` is a succinct way of building a data descriptor that
620triggers function calls upon access to an attribute. Its signature is::
621
622 property(fget=None, fset=None, fdel=None, doc=None) -> property attribute
623
624The documentation shows a typical use to define a managed attribute ``x``::
625
Serhiy Storchakae042a452019-06-10 13:35:52 +0300626 class C:
Georg Brandl45cceeb2010-05-19 21:39:51 +0000627 def getx(self): return self.__x
628 def setx(self, value): self.__x = value
629 def delx(self): del self.__x
630 x = property(getx, setx, delx, "I'm the 'x' property.")
631
632To see how :func:`property` is implemented in terms of the descriptor protocol,
633here is a pure Python equivalent::
634
Serhiy Storchakae042a452019-06-10 13:35:52 +0300635 class Property:
Georg Brandl45cceeb2010-05-19 21:39:51 +0000636 "Emulate PyProperty_Type() in Objects/descrobject.c"
637
638 def __init__(self, fget=None, fset=None, fdel=None, doc=None):
639 self.fget = fget
640 self.fset = fset
641 self.fdel = fdel
Raymond Hettinger632c8c82013-03-10 09:41:18 -0700642 if doc is None and fget is not None:
643 doc = fget.__doc__
Georg Brandl45cceeb2010-05-19 21:39:51 +0000644 self.__doc__ = doc
645
646 def __get__(self, obj, objtype=None):
647 if obj is None:
648 return self
649 if self.fget is None:
Raymond Hettinger632c8c82013-03-10 09:41:18 -0700650 raise AttributeError("unreadable attribute")
Georg Brandl45cceeb2010-05-19 21:39:51 +0000651 return self.fget(obj)
652
653 def __set__(self, obj, value):
654 if self.fset is None:
Raymond Hettinger632c8c82013-03-10 09:41:18 -0700655 raise AttributeError("can't set attribute")
Georg Brandl45cceeb2010-05-19 21:39:51 +0000656 self.fset(obj, value)
657
658 def __delete__(self, obj):
659 if self.fdel is None:
Raymond Hettinger632c8c82013-03-10 09:41:18 -0700660 raise AttributeError("can't delete attribute")
Georg Brandl45cceeb2010-05-19 21:39:51 +0000661 self.fdel(obj)
662
Raymond Hettinger632c8c82013-03-10 09:41:18 -0700663 def getter(self, fget):
664 return type(self)(fget, self.fset, self.fdel, self.__doc__)
665
666 def setter(self, fset):
667 return type(self)(self.fget, fset, self.fdel, self.__doc__)
668
669 def deleter(self, fdel):
670 return type(self)(self.fget, self.fset, fdel, self.__doc__)
671
Georg Brandl45cceeb2010-05-19 21:39:51 +0000672The :func:`property` builtin helps whenever a user interface has granted
673attribute access and then subsequent changes require the intervention of a
674method.
675
676For instance, a spreadsheet class may grant access to a cell value through
677``Cell('b10').value``. Subsequent improvements to the program require the cell
678to be recalculated on every access; however, the programmer does not want to
679affect existing client code accessing the attribute directly. The solution is
680to wrap access to the value attribute in a property data descriptor::
681
Serhiy Storchakae042a452019-06-10 13:35:52 +0300682 class Cell:
Raymond Hettinger8d3d7312020-10-23 12:55:39 -0700683 ...
684
685 @property
686 def value(self):
_ = NaNb066edf2017-06-23 11:54:35 +0800687 "Recalculate the cell before returning value"
Georg Brandl45cceeb2010-05-19 21:39:51 +0000688 self.recalc()
_ = NaNb066edf2017-06-23 11:54:35 +0800689 return self._value
Georg Brandl45cceeb2010-05-19 21:39:51 +0000690
691
692Functions and Methods
693---------------------
694
695Python's object oriented features are built upon a function based environment.
696Using non-data descriptors, the two are merged seamlessly.
697
698Class dictionaries store methods as functions. In a class definition, methods
Raymond Hettinger0d4497b2017-09-25 01:05:49 -0700699are written using :keyword:`def` or :keyword:`lambda`, the usual tools for
700creating functions. Methods only differ from regular functions in that the
Georg Brandl45cceeb2010-05-19 21:39:51 +0000701first argument is reserved for the object instance. By Python convention, the
702instance reference is called *self* but may be called *this* or any other
703variable name.
704
705To support method calls, functions include the :meth:`__get__` method for
706binding methods during attribute access. This means that all functions are
Raymond Hettinger0d4497b2017-09-25 01:05:49 -0700707non-data descriptors which return bound methods when they are invoked from an
Andrés Delfino271818f2018-09-14 14:13:09 -0300708object. In pure Python, it works like this::
Georg Brandl45cceeb2010-05-19 21:39:51 +0000709
Serhiy Storchakae042a452019-06-10 13:35:52 +0300710 class Function:
Raymond Hettinger8d3d7312020-10-23 12:55:39 -0700711 ...
712
Georg Brandl45cceeb2010-05-19 21:39:51 +0000713 def __get__(self, obj, objtype=None):
714 "Simulate func_descr_get() in Objects/funcobject.c"
Raymond Hettinger0d4497b2017-09-25 01:05:49 -0700715 if obj is None:
716 return self
Mariano Anaya1bced562017-06-05 04:46:50 +0200717 return types.MethodType(self, obj)
Georg Brandl45cceeb2010-05-19 21:39:51 +0000718
Raymond Hettinger8d3d7312020-10-23 12:55:39 -0700719Running the following in class in the interpreter shows how the function
720descriptor works in practice::
Georg Brandl45cceeb2010-05-19 21:39:51 +0000721
Raymond Hettinger8d3d7312020-10-23 12:55:39 -0700722 class D:
723 def f(self, x):
724 return x
Raymond Hettinger0d4497b2017-09-25 01:05:49 -0700725
Raymond Hettinger8d3d7312020-10-23 12:55:39 -0700726Access through the class dictionary does not invoke :meth:`__get__`. Instead,
727it just returns the underlying function object::
728
Raymond Hettinger0d4497b2017-09-25 01:05:49 -0700729 >>> D.__dict__['f']
730 <function D.f at 0x00C45070>
731
Raymond Hettinger8d3d7312020-10-23 12:55:39 -0700732Dotted access from a class calls :meth:`__get__` which just returns the
733underlying function unchanged::
734
Raymond Hettinger0d4497b2017-09-25 01:05:49 -0700735 >>> D.f
736 <function D.f at 0x00C45070>
737
Raymond Hettinger8d3d7312020-10-23 12:55:39 -0700738The function has a :term:`qualified name` attribute to support introspection::
739
Raymond Hettinger0d4497b2017-09-25 01:05:49 -0700740 >>> D.f.__qualname__
741 'D.f'
742
Raymond Hettinger8d3d7312020-10-23 12:55:39 -0700743Dotted access from an instance calls :meth:`__get__` which returns a bound
744method object::
745
746 >>> d = D()
Raymond Hettinger0d4497b2017-09-25 01:05:49 -0700747 >>> d.f
Georg Brandl45cceeb2010-05-19 21:39:51 +0000748 <bound method D.f of <__main__.D object at 0x00B18C90>>
749
Raymond Hettinger8d3d7312020-10-23 12:55:39 -0700750Internally, the bound method stores the underlying function and the bound
751instance::
752
Raymond Hettinger0d4497b2017-09-25 01:05:49 -0700753 >>> d.f.__func__
754 <function D.f at 0x1012e5ae8>
755 >>> d.f.__self__
756 <__main__.D object at 0x1012e1f98>
Georg Brandl45cceeb2010-05-19 21:39:51 +0000757
758
759Static Methods and Class Methods
760--------------------------------
761
762Non-data descriptors provide a simple mechanism for variations on the usual
763patterns of binding functions into methods.
764
765To recap, functions have a :meth:`__get__` method so that they can be converted
Serhiy Storchakad65c9492015-11-02 14:10:23 +0200766to a method when accessed as attributes. The non-data descriptor transforms an
Raymond Hettinger8d3d7312020-10-23 12:55:39 -0700767``obj.f(*args)`` call into ``f(obj, *args)``. Calling ``cls.f(*args)``
Georg Brandl45cceeb2010-05-19 21:39:51 +0000768becomes ``f(*args)``.
769
770This chart summarizes the binding and its two most useful variants:
771
772 +-----------------+----------------------+------------------+
773 | Transformation | Called from an | Called from a |
Raymond Hettinger8d3d7312020-10-23 12:55:39 -0700774 | | object | class |
Georg Brandl45cceeb2010-05-19 21:39:51 +0000775 +=================+======================+==================+
776 | function | f(obj, \*args) | f(\*args) |
777 +-----------------+----------------------+------------------+
778 | staticmethod | f(\*args) | f(\*args) |
779 +-----------------+----------------------+------------------+
Raymond Hettinger8d3d7312020-10-23 12:55:39 -0700780 | classmethod | f(type(obj), \*args) | f(cls, \*args) |
Georg Brandl45cceeb2010-05-19 21:39:51 +0000781 +-----------------+----------------------+------------------+
782
783Static methods return the underlying function without changes. Calling either
784``c.f`` or ``C.f`` is the equivalent of a direct lookup into
785``object.__getattribute__(c, "f")`` or ``object.__getattribute__(C, "f")``. As a
786result, the function becomes identically accessible from either an object or a
787class.
788
789Good candidates for static methods are methods that do not reference the
790``self`` variable.
791
792For instance, a statistics package may include a container class for
793experimental data. The class provides normal methods for computing the average,
794mean, median, and other descriptive statistics that depend on the data. However,
795there may be useful functions which are conceptually related but do not depend
796on the data. For instance, ``erf(x)`` is handy conversion routine that comes up
797in statistical work but does not directly depend on a particular dataset.
798It can be called either from an object or the class: ``s.erf(1.5) --> .9332`` or
799``Sample.erf(1.5) --> .9332``.
800
801Since staticmethods return the underlying function with no changes, the example
802calls are unexciting::
803
Raymond Hettinger8d3d7312020-10-23 12:55:39 -0700804 class E:
805 @staticmethod
806 def f(x):
807 print(x)
808
Shubham Aggarwalabbdd1f2019-03-20 08:25:55 +0530809 >>> E.f(3)
Georg Brandl45cceeb2010-05-19 21:39:51 +0000810 3
Shubham Aggarwalabbdd1f2019-03-20 08:25:55 +0530811 >>> E().f(3)
Georg Brandl45cceeb2010-05-19 21:39:51 +0000812 3
813
814Using the non-data descriptor protocol, a pure Python version of
815:func:`staticmethod` would look like this::
816
Serhiy Storchakae042a452019-06-10 13:35:52 +0300817 class StaticMethod:
Serhiy Storchakadba90392016-05-10 12:01:23 +0300818 "Emulate PyStaticMethod_Type() in Objects/funcobject.c"
Georg Brandl45cceeb2010-05-19 21:39:51 +0000819
Serhiy Storchakadba90392016-05-10 12:01:23 +0300820 def __init__(self, f):
821 self.f = f
Georg Brandl45cceeb2010-05-19 21:39:51 +0000822
Serhiy Storchakadba90392016-05-10 12:01:23 +0300823 def __get__(self, obj, objtype=None):
824 return self.f
Georg Brandl45cceeb2010-05-19 21:39:51 +0000825
826Unlike static methods, class methods prepend the class reference to the
827argument list before calling the function. This format is the same
828for whether the caller is an object or a class::
829
Raymond Hettinger8d3d7312020-10-23 12:55:39 -0700830 class F:
831 @classmethod
832 def f(cls, x):
833 return cls.__name__, x
834
835 >>> print(F.f(3))
836 ('F', 3)
837 >>> print(F().f(3))
838 ('F', 3)
Georg Brandl45cceeb2010-05-19 21:39:51 +0000839
840
841This behavior is useful whenever the function only needs to have a class
Raymond Hettinger8d3d7312020-10-23 12:55:39 -0700842reference and does not care about any underlying data. One use for
843classmethods is to create alternate class constructors. The classmethod
Georg Brandl45cceeb2010-05-19 21:39:51 +0000844:func:`dict.fromkeys` creates a new dictionary from a list of keys. The pure
845Python equivalent is::
846
Serhiy Storchakae042a452019-06-10 13:35:52 +0300847 class Dict:
Raymond Hettinger8d3d7312020-10-23 12:55:39 -0700848 ...
849
850 @classmethod
851 def fromkeys(cls, iterable, value=None):
Georg Brandl45cceeb2010-05-19 21:39:51 +0000852 "Emulate dict_fromkeys() in Objects/dictobject.c"
Raymond Hettinger8d3d7312020-10-23 12:55:39 -0700853 d = cls()
Georg Brandl45cceeb2010-05-19 21:39:51 +0000854 for key in iterable:
855 d[key] = value
856 return d
Georg Brandl45cceeb2010-05-19 21:39:51 +0000857
858Now a new dictionary of unique keys can be constructed like this::
859
860 >>> Dict.fromkeys('abracadabra')
861 {'a': None, 'r': None, 'b': None, 'c': None, 'd': None}
862
863Using the non-data descriptor protocol, a pure Python version of
864:func:`classmethod` would look like this::
865
Serhiy Storchakae042a452019-06-10 13:35:52 +0300866 class ClassMethod:
Serhiy Storchakadba90392016-05-10 12:01:23 +0300867 "Emulate PyClassMethod_Type() in Objects/funcobject.c"
Georg Brandl45cceeb2010-05-19 21:39:51 +0000868
Serhiy Storchakadba90392016-05-10 12:01:23 +0300869 def __init__(self, f):
870 self.f = f
Georg Brandl45cceeb2010-05-19 21:39:51 +0000871
Raymond Hettinger8d3d7312020-10-23 12:55:39 -0700872 def __get__(self, obj, cls=None):
873 if cls is None:
874 cls = type(obj)
Raymond Hettinger8e5b0fd2020-10-23 18:37:27 -0700875 if hasattr(obj, '__get__'):
876 return self.f.__get__(cls)
877 return types.MethodType(self.f, cls)
878
879The code path for ``hasattr(obj, '__get__')`` was added in Python 3.9 and
880makes it possible for :func:`classmethod` to support chained decorators.
881For example, a classmethod and property could be chained together::
882
883 class G:
884 @classmethod
885 @property
886 def __doc__(cls):
887 return f'A doc for {cls.__name__!r}'