blob: 0c4580a9a7dbf86b9c2dcf0f096eea6662bb7054 [file] [log] [blame]
Georg Brandl116aa622007-08-15 14:28:22 +00001.. _tut-classes:
2
3*******
4Classes
5*******
6
7Python's class mechanism adds classes to the language with a minimum of new
8syntax and semantics. It is a mixture of the class mechanisms found in C++ and
9Modula-3. As is true for modules, classes in Python do not put an absolute
10barrier between definition and user, but rather rely on the politeness of the
11user not to "break into the definition." The most important features of classes
12are retained with full power, however: the class inheritance mechanism allows
13multiple base classes, a derived class can override any methods of its base
14class or classes, and a method can call the method of a base class with the same
15name. Objects can contain an arbitrary amount of private data.
16
Guido van Rossum0616b792007-08-31 03:25:11 +000017In C++ terminology, normally class members (including the data members) are
18*public* (except see below :ref:`tut-private`),
Georg Brandl116aa622007-08-15 14:28:22 +000019and all member functions are *virtual*. There are no special constructors or
20destructors. As in Modula-3, there are no shorthands for referencing the
21object's members from its methods: the method function is declared with an
22explicit first argument representing the object, which is provided implicitly by
23the call. As in Smalltalk, classes themselves are objects, albeit in the wider
24sense of the word: in Python, all data types are objects. This provides
25semantics for importing and renaming. Unlike C++ and Modula-3, built-in types
26can be used as base classes for extension by the user. Also, like in C++ but
27unlike in Modula-3, most built-in operators with special syntax (arithmetic
28operators, subscripting etc.) can be redefined for class instances.
29
30
31.. _tut-terminology:
32
33A Word About Terminology
34========================
35
36Lacking universally accepted terminology to talk about classes, I will make
37occasional use of Smalltalk and C++ terms. (I would use Modula-3 terms, since
38its object-oriented semantics are closer to those of Python than C++, but I
39expect that few readers have heard of it.)
40
41Objects have individuality, and multiple names (in multiple scopes) can be bound
42to the same object. This is known as aliasing in other languages. This is
43usually not appreciated on a first glance at Python, and can be safely ignored
44when dealing with immutable basic types (numbers, strings, tuples). However,
45aliasing has an (intended!) effect on the semantics of Python code involving
46mutable objects such as lists, dictionaries, and most types representing
47entities outside the program (files, windows, etc.). This is usually used to
48the benefit of the program, since aliases behave like pointers in some respects.
49For example, passing an object is cheap since only a pointer is passed by the
50implementation; and if a function modifies an object passed as an argument, the
51caller will see the change --- this eliminates the need for two different
52argument passing mechanisms as in Pascal.
53
54
55.. _tut-scopes:
56
57Python Scopes and Name Spaces
58=============================
59
60Before introducing classes, I first have to tell you something about Python's
61scope rules. Class definitions play some neat tricks with namespaces, and you
62need to know how scopes and namespaces work to fully understand what's going on.
63Incidentally, knowledge about this subject is useful for any advanced Python
64programmer.
65
66Let's begin with some definitions.
67
68A *namespace* is a mapping from names to objects. Most namespaces are currently
69implemented as Python dictionaries, but that's normally not noticeable in any
70way (except for performance), and it may change in the future. Examples of
71namespaces are: the set of built-in names (functions such as :func:`abs`, and
72built-in exception names); the global names in a module; and the local names in
73a function invocation. In a sense the set of attributes of an object also form
74a namespace. The important thing to know about namespaces is that there is
75absolutely no relation between names in different namespaces; for instance, two
76different modules may both define a function "maximize" without confusion ---
77users of the modules must prefix it with the module name.
78
79By the way, I use the word *attribute* for any name following a dot --- for
80example, in the expression ``z.real``, ``real`` is an attribute of the object
81``z``. Strictly speaking, references to names in modules are attribute
82references: in the expression ``modname.funcname``, ``modname`` is a module
83object and ``funcname`` is an attribute of it. In this case there happens to be
84a straightforward mapping between the module's attributes and the global names
85defined in the module: they share the same namespace! [#]_
86
87Attributes may be read-only or writable. In the latter case, assignment to
88attributes is possible. Module attributes are writable: you can write
89``modname.the_answer = 42``. Writable attributes may also be deleted with the
90:keyword:`del` statement. For example, ``del modname.the_answer`` will remove
91the attribute :attr:`the_answer` from the object named by ``modname``.
92
93Name spaces are created at different moments and have different lifetimes. The
94namespace containing the built-in names is created when the Python interpreter
95starts up, and is never deleted. The global namespace for a module is created
96when the module definition is read in; normally, module namespaces also last
97until the interpreter quits. The statements executed by the top-level
98invocation of the interpreter, either read from a script file or interactively,
99are considered part of a module called :mod:`__main__`, so they have their own
100global namespace. (The built-in names actually also live in a module; this is
101called :mod:`__builtin__`.)
102
103The local namespace for a function is created when the function is called, and
104deleted when the function returns or raises an exception that is not handled
105within the function. (Actually, forgetting would be a better way to describe
106what actually happens.) Of course, recursive invocations each have their own
107local namespace.
108
109A *scope* is a textual region of a Python program where a namespace is directly
110accessible. "Directly accessible" here means that an unqualified reference to a
111name attempts to find the name in the namespace.
112
113Although scopes are determined statically, they are used dynamically. At any
114time during execution, there are at least three nested scopes whose namespaces
115are directly accessible: the innermost scope, which is searched first, contains
116the local names; the namespaces of any enclosing functions, which are searched
117starting with the nearest enclosing scope; the middle scope, searched next,
118contains the current module's global names; and the outermost scope (searched
119last) is the namespace containing built-in names.
120
121If a name is declared global, then all references and assignments go directly to
122the middle scope containing the module's global names. Otherwise, all variables
123found outside of the innermost scope are read-only (an attempt to write to such
124a variable will simply create a *new* local variable in the innermost scope,
125leaving the identically named outer variable unchanged).
126
127Usually, the local scope references the local names of the (textually) current
128function. Outside functions, the local scope references the same namespace as
129the global scope: the module's namespace. Class definitions place yet another
130namespace in the local scope.
131
132It is important to realize that scopes are determined textually: the global
133scope of a function defined in a module is that module's namespace, no matter
134from where or by what alias the function is called. On the other hand, the
135actual search for names is done dynamically, at run time --- however, the
136language definition is evolving towards static name resolution, at "compile"
137time, so don't rely on dynamic name resolution! (In fact, local variables are
138already determined statically.)
139
140A special quirk of Python is that assignments always go into the innermost
141scope. Assignments do not copy data --- they just bind names to objects. The
142same is true for deletions: the statement ``del x`` removes the binding of ``x``
143from the namespace referenced by the local scope. In fact, all operations that
144introduce new names use the local scope: in particular, import statements and
145function definitions bind the module or function name in the local scope. (The
146:keyword:`global` statement can be used to indicate that particular variables
147live in the global scope.)
148
149
150.. _tut-firstclasses:
151
152A First Look at Classes
153=======================
154
155Classes introduce a little bit of new syntax, three new object types, and some
156new semantics.
157
158
159.. _tut-classdefinition:
160
161Class Definition Syntax
162-----------------------
163
164The simplest form of class definition looks like this::
165
166 class ClassName:
167 <statement-1>
168 .
169 .
170 .
171 <statement-N>
172
173Class definitions, like function definitions (:keyword:`def` statements) must be
174executed before they have any effect. (You could conceivably place a class
175definition in a branch of an :keyword:`if` statement, or inside a function.)
176
177In practice, the statements inside a class definition will usually be function
178definitions, but other statements are allowed, and sometimes useful --- we'll
179come back to this later. The function definitions inside a class normally have
180a peculiar form of argument list, dictated by the calling conventions for
181methods --- again, this is explained later.
182
183When a class definition is entered, a new namespace is created, and used as the
184local scope --- thus, all assignments to local variables go into this new
185namespace. In particular, function definitions bind the name of the new
186function here.
187
188When a class definition is left normally (via the end), a *class object* is
189created. This is basically a wrapper around the contents of the namespace
190created by the class definition; we'll learn more about class objects in the
191next section. The original local scope (the one in effect just before the class
192definition was entered) is reinstated, and the class object is bound here to the
193class name given in the class definition header (:class:`ClassName` in the
194example).
195
196
197.. _tut-classobjects:
198
199Class Objects
200-------------
201
202Class objects support two kinds of operations: attribute references and
203instantiation.
204
205*Attribute references* use the standard syntax used for all attribute references
206in Python: ``obj.name``. Valid attribute names are all the names that were in
207the class's namespace when the class object was created. So, if the class
208definition looked like this::
209
210 class MyClass:
211 "A simple example class"
212 i = 12345
213 def f(self):
214 return 'hello world'
215
216then ``MyClass.i`` and ``MyClass.f`` are valid attribute references, returning
217an integer and a function object, respectively. Class attributes can also be
218assigned to, so you can change the value of ``MyClass.i`` by assignment.
219:attr:`__doc__` is also a valid attribute, returning the docstring belonging to
220the class: ``"A simple example class"``.
221
222Class *instantiation* uses function notation. Just pretend that the class
223object is a parameterless function that returns a new instance of the class.
224For example (assuming the above class)::
225
226 x = MyClass()
227
228creates a new *instance* of the class and assigns this object to the local
229variable ``x``.
230
231The instantiation operation ("calling" a class object) creates an empty object.
232Many classes like to create objects with instances customized to a specific
233initial state. Therefore a class may define a special method named
234:meth:`__init__`, like this::
235
236 def __init__(self):
237 self.data = []
238
239When a class defines an :meth:`__init__` method, class instantiation
240automatically invokes :meth:`__init__` for the newly-created class instance. So
241in this example, a new, initialized instance can be obtained by::
242
243 x = MyClass()
244
245Of course, the :meth:`__init__` method may have arguments for greater
246flexibility. In that case, arguments given to the class instantiation operator
247are passed on to :meth:`__init__`. For example, ::
248
249 >>> class Complex:
250 ... def __init__(self, realpart, imagpart):
251 ... self.r = realpart
252 ... self.i = imagpart
253 ...
254 >>> x = Complex(3.0, -4.5)
255 >>> x.r, x.i
256 (3.0, -4.5)
257
258
259.. _tut-instanceobjects:
260
261Instance Objects
262----------------
263
264Now what can we do with instance objects? The only operations understood by
265instance objects are attribute references. There are two kinds of valid
266attribute names, data attributes and methods.
267
268*data attributes* correspond to "instance variables" in Smalltalk, and to "data
269members" in C++. Data attributes need not be declared; like local variables,
270they spring into existence when they are first assigned to. For example, if
271``x`` is the instance of :class:`MyClass` created above, the following piece of
272code will print the value ``16``, without leaving a trace::
273
274 x.counter = 1
275 while x.counter < 10:
276 x.counter = x.counter * 2
Guido van Rossum0616b792007-08-31 03:25:11 +0000277 print(x.counter)
Georg Brandl116aa622007-08-15 14:28:22 +0000278 del x.counter
279
280The other kind of instance attribute reference is a *method*. A method is a
281function that "belongs to" an object. (In Python, the term method is not unique
282to class instances: other object types can have methods as well. For example,
283list objects have methods called append, insert, remove, sort, and so on.
284However, in the following discussion, we'll use the term method exclusively to
285mean methods of class instance objects, unless explicitly stated otherwise.)
286
287.. index:: object: method
288
289Valid method names of an instance object depend on its class. By definition,
290all attributes of a class that are function objects define corresponding
291methods of its instances. So in our example, ``x.f`` is a valid method
292reference, since ``MyClass.f`` is a function, but ``x.i`` is not, since
293``MyClass.i`` is not. But ``x.f`` is not the same thing as ``MyClass.f`` --- it
294is a *method object*, not a function object.
295
296
297.. _tut-methodobjects:
298
299Method Objects
300--------------
301
302Usually, a method is called right after it is bound::
303
304 x.f()
305
306In the :class:`MyClass` example, this will return the string ``'hello world'``.
307However, it is not necessary to call a method right away: ``x.f`` is a method
308object, and can be stored away and called at a later time. For example::
309
310 xf = x.f
311 while True:
Guido van Rossum0616b792007-08-31 03:25:11 +0000312 print(xf())
Georg Brandl116aa622007-08-15 14:28:22 +0000313
314will continue to print ``hello world`` until the end of time.
315
316What exactly happens when a method is called? You may have noticed that
317``x.f()`` was called without an argument above, even though the function
318definition for :meth:`f` specified an argument. What happened to the argument?
319Surely Python raises an exception when a function that requires an argument is
320called without any --- even if the argument isn't actually used...
321
322Actually, you may have guessed the answer: the special thing about methods is
323that the object is passed as the first argument of the function. In our
324example, the call ``x.f()`` is exactly equivalent to ``MyClass.f(x)``. In
325general, calling a method with a list of *n* arguments is equivalent to calling
326the corresponding function with an argument list that is created by inserting
327the method's object before the first argument.
328
329If you still don't understand how methods work, a look at the implementation can
330perhaps clarify matters. When an instance attribute is referenced that isn't a
331data attribute, its class is searched. If the name denotes a valid class
332attribute that is a function object, a method object is created by packing
333(pointers to) the instance object and the function object just found together in
334an abstract object: this is the method object. When the method object is called
335with an argument list, it is unpacked again, a new argument list is constructed
336from the instance object and the original argument list, and the function object
337is called with this new argument list.
338
339
340.. _tut-remarks:
341
342Random Remarks
343==============
344
345.. % [These should perhaps be placed more carefully...]
346
347Data attributes override method attributes with the same name; to avoid
348accidental name conflicts, which may cause hard-to-find bugs in large programs,
349it is wise to use some kind of convention that minimizes the chance of
350conflicts. Possible conventions include capitalizing method names, prefixing
351data attribute names with a small unique string (perhaps just an underscore), or
352using verbs for methods and nouns for data attributes.
353
354Data attributes may be referenced by methods as well as by ordinary users
355("clients") of an object. In other words, classes are not usable to implement
356pure abstract data types. In fact, nothing in Python makes it possible to
357enforce data hiding --- it is all based upon convention. (On the other hand,
358the Python implementation, written in C, can completely hide implementation
359details and control access to an object if necessary; this can be used by
360extensions to Python written in C.)
361
362Clients should use data attributes with care --- clients may mess up invariants
363maintained by the methods by stamping on their data attributes. Note that
364clients may add data attributes of their own to an instance object without
365affecting the validity of the methods, as long as name conflicts are avoided ---
366again, a naming convention can save a lot of headaches here.
367
368There is no shorthand for referencing data attributes (or other methods!) from
369within methods. I find that this actually increases the readability of methods:
370there is no chance of confusing local variables and instance variables when
371glancing through a method.
372
373Often, the first argument of a method is called ``self``. This is nothing more
374than a convention: the name ``self`` has absolutely no special meaning to
375Python. (Note, however, that by not following the convention your code may be
376less readable to other Python programmers, and it is also conceivable that a
377*class browser* program might be written that relies upon such a convention.)
378
379Any function object that is a class attribute defines a method for instances of
380that class. It is not necessary that the function definition is textually
381enclosed in the class definition: assigning a function object to a local
382variable in the class is also ok. For example::
383
384 # Function defined outside the class
385 def f1(self, x, y):
386 return min(x, x+y)
387
388 class C:
389 f = f1
390 def g(self):
391 return 'hello world'
392 h = g
393
394Now ``f``, ``g`` and ``h`` are all attributes of class :class:`C` that refer to
395function objects, and consequently they are all methods of instances of
396:class:`C` --- ``h`` being exactly equivalent to ``g``. Note that this practice
397usually only serves to confuse the reader of a program.
398
399Methods may call other methods by using method attributes of the ``self``
400argument::
401
402 class Bag:
403 def __init__(self):
404 self.data = []
405 def add(self, x):
406 self.data.append(x)
407 def addtwice(self, x):
408 self.add(x)
409 self.add(x)
410
411Methods may reference global names in the same way as ordinary functions. The
412global scope associated with a method is the module containing the class
413definition. (The class itself is never used as a global scope!) While one
414rarely encounters a good reason for using global data in a method, there are
415many legitimate uses of the global scope: for one thing, functions and modules
416imported into the global scope can be used by methods, as well as functions and
417classes defined in it. Usually, the class containing the method is itself
418defined in this global scope, and in the next section we'll find some good
419reasons why a method would want to reference its own class!
420
421
422.. _tut-inheritance:
423
424Inheritance
425===========
426
427Of course, a language feature would not be worthy of the name "class" without
428supporting inheritance. The syntax for a derived class definition looks like
429this::
430
431 class DerivedClassName(BaseClassName):
432 <statement-1>
433 .
434 .
435 .
436 <statement-N>
437
438The name :class:`BaseClassName` must be defined in a scope containing the
439derived class definition. In place of a base class name, other arbitrary
440expressions are also allowed. This can be useful, for example, when the base
441class is defined in another module::
442
443 class DerivedClassName(modname.BaseClassName):
444
445Execution of a derived class definition proceeds the same as for a base class.
446When the class object is constructed, the base class is remembered. This is
447used for resolving attribute references: if a requested attribute is not found
448in the class, the search proceeds to look in the base class. This rule is
449applied recursively if the base class itself is derived from some other class.
450
451There's nothing special about instantiation of derived classes:
452``DerivedClassName()`` creates a new instance of the class. Method references
453are resolved as follows: the corresponding class attribute is searched,
454descending down the chain of base classes if necessary, and the method reference
455is valid if this yields a function object.
456
457Derived classes may override methods of their base classes. Because methods
458have no special privileges when calling other methods of the same object, a
459method of a base class that calls another method defined in the same base class
460may end up calling a method of a derived class that overrides it. (For C++
461programmers: all methods in Python are effectively :keyword:`virtual`.)
462
463An overriding method in a derived class may in fact want to extend rather than
464simply replace the base class method of the same name. There is a simple way to
465call the base class method directly: just call ``BaseClassName.methodname(self,
466arguments)``. This is occasionally useful to clients as well. (Note that this
467only works if the base class is defined or imported directly in the global
468scope.)
469
470
471.. _tut-multiple:
472
473Multiple Inheritance
474--------------------
475
476Python supports a limited form of multiple inheritance as well. A class
477definition with multiple base classes looks like this::
478
479 class DerivedClassName(Base1, Base2, Base3):
480 <statement-1>
481 .
482 .
483 .
484 <statement-N>
485
Georg Brandl85eb8c12007-08-31 16:33:38 +0000486Formerly, the only rule was depth-first, left-to-right. Thus, if an attribute
487was not found in :class:`DerivedClassName`, it was searched in :class:`Base1`,
488then (recursively) in the base classes of :class:`Base1`, and only if it was not
489found there, it was searched in :class:`Base2`, and so on.
Georg Brandl116aa622007-08-15 14:28:22 +0000490
Georg Brandl85eb8c12007-08-31 16:33:38 +0000491In the meantime, the method resolution order changes dynamically to support
492cooperative calls to :func:`super`. This approach is known in some other
493multiple-inheritance languages as call-next-method and is more powerful than the
494super call found in single-inheritance languages.
Georg Brandl116aa622007-08-15 14:28:22 +0000495
Georg Brandl85eb8c12007-08-31 16:33:38 +0000496Dynamic ordering is necessary because all cases of multiple inheritance exhibit
497one or more diamond relationships (where one at least one of the parent classes
498can be accessed through multiple paths from the bottommost class). For example,
499all classes inherit from :class:`object`, so any case of multiple inheritance
500provides more than one path to reach :class:`object`. To keep the base classes
501from being accessed more than once, the dynamic algorithm linearizes the search
502order in a way that preserves the left-to-right ordering specified in each
503class, that calls each parent only once, and that is monotonic (meaning that a
504class can be subclassed without affecting the precedence order of its parents).
505Taken together, these properties make it possible to design reliable and
506extensible classes with multiple inheritance. For more detail, see
Georg Brandl116aa622007-08-15 14:28:22 +0000507http://www.python.org/download/releases/2.3/mro/.
508
509
510.. _tut-private:
511
512Private Variables
513=================
514
515There is limited support for class-private identifiers. Any identifier of the
516form ``__spam`` (at least two leading underscores, at most one trailing
517underscore) is textually replaced with ``_classname__spam``, where ``classname``
518is the current class name with leading underscore(s) stripped. This mangling is
519done without regard to the syntactic position of the identifier, so it can be
520used to define class-private instance and class variables, methods, variables
521stored in globals, and even variables stored in instances. private to this class
522on instances of *other* classes. Truncation may occur when the mangled name
523would be longer than 255 characters. Outside classes, or when the class name
524consists of only underscores, no mangling occurs.
525
526Name mangling is intended to give classes an easy way to define "private"
527instance variables and methods, without having to worry about instance variables
528defined by derived classes, or mucking with instance variables by code outside
529the class. Note that the mangling rules are designed mostly to avoid accidents;
530it still is possible for a determined soul to access or modify a variable that
531is considered private. This can even be useful in special circumstances, such
532as in the debugger, and that's one reason why this loophole is not closed.
533(Buglet: derivation of a class with the same name as the base class makes use of
534private variables of the base class possible.)
535
536Notice that code passed to ``exec()`` or ``eval()`` does not
537consider the classname of the invoking class to be the current class; this is
538similar to the effect of the ``global`` statement, the effect of which is
539likewise restricted to code that is byte-compiled together. The same
540restriction applies to ``getattr()``, ``setattr()`` and ``delattr()``, as well
541as when referencing ``__dict__`` directly.
542
543
544.. _tut-odds:
545
546Odds and Ends
547=============
548
549Sometimes it is useful to have a data type similar to the Pascal "record" or C
550"struct", bundling together a few named data items. An empty class definition
551will do nicely::
552
553 class Employee:
554 pass
555
556 john = Employee() # Create an empty employee record
557
558 # Fill the fields of the record
559 john.name = 'John Doe'
560 john.dept = 'computer lab'
561 john.salary = 1000
562
563A piece of Python code that expects a particular abstract data type can often be
564passed a class that emulates the methods of that data type instead. For
565instance, if you have a function that formats some data from a file object, you
566can define a class with methods :meth:`read` and :meth:`readline` that get the
567data from a string buffer instead, and pass it as an argument.
568
569.. % (Unfortunately, this
570.. % technique has its limitations: a class can't define operations that
571.. % are accessed by special syntax such as sequence subscripting or
572.. % arithmetic operators, and assigning such a ``pseudo-file'' to
573.. % \code{sys.stdin} will not cause the interpreter to read further input
574.. % from it.)
575
576Instance method objects have attributes, too: ``m.im_self`` is the instance
577object with the method :meth:`m`, and ``m.im_func`` is the function object
578corresponding to the method.
579
580
581.. _tut-exceptionclasses:
582
583Exceptions Are Classes Too
584==========================
585
586User-defined exceptions are identified by classes as well. Using this mechanism
587it is possible to create extensible hierarchies of exceptions.
588
589There are two new valid (semantic) forms for the raise statement::
590
591 raise Class, instance
592
593 raise instance
594
595In the first form, ``instance`` must be an instance of :class:`Class` or of a
596class derived from it. The second form is a shorthand for::
597
598 raise instance.__class__, instance
599
600A class in an except clause is compatible with an exception if it is the same
601class or a base class thereof (but not the other way around --- an except clause
602listing a derived class is not compatible with a base class). For example, the
603following code will print B, C, D in that order::
604
605 class B:
606 pass
607 class C(B):
608 pass
609 class D(C):
610 pass
611
612 for c in [B, C, D]:
613 try:
614 raise c()
615 except D:
Guido van Rossum0616b792007-08-31 03:25:11 +0000616 print("D")
Georg Brandl116aa622007-08-15 14:28:22 +0000617 except C:
Guido van Rossum0616b792007-08-31 03:25:11 +0000618 print("C")
Georg Brandl116aa622007-08-15 14:28:22 +0000619 except B:
Guido van Rossum0616b792007-08-31 03:25:11 +0000620 print("B")
Georg Brandl116aa622007-08-15 14:28:22 +0000621
622Note that if the except clauses were reversed (with ``except B`` first), it
623would have printed B, B, B --- the first matching except clause is triggered.
624
625When an error message is printed for an unhandled exception, the exception's
626class name is printed, then a colon and a space, and finally the instance
627converted to a string using the built-in function :func:`str`.
628
629
630.. _tut-iterators:
631
632Iterators
633=========
634
635By now you have probably noticed that most container objects can be looped over
636using a :keyword:`for` statement::
637
638 for element in [1, 2, 3]:
Guido van Rossum0616b792007-08-31 03:25:11 +0000639 print(element)
Georg Brandl116aa622007-08-15 14:28:22 +0000640 for element in (1, 2, 3):
Guido van Rossum0616b792007-08-31 03:25:11 +0000641 print(element)
Georg Brandl116aa622007-08-15 14:28:22 +0000642 for key in {'one':1, 'two':2}:
Guido van Rossum0616b792007-08-31 03:25:11 +0000643 print(key)
Georg Brandl116aa622007-08-15 14:28:22 +0000644 for char in "123":
Guido van Rossum0616b792007-08-31 03:25:11 +0000645 print(char)
Georg Brandl116aa622007-08-15 14:28:22 +0000646 for line in open("myfile.txt"):
Guido van Rossum0616b792007-08-31 03:25:11 +0000647 print(line)
Georg Brandl116aa622007-08-15 14:28:22 +0000648
649This style of access is clear, concise, and convenient. The use of iterators
650pervades and unifies Python. Behind the scenes, the :keyword:`for` statement
651calls :func:`iter` on the container object. The function returns an iterator
652object that defines the method :meth:`__next__` which accesses elements in the
653container one at a time. When there are no more elements, :meth:`__next__`
654raises a :exc:`StopIteration` exception which tells the :keyword:`for` loop to
655terminate. You can call the :meth:`__next__` method using the :func:`next`
656builtin; this example shows how it all works::
657
658 >>> s = 'abc'
659 >>> it = iter(s)
660 >>> it
661 <iterator object at 0x00A1DB50>
662 >>> next(it)
663 'a'
664 >>> next(it)
665 'b'
666 >>> next(it)
667 'c'
668 >>> next(it)
669
670 Traceback (most recent call last):
671 File "<stdin>", line 1, in ?
672 next(it)
673 StopIteration
674
675Having seen the mechanics behind the iterator protocol, it is easy to add
676iterator behavior to your classes. Define a :meth:`__iter__` method which
677returns an object with a :meth:`__next__` method. If the class defines
678:meth:`__next__`, then :meth:`__iter__` can just return ``self``::
679
680 class Reverse:
681 "Iterator for looping over a sequence backwards"
682 def __init__(self, data):
683 self.data = data
684 self.index = len(data)
685 def __iter__(self):
686 return self
687 def __next__(self):
688 if self.index == 0:
689 raise StopIteration
690 self.index = self.index - 1
691 return self.data[self.index]
692
693 >>> for char in Reverse('spam'):
Guido van Rossum0616b792007-08-31 03:25:11 +0000694 ... print(char)
Georg Brandl116aa622007-08-15 14:28:22 +0000695 ...
696 m
697 a
698 p
699 s
700
701
702.. _tut-generators:
703
704Generators
705==========
706
707Generators are a simple and powerful tool for creating iterators. They are
708written like regular functions but use the :keyword:`yield` statement whenever
709they want to return data. Each time :func:`next` is called on it, the generator
710resumes where it left-off (it remembers all the data values and which statement
711was last executed). An example shows that generators can be trivially easy to
712create::
713
714 def reverse(data):
715 for index in range(len(data)-1, -1, -1):
716 yield data[index]
717
718 >>> for char in reverse('golf'):
Guido van Rossum0616b792007-08-31 03:25:11 +0000719 ... print(char)
Georg Brandl116aa622007-08-15 14:28:22 +0000720 ...
721 f
722 l
723 o
724 g
725
726Anything that can be done with generators can also be done with class based
727iterators as described in the previous section. What makes generators so
728compact is that the :meth:`__iter__` and :meth:`__next__` methods are created
729automatically.
730
731Another key feature is that the local variables and execution state are
732automatically saved between calls. This made the function easier to write and
733much more clear than an approach using instance variables like ``self.index``
734and ``self.data``.
735
736In addition to automatic method creation and saving program state, when
737generators terminate, they automatically raise :exc:`StopIteration`. In
738combination, these features make it easy to create iterators with no more effort
739than writing a regular function.
740
741
742.. _tut-genexps:
743
744Generator Expressions
745=====================
746
747Some simple generators can be coded succinctly as expressions using a syntax
748similar to list comprehensions but with parentheses instead of brackets. These
749expressions are designed for situations where the generator is used right away
750by an enclosing function. Generator expressions are more compact but less
751versatile than full generator definitions and tend to be more memory friendly
752than equivalent list comprehensions.
753
754Examples::
755
756 >>> sum(i*i for i in range(10)) # sum of squares
757 285
758
759 >>> xvec = [10, 20, 30]
760 >>> yvec = [7, 5, 3]
761 >>> sum(x*y for x,y in zip(xvec, yvec)) # dot product
762 260
763
764 >>> from math import pi, sin
765 >>> sine_table = dict((x, sin(x*pi/180)) for x in range(0, 91))
766
767 >>> unique_words = set(word for line in page for word in line.split())
768
769 >>> valedictorian = max((student.gpa, student.name) for student in graduates)
770
771 >>> data = 'golf'
772 >>> list(data[i] for i in range(len(data)-1,-1,-1))
773 ['f', 'l', 'o', 'g']
774
775
776
777.. rubric:: Footnotes
778
779.. [#] Except for one thing. Module objects have a secret read-only attribute called
780 :attr:`__dict__` which returns the dictionary used to implement the module's
781 namespace; the name :attr:`__dict__` is an attribute but not a global name.
782 Obviously, using this violates the abstraction of namespace implementation, and
783 should be restricted to things like post-mortem debuggers.
784