blob: 52788272b8cc3766ef24e2c059d6fd0ac29bb385 [file] [log] [blame]
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +00001\documentclass{howto}
2
3% $Id$
4
5\title{What's New in Python 2.2}
Andrew M. Kuchling038215a2001-12-07 14:22:13 +00006\release{0.10}
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +00007\author{A.M. Kuchling}
Andrew M. Kuchling7bf82772001-07-11 18:54:26 +00008\authoraddress{\email{akuchlin@mems-exchange.org}}
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +00009\begin{document}
10\maketitle\tableofcontents
11
12\section{Introduction}
13
Andrew M. Kuchling26c39bf2001-09-10 03:20:53 +000014This article explains the new features in Python 2.2.
Andrew M. Kuchling5e08d102001-12-20 16:33:45 +000015The final release of Python 2.2 is planned for December 2001.
Andrew M. Kuchling26c39bf2001-09-10 03:20:53 +000016
17Python 2.2 can be thought of as the "cleanup release". There are some
18features such as generators and iterators that are completely new, but
19most of the changes, significant and far-reaching though they may be,
20are aimed at cleaning up irregularities and dark corners of the
21language design.
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +000022
Andrew M. Kuchling1497b622001-09-24 14:51:16 +000023This article doesn't attempt to provide a complete specification of
Andrew M. Kuchling26c39bf2001-09-10 03:20:53 +000024the new features, but instead provides a convenient overview. For
25full details, you should refer to the documentation for Python 2.2,
Fred Drake0d002542001-07-17 13:55:33 +000026such as the
Andrew M. Kuchling5e08d102001-12-20 16:33:45 +000027\citetitle[http://www.python.org/doc/2.2/lib/lib.html]{Python
Fred Drake0d002542001-07-17 13:55:33 +000028Library Reference} and the
Andrew M. Kuchling5e08d102001-12-20 16:33:45 +000029\citetitle[http://www.python.org/doc/2.2/ref/ref.html]{Python
30Reference Manual}. If you want to understand the complete
31implementation and design rationale for a change, refer to the PEP for
32a particular new feature.
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +000033
Andrew M. Kuchling1497b622001-09-24 14:51:16 +000034\begin{seealso}
35
Andrew M. Kuchling2dab9c72001-10-31 13:16:10 +000036\seeurl{http://www.unixreview.com/documents/s=1356/urm0109h/0109h.htm}
Andrew M. Kuchling1497b622001-09-24 14:51:16 +000037{``What's So Special About Python 2.2?'' is also about the new 2.2
38features, and was written by Cameron Laird and Kathryn Soraiz.}
39
40\end{seealso}
41
Andrew M. Kuchling8cfa9052001-07-19 01:19:59 +000042
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +000043%======================================================================
Andrew M. Kuchling433b5c42001-10-30 21:36:04 +000044\section{PEPs 252 and 253: Type and Class Changes}
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +000045
Andrew M. Kuchling279e7442001-10-22 02:03:40 +000046The largest and most far-reaching changes in Python 2.2 are to
47Python's model of objects and classes. The changes should be backward
48compatible, so it's likely that your code will continue to run
49unchanged, but the changes provide some amazing new capabilities.
50Before beginning this, the longest and most complicated section of
51this article, I'll provide an overview of the changes and offer some
52comments.
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +000053
Andrew M. Kuchling279e7442001-10-22 02:03:40 +000054A long time ago I wrote a Web page
55(\url{http://www.amk.ca/python/writing/warts.html}) listing flaws in
56Python's design. One of the most significant flaws was that it's
57impossible to subclass Python types implemented in C. In particular,
58it's not possible to subclass built-in types, so you can't just
59subclass, say, lists in order to add a single useful method to them.
60The \module{UserList} module provides a class that supports all of the
61methods of lists and that can be subclassed further, but there's lots
62of C code that expects a regular Python list and won't accept a
63\class{UserList} instance.
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +000064
Andrew M. Kuchling279e7442001-10-22 02:03:40 +000065Python 2.2 fixes this, and in the process adds some exciting new
66capabilities. A brief summary:
Andrew M. Kuchling26c39bf2001-09-10 03:20:53 +000067
Andrew M. Kuchling279e7442001-10-22 02:03:40 +000068\begin{itemize}
Andrew M. Kuchlingd6e40e22001-09-10 16:18:50 +000069
Andrew M. Kuchling279e7442001-10-22 02:03:40 +000070\item You can subclass built-in types such as lists and even integers,
71and your subclasses should work in every place that requires the
72original type.
73
74\item It's now possible to define static and class methods, in addition
75to the instance methods available in previous versions of Python.
76
77\item It's also possible to automatically call methods on accessing or
78setting an instance attribute by using a new mechanism called
79\dfn{properties}. Many uses of \method{__getattr__} can be rewritten
80to use properties instead, making the resulting code simpler and
81faster. As a small side benefit, attributes can now have docstrings,
82too.
83
84\item The list of legal attributes for an instance can be limited to a
85particular set using \dfn{slots}, making it possible to safeguard
86against typos and perhaps make more optimizations possible in future
87versions of Python.
88
89\end{itemize}
90
91Some users have voiced concern about all these changes. Sure, they
92say, the new features are neat and lend themselves to all sorts of
93tricks that weren't possible in previous versions of Python, but
94they also make the language more complicated. Some people have said
95that they've always recommended Python for its simplicity, and feel
96that its simplicity is being lost.
97
98Personally, I think there's no need to worry. Many of the new
99features are quite esoteric, and you can write a lot of Python code
100without ever needed to be aware of them. Writing a simple class is no
101more difficult than it ever was, so you don't need to bother learning
102or teaching them unless they're actually needed. Some very
103complicated tasks that were previously only possible from C will now
104be possible in pure Python, and to my mind that's all for the better.
105
106I'm not going to attempt to cover every single corner case and small
107change that were required to make the new features work. Instead this
Andrew M. Kuchlingcf31d5d2001-10-26 20:37:55 +0000108section will paint only the broad strokes. See section~\ref{sect-rellinks},
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000109``Related Links'', for further sources of information about Python 2.2's new
110object model.
111
112
113\subsection{Old and New Classes}
114
115First, you should know that Python 2.2 really has two kinds of
116classes: classic or old-style classes, and new-style classes. The
117old-style class model is exactly the same as the class model in
118earlier versions of Python. All the new features described in this
119section apply only to new-style classes. This divergence isn't
120intended to last forever; eventually old-style classes will be
121dropped, possibly in Python 3.0.
122
Andrew M. Kuchling4855b022001-10-23 20:26:16 +0000123So how do you define a new-style class? You do it by subclassing an
124existing new-style class. Most of Python's built-in types, such as
125integers, lists, dictionaries, and even files, are new-style classes
126now. A new-style class named \class{object}, the base class for all
127built-in types, has been also been added so if no built-in type is
128suitable, you can just subclass \class{object}:
129
130\begin{verbatim}
131class C(object):
132 def __init__ (self):
133 ...
134 ...
135\end{verbatim}
136
137This means that \keyword{class} statements that don't have any base
Andrew M. Kuchlingcf31d5d2001-10-26 20:37:55 +0000138classes are always classic classes in Python 2.2. (Actually you can
139also change this by setting a module-level variable named
140\member{__metaclass__} --- see \pep{253} for the details --- but it's
141easier to just subclass \keyword{object}.)
Andrew M. Kuchling4855b022001-10-23 20:26:16 +0000142
143The type objects for the built-in types are available as built-ins,
144named using a clever trick. Python has always had built-in functions
145named \function{int()}, \function{float()}, and \function{str()}. In
1462.2, they aren't functions any more, but type objects that behave as
147factories when called.
148
149\begin{verbatim}
150>>> int
151<type 'int'>
152>>> int('123')
153123
154\end{verbatim}
155
156To make the set of types complete, new type objects such as
Andrew M. Kuchling1117d932001-10-29 20:37:47 +0000157\function{dict} and \function{file} have been added. Here's a
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000158more interesting example, adding a \method{lock()} method to file
159objects:
Andrew M. Kuchling4855b022001-10-23 20:26:16 +0000160
161\begin{verbatim}
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000162class LockableFile(file):
163 def lock (self, operation, length=0, start=0, whence=0):
164 import fcntl
165 return fcntl.lockf(self.fileno(), operation,
166 length, start, whence)
Andrew M. Kuchling4855b022001-10-23 20:26:16 +0000167\end{verbatim}
168
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000169The now-obsolete \module{posixfile} module contained a class that
170emulated all of a file object's methods and also added a
171\method{lock()} method, but this class couldn't be passed to internal
172functions that expected a built-in file, something which is possible
173with our new \class{LockableFile}.
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000174
175
176\subsection{Descriptors}
177
178In previous versions of Python, there was no consistent way to
179discover what attributes and methods were supported by an object.
180There were some informal conventions, such as defining
181\member{__members__} and \member{__methods__} attributes that were
182lists of names, but often the author of an extension type or a class
183wouldn't bother to define them. You could fall back on inspecting the
184\member{__dict__} of an object, but when class inheritance or an
185arbitrary \method{__getattr__} hook were in use this could still be
186inaccurate.
187
188The one big idea underlying the new class model is that an API for
189describing the attributes of an object using \dfn{descriptors} has
190been formalized. Descriptors specify the value of an attribute,
191stating whether it's a method or a field. With the descriptor API,
192static methods and class methods become possible, as well as more
193exotic constructs.
194
195Attribute descriptors are objects that live inside class objects, and
196have a few attributes of their own:
197
198\begin{itemize}
199
200\item \member{__name__} is the attribute's name.
201
202\item \member{__doc__} is the attribute's docstring.
203
Andrew M. Kuchling9455df22001-12-03 20:55:37 +0000204\item \method{__get__(\var{object})} is a method that retrieves the
205attribute value from \var{object}.
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000206
Andrew M. Kuchling4f9e2202001-10-29 18:09:42 +0000207\item \method{__set__(\var{object}, \var{value})} sets the attribute
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000208on \var{object} to \var{value}.
209
Andrew M. Kuchlingc54fc312001-12-03 20:58:29 +0000210\item \method{__delete__(\var{object}, \var{value})} deletes the \var{value}
Andrew M. Kuchling9455df22001-12-03 20:55:37 +0000211attribute of \var{object}.
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000212\end{itemize}
213
214For example, when you write \code{obj.x}, the steps that Python
215actually performs are:
216
217\begin{verbatim}
218descriptor = obj.__class__.x
Andrew M. Kuchling7cc13de2001-10-30 14:22:11 +0000219descriptor.__get__(obj)
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000220\end{verbatim}
221
Andrew M. Kuchling7cc13de2001-10-30 14:22:11 +0000222For methods, \method{descriptor.__get__} returns a temporary object that's
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000223callable, and wraps up the instance and the method to be called on it.
224This is also why static methods and class methods are now possible;
225they have descriptors that wrap up just the method, or the method and
226the class. As a brief explanation of these new kinds of methods,
227static methods aren't passed the instance, and therefore resemble
228regular functions. Class methods are passed the class of the object,
Andrew M. Kuchling72a7fb72001-10-30 22:18:21 +0000229but not the object itself. Static and class methods are defined like
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000230this:
231
232\begin{verbatim}
Andrew M. Kuchlingccf04652001-11-26 18:15:44 +0000233class C(object):
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000234 def f(arg1, arg2):
235 ...
236 f = staticmethod(f)
237
238 def g(cls, arg1, arg2):
239 ...
240 g = classmethod(g)
241\end{verbatim}
242
243The \function{staticmethod()} function takes the function
244\function{f}, and returns it wrapped up in a descriptor so it can be
245stored in the class object. You might expect there to be special
246syntax for creating such methods (\code{def static f()},
247\code{defstatic f()}, or something like that) but no such syntax has
248been defined yet; that's been left for future versions.
249
250More new features, such as slots and properties, are also implemented
251as new kinds of descriptors, and it's not difficult to write a
252descriptor class that does something novel. For example, it would be
253possible to write a descriptor class that made it possible to write
254Eiffel-style preconditions and postconditions for a method. A class
255that used this feature might be defined like this:
256
257\begin{verbatim}
258from eiffel import eiffelmethod
259
Andrew M. Kuchlingccf04652001-11-26 18:15:44 +0000260class C(object):
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000261 def f(self, arg1, arg2):
262 # The actual function
263 def pre_f(self):
264 # Check preconditions
265 def post_f(self):
266 # Check postconditions
267
268 f = eiffelmethod(f, pre_f, post_f)
269\end{verbatim}
270
271Note that a person using the new \function{eiffelmethod()} doesn't
272have to understand anything about descriptors. This is why I think
273the new features don't increase the basic complexity of the language.
274There will be a few wizards who need to know about it in order to
275write \function{eiffelmethod()} or the ZODB or whatever, but most
276users will just write code on top of the resulting libraries and
277ignore the implementation details.
278
Andrew M. Kuchling4855b022001-10-23 20:26:16 +0000279\subsection{Multiple Inheritance: The Diamond Rule}
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000280
Andrew M. Kuchling4855b022001-10-23 20:26:16 +0000281Multiple inheritance has also been made more useful through changing
282the rules under which names are resolved. Consider this set of classes
283(diagram taken from \pep{253} by Guido van Rossum):
284
285\begin{verbatim}
286 class A:
287 ^ ^ def save(self): ...
288 / \
289 / \
290 / \
291 / \
292 class B class C:
293 ^ ^ def save(self): ...
294 \ /
295 \ /
296 \ /
297 \ /
298 class D
299\end{verbatim}
300
301The lookup rule for classic classes is simple but not very smart; the
302base classes are searched depth-first, going from left to right. A
303reference to \method{D.save} will search the classes \class{D},
304\class{B}, and then \class{A}, where \method{save()} would be found
305and returned. \method{C.save()} would never be found at all. This is
306bad, because if \class{C}'s \method{save()} method is saving some
307internal state specific to \class{C}, not calling it will result in
308that state never getting saved.
309
310New-style classes follow a different algorithm that's a bit more
311complicated to explain, but does the right thing in this situation.
312
313\begin{enumerate}
314
315\item List all the base classes, following the classic lookup rule and
316include a class multiple times if it's visited repeatedly. In the
317above example, the list of visited classes is [\class{D}, \class{B},
Andrew M. Kuchling28369072001-10-29 15:47:33 +0000318\class{A}, \class{C}, \class{A}].
Andrew M. Kuchling4855b022001-10-23 20:26:16 +0000319
320\item Scan the list for duplicated classes. If any are found, remove
321all but one occurrence, leaving the \emph{last} one in the list. In
322the above example, the list becomes [\class{D}, \class{B}, \class{C},
Andrew M. Kuchling28369072001-10-29 15:47:33 +0000323\class{A}] after dropping duplicates.
Andrew M. Kuchling4855b022001-10-23 20:26:16 +0000324
325\end{enumerate}
326
327Following this rule, referring to \method{D.save()} will return
328\method{C.save()}, which is the behaviour we're after. This lookup
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000329rule is the same as the one followed by Common Lisp.
Andrew M. Kuchling4855b022001-10-23 20:26:16 +0000330
Andrew M. Kuchling433b5c42001-10-30 21:36:04 +0000331% XXX mention super()
332
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000333
334\subsection{Attribute Access}
335
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000336A fair number of sophisticated Python classes define hooks for
337attribute access using \method{__getattr__}; most commonly this is
338done for convenience, to make code more readable by automatically
339mapping an attribute access such as \code{obj.parent} into a method
340call such as \code{obj.get_parent()}. Python 2.2 adds some new ways
341of controlling attribute access.
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000342
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000343First, \method{__getattr__(\var{attr_name})} is still supported by
344new-style classes, and nothing about it has changed. As before, it
345will be called when an attempt is made to access \code{obj.foo} and no
346attribute named \samp{foo} is found in the instance's dictionary.
347
348New-style classes also support a new method,
349\method{__getattribute__(\var{attr_name})}. The difference between
350the two methods is that \method{__getattribute__} is \emph{always}
351called whenever any attribute is accessed, while the old
352\method{__getattr__} is only called if \samp{foo} isn't found in the
353instance's dictionary.
354
355However, Python 2.2's support for \dfn{properties} will often be a
356simpler way to trap attribute references. Writing a
357\method{__getattr__} method is complicated because to avoid recursion
358you can't use regular attribute accesses inside them, and instead have
359to mess around with the contents of \member{__dict__}.
360\method{__getattr__} methods also end up being called by Python when
361it checks for other methods such as \method{__repr__} or
362\method{__coerce__}, and so have to be written with this in mind.
363Finally, calling a function on every attribute access results in a
364sizable performance loss.
365
366\class{property} is a new built-in type that packages up three
367functions that get, set, or delete an attribute, and a docstring. For
368example, if you want to define a \member{size} attribute that's
369computed, but also settable, you could write:
370
371\begin{verbatim}
Andrew M. Kuchlingccf04652001-11-26 18:15:44 +0000372class C(object):
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000373 def get_size (self):
374 result = ... computation ...
375 return result
376 def set_size (self, size):
377 ... compute something based on the size
378 and set internal state appropriately ...
379
380 # Define a property. The 'delete this attribute'
381 # method is defined as None, so the attribute
382 # can't be deleted.
383 size = property(get_size, set_size,
384 None,
385 "Storage size of this instance")
386\end{verbatim}
387
388That is certainly clearer and easier to write than a pair of
389\method{__getattr__}/\method{__setattr__} methods that check for the
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000390\member{size} attribute and handle it specially while retrieving all
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000391other attributes from the instance's \member{__dict__}. Accesses to
392\member{size} are also the only ones which have to perform the work of
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000393calling a function, so references to other attributes run at
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000394their usual speed.
395
396Finally, it's possible to constrain the list of attributes that can be
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000397referenced on an object using the new \member{__slots__} class attribute.
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000398Python objects are usually very dynamic; at any time it's possible to
399define a new attribute on an instance by just doing
400\code{obj.new_attr=1}. This is flexible and convenient, but this
401flexibility can also lead to bugs, as when you meant to write
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000402\code{obj.template = 'a'} but made a typo and wrote
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000403\code{obj.templtae} by accident.
404
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000405A new-style class can define a class attribute named \member{__slots__}
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000406to constrain the list of legal attribute names. An example will make
407this clear:
408
409\begin{verbatim}
410>>> class C(object):
Andrew M. Kuchling038215a2001-12-07 14:22:13 +0000411... __slots__ = ('template', 'name')
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000412...
413>>> obj = C()
414>>> print obj.template
415None
Andrew M. Kuchling28369072001-10-29 15:47:33 +0000416>>> obj.template = 'Test'
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000417>>> print obj.template
418Test
Andrew M. Kuchling28369072001-10-29 15:47:33 +0000419>>> obj.templtae = None
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000420Traceback (most recent call last):
421 File "<stdin>", line 1, in ?
422AttributeError: 'C' object has no attribute 'templtae'
423\end{verbatim}
424
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000425Note how you get an \exception{AttributeError} on the attempt to
426assign to an attribute not listed in \member{__slots__}.
Andrew M. Kuchling4855b022001-10-23 20:26:16 +0000427
428
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000429\subsection{Related Links}
Andrew M. Kuchlingcf31d5d2001-10-26 20:37:55 +0000430\label{sect-rellinks}
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000431
432This section has just been a quick overview of the new features,
433giving enough of an explanation to start you programming, but many
434details have been simplified or ignored. Where should you go to get a
435more complete picture?
436
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000437\url{http://www.python.org/2.2/descrintro.html} is a lengthy tutorial
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000438introduction to the descriptor features, written by Guido van Rossum.
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000439If my description has whetted your appetite, go read this tutorial
440next, because it goes into much more detail about the new features
441while still remaining quite easy to read.
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000442
443Next, there are two relevant PEPs, \pep{252} and \pep{253}. \pep{252}
444is titled "Making Types Look More Like Classes", and covers the
445descriptor API. \pep{253} is titled "Subtyping Built-in Types", and
446describes the changes to type objects that make it possible to subtype
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000447built-in objects. \pep{253} is the more complicated PEP of the two,
448and at a few points the necessary explanations of types and meta-types
449may cause your head to explode. Both PEPs were written and
450implemented by Guido van Rossum, with substantial assistance from the
451rest of the Zope Corp. team.
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000452
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000453Finally, there's the ultimate authority: the source code. Most of the
454machinery for the type handling is in \file{Objects/typeobject.c}, but
455you should only resort to it after all other avenues have been
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000456exhausted, including posting a question to python-list or python-dev.
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +0000457
Andrew M. Kuchling8cfa9052001-07-19 01:19:59 +0000458
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +0000459%======================================================================
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000460\section{PEP 234: Iterators}
461
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000462Another significant addition to 2.2 is an iteration interface at both
463the C and Python levels. Objects can define how they can be looped
464over by callers.
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000465
466In Python versions up to 2.1, the usual way to make \code{for item in
467obj} work is to define a \method{__getitem__()} method that looks
468something like this:
469
470\begin{verbatim}
471 def __getitem__(self, index):
472 return <next item>
473\end{verbatim}
474
475\method{__getitem__()} is more properly used to define an indexing
476operation on an object so that you can write \code{obj[5]} to retrieve
Andrew M. Kuchling8c69c912001-08-07 14:28:58 +0000477the sixth element. It's a bit misleading when you're using this only
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000478to support \keyword{for} loops. Consider some file-like object that
479wants to be looped over; the \var{index} parameter is essentially
480meaningless, as the class probably assumes that a series of
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000481\method{__getitem__()} calls will be made with \var{index}
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000482incrementing by one each time. In other words, the presence of the
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +0000483\method{__getitem__()} method doesn't mean that using \code{file[5]}
484to randomly access the sixth element will work, though it really should.
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000485
486In Python 2.2, iteration can be implemented separately, and
487\method{__getitem__()} methods can be limited to classes that really
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000488do support random access. The basic idea of iterators is
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +0000489simple. A new built-in function, \function{iter(obj)} or
490\code{iter(\var{C}, \var{sentinel})}, is used to get an iterator.
491\function{iter(obj)} returns an iterator for the object \var{obj},
492while \code{iter(\var{C}, \var{sentinel})} returns an iterator that
493will invoke the callable object \var{C} until it returns
494\var{sentinel} to signal that the iterator is done.
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000495
496Python classes can define an \method{__iter__()} method, which should
497create and return a new iterator for the object; if the object is its
498own iterator, this method can just return \code{self}. In particular,
499iterators will usually be their own iterators. Extension types
500implemented in C can implement a \code{tp_iter} function in order to
Andrew M. Kuchling4cf52a92001-07-17 12:48:48 +0000501return an iterator, and extension types that want to behave as
502iterators can define a \code{tp_iternext} function.
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000503
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000504So, after all this, what do iterators actually do? They have one
505required method, \method{next()}, which takes no arguments and returns
506the next value. When there are no more values to be returned, calling
507\method{next()} should raise the \exception{StopIteration} exception.
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000508
509\begin{verbatim}
510>>> L = [1,2,3]
511>>> i = iter(L)
512>>> print i
513<iterator object at 0x8116870>
514>>> i.next()
5151
516>>> i.next()
5172
518>>> i.next()
5193
520>>> i.next()
521Traceback (most recent call last):
522 File "<stdin>", line 1, in ?
523StopIteration
524>>>
525\end{verbatim}
526
527In 2.2, Python's \keyword{for} statement no longer expects a sequence;
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000528it expects something for which \function{iter()} will return an iterator.
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +0000529For backward compatibility and convenience, an iterator is
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000530automatically constructed for sequences that don't implement
531\method{__iter__()} or a \code{tp_iter} slot, so \code{for i in
532[1,2,3]} will still work. Wherever the Python interpreter loops over
533a sequence, it's been changed to use the iterator protocol. This
534means you can do things like this:
535
536\begin{verbatim}
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000537>>> L = [1,2,3]
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000538>>> i = iter(L)
539>>> a,b,c = i
540>>> a,b,c
541(1, 2, 3)
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000542\end{verbatim}
543
Andrew M. Kuchling9e9c1352001-08-11 03:06:50 +0000544Iterator support has been added to some of Python's basic types.
Fred Drake0d002542001-07-17 13:55:33 +0000545Calling \function{iter()} on a dictionary will return an iterator
Andrew M. Kuchling6ea9f0b2001-07-17 14:50:31 +0000546which loops over its keys:
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000547
548\begin{verbatim}
549>>> m = {'Jan': 1, 'Feb': 2, 'Mar': 3, 'Apr': 4, 'May': 5, 'Jun': 6,
550... 'Jul': 7, 'Aug': 8, 'Sep': 9, 'Oct': 10, 'Nov': 11, 'Dec': 12}
551>>> for key in m: print key, m[key]
552...
553Mar 3
554Feb 2
555Aug 8
556Sep 9
557May 5
558Jun 6
559Jul 7
560Jan 1
561Apr 4
562Nov 11
563Dec 12
564Oct 10
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000565\end{verbatim}
566
567That's just the default behaviour. If you want to iterate over keys,
568values, or key/value pairs, you can explicitly call the
569\method{iterkeys()}, \method{itervalues()}, or \method{iteritems()}
Andrew M. Kuchling9e9c1352001-08-11 03:06:50 +0000570methods to get an appropriate iterator. In a minor related change,
571the \keyword{in} operator now works on dictionaries, so
572\code{\var{key} in dict} is now equivalent to
573\code{dict.has_key(\var{key})}.
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000574
Andrew M. Kuchling9e9c1352001-08-11 03:06:50 +0000575Files also provide an iterator, which calls the \method{readline()}
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000576method until there are no more lines in the file. This means you can
577now read each line of a file using code like this:
578
579\begin{verbatim}
580for line in file:
581 # do something for each line
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000582 ...
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000583\end{verbatim}
584
585Note that you can only go forward in an iterator; there's no way to
586get the previous element, reset the iterator, or make a copy of it.
Fred Drake0d002542001-07-17 13:55:33 +0000587An iterator object could provide such additional capabilities, but the
588iterator protocol only requires a \method{next()} method.
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000589
590\begin{seealso}
591
592\seepep{234}{Iterators}{Written by Ka-Ping Yee and GvR; implemented
593by the Python Labs crew, mostly by GvR and Tim Peters.}
594
595\end{seealso}
596
Andrew M. Kuchling8cfa9052001-07-19 01:19:59 +0000597
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000598%======================================================================
599\section{PEP 255: Simple Generators}
600
601Generators are another new feature, one that interacts with the
602introduction of iterators.
603
604You're doubtless familiar with how function calls work in Python or
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +0000605C. When you call a function, it gets a private namespace where its local
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000606variables are created. When the function reaches a \keyword{return}
607statement, the local variables are destroyed and the resulting value
608is returned to the caller. A later call to the same function will get
609a fresh new set of local variables. But, what if the local variables
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000610weren't thrown away on exiting a function? What if you could later
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000611resume the function where it left off? This is what generators
612provide; they can be thought of as resumable functions.
613
614Here's the simplest example of a generator function:
615
616\begin{verbatim}
617def generate_ints(N):
618 for i in range(N):
619 yield i
620\end{verbatim}
621
622A new keyword, \keyword{yield}, was introduced for generators. Any
623function containing a \keyword{yield} statement is a generator
624function; this is detected by Python's bytecode compiler which
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +0000625compiles the function specially as a result. Because a new keyword was
Andrew M. Kuchling4cf52a92001-07-17 12:48:48 +0000626introduced, generators must be explicitly enabled in a module by
627including a \code{from __future__ import generators} statement near
628the top of the module's source code. In Python 2.3 this statement
629will become unnecessary.
630
631When you call a generator function, it doesn't return a single value;
632instead it returns a generator object that supports the iterator
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +0000633protocol. On executing the \keyword{yield} statement, the generator
Andrew M. Kuchling4cf52a92001-07-17 12:48:48 +0000634outputs the value of \code{i}, similar to a \keyword{return}
635statement. The big difference between \keyword{yield} and a
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +0000636\keyword{return} statement is that on reaching a \keyword{yield} the
Andrew M. Kuchling4cf52a92001-07-17 12:48:48 +0000637generator's state of execution is suspended and local variables are
638preserved. On the next call to the generator's \code{.next()} method,
639the function will resume executing immediately after the
640\keyword{yield} statement. (For complicated reasons, the
641\keyword{yield} statement isn't allowed inside the \keyword{try} block
Andrew M. Kuchlingcf31d5d2001-10-26 20:37:55 +0000642of a \code{try...finally} statement; read \pep{255} for a full
Andrew M. Kuchling4cf52a92001-07-17 12:48:48 +0000643explanation of the interaction between \keyword{yield} and
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000644exceptions.)
645
646Here's a sample usage of the \function{generate_ints} generator:
647
648\begin{verbatim}
649>>> gen = generate_ints(3)
650>>> gen
651<generator object at 0x8117f90>
652>>> gen.next()
6530
654>>> gen.next()
6551
656>>> gen.next()
6572
658>>> gen.next()
659Traceback (most recent call last):
660 File "<stdin>", line 1, in ?
661 File "<stdin>", line 2, in generate_ints
662StopIteration
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000663\end{verbatim}
664
665You could equally write \code{for i in generate_ints(5)}, or
666\code{a,b,c = generate_ints(3)}.
667
668Inside a generator function, the \keyword{return} statement can only
Andrew M. Kuchling4cf52a92001-07-17 12:48:48 +0000669be used without a value, and signals the end of the procession of
670values; afterwards the generator cannot return any further values.
671\keyword{return} with a value, such as \code{return 5}, is a syntax
672error inside a generator function. The end of the generator's results
673can also be indicated by raising \exception{StopIteration} manually,
674or by just letting the flow of execution fall off the bottom of the
675function.
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000676
677You could achieve the effect of generators manually by writing your
Andrew M. Kuchling4cf52a92001-07-17 12:48:48 +0000678own class and storing all the local variables of the generator as
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000679instance variables. For example, returning a list of integers could
680be done by setting \code{self.count} to 0, and having the
681\method{next()} method increment \code{self.count} and return it.
Andrew M. Kuchlingc32cc7c2001-07-17 18:25:01 +0000682However, for a moderately complicated generator, writing a
683corresponding class would be much messier.
684\file{Lib/test/test_generators.py} contains a number of more
685interesting examples. The simplest one implements an in-order
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000686traversal of a tree using generators recursively.
687
688\begin{verbatim}
689# A recursive generator that generates Tree leaves in in-order.
690def inorder(t):
691 if t:
692 for x in inorder(t.left):
693 yield x
694 yield t.label
695 for x in inorder(t.right):
696 yield x
697\end{verbatim}
698
699Two other examples in \file{Lib/test/test_generators.py} produce
700solutions for the N-Queens problem (placing $N$ queens on an $NxN$
701chess board so that no queen threatens another) and the Knight's Tour
702(a route that takes a knight to every square of an $NxN$ chessboard
703without visiting any square twice).
704
705The idea of generators comes from other programming languages,
706especially Icon (\url{http://www.cs.arizona.edu/icon/}), where the
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +0000707idea of generators is central. In Icon, every
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000708expression and function call behaves like a generator. One example
709from ``An Overview of the Icon Programming Language'' at
710\url{http://www.cs.arizona.edu/icon/docs/ipd266.htm} gives an idea of
711what this looks like:
712
713\begin{verbatim}
714sentence := "Store it in the neighboring harbor"
715if (i := find("or", sentence)) > 5 then write(i)
716\end{verbatim}
717
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000718In Icon the \function{find()} function returns the indexes at which the
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000719substring ``or'' is found: 3, 23, 33. In the \keyword{if} statement,
720\code{i} is first assigned a value of 3, but 3 is less than 5, so the
721comparison fails, and Icon retries it with the second value of 23. 23
722is greater than 5, so the comparison now succeeds, and the code prints
723the value 23 to the screen.
724
725Python doesn't go nearly as far as Icon in adopting generators as a
726central concept. Generators are considered a new part of the core
727Python language, but learning or using them isn't compulsory; if they
728don't solve any problems that you have, feel free to ignore them.
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +0000729One novel feature of Python's interface as compared to
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000730Icon's is that a generator's state is represented as a concrete object
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000731(the iterator) that can be passed around to other functions or stored
732in a data structure.
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000733
734\begin{seealso}
735
Andrew M. Kuchling4cf52a92001-07-17 12:48:48 +0000736\seepep{255}{Simple Generators}{Written by Neil Schemenauer, Tim
737Peters, Magnus Lie Hetland. Implemented mostly by Neil Schemenauer
738and Tim Peters, with other fixes from the Python Labs crew.}
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000739
740\end{seealso}
741
Andrew M. Kuchling8cfa9052001-07-19 01:19:59 +0000742
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000743%======================================================================
Andrew M. Kuchling2f0047a2001-09-05 14:53:31 +0000744\section{PEP 237: Unifying Long Integers and Integers}
745
Andrew M. Kuchling26c39bf2001-09-10 03:20:53 +0000746In recent versions, the distinction between regular integers, which
747are 32-bit values on most machines, and long integers, which can be of
748arbitrary size, was becoming an annoyance. For example, on platforms
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +0000749that support files larger than \code{2**32} bytes, the
Andrew M. Kuchling26c39bf2001-09-10 03:20:53 +0000750\method{tell()} method of file objects has to return a long integer.
751However, there were various bits of Python that expected plain
752integers and would raise an error if a long integer was provided
Andrew M. Kuchlingd6e40e22001-09-10 16:18:50 +0000753instead. For example, in Python 1.5, only regular integers
Andrew M. Kuchling26c39bf2001-09-10 03:20:53 +0000754could be used as a slice index, and \code{'abc'[1L:]} would raise a
755\exception{TypeError} exception with the message 'slice index must be
756int'.
Andrew M. Kuchling2f0047a2001-09-05 14:53:31 +0000757
Andrew M. Kuchling26c39bf2001-09-10 03:20:53 +0000758Python 2.2 will shift values from short to long integers as required.
759The 'L' suffix is no longer needed to indicate a long integer literal,
760as now the compiler will choose the appropriate type. (Using the 'L'
761suffix will be discouraged in future 2.x versions of Python,
762triggering a warning in Python 2.4, and probably dropped in Python
7633.0.) Many operations that used to raise an \exception{OverflowError}
764will now return a long integer as their result. For example:
765
766\begin{verbatim}
767>>> 1234567890123
Andrew M. Kuchlingd6e40e22001-09-10 16:18:50 +00007681234567890123L
769>>> 2 ** 64
77018446744073709551616L
Andrew M. Kuchling26c39bf2001-09-10 03:20:53 +0000771\end{verbatim}
772
773In most cases, integers and long integers will now be treated
774identically. You can still distinguish them with the
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000775\function{type()} built-in function, but that's rarely needed.
Andrew M. Kuchling26c39bf2001-09-10 03:20:53 +0000776
Andrew M. Kuchling26c39bf2001-09-10 03:20:53 +0000777\begin{seealso}
778
779\seepep{237}{Unifying Long Integers and Integers}{Written by
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000780Moshe Zadka and Guido van Rossum. Implemented mostly by Guido van
781Rossum.}
Andrew M. Kuchling26c39bf2001-09-10 03:20:53 +0000782
783\end{seealso}
Andrew M. Kuchling2f0047a2001-09-05 14:53:31 +0000784
Andrew M. Kuchlingd4707e32001-09-28 20:46:46 +0000785
Andrew M. Kuchling2f0047a2001-09-05 14:53:31 +0000786%======================================================================
Andrew M. Kuchling9e9c1352001-08-11 03:06:50 +0000787\section{PEP 238: Changing the Division Operator}
788
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000789The most controversial change in Python 2.2 heralds the start of an effort
Andrew M. Kuchling9e9c1352001-08-11 03:06:50 +0000790to fix an old design flaw that's been in Python from the beginning.
791Currently Python's division operator, \code{/}, behaves like C's
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +0000792division operator when presented with two integer arguments: it
Andrew M. Kuchling9e9c1352001-08-11 03:06:50 +0000793returns an integer result that's truncated down when there would be
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +0000794a fractional part. For example, \code{3/2} is 1, not 1.5, and
Andrew M. Kuchling9e9c1352001-08-11 03:06:50 +0000795\code{(-1)/2} is -1, not -0.5. This means that the results of divison
796can vary unexpectedly depending on the type of the two operands and
797because Python is dynamically typed, it can be difficult to determine
798the possible types of the operands.
799
800(The controversy is over whether this is \emph{really} a design flaw,
801and whether it's worth breaking existing code to fix this. It's
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000802caused endless discussions on python-dev, and in July 2001 erupted into an
Andrew M. Kuchling9e9c1352001-08-11 03:06:50 +0000803storm of acidly sarcastic postings on \newsgroup{comp.lang.python}. I
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +0000804won't argue for either side here and will stick to describing what's
Andrew M. Kuchlingcf31d5d2001-10-26 20:37:55 +0000805implemented in 2.2. Read \pep{238} for a summary of arguments and
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +0000806counter-arguments.)
Andrew M. Kuchling9e9c1352001-08-11 03:06:50 +0000807
808Because this change might break code, it's being introduced very
809gradually. Python 2.2 begins the transition, but the switch won't be
810complete until Python 3.0.
811
Andrew M. Kuchlingcf31d5d2001-10-26 20:37:55 +0000812First, I'll borrow some terminology from \pep{238}. ``True division'' is the
Andrew M. Kuchling9e9c1352001-08-11 03:06:50 +0000813division that most non-programmers are familiar with: 3/2 is 1.5, 1/4
814is 0.25, and so forth. ``Floor division'' is what Python's \code{/}
815operator currently does when given integer operands; the result is the
816floor of the value returned by true division. ``Classic division'' is
817the current mixed behaviour of \code{/}; it returns the result of
818floor division when the operands are integers, and returns the result
819of true division when one of the operands is a floating-point number.
820
821Here are the changes 2.2 introduces:
822
823\begin{itemize}
824
825\item A new operator, \code{//}, is the floor division operator.
826(Yes, we know it looks like \Cpp's comment symbol.) \code{//}
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000827\emph{always} performs floor division no matter what the types of
Andrew M. Kuchling9e9c1352001-08-11 03:06:50 +0000828its operands are, so \code{1 // 2} is 0 and \code{1.0 // 2.0} is also
8290.0.
830
831\code{//} is always available in Python 2.2; you don't need to enable
832it using a \code{__future__} statement.
833
Andrew M. Kuchling4f9e2202001-10-29 18:09:42 +0000834\item By including a \code{from __future__ import division} in a
Andrew M. Kuchling9e9c1352001-08-11 03:06:50 +0000835module, the \code{/} operator will be changed to return the result of
836true division, so \code{1/2} is 0.5. Without the \code{__future__}
837statement, \code{/} still means classic division. The default meaning
838of \code{/} will not change until Python 3.0.
839
840\item Classes can define methods called \method{__truediv__} and
841\method{__floordiv__} to overload the two division operators. At the
842C level, there are also slots in the \code{PyNumberMethods} structure
843so extension types can define the two operators.
844
Andrew M. Kuchlingcf31d5d2001-10-26 20:37:55 +0000845\item Python 2.2 supports some command-line arguments for testing
846whether code will works with the changed division semantics. Running
847python with \programopt{-Q warn} will cause a warning to be issued
848whenever division is applied to two integers. You can use this to
849find code that's affected by the change and fix it. By default,
850Python 2.2 will simply perform classic division without a warning; the
851warning will be turned on by default in Python 2.3.
Andrew M. Kuchling9e9c1352001-08-11 03:06:50 +0000852
853\end{itemize}
854
855\begin{seealso}
856
857\seepep{238}{Changing the Division Operator}{Written by Moshe Zadka and
858Guido van Rossum. Implemented by Guido van Rossum..}
859
860\end{seealso}
861
862
863%======================================================================
Andrew M. Kuchlinga43e7032001-06-27 20:32:12 +0000864\section{Unicode Changes}
865
Andrew M. Kuchling2cd712b2001-07-16 13:39:08 +0000866Python's Unicode support has been enhanced a bit in 2.2. Unicode
Andrew M. Kuchlinga6d2a042001-07-20 18:34:34 +0000867strings are usually stored as UCS-2, as 16-bit unsigned integers.
Andrew M. Kuchlingf5fec3c2001-07-19 01:48:08 +0000868Python 2.2 can also be compiled to use UCS-4, 32-bit unsigned
869integers, as its internal encoding by supplying
Andrew M. Kuchling433b5c42001-10-30 21:36:04 +0000870\longprogramopt{enable-unicode=ucs4} to the configure script.
871(It's also possible to specify
872\longprogramopt{disable-unicode} to completely disable Unicode
873support.)
874
875When built to use UCS-4 (a ``wide Python''), the interpreter can
876natively handle Unicode characters from U+000000 to U+110000, so the
877range of legal values for the \function{unichr()} function is expanded
Andrew M. Kuchlinga6d2a042001-07-20 18:34:34 +0000878accordingly. Using an interpreter compiled to use UCS-2 (a ``narrow
879Python''), values greater than 65535 will still cause
880\function{unichr()} to raise a \exception{ValueError} exception.
Andrew M. Kuchling433b5c42001-10-30 21:36:04 +0000881This is all described in \pep{261}, ``Support for `wide' Unicode
882characters''; consult it for further details.
Andrew M. Kuchlingab010872001-07-19 14:59:53 +0000883
Andrew M. Kuchling433b5c42001-10-30 21:36:04 +0000884Another change is simpler to explain. Since their introduction,
Andrew M. Kuchlingab010872001-07-19 14:59:53 +0000885Unicode strings have supported an \method{encode()} method to convert
886the string to a selected encoding such as UTF-8 or Latin-1. A
887symmetric \method{decode(\optional{\var{encoding}})} method has been
888added to 8-bit strings (though not to Unicode strings) in 2.2.
889\method{decode()} assumes that the string is in the specified encoding
890and decodes it, returning whatever is returned by the codec.
891
892Using this new feature, codecs have been added for tasks not directly
893related to Unicode. For example, codecs have been added for
894uu-encoding, MIME's base64 encoding, and compression with the
895\module{zlib} module:
Andrew M. Kuchling2cd712b2001-07-16 13:39:08 +0000896
897\begin{verbatim}
898>>> s = """Here is a lengthy piece of redundant, overly verbose,
899... and repetitive text.
900... """
901>>> data = s.encode('zlib')
902>>> data
903'x\x9c\r\xc9\xc1\r\x80 \x10\x04\xc0?Ul...'
904>>> data.decode('zlib')
905'Here is a lengthy piece of redundant, overly verbose,\nand repetitive text.\n'
906>>> print s.encode('uu')
907begin 666 <data>
908M2&5R92!I<R!A(&QE;F=T:'D@<&EE8V4@;V8@<F5D=6YD86YT+"!O=F5R;'D@
909>=F5R8F]S92P*86YD(')E<&5T:71I=F4@=&5X="X*
910
911end
912>>> "sheesh".encode('rot-13')
913'furrfu'
914\end{verbatim}
Andrew M. Kuchlinga43e7032001-06-27 20:32:12 +0000915
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +0000916To convert a class instance to Unicode, a \method{__unicode__} method
Andrew M. Kuchlingcf31d5d2001-10-26 20:37:55 +0000917can be defined by a class, analogous to \method{__str__}.
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +0000918
Andrew M. Kuchlingcf31d5d2001-10-26 20:37:55 +0000919\method{encode()}, \method{decode()}, and \method{__unicode__} were
920implemented by Marc-Andr\'e Lemburg. The changes to support using
921UCS-4 internally were implemented by Fredrik Lundh and Martin von
922L\"owis.
Andrew M. Kuchlinga43e7032001-06-27 20:32:12 +0000923
Andrew M. Kuchlingf5fec3c2001-07-19 01:48:08 +0000924\begin{seealso}
925
Andrew M. Kuchling433b5c42001-10-30 21:36:04 +0000926\seepep{261}{Support for `wide' Unicode characters}{Written by
927Paul Prescod.}
Andrew M. Kuchlingf5fec3c2001-07-19 01:48:08 +0000928
929\end{seealso}
Andrew M. Kuchling8cfa9052001-07-19 01:19:59 +0000930
Andrew M. Kuchlingcf31d5d2001-10-26 20:37:55 +0000931
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000932%======================================================================
933\section{PEP 227: Nested Scopes}
934
935In Python 2.1, statically nested scopes were added as an optional
936feature, to be enabled by a \code{from __future__ import
937nested_scopes} directive. In 2.2 nested scopes no longer need to be
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +0000938specially enabled, and are now always present. The rest of this section
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000939is a copy of the description of nested scopes from my ``What's New in
940Python 2.1'' document; if you read it when 2.1 came out, you can skip
941the rest of this section.
942
943The largest change introduced in Python 2.1, and made complete in 2.2,
944is to Python's scoping rules. In Python 2.0, at any given time there
945are at most three namespaces used to look up variable names: local,
946module-level, and the built-in namespace. This often surprised people
947because it didn't match their intuitive expectations. For example, a
948nested recursive function definition doesn't work:
949
950\begin{verbatim}
951def f():
952 ...
953 def g(value):
954 ...
955 return g(value-1) + 1
956 ...
957\end{verbatim}
958
959The function \function{g()} will always raise a \exception{NameError}
960exception, because the binding of the name \samp{g} isn't in either
961its local namespace or in the module-level namespace. This isn't much
962of a problem in practice (how often do you recursively define interior
963functions like this?), but this also made using the \keyword{lambda}
964statement clumsier, and this was a problem in practice. In code which
965uses \keyword{lambda} you can often find local variables being copied
966by passing them as the default values of arguments.
967
968\begin{verbatim}
969def find(self, name):
970 "Return list of any entries equal to 'name'"
971 L = filter(lambda x, name=name: x == name,
972 self.list_attribute)
973 return L
974\end{verbatim}
975
976The readability of Python code written in a strongly functional style
977suffers greatly as a result.
978
979The most significant change to Python 2.2 is that static scoping has
980been added to the language to fix this problem. As a first effect,
981the \code{name=name} default argument is now unnecessary in the above
982example. Put simply, when a given variable name is not assigned a
983value within a function (by an assignment, or the \keyword{def},
984\keyword{class}, or \keyword{import} statements), references to the
985variable will be looked up in the local namespace of the enclosing
986scope. A more detailed explanation of the rules, and a dissection of
987the implementation, can be found in the PEP.
988
989This change may cause some compatibility problems for code where the
990same variable name is used both at the module level and as a local
991variable within a function that contains further function definitions.
992This seems rather unlikely though, since such code would have been
993pretty confusing to read in the first place.
994
995One side effect of the change is that the \code{from \var{module}
996import *} and \keyword{exec} statements have been made illegal inside
997a function scope under certain conditions. The Python reference
998manual has said all along that \code{from \var{module} import *} is
999only legal at the top level of a module, but the CPython interpreter
1000has never enforced this before. As part of the implementation of
1001nested scopes, the compiler which turns Python source into bytecodes
1002has to generate different code to access variables in a containing
1003scope. \code{from \var{module} import *} and \keyword{exec} make it
1004impossible for the compiler to figure this out, because they add names
1005to the local namespace that are unknowable at compile time.
1006Therefore, if a function contains function definitions or
1007\keyword{lambda} expressions with free variables, the compiler will
1008flag this by raising a \exception{SyntaxError} exception.
1009
1010To make the preceding explanation a bit clearer, here's an example:
1011
1012\begin{verbatim}
1013x = 1
1014def f():
1015 # The next line is a syntax error
1016 exec 'x=2'
1017 def g():
1018 return x
1019\end{verbatim}
1020
1021Line 4 containing the \keyword{exec} statement is a syntax error,
1022since \keyword{exec} would define a new local variable named \samp{x}
1023whose value should be accessed by \function{g()}.
1024
1025This shouldn't be much of a limitation, since \keyword{exec} is rarely
1026used in most Python code (and when it is used, it's often a sign of a
1027poor design anyway).
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +00001028
1029\begin{seealso}
1030
1031\seepep{227}{Statically Nested Scopes}{Written and implemented by
1032Jeremy Hylton.}
1033
1034\end{seealso}
1035
Andrew M. Kuchlinga43e7032001-06-27 20:32:12 +00001036
1037%======================================================================
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +00001038\section{New and Improved Modules}
1039
1040\begin{itemize}
1041
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +00001042 \item The \module{xmlrpclib} module was contributed to the standard
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +00001043 library by Fredrik Lundh, provding support for writing XML-RPC
1044 clients. XML-RPC is a simple remote procedure call protocol built on
Andrew M. Kuchling8c69c912001-08-07 14:28:58 +00001045 top of HTTP and XML. For example, the following snippet retrieves a
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +00001046 list of RSS channels from the O'Reilly Network, and then
1047 lists the recent headlines for one channel:
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +00001048
1049\begin{verbatim}
1050import xmlrpclib
1051s = xmlrpclib.Server(
1052 'http://www.oreillynet.com/meerkat/xml-rpc/server.php')
1053channels = s.meerkat.getChannels()
1054# channels is a list of dictionaries, like this:
1055# [{'id': 4, 'title': 'Freshmeat Daily News'}
1056# {'id': 190, 'title': '32Bits Online'},
1057# {'id': 4549, 'title': '3DGamers'}, ... ]
1058
1059# Get the items for one channel
1060items = s.meerkat.getItems( {'channel': 4} )
1061
1062# 'items' is another list of dictionaries, like this:
1063# [{'link': 'http://freshmeat.net/releases/52719/',
1064# 'description': 'A utility which converts HTML to XSL FO.',
1065# 'title': 'html2fo 0.3 (Default)'}, ... ]
1066\end{verbatim}
1067
Andrew M. Kuchlingd4707e32001-09-28 20:46:46 +00001068The \module{SimpleXMLRPCServer} module makes it easy to create
1069straightforward XML-RPC servers. See \url{http://www.xmlrpc.com/} for
1070more information about XML-RPC.
1071
Andrew M. Kuchling433b5c42001-10-30 21:36:04 +00001072 \item The new \module{hmac} module implements the HMAC
Andrew M. Kuchlingd4707e32001-09-28 20:46:46 +00001073 algorithm described by \rfc{2104}.
Andrew M. Kuchling433b5c42001-10-30 21:36:04 +00001074 (Contributed by Gerhard H\"aring.)
1075
1076 \item Several functions that originally returned lengthy tuples now
1077 return pseudo-sequences that still behave like tuples but also have
1078 mnemonic attributes such as member{st_mtime} or \member{tm_year}.
1079 The enhanced functions include \function{stat()},
1080 \function{fstat()}, \function{statvfs()}, and \function{fstatvfs()}
1081 in the \module{os} module, and \function{localtime()},
1082 \function{gmtime()}, and \function{strptime()} in the \module{time}
1083 module.
1084
1085 For example, to obtain a file's size using the old tuples, you'd end
1086 up writing something like \code{file_size =
1087 os.stat(filename)[stat.ST_SIZE]}, but now this can be written more
1088 clearly as \code{file_size = os.stat(filename).st_size}.
1089
1090 The original patch for this feature was contributed by Nick Mathewson.
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +00001091
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +00001092 \item The Python profiler has been extensively reworked and various
1093 errors in its output have been corrected. (Contributed by Fred
1094 Fred~L. Drake, Jr. and Tim Peters.)
1095
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +00001096 \item The \module{socket} module can be compiled to support IPv6;
Andrew M. Kuchlingddeb1352001-07-16 14:35:52 +00001097 specify the \longprogramopt{enable-ipv6} option to Python's configure
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +00001098 script. (Contributed by Jun-ichiro ``itojun'' Hagino.)
1099
1100 \item Two new format characters were added to the \module{struct}
1101 module for 64-bit integers on platforms that support the C
1102 \ctype{long long} type. \samp{q} is for a signed 64-bit integer,
1103 and \samp{Q} is for an unsigned one. The value is returned in
1104 Python's long integer type. (Contributed by Tim Peters.)
1105
1106 \item In the interpreter's interactive mode, there's a new built-in
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +00001107 function \function{help()} that uses the \module{pydoc} module
1108 introduced in Python 2.1 to provide interactive help.
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +00001109 \code{help(\var{object})} displays any available help text about
1110 \var{object}. \code{help()} with no argument puts you in an online
1111 help utility, where you can enter the names of functions, classes,
1112 or modules to read their help text.
1113 (Contributed by Guido van Rossum, using Ka-Ping Yee's \module{pydoc} module.)
1114
1115 \item Various bugfixes and performance improvements have been made
Andrew M. Kuchling4cf52a92001-07-17 12:48:48 +00001116 to the SRE engine underlying the \module{re} module. For example,
Andrew M. Kuchlingbeb38552001-10-22 14:11:06 +00001117 the \function{re.sub()} and \function{re.split()} functions have
1118 been rewritten in C. Another contributed patch speeds up certain
Andrew M. Kuchling433b5c42001-10-30 21:36:04 +00001119 Unicode character ranges by a factor of two, and a new \method{finditer()}
1120 method that returns an iterator over all the non-overlapping matches in
1121 a given string.
1122 (SRE is maintained by
Andrew M. Kuchlingbeb38552001-10-22 14:11:06 +00001123 Fredrik Lundh. The BIGCHARSET patch was contributed by Martin von
1124 L\"owis.)
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +00001125
Andrew M. Kuchling1efd7ad2001-09-14 16:19:27 +00001126 \item The \module{smtplib} module now supports \rfc{2487}, ``Secure
1127 SMTP over TLS'', so it's now possible to encrypt the SMTP traffic
1128 between a Python program and the mail transport agent being handed a
Andrew M. Kuchling433b5c42001-10-30 21:36:04 +00001129 message. \module{smtplib} also supports SMTP authentication.
1130 (Contributed by Gerhard H\"aring.)
Andrew M. Kuchling1efd7ad2001-09-14 16:19:27 +00001131
Andrew M. Kuchlinga6d2a042001-07-20 18:34:34 +00001132 \item The \module{imaplib} module, maintained by Piers Lauder, has
1133 support for several new extensions: the NAMESPACE extension defined
1134 in \rfc{2342}, SORT, GETACL and SETACL. (Contributed by Anthony
1135 Baxter and Michel Pelletier.)
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +00001136
Andrew M. Kuchling433b5c42001-10-30 21:36:04 +00001137 % XXX should the 'email' module get a section of its own?
Andrew M. Kuchlingd4707e32001-09-28 20:46:46 +00001138 \item The \module{rfc822} module's parsing of email addresses is now
1139 compliant with \rfc{2822}, an update to \rfc{822}. (The module's
1140 name is \emph{not} going to be changed to \samp{rfc2822}.) A new
1141 package, \module{email}, has also been added for parsing and
1142 generating e-mail messages. (Contributed by Barry Warsaw, and
1143 arising out of his work on Mailman.)
Andrew M. Kuchling77707672001-07-31 15:51:16 +00001144
Andrew M. Kuchlingbbde5882001-10-31 13:13:36 +00001145 \item The \module{difflib} module now contains a new \class{Differ}
1146 class for producing human-readable lists of changes (a ``delta'')
1147 between two sequences of lines of text. There are also two
1148 generator functions, \function{ndiff()} and \function{restore()},
1149 which respectively return a delta from two sequences, or one of the
1150 original sequences from a delta. (Grunt work contributed by David
1151 Goodger, from ndiff.py code by Tim Peters who then did the
1152 generatorization.)
1153
Andrew M. Kuchling77707672001-07-31 15:51:16 +00001154 \item New constants \constant{ascii_letters},
1155 \constant{ascii_lowercase}, and \constant{ascii_uppercase} were
1156 added to the \module{string} module. There were several modules in
1157 the standard library that used \constant{string.letters} to mean the
1158 ranges A-Za-z, but that assumption is incorrect when locales are in
1159 use, because \constant{string.letters} varies depending on the set
1160 of legal characters defined by the current locale. The buggy
1161 modules have all been fixed to use \constant{ascii_letters} instead.
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +00001162 (Reported by an unknown person; fixed by Fred~L. Drake, Jr.)
Andrew M. Kuchling77707672001-07-31 15:51:16 +00001163
Andrew M. Kuchling8c69c912001-08-07 14:28:58 +00001164 \item The \module{mimetypes} module now makes it easier to use
1165 alternative MIME-type databases by the addition of a
1166 \class{MimeTypes} class, which takes a list of filenames to be
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +00001167 parsed. (Contributed by Fred~L. Drake, Jr.)
Andrew M. Kuchling8c69c912001-08-07 14:28:58 +00001168
Andrew M. Kuchlingd6e40e22001-09-10 16:18:50 +00001169 \item A \class{Timer} class was added to the \module{threading}
1170 module that allows scheduling an activity to happen at some future
1171 time. (Contributed by Itamar Shtull-Trauring.)
Andrew M. Kuchling2f0047a2001-09-05 14:53:31 +00001172
Andrew M. Kuchling77707672001-07-31 15:51:16 +00001173\end{itemize}
1174
1175
1176%======================================================================
1177\section{Interpreter Changes and Fixes}
1178
1179Some of the changes only affect people who deal with the Python
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +00001180interpreter at the C level because they're writing Python extension modules,
Andrew M. Kuchling77707672001-07-31 15:51:16 +00001181embedding the interpreter, or just hacking on the interpreter itself.
1182If you only write Python code, none of the changes described here will
1183affect you very much.
1184
1185\begin{itemize}
1186
1187 \item Profiling and tracing functions can now be implemented in C,
1188 which can operate at much higher speeds than Python-based functions
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +00001189 and should reduce the overhead of profiling and tracing. This
1190 will be of interest to authors of development environments for
Andrew M. Kuchling77707672001-07-31 15:51:16 +00001191 Python. Two new C functions were added to Python's API,
1192 \cfunction{PyEval_SetProfile()} and \cfunction{PyEval_SetTrace()}.
1193 The existing \function{sys.setprofile()} and
1194 \function{sys.settrace()} functions still exist, and have simply
1195 been changed to use the new C-level interface. (Contributed by Fred
1196 L. Drake, Jr.)
1197
1198 \item Another low-level API, primarily of interest to implementors
1199 of Python debuggers and development tools, was added.
1200 \cfunction{PyInterpreterState_Head()} and
1201 \cfunction{PyInterpreterState_Next()} let a caller walk through all
1202 the existing interpreter objects;
1203 \cfunction{PyInterpreterState_ThreadHead()} and
1204 \cfunction{PyThreadState_Next()} allow looping over all the thread
1205 states for a given interpreter. (Contributed by David Beazley.)
1206
1207 \item A new \samp{et} format sequence was added to
1208 \cfunction{PyArg_ParseTuple}; \samp{et} takes both a parameter and
1209 an encoding name, and converts the parameter to the given encoding
1210 if the parameter turns out to be a Unicode string, or leaves it
1211 alone if it's an 8-bit string, assuming it to already be in the
1212 desired encoding. This differs from the \samp{es} format character,
1213 which assumes that 8-bit strings are in Python's default ASCII
1214 encoding and converts them to the specified new encoding.
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +00001215 (Contributed by M.-A. Lemburg, and used for the MBCS support on
1216 Windows described in the following section.)
Andrew M. Kuchlingcf31d5d2001-10-26 20:37:55 +00001217
1218 \item A different argument parsing function,
1219 \cfunction{PyArg_UnpackTuple()}, has been added that's simpler and
1220 presumably faster. Instead of specifying a format string, the
1221 caller simply gives the minimum and maximum number of arguments
1222 expected, and a set of pointers to \code{PyObject*} variables that
1223 will be filled in with argument values.
1224
Andrew M. Kuchling0ab31b82001-08-29 01:16:54 +00001225 \item Two new flags \constant{METH_NOARGS} and \constant{METH_O} are
1226 available in method definition tables to simplify implementation of
1227 methods with no arguments or a single untyped argument. Calling
1228 such methods is more efficient than calling a corresponding method
1229 that uses \constant{METH_VARARGS}.
1230 Also, the old \constant{METH_OLDARGS} style of writing C methods is
1231 now officially deprecated.
1232
1233\item
1234 Two new wrapper functions, \cfunction{PyOS_snprintf()} and
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +00001235 \cfunction{PyOS_vsnprintf()} were added to provide
Andrew M. Kuchling0ab31b82001-08-29 01:16:54 +00001236 cross-platform implementations for the relatively new
1237 \cfunction{snprintf()} and \cfunction{vsnprintf()} C lib APIs. In
1238 contrast to the standard \cfunction{sprintf()} and
1239 \cfunction{vsprintf()} functions, the Python versions check the
1240 bounds of the buffer used to protect against buffer overruns.
1241 (Contributed by M.-A. Lemburg.)
Andrew M. Kuchling77707672001-07-31 15:51:16 +00001242
Andrew M. Kuchlingccf04652001-11-26 18:15:44 +00001243 \item The \cfunction{_PyTuple_Resize()} function has lost an unused
1244 parameter, so now it takes 2 parameters instead of 3. The third
1245 argument was never used, and can simply be discarded when porting
1246 code from earlier versions to Python 2.2.
1247
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +00001248\end{itemize}
1249
1250
1251%======================================================================
1252\section{Other Changes and Fixes}
1253
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +00001254As usual there were a bunch of other improvements and bugfixes
1255scattered throughout the source tree. A search through the CVS change
Andrew M. Kuchlingadc7df52001-12-20 16:04:24 +00001256logs finds there were 527 patches applied, and 683 bugs fixed; both
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +00001257figures are likely to be underestimates. Some of the more notable
1258changes are:
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +00001259
1260\begin{itemize}
1261
Andrew M. Kuchling0e03f582001-08-30 21:30:16 +00001262 \item The code for the MacOS port for Python, maintained by Jack
1263 Jansen, is now kept in the main Python CVS tree, and many changes
Andrew M. Kuchling279e7442001-10-22 02:03:40 +00001264 have been made to support MacOS~X.
Andrew M. Kuchling0e03f582001-08-30 21:30:16 +00001265
1266The most significant change is the ability to build Python as a
1267framework, enabled by supplying the \longprogramopt{enable-framework}
1268option to the configure script when compiling Python. According to
1269Jack Jansen, ``This installs a self-contained Python installation plus
Andrew M. Kuchling279e7442001-10-22 02:03:40 +00001270the OS~X framework "glue" into
Andrew M. Kuchling0e03f582001-08-30 21:30:16 +00001271\file{/Library/Frameworks/Python.framework} (or another location of
1272choice). For now there is little immediate added benefit to this
1273(actually, there is the disadvantage that you have to change your PATH
1274to be able to find Python), but it is the basis for creating a
1275full-blown Python application, porting the MacPython IDE, possibly
1276using Python as a standard OSA scripting language and much more.''
1277
1278Most of the MacPython toolbox modules, which interface to MacOS APIs
Andrew M. Kuchling279e7442001-10-22 02:03:40 +00001279such as windowing, QuickTime, scripting, etc. have been ported to OS~X,
Andrew M. Kuchlingbeb38552001-10-22 14:11:06 +00001280but they've been left commented out in \file{setup.py}. People who want
Andrew M. Kuchling0e03f582001-08-30 21:30:16 +00001281to experiment with these modules can uncomment them manually.
1282
1283% Jack's original comments:
1284%The main change is the possibility to build Python as a
1285%framework. This installs a self-contained Python installation plus the
1286%OSX framework "glue" into /Library/Frameworks/Python.framework (or
1287%another location of choice). For now there is little immedeate added
1288%benefit to this (actually, there is the disadvantage that you have to
1289%change your PATH to be able to find Python), but it is the basis for
1290%creating a fullblown Python application, porting the MacPython IDE,
1291%possibly using Python as a standard OSA scripting language and much
1292%more. You enable this with "configure --enable-framework".
1293
1294%The other change is that most MacPython toolbox modules, which
1295%interface to all the MacOS APIs such as windowing, quicktime,
1296%scripting, etc. have been ported. Again, most of these are not of
1297%immedeate use, as they need a full application to be really useful, so
1298%they have been commented out in setup.py. People wanting to experiment
1299%can uncomment them. Gestalt and Internet Config modules are enabled by
1300%default.
Andrew M. Kuchling0e03f582001-08-30 21:30:16 +00001301
Andrew M. Kuchling2cd712b2001-07-16 13:39:08 +00001302 \item Keyword arguments passed to builtin functions that don't take them
1303 now cause a \exception{TypeError} exception to be raised, with the
1304 message "\var{function} takes no keyword arguments".
1305
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +00001306 \item Weak references, added in Python 2.1 as an extension module,
1307 are now part of the core because they're used in the implementation
1308 of new-style classes. The \exception{ReferenceError} exception has
1309 therefore moved from the \module{weakref} module to become a
1310 built-in exception.
1311
Andrew M. Kuchling94a7eba2001-08-15 15:55:48 +00001312 \item A new script, \file{Tools/scripts/cleanfuture.py} by Tim
1313 Peters, automatically removes obsolete \code{__future__} statements
1314 from Python source code.
Andrew M. Kuchling2cd712b2001-07-16 13:39:08 +00001315
Andrew M. Kuchling4f9e2202001-10-29 18:09:42 +00001316 \item An additional \var{flags} argument has been added to the
1317 built-in function \function{compile()}, so the behaviour of
1318 \code{__future__} statements can now be correctly observed in
1319 simulated shells, such as those presented by IDLE and other
1320 development environments. This is described in \pep{264}.
1321 (Contributed by Michael Hudson.)
1322
Andrew M. Kuchling2cd712b2001-07-16 13:39:08 +00001323 \item The new license introduced with Python 1.6 wasn't
1324 GPL-compatible. This is fixed by some minor textual changes to the
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +00001325 2.2 license, so it's now legal to embed Python inside a GPLed
1326 program again. Note that Python itself is not GPLed, but instead is
1327 under a license that's essentially equivalent to the BSD license,
1328 same as it always was. The license changes were also applied to the
1329 Python 2.0.1 and 2.1.1 releases.
Andrew M. Kuchling2cd712b2001-07-16 13:39:08 +00001330
Andrew M. Kuchlingf4ccf582001-07-31 01:11:36 +00001331 \item When presented with a Unicode filename on Windows, Python will
1332 now convert it to an MBCS encoded string, as used by the Microsoft
1333 file APIs. As MBCS is explicitly used by the file APIs, Python's
1334 choice of ASCII as the default encoding turns out to be an
Andrew M. Kuchling433b5c42001-10-30 21:36:04 +00001335 annoyance. On Unix, the locale's character set is used if
1336 \function{locale.nl_langinfo(CODESET)} is available. (Windows
1337 support was contributed by Mark Hammond with assistance from
1338 Marc-Andr\'e Lemburg. Unix support was added by Martin von L\"owis.)
Andrew M. Kuchling8cfa9052001-07-19 01:19:59 +00001339
Andrew M. Kuchlingd6e40e22001-09-10 16:18:50 +00001340 \item Large file support is now enabled on Windows. (Contributed by
1341 Tim Peters.)
1342
Andrew M. Kuchling2cd712b2001-07-16 13:39:08 +00001343 \item The \file{Tools/scripts/ftpmirror.py} script
1344 now parses a \file{.netrc} file, if you have one.
Andrew M. Kuchling4cf52a92001-07-17 12:48:48 +00001345 (Contributed by Mike Romberg.)
Andrew M. Kuchling2cd712b2001-07-16 13:39:08 +00001346
Andrew M. Kuchling4cf52a92001-07-17 12:48:48 +00001347 \item Some features of the object returned by the
1348 \function{xrange()} function are now deprecated, and trigger
1349 warnings when they're accessed; they'll disappear in Python 2.3.
1350 \class{xrange} objects tried to pretend they were full sequence
1351 types by supporting slicing, sequence multiplication, and the
1352 \keyword{in} operator, but these features were rarely used and
1353 therefore buggy. The \method{tolist()} method and the
1354 \member{start}, \member{stop}, and \member{step} attributes are also
1355 being deprecated. At the C level, the fourth argument to the
1356 \cfunction{PyRange_New()} function, \samp{repeat}, has also been
1357 deprecated.
1358
Andrew M. Kuchling8cfa9052001-07-19 01:19:59 +00001359 \item There were a bunch of patches to the dictionary
1360 implementation, mostly to fix potential core dumps if a dictionary
1361 contains objects that sneakily changed their hash value, or mutated
1362 the dictionary they were contained in. For a while python-dev fell
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +00001363 into a gentle rhythm of Michael Hudson finding a case that dumped
1364 core, Tim Peters fixing the bug, Michael finding another case, and round
Andrew M. Kuchling8cfa9052001-07-19 01:19:59 +00001365 and round it went.
1366
Andrew M. Kuchling33a3b632001-09-04 21:25:58 +00001367 \item On Windows, Python can now be compiled with Borland C thanks
1368 to a number of patches contributed by Stephen Hansen, though the
1369 result isn't fully functional yet. (But this \emph{is} progress...)
Andrew M. Kuchling8c69c912001-08-07 14:28:58 +00001370
Andrew M. Kuchlingf4ccf582001-07-31 01:11:36 +00001371 \item Another Windows enhancement: Wise Solutions generously offered
1372 PythonLabs use of their InstallerMaster 8.1 system. Earlier
1373 PythonLabs Windows installers used Wise 5.0a, which was beginning to
1374 show its age. (Packaged up by Tim Peters.)
1375
Andrew M. Kuchling8c69c912001-08-07 14:28:58 +00001376 \item Files ending in \samp{.pyw} can now be imported on Windows.
1377 \samp{.pyw} is a Windows-only thing, used to indicate that a script
1378 needs to be run using PYTHONW.EXE instead of PYTHON.EXE in order to
1379 prevent a DOS console from popping up to display the output. This
1380 patch makes it possible to import such scripts, in case they're also
1381 usable as modules. (Implemented by David Bolen.)
1382
Andrew M. Kuchling8cfa9052001-07-19 01:19:59 +00001383 \item On platforms where Python uses the C \cfunction{dlopen()} function
1384 to load extension modules, it's now possible to set the flags used
1385 by \cfunction{dlopen()} using the \function{sys.getdlopenflags()} and
1386 \function{sys.setdlopenflags()} functions. (Contributed by Bram Stolk.)
Andrew M. Kuchling2f0047a2001-09-05 14:53:31 +00001387
Andrew M. Kuchling26c39bf2001-09-10 03:20:53 +00001388 \item The \function{pow()} built-in function no longer supports 3
1389 arguments when floating-point numbers are supplied.
Andrew M. Kuchling1497b622001-09-24 14:51:16 +00001390 \code{pow(\var{x}, \var{y}, \var{z})} returns \code{(x**y) \% z}, but
Andrew M. Kuchling26c39bf2001-09-10 03:20:53 +00001391 this is never useful for floating point numbers, and the final
1392 result varies unpredictably depending on the platform. A call such
Andrew M. Kuchlingd6e40e22001-09-10 16:18:50 +00001393 as \code{pow(2.0, 8.0, 7.0)} will now raise a \exception{TypeError}
Andrew M. Kuchling26c39bf2001-09-10 03:20:53 +00001394 exception.
Andrew M. Kuchling77707672001-07-31 15:51:16 +00001395
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +00001396\end{itemize}
1397
1398
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +00001399%======================================================================
1400\section{Acknowledgements}
1401
1402The author would like to thank the following people for offering
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +00001403suggestions, corrections and assistance with various drafts of this
1404article: Fred Bremmer, Keith Briggs, Andrew Dalke, Fred~L. Drake, Jr.,
Andrew M. Kuchlingccf04652001-11-26 18:15:44 +00001405Carel Fellinger, David Goodger, Mark Hammond, Stephen Hansen, Michael
1406Hudson, Jack Jansen, Marc-Andr\'e Lemburg, Martin von L\"owis, Fredrik
1407Lundh, Michael McLay, Nick Mathewson, Paul Moore, Gustavo Niemeyer,
1408Don O'Donnell, Tim Peters, Jens Quade, Tom Reinhardt, Neil
1409Schemenauer, Guido van Rossum, Greg Ward.
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +00001410
1411\end{document}