blob: 1636c34f5ec30f61767e9d2d2907896f1ff9513b [file] [log] [blame]
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +00001\documentclass{howto}
2
3% $Id$
4
5\title{What's New in Python 2.2}
Andrew M. Kuchlingbbde5882001-10-31 13:13:36 +00006\release{0.09}
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +00007\author{A.M. Kuchling}
Andrew M. Kuchling7bf82772001-07-11 18:54:26 +00008\authoraddress{\email{akuchlin@mems-exchange.org}}
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +00009\begin{document}
10\maketitle\tableofcontents
11
12\section{Introduction}
13
14{\large This document is a draft, and is subject to change until the
Andrew M. Kuchling9e9c1352001-08-11 03:06:50 +000015final version of Python 2.2 is released. Currently it's up to date
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +000016for Python 2.2 beta 1. Please send any comments, bug reports, or
Andrew M. Kuchling9e9c1352001-08-11 03:06:50 +000017questions, no matter how minor, to \email{akuchlin@mems-exchange.org}.
18}
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +000019
Andrew M. Kuchling26c39bf2001-09-10 03:20:53 +000020This article explains the new features in Python 2.2.
21
22Python 2.2 can be thought of as the "cleanup release". There are some
23features such as generators and iterators that are completely new, but
24most of the changes, significant and far-reaching though they may be,
25are aimed at cleaning up irregularities and dark corners of the
26language design.
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +000027
Andrew M. Kuchling1497b622001-09-24 14:51:16 +000028This article doesn't attempt to provide a complete specification of
Andrew M. Kuchling26c39bf2001-09-10 03:20:53 +000029the new features, but instead provides a convenient overview. For
30full details, you should refer to the documentation for Python 2.2,
Fred Drake0d002542001-07-17 13:55:33 +000031such as the
32\citetitle[http://python.sourceforge.net/devel-docs/lib/lib.html]{Python
33Library Reference} and the
34\citetitle[http://python.sourceforge.net/devel-docs/ref/ref.html]{Python
Andrew M. Kuchling26c39bf2001-09-10 03:20:53 +000035Reference Manual}.
36% XXX These \citetitle marks should get the python.org URLs for the final
Fred Drake0d002542001-07-17 13:55:33 +000037% release, just as soon as the docs are published there.
Andrew M. Kuchling26c39bf2001-09-10 03:20:53 +000038If you want to understand the complete implementation and design
39rationale for a change, refer to the PEP for a particular new feature.
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +000040
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +000041The final release of Python 2.2 is planned for December 2001.
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +000042
Andrew M. Kuchling1497b622001-09-24 14:51:16 +000043\begin{seealso}
44
Andrew M. Kuchling2dab9c72001-10-31 13:16:10 +000045\seeurl{http://www.unixreview.com/documents/s=1356/urm0109h/0109h.htm}
Andrew M. Kuchling1497b622001-09-24 14:51:16 +000046{``What's So Special About Python 2.2?'' is also about the new 2.2
47features, and was written by Cameron Laird and Kathryn Soraiz.}
48
49\end{seealso}
50
Andrew M. Kuchling8cfa9052001-07-19 01:19:59 +000051
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +000052%======================================================================
Andrew M. Kuchling433b5c42001-10-30 21:36:04 +000053\section{PEPs 252 and 253: Type and Class Changes}
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +000054
Andrew M. Kuchling279e7442001-10-22 02:03:40 +000055The largest and most far-reaching changes in Python 2.2 are to
56Python's model of objects and classes. The changes should be backward
57compatible, so it's likely that your code will continue to run
58unchanged, but the changes provide some amazing new capabilities.
59Before beginning this, the longest and most complicated section of
60this article, I'll provide an overview of the changes and offer some
61comments.
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +000062
Andrew M. Kuchling279e7442001-10-22 02:03:40 +000063A long time ago I wrote a Web page
64(\url{http://www.amk.ca/python/writing/warts.html}) listing flaws in
65Python's design. One of the most significant flaws was that it's
66impossible to subclass Python types implemented in C. In particular,
67it's not possible to subclass built-in types, so you can't just
68subclass, say, lists in order to add a single useful method to them.
69The \module{UserList} module provides a class that supports all of the
70methods of lists and that can be subclassed further, but there's lots
71of C code that expects a regular Python list and won't accept a
72\class{UserList} instance.
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +000073
Andrew M. Kuchling279e7442001-10-22 02:03:40 +000074Python 2.2 fixes this, and in the process adds some exciting new
75capabilities. A brief summary:
Andrew M. Kuchling26c39bf2001-09-10 03:20:53 +000076
Andrew M. Kuchling279e7442001-10-22 02:03:40 +000077\begin{itemize}
Andrew M. Kuchlingd6e40e22001-09-10 16:18:50 +000078
Andrew M. Kuchling279e7442001-10-22 02:03:40 +000079\item You can subclass built-in types such as lists and even integers,
80and your subclasses should work in every place that requires the
81original type.
82
83\item It's now possible to define static and class methods, in addition
84to the instance methods available in previous versions of Python.
85
86\item It's also possible to automatically call methods on accessing or
87setting an instance attribute by using a new mechanism called
88\dfn{properties}. Many uses of \method{__getattr__} can be rewritten
89to use properties instead, making the resulting code simpler and
90faster. As a small side benefit, attributes can now have docstrings,
91too.
92
93\item The list of legal attributes for an instance can be limited to a
94particular set using \dfn{slots}, making it possible to safeguard
95against typos and perhaps make more optimizations possible in future
96versions of Python.
97
98\end{itemize}
99
100Some users have voiced concern about all these changes. Sure, they
101say, the new features are neat and lend themselves to all sorts of
102tricks that weren't possible in previous versions of Python, but
103they also make the language more complicated. Some people have said
104that they've always recommended Python for its simplicity, and feel
105that its simplicity is being lost.
106
107Personally, I think there's no need to worry. Many of the new
108features are quite esoteric, and you can write a lot of Python code
109without ever needed to be aware of them. Writing a simple class is no
110more difficult than it ever was, so you don't need to bother learning
111or teaching them unless they're actually needed. Some very
112complicated tasks that were previously only possible from C will now
113be possible in pure Python, and to my mind that's all for the better.
114
115I'm not going to attempt to cover every single corner case and small
116change that were required to make the new features work. Instead this
Andrew M. Kuchlingcf31d5d2001-10-26 20:37:55 +0000117section will paint only the broad strokes. See section~\ref{sect-rellinks},
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000118``Related Links'', for further sources of information about Python 2.2's new
119object model.
120
121
122\subsection{Old and New Classes}
123
124First, you should know that Python 2.2 really has two kinds of
125classes: classic or old-style classes, and new-style classes. The
126old-style class model is exactly the same as the class model in
127earlier versions of Python. All the new features described in this
128section apply only to new-style classes. This divergence isn't
129intended to last forever; eventually old-style classes will be
130dropped, possibly in Python 3.0.
131
Andrew M. Kuchling4855b022001-10-23 20:26:16 +0000132So how do you define a new-style class? You do it by subclassing an
133existing new-style class. Most of Python's built-in types, such as
134integers, lists, dictionaries, and even files, are new-style classes
135now. A new-style class named \class{object}, the base class for all
136built-in types, has been also been added so if no built-in type is
137suitable, you can just subclass \class{object}:
138
139\begin{verbatim}
140class C(object):
141 def __init__ (self):
142 ...
143 ...
144\end{verbatim}
145
146This means that \keyword{class} statements that don't have any base
Andrew M. Kuchlingcf31d5d2001-10-26 20:37:55 +0000147classes are always classic classes in Python 2.2. (Actually you can
148also change this by setting a module-level variable named
149\member{__metaclass__} --- see \pep{253} for the details --- but it's
150easier to just subclass \keyword{object}.)
Andrew M. Kuchling4855b022001-10-23 20:26:16 +0000151
152The type objects for the built-in types are available as built-ins,
153named using a clever trick. Python has always had built-in functions
154named \function{int()}, \function{float()}, and \function{str()}. In
1552.2, they aren't functions any more, but type objects that behave as
156factories when called.
157
158\begin{verbatim}
159>>> int
160<type 'int'>
161>>> int('123')
162123
163\end{verbatim}
164
165To make the set of types complete, new type objects such as
Andrew M. Kuchling1117d932001-10-29 20:37:47 +0000166\function{dict} and \function{file} have been added. Here's a
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000167more interesting example, adding a \method{lock()} method to file
168objects:
Andrew M. Kuchling4855b022001-10-23 20:26:16 +0000169
170\begin{verbatim}
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000171class LockableFile(file):
172 def lock (self, operation, length=0, start=0, whence=0):
173 import fcntl
174 return fcntl.lockf(self.fileno(), operation,
175 length, start, whence)
Andrew M. Kuchling4855b022001-10-23 20:26:16 +0000176\end{verbatim}
177
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000178The now-obsolete \module{posixfile} module contained a class that
179emulated all of a file object's methods and also added a
180\method{lock()} method, but this class couldn't be passed to internal
181functions that expected a built-in file, something which is possible
182with our new \class{LockableFile}.
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000183
184
185\subsection{Descriptors}
186
187In previous versions of Python, there was no consistent way to
188discover what attributes and methods were supported by an object.
189There were some informal conventions, such as defining
190\member{__members__} and \member{__methods__} attributes that were
191lists of names, but often the author of an extension type or a class
192wouldn't bother to define them. You could fall back on inspecting the
193\member{__dict__} of an object, but when class inheritance or an
194arbitrary \method{__getattr__} hook were in use this could still be
195inaccurate.
196
197The one big idea underlying the new class model is that an API for
198describing the attributes of an object using \dfn{descriptors} has
199been formalized. Descriptors specify the value of an attribute,
200stating whether it's a method or a field. With the descriptor API,
201static methods and class methods become possible, as well as more
202exotic constructs.
203
204Attribute descriptors are objects that live inside class objects, and
205have a few attributes of their own:
206
207\begin{itemize}
208
209\item \member{__name__} is the attribute's name.
210
211\item \member{__doc__} is the attribute's docstring.
212
213\item \method{__get__(\var{object})} is a method that retrieves the attribute value from \var{object}.
214
Andrew M. Kuchling4f9e2202001-10-29 18:09:42 +0000215\item \method{__set__(\var{object}, \var{value})} sets the attribute
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000216on \var{object} to \var{value}.
217
218\end{itemize}
219
220For example, when you write \code{obj.x}, the steps that Python
221actually performs are:
222
223\begin{verbatim}
224descriptor = obj.__class__.x
Andrew M. Kuchling7cc13de2001-10-30 14:22:11 +0000225descriptor.__get__(obj)
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000226\end{verbatim}
227
Andrew M. Kuchling7cc13de2001-10-30 14:22:11 +0000228For methods, \method{descriptor.__get__} returns a temporary object that's
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000229callable, and wraps up the instance and the method to be called on it.
230This is also why static methods and class methods are now possible;
231they have descriptors that wrap up just the method, or the method and
232the class. As a brief explanation of these new kinds of methods,
233static methods aren't passed the instance, and therefore resemble
234regular functions. Class methods are passed the class of the object,
Andrew M. Kuchling72a7fb72001-10-30 22:18:21 +0000235but not the object itself. Static and class methods are defined like
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000236this:
237
238\begin{verbatim}
239class C:
240 def f(arg1, arg2):
241 ...
242 f = staticmethod(f)
243
244 def g(cls, arg1, arg2):
245 ...
246 g = classmethod(g)
247\end{verbatim}
248
249The \function{staticmethod()} function takes the function
250\function{f}, and returns it wrapped up in a descriptor so it can be
251stored in the class object. You might expect there to be special
252syntax for creating such methods (\code{def static f()},
253\code{defstatic f()}, or something like that) but no such syntax has
254been defined yet; that's been left for future versions.
255
256More new features, such as slots and properties, are also implemented
257as new kinds of descriptors, and it's not difficult to write a
258descriptor class that does something novel. For example, it would be
259possible to write a descriptor class that made it possible to write
260Eiffel-style preconditions and postconditions for a method. A class
261that used this feature might be defined like this:
262
263\begin{verbatim}
264from eiffel import eiffelmethod
265
266class C:
267 def f(self, arg1, arg2):
268 # The actual function
269 def pre_f(self):
270 # Check preconditions
271 def post_f(self):
272 # Check postconditions
273
274 f = eiffelmethod(f, pre_f, post_f)
275\end{verbatim}
276
277Note that a person using the new \function{eiffelmethod()} doesn't
278have to understand anything about descriptors. This is why I think
279the new features don't increase the basic complexity of the language.
280There will be a few wizards who need to know about it in order to
281write \function{eiffelmethod()} or the ZODB or whatever, but most
282users will just write code on top of the resulting libraries and
283ignore the implementation details.
284
Andrew M. Kuchling4855b022001-10-23 20:26:16 +0000285\subsection{Multiple Inheritance: The Diamond Rule}
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000286
Andrew M. Kuchling4855b022001-10-23 20:26:16 +0000287Multiple inheritance has also been made more useful through changing
288the rules under which names are resolved. Consider this set of classes
289(diagram taken from \pep{253} by Guido van Rossum):
290
291\begin{verbatim}
292 class A:
293 ^ ^ def save(self): ...
294 / \
295 / \
296 / \
297 / \
298 class B class C:
299 ^ ^ def save(self): ...
300 \ /
301 \ /
302 \ /
303 \ /
304 class D
305\end{verbatim}
306
307The lookup rule for classic classes is simple but not very smart; the
308base classes are searched depth-first, going from left to right. A
309reference to \method{D.save} will search the classes \class{D},
310\class{B}, and then \class{A}, where \method{save()} would be found
311and returned. \method{C.save()} would never be found at all. This is
312bad, because if \class{C}'s \method{save()} method is saving some
313internal state specific to \class{C}, not calling it will result in
314that state never getting saved.
315
316New-style classes follow a different algorithm that's a bit more
317complicated to explain, but does the right thing in this situation.
318
319\begin{enumerate}
320
321\item List all the base classes, following the classic lookup rule and
322include a class multiple times if it's visited repeatedly. In the
323above example, the list of visited classes is [\class{D}, \class{B},
Andrew M. Kuchling28369072001-10-29 15:47:33 +0000324\class{A}, \class{C}, \class{A}].
Andrew M. Kuchling4855b022001-10-23 20:26:16 +0000325
326\item Scan the list for duplicated classes. If any are found, remove
327all but one occurrence, leaving the \emph{last} one in the list. In
328the above example, the list becomes [\class{D}, \class{B}, \class{C},
Andrew M. Kuchling28369072001-10-29 15:47:33 +0000329\class{A}] after dropping duplicates.
Andrew M. Kuchling4855b022001-10-23 20:26:16 +0000330
331\end{enumerate}
332
333Following this rule, referring to \method{D.save()} will return
334\method{C.save()}, which is the behaviour we're after. This lookup
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000335rule is the same as the one followed by Common Lisp.
Andrew M. Kuchling4855b022001-10-23 20:26:16 +0000336
Andrew M. Kuchling433b5c42001-10-30 21:36:04 +0000337% XXX mention super()
338
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000339
340\subsection{Attribute Access}
341
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000342A fair number of sophisticated Python classes define hooks for
343attribute access using \method{__getattr__}; most commonly this is
344done for convenience, to make code more readable by automatically
345mapping an attribute access such as \code{obj.parent} into a method
346call such as \code{obj.get_parent()}. Python 2.2 adds some new ways
347of controlling attribute access.
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000348
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000349First, \method{__getattr__(\var{attr_name})} is still supported by
350new-style classes, and nothing about it has changed. As before, it
351will be called when an attempt is made to access \code{obj.foo} and no
352attribute named \samp{foo} is found in the instance's dictionary.
353
354New-style classes also support a new method,
355\method{__getattribute__(\var{attr_name})}. The difference between
356the two methods is that \method{__getattribute__} is \emph{always}
357called whenever any attribute is accessed, while the old
358\method{__getattr__} is only called if \samp{foo} isn't found in the
359instance's dictionary.
360
361However, Python 2.2's support for \dfn{properties} will often be a
362simpler way to trap attribute references. Writing a
363\method{__getattr__} method is complicated because to avoid recursion
364you can't use regular attribute accesses inside them, and instead have
365to mess around with the contents of \member{__dict__}.
366\method{__getattr__} methods also end up being called by Python when
367it checks for other methods such as \method{__repr__} or
368\method{__coerce__}, and so have to be written with this in mind.
369Finally, calling a function on every attribute access results in a
370sizable performance loss.
371
372\class{property} is a new built-in type that packages up three
373functions that get, set, or delete an attribute, and a docstring. For
374example, if you want to define a \member{size} attribute that's
375computed, but also settable, you could write:
376
377\begin{verbatim}
378class C:
379 def get_size (self):
380 result = ... computation ...
381 return result
382 def set_size (self, size):
383 ... compute something based on the size
384 and set internal state appropriately ...
385
386 # Define a property. The 'delete this attribute'
387 # method is defined as None, so the attribute
388 # can't be deleted.
389 size = property(get_size, set_size,
390 None,
391 "Storage size of this instance")
392\end{verbatim}
393
394That is certainly clearer and easier to write than a pair of
395\method{__getattr__}/\method{__setattr__} methods that check for the
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000396\member{size} attribute and handle it specially while retrieving all
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000397other attributes from the instance's \member{__dict__}. Accesses to
398\member{size} are also the only ones which have to perform the work of
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000399calling a function, so references to other attributes run at
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000400their usual speed.
401
402Finally, it's possible to constrain the list of attributes that can be
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000403referenced on an object using the new \member{__slots__} class attribute.
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000404Python objects are usually very dynamic; at any time it's possible to
405define a new attribute on an instance by just doing
406\code{obj.new_attr=1}. This is flexible and convenient, but this
407flexibility can also lead to bugs, as when you meant to write
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000408\code{obj.template = 'a'} but made a typo and wrote
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000409\code{obj.templtae} by accident.
410
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000411A new-style class can define a class attribute named \member{__slots__}
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000412to constrain the list of legal attribute names. An example will make
413this clear:
414
415\begin{verbatim}
416>>> class C(object):
417... __slots__ = ['template', 'name']
418...
419>>> obj = C()
420>>> print obj.template
421None
Andrew M. Kuchling28369072001-10-29 15:47:33 +0000422>>> obj.template = 'Test'
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000423>>> print obj.template
424Test
Andrew M. Kuchling28369072001-10-29 15:47:33 +0000425>>> obj.templtae = None
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000426Traceback (most recent call last):
427 File "<stdin>", line 1, in ?
428AttributeError: 'C' object has no attribute 'templtae'
429\end{verbatim}
430
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000431Note how you get an \exception{AttributeError} on the attempt to
432assign to an attribute not listed in \member{__slots__}.
Andrew M. Kuchling4855b022001-10-23 20:26:16 +0000433
434
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000435\subsection{Related Links}
Andrew M. Kuchlingcf31d5d2001-10-26 20:37:55 +0000436\label{sect-rellinks}
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000437
438This section has just been a quick overview of the new features,
439giving enough of an explanation to start you programming, but many
440details have been simplified or ignored. Where should you go to get a
441more complete picture?
442
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000443\url{http://www.python.org/2.2/descrintro.html} is a lengthy tutorial
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000444introduction to the descriptor features, written by Guido van Rossum.
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000445If my description has whetted your appetite, go read this tutorial
446next, because it goes into much more detail about the new features
447while still remaining quite easy to read.
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000448
449Next, there are two relevant PEPs, \pep{252} and \pep{253}. \pep{252}
450is titled "Making Types Look More Like Classes", and covers the
451descriptor API. \pep{253} is titled "Subtyping Built-in Types", and
452describes the changes to type objects that make it possible to subtype
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000453built-in objects. \pep{253} is the more complicated PEP of the two,
454and at a few points the necessary explanations of types and meta-types
455may cause your head to explode. Both PEPs were written and
456implemented by Guido van Rossum, with substantial assistance from the
457rest of the Zope Corp. team.
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000458
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000459Finally, there's the ultimate authority: the source code. Most of the
460machinery for the type handling is in \file{Objects/typeobject.c}, but
461you should only resort to it after all other avenues have been
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000462exhausted, including posting a question to python-list or python-dev.
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +0000463
Andrew M. Kuchling8cfa9052001-07-19 01:19:59 +0000464
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +0000465%======================================================================
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000466\section{PEP 234: Iterators}
467
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000468Another significant addition to 2.2 is an iteration interface at both
469the C and Python levels. Objects can define how they can be looped
470over by callers.
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000471
472In Python versions up to 2.1, the usual way to make \code{for item in
473obj} work is to define a \method{__getitem__()} method that looks
474something like this:
475
476\begin{verbatim}
477 def __getitem__(self, index):
478 return <next item>
479\end{verbatim}
480
481\method{__getitem__()} is more properly used to define an indexing
482operation on an object so that you can write \code{obj[5]} to retrieve
Andrew M. Kuchling8c69c912001-08-07 14:28:58 +0000483the sixth element. It's a bit misleading when you're using this only
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000484to support \keyword{for} loops. Consider some file-like object that
485wants to be looped over; the \var{index} parameter is essentially
486meaningless, as the class probably assumes that a series of
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000487\method{__getitem__()} calls will be made with \var{index}
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000488incrementing by one each time. In other words, the presence of the
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +0000489\method{__getitem__()} method doesn't mean that using \code{file[5]}
490to randomly access the sixth element will work, though it really should.
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000491
492In Python 2.2, iteration can be implemented separately, and
493\method{__getitem__()} methods can be limited to classes that really
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000494do support random access. The basic idea of iterators is
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +0000495simple. A new built-in function, \function{iter(obj)} or
496\code{iter(\var{C}, \var{sentinel})}, is used to get an iterator.
497\function{iter(obj)} returns an iterator for the object \var{obj},
498while \code{iter(\var{C}, \var{sentinel})} returns an iterator that
499will invoke the callable object \var{C} until it returns
500\var{sentinel} to signal that the iterator is done.
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000501
502Python classes can define an \method{__iter__()} method, which should
503create and return a new iterator for the object; if the object is its
504own iterator, this method can just return \code{self}. In particular,
505iterators will usually be their own iterators. Extension types
506implemented in C can implement a \code{tp_iter} function in order to
Andrew M. Kuchling4cf52a92001-07-17 12:48:48 +0000507return an iterator, and extension types that want to behave as
508iterators can define a \code{tp_iternext} function.
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000509
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000510So, after all this, what do iterators actually do? They have one
511required method, \method{next()}, which takes no arguments and returns
512the next value. When there are no more values to be returned, calling
513\method{next()} should raise the \exception{StopIteration} exception.
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000514
515\begin{verbatim}
516>>> L = [1,2,3]
517>>> i = iter(L)
518>>> print i
519<iterator object at 0x8116870>
520>>> i.next()
5211
522>>> i.next()
5232
524>>> i.next()
5253
526>>> i.next()
527Traceback (most recent call last):
528 File "<stdin>", line 1, in ?
529StopIteration
530>>>
531\end{verbatim}
532
533In 2.2, Python's \keyword{for} statement no longer expects a sequence;
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000534it expects something for which \function{iter()} will return an iterator.
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +0000535For backward compatibility and convenience, an iterator is
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000536automatically constructed for sequences that don't implement
537\method{__iter__()} or a \code{tp_iter} slot, so \code{for i in
538[1,2,3]} will still work. Wherever the Python interpreter loops over
539a sequence, it's been changed to use the iterator protocol. This
540means you can do things like this:
541
542\begin{verbatim}
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000543>>> L = [1,2,3]
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000544>>> i = iter(L)
545>>> a,b,c = i
546>>> a,b,c
547(1, 2, 3)
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000548\end{verbatim}
549
Andrew M. Kuchling9e9c1352001-08-11 03:06:50 +0000550Iterator support has been added to some of Python's basic types.
Fred Drake0d002542001-07-17 13:55:33 +0000551Calling \function{iter()} on a dictionary will return an iterator
Andrew M. Kuchling6ea9f0b2001-07-17 14:50:31 +0000552which loops over its keys:
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000553
554\begin{verbatim}
555>>> m = {'Jan': 1, 'Feb': 2, 'Mar': 3, 'Apr': 4, 'May': 5, 'Jun': 6,
556... 'Jul': 7, 'Aug': 8, 'Sep': 9, 'Oct': 10, 'Nov': 11, 'Dec': 12}
557>>> for key in m: print key, m[key]
558...
559Mar 3
560Feb 2
561Aug 8
562Sep 9
563May 5
564Jun 6
565Jul 7
566Jan 1
567Apr 4
568Nov 11
569Dec 12
570Oct 10
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000571\end{verbatim}
572
573That's just the default behaviour. If you want to iterate over keys,
574values, or key/value pairs, you can explicitly call the
575\method{iterkeys()}, \method{itervalues()}, or \method{iteritems()}
Andrew M. Kuchling9e9c1352001-08-11 03:06:50 +0000576methods to get an appropriate iterator. In a minor related change,
577the \keyword{in} operator now works on dictionaries, so
578\code{\var{key} in dict} is now equivalent to
579\code{dict.has_key(\var{key})}.
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000580
Andrew M. Kuchling9e9c1352001-08-11 03:06:50 +0000581Files also provide an iterator, which calls the \method{readline()}
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000582method until there are no more lines in the file. This means you can
583now read each line of a file using code like this:
584
585\begin{verbatim}
586for line in file:
587 # do something for each line
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000588 ...
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000589\end{verbatim}
590
591Note that you can only go forward in an iterator; there's no way to
592get the previous element, reset the iterator, or make a copy of it.
Fred Drake0d002542001-07-17 13:55:33 +0000593An iterator object could provide such additional capabilities, but the
594iterator protocol only requires a \method{next()} method.
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000595
596\begin{seealso}
597
598\seepep{234}{Iterators}{Written by Ka-Ping Yee and GvR; implemented
599by the Python Labs crew, mostly by GvR and Tim Peters.}
600
601\end{seealso}
602
Andrew M. Kuchling8cfa9052001-07-19 01:19:59 +0000603
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000604%======================================================================
605\section{PEP 255: Simple Generators}
606
607Generators are another new feature, one that interacts with the
608introduction of iterators.
609
610You're doubtless familiar with how function calls work in Python or
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +0000611C. When you call a function, it gets a private namespace where its local
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000612variables are created. When the function reaches a \keyword{return}
613statement, the local variables are destroyed and the resulting value
614is returned to the caller. A later call to the same function will get
615a fresh new set of local variables. But, what if the local variables
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000616weren't thrown away on exiting a function? What if you could later
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000617resume the function where it left off? This is what generators
618provide; they can be thought of as resumable functions.
619
620Here's the simplest example of a generator function:
621
622\begin{verbatim}
623def generate_ints(N):
624 for i in range(N):
625 yield i
626\end{verbatim}
627
628A new keyword, \keyword{yield}, was introduced for generators. Any
629function containing a \keyword{yield} statement is a generator
630function; this is detected by Python's bytecode compiler which
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +0000631compiles the function specially as a result. Because a new keyword was
Andrew M. Kuchling4cf52a92001-07-17 12:48:48 +0000632introduced, generators must be explicitly enabled in a module by
633including a \code{from __future__ import generators} statement near
634the top of the module's source code. In Python 2.3 this statement
635will become unnecessary.
636
637When you call a generator function, it doesn't return a single value;
638instead it returns a generator object that supports the iterator
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +0000639protocol. On executing the \keyword{yield} statement, the generator
Andrew M. Kuchling4cf52a92001-07-17 12:48:48 +0000640outputs the value of \code{i}, similar to a \keyword{return}
641statement. The big difference between \keyword{yield} and a
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +0000642\keyword{return} statement is that on reaching a \keyword{yield} the
Andrew M. Kuchling4cf52a92001-07-17 12:48:48 +0000643generator's state of execution is suspended and local variables are
644preserved. On the next call to the generator's \code{.next()} method,
645the function will resume executing immediately after the
646\keyword{yield} statement. (For complicated reasons, the
647\keyword{yield} statement isn't allowed inside the \keyword{try} block
Andrew M. Kuchlingcf31d5d2001-10-26 20:37:55 +0000648of a \code{try...finally} statement; read \pep{255} for a full
Andrew M. Kuchling4cf52a92001-07-17 12:48:48 +0000649explanation of the interaction between \keyword{yield} and
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000650exceptions.)
651
652Here's a sample usage of the \function{generate_ints} generator:
653
654\begin{verbatim}
655>>> gen = generate_ints(3)
656>>> gen
657<generator object at 0x8117f90>
658>>> gen.next()
6590
660>>> gen.next()
6611
662>>> gen.next()
6632
664>>> gen.next()
665Traceback (most recent call last):
666 File "<stdin>", line 1, in ?
667 File "<stdin>", line 2, in generate_ints
668StopIteration
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000669\end{verbatim}
670
671You could equally write \code{for i in generate_ints(5)}, or
672\code{a,b,c = generate_ints(3)}.
673
674Inside a generator function, the \keyword{return} statement can only
Andrew M. Kuchling4cf52a92001-07-17 12:48:48 +0000675be used without a value, and signals the end of the procession of
676values; afterwards the generator cannot return any further values.
677\keyword{return} with a value, such as \code{return 5}, is a syntax
678error inside a generator function. The end of the generator's results
679can also be indicated by raising \exception{StopIteration} manually,
680or by just letting the flow of execution fall off the bottom of the
681function.
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000682
683You could achieve the effect of generators manually by writing your
Andrew M. Kuchling4cf52a92001-07-17 12:48:48 +0000684own class and storing all the local variables of the generator as
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000685instance variables. For example, returning a list of integers could
686be done by setting \code{self.count} to 0, and having the
687\method{next()} method increment \code{self.count} and return it.
Andrew M. Kuchlingc32cc7c2001-07-17 18:25:01 +0000688However, for a moderately complicated generator, writing a
689corresponding class would be much messier.
690\file{Lib/test/test_generators.py} contains a number of more
691interesting examples. The simplest one implements an in-order
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000692traversal of a tree using generators recursively.
693
694\begin{verbatim}
695# A recursive generator that generates Tree leaves in in-order.
696def inorder(t):
697 if t:
698 for x in inorder(t.left):
699 yield x
700 yield t.label
701 for x in inorder(t.right):
702 yield x
703\end{verbatim}
704
705Two other examples in \file{Lib/test/test_generators.py} produce
706solutions for the N-Queens problem (placing $N$ queens on an $NxN$
707chess board so that no queen threatens another) and the Knight's Tour
708(a route that takes a knight to every square of an $NxN$ chessboard
709without visiting any square twice).
710
711The idea of generators comes from other programming languages,
712especially Icon (\url{http://www.cs.arizona.edu/icon/}), where the
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +0000713idea of generators is central. In Icon, every
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000714expression and function call behaves like a generator. One example
715from ``An Overview of the Icon Programming Language'' at
716\url{http://www.cs.arizona.edu/icon/docs/ipd266.htm} gives an idea of
717what this looks like:
718
719\begin{verbatim}
720sentence := "Store it in the neighboring harbor"
721if (i := find("or", sentence)) > 5 then write(i)
722\end{verbatim}
723
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000724In Icon the \function{find()} function returns the indexes at which the
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000725substring ``or'' is found: 3, 23, 33. In the \keyword{if} statement,
726\code{i} is first assigned a value of 3, but 3 is less than 5, so the
727comparison fails, and Icon retries it with the second value of 23. 23
728is greater than 5, so the comparison now succeeds, and the code prints
729the value 23 to the screen.
730
731Python doesn't go nearly as far as Icon in adopting generators as a
732central concept. Generators are considered a new part of the core
733Python language, but learning or using them isn't compulsory; if they
734don't solve any problems that you have, feel free to ignore them.
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +0000735One novel feature of Python's interface as compared to
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000736Icon's is that a generator's state is represented as a concrete object
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000737(the iterator) that can be passed around to other functions or stored
738in a data structure.
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000739
740\begin{seealso}
741
Andrew M. Kuchling4cf52a92001-07-17 12:48:48 +0000742\seepep{255}{Simple Generators}{Written by Neil Schemenauer, Tim
743Peters, Magnus Lie Hetland. Implemented mostly by Neil Schemenauer
744and Tim Peters, with other fixes from the Python Labs crew.}
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000745
746\end{seealso}
747
Andrew M. Kuchling8cfa9052001-07-19 01:19:59 +0000748
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000749%======================================================================
Andrew M. Kuchling2f0047a2001-09-05 14:53:31 +0000750\section{PEP 237: Unifying Long Integers and Integers}
751
Andrew M. Kuchling26c39bf2001-09-10 03:20:53 +0000752In recent versions, the distinction between regular integers, which
753are 32-bit values on most machines, and long integers, which can be of
754arbitrary size, was becoming an annoyance. For example, on platforms
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +0000755that support files larger than \code{2**32} bytes, the
Andrew M. Kuchling26c39bf2001-09-10 03:20:53 +0000756\method{tell()} method of file objects has to return a long integer.
757However, there were various bits of Python that expected plain
758integers and would raise an error if a long integer was provided
Andrew M. Kuchlingd6e40e22001-09-10 16:18:50 +0000759instead. For example, in Python 1.5, only regular integers
Andrew M. Kuchling26c39bf2001-09-10 03:20:53 +0000760could be used as a slice index, and \code{'abc'[1L:]} would raise a
761\exception{TypeError} exception with the message 'slice index must be
762int'.
Andrew M. Kuchling2f0047a2001-09-05 14:53:31 +0000763
Andrew M. Kuchling26c39bf2001-09-10 03:20:53 +0000764Python 2.2 will shift values from short to long integers as required.
765The 'L' suffix is no longer needed to indicate a long integer literal,
766as now the compiler will choose the appropriate type. (Using the 'L'
767suffix will be discouraged in future 2.x versions of Python,
768triggering a warning in Python 2.4, and probably dropped in Python
7693.0.) Many operations that used to raise an \exception{OverflowError}
770will now return a long integer as their result. For example:
771
772\begin{verbatim}
773>>> 1234567890123
Andrew M. Kuchlingd6e40e22001-09-10 16:18:50 +00007741234567890123L
775>>> 2 ** 64
77618446744073709551616L
Andrew M. Kuchling26c39bf2001-09-10 03:20:53 +0000777\end{verbatim}
778
779In most cases, integers and long integers will now be treated
780identically. You can still distinguish them with the
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000781\function{type()} built-in function, but that's rarely needed.
Andrew M. Kuchling26c39bf2001-09-10 03:20:53 +0000782
Andrew M. Kuchling26c39bf2001-09-10 03:20:53 +0000783\begin{seealso}
784
785\seepep{237}{Unifying Long Integers and Integers}{Written by
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000786Moshe Zadka and Guido van Rossum. Implemented mostly by Guido van
787Rossum.}
Andrew M. Kuchling26c39bf2001-09-10 03:20:53 +0000788
789\end{seealso}
Andrew M. Kuchling2f0047a2001-09-05 14:53:31 +0000790
Andrew M. Kuchlingd4707e32001-09-28 20:46:46 +0000791
Andrew M. Kuchling2f0047a2001-09-05 14:53:31 +0000792%======================================================================
Andrew M. Kuchling9e9c1352001-08-11 03:06:50 +0000793\section{PEP 238: Changing the Division Operator}
794
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000795The most controversial change in Python 2.2 heralds the start of an effort
Andrew M. Kuchling9e9c1352001-08-11 03:06:50 +0000796to fix an old design flaw that's been in Python from the beginning.
797Currently Python's division operator, \code{/}, behaves like C's
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +0000798division operator when presented with two integer arguments: it
Andrew M. Kuchling9e9c1352001-08-11 03:06:50 +0000799returns an integer result that's truncated down when there would be
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +0000800a fractional part. For example, \code{3/2} is 1, not 1.5, and
Andrew M. Kuchling9e9c1352001-08-11 03:06:50 +0000801\code{(-1)/2} is -1, not -0.5. This means that the results of divison
802can vary unexpectedly depending on the type of the two operands and
803because Python is dynamically typed, it can be difficult to determine
804the possible types of the operands.
805
806(The controversy is over whether this is \emph{really} a design flaw,
807and whether it's worth breaking existing code to fix this. It's
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000808caused endless discussions on python-dev, and in July 2001 erupted into an
Andrew M. Kuchling9e9c1352001-08-11 03:06:50 +0000809storm of acidly sarcastic postings on \newsgroup{comp.lang.python}. I
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +0000810won't argue for either side here and will stick to describing what's
Andrew M. Kuchlingcf31d5d2001-10-26 20:37:55 +0000811implemented in 2.2. Read \pep{238} for a summary of arguments and
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +0000812counter-arguments.)
Andrew M. Kuchling9e9c1352001-08-11 03:06:50 +0000813
814Because this change might break code, it's being introduced very
815gradually. Python 2.2 begins the transition, but the switch won't be
816complete until Python 3.0.
817
Andrew M. Kuchlingcf31d5d2001-10-26 20:37:55 +0000818First, I'll borrow some terminology from \pep{238}. ``True division'' is the
Andrew M. Kuchling9e9c1352001-08-11 03:06:50 +0000819division that most non-programmers are familiar with: 3/2 is 1.5, 1/4
820is 0.25, and so forth. ``Floor division'' is what Python's \code{/}
821operator currently does when given integer operands; the result is the
822floor of the value returned by true division. ``Classic division'' is
823the current mixed behaviour of \code{/}; it returns the result of
824floor division when the operands are integers, and returns the result
825of true division when one of the operands is a floating-point number.
826
827Here are the changes 2.2 introduces:
828
829\begin{itemize}
830
831\item A new operator, \code{//}, is the floor division operator.
832(Yes, we know it looks like \Cpp's comment symbol.) \code{//}
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000833\emph{always} performs floor division no matter what the types of
Andrew M. Kuchling9e9c1352001-08-11 03:06:50 +0000834its operands are, so \code{1 // 2} is 0 and \code{1.0 // 2.0} is also
8350.0.
836
837\code{//} is always available in Python 2.2; you don't need to enable
838it using a \code{__future__} statement.
839
Andrew M. Kuchling4f9e2202001-10-29 18:09:42 +0000840\item By including a \code{from __future__ import division} in a
Andrew M. Kuchling9e9c1352001-08-11 03:06:50 +0000841module, the \code{/} operator will be changed to return the result of
842true division, so \code{1/2} is 0.5. Without the \code{__future__}
843statement, \code{/} still means classic division. The default meaning
844of \code{/} will not change until Python 3.0.
845
846\item Classes can define methods called \method{__truediv__} and
847\method{__floordiv__} to overload the two division operators. At the
848C level, there are also slots in the \code{PyNumberMethods} structure
849so extension types can define the two operators.
850
Andrew M. Kuchlingcf31d5d2001-10-26 20:37:55 +0000851\item Python 2.2 supports some command-line arguments for testing
852whether code will works with the changed division semantics. Running
853python with \programopt{-Q warn} will cause a warning to be issued
854whenever division is applied to two integers. You can use this to
855find code that's affected by the change and fix it. By default,
856Python 2.2 will simply perform classic division without a warning; the
857warning will be turned on by default in Python 2.3.
Andrew M. Kuchling9e9c1352001-08-11 03:06:50 +0000858
859\end{itemize}
860
861\begin{seealso}
862
863\seepep{238}{Changing the Division Operator}{Written by Moshe Zadka and
864Guido van Rossum. Implemented by Guido van Rossum..}
865
866\end{seealso}
867
868
869%======================================================================
Andrew M. Kuchlinga43e7032001-06-27 20:32:12 +0000870\section{Unicode Changes}
871
Andrew M. Kuchling2cd712b2001-07-16 13:39:08 +0000872Python's Unicode support has been enhanced a bit in 2.2. Unicode
Andrew M. Kuchlinga6d2a042001-07-20 18:34:34 +0000873strings are usually stored as UCS-2, as 16-bit unsigned integers.
Andrew M. Kuchlingf5fec3c2001-07-19 01:48:08 +0000874Python 2.2 can also be compiled to use UCS-4, 32-bit unsigned
875integers, as its internal encoding by supplying
Andrew M. Kuchling433b5c42001-10-30 21:36:04 +0000876\longprogramopt{enable-unicode=ucs4} to the configure script.
877(It's also possible to specify
878\longprogramopt{disable-unicode} to completely disable Unicode
879support.)
880
881When built to use UCS-4 (a ``wide Python''), the interpreter can
882natively handle Unicode characters from U+000000 to U+110000, so the
883range of legal values for the \function{unichr()} function is expanded
Andrew M. Kuchlinga6d2a042001-07-20 18:34:34 +0000884accordingly. Using an interpreter compiled to use UCS-2 (a ``narrow
885Python''), values greater than 65535 will still cause
886\function{unichr()} to raise a \exception{ValueError} exception.
Andrew M. Kuchling433b5c42001-10-30 21:36:04 +0000887This is all described in \pep{261}, ``Support for `wide' Unicode
888characters''; consult it for further details.
Andrew M. Kuchlingab010872001-07-19 14:59:53 +0000889
Andrew M. Kuchling433b5c42001-10-30 21:36:04 +0000890Another change is simpler to explain. Since their introduction,
Andrew M. Kuchlingab010872001-07-19 14:59:53 +0000891Unicode strings have supported an \method{encode()} method to convert
892the string to a selected encoding such as UTF-8 or Latin-1. A
893symmetric \method{decode(\optional{\var{encoding}})} method has been
894added to 8-bit strings (though not to Unicode strings) in 2.2.
895\method{decode()} assumes that the string is in the specified encoding
896and decodes it, returning whatever is returned by the codec.
897
898Using this new feature, codecs have been added for tasks not directly
899related to Unicode. For example, codecs have been added for
900uu-encoding, MIME's base64 encoding, and compression with the
901\module{zlib} module:
Andrew M. Kuchling2cd712b2001-07-16 13:39:08 +0000902
903\begin{verbatim}
904>>> s = """Here is a lengthy piece of redundant, overly verbose,
905... and repetitive text.
906... """
907>>> data = s.encode('zlib')
908>>> data
909'x\x9c\r\xc9\xc1\r\x80 \x10\x04\xc0?Ul...'
910>>> data.decode('zlib')
911'Here is a lengthy piece of redundant, overly verbose,\nand repetitive text.\n'
912>>> print s.encode('uu')
913begin 666 <data>
914M2&5R92!I<R!A(&QE;F=T:'D@<&EE8V4@;V8@<F5D=6YD86YT+"!O=F5R;'D@
915>=F5R8F]S92P*86YD(')E<&5T:71I=F4@=&5X="X*
916
917end
918>>> "sheesh".encode('rot-13')
919'furrfu'
920\end{verbatim}
Andrew M. Kuchlinga43e7032001-06-27 20:32:12 +0000921
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +0000922To convert a class instance to Unicode, a \method{__unicode__} method
Andrew M. Kuchlingcf31d5d2001-10-26 20:37:55 +0000923can be defined by a class, analogous to \method{__str__}.
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +0000924
Andrew M. Kuchlingcf31d5d2001-10-26 20:37:55 +0000925\method{encode()}, \method{decode()}, and \method{__unicode__} were
926implemented by Marc-Andr\'e Lemburg. The changes to support using
927UCS-4 internally were implemented by Fredrik Lundh and Martin von
928L\"owis.
Andrew M. Kuchlinga43e7032001-06-27 20:32:12 +0000929
Andrew M. Kuchlingf5fec3c2001-07-19 01:48:08 +0000930\begin{seealso}
931
Andrew M. Kuchling433b5c42001-10-30 21:36:04 +0000932\seepep{261}{Support for `wide' Unicode characters}{Written by
933Paul Prescod.}
Andrew M. Kuchlingf5fec3c2001-07-19 01:48:08 +0000934
935\end{seealso}
Andrew M. Kuchling8cfa9052001-07-19 01:19:59 +0000936
Andrew M. Kuchlingcf31d5d2001-10-26 20:37:55 +0000937
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000938%======================================================================
939\section{PEP 227: Nested Scopes}
940
941In Python 2.1, statically nested scopes were added as an optional
942feature, to be enabled by a \code{from __future__ import
943nested_scopes} directive. In 2.2 nested scopes no longer need to be
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +0000944specially enabled, and are now always present. The rest of this section
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000945is a copy of the description of nested scopes from my ``What's New in
946Python 2.1'' document; if you read it when 2.1 came out, you can skip
947the rest of this section.
948
949The largest change introduced in Python 2.1, and made complete in 2.2,
950is to Python's scoping rules. In Python 2.0, at any given time there
951are at most three namespaces used to look up variable names: local,
952module-level, and the built-in namespace. This often surprised people
953because it didn't match their intuitive expectations. For example, a
954nested recursive function definition doesn't work:
955
956\begin{verbatim}
957def f():
958 ...
959 def g(value):
960 ...
961 return g(value-1) + 1
962 ...
963\end{verbatim}
964
965The function \function{g()} will always raise a \exception{NameError}
966exception, because the binding of the name \samp{g} isn't in either
967its local namespace or in the module-level namespace. This isn't much
968of a problem in practice (how often do you recursively define interior
969functions like this?), but this also made using the \keyword{lambda}
970statement clumsier, and this was a problem in practice. In code which
971uses \keyword{lambda} you can often find local variables being copied
972by passing them as the default values of arguments.
973
974\begin{verbatim}
975def find(self, name):
976 "Return list of any entries equal to 'name'"
977 L = filter(lambda x, name=name: x == name,
978 self.list_attribute)
979 return L
980\end{verbatim}
981
982The readability of Python code written in a strongly functional style
983suffers greatly as a result.
984
985The most significant change to Python 2.2 is that static scoping has
986been added to the language to fix this problem. As a first effect,
987the \code{name=name} default argument is now unnecessary in the above
988example. Put simply, when a given variable name is not assigned a
989value within a function (by an assignment, or the \keyword{def},
990\keyword{class}, or \keyword{import} statements), references to the
991variable will be looked up in the local namespace of the enclosing
992scope. A more detailed explanation of the rules, and a dissection of
993the implementation, can be found in the PEP.
994
995This change may cause some compatibility problems for code where the
996same variable name is used both at the module level and as a local
997variable within a function that contains further function definitions.
998This seems rather unlikely though, since such code would have been
999pretty confusing to read in the first place.
1000
1001One side effect of the change is that the \code{from \var{module}
1002import *} and \keyword{exec} statements have been made illegal inside
1003a function scope under certain conditions. The Python reference
1004manual has said all along that \code{from \var{module} import *} is
1005only legal at the top level of a module, but the CPython interpreter
1006has never enforced this before. As part of the implementation of
1007nested scopes, the compiler which turns Python source into bytecodes
1008has to generate different code to access variables in a containing
1009scope. \code{from \var{module} import *} and \keyword{exec} make it
1010impossible for the compiler to figure this out, because they add names
1011to the local namespace that are unknowable at compile time.
1012Therefore, if a function contains function definitions or
1013\keyword{lambda} expressions with free variables, the compiler will
1014flag this by raising a \exception{SyntaxError} exception.
1015
1016To make the preceding explanation a bit clearer, here's an example:
1017
1018\begin{verbatim}
1019x = 1
1020def f():
1021 # The next line is a syntax error
1022 exec 'x=2'
1023 def g():
1024 return x
1025\end{verbatim}
1026
1027Line 4 containing the \keyword{exec} statement is a syntax error,
1028since \keyword{exec} would define a new local variable named \samp{x}
1029whose value should be accessed by \function{g()}.
1030
1031This shouldn't be much of a limitation, since \keyword{exec} is rarely
1032used in most Python code (and when it is used, it's often a sign of a
1033poor design anyway).
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +00001034
1035\begin{seealso}
1036
1037\seepep{227}{Statically Nested Scopes}{Written and implemented by
1038Jeremy Hylton.}
1039
1040\end{seealso}
1041
Andrew M. Kuchlinga43e7032001-06-27 20:32:12 +00001042
1043%======================================================================
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +00001044\section{New and Improved Modules}
1045
1046\begin{itemize}
1047
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +00001048 \item The \module{xmlrpclib} module was contributed to the standard
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +00001049 library by Fredrik Lundh, provding support for writing XML-RPC
1050 clients. XML-RPC is a simple remote procedure call protocol built on
Andrew M. Kuchling8c69c912001-08-07 14:28:58 +00001051 top of HTTP and XML. For example, the following snippet retrieves a
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +00001052 list of RSS channels from the O'Reilly Network, and then
1053 lists the recent headlines for one channel:
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +00001054
1055\begin{verbatim}
1056import xmlrpclib
1057s = xmlrpclib.Server(
1058 'http://www.oreillynet.com/meerkat/xml-rpc/server.php')
1059channels = s.meerkat.getChannels()
1060# channels is a list of dictionaries, like this:
1061# [{'id': 4, 'title': 'Freshmeat Daily News'}
1062# {'id': 190, 'title': '32Bits Online'},
1063# {'id': 4549, 'title': '3DGamers'}, ... ]
1064
1065# Get the items for one channel
1066items = s.meerkat.getItems( {'channel': 4} )
1067
1068# 'items' is another list of dictionaries, like this:
1069# [{'link': 'http://freshmeat.net/releases/52719/',
1070# 'description': 'A utility which converts HTML to XSL FO.',
1071# 'title': 'html2fo 0.3 (Default)'}, ... ]
1072\end{verbatim}
1073
Andrew M. Kuchlingd4707e32001-09-28 20:46:46 +00001074The \module{SimpleXMLRPCServer} module makes it easy to create
1075straightforward XML-RPC servers. See \url{http://www.xmlrpc.com/} for
1076more information about XML-RPC.
1077
Andrew M. Kuchling433b5c42001-10-30 21:36:04 +00001078 \item The new \module{hmac} module implements the HMAC
Andrew M. Kuchlingd4707e32001-09-28 20:46:46 +00001079 algorithm described by \rfc{2104}.
Andrew M. Kuchling433b5c42001-10-30 21:36:04 +00001080 (Contributed by Gerhard H\"aring.)
1081
1082 \item Several functions that originally returned lengthy tuples now
1083 return pseudo-sequences that still behave like tuples but also have
1084 mnemonic attributes such as member{st_mtime} or \member{tm_year}.
1085 The enhanced functions include \function{stat()},
1086 \function{fstat()}, \function{statvfs()}, and \function{fstatvfs()}
1087 in the \module{os} module, and \function{localtime()},
1088 \function{gmtime()}, and \function{strptime()} in the \module{time}
1089 module.
1090
1091 For example, to obtain a file's size using the old tuples, you'd end
1092 up writing something like \code{file_size =
1093 os.stat(filename)[stat.ST_SIZE]}, but now this can be written more
1094 clearly as \code{file_size = os.stat(filename).st_size}.
1095
1096 The original patch for this feature was contributed by Nick Mathewson.
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +00001097
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +00001098 \item The Python profiler has been extensively reworked and various
1099 errors in its output have been corrected. (Contributed by Fred
1100 Fred~L. Drake, Jr. and Tim Peters.)
1101
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +00001102 \item The \module{socket} module can be compiled to support IPv6;
Andrew M. Kuchlingddeb1352001-07-16 14:35:52 +00001103 specify the \longprogramopt{enable-ipv6} option to Python's configure
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +00001104 script. (Contributed by Jun-ichiro ``itojun'' Hagino.)
1105
1106 \item Two new format characters were added to the \module{struct}
1107 module for 64-bit integers on platforms that support the C
1108 \ctype{long long} type. \samp{q} is for a signed 64-bit integer,
1109 and \samp{Q} is for an unsigned one. The value is returned in
1110 Python's long integer type. (Contributed by Tim Peters.)
1111
1112 \item In the interpreter's interactive mode, there's a new built-in
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +00001113 function \function{help()} that uses the \module{pydoc} module
1114 introduced in Python 2.1 to provide interactive help.
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +00001115 \code{help(\var{object})} displays any available help text about
1116 \var{object}. \code{help()} with no argument puts you in an online
1117 help utility, where you can enter the names of functions, classes,
1118 or modules to read their help text.
1119 (Contributed by Guido van Rossum, using Ka-Ping Yee's \module{pydoc} module.)
1120
1121 \item Various bugfixes and performance improvements have been made
Andrew M. Kuchling4cf52a92001-07-17 12:48:48 +00001122 to the SRE engine underlying the \module{re} module. For example,
Andrew M. Kuchlingbeb38552001-10-22 14:11:06 +00001123 the \function{re.sub()} and \function{re.split()} functions have
1124 been rewritten in C. Another contributed patch speeds up certain
Andrew M. Kuchling433b5c42001-10-30 21:36:04 +00001125 Unicode character ranges by a factor of two, and a new \method{finditer()}
1126 method that returns an iterator over all the non-overlapping matches in
1127 a given string.
1128 (SRE is maintained by
Andrew M. Kuchlingbeb38552001-10-22 14:11:06 +00001129 Fredrik Lundh. The BIGCHARSET patch was contributed by Martin von
1130 L\"owis.)
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +00001131
Andrew M. Kuchling1efd7ad2001-09-14 16:19:27 +00001132 \item The \module{smtplib} module now supports \rfc{2487}, ``Secure
1133 SMTP over TLS'', so it's now possible to encrypt the SMTP traffic
1134 between a Python program and the mail transport agent being handed a
Andrew M. Kuchling433b5c42001-10-30 21:36:04 +00001135 message. \module{smtplib} also supports SMTP authentication.
1136 (Contributed by Gerhard H\"aring.)
Andrew M. Kuchling1efd7ad2001-09-14 16:19:27 +00001137
Andrew M. Kuchlinga6d2a042001-07-20 18:34:34 +00001138 \item The \module{imaplib} module, maintained by Piers Lauder, has
1139 support for several new extensions: the NAMESPACE extension defined
1140 in \rfc{2342}, SORT, GETACL and SETACL. (Contributed by Anthony
1141 Baxter and Michel Pelletier.)
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +00001142
Andrew M. Kuchling433b5c42001-10-30 21:36:04 +00001143 % XXX should the 'email' module get a section of its own?
Andrew M. Kuchlingd4707e32001-09-28 20:46:46 +00001144 \item The \module{rfc822} module's parsing of email addresses is now
1145 compliant with \rfc{2822}, an update to \rfc{822}. (The module's
1146 name is \emph{not} going to be changed to \samp{rfc2822}.) A new
1147 package, \module{email}, has also been added for parsing and
1148 generating e-mail messages. (Contributed by Barry Warsaw, and
1149 arising out of his work on Mailman.)
Andrew M. Kuchling77707672001-07-31 15:51:16 +00001150
Andrew M. Kuchlingbbde5882001-10-31 13:13:36 +00001151 \item The \module{difflib} module now contains a new \class{Differ}
1152 class for producing human-readable lists of changes (a ``delta'')
1153 between two sequences of lines of text. There are also two
1154 generator functions, \function{ndiff()} and \function{restore()},
1155 which respectively return a delta from two sequences, or one of the
1156 original sequences from a delta. (Grunt work contributed by David
1157 Goodger, from ndiff.py code by Tim Peters who then did the
1158 generatorization.)
1159
Andrew M. Kuchling77707672001-07-31 15:51:16 +00001160 \item New constants \constant{ascii_letters},
1161 \constant{ascii_lowercase}, and \constant{ascii_uppercase} were
1162 added to the \module{string} module. There were several modules in
1163 the standard library that used \constant{string.letters} to mean the
1164 ranges A-Za-z, but that assumption is incorrect when locales are in
1165 use, because \constant{string.letters} varies depending on the set
1166 of legal characters defined by the current locale. The buggy
1167 modules have all been fixed to use \constant{ascii_letters} instead.
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +00001168 (Reported by an unknown person; fixed by Fred~L. Drake, Jr.)
Andrew M. Kuchling77707672001-07-31 15:51:16 +00001169
Andrew M. Kuchling8c69c912001-08-07 14:28:58 +00001170 \item The \module{mimetypes} module now makes it easier to use
1171 alternative MIME-type databases by the addition of a
1172 \class{MimeTypes} class, which takes a list of filenames to be
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +00001173 parsed. (Contributed by Fred~L. Drake, Jr.)
Andrew M. Kuchling8c69c912001-08-07 14:28:58 +00001174
Andrew M. Kuchlingd6e40e22001-09-10 16:18:50 +00001175 \item A \class{Timer} class was added to the \module{threading}
1176 module that allows scheduling an activity to happen at some future
1177 time. (Contributed by Itamar Shtull-Trauring.)
Andrew M. Kuchling2f0047a2001-09-05 14:53:31 +00001178
Andrew M. Kuchling77707672001-07-31 15:51:16 +00001179\end{itemize}
1180
1181
1182%======================================================================
1183\section{Interpreter Changes and Fixes}
1184
1185Some of the changes only affect people who deal with the Python
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +00001186interpreter at the C level because they're writing Python extension modules,
Andrew M. Kuchling77707672001-07-31 15:51:16 +00001187embedding the interpreter, or just hacking on the interpreter itself.
1188If you only write Python code, none of the changes described here will
1189affect you very much.
1190
1191\begin{itemize}
1192
1193 \item Profiling and tracing functions can now be implemented in C,
1194 which can operate at much higher speeds than Python-based functions
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +00001195 and should reduce the overhead of profiling and tracing. This
1196 will be of interest to authors of development environments for
Andrew M. Kuchling77707672001-07-31 15:51:16 +00001197 Python. Two new C functions were added to Python's API,
1198 \cfunction{PyEval_SetProfile()} and \cfunction{PyEval_SetTrace()}.
1199 The existing \function{sys.setprofile()} and
1200 \function{sys.settrace()} functions still exist, and have simply
1201 been changed to use the new C-level interface. (Contributed by Fred
1202 L. Drake, Jr.)
1203
1204 \item Another low-level API, primarily of interest to implementors
1205 of Python debuggers and development tools, was added.
1206 \cfunction{PyInterpreterState_Head()} and
1207 \cfunction{PyInterpreterState_Next()} let a caller walk through all
1208 the existing interpreter objects;
1209 \cfunction{PyInterpreterState_ThreadHead()} and
1210 \cfunction{PyThreadState_Next()} allow looping over all the thread
1211 states for a given interpreter. (Contributed by David Beazley.)
1212
1213 \item A new \samp{et} format sequence was added to
1214 \cfunction{PyArg_ParseTuple}; \samp{et} takes both a parameter and
1215 an encoding name, and converts the parameter to the given encoding
1216 if the parameter turns out to be a Unicode string, or leaves it
1217 alone if it's an 8-bit string, assuming it to already be in the
1218 desired encoding. This differs from the \samp{es} format character,
1219 which assumes that 8-bit strings are in Python's default ASCII
1220 encoding and converts them to the specified new encoding.
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +00001221 (Contributed by M.-A. Lemburg, and used for the MBCS support on
1222 Windows described in the following section.)
Andrew M. Kuchlingcf31d5d2001-10-26 20:37:55 +00001223
1224 \item A different argument parsing function,
1225 \cfunction{PyArg_UnpackTuple()}, has been added that's simpler and
1226 presumably faster. Instead of specifying a format string, the
1227 caller simply gives the minimum and maximum number of arguments
1228 expected, and a set of pointers to \code{PyObject*} variables that
1229 will be filled in with argument values.
1230
Andrew M. Kuchling0ab31b82001-08-29 01:16:54 +00001231 \item Two new flags \constant{METH_NOARGS} and \constant{METH_O} are
1232 available in method definition tables to simplify implementation of
1233 methods with no arguments or a single untyped argument. Calling
1234 such methods is more efficient than calling a corresponding method
1235 that uses \constant{METH_VARARGS}.
1236 Also, the old \constant{METH_OLDARGS} style of writing C methods is
1237 now officially deprecated.
1238
1239\item
1240 Two new wrapper functions, \cfunction{PyOS_snprintf()} and
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +00001241 \cfunction{PyOS_vsnprintf()} were added to provide
Andrew M. Kuchling0ab31b82001-08-29 01:16:54 +00001242 cross-platform implementations for the relatively new
1243 \cfunction{snprintf()} and \cfunction{vsnprintf()} C lib APIs. In
1244 contrast to the standard \cfunction{sprintf()} and
1245 \cfunction{vsprintf()} functions, the Python versions check the
1246 bounds of the buffer used to protect against buffer overruns.
1247 (Contributed by M.-A. Lemburg.)
Andrew M. Kuchling77707672001-07-31 15:51:16 +00001248
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +00001249\end{itemize}
1250
1251
1252%======================================================================
1253\section{Other Changes and Fixes}
1254
Andrew M. Kuchling8cfa9052001-07-19 01:19:59 +00001255% XXX update the patch and bug figures as we go
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +00001256As usual there were a bunch of other improvements and bugfixes
1257scattered throughout the source tree. A search through the CVS change
Andrew M. Kuchling32e32322001-10-22 15:32:05 +00001258logs finds there were 312 patches applied, and 391 bugs fixed; both
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +00001259figures are likely to be underestimates. Some of the more notable
1260changes are:
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +00001261
1262\begin{itemize}
1263
Andrew M. Kuchling0e03f582001-08-30 21:30:16 +00001264 \item The code for the MacOS port for Python, maintained by Jack
1265 Jansen, is now kept in the main Python CVS tree, and many changes
Andrew M. Kuchling279e7442001-10-22 02:03:40 +00001266 have been made to support MacOS~X.
Andrew M. Kuchling0e03f582001-08-30 21:30:16 +00001267
1268The most significant change is the ability to build Python as a
1269framework, enabled by supplying the \longprogramopt{enable-framework}
1270option to the configure script when compiling Python. According to
1271Jack Jansen, ``This installs a self-contained Python installation plus
Andrew M. Kuchling279e7442001-10-22 02:03:40 +00001272the OS~X framework "glue" into
Andrew M. Kuchling0e03f582001-08-30 21:30:16 +00001273\file{/Library/Frameworks/Python.framework} (or another location of
1274choice). For now there is little immediate added benefit to this
1275(actually, there is the disadvantage that you have to change your PATH
1276to be able to find Python), but it is the basis for creating a
1277full-blown Python application, porting the MacPython IDE, possibly
1278using Python as a standard OSA scripting language and much more.''
1279
1280Most of the MacPython toolbox modules, which interface to MacOS APIs
Andrew M. Kuchling279e7442001-10-22 02:03:40 +00001281such as windowing, QuickTime, scripting, etc. have been ported to OS~X,
Andrew M. Kuchlingbeb38552001-10-22 14:11:06 +00001282but they've been left commented out in \file{setup.py}. People who want
Andrew M. Kuchling0e03f582001-08-30 21:30:16 +00001283to experiment with these modules can uncomment them manually.
1284
1285% Jack's original comments:
1286%The main change is the possibility to build Python as a
1287%framework. This installs a self-contained Python installation plus the
1288%OSX framework "glue" into /Library/Frameworks/Python.framework (or
1289%another location of choice). For now there is little immedeate added
1290%benefit to this (actually, there is the disadvantage that you have to
1291%change your PATH to be able to find Python), but it is the basis for
1292%creating a fullblown Python application, porting the MacPython IDE,
1293%possibly using Python as a standard OSA scripting language and much
1294%more. You enable this with "configure --enable-framework".
1295
1296%The other change is that most MacPython toolbox modules, which
1297%interface to all the MacOS APIs such as windowing, quicktime,
1298%scripting, etc. have been ported. Again, most of these are not of
1299%immedeate use, as they need a full application to be really useful, so
1300%they have been commented out in setup.py. People wanting to experiment
1301%can uncomment them. Gestalt and Internet Config modules are enabled by
1302%default.
Andrew M. Kuchling0e03f582001-08-30 21:30:16 +00001303
Andrew M. Kuchling2cd712b2001-07-16 13:39:08 +00001304 \item Keyword arguments passed to builtin functions that don't take them
1305 now cause a \exception{TypeError} exception to be raised, with the
1306 message "\var{function} takes no keyword arguments".
1307
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +00001308 \item Weak references, added in Python 2.1 as an extension module,
1309 are now part of the core because they're used in the implementation
1310 of new-style classes. The \exception{ReferenceError} exception has
1311 therefore moved from the \module{weakref} module to become a
1312 built-in exception.
1313
Andrew M. Kuchling94a7eba2001-08-15 15:55:48 +00001314 \item A new script, \file{Tools/scripts/cleanfuture.py} by Tim
1315 Peters, automatically removes obsolete \code{__future__} statements
1316 from Python source code.
Andrew M. Kuchling2cd712b2001-07-16 13:39:08 +00001317
Andrew M. Kuchling4f9e2202001-10-29 18:09:42 +00001318 \item An additional \var{flags} argument has been added to the
1319 built-in function \function{compile()}, so the behaviour of
1320 \code{__future__} statements can now be correctly observed in
1321 simulated shells, such as those presented by IDLE and other
1322 development environments. This is described in \pep{264}.
1323 (Contributed by Michael Hudson.)
1324
Andrew M. Kuchling2cd712b2001-07-16 13:39:08 +00001325 \item The new license introduced with Python 1.6 wasn't
1326 GPL-compatible. This is fixed by some minor textual changes to the
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +00001327 2.2 license, so it's now legal to embed Python inside a GPLed
1328 program again. Note that Python itself is not GPLed, but instead is
1329 under a license that's essentially equivalent to the BSD license,
1330 same as it always was. The license changes were also applied to the
1331 Python 2.0.1 and 2.1.1 releases.
Andrew M. Kuchling2cd712b2001-07-16 13:39:08 +00001332
Andrew M. Kuchlingf4ccf582001-07-31 01:11:36 +00001333 \item When presented with a Unicode filename on Windows, Python will
1334 now convert it to an MBCS encoded string, as used by the Microsoft
1335 file APIs. As MBCS is explicitly used by the file APIs, Python's
1336 choice of ASCII as the default encoding turns out to be an
Andrew M. Kuchling433b5c42001-10-30 21:36:04 +00001337 annoyance. On Unix, the locale's character set is used if
1338 \function{locale.nl_langinfo(CODESET)} is available. (Windows
1339 support was contributed by Mark Hammond with assistance from
1340 Marc-Andr\'e Lemburg. Unix support was added by Martin von L\"owis.)
Andrew M. Kuchling8cfa9052001-07-19 01:19:59 +00001341
Andrew M. Kuchlingd6e40e22001-09-10 16:18:50 +00001342 \item Large file support is now enabled on Windows. (Contributed by
1343 Tim Peters.)
1344
Andrew M. Kuchling2cd712b2001-07-16 13:39:08 +00001345 \item The \file{Tools/scripts/ftpmirror.py} script
1346 now parses a \file{.netrc} file, if you have one.
Andrew M. Kuchling4cf52a92001-07-17 12:48:48 +00001347 (Contributed by Mike Romberg.)
Andrew M. Kuchling2cd712b2001-07-16 13:39:08 +00001348
Andrew M. Kuchling4cf52a92001-07-17 12:48:48 +00001349 \item Some features of the object returned by the
1350 \function{xrange()} function are now deprecated, and trigger
1351 warnings when they're accessed; they'll disappear in Python 2.3.
1352 \class{xrange} objects tried to pretend they were full sequence
1353 types by supporting slicing, sequence multiplication, and the
1354 \keyword{in} operator, but these features were rarely used and
1355 therefore buggy. The \method{tolist()} method and the
1356 \member{start}, \member{stop}, and \member{step} attributes are also
1357 being deprecated. At the C level, the fourth argument to the
1358 \cfunction{PyRange_New()} function, \samp{repeat}, has also been
1359 deprecated.
1360
Andrew M. Kuchling8cfa9052001-07-19 01:19:59 +00001361 \item There were a bunch of patches to the dictionary
1362 implementation, mostly to fix potential core dumps if a dictionary
1363 contains objects that sneakily changed their hash value, or mutated
1364 the dictionary they were contained in. For a while python-dev fell
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +00001365 into a gentle rhythm of Michael Hudson finding a case that dumped
1366 core, Tim Peters fixing the bug, Michael finding another case, and round
Andrew M. Kuchling8cfa9052001-07-19 01:19:59 +00001367 and round it went.
1368
Andrew M. Kuchling33a3b632001-09-04 21:25:58 +00001369 \item On Windows, Python can now be compiled with Borland C thanks
1370 to a number of patches contributed by Stephen Hansen, though the
1371 result isn't fully functional yet. (But this \emph{is} progress...)
Andrew M. Kuchling8c69c912001-08-07 14:28:58 +00001372
Andrew M. Kuchlingf4ccf582001-07-31 01:11:36 +00001373 \item Another Windows enhancement: Wise Solutions generously offered
1374 PythonLabs use of their InstallerMaster 8.1 system. Earlier
1375 PythonLabs Windows installers used Wise 5.0a, which was beginning to
1376 show its age. (Packaged up by Tim Peters.)
1377
Andrew M. Kuchling8c69c912001-08-07 14:28:58 +00001378 \item Files ending in \samp{.pyw} can now be imported on Windows.
1379 \samp{.pyw} is a Windows-only thing, used to indicate that a script
1380 needs to be run using PYTHONW.EXE instead of PYTHON.EXE in order to
1381 prevent a DOS console from popping up to display the output. This
1382 patch makes it possible to import such scripts, in case they're also
1383 usable as modules. (Implemented by David Bolen.)
1384
Andrew M. Kuchling8cfa9052001-07-19 01:19:59 +00001385 \item On platforms where Python uses the C \cfunction{dlopen()} function
1386 to load extension modules, it's now possible to set the flags used
1387 by \cfunction{dlopen()} using the \function{sys.getdlopenflags()} and
1388 \function{sys.setdlopenflags()} functions. (Contributed by Bram Stolk.)
Andrew M. Kuchling2f0047a2001-09-05 14:53:31 +00001389
Andrew M. Kuchling26c39bf2001-09-10 03:20:53 +00001390 \item The \function{pow()} built-in function no longer supports 3
1391 arguments when floating-point numbers are supplied.
Andrew M. Kuchling1497b622001-09-24 14:51:16 +00001392 \code{pow(\var{x}, \var{y}, \var{z})} returns \code{(x**y) \% z}, but
Andrew M. Kuchling26c39bf2001-09-10 03:20:53 +00001393 this is never useful for floating point numbers, and the final
1394 result varies unpredictably depending on the platform. A call such
Andrew M. Kuchlingd6e40e22001-09-10 16:18:50 +00001395 as \code{pow(2.0, 8.0, 7.0)} will now raise a \exception{TypeError}
Andrew M. Kuchling26c39bf2001-09-10 03:20:53 +00001396 exception.
Andrew M. Kuchling77707672001-07-31 15:51:16 +00001397
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +00001398\end{itemize}
1399
1400
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +00001401%======================================================================
1402\section{Acknowledgements}
1403
1404The author would like to thank the following people for offering
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +00001405suggestions, corrections and assistance with various drafts of this
1406article: Fred Bremmer, Keith Briggs, Andrew Dalke, Fred~L. Drake, Jr.,
Andrew M. Kuchlingbbde5882001-10-31 13:13:36 +00001407Carel Fellinger, David Goodger, Mark Hammond, Stephen Hansen, Michael Hudson, Jack
Andrew M. Kuchling433b5c42001-10-30 21:36:04 +00001408Jansen, Marc-Andr\'e Lemburg, Martin von L\"owis, Fredrik Lundh,
1409Michael McLay, Nick Mathewson, Paul Moore, Tim Peters, Jens Quade, Tom
Andrew M. Kuchling72a7fb72001-10-30 22:18:21 +00001410Reinhardt, Neil Schemenauer, Guido van Rossum, Greg Ward.
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +00001411
1412\end{document}