blob: cacb054c0ded1490805ccdb8f62b011962fc24d6 [file] [log] [blame]
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +00001\documentclass{howto}
2
3% $Id$
4
5\title{What's New in Python 2.2}
Andrew M. Kuchlingbec5b362001-12-21 04:39:11 +00006\release{1.00}
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +00007\author{A.M. Kuchling}
Andrew M. Kuchling7bf82772001-07-11 18:54:26 +00008\authoraddress{\email{akuchlin@mems-exchange.org}}
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +00009\begin{document}
10\maketitle\tableofcontents
11
12\section{Introduction}
13
Andrew M. Kuchlingbec5b362001-12-21 04:39:11 +000014This article explains the new features in Python 2.2, released on
15December 21, 2001.
Andrew M. Kuchling26c39bf2001-09-10 03:20:53 +000016
17Python 2.2 can be thought of as the "cleanup release". There are some
18features such as generators and iterators that are completely new, but
19most of the changes, significant and far-reaching though they may be,
20are aimed at cleaning up irregularities and dark corners of the
21language design.
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +000022
Andrew M. Kuchling1497b622001-09-24 14:51:16 +000023This article doesn't attempt to provide a complete specification of
Andrew M. Kuchling26c39bf2001-09-10 03:20:53 +000024the new features, but instead provides a convenient overview. For
25full details, you should refer to the documentation for Python 2.2,
Fred Drake0d002542001-07-17 13:55:33 +000026such as the
Andrew M. Kuchling5e08d102001-12-20 16:33:45 +000027\citetitle[http://www.python.org/doc/2.2/lib/lib.html]{Python
Fred Drake0d002542001-07-17 13:55:33 +000028Library Reference} and the
Andrew M. Kuchling5e08d102001-12-20 16:33:45 +000029\citetitle[http://www.python.org/doc/2.2/ref/ref.html]{Python
30Reference Manual}. If you want to understand the complete
31implementation and design rationale for a change, refer to the PEP for
32a particular new feature.
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +000033
Andrew M. Kuchling1497b622001-09-24 14:51:16 +000034\begin{seealso}
35
Andrew M. Kuchling2dab9c72001-10-31 13:16:10 +000036\seeurl{http://www.unixreview.com/documents/s=1356/urm0109h/0109h.htm}
Andrew M. Kuchling1497b622001-09-24 14:51:16 +000037{``What's So Special About Python 2.2?'' is also about the new 2.2
38features, and was written by Cameron Laird and Kathryn Soraiz.}
39
40\end{seealso}
41
Andrew M. Kuchling8cfa9052001-07-19 01:19:59 +000042
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +000043%======================================================================
Andrew M. Kuchling433b5c42001-10-30 21:36:04 +000044\section{PEPs 252 and 253: Type and Class Changes}
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +000045
Andrew M. Kuchling279e7442001-10-22 02:03:40 +000046The largest and most far-reaching changes in Python 2.2 are to
47Python's model of objects and classes. The changes should be backward
48compatible, so it's likely that your code will continue to run
49unchanged, but the changes provide some amazing new capabilities.
50Before beginning this, the longest and most complicated section of
51this article, I'll provide an overview of the changes and offer some
52comments.
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +000053
Andrew M. Kuchling279e7442001-10-22 02:03:40 +000054A long time ago I wrote a Web page
55(\url{http://www.amk.ca/python/writing/warts.html}) listing flaws in
56Python's design. One of the most significant flaws was that it's
57impossible to subclass Python types implemented in C. In particular,
58it's not possible to subclass built-in types, so you can't just
59subclass, say, lists in order to add a single useful method to them.
60The \module{UserList} module provides a class that supports all of the
61methods of lists and that can be subclassed further, but there's lots
62of C code that expects a regular Python list and won't accept a
63\class{UserList} instance.
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +000064
Andrew M. Kuchling279e7442001-10-22 02:03:40 +000065Python 2.2 fixes this, and in the process adds some exciting new
66capabilities. A brief summary:
Andrew M. Kuchling26c39bf2001-09-10 03:20:53 +000067
Andrew M. Kuchling279e7442001-10-22 02:03:40 +000068\begin{itemize}
Andrew M. Kuchlingd6e40e22001-09-10 16:18:50 +000069
Andrew M. Kuchling279e7442001-10-22 02:03:40 +000070\item You can subclass built-in types such as lists and even integers,
71and your subclasses should work in every place that requires the
72original type.
73
74\item It's now possible to define static and class methods, in addition
75to the instance methods available in previous versions of Python.
76
77\item It's also possible to automatically call methods on accessing or
78setting an instance attribute by using a new mechanism called
79\dfn{properties}. Many uses of \method{__getattr__} can be rewritten
80to use properties instead, making the resulting code simpler and
81faster. As a small side benefit, attributes can now have docstrings,
82too.
83
84\item The list of legal attributes for an instance can be limited to a
85particular set using \dfn{slots}, making it possible to safeguard
86against typos and perhaps make more optimizations possible in future
87versions of Python.
88
89\end{itemize}
90
91Some users have voiced concern about all these changes. Sure, they
92say, the new features are neat and lend themselves to all sorts of
93tricks that weren't possible in previous versions of Python, but
94they also make the language more complicated. Some people have said
95that they've always recommended Python for its simplicity, and feel
96that its simplicity is being lost.
97
98Personally, I think there's no need to worry. Many of the new
99features are quite esoteric, and you can write a lot of Python code
100without ever needed to be aware of them. Writing a simple class is no
101more difficult than it ever was, so you don't need to bother learning
102or teaching them unless they're actually needed. Some very
103complicated tasks that were previously only possible from C will now
104be possible in pure Python, and to my mind that's all for the better.
105
106I'm not going to attempt to cover every single corner case and small
107change that were required to make the new features work. Instead this
Andrew M. Kuchlingcf31d5d2001-10-26 20:37:55 +0000108section will paint only the broad strokes. See section~\ref{sect-rellinks},
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000109``Related Links'', for further sources of information about Python 2.2's new
110object model.
111
112
113\subsection{Old and New Classes}
114
115First, you should know that Python 2.2 really has two kinds of
116classes: classic or old-style classes, and new-style classes. The
117old-style class model is exactly the same as the class model in
118earlier versions of Python. All the new features described in this
119section apply only to new-style classes. This divergence isn't
120intended to last forever; eventually old-style classes will be
121dropped, possibly in Python 3.0.
122
Andrew M. Kuchling4855b022001-10-23 20:26:16 +0000123So how do you define a new-style class? You do it by subclassing an
124existing new-style class. Most of Python's built-in types, such as
125integers, lists, dictionaries, and even files, are new-style classes
126now. A new-style class named \class{object}, the base class for all
127built-in types, has been also been added so if no built-in type is
128suitable, you can just subclass \class{object}:
129
130\begin{verbatim}
131class C(object):
132 def __init__ (self):
133 ...
134 ...
135\end{verbatim}
136
137This means that \keyword{class} statements that don't have any base
Andrew M. Kuchlingcf31d5d2001-10-26 20:37:55 +0000138classes are always classic classes in Python 2.2. (Actually you can
139also change this by setting a module-level variable named
140\member{__metaclass__} --- see \pep{253} for the details --- but it's
141easier to just subclass \keyword{object}.)
Andrew M. Kuchling4855b022001-10-23 20:26:16 +0000142
143The type objects for the built-in types are available as built-ins,
144named using a clever trick. Python has always had built-in functions
145named \function{int()}, \function{float()}, and \function{str()}. In
1462.2, they aren't functions any more, but type objects that behave as
147factories when called.
148
149\begin{verbatim}
150>>> int
151<type 'int'>
152>>> int('123')
153123
154\end{verbatim}
155
156To make the set of types complete, new type objects such as
Andrew M. Kuchling1117d932001-10-29 20:37:47 +0000157\function{dict} and \function{file} have been added. Here's a
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000158more interesting example, adding a \method{lock()} method to file
159objects:
Andrew M. Kuchling4855b022001-10-23 20:26:16 +0000160
161\begin{verbatim}
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000162class LockableFile(file):
163 def lock (self, operation, length=0, start=0, whence=0):
164 import fcntl
165 return fcntl.lockf(self.fileno(), operation,
166 length, start, whence)
Andrew M. Kuchling4855b022001-10-23 20:26:16 +0000167\end{verbatim}
168
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000169The now-obsolete \module{posixfile} module contained a class that
170emulated all of a file object's methods and also added a
171\method{lock()} method, but this class couldn't be passed to internal
172functions that expected a built-in file, something which is possible
173with our new \class{LockableFile}.
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000174
175
176\subsection{Descriptors}
177
178In previous versions of Python, there was no consistent way to
179discover what attributes and methods were supported by an object.
180There were some informal conventions, such as defining
181\member{__members__} and \member{__methods__} attributes that were
182lists of names, but often the author of an extension type or a class
183wouldn't bother to define them. You could fall back on inspecting the
184\member{__dict__} of an object, but when class inheritance or an
185arbitrary \method{__getattr__} hook were in use this could still be
186inaccurate.
187
188The one big idea underlying the new class model is that an API for
189describing the attributes of an object using \dfn{descriptors} has
190been formalized. Descriptors specify the value of an attribute,
191stating whether it's a method or a field. With the descriptor API,
192static methods and class methods become possible, as well as more
193exotic constructs.
194
195Attribute descriptors are objects that live inside class objects, and
196have a few attributes of their own:
197
198\begin{itemize}
199
200\item \member{__name__} is the attribute's name.
201
202\item \member{__doc__} is the attribute's docstring.
203
Andrew M. Kuchling9455df22001-12-03 20:55:37 +0000204\item \method{__get__(\var{object})} is a method that retrieves the
205attribute value from \var{object}.
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000206
Andrew M. Kuchling4f9e2202001-10-29 18:09:42 +0000207\item \method{__set__(\var{object}, \var{value})} sets the attribute
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000208on \var{object} to \var{value}.
209
Andrew M. Kuchlingc54fc312001-12-03 20:58:29 +0000210\item \method{__delete__(\var{object}, \var{value})} deletes the \var{value}
Andrew M. Kuchling9455df22001-12-03 20:55:37 +0000211attribute of \var{object}.
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000212\end{itemize}
213
214For example, when you write \code{obj.x}, the steps that Python
215actually performs are:
216
217\begin{verbatim}
218descriptor = obj.__class__.x
Andrew M. Kuchling7cc13de2001-10-30 14:22:11 +0000219descriptor.__get__(obj)
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000220\end{verbatim}
221
Andrew M. Kuchling7cc13de2001-10-30 14:22:11 +0000222For methods, \method{descriptor.__get__} returns a temporary object that's
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000223callable, and wraps up the instance and the method to be called on it.
224This is also why static methods and class methods are now possible;
225they have descriptors that wrap up just the method, or the method and
226the class. As a brief explanation of these new kinds of methods,
227static methods aren't passed the instance, and therefore resemble
228regular functions. Class methods are passed the class of the object,
Andrew M. Kuchling72a7fb72001-10-30 22:18:21 +0000229but not the object itself. Static and class methods are defined like
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000230this:
231
232\begin{verbatim}
Andrew M. Kuchlingccf04652001-11-26 18:15:44 +0000233class C(object):
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000234 def f(arg1, arg2):
235 ...
236 f = staticmethod(f)
237
238 def g(cls, arg1, arg2):
239 ...
240 g = classmethod(g)
241\end{verbatim}
242
243The \function{staticmethod()} function takes the function
244\function{f}, and returns it wrapped up in a descriptor so it can be
245stored in the class object. You might expect there to be special
246syntax for creating such methods (\code{def static f()},
247\code{defstatic f()}, or something like that) but no such syntax has
Andrew M. Kuchlingbec5b362001-12-21 04:39:11 +0000248been defined yet; that's been left for future versions of Python.
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000249
250More new features, such as slots and properties, are also implemented
251as new kinds of descriptors, and it's not difficult to write a
252descriptor class that does something novel. For example, it would be
253possible to write a descriptor class that made it possible to write
254Eiffel-style preconditions and postconditions for a method. A class
255that used this feature might be defined like this:
256
257\begin{verbatim}
258from eiffel import eiffelmethod
259
Andrew M. Kuchlingccf04652001-11-26 18:15:44 +0000260class C(object):
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000261 def f(self, arg1, arg2):
262 # The actual function
Andrew M. Kuchlingbec5b362001-12-21 04:39:11 +0000263 ...
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000264 def pre_f(self):
265 # Check preconditions
Andrew M. Kuchlingbec5b362001-12-21 04:39:11 +0000266 ...
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000267 def post_f(self):
268 # Check postconditions
Andrew M. Kuchlingbec5b362001-12-21 04:39:11 +0000269 ...
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000270
271 f = eiffelmethod(f, pre_f, post_f)
272\end{verbatim}
273
274Note that a person using the new \function{eiffelmethod()} doesn't
275have to understand anything about descriptors. This is why I think
276the new features don't increase the basic complexity of the language.
277There will be a few wizards who need to know about it in order to
278write \function{eiffelmethod()} or the ZODB or whatever, but most
279users will just write code on top of the resulting libraries and
280ignore the implementation details.
281
Andrew M. Kuchlingbec5b362001-12-21 04:39:11 +0000282
Andrew M. Kuchling4855b022001-10-23 20:26:16 +0000283\subsection{Multiple Inheritance: The Diamond Rule}
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000284
Andrew M. Kuchling4855b022001-10-23 20:26:16 +0000285Multiple inheritance has also been made more useful through changing
286the rules under which names are resolved. Consider this set of classes
287(diagram taken from \pep{253} by Guido van Rossum):
288
289\begin{verbatim}
290 class A:
291 ^ ^ def save(self): ...
292 / \
293 / \
294 / \
295 / \
296 class B class C:
297 ^ ^ def save(self): ...
298 \ /
299 \ /
300 \ /
301 \ /
302 class D
303\end{verbatim}
304
305The lookup rule for classic classes is simple but not very smart; the
306base classes are searched depth-first, going from left to right. A
307reference to \method{D.save} will search the classes \class{D},
308\class{B}, and then \class{A}, where \method{save()} would be found
309and returned. \method{C.save()} would never be found at all. This is
310bad, because if \class{C}'s \method{save()} method is saving some
311internal state specific to \class{C}, not calling it will result in
312that state never getting saved.
313
314New-style classes follow a different algorithm that's a bit more
315complicated to explain, but does the right thing in this situation.
316
317\begin{enumerate}
318
319\item List all the base classes, following the classic lookup rule and
320include a class multiple times if it's visited repeatedly. In the
321above example, the list of visited classes is [\class{D}, \class{B},
Andrew M. Kuchling28369072001-10-29 15:47:33 +0000322\class{A}, \class{C}, \class{A}].
Andrew M. Kuchling4855b022001-10-23 20:26:16 +0000323
324\item Scan the list for duplicated classes. If any are found, remove
325all but one occurrence, leaving the \emph{last} one in the list. In
326the above example, the list becomes [\class{D}, \class{B}, \class{C},
Andrew M. Kuchling28369072001-10-29 15:47:33 +0000327\class{A}] after dropping duplicates.
Andrew M. Kuchling4855b022001-10-23 20:26:16 +0000328
329\end{enumerate}
330
331Following this rule, referring to \method{D.save()} will return
332\method{C.save()}, which is the behaviour we're after. This lookup
Andrew M. Kuchlingbec5b362001-12-21 04:39:11 +0000333rule is the same as the one followed by Common Lisp. A new built-in
334function, \function{super()}, provides a way to get at a class's
335superclasses without having to reimplement Python's algorithm.
336The most commonly used form will be
337\function{super(\var{class}, \var{obj})}, which returns
338a bound superclass object (not the actual class object). This form
339will be used in methods to call a method in the superclass; for
340example, \class{D}'s \method{save()} method would look like this:
Andrew M. Kuchling4855b022001-10-23 20:26:16 +0000341
Andrew M. Kuchlingbec5b362001-12-21 04:39:11 +0000342\begin{verbatim}
343class D:
344 def save (self):
345 # Call superclass .save()
346 super(D, self).save()
347 # Save D's private information here
348 ...
349\end{verbatim}
350
351\function{super()} can also return unbound superclass objects
352when called as \function{super(\var{class})} or
353\function{super(\var{class1}, \var{class2})}, but this probably won't
354often be useful.
Andrew M. Kuchling433b5c42001-10-30 21:36:04 +0000355
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000356
357\subsection{Attribute Access}
358
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000359A fair number of sophisticated Python classes define hooks for
360attribute access using \method{__getattr__}; most commonly this is
361done for convenience, to make code more readable by automatically
362mapping an attribute access such as \code{obj.parent} into a method
363call such as \code{obj.get_parent()}. Python 2.2 adds some new ways
364of controlling attribute access.
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000365
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000366First, \method{__getattr__(\var{attr_name})} is still supported by
367new-style classes, and nothing about it has changed. As before, it
368will be called when an attempt is made to access \code{obj.foo} and no
369attribute named \samp{foo} is found in the instance's dictionary.
370
371New-style classes also support a new method,
372\method{__getattribute__(\var{attr_name})}. The difference between
373the two methods is that \method{__getattribute__} is \emph{always}
374called whenever any attribute is accessed, while the old
375\method{__getattr__} is only called if \samp{foo} isn't found in the
376instance's dictionary.
377
378However, Python 2.2's support for \dfn{properties} will often be a
379simpler way to trap attribute references. Writing a
380\method{__getattr__} method is complicated because to avoid recursion
381you can't use regular attribute accesses inside them, and instead have
382to mess around with the contents of \member{__dict__}.
383\method{__getattr__} methods also end up being called by Python when
384it checks for other methods such as \method{__repr__} or
385\method{__coerce__}, and so have to be written with this in mind.
386Finally, calling a function on every attribute access results in a
387sizable performance loss.
388
389\class{property} is a new built-in type that packages up three
390functions that get, set, or delete an attribute, and a docstring. For
391example, if you want to define a \member{size} attribute that's
392computed, but also settable, you could write:
393
394\begin{verbatim}
Andrew M. Kuchlingccf04652001-11-26 18:15:44 +0000395class C(object):
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000396 def get_size (self):
397 result = ... computation ...
398 return result
399 def set_size (self, size):
400 ... compute something based on the size
401 and set internal state appropriately ...
402
403 # Define a property. The 'delete this attribute'
404 # method is defined as None, so the attribute
405 # can't be deleted.
406 size = property(get_size, set_size,
407 None,
408 "Storage size of this instance")
409\end{verbatim}
410
411That is certainly clearer and easier to write than a pair of
412\method{__getattr__}/\method{__setattr__} methods that check for the
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000413\member{size} attribute and handle it specially while retrieving all
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000414other attributes from the instance's \member{__dict__}. Accesses to
415\member{size} are also the only ones which have to perform the work of
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000416calling a function, so references to other attributes run at
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000417their usual speed.
418
419Finally, it's possible to constrain the list of attributes that can be
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000420referenced on an object using the new \member{__slots__} class attribute.
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000421Python objects are usually very dynamic; at any time it's possible to
422define a new attribute on an instance by just doing
423\code{obj.new_attr=1}. This is flexible and convenient, but this
424flexibility can also lead to bugs, as when you meant to write
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000425\code{obj.template = 'a'} but made a typo and wrote
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000426\code{obj.templtae} by accident.
427
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000428A new-style class can define a class attribute named \member{__slots__}
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000429to constrain the list of legal attribute names. An example will make
430this clear:
431
432\begin{verbatim}
433>>> class C(object):
Andrew M. Kuchling038215a2001-12-07 14:22:13 +0000434... __slots__ = ('template', 'name')
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000435...
436>>> obj = C()
437>>> print obj.template
438None
Andrew M. Kuchling28369072001-10-29 15:47:33 +0000439>>> obj.template = 'Test'
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000440>>> print obj.template
441Test
Andrew M. Kuchling28369072001-10-29 15:47:33 +0000442>>> obj.templtae = None
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000443Traceback (most recent call last):
444 File "<stdin>", line 1, in ?
445AttributeError: 'C' object has no attribute 'templtae'
446\end{verbatim}
447
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000448Note how you get an \exception{AttributeError} on the attempt to
449assign to an attribute not listed in \member{__slots__}.
Andrew M. Kuchling4855b022001-10-23 20:26:16 +0000450
451
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000452\subsection{Related Links}
Andrew M. Kuchlingcf31d5d2001-10-26 20:37:55 +0000453\label{sect-rellinks}
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000454
455This section has just been a quick overview of the new features,
456giving enough of an explanation to start you programming, but many
457details have been simplified or ignored. Where should you go to get a
458more complete picture?
459
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000460\url{http://www.python.org/2.2/descrintro.html} is a lengthy tutorial
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000461introduction to the descriptor features, written by Guido van Rossum.
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000462If my description has whetted your appetite, go read this tutorial
463next, because it goes into much more detail about the new features
464while still remaining quite easy to read.
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000465
466Next, there are two relevant PEPs, \pep{252} and \pep{253}. \pep{252}
467is titled "Making Types Look More Like Classes", and covers the
468descriptor API. \pep{253} is titled "Subtyping Built-in Types", and
469describes the changes to type objects that make it possible to subtype
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000470built-in objects. \pep{253} is the more complicated PEP of the two,
471and at a few points the necessary explanations of types and meta-types
472may cause your head to explode. Both PEPs were written and
473implemented by Guido van Rossum, with substantial assistance from the
474rest of the Zope Corp. team.
Andrew M. Kuchling279e7442001-10-22 02:03:40 +0000475
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +0000476Finally, there's the ultimate authority: the source code. Most of the
477machinery for the type handling is in \file{Objects/typeobject.c}, but
478you should only resort to it after all other avenues have been
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000479exhausted, including posting a question to python-list or python-dev.
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +0000480
Andrew M. Kuchling8cfa9052001-07-19 01:19:59 +0000481
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +0000482%======================================================================
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000483\section{PEP 234: Iterators}
484
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000485Another significant addition to 2.2 is an iteration interface at both
486the C and Python levels. Objects can define how they can be looped
487over by callers.
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000488
489In Python versions up to 2.1, the usual way to make \code{for item in
490obj} work is to define a \method{__getitem__()} method that looks
491something like this:
492
493\begin{verbatim}
494 def __getitem__(self, index):
495 return <next item>
496\end{verbatim}
497
498\method{__getitem__()} is more properly used to define an indexing
499operation on an object so that you can write \code{obj[5]} to retrieve
Andrew M. Kuchling8c69c91b2001-08-07 14:28:58 +0000500the sixth element. It's a bit misleading when you're using this only
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000501to support \keyword{for} loops. Consider some file-like object that
502wants to be looped over; the \var{index} parameter is essentially
503meaningless, as the class probably assumes that a series of
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000504\method{__getitem__()} calls will be made with \var{index}
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000505incrementing by one each time. In other words, the presence of the
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +0000506\method{__getitem__()} method doesn't mean that using \code{file[5]}
507to randomly access the sixth element will work, though it really should.
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000508
509In Python 2.2, iteration can be implemented separately, and
510\method{__getitem__()} methods can be limited to classes that really
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000511do support random access. The basic idea of iterators is
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +0000512simple. A new built-in function, \function{iter(obj)} or
513\code{iter(\var{C}, \var{sentinel})}, is used to get an iterator.
514\function{iter(obj)} returns an iterator for the object \var{obj},
515while \code{iter(\var{C}, \var{sentinel})} returns an iterator that
516will invoke the callable object \var{C} until it returns
517\var{sentinel} to signal that the iterator is done.
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000518
519Python classes can define an \method{__iter__()} method, which should
520create and return a new iterator for the object; if the object is its
521own iterator, this method can just return \code{self}. In particular,
522iterators will usually be their own iterators. Extension types
523implemented in C can implement a \code{tp_iter} function in order to
Andrew M. Kuchling4cf52a92001-07-17 12:48:48 +0000524return an iterator, and extension types that want to behave as
525iterators can define a \code{tp_iternext} function.
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000526
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000527So, after all this, what do iterators actually do? They have one
528required method, \method{next()}, which takes no arguments and returns
529the next value. When there are no more values to be returned, calling
530\method{next()} should raise the \exception{StopIteration} exception.
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000531
532\begin{verbatim}
533>>> L = [1,2,3]
534>>> i = iter(L)
535>>> print i
536<iterator object at 0x8116870>
537>>> i.next()
5381
539>>> i.next()
5402
541>>> i.next()
5423
543>>> i.next()
544Traceback (most recent call last):
545 File "<stdin>", line 1, in ?
546StopIteration
547>>>
548\end{verbatim}
549
550In 2.2, Python's \keyword{for} statement no longer expects a sequence;
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000551it expects something for which \function{iter()} will return an iterator.
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +0000552For backward compatibility and convenience, an iterator is
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000553automatically constructed for sequences that don't implement
554\method{__iter__()} or a \code{tp_iter} slot, so \code{for i in
555[1,2,3]} will still work. Wherever the Python interpreter loops over
556a sequence, it's been changed to use the iterator protocol. This
557means you can do things like this:
558
559\begin{verbatim}
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000560>>> L = [1,2,3]
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000561>>> i = iter(L)
562>>> a,b,c = i
563>>> a,b,c
564(1, 2, 3)
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000565\end{verbatim}
566
Andrew M. Kuchling9e9c1352001-08-11 03:06:50 +0000567Iterator support has been added to some of Python's basic types.
Fred Drake0d002542001-07-17 13:55:33 +0000568Calling \function{iter()} on a dictionary will return an iterator
Andrew M. Kuchling6ea9f0b2001-07-17 14:50:31 +0000569which loops over its keys:
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000570
571\begin{verbatim}
572>>> m = {'Jan': 1, 'Feb': 2, 'Mar': 3, 'Apr': 4, 'May': 5, 'Jun': 6,
573... 'Jul': 7, 'Aug': 8, 'Sep': 9, 'Oct': 10, 'Nov': 11, 'Dec': 12}
574>>> for key in m: print key, m[key]
575...
576Mar 3
577Feb 2
578Aug 8
579Sep 9
580May 5
581Jun 6
582Jul 7
583Jan 1
584Apr 4
585Nov 11
586Dec 12
587Oct 10
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000588\end{verbatim}
589
590That's just the default behaviour. If you want to iterate over keys,
591values, or key/value pairs, you can explicitly call the
592\method{iterkeys()}, \method{itervalues()}, or \method{iteritems()}
Andrew M. Kuchling9e9c1352001-08-11 03:06:50 +0000593methods to get an appropriate iterator. In a minor related change,
594the \keyword{in} operator now works on dictionaries, so
595\code{\var{key} in dict} is now equivalent to
596\code{dict.has_key(\var{key})}.
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000597
Andrew M. Kuchling9e9c1352001-08-11 03:06:50 +0000598Files also provide an iterator, which calls the \method{readline()}
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000599method until there are no more lines in the file. This means you can
600now read each line of a file using code like this:
601
602\begin{verbatim}
603for line in file:
604 # do something for each line
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000605 ...
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000606\end{verbatim}
607
608Note that you can only go forward in an iterator; there's no way to
609get the previous element, reset the iterator, or make a copy of it.
Fred Drake0d002542001-07-17 13:55:33 +0000610An iterator object could provide such additional capabilities, but the
611iterator protocol only requires a \method{next()} method.
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000612
613\begin{seealso}
614
615\seepep{234}{Iterators}{Written by Ka-Ping Yee and GvR; implemented
616by the Python Labs crew, mostly by GvR and Tim Peters.}
617
618\end{seealso}
619
Andrew M. Kuchling8cfa9052001-07-19 01:19:59 +0000620
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000621%======================================================================
622\section{PEP 255: Simple Generators}
623
624Generators are another new feature, one that interacts with the
625introduction of iterators.
626
627You're doubtless familiar with how function calls work in Python or
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +0000628C. When you call a function, it gets a private namespace where its local
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000629variables are created. When the function reaches a \keyword{return}
630statement, the local variables are destroyed and the resulting value
631is returned to the caller. A later call to the same function will get
632a fresh new set of local variables. But, what if the local variables
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000633weren't thrown away on exiting a function? What if you could later
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000634resume the function where it left off? This is what generators
635provide; they can be thought of as resumable functions.
636
637Here's the simplest example of a generator function:
638
639\begin{verbatim}
640def generate_ints(N):
641 for i in range(N):
642 yield i
643\end{verbatim}
644
645A new keyword, \keyword{yield}, was introduced for generators. Any
646function containing a \keyword{yield} statement is a generator
647function; this is detected by Python's bytecode compiler which
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +0000648compiles the function specially as a result. Because a new keyword was
Andrew M. Kuchling4cf52a92001-07-17 12:48:48 +0000649introduced, generators must be explicitly enabled in a module by
650including a \code{from __future__ import generators} statement near
651the top of the module's source code. In Python 2.3 this statement
652will become unnecessary.
653
654When you call a generator function, it doesn't return a single value;
655instead it returns a generator object that supports the iterator
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +0000656protocol. On executing the \keyword{yield} statement, the generator
Andrew M. Kuchling4cf52a92001-07-17 12:48:48 +0000657outputs the value of \code{i}, similar to a \keyword{return}
658statement. The big difference between \keyword{yield} and a
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +0000659\keyword{return} statement is that on reaching a \keyword{yield} the
Andrew M. Kuchling4cf52a92001-07-17 12:48:48 +0000660generator's state of execution is suspended and local variables are
661preserved. On the next call to the generator's \code{.next()} method,
662the function will resume executing immediately after the
663\keyword{yield} statement. (For complicated reasons, the
664\keyword{yield} statement isn't allowed inside the \keyword{try} block
Andrew M. Kuchlingcf31d5d2001-10-26 20:37:55 +0000665of a \code{try...finally} statement; read \pep{255} for a full
Andrew M. Kuchling4cf52a92001-07-17 12:48:48 +0000666explanation of the interaction between \keyword{yield} and
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000667exceptions.)
668
669Here's a sample usage of the \function{generate_ints} generator:
670
671\begin{verbatim}
672>>> gen = generate_ints(3)
673>>> gen
674<generator object at 0x8117f90>
675>>> gen.next()
6760
677>>> gen.next()
6781
679>>> gen.next()
6802
681>>> gen.next()
682Traceback (most recent call last):
683 File "<stdin>", line 1, in ?
684 File "<stdin>", line 2, in generate_ints
685StopIteration
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000686\end{verbatim}
687
688You could equally write \code{for i in generate_ints(5)}, or
689\code{a,b,c = generate_ints(3)}.
690
691Inside a generator function, the \keyword{return} statement can only
Andrew M. Kuchling4cf52a92001-07-17 12:48:48 +0000692be used without a value, and signals the end of the procession of
693values; afterwards the generator cannot return any further values.
694\keyword{return} with a value, such as \code{return 5}, is a syntax
695error inside a generator function. The end of the generator's results
696can also be indicated by raising \exception{StopIteration} manually,
697or by just letting the flow of execution fall off the bottom of the
698function.
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000699
700You could achieve the effect of generators manually by writing your
Andrew M. Kuchling4cf52a92001-07-17 12:48:48 +0000701own class and storing all the local variables of the generator as
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000702instance variables. For example, returning a list of integers could
703be done by setting \code{self.count} to 0, and having the
704\method{next()} method increment \code{self.count} and return it.
Andrew M. Kuchlingc32cc7c2001-07-17 18:25:01 +0000705However, for a moderately complicated generator, writing a
706corresponding class would be much messier.
707\file{Lib/test/test_generators.py} contains a number of more
708interesting examples. The simplest one implements an in-order
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000709traversal of a tree using generators recursively.
710
711\begin{verbatim}
712# A recursive generator that generates Tree leaves in in-order.
713def inorder(t):
714 if t:
715 for x in inorder(t.left):
716 yield x
717 yield t.label
718 for x in inorder(t.right):
719 yield x
720\end{verbatim}
721
722Two other examples in \file{Lib/test/test_generators.py} produce
723solutions for the N-Queens problem (placing $N$ queens on an $NxN$
724chess board so that no queen threatens another) and the Knight's Tour
725(a route that takes a knight to every square of an $NxN$ chessboard
726without visiting any square twice).
727
728The idea of generators comes from other programming languages,
729especially Icon (\url{http://www.cs.arizona.edu/icon/}), where the
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +0000730idea of generators is central. In Icon, every
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000731expression and function call behaves like a generator. One example
732from ``An Overview of the Icon Programming Language'' at
733\url{http://www.cs.arizona.edu/icon/docs/ipd266.htm} gives an idea of
734what this looks like:
735
736\begin{verbatim}
737sentence := "Store it in the neighboring harbor"
738if (i := find("or", sentence)) > 5 then write(i)
739\end{verbatim}
740
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000741In Icon the \function{find()} function returns the indexes at which the
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000742substring ``or'' is found: 3, 23, 33. In the \keyword{if} statement,
743\code{i} is first assigned a value of 3, but 3 is less than 5, so the
744comparison fails, and Icon retries it with the second value of 23. 23
745is greater than 5, so the comparison now succeeds, and the code prints
746the value 23 to the screen.
747
748Python doesn't go nearly as far as Icon in adopting generators as a
749central concept. Generators are considered a new part of the core
750Python language, but learning or using them isn't compulsory; if they
751don't solve any problems that you have, feel free to ignore them.
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +0000752One novel feature of Python's interface as compared to
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000753Icon's is that a generator's state is represented as a concrete object
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000754(the iterator) that can be passed around to other functions or stored
755in a data structure.
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000756
757\begin{seealso}
758
Andrew M. Kuchling4cf52a92001-07-17 12:48:48 +0000759\seepep{255}{Simple Generators}{Written by Neil Schemenauer, Tim
760Peters, Magnus Lie Hetland. Implemented mostly by Neil Schemenauer
761and Tim Peters, with other fixes from the Python Labs crew.}
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000762
763\end{seealso}
764
Andrew M. Kuchling8cfa9052001-07-19 01:19:59 +0000765
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000766%======================================================================
Andrew M. Kuchling2f0047a2001-09-05 14:53:31 +0000767\section{PEP 237: Unifying Long Integers and Integers}
768
Andrew M. Kuchling26c39bf2001-09-10 03:20:53 +0000769In recent versions, the distinction between regular integers, which
770are 32-bit values on most machines, and long integers, which can be of
771arbitrary size, was becoming an annoyance. For example, on platforms
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +0000772that support files larger than \code{2**32} bytes, the
Andrew M. Kuchling26c39bf2001-09-10 03:20:53 +0000773\method{tell()} method of file objects has to return a long integer.
774However, there were various bits of Python that expected plain
775integers and would raise an error if a long integer was provided
Andrew M. Kuchlingd6e40e22001-09-10 16:18:50 +0000776instead. For example, in Python 1.5, only regular integers
Andrew M. Kuchling26c39bf2001-09-10 03:20:53 +0000777could be used as a slice index, and \code{'abc'[1L:]} would raise a
778\exception{TypeError} exception with the message 'slice index must be
779int'.
Andrew M. Kuchling2f0047a2001-09-05 14:53:31 +0000780
Andrew M. Kuchling26c39bf2001-09-10 03:20:53 +0000781Python 2.2 will shift values from short to long integers as required.
782The 'L' suffix is no longer needed to indicate a long integer literal,
783as now the compiler will choose the appropriate type. (Using the 'L'
784suffix will be discouraged in future 2.x versions of Python,
785triggering a warning in Python 2.4, and probably dropped in Python
7863.0.) Many operations that used to raise an \exception{OverflowError}
787will now return a long integer as their result. For example:
788
789\begin{verbatim}
790>>> 1234567890123
Andrew M. Kuchlingd6e40e22001-09-10 16:18:50 +00007911234567890123L
792>>> 2 ** 64
79318446744073709551616L
Andrew M. Kuchling26c39bf2001-09-10 03:20:53 +0000794\end{verbatim}
795
796In most cases, integers and long integers will now be treated
797identically. You can still distinguish them with the
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000798\function{type()} built-in function, but that's rarely needed.
Andrew M. Kuchling26c39bf2001-09-10 03:20:53 +0000799
Andrew M. Kuchling26c39bf2001-09-10 03:20:53 +0000800\begin{seealso}
801
802\seepep{237}{Unifying Long Integers and Integers}{Written by
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000803Moshe Zadka and Guido van Rossum. Implemented mostly by Guido van
804Rossum.}
Andrew M. Kuchling26c39bf2001-09-10 03:20:53 +0000805
806\end{seealso}
Andrew M. Kuchling2f0047a2001-09-05 14:53:31 +0000807
Andrew M. Kuchlingd4707e32001-09-28 20:46:46 +0000808
Andrew M. Kuchling2f0047a2001-09-05 14:53:31 +0000809%======================================================================
Andrew M. Kuchling9e9c1352001-08-11 03:06:50 +0000810\section{PEP 238: Changing the Division Operator}
811
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000812The most controversial change in Python 2.2 heralds the start of an effort
Andrew M. Kuchling9e9c1352001-08-11 03:06:50 +0000813to fix an old design flaw that's been in Python from the beginning.
814Currently Python's division operator, \code{/}, behaves like C's
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +0000815division operator when presented with two integer arguments: it
Andrew M. Kuchling9e9c1352001-08-11 03:06:50 +0000816returns an integer result that's truncated down when there would be
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +0000817a fractional part. For example, \code{3/2} is 1, not 1.5, and
Andrew M. Kuchling9e9c1352001-08-11 03:06:50 +0000818\code{(-1)/2} is -1, not -0.5. This means that the results of divison
819can vary unexpectedly depending on the type of the two operands and
820because Python is dynamically typed, it can be difficult to determine
821the possible types of the operands.
822
823(The controversy is over whether this is \emph{really} a design flaw,
824and whether it's worth breaking existing code to fix this. It's
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000825caused endless discussions on python-dev, and in July 2001 erupted into an
Andrew M. Kuchling9e9c1352001-08-11 03:06:50 +0000826storm of acidly sarcastic postings on \newsgroup{comp.lang.python}. I
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +0000827won't argue for either side here and will stick to describing what's
Andrew M. Kuchlingcf31d5d2001-10-26 20:37:55 +0000828implemented in 2.2. Read \pep{238} for a summary of arguments and
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +0000829counter-arguments.)
Andrew M. Kuchling9e9c1352001-08-11 03:06:50 +0000830
831Because this change might break code, it's being introduced very
832gradually. Python 2.2 begins the transition, but the switch won't be
833complete until Python 3.0.
834
Andrew M. Kuchlingcf31d5d2001-10-26 20:37:55 +0000835First, I'll borrow some terminology from \pep{238}. ``True division'' is the
Andrew M. Kuchling9e9c1352001-08-11 03:06:50 +0000836division that most non-programmers are familiar with: 3/2 is 1.5, 1/4
837is 0.25, and so forth. ``Floor division'' is what Python's \code{/}
838operator currently does when given integer operands; the result is the
839floor of the value returned by true division. ``Classic division'' is
840the current mixed behaviour of \code{/}; it returns the result of
841floor division when the operands are integers, and returns the result
842of true division when one of the operands is a floating-point number.
843
844Here are the changes 2.2 introduces:
845
846\begin{itemize}
847
848\item A new operator, \code{//}, is the floor division operator.
849(Yes, we know it looks like \Cpp's comment symbol.) \code{//}
Andrew M. Kuchling7aa63c22001-10-30 14:35:03 +0000850\emph{always} performs floor division no matter what the types of
Andrew M. Kuchling9e9c1352001-08-11 03:06:50 +0000851its operands are, so \code{1 // 2} is 0 and \code{1.0 // 2.0} is also
8520.0.
853
854\code{//} is always available in Python 2.2; you don't need to enable
855it using a \code{__future__} statement.
856
Andrew M. Kuchling4f9e2202001-10-29 18:09:42 +0000857\item By including a \code{from __future__ import division} in a
Andrew M. Kuchling9e9c1352001-08-11 03:06:50 +0000858module, the \code{/} operator will be changed to return the result of
859true division, so \code{1/2} is 0.5. Without the \code{__future__}
860statement, \code{/} still means classic division. The default meaning
861of \code{/} will not change until Python 3.0.
862
863\item Classes can define methods called \method{__truediv__} and
864\method{__floordiv__} to overload the two division operators. At the
865C level, there are also slots in the \code{PyNumberMethods} structure
866so extension types can define the two operators.
867
Andrew M. Kuchlingcf31d5d2001-10-26 20:37:55 +0000868\item Python 2.2 supports some command-line arguments for testing
869whether code will works with the changed division semantics. Running
870python with \programopt{-Q warn} will cause a warning to be issued
871whenever division is applied to two integers. You can use this to
872find code that's affected by the change and fix it. By default,
873Python 2.2 will simply perform classic division without a warning; the
874warning will be turned on by default in Python 2.3.
Andrew M. Kuchling9e9c1352001-08-11 03:06:50 +0000875
876\end{itemize}
877
878\begin{seealso}
879
880\seepep{238}{Changing the Division Operator}{Written by Moshe Zadka and
881Guido van Rossum. Implemented by Guido van Rossum..}
882
883\end{seealso}
884
885
886%======================================================================
Andrew M. Kuchlinga43e7032001-06-27 20:32:12 +0000887\section{Unicode Changes}
888
Andrew M. Kuchling2cd712b2001-07-16 13:39:08 +0000889Python's Unicode support has been enhanced a bit in 2.2. Unicode
Andrew M. Kuchlinga6d2a042001-07-20 18:34:34 +0000890strings are usually stored as UCS-2, as 16-bit unsigned integers.
Andrew M. Kuchlingf5fec3c2001-07-19 01:48:08 +0000891Python 2.2 can also be compiled to use UCS-4, 32-bit unsigned
892integers, as its internal encoding by supplying
Andrew M. Kuchling433b5c42001-10-30 21:36:04 +0000893\longprogramopt{enable-unicode=ucs4} to the configure script.
894(It's also possible to specify
895\longprogramopt{disable-unicode} to completely disable Unicode
896support.)
897
898When built to use UCS-4 (a ``wide Python''), the interpreter can
899natively handle Unicode characters from U+000000 to U+110000, so the
900range of legal values for the \function{unichr()} function is expanded
Andrew M. Kuchlinga6d2a042001-07-20 18:34:34 +0000901accordingly. Using an interpreter compiled to use UCS-2 (a ``narrow
902Python''), values greater than 65535 will still cause
903\function{unichr()} to raise a \exception{ValueError} exception.
Andrew M. Kuchling433b5c42001-10-30 21:36:04 +0000904This is all described in \pep{261}, ``Support for `wide' Unicode
905characters''; consult it for further details.
Andrew M. Kuchlingab010872001-07-19 14:59:53 +0000906
Andrew M. Kuchling433b5c42001-10-30 21:36:04 +0000907Another change is simpler to explain. Since their introduction,
Andrew M. Kuchlingab010872001-07-19 14:59:53 +0000908Unicode strings have supported an \method{encode()} method to convert
909the string to a selected encoding such as UTF-8 or Latin-1. A
910symmetric \method{decode(\optional{\var{encoding}})} method has been
911added to 8-bit strings (though not to Unicode strings) in 2.2.
912\method{decode()} assumes that the string is in the specified encoding
913and decodes it, returning whatever is returned by the codec.
914
915Using this new feature, codecs have been added for tasks not directly
916related to Unicode. For example, codecs have been added for
917uu-encoding, MIME's base64 encoding, and compression with the
918\module{zlib} module:
Andrew M. Kuchling2cd712b2001-07-16 13:39:08 +0000919
920\begin{verbatim}
921>>> s = """Here is a lengthy piece of redundant, overly verbose,
922... and repetitive text.
923... """
924>>> data = s.encode('zlib')
925>>> data
926'x\x9c\r\xc9\xc1\r\x80 \x10\x04\xc0?Ul...'
927>>> data.decode('zlib')
928'Here is a lengthy piece of redundant, overly verbose,\nand repetitive text.\n'
929>>> print s.encode('uu')
930begin 666 <data>
931M2&5R92!I<R!A(&QE;F=T:'D@<&EE8V4@;V8@<F5D=6YD86YT+"!O=F5R;'D@
932>=F5R8F]S92P*86YD(')E<&5T:71I=F4@=&5X="X*
933
934end
935>>> "sheesh".encode('rot-13')
936'furrfu'
937\end{verbatim}
Andrew M. Kuchlinga43e7032001-06-27 20:32:12 +0000938
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +0000939To convert a class instance to Unicode, a \method{__unicode__} method
Andrew M. Kuchlingcf31d5d2001-10-26 20:37:55 +0000940can be defined by a class, analogous to \method{__str__}.
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +0000941
Andrew M. Kuchlingcf31d5d2001-10-26 20:37:55 +0000942\method{encode()}, \method{decode()}, and \method{__unicode__} were
943implemented by Marc-Andr\'e Lemburg. The changes to support using
944UCS-4 internally were implemented by Fredrik Lundh and Martin von
945L\"owis.
Andrew M. Kuchlinga43e7032001-06-27 20:32:12 +0000946
Andrew M. Kuchlingf5fec3c2001-07-19 01:48:08 +0000947\begin{seealso}
948
Andrew M. Kuchling433b5c42001-10-30 21:36:04 +0000949\seepep{261}{Support for `wide' Unicode characters}{Written by
950Paul Prescod.}
Andrew M. Kuchlingf5fec3c2001-07-19 01:48:08 +0000951
952\end{seealso}
Andrew M. Kuchling8cfa9052001-07-19 01:19:59 +0000953
Andrew M. Kuchlingcf31d5d2001-10-26 20:37:55 +0000954
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000955%======================================================================
956\section{PEP 227: Nested Scopes}
957
958In Python 2.1, statically nested scopes were added as an optional
959feature, to be enabled by a \code{from __future__ import
960nested_scopes} directive. In 2.2 nested scopes no longer need to be
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +0000961specially enabled, and are now always present. The rest of this section
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +0000962is a copy of the description of nested scopes from my ``What's New in
963Python 2.1'' document; if you read it when 2.1 came out, you can skip
964the rest of this section.
965
966The largest change introduced in Python 2.1, and made complete in 2.2,
967is to Python's scoping rules. In Python 2.0, at any given time there
968are at most three namespaces used to look up variable names: local,
969module-level, and the built-in namespace. This often surprised people
970because it didn't match their intuitive expectations. For example, a
971nested recursive function definition doesn't work:
972
973\begin{verbatim}
974def f():
975 ...
976 def g(value):
977 ...
978 return g(value-1) + 1
979 ...
980\end{verbatim}
981
982The function \function{g()} will always raise a \exception{NameError}
983exception, because the binding of the name \samp{g} isn't in either
984its local namespace or in the module-level namespace. This isn't much
985of a problem in practice (how often do you recursively define interior
986functions like this?), but this also made using the \keyword{lambda}
987statement clumsier, and this was a problem in practice. In code which
988uses \keyword{lambda} you can often find local variables being copied
989by passing them as the default values of arguments.
990
991\begin{verbatim}
992def find(self, name):
993 "Return list of any entries equal to 'name'"
994 L = filter(lambda x, name=name: x == name,
995 self.list_attribute)
996 return L
997\end{verbatim}
998
999The readability of Python code written in a strongly functional style
1000suffers greatly as a result.
1001
1002The most significant change to Python 2.2 is that static scoping has
1003been added to the language to fix this problem. As a first effect,
1004the \code{name=name} default argument is now unnecessary in the above
1005example. Put simply, when a given variable name is not assigned a
1006value within a function (by an assignment, or the \keyword{def},
1007\keyword{class}, or \keyword{import} statements), references to the
1008variable will be looked up in the local namespace of the enclosing
1009scope. A more detailed explanation of the rules, and a dissection of
1010the implementation, can be found in the PEP.
1011
1012This change may cause some compatibility problems for code where the
1013same variable name is used both at the module level and as a local
1014variable within a function that contains further function definitions.
1015This seems rather unlikely though, since such code would have been
1016pretty confusing to read in the first place.
1017
1018One side effect of the change is that the \code{from \var{module}
1019import *} and \keyword{exec} statements have been made illegal inside
1020a function scope under certain conditions. The Python reference
1021manual has said all along that \code{from \var{module} import *} is
1022only legal at the top level of a module, but the CPython interpreter
1023has never enforced this before. As part of the implementation of
1024nested scopes, the compiler which turns Python source into bytecodes
1025has to generate different code to access variables in a containing
1026scope. \code{from \var{module} import *} and \keyword{exec} make it
1027impossible for the compiler to figure this out, because they add names
1028to the local namespace that are unknowable at compile time.
1029Therefore, if a function contains function definitions or
1030\keyword{lambda} expressions with free variables, the compiler will
1031flag this by raising a \exception{SyntaxError} exception.
1032
1033To make the preceding explanation a bit clearer, here's an example:
1034
1035\begin{verbatim}
1036x = 1
1037def f():
1038 # The next line is a syntax error
1039 exec 'x=2'
1040 def g():
1041 return x
1042\end{verbatim}
1043
1044Line 4 containing the \keyword{exec} statement is a syntax error,
1045since \keyword{exec} would define a new local variable named \samp{x}
1046whose value should be accessed by \function{g()}.
1047
1048This shouldn't be much of a limitation, since \keyword{exec} is rarely
1049used in most Python code (and when it is used, it's often a sign of a
1050poor design anyway).
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +00001051
1052\begin{seealso}
1053
1054\seepep{227}{Statically Nested Scopes}{Written and implemented by
1055Jeremy Hylton.}
1056
1057\end{seealso}
1058
Andrew M. Kuchlinga43e7032001-06-27 20:32:12 +00001059
1060%======================================================================
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +00001061\section{New and Improved Modules}
1062
1063\begin{itemize}
1064
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +00001065 \item The \module{xmlrpclib} module was contributed to the standard
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +00001066 library by Fredrik Lundh, provding support for writing XML-RPC
1067 clients. XML-RPC is a simple remote procedure call protocol built on
Andrew M. Kuchling8c69c91b2001-08-07 14:28:58 +00001068 top of HTTP and XML. For example, the following snippet retrieves a
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +00001069 list of RSS channels from the O'Reilly Network, and then
1070 lists the recent headlines for one channel:
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +00001071
1072\begin{verbatim}
1073import xmlrpclib
1074s = xmlrpclib.Server(
1075 'http://www.oreillynet.com/meerkat/xml-rpc/server.php')
1076channels = s.meerkat.getChannels()
1077# channels is a list of dictionaries, like this:
1078# [{'id': 4, 'title': 'Freshmeat Daily News'}
1079# {'id': 190, 'title': '32Bits Online'},
1080# {'id': 4549, 'title': '3DGamers'}, ... ]
1081
1082# Get the items for one channel
1083items = s.meerkat.getItems( {'channel': 4} )
1084
1085# 'items' is another list of dictionaries, like this:
1086# [{'link': 'http://freshmeat.net/releases/52719/',
1087# 'description': 'A utility which converts HTML to XSL FO.',
1088# 'title': 'html2fo 0.3 (Default)'}, ... ]
1089\end{verbatim}
1090
Andrew M. Kuchlingd4707e32001-09-28 20:46:46 +00001091The \module{SimpleXMLRPCServer} module makes it easy to create
1092straightforward XML-RPC servers. See \url{http://www.xmlrpc.com/} for
1093more information about XML-RPC.
1094
Andrew M. Kuchling433b5c42001-10-30 21:36:04 +00001095 \item The new \module{hmac} module implements the HMAC
Andrew M. Kuchlingd4707e32001-09-28 20:46:46 +00001096 algorithm described by \rfc{2104}.
Andrew M. Kuchling433b5c42001-10-30 21:36:04 +00001097 (Contributed by Gerhard H\"aring.)
1098
1099 \item Several functions that originally returned lengthy tuples now
1100 return pseudo-sequences that still behave like tuples but also have
1101 mnemonic attributes such as member{st_mtime} or \member{tm_year}.
1102 The enhanced functions include \function{stat()},
1103 \function{fstat()}, \function{statvfs()}, and \function{fstatvfs()}
1104 in the \module{os} module, and \function{localtime()},
1105 \function{gmtime()}, and \function{strptime()} in the \module{time}
1106 module.
1107
1108 For example, to obtain a file's size using the old tuples, you'd end
1109 up writing something like \code{file_size =
1110 os.stat(filename)[stat.ST_SIZE]}, but now this can be written more
1111 clearly as \code{file_size = os.stat(filename).st_size}.
1112
1113 The original patch for this feature was contributed by Nick Mathewson.
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +00001114
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +00001115 \item The Python profiler has been extensively reworked and various
1116 errors in its output have been corrected. (Contributed by Fred
1117 Fred~L. Drake, Jr. and Tim Peters.)
1118
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +00001119 \item The \module{socket} module can be compiled to support IPv6;
Andrew M. Kuchlingddeb1352001-07-16 14:35:52 +00001120 specify the \longprogramopt{enable-ipv6} option to Python's configure
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +00001121 script. (Contributed by Jun-ichiro ``itojun'' Hagino.)
1122
1123 \item Two new format characters were added to the \module{struct}
1124 module for 64-bit integers on platforms that support the C
1125 \ctype{long long} type. \samp{q} is for a signed 64-bit integer,
1126 and \samp{Q} is for an unsigned one. The value is returned in
1127 Python's long integer type. (Contributed by Tim Peters.)
1128
1129 \item In the interpreter's interactive mode, there's a new built-in
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +00001130 function \function{help()} that uses the \module{pydoc} module
1131 introduced in Python 2.1 to provide interactive help.
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +00001132 \code{help(\var{object})} displays any available help text about
1133 \var{object}. \code{help()} with no argument puts you in an online
1134 help utility, where you can enter the names of functions, classes,
1135 or modules to read their help text.
1136 (Contributed by Guido van Rossum, using Ka-Ping Yee's \module{pydoc} module.)
1137
1138 \item Various bugfixes and performance improvements have been made
Andrew M. Kuchling4cf52a92001-07-17 12:48:48 +00001139 to the SRE engine underlying the \module{re} module. For example,
Andrew M. Kuchlingbeb38552001-10-22 14:11:06 +00001140 the \function{re.sub()} and \function{re.split()} functions have
1141 been rewritten in C. Another contributed patch speeds up certain
Andrew M. Kuchling433b5c42001-10-30 21:36:04 +00001142 Unicode character ranges by a factor of two, and a new \method{finditer()}
1143 method that returns an iterator over all the non-overlapping matches in
1144 a given string.
1145 (SRE is maintained by
Andrew M. Kuchlingbeb38552001-10-22 14:11:06 +00001146 Fredrik Lundh. The BIGCHARSET patch was contributed by Martin von
1147 L\"owis.)
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +00001148
Andrew M. Kuchling1efd7ad2001-09-14 16:19:27 +00001149 \item The \module{smtplib} module now supports \rfc{2487}, ``Secure
1150 SMTP over TLS'', so it's now possible to encrypt the SMTP traffic
1151 between a Python program and the mail transport agent being handed a
Andrew M. Kuchling433b5c42001-10-30 21:36:04 +00001152 message. \module{smtplib} also supports SMTP authentication.
1153 (Contributed by Gerhard H\"aring.)
Andrew M. Kuchling1efd7ad2001-09-14 16:19:27 +00001154
Andrew M. Kuchlinga6d2a042001-07-20 18:34:34 +00001155 \item The \module{imaplib} module, maintained by Piers Lauder, has
1156 support for several new extensions: the NAMESPACE extension defined
1157 in \rfc{2342}, SORT, GETACL and SETACL. (Contributed by Anthony
1158 Baxter and Michel Pelletier.)
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +00001159
Andrew M. Kuchling433b5c42001-10-30 21:36:04 +00001160 % XXX should the 'email' module get a section of its own?
Andrew M. Kuchlingd4707e32001-09-28 20:46:46 +00001161 \item The \module{rfc822} module's parsing of email addresses is now
1162 compliant with \rfc{2822}, an update to \rfc{822}. (The module's
1163 name is \emph{not} going to be changed to \samp{rfc2822}.) A new
1164 package, \module{email}, has also been added for parsing and
1165 generating e-mail messages. (Contributed by Barry Warsaw, and
1166 arising out of his work on Mailman.)
Andrew M. Kuchling77707672001-07-31 15:51:16 +00001167
Andrew M. Kuchlingbbde5882001-10-31 13:13:36 +00001168 \item The \module{difflib} module now contains a new \class{Differ}
1169 class for producing human-readable lists of changes (a ``delta'')
1170 between two sequences of lines of text. There are also two
1171 generator functions, \function{ndiff()} and \function{restore()},
1172 which respectively return a delta from two sequences, or one of the
1173 original sequences from a delta. (Grunt work contributed by David
1174 Goodger, from ndiff.py code by Tim Peters who then did the
1175 generatorization.)
1176
Andrew M. Kuchling77707672001-07-31 15:51:16 +00001177 \item New constants \constant{ascii_letters},
1178 \constant{ascii_lowercase}, and \constant{ascii_uppercase} were
1179 added to the \module{string} module. There were several modules in
1180 the standard library that used \constant{string.letters} to mean the
1181 ranges A-Za-z, but that assumption is incorrect when locales are in
1182 use, because \constant{string.letters} varies depending on the set
1183 of legal characters defined by the current locale. The buggy
1184 modules have all been fixed to use \constant{ascii_letters} instead.
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +00001185 (Reported by an unknown person; fixed by Fred~L. Drake, Jr.)
Andrew M. Kuchling77707672001-07-31 15:51:16 +00001186
Andrew M. Kuchling8c69c91b2001-08-07 14:28:58 +00001187 \item The \module{mimetypes} module now makes it easier to use
1188 alternative MIME-type databases by the addition of a
1189 \class{MimeTypes} class, which takes a list of filenames to be
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +00001190 parsed. (Contributed by Fred~L. Drake, Jr.)
Andrew M. Kuchling8c69c91b2001-08-07 14:28:58 +00001191
Andrew M. Kuchlingd6e40e22001-09-10 16:18:50 +00001192 \item A \class{Timer} class was added to the \module{threading}
1193 module that allows scheduling an activity to happen at some future
1194 time. (Contributed by Itamar Shtull-Trauring.)
Andrew M. Kuchling2f0047a2001-09-05 14:53:31 +00001195
Andrew M. Kuchling77707672001-07-31 15:51:16 +00001196\end{itemize}
1197
1198
1199%======================================================================
1200\section{Interpreter Changes and Fixes}
1201
1202Some of the changes only affect people who deal with the Python
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +00001203interpreter at the C level because they're writing Python extension modules,
Andrew M. Kuchling77707672001-07-31 15:51:16 +00001204embedding the interpreter, or just hacking on the interpreter itself.
1205If you only write Python code, none of the changes described here will
1206affect you very much.
1207
1208\begin{itemize}
1209
1210 \item Profiling and tracing functions can now be implemented in C,
1211 which can operate at much higher speeds than Python-based functions
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +00001212 and should reduce the overhead of profiling and tracing. This
1213 will be of interest to authors of development environments for
Andrew M. Kuchling77707672001-07-31 15:51:16 +00001214 Python. Two new C functions were added to Python's API,
1215 \cfunction{PyEval_SetProfile()} and \cfunction{PyEval_SetTrace()}.
1216 The existing \function{sys.setprofile()} and
1217 \function{sys.settrace()} functions still exist, and have simply
1218 been changed to use the new C-level interface. (Contributed by Fred
1219 L. Drake, Jr.)
1220
1221 \item Another low-level API, primarily of interest to implementors
1222 of Python debuggers and development tools, was added.
1223 \cfunction{PyInterpreterState_Head()} and
1224 \cfunction{PyInterpreterState_Next()} let a caller walk through all
1225 the existing interpreter objects;
1226 \cfunction{PyInterpreterState_ThreadHead()} and
1227 \cfunction{PyThreadState_Next()} allow looping over all the thread
1228 states for a given interpreter. (Contributed by David Beazley.)
1229
1230 \item A new \samp{et} format sequence was added to
1231 \cfunction{PyArg_ParseTuple}; \samp{et} takes both a parameter and
1232 an encoding name, and converts the parameter to the given encoding
1233 if the parameter turns out to be a Unicode string, or leaves it
1234 alone if it's an 8-bit string, assuming it to already be in the
1235 desired encoding. This differs from the \samp{es} format character,
1236 which assumes that 8-bit strings are in Python's default ASCII
1237 encoding and converts them to the specified new encoding.
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +00001238 (Contributed by M.-A. Lemburg, and used for the MBCS support on
1239 Windows described in the following section.)
Andrew M. Kuchlingcf31d5d2001-10-26 20:37:55 +00001240
1241 \item A different argument parsing function,
1242 \cfunction{PyArg_UnpackTuple()}, has been added that's simpler and
1243 presumably faster. Instead of specifying a format string, the
1244 caller simply gives the minimum and maximum number of arguments
1245 expected, and a set of pointers to \code{PyObject*} variables that
1246 will be filled in with argument values.
1247
Andrew M. Kuchling0ab31b82001-08-29 01:16:54 +00001248 \item Two new flags \constant{METH_NOARGS} and \constant{METH_O} are
1249 available in method definition tables to simplify implementation of
1250 methods with no arguments or a single untyped argument. Calling
1251 such methods is more efficient than calling a corresponding method
1252 that uses \constant{METH_VARARGS}.
1253 Also, the old \constant{METH_OLDARGS} style of writing C methods is
1254 now officially deprecated.
1255
1256\item
1257 Two new wrapper functions, \cfunction{PyOS_snprintf()} and
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +00001258 \cfunction{PyOS_vsnprintf()} were added to provide
Andrew M. Kuchling0ab31b82001-08-29 01:16:54 +00001259 cross-platform implementations for the relatively new
1260 \cfunction{snprintf()} and \cfunction{vsnprintf()} C lib APIs. In
1261 contrast to the standard \cfunction{sprintf()} and
1262 \cfunction{vsprintf()} functions, the Python versions check the
1263 bounds of the buffer used to protect against buffer overruns.
1264 (Contributed by M.-A. Lemburg.)
Andrew M. Kuchling77707672001-07-31 15:51:16 +00001265
Andrew M. Kuchlingccf04652001-11-26 18:15:44 +00001266 \item The \cfunction{_PyTuple_Resize()} function has lost an unused
1267 parameter, so now it takes 2 parameters instead of 3. The third
1268 argument was never used, and can simply be discarded when porting
1269 code from earlier versions to Python 2.2.
1270
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +00001271\end{itemize}
1272
1273
1274%======================================================================
1275\section{Other Changes and Fixes}
1276
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +00001277As usual there were a bunch of other improvements and bugfixes
1278scattered throughout the source tree. A search through the CVS change
Andrew M. Kuchlingadc7df52001-12-20 16:04:24 +00001279logs finds there were 527 patches applied, and 683 bugs fixed; both
Andrew M. Kuchling4dbf8712001-07-16 02:17:14 +00001280figures are likely to be underestimates. Some of the more notable
1281changes are:
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +00001282
1283\begin{itemize}
1284
Andrew M. Kuchling0e03f582001-08-30 21:30:16 +00001285 \item The code for the MacOS port for Python, maintained by Jack
1286 Jansen, is now kept in the main Python CVS tree, and many changes
Andrew M. Kuchling279e7442001-10-22 02:03:40 +00001287 have been made to support MacOS~X.
Andrew M. Kuchling0e03f582001-08-30 21:30:16 +00001288
1289The most significant change is the ability to build Python as a
1290framework, enabled by supplying the \longprogramopt{enable-framework}
1291option to the configure script when compiling Python. According to
1292Jack Jansen, ``This installs a self-contained Python installation plus
Andrew M. Kuchling279e7442001-10-22 02:03:40 +00001293the OS~X framework "glue" into
Andrew M. Kuchling0e03f582001-08-30 21:30:16 +00001294\file{/Library/Frameworks/Python.framework} (or another location of
1295choice). For now there is little immediate added benefit to this
1296(actually, there is the disadvantage that you have to change your PATH
1297to be able to find Python), but it is the basis for creating a
1298full-blown Python application, porting the MacPython IDE, possibly
1299using Python as a standard OSA scripting language and much more.''
1300
1301Most of the MacPython toolbox modules, which interface to MacOS APIs
Andrew M. Kuchling279e7442001-10-22 02:03:40 +00001302such as windowing, QuickTime, scripting, etc. have been ported to OS~X,
Andrew M. Kuchlingbeb38552001-10-22 14:11:06 +00001303but they've been left commented out in \file{setup.py}. People who want
Andrew M. Kuchling0e03f582001-08-30 21:30:16 +00001304to experiment with these modules can uncomment them manually.
1305
1306% Jack's original comments:
1307%The main change is the possibility to build Python as a
1308%framework. This installs a self-contained Python installation plus the
1309%OSX framework "glue" into /Library/Frameworks/Python.framework (or
1310%another location of choice). For now there is little immedeate added
1311%benefit to this (actually, there is the disadvantage that you have to
1312%change your PATH to be able to find Python), but it is the basis for
1313%creating a fullblown Python application, porting the MacPython IDE,
1314%possibly using Python as a standard OSA scripting language and much
1315%more. You enable this with "configure --enable-framework".
1316
1317%The other change is that most MacPython toolbox modules, which
1318%interface to all the MacOS APIs such as windowing, quicktime,
1319%scripting, etc. have been ported. Again, most of these are not of
1320%immedeate use, as they need a full application to be really useful, so
1321%they have been commented out in setup.py. People wanting to experiment
1322%can uncomment them. Gestalt and Internet Config modules are enabled by
1323%default.
Andrew M. Kuchling0e03f582001-08-30 21:30:16 +00001324
Andrew M. Kuchling2cd712b2001-07-16 13:39:08 +00001325 \item Keyword arguments passed to builtin functions that don't take them
1326 now cause a \exception{TypeError} exception to be raised, with the
1327 message "\var{function} takes no keyword arguments".
1328
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +00001329 \item Weak references, added in Python 2.1 as an extension module,
1330 are now part of the core because they're used in the implementation
1331 of new-style classes. The \exception{ReferenceError} exception has
1332 therefore moved from the \module{weakref} module to become a
1333 built-in exception.
1334
Andrew M. Kuchling94a7eba2001-08-15 15:55:48 +00001335 \item A new script, \file{Tools/scripts/cleanfuture.py} by Tim
1336 Peters, automatically removes obsolete \code{__future__} statements
1337 from Python source code.
Andrew M. Kuchling2cd712b2001-07-16 13:39:08 +00001338
Andrew M. Kuchling4f9e2202001-10-29 18:09:42 +00001339 \item An additional \var{flags} argument has been added to the
1340 built-in function \function{compile()}, so the behaviour of
1341 \code{__future__} statements can now be correctly observed in
1342 simulated shells, such as those presented by IDLE and other
1343 development environments. This is described in \pep{264}.
1344 (Contributed by Michael Hudson.)
1345
Andrew M. Kuchling2cd712b2001-07-16 13:39:08 +00001346 \item The new license introduced with Python 1.6 wasn't
1347 GPL-compatible. This is fixed by some minor textual changes to the
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +00001348 2.2 license, so it's now legal to embed Python inside a GPLed
1349 program again. Note that Python itself is not GPLed, but instead is
1350 under a license that's essentially equivalent to the BSD license,
1351 same as it always was. The license changes were also applied to the
1352 Python 2.0.1 and 2.1.1 releases.
Andrew M. Kuchling2cd712b2001-07-16 13:39:08 +00001353
Andrew M. Kuchlingf4ccf582001-07-31 01:11:36 +00001354 \item When presented with a Unicode filename on Windows, Python will
1355 now convert it to an MBCS encoded string, as used by the Microsoft
1356 file APIs. As MBCS is explicitly used by the file APIs, Python's
1357 choice of ASCII as the default encoding turns out to be an
Andrew M. Kuchling433b5c42001-10-30 21:36:04 +00001358 annoyance. On Unix, the locale's character set is used if
1359 \function{locale.nl_langinfo(CODESET)} is available. (Windows
1360 support was contributed by Mark Hammond with assistance from
1361 Marc-Andr\'e Lemburg. Unix support was added by Martin von L\"owis.)
Andrew M. Kuchling8cfa9052001-07-19 01:19:59 +00001362
Andrew M. Kuchlingd6e40e22001-09-10 16:18:50 +00001363 \item Large file support is now enabled on Windows. (Contributed by
1364 Tim Peters.)
1365
Andrew M. Kuchling2cd712b2001-07-16 13:39:08 +00001366 \item The \file{Tools/scripts/ftpmirror.py} script
1367 now parses a \file{.netrc} file, if you have one.
Andrew M. Kuchling4cf52a92001-07-17 12:48:48 +00001368 (Contributed by Mike Romberg.)
Andrew M. Kuchling2cd712b2001-07-16 13:39:08 +00001369
Andrew M. Kuchling4cf52a92001-07-17 12:48:48 +00001370 \item Some features of the object returned by the
1371 \function{xrange()} function are now deprecated, and trigger
1372 warnings when they're accessed; they'll disappear in Python 2.3.
1373 \class{xrange} objects tried to pretend they were full sequence
1374 types by supporting slicing, sequence multiplication, and the
1375 \keyword{in} operator, but these features were rarely used and
1376 therefore buggy. The \method{tolist()} method and the
1377 \member{start}, \member{stop}, and \member{step} attributes are also
1378 being deprecated. At the C level, the fourth argument to the
1379 \cfunction{PyRange_New()} function, \samp{repeat}, has also been
1380 deprecated.
1381
Andrew M. Kuchling8cfa9052001-07-19 01:19:59 +00001382 \item There were a bunch of patches to the dictionary
1383 implementation, mostly to fix potential core dumps if a dictionary
1384 contains objects that sneakily changed their hash value, or mutated
1385 the dictionary they were contained in. For a while python-dev fell
Andrew M. Kuchling8b42f012001-10-22 02:00:11 +00001386 into a gentle rhythm of Michael Hudson finding a case that dumped
1387 core, Tim Peters fixing the bug, Michael finding another case, and round
Andrew M. Kuchling8cfa9052001-07-19 01:19:59 +00001388 and round it went.
1389
Andrew M. Kuchling33a3b632001-09-04 21:25:58 +00001390 \item On Windows, Python can now be compiled with Borland C thanks
1391 to a number of patches contributed by Stephen Hansen, though the
1392 result isn't fully functional yet. (But this \emph{is} progress...)
Andrew M. Kuchling8c69c91b2001-08-07 14:28:58 +00001393
Andrew M. Kuchlingf4ccf582001-07-31 01:11:36 +00001394 \item Another Windows enhancement: Wise Solutions generously offered
1395 PythonLabs use of their InstallerMaster 8.1 system. Earlier
1396 PythonLabs Windows installers used Wise 5.0a, which was beginning to
1397 show its age. (Packaged up by Tim Peters.)
1398
Andrew M. Kuchling8c69c91b2001-08-07 14:28:58 +00001399 \item Files ending in \samp{.pyw} can now be imported on Windows.
1400 \samp{.pyw} is a Windows-only thing, used to indicate that a script
1401 needs to be run using PYTHONW.EXE instead of PYTHON.EXE in order to
1402 prevent a DOS console from popping up to display the output. This
1403 patch makes it possible to import such scripts, in case they're also
1404 usable as modules. (Implemented by David Bolen.)
1405
Andrew M. Kuchling8cfa9052001-07-19 01:19:59 +00001406 \item On platforms where Python uses the C \cfunction{dlopen()} function
1407 to load extension modules, it's now possible to set the flags used
1408 by \cfunction{dlopen()} using the \function{sys.getdlopenflags()} and
1409 \function{sys.setdlopenflags()} functions. (Contributed by Bram Stolk.)
Andrew M. Kuchling2f0047a2001-09-05 14:53:31 +00001410
Andrew M. Kuchling26c39bf2001-09-10 03:20:53 +00001411 \item The \function{pow()} built-in function no longer supports 3
1412 arguments when floating-point numbers are supplied.
Andrew M. Kuchling1497b622001-09-24 14:51:16 +00001413 \code{pow(\var{x}, \var{y}, \var{z})} returns \code{(x**y) \% z}, but
Andrew M. Kuchling26c39bf2001-09-10 03:20:53 +00001414 this is never useful for floating point numbers, and the final
1415 result varies unpredictably depending on the platform. A call such
Andrew M. Kuchlingd6e40e22001-09-10 16:18:50 +00001416 as \code{pow(2.0, 8.0, 7.0)} will now raise a \exception{TypeError}
Andrew M. Kuchling26c39bf2001-09-10 03:20:53 +00001417 exception.
Andrew M. Kuchling77707672001-07-31 15:51:16 +00001418
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +00001419\end{itemize}
1420
1421
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +00001422%======================================================================
1423\section{Acknowledgements}
1424
1425The author would like to thank the following people for offering
Andrew M. Kuchlingb83769c2001-10-26 20:07:03 +00001426suggestions, corrections and assistance with various drafts of this
1427article: Fred Bremmer, Keith Briggs, Andrew Dalke, Fred~L. Drake, Jr.,
Andrew M. Kuchlingccf04652001-11-26 18:15:44 +00001428Carel Fellinger, David Goodger, Mark Hammond, Stephen Hansen, Michael
1429Hudson, Jack Jansen, Marc-Andr\'e Lemburg, Martin von L\"owis, Fredrik
1430Lundh, Michael McLay, Nick Mathewson, Paul Moore, Gustavo Niemeyer,
1431Don O'Donnell, Tim Peters, Jens Quade, Tom Reinhardt, Neil
1432Schemenauer, Guido van Rossum, Greg Ward.
Andrew M. Kuchlinga8defaa2001-05-05 16:37:29 +00001433
1434\end{document}