blob: 74419aa80ebce97d50bf874f4744be5f5a116ff5 [file] [log] [blame]
Fred Drakeed0fa3d2003-07-30 19:14:09 +00001\documentclass{howto}
2\usepackage{distutils}
3% $Id$
4
Andrew M. Kuchlingc8f8a812004-07-04 01:26:42 +00005% Don't write extensive text for new sections; I'll do that.
6% Feel free to add commented-out reminders of things that need
7% to be covered. --amk
8
Fred Drakeed0fa3d2003-07-30 19:14:09 +00009\title{What's New in Python 2.4}
Andrew M. Kuchling536183b2004-11-25 01:15:25 +000010\release{1.01}
Fred Drakeed0fa3d2003-07-30 19:14:09 +000011\author{A.M.\ Kuchling}
Fred Drakeb914ef02004-01-02 06:57:50 +000012\authoraddress{
13 \strong{Python Software Foundation}\\
14 Email: \email{amk@amk.ca}
15}
Fred Drakeed0fa3d2003-07-30 19:14:09 +000016
17\begin{document}
18\maketitle
19\tableofcontents
20
Andrew M. Kuchling48715352004-11-30 14:42:24 +000021This article explains the new features in Python 2.4, released on
22November~30, 2004.
Fred Drakeed0fa3d2003-07-30 19:14:09 +000023
Andrew M. Kuchling3bf85f12004-07-05 01:37:07 +000024Python 2.4 is a medium-sized release. It doesn't introduce as many
Andrew M. Kuchling3b790912004-07-04 16:39:40 +000025changes as the radical Python 2.2, but introduces more features than
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +000026the conservative 2.3 release. The most significant new language
27features are function decorators and generator expressions; most other
28changes are to the standard library.
29
Andrew M. Kuchling74666592004-11-19 14:26:23 +000030According to the CVS change logs, there were 481 patches applied and
31502 bugs fixed between Python 2.3 and 2.4. Both figures are likely to
Andrew M. Kuchlinga331e862004-09-10 13:05:22 +000032be underestimates.
33
Fred Drakeed0fa3d2003-07-30 19:14:09 +000034This article doesn't attempt to provide a complete specification of
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +000035every single new feature, but instead provides a brief introduction to
36each feature. For full details, you should refer to the documentation
37for Python 2.4, such as the \citetitle[../lib/lib.html]{Python Library
38Reference} and the \citetitle[../ref/ref.html]{Python Reference
39Manual}. Often you will be referred to the PEP for a particular new
40feature for explanations of the implementation and design rationale.
Fred Drakeed0fa3d2003-07-30 19:14:09 +000041
Andrew M. Kuchling35f2b052003-12-18 13:28:13 +000042
Raymond Hettinger7e0282f2003-11-24 07:14:54 +000043%======================================================================
44\section{PEP 218: Built-In Set Objects}
45
Andrew M. Kuchling3bf85f12004-07-05 01:37:07 +000046Python 2.3 introduced the \module{sets} module. C implementations of
47set data types have now been added to the Python core as two new
48built-in types, \function{set(\var{iterable})} and
49\function{frozenset(\var{iterable})}. They provide high speed
50operations for membership testing, for eliminating duplicates from
51sequences, and for mathematical operations like unions, intersections,
52differences, and symmetric differences.
Raymond Hettinger7e0282f2003-11-24 07:14:54 +000053
54\begin{verbatim}
55>>> a = set('abracadabra') # form a set from a string
56>>> 'z' in a # fast membership testing
57False
58>>> a # unique letters in a
59set(['a', 'r', 'b', 'c', 'd'])
60>>> ''.join(a) # convert back into a string
61'arbcd'
Raymond Hettingerd4462302003-11-26 17:52:45 +000062
Raymond Hettinger7e0282f2003-11-24 07:14:54 +000063>>> b = set('alacazam') # form a second set
64>>> a - b # letters in a but not in b
65set(['r', 'd', 'b'])
66>>> a | b # letters in either a or b
67set(['a', 'c', 'r', 'd', 'b', 'm', 'z', 'l'])
68>>> a & b # letters in both a and b
69set(['a', 'c'])
70>>> a ^ b # letters in a or b but not both
71set(['r', 'd', 'b', 'm', 'z', 'l'])
Raymond Hettingerd4462302003-11-26 17:52:45 +000072
Raymond Hettinger7e0282f2003-11-24 07:14:54 +000073>>> a.add('z') # add a new element
74>>> a.update('wxy') # add multiple new elements
75>>> a
76set(['a', 'c', 'b', 'd', 'r', 'w', 'y', 'x', 'z'])
77>>> a.remove('x') # take one element out
78>>> a
79set(['a', 'c', 'b', 'd', 'r', 'w', 'y', 'z'])
80\end{verbatim}
81
Andrew M. Kuchling3bf85f12004-07-05 01:37:07 +000082The \function{frozenset} type is an immutable version of \function{set}.
Raymond Hettinger7e0282f2003-11-24 07:14:54 +000083Since it is immutable and hashable, it may be used as a dictionary key or
Andrew M. Kuchling3bf85f12004-07-05 01:37:07 +000084as a member of another set.
Raymond Hettinger7e0282f2003-11-24 07:14:54 +000085
Andrew M. Kuchling3bf85f12004-07-05 01:37:07 +000086The \module{sets} module remains in the standard library, and may be
87useful if you wish to subclass the \class{Set} or \class{ImmutableSet}
88classes. There are currently no plans to deprecate the module.
Andrew M. Kuchling35f2b052003-12-18 13:28:13 +000089
Raymond Hettinger7e0282f2003-11-24 07:14:54 +000090\begin{seealso}
91\seepep{218}{Adding a Built-In Set Object Type}{Originally proposed by
92Greg Wilson and ultimately implemented by Raymond Hettinger.}
93\end{seealso}
Fred Drakeed0fa3d2003-07-30 19:14:09 +000094
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +000095
Fred Drakeed0fa3d2003-07-30 19:14:09 +000096%======================================================================
Andrew M. Kuchling35f2b052003-12-18 13:28:13 +000097\section{PEP 237: Unifying Long Integers and Integers}
98
Andrew M. Kuchling3bf85f12004-07-05 01:37:07 +000099The lengthy transition process for this PEP, begun in Python 2.2,
Andrew M. Kuchlingd4be86c2004-07-04 01:44:04 +0000100takes another step forward in Python 2.4. In 2.3, certain integer
101operations that would behave differently after int/long unification
102triggered \exception{FutureWarning} warnings and returned values
Andrew M. Kuchling3bf85f12004-07-05 01:37:07 +0000103limited to 32 or 64 bits (depending on your platform). In 2.4, these
104expressions no longer produce a warning and instead produce a
105different result that's usually a long integer.
Andrew M. Kuchlingd4be86c2004-07-04 01:44:04 +0000106
107The problematic expressions are primarily left shifts and lengthy
Raymond Hettingerca1a7752004-07-12 13:00:45 +0000108hexadecimal and octal constants. For example,
109\code{2 \textless{}\textless{} 32} results
Andrew M. Kuchling3bf85f12004-07-05 01:37:07 +0000110in a warning in 2.3, evaluating to 0 on 32-bit platforms. In Python
1112.4, this expression now returns the correct answer, 8589934592.
Andrew M. Kuchlingd4be86c2004-07-04 01:44:04 +0000112
113\begin{seealso}
114\seepep{237}{Unifying Long Integers and Integers}{Original PEP
Andrew M. Kuchling3bf85f12004-07-05 01:37:07 +0000115written by Moshe Zadka and GvR. The changes for 2.4 were implemented by
Andrew M. Kuchlingd4be86c2004-07-04 01:44:04 +0000116Kalle Svensson.}
117\end{seealso}
Andrew M. Kuchling35f2b052003-12-18 13:28:13 +0000118
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +0000119
Andrew M. Kuchling35f2b052003-12-18 13:28:13 +0000120%======================================================================
Andrew M. Kuchlingc8f8a812004-07-04 01:26:42 +0000121\section{PEP 289: Generator Expressions}
Raymond Hettinger354433a2004-05-19 08:20:33 +0000122
Andrew M. Kuchling38dc2a62004-08-07 13:24:12 +0000123The iterator feature introduced in Python 2.2 and the
124\module{itertools} module make it easier to write programs that loop
125through large data sets without having the entire data set in memory
126at one time. List comprehensions don't fit into this picture very
127well because they produce a Python list object containing all of the
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +0000128items. This unavoidably pulls all of the objects into memory, which
129can be a problem if your data set is very large. When trying to write
Andrew M. Kuchling38dc2a62004-08-07 13:24:12 +0000130a functionally-styled program, it would be natural to write something
Andrew M. Kuchling3bf85f12004-07-05 01:37:07 +0000131like:
Raymond Hettinger354433a2004-05-19 08:20:33 +0000132
133\begin{verbatim}
Andrew M. Kuchlingc8f8a812004-07-04 01:26:42 +0000134links = [link for link in get_all_links() if not link.followed]
135for link in links:
136 ...
Raymond Hettinger354433a2004-05-19 08:20:33 +0000137\end{verbatim}
138
Andrew M. Kuchlingc8f8a812004-07-04 01:26:42 +0000139instead of
Raymond Hettinger354433a2004-05-19 08:20:33 +0000140
141\begin{verbatim}
Andrew M. Kuchlingc8f8a812004-07-04 01:26:42 +0000142for link in get_all_links():
143 if link.followed:
144 continue
145 ...
146\end{verbatim}
Raymond Hettinger354433a2004-05-19 08:20:33 +0000147
Andrew M. Kuchlingc8f8a812004-07-04 01:26:42 +0000148The first form is more concise and perhaps more readable, but if
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +0000149you're dealing with a large number of link objects you'd have to write
150the second form to avoid having all link objects in memory at the same
151time.
Raymond Hettinger354433a2004-05-19 08:20:33 +0000152
Andrew M. Kuchlingc8f8a812004-07-04 01:26:42 +0000153Generator expressions work similarly to list comprehensions but don't
154materialize the entire list; instead they create a generator that will
155return elements one by one. The above example could be written as:
Raymond Hettinger354433a2004-05-19 08:20:33 +0000156
Andrew M. Kuchlingc8f8a812004-07-04 01:26:42 +0000157\begin{verbatim}
158links = (link for link in get_all_links() if not link.followed)
159for link in links:
160 ...
161\end{verbatim}
Raymond Hettinger170a6222004-05-19 19:45:19 +0000162
Andrew M. Kuchlingc8f8a812004-07-04 01:26:42 +0000163Generator expressions always have to be written inside parentheses, as
164in the above example. The parentheses signalling a function call also
165count, so if you want to create a iterator that will be immediately
166passed to a function you could write:
Raymond Hettinger170a6222004-05-19 19:45:19 +0000167
Andrew M. Kuchlingc8f8a812004-07-04 01:26:42 +0000168\begin{verbatim}
169print sum(obj.count for obj in list_all_objects())
170\end{verbatim}
Raymond Hettinger170a6222004-05-19 19:45:19 +0000171
Andrew M. Kuchling3bf85f12004-07-05 01:37:07 +0000172Generator expressions differ from list comprehensions in various small
173ways. Most notably, the loop variable (\var{obj} in the above
174example) is not accessible outside of the generator expression. List
175comprehensions leave the variable assigned to its last value; future
176versions of Python will change this, making list comprehensions match
177generator expressions in this respect.
Raymond Hettinger354433a2004-05-19 08:20:33 +0000178
179\begin{seealso}
180\seepep{289}{Generator Expressions}{Proposed by Raymond Hettinger and
181implemented by Jiwon Seo with early efforts steered by Hye-Shik Chang.}
182\end{seealso}
183
Andrew M. Kuchling87c98b22004-08-25 13:38:46 +0000184
185%======================================================================
186\section{PEP 292: Simpler String Substitutions}
187
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +0000188Some new classes in the standard library provide an alternative
189mechanism for substituting variables into strings; this style of
190substitution may be better for applications where untrained
191users need to edit templates.
Andrew M. Kuchling87c98b22004-08-25 13:38:46 +0000192
193The usual way of substituting variables by name is the \code{\%}
194operator:
195
196\begin{verbatim}
197>>> '%(page)i: %(title)s' % {'page':2, 'title': 'The Best of Times'}
198'2: The Best of Times'
199\end{verbatim}
200
201When writing the template string, it can be easy to forget the
202\samp{i} or \samp{s} after the closing parenthesis. This isn't a big
203problem if the template is in a Python module, because you run the
204code, get an ``Unsupported format character'' \exception{ValueError},
205and fix the problem. However, consider an application such as Mailman
206where template strings or translations are being edited by users who
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +0000207aren't aware of the Python language. The format string's syntax is
208complicated to explain to such users, and if they make a mistake, it's
209difficult to provide helpful feedback to them.
Andrew M. Kuchling87c98b22004-08-25 13:38:46 +0000210
211PEP 292 adds a \class{Template} class to the \module{string} module
Andrew M. Kuchlinge2a66bf2004-12-01 00:45:15 +0000212that uses \samp{\$} to indicate a substitution:
Andrew M. Kuchling87c98b22004-08-25 13:38:46 +0000213
214\begin{verbatim}
215>>> import string
216>>> t = string.Template('$page: $title')
Andrew M. Kuchlinga79ec222004-09-10 11:34:39 +0000217>>> t.substitute({'page':2, 'title': 'The Best of Times'})
Andrew M. Kuchlinge2a66bf2004-12-01 00:45:15 +0000218'2: The Best of Times'
Andrew M. Kuchling87c98b22004-08-25 13:38:46 +0000219\end{verbatim}
220
221% $ Terminate $-mode for Emacs
222
Andrew M. Kuchlinga79ec222004-09-10 11:34:39 +0000223If a key is missing from the dictionary, the \method{substitute} method
224will raise a \exception{KeyError}. There's also a \method{safe_substitute}
225method that ignores missing keys:
Andrew M. Kuchling87c98b22004-08-25 13:38:46 +0000226
227\begin{verbatim}
Andrew M. Kuchlingb69c49c2004-12-01 00:42:41 +0000228>>> t = string.Template('$page: $title')
Andrew M. Kuchlinga79ec222004-09-10 11:34:39 +0000229>>> t.safe_substitute({'page':3})
Andrew M. Kuchlinge2a66bf2004-12-01 00:45:15 +0000230'3: $title'
Andrew M. Kuchling87c98b22004-08-25 13:38:46 +0000231\end{verbatim}
232
Andrew M. Kuchlinga331e862004-09-10 13:05:22 +0000233% $ Terminate math-mode for Emacs
234
235
Andrew M. Kuchling87c98b22004-08-25 13:38:46 +0000236\begin{seealso}
237\seepep{292}{Simpler String Substitutions}{Written and implemented
238by Barry Warsaw.}
239\end{seealso}
240
241
Raymond Hettinger354433a2004-05-19 08:20:33 +0000242%======================================================================
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +0000243\section{PEP 318: Decorators for Functions and Methods}
Andrew M. Kuchlingd91fcbe2004-08-02 12:44:28 +0000244
Andrew M. Kuchling77a602f2004-08-02 13:48:18 +0000245Python 2.2 extended Python's object model by adding static methods and
246class methods, but it didn't extend Python's syntax to provide any new
247way of defining static or class methods. Instead, you had to write a
248\keyword{def} statement in the usual way, and pass the resulting
249method to a \function{staticmethod()} or \function{classmethod()}
250function that would wrap up the function as a method of the new type.
251Your code would look like this:
252
253\begin{verbatim}
254class C:
255 def meth (cls):
256 ...
257
258 meth = classmethod(meth) # Rebind name to wrapped-up class method
259\end{verbatim}
260
261If the method was very long, it would be easy to miss or forget the
262\function{classmethod()} invocation after the function body.
263
264The intention was always to add some syntax to make such definitions
265more readable, but at the time of 2.2's release a good syntax was not
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +0000266obvious. Today a good syntax \emph{still} isn't obvious but users are
267asking for easier access to the feature; a new syntactic feature has
268been added to meet this need.
Andrew M. Kuchling77a602f2004-08-02 13:48:18 +0000269
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +0000270The new feature is called ``function decorators''. The name comes
271from the idea that \function{classmethod}, \function{staticmethod},
272and friends are storing additional information on a function object;
273they're \emph{decorating} functions with more details.
Andrew M. Kuchling77a602f2004-08-02 13:48:18 +0000274
Fred Drake3f5c6542004-08-06 03:34:20 +0000275The notation borrows from Java and uses the \character{@} character as an
Andrew M. Kuchling77a602f2004-08-02 13:48:18 +0000276indicator. Using the new syntax, the example above would be written:
277
278\begin{verbatim}
279class C:
280
281 @classmethod
282 def meth (cls):
283 ...
284
285\end{verbatim}
286
287The \code{@classmethod} is shorthand for the
Fred Drake3f5c6542004-08-06 03:34:20 +0000288\code{meth=classmethod(meth)} assignment. More generally, if you have
Andrew M. Kuchling77a602f2004-08-02 13:48:18 +0000289the following:
290
291\begin{verbatim}
292@A @B @C
293def f ():
294 ...
295\end{verbatim}
296
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +0000297It's equivalent to the following pre-decorator code:
Andrew M. Kuchling77a602f2004-08-02 13:48:18 +0000298
299\begin{verbatim}
300def f(): ...
Andrew M. Kuchlingcebdd3c2004-10-08 18:29:29 +0000301f = A(B(C(f)))
Andrew M. Kuchling77a602f2004-08-02 13:48:18 +0000302\end{verbatim}
303
304Decorators must come on the line before a function definition, and
305can't be on the same line, meaning that \code{@A def f(): ...} is
306illegal. You can only decorate function definitions, either at the
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +0000307module level or inside a class; you can't decorate class definitions.
Andrew M. Kuchling77a602f2004-08-02 13:48:18 +0000308
309A decorator is just a function that takes the function to be decorated
310as an argument and returns either the same function or some new
311callable thing. It's easy to write your own decorators. The
312following simple example just sets an attribute on the function
313object:
314
315\begin{verbatim}
316>>> def deco(func):
317... func.attr = 'decorated'
318... return func
319...
320>>> @deco
321... def f(): pass
322...
323>>> f
324<function f at 0x402ef0d4>
325>>> f.attr
326'decorated'
327>>>
328\end{verbatim}
329
330As a slightly more realistic example, the following decorator checks
331that the supplied argument is an integer:
332
333\begin{verbatim}
334def require_int (func):
335 def wrapper (arg):
336 assert isinstance(arg, int)
337 return func(arg)
338
339 return wrapper
340
341@require_int
342def p1 (arg):
343 print arg
344
345@require_int
346def p2(arg):
347 print arg*2
348\end{verbatim}
349
350An example in \pep{318} contains a fancier version of this idea that
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +0000351lets you both specify the required type and check the returned type.
Andrew M. Kuchling77a602f2004-08-02 13:48:18 +0000352
353Decorator functions can take arguments. If arguments are supplied,
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +0000354your decorator function is called with only those arguments and must
355return a new decorator function; this function must take a single
Andrew M. Kuchling77a602f2004-08-02 13:48:18 +0000356function and return a function, as previously described. In other
357words, \code{@A @B @C(args)} becomes:
358
359\begin{verbatim}
360def f(): ...
361_deco = C(args)
Andrew M. Kuchlingcebdd3c2004-10-08 18:29:29 +0000362f = A(B(_deco(f)))
Andrew M. Kuchling77a602f2004-08-02 13:48:18 +0000363\end{verbatim}
364
365Getting this right can be slightly brain-bending, but it's not too
366difficult.
367
Andrew M. Kuchling87c98b22004-08-25 13:38:46 +0000368A small related change makes the \member{func_name} attribute of
369functions writable. This attribute is used to display function names
370in tracebacks, so decorators should change the name of any new
371function that's constructed and returned.
372
Andrew M. Kuchlingd91fcbe2004-08-02 12:44:28 +0000373\begin{seealso}
374\seepep{318}{Decorators for Functions, Methods and Classes}{Written
Andrew M. Kuchling77a602f2004-08-02 13:48:18 +0000375by Kevin D. Smith, Jim Jewett, and Skip Montanaro. Several people
376wrote patches implementing function decorators, but the one that was
Fred Drakee72bd4d2004-08-02 21:50:26 +0000377actually checked in was patch \#979728, written by Mark Russell.}
Andrew M. Kuchlingd91fcbe2004-08-02 12:44:28 +0000378\end{seealso}
379
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +0000380% XXX add link to decorators module in Wiki
381
382
Andrew M. Kuchlingd91fcbe2004-08-02 12:44:28 +0000383%======================================================================
Andrew M. Kuchling1a420252003-11-08 15:58:49 +0000384\section{PEP 322: Reverse Iteration}
Fred Drakeed0fa3d2003-07-30 19:14:09 +0000385
Fred Drake56fcc232004-05-06 02:55:35 +0000386A new built-in function, \function{reversed(\var{seq})}, takes a sequence
Andrew M. Kuchling3bf85f12004-07-05 01:37:07 +0000387and returns an iterator that loops over the elements of the sequence
Andrew M. Kuchling1a420252003-11-08 15:58:49 +0000388in reverse order.
389
390\begin{verbatim}
Raymond Hettingerbc3cba22003-11-12 16:39:30 +0000391>>> for i in reversed(xrange(1,4)):
Andrew M. Kuchling1a420252003-11-08 15:58:49 +0000392... print i
393...
3943
3952
3961
397\end{verbatim}
398
Andrew M. Kuchling3bf85f12004-07-05 01:37:07 +0000399Compared to extended slicing, such as \code{range(1,4)[::-1]},
400\function{reversed()} is easier to read, runs faster, and uses
401substantially less memory.
Raymond Hettingerbc3cba22003-11-12 16:39:30 +0000402
Andrew M. Kuchling1a420252003-11-08 15:58:49 +0000403Note that \function{reversed()} only accepts sequences, not arbitrary
Raymond Hettingerbc3cba22003-11-12 16:39:30 +0000404iterators. If you want to reverse an iterator, first convert it to
405a list with \function{list()}.
Andrew M. Kuchling1a420252003-11-08 15:58:49 +0000406
407\begin{verbatim}
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +0000408>>> input = open('/etc/passwd', 'r')
Andrew M. Kuchling44a31e12004-01-01 18:33:34 +0000409>>> for line in reversed(list(input)):
Andrew M. Kuchling1a420252003-11-08 15:58:49 +0000410... print line
411...
412root:*:0:0:System Administrator:/var/root:/bin/tcsh
413 ...
414\end{verbatim}
Fred Drakeed0fa3d2003-07-30 19:14:09 +0000415
Andrew M. Kuchlingf7a6b672003-11-08 16:05:37 +0000416\begin{seealso}
417\seepep{322}{Reverse Iteration}{Written and implemented by Raymond Hettinger.}
418
419\end{seealso}
420
Fred Drakeed0fa3d2003-07-30 19:14:09 +0000421
422%======================================================================
Andrew M. Kuchlingc9e7d772004-10-12 15:58:02 +0000423\section{PEP 324: New subprocess Module}
424
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +0000425The standard library provides a number of ways to execute a
426subprocess, offering different features and different levels of
427complexity. \function{os.system(\var{command})} is easy to use, but
428slow (it runs a shell process which executes the command) and
429dangerous (you have to be careful about escaping the shell's
430metacharacters). The \module{popen2} module offers classes that can
431capture standard output and standard error from the subprocess, but
432the naming is confusing. The \module{subprocess} module cleans
433this up, providing a unified interface that offers all the features
434you might need.
Andrew M. Kuchlingc9e7d772004-10-12 15:58:02 +0000435
Andrew M. Kuchlingb6ffc272004-10-12 16:36:57 +0000436Instead of \module{popen2}'s collection of classes,
437\module{subprocess} contains a single class called \class{Popen}
438whose constructor supports a number of different keyword arguments.
Andrew M. Kuchlingc9e7d772004-10-12 15:58:02 +0000439
Andrew M. Kuchlingb6ffc272004-10-12 16:36:57 +0000440\begin{verbatim}
441class Popen(args, bufsize=0, executable=None,
442 stdin=None, stdout=None, stderr=None,
443 preexec_fn=None, close_fds=False, shell=False,
444 cwd=None, env=None, universal_newlines=False,
445 startupinfo=None, creationflags=0):
446\end{verbatim}
Andrew M. Kuchlingc9e7d772004-10-12 15:58:02 +0000447
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +0000448\var{args} is commonly a sequence of strings that will be the
449arguments to the program executed as the subprocess. (If the
450\var{shell} argument is true, \var{args} can be a string which will
451then be passed on to the shell for interpretation, just as
452\function{os.system()} does.)
Andrew M. Kuchlingb6ffc272004-10-12 16:36:57 +0000453
454\var{stdin}, \var{stdout}, and \var{stderr} specify what the
455subprocess's input, output, and error streams will be. You can
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +0000456provide a file object or a file descriptor, or you can use the
457constant \code{subprocess.PIPE} to create a pipe between the
458subprocess and the parent.
Andrew M. Kuchlingb6ffc272004-10-12 16:36:57 +0000459
460The constructor has a number of handy options:
461
462\begin{itemize}
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +0000463 \item \var{close_fds} requests that all file descriptors be closed
464 before running the subprocess.
465
466 \item \var{cwd} specifies the working directory in which the
467 subprocess will be executed (defaulting to whatever the parent's
468 working directory is).
469
Andrew M. Kuchlingb6ffc272004-10-12 16:36:57 +0000470 \item \var{env} is a dictionary specifying environment variables.
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +0000471
472 \item \var{preexec_fn} is a function that gets called before the
473 child is started.
474
475 \item \var{universal_newlines} opens the child's input and output
476 using Python's universal newline feature.
477
Andrew M. Kuchlingb6ffc272004-10-12 16:36:57 +0000478\end{itemize}
479
480Once you've created the \class{Popen} instance,
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +0000481you can call its \method{wait()} method to pause until the subprocess
482has exited, \method{poll()} to check if it's exited without pausing,
483or \method{communicate(\var{data})} to send the string \var{data} to
Andrew M. Kuchlingb6ffc272004-10-12 16:36:57 +0000484the subprocess's standard input. \method{communicate(\var{data})}
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +0000485then reads any data that the subprocess has sent to its standard output
486or standard error, returning a tuple \code{(\var{stdout_data},
487\var{stderr_data})}.
Andrew M. Kuchlingb6ffc272004-10-12 16:36:57 +0000488
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +0000489\function{call()} is a shortcut that passes its arguments along to the
490\class{Popen} constructor, waits for the command to complete, and
491returns the status code of the subprocess. It can serve as a safer
492analog to \function{os.system()}:
Andrew M. Kuchlingb6ffc272004-10-12 16:36:57 +0000493
494\begin{verbatim}
495sts = subprocess.call(['dpkg', '-i', '/tmp/new-package.deb'])
496if sts == 0:
497 # Success
498 ...
499else:
500 # dpkg returned an error
501 ...
502\end{verbatim}
503
504The command is invoked without use of the shell. If you really do want to
505use the shell, you can add \code{shell=True} as a keyword argument and provide
506a string instead of a sequence:
507
508\begin{verbatim}
509sts = subprocess.call('dpkg -i /tmp/new-package.deb', shell=True)
510\end{verbatim}
511
512The PEP takes various examples of shell and Python code and shows how
513they'd be translated into Python code that uses \module{subprocess}.
514Reading this section of the PEP is highly recommended.
Andrew M. Kuchlingc9e7d772004-10-12 15:58:02 +0000515
516\begin{seealso}
Andrew M. Kuchlingb6ffc272004-10-12 16:36:57 +0000517\seepep{324}{subprocess - New process module}{Written and implemented by Peter {\AA}strand, with assistance from Fredrik Lundh and others.}
Andrew M. Kuchlingc9e7d772004-10-12 15:58:02 +0000518\end{seealso}
519
Andrew M. Kuchlingb6ffc272004-10-12 16:36:57 +0000520
Andrew M. Kuchlingc9e7d772004-10-12 15:58:02 +0000521%======================================================================
Raymond Hettinger0fff62f2004-07-01 11:52:15 +0000522\section{PEP 327: Decimal Data Type}
523
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +0000524Python has always supported floating-point (FP) numbers, based on the
525underlying C \ctype{double} type, as a data type. However, while most
Andrew M. Kuchling536183b2004-11-25 01:15:25 +0000526programming languages provide a floating-point type, many people (even
527programmers) are unaware that floating-point numbers don't represent
528certain decimal fractions accurately. The new \class{Decimal} type
529can represent these fractions accurately, up to a user-specified
530precision limit.
531
Raymond Hettinger0fff62f2004-07-01 11:52:15 +0000532
Andrew M. Kuchlingc8f8a812004-07-04 01:26:42 +0000533\subsection{Why is Decimal needed?}
Raymond Hettinger0fff62f2004-07-01 11:52:15 +0000534
Andrew M. Kuchlingc8f8a812004-07-04 01:26:42 +0000535The limitations arise from the representation used for floating-point numbers.
536FP numbers are made up of three components:
537
538\begin{itemize}
Andrew M. Kuchlinge34c3bd2004-08-31 12:21:44 +0000539\item The sign, which is positive or negative.
Andrew M. Kuchlingc8f8a812004-07-04 01:26:42 +0000540\item The mantissa, which is a single-digit binary number
541followed by a fractional part. For example, \code{1.01} in base-2 notation
542is \code{1 + 0/2 + 1/4}, or 1.25 in decimal notation.
543\item The exponent, which tells where the decimal point is located in the number represented.
544\end{itemize}
545
Andrew M. Kuchlinge34c3bd2004-08-31 12:21:44 +0000546For example, the number 1.25 has positive sign, a mantissa value of
5471.01 (in binary), and an exponent of 0 (the decimal point doesn't need
548to be shifted). The number 5 has the same sign and mantissa, but the
549exponent is 2 because the mantissa is multiplied by 4 (2 to the power
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +0000550of the exponent 2); 1.25 * 4 equals 5.
Andrew M. Kuchlingc8f8a812004-07-04 01:26:42 +0000551
552Modern systems usually provide floating-point support that conforms to
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +0000553a standard called IEEE 754. C's \ctype{double} type is usually
554implemented as a 64-bit IEEE 754 number, which uses 52 bits of space
555for the mantissa. This means that numbers can only be specified to 52
556bits of precision. If you're trying to represent numbers whose
Andrew M. Kuchlingc8f8a812004-07-04 01:26:42 +0000557expansion repeats endlessly, the expansion is cut off after 52 bits.
558Unfortunately, most software needs to produce output in base 10, and
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +0000559common fractions in base 10 are often repeating decimals in binary.
560For example, 1.1 decimal is binary \code{1.0001100110011 ...}; .1 =
5611/16 + 1/32 + 1/256 plus an infinite number of additional terms. IEEE
562754 has to chop off that infinitely repeated decimal after 52 digits,
563so the representation is slightly inaccurate.
Andrew M. Kuchlingc8f8a812004-07-04 01:26:42 +0000564
565Sometimes you can see this inaccuracy when the number is printed:
Raymond Hettinger0fff62f2004-07-01 11:52:15 +0000566\begin{verbatim}
Andrew M. Kuchlingc8f8a812004-07-04 01:26:42 +0000567>>> 1.1
5681.1000000000000001
Raymond Hettinger0fff62f2004-07-01 11:52:15 +0000569\end{verbatim}
570
Andrew M. Kuchlingc8f8a812004-07-04 01:26:42 +0000571The inaccuracy isn't always visible when you print the number because
Andrew M. Kuchlinge34c3bd2004-08-31 12:21:44 +0000572the FP-to-decimal-string conversion is provided by the C library, and
573most C libraries try to produce sensible output. Even if it's not
574displayed, however, the inaccuracy is still there and subsequent
575operations can magnify the error.
Raymond Hettinger0fff62f2004-07-01 11:52:15 +0000576
Andrew M. Kuchlingc8f8a812004-07-04 01:26:42 +0000577For many applications this doesn't matter. If I'm plotting points and
578displaying them on my monitor, the difference between 1.1 and
5791.1000000000000001 is too small to be visible. Reports often limit
580output to a certain number of decimal places, and if you round the
581number to two or three or even eight decimal places, the error is
582never apparent. However, for applications where it does matter,
583it's a lot of work to implement your own custom arithmetic routines.
584
Andrew M. Kuchlinge34c3bd2004-08-31 12:21:44 +0000585Hence, the \class{Decimal} type was created.
586
Andrew M. Kuchlingc8f8a812004-07-04 01:26:42 +0000587\subsection{The \class{Decimal} type}
588
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +0000589A new module, \module{decimal}, was added to Python's standard
590library. It contains two classes, \class{Decimal} and
591\class{Context}. \class{Decimal} instances represent numbers, and
592\class{Context} instances are used to wrap up various settings such as
593the precision and default rounding mode.
Andrew M. Kuchlingc8f8a812004-07-04 01:26:42 +0000594
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +0000595\class{Decimal} instances are immutable, like regular Python integers
596and FP numbers; once it's been created, you can't change the value an
597instance represents. \class{Decimal} instances can be created from
598integers or strings:
Raymond Hettinger0fff62f2004-07-01 11:52:15 +0000599
600\begin{verbatim}
Andrew M. Kuchlingc8f8a812004-07-04 01:26:42 +0000601>>> import decimal
602>>> decimal.Decimal(1972)
603Decimal("1972")
604>>> decimal.Decimal("1.1")
605Decimal("1.1")
Raymond Hettinger0fff62f2004-07-01 11:52:15 +0000606\end{verbatim}
607
Andrew M. Kuchlinge34c3bd2004-08-31 12:21:44 +0000608You can also provide tuples containing the sign, the mantissa represented
609as a tuple of decimal digits, and the exponent:
Raymond Hettinger0fff62f2004-07-01 11:52:15 +0000610
611\begin{verbatim}
Andrew M. Kuchlingc8f8a812004-07-04 01:26:42 +0000612>>> decimal.Decimal((1, (1, 4, 7, 5), -2))
613Decimal("-14.75")
Raymond Hettinger0fff62f2004-07-01 11:52:15 +0000614\end{verbatim}
Andrew M. Kuchlingc8f8a812004-07-04 01:26:42 +0000615
Andrew M. Kuchlinge34c3bd2004-08-31 12:21:44 +0000616Cautionary note: the sign bit is a Boolean value, so 0 is positive and
6171 is negative.
Andrew M. Kuchlingc8f8a812004-07-04 01:26:42 +0000618
Andrew M. Kuchlinge34c3bd2004-08-31 12:21:44 +0000619Converting from floating-point numbers poses a bit of a problem:
620should the FP number representing 1.1 turn into the decimal number for
621exactly 1.1, or for 1.1 plus whatever inaccuracies are introduced?
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +0000622The decision was to dodge the issue and leave such a conversion out of
623the API. Instead, you should convert the floating-point number into a
624string using the desired precision and pass the string to the
625\class{Decimal} constructor:
Andrew M. Kuchlingc8f8a812004-07-04 01:26:42 +0000626
627\begin{verbatim}
628>>> f = 1.1
629>>> decimal.Decimal(str(f))
630Decimal("1.1")
Andrew M. Kuchling0ad20f12004-07-21 13:00:06 +0000631>>> decimal.Decimal('%.12f' % f)
632Decimal("1.100000000000")
Andrew M. Kuchlingc8f8a812004-07-04 01:26:42 +0000633\end{verbatim}
634
635Once you have \class{Decimal} instances, you can perform the usual
636mathematical operations on them. One limitation: exponentiation
637requires an integer exponent:
638
639\begin{verbatim}
640>>> a = decimal.Decimal('35.72')
641>>> b = decimal.Decimal('1.73')
642>>> a+b
643Decimal("37.45")
644>>> a-b
645Decimal("33.99")
646>>> a*b
647Decimal("61.7956")
648>>> a/b
Andrew M. Kuchling0ad20f12004-07-21 13:00:06 +0000649Decimal("20.64739884393063583815028902")
Andrew M. Kuchlingc8f8a812004-07-04 01:26:42 +0000650>>> a ** 2
651Decimal("1275.9184")
Andrew M. Kuchling0ad20f12004-07-21 13:00:06 +0000652>>> a**b
653Traceback (most recent call last):
654 ...
655decimal.InvalidOperation: x ** (non-integer)
Andrew M. Kuchlingc8f8a812004-07-04 01:26:42 +0000656\end{verbatim}
657
658You can combine \class{Decimal} instances with integers, but not with
659floating-point numbers:
660
661\begin{verbatim}
662>>> a + 4
663Decimal("39.72")
664>>> a + 4.5
665Traceback (most recent call last):
666 ...
667TypeError: You can interact Decimal only with int, long or Decimal data types.
668>>>
669\end{verbatim}
670
671\class{Decimal} numbers can be used with the \module{math} and
Andrew M. Kuchling0ad20f12004-07-21 13:00:06 +0000672\module{cmath} modules, but note that they'll be immediately converted to
673floating-point numbers before the operation is performed, resulting in
674a possible loss of precision and accuracy. You'll also get back a
675regular floating-point number and not a \class{Decimal}.
Andrew M. Kuchlingc8f8a812004-07-04 01:26:42 +0000676
677\begin{verbatim}
678>>> import math, cmath
679>>> d = decimal.Decimal('123456789012.345')
680>>> math.sqrt(d)
681351364.18288201344
682>>> cmath.sqrt(-d)
683351364.18288201344j
Andrew M. Kuchling0ad20f12004-07-21 13:00:06 +0000684\end{verbatim}
685
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +0000686\class{Decimal} instances have a \method{sqrt()} method that
687returns a \class{Decimal}, but if you need other things such as
688trigonometric functions you'll have to implement them.
Andrew M. Kuchling0ad20f12004-07-21 13:00:06 +0000689
690\begin{verbatim}
Andrew M. Kuchlingc8f8a812004-07-04 01:26:42 +0000691>>> d.sqrt()
Raymond Hettingerca1a7752004-07-12 13:00:45 +0000692Decimal("351364.1828820134592177245001")
Andrew M. Kuchlingc8f8a812004-07-04 01:26:42 +0000693\end{verbatim}
694
695
696\subsection{The \class{Context} type}
697
698Instances of the \class{Context} class encapsulate several settings for
699decimal operations:
700
701\begin{itemize}
702 \item \member{prec} is the precision, the number of decimal places.
703 \item \member{rounding} specifies the rounding mode. The \module{decimal}
704 module has constants for the various possibilities:
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +0000705 \constant{ROUND_DOWN}, \constant{ROUND_CEILING},
706 \constant{ROUND_HALF_EVEN}, and various others.
Andrew M. Kuchling0ad20f12004-07-21 13:00:06 +0000707 \item \member{traps} is a dictionary specifying what happens on
Andrew M. Kuchlingc8f8a812004-07-04 01:26:42 +0000708encountering certain error conditions: either an exception is raised or
709a value is returned. Some examples of error conditions are
710division by zero, loss of precision, and overflow.
711\end{itemize}
712
713There's a thread-local default context available by calling
714\function{getcontext()}; you can change the properties of this context
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +0000715to alter the default precision, rounding, or trap handling. The
716following example shows the effect of changing the precision of the default
717context:
Andrew M. Kuchlingc8f8a812004-07-04 01:26:42 +0000718
719\begin{verbatim}
720>>> decimal.getcontext().prec
72128
722>>> decimal.Decimal(1) / decimal.Decimal(7)
Raymond Hettingerca1a7752004-07-12 13:00:45 +0000723Decimal("0.1428571428571428571428571429")
Andrew M. Kuchlingc8f8a812004-07-04 01:26:42 +0000724>>> decimal.getcontext().prec = 9
725>>> decimal.Decimal(1) / decimal.Decimal(7)
Raymond Hettingerca1a7752004-07-12 13:00:45 +0000726Decimal("0.142857143")
Andrew M. Kuchlingc8f8a812004-07-04 01:26:42 +0000727\end{verbatim}
728
Andrew M. Kuchling0ad20f12004-07-21 13:00:06 +0000729The default action for error conditions is selectable; the module can
730either return a special value such as infinity or not-a-number, or
731exceptions can be raised:
Andrew M. Kuchlingc8f8a812004-07-04 01:26:42 +0000732
733\begin{verbatim}
734>>> decimal.Decimal(1) / decimal.Decimal(0)
Andrew M. Kuchlingc8f8a812004-07-04 01:26:42 +0000735Traceback (most recent call last):
736 ...
737decimal.DivisionByZero: x / 0
Andrew M. Kuchling0ad20f12004-07-21 13:00:06 +0000738>>> decimal.getcontext().traps[decimal.DivisionByZero] = False
739>>> decimal.Decimal(1) / decimal.Decimal(0)
740Decimal("Infinity")
Andrew M. Kuchlingc8f8a812004-07-04 01:26:42 +0000741>>>
742\end{verbatim}
743
744The \class{Context} instance also has various methods for formatting
745numbers such as \method{to_eng_string()} and \method{to_sci_string()}.
746
Andrew M. Kuchling0ad20f12004-07-21 13:00:06 +0000747For more information, see the documentation for the \module{decimal}
748module, which includes a quick-start tutorial and a reference.
749
Raymond Hettinger0fff62f2004-07-01 11:52:15 +0000750\begin{seealso}
751\seepep{327}{Decimal Data Type}{Written by Facundo Batista and implemented
Andrew M. Kuchlingc8f8a812004-07-04 01:26:42 +0000752 by Facundo Batista, Eric Price, Raymond Hettinger, Aahz, and Tim Peters.}
753
Raymond Hettingerca1a7752004-07-12 13:00:45 +0000754\seeurl{http://research.microsoft.com/\textasciitilde hollasch/cgindex/coding/ieeefloat.html}
Andrew M. Kuchlingc8f8a812004-07-04 01:26:42 +0000755{A more detailed overview of the IEEE-754 representation.}
756
757\seeurl{http://www.lahey.com/float.htm}
758{The article uses Fortran code to illustrate many of the problems
759that floating-point inaccuracy can cause.}
760
761\seeurl{http://www2.hursley.ibm.com/decimal/}
762{A description of a decimal-based representation. This representation
763is being proposed as a standard, and underlies the new Python decimal
764type. Much of this material was written by Mike Cowlishaw, designer of the
Raymond Hettingerca1a7752004-07-12 13:00:45 +0000765Rexx language.}
Andrew M. Kuchlingc8f8a812004-07-04 01:26:42 +0000766
Raymond Hettinger0fff62f2004-07-01 11:52:15 +0000767\end{seealso}
768
769
770%======================================================================
Andrew M. Kuchling3294e9d2004-08-31 11:26:23 +0000771\section{PEP 328: Multi-line Imports}
772
773One language change is a small syntactic tweak aimed at making it
774easier to import many names from a module. In a
775\code{from \var{module} import \var{names}} statement,
776\var{names} is a sequence of names separated by commas. If the sequence is
777very long, you can either write multiple imports from the same module,
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +0000778or you can use backslashes to escape the line endings like this:
Andrew M. Kuchling3294e9d2004-08-31 11:26:23 +0000779
780\begin{verbatim}
781from SimpleXMLRPCServer import SimpleXMLRPCServer,\
782 SimpleXMLRPCRequestHandler,\
783 CGIXMLRPCRequestHandler,\
784 resolve_dotted_attribute
785\end{verbatim}
786
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +0000787The syntactic change in Python 2.4 simply allows putting the names
788within parentheses. Python ignores newlines within a parenthesized
Andrew M. Kuchling3294e9d2004-08-31 11:26:23 +0000789expression, so the backslashes are no longer needed:
790
791\begin{verbatim}
792from SimpleXMLRPCServer import (SimpleXMLRPCServer,
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +0000793 SimpleXMLRPCRequestHandler,
794 CGIXMLRPCRequestHandler,
795 resolve_dotted_attribute)
Andrew M. Kuchling3294e9d2004-08-31 11:26:23 +0000796\end{verbatim}
797
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +0000798The PEP also proposes that all \keyword{import} statements be absolute
799imports, with a leading \samp{.} character to indicate a relative
800import. This part of the PEP is not yet implemented, and will have to
801wait for Python 2.5 or some other future version.
Andrew M. Kuchling3294e9d2004-08-31 11:26:23 +0000802
803\begin{seealso}
Fred Drake410eb842004-09-01 04:05:08 +0000804\seepep{328}{Imports: Multi-Line and Absolute/Relative}
805 {Written by Aahz. Multi-line imports were implemented by
806 Dima Dorfman.}
807\end{seealso}
Andrew M. Kuchling3294e9d2004-08-31 11:26:23 +0000808
809
810%======================================================================
Andrew M. Kuchling65a33322004-07-21 12:41:38 +0000811\section{PEP 331: Locale-Independent Float/String Conversions}
812
813The \module{locale} modules lets Python software select various
814conversions and display conventions that are localized to a particular
815country or language. However, the module was careful to not change
816the numeric locale because various functions in Python's
817implementation required that the numeric locale remain set to the
818\code{'C'} locale. Often this was because the code was using the C library's
819\cfunction{atof()} function.
820
821Not setting the numeric locale caused trouble for extensions that used
822third-party C libraries, however, because they wouldn't have the
823correct locale set. The motivating example was GTK+, whose user
824interface widgets weren't displaying numbers in the current locale.
825
826The solution described in the PEP is to add three new functions to the
827Python API that perform ASCII-only conversions, ignoring the locale
828setting:
829
830\begin{itemize}
831 \item \cfunction{PyOS_ascii_strtod(\var{str}, \var{ptr})}
832and \cfunction{PyOS_ascii_atof(\var{str}, \var{ptr})}
833both convert a string to a C \ctype{double}.
834 \item \cfunction{PyOS_ascii_formatd(\var{buffer}, \var{buf_len}, \var{format}, \var{d})} converts a \ctype{double} to an ASCII string.
835\end{itemize}
836
837The code for these functions came from the GLib library
838(\url{http://developer.gnome.org/arch/gtk/glib.html}), whose
839developers kindly relicensed the relevant functions and donated them
840to the Python Software Foundation. The \module{locale} module
841can now change the numeric locale, letting extensions such as GTK+
842produce the correct results.
843
844\begin{seealso}
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +0000845\seepep{331}{Locale-Independent Float/String Conversions}
846{Written by Christian R. Reis, and implemented by Gustavo Carneiro.}
Andrew M. Kuchling65a33322004-07-21 12:41:38 +0000847\end{seealso}
848
849%======================================================================
Fred Drakeed0fa3d2003-07-30 19:14:09 +0000850\section{Other Language Changes}
851
852Here are all of the changes that Python 2.4 makes to the core Python
853language.
854
855\begin{itemize}
Raymond Hettingerd4462302003-11-26 17:52:45 +0000856
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +0000857\item Decorators for functions and methods were added (\pep{318}).
858
859\item Built-in \function{set} and \function{frozenset} types were
860added (\pep{218}). Other new built-ins include the \function{reversed(\var{seq})} function (\pep{322}).
861
862\item Generator expressions were added (\pep{289}).
863
864\item Certain numeric expressions no longer return values restricted to 32 or 64 bits (\pep{237}).
865
866\item You can now put parentheses around the list of names in a
867\code{from \var{module} import \var{names}} statement (\pep{328}).
868
Raymond Hettinger31017ae2004-03-04 08:25:44 +0000869\item The \method{dict.update()} method now accepts the same
870argument forms as the \class{dict} constructor. This includes any
Andrew M. Kuchling7642f7a2004-09-13 15:06:50 +0000871mapping, any iterable of key/value pairs, and keyword arguments.
872(Contributed by Raymond Hettinger.)
Raymond Hettinger31017ae2004-03-04 08:25:44 +0000873
Andrew M. Kuchling3bf85f12004-07-05 01:37:07 +0000874\item The string methods \method{ljust()}, \method{rjust()}, and
Andrew M. Kuchling67087562003-11-26 18:03:48 +0000875\method{center()} now take an optional argument for specifying a
Raymond Hettingerd4462302003-11-26 17:52:45 +0000876fill character other than a space.
Andrew M. Kuchling7642f7a2004-09-13 15:06:50 +0000877(Contributed by Raymond Hettinger.)
Raymond Hettingerd4462302003-11-26 17:52:45 +0000878
Andrew M. Kuchling35f2b052003-12-18 13:28:13 +0000879\item Strings also gained an \method{rsplit()} method that
Raymond Hettingered54d912003-12-31 01:59:18 +0000880works like the \method{split()} method but splits from the end of
Andrew M. Kuchling44a31e12004-01-01 18:33:34 +0000881the string.
Andrew M. Kuchlingb69c49c2004-12-01 00:42:41 +0000882(Contributed by Sean Reifschneider.)
Andrew M. Kuchling35f2b052003-12-18 13:28:13 +0000883
884\begin{verbatim}
Raymond Hettinger7a6d2972004-02-13 19:00:07 +0000885>>> 'www.python.org'.split('.', 1)
886['www', 'python.org']
887'www.python.org'.rsplit('.', 1)
888['www.python', 'org']
889\end{verbatim}
Raymond Hettinger97ef8de2004-01-05 00:29:57 +0000890
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +0000891\item Three keyword parameters, \var{cmp}, \var{key}, and
892\var{reverse}, were added to the \method{sort()} method of lists.
893These parameters make some common usages of \method{sort()} simpler.
894All of these parameters are optional.
Andrew M. Kuchling2fb4d512003-10-21 12:31:16 +0000895
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +0000896For the \var{cmp} parameter, the value should be a comparison function
897that takes two parameters and returns -1, 0, or +1 depending on how
898the parameters compare. This function will then be used to sort the
899list. Previously this was the only parameter that could be provided
900to \method{sort()}.
Andrew M. Kuchling2fb4d512003-10-21 12:31:16 +0000901
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +0000902\var{key} should be a single-parameter function that takes a list
Andrew M. Kuchling2fb4d512003-10-21 12:31:16 +0000903element and returns a comparison key for the element. The list is
Raymond Hettinger607c00f2003-11-12 16:27:50 +0000904then sorted using the comparison keys. The following example sorts a
905list case-insensitively:
Andrew M. Kuchling2fb4d512003-10-21 12:31:16 +0000906
907\begin{verbatim}
908>>> L = ['A', 'b', 'c', 'D']
909>>> L.sort() # Case-sensitive sort
910>>> L
911['A', 'D', 'b', 'c']
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +0000912>>> # Using 'key' parameter to sort list
Andrew M. Kuchling2fb4d512003-10-21 12:31:16 +0000913>>> L.sort(key=lambda x: x.lower())
914>>> L
915['A', 'b', 'c', 'D']
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +0000916>>> # Old-fashioned way
Andrew M. Kuchling2fb4d512003-10-21 12:31:16 +0000917>>> L.sort(cmp=lambda x,y: cmp(x.lower(), y.lower()))
918>>> L
919['A', 'b', 'c', 'D']
920\end{verbatim}
921
922The last example, which uses the \var{cmp} parameter, is the old way
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +0000923to perform a case-insensitive sort. It works but is slower than using
924a \var{key} parameter. Using \var{key} calls \method{lower()} method
925once for each element in the list while using \var{cmp} will call it
926twice for each comparison, so using \var{key} saves on invocations of
927the \method{lower()} method.
Andrew M. Kuchling2fb4d512003-10-21 12:31:16 +0000928
Andrew M. Kuchling981a9182003-11-13 21:33:26 +0000929For simple key functions and comparison functions, it is often
930possible to avoid a \keyword{lambda} expression by using an unbound
Raymond Hettinger607c00f2003-11-12 16:27:50 +0000931method instead. For example, the above case-insensitive sort is best
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +0000932written as:
Raymond Hettinger607c00f2003-11-12 16:27:50 +0000933
934\begin{verbatim}
935>>> L.sort(key=str.lower)
936>>> L
937['A', 'b', 'c', 'D']
938\end{verbatim}
939
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +0000940Finally, the \var{reverse} parameter takes a Boolean value. If the
941value is true, the list will be sorted into reverse order.
942Instead of \code{L.sort() ; L.reverse()}, you can now write
943\code{L.sort(reverse=True)}.
Fred Drakeed0fa3d2003-07-30 19:14:09 +0000944
Andrew M. Kuchling981a9182003-11-13 21:33:26 +0000945The results of sorting are now guaranteed to be stable. This means
946that two entries with equal keys will be returned in the same order as
947they were input. For example, you can sort a list of people by name,
948and then sort the list by age, resulting in a list sorted by age where
949people with the same age are in name-sorted order.
Raymond Hettinger607c00f2003-11-12 16:27:50 +0000950
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +0000951(All changes to \method{sort()} contributed by Raymond Hettinger.)
952
Fred Drake56fcc232004-05-06 02:55:35 +0000953\item There is a new built-in function
954\function{sorted(\var{iterable})} that works like the in-place
Andrew M. Kuchling3bf85f12004-07-05 01:37:07 +0000955\method{list.sort()} method but can be used in
Fred Drake56fcc232004-05-06 02:55:35 +0000956expressions. The differences are:
Raymond Hettinger607c00f2003-11-12 16:27:50 +0000957 \begin{itemize}
Raymond Hettinger7d1dd042003-11-12 16:42:10 +0000958 \item the input may be any iterable;
959 \item a newly formed copy is sorted, leaving the original intact; and
Raymond Hettinger607c00f2003-11-12 16:27:50 +0000960 \item the expression returns the new sorted copy
961 \end{itemize}
Andrew M. Kuchling1a420252003-11-08 15:58:49 +0000962
963\begin{verbatim}
964>>> L = [9,7,8,3,2,4,1,6,5]
Raymond Hettinger64958a12003-12-17 20:43:33 +0000965>>> [10+i for i in sorted(L)] # usable in a list comprehension
Raymond Hettinger607c00f2003-11-12 16:27:50 +0000966[11, 12, 13, 14, 15, 16, 17, 18, 19]
Hye-Shik Chang2b052482004-07-17 13:53:48 +0000967>>> L # original is left unchanged
Andrew M. Kuchlinge3e1eca2004-07-26 18:52:48 +0000968[9,7,8,3,2,4,1,6,5]
969>>> sorted('Monty Python') # any iterable may be an input
970[' ', 'M', 'P', 'h', 'n', 'n', 'o', 'o', 't', 't', 'y', 'y']
Raymond Hettingerd4462302003-11-26 17:52:45 +0000971
972>>> # List the contents of a dict sorted by key values
Raymond Hettinger607c00f2003-11-12 16:27:50 +0000973>>> colormap = dict(red=1, blue=2, green=3, black=4, yellow=5)
Raymond Hettinger64958a12003-12-17 20:43:33 +0000974>>> for k, v in sorted(colormap.iteritems()):
Raymond Hettinger607c00f2003-11-12 16:27:50 +0000975... print k, v
976...
977black 4
978blue 2
979green 3
980red 1
981yellow 5
Andrew M. Kuchling1a420252003-11-08 15:58:49 +0000982\end{verbatim}
983
Andrew M. Kuchling7642f7a2004-09-13 15:06:50 +0000984(Contributed by Raymond Hettinger.)
985
Andrew M. Kuchling87c98b22004-08-25 13:38:46 +0000986\item Integer operations will no longer trigger an \exception{OverflowWarning}.
987The \exception{OverflowWarning} warning will disappear in Python 2.5.
988
Andrew M. Kuchling5e3f9232004-10-07 12:00:33 +0000989\item The interpreter gained a new switch, \programopt{-m}, that
990takes a name, searches for the corresponding module on \code{sys.path},
991and runs the module as a script. For example,
992you can now run the Python profiler with \code{python -m profile}.
993(Contributed by Nick Coghlan.)
994
Andrew M. Kuchlingd0b6d9d2004-07-04 15:35:00 +0000995\item The \function{eval(\var{expr}, \var{globals}, \var{locals})}
Andrew M. Kuchling1455f792004-08-02 12:09:58 +0000996and \function{execfile(\var{filename}, \var{globals}, \var{locals})}
997functions and the \keyword{exec} statement now accept any mapping type
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +0000998for the \var{locals} parameter. Previously this had to be a regular
Andrew M. Kuchling1455f792004-08-02 12:09:58 +0000999Python dictionary. (Contributed by Raymond Hettinger.)
Andrew M. Kuchlingd0b6d9d2004-07-04 15:35:00 +00001000
Raymond Hettinger607c00f2003-11-12 16:27:50 +00001001\item The \function{zip()} built-in function and \function{itertools.izip()}
Andrew M. Kuchling3bf85f12004-07-05 01:37:07 +00001002 now return an empty list if called with no arguments.
1003 Previously they raised a \exception{TypeError}
1004 exception. This makes them more
Raymond Hettinger607c00f2003-11-12 16:27:50 +00001005 suitable for use with variable length argument lists:
1006
1007\begin{verbatim}
1008>>> def transpose(array):
1009... return zip(*array)
1010...
1011>>> transpose([(1,2,3), (4,5,6)])
1012[(1, 4), (2, 5), (3, 6)]
1013>>> transpose([])
1014[]
1015\end{verbatim}
Andrew M. Kuchling7642f7a2004-09-13 15:06:50 +00001016(Contributed by Raymond Hettinger.)
1017
Andrew M. Kuchlingd91fcbe2004-08-02 12:44:28 +00001018\item Encountering a failure while importing a module no longer leaves
1019a partially-initialized module object in \code{sys.modules}. The
1020incomplete module object left behind would fool further imports of the
1021same module into succeeding, leading to confusing errors.
Andrew M. Kuchling067947e2004-11-19 14:43:36 +00001022(Fixed by Tim Peters.)
Andrew M. Kuchlingd91fcbe2004-08-02 12:44:28 +00001023
Andrew M. Kuchling65a33322004-07-21 12:41:38 +00001024\item \constant{None} is now a constant; code that binds a new value to
1025the name \samp{None} is now a syntax error.
Andrew M. Kuchling7642f7a2004-09-13 15:06:50 +00001026(Contributed by Raymond Hettinger.)
Andrew M. Kuchling65a33322004-07-21 12:41:38 +00001027
Fred Drakeed0fa3d2003-07-30 19:14:09 +00001028\end{itemize}
1029
1030
1031%======================================================================
1032\subsection{Optimizations}
1033
1034\begin{itemize}
1035
Raymond Hettingerca1a7752004-07-12 13:00:45 +00001036\item The inner loops for list and tuple slicing
Andrew M. Kuchling65a33322004-07-21 12:41:38 +00001037 were optimized and now run about one-third faster. The inner loops
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +00001038 for dictionaries were also optimized , resulting in performance boosts for
Andrew M. Kuchling65a33322004-07-21 12:41:38 +00001039 \method{keys()}, \method{values()}, \method{items()},
1040 \method{iterkeys()}, \method{itervalues()}, and \method{iteritems()}.
Andrew M. Kuchling7642f7a2004-09-13 15:06:50 +00001041 (Contributed by Raymond Hettinger.)
Raymond Hettingerb7d05db2004-03-08 07:25:05 +00001042
Andrew M. Kuchling3bf85f12004-07-05 01:37:07 +00001043\item The machinery for growing and shrinking lists was optimized for
1044 speed and for space efficiency. Appending and popping from lists now
1045 runs faster due to more efficient code paths and less frequent use of
1046 the underlying system \cfunction{realloc()}. List comprehensions
1047 also benefit. \method{list.extend()} was also optimized and no
1048 longer converts its argument into a temporary list before extending
Andrew M. Kuchling7642f7a2004-09-13 15:06:50 +00001049 the base list. (Contributed by Raymond Hettinger.)
Raymond Hettinger7a6d2972004-02-13 19:00:07 +00001050
Raymond Hettinger97ef8de2004-01-05 00:29:57 +00001051\item \function{list()}, \function{tuple()}, \function{map()},
1052 \function{filter()}, and \function{zip()} now run several times
1053 faster with non-sequence arguments that supply a \method{__len__()}
Andrew M. Kuchling7642f7a2004-09-13 15:06:50 +00001054 method. (Contributed by Raymond Hettinger.)
Raymond Hettinger97ef8de2004-01-05 00:29:57 +00001055
Raymond Hettinger23a0f4e2004-01-05 08:15:20 +00001056\item The methods \method{list.__getitem__()},
Raymond Hettinger97ef8de2004-01-05 00:29:57 +00001057 \method{dict.__getitem__()}, and \method{dict.__contains__()} are
1058 are now implemented as \class{method_descriptor} objects rather
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +00001059 than \class{wrapper_descriptor} objects. This form of
Raymond Hettinger97ef8de2004-01-05 00:29:57 +00001060 access doubles their performance and makes them more suitable for
Raymond Hettinger23a0f4e2004-01-05 08:15:20 +00001061 use as arguments to functionals:
1062 \samp{map(mydict.__getitem__, keylist)}.
Andrew M. Kuchling7642f7a2004-09-13 15:06:50 +00001063 (Contributed by Raymond Hettinger.)
Fred Drakeed0fa3d2003-07-30 19:14:09 +00001064
Fred Draked6d35d92004-06-03 13:31:22 +00001065\item Added a new opcode, \code{LIST_APPEND}, that simplifies
Raymond Hettingerdd80f762004-03-07 07:31:06 +00001066 the generated bytecode for list comprehensions and speeds them up
Andrew M. Kuchling7642f7a2004-09-13 15:06:50 +00001067 by about a third. (Contributed by Raymond Hettinger.)
Raymond Hettingerdd80f762004-03-07 07:31:06 +00001068
Andrew M. Kuchling0c789562004-09-23 20:15:41 +00001069\item The peephole bytecode optimizer has been improved to
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +00001070produce shorter, faster bytecode; remarkably, the resulting bytecode is
Andrew M. Kuchling0c789562004-09-23 20:15:41 +00001071more readable. (Enhanced by Raymond Hettinger.)
1072
Andrew M. Kuchlingac642872004-08-07 13:13:31 +00001073\item String concatenations in statements of the form \code{s = s +
1074"abc"} and \code{s += "abc"} are now performed more efficiently in
1075certain circumstances. This optimization won't be present in other
1076Python implementations such as Jython, so you shouldn't rely on it;
1077using the \method{join()} method of strings is still recommended when
1078you want to efficiently glue a large number of strings together.
Andrew M. Kuchling7642f7a2004-09-13 15:06:50 +00001079(Contributed by Armin Rigo.)
Andrew M. Kuchlingac642872004-08-07 13:13:31 +00001080
Fred Drakeed0fa3d2003-07-30 19:14:09 +00001081\end{itemize}
1082
Raymond Hettingerb2d5a8e2004-11-18 05:51:53 +00001083% pystone is almost useless for comparing different versions of Python;
1084% instead, it excels at predicting relative Python performance on
1085% different machines.
1086% So, this section would be more informative if it used other tools
1087% such as pybench and parrotbench. For a more application oriented
1088% benchmark, try comparing the timings of test_decimal.py under 2.3
1089% and 2.4.
1090
Fred Drakeed0fa3d2003-07-30 19:14:09 +00001091The net result of the 2.4 optimizations is that Python 2.4 runs the
Andrew M. Kuchlingbae8f372004-11-19 14:55:28 +00001092pystone benchmark around 5\% faster than Python 2.3 and 35\% faster
1093than Python 2.2. (pystone is not a particularly good benchmark, but
1094it's the most commonly used measurement of Python's performance. Your
1095own applications may show greater or smaller benefits from Python~2.4.)
Fred Drakeed0fa3d2003-07-30 19:14:09 +00001096
1097
1098%======================================================================
1099\section{New, Improved, and Deprecated Modules}
1100
1101As usual, Python's standard library received a number of enhancements and
1102bug fixes. Here's a partial list of the most notable changes, sorted
1103alphabetically by module name. Consult the
1104\file{Misc/NEWS} file in the source tree for a more
1105complete list of changes, or look through the CVS logs for all the
1106details.
1107
1108\begin{itemize}
1109
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +00001110\item The \module{asyncore} module's \function{loop()} function now
1111 has a \var{count} parameter that lets you perform a limited number
Andrew M. Kuchlingd0b6d9d2004-07-04 15:35:00 +00001112 of passes through the polling loop. The default is still to loop
1113 forever.
1114
Andrew M. Kuchlinga331e862004-09-10 13:05:22 +00001115\item The \module{base64} module now has more complete RFC 3548 support
1116 for Base64, Base32, and Base16 encoding and decoding, including
1117 optional case folding and optional alternative alphabets.
1118 (Contributed by Barry Warsaw.)
Andrew M. Kuchling6aedcfc2003-10-21 12:48:23 +00001119
Raymond Hettinger0c410272004-01-05 10:13:35 +00001120\item The \module{bisect} module now has an underlying C implementation
1121 for improved performance.
1122 (Contributed by Dmitry Vasiliev.)
1123
Andrew M. Kuchling5303a962004-01-18 15:55:51 +00001124\item The CJKCodecs collections of East Asian codecs, maintained
1125by Hye-Shik Chang, was integrated into 2.4.
1126The new encodings are:
1127
1128\begin{itemize}
Andrew M. Kuchling671c5062004-07-28 15:29:39 +00001129 \item Chinese (PRC): gb2312, gbk, gb18030, big5hkscs, hz
Andrew M. Kuchling5303a962004-01-18 15:55:51 +00001130 \item Chinese (ROC): big5, cp950
Andrew M. Kuchling671c5062004-07-28 15:29:39 +00001131 \item Japanese: cp932, euc-jis-2004, euc-jp,
Andrew M. Kuchling5303a962004-01-18 15:55:51 +00001132euc-jisx0213, iso-2022-jp, iso-2022-jp-1, iso-2022-jp-2,
Andrew M. Kuchling671c5062004-07-28 15:29:39 +00001133 iso-2022-jp-3, iso-2022-jp-ext, iso-2022-jp-2004,
1134 shift-jis, shift-jisx0213, shift-jis-2004
Andrew M. Kuchling5303a962004-01-18 15:55:51 +00001135 \item Korean: cp949, euc-kr, johab, iso-2022-kr
1136\end{itemize}
1137
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +00001138\item Some other new encodings were added: HP Roman8,
1139ISO_8859-11, ISO_8859-16, PCTP-154, and TIS-620.
1140
Andrew M. Kuchling579b3e22004-10-05 20:23:34 +00001141\item The UTF-8 and UTF-16 codecs now cope better with receiving partial input.
1142Previously the \class{StreamReader} class would try to read more data,
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +00001143making it impossible to resume decoding from the stream. The
Andrew M. Kuchling579b3e22004-10-05 20:23:34 +00001144\method{read()} method will now return as much data as it can and future
1145calls will resume decoding where previous ones left off.
1146(Implemented by Walter D\"orwald.)
1147
Andrew M. Kuchlingfd0e4942004-02-09 13:23:34 +00001148\item There is a new \module{collections} module for
1149 various specialized collection datatypes.
1150 Currently it contains just one type, \class{deque},
1151 a double-ended queue that supports efficiently adding and removing
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +00001152 elements from either end:
Raymond Hettinger756b3f32004-01-29 06:37:52 +00001153
1154\begin{verbatim}
1155>>> from collections import deque
1156>>> d = deque('ghi') # make a new deque with three items
1157>>> d.append('j') # add a new entry to the right side
1158>>> d.appendleft('f') # add a new entry to the left side
1159>>> d # show the representation of the deque
1160deque(['f', 'g', 'h', 'i', 'j'])
1161>>> d.pop() # return and remove the rightmost item
1162'j'
1163>>> d.popleft() # return and remove the leftmost item
1164'f'
1165>>> list(d) # list the contents of the deque
1166['g', 'h', 'i']
1167>>> 'h' in d # search the deque
1168True
1169\end{verbatim}
1170
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +00001171Several modules, such as the \module{Queue} and \module{threading}
1172modules, now take advantage of \class{collections.deque} for improved
1173performance. (Contributed by Raymond Hettinger.)
Andrew M. Kuchling5303a962004-01-18 15:55:51 +00001174
Fred Drake9f15b5c2004-05-18 04:30:00 +00001175\item The \module{ConfigParser} classes have been enhanced slightly.
1176 The \method{read()} method now returns a list of the files that
1177 were successfully parsed, and the \method{set()} method raises
1178 \exception{TypeError} if passed a \var{value} argument that isn't a
Andrew M. Kuchling067947e2004-11-19 14:43:36 +00001179 string. (Contributed by John Belmonte and David Goodger.)
Fred Drake9f15b5c2004-05-18 04:30:00 +00001180
Andrew M. Kuchlinga331e862004-09-10 13:05:22 +00001181\item The \module{curses} module now supports the ncurses extension
1182 \function{use_default_colors()}. On platforms where the terminal
1183 supports transparency, this makes it possible to use a transparent
1184 background. (Contributed by J\"org Lehmann.)
1185
1186\item The \module{difflib} module now includes an \class{HtmlDiff} class
1187that creates an HTML table showing a side by side comparison
1188of two versions of a text. (Contributed by Dan Gass.)
1189
Andrew M. Kuchling579b3e22004-10-05 20:23:34 +00001190\item The \module{email} package was updated to version 3.0,
1191which dropped various deprecated APIs and removes support for Python
1192versions earlier than 2.3. The 3.0 version of the package uses a new
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +00001193incremental parser for MIME messages, available in the
Andrew M. Kuchling579b3e22004-10-05 20:23:34 +00001194\module{email.FeedParser} module. The new parser doesn't require
1195reading the entire message into memory, and doesn't throw exceptions
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +00001196if a message is malformed; instead it records any problems in the
Andrew M. Kuchling579b3e22004-10-05 20:23:34 +00001197\member{defect} attribute of the message. (Developed by Anthony
1198Baxter, Barry Warsaw, Thomas Wouters, and others.)
Andrew M. Kuchlinga331e862004-09-10 13:05:22 +00001199
Raymond Hettinger607c00f2003-11-12 16:27:50 +00001200\item The \module{heapq} module has been converted to C. The resulting
Andrew M. Kuchlingfd0e4942004-02-09 13:23:34 +00001201 tenfold improvement in speed makes the module suitable for handling
Raymond Hettinger33ecffb2004-06-10 05:03:17 +00001202 high volumes of data. In addition, the module has two new functions
1203 \function{nlargest()} and \function{nsmallest()} that use heaps to
Andrew M. Kuchlingd0b6d9d2004-07-04 15:35:00 +00001204 find the N largest or smallest values in a dataset without the
Andrew M. Kuchling7642f7a2004-09-13 15:06:50 +00001205 expense of a full sort. (Contributed by Raymond Hettinger.)
Andrew M. Kuchling1a420252003-11-08 15:58:49 +00001206
Andrew M. Kuchling0c789562004-09-23 20:15:41 +00001207\item The \module{httplib} module now contains constants for HTTP
1208status codes defined in various HTTP-related RFC documents. Constants
1209have names such as \constant{OK}, \constant{CREATED},
1210\constant{CONTINUE}, and \constant{MOVED_PERMANENTLY}; use pydoc to
1211get a full list. (Contributed by Andrew Eland.)
1212
Andrew M. Kuchlingce4bae62004-07-27 12:13:25 +00001213\item The \module{imaplib} module now supports IMAP's THREAD command
1214(contributed by Yves Dionne) and new \method{deleteacl()} and
1215\method{myrights()} methods (contributed by Arnaud Mazin).
Andrew M. Kuchlingdff9dbd2003-11-20 22:22:19 +00001216
Andrew M. Kuchlingad809552003-12-06 23:19:23 +00001217\item The \module{itertools} module gained a
Andrew M. Kuchling3bf85f12004-07-05 01:37:07 +00001218 \function{groupby(\var{iterable}\optional{, \var{func}})} function.
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +00001219 \var{iterable} is something that can be iterated over to return a
1220 stream of elements, and the optional \var{func} parameter is a
1221 function that takes an element and returns a key value; if omitted,
1222 the key is simply the element itself. \function{groupby()} then
1223 groups the elements into subsequences which have matching values of
1224 the key, and returns a series of 2-tuples containing the key value
1225 and an iterator over the subsequence.
Andrew M. Kuchlingad809552003-12-06 23:19:23 +00001226
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +00001227Here's an example to make this clearer. The \var{key} function simply
1228returns whether a number is even or odd, so the result of
1229\function{groupby()} is to return consecutive runs of odd or even
1230numbers.
Andrew M. Kuchlingad809552003-12-06 23:19:23 +00001231
1232\begin{verbatim}
1233>>> import itertools
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +00001234>>> L = [2, 4, 6, 7, 8, 9, 11, 12, 14]
Andrew M. Kuchlingad809552003-12-06 23:19:23 +00001235>>> for key_val, it in itertools.groupby(L, lambda x: x % 2):
1236... print key_val, list(it)
1237...
12380 [2, 4, 6]
12391 [7]
12400 [8]
12411 [9, 11]
12420 [12, 14]
1243>>>
1244\end{verbatim}
1245
Andrew M. Kuchling3bf85f12004-07-05 01:37:07 +00001246\function{groupby()} is typically used with sorted input. The logic
1247for \function{groupby()} is similar to the \UNIX{} \code{uniq} filter
1248which makes it handy for eliminating, counting, or identifying
1249duplicate elements:
Raymond Hettingerfeb78c92003-12-12 13:13:47 +00001250
1251\begin{verbatim}
1252>>> word = 'abracadabra'
Raymond Hettingered54d912003-12-31 01:59:18 +00001253>>> letters = sorted(word) # Turn string into a sorted list of letters
Raymond Hettinger64958a12003-12-17 20:43:33 +00001254>>> letters
Andrew M. Kuchling4612bc52003-12-16 20:59:37 +00001255['a', 'a', 'a', 'a', 'a', 'b', 'b', 'c', 'd', 'r', 'r']
Andrew M. Kuchling3bf85f12004-07-05 01:37:07 +00001256>>> for k, g in itertools.groupby(letters):
1257... print k, list(g)
1258...
1259a ['a', 'a', 'a', 'a', 'a']
1260b ['b', 'b']
1261c ['c']
1262d ['d']
1263r ['r', 'r']
1264>>> # List unique letters
1265>>> [k for k, g in groupby(letters)]
Raymond Hettingerfeb78c92003-12-12 13:13:47 +00001266['a', 'b', 'c', 'd', 'r']
Johannes Gijsbersd3452252004-09-11 16:50:06 +00001267>>> # Count letter occurrences
Andrew M. Kuchling3bf85f12004-07-05 01:37:07 +00001268>>> [(k, len(list(g))) for k, g in groupby(letters)]
Raymond Hettingerfeb78c92003-12-12 13:13:47 +00001269[('a', 5), ('b', 2), ('c', 1), ('d', 1), ('r', 2)]
Raymond Hettingerfeb78c92003-12-12 13:13:47 +00001270\end{verbatim}
1271
Andrew M. Kuchling7642f7a2004-09-13 15:06:50 +00001272(Contributed by Hye-Shik Chang.)
1273
Raymond Hettingered54d912003-12-31 01:59:18 +00001274\item \module{itertools} also gained a function named
1275\function{tee(\var{iterator}, \var{N})} that returns \var{N} independent
1276iterators that replicate \var{iterator}. If \var{N} is omitted, the
1277default is 2.
Andrew M. Kuchling35f2b052003-12-18 13:28:13 +00001278
1279\begin{verbatim}
1280>>> L = [1,2,3]
1281>>> i1, i2 = itertools.tee(L)
1282>>> i1,i2
1283(<itertools.tee object at 0x402c2080>, <itertools.tee object at 0x402c2090>)
Raymond Hettingered54d912003-12-31 01:59:18 +00001284>>> list(i1) # Run the first iterator to exhaustion
Andrew M. Kuchling35f2b052003-12-18 13:28:13 +00001285[1, 2, 3]
Raymond Hettingered54d912003-12-31 01:59:18 +00001286>>> list(i2) # Run the second iterator to exhaustion
Andrew M. Kuchling35f2b052003-12-18 13:28:13 +00001287[1, 2, 3]
1288>\end{verbatim}
1289
1290Note that \function{tee()} has to keep copies of the values returned
Raymond Hettingered54d912003-12-31 01:59:18 +00001291by the iterator; in the worst case, it may need to keep all of them.
Andrew M. Kuchling44a31e12004-01-01 18:33:34 +00001292This should therefore be used carefully if the leading iterator
Raymond Hettingered54d912003-12-31 01:59:18 +00001293can run far ahead of the trailing iterator in a long stream of inputs.
Andrew M. Kuchling3bf85f12004-07-05 01:37:07 +00001294If the separation is large, then you might as well use
Raymond Hettingered54d912003-12-31 01:59:18 +00001295\function{list()} instead. When the iterators track closely with one
1296another, \function{tee()} is ideal. Possible applications include
1297bookmarking, windowing, or lookahead iterators.
Andrew M. Kuchling7642f7a2004-09-13 15:06:50 +00001298(Contributed by Raymond Hettinger.)
Andrew M. Kuchling35f2b052003-12-18 13:28:13 +00001299
Andrew M. Kuchling5785a132004-07-26 19:28:46 +00001300\item A number of functions were added to the \module{locale}
1301module, such as \function{bind_textdomain_codeset()} to specify a
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +00001302particular encoding and a family of \function{l*gettext()} functions
Andrew M. Kuchling5785a132004-07-26 19:28:46 +00001303that return messages in the chosen encoding.
1304(Contributed by Gustavo Niemeyer.)
1305
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +00001306\item Some keyword arguments were added to the \module{logging}
1307package's \function{basicConfig} function to simplify log
1308configuration. The default behavior is to log messages to standard
1309error, but various keyword arguments can be specified to log to a
1310particular file, change the logging format, or set the logging level.
1311For example:
Andrew M. Kuchlingbcefe692004-07-07 13:01:53 +00001312
1313\begin{verbatim}
1314import logging
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +00001315logging.basicConfig(filename='/var/log/application.log',
1316 level=0, # Log all messages
Andrew M. Kuchlingbcefe692004-07-07 13:01:53 +00001317 format='%(levelname):%(process):%(thread):%(message)')
1318\end{verbatim}
1319
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +00001320Other additions to the \module{logging} package include a
1321\method{log(\var{level}, \var{msg})} convenience method, as well as a
1322\class{TimedRotatingFileHandler} class that rotates its log files at a
1323timed interval. The module already had \class{RotatingFileHandler},
Andrew M. Kuchlingbcefe692004-07-07 13:01:53 +00001324which rotated logs once the file exceeded a certain size. Both
1325classes derive from a new \class{BaseRotatingHandler} class that can
1326be used to implement other rotating handlers.
1327
Andrew M. Kuchling579b3e22004-10-05 20:23:34 +00001328(Changes implemented by Vinay Sajip.)
1329
Andrew M. Kuchling0c789562004-09-23 20:15:41 +00001330\item The \module{marshal} module now shares interned strings on unpacking a
1331data structure. This may shrink the size of certain pickle strings,
1332but the primary effect is to make \file{.pyc} files significantly smaller.
1333(Contributed by Martin von Loewis.)
1334
Andrew M. Kuchling5785a132004-07-26 19:28:46 +00001335\item The \module{nntplib} module's \class{NNTP} class gained
1336\method{description()} and \method{descriptions()} methods to retrieve
1337newsgroup descriptions for a single group or for a range of groups.
1338(Contributed by J\"urgen A. Erhard.)
1339
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +00001340\item Two new functions were added to the \module{operator} module,
Andrew M. Kuchling35f2b052003-12-18 13:28:13 +00001341\function{attrgetter(\var{attr})} and \function{itemgetter(\var{index})}.
1342Both functions return callables that take a single argument and return
Raymond Hettingered54d912003-12-31 01:59:18 +00001343the corresponding attribute or item; these callables make excellent
Andrew M. Kuchlingbcefe692004-07-07 13:01:53 +00001344data extractors when used with \function{map()} or
1345\function{sorted()}. For example:
Andrew M. Kuchling35f2b052003-12-18 13:28:13 +00001346
1347\begin{verbatim}
Raymond Hettingered54d912003-12-31 01:59:18 +00001348>>> L = [('c', 2), ('d', 1), ('a', 4), ('b', 3)]
Andrew M. Kuchling35f2b052003-12-18 13:28:13 +00001349>>> map(operator.itemgetter(0), L)
1350['c', 'd', 'a', 'b']
1351>>> map(operator.itemgetter(1), L)
Raymond Hettingered54d912003-12-31 01:59:18 +00001352[2, 1, 4, 3]
1353>>> sorted(L, key=operator.itemgetter(1)) # Sort list by second tuple item
1354[('d', 1), ('c', 2), ('b', 3), ('a', 4)]
Andrew M. Kuchling35f2b052003-12-18 13:28:13 +00001355\end{verbatim}
1356
Andrew M. Kuchling7642f7a2004-09-13 15:06:50 +00001357(Contributed by Raymond Hettinger.)
1358
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +00001359\item The \module{optparse} module was updated in various ways. The
1360module now passes its messages through \function{gettext.gettext()},
1361making it possible to internationalize Optik's help and error
1362messages. Help messages for options can now include the string
1363\code{'\%default'}, which will be replaced by the option's default
1364value. (Contributed by Greg Ward.)
Andrew M. Kuchlinge30c4d42004-08-07 13:58:02 +00001365
Andrew M. Kuchlingf3958f12004-10-11 19:20:06 +00001366\item The long-term plan is to deprecate the \module{rfc822} module
1367in some future Python release in favor of the \module{email} package.
1368To this end, the \function{email.Utils.formatdate()} function has been
1369changed to make it usable as a replacement for
1370\function{rfc822.formatdate()}. You may want to write new e-mail
1371processing code with this in mind. (Change implemented by Anthony
1372Baxter.)
1373
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +00001374\item A new \function{urandom(\var{n})} function was added to the
1375\module{os} module, returning a string containing \var{n} bytes of
1376random data. This function provides access to platform-specific
1377sources of randomness such as \file{/dev/urandom} on Linux or the
1378Windows CryptoAPI. (Contributed by Trevor Perrin.)
Andrew M. Kuchlingcb7b3f32004-08-30 11:58:04 +00001379
1380\item Another new function: \function{os.path.lexists(\var{path})}
1381returns true if the file specified by \var{path} exists, whether or
1382not it's a symbolic link. This differs from the existing
1383\function{os.path.exists(\var{path})} function, which returns false if
1384\var{path} is a symlink that points to a destination that doesn't exist.
1385(Contributed by Beni Cherniavsky.)
1386
Andrew M. Kuchlingd0b6d9d2004-07-04 15:35:00 +00001387\item A new \function{getsid()} function was added to the
1388\module{posix} module that underlies the \module{os} module.
1389(Contributed by J. Raynor.)
1390
Andrew M. Kuchling067947e2004-11-19 14:43:36 +00001391\item The \module{poplib} module now supports POP over SSL. (Contributed by
1392Hector Urtubia.)
Andrew M. Kuchlingd0b6d9d2004-07-04 15:35:00 +00001393
1394\item The \module{profile} module can now profile C extension functions.
Andrew M. Kuchling7642f7a2004-09-13 15:06:50 +00001395(Contributed by Nick Bastin.)
Andrew M. Kuchlingd0b6d9d2004-07-04 15:35:00 +00001396
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +00001397\item The \module{random} module has a new method called
1398 \method{getrandbits(\var{N})} that returns a long integer \var{N}
1399 bits in length. The existing \method{randrange()} method now uses
1400 \method{getrandbits()} where appropriate, making generation of
1401 arbitrarily large random numbers more efficient. (Contributed by
1402 Raymond Hettinger.)
Andrew M. Kuchling6aedcfc2003-10-21 12:48:23 +00001403
1404\item The regular expression language accepted by the \module{re} module
1405 was extended with simple conditional expressions, written as
Andrew M. Kuchlingab778222004-08-31 12:07:43 +00001406 \regexp{(?(\var{group})\var{A}|\var{B})}. \var{group} is either a
1407 numeric group ID or a group name defined with \regexp{(?P<group>...)}
Andrew M. Kuchling6aedcfc2003-10-21 12:48:23 +00001408 earlier in the expression. If the specified group matched, the
1409 regular expression pattern \var{A} will be tested against the string; if
1410 the group didn't match, the pattern \var{B} will be used instead.
Andrew M. Kuchling067947e2004-11-19 14:43:36 +00001411 (Contributed by Gustavo Niemeyer.)
Raymond Hettinger874ebd52004-05-31 03:15:02 +00001412
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +00001413\item The \module{re} module is also no longer recursive, thanks to a
1414massive amount of work by Gustavo Niemeyer. In a recursive regular
1415expression engine, certain patterns result in a large amount of C
1416stack space being consumed, and it was possible to overflow the stack.
1417For example, if you matched a 30000-byte string of \samp{a} characters
1418against the expression \regexp{(a|b)+}, one stack frame was consumed
1419per character. Python 2.3 tried to check for stack overflow and raise
1420a \exception{RuntimeError} exception, but certain patterns could
1421sidestep the checking and if you were unlucky Python could segfault.
1422Python 2.4's regular expression engine can match this pattern without
1423problems.
Andrew M. Kuchlingab778222004-08-31 12:07:43 +00001424
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +00001425\item A new \function{socketpair()} function, returning a pair of
1426connected sockets, was added to the \module{socket} module.
Andrew M. Kuchling87c98b22004-08-25 13:38:46 +00001427(Contributed by Dave Cole.)
Andrew M. Kuchling7f203b82004-08-09 14:48:28 +00001428
Andrew M. Kuchling87c98b22004-08-25 13:38:46 +00001429\item The \function{sys.exitfunc()} function has been deprecated. Code
1430should be using the existing \module{atexit} module, which correctly
1431handles calling multiple exit functions. Eventually
1432\function{sys.exitfunc()} will become a purely internal interface,
1433accessed only by \module{atexit}.
1434
1435\item The \module{tarfile} module now generates GNU-format tar files
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +00001436by default. (Contributed by Lars Gustaebel.)
Andrew M. Kuchling87c98b22004-08-25 13:38:46 +00001437
Andrew M. Kuchling00457172004-07-15 11:52:40 +00001438\item The \module{threading} module now has an elegantly simple way to support
1439thread-local data. The module contains a \class{local} class whose
1440attribute values are local to different threads.
1441
1442\begin{verbatim}
1443import threading
1444
1445data = threading.local()
1446data.number = 42
1447data.url = ('www.python.org', 80)
1448\end{verbatim}
1449
1450Other threads can assign and retrieve their own values for the
1451\member{number} and \member{url} attributes. You can subclass
1452\class{local} to initialize attributes or to add methods.
1453(Contributed by Jim Fulton.)
1454
Andrew M. Kuchlinga331e862004-09-10 13:05:22 +00001455\item The \module{timeit} module now automatically disables periodic
1456 garbarge collection during the timing loop. This change makes
Andrew M. Kuchling7642f7a2004-09-13 15:06:50 +00001457 consecutive timings more comparable. (Contributed by Raymond Hettinger.)
Andrew M. Kuchlinga331e862004-09-10 13:05:22 +00001458
Raymond Hettinger874ebd52004-05-31 03:15:02 +00001459\item The \module{weakref} module now supports a wider variety of objects
1460 including Python functions, class instances, sets, frozensets, deques,
1461 arrays, files, sockets, and regular expression pattern objects.
Andrew M. Kuchling7642f7a2004-09-13 15:06:50 +00001462 (Contributed by Raymond Hettinger.)
Andrew M. Kuchlingd0b6d9d2004-07-04 15:35:00 +00001463
1464\item The \module{xmlrpclib} module now supports a multi-call extension for
Andrew M. Kuchling00457172004-07-15 11:52:40 +00001465transmitting multiple XML-RPC calls in a single HTTP operation.
Andrew M. Kuchling067947e2004-11-19 14:43:36 +00001466(Contributed by Brian Quinlan.)
Andrew M. Kuchling3d3db962004-08-31 13:57:02 +00001467
1468\item The \module{mpz}, \module{rotor}, and \module{xreadlines} modules have
1469been removed.
Andrew M. Kuchling69f31eb2003-08-13 23:11:04 +00001470
Fred Drakeed0fa3d2003-07-30 19:14:09 +00001471\end{itemize}
1472
1473
1474%======================================================================
Raymond Hettingerca1a7752004-07-12 13:00:45 +00001475% whole new modules get described in subsections here
Fred Drakeed0fa3d2003-07-30 19:14:09 +00001476
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +00001477%=====================
Martin v. Löwis2a6ba902004-05-31 18:22:40 +00001478\subsection{cookielib}
1479
1480The \module{cookielib} library supports client-side handling for HTTP
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +00001481cookies, mirroring the \module{Cookie} module's server-side cookie
1482support. Cookies are stored in cookie jars; the library transparently
1483stores cookies offered by the web server in the cookie jar, and
1484fetches the cookie from the jar when connecting to the server. As in
1485web browsers, policy objects control whether cookies are accepted or
1486not.
Martin v. Löwis2a6ba902004-05-31 18:22:40 +00001487
1488In order to store cookies across sessions, two implementations of
1489cookie jars are provided: one that stores cookies in the Netscape
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +00001490format so applications can use the Mozilla or Lynx cookie files, and
Martin v. Löwis2a6ba902004-05-31 18:22:40 +00001491one that stores cookies in the same format as the Perl libwww libary.
1492
1493\module{urllib2} has been changed to interact with \module{cookielib}:
1494\class{HTTPCookieProcessor} manages a cookie jar that is used when
1495accessing URLs.
Fred Drakeed0fa3d2003-07-30 19:14:09 +00001496
Andrew M. Kuchling067947e2004-11-19 14:43:36 +00001497This module was contributed by John J. Lee.
1498
1499
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +00001500% ==================
Andrew M. Kuchling87c98b22004-08-25 13:38:46 +00001501\subsection{doctest}
1502
1503The \module{doctest} module underwent considerable refactoring thanks
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +00001504to Edward Loper and Tim Peters. Testing can still be as simple as
1505running \function{doctest.testmod()}, but the refactorings allow
1506customizing the module's operation in various ways
Andrew M. Kuchling87c98b22004-08-25 13:38:46 +00001507
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +00001508The new \class{DocTestFinder} class extracts the tests from a given
1509object's docstrings:
1510
1511\begin{verbatim}
1512def f (x, y):
1513 """>>> f(2,2)
15144
1515>>> f(3,2)
15166
1517 """
1518 return x*y
1519
1520finder = doctest.DocTestFinder()
1521
1522# Get list of DocTest instances
1523tests = finder.find(f)
1524\end{verbatim}
1525
1526The new \class{DocTestRunner} class then runs individual tests and can
1527produce a summary of the results:
1528
1529\begin{verbatim}
1530runner = doctest.DocTestRunner()
1531for t in tests:
1532 tried, failed = runner.run(t)
1533
1534runner.summarize(verbose=1)
1535\end{verbatim}
1536
1537The above example produces the following output:
1538
1539\begin{verbatim}
15401 items passed all tests:
1541 2 tests in f
15422 tests in 1 items.
15432 passed and 0 failed.
1544Test passed.
1545\end{verbatim}
1546
1547\class{DocTestRunner} uses an instance of the \class{OutputChecker}
1548class to compare the expected output with the actual output. This
1549class takes a number of different flags that customize its behaviour;
1550ambitious users can also write a completely new subclass of
1551\class{OutputChecker}.
1552
1553The default output checker provides a number of handy features.
1554For example, with the \constant{doctest.ELLIPSIS} option flag,
1555an ellipsis (\samp{...}) in the expected output matches any substring,
1556making it easier to accommodate outputs that vary in minor ways:
1557
1558\begin{verbatim}
1559def o (n):
1560 """>>> o(1)
1561<__main__.C instance at 0x...>
1562>>>
1563"""
1564\end{verbatim}
1565
1566Another special string, \samp{<BLANKLINE>}, matches a blank line:
1567
1568\begin{verbatim}
1569def p (n):
1570 """>>> p(1)
1571<BLANKLINE>
1572>>>
1573"""
1574\end{verbatim}
1575
1576Another new capability is producing a diff-style display of the output
1577by specifying the \constant{doctest.REPORT_UDIFF} (unified diffs),
1578\constant{doctest.REPORT_CDIFF} (context diffs), or
1579\constant{doctest.REPORT_NDIFF} (delta-style) option flags. For example:
1580
1581\begin{verbatim}
1582def g (n):
1583 """>>> g(4)
1584here
1585is
1586a
1587lengthy
1588>>>"""
1589 L = 'here is a rather lengthy list of words'.split()
1590 for word in L[:n]:
1591 print word
1592\end{verbatim}
1593
1594Running the above function's tests with
1595\constant{doctest.REPORT_UDIFF} specified, you get the following output:
1596
1597\begin{verbatim}
1598**********************************************************************
1599File ``t.py'', line 15, in g
1600Failed example:
1601 g(4)
1602Differences (unified diff with -expected +actual):
1603 @@ -2,3 +2,3 @@
1604 is
1605 a
1606 -lengthy
1607 +rather
1608**********************************************************************
1609\end{verbatim}
1610
Andrew M. Kuchling87c98b22004-08-25 13:38:46 +00001611
Fred Drakeed0fa3d2003-07-30 19:14:09 +00001612% ======================================================================
1613\section{Build and C API Changes}
1614
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +00001615Some of the changes to Python's build process and to the C API are:
Fred Drakeed0fa3d2003-07-30 19:14:09 +00001616
1617\begin{itemize}
1618
Andrew M. Kuchling6aedcfc2003-10-21 12:48:23 +00001619 \item Three new convenience macros were added for common return
1620 values from extension functions: \csimplemacro{Py_RETURN_NONE},
1621 \csimplemacro{Py_RETURN_TRUE}, and \csimplemacro{Py_RETURN_FALSE}.
Andrew M. Kuchling7642f7a2004-09-13 15:06:50 +00001622 (Contributed by Brett Cannon.)
Andrew M. Kuchling6aedcfc2003-10-21 12:48:23 +00001623
Andrew M. Kuchling5785a132004-07-26 19:28:46 +00001624 \item Another new macro, \csimplemacro{Py_CLEAR(\var{obj})},
1625 decreases the reference count of \var{obj} and sets \var{obj} to the
Andrew M. Kuchling7642f7a2004-09-13 15:06:50 +00001626 null pointer. (Contributed by Jim Fulton.)
Andrew M. Kuchling5785a132004-07-26 19:28:46 +00001627
Fred Drakece3caf22004-02-12 18:13:12 +00001628 \item A new function, \cfunction{PyTuple_Pack(\var{N}, \var{obj1},
1629 \var{obj2}, ..., \var{objN})}, constructs tuples from a variable
Andrew M. Kuchling7642f7a2004-09-13 15:06:50 +00001630 length argument list of Python objects. (Contributed by Raymond Hettinger.)
Fred Drakeed0fa3d2003-07-30 19:14:09 +00001631
Fred Drakece3caf22004-02-12 18:13:12 +00001632 \item A new function, \cfunction{PyDict_Contains(\var{d}, \var{k})},
1633 implements fast dictionary lookups without masking exceptions raised
Andrew M. Kuchling7642f7a2004-09-13 15:06:50 +00001634 during the look-up process. (Contributed by Raymond Hettinger.)
Raymond Hettingerd4462302003-11-26 17:52:45 +00001635
Andrew M. Kuchling0c789562004-09-23 20:15:41 +00001636 \item The \csimplemacro{Py_IS_NAN(\var{X})} macro returns 1 if
1637 its float or double argument \var{X} is a NaN.
1638 (Contributed by Tim Peters.)
1639
Andrew M. Kuchlingf3958f12004-10-11 19:20:06 +00001640 \item C code can avoid unnecessary locking by using the new
1641 \cfunction{PyEval_ThreadsInitialized()} function to tell
1642 if any thread operations have been performed. If this function
1643 returns false, no lock operations are needed.
1644 (Contributed by Nick Coghlan.)
1645
Andrew M. Kuchlinge30c4d42004-08-07 13:58:02 +00001646 \item A new function, \cfunction{PyArg_VaParseTupleAndKeywords()},
1647 is the same as \cfunction{PyArg_ParseTupleAndKeywords()} but takes a
1648 \ctype{va_list} instead of a number of arguments.
1649 (Contributed by Greg Chapman.)
1650
Fred Drakece3caf22004-02-12 18:13:12 +00001651 \item A new method flag, \constant{METH_COEXISTS}, allows a function
Andrew M. Kuchling71432f12004-07-05 01:40:07 +00001652 defined in slots to co-exist with a \ctype{PyCFunction} having the
1653 same name. This can halve the access time for a method such as
Andrew M. Kuchling7642f7a2004-09-13 15:06:50 +00001654 \method{set.__contains__()}. (Contributed by Raymond Hettinger.)
Andrew M. Kuchlingd0b6d9d2004-07-04 15:35:00 +00001655
Andrew M. Kuchling87c98b22004-08-25 13:38:46 +00001656 \item Python can now be built with additional profiling for the
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +00001657 interpreter itself, intended as an aid to people developing the
Andrew M. Kuchling87c98b22004-08-25 13:38:46 +00001658 Python core. Providing \longprogramopt{--enable-profiling} to the
1659 \program{configure} script will let you profile the interpreter with
1660 \program{gprof}, and providing the \longprogramopt{--with-tsc}
1661 switch enables profiling using the Pentium's Time-Stamp-Counter
Andrew M. Kuchling067947e2004-11-19 14:43:36 +00001662 register. Note that the \longprogramopt{--with-tsc} switch is slightly
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +00001663 misnamed, because the profiling feature also works on the PowerPC
1664 platform, though that processor architecture doesn't call that
Andrew M. Kuchling067947e2004-11-19 14:43:36 +00001665 register ``the TSC register''. (Contributed by Jeremy Hylton.)
1666
Andrew M. Kuchlingd0b6d9d2004-07-04 15:35:00 +00001667 \item The \ctype{tracebackobject} type has been renamed to \ctype{PyTracebackObject}.
Raymond Hettinger97ef8de2004-01-05 00:29:57 +00001668
Fred Drakeed0fa3d2003-07-30 19:14:09 +00001669\end{itemize}
1670
1671
1672%======================================================================
1673\subsection{Port-Specific Changes}
1674
Raymond Hettinger97ef8de2004-01-05 00:29:57 +00001675\begin{itemize}
1676
1677\item The Windows port now builds under MSVC++ 7.1 as well as version 6.
Andrew M. Kuchling7642f7a2004-09-13 15:06:50 +00001678 (Contributed by Martin von Loewis.)
Raymond Hettinger97ef8de2004-01-05 00:29:57 +00001679
1680\end{itemize}
Fred Drakeed0fa3d2003-07-30 19:14:09 +00001681
1682
Fred Drakeed0fa3d2003-07-30 19:14:09 +00001683
1684%======================================================================
1685\section{Porting to Python 2.4}
1686
1687This section lists previously described changes that may require
1688changes to your code:
1689
1690\begin{itemize}
1691
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +00001692\item Left shifts and hexadecimal/octal constants that are too
1693 large no longer trigger a \exception{FutureWarning} and return
1694 a value limited to 32 or 64 bits; instead they return a long integer.
1695
1696\item Integer operations will no longer trigger an \exception{OverflowWarning}.
1697The \exception{OverflowWarning} warning will disappear in Python 2.5.
1698
Raymond Hettinger607c00f2003-11-12 16:27:50 +00001699\item The \function{zip()} built-in function and \function{itertools.izip()}
1700 now return an empty list instead of raising a \exception{TypeError}
1701 exception if called with no arguments.
Andrew M. Kuchling6aedcfc2003-10-21 12:48:23 +00001702
1703\item \function{dircache.listdir()} now passes exceptions to the caller
1704 instead of returning empty lists.
Fred Drakeed0fa3d2003-07-30 19:14:09 +00001705
Andrew M. Kuchling71432f12004-07-05 01:40:07 +00001706\item \function{LexicalHandler.startDTD()} used to receive the public and
1707 system IDs in the wrong order. This has been corrected; applications
Fred Drake56fcc232004-05-06 02:55:35 +00001708 relying on the wrong order need to be fixed.
Martin v. Löwis456ab1d2004-05-06 01:54:36 +00001709
Andrew M. Kuchling71432f12004-07-05 01:40:07 +00001710\item \function{fcntl.ioctl} now warns if the \var{mutate}
1711 argument is omitted and relevant.
Martin v. Löwis77ca6c42004-06-03 12:47:26 +00001712
Andrew M. Kuchling87c98b22004-08-25 13:38:46 +00001713\item The \module{tarfile} module now generates GNU-format tar files
1714by default.
1715
Andrew M. Kuchlingf8c075c2004-11-09 02:58:02 +00001716\item Encountering a failure while importing a module no longer leaves
1717a partially-initialized module object in \code{sys.modules}.
1718
1719\item \constant{None} is now a constant; code that binds a new value to
1720the name \samp{None} is now a syntax error.
1721
Fred Drakeed0fa3d2003-07-30 19:14:09 +00001722\end{itemize}
1723
1724
1725%======================================================================
1726\section{Acknowledgements \label{acks}}
1727
1728The author would like to thank the following people for offering
1729suggestions, corrections and assistance with various drafts of this
Andrew M. Kuchling536183b2004-11-25 01:15:25 +00001730article: Koray Can, Hye-Shik Chang, Michael Dyck, Raymond Hettinger,
Andrew M. Kuchlingb69c49c2004-12-01 00:42:41 +00001731Brian Hurt, Hamish Lawson, Fredrik Lundh, Sean Reifschneider.
Fred Drakeed0fa3d2003-07-30 19:14:09 +00001732
1733\end{document}