blob: 0da9b5a54c19cc5f96132b084e6e83a8f7e13370 [file] [log] [blame]
Fred Drake2db76802004-12-01 05:05:47 +00001\documentclass{howto}
2\usepackage{distutils}
3% $Id$
4
Fred Drake2db76802004-12-01 05:05:47 +00005
6\title{What's New in Python 2.5}
7\release{0.0}
Andrew M. Kuchling92e24952004-12-03 13:54:09 +00008\author{A.M. Kuchling}
9\authoraddress{\email{amk@amk.ca}}
Fred Drake2db76802004-12-01 05:05:47 +000010
11\begin{document}
12\maketitle
13\tableofcontents
14
15This article explains the new features in Python 2.5. No release date
Andrew M. Kuchling5eefdca2006-02-08 11:36:09 +000016for Python 2.5 has been set; it will probably be released in the
17autumn of 2006.
Fred Drake2db76802004-12-01 05:05:47 +000018
Andrew M. Kuchling437567c2006-03-07 20:48:55 +000019% XXX Compare with previous release in 2 - 3 sentences here.
Fred Drake2db76802004-12-01 05:05:47 +000020
21This article doesn't attempt to provide a complete specification of
22the new features, but instead provides a convenient overview. For
23full details, you should refer to the documentation for Python 2.5.
Andrew M. Kuchling437567c2006-03-07 20:48:55 +000024% XXX add hyperlink when the documentation becomes available online.
Fred Drake2db76802004-12-01 05:05:47 +000025If you want to understand the complete implementation and design
26rationale, refer to the PEP for a particular new feature.
27
28
29%======================================================================
Andrew M. Kuchling437567c2006-03-07 20:48:55 +000030\section{PEP 308: Conditional Expressions}
31
Andrew M. Kuchlinge362d932006-03-09 13:56:25 +000032For a long time, people have been requesting a way to write
33conditional expressions, expressions that return value A or value B
34depending on whether a Boolean value is true or false. A conditional
35expression lets you write a single assignment statement that has the
36same effect as the following:
37
38\begin{verbatim}
39if condition:
40 x = true_value
41else:
42 x = false_value
43\end{verbatim}
44
45There have been endless tedious discussions of syntax on both
46python-dev and comp.lang.python, and even a vote that found the
47majority of voters wanted some way to write conditional expressions,
48but there was no syntax that was clearly preferred by a majority.
49Candidates include C's \code{cond ? true_v : false_v},
50\code{if cond then true_v else false_v}, and 16 other variations.
51
52GvR eventually chose a surprising syntax:
53
54\begin{verbatim}
55x = true_value if condition else false_value
56\end{verbatim}
57
58Evaluation is still lazy as in existing Boolean expression, so the
59evaluation jumps around a bit. The \var{condition} expression is
60evaluated first, and the \var{true_value} expression is evaluated only
61if the condition was true. Similarly, the \var{false_value}
62expression is only evaluated when the condition is false.
63
64This syntax may seem strange and backwards; why does the condition go
65in the \emph{middle} of the expression, and not in the front as in C's
66\code{c ? x : y}? The decision was checked by applying the new syntax
67to the modules in the standard library and seeing how the resulting
68code read. In many cases where a conditional expression is used, one
69value seems to be the 'common case' and one value is an 'exceptional
70case', used only on rarer occasions when the condition isn't met. The
71conditional syntax makes this pattern a bit more obvious:
72
73\begin{verbatim}
74contents = ((doc + '\n') if doc else '')
75\end{verbatim}
76
77I read the above statement as meaning ``here \var{contents} is
Andrew M. Kuchlingd0fcc022006-03-09 13:57:28 +000078usually assigned a value of \code{doc+'\e n'}; sometimes
Andrew M. Kuchlinge362d932006-03-09 13:56:25 +000079\var{doc} is empty, in which special case an empty string is returned.''
80I doubt I will use conditional expressions very often where there
81isn't a clear common and uncommon case.
82
83There was some discussion of whether the language should require
84surrounding conditional expressions with parentheses. The decision
85was made to \emph{not} require parentheses in the Python language's
86grammar, but as a matter of style I think you should always use them.
87Consider these two statements:
88
89\begin{verbatim}
90# First version -- no parens
91level = 1 if logging else 0
92
93# Second version -- with parens
94level = (1 if logging else 0)
95\end{verbatim}
96
97In the first version, I think a reader's eye might group the statement
98into 'level = 1', 'if logging', 'else 0', and think that the condition
99decides whether the assignment to \var{level} is performed. The
100second version reads better, in my opinion, because it makes it clear
101that the assignment is always performed and the choice is being made
102between two values.
103
104Another reason for including the brackets: a few odd combinations of
105list comprehensions and lambdas could look like incorrect conditional
106expressions. See \pep{308} for some examples. If you put parentheses
107around your conditional expressions, you won't run into this case.
108
109
110\begin{seealso}
111
112\seepep{308}{Conditional Expressions}{PEP written by
113Guido van Rossum and Raymond D. Hettinger; implemented by Thomas
114Wouters.}
115
116\end{seealso}
Andrew M. Kuchling437567c2006-03-07 20:48:55 +0000117
118
119%======================================================================
Andrew M. Kuchling3e41b052005-03-01 00:53:46 +0000120\section{PEP 309: Partial Function Application}
Fred Drake2db76802004-12-01 05:05:47 +0000121
Andrew M. Kuchlingb1c96fd2005-03-20 21:42:04 +0000122The \module{functional} module is intended to contain tools for
Andrew M. Kuchling437567c2006-03-07 20:48:55 +0000123functional-style programming. Currently it only contains a
124\class{partial()} function, but new functions will probably be added
125in future versions of Python.
Andrew M. Kuchlingb1c96fd2005-03-20 21:42:04 +0000126
Andrew M. Kuchling4b000cd2005-04-09 15:51:44 +0000127For programs written in a functional style, it can be useful to
128construct variants of existing functions that have some of the
129parameters filled in. Consider a Python function \code{f(a, b, c)};
130you could create a new function \code{g(b, c)} that was equivalent to
131\code{f(1, b, c)}. This is called ``partial function application'',
132and is provided by the \class{partial} class in the new
133\module{functional} module.
134
135The constructor for \class{partial} takes the arguments
136\code{(\var{function}, \var{arg1}, \var{arg2}, ...
137\var{kwarg1}=\var{value1}, \var{kwarg2}=\var{value2})}. The resulting
138object is callable, so you can just call it to invoke \var{function}
139with the filled-in arguments.
140
141Here's a small but realistic example:
142
143\begin{verbatim}
144import functional
145
146def log (message, subsystem):
147 "Write the contents of 'message' to the specified subsystem."
148 print '%s: %s' % (subsystem, message)
149 ...
150
151server_log = functional.partial(log, subsystem='server')
Andrew M. Kuchling437567c2006-03-07 20:48:55 +0000152server_log('Unable to open socket')
Andrew M. Kuchling4b000cd2005-04-09 15:51:44 +0000153\end{verbatim}
154
Andrew M. Kuchling6af7fe02005-08-02 17:20:36 +0000155Here's another example, from a program that uses PyGTk. Here a
156context-sensitive pop-up menu is being constructed dynamically. The
157callback provided for the menu option is a partially applied version
158of the \method{open_item()} method, where the first argument has been
159provided.
Andrew M. Kuchling4b000cd2005-04-09 15:51:44 +0000160
Andrew M. Kuchling6af7fe02005-08-02 17:20:36 +0000161\begin{verbatim}
162...
163class Application:
164 def open_item(self, path):
165 ...
166 def init (self):
167 open_func = functional.partial(self.open_item, item_path)
168 popup_menu.append( ("Open", open_func, 1) )
169\end{verbatim}
Andrew M. Kuchlingb1c96fd2005-03-20 21:42:04 +0000170
171
172\begin{seealso}
173
174\seepep{309}{Partial Function Application}{PEP proposed and written by
175Peter Harris; implemented by Hye-Shik Chang, with adaptations by
176Raymond Hettinger.}
177
178\end{seealso}
Fred Drake2db76802004-12-01 05:05:47 +0000179
180
181%======================================================================
Fred Drakedb7b0022005-03-20 22:19:47 +0000182\section{PEP 314: Metadata for Python Software Packages v1.1}
183
Andrew M. Kuchlingd8d732e2005-04-09 23:59:41 +0000184Some simple dependency support was added to Distutils. The
Andrew M. Kuchling437567c2006-03-07 20:48:55 +0000185\function{setup()} function now has \code{requires}, \code{provides},
186and \code{obsoletes} keyword parameters. When you build a source
187distribution using the \code{sdist} command, the dependency
188information will be recorded in the \file{PKG-INFO} file.
Andrew M. Kuchlingd8d732e2005-04-09 23:59:41 +0000189
Andrew M. Kuchling437567c2006-03-07 20:48:55 +0000190Another new keyword parameter is \code{download_url}, which should be
191set to a URL for the package's source code. This means it's now
192possible to look up an entry in the package index, determine the
193dependencies for a package, and download the required packages.
Andrew M. Kuchlingd8d732e2005-04-09 23:59:41 +0000194
195% XXX put example here
196
197\begin{seealso}
198
199\seepep{314}{Metadata for Python Software Packages v1.1}{PEP proposed
200and written by A.M. Kuchling, Richard Jones, and Fred Drake;
201implemented by Richard Jones and Fred Drake.}
202
203\end{seealso}
Fred Drakedb7b0022005-03-20 22:19:47 +0000204
205
206%======================================================================
Andrew M. Kuchling437567c2006-03-07 20:48:55 +0000207\section{PEP 328: Absolute and Relative Imports}
208
209% XXX write this
210
211
212%======================================================================
Andrew M. Kuchling21d3a7c2006-03-15 11:53:09 +0000213\section{PEP 338: Executing Modules as Scripts}
214
Andrew M. Kuchlingb182db42006-03-17 21:48:46 +0000215The \programopt{-m} switch added in Python 2.4 to execute a module as
216a script gained a few more abilities. Instead of being implemented in
217C code inside the Python interpreter, the switch now uses an
218implementation in a new module, \module{runpy}.
219
220The \module{runpy} module implements a more sophisticated import
221mechanism so that it's now possible to run modules in a package such
222as \module{pychecker.checker}. The module also supports alternative
223import mechanisms such as the \module{zipimport} module. (This means
224you can add a .zip archive's path to \code{sys.path} and then use the
225\programopt{-m} switch to execute code from the archive.
226
227
228\begin{seealso}
229
230\seepep{338}{Executing modules as scripts}{PEP written and
231implemented by Nick Coghlan.}
232
233\end{seealso}
Andrew M. Kuchling21d3a7c2006-03-15 11:53:09 +0000234
235
236%======================================================================
Andrew M. Kuchling437567c2006-03-07 20:48:55 +0000237\section{PEP 341: Unified try/except/finally}
238
239% XXX write this
240
241
242%======================================================================
Andrew M. Kuchlinga2e21cb2005-08-02 17:13:21 +0000243\section{PEP 342: New Generator Features}
244
Andrew M. Kuchling150e3492005-08-23 00:56:06 +0000245As introduced in Python 2.3, generators only produce output; once a
Andrew M. Kuchling437567c2006-03-07 20:48:55 +0000246generator's code is invoked to create an iterator, there's no way to
247pass any new information into the function when its execution is
248resumed. Hackish solutions to this include making the generator's
249code look at a global variable and then changing the global variable's
250value, or passing in some mutable object that callers then modify.
251Python 2.5 adds the ability to pass values \emph{into} a generator.
Andrew M. Kuchling150e3492005-08-23 00:56:06 +0000252
253To refresh your memory of basic generators, here's a simple example:
254
255\begin{verbatim}
256def counter (maximum):
257 i = 0
258 while i < maximum:
259 yield i
260 i += 1
261\end{verbatim}
262
Andrew M. Kuchling07382062005-08-27 18:45:47 +0000263When you call \code{counter(10)}, the result is an iterator that
264returns the values from 0 up to 9. On encountering the
265\keyword{yield} statement, the iterator returns the provided value and
266suspends the function's execution, preserving the local variables.
267Execution resumes on the following call to the iterator's
Andrew M. Kuchling437567c2006-03-07 20:48:55 +0000268\method{next()} method, picking up after the \keyword{yield} statement.
Andrew M. Kuchling150e3492005-08-23 00:56:06 +0000269
Andrew M. Kuchling07382062005-08-27 18:45:47 +0000270In Python 2.3, \keyword{yield} was a statement; it didn't return any
271value. In 2.5, \keyword{yield} is now an expression, returning a
272value that can be assigned to a variable or otherwise operated on:
Andrew M. Kuchlinga2e21cb2005-08-02 17:13:21 +0000273
Andrew M. Kuchling07382062005-08-27 18:45:47 +0000274\begin{verbatim}
275val = (yield i)
276\end{verbatim}
277
278I recommend that you always put parentheses around a \keyword{yield}
279expression when you're doing something with the returned value, as in
280the above example. The parentheses aren't always necessary, but it's
281easier to always add them instead of having to remember when they're
Andrew M. Kuchling437567c2006-03-07 20:48:55 +0000282needed.\footnote{The exact rules are that a \keyword{yield}-expression must
Andrew M. Kuchling07382062005-08-27 18:45:47 +0000283always be parenthesized except when it occurs at the top-level
Andrew M. Kuchling437567c2006-03-07 20:48:55 +0000284expression on the right-hand side of an assignment, meaning you can
285write \code{val = yield i} but have to use parentheses when there's an
286operation, as in \code{val = (yield i) + 12}.}
Andrew M. Kuchling07382062005-08-27 18:45:47 +0000287
288Values are sent into a generator by calling its
289\method{send(\var{value})} method. The generator's code is then
Andrew M. Kuchling437567c2006-03-07 20:48:55 +0000290resumed and the \keyword{yield} expression returns the specified
291\var{value}. If the regular \method{next()} method is called, the
292\keyword{yield} returns \constant{None}.
Andrew M. Kuchling07382062005-08-27 18:45:47 +0000293
294Here's the previous example, modified to allow changing the value of
295the internal counter.
296
297\begin{verbatim}
298def counter (maximum):
299 i = 0
300 while i < maximum:
301 val = (yield i)
302 # If value provided, change counter
303 if val is not None:
304 i = val
305 else:
306 i += 1
307\end{verbatim}
308
309And here's an example of changing the counter:
310
311\begin{verbatim}
312>>> it = counter(10)
313>>> print it.next()
3140
315>>> print it.next()
3161
317>>> print it.send(8)
3188
319>>> print it.next()
3209
321>>> print it.next()
322Traceback (most recent call last):
323 File ``t.py'', line 15, in ?
324 print it.next()
325StopIteration
Andrew M. Kuchlingc2033702005-08-29 13:30:12 +0000326\end{verbatim}
Andrew M. Kuchling07382062005-08-27 18:45:47 +0000327
Andrew M. Kuchling437567c2006-03-07 20:48:55 +0000328Because \keyword{yield} will often be returning \constant{None}, you
329should always check for this case. Don't just use its value in
330expressions unless you're sure that the \method{send()} method
331will be the only method used resume your generator function.
Andrew M. Kuchling07382062005-08-27 18:45:47 +0000332
Andrew M. Kuchling437567c2006-03-07 20:48:55 +0000333In addition to \method{send()}, there are two other new methods on
334generators:
Andrew M. Kuchling07382062005-08-27 18:45:47 +0000335
336\begin{itemize}
337
338 \item \method{throw(\var{type}, \var{value}=None,
339 \var{traceback}=None)} is used to raise an exception inside the
340 generator; the exception is raised by the \keyword{yield} expression
341 where the generator's execution is paused.
342
343 \item \method{close()} raises a new \exception{GeneratorExit}
344 exception inside the generator to terminate the iteration.
345 On receiving this
346 exception, the generator's code must either raise
347 \exception{GeneratorExit} or \exception{StopIteration}; catching the
348 exception and doing anything else is illegal and will trigger
349 a \exception{RuntimeError}. \method{close()} will also be called by
350 Python's garbage collection when the generator is garbage-collected.
351
352 If you need to run cleanup code in case of a \exception{GeneratorExit},
353 I suggest using a \code{try: ... finally:} suite instead of
354 catching \exception{GeneratorExit}.
355
356\end{itemize}
357
358The cumulative effect of these changes is to turn generators from
359one-way producers of information into both producers and consumers.
Andrew M. Kuchling437567c2006-03-07 20:48:55 +0000360
Andrew M. Kuchling07382062005-08-27 18:45:47 +0000361Generators also become \emph{coroutines}, a more generalized form of
Andrew M. Kuchling437567c2006-03-07 20:48:55 +0000362subroutines. Subroutines are entered at one point and exited at
Andrew M. Kuchling07382062005-08-27 18:45:47 +0000363another point (the top of the function, and a \keyword{return
364statement}), but coroutines can be entered, exited, and resumed at
Andrew M. Kuchling437567c2006-03-07 20:48:55 +0000365many different points (the \keyword{yield} statements).
Andrew M. Kuchling07382062005-08-27 18:45:47 +0000366
Andrew M. Kuchling437567c2006-03-07 20:48:55 +0000367
Andrew M. Kuchlinga2e21cb2005-08-02 17:13:21 +0000368\begin{seealso}
369
370\seepep{342}{Coroutines via Enhanced Generators}{PEP written by
371Guido van Rossum and Phillip J. Eby;
Andrew M. Kuchling07382062005-08-27 18:45:47 +0000372implemented by Phillip J. Eby. Includes examples of
373some fancier uses of generators as coroutines.}
374
375\seeurl{http://en.wikipedia.org/wiki/Coroutine}{The Wikipedia entry for
376coroutines.}
377
Neal Norwitz09179882006-03-04 23:31:45 +0000378\seeurl{http://www.sidhe.org/\~{}dan/blog/archives/000178.html}{An
Andrew M. Kuchling07382062005-08-27 18:45:47 +0000379explanation of coroutines from a Perl point of view, written by Dan
380Sugalski.}
Andrew M. Kuchlinga2e21cb2005-08-02 17:13:21 +0000381
382\end{seealso}
383
384
385%======================================================================
Andrew M. Kuchling437567c2006-03-07 20:48:55 +0000386\section{PEP 343: The 'with' statement}
387
388% XXX write this
389
390
391%======================================================================
Andrew M. Kuchling8f4d2552006-03-08 01:50:20 +0000392\section{PEP 352: Exceptions as New-Style Classes}
393
Andrew M. Kuchlingaeadf952006-03-09 19:06:05 +0000394Exception classes can now be new-style classes, not just classic classes,
395and the built-in \exception{Exception} class and all
396
397The inheritance hierarchy for exceptions has been rearranged a bit.
398In 2.5, the inheritance relationships are:
399
400\begin{verbatim}
401BaseException # New in Python 2.5
402|- KeyboardInterrupt
403|- SystemExit
404|- Exception
405 |- (all other current built-in exceptions)
406\end{verbatim}
407
408This rearrangement was done because people often want to catch all
409exceptions that indicate program errors. \exception{KeyboardInterrupt} and
410\exception{SystemExit} aren't errors, though, and usually represent an explicit
411action such as the user hitting Control-C or code calling
412\function{sys.exit()}. A bare \code{except:} will catch all exceptions,
413so you commonly need to list \exception{KeyboardInterrupt} and
414\exception{SystemExit} in order to re-raise them. The usual pattern is:
415
416\begin{verbatim}
417try:
418 ...
419except (KeyboardInterrupt, SystemExit):
420 raise
421except:
422 # Log error...
423 # Continue running program...
424\end{verbatim}
425
426In Python 2.5, you can now write \code{except Exception} to achieve
427the same result, catching all the exceptions that usually indicate errors
428but leaving \exception{KeyboardInterrupt} and
429\exception{SystemExit} alone. As in previous versions,
430a bare \code{except:} still catches all exceptions.
431
432The goal for Python 3.0 is to require any class raised as an exception
433to derive from \exception{BaseException} or some descendant of
434\exception{BaseException}, and future releases in the
435Python 2.x series may begin to enforce this constraint. Therefore, I
436suggest you begin making all your exception classes derive from
437\exception{Exception} now. It's been suggested that the bare
438\code{except:} form should be removed in Python 3.0, but Guido van~Rossum
439hasn't decided whether to do this or not.
440
441Raising of strings as exceptions, as in the statement \code{raise
442"Error occurred"}, is deprecated in Python 2.5 and will trigger a
443warning. The aim is to be able to remove the string-exception feature
444in a few releases.
445
446
447\begin{seealso}
448
449\seepep{352}{}{PEP written by
450Brett Cannon and Guido van Rossum; implemented by Brett Cannon.}
451
452\end{seealso}
Andrew M. Kuchling8f4d2552006-03-08 01:50:20 +0000453
454
455%======================================================================
Andrew M. Kuchling437567c2006-03-07 20:48:55 +0000456\section{PEP 357: The '__index__' method}
457
458% XXX write this
459
460
461%======================================================================
Fred Drake2db76802004-12-01 05:05:47 +0000462\section{Other Language Changes}
463
464Here are all of the changes that Python 2.5 makes to the core Python
465language.
466
467\begin{itemize}
Andrew M. Kuchling1cae3f52004-12-03 14:57:21 +0000468
469\item The \function{min()} and \function{max()} built-in functions
470gained a \code{key} keyword argument analogous to the \code{key}
Andrew M. Kuchlinge9b1bf42005-03-20 19:26:30 +0000471argument for \method{sort()}. This argument supplies a function
Andrew M. Kuchling1cae3f52004-12-03 14:57:21 +0000472that takes a single argument and is called for every value in the list;
473\function{min()}/\function{max()} will return the element with the
474smallest/largest return value from this function.
475For example, to find the longest string in a list, you can do:
476
477\begin{verbatim}
478L = ['medium', 'longest', 'short']
479# Prints 'longest'
480print max(L, key=len)
481# Prints 'short', because lexicographically 'short' has the largest value
482print max(L)
483\end{verbatim}
484
485(Contributed by Steven Bethard and Raymond Hettinger.)
Fred Drake2db76802004-12-01 05:05:47 +0000486
Andrew M. Kuchling150e3492005-08-23 00:56:06 +0000487\item Two new built-in functions, \function{any()} and
488\function{all()}, evaluate whether an iterator contains any true or
489false values. \function{any()} returns \constant{True} if any value
490returned by the iterator is true; otherwise it will return
491\constant{False}. \function{all()} returns \constant{True} only if
492all of the values returned by the iterator evaluate as being true.
493
494% XXX who added?
495
496
Andrew M. Kuchlinge9b1bf42005-03-20 19:26:30 +0000497\item The list of base classes in a class definition can now be empty.
498As an example, this is now legal:
499
500\begin{verbatim}
501class C():
502 pass
503\end{verbatim}
504(Implemented by Brett Cannon.)
505
Fred Drake2db76802004-12-01 05:05:47 +0000506\end{itemize}
507
508
509%======================================================================
Andrew M. Kuchlingda376042006-03-17 15:56:41 +0000510\subsection{Interactive Interpreter Changes}
511
512In the interactive interpreter, \code{quit} and \code{exit}
513have long been strings so that new users get a somewhat helpful message
514when they try to quit:
515
516\begin{verbatim}
517>>> quit
518'Use Ctrl-D (i.e. EOF) to exit.'
519\end{verbatim}
520
521In Python 2.5, \code{quit} and \code{exit} are now objects that still
522produce string representations of themselves, but are also callable.
523Newbies who try \code{quit()} or \code{exit()} will now exit the
524interpreter as they expect. (Implemented by Georg Brandl.)
525
526
527%======================================================================
Fred Drake2db76802004-12-01 05:05:47 +0000528\subsection{Optimizations}
529
530\begin{itemize}
531
Andrew M. Kuchling150e3492005-08-23 00:56:06 +0000532\item When they were introduced
533in Python 2.4, the built-in \class{set} and \class{frozenset} types
534were built on top of Python's dictionary type.
535In 2.5 the internal data structure has been customized for implementing sets,
536and as a result sets will use a third less memory and are somewhat faster.
537(Implemented by Raymond Hettinger.)
Fred Drake2db76802004-12-01 05:05:47 +0000538
539\end{itemize}
540
541The net result of the 2.5 optimizations is that Python 2.5 runs the
Andrew M. Kuchling92e24952004-12-03 13:54:09 +0000542pystone benchmark around XX\% faster than Python 2.4.
Fred Drake2db76802004-12-01 05:05:47 +0000543
544
545%======================================================================
546\section{New, Improved, and Deprecated Modules}
547
548As usual, Python's standard library received a number of enhancements and
549bug fixes. Here's a partial list of the most notable changes, sorted
550alphabetically by module name. Consult the
551\file{Misc/NEWS} file in the source tree for a more
Andrew M. Kuchlingf688cc52006-03-10 18:50:08 +0000552complete list of changes, or look through the SVN logs for all the
Fred Drake2db76802004-12-01 05:05:47 +0000553details.
554
555\begin{itemize}
556
Andrew M. Kuchlingaeadf952006-03-09 19:06:05 +0000557% ctypes added
558
Andrew M. Kuchling150e3492005-08-23 00:56:06 +0000559% collections.deque now has .remove()
560
Andrew M. Kuchling3e41b052005-03-01 00:53:46 +0000561% the cPickle module no longer accepts the deprecated None option in the
562% args tuple returned by __reduce__().
563
564% csv module improvements
565
566% datetime.datetime() now has a strptime class method which can be used to
567% create datetime object using a string and format.
568
Andrew M. Kuchlingda376042006-03-17 15:56:41 +0000569\item In the \module{gc} module, the new \function{get_count()} function
570returns a 3-tuple containing the current collection counts for the
571three GC generations. This is accounting information for the garbage
572collector; when these counts reach a specified threshold, a garbage
573collection sweep will be made. The existing \function{gc.collect()}
574function now takes an optional \var{generation} argument of 0, 1, or 2
575to specify which generation to collect.
576
Andrew M. Kuchling150e3492005-08-23 00:56:06 +0000577\item A new \module{hashlib} module has been added to replace the
578\module{md5} and \module{sha} modules. \module{hashlib} adds support
579for additional secure hashes (SHA-224, SHA-256, SHA-384, and SHA-512).
580When available, the module uses OpenSSL for fast platform optimized
581implementations of algorithms. The old \module{md5} and \module{sha}
582modules still exist as wrappers around hashlib to preserve backwards
583compatibility. (Contributed by Gregory P. Smith.)
584
Andrew M. Kuchlinge9b1bf42005-03-20 19:26:30 +0000585\item The \function{nsmallest()} and
586\function{nlargest()} functions in the \module{heapq} module
587now support a \code{key} keyword argument similar to the one
588provided by the \function{min()}/\function{max()} functions
589and the \method{sort()} methods. For example:
590Example:
591
592\begin{verbatim}
593>>> import heapq
594>>> L = ["short", 'medium', 'longest', 'longer still']
595>>> heapq.nsmallest(2, L) # Return two lowest elements, lexicographically
596['longer still', 'longest']
597>>> heapq.nsmallest(2, L, key=len) # Return two shortest elements
598['short', 'medium']
599\end{verbatim}
600
601(Contributed by Raymond Hettinger.)
602
Andrew M. Kuchling511a3a82005-03-20 19:52:18 +0000603\item The \function{itertools.islice()} function now accepts
604\code{None} for the start and step arguments. This makes it more
605compatible with the attributes of slice objects, so that you can now write
606the following:
607
608\begin{verbatim}
609s = slice(5) # Create slice object
610itertools.islice(iterable, s.start, s.stop, s.step)
611\end{verbatim}
612
613(Contributed by Raymond Hettinger.)
Andrew M. Kuchling3e41b052005-03-01 00:53:46 +0000614
Andrew M. Kuchling150e3492005-08-23 00:56:06 +0000615\item The \module{operator} module's \function{itemgetter()}
616and \function{attrgetter()} functions now support multiple fields.
617A call such as \code{operator.attrgetter('a', 'b')}
618will return a function
619that retrieves the \member{a} and \member{b} attributes. Combining
620this new feature with the \method{sort()} method's \code{key} parameter
621lets you easily sort lists using multiple fields.
622
623% XXX who added?
624
Andrew M. Kuchling3e41b052005-03-01 00:53:46 +0000625
Andrew M. Kuchlinge9b1bf42005-03-20 19:26:30 +0000626\item The \module{os} module underwent a number of changes. The
627\member{stat_float_times} variable now defaults to true, meaning that
628\function{os.stat()} will now return time values as floats. (This
629doesn't necessarily mean that \function{os.stat()} will return times
630that are precise to fractions of a second; not all systems support
631such precision.)
Andrew M. Kuchling3e41b052005-03-01 00:53:46 +0000632
Andrew M. Kuchling150e3492005-08-23 00:56:06 +0000633Constants named \member{os.SEEK_SET}, \member{os.SEEK_CUR}, and
Andrew M. Kuchlinge9b1bf42005-03-20 19:26:30 +0000634\member{os.SEEK_END} have been added; these are the parameters to the
Andrew M. Kuchling150e3492005-08-23 00:56:06 +0000635\function{os.lseek()} function. Two new constants for locking are
636\member{os.O_SHLOCK} and \member{os.O_EXLOCK}.
637
638On FreeBSD, the \function{os.stat()} function now returns
639times with nanosecond resolution, and the returned object
640now has \member{st_gen} and \member{st_birthtime}.
641The \member{st_flags} member is also available, if the platform supports it.
642% XXX patch 1180695, 1212117
643
Andrew M. Kuchling01e3d262006-03-17 15:38:39 +0000644\item The old \module{regex} and \module{regsub} modules, which have been
645deprecated ever since Python 2.0, have finally been deleted.
Andrew M. Kuchlingf4b06602006-03-17 15:39:52 +0000646Other deleted modules: \module{statcache}, \module{tzparse},
647\module{whrandom}.
Andrew M. Kuchling01e3d262006-03-17 15:38:39 +0000648
649\item The \file{lib-old} directory,
650which includes ancient modules such as \module{dircmp} and
651\module{ni}, was also deleted. \file{lib-old} wasn't on the default
652\code{sys.path}, so unless your programs explicitly added the directory to
653\code{sys.path}, this removal shouldn't affect your code.
654
Andrew M. Kuchling4678dc82006-01-15 16:11:28 +0000655\item The \module{socket} module now supports \constant{AF_NETLINK}
656sockets on Linux, thanks to a patch from Philippe Biondi.
657Netlink sockets are a Linux-specific mechanism for communications
658between a user-space process and kernel code; an introductory
659article about them is at \url{http://www.linuxjournal.com/article/7356}.
660In Python code, netlink addresses are represented as a tuple of 2 integers,
661\code{(\var{pid}, \var{group_mask})}.
662
Andrew M. Kuchling150e3492005-08-23 00:56:06 +0000663\item New module: \module{spwd} provides functions for accessing the
664shadow password database on systems that support it.
665% XXX give example
Fred Drake2db76802004-12-01 05:05:47 +0000666
Andrew M. Kuchlinge9b1bf42005-03-20 19:26:30 +0000667\item The \class{TarFile} class in the \module{tarfile} module now has
Georg Brandl08c02db2005-07-22 18:39:19 +0000668an \method{extractall()} method that extracts all members from the
Andrew M. Kuchlinge9b1bf42005-03-20 19:26:30 +0000669archive into the current working directory. It's also possible to set
670a different directory as the extraction target, and to unpack only a
Andrew M. Kuchling150e3492005-08-23 00:56:06 +0000671subset of the archive's members.
Andrew M. Kuchlinge9b1bf42005-03-20 19:26:30 +0000672
Andrew M. Kuchling150e3492005-08-23 00:56:06 +0000673A tarfile's compression can be autodetected by
674using the mode \code{'r|*'}.
675% patch 918101
676(Contributed by Lars Gust\"abel.)
Gregory P. Smithf21a5f72005-08-21 18:45:59 +0000677
Andrew M. Kuchlingf688cc52006-03-10 18:50:08 +0000678\item The \module{unicodedata} module has been updated to use version 4.1.0
679of the Unicode character database. Version 3.2.0 is required
680by some specifications, so it's still available as
681\member{unicodedata.db_3_2_0}.
682
Fredrik Lundh7e0aef02005-12-12 18:54:55 +0000683\item A new package \module{xml.etree} has been added, which contains
684a subset of the ElementTree XML library. Available modules are
685\module{ElementTree}, \module{ElementPath}, and
686\module{ElementInclude}, from ElementTree 1.2.6. (Contributed by
687Fredrik Lundh.)
688
Andrew M. Kuchling150e3492005-08-23 00:56:06 +0000689\item The \module{xmlrpclib} module now supports returning
690 \class{datetime} objects for the XML-RPC date type. Supply
691 \code{use_datetime=True} to the \function{loads()} function
692 or the \class{Unmarshaller} class to enable this feature.
693% XXX patch 1120353
694
Gregory P. Smithf21a5f72005-08-21 18:45:59 +0000695
Fred Drake114b8ca2005-03-21 05:47:11 +0000696\end{itemize}
Andrew M. Kuchlinge9b1bf42005-03-20 19:26:30 +0000697
Fred Drake2db76802004-12-01 05:05:47 +0000698
699
700%======================================================================
701% whole new modules get described in \subsections here
702
Andrew M. Kuchling150e3492005-08-23 00:56:06 +0000703% XXX new distutils features: upload
704
Fredrik Lundh7e0aef02005-12-12 18:54:55 +0000705% XXX should hashlib perhaps be described here instead?
706% XXX should xml.etree perhaps be described here instead?
Andrew M. Kuchling150e3492005-08-23 00:56:06 +0000707
708
Fred Drake2db76802004-12-01 05:05:47 +0000709
710% ======================================================================
711\section{Build and C API Changes}
712
713Changes to Python's build process and to the C API include:
714
715\begin{itemize}
716
Andrew M. Kuchlingdb85ed52005-10-23 21:52:59 +0000717\item The design of the bytecode compiler has changed a great deal, no
718longer generating bytecode by traversing the parse tree. Instead
719the parse tree is converted to an abstract syntax tree (or AST), and it is
720the abstract syntax tree that's traversed to produce the bytecode.
721
722No documentation has been written for the AST code yet. To start
723learning about it, read the definition of the various AST nodes in
724\file{Parser/Python.asdl}. A Python script reads this file and
725generates a set of C structure definitions in
726\file{Include/Python-ast.h}. The \cfunction{PyParser_ASTFromString()}
727and \cfunction{PyParser_ASTFromFile()}, defined in
728\file{Include/pythonrun.h}, take Python source as input and return the
729root of an AST representing the contents. This AST can then be turned
730into a code object by \cfunction{PyAST_Compile()}. For more
731information, read the source code, and then ask questions on
732python-dev.
733
734% List of names taken from Jeremy's python-dev post at
735% http://mail.python.org/pipermail/python-dev/2005-October/057500.html
736The AST code was developed under Jeremy Hylton's management, and
737implemented by (in alphabetical order) Brett Cannon, Nick Coghlan,
738Grant Edwards, John Ehresman, Kurt Kaiser, Neal Norwitz, Tim Peters,
739Armin Rigo, and Neil Schemenauer, plus the participants in a number of
740AST sprints at conferences such as PyCon.
741
Andrew M. Kuchling150e3492005-08-23 00:56:06 +0000742\item The built-in set types now have an official C API. Call
743\cfunction{PySet_New()} and \cfunction{PyFrozenSet_New()} to create a
744new set, \cfunction{PySet_Add()} and \cfunction{PySet_Discard()} to
745add and remove elements, and \cfunction{PySet_Contains} and
746\cfunction{PySet_Size} to examine the set's state.
747
748\item The \cfunction{PyRange_New()} function was removed. It was
749never documented, never used in the core code, and had dangerously lax
750error checking.
Fred Drake2db76802004-12-01 05:05:47 +0000751
752\end{itemize}
753
754
755%======================================================================
756\subsection{Port-Specific Changes}
757
758Platform-specific changes go here.
759
760
761%======================================================================
762\section{Other Changes and Fixes \label{section-other}}
763
764As usual, there were a bunch of other improvements and bugfixes
Andrew M. Kuchlingf688cc52006-03-10 18:50:08 +0000765scattered throughout the source tree. A search through the SVN change
Fred Drake2db76802004-12-01 05:05:47 +0000766logs finds there were XXX patches applied and YYY bugs fixed between
Andrew M. Kuchling92e24952004-12-03 13:54:09 +0000767Python 2.4 and 2.5. Both figures are likely to be underestimates.
Fred Drake2db76802004-12-01 05:05:47 +0000768
769Some of the more notable changes are:
770
771\begin{itemize}
772
Andrew M. Kuchling01e3d262006-03-17 15:38:39 +0000773\item Evan Jones's patch to obmalloc, first described in a talk
774at PyCon DC 2005, was applied. Python 2.4 allocated small objects in
775256K-sized arenas, but never freed arenas. With this patch, Python
776will free arenas when they're empty. The net effect is that on some
777platforms, when you allocate many objects, Python's memory usage may
778actually drop when you delete them, and the memory may be returned to
779the operating system. (Implemented by Evan Jones, and reworked by Tim
780Peters.)
Fred Drake2db76802004-12-01 05:05:47 +0000781
782\end{itemize}
783
784
785%======================================================================
786\section{Porting to Python 2.5}
787
788This section lists previously described changes that may require
789changes to your code:
790
791\begin{itemize}
792
Andrew M. Kuchling3e41b052005-03-01 00:53:46 +0000793% the pickle module no longer uses the deprecated bin parameter.
Fred Drake2db76802004-12-01 05:05:47 +0000794
795\end{itemize}
796
797
798%======================================================================
799\section{Acknowledgements \label{acks}}
800
801The author would like to thank the following people for offering
802suggestions, corrections and assistance with various drafts of this
803article: .
804
805\end{document}