| \documentclass{howto} |
| \usepackage{distutils} |
| % $Id$ |
| |
| |
| \title{What's New in Python 2.5} |
| \release{0.0} |
| \author{A.M. Kuchling} |
| \authoraddress{\email{amk@amk.ca}} |
| |
| \begin{document} |
| \maketitle |
| \tableofcontents |
| |
| This article explains the new features in Python 2.5. No release date |
| for Python 2.5 has been set; it will probably be released in early 2006. |
| |
| % Compare with previous release in 2 - 3 sentences here. |
| |
| This article doesn't attempt to provide a complete specification of |
| the new features, but instead provides a convenient overview. For |
| full details, you should refer to the documentation for Python 2.5. |
| % add hyperlink when the documentation becomes available online. |
| If you want to understand the complete implementation and design |
| rationale, refer to the PEP for a particular new feature. |
| |
| |
| %====================================================================== |
| \section{PEP 309: Partial Function Application} |
| |
| The \module{functional} module is intended to contain tools for |
| functional-style programming. Currently it only contains |
| \class{partial}, but new functions will probably be added in future |
| versions of Python. |
| |
| For programs written in a functional style, it can be useful to |
| construct variants of existing functions that have some of the |
| parameters filled in. Consider a Python function \code{f(a, b, c)}; |
| you could create a new function \code{g(b, c)} that was equivalent to |
| \code{f(1, b, c)}. This is called ``partial function application'', |
| and is provided by the \class{partial} class in the new |
| \module{functional} module. |
| |
| The constructor for \class{partial} takes the arguments |
| \code{(\var{function}, \var{arg1}, \var{arg2}, ... |
| \var{kwarg1}=\var{value1}, \var{kwarg2}=\var{value2})}. The resulting |
| object is callable, so you can just call it to invoke \var{function} |
| with the filled-in arguments. |
| |
| Here's a small but realistic example: |
| |
| \begin{verbatim} |
| import functional |
| |
| def log (message, subsystem): |
| "Write the contents of 'message' to the specified subsystem." |
| print '%s: %s' % (subsystem, message) |
| ... |
| |
| server_log = functional.partial(log, subsystem='server') |
| \end{verbatim} |
| |
| Here's another example, from a program that uses PyGTk. Here a |
| context-sensitive pop-up menu is being constructed dynamically. The |
| callback provided for the menu option is a partially applied version |
| of the \method{open_item()} method, where the first argument has been |
| provided. |
| |
| \begin{verbatim} |
| ... |
| class Application: |
| def open_item(self, path): |
| ... |
| def init (self): |
| open_func = functional.partial(self.open_item, item_path) |
| popup_menu.append( ("Open", open_func, 1) ) |
| \end{verbatim} |
| |
| |
| \begin{seealso} |
| |
| \seepep{309}{Partial Function Application}{PEP proposed and written by |
| Peter Harris; implemented by Hye-Shik Chang, with adaptations by |
| Raymond Hettinger.} |
| |
| \end{seealso} |
| |
| |
| %====================================================================== |
| \section{PEP 314: Metadata for Python Software Packages v1.1} |
| |
| Some simple dependency support was added to Distutils. The |
| \function{setup()} function now has \code{requires},\code{provides}, |
| and \code{obsoletes}. When you build a source distribution using the |
| \code{sdist} command, the dependency information will be recorded in |
| the \file{PKG-INFO} file. |
| |
| Another new keyword is \code{download_url}, which should be set to a |
| URL for the package's source code. This means it's now possible to |
| look up an entry in the package index, determine the dependencies for |
| a package, and download the required packages. |
| |
| % XXX put example here |
| |
| \begin{seealso} |
| |
| \seepep{314}{Metadata for Python Software Packages v1.1}{PEP proposed |
| and written by A.M. Kuchling, Richard Jones, and Fred Drake; |
| implemented by Richard Jones and Fred Drake.} |
| |
| \end{seealso} |
| |
| |
| %====================================================================== |
| \section{PEP 342: New Generator Features} |
| |
| As introduced in Python 2.3, generators only produce output; once a |
| generator's code was invoked to create an iterator, there's no way to |
| pass new parameters into the function when its execution is resumed. |
| Hackish solutions to this include making the generator's code look at |
| a global variable and then changing the global variable's value, or |
| passing in some mutable object that callers then modify. Python |
| 2.5 adds the ability to pass values \emph{into} a generator. |
| |
| To refresh your memory of basic generators, here's a simple example: |
| |
| \begin{verbatim} |
| def counter (maximum): |
| i = 0 |
| while i < maximum: |
| yield i |
| i += 1 |
| \end{verbatim} |
| |
| When you call \code{counter(10)}, the result is an iterator that |
| returns the values from 0 up to 9. On encountering the |
| \keyword{yield} statement, the iterator returns the provided value and |
| suspends the function's execution, preserving the local variables. |
| Execution resumes on the following call to the iterator's |
| \method{next()} method, picking up after the \keyword{yield}. |
| |
| In Python 2.3, \keyword{yield} was a statement; it didn't return any |
| value. In 2.5, \keyword{yield} is now an expression, returning a |
| value that can be assigned to a variable or otherwise operated on: |
| |
| \begin{verbatim} |
| val = (yield i) |
| \end{verbatim} |
| |
| I recommend that you always put parentheses around a \keyword{yield} |
| expression when you're doing something with the returned value, as in |
| the above example. The parentheses aren't always necessary, but it's |
| easier to always add them instead of having to remember when they're |
| needed. The exact rules are that a \keyword{yield}-expression must |
| always be parenthesized except when it occurs at the top-level |
| expression on the right-hand side of an assignment, meaning |
| you can to write \code{val = yield i} but \code{val = (yield i) + 12}. |
| |
| Values are sent into a generator by calling its |
| \method{send(\var{value})} method. The generator's code is then |
| resumed and the \keyword{yield} expression produces \var{value}. |
| If the regular \method{next()} method is called, the \keyword{yield} |
| returns \constant{None}. |
| |
| Here's the previous example, modified to allow changing the value of |
| the internal counter. |
| |
| \begin{verbatim} |
| def counter (maximum): |
| i = 0 |
| while i < maximum: |
| val = (yield i) |
| # If value provided, change counter |
| if val is not None: |
| i = val |
| else: |
| i += 1 |
| \end{verbatim} |
| |
| And here's an example of changing the counter: |
| |
| \begin{verbatim} |
| >>> it = counter(10) |
| >>> print it.next() |
| 0 |
| >>> print it.next() |
| 1 |
| >>> print it.send(8) |
| 8 |
| >>> print it.next() |
| 9 |
| >>> print it.next() |
| Traceback (most recent call last): |
| File ``t.py'', line 15, in ? |
| print it.next() |
| StopIteration |
| \end{verbatim} |
| |
| Because \keyword{yield} will often be returning \constant{None}, |
| you shouldn't just use its value in expressions unless you're sure |
| that only the \method{send()} method will be used. |
| |
| There are two other new methods on generators in addition to |
| \method{send()}: |
| |
| \begin{itemize} |
| |
| \item \method{throw(\var{type}, \var{value}=None, |
| \var{traceback}=None)} is used to raise an exception inside the |
| generator; the exception is raised by the \keyword{yield} expression |
| where the generator's execution is paused. |
| |
| \item \method{close()} raises a new \exception{GeneratorExit} |
| exception inside the generator to terminate the iteration. |
| On receiving this |
| exception, the generator's code must either raise |
| \exception{GeneratorExit} or \exception{StopIteration}; catching the |
| exception and doing anything else is illegal and will trigger |
| a \exception{RuntimeError}. \method{close()} will also be called by |
| Python's garbage collection when the generator is garbage-collected. |
| |
| If you need to run cleanup code in case of a \exception{GeneratorExit}, |
| I suggest using a \code{try: ... finally:} suite instead of |
| catching \exception{GeneratorExit}. |
| |
| \end{itemize} |
| |
| The cumulative effect of these changes is to turn generators from |
| one-way producers of information into both producers and consumers. |
| Generators also become \emph{coroutines}, a more generalized form of |
| subroutines; subroutines are entered at one point and exited at |
| another point (the top of the function, and a \keyword{return |
| statement}), but coroutines can be entered, exited, and resumed at |
| many different points (the \keyword{yield} statements).science term |
| |
| |
| \begin{seealso} |
| |
| \seepep{342}{Coroutines via Enhanced Generators}{PEP written by |
| Guido van Rossum and Phillip J. Eby; |
| implemented by Phillip J. Eby. Includes examples of |
| some fancier uses of generators as coroutines.} |
| |
| \seeurl{http://en.wikipedia.org/wiki/Coroutine}{The Wikipedia entry for |
| coroutines.} |
| |
| \seeurl{http://www.sidhe.org/~dan/blog/archives/000178.html}{An |
| explanation of coroutines from a Perl point of view, written by Dan |
| Sugalski.} |
| |
| \end{seealso} |
| |
| |
| %====================================================================== |
| \section{Other Language Changes} |
| |
| Here are all of the changes that Python 2.5 makes to the core Python |
| language. |
| |
| \begin{itemize} |
| |
| \item The \function{min()} and \function{max()} built-in functions |
| gained a \code{key} keyword argument analogous to the \code{key} |
| argument for \method{sort()}. This argument supplies a function |
| that takes a single argument and is called for every value in the list; |
| \function{min()}/\function{max()} will return the element with the |
| smallest/largest return value from this function. |
| For example, to find the longest string in a list, you can do: |
| |
| \begin{verbatim} |
| L = ['medium', 'longest', 'short'] |
| # Prints 'longest' |
| print max(L, key=len) |
| # Prints 'short', because lexicographically 'short' has the largest value |
| print max(L) |
| \end{verbatim} |
| |
| (Contributed by Steven Bethard and Raymond Hettinger.) |
| |
| \item Two new built-in functions, \function{any()} and |
| \function{all()}, evaluate whether an iterator contains any true or |
| false values. \function{any()} returns \constant{True} if any value |
| returned by the iterator is true; otherwise it will return |
| \constant{False}. \function{all()} returns \constant{True} only if |
| all of the values returned by the iterator evaluate as being true. |
| |
| % XXX who added? |
| |
| |
| \item The list of base classes in a class definition can now be empty. |
| As an example, this is now legal: |
| |
| \begin{verbatim} |
| class C(): |
| pass |
| \end{verbatim} |
| (Implemented by Brett Cannon.) |
| |
| \end{itemize} |
| |
| |
| %====================================================================== |
| \subsection{Optimizations} |
| |
| \begin{itemize} |
| |
| \item When they were introduced |
| in Python 2.4, the built-in \class{set} and \class{frozenset} types |
| were built on top of Python's dictionary type. |
| In 2.5 the internal data structure has been customized for implementing sets, |
| and as a result sets will use a third less memory and are somewhat faster. |
| (Implemented by Raymond Hettinger.) |
| |
| \end{itemize} |
| |
| The net result of the 2.5 optimizations is that Python 2.5 runs the |
| pystone benchmark around XX\% faster than Python 2.4. |
| |
| |
| %====================================================================== |
| \section{New, Improved, and Deprecated Modules} |
| |
| As usual, Python's standard library received a number of enhancements and |
| bug fixes. Here's a partial list of the most notable changes, sorted |
| alphabetically by module name. Consult the |
| \file{Misc/NEWS} file in the source tree for a more |
| complete list of changes, or look through the CVS logs for all the |
| details. |
| |
| \begin{itemize} |
| |
| % collections.deque now has .remove() |
| |
| % the cPickle module no longer accepts the deprecated None option in the |
| % args tuple returned by __reduce__(). |
| |
| % csv module improvements |
| |
| % datetime.datetime() now has a strptime class method which can be used to |
| % create datetime object using a string and format. |
| |
| \item A new \module{hashlib} module has been added to replace the |
| \module{md5} and \module{sha} modules. \module{hashlib} adds support |
| for additional secure hashes (SHA-224, SHA-256, SHA-384, and SHA-512). |
| When available, the module uses OpenSSL for fast platform optimized |
| implementations of algorithms. The old \module{md5} and \module{sha} |
| modules still exist as wrappers around hashlib to preserve backwards |
| compatibility. (Contributed by Gregory P. Smith.) |
| |
| \item The \function{nsmallest()} and |
| \function{nlargest()} functions in the \module{heapq} module |
| now support a \code{key} keyword argument similar to the one |
| provided by the \function{min()}/\function{max()} functions |
| and the \method{sort()} methods. For example: |
| Example: |
| |
| \begin{verbatim} |
| >>> import heapq |
| >>> L = ["short", 'medium', 'longest', 'longer still'] |
| >>> heapq.nsmallest(2, L) # Return two lowest elements, lexicographically |
| ['longer still', 'longest'] |
| >>> heapq.nsmallest(2, L, key=len) # Return two shortest elements |
| ['short', 'medium'] |
| \end{verbatim} |
| |
| (Contributed by Raymond Hettinger.) |
| |
| \item The \function{itertools.islice()} function now accepts |
| \code{None} for the start and step arguments. This makes it more |
| compatible with the attributes of slice objects, so that you can now write |
| the following: |
| |
| \begin{verbatim} |
| s = slice(5) # Create slice object |
| itertools.islice(iterable, s.start, s.stop, s.step) |
| \end{verbatim} |
| |
| (Contributed by Raymond Hettinger.) |
| |
| \item The \module{operator} module's \function{itemgetter()} |
| and \function{attrgetter()} functions now support multiple fields. |
| A call such as \code{operator.attrgetter('a', 'b')} |
| will return a function |
| that retrieves the \member{a} and \member{b} attributes. Combining |
| this new feature with the \method{sort()} method's \code{key} parameter |
| lets you easily sort lists using multiple fields. |
| |
| % XXX who added? |
| |
| |
| \item The \module{os} module underwent a number of changes. The |
| \member{stat_float_times} variable now defaults to true, meaning that |
| \function{os.stat()} will now return time values as floats. (This |
| doesn't necessarily mean that \function{os.stat()} will return times |
| that are precise to fractions of a second; not all systems support |
| such precision.) |
| |
| Constants named \member{os.SEEK_SET}, \member{os.SEEK_CUR}, and |
| \member{os.SEEK_END} have been added; these are the parameters to the |
| \function{os.lseek()} function. Two new constants for locking are |
| \member{os.O_SHLOCK} and \member{os.O_EXLOCK}. |
| |
| On FreeBSD, the \function{os.stat()} function now returns |
| times with nanosecond resolution, and the returned object |
| now has \member{st_gen} and \member{st_birthtime}. |
| The \member{st_flags} member is also available, if the platform supports it. |
| % XXX patch 1180695, 1212117 |
| |
| \item The \module{socket} module now supports \constant{AF_NETLINK} |
| sockets on Linux, thanks to a patch from Philippe Biondi. |
| Netlink sockets are a Linux-specific mechanism for communications |
| between a user-space process and kernel code; an introductory |
| article about them is at \url{http://www.linuxjournal.com/article/7356}. |
| In Python code, netlink addresses are represented as a tuple of 2 integers, |
| \code{(\var{pid}, \var{group_mask})}. |
| |
| \item New module: \module{spwd} provides functions for accessing the |
| shadow password database on systems that support it. |
| % XXX give example |
| |
| \item The \class{TarFile} class in the \module{tarfile} module now has |
| an \method{extractall()} method that extracts all members from the |
| archive into the current working directory. It's also possible to set |
| a different directory as the extraction target, and to unpack only a |
| subset of the archive's members. |
| |
| A tarfile's compression can be autodetected by |
| using the mode \code{'r|*'}. |
| % patch 918101 |
| (Contributed by Lars Gust\"abel.) |
| |
| \item A new package \module{xml.etree} has been added, which contains |
| a subset of the ElementTree XML library. Available modules are |
| \module{ElementTree}, \module{ElementPath}, and |
| \module{ElementInclude}, from ElementTree 1.2.6. (Contributed by |
| Fredrik Lundh.) |
| |
| \item The \module{xmlrpclib} module now supports returning |
| \class{datetime} objects for the XML-RPC date type. Supply |
| \code{use_datetime=True} to the \function{loads()} function |
| or the \class{Unmarshaller} class to enable this feature. |
| % XXX patch 1120353 |
| |
| |
| \end{itemize} |
| |
| |
| |
| %====================================================================== |
| % whole new modules get described in \subsections here |
| |
| % XXX new distutils features: upload |
| |
| % XXX should hashlib perhaps be described here instead? |
| % XXX should xml.etree perhaps be described here instead? |
| |
| |
| |
| % ====================================================================== |
| \section{Build and C API Changes} |
| |
| Changes to Python's build process and to the C API include: |
| |
| \begin{itemize} |
| |
| \item The design of the bytecode compiler has changed a great deal, no |
| longer generating bytecode by traversing the parse tree. Instead |
| the parse tree is converted to an abstract syntax tree (or AST), and it is |
| the abstract syntax tree that's traversed to produce the bytecode. |
| |
| No documentation has been written for the AST code yet. To start |
| learning about it, read the definition of the various AST nodes in |
| \file{Parser/Python.asdl}. A Python script reads this file and |
| generates a set of C structure definitions in |
| \file{Include/Python-ast.h}. The \cfunction{PyParser_ASTFromString()} |
| and \cfunction{PyParser_ASTFromFile()}, defined in |
| \file{Include/pythonrun.h}, take Python source as input and return the |
| root of an AST representing the contents. This AST can then be turned |
| into a code object by \cfunction{PyAST_Compile()}. For more |
| information, read the source code, and then ask questions on |
| python-dev. |
| |
| % List of names taken from Jeremy's python-dev post at |
| % http://mail.python.org/pipermail/python-dev/2005-October/057500.html |
| The AST code was developed under Jeremy Hylton's management, and |
| implemented by (in alphabetical order) Brett Cannon, Nick Coghlan, |
| Grant Edwards, John Ehresman, Kurt Kaiser, Neal Norwitz, Tim Peters, |
| Armin Rigo, and Neil Schemenauer, plus the participants in a number of |
| AST sprints at conferences such as PyCon. |
| |
| \item The built-in set types now have an official C API. Call |
| \cfunction{PySet_New()} and \cfunction{PyFrozenSet_New()} to create a |
| new set, \cfunction{PySet_Add()} and \cfunction{PySet_Discard()} to |
| add and remove elements, and \cfunction{PySet_Contains} and |
| \cfunction{PySet_Size} to examine the set's state. |
| |
| \item The \cfunction{PyRange_New()} function was removed. It was |
| never documented, never used in the core code, and had dangerously lax |
| error checking. |
| |
| \end{itemize} |
| |
| |
| %====================================================================== |
| \subsection{Port-Specific Changes} |
| |
| Platform-specific changes go here. |
| |
| |
| %====================================================================== |
| \section{Other Changes and Fixes \label{section-other}} |
| |
| As usual, there were a bunch of other improvements and bugfixes |
| scattered throughout the source tree. A search through the CVS change |
| logs finds there were XXX patches applied and YYY bugs fixed between |
| Python 2.4 and 2.5. Both figures are likely to be underestimates. |
| |
| Some of the more notable changes are: |
| |
| \begin{itemize} |
| |
| \item Details go here. |
| |
| \end{itemize} |
| |
| |
| %====================================================================== |
| \section{Porting to Python 2.5} |
| |
| This section lists previously described changes that may require |
| changes to your code: |
| |
| \begin{itemize} |
| |
| \item Some old deprecated modules (\module{statcache}, \module{tzparse}, |
| \module{whrandom}) have been moved to \file{Lib/lib-old}. |
| You can get access to these modules again by adding the directory |
| to your \code{sys.path}: |
| |
| \begin{verbatim} |
| import os |
| from distutils import sysconfig |
| |
| lib_dir = sysconfig.get_python_lib(standard_lib=True) |
| old_dir = os.path.join(lib_dir, 'lib-old') |
| sys.path.append(old_dir) |
| \end{verbatim} |
| |
| Doing so is discouraged, however; it's better to update any code that |
| still uses these modules. |
| |
| % the pickle module no longer uses the deprecated bin parameter. |
| |
| \end{itemize} |
| |
| |
| %====================================================================== |
| \section{Acknowledgements \label{acks}} |
| |
| The author would like to thank the following people for offering |
| suggestions, corrections and assistance with various drafts of this |
| article: . |
| |
| \end{document} |