blob: 7f405213db9d5a7acf4c4b496bb897a74f424962 [file] [log] [blame]
Georg Brandl116aa622007-08-15 14:28:22 +00001****************************
Georg Brandl48310cd2009-01-03 21:18:54 +00002 What's New in Python 2.1
Georg Brandl116aa622007-08-15 14:28:22 +00003****************************
4
5:Author: A.M. Kuchling
6
7.. |release| replace:: 1.01
8
Christian Heimes5b5e81c2007-12-31 16:14:33 +00009.. $Id: whatsnew21.tex 50964 2006-07-30 03:03:43Z fred.drake $
Georg Brandl116aa622007-08-15 14:28:22 +000010
11
12Introduction
13============
14
15This article explains the new features in Python 2.1. While there aren't as
16many changes in 2.1 as there were in Python 2.0, there are still some pleasant
17surprises in store. 2.1 is the first release to be steered through the use of
18Python Enhancement Proposals, or PEPs, so most of the sizable changes have
19accompanying PEPs that provide more complete documentation and a design
20rationale for the change. This article doesn't attempt to document the new
21features completely, but simply provides an overview of the new features for
22Python programmers. Refer to the Python 2.1 documentation, or to the specific
23PEP, for more details about any new feature that particularly interests you.
24
25One recent goal of the Python development team has been to accelerate the pace
26of new releases, with a new release coming every 6 to 9 months. 2.1 is the first
27release to come out at this faster pace, with the first alpha appearing in
28January, 3 months after the final version of 2.0 was released.
29
30The final release of Python 2.1 was made on April 17, 2001.
31
Christian Heimes5b5e81c2007-12-31 16:14:33 +000032.. ======================================================================
Georg Brandl116aa622007-08-15 14:28:22 +000033
34
35PEP 227: Nested Scopes
36======================
37
38The largest change in Python 2.1 is to Python's scoping rules. In Python 2.0,
39at any given time there are at most three namespaces used to look up variable
40names: local, module-level, and the built-in namespace. This often surprised
41people because it didn't match their intuitive expectations. For example, a
42nested recursive function definition doesn't work::
43
44 def f():
45 ...
46 def g(value):
47 ...
48 return g(value-1) + 1
49 ...
50
51The function :func:`g` will always raise a :exc:`NameError` exception, because
52the binding of the name ``g`` isn't in either its local namespace or in the
53module-level namespace. This isn't much of a problem in practice (how often do
54you recursively define interior functions like this?), but this also made using
55the :keyword:`lambda` statement clumsier, and this was a problem in practice.
56In code which uses :keyword:`lambda` you can often find local variables being
57copied by passing them as the default values of arguments. ::
58
59 def find(self, name):
60 "Return list of any entries equal to 'name'"
61 L = filter(lambda x, name=name: x == name,
62 self.list_attribute)
63 return L
64
65The readability of Python code written in a strongly functional style suffers
66greatly as a result.
67
68The most significant change to Python 2.1 is that static scoping has been added
69to the language to fix this problem. As a first effect, the ``name=name``
70default argument is now unnecessary in the above example. Put simply, when a
71given variable name is not assigned a value within a function (by an assignment,
72or the :keyword:`def`, :keyword:`class`, or :keyword:`import` statements),
73references to the variable will be looked up in the local namespace of the
74enclosing scope. A more detailed explanation of the rules, and a dissection of
75the implementation, can be found in the PEP.
76
77This change may cause some compatibility problems for code where the same
78variable name is used both at the module level and as a local variable within a
79function that contains further function definitions. This seems rather unlikely
80though, since such code would have been pretty confusing to read in the first
81place.
82
83One side effect of the change is that the ``from module import *`` and
84:keyword:`exec` statements have been made illegal inside a function scope under
85certain conditions. The Python reference manual has said all along that ``from
86module import *`` is only legal at the top level of a module, but the CPython
87interpreter has never enforced this before. As part of the implementation of
88nested scopes, the compiler which turns Python source into bytecodes has to
89generate different code to access variables in a containing scope. ``from
90module import *`` and :keyword:`exec` make it impossible for the compiler to
91figure this out, because they add names to the local namespace that are
92unknowable at compile time. Therefore, if a function contains function
93definitions or :keyword:`lambda` expressions with free variables, the compiler
94will flag this by raising a :exc:`SyntaxError` exception.
95
96To make the preceding explanation a bit clearer, here's an example::
97
98 x = 1
99 def f():
100 # The next line is a syntax error
Georg Brandl48310cd2009-01-03 21:18:54 +0000101 exec 'x=2'
Georg Brandl116aa622007-08-15 14:28:22 +0000102 def g():
103 return x
104
105Line 4 containing the :keyword:`exec` statement is a syntax error, since
106:keyword:`exec` would define a new local variable named ``x`` whose value should
107be accessed by :func:`g`.
108
109This shouldn't be much of a limitation, since :keyword:`exec` is rarely used in
110most Python code (and when it is used, it's often a sign of a poor design
111anyway).
112
113Compatibility concerns have led to nested scopes being introduced gradually; in
114Python 2.1, they aren't enabled by default, but can be turned on within a module
115by using a future statement as described in PEP 236. (See the following section
116for further discussion of PEP 236.) In Python 2.2, nested scopes will become
117the default and there will be no way to turn them off, but users will have had
118all of 2.1's lifetime to fix any breakage resulting from their introduction.
119
120
121.. seealso::
122
123 :pep:`227` - Statically Nested Scopes
124 Written and implemented by Jeremy Hylton.
125
Christian Heimes5b5e81c2007-12-31 16:14:33 +0000126.. ======================================================================
Georg Brandl116aa622007-08-15 14:28:22 +0000127
128
129PEP 236: __future__ Directives
130==============================
131
132The reaction to nested scopes was widespread concern about the dangers of
133breaking code with the 2.1 release, and it was strong enough to make the
134Pythoneers take a more conservative approach. This approach consists of
135introducing a convention for enabling optional functionality in release N that
136will become compulsory in release N+1.
137
138The syntax uses a ``from...import`` statement using the reserved module name
139:mod:`__future__`. Nested scopes can be enabled by the following statement::
140
141 from __future__ import nested_scopes
142
143While it looks like a normal :keyword:`import` statement, it's not; there are
144strict rules on where such a future statement can be put. They can only be at
145the top of a module, and must precede any Python code or regular
146:keyword:`import` statements. This is because such statements can affect how
147the Python bytecode compiler parses code and generates bytecode, so they must
148precede any statement that will result in bytecodes being produced.
149
150
151.. seealso::
152
153 :pep:`236` - Back to the :mod:`__future__`
154 Written by Tim Peters, and primarily implemented by Jeremy Hylton.
155
Christian Heimes5b5e81c2007-12-31 16:14:33 +0000156.. ======================================================================
Georg Brandl116aa622007-08-15 14:28:22 +0000157
158
159PEP 207: Rich Comparisons
160=========================
161
162In earlier versions, Python's support for implementing comparisons on user-
163defined classes and extension types was quite simple. Classes could implement a
164:meth:`__cmp__` method that was given two instances of a class, and could only
165return 0 if they were equal or +1 or -1 if they weren't; the method couldn't
166raise an exception or return anything other than a Boolean value. Users of
167Numeric Python often found this model too weak and restrictive, because in the
168number-crunching programs that numeric Python is used for, it would be more
169useful to be able to perform elementwise comparisons of two matrices, returning
170a matrix containing the results of a given comparison for each element. If the
171two matrices are of different sizes, then the compare has to be able to raise an
172exception to signal the error.
173
174In Python 2.1, rich comparisons were added in order to support this need.
175Python classes can now individually overload each of the ``<``, ``<=``, ``>``,
176``>=``, ``==``, and ``!=`` operations. The new magic method names are:
177
178+-----------+----------------+
179| Operation | Method name |
180+===========+================+
181| ``<`` | :meth:`__lt__` |
182+-----------+----------------+
183| ``<=`` | :meth:`__le__` |
184+-----------+----------------+
185| ``>`` | :meth:`__gt__` |
186+-----------+----------------+
187| ``>=`` | :meth:`__ge__` |
188+-----------+----------------+
189| ``==`` | :meth:`__eq__` |
190+-----------+----------------+
191| ``!=`` | :meth:`__ne__` |
192+-----------+----------------+
193
194(The magic methods are named after the corresponding Fortran operators ``.LT.``.
195``.LE.``, &c. Numeric programmers are almost certainly quite familiar with
196these names and will find them easy to remember.)
197
198Each of these magic methods is of the form ``method(self, other)``, where
199``self`` will be the object on the left-hand side of the operator, while
200``other`` will be the object on the right-hand side. For example, the
201expression ``A < B`` will cause ``A.__lt__(B)`` to be called.
202
203Each of these magic methods can return anything at all: a Boolean, a matrix, a
204list, or any other Python object. Alternatively they can raise an exception if
205the comparison is impossible, inconsistent, or otherwise meaningless.
206
207The built-in :func:`cmp(A,B)` function can use the rich comparison machinery,
208and now accepts an optional argument specifying which comparison operation to
209use; this is given as one of the strings ``"<"``, ``"<="``, ``">"``, ``">="``,
210``"=="``, or ``"!="``. If called without the optional third argument,
211:func:`cmp` will only return -1, 0, or +1 as in previous versions of Python;
212otherwise it will call the appropriate method and can return any Python object.
213
214There are also corresponding changes of interest to C programmers; there's a new
215slot ``tp_richcmp`` in type objects and an API for performing a given rich
216comparison. I won't cover the C API here, but will refer you to PEP 207, or to
2172.1's C API documentation, for the full list of related functions.
218
219
220.. seealso::
221
222 :pep:`207` - Rich Comparisions
223 Written by Guido van Rossum, heavily based on earlier work by David Ascher, and
224 implemented by Guido van Rossum.
225
Christian Heimes5b5e81c2007-12-31 16:14:33 +0000226.. ======================================================================
Georg Brandl116aa622007-08-15 14:28:22 +0000227
228
229PEP 230: Warning Framework
230==========================
231
232Over its 10 years of existence, Python has accumulated a certain number of
233obsolete modules and features along the way. It's difficult to know when a
234feature is safe to remove, since there's no way of knowing how much code uses it
235--- perhaps no programs depend on the feature, or perhaps many do. To enable
236removing old features in a more structured way, a warning framework was added.
237When the Python developers want to get rid of a feature, it will first trigger a
238warning in the next version of Python. The following Python version can then
239drop the feature, and users will have had a full release cycle to remove uses of
240the old feature.
241
242Python 2.1 adds the warning framework to be used in this scheme. It adds a
243:mod:`warnings` module that provide functions to issue warnings, and to filter
244out warnings that you don't want to be displayed. Third-party modules can also
245use this framework to deprecate old features that they no longer wish to
246support.
247
248For example, in Python 2.1 the :mod:`regex` module is deprecated, so importing
249it causes a warning to be printed::
250
251 >>> import regex
252 __main__:1: DeprecationWarning: the regex module
253 is deprecated; please use the re module
254 >>>
255
256Warnings can be issued by calling the :func:`warnings.warn` function::
257
258 warnings.warn("feature X no longer supported")
259
260The first parameter is the warning message; an additional optional parameters
261can be used to specify a particular warning category.
262
263Filters can be added to disable certain warnings; a regular expression pattern
264can be applied to the message or to the module name in order to suppress a
265warning. For example, you may have a program that uses the :mod:`regex` module
266and not want to spare the time to convert it to use the :mod:`re` module right
267now. The warning can be suppressed by calling ::
268
269 import warnings
270 warnings.filterwarnings(action = 'ignore',
271 message='.*regex module is deprecated',
272 category=DeprecationWarning,
273 module = '__main__')
274
275This adds a filter that will apply only to warnings of the class
276:class:`DeprecationWarning` triggered in the :mod:`__main__` module, and applies
277a regular expression to only match the message about the :mod:`regex` module
278being deprecated, and will cause such warnings to be ignored. Warnings can also
279be printed only once, printed every time the offending code is executed, or
280turned into exceptions that will cause the program to stop (unless the
281exceptions are caught in the usual way, of course).
282
283Functions were also added to Python's C API for issuing warnings; refer to PEP
284230 or to Python's API documentation for the details.
285
286
287.. seealso::
288
289 :pep:`5` - Guidelines for Language Evolution
290 Written by Paul Prescod, to specify procedures to be followed when removing old
291 features from Python. The policy described in this PEP hasn't been officially
292 adopted, but the eventual policy probably won't be too different from Prescod's
293 proposal.
294
295 :pep:`230` - Warning Framework
296 Written and implemented by Guido van Rossum.
297
Christian Heimes5b5e81c2007-12-31 16:14:33 +0000298.. ======================================================================
Georg Brandl116aa622007-08-15 14:28:22 +0000299
300
301PEP 229: New Build System
302=========================
303
304When compiling Python, the user had to go in and edit the :file:`Modules/Setup`
305file in order to enable various additional modules; the default set is
306relatively small and limited to modules that compile on most Unix platforms.
307This means that on Unix platforms with many more features, most notably Linux,
308Python installations often don't contain all useful modules they could.
309
310Python 2.0 added the Distutils, a set of modules for distributing and installing
311extensions. In Python 2.1, the Distutils are used to compile much of the
312standard library of extension modules, autodetecting which ones are supported on
313the current machine. It's hoped that this will make Python installations easier
314and more featureful.
315
316Instead of having to edit the :file:`Modules/Setup` file in order to enable
317modules, a :file:`setup.py` script in the top directory of the Python source
318distribution is run at build time, and attempts to discover which modules can be
319enabled by examining the modules and header files on the system. If a module is
320configured in :file:`Modules/Setup`, the :file:`setup.py` script won't attempt
321to compile that module and will defer to the :file:`Modules/Setup` file's
322contents. This provides a way to specific any strange command-line flags or
323libraries that are required for a specific platform.
324
325In another far-reaching change to the build mechanism, Neil Schemenauer
326restructured things so Python now uses a single makefile that isn't recursive,
327instead of makefiles in the top directory and in each of the :file:`Python/`,
328:file:`Parser/`, :file:`Objects/`, and :file:`Modules/` subdirectories. This
329makes building Python faster and also makes hacking the Makefiles clearer and
330simpler.
331
332
333.. seealso::
334
335 :pep:`229` - Using Distutils to Build Python
336 Written and implemented by A.M. Kuchling.
337
Christian Heimes5b5e81c2007-12-31 16:14:33 +0000338.. ======================================================================
Georg Brandl116aa622007-08-15 14:28:22 +0000339
340
341PEP 205: Weak References
342========================
343
344Weak references, available through the :mod:`weakref` module, are a minor but
345useful new data type in the Python programmer's toolbox.
346
347Storing a reference to an object (say, in a dictionary or a list) has the side
348effect of keeping that object alive forever. There are a few specific cases
349where this behaviour is undesirable, object caches being the most common one,
350and another being circular references in data structures such as trees.
351
352For example, consider a memoizing function that caches the results of another
353function :func:`f(x)` by storing the function's argument and its result in a
354dictionary::
355
356 _cache = {}
357 def memoize(x):
358 if _cache.has_key(x):
359 return _cache[x]
360
361 retval = f(x)
362
363 # Cache the returned object
364 _cache[x] = retval
365
366 return retval
367
368This version works for simple things such as integers, but it has a side effect;
369the ``_cache`` dictionary holds a reference to the return values, so they'll
370never be deallocated until the Python process exits and cleans up This isn't
371very noticeable for integers, but if :func:`f` returns an object, or a data
372structure that takes up a lot of memory, this can be a problem.
373
374Weak references provide a way to implement a cache that won't keep objects alive
375beyond their time. If an object is only accessible through weak references, the
376object will be deallocated and the weak references will now indicate that the
377object it referred to no longer exists. A weak reference to an object *obj* is
378created by calling ``wr = weakref.ref(obj)``. The object being referred to is
379returned by calling the weak reference as if it were a function: ``wr()``. It
380will return the referenced object, or ``None`` if the object no longer exists.
381
382This makes it possible to write a :func:`memoize` function whose cache doesn't
383keep objects alive, by storing weak references in the cache. ::
384
385 _cache = {}
386 def memoize(x):
387 if _cache.has_key(x):
388 obj = _cache[x]()
389 # If weak reference object still exists,
390 # return it
391 if obj is not None: return obj
392
393 retval = f(x)
394
395 # Cache a weak reference
396 _cache[x] = weakref.ref(retval)
397
398 return retval
399
400The :mod:`weakref` module also allows creating proxy objects which behave like
401weak references --- an object referenced only by proxy objects is deallocated --
402but instead of requiring an explicit call to retrieve the object, the proxy
403transparently forwards all operations to the object as long as the object still
404exists. If the object is deallocated, attempting to use a proxy will cause a
405:exc:`weakref.ReferenceError` exception to be raised. ::
406
407 proxy = weakref.proxy(obj)
408 proxy.attr # Equivalent to obj.attr
409 proxy.meth() # Equivalent to obj.meth()
410 del obj
411 proxy.attr # raises weakref.ReferenceError
412
413
414.. seealso::
415
416 :pep:`205` - Weak References
417 Written and implemented by Fred L. Drake, Jr.
418
Christian Heimes5b5e81c2007-12-31 16:14:33 +0000419.. ======================================================================
Georg Brandl116aa622007-08-15 14:28:22 +0000420
421
422PEP 232: Function Attributes
423============================
424
425In Python 2.1, functions can now have arbitrary information attached to them.
426People were often using docstrings to hold information about functions and
427methods, because the ``__doc__`` attribute was the only way of attaching any
428information to a function. For example, in the Zope Web application server,
429functions are marked as safe for public access by having a docstring, and in
430John Aycock's SPARK parsing framework, docstrings hold parts of the BNF grammar
431to be parsed. This overloading is unfortunate, since docstrings are really
432intended to hold a function's documentation; for example, it means you can't
433properly document functions intended for private use in Zope.
434
435Arbitrary attributes can now be set and retrieved on functions using the regular
436Python syntax::
437
438 def f(): pass
439
440 f.publish = 1
441 f.secure = 1
442 f.grammar = "A ::= B (C D)*"
443
444The dictionary containing attributes can be accessed as the function's
445:attr:`__dict__`. Unlike the :attr:`__dict__` attribute of class instances, in
446functions you can actually assign a new dictionary to :attr:`__dict__`, though
447the new value is restricted to a regular Python dictionary; you *can't* be
448tricky and set it to a :class:`UserDict` instance, or any other random object
449that behaves like a mapping.
450
451
452.. seealso::
453
454 :pep:`232` - Function Attributes
455 Written and implemented by Barry Warsaw.
456
Christian Heimes5b5e81c2007-12-31 16:14:33 +0000457.. ======================================================================
Georg Brandl116aa622007-08-15 14:28:22 +0000458
459
460PEP 235: Importing Modules on Case-Insensitive Platforms
461========================================================
462
463Some operating systems have filesystems that are case-insensitive, MacOS and
464Windows being the primary examples; on these systems, it's impossible to
465distinguish the filenames ``FILE.PY`` and ``file.py``, even though they do store
466the file's name in its original case (they're case-preserving, too).
467
468In Python 2.1, the :keyword:`import` statement will work to simulate case-
469sensitivity on case-insensitive platforms. Python will now search for the first
470case-sensitive match by default, raising an :exc:`ImportError` if no such file
471is found, so ``import file`` will not import a module named ``FILE.PY``. Case-
472insensitive matching can be requested by setting the :envvar:`PYTHONCASEOK`
473environment variable before starting the Python interpreter.
474
Christian Heimes5b5e81c2007-12-31 16:14:33 +0000475.. ======================================================================
Georg Brandl116aa622007-08-15 14:28:22 +0000476
477
478PEP 217: Interactive Display Hook
479=================================
480
481When using the Python interpreter interactively, the output of commands is
482displayed using the built-in :func:`repr` function. In Python 2.1, the variable
483:func:`sys.displayhook` can be set to a callable object which will be called
484instead of :func:`repr`. For example, you can set it to a special pretty-
485printing function::
486
487 >>> # Create a recursive data structure
488 ... L = [1,2,3]
489 >>> L.append(L)
490 >>> L # Show Python's default output
491 [1, 2, 3, [...]]
492 >>> # Use pprint.pprint() as the display function
493 ... import sys, pprint
494 >>> sys.displayhook = pprint.pprint
495 >>> L
496 [1, 2, 3, <Recursion on list with id=135143996>]
497 >>>
498
499
500.. seealso::
501
502 :pep:`217` - Display Hook for Interactive Use
503 Written and implemented by Moshe Zadka.
504
Christian Heimes5b5e81c2007-12-31 16:14:33 +0000505.. ======================================================================
Georg Brandl116aa622007-08-15 14:28:22 +0000506
507
508PEP 208: New Coercion Model
509===========================
510
511How numeric coercion is done at the C level was significantly modified. This
512will only affect the authors of C extensions to Python, allowing them more
513flexibility in writing extension types that support numeric operations.
514
515Extension types can now set the type flag ``Py_TPFLAGS_CHECKTYPES`` in their
516``PyTypeObject`` structure to indicate that they support the new coercion model.
517In such extension types, the numeric slot functions can no longer assume that
518they'll be passed two arguments of the same type; instead they may be passed two
519arguments of differing types, and can then perform their own internal coercion.
520If the slot function is passed a type it can't handle, it can indicate the
521failure by returning a reference to the ``Py_NotImplemented`` singleton value.
522The numeric functions of the other type will then be tried, and perhaps they can
523handle the operation; if the other type also returns ``Py_NotImplemented``, then
524a :exc:`TypeError` will be raised. Numeric methods written in Python can also
525return ``Py_NotImplemented``, causing the interpreter to act as if the method
526did not exist (perhaps raising a :exc:`TypeError`, perhaps trying another
527object's numeric methods).
528
529
530.. seealso::
531
532 :pep:`208` - Reworking the Coercion Model
533 Written and implemented by Neil Schemenauer, heavily based upon earlier work by
534 Marc-André Lemburg. Read this to understand the fine points of how numeric
535 operations will now be processed at the C level.
536
Christian Heimes5b5e81c2007-12-31 16:14:33 +0000537.. ======================================================================
Georg Brandl116aa622007-08-15 14:28:22 +0000538
539
540PEP 241: Metadata in Python Packages
541====================================
542
543A common complaint from Python users is that there's no single catalog of all
544the Python modules in existence. T. Middleton's Vaults of Parnassus at
545http://www.vex.net/parnassus/ are the largest catalog of Python modules, but
546registering software at the Vaults is optional, and many people don't bother.
547
548As a first small step toward fixing the problem, Python software packaged using
549the Distutils :command:`sdist` command will include a file named
550:file:`PKG-INFO` containing information about the package such as its name,
551version, and author (metadata, in cataloguing terminology). PEP 241 contains
552the full list of fields that can be present in the :file:`PKG-INFO` file. As
553people began to package their software using Python 2.1, more and more packages
554will include metadata, making it possible to build automated cataloguing systems
555and experiment with them. With the result experience, perhaps it'll be possible
556to design a really good catalog and then build support for it into Python 2.2.
557For example, the Distutils :command:`sdist` and :command:`bdist_\*` commands
558could support a :option:`upload` option that would automatically upload your
559package to a catalog server.
560
561You can start creating packages containing :file:`PKG-INFO` even if you're not
562using Python 2.1, since a new release of the Distutils will be made for users of
563earlier Python versions. Version 1.0.2 of the Distutils includes the changes
564described in PEP 241, as well as various bugfixes and enhancements. It will be
565available from the Distutils SIG at http://www.python.org/sigs/distutils-sig/.
566
567
568.. seealso::
569
570 :pep:`241` - Metadata for Python Software Packages
571 Written and implemented by A.M. Kuchling.
572
573 :pep:`243` - Module Repository Upload Mechanism
574 Written by Sean Reifschneider, this draft PEP describes a proposed mechanism for
575 uploading Python packages to a central server.
576
Christian Heimes5b5e81c2007-12-31 16:14:33 +0000577.. ======================================================================
Georg Brandl116aa622007-08-15 14:28:22 +0000578
579
580New and Improved Modules
581========================
582
583* Ka-Ping Yee contributed two new modules: :mod:`inspect.py`, a module for
584 getting information about live Python code, and :mod:`pydoc.py`, a module for
585 interactively converting docstrings to HTML or text. As a bonus,
586 :file:`Tools/scripts/pydoc`, which is now automatically installed, uses
587 :mod:`pydoc.py` to display documentation given a Python module, package, or
588 class name. For example, ``pydoc xml.dom`` displays the following::
589
590 Python Library Documentation: package xml.dom in xml
591
592 NAME
593 xml.dom - W3C Document Object Model implementation for Python.
594
595 FILE
596 /usr/local/lib/python2.1/xml/dom/__init__.pyc
597
598 DESCRIPTION
599 The Python mapping of the Document Object Model is documented in the
600 Python Library Reference in the section on the xml.dom package.
601
602 This package contains the following modules:
603 ...
604
605 :file:`pydoc` also includes a Tk-based interactive help browser. :file:`pydoc`
606 quickly becomes addictive; try it out!
607
608* Two different modules for unit testing were added to the standard library.
609 The :mod:`doctest` module, contributed by Tim Peters, provides a testing
610 framework based on running embedded examples in docstrings and comparing the
611 results against the expected output. PyUnit, contributed by Steve Purcell, is a
612 unit testing framework inspired by JUnit, which was in turn an adaptation of
613 Kent Beck's Smalltalk testing framework. See http://pyunit.sourceforge.net/ for
614 more information about PyUnit.
615
616* The :mod:`difflib` module contains a class, :class:`SequenceMatcher`, which
617 compares two sequences and computes the changes required to transform one
618 sequence into the other. For example, this module can be used to write a tool
619 similar to the Unix :program:`diff` program, and in fact the sample program
620 :file:`Tools/scripts/ndiff.py` demonstrates how to write such a script.
621
622* :mod:`curses.panel`, a wrapper for the panel library, part of ncurses and of
623 SYSV curses, was contributed by Thomas Gellekum. The panel library provides
624 windows with the additional feature of depth. Windows can be moved higher or
625 lower in the depth ordering, and the panel library figures out where panels
626 overlap and which sections are visible.
627
628* The PyXML package has gone through a few releases since Python 2.0, and Python
629 2.1 includes an updated version of the :mod:`xml` package. Some of the
630 noteworthy changes include support for Expat 1.2 and later versions, the ability
631 for Expat parsers to handle files in any encoding supported by Python, and
632 various bugfixes for SAX, DOM, and the :mod:`minidom` module.
633
634* Ping also contributed another hook for handling uncaught exceptions.
635 :func:`sys.excepthook` can be set to a callable object. When an exception isn't
636 caught by any :keyword:`try`...\ :keyword:`except` blocks, the exception will be
637 passed to :func:`sys.excepthook`, which can then do whatever it likes. At the
638 Ninth Python Conference, Ping demonstrated an application for this hook:
639 printing an extended traceback that not only lists the stack frames, but also
640 lists the function arguments and the local variables for each frame.
641
642* Various functions in the :mod:`time` module, such as :func:`asctime` and
643 :func:`localtime`, require a floating point argument containing the time in
644 seconds since the epoch. The most common use of these functions is to work with
645 the current time, so the floating point argument has been made optional; when a
646 value isn't provided, the current time will be used. For example, log file
647 entries usually need a string containing the current time; in Python 2.1,
648 ``time.asctime()`` can be used, instead of the lengthier
649 ``time.asctime(time.localtime(time.time()))`` that was previously required.
650
651 This change was proposed and implemented by Thomas Wouters.
652
653* The :mod:`ftplib` module now defaults to retrieving files in passive mode,
654 because passive mode is more likely to work from behind a firewall. This
655 request came from the Debian bug tracking system, since other Debian packages
656 use :mod:`ftplib` to retrieve files and then don't work from behind a firewall.
657 It's deemed unlikely that this will cause problems for anyone, because Netscape
658 defaults to passive mode and few people complain, but if passive mode is
659 unsuitable for your application or network setup, call :meth:`set_pasv(0)` on
660 FTP objects to disable passive mode.
661
662* Support for raw socket access has been added to the :mod:`socket` module,
663 contributed by Grant Edwards.
664
665* The :mod:`pstats` module now contains a simple interactive statistics browser
666 for displaying timing profiles for Python programs, invoked when the module is
667 run as a script. Contributed by Eric S. Raymond.
668
669* A new implementation-dependent function, :func:`sys._getframe([depth])`, has
670 been added to return a given frame object from the current call stack.
671 :func:`sys._getframe` returns the frame at the top of the call stack; if the
672 optional integer argument *depth* is supplied, the function returns the frame
673 that is *depth* calls below the top of the stack. For example,
674 ``sys._getframe(1)`` returns the caller's frame object.
675
676 This function is only present in CPython, not in Jython or the .NET
677 implementation. Use it for debugging, and resist the temptation to put it into
678 production code.
679
Christian Heimes5b5e81c2007-12-31 16:14:33 +0000680.. ======================================================================
Georg Brandl116aa622007-08-15 14:28:22 +0000681
682
683Other Changes and Fixes
684=======================
685
686There were relatively few smaller changes made in Python 2.1 due to the shorter
687release cycle. A search through the CVS change logs turns up 117 patches
688applied, and 136 bugs fixed; both figures are likely to be underestimates. Some
689of the more notable changes are:
690
691* A specialized object allocator is now optionally available, that should be
692 faster than the system :func:`malloc` and have less memory overhead. The
693 allocator uses C's :func:`malloc` function to get large pools of memory, and
694 then fulfills smaller memory requests from these pools. It can be enabled by
695 providing the :option:`--with-pymalloc` option to the :program:`configure`
696 script; see :file:`Objects/obmalloc.c` for the implementation details.
697
698 Authors of C extension modules should test their code with the object allocator
699 enabled, because some incorrect code may break, causing core dumps at runtime.
700 There are a bunch of memory allocation functions in Python's C API that have
701 previously been just aliases for the C library's :func:`malloc` and
702 :func:`free`, meaning that if you accidentally called mismatched functions, the
703 error wouldn't be noticeable. When the object allocator is enabled, these
704 functions aren't aliases of :func:`malloc` and :func:`free` any more, and
705 calling the wrong function to free memory will get you a core dump. For
706 example, if memory was allocated using :func:`PyMem_New`, it has to be freed
707 using :func:`PyMem_Del`, not :func:`free`. A few modules included with Python
708 fell afoul of this and had to be fixed; doubtless there are more third-party
709 modules that will have the same problem.
710
711 The object allocator was contributed by Vladimir Marangozov.
712
713* The speed of line-oriented file I/O has been improved because people often
714 complain about its lack of speed, and because it's often been used as a naïve
715 benchmark. The :meth:`readline` method of file objects has therefore been
716 rewritten to be much faster. The exact amount of the speedup will vary from
717 platform to platform depending on how slow the C library's :func:`getc` was, but
718 is around 66%, and potentially much faster on some particular operating systems.
719 Tim Peters did much of the benchmarking and coding for this change, motivated by
720 a discussion in comp.lang.python.
721
722 A new module and method for file objects was also added, contributed by Jeff
723 Epler. The new method, :meth:`xreadlines`, is similar to the existing
724 :func:`xrange` built-in. :func:`xreadlines` returns an opaque sequence object
725 that only supports being iterated over, reading a line on every iteration but
726 not reading the entire file into memory as the existing :meth:`readlines` method
727 does. You'd use it like this::
728
729 for line in sys.stdin.xreadlines():
730 # ... do something for each line ...
731 ...
732
733 For a fuller discussion of the line I/O changes, see the python-dev summary for
Ezio Melotti9e8346c2010-04-05 13:57:55 +0000734 January 1-15, 2001 at http://www.python.org/dev/summary/2001-01-1/.
Georg Brandl116aa622007-08-15 14:28:22 +0000735
736* A new method, :meth:`popitem`, was added to dictionaries to enable
737 destructively iterating through the contents of a dictionary; this can be faster
738 for large dictionaries because there's no need to construct a list containing
739 all the keys or values. ``D.popitem()`` removes a random ``(key, value)`` pair
740 from the dictionary ``D`` and returns it as a 2-tuple. This was implemented
741 mostly by Tim Peters and Guido van Rossum, after a suggestion and preliminary
742 patch by Moshe Zadka.
743
744* Modules can now control which names are imported when ``from module import *``
745 is used, by defining an ``__all__`` attribute containing a list of names that
746 will be imported. One common complaint is that if the module imports other
747 modules such as :mod:`sys` or :mod:`string`, ``from module import *`` will add
748 them to the importing module's namespace. To fix this, simply list the public
749 names in ``__all__``::
750
751 # List public names
752 __all__ = ['Database', 'open']
753
754 A stricter version of this patch was first suggested and implemented by Ben
755 Wolfson, but after some python-dev discussion, a weaker final version was
756 checked in.
757
758* Applying :func:`repr` to strings previously used octal escapes for
759 non-printable characters; for example, a newline was ``'\012'``. This was a
760 vestigial trace of Python's C ancestry, but today octal is of very little
761 practical use. Ka-Ping Yee suggested using hex escapes instead of octal ones,
762 and using the ``\n``, ``\t``, ``\r`` escapes for the appropriate characters,
763 and implemented this new formatting.
764
765* Syntax errors detected at compile-time can now raise exceptions containing the
766 filename and line number of the error, a pleasant side effect of the compiler
767 reorganization done by Jeremy Hylton.
768
769* C extensions which import other modules have been changed to use
770 :func:`PyImport_ImportModule`, which means that they will use any import hooks
771 that have been installed. This is also encouraged for third-party extensions
772 that need to import some other module from C code.
773
774* The size of the Unicode character database was shrunk by another 340K thanks
775 to Fredrik Lundh.
776
777* Some new ports were contributed: MacOS X (by Steven Majewski), Cygwin (by
778 Jason Tishler); RISCOS (by Dietmar Schwertberger); Unixware 7 (by Billy G.
779 Allie).
780
781And there's the usual list of minor bugfixes, minor memory leaks, docstring
782edits, and other tweaks, too lengthy to be worth itemizing; see the CVS logs for
783the full details if you want them.
784
Christian Heimes5b5e81c2007-12-31 16:14:33 +0000785.. ======================================================================
Georg Brandl116aa622007-08-15 14:28:22 +0000786
787
788Acknowledgements
789================
790
791The author would like to thank the following people for offering suggestions on
792various drafts of this article: Graeme Cross, David Goodger, Jay Graves, Michael
793Hudson, Marc-André Lemburg, Fredrik Lundh, Neil Schemenauer, Thomas Wouters.
794