Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1 | **************************** |
Georg Brandl | 48310cd | 2009-01-03 21:18:54 +0000 | [diff] [blame] | 2 | What's New in Python 2.1 |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 3 | **************************** |
| 4 | |
| 5 | :Author: A.M. Kuchling |
| 6 | |
| 7 | .. |release| replace:: 1.01 |
| 8 | |
Christian Heimes | 5b5e81c | 2007-12-31 16:14:33 +0000 | [diff] [blame] | 9 | .. $Id: whatsnew21.tex 50964 2006-07-30 03:03:43Z fred.drake $ |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 10 | |
| 11 | |
| 12 | Introduction |
| 13 | ============ |
| 14 | |
| 15 | This article explains the new features in Python 2.1. While there aren't as |
| 16 | many changes in 2.1 as there were in Python 2.0, there are still some pleasant |
| 17 | surprises in store. 2.1 is the first release to be steered through the use of |
| 18 | Python Enhancement Proposals, or PEPs, so most of the sizable changes have |
| 19 | accompanying PEPs that provide more complete documentation and a design |
| 20 | rationale for the change. This article doesn't attempt to document the new |
| 21 | features completely, but simply provides an overview of the new features for |
| 22 | Python programmers. Refer to the Python 2.1 documentation, or to the specific |
| 23 | PEP, for more details about any new feature that particularly interests you. |
| 24 | |
| 25 | One recent goal of the Python development team has been to accelerate the pace |
| 26 | of new releases, with a new release coming every 6 to 9 months. 2.1 is the first |
| 27 | release to come out at this faster pace, with the first alpha appearing in |
| 28 | January, 3 months after the final version of 2.0 was released. |
| 29 | |
| 30 | The final release of Python 2.1 was made on April 17, 2001. |
| 31 | |
Christian Heimes | 5b5e81c | 2007-12-31 16:14:33 +0000 | [diff] [blame] | 32 | .. ====================================================================== |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 33 | |
| 34 | |
| 35 | PEP 227: Nested Scopes |
| 36 | ====================== |
| 37 | |
| 38 | The largest change in Python 2.1 is to Python's scoping rules. In Python 2.0, |
| 39 | at any given time there are at most three namespaces used to look up variable |
| 40 | names: local, module-level, and the built-in namespace. This often surprised |
| 41 | people because it didn't match their intuitive expectations. For example, a |
| 42 | nested recursive function definition doesn't work:: |
| 43 | |
| 44 | def f(): |
| 45 | ... |
| 46 | def g(value): |
| 47 | ... |
| 48 | return g(value-1) + 1 |
| 49 | ... |
| 50 | |
| 51 | The function :func:`g` will always raise a :exc:`NameError` exception, because |
| 52 | the binding of the name ``g`` isn't in either its local namespace or in the |
| 53 | module-level namespace. This isn't much of a problem in practice (how often do |
| 54 | you recursively define interior functions like this?), but this also made using |
| 55 | the :keyword:`lambda` statement clumsier, and this was a problem in practice. |
| 56 | In code which uses :keyword:`lambda` you can often find local variables being |
| 57 | copied by passing them as the default values of arguments. :: |
| 58 | |
| 59 | def find(self, name): |
| 60 | "Return list of any entries equal to 'name'" |
| 61 | L = filter(lambda x, name=name: x == name, |
| 62 | self.list_attribute) |
| 63 | return L |
| 64 | |
| 65 | The readability of Python code written in a strongly functional style suffers |
| 66 | greatly as a result. |
| 67 | |
| 68 | The most significant change to Python 2.1 is that static scoping has been added |
| 69 | to the language to fix this problem. As a first effect, the ``name=name`` |
| 70 | default argument is now unnecessary in the above example. Put simply, when a |
| 71 | given variable name is not assigned a value within a function (by an assignment, |
| 72 | or the :keyword:`def`, :keyword:`class`, or :keyword:`import` statements), |
| 73 | references to the variable will be looked up in the local namespace of the |
| 74 | enclosing scope. A more detailed explanation of the rules, and a dissection of |
| 75 | the implementation, can be found in the PEP. |
| 76 | |
| 77 | This change may cause some compatibility problems for code where the same |
| 78 | variable name is used both at the module level and as a local variable within a |
| 79 | function that contains further function definitions. This seems rather unlikely |
| 80 | though, since such code would have been pretty confusing to read in the first |
| 81 | place. |
| 82 | |
| 83 | One side effect of the change is that the ``from module import *`` and |
| 84 | :keyword:`exec` statements have been made illegal inside a function scope under |
| 85 | certain conditions. The Python reference manual has said all along that ``from |
| 86 | module import *`` is only legal at the top level of a module, but the CPython |
| 87 | interpreter has never enforced this before. As part of the implementation of |
| 88 | nested scopes, the compiler which turns Python source into bytecodes has to |
| 89 | generate different code to access variables in a containing scope. ``from |
| 90 | module import *`` and :keyword:`exec` make it impossible for the compiler to |
| 91 | figure this out, because they add names to the local namespace that are |
| 92 | unknowable at compile time. Therefore, if a function contains function |
| 93 | definitions or :keyword:`lambda` expressions with free variables, the compiler |
| 94 | will flag this by raising a :exc:`SyntaxError` exception. |
| 95 | |
| 96 | To make the preceding explanation a bit clearer, here's an example:: |
| 97 | |
| 98 | x = 1 |
| 99 | def f(): |
| 100 | # The next line is a syntax error |
Georg Brandl | 48310cd | 2009-01-03 21:18:54 +0000 | [diff] [blame] | 101 | exec 'x=2' |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 102 | def g(): |
| 103 | return x |
| 104 | |
| 105 | Line 4 containing the :keyword:`exec` statement is a syntax error, since |
| 106 | :keyword:`exec` would define a new local variable named ``x`` whose value should |
| 107 | be accessed by :func:`g`. |
| 108 | |
| 109 | This shouldn't be much of a limitation, since :keyword:`exec` is rarely used in |
| 110 | most Python code (and when it is used, it's often a sign of a poor design |
| 111 | anyway). |
| 112 | |
| 113 | Compatibility concerns have led to nested scopes being introduced gradually; in |
| 114 | Python 2.1, they aren't enabled by default, but can be turned on within a module |
| 115 | by using a future statement as described in PEP 236. (See the following section |
| 116 | for further discussion of PEP 236.) In Python 2.2, nested scopes will become |
| 117 | the default and there will be no way to turn them off, but users will have had |
| 118 | all of 2.1's lifetime to fix any breakage resulting from their introduction. |
| 119 | |
| 120 | |
| 121 | .. seealso:: |
| 122 | |
| 123 | :pep:`227` - Statically Nested Scopes |
| 124 | Written and implemented by Jeremy Hylton. |
| 125 | |
Christian Heimes | 5b5e81c | 2007-12-31 16:14:33 +0000 | [diff] [blame] | 126 | .. ====================================================================== |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 127 | |
| 128 | |
| 129 | PEP 236: __future__ Directives |
| 130 | ============================== |
| 131 | |
| 132 | The reaction to nested scopes was widespread concern about the dangers of |
| 133 | breaking code with the 2.1 release, and it was strong enough to make the |
| 134 | Pythoneers take a more conservative approach. This approach consists of |
| 135 | introducing a convention for enabling optional functionality in release N that |
| 136 | will become compulsory in release N+1. |
| 137 | |
| 138 | The syntax uses a ``from...import`` statement using the reserved module name |
| 139 | :mod:`__future__`. Nested scopes can be enabled by the following statement:: |
| 140 | |
| 141 | from __future__ import nested_scopes |
| 142 | |
| 143 | While it looks like a normal :keyword:`import` statement, it's not; there are |
| 144 | strict rules on where such a future statement can be put. They can only be at |
| 145 | the top of a module, and must precede any Python code or regular |
| 146 | :keyword:`import` statements. This is because such statements can affect how |
| 147 | the Python bytecode compiler parses code and generates bytecode, so they must |
| 148 | precede any statement that will result in bytecodes being produced. |
| 149 | |
| 150 | |
| 151 | .. seealso:: |
| 152 | |
| 153 | :pep:`236` - Back to the :mod:`__future__` |
| 154 | Written by Tim Peters, and primarily implemented by Jeremy Hylton. |
| 155 | |
Christian Heimes | 5b5e81c | 2007-12-31 16:14:33 +0000 | [diff] [blame] | 156 | .. ====================================================================== |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 157 | |
| 158 | |
| 159 | PEP 207: Rich Comparisons |
| 160 | ========================= |
| 161 | |
| 162 | In earlier versions, Python's support for implementing comparisons on user- |
| 163 | defined classes and extension types was quite simple. Classes could implement a |
| 164 | :meth:`__cmp__` method that was given two instances of a class, and could only |
| 165 | return 0 if they were equal or +1 or -1 if they weren't; the method couldn't |
| 166 | raise an exception or return anything other than a Boolean value. Users of |
| 167 | Numeric Python often found this model too weak and restrictive, because in the |
| 168 | number-crunching programs that numeric Python is used for, it would be more |
| 169 | useful to be able to perform elementwise comparisons of two matrices, returning |
| 170 | a matrix containing the results of a given comparison for each element. If the |
| 171 | two matrices are of different sizes, then the compare has to be able to raise an |
| 172 | exception to signal the error. |
| 173 | |
| 174 | In Python 2.1, rich comparisons were added in order to support this need. |
| 175 | Python classes can now individually overload each of the ``<``, ``<=``, ``>``, |
| 176 | ``>=``, ``==``, and ``!=`` operations. The new magic method names are: |
| 177 | |
| 178 | +-----------+----------------+ |
| 179 | | Operation | Method name | |
| 180 | +===========+================+ |
| 181 | | ``<`` | :meth:`__lt__` | |
| 182 | +-----------+----------------+ |
| 183 | | ``<=`` | :meth:`__le__` | |
| 184 | +-----------+----------------+ |
| 185 | | ``>`` | :meth:`__gt__` | |
| 186 | +-----------+----------------+ |
| 187 | | ``>=`` | :meth:`__ge__` | |
| 188 | +-----------+----------------+ |
| 189 | | ``==`` | :meth:`__eq__` | |
| 190 | +-----------+----------------+ |
| 191 | | ``!=`` | :meth:`__ne__` | |
| 192 | +-----------+----------------+ |
| 193 | |
| 194 | (The magic methods are named after the corresponding Fortran operators ``.LT.``. |
| 195 | ``.LE.``, &c. Numeric programmers are almost certainly quite familiar with |
| 196 | these names and will find them easy to remember.) |
| 197 | |
| 198 | Each of these magic methods is of the form ``method(self, other)``, where |
| 199 | ``self`` will be the object on the left-hand side of the operator, while |
| 200 | ``other`` will be the object on the right-hand side. For example, the |
| 201 | expression ``A < B`` will cause ``A.__lt__(B)`` to be called. |
| 202 | |
| 203 | Each of these magic methods can return anything at all: a Boolean, a matrix, a |
| 204 | list, or any other Python object. Alternatively they can raise an exception if |
| 205 | the comparison is impossible, inconsistent, or otherwise meaningless. |
| 206 | |
| 207 | The built-in :func:`cmp(A,B)` function can use the rich comparison machinery, |
| 208 | and now accepts an optional argument specifying which comparison operation to |
| 209 | use; this is given as one of the strings ``"<"``, ``"<="``, ``">"``, ``">="``, |
| 210 | ``"=="``, or ``"!="``. If called without the optional third argument, |
| 211 | :func:`cmp` will only return -1, 0, or +1 as in previous versions of Python; |
| 212 | otherwise it will call the appropriate method and can return any Python object. |
| 213 | |
| 214 | There are also corresponding changes of interest to C programmers; there's a new |
| 215 | slot ``tp_richcmp`` in type objects and an API for performing a given rich |
| 216 | comparison. I won't cover the C API here, but will refer you to PEP 207, or to |
| 217 | 2.1's C API documentation, for the full list of related functions. |
| 218 | |
| 219 | |
| 220 | .. seealso:: |
| 221 | |
| 222 | :pep:`207` - Rich Comparisions |
| 223 | Written by Guido van Rossum, heavily based on earlier work by David Ascher, and |
| 224 | implemented by Guido van Rossum. |
| 225 | |
Christian Heimes | 5b5e81c | 2007-12-31 16:14:33 +0000 | [diff] [blame] | 226 | .. ====================================================================== |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 227 | |
| 228 | |
| 229 | PEP 230: Warning Framework |
| 230 | ========================== |
| 231 | |
| 232 | Over its 10 years of existence, Python has accumulated a certain number of |
| 233 | obsolete modules and features along the way. It's difficult to know when a |
| 234 | feature is safe to remove, since there's no way of knowing how much code uses it |
| 235 | --- perhaps no programs depend on the feature, or perhaps many do. To enable |
| 236 | removing old features in a more structured way, a warning framework was added. |
| 237 | When the Python developers want to get rid of a feature, it will first trigger a |
| 238 | warning in the next version of Python. The following Python version can then |
| 239 | drop the feature, and users will have had a full release cycle to remove uses of |
| 240 | the old feature. |
| 241 | |
| 242 | Python 2.1 adds the warning framework to be used in this scheme. It adds a |
| 243 | :mod:`warnings` module that provide functions to issue warnings, and to filter |
| 244 | out warnings that you don't want to be displayed. Third-party modules can also |
| 245 | use this framework to deprecate old features that they no longer wish to |
| 246 | support. |
| 247 | |
| 248 | For example, in Python 2.1 the :mod:`regex` module is deprecated, so importing |
| 249 | it causes a warning to be printed:: |
| 250 | |
| 251 | >>> import regex |
| 252 | __main__:1: DeprecationWarning: the regex module |
| 253 | is deprecated; please use the re module |
| 254 | >>> |
| 255 | |
| 256 | Warnings can be issued by calling the :func:`warnings.warn` function:: |
| 257 | |
| 258 | warnings.warn("feature X no longer supported") |
| 259 | |
| 260 | The first parameter is the warning message; an additional optional parameters |
| 261 | can be used to specify a particular warning category. |
| 262 | |
| 263 | Filters can be added to disable certain warnings; a regular expression pattern |
| 264 | can be applied to the message or to the module name in order to suppress a |
| 265 | warning. For example, you may have a program that uses the :mod:`regex` module |
| 266 | and not want to spare the time to convert it to use the :mod:`re` module right |
| 267 | now. The warning can be suppressed by calling :: |
| 268 | |
| 269 | import warnings |
| 270 | warnings.filterwarnings(action = 'ignore', |
| 271 | message='.*regex module is deprecated', |
| 272 | category=DeprecationWarning, |
| 273 | module = '__main__') |
| 274 | |
| 275 | This adds a filter that will apply only to warnings of the class |
| 276 | :class:`DeprecationWarning` triggered in the :mod:`__main__` module, and applies |
| 277 | a regular expression to only match the message about the :mod:`regex` module |
| 278 | being deprecated, and will cause such warnings to be ignored. Warnings can also |
| 279 | be printed only once, printed every time the offending code is executed, or |
| 280 | turned into exceptions that will cause the program to stop (unless the |
| 281 | exceptions are caught in the usual way, of course). |
| 282 | |
| 283 | Functions were also added to Python's C API for issuing warnings; refer to PEP |
| 284 | 230 or to Python's API documentation for the details. |
| 285 | |
| 286 | |
| 287 | .. seealso:: |
| 288 | |
| 289 | :pep:`5` - Guidelines for Language Evolution |
| 290 | Written by Paul Prescod, to specify procedures to be followed when removing old |
| 291 | features from Python. The policy described in this PEP hasn't been officially |
| 292 | adopted, but the eventual policy probably won't be too different from Prescod's |
| 293 | proposal. |
| 294 | |
| 295 | :pep:`230` - Warning Framework |
| 296 | Written and implemented by Guido van Rossum. |
| 297 | |
Christian Heimes | 5b5e81c | 2007-12-31 16:14:33 +0000 | [diff] [blame] | 298 | .. ====================================================================== |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 299 | |
| 300 | |
| 301 | PEP 229: New Build System |
| 302 | ========================= |
| 303 | |
| 304 | When compiling Python, the user had to go in and edit the :file:`Modules/Setup` |
| 305 | file in order to enable various additional modules; the default set is |
| 306 | relatively small and limited to modules that compile on most Unix platforms. |
| 307 | This means that on Unix platforms with many more features, most notably Linux, |
| 308 | Python installations often don't contain all useful modules they could. |
| 309 | |
| 310 | Python 2.0 added the Distutils, a set of modules for distributing and installing |
| 311 | extensions. In Python 2.1, the Distutils are used to compile much of the |
| 312 | standard library of extension modules, autodetecting which ones are supported on |
| 313 | the current machine. It's hoped that this will make Python installations easier |
| 314 | and more featureful. |
| 315 | |
| 316 | Instead of having to edit the :file:`Modules/Setup` file in order to enable |
| 317 | modules, a :file:`setup.py` script in the top directory of the Python source |
| 318 | distribution is run at build time, and attempts to discover which modules can be |
| 319 | enabled by examining the modules and header files on the system. If a module is |
| 320 | configured in :file:`Modules/Setup`, the :file:`setup.py` script won't attempt |
| 321 | to compile that module and will defer to the :file:`Modules/Setup` file's |
| 322 | contents. This provides a way to specific any strange command-line flags or |
| 323 | libraries that are required for a specific platform. |
| 324 | |
| 325 | In another far-reaching change to the build mechanism, Neil Schemenauer |
| 326 | restructured things so Python now uses a single makefile that isn't recursive, |
| 327 | instead of makefiles in the top directory and in each of the :file:`Python/`, |
| 328 | :file:`Parser/`, :file:`Objects/`, and :file:`Modules/` subdirectories. This |
| 329 | makes building Python faster and also makes hacking the Makefiles clearer and |
| 330 | simpler. |
| 331 | |
| 332 | |
| 333 | .. seealso:: |
| 334 | |
| 335 | :pep:`229` - Using Distutils to Build Python |
| 336 | Written and implemented by A.M. Kuchling. |
| 337 | |
Christian Heimes | 5b5e81c | 2007-12-31 16:14:33 +0000 | [diff] [blame] | 338 | .. ====================================================================== |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 339 | |
| 340 | |
| 341 | PEP 205: Weak References |
| 342 | ======================== |
| 343 | |
| 344 | Weak references, available through the :mod:`weakref` module, are a minor but |
| 345 | useful new data type in the Python programmer's toolbox. |
| 346 | |
| 347 | Storing a reference to an object (say, in a dictionary or a list) has the side |
| 348 | effect of keeping that object alive forever. There are a few specific cases |
| 349 | where this behaviour is undesirable, object caches being the most common one, |
| 350 | and another being circular references in data structures such as trees. |
| 351 | |
| 352 | For example, consider a memoizing function that caches the results of another |
| 353 | function :func:`f(x)` by storing the function's argument and its result in a |
| 354 | dictionary:: |
| 355 | |
| 356 | _cache = {} |
| 357 | def memoize(x): |
| 358 | if _cache.has_key(x): |
| 359 | return _cache[x] |
| 360 | |
| 361 | retval = f(x) |
| 362 | |
| 363 | # Cache the returned object |
| 364 | _cache[x] = retval |
| 365 | |
| 366 | return retval |
| 367 | |
| 368 | This version works for simple things such as integers, but it has a side effect; |
| 369 | the ``_cache`` dictionary holds a reference to the return values, so they'll |
| 370 | never be deallocated until the Python process exits and cleans up This isn't |
| 371 | very noticeable for integers, but if :func:`f` returns an object, or a data |
| 372 | structure that takes up a lot of memory, this can be a problem. |
| 373 | |
| 374 | Weak references provide a way to implement a cache that won't keep objects alive |
| 375 | beyond their time. If an object is only accessible through weak references, the |
| 376 | object will be deallocated and the weak references will now indicate that the |
| 377 | object it referred to no longer exists. A weak reference to an object *obj* is |
| 378 | created by calling ``wr = weakref.ref(obj)``. The object being referred to is |
| 379 | returned by calling the weak reference as if it were a function: ``wr()``. It |
| 380 | will return the referenced object, or ``None`` if the object no longer exists. |
| 381 | |
| 382 | This makes it possible to write a :func:`memoize` function whose cache doesn't |
| 383 | keep objects alive, by storing weak references in the cache. :: |
| 384 | |
| 385 | _cache = {} |
| 386 | def memoize(x): |
| 387 | if _cache.has_key(x): |
| 388 | obj = _cache[x]() |
| 389 | # If weak reference object still exists, |
| 390 | # return it |
| 391 | if obj is not None: return obj |
| 392 | |
| 393 | retval = f(x) |
| 394 | |
| 395 | # Cache a weak reference |
| 396 | _cache[x] = weakref.ref(retval) |
| 397 | |
| 398 | return retval |
| 399 | |
| 400 | The :mod:`weakref` module also allows creating proxy objects which behave like |
| 401 | weak references --- an object referenced only by proxy objects is deallocated -- |
| 402 | but instead of requiring an explicit call to retrieve the object, the proxy |
| 403 | transparently forwards all operations to the object as long as the object still |
| 404 | exists. If the object is deallocated, attempting to use a proxy will cause a |
| 405 | :exc:`weakref.ReferenceError` exception to be raised. :: |
| 406 | |
| 407 | proxy = weakref.proxy(obj) |
| 408 | proxy.attr # Equivalent to obj.attr |
| 409 | proxy.meth() # Equivalent to obj.meth() |
| 410 | del obj |
| 411 | proxy.attr # raises weakref.ReferenceError |
| 412 | |
| 413 | |
| 414 | .. seealso:: |
| 415 | |
| 416 | :pep:`205` - Weak References |
| 417 | Written and implemented by Fred L. Drake, Jr. |
| 418 | |
Christian Heimes | 5b5e81c | 2007-12-31 16:14:33 +0000 | [diff] [blame] | 419 | .. ====================================================================== |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 420 | |
| 421 | |
| 422 | PEP 232: Function Attributes |
| 423 | ============================ |
| 424 | |
| 425 | In Python 2.1, functions can now have arbitrary information attached to them. |
| 426 | People were often using docstrings to hold information about functions and |
| 427 | methods, because the ``__doc__`` attribute was the only way of attaching any |
| 428 | information to a function. For example, in the Zope Web application server, |
| 429 | functions are marked as safe for public access by having a docstring, and in |
| 430 | John Aycock's SPARK parsing framework, docstrings hold parts of the BNF grammar |
| 431 | to be parsed. This overloading is unfortunate, since docstrings are really |
| 432 | intended to hold a function's documentation; for example, it means you can't |
| 433 | properly document functions intended for private use in Zope. |
| 434 | |
| 435 | Arbitrary attributes can now be set and retrieved on functions using the regular |
| 436 | Python syntax:: |
| 437 | |
| 438 | def f(): pass |
| 439 | |
| 440 | f.publish = 1 |
| 441 | f.secure = 1 |
| 442 | f.grammar = "A ::= B (C D)*" |
| 443 | |
| 444 | The dictionary containing attributes can be accessed as the function's |
| 445 | :attr:`__dict__`. Unlike the :attr:`__dict__` attribute of class instances, in |
| 446 | functions you can actually assign a new dictionary to :attr:`__dict__`, though |
| 447 | the new value is restricted to a regular Python dictionary; you *can't* be |
| 448 | tricky and set it to a :class:`UserDict` instance, or any other random object |
| 449 | that behaves like a mapping. |
| 450 | |
| 451 | |
| 452 | .. seealso:: |
| 453 | |
| 454 | :pep:`232` - Function Attributes |
| 455 | Written and implemented by Barry Warsaw. |
| 456 | |
Christian Heimes | 5b5e81c | 2007-12-31 16:14:33 +0000 | [diff] [blame] | 457 | .. ====================================================================== |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 458 | |
| 459 | |
| 460 | PEP 235: Importing Modules on Case-Insensitive Platforms |
| 461 | ======================================================== |
| 462 | |
| 463 | Some operating systems have filesystems that are case-insensitive, MacOS and |
| 464 | Windows being the primary examples; on these systems, it's impossible to |
| 465 | distinguish the filenames ``FILE.PY`` and ``file.py``, even though they do store |
| 466 | the file's name in its original case (they're case-preserving, too). |
| 467 | |
| 468 | In Python 2.1, the :keyword:`import` statement will work to simulate case- |
| 469 | sensitivity on case-insensitive platforms. Python will now search for the first |
| 470 | case-sensitive match by default, raising an :exc:`ImportError` if no such file |
| 471 | is found, so ``import file`` will not import a module named ``FILE.PY``. Case- |
| 472 | insensitive matching can be requested by setting the :envvar:`PYTHONCASEOK` |
| 473 | environment variable before starting the Python interpreter. |
| 474 | |
Christian Heimes | 5b5e81c | 2007-12-31 16:14:33 +0000 | [diff] [blame] | 475 | .. ====================================================================== |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 476 | |
| 477 | |
| 478 | PEP 217: Interactive Display Hook |
| 479 | ================================= |
| 480 | |
| 481 | When using the Python interpreter interactively, the output of commands is |
| 482 | displayed using the built-in :func:`repr` function. In Python 2.1, the variable |
| 483 | :func:`sys.displayhook` can be set to a callable object which will be called |
| 484 | instead of :func:`repr`. For example, you can set it to a special pretty- |
| 485 | printing function:: |
| 486 | |
| 487 | >>> # Create a recursive data structure |
| 488 | ... L = [1,2,3] |
| 489 | >>> L.append(L) |
| 490 | >>> L # Show Python's default output |
| 491 | [1, 2, 3, [...]] |
| 492 | >>> # Use pprint.pprint() as the display function |
| 493 | ... import sys, pprint |
| 494 | >>> sys.displayhook = pprint.pprint |
| 495 | >>> L |
| 496 | [1, 2, 3, <Recursion on list with id=135143996>] |
| 497 | >>> |
| 498 | |
| 499 | |
| 500 | .. seealso:: |
| 501 | |
| 502 | :pep:`217` - Display Hook for Interactive Use |
| 503 | Written and implemented by Moshe Zadka. |
| 504 | |
Christian Heimes | 5b5e81c | 2007-12-31 16:14:33 +0000 | [diff] [blame] | 505 | .. ====================================================================== |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 506 | |
| 507 | |
| 508 | PEP 208: New Coercion Model |
| 509 | =========================== |
| 510 | |
| 511 | How numeric coercion is done at the C level was significantly modified. This |
| 512 | will only affect the authors of C extensions to Python, allowing them more |
| 513 | flexibility in writing extension types that support numeric operations. |
| 514 | |
| 515 | Extension types can now set the type flag ``Py_TPFLAGS_CHECKTYPES`` in their |
| 516 | ``PyTypeObject`` structure to indicate that they support the new coercion model. |
| 517 | In such extension types, the numeric slot functions can no longer assume that |
| 518 | they'll be passed two arguments of the same type; instead they may be passed two |
| 519 | arguments of differing types, and can then perform their own internal coercion. |
| 520 | If the slot function is passed a type it can't handle, it can indicate the |
| 521 | failure by returning a reference to the ``Py_NotImplemented`` singleton value. |
| 522 | The numeric functions of the other type will then be tried, and perhaps they can |
| 523 | handle the operation; if the other type also returns ``Py_NotImplemented``, then |
| 524 | a :exc:`TypeError` will be raised. Numeric methods written in Python can also |
| 525 | return ``Py_NotImplemented``, causing the interpreter to act as if the method |
| 526 | did not exist (perhaps raising a :exc:`TypeError`, perhaps trying another |
| 527 | object's numeric methods). |
| 528 | |
| 529 | |
| 530 | .. seealso:: |
| 531 | |
| 532 | :pep:`208` - Reworking the Coercion Model |
| 533 | Written and implemented by Neil Schemenauer, heavily based upon earlier work by |
| 534 | Marc-André Lemburg. Read this to understand the fine points of how numeric |
| 535 | operations will now be processed at the C level. |
| 536 | |
Christian Heimes | 5b5e81c | 2007-12-31 16:14:33 +0000 | [diff] [blame] | 537 | .. ====================================================================== |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 538 | |
| 539 | |
| 540 | PEP 241: Metadata in Python Packages |
| 541 | ==================================== |
| 542 | |
| 543 | A common complaint from Python users is that there's no single catalog of all |
| 544 | the Python modules in existence. T. Middleton's Vaults of Parnassus at |
| 545 | http://www.vex.net/parnassus/ are the largest catalog of Python modules, but |
| 546 | registering software at the Vaults is optional, and many people don't bother. |
| 547 | |
| 548 | As a first small step toward fixing the problem, Python software packaged using |
| 549 | the Distutils :command:`sdist` command will include a file named |
| 550 | :file:`PKG-INFO` containing information about the package such as its name, |
| 551 | version, and author (metadata, in cataloguing terminology). PEP 241 contains |
| 552 | the full list of fields that can be present in the :file:`PKG-INFO` file. As |
| 553 | people began to package their software using Python 2.1, more and more packages |
| 554 | will include metadata, making it possible to build automated cataloguing systems |
| 555 | and experiment with them. With the result experience, perhaps it'll be possible |
| 556 | to design a really good catalog and then build support for it into Python 2.2. |
| 557 | For example, the Distutils :command:`sdist` and :command:`bdist_\*` commands |
| 558 | could support a :option:`upload` option that would automatically upload your |
| 559 | package to a catalog server. |
| 560 | |
| 561 | You can start creating packages containing :file:`PKG-INFO` even if you're not |
| 562 | using Python 2.1, since a new release of the Distutils will be made for users of |
| 563 | earlier Python versions. Version 1.0.2 of the Distutils includes the changes |
| 564 | described in PEP 241, as well as various bugfixes and enhancements. It will be |
| 565 | available from the Distutils SIG at http://www.python.org/sigs/distutils-sig/. |
| 566 | |
| 567 | |
| 568 | .. seealso:: |
| 569 | |
| 570 | :pep:`241` - Metadata for Python Software Packages |
| 571 | Written and implemented by A.M. Kuchling. |
| 572 | |
| 573 | :pep:`243` - Module Repository Upload Mechanism |
| 574 | Written by Sean Reifschneider, this draft PEP describes a proposed mechanism for |
| 575 | uploading Python packages to a central server. |
| 576 | |
Christian Heimes | 5b5e81c | 2007-12-31 16:14:33 +0000 | [diff] [blame] | 577 | .. ====================================================================== |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 578 | |
| 579 | |
| 580 | New and Improved Modules |
| 581 | ======================== |
| 582 | |
| 583 | * Ka-Ping Yee contributed two new modules: :mod:`inspect.py`, a module for |
| 584 | getting information about live Python code, and :mod:`pydoc.py`, a module for |
| 585 | interactively converting docstrings to HTML or text. As a bonus, |
| 586 | :file:`Tools/scripts/pydoc`, which is now automatically installed, uses |
| 587 | :mod:`pydoc.py` to display documentation given a Python module, package, or |
| 588 | class name. For example, ``pydoc xml.dom`` displays the following:: |
| 589 | |
| 590 | Python Library Documentation: package xml.dom in xml |
| 591 | |
| 592 | NAME |
| 593 | xml.dom - W3C Document Object Model implementation for Python. |
| 594 | |
| 595 | FILE |
| 596 | /usr/local/lib/python2.1/xml/dom/__init__.pyc |
| 597 | |
| 598 | DESCRIPTION |
| 599 | The Python mapping of the Document Object Model is documented in the |
| 600 | Python Library Reference in the section on the xml.dom package. |
| 601 | |
| 602 | This package contains the following modules: |
| 603 | ... |
| 604 | |
| 605 | :file:`pydoc` also includes a Tk-based interactive help browser. :file:`pydoc` |
| 606 | quickly becomes addictive; try it out! |
| 607 | |
| 608 | * Two different modules for unit testing were added to the standard library. |
| 609 | The :mod:`doctest` module, contributed by Tim Peters, provides a testing |
| 610 | framework based on running embedded examples in docstrings and comparing the |
| 611 | results against the expected output. PyUnit, contributed by Steve Purcell, is a |
| 612 | unit testing framework inspired by JUnit, which was in turn an adaptation of |
| 613 | Kent Beck's Smalltalk testing framework. See http://pyunit.sourceforge.net/ for |
| 614 | more information about PyUnit. |
| 615 | |
| 616 | * The :mod:`difflib` module contains a class, :class:`SequenceMatcher`, which |
| 617 | compares two sequences and computes the changes required to transform one |
| 618 | sequence into the other. For example, this module can be used to write a tool |
| 619 | similar to the Unix :program:`diff` program, and in fact the sample program |
| 620 | :file:`Tools/scripts/ndiff.py` demonstrates how to write such a script. |
| 621 | |
| 622 | * :mod:`curses.panel`, a wrapper for the panel library, part of ncurses and of |
| 623 | SYSV curses, was contributed by Thomas Gellekum. The panel library provides |
| 624 | windows with the additional feature of depth. Windows can be moved higher or |
| 625 | lower in the depth ordering, and the panel library figures out where panels |
| 626 | overlap and which sections are visible. |
| 627 | |
| 628 | * The PyXML package has gone through a few releases since Python 2.0, and Python |
| 629 | 2.1 includes an updated version of the :mod:`xml` package. Some of the |
| 630 | noteworthy changes include support for Expat 1.2 and later versions, the ability |
| 631 | for Expat parsers to handle files in any encoding supported by Python, and |
| 632 | various bugfixes for SAX, DOM, and the :mod:`minidom` module. |
| 633 | |
| 634 | * Ping also contributed another hook for handling uncaught exceptions. |
| 635 | :func:`sys.excepthook` can be set to a callable object. When an exception isn't |
| 636 | caught by any :keyword:`try`...\ :keyword:`except` blocks, the exception will be |
| 637 | passed to :func:`sys.excepthook`, which can then do whatever it likes. At the |
| 638 | Ninth Python Conference, Ping demonstrated an application for this hook: |
| 639 | printing an extended traceback that not only lists the stack frames, but also |
| 640 | lists the function arguments and the local variables for each frame. |
| 641 | |
| 642 | * Various functions in the :mod:`time` module, such as :func:`asctime` and |
| 643 | :func:`localtime`, require a floating point argument containing the time in |
| 644 | seconds since the epoch. The most common use of these functions is to work with |
| 645 | the current time, so the floating point argument has been made optional; when a |
| 646 | value isn't provided, the current time will be used. For example, log file |
| 647 | entries usually need a string containing the current time; in Python 2.1, |
| 648 | ``time.asctime()`` can be used, instead of the lengthier |
| 649 | ``time.asctime(time.localtime(time.time()))`` that was previously required. |
| 650 | |
| 651 | This change was proposed and implemented by Thomas Wouters. |
| 652 | |
| 653 | * The :mod:`ftplib` module now defaults to retrieving files in passive mode, |
| 654 | because passive mode is more likely to work from behind a firewall. This |
| 655 | request came from the Debian bug tracking system, since other Debian packages |
| 656 | use :mod:`ftplib` to retrieve files and then don't work from behind a firewall. |
| 657 | It's deemed unlikely that this will cause problems for anyone, because Netscape |
| 658 | defaults to passive mode and few people complain, but if passive mode is |
| 659 | unsuitable for your application or network setup, call :meth:`set_pasv(0)` on |
| 660 | FTP objects to disable passive mode. |
| 661 | |
| 662 | * Support for raw socket access has been added to the :mod:`socket` module, |
| 663 | contributed by Grant Edwards. |
| 664 | |
| 665 | * The :mod:`pstats` module now contains a simple interactive statistics browser |
| 666 | for displaying timing profiles for Python programs, invoked when the module is |
| 667 | run as a script. Contributed by Eric S. Raymond. |
| 668 | |
| 669 | * A new implementation-dependent function, :func:`sys._getframe([depth])`, has |
| 670 | been added to return a given frame object from the current call stack. |
| 671 | :func:`sys._getframe` returns the frame at the top of the call stack; if the |
| 672 | optional integer argument *depth* is supplied, the function returns the frame |
| 673 | that is *depth* calls below the top of the stack. For example, |
| 674 | ``sys._getframe(1)`` returns the caller's frame object. |
| 675 | |
| 676 | This function is only present in CPython, not in Jython or the .NET |
| 677 | implementation. Use it for debugging, and resist the temptation to put it into |
| 678 | production code. |
| 679 | |
Christian Heimes | 5b5e81c | 2007-12-31 16:14:33 +0000 | [diff] [blame] | 680 | .. ====================================================================== |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 681 | |
| 682 | |
| 683 | Other Changes and Fixes |
| 684 | ======================= |
| 685 | |
| 686 | There were relatively few smaller changes made in Python 2.1 due to the shorter |
| 687 | release cycle. A search through the CVS change logs turns up 117 patches |
| 688 | applied, and 136 bugs fixed; both figures are likely to be underestimates. Some |
| 689 | of the more notable changes are: |
| 690 | |
| 691 | * A specialized object allocator is now optionally available, that should be |
| 692 | faster than the system :func:`malloc` and have less memory overhead. The |
| 693 | allocator uses C's :func:`malloc` function to get large pools of memory, and |
| 694 | then fulfills smaller memory requests from these pools. It can be enabled by |
| 695 | providing the :option:`--with-pymalloc` option to the :program:`configure` |
| 696 | script; see :file:`Objects/obmalloc.c` for the implementation details. |
| 697 | |
| 698 | Authors of C extension modules should test their code with the object allocator |
| 699 | enabled, because some incorrect code may break, causing core dumps at runtime. |
| 700 | There are a bunch of memory allocation functions in Python's C API that have |
| 701 | previously been just aliases for the C library's :func:`malloc` and |
| 702 | :func:`free`, meaning that if you accidentally called mismatched functions, the |
| 703 | error wouldn't be noticeable. When the object allocator is enabled, these |
| 704 | functions aren't aliases of :func:`malloc` and :func:`free` any more, and |
| 705 | calling the wrong function to free memory will get you a core dump. For |
| 706 | example, if memory was allocated using :func:`PyMem_New`, it has to be freed |
| 707 | using :func:`PyMem_Del`, not :func:`free`. A few modules included with Python |
| 708 | fell afoul of this and had to be fixed; doubtless there are more third-party |
| 709 | modules that will have the same problem. |
| 710 | |
| 711 | The object allocator was contributed by Vladimir Marangozov. |
| 712 | |
| 713 | * The speed of line-oriented file I/O has been improved because people often |
| 714 | complain about its lack of speed, and because it's often been used as a naïve |
| 715 | benchmark. The :meth:`readline` method of file objects has therefore been |
| 716 | rewritten to be much faster. The exact amount of the speedup will vary from |
| 717 | platform to platform depending on how slow the C library's :func:`getc` was, but |
| 718 | is around 66%, and potentially much faster on some particular operating systems. |
| 719 | Tim Peters did much of the benchmarking and coding for this change, motivated by |
| 720 | a discussion in comp.lang.python. |
| 721 | |
| 722 | A new module and method for file objects was also added, contributed by Jeff |
| 723 | Epler. The new method, :meth:`xreadlines`, is similar to the existing |
| 724 | :func:`xrange` built-in. :func:`xreadlines` returns an opaque sequence object |
| 725 | that only supports being iterated over, reading a line on every iteration but |
| 726 | not reading the entire file into memory as the existing :meth:`readlines` method |
| 727 | does. You'd use it like this:: |
| 728 | |
| 729 | for line in sys.stdin.xreadlines(): |
| 730 | # ... do something for each line ... |
| 731 | ... |
| 732 | |
| 733 | For a fuller discussion of the line I/O changes, see the python-dev summary for |
Ezio Melotti | 9e8346c | 2010-04-05 13:57:55 +0000 | [diff] [blame] | 734 | January 1-15, 2001 at http://www.python.org/dev/summary/2001-01-1/. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 735 | |
| 736 | * A new method, :meth:`popitem`, was added to dictionaries to enable |
| 737 | destructively iterating through the contents of a dictionary; this can be faster |
| 738 | for large dictionaries because there's no need to construct a list containing |
| 739 | all the keys or values. ``D.popitem()`` removes a random ``(key, value)`` pair |
| 740 | from the dictionary ``D`` and returns it as a 2-tuple. This was implemented |
| 741 | mostly by Tim Peters and Guido van Rossum, after a suggestion and preliminary |
| 742 | patch by Moshe Zadka. |
| 743 | |
| 744 | * Modules can now control which names are imported when ``from module import *`` |
| 745 | is used, by defining an ``__all__`` attribute containing a list of names that |
| 746 | will be imported. One common complaint is that if the module imports other |
| 747 | modules such as :mod:`sys` or :mod:`string`, ``from module import *`` will add |
| 748 | them to the importing module's namespace. To fix this, simply list the public |
| 749 | names in ``__all__``:: |
| 750 | |
| 751 | # List public names |
| 752 | __all__ = ['Database', 'open'] |
| 753 | |
| 754 | A stricter version of this patch was first suggested and implemented by Ben |
| 755 | Wolfson, but after some python-dev discussion, a weaker final version was |
| 756 | checked in. |
| 757 | |
| 758 | * Applying :func:`repr` to strings previously used octal escapes for |
| 759 | non-printable characters; for example, a newline was ``'\012'``. This was a |
| 760 | vestigial trace of Python's C ancestry, but today octal is of very little |
| 761 | practical use. Ka-Ping Yee suggested using hex escapes instead of octal ones, |
| 762 | and using the ``\n``, ``\t``, ``\r`` escapes for the appropriate characters, |
| 763 | and implemented this new formatting. |
| 764 | |
| 765 | * Syntax errors detected at compile-time can now raise exceptions containing the |
| 766 | filename and line number of the error, a pleasant side effect of the compiler |
| 767 | reorganization done by Jeremy Hylton. |
| 768 | |
| 769 | * C extensions which import other modules have been changed to use |
| 770 | :func:`PyImport_ImportModule`, which means that they will use any import hooks |
| 771 | that have been installed. This is also encouraged for third-party extensions |
| 772 | that need to import some other module from C code. |
| 773 | |
| 774 | * The size of the Unicode character database was shrunk by another 340K thanks |
| 775 | to Fredrik Lundh. |
| 776 | |
| 777 | * Some new ports were contributed: MacOS X (by Steven Majewski), Cygwin (by |
| 778 | Jason Tishler); RISCOS (by Dietmar Schwertberger); Unixware 7 (by Billy G. |
| 779 | Allie). |
| 780 | |
| 781 | And there's the usual list of minor bugfixes, minor memory leaks, docstring |
| 782 | edits, and other tweaks, too lengthy to be worth itemizing; see the CVS logs for |
| 783 | the full details if you want them. |
| 784 | |
Christian Heimes | 5b5e81c | 2007-12-31 16:14:33 +0000 | [diff] [blame] | 785 | .. ====================================================================== |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 786 | |
| 787 | |
| 788 | Acknowledgements |
| 789 | ================ |
| 790 | |
| 791 | The author would like to thank the following people for offering suggestions on |
| 792 | various drafts of this article: Graeme Cross, David Goodger, Jay Graves, Michael |
| 793 | Hudson, Marc-André Lemburg, Fredrik Lundh, Neil Schemenauer, Thomas Wouters. |
| 794 | |