blob: 4a2b18fb53b3aa82cbd92eee86361640c5b02598 [file] [log] [blame]
Fred Drake03e10312002-03-26 19:17:43 +00001\documentclass{howto}
Fred Drake693aea22003-02-07 14:52:18 +00002\usepackage{distutils}
Andrew M. Kuchling03594bb2002-03-27 02:29:48 +00003% $Id$
4
5\title{What's New in Python 2.3}
Andrew M. Kuchlingd87eeb92003-02-18 00:56:56 +00006\release{0.09}
Fred Drakeaac8c582003-01-17 22:50:10 +00007\author{A.M.\ Kuchling}
Andrew M. Kuchlingbc5e3cc2002-11-05 00:26:33 +00008\authoraddress{\email{amk@amk.ca}}
Fred Drake03e10312002-03-26 19:17:43 +00009
10\begin{document}
11\maketitle
12\tableofcontents
13
Andrew M. Kuchlingc61ec522002-08-04 01:20:05 +000014% MacOS framework-related changes (section of its own, probably)
Andrew M. Kuchlingf70a0a82002-06-10 13:22:46 +000015
Andrew M. Kuchling03594bb2002-03-27 02:29:48 +000016%\section{Introduction \label{intro}}
17
Andrew M. Kuchling974ab9d2002-12-31 01:20:30 +000018{\large This article is a draft, and is currently up to date for
19Python 2.3alpha1. Please send any additions, comments or errata to
20the author.}
Andrew M. Kuchling03594bb2002-03-27 02:29:48 +000021
22This article explains the new features in Python 2.3. The tentative
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +000023release date of Python 2.3 is currently scheduled for mid-2003.
Andrew M. Kuchling03594bb2002-03-27 02:29:48 +000024
25This article doesn't attempt to provide a complete specification of
26the new features, but instead provides a convenient overview. For
27full details, you should refer to the documentation for Python 2.3,
Fred Drake693aea22003-02-07 14:52:18 +000028such as the \citetitle[../lib/lib.html]{Python Library Reference} and
29the \citetitle[../ref/ref.html]{Python Reference Manual}. If you want
30to understand the complete implementation and design rationale for a
31change, refer to the PEP for a particular new feature.
Fred Drake03e10312002-03-26 19:17:43 +000032
33
Andrew M. Kuchling03594bb2002-03-27 02:29:48 +000034%======================================================================
Andrew M. Kuchlingbc465102002-08-20 01:34:06 +000035\section{PEP 218: A Standard Set Datatype}
36
37The new \module{sets} module contains an implementation of a set
38datatype. The \class{Set} class is for mutable sets, sets that can
39have members added and removed. The \class{ImmutableSet} class is for
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +000040sets that can't be modified, and instances of \class{ImmutableSet} can
41therefore be used as dictionary keys. Sets are built on top of
42dictionaries, so the elements within a set must be hashable.
Andrew M. Kuchlingbc465102002-08-20 01:34:06 +000043
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +000044Here's a simple example:
Andrew M. Kuchlingbc465102002-08-20 01:34:06 +000045
46\begin{verbatim}
47>>> import sets
48>>> S = sets.Set([1,2,3])
49>>> S
50Set([1, 2, 3])
51>>> 1 in S
52True
53>>> 0 in S
54False
55>>> S.add(5)
56>>> S.remove(3)
57>>> S
58Set([1, 2, 5])
Fred Drake5c4cf152002-11-13 14:59:06 +000059>>>
Andrew M. Kuchlingbc465102002-08-20 01:34:06 +000060\end{verbatim}
61
62The union and intersection of sets can be computed with the
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +000063\method{union()} and \method{intersection()} methods or
64alternatively using the bitwise operators \code{\&} and \code{|}.
Andrew M. Kuchlingbc465102002-08-20 01:34:06 +000065Mutable sets also have in-place versions of these methods,
66\method{union_update()} and \method{intersection_update()}.
67
68\begin{verbatim}
69>>> S1 = sets.Set([1,2,3])
70>>> S2 = sets.Set([4,5,6])
71>>> S1.union(S2)
72Set([1, 2, 3, 4, 5, 6])
73>>> S1 | S2 # Alternative notation
74Set([1, 2, 3, 4, 5, 6])
Fred Drake5c4cf152002-11-13 14:59:06 +000075>>> S1.intersection(S2)
Andrew M. Kuchlingbc465102002-08-20 01:34:06 +000076Set([])
77>>> S1 & S2 # Alternative notation
78Set([])
79>>> S1.union_update(S2)
Andrew M. Kuchlingbc465102002-08-20 01:34:06 +000080>>> S1
81Set([1, 2, 3, 4, 5, 6])
Fred Drake5c4cf152002-11-13 14:59:06 +000082>>>
Andrew M. Kuchlingbc465102002-08-20 01:34:06 +000083\end{verbatim}
84
85It's also possible to take the symmetric difference of two sets. This
86is the set of all elements in the union that aren't in the
87intersection. An alternative way of expressing the symmetric
88difference is that it contains all elements that are in exactly one
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +000089set. Again, there's an alternative notation (\code{\^}), and an
90in-place version with the ungainly name
Andrew M. Kuchlingbc465102002-08-20 01:34:06 +000091\method{symmetric_difference_update()}.
92
93\begin{verbatim}
94>>> S1 = sets.Set([1,2,3,4])
95>>> S2 = sets.Set([3,4,5,6])
96>>> S1.symmetric_difference(S2)
97Set([1, 2, 5, 6])
98>>> S1 ^ S2
99Set([1, 2, 5, 6])
100>>>
101\end{verbatim}
102
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +0000103There are also \method{issubset()} and \method{issuperset()} methods
Michael W. Hudson065f5fa2003-02-10 19:24:50 +0000104for checking whether one set is a subset or superset of another:
Andrew M. Kuchlingbc465102002-08-20 01:34:06 +0000105
106\begin{verbatim}
107>>> S1 = sets.Set([1,2,3])
108>>> S2 = sets.Set([2,3])
109>>> S2.issubset(S1)
110True
111>>> S1.issubset(S2)
112False
113>>> S1.issuperset(S2)
114True
115>>>
116\end{verbatim}
117
118
119\begin{seealso}
120
121\seepep{218}{Adding a Built-In Set Object Type}{PEP written by Greg V. Wilson.
122Implemented by Greg V. Wilson, Alex Martelli, and GvR.}
123
124\end{seealso}
125
126
127
128%======================================================================
Andrew M. Kuchling517109b2002-05-07 21:01:16 +0000129\section{PEP 255: Simple Generators\label{section-generators}}
Andrew M. Kuchlingf4dd65d2002-04-01 19:28:09 +0000130
131In Python 2.2, generators were added as an optional feature, to be
132enabled by a \code{from __future__ import generators} directive. In
1332.3 generators no longer need to be specially enabled, and are now
134always present; this means that \keyword{yield} is now always a
135keyword. The rest of this section is a copy of the description of
136generators from the ``What's New in Python 2.2'' document; if you read
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +0000137it back when Python 2.2 came out, you can skip the rest of this section.
Andrew M. Kuchlingf4dd65d2002-04-01 19:28:09 +0000138
Andrew M. Kuchling517109b2002-05-07 21:01:16 +0000139You're doubtless familiar with how function calls work in Python or C.
140When you call a function, it gets a private namespace where its local
Andrew M. Kuchlingf4dd65d2002-04-01 19:28:09 +0000141variables are created. When the function reaches a \keyword{return}
142statement, the local variables are destroyed and the resulting value
143is returned to the caller. A later call to the same function will get
Andrew M. Kuchling517109b2002-05-07 21:01:16 +0000144a fresh new set of local variables. But, what if the local variables
Andrew M. Kuchlingf4dd65d2002-04-01 19:28:09 +0000145weren't thrown away on exiting a function? What if you could later
146resume the function where it left off? This is what generators
147provide; they can be thought of as resumable functions.
148
149Here's the simplest example of a generator function:
150
151\begin{verbatim}
152def generate_ints(N):
153 for i in range(N):
154 yield i
155\end{verbatim}
156
157A new keyword, \keyword{yield}, was introduced for generators. Any
158function containing a \keyword{yield} statement is a generator
159function; this is detected by Python's bytecode compiler which
Fred Drake5c4cf152002-11-13 14:59:06 +0000160compiles the function specially as a result.
Andrew M. Kuchlingf4dd65d2002-04-01 19:28:09 +0000161
162When you call a generator function, it doesn't return a single value;
163instead it returns a generator object that supports the iterator
164protocol. On executing the \keyword{yield} statement, the generator
165outputs the value of \code{i}, similar to a \keyword{return}
166statement. The big difference between \keyword{yield} and a
167\keyword{return} statement is that on reaching a \keyword{yield} the
168generator's state of execution is suspended and local variables are
169preserved. On the next call to the generator's \code{.next()} method,
170the function will resume executing immediately after the
171\keyword{yield} statement. (For complicated reasons, the
172\keyword{yield} statement isn't allowed inside the \keyword{try} block
Fred Drakeaac8c582003-01-17 22:50:10 +0000173of a \keyword{try}...\keyword{finally} statement; read \pep{255} for a full
Andrew M. Kuchlingf4dd65d2002-04-01 19:28:09 +0000174explanation of the interaction between \keyword{yield} and
175exceptions.)
176
Fred Drakeaac8c582003-01-17 22:50:10 +0000177Here's a sample usage of the \function{generate_ints()} generator:
Andrew M. Kuchlingf4dd65d2002-04-01 19:28:09 +0000178
179\begin{verbatim}
180>>> gen = generate_ints(3)
181>>> gen
182<generator object at 0x8117f90>
183>>> gen.next()
1840
185>>> gen.next()
1861
187>>> gen.next()
1882
189>>> gen.next()
190Traceback (most recent call last):
Andrew M. Kuchling9f6e1042002-06-17 13:40:04 +0000191 File "stdin", line 1, in ?
192 File "stdin", line 2, in generate_ints
Andrew M. Kuchlingf4dd65d2002-04-01 19:28:09 +0000193StopIteration
194\end{verbatim}
195
196You could equally write \code{for i in generate_ints(5)}, or
197\code{a,b,c = generate_ints(3)}.
198
199Inside a generator function, the \keyword{return} statement can only
200be used without a value, and signals the end of the procession of
201values; afterwards the generator cannot return any further values.
202\keyword{return} with a value, such as \code{return 5}, is a syntax
203error inside a generator function. The end of the generator's results
204can also be indicated by raising \exception{StopIteration} manually,
205or by just letting the flow of execution fall off the bottom of the
206function.
207
208You could achieve the effect of generators manually by writing your
209own class and storing all the local variables of the generator as
210instance variables. For example, returning a list of integers could
211be done by setting \code{self.count} to 0, and having the
212\method{next()} method increment \code{self.count} and return it.
213However, for a moderately complicated generator, writing a
214corresponding class would be much messier.
215\file{Lib/test/test_generators.py} contains a number of more
216interesting examples. The simplest one implements an in-order
217traversal of a tree using generators recursively.
218
219\begin{verbatim}
220# A recursive generator that generates Tree leaves in in-order.
221def inorder(t):
222 if t:
223 for x in inorder(t.left):
224 yield x
225 yield t.label
226 for x in inorder(t.right):
227 yield x
228\end{verbatim}
229
230Two other examples in \file{Lib/test/test_generators.py} produce
231solutions for the N-Queens problem (placing $N$ queens on an $NxN$
232chess board so that no queen threatens another) and the Knight's Tour
233(a route that takes a knight to every square of an $NxN$ chessboard
Fred Drake5c4cf152002-11-13 14:59:06 +0000234without visiting any square twice).
Andrew M. Kuchlingf4dd65d2002-04-01 19:28:09 +0000235
236The idea of generators comes from other programming languages,
237especially Icon (\url{http://www.cs.arizona.edu/icon/}), where the
238idea of generators is central. In Icon, every
239expression and function call behaves like a generator. One example
240from ``An Overview of the Icon Programming Language'' at
241\url{http://www.cs.arizona.edu/icon/docs/ipd266.htm} gives an idea of
242what this looks like:
243
244\begin{verbatim}
245sentence := "Store it in the neighboring harbor"
246if (i := find("or", sentence)) > 5 then write(i)
247\end{verbatim}
248
249In Icon the \function{find()} function returns the indexes at which the
250substring ``or'' is found: 3, 23, 33. In the \keyword{if} statement,
251\code{i} is first assigned a value of 3, but 3 is less than 5, so the
252comparison fails, and Icon retries it with the second value of 23. 23
253is greater than 5, so the comparison now succeeds, and the code prints
254the value 23 to the screen.
255
256Python doesn't go nearly as far as Icon in adopting generators as a
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +0000257central concept. Generators are considered part of the core
Andrew M. Kuchlingf4dd65d2002-04-01 19:28:09 +0000258Python language, but learning or using them isn't compulsory; if they
259don't solve any problems that you have, feel free to ignore them.
260One novel feature of Python's interface as compared to
261Icon's is that a generator's state is represented as a concrete object
262(the iterator) that can be passed around to other functions or stored
263in a data structure.
264
265\begin{seealso}
266
267\seepep{255}{Simple Generators}{Written by Neil Schemenauer, Tim
268Peters, Magnus Lie Hetland. Implemented mostly by Neil Schemenauer
269and Tim Peters, with other fixes from the Python Labs crew.}
270
271\end{seealso}
272
273
274%======================================================================
Fred Drake13090e12002-08-22 16:51:08 +0000275\section{PEP 263: Source Code Encodings \label{section-encodings}}
Andrew M. Kuchling950725f2002-08-06 01:40:48 +0000276
277Python source files can now be declared as being in different
278character set encodings. Encodings are declared by including a
279specially formatted comment in the first or second line of the source
280file. For example, a UTF-8 file can be declared with:
281
282\begin{verbatim}
283#!/usr/bin/env python
284# -*- coding: UTF-8 -*-
285\end{verbatim}
286
287Without such an encoding declaration, the default encoding used is
Andrew M. Kuchlingacddabc2003-02-18 00:43:24 +00002887-bit ASCII. Executing or importing modules containing string
289literals with 8-bit characters and no encoding declaration will result
290in a \exception{DeprecationWarning} being signalled by Python 2.3; in
2912.4 this will be a syntax error.
Andrew M. Kuchling950725f2002-08-06 01:40:48 +0000292
Andrew M. Kuchlingacddabc2003-02-18 00:43:24 +0000293The encoding declaration only affects Unicode string literals, which
294will be converted to Unicode using the specified encoding. Note that
295Python identifiers are still restricted to ASCII characters, so you
296can't have variable names that use characters outside of the usual
297alphanumerics.
Andrew M. Kuchling950725f2002-08-06 01:40:48 +0000298
299\begin{seealso}
300
301\seepep{263}{Defining Python Source Code Encodings}{Written by
Martin v. Löwisbd5e38d2002-10-07 18:52:29 +0000302Marc-Andr\'e Lemburg and Martin von L\"owis; implemented by SUZUKI
303Hisao and Martin von L\"owis.}
Andrew M. Kuchling950725f2002-08-06 01:40:48 +0000304
305\end{seealso}
306
307
308%======================================================================
Martin v. Löwisbd5e38d2002-10-07 18:52:29 +0000309\section{PEP 277: Unicode file name support for Windows NT}
Andrew M. Kuchling0f345562002-10-04 22:34:11 +0000310
Martin v. Löwisbd5e38d2002-10-07 18:52:29 +0000311On Windows NT, 2000, and XP, the system stores file names as Unicode
Andrew M. Kuchling0a6fa962002-10-09 12:11:10 +0000312strings. Traditionally, Python has represented file names as byte
313strings, which is inadequate because it renders some file names
Martin v. Löwisbd5e38d2002-10-07 18:52:29 +0000314inaccessible.
315
Andrew M. Kuchling0a6fa962002-10-09 12:11:10 +0000316Python now allows using arbitrary Unicode strings (within the
317limitations of the file system) for all functions that expect file
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +0000318names, most notably the \function{open()} built-in function. If a Unicode
319string is passed to \function{os.listdir()}, Python now returns a list
Andrew M. Kuchling0a6fa962002-10-09 12:11:10 +0000320of Unicode strings. A new function, \function{os.getcwdu()}, returns
321the current directory as a Unicode string.
Martin v. Löwisbd5e38d2002-10-07 18:52:29 +0000322
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +0000323Byte strings still work as file names, and on Windows Python will
324transparently convert them to Unicode using the \code{mbcs} encoding.
Martin v. Löwisbd5e38d2002-10-07 18:52:29 +0000325
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +0000326Other systems also allow Unicode strings as file names but convert
327them to byte strings before passing them to the system, which can
328cause a \exception{UnicodeError} to be raised. Applications can test
329whether arbitrary Unicode strings are supported as file names by
Andrew M. Kuchlingb9ba4e62003-02-03 15:16:15 +0000330checking \member{os.path.supports_unicode_filenames}, a Boolean value.
Martin v. Löwisbd5e38d2002-10-07 18:52:29 +0000331
Andrew M. Kuchling563389f2003-03-02 02:31:58 +0000332Under MacOS, \function{os.listdir()} may now return Unicode filenames.
333
Martin v. Löwisbd5e38d2002-10-07 18:52:29 +0000334\begin{seealso}
335
336\seepep{277}{Unicode file name support for Windows NT}{Written by Neil
337Hodgson; implemented by Neil Hodgson, Martin von L\"owis, and Mark
338Hammond.}
339
340\end{seealso}
Andrew M. Kuchling0f345562002-10-04 22:34:11 +0000341
342
343%======================================================================
Andrew M. Kuchlingf3676512002-04-15 02:27:55 +0000344\section{PEP 278: Universal Newline Support}
345
Andrew M. Kuchling821013e2002-05-06 17:46:39 +0000346The three major operating systems used today are Microsoft Windows,
Andrew M. Kuchling517109b2002-05-07 21:01:16 +0000347Apple's Macintosh OS, and the various \UNIX\ derivatives. A minor
Andrew M. Kuchling821013e2002-05-06 17:46:39 +0000348irritation is that these three platforms all use different characters
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +0000349to mark the ends of lines in text files. \UNIX\ uses the linefeed
350(ASCII character 10), while MacOS uses the carriage return (ASCII
351character 13), and Windows uses a two-character sequence containing a
352carriage return plus a newline.
Andrew M. Kuchlingf3676512002-04-15 02:27:55 +0000353
Andrew M. Kuchling821013e2002-05-06 17:46:39 +0000354Python's file objects can now support end of line conventions other
355than the one followed by the platform on which Python is running.
Fred Drake5c4cf152002-11-13 14:59:06 +0000356Opening a file with the mode \code{'U'} or \code{'rU'} will open a file
Andrew M. Kuchling821013e2002-05-06 17:46:39 +0000357for reading in universal newline mode. All three line ending
Fred Drake5c4cf152002-11-13 14:59:06 +0000358conventions will be translated to a \character{\e n} in the strings
Andrew M. Kuchling821013e2002-05-06 17:46:39 +0000359returned by the various file methods such as \method{read()} and
Fred Drake5c4cf152002-11-13 14:59:06 +0000360\method{readline()}.
Andrew M. Kuchlingf3676512002-04-15 02:27:55 +0000361
Andrew M. Kuchling821013e2002-05-06 17:46:39 +0000362Universal newline support is also used when importing modules and when
363executing a file with the \function{execfile()} function. This means
364that Python modules can be shared between all three operating systems
365without needing to convert the line-endings.
366
Fred Drake5c4cf152002-11-13 14:59:06 +0000367This feature can be disabled at compile-time by specifying
Andrew M. Kuchling821013e2002-05-06 17:46:39 +0000368\longprogramopt{without-universal-newlines} when running Python's
Fred Drake5c4cf152002-11-13 14:59:06 +0000369\program{configure} script.
Andrew M. Kuchlingf3676512002-04-15 02:27:55 +0000370
371\begin{seealso}
372
Fred Drake5c4cf152002-11-13 14:59:06 +0000373\seepep{278}{Universal Newline Support}{Written
Andrew M. Kuchlingf3676512002-04-15 02:27:55 +0000374and implemented by Jack Jansen.}
375
376\end{seealso}
377
Andrew M. Kuchlingfad2f592002-05-10 21:00:05 +0000378
379%======================================================================
Andrew M. Kuchlinge995d162002-07-11 20:09:50 +0000380\section{PEP 279: The \function{enumerate()} Built-in Function\label{section-enumerate}}
Andrew M. Kuchlingfad2f592002-05-10 21:00:05 +0000381
382A new built-in function, \function{enumerate()}, will make
383certain loops a bit clearer. \code{enumerate(thing)}, where
384\var{thing} is either an iterator or a sequence, returns a iterator
385that will return \code{(0, \var{thing[0]})}, \code{(1,
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +0000386\var{thing[1]})}, \code{(2, \var{thing[2]})}, and so forth.
387
388Fairly often you'll see code to change every element of a list that
389looks like this:
Andrew M. Kuchlingfad2f592002-05-10 21:00:05 +0000390
391\begin{verbatim}
392for i in range(len(L)):
393 item = L[i]
394 # ... compute some result based on item ...
395 L[i] = result
396\end{verbatim}
397
398This can be rewritten using \function{enumerate()} as:
399
400\begin{verbatim}
401for i, item in enumerate(L):
402 # ... compute some result based on item ...
403 L[i] = result
404\end{verbatim}
405
406
407\begin{seealso}
408
Fred Drake5c4cf152002-11-13 14:59:06 +0000409\seepep{279}{The enumerate() built-in function}{Written
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +0000410and implemented by Raymond D. Hettinger.}
Andrew M. Kuchlingfad2f592002-05-10 21:00:05 +0000411
412\end{seealso}
413
414
Andrew M. Kuchlingf3676512002-04-15 02:27:55 +0000415%======================================================================
Andrew M. Kuchling28f2f882002-11-14 14:14:16 +0000416\section{PEP 282: The \module{logging} Package}
417
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +0000418A standard package for writing logs, \module{logging}, has been added
419to Python 2.3. It provides a powerful and flexible mechanism for
Andrew M. Kuchling366c10c2002-11-14 23:07:57 +0000420components to generate logging output which can then be filtered and
421processed in various ways. A standard configuration file format can
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +0000422be used to control the logging behavior of a program. Python's
423standard library includes handlers that will write log records to
424standard error or to a file or socket, send them to the system log, or
425even e-mail them to a particular address, and of course it's also
426possible to write your own handler classes.
Andrew M. Kuchling28f2f882002-11-14 14:14:16 +0000427
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +0000428The \class{Logger} class is the primary class.
Andrew M. Kuchling366c10c2002-11-14 23:07:57 +0000429Most application code will deal with one or more \class{Logger}
430objects, each one used by a particular subsystem of the application.
431Each \class{Logger} is identified by a name, and names are organized
432into a hierarchy using \samp{.} as the component separator. For
433example, you might have \class{Logger} instances named \samp{server},
434\samp{server.auth} and \samp{server.network}. The latter two
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +0000435instances are below \samp{server} in the hierarchy. This means that
436if you turn up the verbosity for \samp{server} or direct \samp{server}
437messages to a different handler, the changes will also apply to
438records logged to \samp{server.auth} and \samp{server.network}.
439There's also a root \class{Logger} that's the parent of all other
440loggers.
Andrew M. Kuchling28f2f882002-11-14 14:14:16 +0000441
Andrew M. Kuchling366c10c2002-11-14 23:07:57 +0000442For simple uses, the \module{logging} package contains some
443convenience functions that always use the root log:
Andrew M. Kuchling28f2f882002-11-14 14:14:16 +0000444
445\begin{verbatim}
446import logging
447
448logging.debug('Debugging information')
449logging.info('Informational message')
Andrew M. Kuchling37495072003-02-19 13:46:18 +0000450logging.warning('Warning:config file %s not found', 'server.conf')
Andrew M. Kuchling28f2f882002-11-14 14:14:16 +0000451logging.error('Error occurred')
452logging.critical('Critical error -- shutting down')
453\end{verbatim}
454
455This produces the following output:
456
457\begin{verbatim}
Andrew M. Kuchling37495072003-02-19 13:46:18 +0000458WARNING:root:Warning:config file server.conf not found
Andrew M. Kuchling28f2f882002-11-14 14:14:16 +0000459ERROR:root:Error occurred
460CRITICAL:root:Critical error -- shutting down
461\end{verbatim}
462
463In the default configuration, informational and debugging messages are
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +0000464suppressed and the output is sent to standard error. You can enable
465the display of information and debugging messages by calling the
466\method{setLevel()} method on the root logger.
Andrew M. Kuchling366c10c2002-11-14 23:07:57 +0000467
Andrew M. Kuchling37495072003-02-19 13:46:18 +0000468Notice the \function{warning()} call's use of string formatting
Andrew M. Kuchling366c10c2002-11-14 23:07:57 +0000469operators; all of the functions for logging messages take the
470arguments \code{(\var{msg}, \var{arg1}, \var{arg2}, ...)} and log the
471string resulting from \code{\var{msg} \% (\var{arg1}, \var{arg2},
472...)}.
Andrew M. Kuchling28f2f882002-11-14 14:14:16 +0000473
474There's also an \function{exception()} function that records the most
475recent traceback. Any of the other functions will also record the
Andrew M. Kuchling366c10c2002-11-14 23:07:57 +0000476traceback if you specify a true value for the keyword argument
Fred Drakeaac8c582003-01-17 22:50:10 +0000477\var{exc_info}.
Andrew M. Kuchling28f2f882002-11-14 14:14:16 +0000478
479\begin{verbatim}
480def f():
481 try: 1/0
482 except: logging.exception('Problem recorded')
483
484f()
485\end{verbatim}
486
487This produces the following output:
488
489\begin{verbatim}
490ERROR:root:Problem recorded
491Traceback (most recent call last):
492 File "t.py", line 6, in f
493 1/0
494ZeroDivisionError: integer division or modulo by zero
495\end{verbatim}
496
Andrew M. Kuchling366c10c2002-11-14 23:07:57 +0000497Slightly more advanced programs will use a logger other than the root
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +0000498logger. The \function{getLogger(\var{name})} function is used to get
499a particular log, creating it if it doesn't exist yet.
Andrew M. Kuchlingb1e4bf92002-12-03 13:35:17 +0000500\function{getLogger(None)} returns the root logger.
501
Andrew M. Kuchling28f2f882002-11-14 14:14:16 +0000502
503\begin{verbatim}
504log = logging.getLogger('server')
505 ...
506log.info('Listening on port %i', port)
507 ...
508log.critical('Disk full')
509 ...
510\end{verbatim}
511
Andrew M. Kuchling366c10c2002-11-14 23:07:57 +0000512Log records are usually propagated up the hierarchy, so a message
513logged to \samp{server.auth} is also seen by \samp{server} and
514\samp{root}, but a handler can prevent this by setting its
Fred Drakeaac8c582003-01-17 22:50:10 +0000515\member{propagate} attribute to \constant{False}.
Andrew M. Kuchling366c10c2002-11-14 23:07:57 +0000516
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +0000517There are more classes provided by the \module{logging} package that
518can be customized. When a \class{Logger} instance is told to log a
519message, it creates a \class{LogRecord} instance that is sent to any
520number of different \class{Handler} instances. Loggers and handlers
521can also have an attached list of filters, and each filter can cause
522the \class{LogRecord} to be ignored or can modify the record before
523passing it along. \class{LogRecord} instances are converted to text
524for output by a \class{Formatter} class. All of these classes can be
525replaced by your own specially-written classes.
526
Andrew M. Kuchling366c10c2002-11-14 23:07:57 +0000527With all of these features the \module{logging} package should provide
528enough flexibility for even the most complicated applications. This
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +0000529is only a partial overview of the \module{logging} package, so please
530see the \ulink{package's reference
Fred Drake693aea22003-02-07 14:52:18 +0000531documentation}{../lib/module-logging.html} for all of the details.
532Reading \pep{282} will also be helpful.
Andrew M. Kuchling28f2f882002-11-14 14:14:16 +0000533
534
535\begin{seealso}
536
537\seepep{282}{A Logging System}{Written by Vinay Sajip and Trent Mick;
538implemented by Vinay Sajip.}
539
540\end{seealso}
541
542
543%======================================================================
Andrew M. Kuchling517109b2002-05-07 21:01:16 +0000544\section{PEP 285: The \class{bool} Type\label{section-bool}}
545
Andrew M. Kuchling821013e2002-05-06 17:46:39 +0000546A Boolean type was added to Python 2.3. Two new constants were added
547to the \module{__builtin__} module, \constant{True} and
Andrew M. Kuchling5a224532003-01-03 16:52:27 +0000548\constant{False}. (\constant{True} and
549\constant{False} constants were added to the built-ins
Michael W. Hudson065f5fa2003-02-10 19:24:50 +0000550in Python 2.2.1, but the 2.2.1 versions simply have integer values of
Andrew M. Kuchling5a224532003-01-03 16:52:27 +00005511 and 0 and aren't a different type.)
552
553The type object for this new type is named
Andrew M. Kuchling821013e2002-05-06 17:46:39 +0000554\class{bool}; the constructor for it takes any Python value and
555converts it to \constant{True} or \constant{False}.
556
557\begin{verbatim}
558>>> bool(1)
559True
560>>> bool(0)
561False
562>>> bool([])
563False
564>>> bool( (1,) )
565True
566\end{verbatim}
567
568Most of the standard library modules and built-in functions have been
569changed to return Booleans.
570
571\begin{verbatim}
Andrew M. Kuchling517109b2002-05-07 21:01:16 +0000572>>> obj = []
573>>> hasattr(obj, 'append')
Andrew M. Kuchling821013e2002-05-06 17:46:39 +0000574True
Andrew M. Kuchling517109b2002-05-07 21:01:16 +0000575>>> isinstance(obj, list)
Andrew M. Kuchling821013e2002-05-06 17:46:39 +0000576True
Andrew M. Kuchling517109b2002-05-07 21:01:16 +0000577>>> isinstance(obj, tuple)
Andrew M. Kuchling821013e2002-05-06 17:46:39 +0000578False
579\end{verbatim}
580
581Python's Booleans were added with the primary goal of making code
582clearer. For example, if you're reading a function and encounter the
Fred Drake5c4cf152002-11-13 14:59:06 +0000583statement \code{return 1}, you might wonder whether the \code{1}
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +0000584represents a Boolean truth value, an index, or a
Andrew M. Kuchling821013e2002-05-06 17:46:39 +0000585coefficient that multiplies some other quantity. If the statement is
586\code{return True}, however, the meaning of the return value is quite
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +0000587clear.
Andrew M. Kuchling821013e2002-05-06 17:46:39 +0000588
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +0000589Python's Booleans were \emph{not} added for the sake of strict
590type-checking. A very strict language such as Pascal would also
591prevent you performing arithmetic with Booleans, and would require
592that the expression in an \keyword{if} statement always evaluate to a
593Boolean. Python is not this strict, and it never will be, as
594\pep{285} explicitly says. This means you can still use any
595expression in an \keyword{if} statement, even ones that evaluate to a
596list or tuple or some random object, and the Boolean type is a
597subclass of the \class{int} class so that arithmetic using a Boolean
598still works.
Andrew M. Kuchling821013e2002-05-06 17:46:39 +0000599
600\begin{verbatim}
601>>> True + 1
6022
603>>> False + 1
6041
605>>> False * 75
6060
607>>> True * 75
60875
609\end{verbatim}
610
611To sum up \constant{True} and \constant{False} in a sentence: they're
612alternative ways to spell the integer values 1 and 0, with the single
613difference that \function{str()} and \function{repr()} return the
Fred Drake5c4cf152002-11-13 14:59:06 +0000614strings \code{'True'} and \code{'False'} instead of \code{'1'} and
615\code{'0'}.
Andrew M. Kuchling3a52ff62002-04-03 22:44:47 +0000616
617\begin{seealso}
618
619\seepep{285}{Adding a bool type}{Written and implemented by GvR.}
620
621\end{seealso}
622
Michael W. Hudson5efaf7e2002-06-11 10:55:12 +0000623
Andrew M. Kuchling65b72822002-09-03 00:53:21 +0000624%======================================================================
625\section{PEP 293: Codec Error Handling Callbacks}
626
Martin v. Löwis20eae692002-10-07 19:01:07 +0000627When encoding a Unicode string into a byte string, unencodable
Andrew M. Kuchling0a6fa962002-10-09 12:11:10 +0000628characters may be encountered. So far, Python has allowed specifying
629the error processing as either ``strict'' (raising
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +0000630\exception{UnicodeError}), ``ignore'' (skipping the character), or
631``replace'' (using a question mark in the output string), with
632``strict'' being the default behavior. It may be desirable to specify
633alternative processing of such errors, such as inserting an XML
634character reference or HTML entity reference into the converted
635string.
Martin v. Löwis20eae692002-10-07 19:01:07 +0000636
Andrew M. Kuchlingb492fa92002-11-27 19:11:10 +0000637Python now has a flexible framework to add different processing
Andrew M. Kuchling0a6fa962002-10-09 12:11:10 +0000638strategies. New error handlers can be added with
Martin v. Löwis20eae692002-10-07 19:01:07 +0000639\function{codecs.register_error}. Codecs then can access the error
Andrew M. Kuchling0a6fa962002-10-09 12:11:10 +0000640handler with \function{codecs.lookup_error}. An equivalent C API has
641been added for codecs written in C. The error handler gets the
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +0000642necessary state information such as the string being converted, the
Andrew M. Kuchling0a6fa962002-10-09 12:11:10 +0000643position in the string where the error was detected, and the target
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +0000644encoding. The handler can then either raise an exception or return a
Andrew M. Kuchling0a6fa962002-10-09 12:11:10 +0000645replacement string.
Martin v. Löwis20eae692002-10-07 19:01:07 +0000646
647Two additional error handlers have been implemented using this
Andrew M. Kuchling0a6fa962002-10-09 12:11:10 +0000648framework: ``backslashreplace'' uses Python backslash quoting to
Andrew M. Kuchlingb492fa92002-11-27 19:11:10 +0000649represent unencodable characters and ``xmlcharrefreplace'' emits
Martin v. Löwis20eae692002-10-07 19:01:07 +0000650XML character references.
Andrew M. Kuchling65b72822002-09-03 00:53:21 +0000651
652\begin{seealso}
653
Fred Drake5c4cf152002-11-13 14:59:06 +0000654\seepep{293}{Codec Error Handling Callbacks}{Written and implemented by
Andrew M. Kuchling0a6fa962002-10-09 12:11:10 +0000655Walter D\"orwald.}
Andrew M. Kuchling65b72822002-09-03 00:53:21 +0000656
657\end{seealso}
658
659
660%======================================================================
Andrew M. Kuchling974ab9d2002-12-31 01:20:30 +0000661\section{PEP 273: Importing Modules from Zip Archives}
662
663The new \module{zipimport} module adds support for importing
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +0000664modules from a ZIP-format archive. You don't need to import the
Andrew M. Kuchling974ab9d2002-12-31 01:20:30 +0000665module explicitly; it will be automatically imported if a ZIP
666archive's filename is added to \code{sys.path}. For example:
667
668\begin{verbatim}
669amk@nyman:~/src/python$ unzip -l /tmp/example.zip
670Archive: /tmp/example.zip
671 Length Date Time Name
672 -------- ---- ---- ----
673 8467 11-26-02 22:30 jwzthreading.py
674 -------- -------
675 8467 1 file
676amk@nyman:~/src/python$ ./python
677Python 2.3a0 (#1, Dec 30 2002, 19:54:32)
Andrew M. Kuchling974ab9d2002-12-31 01:20:30 +0000678>>> import sys
679>>> sys.path.insert(0, '/tmp/example.zip') # Add .zip file to front of path
680>>> import jwzthreading
681>>> jwzthreading.__file__
682'/tmp/example.zip/jwzthreading.py'
683>>>
684\end{verbatim}
685
686An entry in \code{sys.path} can now be the filename of a ZIP archive.
687The ZIP archive can contain any kind of files, but only files named
Fred Drakeaac8c582003-01-17 22:50:10 +0000688\file{*.py}, \file{*.pyc}, or \file{*.pyo} can be imported. If an
689archive only contains \file{*.py} files, Python will not attempt to
690modify the archive by adding the corresponding \file{*.pyc} file, meaning
691that if a ZIP archive doesn't contain \file{*.pyc} files, importing may be
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +0000692rather slow.
Andrew M. Kuchling974ab9d2002-12-31 01:20:30 +0000693
694A path within the archive can also be specified to only import from a
695subdirectory; for example, the path \file{/tmp/example.zip/lib/}
696would only import from the \file{lib/} subdirectory within the
697archive.
698
Andrew M. Kuchling974ab9d2002-12-31 01:20:30 +0000699\begin{seealso}
700
701\seepep{273}{Import Modules from Zip Archives}{Written by James C. Ahlstrom,
702who also provided an implementation.
703Python 2.3 follows the specification in \pep{273},
Andrew M. Kuchlingae3bbf52002-12-31 14:03:45 +0000704but uses an implementation written by Just van~Rossum
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +0000705that uses the import hooks described in \pep{302}.
706See section~\ref{section-pep302} for a description of the new import hooks.
707}
Andrew M. Kuchling974ab9d2002-12-31 01:20:30 +0000708
709\end{seealso}
710
711%======================================================================
Fred Drake693aea22003-02-07 14:52:18 +0000712\section{PEP 301: Package Index and Metadata for
713Distutils\label{section-pep301}}
Andrew M. Kuchling87cebbf2003-01-03 16:24:28 +0000714
Andrew M. Kuchling5a224532003-01-03 16:52:27 +0000715Support for the long-requested Python catalog makes its first
716appearance in 2.3.
717
Fred Drake693aea22003-02-07 14:52:18 +0000718The core component is the new Distutils \command{register} command.
719Running \code{python setup.py register} will collect the metadata
Andrew M. Kuchling5a224532003-01-03 16:52:27 +0000720describing a package, such as its name, version, maintainer,
Andrew M. Kuchlingc61402b2003-02-26 19:00:52 +0000721description, \&c., and send it to a central catalog server. The
722catalog is available from \url{http://www.python.org/pypi}.
Andrew M. Kuchling5a224532003-01-03 16:52:27 +0000723
724To make the catalog a bit more useful, a new optional
Fred Drake693aea22003-02-07 14:52:18 +0000725\var{classifiers} keyword argument has been added to the Distutils
Andrew M. Kuchling5a224532003-01-03 16:52:27 +0000726\function{setup()} function. A list of
Fred Drake693aea22003-02-07 14:52:18 +0000727\ulink{Trove}{http://catb.org/\textasciitilde esr/trove/}-style
728strings can be supplied to help classify the software.
Andrew M. Kuchling5a224532003-01-03 16:52:27 +0000729
Andrew M. Kuchlinga31bb372003-01-27 16:36:34 +0000730Here's an example \file{setup.py} with classifiers, written to be compatible
731with older versions of the Distutils:
Andrew M. Kuchling5a224532003-01-03 16:52:27 +0000732
733\begin{verbatim}
Andrew M. Kuchlinga31bb372003-01-27 16:36:34 +0000734from distutils import core
Fred Drake693aea22003-02-07 14:52:18 +0000735kw = {'name': "Quixote",
Andrew M. Kuchlinga31bb372003-01-27 16:36:34 +0000736 'version': "0.5.1",
737 'description': "A highly Pythonic Web application framework",
Fred Drake693aea22003-02-07 14:52:18 +0000738 # ...
739 }
Andrew M. Kuchlinga31bb372003-01-27 16:36:34 +0000740
Fred Drake693aea22003-02-07 14:52:18 +0000741if ( hasattr(core, 'setup_keywords') and
742 'classifiers' in core.setup_keywords):
743 kw['classifiers'] = \
744 ['Topic :: Internet :: WWW/HTTP :: Dynamic Content',
745 'Environment :: No Input/Output (Daemon)',
746 'Intended Audience :: Developers'],
Andrew M. Kuchlinga31bb372003-01-27 16:36:34 +0000747
Fred Drake693aea22003-02-07 14:52:18 +0000748core.setup(**kw)
Andrew M. Kuchling5a224532003-01-03 16:52:27 +0000749\end{verbatim}
750
751The full list of classifiers can be obtained by running
752\code{python setup.py register --list-classifiers}.
Andrew M. Kuchling87cebbf2003-01-03 16:24:28 +0000753
754\begin{seealso}
755
Fred Drake693aea22003-02-07 14:52:18 +0000756\seepep{301}{Package Index and Metadata for Distutils}{Written and
757implemented by Richard Jones.}
Andrew M. Kuchling87cebbf2003-01-03 16:24:28 +0000758
759\end{seealso}
760
761
762%======================================================================
Andrew M. Kuchling974ab9d2002-12-31 01:20:30 +0000763\section{PEP 302: New Import Hooks \label{section-pep302}}
764
765While it's been possible to write custom import hooks ever since the
766\module{ihooks} module was introduced in Python 1.3, no one has ever
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +0000767been really happy with it because writing new import hooks is
768difficult and messy. There have been various proposed alternatives
769such as the \module{imputil} and \module{iu} modules, but none of them
770has ever gained much acceptance, and none of them were easily usable
771from \C{} code.
Andrew M. Kuchling974ab9d2002-12-31 01:20:30 +0000772
773\pep{302} borrows ideas from its predecessors, especially from
774Gordon McMillan's \module{iu} module. Three new items
775are added to the \module{sys} module:
776
777\begin{itemize}
Andrew M. Kuchlingd5ac8d02003-01-02 21:33:15 +0000778 \item \code{sys.path_hooks} is a list of callable objects; most
Fred Drakeaac8c582003-01-17 22:50:10 +0000779 often they'll be classes. Each callable takes a string containing a
780 path and either returns an importer object that will handle imports
781 from this path or raises an \exception{ImportError} exception if it
782 can't handle this path.
Andrew M. Kuchling974ab9d2002-12-31 01:20:30 +0000783
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +0000784 \item \code{sys.path_importer_cache} caches importer objects for
Fred Drakeaac8c582003-01-17 22:50:10 +0000785 each path, so \code{sys.path_hooks} will only need to be traversed
786 once for each path.
Andrew M. Kuchling974ab9d2002-12-31 01:20:30 +0000787
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +0000788 \item \code{sys.meta_path} is a list of importer objects that will
789 be traversed before \code{sys.path} is checked. This list is
790 initially empty, but user code can add objects to it. Additional
791 built-in and frozen modules can be imported by an object added to
792 this list.
Andrew M. Kuchling974ab9d2002-12-31 01:20:30 +0000793
794\end{itemize}
795
796Importer objects must have a single method,
797\method{find_module(\var{fullname}, \var{path}=None)}. \var{fullname}
798will be a module or package name, e.g. \samp{string} or
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +0000799\samp{distutils.core}. \method{find_module()} must return a loader object
Andrew M. Kuchling974ab9d2002-12-31 01:20:30 +0000800that has a single method, \method{load_module(\var{fullname})}, that
801creates and returns the corresponding module object.
802
803Pseudo-code for Python's new import logic, therefore, looks something
804like this (simplified a bit; see \pep{302} for the full details):
805
806\begin{verbatim}
807for mp in sys.meta_path:
808 loader = mp(fullname)
809 if loader is not None:
Andrew M. Kuchlingd5ac8d02003-01-02 21:33:15 +0000810 <module> = loader.load_module(fullname)
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +0000811
Andrew M. Kuchling974ab9d2002-12-31 01:20:30 +0000812for path in sys.path:
813 for hook in sys.path_hooks:
Andrew M. Kuchlingd5ac8d02003-01-02 21:33:15 +0000814 try:
815 importer = hook(path)
816 except ImportError:
817 # ImportError, so try the other path hooks
818 pass
819 else:
Andrew M. Kuchling974ab9d2002-12-31 01:20:30 +0000820 loader = importer.find_module(fullname)
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +0000821 <module> = loader.load_module(fullname)
Andrew M. Kuchling974ab9d2002-12-31 01:20:30 +0000822
823# Not found!
824raise ImportError
825\end{verbatim}
826
827\begin{seealso}
828
829\seepep{302}{New Import Hooks}{Written by Just van~Rossum and Paul Moore.
Andrew M. Kuchlingae3bbf52002-12-31 14:03:45 +0000830Implemented by Just van~Rossum.
Andrew M. Kuchling974ab9d2002-12-31 01:20:30 +0000831}
832
833\end{seealso}
834
835
836%======================================================================
Andrew M. Kuchlinga978e102003-03-21 18:10:12 +0000837\section{PEP 305: Comma-separated Files \label{section-pep305}}
838
839Comma-separated files are a format frequently used for exporting data
840from databases and spreadsheets. Python 2.3 adds a parser for
841comma-separated files.
842The format is deceptively simple at first glance:
843
844\begin{verbatim}
845Costs,150,200,3.95
846\end{verbatim}
847
848Read a line and call \code{line.split(',')}: what could be simpler?
849But toss in string data that can contain commas, and things get more
850complicated:
851
852\begin{verbatim}
853"Costs",150,200,3.95,"Includes taxes, shipping, and sundry items"
854\end{verbatim}
855
856A big ugly regular expression can parse this, but using the new
857\module{csv} package is much simpler:
858
859\begin{verbatim}
860from csv import csv
861
862input = open('datafile', 'rb')
863reader = csv.reader(input)
864for line in reader:
865 print line
866\end{verbatim}
867
868The \function{reader} function takes a number of different options.
869The field separator isn't limited to the comma and can be changed to
870any character, and so can the quoting and line-ending characters.
871
872Different dialects of comma-separated files can be defined and
873registered; currently there are two, both for Microsoft Excel.
874A separate \class{csv.writer} class will generate comma-separated files
875from a succession of tuples or lists, quoting strings that contain the
876delimiter.
877
878\begin{seealso}
879
880\seepep{305}{CSV File API}{Written and implemented
881by Kevin Altis, Dave Cole, Andrew McNamara, Skip Montanaro, Cliff Wells.
882}
883
884\end{seealso}
885
886%======================================================================
Andrew M. Kuchlinge995d162002-07-11 20:09:50 +0000887\section{Extended Slices\label{section-slices}}
Michael W. Hudson5efaf7e2002-06-11 10:55:12 +0000888
Andrew M. Kuchlinge995d162002-07-11 20:09:50 +0000889Ever since Python 1.4, the slicing syntax has supported an optional
890third ``step'' or ``stride'' argument. For example, these are all
891legal Python syntax: \code{L[1:10:2]}, \code{L[:-1:1]},
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +0000892\code{L[::-1]}. This was added to Python at the request of
893the developers of Numerical Python, which uses the third argument
894extensively. However, Python's built-in list, tuple, and string
895sequence types have never supported this feature, and you got a
896\exception{TypeError} if you tried it. Michael Hudson contributed a
897patch to fix this shortcoming.
Andrew M. Kuchlinge995d162002-07-11 20:09:50 +0000898
899For example, you can now easily extract the elements of a list that
900have even indexes:
Fred Drakedf872a22002-07-03 12:02:01 +0000901
902\begin{verbatim}
903>>> L = range(10)
904>>> L[::2]
905[0, 2, 4, 6, 8]
906\end{verbatim}
907
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +0000908Negative values also work to make a copy of the same list in reverse
909order:
Fred Drakedf872a22002-07-03 12:02:01 +0000910
911\begin{verbatim}
912>>> L[::-1]
913[9, 8, 7, 6, 5, 4, 3, 2, 1, 0]
914\end{verbatim}
Andrew M. Kuchling3a52ff62002-04-03 22:44:47 +0000915
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +0000916This also works for tuples, arrays, and strings:
Andrew M. Kuchlinge995d162002-07-11 20:09:50 +0000917
918\begin{verbatim}
919>>> s='abcd'
920>>> s[::2]
921'ac'
922>>> s[::-1]
923'dcba'
924\end{verbatim}
925
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +0000926If you have a mutable sequence such as a list or an array you can
Michael W. Hudson4da01ed2002-07-19 15:48:56 +0000927assign to or delete an extended slice, but there are some differences
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +0000928between assignment to extended and regular slices. Assignment to a
929regular slice can be used to change the length of the sequence:
Michael W. Hudson4da01ed2002-07-19 15:48:56 +0000930
931\begin{verbatim}
932>>> a = range(3)
933>>> a
934[0, 1, 2]
935>>> a[1:3] = [4, 5, 6]
936>>> a
937[0, 4, 5, 6]
938\end{verbatim}
939
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +0000940Extended slices aren't this flexible. When assigning to an extended
941slice the list on the right hand side of the statement must contain
942the same number of items as the slice it is replacing:
Michael W. Hudson4da01ed2002-07-19 15:48:56 +0000943
944\begin{verbatim}
945>>> a = range(4)
946>>> a
947[0, 1, 2, 3]
948>>> a[::2]
949[0, 2]
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +0000950>>> a[::2] = [0, -1]
Michael W. Hudson4da01ed2002-07-19 15:48:56 +0000951>>> a
952[0, 1, -1, 3]
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +0000953>>> a[::2] = [0,1,2]
Michael W. Hudson4da01ed2002-07-19 15:48:56 +0000954Traceback (most recent call last):
955 File "<stdin>", line 1, in ?
Raymond Hettingeree1bded2003-01-17 16:20:23 +0000956ValueError: attempt to assign sequence of size 3 to extended slice of size 2
Michael W. Hudson4da01ed2002-07-19 15:48:56 +0000957\end{verbatim}
958
959Deletion is more straightforward:
960
961\begin{verbatim}
962>>> a = range(4)
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +0000963>>> a
964[0, 1, 2, 3]
Michael W. Hudson4da01ed2002-07-19 15:48:56 +0000965>>> a[::2]
966[0, 2]
967>>> del a[::2]
968>>> a
969[1, 3]
970\end{verbatim}
971
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +0000972One can also now pass slice objects to the
973\method{__getitem__} methods of the built-in sequences:
Michael W. Hudson4da01ed2002-07-19 15:48:56 +0000974
975\begin{verbatim}
976>>> range(10).__getitem__(slice(0, 5, 2))
977[0, 2, 4]
978\end{verbatim}
979
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +0000980Or use slice objects directly in subscripts:
Michael W. Hudson4da01ed2002-07-19 15:48:56 +0000981
982\begin{verbatim}
983>>> range(10)[slice(0, 5, 2)]
984[0, 2, 4]
985\end{verbatim}
986
Andrew M. Kuchlingb6f79592002-11-29 19:43:45 +0000987To simplify implementing sequences that support extended slicing,
988slice objects now have a method \method{indices(\var{length})} which,
Fred Drakeaac8c582003-01-17 22:50:10 +0000989given the length of a sequence, returns a \code{(\var{start},
990\var{stop}, \var{step})} tuple that can be passed directly to
991\function{range()}.
Andrew M. Kuchlingb6f79592002-11-29 19:43:45 +0000992\method{indices()} handles omitted and out-of-bounds indices in a
993manner consistent with regular slices (and this innocuous phrase hides
994a welter of confusing details!). The method is intended to be used
995like this:
Michael W. Hudson4da01ed2002-07-19 15:48:56 +0000996
997\begin{verbatim}
998class FakeSeq:
999 ...
1000 def calc_item(self, i):
1001 ...
1002 def __getitem__(self, item):
1003 if isinstance(item, slice):
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +00001004 indices = item.indices(len(self))
1005 return FakeSeq([self.calc_item(i) in range(*indices)])
Fred Drake5c4cf152002-11-13 14:59:06 +00001006 else:
Michael W. Hudson4da01ed2002-07-19 15:48:56 +00001007 return self.calc_item(i)
1008\end{verbatim}
1009
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +00001010From this example you can also see that the built-in \class{slice}
Andrew M. Kuchling90e9a792002-08-15 00:40:21 +00001011object is now the type object for the slice type, and is no longer a
1012function. This is consistent with Python 2.2, where \class{int},
1013\class{str}, etc., underwent the same change.
1014
Andrew M. Kuchlinge995d162002-07-11 20:09:50 +00001015
Andrew M. Kuchling3a52ff62002-04-03 22:44:47 +00001016%======================================================================
Fred Drakedf872a22002-07-03 12:02:01 +00001017\section{Other Language Changes}
Andrew M. Kuchling517109b2002-05-07 21:01:16 +00001018
Andrew M. Kuchlinge995d162002-07-11 20:09:50 +00001019Here are all of the changes that Python 2.3 makes to the core Python
1020language.
Andrew M. Kuchling517109b2002-05-07 21:01:16 +00001021
Andrew M. Kuchlinge995d162002-07-11 20:09:50 +00001022\begin{itemize}
1023\item The \keyword{yield} statement is now always a keyword, as
1024described in section~\ref{section-generators} of this document.
Andrew M. Kuchling517109b2002-05-07 21:01:16 +00001025
Fred Drake5c4cf152002-11-13 14:59:06 +00001026\item A new built-in function \function{enumerate()}
Andrew M. Kuchlinge995d162002-07-11 20:09:50 +00001027was added, as described in section~\ref{section-enumerate} of this
1028document.
Andrew M. Kuchling517109b2002-05-07 21:01:16 +00001029
Andrew M. Kuchlinge995d162002-07-11 20:09:50 +00001030\item Two new constants, \constant{True} and \constant{False} were
1031added along with the built-in \class{bool} type, as described in
1032section~\ref{section-bool} of this document.
Andrew M. Kuchling517109b2002-05-07 21:01:16 +00001033
Andrew M. Kuchling495172c2002-11-20 13:50:15 +00001034\item The \function{int()} type constructor will now return a long
1035integer instead of raising an \exception{OverflowError} when a string
1036or floating-point number is too large to fit into an integer. This
1037can lead to the paradoxical result that
Andrew M. Kuchling974ab9d2002-12-31 01:20:30 +00001038\code{isinstance(int(\var{expression}), int)} is false, but that seems
1039unlikely to cause problems in practice.
Andrew M. Kuchling495172c2002-11-20 13:50:15 +00001040
Fred Drake5c4cf152002-11-13 14:59:06 +00001041\item Built-in types now support the extended slicing syntax,
Andrew M. Kuchlinge995d162002-07-11 20:09:50 +00001042as described in section~\ref{section-slices} of this document.
Andrew M. Kuchling517109b2002-05-07 21:01:16 +00001043
Andrew M. Kuchlinge995d162002-07-11 20:09:50 +00001044\item Dictionaries have a new method, \method{pop(\var{key})}, that
1045returns the value corresponding to \var{key} and removes that
1046key/value pair from the dictionary. \method{pop()} will raise a
1047\exception{KeyError} if the requested key isn't present in the
1048dictionary:
1049
1050\begin{verbatim}
1051>>> d = {1:2}
1052>>> d
1053{1: 2}
1054>>> d.pop(4)
1055Traceback (most recent call last):
Andrew M. Kuchling28f2f882002-11-14 14:14:16 +00001056 File "stdin", line 1, in ?
Andrew M. Kuchlinge995d162002-07-11 20:09:50 +00001057KeyError: 4
1058>>> d.pop(1)
10592
1060>>> d.pop(1)
1061Traceback (most recent call last):
Andrew M. Kuchling28f2f882002-11-14 14:14:16 +00001062 File "stdin", line 1, in ?
Raymond Hettingeree1bded2003-01-17 16:20:23 +00001063KeyError: 'pop(): dictionary is empty'
Andrew M. Kuchlinge995d162002-07-11 20:09:50 +00001064>>> d
1065{}
1066>>>
1067\end{verbatim}
1068
Andrew M. Kuchlingb492fa92002-11-27 19:11:10 +00001069There's also a new class method,
1070\method{dict.fromkeys(\var{iterable}, \var{value})}, that
1071creates a dictionary with keys taken from the supplied iterator
1072\var{iterable} and all values set to \var{value}, defaulting to
1073\code{None}.
1074
1075(Patches contributed by Raymond Hettinger.)
Andrew M. Kuchlinge995d162002-07-11 20:09:50 +00001076
Andrew M. Kuchling974ab9d2002-12-31 01:20:30 +00001077Also, the \function{dict()} constructor now accepts keyword arguments to
Raymond Hettinger45bda572002-12-14 20:20:45 +00001078simplify creating small dictionaries:
Andrew M. Kuchling449a87d2002-12-11 15:03:51 +00001079
1080\begin{verbatim}
1081>>> dict(red=1, blue=2, green=3, black=4)
1082{'blue': 2, 'black': 4, 'green': 3, 'red': 1}
1083\end{verbatim}
1084
Andrew M. Kuchlingae3bbf52002-12-31 14:03:45 +00001085(Contributed by Just van~Rossum.)
Andrew M. Kuchling449a87d2002-12-11 15:03:51 +00001086
Andrew M. Kuchling7a82b8c2002-11-04 20:17:24 +00001087\item The \keyword{assert} statement no longer checks the \code{__debug__}
Andrew M. Kuchling6974aa92002-08-20 00:54:36 +00001088flag, so you can no longer disable assertions by assigning to \code{__debug__}.
Fred Drake5c4cf152002-11-13 14:59:06 +00001089Running Python with the \programopt{-O} switch will still generate
Andrew M. Kuchling6974aa92002-08-20 00:54:36 +00001090code that doesn't execute any assertions.
1091
1092\item Most type objects are now callable, so you can use them
1093to create new objects such as functions, classes, and modules. (This
1094means that the \module{new} module can be deprecated in a future
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +00001095Python version, because you can now use the type objects available in
1096the \module{types} module.)
Andrew M. Kuchling6974aa92002-08-20 00:54:36 +00001097% XXX should new.py use PendingDeprecationWarning?
1098For example, you can create a new module object with the following code:
1099
1100\begin{verbatim}
1101>>> import types
1102>>> m = types.ModuleType('abc','docstring')
1103>>> m
1104<module 'abc' (built-in)>
1105>>> m.__doc__
1106'docstring'
1107\end{verbatim}
1108
Fred Drake5c4cf152002-11-13 14:59:06 +00001109\item
Andrew M. Kuchling6974aa92002-08-20 00:54:36 +00001110A new warning, \exception{PendingDeprecationWarning} was added to
1111indicate features which are in the process of being
1112deprecated. The warning will \emph{not} be printed by default. To
1113check for use of features that will be deprecated in the future,
1114supply \programopt{-Walways::PendingDeprecationWarning::} on the
1115command line or use \function{warnings.filterwarnings()}.
1116
Andrew M. Kuchlingc1dd1742003-01-13 13:59:22 +00001117\item The process of deprecating string-based exceptions, as
1118in \code{raise "Error occurred"}, has begun. Raising a string will
1119now trigger \exception{PendingDeprecationWarning}.
1120
Andrew M. Kuchling6974aa92002-08-20 00:54:36 +00001121\item Using \code{None} as a variable name will now result in a
1122\exception{SyntaxWarning} warning. In a future version of Python,
1123\code{None} may finally become a keyword.
1124
Andrew M. Kuchlingb60ea3f2002-11-15 14:37:10 +00001125\item The method resolution order used by new-style classes has
1126changed, though you'll only notice the difference if you have a really
1127complicated inheritance hierarchy. (Classic classes are unaffected by
1128this change.) Python 2.2 originally used a topological sort of a
1129class's ancestors, but 2.3 now uses the C3 algorithm as described in
Andrew M. Kuchling6f429c32002-11-19 13:09:00 +00001130the paper \ulink{``A Monotonic Superclass Linearization for
1131Dylan''}{http://www.webcom.com/haahr/dylan/linearization-oopsla96.html}.
Andrew M. Kuchlingc1dd1742003-01-13 13:59:22 +00001132To understand the motivation for this change,
1133read Michele Simionato's article
Fred Drake693aea22003-02-07 14:52:18 +00001134\ulink{``Python 2.3 Method Resolution Order''}
Andrew M. Kuchlingb8a39052003-02-07 20:22:33 +00001135 {http://www.python.org/2.3/mro.html}, or
Andrew M. Kuchlingc1dd1742003-01-13 13:59:22 +00001136read the thread on python-dev starting with the message at
Andrew M. Kuchlingb60ea3f2002-11-15 14:37:10 +00001137\url{http://mail.python.org/pipermail/python-dev/2002-October/029035.html}.
1138Samuele Pedroni first pointed out the problem and also implemented the
1139fix by coding the C3 algorithm.
1140
Andrew M. Kuchlingdcfd8252002-09-13 22:21:42 +00001141\item Python runs multithreaded programs by switching between threads
1142after executing N bytecodes. The default value for N has been
1143increased from 10 to 100 bytecodes, speeding up single-threaded
1144applications by reducing the switching overhead. Some multithreaded
1145applications may suffer slower response time, but that's easily fixed
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +00001146by setting the limit back to a lower number using
Andrew M. Kuchlingdcfd8252002-09-13 22:21:42 +00001147\function{sys.setcheckinterval(\var{N})}.
1148
Andrew M. Kuchling6974aa92002-08-20 00:54:36 +00001149\item One minor but far-reaching change is that the names of extension
1150types defined by the modules included with Python now contain the
Fred Drake5c4cf152002-11-13 14:59:06 +00001151module and a \character{.} in front of the type name. For example, in
Andrew M. Kuchling6974aa92002-08-20 00:54:36 +00001152Python 2.2, if you created a socket and printed its
1153\member{__class__}, you'd get this output:
1154
1155\begin{verbatim}
1156>>> s = socket.socket()
1157>>> s.__class__
1158<type 'socket'>
1159\end{verbatim}
1160
1161In 2.3, you get this:
1162\begin{verbatim}
1163>>> s.__class__
1164<type '_socket.socket'>
1165\end{verbatim}
1166
Michael W. Hudson96bc3b42002-11-26 14:48:23 +00001167\item One of the noted incompatibilities between old- and new-style
1168 classes has been removed: you can now assign to the
1169 \member{__name__} and \member{__bases__} attributes of new-style
1170 classes. There are some restrictions on what can be assigned to
1171 \member{__bases__} along the lines of those relating to assigning to
1172 an instance's \member{__class__} attribute.
1173
Andrew M. Kuchling6974aa92002-08-20 00:54:36 +00001174\end{itemize}
1175
1176
Andrew M. Kuchling366c10c2002-11-14 23:07:57 +00001177%======================================================================
Andrew M. Kuchling6974aa92002-08-20 00:54:36 +00001178\subsection{String Changes}
1179
1180\begin{itemize}
1181
Fred Drakeaac8c582003-01-17 22:50:10 +00001182\item The \keyword{in} operator now works differently for strings.
Andrew M. Kuchling6974aa92002-08-20 00:54:36 +00001183Previously, when evaluating \code{\var{X} in \var{Y}} where \var{X}
1184and \var{Y} are strings, \var{X} could only be a single character.
1185That's now changed; \var{X} can be a string of any length, and
1186\code{\var{X} in \var{Y}} will return \constant{True} if \var{X} is a
1187substring of \var{Y}. If \var{X} is the empty string, the result is
1188always \constant{True}.
1189
1190\begin{verbatim}
1191>>> 'ab' in 'abcd'
1192True
1193>>> 'ad' in 'abcd'
1194False
1195>>> '' in 'abcd'
1196True
1197\end{verbatim}
1198
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +00001199Note that this doesn't tell you where the substring starts; if you
1200need that information, you must use the \method{find()} method
1201instead.
Andrew M. Kuchling6974aa92002-08-20 00:54:36 +00001202
Andrew M. Kuchlinge995d162002-07-11 20:09:50 +00001203\item The \method{strip()}, \method{lstrip()}, and \method{rstrip()}
1204string methods now have an optional argument for specifying the
1205characters to strip. The default is still to remove all whitespace
1206characters:
1207
1208\begin{verbatim}
1209>>> ' abc '.strip()
1210'abc'
1211>>> '><><abc<><><>'.strip('<>')
1212'abc'
1213>>> '><><abc<><><>\n'.strip('<>')
1214'abc<><><>\n'
1215>>> u'\u4000\u4001abc\u4000'.strip(u'\u4000')
1216u'\u4001abc'
1217>>>
1218\end{verbatim}
1219
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +00001220(Suggested by Simon Brunning and implemented by Walter D\"orwald.)
Andrew M. Kuchling346386f2002-07-12 20:24:42 +00001221
Andrew M. Kuchlinge995d162002-07-11 20:09:50 +00001222\item The \method{startswith()} and \method{endswith()}
1223string methods now accept negative numbers for the start and end
1224parameters.
1225
1226\item Another new string method is \method{zfill()}, originally a
1227function in the \module{string} module. \method{zfill()} pads a
1228numeric string with zeros on the left until it's the specified width.
1229Note that the \code{\%} operator is still more flexible and powerful
1230than \method{zfill()}.
1231
1232\begin{verbatim}
1233>>> '45'.zfill(4)
1234'0045'
1235>>> '12345'.zfill(4)
1236'12345'
1237>>> 'goofy'.zfill(6)
1238'0goofy'
1239\end{verbatim}
1240
Andrew M. Kuchling346386f2002-07-12 20:24:42 +00001241(Contributed by Walter D\"orwald.)
1242
Fred Drake5c4cf152002-11-13 14:59:06 +00001243\item A new type object, \class{basestring}, has been added.
Andrew M. Kuchling20e5abc2002-07-11 20:50:34 +00001244 Both 8-bit strings and Unicode strings inherit from this type, so
1245 \code{isinstance(obj, basestring)} will return \constant{True} for
1246 either kind of string. It's a completely abstract type, so you
1247 can't create \class{basestring} instances.
1248
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +00001249\item Interned strings are no longer immortal, and will now be
Andrew M. Kuchling6974aa92002-08-20 00:54:36 +00001250garbage-collected in the usual way when the only reference to them is
1251from the internal dictionary of interned strings. (Implemented by
1252Oren Tirosh.)
1253
1254\end{itemize}
1255
1256
Andrew M. Kuchling366c10c2002-11-14 23:07:57 +00001257%======================================================================
Andrew M. Kuchling6974aa92002-08-20 00:54:36 +00001258\subsection{Optimizations}
1259
1260\begin{itemize}
1261
Andrew M. Kuchling974ab9d2002-12-31 01:20:30 +00001262\item The creation of new-style class instances has been made much
1263faster; they're now faster than classic classes!
1264
Andrew M. Kuchling950725f2002-08-06 01:40:48 +00001265\item The \method{sort()} method of list objects has been extensively
1266rewritten by Tim Peters, and the implementation is significantly
1267faster.
1268
Andrew M. Kuchling6974aa92002-08-20 00:54:36 +00001269\item Multiplication of large long integers is now much faster thanks
1270to an implementation of Karatsuba multiplication, an algorithm that
1271scales better than the O(n*n) required for the grade-school
1272multiplication algorithm. (Original patch by Christopher A. Craig,
1273and significantly reworked by Tim Peters.)
Andrew M. Kuchling20e5abc2002-07-11 20:50:34 +00001274
Andrew M. Kuchling6974aa92002-08-20 00:54:36 +00001275\item The \code{SET_LINENO} opcode is now gone. This may provide a
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +00001276small speed increase, depending on your compiler's idiosyncrasies.
1277See section~\ref{section-other} for a longer explanation.
Andrew M. Kuchling6974aa92002-08-20 00:54:36 +00001278(Removed by Michael Hudson.)
Andrew M. Kuchling20e5abc2002-07-11 20:50:34 +00001279
Andrew M. Kuchling449a87d2002-12-11 15:03:51 +00001280\item \function{xrange()} objects now have their own iterator, making
1281\code{for i in xrange(n)} slightly faster than
1282\code{for i in range(n)}. (Patch by Raymond Hettinger.)
1283
Andrew M. Kuchling6974aa92002-08-20 00:54:36 +00001284\item A number of small rearrangements have been made in various
1285hotspots to improve performance, inlining a function here, removing
1286some code there. (Implemented mostly by GvR, but lots of people have
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +00001287contributed single changes.)
Andrew M. Kuchlinge995d162002-07-11 20:09:50 +00001288
1289\end{itemize}
Neal Norwitzd68f5172002-05-29 15:54:55 +00001290
Andrew M. Kuchling6974aa92002-08-20 00:54:36 +00001291
Andrew M. Kuchling517109b2002-05-07 21:01:16 +00001292%======================================================================
Andrew M. Kuchlingef893fe2003-01-06 20:04:17 +00001293\section{New, Improved, and Deprecated Modules}
Andrew M. Kuchling03594bb2002-03-27 02:29:48 +00001294
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +00001295As usual, Python's standard library received a number of enhancements and
Andrew M. Kuchlinga982eb12002-07-22 18:57:36 +00001296bug fixes. Here's a partial list of the most notable changes, sorted
1297alphabetically by module name. Consult the
1298\file{Misc/NEWS} file in the source tree for a more
1299complete list of changes, or look through the CVS logs for all the
1300details.
Andrew M. Kuchling821013e2002-05-06 17:46:39 +00001301
1302\begin{itemize}
1303
Andrew M. Kuchlinga982eb12002-07-22 18:57:36 +00001304\item The \module{array} module now supports arrays of Unicode
Fred Drake5c4cf152002-11-13 14:59:06 +00001305characters using the \character{u} format character. Arrays also now
Andrew M. Kuchlinga982eb12002-07-22 18:57:36 +00001306support using the \code{+=} assignment operator to add another array's
1307contents, and the \code{*=} assignment operator to repeat an array.
1308(Contributed by Jason Orendorff.)
1309
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +00001310\item The \module{bsddb} module has been replaced by version 4.1.1
Andrew M. Kuchling669249e2002-11-19 13:05:33 +00001311of the \ulink{PyBSDDB}{http://pybsddb.sourceforge.net} package,
1312providing a more complete interface to the transactional features of
1313the BerkeleyDB library.
1314The old version of the module has been renamed to
1315\module{bsddb185} and is no longer built automatically; you'll
1316have to edit \file{Modules/Setup} to enable it. Note that the new
1317\module{bsddb} package is intended to be compatible with the
1318old module, so be sure to file bugs if you discover any
Skip Montanaro959c7722003-03-07 15:45:15 +00001319incompatibilities. When upgrading to Python 2.3, if you also change
1320the underlying BerkeleyDB library, you will almost certainly have to
1321convert your database files to the new version. You can do this
1322fairly easily with the new scripts \file{db2pickle.py} and
1323\file{pickle2db.py} which you will find in the distribution's
1324Tools/scripts directory. If you've already been using the PyBSDDB
1325package, importing it as \module{bsddb3}, you will have to change your
1326\code{import} statements.
Andrew M. Kuchling669249e2002-11-19 13:05:33 +00001327
Fred Drake5c4cf152002-11-13 14:59:06 +00001328\item The Distutils \class{Extension} class now supports
1329an extra constructor argument named \var{depends} for listing
Andrew M. Kuchlinga982eb12002-07-22 18:57:36 +00001330additional source files that an extension depends on. This lets
1331Distutils recompile the module if any of the dependency files are
Fred Drake5c4cf152002-11-13 14:59:06 +00001332modified. For example, if \file{sampmodule.c} includes the header
Andrew M. Kuchlinga982eb12002-07-22 18:57:36 +00001333file \file{sample.h}, you would create the \class{Extension} object like
1334this:
1335
1336\begin{verbatim}
1337ext = Extension("samp",
1338 sources=["sampmodule.c"],
1339 depends=["sample.h"])
1340\end{verbatim}
1341
1342Modifying \file{sample.h} would then cause the module to be recompiled.
1343(Contributed by Jeremy Hylton.)
1344
Andrew M. Kuchlingdc3f7e12002-11-04 20:05:10 +00001345\item Other minor changes to Distutils:
1346it now checks for the \envvar{CC}, \envvar{CFLAGS}, \envvar{CPP},
1347\envvar{LDFLAGS}, and \envvar{CPPFLAGS} environment variables, using
1348them to override the settings in Python's configuration (contributed
Andrew M. Kuchlinga31bb372003-01-27 16:36:34 +00001349by Robert Weber).
Andrew M. Kuchlingdc3f7e12002-11-04 20:05:10 +00001350
Andrew M. Kuchlinga982eb12002-07-22 18:57:36 +00001351\item The \module{getopt} module gained a new function,
1352\function{gnu_getopt()}, that supports the same arguments as the existing
Fred Drake5c4cf152002-11-13 14:59:06 +00001353\function{getopt()} function but uses GNU-style scanning mode.
Andrew M. Kuchlinga982eb12002-07-22 18:57:36 +00001354The existing \function{getopt()} stops processing options as soon as a
1355non-option argument is encountered, but in GNU-style mode processing
1356continues, meaning that options and arguments can be mixed. For
1357example:
1358
1359\begin{verbatim}
1360>>> getopt.getopt(['-f', 'filename', 'output', '-v'], 'f:v')
1361([('-f', 'filename')], ['output', '-v'])
1362>>> getopt.gnu_getopt(['-f', 'filename', 'output', '-v'], 'f:v')
1363([('-f', 'filename'), ('-v', '')], ['output'])
1364\end{verbatim}
1365
1366(Contributed by Peter \AA{strand}.)
1367
1368\item The \module{grp}, \module{pwd}, and \module{resource} modules
Fred Drake5c4cf152002-11-13 14:59:06 +00001369now return enhanced tuples:
Andrew M. Kuchlinga982eb12002-07-22 18:57:36 +00001370
1371\begin{verbatim}
1372>>> import grp
1373>>> g = grp.getgrnam('amk')
1374>>> g.gr_name, g.gr_gid
1375('amk', 500)
1376\end{verbatim}
1377
Andrew M. Kuchling974ab9d2002-12-31 01:20:30 +00001378\item The \module{gzip} module can now handle files exceeding 2~Gb.
1379
Andrew M. Kuchling950725f2002-08-06 01:40:48 +00001380\item The new \module{heapq} module contains an implementation of a
1381heap queue algorithm. A heap is an array-like data structure that
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +00001382keeps items in a partially sorted order such that, for every index
1383\var{k}, \code{heap[\var{k}] <= heap[2*\var{k}+1]} and
1384\code{heap[\var{k}] <= heap[2*\var{k}+2]}. This makes it quick to
1385remove the smallest item, and inserting a new item while maintaining
1386the heap property is O(lg~n). (See
Andrew M. Kuchling950725f2002-08-06 01:40:48 +00001387\url{http://www.nist.gov/dads/HTML/priorityque.html} for more
1388information about the priority queue data structure.)
1389
Andrew M. Kuchling8a61f492002-11-13 13:24:41 +00001390The \module{heapq} module provides \function{heappush()} and
Andrew M. Kuchling950725f2002-08-06 01:40:48 +00001391\function{heappop()} functions for adding and removing items while
1392maintaining the heap property on top of some other mutable Python
1393sequence type. For example:
1394
1395\begin{verbatim}
1396>>> import heapq
1397>>> heap = []
1398>>> for item in [3, 7, 5, 11, 1]:
1399... heapq.heappush(heap, item)
1400...
1401>>> heap
1402[1, 3, 5, 11, 7]
1403>>> heapq.heappop(heap)
14041
1405>>> heapq.heappop(heap)
14063
1407>>> heap
1408[5, 7, 11]
Andrew M. Kuchling950725f2002-08-06 01:40:48 +00001409\end{verbatim}
1410
1411(Contributed by Kevin O'Connor.)
Andrew M. Kuchlinga982eb12002-07-22 18:57:36 +00001412
Andrew M. Kuchling87cebbf2003-01-03 16:24:28 +00001413\item The \module{imaplib} module now supports IMAP over SSL.
1414(Contributed by Piers Lauder and Tino Lange.)
1415
Andrew M. Kuchling41c3e002003-03-02 02:13:52 +00001416\item The \module{itertools} contains a number of useful functions for
1417use with iterators, inspired by various functions provided by the ML
1418and Haskell languages. For example,
1419\code{itertools.ifilter(predicate, iterator)} returns all elements in
1420the iterator for which the function \function{predicate()} returns
Andrew M. Kuchling563389f2003-03-02 02:31:58 +00001421\constant{True}, and \code{itertools.repeat(obj, \var{N})} returns
Andrew M. Kuchling41c3e002003-03-02 02:13:52 +00001422\code{obj} \var{N} times. There are a number of other functions in
1423the module; see the \ulink{package's reference
1424documentation}{../lib/module-itertools.html} for details.
Raymond Hettinger5284b442003-03-09 07:19:38 +00001425(Contributed by Raymond Hettinger.)
Fred Drakecade7132003-02-19 16:08:08 +00001426
Fred Drake5c4cf152002-11-13 14:59:06 +00001427\item Two new functions in the \module{math} module,
Andrew M. Kuchlinga982eb12002-07-22 18:57:36 +00001428\function{degrees(\var{rads})} and \function{radians(\var{degs})},
Fred Drake5c4cf152002-11-13 14:59:06 +00001429convert between radians and degrees. Other functions in the
Andrew M. Kuchling8e5b53b2002-12-15 20:17:38 +00001430\module{math} module such as \function{math.sin()} and
1431\function{math.cos()} have always required input values measured in
1432radians. Also, an optional \var{base} argument was added to
1433\function{math.log()} to make it easier to compute logarithms for
1434bases other than \code{e} and \code{10}. (Contributed by Raymond
1435Hettinger.)
Andrew M. Kuchlinga982eb12002-07-22 18:57:36 +00001436
Andrew M. Kuchlingae3bbf52002-12-31 14:03:45 +00001437\item Several new functions (\function{getpgid()}, \function{killpg()},
1438\function{lchown()}, \function{loadavg()}, \function{major()}, \function{makedev()},
1439\function{minor()}, and \function{mknod()}) were added to the
Andrew M. Kuchlingc309cca2002-10-10 16:04:08 +00001440\module{posix} module that underlies the \module{os} module.
Andrew M. Kuchlingae3bbf52002-12-31 14:03:45 +00001441(Contributed by Gustavo Niemeyer, Geert Jansen, and Denis S. Otkidach.)
Andrew M. Kuchlinga982eb12002-07-22 18:57:36 +00001442
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +00001443\item In the \module{os} module, the \function{*stat()} family of functions can now report
1444fractions of a second in a timestamp. Such time stamps are
1445represented as floats, similar to \function{time.time()}.
1446
1447During testing, it was found that some applications will break if time
1448stamps are floats. For compatibility, when using the tuple interface
1449of the \class{stat_result} time stamps will be represented as integers.
1450When using named fields (a feature first introduced in Python 2.2),
1451time stamps are still represented as integers, unless
1452\function{os.stat_float_times()} is invoked to enable float return
1453values:
1454
1455\begin{verbatim}
1456>>> os.stat("/tmp").st_mtime
14571034791200
1458>>> os.stat_float_times(True)
1459>>> os.stat("/tmp").st_mtime
14601034791200.6335014
1461\end{verbatim}
1462
1463In Python 2.4, the default will change to always returning floats.
1464
1465Application developers should enable this feature only if all their
1466libraries work properly when confronted with floating point time
1467stamps, or if they use the tuple API. If used, the feature should be
1468activated on an application level instead of trying to enable it on a
1469per-use basis.
1470
Andrew M. Kuchling53262572002-12-01 14:00:21 +00001471\item The old and never-documented \module{linuxaudiodev} module has
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +00001472been deprecated, and a new version named \module{ossaudiodev} has been
1473added. The module was renamed because the OSS sound drivers can be
1474used on platforms other than Linux, and the interface has also been
1475tidied and brought up to date in various ways. (Contributed by Greg
Greg Wardaa1d3aa2003-01-03 18:03:21 +00001476Ward and Nicholas FitzRoy-Dale.)
Andrew M. Kuchling53262572002-12-01 14:00:21 +00001477
Fred Drake5c4cf152002-11-13 14:59:06 +00001478\item The parser objects provided by the \module{pyexpat} module
Andrew M. Kuchlinga982eb12002-07-22 18:57:36 +00001479can now optionally buffer character data, resulting in fewer calls to
1480your character data handler and therefore faster performance. Setting
Fred Drake5c4cf152002-11-13 14:59:06 +00001481the parser object's \member{buffer_text} attribute to \constant{True}
Andrew M. Kuchlinga982eb12002-07-22 18:57:36 +00001482will enable buffering.
1483
Andrew M. Kuchling8a61f492002-11-13 13:24:41 +00001484\item The \function{sample(\var{population}, \var{k})} function was
1485added to the \module{random} module. \var{population} is a sequence
Fred Drakeaac8c582003-01-17 22:50:10 +00001486or \class{xrange} object containing the elements of a population, and
1487\function{sample()}
Andrew M. Kuchling8a61f492002-11-13 13:24:41 +00001488chooses \var{k} elements from the population without replacing chosen
1489elements. \var{k} can be any value up to \code{len(\var{population})}.
1490For example:
1491
1492\begin{verbatim}
Andrew M. Kuchling449a87d2002-12-11 15:03:51 +00001493>>> days = ['Mo', 'Tu', 'We', 'Th', 'Fr', 'St', 'Sn']
Michael W. Hudsoncfd38842002-12-17 16:15:34 +00001494>>> random.sample(days, 3) # Choose 3 elements
Andrew M. Kuchling449a87d2002-12-11 15:03:51 +00001495['St', 'Sn', 'Th']
Michael W. Hudsoncfd38842002-12-17 16:15:34 +00001496>>> random.sample(days, 7) # Choose 7 elements
Andrew M. Kuchling449a87d2002-12-11 15:03:51 +00001497['Tu', 'Th', 'Mo', 'We', 'St', 'Fr', 'Sn']
Michael W. Hudsoncfd38842002-12-17 16:15:34 +00001498>>> random.sample(days, 7) # Choose 7 again
Andrew M. Kuchling449a87d2002-12-11 15:03:51 +00001499['We', 'Mo', 'Sn', 'Fr', 'Tu', 'St', 'Th']
Michael W. Hudsoncfd38842002-12-17 16:15:34 +00001500>>> random.sample(days, 8) # Can't choose eight
Andrew M. Kuchling8a61f492002-11-13 13:24:41 +00001501Traceback (most recent call last):
Andrew M. Kuchling28f2f882002-11-14 14:14:16 +00001502 File "<stdin>", line 1, in ?
Andrew M. Kuchling449a87d2002-12-11 15:03:51 +00001503 File "random.py", line 414, in sample
1504 raise ValueError, "sample larger than population"
Andrew M. Kuchling8a61f492002-11-13 13:24:41 +00001505ValueError: sample larger than population
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +00001506>>> random.sample(xrange(1,10000,2), 10) # Choose ten odd nos. under 10000
Andrew M. Kuchling449a87d2002-12-11 15:03:51 +00001507[3407, 3805, 1505, 7023, 2401, 2267, 9733, 3151, 8083, 9195]
Andrew M. Kuchling8a61f492002-11-13 13:24:41 +00001508\end{verbatim}
Andrew M. Kuchling974ab9d2002-12-31 01:20:30 +00001509
1510The \module{random} module now uses a new algorithm, the Mersenne
1511Twister, implemented in C. It's faster and more extensively studied
1512than the previous algorithm.
1513
1514(All changes contributed by Raymond Hettinger.)
Andrew M. Kuchling8a61f492002-11-13 13:24:41 +00001515
Andrew M. Kuchlinga982eb12002-07-22 18:57:36 +00001516\item The \module{readline} module also gained a number of new
1517functions: \function{get_history_item()},
1518\function{get_current_history_length()}, and \function{redisplay()}.
1519
Andrew M. Kuchlingef893fe2003-01-06 20:04:17 +00001520\item The \module{rexec} and \module{Bastion} modules have been
1521declared dead, and attempts to import them will fail with a
1522\exception{RuntimeError}. New-style classes provide new ways to break
1523out of the restricted execution environment provided by
1524\module{rexec}, and no one has interest in fixing them or time to do
1525so. If you have applications using \module{rexec}, rewrite them to
1526use something else.
1527
1528(Sticking with Python 2.2 or 2.1 will not make your applications any
1529safer, because there are known bugs in the \module{rexec} module in
1530those versions. I repeat, if you're using \module{rexec}, stop using
1531it immediately.)
1532
Andrew M. Kuchling974ab9d2002-12-31 01:20:30 +00001533\item The \module{shutil} module gained a \function{move(\var{src},
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +00001534\var{dest})} function that recursively moves a file or directory to a new
Andrew M. Kuchling974ab9d2002-12-31 01:20:30 +00001535location.
1536
Andrew M. Kuchlinga982eb12002-07-22 18:57:36 +00001537\item Support for more advanced POSIX signal handling was added
Michael W. Hudson43ed43b2003-03-13 13:56:53 +00001538to the \module{signal} but then removed again as it proved impossible
1539to make it work reliably across platforms.
Andrew M. Kuchlinga982eb12002-07-22 18:57:36 +00001540
1541\item The \module{socket} module now supports timeouts. You
1542can call the \method{settimeout(\var{t})} method on a socket object to
1543set a timeout of \var{t} seconds. Subsequent socket operations that
1544take longer than \var{t} seconds to complete will abort and raise a
Fred Drake5c4cf152002-11-13 14:59:06 +00001545\exception{socket.error} exception.
Andrew M. Kuchlinga982eb12002-07-22 18:57:36 +00001546
1547The original timeout implementation was by Tim O'Malley. Michael
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +00001548Gilfix integrated it into the Python \module{socket} module and
1549shepherded it through a lengthy review. After the code was checked
1550in, Guido van~Rossum rewrote parts of it. (This is a good example of
1551a collaborative development process in action.)
Andrew M. Kuchlinga982eb12002-07-22 18:57:36 +00001552
Mark Hammond8af50bc2002-12-03 06:13:35 +00001553\item On Windows, the \module{socket} module now ships with Secure
Michael W. Hudson065f5fa2003-02-10 19:24:50 +00001554Sockets Layer (SSL) support.
Mark Hammond8af50bc2002-12-03 06:13:35 +00001555
Andrew M. Kuchling563389f2003-03-02 02:31:58 +00001556\item The value of the C \constant{PYTHON_API_VERSION} macro is now
1557exposed at the Python level as \code{sys.api_version}. The current
1558exception can be cleared by calling the new \function{sys.exc_clear()}
1559function.
Andrew M. Kuchlingdcfd8252002-09-13 22:21:42 +00001560
Andrew M. Kuchling674b0bf2003-01-07 00:07:19 +00001561\item The new \module{tarfile} module
Neal Norwitz55d555f2003-01-08 05:27:42 +00001562allows reading from and writing to \program{tar}-format archive files.
Andrew M. Kuchling674b0bf2003-01-07 00:07:19 +00001563(Contributed by Lars Gust\"abel.)
1564
Andrew M. Kuchling20e5abc2002-07-11 20:50:34 +00001565\item The new \module{textwrap} module contains functions for wrapping
Andrew M. Kuchlingd003a2a2002-06-26 13:23:55 +00001566strings containing paragraphs of text. The \function{wrap(\var{text},
1567\var{width})} function takes a string and returns a list containing
1568the text split into lines of no more than the chosen width. The
1569\function{fill(\var{text}, \var{width})} function returns a single
1570string, reformatted to fit into lines no longer than the chosen width.
1571(As you can guess, \function{fill()} is built on top of
1572\function{wrap()}. For example:
1573
1574\begin{verbatim}
1575>>> import textwrap
1576>>> paragraph = "Not a whit, we defy augury: ... more text ..."
1577>>> textwrap.wrap(paragraph, 60)
Fred Drake5c4cf152002-11-13 14:59:06 +00001578["Not a whit, we defy augury: there's a special providence in",
1579 "the fall of a sparrow. If it be now, 'tis not to come; if it",
Andrew M. Kuchlingd003a2a2002-06-26 13:23:55 +00001580 ...]
1581>>> print textwrap.fill(paragraph, 35)
1582Not a whit, we defy augury: there's
1583a special providence in the fall of
1584a sparrow. If it be now, 'tis not
1585to come; if it be not to come, it
1586will be now; if it be not now, yet
1587it will come: the readiness is all.
Fred Drake5c4cf152002-11-13 14:59:06 +00001588>>>
Andrew M. Kuchlingd003a2a2002-06-26 13:23:55 +00001589\end{verbatim}
1590
1591The module also contains a \class{TextWrapper} class that actually
Fred Drake5c4cf152002-11-13 14:59:06 +00001592implements the text wrapping strategy. Both the
Andrew M. Kuchlingd003a2a2002-06-26 13:23:55 +00001593\class{TextWrapper} class and the \function{wrap()} and
1594\function{fill()} functions support a number of additional keyword
1595arguments for fine-tuning the formatting; consult the module's
Fred Drake5c4cf152002-11-13 14:59:06 +00001596documentation for details.
Andrew M. Kuchling366c10c2002-11-14 23:07:57 +00001597%XXX add a link to the module docs?
Andrew M. Kuchlingd003a2a2002-06-26 13:23:55 +00001598(Contributed by Greg Ward.)
1599
Andrew M. Kuchling974ab9d2002-12-31 01:20:30 +00001600\item The \module{thread} and \module{threading} modules now have
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +00001601companion modules, \module{dummy_thread} and \module{dummy_threading},
1602that provide a do-nothing implementation of the \module{thread}
1603module's interface for platforms where threads are not supported. The
1604intention is to simplify thread-aware modules (ones that \emph{don't}
1605rely on threads to run) by putting the following code at the top:
Andrew M. Kuchling974ab9d2002-12-31 01:20:30 +00001606
1607% XXX why as _threading?
1608\begin{verbatim}
1609try:
1610 import threading as _threading
1611except ImportError:
1612 import dummy_threading as _threading
1613\end{verbatim}
1614
1615Code can then call functions and use classes in \module{_threading}
1616whether or not threads are supported, avoiding an \keyword{if}
1617statement and making the code slightly clearer. This module will not
1618magically make multithreaded code run without threads; code that waits
1619for another thread to return or to do something will simply hang
1620forever.
1621
Andrew M. Kuchlingef5d06b2002-07-22 19:21:06 +00001622\item The \module{time} module's \function{strptime()} function has
Fred Drake5c4cf152002-11-13 14:59:06 +00001623long been an annoyance because it uses the platform C library's
Andrew M. Kuchlingef5d06b2002-07-22 19:21:06 +00001624\function{strptime()} implementation, and different platforms
1625sometimes have odd bugs. Brett Cannon contributed a portable
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +00001626implementation that's written in pure Python and should behave
Andrew M. Kuchlingef5d06b2002-07-22 19:21:06 +00001627identically on all platforms.
1628
Andrew M. Kuchling6c50df22002-12-13 12:53:16 +00001629\item The \module{UserDict} module has a new \class{DictMixin} class which
Andrew M. Kuchling449a87d2002-12-11 15:03:51 +00001630defines all dictionary methods for classes that already have a minimum
1631mapping interface. This greatly simplifies writing classes that need
1632to be substitutable for dictionaries, such as the classes in
1633the \module{shelve} module.
1634
1635Adding the mixin as a superclass provides the full dictionary
1636interface whenever the class defines \method{__getitem__},
Andrew M. Kuchling6c50df22002-12-13 12:53:16 +00001637\method{__setitem__}, \method{__delitem__}, and \method{keys}.
Andrew M. Kuchling449a87d2002-12-11 15:03:51 +00001638For example:
1639
1640\begin{verbatim}
1641>>> import UserDict
1642>>> class SeqDict(UserDict.DictMixin):
1643 """Dictionary lookalike implemented with lists."""
1644 def __init__(self):
1645 self.keylist = []
1646 self.valuelist = []
1647 def __getitem__(self, key):
1648 try:
1649 i = self.keylist.index(key)
1650 except ValueError:
1651 raise KeyError
1652 return self.valuelist[i]
1653 def __setitem__(self, key, value):
1654 try:
1655 i = self.keylist.index(key)
1656 self.valuelist[i] = value
1657 except ValueError:
1658 self.keylist.append(key)
1659 self.valuelist.append(value)
1660 def __delitem__(self, key):
1661 try:
1662 i = self.keylist.index(key)
1663 except ValueError:
1664 raise KeyError
1665 self.keylist.pop(i)
1666 self.valuelist.pop(i)
1667 def keys(self):
1668 return list(self.keylist)
1669
1670>>> s = SeqDict()
1671>>> dir(s) # See that other dictionary methods are implemented
1672['__cmp__', '__contains__', '__delitem__', '__doc__', '__getitem__',
1673 '__init__', '__iter__', '__len__', '__module__', '__repr__',
1674 '__setitem__', 'clear', 'get', 'has_key', 'items', 'iteritems',
1675 'iterkeys', 'itervalues', 'keylist', 'keys', 'pop', 'popitem',
1676 'setdefault', 'update', 'valuelist', 'values']
Neal Norwitzc7d8c682002-12-24 14:51:43 +00001677\end{verbatim}
Andrew M. Kuchling449a87d2002-12-11 15:03:51 +00001678
1679(Contributed by Raymond Hettinger.)
1680
Andrew M. Kuchling20e5abc2002-07-11 20:50:34 +00001681\item The DOM implementation
1682in \module{xml.dom.minidom} can now generate XML output in a
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +00001683particular encoding by providing an optional encoding argument to
Andrew M. Kuchling20e5abc2002-07-11 20:50:34 +00001684the \method{toxml()} and \method{toprettyxml()} methods of DOM nodes.
1685
Andrew M. Kuchlingef893fe2003-01-06 20:04:17 +00001686item The \module{Tix} module has received various bug fixes and
1687updates for the current version of the Tix package.
1688
Andrew M. Kuchling6c50df22002-12-13 12:53:16 +00001689\item The \module{Tkinter} module now works with a thread-enabled
1690version of Tcl. Tcl's threading model requires that widgets only be
1691accessed from the thread in which they're created; accesses from
1692another thread can cause Tcl to panic. For certain Tcl interfaces,
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +00001693\module{Tkinter} will now automatically avoid this
1694when a widget is accessed from a different thread by marshalling a
1695command, passing it to the correct thread, and waiting for the
1696results. Other interfaces can't be handled automatically but
1697\module{Tkinter} will now raise an exception on such an access so that
1698at least you can find out about the problem. See
Andrew M. Kuchling6c50df22002-12-13 12:53:16 +00001699\url{http://mail.python.org/pipermail/python-dev/2002-December/031107.html}
1700for a more detailed explanation of this change. (Implemented by
1701Martin von L\"owis.)
1702
Andrew M. Kuchlingb492fa92002-11-27 19:11:10 +00001703\item Calling Tcl methods through \module{_tkinter} no longer
1704returns only strings. Instead, if Tcl returns other objects those
1705objects are converted to their Python equivalent, if one exists, or
1706wrapped with a \class{_tkinter.Tcl_Obj} object if no Python equivalent
Raymond Hettinger45bda572002-12-14 20:20:45 +00001707exists. This behavior can be controlled through the
Andrew M. Kuchlingb492fa92002-11-27 19:11:10 +00001708\method{wantobjects()} method of \class{tkapp} objects.
Martin v. Löwis39b48522002-11-26 09:47:25 +00001709
Andrew M. Kuchlingb492fa92002-11-27 19:11:10 +00001710When using \module{_tkinter} through the \module{Tkinter} module (as
1711most Tkinter applications will), this feature is always activated. It
1712should not cause compatibility problems, since Tkinter would always
1713convert string results to Python types where possible.
Martin v. Löwis39b48522002-11-26 09:47:25 +00001714
Raymond Hettinger45bda572002-12-14 20:20:45 +00001715If any incompatibilities are found, the old behavior can be restored
Andrew M. Kuchlingb492fa92002-11-27 19:11:10 +00001716by setting the \member{wantobjects} variable in the \module{Tkinter}
1717module to false before creating the first \class{tkapp} object.
Martin v. Löwis39b48522002-11-26 09:47:25 +00001718
1719\begin{verbatim}
1720import Tkinter
Martin v. Löwis8c8aa5d2002-11-26 21:39:48 +00001721Tkinter.wantobjects = 0
Martin v. Löwis39b48522002-11-26 09:47:25 +00001722\end{verbatim}
1723
Andrew M. Kuchling6c50df22002-12-13 12:53:16 +00001724Any breakage caused by this change should be reported as a bug.
Martin v. Löwis39b48522002-11-26 09:47:25 +00001725
Andrew M. Kuchling821013e2002-05-06 17:46:39 +00001726\end{itemize}
1727
Andrew M. Kuchling03594bb2002-03-27 02:29:48 +00001728
Andrew M. Kuchlingef5d06b2002-07-22 19:21:06 +00001729%======================================================================
Andrew M. Kuchlinga974b392003-01-13 19:09:03 +00001730\subsection{Date/Time Type}
1731
1732Date and time types suitable for expressing timestamps were added as
1733the \module{datetime} module. The types don't support different
1734calendars or many fancy features, and just stick to the basics of
1735representing time.
1736
1737The three primary types are: \class{date}, representing a day, month,
1738and year; \class{time}, consisting of hour, minute, and second; and
1739\class{datetime}, which contains all the attributes of both
Andrew M. Kuchlingc71bb972003-03-21 17:23:07 +00001740\class{date} and \class{time}. There's also a
1741\class{timedelta} class representing differences between two points
Andrew M. Kuchlinga974b392003-01-13 19:09:03 +00001742in time, and time zone logic is implemented by classes inheriting from
1743the abstract \class{tzinfo} class.
1744
1745You can create instances of \class{date} and \class{time} by either
1746supplying keyword arguments to the appropriate constructor,
1747e.g. \code{datetime.date(year=1972, month=10, day=15)}, or by using
Andrew M. Kuchlingc71bb972003-03-21 17:23:07 +00001748one of a number of class methods. For example, the \method{date.today()}
Andrew M. Kuchlinga974b392003-01-13 19:09:03 +00001749class method returns the current local date.
1750
1751Once created, instances of the date/time classes are all immutable.
1752There are a number of methods for producing formatted strings from
1753objects:
1754
1755\begin{verbatim}
1756>>> import datetime
1757>>> now = datetime.datetime.now()
1758>>> now.isoformat()
1759'2002-12-30T21:27:03.994956'
1760>>> now.ctime() # Only available on date, datetime
1761'Mon Dec 30 21:27:03 2002'
Raymond Hettingeree1bded2003-01-17 16:20:23 +00001762>>> now.strftime('%Y %d %b')
Andrew M. Kuchlinga974b392003-01-13 19:09:03 +00001763'2002 30 Dec'
1764\end{verbatim}
1765
1766The \method{replace()} method allows modifying one or more fields
1767of a \class{date} or \class{datetime} instance:
1768
1769\begin{verbatim}
1770>>> d = datetime.datetime.now()
1771>>> d
1772datetime.datetime(2002, 12, 30, 22, 15, 38, 827738)
1773>>> d.replace(year=2001, hour = 12)
1774datetime.datetime(2001, 12, 30, 12, 15, 38, 827738)
1775>>>
1776\end{verbatim}
1777
1778Instances can be compared, hashed, and converted to strings (the
1779result is the same as that of \method{isoformat()}). \class{date} and
1780\class{datetime} instances can be subtracted from each other, and
Andrew M. Kuchlingc71bb972003-03-21 17:23:07 +00001781added to \class{timedelta} instances. The largest missing feature is
1782that there's no support for parsing strings and getting back a
1783\class{date} or \class{datetime}.
Andrew M. Kuchlinga974b392003-01-13 19:09:03 +00001784
1785For more information, refer to the \ulink{module's reference
Fred Drake693aea22003-02-07 14:52:18 +00001786documentation}{..//lib/module-datetime.html}.
Andrew M. Kuchlinga974b392003-01-13 19:09:03 +00001787(Contributed by Tim Peters.)
1788
1789
1790%======================================================================
Andrew M. Kuchling24d5a522002-11-14 23:40:42 +00001791\subsection{The \module{optparse} Module}
Andrew M. Kuchling366c10c2002-11-14 23:07:57 +00001792
Andrew M. Kuchling24d5a522002-11-14 23:40:42 +00001793The \module{getopt} module provides simple parsing of command-line
1794arguments. The new \module{optparse} module (originally named Optik)
1795provides more elaborate command-line parsing that follows the Unix
1796conventions, automatically creates the output for \longprogramopt{help},
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +00001797and can perform different actions for different options.
Andrew M. Kuchling24d5a522002-11-14 23:40:42 +00001798
1799You start by creating an instance of \class{OptionParser} and telling
1800it what your program's options are.
1801
1802\begin{verbatim}
Andrew M. Kuchling7ee9b512003-02-18 00:48:23 +00001803import sys
Andrew M. Kuchling24d5a522002-11-14 23:40:42 +00001804from optparse import OptionParser
1805
1806op = OptionParser()
1807op.add_option('-i', '--input',
1808 action='store', type='string', dest='input',
1809 help='set input filename')
1810op.add_option('-l', '--length',
1811 action='store', type='int', dest='length',
1812 help='set maximum length of output')
1813\end{verbatim}
1814
1815Parsing a command line is then done by calling the \method{parse_args()}
1816method.
1817
1818\begin{verbatim}
Andrew M. Kuchling7ee9b512003-02-18 00:48:23 +00001819import optparse
1820
1821options, args = optparse.parse_args(sys.argv[1:])
Andrew M. Kuchling24d5a522002-11-14 23:40:42 +00001822print options
1823print args
1824\end{verbatim}
1825
1826This returns an object containing all of the option values,
1827and a list of strings containing the remaining arguments.
1828
1829Invoking the script with the various arguments now works as you'd
1830expect it to. Note that the length argument is automatically
1831converted to an integer.
1832
1833\begin{verbatim}
1834$ ./python opt.py -i data arg1
1835<Values at 0x400cad4c: {'input': 'data', 'length': None}>
1836['arg1']
1837$ ./python opt.py --input=data --length=4
1838<Values at 0x400cad2c: {'input': 'data', 'length': 4}>
Andrew M. Kuchling7ee9b512003-02-18 00:48:23 +00001839[]
Andrew M. Kuchling24d5a522002-11-14 23:40:42 +00001840$
1841\end{verbatim}
1842
1843The help message is automatically generated for you:
1844
1845\begin{verbatim}
1846$ ./python opt.py --help
1847usage: opt.py [options]
1848
1849options:
1850 -h, --help show this help message and exit
1851 -iINPUT, --input=INPUT
1852 set input filename
1853 -lLENGTH, --length=LENGTH
1854 set maximum length of output
1855$
1856\end{verbatim}
Andrew M. Kuchling669249e2002-11-19 13:05:33 +00001857% $ prevent Emacs tex-mode from getting confused
Andrew M. Kuchling24d5a522002-11-14 23:40:42 +00001858
1859Optik was written by Greg Ward, with suggestions from the readers of
1860the Getopt SIG.
1861
1862\begin{seealso}
Fred Drake693aea22003-02-07 14:52:18 +00001863\seeurl{http://optik.sourceforge.net/}
Andrew M. Kuchling24d5a522002-11-14 23:40:42 +00001864{The Optik site has tutorial and reference documentation for
1865\module{optparse}.
1866% XXX change to point to Python docs, when those docs get written.
1867}
1868\end{seealso}
Andrew M. Kuchling366c10c2002-11-14 23:07:57 +00001869
1870
1871%======================================================================
Andrew M. Kuchlingef5d06b2002-07-22 19:21:06 +00001872\section{Specialized Object Allocator (pymalloc)\label{section-pymalloc}}
1873
Andrew M. Kuchlingc71bb972003-03-21 17:23:07 +00001874Pymalloc, a specialized object allocator written by Vladimir
1875Marangozov, was a feature added to Python 2.1. Pymalloc is intended
1876to be faster than the system \cfunction{malloc()} and to have less
1877memory overhead for allocation patterns typical of Python programs.
1878The allocator uses C's \cfunction{malloc()} function to get large
1879pools of memory and then fulfills smaller memory requests from these
1880pools.
Andrew M. Kuchlingef5d06b2002-07-22 19:21:06 +00001881
1882In 2.1 and 2.2, pymalloc was an experimental feature and wasn't
Andrew M. Kuchlingc71bb972003-03-21 17:23:07 +00001883enabled by default; you had to explicitly enable it when compiling
1884Python by providing the
Andrew M. Kuchlingef5d06b2002-07-22 19:21:06 +00001885\longprogramopt{with-pymalloc} option to the \program{configure}
1886script. In 2.3, pymalloc has had further enhancements and is now
1887enabled by default; you'll have to supply
1888\longprogramopt{without-pymalloc} to disable it.
1889
1890This change is transparent to code written in Python; however,
1891pymalloc may expose bugs in C extensions. Authors of C extension
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +00001892modules should test their code with pymalloc enabled,
1893because some incorrect code may cause core dumps at runtime.
1894
1895There's one particularly common error that causes problems. There are
1896a number of memory allocation functions in Python's C API that have
1897previously just been aliases for the C library's \cfunction{malloc()}
Andrew M. Kuchlingef5d06b2002-07-22 19:21:06 +00001898and \cfunction{free()}, meaning that if you accidentally called
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +00001899mismatched functions the error wouldn't be noticeable. When the
Andrew M. Kuchlingef5d06b2002-07-22 19:21:06 +00001900object allocator is enabled, these functions aren't aliases of
1901\cfunction{malloc()} and \cfunction{free()} any more, and calling the
1902wrong function to free memory may get you a core dump. For example,
1903if memory was allocated using \cfunction{PyObject_Malloc()}, it has to
1904be freed using \cfunction{PyObject_Free()}, not \cfunction{free()}. A
1905few modules included with Python fell afoul of this and had to be
1906fixed; doubtless there are more third-party modules that will have the
1907same problem.
1908
1909As part of this change, the confusing multiple interfaces for
1910allocating memory have been consolidated down into two API families.
1911Memory allocated with one family must not be manipulated with
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +00001912functions from the other family. There is one family for allocating
1913chunks of memory, and another family of functions specifically for
1914allocating Python objects.
Andrew M. Kuchlingef5d06b2002-07-22 19:21:06 +00001915
1916\begin{itemize}
1917 \item To allocate and free an undistinguished chunk of memory use
1918 the ``raw memory'' family: \cfunction{PyMem_Malloc()},
1919 \cfunction{PyMem_Realloc()}, and \cfunction{PyMem_Free()}.
1920
1921 \item The ``object memory'' family is the interface to the pymalloc
1922 facility described above and is biased towards a large number of
1923 ``small'' allocations: \cfunction{PyObject_Malloc},
1924 \cfunction{PyObject_Realloc}, and \cfunction{PyObject_Free}.
1925
1926 \item To allocate and free Python objects, use the ``object'' family
1927 \cfunction{PyObject_New()}, \cfunction{PyObject_NewVar()}, and
1928 \cfunction{PyObject_Del()}.
1929\end{itemize}
1930
1931Thanks to lots of work by Tim Peters, pymalloc in 2.3 also provides
1932debugging features to catch memory overwrites and doubled frees in
1933both extension modules and in the interpreter itself. To enable this
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +00001934support, compile a debugging version of the Python interpreter by
1935running \program{configure} with \longprogramopt{with-pydebug}.
Andrew M. Kuchlingef5d06b2002-07-22 19:21:06 +00001936
1937To aid extension writers, a header file \file{Misc/pymemcompat.h} is
1938distributed with the source to Python 2.3 that allows Python
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +00001939extensions to use the 2.3 interfaces to memory allocation while
1940compiling against any version of Python since 1.5.2. You would copy
1941the file from Python's source distribution and bundle it with the
1942source of your extension.
Andrew M. Kuchlingef5d06b2002-07-22 19:21:06 +00001943
1944\begin{seealso}
1945
1946\seeurl{http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/Objects/obmalloc.c}
1947{For the full details of the pymalloc implementation, see
1948the comments at the top of the file \file{Objects/obmalloc.c} in the
1949Python source code. The above link points to the file within the
1950SourceForge CVS browser.}
1951
1952\end{seealso}
1953
1954
Andrew M. Kuchling821013e2002-05-06 17:46:39 +00001955% ======================================================================
1956\section{Build and C API Changes}
1957
Andrew M. Kuchling3c305d92002-07-22 18:50:11 +00001958Changes to Python's build process and to the C API include:
Andrew M. Kuchling821013e2002-05-06 17:46:39 +00001959
1960\begin{itemize}
1961
Andrew M. Kuchlingef5d06b2002-07-22 19:21:06 +00001962\item The C-level interface to the garbage collector has been changed,
1963to make it easier to write extension types that support garbage
1964collection, and to make it easier to debug misuses of the functions.
1965Various functions have slightly different semantics, so a bunch of
1966functions had to be renamed. Extensions that use the old API will
1967still compile but will \emph{not} participate in garbage collection,
1968so updating them for 2.3 should be considered fairly high priority.
1969
1970To upgrade an extension module to the new API, perform the following
1971steps:
1972
1973\begin{itemize}
1974
1975\item Rename \cfunction{Py_TPFLAGS_GC} to \cfunction{PyTPFLAGS_HAVE_GC}.
1976
1977\item Use \cfunction{PyObject_GC_New} or \cfunction{PyObject_GC_NewVar} to
1978allocate objects, and \cfunction{PyObject_GC_Del} to deallocate them.
1979
1980\item Rename \cfunction{PyObject_GC_Init} to \cfunction{PyObject_GC_Track} and
1981\cfunction{PyObject_GC_Fini} to \cfunction{PyObject_GC_UnTrack}.
1982
1983\item Remove \cfunction{PyGC_HEAD_SIZE} from object size calculations.
1984
1985\item Remove calls to \cfunction{PyObject_AS_GC} and \cfunction{PyObject_FROM_GC}.
1986
1987\end{itemize}
1988
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +00001989\item The cycle detection implementation used by the garbage collection
1990has proven to be stable, so it's now being made mandatory; you can no
1991longer compile Python without it, and the
1992\longprogramopt{with-cycle-gc} switch to \program{configure} has been removed.
Andrew M. Kuchling974ab9d2002-12-31 01:20:30 +00001993
Andrew M. Kuchling517109b2002-05-07 21:01:16 +00001994\item Python can now optionally be built as a shared library
1995(\file{libpython2.3.so}) by supplying \longprogramopt{enable-shared}
Fred Drake5c4cf152002-11-13 14:59:06 +00001996when running Python's \program{configure} script. (Contributed by Ondrej
Andrew M. Kuchlingfad2f592002-05-10 21:00:05 +00001997Palkovsky.)
Andrew M. Kuchlingf4dd65d2002-04-01 19:28:09 +00001998
Michael W. Hudsondd32a912002-08-15 14:59:02 +00001999\item The \csimplemacro{DL_EXPORT} and \csimplemacro{DL_IMPORT} macros
2000are now deprecated. Initialization functions for Python extension
2001modules should now be declared using the new macro
Andrew M. Kuchling3c305d92002-07-22 18:50:11 +00002002\csimplemacro{PyMODINIT_FUNC}, while the Python core will generally
2003use the \csimplemacro{PyAPI_FUNC} and \csimplemacro{PyAPI_DATA}
2004macros.
Neal Norwitzbba23a82002-07-22 13:18:59 +00002005
Fred Drake5c4cf152002-11-13 14:59:06 +00002006\item The interpreter can be compiled without any docstrings for
Andrew M. Kuchlinge995d162002-07-11 20:09:50 +00002007the built-in functions and modules by supplying
Fred Drake5c4cf152002-11-13 14:59:06 +00002008\longprogramopt{without-doc-strings} to the \program{configure} script.
Andrew M. Kuchlinge995d162002-07-11 20:09:50 +00002009This makes the Python executable about 10\% smaller, but will also
2010mean that you can't get help for Python's built-ins. (Contributed by
2011Gustavo Niemeyer.)
2012
Andrew M. Kuchling517109b2002-05-07 21:01:16 +00002013\item The \cfunction{PyArg_NoArgs()} macro is now deprecated, and code
Andrew M. Kuchling7845e7c2002-07-11 19:27:46 +00002014that uses it should be changed. For Python 2.2 and later, the method
2015definition table can specify the
Fred Drake5c4cf152002-11-13 14:59:06 +00002016\constant{METH_NOARGS} flag, signalling that there are no arguments, and
Andrew M. Kuchling7845e7c2002-07-11 19:27:46 +00002017the argument checking can then be removed. If compatibility with
2018pre-2.2 versions of Python is important, the code could use
Fred Drakeaac8c582003-01-17 22:50:10 +00002019\code{PyArg_ParseTuple(\var{args}, "")} instead, but this will be slower
Andrew M. Kuchling7845e7c2002-07-11 19:27:46 +00002020than using \constant{METH_NOARGS}.
Andrew M. Kuchling03594bb2002-03-27 02:29:48 +00002021
Andrew M. Kuchling517109b2002-05-07 21:01:16 +00002022\item A new function, \cfunction{PyObject_DelItemString(\var{mapping},
2023char *\var{key})} was added
Fred Drake5c4cf152002-11-13 14:59:06 +00002024as shorthand for
Andrew M. Kuchling517109b2002-05-07 21:01:16 +00002025\code{PyObject_DelItem(\var{mapping}, PyString_New(\var{key})}.
Andrew M. Kuchling03594bb2002-03-27 02:29:48 +00002026
Andrew M. Kuchling366c10c2002-11-14 23:07:57 +00002027\item The \method{xreadlines()} method of file objects, introduced in
2028Python 2.1, is no longer necessary because files now behave as their
2029own iterator. \method{xreadlines()} was originally introduced as a
2030faster way to loop over all the lines in a file, but now you can
2031simply write \code{for line in file_obj}.
2032
Andrew M. Kuchling517109b2002-05-07 21:01:16 +00002033\item File objects now manage their internal string buffer
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +00002034differently, increasing it exponentially when needed. This results in
2035the benchmark tests in \file{Lib/test/test_bufio.py} speeding up
2036considerably (from 57 seconds to 1.7 seconds, according to one
2037measurement).
Andrew M. Kuchling517109b2002-05-07 21:01:16 +00002038
Andrew M. Kuchling72b58e02002-05-29 17:30:34 +00002039\item It's now possible to define class and static methods for a C
2040extension type by setting either the \constant{METH_CLASS} or
2041\constant{METH_STATIC} flags in a method's \ctype{PyMethodDef}
2042structure.
Andrew M. Kuchling45afd542002-04-02 14:25:25 +00002043
Andrew M. Kuchling346386f2002-07-12 20:24:42 +00002044\item Python now includes a copy of the Expat XML parser's source code,
2045removing any dependence on a system version or local installation of
Fred Drake5c4cf152002-11-13 14:59:06 +00002046Expat.
Andrew M. Kuchling346386f2002-07-12 20:24:42 +00002047
Michael W. Hudson3e245d82003-02-11 14:19:56 +00002048\item If you dynamically allocate type objects in your extension, you
Neal Norwitzada859c2003-02-11 14:30:39 +00002049should be aware of a change in the rules relating to the
Michael W. Hudson3e245d82003-02-11 14:19:56 +00002050\member{__module__} and \member{__name__} attributes. In summary,
2051you will want to ensure the type's dictionary contains a
2052\code{'__module__'} key; making the module name the part of the type
2053name leading up to the final period will no longer have the desired
2054effect. For more detail, read the API reference documentation or the
2055source.
2056
Andrew M. Kuchling821013e2002-05-06 17:46:39 +00002057\end{itemize}
2058
Andrew M. Kuchling366c10c2002-11-14 23:07:57 +00002059
Andrew M. Kuchling974ab9d2002-12-31 01:20:30 +00002060%======================================================================
Andrew M. Kuchling821013e2002-05-06 17:46:39 +00002061\subsection{Port-Specific Changes}
2062
Andrew M. Kuchling187b1d82002-05-29 19:20:57 +00002063Support for a port to IBM's OS/2 using the EMX runtime environment was
2064merged into the main Python source tree. EMX is a POSIX emulation
2065layer over the OS/2 system APIs. The Python port for EMX tries to
2066support all the POSIX-like capability exposed by the EMX runtime, and
2067mostly succeeds; \function{fork()} and \function{fcntl()} are
2068restricted by the limitations of the underlying emulation layer. The
2069standard OS/2 port, which uses IBM's Visual Age compiler, also gained
2070support for case-sensitive import semantics as part of the integration
2071of the EMX port into CVS. (Contributed by Andrew MacIntyre.)
Andrew M. Kuchling03594bb2002-03-27 02:29:48 +00002072
Andrew M. Kuchling72b58e02002-05-29 17:30:34 +00002073On MacOS, most toolbox modules have been weaklinked to improve
2074backward compatibility. This means that modules will no longer fail
2075to load if a single routine is missing on the curent OS version.
Andrew M. Kuchling187b1d82002-05-29 19:20:57 +00002076Instead calling the missing routine will raise an exception.
2077(Contributed by Jack Jansen.)
Andrew M. Kuchling03594bb2002-03-27 02:29:48 +00002078
Andrew M. Kuchling187b1d82002-05-29 19:20:57 +00002079The RPM spec files, found in the \file{Misc/RPM/} directory in the
2080Python source distribution, were updated for 2.3. (Contributed by
2081Sean Reifschneider.)
Fred Drake03e10312002-03-26 19:17:43 +00002082
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +00002083Other new platforms now supported by Python include AtheOS
Fred Drake693aea22003-02-07 14:52:18 +00002084(\url{http://www.atheos.cx/}), GNU/Hurd, and OpenVMS.
Andrew M. Kuchling20e5abc2002-07-11 20:50:34 +00002085
Fred Drake03e10312002-03-26 19:17:43 +00002086
2087%======================================================================
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +00002088\section{Other Changes and Fixes \label{section-other}}
Andrew M. Kuchling517109b2002-05-07 21:01:16 +00002089
Andrew M. Kuchling7a82b8c2002-11-04 20:17:24 +00002090As usual, there were a bunch of other improvements and bugfixes
2091scattered throughout the source tree. A search through the CVS change
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +00002092logs finds there were 121 patches applied and 103 bugs fixed between
Andrew M. Kuchling7a82b8c2002-11-04 20:17:24 +00002093Python 2.2 and 2.3. Both figures are likely to be underestimates.
2094
2095Some of the more notable changes are:
Andrew M. Kuchling517109b2002-05-07 21:01:16 +00002096
2097\begin{itemize}
2098
Fred Drake54fe3fd2002-11-26 22:07:35 +00002099\item The \file{regrtest.py} script now provides a way to allow ``all
2100resources except \var{foo}.'' A resource name passed to the
2101\programopt{-u} option can now be prefixed with a hyphen
2102(\character{-}) to mean ``remove this resource.'' For example, the
2103option `\code{\programopt{-u}all,-bsddb}' could be used to enable the
2104use of all resources except \code{bsddb}.
2105
Andrew M. Kuchling517109b2002-05-07 21:01:16 +00002106\item The tools used to build the documentation now work under Cygwin
2107as well as \UNIX.
2108
Michael W. Hudsondd32a912002-08-15 14:59:02 +00002109\item The \code{SET_LINENO} opcode has been removed. Back in the
2110mists of time, this opcode was needed to produce line numbers in
2111tracebacks and support trace functions (for, e.g., \module{pdb}).
2112Since Python 1.5, the line numbers in tracebacks have been computed
2113using a different mechanism that works with ``python -O''. For Python
21142.3 Michael Hudson implemented a similar scheme to determine when to
2115call the trace function, removing the need for \code{SET_LINENO}
2116entirely.
2117
Andrew M. Kuchling7a82b8c2002-11-04 20:17:24 +00002118It would be difficult to detect any resulting difference from Python
2119code, apart from a slight speed up when Python is run without
Michael W. Hudsondd32a912002-08-15 14:59:02 +00002120\programopt{-O}.
2121
2122C extensions that access the \member{f_lineno} field of frame objects
2123should instead call \code{PyCode_Addr2Line(f->f_code, f->f_lasti)}.
2124This will have the added effect of making the code work as desired
2125under ``python -O'' in earlier versions of Python.
2126
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +00002127A nifty new feature is that trace functions can now assign to the
2128\member{f_lineno} attribute of frame objects, changing the line that
2129will be executed next. A \samp{jump} command has been added to the
2130\module{pdb} debugger taking advantage of this new feature.
2131(Implemented by Richie Hindle.)
Andrew M. Kuchling974ab9d2002-12-31 01:20:30 +00002132
Andrew M. Kuchling517109b2002-05-07 21:01:16 +00002133\end{itemize}
2134
Andrew M. Kuchling187b1d82002-05-29 19:20:57 +00002135
Andrew M. Kuchling517109b2002-05-07 21:01:16 +00002136%======================================================================
Andrew M. Kuchling950725f2002-08-06 01:40:48 +00002137\section{Porting to Python 2.3}
2138
Andrew M. Kuchlingf15fb292002-12-31 18:34:54 +00002139This section lists previously described changes that may require
2140changes to your code:
Andrew M. Kuchling8a61f492002-11-13 13:24:41 +00002141
2142\begin{itemize}
2143
2144\item \keyword{yield} is now always a keyword; if it's used as a
2145variable name in your code, a different name must be chosen.
2146
Andrew M. Kuchling8a61f492002-11-13 13:24:41 +00002147\item For strings \var{X} and \var{Y}, \code{\var{X} in \var{Y}} now works
2148if \var{X} is more than one character long.
2149
Andrew M. Kuchling495172c2002-11-20 13:50:15 +00002150\item The \function{int()} type constructor will now return a long
2151integer instead of raising an \exception{OverflowError} when a string
2152or floating-point number is too large to fit into an integer.
2153
Andrew M. Kuchlingacddabc2003-02-18 00:43:24 +00002154\item If you have Unicode strings that contain 8-bit characters, you
2155must declare the file's encoding (UTF-8, Latin-1, or whatever) by
2156adding a comment to the top of the file. See
2157section~\ref{section-encodings} for more information.
2158
Andrew M. Kuchlingb492fa92002-11-27 19:11:10 +00002159\item Calling Tcl methods through \module{_tkinter} no longer
2160returns only strings. Instead, if Tcl returns other objects those
2161objects are converted to their Python equivalent, if one exists, or
2162wrapped with a \class{_tkinter.Tcl_Obj} object if no Python equivalent
2163exists.
2164
Andrew M. Kuchling80fd7852003-02-06 15:14:04 +00002165\item Large octal and hex literals such as
Andrew M. Kuchling72df65a2003-02-10 15:08:16 +00002166\code{0xffffffff} now trigger a \exception{FutureWarning}. Currently
Andrew M. Kuchling80fd7852003-02-06 15:14:04 +00002167they're stored as 32-bit numbers and result in a negative value, but
Andrew M. Kuchling72df65a2003-02-10 15:08:16 +00002168in Python 2.4 they'll become positive long integers.
2169
2170There are a few ways to fix this warning. If you really need a
2171positive number, just add an \samp{L} to the end of the literal. If
2172you're trying to get a 32-bit integer with low bits set and have
2173previously used an expression such as \code{~(1 << 31)}, it's probably
2174clearest to start with all bits set and clear the desired upper bits.
2175For example, to clear just the top bit (bit 31), you could write
2176\code{0xffffffffL {\&}{\textasciitilde}(1L<<31)}.
Andrew M. Kuchling80fd7852003-02-06 15:14:04 +00002177
Andrew M. Kuchling495172c2002-11-20 13:50:15 +00002178\item You can no longer disable assertions by assigning to \code{__debug__}.
2179
Andrew M. Kuchling8a61f492002-11-13 13:24:41 +00002180\item The Distutils \function{setup()} function has gained various new
Fred Drake5c4cf152002-11-13 14:59:06 +00002181keyword arguments such as \var{depends}. Old versions of the
Andrew M. Kuchling8a61f492002-11-13 13:24:41 +00002182Distutils will abort if passed unknown keywords. The fix is to check
2183for the presence of the new \function{get_distutil_options()} function
2184in your \file{setup.py} if you want to only support the new keywords
2185with a version of the Distutils that supports them:
2186
2187\begin{verbatim}
2188from distutils import core
2189
2190kw = {'sources': 'foo.c', ...}
2191if hasattr(core, 'get_distutil_options'):
2192 kw['depends'] = ['foo.h']
Fred Drake5c4cf152002-11-13 14:59:06 +00002193ext = Extension(**kw)
Andrew M. Kuchling8a61f492002-11-13 13:24:41 +00002194\end{verbatim}
2195
Andrew M. Kuchling495172c2002-11-20 13:50:15 +00002196\item Using \code{None} as a variable name will now result in a
2197\exception{SyntaxWarning} warning.
2198
2199\item Names of extension types defined by the modules included with
2200Python now contain the module and a \character{.} in front of the type
2201name.
2202
Andrew M. Kuchling8a61f492002-11-13 13:24:41 +00002203\end{itemize}
Andrew M. Kuchling950725f2002-08-06 01:40:48 +00002204
2205
2206%======================================================================
Fred Drake03e10312002-03-26 19:17:43 +00002207\section{Acknowledgements \label{acks}}
2208
Andrew M. Kuchling03594bb2002-03-27 02:29:48 +00002209The author would like to thank the following people for offering
2210suggestions, corrections and assistance with various drafts of this
Andrew M. Kuchlingb9ba4e62003-02-03 15:16:15 +00002211article: Simon Brunning, Michael Chermside, Andrew Dalke, Scott David
2212Daniels, Fred~L. Drake, Jr., Kelly Gerber, Raymond Hettinger, Michael
Andrew M. Kuchlingd87eeb92003-02-18 00:56:56 +00002213Hudson, Chris Lambert, Detlef Lannert, Martin von L\"owis, Andrew MacIntyre, Lalo
Andrew M. Kuchlingb9ba4e62003-02-03 15:16:15 +00002214Martins, Gustavo Niemeyer, Neal Norwitz, Hans Nowak, Chris Reedy,
Andrew M. Kuchlingd87eeb92003-02-18 00:56:56 +00002215Vinay Sajip, Neil Schemenauer, Roman Suzi, Jason Tishler, Just van~Rossum.
Fred Drake03e10312002-03-26 19:17:43 +00002216
2217\end{document}