Fred Drake | ed0fa3d | 2003-07-30 19:14:09 +0000 | [diff] [blame] | 1 | \documentclass{howto} |
| 2 | \usepackage{distutils} |
| 3 | % $Id$ |
| 4 | |
| 5 | \title{What's New in Python 2.4} |
| 6 | \release{0.0} |
| 7 | \author{A.M.\ Kuchling} |
Fred Drake | b914ef0 | 2004-01-02 06:57:50 +0000 | [diff] [blame] | 8 | \authoraddress{ |
| 9 | \strong{Python Software Foundation}\\ |
| 10 | Email: \email{amk@amk.ca} |
| 11 | } |
Fred Drake | ed0fa3d | 2003-07-30 19:14:09 +0000 | [diff] [blame] | 12 | |
| 13 | \begin{document} |
| 14 | \maketitle |
| 15 | \tableofcontents |
| 16 | |
| 17 | This article explains the new features in Python 2.4. No release date |
Raymond Hettinger | d446230 | 2003-11-26 17:52:45 +0000 | [diff] [blame] | 18 | for Python 2.4 has been set; expect that this will happen mid-2004. |
Fred Drake | ed0fa3d | 2003-07-30 19:14:09 +0000 | [diff] [blame] | 19 | |
| 20 | While Python 2.3 was primarily a library development release, Python |
| 21 | 2.4 may extend the core language and interpreter in |
| 22 | as-yet-undetermined ways. |
| 23 | |
| 24 | This article doesn't attempt to provide a complete specification of |
| 25 | the new features, but instead provides a convenient overview. For |
Andrew M. Kuchling | 35f2b05 | 2003-12-18 13:28:13 +0000 | [diff] [blame] | 26 | full details, you should refer to the documentation for Python 2.4, |
| 27 | such as the \citetitle[../lib/lib.html]{Python Library Reference} and |
| 28 | the \citetitle[../ref/ref.html]{Python Reference Manual}. |
Fred Drake | ed0fa3d | 2003-07-30 19:14:09 +0000 | [diff] [blame] | 29 | If you want to understand the complete implementation and design |
| 30 | rationale, refer to the PEP for a particular new feature. |
| 31 | |
Andrew M. Kuchling | 35f2b05 | 2003-12-18 13:28:13 +0000 | [diff] [blame] | 32 | |
Raymond Hettinger | 7e0282f | 2003-11-24 07:14:54 +0000 | [diff] [blame] | 33 | %====================================================================== |
| 34 | \section{PEP 218: Built-In Set Objects} |
| 35 | |
| 36 | Two new built-in types, \function{set(iterable)} and |
| 37 | \function{frozenset(iterable)} provide high speed data types for |
| 38 | membership testing, for eliminating duplicates from sequences, and |
| 39 | for mathematical operations like unions, intersections, differences, |
| 40 | and symmetric differences. |
| 41 | |
| 42 | \begin{verbatim} |
| 43 | >>> a = set('abracadabra') # form a set from a string |
| 44 | >>> 'z' in a # fast membership testing |
| 45 | False |
| 46 | >>> a # unique letters in a |
| 47 | set(['a', 'r', 'b', 'c', 'd']) |
| 48 | >>> ''.join(a) # convert back into a string |
| 49 | 'arbcd' |
Raymond Hettinger | d446230 | 2003-11-26 17:52:45 +0000 | [diff] [blame] | 50 | |
Raymond Hettinger | 7e0282f | 2003-11-24 07:14:54 +0000 | [diff] [blame] | 51 | >>> b = set('alacazam') # form a second set |
| 52 | >>> a - b # letters in a but not in b |
| 53 | set(['r', 'd', 'b']) |
| 54 | >>> a | b # letters in either a or b |
| 55 | set(['a', 'c', 'r', 'd', 'b', 'm', 'z', 'l']) |
| 56 | >>> a & b # letters in both a and b |
| 57 | set(['a', 'c']) |
| 58 | >>> a ^ b # letters in a or b but not both |
| 59 | set(['r', 'd', 'b', 'm', 'z', 'l']) |
Raymond Hettinger | d446230 | 2003-11-26 17:52:45 +0000 | [diff] [blame] | 60 | |
Raymond Hettinger | 7e0282f | 2003-11-24 07:14:54 +0000 | [diff] [blame] | 61 | >>> a.add('z') # add a new element |
| 62 | >>> a.update('wxy') # add multiple new elements |
| 63 | >>> a |
| 64 | set(['a', 'c', 'b', 'd', 'r', 'w', 'y', 'x', 'z']) |
| 65 | >>> a.remove('x') # take one element out |
| 66 | >>> a |
| 67 | set(['a', 'c', 'b', 'd', 'r', 'w', 'y', 'z']) |
| 68 | \end{verbatim} |
| 69 | |
| 70 | The type \function{frozenset()} is an immutable version of \function{set()}. |
| 71 | Since it is immutable and hashable, it may be used as a dictionary key or |
| 72 | as a member of another set. Accordingly, it does not have methods |
| 73 | like \method{add()} and \method{remove()} which could alter its contents. |
| 74 | |
Andrew M. Kuchling | 35f2b05 | 2003-12-18 13:28:13 +0000 | [diff] [blame] | 75 | % XXX what happens to the sets module? |
Raymond Hettinger | ed54d91 | 2003-12-31 01:59:18 +0000 | [diff] [blame] | 76 | % The current thinking is that the sets module will be left alone. |
| 77 | % That way, existing code will continue to run without alteration. |
| 78 | % Also, the module provides an autoconversion feature not supported by set() |
| 79 | % and frozenset(). |
Andrew M. Kuchling | 35f2b05 | 2003-12-18 13:28:13 +0000 | [diff] [blame] | 80 | |
Raymond Hettinger | 7e0282f | 2003-11-24 07:14:54 +0000 | [diff] [blame] | 81 | \begin{seealso} |
| 82 | \seepep{218}{Adding a Built-In Set Object Type}{Originally proposed by |
| 83 | Greg Wilson and ultimately implemented by Raymond Hettinger.} |
| 84 | \end{seealso} |
Fred Drake | ed0fa3d | 2003-07-30 19:14:09 +0000 | [diff] [blame] | 85 | |
| 86 | %====================================================================== |
Andrew M. Kuchling | 35f2b05 | 2003-12-18 13:28:13 +0000 | [diff] [blame] | 87 | \section{PEP 237: Unifying Long Integers and Integers} |
| 88 | |
| 89 | XXX write this. |
| 90 | |
| 91 | %====================================================================== |
Andrew M. Kuchling | 1a42025 | 2003-11-08 15:58:49 +0000 | [diff] [blame] | 92 | \section{PEP 322: Reverse Iteration} |
Fred Drake | ed0fa3d | 2003-07-30 19:14:09 +0000 | [diff] [blame] | 93 | |
Andrew M. Kuchling | 1a42025 | 2003-11-08 15:58:49 +0000 | [diff] [blame] | 94 | A new built-in function, \function{reversed(seq)}, takes a sequence |
| 95 | and returns an iterator that returns the elements of the sequence |
| 96 | in reverse order. |
| 97 | |
| 98 | \begin{verbatim} |
Raymond Hettinger | bc3cba2 | 2003-11-12 16:39:30 +0000 | [diff] [blame] | 99 | >>> for i in reversed(xrange(1,4)): |
Andrew M. Kuchling | 1a42025 | 2003-11-08 15:58:49 +0000 | [diff] [blame] | 100 | ... print i |
| 101 | ... |
| 102 | 3 |
| 103 | 2 |
| 104 | 1 |
| 105 | \end{verbatim} |
| 106 | |
Raymond Hettinger | bc3cba2 | 2003-11-12 16:39:30 +0000 | [diff] [blame] | 107 | Compared to extended slicing, \code{range(1,4)[::-1]}, \function{reversed()} |
| 108 | is easier to read, runs faster, and uses substantially less memory. |
| 109 | |
Andrew M. Kuchling | 1a42025 | 2003-11-08 15:58:49 +0000 | [diff] [blame] | 110 | Note that \function{reversed()} only accepts sequences, not arbitrary |
Raymond Hettinger | bc3cba2 | 2003-11-12 16:39:30 +0000 | [diff] [blame] | 111 | iterators. If you want to reverse an iterator, first convert it to |
| 112 | a list with \function{list()}. |
Andrew M. Kuchling | 1a42025 | 2003-11-08 15:58:49 +0000 | [diff] [blame] | 113 | |
| 114 | \begin{verbatim} |
Andrew M. Kuchling | 44a31e1 | 2004-01-01 18:33:34 +0000 | [diff] [blame] | 115 | >>> input= open('/etc/passwd', 'r') |
| 116 | >>> for line in reversed(list(input)): |
Andrew M. Kuchling | 1a42025 | 2003-11-08 15:58:49 +0000 | [diff] [blame] | 117 | ... print line |
| 118 | ... |
| 119 | root:*:0:0:System Administrator:/var/root:/bin/tcsh |
| 120 | ... |
| 121 | \end{verbatim} |
Fred Drake | ed0fa3d | 2003-07-30 19:14:09 +0000 | [diff] [blame] | 122 | |
Andrew M. Kuchling | f7a6b67 | 2003-11-08 16:05:37 +0000 | [diff] [blame] | 123 | \begin{seealso} |
| 124 | \seepep{322}{Reverse Iteration}{Written and implemented by Raymond Hettinger.} |
| 125 | |
| 126 | \end{seealso} |
| 127 | |
Fred Drake | ed0fa3d | 2003-07-30 19:14:09 +0000 | [diff] [blame] | 128 | |
| 129 | %====================================================================== |
| 130 | \section{Other Language Changes} |
| 131 | |
| 132 | Here are all of the changes that Python 2.4 makes to the core Python |
| 133 | language. |
| 134 | |
| 135 | \begin{itemize} |
Raymond Hettinger | d446230 | 2003-11-26 17:52:45 +0000 | [diff] [blame] | 136 | |
Raymond Hettinger | 31017ae | 2004-03-04 08:25:44 +0000 | [diff] [blame] | 137 | \item The \method{dict.update()} method now accepts the same |
| 138 | argument forms as the \class{dict} constructor. This includes any |
| 139 | mapping, any iterable of key/value pairs, and/or keyword arguments. |
| 140 | |
Raymond Hettinger | d446230 | 2003-11-26 17:52:45 +0000 | [diff] [blame] | 141 | \item The string methods, \method{ljust()}, \method{rjust()}, and |
Andrew M. Kuchling | 6708756 | 2003-11-26 18:03:48 +0000 | [diff] [blame] | 142 | \method{center()} now take an optional argument for specifying a |
Raymond Hettinger | d446230 | 2003-11-26 17:52:45 +0000 | [diff] [blame] | 143 | fill character other than a space. |
| 144 | |
Andrew M. Kuchling | 35f2b05 | 2003-12-18 13:28:13 +0000 | [diff] [blame] | 145 | \item Strings also gained an \method{rsplit()} method that |
Raymond Hettinger | ed54d91 | 2003-12-31 01:59:18 +0000 | [diff] [blame] | 146 | works like the \method{split()} method but splits from the end of |
Andrew M. Kuchling | 44a31e1 | 2004-01-01 18:33:34 +0000 | [diff] [blame] | 147 | the string. |
Andrew M. Kuchling | 35f2b05 | 2003-12-18 13:28:13 +0000 | [diff] [blame] | 148 | |
| 149 | \begin{verbatim} |
Raymond Hettinger | 7a6d297 | 2004-02-13 19:00:07 +0000 | [diff] [blame] | 150 | >>> 'www.python.org'.split('.', 1) |
| 151 | ['www', 'python.org'] |
| 152 | 'www.python.org'.rsplit('.', 1) |
| 153 | ['www.python', 'org'] |
| 154 | \end{verbatim} |
Raymond Hettinger | 97ef8de | 2004-01-05 00:29:57 +0000 | [diff] [blame] | 155 | |
Andrew M. Kuchling | 2fb4d51 | 2003-10-21 12:31:16 +0000 | [diff] [blame] | 156 | \item The \method{sort()} method of lists gained three keyword |
| 157 | arguments, \var{cmp}, \var{key}, and \var{reverse}. These arguments |
| 158 | make some common usages of \method{sort()} simpler. All are optional. |
| 159 | |
| 160 | \var{cmp} is the same as the previous single argument to |
| 161 | \method{sort()}; if provided, the value should be a comparison |
| 162 | function that takes two arguments and returns -1, 0, or +1 depending |
| 163 | on how the arguments compare. |
| 164 | |
| 165 | \var{key} should be a single-argument function that takes a list |
| 166 | element and returns a comparison key for the element. The list is |
Raymond Hettinger | 607c00f | 2003-11-12 16:27:50 +0000 | [diff] [blame] | 167 | then sorted using the comparison keys. The following example sorts a |
| 168 | list case-insensitively: |
Andrew M. Kuchling | 2fb4d51 | 2003-10-21 12:31:16 +0000 | [diff] [blame] | 169 | |
| 170 | \begin{verbatim} |
| 171 | >>> L = ['A', 'b', 'c', 'D'] |
| 172 | >>> L.sort() # Case-sensitive sort |
| 173 | >>> L |
| 174 | ['A', 'D', 'b', 'c'] |
| 175 | >>> L.sort(key=lambda x: x.lower()) |
| 176 | >>> L |
| 177 | ['A', 'b', 'c', 'D'] |
| 178 | >>> L.sort(cmp=lambda x,y: cmp(x.lower(), y.lower())) |
| 179 | >>> L |
| 180 | ['A', 'b', 'c', 'D'] |
| 181 | \end{verbatim} |
| 182 | |
| 183 | The last example, which uses the \var{cmp} parameter, is the old way |
Raymond Hettinger | ed54d91 | 2003-12-31 01:59:18 +0000 | [diff] [blame] | 184 | to perform a case-insensitive sort. It works but is slower than |
Andrew M. Kuchling | 2fb4d51 | 2003-10-21 12:31:16 +0000 | [diff] [blame] | 185 | using a \var{key} parameter. Using \var{key} results in calling the |
| 186 | \method{lower()} method once for each element in the list while using |
| 187 | \var{cmp} will call the method twice for each comparison. |
| 188 | |
Andrew M. Kuchling | 981a918 | 2003-11-13 21:33:26 +0000 | [diff] [blame] | 189 | For simple key functions and comparison functions, it is often |
| 190 | possible to avoid a \keyword{lambda} expression by using an unbound |
Raymond Hettinger | 607c00f | 2003-11-12 16:27:50 +0000 | [diff] [blame] | 191 | method instead. For example, the above case-insensitive sort is best |
| 192 | coded as: |
| 193 | |
| 194 | \begin{verbatim} |
| 195 | >>> L.sort(key=str.lower) |
| 196 | >>> L |
| 197 | ['A', 'b', 'c', 'D'] |
| 198 | \end{verbatim} |
| 199 | |
Andrew M. Kuchling | 2fb4d51 | 2003-10-21 12:31:16 +0000 | [diff] [blame] | 200 | The \var{reverse} parameter should have a Boolean value. If the value is |
| 201 | \constant{True}, the list will be sorted into reverse order. Instead |
Raymond Hettinger | 607c00f | 2003-11-12 16:27:50 +0000 | [diff] [blame] | 202 | of \code{L.sort(lambda x,y: cmp(y.score, x.score))}, you can now write: |
| 203 | \code{L.sort(key = lambda x: x.score, reverse=True)}. |
Fred Drake | ed0fa3d | 2003-07-30 19:14:09 +0000 | [diff] [blame] | 204 | |
Andrew M. Kuchling | 981a918 | 2003-11-13 21:33:26 +0000 | [diff] [blame] | 205 | The results of sorting are now guaranteed to be stable. This means |
| 206 | that two entries with equal keys will be returned in the same order as |
| 207 | they were input. For example, you can sort a list of people by name, |
| 208 | and then sort the list by age, resulting in a list sorted by age where |
| 209 | people with the same age are in name-sorted order. |
Raymond Hettinger | 607c00f | 2003-11-12 16:27:50 +0000 | [diff] [blame] | 210 | |
Andrew M. Kuchling | 35f2b05 | 2003-12-18 13:28:13 +0000 | [diff] [blame] | 211 | \item There is a new built-in function \function{sorted(iterable)} that works |
Raymond Hettinger | 64958a1 | 2003-12-17 20:43:33 +0000 | [diff] [blame] | 212 | like the in-place \method{list.sort()} method but has been made suitable |
| 213 | for use in expressions. The differences are: |
Raymond Hettinger | 607c00f | 2003-11-12 16:27:50 +0000 | [diff] [blame] | 214 | \begin{itemize} |
Raymond Hettinger | 7d1dd04 | 2003-11-12 16:42:10 +0000 | [diff] [blame] | 215 | \item the input may be any iterable; |
| 216 | \item a newly formed copy is sorted, leaving the original intact; and |
Raymond Hettinger | 607c00f | 2003-11-12 16:27:50 +0000 | [diff] [blame] | 217 | \item the expression returns the new sorted copy |
| 218 | \end{itemize} |
Andrew M. Kuchling | 1a42025 | 2003-11-08 15:58:49 +0000 | [diff] [blame] | 219 | |
| 220 | \begin{verbatim} |
| 221 | >>> L = [9,7,8,3,2,4,1,6,5] |
Raymond Hettinger | 64958a1 | 2003-12-17 20:43:33 +0000 | [diff] [blame] | 222 | >>> [10+i for i in sorted(L)] # usable in a list comprehension |
Raymond Hettinger | 607c00f | 2003-11-12 16:27:50 +0000 | [diff] [blame] | 223 | [11, 12, 13, 14, 15, 16, 17, 18, 19] |
| 224 | >>> L = [9,7,8,3,2,4,1,6,5] # original is left unchanged |
| 225 | [9,7,8,3,2,4,1,6,5] |
Raymond Hettinger | d446230 | 2003-11-26 17:52:45 +0000 | [diff] [blame] | 226 | |
Raymond Hettinger | 64958a1 | 2003-12-17 20:43:33 +0000 | [diff] [blame] | 227 | >>> sorted('Monte Python') # any iterable may be an input |
Raymond Hettinger | 607c00f | 2003-11-12 16:27:50 +0000 | [diff] [blame] | 228 | [' ', 'M', 'P', 'e', 'h', 'n', 'n', 'o', 'o', 't', 't', 'y'] |
Raymond Hettinger | d446230 | 2003-11-26 17:52:45 +0000 | [diff] [blame] | 229 | |
| 230 | >>> # List the contents of a dict sorted by key values |
Raymond Hettinger | 607c00f | 2003-11-12 16:27:50 +0000 | [diff] [blame] | 231 | >>> colormap = dict(red=1, blue=2, green=3, black=4, yellow=5) |
Raymond Hettinger | 64958a1 | 2003-12-17 20:43:33 +0000 | [diff] [blame] | 232 | >>> for k, v in sorted(colormap.iteritems()): |
Raymond Hettinger | 607c00f | 2003-11-12 16:27:50 +0000 | [diff] [blame] | 233 | ... print k, v |
| 234 | ... |
| 235 | black 4 |
| 236 | blue 2 |
| 237 | green 3 |
| 238 | red 1 |
| 239 | yellow 5 |
| 240 | |
Andrew M. Kuchling | 1a42025 | 2003-11-08 15:58:49 +0000 | [diff] [blame] | 241 | \end{verbatim} |
| 242 | |
Raymond Hettinger | 607c00f | 2003-11-12 16:27:50 +0000 | [diff] [blame] | 243 | \item The \function{zip()} built-in function and \function{itertools.izip()} |
Andrew M. Kuchling | 6708756 | 2003-11-26 18:03:48 +0000 | [diff] [blame] | 244 | now return an empty list instead of raising a \exception{TypeError} |
Andrew M. Kuchling | 44a31e1 | 2004-01-01 18:33:34 +0000 | [diff] [blame] | 245 | exception if called with no arguments. This makes them more |
Raymond Hettinger | 607c00f | 2003-11-12 16:27:50 +0000 | [diff] [blame] | 246 | suitable for use with variable length argument lists: |
| 247 | |
| 248 | \begin{verbatim} |
| 249 | >>> def transpose(array): |
| 250 | ... return zip(*array) |
| 251 | ... |
| 252 | >>> transpose([(1,2,3), (4,5,6)]) |
| 253 | [(1, 4), (2, 5), (3, 6)] |
| 254 | >>> transpose([]) |
| 255 | [] |
| 256 | \end{verbatim} |
Andrew M. Kuchling | 6aedcfc | 2003-10-21 12:48:23 +0000 | [diff] [blame] | 257 | |
Fred Drake | ed0fa3d | 2003-07-30 19:14:09 +0000 | [diff] [blame] | 258 | \end{itemize} |
| 259 | |
| 260 | |
| 261 | %====================================================================== |
| 262 | \subsection{Optimizations} |
| 263 | |
| 264 | \begin{itemize} |
| 265 | |
Raymond Hettinger | b7d05db | 2004-03-08 07:25:05 +0000 | [diff] [blame] | 266 | \item The inner loops for \class{list} and \class{tuple} slicing |
Raymond Hettinger | ade08ea | 2004-03-18 09:48:12 +0000 | [diff] [blame^] | 267 | were optimized and now run about one-third faster. The inner |
| 268 | loops were also optimized for \class{dict} with performance |
| 269 | boosts to \method{keys()}, \method{values()}, \method{items()}, |
| 270 | \method{\iterkeys()}, \method{\itervalues()}, and \method{iteritems()}. |
Raymond Hettinger | b7d05db | 2004-03-08 07:25:05 +0000 | [diff] [blame] | 271 | |
Raymond Hettinger | 7a6d297 | 2004-02-13 19:00:07 +0000 | [diff] [blame] | 272 | \item The machinery for growing and shrinking lists was optimized |
Raymond Hettinger | ab517d2 | 2004-02-14 18:34:46 +0000 | [diff] [blame] | 273 | for speed and for space efficiency. Small lists (under eight elements) |
| 274 | never over-allocate by more than three elements. Large lists do not |
Raymond Hettinger | 7a6d297 | 2004-02-13 19:00:07 +0000 | [diff] [blame] | 275 | over-allocate by more than 1/8th. Appending and popping from lists |
| 276 | now runs faster due to more efficient code paths and less frequent |
| 277 | use of the underlying system realloc(). List comprehensions also |
| 278 | benefit. The amount of improvement varies between systems and shows |
| 279 | the greatest improvement on systems with poor realloc() implementations. |
Raymond Hettinger | 79b5cf1 | 2004-02-17 10:46:32 +0000 | [diff] [blame] | 280 | \method{list.extend()} was also optimized and no longer converts its |
| 281 | argument into a temporary list prior to extending the base list. |
Raymond Hettinger | 7a6d297 | 2004-02-13 19:00:07 +0000 | [diff] [blame] | 282 | |
Raymond Hettinger | 97ef8de | 2004-01-05 00:29:57 +0000 | [diff] [blame] | 283 | \item \function{list()}, \function{tuple()}, \function{map()}, |
| 284 | \function{filter()}, and \function{zip()} now run several times |
| 285 | faster with non-sequence arguments that supply a \method{__len__()} |
| 286 | method. Previously, the pre-sizing optimization only applied to |
| 287 | sequence arguments. |
| 288 | |
Raymond Hettinger | 23a0f4e | 2004-01-05 08:15:20 +0000 | [diff] [blame] | 289 | \item The methods \method{list.__getitem__()}, |
Raymond Hettinger | 97ef8de | 2004-01-05 00:29:57 +0000 | [diff] [blame] | 290 | \method{dict.__getitem__()}, and \method{dict.__contains__()} are |
| 291 | are now implemented as \class{method_descriptor} objects rather |
| 292 | than \class{wrapper_descriptor} objects. This form of optimized |
| 293 | access doubles their performance and makes them more suitable for |
Raymond Hettinger | 23a0f4e | 2004-01-05 08:15:20 +0000 | [diff] [blame] | 294 | use as arguments to functionals: |
| 295 | \samp{map(mydict.__getitem__, keylist)}. |
Fred Drake | ed0fa3d | 2003-07-30 19:14:09 +0000 | [diff] [blame] | 296 | |
Raymond Hettinger | dd80f76 | 2004-03-07 07:31:06 +0000 | [diff] [blame] | 297 | \item Added an newcode opcode, \code{LIST_APPEND}, that simplifies |
| 298 | the generated bytecode for list comprehensions and speeds them up |
| 299 | by about a third. |
| 300 | |
Fred Drake | ed0fa3d | 2003-07-30 19:14:09 +0000 | [diff] [blame] | 301 | \end{itemize} |
| 302 | |
| 303 | The net result of the 2.4 optimizations is that Python 2.4 runs the |
| 304 | pystone benchmark around XX\% faster than Python 2.3 and YY\% faster |
| 305 | than Python 2.2. |
| 306 | |
| 307 | |
| 308 | %====================================================================== |
| 309 | \section{New, Improved, and Deprecated Modules} |
| 310 | |
| 311 | As usual, Python's standard library received a number of enhancements and |
| 312 | bug fixes. Here's a partial list of the most notable changes, sorted |
| 313 | alphabetically by module name. Consult the |
| 314 | \file{Misc/NEWS} file in the source tree for a more |
| 315 | complete list of changes, or look through the CVS logs for all the |
| 316 | details. |
| 317 | |
| 318 | \begin{itemize} |
| 319 | |
Andrew M. Kuchling | 69f31eb | 2003-08-13 23:11:04 +0000 | [diff] [blame] | 320 | \item The \module{curses} modules now supports the ncurses extension |
| 321 | \function{use_default_colors()}. On platforms where the terminal |
| 322 | supports transparency, this makes it possible to use a transparent background. |
| 323 | (Contributed by J\"org Lehmann.) |
Andrew M. Kuchling | 6aedcfc | 2003-10-21 12:48:23 +0000 | [diff] [blame] | 324 | |
Raymond Hettinger | 0c41027 | 2004-01-05 10:13:35 +0000 | [diff] [blame] | 325 | \item The \module{bisect} module now has an underlying C implementation |
| 326 | for improved performance. |
| 327 | (Contributed by Dmitry Vasiliev.) |
| 328 | |
Andrew M. Kuchling | 5303a96 | 2004-01-18 15:55:51 +0000 | [diff] [blame] | 329 | \item The CJKCodecs collections of East Asian codecs, maintained |
| 330 | by Hye-Shik Chang, was integrated into 2.4. |
| 331 | The new encodings are: |
| 332 | |
| 333 | \begin{itemize} |
| 334 | \item Chinese (PRC): gb2312, gbk, gb18030, hz |
| 335 | \item Chinese (ROC): big5, cp950 |
| 336 | \item Japanese: cp932, shift-jis, shift-jisx0213, euc-jp, |
| 337 | euc-jisx0213, iso-2022-jp, iso-2022-jp-1, iso-2022-jp-2, |
| 338 | iso-2022-jp-3, iso-2022-jp-ext |
| 339 | \item Korean: cp949, euc-kr, johab, iso-2022-kr |
| 340 | \end{itemize} |
| 341 | |
Andrew M. Kuchling | fd0e494 | 2004-02-09 13:23:34 +0000 | [diff] [blame] | 342 | \item There is a new \module{collections} module for |
| 343 | various specialized collection datatypes. |
| 344 | Currently it contains just one type, \class{deque}, |
| 345 | a double-ended queue that supports efficiently adding and removing |
| 346 | elements from either end. |
Raymond Hettinger | 756b3f3 | 2004-01-29 06:37:52 +0000 | [diff] [blame] | 347 | |
| 348 | \begin{verbatim} |
| 349 | >>> from collections import deque |
| 350 | >>> d = deque('ghi') # make a new deque with three items |
| 351 | >>> d.append('j') # add a new entry to the right side |
| 352 | >>> d.appendleft('f') # add a new entry to the left side |
| 353 | >>> d # show the representation of the deque |
| 354 | deque(['f', 'g', 'h', 'i', 'j']) |
| 355 | >>> d.pop() # return and remove the rightmost item |
| 356 | 'j' |
| 357 | >>> d.popleft() # return and remove the leftmost item |
| 358 | 'f' |
| 359 | >>> list(d) # list the contents of the deque |
| 360 | ['g', 'h', 'i'] |
| 361 | >>> 'h' in d # search the deque |
| 362 | True |
| 363 | \end{verbatim} |
| 364 | |
Andrew M. Kuchling | fd0e494 | 2004-02-09 13:23:34 +0000 | [diff] [blame] | 365 | Several modules now take advantage of \class{collections.deque} for |
Raymond Hettinger | 756b3f3 | 2004-01-29 06:37:52 +0000 | [diff] [blame] | 366 | improved performance: \module{Queue}, \module{mutex}, \module{shlex} |
| 367 | \module{threading}, and \module{pydoc}. |
Andrew M. Kuchling | 5303a96 | 2004-01-18 15:55:51 +0000 | [diff] [blame] | 368 | |
Raymond Hettinger | 607c00f | 2003-11-12 16:27:50 +0000 | [diff] [blame] | 369 | \item The \module{heapq} module has been converted to C. The resulting |
Andrew M. Kuchling | fd0e494 | 2004-02-09 13:23:34 +0000 | [diff] [blame] | 370 | tenfold improvement in speed makes the module suitable for handling |
Raymond Hettinger | 607c00f | 2003-11-12 16:27:50 +0000 | [diff] [blame] | 371 | high volumes of data. |
Andrew M. Kuchling | 1a42025 | 2003-11-08 15:58:49 +0000 | [diff] [blame] | 372 | |
Andrew M. Kuchling | dff9dbd | 2003-11-20 22:22:19 +0000 | [diff] [blame] | 373 | \item The \module{imaplib} module now supports IMAP's THREAD command. |
| 374 | (Contributed by Yves Dionne.) |
| 375 | |
Andrew M. Kuchling | ad80955 | 2003-12-06 23:19:23 +0000 | [diff] [blame] | 376 | \item The \module{itertools} module gained a |
| 377 | \function{groupby(\var{iterable}\optional{, \var{func}})} function, |
| 378 | inspired by the GROUP BY clause from SQL. |
| 379 | \var{iterable} returns a succession of elements, and the optional |
| 380 | \var{func} is a function that takes an element and returns a key |
| 381 | value; if omitted, the key is simply the element itself. |
| 382 | \function{groupby()} then groups the elements into subsequences |
| 383 | which have matching values of the key, and returns a series of 2-tuples |
| 384 | containing the key value and an iterator over the subsequence. |
| 385 | |
| 386 | Here's an example. The \var{key} function simply returns whether a |
| 387 | number is even or odd, so the result of \function{groupby()} is to |
| 388 | return consecutive runs of odd or even numbers. |
| 389 | |
| 390 | \begin{verbatim} |
| 391 | >>> import itertools |
| 392 | >>> L = [2,4,6, 7,8,9,11, 12, 14] |
| 393 | >>> for key_val, it in itertools.groupby(L, lambda x: x % 2): |
| 394 | ... print key_val, list(it) |
| 395 | ... |
| 396 | 0 [2, 4, 6] |
| 397 | 1 [7] |
| 398 | 0 [8] |
| 399 | 1 [9, 11] |
| 400 | 0 [12, 14] |
| 401 | >>> |
| 402 | \end{verbatim} |
| 403 | |
Raymond Hettinger | feb78c9 | 2003-12-12 13:13:47 +0000 | [diff] [blame] | 404 | Like its SQL counterpart, \function{groupby()} is typically used with |
| 405 | sorted input. The logic for \function{groupby()} is similar to the |
| 406 | \UNIX{} \code{uniq} filter which makes it handy for eliminating, |
| 407 | counting, or identifying duplicate elements: |
| 408 | |
| 409 | \begin{verbatim} |
| 410 | >>> word = 'abracadabra' |
Raymond Hettinger | ed54d91 | 2003-12-31 01:59:18 +0000 | [diff] [blame] | 411 | >>> letters = sorted(word) # Turn string into a sorted list of letters |
Raymond Hettinger | 64958a1 | 2003-12-17 20:43:33 +0000 | [diff] [blame] | 412 | >>> letters |
Andrew M. Kuchling | 4612bc5 | 2003-12-16 20:59:37 +0000 | [diff] [blame] | 413 | ['a', 'a', 'a', 'a', 'a', 'b', 'b', 'c', 'd', 'r', 'r'] |
Raymond Hettinger | ed54d91 | 2003-12-31 01:59:18 +0000 | [diff] [blame] | 414 | >>> [k for k, g in groupby(letters)] # List unique letters |
Raymond Hettinger | feb78c9 | 2003-12-12 13:13:47 +0000 | [diff] [blame] | 415 | ['a', 'b', 'c', 'd', 'r'] |
Raymond Hettinger | ed54d91 | 2003-12-31 01:59:18 +0000 | [diff] [blame] | 416 | >>> [(k, len(list(g))) for k, g in groupby(letters)] # Count letter occurences |
Raymond Hettinger | feb78c9 | 2003-12-12 13:13:47 +0000 | [diff] [blame] | 417 | [('a', 5), ('b', 2), ('c', 1), ('d', 1), ('r', 2)] |
Raymond Hettinger | ed54d91 | 2003-12-31 01:59:18 +0000 | [diff] [blame] | 418 | >>> [k for k, g in groupby(letters) if len(list(g)) > 1] # List duplicated letters |
Raymond Hettinger | feb78c9 | 2003-12-12 13:13:47 +0000 | [diff] [blame] | 419 | ['a', 'b', 'r'] |
| 420 | \end{verbatim} |
| 421 | |
Raymond Hettinger | ed54d91 | 2003-12-31 01:59:18 +0000 | [diff] [blame] | 422 | \item \module{itertools} also gained a function named |
| 423 | \function{tee(\var{iterator}, \var{N})} that returns \var{N} independent |
| 424 | iterators that replicate \var{iterator}. If \var{N} is omitted, the |
| 425 | default is 2. |
Andrew M. Kuchling | 35f2b05 | 2003-12-18 13:28:13 +0000 | [diff] [blame] | 426 | |
| 427 | \begin{verbatim} |
| 428 | >>> L = [1,2,3] |
| 429 | >>> i1, i2 = itertools.tee(L) |
| 430 | >>> i1,i2 |
| 431 | (<itertools.tee object at 0x402c2080>, <itertools.tee object at 0x402c2090>) |
Raymond Hettinger | ed54d91 | 2003-12-31 01:59:18 +0000 | [diff] [blame] | 432 | >>> list(i1) # Run the first iterator to exhaustion |
Andrew M. Kuchling | 35f2b05 | 2003-12-18 13:28:13 +0000 | [diff] [blame] | 433 | [1, 2, 3] |
Raymond Hettinger | ed54d91 | 2003-12-31 01:59:18 +0000 | [diff] [blame] | 434 | >>> list(i2) # Run the second iterator to exhaustion |
Andrew M. Kuchling | 35f2b05 | 2003-12-18 13:28:13 +0000 | [diff] [blame] | 435 | [1, 2, 3] |
| 436 | >\end{verbatim} |
| 437 | |
| 438 | Note that \function{tee()} has to keep copies of the values returned |
Raymond Hettinger | ed54d91 | 2003-12-31 01:59:18 +0000 | [diff] [blame] | 439 | by the iterator; in the worst case, it may need to keep all of them. |
Andrew M. Kuchling | 44a31e1 | 2004-01-01 18:33:34 +0000 | [diff] [blame] | 440 | This should therefore be used carefully if the leading iterator |
Raymond Hettinger | ed54d91 | 2003-12-31 01:59:18 +0000 | [diff] [blame] | 441 | can run far ahead of the trailing iterator in a long stream of inputs. |
Andrew M. Kuchling | 44a31e1 | 2004-01-01 18:33:34 +0000 | [diff] [blame] | 442 | If the separation is large, then it becomes preferable to use |
Raymond Hettinger | ed54d91 | 2003-12-31 01:59:18 +0000 | [diff] [blame] | 443 | \function{list()} instead. When the iterators track closely with one |
| 444 | another, \function{tee()} is ideal. Possible applications include |
| 445 | bookmarking, windowing, or lookahead iterators. |
Andrew M. Kuchling | 35f2b05 | 2003-12-18 13:28:13 +0000 | [diff] [blame] | 446 | |
Andrew M. Kuchling | dff9dbd | 2003-11-20 22:22:19 +0000 | [diff] [blame] | 447 | \item A new \function{getsid()} function was added to the |
| 448 | \module{posix} module that underlies the \module{os} module. |
| 449 | (Contributed by J. Raynor.) |
| 450 | |
Andrew M. Kuchling | 35f2b05 | 2003-12-18 13:28:13 +0000 | [diff] [blame] | 451 | \item The \module{operator} module gained two new functions, |
| 452 | \function{attrgetter(\var{attr})} and \function{itemgetter(\var{index})}. |
| 453 | Both functions return callables that take a single argument and return |
Raymond Hettinger | ed54d91 | 2003-12-31 01:59:18 +0000 | [diff] [blame] | 454 | the corresponding attribute or item; these callables make excellent |
| 455 | data extractors when used with \function{map()} or \function{sorted()}. |
| 456 | For example: |
Andrew M. Kuchling | 35f2b05 | 2003-12-18 13:28:13 +0000 | [diff] [blame] | 457 | |
| 458 | \begin{verbatim} |
Raymond Hettinger | ed54d91 | 2003-12-31 01:59:18 +0000 | [diff] [blame] | 459 | >>> L = [('c', 2), ('d', 1), ('a', 4), ('b', 3)] |
Andrew M. Kuchling | 35f2b05 | 2003-12-18 13:28:13 +0000 | [diff] [blame] | 460 | >>> map(operator.itemgetter(0), L) |
| 461 | ['c', 'd', 'a', 'b'] |
| 462 | >>> map(operator.itemgetter(1), L) |
Raymond Hettinger | ed54d91 | 2003-12-31 01:59:18 +0000 | [diff] [blame] | 463 | [2, 1, 4, 3] |
| 464 | >>> sorted(L, key=operator.itemgetter(1)) # Sort list by second tuple item |
| 465 | [('d', 1), ('c', 2), ('b', 3), ('a', 4)] |
Andrew M. Kuchling | 35f2b05 | 2003-12-18 13:28:13 +0000 | [diff] [blame] | 466 | \end{verbatim} |
| 467 | |
Andrew M. Kuchling | 6aedcfc | 2003-10-21 12:48:23 +0000 | [diff] [blame] | 468 | \item The \module{random} module has a new method called \method{getrandbits(N)} |
Raymond Hettinger | 607c00f | 2003-11-12 16:27:50 +0000 | [diff] [blame] | 469 | which returns an N-bit long integer. This method supports the existing |
| 470 | \method{randrange()} method, making it possible to efficiently generate |
Andrew M. Kuchling | 44a31e1 | 2004-01-01 18:33:34 +0000 | [diff] [blame] | 471 | arbitrarily large random numbers. |
Andrew M. Kuchling | 6aedcfc | 2003-10-21 12:48:23 +0000 | [diff] [blame] | 472 | |
| 473 | \item The regular expression language accepted by the \module{re} module |
| 474 | was extended with simple conditional expressions, written as |
| 475 | \code{(?(\var{group})\var{A}|\var{B})}. \var{group} is either a |
| 476 | numeric group ID or a group name defined with \code{(?P<group>...)} |
| 477 | earlier in the expression. If the specified group matched, the |
| 478 | regular expression pattern \var{A} will be tested against the string; if |
| 479 | the group didn't match, the pattern \var{B} will be used instead. |
Andrew M. Kuchling | 69f31eb | 2003-08-13 23:11:04 +0000 | [diff] [blame] | 480 | |
Fred Drake | ed0fa3d | 2003-07-30 19:14:09 +0000 | [diff] [blame] | 481 | \end{itemize} |
| 482 | |
| 483 | |
| 484 | %====================================================================== |
| 485 | % whole new modules get described in \subsections here |
| 486 | |
| 487 | |
| 488 | % ====================================================================== |
| 489 | \section{Build and C API Changes} |
| 490 | |
| 491 | Changes to Python's build process and to the C API include: |
| 492 | |
| 493 | \begin{itemize} |
| 494 | |
Andrew M. Kuchling | 6aedcfc | 2003-10-21 12:48:23 +0000 | [diff] [blame] | 495 | \item Three new convenience macros were added for common return |
| 496 | values from extension functions: \csimplemacro{Py_RETURN_NONE}, |
| 497 | \csimplemacro{Py_RETURN_TRUE}, and \csimplemacro{Py_RETURN_FALSE}. |
| 498 | |
Fred Drake | ce3caf2 | 2004-02-12 18:13:12 +0000 | [diff] [blame] | 499 | \item A new function, \cfunction{PyTuple_Pack(\var{N}, \var{obj1}, |
| 500 | \var{obj2}, ..., \var{objN})}, constructs tuples from a variable |
| 501 | length argument list of Python objects. |
Fred Drake | ed0fa3d | 2003-07-30 19:14:09 +0000 | [diff] [blame] | 502 | |
Fred Drake | ce3caf2 | 2004-02-12 18:13:12 +0000 | [diff] [blame] | 503 | \item A new function, \cfunction{PyDict_Contains(\var{d}, \var{k})}, |
| 504 | implements fast dictionary lookups without masking exceptions raised |
| 505 | during the look-up process. |
Raymond Hettinger | d446230 | 2003-11-26 17:52:45 +0000 | [diff] [blame] | 506 | |
Fred Drake | ce3caf2 | 2004-02-12 18:13:12 +0000 | [diff] [blame] | 507 | \item A new method flag, \constant{METH_COEXISTS}, allows a function |
Raymond Hettinger | 97ef8de | 2004-01-05 00:29:57 +0000 | [diff] [blame] | 508 | defined in slots to co-exist with a PyCFunction having the same name. |
| 509 | This can halve the access to time to a method such as |
| 510 | \method{set.__contains__()} |
| 511 | |
Fred Drake | ed0fa3d | 2003-07-30 19:14:09 +0000 | [diff] [blame] | 512 | \end{itemize} |
| 513 | |
| 514 | |
| 515 | %====================================================================== |
| 516 | \subsection{Port-Specific Changes} |
| 517 | |
Raymond Hettinger | 97ef8de | 2004-01-05 00:29:57 +0000 | [diff] [blame] | 518 | \begin{itemize} |
| 519 | |
| 520 | \item The Windows port now builds under MSVC++ 7.1 as well as version 6. |
| 521 | |
| 522 | \end{itemize} |
Fred Drake | ed0fa3d | 2003-07-30 19:14:09 +0000 | [diff] [blame] | 523 | |
| 524 | |
| 525 | %====================================================================== |
| 526 | \section{Other Changes and Fixes \label{section-other}} |
| 527 | |
| 528 | As usual, there were a bunch of other improvements and bugfixes |
| 529 | scattered throughout the source tree. A search through the CVS change |
| 530 | logs finds there were XXX patches applied and YYY bugs fixed between |
| 531 | Python 2.3 and 2.4. Both figures are likely to be underestimates. |
| 532 | |
| 533 | Some of the more notable changes are: |
| 534 | |
| 535 | \begin{itemize} |
| 536 | |
Raymond Hettinger | 97ef8de | 2004-01-05 00:29:57 +0000 | [diff] [blame] | 537 | \item The \module{timeit} module now automatically disables periodic |
| 538 | garbarge collection during the timing loop. This change makes |
| 539 | consecutive timings more comparable. |
| 540 | |
| 541 | \item The \module{base64} module now has more complete RFC 3548 support |
| 542 | for Base64, Base32, and Base16 encoding and decoding, including |
| 543 | optional case folding and optional alternative alphabets. |
| 544 | (Contributed by Barry Warsaw.) |
Fred Drake | ed0fa3d | 2003-07-30 19:14:09 +0000 | [diff] [blame] | 545 | |
| 546 | \end{itemize} |
| 547 | |
| 548 | |
| 549 | %====================================================================== |
| 550 | \section{Porting to Python 2.4} |
| 551 | |
| 552 | This section lists previously described changes that may require |
| 553 | changes to your code: |
| 554 | |
| 555 | \begin{itemize} |
| 556 | |
Raymond Hettinger | 607c00f | 2003-11-12 16:27:50 +0000 | [diff] [blame] | 557 | \item The \function{zip()} built-in function and \function{itertools.izip()} |
| 558 | now return an empty list instead of raising a \exception{TypeError} |
| 559 | exception if called with no arguments. |
Andrew M. Kuchling | 6aedcfc | 2003-10-21 12:48:23 +0000 | [diff] [blame] | 560 | |
| 561 | \item \function{dircache.listdir()} now passes exceptions to the caller |
| 562 | instead of returning empty lists. |
Fred Drake | ed0fa3d | 2003-07-30 19:14:09 +0000 | [diff] [blame] | 563 | |
| 564 | \end{itemize} |
| 565 | |
| 566 | |
| 567 | %====================================================================== |
| 568 | \section{Acknowledgements \label{acks}} |
| 569 | |
| 570 | The author would like to thank the following people for offering |
| 571 | suggestions, corrections and assistance with various drafts of this |
Andrew M. Kuchling | 981a918 | 2003-11-13 21:33:26 +0000 | [diff] [blame] | 572 | article: Raymond Hettinger. |
Fred Drake | ed0fa3d | 2003-07-30 19:14:09 +0000 | [diff] [blame] | 573 | |
| 574 | \end{document} |