Fred Drake | 03e1031 | 2002-03-26 19:17:43 +0000 | [diff] [blame] | 1 | \documentclass{howto} |
Fred Drake | 03e1031 | 2002-03-26 19:17:43 +0000 | [diff] [blame] | 2 | |
Andrew M. Kuchling | 03594bb | 2002-03-27 02:29:48 +0000 | [diff] [blame] | 3 | % $Id$ |
| 4 | |
| 5 | \title{What's New in Python 2.3} |
| 6 | \release{0.01} |
| 7 | \author{A.M. Kuchling} |
| 8 | \authoraddress{\email{akuchlin@mems-exchange.org}} |
Fred Drake | 03e1031 | 2002-03-26 19:17:43 +0000 | [diff] [blame] | 9 | |
| 10 | \begin{document} |
| 11 | \maketitle |
| 12 | \tableofcontents |
| 13 | |
Andrew M. Kuchling | 03594bb | 2002-03-27 02:29:48 +0000 | [diff] [blame] | 14 | %\section{Introduction \label{intro}} |
| 15 | |
| 16 | {\large This article is a draft, and is currently up to date for some |
| 17 | random version of the CVS tree around March 26 2002. Please send any |
| 18 | additions, comments or errata to the author.} |
| 19 | |
| 20 | This article explains the new features in Python 2.3. The tentative |
| 21 | release date of Python 2.3 is currently scheduled for August 30 2002. |
| 22 | |
| 23 | This article doesn't attempt to provide a complete specification of |
| 24 | the new features, but instead provides a convenient overview. For |
| 25 | full details, you should refer to the documentation for Python 2.3, |
| 26 | such as the |
| 27 | \citetitle[http://www.python.org/doc/2.3/lib/lib.html]{Python Library |
| 28 | Reference} and the |
| 29 | \citetitle[http://www.python.org/doc/2.3/ref/ref.html]{Python |
| 30 | Reference Manual}. If you want to understand the complete |
| 31 | implementation and design rationale for a change, refer to the PEP for |
| 32 | a particular new feature. |
Fred Drake | 03e1031 | 2002-03-26 19:17:43 +0000 | [diff] [blame] | 33 | |
| 34 | |
Andrew M. Kuchling | 03594bb | 2002-03-27 02:29:48 +0000 | [diff] [blame] | 35 | %====================================================================== |
Andrew M. Kuchling | f4dd65d | 2002-04-01 19:28:09 +0000 | [diff] [blame] | 36 | \section{PEP 255: Simple Generators} |
| 37 | |
| 38 | In Python 2.2, generators were added as an optional feature, to be |
| 39 | enabled by a \code{from __future__ import generators} directive. In |
| 40 | 2.3 generators no longer need to be specially enabled, and are now |
| 41 | always present; this means that \keyword{yield} is now always a |
| 42 | keyword. The rest of this section is a copy of the description of |
| 43 | generators from the ``What's New in Python 2.2'' document; if you read |
| 44 | it when 2.2 came out, you can skip the rest of this section. |
| 45 | |
| 46 | Generators are a new feature that interacts with the iterators |
| 47 | introduced in Python 2.2. |
| 48 | |
| 49 | You're doubtless familiar with how function calls work in Python or |
| 50 | C. When you call a function, it gets a private namespace where its local |
| 51 | variables are created. When the function reaches a \keyword{return} |
| 52 | statement, the local variables are destroyed and the resulting value |
| 53 | is returned to the caller. A later call to the same function will get |
| 54 | a fresh new set of local variables. But, what if the local variables |
| 55 | weren't thrown away on exiting a function? What if you could later |
| 56 | resume the function where it left off? This is what generators |
| 57 | provide; they can be thought of as resumable functions. |
| 58 | |
| 59 | Here's the simplest example of a generator function: |
| 60 | |
| 61 | \begin{verbatim} |
| 62 | def generate_ints(N): |
| 63 | for i in range(N): |
| 64 | yield i |
| 65 | \end{verbatim} |
| 66 | |
| 67 | A new keyword, \keyword{yield}, was introduced for generators. Any |
| 68 | function containing a \keyword{yield} statement is a generator |
| 69 | function; this is detected by Python's bytecode compiler which |
| 70 | compiles the function specially as a result. |
| 71 | |
| 72 | When you call a generator function, it doesn't return a single value; |
| 73 | instead it returns a generator object that supports the iterator |
| 74 | protocol. On executing the \keyword{yield} statement, the generator |
| 75 | outputs the value of \code{i}, similar to a \keyword{return} |
| 76 | statement. The big difference between \keyword{yield} and a |
| 77 | \keyword{return} statement is that on reaching a \keyword{yield} the |
| 78 | generator's state of execution is suspended and local variables are |
| 79 | preserved. On the next call to the generator's \code{.next()} method, |
| 80 | the function will resume executing immediately after the |
| 81 | \keyword{yield} statement. (For complicated reasons, the |
| 82 | \keyword{yield} statement isn't allowed inside the \keyword{try} block |
| 83 | of a \code{try...finally} statement; read \pep{255} for a full |
| 84 | explanation of the interaction between \keyword{yield} and |
| 85 | exceptions.) |
| 86 | |
| 87 | Here's a sample usage of the \function{generate_ints} generator: |
| 88 | |
| 89 | \begin{verbatim} |
| 90 | >>> gen = generate_ints(3) |
| 91 | >>> gen |
| 92 | <generator object at 0x8117f90> |
| 93 | >>> gen.next() |
| 94 | 0 |
| 95 | >>> gen.next() |
| 96 | 1 |
| 97 | >>> gen.next() |
| 98 | 2 |
| 99 | >>> gen.next() |
| 100 | Traceback (most recent call last): |
| 101 | File "<stdin>", line 1, in ? |
| 102 | File "<stdin>", line 2, in generate_ints |
| 103 | StopIteration |
| 104 | \end{verbatim} |
| 105 | |
| 106 | You could equally write \code{for i in generate_ints(5)}, or |
| 107 | \code{a,b,c = generate_ints(3)}. |
| 108 | |
| 109 | Inside a generator function, the \keyword{return} statement can only |
| 110 | be used without a value, and signals the end of the procession of |
| 111 | values; afterwards the generator cannot return any further values. |
| 112 | \keyword{return} with a value, such as \code{return 5}, is a syntax |
| 113 | error inside a generator function. The end of the generator's results |
| 114 | can also be indicated by raising \exception{StopIteration} manually, |
| 115 | or by just letting the flow of execution fall off the bottom of the |
| 116 | function. |
| 117 | |
| 118 | You could achieve the effect of generators manually by writing your |
| 119 | own class and storing all the local variables of the generator as |
| 120 | instance variables. For example, returning a list of integers could |
| 121 | be done by setting \code{self.count} to 0, and having the |
| 122 | \method{next()} method increment \code{self.count} and return it. |
| 123 | However, for a moderately complicated generator, writing a |
| 124 | corresponding class would be much messier. |
| 125 | \file{Lib/test/test_generators.py} contains a number of more |
| 126 | interesting examples. The simplest one implements an in-order |
| 127 | traversal of a tree using generators recursively. |
| 128 | |
| 129 | \begin{verbatim} |
| 130 | # A recursive generator that generates Tree leaves in in-order. |
| 131 | def inorder(t): |
| 132 | if t: |
| 133 | for x in inorder(t.left): |
| 134 | yield x |
| 135 | yield t.label |
| 136 | for x in inorder(t.right): |
| 137 | yield x |
| 138 | \end{verbatim} |
| 139 | |
| 140 | Two other examples in \file{Lib/test/test_generators.py} produce |
| 141 | solutions for the N-Queens problem (placing $N$ queens on an $NxN$ |
| 142 | chess board so that no queen threatens another) and the Knight's Tour |
| 143 | (a route that takes a knight to every square of an $NxN$ chessboard |
| 144 | without visiting any square twice). |
| 145 | |
| 146 | The idea of generators comes from other programming languages, |
| 147 | especially Icon (\url{http://www.cs.arizona.edu/icon/}), where the |
| 148 | idea of generators is central. In Icon, every |
| 149 | expression and function call behaves like a generator. One example |
| 150 | from ``An Overview of the Icon Programming Language'' at |
| 151 | \url{http://www.cs.arizona.edu/icon/docs/ipd266.htm} gives an idea of |
| 152 | what this looks like: |
| 153 | |
| 154 | \begin{verbatim} |
| 155 | sentence := "Store it in the neighboring harbor" |
| 156 | if (i := find("or", sentence)) > 5 then write(i) |
| 157 | \end{verbatim} |
| 158 | |
| 159 | In Icon the \function{find()} function returns the indexes at which the |
| 160 | substring ``or'' is found: 3, 23, 33. In the \keyword{if} statement, |
| 161 | \code{i} is first assigned a value of 3, but 3 is less than 5, so the |
| 162 | comparison fails, and Icon retries it with the second value of 23. 23 |
| 163 | is greater than 5, so the comparison now succeeds, and the code prints |
| 164 | the value 23 to the screen. |
| 165 | |
| 166 | Python doesn't go nearly as far as Icon in adopting generators as a |
| 167 | central concept. Generators are considered a new part of the core |
| 168 | Python language, but learning or using them isn't compulsory; if they |
| 169 | don't solve any problems that you have, feel free to ignore them. |
| 170 | One novel feature of Python's interface as compared to |
| 171 | Icon's is that a generator's state is represented as a concrete object |
| 172 | (the iterator) that can be passed around to other functions or stored |
| 173 | in a data structure. |
| 174 | |
| 175 | \begin{seealso} |
| 176 | |
| 177 | \seepep{255}{Simple Generators}{Written by Neil Schemenauer, Tim |
| 178 | Peters, Magnus Lie Hetland. Implemented mostly by Neil Schemenauer |
| 179 | and Tim Peters, with other fixes from the Python Labs crew.} |
| 180 | |
| 181 | \end{seealso} |
| 182 | |
| 183 | |
| 184 | %====================================================================== |
Andrew M. Kuchling | f367651 | 2002-04-15 02:27:55 +0000 | [diff] [blame] | 185 | \section{PEP 278: Universal Newline Support} |
| 186 | |
| 187 | XXX write this section |
| 188 | |
| 189 | %Highlights: import and friends will understand any of \r, \n and \r\n |
| 190 | %as end of line. Python file input will do the same if you use mode 'U'. |
| 191 | %Everything can be disabled by configuring with --without-universal-newlines. |
| 192 | |
| 193 | |
| 194 | \begin{seealso} |
| 195 | |
| 196 | \seepep{278}{Universal Newline Support}{Written |
| 197 | and implemented by Jack Jansen.} |
| 198 | |
| 199 | \end{seealso} |
| 200 | |
| 201 | %====================================================================== |
Andrew M. Kuchling | 3a52ff6 | 2002-04-03 22:44:47 +0000 | [diff] [blame] | 202 | \section{PEP 285: The \class{bool} Type} |
| 203 | |
| 204 | XXX write this section |
| 205 | |
| 206 | \begin{seealso} |
| 207 | |
| 208 | \seepep{285}{Adding a bool type}{Written and implemented by GvR.} |
| 209 | |
| 210 | \end{seealso} |
| 211 | |
| 212 | |
| 213 | %====================================================================== |
Andrew M. Kuchling | 03594bb | 2002-03-27 02:29:48 +0000 | [diff] [blame] | 214 | \section{New and Improved Modules} |
| 215 | |
| 216 | arraymodule.c: - add Py_UNICODE arrays |
| 217 | - support +=, *= |
| 218 | |
| 219 | Return enhanced tuples in grpmodule |
| 220 | |
Neal Norwitz | b384c72 | 2002-04-15 12:46:11 +0000 | [diff] [blame] | 221 | posixmodule: killpg, mknod |
Andrew M. Kuchling | 03594bb | 2002-03-27 02:29:48 +0000 | [diff] [blame] | 222 | |
| 223 | Expat is now included with the Python source |
| 224 | |
| 225 | Readline: Add get_history_item, get_current_history_length, and |
| 226 | redisplay functions. |
| 227 | |
Andrew M. Kuchling | f367651 | 2002-04-15 02:27:55 +0000 | [diff] [blame] | 228 | Add optional arg to string methods strip(), lstrip(), rstrip(). |
| 229 | The optional arg specifies characters to delete. |
| 230 | |
Neal Norwitz | b384c72 | 2002-04-15 12:46:11 +0000 | [diff] [blame] | 231 | Add dict method pop(). |
Andrew M. Kuchling | 03594bb | 2002-03-27 02:29:48 +0000 | [diff] [blame] | 232 | |
| 233 | %====================================================================== |
| 234 | \section{Interpreter Changes and Fixes} |
| 235 | |
| 236 | XXX bug? Change the version string from "2.2+" to "2.3a0". disutils peels off |
| 237 | the first 3 characters of this string in several places, so for as long |
| 238 | as they remain "2.2" it confuses the heck out of attempts to build 2.3 |
| 239 | stuff using distutils. |
| 240 | |
| 241 | file object can now be subtyped (did this not work before?) |
| 242 | |
| 243 | yield is now always available |
| 244 | |
| 245 | This adds the module name and a dot in front of the type name in every |
| 246 | type object initializer, except for built-in types (and those that |
| 247 | already had this). Note that it touches lots of Mac modules -- I have |
| 248 | no way to test these but the changes look right. Apologies if they're |
| 249 | not. This also touches the weakref docs, which contains a sample type |
| 250 | object initializer. It also touches the mmap test output, because the |
| 251 | mmap type's repr is included in that output. It touches object.h to |
| 252 | put the correct description in a comment. |
| 253 | |
| 254 | File objects: Grow the string buffer at a mildly exponential rate for |
| 255 | the getc version of get_line. This makes test_bufio finish in 1.7 |
| 256 | seconds instead of 57 seconds on my machine (with Py_DEBUG defined). |
| 257 | |
| 258 | %====================================================================== |
| 259 | \section{Other Changes and Fixes} |
| 260 | |
| 261 | |
| 262 | % ====================================================================== |
| 263 | \section{C Interface Changes} |
| 264 | |
Andrew M. Kuchling | f4dd65d | 2002-04-01 19:28:09 +0000 | [diff] [blame] | 265 | Patch \#527027: Allow building python as shared library with |
| 266 | --enable-shared |
| 267 | |
Andrew M. Kuchling | 03594bb | 2002-03-27 02:29:48 +0000 | [diff] [blame] | 268 | pymalloc is now enabled by default (also mention debug-mode pymalloc) |
| 269 | |
Andrew M. Kuchling | f4dd65d | 2002-04-01 19:28:09 +0000 | [diff] [blame] | 270 | Memory API reworking -- which functions are deprecated? |
Andrew M. Kuchling | 03594bb | 2002-03-27 02:29:48 +0000 | [diff] [blame] | 271 | |
| 272 | PyObject_DelItemString() added |
| 273 | |
| 274 | PyArg_NoArgs macro is now deprecated |
| 275 | |
Andrew M. Kuchling | 45afd54 | 2002-04-02 14:25:25 +0000 | [diff] [blame] | 276 | === |
| 277 | Introduce two new flag bits that can be set in a PyMethodDef method |
| 278 | descriptor, as used for the tp_methods slot of a type. These new flag |
| 279 | bits are both optional, and mutually exclusive. Most methods will not |
| 280 | use either. These flags are used to create special method types which |
| 281 | exist in the same namespace as normal methods without having to use |
| 282 | tedious construction code to insert the new special method objects in |
| 283 | the type's tp_dict after PyType_Ready() has been called. |
| 284 | |
| 285 | If METH_CLASS is specified, the method will represent a class method |
| 286 | like that returned by the classmethod() built-in. |
| 287 | |
| 288 | If METH_STATIC is specified, the method will represent a static method |
| 289 | like that returned by the staticmethod() built-in. |
| 290 | |
| 291 | These flags may not be used in the PyMethodDef table for modules since |
| 292 | these special method types are not meaningful in that case; a |
| 293 | ValueError will be raised if these flags are found in that context. |
| 294 | === |
| 295 | |
Andrew M. Kuchling | 03594bb | 2002-03-27 02:29:48 +0000 | [diff] [blame] | 296 | Ports: |
| 297 | |
| 298 | OS/2 EMX port |
| 299 | |
| 300 | MacOS: Weaklink most toolbox modules, improving backward |
| 301 | compatibility. Modules will no longer fail to load if a single routine |
| 302 | is missing on the curent OS version, in stead calling the missing |
| 303 | routine will raise an exception. Should finally fix 531398. 2.2.1 |
| 304 | candidate. Also blacklisted some constants with definitions that |
| 305 | were not Python-compatible. |
| 306 | |
| 307 | Checked in Sean Reifschneider's RPM spec file and patches. Bugfix candidate. |
Fred Drake | 03e1031 | 2002-03-26 19:17:43 +0000 | [diff] [blame] | 308 | |
| 309 | |
| 310 | %====================================================================== |
| 311 | \section{Acknowledgements \label{acks}} |
| 312 | |
Andrew M. Kuchling | 03594bb | 2002-03-27 02:29:48 +0000 | [diff] [blame] | 313 | The author would like to thank the following people for offering |
| 314 | suggestions, corrections and assistance with various drafts of this |
| 315 | article: Fred~L. Drake, Jr. |
Fred Drake | 03e1031 | 2002-03-26 19:17:43 +0000 | [diff] [blame] | 316 | |
| 317 | \end{document} |
Andrew M. Kuchling | 03594bb | 2002-03-27 02:29:48 +0000 | [diff] [blame] | 318 | |
| 319 | |
| 320 | |