Georg Brandl | c47408a | 2008-12-04 18:44:53 +0000 | [diff] [blame] | 1 | **************************** |
Georg Brandl | 48310cd | 2009-01-03 21:18:54 +0000 | [diff] [blame] | 2 | What's New In Python 3.1 |
Georg Brandl | c47408a | 2008-12-04 18:44:53 +0000 | [diff] [blame] | 3 | **************************** |
| 4 | |
Raymond Hettinger | 1f251a0 | 2009-04-04 10:47:35 +0000 | [diff] [blame] | 5 | :Author: Raymond Hettinger |
Benjamin Peterson | 3432829 | 2008-12-05 03:05:29 +0000 | [diff] [blame] | 6 | :Release: |release| |
| 7 | :Date: |today| |
Georg Brandl | c47408a | 2008-12-04 18:44:53 +0000 | [diff] [blame] | 8 | |
| 9 | .. $Id$ |
| 10 | Rules for maintenance: |
Georg Brandl | 48310cd | 2009-01-03 21:18:54 +0000 | [diff] [blame] | 11 | |
Georg Brandl | c47408a | 2008-12-04 18:44:53 +0000 | [diff] [blame] | 12 | * Anyone can add text to this document. Do not spend very much time |
| 13 | on the wording of your changes, because your text will probably |
| 14 | get rewritten to some degree. |
Georg Brandl | 48310cd | 2009-01-03 21:18:54 +0000 | [diff] [blame] | 15 | |
Georg Brandl | c47408a | 2008-12-04 18:44:53 +0000 | [diff] [blame] | 16 | * The maintainer will go through Misc/NEWS periodically and add |
| 17 | changes; it's therefore more important to add your changes to |
Raymond Hettinger | b9b3319 | 2009-04-08 09:38:32 +0000 | [diff] [blame] | 18 | Misc/NEWS than to this file. |
Georg Brandl | 48310cd | 2009-01-03 21:18:54 +0000 | [diff] [blame] | 19 | |
Georg Brandl | c47408a | 2008-12-04 18:44:53 +0000 | [diff] [blame] | 20 | * This is not a complete list of every single change; completeness |
| 21 | is the purpose of Misc/NEWS. Some changes I consider too small |
| 22 | or esoteric to include. If such a change is added to the text, |
| 23 | I'll just remove it. (This is another reason you shouldn't spend |
| 24 | too much time on writing your addition.) |
Georg Brandl | 48310cd | 2009-01-03 21:18:54 +0000 | [diff] [blame] | 25 | |
Georg Brandl | c47408a | 2008-12-04 18:44:53 +0000 | [diff] [blame] | 26 | * If you want to draw your new text to the attention of the |
| 27 | maintainer, add 'XXX' to the beginning of the paragraph or |
| 28 | section. |
Georg Brandl | 48310cd | 2009-01-03 21:18:54 +0000 | [diff] [blame] | 29 | |
Georg Brandl | c47408a | 2008-12-04 18:44:53 +0000 | [diff] [blame] | 30 | * It's OK to just add a fragmentary note about a change. For |
| 31 | example: "XXX Describe the transmogrify() function added to the |
| 32 | socket module." The maintainer will research the change and |
| 33 | write the necessary text. |
Georg Brandl | 48310cd | 2009-01-03 21:18:54 +0000 | [diff] [blame] | 34 | |
Georg Brandl | c47408a | 2008-12-04 18:44:53 +0000 | [diff] [blame] | 35 | * You can comment out your additions if you like, but it's not |
| 36 | necessary (especially when a final release is some months away). |
Georg Brandl | 48310cd | 2009-01-03 21:18:54 +0000 | [diff] [blame] | 37 | |
Georg Brandl | c47408a | 2008-12-04 18:44:53 +0000 | [diff] [blame] | 38 | * Credit the author of a patch or bugfix. Just the name is |
Raymond Hettinger | b9b3319 | 2009-04-08 09:38:32 +0000 | [diff] [blame] | 39 | sufficient; the e-mail address isn't necessary. |
Georg Brandl | 48310cd | 2009-01-03 21:18:54 +0000 | [diff] [blame] | 40 | |
Georg Brandl | c47408a | 2008-12-04 18:44:53 +0000 | [diff] [blame] | 41 | * It's helpful to add the bug/patch number as a comment: |
Georg Brandl | 48310cd | 2009-01-03 21:18:54 +0000 | [diff] [blame] | 42 | |
Georg Brandl | c47408a | 2008-12-04 18:44:53 +0000 | [diff] [blame] | 43 | % Patch 12345 |
| 44 | XXX Describe the transmogrify() function added to the socket |
| 45 | module. |
| 46 | (Contributed by P.Y. Developer.) |
Georg Brandl | 48310cd | 2009-01-03 21:18:54 +0000 | [diff] [blame] | 47 | |
Georg Brandl | c47408a | 2008-12-04 18:44:53 +0000 | [diff] [blame] | 48 | This saves the maintainer the effort of going through the SVN log |
Raymond Hettinger | b9b3319 | 2009-04-08 09:38:32 +0000 | [diff] [blame] | 49 | when researching a change. |
Georg Brandl | c47408a | 2008-12-04 18:44:53 +0000 | [diff] [blame] | 50 | |
| 51 | This article explains the new features in Python 3.1, compared to 3.0. |
| 52 | |
Georg Brandl | c47408a | 2008-12-04 18:44:53 +0000 | [diff] [blame] | 53 | |
Raymond Hettinger | 1f251a0 | 2009-04-04 10:47:35 +0000 | [diff] [blame] | 54 | PEP 372: Ordered Dictionaries |
| 55 | ============================= |
| 56 | |
| 57 | Regular Python dictionaries iterate over key/value pairs in arbitrary order. |
| 58 | Over the years, a number of authors have written alternative implementations |
| 59 | that remember the order that the keys were originally inserted. Based on |
Raymond Hettinger | d621dd7 | 2009-04-14 08:16:50 +0000 | [diff] [blame] | 60 | the experiences from those implementations, a new |
| 61 | :class:`collections.OrderedDict` class has been introduced. |
Raymond Hettinger | 1f251a0 | 2009-04-04 10:47:35 +0000 | [diff] [blame] | 62 | |
| 63 | The OrderedDict API is substantially the same as regular dictionaries |
| 64 | but will iterate over keys and values in a guaranteed order depending on |
| 65 | when a key was first inserted. If a new entry overwrites an existing entry, |
| 66 | the original insertion position is left unchanged. Deleting an entry and |
| 67 | reinserting it will move it to the end. |
| 68 | |
| 69 | The standard library now supports use of ordered dictionaries in several |
Raymond Hettinger | 7f5d746 | 2009-04-14 08:05:31 +0000 | [diff] [blame] | 70 | modules. The :mod:`configparser` module uses them by default. This lets |
Raymond Hettinger | 1f251a0 | 2009-04-04 10:47:35 +0000 | [diff] [blame] | 71 | configuration files be read, modified, and then written back in their original |
Raymond Hettinger | d621dd7 | 2009-04-14 08:16:50 +0000 | [diff] [blame] | 72 | order. The *_asdict()* method for :func:`collections.namedtuple` now |
Raymond Hettinger | c4f6d29 | 2009-04-04 12:35:58 +0000 | [diff] [blame] | 73 | returns an ordered dictionary with the values appearing in the same order as |
| 74 | the underlying tuple indicies. The :mod:`json` module is being built-out with |
| 75 | an *object_pairs_hook* to allow OrderedDicts to be built by the decoder. |
Raymond Hettinger | 347396a | 2009-04-07 23:10:59 +0000 | [diff] [blame] | 76 | Support was also added for third-party tools like `PyYAML <http://pyyaml.org/>`_. |
Raymond Hettinger | 1f251a0 | 2009-04-04 10:47:35 +0000 | [diff] [blame] | 77 | |
| 78 | .. seealso:: |
| 79 | |
| 80 | :pep:`372` - Ordered Dictionaries |
Raymond Hettinger | f84dfe5 | 2009-04-04 13:13:56 +0000 | [diff] [blame] | 81 | PEP written by Armin Ronacher and Raymond Hettinger. Implementation |
| 82 | written by Raymond Hettinger. |
Raymond Hettinger | 1f251a0 | 2009-04-04 10:47:35 +0000 | [diff] [blame] | 83 | |
Raymond Hettinger | b9b3319 | 2009-04-08 09:38:32 +0000 | [diff] [blame] | 84 | |
Raymond Hettinger | 1f251a0 | 2009-04-04 10:47:35 +0000 | [diff] [blame] | 85 | PEP 378: Format Specifier for Thousands Separator |
| 86 | ================================================= |
| 87 | |
| 88 | The builtin :func:`format` function and the :meth:`str.format` method use |
| 89 | a mini-language that now includes a simple, non-locale aware way to format |
| 90 | a number with a thousands separator. That provides a way to humanize a |
| 91 | program's output, improving its professional appearance and readability:: |
| 92 | |
Raymond Hettinger | 0422e14 | 2009-04-17 18:58:06 +0000 | [diff] [blame] | 93 | >>> format(1234567, ',d') |
| 94 | '1,234,567' |
| 95 | >>> format(1234567.89, ',.2f') |
| 96 | '1,234,567.89' |
Raymond Hettinger | d48ed2e | 2009-05-14 22:48:19 +0000 | [diff] [blame] | 97 | >>> format(12345.6 + 8901234.12j, ',f') |
| 98 | '12,345.600000+8,901,234.120000j' |
Raymond Hettinger | 1f251a0 | 2009-04-04 10:47:35 +0000 | [diff] [blame] | 99 | >>> format(Decimal('1234567.89'), ',f') |
| 100 | '1,234,567.89' |
| 101 | |
Raymond Hettinger | d48ed2e | 2009-05-14 22:48:19 +0000 | [diff] [blame] | 102 | The supported types are :class:`int`, :class:`float`, :class:`complex` |
| 103 | and :class:`decimal.Decimal`. |
Raymond Hettinger | c548b6e | 2009-04-17 10:09:27 +0000 | [diff] [blame] | 104 | |
Raymond Hettinger | 1f251a0 | 2009-04-04 10:47:35 +0000 | [diff] [blame] | 105 | Discussions are underway about how to specify alternative separators |
Raymond Hettinger | c4f6d29 | 2009-04-04 12:35:58 +0000 | [diff] [blame] | 106 | like dots, spaces, apostrophes, or underscores. Locale-aware applications |
| 107 | should use the existing *n* format specifier which already has some support |
| 108 | for thousands separators. |
Raymond Hettinger | 1f251a0 | 2009-04-04 10:47:35 +0000 | [diff] [blame] | 109 | |
| 110 | .. seealso:: |
| 111 | |
| 112 | :pep:`378` - Format Specifier for Thousands Separator |
Raymond Hettinger | 55fc9ce | 2009-04-14 20:45:17 +0000 | [diff] [blame] | 113 | PEP written by Raymond Hettinger and implemented by Eric Smith and |
Raymond Hettinger | 1f251a0 | 2009-04-04 10:47:35 +0000 | [diff] [blame] | 114 | Mark Dickinson. |
| 115 | |
| 116 | |
Mark Dickinson | 54bc1ec | 2008-12-17 16:19:07 +0000 | [diff] [blame] | 117 | Other Language Changes |
| 118 | ====================== |
| 119 | |
| 120 | Some smaller changes made to the core Python language are: |
| 121 | |
| 122 | * The :func:`int` type gained a ``bit_length`` method that returns the |
| 123 | number of bits necessary to represent its argument in binary:: |
| 124 | |
| 125 | >>> n = 37 |
| 126 | >>> bin(37) |
| 127 | '0b100101' |
| 128 | >>> n.bit_length() |
| 129 | 6 |
| 130 | >>> n = 2**123-1 |
| 131 | >>> n.bit_length() |
| 132 | 123 |
| 133 | >>> (n+1).bit_length() |
| 134 | 124 |
| 135 | |
Raymond Hettinger | c4f6d29 | 2009-04-04 12:35:58 +0000 | [diff] [blame] | 136 | (Contributed by Fredrik Johansson, Victor Stinner, Raymond Hettinger, |
| 137 | and Mark Dickinson; :issue:`3439`.) |
Mark Dickinson | 54bc1ec | 2008-12-17 16:19:07 +0000 | [diff] [blame] | 138 | |
Raymond Hettinger | 79d0b0e | 2009-04-07 07:11:00 +0000 | [diff] [blame] | 139 | * The fields in :func:`format` strings can now be automatically |
| 140 | numbered:: |
| 141 | |
| 142 | >>> 'Sir {} of {}'.format('Gallahad', 'Camelot') |
| 143 | 'Sir Gallahad of Camelot' |
| 144 | |
| 145 | Formerly, the string would have required numbered fields such as: |
| 146 | ``'Sir {0} of {1}'``. |
| 147 | |
| 148 | (Contributed by Eric Smith; :issue:`5237`.) |
| 149 | |
Raymond Hettinger | bd3da6b | 2009-05-15 16:16:12 +0000 | [diff] [blame] | 150 | * The :func:`string.maketrans` function is deprecated and is replaced by new |
| 151 | static methods, :meth:`bytes.maketrans` and :meth:`bytearray.maketrans`. |
| 152 | This change solves the confusion around which types were supported by the |
| 153 | :mod:`string` module. Now, :class:`str`, :class:`bytes`, and |
| 154 | :class:`bytearray` each have their own **maketrans** and **translate** |
| 155 | methods with intermediate translation tables of the appropriate type. |
| 156 | |
| 157 | (Contributed by Georg Brandl; :issue:`5675`.) |
| 158 | |
Raymond Hettinger | 81c0dce | 2009-05-27 09:12:18 +0000 | [diff] [blame] | 159 | * The syntax of the :keyword:`with` statement now allows multiple context |
| 160 | managers in a single statement:: |
| 161 | |
| 162 | >>> with open('mylog.txt') as infile, open('a.out', 'w') as outfile: |
| 163 | ... for line in infile: |
| 164 | ... if '<critical>' in line: |
| 165 | ... outfile.write(line) |
| 166 | |
Raymond Hettinger | 91e3b9d | 2009-05-28 22:20:03 +0000 | [diff] [blame] | 167 | With the new syntax, the :func:`contextlib.nested` function is no longer |
Raymond Hettinger | 323ee0c | 2009-05-28 22:49:33 +0000 | [diff] [blame] | 168 | needed and is now deprecated. |
Raymond Hettinger | 91e3b9d | 2009-05-28 22:20:03 +0000 | [diff] [blame] | 169 | |
Georg Brandl | ed23ab6 | 2009-05-27 19:46:38 +0000 | [diff] [blame] | 170 | (Contributed by Georg Brandl and Mattias Brändström; |
Raymond Hettinger | 81c0dce | 2009-05-27 09:12:18 +0000 | [diff] [blame] | 171 | `appspot issue 53094 <http://codereview.appspot.com/53094>`_.) |
| 172 | |
Raymond Hettinger | 79d0b0e | 2009-04-07 07:11:00 +0000 | [diff] [blame] | 173 | * ``round(x, n)`` now returns an integer if *x* is an integer. |
| 174 | Previously it returned a float:: |
| 175 | |
| 176 | >>> round(1123, -2) |
| 177 | 1100 |
| 178 | |
| 179 | (Contributed by Mark Dickinson; :issue:`4707`.) |
| 180 | |
Raymond Hettinger | 8648e50 | 2009-04-17 00:11:54 +0000 | [diff] [blame] | 181 | * Python now uses David Gay's algorithm for finding the shortest floating |
| 182 | point representation that doesn't change its value. This should help |
Eric Smith | 86c06bc | 2009-04-17 12:33:24 +0000 | [diff] [blame] | 183 | mitigate some of the confusion surrounding binary floating point |
Raymond Hettinger | 8648e50 | 2009-04-17 00:11:54 +0000 | [diff] [blame] | 184 | numbers. |
| 185 | |
| 186 | The significance is easily seen with a number like ``1.1`` which does not |
| 187 | have an exact equivalent in binary floating point. Since there is no exact |
Raymond Hettinger | fee346b | 2009-04-17 09:45:19 +0000 | [diff] [blame] | 188 | equivalent, an expression like ``float('1.1')`` evaluates to the nearest |
Raymond Hettinger | 8648e50 | 2009-04-17 00:11:54 +0000 | [diff] [blame] | 189 | representable value which is ``0x1.199999999999ap+0`` in hex or |
| 190 | ``1.100000000000000088817841970012523233890533447265625`` in decimal. That |
| 191 | nearest value was and still is used in subsequent floating point |
| 192 | calculations. |
| 193 | |
| 194 | What is new is how the number gets displayed. Formerly, Python used a |
| 195 | simple approach. The value of ``repr(1.1)`` was computed as ``format(1.1, |
Raymond Hettinger | fee346b | 2009-04-17 09:45:19 +0000 | [diff] [blame] | 196 | '.17g')`` which evaluated to ``'1.1000000000000001'``. The advantage of |
Raymond Hettinger | 8648e50 | 2009-04-17 00:11:54 +0000 | [diff] [blame] | 197 | using 17 digits was that it relied on IEEE-754 guarantees to assure that |
| 198 | ``eval(repr(1.1))`` would round-trip exactly to its original value. The |
| 199 | disadvantage is that many people found the output to be confusing (mistaking |
| 200 | intrinsic limitations of binary floating point representation as being a |
| 201 | problem with Python itself). |
| 202 | |
Raymond Hettinger | fee346b | 2009-04-17 09:45:19 +0000 | [diff] [blame] | 203 | The new algorithm for ``repr(1.1)`` is smarter and returns ``'1.1'``. |
Raymond Hettinger | 8648e50 | 2009-04-17 00:11:54 +0000 | [diff] [blame] | 204 | Effectively, it searches all equivalent string representations (ones that |
Raymond Hettinger | fee346b | 2009-04-17 09:45:19 +0000 | [diff] [blame] | 205 | get stored with the same underlying float value) and returns the shortest |
Raymond Hettinger | 8648e50 | 2009-04-17 00:11:54 +0000 | [diff] [blame] | 206 | representation. |
| 207 | |
| 208 | The new algorithm tends to emit cleaner representations when possible, but |
| 209 | it does not change the underlying values. So, it is still the case that |
| 210 | ``1.1 + 2.2 != 3.3`` even though the representations may suggest otherwise. |
| 211 | |
| 212 | The new algorithm depends on certain features in the underlying floating |
| 213 | point implementation. If the required features are not found, the old |
| 214 | algorithm will continue to be used. Also, the text pickle protocols |
| 215 | assure cross-platform portability by using the old algorithm. |
| 216 | |
| 217 | (Contributed by Eric Smith and Mark Dickinson; :issue:`1580`) |
Raymond Hettinger | 79d0b0e | 2009-04-07 07:11:00 +0000 | [diff] [blame] | 218 | |
| 219 | New, Improved, and Deprecated Modules |
| 220 | ===================================== |
| 221 | |
Raymond Hettinger | 1f251a0 | 2009-04-04 10:47:35 +0000 | [diff] [blame] | 222 | * Added a :class:`collections.Counter` class to support convenient |
| 223 | counting of unique items in a sequence or iterable:: |
| 224 | |
| 225 | >>> Counter(['red', 'blue', 'red', 'green', 'blue', 'blue']) |
| 226 | Counter({'blue': 3, 'red': 2, 'green': 1}) |
| 227 | |
| 228 | (Contributed by Raymond Hettinger; :issue:`1696199`.) |
| 229 | |
Raymond Hettinger | c1bd49a | 2009-04-06 23:11:08 +0000 | [diff] [blame] | 230 | * Added a new module, :mod:`tkinter.ttk` for access to the Tk themed widget set. |
Raymond Hettinger | dbd51b5 | 2009-04-06 22:45:52 +0000 | [diff] [blame] | 231 | The basic idea of ttk is to separate, to the extent possible, the code |
Raymond Hettinger | f84dfe5 | 2009-04-04 13:13:56 +0000 | [diff] [blame] | 232 | implementing a widget's behavior from the code implementing its appearance. |
| 233 | |
Raymond Hettinger | d1b3de3 | 2009-04-08 00:09:26 +0000 | [diff] [blame] | 234 | (Contributed by Guilherme Polo; :issue:`2983`.) |
Raymond Hettinger | f84dfe5 | 2009-04-04 13:13:56 +0000 | [diff] [blame] | 235 | |
Raymond Hettinger | f4cc2c4 | 2009-04-06 22:39:03 +0000 | [diff] [blame] | 236 | * The :class:`gzip.GzipFile` and :class:`bz2.BZ2File` classes now support |
| 237 | the context manager protocol:: |
Raymond Hettinger | 1f251a0 | 2009-04-04 10:47:35 +0000 | [diff] [blame] | 238 | |
Raymond Hettinger | f4cc2c4 | 2009-04-06 22:39:03 +0000 | [diff] [blame] | 239 | >>> # Automatically close file after writing |
| 240 | >>> with gzip.GzipFile(filename, "wb") as f: |
| 241 | ... f.write(b"xxx") |
| 242 | |
| 243 | (Contributed by Antoine Pitrou.) |
Raymond Hettinger | 1f251a0 | 2009-04-04 10:47:35 +0000 | [diff] [blame] | 244 | |
Raymond Hettinger | bf1537c | 2009-04-09 22:34:46 +0000 | [diff] [blame] | 245 | * The :mod:`decimal` module now supports methods for creating a |
Raymond Hettinger | 76efa2b | 2009-04-06 23:11:47 +0000 | [diff] [blame] | 246 | decimal object from a binary :class:`float`. The conversion is |
Raymond Hettinger | 1f251a0 | 2009-04-04 10:47:35 +0000 | [diff] [blame] | 247 | exact but can sometimes be surprising:: |
| 248 | |
| 249 | >>> Decimal.from_float(1.1) |
| 250 | Decimal('1.100000000000000088817841970012523233890533447265625') |
| 251 | |
| 252 | The long decimal result shows the actual binary fraction being |
| 253 | stored for *1.1*. The fraction has many digits because *1.1* cannot |
| 254 | be exactly represented in binary. |
| 255 | |
| 256 | (Contributed by Raymond Hettinger and Mark Dickinson.) |
| 257 | |
Raymond Hettinger | e7ec57d | 2009-04-04 11:08:48 +0000 | [diff] [blame] | 258 | * The :mod:`itertools` module grew two new functions. The |
| 259 | :func:`itertools.combinations_with_replacement` function is one of |
| 260 | four for generating combinatorics including permutations and Cartesian |
| 261 | products. The :func:`itertools.compress` function mimics its namesake |
| 262 | from APL. Also, the existing :func:`itertools.count` function now has |
| 263 | an optional *step* argument and can accept any type of counting |
| 264 | sequence including :class:`fractions.Fraction` and |
Raymond Hettinger | 8d97ccb | 2009-04-06 17:55:05 +0000 | [diff] [blame] | 265 | :class:`decimal.Decimal`:: |
| 266 | |
| 267 | >>> [p+q for p,q in combinations_with_replacement('LOVE', 2)] |
| 268 | ['LL', 'LO', 'LV', 'LE', 'OO', 'OV', 'OE', 'VV', 'VE', 'EE'] |
| 269 | |
| 270 | >>> list(compress(data=range(10), selectors=[0,0,1,1,0,1,0,1,0,0])) |
| 271 | [2, 3, 5, 7] |
| 272 | |
| 273 | >>> c = count(start=Fraction(1,2), step=Fraction(1,6)) |
Raymond Hettinger | bd3da6b | 2009-05-15 16:16:12 +0000 | [diff] [blame] | 274 | >>> [next(c), next(c), next(c), next(c)] |
| 275 | [Fraction(1, 2), Fraction(2, 3), Fraction(5, 6), Fraction(1, 1)] |
Raymond Hettinger | e7ec57d | 2009-04-04 11:08:48 +0000 | [diff] [blame] | 276 | |
| 277 | (Contributed by Raymond Hettinger.) |
| 278 | |
Raymond Hettinger | f41857e | 2009-04-04 11:59:00 +0000 | [diff] [blame] | 279 | * :func:`collections.namedtuple` now supports a keyword argument |
| 280 | *rename* which lets invalid fieldnames be automatically converted to |
| 281 | positional names in the form _0, _1, etc. This is useful when |
| 282 | the field names are being created by an external source such as a |
Raymond Hettinger | 4c0b1e4 | 2009-04-08 07:49:03 +0000 | [diff] [blame] | 283 | CSV header, SQL field list, or user input:: |
| 284 | |
Raymond Hettinger | 2a39e0f | 2009-04-08 22:50:09 +0000 | [diff] [blame] | 285 | >>> query = input() |
| 286 | SELECT region, dept, count(*) FROM main GROUPBY region, dept |
Raymond Hettinger | 4c0b1e4 | 2009-04-08 07:49:03 +0000 | [diff] [blame] | 287 | |
| 288 | >>> cursor.execute(query) |
| 289 | >>> query_fields = [desc[0] for desc in cursor.description] |
| 290 | >>> UserQuery = namedtuple('UserQuery', query_fields, rename=True) |
| 291 | >>> pprint.pprint([UserQuery(*row) for row in cursor]) |
| 292 | [UserQuery(region='South', dept='Shipping', _2=185), |
| 293 | UserQuery(region='North', dept='Accounting', _2=37), |
| 294 | UserQuery(region='West', dept='Sales', _2=419)] |
Raymond Hettinger | f41857e | 2009-04-04 11:59:00 +0000 | [diff] [blame] | 295 | |
| 296 | (Contributed by Raymond Hettinger; :issue:`1818`.) |
| 297 | |
Raymond Hettinger | f41857e | 2009-04-04 11:59:00 +0000 | [diff] [blame] | 298 | * The :func:`re.sub`, :func:`re.subn` and :func:`re.split` functions now |
| 299 | accept a flags parameter. |
| 300 | |
| 301 | (Contributed by Gregory Smith.) |
| 302 | |
Raymond Hettinger | d621dd7 | 2009-04-14 08:16:50 +0000 | [diff] [blame] | 303 | * The :mod:`logging` module now implements a simple :class:`logging.NullHandler` |
Raymond Hettinger | 35a8836 | 2009-04-09 00:08:24 +0000 | [diff] [blame] | 304 | class for applications that are not using logging but are calling |
| 305 | library code that does. Setting-up a null handler will suppress |
Raymond Hettinger | d621dd7 | 2009-04-14 08:16:50 +0000 | [diff] [blame] | 306 | spurious warnings such as "No handlers could be found for logger foo":: |
Raymond Hettinger | 35a8836 | 2009-04-09 00:08:24 +0000 | [diff] [blame] | 307 | |
| 308 | >>> h = logging.NullHandler() |
| 309 | >>> logging.getLogger("foo").addHandler(h) |
| 310 | |
Raymond Hettinger | d621dd7 | 2009-04-14 08:16:50 +0000 | [diff] [blame] | 311 | (Contributed by Vinay Sajip; :issue:`4384`). |
Raymond Hettinger | 35a8836 | 2009-04-09 00:08:24 +0000 | [diff] [blame] | 312 | |
Raymond Hettinger | f41857e | 2009-04-04 11:59:00 +0000 | [diff] [blame] | 313 | * The :mod:`runpy` module which supports the ``-m`` command line switch |
| 314 | now supports the execution of packages by looking for and executing |
| 315 | a ``__main__`` submodule when a package name is supplied. |
| 316 | |
| 317 | (Contributed by Andi Vajda; :issue:`4195`.) |
| 318 | |
| 319 | * The :mod:`pdb` module can now access and display source code loaded via |
| 320 | :mod:`zipimport` (or any other conformant :pep:`302` loader). |
| 321 | |
| 322 | (Contributed by Alexander Belopolsky; :issue:`4201`.) |
| 323 | |
| 324 | * :class:`functools.partial` objects can now be pickled. |
| 325 | |
| 326 | (Suggested by Antoine Pitrou and Jesse Noller. Implemented by |
| 327 | Jack Diedrich; :issue:`5228`.) |
| 328 | |
Raymond Hettinger | 8e33051 | 2009-04-04 13:20:55 +0000 | [diff] [blame] | 329 | * Add :mod:`pydoc` help topics for symbols so that ``help('@')`` |
| 330 | works as expected in the interactive environment. |
| 331 | |
| 332 | (Contributed by David Laban; :issue:`4739`.) |
| 333 | |
Raymond Hettinger | c4f6d29 | 2009-04-04 12:35:58 +0000 | [diff] [blame] | 334 | * The :mod:`unittest` module now supports skipping individual tests or classes |
| 335 | of tests. And it supports marking a test as a expected failure, a test that |
Raymond Hettinger | 8daab40 | 2009-04-04 13:01:19 +0000 | [diff] [blame] | 336 | is known to be broken, but shouldn't be counted as a failure on a |
Raymond Hettinger | 8d97ccb | 2009-04-06 17:55:05 +0000 | [diff] [blame] | 337 | TestResult:: |
| 338 | |
| 339 | class TestGizmo(unittest.TestCase): |
| 340 | |
| 341 | @unittest.skipUnless(sys.platform.startswith("win"), "requires Windows") |
| 342 | def test_gizmo_on_windows(self): |
| 343 | ... |
| 344 | |
| 345 | @unittest.expectedFailure |
| 346 | def test_gimzo_without_required_library(self): |
| 347 | ... |
Raymond Hettinger | f41857e | 2009-04-04 11:59:00 +0000 | [diff] [blame] | 348 | |
Raymond Hettinger | 55fc9ce | 2009-04-14 20:45:17 +0000 | [diff] [blame] | 349 | Also, tests for exceptions have been builtout to work with context managers |
| 350 | using the :keyword:`with` statement:: |
Raymond Hettinger | 35a8836 | 2009-04-09 00:08:24 +0000 | [diff] [blame] | 351 | |
| 352 | def test_division_by_zero(self): |
| 353 | with self.assertRaises(ZeroDivisionError): |
| 354 | x / 0 |
| 355 | |
| 356 | In addition, several new assertion methods were added including |
| 357 | :func:`assertSetEqual`, :func:`assertDictEqual`, |
| 358 | :func:`assertDictContainsSubset`, :func:`assertListEqual`, |
| 359 | :func:`assertTupleEqual`, :func:`assertSequenceEqual`, |
| 360 | :func:`assertRaisesRegexp`, :func:`assertIsNone`, |
Michael Foord | 5859b86 | 2009-04-25 20:47:43 +0000 | [diff] [blame] | 361 | and :func:`assertIsNotNone`. |
Raymond Hettinger | 35a8836 | 2009-04-09 00:08:24 +0000 | [diff] [blame] | 362 | |
| 363 | (Contributed by Benjamin Peterson and Antoine Pitrou.) |
| 364 | |
Raymond Hettinger | be3a821 | 2009-04-09 00:18:29 +0000 | [diff] [blame] | 365 | * The :mod:`io` module has three new constants for the :meth:`seek` |
Raymond Hettinger | 35a8836 | 2009-04-09 00:08:24 +0000 | [diff] [blame] | 366 | method :data:`SEEK_SET`, :data:`SEEK_CUR`, and :data:`SEEK_END`. |
| 367 | |
| 368 | * The :attr:`sys.version_info` tuple is now a named tuple:: |
| 369 | |
| 370 | >>> sys.version_info |
| 371 | sys.version_info(major=3, minor=1, micro=0, releaselevel='alpha', serial=2) |
| 372 | |
| 373 | (Contributed by Ross Light; :issue:`4285`.) |
Raymond Hettinger | c4f6d29 | 2009-04-04 12:35:58 +0000 | [diff] [blame] | 374 | |
Raymond Hettinger | daafea3 | 2009-05-15 15:21:33 +0000 | [diff] [blame] | 375 | * The :mod:`nntplib` and :mod:`imaplib` modules now support IPv6. |
Raymond Hettinger | 9dd7476 | 2009-05-15 14:57:35 +0000 | [diff] [blame] | 376 | |
| 377 | (Contributed by Derek Morr; :issue:`1655` and :issue:`1664`.) |
| 378 | |
Raymond Hettinger | 662bad8 | 2009-06-09 21:01:05 +0000 | [diff] [blame] | 379 | * The :mod:`pickle` module has been adapted for better interoperability with |
| 380 | Python 2.x when used with protocol 2 or lower. The reorganization of the |
| 381 | standard library changed the formal reference for many objects. For |
| 382 | example, ``__builtin__.set`` in Python 2 is called ``builtins.set`` in Python |
| 383 | 3. This change cofounded efforts to share data between different versions of |
| 384 | Python. But now when protocol 2 or lower is selected, the pickler will |
| 385 | automatically use the old Python 2 names for both loading and dumping. This |
| 386 | remapping is turned-on by default but can be disabled with the *fix_imports* |
| 387 | option:: |
| 388 | |
| 389 | >>> s = {1, 2, 3} |
| 390 | >>> pickle.dumps(s, protocol=0) |
| 391 | b'c__builtin__\nset\np0\n((lp1\nL1L\naL2L\naL3L\natp2\nRp3\n.' |
| 392 | >>> pickle.dumps(s, protocol=0, fix_imports=False) |
| 393 | b'cbuiltins\nset\np0\n((lp1\nL1L\naL2L\naL3L\natp2\nRp3\n.' |
| 394 | |
| 395 | An unfortunate but unavoidable side-effect of this change is that protocol 2 |
| 396 | pickles produced by Python 3.1 won't be readable with Python 3.0. The latest |
| 397 | pickle protocol, protocol 3, should be used when migrating data between |
| 398 | Python 3.x implementations, as it doesn't attempt to remain compatible with |
| 399 | Python 2.x. |
| 400 | |
| 401 | (Contributed by Alexandre Vassalotti and Antoine Pitrou, :issue:`6137`.) |
| 402 | |
Raymond Hettinger | c4f6d29 | 2009-04-04 12:35:58 +0000 | [diff] [blame] | 403 | * A new module, :mod:`importlib` was added. It provides a complete, portable, |
Raymond Hettinger | 55fc9ce | 2009-04-14 20:45:17 +0000 | [diff] [blame] | 404 | pure Python reference implementation of the :keyword:`import` statement and its |
Benjamin Peterson | 3fa0fb4 | 2009-04-04 12:42:53 +0000 | [diff] [blame] | 405 | counterpart, the :func:`__import__` function. It represents a substantial |
Raymond Hettinger | c4f6d29 | 2009-04-04 12:35:58 +0000 | [diff] [blame] | 406 | step forward in documenting and defining the actions that take place during |
| 407 | imports. |
| 408 | |
| 409 | (Contributed by Brett Cannon.) |
Raymond Hettinger | 1f251a0 | 2009-04-04 10:47:35 +0000 | [diff] [blame] | 410 | |
Antoine Pitrou | b556452 | 2009-03-28 19:45:26 +0000 | [diff] [blame] | 411 | Optimizations |
Raymond Hettinger | 79d0b0e | 2009-04-07 07:11:00 +0000 | [diff] [blame] | 412 | ============= |
Antoine Pitrou | b556452 | 2009-03-28 19:45:26 +0000 | [diff] [blame] | 413 | |
| 414 | Major performance enhancements have been added: |
| 415 | |
| 416 | * The new I/O library (as defined in :pep:`3116`) was mostly written in |
| 417 | Python and quickly proved to be a problematic bottleneck in Python 3.0. |
| 418 | In Python 3.1, the I/O library has been entirely rewritten in C and is |
| 419 | 2 to 20 times faster depending on the task at hand. The pure Python |
| 420 | version is still available for experimentation purposes through |
| 421 | the ``_pyio`` module. |
| 422 | |
| 423 | (Contributed by Amaury Forgeot d'Arc and Antoine Pitrou.) |
| 424 | |
Raymond Hettinger | 8daab40 | 2009-04-04 13:01:19 +0000 | [diff] [blame] | 425 | * Added a heuristic so that tuples and dicts containing only untrackable objects |
Raymond Hettinger | e7ec57d | 2009-04-04 11:08:48 +0000 | [diff] [blame] | 426 | are not tracked by the garbage collector. This can reduce the size of |
| 427 | collections and therefore the garbage collection overhead on long-running |
| 428 | programs, depending on their particular use of datatypes. |
| 429 | |
| 430 | (Contributed by Antoine Pitrou, :issue:`4688`.) |
| 431 | |
Raymond Hettinger | f41857e | 2009-04-04 11:59:00 +0000 | [diff] [blame] | 432 | * Enabling a configure option named ``--with-computed-gotos`` |
| 433 | on compilers that support it (notably: gcc, SunPro, icc), the bytecode |
| 434 | evaluation loop is compiled with a new dispatch mechanism which gives |
Raymond Hettinger | 2a02777 | 2009-04-04 12:46:57 +0000 | [diff] [blame] | 435 | speedups of up to 20%, depending on the system, the compiler, and |
| 436 | the benchmark. |
Raymond Hettinger | f41857e | 2009-04-04 11:59:00 +0000 | [diff] [blame] | 437 | |
Raymond Hettinger | 2a02777 | 2009-04-04 12:46:57 +0000 | [diff] [blame] | 438 | (Contributed by Antoine Pitrou along with a number of other participants, |
| 439 | :issue:`4753`). |
Raymond Hettinger | f41857e | 2009-04-04 11:59:00 +0000 | [diff] [blame] | 440 | |
| 441 | * The decoding of UTF-8, UTF-16 and LATIN-1 is now two to four times |
| 442 | faster. |
| 443 | |
| 444 | (Contributed by Antoine Pitrou and Amaury Forgeot d'Arc, :issue:`4868`.) |
| 445 | |
Raymond Hettinger | bd3da6b | 2009-05-15 16:16:12 +0000 | [diff] [blame] | 446 | * The :mod:`json` module now has a C extension to substantially improve |
Raymond Hettinger | d48ed2e | 2009-05-14 22:48:19 +0000 | [diff] [blame] | 447 | its performance. In addition, the API was modified so that json works |
| 448 | only with :class:`str`, not with :class:`bytes`. That change makes the |
Raymond Hettinger | bd3da6b | 2009-05-15 16:16:12 +0000 | [diff] [blame] | 449 | module closely match the `JSON specification <http://json.org/>`_ |
Raymond Hettinger | d48ed2e | 2009-05-14 22:48:19 +0000 | [diff] [blame] | 450 | which is defined in terms of Unicode. |
Raymond Hettinger | 9dd7476 | 2009-05-15 14:57:35 +0000 | [diff] [blame] | 451 | |
Raymond Hettinger | d48ed2e | 2009-05-14 22:48:19 +0000 | [diff] [blame] | 452 | (Contributed by Bob Ippolito and converted to Py3.1 by Antoine Pitrou |
| 453 | and Benjamin Peterson; :issue:`4136`.) |
Raymond Hettinger | c4f6d29 | 2009-04-04 12:35:58 +0000 | [diff] [blame] | 454 | |
Raymond Hettinger | d48ed2e | 2009-05-14 22:48:19 +0000 | [diff] [blame] | 455 | * Unpickling now interns the attribute names of pickled objects. This saves |
| 456 | memory and allows pickles to be smaller. |
Raymond Hettinger | 9dd7476 | 2009-05-15 14:57:35 +0000 | [diff] [blame] | 457 | |
Raymond Hettinger | d48ed2e | 2009-05-14 22:48:19 +0000 | [diff] [blame] | 458 | (Contributed by Jake McGuire and Antoine Pitrou; :issue:`5084`.) |
Raymond Hettinger | 35a8836 | 2009-04-09 00:08:24 +0000 | [diff] [blame] | 459 | |
Raymond Hettinger | eb81695 | 2009-05-29 01:22:18 +0000 | [diff] [blame] | 460 | IDLE |
| 461 | ==== |
| 462 | |
| 463 | * IDLE's format menu now provides an option to strip trailing whitespace |
| 464 | from a source file. |
| 465 | |
| 466 | (Contributed by Roger D. Serwy; :issue:`5150`.) |
| 467 | |
Raymond Hettinger | 35a8836 | 2009-04-09 00:08:24 +0000 | [diff] [blame] | 468 | Build and C API Changes |
| 469 | ======================= |
| 470 | |
| 471 | Changes to Python's build process and to the C API include: |
Raymond Hettinger | c4f6d29 | 2009-04-04 12:35:58 +0000 | [diff] [blame] | 472 | |
| 473 | * Integers are now stored internally either in base 2**15 or in base |
| 474 | 2**30, the base being determined at build time. Previously, they |
| 475 | were always stored in base 2**15. Using base 2**30 gives |
| 476 | significant performance improvements on 64-bit machines, but |
| 477 | benchmark results on 32-bit machines have been mixed. Therefore, |
| 478 | the default is to use base 2**30 on 64-bit machines and base 2**15 |
| 479 | on 32-bit machines; on Unix, there's a new configure option |
| 480 | ``--enable-big-digits`` that can be used to override this default. |
| 481 | |
| 482 | Apart from the performance improvements this change should be invisible to |
| 483 | end users, with one exception: for testing and debugging purposes there's a |
Raymond Hettinger | 79d0b0e | 2009-04-07 07:11:00 +0000 | [diff] [blame] | 484 | new :attr:`sys.int_info` that provides information about the |
Raymond Hettinger | c4f6d29 | 2009-04-04 12:35:58 +0000 | [diff] [blame] | 485 | internal format, giving the number of bits per digit and the size in bytes |
| 486 | of the C type used to store each digit:: |
| 487 | |
| 488 | >>> import sys |
| 489 | >>> sys.int_info |
| 490 | sys.int_info(bits_per_digit=30, sizeof_digit=4) |
| 491 | |
| 492 | (Contributed by Mark Dickinson; :issue:`4258`.) |
Antoine Pitrou | b556452 | 2009-03-28 19:45:26 +0000 | [diff] [blame] | 493 | |
Raymond Hettinger | 35a8836 | 2009-04-09 00:08:24 +0000 | [diff] [blame] | 494 | * The :cfunc:`PyLong_AsUnsignedLongLong()` function now handles a negative |
| 495 | *pylong* by raising :exc:`OverflowError` instead of :exc:`TypeError`. |
| 496 | |
| 497 | (Contributed by Mark Dickinson and Lisandro Dalcrin; :issue:`5175`.) |
| 498 | |
| 499 | * Deprecated :cfunc:`PyNumber_Int`. Use :cfunc:`PyNumber_Long` instead. |
| 500 | |
Raymond Hettinger | be3a821 | 2009-04-09 00:18:29 +0000 | [diff] [blame] | 501 | (Contributed by Mark Dickinson; :issue:`4910`.) |
Raymond Hettinger | 1a6b73d | 2009-04-17 20:55:52 +0000 | [diff] [blame] | 502 | |
Raymond Hettinger | bd3da6b | 2009-05-15 16:16:12 +0000 | [diff] [blame] | 503 | * Added a new :cfunc:`PyOS_string_to_double` function to replace the |
| 504 | deprecated functions :cfunc:`PyOS_ascii_strtod` and :cfunc:`PyOS_ascii_atof`. |
| 505 | |
| 506 | (Contributed by Mark Dickinson; :issue:`5914`.) |
| 507 | |
Raymond Hettinger | 4f20a74 | 2009-05-14 23:17:38 +0000 | [diff] [blame] | 508 | * Added :ctype:`PyCapsule` as a replacement for the :ctype:`PyCObject` API. |
| 509 | The principal difference is that the new type has a well defined interface |
| 510 | for passing typing safety information and a less complicated signature |
Raymond Hettinger | 9dd7476 | 2009-05-15 14:57:35 +0000 | [diff] [blame] | 511 | for calling a destructor. The old type had a problematic API and is now |
| 512 | deprecated. |
Raymond Hettinger | 4f20a74 | 2009-05-14 23:17:38 +0000 | [diff] [blame] | 513 | |
| 514 | (Contributed by Larry Hastings; :issue:`5630`.) |
| 515 | |
Raymond Hettinger | 1a6b73d | 2009-04-17 20:55:52 +0000 | [diff] [blame] | 516 | Porting to Python 3.1 |
| 517 | ===================== |
| 518 | |
| 519 | This section lists previously described changes and other bugfixes |
| 520 | that may require changes to your code: |
| 521 | |
| 522 | * The new floating point string representations can break existing doctests. |
| 523 | For example:: |
| 524 | |
| 525 | def e(): |
| 526 | '''Compute the base of natural logarithms. |
| 527 | |
| 528 | >>> e() |
| 529 | 2.7182818284590451 |
| 530 | |
| 531 | ''' |
| 532 | return sum(1/math.factorial(x) for x in reversed(range(30))) |
| 533 | |
| 534 | doctest.testmod() |
| 535 | |
| 536 | ********************************************************************** |
| 537 | Failed example: |
| 538 | e() |
| 539 | Expected: |
| 540 | 2.7182818284590451 |
| 541 | Got: |
| 542 | 2.718281828459045 |
| 543 | ********************************************************************** |
Raymond Hettinger | 5b4a54c | 2009-06-09 21:07:46 +0000 | [diff] [blame^] | 544 | |
| 545 | * The automatic name remapping in the pickle module for protocol 2 or lower can |
| 546 | make Python 3.1 pickles unreadable in Python 3.0. One solution is to use |
| 547 | protocol 3. Another solution is to set the *fix_imports* option to **False**. |
| 548 | See the discussion above for more details. |