blob: 64ae1c1936587b0748fe5c0019ac0ffeb1c47695 [file] [log] [blame]
Georg Brandlc47408a2008-12-04 18:44:53 +00001****************************
Georg Brandl48310cd2009-01-03 21:18:54 +00002 What's New In Python 3.1
Georg Brandlc47408a2008-12-04 18:44:53 +00003****************************
4
Raymond Hettinger1f251a02009-04-04 10:47:35 +00005:Author: Raymond Hettinger
Benjamin Peterson34328292008-12-05 03:05:29 +00006:Release: |release|
7:Date: |today|
Georg Brandlc47408a2008-12-04 18:44:53 +00008
9.. $Id$
10 Rules for maintenance:
Georg Brandl48310cd2009-01-03 21:18:54 +000011
Georg Brandlc47408a2008-12-04 18:44:53 +000012 * Anyone can add text to this document. Do not spend very much time
13 on the wording of your changes, because your text will probably
14 get rewritten to some degree.
Georg Brandl48310cd2009-01-03 21:18:54 +000015
Georg Brandlc47408a2008-12-04 18:44:53 +000016 * The maintainer will go through Misc/NEWS periodically and add
17 changes; it's therefore more important to add your changes to
Raymond Hettingerb9b33192009-04-08 09:38:32 +000018 Misc/NEWS than to this file.
Georg Brandl48310cd2009-01-03 21:18:54 +000019
Georg Brandlc47408a2008-12-04 18:44:53 +000020 * This is not a complete list of every single change; completeness
21 is the purpose of Misc/NEWS. Some changes I consider too small
22 or esoteric to include. If such a change is added to the text,
23 I'll just remove it. (This is another reason you shouldn't spend
24 too much time on writing your addition.)
Georg Brandl48310cd2009-01-03 21:18:54 +000025
Georg Brandlc47408a2008-12-04 18:44:53 +000026 * If you want to draw your new text to the attention of the
27 maintainer, add 'XXX' to the beginning of the paragraph or
28 section.
Georg Brandl48310cd2009-01-03 21:18:54 +000029
Georg Brandlc47408a2008-12-04 18:44:53 +000030 * It's OK to just add a fragmentary note about a change. For
31 example: "XXX Describe the transmogrify() function added to the
32 socket module." The maintainer will research the change and
33 write the necessary text.
Georg Brandl48310cd2009-01-03 21:18:54 +000034
Georg Brandlc47408a2008-12-04 18:44:53 +000035 * You can comment out your additions if you like, but it's not
36 necessary (especially when a final release is some months away).
Georg Brandl48310cd2009-01-03 21:18:54 +000037
Georg Brandlc47408a2008-12-04 18:44:53 +000038 * Credit the author of a patch or bugfix. Just the name is
Raymond Hettingerb9b33192009-04-08 09:38:32 +000039 sufficient; the e-mail address isn't necessary.
Georg Brandl48310cd2009-01-03 21:18:54 +000040
Georg Brandlc47408a2008-12-04 18:44:53 +000041 * It's helpful to add the bug/patch number as a comment:
Georg Brandl48310cd2009-01-03 21:18:54 +000042
Georg Brandlc47408a2008-12-04 18:44:53 +000043 % Patch 12345
44 XXX Describe the transmogrify() function added to the socket
45 module.
46 (Contributed by P.Y. Developer.)
Georg Brandl48310cd2009-01-03 21:18:54 +000047
Georg Brandlc47408a2008-12-04 18:44:53 +000048 This saves the maintainer the effort of going through the SVN log
Raymond Hettingerb9b33192009-04-08 09:38:32 +000049 when researching a change.
Georg Brandlc47408a2008-12-04 18:44:53 +000050
51This article explains the new features in Python 3.1, compared to 3.0.
52
Georg Brandlc47408a2008-12-04 18:44:53 +000053
Raymond Hettinger1f251a02009-04-04 10:47:35 +000054PEP 372: Ordered Dictionaries
55=============================
56
57Regular Python dictionaries iterate over key/value pairs in arbitrary order.
58Over the years, a number of authors have written alternative implementations
59that remember the order that the keys were originally inserted. Based on
Raymond Hettingerd621dd72009-04-14 08:16:50 +000060the experiences from those implementations, a new
61:class:`collections.OrderedDict` class has been introduced.
Raymond Hettinger1f251a02009-04-04 10:47:35 +000062
63The OrderedDict API is substantially the same as regular dictionaries
64but will iterate over keys and values in a guaranteed order depending on
65when a key was first inserted. If a new entry overwrites an existing entry,
66the original insertion position is left unchanged. Deleting an entry and
67reinserting it will move it to the end.
68
69The standard library now supports use of ordered dictionaries in several
Raymond Hettinger7f5d7462009-04-14 08:05:31 +000070modules. The :mod:`configparser` module uses them by default. This lets
Raymond Hettinger1f251a02009-04-04 10:47:35 +000071configuration files be read, modified, and then written back in their original
Raymond Hettingerd621dd72009-04-14 08:16:50 +000072order. The *_asdict()* method for :func:`collections.namedtuple` now
Raymond Hettingerc4f6d292009-04-04 12:35:58 +000073returns an ordered dictionary with the values appearing in the same order as
74the underlying tuple indicies. The :mod:`json` module is being built-out with
75an *object_pairs_hook* to allow OrderedDicts to be built by the decoder.
Raymond Hettinger347396a2009-04-07 23:10:59 +000076Support was also added for third-party tools like `PyYAML <http://pyyaml.org/>`_.
Raymond Hettinger1f251a02009-04-04 10:47:35 +000077
78.. seealso::
79
80 :pep:`372` - Ordered Dictionaries
Raymond Hettingerf84dfe52009-04-04 13:13:56 +000081 PEP written by Armin Ronacher and Raymond Hettinger. Implementation
82 written by Raymond Hettinger.
Raymond Hettinger1f251a02009-04-04 10:47:35 +000083
Raymond Hettingerb9b33192009-04-08 09:38:32 +000084
Raymond Hettinger1f251a02009-04-04 10:47:35 +000085PEP 378: Format Specifier for Thousands Separator
86=================================================
87
Georg Brandlc4a55fc2010-02-06 18:46:57 +000088The built-in :func:`format` function and the :meth:`str.format` method use
Raymond Hettinger1f251a02009-04-04 10:47:35 +000089a mini-language that now includes a simple, non-locale aware way to format
90a number with a thousands separator. That provides a way to humanize a
91program's output, improving its professional appearance and readability::
92
Raymond Hettinger0422e142009-04-17 18:58:06 +000093 >>> format(1234567, ',d')
94 '1,234,567'
95 >>> format(1234567.89, ',.2f')
96 '1,234,567.89'
Raymond Hettingerd48ed2e2009-05-14 22:48:19 +000097 >>> format(12345.6 + 8901234.12j, ',f')
98 '12,345.600000+8,901,234.120000j'
Raymond Hettinger1f251a02009-04-04 10:47:35 +000099 >>> format(Decimal('1234567.89'), ',f')
100 '1,234,567.89'
101
Raymond Hettingerd48ed2e2009-05-14 22:48:19 +0000102The supported types are :class:`int`, :class:`float`, :class:`complex`
103and :class:`decimal.Decimal`.
Raymond Hettingerc548b6e2009-04-17 10:09:27 +0000104
Raymond Hettinger1f251a02009-04-04 10:47:35 +0000105Discussions are underway about how to specify alternative separators
Raymond Hettingerc4f6d292009-04-04 12:35:58 +0000106like dots, spaces, apostrophes, or underscores. Locale-aware applications
107should use the existing *n* format specifier which already has some support
108for thousands separators.
Raymond Hettinger1f251a02009-04-04 10:47:35 +0000109
110.. seealso::
111
112 :pep:`378` - Format Specifier for Thousands Separator
Raymond Hettinger55fc9ce2009-04-14 20:45:17 +0000113 PEP written by Raymond Hettinger and implemented by Eric Smith and
Raymond Hettinger1f251a02009-04-04 10:47:35 +0000114 Mark Dickinson.
115
116
Mark Dickinson54bc1ec2008-12-17 16:19:07 +0000117Other Language Changes
118======================
119
120Some smaller changes made to the core Python language are:
121
Andrew M. Kuchling7718e5e2010-02-22 15:24:43 +0000122* Directories and zip archives containing a :file:`__main__.py`
123 file can now be executed directly by passing their name to the
124 interpreter. The directory/zipfile is automatically inserted as the
125 first entry in sys.path. (Suggestion and initial patch by Andy Chu;
126 revised patch by Phillip J. Eby and Nick Coghlan; :issue:`1739468`.)
127
Mark Dickinson54bc1ec2008-12-17 16:19:07 +0000128* The :func:`int` type gained a ``bit_length`` method that returns the
129 number of bits necessary to represent its argument in binary::
130
131 >>> n = 37
132 >>> bin(37)
133 '0b100101'
134 >>> n.bit_length()
135 6
136 >>> n = 2**123-1
137 >>> n.bit_length()
138 123
139 >>> (n+1).bit_length()
140 124
141
Raymond Hettingerc4f6d292009-04-04 12:35:58 +0000142 (Contributed by Fredrik Johansson, Victor Stinner, Raymond Hettinger,
143 and Mark Dickinson; :issue:`3439`.)
Mark Dickinson54bc1ec2008-12-17 16:19:07 +0000144
Raymond Hettinger79d0b0e2009-04-07 07:11:00 +0000145* The fields in :func:`format` strings can now be automatically
146 numbered::
147
148 >>> 'Sir {} of {}'.format('Gallahad', 'Camelot')
149 'Sir Gallahad of Camelot'
150
151 Formerly, the string would have required numbered fields such as:
152 ``'Sir {0} of {1}'``.
153
154 (Contributed by Eric Smith; :issue:`5237`.)
155
Raymond Hettingerbd3da6b2009-05-15 16:16:12 +0000156* The :func:`string.maketrans` function is deprecated and is replaced by new
157 static methods, :meth:`bytes.maketrans` and :meth:`bytearray.maketrans`.
158 This change solves the confusion around which types were supported by the
159 :mod:`string` module. Now, :class:`str`, :class:`bytes`, and
160 :class:`bytearray` each have their own **maketrans** and **translate**
161 methods with intermediate translation tables of the appropriate type.
162
163 (Contributed by Georg Brandl; :issue:`5675`.)
164
Raymond Hettinger81c0dce2009-05-27 09:12:18 +0000165* The syntax of the :keyword:`with` statement now allows multiple context
166 managers in a single statement::
167
168 >>> with open('mylog.txt') as infile, open('a.out', 'w') as outfile:
169 ... for line in infile:
170 ... if '<critical>' in line:
171 ... outfile.write(line)
172
Raymond Hettinger91e3b9d2009-05-28 22:20:03 +0000173 With the new syntax, the :func:`contextlib.nested` function is no longer
Raymond Hettinger323ee0c2009-05-28 22:49:33 +0000174 needed and is now deprecated.
Raymond Hettinger91e3b9d2009-05-28 22:20:03 +0000175
Georg Brandled23ab62009-05-27 19:46:38 +0000176 (Contributed by Georg Brandl and Mattias Brändström;
Raymond Hettinger81c0dce2009-05-27 09:12:18 +0000177 `appspot issue 53094 <http://codereview.appspot.com/53094>`_.)
178
Raymond Hettinger79d0b0e2009-04-07 07:11:00 +0000179* ``round(x, n)`` now returns an integer if *x* is an integer.
180 Previously it returned a float::
181
182 >>> round(1123, -2)
183 1100
184
185 (Contributed by Mark Dickinson; :issue:`4707`.)
186
Raymond Hettinger8648e502009-04-17 00:11:54 +0000187* Python now uses David Gay's algorithm for finding the shortest floating
188 point representation that doesn't change its value. This should help
Eric Smith86c06bc2009-04-17 12:33:24 +0000189 mitigate some of the confusion surrounding binary floating point
Raymond Hettinger8648e502009-04-17 00:11:54 +0000190 numbers.
191
192 The significance is easily seen with a number like ``1.1`` which does not
193 have an exact equivalent in binary floating point. Since there is no exact
Raymond Hettingerfee346b2009-04-17 09:45:19 +0000194 equivalent, an expression like ``float('1.1')`` evaluates to the nearest
Raymond Hettinger8648e502009-04-17 00:11:54 +0000195 representable value which is ``0x1.199999999999ap+0`` in hex or
196 ``1.100000000000000088817841970012523233890533447265625`` in decimal. That
197 nearest value was and still is used in subsequent floating point
198 calculations.
199
200 What is new is how the number gets displayed. Formerly, Python used a
201 simple approach. The value of ``repr(1.1)`` was computed as ``format(1.1,
Raymond Hettingerfee346b2009-04-17 09:45:19 +0000202 '.17g')`` which evaluated to ``'1.1000000000000001'``. The advantage of
Raymond Hettinger8648e502009-04-17 00:11:54 +0000203 using 17 digits was that it relied on IEEE-754 guarantees to assure that
204 ``eval(repr(1.1))`` would round-trip exactly to its original value. The
205 disadvantage is that many people found the output to be confusing (mistaking
206 intrinsic limitations of binary floating point representation as being a
207 problem with Python itself).
208
Raymond Hettingerfee346b2009-04-17 09:45:19 +0000209 The new algorithm for ``repr(1.1)`` is smarter and returns ``'1.1'``.
Raymond Hettinger8648e502009-04-17 00:11:54 +0000210 Effectively, it searches all equivalent string representations (ones that
Raymond Hettingerfee346b2009-04-17 09:45:19 +0000211 get stored with the same underlying float value) and returns the shortest
Raymond Hettinger8648e502009-04-17 00:11:54 +0000212 representation.
213
214 The new algorithm tends to emit cleaner representations when possible, but
215 it does not change the underlying values. So, it is still the case that
216 ``1.1 + 2.2 != 3.3`` even though the representations may suggest otherwise.
217
218 The new algorithm depends on certain features in the underlying floating
219 point implementation. If the required features are not found, the old
220 algorithm will continue to be used. Also, the text pickle protocols
221 assure cross-platform portability by using the old algorithm.
222
223 (Contributed by Eric Smith and Mark Dickinson; :issue:`1580`)
Raymond Hettinger79d0b0e2009-04-07 07:11:00 +0000224
225New, Improved, and Deprecated Modules
226=====================================
227
Raymond Hettinger1f251a02009-04-04 10:47:35 +0000228* Added a :class:`collections.Counter` class to support convenient
229 counting of unique items in a sequence or iterable::
230
231 >>> Counter(['red', 'blue', 'red', 'green', 'blue', 'blue'])
232 Counter({'blue': 3, 'red': 2, 'green': 1})
233
234 (Contributed by Raymond Hettinger; :issue:`1696199`.)
235
Raymond Hettingerc1bd49a2009-04-06 23:11:08 +0000236* Added a new module, :mod:`tkinter.ttk` for access to the Tk themed widget set.
Raymond Hettingerdbd51b52009-04-06 22:45:52 +0000237 The basic idea of ttk is to separate, to the extent possible, the code
Raymond Hettingerf84dfe52009-04-04 13:13:56 +0000238 implementing a widget's behavior from the code implementing its appearance.
239
Raymond Hettingerd1b3de32009-04-08 00:09:26 +0000240 (Contributed by Guilherme Polo; :issue:`2983`.)
Raymond Hettingerf84dfe52009-04-04 13:13:56 +0000241
Raymond Hettingerf4cc2c42009-04-06 22:39:03 +0000242* The :class:`gzip.GzipFile` and :class:`bz2.BZ2File` classes now support
243 the context manager protocol::
Raymond Hettinger1f251a02009-04-04 10:47:35 +0000244
Raymond Hettingerf4cc2c42009-04-06 22:39:03 +0000245 >>> # Automatically close file after writing
246 >>> with gzip.GzipFile(filename, "wb") as f:
247 ... f.write(b"xxx")
248
249 (Contributed by Antoine Pitrou.)
Raymond Hettinger1f251a02009-04-04 10:47:35 +0000250
Raymond Hettingerbf1537c2009-04-09 22:34:46 +0000251* The :mod:`decimal` module now supports methods for creating a
Raymond Hettinger76efa2b2009-04-06 23:11:47 +0000252 decimal object from a binary :class:`float`. The conversion is
Raymond Hettinger1f251a02009-04-04 10:47:35 +0000253 exact but can sometimes be surprising::
254
255 >>> Decimal.from_float(1.1)
256 Decimal('1.100000000000000088817841970012523233890533447265625')
257
258 The long decimal result shows the actual binary fraction being
259 stored for *1.1*. The fraction has many digits because *1.1* cannot
260 be exactly represented in binary.
261
262 (Contributed by Raymond Hettinger and Mark Dickinson.)
263
Raymond Hettingere7ec57d2009-04-04 11:08:48 +0000264* The :mod:`itertools` module grew two new functions. The
265 :func:`itertools.combinations_with_replacement` function is one of
266 four for generating combinatorics including permutations and Cartesian
267 products. The :func:`itertools.compress` function mimics its namesake
268 from APL. Also, the existing :func:`itertools.count` function now has
269 an optional *step* argument and can accept any type of counting
270 sequence including :class:`fractions.Fraction` and
Raymond Hettinger8d97ccb2009-04-06 17:55:05 +0000271 :class:`decimal.Decimal`::
272
273 >>> [p+q for p,q in combinations_with_replacement('LOVE', 2)]
274 ['LL', 'LO', 'LV', 'LE', 'OO', 'OV', 'OE', 'VV', 'VE', 'EE']
275
276 >>> list(compress(data=range(10), selectors=[0,0,1,1,0,1,0,1,0,0]))
277 [2, 3, 5, 7]
278
279 >>> c = count(start=Fraction(1,2), step=Fraction(1,6))
Raymond Hettingerbd3da6b2009-05-15 16:16:12 +0000280 >>> [next(c), next(c), next(c), next(c)]
281 [Fraction(1, 2), Fraction(2, 3), Fraction(5, 6), Fraction(1, 1)]
Raymond Hettingere7ec57d2009-04-04 11:08:48 +0000282
283 (Contributed by Raymond Hettinger.)
284
Raymond Hettingerf41857e2009-04-04 11:59:00 +0000285* :func:`collections.namedtuple` now supports a keyword argument
286 *rename* which lets invalid fieldnames be automatically converted to
287 positional names in the form _0, _1, etc. This is useful when
288 the field names are being created by an external source such as a
Raymond Hettinger4c0b1e42009-04-08 07:49:03 +0000289 CSV header, SQL field list, or user input::
290
Raymond Hettinger2a39e0f2009-04-08 22:50:09 +0000291 >>> query = input()
292 SELECT region, dept, count(*) FROM main GROUPBY region, dept
Raymond Hettinger4c0b1e42009-04-08 07:49:03 +0000293
294 >>> cursor.execute(query)
295 >>> query_fields = [desc[0] for desc in cursor.description]
296 >>> UserQuery = namedtuple('UserQuery', query_fields, rename=True)
297 >>> pprint.pprint([UserQuery(*row) for row in cursor])
298 [UserQuery(region='South', dept='Shipping', _2=185),
299 UserQuery(region='North', dept='Accounting', _2=37),
300 UserQuery(region='West', dept='Sales', _2=419)]
Raymond Hettingerf41857e2009-04-04 11:59:00 +0000301
302 (Contributed by Raymond Hettinger; :issue:`1818`.)
303
Raymond Hettingerf41857e2009-04-04 11:59:00 +0000304* The :func:`re.sub`, :func:`re.subn` and :func:`re.split` functions now
305 accept a flags parameter.
306
307 (Contributed by Gregory Smith.)
308
Raymond Hettingerd621dd72009-04-14 08:16:50 +0000309* The :mod:`logging` module now implements a simple :class:`logging.NullHandler`
Raymond Hettinger35a88362009-04-09 00:08:24 +0000310 class for applications that are not using logging but are calling
311 library code that does. Setting-up a null handler will suppress
Raymond Hettingerd621dd72009-04-14 08:16:50 +0000312 spurious warnings such as "No handlers could be found for logger foo"::
Raymond Hettinger35a88362009-04-09 00:08:24 +0000313
314 >>> h = logging.NullHandler()
315 >>> logging.getLogger("foo").addHandler(h)
316
Raymond Hettingerd621dd72009-04-14 08:16:50 +0000317 (Contributed by Vinay Sajip; :issue:`4384`).
Raymond Hettinger35a88362009-04-09 00:08:24 +0000318
Raymond Hettingerf41857e2009-04-04 11:59:00 +0000319* The :mod:`runpy` module which supports the ``-m`` command line switch
320 now supports the execution of packages by looking for and executing
321 a ``__main__`` submodule when a package name is supplied.
322
323 (Contributed by Andi Vajda; :issue:`4195`.)
324
325* The :mod:`pdb` module can now access and display source code loaded via
326 :mod:`zipimport` (or any other conformant :pep:`302` loader).
327
328 (Contributed by Alexander Belopolsky; :issue:`4201`.)
329
330* :class:`functools.partial` objects can now be pickled.
331
332 (Suggested by Antoine Pitrou and Jesse Noller. Implemented by
Georg Brandl6faee4e2010-09-21 14:48:28 +0000333 Jack Diederich; :issue:`5228`.)
Raymond Hettingerf41857e2009-04-04 11:59:00 +0000334
Raymond Hettinger8e330512009-04-04 13:20:55 +0000335* Add :mod:`pydoc` help topics for symbols so that ``help('@')``
336 works as expected in the interactive environment.
337
338 (Contributed by David Laban; :issue:`4739`.)
339
Raymond Hettingerc4f6d292009-04-04 12:35:58 +0000340* The :mod:`unittest` module now supports skipping individual tests or classes
341 of tests. And it supports marking a test as a expected failure, a test that
Raymond Hettinger8daab402009-04-04 13:01:19 +0000342 is known to be broken, but shouldn't be counted as a failure on a
Raymond Hettinger8d97ccb2009-04-06 17:55:05 +0000343 TestResult::
344
345 class TestGizmo(unittest.TestCase):
346
347 @unittest.skipUnless(sys.platform.startswith("win"), "requires Windows")
348 def test_gizmo_on_windows(self):
349 ...
350
351 @unittest.expectedFailure
352 def test_gimzo_without_required_library(self):
353 ...
Raymond Hettingerf41857e2009-04-04 11:59:00 +0000354
Raymond Hettinger55fc9ce2009-04-14 20:45:17 +0000355 Also, tests for exceptions have been builtout to work with context managers
356 using the :keyword:`with` statement::
Raymond Hettinger35a88362009-04-09 00:08:24 +0000357
358 def test_division_by_zero(self):
359 with self.assertRaises(ZeroDivisionError):
360 x / 0
361
362 In addition, several new assertion methods were added including
363 :func:`assertSetEqual`, :func:`assertDictEqual`,
364 :func:`assertDictContainsSubset`, :func:`assertListEqual`,
365 :func:`assertTupleEqual`, :func:`assertSequenceEqual`,
366 :func:`assertRaisesRegexp`, :func:`assertIsNone`,
Michael Foord5859b862009-04-25 20:47:43 +0000367 and :func:`assertIsNotNone`.
Raymond Hettinger35a88362009-04-09 00:08:24 +0000368
369 (Contributed by Benjamin Peterson and Antoine Pitrou.)
370
Raymond Hettingerbe3a8212009-04-09 00:18:29 +0000371* The :mod:`io` module has three new constants for the :meth:`seek`
Raymond Hettinger35a88362009-04-09 00:08:24 +0000372 method :data:`SEEK_SET`, :data:`SEEK_CUR`, and :data:`SEEK_END`.
373
374* The :attr:`sys.version_info` tuple is now a named tuple::
375
376 >>> sys.version_info
377 sys.version_info(major=3, minor=1, micro=0, releaselevel='alpha', serial=2)
378
379 (Contributed by Ross Light; :issue:`4285`.)
Raymond Hettingerc4f6d292009-04-04 12:35:58 +0000380
Raymond Hettingerdaafea32009-05-15 15:21:33 +0000381* The :mod:`nntplib` and :mod:`imaplib` modules now support IPv6.
Raymond Hettinger9dd74762009-05-15 14:57:35 +0000382
383 (Contributed by Derek Morr; :issue:`1655` and :issue:`1664`.)
384
Raymond Hettinger662bad82009-06-09 21:01:05 +0000385* The :mod:`pickle` module has been adapted for better interoperability with
386 Python 2.x when used with protocol 2 or lower. The reorganization of the
387 standard library changed the formal reference for many objects. For
388 example, ``__builtin__.set`` in Python 2 is called ``builtins.set`` in Python
Raymond Hettinger5c963082009-06-10 15:38:52 +0000389 3. This change confounded efforts to share data between different versions of
Raymond Hettinger662bad82009-06-09 21:01:05 +0000390 Python. But now when protocol 2 or lower is selected, the pickler will
391 automatically use the old Python 2 names for both loading and dumping. This
392 remapping is turned-on by default but can be disabled with the *fix_imports*
393 option::
394
395 >>> s = {1, 2, 3}
396 >>> pickle.dumps(s, protocol=0)
397 b'c__builtin__\nset\np0\n((lp1\nL1L\naL2L\naL3L\natp2\nRp3\n.'
398 >>> pickle.dumps(s, protocol=0, fix_imports=False)
399 b'cbuiltins\nset\np0\n((lp1\nL1L\naL2L\naL3L\natp2\nRp3\n.'
400
401 An unfortunate but unavoidable side-effect of this change is that protocol 2
402 pickles produced by Python 3.1 won't be readable with Python 3.0. The latest
403 pickle protocol, protocol 3, should be used when migrating data between
404 Python 3.x implementations, as it doesn't attempt to remain compatible with
405 Python 2.x.
406
407 (Contributed by Alexandre Vassalotti and Antoine Pitrou, :issue:`6137`.)
408
Raymond Hettingerc4f6d292009-04-04 12:35:58 +0000409* A new module, :mod:`importlib` was added. It provides a complete, portable,
Raymond Hettinger55fc9ce2009-04-14 20:45:17 +0000410 pure Python reference implementation of the :keyword:`import` statement and its
Benjamin Peterson3fa0fb42009-04-04 12:42:53 +0000411 counterpart, the :func:`__import__` function. It represents a substantial
Raymond Hettingerc4f6d292009-04-04 12:35:58 +0000412 step forward in documenting and defining the actions that take place during
413 imports.
414
415 (Contributed by Brett Cannon.)
Raymond Hettinger1f251a02009-04-04 10:47:35 +0000416
Antoine Pitroub5564522009-03-28 19:45:26 +0000417Optimizations
Raymond Hettinger79d0b0e2009-04-07 07:11:00 +0000418=============
Antoine Pitroub5564522009-03-28 19:45:26 +0000419
420Major performance enhancements have been added:
421
422* The new I/O library (as defined in :pep:`3116`) was mostly written in
423 Python and quickly proved to be a problematic bottleneck in Python 3.0.
424 In Python 3.1, the I/O library has been entirely rewritten in C and is
425 2 to 20 times faster depending on the task at hand. The pure Python
426 version is still available for experimentation purposes through
427 the ``_pyio`` module.
428
429 (Contributed by Amaury Forgeot d'Arc and Antoine Pitrou.)
430
Raymond Hettinger8daab402009-04-04 13:01:19 +0000431* Added a heuristic so that tuples and dicts containing only untrackable objects
Raymond Hettingere7ec57d2009-04-04 11:08:48 +0000432 are not tracked by the garbage collector. This can reduce the size of
433 collections and therefore the garbage collection overhead on long-running
434 programs, depending on their particular use of datatypes.
435
436 (Contributed by Antoine Pitrou, :issue:`4688`.)
437
Raymond Hettingerf41857e2009-04-04 11:59:00 +0000438* Enabling a configure option named ``--with-computed-gotos``
439 on compilers that support it (notably: gcc, SunPro, icc), the bytecode
440 evaluation loop is compiled with a new dispatch mechanism which gives
Raymond Hettinger2a027772009-04-04 12:46:57 +0000441 speedups of up to 20%, depending on the system, the compiler, and
442 the benchmark.
Raymond Hettingerf41857e2009-04-04 11:59:00 +0000443
Raymond Hettinger2a027772009-04-04 12:46:57 +0000444 (Contributed by Antoine Pitrou along with a number of other participants,
445 :issue:`4753`).
Raymond Hettingerf41857e2009-04-04 11:59:00 +0000446
447* The decoding of UTF-8, UTF-16 and LATIN-1 is now two to four times
448 faster.
449
450 (Contributed by Antoine Pitrou and Amaury Forgeot d'Arc, :issue:`4868`.)
451
Raymond Hettingerbd3da6b2009-05-15 16:16:12 +0000452* The :mod:`json` module now has a C extension to substantially improve
Raymond Hettingerd48ed2e2009-05-14 22:48:19 +0000453 its performance. In addition, the API was modified so that json works
454 only with :class:`str`, not with :class:`bytes`. That change makes the
Raymond Hettingerbd3da6b2009-05-15 16:16:12 +0000455 module closely match the `JSON specification <http://json.org/>`_
Raymond Hettingerd48ed2e2009-05-14 22:48:19 +0000456 which is defined in terms of Unicode.
Raymond Hettinger9dd74762009-05-15 14:57:35 +0000457
Raymond Hettingerd48ed2e2009-05-14 22:48:19 +0000458 (Contributed by Bob Ippolito and converted to Py3.1 by Antoine Pitrou
459 and Benjamin Peterson; :issue:`4136`.)
Raymond Hettingerc4f6d292009-04-04 12:35:58 +0000460
Raymond Hettingerd48ed2e2009-05-14 22:48:19 +0000461* Unpickling now interns the attribute names of pickled objects. This saves
462 memory and allows pickles to be smaller.
Raymond Hettinger9dd74762009-05-15 14:57:35 +0000463
Raymond Hettingerd48ed2e2009-05-14 22:48:19 +0000464 (Contributed by Jake McGuire and Antoine Pitrou; :issue:`5084`.)
Raymond Hettinger35a88362009-04-09 00:08:24 +0000465
Raymond Hettingereb816952009-05-29 01:22:18 +0000466IDLE
467====
468
469* IDLE's format menu now provides an option to strip trailing whitespace
470 from a source file.
471
472 (Contributed by Roger D. Serwy; :issue:`5150`.)
473
Raymond Hettinger35a88362009-04-09 00:08:24 +0000474Build and C API Changes
475=======================
476
477Changes to Python's build process and to the C API include:
Raymond Hettingerc4f6d292009-04-04 12:35:58 +0000478
479* Integers are now stored internally either in base 2**15 or in base
480 2**30, the base being determined at build time. Previously, they
481 were always stored in base 2**15. Using base 2**30 gives
482 significant performance improvements on 64-bit machines, but
483 benchmark results on 32-bit machines have been mixed. Therefore,
484 the default is to use base 2**30 on 64-bit machines and base 2**15
485 on 32-bit machines; on Unix, there's a new configure option
486 ``--enable-big-digits`` that can be used to override this default.
487
488 Apart from the performance improvements this change should be invisible to
489 end users, with one exception: for testing and debugging purposes there's a
Raymond Hettinger79d0b0e2009-04-07 07:11:00 +0000490 new :attr:`sys.int_info` that provides information about the
Raymond Hettingerc4f6d292009-04-04 12:35:58 +0000491 internal format, giving the number of bits per digit and the size in bytes
492 of the C type used to store each digit::
493
494 >>> import sys
495 >>> sys.int_info
496 sys.int_info(bits_per_digit=30, sizeof_digit=4)
497
498 (Contributed by Mark Dickinson; :issue:`4258`.)
Antoine Pitroub5564522009-03-28 19:45:26 +0000499
Georg Brandl60203b42010-10-06 10:11:56 +0000500* The :c:func:`PyLong_AsUnsignedLongLong()` function now handles a negative
Raymond Hettinger35a88362009-04-09 00:08:24 +0000501 *pylong* by raising :exc:`OverflowError` instead of :exc:`TypeError`.
502
503 (Contributed by Mark Dickinson and Lisandro Dalcrin; :issue:`5175`.)
504
Georg Brandl60203b42010-10-06 10:11:56 +0000505* Deprecated :c:func:`PyNumber_Int`. Use :c:func:`PyNumber_Long` instead.
Raymond Hettinger35a88362009-04-09 00:08:24 +0000506
Raymond Hettingerbe3a8212009-04-09 00:18:29 +0000507 (Contributed by Mark Dickinson; :issue:`4910`.)
Raymond Hettinger1a6b73d2009-04-17 20:55:52 +0000508
Georg Brandl60203b42010-10-06 10:11:56 +0000509* Added a new :c:func:`PyOS_string_to_double` function to replace the
510 deprecated functions :c:func:`PyOS_ascii_strtod` and :c:func:`PyOS_ascii_atof`.
Raymond Hettingerbd3da6b2009-05-15 16:16:12 +0000511
512 (Contributed by Mark Dickinson; :issue:`5914`.)
513
Georg Brandl60203b42010-10-06 10:11:56 +0000514* Added :c:type:`PyCapsule` as a replacement for the :c:type:`PyCObject` API.
Raymond Hettinger4f20a742009-05-14 23:17:38 +0000515 The principal difference is that the new type has a well defined interface
516 for passing typing safety information and a less complicated signature
Raymond Hettinger9dd74762009-05-15 14:57:35 +0000517 for calling a destructor. The old type had a problematic API and is now
518 deprecated.
Raymond Hettinger4f20a742009-05-14 23:17:38 +0000519
520 (Contributed by Larry Hastings; :issue:`5630`.)
521
Raymond Hettinger1a6b73d2009-04-17 20:55:52 +0000522Porting to Python 3.1
523=====================
524
525This section lists previously described changes and other bugfixes
526that may require changes to your code:
527
528* The new floating point string representations can break existing doctests.
529 For example::
530
531 def e():
532 '''Compute the base of natural logarithms.
533
534 >>> e()
535 2.7182818284590451
536
537 '''
538 return sum(1/math.factorial(x) for x in reversed(range(30)))
539
540 doctest.testmod()
541
542 **********************************************************************
543 Failed example:
544 e()
545 Expected:
546 2.7182818284590451
547 Got:
548 2.718281828459045
549 **********************************************************************
Raymond Hettinger5b4a54c2009-06-09 21:07:46 +0000550
551* The automatic name remapping in the pickle module for protocol 2 or lower can
552 make Python 3.1 pickles unreadable in Python 3.0. One solution is to use
553 protocol 3. Another solution is to set the *fix_imports* option to **False**.
554 See the discussion above for more details.