blob: 5d397fec26044e48a8281b39736b7e20ceec82b1 [file] [log] [blame]
Georg Brandlb80f5112012-09-30 09:11:58 +02001****************************
2 What's New In Python 3.4
3****************************
4
Georg Brandl3b80d342012-10-28 13:37:54 +01005.. :Author: Someone <email>
6 (uncomment if there is a principal author)
Georg Brandlb80f5112012-09-30 09:11:58 +02007
8.. Rules for maintenance:
9
Nick Coghlan03074fd2012-09-30 18:51:53 +053010 * Anyone can add text to this document, but the maintainer reserves the
11 right to rewrite any additions. In particular, for obscure or esoteric
12 features, the maintainer may reduce any addition to a simple reference to
13 the new documentation rather than explaining the feature inline.
Georg Brandlb80f5112012-09-30 09:11:58 +020014
Nick Coghlan03074fd2012-09-30 18:51:53 +053015 * While the maintainer will periodically go through Misc/NEWS
16 and add changes, it's best not to rely on this. We know from experience
17 that any changes that aren't in the What's New documentation around the
18 time of the original release will remain largely unknown to the community
19 for years, even if they're added later. We also know from experience that
20 other priorities can arise, and the maintainer will run out of time to do
R David Murray061efb12013-12-24 12:35:59 -050021 updates -- in such cases, end users will be much better served by partial
Nick Coghlan03074fd2012-09-30 18:51:53 +053022 notifications that at least give a hint about new features to
23 investigate.
Georg Brandlb80f5112012-09-30 09:11:58 +020024
25 * This is not a complete list of every single change; completeness
Nick Coghlan03074fd2012-09-30 18:51:53 +053026 is the purpose of Misc/NEWS. The What's New should focus on changes that
27 are visible to Python *users* and that *require* a feature release (i.e.
28 most bug fixes should only be recorded in Misc/NEWS)
29
30 * PEPs should not be marked Final until they have an entry in What's New.
31 A placeholder entry that is just a section header and a link to the PEP
32 (e.g ":pep:`397` has been implemented") is acceptable. If a PEP has been
33 implemented and noted in What's New, don't forget to mark it as Final!
Georg Brandlb80f5112012-09-30 09:11:58 +020034
35 * If you want to draw your new text to the attention of the
36 maintainer, add 'XXX' to the beginning of the paragraph or
37 section.
38
Nick Coghlan03074fd2012-09-30 18:51:53 +053039 * It's OK to add just a very brief note about a change. For
40 example: "The :ref:`~socket.transmogrify()` function was added to the
41 :mod:`socket` module." The maintainer will research the change and
42 write the necessary text (if appropriate). The advantage of doing this
43 is that even if no more descriptive text is ever added, readers will at
44 least have a notification that the new feature exists and a link to the
45 relevant documentation.
Georg Brandlb80f5112012-09-30 09:11:58 +020046
47 * You can comment out your additions if you like, but it's not
48 necessary (especially when a final release is some months away).
49
50 * Credit the author of a patch or bugfix. Just the name is
51 sufficient; the e-mail address isn't necessary.
52
53 * It's helpful to add the bug/patch number as a comment:
54
Nick Coghlan03074fd2012-09-30 18:51:53 +053055 The :ref:`~socket.transmogrify()` function was added to the
56 :mod:`socket` module. (Contributed by P.Y. Developer in :issue:`12345`.)
Georg Brandlb80f5112012-09-30 09:11:58 +020057
58 This saves the maintainer the effort of going through the Mercurial log
59 when researching a change.
60
Nick Coghlan03074fd2012-09-30 18:51:53 +053061 * Cross referencing tip: :ref:`mod.attr` will display as ``mod.attr``,
62 while :ref:`~mod.attr` will display as ``attr``.
63
Georg Brandlb80f5112012-09-30 09:11:58 +020064This article explains the new features in Python 3.4, compared to 3.3.
65
Nick Coghlan03074fd2012-09-30 18:51:53 +053066.. Python 3.4 was released on TBD.
Georg Brandlb80f5112012-09-30 09:11:58 +020067
Ezio Melotti25bbe5e2012-11-17 19:30:48 +020068For full details, see the
69`changelog <http://docs.python.org/3.4/whatsnew/changelog.html>`_.
Georg Brandlb80f5112012-09-30 09:11:58 +020070
71.. note:: Prerelease users should be aware that this document is currently in
72 draft form. It will be updated substantially as Python 3.4 moves towards
73 release, so it's worth checking back even after reading earlier versions.
74
75
76.. seealso::
77
R David Murray061efb12013-12-24 12:35:59 -050078 :pep:`429` -- Python 3.4 Release Schedule
Georg Brandlb80f5112012-09-30 09:11:58 +020079
80
R David Murraye6082552014-01-03 16:15:45 -050081
R David Murraye7cf6782013-12-24 14:51:25 -050082Summary -- Release Highlights
Georg Brandlb80f5112012-09-30 09:11:58 +020083=============================
84
Kristjan Valur Jonssona1e82442013-03-26 13:56:14 +000085.. This section singles out the most important changes in Python 3.4.
Georg Brandlb80f5112012-09-30 09:11:58 +020086 Brevity is key.
87
88New syntax features:
89
R David Murraye0f7a782013-12-20 16:04:29 -050090* No new syntax features were added in Python 3.4.
Georg Brandlb80f5112012-09-30 09:11:58 +020091
R David Murray809487e2013-12-22 20:49:40 -050092New expected features for Python implementations:
93
R David Murray68790662013-12-23 11:17:51 -050094* :ref:`pip should always be "available" <whatsnew-pep-453>` (:pep:`453`).
R David Murray809487e2013-12-22 20:49:40 -050095* :ref:`Make newly created file descriptors non-inheritable <whatsnew-pep-446>`
96 (:pep:`446`).
97* command line option for :ref:`isolated mode <using-on-misc-options>`,
98 (:issue:`16499`).
R David Murray8d856782013-12-23 10:28:57 -050099* :ref:`improvements in the handling of codecs <codec-handling-improvements>`
100 that are not text encodings (multiple issues).
R David Murray809487e2013-12-22 20:49:40 -0500101* :ref:`A ModuleSpec Type <whatsnew-pep-451>` for the Import System
R David Murraye9b74d42013-12-22 21:05:04 -0500102 (:pep:`451`). (Affects importer authors.)
R David Murray809487e2013-12-22 20:49:40 -0500103
Georg Brandlb80f5112012-09-30 09:11:58 +0200104New library modules:
105
R David Murray9217dad2013-12-23 21:08:28 -0500106* :mod:`asyncio`: :ref:`New provisional API for asynchronous IO
107 <whatsnew-asyncio>` (:pep:`3156`).
108* :mod:`ensurepip`: :ref:`Bootstrapping the pip installer <whatsnew-ensurepip>`
109 (:pep:`453`).
110* :mod:`enum`: :ref:`Support for enumeration types <whatsnew-enum>`
111 (:pep:`435`).
112* :mod:`pathlib`: :ref:`Object-oriented filesystem paths <whatsnew-pathlib>`
113 (:pep:`428`).
R David Murrayf9976e72013-12-23 10:32:02 -0500114* :mod:`selectors`: :ref:`High-level and efficient I/O multiplexing
115 <whatsnew-selectors>`, built upon the :mod:`select` module primitives (part
116 of :pep:`3156`).
R David Murray9217dad2013-12-23 21:08:28 -0500117* :mod:`statistics`: A basic :ref:`numerically stable statistics library
118 <whatsnew-statistics>` (:pep:`450`).
119* :mod:`tracemalloc`: :ref:`Trace Python memory allocations
120 <whatsnew-tracemalloc>` (:pep:`454`).
Georg Brandlb80f5112012-09-30 09:11:58 +0200121
Georg Brandlb80f5112012-09-30 09:11:58 +0200122Significantly Improved Library Modules:
123
R David Murray0a102162013-12-20 15:00:54 -0500124* :ref:`Single-dispatch generic functions <whatsnew-singledispatch>` in
R David Murray8d856782013-12-23 10:28:57 -0500125 :mod:`functools` (:pep:`443`).
126* New :mod:`pickle` :ref:`protocol 4 <whatsnew-protocol-4>` (:pep:`3154`).
R David Murray8d856782013-12-23 10:28:57 -0500127* :ref:`TLSv1.1 and TLSv1.2 support <whatsnew-tls-11-12>` for :mod:`ssl`
128 (:issue:`16692`).
R David Murrayac186222013-12-20 17:23:57 -0500129* :mod:`multiprocessing` now has :ref:`an option to avoid using os.fork
130 on Unix <whatsnew-multiprocessing-no-fork>` (:issue:`8713`).
R David Murray6adb4542013-12-20 13:10:43 -0500131* :mod:`email` has a new submodule, :mod:`~email.contentmanager`, and
132 a new :mod:`~email.message.Message` subclass
R David Murray26b80cfd2013-12-20 17:26:52 -0500133 (:class:`~email.contentmanager.EmailMessage`) that :ref:`simplify MIME
R David Murray8d856782013-12-23 10:28:57 -0500134 handling <whatsnew_email_contentmanager>` (:issue:`18891`).
Ronald Oussoren6db66532014-01-15 11:32:35 +0100135* :mod:`plistlib` has a cleaned up interface and support for binary
136 plist files (:issue:`14455`)
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +0200137
Nick Coghlan367df122013-10-27 01:57:34 +1000138CPython implementation improvements:
Georg Brandlb80f5112012-09-30 09:11:58 +0200139
R David Murraya93ca972013-12-22 14:10:21 -0500140* :ref:`Safe object finalization <whatsnew-pep-442>` (:pep:`442`).
R David Murrayca794612013-12-22 14:05:11 -0500141* Leveraging :pep:`442`, :ref:`module globals are no longer set to None
142 during finalization <whatsnew-pep-442>`, in most cases (:issue:`18214`).
R David Murraya93ca972013-12-22 14:10:21 -0500143* :ref:`Configurable memory allocators <whatsnew-pep-445>` (:pep:`445`).
144* :ref:`Secure and interchangeable hash algorithm <whatsnew-pep-456>`
145 (:pep:`456`).
146* :ref:`Argument Clinic <whatsnew-pep-436>` (:pep:`436`).
R David Murray8f7664a2013-12-22 20:40:11 -0500147* The :mod:`marshal` format has been made :ref:`more compact and efficient
148 <whatsnew-marshal-3>` (:issue:`16475`).
Georg Brandlb80f5112012-09-30 09:11:58 +0200149
R David Murray347f9c72013-12-23 21:30:06 -0500150Please read on for a comprehensive list of user-facing changes, including many
151other smaller improvements, CPython optimizations, deprecations, and potential
152porting issues.
Georg Brandlb80f5112012-09-30 09:11:58 +0200153
Nick Coghland0cf0632013-11-11 22:11:55 +1000154
R David Murraye6082552014-01-03 16:15:45 -0500155
R David Murraye7cf6782013-12-24 14:51:25 -0500156New Expected Features for Python Implementations
R David Murray809487e2013-12-22 20:49:40 -0500157================================================
158
R David Murray68790662013-12-23 11:17:51 -0500159.. _whatsnew-pep-453:
160
R David Murraye7cf6782013-12-24 14:51:25 -0500161PEP 453: Explicit Bootstrapping of PIP in Python Installations
R David Murray809487e2013-12-22 20:49:40 -0500162--------------------------------------------------------------
Nick Coghland0cf0632013-11-11 22:11:55 +1000163
164The new :mod:`ensurepip` module (defined in :pep:`453`) provides a standard
165cross-platform mechanism to boostrap the pip installer into Python
166installations and virtual environments.
167
Nick Coghlan7bc4b3b2013-11-23 11:59:40 +1000168The :mod:`venv` module and the :command:`pyvenv` utility make use of this
169module to make ``pip`` readily available in virtual environments. When
170using the command line interface, ``pip`` is installed by default, while
171for the module API installation of ``pip`` must be requested explicitly.
172
173For CPython source builds on POSIX systems, the ``make install`` and
174``make altinstall`` commands bootstrap ``pip`` by default. This behaviour
175can be controlled through configure options, and overridden through
176Makefile options.
177
Ned Deily44a0db02013-11-22 22:39:09 -0800178On Windows and Mac OS X, the CPython installers now offer the option to
179install ``pip`` along with CPython itself.
Nick Coghlan7bc4b3b2013-11-23 11:59:40 +1000180
R David Murray68790662013-12-23 11:17:51 -0500181As `discussed in the PEP`__, platform packagers may choose not to install
182``pip`` by default, as long as the command ``pip``, when invoked, provides
183clear and simple directions on how to install ``pip`` on the platform.
184
185__ http://www.python.org/dev/peps/pep-0453/#recommendations-for-downstream-distributors
186
Nick Coghland0cf0632013-11-11 22:11:55 +1000187.. note::
188
Nick Coghlan7bc4b3b2013-11-23 11:59:40 +1000189 The implementation of PEP 453 is still a work in progress. Refer to
190 :issue:`19347` for the progress on additional steps:
Nick Coghland0cf0632013-11-11 22:11:55 +1000191
Nick Coghlan7bc4b3b2013-11-23 11:59:40 +1000192 * Having the binary installers install ``pip`` by default
193 * Recommending the use of ``pip`` in the "Installing Python Module"
194 documentation.
Nick Coghland0cf0632013-11-11 22:11:55 +1000195
196.. seealso::
197
R David Murray061efb12013-12-24 12:35:59 -0500198 :pep:`453` -- Explicit bootstrapping of pip in Python installations
Nick Coghland0cf0632013-11-11 22:11:55 +1000199 PEP written by Donald Stufft and Nick Coghlan, implemented by
Nick Coghlan7bc4b3b2013-11-23 11:59:40 +1000200 Donald Stufft, Nick Coghlan, Martin von Löwis and Ned Deily.
Nick Coghland0cf0632013-11-11 22:11:55 +1000201
202
R David Murrayf9909c22013-12-20 14:50:12 -0500203.. _whatsnew-pep-446:
Victor Stinnerdaf45552013-08-28 00:53:59 +0200204
R David Murraye7cf6782013-12-24 14:51:25 -0500205PEP 446: Make Newly Created File Descriptors Non-Inheritable
R David Murray809487e2013-12-22 20:49:40 -0500206------------------------------------------------------------
Victor Stinnerdaf45552013-08-28 00:53:59 +0200207
Nick Coghlan367df122013-10-27 01:57:34 +1000208:pep:`446` makes newly created file descriptors :ref:`non-inheritable
Georg Brandl5642ff92013-09-15 10:37:57 +0200209<fd_inheritance>`. New functions and methods:
Victor Stinnerdaf45552013-08-28 00:53:59 +0200210
211* :func:`os.get_inheritable`, :func:`os.set_inheritable`
212* :func:`os.get_handle_inheritable`, :func:`os.set_handle_inheritable`
213* :meth:`socket.socket.get_inheritable`, :meth:`socket.socket.set_inheritable`
214
Antoine Pitrou796564c2013-07-30 19:59:21 +0200215.. seealso::
216
R David Murray061efb12013-12-24 12:35:59 -0500217 :pep:`446` -- Make newly created file descriptors non-inheritable
Nick Coghlan367df122013-10-27 01:57:34 +1000218 PEP written and implemented by Victor Stinner.
Antoine Pitrou796564c2013-07-30 19:59:21 +0200219
Georg Brandlb80f5112012-09-30 09:11:58 +0200220
Nick Coghlan9c1aed82013-11-23 11:13:36 +1000221.. _codec-handling-improvements:
222
R David Murraye7cf6782013-12-24 14:51:25 -0500223Improvements to Codec Handling
R David Murray809487e2013-12-22 20:49:40 -0500224------------------------------
Nick Coghlan8b097b42013-11-13 23:49:21 +1000225
226Since it was first introduced, the :mod:`codecs` module has always been
227intended to operate as a type-neutral dynamic encoding and decoding
228system. However, its close coupling with the Python text model, especially
229the type restricted convenience methods on the builtin :class:`str`,
230:class:`bytes` and :class:`bytearray` types, has historically obscured that
231fact.
232
233As a key step in clarifying the situation, the :meth:`codecs.encode` and
234:meth:`codecs.decode` convenience functions are now properly documented in
235Python 2.7, 3.3 and 3.4. These functions have existed in the :mod:`codecs`
Nick Coghlan9c1aed82013-11-23 11:13:36 +1000236module (and have been covered by the regression test suite) since Python 2.4,
Nick Coghlan8b097b42013-11-13 23:49:21 +1000237but were previously only discoverable through runtime introspection.
238
239Unlike the convenience methods on :class:`str`, :class:`bytes` and
240:class:`bytearray`, these convenience functions support arbitrary codecs
241in both Python 2 and Python 3, rather than being limited to Unicode text
242encodings (in Python 3) or ``basestring`` <-> ``basestring`` conversions
243(in Python 2).
244
Nick Coghlan8afc8f62013-11-22 23:00:22 +1000245In Python 3.4, the interpreter is able to identify the known non-text
246encodings provided in the standard library and direct users towards these
247general purpose convenience functions when appropriate::
Nick Coghlan8b097b42013-11-13 23:49:21 +1000248
Nick Coghlan9c1aed82013-11-23 11:13:36 +1000249 >>> b"abcdef".decode("hex")
Nick Coghlan8b097b42013-11-13 23:49:21 +1000250 Traceback (most recent call last):
251 File "<stdin>", line 1, in <module>
Nick Coghlan9c1aed82013-11-23 11:13:36 +1000252 LookupError: 'hex' is not a text encoding; use codecs.decode() to handle arbitrary codecs
Nick Coghlan8b097b42013-11-13 23:49:21 +1000253
Nick Coghlan9c1aed82013-11-23 11:13:36 +1000254 >>> "hello".encode("rot13")
Nick Coghlan8b097b42013-11-13 23:49:21 +1000255 Traceback (most recent call last):
256 File "<stdin>", line 1, in <module>
Nick Coghlan9c1aed82013-11-23 11:13:36 +1000257 LookupError: 'rot13' is not a text encoding; use codecs.encode() to handle arbitrary codecs
Nick Coghlan8b097b42013-11-13 23:49:21 +1000258
259In a related change, whenever it is feasible without breaking backwards
260compatibility, exceptions raised during encoding and decoding operations
261will be wrapped in a chained exception of the same type that mentions the
262name of the codec responsible for producing the error::
263
Nick Coghlan9c1aed82013-11-23 11:13:36 +1000264 >>> import codecs
265
266 >>> codecs.decode(b"abcdefgh", "hex")
Nick Coghlan8afc8f62013-11-22 23:00:22 +1000267 binascii.Error: Non-hexadecimal digit found
Nick Coghlan8b097b42013-11-13 23:49:21 +1000268
269 The above exception was the direct cause of the following exception:
270
271 Traceback (most recent call last):
272 File "<stdin>", line 1, in <module>
Nick Coghlan9c1aed82013-11-23 11:13:36 +1000273 binascii.Error: decoding with 'hex' codec failed (Error: Non-hexadecimal digit found)
Nick Coghlan8b097b42013-11-13 23:49:21 +1000274
Nick Coghlan9c1aed82013-11-23 11:13:36 +1000275 >>> codecs.encode("hello", "bz2")
Nick Coghlan8b097b42013-11-13 23:49:21 +1000276 TypeError: 'str' does not support the buffer interface
277
278 The above exception was the direct cause of the following exception:
279
280 Traceback (most recent call last):
281 File "<stdin>", line 1, in <module>
Nick Coghlan9c1aed82013-11-23 11:13:36 +1000282 TypeError: encoding with 'bz2' codec failed (TypeError: 'str' does not support the buffer interface)
Nick Coghlan8b097b42013-11-13 23:49:21 +1000283
Nick Coghlan9c1aed82013-11-23 11:13:36 +1000284Finally, as the examples above show, these improvements have permitted
285the restoration of the convenience aliases for the non-Unicode codecs that
286were themselves restored in Python 3.2. This means that encoding binary data
287to and from its hexadecimal representation (for example) can now be written
288as::
289
290 >>> from codecs import encode, decode
291 >>> encode(b"hello", "hex")
292 b'68656c6c6f'
293 >>> decode(b"68656c6c6f", "hex")
294 b'hello'
295
296The binary and text transforms provided in the standard library are detailed
297in :ref:`binary-transforms` and :ref:`text-transforms`.
298
R David Murray9217dad2013-12-23 21:08:28 -0500299(Contributed by Nick Coghlan in :issue:`7475`, , :issue:`17827`,
Nick Coghlan9c1aed82013-11-23 11:13:36 +1000300:issue:`17828` and :issue:`19619`)
Nick Coghlan8b097b42013-11-13 23:49:21 +1000301
R David Murray809487e2013-12-22 20:49:40 -0500302.. _whatsnew-pep-451:
Eric Snowb523f842013-11-22 09:05:39 -0700303
304PEP 451: A ModuleSpec Type for the Import System
R David Murray809487e2013-12-22 20:49:40 -0500305------------------------------------------------
Eric Snowb523f842013-11-22 09:05:39 -0700306
R David Murrayf85b2a82013-12-22 21:06:13 -0500307:pep:`451` provides an encapsulation of the information about a module that the
308import machinery will use to load it (that is, a module specification). This
309helps simplify both the import implementation and several import-related APIs.
310The change is also a stepping stone for `several future import-related
311improvements`__.
Eric Snowb523f842013-11-22 09:05:39 -0700312
R David Murraye9b74d42013-12-22 21:05:04 -0500313__ https://mail.python.org/pipermail/python-dev/2013-November/130111.html
Eric Snowb523f842013-11-22 09:05:39 -0700314
315The public-facing changes from the PEP are entirely backward-compatible.
R David Murrayf85b2a82013-12-22 21:06:13 -0500316Furthermore, they should be transparent to everyone but importer authors. Key
317finder and loader methods have been deprecated, but they will continue working.
318New importers should use the new methods described in the PEP. Existing
319importers should be updated to implement the new methods.
Eric Snowb523f842013-11-22 09:05:39 -0700320
Nick Coghlan8b097b42013-11-13 23:49:21 +1000321
Georg Brandlb80f5112012-09-30 09:11:58 +0200322Other Language Changes
R David Murray809487e2013-12-22 20:49:40 -0500323----------------------
Georg Brandlb80f5112012-09-30 09:11:58 +0200324
325Some smaller changes made to the core Python language are:
326
Ezio Melotti34808e22013-10-12 16:36:13 +0300327* Unicode database updated to UCD version 6.3.
Georg Brandlb80f5112012-09-30 09:11:58 +0200328
R David Murray9a2f1392013-06-28 13:31:19 -0400329* :func:`min` and :func:`max` now accept a *default* argument that can be used
330 to specify the value they return if the iterable they are evaluating has no
331 elements. Contributed by Julian Berman in :issue:`18111`.
Georg Brandlb80f5112012-09-30 09:11:58 +0200332
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +0200333* Module objects are now :mod:`weakref`'able.
334
Nick Coghlan0acceb72013-10-20 13:22:21 +1000335* Module ``__file__`` attributes (and related values) should now always
336 contain absolute paths by default, with the sole exception of
337 ``__main__.__file__`` when a script has been executed directly using
338 a relative path (Contributed by Brett Cannon in :issue:`18416`).
339
Serhiy Storchaka58cf6072013-11-19 11:32:41 +0200340* Now all the UTF-\* codecs (except UTF-7) reject surrogates during both
341 encoding and decoding unless the ``surrogatepass`` error handler is used,
342 with the exception of the UTF-16 decoder that accepts valid surrogate pairs,
343 and the UTF-16 encoder that produces them while encoding non-BMP characters.
344 Contributed by Victor Stinner, Kang-Hao (Kenny) Lu and Serhiy Storchaka in
345 :issue:`12892`.
346
Georg Brandlb80f5112012-09-30 09:11:58 +0200347
R David Murraye6082552014-01-03 16:15:45 -0500348
Georg Brandlb80f5112012-09-30 09:11:58 +0200349New Modules
350===========
351
Nick Coghlan0acceb72013-10-20 13:22:21 +1000352
R David Murray9217dad2013-12-23 21:08:28 -0500353.. _whatsnew-asyncio:
354
Nick Coghlan0acceb72013-10-20 13:22:21 +1000355asyncio
356-------
357
358The new :mod:`asyncio` module (defined in :pep:`3156`) provides a standard
359pluggable event loop model for Python, providing solid asynchronous IO
360support in the standard library, and making it easier for other event loop
361implementations to interoperate with the standard library and each other.
362
363For Python 3.4, this module is considered a :term:`provisional API`.
364
Nick Coghlan367df122013-10-27 01:57:34 +1000365.. seealso::
366
R David Murray061efb12013-12-24 12:35:59 -0500367 :pep:`3156` -- Asynchronous IO Support Rebooted: the "asyncio" Module
Nick Coghlan367df122013-10-27 01:57:34 +1000368 PEP written and implementation led by Guido van Rossum.
369
R David Murray68790662013-12-23 11:17:51 -0500370
R David Murray9217dad2013-12-23 21:08:28 -0500371.. _whatsnew-ensurepip:
R David Murray68790662013-12-23 11:17:51 -0500372
373ensurepip
374---------
375
376The new :mod:`ensurepip` module is the primary infrastructure for the
377:pep:`453` implementation. In the normal course of events end users will not
378need to interact with this module, but it can be used to manually bootstrap
379``pip`` if the automated bootstrapping into an installation or virtual
380environment was declined.
381
382:mod:`ensurepip` includes a bundled copy of ``pip``, up-to-date as of the first
383release candidate of the release of CPython with which it ships (this applies
384to both maintenance releases and feature releases). ``ensurepip`` does not
385access the internet. (If the installation has Internet access, it is of course
386possible to upgrade ``pip`` to a release more recent than the bundled ``pip``
387by using the bundled ``pip`` command itself once it is installed.)
388
389The module is named *ensure*\ pip because if called when ``pip`` is already
390installed, it does nothing. It also has an ``--upgrade`` option that will
391cause it to install the bundled copy of ``pip`` if the existing installed
392version of ``pip`` is older than the bundled copy.
393
394
R David Murray9217dad2013-12-23 21:08:28 -0500395.. _whatsnew-enum:
396
Nick Coghlan0acceb72013-10-20 13:22:21 +1000397enum
398----
399
Nick Coghlan367df122013-10-27 01:57:34 +1000400The new :mod:`enum` module (defined in :pep:`435`) provides a standard
401implementation of enumeration types, allowing other modules (such as
402:mod:`socket`) to provide more informative error messages and better
403debugging support by replacing opaque integer constants with backwards
404compatible enumeration values.
405
406.. seealso::
407
R David Murray061efb12013-12-24 12:35:59 -0500408 :pep:`435` -- Adding an Enum type to the Python standard library
Nick Coghlan367df122013-10-27 01:57:34 +1000409 PEP written by Barry Warsaw, Eli Bendersky and Ethan Furman,
410 implemented by Ethan Furman.
Nick Coghlan0acceb72013-10-20 13:22:21 +1000411
412
R David Murray9217dad2013-12-23 21:08:28 -0500413.. _whatsnew-pathlib:
414
Antoine Pitrou31119e42013-11-22 17:38:12 +0100415pathlib
416-------
417
418The new :mod:`pathlib` module offers classes representing filesystem paths
419with semantics appropriate for different operating systems. Path classes are
420divided between *pure paths*, which provide purely computational operations
421without I/O, and *concrete paths*, which inherit from pure paths but also
422provide I/O operations.
423
424For Python 3.4, this module is considered a :term:`provisional API`.
425
426.. seealso::
427
R David Murray061efb12013-12-24 12:35:59 -0500428 :pep:`428` -- The pathlib module -- object-oriented filesystem paths
Antoine Pitrou31119e42013-11-22 17:38:12 +0100429 PEP written and implemented by Antoine Pitrou.
430
431
R David Murrayf9976e72013-12-23 10:32:02 -0500432.. _whatsnew-selectors:
433
Charles-François Natali243d8d82013-09-04 19:02:49 +0200434selectors
435---------
Georg Brandlb80f5112012-09-30 09:11:58 +0200436
Nick Coghlan0acceb72013-10-20 13:22:21 +1000437The new :mod:`selectors` module (created as part of implementing :pep:`3156`)
438allows high-level and efficient I/O multiplexing, built upon the
439:mod:`select` module primitives.
440
441
R David Murray9217dad2013-12-23 21:08:28 -0500442.. _whatsnew-statistics:
443
Nick Coghlan0acceb72013-10-20 13:22:21 +1000444statistics
445----------
446
447The new :mod:`statistics` module (defined in :pep:`450`) offers some core
448statistics functionality directly in the standard library. This module
449supports calculation of the mean, median, mode, variance and standard
450deviation of a data series.
Georg Brandlb80f5112012-09-30 09:11:58 +0200451
Nick Coghlan367df122013-10-27 01:57:34 +1000452.. seealso::
453
R David Murray061efb12013-12-24 12:35:59 -0500454 :pep:`450` -- Adding A Statistics Module To The Standard Library
Nick Coghlan367df122013-10-27 01:57:34 +1000455 PEP written and implemented by Steven D'Aprano
456
R David Murray9217dad2013-12-23 21:08:28 -0500457.. _whatsnew-tracemalloc:
Georg Brandlb80f5112012-09-30 09:11:58 +0200458
R David Murray8b2d6822013-12-31 15:06:05 -0500459
Victor Stinnerd2736af2013-11-25 09:40:27 +0100460tracemalloc
461-----------
462
463The new :mod:`tracemalloc` module (defined in :pep:`454`) is a debug tool to
464trace memory blocks allocated by Python. It provides the following information:
465
466* Traceback where an object was allocated
467* Statistics on allocated memory blocks per filename and per line number:
468 total size, number and average size of allocated memory blocks
469* Compute the differences between two snapshots to detect memory leaks
470
471.. seealso::
472
R David Murray061efb12013-12-24 12:35:59 -0500473 :pep:`454` -- Add a new tracemalloc module to trace Python memory allocations
Victor Stinnerd2736af2013-11-25 09:40:27 +0100474 PEP written and implemented by Victor Stinner
475
476
R David Murraye6082552014-01-03 16:15:45 -0500477
Georg Brandlb80f5112012-09-30 09:11:58 +0200478Improved Modules
479================
480
R David Murray6e390152013-12-24 22:28:04 -0500481
R David Murray3edcc782013-12-24 16:13:32 -0500482abc
483---
484
485New function :func:`abc.get_cache_token` can be used to know when to invalidate
486caches that are affected by changes in the object graph. (Contributed
487by Łukasz Langa in :issue:`16832`.)
R David Murray72420ff2013-12-24 10:46:44 -0500488
R David Murray2691ee62013-12-28 23:15:12 -0500489New class :class:`~abc.ABC` has :class:`~abc.ABCMeta` as its meta class.
490Using ``ABC`` as a base class has essentially the same effect as specifying
491``metaclass=abc.ABCMeta``, but is simpler to type and easier to read.
492(Contributed by Bruno Dupuis in :issue:`16049`.)
493
494
Antoine Pitroud6cbd342013-08-12 20:48:15 +0200495aifc
496----
497
498The :meth:`~aifc.getparams` method now returns a namedtuple rather than a
499plain tuple. (Contributed by Claudiu Popa in :issue:`17818`.)
500
R David Murrayd592bb22013-12-31 13:45:38 -0500501:func:`aifc.open` now supports the context manager protocol: when used in a
502:keyword:`with` block, the :meth:`~aifc.aifc.close` method of the returned
503object will be called automatically at the end of the block. (Contributed by
504Serhiy Storchacha in :issue:`16486`.)
505
Antoine Pitroud6cbd342013-08-12 20:48:15 +0200506
R David Murrayfced3ec2013-12-31 11:18:01 -0500507argparse
508--------
509
510The :class:`~argparse.FileType` class now accepts *encoding* and
511*errors* arguments, which are passed through to :func:`open`. (Contributed
512by Lucas Maystre in :issue:`11175`.)
513
514
Serhiy Storchakaeaea5e92013-10-19 21:10:46 +0300515audioop
516-------
517
518Added support for 24-bit samples (:issue:`12866`).
519
Serhiy Storchaka3062c9a2013-11-23 22:26:01 +0200520Added the :func:`~audioop.byteswap` function to convert big-endian samples
521to little-endian and vice versa (:issue:`19641`).
522
Serhiy Storchakaeaea5e92013-10-19 21:10:46 +0300523
Nick Coghland4fdbcc2013-11-14 00:24:31 +1000524base64
525------
526
527The encoding and decoding functions in :mod:`base64` now accept any
528:term:`bytes-like object` in cases where it previously required a
Georg Brandled007d52013-11-24 16:09:26 +0100529:class:`bytes` or :class:`bytearray` instance (:issue:`17839`).
Nick Coghland4fdbcc2013-11-14 00:24:31 +1000530
531
Antoine Pitroud6cbd342013-08-12 20:48:15 +0200532colorsys
533--------
534
535The number of digits in the coefficients for the RGB --- YIQ conversions have
536been expanded so that they match the FCC NTSC versions. The change in
537results should be less than 1% and may better match results found elsewhere.
538
R David Murray8e37d5d2013-04-13 14:49:48 -0400539
Nick Coghlanb4534ae2013-10-13 23:23:08 +1000540contextlib
541----------
542
Nick Coghlan240f86d2013-10-17 23:40:57 +1000543The new :class:`contextlib.suppress` context manager helps to clarify the
544intent of code that deliberately suppresses exceptions from a single
545statement. (Contributed by Raymond Hettinger in :issue:`15806` and
546Zero Piraeus in :issue:`19266`)
547
Victor Stinner6633c392013-10-21 13:27:11 +0200548The new :func:`contextlib.redirect_stdout` context manager makes it easier
Nick Coghlanb4534ae2013-10-13 23:23:08 +1000549for utility scripts to handle inflexible APIs that don't provide any
550options to retrieve their output as a string or direct it to somewhere
Nick Coghlan0acceb72013-10-20 13:22:21 +1000551other than :data:`sys.stdout`. In conjunction with :class:`io.StringIO`,
552this context manager is also useful for checking expected output from
553command line utilities. (Contribute by Raymond Hettinger in :issue:`15805`)
554
555The :mod:`contextlib` documentation has also been updated to include a
556:ref:`discussion <single-use-reusable-and-reentrant-cms>` of the
557differences between single use, reusable and reentrant context managers.
Nick Coghlanb4534ae2013-10-13 23:23:08 +1000558
559
Nick Coghlanb39fd0c2013-05-06 23:59:20 +1000560dis
561---
562
R David Murray0bce6e72014-01-07 14:30:17 -0500563Functions :func:`~dis.show_code`, :func:`~dis.dis`, :func:`~dis.distb`, and
564:func:`~dis.disassemble` now accept a keyword-only *file* argument that
565controls where they write their output.
Nick Coghlanb39fd0c2013-05-06 23:59:20 +1000566
R David Murray0bce6e72014-01-07 14:30:17 -0500567The :mod:`dis` module is now built around an :class:`~dis.Instruction` class
568that provides object oriented access to the details of each individual bytecode
569operation.
570
571A new method, :func:`~dis.get_instructions`, provides an iterator that emits
572the Instruction stream for a given piece of Python code. Thus it is now
573possible to write a program that inspects and manipulates a bytecode
574object in ways different from those provided by the :mod:`~dis` module
575itself. For example::
576
577 >>> import dis
578 >>> for instr in dis.get_instructions(lambda x: x + 1):
579 ... print(instr.opname)
580 LOAD_FAST
581 LOAD_CONST
582 BINARY_ADD
583 RETURN_VALUE
584
585The various display tools in the :mod:`dis` module have been rewritten to use
586these new components.
587
588In addition, a new application-friendly class :class:`~dis.Bytecode` provides
589an object-oriented API for inspecting bytecode in both in human-readable form
590and for iterating over instructions. The :class:`~dis.Bytecode` constructor
591takes the same arguments that :func:`~dis.get_instruction` does (plus an
592optional *current_offset*), and the resulting object can be iterated to produce
593:class:`~dis.Instruction` objects. But it also has a :mod:`~dis.Bytecode.dis`
594method, equivalent to calling :mod:`~dis.dis` on the constructor argument, but
595returned as a multi-line string::
596
597 >>> bytecode = dis.Bytecode(lambda x: x +1, current_offset=3)
598 >>> for instr in bytecode:
599 ... print('{} ({})'.format(instr.opname, instr.opcode))
600 LOAD_FAST (124)
601 LOAD_CONST (100)
602 BINARY_ADD (23)
603 RETURN_VALUE (83)
604 >>> bytecode.dis().splitlines() # doctest: +NORMALIZE_WHITESPACE
605 [' 1 0 LOAD_FAST 0 (x)',
606 ' --> 3 LOAD_CONST 1 (1)',
607 ' 6 BINARY_ADD',
608 ' 7 RETURN_VALUE']
609
610:class:`~dis.Bytecode` also has a class method,
611:meth:`~dis.Bytecode.from_traceback`, that provides the ability to manipulate a
612traceback (that is, ``print(Bytecode.from_traceback(tb).dis())`` is equivalent
613to ``distb(tb)``).
Nick Coghlanb39fd0c2013-05-06 23:59:20 +1000614
Nick Coghlan50c48b82013-11-23 00:57:00 +1000615(Contributed by Nick Coghlan, Ryan Kelly and Thomas Kluyver in :issue:`11816`
616and Claudiu Popa in :issue:`17916`)
Nick Coghlanb39fd0c2013-05-06 23:59:20 +1000617
Antoine Pitroud6cbd342013-08-12 20:48:15 +0200618
R David Murray5a9d7062012-11-21 15:09:21 -0500619doctest
620-------
621
R David Murray473f45b2013-12-27 17:01:16 -0500622A new :ref:`option flag <doctest-options>`, :data:`~doctest.FAIL_FAST`, halts
623test running as soon as the first failure is detected. (Contributed by R.
624David Murray and Daniel Urban in :issue:`16522`.)
R David Murray5707d502013-06-23 14:24:13 -0400625
R David Murray473f45b2013-12-27 17:01:16 -0500626The :mod:`doctest` command line interface now uses :mod:`argparse`, and has two
627new options, ``-o`` and ``-f``. ``-o`` allows :ref:`doctest options
628<doctest-options>` to be specified on the command line, and ``-f`` is a
629shorthand for ``-o FAIL_FAST`` (to parallel the similar option supported by the
630:mod:`unittest` CLI). (Contributed by R. David Murray in :issue:`11390`.)
Georg Brandlb80f5112012-09-30 09:11:58 +0200631
R David Murray8e37d5d2013-04-13 14:49:48 -0400632
R David Murraybb17d2b2013-08-09 16:15:28 -0400633email
634-----
635
636:meth:`~email.message.Message.as_string` now accepts a *policy* argument to
637override the default policy of the message when generating a string
638representation of it. This means that ``as_string`` can now be used in more
639circumstances, instead of having to create and use a :mod:`~email.generator` in
640order to pass formatting parameters to its ``flatten`` method.
641
642New method :meth:`~email.message.Message.as_bytes` added to produce a bytes
643representation of the message in a fashion similar to how ``as_string``
644produces a string representation. It does not accept the *maxheaderlen*
645argument, but does accept the *unixfrom* and *policy* arguments. The
646:class:`~email.message.Message` :meth:`~email.message.Message.__bytes__` method
647calls it, meaning that ``bytes(mymsg)`` will now produce the intuitive
648result: a bytes object containing the fully formatted message.
649
650(Contributed by R. David Murray in :issue:`18600`.)
651
R David Murray26b80cfd2013-12-20 17:26:52 -0500652.. _whatsnew_email_contentmanager:
653
R David Murray3da240f2013-10-16 22:48:40 -0400654A pair of new subclasses of :class:`~email.message.Message` have been added,
655along with a new sub-module, :mod:`~email.contentmanager`. All documentation
656is currently in the new module, which is being added as part of the new
Nick Coghlan240f86d2013-10-17 23:40:57 +1000657:term:`provisional <provisional package>` email API. These classes provide a
R David Murray3da240f2013-10-16 22:48:40 -0400658number of new methods that make extracting content from and inserting content
659into email messages much easier. See the :mod:`~email.contentmanager`
660documentation for details.
661
662These API additions complete the bulk of the work that was planned as part of
663the email6 project. The currently provisional API is scheduled to become final
664in Python 3.5 (possibly with a few minor additions in the area of error
665handling).
666
667(Contributed by R. David Murray in :issue:`18891`.)
668
R David Murraybb17d2b2013-08-09 16:15:28 -0400669
Victor Stinner854ffcb2013-06-21 00:36:30 +0200670functools
671---------
672
Nick Coghlanf4cb48a2013-11-03 16:41:46 +1000673The new :func:`~functools.partialmethod` descriptor bring partial argument
674application to descriptors, just as :func:`~functools.partial` provides
675for normal callables. The new descriptor also makes it easier to get
676arbitrary callables (including :func:`~functools.partial` instances)
677to behave like normal instance methods when included in a class definition.
678
679(Contributed by Alon Horev and Nick Coghlan in :issue:`4331`)
680
R David Murray0a102162013-12-20 15:00:54 -0500681.. _whatsnew-singledispatch:
682
Nick Coghlanf4cb48a2013-11-03 16:41:46 +1000683The new :func:`~functools.singledispatch` decorator brings support for
684single-dispatch generic functions to the Python standard library. Where
685object oriented programming focuses on grouping multiple operations on a
686common set of data into a class, a generic function focuses on grouping
687multiple implementations of an operation that allows it to work with
688*different* kinds of data.
689
690.. seealso::
691
R David Murray061efb12013-12-24 12:35:59 -0500692 :pep:`443` -- Single-dispatch generic functions
Nick Coghlanf4cb48a2013-11-03 16:41:46 +1000693 PEP written and implemented by Łukasz Langa.
Victor Stinner854ffcb2013-06-21 00:36:30 +0200694
R David Murrayd2653812013-12-27 14:06:15 -0500695A pure-python version of the :func:`~functools.partial` function is now in the
696stdlib; in CPython it is overridden by the C accelerated version, but it is
697available for other implementations to use. (Contributed by Brian Thorne in
698:issue:`12428`.)
699
Nick Coghlane8c45d62013-07-28 20:00:01 +1000700
R David Murray0e814632013-12-26 15:11:28 -0500701gc
702--
703
R David Murraya101bdb2014-01-06 16:32:05 -0500704New function :func:`~gc.get_stats` returns a list of three per-generation
705dictionaries containing the collections statistics since interpreter startup.
706(Contributed by Antoine Pitrou in :issue:`16351`.)
R David Murray0e814632013-12-26 15:11:28 -0500707
708
Christian Heimese92ef132013-10-13 00:52:43 +0200709hashlib
710-------
711
712New :func:`hashlib.pbkdf2_hmac` function.
Christian Heimese92ef132013-10-13 00:52:43 +0200713(Contributed by Christian Heimes in :issue:`18582`)
714
715
Ezio Melotti250a06c2013-11-25 06:18:47 +0200716html
717----
718
719Added a new :func:`html.unescape` function that converts HTML5 character
720references to the corresponding Unicode characters.
Ezio Melotti250a06c2013-11-25 06:18:47 +0200721(Contributed by Ezio Melotti in :issue:`2927`)
722
723Added a new *convert_charrefs* keyword argument to
724:class:`~html.parser.HTMLParser` that, when ``True``, automatically converts
725all character references. For backward-compatibility, its value defaults
726to ``False``, but it will change to ``True`` in future versions, so you
727are invited to set it explicitly and update your code to use this new feature.
Ezio Melotti250a06c2013-11-25 06:18:47 +0200728(Contributed by Ezio Melotti in :issue:`13633`)
729
730The *strict* argument of :class:`~html.parser.HTMLParser` is now deprecated.
Ezio Melotti250a06c2013-11-25 06:18:47 +0200731(Contributed by Ezio Melotti in :issue:`15114`)
732
733
R David Murraya475a8d2014-01-03 13:03:00 -0500734http
735----
736
737:meth:`~http.server.BaseHTTPRequestHandler.send_error` now accepts an
Zachary Ware335957e2014-01-13 16:08:54 -0600738optional additional *explain* parameter which can be used to provide an
R David Murraya475a8d2014-01-03 13:03:00 -0500739extended error description, overriding the hardcoded default if there is one.
740This extended error description will be formatted using the
741:attr:`~http.server.HTTP.error_message_format` attribute and sent as the body
742of the error response. (Contributed by Karl Cow in :issue:`12921`.)
743
744
745
R David Murrayd2653812013-12-27 14:06:15 -0500746importlib
747---------
748
749The :class:`~importlib.abc.InspectLoader` ABC defines a new method,
750:meth:`~importlib.abc.InspectLoader.source_to_code` that accepts source
751data and a path and returns a code object. The default implementation
752is equivalent to ``compile(data, path, 'exec', dont_inherit=True)``.
753(Contributed by Eric Snow and Brett Cannon in :issue:`15627`.)
754
R David Murray0ae7ae12014-01-08 18:16:02 -0500755:class:`~importlib.abc.InspectLoader` also now has a default implementation
756for the :meth:`~importlib.abc.InspectLoader.get_code` method. However,
757it will normally be desirable to override the default implementation
758for performance reasons. (Contributed by Brett Cannon in :issue:`18072`.)
759
R David Murrayd2653812013-12-27 14:06:15 -0500760
Antoine Pitroud6cbd342013-08-12 20:48:15 +0200761inspect
762-------
763
Nick Coghlanf94a16b2013-09-22 22:46:49 +1000764
Nick Coghlan367df122013-10-27 01:57:34 +1000765The inspect module now offers a basic :ref:`command line interface
766<inspect-module-cli>` to quickly display source code and other
767information for modules, classes and functions. (Contributed by Claudiu Popa
768and Nick Coghlan in :issue:`18626`)
Nick Coghlanf94a16b2013-09-22 22:46:49 +1000769
Antoine Pitroud6cbd342013-08-12 20:48:15 +0200770:func:`~inspect.unwrap` makes it easy to unravel wrapper function chains
771created by :func:`functools.wraps` (and any other API that sets the
Nick Coghlan367df122013-10-27 01:57:34 +1000772``__wrapped__`` attribute on a wrapper function). (Contributed by
773Daniel Urban, Aaron Iles and Nick Coghlan in :issue:`13266`)
774
775As part of the implementation of the new :mod:`enum` module, the
776:mod:`inspect` module now has substantially better support for custom
777``__dir__`` methods and dynamic class attributes provided through
778metaclasses (Contributed by Ethan Furman in :issue:`18929` and
779:issue:`19030`)
780
Antoine Pitroud6cbd342013-08-12 20:48:15 +0200781
R David Murray9cf617b2014-01-04 18:55:01 -0500782logging
783-------
784
785The :class:`~logging.handlers.TimedRotatingFileHandler` has a new *atTime*
786parameter that can be used to specify the time of day when rollover should
787happen. (Contributed by Ronald Oussoren in :issue:`9556`.)
788
789
R David Murray8f7664a2013-12-22 20:40:11 -0500790.. _whatsnew-marshal-3:
791
792marshal
793-------
794
795The default :mod:`marshal` version has been bumped to 3. The code implementing
796the new version restores the Python2 behavior of recording only one copy of
797interned strings and preserving the interning on deserialization, and extends
798this "one copy" ability to any object type (including handling recursive
799references). This reduces both the size of ``.pyc`` files and the amount of
800memory a module occupies in memory when it is loaded from a ``.pyc`` (or
801``.pyo``) file. (Contributed by Kristján Valur Jónsson in :issue:`16475`.)
802
803
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +0200804mmap
805----
806
807mmap objects can now be weakref'ed.
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +0200808(Contributed by Valerie Lambert in :issue:`4885`.)
809
810
R David Murray33ef1ae2013-12-31 16:04:50 -0500811mock
812----
813
814:mod:`~unittest.mock` objects now inspect their specification signatures when
815matching calls, which means an argument can now be matched by either position
816or name, instead of only by position. (Contributed by Antoine Pitrou in
817:issue:`17015`.)
818
R David Murray57fcf762014-01-03 23:31:54 -0500819:func:`~mock.mock_open` objects now have ``readline`` and ``readlines``
820methods. (Contributed by Toshio Kuratomi in :issue:`17467`.)
821
R David Murray33ef1ae2013-12-31 16:04:50 -0500822
Richard Oudkerk84ed9a62013-08-14 15:35:41 +0100823multiprocessing
824---------------
825
R David Murrayac186222013-12-20 17:23:57 -0500826.. _whatsnew-multiprocessing-no-fork:
827
828On Unix, two new :ref:`start methods <multiprocessing-start-methods>`
829(``spawn`` and ``forkserver``) have been added for starting processes using
830:mod:`multiprocessing`. These make the mixing of processes with threads more
831robust, and the ``spawn`` method matches the semantics that multiprocessing has
832always used on Windows. (Contributed by Richard Oudkerk in :issue:`8713`).
Richard Oudkerk84ed9a62013-08-14 15:35:41 +0100833
834Also, except when using the old *fork* start method, child processes
R David Murrayac186222013-12-20 17:23:57 -0500835will no longer inherit unneeded handles/file descriptors from their parents
836(part of :issue:`8713`).
Richard Oudkerk84ed9a62013-08-14 15:35:41 +0100837
Nick Coghlan9a767352013-12-17 22:17:26 +1000838:mod:`multiprocessing` now relies on :mod:`runpy` (which implements the
839``-m`` switch) to initialise ``__main__`` appropriately in child processes
840when using the ``spawn`` or ``forkserver`` start methods. This resolves some
841edge cases where combining multiprocessing, the ``-m`` command line switch
842and explicit relative imports could cause obscure failures in child
843processes. (Contributed by Nick Coghlan in :issue:`19946`)
844
Richard Oudkerk84ed9a62013-08-14 15:35:41 +0100845
R David Murray98358272014-01-05 20:52:06 -0500846operator
847--------
848
849There is now a pure-python version of the :mod:`operator` module available for
850reference and for use by alternate implementations of Python. (Contributed by
851Zachary Ware in :issue:`16694`.)
852
853
Victor Stinnerdaf45552013-08-28 00:53:59 +0200854os
855--
856
Georg Brandlc6ebbef2013-09-16 04:03:12 +0200857New functions to get and set the :ref:`inheritable flag <fd_inheritance>` of a file
Victor Stinnerdaf45552013-08-28 00:53:59 +0200858descriptors or a Windows handle:
859
860* :func:`os.get_inheritable`, :func:`os.set_inheritable`
861* :func:`os.get_handle_inheritable`, :func:`os.set_handle_inheritable`
862
R David Murray66bf12a2014-01-08 17:21:22 -0500863The :mod:`os` module now provides a :func:`~os.cpu_count` function, analogous to
864the :func:`multiprocessing.cpu_count` function (which is now implemented in
865terms of the new :mod:`os` function). (Contributed by Trent Nelson, Yogesh
866Chaudhari, Victor Stinner, and Charles-François Natali in :issue:`17914`.)
867
Victor Stinnerdaf45552013-08-28 00:53:59 +0200868
R David Murray78d692f2013-10-10 17:23:26 -0400869pdb
870---
871
872The ``print`` command has been removed from :mod:`pdb`, restoring access to the
873``print`` function.
874
875Rationale: Python2's ``pdb`` did not have a ``print`` command; instead,
876entering ``print`` executed the ``print`` statement. In Python3 ``print`` was
877mistakenly made an alias for the pdb :pdbcmd:`p` command. ``p``, however,
878prints the ``repr`` of its argument, not the ``str`` like the Python2 ``print``
879command did. Worse, the Python3 ``pdb print`` command shadowed the Python3
880``print`` function, making it inaccessible at the ``pdb`` prompt.
881
882(Contributed by Connor Osborn in :issue:`18764`.)
883
884
R David Murray809487e2013-12-22 20:49:40 -0500885.. _whatsnew-protocol-4:
886
R David Murray5ea95242013-12-24 15:59:57 -0500887pickle
R David Murray809487e2013-12-22 20:49:40 -0500888------
889
890protocol 4
891
892:mod:`pickle` now supports (but does not use by default) a new pickle protocol,
893protocol 4. This new protocol addresses a number of issues that were present
894in previous protocols, such as the serialization of nested classes, very large
895strings and containers, or classes whose :meth:`__new__` method takes
896keyword-only arguments. It also provides some efficiency improvements.
897
898.. seealso::
899
R David Murray061efb12013-12-24 12:35:59 -0500900 :pep:`3154` -- Pickle protocol 4
R David Murray809487e2013-12-22 20:49:40 -0500901 PEP written by Antoine Pitrou and implemented by Alexandre Vassalotti.
902
903
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +0200904poplib
905------
906
R David Murray473f45b2013-12-27 17:01:16 -0500907Two new methods have been added to :mod:`poplib`: :meth:`~poplib.POP3.capa`,
908which returns the list of capabilities advertised by the POP server, and
909:meth:`~poplib.POP3.stls`, which switches a clear-text POP3 session into an
910encrypted POP3 session if the POP server supports it. (Contributed by Lorenzo
911Catucci in :issue:`4473`.)
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +0200912
913
Serhiy Storchaka7c411a42013-10-02 11:56:18 +0300914pprint
915------
916
Christian Heimese1bfd3e2013-10-21 12:32:21 +0200917The :mod:`pprint` module now supports *compact* mode for formatting long
Serhiy Storchaka7c411a42013-10-02 11:56:18 +0300918sequences (:issue:`19132`).
919
R David Murrayb231b2b2014-01-04 17:11:23 -0500920Long strings are now wrapped using Python's normal line continuation
921syntax (Contributed by Antoine Pitrou in :issue:`17150`.)
922
Serhiy Storchaka7c411a42013-10-02 11:56:18 +0300923
R David Murray6e390152013-12-24 22:28:04 -0500924pty
925---
926
927:func:`pty.spawn` now returns the status value from :func:`os.waitpid` on
928the child process, instead of ``None``. (Contributed by Gregory P. Smith.)
929
930
Nick Coghlan367df122013-10-27 01:57:34 +1000931pydoc
932-----
933
934While significant changes have not been made to :mod:`pydoc` directly,
935its handling of custom ``__dir__`` methods and various descriptor
936behaviours has been improved substantially by the underlying changes in
937the :mod:`inspect` module.
938
939
Serhiy Storchaka32eddc12013-11-23 23:20:30 +0200940re
941--
942
943Added :func:`re.fullmatch` function and :meth:`regex.fullmatch` method,
944which anchor the pattern at both ends of the string to match.
945(Contributed by Matthew Barnett in :issue:`16203`.)
946
Ezio Melottidd7e2912013-11-25 23:20:20 +0200947The repr of :ref:`regex objects <re-objects>` now includes the pattern
948and the flags; the repr of :ref:`match objects <match-objects>` now
949includes the start, end, and the part of the string that matched.
Ezio Melottidd7e2912013-11-25 23:20:20 +0200950(Contributed by Serhiy Storchaka in :issue:`13592` and :issue:`17087`.)
951
952
Christian Heimesb7bd5df2013-10-22 11:21:54 +0200953resource
954--------
955
956New :func:`resource.prlimit` function and Linux specific constants.
957(Contributed by Christian Heimes in :issue:`16595` and :issue:`19324`.)
958
R David Murray575fb312013-12-25 23:21:03 -0500959
R David Murray2bc930f2013-12-31 11:17:21 -0500960select
961------
962
963:class:`~select.epoll` objects now support the context management protocol.
964When used in a :keyword:`with` statement, the :meth:`~select.epoll.close`
965method will be called automatically at the end of the block. (Contributed
966by Serhiy Storchaka in :issue:`16488`.)
967
968
R David Murray575fb312013-12-25 23:21:03 -0500969shelve
970------
971
972:class:`~shelve.Shelf` instances may now be used in :keyword:`with` statements,
973and will be automatically closed at the end of the :keyword:`with` block.
974(Contributed by Filip Gruszczyński in :issue:`13896`.)
975
976
R David Murrayf1e4fdc2014-01-21 18:30:42 -0500977smtpd
978-----
979
980The :class:`~smtpd.SMTPServer` and :class:`~smtpd.SMTPChannel` classes now
981accept a *map* keyword argument, which if specified is passed in to
982:class:`asynchat.async_chat` as its *map* argument. This allows an application
983to avoid affecting the global socket map. (Contributed by Vinay Sajip in
984:issue:`11959`.)
985
986
R David Murray8e37d5d2013-04-13 14:49:48 -0400987smtplib
988-------
989
R David Murray8a345962013-04-14 06:46:35 -0400990:exc:`~smtplib.SMTPException` is now a subclass of :exc:`OSError`, which allows
R David Murray8e37d5d2013-04-13 14:49:48 -0400991both socket level errors and SMTP protocol level errors to be caught in one
992try/except statement by code that only cares whether or not an error occurred.
993(:issue:`2118`).
994
Antoine Pitroud6cbd342013-08-12 20:48:15 +0200995
Victor Stinnerdaf45552013-08-28 00:53:59 +0200996socket
997------
998
R David Murray33ef1ae2013-12-31 16:04:50 -0500999The socket module now supports the :data:`~socket.CAN_BCM` protocol on
1000platforms that support it. (Contributed by Brian Thorne in :issue:`15359`.)
1001
Georg Brandl5642ff92013-09-15 10:37:57 +02001002Socket objects have new methods to get or set their :ref:`inheritable flag
R David Murray33ef1ae2013-12-31 16:04:50 -05001003<fd_inheritance>`, :meth:`~socket.socket.get_inheritable` and
1004:meth:`~socket.socket.set_inheritable`.
Victor Stinnerdaf45552013-08-28 00:53:59 +02001005
R David Murray33ef1ae2013-12-31 16:04:50 -05001006The ``socket.AF_*`` and ``socket.SOCK_*`` constants are now enumeration values
1007using the new :mod:`enum` module. This allows meaningful names to be printed
1008during debugging, instead of integer "magic numbers".
Victor Stinnerdaf45552013-08-28 00:53:59 +02001009
R David Murray66bf12a2014-01-08 17:21:22 -05001010The :data:`~socket.AF_LINK` constant is now available on BSD and OSX.
1011
R David Murray8b2d6822013-12-31 15:06:05 -05001012
R David Murray9025f1c2014-01-02 13:44:18 -05001013sqlite3
1014-------
1015
1016A new boolean parameter, *uri*, to the :func:`~sqlite3.connect` function can
1017be used to indicate that the *database* parameter is a ``uri`` (see
1018the `SQLite URI documentation <http://www.sqlite.org/uri.html>`_).
1019(Contributed by poq in :issue:`13773`.)
1020
1021
Christian Heimes24cd4cf2013-06-22 19:31:58 +02001022ssl
1023---
1024
R David Murray748bad22013-12-20 17:08:39 -05001025.. _whatsnew-tls-11-12:
Christian Heimes24cd4cf2013-06-22 19:31:58 +02001026
R David Murray748bad22013-12-20 17:08:39 -05001027:data:`~ssl.PROTOCOL_TLSv1_1` and :data:`~ssl.PROTOCOL_TLSv1_2` (TLSv1.1 and
1028TLSv1.2 support) have been added; support for these protocols is only available if
1029Python is linked with OpenSSL 1.0.1 or later. (Contributed by Michele Orrù and
1030Antoine Pitrou in :issue:`16692`)
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +02001031
R David Murrayf1e4fdc2014-01-21 18:30:42 -05001032New function :func:`~ssl.get_default_verify_paths` returns
1033a named tuple of the paths and environment variables that the
1034:meth:`~ssl.SSLContext.set_default_verify_paths` method uses to set
1035OpenSSL's default ``cafile`` and ``capath``. This can be an aid in
1036debugging default verification issues. (Contributed by Christian Heimes
1037in :issue:`18143`.)
1038
1039:class:`~ssl.SSLContext` has a new method,
1040:meth:`~ssl.SSLContext.cert_store_stats`, that reports the number of loaded
1041``X.509`` certs, ``X.509 CA`` certs, and certificate revocation lists (``crl``\
1042s), as well as a :meth:`~ssl.SSLContext.get_ca_certs` method that returns a
1043list of the loaded ``CA`` certificates. (Contributed by Christian Heimes in
1044and :issue:`18147`.)
Christian Heimes24cd4cf2013-06-22 19:31:58 +02001045
R David Murray748bad22013-12-20 17:08:39 -05001046Add :func:`ssl.enum_cert_store` to retrieve certificates and CRL from Windows'
1047cert store. (Contributed by Christian Heimes in :issue:`17134`.)
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +02001048
1049Support for server-side SNI using the new
1050:meth:`ssl.SSLContext.set_servername_callback` method.
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +02001051(Contributed by Daniel Black in :issue:`8109`.)
1052
1053
Antoine Pitroud6cbd342013-08-12 20:48:15 +02001054stat
1055----
1056
1057The :mod:`stat` module is now backed by a C implementation in :mod:`_stat`. A C
1058implementation is required as most of the values aren't standardized and
1059platform-dependent. (Contributed by Christian Heimes in :issue:`11016`.)
1060
1061The module supports new file types: door, event port and whiteout.
1062
1063
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +02001064struct
1065------
1066
R David Murray98358272014-01-05 20:52:06 -05001067:mod:`struct` now supports the streamed unpacking of a buffer containing
1068repeated instances of a given format of data. Both a module level
1069:mod:`~struct.iter_unpack` function and a :meth:`struct.Struct.iter_unpack`
1070method on compiled formats have been added. (Contributed by Antoine Pitrou in
1071:issue:`17804`.)
1072
1073
1074subprocess
1075----------
1076
1077:func:`~subprocess.check_output` now accepts an *input* argument that can
1078be used to provide the contents of ``stdin`` for the command that is run.
1079(Contributed by Zack Weinberg in :issue:`16624`.)
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +02001080
1081
Serhiy Storchakae06a8962013-09-04 00:43:03 +03001082sunau
1083-----
1084
1085The :meth:`~sunau.getparams` method now returns a namedtuple rather than a
1086plain tuple. (Contributed by Claudiu Popa in :issue:`18901`.)
1087
Serhiy Storchaka34d20132013-09-05 17:01:53 +03001088:meth:`sunau.open` now supports the context manager protocol (:issue:`18878`).
1089
Serhiy Storchakae06a8962013-09-04 00:43:03 +03001090
R David Murrayd17aba72013-12-24 14:46:23 -05001091sys
1092---
1093
1094New function :func:`sys.getallocatedblocks` returns the current number of
1095blocks allocated by the interpreter (in CPython with the default
1096``--with-pymalloc`` setting, this is allocations made through the
1097:c:func:`PyObject_Malloc` API). This can be useful for tracking memory leaks,
1098especially if automated via a test suite. (Contributed by Antoine Pitrou
1099in :issue:`13390`.)
1100
1101
Andrew Kuchling173a1572013-09-15 18:15:56 -04001102traceback
1103---------
1104
1105A new :func:`traceback.clear_frames` function takes a traceback object
1106and clears the local variables in all of the frames it references,
1107reducing the amount of memory consumed (:issue:`1565525`).
1108
1109
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +02001110urllib
1111------
1112
R David Murray473f45b2013-12-27 17:01:16 -05001113:mod:`urllib.request` now supports ``data:`` URLs via the
1114:class:`~urllib.request.DataHandler` class. (Contributed by Mathias Panzenböck
1115in :issue:`16423`.)
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +02001116
1117
1118unittest
1119--------
1120
R David Murray57fcf762014-01-03 23:31:54 -05001121The :class:`~unittest.TestCase` class has a new method,
1122:meth:`~unittest.TestCase.subTest`, that produces a context manager whose
1123:keyword:`with` block becomes a "sub-test". This context manager allows a test
1124method to dynamically generate subtests by, say, calling the ``subTest``
1125context manager inside a loop. A single test method can thereby produce an
1126indefinite number of separately-identified and separately-counted tests, all of
1127which will run even if one or more of them fail. For example::
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +02001128
R David Murray57fcf762014-01-03 23:31:54 -05001129 class NumbersTest(unittest.TestCase):
1130 def test_even(self):
1131 for i in range(6):
1132 with self.subTest(i=1):
1133 self.assertEqual(i % 2, 0)
1134
1135will result in six subtests, each identified in the unittest verbose output
1136with a label consisting of the variable name ``i`` and a particular value for
1137that variable (``i=0``, ``i=1``, etc). See :ref:`subtests` for the full
1138version of this example. (Contributed by Antoine Pitrou in :issue:`16997`.)
1139
1140:func:`unittest.main` now accepts an iterable of test names for
R David Murray9025f1c2014-01-02 13:44:18 -05001141*defaultTest*, where previously it only accepted a single test name as a
1142string. (Contributed by Jyrki Pulliainen in :issue:`15132`.)
1143
R David Murray42fa1102014-01-03 13:03:36 -05001144If :class:`~unittest.SkipTest` is raised during test discovery (that is, at the
1145module level in the test file), it is now reported as a skip instead of an
1146error. (Contributed by Zach Ware in :issue:`16935`.)
1147
R David Murraydb085f52014-01-03 15:46:24 -05001148:meth:`~unittest.TestLoader.discover` now sorts the discovered files to provide
1149consistent test ordering. (Contributed by Martin Melin and Jeff Ramnani in
1150:issue:`16709`.)
1151
R David Murray8e37d5d2013-04-13 14:49:48 -04001152
R David Murray575fb312013-12-25 23:21:03 -05001153venv
1154----
1155
1156:mod:`venv` now includes activation scripts for the ``csh`` and ``fish``
1157shells (Contributed by Andrew Svetlov in :issue:`15417`.)
1158
1159
R David Murray671cd322013-04-10 12:31:43 -04001160wave
1161----
1162
1163The :meth:`~wave.getparams` method now returns a namedtuple rather than a
1164plain tuple. (Contributed by Claudiu Popa in :issue:`17487`.)
1165
R David Murrayc91d5ee2013-07-31 13:46:08 -04001166:meth:`wave.open` now supports the context manager protocol. (Contributed
1167by Claudiu Popa in :issue:`17616`.)
1168
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +02001169
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +02001170weakref
1171-------
1172
1173New :class:`~weakref.WeakMethod` class simulates weak references to bound
Nick Coghlanbe57ab82013-09-22 21:26:30 +10001174methods. (Contributed by Antoine Pitrou in :issue:`14631`.)
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +02001175
Nick Coghlanbe57ab82013-09-22 21:26:30 +10001176New :class:`~weakref.finalize` class makes it possible to register a callback
1177to be invoked when an object is garbage collected, without needing to
1178carefully manage the lifecycle of the weak reference itself. (Contributed by
1179Richard Oudkerk in :issue:`15528`)
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +02001180
R David Murraya101bdb2014-01-06 16:32:05 -05001181The callback, if any, associated with a :class:`~weakref.ref` is now
1182exposed via the :attr:`~weakref.ref.__callback__` attribute. (Contributed
1183by Mark Dickinson in :issue:`17643`.)
1184
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +02001185
1186xml.etree
1187---------
1188
R David Murray410d3202014-01-04 23:52:50 -05001189A new parser, :class:`~xml.etree.ElementTree.XMLPullParser`, allows a
1190non-blocking applications to parse XML documents. An example can be
1191seen at :ref:`elementtree-pull-parsing`. (Contributed by Antoine
1192Pitrou in :issue:`17741`.)
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +02001193
R David Murray575fb312013-12-25 23:21:03 -05001194The :mod:`xml.etree.ElementTree` :func:`~xml.etree.ElementTree.tostring` and
1195:func:`~xml.etree.ElementTree.tostringlist` functions, and the
1196:class:`~xml.etree.ElementTree.ElementTree`
1197:meth:`~xml.etree.ElementTree.ElementTree.write` method, now have a
1198*short_empty_elements* :ref:`keyword-only parameter <keyword-only_parameter>`
1199providing control over whether elements with no content are written in
1200abbreviated (``<tag />``) or expanded (``<tag></tag>``) form. (Contributed by
1201Ariel Poliak and Serhiy Storchaka in :issue:`14377`.)
1202
Christian Tismer59202e52013-10-21 03:59:23 +02001203
1204zipfile.PyZipfile
1205-----------------
1206
1207Add a filter function to ignore some packages (tests for instance),
1208:meth:`~zipfile.PyZipFile.writepy`.
Christian Tismer59202e52013-10-21 03:59:23 +02001209(Contributed by Christian Tismer in :issue:`19274`.)
1210
1211
R David Murraye6082552014-01-03 16:15:45 -05001212
Nick Coghlan367df122013-10-27 01:57:34 +10001213CPython Implementation Changes
1214==============================
1215
1216
R David Murrayc16dfe12013-12-21 12:32:10 -05001217.. _whatsnew-pep-445:
Nick Coghlan367df122013-10-27 01:57:34 +10001218
R David Murraye7cf6782013-12-24 14:51:25 -05001219PEP 445: Customization of CPython Memory Allocators
Nick Coghlan367df122013-10-27 01:57:34 +10001220---------------------------------------------------
1221
1222:pep:`445` adds new C level interfaces to customize memory allocation in
1223the CPython interpreter.
1224
1225.. seealso::
1226
R David Murray061efb12013-12-24 12:35:59 -05001227 :pep:`445` -- Add new APIs to customize Python memory allocators
Nick Coghlan367df122013-10-27 01:57:34 +10001228 PEP written and implemented by Victor Stinner.
1229
1230
R David Murrayc16dfe12013-12-21 12:32:10 -05001231.. _whatsnew-pep-442:
Nick Coghlan367df122013-10-27 01:57:34 +10001232
R David Murraye7cf6782013-12-24 14:51:25 -05001233PEP 442: Safe Object Finalization
Nick Coghlan367df122013-10-27 01:57:34 +10001234---------------------------------
1235
1236:pep:`442` removes the current limitations and quirks of object finalization
1237in CPython. With it, objects with :meth:`__del__` methods, as well as
1238generators with :keyword:`finally` clauses, can be finalized when they are
1239part of a reference cycle.
1240
1241As part of this change, module globals are no longer forcibly set to
1242:const:`None` during interpreter shutdown in most cases, instead relying
R David Murrayca794612013-12-22 14:05:11 -05001243on the normal operation of the cyclic garbage collector. This avoids a
1244whole class of interpreter-shutdown-time errors, usually involving
1245``__del__`` methods, that have plagued Python since the cyclic GC
1246was first introduced.
Nick Coghlan367df122013-10-27 01:57:34 +10001247
1248.. seealso::
1249
R David Murray061efb12013-12-24 12:35:59 -05001250 :pep:`442` -- Safe object finalization
Nick Coghlan367df122013-10-27 01:57:34 +10001251 PEP written and implemented by Antoine Pitrou.
1252
1253
R David Murrayc16dfe12013-12-21 12:32:10 -05001254.. _whatsnew-pep-456:
1255
1256PEP 456: Secure and Interchangeable Hash Algorithm
1257--------------------------------------------------
1258
1259:pep:`456` follows up on earlier security fix work done on Python's hash
1260algorithm to address certain DOS attacks to which public facing APIs backed by
1261dictionary lookups may be subject. (See :issue:`14621` for the start of the
1262current round of improvements.) The PEP unifies CPython's hash code to make it
1263easier for a packager to substitute a different hash algorithm, and switches
1264Python's default implementation to a SipHash implementation on platforms that
1265have a 64 bit data type. Any performance differences in comparison with the
1266older FNV algorithm are trivial.
1267
1268The PEP adds additional fields to the :func:`sys.hash_info` struct sequence to
1269describe the hash algorithm in use by the currently executing binary. Otherwise,
1270the PEP does not alter any existing CPython APIs.
1271
1272
R David Murray6dd18302013-12-24 12:23:56 -05001273.. _whatsnew-pep-436:
1274
1275PEP 436: Argument Clinic
1276------------------------
1277
1278"Argument Clinic" (:pep:`436`) is now part of the CPython build process
1279and can be used to simplify the process of defining and maintaining
1280accurate signatures for builtins and standard library extension modules
1281implemented in C.
1282
1283.. note::
1284 The Argument Clinic PEP is not fully up to date with the state of the
1285 implementation. This has been deemed acceptable by the release manager
1286 and core development team in this case, as Argument Clinic will not
1287 be made available as a public API for third party use in Python 3.4.
1288
1289.. seealso::
1290
R David Murray061efb12013-12-24 12:35:59 -05001291 :pep:`436` -- The Argument Clinic DSL
R David Murray6dd18302013-12-24 12:23:56 -05001292 PEP written and implemented by Larry Hastings.
1293
1294
R David Murraye7cf6782013-12-24 14:51:25 -05001295Other Build and C API Changes
Nick Coghlan367df122013-10-27 01:57:34 +10001296-----------------------------
Georg Brandlb80f5112012-09-30 09:11:58 +02001297
Nick Coghlan7d270ee2013-10-17 22:35:35 +10001298* The new :c:func:`Py_SetStandardStreamEncoding` pre-initialization API
1299 allows applications embedding the CPython interpreter to reliably force
1300 a particular encoding and error handler for the standard streams
1301 (Contributed by Bastien Montagne and Nick Coghlan in :issue:`16129`)
Georg Brandlb80f5112012-09-30 09:11:58 +02001302
Nick Coghlan0acceb72013-10-20 13:22:21 +10001303* Most Python C APIs that don't mutate string arguments are now correctly
1304 marked as accepting ``const char *`` rather than ``char *`` (Contributed
1305 by Serhiy Storchaka in :issue:`1772673`).
1306
R David Murrayd91ba202013-12-24 12:13:44 -05001307* New shell version of ``python-config``; can be used even when a python
1308 interpreter is not available (for example, in cross compilation scenarios).
1309
Georg Brandlb80f5112012-09-30 09:11:58 +02001310
R David Murraye6082552014-01-03 16:15:45 -05001311
R David Murraye7cf6782013-12-24 14:51:25 -05001312Other Improvements
R David Murraybcaaecf2013-12-23 21:23:36 -05001313==================
1314
1315* Tab-completion is now enabled by default in the interactive interpreter.
1316 (Contributed by Antoine Pitrou and Éric Araujo in :issue:`5845`.)
1317
1318* Invoking the Python interpreter with ``--version`` now outputs the version to
1319 standard output instead of standard error (:issue:`18338`). Similar changes
1320 were made to :mod:`argparse` (:issue:`18920`) and other modules that have
1321 script-like invocation capabilities (:issue:`18922`).
1322
R David Murray72420ff2013-12-24 10:46:44 -05001323* The CPython Windows installer now adds ``.py`` to the :envvar:`PATHEXT`
1324 variable when extensions are registered, allowing users to run a python
1325 script at the windows command prompt by just typing its name without the
1326 ``.py`` extension. (Contributed by Paul Moore in :issue:`18569`.)
1327
R David Murrayd17aba72013-12-24 14:46:23 -05001328* A new ``make`` target `coverage-report
1329 <http://docs.python.org/devguide/coverage.html#measuring-coverage-of-c-code-with-gcov-and-lcov>`_
1330 will build python, run the test suite, and generate an HTML coverage report
1331 for the C codebase using ``gcov`` and `lcov
1332 <http://ltp.sourceforge.net/coverage/lcov.php>`_.
1333
1334* The ``-R`` option to the :ref:`python regression test suite <regrtest>` now
1335 also checks for memory allocation leaks, using
1336 :func:`sys.getallocatedblocks()`. (Contributed by Antoine Pitrou in
1337 :issue:`13390`).
1338
R David Murraybcaaecf2013-12-23 21:23:36 -05001339
R David Murraye6082552014-01-03 16:15:45 -05001340
R David Murrayd74d09a2013-12-24 15:33:02 -05001341Significant Optimizations
1342=========================
R David Murraybcaaecf2013-12-23 21:23:36 -05001343
R David Murray4908f4a2014-01-04 18:07:20 -05001344* The UTF-32 decoder is now 3x to 4x faster. (Contributed by Serhiy Storchaka
1345 in :issue:`14625`.)
R David Murraybcaaecf2013-12-23 21:23:36 -05001346
1347* The cost of hash collisions for sets is now reduced. Each hash table
1348 probe now checks a series of consecutive, adjacent key/hash pairs before
1349 continuing to make random probes through the hash table. This exploits
1350 cache locality to make collision resolution less expensive.
R David Murraybcaaecf2013-12-23 21:23:36 -05001351 The collision resolution scheme can be described as a hybrid of linear
1352 probing and open addressing. The number of additional linear probes
1353 defaults to nine. This can be changed at compile-time by defining
1354 LINEAR_PROBES to be any value. Set LINEAR_PROBES=0 to turn-off
R David Murray59171722014-01-03 15:52:22 -05001355 linear probing entirely. (Contributed by Raymond Hettinger in
1356 :issue:`18771`.)
R David Murraybcaaecf2013-12-23 21:23:36 -05001357
1358* The interpreter starts about 30% faster. A couple of measures lead to the
1359 speedup. The interpreter loads fewer modules on startup, e.g. the :mod:`re`,
1360 :mod:`collections` and :mod:`locale` modules and their dependencies are no
1361 longer imported by default. The marshal module has been improved to load
R David Murray59171722014-01-03 15:52:22 -05001362 compiled Python code faster. (Contributed by Antoine Pitrou, Christian
1363 Heimes and Victor Stinner in :issue:`19219`, :issue:`19218`, :issue:`19209`,
1364 :issue:`19205` and :issue:`9548`)
R David Murraybcaaecf2013-12-23 21:23:36 -05001365
R David Murray6e390152013-12-24 22:28:04 -05001366* :class:`bz2.BZ2File` is now as fast or faster than the Python2 version for
1367 most cases. :class:`lzma.LZMAFile` has also been optimized. (Contributed by
1368 Serhiy Storchaka and Nadeem Vawda in :issue:`16034`.)
1369
R David Murray8b2d6822013-12-31 15:06:05 -05001370* :func:`random.getrandbits` is 20%-40% faster for small integers (the most
1371 common use case). (Contributed by Serhiy Storchaka in :issue:`16674`).
1372
R David Murrayb231b2b2014-01-04 17:11:23 -05001373* By taking advantage of the new storage format for strings, pickling of
1374 strings is now significantly faster. (Contributed by Victor Stinner and
1375 Antoine Pitrou in :issue:`15596`.)
1376
R David Murray66bf12a2014-01-08 17:21:22 -05001377* A performance issue in :meth:`io.FileIO.readall` has been solved. This
1378 particularly affects Windows, and significantly speeds up the case of piping
1379 significant amounts of data through :mod:`subprocess`. (Contributed
1380 by Richard Oudkerk in :issue:`15758`.)
1381
R David Murraybcaaecf2013-12-23 21:23:36 -05001382
R David Murraye6082552014-01-03 16:15:45 -05001383
Georg Brandlb80f5112012-09-30 09:11:58 +02001384Deprecated
1385==========
1386
R David Murrayd74d09a2013-12-24 15:33:02 -05001387This section covers various APIs and other features that have been deprecated
1388in Python 3.4, and will be removed in Python 3.5 or later. In most (but not
1389all) cases, using the deprecated APIs will produce a :exc:`DeprecationWarning`
1390when the interpreter is run with deprecation warnings enabled (for example, by
1391using ``-Wd``).
1392
Georg Brandlb80f5112012-09-30 09:11:58 +02001393
R David Murraye7cf6782013-12-24 14:51:25 -05001394Deprecated Python Modules, Functions and Methods
Georg Brandlb80f5112012-09-30 09:11:58 +02001395------------------------------------------------
1396
Terry Jan Reedy2b6c26e2013-03-21 19:36:26 -04001397* :meth:`difflib.SequenceMatcher.isbjunk` and
Andrew Kuchling0d0813a2013-06-15 13:29:09 -04001398 :meth:`difflib.SequenceMatcher.isbpopular` were removed: use ``x in sm.bjunk`` and
1399 ``x in sm.bpopular``, where *sm* is a :class:`~difflib.SequenceMatcher` object.
Georg Brandlb80f5112012-09-30 09:11:58 +02001400
Brett Cannon82b3d6a2013-06-14 22:37:11 -04001401* :func:`importlib.util.module_for_loader` is pending deprecation. Using
1402 :func:`importlib.util.module_to_load` and
1403 :meth:`importlib.abc.Loader.init_module_attrs` allows subclasses of a loader
1404 to more easily customize module loading.
1405
Brett Cannone4f41de2013-06-16 13:13:40 -04001406* The :mod:`imp` module is pending deprecation. To keep compatibility with
1407 Python 2/3 code bases, the module's removal is currently not scheduled.
1408
Brett Cannon1448ecf2013-10-04 11:38:59 -04001409* The :mod:`formatter` module is pending deprecation and is slated for removal
1410 in Python 3.6.
1411
Christian Heimes634919a2013-11-20 17:23:06 +01001412* MD5 as default digestmod for :mod:`hmac` is deprecated. Python 3.6 will
1413 require an explicit digest name or constructor as *digestmod* argument.
1414
R David Murray9025f1c2014-01-02 13:44:18 -05001415* The internal ``Netrc`` class in the :mod:`ftplib` module has been documented
1416 as deprecated in its docstring for quite some time. It now emits a
1417 :exc:`DeprecationWarning` and will be removed completely in Python 3.5.
1418
Georg Brandlb80f5112012-09-30 09:11:58 +02001419
R David Murraye7cf6782013-12-24 14:51:25 -05001420Deprecated Functions and Types in the C API
Georg Brandlb80f5112012-09-30 09:11:58 +02001421-------------------------------------------
1422
R David Murrayd74d09a2013-12-24 15:33:02 -05001423XXX: None so far
Georg Brandlb80f5112012-09-30 09:11:58 +02001424
1425
R David Murraye7cf6782013-12-24 14:51:25 -05001426Deprecated Features
Georg Brandlb80f5112012-09-30 09:11:58 +02001427-------------------
1428
R David Murray5ea95242013-12-24 15:59:57 -05001429* Running :ref:`idle` with the ``-n`` flag (no subprocess) is deprecated.
1430 However, the feature will not be removed until :issue:`18823` is resolved.
1431
Antoine Pitrou3b2f0f02013-10-25 21:39:26 +02001432* The site module adding a "site-python" directory to sys.path, if it
1433 exists, is deprecated (:issue:`19375`).
Georg Brandlb80f5112012-09-30 09:11:58 +02001434
1435
R David Murray59171722014-01-03 15:52:22 -05001436
R David Murray72420ff2013-12-24 10:46:44 -05001437Removed
1438=======
1439
R David Murray59171722014-01-03 15:52:22 -05001440
1441Operating Systems No Longer Supported
1442-------------------------------------
1443
1444Support for the following operating systems has been removed from the source
1445and build tools:
1446
1447* OS/2 (:issue:`16135`).
1448* Windows 2000 (changeset e52df05b496a).
1449* VMS (:issue:`16136`).
1450
1451
1452API and Feature Removals
1453------------------------
1454
R David Murrayd2653812013-12-27 14:06:15 -05001455The following obsolete and previously deprecated APIs and features have been
R David Murray59171722014-01-03 15:52:22 -05001456removed:
R David Murray076dead2013-12-24 14:59:50 -05001457
R David Murray72420ff2013-12-24 10:46:44 -05001458* The unmaintained ``Misc/TextMate`` and ``Misc/vim`` directories have been
R David Murrayd91ba202013-12-24 12:13:44 -05001459 removed (see the `devguide <http://docs.python.org/devguide>`_
R David Murray00569362014-01-03 13:04:25 -05001460 for suggestions on what to use instead).
R David Murray72420ff2013-12-24 10:46:44 -05001461
R David Murrayd17aba72013-12-24 14:46:23 -05001462* The ``SO`` makefile macro is removed (it was replaced by the
1463 ``SHLIB_SUFFIX`` and ``EXT_SUFFIX`` macros) (:issue:`16754`).
1464
R David Murrayd74d09a2013-12-24 15:33:02 -05001465* The ``PyThreadState.tick_counter`` field has been removed; its value has
1466 been meaningless since Python 3.2, when the "new GIL" was introduced.
1467
R David Murrayd2653812013-12-27 14:06:15 -05001468* ``PyLoader`` and ``PyPycLoader`` have been removed from :mod:`importlib`.
1469 (Contributed by Taras Lyapun in :issue:`15641`.)
1470
R David Murraya2924ca2014-01-03 14:06:01 -05001471* The *strict* argument to :class:`~http.client.HTTPConnection` and
1472 :class:`~http.client.HTTPSConnection` has been removed. HTTP 0.9-style
1473 "Simple Responses" are no longer supported.
R David Murray00569362014-01-03 13:04:25 -05001474
R David Murraydb085f52014-01-03 15:46:24 -05001475* The deprecated :mod:`urllib.request.Request` getter and setter methods
1476 ``add_data``, ``has_data``, ``get_data``, ``get_type``, ``get_host``,
1477 ``get_selector``, ``set_proxy``, ``get_origin_req_host``, and
1478 ``is_unverifiable`` have been removed (use direct attribute access instead).
1479
R David Murray244ad602014-01-04 21:17:52 -05001480* Support for loading the deprecated ``TYPE_INT64`` has been removed from
1481 :mod:`marshal`. (Contributed by Dan Riti in :issue:`15480`.)
1482
R David Murray72420ff2013-12-24 10:46:44 -05001483
R David Murray98358272014-01-05 20:52:06 -05001484Code Cleanups
1485-------------
1486
1487* The unused and undocumented internal ``Scanner`` class has been removed from
1488 the :mod:`pydoc` module.
1489
1490
R David Murraye6082552014-01-03 16:15:45 -05001491
Benjamin Peterson88f3b232012-10-04 12:45:10 -04001492Porting to Python 3.4
Georg Brandlb80f5112012-09-30 09:11:58 +02001493=====================
1494
1495This section lists previously described changes and other bugfixes
1496that may require changes to your code.
1497
R David Murrayd74d09a2013-12-24 15:33:02 -05001498Changes in the Python API
1499-------------------------
1500
Brett Cannon777622b2013-04-09 17:03:10 -04001501* The ABCs defined in :mod:`importlib.abc` now either raise the appropriate
1502 exception or return a default value instead of raising
1503 :exc:`NotImplementedError` blindly. This will only affect code calling
1504 :func:`super` and falling through all the way to the ABCs. For compatibility,
1505 catch both :exc:`NotImplementedError` or the appropriate exception as needed.
Brett Cannon4c14b5d2013-05-04 13:56:58 -04001506
1507* The module type now initializes the :attr:`__package__` and :attr:`__loader__`
1508 attributes to ``None`` by default. To determine if these attributes were set
1509 in a backwards-compatible fashion, use e.g.
Brett Cannon3dc48d62013-05-28 18:35:54 -04001510 ``getattr(module, '__loader__', None) is not None``.
1511
1512* :meth:`importlib.util.module_for_loader` now sets ``__loader__`` and
1513 ``__package__`` unconditionally to properly support reloading. If this is not
1514 desired then you will need to set these attributes manually. You can use
Brett Cannon028d5122013-05-31 18:02:11 -04001515 :func:`importlib.util.module_to_load` for module management.
Brett Cannon3e0651b2013-05-31 23:18:39 -04001516
1517* Import now resets relevant attributes (e.g. ``__name__``, ``__loader__``,
1518 ``__package__``, ``__file__``, ``__cached__``) unconditionally when reloading.
1519
Brett Cannon1448ecf2013-10-04 11:38:59 -04001520* Frozen packages no longer set ``__path__`` to a list containing the package
1521 name but an empty list instead. Determing if a module is a package should be
1522 done using ``hasattr(module, '__path__')``.
Brett Cannon8f5ac512013-06-12 23:29:18 -04001523
Brett Cannon33915eb2013-06-14 18:33:00 -04001524* :func:`py_compile.compile` now raises :exc:`FileExistsError` if the file path
1525 it would write to is a symlink or a non-regular file. This is to act as a
1526 warning that import will overwrite those files with a regular file regardless
1527 of what type of file path they were originally.
Brett Cannonf4375ef2013-06-16 18:05:54 -04001528
1529* :meth:`importlib.abc.SourceLoader.get_source` no longer raises
1530 :exc:`ImportError` when the source code being loaded triggers a
1531 :exc:`SyntaxError` or :exc:`UnicodeDecodeError`. As :exc:`ImportError` is
1532 meant to be raised only when source code cannot be found but it should, it was
1533 felt to be over-reaching/overloading of that meaning when the source code is
1534 found but improperly structured. If you were catching ImportError before and
1535 wish to continue to ignore syntax or decoding issues, catch all three
Victor Stinner84e33c82013-06-21 00:31:55 +02001536 exceptions now.
Nick Coghlan24c05bc2013-07-15 21:13:08 +10001537
1538* :func:`functools.update_wrapper` and :func:`functools.wraps` now correctly
Nick Coghlan367df122013-10-27 01:57:34 +10001539 set the ``__wrapped__`` attribute to the function being wrapper, even if
1540 that function also had its ``__wrapped__`` attribute set. This means
1541 ``__wrapped__`` attributes now correctly link a stack of decorated
1542 functions rather than every ``__wrapped__`` attribute in the chain
1543 referring to the innermost function. Introspection libraries that
1544 assumed the previous behaviour was intentional can use
1545 :func:`inspect.unwrap` to access the first function in the chain that has
1546 no ``__wrapped__`` attribute.
Victor Stinner2fe9bac2013-10-10 16:18:20 +02001547
Georg Brandl0f5bff22013-10-19 17:46:38 +02001548* :class:`importlib.machinery.PathFinder` now passes on the current working
Brett Cannon27e27f72013-10-18 11:39:04 -04001549 directory to objects in :data:`sys.path_hooks` for the empty string. This
1550 results in :data:`sys.path_importer_cache` never containing ``''``, thus
1551 iterating through :data:`sys.path_importer_cache` based on :data:`sys.path`
1552 will not find all keys. A module's ``__file__`` when imported in the current
1553 working directory will also now have an absolute path, including when using
1554 ``-m`` with the interpreter (this does not influence when the path to a file
1555 is specified on the command-line).
Victor Stinner2748bc72013-12-13 10:57:04 +01001556
R David Murraya2924ca2014-01-03 14:06:01 -05001557* The removal of the *strict* argument to :class:`~http.client.HTTPConnection`
1558 and :class:`~http.client.HTTPSConnection` changes the meaning of the
1559 remaining arguments if you are specifying them positionally rather than by
1560 keyword. If you've been paying attention to deprecation warnings your code
1561 should already be specifying any additional arguments via keywords.
1562
R David Murray4908f4a2014-01-04 18:07:20 -05001563* Strings between ``from __future__ import ...`` statements now *always* raise
1564 a :exc:`SyntaxError`. Previously if there was no leading docstring, an
1565 interstitial string would sometimes be ignored. This brings CPython into
1566 compliance with the language spec; Jython and PyPy already were.
1567 (:issue:`17434`).
1568
R David Murraya101bdb2014-01-06 16:32:05 -05001569* :meth:`ssl.SSLSocket.getpeercert` and :meth:`ssl.SSLSocket.do_handshake`
1570 now raise an :exc:`OSError` with ``ENOTCONN`` when the ``SSLSocket`` is not
1571 connected, instead of the previous behavior of raising an
1572 :exc:`AttributError`. In addition, :meth:`~ssl.SSLSocket.getpeercert`
1573 will raise a :exc:`ValueError` if the handshake has not yet been done.
1574
R David Murray0ae7ae12014-01-08 18:16:02 -05001575* :func:`base64.b32decode` now raises a :exc:`binascii.Error` when the
1576 input string contains non-b32-alphabet characters, instead of a
1577 :exc:`TypeError`. This particular :exc:`TypeError` was missed when the other
1578 :exc:`TypeError`\ s were converted. (Contributed by Serhiy Storchaka in
1579 :issue:`18011`.) Note: this change was also inadvertently applied in Python
1580 3.3.3.
1581
Brett Cannonc089f702014-01-17 11:03:19 -05001582* The :attr:`~cgi.FieldStorage.file` attribute is now automatically closed when
1583 the creating :class:`cgi.FieldStorage` instance is garbage collected. If you
1584 were pulling the file object out separately from the :class:`cgi.FieldStorage`
1585 instance and not keeping the instance alive, then you should either store the
1586 entire :class:`cgi.FieldStorage` instance or read the contents of the file
1587 before the :class:`cgi.FieldStorage` instance is garbage collected.
1588
R David Murrayd2653812013-12-27 14:06:15 -05001589
Victor Stinner774b2e02013-12-13 14:33:01 +01001590Changes in the C API
1591--------------------
1592
1593* :c:func:`PyErr_SetImportError` now sets :exc:`TypeError` when its **msg**
1594 argument is not set. Previously only ``NULL`` was returned with no exception
1595 set.
1596
1597* The result of the :c:data:`PyOS_ReadlineFunctionPointer` callback must
1598 now be a string allocated by :c:func:`PyMem_RawMalloc` or
1599 :c:func:`PyMem_RawRealloc`, or *NULL* if an error occurred, instead of a
1600 string allocated by :c:func:`PyMem_Malloc` or :c:func:`PyMem_Realloc`.
1601
1602* :c:func:`PyThread_set_key_value` now always set the value. In Python
Victor Stinner2748bc72013-12-13 10:57:04 +01001603 3.3, the function did nothing if the key already exists (if the current
1604 value is a non-NULL pointer).
1605
Victor Stinner774b2e02013-12-13 14:33:01 +01001606* The ``f_tstate`` (thread state) field of the :c:type:`PyFrameObject`
1607 structure has been removed to fix a bug: see :issue:`14432` for the
1608 rationale.
1609