blob: a5bbdf029ebb9b923b1cf7221d429a9281a19ead [file] [log] [blame]
Georg Brandlb80f5112012-09-30 09:11:58 +02001****************************
2 What's New In Python 3.4
3****************************
4
Georg Brandl3b80d342012-10-28 13:37:54 +01005.. :Author: Someone <email>
6 (uncomment if there is a principal author)
Georg Brandlb80f5112012-09-30 09:11:58 +02007
8.. Rules for maintenance:
9
Nick Coghlan03074fd2012-09-30 18:51:53 +053010 * Anyone can add text to this document, but the maintainer reserves the
11 right to rewrite any additions. In particular, for obscure or esoteric
12 features, the maintainer may reduce any addition to a simple reference to
13 the new documentation rather than explaining the feature inline.
Georg Brandlb80f5112012-09-30 09:11:58 +020014
Nick Coghlan03074fd2012-09-30 18:51:53 +053015 * While the maintainer will periodically go through Misc/NEWS
16 and add changes, it's best not to rely on this. We know from experience
17 that any changes that aren't in the What's New documentation around the
18 time of the original release will remain largely unknown to the community
19 for years, even if they're added later. We also know from experience that
20 other priorities can arise, and the maintainer will run out of time to do
R David Murray061efb12013-12-24 12:35:59 -050021 updates -- in such cases, end users will be much better served by partial
Nick Coghlan03074fd2012-09-30 18:51:53 +053022 notifications that at least give a hint about new features to
23 investigate.
Georg Brandlb80f5112012-09-30 09:11:58 +020024
25 * This is not a complete list of every single change; completeness
Nick Coghlan03074fd2012-09-30 18:51:53 +053026 is the purpose of Misc/NEWS. The What's New should focus on changes that
27 are visible to Python *users* and that *require* a feature release (i.e.
28 most bug fixes should only be recorded in Misc/NEWS)
29
30 * PEPs should not be marked Final until they have an entry in What's New.
31 A placeholder entry that is just a section header and a link to the PEP
32 (e.g ":pep:`397` has been implemented") is acceptable. If a PEP has been
33 implemented and noted in What's New, don't forget to mark it as Final!
Georg Brandlb80f5112012-09-30 09:11:58 +020034
35 * If you want to draw your new text to the attention of the
36 maintainer, add 'XXX' to the beginning of the paragraph or
37 section.
38
Nick Coghlan03074fd2012-09-30 18:51:53 +053039 * It's OK to add just a very brief note about a change. For
40 example: "The :ref:`~socket.transmogrify()` function was added to the
41 :mod:`socket` module." The maintainer will research the change and
42 write the necessary text (if appropriate). The advantage of doing this
43 is that even if no more descriptive text is ever added, readers will at
44 least have a notification that the new feature exists and a link to the
45 relevant documentation.
Georg Brandlb80f5112012-09-30 09:11:58 +020046
47 * You can comment out your additions if you like, but it's not
48 necessary (especially when a final release is some months away).
49
50 * Credit the author of a patch or bugfix. Just the name is
51 sufficient; the e-mail address isn't necessary.
52
53 * It's helpful to add the bug/patch number as a comment:
54
Nick Coghlan03074fd2012-09-30 18:51:53 +053055 The :ref:`~socket.transmogrify()` function was added to the
56 :mod:`socket` module. (Contributed by P.Y. Developer in :issue:`12345`.)
Georg Brandlb80f5112012-09-30 09:11:58 +020057
58 This saves the maintainer the effort of going through the Mercurial log
59 when researching a change.
60
Nick Coghlan03074fd2012-09-30 18:51:53 +053061 * Cross referencing tip: :ref:`mod.attr` will display as ``mod.attr``,
62 while :ref:`~mod.attr` will display as ``attr``.
63
Georg Brandlb80f5112012-09-30 09:11:58 +020064This article explains the new features in Python 3.4, compared to 3.3.
65
Nick Coghlan03074fd2012-09-30 18:51:53 +053066.. Python 3.4 was released on TBD.
Georg Brandlb80f5112012-09-30 09:11:58 +020067
Ezio Melotti25bbe5e2012-11-17 19:30:48 +020068For full details, see the
69`changelog <http://docs.python.org/3.4/whatsnew/changelog.html>`_.
Georg Brandlb80f5112012-09-30 09:11:58 +020070
71.. note:: Prerelease users should be aware that this document is currently in
72 draft form. It will be updated substantially as Python 3.4 moves towards
73 release, so it's worth checking back even after reading earlier versions.
74
75
76.. seealso::
77
R David Murray061efb12013-12-24 12:35:59 -050078 :pep:`429` -- Python 3.4 Release Schedule
Georg Brandlb80f5112012-09-30 09:11:58 +020079
80
R David Murraye6082552014-01-03 16:15:45 -050081
R David Murraye7cf6782013-12-24 14:51:25 -050082Summary -- Release Highlights
Georg Brandlb80f5112012-09-30 09:11:58 +020083=============================
84
Kristjan Valur Jonssona1e82442013-03-26 13:56:14 +000085.. This section singles out the most important changes in Python 3.4.
Georg Brandlb80f5112012-09-30 09:11:58 +020086 Brevity is key.
87
88New syntax features:
89
R David Murraye0f7a782013-12-20 16:04:29 -050090* No new syntax features were added in Python 3.4.
Georg Brandlb80f5112012-09-30 09:11:58 +020091
R David Murray809487e2013-12-22 20:49:40 -050092New expected features for Python implementations:
93
R David Murray68790662013-12-23 11:17:51 -050094* :ref:`pip should always be "available" <whatsnew-pep-453>` (:pep:`453`).
R David Murray809487e2013-12-22 20:49:40 -050095* :ref:`Make newly created file descriptors non-inheritable <whatsnew-pep-446>`
96 (:pep:`446`).
97* command line option for :ref:`isolated mode <using-on-misc-options>`,
98 (:issue:`16499`).
R David Murray8d856782013-12-23 10:28:57 -050099* :ref:`improvements in the handling of codecs <codec-handling-improvements>`
100 that are not text encodings (multiple issues).
R David Murray809487e2013-12-22 20:49:40 -0500101* :ref:`A ModuleSpec Type <whatsnew-pep-451>` for the Import System
R David Murraye9b74d42013-12-22 21:05:04 -0500102 (:pep:`451`). (Affects importer authors.)
R David Murray809487e2013-12-22 20:49:40 -0500103
Georg Brandlb80f5112012-09-30 09:11:58 +0200104New library modules:
105
R David Murray9217dad2013-12-23 21:08:28 -0500106* :mod:`asyncio`: :ref:`New provisional API for asynchronous IO
107 <whatsnew-asyncio>` (:pep:`3156`).
108* :mod:`ensurepip`: :ref:`Bootstrapping the pip installer <whatsnew-ensurepip>`
109 (:pep:`453`).
110* :mod:`enum`: :ref:`Support for enumeration types <whatsnew-enum>`
111 (:pep:`435`).
112* :mod:`pathlib`: :ref:`Object-oriented filesystem paths <whatsnew-pathlib>`
113 (:pep:`428`).
R David Murrayf9976e72013-12-23 10:32:02 -0500114* :mod:`selectors`: :ref:`High-level and efficient I/O multiplexing
115 <whatsnew-selectors>`, built upon the :mod:`select` module primitives (part
116 of :pep:`3156`).
R David Murray9217dad2013-12-23 21:08:28 -0500117* :mod:`statistics`: A basic :ref:`numerically stable statistics library
118 <whatsnew-statistics>` (:pep:`450`).
119* :mod:`tracemalloc`: :ref:`Trace Python memory allocations
120 <whatsnew-tracemalloc>` (:pep:`454`).
Georg Brandlb80f5112012-09-30 09:11:58 +0200121
Georg Brandlb80f5112012-09-30 09:11:58 +0200122Significantly Improved Library Modules:
123
R David Murray0a102162013-12-20 15:00:54 -0500124* :ref:`Single-dispatch generic functions <whatsnew-singledispatch>` in
R David Murray8d856782013-12-23 10:28:57 -0500125 :mod:`functools` (:pep:`443`).
126* New :mod:`pickle` :ref:`protocol 4 <whatsnew-protocol-4>` (:pep:`3154`).
R David Murray8d856782013-12-23 10:28:57 -0500127* :ref:`TLSv1.1 and TLSv1.2 support <whatsnew-tls-11-12>` for :mod:`ssl`
128 (:issue:`16692`).
R David Murrayac186222013-12-20 17:23:57 -0500129* :mod:`multiprocessing` now has :ref:`an option to avoid using os.fork
130 on Unix <whatsnew-multiprocessing-no-fork>` (:issue:`8713`).
R David Murray6adb4542013-12-20 13:10:43 -0500131* :mod:`email` has a new submodule, :mod:`~email.contentmanager`, and
132 a new :mod:`~email.message.Message` subclass
R David Murray26b80cfd2013-12-20 17:26:52 -0500133 (:class:`~email.contentmanager.EmailMessage`) that :ref:`simplify MIME
R David Murray8d856782013-12-23 10:28:57 -0500134 handling <whatsnew_email_contentmanager>` (:issue:`18891`).
Ronald Oussoren6db66532014-01-15 11:32:35 +0100135* :mod:`plistlib` has a cleaned up interface and support for binary
136 plist files (:issue:`14455`)
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +0200137
Nick Coghlan367df122013-10-27 01:57:34 +1000138CPython implementation improvements:
Georg Brandlb80f5112012-09-30 09:11:58 +0200139
R David Murraya93ca972013-12-22 14:10:21 -0500140* :ref:`Safe object finalization <whatsnew-pep-442>` (:pep:`442`).
R David Murrayca794612013-12-22 14:05:11 -0500141* Leveraging :pep:`442`, :ref:`module globals are no longer set to None
142 during finalization <whatsnew-pep-442>`, in most cases (:issue:`18214`).
R David Murraya93ca972013-12-22 14:10:21 -0500143* :ref:`Configurable memory allocators <whatsnew-pep-445>` (:pep:`445`).
144* :ref:`Secure and interchangeable hash algorithm <whatsnew-pep-456>`
145 (:pep:`456`).
146* :ref:`Argument Clinic <whatsnew-pep-436>` (:pep:`436`).
R David Murray8f7664a2013-12-22 20:40:11 -0500147* The :mod:`marshal` format has been made :ref:`more compact and efficient
148 <whatsnew-marshal-3>` (:issue:`16475`).
Georg Brandlb80f5112012-09-30 09:11:58 +0200149
R David Murray347f9c72013-12-23 21:30:06 -0500150Please read on for a comprehensive list of user-facing changes, including many
151other smaller improvements, CPython optimizations, deprecations, and potential
152porting issues.
Georg Brandlb80f5112012-09-30 09:11:58 +0200153
Nick Coghland0cf0632013-11-11 22:11:55 +1000154
R David Murraye6082552014-01-03 16:15:45 -0500155
R David Murraye7cf6782013-12-24 14:51:25 -0500156New Expected Features for Python Implementations
R David Murray809487e2013-12-22 20:49:40 -0500157================================================
158
R David Murray68790662013-12-23 11:17:51 -0500159.. _whatsnew-pep-453:
160
R David Murraye7cf6782013-12-24 14:51:25 -0500161PEP 453: Explicit Bootstrapping of PIP in Python Installations
R David Murray809487e2013-12-22 20:49:40 -0500162--------------------------------------------------------------
Nick Coghland0cf0632013-11-11 22:11:55 +1000163
164The new :mod:`ensurepip` module (defined in :pep:`453`) provides a standard
165cross-platform mechanism to boostrap the pip installer into Python
166installations and virtual environments.
167
Nick Coghlan7bc4b3b2013-11-23 11:59:40 +1000168The :mod:`venv` module and the :command:`pyvenv` utility make use of this
169module to make ``pip`` readily available in virtual environments. When
170using the command line interface, ``pip`` is installed by default, while
171for the module API installation of ``pip`` must be requested explicitly.
172
173For CPython source builds on POSIX systems, the ``make install`` and
174``make altinstall`` commands bootstrap ``pip`` by default. This behaviour
175can be controlled through configure options, and overridden through
176Makefile options.
177
Ned Deily44a0db02013-11-22 22:39:09 -0800178On Windows and Mac OS X, the CPython installers now offer the option to
179install ``pip`` along with CPython itself.
Nick Coghlan7bc4b3b2013-11-23 11:59:40 +1000180
R David Murray68790662013-12-23 11:17:51 -0500181As `discussed in the PEP`__, platform packagers may choose not to install
182``pip`` by default, as long as the command ``pip``, when invoked, provides
183clear and simple directions on how to install ``pip`` on the platform.
184
185__ http://www.python.org/dev/peps/pep-0453/#recommendations-for-downstream-distributors
186
Nick Coghland0cf0632013-11-11 22:11:55 +1000187.. note::
188
Nick Coghlan7bc4b3b2013-11-23 11:59:40 +1000189 The implementation of PEP 453 is still a work in progress. Refer to
190 :issue:`19347` for the progress on additional steps:
Nick Coghland0cf0632013-11-11 22:11:55 +1000191
Nick Coghlan7bc4b3b2013-11-23 11:59:40 +1000192 * Having the binary installers install ``pip`` by default
193 * Recommending the use of ``pip`` in the "Installing Python Module"
194 documentation.
Nick Coghland0cf0632013-11-11 22:11:55 +1000195
196.. seealso::
197
R David Murray061efb12013-12-24 12:35:59 -0500198 :pep:`453` -- Explicit bootstrapping of pip in Python installations
Nick Coghland0cf0632013-11-11 22:11:55 +1000199 PEP written by Donald Stufft and Nick Coghlan, implemented by
Nick Coghlan7bc4b3b2013-11-23 11:59:40 +1000200 Donald Stufft, Nick Coghlan, Martin von Löwis and Ned Deily.
Nick Coghland0cf0632013-11-11 22:11:55 +1000201
202
R David Murrayf9909c22013-12-20 14:50:12 -0500203.. _whatsnew-pep-446:
Victor Stinnerdaf45552013-08-28 00:53:59 +0200204
R David Murraye7cf6782013-12-24 14:51:25 -0500205PEP 446: Make Newly Created File Descriptors Non-Inheritable
R David Murray809487e2013-12-22 20:49:40 -0500206------------------------------------------------------------
Victor Stinnerdaf45552013-08-28 00:53:59 +0200207
Nick Coghlan367df122013-10-27 01:57:34 +1000208:pep:`446` makes newly created file descriptors :ref:`non-inheritable
Georg Brandl5642ff92013-09-15 10:37:57 +0200209<fd_inheritance>`. New functions and methods:
Victor Stinnerdaf45552013-08-28 00:53:59 +0200210
211* :func:`os.get_inheritable`, :func:`os.set_inheritable`
212* :func:`os.get_handle_inheritable`, :func:`os.set_handle_inheritable`
213* :meth:`socket.socket.get_inheritable`, :meth:`socket.socket.set_inheritable`
214
Antoine Pitrou796564c2013-07-30 19:59:21 +0200215.. seealso::
216
R David Murray061efb12013-12-24 12:35:59 -0500217 :pep:`446` -- Make newly created file descriptors non-inheritable
Nick Coghlan367df122013-10-27 01:57:34 +1000218 PEP written and implemented by Victor Stinner.
Antoine Pitrou796564c2013-07-30 19:59:21 +0200219
Georg Brandlb80f5112012-09-30 09:11:58 +0200220
Nick Coghlan9c1aed82013-11-23 11:13:36 +1000221.. _codec-handling-improvements:
222
R David Murraye7cf6782013-12-24 14:51:25 -0500223Improvements to Codec Handling
R David Murray809487e2013-12-22 20:49:40 -0500224------------------------------
Nick Coghlan8b097b42013-11-13 23:49:21 +1000225
226Since it was first introduced, the :mod:`codecs` module has always been
227intended to operate as a type-neutral dynamic encoding and decoding
228system. However, its close coupling with the Python text model, especially
229the type restricted convenience methods on the builtin :class:`str`,
230:class:`bytes` and :class:`bytearray` types, has historically obscured that
231fact.
232
233As a key step in clarifying the situation, the :meth:`codecs.encode` and
234:meth:`codecs.decode` convenience functions are now properly documented in
235Python 2.7, 3.3 and 3.4. These functions have existed in the :mod:`codecs`
Nick Coghlan9c1aed82013-11-23 11:13:36 +1000236module (and have been covered by the regression test suite) since Python 2.4,
Nick Coghlan8b097b42013-11-13 23:49:21 +1000237but were previously only discoverable through runtime introspection.
238
239Unlike the convenience methods on :class:`str`, :class:`bytes` and
240:class:`bytearray`, these convenience functions support arbitrary codecs
241in both Python 2 and Python 3, rather than being limited to Unicode text
242encodings (in Python 3) or ``basestring`` <-> ``basestring`` conversions
243(in Python 2).
244
Nick Coghlan8afc8f62013-11-22 23:00:22 +1000245In Python 3.4, the interpreter is able to identify the known non-text
246encodings provided in the standard library and direct users towards these
247general purpose convenience functions when appropriate::
Nick Coghlan8b097b42013-11-13 23:49:21 +1000248
Nick Coghlan9c1aed82013-11-23 11:13:36 +1000249 >>> b"abcdef".decode("hex")
Nick Coghlan8b097b42013-11-13 23:49:21 +1000250 Traceback (most recent call last):
251 File "<stdin>", line 1, in <module>
Nick Coghlan9c1aed82013-11-23 11:13:36 +1000252 LookupError: 'hex' is not a text encoding; use codecs.decode() to handle arbitrary codecs
Nick Coghlan8b097b42013-11-13 23:49:21 +1000253
Nick Coghlan9c1aed82013-11-23 11:13:36 +1000254 >>> "hello".encode("rot13")
Nick Coghlan8b097b42013-11-13 23:49:21 +1000255 Traceback (most recent call last):
256 File "<stdin>", line 1, in <module>
Nick Coghlan9c1aed82013-11-23 11:13:36 +1000257 LookupError: 'rot13' is not a text encoding; use codecs.encode() to handle arbitrary codecs
Nick Coghlan8b097b42013-11-13 23:49:21 +1000258
259In a related change, whenever it is feasible without breaking backwards
260compatibility, exceptions raised during encoding and decoding operations
261will be wrapped in a chained exception of the same type that mentions the
262name of the codec responsible for producing the error::
263
Nick Coghlan9c1aed82013-11-23 11:13:36 +1000264 >>> import codecs
265
266 >>> codecs.decode(b"abcdefgh", "hex")
Nick Coghlan8afc8f62013-11-22 23:00:22 +1000267 binascii.Error: Non-hexadecimal digit found
Nick Coghlan8b097b42013-11-13 23:49:21 +1000268
269 The above exception was the direct cause of the following exception:
270
271 Traceback (most recent call last):
272 File "<stdin>", line 1, in <module>
Nick Coghlan9c1aed82013-11-23 11:13:36 +1000273 binascii.Error: decoding with 'hex' codec failed (Error: Non-hexadecimal digit found)
Nick Coghlan8b097b42013-11-13 23:49:21 +1000274
Nick Coghlan9c1aed82013-11-23 11:13:36 +1000275 >>> codecs.encode("hello", "bz2")
Nick Coghlan8b097b42013-11-13 23:49:21 +1000276 TypeError: 'str' does not support the buffer interface
277
278 The above exception was the direct cause of the following exception:
279
280 Traceback (most recent call last):
281 File "<stdin>", line 1, in <module>
Nick Coghlan9c1aed82013-11-23 11:13:36 +1000282 TypeError: encoding with 'bz2' codec failed (TypeError: 'str' does not support the buffer interface)
Nick Coghlan8b097b42013-11-13 23:49:21 +1000283
Nick Coghlan9c1aed82013-11-23 11:13:36 +1000284Finally, as the examples above show, these improvements have permitted
285the restoration of the convenience aliases for the non-Unicode codecs that
286were themselves restored in Python 3.2. This means that encoding binary data
287to and from its hexadecimal representation (for example) can now be written
288as::
289
290 >>> from codecs import encode, decode
291 >>> encode(b"hello", "hex")
292 b'68656c6c6f'
293 >>> decode(b"68656c6c6f", "hex")
294 b'hello'
295
296The binary and text transforms provided in the standard library are detailed
297in :ref:`binary-transforms` and :ref:`text-transforms`.
298
R David Murray9217dad2013-12-23 21:08:28 -0500299(Contributed by Nick Coghlan in :issue:`7475`, , :issue:`17827`,
Nick Coghlan9c1aed82013-11-23 11:13:36 +1000300:issue:`17828` and :issue:`19619`)
Nick Coghlan8b097b42013-11-13 23:49:21 +1000301
R David Murray809487e2013-12-22 20:49:40 -0500302.. _whatsnew-pep-451:
Eric Snowb523f842013-11-22 09:05:39 -0700303
304PEP 451: A ModuleSpec Type for the Import System
R David Murray809487e2013-12-22 20:49:40 -0500305------------------------------------------------
Eric Snowb523f842013-11-22 09:05:39 -0700306
R David Murrayf85b2a82013-12-22 21:06:13 -0500307:pep:`451` provides an encapsulation of the information about a module that the
308import machinery will use to load it (that is, a module specification). This
309helps simplify both the import implementation and several import-related APIs.
310The change is also a stepping stone for `several future import-related
311improvements`__.
Eric Snowb523f842013-11-22 09:05:39 -0700312
R David Murraye9b74d42013-12-22 21:05:04 -0500313__ https://mail.python.org/pipermail/python-dev/2013-November/130111.html
Eric Snowb523f842013-11-22 09:05:39 -0700314
315The public-facing changes from the PEP are entirely backward-compatible.
R David Murrayf85b2a82013-12-22 21:06:13 -0500316Furthermore, they should be transparent to everyone but importer authors. Key
317finder and loader methods have been deprecated, but they will continue working.
318New importers should use the new methods described in the PEP. Existing
319importers should be updated to implement the new methods.
Eric Snowb523f842013-11-22 09:05:39 -0700320
Nick Coghlan8b097b42013-11-13 23:49:21 +1000321
Georg Brandlb80f5112012-09-30 09:11:58 +0200322Other Language Changes
R David Murray809487e2013-12-22 20:49:40 -0500323----------------------
Georg Brandlb80f5112012-09-30 09:11:58 +0200324
325Some smaller changes made to the core Python language are:
326
Ezio Melotti34808e22013-10-12 16:36:13 +0300327* Unicode database updated to UCD version 6.3.
Georg Brandlb80f5112012-09-30 09:11:58 +0200328
R David Murray9a2f1392013-06-28 13:31:19 -0400329* :func:`min` and :func:`max` now accept a *default* argument that can be used
330 to specify the value they return if the iterable they are evaluating has no
331 elements. Contributed by Julian Berman in :issue:`18111`.
Georg Brandlb80f5112012-09-30 09:11:58 +0200332
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +0200333* Module objects are now :mod:`weakref`'able.
334
Nick Coghlan0acceb72013-10-20 13:22:21 +1000335* Module ``__file__`` attributes (and related values) should now always
336 contain absolute paths by default, with the sole exception of
337 ``__main__.__file__`` when a script has been executed directly using
338 a relative path (Contributed by Brett Cannon in :issue:`18416`).
339
Serhiy Storchaka58cf6072013-11-19 11:32:41 +0200340* Now all the UTF-\* codecs (except UTF-7) reject surrogates during both
341 encoding and decoding unless the ``surrogatepass`` error handler is used,
342 with the exception of the UTF-16 decoder that accepts valid surrogate pairs,
343 and the UTF-16 encoder that produces them while encoding non-BMP characters.
344 Contributed by Victor Stinner, Kang-Hao (Kenny) Lu and Serhiy Storchaka in
345 :issue:`12892`.
346
Georg Brandlb80f5112012-09-30 09:11:58 +0200347
R David Murraye6082552014-01-03 16:15:45 -0500348
Georg Brandlb80f5112012-09-30 09:11:58 +0200349New Modules
350===========
351
Nick Coghlan0acceb72013-10-20 13:22:21 +1000352
R David Murray9217dad2013-12-23 21:08:28 -0500353.. _whatsnew-asyncio:
354
Nick Coghlan0acceb72013-10-20 13:22:21 +1000355asyncio
356-------
357
358The new :mod:`asyncio` module (defined in :pep:`3156`) provides a standard
359pluggable event loop model for Python, providing solid asynchronous IO
360support in the standard library, and making it easier for other event loop
361implementations to interoperate with the standard library and each other.
362
363For Python 3.4, this module is considered a :term:`provisional API`.
364
Nick Coghlan367df122013-10-27 01:57:34 +1000365.. seealso::
366
R David Murray061efb12013-12-24 12:35:59 -0500367 :pep:`3156` -- Asynchronous IO Support Rebooted: the "asyncio" Module
Nick Coghlan367df122013-10-27 01:57:34 +1000368 PEP written and implementation led by Guido van Rossum.
369
R David Murray68790662013-12-23 11:17:51 -0500370
R David Murray9217dad2013-12-23 21:08:28 -0500371.. _whatsnew-ensurepip:
R David Murray68790662013-12-23 11:17:51 -0500372
373ensurepip
374---------
375
376The new :mod:`ensurepip` module is the primary infrastructure for the
377:pep:`453` implementation. In the normal course of events end users will not
378need to interact with this module, but it can be used to manually bootstrap
379``pip`` if the automated bootstrapping into an installation or virtual
380environment was declined.
381
382:mod:`ensurepip` includes a bundled copy of ``pip``, up-to-date as of the first
383release candidate of the release of CPython with which it ships (this applies
384to both maintenance releases and feature releases). ``ensurepip`` does not
385access the internet. (If the installation has Internet access, it is of course
386possible to upgrade ``pip`` to a release more recent than the bundled ``pip``
387by using the bundled ``pip`` command itself once it is installed.)
388
389The module is named *ensure*\ pip because if called when ``pip`` is already
390installed, it does nothing. It also has an ``--upgrade`` option that will
391cause it to install the bundled copy of ``pip`` if the existing installed
392version of ``pip`` is older than the bundled copy.
393
394
R David Murray9217dad2013-12-23 21:08:28 -0500395.. _whatsnew-enum:
396
Nick Coghlan0acceb72013-10-20 13:22:21 +1000397enum
398----
399
Nick Coghlan367df122013-10-27 01:57:34 +1000400The new :mod:`enum` module (defined in :pep:`435`) provides a standard
401implementation of enumeration types, allowing other modules (such as
402:mod:`socket`) to provide more informative error messages and better
403debugging support by replacing opaque integer constants with backwards
404compatible enumeration values.
405
406.. seealso::
407
R David Murray061efb12013-12-24 12:35:59 -0500408 :pep:`435` -- Adding an Enum type to the Python standard library
Nick Coghlan367df122013-10-27 01:57:34 +1000409 PEP written by Barry Warsaw, Eli Bendersky and Ethan Furman,
410 implemented by Ethan Furman.
Nick Coghlan0acceb72013-10-20 13:22:21 +1000411
412
R David Murray9217dad2013-12-23 21:08:28 -0500413.. _whatsnew-pathlib:
414
Antoine Pitrou31119e42013-11-22 17:38:12 +0100415pathlib
416-------
417
418The new :mod:`pathlib` module offers classes representing filesystem paths
419with semantics appropriate for different operating systems. Path classes are
420divided between *pure paths*, which provide purely computational operations
421without I/O, and *concrete paths*, which inherit from pure paths but also
422provide I/O operations.
423
424For Python 3.4, this module is considered a :term:`provisional API`.
425
426.. seealso::
427
R David Murray061efb12013-12-24 12:35:59 -0500428 :pep:`428` -- The pathlib module -- object-oriented filesystem paths
Antoine Pitrou31119e42013-11-22 17:38:12 +0100429 PEP written and implemented by Antoine Pitrou.
430
431
R David Murrayf9976e72013-12-23 10:32:02 -0500432.. _whatsnew-selectors:
433
Charles-François Natali243d8d82013-09-04 19:02:49 +0200434selectors
435---------
Georg Brandlb80f5112012-09-30 09:11:58 +0200436
Nick Coghlan0acceb72013-10-20 13:22:21 +1000437The new :mod:`selectors` module (created as part of implementing :pep:`3156`)
438allows high-level and efficient I/O multiplexing, built upon the
439:mod:`select` module primitives.
440
441
R David Murray9217dad2013-12-23 21:08:28 -0500442.. _whatsnew-statistics:
443
Nick Coghlan0acceb72013-10-20 13:22:21 +1000444statistics
445----------
446
447The new :mod:`statistics` module (defined in :pep:`450`) offers some core
448statistics functionality directly in the standard library. This module
449supports calculation of the mean, median, mode, variance and standard
450deviation of a data series.
Georg Brandlb80f5112012-09-30 09:11:58 +0200451
Nick Coghlan367df122013-10-27 01:57:34 +1000452.. seealso::
453
R David Murray061efb12013-12-24 12:35:59 -0500454 :pep:`450` -- Adding A Statistics Module To The Standard Library
Nick Coghlan367df122013-10-27 01:57:34 +1000455 PEP written and implemented by Steven D'Aprano
456
R David Murray9217dad2013-12-23 21:08:28 -0500457.. _whatsnew-tracemalloc:
Georg Brandlb80f5112012-09-30 09:11:58 +0200458
R David Murray8b2d6822013-12-31 15:06:05 -0500459
Victor Stinnerd2736af2013-11-25 09:40:27 +0100460tracemalloc
461-----------
462
463The new :mod:`tracemalloc` module (defined in :pep:`454`) is a debug tool to
464trace memory blocks allocated by Python. It provides the following information:
465
466* Traceback where an object was allocated
467* Statistics on allocated memory blocks per filename and per line number:
468 total size, number and average size of allocated memory blocks
469* Compute the differences between two snapshots to detect memory leaks
470
471.. seealso::
472
R David Murray061efb12013-12-24 12:35:59 -0500473 :pep:`454` -- Add a new tracemalloc module to trace Python memory allocations
Victor Stinnerd2736af2013-11-25 09:40:27 +0100474 PEP written and implemented by Victor Stinner
475
476
R David Murraye6082552014-01-03 16:15:45 -0500477
Georg Brandlb80f5112012-09-30 09:11:58 +0200478Improved Modules
479================
480
R David Murray6e390152013-12-24 22:28:04 -0500481
R David Murray3edcc782013-12-24 16:13:32 -0500482abc
483---
484
485New function :func:`abc.get_cache_token` can be used to know when to invalidate
486caches that are affected by changes in the object graph. (Contributed
487by Łukasz Langa in :issue:`16832`.)
R David Murray72420ff2013-12-24 10:46:44 -0500488
R David Murray2691ee62013-12-28 23:15:12 -0500489New class :class:`~abc.ABC` has :class:`~abc.ABCMeta` as its meta class.
490Using ``ABC`` as a base class has essentially the same effect as specifying
491``metaclass=abc.ABCMeta``, but is simpler to type and easier to read.
492(Contributed by Bruno Dupuis in :issue:`16049`.)
493
494
Antoine Pitroud6cbd342013-08-12 20:48:15 +0200495aifc
496----
497
498The :meth:`~aifc.getparams` method now returns a namedtuple rather than a
499plain tuple. (Contributed by Claudiu Popa in :issue:`17818`.)
500
R David Murrayd592bb22013-12-31 13:45:38 -0500501:func:`aifc.open` now supports the context manager protocol: when used in a
502:keyword:`with` block, the :meth:`~aifc.aifc.close` method of the returned
503object will be called automatically at the end of the block. (Contributed by
504Serhiy Storchacha in :issue:`16486`.)
505
Antoine Pitroud6cbd342013-08-12 20:48:15 +0200506
R David Murrayfced3ec2013-12-31 11:18:01 -0500507argparse
508--------
509
510The :class:`~argparse.FileType` class now accepts *encoding* and
511*errors* arguments, which are passed through to :func:`open`. (Contributed
512by Lucas Maystre in :issue:`11175`.)
513
514
Serhiy Storchakaeaea5e92013-10-19 21:10:46 +0300515audioop
516-------
517
518Added support for 24-bit samples (:issue:`12866`).
519
Serhiy Storchaka3062c9a2013-11-23 22:26:01 +0200520Added the :func:`~audioop.byteswap` function to convert big-endian samples
521to little-endian and vice versa (:issue:`19641`).
522
Serhiy Storchakaeaea5e92013-10-19 21:10:46 +0300523
Nick Coghland4fdbcc2013-11-14 00:24:31 +1000524base64
525------
526
527The encoding and decoding functions in :mod:`base64` now accept any
528:term:`bytes-like object` in cases where it previously required a
Georg Brandled007d52013-11-24 16:09:26 +0100529:class:`bytes` or :class:`bytearray` instance (:issue:`17839`).
Nick Coghland4fdbcc2013-11-14 00:24:31 +1000530
531
Antoine Pitroud6cbd342013-08-12 20:48:15 +0200532colorsys
533--------
534
535The number of digits in the coefficients for the RGB --- YIQ conversions have
536been expanded so that they match the FCC NTSC versions. The change in
537results should be less than 1% and may better match results found elsewhere.
538
R David Murray8e37d5d2013-04-13 14:49:48 -0400539
Nick Coghlanb4534ae2013-10-13 23:23:08 +1000540contextlib
541----------
542
Nick Coghlan240f86d2013-10-17 23:40:57 +1000543The new :class:`contextlib.suppress` context manager helps to clarify the
544intent of code that deliberately suppresses exceptions from a single
545statement. (Contributed by Raymond Hettinger in :issue:`15806` and
546Zero Piraeus in :issue:`19266`)
547
Victor Stinner6633c392013-10-21 13:27:11 +0200548The new :func:`contextlib.redirect_stdout` context manager makes it easier
Nick Coghlanb4534ae2013-10-13 23:23:08 +1000549for utility scripts to handle inflexible APIs that don't provide any
550options to retrieve their output as a string or direct it to somewhere
Nick Coghlan0acceb72013-10-20 13:22:21 +1000551other than :data:`sys.stdout`. In conjunction with :class:`io.StringIO`,
552this context manager is also useful for checking expected output from
553command line utilities. (Contribute by Raymond Hettinger in :issue:`15805`)
554
555The :mod:`contextlib` documentation has also been updated to include a
556:ref:`discussion <single-use-reusable-and-reentrant-cms>` of the
557differences between single use, reusable and reentrant context managers.
Nick Coghlanb4534ae2013-10-13 23:23:08 +1000558
559
Nick Coghlanb39fd0c2013-05-06 23:59:20 +1000560dis
561---
562
R David Murray0bce6e72014-01-07 14:30:17 -0500563Functions :func:`~dis.show_code`, :func:`~dis.dis`, :func:`~dis.distb`, and
564:func:`~dis.disassemble` now accept a keyword-only *file* argument that
565controls where they write their output.
Nick Coghlanb39fd0c2013-05-06 23:59:20 +1000566
R David Murray0bce6e72014-01-07 14:30:17 -0500567The :mod:`dis` module is now built around an :class:`~dis.Instruction` class
568that provides object oriented access to the details of each individual bytecode
569operation.
570
571A new method, :func:`~dis.get_instructions`, provides an iterator that emits
572the Instruction stream for a given piece of Python code. Thus it is now
573possible to write a program that inspects and manipulates a bytecode
574object in ways different from those provided by the :mod:`~dis` module
575itself. For example::
576
577 >>> import dis
578 >>> for instr in dis.get_instructions(lambda x: x + 1):
579 ... print(instr.opname)
580 LOAD_FAST
581 LOAD_CONST
582 BINARY_ADD
583 RETURN_VALUE
584
585The various display tools in the :mod:`dis` module have been rewritten to use
586these new components.
587
588In addition, a new application-friendly class :class:`~dis.Bytecode` provides
589an object-oriented API for inspecting bytecode in both in human-readable form
590and for iterating over instructions. The :class:`~dis.Bytecode` constructor
591takes the same arguments that :func:`~dis.get_instruction` does (plus an
592optional *current_offset*), and the resulting object can be iterated to produce
593:class:`~dis.Instruction` objects. But it also has a :mod:`~dis.Bytecode.dis`
594method, equivalent to calling :mod:`~dis.dis` on the constructor argument, but
595returned as a multi-line string::
596
597 >>> bytecode = dis.Bytecode(lambda x: x +1, current_offset=3)
598 >>> for instr in bytecode:
599 ... print('{} ({})'.format(instr.opname, instr.opcode))
600 LOAD_FAST (124)
601 LOAD_CONST (100)
602 BINARY_ADD (23)
603 RETURN_VALUE (83)
604 >>> bytecode.dis().splitlines() # doctest: +NORMALIZE_WHITESPACE
605 [' 1 0 LOAD_FAST 0 (x)',
606 ' --> 3 LOAD_CONST 1 (1)',
607 ' 6 BINARY_ADD',
608 ' 7 RETURN_VALUE']
609
610:class:`~dis.Bytecode` also has a class method,
611:meth:`~dis.Bytecode.from_traceback`, that provides the ability to manipulate a
612traceback (that is, ``print(Bytecode.from_traceback(tb).dis())`` is equivalent
613to ``distb(tb)``).
Nick Coghlanb39fd0c2013-05-06 23:59:20 +1000614
Nick Coghlan50c48b82013-11-23 00:57:00 +1000615(Contributed by Nick Coghlan, Ryan Kelly and Thomas Kluyver in :issue:`11816`
616and Claudiu Popa in :issue:`17916`)
Nick Coghlanb39fd0c2013-05-06 23:59:20 +1000617
Antoine Pitroud6cbd342013-08-12 20:48:15 +0200618
R David Murray5a9d7062012-11-21 15:09:21 -0500619doctest
620-------
621
R David Murray473f45b2013-12-27 17:01:16 -0500622A new :ref:`option flag <doctest-options>`, :data:`~doctest.FAIL_FAST`, halts
623test running as soon as the first failure is detected. (Contributed by R.
624David Murray and Daniel Urban in :issue:`16522`.)
R David Murray5707d502013-06-23 14:24:13 -0400625
R David Murray473f45b2013-12-27 17:01:16 -0500626The :mod:`doctest` command line interface now uses :mod:`argparse`, and has two
627new options, ``-o`` and ``-f``. ``-o`` allows :ref:`doctest options
628<doctest-options>` to be specified on the command line, and ``-f`` is a
629shorthand for ``-o FAIL_FAST`` (to parallel the similar option supported by the
630:mod:`unittest` CLI). (Contributed by R. David Murray in :issue:`11390`.)
Georg Brandlb80f5112012-09-30 09:11:58 +0200631
R David Murray8e37d5d2013-04-13 14:49:48 -0400632
R David Murraybb17d2b2013-08-09 16:15:28 -0400633email
634-----
635
636:meth:`~email.message.Message.as_string` now accepts a *policy* argument to
637override the default policy of the message when generating a string
638representation of it. This means that ``as_string`` can now be used in more
639circumstances, instead of having to create and use a :mod:`~email.generator` in
640order to pass formatting parameters to its ``flatten`` method.
641
642New method :meth:`~email.message.Message.as_bytes` added to produce a bytes
643representation of the message in a fashion similar to how ``as_string``
644produces a string representation. It does not accept the *maxheaderlen*
645argument, but does accept the *unixfrom* and *policy* arguments. The
646:class:`~email.message.Message` :meth:`~email.message.Message.__bytes__` method
647calls it, meaning that ``bytes(mymsg)`` will now produce the intuitive
648result: a bytes object containing the fully formatted message.
649
650(Contributed by R. David Murray in :issue:`18600`.)
651
R David Murray26b80cfd2013-12-20 17:26:52 -0500652.. _whatsnew_email_contentmanager:
653
R David Murray3da240f2013-10-16 22:48:40 -0400654A pair of new subclasses of :class:`~email.message.Message` have been added,
655along with a new sub-module, :mod:`~email.contentmanager`. All documentation
656is currently in the new module, which is being added as part of the new
Nick Coghlan240f86d2013-10-17 23:40:57 +1000657:term:`provisional <provisional package>` email API. These classes provide a
R David Murray3da240f2013-10-16 22:48:40 -0400658number of new methods that make extracting content from and inserting content
659into email messages much easier. See the :mod:`~email.contentmanager`
660documentation for details.
661
662These API additions complete the bulk of the work that was planned as part of
663the email6 project. The currently provisional API is scheduled to become final
664in Python 3.5 (possibly with a few minor additions in the area of error
665handling).
666
667(Contributed by R. David Murray in :issue:`18891`.)
668
R David Murraybb17d2b2013-08-09 16:15:28 -0400669
Victor Stinner854ffcb2013-06-21 00:36:30 +0200670functools
671---------
672
Nick Coghlanf4cb48a2013-11-03 16:41:46 +1000673The new :func:`~functools.partialmethod` descriptor bring partial argument
674application to descriptors, just as :func:`~functools.partial` provides
675for normal callables. The new descriptor also makes it easier to get
676arbitrary callables (including :func:`~functools.partial` instances)
677to behave like normal instance methods when included in a class definition.
678
679(Contributed by Alon Horev and Nick Coghlan in :issue:`4331`)
680
R David Murray0a102162013-12-20 15:00:54 -0500681.. _whatsnew-singledispatch:
682
Nick Coghlanf4cb48a2013-11-03 16:41:46 +1000683The new :func:`~functools.singledispatch` decorator brings support for
684single-dispatch generic functions to the Python standard library. Where
685object oriented programming focuses on grouping multiple operations on a
686common set of data into a class, a generic function focuses on grouping
687multiple implementations of an operation that allows it to work with
688*different* kinds of data.
689
690.. seealso::
691
R David Murray061efb12013-12-24 12:35:59 -0500692 :pep:`443` -- Single-dispatch generic functions
Nick Coghlanf4cb48a2013-11-03 16:41:46 +1000693 PEP written and implemented by Łukasz Langa.
Victor Stinner854ffcb2013-06-21 00:36:30 +0200694
R David Murrayd2653812013-12-27 14:06:15 -0500695A pure-python version of the :func:`~functools.partial` function is now in the
696stdlib; in CPython it is overridden by the C accelerated version, but it is
697available for other implementations to use. (Contributed by Brian Thorne in
698:issue:`12428`.)
699
Nick Coghlane8c45d62013-07-28 20:00:01 +1000700
R David Murray0e814632013-12-26 15:11:28 -0500701gc
702--
703
R David Murraya101bdb2014-01-06 16:32:05 -0500704New function :func:`~gc.get_stats` returns a list of three per-generation
705dictionaries containing the collections statistics since interpreter startup.
706(Contributed by Antoine Pitrou in :issue:`16351`.)
R David Murray0e814632013-12-26 15:11:28 -0500707
708
Christian Heimese92ef132013-10-13 00:52:43 +0200709hashlib
710-------
711
712New :func:`hashlib.pbkdf2_hmac` function.
Christian Heimese92ef132013-10-13 00:52:43 +0200713(Contributed by Christian Heimes in :issue:`18582`)
714
715
Ezio Melotti250a06c2013-11-25 06:18:47 +0200716html
717----
718
719Added a new :func:`html.unescape` function that converts HTML5 character
720references to the corresponding Unicode characters.
Ezio Melotti250a06c2013-11-25 06:18:47 +0200721(Contributed by Ezio Melotti in :issue:`2927`)
722
723Added a new *convert_charrefs* keyword argument to
724:class:`~html.parser.HTMLParser` that, when ``True``, automatically converts
725all character references. For backward-compatibility, its value defaults
726to ``False``, but it will change to ``True`` in future versions, so you
727are invited to set it explicitly and update your code to use this new feature.
Ezio Melotti250a06c2013-11-25 06:18:47 +0200728(Contributed by Ezio Melotti in :issue:`13633`)
729
730The *strict* argument of :class:`~html.parser.HTMLParser` is now deprecated.
Ezio Melotti250a06c2013-11-25 06:18:47 +0200731(Contributed by Ezio Melotti in :issue:`15114`)
732
733
R David Murraya475a8d2014-01-03 13:03:00 -0500734http
735----
736
737:meth:`~http.server.BaseHTTPRequestHandler.send_error` now accepts an
Zachary Ware335957e2014-01-13 16:08:54 -0600738optional additional *explain* parameter which can be used to provide an
R David Murraya475a8d2014-01-03 13:03:00 -0500739extended error description, overriding the hardcoded default if there is one.
740This extended error description will be formatted using the
741:attr:`~http.server.HTTP.error_message_format` attribute and sent as the body
742of the error response. (Contributed by Karl Cow in :issue:`12921`.)
743
744
745
R David Murrayd2653812013-12-27 14:06:15 -0500746importlib
747---------
748
749The :class:`~importlib.abc.InspectLoader` ABC defines a new method,
750:meth:`~importlib.abc.InspectLoader.source_to_code` that accepts source
751data and a path and returns a code object. The default implementation
752is equivalent to ``compile(data, path, 'exec', dont_inherit=True)``.
753(Contributed by Eric Snow and Brett Cannon in :issue:`15627`.)
754
R David Murray0ae7ae12014-01-08 18:16:02 -0500755:class:`~importlib.abc.InspectLoader` also now has a default implementation
756for the :meth:`~importlib.abc.InspectLoader.get_code` method. However,
757it will normally be desirable to override the default implementation
758for performance reasons. (Contributed by Brett Cannon in :issue:`18072`.)
759
R David Murrayd2653812013-12-27 14:06:15 -0500760
Antoine Pitroud6cbd342013-08-12 20:48:15 +0200761inspect
762-------
763
Nick Coghlanf94a16b2013-09-22 22:46:49 +1000764
Nick Coghlan367df122013-10-27 01:57:34 +1000765The inspect module now offers a basic :ref:`command line interface
766<inspect-module-cli>` to quickly display source code and other
767information for modules, classes and functions. (Contributed by Claudiu Popa
768and Nick Coghlan in :issue:`18626`)
Nick Coghlanf94a16b2013-09-22 22:46:49 +1000769
Antoine Pitroud6cbd342013-08-12 20:48:15 +0200770:func:`~inspect.unwrap` makes it easy to unravel wrapper function chains
771created by :func:`functools.wraps` (and any other API that sets the
Nick Coghlan367df122013-10-27 01:57:34 +1000772``__wrapped__`` attribute on a wrapper function). (Contributed by
773Daniel Urban, Aaron Iles and Nick Coghlan in :issue:`13266`)
774
775As part of the implementation of the new :mod:`enum` module, the
776:mod:`inspect` module now has substantially better support for custom
777``__dir__`` methods and dynamic class attributes provided through
778metaclasses (Contributed by Ethan Furman in :issue:`18929` and
779:issue:`19030`)
780
Antoine Pitroud6cbd342013-08-12 20:48:15 +0200781
R David Murray9cf617b2014-01-04 18:55:01 -0500782logging
783-------
784
785The :class:`~logging.handlers.TimedRotatingFileHandler` has a new *atTime*
786parameter that can be used to specify the time of day when rollover should
787happen. (Contributed by Ronald Oussoren in :issue:`9556`.)
788
789
R David Murray8f7664a2013-12-22 20:40:11 -0500790.. _whatsnew-marshal-3:
791
792marshal
793-------
794
795The default :mod:`marshal` version has been bumped to 3. The code implementing
796the new version restores the Python2 behavior of recording only one copy of
797interned strings and preserving the interning on deserialization, and extends
798this "one copy" ability to any object type (including handling recursive
799references). This reduces both the size of ``.pyc`` files and the amount of
800memory a module occupies in memory when it is loaded from a ``.pyc`` (or
801``.pyo``) file. (Contributed by Kristján Valur Jónsson in :issue:`16475`.)
802
803
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +0200804mmap
805----
806
807mmap objects can now be weakref'ed.
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +0200808(Contributed by Valerie Lambert in :issue:`4885`.)
809
810
R David Murray33ef1ae2013-12-31 16:04:50 -0500811mock
812----
813
814:mod:`~unittest.mock` objects now inspect their specification signatures when
815matching calls, which means an argument can now be matched by either position
816or name, instead of only by position. (Contributed by Antoine Pitrou in
817:issue:`17015`.)
818
R David Murray57fcf762014-01-03 23:31:54 -0500819:func:`~mock.mock_open` objects now have ``readline`` and ``readlines``
820methods. (Contributed by Toshio Kuratomi in :issue:`17467`.)
821
R David Murray33ef1ae2013-12-31 16:04:50 -0500822
Richard Oudkerk84ed9a62013-08-14 15:35:41 +0100823multiprocessing
824---------------
825
R David Murrayac186222013-12-20 17:23:57 -0500826.. _whatsnew-multiprocessing-no-fork:
827
828On Unix, two new :ref:`start methods <multiprocessing-start-methods>`
829(``spawn`` and ``forkserver``) have been added for starting processes using
830:mod:`multiprocessing`. These make the mixing of processes with threads more
831robust, and the ``spawn`` method matches the semantics that multiprocessing has
832always used on Windows. (Contributed by Richard Oudkerk in :issue:`8713`).
Richard Oudkerk84ed9a62013-08-14 15:35:41 +0100833
834Also, except when using the old *fork* start method, child processes
R David Murrayac186222013-12-20 17:23:57 -0500835will no longer inherit unneeded handles/file descriptors from their parents
836(part of :issue:`8713`).
Richard Oudkerk84ed9a62013-08-14 15:35:41 +0100837
Nick Coghlan9a767352013-12-17 22:17:26 +1000838:mod:`multiprocessing` now relies on :mod:`runpy` (which implements the
839``-m`` switch) to initialise ``__main__`` appropriately in child processes
840when using the ``spawn`` or ``forkserver`` start methods. This resolves some
841edge cases where combining multiprocessing, the ``-m`` command line switch
842and explicit relative imports could cause obscure failures in child
843processes. (Contributed by Nick Coghlan in :issue:`19946`)
844
Richard Oudkerk84ed9a62013-08-14 15:35:41 +0100845
R David Murray98358272014-01-05 20:52:06 -0500846operator
847--------
848
849There is now a pure-python version of the :mod:`operator` module available for
850reference and for use by alternate implementations of Python. (Contributed by
851Zachary Ware in :issue:`16694`.)
852
853
Victor Stinnerdaf45552013-08-28 00:53:59 +0200854os
855--
856
Georg Brandlc6ebbef2013-09-16 04:03:12 +0200857New functions to get and set the :ref:`inheritable flag <fd_inheritance>` of a file
Victor Stinnerdaf45552013-08-28 00:53:59 +0200858descriptors or a Windows handle:
859
860* :func:`os.get_inheritable`, :func:`os.set_inheritable`
861* :func:`os.get_handle_inheritable`, :func:`os.set_handle_inheritable`
862
R David Murray66bf12a2014-01-08 17:21:22 -0500863The :mod:`os` module now provides a :func:`~os.cpu_count` function, analogous to
864the :func:`multiprocessing.cpu_count` function (which is now implemented in
865terms of the new :mod:`os` function). (Contributed by Trent Nelson, Yogesh
866Chaudhari, Victor Stinner, and Charles-François Natali in :issue:`17914`.)
867
Victor Stinnerdaf45552013-08-28 00:53:59 +0200868
R David Murray78d692f2013-10-10 17:23:26 -0400869pdb
870---
871
872The ``print`` command has been removed from :mod:`pdb`, restoring access to the
873``print`` function.
874
875Rationale: Python2's ``pdb`` did not have a ``print`` command; instead,
876entering ``print`` executed the ``print`` statement. In Python3 ``print`` was
877mistakenly made an alias for the pdb :pdbcmd:`p` command. ``p``, however,
878prints the ``repr`` of its argument, not the ``str`` like the Python2 ``print``
879command did. Worse, the Python3 ``pdb print`` command shadowed the Python3
880``print`` function, making it inaccessible at the ``pdb`` prompt.
881
882(Contributed by Connor Osborn in :issue:`18764`.)
883
884
R David Murray809487e2013-12-22 20:49:40 -0500885.. _whatsnew-protocol-4:
886
R David Murray5ea95242013-12-24 15:59:57 -0500887pickle
R David Murray809487e2013-12-22 20:49:40 -0500888------
889
890protocol 4
891
892:mod:`pickle` now supports (but does not use by default) a new pickle protocol,
893protocol 4. This new protocol addresses a number of issues that were present
894in previous protocols, such as the serialization of nested classes, very large
895strings and containers, or classes whose :meth:`__new__` method takes
896keyword-only arguments. It also provides some efficiency improvements.
897
898.. seealso::
899
R David Murray061efb12013-12-24 12:35:59 -0500900 :pep:`3154` -- Pickle protocol 4
R David Murray809487e2013-12-22 20:49:40 -0500901 PEP written by Antoine Pitrou and implemented by Alexandre Vassalotti.
902
903
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +0200904poplib
905------
906
R David Murray473f45b2013-12-27 17:01:16 -0500907Two new methods have been added to :mod:`poplib`: :meth:`~poplib.POP3.capa`,
908which returns the list of capabilities advertised by the POP server, and
909:meth:`~poplib.POP3.stls`, which switches a clear-text POP3 session into an
910encrypted POP3 session if the POP server supports it. (Contributed by Lorenzo
911Catucci in :issue:`4473`.)
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +0200912
913
Serhiy Storchaka7c411a42013-10-02 11:56:18 +0300914pprint
915------
916
Christian Heimese1bfd3e2013-10-21 12:32:21 +0200917The :mod:`pprint` module now supports *compact* mode for formatting long
Serhiy Storchaka7c411a42013-10-02 11:56:18 +0300918sequences (:issue:`19132`).
919
R David Murrayb231b2b2014-01-04 17:11:23 -0500920Long strings are now wrapped using Python's normal line continuation
921syntax (Contributed by Antoine Pitrou in :issue:`17150`.)
922
Serhiy Storchaka7c411a42013-10-02 11:56:18 +0300923
R David Murray6e390152013-12-24 22:28:04 -0500924pty
925---
926
927:func:`pty.spawn` now returns the status value from :func:`os.waitpid` on
928the child process, instead of ``None``. (Contributed by Gregory P. Smith.)
929
930
Nick Coghlan367df122013-10-27 01:57:34 +1000931pydoc
932-----
933
934While significant changes have not been made to :mod:`pydoc` directly,
935its handling of custom ``__dir__`` methods and various descriptor
936behaviours has been improved substantially by the underlying changes in
937the :mod:`inspect` module.
938
939
Serhiy Storchaka32eddc12013-11-23 23:20:30 +0200940re
941--
942
943Added :func:`re.fullmatch` function and :meth:`regex.fullmatch` method,
944which anchor the pattern at both ends of the string to match.
945(Contributed by Matthew Barnett in :issue:`16203`.)
946
Ezio Melottidd7e2912013-11-25 23:20:20 +0200947The repr of :ref:`regex objects <re-objects>` now includes the pattern
948and the flags; the repr of :ref:`match objects <match-objects>` now
949includes the start, end, and the part of the string that matched.
Ezio Melottidd7e2912013-11-25 23:20:20 +0200950(Contributed by Serhiy Storchaka in :issue:`13592` and :issue:`17087`.)
951
952
Christian Heimesb7bd5df2013-10-22 11:21:54 +0200953resource
954--------
955
956New :func:`resource.prlimit` function and Linux specific constants.
957(Contributed by Christian Heimes in :issue:`16595` and :issue:`19324`.)
958
R David Murray575fb312013-12-25 23:21:03 -0500959
R David Murray2bc930f2013-12-31 11:17:21 -0500960select
961------
962
963:class:`~select.epoll` objects now support the context management protocol.
964When used in a :keyword:`with` statement, the :meth:`~select.epoll.close`
965method will be called automatically at the end of the block. (Contributed
966by Serhiy Storchaka in :issue:`16488`.)
967
968
R David Murray575fb312013-12-25 23:21:03 -0500969shelve
970------
971
972:class:`~shelve.Shelf` instances may now be used in :keyword:`with` statements,
973and will be automatically closed at the end of the :keyword:`with` block.
974(Contributed by Filip Gruszczyński in :issue:`13896`.)
975
976
R David Murray8e37d5d2013-04-13 14:49:48 -0400977smtplib
978-------
979
R David Murray8a345962013-04-14 06:46:35 -0400980:exc:`~smtplib.SMTPException` is now a subclass of :exc:`OSError`, which allows
R David Murray8e37d5d2013-04-13 14:49:48 -0400981both socket level errors and SMTP protocol level errors to be caught in one
982try/except statement by code that only cares whether or not an error occurred.
983(:issue:`2118`).
984
Antoine Pitroud6cbd342013-08-12 20:48:15 +0200985
Victor Stinnerdaf45552013-08-28 00:53:59 +0200986socket
987------
988
R David Murray33ef1ae2013-12-31 16:04:50 -0500989The socket module now supports the :data:`~socket.CAN_BCM` protocol on
990platforms that support it. (Contributed by Brian Thorne in :issue:`15359`.)
991
Georg Brandl5642ff92013-09-15 10:37:57 +0200992Socket objects have new methods to get or set their :ref:`inheritable flag
R David Murray33ef1ae2013-12-31 16:04:50 -0500993<fd_inheritance>`, :meth:`~socket.socket.get_inheritable` and
994:meth:`~socket.socket.set_inheritable`.
Victor Stinnerdaf45552013-08-28 00:53:59 +0200995
R David Murray33ef1ae2013-12-31 16:04:50 -0500996The ``socket.AF_*`` and ``socket.SOCK_*`` constants are now enumeration values
997using the new :mod:`enum` module. This allows meaningful names to be printed
998during debugging, instead of integer "magic numbers".
Victor Stinnerdaf45552013-08-28 00:53:59 +0200999
R David Murray66bf12a2014-01-08 17:21:22 -05001000The :data:`~socket.AF_LINK` constant is now available on BSD and OSX.
1001
R David Murray8b2d6822013-12-31 15:06:05 -05001002
R David Murray9025f1c2014-01-02 13:44:18 -05001003sqlite3
1004-------
1005
1006A new boolean parameter, *uri*, to the :func:`~sqlite3.connect` function can
1007be used to indicate that the *database* parameter is a ``uri`` (see
1008the `SQLite URI documentation <http://www.sqlite.org/uri.html>`_).
1009(Contributed by poq in :issue:`13773`.)
1010
1011
Christian Heimes24cd4cf2013-06-22 19:31:58 +02001012ssl
1013---
1014
R David Murray748bad22013-12-20 17:08:39 -05001015.. _whatsnew-tls-11-12:
Christian Heimes24cd4cf2013-06-22 19:31:58 +02001016
R David Murray748bad22013-12-20 17:08:39 -05001017:data:`~ssl.PROTOCOL_TLSv1_1` and :data:`~ssl.PROTOCOL_TLSv1_2` (TLSv1.1 and
1018TLSv1.2 support) have been added; support for these protocols is only available if
1019Python is linked with OpenSSL 1.0.1 or later. (Contributed by Michele Orrù and
1020Antoine Pitrou in :issue:`16692`)
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +02001021
R David Murray748bad22013-12-20 17:08:39 -05001022New diagnostic functions :func:`~ssl.get_default_verify_paths`,
1023:meth:`~ssl.SSLContext.cert_store_stats` and
1024:meth:`~ssl.SSLContext.get_ca_certs` (Contributed by Christian Heimes
1025in :issue:`18143` and :issue:`18147`)
Christian Heimes24cd4cf2013-06-22 19:31:58 +02001026
R David Murray748bad22013-12-20 17:08:39 -05001027Add :func:`ssl.enum_cert_store` to retrieve certificates and CRL from Windows'
1028cert store. (Contributed by Christian Heimes in :issue:`17134`.)
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +02001029
1030Support for server-side SNI using the new
1031:meth:`ssl.SSLContext.set_servername_callback` method.
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +02001032(Contributed by Daniel Black in :issue:`8109`.)
1033
1034
Antoine Pitroud6cbd342013-08-12 20:48:15 +02001035stat
1036----
1037
1038The :mod:`stat` module is now backed by a C implementation in :mod:`_stat`. A C
1039implementation is required as most of the values aren't standardized and
1040platform-dependent. (Contributed by Christian Heimes in :issue:`11016`.)
1041
1042The module supports new file types: door, event port and whiteout.
1043
1044
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +02001045struct
1046------
1047
R David Murray98358272014-01-05 20:52:06 -05001048:mod:`struct` now supports the streamed unpacking of a buffer containing
1049repeated instances of a given format of data. Both a module level
1050:mod:`~struct.iter_unpack` function and a :meth:`struct.Struct.iter_unpack`
1051method on compiled formats have been added. (Contributed by Antoine Pitrou in
1052:issue:`17804`.)
1053
1054
1055subprocess
1056----------
1057
1058:func:`~subprocess.check_output` now accepts an *input* argument that can
1059be used to provide the contents of ``stdin`` for the command that is run.
1060(Contributed by Zack Weinberg in :issue:`16624`.)
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +02001061
1062
Serhiy Storchakae06a8962013-09-04 00:43:03 +03001063sunau
1064-----
1065
1066The :meth:`~sunau.getparams` method now returns a namedtuple rather than a
1067plain tuple. (Contributed by Claudiu Popa in :issue:`18901`.)
1068
Serhiy Storchaka34d20132013-09-05 17:01:53 +03001069:meth:`sunau.open` now supports the context manager protocol (:issue:`18878`).
1070
Serhiy Storchakae06a8962013-09-04 00:43:03 +03001071
R David Murrayd17aba72013-12-24 14:46:23 -05001072sys
1073---
1074
1075New function :func:`sys.getallocatedblocks` returns the current number of
1076blocks allocated by the interpreter (in CPython with the default
1077``--with-pymalloc`` setting, this is allocations made through the
1078:c:func:`PyObject_Malloc` API). This can be useful for tracking memory leaks,
1079especially if automated via a test suite. (Contributed by Antoine Pitrou
1080in :issue:`13390`.)
1081
1082
Andrew Kuchling173a1572013-09-15 18:15:56 -04001083traceback
1084---------
1085
1086A new :func:`traceback.clear_frames` function takes a traceback object
1087and clears the local variables in all of the frames it references,
1088reducing the amount of memory consumed (:issue:`1565525`).
1089
1090
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +02001091urllib
1092------
1093
R David Murray473f45b2013-12-27 17:01:16 -05001094:mod:`urllib.request` now supports ``data:`` URLs via the
1095:class:`~urllib.request.DataHandler` class. (Contributed by Mathias Panzenböck
1096in :issue:`16423`.)
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +02001097
1098
1099unittest
1100--------
1101
R David Murray57fcf762014-01-03 23:31:54 -05001102The :class:`~unittest.TestCase` class has a new method,
1103:meth:`~unittest.TestCase.subTest`, that produces a context manager whose
1104:keyword:`with` block becomes a "sub-test". This context manager allows a test
1105method to dynamically generate subtests by, say, calling the ``subTest``
1106context manager inside a loop. A single test method can thereby produce an
1107indefinite number of separately-identified and separately-counted tests, all of
1108which will run even if one or more of them fail. For example::
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +02001109
R David Murray57fcf762014-01-03 23:31:54 -05001110 class NumbersTest(unittest.TestCase):
1111 def test_even(self):
1112 for i in range(6):
1113 with self.subTest(i=1):
1114 self.assertEqual(i % 2, 0)
1115
1116will result in six subtests, each identified in the unittest verbose output
1117with a label consisting of the variable name ``i`` and a particular value for
1118that variable (``i=0``, ``i=1``, etc). See :ref:`subtests` for the full
1119version of this example. (Contributed by Antoine Pitrou in :issue:`16997`.)
1120
1121:func:`unittest.main` now accepts an iterable of test names for
R David Murray9025f1c2014-01-02 13:44:18 -05001122*defaultTest*, where previously it only accepted a single test name as a
1123string. (Contributed by Jyrki Pulliainen in :issue:`15132`.)
1124
R David Murray42fa1102014-01-03 13:03:36 -05001125If :class:`~unittest.SkipTest` is raised during test discovery (that is, at the
1126module level in the test file), it is now reported as a skip instead of an
1127error. (Contributed by Zach Ware in :issue:`16935`.)
1128
R David Murraydb085f52014-01-03 15:46:24 -05001129:meth:`~unittest.TestLoader.discover` now sorts the discovered files to provide
1130consistent test ordering. (Contributed by Martin Melin and Jeff Ramnani in
1131:issue:`16709`.)
1132
R David Murray8e37d5d2013-04-13 14:49:48 -04001133
R David Murray575fb312013-12-25 23:21:03 -05001134venv
1135----
1136
1137:mod:`venv` now includes activation scripts for the ``csh`` and ``fish``
1138shells (Contributed by Andrew Svetlov in :issue:`15417`.)
1139
1140
R David Murray671cd322013-04-10 12:31:43 -04001141wave
1142----
1143
1144The :meth:`~wave.getparams` method now returns a namedtuple rather than a
1145plain tuple. (Contributed by Claudiu Popa in :issue:`17487`.)
1146
R David Murrayc91d5ee2013-07-31 13:46:08 -04001147:meth:`wave.open` now supports the context manager protocol. (Contributed
1148by Claudiu Popa in :issue:`17616`.)
1149
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +02001150
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +02001151weakref
1152-------
1153
1154New :class:`~weakref.WeakMethod` class simulates weak references to bound
Nick Coghlanbe57ab82013-09-22 21:26:30 +10001155methods. (Contributed by Antoine Pitrou in :issue:`14631`.)
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +02001156
Nick Coghlanbe57ab82013-09-22 21:26:30 +10001157New :class:`~weakref.finalize` class makes it possible to register a callback
1158to be invoked when an object is garbage collected, without needing to
1159carefully manage the lifecycle of the weak reference itself. (Contributed by
1160Richard Oudkerk in :issue:`15528`)
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +02001161
R David Murraya101bdb2014-01-06 16:32:05 -05001162The callback, if any, associated with a :class:`~weakref.ref` is now
1163exposed via the :attr:`~weakref.ref.__callback__` attribute. (Contributed
1164by Mark Dickinson in :issue:`17643`.)
1165
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +02001166
1167xml.etree
1168---------
1169
R David Murray410d3202014-01-04 23:52:50 -05001170A new parser, :class:`~xml.etree.ElementTree.XMLPullParser`, allows a
1171non-blocking applications to parse XML documents. An example can be
1172seen at :ref:`elementtree-pull-parsing`. (Contributed by Antoine
1173Pitrou in :issue:`17741`.)
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +02001174
R David Murray575fb312013-12-25 23:21:03 -05001175The :mod:`xml.etree.ElementTree` :func:`~xml.etree.ElementTree.tostring` and
1176:func:`~xml.etree.ElementTree.tostringlist` functions, and the
1177:class:`~xml.etree.ElementTree.ElementTree`
1178:meth:`~xml.etree.ElementTree.ElementTree.write` method, now have a
1179*short_empty_elements* :ref:`keyword-only parameter <keyword-only_parameter>`
1180providing control over whether elements with no content are written in
1181abbreviated (``<tag />``) or expanded (``<tag></tag>``) form. (Contributed by
1182Ariel Poliak and Serhiy Storchaka in :issue:`14377`.)
1183
Christian Tismer59202e52013-10-21 03:59:23 +02001184
1185zipfile.PyZipfile
1186-----------------
1187
1188Add a filter function to ignore some packages (tests for instance),
1189:meth:`~zipfile.PyZipFile.writepy`.
Christian Tismer59202e52013-10-21 03:59:23 +02001190(Contributed by Christian Tismer in :issue:`19274`.)
1191
1192
R David Murraye6082552014-01-03 16:15:45 -05001193
Nick Coghlan367df122013-10-27 01:57:34 +10001194CPython Implementation Changes
1195==============================
1196
1197
R David Murrayc16dfe12013-12-21 12:32:10 -05001198.. _whatsnew-pep-445:
Nick Coghlan367df122013-10-27 01:57:34 +10001199
R David Murraye7cf6782013-12-24 14:51:25 -05001200PEP 445: Customization of CPython Memory Allocators
Nick Coghlan367df122013-10-27 01:57:34 +10001201---------------------------------------------------
1202
1203:pep:`445` adds new C level interfaces to customize memory allocation in
1204the CPython interpreter.
1205
1206.. seealso::
1207
R David Murray061efb12013-12-24 12:35:59 -05001208 :pep:`445` -- Add new APIs to customize Python memory allocators
Nick Coghlan367df122013-10-27 01:57:34 +10001209 PEP written and implemented by Victor Stinner.
1210
1211
R David Murrayc16dfe12013-12-21 12:32:10 -05001212.. _whatsnew-pep-442:
Nick Coghlan367df122013-10-27 01:57:34 +10001213
R David Murraye7cf6782013-12-24 14:51:25 -05001214PEP 442: Safe Object Finalization
Nick Coghlan367df122013-10-27 01:57:34 +10001215---------------------------------
1216
1217:pep:`442` removes the current limitations and quirks of object finalization
1218in CPython. With it, objects with :meth:`__del__` methods, as well as
1219generators with :keyword:`finally` clauses, can be finalized when they are
1220part of a reference cycle.
1221
1222As part of this change, module globals are no longer forcibly set to
1223:const:`None` during interpreter shutdown in most cases, instead relying
R David Murrayca794612013-12-22 14:05:11 -05001224on the normal operation of the cyclic garbage collector. This avoids a
1225whole class of interpreter-shutdown-time errors, usually involving
1226``__del__`` methods, that have plagued Python since the cyclic GC
1227was first introduced.
Nick Coghlan367df122013-10-27 01:57:34 +10001228
1229.. seealso::
1230
R David Murray061efb12013-12-24 12:35:59 -05001231 :pep:`442` -- Safe object finalization
Nick Coghlan367df122013-10-27 01:57:34 +10001232 PEP written and implemented by Antoine Pitrou.
1233
1234
R David Murrayc16dfe12013-12-21 12:32:10 -05001235.. _whatsnew-pep-456:
1236
1237PEP 456: Secure and Interchangeable Hash Algorithm
1238--------------------------------------------------
1239
1240:pep:`456` follows up on earlier security fix work done on Python's hash
1241algorithm to address certain DOS attacks to which public facing APIs backed by
1242dictionary lookups may be subject. (See :issue:`14621` for the start of the
1243current round of improvements.) The PEP unifies CPython's hash code to make it
1244easier for a packager to substitute a different hash algorithm, and switches
1245Python's default implementation to a SipHash implementation on platforms that
1246have a 64 bit data type. Any performance differences in comparison with the
1247older FNV algorithm are trivial.
1248
1249The PEP adds additional fields to the :func:`sys.hash_info` struct sequence to
1250describe the hash algorithm in use by the currently executing binary. Otherwise,
1251the PEP does not alter any existing CPython APIs.
1252
1253
R David Murray6dd18302013-12-24 12:23:56 -05001254.. _whatsnew-pep-436:
1255
1256PEP 436: Argument Clinic
1257------------------------
1258
1259"Argument Clinic" (:pep:`436`) is now part of the CPython build process
1260and can be used to simplify the process of defining and maintaining
1261accurate signatures for builtins and standard library extension modules
1262implemented in C.
1263
1264.. note::
1265 The Argument Clinic PEP is not fully up to date with the state of the
1266 implementation. This has been deemed acceptable by the release manager
1267 and core development team in this case, as Argument Clinic will not
1268 be made available as a public API for third party use in Python 3.4.
1269
1270.. seealso::
1271
R David Murray061efb12013-12-24 12:35:59 -05001272 :pep:`436` -- The Argument Clinic DSL
R David Murray6dd18302013-12-24 12:23:56 -05001273 PEP written and implemented by Larry Hastings.
1274
1275
R David Murraye7cf6782013-12-24 14:51:25 -05001276Other Build and C API Changes
Nick Coghlan367df122013-10-27 01:57:34 +10001277-----------------------------
Georg Brandlb80f5112012-09-30 09:11:58 +02001278
Nick Coghlan7d270ee2013-10-17 22:35:35 +10001279* The new :c:func:`Py_SetStandardStreamEncoding` pre-initialization API
1280 allows applications embedding the CPython interpreter to reliably force
1281 a particular encoding and error handler for the standard streams
1282 (Contributed by Bastien Montagne and Nick Coghlan in :issue:`16129`)
Georg Brandlb80f5112012-09-30 09:11:58 +02001283
Nick Coghlan0acceb72013-10-20 13:22:21 +10001284* Most Python C APIs that don't mutate string arguments are now correctly
1285 marked as accepting ``const char *`` rather than ``char *`` (Contributed
1286 by Serhiy Storchaka in :issue:`1772673`).
1287
R David Murrayd91ba202013-12-24 12:13:44 -05001288* New shell version of ``python-config``; can be used even when a python
1289 interpreter is not available (for example, in cross compilation scenarios).
1290
Georg Brandlb80f5112012-09-30 09:11:58 +02001291
R David Murraye6082552014-01-03 16:15:45 -05001292
R David Murraye7cf6782013-12-24 14:51:25 -05001293Other Improvements
R David Murraybcaaecf2013-12-23 21:23:36 -05001294==================
1295
1296* Tab-completion is now enabled by default in the interactive interpreter.
1297 (Contributed by Antoine Pitrou and Éric Araujo in :issue:`5845`.)
1298
1299* Invoking the Python interpreter with ``--version`` now outputs the version to
1300 standard output instead of standard error (:issue:`18338`). Similar changes
1301 were made to :mod:`argparse` (:issue:`18920`) and other modules that have
1302 script-like invocation capabilities (:issue:`18922`).
1303
R David Murray72420ff2013-12-24 10:46:44 -05001304* The CPython Windows installer now adds ``.py`` to the :envvar:`PATHEXT`
1305 variable when extensions are registered, allowing users to run a python
1306 script at the windows command prompt by just typing its name without the
1307 ``.py`` extension. (Contributed by Paul Moore in :issue:`18569`.)
1308
R David Murrayd17aba72013-12-24 14:46:23 -05001309* A new ``make`` target `coverage-report
1310 <http://docs.python.org/devguide/coverage.html#measuring-coverage-of-c-code-with-gcov-and-lcov>`_
1311 will build python, run the test suite, and generate an HTML coverage report
1312 for the C codebase using ``gcov`` and `lcov
1313 <http://ltp.sourceforge.net/coverage/lcov.php>`_.
1314
1315* The ``-R`` option to the :ref:`python regression test suite <regrtest>` now
1316 also checks for memory allocation leaks, using
1317 :func:`sys.getallocatedblocks()`. (Contributed by Antoine Pitrou in
1318 :issue:`13390`).
1319
R David Murraybcaaecf2013-12-23 21:23:36 -05001320
R David Murraye6082552014-01-03 16:15:45 -05001321
R David Murrayd74d09a2013-12-24 15:33:02 -05001322Significant Optimizations
1323=========================
R David Murraybcaaecf2013-12-23 21:23:36 -05001324
R David Murray4908f4a2014-01-04 18:07:20 -05001325* The UTF-32 decoder is now 3x to 4x faster. (Contributed by Serhiy Storchaka
1326 in :issue:`14625`.)
R David Murraybcaaecf2013-12-23 21:23:36 -05001327
1328* The cost of hash collisions for sets is now reduced. Each hash table
1329 probe now checks a series of consecutive, adjacent key/hash pairs before
1330 continuing to make random probes through the hash table. This exploits
1331 cache locality to make collision resolution less expensive.
R David Murraybcaaecf2013-12-23 21:23:36 -05001332 The collision resolution scheme can be described as a hybrid of linear
1333 probing and open addressing. The number of additional linear probes
1334 defaults to nine. This can be changed at compile-time by defining
1335 LINEAR_PROBES to be any value. Set LINEAR_PROBES=0 to turn-off
R David Murray59171722014-01-03 15:52:22 -05001336 linear probing entirely. (Contributed by Raymond Hettinger in
1337 :issue:`18771`.)
R David Murraybcaaecf2013-12-23 21:23:36 -05001338
1339* The interpreter starts about 30% faster. A couple of measures lead to the
1340 speedup. The interpreter loads fewer modules on startup, e.g. the :mod:`re`,
1341 :mod:`collections` and :mod:`locale` modules and their dependencies are no
1342 longer imported by default. The marshal module has been improved to load
R David Murray59171722014-01-03 15:52:22 -05001343 compiled Python code faster. (Contributed by Antoine Pitrou, Christian
1344 Heimes and Victor Stinner in :issue:`19219`, :issue:`19218`, :issue:`19209`,
1345 :issue:`19205` and :issue:`9548`)
R David Murraybcaaecf2013-12-23 21:23:36 -05001346
R David Murray6e390152013-12-24 22:28:04 -05001347* :class:`bz2.BZ2File` is now as fast or faster than the Python2 version for
1348 most cases. :class:`lzma.LZMAFile` has also been optimized. (Contributed by
1349 Serhiy Storchaka and Nadeem Vawda in :issue:`16034`.)
1350
R David Murray8b2d6822013-12-31 15:06:05 -05001351* :func:`random.getrandbits` is 20%-40% faster for small integers (the most
1352 common use case). (Contributed by Serhiy Storchaka in :issue:`16674`).
1353
R David Murrayb231b2b2014-01-04 17:11:23 -05001354* By taking advantage of the new storage format for strings, pickling of
1355 strings is now significantly faster. (Contributed by Victor Stinner and
1356 Antoine Pitrou in :issue:`15596`.)
1357
R David Murray66bf12a2014-01-08 17:21:22 -05001358* A performance issue in :meth:`io.FileIO.readall` has been solved. This
1359 particularly affects Windows, and significantly speeds up the case of piping
1360 significant amounts of data through :mod:`subprocess`. (Contributed
1361 by Richard Oudkerk in :issue:`15758`.)
1362
R David Murraybcaaecf2013-12-23 21:23:36 -05001363
R David Murraye6082552014-01-03 16:15:45 -05001364
Georg Brandlb80f5112012-09-30 09:11:58 +02001365Deprecated
1366==========
1367
R David Murrayd74d09a2013-12-24 15:33:02 -05001368This section covers various APIs and other features that have been deprecated
1369in Python 3.4, and will be removed in Python 3.5 or later. In most (but not
1370all) cases, using the deprecated APIs will produce a :exc:`DeprecationWarning`
1371when the interpreter is run with deprecation warnings enabled (for example, by
1372using ``-Wd``).
1373
Georg Brandlb80f5112012-09-30 09:11:58 +02001374
R David Murraye7cf6782013-12-24 14:51:25 -05001375Deprecated Python Modules, Functions and Methods
Georg Brandlb80f5112012-09-30 09:11:58 +02001376------------------------------------------------
1377
Terry Jan Reedy2b6c26e2013-03-21 19:36:26 -04001378* :meth:`difflib.SequenceMatcher.isbjunk` and
Andrew Kuchling0d0813a2013-06-15 13:29:09 -04001379 :meth:`difflib.SequenceMatcher.isbpopular` were removed: use ``x in sm.bjunk`` and
1380 ``x in sm.bpopular``, where *sm* is a :class:`~difflib.SequenceMatcher` object.
Georg Brandlb80f5112012-09-30 09:11:58 +02001381
Brett Cannon82b3d6a2013-06-14 22:37:11 -04001382* :func:`importlib.util.module_for_loader` is pending deprecation. Using
1383 :func:`importlib.util.module_to_load` and
1384 :meth:`importlib.abc.Loader.init_module_attrs` allows subclasses of a loader
1385 to more easily customize module loading.
1386
Brett Cannone4f41de2013-06-16 13:13:40 -04001387* The :mod:`imp` module is pending deprecation. To keep compatibility with
1388 Python 2/3 code bases, the module's removal is currently not scheduled.
1389
Brett Cannon1448ecf2013-10-04 11:38:59 -04001390* The :mod:`formatter` module is pending deprecation and is slated for removal
1391 in Python 3.6.
1392
Christian Heimes634919a2013-11-20 17:23:06 +01001393* MD5 as default digestmod for :mod:`hmac` is deprecated. Python 3.6 will
1394 require an explicit digest name or constructor as *digestmod* argument.
1395
R David Murray9025f1c2014-01-02 13:44:18 -05001396* The internal ``Netrc`` class in the :mod:`ftplib` module has been documented
1397 as deprecated in its docstring for quite some time. It now emits a
1398 :exc:`DeprecationWarning` and will be removed completely in Python 3.5.
1399
Georg Brandlb80f5112012-09-30 09:11:58 +02001400
R David Murraye7cf6782013-12-24 14:51:25 -05001401Deprecated Functions and Types in the C API
Georg Brandlb80f5112012-09-30 09:11:58 +02001402-------------------------------------------
1403
R David Murrayd74d09a2013-12-24 15:33:02 -05001404XXX: None so far
Georg Brandlb80f5112012-09-30 09:11:58 +02001405
1406
R David Murraye7cf6782013-12-24 14:51:25 -05001407Deprecated Features
Georg Brandlb80f5112012-09-30 09:11:58 +02001408-------------------
1409
R David Murray5ea95242013-12-24 15:59:57 -05001410* Running :ref:`idle` with the ``-n`` flag (no subprocess) is deprecated.
1411 However, the feature will not be removed until :issue:`18823` is resolved.
1412
Antoine Pitrou3b2f0f02013-10-25 21:39:26 +02001413* The site module adding a "site-python" directory to sys.path, if it
1414 exists, is deprecated (:issue:`19375`).
Georg Brandlb80f5112012-09-30 09:11:58 +02001415
1416
R David Murray59171722014-01-03 15:52:22 -05001417
R David Murray72420ff2013-12-24 10:46:44 -05001418Removed
1419=======
1420
R David Murray59171722014-01-03 15:52:22 -05001421
1422Operating Systems No Longer Supported
1423-------------------------------------
1424
1425Support for the following operating systems has been removed from the source
1426and build tools:
1427
1428* OS/2 (:issue:`16135`).
1429* Windows 2000 (changeset e52df05b496a).
1430* VMS (:issue:`16136`).
1431
1432
1433API and Feature Removals
1434------------------------
1435
R David Murrayd2653812013-12-27 14:06:15 -05001436The following obsolete and previously deprecated APIs and features have been
R David Murray59171722014-01-03 15:52:22 -05001437removed:
R David Murray076dead2013-12-24 14:59:50 -05001438
R David Murray72420ff2013-12-24 10:46:44 -05001439* The unmaintained ``Misc/TextMate`` and ``Misc/vim`` directories have been
R David Murrayd91ba202013-12-24 12:13:44 -05001440 removed (see the `devguide <http://docs.python.org/devguide>`_
R David Murray00569362014-01-03 13:04:25 -05001441 for suggestions on what to use instead).
R David Murray72420ff2013-12-24 10:46:44 -05001442
R David Murrayd17aba72013-12-24 14:46:23 -05001443* The ``SO`` makefile macro is removed (it was replaced by the
1444 ``SHLIB_SUFFIX`` and ``EXT_SUFFIX`` macros) (:issue:`16754`).
1445
R David Murrayd74d09a2013-12-24 15:33:02 -05001446* The ``PyThreadState.tick_counter`` field has been removed; its value has
1447 been meaningless since Python 3.2, when the "new GIL" was introduced.
1448
R David Murrayd2653812013-12-27 14:06:15 -05001449* ``PyLoader`` and ``PyPycLoader`` have been removed from :mod:`importlib`.
1450 (Contributed by Taras Lyapun in :issue:`15641`.)
1451
R David Murraya2924ca2014-01-03 14:06:01 -05001452* The *strict* argument to :class:`~http.client.HTTPConnection` and
1453 :class:`~http.client.HTTPSConnection` has been removed. HTTP 0.9-style
1454 "Simple Responses" are no longer supported.
R David Murray00569362014-01-03 13:04:25 -05001455
R David Murraydb085f52014-01-03 15:46:24 -05001456* The deprecated :mod:`urllib.request.Request` getter and setter methods
1457 ``add_data``, ``has_data``, ``get_data``, ``get_type``, ``get_host``,
1458 ``get_selector``, ``set_proxy``, ``get_origin_req_host``, and
1459 ``is_unverifiable`` have been removed (use direct attribute access instead).
1460
R David Murray244ad602014-01-04 21:17:52 -05001461* Support for loading the deprecated ``TYPE_INT64`` has been removed from
1462 :mod:`marshal`. (Contributed by Dan Riti in :issue:`15480`.)
1463
R David Murray72420ff2013-12-24 10:46:44 -05001464
R David Murray98358272014-01-05 20:52:06 -05001465Code Cleanups
1466-------------
1467
1468* The unused and undocumented internal ``Scanner`` class has been removed from
1469 the :mod:`pydoc` module.
1470
1471
R David Murraye6082552014-01-03 16:15:45 -05001472
Benjamin Peterson88f3b232012-10-04 12:45:10 -04001473Porting to Python 3.4
Georg Brandlb80f5112012-09-30 09:11:58 +02001474=====================
1475
1476This section lists previously described changes and other bugfixes
1477that may require changes to your code.
1478
R David Murrayd74d09a2013-12-24 15:33:02 -05001479Changes in the Python API
1480-------------------------
1481
Brett Cannon777622b2013-04-09 17:03:10 -04001482* The ABCs defined in :mod:`importlib.abc` now either raise the appropriate
1483 exception or return a default value instead of raising
1484 :exc:`NotImplementedError` blindly. This will only affect code calling
1485 :func:`super` and falling through all the way to the ABCs. For compatibility,
1486 catch both :exc:`NotImplementedError` or the appropriate exception as needed.
Brett Cannon4c14b5d2013-05-04 13:56:58 -04001487
1488* The module type now initializes the :attr:`__package__` and :attr:`__loader__`
1489 attributes to ``None`` by default. To determine if these attributes were set
1490 in a backwards-compatible fashion, use e.g.
Brett Cannon3dc48d62013-05-28 18:35:54 -04001491 ``getattr(module, '__loader__', None) is not None``.
1492
1493* :meth:`importlib.util.module_for_loader` now sets ``__loader__`` and
1494 ``__package__`` unconditionally to properly support reloading. If this is not
1495 desired then you will need to set these attributes manually. You can use
Brett Cannon028d5122013-05-31 18:02:11 -04001496 :func:`importlib.util.module_to_load` for module management.
Brett Cannon3e0651b2013-05-31 23:18:39 -04001497
1498* Import now resets relevant attributes (e.g. ``__name__``, ``__loader__``,
1499 ``__package__``, ``__file__``, ``__cached__``) unconditionally when reloading.
1500
Brett Cannon1448ecf2013-10-04 11:38:59 -04001501* Frozen packages no longer set ``__path__`` to a list containing the package
1502 name but an empty list instead. Determing if a module is a package should be
1503 done using ``hasattr(module, '__path__')``.
Brett Cannon8f5ac512013-06-12 23:29:18 -04001504
Brett Cannon33915eb2013-06-14 18:33:00 -04001505* :func:`py_compile.compile` now raises :exc:`FileExistsError` if the file path
1506 it would write to is a symlink or a non-regular file. This is to act as a
1507 warning that import will overwrite those files with a regular file regardless
1508 of what type of file path they were originally.
Brett Cannonf4375ef2013-06-16 18:05:54 -04001509
1510* :meth:`importlib.abc.SourceLoader.get_source` no longer raises
1511 :exc:`ImportError` when the source code being loaded triggers a
1512 :exc:`SyntaxError` or :exc:`UnicodeDecodeError`. As :exc:`ImportError` is
1513 meant to be raised only when source code cannot be found but it should, it was
1514 felt to be over-reaching/overloading of that meaning when the source code is
1515 found but improperly structured. If you were catching ImportError before and
1516 wish to continue to ignore syntax or decoding issues, catch all three
Victor Stinner84e33c82013-06-21 00:31:55 +02001517 exceptions now.
Nick Coghlan24c05bc2013-07-15 21:13:08 +10001518
1519* :func:`functools.update_wrapper` and :func:`functools.wraps` now correctly
Nick Coghlan367df122013-10-27 01:57:34 +10001520 set the ``__wrapped__`` attribute to the function being wrapper, even if
1521 that function also had its ``__wrapped__`` attribute set. This means
1522 ``__wrapped__`` attributes now correctly link a stack of decorated
1523 functions rather than every ``__wrapped__`` attribute in the chain
1524 referring to the innermost function. Introspection libraries that
1525 assumed the previous behaviour was intentional can use
1526 :func:`inspect.unwrap` to access the first function in the chain that has
1527 no ``__wrapped__`` attribute.
Victor Stinner2fe9bac2013-10-10 16:18:20 +02001528
Georg Brandl0f5bff22013-10-19 17:46:38 +02001529* :class:`importlib.machinery.PathFinder` now passes on the current working
Brett Cannon27e27f72013-10-18 11:39:04 -04001530 directory to objects in :data:`sys.path_hooks` for the empty string. This
1531 results in :data:`sys.path_importer_cache` never containing ``''``, thus
1532 iterating through :data:`sys.path_importer_cache` based on :data:`sys.path`
1533 will not find all keys. A module's ``__file__`` when imported in the current
1534 working directory will also now have an absolute path, including when using
1535 ``-m`` with the interpreter (this does not influence when the path to a file
1536 is specified on the command-line).
Victor Stinner2748bc72013-12-13 10:57:04 +01001537
R David Murraya2924ca2014-01-03 14:06:01 -05001538* The removal of the *strict* argument to :class:`~http.client.HTTPConnection`
1539 and :class:`~http.client.HTTPSConnection` changes the meaning of the
1540 remaining arguments if you are specifying them positionally rather than by
1541 keyword. If you've been paying attention to deprecation warnings your code
1542 should already be specifying any additional arguments via keywords.
1543
R David Murray4908f4a2014-01-04 18:07:20 -05001544* Strings between ``from __future__ import ...`` statements now *always* raise
1545 a :exc:`SyntaxError`. Previously if there was no leading docstring, an
1546 interstitial string would sometimes be ignored. This brings CPython into
1547 compliance with the language spec; Jython and PyPy already were.
1548 (:issue:`17434`).
1549
R David Murraya101bdb2014-01-06 16:32:05 -05001550* :meth:`ssl.SSLSocket.getpeercert` and :meth:`ssl.SSLSocket.do_handshake`
1551 now raise an :exc:`OSError` with ``ENOTCONN`` when the ``SSLSocket`` is not
1552 connected, instead of the previous behavior of raising an
1553 :exc:`AttributError`. In addition, :meth:`~ssl.SSLSocket.getpeercert`
1554 will raise a :exc:`ValueError` if the handshake has not yet been done.
1555
R David Murray0ae7ae12014-01-08 18:16:02 -05001556* :func:`base64.b32decode` now raises a :exc:`binascii.Error` when the
1557 input string contains non-b32-alphabet characters, instead of a
1558 :exc:`TypeError`. This particular :exc:`TypeError` was missed when the other
1559 :exc:`TypeError`\ s were converted. (Contributed by Serhiy Storchaka in
1560 :issue:`18011`.) Note: this change was also inadvertently applied in Python
1561 3.3.3.
1562
Brett Cannonc089f702014-01-17 11:03:19 -05001563* The :attr:`~cgi.FieldStorage.file` attribute is now automatically closed when
1564 the creating :class:`cgi.FieldStorage` instance is garbage collected. If you
1565 were pulling the file object out separately from the :class:`cgi.FieldStorage`
1566 instance and not keeping the instance alive, then you should either store the
1567 entire :class:`cgi.FieldStorage` instance or read the contents of the file
1568 before the :class:`cgi.FieldStorage` instance is garbage collected.
1569
R David Murrayd2653812013-12-27 14:06:15 -05001570
Victor Stinner774b2e02013-12-13 14:33:01 +01001571Changes in the C API
1572--------------------
1573
1574* :c:func:`PyErr_SetImportError` now sets :exc:`TypeError` when its **msg**
1575 argument is not set. Previously only ``NULL`` was returned with no exception
1576 set.
1577
1578* The result of the :c:data:`PyOS_ReadlineFunctionPointer` callback must
1579 now be a string allocated by :c:func:`PyMem_RawMalloc` or
1580 :c:func:`PyMem_RawRealloc`, or *NULL* if an error occurred, instead of a
1581 string allocated by :c:func:`PyMem_Malloc` or :c:func:`PyMem_Realloc`.
1582
1583* :c:func:`PyThread_set_key_value` now always set the value. In Python
Victor Stinner2748bc72013-12-13 10:57:04 +01001584 3.3, the function did nothing if the key already exists (if the current
1585 value is a non-NULL pointer).
1586
Victor Stinner774b2e02013-12-13 14:33:01 +01001587* The ``f_tstate`` (thread state) field of the :c:type:`PyFrameObject`
1588 structure has been removed to fix a bug: see :issue:`14432` for the
1589 rationale.
1590