blob: 78d58415b28abc082abc59e37ffaa1466d5cc4d3 [file] [log] [blame]
Georg Brandlb80f5112012-09-30 09:11:58 +02001****************************
2 What's New In Python 3.4
3****************************
4
Georg Brandl3b80d342012-10-28 13:37:54 +01005.. :Author: Someone <email>
6 (uncomment if there is a principal author)
Georg Brandlb80f5112012-09-30 09:11:58 +02007
8.. Rules for maintenance:
9
Nick Coghlan03074fd2012-09-30 18:51:53 +053010 * Anyone can add text to this document, but the maintainer reserves the
11 right to rewrite any additions. In particular, for obscure or esoteric
12 features, the maintainer may reduce any addition to a simple reference to
13 the new documentation rather than explaining the feature inline.
Georg Brandlb80f5112012-09-30 09:11:58 +020014
Nick Coghlan03074fd2012-09-30 18:51:53 +053015 * While the maintainer will periodically go through Misc/NEWS
16 and add changes, it's best not to rely on this. We know from experience
17 that any changes that aren't in the What's New documentation around the
18 time of the original release will remain largely unknown to the community
19 for years, even if they're added later. We also know from experience that
20 other priorities can arise, and the maintainer will run out of time to do
R David Murray061efb12013-12-24 12:35:59 -050021 updates -- in such cases, end users will be much better served by partial
Nick Coghlan03074fd2012-09-30 18:51:53 +053022 notifications that at least give a hint about new features to
23 investigate.
Georg Brandlb80f5112012-09-30 09:11:58 +020024
25 * This is not a complete list of every single change; completeness
Nick Coghlan03074fd2012-09-30 18:51:53 +053026 is the purpose of Misc/NEWS. The What's New should focus on changes that
27 are visible to Python *users* and that *require* a feature release (i.e.
28 most bug fixes should only be recorded in Misc/NEWS)
29
30 * PEPs should not be marked Final until they have an entry in What's New.
31 A placeholder entry that is just a section header and a link to the PEP
32 (e.g ":pep:`397` has been implemented") is acceptable. If a PEP has been
33 implemented and noted in What's New, don't forget to mark it as Final!
Georg Brandlb80f5112012-09-30 09:11:58 +020034
35 * If you want to draw your new text to the attention of the
36 maintainer, add 'XXX' to the beginning of the paragraph or
37 section.
38
Nick Coghlan03074fd2012-09-30 18:51:53 +053039 * It's OK to add just a very brief note about a change. For
40 example: "The :ref:`~socket.transmogrify()` function was added to the
41 :mod:`socket` module." The maintainer will research the change and
42 write the necessary text (if appropriate). The advantage of doing this
43 is that even if no more descriptive text is ever added, readers will at
44 least have a notification that the new feature exists and a link to the
45 relevant documentation.
Georg Brandlb80f5112012-09-30 09:11:58 +020046
47 * You can comment out your additions if you like, but it's not
48 necessary (especially when a final release is some months away).
49
50 * Credit the author of a patch or bugfix. Just the name is
51 sufficient; the e-mail address isn't necessary.
52
53 * It's helpful to add the bug/patch number as a comment:
54
Nick Coghlan03074fd2012-09-30 18:51:53 +053055 The :ref:`~socket.transmogrify()` function was added to the
56 :mod:`socket` module. (Contributed by P.Y. Developer in :issue:`12345`.)
Georg Brandlb80f5112012-09-30 09:11:58 +020057
58 This saves the maintainer the effort of going through the Mercurial log
59 when researching a change.
60
Nick Coghlan03074fd2012-09-30 18:51:53 +053061 * Cross referencing tip: :ref:`mod.attr` will display as ``mod.attr``,
62 while :ref:`~mod.attr` will display as ``attr``.
63
Georg Brandlb80f5112012-09-30 09:11:58 +020064This article explains the new features in Python 3.4, compared to 3.3.
65
Nick Coghlan03074fd2012-09-30 18:51:53 +053066.. Python 3.4 was released on TBD.
Georg Brandlb80f5112012-09-30 09:11:58 +020067
Ezio Melotti25bbe5e2012-11-17 19:30:48 +020068For full details, see the
69`changelog <http://docs.python.org/3.4/whatsnew/changelog.html>`_.
Georg Brandlb80f5112012-09-30 09:11:58 +020070
71.. note:: Prerelease users should be aware that this document is currently in
72 draft form. It will be updated substantially as Python 3.4 moves towards
73 release, so it's worth checking back even after reading earlier versions.
74
75
76.. seealso::
77
R David Murray061efb12013-12-24 12:35:59 -050078 :pep:`429` -- Python 3.4 Release Schedule
Georg Brandlb80f5112012-09-30 09:11:58 +020079
80
R David Murraye6082552014-01-03 16:15:45 -050081
R David Murraye7cf6782013-12-24 14:51:25 -050082Summary -- Release Highlights
Georg Brandlb80f5112012-09-30 09:11:58 +020083=============================
84
Kristjan Valur Jonssona1e82442013-03-26 13:56:14 +000085.. This section singles out the most important changes in Python 3.4.
Georg Brandlb80f5112012-09-30 09:11:58 +020086 Brevity is key.
87
88New syntax features:
89
R David Murraye0f7a782013-12-20 16:04:29 -050090* No new syntax features were added in Python 3.4.
Georg Brandlb80f5112012-09-30 09:11:58 +020091
R David Murray809487e2013-12-22 20:49:40 -050092New expected features for Python implementations:
93
R David Murray68790662013-12-23 11:17:51 -050094* :ref:`pip should always be "available" <whatsnew-pep-453>` (:pep:`453`).
R David Murray809487e2013-12-22 20:49:40 -050095* :ref:`Make newly created file descriptors non-inheritable <whatsnew-pep-446>`
96 (:pep:`446`).
97* command line option for :ref:`isolated mode <using-on-misc-options>`,
98 (:issue:`16499`).
R David Murray8d856782013-12-23 10:28:57 -050099* :ref:`improvements in the handling of codecs <codec-handling-improvements>`
100 that are not text encodings (multiple issues).
R David Murray809487e2013-12-22 20:49:40 -0500101* :ref:`A ModuleSpec Type <whatsnew-pep-451>` for the Import System
R David Murraye9b74d42013-12-22 21:05:04 -0500102 (:pep:`451`). (Affects importer authors.)
R David Murray809487e2013-12-22 20:49:40 -0500103
Georg Brandlb80f5112012-09-30 09:11:58 +0200104New library modules:
105
R David Murray9217dad2013-12-23 21:08:28 -0500106* :mod:`asyncio`: :ref:`New provisional API for asynchronous IO
107 <whatsnew-asyncio>` (:pep:`3156`).
108* :mod:`ensurepip`: :ref:`Bootstrapping the pip installer <whatsnew-ensurepip>`
109 (:pep:`453`).
110* :mod:`enum`: :ref:`Support for enumeration types <whatsnew-enum>`
111 (:pep:`435`).
112* :mod:`pathlib`: :ref:`Object-oriented filesystem paths <whatsnew-pathlib>`
113 (:pep:`428`).
R David Murrayf9976e72013-12-23 10:32:02 -0500114* :mod:`selectors`: :ref:`High-level and efficient I/O multiplexing
115 <whatsnew-selectors>`, built upon the :mod:`select` module primitives (part
116 of :pep:`3156`).
R David Murray9217dad2013-12-23 21:08:28 -0500117* :mod:`statistics`: A basic :ref:`numerically stable statistics library
118 <whatsnew-statistics>` (:pep:`450`).
119* :mod:`tracemalloc`: :ref:`Trace Python memory allocations
120 <whatsnew-tracemalloc>` (:pep:`454`).
Georg Brandlb80f5112012-09-30 09:11:58 +0200121
Georg Brandlb80f5112012-09-30 09:11:58 +0200122Significantly Improved Library Modules:
123
R David Murray0a102162013-12-20 15:00:54 -0500124* :ref:`Single-dispatch generic functions <whatsnew-singledispatch>` in
R David Murray8d856782013-12-23 10:28:57 -0500125 :mod:`functools` (:pep:`443`).
126* New :mod:`pickle` :ref:`protocol 4 <whatsnew-protocol-4>` (:pep:`3154`).
R David Murray8d856782013-12-23 10:28:57 -0500127* :ref:`TLSv1.1 and TLSv1.2 support <whatsnew-tls-11-12>` for :mod:`ssl`
128 (:issue:`16692`).
R David Murrayac186222013-12-20 17:23:57 -0500129* :mod:`multiprocessing` now has :ref:`an option to avoid using os.fork
130 on Unix <whatsnew-multiprocessing-no-fork>` (:issue:`8713`).
R David Murray6adb4542013-12-20 13:10:43 -0500131* :mod:`email` has a new submodule, :mod:`~email.contentmanager`, and
132 a new :mod:`~email.message.Message` subclass
R David Murray26b80cfd2013-12-20 17:26:52 -0500133 (:class:`~email.contentmanager.EmailMessage`) that :ref:`simplify MIME
R David Murray8d856782013-12-23 10:28:57 -0500134 handling <whatsnew_email_contentmanager>` (:issue:`18891`).
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +0200135
Nick Coghlan367df122013-10-27 01:57:34 +1000136CPython implementation improvements:
Georg Brandlb80f5112012-09-30 09:11:58 +0200137
R David Murraya93ca972013-12-22 14:10:21 -0500138* :ref:`Safe object finalization <whatsnew-pep-442>` (:pep:`442`).
R David Murrayca794612013-12-22 14:05:11 -0500139* Leveraging :pep:`442`, :ref:`module globals are no longer set to None
140 during finalization <whatsnew-pep-442>`, in most cases (:issue:`18214`).
R David Murraya93ca972013-12-22 14:10:21 -0500141* :ref:`Configurable memory allocators <whatsnew-pep-445>` (:pep:`445`).
142* :ref:`Secure and interchangeable hash algorithm <whatsnew-pep-456>`
143 (:pep:`456`).
144* :ref:`Argument Clinic <whatsnew-pep-436>` (:pep:`436`).
R David Murray8f7664a2013-12-22 20:40:11 -0500145* The :mod:`marshal` format has been made :ref:`more compact and efficient
146 <whatsnew-marshal-3>` (:issue:`16475`).
Georg Brandlb80f5112012-09-30 09:11:58 +0200147
R David Murray347f9c72013-12-23 21:30:06 -0500148Please read on for a comprehensive list of user-facing changes, including many
149other smaller improvements, CPython optimizations, deprecations, and potential
150porting issues.
Georg Brandlb80f5112012-09-30 09:11:58 +0200151
Nick Coghland0cf0632013-11-11 22:11:55 +1000152
R David Murraye6082552014-01-03 16:15:45 -0500153
R David Murraye7cf6782013-12-24 14:51:25 -0500154New Expected Features for Python Implementations
R David Murray809487e2013-12-22 20:49:40 -0500155================================================
156
R David Murray68790662013-12-23 11:17:51 -0500157.. _whatsnew-pep-453:
158
R David Murraye7cf6782013-12-24 14:51:25 -0500159PEP 453: Explicit Bootstrapping of PIP in Python Installations
R David Murray809487e2013-12-22 20:49:40 -0500160--------------------------------------------------------------
Nick Coghland0cf0632013-11-11 22:11:55 +1000161
162The new :mod:`ensurepip` module (defined in :pep:`453`) provides a standard
163cross-platform mechanism to boostrap the pip installer into Python
164installations and virtual environments.
165
Nick Coghlan7bc4b3b2013-11-23 11:59:40 +1000166The :mod:`venv` module and the :command:`pyvenv` utility make use of this
167module to make ``pip`` readily available in virtual environments. When
168using the command line interface, ``pip`` is installed by default, while
169for the module API installation of ``pip`` must be requested explicitly.
170
171For CPython source builds on POSIX systems, the ``make install`` and
172``make altinstall`` commands bootstrap ``pip`` by default. This behaviour
173can be controlled through configure options, and overridden through
174Makefile options.
175
Ned Deily44a0db02013-11-22 22:39:09 -0800176On Windows and Mac OS X, the CPython installers now offer the option to
177install ``pip`` along with CPython itself.
Nick Coghlan7bc4b3b2013-11-23 11:59:40 +1000178
R David Murray68790662013-12-23 11:17:51 -0500179As `discussed in the PEP`__, platform packagers may choose not to install
180``pip`` by default, as long as the command ``pip``, when invoked, provides
181clear and simple directions on how to install ``pip`` on the platform.
182
183__ http://www.python.org/dev/peps/pep-0453/#recommendations-for-downstream-distributors
184
Nick Coghland0cf0632013-11-11 22:11:55 +1000185.. note::
186
Nick Coghlan7bc4b3b2013-11-23 11:59:40 +1000187 The implementation of PEP 453 is still a work in progress. Refer to
188 :issue:`19347` for the progress on additional steps:
Nick Coghland0cf0632013-11-11 22:11:55 +1000189
Nick Coghlan7bc4b3b2013-11-23 11:59:40 +1000190 * Having the binary installers install ``pip`` by default
191 * Recommending the use of ``pip`` in the "Installing Python Module"
192 documentation.
Nick Coghland0cf0632013-11-11 22:11:55 +1000193
194.. seealso::
195
R David Murray061efb12013-12-24 12:35:59 -0500196 :pep:`453` -- Explicit bootstrapping of pip in Python installations
Nick Coghland0cf0632013-11-11 22:11:55 +1000197 PEP written by Donald Stufft and Nick Coghlan, implemented by
Nick Coghlan7bc4b3b2013-11-23 11:59:40 +1000198 Donald Stufft, Nick Coghlan, Martin von Löwis and Ned Deily.
Nick Coghland0cf0632013-11-11 22:11:55 +1000199
200
R David Murrayf9909c22013-12-20 14:50:12 -0500201.. _whatsnew-pep-446:
Victor Stinnerdaf45552013-08-28 00:53:59 +0200202
R David Murraye7cf6782013-12-24 14:51:25 -0500203PEP 446: Make Newly Created File Descriptors Non-Inheritable
R David Murray809487e2013-12-22 20:49:40 -0500204------------------------------------------------------------
Victor Stinnerdaf45552013-08-28 00:53:59 +0200205
Nick Coghlan367df122013-10-27 01:57:34 +1000206:pep:`446` makes newly created file descriptors :ref:`non-inheritable
Georg Brandl5642ff92013-09-15 10:37:57 +0200207<fd_inheritance>`. New functions and methods:
Victor Stinnerdaf45552013-08-28 00:53:59 +0200208
209* :func:`os.get_inheritable`, :func:`os.set_inheritable`
210* :func:`os.get_handle_inheritable`, :func:`os.set_handle_inheritable`
211* :meth:`socket.socket.get_inheritable`, :meth:`socket.socket.set_inheritable`
212
Antoine Pitrou796564c2013-07-30 19:59:21 +0200213.. seealso::
214
R David Murray061efb12013-12-24 12:35:59 -0500215 :pep:`446` -- Make newly created file descriptors non-inheritable
Nick Coghlan367df122013-10-27 01:57:34 +1000216 PEP written and implemented by Victor Stinner.
Antoine Pitrou796564c2013-07-30 19:59:21 +0200217
Georg Brandlb80f5112012-09-30 09:11:58 +0200218
Nick Coghlan9c1aed82013-11-23 11:13:36 +1000219.. _codec-handling-improvements:
220
R David Murraye7cf6782013-12-24 14:51:25 -0500221Improvements to Codec Handling
R David Murray809487e2013-12-22 20:49:40 -0500222------------------------------
Nick Coghlan8b097b42013-11-13 23:49:21 +1000223
224Since it was first introduced, the :mod:`codecs` module has always been
225intended to operate as a type-neutral dynamic encoding and decoding
226system. However, its close coupling with the Python text model, especially
227the type restricted convenience methods on the builtin :class:`str`,
228:class:`bytes` and :class:`bytearray` types, has historically obscured that
229fact.
230
231As a key step in clarifying the situation, the :meth:`codecs.encode` and
232:meth:`codecs.decode` convenience functions are now properly documented in
233Python 2.7, 3.3 and 3.4. These functions have existed in the :mod:`codecs`
Nick Coghlan9c1aed82013-11-23 11:13:36 +1000234module (and have been covered by the regression test suite) since Python 2.4,
Nick Coghlan8b097b42013-11-13 23:49:21 +1000235but were previously only discoverable through runtime introspection.
236
237Unlike the convenience methods on :class:`str`, :class:`bytes` and
238:class:`bytearray`, these convenience functions support arbitrary codecs
239in both Python 2 and Python 3, rather than being limited to Unicode text
240encodings (in Python 3) or ``basestring`` <-> ``basestring`` conversions
241(in Python 2).
242
Nick Coghlan8afc8f62013-11-22 23:00:22 +1000243In Python 3.4, the interpreter is able to identify the known non-text
244encodings provided in the standard library and direct users towards these
245general purpose convenience functions when appropriate::
Nick Coghlan8b097b42013-11-13 23:49:21 +1000246
Nick Coghlan9c1aed82013-11-23 11:13:36 +1000247 >>> b"abcdef".decode("hex")
Nick Coghlan8b097b42013-11-13 23:49:21 +1000248 Traceback (most recent call last):
249 File "<stdin>", line 1, in <module>
Nick Coghlan9c1aed82013-11-23 11:13:36 +1000250 LookupError: 'hex' is not a text encoding; use codecs.decode() to handle arbitrary codecs
Nick Coghlan8b097b42013-11-13 23:49:21 +1000251
Nick Coghlan9c1aed82013-11-23 11:13:36 +1000252 >>> "hello".encode("rot13")
Nick Coghlan8b097b42013-11-13 23:49:21 +1000253 Traceback (most recent call last):
254 File "<stdin>", line 1, in <module>
Nick Coghlan9c1aed82013-11-23 11:13:36 +1000255 LookupError: 'rot13' is not a text encoding; use codecs.encode() to handle arbitrary codecs
Nick Coghlan8b097b42013-11-13 23:49:21 +1000256
257In a related change, whenever it is feasible without breaking backwards
258compatibility, exceptions raised during encoding and decoding operations
259will be wrapped in a chained exception of the same type that mentions the
260name of the codec responsible for producing the error::
261
Nick Coghlan9c1aed82013-11-23 11:13:36 +1000262 >>> import codecs
263
264 >>> codecs.decode(b"abcdefgh", "hex")
Nick Coghlan8afc8f62013-11-22 23:00:22 +1000265 binascii.Error: Non-hexadecimal digit found
Nick Coghlan8b097b42013-11-13 23:49:21 +1000266
267 The above exception was the direct cause of the following exception:
268
269 Traceback (most recent call last):
270 File "<stdin>", line 1, in <module>
Nick Coghlan9c1aed82013-11-23 11:13:36 +1000271 binascii.Error: decoding with 'hex' codec failed (Error: Non-hexadecimal digit found)
Nick Coghlan8b097b42013-11-13 23:49:21 +1000272
Nick Coghlan9c1aed82013-11-23 11:13:36 +1000273 >>> codecs.encode("hello", "bz2")
Nick Coghlan8b097b42013-11-13 23:49:21 +1000274 TypeError: 'str' does not support the buffer interface
275
276 The above exception was the direct cause of the following exception:
277
278 Traceback (most recent call last):
279 File "<stdin>", line 1, in <module>
Nick Coghlan9c1aed82013-11-23 11:13:36 +1000280 TypeError: encoding with 'bz2' codec failed (TypeError: 'str' does not support the buffer interface)
Nick Coghlan8b097b42013-11-13 23:49:21 +1000281
Nick Coghlan9c1aed82013-11-23 11:13:36 +1000282Finally, as the examples above show, these improvements have permitted
283the restoration of the convenience aliases for the non-Unicode codecs that
284were themselves restored in Python 3.2. This means that encoding binary data
285to and from its hexadecimal representation (for example) can now be written
286as::
287
288 >>> from codecs import encode, decode
289 >>> encode(b"hello", "hex")
290 b'68656c6c6f'
291 >>> decode(b"68656c6c6f", "hex")
292 b'hello'
293
294The binary and text transforms provided in the standard library are detailed
295in :ref:`binary-transforms` and :ref:`text-transforms`.
296
R David Murray9217dad2013-12-23 21:08:28 -0500297(Contributed by Nick Coghlan in :issue:`7475`, , :issue:`17827`,
Nick Coghlan9c1aed82013-11-23 11:13:36 +1000298:issue:`17828` and :issue:`19619`)
Nick Coghlan8b097b42013-11-13 23:49:21 +1000299
R David Murray809487e2013-12-22 20:49:40 -0500300.. _whatsnew-pep-451:
Eric Snowb523f842013-11-22 09:05:39 -0700301
302PEP 451: A ModuleSpec Type for the Import System
R David Murray809487e2013-12-22 20:49:40 -0500303------------------------------------------------
Eric Snowb523f842013-11-22 09:05:39 -0700304
R David Murrayf85b2a82013-12-22 21:06:13 -0500305:pep:`451` provides an encapsulation of the information about a module that the
306import machinery will use to load it (that is, a module specification). This
307helps simplify both the import implementation and several import-related APIs.
308The change is also a stepping stone for `several future import-related
309improvements`__.
Eric Snowb523f842013-11-22 09:05:39 -0700310
R David Murraye9b74d42013-12-22 21:05:04 -0500311__ https://mail.python.org/pipermail/python-dev/2013-November/130111.html
Eric Snowb523f842013-11-22 09:05:39 -0700312
313The public-facing changes from the PEP are entirely backward-compatible.
R David Murrayf85b2a82013-12-22 21:06:13 -0500314Furthermore, they should be transparent to everyone but importer authors. Key
315finder and loader methods have been deprecated, but they will continue working.
316New importers should use the new methods described in the PEP. Existing
317importers should be updated to implement the new methods.
Eric Snowb523f842013-11-22 09:05:39 -0700318
Nick Coghlan8b097b42013-11-13 23:49:21 +1000319
Georg Brandlb80f5112012-09-30 09:11:58 +0200320Other Language Changes
R David Murray809487e2013-12-22 20:49:40 -0500321----------------------
Georg Brandlb80f5112012-09-30 09:11:58 +0200322
323Some smaller changes made to the core Python language are:
324
Ezio Melotti34808e22013-10-12 16:36:13 +0300325* Unicode database updated to UCD version 6.3.
Georg Brandlb80f5112012-09-30 09:11:58 +0200326
R David Murray9a2f1392013-06-28 13:31:19 -0400327* :func:`min` and :func:`max` now accept a *default* argument that can be used
328 to specify the value they return if the iterable they are evaluating has no
329 elements. Contributed by Julian Berman in :issue:`18111`.
Georg Brandlb80f5112012-09-30 09:11:58 +0200330
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +0200331* Module objects are now :mod:`weakref`'able.
332
Nick Coghlan0acceb72013-10-20 13:22:21 +1000333* Module ``__file__`` attributes (and related values) should now always
334 contain absolute paths by default, with the sole exception of
335 ``__main__.__file__`` when a script has been executed directly using
336 a relative path (Contributed by Brett Cannon in :issue:`18416`).
337
Serhiy Storchaka58cf6072013-11-19 11:32:41 +0200338* Now all the UTF-\* codecs (except UTF-7) reject surrogates during both
339 encoding and decoding unless the ``surrogatepass`` error handler is used,
340 with the exception of the UTF-16 decoder that accepts valid surrogate pairs,
341 and the UTF-16 encoder that produces them while encoding non-BMP characters.
342 Contributed by Victor Stinner, Kang-Hao (Kenny) Lu and Serhiy Storchaka in
343 :issue:`12892`.
344
Georg Brandlb80f5112012-09-30 09:11:58 +0200345
R David Murraye6082552014-01-03 16:15:45 -0500346
Georg Brandlb80f5112012-09-30 09:11:58 +0200347New Modules
348===========
349
Nick Coghlan0acceb72013-10-20 13:22:21 +1000350
R David Murray9217dad2013-12-23 21:08:28 -0500351.. _whatsnew-asyncio:
352
Nick Coghlan0acceb72013-10-20 13:22:21 +1000353asyncio
354-------
355
356The new :mod:`asyncio` module (defined in :pep:`3156`) provides a standard
357pluggable event loop model for Python, providing solid asynchronous IO
358support in the standard library, and making it easier for other event loop
359implementations to interoperate with the standard library and each other.
360
361For Python 3.4, this module is considered a :term:`provisional API`.
362
Nick Coghlan367df122013-10-27 01:57:34 +1000363.. seealso::
364
R David Murray061efb12013-12-24 12:35:59 -0500365 :pep:`3156` -- Asynchronous IO Support Rebooted: the "asyncio" Module
Nick Coghlan367df122013-10-27 01:57:34 +1000366 PEP written and implementation led by Guido van Rossum.
367
R David Murray68790662013-12-23 11:17:51 -0500368
R David Murray9217dad2013-12-23 21:08:28 -0500369.. _whatsnew-ensurepip:
R David Murray68790662013-12-23 11:17:51 -0500370
371ensurepip
372---------
373
374The new :mod:`ensurepip` module is the primary infrastructure for the
375:pep:`453` implementation. In the normal course of events end users will not
376need to interact with this module, but it can be used to manually bootstrap
377``pip`` if the automated bootstrapping into an installation or virtual
378environment was declined.
379
380:mod:`ensurepip` includes a bundled copy of ``pip``, up-to-date as of the first
381release candidate of the release of CPython with which it ships (this applies
382to both maintenance releases and feature releases). ``ensurepip`` does not
383access the internet. (If the installation has Internet access, it is of course
384possible to upgrade ``pip`` to a release more recent than the bundled ``pip``
385by using the bundled ``pip`` command itself once it is installed.)
386
387The module is named *ensure*\ pip because if called when ``pip`` is already
388installed, it does nothing. It also has an ``--upgrade`` option that will
389cause it to install the bundled copy of ``pip`` if the existing installed
390version of ``pip`` is older than the bundled copy.
391
392
R David Murray9217dad2013-12-23 21:08:28 -0500393.. _whatsnew-enum:
394
Nick Coghlan0acceb72013-10-20 13:22:21 +1000395enum
396----
397
Nick Coghlan367df122013-10-27 01:57:34 +1000398The new :mod:`enum` module (defined in :pep:`435`) provides a standard
399implementation of enumeration types, allowing other modules (such as
400:mod:`socket`) to provide more informative error messages and better
401debugging support by replacing opaque integer constants with backwards
402compatible enumeration values.
403
404.. seealso::
405
R David Murray061efb12013-12-24 12:35:59 -0500406 :pep:`435` -- Adding an Enum type to the Python standard library
Nick Coghlan367df122013-10-27 01:57:34 +1000407 PEP written by Barry Warsaw, Eli Bendersky and Ethan Furman,
408 implemented by Ethan Furman.
Nick Coghlan0acceb72013-10-20 13:22:21 +1000409
410
R David Murray9217dad2013-12-23 21:08:28 -0500411.. _whatsnew-pathlib:
412
Antoine Pitrou31119e42013-11-22 17:38:12 +0100413pathlib
414-------
415
416The new :mod:`pathlib` module offers classes representing filesystem paths
417with semantics appropriate for different operating systems. Path classes are
418divided between *pure paths*, which provide purely computational operations
419without I/O, and *concrete paths*, which inherit from pure paths but also
420provide I/O operations.
421
422For Python 3.4, this module is considered a :term:`provisional API`.
423
424.. seealso::
425
R David Murray061efb12013-12-24 12:35:59 -0500426 :pep:`428` -- The pathlib module -- object-oriented filesystem paths
Antoine Pitrou31119e42013-11-22 17:38:12 +0100427 PEP written and implemented by Antoine Pitrou.
428
429
R David Murrayf9976e72013-12-23 10:32:02 -0500430.. _whatsnew-selectors:
431
Charles-François Natali243d8d82013-09-04 19:02:49 +0200432selectors
433---------
Georg Brandlb80f5112012-09-30 09:11:58 +0200434
Nick Coghlan0acceb72013-10-20 13:22:21 +1000435The new :mod:`selectors` module (created as part of implementing :pep:`3156`)
436allows high-level and efficient I/O multiplexing, built upon the
437:mod:`select` module primitives.
438
439
R David Murray9217dad2013-12-23 21:08:28 -0500440.. _whatsnew-statistics:
441
Nick Coghlan0acceb72013-10-20 13:22:21 +1000442statistics
443----------
444
445The new :mod:`statistics` module (defined in :pep:`450`) offers some core
446statistics functionality directly in the standard library. This module
447supports calculation of the mean, median, mode, variance and standard
448deviation of a data series.
Georg Brandlb80f5112012-09-30 09:11:58 +0200449
Nick Coghlan367df122013-10-27 01:57:34 +1000450.. seealso::
451
R David Murray061efb12013-12-24 12:35:59 -0500452 :pep:`450` -- Adding A Statistics Module To The Standard Library
Nick Coghlan367df122013-10-27 01:57:34 +1000453 PEP written and implemented by Steven D'Aprano
454
R David Murray9217dad2013-12-23 21:08:28 -0500455.. _whatsnew-tracemalloc:
Georg Brandlb80f5112012-09-30 09:11:58 +0200456
R David Murray8b2d6822013-12-31 15:06:05 -0500457
Victor Stinnerd2736af2013-11-25 09:40:27 +0100458tracemalloc
459-----------
460
461The new :mod:`tracemalloc` module (defined in :pep:`454`) is a debug tool to
462trace memory blocks allocated by Python. It provides the following information:
463
464* Traceback where an object was allocated
465* Statistics on allocated memory blocks per filename and per line number:
466 total size, number and average size of allocated memory blocks
467* Compute the differences between two snapshots to detect memory leaks
468
469.. seealso::
470
R David Murray061efb12013-12-24 12:35:59 -0500471 :pep:`454` -- Add a new tracemalloc module to trace Python memory allocations
Victor Stinnerd2736af2013-11-25 09:40:27 +0100472 PEP written and implemented by Victor Stinner
473
474
R David Murraye6082552014-01-03 16:15:45 -0500475
Georg Brandlb80f5112012-09-30 09:11:58 +0200476Improved Modules
477================
478
R David Murray6e390152013-12-24 22:28:04 -0500479
R David Murray3edcc782013-12-24 16:13:32 -0500480abc
481---
482
483New function :func:`abc.get_cache_token` can be used to know when to invalidate
484caches that are affected by changes in the object graph. (Contributed
485by Łukasz Langa in :issue:`16832`.)
R David Murray72420ff2013-12-24 10:46:44 -0500486
R David Murray2691ee62013-12-28 23:15:12 -0500487New class :class:`~abc.ABC` has :class:`~abc.ABCMeta` as its meta class.
488Using ``ABC`` as a base class has essentially the same effect as specifying
489``metaclass=abc.ABCMeta``, but is simpler to type and easier to read.
490(Contributed by Bruno Dupuis in :issue:`16049`.)
491
492
Antoine Pitroud6cbd342013-08-12 20:48:15 +0200493aifc
494----
495
496The :meth:`~aifc.getparams` method now returns a namedtuple rather than a
497plain tuple. (Contributed by Claudiu Popa in :issue:`17818`.)
498
R David Murrayd592bb22013-12-31 13:45:38 -0500499:func:`aifc.open` now supports the context manager protocol: when used in a
500:keyword:`with` block, the :meth:`~aifc.aifc.close` method of the returned
501object will be called automatically at the end of the block. (Contributed by
502Serhiy Storchacha in :issue:`16486`.)
503
Antoine Pitroud6cbd342013-08-12 20:48:15 +0200504
R David Murrayfced3ec2013-12-31 11:18:01 -0500505argparse
506--------
507
508The :class:`~argparse.FileType` class now accepts *encoding* and
509*errors* arguments, which are passed through to :func:`open`. (Contributed
510by Lucas Maystre in :issue:`11175`.)
511
512
Serhiy Storchakaeaea5e92013-10-19 21:10:46 +0300513audioop
514-------
515
516Added support for 24-bit samples (:issue:`12866`).
517
Serhiy Storchaka3062c9a2013-11-23 22:26:01 +0200518Added the :func:`~audioop.byteswap` function to convert big-endian samples
519to little-endian and vice versa (:issue:`19641`).
520
Serhiy Storchakaeaea5e92013-10-19 21:10:46 +0300521
Nick Coghland4fdbcc2013-11-14 00:24:31 +1000522base64
523------
524
525The encoding and decoding functions in :mod:`base64` now accept any
526:term:`bytes-like object` in cases where it previously required a
Georg Brandled007d52013-11-24 16:09:26 +0100527:class:`bytes` or :class:`bytearray` instance (:issue:`17839`).
Nick Coghland4fdbcc2013-11-14 00:24:31 +1000528
529
Antoine Pitroud6cbd342013-08-12 20:48:15 +0200530colorsys
531--------
532
533The number of digits in the coefficients for the RGB --- YIQ conversions have
534been expanded so that they match the FCC NTSC versions. The change in
535results should be less than 1% and may better match results found elsewhere.
536
R David Murray8e37d5d2013-04-13 14:49:48 -0400537
Nick Coghlanb4534ae2013-10-13 23:23:08 +1000538contextlib
539----------
540
Nick Coghlan240f86d2013-10-17 23:40:57 +1000541The new :class:`contextlib.suppress` context manager helps to clarify the
542intent of code that deliberately suppresses exceptions from a single
543statement. (Contributed by Raymond Hettinger in :issue:`15806` and
544Zero Piraeus in :issue:`19266`)
545
Victor Stinner6633c392013-10-21 13:27:11 +0200546The new :func:`contextlib.redirect_stdout` context manager makes it easier
Nick Coghlanb4534ae2013-10-13 23:23:08 +1000547for utility scripts to handle inflexible APIs that don't provide any
548options to retrieve their output as a string or direct it to somewhere
Nick Coghlan0acceb72013-10-20 13:22:21 +1000549other than :data:`sys.stdout`. In conjunction with :class:`io.StringIO`,
550this context manager is also useful for checking expected output from
551command line utilities. (Contribute by Raymond Hettinger in :issue:`15805`)
552
553The :mod:`contextlib` documentation has also been updated to include a
554:ref:`discussion <single-use-reusable-and-reentrant-cms>` of the
555differences between single use, reusable and reentrant context managers.
Nick Coghlanb4534ae2013-10-13 23:23:08 +1000556
557
Nick Coghlanb39fd0c2013-05-06 23:59:20 +1000558dis
559---
560
Serhiy Storchaka98b28fd2013-10-13 23:12:09 +0300561The :mod:`dis` module is now built around an :class:`~dis.Instruction` class
562that provides details of individual bytecode operations and a
563:func:`~dis.get_instructions` iterator that emits the Instruction stream for a
Nick Coghlanb39fd0c2013-05-06 23:59:20 +1000564given piece of Python code. The various display tools in the :mod:`dis`
565module have been updated to be based on these new components.
566
567The new :class:`dis.Bytecode` class provides an object-oriented API for
568inspecting bytecode, both in human-readable form and for iterating over
569instructions.
570
Nick Coghlan50c48b82013-11-23 00:57:00 +1000571(Contributed by Nick Coghlan, Ryan Kelly and Thomas Kluyver in :issue:`11816`
572and Claudiu Popa in :issue:`17916`)
Nick Coghlanb39fd0c2013-05-06 23:59:20 +1000573
Antoine Pitroud6cbd342013-08-12 20:48:15 +0200574
R David Murray5a9d7062012-11-21 15:09:21 -0500575doctest
576-------
577
R David Murray473f45b2013-12-27 17:01:16 -0500578A new :ref:`option flag <doctest-options>`, :data:`~doctest.FAIL_FAST`, halts
579test running as soon as the first failure is detected. (Contributed by R.
580David Murray and Daniel Urban in :issue:`16522`.)
R David Murray5707d502013-06-23 14:24:13 -0400581
R David Murray473f45b2013-12-27 17:01:16 -0500582The :mod:`doctest` command line interface now uses :mod:`argparse`, and has two
583new options, ``-o`` and ``-f``. ``-o`` allows :ref:`doctest options
584<doctest-options>` to be specified on the command line, and ``-f`` is a
585shorthand for ``-o FAIL_FAST`` (to parallel the similar option supported by the
586:mod:`unittest` CLI). (Contributed by R. David Murray in :issue:`11390`.)
Georg Brandlb80f5112012-09-30 09:11:58 +0200587
R David Murray8e37d5d2013-04-13 14:49:48 -0400588
R David Murraybb17d2b2013-08-09 16:15:28 -0400589email
590-----
591
592:meth:`~email.message.Message.as_string` now accepts a *policy* argument to
593override the default policy of the message when generating a string
594representation of it. This means that ``as_string`` can now be used in more
595circumstances, instead of having to create and use a :mod:`~email.generator` in
596order to pass formatting parameters to its ``flatten`` method.
597
598New method :meth:`~email.message.Message.as_bytes` added to produce a bytes
599representation of the message in a fashion similar to how ``as_string``
600produces a string representation. It does not accept the *maxheaderlen*
601argument, but does accept the *unixfrom* and *policy* arguments. The
602:class:`~email.message.Message` :meth:`~email.message.Message.__bytes__` method
603calls it, meaning that ``bytes(mymsg)`` will now produce the intuitive
604result: a bytes object containing the fully formatted message.
605
606(Contributed by R. David Murray in :issue:`18600`.)
607
R David Murray26b80cfd2013-12-20 17:26:52 -0500608.. _whatsnew_email_contentmanager:
609
R David Murray3da240f2013-10-16 22:48:40 -0400610A pair of new subclasses of :class:`~email.message.Message` have been added,
611along with a new sub-module, :mod:`~email.contentmanager`. All documentation
612is currently in the new module, which is being added as part of the new
Nick Coghlan240f86d2013-10-17 23:40:57 +1000613:term:`provisional <provisional package>` email API. These classes provide a
R David Murray3da240f2013-10-16 22:48:40 -0400614number of new methods that make extracting content from and inserting content
615into email messages much easier. See the :mod:`~email.contentmanager`
616documentation for details.
617
618These API additions complete the bulk of the work that was planned as part of
619the email6 project. The currently provisional API is scheduled to become final
620in Python 3.5 (possibly with a few minor additions in the area of error
621handling).
622
623(Contributed by R. David Murray in :issue:`18891`.)
624
R David Murraybb17d2b2013-08-09 16:15:28 -0400625
Victor Stinner854ffcb2013-06-21 00:36:30 +0200626functools
627---------
628
Nick Coghlanf4cb48a2013-11-03 16:41:46 +1000629The new :func:`~functools.partialmethod` descriptor bring partial argument
630application to descriptors, just as :func:`~functools.partial` provides
631for normal callables. The new descriptor also makes it easier to get
632arbitrary callables (including :func:`~functools.partial` instances)
633to behave like normal instance methods when included in a class definition.
634
635(Contributed by Alon Horev and Nick Coghlan in :issue:`4331`)
636
R David Murray0a102162013-12-20 15:00:54 -0500637.. _whatsnew-singledispatch:
638
Nick Coghlanf4cb48a2013-11-03 16:41:46 +1000639The new :func:`~functools.singledispatch` decorator brings support for
640single-dispatch generic functions to the Python standard library. Where
641object oriented programming focuses on grouping multiple operations on a
642common set of data into a class, a generic function focuses on grouping
643multiple implementations of an operation that allows it to work with
644*different* kinds of data.
645
646.. seealso::
647
R David Murray061efb12013-12-24 12:35:59 -0500648 :pep:`443` -- Single-dispatch generic functions
Nick Coghlanf4cb48a2013-11-03 16:41:46 +1000649 PEP written and implemented by Łukasz Langa.
Victor Stinner854ffcb2013-06-21 00:36:30 +0200650
R David Murrayd2653812013-12-27 14:06:15 -0500651A pure-python version of the :func:`~functools.partial` function is now in the
652stdlib; in CPython it is overridden by the C accelerated version, but it is
653available for other implementations to use. (Contributed by Brian Thorne in
654:issue:`12428`.)
655
Nick Coghlane8c45d62013-07-28 20:00:01 +1000656
R David Murray0e814632013-12-26 15:11:28 -0500657gc
658--
659
660New :func:`~gc.get_stats` returns a list of three per-generation dictionaries
661containing the collections statistics since interpreter startup. (Contributed
662by Antoine Pitrou in :issue:`16351`.)
663
664
Christian Heimese92ef132013-10-13 00:52:43 +0200665hashlib
666-------
667
668New :func:`hashlib.pbkdf2_hmac` function.
Christian Heimese92ef132013-10-13 00:52:43 +0200669(Contributed by Christian Heimes in :issue:`18582`)
670
671
Ezio Melotti250a06c2013-11-25 06:18:47 +0200672html
673----
674
675Added a new :func:`html.unescape` function that converts HTML5 character
676references to the corresponding Unicode characters.
Ezio Melotti250a06c2013-11-25 06:18:47 +0200677(Contributed by Ezio Melotti in :issue:`2927`)
678
679Added a new *convert_charrefs* keyword argument to
680:class:`~html.parser.HTMLParser` that, when ``True``, automatically converts
681all character references. For backward-compatibility, its value defaults
682to ``False``, but it will change to ``True`` in future versions, so you
683are invited to set it explicitly and update your code to use this new feature.
Ezio Melotti250a06c2013-11-25 06:18:47 +0200684(Contributed by Ezio Melotti in :issue:`13633`)
685
686The *strict* argument of :class:`~html.parser.HTMLParser` is now deprecated.
Ezio Melotti250a06c2013-11-25 06:18:47 +0200687(Contributed by Ezio Melotti in :issue:`15114`)
688
689
R David Murraya475a8d2014-01-03 13:03:00 -0500690http
691----
692
693:meth:`~http.server.BaseHTTPRequestHandler.send_error` now accepts an
694optional additional *exaplain* parameter which can be used to provide an
695extended error description, overriding the hardcoded default if there is one.
696This extended error description will be formatted using the
697:attr:`~http.server.HTTP.error_message_format` attribute and sent as the body
698of the error response. (Contributed by Karl Cow in :issue:`12921`.)
699
700
701
R David Murrayd2653812013-12-27 14:06:15 -0500702importlib
703---------
704
705The :class:`~importlib.abc.InspectLoader` ABC defines a new method,
706:meth:`~importlib.abc.InspectLoader.source_to_code` that accepts source
707data and a path and returns a code object. The default implementation
708is equivalent to ``compile(data, path, 'exec', dont_inherit=True)``.
709(Contributed by Eric Snow and Brett Cannon in :issue:`15627`.)
710
711
Antoine Pitroud6cbd342013-08-12 20:48:15 +0200712inspect
713-------
714
Nick Coghlanf94a16b2013-09-22 22:46:49 +1000715
Nick Coghlan367df122013-10-27 01:57:34 +1000716The inspect module now offers a basic :ref:`command line interface
717<inspect-module-cli>` to quickly display source code and other
718information for modules, classes and functions. (Contributed by Claudiu Popa
719and Nick Coghlan in :issue:`18626`)
Nick Coghlanf94a16b2013-09-22 22:46:49 +1000720
Antoine Pitroud6cbd342013-08-12 20:48:15 +0200721:func:`~inspect.unwrap` makes it easy to unravel wrapper function chains
722created by :func:`functools.wraps` (and any other API that sets the
Nick Coghlan367df122013-10-27 01:57:34 +1000723``__wrapped__`` attribute on a wrapper function). (Contributed by
724Daniel Urban, Aaron Iles and Nick Coghlan in :issue:`13266`)
725
726As part of the implementation of the new :mod:`enum` module, the
727:mod:`inspect` module now has substantially better support for custom
728``__dir__`` methods and dynamic class attributes provided through
729metaclasses (Contributed by Ethan Furman in :issue:`18929` and
730:issue:`19030`)
731
Antoine Pitroud6cbd342013-08-12 20:48:15 +0200732
R David Murray8f7664a2013-12-22 20:40:11 -0500733.. _whatsnew-marshal-3:
734
735marshal
736-------
737
738The default :mod:`marshal` version has been bumped to 3. The code implementing
739the new version restores the Python2 behavior of recording only one copy of
740interned strings and preserving the interning on deserialization, and extends
741this "one copy" ability to any object type (including handling recursive
742references). This reduces both the size of ``.pyc`` files and the amount of
743memory a module occupies in memory when it is loaded from a ``.pyc`` (or
744``.pyo``) file. (Contributed by Kristján Valur Jónsson in :issue:`16475`.)
745
746
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +0200747mmap
748----
749
750mmap objects can now be weakref'ed.
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +0200751(Contributed by Valerie Lambert in :issue:`4885`.)
752
753
R David Murray33ef1ae2013-12-31 16:04:50 -0500754mock
755----
756
757:mod:`~unittest.mock` objects now inspect their specification signatures when
758matching calls, which means an argument can now be matched by either position
759or name, instead of only by position. (Contributed by Antoine Pitrou in
760:issue:`17015`.)
761
R David Murray57fcf762014-01-03 23:31:54 -0500762:func:`~mock.mock_open` objects now have ``readline`` and ``readlines``
763methods. (Contributed by Toshio Kuratomi in :issue:`17467`.)
764
R David Murray33ef1ae2013-12-31 16:04:50 -0500765
Richard Oudkerk84ed9a62013-08-14 15:35:41 +0100766multiprocessing
767---------------
768
R David Murrayac186222013-12-20 17:23:57 -0500769.. _whatsnew-multiprocessing-no-fork:
770
771On Unix, two new :ref:`start methods <multiprocessing-start-methods>`
772(``spawn`` and ``forkserver``) have been added for starting processes using
773:mod:`multiprocessing`. These make the mixing of processes with threads more
774robust, and the ``spawn`` method matches the semantics that multiprocessing has
775always used on Windows. (Contributed by Richard Oudkerk in :issue:`8713`).
Richard Oudkerk84ed9a62013-08-14 15:35:41 +0100776
777Also, except when using the old *fork* start method, child processes
R David Murrayac186222013-12-20 17:23:57 -0500778will no longer inherit unneeded handles/file descriptors from their parents
779(part of :issue:`8713`).
Richard Oudkerk84ed9a62013-08-14 15:35:41 +0100780
Nick Coghlan9a767352013-12-17 22:17:26 +1000781:mod:`multiprocessing` now relies on :mod:`runpy` (which implements the
782``-m`` switch) to initialise ``__main__`` appropriately in child processes
783when using the ``spawn`` or ``forkserver`` start methods. This resolves some
784edge cases where combining multiprocessing, the ``-m`` command line switch
785and explicit relative imports could cause obscure failures in child
786processes. (Contributed by Nick Coghlan in :issue:`19946`)
787
Richard Oudkerk84ed9a62013-08-14 15:35:41 +0100788
Victor Stinnerdaf45552013-08-28 00:53:59 +0200789os
790--
791
Georg Brandlc6ebbef2013-09-16 04:03:12 +0200792New functions to get and set the :ref:`inheritable flag <fd_inheritance>` of a file
Victor Stinnerdaf45552013-08-28 00:53:59 +0200793descriptors or a Windows handle:
794
795* :func:`os.get_inheritable`, :func:`os.set_inheritable`
796* :func:`os.get_handle_inheritable`, :func:`os.set_handle_inheritable`
797
798
R David Murray78d692f2013-10-10 17:23:26 -0400799pdb
800---
801
802The ``print`` command has been removed from :mod:`pdb`, restoring access to the
803``print`` function.
804
805Rationale: Python2's ``pdb`` did not have a ``print`` command; instead,
806entering ``print`` executed the ``print`` statement. In Python3 ``print`` was
807mistakenly made an alias for the pdb :pdbcmd:`p` command. ``p``, however,
808prints the ``repr`` of its argument, not the ``str`` like the Python2 ``print``
809command did. Worse, the Python3 ``pdb print`` command shadowed the Python3
810``print`` function, making it inaccessible at the ``pdb`` prompt.
811
812(Contributed by Connor Osborn in :issue:`18764`.)
813
814
R David Murray809487e2013-12-22 20:49:40 -0500815.. _whatsnew-protocol-4:
816
R David Murray5ea95242013-12-24 15:59:57 -0500817pickle
R David Murray809487e2013-12-22 20:49:40 -0500818------
819
820protocol 4
821
822:mod:`pickle` now supports (but does not use by default) a new pickle protocol,
823protocol 4. This new protocol addresses a number of issues that were present
824in previous protocols, such as the serialization of nested classes, very large
825strings and containers, or classes whose :meth:`__new__` method takes
826keyword-only arguments. It also provides some efficiency improvements.
827
828.. seealso::
829
R David Murray061efb12013-12-24 12:35:59 -0500830 :pep:`3154` -- Pickle protocol 4
R David Murray809487e2013-12-22 20:49:40 -0500831 PEP written by Antoine Pitrou and implemented by Alexandre Vassalotti.
832
833
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +0200834poplib
835------
836
R David Murray473f45b2013-12-27 17:01:16 -0500837Two new methods have been added to :mod:`poplib`: :meth:`~poplib.POP3.capa`,
838which returns the list of capabilities advertised by the POP server, and
839:meth:`~poplib.POP3.stls`, which switches a clear-text POP3 session into an
840encrypted POP3 session if the POP server supports it. (Contributed by Lorenzo
841Catucci in :issue:`4473`.)
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +0200842
843
Serhiy Storchaka7c411a42013-10-02 11:56:18 +0300844pprint
845------
846
Christian Heimese1bfd3e2013-10-21 12:32:21 +0200847The :mod:`pprint` module now supports *compact* mode for formatting long
Serhiy Storchaka7c411a42013-10-02 11:56:18 +0300848sequences (:issue:`19132`).
849
R David Murrayb231b2b2014-01-04 17:11:23 -0500850Long strings are now wrapped using Python's normal line continuation
851syntax (Contributed by Antoine Pitrou in :issue:`17150`.)
852
Serhiy Storchaka7c411a42013-10-02 11:56:18 +0300853
R David Murray6e390152013-12-24 22:28:04 -0500854pty
855---
856
857:func:`pty.spawn` now returns the status value from :func:`os.waitpid` on
858the child process, instead of ``None``. (Contributed by Gregory P. Smith.)
859
860
Nick Coghlan367df122013-10-27 01:57:34 +1000861pydoc
862-----
863
864While significant changes have not been made to :mod:`pydoc` directly,
865its handling of custom ``__dir__`` methods and various descriptor
866behaviours has been improved substantially by the underlying changes in
867the :mod:`inspect` module.
868
869
Serhiy Storchaka32eddc12013-11-23 23:20:30 +0200870re
871--
872
873Added :func:`re.fullmatch` function and :meth:`regex.fullmatch` method,
874which anchor the pattern at both ends of the string to match.
875(Contributed by Matthew Barnett in :issue:`16203`.)
876
Ezio Melottidd7e2912013-11-25 23:20:20 +0200877The repr of :ref:`regex objects <re-objects>` now includes the pattern
878and the flags; the repr of :ref:`match objects <match-objects>` now
879includes the start, end, and the part of the string that matched.
Ezio Melottidd7e2912013-11-25 23:20:20 +0200880(Contributed by Serhiy Storchaka in :issue:`13592` and :issue:`17087`.)
881
882
Christian Heimesb7bd5df2013-10-22 11:21:54 +0200883resource
884--------
885
886New :func:`resource.prlimit` function and Linux specific constants.
887(Contributed by Christian Heimes in :issue:`16595` and :issue:`19324`.)
888
R David Murray575fb312013-12-25 23:21:03 -0500889
R David Murray2bc930f2013-12-31 11:17:21 -0500890select
891------
892
893:class:`~select.epoll` objects now support the context management protocol.
894When used in a :keyword:`with` statement, the :meth:`~select.epoll.close`
895method will be called automatically at the end of the block. (Contributed
896by Serhiy Storchaka in :issue:`16488`.)
897
898
R David Murray575fb312013-12-25 23:21:03 -0500899shelve
900------
901
902:class:`~shelve.Shelf` instances may now be used in :keyword:`with` statements,
903and will be automatically closed at the end of the :keyword:`with` block.
904(Contributed by Filip Gruszczyński in :issue:`13896`.)
905
906
R David Murray8e37d5d2013-04-13 14:49:48 -0400907smtplib
908-------
909
R David Murray8a345962013-04-14 06:46:35 -0400910:exc:`~smtplib.SMTPException` is now a subclass of :exc:`OSError`, which allows
R David Murray8e37d5d2013-04-13 14:49:48 -0400911both socket level errors and SMTP protocol level errors to be caught in one
912try/except statement by code that only cares whether or not an error occurred.
913(:issue:`2118`).
914
Antoine Pitroud6cbd342013-08-12 20:48:15 +0200915
Victor Stinnerdaf45552013-08-28 00:53:59 +0200916socket
917------
918
R David Murray33ef1ae2013-12-31 16:04:50 -0500919The socket module now supports the :data:`~socket.CAN_BCM` protocol on
920platforms that support it. (Contributed by Brian Thorne in :issue:`15359`.)
921
Georg Brandl5642ff92013-09-15 10:37:57 +0200922Socket objects have new methods to get or set their :ref:`inheritable flag
R David Murray33ef1ae2013-12-31 16:04:50 -0500923<fd_inheritance>`, :meth:`~socket.socket.get_inheritable` and
924:meth:`~socket.socket.set_inheritable`.
Victor Stinnerdaf45552013-08-28 00:53:59 +0200925
R David Murray33ef1ae2013-12-31 16:04:50 -0500926The ``socket.AF_*`` and ``socket.SOCK_*`` constants are now enumeration values
927using the new :mod:`enum` module. This allows meaningful names to be printed
928during debugging, instead of integer "magic numbers".
Victor Stinnerdaf45552013-08-28 00:53:59 +0200929
R David Murray8b2d6822013-12-31 15:06:05 -0500930
R David Murray9025f1c2014-01-02 13:44:18 -0500931sqlite3
932-------
933
934A new boolean parameter, *uri*, to the :func:`~sqlite3.connect` function can
935be used to indicate that the *database* parameter is a ``uri`` (see
936the `SQLite URI documentation <http://www.sqlite.org/uri.html>`_).
937(Contributed by poq in :issue:`13773`.)
938
939
Christian Heimes24cd4cf2013-06-22 19:31:58 +0200940ssl
941---
942
R David Murray748bad22013-12-20 17:08:39 -0500943.. _whatsnew-tls-11-12:
Christian Heimes24cd4cf2013-06-22 19:31:58 +0200944
R David Murray748bad22013-12-20 17:08:39 -0500945:data:`~ssl.PROTOCOL_TLSv1_1` and :data:`~ssl.PROTOCOL_TLSv1_2` (TLSv1.1 and
946TLSv1.2 support) have been added; support for these protocols is only available if
947Python is linked with OpenSSL 1.0.1 or later. (Contributed by Michele Orrù and
948Antoine Pitrou in :issue:`16692`)
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +0200949
R David Murray748bad22013-12-20 17:08:39 -0500950New diagnostic functions :func:`~ssl.get_default_verify_paths`,
951:meth:`~ssl.SSLContext.cert_store_stats` and
952:meth:`~ssl.SSLContext.get_ca_certs` (Contributed by Christian Heimes
953in :issue:`18143` and :issue:`18147`)
Christian Heimes24cd4cf2013-06-22 19:31:58 +0200954
R David Murray748bad22013-12-20 17:08:39 -0500955Add :func:`ssl.enum_cert_store` to retrieve certificates and CRL from Windows'
956cert store. (Contributed by Christian Heimes in :issue:`17134`.)
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +0200957
958Support for server-side SNI using the new
959:meth:`ssl.SSLContext.set_servername_callback` method.
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +0200960(Contributed by Daniel Black in :issue:`8109`.)
961
962
Antoine Pitroud6cbd342013-08-12 20:48:15 +0200963stat
964----
965
966The :mod:`stat` module is now backed by a C implementation in :mod:`_stat`. A C
967implementation is required as most of the values aren't standardized and
968platform-dependent. (Contributed by Christian Heimes in :issue:`11016`.)
969
970The module supports new file types: door, event port and whiteout.
971
972
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +0200973struct
974------
975
976Streaming struct unpacking using :func:`struct.iter_unpack`.
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +0200977(Contributed by Antoine Pitrou in :issue:`17804`.)
978
979
Serhiy Storchakae06a8962013-09-04 00:43:03 +0300980sunau
981-----
982
983The :meth:`~sunau.getparams` method now returns a namedtuple rather than a
984plain tuple. (Contributed by Claudiu Popa in :issue:`18901`.)
985
Serhiy Storchaka34d20132013-09-05 17:01:53 +0300986:meth:`sunau.open` now supports the context manager protocol (:issue:`18878`).
987
Serhiy Storchakae06a8962013-09-04 00:43:03 +0300988
R David Murrayd17aba72013-12-24 14:46:23 -0500989sys
990---
991
992New function :func:`sys.getallocatedblocks` returns the current number of
993blocks allocated by the interpreter (in CPython with the default
994``--with-pymalloc`` setting, this is allocations made through the
995:c:func:`PyObject_Malloc` API). This can be useful for tracking memory leaks,
996especially if automated via a test suite. (Contributed by Antoine Pitrou
997in :issue:`13390`.)
998
999
Andrew Kuchling173a1572013-09-15 18:15:56 -04001000traceback
1001---------
1002
1003A new :func:`traceback.clear_frames` function takes a traceback object
1004and clears the local variables in all of the frames it references,
1005reducing the amount of memory consumed (:issue:`1565525`).
1006
1007
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +02001008urllib
1009------
1010
R David Murray473f45b2013-12-27 17:01:16 -05001011:mod:`urllib.request` now supports ``data:`` URLs via the
1012:class:`~urllib.request.DataHandler` class. (Contributed by Mathias Panzenböck
1013in :issue:`16423`.)
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +02001014
1015
1016unittest
1017--------
1018
R David Murray57fcf762014-01-03 23:31:54 -05001019The :class:`~unittest.TestCase` class has a new method,
1020:meth:`~unittest.TestCase.subTest`, that produces a context manager whose
1021:keyword:`with` block becomes a "sub-test". This context manager allows a test
1022method to dynamically generate subtests by, say, calling the ``subTest``
1023context manager inside a loop. A single test method can thereby produce an
1024indefinite number of separately-identified and separately-counted tests, all of
1025which will run even if one or more of them fail. For example::
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +02001026
R David Murray57fcf762014-01-03 23:31:54 -05001027 class NumbersTest(unittest.TestCase):
1028 def test_even(self):
1029 for i in range(6):
1030 with self.subTest(i=1):
1031 self.assertEqual(i % 2, 0)
1032
1033will result in six subtests, each identified in the unittest verbose output
1034with a label consisting of the variable name ``i`` and a particular value for
1035that variable (``i=0``, ``i=1``, etc). See :ref:`subtests` for the full
1036version of this example. (Contributed by Antoine Pitrou in :issue:`16997`.)
1037
1038:func:`unittest.main` now accepts an iterable of test names for
R David Murray9025f1c2014-01-02 13:44:18 -05001039*defaultTest*, where previously it only accepted a single test name as a
1040string. (Contributed by Jyrki Pulliainen in :issue:`15132`.)
1041
R David Murray42fa1102014-01-03 13:03:36 -05001042If :class:`~unittest.SkipTest` is raised during test discovery (that is, at the
1043module level in the test file), it is now reported as a skip instead of an
1044error. (Contributed by Zach Ware in :issue:`16935`.)
1045
R David Murraydb085f52014-01-03 15:46:24 -05001046:meth:`~unittest.TestLoader.discover` now sorts the discovered files to provide
1047consistent test ordering. (Contributed by Martin Melin and Jeff Ramnani in
1048:issue:`16709`.)
1049
R David Murray8e37d5d2013-04-13 14:49:48 -04001050
R David Murray575fb312013-12-25 23:21:03 -05001051venv
1052----
1053
1054:mod:`venv` now includes activation scripts for the ``csh`` and ``fish``
1055shells (Contributed by Andrew Svetlov in :issue:`15417`.)
1056
1057
R David Murray671cd322013-04-10 12:31:43 -04001058wave
1059----
1060
1061The :meth:`~wave.getparams` method now returns a namedtuple rather than a
1062plain tuple. (Contributed by Claudiu Popa in :issue:`17487`.)
1063
R David Murrayc91d5ee2013-07-31 13:46:08 -04001064:meth:`wave.open` now supports the context manager protocol. (Contributed
1065by Claudiu Popa in :issue:`17616`.)
1066
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +02001067
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +02001068weakref
1069-------
1070
1071New :class:`~weakref.WeakMethod` class simulates weak references to bound
Nick Coghlanbe57ab82013-09-22 21:26:30 +10001072methods. (Contributed by Antoine Pitrou in :issue:`14631`.)
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +02001073
Nick Coghlanbe57ab82013-09-22 21:26:30 +10001074New :class:`~weakref.finalize` class makes it possible to register a callback
1075to be invoked when an object is garbage collected, without needing to
1076carefully manage the lifecycle of the weak reference itself. (Contributed by
1077Richard Oudkerk in :issue:`15528`)
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +02001078
1079
1080xml.etree
1081---------
1082
1083Add an event-driven parser for non-blocking applications,
Eli Benderskyb5869342013-08-30 05:51:20 -07001084:class:`~xml.etree.ElementTree.XMLPullParser`.
Eli Benderskyb5869342013-08-30 05:51:20 -07001085(Contributed by Antoine Pitrou in :issue:`17741`.)
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +02001086
R David Murray575fb312013-12-25 23:21:03 -05001087The :mod:`xml.etree.ElementTree` :func:`~xml.etree.ElementTree.tostring` and
1088:func:`~xml.etree.ElementTree.tostringlist` functions, and the
1089:class:`~xml.etree.ElementTree.ElementTree`
1090:meth:`~xml.etree.ElementTree.ElementTree.write` method, now have a
1091*short_empty_elements* :ref:`keyword-only parameter <keyword-only_parameter>`
1092providing control over whether elements with no content are written in
1093abbreviated (``<tag />``) or expanded (``<tag></tag>``) form. (Contributed by
1094Ariel Poliak and Serhiy Storchaka in :issue:`14377`.)
1095
Christian Tismer59202e52013-10-21 03:59:23 +02001096
1097zipfile.PyZipfile
1098-----------------
1099
1100Add a filter function to ignore some packages (tests for instance),
1101:meth:`~zipfile.PyZipFile.writepy`.
Christian Tismer59202e52013-10-21 03:59:23 +02001102(Contributed by Christian Tismer in :issue:`19274`.)
1103
1104
R David Murraye6082552014-01-03 16:15:45 -05001105
Nick Coghlan367df122013-10-27 01:57:34 +10001106CPython Implementation Changes
1107==============================
1108
1109
R David Murrayc16dfe12013-12-21 12:32:10 -05001110.. _whatsnew-pep-445:
Nick Coghlan367df122013-10-27 01:57:34 +10001111
R David Murraye7cf6782013-12-24 14:51:25 -05001112PEP 445: Customization of CPython Memory Allocators
Nick Coghlan367df122013-10-27 01:57:34 +10001113---------------------------------------------------
1114
1115:pep:`445` adds new C level interfaces to customize memory allocation in
1116the CPython interpreter.
1117
1118.. seealso::
1119
R David Murray061efb12013-12-24 12:35:59 -05001120 :pep:`445` -- Add new APIs to customize Python memory allocators
Nick Coghlan367df122013-10-27 01:57:34 +10001121 PEP written and implemented by Victor Stinner.
1122
1123
R David Murrayc16dfe12013-12-21 12:32:10 -05001124.. _whatsnew-pep-442:
Nick Coghlan367df122013-10-27 01:57:34 +10001125
R David Murraye7cf6782013-12-24 14:51:25 -05001126PEP 442: Safe Object Finalization
Nick Coghlan367df122013-10-27 01:57:34 +10001127---------------------------------
1128
1129:pep:`442` removes the current limitations and quirks of object finalization
1130in CPython. With it, objects with :meth:`__del__` methods, as well as
1131generators with :keyword:`finally` clauses, can be finalized when they are
1132part of a reference cycle.
1133
1134As part of this change, module globals are no longer forcibly set to
1135:const:`None` during interpreter shutdown in most cases, instead relying
R David Murrayca794612013-12-22 14:05:11 -05001136on the normal operation of the cyclic garbage collector. This avoids a
1137whole class of interpreter-shutdown-time errors, usually involving
1138``__del__`` methods, that have plagued Python since the cyclic GC
1139was first introduced.
Nick Coghlan367df122013-10-27 01:57:34 +10001140
1141.. seealso::
1142
R David Murray061efb12013-12-24 12:35:59 -05001143 :pep:`442` -- Safe object finalization
Nick Coghlan367df122013-10-27 01:57:34 +10001144 PEP written and implemented by Antoine Pitrou.
1145
1146
R David Murrayc16dfe12013-12-21 12:32:10 -05001147.. _whatsnew-pep-456:
1148
1149PEP 456: Secure and Interchangeable Hash Algorithm
1150--------------------------------------------------
1151
1152:pep:`456` follows up on earlier security fix work done on Python's hash
1153algorithm to address certain DOS attacks to which public facing APIs backed by
1154dictionary lookups may be subject. (See :issue:`14621` for the start of the
1155current round of improvements.) The PEP unifies CPython's hash code to make it
1156easier for a packager to substitute a different hash algorithm, and switches
1157Python's default implementation to a SipHash implementation on platforms that
1158have a 64 bit data type. Any performance differences in comparison with the
1159older FNV algorithm are trivial.
1160
1161The PEP adds additional fields to the :func:`sys.hash_info` struct sequence to
1162describe the hash algorithm in use by the currently executing binary. Otherwise,
1163the PEP does not alter any existing CPython APIs.
1164
1165
R David Murray6dd18302013-12-24 12:23:56 -05001166.. _whatsnew-pep-436:
1167
1168PEP 436: Argument Clinic
1169------------------------
1170
1171"Argument Clinic" (:pep:`436`) is now part of the CPython build process
1172and can be used to simplify the process of defining and maintaining
1173accurate signatures for builtins and standard library extension modules
1174implemented in C.
1175
1176.. note::
1177 The Argument Clinic PEP is not fully up to date with the state of the
1178 implementation. This has been deemed acceptable by the release manager
1179 and core development team in this case, as Argument Clinic will not
1180 be made available as a public API for third party use in Python 3.4.
1181
1182.. seealso::
1183
R David Murray061efb12013-12-24 12:35:59 -05001184 :pep:`436` -- The Argument Clinic DSL
R David Murray6dd18302013-12-24 12:23:56 -05001185 PEP written and implemented by Larry Hastings.
1186
1187
R David Murraye7cf6782013-12-24 14:51:25 -05001188Other Build and C API Changes
Nick Coghlan367df122013-10-27 01:57:34 +10001189-----------------------------
Georg Brandlb80f5112012-09-30 09:11:58 +02001190
Nick Coghlan7d270ee2013-10-17 22:35:35 +10001191* The new :c:func:`Py_SetStandardStreamEncoding` pre-initialization API
1192 allows applications embedding the CPython interpreter to reliably force
1193 a particular encoding and error handler for the standard streams
1194 (Contributed by Bastien Montagne and Nick Coghlan in :issue:`16129`)
Georg Brandlb80f5112012-09-30 09:11:58 +02001195
Nick Coghlan0acceb72013-10-20 13:22:21 +10001196* Most Python C APIs that don't mutate string arguments are now correctly
1197 marked as accepting ``const char *`` rather than ``char *`` (Contributed
1198 by Serhiy Storchaka in :issue:`1772673`).
1199
R David Murrayd91ba202013-12-24 12:13:44 -05001200* New shell version of ``python-config``; can be used even when a python
1201 interpreter is not available (for example, in cross compilation scenarios).
1202
Georg Brandlb80f5112012-09-30 09:11:58 +02001203
R David Murraye6082552014-01-03 16:15:45 -05001204
R David Murraye7cf6782013-12-24 14:51:25 -05001205Other Improvements
R David Murraybcaaecf2013-12-23 21:23:36 -05001206==================
1207
1208* Tab-completion is now enabled by default in the interactive interpreter.
1209 (Contributed by Antoine Pitrou and Éric Araujo in :issue:`5845`.)
1210
1211* Invoking the Python interpreter with ``--version`` now outputs the version to
1212 standard output instead of standard error (:issue:`18338`). Similar changes
1213 were made to :mod:`argparse` (:issue:`18920`) and other modules that have
1214 script-like invocation capabilities (:issue:`18922`).
1215
R David Murray72420ff2013-12-24 10:46:44 -05001216* The CPython Windows installer now adds ``.py`` to the :envvar:`PATHEXT`
1217 variable when extensions are registered, allowing users to run a python
1218 script at the windows command prompt by just typing its name without the
1219 ``.py`` extension. (Contributed by Paul Moore in :issue:`18569`.)
1220
R David Murrayd17aba72013-12-24 14:46:23 -05001221* A new ``make`` target `coverage-report
1222 <http://docs.python.org/devguide/coverage.html#measuring-coverage-of-c-code-with-gcov-and-lcov>`_
1223 will build python, run the test suite, and generate an HTML coverage report
1224 for the C codebase using ``gcov`` and `lcov
1225 <http://ltp.sourceforge.net/coverage/lcov.php>`_.
1226
1227* The ``-R`` option to the :ref:`python regression test suite <regrtest>` now
1228 also checks for memory allocation leaks, using
1229 :func:`sys.getallocatedblocks()`. (Contributed by Antoine Pitrou in
1230 :issue:`13390`).
1231
R David Murraybcaaecf2013-12-23 21:23:36 -05001232
R David Murraye6082552014-01-03 16:15:45 -05001233
R David Murrayd74d09a2013-12-24 15:33:02 -05001234Significant Optimizations
1235=========================
R David Murraybcaaecf2013-12-23 21:23:36 -05001236
Serhiy Storchaka30793282014-01-04 22:44:01 +02001237* The UTF-32 decoder is now 3x to 4x faster.
R David Murraybcaaecf2013-12-23 21:23:36 -05001238
1239* The cost of hash collisions for sets is now reduced. Each hash table
1240 probe now checks a series of consecutive, adjacent key/hash pairs before
1241 continuing to make random probes through the hash table. This exploits
1242 cache locality to make collision resolution less expensive.
R David Murraybcaaecf2013-12-23 21:23:36 -05001243 The collision resolution scheme can be described as a hybrid of linear
1244 probing and open addressing. The number of additional linear probes
1245 defaults to nine. This can be changed at compile-time by defining
1246 LINEAR_PROBES to be any value. Set LINEAR_PROBES=0 to turn-off
R David Murray59171722014-01-03 15:52:22 -05001247 linear probing entirely. (Contributed by Raymond Hettinger in
1248 :issue:`18771`.)
R David Murraybcaaecf2013-12-23 21:23:36 -05001249
1250* The interpreter starts about 30% faster. A couple of measures lead to the
1251 speedup. The interpreter loads fewer modules on startup, e.g. the :mod:`re`,
1252 :mod:`collections` and :mod:`locale` modules and their dependencies are no
1253 longer imported by default. The marshal module has been improved to load
R David Murray59171722014-01-03 15:52:22 -05001254 compiled Python code faster. (Contributed by Antoine Pitrou, Christian
1255 Heimes and Victor Stinner in :issue:`19219`, :issue:`19218`, :issue:`19209`,
1256 :issue:`19205` and :issue:`9548`)
R David Murraybcaaecf2013-12-23 21:23:36 -05001257
R David Murray6e390152013-12-24 22:28:04 -05001258* :class:`bz2.BZ2File` is now as fast or faster than the Python2 version for
1259 most cases. :class:`lzma.LZMAFile` has also been optimized. (Contributed by
1260 Serhiy Storchaka and Nadeem Vawda in :issue:`16034`.)
1261
R David Murray8b2d6822013-12-31 15:06:05 -05001262* :func:`random.getrandbits` is 20%-40% faster for small integers (the most
1263 common use case). (Contributed by Serhiy Storchaka in :issue:`16674`).
1264
R David Murrayb231b2b2014-01-04 17:11:23 -05001265* By taking advantage of the new storage format for strings, pickling of
1266 strings is now significantly faster. (Contributed by Victor Stinner and
1267 Antoine Pitrou in :issue:`15596`.)
1268
R David Murraybcaaecf2013-12-23 21:23:36 -05001269
R David Murraye6082552014-01-03 16:15:45 -05001270
Georg Brandlb80f5112012-09-30 09:11:58 +02001271Deprecated
1272==========
1273
R David Murrayd74d09a2013-12-24 15:33:02 -05001274This section covers various APIs and other features that have been deprecated
1275in Python 3.4, and will be removed in Python 3.5 or later. In most (but not
1276all) cases, using the deprecated APIs will produce a :exc:`DeprecationWarning`
1277when the interpreter is run with deprecation warnings enabled (for example, by
1278using ``-Wd``).
1279
Georg Brandlb80f5112012-09-30 09:11:58 +02001280
R David Murraye7cf6782013-12-24 14:51:25 -05001281Deprecated Python Modules, Functions and Methods
Georg Brandlb80f5112012-09-30 09:11:58 +02001282------------------------------------------------
1283
Terry Jan Reedy2b6c26e2013-03-21 19:36:26 -04001284* :meth:`difflib.SequenceMatcher.isbjunk` and
Andrew Kuchling0d0813a2013-06-15 13:29:09 -04001285 :meth:`difflib.SequenceMatcher.isbpopular` were removed: use ``x in sm.bjunk`` and
1286 ``x in sm.bpopular``, where *sm* is a :class:`~difflib.SequenceMatcher` object.
Georg Brandlb80f5112012-09-30 09:11:58 +02001287
Brett Cannon82b3d6a2013-06-14 22:37:11 -04001288* :func:`importlib.util.module_for_loader` is pending deprecation. Using
1289 :func:`importlib.util.module_to_load` and
1290 :meth:`importlib.abc.Loader.init_module_attrs` allows subclasses of a loader
1291 to more easily customize module loading.
1292
Brett Cannone4f41de2013-06-16 13:13:40 -04001293* The :mod:`imp` module is pending deprecation. To keep compatibility with
1294 Python 2/3 code bases, the module's removal is currently not scheduled.
1295
Brett Cannon1448ecf2013-10-04 11:38:59 -04001296* The :mod:`formatter` module is pending deprecation and is slated for removal
1297 in Python 3.6.
1298
Christian Heimes634919a2013-11-20 17:23:06 +01001299* MD5 as default digestmod for :mod:`hmac` is deprecated. Python 3.6 will
1300 require an explicit digest name or constructor as *digestmod* argument.
1301
R David Murray9025f1c2014-01-02 13:44:18 -05001302* The internal ``Netrc`` class in the :mod:`ftplib` module has been documented
1303 as deprecated in its docstring for quite some time. It now emits a
1304 :exc:`DeprecationWarning` and will be removed completely in Python 3.5.
1305
Georg Brandlb80f5112012-09-30 09:11:58 +02001306
R David Murraye7cf6782013-12-24 14:51:25 -05001307Deprecated Functions and Types in the C API
Georg Brandlb80f5112012-09-30 09:11:58 +02001308-------------------------------------------
1309
R David Murrayd74d09a2013-12-24 15:33:02 -05001310XXX: None so far
Georg Brandlb80f5112012-09-30 09:11:58 +02001311
1312
R David Murraye7cf6782013-12-24 14:51:25 -05001313Deprecated Features
Georg Brandlb80f5112012-09-30 09:11:58 +02001314-------------------
1315
R David Murray5ea95242013-12-24 15:59:57 -05001316* Running :ref:`idle` with the ``-n`` flag (no subprocess) is deprecated.
1317 However, the feature will not be removed until :issue:`18823` is resolved.
1318
Antoine Pitrou3b2f0f02013-10-25 21:39:26 +02001319* The site module adding a "site-python" directory to sys.path, if it
1320 exists, is deprecated (:issue:`19375`).
Georg Brandlb80f5112012-09-30 09:11:58 +02001321
1322
R David Murray59171722014-01-03 15:52:22 -05001323
R David Murray72420ff2013-12-24 10:46:44 -05001324Removed
1325=======
1326
R David Murray59171722014-01-03 15:52:22 -05001327
1328Operating Systems No Longer Supported
1329-------------------------------------
1330
1331Support for the following operating systems has been removed from the source
1332and build tools:
1333
1334* OS/2 (:issue:`16135`).
1335* Windows 2000 (changeset e52df05b496a).
1336* VMS (:issue:`16136`).
1337
1338
1339API and Feature Removals
1340------------------------
1341
R David Murrayd2653812013-12-27 14:06:15 -05001342The following obsolete and previously deprecated APIs and features have been
R David Murray59171722014-01-03 15:52:22 -05001343removed:
R David Murray076dead2013-12-24 14:59:50 -05001344
R David Murray72420ff2013-12-24 10:46:44 -05001345* The unmaintained ``Misc/TextMate`` and ``Misc/vim`` directories have been
R David Murrayd91ba202013-12-24 12:13:44 -05001346 removed (see the `devguide <http://docs.python.org/devguide>`_
R David Murray00569362014-01-03 13:04:25 -05001347 for suggestions on what to use instead).
R David Murray72420ff2013-12-24 10:46:44 -05001348
R David Murrayd17aba72013-12-24 14:46:23 -05001349* The ``SO`` makefile macro is removed (it was replaced by the
1350 ``SHLIB_SUFFIX`` and ``EXT_SUFFIX`` macros) (:issue:`16754`).
1351
R David Murrayd74d09a2013-12-24 15:33:02 -05001352* The ``PyThreadState.tick_counter`` field has been removed; its value has
1353 been meaningless since Python 3.2, when the "new GIL" was introduced.
1354
R David Murrayd2653812013-12-27 14:06:15 -05001355* ``PyLoader`` and ``PyPycLoader`` have been removed from :mod:`importlib`.
1356 (Contributed by Taras Lyapun in :issue:`15641`.)
1357
R David Murraya2924ca2014-01-03 14:06:01 -05001358* The *strict* argument to :class:`~http.client.HTTPConnection` and
1359 :class:`~http.client.HTTPSConnection` has been removed. HTTP 0.9-style
1360 "Simple Responses" are no longer supported.
R David Murray00569362014-01-03 13:04:25 -05001361
R David Murraydb085f52014-01-03 15:46:24 -05001362* The deprecated :mod:`urllib.request.Request` getter and setter methods
1363 ``add_data``, ``has_data``, ``get_data``, ``get_type``, ``get_host``,
1364 ``get_selector``, ``set_proxy``, ``get_origin_req_host``, and
1365 ``is_unverifiable`` have been removed (use direct attribute access instead).
1366
R David Murray72420ff2013-12-24 10:46:44 -05001367
R David Murraye6082552014-01-03 16:15:45 -05001368
Benjamin Peterson88f3b232012-10-04 12:45:10 -04001369Porting to Python 3.4
Georg Brandlb80f5112012-09-30 09:11:58 +02001370=====================
1371
1372This section lists previously described changes and other bugfixes
1373that may require changes to your code.
1374
R David Murrayd74d09a2013-12-24 15:33:02 -05001375Changes in the Python API
1376-------------------------
1377
Brett Cannon777622b2013-04-09 17:03:10 -04001378* The ABCs defined in :mod:`importlib.abc` now either raise the appropriate
1379 exception or return a default value instead of raising
1380 :exc:`NotImplementedError` blindly. This will only affect code calling
1381 :func:`super` and falling through all the way to the ABCs. For compatibility,
1382 catch both :exc:`NotImplementedError` or the appropriate exception as needed.
Brett Cannon4c14b5d2013-05-04 13:56:58 -04001383
1384* The module type now initializes the :attr:`__package__` and :attr:`__loader__`
1385 attributes to ``None`` by default. To determine if these attributes were set
1386 in a backwards-compatible fashion, use e.g.
Brett Cannon3dc48d62013-05-28 18:35:54 -04001387 ``getattr(module, '__loader__', None) is not None``.
1388
1389* :meth:`importlib.util.module_for_loader` now sets ``__loader__`` and
1390 ``__package__`` unconditionally to properly support reloading. If this is not
1391 desired then you will need to set these attributes manually. You can use
Brett Cannon028d5122013-05-31 18:02:11 -04001392 :func:`importlib.util.module_to_load` for module management.
Brett Cannon3e0651b2013-05-31 23:18:39 -04001393
1394* Import now resets relevant attributes (e.g. ``__name__``, ``__loader__``,
1395 ``__package__``, ``__file__``, ``__cached__``) unconditionally when reloading.
1396
Brett Cannon1448ecf2013-10-04 11:38:59 -04001397* Frozen packages no longer set ``__path__`` to a list containing the package
1398 name but an empty list instead. Determing if a module is a package should be
1399 done using ``hasattr(module, '__path__')``.
Brett Cannon8f5ac512013-06-12 23:29:18 -04001400
Brett Cannon33915eb2013-06-14 18:33:00 -04001401* :func:`py_compile.compile` now raises :exc:`FileExistsError` if the file path
1402 it would write to is a symlink or a non-regular file. This is to act as a
1403 warning that import will overwrite those files with a regular file regardless
1404 of what type of file path they were originally.
Brett Cannonf4375ef2013-06-16 18:05:54 -04001405
1406* :meth:`importlib.abc.SourceLoader.get_source` no longer raises
1407 :exc:`ImportError` when the source code being loaded triggers a
1408 :exc:`SyntaxError` or :exc:`UnicodeDecodeError`. As :exc:`ImportError` is
1409 meant to be raised only when source code cannot be found but it should, it was
1410 felt to be over-reaching/overloading of that meaning when the source code is
1411 found but improperly structured. If you were catching ImportError before and
1412 wish to continue to ignore syntax or decoding issues, catch all three
Victor Stinner84e33c82013-06-21 00:31:55 +02001413 exceptions now.
Nick Coghlan24c05bc2013-07-15 21:13:08 +10001414
1415* :func:`functools.update_wrapper` and :func:`functools.wraps` now correctly
Nick Coghlan367df122013-10-27 01:57:34 +10001416 set the ``__wrapped__`` attribute to the function being wrapper, even if
1417 that function also had its ``__wrapped__`` attribute set. This means
1418 ``__wrapped__`` attributes now correctly link a stack of decorated
1419 functions rather than every ``__wrapped__`` attribute in the chain
1420 referring to the innermost function. Introspection libraries that
1421 assumed the previous behaviour was intentional can use
1422 :func:`inspect.unwrap` to access the first function in the chain that has
1423 no ``__wrapped__`` attribute.
Victor Stinner2fe9bac2013-10-10 16:18:20 +02001424
Georg Brandl0f5bff22013-10-19 17:46:38 +02001425* :class:`importlib.machinery.PathFinder` now passes on the current working
Brett Cannon27e27f72013-10-18 11:39:04 -04001426 directory to objects in :data:`sys.path_hooks` for the empty string. This
1427 results in :data:`sys.path_importer_cache` never containing ``''``, thus
1428 iterating through :data:`sys.path_importer_cache` based on :data:`sys.path`
1429 will not find all keys. A module's ``__file__`` when imported in the current
1430 working directory will also now have an absolute path, including when using
1431 ``-m`` with the interpreter (this does not influence when the path to a file
1432 is specified on the command-line).
Victor Stinner2748bc72013-12-13 10:57:04 +01001433
R David Murraya2924ca2014-01-03 14:06:01 -05001434* The removal of the *strict* argument to :class:`~http.client.HTTPConnection`
1435 and :class:`~http.client.HTTPSConnection` changes the meaning of the
1436 remaining arguments if you are specifying them positionally rather than by
1437 keyword. If you've been paying attention to deprecation warnings your code
1438 should already be specifying any additional arguments via keywords.
1439
R David Murrayd2653812013-12-27 14:06:15 -05001440
Victor Stinner774b2e02013-12-13 14:33:01 +01001441Changes in the C API
1442--------------------
1443
1444* :c:func:`PyErr_SetImportError` now sets :exc:`TypeError` when its **msg**
1445 argument is not set. Previously only ``NULL`` was returned with no exception
1446 set.
1447
1448* The result of the :c:data:`PyOS_ReadlineFunctionPointer` callback must
1449 now be a string allocated by :c:func:`PyMem_RawMalloc` or
1450 :c:func:`PyMem_RawRealloc`, or *NULL* if an error occurred, instead of a
1451 string allocated by :c:func:`PyMem_Malloc` or :c:func:`PyMem_Realloc`.
1452
1453* :c:func:`PyThread_set_key_value` now always set the value. In Python
Victor Stinner2748bc72013-12-13 10:57:04 +01001454 3.3, the function did nothing if the key already exists (if the current
1455 value is a non-NULL pointer).
1456
Victor Stinner774b2e02013-12-13 14:33:01 +01001457* The ``f_tstate`` (thread state) field of the :c:type:`PyFrameObject`
1458 structure has been removed to fix a bug: see :issue:`14432` for the
1459 rationale.
1460