blob: 73225bc43fe5b9521c3a44e8f1b14bbd6c1cc05b [file] [log] [blame]
Georg Brandlb80f5112012-09-30 09:11:58 +02001****************************
2 What's New In Python 3.4
3****************************
4
Georg Brandl3b80d342012-10-28 13:37:54 +01005.. :Author: Someone <email>
6 (uncomment if there is a principal author)
Georg Brandlb80f5112012-09-30 09:11:58 +02007
8.. Rules for maintenance:
9
Nick Coghlan03074fd2012-09-30 18:51:53 +053010 * Anyone can add text to this document, but the maintainer reserves the
11 right to rewrite any additions. In particular, for obscure or esoteric
12 features, the maintainer may reduce any addition to a simple reference to
13 the new documentation rather than explaining the feature inline.
Georg Brandlb80f5112012-09-30 09:11:58 +020014
Nick Coghlan03074fd2012-09-30 18:51:53 +053015 * While the maintainer will periodically go through Misc/NEWS
16 and add changes, it's best not to rely on this. We know from experience
17 that any changes that aren't in the What's New documentation around the
18 time of the original release will remain largely unknown to the community
19 for years, even if they're added later. We also know from experience that
20 other priorities can arise, and the maintainer will run out of time to do
21 updates - in such cases, end users will be much better served by partial
22 notifications that at least give a hint about new features to
23 investigate.
Georg Brandlb80f5112012-09-30 09:11:58 +020024
25 * This is not a complete list of every single change; completeness
Nick Coghlan03074fd2012-09-30 18:51:53 +053026 is the purpose of Misc/NEWS. The What's New should focus on changes that
27 are visible to Python *users* and that *require* a feature release (i.e.
28 most bug fixes should only be recorded in Misc/NEWS)
29
30 * PEPs should not be marked Final until they have an entry in What's New.
31 A placeholder entry that is just a section header and a link to the PEP
32 (e.g ":pep:`397` has been implemented") is acceptable. If a PEP has been
33 implemented and noted in What's New, don't forget to mark it as Final!
Georg Brandlb80f5112012-09-30 09:11:58 +020034
35 * If you want to draw your new text to the attention of the
36 maintainer, add 'XXX' to the beginning of the paragraph or
37 section.
38
Nick Coghlan03074fd2012-09-30 18:51:53 +053039 * It's OK to add just a very brief note about a change. For
40 example: "The :ref:`~socket.transmogrify()` function was added to the
41 :mod:`socket` module." The maintainer will research the change and
42 write the necessary text (if appropriate). The advantage of doing this
43 is that even if no more descriptive text is ever added, readers will at
44 least have a notification that the new feature exists and a link to the
45 relevant documentation.
Georg Brandlb80f5112012-09-30 09:11:58 +020046
47 * You can comment out your additions if you like, but it's not
48 necessary (especially when a final release is some months away).
49
50 * Credit the author of a patch or bugfix. Just the name is
51 sufficient; the e-mail address isn't necessary.
52
53 * It's helpful to add the bug/patch number as a comment:
54
Nick Coghlan03074fd2012-09-30 18:51:53 +053055 The :ref:`~socket.transmogrify()` function was added to the
56 :mod:`socket` module. (Contributed by P.Y. Developer in :issue:`12345`.)
Georg Brandlb80f5112012-09-30 09:11:58 +020057
58 This saves the maintainer the effort of going through the Mercurial log
59 when researching a change.
60
Nick Coghlan03074fd2012-09-30 18:51:53 +053061 * Cross referencing tip: :ref:`mod.attr` will display as ``mod.attr``,
62 while :ref:`~mod.attr` will display as ``attr``.
63
Georg Brandlb80f5112012-09-30 09:11:58 +020064This article explains the new features in Python 3.4, compared to 3.3.
65
Nick Coghlan03074fd2012-09-30 18:51:53 +053066.. Python 3.4 was released on TBD.
Georg Brandlb80f5112012-09-30 09:11:58 +020067
Ezio Melotti25bbe5e2012-11-17 19:30:48 +020068For full details, see the
69`changelog <http://docs.python.org/3.4/whatsnew/changelog.html>`_.
Georg Brandlb80f5112012-09-30 09:11:58 +020070
71.. note:: Prerelease users should be aware that this document is currently in
72 draft form. It will be updated substantially as Python 3.4 moves towards
73 release, so it's worth checking back even after reading earlier versions.
74
75
76.. seealso::
77
Nick Coghlan367df122013-10-27 01:57:34 +100078 :pep:`429` - Python 3.4 Release Schedule
Georg Brandlb80f5112012-09-30 09:11:58 +020079
80
81Summary -- Release highlights
82=============================
83
Kristjan Valur Jonssona1e82442013-03-26 13:56:14 +000084.. This section singles out the most important changes in Python 3.4.
Georg Brandlb80f5112012-09-30 09:11:58 +020085 Brevity is key.
86
87New syntax features:
88
Nick Coghlan0acceb72013-10-20 13:22:21 +100089* No new syntax features are planned for Python 3.4.
Georg Brandlb80f5112012-09-30 09:11:58 +020090
91New library modules:
92
Nick Coghlan0acceb72013-10-20 13:22:21 +100093* :mod:`asyncio`: New provisonal API for asynchronous IO (:pep:`3156`).
94* :mod:`enum`: Support for enumeration types (:pep:`435`).
Victor Stinner4aea4a02013-09-04 20:30:34 +020095* :mod:`selectors`: High-level and efficient I/O multiplexing, built upon the
96 :mod:`select` module primitives.
Nick Coghlan0acceb72013-10-20 13:22:21 +100097* :mod:`statistics`: A basic numerically stable statistics library (:pep:`450`).
Georg Brandlb80f5112012-09-30 09:11:58 +020098
Nick Coghlan367df122013-10-27 01:57:34 +100099New expected features for Python implementations:
Georg Brandlb80f5112012-09-30 09:11:58 +0200100
Victor Stinnerdaf45552013-08-28 00:53:59 +0200101* :ref:`PEP 446: Make newly created file descriptors non-inheritable <pep-446>`.
Nick Coghlan367df122013-10-27 01:57:34 +1000102* command line option for :ref:`isolated mode <using-on-misc-options>`,
103 (:issue:`16499`).
Georg Brandlb80f5112012-09-30 09:11:58 +0200104
105Significantly Improved Library Modules:
106
Nick Coghlan367df122013-10-27 01:57:34 +1000107* Single-dispatch generic functions in :mod:`functoools` (:pep:`443`)
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +0200108* SHA-3 (Keccak) support for :mod:`hashlib`.
109* TLSv1.1 and TLSv1.2 support for :mod:`ssl`.
Richard Oudkerk84ed9a62013-08-14 15:35:41 +0100110* :mod:`multiprocessing` now has option to avoid using :func:`os.fork`
111 on Unix (:issue:`8713`).
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +0200112
Nick Coghlan367df122013-10-27 01:57:34 +1000113CPython implementation improvements:
Georg Brandlb80f5112012-09-30 09:11:58 +0200114
Nick Coghlan367df122013-10-27 01:57:34 +1000115* :ref:`PEP 442: Safe object finalization <pep-442>`
116* :ref:`PEP 445: Configurable memory allocators <pep-445>`
117* Improve finalization of Python modules to avoid setting their globals
118 to None, in most cases (:issue:`18214`).
119* A more efficient :mod:`marshal` format (:issue:`16475`).
120* "Argument Clinic", an initial step towards providing improved introspection
121 support for builtin and standard library extension types implemented in C
122 (:pep:`436`)
Georg Brandlb80f5112012-09-30 09:11:58 +0200123
124Please read on for a comprehensive list of user-facing changes.
125
Victor Stinnerdaf45552013-08-28 00:53:59 +0200126.. _pep-446:
127
128PEP 446: Make newly created file descriptors non-inheritable
129============================================================
130
Nick Coghlan367df122013-10-27 01:57:34 +1000131:pep:`446` makes newly created file descriptors :ref:`non-inheritable
Georg Brandl5642ff92013-09-15 10:37:57 +0200132<fd_inheritance>`. New functions and methods:
Victor Stinnerdaf45552013-08-28 00:53:59 +0200133
134* :func:`os.get_inheritable`, :func:`os.set_inheritable`
135* :func:`os.get_handle_inheritable`, :func:`os.set_handle_inheritable`
136* :meth:`socket.socket.get_inheritable`, :meth:`socket.socket.set_inheritable`
137
Antoine Pitrou796564c2013-07-30 19:59:21 +0200138.. seealso::
139
Nick Coghlan367df122013-10-27 01:57:34 +1000140 :pep:`446` - Make newly created file descriptors non-inheritable
141 PEP written and implemented by Victor Stinner.
Antoine Pitrou796564c2013-07-30 19:59:21 +0200142
Georg Brandlb80f5112012-09-30 09:11:58 +0200143
144Other Language Changes
145======================
146
147Some smaller changes made to the core Python language are:
148
Ezio Melotti34808e22013-10-12 16:36:13 +0300149* Unicode database updated to UCD version 6.3.
Georg Brandlb80f5112012-09-30 09:11:58 +0200150
R David Murray9a2f1392013-06-28 13:31:19 -0400151* :func:`min` and :func:`max` now accept a *default* argument that can be used
152 to specify the value they return if the iterable they are evaluating has no
153 elements. Contributed by Julian Berman in :issue:`18111`.
Georg Brandlb80f5112012-09-30 09:11:58 +0200154
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +0200155* Module objects are now :mod:`weakref`'able.
156
Nick Coghlan0acceb72013-10-20 13:22:21 +1000157* Module ``__file__`` attributes (and related values) should now always
158 contain absolute paths by default, with the sole exception of
159 ``__main__.__file__`` when a script has been executed directly using
160 a relative path (Contributed by Brett Cannon in :issue:`18416`).
161
Georg Brandlb80f5112012-09-30 09:11:58 +0200162
163New Modules
164===========
165
Nick Coghlan0acceb72013-10-20 13:22:21 +1000166
167asyncio
168-------
169
170The new :mod:`asyncio` module (defined in :pep:`3156`) provides a standard
171pluggable event loop model for Python, providing solid asynchronous IO
172support in the standard library, and making it easier for other event loop
173implementations to interoperate with the standard library and each other.
174
175For Python 3.4, this module is considered a :term:`provisional API`.
176
Nick Coghlan367df122013-10-27 01:57:34 +1000177.. seealso::
178
179 :pep:`3156` - Asynchronous IO Support Rebooted: the "asyncio" Module
180 PEP written and implementation led by Guido van Rossum.
181
Nick Coghlan0acceb72013-10-20 13:22:21 +1000182enum
183----
184
Nick Coghlan367df122013-10-27 01:57:34 +1000185The new :mod:`enum` module (defined in :pep:`435`) provides a standard
186implementation of enumeration types, allowing other modules (such as
187:mod:`socket`) to provide more informative error messages and better
188debugging support by replacing opaque integer constants with backwards
189compatible enumeration values.
190
191.. seealso::
192
193 :pep:`435` - Adding an Enum type to the Python standard library
194 PEP written by Barry Warsaw, Eli Bendersky and Ethan Furman,
195 implemented by Ethan Furman.
Nick Coghlan0acceb72013-10-20 13:22:21 +1000196
197
Charles-François Natali243d8d82013-09-04 19:02:49 +0200198selectors
199---------
Georg Brandlb80f5112012-09-30 09:11:58 +0200200
Nick Coghlan0acceb72013-10-20 13:22:21 +1000201The new :mod:`selectors` module (created as part of implementing :pep:`3156`)
202allows high-level and efficient I/O multiplexing, built upon the
203:mod:`select` module primitives.
204
205
206statistics
207----------
208
209The new :mod:`statistics` module (defined in :pep:`450`) offers some core
210statistics functionality directly in the standard library. This module
211supports calculation of the mean, median, mode, variance and standard
212deviation of a data series.
Georg Brandlb80f5112012-09-30 09:11:58 +0200213
Nick Coghlan367df122013-10-27 01:57:34 +1000214.. seealso::
215
216 :pep:`450` - Adding A Statistics Module To The Standard Library
217 PEP written and implemented by Steven D'Aprano
218
Georg Brandlb80f5112012-09-30 09:11:58 +0200219
220Improved Modules
221================
222
Antoine Pitroud6cbd342013-08-12 20:48:15 +0200223aifc
224----
225
226The :meth:`~aifc.getparams` method now returns a namedtuple rather than a
227plain tuple. (Contributed by Claudiu Popa in :issue:`17818`.)
228
229
Serhiy Storchakaeaea5e92013-10-19 21:10:46 +0300230audioop
231-------
232
233Added support for 24-bit samples (:issue:`12866`).
234
235
Nick Coghlane206b6e2013-10-14 00:55:46 +1000236codecs
237------
238
239The :meth:`codecs.encode` and :meth:`codecs.decode` convenience functions are
240now properly documented. These functions have existed in the :mod:`codecs`
241module since ~2004, but were previously only discoverable through runtime
242introspection.
243
244Unlike the convenience methods on :class:`str`, :class:`bytes` and
245:class:`bytearray`, these convenience functions support arbitrary codecs,
246rather than being limited to Unicode text encodings.
247
248
Antoine Pitroud6cbd342013-08-12 20:48:15 +0200249colorsys
250--------
251
252The number of digits in the coefficients for the RGB --- YIQ conversions have
253been expanded so that they match the FCC NTSC versions. The change in
254results should be less than 1% and may better match results found elsewhere.
255
R David Murray8e37d5d2013-04-13 14:49:48 -0400256
Nick Coghlanb4534ae2013-10-13 23:23:08 +1000257contextlib
258----------
259
Nick Coghlan240f86d2013-10-17 23:40:57 +1000260The new :class:`contextlib.suppress` context manager helps to clarify the
261intent of code that deliberately suppresses exceptions from a single
262statement. (Contributed by Raymond Hettinger in :issue:`15806` and
263Zero Piraeus in :issue:`19266`)
264
Victor Stinner6633c392013-10-21 13:27:11 +0200265The new :func:`contextlib.redirect_stdout` context manager makes it easier
Nick Coghlanb4534ae2013-10-13 23:23:08 +1000266for utility scripts to handle inflexible APIs that don't provide any
267options to retrieve their output as a string or direct it to somewhere
Nick Coghlan0acceb72013-10-20 13:22:21 +1000268other than :data:`sys.stdout`. In conjunction with :class:`io.StringIO`,
269this context manager is also useful for checking expected output from
270command line utilities. (Contribute by Raymond Hettinger in :issue:`15805`)
271
272The :mod:`contextlib` documentation has also been updated to include a
273:ref:`discussion <single-use-reusable-and-reentrant-cms>` of the
274differences between single use, reusable and reentrant context managers.
Nick Coghlanb4534ae2013-10-13 23:23:08 +1000275
276
Nick Coghlanb39fd0c2013-05-06 23:59:20 +1000277dis
278---
279
Serhiy Storchaka98b28fd2013-10-13 23:12:09 +0300280The :mod:`dis` module is now built around an :class:`~dis.Instruction` class
281that provides details of individual bytecode operations and a
282:func:`~dis.get_instructions` iterator that emits the Instruction stream for a
Nick Coghlanb39fd0c2013-05-06 23:59:20 +1000283given piece of Python code. The various display tools in the :mod:`dis`
284module have been updated to be based on these new components.
285
286The new :class:`dis.Bytecode` class provides an object-oriented API for
287inspecting bytecode, both in human-readable form and for iterating over
288instructions.
289
290(Contributed by Nick Coghlan, Ryan Kelly and Thomas Kluyver in :issue:`11816`)
291
Antoine Pitroud6cbd342013-08-12 20:48:15 +0200292
R David Murray5a9d7062012-11-21 15:09:21 -0500293doctest
294-------
295
R David Murray5707d502013-06-23 14:24:13 -0400296Added :data:`~doctest.FAIL_FAST` flag to halt test running as soon as the first
297failure is detected. (Contributed by R. David Murray and Daniel Urban in
298:issue:`16522`.)
299
300Updated the doctest command line interface to use :mod:`argparse`, and added
301``-o`` and ``-f`` options to the interface. ``-o`` allows doctest options to
302be specified on the command line, and ``-f`` is a shorthand for ``-o
303FAIL_FAST`` (to parallel the similar option supported by the :mod:`unittest`
304CLI). (Contributed by R. David Murray in :issue:`11390`.)
Georg Brandlb80f5112012-09-30 09:11:58 +0200305
R David Murray8e37d5d2013-04-13 14:49:48 -0400306
R David Murraybb17d2b2013-08-09 16:15:28 -0400307email
308-----
309
310:meth:`~email.message.Message.as_string` now accepts a *policy* argument to
311override the default policy of the message when generating a string
312representation of it. This means that ``as_string`` can now be used in more
313circumstances, instead of having to create and use a :mod:`~email.generator` in
314order to pass formatting parameters to its ``flatten`` method.
315
316New method :meth:`~email.message.Message.as_bytes` added to produce a bytes
317representation of the message in a fashion similar to how ``as_string``
318produces a string representation. It does not accept the *maxheaderlen*
319argument, but does accept the *unixfrom* and *policy* arguments. The
320:class:`~email.message.Message` :meth:`~email.message.Message.__bytes__` method
321calls it, meaning that ``bytes(mymsg)`` will now produce the intuitive
322result: a bytes object containing the fully formatted message.
323
324(Contributed by R. David Murray in :issue:`18600`.)
325
R David Murray3da240f2013-10-16 22:48:40 -0400326A pair of new subclasses of :class:`~email.message.Message` have been added,
327along with a new sub-module, :mod:`~email.contentmanager`. All documentation
328is currently in the new module, which is being added as part of the new
Nick Coghlan240f86d2013-10-17 23:40:57 +1000329:term:`provisional <provisional package>` email API. These classes provide a
R David Murray3da240f2013-10-16 22:48:40 -0400330number of new methods that make extracting content from and inserting content
331into email messages much easier. See the :mod:`~email.contentmanager`
332documentation for details.
333
334These API additions complete the bulk of the work that was planned as part of
335the email6 project. The currently provisional API is scheduled to become final
336in Python 3.5 (possibly with a few minor additions in the area of error
337handling).
338
339(Contributed by R. David Murray in :issue:`18891`.)
340
R David Murraybb17d2b2013-08-09 16:15:28 -0400341
Victor Stinner854ffcb2013-06-21 00:36:30 +0200342functools
343---------
344
345New :func:`functools.singledispatch` decorator: see the :pep:`443`.
346
Nick Coghlane8c45d62013-07-28 20:00:01 +1000347
Christian Heimese92ef132013-10-13 00:52:43 +0200348hashlib
349-------
350
351New :func:`hashlib.pbkdf2_hmac` function.
352
353(Contributed by Christian Heimes in :issue:`18582`)
354
355
Antoine Pitroud6cbd342013-08-12 20:48:15 +0200356inspect
357-------
358
Nick Coghlanf94a16b2013-09-22 22:46:49 +1000359
Nick Coghlan367df122013-10-27 01:57:34 +1000360The inspect module now offers a basic :ref:`command line interface
361<inspect-module-cli>` to quickly display source code and other
362information for modules, classes and functions. (Contributed by Claudiu Popa
363and Nick Coghlan in :issue:`18626`)
Nick Coghlanf94a16b2013-09-22 22:46:49 +1000364
Antoine Pitroud6cbd342013-08-12 20:48:15 +0200365:func:`~inspect.unwrap` makes it easy to unravel wrapper function chains
366created by :func:`functools.wraps` (and any other API that sets the
Nick Coghlan367df122013-10-27 01:57:34 +1000367``__wrapped__`` attribute on a wrapper function). (Contributed by
368Daniel Urban, Aaron Iles and Nick Coghlan in :issue:`13266`)
369
370As part of the implementation of the new :mod:`enum` module, the
371:mod:`inspect` module now has substantially better support for custom
372``__dir__`` methods and dynamic class attributes provided through
373metaclasses (Contributed by Ethan Furman in :issue:`18929` and
374:issue:`19030`)
375
Antoine Pitroud6cbd342013-08-12 20:48:15 +0200376
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +0200377mmap
378----
379
380mmap objects can now be weakref'ed.
381
382(Contributed by Valerie Lambert in :issue:`4885`.)
383
384
Richard Oudkerk84ed9a62013-08-14 15:35:41 +0100385multiprocessing
386---------------
387
388On Unix two new *start methods* have been added for starting processes
389using :mod:`multiprocessing`. These make the mixing of processes with
390threads more robust. See :issue:`8713`.
391
392Also, except when using the old *fork* start method, child processes
393will no longer inherit unneeded handles/file descriptors from their parents.
394
395
Victor Stinnerdaf45552013-08-28 00:53:59 +0200396os
397--
398
Georg Brandlc6ebbef2013-09-16 04:03:12 +0200399New functions to get and set the :ref:`inheritable flag <fd_inheritance>` of a file
Victor Stinnerdaf45552013-08-28 00:53:59 +0200400descriptors or a Windows handle:
401
402* :func:`os.get_inheritable`, :func:`os.set_inheritable`
403* :func:`os.get_handle_inheritable`, :func:`os.set_handle_inheritable`
404
405
R David Murray78d692f2013-10-10 17:23:26 -0400406pdb
407---
408
409The ``print`` command has been removed from :mod:`pdb`, restoring access to the
410``print`` function.
411
412Rationale: Python2's ``pdb`` did not have a ``print`` command; instead,
413entering ``print`` executed the ``print`` statement. In Python3 ``print`` was
414mistakenly made an alias for the pdb :pdbcmd:`p` command. ``p``, however,
415prints the ``repr`` of its argument, not the ``str`` like the Python2 ``print``
416command did. Worse, the Python3 ``pdb print`` command shadowed the Python3
417``print`` function, making it inaccessible at the ``pdb`` prompt.
418
419(Contributed by Connor Osborn in :issue:`18764`.)
420
421
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +0200422poplib
423------
424
425New :meth:`~poplib.POP3.stls` method to switch a clear-text POP3 session into
426an encrypted POP3 session.
427
428New :meth:`~poplib.POP3.capa` method to query the capabilities advertised by the
429POP3 server.
430
431(Contributed by Lorenzo Catucci in :issue:`4473`.)
432
433
Serhiy Storchaka7c411a42013-10-02 11:56:18 +0300434pprint
435------
436
Christian Heimese1bfd3e2013-10-21 12:32:21 +0200437The :mod:`pprint` module now supports *compact* mode for formatting long
Serhiy Storchaka7c411a42013-10-02 11:56:18 +0300438sequences (:issue:`19132`).
439
440
Nick Coghlan367df122013-10-27 01:57:34 +1000441pydoc
442-----
443
444While significant changes have not been made to :mod:`pydoc` directly,
445its handling of custom ``__dir__`` methods and various descriptor
446behaviours has been improved substantially by the underlying changes in
447the :mod:`inspect` module.
448
449
Christian Heimesb7bd5df2013-10-22 11:21:54 +0200450resource
451--------
452
453New :func:`resource.prlimit` function and Linux specific constants.
454(Contributed by Christian Heimes in :issue:`16595` and :issue:`19324`.)
455
R David Murray8e37d5d2013-04-13 14:49:48 -0400456smtplib
457-------
458
R David Murray8a345962013-04-14 06:46:35 -0400459:exc:`~smtplib.SMTPException` is now a subclass of :exc:`OSError`, which allows
R David Murray8e37d5d2013-04-13 14:49:48 -0400460both socket level errors and SMTP protocol level errors to be caught in one
461try/except statement by code that only cares whether or not an error occurred.
462(:issue:`2118`).
463
Antoine Pitroud6cbd342013-08-12 20:48:15 +0200464
Victor Stinnerdaf45552013-08-28 00:53:59 +0200465socket
466------
467
Georg Brandl5642ff92013-09-15 10:37:57 +0200468Socket objects have new methods to get or set their :ref:`inheritable flag
469<fd_inheritance>`:
Victor Stinnerdaf45552013-08-28 00:53:59 +0200470
471* :meth:`socket.socket.get_inheritable`, :meth:`socket.socket.set_inheritable`
472
Eli Bendersky34567ec2013-08-31 15:18:48 -0700473The ``socket.AF_*`` and ``socket.SOCK_*`` constants are enumeration values,
474using the new :mod:`enum` module. This allows descriptive reporting during
475debugging, instead of seeing integer "magic numbers".
Victor Stinnerdaf45552013-08-28 00:53:59 +0200476
Christian Heimes24cd4cf2013-06-22 19:31:58 +0200477ssl
478---
479
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +0200480TLSv1.1 and TLSv1.2 support.
Christian Heimes70833a82013-06-22 19:34:17 +0200481
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +0200482(Contributed by Michele Orrù and Antoine Pitrou in :issue:`16692`)
Christian Heimes24cd4cf2013-06-22 19:31:58 +0200483
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +0200484* New diagnostic functions :func:`~ssl.get_default_verify_paths`,
485 :meth:`~ssl.SSLContext.cert_store_stats` and
486 :meth:`~ssl.SSLContext.get_ca_certs`
487
488* Add :func:`ssl.enum_cert_store` to retrieve certificates and CRL from Windows'
489 cert store.
Christian Heimes24cd4cf2013-06-22 19:31:58 +0200490
491(Contributed by Christian Heimes in :issue:`18143`, :issue:`18147` and
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +0200492 :issue:`17134`.)
493
494Support for server-side SNI using the new
495:meth:`ssl.SSLContext.set_servername_callback` method.
496
497(Contributed by Daniel Black in :issue:`8109`.)
498
499
Antoine Pitroud6cbd342013-08-12 20:48:15 +0200500stat
501----
502
503The :mod:`stat` module is now backed by a C implementation in :mod:`_stat`. A C
504implementation is required as most of the values aren't standardized and
505platform-dependent. (Contributed by Christian Heimes in :issue:`11016`.)
506
507The module supports new file types: door, event port and whiteout.
508
509
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +0200510struct
511------
512
513Streaming struct unpacking using :func:`struct.iter_unpack`.
514
515(Contributed by Antoine Pitrou in :issue:`17804`.)
516
517
Serhiy Storchakae06a8962013-09-04 00:43:03 +0300518sunau
519-----
520
521The :meth:`~sunau.getparams` method now returns a namedtuple rather than a
522plain tuple. (Contributed by Claudiu Popa in :issue:`18901`.)
523
Serhiy Storchaka34d20132013-09-05 17:01:53 +0300524:meth:`sunau.open` now supports the context manager protocol (:issue:`18878`).
525
Serhiy Storchakae06a8962013-09-04 00:43:03 +0300526
Andrew Kuchling173a1572013-09-15 18:15:56 -0400527traceback
528---------
529
530A new :func:`traceback.clear_frames` function takes a traceback object
531and clears the local variables in all of the frames it references,
532reducing the amount of memory consumed (:issue:`1565525`).
533
534
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +0200535urllib
536------
537
538Add support.for ``data:`` URLs in :mod:`urllib.request`.
539
540(Contributed by Mathias Panzenböck in :issue:`16423`.)
541
542
543unittest
544--------
545
546Support for easy dynamically-generated subtests using the
547:meth:`~unittest.TestCase.subTest` context manager.
548
549(Contributed by Antoine Pitrou in :issue:`16997`.)
550
R David Murray8e37d5d2013-04-13 14:49:48 -0400551
R David Murray671cd322013-04-10 12:31:43 -0400552wave
553----
554
555The :meth:`~wave.getparams` method now returns a namedtuple rather than a
556plain tuple. (Contributed by Claudiu Popa in :issue:`17487`.)
557
R David Murrayc91d5ee2013-07-31 13:46:08 -0400558:meth:`wave.open` now supports the context manager protocol. (Contributed
559by Claudiu Popa in :issue:`17616`.)
560
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +0200561
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +0200562weakref
563-------
564
565New :class:`~weakref.WeakMethod` class simulates weak references to bound
Nick Coghlanbe57ab82013-09-22 21:26:30 +1000566methods. (Contributed by Antoine Pitrou in :issue:`14631`.)
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +0200567
Nick Coghlanbe57ab82013-09-22 21:26:30 +1000568New :class:`~weakref.finalize` class makes it possible to register a callback
569to be invoked when an object is garbage collected, without needing to
570carefully manage the lifecycle of the weak reference itself. (Contributed by
571Richard Oudkerk in :issue:`15528`)
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +0200572
573
574xml.etree
575---------
576
577Add an event-driven parser for non-blocking applications,
Eli Benderskyb5869342013-08-30 05:51:20 -0700578:class:`~xml.etree.ElementTree.XMLPullParser`.
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +0200579
Eli Benderskyb5869342013-08-30 05:51:20 -0700580(Contributed by Antoine Pitrou in :issue:`17741`.)
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +0200581
Christian Tismer59202e52013-10-21 03:59:23 +0200582
583zipfile.PyZipfile
584-----------------
585
586Add a filter function to ignore some packages (tests for instance),
587:meth:`~zipfile.PyZipFile.writepy`.
588
589(Contributed by Christian Tismer in :issue:`19274`.)
590
591
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +0200592Other improvements
593==================
594
595Tab-completion is now enabled by default in the interactive interpreter.
596
597(Contributed by Antoine Pitrou and Éric Araujo in :issue:`5845`.)
598
Eli Bendersky96d848a2013-09-06 06:55:58 -0700599Python invocation changes
600=========================
601
602Invoking the Python interpreter with ``--version`` now outputs the version to
603standard output instead of standard error (:issue:`18338`). Similar changes
604were made to :mod:`argparse` (:issue:`18920`) and other modules that have
605script-like invocation capabilities (:issue:`18922`).
Antoine Pitrouf89aa9a2013-08-12 20:46:47 +0200606
Georg Brandlb80f5112012-09-30 09:11:58 +0200607Optimizations
608=============
609
610Major performance enhancements have been added:
611
Victor Stinnere64322e2012-10-30 23:12:47 +0100612* The UTF-32 decoder is now 3x to 4x faster.
Georg Brandlb80f5112012-09-30 09:11:58 +0200613
Raymond Hettingerc301b552013-08-19 09:12:20 -0700614* The cost of hash collisions for sets is now reduced. Each hash table
Raymond Hettinger8408dc52013-09-15 14:57:15 -0700615 probe now checks a series of consecutive, adjacent key/hash pairs before
616 continuing to make random probes through the hash table. This exploits
617 cache locality to make collision resolution less expensive.
618
619 The collision resolution scheme can be described as a hybrid of linear
620 probing and open addressing. The number of additional linear probes
621 defaults to nine. This can be changed at compile-time by defining
622 LINEAR_PROBES to be any value. Set LINEAR_PROBES=0 to turn-off
623 linear probing entirely.
Raymond Hettingerc301b552013-08-19 09:12:20 -0700624
Christian Heimes086b1af2013-10-22 11:49:34 +0200625 (Contributed by Raymond Hettinger in :issue:`18771`.)
Raymond Hettingerc301b552013-08-19 09:12:20 -0700626
Christian Heimes17ecd1d2013-10-13 03:10:06 +0200627* The interpreter starts about 30% faster. A couple of measures lead to the
Eric V. Smith57841dd2013-10-13 00:36:08 -0400628 speedup. The interpreter loads fewer modules on startup, e.g. the :mod:`re`,
Christian Heimes17ecd1d2013-10-13 03:10:06 +0200629 :mod:`collections` and :mod:`locale` modules and their dependencies are no
630 longer imported by default. The marshal module has been improved to load
631 compiled Python code faster.
632
633 (Contributed by Antoine Pitrou, Christian Heimes and Victor Stinner in
634 :issue:`19219`, :issue:`19218`, :issue:`19209`, :issue:`19205` and
635 :issue:`9548`)
636
637
Nick Coghlan367df122013-10-27 01:57:34 +1000638CPython Implementation Changes
639==============================
640
641
642.. _pep-445:
643
644PEP 445: Customization of CPython memory allocators
645---------------------------------------------------
646
647:pep:`445` adds new C level interfaces to customize memory allocation in
648the CPython interpreter.
649
650.. seealso::
651
652 :pep:`445` - Add new APIs to customize Python memory allocators
653 PEP written and implemented by Victor Stinner.
654
655
656.. _pep-442:
657
658PEP 442: Safe object finalization
659---------------------------------
660
661:pep:`442` removes the current limitations and quirks of object finalization
662in CPython. With it, objects with :meth:`__del__` methods, as well as
663generators with :keyword:`finally` clauses, can be finalized when they are
664part of a reference cycle.
665
666As part of this change, module globals are no longer forcibly set to
667:const:`None` during interpreter shutdown in most cases, instead relying
668on the normal operation of the cyclic garbage collector.
669
670.. seealso::
671
672 :pep:`442` - Safe object finalization
673 PEP written and implemented by Antoine Pitrou.
674
675
676Other build and C API changes
677-----------------------------
Georg Brandlb80f5112012-09-30 09:11:58 +0200678
679Changes to Python's build process and to the C API include:
680
Nick Coghlan7d270ee2013-10-17 22:35:35 +1000681* The new :c:func:`Py_SetStandardStreamEncoding` pre-initialization API
682 allows applications embedding the CPython interpreter to reliably force
683 a particular encoding and error handler for the standard streams
684 (Contributed by Bastien Montagne and Nick Coghlan in :issue:`16129`)
Georg Brandlb80f5112012-09-30 09:11:58 +0200685
Nick Coghlan0acceb72013-10-20 13:22:21 +1000686* Most Python C APIs that don't mutate string arguments are now correctly
687 marked as accepting ``const char *`` rather than ``char *`` (Contributed
688 by Serhiy Storchaka in :issue:`1772673`).
689
690* "Argument Clinic" (:pep:`436`) is now part of the CPython build process
691 and can be used to simplify the process of defining and maintaining
692 accurate signatures for builtins and standard library extension modules
693 implemented in C.
694
Nick Coghlan367df122013-10-27 01:57:34 +1000695 .. note::
696 The Argument Clinic PEP is not fully up to date with the state of the
697 implementation. This has been deemed acceptable by the release manager
698 and core development team in this case, as Argument Clinic will not
699 be made available as a public API for third party use in Python 3.4.
700
Georg Brandlb80f5112012-09-30 09:11:58 +0200701
702Deprecated
703==========
704
705Unsupported Operating Systems
706-----------------------------
707
Victor Stinnerf3fd13b2013-08-04 10:30:57 +0200708* OS/2
Victor Stinnerf3fd13b2013-08-04 10:30:57 +0200709* Windows 2000
Georg Brandlb80f5112012-09-30 09:11:58 +0200710
711
712Deprecated Python modules, functions and methods
713------------------------------------------------
714
Terry Jan Reedy2b6c26e2013-03-21 19:36:26 -0400715* :meth:`difflib.SequenceMatcher.isbjunk` and
Andrew Kuchling0d0813a2013-06-15 13:29:09 -0400716 :meth:`difflib.SequenceMatcher.isbpopular` were removed: use ``x in sm.bjunk`` and
717 ``x in sm.bpopular``, where *sm* is a :class:`~difflib.SequenceMatcher` object.
Georg Brandlb80f5112012-09-30 09:11:58 +0200718
Brett Cannon82b3d6a2013-06-14 22:37:11 -0400719* :func:`importlib.util.module_for_loader` is pending deprecation. Using
720 :func:`importlib.util.module_to_load` and
721 :meth:`importlib.abc.Loader.init_module_attrs` allows subclasses of a loader
722 to more easily customize module loading.
723
Brett Cannone4f41de2013-06-16 13:13:40 -0400724* The :mod:`imp` module is pending deprecation. To keep compatibility with
725 Python 2/3 code bases, the module's removal is currently not scheduled.
726
Brett Cannon1448ecf2013-10-04 11:38:59 -0400727* The :mod:`formatter` module is pending deprecation and is slated for removal
728 in Python 3.6.
729
Georg Brandlb80f5112012-09-30 09:11:58 +0200730
731Deprecated functions and types of the C API
732-------------------------------------------
733
Victor Stinner3dd263f2013-10-23 18:54:43 +0200734* The ``PyThreadState.tick_counter`` field has been removed: its value was
735 meaningless since Python 3.2 ("new GIL").
Georg Brandlb80f5112012-09-30 09:11:58 +0200736
737
738Deprecated features
739-------------------
740
Antoine Pitrou3b2f0f02013-10-25 21:39:26 +0200741* The site module adding a "site-python" directory to sys.path, if it
742 exists, is deprecated (:issue:`19375`).
Georg Brandlb80f5112012-09-30 09:11:58 +0200743
744
Benjamin Peterson88f3b232012-10-04 12:45:10 -0400745Porting to Python 3.4
Georg Brandlb80f5112012-09-30 09:11:58 +0200746=====================
747
748This section lists previously described changes and other bugfixes
749that may require changes to your code.
750
Brett Cannon777622b2013-04-09 17:03:10 -0400751* The ABCs defined in :mod:`importlib.abc` now either raise the appropriate
752 exception or return a default value instead of raising
753 :exc:`NotImplementedError` blindly. This will only affect code calling
754 :func:`super` and falling through all the way to the ABCs. For compatibility,
755 catch both :exc:`NotImplementedError` or the appropriate exception as needed.
Brett Cannon4c14b5d2013-05-04 13:56:58 -0400756
757* The module type now initializes the :attr:`__package__` and :attr:`__loader__`
758 attributes to ``None`` by default. To determine if these attributes were set
759 in a backwards-compatible fashion, use e.g.
Brett Cannon3dc48d62013-05-28 18:35:54 -0400760 ``getattr(module, '__loader__', None) is not None``.
761
762* :meth:`importlib.util.module_for_loader` now sets ``__loader__`` and
763 ``__package__`` unconditionally to properly support reloading. If this is not
764 desired then you will need to set these attributes manually. You can use
Brett Cannon028d5122013-05-31 18:02:11 -0400765 :func:`importlib.util.module_to_load` for module management.
Brett Cannon3e0651b2013-05-31 23:18:39 -0400766
767* Import now resets relevant attributes (e.g. ``__name__``, ``__loader__``,
768 ``__package__``, ``__file__``, ``__cached__``) unconditionally when reloading.
769
Brett Cannon1448ecf2013-10-04 11:38:59 -0400770* Frozen packages no longer set ``__path__`` to a list containing the package
771 name but an empty list instead. Determing if a module is a package should be
772 done using ``hasattr(module, '__path__')``.
Brett Cannon8f5ac512013-06-12 23:29:18 -0400773
774* :c:func:`PyErr_SetImportError` now sets :exc:`TypeError` when its **msg**
Brett Cannon82da8882013-07-04 17:48:16 -0400775 argument is not set. Previously only ``NULL`` was returned with no exception
776 set.
Brett Cannon33915eb2013-06-14 18:33:00 -0400777
778* :func:`py_compile.compile` now raises :exc:`FileExistsError` if the file path
779 it would write to is a symlink or a non-regular file. This is to act as a
780 warning that import will overwrite those files with a regular file regardless
781 of what type of file path they were originally.
Brett Cannonf4375ef2013-06-16 18:05:54 -0400782
783* :meth:`importlib.abc.SourceLoader.get_source` no longer raises
784 :exc:`ImportError` when the source code being loaded triggers a
785 :exc:`SyntaxError` or :exc:`UnicodeDecodeError`. As :exc:`ImportError` is
786 meant to be raised only when source code cannot be found but it should, it was
787 felt to be over-reaching/overloading of that meaning when the source code is
788 found but improperly structured. If you were catching ImportError before and
789 wish to continue to ignore syntax or decoding issues, catch all three
Victor Stinner84e33c82013-06-21 00:31:55 +0200790 exceptions now.
Nick Coghlan24c05bc2013-07-15 21:13:08 +1000791
792* :func:`functools.update_wrapper` and :func:`functools.wraps` now correctly
Nick Coghlan367df122013-10-27 01:57:34 +1000793 set the ``__wrapped__`` attribute to the function being wrapper, even if
794 that function also had its ``__wrapped__`` attribute set. This means
795 ``__wrapped__`` attributes now correctly link a stack of decorated
796 functions rather than every ``__wrapped__`` attribute in the chain
797 referring to the innermost function. Introspection libraries that
798 assumed the previous behaviour was intentional can use
799 :func:`inspect.unwrap` to access the first function in the chain that has
800 no ``__wrapped__`` attribute.
Victor Stinner2fe9bac2013-10-10 16:18:20 +0200801
Georg Brandlc2228c82013-10-12 13:24:55 +0200802* (C API) The result of the :c:data:`PyOS_ReadlineFunctionPointer` callback must
Victor Stinner2fe9bac2013-10-10 16:18:20 +0200803 now be a string allocated by :c:func:`PyMem_RawMalloc` or
804 :c:func:`PyMem_RawRealloc`, or *NULL* if an error occurred, instead of a
805 string allocated by :c:func:`PyMem_Malloc` or :c:func:`PyMem_Realloc`.
806
Georg Brandl0f5bff22013-10-19 17:46:38 +0200807* :class:`importlib.machinery.PathFinder` now passes on the current working
Brett Cannon27e27f72013-10-18 11:39:04 -0400808 directory to objects in :data:`sys.path_hooks` for the empty string. This
809 results in :data:`sys.path_importer_cache` never containing ``''``, thus
810 iterating through :data:`sys.path_importer_cache` based on :data:`sys.path`
811 will not find all keys. A module's ``__file__`` when imported in the current
812 working directory will also now have an absolute path, including when using
813 ``-m`` with the interpreter (this does not influence when the path to a file
814 is specified on the command-line).
815
Georg Brandl0f5bff22013-10-19 17:46:38 +0200816* :class:`importlib.machinery.FileFinder` no longer special-cases the empty string
Brett Cannon27e27f72013-10-18 11:39:04 -0400817 to be changed to ``'.'``.