Georg Brandl | b80f511 | 2012-09-30 09:11:58 +0200 | [diff] [blame] | 1 | **************************** |
| 2 | What's New In Python 3.4 |
| 3 | **************************** |
| 4 | |
Georg Brandl | 3b80d34 | 2012-10-28 13:37:54 +0100 | [diff] [blame] | 5 | .. :Author: Someone <email> |
| 6 | (uncomment if there is a principal author) |
Georg Brandl | b80f511 | 2012-09-30 09:11:58 +0200 | [diff] [blame] | 7 | |
| 8 | .. Rules for maintenance: |
| 9 | |
Nick Coghlan | 03074fd | 2012-09-30 18:51:53 +0530 | [diff] [blame] | 10 | * Anyone can add text to this document, but the maintainer reserves the |
| 11 | right to rewrite any additions. In particular, for obscure or esoteric |
| 12 | features, the maintainer may reduce any addition to a simple reference to |
| 13 | the new documentation rather than explaining the feature inline. |
Georg Brandl | b80f511 | 2012-09-30 09:11:58 +0200 | [diff] [blame] | 14 | |
Nick Coghlan | 03074fd | 2012-09-30 18:51:53 +0530 | [diff] [blame] | 15 | * While the maintainer will periodically go through Misc/NEWS |
| 16 | and add changes, it's best not to rely on this. We know from experience |
| 17 | that any changes that aren't in the What's New documentation around the |
| 18 | time of the original release will remain largely unknown to the community |
| 19 | for years, even if they're added later. We also know from experience that |
| 20 | other priorities can arise, and the maintainer will run out of time to do |
R David Murray | 061efb1 | 2013-12-24 12:35:59 -0500 | [diff] [blame] | 21 | updates -- in such cases, end users will be much better served by partial |
Nick Coghlan | 03074fd | 2012-09-30 18:51:53 +0530 | [diff] [blame] | 22 | notifications that at least give a hint about new features to |
| 23 | investigate. |
Georg Brandl | b80f511 | 2012-09-30 09:11:58 +0200 | [diff] [blame] | 24 | |
| 25 | * This is not a complete list of every single change; completeness |
Nick Coghlan | 03074fd | 2012-09-30 18:51:53 +0530 | [diff] [blame] | 26 | is the purpose of Misc/NEWS. The What's New should focus on changes that |
| 27 | are visible to Python *users* and that *require* a feature release (i.e. |
| 28 | most bug fixes should only be recorded in Misc/NEWS) |
| 29 | |
| 30 | * PEPs should not be marked Final until they have an entry in What's New. |
| 31 | A placeholder entry that is just a section header and a link to the PEP |
| 32 | (e.g ":pep:`397` has been implemented") is acceptable. If a PEP has been |
| 33 | implemented and noted in What's New, don't forget to mark it as Final! |
Georg Brandl | b80f511 | 2012-09-30 09:11:58 +0200 | [diff] [blame] | 34 | |
| 35 | * If you want to draw your new text to the attention of the |
| 36 | maintainer, add 'XXX' to the beginning of the paragraph or |
| 37 | section. |
| 38 | |
Nick Coghlan | 03074fd | 2012-09-30 18:51:53 +0530 | [diff] [blame] | 39 | * It's OK to add just a very brief note about a change. For |
| 40 | example: "The :ref:`~socket.transmogrify()` function was added to the |
| 41 | :mod:`socket` module." The maintainer will research the change and |
| 42 | write the necessary text (if appropriate). The advantage of doing this |
| 43 | is that even if no more descriptive text is ever added, readers will at |
| 44 | least have a notification that the new feature exists and a link to the |
| 45 | relevant documentation. |
Georg Brandl | b80f511 | 2012-09-30 09:11:58 +0200 | [diff] [blame] | 46 | |
| 47 | * You can comment out your additions if you like, but it's not |
| 48 | necessary (especially when a final release is some months away). |
| 49 | |
| 50 | * Credit the author of a patch or bugfix. Just the name is |
| 51 | sufficient; the e-mail address isn't necessary. |
| 52 | |
| 53 | * It's helpful to add the bug/patch number as a comment: |
| 54 | |
Nick Coghlan | 03074fd | 2012-09-30 18:51:53 +0530 | [diff] [blame] | 55 | The :ref:`~socket.transmogrify()` function was added to the |
| 56 | :mod:`socket` module. (Contributed by P.Y. Developer in :issue:`12345`.) |
Georg Brandl | b80f511 | 2012-09-30 09:11:58 +0200 | [diff] [blame] | 57 | |
| 58 | This saves the maintainer the effort of going through the Mercurial log |
| 59 | when researching a change. |
| 60 | |
Nick Coghlan | 03074fd | 2012-09-30 18:51:53 +0530 | [diff] [blame] | 61 | * Cross referencing tip: :ref:`mod.attr` will display as ``mod.attr``, |
| 62 | while :ref:`~mod.attr` will display as ``attr``. |
| 63 | |
Georg Brandl | b80f511 | 2012-09-30 09:11:58 +0200 | [diff] [blame] | 64 | This article explains the new features in Python 3.4, compared to 3.3. |
| 65 | |
Nick Coghlan | 03074fd | 2012-09-30 18:51:53 +0530 | [diff] [blame] | 66 | .. Python 3.4 was released on TBD. |
Georg Brandl | b80f511 | 2012-09-30 09:11:58 +0200 | [diff] [blame] | 67 | |
Ezio Melotti | 25bbe5e | 2012-11-17 19:30:48 +0200 | [diff] [blame] | 68 | For full details, see the |
| 69 | `changelog <http://docs.python.org/3.4/whatsnew/changelog.html>`_. |
Georg Brandl | b80f511 | 2012-09-30 09:11:58 +0200 | [diff] [blame] | 70 | |
| 71 | .. note:: Prerelease users should be aware that this document is currently in |
| 72 | draft form. It will be updated substantially as Python 3.4 moves towards |
| 73 | release, so it's worth checking back even after reading earlier versions. |
| 74 | |
| 75 | |
| 76 | .. seealso:: |
| 77 | |
R David Murray | 061efb1 | 2013-12-24 12:35:59 -0500 | [diff] [blame] | 78 | :pep:`429` -- Python 3.4 Release Schedule |
Georg Brandl | b80f511 | 2012-09-30 09:11:58 +0200 | [diff] [blame] | 79 | |
| 80 | |
R David Murray | e608255 | 2014-01-03 16:15:45 -0500 | [diff] [blame] | 81 | |
R David Murray | e7cf678 | 2013-12-24 14:51:25 -0500 | [diff] [blame] | 82 | Summary -- Release Highlights |
Georg Brandl | b80f511 | 2012-09-30 09:11:58 +0200 | [diff] [blame] | 83 | ============================= |
| 84 | |
Kristjan Valur Jonsson | a1e8244 | 2013-03-26 13:56:14 +0000 | [diff] [blame] | 85 | .. This section singles out the most important changes in Python 3.4. |
Georg Brandl | b80f511 | 2012-09-30 09:11:58 +0200 | [diff] [blame] | 86 | Brevity is key. |
| 87 | |
| 88 | New syntax features: |
| 89 | |
R David Murray | e0f7a78 | 2013-12-20 16:04:29 -0500 | [diff] [blame] | 90 | * No new syntax features were added in Python 3.4. |
Georg Brandl | b80f511 | 2012-09-30 09:11:58 +0200 | [diff] [blame] | 91 | |
R David Murray | 809487e | 2013-12-22 20:49:40 -0500 | [diff] [blame] | 92 | New expected features for Python implementations: |
| 93 | |
R David Murray | 6879066 | 2013-12-23 11:17:51 -0500 | [diff] [blame] | 94 | * :ref:`pip should always be "available" <whatsnew-pep-453>` (:pep:`453`). |
R David Murray | 809487e | 2013-12-22 20:49:40 -0500 | [diff] [blame] | 95 | * :ref:`Make newly created file descriptors non-inheritable <whatsnew-pep-446>` |
| 96 | (:pep:`446`). |
| 97 | * command line option for :ref:`isolated mode <using-on-misc-options>`, |
| 98 | (:issue:`16499`). |
R David Murray | 8d85678 | 2013-12-23 10:28:57 -0500 | [diff] [blame] | 99 | * :ref:`improvements in the handling of codecs <codec-handling-improvements>` |
| 100 | that are not text encodings (multiple issues). |
R David Murray | 809487e | 2013-12-22 20:49:40 -0500 | [diff] [blame] | 101 | * :ref:`A ModuleSpec Type <whatsnew-pep-451>` for the Import System |
R David Murray | e9b74d4 | 2013-12-22 21:05:04 -0500 | [diff] [blame] | 102 | (:pep:`451`). (Affects importer authors.) |
R David Murray | 809487e | 2013-12-22 20:49:40 -0500 | [diff] [blame] | 103 | |
Georg Brandl | b80f511 | 2012-09-30 09:11:58 +0200 | [diff] [blame] | 104 | New library modules: |
| 105 | |
R David Murray | 9217dad | 2013-12-23 21:08:28 -0500 | [diff] [blame] | 106 | * :mod:`asyncio`: :ref:`New provisional API for asynchronous IO |
| 107 | <whatsnew-asyncio>` (:pep:`3156`). |
| 108 | * :mod:`ensurepip`: :ref:`Bootstrapping the pip installer <whatsnew-ensurepip>` |
| 109 | (:pep:`453`). |
| 110 | * :mod:`enum`: :ref:`Support for enumeration types <whatsnew-enum>` |
| 111 | (:pep:`435`). |
| 112 | * :mod:`pathlib`: :ref:`Object-oriented filesystem paths <whatsnew-pathlib>` |
| 113 | (:pep:`428`). |
R David Murray | f9976e7 | 2013-12-23 10:32:02 -0500 | [diff] [blame] | 114 | * :mod:`selectors`: :ref:`High-level and efficient I/O multiplexing |
| 115 | <whatsnew-selectors>`, built upon the :mod:`select` module primitives (part |
| 116 | of :pep:`3156`). |
R David Murray | 9217dad | 2013-12-23 21:08:28 -0500 | [diff] [blame] | 117 | * :mod:`statistics`: A basic :ref:`numerically stable statistics library |
| 118 | <whatsnew-statistics>` (:pep:`450`). |
| 119 | * :mod:`tracemalloc`: :ref:`Trace Python memory allocations |
| 120 | <whatsnew-tracemalloc>` (:pep:`454`). |
Georg Brandl | b80f511 | 2012-09-30 09:11:58 +0200 | [diff] [blame] | 121 | |
Georg Brandl | b80f511 | 2012-09-30 09:11:58 +0200 | [diff] [blame] | 122 | Significantly Improved Library Modules: |
| 123 | |
R David Murray | 0a10216 | 2013-12-20 15:00:54 -0500 | [diff] [blame] | 124 | * :ref:`Single-dispatch generic functions <whatsnew-singledispatch>` in |
R David Murray | 8d85678 | 2013-12-23 10:28:57 -0500 | [diff] [blame] | 125 | :mod:`functools` (:pep:`443`). |
| 126 | * New :mod:`pickle` :ref:`protocol 4 <whatsnew-protocol-4>` (:pep:`3154`). |
R David Murray | 8d85678 | 2013-12-23 10:28:57 -0500 | [diff] [blame] | 127 | * :ref:`TLSv1.1 and TLSv1.2 support <whatsnew-tls-11-12>` for :mod:`ssl` |
| 128 | (:issue:`16692`). |
R David Murray | ac18622 | 2013-12-20 17:23:57 -0500 | [diff] [blame] | 129 | * :mod:`multiprocessing` now has :ref:`an option to avoid using os.fork |
| 130 | on Unix <whatsnew-multiprocessing-no-fork>` (:issue:`8713`). |
R David Murray | 6adb454 | 2013-12-20 13:10:43 -0500 | [diff] [blame] | 131 | * :mod:`email` has a new submodule, :mod:`~email.contentmanager`, and |
| 132 | a new :mod:`~email.message.Message` subclass |
R David Murray | 26b80cfd | 2013-12-20 17:26:52 -0500 | [diff] [blame] | 133 | (:class:`~email.contentmanager.EmailMessage`) that :ref:`simplify MIME |
R David Murray | 8d85678 | 2013-12-23 10:28:57 -0500 | [diff] [blame] | 134 | handling <whatsnew_email_contentmanager>` (:issue:`18891`). |
Ronald Oussoren | 6db6653 | 2014-01-15 11:32:35 +0100 | [diff] [blame] | 135 | * :mod:`plistlib` has a cleaned up interface and support for binary |
| 136 | plist files (:issue:`14455`) |
Antoine Pitrou | f89aa9a | 2013-08-12 20:46:47 +0200 | [diff] [blame] | 137 | |
Nick Coghlan | 367df12 | 2013-10-27 01:57:34 +1000 | [diff] [blame] | 138 | CPython implementation improvements: |
Georg Brandl | b80f511 | 2012-09-30 09:11:58 +0200 | [diff] [blame] | 139 | |
R David Murray | a93ca97 | 2013-12-22 14:10:21 -0500 | [diff] [blame] | 140 | * :ref:`Safe object finalization <whatsnew-pep-442>` (:pep:`442`). |
R David Murray | ca79461 | 2013-12-22 14:05:11 -0500 | [diff] [blame] | 141 | * Leveraging :pep:`442`, :ref:`module globals are no longer set to None |
| 142 | during finalization <whatsnew-pep-442>`, in most cases (:issue:`18214`). |
R David Murray | a93ca97 | 2013-12-22 14:10:21 -0500 | [diff] [blame] | 143 | * :ref:`Configurable memory allocators <whatsnew-pep-445>` (:pep:`445`). |
| 144 | * :ref:`Secure and interchangeable hash algorithm <whatsnew-pep-456>` |
| 145 | (:pep:`456`). |
| 146 | * :ref:`Argument Clinic <whatsnew-pep-436>` (:pep:`436`). |
R David Murray | 8f7664a | 2013-12-22 20:40:11 -0500 | [diff] [blame] | 147 | * The :mod:`marshal` format has been made :ref:`more compact and efficient |
| 148 | <whatsnew-marshal-3>` (:issue:`16475`). |
Georg Brandl | b80f511 | 2012-09-30 09:11:58 +0200 | [diff] [blame] | 149 | |
R David Murray | 347f9c7 | 2013-12-23 21:30:06 -0500 | [diff] [blame] | 150 | Please read on for a comprehensive list of user-facing changes, including many |
| 151 | other smaller improvements, CPython optimizations, deprecations, and potential |
| 152 | porting issues. |
Georg Brandl | b80f511 | 2012-09-30 09:11:58 +0200 | [diff] [blame] | 153 | |
Nick Coghlan | d0cf063 | 2013-11-11 22:11:55 +1000 | [diff] [blame] | 154 | |
R David Murray | e608255 | 2014-01-03 16:15:45 -0500 | [diff] [blame] | 155 | |
R David Murray | e7cf678 | 2013-12-24 14:51:25 -0500 | [diff] [blame] | 156 | New Expected Features for Python Implementations |
R David Murray | 809487e | 2013-12-22 20:49:40 -0500 | [diff] [blame] | 157 | ================================================ |
| 158 | |
R David Murray | 6879066 | 2013-12-23 11:17:51 -0500 | [diff] [blame] | 159 | .. _whatsnew-pep-453: |
| 160 | |
R David Murray | e7cf678 | 2013-12-24 14:51:25 -0500 | [diff] [blame] | 161 | PEP 453: Explicit Bootstrapping of PIP in Python Installations |
R David Murray | 809487e | 2013-12-22 20:49:40 -0500 | [diff] [blame] | 162 | -------------------------------------------------------------- |
Nick Coghlan | d0cf063 | 2013-11-11 22:11:55 +1000 | [diff] [blame] | 163 | |
| 164 | The new :mod:`ensurepip` module (defined in :pep:`453`) provides a standard |
| 165 | cross-platform mechanism to boostrap the pip installer into Python |
| 166 | installations and virtual environments. |
| 167 | |
Nick Coghlan | 7bc4b3b | 2013-11-23 11:59:40 +1000 | [diff] [blame] | 168 | The :mod:`venv` module and the :command:`pyvenv` utility make use of this |
| 169 | module to make ``pip`` readily available in virtual environments. When |
| 170 | using the command line interface, ``pip`` is installed by default, while |
| 171 | for the module API installation of ``pip`` must be requested explicitly. |
| 172 | |
| 173 | For CPython source builds on POSIX systems, the ``make install`` and |
| 174 | ``make altinstall`` commands bootstrap ``pip`` by default. This behaviour |
| 175 | can be controlled through configure options, and overridden through |
| 176 | Makefile options. |
| 177 | |
Ned Deily | 44a0db0 | 2013-11-22 22:39:09 -0800 | [diff] [blame] | 178 | On Windows and Mac OS X, the CPython installers now offer the option to |
| 179 | install ``pip`` along with CPython itself. |
Nick Coghlan | 7bc4b3b | 2013-11-23 11:59:40 +1000 | [diff] [blame] | 180 | |
R David Murray | 6879066 | 2013-12-23 11:17:51 -0500 | [diff] [blame] | 181 | As `discussed in the PEP`__, platform packagers may choose not to install |
| 182 | ``pip`` by default, as long as the command ``pip``, when invoked, provides |
| 183 | clear and simple directions on how to install ``pip`` on the platform. |
| 184 | |
| 185 | __ http://www.python.org/dev/peps/pep-0453/#recommendations-for-downstream-distributors |
| 186 | |
Nick Coghlan | d0cf063 | 2013-11-11 22:11:55 +1000 | [diff] [blame] | 187 | .. note:: |
| 188 | |
Nick Coghlan | 7bc4b3b | 2013-11-23 11:59:40 +1000 | [diff] [blame] | 189 | The implementation of PEP 453 is still a work in progress. Refer to |
| 190 | :issue:`19347` for the progress on additional steps: |
Nick Coghlan | d0cf063 | 2013-11-11 22:11:55 +1000 | [diff] [blame] | 191 | |
Nick Coghlan | 7bc4b3b | 2013-11-23 11:59:40 +1000 | [diff] [blame] | 192 | * Having the binary installers install ``pip`` by default |
| 193 | * Recommending the use of ``pip`` in the "Installing Python Module" |
| 194 | documentation. |
Nick Coghlan | d0cf063 | 2013-11-11 22:11:55 +1000 | [diff] [blame] | 195 | |
| 196 | .. seealso:: |
| 197 | |
R David Murray | 061efb1 | 2013-12-24 12:35:59 -0500 | [diff] [blame] | 198 | :pep:`453` -- Explicit bootstrapping of pip in Python installations |
Nick Coghlan | d0cf063 | 2013-11-11 22:11:55 +1000 | [diff] [blame] | 199 | PEP written by Donald Stufft and Nick Coghlan, implemented by |
Nick Coghlan | 7bc4b3b | 2013-11-23 11:59:40 +1000 | [diff] [blame] | 200 | Donald Stufft, Nick Coghlan, Martin von Löwis and Ned Deily. |
Nick Coghlan | d0cf063 | 2013-11-11 22:11:55 +1000 | [diff] [blame] | 201 | |
| 202 | |
R David Murray | f9909c2 | 2013-12-20 14:50:12 -0500 | [diff] [blame] | 203 | .. _whatsnew-pep-446: |
Victor Stinner | daf4555 | 2013-08-28 00:53:59 +0200 | [diff] [blame] | 204 | |
R David Murray | e7cf678 | 2013-12-24 14:51:25 -0500 | [diff] [blame] | 205 | PEP 446: Make Newly Created File Descriptors Non-Inheritable |
R David Murray | 809487e | 2013-12-22 20:49:40 -0500 | [diff] [blame] | 206 | ------------------------------------------------------------ |
Victor Stinner | daf4555 | 2013-08-28 00:53:59 +0200 | [diff] [blame] | 207 | |
Nick Coghlan | 367df12 | 2013-10-27 01:57:34 +1000 | [diff] [blame] | 208 | :pep:`446` makes newly created file descriptors :ref:`non-inheritable |
Georg Brandl | 5642ff9 | 2013-09-15 10:37:57 +0200 | [diff] [blame] | 209 | <fd_inheritance>`. New functions and methods: |
Victor Stinner | daf4555 | 2013-08-28 00:53:59 +0200 | [diff] [blame] | 210 | |
| 211 | * :func:`os.get_inheritable`, :func:`os.set_inheritable` |
| 212 | * :func:`os.get_handle_inheritable`, :func:`os.set_handle_inheritable` |
| 213 | * :meth:`socket.socket.get_inheritable`, :meth:`socket.socket.set_inheritable` |
| 214 | |
Antoine Pitrou | 796564c | 2013-07-30 19:59:21 +0200 | [diff] [blame] | 215 | .. seealso:: |
| 216 | |
R David Murray | 061efb1 | 2013-12-24 12:35:59 -0500 | [diff] [blame] | 217 | :pep:`446` -- Make newly created file descriptors non-inheritable |
Nick Coghlan | 367df12 | 2013-10-27 01:57:34 +1000 | [diff] [blame] | 218 | PEP written and implemented by Victor Stinner. |
Antoine Pitrou | 796564c | 2013-07-30 19:59:21 +0200 | [diff] [blame] | 219 | |
Georg Brandl | b80f511 | 2012-09-30 09:11:58 +0200 | [diff] [blame] | 220 | |
Nick Coghlan | 9c1aed8 | 2013-11-23 11:13:36 +1000 | [diff] [blame] | 221 | .. _codec-handling-improvements: |
| 222 | |
R David Murray | e7cf678 | 2013-12-24 14:51:25 -0500 | [diff] [blame] | 223 | Improvements to Codec Handling |
R David Murray | 809487e | 2013-12-22 20:49:40 -0500 | [diff] [blame] | 224 | ------------------------------ |
Nick Coghlan | 8b097b4 | 2013-11-13 23:49:21 +1000 | [diff] [blame] | 225 | |
| 226 | Since it was first introduced, the :mod:`codecs` module has always been |
| 227 | intended to operate as a type-neutral dynamic encoding and decoding |
| 228 | system. However, its close coupling with the Python text model, especially |
| 229 | the type restricted convenience methods on the builtin :class:`str`, |
| 230 | :class:`bytes` and :class:`bytearray` types, has historically obscured that |
| 231 | fact. |
| 232 | |
| 233 | As a key step in clarifying the situation, the :meth:`codecs.encode` and |
| 234 | :meth:`codecs.decode` convenience functions are now properly documented in |
| 235 | Python 2.7, 3.3 and 3.4. These functions have existed in the :mod:`codecs` |
Nick Coghlan | 9c1aed8 | 2013-11-23 11:13:36 +1000 | [diff] [blame] | 236 | module (and have been covered by the regression test suite) since Python 2.4, |
Nick Coghlan | 8b097b4 | 2013-11-13 23:49:21 +1000 | [diff] [blame] | 237 | but were previously only discoverable through runtime introspection. |
| 238 | |
| 239 | Unlike the convenience methods on :class:`str`, :class:`bytes` and |
| 240 | :class:`bytearray`, these convenience functions support arbitrary codecs |
| 241 | in both Python 2 and Python 3, rather than being limited to Unicode text |
| 242 | encodings (in Python 3) or ``basestring`` <-> ``basestring`` conversions |
| 243 | (in Python 2). |
| 244 | |
Nick Coghlan | 8afc8f6 | 2013-11-22 23:00:22 +1000 | [diff] [blame] | 245 | In Python 3.4, the interpreter is able to identify the known non-text |
| 246 | encodings provided in the standard library and direct users towards these |
| 247 | general purpose convenience functions when appropriate:: |
Nick Coghlan | 8b097b4 | 2013-11-13 23:49:21 +1000 | [diff] [blame] | 248 | |
Nick Coghlan | 9c1aed8 | 2013-11-23 11:13:36 +1000 | [diff] [blame] | 249 | >>> b"abcdef".decode("hex") |
Nick Coghlan | 8b097b4 | 2013-11-13 23:49:21 +1000 | [diff] [blame] | 250 | Traceback (most recent call last): |
| 251 | File "<stdin>", line 1, in <module> |
Nick Coghlan | 9c1aed8 | 2013-11-23 11:13:36 +1000 | [diff] [blame] | 252 | LookupError: 'hex' is not a text encoding; use codecs.decode() to handle arbitrary codecs |
Nick Coghlan | 8b097b4 | 2013-11-13 23:49:21 +1000 | [diff] [blame] | 253 | |
Nick Coghlan | 9c1aed8 | 2013-11-23 11:13:36 +1000 | [diff] [blame] | 254 | >>> "hello".encode("rot13") |
Nick Coghlan | 8b097b4 | 2013-11-13 23:49:21 +1000 | [diff] [blame] | 255 | Traceback (most recent call last): |
| 256 | File "<stdin>", line 1, in <module> |
Nick Coghlan | 9c1aed8 | 2013-11-23 11:13:36 +1000 | [diff] [blame] | 257 | LookupError: 'rot13' is not a text encoding; use codecs.encode() to handle arbitrary codecs |
Nick Coghlan | 8b097b4 | 2013-11-13 23:49:21 +1000 | [diff] [blame] | 258 | |
| 259 | In a related change, whenever it is feasible without breaking backwards |
| 260 | compatibility, exceptions raised during encoding and decoding operations |
| 261 | will be wrapped in a chained exception of the same type that mentions the |
| 262 | name of the codec responsible for producing the error:: |
| 263 | |
Nick Coghlan | 9c1aed8 | 2013-11-23 11:13:36 +1000 | [diff] [blame] | 264 | >>> import codecs |
| 265 | |
| 266 | >>> codecs.decode(b"abcdefgh", "hex") |
Nick Coghlan | 77b286b | 2014-01-27 00:53:38 +1000 | [diff] [blame] | 267 | Traceback (most recent call last): |
| 268 | File "/usr/lib/python3.4/encodings/hex_codec.py", line 20, in hex_decode |
| 269 | return (binascii.a2b_hex(input), len(input)) |
Nick Coghlan | 8afc8f6 | 2013-11-22 23:00:22 +1000 | [diff] [blame] | 270 | binascii.Error: Non-hexadecimal digit found |
Nick Coghlan | 8b097b4 | 2013-11-13 23:49:21 +1000 | [diff] [blame] | 271 | |
| 272 | The above exception was the direct cause of the following exception: |
| 273 | |
| 274 | Traceback (most recent call last): |
| 275 | File "<stdin>", line 1, in <module> |
Nick Coghlan | 9c1aed8 | 2013-11-23 11:13:36 +1000 | [diff] [blame] | 276 | binascii.Error: decoding with 'hex' codec failed (Error: Non-hexadecimal digit found) |
Nick Coghlan | 8b097b4 | 2013-11-13 23:49:21 +1000 | [diff] [blame] | 277 | |
Nick Coghlan | 9c1aed8 | 2013-11-23 11:13:36 +1000 | [diff] [blame] | 278 | >>> codecs.encode("hello", "bz2") |
Nick Coghlan | 77b286b | 2014-01-27 00:53:38 +1000 | [diff] [blame] | 279 | Traceback (most recent call last): |
| 280 | File "/usr/lib/python3.4/encodings/bz2_codec.py", line 17, in bz2_encode |
| 281 | return (bz2.compress(input), len(input)) |
| 282 | File "/usr/lib/python3.4/bz2.py", line 498, in compress |
| 283 | return comp.compress(data) + comp.flush() |
Nick Coghlan | 8b097b4 | 2013-11-13 23:49:21 +1000 | [diff] [blame] | 284 | TypeError: 'str' does not support the buffer interface |
| 285 | |
| 286 | The above exception was the direct cause of the following exception: |
| 287 | |
| 288 | Traceback (most recent call last): |
| 289 | File "<stdin>", line 1, in <module> |
Nick Coghlan | 9c1aed8 | 2013-11-23 11:13:36 +1000 | [diff] [blame] | 290 | TypeError: encoding with 'bz2' codec failed (TypeError: 'str' does not support the buffer interface) |
Nick Coghlan | 8b097b4 | 2013-11-13 23:49:21 +1000 | [diff] [blame] | 291 | |
Nick Coghlan | 9c1aed8 | 2013-11-23 11:13:36 +1000 | [diff] [blame] | 292 | Finally, as the examples above show, these improvements have permitted |
| 293 | the restoration of the convenience aliases for the non-Unicode codecs that |
| 294 | were themselves restored in Python 3.2. This means that encoding binary data |
| 295 | to and from its hexadecimal representation (for example) can now be written |
| 296 | as:: |
| 297 | |
| 298 | >>> from codecs import encode, decode |
| 299 | >>> encode(b"hello", "hex") |
| 300 | b'68656c6c6f' |
| 301 | >>> decode(b"68656c6c6f", "hex") |
| 302 | b'hello' |
| 303 | |
| 304 | The binary and text transforms provided in the standard library are detailed |
| 305 | in :ref:`binary-transforms` and :ref:`text-transforms`. |
| 306 | |
R David Murray | 9217dad | 2013-12-23 21:08:28 -0500 | [diff] [blame] | 307 | (Contributed by Nick Coghlan in :issue:`7475`, , :issue:`17827`, |
Nick Coghlan | 9c1aed8 | 2013-11-23 11:13:36 +1000 | [diff] [blame] | 308 | :issue:`17828` and :issue:`19619`) |
Nick Coghlan | 8b097b4 | 2013-11-13 23:49:21 +1000 | [diff] [blame] | 309 | |
R David Murray | 809487e | 2013-12-22 20:49:40 -0500 | [diff] [blame] | 310 | .. _whatsnew-pep-451: |
Eric Snow | b523f84 | 2013-11-22 09:05:39 -0700 | [diff] [blame] | 311 | |
| 312 | PEP 451: A ModuleSpec Type for the Import System |
R David Murray | 809487e | 2013-12-22 20:49:40 -0500 | [diff] [blame] | 313 | ------------------------------------------------ |
Eric Snow | b523f84 | 2013-11-22 09:05:39 -0700 | [diff] [blame] | 314 | |
R David Murray | f85b2a8 | 2013-12-22 21:06:13 -0500 | [diff] [blame] | 315 | :pep:`451` provides an encapsulation of the information about a module that the |
| 316 | import machinery will use to load it (that is, a module specification). This |
| 317 | helps simplify both the import implementation and several import-related APIs. |
| 318 | The change is also a stepping stone for `several future import-related |
| 319 | improvements`__. |
Eric Snow | b523f84 | 2013-11-22 09:05:39 -0700 | [diff] [blame] | 320 | |
R David Murray | e9b74d4 | 2013-12-22 21:05:04 -0500 | [diff] [blame] | 321 | __ https://mail.python.org/pipermail/python-dev/2013-November/130111.html |
Eric Snow | b523f84 | 2013-11-22 09:05:39 -0700 | [diff] [blame] | 322 | |
| 323 | The public-facing changes from the PEP are entirely backward-compatible. |
R David Murray | f85b2a8 | 2013-12-22 21:06:13 -0500 | [diff] [blame] | 324 | Furthermore, they should be transparent to everyone but importer authors. Key |
| 325 | finder and loader methods have been deprecated, but they will continue working. |
| 326 | New importers should use the new methods described in the PEP. Existing |
| 327 | importers should be updated to implement the new methods. |
Eric Snow | b523f84 | 2013-11-22 09:05:39 -0700 | [diff] [blame] | 328 | |
Nick Coghlan | 8b097b4 | 2013-11-13 23:49:21 +1000 | [diff] [blame] | 329 | |
Georg Brandl | b80f511 | 2012-09-30 09:11:58 +0200 | [diff] [blame] | 330 | Other Language Changes |
R David Murray | 809487e | 2013-12-22 20:49:40 -0500 | [diff] [blame] | 331 | ---------------------- |
Georg Brandl | b80f511 | 2012-09-30 09:11:58 +0200 | [diff] [blame] | 332 | |
| 333 | Some smaller changes made to the core Python language are: |
| 334 | |
Ezio Melotti | 34808e2 | 2013-10-12 16:36:13 +0300 | [diff] [blame] | 335 | * Unicode database updated to UCD version 6.3. |
Georg Brandl | b80f511 | 2012-09-30 09:11:58 +0200 | [diff] [blame] | 336 | |
R David Murray | 9a2f139 | 2013-06-28 13:31:19 -0400 | [diff] [blame] | 337 | * :func:`min` and :func:`max` now accept a *default* argument that can be used |
| 338 | to specify the value they return if the iterable they are evaluating has no |
| 339 | elements. Contributed by Julian Berman in :issue:`18111`. |
Georg Brandl | b80f511 | 2012-09-30 09:11:58 +0200 | [diff] [blame] | 340 | |
Antoine Pitrou | f89aa9a | 2013-08-12 20:46:47 +0200 | [diff] [blame] | 341 | * Module objects are now :mod:`weakref`'able. |
| 342 | |
Nick Coghlan | 0acceb7 | 2013-10-20 13:22:21 +1000 | [diff] [blame] | 343 | * Module ``__file__`` attributes (and related values) should now always |
| 344 | contain absolute paths by default, with the sole exception of |
| 345 | ``__main__.__file__`` when a script has been executed directly using |
| 346 | a relative path (Contributed by Brett Cannon in :issue:`18416`). |
| 347 | |
Serhiy Storchaka | 58cf607 | 2013-11-19 11:32:41 +0200 | [diff] [blame] | 348 | * Now all the UTF-\* codecs (except UTF-7) reject surrogates during both |
| 349 | encoding and decoding unless the ``surrogatepass`` error handler is used, |
| 350 | with the exception of the UTF-16 decoder that accepts valid surrogate pairs, |
| 351 | and the UTF-16 encoder that produces them while encoding non-BMP characters. |
| 352 | Contributed by Victor Stinner, Kang-Hao (Kenny) Lu and Serhiy Storchaka in |
| 353 | :issue:`12892`. |
| 354 | |
Georg Brandl | b80f511 | 2012-09-30 09:11:58 +0200 | [diff] [blame] | 355 | |
R David Murray | e608255 | 2014-01-03 16:15:45 -0500 | [diff] [blame] | 356 | |
Georg Brandl | b80f511 | 2012-09-30 09:11:58 +0200 | [diff] [blame] | 357 | New Modules |
| 358 | =========== |
| 359 | |
Nick Coghlan | 0acceb7 | 2013-10-20 13:22:21 +1000 | [diff] [blame] | 360 | |
R David Murray | 9217dad | 2013-12-23 21:08:28 -0500 | [diff] [blame] | 361 | .. _whatsnew-asyncio: |
| 362 | |
Nick Coghlan | 0acceb7 | 2013-10-20 13:22:21 +1000 | [diff] [blame] | 363 | asyncio |
| 364 | ------- |
| 365 | |
| 366 | The new :mod:`asyncio` module (defined in :pep:`3156`) provides a standard |
| 367 | pluggable event loop model for Python, providing solid asynchronous IO |
| 368 | support in the standard library, and making it easier for other event loop |
| 369 | implementations to interoperate with the standard library and each other. |
| 370 | |
| 371 | For Python 3.4, this module is considered a :term:`provisional API`. |
| 372 | |
Nick Coghlan | 367df12 | 2013-10-27 01:57:34 +1000 | [diff] [blame] | 373 | .. seealso:: |
| 374 | |
R David Murray | 061efb1 | 2013-12-24 12:35:59 -0500 | [diff] [blame] | 375 | :pep:`3156` -- Asynchronous IO Support Rebooted: the "asyncio" Module |
Nick Coghlan | 367df12 | 2013-10-27 01:57:34 +1000 | [diff] [blame] | 376 | PEP written and implementation led by Guido van Rossum. |
| 377 | |
R David Murray | 6879066 | 2013-12-23 11:17:51 -0500 | [diff] [blame] | 378 | |
R David Murray | 9217dad | 2013-12-23 21:08:28 -0500 | [diff] [blame] | 379 | .. _whatsnew-ensurepip: |
R David Murray | 6879066 | 2013-12-23 11:17:51 -0500 | [diff] [blame] | 380 | |
| 381 | ensurepip |
| 382 | --------- |
| 383 | |
| 384 | The new :mod:`ensurepip` module is the primary infrastructure for the |
| 385 | :pep:`453` implementation. In the normal course of events end users will not |
| 386 | need to interact with this module, but it can be used to manually bootstrap |
| 387 | ``pip`` if the automated bootstrapping into an installation or virtual |
| 388 | environment was declined. |
| 389 | |
| 390 | :mod:`ensurepip` includes a bundled copy of ``pip``, up-to-date as of the first |
| 391 | release candidate of the release of CPython with which it ships (this applies |
| 392 | to both maintenance releases and feature releases). ``ensurepip`` does not |
| 393 | access the internet. (If the installation has Internet access, it is of course |
| 394 | possible to upgrade ``pip`` to a release more recent than the bundled ``pip`` |
| 395 | by using the bundled ``pip`` command itself once it is installed.) |
| 396 | |
| 397 | The module is named *ensure*\ pip because if called when ``pip`` is already |
| 398 | installed, it does nothing. It also has an ``--upgrade`` option that will |
| 399 | cause it to install the bundled copy of ``pip`` if the existing installed |
| 400 | version of ``pip`` is older than the bundled copy. |
| 401 | |
| 402 | |
R David Murray | 9217dad | 2013-12-23 21:08:28 -0500 | [diff] [blame] | 403 | .. _whatsnew-enum: |
| 404 | |
Nick Coghlan | 0acceb7 | 2013-10-20 13:22:21 +1000 | [diff] [blame] | 405 | enum |
| 406 | ---- |
| 407 | |
Nick Coghlan | 367df12 | 2013-10-27 01:57:34 +1000 | [diff] [blame] | 408 | The new :mod:`enum` module (defined in :pep:`435`) provides a standard |
| 409 | implementation of enumeration types, allowing other modules (such as |
| 410 | :mod:`socket`) to provide more informative error messages and better |
| 411 | debugging support by replacing opaque integer constants with backwards |
| 412 | compatible enumeration values. |
| 413 | |
| 414 | .. seealso:: |
| 415 | |
R David Murray | 061efb1 | 2013-12-24 12:35:59 -0500 | [diff] [blame] | 416 | :pep:`435` -- Adding an Enum type to the Python standard library |
Nick Coghlan | 367df12 | 2013-10-27 01:57:34 +1000 | [diff] [blame] | 417 | PEP written by Barry Warsaw, Eli Bendersky and Ethan Furman, |
| 418 | implemented by Ethan Furman. |
Nick Coghlan | 0acceb7 | 2013-10-20 13:22:21 +1000 | [diff] [blame] | 419 | |
| 420 | |
R David Murray | 9217dad | 2013-12-23 21:08:28 -0500 | [diff] [blame] | 421 | .. _whatsnew-pathlib: |
| 422 | |
Antoine Pitrou | 31119e4 | 2013-11-22 17:38:12 +0100 | [diff] [blame] | 423 | pathlib |
| 424 | ------- |
| 425 | |
| 426 | The new :mod:`pathlib` module offers classes representing filesystem paths |
| 427 | with semantics appropriate for different operating systems. Path classes are |
| 428 | divided between *pure paths*, which provide purely computational operations |
| 429 | without I/O, and *concrete paths*, which inherit from pure paths but also |
| 430 | provide I/O operations. |
| 431 | |
| 432 | For Python 3.4, this module is considered a :term:`provisional API`. |
| 433 | |
| 434 | .. seealso:: |
| 435 | |
R David Murray | 061efb1 | 2013-12-24 12:35:59 -0500 | [diff] [blame] | 436 | :pep:`428` -- The pathlib module -- object-oriented filesystem paths |
Antoine Pitrou | 31119e4 | 2013-11-22 17:38:12 +0100 | [diff] [blame] | 437 | PEP written and implemented by Antoine Pitrou. |
| 438 | |
| 439 | |
R David Murray | f9976e7 | 2013-12-23 10:32:02 -0500 | [diff] [blame] | 440 | .. _whatsnew-selectors: |
| 441 | |
Charles-François Natali | 243d8d8 | 2013-09-04 19:02:49 +0200 | [diff] [blame] | 442 | selectors |
| 443 | --------- |
Georg Brandl | b80f511 | 2012-09-30 09:11:58 +0200 | [diff] [blame] | 444 | |
Nick Coghlan | 0acceb7 | 2013-10-20 13:22:21 +1000 | [diff] [blame] | 445 | The new :mod:`selectors` module (created as part of implementing :pep:`3156`) |
| 446 | allows high-level and efficient I/O multiplexing, built upon the |
| 447 | :mod:`select` module primitives. |
| 448 | |
| 449 | |
R David Murray | 9217dad | 2013-12-23 21:08:28 -0500 | [diff] [blame] | 450 | .. _whatsnew-statistics: |
| 451 | |
Nick Coghlan | 0acceb7 | 2013-10-20 13:22:21 +1000 | [diff] [blame] | 452 | statistics |
| 453 | ---------- |
| 454 | |
| 455 | The new :mod:`statistics` module (defined in :pep:`450`) offers some core |
| 456 | statistics functionality directly in the standard library. This module |
| 457 | supports calculation of the mean, median, mode, variance and standard |
| 458 | deviation of a data series. |
Georg Brandl | b80f511 | 2012-09-30 09:11:58 +0200 | [diff] [blame] | 459 | |
Nick Coghlan | 367df12 | 2013-10-27 01:57:34 +1000 | [diff] [blame] | 460 | .. seealso:: |
| 461 | |
R David Murray | 061efb1 | 2013-12-24 12:35:59 -0500 | [diff] [blame] | 462 | :pep:`450` -- Adding A Statistics Module To The Standard Library |
Nick Coghlan | 367df12 | 2013-10-27 01:57:34 +1000 | [diff] [blame] | 463 | PEP written and implemented by Steven D'Aprano |
| 464 | |
R David Murray | 9217dad | 2013-12-23 21:08:28 -0500 | [diff] [blame] | 465 | .. _whatsnew-tracemalloc: |
Georg Brandl | b80f511 | 2012-09-30 09:11:58 +0200 | [diff] [blame] | 466 | |
R David Murray | 8b2d682 | 2013-12-31 15:06:05 -0500 | [diff] [blame] | 467 | |
Victor Stinner | d2736af | 2013-11-25 09:40:27 +0100 | [diff] [blame] | 468 | tracemalloc |
| 469 | ----------- |
| 470 | |
| 471 | The new :mod:`tracemalloc` module (defined in :pep:`454`) is a debug tool to |
| 472 | trace memory blocks allocated by Python. It provides the following information: |
| 473 | |
| 474 | * Traceback where an object was allocated |
| 475 | * Statistics on allocated memory blocks per filename and per line number: |
| 476 | total size, number and average size of allocated memory blocks |
| 477 | * Compute the differences between two snapshots to detect memory leaks |
| 478 | |
| 479 | .. seealso:: |
| 480 | |
R David Murray | 061efb1 | 2013-12-24 12:35:59 -0500 | [diff] [blame] | 481 | :pep:`454` -- Add a new tracemalloc module to trace Python memory allocations |
Victor Stinner | d2736af | 2013-11-25 09:40:27 +0100 | [diff] [blame] | 482 | PEP written and implemented by Victor Stinner |
| 483 | |
| 484 | |
R David Murray | e608255 | 2014-01-03 16:15:45 -0500 | [diff] [blame] | 485 | |
Georg Brandl | b80f511 | 2012-09-30 09:11:58 +0200 | [diff] [blame] | 486 | Improved Modules |
| 487 | ================ |
| 488 | |
R David Murray | 6e39015 | 2013-12-24 22:28:04 -0500 | [diff] [blame] | 489 | |
R David Murray | 3edcc78 | 2013-12-24 16:13:32 -0500 | [diff] [blame] | 490 | abc |
| 491 | --- |
| 492 | |
| 493 | New function :func:`abc.get_cache_token` can be used to know when to invalidate |
| 494 | caches that are affected by changes in the object graph. (Contributed |
| 495 | by Łukasz Langa in :issue:`16832`.) |
R David Murray | 72420ff | 2013-12-24 10:46:44 -0500 | [diff] [blame] | 496 | |
R David Murray | 2691ee6 | 2013-12-28 23:15:12 -0500 | [diff] [blame] | 497 | New class :class:`~abc.ABC` has :class:`~abc.ABCMeta` as its meta class. |
| 498 | Using ``ABC`` as a base class has essentially the same effect as specifying |
| 499 | ``metaclass=abc.ABCMeta``, but is simpler to type and easier to read. |
| 500 | (Contributed by Bruno Dupuis in :issue:`16049`.) |
| 501 | |
| 502 | |
Antoine Pitrou | d6cbd34 | 2013-08-12 20:48:15 +0200 | [diff] [blame] | 503 | aifc |
| 504 | ---- |
| 505 | |
| 506 | The :meth:`~aifc.getparams` method now returns a namedtuple rather than a |
| 507 | plain tuple. (Contributed by Claudiu Popa in :issue:`17818`.) |
| 508 | |
R David Murray | d592bb2 | 2013-12-31 13:45:38 -0500 | [diff] [blame] | 509 | :func:`aifc.open` now supports the context manager protocol: when used in a |
| 510 | :keyword:`with` block, the :meth:`~aifc.aifc.close` method of the returned |
| 511 | object will be called automatically at the end of the block. (Contributed by |
| 512 | Serhiy Storchacha in :issue:`16486`.) |
| 513 | |
Antoine Pitrou | d6cbd34 | 2013-08-12 20:48:15 +0200 | [diff] [blame] | 514 | |
R David Murray | fced3ec | 2013-12-31 11:18:01 -0500 | [diff] [blame] | 515 | argparse |
| 516 | -------- |
| 517 | |
| 518 | The :class:`~argparse.FileType` class now accepts *encoding* and |
| 519 | *errors* arguments, which are passed through to :func:`open`. (Contributed |
| 520 | by Lucas Maystre in :issue:`11175`.) |
| 521 | |
| 522 | |
Serhiy Storchaka | eaea5e9 | 2013-10-19 21:10:46 +0300 | [diff] [blame] | 523 | audioop |
| 524 | ------- |
| 525 | |
| 526 | Added support for 24-bit samples (:issue:`12866`). |
| 527 | |
Serhiy Storchaka | 3062c9a | 2013-11-23 22:26:01 +0200 | [diff] [blame] | 528 | Added the :func:`~audioop.byteswap` function to convert big-endian samples |
| 529 | to little-endian and vice versa (:issue:`19641`). |
| 530 | |
Serhiy Storchaka | eaea5e9 | 2013-10-19 21:10:46 +0300 | [diff] [blame] | 531 | |
Nick Coghlan | d4fdbcc | 2013-11-14 00:24:31 +1000 | [diff] [blame] | 532 | base64 |
| 533 | ------ |
| 534 | |
| 535 | The encoding and decoding functions in :mod:`base64` now accept any |
| 536 | :term:`bytes-like object` in cases where it previously required a |
Georg Brandl | ed007d5 | 2013-11-24 16:09:26 +0100 | [diff] [blame] | 537 | :class:`bytes` or :class:`bytearray` instance (:issue:`17839`). |
Nick Coghlan | d4fdbcc | 2013-11-14 00:24:31 +1000 | [diff] [blame] | 538 | |
| 539 | |
Antoine Pitrou | d6cbd34 | 2013-08-12 20:48:15 +0200 | [diff] [blame] | 540 | colorsys |
| 541 | -------- |
| 542 | |
| 543 | The number of digits in the coefficients for the RGB --- YIQ conversions have |
| 544 | been expanded so that they match the FCC NTSC versions. The change in |
| 545 | results should be less than 1% and may better match results found elsewhere. |
| 546 | |
R David Murray | 8e37d5d | 2013-04-13 14:49:48 -0400 | [diff] [blame] | 547 | |
Nick Coghlan | b4534ae | 2013-10-13 23:23:08 +1000 | [diff] [blame] | 548 | contextlib |
| 549 | ---------- |
| 550 | |
Nick Coghlan | 240f86d | 2013-10-17 23:40:57 +1000 | [diff] [blame] | 551 | The new :class:`contextlib.suppress` context manager helps to clarify the |
| 552 | intent of code that deliberately suppresses exceptions from a single |
| 553 | statement. (Contributed by Raymond Hettinger in :issue:`15806` and |
| 554 | Zero Piraeus in :issue:`19266`) |
| 555 | |
Victor Stinner | 6633c39 | 2013-10-21 13:27:11 +0200 | [diff] [blame] | 556 | The new :func:`contextlib.redirect_stdout` context manager makes it easier |
Nick Coghlan | b4534ae | 2013-10-13 23:23:08 +1000 | [diff] [blame] | 557 | for utility scripts to handle inflexible APIs that don't provide any |
| 558 | options to retrieve their output as a string or direct it to somewhere |
Nick Coghlan | 0acceb7 | 2013-10-20 13:22:21 +1000 | [diff] [blame] | 559 | other than :data:`sys.stdout`. In conjunction with :class:`io.StringIO`, |
| 560 | this context manager is also useful for checking expected output from |
| 561 | command line utilities. (Contribute by Raymond Hettinger in :issue:`15805`) |
| 562 | |
| 563 | The :mod:`contextlib` documentation has also been updated to include a |
| 564 | :ref:`discussion <single-use-reusable-and-reentrant-cms>` of the |
| 565 | differences between single use, reusable and reentrant context managers. |
Nick Coghlan | b4534ae | 2013-10-13 23:23:08 +1000 | [diff] [blame] | 566 | |
| 567 | |
Nick Coghlan | b39fd0c | 2013-05-06 23:59:20 +1000 | [diff] [blame] | 568 | dis |
| 569 | --- |
| 570 | |
R David Murray | 0bce6e7 | 2014-01-07 14:30:17 -0500 | [diff] [blame] | 571 | Functions :func:`~dis.show_code`, :func:`~dis.dis`, :func:`~dis.distb`, and |
| 572 | :func:`~dis.disassemble` now accept a keyword-only *file* argument that |
| 573 | controls where they write their output. |
Nick Coghlan | b39fd0c | 2013-05-06 23:59:20 +1000 | [diff] [blame] | 574 | |
R David Murray | 0bce6e7 | 2014-01-07 14:30:17 -0500 | [diff] [blame] | 575 | The :mod:`dis` module is now built around an :class:`~dis.Instruction` class |
| 576 | that provides object oriented access to the details of each individual bytecode |
| 577 | operation. |
| 578 | |
| 579 | A new method, :func:`~dis.get_instructions`, provides an iterator that emits |
| 580 | the Instruction stream for a given piece of Python code. Thus it is now |
| 581 | possible to write a program that inspects and manipulates a bytecode |
| 582 | object in ways different from those provided by the :mod:`~dis` module |
| 583 | itself. For example:: |
| 584 | |
| 585 | >>> import dis |
| 586 | >>> for instr in dis.get_instructions(lambda x: x + 1): |
| 587 | ... print(instr.opname) |
| 588 | LOAD_FAST |
| 589 | LOAD_CONST |
| 590 | BINARY_ADD |
| 591 | RETURN_VALUE |
| 592 | |
| 593 | The various display tools in the :mod:`dis` module have been rewritten to use |
| 594 | these new components. |
| 595 | |
| 596 | In addition, a new application-friendly class :class:`~dis.Bytecode` provides |
| 597 | an object-oriented API for inspecting bytecode in both in human-readable form |
| 598 | and for iterating over instructions. The :class:`~dis.Bytecode` constructor |
| 599 | takes the same arguments that :func:`~dis.get_instruction` does (plus an |
| 600 | optional *current_offset*), and the resulting object can be iterated to produce |
| 601 | :class:`~dis.Instruction` objects. But it also has a :mod:`~dis.Bytecode.dis` |
| 602 | method, equivalent to calling :mod:`~dis.dis` on the constructor argument, but |
| 603 | returned as a multi-line string:: |
| 604 | |
| 605 | >>> bytecode = dis.Bytecode(lambda x: x +1, current_offset=3) |
| 606 | >>> for instr in bytecode: |
| 607 | ... print('{} ({})'.format(instr.opname, instr.opcode)) |
| 608 | LOAD_FAST (124) |
| 609 | LOAD_CONST (100) |
| 610 | BINARY_ADD (23) |
| 611 | RETURN_VALUE (83) |
| 612 | >>> bytecode.dis().splitlines() # doctest: +NORMALIZE_WHITESPACE |
| 613 | [' 1 0 LOAD_FAST 0 (x)', |
| 614 | ' --> 3 LOAD_CONST 1 (1)', |
| 615 | ' 6 BINARY_ADD', |
| 616 | ' 7 RETURN_VALUE'] |
| 617 | |
| 618 | :class:`~dis.Bytecode` also has a class method, |
| 619 | :meth:`~dis.Bytecode.from_traceback`, that provides the ability to manipulate a |
| 620 | traceback (that is, ``print(Bytecode.from_traceback(tb).dis())`` is equivalent |
| 621 | to ``distb(tb)``). |
Nick Coghlan | b39fd0c | 2013-05-06 23:59:20 +1000 | [diff] [blame] | 622 | |
Nick Coghlan | 50c48b8 | 2013-11-23 00:57:00 +1000 | [diff] [blame] | 623 | (Contributed by Nick Coghlan, Ryan Kelly and Thomas Kluyver in :issue:`11816` |
| 624 | and Claudiu Popa in :issue:`17916`) |
Nick Coghlan | b39fd0c | 2013-05-06 23:59:20 +1000 | [diff] [blame] | 625 | |
Antoine Pitrou | d6cbd34 | 2013-08-12 20:48:15 +0200 | [diff] [blame] | 626 | |
R David Murray | 5a9d706 | 2012-11-21 15:09:21 -0500 | [diff] [blame] | 627 | doctest |
| 628 | ------- |
| 629 | |
R David Murray | 473f45b | 2013-12-27 17:01:16 -0500 | [diff] [blame] | 630 | A new :ref:`option flag <doctest-options>`, :data:`~doctest.FAIL_FAST`, halts |
| 631 | test running as soon as the first failure is detected. (Contributed by R. |
| 632 | David Murray and Daniel Urban in :issue:`16522`.) |
R David Murray | 5707d50 | 2013-06-23 14:24:13 -0400 | [diff] [blame] | 633 | |
R David Murray | 473f45b | 2013-12-27 17:01:16 -0500 | [diff] [blame] | 634 | The :mod:`doctest` command line interface now uses :mod:`argparse`, and has two |
| 635 | new options, ``-o`` and ``-f``. ``-o`` allows :ref:`doctest options |
| 636 | <doctest-options>` to be specified on the command line, and ``-f`` is a |
| 637 | shorthand for ``-o FAIL_FAST`` (to parallel the similar option supported by the |
| 638 | :mod:`unittest` CLI). (Contributed by R. David Murray in :issue:`11390`.) |
Georg Brandl | b80f511 | 2012-09-30 09:11:58 +0200 | [diff] [blame] | 639 | |
R David Murray | 8e37d5d | 2013-04-13 14:49:48 -0400 | [diff] [blame] | 640 | |
R David Murray | bb17d2b | 2013-08-09 16:15:28 -0400 | [diff] [blame] | 641 | email |
| 642 | ----- |
| 643 | |
| 644 | :meth:`~email.message.Message.as_string` now accepts a *policy* argument to |
| 645 | override the default policy of the message when generating a string |
| 646 | representation of it. This means that ``as_string`` can now be used in more |
| 647 | circumstances, instead of having to create and use a :mod:`~email.generator` in |
| 648 | order to pass formatting parameters to its ``flatten`` method. |
| 649 | |
| 650 | New method :meth:`~email.message.Message.as_bytes` added to produce a bytes |
| 651 | representation of the message in a fashion similar to how ``as_string`` |
| 652 | produces a string representation. It does not accept the *maxheaderlen* |
| 653 | argument, but does accept the *unixfrom* and *policy* arguments. The |
| 654 | :class:`~email.message.Message` :meth:`~email.message.Message.__bytes__` method |
| 655 | calls it, meaning that ``bytes(mymsg)`` will now produce the intuitive |
| 656 | result: a bytes object containing the fully formatted message. |
| 657 | |
| 658 | (Contributed by R. David Murray in :issue:`18600`.) |
| 659 | |
R David Murray | 26b80cfd | 2013-12-20 17:26:52 -0500 | [diff] [blame] | 660 | .. _whatsnew_email_contentmanager: |
| 661 | |
R David Murray | 3da240f | 2013-10-16 22:48:40 -0400 | [diff] [blame] | 662 | A pair of new subclasses of :class:`~email.message.Message` have been added, |
| 663 | along with a new sub-module, :mod:`~email.contentmanager`. All documentation |
| 664 | is currently in the new module, which is being added as part of the new |
Nick Coghlan | 240f86d | 2013-10-17 23:40:57 +1000 | [diff] [blame] | 665 | :term:`provisional <provisional package>` email API. These classes provide a |
R David Murray | 3da240f | 2013-10-16 22:48:40 -0400 | [diff] [blame] | 666 | number of new methods that make extracting content from and inserting content |
| 667 | into email messages much easier. See the :mod:`~email.contentmanager` |
| 668 | documentation for details. |
| 669 | |
| 670 | These API additions complete the bulk of the work that was planned as part of |
| 671 | the email6 project. The currently provisional API is scheduled to become final |
| 672 | in Python 3.5 (possibly with a few minor additions in the area of error |
| 673 | handling). |
| 674 | |
| 675 | (Contributed by R. David Murray in :issue:`18891`.) |
| 676 | |
R David Murray | bb17d2b | 2013-08-09 16:15:28 -0400 | [diff] [blame] | 677 | |
Victor Stinner | 854ffcb | 2013-06-21 00:36:30 +0200 | [diff] [blame] | 678 | functools |
| 679 | --------- |
| 680 | |
Andrew Kuchling | fe0f0b0 | 2014-01-31 12:17:53 -0500 | [diff] [blame^] | 681 | The new :func:`~functools.partialmethod` descriptor brings partial argument |
Nick Coghlan | f4cb48a | 2013-11-03 16:41:46 +1000 | [diff] [blame] | 682 | application to descriptors, just as :func:`~functools.partial` provides |
| 683 | for normal callables. The new descriptor also makes it easier to get |
| 684 | arbitrary callables (including :func:`~functools.partial` instances) |
| 685 | to behave like normal instance methods when included in a class definition. |
| 686 | |
| 687 | (Contributed by Alon Horev and Nick Coghlan in :issue:`4331`) |
| 688 | |
R David Murray | 0a10216 | 2013-12-20 15:00:54 -0500 | [diff] [blame] | 689 | .. _whatsnew-singledispatch: |
| 690 | |
Nick Coghlan | f4cb48a | 2013-11-03 16:41:46 +1000 | [diff] [blame] | 691 | The new :func:`~functools.singledispatch` decorator brings support for |
| 692 | single-dispatch generic functions to the Python standard library. Where |
| 693 | object oriented programming focuses on grouping multiple operations on a |
| 694 | common set of data into a class, a generic function focuses on grouping |
| 695 | multiple implementations of an operation that allows it to work with |
| 696 | *different* kinds of data. |
| 697 | |
| 698 | .. seealso:: |
| 699 | |
R David Murray | 061efb1 | 2013-12-24 12:35:59 -0500 | [diff] [blame] | 700 | :pep:`443` -- Single-dispatch generic functions |
Nick Coghlan | f4cb48a | 2013-11-03 16:41:46 +1000 | [diff] [blame] | 701 | PEP written and implemented by Łukasz Langa. |
Victor Stinner | 854ffcb | 2013-06-21 00:36:30 +0200 | [diff] [blame] | 702 | |
R David Murray | d265381 | 2013-12-27 14:06:15 -0500 | [diff] [blame] | 703 | A pure-python version of the :func:`~functools.partial` function is now in the |
| 704 | stdlib; in CPython it is overridden by the C accelerated version, but it is |
| 705 | available for other implementations to use. (Contributed by Brian Thorne in |
| 706 | :issue:`12428`.) |
| 707 | |
Nick Coghlan | e8c45d6 | 2013-07-28 20:00:01 +1000 | [diff] [blame] | 708 | |
R David Murray | 0e81463 | 2013-12-26 15:11:28 -0500 | [diff] [blame] | 709 | gc |
| 710 | -- |
| 711 | |
R David Murray | a101bdb | 2014-01-06 16:32:05 -0500 | [diff] [blame] | 712 | New function :func:`~gc.get_stats` returns a list of three per-generation |
| 713 | dictionaries containing the collections statistics since interpreter startup. |
| 714 | (Contributed by Antoine Pitrou in :issue:`16351`.) |
R David Murray | 0e81463 | 2013-12-26 15:11:28 -0500 | [diff] [blame] | 715 | |
| 716 | |
Christian Heimes | e92ef13 | 2013-10-13 00:52:43 +0200 | [diff] [blame] | 717 | hashlib |
| 718 | ------- |
| 719 | |
| 720 | New :func:`hashlib.pbkdf2_hmac` function. |
Christian Heimes | e92ef13 | 2013-10-13 00:52:43 +0200 | [diff] [blame] | 721 | (Contributed by Christian Heimes in :issue:`18582`) |
| 722 | |
| 723 | |
Ezio Melotti | 250a06c | 2013-11-25 06:18:47 +0200 | [diff] [blame] | 724 | html |
| 725 | ---- |
| 726 | |
| 727 | Added a new :func:`html.unescape` function that converts HTML5 character |
| 728 | references to the corresponding Unicode characters. |
Ezio Melotti | 250a06c | 2013-11-25 06:18:47 +0200 | [diff] [blame] | 729 | (Contributed by Ezio Melotti in :issue:`2927`) |
| 730 | |
| 731 | Added a new *convert_charrefs* keyword argument to |
| 732 | :class:`~html.parser.HTMLParser` that, when ``True``, automatically converts |
| 733 | all character references. For backward-compatibility, its value defaults |
| 734 | to ``False``, but it will change to ``True`` in future versions, so you |
| 735 | are invited to set it explicitly and update your code to use this new feature. |
Ezio Melotti | 250a06c | 2013-11-25 06:18:47 +0200 | [diff] [blame] | 736 | (Contributed by Ezio Melotti in :issue:`13633`) |
| 737 | |
| 738 | The *strict* argument of :class:`~html.parser.HTMLParser` is now deprecated. |
Ezio Melotti | 250a06c | 2013-11-25 06:18:47 +0200 | [diff] [blame] | 739 | (Contributed by Ezio Melotti in :issue:`15114`) |
| 740 | |
| 741 | |
R David Murray | a475a8d | 2014-01-03 13:03:00 -0500 | [diff] [blame] | 742 | http |
| 743 | ---- |
| 744 | |
| 745 | :meth:`~http.server.BaseHTTPRequestHandler.send_error` now accepts an |
Zachary Ware | 335957e | 2014-01-13 16:08:54 -0600 | [diff] [blame] | 746 | optional additional *explain* parameter which can be used to provide an |
R David Murray | a475a8d | 2014-01-03 13:03:00 -0500 | [diff] [blame] | 747 | extended error description, overriding the hardcoded default if there is one. |
| 748 | This extended error description will be formatted using the |
| 749 | :attr:`~http.server.HTTP.error_message_format` attribute and sent as the body |
| 750 | of the error response. (Contributed by Karl Cow in :issue:`12921`.) |
| 751 | |
| 752 | |
| 753 | |
R David Murray | d265381 | 2013-12-27 14:06:15 -0500 | [diff] [blame] | 754 | importlib |
| 755 | --------- |
| 756 | |
| 757 | The :class:`~importlib.abc.InspectLoader` ABC defines a new method, |
| 758 | :meth:`~importlib.abc.InspectLoader.source_to_code` that accepts source |
| 759 | data and a path and returns a code object. The default implementation |
| 760 | is equivalent to ``compile(data, path, 'exec', dont_inherit=True)``. |
| 761 | (Contributed by Eric Snow and Brett Cannon in :issue:`15627`.) |
| 762 | |
R David Murray | 0ae7ae1 | 2014-01-08 18:16:02 -0500 | [diff] [blame] | 763 | :class:`~importlib.abc.InspectLoader` also now has a default implementation |
| 764 | for the :meth:`~importlib.abc.InspectLoader.get_code` method. However, |
| 765 | it will normally be desirable to override the default implementation |
| 766 | for performance reasons. (Contributed by Brett Cannon in :issue:`18072`.) |
| 767 | |
R David Murray | d265381 | 2013-12-27 14:06:15 -0500 | [diff] [blame] | 768 | |
Antoine Pitrou | d6cbd34 | 2013-08-12 20:48:15 +0200 | [diff] [blame] | 769 | inspect |
| 770 | ------- |
| 771 | |
Nick Coghlan | f94a16b | 2013-09-22 22:46:49 +1000 | [diff] [blame] | 772 | |
Nick Coghlan | 367df12 | 2013-10-27 01:57:34 +1000 | [diff] [blame] | 773 | The inspect module now offers a basic :ref:`command line interface |
| 774 | <inspect-module-cli>` to quickly display source code and other |
| 775 | information for modules, classes and functions. (Contributed by Claudiu Popa |
| 776 | and Nick Coghlan in :issue:`18626`) |
Nick Coghlan | f94a16b | 2013-09-22 22:46:49 +1000 | [diff] [blame] | 777 | |
Antoine Pitrou | d6cbd34 | 2013-08-12 20:48:15 +0200 | [diff] [blame] | 778 | :func:`~inspect.unwrap` makes it easy to unravel wrapper function chains |
| 779 | created by :func:`functools.wraps` (and any other API that sets the |
Nick Coghlan | 367df12 | 2013-10-27 01:57:34 +1000 | [diff] [blame] | 780 | ``__wrapped__`` attribute on a wrapper function). (Contributed by |
| 781 | Daniel Urban, Aaron Iles and Nick Coghlan in :issue:`13266`) |
| 782 | |
| 783 | As part of the implementation of the new :mod:`enum` module, the |
| 784 | :mod:`inspect` module now has substantially better support for custom |
| 785 | ``__dir__`` methods and dynamic class attributes provided through |
| 786 | metaclasses (Contributed by Ethan Furman in :issue:`18929` and |
| 787 | :issue:`19030`) |
| 788 | |
Yury Selivanov | d82eddc | 2014-01-29 11:24:39 -0500 | [diff] [blame] | 789 | :func:`~inspect.getfullargspec` and :func:`~inspect.getargspec` |
| 790 | now use the :func:`~inspect.signature` API. This allows them to |
| 791 | support much broader range of functions, including some builtins and |
| 792 | callables that follow ``__signature__`` protocol. It is still |
| 793 | recommended to update your code to use :func:`~inspect.signature` |
| 794 | directly. (Contributed by Yury Selivanov in :issue:`17481`) |
| 795 | |
Antoine Pitrou | d6cbd34 | 2013-08-12 20:48:15 +0200 | [diff] [blame] | 796 | |
R David Murray | 9cf617b | 2014-01-04 18:55:01 -0500 | [diff] [blame] | 797 | logging |
| 798 | ------- |
| 799 | |
| 800 | The :class:`~logging.handlers.TimedRotatingFileHandler` has a new *atTime* |
| 801 | parameter that can be used to specify the time of day when rollover should |
| 802 | happen. (Contributed by Ronald Oussoren in :issue:`9556`.) |
| 803 | |
| 804 | |
R David Murray | 8f7664a | 2013-12-22 20:40:11 -0500 | [diff] [blame] | 805 | .. _whatsnew-marshal-3: |
| 806 | |
| 807 | marshal |
| 808 | ------- |
| 809 | |
| 810 | The default :mod:`marshal` version has been bumped to 3. The code implementing |
| 811 | the new version restores the Python2 behavior of recording only one copy of |
| 812 | interned strings and preserving the interning on deserialization, and extends |
| 813 | this "one copy" ability to any object type (including handling recursive |
| 814 | references). This reduces both the size of ``.pyc`` files and the amount of |
| 815 | memory a module occupies in memory when it is loaded from a ``.pyc`` (or |
| 816 | ``.pyo``) file. (Contributed by Kristján Valur Jónsson in :issue:`16475`.) |
| 817 | |
| 818 | |
Antoine Pitrou | f89aa9a | 2013-08-12 20:46:47 +0200 | [diff] [blame] | 819 | mmap |
| 820 | ---- |
| 821 | |
| 822 | mmap objects can now be weakref'ed. |
Antoine Pitrou | f89aa9a | 2013-08-12 20:46:47 +0200 | [diff] [blame] | 823 | (Contributed by Valerie Lambert in :issue:`4885`.) |
| 824 | |
| 825 | |
R David Murray | 33ef1ae | 2013-12-31 16:04:50 -0500 | [diff] [blame] | 826 | mock |
| 827 | ---- |
| 828 | |
| 829 | :mod:`~unittest.mock` objects now inspect their specification signatures when |
| 830 | matching calls, which means an argument can now be matched by either position |
| 831 | or name, instead of only by position. (Contributed by Antoine Pitrou in |
| 832 | :issue:`17015`.) |
| 833 | |
R David Murray | 57fcf76 | 2014-01-03 23:31:54 -0500 | [diff] [blame] | 834 | :func:`~mock.mock_open` objects now have ``readline`` and ``readlines`` |
| 835 | methods. (Contributed by Toshio Kuratomi in :issue:`17467`.) |
| 836 | |
R David Murray | 33ef1ae | 2013-12-31 16:04:50 -0500 | [diff] [blame] | 837 | |
Richard Oudkerk | 84ed9a6 | 2013-08-14 15:35:41 +0100 | [diff] [blame] | 838 | multiprocessing |
| 839 | --------------- |
| 840 | |
R David Murray | ac18622 | 2013-12-20 17:23:57 -0500 | [diff] [blame] | 841 | .. _whatsnew-multiprocessing-no-fork: |
| 842 | |
| 843 | On Unix, two new :ref:`start methods <multiprocessing-start-methods>` |
| 844 | (``spawn`` and ``forkserver``) have been added for starting processes using |
| 845 | :mod:`multiprocessing`. These make the mixing of processes with threads more |
| 846 | robust, and the ``spawn`` method matches the semantics that multiprocessing has |
| 847 | always used on Windows. (Contributed by Richard Oudkerk in :issue:`8713`). |
Richard Oudkerk | 84ed9a6 | 2013-08-14 15:35:41 +0100 | [diff] [blame] | 848 | |
| 849 | Also, except when using the old *fork* start method, child processes |
R David Murray | ac18622 | 2013-12-20 17:23:57 -0500 | [diff] [blame] | 850 | will no longer inherit unneeded handles/file descriptors from their parents |
| 851 | (part of :issue:`8713`). |
Richard Oudkerk | 84ed9a6 | 2013-08-14 15:35:41 +0100 | [diff] [blame] | 852 | |
Nick Coghlan | 9a76735 | 2013-12-17 22:17:26 +1000 | [diff] [blame] | 853 | :mod:`multiprocessing` now relies on :mod:`runpy` (which implements the |
| 854 | ``-m`` switch) to initialise ``__main__`` appropriately in child processes |
| 855 | when using the ``spawn`` or ``forkserver`` start methods. This resolves some |
| 856 | edge cases where combining multiprocessing, the ``-m`` command line switch |
| 857 | and explicit relative imports could cause obscure failures in child |
| 858 | processes. (Contributed by Nick Coghlan in :issue:`19946`) |
| 859 | |
Richard Oudkerk | 84ed9a6 | 2013-08-14 15:35:41 +0100 | [diff] [blame] | 860 | |
R David Murray | 9835827 | 2014-01-05 20:52:06 -0500 | [diff] [blame] | 861 | operator |
| 862 | -------- |
| 863 | |
| 864 | There is now a pure-python version of the :mod:`operator` module available for |
| 865 | reference and for use by alternate implementations of Python. (Contributed by |
| 866 | Zachary Ware in :issue:`16694`.) |
| 867 | |
| 868 | |
Victor Stinner | daf4555 | 2013-08-28 00:53:59 +0200 | [diff] [blame] | 869 | os |
| 870 | -- |
| 871 | |
Georg Brandl | c6ebbef | 2013-09-16 04:03:12 +0200 | [diff] [blame] | 872 | New functions to get and set the :ref:`inheritable flag <fd_inheritance>` of a file |
Victor Stinner | daf4555 | 2013-08-28 00:53:59 +0200 | [diff] [blame] | 873 | descriptors or a Windows handle: |
| 874 | |
| 875 | * :func:`os.get_inheritable`, :func:`os.set_inheritable` |
| 876 | * :func:`os.get_handle_inheritable`, :func:`os.set_handle_inheritable` |
| 877 | |
R David Murray | 66bf12a | 2014-01-08 17:21:22 -0500 | [diff] [blame] | 878 | The :mod:`os` module now provides a :func:`~os.cpu_count` function, analogous to |
| 879 | the :func:`multiprocessing.cpu_count` function (which is now implemented in |
| 880 | terms of the new :mod:`os` function). (Contributed by Trent Nelson, Yogesh |
| 881 | Chaudhari, Victor Stinner, and Charles-François Natali in :issue:`17914`.) |
| 882 | |
Victor Stinner | daf4555 | 2013-08-28 00:53:59 +0200 | [diff] [blame] | 883 | |
R David Murray | 78d692f | 2013-10-10 17:23:26 -0400 | [diff] [blame] | 884 | pdb |
| 885 | --- |
| 886 | |
| 887 | The ``print`` command has been removed from :mod:`pdb`, restoring access to the |
| 888 | ``print`` function. |
| 889 | |
| 890 | Rationale: Python2's ``pdb`` did not have a ``print`` command; instead, |
| 891 | entering ``print`` executed the ``print`` statement. In Python3 ``print`` was |
| 892 | mistakenly made an alias for the pdb :pdbcmd:`p` command. ``p``, however, |
| 893 | prints the ``repr`` of its argument, not the ``str`` like the Python2 ``print`` |
| 894 | command did. Worse, the Python3 ``pdb print`` command shadowed the Python3 |
| 895 | ``print`` function, making it inaccessible at the ``pdb`` prompt. |
| 896 | |
| 897 | (Contributed by Connor Osborn in :issue:`18764`.) |
| 898 | |
| 899 | |
R David Murray | 809487e | 2013-12-22 20:49:40 -0500 | [diff] [blame] | 900 | .. _whatsnew-protocol-4: |
| 901 | |
R David Murray | 5ea9524 | 2013-12-24 15:59:57 -0500 | [diff] [blame] | 902 | pickle |
R David Murray | 809487e | 2013-12-22 20:49:40 -0500 | [diff] [blame] | 903 | ------ |
| 904 | |
| 905 | protocol 4 |
| 906 | |
| 907 | :mod:`pickle` now supports (but does not use by default) a new pickle protocol, |
| 908 | protocol 4. This new protocol addresses a number of issues that were present |
| 909 | in previous protocols, such as the serialization of nested classes, very large |
| 910 | strings and containers, or classes whose :meth:`__new__` method takes |
| 911 | keyword-only arguments. It also provides some efficiency improvements. |
| 912 | |
| 913 | .. seealso:: |
| 914 | |
R David Murray | 061efb1 | 2013-12-24 12:35:59 -0500 | [diff] [blame] | 915 | :pep:`3154` -- Pickle protocol 4 |
R David Murray | 809487e | 2013-12-22 20:49:40 -0500 | [diff] [blame] | 916 | PEP written by Antoine Pitrou and implemented by Alexandre Vassalotti. |
| 917 | |
| 918 | |
Antoine Pitrou | f89aa9a | 2013-08-12 20:46:47 +0200 | [diff] [blame] | 919 | poplib |
| 920 | ------ |
| 921 | |
R David Murray | 473f45b | 2013-12-27 17:01:16 -0500 | [diff] [blame] | 922 | Two new methods have been added to :mod:`poplib`: :meth:`~poplib.POP3.capa`, |
| 923 | which returns the list of capabilities advertised by the POP server, and |
| 924 | :meth:`~poplib.POP3.stls`, which switches a clear-text POP3 session into an |
| 925 | encrypted POP3 session if the POP server supports it. (Contributed by Lorenzo |
| 926 | Catucci in :issue:`4473`.) |
Antoine Pitrou | f89aa9a | 2013-08-12 20:46:47 +0200 | [diff] [blame] | 927 | |
| 928 | |
Serhiy Storchaka | 7c411a4 | 2013-10-02 11:56:18 +0300 | [diff] [blame] | 929 | pprint |
| 930 | ------ |
| 931 | |
Christian Heimes | e1bfd3e | 2013-10-21 12:32:21 +0200 | [diff] [blame] | 932 | The :mod:`pprint` module now supports *compact* mode for formatting long |
Serhiy Storchaka | 7c411a4 | 2013-10-02 11:56:18 +0300 | [diff] [blame] | 933 | sequences (:issue:`19132`). |
| 934 | |
R David Murray | b231b2b | 2014-01-04 17:11:23 -0500 | [diff] [blame] | 935 | Long strings are now wrapped using Python's normal line continuation |
| 936 | syntax (Contributed by Antoine Pitrou in :issue:`17150`.) |
| 937 | |
Serhiy Storchaka | 7c411a4 | 2013-10-02 11:56:18 +0300 | [diff] [blame] | 938 | |
R David Murray | 6e39015 | 2013-12-24 22:28:04 -0500 | [diff] [blame] | 939 | pty |
| 940 | --- |
| 941 | |
| 942 | :func:`pty.spawn` now returns the status value from :func:`os.waitpid` on |
| 943 | the child process, instead of ``None``. (Contributed by Gregory P. Smith.) |
| 944 | |
| 945 | |
Nick Coghlan | 367df12 | 2013-10-27 01:57:34 +1000 | [diff] [blame] | 946 | pydoc |
| 947 | ----- |
| 948 | |
| 949 | While significant changes have not been made to :mod:`pydoc` directly, |
| 950 | its handling of custom ``__dir__`` methods and various descriptor |
| 951 | behaviours has been improved substantially by the underlying changes in |
| 952 | the :mod:`inspect` module. |
| 953 | |
| 954 | |
Serhiy Storchaka | 32eddc1 | 2013-11-23 23:20:30 +0200 | [diff] [blame] | 955 | re |
| 956 | -- |
| 957 | |
| 958 | Added :func:`re.fullmatch` function and :meth:`regex.fullmatch` method, |
| 959 | which anchor the pattern at both ends of the string to match. |
| 960 | (Contributed by Matthew Barnett in :issue:`16203`.) |
| 961 | |
Ezio Melotti | dd7e291 | 2013-11-25 23:20:20 +0200 | [diff] [blame] | 962 | The repr of :ref:`regex objects <re-objects>` now includes the pattern |
| 963 | and the flags; the repr of :ref:`match objects <match-objects>` now |
| 964 | includes the start, end, and the part of the string that matched. |
Ezio Melotti | dd7e291 | 2013-11-25 23:20:20 +0200 | [diff] [blame] | 965 | (Contributed by Serhiy Storchaka in :issue:`13592` and :issue:`17087`.) |
| 966 | |
| 967 | |
Christian Heimes | b7bd5df | 2013-10-22 11:21:54 +0200 | [diff] [blame] | 968 | resource |
| 969 | -------- |
| 970 | |
| 971 | New :func:`resource.prlimit` function and Linux specific constants. |
| 972 | (Contributed by Christian Heimes in :issue:`16595` and :issue:`19324`.) |
| 973 | |
R David Murray | 575fb31 | 2013-12-25 23:21:03 -0500 | [diff] [blame] | 974 | |
R David Murray | 2bc930f | 2013-12-31 11:17:21 -0500 | [diff] [blame] | 975 | select |
| 976 | ------ |
| 977 | |
| 978 | :class:`~select.epoll` objects now support the context management protocol. |
| 979 | When used in a :keyword:`with` statement, the :meth:`~select.epoll.close` |
| 980 | method will be called automatically at the end of the block. (Contributed |
| 981 | by Serhiy Storchaka in :issue:`16488`.) |
| 982 | |
| 983 | |
R David Murray | 575fb31 | 2013-12-25 23:21:03 -0500 | [diff] [blame] | 984 | shelve |
| 985 | ------ |
| 986 | |
| 987 | :class:`~shelve.Shelf` instances may now be used in :keyword:`with` statements, |
| 988 | and will be automatically closed at the end of the :keyword:`with` block. |
| 989 | (Contributed by Filip Gruszczyński in :issue:`13896`.) |
| 990 | |
| 991 | |
R David Murray | f1e4fdc | 2014-01-21 18:30:42 -0500 | [diff] [blame] | 992 | smtpd |
| 993 | ----- |
| 994 | |
| 995 | The :class:`~smtpd.SMTPServer` and :class:`~smtpd.SMTPChannel` classes now |
| 996 | accept a *map* keyword argument, which if specified is passed in to |
| 997 | :class:`asynchat.async_chat` as its *map* argument. This allows an application |
| 998 | to avoid affecting the global socket map. (Contributed by Vinay Sajip in |
| 999 | :issue:`11959`.) |
| 1000 | |
| 1001 | |
R David Murray | 8e37d5d | 2013-04-13 14:49:48 -0400 | [diff] [blame] | 1002 | smtplib |
| 1003 | ------- |
| 1004 | |
R David Murray | 8a34596 | 2013-04-14 06:46:35 -0400 | [diff] [blame] | 1005 | :exc:`~smtplib.SMTPException` is now a subclass of :exc:`OSError`, which allows |
R David Murray | 8e37d5d | 2013-04-13 14:49:48 -0400 | [diff] [blame] | 1006 | both socket level errors and SMTP protocol level errors to be caught in one |
| 1007 | try/except statement by code that only cares whether or not an error occurred. |
| 1008 | (:issue:`2118`). |
| 1009 | |
Antoine Pitrou | d6cbd34 | 2013-08-12 20:48:15 +0200 | [diff] [blame] | 1010 | |
Victor Stinner | daf4555 | 2013-08-28 00:53:59 +0200 | [diff] [blame] | 1011 | socket |
| 1012 | ------ |
| 1013 | |
R David Murray | 33ef1ae | 2013-12-31 16:04:50 -0500 | [diff] [blame] | 1014 | The socket module now supports the :data:`~socket.CAN_BCM` protocol on |
| 1015 | platforms that support it. (Contributed by Brian Thorne in :issue:`15359`.) |
| 1016 | |
Georg Brandl | 5642ff9 | 2013-09-15 10:37:57 +0200 | [diff] [blame] | 1017 | Socket objects have new methods to get or set their :ref:`inheritable flag |
R David Murray | 33ef1ae | 2013-12-31 16:04:50 -0500 | [diff] [blame] | 1018 | <fd_inheritance>`, :meth:`~socket.socket.get_inheritable` and |
| 1019 | :meth:`~socket.socket.set_inheritable`. |
Victor Stinner | daf4555 | 2013-08-28 00:53:59 +0200 | [diff] [blame] | 1020 | |
R David Murray | 33ef1ae | 2013-12-31 16:04:50 -0500 | [diff] [blame] | 1021 | The ``socket.AF_*`` and ``socket.SOCK_*`` constants are now enumeration values |
| 1022 | using the new :mod:`enum` module. This allows meaningful names to be printed |
| 1023 | during debugging, instead of integer "magic numbers". |
Victor Stinner | daf4555 | 2013-08-28 00:53:59 +0200 | [diff] [blame] | 1024 | |
R David Murray | 66bf12a | 2014-01-08 17:21:22 -0500 | [diff] [blame] | 1025 | The :data:`~socket.AF_LINK` constant is now available on BSD and OSX. |
| 1026 | |
R David Murray | 8b2d682 | 2013-12-31 15:06:05 -0500 | [diff] [blame] | 1027 | |
R David Murray | 9025f1c | 2014-01-02 13:44:18 -0500 | [diff] [blame] | 1028 | sqlite3 |
| 1029 | ------- |
| 1030 | |
| 1031 | A new boolean parameter, *uri*, to the :func:`~sqlite3.connect` function can |
| 1032 | be used to indicate that the *database* parameter is a ``uri`` (see |
| 1033 | the `SQLite URI documentation <http://www.sqlite.org/uri.html>`_). |
| 1034 | (Contributed by poq in :issue:`13773`.) |
| 1035 | |
| 1036 | |
Christian Heimes | 24cd4cf | 2013-06-22 19:31:58 +0200 | [diff] [blame] | 1037 | ssl |
| 1038 | --- |
| 1039 | |
R David Murray | 748bad2 | 2013-12-20 17:08:39 -0500 | [diff] [blame] | 1040 | .. _whatsnew-tls-11-12: |
Christian Heimes | 24cd4cf | 2013-06-22 19:31:58 +0200 | [diff] [blame] | 1041 | |
R David Murray | 748bad2 | 2013-12-20 17:08:39 -0500 | [diff] [blame] | 1042 | :data:`~ssl.PROTOCOL_TLSv1_1` and :data:`~ssl.PROTOCOL_TLSv1_2` (TLSv1.1 and |
| 1043 | TLSv1.2 support) have been added; support for these protocols is only available if |
| 1044 | Python is linked with OpenSSL 1.0.1 or later. (Contributed by Michele Orrù and |
| 1045 | Antoine Pitrou in :issue:`16692`) |
Antoine Pitrou | f89aa9a | 2013-08-12 20:46:47 +0200 | [diff] [blame] | 1046 | |
R David Murray | f1e4fdc | 2014-01-21 18:30:42 -0500 | [diff] [blame] | 1047 | New function :func:`~ssl.get_default_verify_paths` returns |
| 1048 | a named tuple of the paths and environment variables that the |
| 1049 | :meth:`~ssl.SSLContext.set_default_verify_paths` method uses to set |
| 1050 | OpenSSL's default ``cafile`` and ``capath``. This can be an aid in |
| 1051 | debugging default verification issues. (Contributed by Christian Heimes |
| 1052 | in :issue:`18143`.) |
| 1053 | |
| 1054 | :class:`~ssl.SSLContext` has a new method, |
| 1055 | :meth:`~ssl.SSLContext.cert_store_stats`, that reports the number of loaded |
| 1056 | ``X.509`` certs, ``X.509 CA`` certs, and certificate revocation lists (``crl``\ |
| 1057 | s), as well as a :meth:`~ssl.SSLContext.get_ca_certs` method that returns a |
| 1058 | list of the loaded ``CA`` certificates. (Contributed by Christian Heimes in |
| 1059 | and :issue:`18147`.) |
Christian Heimes | 24cd4cf | 2013-06-22 19:31:58 +0200 | [diff] [blame] | 1060 | |
R David Murray | 748bad2 | 2013-12-20 17:08:39 -0500 | [diff] [blame] | 1061 | Add :func:`ssl.enum_cert_store` to retrieve certificates and CRL from Windows' |
| 1062 | cert store. (Contributed by Christian Heimes in :issue:`17134`.) |
Antoine Pitrou | f89aa9a | 2013-08-12 20:46:47 +0200 | [diff] [blame] | 1063 | |
| 1064 | Support for server-side SNI using the new |
| 1065 | :meth:`ssl.SSLContext.set_servername_callback` method. |
Antoine Pitrou | f89aa9a | 2013-08-12 20:46:47 +0200 | [diff] [blame] | 1066 | (Contributed by Daniel Black in :issue:`8109`.) |
| 1067 | |
| 1068 | |
Antoine Pitrou | d6cbd34 | 2013-08-12 20:48:15 +0200 | [diff] [blame] | 1069 | stat |
| 1070 | ---- |
| 1071 | |
| 1072 | The :mod:`stat` module is now backed by a C implementation in :mod:`_stat`. A C |
| 1073 | implementation is required as most of the values aren't standardized and |
| 1074 | platform-dependent. (Contributed by Christian Heimes in :issue:`11016`.) |
| 1075 | |
| 1076 | The module supports new file types: door, event port and whiteout. |
| 1077 | |
| 1078 | |
Antoine Pitrou | f89aa9a | 2013-08-12 20:46:47 +0200 | [diff] [blame] | 1079 | struct |
| 1080 | ------ |
| 1081 | |
R David Murray | 9835827 | 2014-01-05 20:52:06 -0500 | [diff] [blame] | 1082 | :mod:`struct` now supports the streamed unpacking of a buffer containing |
| 1083 | repeated instances of a given format of data. Both a module level |
| 1084 | :mod:`~struct.iter_unpack` function and a :meth:`struct.Struct.iter_unpack` |
| 1085 | method on compiled formats have been added. (Contributed by Antoine Pitrou in |
| 1086 | :issue:`17804`.) |
| 1087 | |
| 1088 | |
| 1089 | subprocess |
| 1090 | ---------- |
| 1091 | |
| 1092 | :func:`~subprocess.check_output` now accepts an *input* argument that can |
| 1093 | be used to provide the contents of ``stdin`` for the command that is run. |
| 1094 | (Contributed by Zack Weinberg in :issue:`16624`.) |
Antoine Pitrou | f89aa9a | 2013-08-12 20:46:47 +0200 | [diff] [blame] | 1095 | |
| 1096 | |
Serhiy Storchaka | e06a896 | 2013-09-04 00:43:03 +0300 | [diff] [blame] | 1097 | sunau |
| 1098 | ----- |
| 1099 | |
| 1100 | The :meth:`~sunau.getparams` method now returns a namedtuple rather than a |
| 1101 | plain tuple. (Contributed by Claudiu Popa in :issue:`18901`.) |
| 1102 | |
Serhiy Storchaka | 34d2013 | 2013-09-05 17:01:53 +0300 | [diff] [blame] | 1103 | :meth:`sunau.open` now supports the context manager protocol (:issue:`18878`). |
| 1104 | |
Serhiy Storchaka | e06a896 | 2013-09-04 00:43:03 +0300 | [diff] [blame] | 1105 | |
R David Murray | d17aba7 | 2013-12-24 14:46:23 -0500 | [diff] [blame] | 1106 | sys |
| 1107 | --- |
| 1108 | |
| 1109 | New function :func:`sys.getallocatedblocks` returns the current number of |
| 1110 | blocks allocated by the interpreter (in CPython with the default |
| 1111 | ``--with-pymalloc`` setting, this is allocations made through the |
| 1112 | :c:func:`PyObject_Malloc` API). This can be useful for tracking memory leaks, |
| 1113 | especially if automated via a test suite. (Contributed by Antoine Pitrou |
| 1114 | in :issue:`13390`.) |
| 1115 | |
| 1116 | |
Andrew Kuchling | 173a157 | 2013-09-15 18:15:56 -0400 | [diff] [blame] | 1117 | traceback |
| 1118 | --------- |
| 1119 | |
| 1120 | A new :func:`traceback.clear_frames` function takes a traceback object |
| 1121 | and clears the local variables in all of the frames it references, |
| 1122 | reducing the amount of memory consumed (:issue:`1565525`). |
| 1123 | |
| 1124 | |
Antoine Pitrou | f89aa9a | 2013-08-12 20:46:47 +0200 | [diff] [blame] | 1125 | urllib |
| 1126 | ------ |
| 1127 | |
R David Murray | 473f45b | 2013-12-27 17:01:16 -0500 | [diff] [blame] | 1128 | :mod:`urllib.request` now supports ``data:`` URLs via the |
| 1129 | :class:`~urllib.request.DataHandler` class. (Contributed by Mathias Panzenböck |
| 1130 | in :issue:`16423`.) |
Antoine Pitrou | f89aa9a | 2013-08-12 20:46:47 +0200 | [diff] [blame] | 1131 | |
| 1132 | |
| 1133 | unittest |
| 1134 | -------- |
| 1135 | |
R David Murray | 57fcf76 | 2014-01-03 23:31:54 -0500 | [diff] [blame] | 1136 | The :class:`~unittest.TestCase` class has a new method, |
| 1137 | :meth:`~unittest.TestCase.subTest`, that produces a context manager whose |
| 1138 | :keyword:`with` block becomes a "sub-test". This context manager allows a test |
| 1139 | method to dynamically generate subtests by, say, calling the ``subTest`` |
| 1140 | context manager inside a loop. A single test method can thereby produce an |
| 1141 | indefinite number of separately-identified and separately-counted tests, all of |
| 1142 | which will run even if one or more of them fail. For example:: |
Antoine Pitrou | f89aa9a | 2013-08-12 20:46:47 +0200 | [diff] [blame] | 1143 | |
R David Murray | 57fcf76 | 2014-01-03 23:31:54 -0500 | [diff] [blame] | 1144 | class NumbersTest(unittest.TestCase): |
| 1145 | def test_even(self): |
| 1146 | for i in range(6): |
| 1147 | with self.subTest(i=1): |
| 1148 | self.assertEqual(i % 2, 0) |
| 1149 | |
| 1150 | will result in six subtests, each identified in the unittest verbose output |
| 1151 | with a label consisting of the variable name ``i`` and a particular value for |
| 1152 | that variable (``i=0``, ``i=1``, etc). See :ref:`subtests` for the full |
| 1153 | version of this example. (Contributed by Antoine Pitrou in :issue:`16997`.) |
| 1154 | |
| 1155 | :func:`unittest.main` now accepts an iterable of test names for |
R David Murray | 9025f1c | 2014-01-02 13:44:18 -0500 | [diff] [blame] | 1156 | *defaultTest*, where previously it only accepted a single test name as a |
| 1157 | string. (Contributed by Jyrki Pulliainen in :issue:`15132`.) |
| 1158 | |
R David Murray | 42fa110 | 2014-01-03 13:03:36 -0500 | [diff] [blame] | 1159 | If :class:`~unittest.SkipTest` is raised during test discovery (that is, at the |
| 1160 | module level in the test file), it is now reported as a skip instead of an |
| 1161 | error. (Contributed by Zach Ware in :issue:`16935`.) |
| 1162 | |
R David Murray | db085f5 | 2014-01-03 15:46:24 -0500 | [diff] [blame] | 1163 | :meth:`~unittest.TestLoader.discover` now sorts the discovered files to provide |
| 1164 | consistent test ordering. (Contributed by Martin Melin and Jeff Ramnani in |
| 1165 | :issue:`16709`.) |
| 1166 | |
R David Murray | 8e37d5d | 2013-04-13 14:49:48 -0400 | [diff] [blame] | 1167 | |
R David Murray | 575fb31 | 2013-12-25 23:21:03 -0500 | [diff] [blame] | 1168 | venv |
| 1169 | ---- |
| 1170 | |
| 1171 | :mod:`venv` now includes activation scripts for the ``csh`` and ``fish`` |
| 1172 | shells (Contributed by Andrew Svetlov in :issue:`15417`.) |
| 1173 | |
| 1174 | |
R David Murray | 671cd32 | 2013-04-10 12:31:43 -0400 | [diff] [blame] | 1175 | wave |
| 1176 | ---- |
| 1177 | |
| 1178 | The :meth:`~wave.getparams` method now returns a namedtuple rather than a |
| 1179 | plain tuple. (Contributed by Claudiu Popa in :issue:`17487`.) |
| 1180 | |
R David Murray | c91d5ee | 2013-07-31 13:46:08 -0400 | [diff] [blame] | 1181 | :meth:`wave.open` now supports the context manager protocol. (Contributed |
| 1182 | by Claudiu Popa in :issue:`17616`.) |
| 1183 | |
Antoine Pitrou | f89aa9a | 2013-08-12 20:46:47 +0200 | [diff] [blame] | 1184 | |
Antoine Pitrou | f89aa9a | 2013-08-12 20:46:47 +0200 | [diff] [blame] | 1185 | weakref |
| 1186 | ------- |
| 1187 | |
| 1188 | New :class:`~weakref.WeakMethod` class simulates weak references to bound |
Nick Coghlan | be57ab8 | 2013-09-22 21:26:30 +1000 | [diff] [blame] | 1189 | methods. (Contributed by Antoine Pitrou in :issue:`14631`.) |
Antoine Pitrou | f89aa9a | 2013-08-12 20:46:47 +0200 | [diff] [blame] | 1190 | |
Nick Coghlan | be57ab8 | 2013-09-22 21:26:30 +1000 | [diff] [blame] | 1191 | New :class:`~weakref.finalize` class makes it possible to register a callback |
| 1192 | to be invoked when an object is garbage collected, without needing to |
| 1193 | carefully manage the lifecycle of the weak reference itself. (Contributed by |
| 1194 | Richard Oudkerk in :issue:`15528`) |
Antoine Pitrou | f89aa9a | 2013-08-12 20:46:47 +0200 | [diff] [blame] | 1195 | |
R David Murray | a101bdb | 2014-01-06 16:32:05 -0500 | [diff] [blame] | 1196 | The callback, if any, associated with a :class:`~weakref.ref` is now |
| 1197 | exposed via the :attr:`~weakref.ref.__callback__` attribute. (Contributed |
| 1198 | by Mark Dickinson in :issue:`17643`.) |
| 1199 | |
Antoine Pitrou | f89aa9a | 2013-08-12 20:46:47 +0200 | [diff] [blame] | 1200 | |
| 1201 | xml.etree |
| 1202 | --------- |
| 1203 | |
R David Murray | 410d320 | 2014-01-04 23:52:50 -0500 | [diff] [blame] | 1204 | A new parser, :class:`~xml.etree.ElementTree.XMLPullParser`, allows a |
| 1205 | non-blocking applications to parse XML documents. An example can be |
| 1206 | seen at :ref:`elementtree-pull-parsing`. (Contributed by Antoine |
| 1207 | Pitrou in :issue:`17741`.) |
Antoine Pitrou | f89aa9a | 2013-08-12 20:46:47 +0200 | [diff] [blame] | 1208 | |
R David Murray | 575fb31 | 2013-12-25 23:21:03 -0500 | [diff] [blame] | 1209 | The :mod:`xml.etree.ElementTree` :func:`~xml.etree.ElementTree.tostring` and |
| 1210 | :func:`~xml.etree.ElementTree.tostringlist` functions, and the |
| 1211 | :class:`~xml.etree.ElementTree.ElementTree` |
| 1212 | :meth:`~xml.etree.ElementTree.ElementTree.write` method, now have a |
| 1213 | *short_empty_elements* :ref:`keyword-only parameter <keyword-only_parameter>` |
| 1214 | providing control over whether elements with no content are written in |
| 1215 | abbreviated (``<tag />``) or expanded (``<tag></tag>``) form. (Contributed by |
| 1216 | Ariel Poliak and Serhiy Storchaka in :issue:`14377`.) |
| 1217 | |
Christian Tismer | 59202e5 | 2013-10-21 03:59:23 +0200 | [diff] [blame] | 1218 | |
| 1219 | zipfile.PyZipfile |
| 1220 | ----------------- |
| 1221 | |
| 1222 | Add a filter function to ignore some packages (tests for instance), |
| 1223 | :meth:`~zipfile.PyZipFile.writepy`. |
Christian Tismer | 59202e5 | 2013-10-21 03:59:23 +0200 | [diff] [blame] | 1224 | (Contributed by Christian Tismer in :issue:`19274`.) |
| 1225 | |
| 1226 | |
R David Murray | e608255 | 2014-01-03 16:15:45 -0500 | [diff] [blame] | 1227 | |
Nick Coghlan | 367df12 | 2013-10-27 01:57:34 +1000 | [diff] [blame] | 1228 | CPython Implementation Changes |
| 1229 | ============================== |
| 1230 | |
| 1231 | |
R David Murray | c16dfe1 | 2013-12-21 12:32:10 -0500 | [diff] [blame] | 1232 | .. _whatsnew-pep-445: |
Nick Coghlan | 367df12 | 2013-10-27 01:57:34 +1000 | [diff] [blame] | 1233 | |
R David Murray | e7cf678 | 2013-12-24 14:51:25 -0500 | [diff] [blame] | 1234 | PEP 445: Customization of CPython Memory Allocators |
Nick Coghlan | 367df12 | 2013-10-27 01:57:34 +1000 | [diff] [blame] | 1235 | --------------------------------------------------- |
| 1236 | |
| 1237 | :pep:`445` adds new C level interfaces to customize memory allocation in |
| 1238 | the CPython interpreter. |
| 1239 | |
| 1240 | .. seealso:: |
| 1241 | |
R David Murray | 061efb1 | 2013-12-24 12:35:59 -0500 | [diff] [blame] | 1242 | :pep:`445` -- Add new APIs to customize Python memory allocators |
Nick Coghlan | 367df12 | 2013-10-27 01:57:34 +1000 | [diff] [blame] | 1243 | PEP written and implemented by Victor Stinner. |
| 1244 | |
| 1245 | |
R David Murray | c16dfe1 | 2013-12-21 12:32:10 -0500 | [diff] [blame] | 1246 | .. _whatsnew-pep-442: |
Nick Coghlan | 367df12 | 2013-10-27 01:57:34 +1000 | [diff] [blame] | 1247 | |
R David Murray | e7cf678 | 2013-12-24 14:51:25 -0500 | [diff] [blame] | 1248 | PEP 442: Safe Object Finalization |
Nick Coghlan | 367df12 | 2013-10-27 01:57:34 +1000 | [diff] [blame] | 1249 | --------------------------------- |
| 1250 | |
| 1251 | :pep:`442` removes the current limitations and quirks of object finalization |
| 1252 | in CPython. With it, objects with :meth:`__del__` methods, as well as |
| 1253 | generators with :keyword:`finally` clauses, can be finalized when they are |
| 1254 | part of a reference cycle. |
| 1255 | |
| 1256 | As part of this change, module globals are no longer forcibly set to |
| 1257 | :const:`None` during interpreter shutdown in most cases, instead relying |
R David Murray | ca79461 | 2013-12-22 14:05:11 -0500 | [diff] [blame] | 1258 | on the normal operation of the cyclic garbage collector. This avoids a |
| 1259 | whole class of interpreter-shutdown-time errors, usually involving |
| 1260 | ``__del__`` methods, that have plagued Python since the cyclic GC |
| 1261 | was first introduced. |
Nick Coghlan | 367df12 | 2013-10-27 01:57:34 +1000 | [diff] [blame] | 1262 | |
| 1263 | .. seealso:: |
| 1264 | |
R David Murray | 061efb1 | 2013-12-24 12:35:59 -0500 | [diff] [blame] | 1265 | :pep:`442` -- Safe object finalization |
Nick Coghlan | 367df12 | 2013-10-27 01:57:34 +1000 | [diff] [blame] | 1266 | PEP written and implemented by Antoine Pitrou. |
| 1267 | |
| 1268 | |
R David Murray | c16dfe1 | 2013-12-21 12:32:10 -0500 | [diff] [blame] | 1269 | .. _whatsnew-pep-456: |
| 1270 | |
| 1271 | PEP 456: Secure and Interchangeable Hash Algorithm |
| 1272 | -------------------------------------------------- |
| 1273 | |
| 1274 | :pep:`456` follows up on earlier security fix work done on Python's hash |
| 1275 | algorithm to address certain DOS attacks to which public facing APIs backed by |
| 1276 | dictionary lookups may be subject. (See :issue:`14621` for the start of the |
| 1277 | current round of improvements.) The PEP unifies CPython's hash code to make it |
| 1278 | easier for a packager to substitute a different hash algorithm, and switches |
| 1279 | Python's default implementation to a SipHash implementation on platforms that |
| 1280 | have a 64 bit data type. Any performance differences in comparison with the |
| 1281 | older FNV algorithm are trivial. |
| 1282 | |
| 1283 | The PEP adds additional fields to the :func:`sys.hash_info` struct sequence to |
| 1284 | describe the hash algorithm in use by the currently executing binary. Otherwise, |
| 1285 | the PEP does not alter any existing CPython APIs. |
| 1286 | |
| 1287 | |
R David Murray | 6dd1830 | 2013-12-24 12:23:56 -0500 | [diff] [blame] | 1288 | .. _whatsnew-pep-436: |
| 1289 | |
| 1290 | PEP 436: Argument Clinic |
| 1291 | ------------------------ |
| 1292 | |
| 1293 | "Argument Clinic" (:pep:`436`) is now part of the CPython build process |
| 1294 | and can be used to simplify the process of defining and maintaining |
| 1295 | accurate signatures for builtins and standard library extension modules |
| 1296 | implemented in C. |
| 1297 | |
| 1298 | .. note:: |
| 1299 | The Argument Clinic PEP is not fully up to date with the state of the |
| 1300 | implementation. This has been deemed acceptable by the release manager |
| 1301 | and core development team in this case, as Argument Clinic will not |
| 1302 | be made available as a public API for third party use in Python 3.4. |
| 1303 | |
| 1304 | .. seealso:: |
| 1305 | |
R David Murray | 061efb1 | 2013-12-24 12:35:59 -0500 | [diff] [blame] | 1306 | :pep:`436` -- The Argument Clinic DSL |
R David Murray | 6dd1830 | 2013-12-24 12:23:56 -0500 | [diff] [blame] | 1307 | PEP written and implemented by Larry Hastings. |
| 1308 | |
| 1309 | |
R David Murray | e7cf678 | 2013-12-24 14:51:25 -0500 | [diff] [blame] | 1310 | Other Build and C API Changes |
Nick Coghlan | 367df12 | 2013-10-27 01:57:34 +1000 | [diff] [blame] | 1311 | ----------------------------- |
Georg Brandl | b80f511 | 2012-09-30 09:11:58 +0200 | [diff] [blame] | 1312 | |
Nick Coghlan | 7d270ee | 2013-10-17 22:35:35 +1000 | [diff] [blame] | 1313 | * The new :c:func:`Py_SetStandardStreamEncoding` pre-initialization API |
| 1314 | allows applications embedding the CPython interpreter to reliably force |
| 1315 | a particular encoding and error handler for the standard streams |
| 1316 | (Contributed by Bastien Montagne and Nick Coghlan in :issue:`16129`) |
Georg Brandl | b80f511 | 2012-09-30 09:11:58 +0200 | [diff] [blame] | 1317 | |
Nick Coghlan | 0acceb7 | 2013-10-20 13:22:21 +1000 | [diff] [blame] | 1318 | * Most Python C APIs that don't mutate string arguments are now correctly |
| 1319 | marked as accepting ``const char *`` rather than ``char *`` (Contributed |
| 1320 | by Serhiy Storchaka in :issue:`1772673`). |
| 1321 | |
R David Murray | d91ba20 | 2013-12-24 12:13:44 -0500 | [diff] [blame] | 1322 | * New shell version of ``python-config``; can be used even when a python |
| 1323 | interpreter is not available (for example, in cross compilation scenarios). |
| 1324 | |
Georg Brandl | b80f511 | 2012-09-30 09:11:58 +0200 | [diff] [blame] | 1325 | |
R David Murray | e608255 | 2014-01-03 16:15:45 -0500 | [diff] [blame] | 1326 | |
R David Murray | e7cf678 | 2013-12-24 14:51:25 -0500 | [diff] [blame] | 1327 | Other Improvements |
R David Murray | bcaaecf | 2013-12-23 21:23:36 -0500 | [diff] [blame] | 1328 | ================== |
| 1329 | |
| 1330 | * Tab-completion is now enabled by default in the interactive interpreter. |
| 1331 | (Contributed by Antoine Pitrou and Éric Araujo in :issue:`5845`.) |
| 1332 | |
| 1333 | * Invoking the Python interpreter with ``--version`` now outputs the version to |
| 1334 | standard output instead of standard error (:issue:`18338`). Similar changes |
| 1335 | were made to :mod:`argparse` (:issue:`18920`) and other modules that have |
| 1336 | script-like invocation capabilities (:issue:`18922`). |
| 1337 | |
R David Murray | 72420ff | 2013-12-24 10:46:44 -0500 | [diff] [blame] | 1338 | * The CPython Windows installer now adds ``.py`` to the :envvar:`PATHEXT` |
| 1339 | variable when extensions are registered, allowing users to run a python |
| 1340 | script at the windows command prompt by just typing its name without the |
| 1341 | ``.py`` extension. (Contributed by Paul Moore in :issue:`18569`.) |
| 1342 | |
R David Murray | d17aba7 | 2013-12-24 14:46:23 -0500 | [diff] [blame] | 1343 | * A new ``make`` target `coverage-report |
| 1344 | <http://docs.python.org/devguide/coverage.html#measuring-coverage-of-c-code-with-gcov-and-lcov>`_ |
| 1345 | will build python, run the test suite, and generate an HTML coverage report |
| 1346 | for the C codebase using ``gcov`` and `lcov |
| 1347 | <http://ltp.sourceforge.net/coverage/lcov.php>`_. |
| 1348 | |
| 1349 | * The ``-R`` option to the :ref:`python regression test suite <regrtest>` now |
| 1350 | also checks for memory allocation leaks, using |
| 1351 | :func:`sys.getallocatedblocks()`. (Contributed by Antoine Pitrou in |
| 1352 | :issue:`13390`). |
| 1353 | |
R David Murray | bcaaecf | 2013-12-23 21:23:36 -0500 | [diff] [blame] | 1354 | |
R David Murray | e608255 | 2014-01-03 16:15:45 -0500 | [diff] [blame] | 1355 | |
R David Murray | d74d09a | 2013-12-24 15:33:02 -0500 | [diff] [blame] | 1356 | Significant Optimizations |
| 1357 | ========================= |
R David Murray | bcaaecf | 2013-12-23 21:23:36 -0500 | [diff] [blame] | 1358 | |
R David Murray | 4908f4a | 2014-01-04 18:07:20 -0500 | [diff] [blame] | 1359 | * The UTF-32 decoder is now 3x to 4x faster. (Contributed by Serhiy Storchaka |
| 1360 | in :issue:`14625`.) |
R David Murray | bcaaecf | 2013-12-23 21:23:36 -0500 | [diff] [blame] | 1361 | |
| 1362 | * The cost of hash collisions for sets is now reduced. Each hash table |
| 1363 | probe now checks a series of consecutive, adjacent key/hash pairs before |
| 1364 | continuing to make random probes through the hash table. This exploits |
| 1365 | cache locality to make collision resolution less expensive. |
R David Murray | bcaaecf | 2013-12-23 21:23:36 -0500 | [diff] [blame] | 1366 | The collision resolution scheme can be described as a hybrid of linear |
| 1367 | probing and open addressing. The number of additional linear probes |
| 1368 | defaults to nine. This can be changed at compile-time by defining |
| 1369 | LINEAR_PROBES to be any value. Set LINEAR_PROBES=0 to turn-off |
R David Murray | 5917172 | 2014-01-03 15:52:22 -0500 | [diff] [blame] | 1370 | linear probing entirely. (Contributed by Raymond Hettinger in |
| 1371 | :issue:`18771`.) |
R David Murray | bcaaecf | 2013-12-23 21:23:36 -0500 | [diff] [blame] | 1372 | |
| 1373 | * The interpreter starts about 30% faster. A couple of measures lead to the |
| 1374 | speedup. The interpreter loads fewer modules on startup, e.g. the :mod:`re`, |
| 1375 | :mod:`collections` and :mod:`locale` modules and their dependencies are no |
| 1376 | longer imported by default. The marshal module has been improved to load |
R David Murray | 5917172 | 2014-01-03 15:52:22 -0500 | [diff] [blame] | 1377 | compiled Python code faster. (Contributed by Antoine Pitrou, Christian |
| 1378 | Heimes and Victor Stinner in :issue:`19219`, :issue:`19218`, :issue:`19209`, |
| 1379 | :issue:`19205` and :issue:`9548`) |
R David Murray | bcaaecf | 2013-12-23 21:23:36 -0500 | [diff] [blame] | 1380 | |
R David Murray | 6e39015 | 2013-12-24 22:28:04 -0500 | [diff] [blame] | 1381 | * :class:`bz2.BZ2File` is now as fast or faster than the Python2 version for |
| 1382 | most cases. :class:`lzma.LZMAFile` has also been optimized. (Contributed by |
| 1383 | Serhiy Storchaka and Nadeem Vawda in :issue:`16034`.) |
| 1384 | |
R David Murray | 8b2d682 | 2013-12-31 15:06:05 -0500 | [diff] [blame] | 1385 | * :func:`random.getrandbits` is 20%-40% faster for small integers (the most |
| 1386 | common use case). (Contributed by Serhiy Storchaka in :issue:`16674`). |
| 1387 | |
R David Murray | b231b2b | 2014-01-04 17:11:23 -0500 | [diff] [blame] | 1388 | * By taking advantage of the new storage format for strings, pickling of |
| 1389 | strings is now significantly faster. (Contributed by Victor Stinner and |
| 1390 | Antoine Pitrou in :issue:`15596`.) |
| 1391 | |
R David Murray | 66bf12a | 2014-01-08 17:21:22 -0500 | [diff] [blame] | 1392 | * A performance issue in :meth:`io.FileIO.readall` has been solved. This |
| 1393 | particularly affects Windows, and significantly speeds up the case of piping |
| 1394 | significant amounts of data through :mod:`subprocess`. (Contributed |
| 1395 | by Richard Oudkerk in :issue:`15758`.) |
| 1396 | |
R David Murray | bcaaecf | 2013-12-23 21:23:36 -0500 | [diff] [blame] | 1397 | |
R David Murray | e608255 | 2014-01-03 16:15:45 -0500 | [diff] [blame] | 1398 | |
Georg Brandl | b80f511 | 2012-09-30 09:11:58 +0200 | [diff] [blame] | 1399 | Deprecated |
| 1400 | ========== |
| 1401 | |
R David Murray | d74d09a | 2013-12-24 15:33:02 -0500 | [diff] [blame] | 1402 | This section covers various APIs and other features that have been deprecated |
| 1403 | in Python 3.4, and will be removed in Python 3.5 or later. In most (but not |
| 1404 | all) cases, using the deprecated APIs will produce a :exc:`DeprecationWarning` |
| 1405 | when the interpreter is run with deprecation warnings enabled (for example, by |
| 1406 | using ``-Wd``). |
| 1407 | |
Georg Brandl | b80f511 | 2012-09-30 09:11:58 +0200 | [diff] [blame] | 1408 | |
R David Murray | e7cf678 | 2013-12-24 14:51:25 -0500 | [diff] [blame] | 1409 | Deprecated Python Modules, Functions and Methods |
Georg Brandl | b80f511 | 2012-09-30 09:11:58 +0200 | [diff] [blame] | 1410 | ------------------------------------------------ |
| 1411 | |
Terry Jan Reedy | 2b6c26e | 2013-03-21 19:36:26 -0400 | [diff] [blame] | 1412 | * :meth:`difflib.SequenceMatcher.isbjunk` and |
Andrew Kuchling | 0d0813a | 2013-06-15 13:29:09 -0400 | [diff] [blame] | 1413 | :meth:`difflib.SequenceMatcher.isbpopular` were removed: use ``x in sm.bjunk`` and |
| 1414 | ``x in sm.bpopular``, where *sm* is a :class:`~difflib.SequenceMatcher` object. |
Georg Brandl | b80f511 | 2012-09-30 09:11:58 +0200 | [diff] [blame] | 1415 | |
Brett Cannon | 82b3d6a | 2013-06-14 22:37:11 -0400 | [diff] [blame] | 1416 | * :func:`importlib.util.module_for_loader` is pending deprecation. Using |
| 1417 | :func:`importlib.util.module_to_load` and |
| 1418 | :meth:`importlib.abc.Loader.init_module_attrs` allows subclasses of a loader |
| 1419 | to more easily customize module loading. |
| 1420 | |
Brett Cannon | e4f41de | 2013-06-16 13:13:40 -0400 | [diff] [blame] | 1421 | * The :mod:`imp` module is pending deprecation. To keep compatibility with |
| 1422 | Python 2/3 code bases, the module's removal is currently not scheduled. |
| 1423 | |
Brett Cannon | 1448ecf | 2013-10-04 11:38:59 -0400 | [diff] [blame] | 1424 | * The :mod:`formatter` module is pending deprecation and is slated for removal |
| 1425 | in Python 3.6. |
| 1426 | |
Christian Heimes | 634919a | 2013-11-20 17:23:06 +0100 | [diff] [blame] | 1427 | * MD5 as default digestmod for :mod:`hmac` is deprecated. Python 3.6 will |
| 1428 | require an explicit digest name or constructor as *digestmod* argument. |
| 1429 | |
R David Murray | 9025f1c | 2014-01-02 13:44:18 -0500 | [diff] [blame] | 1430 | * The internal ``Netrc`` class in the :mod:`ftplib` module has been documented |
| 1431 | as deprecated in its docstring for quite some time. It now emits a |
| 1432 | :exc:`DeprecationWarning` and will be removed completely in Python 3.5. |
| 1433 | |
Georg Brandl | b80f511 | 2012-09-30 09:11:58 +0200 | [diff] [blame] | 1434 | |
R David Murray | e7cf678 | 2013-12-24 14:51:25 -0500 | [diff] [blame] | 1435 | Deprecated Functions and Types in the C API |
Georg Brandl | b80f511 | 2012-09-30 09:11:58 +0200 | [diff] [blame] | 1436 | ------------------------------------------- |
| 1437 | |
R David Murray | d74d09a | 2013-12-24 15:33:02 -0500 | [diff] [blame] | 1438 | XXX: None so far |
Georg Brandl | b80f511 | 2012-09-30 09:11:58 +0200 | [diff] [blame] | 1439 | |
| 1440 | |
R David Murray | e7cf678 | 2013-12-24 14:51:25 -0500 | [diff] [blame] | 1441 | Deprecated Features |
Georg Brandl | b80f511 | 2012-09-30 09:11:58 +0200 | [diff] [blame] | 1442 | ------------------- |
| 1443 | |
R David Murray | 5ea9524 | 2013-12-24 15:59:57 -0500 | [diff] [blame] | 1444 | * Running :ref:`idle` with the ``-n`` flag (no subprocess) is deprecated. |
| 1445 | However, the feature will not be removed until :issue:`18823` is resolved. |
| 1446 | |
Antoine Pitrou | 3b2f0f0 | 2013-10-25 21:39:26 +0200 | [diff] [blame] | 1447 | * The site module adding a "site-python" directory to sys.path, if it |
| 1448 | exists, is deprecated (:issue:`19375`). |
Georg Brandl | b80f511 | 2012-09-30 09:11:58 +0200 | [diff] [blame] | 1449 | |
| 1450 | |
R David Murray | 5917172 | 2014-01-03 15:52:22 -0500 | [diff] [blame] | 1451 | |
R David Murray | 72420ff | 2013-12-24 10:46:44 -0500 | [diff] [blame] | 1452 | Removed |
| 1453 | ======= |
| 1454 | |
R David Murray | 5917172 | 2014-01-03 15:52:22 -0500 | [diff] [blame] | 1455 | |
| 1456 | Operating Systems No Longer Supported |
| 1457 | ------------------------------------- |
| 1458 | |
| 1459 | Support for the following operating systems has been removed from the source |
| 1460 | and build tools: |
| 1461 | |
| 1462 | * OS/2 (:issue:`16135`). |
| 1463 | * Windows 2000 (changeset e52df05b496a). |
| 1464 | * VMS (:issue:`16136`). |
| 1465 | |
| 1466 | |
| 1467 | API and Feature Removals |
| 1468 | ------------------------ |
| 1469 | |
R David Murray | d265381 | 2013-12-27 14:06:15 -0500 | [diff] [blame] | 1470 | The following obsolete and previously deprecated APIs and features have been |
R David Murray | 5917172 | 2014-01-03 15:52:22 -0500 | [diff] [blame] | 1471 | removed: |
R David Murray | 076dead | 2013-12-24 14:59:50 -0500 | [diff] [blame] | 1472 | |
R David Murray | 72420ff | 2013-12-24 10:46:44 -0500 | [diff] [blame] | 1473 | * The unmaintained ``Misc/TextMate`` and ``Misc/vim`` directories have been |
R David Murray | d91ba20 | 2013-12-24 12:13:44 -0500 | [diff] [blame] | 1474 | removed (see the `devguide <http://docs.python.org/devguide>`_ |
R David Murray | 0056936 | 2014-01-03 13:04:25 -0500 | [diff] [blame] | 1475 | for suggestions on what to use instead). |
R David Murray | 72420ff | 2013-12-24 10:46:44 -0500 | [diff] [blame] | 1476 | |
R David Murray | d17aba7 | 2013-12-24 14:46:23 -0500 | [diff] [blame] | 1477 | * The ``SO`` makefile macro is removed (it was replaced by the |
| 1478 | ``SHLIB_SUFFIX`` and ``EXT_SUFFIX`` macros) (:issue:`16754`). |
| 1479 | |
R David Murray | d74d09a | 2013-12-24 15:33:02 -0500 | [diff] [blame] | 1480 | * The ``PyThreadState.tick_counter`` field has been removed; its value has |
| 1481 | been meaningless since Python 3.2, when the "new GIL" was introduced. |
| 1482 | |
R David Murray | d265381 | 2013-12-27 14:06:15 -0500 | [diff] [blame] | 1483 | * ``PyLoader`` and ``PyPycLoader`` have been removed from :mod:`importlib`. |
| 1484 | (Contributed by Taras Lyapun in :issue:`15641`.) |
| 1485 | |
R David Murray | a2924ca | 2014-01-03 14:06:01 -0500 | [diff] [blame] | 1486 | * The *strict* argument to :class:`~http.client.HTTPConnection` and |
| 1487 | :class:`~http.client.HTTPSConnection` has been removed. HTTP 0.9-style |
| 1488 | "Simple Responses" are no longer supported. |
R David Murray | 0056936 | 2014-01-03 13:04:25 -0500 | [diff] [blame] | 1489 | |
R David Murray | db085f5 | 2014-01-03 15:46:24 -0500 | [diff] [blame] | 1490 | * The deprecated :mod:`urllib.request.Request` getter and setter methods |
| 1491 | ``add_data``, ``has_data``, ``get_data``, ``get_type``, ``get_host``, |
| 1492 | ``get_selector``, ``set_proxy``, ``get_origin_req_host``, and |
| 1493 | ``is_unverifiable`` have been removed (use direct attribute access instead). |
| 1494 | |
R David Murray | 244ad60 | 2014-01-04 21:17:52 -0500 | [diff] [blame] | 1495 | * Support for loading the deprecated ``TYPE_INT64`` has been removed from |
| 1496 | :mod:`marshal`. (Contributed by Dan Riti in :issue:`15480`.) |
| 1497 | |
Yury Selivanov | 2393dca | 2014-01-27 15:07:58 -0500 | [diff] [blame] | 1498 | * :class:`inspect.Signature`: positional-only parameters are now required |
| 1499 | to have a valid name. |
| 1500 | |
R David Murray | 72420ff | 2013-12-24 10:46:44 -0500 | [diff] [blame] | 1501 | |
R David Murray | 9835827 | 2014-01-05 20:52:06 -0500 | [diff] [blame] | 1502 | Code Cleanups |
| 1503 | ------------- |
| 1504 | |
| 1505 | * The unused and undocumented internal ``Scanner`` class has been removed from |
| 1506 | the :mod:`pydoc` module. |
| 1507 | |
| 1508 | |
R David Murray | e608255 | 2014-01-03 16:15:45 -0500 | [diff] [blame] | 1509 | |
Benjamin Peterson | 88f3b23 | 2012-10-04 12:45:10 -0400 | [diff] [blame] | 1510 | Porting to Python 3.4 |
Georg Brandl | b80f511 | 2012-09-30 09:11:58 +0200 | [diff] [blame] | 1511 | ===================== |
| 1512 | |
| 1513 | This section lists previously described changes and other bugfixes |
| 1514 | that may require changes to your code. |
| 1515 | |
R David Murray | d74d09a | 2013-12-24 15:33:02 -0500 | [diff] [blame] | 1516 | Changes in the Python API |
| 1517 | ------------------------- |
| 1518 | |
Brett Cannon | 777622b | 2013-04-09 17:03:10 -0400 | [diff] [blame] | 1519 | * The ABCs defined in :mod:`importlib.abc` now either raise the appropriate |
| 1520 | exception or return a default value instead of raising |
| 1521 | :exc:`NotImplementedError` blindly. This will only affect code calling |
| 1522 | :func:`super` and falling through all the way to the ABCs. For compatibility, |
| 1523 | catch both :exc:`NotImplementedError` or the appropriate exception as needed. |
Brett Cannon | 4c14b5d | 2013-05-04 13:56:58 -0400 | [diff] [blame] | 1524 | |
| 1525 | * The module type now initializes the :attr:`__package__` and :attr:`__loader__` |
| 1526 | attributes to ``None`` by default. To determine if these attributes were set |
| 1527 | in a backwards-compatible fashion, use e.g. |
Brett Cannon | 3dc48d6 | 2013-05-28 18:35:54 -0400 | [diff] [blame] | 1528 | ``getattr(module, '__loader__', None) is not None``. |
| 1529 | |
| 1530 | * :meth:`importlib.util.module_for_loader` now sets ``__loader__`` and |
| 1531 | ``__package__`` unconditionally to properly support reloading. If this is not |
| 1532 | desired then you will need to set these attributes manually. You can use |
Brett Cannon | 028d512 | 2013-05-31 18:02:11 -0400 | [diff] [blame] | 1533 | :func:`importlib.util.module_to_load` for module management. |
Brett Cannon | 3e0651b | 2013-05-31 23:18:39 -0400 | [diff] [blame] | 1534 | |
| 1535 | * Import now resets relevant attributes (e.g. ``__name__``, ``__loader__``, |
| 1536 | ``__package__``, ``__file__``, ``__cached__``) unconditionally when reloading. |
| 1537 | |
Brett Cannon | 1448ecf | 2013-10-04 11:38:59 -0400 | [diff] [blame] | 1538 | * Frozen packages no longer set ``__path__`` to a list containing the package |
| 1539 | name but an empty list instead. Determing if a module is a package should be |
| 1540 | done using ``hasattr(module, '__path__')``. |
Brett Cannon | 8f5ac51 | 2013-06-12 23:29:18 -0400 | [diff] [blame] | 1541 | |
Brett Cannon | 33915eb | 2013-06-14 18:33:00 -0400 | [diff] [blame] | 1542 | * :func:`py_compile.compile` now raises :exc:`FileExistsError` if the file path |
| 1543 | it would write to is a symlink or a non-regular file. This is to act as a |
| 1544 | warning that import will overwrite those files with a regular file regardless |
| 1545 | of what type of file path they were originally. |
Brett Cannon | f4375ef | 2013-06-16 18:05:54 -0400 | [diff] [blame] | 1546 | |
| 1547 | * :meth:`importlib.abc.SourceLoader.get_source` no longer raises |
| 1548 | :exc:`ImportError` when the source code being loaded triggers a |
| 1549 | :exc:`SyntaxError` or :exc:`UnicodeDecodeError`. As :exc:`ImportError` is |
| 1550 | meant to be raised only when source code cannot be found but it should, it was |
| 1551 | felt to be over-reaching/overloading of that meaning when the source code is |
| 1552 | found but improperly structured. If you were catching ImportError before and |
| 1553 | wish to continue to ignore syntax or decoding issues, catch all three |
Victor Stinner | 84e33c8 | 2013-06-21 00:31:55 +0200 | [diff] [blame] | 1554 | exceptions now. |
Nick Coghlan | 24c05bc | 2013-07-15 21:13:08 +1000 | [diff] [blame] | 1555 | |
| 1556 | * :func:`functools.update_wrapper` and :func:`functools.wraps` now correctly |
Nick Coghlan | 367df12 | 2013-10-27 01:57:34 +1000 | [diff] [blame] | 1557 | set the ``__wrapped__`` attribute to the function being wrapper, even if |
| 1558 | that function also had its ``__wrapped__`` attribute set. This means |
| 1559 | ``__wrapped__`` attributes now correctly link a stack of decorated |
| 1560 | functions rather than every ``__wrapped__`` attribute in the chain |
| 1561 | referring to the innermost function. Introspection libraries that |
| 1562 | assumed the previous behaviour was intentional can use |
| 1563 | :func:`inspect.unwrap` to access the first function in the chain that has |
| 1564 | no ``__wrapped__`` attribute. |
Victor Stinner | 2fe9bac | 2013-10-10 16:18:20 +0200 | [diff] [blame] | 1565 | |
Georg Brandl | 0f5bff2 | 2013-10-19 17:46:38 +0200 | [diff] [blame] | 1566 | * :class:`importlib.machinery.PathFinder` now passes on the current working |
Brett Cannon | 27e27f7 | 2013-10-18 11:39:04 -0400 | [diff] [blame] | 1567 | directory to objects in :data:`sys.path_hooks` for the empty string. This |
| 1568 | results in :data:`sys.path_importer_cache` never containing ``''``, thus |
| 1569 | iterating through :data:`sys.path_importer_cache` based on :data:`sys.path` |
| 1570 | will not find all keys. A module's ``__file__`` when imported in the current |
| 1571 | working directory will also now have an absolute path, including when using |
| 1572 | ``-m`` with the interpreter (this does not influence when the path to a file |
| 1573 | is specified on the command-line). |
Victor Stinner | 2748bc7 | 2013-12-13 10:57:04 +0100 | [diff] [blame] | 1574 | |
R David Murray | a2924ca | 2014-01-03 14:06:01 -0500 | [diff] [blame] | 1575 | * The removal of the *strict* argument to :class:`~http.client.HTTPConnection` |
| 1576 | and :class:`~http.client.HTTPSConnection` changes the meaning of the |
| 1577 | remaining arguments if you are specifying them positionally rather than by |
| 1578 | keyword. If you've been paying attention to deprecation warnings your code |
| 1579 | should already be specifying any additional arguments via keywords. |
| 1580 | |
R David Murray | 4908f4a | 2014-01-04 18:07:20 -0500 | [diff] [blame] | 1581 | * Strings between ``from __future__ import ...`` statements now *always* raise |
| 1582 | a :exc:`SyntaxError`. Previously if there was no leading docstring, an |
| 1583 | interstitial string would sometimes be ignored. This brings CPython into |
| 1584 | compliance with the language spec; Jython and PyPy already were. |
| 1585 | (:issue:`17434`). |
| 1586 | |
R David Murray | a101bdb | 2014-01-06 16:32:05 -0500 | [diff] [blame] | 1587 | * :meth:`ssl.SSLSocket.getpeercert` and :meth:`ssl.SSLSocket.do_handshake` |
| 1588 | now raise an :exc:`OSError` with ``ENOTCONN`` when the ``SSLSocket`` is not |
| 1589 | connected, instead of the previous behavior of raising an |
| 1590 | :exc:`AttributError`. In addition, :meth:`~ssl.SSLSocket.getpeercert` |
| 1591 | will raise a :exc:`ValueError` if the handshake has not yet been done. |
| 1592 | |
R David Murray | 0ae7ae1 | 2014-01-08 18:16:02 -0500 | [diff] [blame] | 1593 | * :func:`base64.b32decode` now raises a :exc:`binascii.Error` when the |
| 1594 | input string contains non-b32-alphabet characters, instead of a |
| 1595 | :exc:`TypeError`. This particular :exc:`TypeError` was missed when the other |
| 1596 | :exc:`TypeError`\ s were converted. (Contributed by Serhiy Storchaka in |
| 1597 | :issue:`18011`.) Note: this change was also inadvertently applied in Python |
| 1598 | 3.3.3. |
| 1599 | |
Brett Cannon | c089f70 | 2014-01-17 11:03:19 -0500 | [diff] [blame] | 1600 | * The :attr:`~cgi.FieldStorage.file` attribute is now automatically closed when |
| 1601 | the creating :class:`cgi.FieldStorage` instance is garbage collected. If you |
| 1602 | were pulling the file object out separately from the :class:`cgi.FieldStorage` |
| 1603 | instance and not keeping the instance alive, then you should either store the |
| 1604 | entire :class:`cgi.FieldStorage` instance or read the contents of the file |
| 1605 | before the :class:`cgi.FieldStorage` instance is garbage collected. |
| 1606 | |
R David Murray | d265381 | 2013-12-27 14:06:15 -0500 | [diff] [blame] | 1607 | |
Victor Stinner | 774b2e0 | 2013-12-13 14:33:01 +0100 | [diff] [blame] | 1608 | Changes in the C API |
| 1609 | -------------------- |
| 1610 | |
| 1611 | * :c:func:`PyErr_SetImportError` now sets :exc:`TypeError` when its **msg** |
| 1612 | argument is not set. Previously only ``NULL`` was returned with no exception |
| 1613 | set. |
| 1614 | |
| 1615 | * The result of the :c:data:`PyOS_ReadlineFunctionPointer` callback must |
| 1616 | now be a string allocated by :c:func:`PyMem_RawMalloc` or |
| 1617 | :c:func:`PyMem_RawRealloc`, or *NULL* if an error occurred, instead of a |
| 1618 | string allocated by :c:func:`PyMem_Malloc` or :c:func:`PyMem_Realloc`. |
| 1619 | |
| 1620 | * :c:func:`PyThread_set_key_value` now always set the value. In Python |
Victor Stinner | 2748bc7 | 2013-12-13 10:57:04 +0100 | [diff] [blame] | 1621 | 3.3, the function did nothing if the key already exists (if the current |
| 1622 | value is a non-NULL pointer). |
| 1623 | |
Victor Stinner | 774b2e0 | 2013-12-13 14:33:01 +0100 | [diff] [blame] | 1624 | * The ``f_tstate`` (thread state) field of the :c:type:`PyFrameObject` |
| 1625 | structure has been removed to fix a bug: see :issue:`14432` for the |
| 1626 | rationale. |