| **************************** |
| What's New In Python 3.5 |
| **************************** |
| |
| :Release: |release| |
| :Date: |today| |
| |
| .. Rules for maintenance: |
| |
| * Anyone can add text to this document. Do not spend very much time |
| on the wording of your changes, because your text will probably |
| get rewritten to some degree. |
| |
| * The maintainer will go through Misc/NEWS periodically and add |
| changes; it's therefore more important to add your changes to |
| Misc/NEWS than to this file. |
| |
| * This is not a complete list of every single change; completeness |
| is the purpose of Misc/NEWS. Some changes I consider too small |
| or esoteric to include. If such a change is added to the text, |
| I'll just remove it. (This is another reason you shouldn't spend |
| too much time on writing your addition.) |
| |
| * If you want to draw your new text to the attention of the |
| maintainer, add 'XXX' to the beginning of the paragraph or |
| section. |
| |
| * It's OK to just add a fragmentary note about a change. For |
| example: "XXX Describe the transmogrify() function added to the |
| socket module." The maintainer will research the change and |
| write the necessary text. |
| |
| * You can comment out your additions if you like, but it's not |
| necessary (especially when a final release is some months away). |
| |
| * Credit the author of a patch or bugfix. Just the name is |
| sufficient; the e-mail address isn't necessary. |
| |
| * It's helpful to add the bug/patch number as a comment: |
| |
| XXX Describe the transmogrify() function added to the socket |
| module. |
| (Contributed by P.Y. Developer in :issue:`12345`.) |
| |
| This saves the maintainer the effort of going through the Mercurial log |
| when researching a change. |
| |
| This article explains the new features in Python 3.5, compared to 3.4. |
| |
| For full details, see the :source:`Misc/NEWS` file. |
| |
| .. note:: Prerelease users should be aware that this document is currently in |
| draft form. It will be updated substantially as Python 3.5 moves towards |
| release, so it's worth checking back even after reading earlier versions. |
| |
| |
| .. seealso:: |
| |
| :pep:`478` - Python 3.5 Release Schedule |
| |
| |
| Summary -- Release highlights |
| ============================= |
| |
| .. This section singles out the most important changes in Python 3.3. |
| Brevity is key. |
| |
| New syntax features: |
| |
| * None yet. |
| |
| New library modules: |
| |
| * None yet. |
| |
| New built-in features: |
| |
| * None yet. |
| |
| Implementation improvements: |
| |
| * When the ``LC_TYPE`` locale is the POSIX locale (``C`` locale), |
| :py:data:`sys.stdin` and :py:data:`sys.stdout` are now using the |
| ``surrogateescape`` error handler, instead of the ``strict`` error handler |
| (:issue:`19977`). |
| |
| Significantly Improved Library Modules: |
| |
| * None yet. |
| |
| Security improvements: |
| |
| * None yet. |
| |
| Please read on for a comprehensive list of user-facing changes. |
| |
| |
| .. PEP-sized items next. |
| |
| .. _pep-4XX: |
| |
| .. PEP 4XX: Virtual Environments |
| .. ============================= |
| |
| |
| .. (Implemented by Foo Bar.) |
| |
| .. .. seealso:: |
| |
| :pep:`4XX` - Python Virtual Environments |
| PEP written by Carl Meyer |
| |
| |
| |
| PEP 471 - os.scandir() function -- a better and faster directory iterator |
| ------------------------------------------------------------------------- |
| |
| :pep:`471` adds a new directory iteration function, :func:`os.scandir`, |
| to the standard library. Additionally, :func:`os.walk` is now |
| implemented using :func:`os.scandir`, which speeds it up by 3-5 times |
| on POSIX systems and by 7-20 times on Windows systems. |
| |
| PEP and implementation written by Ben Hoyt with the help of Victor Stinner. |
| |
| .. seealso:: |
| |
| :pep:`471` -- os.scandir() function -- a better and faster directory |
| iterator |
| |
| |
| PEP 475: Retry system calls failing with EINTR |
| ---------------------------------------------- |
| |
| :pep:`475` adds support for automatic retry of system calls failing with EINTR: |
| this means that user code doesn't have to deal with EINTR or InterruptedError |
| manually, and should make it more robust against asynchronous signal reception. |
| |
| .. seealso:: |
| |
| :pep:`475` -- Retry system calls failing with EINTR |
| |
| |
| PEP 486: Make the Python Launcher aware of virtual environments |
| --------------------------------------------------------------- |
| |
| :pep:`486` makes the Windows launcher (see :pep:`397`) aware of an active |
| virtual environment. When the default interpreter would be used and the |
| ``VIRTUAL_ENV`` environment variable is set, the interpreter in the virtual |
| environment will be used. |
| |
| .. seealso:: |
| |
| :pep:`486` -- Make the Python Launcher aware of virtual environments |
| |
| Other Language Changes |
| ====================== |
| |
| Some smaller changes made to the core Python language are: |
| |
| * Added the ``'namereplace'`` error handlers. The ``'backslashreplace'`` |
| error handlers now works with decoding and translating. |
| (Contributed by Serhiy Storchaka in :issue:`19676` and :issue:`22286`.) |
| |
| |
| |
| New Modules |
| =========== |
| |
| .. module name |
| .. ----------- |
| |
| * None yet. |
| |
| |
| Improved Modules |
| ================ |
| |
| argparse |
| -------- |
| |
| * :class:`~argparse.ArgumentParser` now allows to disable |
| :ref:`abbreviated usage <prefix-matching>` of long options by setting |
| :ref:`allow_abbrev` to ``False``. |
| (Contributed by Jonathan Paugh, Steven Bethard, paul j3 and Daniel Eriksson.) |
| |
| cgi |
| --- |
| |
| * :class:`~cgi.FieldStorage` now supports the context management protocol. |
| (Contributed by Berker Peksag in :issue:`20289`.) |
| |
| code |
| ---- |
| |
| * The :func:`code.InteractiveInterpreter.showtraceback` method now prints |
| the full chained traceback, just like the interactive interpreter. |
| (Contributed by Claudiu Popa in :issue:`17442`.) |
| |
| compileall |
| ---------- |
| |
| * :func:`compileall.compile_dir` and :mod:`compileall`'s command-line interface |
| can now do parallel bytecode compilation. |
| (Contributed by Claudiu Popa in :issue:`16104`.) |
| |
| contextlib |
| ---------- |
| |
| * The new :func:`contextlib.redirect_stderr` context manager(similar to |
| :func:`contextlib.redirect_stdout`) makes it easier for utility scripts to |
| handle inflexible APIs that write their output to :data:`sys.stderr` and |
| don't provide any options to redirect it. |
| (Contributed by Berker Peksag in :issue:`22389`.) |
| |
| distutils |
| --------- |
| |
| * The ``build`` and ``build_ext`` commands now accept a ``-j`` |
| option to enable parallel building of extension modules. |
| (Contributed by Antoine Pitrou in :issue:`5309`.) |
| |
| doctest |
| ------- |
| |
| * :func:`doctest.DocTestSuite` returns an empty :class:`unittest.TestSuite` if |
| *module* contains no docstrings instead of raising :exc:`ValueError`. |
| (Contributed by Glenn Jones in :issue:`15916`.) |
| |
| glob |
| ---- |
| |
| * :func:`~glob.iglob` and :func:`~glob.glob` now support recursive search in |
| subdirectories using the "``**``" pattern. |
| (Contributed by Serhiy Storchaka in :issue:`13968`.) |
| |
| imaplib |
| ------- |
| |
| * :class:`IMAP4` now supports the context management protocol. When used in a |
| :keyword:`with` statement, the IMAP4 ``LOGOUT`` command will be called |
| automatically at the end of the block. (Contributed by Tarek Ziadé and |
| Serhiy Storchaka in :issue:`4972`.) |
| |
| imghdr |
| ------ |
| |
| * :func:`~imghdr.what` now recognizes the `OpenEXR <http://www.openexr.com>`_ |
| format. (Contributed by Martin Vignali and Claudiu Popa in :issue:`20295`.) |
| |
| importlib |
| --------- |
| |
| * :class:`importlib.util.LazyLoader` allows for the lazy loading of modules in |
| applications where startup time is paramount. |
| (Contributed by Brett Cannon in :issue:`17621`.) |
| |
| * :func:`importlib.abc.InspectLoader.source_to_code` is now a |
| static method to make it easier to work with source code in a string. |
| With a module object that you want to initialize you can then use |
| ``exec(code, module.__dict__)`` to execute the code in the module. |
| |
| * :func:`importlib.util.module_from_spec` is now the preferred way to create a |
| new module. Compared to :class:`types.ModuleType`, this new function will set |
| the various import-controlled attributes based on the passed-in spec object. |
| |
| inspect |
| ------- |
| |
| * :class:`inspect.Signature` and :class:`inspect.Parameter` are now |
| picklable and hashable. (Contributed by Yury Selivanov in :issue:`20726` |
| and :issue:`20334`.) |
| |
| * New class method :meth:`inspect.Signature.from_callable`, which makes |
| subclassing of :class:`~inspect.Signature` easier. (Contributed |
| by Yury Selivanov and Eric Snow in :issue:`17373`.) |
| |
| ipaddress |
| --------- |
| |
| * :class:`ipaddress.IPv4Network` and :class:`ipaddress.IPv6Network` now |
| accept an ``(address, netmask)`` tuple argument, so as to easily construct |
| network objects from existing addresses. (Contributed by Peter Moody |
| and Antoine Pitrou in :issue:`16531`.) |
| |
| json |
| ---- |
| |
| * The output of :mod:`json.tool` command line interface is now in the same |
| order as the input. Use the :option:`--sort-keys` option to sort the output |
| of dictionaries alphabetically by key. (Contributed by Berker Peksag in |
| :issue:`21650`.) |
| |
| * JSON decoder now raises :exc:`json.JSONDecodeError` instead of |
| :exc:`ValueError`. (Contributed by Serhiy Storchaka in :issue:`19361`.) |
| |
| os |
| -- |
| |
| * New :func:`os.scandir` function that exposes file information from |
| the operating system when listing a directory. :func:`os.scandir` |
| returns an iterator of :class:`os.DirEntry` objects corresponding to |
| the entries in the directory given by *path*. (Contributed by Ben |
| Hoyt with the help of Victor Stinner in :issue:`22524`.) |
| |
| * :class:`os.stat_result` now has a :attr:`~os.stat_result.st_file_attributes` |
| attribute on Windows. (Contributed by Ben Hoyt in :issue:`21719`.) |
| |
| re |
| -- |
| |
| * Number of capturing groups in regular expression is no longer limited by 100. |
| (Contributed by Serhiy Storchaka in :issue:`22437`.) |
| |
| * Now unmatched groups are replaced with empty strings in :func:`re.sub` |
| and :func:`re.subn`. (Contributed by Serhiy Storchaka in :issue:`1519638`.) |
| |
| math |
| ---- |
| |
| * :data:`math.inf` and :data:`math.nan` constants added. (Contributed by Mark |
| Dickinson in :issue:`23185`.) |
| |
| shutil |
| ------ |
| |
| * :func:`~shutil.move` now accepts a *copy_function* argument, allowing, |
| for example, :func:`~shutil.copy` to be used instead of the default |
| :func:`~shutil.copy2` if there is a need to ignore metadata. (Contributed by |
| Claudiu Popa in :issue:`19840`.) |
| |
| signal |
| ------ |
| |
| * Different constants of :mod:`signal` module are now enumeration values using |
| the :mod:`enum` module. This allows meaningful names to be printed during |
| debugging, instead of integer “magic numbers”. (Contributed by Giampaolo |
| Rodola' in :issue:`21076`.) |
| |
| smtpd |
| ----- |
| |
| * Both :class:`~smtpd.SMTPServer` and :class:`smtpd.SMTPChannel` now accept a |
| *decode_data* keyword to determine if the DATA portion of the SMTP |
| transaction is decoded using the ``utf-8`` codec or is instead provided to |
| :meth:`~smtpd.SMTPServer.process_message` as a byte string. The default |
| is ``True`` for backward compatibility reasons, but will change to ``False`` |
| in Python 3.6. (Contributed by Maciej Szulik in :issue:`19662`.) |
| |
| * It is now possible to provide, directly or via name resolution, IPv6 |
| addresses in the :class:`~smtpd.SMTPServer` constructor, and have it |
| successfully connect. (Contributed by Milan Oberkirch in :issue:`14758`.) |
| |
| * :mod:`~smtpd.SMTPServer` now supports :rfc:`6531` via the *enable_SMTPUTF8* |
| constructor argument and a user-provided |
| :meth:`~smtpd.SMTPServer.process_smtputf8_message` method. |
| |
| smtplib |
| ------- |
| |
| * A new :meth:`~smtplib.SMTP.auth` method provides a convenient way to |
| implement custom authentication mechanisms. |
| (Contributed by Milan Oberkirch in :issue:`15014`.) |
| |
| sndhdr |
| ------ |
| |
| * :func:`~sndhdr.what` and :func:`~sndhdr.whathdr` now return |
| :func:`~collections.namedtuple`. |
| (Contributed by Claudiu Popa in :issue:`18615`.) |
| |
| socket |
| ------ |
| |
| * New :meth:`socket.socket.sendfile` method allows to send a file over a socket |
| by using high-performance :func:`os.sendfile` function on UNIX resulting in |
| uploads being from 2x to 3x faster than when using plain |
| :meth:`socket.socket.send`. |
| (Contributed by Giampaolo Rodola' in :issue:`17552`.) |
| |
| sysconfig |
| --------- |
| |
| * The user scripts directory on Windows is now versioned. |
| (Contributed by Paul Moore in :issue:`23437`.) |
| |
| |
| tarfile |
| ------- |
| |
| * The :func:`tarfile.open` function now supports ``'x'`` (exclusive creation) |
| mode. (Contributed by Berker Peksag in :issue:`21717`.) |
| |
| time |
| ---- |
| |
| * The :func:`time.monotonic` function is now always available. (Contributed by |
| Victor Stinner in :issue:`22043`.) |
| |
| urllib |
| ------ |
| |
| * A new :class:`urllib.request.HTTPBasicPriorAuthHandler` allows HTTP Basic |
| Authentication credentials to be sent unconditionally with the first HTTP |
| request, rather than waiting for a HTTP 401 Unauthorized response from the |
| server. |
| (Contributed by Matej Cepl in :issue:`19494`.) |
| |
| wsgiref |
| ------- |
| |
| * *headers* parameter of :class:`wsgiref.headers.Headers` is now optional. |
| (Contributed by Pablo Torres Navarrete and SilentGhost in :issue:`5800`.) |
| |
| xmlrpc |
| ------ |
| |
| * :class:`xmlrpc.client.ServerProxy` is now a :term:`context manager`. |
| (Contributed by Claudiu Popa in :issue:`20627`.) |
| |
| |
| Optimizations |
| ============= |
| |
| The following performance enhancements have been added: |
| |
| * :func:`os.walk` has been sped up by 3-5x on POSIX systems and 7-20x |
| on Windows. This was done using the new :func:`os.scandir` function, |
| which exposes file information from the underlying ``readdir`` and |
| ``FindFirstFile``/``FindNextFile`` system calls. (Contributed by |
| Ben Hoyt with help from Victor Stinner in :issue:`23605`.) |
| |
| * Construction of ``bytes(int)`` (filled by zero bytes) is faster and use less |
| memory for large objects. ``calloc()`` is used instead of ``malloc()`` to |
| allocate memory for these objects. |
| |
| * Some operations on :class:`~ipaddress.IPv4Network` and |
| :class:`~ipaddress.IPv6Network` have been massively sped up, such as |
| :meth:`~ipaddress.IPv4Network.subnets`, :meth:`~ipaddress.IPv4Network.supernet`, |
| :func:`~ipaddress.summarize_address_range`, :func:`~ipaddress.collapse_addresses`. |
| The speed up can range from 3x to 15x. |
| (:issue:`21486`, :issue:`21487`, :issue:`20826`) |
| |
| * Many operations on :class:`io.BytesIO` are now 50% to 100% faster. |
| (Contributed by Serhiy Storchaka in :issue:`15381` and David Wilson in |
| :issue:`22003`.) |
| |
| * :func:`marshal.dumps` is now faster (65%-85% with versions 3--4, 20-25% with |
| versions 0--2 on typical data, and up to 5x in best cases). |
| (Contributed by Serhiy Storchaka in :issue:`20416` and :issue:`23344`.) |
| |
| |
| Build and C API Changes |
| ======================= |
| |
| Changes to Python's build process and to the C API include: |
| |
| * New ``calloc`` functions: |
| |
| * :c:func:`PyMem_RawCalloc` |
| * :c:func:`PyMem_Calloc` |
| * :c:func:`PyObject_Calloc` |
| * :c:func:`_PyObject_GC_Calloc` |
| |
| |
| Deprecated |
| ========== |
| |
| Unsupported Operating Systems |
| ----------------------------- |
| |
| * None yet. |
| |
| |
| Deprecated Python modules, functions and methods |
| ------------------------------------------------ |
| |
| * The :mod:`formatter` module has now graduated to full deprecation and is still |
| slated for removal in Python 3.6. |
| |
| * :mod:`smtpd` has in the past always decoded the DATA portion of email |
| messages using the ``utf-8`` codec. This can now be controlled by the new |
| *decode_data* keyword to :class:`~smtpd.SMTPServer`. The default value is |
| ``True``, but this default is deprecated. Specify the *decode_data* keyword |
| with an appropriate value to avoid the deprecation warning. |
| |
| |
| Deprecated functions and types of the C API |
| ------------------------------------------- |
| |
| * None yet. |
| |
| |
| Deprecated features |
| ------------------- |
| |
| * None yet. |
| |
| |
| Removed |
| ======= |
| |
| API and Feature Removals |
| ------------------------ |
| |
| The following obsolete and previously deprecated APIs and features have been |
| removed: |
| |
| * The ``__version__`` attribute has been dropped from the email package. The |
| email code hasn't been shipped separately from the stdlib for a long time, |
| and the ``__version__`` string was not updated in the last few releases. |
| |
| * The internal ``Netrc`` class in the :mod:`ftplib` module was deprecated in |
| 3.4, and has now been removed. |
| (Contributed by Matt Chaput in :issue:`6623`.) |
| |
| Porting to Python 3.5 |
| ===================== |
| |
| This section lists previously described changes and other bugfixes |
| that may require changes to your code. |
| |
| Changes in the Python API |
| ------------------------- |
| |
| * Before Python 3.5, a :class:`datetime.time` object was considered to be false |
| if it represented midnight in UTC. This behavior was considered obscure and |
| error-prone and has been removed in Python 3.5. See :issue:`13936` for full |
| details. |
| |
| * :meth:`ssl.SSLSocket.send()` now raises either :exc:`ssl.SSLWantReadError` |
| or :exc:`ssl.SSLWantWriteError` on a non-blocking socket if the operation |
| would block. Previously, it would return 0. See :issue:`20951`. |
| |
| * The ``__name__`` attribute of generator is now set from the function name, |
| instead of being set from the code name. Use ``gen.gi_code.co_name`` to |
| retrieve the code name. Generators also have a new ``__qualname__`` |
| attribute, the qualified name, which is now used for the representation |
| of a generator (``repr(gen)``). See :issue:`21205`. |
| |
| * The deprecated "strict" mode and argument of :class:`~html.parser.HTMLParser`, |
| :meth:`HTMLParser.error`, and the :exc:`HTMLParserError` exception have been |
| removed. (Contributed by Ezio Melotti in :issue:`15114`.) |
| The *convert_charrefs* argument of :class:`~html.parser.HTMLParser` is |
| now ``True`` by default. (Contributed by Berker Peksag in :issue:`21047`.) |
| |
| * Although it is not formally part of the API, it is worth noting for porting |
| purposes (ie: fixing tests) that error messages that were previously of the |
| form "'sometype' does not support the buffer protocol" are now of the form "a |
| bytes-like object is required, not 'sometype'". (Contributed by Ezio Melotti |
| in :issue:`16518`.) |
| |
| * If the current directory is set to a directory that no longer exists then |
| :exc:`FileNotFoundError` will no longer be raised and instead |
| :meth:`~importlib.machinery.FileFinder.find_spec` will return ``None`` |
| **without** caching ``None`` in :data:`sys.path_importer_cache` which is |
| different than the typical case (:issue:`22834`). |
| |
| * HTTP status code and messages from :mod:`http.client` and :mod:`http.server` |
| were refactored into a common :class:`~http.HTTPStatus` enum. The values in |
| :mod:`http.client` and :mod:`http.server` remain available for backwards |
| compatibility. (Contributed by Demian Brecht in :issue:`21793`.) |
| |
| * When an import loader defines :meth:`~importlib.machinery.Loader.exec_module` |
| it is now expected to also define |
| :meth:`~importlib.machinery.Loader.create_module` (raises a |
| :exc:`DeprecationWarning` now, will be an error in Python 3.6). If the loader |
| inherits from :class:`importlib.abc.Loader` then there is nothing to do, else |
| simply define :meth:`~importlib.machinery.Loader.create_module` to return |
| ``None`` (:issue:`23014`). |
| |
| * :func:`re.split` always ignored empty pattern matches, so the ``'x*'`` |
| pattern worked the same as ``'x+'``, and the ``'\b'`` pattern never worked. |
| Now :func:`re.split` raises a warning if the pattern could match |
| an empty string. For compatibility use patterns that never match an empty |
| string (e.g. ``'x+'`` instead of ``'x*'``). Patterns that could only match |
| an empty string (such as ``'\b'``) now raise an error. |
| |
| Changes in the C API |
| -------------------- |
| |
| * The undocumented :c:member:`~PyMemoryViewObject.format` member of the |
| (non-public) :c:type:`PyMemoryViewObject` structure has been removed. |
| |
| All extensions relying on the relevant parts in ``memoryobject.h`` |
| must be rebuilt. |
| |
| * The :c:type:`PyMemAllocator` structure was renamed to |
| :c:type:`PyMemAllocatorEx` and a new ``calloc`` field was added. |
| |
| * Removed non-documented macro :c:macro:`PyObject_REPR` which leaked references. |
| Use format character ``%R`` in :c:func:`PyUnicode_FromFormat`-like functions |
| to format the :func:`repr` of the object. |
| |
| * Because the lack of the :attr:`__module__` attribute breaks pickling and |
| introspection, a deprecation warning now is raised for builtin type without |
| the :attr:`__module__` attribute. Would be an AttributeError in future. |
| (:issue:`20204`) |