| **************************** |
| What's New In Python 3.7 |
| **************************** |
| |
| :Release: |release| |
| :Date: |today| |
| |
| .. Rules for maintenance: |
| |
| * Anyone can add text to this document. Do not spend very much time |
| on the wording of your changes, because your text will probably |
| get rewritten to some degree. |
| |
| * The maintainer will go through Misc/NEWS periodically and add |
| changes; it's therefore more important to add your changes to |
| Misc/NEWS than to this file. |
| |
| * This is not a complete list of every single change; completeness |
| is the purpose of Misc/NEWS. Some changes I consider too small |
| or esoteric to include. If such a change is added to the text, |
| I'll just remove it. (This is another reason you shouldn't spend |
| too much time on writing your addition.) |
| |
| * If you want to draw your new text to the attention of the |
| maintainer, add 'XXX' to the beginning of the paragraph or |
| section. |
| |
| * It's OK to just add a fragmentary note about a change. For |
| example: "XXX Describe the transmogrify() function added to the |
| socket module." The maintainer will research the change and |
| write the necessary text. |
| |
| * You can comment out your additions if you like, but it's not |
| necessary (especially when a final release is some months away). |
| |
| * Credit the author of a patch or bugfix. Just the name is |
| sufficient; the e-mail address isn't necessary. |
| |
| * It's helpful to add the bug/patch number as a comment: |
| |
| XXX Describe the transmogrify() function added to the socket |
| module. |
| (Contributed by P.Y. Developer in :issue:`12345`.) |
| |
| This saves the maintainer the effort of going through the Mercurial log |
| when researching a change. |
| |
| This article explains the new features in Python 3.7, compared to 3.6. |
| |
| For full details, see the :ref:`changelog <changelog>`. |
| |
| .. note:: |
| |
| Prerelease users should be aware that this document is currently in draft |
| form. It will be updated substantially as Python 3.7 moves towards release, |
| so it's worth checking back even after reading earlier versions. |
| |
| |
| Summary -- Release highlights |
| ============================= |
| |
| .. This section singles out the most important changes in Python 3.7. |
| Brevity is key. |
| |
| |
| .. PEP-sized items next. |
| |
| |
| |
| New Features |
| ============ |
| |
| .. _whatsnew37-pep538: |
| |
| PEP 538: Legacy C Locale Coercion |
| --------------------------------- |
| |
| An ongoing challenge within the Python 3 series has been determining a sensible |
| default strategy for handling the "7-bit ASCII" text encoding assumption |
| currently implied by the use of the default C locale on non-Windows platforms. |
| |
| :pep:`538` updates the default interpreter command line interface to |
| automatically coerce that locale to an available UTF-8 based locale as |
| described in the documentation of the new :envvar:`PYTHONCOERCECLOCALE` |
| environment variable. Automatically setting ``LC_CTYPE`` this way means that |
| both the core interpreter and locale-aware C extensions (such as |
| :mod:`readline`) will assume the use of UTF-8 as the default text encoding, |
| rather than ASCII. |
| |
| The platform support definition in :pep:`11` has also been updated to limit |
| full text handling support to suitably configured non-ASCII based locales. |
| |
| As part of this change, the default error handler for ``stdin`` and ``stdout`` |
| is now ``surrogateescape`` (rather than ``strict``) when using any of the |
| defined coercion target locales (currently ``C.UTF-8``, ``C.utf8``, and |
| ``UTF-8``). The default error handler for ``stderr`` continues to be |
| ``backslashreplace``, regardless of locale. |
| |
| Locale coercion is silent by default, but to assist in debugging potentially |
| locale related integration problems, explicit warnings (emitted directly on |
| ``stderr`` can be requested by setting ``PYTHONCOERCECLOCALE=warn``. This |
| setting will also cause the Python runtime to emit a warning if the legacy C |
| locale remains active when the core interpreter is initialized. |
| |
| .. seealso:: |
| |
| :pep:`538` -- Coercing the legacy C locale to a UTF-8 based locale |
| PEP written and implemented by Nick Coghlan. |
| |
| .. _whatsnew37-pep553: |
| |
| PEP 553: Built-in breakpoint() |
| ------------------------------ |
| |
| :pep:`553` describes a new built-in called ``breakpoint()`` which makes it |
| easy and consistent to enter the Python debugger. Built-in ``breakpoint()`` |
| calls ``sys.breakpointhook()``. By default, this latter imports ``pdb`` and |
| then calls ``pdb.set_trace()``, but by binding ``sys.breakpointhook()`` to the |
| function of your choosing, ``breakpoint()`` can enter any debugger. Or, the |
| environment variable :envvar:`PYTHONBREAKPOINT` can be set to the callable of |
| your debugger of choice. Set ``PYTHONBREAKPOINT=0`` to completely disable |
| built-in ``breakpoint()``. |
| |
| .. seealso:: |
| |
| :pep:`553` -- Built-in breakpoint() |
| PEP written and implemented by Barry Warsaw |
| |
| |
| .. _whatsnew37-pep539: |
| |
| PEP 539: A New C-API for Thread-Local Storage in CPython |
| -------------------------------------------------------- |
| |
| While Python provides a C API for thread-local storage support; the existing |
| :ref:`Thread Local Storage (TLS) API <thread-local-storage-api>` has used |
| :c:type:`int` to represent TLS keys across all platforms. This has not |
| generally been a problem for officially-support platforms, but that is neither |
| POSIX-compliant, nor portable in any practical sense. |
| |
| :pep:`539` changes this by providing a new :ref:`Thread Specific Storage (TSS) |
| API <thread-specific-storage-api>` to CPython which supersedes use of the |
| existing TLS API within the CPython interpreter, while deprecating the existing |
| API. The TSS API uses a new type :c:type:`Py_tss_t` instead of :c:type:`int` |
| to represent TSS keys--an opaque type the definition of which may depend on |
| the underlying TLS implementation. Therefore, this will allow to build CPython |
| on platforms where the native TLS key is defined in a way that cannot be safely |
| cast to :c:type:`int`. |
| |
| Note that on platforms where the native TLS key is defined in a way that cannot |
| be safely cast to :c:type:`int`, all functions of the existing TLS API will be |
| no-op and immediately return failure. This indicates clearly that the old API |
| is not supported on platforms where it cannot be used reliably, and that no |
| effort will be made to add such support. |
| |
| .. seealso:: |
| |
| :pep:`539` -- A New C-API for Thread-Local Storage in CPython |
| PEP written by Erik M. Bray; implementation by Masayuki Yamamoto. |
| |
| |
| PEP 562: Customization of access to module attributes |
| ----------------------------------------------------- |
| |
| It is sometimes convenient to customize or otherwise have control over access |
| to module attributes. A typical example is managing deprecation warnings. |
| Typical workarounds are assigning ``__class__`` of a module object to |
| a custom subclass of :class:`types.ModuleType` or replacing the ``sys.modules`` |
| item with a custom wrapper instance. This procedure is now simplified by |
| recognizing ``__getattr__`` defined directly in a module that would act like |
| a normal ``__getattr__`` method, except that it will be defined on module |
| *instances*. |
| |
| .. seealso:: |
| |
| :pep:`562` -- Module ``__getattr__`` and ``__dir__`` |
| PEP written and implemented by Ivan Levkivskyi |
| |
| |
| PEP 564: Add new time functions with nanosecond resolution |
| ---------------------------------------------------------- |
| |
| Add six new "nanosecond" variants of existing functions to the :mod:`time` |
| module: |
| |
| * :func:`time.clock_gettime_ns` |
| * :func:`time.clock_settime_ns` |
| * :func:`time.monotonic_ns` |
| * :func:`time.perf_counter_ns` |
| * :func:`time.process_time_ns` |
| * :func:`time.time_ns` |
| |
| While similar to the existing functions without the ``_ns`` suffix, they |
| provide nanosecond resolution: they return a number of nanoseconds as a Python |
| ``int``. |
| |
| The ``time.time_ns()`` resolution is 3 times better than the ``time.time()`` |
| resolution on Linux and Windows. |
| |
| .. seealso:: |
| |
| :pep:`564` -- Add new time functions with nanosecond resolution |
| PEP written and implemented by Victor Stinner |
| |
| |
| PEP 540: Add a new UTF-8 mode |
| ----------------------------- |
| |
| Add a new UTF-8 mode to ignore the locale, use the UTF-8 encoding, and change |
| :data:`sys.stdin` and :data:`sys.stdout` error handlers to ``surrogateescape``. |
| This mode is enabled by default in the POSIX locale, but otherwise disabled by |
| default. |
| |
| The new :option:`-X` ``utf8`` command line option and :envvar:`PYTHONUTF8` |
| environment variable are added to control the UTF-8 mode. |
| |
| .. seealso:: |
| |
| :pep:`540` -- Add a new UTF-8 mode |
| PEP written and implemented by Victor Stinner |
| |
| |
| New Development Mode: -X dev |
| ---------------------------- |
| |
| Add a new "development mode": :option:`-X` ``dev`` command line option and |
| :envvar:`PYTHONDEVMODE` environment variable to enable CPython's "development |
| mode", introducing additional runtime checks which are too expensive to be |
| enabled by default. See :option:`-X` ``dev`` documentation for the effects of |
| the development mode. |
| |
| Hash-based pycs |
| --------------- |
| |
| Python has traditionally checked the up-to-dateness of bytecode cache files |
| (i.e., ``.pyc`` files) by comparing the source metadata (last-modified timestamp |
| and size) with source metadata saved in the cache file header when it was |
| generated. While effective, this invalidation method has its drawbacks. When |
| filesystem timestamps are too coarse, Python can miss source updates, leading to |
| user confusion. Additionally, having a timestamp in the cache file is |
| problematic for `build reproduciblity <https://reproducible-builds.org/>`_ and |
| content-based build systems. |
| |
| :pep:`552` extends the pyc format to allow the hash of the source file to be |
| used for invalidation instead of the source timestamp. Such ``.pyc`` files are |
| called "hash-based". By default, Python still uses timestamp-based invalidation |
| and does not generate hash-based ``.pyc`` files at runtime. Hash-based ``.pyc`` |
| files may be generated with :mod:`py_compile` or :mod:`compileall`. |
| |
| Hash-based ``.pyc`` files come in two variants: checked and unchecked. Python |
| validates checked hash-based ``.pyc`` files against the corresponding source |
| files at runtime but doesn't do so for unchecked hash-based pycs. Unchecked |
| hash-based ``.pyc`` files are a useful performance optimization for environments |
| where a system external to Python (e.g., the build system) is responsible for |
| keeping ``.pyc`` files up-to-date. |
| |
| See :ref:`pyc-invalidation` for more information. |
| |
| |
| Other Language Changes |
| ====================== |
| |
| * More than 255 arguments can now be passed to a function, and a function can |
| now have more than 255 parameters. (Contributed by Serhiy Storchaka in |
| :issue:`12844` and :issue:`18896`.) |
| |
| * :meth:`bytes.fromhex` and :meth:`bytearray.fromhex` now ignore all ASCII |
| whitespace, not only spaces. (Contributed by Robert Xiao in :issue:`28927`.) |
| |
| * :exc:`ImportError` now displays module name and module ``__file__`` path when |
| ``from ... import ...`` fails. (Contributed by Matthias Bussonnier in :issue:`29546`.) |
| |
| * Circular imports involving absolute imports with binding a submodule to |
| a name are now supported. |
| (Contributed by Serhiy Storchaka in :issue:`30024`.) |
| |
| * ``object.__format__(x, '')`` is now equivalent to ``str(x)`` rather than |
| ``format(str(self), '')``. |
| (Contributed by Serhiy Storchaka in :issue:`28974`.) |
| |
| |
| New Modules |
| =========== |
| |
| * None yet. |
| |
| |
| Improved Modules |
| ================ |
| |
| |
| argparse |
| -------- |
| |
| The :meth:`~argparse.ArgumentParser.parse_intermixed_args` supports letting |
| the user intermix options and positional arguments on the command line, |
| as is possible in many unix commands. It supports most but not all |
| argparse features. (Contributed by paul.j3 in :issue:`14191`.) |
| |
| |
| binascii |
| -------- |
| |
| The :func:`~binascii.b2a_uu` function now accepts an optional *backtick* |
| keyword argument. When it's true, zeros are represented by ``'`'`` |
| instead of spaces. (Contributed by Xiang Zhang in :issue:`30103`.) |
| |
| |
| calendar |
| -------- |
| |
| The class :class:`~calendar.HTMLCalendar` has new class attributes which ease |
| the customisation of the CSS classes in the produced HTML calendar. |
| (Contributed by Oz Tiram in :issue:`30095`.) |
| |
| cgi |
| --- |
| |
| :func:`~cgi.parse_multipart` returns the same results as |
| :class:`~FieldStorage` : for non-file fields, the value associated to a key |
| is a list of strings, not bytes. |
| (Contributed by Pierre Quentel in :issue:`29979`.) |
| |
| contextlib |
| ---------- |
| |
| :func:`~contextlib.asynccontextmanager` and |
| :class:`~contextlib.AbstractAsyncContextManager` have been added. (Contributed |
| by Jelle Zijlstra in :issue:`29679` and :issue:`30241`.) |
| |
| cProfile |
| -------- |
| |
| cProfile command line now accepts `-m module_name` as an alternative to |
| script path. (Contributed by Sanyam Khurana in :issue:`21862`.) |
| |
| crypt |
| ----- |
| |
| Added support for the Blowfish method. |
| (Contributed by Serhiy Storchaka in :issue:`31664`.) |
| |
| The :func:`~crypt.mksalt` function now allows to specify the number of rounds |
| for hashing. (Contributed by Serhiy Storchaka in :issue:`31702`.) |
| |
| dis |
| --- |
| |
| The :func:`~dis.dis` function now is able to |
| disassemble nested code objects (the code of comprehensions, generator |
| expressions and nested functions, and the code used for building nested |
| classes). (Contributed by Serhiy Storchaka in :issue:`11822`.) |
| |
| distutils |
| --------- |
| |
| README.rst is now included in the list of distutils standard READMEs and |
| therefore included in source distributions. |
| (Contributed by Ryan Gonzalez in :issue:`11913`.) |
| |
| :class:`distutils.core.setup` now warns if the ``classifiers``, ``keywords`` |
| and ``platforms`` fields are not specified as a list or a string. |
| (Contributed by Berker Peksag in :issue:`19610`.) |
| |
| http.client |
| ----------- |
| |
| Add Configurable *blocksize* to ``HTTPConnection`` and |
| ``HTTPSConnection`` for improved upload throughput. |
| (Contributed by Nir Soffer in :issue:`31945`.) |
| |
| http.server |
| ----------- |
| |
| :class:`~http.server.SimpleHTTPRequestHandler` supports the HTTP |
| If-Modified-Since header. The server returns the 304 response status if the |
| target file was not modified after the time specified in the header. |
| (Contributed by Pierre Quentel in :issue:`29654`.) |
| |
| Add the parameter ``directory`` to the :class:`~http.server.SimpleHTTPRequestHandler` |
| and the ``--directory`` to the command line of the module :mod:`~http.server`. |
| With this parameter, the server serves the specified directory, by default it uses the current working directory. |
| (Contributed by Stéphane Wirtel and Julien Palard in :issue:`28707`.) |
| |
| locale |
| ------ |
| |
| Added another argument *monetary* in :meth:`format_string` of :mod:`locale`. |
| If *monetary* is true, the conversion uses monetary thousands separator and |
| grouping strings. (Contributed by Garvit in :issue:`10379`.) |
| |
| The :func:`locale.getpreferredencoding` function now always returns ``'UTF-8'`` |
| on Android or in the UTF-8 mode (:option:`-X` ``utf8`` option), the locale and |
| the *do_setlocale* argument are ignored. |
| |
| math |
| ---- |
| |
| New :func:`~math.remainder` function, implementing the IEEE 754-style remainder |
| operation. (Contributed by Mark Dickinson in :issue:`29962`.) |
| |
| os |
| -- |
| |
| Added support for :class:`bytes` paths in :func:`~os.fwalk`. (Contributed by |
| Serhiy Storchaka in :issue:`28682`.) |
| |
| Added support for :ref:`file descriptors <path_fd>` in :func:`~os.scandir` |
| on Unix. (Contributed by Serhiy Storchaka in :issue:`25996`.) |
| |
| New function :func:`os.register_at_fork` allows registering Python callbacks |
| to be executed on a process fork. (Contributed by Antoine Pitrou in |
| :issue:`16500`.) |
| |
| pdb |
| --- |
| |
| :func:`~pdb.set_trace` now takes an optional *header* keyword-only |
| argument. If given, this is printed to the console just before debugging |
| begins. (Contributed by Barry Warsaw in :issue:`31389`.) |
| |
| re |
| -- |
| |
| The flags :const:`re.ASCII`, :const:`re.LOCALE` and :const:`re.UNICODE` |
| can be set within the scope of a group. |
| (Contributed by Serhiy Storchaka in :issue:`31690`.) |
| |
| :func:`re.split` now supports splitting on a pattern like ``r'\b'``, |
| ``'^$'`` or ``(?=-)`` that matches an empty string. |
| (Contributed by Serhiy Storchaka in :issue:`25054`.) |
| |
| string |
| ------ |
| |
| :class:`string.Template` now lets you to optionally modify the regular |
| expression pattern for braced placeholders and non-braced placeholders |
| separately. (Contributed by Barry Warsaw in :issue:`1198569`.) |
| |
| subprocess |
| ---------- |
| |
| On Windows the default for *close_fds* was changed from :const:`False` to |
| :const:`True` when redirecting the standard handles. It's now possible to set |
| *close_fds* to :const:`True` when redirecting the standard handles. See |
| :class:`subprocess.Popen`. |
| |
| This means that *close_fds* now defaults to :const:`True` on all supported |
| platforms. |
| |
| sys |
| --- |
| |
| Added :attr:`sys.flags.dev_mode` flag for the new development mode. |
| |
| time |
| ---- |
| |
| The :pep:`564` added six new functions with nanosecond resolution: |
| |
| * :func:`time.clock_gettime_ns` |
| * :func:`time.clock_settime_ns` |
| * :func:`time.monotonic_ns` |
| * :func:`time.perf_counter_ns` |
| * :func:`time.process_time_ns` |
| * :func:`time.time_ns` |
| |
| Add new clock identifiers: |
| |
| * :data:`time.CLOCK_BOOTTIME` (Linux): Identical to |
| :data:`time.CLOCK_MONOTONIC`, except it also includes any time that the |
| system is suspended. |
| * :data:`time.CLOCK_PROF` (FreeBSD, NetBSD and OpenBSD): High-resolution |
| per-process timer from the CPU. |
| * :data:`time.CLOCK_UPTIME` (FreeBSD, OpenBSD): Time whose absolute value is |
| the time the system has been running and not suspended, providing accurate |
| uptime measurement, both absolute and interval. |
| |
| Added functions :func:`time.thread_time` and :func:`time.thread_time_ns` |
| to get per-thread CPU time measurements. |
| (Contributed by Antoine Pitrou in :issue:`32025`.) |
| |
| |
| unittest |
| -------- |
| Added new command-line option ``-k`` to filter tests to run with a substring or |
| Unix shell-like pattern. For example, ``python -m unittest -k foo`` runs the |
| tests ``foo_tests.SomeTest.test_something``, ``bar_tests.SomeTest.test_foo``, |
| but not ``bar_tests.FooTest.test_something``. |
| |
| |
| unittest.mock |
| ------------- |
| |
| The :const:`~unittest.mock.sentinel` attributes now preserve their identity |
| when they are :mod:`copied <copy>` or :mod:`pickled <pickle>`. (Contributed by |
| Serhiy Storchaka in :issue:`20804`.) |
| |
| New function :const:`~unittest.mock.seal` will disable the creation of mock |
| children by preventing to get or set any new attribute on the sealed mock. |
| The sealing process is performed recursively. (Contributed by Mario Corchero |
| in :issue:`30541`.) |
| |
| xmlrpc.server |
| ------------- |
| |
| :meth:`register_function` of :class:`xmlrpc.server.SimpleXMLRPCDispatcher` and |
| its subclasses can be used as a decorator. (Contributed by Xiang Zhang in |
| :issue:`7769`.) |
| |
| unicodedata |
| ----------- |
| |
| The internal :mod:`unicodedata` database has been upgraded to use `Unicode 10 |
| <http://www.unicode.org/versions/Unicode10.0.0/>`_. (Contributed by Benjamin |
| Peterson.) |
| |
| urllib.parse |
| ------------ |
| |
| :func:`urllib.parse.quote` has been updated from RFC 2396 to RFC 3986, |
| adding `~` to the set of characters that is never quoted by default. |
| (Contributed by Christian Theune and Ratnadeep Debnath in :issue:`16285`.) |
| |
| uu |
| -- |
| |
| Function :func:`~uu.encode` now accepts an optional *backtick* |
| keyword argument. When it's true, zeros are represented by ``'`'`` |
| instead of spaces. (Contributed by Xiang Zhang in :issue:`30103`.) |
| |
| warnings |
| -------- |
| |
| The initialization of the default warnings filters has changed as follows: |
| |
| * warnings enabled via command line options (including those for :option:`-b` |
| and the new CPython-specific ``-X dev`` option) are always passed to the |
| warnings machinery via the ``sys.warnoptions`` attribute. |
| * warnings filters enabled via the command line or the environment now have the |
| following precedence order: |
| |
| * the ``BytesWarning`` filter for :option:`-b` (or ``-bb``) |
| * any filters specified with :option:`-W` |
| * any filters specified with :envvar:`PYTHONWARNINGS` |
| * any other CPython specific filters (e.g. the ``default`` filter added |
| for the new ``-X dev`` mode) |
| * any implicit filters defined directly by the warnings machinery |
| * in CPython debug builds, all warnings are now displayed by default (the |
| implicit filter list is empty) |
| |
| (Contributed by Nick Coghlan and Victor Stinner in :issue:`20361`, |
| :issue:`32043`, and :issue:`32230`) |
| |
| xml.etree |
| --------- |
| |
| :ref:`ElementPath <elementtree-xpath>` predicates in the :meth:`find` |
| methods can now compare text of the current node with ``[. = "text"]``, |
| not only text in children. Predicates also allow adding spaces for |
| better readability. (Contributed by Stefan Behnel in :issue:`31648`.) |
| |
| zipapp |
| ------ |
| |
| Function :func:`zipapp.create_archive` now accepts an optional *filter* |
| argument to allow the user to select which files should be included in the |
| archive, and an optional *compressed* argument to generate a compressed |
| archive. |
| |
| A command line option ``--compress`` has also been added to support |
| compression. |
| |
| |
| Optimizations |
| ============= |
| |
| * Added two new opcodes: ``LOAD_METHOD`` and ``CALL_METHOD`` to avoid |
| instantiation of bound method objects for method calls, which results |
| in method calls being faster up to 20%. (Contributed by Yury Selivanov and |
| INADA Naoki in :issue:`26110`.) |
| |
| * Searching some unlucky Unicode characters (like Ukrainian capital "Є") |
| in a string was up to 25 times slower than searching other characters. |
| Now it is slower only by 3 times in the worst case. |
| (Contributed by Serhiy Storchaka in :issue:`24821`.) |
| |
| * Fast implementation from standard C library is now used for functions |
| :func:`~math.erf` and :func:`~math.erfc` in the :mod:`math` module. |
| (Contributed by Serhiy Storchaka in :issue:`26121`.) |
| |
| * The :func:`os.fwalk` function has been sped up by 2 times. This was done |
| using the :func:`os.scandir` function. |
| (Contributed by Serhiy Storchaka in :issue:`25996`.) |
| |
| * The :func:`shutil.rmtree` function has been sped up to 20--40%. |
| This was done using the :func:`os.scandir` function. |
| (Contributed by Serhiy Storchaka in :issue:`28564`.) |
| |
| * Optimized case-insensitive matching and searching of :mod:`regular |
| expressions <re>`. Searching some patterns can now be up to 20 times faster. |
| (Contributed by Serhiy Storchaka in :issue:`30285`.) |
| |
| * :func:`re.compile` now converts ``flags`` parameter to int object if |
| it is ``RegexFlag``. It is now as fast as Python 3.5, and faster than |
| Python 3.6 by about 10% depending on the pattern. |
| (Contributed by INADA Naoki in :issue:`31671`.) |
| |
| * :meth:`selectors.EpollSelector.modify`, :meth:`selectors.PollSelector.modify` |
| and :meth:`selectors.DevpollSelector.modify` may be around 10% faster under |
| heavy loads. (Contributed by Giampaolo Rodola' in :issue:`30014`) |
| |
| * Constant folding is moved from peephole optimizer to new AST optimizer. |
| (Contributed by Eugene Toder and INADA Naoki in :issue:`29469`) |
| |
| Build and C API Changes |
| ======================= |
| |
| * A full copy of libffi is no longer bundled for use when building the |
| :mod:`_ctypes <ctypes>` module on non-OSX UNIX platforms. An installed copy |
| of libffi is now required when building ``_ctypes`` on such platforms. |
| Contributed by Zachary Ware in :issue:`27979`. |
| |
| * The fields :c:member:`name` and :c:member:`doc` of structures |
| :c:type:`PyMemberDef`, :c:type:`PyGetSetDef`, |
| :c:type:`PyStructSequence_Field`, :c:type:`PyStructSequence_Desc`, |
| and :c:type:`wrapperbase` are now of type ``const char *`` rather of |
| ``char *``. (Contributed by Serhiy Storchaka in :issue:`28761`.) |
| |
| * The result of :c:func:`PyUnicode_AsUTF8AndSize` and :c:func:`PyUnicode_AsUTF8` |
| is now of type ``const char *`` rather of ``char *``. (Contributed by Serhiy |
| Storchaka in :issue:`28769`.) |
| |
| * The result of :c:func:`PyMapping_Keys`, :c:func:`PyMapping_Values` and |
| :c:func:`PyMapping_Items` is now always a list, rather than a list or a |
| tuple. (Contributed by Oren Milman in :issue:`28280`.) |
| |
| * Added functions :c:func:`PySlice_Unpack` and :c:func:`PySlice_AdjustIndices`. |
| (Contributed by Serhiy Storchaka in :issue:`27867`.) |
| |
| * :c:func:`PyOS_AfterFork` is deprecated in favour of the new functions |
| :c:func:`PyOS_BeforeFork`, :c:func:`PyOS_AfterFork_Parent` and |
| :c:func:`PyOS_AfterFork_Child`. (Contributed by Antoine Pitrou in |
| :issue:`16500`.) |
| |
| * The Windows build process no longer depends on Subversion to pull in external |
| sources, a Python script is used to download zipfiles from GitHub instead. |
| If Python 3.6 is not found on the system (via ``py -3.6``), NuGet is used to |
| download a copy of 32-bit Python for this purpose. (Contributed by Zachary |
| Ware in :issue:`30450`.) |
| |
| * The ``PyExc_RecursionErrorInst`` singleton that was part of the public API |
| has been removed as its members being never cleared may cause a segfault |
| during finalization of the interpreter. Contributed by Xavier de Gaye in |
| :issue:`22898` and :issue:`30697`. |
| |
| * Support for building ``--without-threads`` is removed. |
| (Contributed by Antoine Pitrou in :issue:`31370`.). |
| |
| |
| Other CPython Implementation Changes |
| ==================================== |
| |
| * Trace hooks may now opt out of receiving ``line`` events from the interpreter |
| by setting the new ``f_trace_lines`` attribute to :const:`False` on the frame |
| being traced. (Contributed by Nick Coghlan in :issue:`31344`.) |
| |
| * Trace hooks may now opt in to receiving ``opcode`` events from the interpreter |
| by setting the new ``f_trace_opcodes`` attribute to :const:`True` on the frame |
| being traced. (Contributed by Nick Coghlan in :issue:`31344`.) |
| |
| |
| Deprecated |
| ========== |
| |
| * Yield expressions (both ``yield`` and ``yield from`` clauses) are now deprecated |
| in comprehensions and generator expressions (aside from the iterable expression |
| in the leftmost :keyword:`for` clause). This ensures that comprehensions |
| always immediately return a container of the appropriate type (rather than |
| potentially returning a :term:`generator iterator` object), while generator |
| expressions won't attempt to interleave their implicit output with the output |
| from any explicit yield expressions. |
| |
| In Python 3.7, such expressions emit :exc:`DeprecationWarning` when compiled, |
| in Python 3.8+ they will emit :exc:`SyntaxError`. (Contributed by Serhiy |
| Storchaka in :issue:`10544`.) |
| |
| - Function :c:func:`PySlice_GetIndicesEx` is deprecated and replaced with |
| a macro if ``Py_LIMITED_API`` is not set or set to the value between |
| ``0x03050400`` and ``0x03060000`` (not including) or ``0x03060100`` or |
| higher. (Contributed by Serhiy Storchaka in :issue:`27867`.) |
| |
| - Deprecated :meth:`format` from :mod:`locale`, use the :meth:`format_string` |
| instead. (Contributed by Garvit in :issue:`10379`.) |
| |
| - Methods |
| :meth:`MetaPathFinder.find_module() <importlib.abc.MetaPathFinder.find_module>` |
| (replaced by |
| :meth:`MetaPathFinder.find_spec() <importlib.abc.MetaPathFinder.find_spec>` |
| ) and |
| :meth:`PathEntryFinder.find_loader() <importlib.abc.PathEntryFinder.find_loader>` |
| (replaced by |
| :meth:`PathEntryFinder.find_spec() <importlib.abc.PathEntryFinder.find_spec>`) |
| both deprecated in Python 3.4 now emit :exc:`DeprecationWarning`. (Contributed |
| by Matthias Bussonnier in :issue:`29576`) |
| |
| - Using non-integer value for selecting a plural form in :mod:`gettext` is |
| now deprecated. It never correctly worked. (Contributed by Serhiy Storchaka |
| in :issue:`28692`.) |
| |
| - The :mod:`macpath` is now deprecated and will be removed in Python 3.8. |
| |
| |
| Changes in the C API |
| -------------------- |
| |
| - The type of results of :c:func:`PyThread_start_new_thread` and |
| :c:func:`PyThread_get_thread_ident`, and the *id* parameter of |
| :c:func:`PyThreadState_SetAsyncExc` changed from :c:type:`long` to |
| :c:type:`unsigned long`. |
| (Contributed by Serhiy Storchaka in :issue:`6532`.) |
| |
| - :c:func:`PyUnicode_AsWideCharString` now raises a :exc:`ValueError` if the |
| second argument is *NULL* and the :c:type:`wchar_t*` string contains null |
| characters. (Contributed by Serhiy Storchaka in :issue:`30708`.) |
| |
| |
| Windows Only |
| ------------ |
| - The python launcher, (py.exe), can accept 32 & 64 bit specifiers **without** |
| having to specify a minor version as well. So ``py -3-32`` and ``py -3-64`` |
| become valid as well as ``py -3.7-32``, also the -*m*-64 and -*m.n*-64 forms |
| are now accepted to force 64 bit python even if 32 bit would have otherwise |
| been used. If the specified version is not available py.exe will error exit. |
| (Contributed by Steve Barnes in :issue:`30291`.) |
| |
| - The launcher can be run as "py -0" to produce a list of the installed pythons, |
| *with default marked with an asterix*. Running "py -0p" will include the paths. |
| If py is run with a version specifier that cannot be matched it will also print |
| the *short form* list of available specifiers. |
| (Contributed by Steve Barnes in :issue:`30362`.) |
| |
| Removed |
| ======= |
| |
| API and Feature Removals |
| ------------------------ |
| |
| * The ``os.stat_float_times()`` function has been removed. It was introduced in |
| Python 2.3 for backward compatibility with Python 2.2, and was deprecated |
| since Python 3.1. |
| |
| * Unknown escapes consisting of ``'\'`` and an ASCII letter in replacement |
| templates for :func:`re.sub` were deprecated in Python 3.5, and will now |
| cause an error. |
| |
| * Removed support of the *exclude* argument in :meth:`tarfile.TarFile.add`. |
| It was deprecated in Python 2.7 and 3.2. Use the *filter* argument instead. |
| |
| * The ``splitunc()`` function in the :mod:`ntpath` module was deprecated in |
| Python 3.1, and has now been removed. Use the :func:`~os.path.splitdrive` |
| function instead. |
| |
| * :func:`collections.namedtuple` no longer supports the *verbose* parameter |
| or ``_source`` attribute which showed the generated source code for the |
| named tuple class. This was part of an optimization designed to speed-up |
| class creation. (Contributed by Jelle Zijlstra with further improvements |
| by INADA Naoki, Serhiy Storchaka, and Raymond Hettinger in :issue:`28638`.) |
| |
| * Functions :func:`bool`, :func:`float`, :func:`list` and :func:`tuple` no |
| longer take keyword arguments. The first argument of :func:`int` can now |
| be passed only as positional argument. |
| |
| * Removed previously deprecated in Python 2.4 classes ``Plist``, ``Dict`` and |
| ``_InternalDict`` in the :mod:`plistlib` module. Dict values in the result |
| of functions :func:`~plistlib.readPlist` and |
| :func:`~plistlib.readPlistFromBytes` are now normal dicts. You no longer |
| can use attribute access to access items of these dictionaries. |
| |
| |
| Porting to Python 3.7 |
| ===================== |
| |
| This section lists previously described changes and other bugfixes |
| that may require changes to your code. |
| |
| |
| Changes in Python behavior |
| -------------------------- |
| |
| * Due to an oversight, earlier Python versions erroneously accepted the |
| following syntax:: |
| |
| f(1 for x in [1],) |
| |
| class C(1 for x in [1]): |
| pass |
| |
| Python 3.7 now correctly raises a :exc:`SyntaxError`, as a generator |
| expression always needs to be directly inside a set of parentheses |
| and cannot have a comma on either side, and the duplication of the |
| parentheses can be omitted only on calls. |
| (Contributed by Serhiy Storchaka in :issue:`32012` and :issue:`32023`.) |
| |
| |
| Changes in the Python API |
| ------------------------- |
| |
| * The ``asyncio.windows_utils.socketpair()`` function has been |
| removed: use directly :func:`socket.socketpair` which is available on all |
| platforms since Python 3.5 (before, it wasn't available on Windows). |
| ``asyncio.windows_utils.socketpair()`` was just an alias to |
| ``socket.socketpair`` on Python 3.5 and newer. |
| |
| * :mod:`asyncio`: The module doesn't export :mod:`selectors` and |
| :mod:`_overlapped` modules as ``asyncio.selectors`` and |
| ``asyncio._overlapped``. Replace ``from asyncio import selectors`` with |
| ``import selectors`` for example. |
| |
| * :meth:`pkgutil.walk_packages` now raises ValueError if *path* is a string. |
| Previously an empty list was returned. (Contributed by Sanyam Khurana in |
| :issue:`24744`.) |
| |
| * A format string argument for :meth:`string.Formatter.format` |
| is now :ref:`positional-only <positional-only_parameter>`. |
| Passing it as a keyword argument was deprecated in Python 3.5. (Contributed |
| by Serhiy Storchaka in :issue:`29193`.) |
| |
| * Attributes :attr:`~http.cookies.Morsel.key`, |
| :attr:`~http.cookies.Morsel.value` and |
| :attr:`~http.cookies.Morsel.coded_value` of class |
| :class:`http.cookies.Morsel` are now read-only. |
| Assigning to them was deprecated in Python 3.5. |
| Use the :meth:`~http.cookies.Morsel.set` method for setting them. |
| (Contributed by Serhiy Storchaka in :issue:`29192`.) |
| |
| * ``Module``, ``FunctionDef``, ``AsyncFunctionDef``, and |
| ``ClassDef`` AST nodes now have a new ``docstring`` field. |
| The first statement in their body is not considered as a docstring |
| anymore. ``co_firstlineno`` and ``co_lnotab`` of code object for class |
| and module are affected by this change. (Contributed by INADA Naoki and |
| Eugene Toder in :issue:`29463`.) |
| |
| * The *mode* argument of :func:`os.makedirs` no longer affects the file |
| permission bits of newly-created intermediate-level directories. |
| To set their file permission bits you can set the umask before invoking |
| ``makedirs()``. |
| (Contributed by Serhiy Storchaka in :issue:`19930`.) |
| |
| * The :attr:`struct.Struct.format` type is now :class:`str` instead of |
| :class:`bytes`. (Contributed by Victor Stinner in :issue:`21071`.) |
| |
| * Due to internal changes in :mod:`socket` you won't be able to |
| :func:`socket.fromshare` a socket :func:`~socket.socket.share`-ed in older |
| Python versions. |
| |
| * ``repr`` for :class:`datetime.timedelta` has changed to include keyword arguments |
| in the output. (Contributed by Utkarsh Upadhyay in :issue:`30302`.) |
| |
| * Because :func:`shutil.rmtree` is now implemented using the :func:`os.scandir` |
| function, the user specified handler *onerror* is now called with the first |
| argument ``os.scandir`` instead of ``os.listdir`` when listing the direcory |
| is failed. |
| |
| * Support of nested sets and set operations in regular expressions as in |
| `Unicode Technical Standard #18`_ might be added in the future. This would |
| change the syntax, so to facilitate this change a :exc:`FutureWarning` will |
| be raised in ambiguous cases for the time being. |
| That include sets starting with a literal ``'['`` or containing literal |
| character sequences ``'--'``, ``'&&'``, ``'~~'``, and ``'||'``. To |
| avoid a warning escape them with a backslash. |
| (Contributed by Serhiy Storchaka in :issue:`30349`.) |
| |
| * The result of splitting a string on a :mod:`regular expression <re>` |
| that could match an empty string has been changed. For example |
| splitting on ``r'\s*'`` will now split not only on whitespaces as it |
| did previously, but also between any pair of non-whitespace |
| characters. The previous behavior can be restored by changing the pattern |
| to ``r'\s+'``. A :exc:`FutureWarning` was emitted for such patterns since |
| Python 3.5. |
| |
| For patterns that match both empty and non-empty strings, the result of |
| searching for all matches may also be changed in other cases. For example |
| in the string ``'a\n\n'``, the pattern ``r'(?m)^\s*?$'`` will not only |
| match empty strings at positions 2 and 3, but also the string ``'\n'`` at |
| positions 2--3. To match only blank lines, the pattern should be rewritten |
| as ``r'(?m)^[^\S\n]*$'``. |
| |
| (Contributed by Serhiy Storchaka in :issue:`25054`.) |
| |
| * :class:`tracemalloc.Traceback` frames are now sorted from oldest to most |
| recent to be more consistent with :mod:`traceback`. |
| (Contributed by Jesse Bakker in :issue:`32121`.) |
| |
| .. _Unicode Technical Standard #18: https://unicode.org/reports/tr18/ |
| |
| * On Windows the default for the *close_fds* argument of |
| :class:`subprocess.Popen` was changed from :const:`False` to :const:`True` |
| when redirecting the standard handles. If you previously depended on handles |
| being inherited when using :class:`subprocess.Popen` with standard io |
| redirection, you will have to pass ``close_fds=False`` to preserve the |
| previous behaviour, or use |
| :attr:`STARTUPINFO.lpAttributeList <subprocess.STARTUPINFO.lpAttributeList>`. |
| |
| |
| Changes in the C API |
| -------------------- |
| |
| * The function :c:func:`PySlice_GetIndicesEx` is considered not safe for |
| resizable sequences. If the slice indices are not instances of :class:`int`, |
| but objects that implement the :meth:`!__index__` method, the sequence can be |
| resized after passing its length to :c:func:`!PySlice_GetIndicesEx`. This |
| can lead to returning indices out of the length of the sequence. For |
| avoiding possible problems use new functions :c:func:`PySlice_Unpack` and |
| :c:func:`PySlice_AdjustIndices`. |
| (Contributed by Serhiy Storchaka in :issue:`27867`.) |
| |
| |
| CPython bytecode changes |
| ------------------------ |
| |
| * Added two new opcodes: :opcode:`LOAD_METHOD` and :opcode:`CALL_METHOD`. |
| (Contributed by Yury Selivanov and INADA Naoki in :issue:`26110`.) |
| |
| |
| Other CPython implementation changes |
| ------------------------------------ |
| |
| * In preparation for potential future changes to the public CPython runtime |
| initialization API (see :pep:`432` for details), CPython's internal startup |
| and configuration management logic has been significantly refactored. While |
| these updates are intended to be entirely transparent to both embedding |
| applications and users of the regular CPython CLI, they're being mentioned |
| here as the refactoring changes the internal order of various operations |
| during interpreter startup, and hence may uncover previously latent defects, |
| either in embedding applications, or in CPython itself. |
| (Contributed by Nick Coghlan and Eric Snow as part of :issue:`22257`.) |
| |
| * Due to changes in the way the default warnings filters are configured, |
| setting ``Py_BytesWarningFlag`` to a value greater than one is no longer |
| sufficient to both emit ``BytesWarning`` messages and have them converted |
| to exceptions. Instead, the flag must be set (to cause the warnings to be |
| emitted in the first place), and an explicit ``error::BytesWarning`` |
| warnings filter added to convert them to exceptions. |
| |
| Documentation |
| ============= |
| |
| .. _whatsnew37-pep545: |
| |
| PEP 545: Python Documentation Translations |
| ------------------------------------------ |
| |
| :pep:`545` describes the process to translate Python documentation, |
| and two translations have been added: |
| |
| - Japanese: https://docs.python.org/ja/ and associated GitHub |
| repository: https://github.com/python/python-docs-ja |
| |
| - French: https://docs.python.org/fr/ and associated GitHub |
| repository: https://github.com/python/python-docs-fr |
| |
| (Contributed by Julien Palard, Inada Naoki, and Victor Stinner in |
| :issue:`26546`.) |