| **************************** |
| What's New In Python 3.6 |
| **************************** |
| |
| :Release: |release| |
| :Date: |today| |
| |
| .. Rules for maintenance: |
| |
| * Anyone can add text to this document. Do not spend very much time |
| on the wording of your changes, because your text will probably |
| get rewritten to some degree. |
| |
| * The maintainer will go through Misc/NEWS periodically and add |
| changes; it's therefore more important to add your changes to |
| Misc/NEWS than to this file. |
| |
| * This is not a complete list of every single change; completeness |
| is the purpose of Misc/NEWS. Some changes I consider too small |
| or esoteric to include. If such a change is added to the text, |
| I'll just remove it. (This is another reason you shouldn't spend |
| too much time on writing your addition.) |
| |
| * If you want to draw your new text to the attention of the |
| maintainer, add 'XXX' to the beginning of the paragraph or |
| section. |
| |
| * It's OK to just add a fragmentary note about a change. For |
| example: "XXX Describe the transmogrify() function added to the |
| socket module." The maintainer will research the change and |
| write the necessary text. |
| |
| * You can comment out your additions if you like, but it's not |
| necessary (especially when a final release is some months away). |
| |
| * Credit the author of a patch or bugfix. Just the name is |
| sufficient; the e-mail address isn't necessary. |
| |
| * It's helpful to add the bug/patch number as a comment: |
| |
| XXX Describe the transmogrify() function added to the socket |
| module. |
| (Contributed by P.Y. Developer in :issue:`12345`.) |
| |
| This saves the maintainer the effort of going through the Mercurial log |
| when researching a change. |
| |
| This article explains the new features in Python 3.6, compared to 3.5. |
| |
| For full details, see the :source:`Misc/NEWS` file. |
| |
| .. note:: |
| |
| Prerelease users should be aware that this document is currently in draft |
| form. It will be updated substantially as Python 3.6 moves towards release, |
| so it's worth checking back even after reading earlier versions. |
| |
| |
| Summary -- Release highlights |
| ============================= |
| |
| .. This section singles out the most important changes in Python 3.6. |
| Brevity is key. |
| |
| * PEP 498: :ref:`Formatted string literals <whatsnew-fstrings>` |
| |
| .. PEP-sized items next. |
| |
| .. _pep-4XX: |
| |
| .. PEP 4XX: Virtual Environments |
| .. ============================= |
| |
| |
| .. (Implemented by Foo Bar.) |
| |
| .. .. seealso:: |
| |
| :pep:`4XX` - Python Virtual Environments |
| PEP written by Carl Meyer |
| |
| |
| New Features |
| ============ |
| |
| .. _whatsnew-fstrings: |
| |
| PEP 498: Formatted string literals |
| ---------------------------------- |
| |
| Formatted string literals are a new kind of string literal, prefixed |
| with ``'f'``. They are similar to the format strings accepted by |
| :meth:`str.format`. They contain replacement fields surrounded by |
| curly braces. The replacement fields are expressions, which are |
| evaluated at run time, and then formatted using the :func:`format` protocol. |
| |
| >>> name = "Fred" |
| >>> f"He said his name is {name}." |
| 'He said his name is Fred.' |
| |
| See :pep:`498` and the main documentation at :ref:`f-strings`. |
| |
| |
| PYTHONMALLOC environment variable |
| --------------------------------- |
| |
| The new :envvar:`PYTHONMALLOC` environment variable allows to set the Python |
| memory allocators and/or install debug hooks. |
| |
| It is now possible to install debug hooks on Python memory allocators on Python |
| compiled in release mode using ``PYTHONMALLOC=debug``. Effects of debug hooks: |
| |
| * Newly allocated memory is filled with the byte ``0xCB`` |
| * Freed memory is filled with the byte ``0xDB`` |
| * Detect violations of Python memory allocator API. For example, |
| :c:func:`PyObject_Free` called on a memory block allocated by |
| :c:func:`PyMem_Malloc`. |
| * Detect write before the start of the buffer (buffer underflow) |
| * Detect write after the end of the buffer (buffer overflow) |
| * Check that the :term:`GIL <global interpreter lock>` is held when allocator |
| functions of :c:data:`PYMEM_DOMAIN_OBJ` (ex: :c:func:`PyObject_Malloc`) and |
| :c:data:`PYMEM_DOMAIN_MEM` (ex: :c:func:`PyMem_Malloc`) domains are called. |
| |
| Checking if the GIL is held is also a new feature of Python 3.6. |
| |
| See the :c:func:`PyMem_SetupDebugHooks` function for debug hooks on Python |
| memory allocators. |
| |
| It is now also possible to force the usage of the :c:func:`malloc` allocator of |
| the C library for all Python memory allocations using ``PYTHONMALLOC=malloc``. |
| It helps to use external memory debuggers like Valgrind on a Python compiled in |
| release mode. |
| |
| On error, the debug hooks on Python memory allocators now use the |
| :mod:`tracemalloc` module to get the traceback where a memory block was |
| allocated. |
| |
| Example of fatal error on buffer overflow using |
| ``python3.6 -X tracemalloc=5`` (store 5 frames in traces):: |
| |
| Debug memory block at address p=0x7fbcd41666f8: API 'o' |
| 4 bytes originally requested |
| The 7 pad bytes at p-7 are FORBIDDENBYTE, as expected. |
| The 8 pad bytes at tail=0x7fbcd41666fc are not all FORBIDDENBYTE (0xfb): |
| at tail+0: 0x02 *** OUCH |
| at tail+1: 0xfb |
| at tail+2: 0xfb |
| at tail+3: 0xfb |
| at tail+4: 0xfb |
| at tail+5: 0xfb |
| at tail+6: 0xfb |
| at tail+7: 0xfb |
| The block was made by call #1233329 to debug malloc/realloc. |
| Data at p: 1a 2b 30 00 |
| |
| Memory block allocated at (most recent call first): |
| File "test/test_bytes.py", line 323 |
| File "unittest/case.py", line 600 |
| File "unittest/case.py", line 648 |
| File "unittest/suite.py", line 122 |
| File "unittest/suite.py", line 84 |
| |
| Fatal Python error: bad trailing pad byte |
| |
| Current thread 0x00007fbcdbd32700 (most recent call first): |
| File "test/test_bytes.py", line 323 in test_hex |
| File "unittest/case.py", line 600 in run |
| File "unittest/case.py", line 648 in __call__ |
| File "unittest/suite.py", line 122 in run |
| File "unittest/suite.py", line 84 in __call__ |
| File "unittest/suite.py", line 122 in run |
| File "unittest/suite.py", line 84 in __call__ |
| ... |
| |
| (Contributed by Victor Stinner in :issue:`26516` and :issue:`26564`.) |
| |
| |
| Other Language Changes |
| ====================== |
| |
| * None yet. |
| |
| |
| New Modules |
| =========== |
| |
| * None yet. |
| |
| |
| Improved Modules |
| ================ |
| |
| datetime |
| -------- |
| |
| The :meth:`datetime.strftime() <datetime.datetime.strftime>` and |
| :meth:`date.strftime() <datetime.date.strftime>` methods now support ISO 8601 date |
| directives ``%G``, ``%u`` and ``%V``. |
| (Contributed by Ashley Anderson in :issue:`12006`.) |
| |
| |
| faulthandler |
| ------------ |
| |
| On Windows, the :mod:`faulthandler` module now installs an handler for Windows |
| exceptions: see :func:`faulthandler.enable`. (Contributed by Victor Stinner in |
| :issue:`23848`.) |
| |
| |
| os |
| -- |
| |
| A new :meth:`~os.scandir.close` method allows explicitly closing a |
| :func:`~os.scandir` iterator. The :func:`~os.scandir` iterator now |
| supports the :term:`context manager` protocol. If a :func:`scandir` |
| iterator is neither exhausted nor explicitly closed a :exc:`ResourceWarning` |
| will be emitted in its destructor. |
| (Contributed by Serhiy Storchaka in :issue:`25994`.) |
| |
| |
| pickle |
| ------ |
| |
| Objects that need calling ``__new__`` with keyword arguments can now be pickled |
| using :ref:`pickle protocols <pickle-protocols>` older than protocol version 4. |
| Protocol version 4 already supports this case. (Contributed by Serhiy |
| Storchaka in :issue:`24164`.) |
| |
| |
| rlcompleter |
| ----------- |
| |
| Private and special attribute names now are omitted unless the prefix starts |
| with underscores. A space or a colon is added after some completed keywords. |
| (Contributed by Serhiy Storchaka in :issue:`25011` and :issue:`25209`.) |
| |
| Names of most attributes listed by :func:`dir` are now completed. |
| Previously, names of properties and slots which were not yet created on |
| an instance were excluded. (Contributed by Martin Panter in :issue:`25590`.) |
| |
| |
| telnetlib |
| --------- |
| |
| :class:`~telnetlib.Telnet` is now a context manager (contributed by |
| Stéphane Wirtel in :issue:`25485`). |
| |
| |
| unittest.mock |
| ------------- |
| |
| The :class:`~unittest.mock.Mock` class has the following improvements: |
| |
| * Two new methods, :meth:`Mock.assert_called() |
| <unittest.mock.Mock.assert_called>` and :meth:`Mock.assert_called_once() |
| <unittest.mock.Mock.assert_called_once>` to check if the mock object |
| was called. |
| (Contributed by Amit Saha in :issue:`26323`.) |
| |
| |
| urllib.robotparser |
| ------------------ |
| |
| :class:`~urllib.robotparser.RobotFileParser` now supports the ``Crawl-delay`` and |
| ``Request-rate`` extensions. |
| (Contributed by Nikolay Bogoychev in :issue:`16099`.) |
| |
| |
| warnings |
| -------- |
| |
| A new optional *source* parameter has been added to the |
| :func:`warnings.warn_explicit` function: the destroyed object which emitted a |
| :exc:`ResourceWarning`. A *source* attribute has also been added to |
| :class:`warnings.WarningMessage` (contributed by Victor Stinner in |
| :issue:`26568` and :issue:`26567`). |
| |
| When a :exc:`ResourceWarning` warning is logged, the :mod:`tracemalloc` is now |
| used to try to retrieve the traceback where the detroyed object was allocated. |
| |
| Example with the script ``example.py``:: |
| |
| import warnings |
| |
| def func(): |
| return open(__file__) |
| |
| f = func() |
| f = None |
| |
| Output of the command ``python3.6 -Wd -X tracemalloc=5 example.py``:: |
| |
| example.py:7: ResourceWarning: unclosed file <_io.TextIOWrapper name='example.py' mode='r' encoding='UTF-8'> |
| f = None |
| Object allocated at (most recent call first): |
| File "example.py", lineno 4 |
| return open(__file__) |
| File "example.py", lineno 6 |
| f = func() |
| |
| The "Object allocated at" traceback is new and only displayed if |
| :mod:`tracemalloc` is tracing Python memory allocations and if the |
| :mod:`warnings` was already imported. |
| |
| |
| zipfile |
| ------- |
| |
| A new :meth:`ZipInfo.from_file() <zipfile.ZipInfo.from_file>` class method |
| allows making a :class:`~zipfile.ZipInfo` instance from a filesystem file. |
| A new :meth:`ZipInfo.is_dir() <zipfile.ZipInfo.is_dir>` method can be used |
| to check if the :class:`~zipfile.ZipInfo` instance represents a directory. |
| (Contributed by Thomas Kluyver in :issue:`26039`.) |
| |
| |
| zlib |
| ---- |
| |
| The :func:`~zlib.compress` function now accepts keyword arguments. |
| (Contributed by Aviv Palivoda in :issue:`26243`.) |
| |
| |
| Optimizations |
| ============= |
| |
| * The ASCII decoder is now up to 60 times as fast for error handlers |
| ``surrogateescape``, ``ignore`` and ``replace`` (Contributed |
| by Victor Stinner in :issue:`24870`). |
| |
| * The ASCII and the Latin1 encoders are now up to 3 times as fast for the |
| error handler ``surrogateescape`` (Contributed by Victor Stinner in :issue:`25227`). |
| |
| * The UTF-8 encoder is now up to 75 times as fast for error handlers |
| ``ignore``, ``replace``, ``surrogateescape``, ``surrogatepass`` (Contributed |
| by Victor Stinner in :issue:`25267`). |
| |
| * The UTF-8 decoder is now up to 15 times as fast for error handlers |
| ``ignore``, ``replace`` and ``surrogateescape`` (Contributed |
| by Victor Stinner in :issue:`25301`). |
| |
| * ``bytes % args`` is now up to 2 times faster. (Contributed by Victor Stinner |
| in :issue:`25349`). |
| |
| * ``bytearray % args`` is now between 2.5 and 5 times faster. (Contributed by |
| Victor Stinner in :issue:`25399`). |
| |
| * Optimize :meth:`bytes.fromhex` and :meth:`bytearray.fromhex`: they are now |
| between 2x and 3.5x faster. (Contributed by Victor Stinner in :issue:`25401`). |
| |
| * Optimize ``bytes.replace(b'', b'.')`` and ``bytearray.replace(b'', b'.')``: |
| up to 80% faster. (Contributed by Josh Snider in :issue:`26574`). |
| |
| |
| Build and C API Changes |
| ======================= |
| |
| * New :c:func:`Py_FinalizeEx` API which indicates if flushing buffered data |
| failed (:issue:`5319`). |
| |
| |
| Deprecated |
| ========== |
| |
| New Keywords |
| ------------ |
| |
| ``async`` and ``await`` are not recommended to be used as variable, class, |
| function or module names. Introduced by :pep:`492` in Python 3.5, they will |
| become proper keywords in Python 3.7. |
| |
| |
| Deprecated Python modules, functions and methods |
| ------------------------------------------------ |
| |
| * :meth:`importlib.machinery.SourceFileLoader` and |
| :meth:`importlib.machinery.SourcelessFileLoader` are now deprecated. They |
| were the only remaining implementations of |
| :meth:`importlib.abc.Loader.load_module` in :mod:`importlib` that had not |
| been deprecated in previous versions of Python in favour of |
| :meth:`importlib.abc.Loader.exec_module`. |
| |
| |
| Deprecated functions and types of the C API |
| ------------------------------------------- |
| |
| * None yet. |
| |
| |
| Deprecated features |
| ------------------- |
| |
| * The ``pyvenv`` script has been deprecated in favour of ``python3 -m venv``. |
| This prevents confusion as to what Python interpreter ``pyvenv`` is |
| connected to and thus what Python interpreter will be used by the virtual |
| environment. (Contributed by Brett Cannon in :issue:`25154`.) |
| |
| * When performing a relative import, falling back on ``__name__`` and |
| ``__path__`` from the calling module when ``__spec__`` or |
| ``__package__`` are not defined now raises an :exc:`ImportWarning`. |
| (Contributed by Rose Ames in :issue:`25791`.) |
| |
| |
| Deprecated Python behavior |
| -------------------------- |
| |
| * Raising the :exc:`StopIteration` exception inside a generator will now generate a |
| :exc:`DeprecationWarning`, and will trigger a :exc:`RuntimeError` in Python 3.7. |
| See :ref:`whatsnew-pep-479` for details. |
| |
| |
| Removed |
| ======= |
| |
| API and Feature Removals |
| ------------------------ |
| |
| * ``inspect.getmoduleinfo()`` was removed (was deprecated since CPython 3.3). |
| :func:`inspect.getmodulename` should be used for obtaining the module |
| name for a given path. |
| |
| * ``traceback.Ignore`` class and ``traceback.usage``, ``traceback.modname``, |
| ``traceback.fullmodname``, ``traceback.find_lines_from_code``, |
| ``traceback.find_lines``, ``traceback.find_strings``, |
| ``traceback.find_executable_lines`` methods were removed from the |
| :mod:`traceback` module. They were undocumented methods deprecated since |
| Python 3.2 and equivalent functionality is available from private methods. |
| |
| |
| Porting to Python 3.6 |
| ===================== |
| |
| This section lists previously described changes and other bugfixes |
| that may require changes to your code. |
| |
| Changes in the Python API |
| ------------------------- |
| |
| * The format of the ``co_lnotab`` attribute of code objects changed to support |
| negative line number delta. By default, Python does not emit bytecode with |
| negative line number delta. Functions using ``frame.f_lineno``, |
| ``PyFrame_GetLineNumber()`` or ``PyCode_Addr2Line()`` are not affected. |
| Functions decoding directly ``co_lnotab`` should be updated to use a signed |
| 8-bit integer type for the line number delta, but it's only required to |
| support applications using negative line number delta. See |
| ``Objects/lnotab_notes.txt`` for the ``co_lnotab`` format and how to decode |
| it, and see the :pep:`511` for the rationale. |
| |
| * The functions in the :mod:`compileall` module now return booleans instead |
| of ``1`` or ``0`` to represent success or failure, respectively. Thanks to |
| booleans being a subclass of integers, this should only be an issue if you |
| were doing identity checks for ``1`` or ``0``. See :issue:`25768`. |
| |
| * Reading the :attr:`~urllib.parse.SplitResult.port` attribute of |
| :func:`urllib.parse.urlsplit` and :func:`~urllib.parse.urlparse` results |
| now raises :exc:`ValueError` for out-of-range values, rather than |
| returning :const:`None`. See :issue:`20059`. |
| |
| * The :mod:`imp` module now raises a :exc:`DeprecationWarning` instead of |
| :exc:`PendingDeprecationWarning`. |
| |
| * The following modules have had missing APIs added to their :attr:`__all__` |
| attributes to match the documented APIs: :mod:`calendar`, :mod:`csv`, |
| :mod:`~xml.etree.ElementTree`, :mod:`enum`, |
| :mod:`fileinput`, :mod:`ftplib`, :mod:`logging`, |
| :mod:`optparse`, :mod:`tarfile`, :mod:`threading` and |
| :mod:`wave`. This means they will export new symbols when ``import *`` |
| is used. See :issue:`23883`. |
| |
| * When performing a relative import, if ``__package__`` does not compare equal |
| to ``__spec__.parent`` then :exc:`ImportWarning` is raised. |
| (Contributed by Brett Cannon in :issue:`25791`.) |
| |
| * When a relative import is performed and no parent package is known, then |
| :exc:`ImportError` will be raised. Previously, :exc:`SystemError` could be |
| raised. (Contributed by Brett Cannon in :issue:`18018`.) |
| |
| * Servers based on the :mod:`socketserver` module, including those |
| defined in :mod:`http.server`, :mod:`xmlrpc.server` and |
| :mod:`wsgiref.simple_server`, now only catch exceptions derived |
| from :exc:`Exception`. Therefore if a request handler raises |
| an exception like :exc:`SystemExit` or :exc:`KeyboardInterrupt`, |
| :meth:`~socketserver.BaseServer.handle_error` is no longer called, and |
| the exception will stop a single-threaded server. (Contributed by |
| Martin Panter in :issue:`23430`.) |
| |
| * :func:`spwd.getspnam` now raises a :exc:`PermissionError` instead of |
| :exc:`KeyError` if the user doesn't have privileges. |
| |
| Changes in the C API |
| -------------------- |
| |
| * :c:func:`Py_Exit` (and the main interpreter) now override the exit status |
| with 120 if flushing buffered data failed. See :issue:`5319`. |