| **************************** |
| What's New In Python 3.5 |
| **************************** |
| |
| :Release: |release| |
| :Date: |today| |
| |
| .. Rules for maintenance: |
| |
| * Anyone can add text to this document. Do not spend very much time |
| on the wording of your changes, because your text will probably |
| get rewritten to some degree. |
| |
| * The maintainer will go through Misc/NEWS periodically and add |
| changes; it's therefore more important to add your changes to |
| Misc/NEWS than to this file. |
| |
| * This is not a complete list of every single change; completeness |
| is the purpose of Misc/NEWS. Some changes I consider too small |
| or esoteric to include. If such a change is added to the text, |
| I'll just remove it. (This is another reason you shouldn't spend |
| too much time on writing your addition.) |
| |
| * If you want to draw your new text to the attention of the |
| maintainer, add 'XXX' to the beginning of the paragraph or |
| section. |
| |
| * It's OK to just add a fragmentary note about a change. For |
| example: "XXX Describe the transmogrify() function added to the |
| socket module." The maintainer will research the change and |
| write the necessary text. |
| |
| * You can comment out your additions if you like, but it's not |
| necessary (especially when a final release is some months away). |
| |
| * Credit the author of a patch or bugfix. Just the name is |
| sufficient; the e-mail address isn't necessary. |
| |
| * It's helpful to add the bug/patch number as a comment: |
| |
| XXX Describe the transmogrify() function added to the socket |
| module. |
| (Contributed by P.Y. Developer in :issue:`12345`.) |
| |
| This saves the maintainer the effort of going through the Mercurial log |
| when researching a change. |
| |
| This article explains the new features in Python 3.5, compared to 3.4. |
| |
| For full details, see the :source:`Misc/NEWS` file. |
| |
| .. note:: |
| |
| Prerelease users should be aware that this document is currently in draft |
| form. It will be updated substantially as Python 3.5 moves towards release, |
| so it's worth checking back even after reading earlier versions. |
| |
| |
| .. seealso:: |
| |
| :pep:`478` - Python 3.5 Release Schedule |
| |
| |
| Summary -- Release highlights |
| ============================= |
| |
| .. This section singles out the most important changes in Python 3.5. |
| Brevity is key. |
| |
| New syntax features: |
| |
| * :pep:`465`, a new matrix multiplication operator: ``a @ b``. |
| * :pep:`492`, coroutines with async and await syntax. |
| |
| New library modules: |
| |
| * :mod:`zipapp`: :ref:`Improving Python ZIP Application Support |
| <whatsnew-zipapp>` (:pep:`441`). |
| |
| New built-in features: |
| |
| * ``bytes % args``, ``bytearray % args``: :pep:`461` - Adding ``%`` formatting |
| to bytes and bytearray |
| * ``b'\xf0\x9f\x90\x8d'.hex()``, ``bytearray(b'\xf0\x9f\x90\x8d').hex()``, |
| ``memoryview(b'\xf0\x9f\x90\x8d').hex()``: :issue:`9951` - A ``hex`` method |
| has been added to bytes, bytearray, and memoryview. |
| |
| Implementation improvements: |
| |
| * When the ``LC_TYPE`` locale is the POSIX locale (``C`` locale), |
| :py:data:`sys.stdin` and :py:data:`sys.stdout` are now using the |
| ``surrogateescape`` error handler, instead of the ``strict`` error handler |
| (:issue:`19977`). |
| |
| * :pep:`488`, the elimination of ``.pyo`` files. |
| * :pep:`489`, multi-phase initialization of extension modules. |
| |
| Significantly Improved Library Modules: |
| |
| * :class:`collections.OrderedDict` is now implemented in C, which improves |
| its performance between 4x to 100x times. Contributed by Eric Snow in |
| :issue:`16991`. |
| |
| * You may now pass bytes to the :mod:`tempfile` module's APIs and it will |
| return the temporary pathname as bytes instead of str. It also accepts |
| a value of ``None`` on parameters where only str was accepted in the past to |
| do the right thing based on the types of the other inputs. Two functions, |
| :func:`gettempdirb` and :func:`gettempprefixb`, have been added to go along |
| with this. This behavior matches that of the :mod:`os` APIs. |
| |
| Security improvements: |
| |
| * None yet. |
| |
| Please read on for a comprehensive list of user-facing changes. |
| |
| |
| .. PEP-sized items next. |
| |
| .. _pep-4XX: |
| |
| .. PEP 4XX: Virtual Environments |
| .. ============================= |
| |
| |
| .. (Implemented by Foo Bar.) |
| |
| .. .. seealso:: |
| |
| :pep:`4XX` - Python Virtual Environments |
| PEP written by Carl Meyer |
| |
| |
| PEP 492 - Coroutines with async and await syntax |
| ------------------------------------------------ |
| |
| The PEP added dedicated syntax for declaring :term:`coroutines <coroutine>`, |
| :keyword:`await` expressions, new asynchronous :keyword:`async for` |
| and :keyword:`async with` statements. |
| |
| Example:: |
| |
| async def read_data(db): |
| async with db.transaction(): |
| data = await db.fetch('SELECT ...') |
| |
| PEP written and implemented by Yury Selivanov. |
| |
| .. seealso:: |
| |
| :pep:`492` -- Coroutines with async and await syntax |
| |
| |
| PEP 461 - Formatting support for bytes and bytearray |
| ---------------------------------------------------- |
| |
| This PEP proposes adding % formatting operations similar to Python 2's ``str`` |
| type to :class:`bytes` and :class:`bytearray`. |
| |
| Examples:: |
| |
| >>> b'Hello %s!' % b'World' |
| b'Hello World!' |
| >>> b'x=%i y=%f' % (1, 2.5) |
| b'x=1 y=2.500000' |
| |
| Unicode is not allowed for ``%s``, but it is accepted by ``%a`` (equivalent of |
| ``repr(obj).encode('ascii', 'backslashreplace')``):: |
| |
| >>> b'Hello %s!' % 'World' |
| Traceback (most recent call last): |
| File "<stdin>", line 1, in <module> |
| TypeError: %b requires bytes, or an object that implements __bytes__, not 'str' |
| >>> b'price: %a' % '10€' |
| b"price: '10\\u20ac'" |
| |
| .. seealso:: |
| |
| :pep:`461` -- Adding % formatting to bytes and bytearray |
| |
| |
| PEP 465 - A dedicated infix operator for matrix multiplication |
| -------------------------------------------------------------- |
| |
| This PEP proposes a new binary operator to be used for matrix multiplication, |
| called ``@``. (Mnemonic: ``@`` is ``*`` for mATrices.) |
| |
| .. seealso:: |
| |
| :pep:`465` -- A dedicated infix operator for matrix multiplication |
| |
| |
| PEP 471 - os.scandir() function -- a better and faster directory iterator |
| ------------------------------------------------------------------------- |
| |
| :pep:`471` adds a new directory iteration function, :func:`os.scandir`, |
| to the standard library. Additionally, :func:`os.walk` is now |
| implemented using :func:`os.scandir`, which speeds it up by 3-5 times |
| on POSIX systems and by 7-20 times on Windows systems. |
| |
| PEP and implementation written by Ben Hoyt with the help of Victor Stinner. |
| |
| .. seealso:: |
| |
| :pep:`471` -- os.scandir() function -- a better and faster directory |
| iterator |
| |
| |
| PEP 475: Retry system calls failing with EINTR |
| ---------------------------------------------- |
| |
| :pep:`475` adds support for automatic retry of system calls failing with |
| :py:data:`~errno.EINTR`: this means that user code doesn't have to deal with |
| EINTR or :exc:`InterruptedError` manually, and should make it more robust |
| against asynchronous signal reception. |
| |
| .. seealso:: |
| |
| :pep:`475` -- Retry system calls failing with EINTR |
| |
| |
| PEP 479: Change StopIteration handling inside generators |
| -------------------------------------------------------- |
| |
| :pep:`479` changes the behavior of generators: when a :exc:`StopIteration` |
| exception is raised inside a generator, it is replaced with a |
| :exc:`RuntimeError`. To enable the feature a ``__future__`` import should |
| be used:: |
| |
| from __future__ import generator_stop |
| |
| Without a ``__future__`` import, a :exc:`PendingDeprecationWarning` will be |
| raised. |
| |
| PEP written by Chris Angelico and Guido van Rossum. Implemented by |
| Chris Angelico, Yury Selivanov and Nick Coghlan. |
| |
| .. seealso:: |
| |
| :pep:`479` -- Change StopIteration handling inside generators |
| |
| |
| PEP 486: Make the Python Launcher aware of virtual environments |
| --------------------------------------------------------------- |
| |
| :pep:`486` makes the Windows launcher (see :pep:`397`) aware of an active |
| virtual environment. When the default interpreter would be used and the |
| ``VIRTUAL_ENV`` environment variable is set, the interpreter in the virtual |
| environment will be used. |
| |
| .. seealso:: |
| |
| :pep:`486` -- Make the Python Launcher aware of virtual environments |
| |
| |
| PEP 488: Elimination of PYO files |
| --------------------------------- |
| |
| :pep:`488` does away with the concept of ``.pyo`` files. This means that |
| ``.pyc`` files represent both unoptimized and optimized bytecode. To prevent the |
| need to constantly regenerate bytecode files, ``.pyc`` files now have an |
| optional ``opt-`` tag in their name when the bytecode is optimized. This has the |
| side-effect of no more bytecode file name clashes when running under either |
| ``-O`` or ``-OO``. Consequently, bytecode files generated from ``-O``, and |
| ``-OO`` may now exist simultaneously. :func:`importlib.util.cache_from_source` |
| has an updated API to help with this change. |
| |
| .. seealso:: |
| |
| :pep:`488` -- Elimination of PYO files |
| |
| |
| PEP 489: Multi-phase extension module initialization |
| ---------------------------------------------------- |
| |
| :pep:`489` updates extension module initialization to take advantage of the |
| two step module loading mechanism introduced by :pep:`451` in Python 3.4. |
| |
| This change brings the import semantics of extension modules that opt-in to |
| using the new mechanism much closer to those of Python source and bytecode |
| modules, including the ability to any valid identifier as a module name, |
| rather than being restricted to ASCII. |
| |
| .. seealso:: |
| |
| :pep:`488` -- Multi-phase extension module initialization |
| |
| PEP 485: A function for testing approximate equality |
| ---------------------------------------------------- |
| |
| :pep:`485` adds the :func:`math.isclose` and :func:`cmath.isclose` |
| functions which tell whether two values are approximately equal or |
| "close" to each other. Whether or not two values are considered |
| close is determined according to given absolute and relative tolerances. |
| |
| .. seealso:: |
| |
| :pep:`485` -- A function for testing approximate equality |
| |
| Other Language Changes |
| ====================== |
| |
| Some smaller changes made to the core Python language are: |
| |
| * Added the ``'namereplace'`` error handlers. The ``'backslashreplace'`` |
| error handlers now works with decoding and translating. |
| (Contributed by Serhiy Storchaka in :issue:`19676` and :issue:`22286`.) |
| |
| * The :option:`-b` option now affects comparisons of :class:`bytes` with |
| :class:`int`. (Contributed by Serhiy Storchaka in :issue:`23681`) |
| |
| * New Kazakh :ref:`codec <standard-encodings>` ``kz1048``. (Contributed by |
| Serhiy Storchaka in :issue:`22682`.) |
| |
| * Property docstrings are now writable. This is especially useful for |
| :func:`collections.namedtuple` docstrings. |
| (Contributed by Berker Peksag in :issue:`24064`.) |
| |
| * New Tajik :ref:`codec <standard-encodings>` ``koi8_t``. (Contributed by |
| Serhiy Storchaka in :issue:`22681`.) |
| |
| |
| New Modules |
| =========== |
| |
| .. _whatsnew-zipapp: |
| |
| zipapp |
| ------ |
| |
| The new :mod:`zipapp` module (specified in :pep:`441`) provides an API and |
| command line tool for creating executable Python Zip Applications, which |
| were introduced in Python 2.6 in :issue:`1739468` but which were not well |
| publicised, either at the time or since. |
| |
| With the new module, bundling your application is as simple as putting all |
| the files, including a ``__main__.py`` file, into a directory ``myapp`` |
| and running:: |
| |
| $ python -m zipapp myapp |
| $ python myapp.pyz |
| |
| |
| Improved Modules |
| ================ |
| |
| argparse |
| -------- |
| |
| * :class:`~argparse.ArgumentParser` now allows to disable |
| :ref:`abbreviated usage <prefix-matching>` of long options by setting |
| :ref:`allow_abbrev` to ``False``. |
| (Contributed by Jonathan Paugh, Steven Bethard, paul j3 and Daniel Eriksson.) |
| |
| cgi |
| --- |
| |
| * :class:`~cgi.FieldStorage` now supports the context management protocol. |
| (Contributed by Berker Peksag in :issue:`20289`.) |
| |
| cmath |
| ----- |
| |
| * :func:`cmath.isclose` function added. |
| (Contributed by Chris Barker and Tal Einat in :issue:`24270`.) |
| |
| |
| code |
| ---- |
| |
| * The :func:`code.InteractiveInterpreter.showtraceback` method now prints |
| the full chained traceback, just like the interactive interpreter. |
| (Contributed by Claudiu Popa in :issue:`17442`.) |
| |
| collections |
| ----------- |
| |
| * You can now update docstrings produced by :func:`collections.namedtuple`:: |
| |
| Point = namedtuple('Point', ['x', 'y']) |
| Point.__doc__ = 'ordered pair' |
| Point.x.__doc__ = 'abscissa' |
| Point.y.__doc__ = 'ordinate' |
| |
| (Contributed by Berker Peksag in :issue:`24064`.) |
| |
| compileall |
| ---------- |
| |
| * :func:`compileall.compile_dir` and :mod:`compileall`'s command-line interface |
| can now do parallel bytecode compilation. |
| (Contributed by Claudiu Popa in :issue:`16104`.) |
| |
| contextlib |
| ---------- |
| |
| * The new :func:`contextlib.redirect_stderr` context manager(similar to |
| :func:`contextlib.redirect_stdout`) makes it easier for utility scripts to |
| handle inflexible APIs that write their output to :data:`sys.stderr` and |
| don't provide any options to redirect it. |
| (Contributed by Berker Peksag in :issue:`22389`.) |
| |
| curses |
| ------ |
| * The new :func:`curses.update_lines_cols` function updates the variables |
| :envvar:`curses.LINES` and :envvar:`curses.COLS`. |
| |
| difflib |
| ------- |
| |
| * The charset of the HTML document generated by :meth:`difflib.HtmlDiff.make_file` |
| can now be customized by using *charset* keyword-only parameter. The default |
| charset of HTML document changed from ``'ISO-8859-1'`` to ``'utf-8'``. |
| (Contributed by Berker Peksag in :issue:`2052`.) |
| |
| * It's now possible to compare lists of byte strings with |
| :func:`difflib.diff_bytes` (fixes a regression from Python 2). |
| |
| distutils |
| --------- |
| |
| * The ``build`` and ``build_ext`` commands now accept a ``-j`` |
| option to enable parallel building of extension modules. |
| (Contributed by Antoine Pitrou in :issue:`5309`.) |
| |
| * Added support for the LZMA compression. |
| (Contributed by Serhiy Storchaka in :issue:`16314`.) |
| |
| doctest |
| ------- |
| |
| * :func:`doctest.DocTestSuite` returns an empty :class:`unittest.TestSuite` if |
| *module* contains no docstrings instead of raising :exc:`ValueError`. |
| (Contributed by Glenn Jones in :issue:`15916`.) |
| |
| email |
| ----- |
| |
| * A new policy option :attr:`~email.policy.Policy.mangle_from_` controls |
| whether or not lines that start with "From " in email bodies are prefixed with |
| a '>' character by generators. The default is ``True`` for |
| :attr:`~email.policy.compat32` and ``False`` for all other policies. |
| (Contributed by Milan Oberkirch in :issue:`20098`.) |
| |
| * A new method :meth:`~email.message.Message.get_content_disposition` provides |
| easy access to a canonical value for the :mailheader:`Content-Disposition` |
| header (``None`` if there is no such header). (Contributed by Abhilash Raj |
| in :issue:`21083`.) |
| |
| * A new policy option :attr:`~email.policy.EmailPolicy.utf8` can be set |
| ``True`` to encode email headers using the utf8 charset instead of using |
| encoded words. This allows ``Messages`` to be formatted according to |
| :rfc:`6532` and used with an SMTP server that supports the :rfc:`6531` |
| ``SMTPUTF8`` extension. (Contributed by R. David Murray in :issue:`24211`.) |
| |
| glob |
| ---- |
| |
| * :func:`~glob.iglob` and :func:`~glob.glob` now support recursive search in |
| subdirectories using the "``**``" pattern. |
| (Contributed by Serhiy Storchaka in :issue:`13968`.) |
| |
| idlelib and IDLE |
| ---------------- |
| |
| Since idlelib implements the IDLE shell and editor and is not intended for |
| import by other programs, it gets improvements with every release. See |
| :file:`Lib/idlelib/NEWS.txt` for a cumulative list of changes since 3.4.0, |
| as well as changes made in future 3.5.x releases. This file is also available |
| from the IDLE Help -> About Idle dialog. |
| |
| imaplib |
| ------- |
| |
| * :class:`IMAP4` now supports the context management protocol. When used in a |
| :keyword:`with` statement, the IMAP4 ``LOGOUT`` command will be called |
| automatically at the end of the block. (Contributed by Tarek Ziadé and |
| Serhiy Storchaka in :issue:`4972`.) |
| |
| * :mod:`imaplib` now supports :rfc:`5161`: the :meth:`~imaplib.IMAP4.enable` |
| extension), and :rfc:`6855`: utf-8 support (internationalized email, via the |
| ``UTF8=ACCEPT`` argument to :meth:`~imaplib.IMAP4.enable`). A new attribute, |
| :attr:`~imaplib.IMAP4.utf8_enabled`, tracks whether or not :rfc:`6855` |
| support is enabled. Milan Oberkirch, R. David Murray, and Maciej Szulik in |
| :issue:`21800`.) |
| |
| * :mod:`imaplib` now automatically encodes non-ASCII string usernames and |
| passwords using ``UTF8``, as recommended by the RFCs. (Contributed by Milan |
| Oberkirch in :issue:`21800`.) |
| |
| imghdr |
| ------ |
| |
| * :func:`~imghdr.what` now recognizes the `OpenEXR <http://www.openexr.com>`_ |
| format. (Contributed by Martin Vignali and Claudiu Popa in :issue:`20295`.) |
| |
| importlib |
| --------- |
| |
| * :class:`importlib.util.LazyLoader` allows for the lazy loading of modules in |
| applications where startup time is paramount. |
| (Contributed by Brett Cannon in :issue:`17621`.) |
| |
| * :func:`importlib.abc.InspectLoader.source_to_code` is now a |
| static method to make it easier to work with source code in a string. |
| With a module object that you want to initialize you can then use |
| ``exec(code, module.__dict__)`` to execute the code in the module. |
| |
| * :func:`importlib.util.module_from_spec` is now the preferred way to create a |
| new module. Compared to :class:`types.ModuleType`, this new function will set |
| the various import-controlled attributes based on the passed-in spec object. |
| |
| inspect |
| ------- |
| |
| * :class:`inspect.Signature` and :class:`inspect.Parameter` are now |
| picklable and hashable. (Contributed by Yury Selivanov in :issue:`20726` |
| and :issue:`20334`.) |
| |
| * New method :meth:`inspect.BoundArguments.apply_defaults`. (Contributed |
| by Yury Selivanov in :issue:`24190`.) |
| |
| * New class method :meth:`inspect.Signature.from_callable`, which makes |
| subclassing of :class:`~inspect.Signature` easier. (Contributed |
| by Yury Selivanov and Eric Snow in :issue:`17373`.) |
| |
| * New argument ``follow_wrapped`` for :func:`inspect.signature`. |
| (Contributed by Yury Selivanov in :issue:`20691`.) |
| |
| * New :func:`~inspect.iscoroutine`, :func:`~inspect.iscoroutinefunction`, |
| and :func:`~inspect.isawaitable` functions. (Contributed by Yury Selivanov |
| in :issue:`24017`.) |
| |
| ipaddress |
| --------- |
| |
| * :class:`ipaddress.IPv4Network` and :class:`ipaddress.IPv6Network` now |
| accept an ``(address, netmask)`` tuple argument, so as to easily construct |
| network objects from existing addresses. (Contributed by Peter Moody |
| and Antoine Pitrou in :issue:`16531`.) |
| |
| json |
| ---- |
| |
| * The output of :mod:`json.tool` command line interface is now in the same |
| order as the input. Use the :option:`--sort-keys` option to sort the output |
| of dictionaries alphabetically by key. (Contributed by Berker Peksag in |
| :issue:`21650`.) |
| |
| * JSON decoder now raises :exc:`json.JSONDecodeError` instead of |
| :exc:`ValueError`. (Contributed by Serhiy Storchaka in :issue:`19361`.) |
| |
| os |
| -- |
| |
| * New :func:`os.scandir` function that exposes file information from |
| the operating system when listing a directory. :func:`os.scandir` |
| returns an iterator of :class:`os.DirEntry` objects corresponding to |
| the entries in the directory given by *path*. (Contributed by Ben |
| Hoyt with the help of Victor Stinner in :issue:`22524`.) |
| |
| * :class:`os.stat_result` now has a :attr:`~os.stat_result.st_file_attributes` |
| attribute on Windows. (Contributed by Ben Hoyt in :issue:`21719`.) |
| |
| os.path |
| ------- |
| |
| * New :func:`~os.path.commonpath` function that extracts common path prefix. |
| Unlike the :func:`~os.path.commonprefix` function, it always returns a valid |
| patch. (Contributed by Rafik Draoui and Serhiy Storchaka in :issue:`10395`.) |
| |
| pickle |
| ------ |
| |
| * Serializing more "lookupable" objects (such as unbound methods or nested |
| classes) now are supported with pickle protocols < 4. |
| (Contributed by Serhiy Storchaka in :issue:`23611`.) |
| |
| poplib |
| ------ |
| |
| * A new command :meth:`~poplib.POP3.utf8` enables :rfc:`6856` |
| (internationalized email) support if the POP server supports it. (Contributed |
| by Milan OberKirch in :issue:`21804`.) |
| |
| re |
| -- |
| |
| * Number of capturing groups in regular expression is no longer limited by 100. |
| (Contributed by Serhiy Storchaka in :issue:`22437`.) |
| |
| * Now unmatched groups are replaced with empty strings in :func:`re.sub` |
| and :func:`re.subn`. (Contributed by Serhiy Storchaka in :issue:`1519638`.) |
| |
| math |
| ---- |
| |
| * :data:`math.inf` and :data:`math.nan` constants added. (Contributed by Mark |
| Dickinson in :issue:`23185`.) |
| * :func:`math.isclose` function added. |
| (Contributed by Chris Barker and Tal Einat in :issue:`24270`.) |
| |
| shutil |
| ------ |
| |
| * :func:`~shutil.move` now accepts a *copy_function* argument, allowing, |
| for example, :func:`~shutil.copy` to be used instead of the default |
| :func:`~shutil.copy2` if there is a need to ignore metadata. (Contributed by |
| Claudiu Popa in :issue:`19840`.) |
| |
| signal |
| ------ |
| |
| * On Windows, :func:`signal.set_wakeup_fd` now also supports socket handles. |
| (Contributed by Victor Stinner in :issue:`22018`.) |
| |
| * Different constants of :mod:`signal` module are now enumeration values using |
| the :mod:`enum` module. This allows meaningful names to be printed during |
| debugging, instead of integer “magic numbers”. (Contributed by Giampaolo |
| Rodola' in :issue:`21076`.) |
| |
| smtpd |
| ----- |
| |
| * Both :class:`~smtpd.SMTPServer` and :class:`smtpd.SMTPChannel` now accept a |
| *decode_data* keyword to determine if the DATA portion of the SMTP |
| transaction is decoded using the ``utf-8`` codec or is instead provided to |
| :meth:`~smtpd.SMTPServer.process_message` as a byte string. The default |
| is ``True`` for backward compatibility reasons, but will change to ``False`` |
| in Python 3.6. If *decode_data* is set to ``False``, the |
| :meth:`~smtpd.SMTPServer.process_message` method must be prepared to accept |
| keyword arguments. (Contributed by Maciej Szulik in :issue:`19662`.) |
| |
| * :class:`~smtpd.SMTPServer` now advertises the ``8BITMIME`` extension |
| (:rfc:`6152`) if if *decode_data* has been set ``True``. If the client |
| specifies ``BODY=8BITMIME`` on the ``MAIL`` command, it is passed to |
| :meth:`~smtpd.SMTPServer.process_message` via the ``mail_options`` keyword. |
| (Contributed by Milan Oberkirch and R. David Murray in :issue:`21795`.) |
| |
| * :class:`~smtpd.SMTPServer` now supports the ``SMTPUTF8`` extension |
| (:rfc:`6531`: Internationalized Email). If the client specified ``SMTPUTF8 |
| BODY=8BITMIME`` on the ``MAIL`` command, they are passed to |
| :meth:`~smtpd.SMTPServer.process_message` via the ``mail_options`` keyword. |
| It is the responsibility of the :meth:`~smtpd.SMTPServer.process_message` |
| method to correctly handle the ``SMTPUTF8`` data. (Contributed by Milan |
| Oberkirch in :issue:`21725`.) |
| |
| * It is now possible to provide, directly or via name resolution, IPv6 |
| addresses in the :class:`~smtpd.SMTPServer` constructor, and have it |
| successfully connect. (Contributed by Milan Oberkirch in :issue:`14758`.) |
| |
| smtplib |
| ------- |
| |
| * A new :meth:`~smtplib.SMTP.auth` method provides a convenient way to |
| implement custom authentication mechanisms. |
| (Contributed by Milan Oberkirch in :issue:`15014`.) |
| |
| * Additional debuglevel (2) shows timestamps for debug messages in |
| :class:`smtplib.SMTP`. (Contributed by Gavin Chappell and Maciej Szulik in |
| :issue:`16914`.) |
| |
| * :mod:`smtplib` now supports :rfc:`6531` (SMTPUTF8) in both the |
| :meth:`~smtplib.SMTP.sendmail` and :meth:`~smtplib.SMTP.send_message` |
| commands. (Contributed by Milan Oberkirch and R. David Murray in |
| :issue:`22027`.) |
| |
| sndhdr |
| ------ |
| |
| * :func:`~sndhdr.what` and :func:`~sndhdr.whathdr` now return |
| :func:`~collections.namedtuple`. |
| (Contributed by Claudiu Popa in :issue:`18615`.) |
| |
| socket |
| ------ |
| |
| * New :meth:`socket.socket.sendfile` method allows to send a file over a socket |
| by using high-performance :func:`os.sendfile` function on UNIX resulting in |
| uploads being from 2x to 3x faster than when using plain |
| :meth:`socket.socket.send`. |
| (Contributed by Giampaolo Rodola' in :issue:`17552`.) |
| |
| subprocess |
| ---------- |
| |
| * The new :func:`subprocess.run` function runs subprocesses and returns a |
| :class:`subprocess.CompletedProcess` object. It Provides a more consistent |
| API than :func:`~subprocess.call`, :func:`~subprocess.check_call` and |
| :func:`~subprocess.check_output`. |
| |
| sysconfig |
| --------- |
| |
| * The user scripts directory on Windows is now versioned. |
| (Contributed by Paul Moore in :issue:`23437`.) |
| |
| tarfile |
| ------- |
| |
| * The :func:`tarfile.open` function now supports ``'x'`` (exclusive creation) |
| mode. (Contributed by Berker Peksag in :issue:`21717`.) |
| |
| * The :meth:`~tarfile.TarFile.extractall` and :meth:`~tarfile.TarFile.extract` |
| methods now take a keyword parameter *numeric_only*. If set to ``True``, |
| the extracted files and directories will be owned by the numeric uid and gid |
| from the tarfile. If set to ``False`` (the default, and the behavior in |
| versions prior to 3.5), they will be owned bythe named user and group in the |
| tarfile. (Contributed by Michael Vogt and Eric Smith in :issue:`23193`.) |
| |
| time |
| ---- |
| |
| * The :func:`time.monotonic` function is now always available. (Contributed by |
| Victor Stinner in :issue:`22043`.) |
| |
| tkinter |
| ------- |
| |
| * The :mod:`tkinter._fix` module used for setting up the Tcl/Tk environment |
| on Windows has been replaced by a private function in the :mod:`_tkinter` |
| module which makes no permanent changes to environment variables. |
| (Contributed by Zachary Ware in :issue:`20035`.) |
| |
| types |
| ----- |
| |
| * New :func:`~types.coroutine` function. (Contributed by Yury Selivanov |
| in :issue:`24017`.) |
| |
| urllib |
| ------ |
| |
| * A new :class:`~urllib.request.HTTPPasswordMgrWithPriorAuth` allows HTTP Basic |
| Authentication credentials to be managed so as to eliminate unnecessary |
| ``401`` response handling, or to unconditionally send credentials |
| on the first request in order to communicate with servers that return a |
| ``404`` response instead of a ``401`` if the ``Authorization`` header is not |
| sent. (Contributed by Matej Cepl in :issue:`19494` and Akshit Khurana in |
| :issue:`7159`.) |
| |
| * A new :func:`~urllib.parse.urlencode` parameter *quote_via* provides a way to |
| control the encoding of query parts if needed. (Contributed by Samwyse and |
| Arnon Yaari in :issue:`13866`.) |
| |
| wsgiref |
| ------- |
| |
| * *headers* parameter of :class:`wsgiref.headers.Headers` is now optional. |
| (Contributed by Pablo Torres Navarrete and SilentGhost in :issue:`5800`.) |
| |
| xmlrpc |
| ------ |
| |
| * :class:`xmlrpc.client.ServerProxy` is now a :term:`context manager`. |
| (Contributed by Claudiu Popa in :issue:`20627`.) |
| |
| xml.sax |
| ------- |
| |
| * SAX parsers now support a character stream of |
| :class:`~xml.sax.xmlreader.InputSource` object. |
| (Contributed by Serhiy Storchaka in :issue:`2175`.) |
| |
| faulthandler |
| ------------ |
| |
| * :func:`~faulthandler.enable`, :func:`~faulthandler.register`, |
| :func:`~faulthandler.dump_traceback` and |
| :func:`~faulthandler.dump_traceback_later` functions now accept file |
| descriptors. (Contributed by Wei Wu in :issue:`23566`.) |
| |
| zipfile |
| ------- |
| |
| * Added support for writing ZIP files to unseekable streams. |
| (Contributed by Serhiy Storchaka in :issue:`23252`.) |
| |
| * The :func:`zipfile.ZipFile.open` function now supports ``'x'`` (exclusive |
| creation) mode. (Contributed by Serhiy Storchaka in :issue:`21717`.) |
| |
| |
| Optimizations |
| ============= |
| |
| The following performance enhancements have been added: |
| |
| * :func:`os.walk` has been sped up by 3-5x on POSIX systems and 7-20x |
| on Windows. This was done using the new :func:`os.scandir` function, |
| which exposes file information from the underlying ``readdir`` and |
| ``FindFirstFile``/``FindNextFile`` system calls. (Contributed by |
| Ben Hoyt with help from Victor Stinner in :issue:`23605`.) |
| |
| * Construction of ``bytes(int)`` (filled by zero bytes) is faster and uses less |
| memory for large objects. ``calloc()`` is used instead of ``malloc()`` to |
| allocate memory for these objects. |
| |
| * Some operations on :class:`~ipaddress.IPv4Network` and |
| :class:`~ipaddress.IPv6Network` have been massively sped up, such as |
| :meth:`~ipaddress.IPv4Network.subnets`, :meth:`~ipaddress.IPv4Network.supernet`, |
| :func:`~ipaddress.summarize_address_range`, :func:`~ipaddress.collapse_addresses`. |
| The speed up can range from 3x to 15x. |
| (:issue:`21486`, :issue:`21487`, :issue:`20826`) |
| |
| * Many operations on :class:`io.BytesIO` are now 50% to 100% faster. |
| (Contributed by Serhiy Storchaka in :issue:`15381` and David Wilson in |
| :issue:`22003`.) |
| |
| * :func:`marshal.dumps` is now faster (65%-85% with versions 3--4, 20-25% with |
| versions 0--2 on typical data, and up to 5x in best cases). |
| (Contributed by Serhiy Storchaka in :issue:`20416` and :issue:`23344`.) |
| |
| * The UTF-32 encoder is now 3x to 7x faster. (Contributed by Serhiy Storchaka |
| in :issue:`15027`.) |
| |
| |
| Build and C API Changes |
| ======================= |
| |
| Changes to Python's build process and to the C API include: |
| |
| * New ``calloc`` functions: |
| |
| * :c:func:`PyMem_RawCalloc` |
| * :c:func:`PyMem_Calloc` |
| * :c:func:`PyObject_Calloc` |
| * :c:func:`_PyObject_GC_Calloc` |
| |
| |
| Deprecated |
| ========== |
| |
| New Keywords |
| ------------ |
| |
| ``async`` and ``await`` are not recommended to be used as variable, class or |
| function names. Introduced by :pep:`492` in Python 3.5, they will become |
| proper keywords in Python 3.7. |
| |
| |
| Unsupported Operating Systems |
| ----------------------------- |
| |
| * Windows XP - Per :PEP:`11`, Microsoft support of Windows XP has ended. |
| |
| |
| Deprecated Python modules, functions and methods |
| ------------------------------------------------ |
| |
| * The :mod:`formatter` module has now graduated to full deprecation and is still |
| slated for removal in Python 3.6. |
| |
| * :mod:`smtpd` has in the past always decoded the DATA portion of email |
| messages using the ``utf-8`` codec. This can now be controlled by the new |
| *decode_data* keyword to :class:`~smtpd.SMTPServer`. The default value is |
| ``True``, but this default is deprecated. Specify the *decode_data* keyword |
| with an appropriate value to avoid the deprecation warning. |
| |
| * Directly assigning values to the :attr:`~http.cookies.Morsel.key`, |
| :attr:`~http.cookies.Morsel.value` and |
| :attr:`~http.cookies.Morsel.coded_value` of :class:`~http.cookies.Morsel` |
| objects is deprecated. Use the :func:`~http.cookies.Morsel.set` method |
| instead. In addition, the undocumented *LegalChars* parameter of |
| :func:`~http.cookies.Morsel.set` is deprecated, and is now ignored. |
| |
| * Passing a format string as keyword argument *format_string* to the |
| :meth:`~string.Formatter.format` method of the :class:`string.Formatter` |
| class has been deprecated. |
| |
| * :func:`platform.dist` and :func:`platform.linux_distribution` functions are |
| now deprecated and will be removed in Python 3.7. Linux distributions use |
| too many different ways of describing themselves, so the functionality is |
| left to a package. |
| (Contributed by Vajrasky Kok and Berker Peksag in :issue:`1322`.) |
| |
| * The previously undocumented ``from_function`` and ``from_builtin`` methods of |
| :class:`inspect.Signature` are deprecated. Use new |
| :meth:`inspect.Signature.from_callable` instead. (Contributed by Yury |
| Selivanov in :issue:`24248`.) |
| |
| * :func:`inspect.getargspec` is deprecated and scheduled to be removed in |
| Python 3.6. (See :issue:`20438` for details.) |
| |
| * :func:`~inspect.getfullargspec`, :func:`~inspect.getargvalues`, |
| :func:`~inspect.getcallargs`, :func:`~inspect.getargvalues`, |
| :func:`~inspect.formatargspec`, and :func:`~inspect.formatargvalues` are |
| deprecated in favor of :func:`inspect.signature` API. (See :issue:`20438` |
| for details.) |
| |
| |
| Deprecated functions and types of the C API |
| ------------------------------------------- |
| |
| * None yet. |
| |
| |
| Deprecated features |
| ------------------- |
| |
| * None yet. |
| |
| |
| Removed |
| ======= |
| |
| API and Feature Removals |
| ------------------------ |
| |
| The following obsolete and previously deprecated APIs and features have been |
| removed: |
| |
| * The ``__version__`` attribute has been dropped from the email package. The |
| email code hasn't been shipped separately from the stdlib for a long time, |
| and the ``__version__`` string was not updated in the last few releases. |
| |
| * The internal ``Netrc`` class in the :mod:`ftplib` module was deprecated in |
| 3.4, and has now been removed. |
| (Contributed by Matt Chaput in :issue:`6623`.) |
| |
| * The concept of ``.pyo`` files has been removed. |
| |
| * The JoinableQueue class in the provisional asyncio module was deprecated |
| in 3.4.4 and is now removed (:issue:`23464`). |
| |
| |
| Porting to Python 3.5 |
| ===================== |
| |
| This section lists previously described changes and other bugfixes |
| that may require changes to your code. |
| |
| Changes in the Python API |
| ------------------------- |
| |
| * :pep:`475`: Examples of functions which are now retried when interrupted |
| instead of raising :exc:`InterruptedError` if the signal handler does not |
| raise an exception: |
| |
| - :func:`open`, :func:`os.open`, :func:`io.open` |
| - functions of the :mod:`faulthandler` module |
| - :mod:`os` functions: |
| |
| * :func:`os.fchdir` |
| * :func:`os.fchmod` |
| * :func:`os.fchown` |
| * :func:`os.fdatasync` |
| * :func:`os.fstat` |
| * :func:`os.fstatvfs` |
| * :func:`os.fsync` |
| * :func:`os.ftruncate` |
| * :func:`os.mkfifo` |
| * :func:`os.mknod` |
| * :func:`os.posix_fadvise` |
| * :func:`os.posix_fallocate` |
| * :func:`os.pread` |
| * :func:`os.pwrite` |
| * :func:`os.read` |
| * :func:`os.readv` |
| * :func:`os.sendfile` |
| * :func:`os.wait3` |
| * :func:`os.wait4` |
| * :func:`os.wait` |
| * :func:`os.waitid` |
| * :func:`os.waitpid` |
| * :func:`os.write` |
| * :func:`os.writev` |
| * special cases: :func:`os.close` and :func:`os.dup2` now ignore |
| :py:data:`~errno.EINTR` error, the syscall is not retried (see the PEP |
| for the rationale) |
| |
| - :func:`select.select`, :func:`select.poll.poll`, :func:`select.epoll.poll`, |
| :func:`select.kqueue.control`, :func:`select.devpoll.poll` |
| - :func:`socket.socket` methods: |
| |
| * :meth:`~socket.socket.accept` |
| * :meth:`~socket.socket.connect` (except for non-blocking sockets) |
| * :meth:`~socket.socket.recv` |
| * :meth:`~socket.socket.recvfrom` |
| * :meth:`~socket.socket.recvmsg` |
| * :meth:`~socket.socket.send` |
| * :meth:`~socket.socket.sendall` |
| * :meth:`~socket.socket.sendmsg` |
| * :meth:`~socket.socket.sendto` |
| |
| - :func:`signal.sigtimedwait`, :func:`signal.sigwaitinfo` |
| - :func:`time.sleep` |
| |
| * Before Python 3.5, a :class:`datetime.time` object was considered to be false |
| if it represented midnight in UTC. This behavior was considered obscure and |
| error-prone and has been removed in Python 3.5. See :issue:`13936` for full |
| details. |
| |
| * :meth:`ssl.SSLSocket.send()` now raises either :exc:`ssl.SSLWantReadError` |
| or :exc:`ssl.SSLWantWriteError` on a non-blocking socket if the operation |
| would block. Previously, it would return 0. See :issue:`20951`. |
| |
| * The ``__name__`` attribute of generator is now set from the function name, |
| instead of being set from the code name. Use ``gen.gi_code.co_name`` to |
| retrieve the code name. Generators also have a new ``__qualname__`` |
| attribute, the qualified name, which is now used for the representation |
| of a generator (``repr(gen)``). See :issue:`21205`. |
| |
| * The deprecated "strict" mode and argument of :class:`~html.parser.HTMLParser`, |
| :meth:`HTMLParser.error`, and the :exc:`HTMLParserError` exception have been |
| removed. (Contributed by Ezio Melotti in :issue:`15114`.) |
| The *convert_charrefs* argument of :class:`~html.parser.HTMLParser` is |
| now ``True`` by default. (Contributed by Berker Peksag in :issue:`21047`.) |
| |
| * Although it is not formally part of the API, it is worth noting for porting |
| purposes (ie: fixing tests) that error messages that were previously of the |
| form "'sometype' does not support the buffer protocol" are now of the form "a |
| bytes-like object is required, not 'sometype'". (Contributed by Ezio Melotti |
| in :issue:`16518`.) |
| |
| * If the current directory is set to a directory that no longer exists then |
| :exc:`FileNotFoundError` will no longer be raised and instead |
| :meth:`~importlib.machinery.FileFinder.find_spec` will return ``None`` |
| **without** caching ``None`` in :data:`sys.path_importer_cache` which is |
| different than the typical case (:issue:`22834`). |
| |
| * HTTP status code and messages from :mod:`http.client` and :mod:`http.server` |
| were refactored into a common :class:`~http.HTTPStatus` enum. The values in |
| :mod:`http.client` and :mod:`http.server` remain available for backwards |
| compatibility. (Contributed by Demian Brecht in :issue:`21793`.) |
| |
| * When an import loader defines :meth:`~importlib.machinery.Loader.exec_module` |
| it is now expected to also define |
| :meth:`~importlib.machinery.Loader.create_module` (raises a |
| :exc:`DeprecationWarning` now, will be an error in Python 3.6). If the loader |
| inherits from :class:`importlib.abc.Loader` then there is nothing to do, else |
| simply define :meth:`~importlib.machinery.Loader.create_module` to return |
| ``None`` (:issue:`23014`). |
| |
| * :func:`re.split` always ignored empty pattern matches, so the ``'x*'`` |
| pattern worked the same as ``'x+'``, and the ``'\b'`` pattern never worked. |
| Now :func:`re.split` raises a warning if the pattern could match |
| an empty string. For compatibility use patterns that never match an empty |
| string (e.g. ``'x+'`` instead of ``'x*'``). Patterns that could only match |
| an empty string (such as ``'\b'``) now raise an error. |
| |
| * The :class:`~http.cookies.Morsel` dict-like interface has been made self |
| consistent: morsel comparison now takes the :attr:`~http.cookies.Morsel.key` |
| and :attr:`~http.cookies.Morsel.value` into account, |
| :meth:`~http.cookies.Morsel.copy` now results in a |
| :class:`~http.cookies.Morsel` instance rather than a :class:`dict`, and |
| :meth:`~http.cookies.Morsel.update` will now raise an exception if any of the |
| keys in the update dictionary are invalid. In addition, the undocumented |
| *LegalChars* parameter of :func:`~http.cookies.Morsel.set` is deprecated and |
| is now ignored. (:issue:`2211`) |
| |
| * :pep:`488` has removed ``.pyo`` files from Python and introduced the optional |
| ``opt-`` tag in ``.pyc`` file names. The |
| :func:`importlib.util.cache_from_source` has gained an *optimization* |
| parameter to help control the ``opt-`` tag. Because of this, the |
| *debug_override* parameter of the function is now deprecated. `.pyo` files |
| are also no longer supported as a file argument to the Python interpreter and |
| thus serve no purpose when distributed on their own (i.e. sourcless code |
| distribution). Due to the fact that the magic number for bytecode has changed |
| in Python 3.5, all old `.pyo` files from previous versions of Python are |
| invalid regardless of this PEP. |
| |
| * The :mod:`socket` module now exports the CAN_RAW_FD_FRAMES constant on linux |
| 3.6 and greater. |
| |
| * The `pygettext.py` Tool now uses the standard +NNNN format for timezones in |
| the POT-Creation-Date header. |
| |
| * The :mod:`smtplib` module now uses :data:`sys.stderr` instead of previous |
| module level :data:`stderr` variable for debug output. If your (test) |
| program depends on patching the module level variable to capture the debug |
| output, you will need to update it to capture sys.stderr instead. |
| |
| * The :meth:`str.startswith` and :meth:`str.endswith` methods no longer return |
| ``True`` when finding the empty string and the indexes are completely out of |
| range. See :issue:`24284`. |
| |
| Changes in the C API |
| -------------------- |
| |
| * The undocumented :c:member:`~PyMemoryViewObject.format` member of the |
| (non-public) :c:type:`PyMemoryViewObject` structure has been removed. |
| |
| All extensions relying on the relevant parts in ``memoryobject.h`` |
| must be rebuilt. |
| |
| * The :c:type:`PyMemAllocator` structure was renamed to |
| :c:type:`PyMemAllocatorEx` and a new ``calloc`` field was added. |
| |
| * Removed non-documented macro :c:macro:`PyObject_REPR` which leaked references. |
| Use format character ``%R`` in :c:func:`PyUnicode_FromFormat`-like functions |
| to format the :func:`repr` of the object. |
| |
| * Because the lack of the :attr:`__module__` attribute breaks pickling and |
| introspection, a deprecation warning now is raised for builtin type without |
| the :attr:`__module__` attribute. Would be an AttributeError in future. |
| (:issue:`20204`) |
| |
| * As part of PEP 492 implementation, ``tp_reserved`` slot of |
| :c:type:`PyTypeObject` was replaced with a |
| :c:member:`PyTypeObject.tp_as_async` slot. |