blob: 7a51a9dbfb83adea5b7ba55e8774ecef31bda4cc [file] [log] [blame]
Pablo Galindod4fe0982020-05-19 03:33:01 +01001****************************
2 What's New In Python 3.10
3****************************
4
5:Release: |release|
6:Date: |today|
7
8.. Rules for maintenance:
9
10 * Anyone can add text to this document. Do not spend very much time
11 on the wording of your changes, because your text will probably
12 get rewritten to some degree.
13
14 * The maintainer will go through Misc/NEWS periodically and add
15 changes; it's therefore more important to add your changes to
16 Misc/NEWS than to this file.
17
18 * This is not a complete list of every single change; completeness
19 is the purpose of Misc/NEWS. Some changes I consider too small
20 or esoteric to include. If such a change is added to the text,
21 I'll just remove it. (This is another reason you shouldn't spend
22 too much time on writing your addition.)
23
24 * If you want to draw your new text to the attention of the
25 maintainer, add 'XXX' to the beginning of the paragraph or
26 section.
27
28 * It's OK to just add a fragmentary note about a change. For
29 example: "XXX Describe the transmogrify() function added to the
30 socket module." The maintainer will research the change and
31 write the necessary text.
32
33 * You can comment out your additions if you like, but it's not
34 necessary (especially when a final release is some months away).
35
36 * Credit the author of a patch or bugfix. Just the name is
37 sufficient; the e-mail address isn't necessary.
38
39 * It's helpful to add the bug/patch number as a comment:
40
41 XXX Describe the transmogrify() function added to the socket
42 module.
43 (Contributed by P.Y. Developer in :issue:`12345`.)
44
45 This saves the maintainer the effort of going through the Mercurial log
46 when researching a change.
47
48This article explains the new features in Python 3.10, compared to 3.9.
49
Ned Deily29251b72020-05-19 07:39:29 -040050For full details, see the :ref:`changelog <changelog>`.
Pablo Galindod4fe0982020-05-19 03:33:01 +010051
52.. note::
53
54 Prerelease users should be aware that this document is currently in draft
55 form. It will be updated substantially as Python 3.10 moves towards release,
56 so it's worth checking back even after reading earlier versions.
57
58
59Summary -- Release highlights
60=============================
61
62.. This section singles out the most important changes in Python 3.10.
63 Brevity is key.
64
65
66.. PEP-sized items next.
67
68
69
70New Features
71============
72
Batuhan Taskaya044a1042020-10-06 23:03:02 +030073.. _whatsnew310-pep563:
74
75PEP 563: Postponed Evaluation of Annotations Becomes Default
76------------------------------------------------------------
77
78In Python 3.7, postponed evaluation of annotations was added,
79to be enabled with a ``from __future__ import annotations``
80directive. In 3.10 this became the default behavior, even
81without that future directive. With this being default, all
82annotations stored in :attr:`__annotations__` will be strings.
83If needed, annotations can be resolved at runtime using
84:func:`typing.get_type_hints`. See :pep:`563` for a full
85description. Also, the :func:`inspect.signature` will try to
86resolve types from now on, and when it fails it will fall back to
87showing the string annotations. (Contributed by Batuhan Taskaya
88in :issue:`38605`.)
89
Niklas Fiekas8bd216d2020-05-29 18:28:02 +020090* The :class:`int` type has a new method :meth:`int.bit_count`, returning the
91 number of ones in the binary expansion of a given integer, also known
92 as the population count. (Contributed by Niklas Fiekas in :issue:`29882`.)
Pablo Galindod4fe0982020-05-19 03:33:01 +010093
Dennis Sweeney3ee0e482020-06-12 13:19:25 -040094* The views returned by :meth:`dict.keys`, :meth:`dict.values` and
95 :meth:`dict.items` now all have a ``mapping`` attribute that gives a
96 :class:`types.MappingProxyType` object wrapping the original
97 dictionary. (Contributed by Dennis Sweeney in :issue:`40890`.)
98
Ram Rachum59cf8532020-06-19 23:39:22 +030099* :pep:`618`: The :func:`zip` function now has an optional ``strict`` flag, used
100 to require that all the iterables have an equal length.
101
Mikhail Golubev4f3c2502020-10-08 00:44:31 +0300102PEP 613: TypeAlias Annotation
103-----------------------------
104
105:pep:`484` introduced the concept of type aliases, only requiring them to be
106top-level unannotated assignments. This simplicity sometimes made it difficult
107for type checkers to distinguish between type aliases and ordinary assignments,
108especially when forward references or invalid types were involved. Compare::
109
110 StrCache = 'Cache[str]' # a type alias
111 LOG_PREFIX = 'LOG[DEBUG]' # a module constant
112
113Now the :mod:`typing` module has a special annotation :data:`TypeAlias` to
114declare type aliases more explicitly::
115
116 StrCache: TypeAlias = 'Cache[str]' # a type alias
117 LOG_PREFIX = 'LOG[DEBUG]' # a module constant
118
119See :pep:`613` for more details.
120
121(Contributed by Mikhail Golubev in :issue:`41923`.)
122
kj8d17d2b2020-11-25 11:59:59 +0700123PEP 604: New Type Union Operator
124--------------------------------
Fidget-Spinner8e1dd552020-10-05 12:40:52 +0800125
126A new type union operator was introduced which enables the syntax ``X | Y``.
127This provides a cleaner way of expressing 'either type X or type Y' instead of
128using :data:`typing.Union`, especially in type hints (annotations).
129
130In previous versions of Python, to apply a type hint for functions accepting
131arguments of multiple types, :data:`typing.Union` was used::
132
133 def square(number: Union[int, float]) -> Union[int, float]:
134 return number ** 2
135
136
kjd21cb2d2020-10-31 23:08:17 +0800137Type hints can now be written in a more succinct manner::
Fidget-Spinner8e1dd552020-10-05 12:40:52 +0800138
139 def square(number: int | float) -> int | float:
140 return number ** 2
141
142
143See :pep:`604` for more details.
144
145(Contributed by Maggie Moss and Philippe Prados in :issue:`41428`.)
Pablo Galindod4fe0982020-05-19 03:33:01 +0100146
Ken Jin11276cd2021-01-02 08:45:50 +0800147PEP 612: Parameter Specification Variables
148------------------------------------------
149
150Two new options to improve the information provided to static type checkers for
151:pep:`484`\ 's ``Callable`` have been added to the :mod:`typing` module.
152
153The first is the parameter specification variable. They are used to forward the
154parameter types of one callable to another callable -- a pattern commonly
155found in higher order functions and decorators. Examples of usage can be found
156in :class:`typing.ParamSpec`. Previously, there was no easy way to type annotate
157dependency of parameter types in such a precise manner.
158
159The second option is the new ``Concatenate`` operator. It's used in conjunction
160with parameter specification variables to type annotate a higher order callable
161which adds or removes parameters of another callable. Examples of usage can
162be found in :class:`typing.Concatenate`.
163
164See :class:`typing.Callable`, :class:`typing.ParamSpec`,
165:class:`typing.Concatenate` and :pep:`612` for more details.
166
167(Contributed by Ken Jin in :issue:`41559`.)
168
Pablo Galindod4fe0982020-05-19 03:33:01 +0100169Other Language Changes
170======================
171
Serhiy Storchaka578c3952020-05-26 18:43:38 +0300172* Builtin and extension functions that take integer arguments no longer accept
173 :class:`~decimal.Decimal`\ s, :class:`~fractions.Fraction`\ s and other
174 objects that can be converted to integers only with a loss (e.g. that have
175 the :meth:`~object.__int__` method but do not have the
176 :meth:`~object.__index__` method).
177 (Contributed by Serhiy Storchaka in :issue:`37999`.)
Pablo Galindod4fe0982020-05-19 03:33:01 +0100178
Lysandros Nikolaoua85fefe2020-11-19 01:49:28 +0200179* Assignment expressions can now be used unparenthesized within set literals
180 and set comprehensions, as well as in sequence indexes (but not slices).
181
Pablo Galindod4fe0982020-05-19 03:33:01 +0100182
183New Modules
184===========
185
186* None yet.
187
188
189Improved Modules
190================
191
Tomáš Hrnčiarfb35fa42021-01-12 01:41:35 +0100192argparse
193--------
194
195Misleading phrase "optional arguments" was replaced with "options" in argparse help. Some tests might require adaptation if they rely on exact output match.
196(Contributed by Raymond Hettinger in :issue:`9694`.)
197
Filipe Laíns4ce6faa2020-08-10 15:48:20 +0100198base64
199------
200
201Add :func:`base64.b32hexencode` and :func:`base64.b32hexdecode` to support the
202Base32 Encoding with Extended Hex Alphabet.
203
Hai Shid332e7b2020-09-29 05:41:11 +0800204codecs
205------
206
207Add a :func:`codecs.unregister` function to unregister a codec search function.
208(Contributed by Hai Shi in :issue:`41842`.)
209
kjd75f6f72020-12-19 01:39:26 +0800210collections.abc
211---------------
212
213The ``__args__`` of the :ref:`parameterized generic <types-genericalias>` for
214:class:`collections.abc.Callable` are now consistent with :data:`typing.Callable`.
215:class:`collections.abc.Callable` generic now flattens type parameters, similar
216to what :data:`typing.Callable` currently does. This means that
217``collections.abc.Callable[[int, str], str]`` will have ``__args__`` of
218``(int, str, str)``; previously this was ``([int, str], str)``. To allow this
219change, :class:`types.GenericAlias` can now be subclassed, and a subclass will
220be returned when subscripting the :class:`collections.abc.Callable` type. Note
221that a :exc:`TypeError` may be raised for invalid forms of parameterizing
222:class:`collections.abc.Callable` which may have passed silently in Python 3.9.
223(Contributed by Ken Jin in :issue:`42195`.)
224
Joongi Kim3eb28462020-11-11 00:19:11 +0900225contextlib
226----------
227
228Add a :func:`contextlib.aclosing` context manager to safely close async generators
229and objects representing asynchronously released resources.
230(Contributed by Joongi Kim and John Belmonte in :issue:`41229`.)
231
Tom Gringauz9c98e8c2020-11-18 00:58:35 +0200232Add asynchronous context manager support to :func:`contextlib.nullcontext`.
233(Contributed by Tom Gringauz in :issue:`41543`.)
234
Hans Petter Janssonda4e09f2020-08-03 22:51:33 -0500235curses
236------
237
238The extended color functions added in ncurses 6.1 will be used transparently
239by :func:`curses.color_content`, :func:`curses.init_color`,
240:func:`curses.init_pair`, and :func:`curses.pair_content`. A new function,
241:func:`curses.has_extended_color_support`, indicates whether extended color
242support is provided by the underlying ncurses library.
243(Contributed by Jeffrey Kintscher and Hans Petter Jansson in :issue:`36982`.)
244
Victor Stinner0e2a0f72021-01-09 00:35:01 +0100245distutils
246---------
247
248The ``bdist_wininst`` command deprecated in Python 3.8 has been removed.
249The ``bidst_wheel`` command is now recommended to distribute binary packages
250on Windows.
251(Contributed by Victor Stinner in :issue:`42802`.)
252
Brett Cannon825ac382020-11-06 18:45:56 -0800253doctest
254-------
255
256When a module does not define ``__loader__``, fall back to ``__spec__.loader``.
257(Contributed by Brett Cannon in :issue:`42133`.)
258
Hai Shic5b049b2020-10-14 23:43:31 +0800259encodings
260---------
261:func:`encodings.normalize_encoding` now ignores non-ASCII characters.
262(Contributed by Hai Shi in :issue:`39337`.)
263
Serhiy Storchaka8a64cea2020-06-18 22:08:27 +0300264glob
265----
266
267Added the *root_dir* and *dir_fd* parameters in :func:`~glob.glob` and
268:func:`~glob.iglob` which allow to specify the root directory for searching.
269(Contributed by Serhiy Storchaka in :issue:`38144`.)
270
Brett Cannon825ac382020-11-06 18:45:56 -0800271inspect
272-------
273
274When a module does not define ``__loader__``, fall back to ``__spec__.loader``.
275(Contributed by Brett Cannon in :issue:`42133`.)
276
Batuhan Taskayaeee1c772020-12-24 01:45:13 +0300277Added *globalns* and *localns* parameters in :func:`~inspect.signature` and
278:meth:`inspect.Signature.from_callable` to retrieve the annotations in given
279local and global namespaces.
280(Contributed by Batuhan Taskaya in :issue:`41960`.)
281
Brett Cannon825ac382020-11-06 18:45:56 -0800282linecache
283---------
284
285When a module does not define ``__loader__``, fall back to ``__spec__.loader``.
286(Contributed by Brett Cannon in :issue:`42133`.)
287
pxinwr3405e052020-08-07 13:21:52 +0800288os
289--
290
291Added :func:`os.cpu_count()` support for VxWorks RTOS.
292(Contributed by Peixing Xin in :issue:`41440`.)
293
Christian Heimescd9fed62020-11-13 19:48:52 +0100294Added a new function :func:`os.eventfd` and related helpers to wrap the
295``eventfd2`` syscall on Linux.
296(Contributed by Christian Heimes in :issue:`41001`.)
297
Pablo Galindoa57b3d32020-11-17 00:00:38 +0000298Added :func:`os.splice()` that allows to move data between two file
299descriptors without copying between kernel address space and user
300address space, where one of the file descriptors must refer to a
301pipe. (Contributed by Pablo Galindo in :issue:`41625`.)
302
Joshua Cannon45205842020-11-20 09:40:39 -0600303pathlib
304-------
305
Yaroslav Pankovych79d2e622020-11-23 22:06:22 +0200306Added slice support to :attr:`PurePath.parents <pathlib.PurePath.parents>`.
Joshua Cannon45205842020-11-20 09:40:39 -0600307(Contributed by Joshua Cannon in :issue:`35498`)
308
Yaroslav Pankovych79d2e622020-11-23 22:06:22 +0200309Added negative indexing support to :attr:`PurePath.parents
310<pathlib.PurePath.parents>`.
311(Contributed by Yaroslav Pankovych in :issue:`21041`)
312
Christian Heimes5c73afc2020-11-30 22:34:45 +0100313platform
314--------
315
316Added :func:`platform.freedesktop_os_release()` to retrieve operation system
317identification from `freedesktop.org os-release
318<https://www.freedesktop.org/software/systemd/man/os-release.html>`_ standard file.
319(Contributed by Christian Heimes in :issue:`28468`)
320
Gregory Schevchenkodaff3902020-07-25 22:58:45 +0300321py_compile
322----------
323
324Added ``--quiet`` option to command-line interface of :mod:`py_compile`.
325(Contributed by Gregory Schevchenko in :issue:`38731`.)
326
Zackery Spytzdf592732020-10-29 03:44:35 -0600327shelve
328------
329
330The :mod:`shelve` module now uses :data:`pickle.DEFAULT_PROTOCOL` by default
331instead of :mod:`pickle` protocol ``3`` when creating shelves.
332(Contributed by Zackery Spytz in :issue:`34204`.)
333
Brett Cannon825ac382020-11-06 18:45:56 -0800334site
335----
336
337When a module does not define ``__loader__``, fall back to ``__spec__.loader``.
338(Contributed by Brett Cannon in :issue:`42133`.)
339
Christian Heimes03c8ddd2020-11-20 09:26:07 +0100340socket
341------
342
343The exception :exc:`socket.timeout` is now an alias of :exc:`TimeoutError`.
344(Contributed by Christian Heimes in :issue:`42413`.)
345
Victor Stinnerdd8a93e2020-06-30 00:49:03 +0200346sys
347---
348
349Add :data:`sys.orig_argv` attribute: the list of the original command line
350arguments passed to the Python executable.
351(Contributed by Victor Stinner in :issue:`23427`.)
352
Mario Corchero0001a1b2020-11-04 10:27:43 +0100353threading
354---------
355
356Added :func:`threading.gettrace` and :func:`threading.getprofile` to
357retrieve the functions set by :func:`threading.settrace` and
358:func:`threading.setprofile` respectively.
359(Contributed by Mario Corchero in :issue:`42251`.)
360
Mario Corchero750c5ab2020-11-12 18:27:44 +0100361Add :data:`threading.__excepthook__` to allow retrieving the original value
362of :func:`threading.excepthook` in case it is set to a broken or a different
363value.
364(Contributed by Mario Corchero in :issue:`42308`.)
365
Zackery Spytz91e93792020-11-05 15:18:44 -0700366traceback
367---------
368
369The :func:`~traceback.format_exception`,
370:func:`~traceback.format_exception_only`, and
371:func:`~traceback.print_exception` functions can now take an exception object
372as a positional-only argument.
373(Contributed by Zackery Spytz and Matthias Bussonnier in :issue:`26389`.)
374
Bas van Beek0d0e9fe2020-09-22 17:55:34 +0200375types
376-----
377
378Reintroduced the :data:`types.EllipsisType`, :data:`types.NoneType`
379and :data:`types.NotImplementedType` classes, providing a new set
380of types readily interpretable by type checkers.
381(Contributed by Bas van Beek in :issue:`41810`.)
382
kj46873382020-11-19 11:44:24 +0700383typing
384------
385
386The behavior of :class:`typing.Literal` was changed to conform with :pep:`586`
387and to match the behavior of static type checkers specified in the PEP.
388
3891. ``Literal`` now de-duplicates parameters.
3902. Equality comparisons between ``Literal`` objects are now order independent.
3913. ``Literal`` comparisons now respects types. For example,
392 ``Literal[0] == Literal[False]`` previously evaluated to ``True``. It is
393 now ``False``. To support this change, the internally used type cache now
394 supports differentiating types.
3954. ``Literal`` objects will now raise a :exc:`TypeError` exception during
396 equality comparisons if one of their parameters are not :term:`immutable`.
397 Note that declaring ``Literal`` with mutable parameters will not throw
398 an error::
399
400 >>> from typing import Literal
401 >>> Literal[{0}]
402 >>> Literal[{0}] == Literal[{False}]
403 Traceback (most recent call last):
404 File "<stdin>", line 1, in <module>
405 TypeError: unhashable type: 'set'
406
407(Contributed by Yurii Karabas in :issue:`42345`.)
408
Mark Dickinsonc8c70e72020-09-19 21:38:11 +0100409unittest
410--------
411
412Add new method :meth:`~unittest.TestCase.assertNoLogs` to complement the
413existing :meth:`~unittest.TestCase.assertLogs`. (Contributed by Kit Yan Choi
414in :issue:`39385`.)
415
Zackery Spytze28b8c92020-08-09 04:50:53 -0600416xml
417---
418
419Add a :class:`~xml.sax.handler.LexicalHandler` class to the
420:mod:`xml.sax.handler` module.
421(Contributed by Jonathan Gossage and Zackery Spytz in :issue:`35018`.)
422
Brett Cannond2e94bb2020-11-13 15:14:58 -0800423zipimport
424---------
425Add methods related to :pep:`451`: :meth:`~zipimport.zipimporter.find_spec`,
426:meth:`zipimport.zipimporter.create_module`, and
427:meth:`zipimport.zipimporter.exec_module`.
428(Contributed by Brett Cannon in :issue:`42131`.
429
Serhiy Storchaka8a64cea2020-06-18 22:08:27 +0300430
Pablo Galindod4fe0982020-05-19 03:33:01 +0100431Optimizations
432=============
433
Serhiy Storchaka12f43342020-07-20 15:53:55 +0300434* Constructors :func:`str`, :func:`bytes` and :func:`bytearray` are now faster
435 (around 30--40% for small objects).
436 (Contributed by Serhiy Storchaka in :issue:`41334`.)
437
Victor Stinner2c2a4f32020-06-18 01:20:51 +0200438* The :mod:`runpy` module now imports fewer modules.
Victor Stinner4c18fc82020-06-17 23:58:58 +0200439 The ``python3 -m module-name`` command startup time is 1.3x faster in
440 average.
441 (Contributed by Victor Stinner in :issue:`41006`.)
442
Pablo Galindo9e8fe192021-01-03 04:37:46 +0000443* The ``LOAD_ATTR`` instruction now uses new "per opcode cache" mechanism. It
444 is about 36% faster now. This makes optimized ``LOAD_ATTR`` instructions the
445 current most performance attribute access method (faster than slots).
446 (Contributed by Pablo Galindo and Yury Selivanov in :issue:`42093`, based on
447 ideas implemented originally in PyPy and MicroPython.)
Pablo Galindod4fe0982020-05-19 03:33:01 +0100448
Pablo Galindob451b0e2020-10-21 22:46:52 +0100449* When building Python with ``--enable-optimizations`` now
450 ``-fno-semantic-interposition`` is added to both the compile and link line.
451 This speeds builds of the Python interpreter created with ``--enable-shared``
452 with ``gcc`` by up to 30%. See `this article
453 <https://developers.redhat.com/blog/2020/06/25/red-hat-enterprise-linux-8-2-brings-faster-python-3-8-run-speeds/>`_
454 for more details. (Contributed by Victor Stinner and Pablo Galindo in
Brett Cannon2de50972020-12-04 15:39:21 -0800455 :issue:`38980`.)
456
Pablo Galindob451b0e2020-10-21 22:46:52 +0100457
Yurii Karabas73019792020-11-25 12:43:18 +0200458* Function parameters and their annotations are no longer computed at runtime,
459 but rather at compilation time. They are stored as a tuple of strings at the
460 bytecode level. It is now around 100% faster to create a function with parameter
461 annotations. (Contributed by Yurii Karabas and Inada Naoki in :issue:`42202`)
462
Pablo Galindod4fe0982020-05-19 03:33:01 +0100463Deprecated
464==========
465
Brett Cannon04523c52020-10-23 18:10:54 -0700466* Starting in this release, there will be a concerted effort to begin
467 cleaning up old import semantics that were kept for Python 2.7
468 compatibility. Specifically,
469 :meth:`~importlib.abc.PathEntryFinder.find_loader`/:meth:`~importlib.abc.Finder.find_module`
470 (superseded by :meth:`~importlib.abc.Finder.find_spec`),
471 :meth:`~importlib.abc.Loader.load_module`
472 (superseded by :meth:`~importlib.abc.Loader.exec_module`),
473 :meth:`~importlib.abc.Loader.module_repr` (which the import system
474 takes care of for you), the ``__package__`` attribute
475 (superseded by ``__spec__.parent``), the ``__loader__`` attribute
476 (superseded by ``__spec__.loader``), and the ``__cached__`` attribute
477 (superseded by ``__spec__.cached``) will slowly be removed (as well
478 as other classes and methods in :mod:`importlib`).
479 :exc:`ImportWarning` and/or :exc:`DeprecationWarning` will be raised
480 as appropriate to help identify code which needs updating during
481 this transition.
482
Brett Cannon2de50972020-12-04 15:39:21 -0800483* The various ``load_module()`` methods of :mod:`importlib` have been
484 documented as deprecated since Python 3.6, but will now also trigger
485 a :exc:`DeprecationWarning`. Use
486 :meth:`~importlib.abc.Loader.exec_module` instead.
487 (Contributed by Brett Cannon in :issue:`26131`.)
488
489* :meth:`zimport.zipimporter.load_module` has been deprecated in
490 preference for :meth:`~zipimport.zipimporter.exec_module`.
491 (Contributed by Brett Cannon in :issue:`26131`.)
492
493* The use of :meth:`~importlib.abc.Loader.load_module` by the import
494 system now triggers an :exc:`ImportWarning` as
495 :meth:`~importlib.abc.Loader.exec_module` is preferred.
496 (Contributed by Brett Cannon in :issue:`26131`.)
497
Erlend Egeberg Aaslanda1f401a2020-11-17 16:55:12 +0100498* ``sqlite3.OptimizedUnicode`` has been undocumented and obsolete since Python
499 3.3, when it was made an alias to :class:`str`. It is now deprecated,
500 scheduled for removal in Python 3.12.
501 (Contributed by Erlend E. Aasland in :issue:`42264`.)
502
Erlend Egeberg Aaslandddb5e112021-01-06 01:36:04 +0100503* The undocumented built-in function ``sqlite3.enable_shared_cache`` is now
504 deprecated, scheduled for removal in Python 3.12. Its use is strongly
505 discouraged by the SQLite3 documentation. See `the SQLite3 docs
506 <https://sqlite.org/c3ref/enable_shared_cache.html/>`_ for more details.
507 If shared cache must be used, open the database in URI mode using the
508 ``cache=shared`` query parameter.
509 (Contributed by Erlend E. Aasland in :issue:`24464`.)
510
Pablo Galindod4fe0982020-05-19 03:33:01 +0100511
Pablo Galindod4fe0982020-05-19 03:33:01 +0100512Removed
513=======
514
Serhiy Storchakae2ec0b22020-10-09 14:14:37 +0300515* Removed special methods ``__int__``, ``__float__``, ``__floordiv__``,
516 ``__mod__``, ``__divmod__``, ``__rfloordiv__``, ``__rmod__`` and
517 ``__rdivmod__`` of the :class:`complex` class. They always raised
518 a :exc:`TypeError`.
519 (Contributed by Serhiy Storchaka in :issue:`41974`.)
520
Berker Peksagd4d127f2020-07-16 09:38:58 +0300521* The ``ParserBase.error()`` method from the private and undocumented ``_markupbase``
522 module has been removed. :class:`html.parser.HTMLParser` is the only subclass of
523 ``ParserBase`` and its ``error()`` implementation has already been removed in
524 Python 3.5.
525 (Contributed by Berker Peksag in :issue:`31844`.)
526
Victor Stinner84f73822020-10-27 04:36:22 +0100527* Removed the ``unicodedata.ucnhash_CAPI`` attribute which was an internal
528 PyCapsule object. The related private ``_PyUnicode_Name_CAPI`` structure was
529 moved to the internal C API.
530 (Contributed by Victor Stinner in :issue:`42157`.)
531
Lysandros Nikolaouc26d5912020-11-16 20:46:37 +0200532* Removed the ``parser`` module, which was deprecated in 3.9 due to the
533 switch to the new PEG parser, as well as all the C source and header files
534 that were only being used by the old parser, including ``node.h``, ``parser.h``,
535 ``graminit.h`` and ``grammar.h``.
536
537* Removed the Public C API functions :c:func:`PyParser_SimpleParseStringFlags`,
538 :c:func:`PyParser_SimpleParseStringFlagsFilename`,
539 :c:func:`PyParser_SimpleParseFileFlags` and :c:func:`PyNode_Compile`
540 that were deprecated in 3.9 due to the switch to the new PEG parser.
541
Dong-hee Nabe319c02020-11-25 22:17:30 +0900542* Removed the ``formatter`` module, which was deprecated in Python 3.4.
543 It is somewhat obsolete, little used, and not tested. It was originally
544 scheduled to be removed in Python 3.6, but such removals were delayed until
545 after Python 2.7 EOL. Existing users should copy whatever classes they use
546 into their code.
547 (Contributed by Dong-hee Na and Terry J. Reedy in :issue:`42299`.)
Pablo Galindod4fe0982020-05-19 03:33:01 +0100548
Hai Shi0f91f582020-12-08 22:42:42 +0800549* Removed the :c:func:`PyModule_GetWarningsModule` function that was useless
550 now due to the _warnings module was converted to a builtin module in 2.6.
551 (Contributed by Hai Shi in :issue:`42599`.)
552
Hugo van Kemenadec47c78b2021-01-13 01:16:37 +0200553* Remove deprecated aliases to :ref:`collections-abstract-base-classes` from
554 the :mod:`collections` module.
555 (Contributed by Victor Stinner in :issue:`37324`.)
556
Hai Shi0f91f582020-12-08 22:42:42 +0800557
Pablo Galindod4fe0982020-05-19 03:33:01 +0100558Porting to Python 3.10
559======================
560
561This section lists previously described changes and other bugfixes
562that may require changes to your code.
563
564
Zackery Spytz91e93792020-11-05 15:18:44 -0700565Changes in the Python API
566-------------------------
567
568* The *etype* parameters of the :func:`~traceback.format_exception`,
569 :func:`~traceback.format_exception_only`, and
570 :func:`~traceback.print_exception` functions in the :mod:`traceback` module
571 have been renamed to *exc*.
572 (Contributed by Zackery Spytz and Matthias Bussonnier in :issue:`26389`.)
573
Victor Stinner357704c2020-12-14 23:07:54 +0100574* :mod:`atexit`: At Python exit, if a callback registered with
575 :func:`atexit.register` fails, its exception is now logged. Previously, only
576 some exceptions were logged, and the last exception was always silently
577 ignored.
578 (Contributed by Victor Stinner in :issue:`42639`.)
579
kjd75f6f72020-12-19 01:39:26 +0800580* :class:`collections.abc.Callable` generic now flattens type parameters, similar
581 to what :data:`typing.Callable` currently does. This means that
582 ``collections.abc.Callable[[int, str], str]`` will have ``__args__`` of
583 ``(int, str, str)``; previously this was ``([int, str], str)``. Code which
584 accesses the arguments via :func:`typing.get_args` or ``__args__`` need to account
585 for this change. Furthermore, :exc:`TypeError` may be raised for invalid forms
586 of parameterizing :class:`collections.abc.Callable` which may have passed
587 silently in Python 3.9.
588 (Contributed by Ken Jin in :issue:`42195`.)
Victor Stinner357704c2020-12-14 23:07:54 +0100589
Erlend Egeberg Aaslandf4936ad2020-12-31 14:16:50 +0100590* :meth:`socket.htons` and :meth:`socket.ntohs` now raise :exc:`OverflowError`
591 instead of :exc:`DeprecationWarning` if the given parameter will not fit in
592 a 16-bit unsigned integer.
593 (Contributed by Erlend E. Aasland in :issue:`42393`.)
594
595
Yurii Karabas73019792020-11-25 12:43:18 +0200596CPython bytecode changes
597========================
598
599* The ``MAKE_FUNCTION`` instruction accepts tuple of strings as annotations
600 instead of dictionary.
601 (Contributed by Yurii Karabas and Inada Naoki in :issue:`42202`)
Dong-hee Naad3252b2020-05-26 01:52:54 +0900602
603Build Changes
604=============
605
Victor Stinner7ab92d52020-06-16 00:54:44 +0200606* The C99 functions :c:func:`snprintf` and :c:func:`vsnprintf` are now required
607 to build Python.
608 (Contributed by Victor Stinner in :issue:`36020`.)
609
Erlend Egeberg Aaslandcf0b2392021-01-06 01:02:43 +0100610* :mod:`sqlite3` requires SQLite 3.7.15 or higher. (Contributed by Sergey Fedoseev
611 and Erlend E. Aasland :issue:`40744` and :issue:`40810`.)
Erlend Egeberg Aasland207c3212020-09-07 23:26:54 +0200612
Victor Stinner357704c2020-12-14 23:07:54 +0100613* The :mod:`atexit` module must now always be built as a built-in module.
614 (Contributed by Victor Stinner in :issue:`42639`.)
Erlend Egeberg Aasland207c3212020-09-07 23:26:54 +0200615
pxinwr277ce302020-12-30 20:50:39 +0800616* Added ``--disable-test-modules`` option to the ``configure`` script:
617 don't build nor install test modules.
618 (Contributed by Xavier de Gaye, Thomas Petazzoni and Peixing Xin in :issue:`27640`.)
619
Dong-hee Naad3252b2020-05-26 01:52:54 +0900620
621C API Changes
622=============
623
624New Features
625------------
626
Victor Stinnerdd8a93e2020-06-30 00:49:03 +0200627* The result of :c:func:`PyNumber_Index` now always has exact type :class:`int`.
Serhiy Storchaka5f4b229d2020-05-28 10:33:45 +0300628 Previously, the result could have been an instance of a subclass of ``int``.
629 (Contributed by Serhiy Storchaka in :issue:`40792`.)
630
Victor Stinnerdd8a93e2020-06-30 00:49:03 +0200631* Add a new :c:member:`~PyConfig.orig_argv` member to the :c:type:`PyConfig`
632 structure: the list of the original command line arguments passed to the
633 Python executable.
634 (Contributed by Victor Stinner in :issue:`23427`.)
Dong-hee Naad3252b2020-05-26 01:52:54 +0900635
Zackery Spytz2e4dd332020-09-23 12:43:45 -0600636* The :c:func:`PyDateTime_DATE_GET_TZINFO` and
637 :c:func:`PyDateTime_TIME_GET_TZINFO` macros have been added for accessing
638 the ``tzinfo`` attributes of :class:`datetime.datetime` and
639 :class:`datetime.time` objects.
640 (Contributed by Zackery Spytz in :issue:`30155`.)
641
Hai Shid332e7b2020-09-29 05:41:11 +0800642* Add a :c:func:`PyCodec_Unregister` function to unregister a codec
643 search function.
644 (Contributed by Hai Shi in :issue:`41842`.)
645
Vladimir Matveev24a54c02020-10-12 12:10:42 -0700646* The :c:func:`PyIter_Send` function was added to allow
Vladimir Matveev037245c2020-10-09 17:15:15 -0700647 sending value into iterator without raising ``StopIteration`` exception.
648 (Contributed by Vladimir Matveev in :issue:`41756`.)
649
Alex Gaynor3a8fdb22020-10-19 18:17:50 -0400650* Added :c:func:`PyUnicode_AsUTF8AndSize` to the limited C API.
651 (Contributed by Alex Gaynor in :issue:`41784`.)
652
Victor Stinner80218752020-11-04 13:59:15 +0100653* Added :c:func:`PyModule_AddObjectRef` function: similar to
Victor Stinner95ce7cd2020-11-11 01:52:26 +0100654 :c:func:`PyModule_AddObject` but don't steal a reference to the value on
Victor Stinner80218752020-11-04 13:59:15 +0100655 success.
656 (Contributed by Victor Stinner in :issue:`1635741`.)
657
Victor Stinner53a03aa2020-11-05 15:02:12 +0100658* Added :c:func:`Py_NewRef` and :c:func:`Py_XNewRef` functions to increment the
659 reference count of an object and return the object.
660 (Contributed by Victor Stinner in :issue:`42262`.)
661
Serhiy Storchaka686c2032020-11-22 13:25:02 +0200662* The :c:func:`PyType_FromSpecWithBases` and :c:func:`PyType_FromModuleAndSpec`
663 functions now accept a single class as the *bases* argument.
664 (Contributed by Serhiy Storchaka in :issue:`42423`.)
665
Hai Shi88c2cfd2020-11-07 00:04:47 +0800666* The :c:func:`PyType_FromModuleAndSpec` function now accepts NULL ``tp_doc``
667 slot.
668 (Contributed by Hai Shi in :issue:`41832`.)
669
Hai Shia13b26c2020-11-11 04:53:46 +0800670* The :c:func:`PyType_GetSlot` function can accept static types.
671 (Contributed by Hai Shi and Petr Viktorin in :issue:`41073`.)
672
Alex Gaynor3a8fdb22020-10-19 18:17:50 -0400673
Dong-hee Naad3252b2020-05-26 01:52:54 +0900674Porting to Python 3.10
675----------------------
676
Victor Stinner37bb2892020-06-19 11:45:31 +0200677* The ``PY_SSIZE_T_CLEAN`` macro must now be defined to use
678 :c:func:`PyArg_ParseTuple` and :c:func:`Py_BuildValue` formats which use
679 ``#``: ``es#``, ``et#``, ``s#``, ``u#``, ``y#``, ``z#``, ``U#`` and ``Z#``.
680 See :ref:`Parsing arguments and building values
681 <arg-parsing>` and the :pep:`353`.
682 (Contributed by Victor Stinner in :issue:`40943`.)
683
Victor Stinnerfe2978b2020-05-27 14:55:10 +0200684* Since :c:func:`Py_REFCNT()` is changed to the inline static function,
685 ``Py_REFCNT(obj) = new_refcnt`` must be replaced with ``Py_SET_REFCNT(obj, new_refcnt)``:
Victor Stinnerdc24b8a2020-06-04 22:10:43 +0200686 see :c:func:`Py_SET_REFCNT()` (available since Python 3.9). For backward
687 compatibility, this macro can be used::
688
689 #if PY_VERSION_HEX < 0x030900A4
690 # define Py_SET_REFCNT(obj, refcnt) ((Py_REFCNT(obj) = (refcnt)), (void)0)
691 #endif
692
Victor Stinnerfe2978b2020-05-27 14:55:10 +0200693 (Contributed by Victor Stinner in :issue:`39573`.)
694
Victor Stinner59d3dce2020-06-02 14:03:25 +0200695* Calling :c:func:`PyDict_GetItem` without :term:`GIL` held had been allowed
696 for historical reason. It is no longer allowed.
697 (Contributed by Victor Stinner in :issue:`40839`.)
698
Inada Naoki038dd0f2020-06-30 15:26:56 +0900699* ``PyUnicode_FromUnicode(NULL, size)`` and ``PyUnicode_FromStringAndSize(NULL, size)``
700 raise ``DeprecationWarning`` now. Use :c:func:`PyUnicode_New` to allocate
701 Unicode object without initial data.
702 (Contributed by Inada Naoki in :issue:`36346`.)
703
Victor Stinner47e1afd2020-10-26 16:43:47 +0100704* The private ``_PyUnicode_Name_CAPI`` structure of the PyCapsule API
Victor Stinner84f73822020-10-27 04:36:22 +0100705 ``unicodedata.ucnhash_CAPI`` has been moved to the internal C API.
Victor Stinner920cb642020-10-26 19:19:36 +0100706 (Contributed by Victor Stinner in :issue:`42157`.)
Victor Stinner47e1afd2020-10-26 16:43:47 +0100707
Victor Stinnerace3f9a2020-11-10 21:10:22 +0100708* :c:func:`Py_GetPath`, :c:func:`Py_GetPrefix`, :c:func:`Py_GetExecPrefix`,
709 :c:func:`Py_GetProgramFullPath`, :c:func:`Py_GetPythonHome` and
710 :c:func:`Py_GetProgramName` functions now return ``NULL`` if called before
711 :c:func:`Py_Initialize` (before Python is initialized). Use the new
712 :ref:`Python Initialization Configuration API <init-config>` to get the
713 :ref:`Python Path Configuration. <init-path-config>`.
714 (Contributed by Victor Stinner in :issue:`42260`.)
715
Victor Stinner0ef96c22020-12-07 11:56:20 +0100716* :c:func:`PyList_SET_ITEM`, :c:func:`PyTuple_SET_ITEM` and
717 :c:func:`PyCell_SET` macros can no longer be used as l-value or r-value.
718 For example, ``x = PyList_SET_ITEM(a, b, c)`` and
719 ``PyList_SET_ITEM(a, b, c) = x`` now fail with a compiler error. It prevents
720 bugs like ``if (PyList_SET_ITEM (a, b, c) < 0) ...`` test.
721 (Contributed by Zackery Spytz and Victor Stinner in :issue:`30459`.)
722
Victor Stinner583ee5a2020-10-02 14:49:00 +0200723Deprecated
724----------
725
726* The ``PyUnicode_InternImmortal()`` function is now deprecated
727 and will be removed in Python 3.12: use :c:func:`PyUnicode_InternInPlace`
728 instead.
729 (Contributed by Victor Stinner in :issue:`41692`.)
730
Dong-hee Naad3252b2020-05-26 01:52:54 +0900731Removed
732-------
Inada Naoki6f8a6ee2020-06-26 08:07:22 +0900733
734* ``PyObject_AsCharBuffer()``, ``PyObject_AsReadBuffer()``, ``PyObject_CheckReadBuffer()``,
735 and ``PyObject_AsWriteBuffer()`` are removed. Please migrate to new buffer protocol;
736 :c:func:`PyObject_GetBuffer` and :c:func:`PyBuffer_Release`.
Inada Naoki20a79022020-06-27 18:22:09 +0900737 (Contributed by Inada Naoki in :issue:`41103`.)
738
739* Removed ``Py_UNICODE_str*`` functions manipulating ``Py_UNICODE*`` strings.
740 (Contributed by Inada Naoki in :issue:`41123`.)
741
742 * ``Py_UNICODE_strlen``: use :c:func:`PyUnicode_GetLength` or
743 :c:macro:`PyUnicode_GET_LENGTH`
744 * ``Py_UNICODE_strcat``: use :c:func:`PyUnicode_CopyCharacters` or
745 :c:func:`PyUnicode_FromFormat`
746 * ``Py_UNICODE_strcpy``, ``Py_UNICODE_strncpy``: use
747 :c:func:`PyUnicode_CopyCharacters` or :c:func:`PyUnicode_Substring`
748 * ``Py_UNICODE_strcmp``: use :c:func:`PyUnicode_Compare`
749 * ``Py_UNICODE_strncmp``: use :c:func:`PyUnicode_Tailmatch`
750 * ``Py_UNICODE_strchr``, ``Py_UNICODE_strrchr``: use
751 :c:func:`PyUnicode_FindChar`
Inada Naokid9f2a132020-06-29 10:46:51 +0900752
753* Removed ``PyUnicode_GetMax()``. Please migrate to new (:pep:`393`) APIs.
754 (Contributed by Inada Naoki in :issue:`41103`.)
Inada Naokie4f1fe62020-06-29 13:00:43 +0900755
756* Removed ``PyLong_FromUnicode()``. Please migrate to :c:func:`PyLong_FromUnicodeObject`.
757 (Contributed by Inada Naoki in :issue:`41103`.)
Inada Naokib3332662020-06-30 12:23:07 +0900758
759* Removed ``PyUnicode_AsUnicodeCopy()``. Please use :c:func:`PyUnicode_AsUCS4Copy` or
760 :c:func:`PyUnicode_AsWideCharString`
761 (Contributed by Inada Naoki in :issue:`41103`.)
Victor Stinner19c3ac92020-09-23 14:04:57 +0200762
763* Removed ``_Py_CheckRecursionLimit`` variable: it has been replaced by
764 ``ceval.recursion_limit`` of the :c:type:`PyInterpreterState` structure.
765 (Contributed by Victor Stinner in :issue:`41834`.)
Serhiy Storchakadcc54212020-10-05 12:32:00 +0300766
767* Removed undocumented macros ``Py_ALLOW_RECURSION`` and
768 ``Py_END_ALLOW_RECURSION`` and the ``recursion_critical`` field of the
769 :c:type:`PyInterpreterState` structure.
770 (Contributed by Serhiy Storchaka in :issue:`41936`.)
Victor Stinner296a7962020-11-17 16:22:23 +0100771
772* Removed the undocumented ``PyOS_InitInterrupts()`` function. Initializing
773 Python already implicitly installs signal handlers: see
774 :c:member:`PyConfig.install_signal_handlers`.
775 (Contributed by Victor Stinner in :issue:`41713`.)