blob: c4c282e5a04eae90872ce043dccd9e4bd294ef21 [file] [log] [blame]
Pablo Galindod4fe0982020-05-19 03:33:01 +01001****************************
2 What's New In Python 3.10
3****************************
4
5:Release: |release|
6:Date: |today|
7
8.. Rules for maintenance:
9
10 * Anyone can add text to this document. Do not spend very much time
11 on the wording of your changes, because your text will probably
12 get rewritten to some degree.
13
14 * The maintainer will go through Misc/NEWS periodically and add
15 changes; it's therefore more important to add your changes to
16 Misc/NEWS than to this file.
17
18 * This is not a complete list of every single change; completeness
19 is the purpose of Misc/NEWS. Some changes I consider too small
20 or esoteric to include. If such a change is added to the text,
21 I'll just remove it. (This is another reason you shouldn't spend
22 too much time on writing your addition.)
23
24 * If you want to draw your new text to the attention of the
25 maintainer, add 'XXX' to the beginning of the paragraph or
26 section.
27
28 * It's OK to just add a fragmentary note about a change. For
29 example: "XXX Describe the transmogrify() function added to the
30 socket module." The maintainer will research the change and
31 write the necessary text.
32
33 * You can comment out your additions if you like, but it's not
34 necessary (especially when a final release is some months away).
35
36 * Credit the author of a patch or bugfix. Just the name is
37 sufficient; the e-mail address isn't necessary.
38
39 * It's helpful to add the bug/patch number as a comment:
40
41 XXX Describe the transmogrify() function added to the socket
42 module.
43 (Contributed by P.Y. Developer in :issue:`12345`.)
44
45 This saves the maintainer the effort of going through the Mercurial log
46 when researching a change.
47
48This article explains the new features in Python 3.10, compared to 3.9.
49
Ned Deily29251b72020-05-19 07:39:29 -040050For full details, see the :ref:`changelog <changelog>`.
Pablo Galindod4fe0982020-05-19 03:33:01 +010051
52.. note::
53
54 Prerelease users should be aware that this document is currently in draft
55 form. It will be updated substantially as Python 3.10 moves towards release,
56 so it's worth checking back even after reading earlier versions.
57
58
59Summary -- Release highlights
60=============================
61
62.. This section singles out the most important changes in Python 3.10.
63 Brevity is key.
64
65
66.. PEP-sized items next.
67
68
69
70New Features
71============
72
Batuhan Taskaya044a1042020-10-06 23:03:02 +030073.. _whatsnew310-pep563:
74
Pablo Galindo7c8e0b02021-01-25 23:15:51 +000075Parenthesized context managers
76------------------------------
77
78Using enclosing parentheses for continuation across multiple lines
79in context managers is now supported. This allows formatting a long
80collection of context managers in multiple lines in a similar way
81as it was previously possible with import statements. For instance,
82all these examples are now valid:
83
84.. code-block:: python
85
86 with (CtxManager() as example):
87 ...
88
89 with (
90 CtxManager1(),
91 CtxManager2()
92 ):
93 ...
94
95 with (CtxManager1() as example,
96 CtxManager2()):
97 ...
98
99 with (CtxManager1(),
100 CtxManager2() as example):
101 ...
102
103 with (
104 CtxManager1() as example1,
105 CtxManager2() as example2
106 ):
107 ...
108
109it is also possible to use a trailing comma at the end of the
110enclosed group:
111
112.. code-block:: python
113
114 with (
115 CtxManager1() as example1,
116 CtxManager2() as example2,
117 CtxManager3() as example3,
118 ):
119 ...
120
121This new syntax uses the non LL(1) capacities of the new parser.
122Check :pep:`617` for more details.
123
124(Contributed by Guido van Rossum, Pablo Galindo and Lysandros Nikolaou
125in :issue:`12782` and :issue:`40334`.)
126
127
Ken Jin727a68b2021-03-03 08:52:03 +0800128Better error messages in the parser
129-----------------------------------
Batuhan Taskaya044a1042020-10-06 23:03:02 +0300130
Ken Jin727a68b2021-03-03 08:52:03 +0800131When parsing code that contains unclosed parentheses or brackets the interpreter
132now includes the location of the unclosed bracket of parentheses instead of displaying
133*SyntaxError: unexpected EOF while parsing* or pointing to some incorrect location.
134For instance, consider the following code (notice the unclosed '{'):
Batuhan Taskaya044a1042020-10-06 23:03:02 +0300135
Ken Jin727a68b2021-03-03 08:52:03 +0800136.. code-block:: python
Pablo Galindod4fe0982020-05-19 03:33:01 +0100137
Ken Jin727a68b2021-03-03 08:52:03 +0800138 expected = {9: 1, 18: 2, 19: 2, 27: 3, 28: 3, 29: 3, 36: 4, 37: 4,
139 38: 4, 39: 4, 45: 5, 46: 5, 47: 5, 48: 5, 49: 5, 54: 6,
140 some_other_code = foo()
Dennis Sweeney3ee0e482020-06-12 13:19:25 -0400141
Ken Jin727a68b2021-03-03 08:52:03 +0800142previous versions of the interpreter reported confusing places as the location of
143the syntax error:
Ram Rachum59cf8532020-06-19 23:39:22 +0300144
Ken Jin727a68b2021-03-03 08:52:03 +0800145.. code-block:: text
Mikhail Golubev4f3c2502020-10-08 00:44:31 +0300146
Ken Jin727a68b2021-03-03 08:52:03 +0800147 File "example.py", line 3
148 some_other_code = foo()
149 ^
150 SyntaxError: invalid syntax
Mikhail Golubev4f3c2502020-10-08 00:44:31 +0300151
Ken Jin727a68b2021-03-03 08:52:03 +0800152but in Python3.10 a more informative error is emitted:
Mikhail Golubev4f3c2502020-10-08 00:44:31 +0300153
Ken Jin727a68b2021-03-03 08:52:03 +0800154.. code-block:: text
Mikhail Golubev4f3c2502020-10-08 00:44:31 +0300155
Ken Jin727a68b2021-03-03 08:52:03 +0800156 File "example.py", line 1
157 expected = {9: 1, 18: 2, 19: 2, 27: 3, 28: 3, 29: 3, 36: 4, 37: 4,
158 ^
159 SyntaxError: '{' was never closed
Fidget-Spinner8e1dd552020-10-05 12:40:52 +0800160
161
Ken Jin727a68b2021-03-03 08:52:03 +0800162In a similar way, errors involving unclosed string literals (single and triple
163quoted) now point to the start of the string instead of reporting EOF/EOL.
Fidget-Spinner8e1dd552020-10-05 12:40:52 +0800164
Ken Jin727a68b2021-03-03 08:52:03 +0800165These improvements are inspired by previous work in the PyPy interpreter.
Fidget-Spinner8e1dd552020-10-05 12:40:52 +0800166
Ken Jin727a68b2021-03-03 08:52:03 +0800167(Contributed by Pablo Galindo in :issue:`42864` and Batuhan Taskaya in
168:issue:`40176`.)
Fidget-Spinner8e1dd552020-10-05 12:40:52 +0800169
Ken Jin11276cd2021-01-02 08:45:50 +0800170
Carol Willing41934b32021-02-28 15:43:17 -0800171PEP 634: Structural Pattern Matching
172------------------------------------
173
174Structural pattern matching has been added in the form of a *match statement*
175and *case statements* of patterns with associated actions. Patterns
176consist of sequences, mappings, primitive data types as well as class instances.
177Pattern matching enables programs to extract information from complex data types,
178branch on the structure of data, and apply specific actions based on different
179forms of data.
180
181Syntax and operations
182~~~~~~~~~~~~~~~~~~~~~
183
184The generic syntax of pattern matching is::
185
186 match subject:
187 case <pattern_1>:
188 <action_1>
189 case <pattern_2>:
190 <action_2>
191 case <pattern_3>:
192 <action_3>
193 case _:
194 <action_wildcard>
195
196A match statement takes an expression and compares its value to successive
197patterns given as one or more case blocks. Specifically, pattern matching
198operates by:
199
200 1. using data with type and shape (the ``subject``)
201 2. evaluating the ``subject`` in the ``match`` statement
202 3. comparing the subject with each pattern in a ``case`` statement
203 from top to bottom until a match is confirmed.
204 4. executing the action associated with the pattern of the confirmed
205 match
206 5. If an exact match is not confirmed, the last case, a wildcard ``_``,
207 if provided, will be used as the matching case. If an exact match is
208 not confirmed and a wildcard case does not exists, the entire match
209 block is a no-op.
210
211Declarative approach
212~~~~~~~~~~~~~~~~~~~~
213
214Readers may be aware of pattern matching through the simple example of matching
215a subject (data object) to a literal (pattern) with the switch statement found
216in C, Java or JavaScript (and many other languages). Often the switch statement
217is used for comparison of an object/expression with case statements containing
218literals.
219
220More powerful examples of pattern matching can be found in languages, such as
221Scala and Elixir. With structural pattern matching, the approach is "declarative" and
222explicitly states the conditions (the patterns) for data to match.
223
224While an "imperative" series of instructions using nested "if" statements
225could be used to accomplish something similar to structural pattern matching,
226it is less clear than the "declarative" approach. Instead the "declarative"
227approach states the conditions to meet for a match and is more readable through
228its explicit patterns. While structural pattern matching can be used in its
229simplest form comparing a variable to a literal in a case statement, its
230true value for Python lies in its handling of the subject's type and shape.
231
232Simple pattern: match to a literal
233~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
234
235Let's look at this example as pattern matching in its simplest form: a value,
236the subject, being matched to several literals, the patterns. In the example
237below, ``status`` is the subject of the match statement. The patterns are
238each of the case statements, where literals represent request status codes.
239The associated action to the case is executed after a match::
240
241 def http_error(status):
242 match status:
243 case 400:
244 return "Bad request"
245 case 404:
246 return "Not found"
247 case 418:
248 return "I'm a teapot"
249 case _:
250 return "Something's wrong with the Internet"
251
252If the above function is passed a ``status`` of 418, "I'm a teapot" is returned.
253If the above function is passed a ``status`` of 500, the case statement with
254``_`` will match as a wildcard, and "Something's wrong with the Internet" is
255returned.
256Note the last block: the variable name, ``_``, acts as a *wildcard* and insures
257the subject will always match. The use of ``_`` is optional.
258
259You can combine several literals in a single pattern using ``|`` ("or")::
260
261 case 401 | 403 | 404:
262 return "Not allowed"
263
264Behavior without the wildcard
265^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
266
267If we modify the above example by removing the last case block, the example
268becomes::
269
270 def http_error(status):
271 match status:
272 case 400:
273 return "Bad request"
274 case 404:
275 return "Not found"
276 case 418:
277 return "I'm a teapot"
278
279Without the use of ``_`` in a case statement, a match may not exist. If no
280match exists, the behavior is a no-op. For example, if ``status`` of 500 is
281passed, a no-op occurs.
282
Julien Palardb04f1cb2021-03-03 11:32:12 +0100283Patterns with a literal and variable
284~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Carol Willing41934b32021-02-28 15:43:17 -0800285
286Patterns can look like unpacking assignments, and a pattern may be used to bind
287variables. In this example, a data point can be unpacked to its x-coordinate
288and y-coordinate::
289
290 # point is an (x, y) tuple
291 match point:
292 case (0, 0):
293 print("Origin")
294 case (0, y):
295 print(f"Y={y}")
296 case (x, 0):
297 print(f"X={x}")
298 case (x, y):
299 print(f"X={x}, Y={y}")
300 case _:
301 raise ValueError("Not a point")
302
303The first pattern has two literals, ``(0, 0)``, and may be thought of as an
304extension of the literal pattern shown above. The next two patterns combine a
305literal and a variable, and the variable *binds* a value from the subject
306(``point``). The fourth pattern captures two values, which makes it
307conceptually similar to the unpacking assignment ``(x, y) = point``.
308
309Patterns and classes
310~~~~~~~~~~~~~~~~~~~~
311
312If you are using classes to structure your data, you can use as a pattern
313the class name followed by an argument list resembling a constructor. This
314pattern has the ability to capture class attributes into variables::
315
316 class Point:
317 x: int
318 y: int
319
320 def location(point):
321 match point:
322 case Point(x=0, y=0):
323 print("Origin is the point's location.")
324 case Point(x=0, y=y):
325 print(f"Y={y} and the point is on the y-axis.")
326 case Point(x=x, y=0):
327 print(f"X={x} and the point is on the x-axis.")
328 case Point():
329 print("The point is located somewhere else on the plane.")
330 case _:
331 print("Not a point")
332
333Patterns with positional parameters
334^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
335
336You can use positional parameters with some builtin classes that provide an
337ordering for their attributes (e.g. dataclasses). You can also define a specific
338position for attributes in patterns by setting the ``__match_args__`` special
339attribute in your classes. If it's set to ("x", "y"), the following patterns
340are all equivalent (and all bind the ``y`` attribute to the ``var`` variable)::
341
342 Point(1, var)
343 Point(1, y=var)
344 Point(x=1, y=var)
345 Point(y=var, x=1)
346
347Nested patterns
348~~~~~~~~~~~~~~~
349
350Patterns can be arbitrarily nested. For example, if our data is a short
351list of points, it could be matched like this::
352
353 match points:
354 case []:
355 print("No points in the list.")
356 case [Point(0, 0)]:
357 print("The origin is the only point in the list.")
358 case [Point(x, y)]:
359 print(f"A single point {x}, {y} is in the list.")
360 case [Point(0, y1), Point(0, y2)]:
361 print(f"Two points on the Y axis at {y1}, {y2} are in the list.")
362 case _:
363 print("Something else is found in the list.")
364
365Complex patterns and the wildcard
366~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
367
368To this point, the examples have used ``_`` alone in the last case statement.
369A wildcard can be used in more complex patterns, such as ``('error', code, _)``.
370For example::
371
372 match test_variable:
373 case ('warning', code, 40):
374 print("A warning has been received.")
375 case ('error', code, _):
376 print(f"An error {code} occured.")
377
378In the above case, ``test_variable`` will match for ('error', code, 100) and
379('error', code, 800).
380
381Guard
382~~~~~
383
384We can add an ``if`` clause to a pattern, known as a "guard". If the
385guard is false, ``match`` goes on to try the next case block. Note
386that value capture happens before the guard is evaluated::
387
388 match point:
389 case Point(x, y) if x == y:
390 print(f"The point is located on the diagonal Y=X at {x}.")
391 case Point(x, y):
392 print(f"Point is not on the diagonal.")
393
394Other Key Features
395~~~~~~~~~~~~~~~~~~
396
397Several other key features:
398
399- Like unpacking assignments, tuple and list patterns have exactly the
400 same meaning and actually match arbitrary sequences. Technically,
401 the subject must be an instance of ``collections.abc.Sequence``.
402 Therefore, an important exception is that patterns don't match iterators.
403 Also, to prevent a common mistake, sequence patterns don't match strings.
404
405- Sequence patterns support wildcards: ``[x, y, *rest]`` and ``(x, y,
406 *rest)`` work similar to wildcards in unpacking assignments. The
407 name after ``*`` may also be ``_``, so ``(x, y, *_)`` matches a sequence
408 of at least two items without binding the remaining items.
409
410- Mapping patterns: ``{"bandwidth": b, "latency": l}`` captures the
411 ``"bandwidth"`` and ``"latency"`` values from a dict. Unlike sequence
412 patterns, extra keys are ignored. A wildcard ``**rest`` is also
413 supported. (But ``**_`` would be redundant, so it not allowed.)
414
415- Subpatterns may be captured using the ``as`` keyword::
416
417 case (Point(x1, y1), Point(x2, y2) as p2): ...
418
419 This binds x1, y1, x2, y2 like you would expect without the ``as`` clause,
420 and p2 to the entire second item of the subject.
421
422- Most literals are compared by equality. However, the singletons ``True``,
423 ``False`` and ``None`` are compared by identity.
424
425- Named constants may be used in patterns. These named constants must be
426 dotted names to prevent the constant from being interpreted as a capture
427 variable::
428
429 from enum import Enum
430 class Color(Enum):
431 RED = 0
432 GREEN = 1
433 BLUE = 2
434
435 match color:
436 case Color.RED:
437 print("I see red!")
438 case Color.GREEN:
439 print("Grass is green")
440 case Color.BLUE:
441 print("I'm feeling the blues :(")
442
443For the full specification see :pep:`634`. Motivation and rationale
444are in :pep:`635`, and a longer tutorial is in :pep:`636`.
445
Pablo Galindo805ede82021-01-21 17:36:35 +0000446
Ken Jin727a68b2021-03-03 08:52:03 +0800447New Features Related to Type Annotations
448========================================
Pablo Galindo805ede82021-01-21 17:36:35 +0000449
Ken Jin727a68b2021-03-03 08:52:03 +0800450This section covers major changes affecting :pep:`484` type annotations and
451the :mod:`typing` module.
Pablo Galindo805ede82021-01-21 17:36:35 +0000452
453
Ken Jin727a68b2021-03-03 08:52:03 +0800454PEP 563: Postponed Evaluation of Annotations Becomes Default
455------------------------------------------------------------
Pablo Galindo805ede82021-01-21 17:36:35 +0000456
Ken Jin727a68b2021-03-03 08:52:03 +0800457In Python 3.7, postponed evaluation of annotations was added,
458to be enabled with a ``from __future__ import annotations``
459directive. In 3.10 this became the default behavior, even
460without that future directive. With this being default, all
461annotations stored in :attr:`__annotations__` will be strings.
462If needed, annotations can be resolved at runtime using
463:func:`typing.get_type_hints`. See :pep:`563` for a full
464description. Also, the :func:`inspect.signature` will try to
465resolve types from now on, and when it fails it will fall back to
466showing the string annotations. (Contributed by Batuhan Taskaya
467in :issue:`38605`.)
Pablo Galindo805ede82021-01-21 17:36:35 +0000468
Ken Jin727a68b2021-03-03 08:52:03 +0800469
470PEP 604: New Type Union Operator
471--------------------------------
472
473A new type union operator was introduced which enables the syntax ``X | Y``.
474This provides a cleaner way of expressing 'either type X or type Y' instead of
475using :data:`typing.Union`, especially in type hints (annotations).
476
477In previous versions of Python, to apply a type hint for functions accepting
478arguments of multiple types, :data:`typing.Union` was used::
479
480 def square(number: Union[int, float]) -> Union[int, float]:
481 return number ** 2
482
483
484Type hints can now be written in a more succinct manner::
485
486 def square(number: int | float) -> int | float:
487 return number ** 2
488
489
490This new syntax is also accepted as the second argument to :func:`isinstance`
491and :func:`issubclass`::
492
493 >>> isinstance(1, int | str)
494 True
495
496See :ref:`types-union` and :pep:`604` for more details.
497
498(Contributed by Maggie Moss and Philippe Prados in :issue:`41428`.)
499
500
501PEP 612: Parameter Specification Variables
502------------------------------------------
503
504Two new options to improve the information provided to static type checkers for
505:pep:`484`\ 's ``Callable`` have been added to the :mod:`typing` module.
506
507The first is the parameter specification variable. They are used to forward the
508parameter types of one callable to another callable -- a pattern commonly
509found in higher order functions and decorators. Examples of usage can be found
510in :class:`typing.ParamSpec`. Previously, there was no easy way to type annotate
511dependency of parameter types in such a precise manner.
512
513The second option is the new ``Concatenate`` operator. It's used in conjunction
514with parameter specification variables to type annotate a higher order callable
515which adds or removes parameters of another callable. Examples of usage can
516be found in :class:`typing.Concatenate`.
517
518See :class:`typing.Callable`, :class:`typing.ParamSpec`,
519:class:`typing.Concatenate` and :pep:`612` for more details.
520
521(Contributed by Ken Jin in :issue:`41559`.)
522
523
524PEP 613: TypeAlias Annotation
525-----------------------------
526
527:pep:`484` introduced the concept of type aliases, only requiring them to be
528top-level unannotated assignments. This simplicity sometimes made it difficult
529for type checkers to distinguish between type aliases and ordinary assignments,
530especially when forward references or invalid types were involved. Compare::
531
532 StrCache = 'Cache[str]' # a type alias
533 LOG_PREFIX = 'LOG[DEBUG]' # a module constant
534
535Now the :mod:`typing` module has a special annotation :data:`TypeAlias` to
536declare type aliases more explicitly::
537
538 StrCache: TypeAlias = 'Cache[str]' # a type alias
539 LOG_PREFIX = 'LOG[DEBUG]' # a module constant
540
541See :pep:`613` for more details.
542
543(Contributed by Mikhail Golubev in :issue:`41923`.)
544
Pablo Galindo805ede82021-01-21 17:36:35 +0000545
Pablo Galindod4fe0982020-05-19 03:33:01 +0100546Other Language Changes
547======================
548
Ken Jin727a68b2021-03-03 08:52:03 +0800549* The :class:`int` type has a new method :meth:`int.bit_count`, returning the
550 number of ones in the binary expansion of a given integer, also known
551 as the population count. (Contributed by Niklas Fiekas in :issue:`29882`.)
552
553* The views returned by :meth:`dict.keys`, :meth:`dict.values` and
554 :meth:`dict.items` now all have a ``mapping`` attribute that gives a
555 :class:`types.MappingProxyType` object wrapping the original
556 dictionary. (Contributed by Dennis Sweeney in :issue:`40890`.)
557
558* :pep:`618`: The :func:`zip` function now has an optional ``strict`` flag, used
559 to require that all the iterables have an equal length.
560
Serhiy Storchaka578c3952020-05-26 18:43:38 +0300561* Builtin and extension functions that take integer arguments no longer accept
562 :class:`~decimal.Decimal`\ s, :class:`~fractions.Fraction`\ s and other
563 objects that can be converted to integers only with a loss (e.g. that have
564 the :meth:`~object.__int__` method but do not have the
565 :meth:`~object.__index__` method).
566 (Contributed by Serhiy Storchaka in :issue:`37999`.)
Ken Jin727a68b2021-03-03 08:52:03 +0800567
Alexcc02b4f2021-02-26 21:58:39 +0200568* If :func:`object.__ipow__` returns :const:`NotImplemented`, the operator will
569 correctly fall back to :func:`object.__pow__` and :func:`object.__rpow__` as expected.
570 (Contributed by Alex Shkop in :issue:`38302`.)
Pablo Galindod4fe0982020-05-19 03:33:01 +0100571
Lysandros Nikolaoua85fefe2020-11-19 01:49:28 +0200572* Assignment expressions can now be used unparenthesized within set literals
573 and set comprehensions, as well as in sequence indexes (but not slices).
574
Victor Stinnera3c3ffa2021-02-18 12:35:37 +0100575* Functions have a new ``__builtins__`` attribute which is used to look for
576 builtin symbols when a function is executed, instead of looking into
Victor Stinner46496f92021-02-20 15:17:18 +0100577 ``__globals__['__builtins__']``. The attribute is initialized from
578 ``__globals__["__builtins__"]`` if it exists, else from the current builtins.
Victor Stinnera3c3ffa2021-02-18 12:35:37 +0100579 (Contributed by Mark Shannon in :issue:`42990`.)
580
Pablo Galindod4fe0982020-05-19 03:33:01 +0100581
582New Modules
583===========
584
585* None yet.
586
587
588Improved Modules
589================
590
Tomáš Hrnčiarfb35fa42021-01-12 01:41:35 +0100591argparse
592--------
593
594Misleading phrase "optional arguments" was replaced with "options" in argparse help. Some tests might require adaptation if they rely on exact output match.
595(Contributed by Raymond Hettinger in :issue:`9694`.)
596
Filipe Laíns4ce6faa2020-08-10 15:48:20 +0100597base64
598------
599
600Add :func:`base64.b32hexencode` and :func:`base64.b32hexdecode` to support the
601Base32 Encoding with Extended Hex Alphabet.
602
Hai Shid332e7b2020-09-29 05:41:11 +0800603codecs
604------
605
606Add a :func:`codecs.unregister` function to unregister a codec search function.
607(Contributed by Hai Shi in :issue:`41842`.)
608
kjd75f6f72020-12-19 01:39:26 +0800609collections.abc
610---------------
611
612The ``__args__`` of the :ref:`parameterized generic <types-genericalias>` for
613:class:`collections.abc.Callable` are now consistent with :data:`typing.Callable`.
614:class:`collections.abc.Callable` generic now flattens type parameters, similar
615to what :data:`typing.Callable` currently does. This means that
616``collections.abc.Callable[[int, str], str]`` will have ``__args__`` of
617``(int, str, str)``; previously this was ``([int, str], str)``. To allow this
618change, :class:`types.GenericAlias` can now be subclassed, and a subclass will
619be returned when subscripting the :class:`collections.abc.Callable` type. Note
620that a :exc:`TypeError` may be raised for invalid forms of parameterizing
621:class:`collections.abc.Callable` which may have passed silently in Python 3.9.
622(Contributed by Ken Jin in :issue:`42195`.)
623
Joongi Kim3eb28462020-11-11 00:19:11 +0900624contextlib
625----------
626
627Add a :func:`contextlib.aclosing` context manager to safely close async generators
628and objects representing asynchronously released resources.
629(Contributed by Joongi Kim and John Belmonte in :issue:`41229`.)
630
Tom Gringauz9c98e8c2020-11-18 00:58:35 +0200631Add asynchronous context manager support to :func:`contextlib.nullcontext`.
632(Contributed by Tom Gringauz in :issue:`41543`.)
633
Hans Petter Janssonda4e09f2020-08-03 22:51:33 -0500634curses
635------
636
637The extended color functions added in ncurses 6.1 will be used transparently
638by :func:`curses.color_content`, :func:`curses.init_color`,
639:func:`curses.init_pair`, and :func:`curses.pair_content`. A new function,
640:func:`curses.has_extended_color_support`, indicates whether extended color
641support is provided by the underlying ncurses library.
642(Contributed by Jeffrey Kintscher and Hans Petter Jansson in :issue:`36982`.)
643
Zackery Spytz14cfa322021-01-14 02:40:09 -0700644The ``BUTTON5_*`` constants are now exposed in the :mod:`curses` module if
645they are provided by the underlying curses library.
646(Contributed by Zackery Spytz in :issue:`39273`.)
647
Steve Dower62949f62021-01-29 21:48:55 +0000648.. _distutils-deprecated:
649
Victor Stinner0e2a0f72021-01-09 00:35:01 +0100650distutils
651---------
652
Steve Dower62949f62021-01-29 21:48:55 +0000653The entire ``distutils`` package is deprecated, to be removed in Python
6543.12. Its functionality for specifying package builds has already been
655completely replaced by third-party packages ``setuptools`` and
656``packaging``, and most other commonly used APIs are available elsewhere
657in the standard library (such as :mod:`platform`, :mod:`shutil`,
658:mod:`subprocess` or :mod:`sysconfig`). There are no plans to migrate
659any other functionality from ``distutils``, and applications that are
660using other functions should plan to make private copies of the code.
661Refer to :pep:`632` for discussion.
662
Victor Stinner0e2a0f72021-01-09 00:35:01 +0100663The ``bdist_wininst`` command deprecated in Python 3.8 has been removed.
ravcio6cd5b012021-01-21 11:23:46 +0100664The ``bdist_wheel`` command is now recommended to distribute binary packages
Victor Stinner0e2a0f72021-01-09 00:35:01 +0100665on Windows.
666(Contributed by Victor Stinner in :issue:`42802`.)
667
Brett Cannon825ac382020-11-06 18:45:56 -0800668doctest
669-------
670
671When a module does not define ``__loader__``, fall back to ``__spec__.loader``.
672(Contributed by Brett Cannon in :issue:`42133`.)
673
Hai Shic5b049b2020-10-14 23:43:31 +0800674encodings
675---------
Pablo Galindob4f90892021-03-10 00:53:57 +0000676
Hai Shic5b049b2020-10-14 23:43:31 +0800677:func:`encodings.normalize_encoding` now ignores non-ASCII characters.
678(Contributed by Hai Shi in :issue:`39337`.)
679
Pablo Galindob4f90892021-03-10 00:53:57 +0000680gc
681--
682
683Added audit hooks for :func:`gc.get_objects`, :func:`gc.get_referrers` and
684:func:`gc.get_referents`. (Contributed by Pablo Galindo in :issue:`43439`.)
685
Serhiy Storchaka8a64cea2020-06-18 22:08:27 +0300686glob
687----
688
689Added the *root_dir* and *dir_fd* parameters in :func:`~glob.glob` and
690:func:`~glob.iglob` which allow to specify the root directory for searching.
691(Contributed by Serhiy Storchaka in :issue:`38144`.)
692
Jason R. Coombs35d50682021-03-14 22:20:49 -0400693importlib.metadata
694------------------
695
696Feature parity with ``importlib_metadata`` 3.7.
697
698:func:`importlib.metadata.entry_points` now provides a nicer experience
699for selecting entry points by group and name through a new
700:class:`importlib.metadata.EntryPoints` class.
701
702Added :func:`importlib.metadata.packages_distributions` for resolving
703top-level Python modules and packages to their
704:class:`importlib.metadata.Distribution`.
705
Brett Cannon825ac382020-11-06 18:45:56 -0800706inspect
707-------
708
709When a module does not define ``__loader__``, fall back to ``__spec__.loader``.
710(Contributed by Brett Cannon in :issue:`42133`.)
711
Batuhan Taskayaeee1c772020-12-24 01:45:13 +0300712Added *globalns* and *localns* parameters in :func:`~inspect.signature` and
713:meth:`inspect.Signature.from_callable` to retrieve the annotations in given
714local and global namespaces.
715(Contributed by Batuhan Taskaya in :issue:`41960`.)
716
Brett Cannon825ac382020-11-06 18:45:56 -0800717linecache
718---------
719
720When a module does not define ``__loader__``, fall back to ``__spec__.loader``.
721(Contributed by Brett Cannon in :issue:`42133`.)
722
pxinwr3405e052020-08-07 13:21:52 +0800723os
724--
725
726Added :func:`os.cpu_count()` support for VxWorks RTOS.
727(Contributed by Peixing Xin in :issue:`41440`.)
728
Christian Heimescd9fed62020-11-13 19:48:52 +0100729Added a new function :func:`os.eventfd` and related helpers to wrap the
730``eventfd2`` syscall on Linux.
731(Contributed by Christian Heimes in :issue:`41001`.)
732
Pablo Galindoa57b3d32020-11-17 00:00:38 +0000733Added :func:`os.splice()` that allows to move data between two file
734descriptors without copying between kernel address space and user
735address space, where one of the file descriptors must refer to a
736pipe. (Contributed by Pablo Galindo in :issue:`41625`.)
737
Dong-hee Naf917c242021-02-04 08:32:55 +0900738Added :data:`~os.O_EVTONLY`, :data:`~os.O_FSYNC`, :data:`~os.O_SYMLINK`
739and :data:`~os.O_NOFOLLOW_ANY` for macOS.
740(Contributed by Dong-hee Na in :issue:`43106`.)
741
Joshua Cannon45205842020-11-20 09:40:39 -0600742pathlib
743-------
744
Yaroslav Pankovych79d2e622020-11-23 22:06:22 +0200745Added slice support to :attr:`PurePath.parents <pathlib.PurePath.parents>`.
Joshua Cannon45205842020-11-20 09:40:39 -0600746(Contributed by Joshua Cannon in :issue:`35498`)
747
Yaroslav Pankovych79d2e622020-11-23 22:06:22 +0200748Added negative indexing support to :attr:`PurePath.parents
749<pathlib.PurePath.parents>`.
750(Contributed by Yaroslav Pankovych in :issue:`21041`)
751
Christian Heimes5c73afc2020-11-30 22:34:45 +0100752platform
753--------
754
755Added :func:`platform.freedesktop_os_release()` to retrieve operation system
756identification from `freedesktop.org os-release
757<https://www.freedesktop.org/software/systemd/man/os-release.html>`_ standard file.
758(Contributed by Christian Heimes in :issue:`28468`)
759
Gregory Schevchenkodaff3902020-07-25 22:58:45 +0300760py_compile
761----------
762
763Added ``--quiet`` option to command-line interface of :mod:`py_compile`.
764(Contributed by Gregory Schevchenko in :issue:`38731`.)
765
Aviral Srivastava000cde52021-02-01 09:38:44 -0800766pyclbr
767------
768
769Added an ``end_lineno`` attribute to the ``Function`` and ``Class``
770objects in the tree returned by :func:`pyclbr.readline` and
771:func:`pyclbr.readline_ex`. It matches the existing (start) ``lineno``.
772(Contributed by Aviral Srivastava in :issue:`38307`.)
773
Zackery Spytzdf592732020-10-29 03:44:35 -0600774shelve
775------
776
777The :mod:`shelve` module now uses :data:`pickle.DEFAULT_PROTOCOL` by default
778instead of :mod:`pickle` protocol ``3`` when creating shelves.
779(Contributed by Zackery Spytz in :issue:`34204`.)
780
Brett Cannon825ac382020-11-06 18:45:56 -0800781site
782----
783
784When a module does not define ``__loader__``, fall back to ``__spec__.loader``.
785(Contributed by Brett Cannon in :issue:`42133`.)
786
Christian Heimes03c8ddd2020-11-20 09:26:07 +0100787socket
788------
789
790The exception :exc:`socket.timeout` is now an alias of :exc:`TimeoutError`.
791(Contributed by Christian Heimes in :issue:`42413`.)
792
Victor Stinnerdd8a93e2020-06-30 00:49:03 +0200793sys
794---
795
796Add :data:`sys.orig_argv` attribute: the list of the original command line
797arguments passed to the Python executable.
798(Contributed by Victor Stinner in :issue:`23427`.)
799
Victor Stinner9852cb32021-01-25 23:12:50 +0100800Add :data:`sys.stdlib_module_names`, containing the list of the standard library
Victor Stinnerdb584bd2021-01-25 13:24:42 +0100801module names.
802(Contributed by Victor Stinner in :issue:`42955`.)
803
Antoine Pitrouba251c22021-03-11 23:35:45 +0100804_thread
805-------
806
807:func:`_thread.interrupt_main` now takes an optional signal number to
808simulate (the default is still :data:`signal.SIGINT`).
809(Contributed by Antoine Pitrou in :issue:`43356`.)
810
Mario Corchero0001a1b2020-11-04 10:27:43 +0100811threading
812---------
813
814Added :func:`threading.gettrace` and :func:`threading.getprofile` to
815retrieve the functions set by :func:`threading.settrace` and
816:func:`threading.setprofile` respectively.
817(Contributed by Mario Corchero in :issue:`42251`.)
818
Mario Corchero750c5ab2020-11-12 18:27:44 +0100819Add :data:`threading.__excepthook__` to allow retrieving the original value
820of :func:`threading.excepthook` in case it is set to a broken or a different
821value.
822(Contributed by Mario Corchero in :issue:`42308`.)
823
Zackery Spytz91e93792020-11-05 15:18:44 -0700824traceback
825---------
826
827The :func:`~traceback.format_exception`,
828:func:`~traceback.format_exception_only`, and
829:func:`~traceback.print_exception` functions can now take an exception object
830as a positional-only argument.
831(Contributed by Zackery Spytz and Matthias Bussonnier in :issue:`26389`.)
832
Bas van Beek0d0e9fe2020-09-22 17:55:34 +0200833types
834-----
835
836Reintroduced the :data:`types.EllipsisType`, :data:`types.NoneType`
837and :data:`types.NotImplementedType` classes, providing a new set
838of types readily interpretable by type checkers.
839(Contributed by Bas van Beek in :issue:`41810`.)
840
kj46873382020-11-19 11:44:24 +0700841typing
842------
843
Ken Jin727a68b2021-03-03 08:52:03 +0800844For major changes, see `New Features Related to Type Annotations`_.
845
kj46873382020-11-19 11:44:24 +0700846The behavior of :class:`typing.Literal` was changed to conform with :pep:`586`
847and to match the behavior of static type checkers specified in the PEP.
848
8491. ``Literal`` now de-duplicates parameters.
8502. Equality comparisons between ``Literal`` objects are now order independent.
8513. ``Literal`` comparisons now respects types. For example,
852 ``Literal[0] == Literal[False]`` previously evaluated to ``True``. It is
853 now ``False``. To support this change, the internally used type cache now
854 supports differentiating types.
8554. ``Literal`` objects will now raise a :exc:`TypeError` exception during
856 equality comparisons if one of their parameters are not :term:`immutable`.
857 Note that declaring ``Literal`` with mutable parameters will not throw
858 an error::
859
860 >>> from typing import Literal
861 >>> Literal[{0}]
862 >>> Literal[{0}] == Literal[{False}]
863 Traceback (most recent call last):
864 File "<stdin>", line 1, in <module>
865 TypeError: unhashable type: 'set'
866
867(Contributed by Yurii Karabas in :issue:`42345`.)
868
Mark Dickinsonc8c70e72020-09-19 21:38:11 +0100869unittest
870--------
871
872Add new method :meth:`~unittest.TestCase.assertNoLogs` to complement the
873existing :meth:`~unittest.TestCase.assertLogs`. (Contributed by Kit Yan Choi
874in :issue:`39385`.)
875
Adam Goldschmidtfcbe0cb2021-02-15 00:41:57 +0200876urllib.parse
877------------
878
879Python versions earlier than Python 3.10 allowed using both ``;`` and ``&`` as
880query parameter separators in :func:`urllib.parse.parse_qs` and
881:func:`urllib.parse.parse_qsl`. Due to security concerns, and to conform with
882newer W3C recommendations, this has been changed to allow only a single
883separator key, with ``&`` as the default. This change also affects
884:func:`cgi.parse` and :func:`cgi.parse_multipart` as they use the affected
885functions internally. For more details, please see their respective
886documentation.
887(Contributed by Adam Goldschmidt, Senthil Kumaran and Ken Jin in :issue:`42967`.)
888
Zackery Spytze28b8c92020-08-09 04:50:53 -0600889xml
890---
891
892Add a :class:`~xml.sax.handler.LexicalHandler` class to the
893:mod:`xml.sax.handler` module.
894(Contributed by Jonathan Gossage and Zackery Spytz in :issue:`35018`.)
895
Brett Cannond2e94bb2020-11-13 15:14:58 -0800896zipimport
897---------
898Add methods related to :pep:`451`: :meth:`~zipimport.zipimporter.find_spec`,
899:meth:`zipimport.zipimporter.create_module`, and
900:meth:`zipimport.zipimporter.exec_module`.
901(Contributed by Brett Cannon in :issue:`42131`.
902
Serhiy Storchaka8a64cea2020-06-18 22:08:27 +0300903
Pablo Galindod4fe0982020-05-19 03:33:01 +0100904Optimizations
905=============
906
Serhiy Storchaka12f43342020-07-20 15:53:55 +0300907* Constructors :func:`str`, :func:`bytes` and :func:`bytearray` are now faster
908 (around 30--40% for small objects).
909 (Contributed by Serhiy Storchaka in :issue:`41334`.)
910
Victor Stinner2c2a4f32020-06-18 01:20:51 +0200911* The :mod:`runpy` module now imports fewer modules.
Victor Stinner4c18fc82020-06-17 23:58:58 +0200912 The ``python3 -m module-name`` command startup time is 1.3x faster in
913 average.
914 (Contributed by Victor Stinner in :issue:`41006`.)
915
Pablo Galindo9e8fe192021-01-03 04:37:46 +0000916* The ``LOAD_ATTR`` instruction now uses new "per opcode cache" mechanism. It
Pablo Galindoa776da92021-01-31 22:55:48 +0000917 is about 36% faster now for regular attributes and 44% faster for slots.
918 (Contributed by Pablo Galindo and Yury Selivanov in :issue:`42093` and Guido
919 van Rossum in :issue:`42927`, based on ideas implemented originally in PyPy
920 and MicroPython.)
Pablo Galindod4fe0982020-05-19 03:33:01 +0100921
Pablo Galindob451b0e2020-10-21 22:46:52 +0100922* When building Python with ``--enable-optimizations`` now
923 ``-fno-semantic-interposition`` is added to both the compile and link line.
924 This speeds builds of the Python interpreter created with ``--enable-shared``
925 with ``gcc`` by up to 30%. See `this article
926 <https://developers.redhat.com/blog/2020/06/25/red-hat-enterprise-linux-8-2-brings-faster-python-3-8-run-speeds/>`_
927 for more details. (Contributed by Victor Stinner and Pablo Galindo in
Brett Cannon2de50972020-12-04 15:39:21 -0800928 :issue:`38980`.)
929
Pablo Galindob451b0e2020-10-21 22:46:52 +0100930
Yurii Karabas73019792020-11-25 12:43:18 +0200931* Function parameters and their annotations are no longer computed at runtime,
932 but rather at compilation time. They are stored as a tuple of strings at the
Pablo Galindo8747c1f2021-03-04 01:29:29 +0000933 bytecode level. It is now around 2 times faster to create a function with
934 parameter annotations. (Contributed by Yurii Karabas and Inada Naoki
935 in :issue:`42202`)
Yurii Karabas73019792020-11-25 12:43:18 +0200936
Dennis Sweeneye8f5ddd2021-02-28 16:32:04 -0500937* Substring search functions such as ``str1 in str2`` and ``str2.find(str1)``
938 now sometimes use Crochemore & Perrin's "Two-Way" string searching
939 algorithm to avoid quadratic behavior on long strings. (Contributed
940 by Dennis Sweeney in :issue:`41972`)
941
Pablo Galindod4fe0982020-05-19 03:33:01 +0100942Deprecated
943==========
944
Brett Cannon04523c52020-10-23 18:10:54 -0700945* Starting in this release, there will be a concerted effort to begin
946 cleaning up old import semantics that were kept for Python 2.7
947 compatibility. Specifically,
948 :meth:`~importlib.abc.PathEntryFinder.find_loader`/:meth:`~importlib.abc.Finder.find_module`
949 (superseded by :meth:`~importlib.abc.Finder.find_spec`),
950 :meth:`~importlib.abc.Loader.load_module`
951 (superseded by :meth:`~importlib.abc.Loader.exec_module`),
952 :meth:`~importlib.abc.Loader.module_repr` (which the import system
953 takes care of for you), the ``__package__`` attribute
954 (superseded by ``__spec__.parent``), the ``__loader__`` attribute
955 (superseded by ``__spec__.loader``), and the ``__cached__`` attribute
956 (superseded by ``__spec__.cached``) will slowly be removed (as well
957 as other classes and methods in :mod:`importlib`).
958 :exc:`ImportWarning` and/or :exc:`DeprecationWarning` will be raised
959 as appropriate to help identify code which needs updating during
960 this transition.
961
Steve Dower62949f62021-01-29 21:48:55 +0000962* The entire ``distutils`` namespace is deprecated, to be removed in
963 Python 3.12. Refer to the :ref:`module changes <distutils-deprecated>`
964 section for more information.
965
Serhiy Storchakaf066bd92021-01-25 23:02:04 +0200966* Non-integer arguments to :func:`random.randrange` are deprecated.
967 The :exc:`ValueError` is deprecated in favor of a :exc:`TypeError`.
968 (Contributed by Serhiy Storchaka and Raymond Hettinger in :issue:`37319`.)
969
Brett Cannon2de50972020-12-04 15:39:21 -0800970* The various ``load_module()`` methods of :mod:`importlib` have been
971 documented as deprecated since Python 3.6, but will now also trigger
972 a :exc:`DeprecationWarning`. Use
973 :meth:`~importlib.abc.Loader.exec_module` instead.
974 (Contributed by Brett Cannon in :issue:`26131`.)
975
976* :meth:`zimport.zipimporter.load_module` has been deprecated in
977 preference for :meth:`~zipimport.zipimporter.exec_module`.
978 (Contributed by Brett Cannon in :issue:`26131`.)
979
980* The use of :meth:`~importlib.abc.Loader.load_module` by the import
981 system now triggers an :exc:`ImportWarning` as
982 :meth:`~importlib.abc.Loader.exec_module` is preferred.
983 (Contributed by Brett Cannon in :issue:`26131`.)
984
Erlend Egeberg Aaslanda1f401a2020-11-17 16:55:12 +0100985* ``sqlite3.OptimizedUnicode`` has been undocumented and obsolete since Python
986 3.3, when it was made an alias to :class:`str`. It is now deprecated,
987 scheduled for removal in Python 3.12.
988 (Contributed by Erlend E. Aasland in :issue:`42264`.)
989
Erlend Egeberg Aaslandddb5e112021-01-06 01:36:04 +0100990* The undocumented built-in function ``sqlite3.enable_shared_cache`` is now
991 deprecated, scheduled for removal in Python 3.12. Its use is strongly
992 discouraged by the SQLite3 documentation. See `the SQLite3 docs
Tom Forbes749d40a2021-02-10 17:56:16 +0000993 <https://sqlite.org/c3ref/enable_shared_cache.html>`_ for more details.
Erlend Egeberg Aaslandddb5e112021-01-06 01:36:04 +0100994 If shared cache must be used, open the database in URI mode using the
995 ``cache=shared`` query parameter.
996 (Contributed by Erlend E. Aasland in :issue:`24464`.)
997
Pablo Galindod4fe0982020-05-19 03:33:01 +0100998
Pablo Galindod4fe0982020-05-19 03:33:01 +0100999Removed
1000=======
1001
Serhiy Storchakae2ec0b22020-10-09 14:14:37 +03001002* Removed special methods ``__int__``, ``__float__``, ``__floordiv__``,
1003 ``__mod__``, ``__divmod__``, ``__rfloordiv__``, ``__rmod__`` and
1004 ``__rdivmod__`` of the :class:`complex` class. They always raised
1005 a :exc:`TypeError`.
1006 (Contributed by Serhiy Storchaka in :issue:`41974`.)
1007
Berker Peksagd4d127f2020-07-16 09:38:58 +03001008* The ``ParserBase.error()`` method from the private and undocumented ``_markupbase``
1009 module has been removed. :class:`html.parser.HTMLParser` is the only subclass of
1010 ``ParserBase`` and its ``error()`` implementation has already been removed in
1011 Python 3.5.
1012 (Contributed by Berker Peksag in :issue:`31844`.)
1013
Victor Stinner84f73822020-10-27 04:36:22 +01001014* Removed the ``unicodedata.ucnhash_CAPI`` attribute which was an internal
1015 PyCapsule object. The related private ``_PyUnicode_Name_CAPI`` structure was
1016 moved to the internal C API.
1017 (Contributed by Victor Stinner in :issue:`42157`.)
1018
Lysandros Nikolaouc26d5912020-11-16 20:46:37 +02001019* Removed the ``parser`` module, which was deprecated in 3.9 due to the
1020 switch to the new PEG parser, as well as all the C source and header files
1021 that were only being used by the old parser, including ``node.h``, ``parser.h``,
1022 ``graminit.h`` and ``grammar.h``.
1023
1024* Removed the Public C API functions :c:func:`PyParser_SimpleParseStringFlags`,
1025 :c:func:`PyParser_SimpleParseStringFlagsFilename`,
1026 :c:func:`PyParser_SimpleParseFileFlags` and :c:func:`PyNode_Compile`
1027 that were deprecated in 3.9 due to the switch to the new PEG parser.
1028
Dong-hee Nabe319c02020-11-25 22:17:30 +09001029* Removed the ``formatter`` module, which was deprecated in Python 3.4.
1030 It is somewhat obsolete, little used, and not tested. It was originally
1031 scheduled to be removed in Python 3.6, but such removals were delayed until
1032 after Python 2.7 EOL. Existing users should copy whatever classes they use
1033 into their code.
1034 (Contributed by Dong-hee Na and Terry J. Reedy in :issue:`42299`.)
Pablo Galindod4fe0982020-05-19 03:33:01 +01001035
Hai Shi0f91f582020-12-08 22:42:42 +08001036* Removed the :c:func:`PyModule_GetWarningsModule` function that was useless
1037 now due to the _warnings module was converted to a builtin module in 2.6.
1038 (Contributed by Hai Shi in :issue:`42599`.)
1039
Hugo van Kemenadec47c78b2021-01-13 01:16:37 +02001040* Remove deprecated aliases to :ref:`collections-abstract-base-classes` from
1041 the :mod:`collections` module.
1042 (Contributed by Victor Stinner in :issue:`37324`.)
1043
Ken Jindcea78f2021-01-20 16:16:12 -08001044* The ``loop`` parameter has been removed from most of :mod:`asyncio`\ 's
1045 :doc:`high-level API <../library/asyncio-api-index>` following deprecation
1046 in Python 3.8. The motivation behind this change is multifold:
1047
1048 1. This simplifies the high-level API.
1049 2. The functions in the high-level API have been implicitly getting the
1050 current thread's running event loop since Python 3.7. There isn't a need to
1051 pass the event loop to the API in most normal use cases.
1052 3. Event loop passing is error-prone especially when dealing with loops
1053 running in different threads.
1054
1055 Note that the low-level API will still accept ``loop``.
1056 See `Changes in the Python API`_ for examples of how to replace existing code.
1057
1058 (Contributed by Yurii Karabas, Andrew Svetlov, Yury Selivanov and Kyle Stanley
1059 in :issue:`42392`.)
1060
Hai Shi0f91f582020-12-08 22:42:42 +08001061
Pablo Galindod4fe0982020-05-19 03:33:01 +01001062Porting to Python 3.10
1063======================
1064
1065This section lists previously described changes and other bugfixes
1066that may require changes to your code.
1067
1068
Zackery Spytz91e93792020-11-05 15:18:44 -07001069Changes in the Python API
1070-------------------------
1071
1072* The *etype* parameters of the :func:`~traceback.format_exception`,
1073 :func:`~traceback.format_exception_only`, and
1074 :func:`~traceback.print_exception` functions in the :mod:`traceback` module
1075 have been renamed to *exc*.
1076 (Contributed by Zackery Spytz and Matthias Bussonnier in :issue:`26389`.)
1077
Victor Stinner357704c2020-12-14 23:07:54 +01001078* :mod:`atexit`: At Python exit, if a callback registered with
1079 :func:`atexit.register` fails, its exception is now logged. Previously, only
1080 some exceptions were logged, and the last exception was always silently
1081 ignored.
1082 (Contributed by Victor Stinner in :issue:`42639`.)
1083
kjd75f6f72020-12-19 01:39:26 +08001084* :class:`collections.abc.Callable` generic now flattens type parameters, similar
1085 to what :data:`typing.Callable` currently does. This means that
1086 ``collections.abc.Callable[[int, str], str]`` will have ``__args__`` of
1087 ``(int, str, str)``; previously this was ``([int, str], str)``. Code which
1088 accesses the arguments via :func:`typing.get_args` or ``__args__`` need to account
1089 for this change. Furthermore, :exc:`TypeError` may be raised for invalid forms
1090 of parameterizing :class:`collections.abc.Callable` which may have passed
1091 silently in Python 3.9.
1092 (Contributed by Ken Jin in :issue:`42195`.)
Victor Stinner357704c2020-12-14 23:07:54 +01001093
Erlend Egeberg Aaslandf4936ad2020-12-31 14:16:50 +01001094* :meth:`socket.htons` and :meth:`socket.ntohs` now raise :exc:`OverflowError`
1095 instead of :exc:`DeprecationWarning` if the given parameter will not fit in
1096 a 16-bit unsigned integer.
1097 (Contributed by Erlend E. Aasland in :issue:`42393`.)
1098
Ken Jindcea78f2021-01-20 16:16:12 -08001099* The ``loop`` parameter has been removed from most of :mod:`asyncio`\ 's
1100 :doc:`high-level API <../library/asyncio-api-index>` following deprecation
1101 in Python 3.8.
1102
1103 A coroutine that currently look like this::
1104
1105 async def foo(loop):
1106 await asyncio.sleep(1, loop=loop)
1107
1108 Should be replaced with this::
1109
1110 async def foo():
1111 await asyncio.sleep(1)
1112
1113 If ``foo()`` was specifically designed *not* to run in the current thread's
1114 running event loop (e.g. running in another thread's event loop), consider
1115 using :func:`asyncio.run_coroutine_threadsafe` instead.
1116
1117 (Contributed by Yurii Karabas, Andrew Svetlov, Yury Selivanov and Kyle Stanley
1118 in :issue:`42392`.)
Erlend Egeberg Aaslandf4936ad2020-12-31 14:16:50 +01001119
Victor Stinner46496f92021-02-20 15:17:18 +01001120* The :data:`types.FunctionType` constructor now inherits the current builtins
1121 if the *globals* dictionary has no ``"__builtins__"`` key, rather than using
1122 ``{"None": None}`` as builtins: same behavior as :func:`eval` and
1123 :func:`exec` functions. Defining a function with ``def function(...): ...``
1124 in Python is not affected, globals cannot be overriden with this syntax: it
1125 also inherits the current builtins.
1126 (Contributed by Victor Stinner in :issue:`42990`.)
1127
Yurii Karabas73019792020-11-25 12:43:18 +02001128CPython bytecode changes
1129========================
1130
1131* The ``MAKE_FUNCTION`` instruction accepts tuple of strings as annotations
1132 instead of dictionary.
1133 (Contributed by Yurii Karabas and Inada Naoki in :issue:`42202`)
Dong-hee Naad3252b2020-05-26 01:52:54 +09001134
1135Build Changes
1136=============
1137
Victor Stinner7ab92d52020-06-16 00:54:44 +02001138* The C99 functions :c:func:`snprintf` and :c:func:`vsnprintf` are now required
1139 to build Python.
1140 (Contributed by Victor Stinner in :issue:`36020`.)
1141
Erlend Egeberg Aaslandcf0b2392021-01-06 01:02:43 +01001142* :mod:`sqlite3` requires SQLite 3.7.15 or higher. (Contributed by Sergey Fedoseev
1143 and Erlend E. Aasland :issue:`40744` and :issue:`40810`.)
Erlend Egeberg Aasland207c3212020-09-07 23:26:54 +02001144
Victor Stinner357704c2020-12-14 23:07:54 +01001145* The :mod:`atexit` module must now always be built as a built-in module.
1146 (Contributed by Victor Stinner in :issue:`42639`.)
Erlend Egeberg Aasland207c3212020-09-07 23:26:54 +02001147
pxinwr277ce302020-12-30 20:50:39 +08001148* Added ``--disable-test-modules`` option to the ``configure`` script:
1149 don't build nor install test modules.
1150 (Contributed by Xavier de Gaye, Thomas Petazzoni and Peixing Xin in :issue:`27640`.)
1151
Victor Stinner75e59a92021-01-20 17:07:21 +01001152* Add ``--with-wheel-pkg-dir=PATH`` option to the ``./configure`` script. If
1153 specified, the :mod:`ensurepip` module looks for ``setuptools`` and ``pip``
1154 wheel packages in this directory: if both are present, these wheel packages
1155 are used instead of ensurepip bundled wheel packages.
1156
1157 Some Linux distribution packaging policies recommend against bundling
1158 dependencies. For example, Fedora installs wheel packages in the
1159 ``/usr/share/python-wheels/`` directory and don't install the
1160 ``ensurepip._bundled`` package.
1161
1162 (Contributed by Victor Stinner in :issue:`42856`.)
1163
Victor Stinner801bb0b2021-02-17 11:14:42 +01001164* Add a new configure ``--without-static-libpython`` option to not build the
1165 ``libpythonMAJOR.MINOR.a`` static library and not install the ``python.o``
1166 object file.
1167
1168 (Contributed by Victor Stinner in :issue:`43103`.)
1169
Ned Deilya65b0502021-03-01 00:27:20 -05001170* The ``configure`` script now uses the ``pkg-config`` utility, if available,
1171 to detect the location of Tcl/Tk headers and libraries. As before, those
1172 locations can be explicitly specified with the ``--with-tcltk-includes``
1173 and ``--with-tcltk-libs`` configuration options.
1174 (Contributed by Manolis Stamatogiannakis in :issue:`42603`.)
1175
Dong-hee Naad3252b2020-05-26 01:52:54 +09001176
1177C API Changes
1178=============
1179
1180New Features
1181------------
1182
Victor Stinnerdd8a93e2020-06-30 00:49:03 +02001183* The result of :c:func:`PyNumber_Index` now always has exact type :class:`int`.
Serhiy Storchaka5f4b229d2020-05-28 10:33:45 +03001184 Previously, the result could have been an instance of a subclass of ``int``.
1185 (Contributed by Serhiy Storchaka in :issue:`40792`.)
1186
Victor Stinnerdd8a93e2020-06-30 00:49:03 +02001187* Add a new :c:member:`~PyConfig.orig_argv` member to the :c:type:`PyConfig`
1188 structure: the list of the original command line arguments passed to the
1189 Python executable.
1190 (Contributed by Victor Stinner in :issue:`23427`.)
Dong-hee Naad3252b2020-05-26 01:52:54 +09001191
Zackery Spytz2e4dd332020-09-23 12:43:45 -06001192* The :c:func:`PyDateTime_DATE_GET_TZINFO` and
1193 :c:func:`PyDateTime_TIME_GET_TZINFO` macros have been added for accessing
1194 the ``tzinfo`` attributes of :class:`datetime.datetime` and
1195 :class:`datetime.time` objects.
1196 (Contributed by Zackery Spytz in :issue:`30155`.)
1197
Hai Shid332e7b2020-09-29 05:41:11 +08001198* Add a :c:func:`PyCodec_Unregister` function to unregister a codec
1199 search function.
1200 (Contributed by Hai Shi in :issue:`41842`.)
1201
Vladimir Matveev24a54c02020-10-12 12:10:42 -07001202* The :c:func:`PyIter_Send` function was added to allow
Vladimir Matveev037245c2020-10-09 17:15:15 -07001203 sending value into iterator without raising ``StopIteration`` exception.
1204 (Contributed by Vladimir Matveev in :issue:`41756`.)
1205
Alex Gaynor3a8fdb22020-10-19 18:17:50 -04001206* Added :c:func:`PyUnicode_AsUTF8AndSize` to the limited C API.
1207 (Contributed by Alex Gaynor in :issue:`41784`.)
1208
Victor Stinner80218752020-11-04 13:59:15 +01001209* Added :c:func:`PyModule_AddObjectRef` function: similar to
Victor Stinner95ce7cd2020-11-11 01:52:26 +01001210 :c:func:`PyModule_AddObject` but don't steal a reference to the value on
Victor Stinner80218752020-11-04 13:59:15 +01001211 success.
1212 (Contributed by Victor Stinner in :issue:`1635741`.)
1213
Victor Stinner53a03aa2020-11-05 15:02:12 +01001214* Added :c:func:`Py_NewRef` and :c:func:`Py_XNewRef` functions to increment the
1215 reference count of an object and return the object.
1216 (Contributed by Victor Stinner in :issue:`42262`.)
1217
Serhiy Storchaka686c2032020-11-22 13:25:02 +02001218* The :c:func:`PyType_FromSpecWithBases` and :c:func:`PyType_FromModuleAndSpec`
1219 functions now accept a single class as the *bases* argument.
1220 (Contributed by Serhiy Storchaka in :issue:`42423`.)
1221
Hai Shi88c2cfd2020-11-07 00:04:47 +08001222* The :c:func:`PyType_FromModuleAndSpec` function now accepts NULL ``tp_doc``
1223 slot.
1224 (Contributed by Hai Shi in :issue:`41832`.)
1225
Hai Shia13b26c2020-11-11 04:53:46 +08001226* The :c:func:`PyType_GetSlot` function can accept static types.
1227 (Contributed by Hai Shi and Petr Viktorin in :issue:`41073`.)
1228
Pablo Galindod439fb32021-02-20 18:03:08 +00001229* Add a new :c:func:`PySet_CheckExact` function to the C-API to check if an
1230 object is an instance of :class:`set` but not an instance of a subtype.
1231 (Contributed by Pablo Galindo in :issue:`43277`.)
Alex Gaynor3a8fdb22020-10-19 18:17:50 -04001232
Antoine Pitrouba251c22021-03-11 23:35:45 +01001233* Added :c:func:`PyErr_SetInterruptEx` which allows passing a signal number
1234 to simulate.
1235 (Contributed by Antoine Pitrou in :issue:`43356`.)
1236
1237
Dong-hee Naad3252b2020-05-26 01:52:54 +09001238Porting to Python 3.10
1239----------------------
1240
Victor Stinner37bb2892020-06-19 11:45:31 +02001241* The ``PY_SSIZE_T_CLEAN`` macro must now be defined to use
1242 :c:func:`PyArg_ParseTuple` and :c:func:`Py_BuildValue` formats which use
1243 ``#``: ``es#``, ``et#``, ``s#``, ``u#``, ``y#``, ``z#``, ``U#`` and ``Z#``.
1244 See :ref:`Parsing arguments and building values
1245 <arg-parsing>` and the :pep:`353`.
1246 (Contributed by Victor Stinner in :issue:`40943`.)
1247
Victor Stinnerfe2978b2020-05-27 14:55:10 +02001248* Since :c:func:`Py_REFCNT()` is changed to the inline static function,
1249 ``Py_REFCNT(obj) = new_refcnt`` must be replaced with ``Py_SET_REFCNT(obj, new_refcnt)``:
Victor Stinnerdc24b8a2020-06-04 22:10:43 +02001250 see :c:func:`Py_SET_REFCNT()` (available since Python 3.9). For backward
1251 compatibility, this macro can be used::
1252
1253 #if PY_VERSION_HEX < 0x030900A4
1254 # define Py_SET_REFCNT(obj, refcnt) ((Py_REFCNT(obj) = (refcnt)), (void)0)
1255 #endif
1256
Victor Stinnerfe2978b2020-05-27 14:55:10 +02001257 (Contributed by Victor Stinner in :issue:`39573`.)
1258
Victor Stinner59d3dce2020-06-02 14:03:25 +02001259* Calling :c:func:`PyDict_GetItem` without :term:`GIL` held had been allowed
1260 for historical reason. It is no longer allowed.
1261 (Contributed by Victor Stinner in :issue:`40839`.)
1262
Inada Naoki038dd0f2020-06-30 15:26:56 +09001263* ``PyUnicode_FromUnicode(NULL, size)`` and ``PyUnicode_FromStringAndSize(NULL, size)``
1264 raise ``DeprecationWarning`` now. Use :c:func:`PyUnicode_New` to allocate
1265 Unicode object without initial data.
1266 (Contributed by Inada Naoki in :issue:`36346`.)
1267
Victor Stinner47e1afd2020-10-26 16:43:47 +01001268* The private ``_PyUnicode_Name_CAPI`` structure of the PyCapsule API
Victor Stinner84f73822020-10-27 04:36:22 +01001269 ``unicodedata.ucnhash_CAPI`` has been moved to the internal C API.
Victor Stinner920cb642020-10-26 19:19:36 +01001270 (Contributed by Victor Stinner in :issue:`42157`.)
Victor Stinner47e1afd2020-10-26 16:43:47 +01001271
Victor Stinnerace3f9a2020-11-10 21:10:22 +01001272* :c:func:`Py_GetPath`, :c:func:`Py_GetPrefix`, :c:func:`Py_GetExecPrefix`,
1273 :c:func:`Py_GetProgramFullPath`, :c:func:`Py_GetPythonHome` and
1274 :c:func:`Py_GetProgramName` functions now return ``NULL`` if called before
1275 :c:func:`Py_Initialize` (before Python is initialized). Use the new
1276 :ref:`Python Initialization Configuration API <init-config>` to get the
1277 :ref:`Python Path Configuration. <init-path-config>`.
1278 (Contributed by Victor Stinner in :issue:`42260`.)
1279
Victor Stinner0ef96c22020-12-07 11:56:20 +01001280* :c:func:`PyList_SET_ITEM`, :c:func:`PyTuple_SET_ITEM` and
1281 :c:func:`PyCell_SET` macros can no longer be used as l-value or r-value.
1282 For example, ``x = PyList_SET_ITEM(a, b, c)`` and
1283 ``PyList_SET_ITEM(a, b, c) = x`` now fail with a compiler error. It prevents
1284 bugs like ``if (PyList_SET_ITEM (a, b, c) < 0) ...`` test.
1285 (Contributed by Zackery Spytz and Victor Stinner in :issue:`30459`.)
1286
Nicholas Sim4a6bf272021-02-19 22:55:46 +08001287* The non-limited API files ``odictobject.h``, ``parser_interface.h``,
1288 ``picklebufobject.h``, ``pyarena.h``, ``pyctype.h``, ``pydebug.h``,
1289 ``pyfpe.h``, and ``pytime.h`` have been moved to the ``Include/cpython``
1290 directory. These files must not be included directly, as they are already
1291 included in ``Python.h``: :ref:`Include Files <api-includes>`. If they have
1292 been included directly, consider including ``Python.h`` instead.
1293 (Contributed by Nicholas Sim in :issue:`35134`)
1294
Victor Stinner583ee5a2020-10-02 14:49:00 +02001295Deprecated
1296----------
1297
1298* The ``PyUnicode_InternImmortal()`` function is now deprecated
1299 and will be removed in Python 3.12: use :c:func:`PyUnicode_InternInPlace`
1300 instead.
1301 (Contributed by Victor Stinner in :issue:`41692`.)
1302
Dong-hee Naad3252b2020-05-26 01:52:54 +09001303Removed
1304-------
Inada Naoki6f8a6ee2020-06-26 08:07:22 +09001305
1306* ``PyObject_AsCharBuffer()``, ``PyObject_AsReadBuffer()``, ``PyObject_CheckReadBuffer()``,
1307 and ``PyObject_AsWriteBuffer()`` are removed. Please migrate to new buffer protocol;
1308 :c:func:`PyObject_GetBuffer` and :c:func:`PyBuffer_Release`.
Inada Naoki20a79022020-06-27 18:22:09 +09001309 (Contributed by Inada Naoki in :issue:`41103`.)
1310
1311* Removed ``Py_UNICODE_str*`` functions manipulating ``Py_UNICODE*`` strings.
1312 (Contributed by Inada Naoki in :issue:`41123`.)
1313
1314 * ``Py_UNICODE_strlen``: use :c:func:`PyUnicode_GetLength` or
1315 :c:macro:`PyUnicode_GET_LENGTH`
1316 * ``Py_UNICODE_strcat``: use :c:func:`PyUnicode_CopyCharacters` or
1317 :c:func:`PyUnicode_FromFormat`
1318 * ``Py_UNICODE_strcpy``, ``Py_UNICODE_strncpy``: use
1319 :c:func:`PyUnicode_CopyCharacters` or :c:func:`PyUnicode_Substring`
1320 * ``Py_UNICODE_strcmp``: use :c:func:`PyUnicode_Compare`
1321 * ``Py_UNICODE_strncmp``: use :c:func:`PyUnicode_Tailmatch`
1322 * ``Py_UNICODE_strchr``, ``Py_UNICODE_strrchr``: use
1323 :c:func:`PyUnicode_FindChar`
Inada Naokid9f2a132020-06-29 10:46:51 +09001324
1325* Removed ``PyUnicode_GetMax()``. Please migrate to new (:pep:`393`) APIs.
1326 (Contributed by Inada Naoki in :issue:`41103`.)
Inada Naokie4f1fe62020-06-29 13:00:43 +09001327
1328* Removed ``PyLong_FromUnicode()``. Please migrate to :c:func:`PyLong_FromUnicodeObject`.
1329 (Contributed by Inada Naoki in :issue:`41103`.)
Inada Naokib3332662020-06-30 12:23:07 +09001330
1331* Removed ``PyUnicode_AsUnicodeCopy()``. Please use :c:func:`PyUnicode_AsUCS4Copy` or
1332 :c:func:`PyUnicode_AsWideCharString`
1333 (Contributed by Inada Naoki in :issue:`41103`.)
Victor Stinner19c3ac92020-09-23 14:04:57 +02001334
1335* Removed ``_Py_CheckRecursionLimit`` variable: it has been replaced by
1336 ``ceval.recursion_limit`` of the :c:type:`PyInterpreterState` structure.
1337 (Contributed by Victor Stinner in :issue:`41834`.)
Serhiy Storchakadcc54212020-10-05 12:32:00 +03001338
1339* Removed undocumented macros ``Py_ALLOW_RECURSION`` and
1340 ``Py_END_ALLOW_RECURSION`` and the ``recursion_critical`` field of the
1341 :c:type:`PyInterpreterState` structure.
1342 (Contributed by Serhiy Storchaka in :issue:`41936`.)
Victor Stinner296a7962020-11-17 16:22:23 +01001343
1344* Removed the undocumented ``PyOS_InitInterrupts()`` function. Initializing
1345 Python already implicitly installs signal handlers: see
1346 :c:member:`PyConfig.install_signal_handlers`.
1347 (Contributed by Victor Stinner in :issue:`41713`.)