blob: 51b3331be89e27ae91487718a0d1e3fca9cbb072 [file] [log] [blame]
Pablo Galindod4fe0982020-05-19 03:33:01 +01001****************************
2 What's New In Python 3.10
3****************************
4
5:Release: |release|
6:Date: |today|
7
8.. Rules for maintenance:
9
10 * Anyone can add text to this document. Do not spend very much time
11 on the wording of your changes, because your text will probably
12 get rewritten to some degree.
13
14 * The maintainer will go through Misc/NEWS periodically and add
15 changes; it's therefore more important to add your changes to
16 Misc/NEWS than to this file.
17
18 * This is not a complete list of every single change; completeness
19 is the purpose of Misc/NEWS. Some changes I consider too small
20 or esoteric to include. If such a change is added to the text,
21 I'll just remove it. (This is another reason you shouldn't spend
22 too much time on writing your addition.)
23
24 * If you want to draw your new text to the attention of the
25 maintainer, add 'XXX' to the beginning of the paragraph or
26 section.
27
28 * It's OK to just add a fragmentary note about a change. For
29 example: "XXX Describe the transmogrify() function added to the
30 socket module." The maintainer will research the change and
31 write the necessary text.
32
33 * You can comment out your additions if you like, but it's not
34 necessary (especially when a final release is some months away).
35
36 * Credit the author of a patch or bugfix. Just the name is
37 sufficient; the e-mail address isn't necessary.
38
39 * It's helpful to add the bug/patch number as a comment:
40
41 XXX Describe the transmogrify() function added to the socket
42 module.
43 (Contributed by P.Y. Developer in :issue:`12345`.)
44
45 This saves the maintainer the effort of going through the Mercurial log
46 when researching a change.
47
48This article explains the new features in Python 3.10, compared to 3.9.
49
Ned Deily29251b72020-05-19 07:39:29 -040050For full details, see the :ref:`changelog <changelog>`.
Pablo Galindod4fe0982020-05-19 03:33:01 +010051
52.. note::
53
54 Prerelease users should be aware that this document is currently in draft
55 form. It will be updated substantially as Python 3.10 moves towards release,
56 so it's worth checking back even after reading earlier versions.
57
58
59Summary -- Release highlights
60=============================
61
62.. This section singles out the most important changes in Python 3.10.
63 Brevity is key.
64
65
66.. PEP-sized items next.
67
68
69
70New Features
71============
72
Batuhan Taskaya044a1042020-10-06 23:03:02 +030073.. _whatsnew310-pep563:
74
Pablo Galindo7c8e0b02021-01-25 23:15:51 +000075Parenthesized context managers
76------------------------------
77
78Using enclosing parentheses for continuation across multiple lines
79in context managers is now supported. This allows formatting a long
80collection of context managers in multiple lines in a similar way
81as it was previously possible with import statements. For instance,
82all these examples are now valid:
83
84.. code-block:: python
85
86 with (CtxManager() as example):
87 ...
88
89 with (
90 CtxManager1(),
91 CtxManager2()
92 ):
93 ...
94
95 with (CtxManager1() as example,
96 CtxManager2()):
97 ...
98
99 with (CtxManager1(),
100 CtxManager2() as example):
101 ...
102
103 with (
104 CtxManager1() as example1,
105 CtxManager2() as example2
106 ):
107 ...
108
109it is also possible to use a trailing comma at the end of the
110enclosed group:
111
112.. code-block:: python
113
114 with (
115 CtxManager1() as example1,
116 CtxManager2() as example2,
117 CtxManager3() as example3,
118 ):
119 ...
120
121This new syntax uses the non LL(1) capacities of the new parser.
122Check :pep:`617` for more details.
123
124(Contributed by Guido van Rossum, Pablo Galindo and Lysandros Nikolaou
125in :issue:`12782` and :issue:`40334`.)
126
127
Ken Jin727a68b2021-03-03 08:52:03 +0800128Better error messages in the parser
129-----------------------------------
Batuhan Taskaya044a1042020-10-06 23:03:02 +0300130
Ken Jin727a68b2021-03-03 08:52:03 +0800131When parsing code that contains unclosed parentheses or brackets the interpreter
132now includes the location of the unclosed bracket of parentheses instead of displaying
133*SyntaxError: unexpected EOF while parsing* or pointing to some incorrect location.
134For instance, consider the following code (notice the unclosed '{'):
Batuhan Taskaya044a1042020-10-06 23:03:02 +0300135
Ken Jin727a68b2021-03-03 08:52:03 +0800136.. code-block:: python
Pablo Galindod4fe0982020-05-19 03:33:01 +0100137
Ken Jin727a68b2021-03-03 08:52:03 +0800138 expected = {9: 1, 18: 2, 19: 2, 27: 3, 28: 3, 29: 3, 36: 4, 37: 4,
139 38: 4, 39: 4, 45: 5, 46: 5, 47: 5, 48: 5, 49: 5, 54: 6,
140 some_other_code = foo()
Dennis Sweeney3ee0e482020-06-12 13:19:25 -0400141
Ken Jin727a68b2021-03-03 08:52:03 +0800142previous versions of the interpreter reported confusing places as the location of
143the syntax error:
Ram Rachum59cf8532020-06-19 23:39:22 +0300144
Ken Jin727a68b2021-03-03 08:52:03 +0800145.. code-block:: text
Mikhail Golubev4f3c2502020-10-08 00:44:31 +0300146
Ken Jin727a68b2021-03-03 08:52:03 +0800147 File "example.py", line 3
148 some_other_code = foo()
149 ^
150 SyntaxError: invalid syntax
Mikhail Golubev4f3c2502020-10-08 00:44:31 +0300151
Ken Jin727a68b2021-03-03 08:52:03 +0800152but in Python3.10 a more informative error is emitted:
Mikhail Golubev4f3c2502020-10-08 00:44:31 +0300153
Ken Jin727a68b2021-03-03 08:52:03 +0800154.. code-block:: text
Mikhail Golubev4f3c2502020-10-08 00:44:31 +0300155
Ken Jin727a68b2021-03-03 08:52:03 +0800156 File "example.py", line 1
157 expected = {9: 1, 18: 2, 19: 2, 27: 3, 28: 3, 29: 3, 36: 4, 37: 4,
158 ^
159 SyntaxError: '{' was never closed
Fidget-Spinner8e1dd552020-10-05 12:40:52 +0800160
161
Ken Jin727a68b2021-03-03 08:52:03 +0800162In a similar way, errors involving unclosed string literals (single and triple
163quoted) now point to the start of the string instead of reporting EOF/EOL.
Fidget-Spinner8e1dd552020-10-05 12:40:52 +0800164
Ken Jin727a68b2021-03-03 08:52:03 +0800165These improvements are inspired by previous work in the PyPy interpreter.
Fidget-Spinner8e1dd552020-10-05 12:40:52 +0800166
Ken Jin727a68b2021-03-03 08:52:03 +0800167(Contributed by Pablo Galindo in :issue:`42864` and Batuhan Taskaya in
168:issue:`40176`.)
Fidget-Spinner8e1dd552020-10-05 12:40:52 +0800169
Mark Shannon6086ae72021-03-16 13:43:58 +0000170PEP 626: Precise line numbers for debugging and other tools
171-----------------------------------------------------------
172
173PEP 626 brings more precise and reliable line numbers for debugging, profiling and coverage tools.
174Tracing events, with the correct line number, are generated for all lines of code executed and only for lines of code that are executed.
175
176The ``f_lineo`` attribute of frame objects will always contain the expected line number.
177
Mark Shannon148bc052021-03-19 17:30:24 +0000178The ``co_lnotab`` attribute of code objects is deprecated and will be removed in 3.12.
179Code that needs to convert from offset to line number should use the new ``co_lines()`` method instead.
Ken Jin11276cd2021-01-02 08:45:50 +0800180
Carol Willing41934b32021-02-28 15:43:17 -0800181PEP 634: Structural Pattern Matching
182------------------------------------
183
184Structural pattern matching has been added in the form of a *match statement*
185and *case statements* of patterns with associated actions. Patterns
186consist of sequences, mappings, primitive data types as well as class instances.
187Pattern matching enables programs to extract information from complex data types,
188branch on the structure of data, and apply specific actions based on different
189forms of data.
190
191Syntax and operations
192~~~~~~~~~~~~~~~~~~~~~
193
194The generic syntax of pattern matching is::
195
196 match subject:
197 case <pattern_1>:
198 <action_1>
199 case <pattern_2>:
200 <action_2>
201 case <pattern_3>:
202 <action_3>
203 case _:
204 <action_wildcard>
205
206A match statement takes an expression and compares its value to successive
207patterns given as one or more case blocks. Specifically, pattern matching
208operates by:
209
210 1. using data with type and shape (the ``subject``)
211 2. evaluating the ``subject`` in the ``match`` statement
212 3. comparing the subject with each pattern in a ``case`` statement
213 from top to bottom until a match is confirmed.
214 4. executing the action associated with the pattern of the confirmed
215 match
216 5. If an exact match is not confirmed, the last case, a wildcard ``_``,
217 if provided, will be used as the matching case. If an exact match is
218 not confirmed and a wildcard case does not exists, the entire match
219 block is a no-op.
220
221Declarative approach
222~~~~~~~~~~~~~~~~~~~~
223
224Readers may be aware of pattern matching through the simple example of matching
225a subject (data object) to a literal (pattern) with the switch statement found
226in C, Java or JavaScript (and many other languages). Often the switch statement
227is used for comparison of an object/expression with case statements containing
228literals.
229
230More powerful examples of pattern matching can be found in languages, such as
231Scala and Elixir. With structural pattern matching, the approach is "declarative" and
232explicitly states the conditions (the patterns) for data to match.
233
234While an "imperative" series of instructions using nested "if" statements
235could be used to accomplish something similar to structural pattern matching,
236it is less clear than the "declarative" approach. Instead the "declarative"
237approach states the conditions to meet for a match and is more readable through
238its explicit patterns. While structural pattern matching can be used in its
239simplest form comparing a variable to a literal in a case statement, its
240true value for Python lies in its handling of the subject's type and shape.
241
242Simple pattern: match to a literal
243~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
244
245Let's look at this example as pattern matching in its simplest form: a value,
246the subject, being matched to several literals, the patterns. In the example
247below, ``status`` is the subject of the match statement. The patterns are
248each of the case statements, where literals represent request status codes.
249The associated action to the case is executed after a match::
250
251 def http_error(status):
252 match status:
253 case 400:
254 return "Bad request"
255 case 404:
256 return "Not found"
257 case 418:
258 return "I'm a teapot"
259 case _:
260 return "Something's wrong with the Internet"
261
262If the above function is passed a ``status`` of 418, "I'm a teapot" is returned.
263If the above function is passed a ``status`` of 500, the case statement with
264``_`` will match as a wildcard, and "Something's wrong with the Internet" is
265returned.
266Note the last block: the variable name, ``_``, acts as a *wildcard* and insures
267the subject will always match. The use of ``_`` is optional.
268
269You can combine several literals in a single pattern using ``|`` ("or")::
270
271 case 401 | 403 | 404:
272 return "Not allowed"
273
274Behavior without the wildcard
275^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
276
277If we modify the above example by removing the last case block, the example
278becomes::
279
280 def http_error(status):
281 match status:
282 case 400:
283 return "Bad request"
284 case 404:
285 return "Not found"
286 case 418:
287 return "I'm a teapot"
288
289Without the use of ``_`` in a case statement, a match may not exist. If no
290match exists, the behavior is a no-op. For example, if ``status`` of 500 is
291passed, a no-op occurs.
292
Julien Palardb04f1cb2021-03-03 11:32:12 +0100293Patterns with a literal and variable
294~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Carol Willing41934b32021-02-28 15:43:17 -0800295
296Patterns can look like unpacking assignments, and a pattern may be used to bind
297variables. In this example, a data point can be unpacked to its x-coordinate
298and y-coordinate::
299
300 # point is an (x, y) tuple
301 match point:
302 case (0, 0):
303 print("Origin")
304 case (0, y):
305 print(f"Y={y}")
306 case (x, 0):
307 print(f"X={x}")
308 case (x, y):
309 print(f"X={x}, Y={y}")
310 case _:
311 raise ValueError("Not a point")
312
313The first pattern has two literals, ``(0, 0)``, and may be thought of as an
314extension of the literal pattern shown above. The next two patterns combine a
315literal and a variable, and the variable *binds* a value from the subject
316(``point``). The fourth pattern captures two values, which makes it
317conceptually similar to the unpacking assignment ``(x, y) = point``.
318
319Patterns and classes
320~~~~~~~~~~~~~~~~~~~~
321
322If you are using classes to structure your data, you can use as a pattern
323the class name followed by an argument list resembling a constructor. This
324pattern has the ability to capture class attributes into variables::
325
326 class Point:
327 x: int
328 y: int
329
330 def location(point):
331 match point:
332 case Point(x=0, y=0):
333 print("Origin is the point's location.")
334 case Point(x=0, y=y):
335 print(f"Y={y} and the point is on the y-axis.")
336 case Point(x=x, y=0):
337 print(f"X={x} and the point is on the x-axis.")
338 case Point():
339 print("The point is located somewhere else on the plane.")
340 case _:
341 print("Not a point")
342
343Patterns with positional parameters
344^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
345
346You can use positional parameters with some builtin classes that provide an
347ordering for their attributes (e.g. dataclasses). You can also define a specific
348position for attributes in patterns by setting the ``__match_args__`` special
349attribute in your classes. If it's set to ("x", "y"), the following patterns
350are all equivalent (and all bind the ``y`` attribute to the ``var`` variable)::
351
352 Point(1, var)
353 Point(1, y=var)
354 Point(x=1, y=var)
355 Point(y=var, x=1)
356
357Nested patterns
358~~~~~~~~~~~~~~~
359
360Patterns can be arbitrarily nested. For example, if our data is a short
361list of points, it could be matched like this::
362
363 match points:
364 case []:
365 print("No points in the list.")
366 case [Point(0, 0)]:
367 print("The origin is the only point in the list.")
368 case [Point(x, y)]:
369 print(f"A single point {x}, {y} is in the list.")
370 case [Point(0, y1), Point(0, y2)]:
371 print(f"Two points on the Y axis at {y1}, {y2} are in the list.")
372 case _:
373 print("Something else is found in the list.")
374
375Complex patterns and the wildcard
376~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
377
378To this point, the examples have used ``_`` alone in the last case statement.
379A wildcard can be used in more complex patterns, such as ``('error', code, _)``.
380For example::
381
382 match test_variable:
383 case ('warning', code, 40):
384 print("A warning has been received.")
385 case ('error', code, _):
386 print(f"An error {code} occured.")
387
388In the above case, ``test_variable`` will match for ('error', code, 100) and
389('error', code, 800).
390
391Guard
392~~~~~
393
394We can add an ``if`` clause to a pattern, known as a "guard". If the
395guard is false, ``match`` goes on to try the next case block. Note
396that value capture happens before the guard is evaluated::
397
398 match point:
399 case Point(x, y) if x == y:
400 print(f"The point is located on the diagonal Y=X at {x}.")
401 case Point(x, y):
402 print(f"Point is not on the diagonal.")
403
404Other Key Features
405~~~~~~~~~~~~~~~~~~
406
407Several other key features:
408
409- Like unpacking assignments, tuple and list patterns have exactly the
410 same meaning and actually match arbitrary sequences. Technically,
411 the subject must be an instance of ``collections.abc.Sequence``.
412 Therefore, an important exception is that patterns don't match iterators.
413 Also, to prevent a common mistake, sequence patterns don't match strings.
414
415- Sequence patterns support wildcards: ``[x, y, *rest]`` and ``(x, y,
416 *rest)`` work similar to wildcards in unpacking assignments. The
417 name after ``*`` may also be ``_``, so ``(x, y, *_)`` matches a sequence
418 of at least two items without binding the remaining items.
419
420- Mapping patterns: ``{"bandwidth": b, "latency": l}`` captures the
421 ``"bandwidth"`` and ``"latency"`` values from a dict. Unlike sequence
422 patterns, extra keys are ignored. A wildcard ``**rest`` is also
423 supported. (But ``**_`` would be redundant, so it not allowed.)
424
425- Subpatterns may be captured using the ``as`` keyword::
426
427 case (Point(x1, y1), Point(x2, y2) as p2): ...
428
429 This binds x1, y1, x2, y2 like you would expect without the ``as`` clause,
430 and p2 to the entire second item of the subject.
431
432- Most literals are compared by equality. However, the singletons ``True``,
433 ``False`` and ``None`` are compared by identity.
434
435- Named constants may be used in patterns. These named constants must be
436 dotted names to prevent the constant from being interpreted as a capture
437 variable::
438
439 from enum import Enum
440 class Color(Enum):
441 RED = 0
442 GREEN = 1
443 BLUE = 2
444
445 match color:
446 case Color.RED:
447 print("I see red!")
448 case Color.GREEN:
449 print("Grass is green")
450 case Color.BLUE:
451 print("I'm feeling the blues :(")
452
453For the full specification see :pep:`634`. Motivation and rationale
454are in :pep:`635`, and a longer tutorial is in :pep:`636`.
455
Pablo Galindo805ede82021-01-21 17:36:35 +0000456
Ken Jin727a68b2021-03-03 08:52:03 +0800457New Features Related to Type Annotations
458========================================
Pablo Galindo805ede82021-01-21 17:36:35 +0000459
Ken Jin727a68b2021-03-03 08:52:03 +0800460This section covers major changes affecting :pep:`484` type annotations and
461the :mod:`typing` module.
Pablo Galindo805ede82021-01-21 17:36:35 +0000462
463
Ken Jin727a68b2021-03-03 08:52:03 +0800464PEP 563: Postponed Evaluation of Annotations Becomes Default
465------------------------------------------------------------
Pablo Galindo805ede82021-01-21 17:36:35 +0000466
Ken Jin727a68b2021-03-03 08:52:03 +0800467In Python 3.7, postponed evaluation of annotations was added,
468to be enabled with a ``from __future__ import annotations``
469directive. In 3.10 this became the default behavior, even
470without that future directive. With this being default, all
471annotations stored in :attr:`__annotations__` will be strings.
472If needed, annotations can be resolved at runtime using
473:func:`typing.get_type_hints`. See :pep:`563` for a full
474description. Also, the :func:`inspect.signature` will try to
475resolve types from now on, and when it fails it will fall back to
476showing the string annotations. (Contributed by Batuhan Taskaya
477in :issue:`38605`.)
Pablo Galindo805ede82021-01-21 17:36:35 +0000478
Ken Jin727a68b2021-03-03 08:52:03 +0800479
480PEP 604: New Type Union Operator
481--------------------------------
482
483A new type union operator was introduced which enables the syntax ``X | Y``.
484This provides a cleaner way of expressing 'either type X or type Y' instead of
485using :data:`typing.Union`, especially in type hints (annotations).
486
487In previous versions of Python, to apply a type hint for functions accepting
488arguments of multiple types, :data:`typing.Union` was used::
489
490 def square(number: Union[int, float]) -> Union[int, float]:
491 return number ** 2
492
493
494Type hints can now be written in a more succinct manner::
495
496 def square(number: int | float) -> int | float:
497 return number ** 2
498
499
500This new syntax is also accepted as the second argument to :func:`isinstance`
501and :func:`issubclass`::
502
503 >>> isinstance(1, int | str)
504 True
505
506See :ref:`types-union` and :pep:`604` for more details.
507
508(Contributed by Maggie Moss and Philippe Prados in :issue:`41428`.)
509
510
511PEP 612: Parameter Specification Variables
512------------------------------------------
513
514Two new options to improve the information provided to static type checkers for
515:pep:`484`\ 's ``Callable`` have been added to the :mod:`typing` module.
516
517The first is the parameter specification variable. They are used to forward the
518parameter types of one callable to another callable -- a pattern commonly
519found in higher order functions and decorators. Examples of usage can be found
520in :class:`typing.ParamSpec`. Previously, there was no easy way to type annotate
521dependency of parameter types in such a precise manner.
522
523The second option is the new ``Concatenate`` operator. It's used in conjunction
524with parameter specification variables to type annotate a higher order callable
525which adds or removes parameters of another callable. Examples of usage can
526be found in :class:`typing.Concatenate`.
527
528See :class:`typing.Callable`, :class:`typing.ParamSpec`,
529:class:`typing.Concatenate` and :pep:`612` for more details.
530
531(Contributed by Ken Jin in :issue:`41559`.)
532
533
534PEP 613: TypeAlias Annotation
535-----------------------------
536
537:pep:`484` introduced the concept of type aliases, only requiring them to be
538top-level unannotated assignments. This simplicity sometimes made it difficult
539for type checkers to distinguish between type aliases and ordinary assignments,
540especially when forward references or invalid types were involved. Compare::
541
542 StrCache = 'Cache[str]' # a type alias
543 LOG_PREFIX = 'LOG[DEBUG]' # a module constant
544
545Now the :mod:`typing` module has a special annotation :data:`TypeAlias` to
546declare type aliases more explicitly::
547
548 StrCache: TypeAlias = 'Cache[str]' # a type alias
549 LOG_PREFIX = 'LOG[DEBUG]' # a module constant
550
551See :pep:`613` for more details.
552
553(Contributed by Mikhail Golubev in :issue:`41923`.)
554
Pablo Galindo805ede82021-01-21 17:36:35 +0000555
Pablo Galindod4fe0982020-05-19 03:33:01 +0100556Other Language Changes
557======================
558
Ken Jin727a68b2021-03-03 08:52:03 +0800559* The :class:`int` type has a new method :meth:`int.bit_count`, returning the
560 number of ones in the binary expansion of a given integer, also known
561 as the population count. (Contributed by Niklas Fiekas in :issue:`29882`.)
562
563* The views returned by :meth:`dict.keys`, :meth:`dict.values` and
564 :meth:`dict.items` now all have a ``mapping`` attribute that gives a
565 :class:`types.MappingProxyType` object wrapping the original
566 dictionary. (Contributed by Dennis Sweeney in :issue:`40890`.)
567
568* :pep:`618`: The :func:`zip` function now has an optional ``strict`` flag, used
569 to require that all the iterables have an equal length.
570
Serhiy Storchaka578c3952020-05-26 18:43:38 +0300571* Builtin and extension functions that take integer arguments no longer accept
572 :class:`~decimal.Decimal`\ s, :class:`~fractions.Fraction`\ s and other
573 objects that can be converted to integers only with a loss (e.g. that have
574 the :meth:`~object.__int__` method but do not have the
575 :meth:`~object.__index__` method).
576 (Contributed by Serhiy Storchaka in :issue:`37999`.)
Ken Jin727a68b2021-03-03 08:52:03 +0800577
Alexcc02b4f2021-02-26 21:58:39 +0200578* If :func:`object.__ipow__` returns :const:`NotImplemented`, the operator will
579 correctly fall back to :func:`object.__pow__` and :func:`object.__rpow__` as expected.
580 (Contributed by Alex Shkop in :issue:`38302`.)
Pablo Galindod4fe0982020-05-19 03:33:01 +0100581
Lysandros Nikolaoua85fefe2020-11-19 01:49:28 +0200582* Assignment expressions can now be used unparenthesized within set literals
583 and set comprehensions, as well as in sequence indexes (but not slices).
584
Victor Stinnera3c3ffa2021-02-18 12:35:37 +0100585* Functions have a new ``__builtins__`` attribute which is used to look for
586 builtin symbols when a function is executed, instead of looking into
Victor Stinner46496f92021-02-20 15:17:18 +0100587 ``__globals__['__builtins__']``. The attribute is initialized from
588 ``__globals__["__builtins__"]`` if it exists, else from the current builtins.
Victor Stinnera3c3ffa2021-02-18 12:35:37 +0100589 (Contributed by Mark Shannon in :issue:`42990`.)
590
Pablo Galindod4fe0982020-05-19 03:33:01 +0100591
592New Modules
593===========
594
595* None yet.
596
597
598Improved Modules
599================
600
Tomáš Hrnčiarfb35fa42021-01-12 01:41:35 +0100601argparse
602--------
603
604Misleading phrase "optional arguments" was replaced with "options" in argparse help. Some tests might require adaptation if they rely on exact output match.
605(Contributed by Raymond Hettinger in :issue:`9694`.)
606
Filipe Laíns4ce6faa2020-08-10 15:48:20 +0100607base64
608------
609
610Add :func:`base64.b32hexencode` and :func:`base64.b32hexdecode` to support the
611Base32 Encoding with Extended Hex Alphabet.
612
Hai Shid332e7b2020-09-29 05:41:11 +0800613codecs
614------
615
616Add a :func:`codecs.unregister` function to unregister a codec search function.
617(Contributed by Hai Shi in :issue:`41842`.)
618
kjd75f6f72020-12-19 01:39:26 +0800619collections.abc
620---------------
621
622The ``__args__`` of the :ref:`parameterized generic <types-genericalias>` for
623:class:`collections.abc.Callable` are now consistent with :data:`typing.Callable`.
624:class:`collections.abc.Callable` generic now flattens type parameters, similar
625to what :data:`typing.Callable` currently does. This means that
626``collections.abc.Callable[[int, str], str]`` will have ``__args__`` of
627``(int, str, str)``; previously this was ``([int, str], str)``. To allow this
628change, :class:`types.GenericAlias` can now be subclassed, and a subclass will
629be returned when subscripting the :class:`collections.abc.Callable` type. Note
630that a :exc:`TypeError` may be raised for invalid forms of parameterizing
631:class:`collections.abc.Callable` which may have passed silently in Python 3.9.
632(Contributed by Ken Jin in :issue:`42195`.)
633
Joongi Kim3eb28462020-11-11 00:19:11 +0900634contextlib
635----------
636
637Add a :func:`contextlib.aclosing` context manager to safely close async generators
638and objects representing asynchronously released resources.
639(Contributed by Joongi Kim and John Belmonte in :issue:`41229`.)
640
Tom Gringauz9c98e8c2020-11-18 00:58:35 +0200641Add asynchronous context manager support to :func:`contextlib.nullcontext`.
642(Contributed by Tom Gringauz in :issue:`41543`.)
643
Hans Petter Janssonda4e09f2020-08-03 22:51:33 -0500644curses
645------
646
647The extended color functions added in ncurses 6.1 will be used transparently
648by :func:`curses.color_content`, :func:`curses.init_color`,
649:func:`curses.init_pair`, and :func:`curses.pair_content`. A new function,
650:func:`curses.has_extended_color_support`, indicates whether extended color
651support is provided by the underlying ncurses library.
652(Contributed by Jeffrey Kintscher and Hans Petter Jansson in :issue:`36982`.)
653
Zackery Spytz14cfa322021-01-14 02:40:09 -0700654The ``BUTTON5_*`` constants are now exposed in the :mod:`curses` module if
655they are provided by the underlying curses library.
656(Contributed by Zackery Spytz in :issue:`39273`.)
657
Steve Dower62949f62021-01-29 21:48:55 +0000658.. _distutils-deprecated:
659
Victor Stinner0e2a0f72021-01-09 00:35:01 +0100660distutils
661---------
662
Steve Dower62949f62021-01-29 21:48:55 +0000663The entire ``distutils`` package is deprecated, to be removed in Python
6643.12. Its functionality for specifying package builds has already been
665completely replaced by third-party packages ``setuptools`` and
666``packaging``, and most other commonly used APIs are available elsewhere
667in the standard library (such as :mod:`platform`, :mod:`shutil`,
668:mod:`subprocess` or :mod:`sysconfig`). There are no plans to migrate
669any other functionality from ``distutils``, and applications that are
670using other functions should plan to make private copies of the code.
671Refer to :pep:`632` for discussion.
672
Victor Stinner0e2a0f72021-01-09 00:35:01 +0100673The ``bdist_wininst`` command deprecated in Python 3.8 has been removed.
ravcio6cd5b012021-01-21 11:23:46 +0100674The ``bdist_wheel`` command is now recommended to distribute binary packages
Victor Stinner0e2a0f72021-01-09 00:35:01 +0100675on Windows.
676(Contributed by Victor Stinner in :issue:`42802`.)
677
Brett Cannon825ac382020-11-06 18:45:56 -0800678doctest
679-------
680
681When a module does not define ``__loader__``, fall back to ``__spec__.loader``.
682(Contributed by Brett Cannon in :issue:`42133`.)
683
Hai Shic5b049b2020-10-14 23:43:31 +0800684encodings
685---------
Pablo Galindob4f90892021-03-10 00:53:57 +0000686
Hai Shic5b049b2020-10-14 23:43:31 +0800687:func:`encodings.normalize_encoding` now ignores non-ASCII characters.
688(Contributed by Hai Shi in :issue:`39337`.)
689
Pablo Galindob4f90892021-03-10 00:53:57 +0000690gc
691--
692
693Added audit hooks for :func:`gc.get_objects`, :func:`gc.get_referrers` and
694:func:`gc.get_referents`. (Contributed by Pablo Galindo in :issue:`43439`.)
695
Serhiy Storchaka8a64cea2020-06-18 22:08:27 +0300696glob
697----
698
699Added the *root_dir* and *dir_fd* parameters in :func:`~glob.glob` and
700:func:`~glob.iglob` which allow to specify the root directory for searching.
701(Contributed by Serhiy Storchaka in :issue:`38144`.)
702
Jason R. Coombs35d50682021-03-14 22:20:49 -0400703importlib.metadata
704------------------
705
706Feature parity with ``importlib_metadata`` 3.7.
707
708:func:`importlib.metadata.entry_points` now provides a nicer experience
709for selecting entry points by group and name through a new
710:class:`importlib.metadata.EntryPoints` class.
711
712Added :func:`importlib.metadata.packages_distributions` for resolving
713top-level Python modules and packages to their
714:class:`importlib.metadata.Distribution`.
715
Brett Cannon825ac382020-11-06 18:45:56 -0800716inspect
717-------
718
719When a module does not define ``__loader__``, fall back to ``__spec__.loader``.
720(Contributed by Brett Cannon in :issue:`42133`.)
721
Batuhan Taskayaeee1c772020-12-24 01:45:13 +0300722Added *globalns* and *localns* parameters in :func:`~inspect.signature` and
723:meth:`inspect.Signature.from_callable` to retrieve the annotations in given
724local and global namespaces.
725(Contributed by Batuhan Taskaya in :issue:`41960`.)
726
Brett Cannon825ac382020-11-06 18:45:56 -0800727linecache
728---------
729
730When a module does not define ``__loader__``, fall back to ``__spec__.loader``.
731(Contributed by Brett Cannon in :issue:`42133`.)
732
pxinwr3405e052020-08-07 13:21:52 +0800733os
734--
735
736Added :func:`os.cpu_count()` support for VxWorks RTOS.
737(Contributed by Peixing Xin in :issue:`41440`.)
738
Christian Heimescd9fed62020-11-13 19:48:52 +0100739Added a new function :func:`os.eventfd` and related helpers to wrap the
740``eventfd2`` syscall on Linux.
741(Contributed by Christian Heimes in :issue:`41001`.)
742
Pablo Galindoa57b3d32020-11-17 00:00:38 +0000743Added :func:`os.splice()` that allows to move data between two file
744descriptors without copying between kernel address space and user
745address space, where one of the file descriptors must refer to a
746pipe. (Contributed by Pablo Galindo in :issue:`41625`.)
747
Dong-hee Naf917c242021-02-04 08:32:55 +0900748Added :data:`~os.O_EVTONLY`, :data:`~os.O_FSYNC`, :data:`~os.O_SYMLINK`
749and :data:`~os.O_NOFOLLOW_ANY` for macOS.
750(Contributed by Dong-hee Na in :issue:`43106`.)
751
Joshua Cannon45205842020-11-20 09:40:39 -0600752pathlib
753-------
754
Yaroslav Pankovych79d2e622020-11-23 22:06:22 +0200755Added slice support to :attr:`PurePath.parents <pathlib.PurePath.parents>`.
Joshua Cannon45205842020-11-20 09:40:39 -0600756(Contributed by Joshua Cannon in :issue:`35498`)
757
Yaroslav Pankovych79d2e622020-11-23 22:06:22 +0200758Added negative indexing support to :attr:`PurePath.parents
759<pathlib.PurePath.parents>`.
760(Contributed by Yaroslav Pankovych in :issue:`21041`)
761
Christian Heimes5c73afc2020-11-30 22:34:45 +0100762platform
763--------
764
765Added :func:`platform.freedesktop_os_release()` to retrieve operation system
766identification from `freedesktop.org os-release
767<https://www.freedesktop.org/software/systemd/man/os-release.html>`_ standard file.
768(Contributed by Christian Heimes in :issue:`28468`)
769
Gregory Schevchenkodaff3902020-07-25 22:58:45 +0300770py_compile
771----------
772
773Added ``--quiet`` option to command-line interface of :mod:`py_compile`.
774(Contributed by Gregory Schevchenko in :issue:`38731`.)
775
Aviral Srivastava000cde52021-02-01 09:38:44 -0800776pyclbr
777------
778
779Added an ``end_lineno`` attribute to the ``Function`` and ``Class``
780objects in the tree returned by :func:`pyclbr.readline` and
781:func:`pyclbr.readline_ex`. It matches the existing (start) ``lineno``.
782(Contributed by Aviral Srivastava in :issue:`38307`.)
783
Zackery Spytzdf592732020-10-29 03:44:35 -0600784shelve
785------
786
787The :mod:`shelve` module now uses :data:`pickle.DEFAULT_PROTOCOL` by default
788instead of :mod:`pickle` protocol ``3`` when creating shelves.
789(Contributed by Zackery Spytz in :issue:`34204`.)
790
Brett Cannon825ac382020-11-06 18:45:56 -0800791site
792----
793
794When a module does not define ``__loader__``, fall back to ``__spec__.loader``.
795(Contributed by Brett Cannon in :issue:`42133`.)
796
Christian Heimes03c8ddd2020-11-20 09:26:07 +0100797socket
798------
799
800The exception :exc:`socket.timeout` is now an alias of :exc:`TimeoutError`.
801(Contributed by Christian Heimes in :issue:`42413`.)
802
Victor Stinnerdd8a93e2020-06-30 00:49:03 +0200803sys
804---
805
806Add :data:`sys.orig_argv` attribute: the list of the original command line
807arguments passed to the Python executable.
808(Contributed by Victor Stinner in :issue:`23427`.)
809
Victor Stinner9852cb32021-01-25 23:12:50 +0100810Add :data:`sys.stdlib_module_names`, containing the list of the standard library
Victor Stinnerdb584bd2021-01-25 13:24:42 +0100811module names.
812(Contributed by Victor Stinner in :issue:`42955`.)
813
Antoine Pitrouba251c22021-03-11 23:35:45 +0100814_thread
815-------
816
817:func:`_thread.interrupt_main` now takes an optional signal number to
818simulate (the default is still :data:`signal.SIGINT`).
819(Contributed by Antoine Pitrou in :issue:`43356`.)
820
Mario Corchero0001a1b2020-11-04 10:27:43 +0100821threading
822---------
823
824Added :func:`threading.gettrace` and :func:`threading.getprofile` to
825retrieve the functions set by :func:`threading.settrace` and
826:func:`threading.setprofile` respectively.
827(Contributed by Mario Corchero in :issue:`42251`.)
828
Mario Corchero750c5ab2020-11-12 18:27:44 +0100829Add :data:`threading.__excepthook__` to allow retrieving the original value
830of :func:`threading.excepthook` in case it is set to a broken or a different
831value.
832(Contributed by Mario Corchero in :issue:`42308`.)
833
Zackery Spytz91e93792020-11-05 15:18:44 -0700834traceback
835---------
836
837The :func:`~traceback.format_exception`,
838:func:`~traceback.format_exception_only`, and
839:func:`~traceback.print_exception` functions can now take an exception object
840as a positional-only argument.
841(Contributed by Zackery Spytz and Matthias Bussonnier in :issue:`26389`.)
842
Bas van Beek0d0e9fe2020-09-22 17:55:34 +0200843types
844-----
845
846Reintroduced the :data:`types.EllipsisType`, :data:`types.NoneType`
847and :data:`types.NotImplementedType` classes, providing a new set
848of types readily interpretable by type checkers.
849(Contributed by Bas van Beek in :issue:`41810`.)
850
kj46873382020-11-19 11:44:24 +0700851typing
852------
853
Ken Jin727a68b2021-03-03 08:52:03 +0800854For major changes, see `New Features Related to Type Annotations`_.
855
kj46873382020-11-19 11:44:24 +0700856The behavior of :class:`typing.Literal` was changed to conform with :pep:`586`
857and to match the behavior of static type checkers specified in the PEP.
858
8591. ``Literal`` now de-duplicates parameters.
8602. Equality comparisons between ``Literal`` objects are now order independent.
8613. ``Literal`` comparisons now respects types. For example,
862 ``Literal[0] == Literal[False]`` previously evaluated to ``True``. It is
863 now ``False``. To support this change, the internally used type cache now
864 supports differentiating types.
8654. ``Literal`` objects will now raise a :exc:`TypeError` exception during
866 equality comparisons if one of their parameters are not :term:`immutable`.
867 Note that declaring ``Literal`` with mutable parameters will not throw
868 an error::
869
870 >>> from typing import Literal
871 >>> Literal[{0}]
872 >>> Literal[{0}] == Literal[{False}]
873 Traceback (most recent call last):
874 File "<stdin>", line 1, in <module>
875 TypeError: unhashable type: 'set'
876
877(Contributed by Yurii Karabas in :issue:`42345`.)
878
Mark Dickinsonc8c70e72020-09-19 21:38:11 +0100879unittest
880--------
881
882Add new method :meth:`~unittest.TestCase.assertNoLogs` to complement the
883existing :meth:`~unittest.TestCase.assertLogs`. (Contributed by Kit Yan Choi
884in :issue:`39385`.)
885
Adam Goldschmidtfcbe0cb2021-02-15 00:41:57 +0200886urllib.parse
887------------
888
889Python versions earlier than Python 3.10 allowed using both ``;`` and ``&`` as
890query parameter separators in :func:`urllib.parse.parse_qs` and
891:func:`urllib.parse.parse_qsl`. Due to security concerns, and to conform with
892newer W3C recommendations, this has been changed to allow only a single
893separator key, with ``&`` as the default. This change also affects
894:func:`cgi.parse` and :func:`cgi.parse_multipart` as they use the affected
895functions internally. For more details, please see their respective
896documentation.
897(Contributed by Adam Goldschmidt, Senthil Kumaran and Ken Jin in :issue:`42967`.)
898
Zackery Spytze28b8c92020-08-09 04:50:53 -0600899xml
900---
901
902Add a :class:`~xml.sax.handler.LexicalHandler` class to the
903:mod:`xml.sax.handler` module.
904(Contributed by Jonathan Gossage and Zackery Spytz in :issue:`35018`.)
905
Brett Cannond2e94bb2020-11-13 15:14:58 -0800906zipimport
907---------
908Add methods related to :pep:`451`: :meth:`~zipimport.zipimporter.find_spec`,
909:meth:`zipimport.zipimporter.create_module`, and
910:meth:`zipimport.zipimporter.exec_module`.
911(Contributed by Brett Cannon in :issue:`42131`.
912
Serhiy Storchaka8a64cea2020-06-18 22:08:27 +0300913
Pablo Galindod4fe0982020-05-19 03:33:01 +0100914Optimizations
915=============
916
Serhiy Storchaka12f43342020-07-20 15:53:55 +0300917* Constructors :func:`str`, :func:`bytes` and :func:`bytearray` are now faster
918 (around 30--40% for small objects).
919 (Contributed by Serhiy Storchaka in :issue:`41334`.)
920
Victor Stinner2c2a4f32020-06-18 01:20:51 +0200921* The :mod:`runpy` module now imports fewer modules.
Victor Stinner4c18fc82020-06-17 23:58:58 +0200922 The ``python3 -m module-name`` command startup time is 1.3x faster in
923 average.
924 (Contributed by Victor Stinner in :issue:`41006`.)
925
Pablo Galindo9e8fe192021-01-03 04:37:46 +0000926* The ``LOAD_ATTR`` instruction now uses new "per opcode cache" mechanism. It
Pablo Galindoa776da92021-01-31 22:55:48 +0000927 is about 36% faster now for regular attributes and 44% faster for slots.
928 (Contributed by Pablo Galindo and Yury Selivanov in :issue:`42093` and Guido
929 van Rossum in :issue:`42927`, based on ideas implemented originally in PyPy
930 and MicroPython.)
Pablo Galindod4fe0982020-05-19 03:33:01 +0100931
Pablo Galindob451b0e2020-10-21 22:46:52 +0100932* When building Python with ``--enable-optimizations`` now
933 ``-fno-semantic-interposition`` is added to both the compile and link line.
934 This speeds builds of the Python interpreter created with ``--enable-shared``
935 with ``gcc`` by up to 30%. See `this article
936 <https://developers.redhat.com/blog/2020/06/25/red-hat-enterprise-linux-8-2-brings-faster-python-3-8-run-speeds/>`_
937 for more details. (Contributed by Victor Stinner and Pablo Galindo in
Brett Cannon2de50972020-12-04 15:39:21 -0800938 :issue:`38980`.)
939
Pablo Galindob451b0e2020-10-21 22:46:52 +0100940
Yurii Karabas73019792020-11-25 12:43:18 +0200941* Function parameters and their annotations are no longer computed at runtime,
942 but rather at compilation time. They are stored as a tuple of strings at the
Pablo Galindo8747c1f2021-03-04 01:29:29 +0000943 bytecode level. It is now around 2 times faster to create a function with
944 parameter annotations. (Contributed by Yurii Karabas and Inada Naoki
945 in :issue:`42202`)
Yurii Karabas73019792020-11-25 12:43:18 +0200946
Dennis Sweeneye8f5ddd2021-02-28 16:32:04 -0500947* Substring search functions such as ``str1 in str2`` and ``str2.find(str1)``
948 now sometimes use Crochemore & Perrin's "Two-Way" string searching
949 algorithm to avoid quadratic behavior on long strings. (Contributed
950 by Dennis Sweeney in :issue:`41972`)
951
Pablo Galindod4fe0982020-05-19 03:33:01 +0100952Deprecated
953==========
954
Brett Cannon04523c52020-10-23 18:10:54 -0700955* Starting in this release, there will be a concerted effort to begin
956 cleaning up old import semantics that were kept for Python 2.7
957 compatibility. Specifically,
958 :meth:`~importlib.abc.PathEntryFinder.find_loader`/:meth:`~importlib.abc.Finder.find_module`
959 (superseded by :meth:`~importlib.abc.Finder.find_spec`),
960 :meth:`~importlib.abc.Loader.load_module`
961 (superseded by :meth:`~importlib.abc.Loader.exec_module`),
962 :meth:`~importlib.abc.Loader.module_repr` (which the import system
963 takes care of for you), the ``__package__`` attribute
964 (superseded by ``__spec__.parent``), the ``__loader__`` attribute
965 (superseded by ``__spec__.loader``), and the ``__cached__`` attribute
966 (superseded by ``__spec__.cached``) will slowly be removed (as well
967 as other classes and methods in :mod:`importlib`).
968 :exc:`ImportWarning` and/or :exc:`DeprecationWarning` will be raised
969 as appropriate to help identify code which needs updating during
970 this transition.
971
Steve Dower62949f62021-01-29 21:48:55 +0000972* The entire ``distutils`` namespace is deprecated, to be removed in
973 Python 3.12. Refer to the :ref:`module changes <distutils-deprecated>`
974 section for more information.
975
Serhiy Storchakaf066bd92021-01-25 23:02:04 +0200976* Non-integer arguments to :func:`random.randrange` are deprecated.
977 The :exc:`ValueError` is deprecated in favor of a :exc:`TypeError`.
978 (Contributed by Serhiy Storchaka and Raymond Hettinger in :issue:`37319`.)
979
Brett Cannon2de50972020-12-04 15:39:21 -0800980* The various ``load_module()`` methods of :mod:`importlib` have been
981 documented as deprecated since Python 3.6, but will now also trigger
982 a :exc:`DeprecationWarning`. Use
983 :meth:`~importlib.abc.Loader.exec_module` instead.
984 (Contributed by Brett Cannon in :issue:`26131`.)
985
986* :meth:`zimport.zipimporter.load_module` has been deprecated in
987 preference for :meth:`~zipimport.zipimporter.exec_module`.
988 (Contributed by Brett Cannon in :issue:`26131`.)
989
990* The use of :meth:`~importlib.abc.Loader.load_module` by the import
991 system now triggers an :exc:`ImportWarning` as
992 :meth:`~importlib.abc.Loader.exec_module` is preferred.
993 (Contributed by Brett Cannon in :issue:`26131`.)
994
Erlend Egeberg Aaslanda1f401a2020-11-17 16:55:12 +0100995* ``sqlite3.OptimizedUnicode`` has been undocumented and obsolete since Python
996 3.3, when it was made an alias to :class:`str`. It is now deprecated,
997 scheduled for removal in Python 3.12.
998 (Contributed by Erlend E. Aasland in :issue:`42264`.)
999
Erlend Egeberg Aaslandddb5e112021-01-06 01:36:04 +01001000* The undocumented built-in function ``sqlite3.enable_shared_cache`` is now
1001 deprecated, scheduled for removal in Python 3.12. Its use is strongly
1002 discouraged by the SQLite3 documentation. See `the SQLite3 docs
Tom Forbes749d40a2021-02-10 17:56:16 +00001003 <https://sqlite.org/c3ref/enable_shared_cache.html>`_ for more details.
Erlend Egeberg Aaslandddb5e112021-01-06 01:36:04 +01001004 If shared cache must be used, open the database in URI mode using the
1005 ``cache=shared`` query parameter.
1006 (Contributed by Erlend E. Aasland in :issue:`24464`.)
1007
Pablo Galindod4fe0982020-05-19 03:33:01 +01001008
Pablo Galindod4fe0982020-05-19 03:33:01 +01001009Removed
1010=======
1011
Serhiy Storchakae2ec0b22020-10-09 14:14:37 +03001012* Removed special methods ``__int__``, ``__float__``, ``__floordiv__``,
1013 ``__mod__``, ``__divmod__``, ``__rfloordiv__``, ``__rmod__`` and
1014 ``__rdivmod__`` of the :class:`complex` class. They always raised
1015 a :exc:`TypeError`.
1016 (Contributed by Serhiy Storchaka in :issue:`41974`.)
1017
Berker Peksagd4d127f2020-07-16 09:38:58 +03001018* The ``ParserBase.error()`` method from the private and undocumented ``_markupbase``
1019 module has been removed. :class:`html.parser.HTMLParser` is the only subclass of
1020 ``ParserBase`` and its ``error()`` implementation has already been removed in
1021 Python 3.5.
1022 (Contributed by Berker Peksag in :issue:`31844`.)
1023
Victor Stinner84f73822020-10-27 04:36:22 +01001024* Removed the ``unicodedata.ucnhash_CAPI`` attribute which was an internal
1025 PyCapsule object. The related private ``_PyUnicode_Name_CAPI`` structure was
1026 moved to the internal C API.
1027 (Contributed by Victor Stinner in :issue:`42157`.)
1028
Lysandros Nikolaouc26d5912020-11-16 20:46:37 +02001029* Removed the ``parser`` module, which was deprecated in 3.9 due to the
1030 switch to the new PEG parser, as well as all the C source and header files
1031 that were only being used by the old parser, including ``node.h``, ``parser.h``,
1032 ``graminit.h`` and ``grammar.h``.
1033
1034* Removed the Public C API functions :c:func:`PyParser_SimpleParseStringFlags`,
1035 :c:func:`PyParser_SimpleParseStringFlagsFilename`,
1036 :c:func:`PyParser_SimpleParseFileFlags` and :c:func:`PyNode_Compile`
1037 that were deprecated in 3.9 due to the switch to the new PEG parser.
1038
Dong-hee Nabe319c02020-11-25 22:17:30 +09001039* Removed the ``formatter`` module, which was deprecated in Python 3.4.
1040 It is somewhat obsolete, little used, and not tested. It was originally
1041 scheduled to be removed in Python 3.6, but such removals were delayed until
1042 after Python 2.7 EOL. Existing users should copy whatever classes they use
1043 into their code.
1044 (Contributed by Dong-hee Na and Terry J. Reedy in :issue:`42299`.)
Pablo Galindod4fe0982020-05-19 03:33:01 +01001045
Hai Shi0f91f582020-12-08 22:42:42 +08001046* Removed the :c:func:`PyModule_GetWarningsModule` function that was useless
1047 now due to the _warnings module was converted to a builtin module in 2.6.
1048 (Contributed by Hai Shi in :issue:`42599`.)
1049
Hugo van Kemenadec47c78b2021-01-13 01:16:37 +02001050* Remove deprecated aliases to :ref:`collections-abstract-base-classes` from
1051 the :mod:`collections` module.
1052 (Contributed by Victor Stinner in :issue:`37324`.)
1053
Ken Jindcea78f2021-01-20 16:16:12 -08001054* The ``loop`` parameter has been removed from most of :mod:`asyncio`\ 's
1055 :doc:`high-level API <../library/asyncio-api-index>` following deprecation
1056 in Python 3.8. The motivation behind this change is multifold:
1057
1058 1. This simplifies the high-level API.
1059 2. The functions in the high-level API have been implicitly getting the
1060 current thread's running event loop since Python 3.7. There isn't a need to
1061 pass the event loop to the API in most normal use cases.
1062 3. Event loop passing is error-prone especially when dealing with loops
1063 running in different threads.
1064
1065 Note that the low-level API will still accept ``loop``.
1066 See `Changes in the Python API`_ for examples of how to replace existing code.
1067
1068 (Contributed by Yurii Karabas, Andrew Svetlov, Yury Selivanov and Kyle Stanley
1069 in :issue:`42392`.)
1070
Hai Shi0f91f582020-12-08 22:42:42 +08001071
Pablo Galindod4fe0982020-05-19 03:33:01 +01001072Porting to Python 3.10
1073======================
1074
1075This section lists previously described changes and other bugfixes
1076that may require changes to your code.
1077
1078
Zackery Spytz91e93792020-11-05 15:18:44 -07001079Changes in the Python API
1080-------------------------
1081
1082* The *etype* parameters of the :func:`~traceback.format_exception`,
1083 :func:`~traceback.format_exception_only`, and
1084 :func:`~traceback.print_exception` functions in the :mod:`traceback` module
1085 have been renamed to *exc*.
1086 (Contributed by Zackery Spytz and Matthias Bussonnier in :issue:`26389`.)
1087
Victor Stinner357704c2020-12-14 23:07:54 +01001088* :mod:`atexit`: At Python exit, if a callback registered with
1089 :func:`atexit.register` fails, its exception is now logged. Previously, only
1090 some exceptions were logged, and the last exception was always silently
1091 ignored.
1092 (Contributed by Victor Stinner in :issue:`42639`.)
1093
kjd75f6f72020-12-19 01:39:26 +08001094* :class:`collections.abc.Callable` generic now flattens type parameters, similar
1095 to what :data:`typing.Callable` currently does. This means that
1096 ``collections.abc.Callable[[int, str], str]`` will have ``__args__`` of
1097 ``(int, str, str)``; previously this was ``([int, str], str)``. Code which
1098 accesses the arguments via :func:`typing.get_args` or ``__args__`` need to account
1099 for this change. Furthermore, :exc:`TypeError` may be raised for invalid forms
1100 of parameterizing :class:`collections.abc.Callable` which may have passed
1101 silently in Python 3.9.
1102 (Contributed by Ken Jin in :issue:`42195`.)
Victor Stinner357704c2020-12-14 23:07:54 +01001103
Erlend Egeberg Aaslandf4936ad2020-12-31 14:16:50 +01001104* :meth:`socket.htons` and :meth:`socket.ntohs` now raise :exc:`OverflowError`
1105 instead of :exc:`DeprecationWarning` if the given parameter will not fit in
1106 a 16-bit unsigned integer.
1107 (Contributed by Erlend E. Aasland in :issue:`42393`.)
1108
Ken Jindcea78f2021-01-20 16:16:12 -08001109* The ``loop`` parameter has been removed from most of :mod:`asyncio`\ 's
1110 :doc:`high-level API <../library/asyncio-api-index>` following deprecation
1111 in Python 3.8.
1112
1113 A coroutine that currently look like this::
1114
1115 async def foo(loop):
1116 await asyncio.sleep(1, loop=loop)
1117
1118 Should be replaced with this::
1119
1120 async def foo():
1121 await asyncio.sleep(1)
1122
1123 If ``foo()`` was specifically designed *not* to run in the current thread's
1124 running event loop (e.g. running in another thread's event loop), consider
1125 using :func:`asyncio.run_coroutine_threadsafe` instead.
1126
1127 (Contributed by Yurii Karabas, Andrew Svetlov, Yury Selivanov and Kyle Stanley
1128 in :issue:`42392`.)
Erlend Egeberg Aaslandf4936ad2020-12-31 14:16:50 +01001129
Victor Stinner46496f92021-02-20 15:17:18 +01001130* The :data:`types.FunctionType` constructor now inherits the current builtins
1131 if the *globals* dictionary has no ``"__builtins__"`` key, rather than using
1132 ``{"None": None}`` as builtins: same behavior as :func:`eval` and
1133 :func:`exec` functions. Defining a function with ``def function(...): ...``
1134 in Python is not affected, globals cannot be overriden with this syntax: it
1135 also inherits the current builtins.
1136 (Contributed by Victor Stinner in :issue:`42990`.)
1137
Yurii Karabas73019792020-11-25 12:43:18 +02001138CPython bytecode changes
1139========================
1140
1141* The ``MAKE_FUNCTION`` instruction accepts tuple of strings as annotations
1142 instead of dictionary.
1143 (Contributed by Yurii Karabas and Inada Naoki in :issue:`42202`)
Dong-hee Naad3252b2020-05-26 01:52:54 +09001144
1145Build Changes
1146=============
1147
Victor Stinner7ab92d52020-06-16 00:54:44 +02001148* The C99 functions :c:func:`snprintf` and :c:func:`vsnprintf` are now required
1149 to build Python.
1150 (Contributed by Victor Stinner in :issue:`36020`.)
1151
Erlend Egeberg Aaslandcf0b2392021-01-06 01:02:43 +01001152* :mod:`sqlite3` requires SQLite 3.7.15 or higher. (Contributed by Sergey Fedoseev
1153 and Erlend E. Aasland :issue:`40744` and :issue:`40810`.)
Erlend Egeberg Aasland207c3212020-09-07 23:26:54 +02001154
Victor Stinner357704c2020-12-14 23:07:54 +01001155* The :mod:`atexit` module must now always be built as a built-in module.
1156 (Contributed by Victor Stinner in :issue:`42639`.)
Erlend Egeberg Aasland207c3212020-09-07 23:26:54 +02001157
pxinwr277ce302020-12-30 20:50:39 +08001158* Added ``--disable-test-modules`` option to the ``configure`` script:
1159 don't build nor install test modules.
1160 (Contributed by Xavier de Gaye, Thomas Petazzoni and Peixing Xin in :issue:`27640`.)
1161
Victor Stinner75e59a92021-01-20 17:07:21 +01001162* Add ``--with-wheel-pkg-dir=PATH`` option to the ``./configure`` script. If
1163 specified, the :mod:`ensurepip` module looks for ``setuptools`` and ``pip``
1164 wheel packages in this directory: if both are present, these wheel packages
1165 are used instead of ensurepip bundled wheel packages.
1166
1167 Some Linux distribution packaging policies recommend against bundling
1168 dependencies. For example, Fedora installs wheel packages in the
1169 ``/usr/share/python-wheels/`` directory and don't install the
1170 ``ensurepip._bundled`` package.
1171
1172 (Contributed by Victor Stinner in :issue:`42856`.)
1173
Victor Stinner801bb0b2021-02-17 11:14:42 +01001174* Add a new configure ``--without-static-libpython`` option to not build the
1175 ``libpythonMAJOR.MINOR.a`` static library and not install the ``python.o``
1176 object file.
1177
1178 (Contributed by Victor Stinner in :issue:`43103`.)
1179
Ned Deilya65b0502021-03-01 00:27:20 -05001180* The ``configure`` script now uses the ``pkg-config`` utility, if available,
1181 to detect the location of Tcl/Tk headers and libraries. As before, those
1182 locations can be explicitly specified with the ``--with-tcltk-includes``
1183 and ``--with-tcltk-libs`` configuration options.
1184 (Contributed by Manolis Stamatogiannakis in :issue:`42603`.)
1185
Christian Heimes32eba612021-03-19 10:29:25 +01001186* Add ``--with-openssl-rpath`` option to ``configure`` script. The option
1187 simplifies building Python with a custom OpenSSL installation, e.g.
1188 ``./configure --with-openssl=/path/to/openssl --with-openssl-rpath=auto``.
1189 (Contributed by Christian Heimes in :issue:`43466`.)
1190
1191
Dong-hee Naad3252b2020-05-26 01:52:54 +09001192
1193C API Changes
1194=============
1195
1196New Features
1197------------
1198
Victor Stinnerdd8a93e2020-06-30 00:49:03 +02001199* The result of :c:func:`PyNumber_Index` now always has exact type :class:`int`.
Serhiy Storchaka5f4b229d2020-05-28 10:33:45 +03001200 Previously, the result could have been an instance of a subclass of ``int``.
1201 (Contributed by Serhiy Storchaka in :issue:`40792`.)
1202
Victor Stinnerdd8a93e2020-06-30 00:49:03 +02001203* Add a new :c:member:`~PyConfig.orig_argv` member to the :c:type:`PyConfig`
1204 structure: the list of the original command line arguments passed to the
1205 Python executable.
1206 (Contributed by Victor Stinner in :issue:`23427`.)
Dong-hee Naad3252b2020-05-26 01:52:54 +09001207
Zackery Spytz2e4dd332020-09-23 12:43:45 -06001208* The :c:func:`PyDateTime_DATE_GET_TZINFO` and
1209 :c:func:`PyDateTime_TIME_GET_TZINFO` macros have been added for accessing
1210 the ``tzinfo`` attributes of :class:`datetime.datetime` and
1211 :class:`datetime.time` objects.
1212 (Contributed by Zackery Spytz in :issue:`30155`.)
1213
Hai Shid332e7b2020-09-29 05:41:11 +08001214* Add a :c:func:`PyCodec_Unregister` function to unregister a codec
1215 search function.
1216 (Contributed by Hai Shi in :issue:`41842`.)
1217
Vladimir Matveev24a54c02020-10-12 12:10:42 -07001218* The :c:func:`PyIter_Send` function was added to allow
Vladimir Matveev037245c2020-10-09 17:15:15 -07001219 sending value into iterator without raising ``StopIteration`` exception.
1220 (Contributed by Vladimir Matveev in :issue:`41756`.)
1221
Alex Gaynor3a8fdb22020-10-19 18:17:50 -04001222* Added :c:func:`PyUnicode_AsUTF8AndSize` to the limited C API.
1223 (Contributed by Alex Gaynor in :issue:`41784`.)
1224
Victor Stinner80218752020-11-04 13:59:15 +01001225* Added :c:func:`PyModule_AddObjectRef` function: similar to
Victor Stinner95ce7cd2020-11-11 01:52:26 +01001226 :c:func:`PyModule_AddObject` but don't steal a reference to the value on
Victor Stinner80218752020-11-04 13:59:15 +01001227 success.
1228 (Contributed by Victor Stinner in :issue:`1635741`.)
1229
Victor Stinner53a03aa2020-11-05 15:02:12 +01001230* Added :c:func:`Py_NewRef` and :c:func:`Py_XNewRef` functions to increment the
1231 reference count of an object and return the object.
1232 (Contributed by Victor Stinner in :issue:`42262`.)
1233
Serhiy Storchaka686c2032020-11-22 13:25:02 +02001234* The :c:func:`PyType_FromSpecWithBases` and :c:func:`PyType_FromModuleAndSpec`
1235 functions now accept a single class as the *bases* argument.
1236 (Contributed by Serhiy Storchaka in :issue:`42423`.)
1237
Hai Shi88c2cfd2020-11-07 00:04:47 +08001238* The :c:func:`PyType_FromModuleAndSpec` function now accepts NULL ``tp_doc``
1239 slot.
1240 (Contributed by Hai Shi in :issue:`41832`.)
1241
Hai Shia13b26c2020-11-11 04:53:46 +08001242* The :c:func:`PyType_GetSlot` function can accept static types.
1243 (Contributed by Hai Shi and Petr Viktorin in :issue:`41073`.)
1244
Pablo Galindod439fb32021-02-20 18:03:08 +00001245* Add a new :c:func:`PySet_CheckExact` function to the C-API to check if an
1246 object is an instance of :class:`set` but not an instance of a subtype.
1247 (Contributed by Pablo Galindo in :issue:`43277`.)
Alex Gaynor3a8fdb22020-10-19 18:17:50 -04001248
Antoine Pitrouba251c22021-03-11 23:35:45 +01001249* Added :c:func:`PyErr_SetInterruptEx` which allows passing a signal number
1250 to simulate.
1251 (Contributed by Antoine Pitrou in :issue:`43356`.)
1252
1253
Dong-hee Naad3252b2020-05-26 01:52:54 +09001254Porting to Python 3.10
1255----------------------
1256
Victor Stinner37bb2892020-06-19 11:45:31 +02001257* The ``PY_SSIZE_T_CLEAN`` macro must now be defined to use
1258 :c:func:`PyArg_ParseTuple` and :c:func:`Py_BuildValue` formats which use
1259 ``#``: ``es#``, ``et#``, ``s#``, ``u#``, ``y#``, ``z#``, ``U#`` and ``Z#``.
1260 See :ref:`Parsing arguments and building values
1261 <arg-parsing>` and the :pep:`353`.
1262 (Contributed by Victor Stinner in :issue:`40943`.)
1263
Victor Stinnerfe2978b2020-05-27 14:55:10 +02001264* Since :c:func:`Py_REFCNT()` is changed to the inline static function,
1265 ``Py_REFCNT(obj) = new_refcnt`` must be replaced with ``Py_SET_REFCNT(obj, new_refcnt)``:
Victor Stinnerdc24b8a2020-06-04 22:10:43 +02001266 see :c:func:`Py_SET_REFCNT()` (available since Python 3.9). For backward
1267 compatibility, this macro can be used::
1268
1269 #if PY_VERSION_HEX < 0x030900A4
1270 # define Py_SET_REFCNT(obj, refcnt) ((Py_REFCNT(obj) = (refcnt)), (void)0)
1271 #endif
1272
Victor Stinnerfe2978b2020-05-27 14:55:10 +02001273 (Contributed by Victor Stinner in :issue:`39573`.)
1274
Victor Stinner59d3dce2020-06-02 14:03:25 +02001275* Calling :c:func:`PyDict_GetItem` without :term:`GIL` held had been allowed
1276 for historical reason. It is no longer allowed.
1277 (Contributed by Victor Stinner in :issue:`40839`.)
1278
Inada Naoki038dd0f2020-06-30 15:26:56 +09001279* ``PyUnicode_FromUnicode(NULL, size)`` and ``PyUnicode_FromStringAndSize(NULL, size)``
1280 raise ``DeprecationWarning`` now. Use :c:func:`PyUnicode_New` to allocate
1281 Unicode object without initial data.
1282 (Contributed by Inada Naoki in :issue:`36346`.)
1283
Victor Stinner47e1afd2020-10-26 16:43:47 +01001284* The private ``_PyUnicode_Name_CAPI`` structure of the PyCapsule API
Victor Stinner84f73822020-10-27 04:36:22 +01001285 ``unicodedata.ucnhash_CAPI`` has been moved to the internal C API.
Victor Stinner920cb642020-10-26 19:19:36 +01001286 (Contributed by Victor Stinner in :issue:`42157`.)
Victor Stinner47e1afd2020-10-26 16:43:47 +01001287
Victor Stinnerace3f9a2020-11-10 21:10:22 +01001288* :c:func:`Py_GetPath`, :c:func:`Py_GetPrefix`, :c:func:`Py_GetExecPrefix`,
1289 :c:func:`Py_GetProgramFullPath`, :c:func:`Py_GetPythonHome` and
1290 :c:func:`Py_GetProgramName` functions now return ``NULL`` if called before
1291 :c:func:`Py_Initialize` (before Python is initialized). Use the new
1292 :ref:`Python Initialization Configuration API <init-config>` to get the
1293 :ref:`Python Path Configuration. <init-path-config>`.
1294 (Contributed by Victor Stinner in :issue:`42260`.)
1295
Victor Stinner0ef96c22020-12-07 11:56:20 +01001296* :c:func:`PyList_SET_ITEM`, :c:func:`PyTuple_SET_ITEM` and
1297 :c:func:`PyCell_SET` macros can no longer be used as l-value or r-value.
1298 For example, ``x = PyList_SET_ITEM(a, b, c)`` and
1299 ``PyList_SET_ITEM(a, b, c) = x`` now fail with a compiler error. It prevents
1300 bugs like ``if (PyList_SET_ITEM (a, b, c) < 0) ...`` test.
1301 (Contributed by Zackery Spytz and Victor Stinner in :issue:`30459`.)
1302
Nicholas Sim4a6bf272021-02-19 22:55:46 +08001303* The non-limited API files ``odictobject.h``, ``parser_interface.h``,
1304 ``picklebufobject.h``, ``pyarena.h``, ``pyctype.h``, ``pydebug.h``,
1305 ``pyfpe.h``, and ``pytime.h`` have been moved to the ``Include/cpython``
1306 directory. These files must not be included directly, as they are already
1307 included in ``Python.h``: :ref:`Include Files <api-includes>`. If they have
1308 been included directly, consider including ``Python.h`` instead.
1309 (Contributed by Nicholas Sim in :issue:`35134`)
1310
Victor Stinner583ee5a2020-10-02 14:49:00 +02001311Deprecated
1312----------
1313
1314* The ``PyUnicode_InternImmortal()`` function is now deprecated
1315 and will be removed in Python 3.12: use :c:func:`PyUnicode_InternInPlace`
1316 instead.
1317 (Contributed by Victor Stinner in :issue:`41692`.)
1318
Dong-hee Naad3252b2020-05-26 01:52:54 +09001319Removed
1320-------
Inada Naoki6f8a6ee2020-06-26 08:07:22 +09001321
1322* ``PyObject_AsCharBuffer()``, ``PyObject_AsReadBuffer()``, ``PyObject_CheckReadBuffer()``,
1323 and ``PyObject_AsWriteBuffer()`` are removed. Please migrate to new buffer protocol;
1324 :c:func:`PyObject_GetBuffer` and :c:func:`PyBuffer_Release`.
Inada Naoki20a79022020-06-27 18:22:09 +09001325 (Contributed by Inada Naoki in :issue:`41103`.)
1326
1327* Removed ``Py_UNICODE_str*`` functions manipulating ``Py_UNICODE*`` strings.
1328 (Contributed by Inada Naoki in :issue:`41123`.)
1329
1330 * ``Py_UNICODE_strlen``: use :c:func:`PyUnicode_GetLength` or
1331 :c:macro:`PyUnicode_GET_LENGTH`
1332 * ``Py_UNICODE_strcat``: use :c:func:`PyUnicode_CopyCharacters` or
1333 :c:func:`PyUnicode_FromFormat`
1334 * ``Py_UNICODE_strcpy``, ``Py_UNICODE_strncpy``: use
1335 :c:func:`PyUnicode_CopyCharacters` or :c:func:`PyUnicode_Substring`
1336 * ``Py_UNICODE_strcmp``: use :c:func:`PyUnicode_Compare`
1337 * ``Py_UNICODE_strncmp``: use :c:func:`PyUnicode_Tailmatch`
1338 * ``Py_UNICODE_strchr``, ``Py_UNICODE_strrchr``: use
1339 :c:func:`PyUnicode_FindChar`
Inada Naokid9f2a132020-06-29 10:46:51 +09001340
1341* Removed ``PyUnicode_GetMax()``. Please migrate to new (:pep:`393`) APIs.
1342 (Contributed by Inada Naoki in :issue:`41103`.)
Inada Naokie4f1fe62020-06-29 13:00:43 +09001343
1344* Removed ``PyLong_FromUnicode()``. Please migrate to :c:func:`PyLong_FromUnicodeObject`.
1345 (Contributed by Inada Naoki in :issue:`41103`.)
Inada Naokib3332662020-06-30 12:23:07 +09001346
1347* Removed ``PyUnicode_AsUnicodeCopy()``. Please use :c:func:`PyUnicode_AsUCS4Copy` or
1348 :c:func:`PyUnicode_AsWideCharString`
1349 (Contributed by Inada Naoki in :issue:`41103`.)
Victor Stinner19c3ac92020-09-23 14:04:57 +02001350
1351* Removed ``_Py_CheckRecursionLimit`` variable: it has been replaced by
1352 ``ceval.recursion_limit`` of the :c:type:`PyInterpreterState` structure.
1353 (Contributed by Victor Stinner in :issue:`41834`.)
Serhiy Storchakadcc54212020-10-05 12:32:00 +03001354
1355* Removed undocumented macros ``Py_ALLOW_RECURSION`` and
1356 ``Py_END_ALLOW_RECURSION`` and the ``recursion_critical`` field of the
1357 :c:type:`PyInterpreterState` structure.
1358 (Contributed by Serhiy Storchaka in :issue:`41936`.)
Victor Stinner296a7962020-11-17 16:22:23 +01001359
1360* Removed the undocumented ``PyOS_InitInterrupts()`` function. Initializing
1361 Python already implicitly installs signal handlers: see
1362 :c:member:`PyConfig.install_signal_handlers`.
1363 (Contributed by Victor Stinner in :issue:`41713`.)
Victor Stinnereec8e612021-03-18 14:57:49 +01001364
1365* Remove the ``PyAST_Validate()`` function. It is no longer possible to build a
1366 AST object (``mod_ty`` type) with the public C API. The function was already
1367 excluded from the limited C API (:pep:`384`).
1368 (Contributed by Victor Stinner in :issue:`43244`.)
Victor Stinner28ad12f2021-03-19 12:41:49 +01001369
1370* Remove the ``symtable.h`` header file and the undocumented functions:
1371
1372 * ``PyST_GetScope()``
1373 * ``PySymtable_Build()``
1374 * ``PySymtable_BuildObject()``
1375 * ``PySymtable_Free()``
1376 * ``Py_SymtableString()``
1377 * ``Py_SymtableStringObject()``
1378
1379 The ``Py_SymtableString()`` function was part the stable ABI by mistake but
1380 it could not be used, because the ``symtable.h`` header file was excluded
1381 from the limited C API.
1382
1383 The Python :mod:`symtable` module remains available and is unchanged.
1384 (Contributed by Victor Stinner in :issue:`43244`.)