Ned Deily | 07a1892 | 2018-01-31 18:12:38 -0500 | [diff] [blame] | 1 | **************************** |
| 2 | What's New In Python 3.8 |
| 3 | **************************** |
| 4 | |
Ned Deily | 07a1892 | 2018-01-31 18:12:38 -0500 | [diff] [blame] | 5 | .. Rules for maintenance: |
| 6 | |
| 7 | * Anyone can add text to this document. Do not spend very much time |
| 8 | on the wording of your changes, because your text will probably |
| 9 | get rewritten to some degree. |
| 10 | |
| 11 | * The maintainer will go through Misc/NEWS periodically and add |
| 12 | changes; it's therefore more important to add your changes to |
| 13 | Misc/NEWS than to this file. |
| 14 | |
| 15 | * This is not a complete list of every single change; completeness |
| 16 | is the purpose of Misc/NEWS. Some changes I consider too small |
| 17 | or esoteric to include. If such a change is added to the text, |
| 18 | I'll just remove it. (This is another reason you shouldn't spend |
| 19 | too much time on writing your addition.) |
| 20 | |
| 21 | * If you want to draw your new text to the attention of the |
| 22 | maintainer, add 'XXX' to the beginning of the paragraph or |
| 23 | section. |
| 24 | |
| 25 | * It's OK to just add a fragmentary note about a change. For |
| 26 | example: "XXX Describe the transmogrify() function added to the |
| 27 | socket module." The maintainer will research the change and |
| 28 | write the necessary text. |
| 29 | |
| 30 | * You can comment out your additions if you like, but it's not |
| 31 | necessary (especially when a final release is some months away). |
| 32 | |
| 33 | * Credit the author of a patch or bugfix. Just the name is |
| 34 | sufficient; the e-mail address isn't necessary. |
| 35 | |
| 36 | * It's helpful to add the bug/patch number as a comment: |
| 37 | |
| 38 | XXX Describe the transmogrify() function added to the socket |
| 39 | module. |
| 40 | (Contributed by P.Y. Developer in :issue:`12345`.) |
| 41 | |
| 42 | This saves the maintainer the effort of going through the Mercurial log |
| 43 | when researching a change. |
| 44 | |
| 45 | This article explains the new features in Python 3.8, compared to 3.7. |
| 46 | |
Ned Deily | 45ab51c | 2018-02-28 13:58:38 -0500 | [diff] [blame] | 47 | For full details, see the :ref:`changelog <changelog>`. |
Ned Deily | 07a1892 | 2018-01-31 18:12:38 -0500 | [diff] [blame] | 48 | |
| 49 | .. note:: |
| 50 | |
| 51 | Prerelease users should be aware that this document is currently in draft |
| 52 | form. It will be updated substantially as Python 3.8 moves towards release, |
| 53 | so it's worth checking back even after reading earlier versions. |
| 54 | |
| 55 | |
| 56 | Summary -- Release highlights |
| 57 | ============================= |
| 58 | |
| 59 | .. This section singles out the most important changes in Python 3.8. |
| 60 | Brevity is key. |
| 61 | |
| 62 | |
| 63 | .. PEP-sized items next. |
| 64 | |
| 65 | |
| 66 | |
| 67 | New Features |
| 68 | ============ |
| 69 | |
Guido van Rossum | 09d434c | 2019-04-24 11:30:17 -0700 | [diff] [blame] | 70 | Assignment expressions |
| 71 | ---------------------- |
| 72 | |
| 73 | There is new syntax (the "walrus operator", ``:=``) to assign values |
| 74 | to variables as part of an expression. Example:: |
| 75 | |
| 76 | if (n := len(a)) > 10: |
| 77 | print(f"List is too long ({n} elements, expected <= 10)") |
| 78 | |
| 79 | See :pep:`572` for a full description. |
| 80 | |
| 81 | (Contributed by Emily Morehouse in :issue:`35224`.) |
| 82 | |
| 83 | .. TODO: Emily will sprint on docs at PyCon US 2019. |
| 84 | |
| 85 | |
Guido van Rossum | 843bf42 | 2019-04-29 05:49:30 -0700 | [diff] [blame] | 86 | Positional-only parameters |
| 87 | -------------------------- |
| 88 | |
| 89 | There is new syntax (``/``) to indicate that some function parameters |
| 90 | must be specified positionally (i.e., cannot be used as keyword |
| 91 | arguments). This is the same notation as shown by ``help()`` for |
| 92 | functions implemented in C (produced by Larry Hastings' "Argument |
| 93 | Clinic" tool). Example:: |
| 94 | |
| 95 | def pow(x, y, z=None, /): |
| 96 | r = x**y |
| 97 | if z is not None: |
| 98 | r %= z |
| 99 | return r |
| 100 | |
| 101 | Now ``pow(2, 10)`` and ``pow(2, 10, 17)`` are valid calls, but |
| 102 | ``pow(x=2, y=10)`` and ``pow(2, 10, z=17)`` are invalid. |
| 103 | |
| 104 | See :pep:`570` for a full description. |
| 105 | |
| 106 | (Contributed by Pablo Galindo in :issue:`36540`.) |
| 107 | |
| 108 | .. TODO: Pablo will sprint on docs at PyCon US 2019. |
| 109 | |
| 110 | |
Nick Coghlan | 16eb3bc | 2018-06-20 21:25:01 +1000 | [diff] [blame] | 111 | Parallel filesystem cache for compiled bytecode files |
| 112 | ----------------------------------------------------- |
| 113 | |
| 114 | The new :envvar:`PYTHONPYCACHEPREFIX` setting (also available as |
| 115 | :option:`-X` ``pycache_prefix``) configures the implicit bytecode |
| 116 | cache to use a separate parallel filesystem tree, rather than |
| 117 | the default ``__pycache__`` subdirectories within each source |
| 118 | directory. |
| 119 | |
| 120 | The location of the cache is reported in :data:`sys.pycache_prefix` |
| 121 | (:const:`None` indicates the default location in ``__pycache__`` |
| 122 | subdirectories). |
| 123 | |
| 124 | (Contributed by Carl Meyer in :issue:`33499`.) |
Ned Deily | 07a1892 | 2018-01-31 18:12:38 -0500 | [diff] [blame] | 125 | |
Victor Stinner | 4046069 | 2019-04-26 17:56:44 +0200 | [diff] [blame] | 126 | Debug build uses the same ABI as release build |
| 127 | ----------------------------------------------- |
| 128 | |
Paul Ganssle | 5c403b2 | 2019-04-27 14:14:35 -0400 | [diff] [blame] | 129 | Python now uses the same ABI whether it built in release or debug mode. On |
| 130 | Unix, when Python is built in debug mode, it is now possible to load C |
| 131 | extensions built in release mode and C extensions built using the stable ABI. |
Victor Stinner | 4046069 | 2019-04-26 17:56:44 +0200 | [diff] [blame] | 132 | |
Paul Ganssle | 5c403b2 | 2019-04-27 14:14:35 -0400 | [diff] [blame] | 133 | Release builds and debug builds are now ABI compatible: defining the |
| 134 | ``Py_DEBUG`` macro no longer implies the ``Py_TRACE_REFS`` macro, which |
| 135 | introduces the only ABI incompatibility. The ``Py_TRACE_REFS`` macro, which |
| 136 | adds the :func:`sys.getobjects` function and the :envvar:`PYTHONDUMPREFS` |
| 137 | environment variable, can be set using the new ``./configure --with-trace-refs`` |
| 138 | build option. |
Victor Stinner | 4046069 | 2019-04-26 17:56:44 +0200 | [diff] [blame] | 139 | (Contributed by Victor Stinner in :issue:`36465`.) |
| 140 | |
E. M. Bray | c994c8f | 2019-05-24 17:33:47 +0200 | [diff] [blame] | 141 | On Unix, C extensions are no longer linked to libpython except on Android |
| 142 | and Cygwin. |
Victor Stinner | 4ebcd7e | 2019-05-11 04:10:03 +0200 | [diff] [blame] | 143 | It is now possible |
Paul Ganssle | 5c403b2 | 2019-04-27 14:14:35 -0400 | [diff] [blame] | 144 | for a statically linked Python to load a C extension built using a shared |
| 145 | library Python. |
Victor Stinner | 4046069 | 2019-04-26 17:56:44 +0200 | [diff] [blame] | 146 | (Contributed by Victor Stinner in :issue:`21536`.) |
| 147 | |
| 148 | On Unix, when Python is built in debug mode, import now also looks for C |
| 149 | extensions compiled in release mode and for C extensions compiled with the |
| 150 | stable ABI. |
| 151 | (Contributed by Victor Stinner in :issue:`36722`.) |
| 152 | |
Victor Stinner | 0a8e572 | 2019-05-23 03:30:23 +0200 | [diff] [blame] | 153 | To embed Python into an application, a new ``--embed`` option must be passed to |
| 154 | ``python3-config --libs --embed`` to get ``-lpython3.8`` (link the application |
| 155 | to libpython). To support both 3.8 and older, try ``python3-config --libs |
| 156 | --embed`` first and fallback to ``python3-config --libs`` (without ``--embed``) |
| 157 | if the previous command fails. |
| 158 | |
| 159 | Add a pkg-config ``python-3.8-embed`` module to embed Python into an |
| 160 | application: ``pkg-config python-3.8-embed --libs`` includes ``-lpython3.8``. |
| 161 | To support both 3.8 and older, try ``pkg-config python-X.Y-embed --libs`` first |
| 162 | and fallback to ``pkg-config python-X.Y --libs`` (without ``--embed``) if the |
| 163 | previous command fails (replace ``X.Y`` with the Python version). |
| 164 | |
| 165 | On the other hand, ``pkg-config python3.8 --libs`` no longer contains |
| 166 | ``-lpython3.8``. C extensions must not be linked to libpython (except on |
E. M. Bray | c994c8f | 2019-05-24 17:33:47 +0200 | [diff] [blame] | 167 | Android and Cygwin, whose cases are handled by the script); |
| 168 | this change is backward incompatible on purpose. |
Victor Stinner | 0a8e572 | 2019-05-23 03:30:23 +0200 | [diff] [blame] | 169 | (Contributed by Victor Stinner in :issue:`36721`.) |
| 170 | |
Eric V. Smith | 9a4135e | 2019-05-08 16:28:48 -0400 | [diff] [blame] | 171 | f-strings now support = for quick and easy debugging |
| 172 | ----------------------------------------------------- |
| 173 | |
| 174 | Add ``=`` specifier to f-strings. ``f'{expr=}'`` expands |
| 175 | to the text of the expression, an equal sign, then the repr of the |
| 176 | evaluated expression. So:: |
| 177 | |
| 178 | x = 3 |
| 179 | print(f'{x*9 + 15=}') |
| 180 | |
| 181 | Would print ``x*9 + 15=42``. |
| 182 | |
| 183 | (Contributed by Eric V. Smith and Larry Hastings in :issue:`36817`.) |
| 184 | |
Victor Stinner | 331a6a5 | 2019-05-27 16:39:22 +0200 | [diff] [blame] | 185 | PEP 587: Python Initialization Configuration |
| 186 | -------------------------------------------- |
| 187 | |
| 188 | The :pep:`587` adds a new C API to configure the Python Initialization |
| 189 | providing finer control on the whole configuration and better error reporting. |
| 190 | |
| 191 | New structures: |
| 192 | |
| 193 | * :c:type:`PyConfig` |
| 194 | * :c:type:`PyPreConfig` |
| 195 | * :c:type:`PyStatus` |
| 196 | * :c:type:`PyWideStringList` |
| 197 | |
| 198 | New functions: |
| 199 | |
| 200 | * :c:func:`PyConfig_Clear` |
| 201 | * :c:func:`PyConfig_InitIsolatedConfig` |
| 202 | * :c:func:`PyConfig_InitPythonConfig` |
| 203 | * :c:func:`PyConfig_Read` |
| 204 | * :c:func:`PyConfig_SetArgv` |
| 205 | * :c:func:`PyConfig_SetBytesArgv` |
| 206 | * :c:func:`PyConfig_SetBytesString` |
| 207 | * :c:func:`PyConfig_SetString` |
| 208 | * :c:func:`PyPreConfig_InitIsolatedConfig` |
| 209 | * :c:func:`PyPreConfig_InitPythonConfig` |
| 210 | * :c:func:`PyStatus_Error` |
| 211 | * :c:func:`PyStatus_Exception` |
| 212 | * :c:func:`PyStatus_Exit` |
| 213 | * :c:func:`PyStatus_IsError` |
| 214 | * :c:func:`PyStatus_IsExit` |
| 215 | * :c:func:`PyStatus_NoMemory` |
| 216 | * :c:func:`PyStatus_Ok` |
| 217 | * :c:func:`PyWideStringList_Append` |
| 218 | * :c:func:`PyWideStringList_Insert` |
| 219 | * :c:func:`Py_BytesMain` |
| 220 | * :c:func:`Py_ExitStatusException` |
| 221 | * :c:func:`Py_InitializeFromConfig` |
| 222 | * :c:func:`Py_PreInitialize` |
| 223 | * :c:func:`Py_PreInitializeFromArgs` |
| 224 | * :c:func:`Py_PreInitializeFromBytesArgs` |
| 225 | * :c:func:`Py_RunMain` |
| 226 | |
| 227 | This PEP also adds ``_PyRuntimeState.preconfig`` (:c:type:`PyPreConfig` type) |
| 228 | and ``PyInterpreterState.config`` (:c:type:`PyConfig` type) fields to these |
| 229 | internal structures. ``PyInterpreterState.config`` becomes the new |
| 230 | reference configuration, replacing global configuration variables and |
| 231 | other private variables. |
| 232 | |
| 233 | See :ref:`Python Initialization Configuration <init-config>` for the |
| 234 | documentation. |
| 235 | |
| 236 | See :pep:`587` for a full description. |
| 237 | |
| 238 | (Contributed by Victor Stinner in :issue:`36763`.) |
| 239 | |
Ned Deily | 07a1892 | 2018-01-31 18:12:38 -0500 | [diff] [blame] | 240 | |
| 241 | Other Language Changes |
| 242 | ====================== |
| 243 | |
Serhiy Storchaka | fe2bbb1 | 2018-03-18 09:56:52 +0200 | [diff] [blame] | 244 | * A :keyword:`continue` statement was illegal in the :keyword:`finally` clause |
| 245 | due to a problem with the implementation. In Python 3.8 this restriction |
| 246 | was lifted. |
| 247 | (Contributed by Serhiy Storchaka in :issue:`32489`.) |
| 248 | |
Serhiy Storchaka | b2e2025 | 2018-10-20 00:46:31 +0300 | [diff] [blame] | 249 | * The :class:`int` type now has a new :meth:`~int.as_integer_ratio` method |
| 250 | compatible with the existing :meth:`float.as_integer_ratio` method. |
Lisa Roach | 5ac7043 | 2018-09-13 23:56:23 -0700 | [diff] [blame] | 251 | (Contributed by Lisa Roach in :issue:`33073`.) |
| 252 | |
Serhiy Storchaka | a445feb | 2018-02-10 00:08:17 +0200 | [diff] [blame] | 253 | * Added support of ``\N{name}`` escapes in :mod:`regular expressions <re>`. |
| 254 | (Contributed by Jonathan Eunice and Serhiy Storchaka in :issue:`30688`.) |
Ned Deily | 07a1892 | 2018-01-31 18:12:38 -0500 | [diff] [blame] | 255 | |
Rémi Lapeyre | 6531bf6 | 2018-11-06 01:38:54 +0100 | [diff] [blame] | 256 | * Dict and dictviews are now iterable in reversed insertion order using |
| 257 | :func:`reversed`. (Contributed by Rémi Lapeyre in :issue:`33462`.) |
| 258 | |
Benjamin Peterson | c9a71dd | 2018-09-12 17:14:39 -0700 | [diff] [blame] | 259 | * The syntax allowed for keyword names in function calls was further |
| 260 | restricted. In particular, ``f((keyword)=arg)`` is no longer allowed. It was |
| 261 | never intended to permit more than a bare name on the left-hand side of a |
| 262 | keyword argument assignment term. See :issue:`34641`. |
Ned Deily | 07a1892 | 2018-01-31 18:12:38 -0500 | [diff] [blame] | 263 | |
jChapman | 8fabae3 | 2018-09-22 21:13:10 -0400 | [diff] [blame] | 264 | * Iterable unpacking is now allowed without parentheses in :keyword:`yield` |
| 265 | and :keyword:`return` statements. |
| 266 | (Contributed by David Cuthbert and Jordan Chapman in :issue:`32117`.) |
| 267 | |
Serhiy Storchaka | 6543912 | 2018-10-19 17:42:06 +0300 | [diff] [blame] | 268 | * A backslash-character pair that is not a valid escape sequence generates |
| 269 | a :exc:`DeprecationWarning` since Python 3.6. In Python 3.8 it generates |
| 270 | a :exc:`SyntaxWarning` instead. |
| 271 | (Contributed by Serhiy Storchaka in :issue:`32912`.) |
| 272 | |
Serhiy Storchaka | 62e4481 | 2019-02-16 08:12:19 +0200 | [diff] [blame] | 273 | * The compiler now produces a :exc:`SyntaxWarning` in some cases when a comma |
| 274 | is missed before tuple or list. For example:: |
| 275 | |
| 276 | data = [ |
| 277 | (1, 2, 3) # oops, missing comma! |
| 278 | (4, 5, 6) |
| 279 | ] |
| 280 | |
| 281 | (Contributed by Serhiy Storchaka in :issue:`15248`.) |
| 282 | |
Paul Ganssle | d9503c3 | 2019-02-08 11:02:00 -0500 | [diff] [blame] | 283 | * Arithmetic operations between subclasses of :class:`datetime.date` or |
| 284 | :class:`datetime.datetime` and :class:`datetime.timedelta` objects now return |
| 285 | an instance of the subclass, rather than the base class. This also affects |
| 286 | the return type of operations whose implementation (directly or indirectly) |
| 287 | uses :class:`datetime.timedelta` arithmetic, such as |
| 288 | :meth:`datetime.datetime.astimezone`. |
| 289 | (Contributed by Paul Ganssle in :issue:`32417`.) |
| 290 | |
Gregory P. Smith | 06babb2 | 2019-02-23 10:43:49 -0800 | [diff] [blame] | 291 | * When the Python interpreter is interrupted by Ctrl-C (SIGINT) and the |
| 292 | resulting :exc:`KeyboardInterrupt` exception is not caught, the Python process |
| 293 | now exits via a SIGINT signal or with the correct exit code such that the |
| 294 | calling process can detect that it died due to a Ctrl-C. Shells on POSIX |
| 295 | and Windows use this to properly terminate scripts in interactive sessions. |
| 296 | (Contributed by Google via Gregory P. Smith in :issue:`1054041`.) |
| 297 | |
Victor Stinner | a9f05d6 | 2019-05-24 23:57:23 +0200 | [diff] [blame] | 298 | * Added new ``replace()`` method to the code type (:class:`types.CodeType`). |
| 299 | (Contributed by Victor Stinner in :issue:`37032`.) |
| 300 | |
Serhiy Storchaka | 6543912 | 2018-10-19 17:42:06 +0300 | [diff] [blame] | 301 | |
Ned Deily | 07a1892 | 2018-01-31 18:12:38 -0500 | [diff] [blame] | 302 | New Modules |
| 303 | =========== |
| 304 | |
| 305 | * None yet. |
| 306 | |
| 307 | |
| 308 | Improved Modules |
| 309 | ================ |
| 310 | |
Raymond Hettinger | 0bb4bdf | 2019-01-31 00:59:50 -0800 | [diff] [blame] | 311 | |
Victor Stinner | 6ea29c5 | 2018-09-25 08:27:08 -0700 | [diff] [blame] | 312 | asyncio |
| 313 | ------- |
| 314 | |
| 315 | On Windows, the default event loop is now :class:`~asyncio.ProactorEventLoop`. |
| 316 | |
Matthias Bussonnier | 2ddbd21 | 2019-05-22 12:07:45 -0700 | [diff] [blame] | 317 | builtins |
| 318 | -------- |
| 319 | |
| 320 | The :func:`compile` built-in has been improved to accept the |
| 321 | ``ast.PyCF_ALLOW_TOP_LEVEL_AWAIT`` flag. With this new flag passed, |
| 322 | :func:`compile` will allow top-level ``await``, ``async for`` and ``async with`` |
| 323 | constructs that are usually considered invalid syntax. Asynchronous code object |
| 324 | marked with the ``CO_COROUTINE`` flag may then be returned. |
| 325 | |
| 326 | (Contributed by Matthias Bussonnier in :issue:`34616`) |
Terry Jan Reedy | fdcb5ae | 2018-09-25 12:45:27 -0400 | [diff] [blame] | 327 | |
Raymond Hettinger | 482b6b5 | 2019-05-01 17:48:13 -0700 | [diff] [blame] | 328 | collections |
| 329 | ----------- |
| 330 | |
| 331 | The :meth:`_asdict()` method for :func:`collections.namedtuple` now returns |
Daniel Porteous | 0522291 | 2019-05-02 04:20:59 -0400 | [diff] [blame] | 332 | a :class:`dict` instead of a :class:`collections.OrderedDict`. This works because |
| 333 | regular dicts have guaranteed ordering since Python 3.7. If the extra |
Raymond Hettinger | 482b6b5 | 2019-05-01 17:48:13 -0700 | [diff] [blame] | 334 | features of :class:`OrderedDict` are required, the suggested remediation is |
| 335 | to cast the result to the desired type: ``OrderedDict(nt._asdict())``. |
| 336 | (Contributed by Raymond Hettinger in :issue:`35864`.) |
| 337 | |
| 338 | |
Steve Dower | 2438cdf | 2019-03-29 16:37:16 -0700 | [diff] [blame] | 339 | ctypes |
| 340 | ------ |
| 341 | |
| 342 | On Windows, :class:`~ctypes.CDLL` and subclasses now accept a *winmode* parameter |
| 343 | to specify flags for the underlying ``LoadLibraryEx`` call. The default flags are |
| 344 | set to only load DLL dependencies from trusted locations, including the path |
| 345 | where the DLL is stored (if a full or partial path is used to load the initial |
| 346 | DLL) and paths added by :func:`~os.add_dll_directory`. |
| 347 | |
| 348 | |
Raymond Hettinger | b821868 | 2019-05-26 11:27:35 -0700 | [diff] [blame] | 349 | functools |
| 350 | --------- |
| 351 | |
| 352 | :func:`functools.lru_cache` can now be used as a straight decorator rather |
| 353 | than as a function returning a decorator. So both of these are now supported:: |
| 354 | |
| 355 | @lru_cache |
| 356 | def f(x): |
| 357 | ... |
| 358 | |
| 359 | @lru_cache(maxsize=256) |
| 360 | def f(x): |
| 361 | ... |
| 362 | |
| 363 | (Contributed by Raymond Hettinger in :issue:`36772`.) |
| 364 | |
| 365 | |
Paul Ganssle | 88c0937 | 2019-04-29 09:22:03 -0400 | [diff] [blame] | 366 | datetime |
| 367 | -------- |
| 368 | |
| 369 | Added new alternate constructors :meth:`datetime.date.fromisocalendar` and |
| 370 | :meth:`datetime.datetime.fromisocalendar`, which construct :class:`date` and |
| 371 | :class:`datetime` objects respectively from ISO year, week number and weekday; |
| 372 | these are the inverse of each class's ``isocalendar`` method. |
| 373 | (Contributed by Paul Ganssle in :issue:`36004`.) |
| 374 | |
| 375 | |
Cheryl Sabella | 637a33b | 2018-11-07 09:12:20 -0500 | [diff] [blame] | 376 | gettext |
| 377 | ------- |
| 378 | |
| 379 | Added :func:`~gettext.pgettext` and its variants. |
| 380 | (Contributed by Franz Glasner, Éric Araujo, and Cheryl Sabella in :issue:`2504`.) |
| 381 | |
Raymond Hettinger | d1e768a | 2019-03-25 13:01:13 -0700 | [diff] [blame] | 382 | inspect |
| 383 | ------- |
| 384 | |
| 385 | The :func:`inspect.getdoc` function can now find docstrings for ``__slots__`` |
| 386 | if that attribute is a :class:`dict` where the values are docstrings. |
| 387 | This provides documentation options similar to what we already have |
| 388 | for :func:`property`, :func:`classmethod`, and :func:`staticmethod`:: |
| 389 | |
| 390 | class AudioClip: |
| 391 | __slots__ = {'bit_rate': 'expressed in kilohertz to one decimal place', |
| 392 | 'duration': 'in seconds, rounded up to an integer'} |
| 393 | def __init__(self, bit_rate, duration): |
| 394 | self.bit_rate = round(bit_rate / 1000.0, 1) |
| 395 | self.duration = ceil(duration) |
Pablo Galindo | 175421b | 2019-02-23 03:02:06 +0000 | [diff] [blame] | 396 | |
Victor Stinner | bc2aa81 | 2019-05-23 03:45:09 +0200 | [diff] [blame] | 397 | io |
| 398 | -- |
| 399 | |
| 400 | In development mode (:option:`-X` ``env``) and in debug build, the |
| 401 | :class:`io.IOBase` finalizer now logs the exception if the ``close()`` method |
| 402 | fails. The exception is ignored silently by default in release build. |
| 403 | (Contributed by Victor Stinner in :issue:`18748`.) |
| 404 | |
| 405 | |
Pablo Galindo | 175421b | 2019-02-23 03:02:06 +0000 | [diff] [blame] | 406 | gc |
| 407 | -- |
| 408 | |
| 409 | :func:`~gc.get_objects` can now receive an optional *generation* parameter |
| 410 | indicating a generation to get objects from. Contributed in |
| 411 | :issue:`36016` by Pablo Galindo. |
| 412 | |
| 413 | |
guoci | 0e7497c | 2018-11-07 04:50:23 -0500 | [diff] [blame] | 414 | gzip |
| 415 | ---- |
| 416 | |
| 417 | Added the *mtime* parameter to :func:`gzip.compress` for reproducible output. |
| 418 | (Contributed by Guo Ci Teo in :issue:`34898`.) |
| 419 | |
Zackery Spytz | cf599f6 | 2019-05-13 01:50:52 -0600 | [diff] [blame] | 420 | A :exc:`~gzip.BadGzipFile` exception is now raised instead of :exc:`OSError` |
| 421 | for certain types of invalid or corrupt gzip files. |
| 422 | (Contributed by Filip Gruszczyński, Michele Orrù, and Zackery Spytz in |
| 423 | :issue:`6584`.) |
| 424 | |
guoci | 0e7497c | 2018-11-07 04:50:23 -0500 | [diff] [blame] | 425 | |
Terry Jan Reedy | fdcb5ae | 2018-09-25 12:45:27 -0400 | [diff] [blame] | 426 | idlelib and IDLE |
| 427 | ---------------- |
| 428 | |
| 429 | Output over N lines (50 by default) is squeezed down to a button. |
| 430 | N can be changed in the PyShell section of the General page of the |
| 431 | Settings dialog. Fewer, but possibly extra long, lines can be squeezed by |
| 432 | right clicking on the output. Squeezed output can be expanded in place |
| 433 | by double-clicking the button or into the clipboard or a separate window |
| 434 | by right-clicking the button. (Contributed by Tal Einat in :issue:`1529353`.) |
| 435 | |
| 436 | The changes above have been backported to 3.7 maintenance releases. |
| 437 | |
| 438 | |
HongWeipeng | f194479 | 2018-11-07 18:09:32 +0800 | [diff] [blame] | 439 | json.tool |
| 440 | --------- |
| 441 | |
| 442 | Add option ``--json-lines`` to parse every input line as separate JSON object. |
| 443 | (Contributed by Weipeng Hong in :issue:`31553`.) |
| 444 | |
Pablo Galindo | bc09851 | 2019-02-07 07:04:02 +0000 | [diff] [blame] | 445 | |
| 446 | math |
| 447 | ---- |
| 448 | |
Raymond Hettinger | 3ff5962 | 2019-02-16 11:00:42 -0800 | [diff] [blame] | 449 | Added new function :func:`math.dist` for computing Euclidean distance |
| 450 | between two points. (Contributed by Raymond Hettinger in :issue:`33089`.) |
| 451 | |
| 452 | Expanded the :func:`math.hypot` function to handle multiple dimensions. |
| 453 | Formerly, it only supported the 2-D case. |
| 454 | (Contributed by Raymond Hettinger in :issue:`33089`.) |
| 455 | |
Pablo Galindo | bc09851 | 2019-02-07 07:04:02 +0000 | [diff] [blame] | 456 | Added new function, :func:`math.prod`, as analogous function to :func:`sum` |
| 457 | that returns the product of a 'start' value (default: 1) times an iterable of |
Raymond Hettinger | 3ff5962 | 2019-02-16 11:00:42 -0800 | [diff] [blame] | 458 | numbers. (Contributed by Pablo Galindo in :issue:`35606`) |
Pablo Galindo | bc09851 | 2019-02-07 07:04:02 +0000 | [diff] [blame] | 459 | |
Mark Dickinson | 73934b9 | 2019-05-18 12:29:50 +0100 | [diff] [blame] | 460 | Added new function :func:`math.isqrt` for computing integer square roots. |
| 461 | (Contributed by Mark Dickinson in :issue:`36887`.) |
| 462 | |
Zackery Spytz | 02db696 | 2019-05-27 10:48:17 -0600 | [diff] [blame] | 463 | |
| 464 | mmap |
| 465 | ---- |
| 466 | |
| 467 | The :class:`mmap.mmap` class now has an :meth:`~mmap.mmap.madvise` method to |
| 468 | access the ``madvise()`` system call. |
| 469 | (Contributed by Zackery Spytz in :issue:`32941`.) |
| 470 | |
| 471 | |
Victor Stinner | 17a5588 | 2019-05-28 16:02:50 +0200 | [diff] [blame] | 472 | multiprocessing |
| 473 | --------------- |
| 474 | |
| 475 | Added new :mod:`multiprocessing.shared_memory` module. |
| 476 | (Contributed Davin Potts in :issue:`35813`.) |
| 477 | |
| 478 | On macOS, the *spawn* start method is now used by default. |
| 479 | (Contributed by Victor Stinner in :issue:`33725`.) |
| 480 | |
| 481 | |
Steve Dower | 2438cdf | 2019-03-29 16:37:16 -0700 | [diff] [blame] | 482 | os |
| 483 | -- |
| 484 | |
| 485 | Added new function :func:`~os.add_dll_directory` on Windows for providing |
| 486 | additional search paths for native dependencies when importing extension |
| 487 | modules or loading DLLs using :mod:`ctypes`. |
| 488 | |
Pablo Galindo | bc09851 | 2019-02-07 07:04:02 +0000 | [diff] [blame] | 489 | |
Serhiy Storchaka | 0185f34 | 2018-09-18 11:28:51 +0300 | [diff] [blame] | 490 | os.path |
| 491 | ------- |
| 492 | |
| 493 | :mod:`os.path` functions that return a boolean result like |
| 494 | :func:`~os.path.exists`, :func:`~os.path.lexists`, :func:`~os.path.isdir`, |
| 495 | :func:`~os.path.isfile`, :func:`~os.path.islink`, and :func:`~os.path.ismount` |
| 496 | now return ``False`` instead of raising :exc:`ValueError` or its subclasses |
| 497 | :exc:`UnicodeEncodeError` and :exc:`UnicodeDecodeError` for paths that contain |
| 498 | characters or bytes unrepresentable at the OS level. |
| 499 | (Contributed by Serhiy Storchaka in :issue:`33721`.) |
| 500 | |
Steve Dower | 8ef864d | 2019-03-12 15:15:26 -0700 | [diff] [blame] | 501 | :func:`~os.path.expanduser` on Windows now prefers the :envvar:`USERPROFILE` |
| 502 | environment variable and does not use :envvar:`HOME`, which is not normally set |
| 503 | for regular user accounts. |
| 504 | |
Serhiy Storchaka | b232df9 | 2018-10-30 13:22:42 +0200 | [diff] [blame] | 505 | |
| 506 | ncurses |
| 507 | ------- |
| 508 | |
| 509 | Added a new variable holding structured version information for the |
| 510 | underlying ncurses library: :data:`~curses.ncurses_version`. |
| 511 | (Contributed by Serhiy Storchaka in :issue:`31680`.) |
| 512 | |
| 513 | |
Serhiy Storchaka | 0185f34 | 2018-09-18 11:28:51 +0300 | [diff] [blame] | 514 | pathlib |
| 515 | ------- |
| 516 | |
| 517 | :mod:`pathlib.Path` methods that return a boolean result like |
| 518 | :meth:`~pathlib.Path.exists()`, :meth:`~pathlib.Path.is_dir()`, |
| 519 | :meth:`~pathlib.Path.is_file()`, :meth:`~pathlib.Path.is_mount()`, |
| 520 | :meth:`~pathlib.Path.is_symlink()`, :meth:`~pathlib.Path.is_block_device()`, |
| 521 | :meth:`~pathlib.Path.is_char_device()`, :meth:`~pathlib.Path.is_fifo()`, |
| 522 | :meth:`~pathlib.Path.is_socket()` now return ``False`` instead of raising |
| 523 | :exc:`ValueError` or its subclass :exc:`UnicodeEncodeError` for paths that |
| 524 | contain characters unrepresentable at the OS level. |
| 525 | (Contributed by Serhiy Storchaka in :issue:`33721`.) |
| 526 | |
Joannah Nanjekye | 6b5b013 | 2019-05-04 11:27:10 -0400 | [diff] [blame] | 527 | Added :meth:`pathlib.Path.link_to()` which creates a hard link pointing |
| 528 | to a path. |
| 529 | (Contributed by Joannah Nanjekye in :issue:`26978`) |
| 530 | |
jab | 9e00d9e | 2018-12-28 13:03:40 -0500 | [diff] [blame] | 531 | |
Jon Janzen | c981ad1 | 2019-05-15 22:14:38 +0200 | [diff] [blame] | 532 | plistlib |
| 533 | -------- |
| 534 | |
| 535 | Added new :class:`plistlib.UID` and enabled support for reading and writing |
| 536 | NSKeyedArchiver-encoded binary plists. |
| 537 | (Contributed by Jon Janzen in :issue:`26707`.) |
| 538 | |
| 539 | |
Giampaolo Rodola | eb7e29f | 2019-04-09 00:34:02 +0200 | [diff] [blame] | 540 | socket |
| 541 | ------ |
| 542 | |
| 543 | Added :meth:`~socket.create_server()` and :meth:`~socket.has_dualstack_ipv6()` |
| 544 | convenience functions to automate the necessary tasks usually involved when |
| 545 | creating a server socket, including accepting both IPv4 and IPv6 connections |
| 546 | on the same socket. (Contributed by Giampaolo Rodola in :issue:`17561`.) |
| 547 | |
| 548 | |
jab | 9e00d9e | 2018-12-28 13:03:40 -0500 | [diff] [blame] | 549 | shutil |
| 550 | ------ |
| 551 | |
| 552 | :func:`shutil.copytree` now accepts a new ``dirs_exist_ok`` keyword argument. |
| 553 | (Contributed by Josh Bronson in :issue:`20849`.) |
| 554 | |
CAM Gerlach | 89a8944 | 2019-04-06 23:47:49 -0500 | [diff] [blame] | 555 | :func:`shutil.make_archive` now defaults to the modern pax (POSIX.1-2001) |
| 556 | format for new archives to improve portability and standards conformance, |
| 557 | inherited from the corresponding change to the :mod:`tarfile` module. |
| 558 | (Contributed by C.A.M. Gerlach in :issue:`30661`.) |
| 559 | |
jab | 9e00d9e | 2018-12-28 13:03:40 -0500 | [diff] [blame] | 560 | |
Christian Heimes | 9fb051f | 2018-09-23 08:32:31 +0200 | [diff] [blame] | 561 | ssl |
| 562 | --- |
| 563 | |
| 564 | Added :attr:`SSLContext.post_handshake_auth` to enable and |
| 565 | :meth:`ssl.SSLSocket.verify_client_post_handshake` to initiate TLS 1.3 |
| 566 | post-handshake authentication. |
| 567 | (Contributed by Christian Heimes in :issue:`34670`.) |
| 568 | |
Raymond Hettinger | 47d9987 | 2019-02-21 15:06:29 -0800 | [diff] [blame] | 569 | |
| 570 | statistics |
| 571 | ---------- |
| 572 | |
| 573 | Added :func:`statistics.fmean` as a faster, floating point variant of |
| 574 | :func:`statistics.mean()`. (Contributed by Raymond Hettinger and |
| 575 | Steven D'Aprano in :issue:`35904`.) |
| 576 | |
Raymond Hettinger | 6463ba3 | 2019-04-07 09:20:03 -0700 | [diff] [blame] | 577 | Added :func:`statistics.geometric_mean()` |
| 578 | (Contributed by Raymond Hettinger in :issue:`27181`.) |
| 579 | |
Raymond Hettinger | fc06a19 | 2019-03-12 00:43:27 -0700 | [diff] [blame] | 580 | Added :func:`statistics.multimode` that returns a list of the most |
| 581 | common values. (Contributed by Raymond Hettinger in :issue:`35892`.) |
| 582 | |
Raymond Hettinger | 9013ccf | 2019-04-23 00:06:35 -0700 | [diff] [blame] | 583 | Added :func:`statistics.quantiles` that divides data or a distribution |
| 584 | in to equiprobable intervals (e.g. quartiles, deciles, or percentiles). |
| 585 | (Contributed by Raymond Hettinger in :issue:`36546`.) |
| 586 | |
Raymond Hettinger | 11c7953 | 2019-02-23 14:44:07 -0800 | [diff] [blame] | 587 | Added :class:`statistics.NormalDist`, a tool for creating |
| 588 | and manipulating normal distributions of a random variable. |
| 589 | (Contributed by Raymond Hettinger in :issue:`36018`.) |
| 590 | |
| 591 | :: |
| 592 | |
| 593 | >>> temperature_feb = NormalDist.from_samples([4, 12, -3, 2, 7, 14]) |
Raymond Hettinger | 671d782 | 2019-05-01 17:49:12 -0700 | [diff] [blame] | 594 | >>> temperature_feb.mean |
| 595 | 6.0 |
| 596 | >>> temperature_feb.stdev |
| 597 | 6.356099432828281 |
Raymond Hettinger | 11c7953 | 2019-02-23 14:44:07 -0800 | [diff] [blame] | 598 | |
| 599 | >>> temperature_feb.cdf(3) # Chance of being under 3 degrees |
| 600 | 0.3184678262814532 |
| 601 | >>> # Relative chance of being 7 degrees versus 10 degrees |
| 602 | >>> temperature_feb.pdf(7) / temperature_feb.pdf(10) |
| 603 | 1.2039930378537762 |
| 604 | |
Raymond Hettinger | 671d782 | 2019-05-01 17:49:12 -0700 | [diff] [blame] | 605 | >>> el_niño = NormalDist(4, 2.5) |
| 606 | >>> temperature_feb += el_niño # Add in a climate effect |
Raymond Hettinger | 11c7953 | 2019-02-23 14:44:07 -0800 | [diff] [blame] | 607 | >>> temperature_feb |
| 608 | NormalDist(mu=10.0, sigma=6.830080526611674) |
| 609 | |
| 610 | >>> temperature_feb * (9/5) + 32 # Convert to Fahrenheit |
| 611 | NormalDist(mu=50.0, sigma=12.294144947901014) |
| 612 | >>> temperature_feb.samples(3) # Generate random samples |
| 613 | [7.672102882379219, 12.000027119750287, 4.647488369766392] |
| 614 | |
Raymond Hettinger | 47d9987 | 2019-02-21 15:06:29 -0800 | [diff] [blame] | 615 | |
Victor Stinner | ef9d9b6 | 2019-05-22 11:28:22 +0200 | [diff] [blame] | 616 | sys |
| 617 | --- |
| 618 | |
| 619 | Add new :func:`sys.unraisablehook` function which can be overridden to control |
| 620 | how "unraisable exceptions" are handled. It is called when an exception has |
| 621 | occurred but there is no way for Python to handle it. For example, when a |
| 622 | destructor raises an exception or during garbage collection |
| 623 | (:func:`gc.collect`). |
| 624 | |
| 625 | |
CAM Gerlach | e680c3d | 2019-03-21 09:44:51 -0500 | [diff] [blame] | 626 | tarfile |
| 627 | ------- |
| 628 | |
| 629 | The :mod:`tarfile` module now defaults to the modern pax (POSIX.1-2001) |
| 630 | format for new archives, instead of the previous GNU-specific one. |
| 631 | This improves cross-platform portability with a consistent encoding (UTF-8) |
| 632 | in a standardized and extensible format, and offers several other benefits. |
| 633 | (Contributed by C.A.M. Gerlach in :issue:`36268`.) |
| 634 | |
| 635 | |
Victor Stinner | cd590a7 | 2019-05-28 00:39:52 +0200 | [diff] [blame] | 636 | threading |
| 637 | --------- |
| 638 | |
| 639 | Add a new :func:`threading.excepthook` function which handles uncaught |
| 640 | :meth:`threading.Thread.run` exception. It can be overridden to control how |
| 641 | uncaught :meth:`threading.Thread.run` exceptions are handled. |
| 642 | (Contributed by Victor Stinner in :issue:`1230540`.) |
| 643 | |
| 644 | |
Tal Einat | dfba1f6 | 2018-10-24 10:20:05 +0300 | [diff] [blame] | 645 | tokenize |
| 646 | -------- |
| 647 | |
| 648 | The :mod:`tokenize` module now implicitly emits a ``NEWLINE`` token when |
| 649 | provided with input that does not have a trailing new line. This behavior |
| 650 | now matches what the C tokenizer does internally. |
| 651 | (Contributed by Ammar Askar in :issue:`33899`.) |
| 652 | |
Juliette Monsel | af5658a | 2018-10-08 18:29:24 +0200 | [diff] [blame] | 653 | tkinter |
| 654 | ------- |
| 655 | |
| 656 | Added methods :meth:`~tkinter.Spinbox.selection_from`, |
| 657 | :meth:`~tkinter.Spinbox.selection_present`, |
| 658 | :meth:`~tkinter.Spinbox.selection_range` and |
| 659 | :meth:`~tkinter.Spinbox.selection_to` |
| 660 | in the :class:`tkinter.Spinbox` class. |
| 661 | (Contributed by Juliette Monsel in :issue:`34829`.) |
| 662 | |
Juliette Monsel | bf03471 | 2018-10-12 18:44:10 +0200 | [diff] [blame] | 663 | Added method :meth:`~tkinter.Canvas.moveto` |
| 664 | in the :class:`tkinter.Canvas` class. |
| 665 | (Contributed by Juliette Monsel in :issue:`23831`.) |
| 666 | |
Zackery Spytz | 50866e9 | 2019-04-05 04:17:13 -0600 | [diff] [blame] | 667 | The :class:`tkinter.PhotoImage` class now has |
| 668 | :meth:`~tkinter.PhotoImage.transparency_get` and |
| 669 | :meth:`~tkinter.PhotoImage.transparency_set` methods. (Contributed by |
| 670 | Zackery Spytz in :issue:`25451`.) |
| 671 | |
Joannah Nanjekye | 572168a | 2019-01-10 19:56:38 +0300 | [diff] [blame] | 672 | time |
| 673 | ---- |
| 674 | |
| 675 | Added new clock :data:`~time.CLOCK_UPTIME_RAW` for macOS 10.12. |
| 676 | (Contributed by Joannah Nanjekye in :issue:`35702`.) |
| 677 | |
Max Bélanger | 2810dd7 | 2018-11-04 15:58:24 -0800 | [diff] [blame] | 678 | unicodedata |
| 679 | ----------- |
| 680 | |
Benjamin Peterson | 3aca40d | 2019-05-08 20:59:35 -0700 | [diff] [blame] | 681 | * The :mod:`unicodedata` module has been upgraded to use the `Unicode 12.1.0 |
| 682 | <http://blog.unicode.org/2019/05/unicode-12-1-en.html>`_ release. |
Raymond Hettinger | 482b6b5 | 2019-05-01 17:48:13 -0700 | [diff] [blame] | 683 | |
Max Bélanger | 2810dd7 | 2018-11-04 15:58:24 -0800 | [diff] [blame] | 684 | * New function :func:`~unicodedata.is_normalized` can be used to verify a string |
| 685 | is in a specific normal form. (Contributed by Max Belanger and David Euresti in |
| 686 | :issue:`32285`). |
| 687 | |
Raymond Hettinger | 482b6b5 | 2019-05-01 17:48:13 -0700 | [diff] [blame] | 688 | |
Lisa Roach | 0f221d0 | 2018-11-08 18:34:33 -0800 | [diff] [blame] | 689 | unittest |
| 690 | -------- |
| 691 | |
Lisa Roach | 77b3b77 | 2019-05-20 09:19:53 -0700 | [diff] [blame] | 692 | * XXX Added :class:`AsyncMock` to support an asynchronous version of :class:`Mock`. |
| 693 | Appropriate new assert functions for testing have been added as well. |
| 694 | (Contributed by Lisa Roach in :issue:`26467`). |
| 695 | |
Lisa Roach | 0f221d0 | 2018-11-08 18:34:33 -0800 | [diff] [blame] | 696 | * Added :func:`~unittest.addModuleCleanup()` and |
| 697 | :meth:`~unittest.TestCase.addClassCleanup()` to unittest to support |
| 698 | cleanups for :func:`~unittest.setUpModule()` and |
| 699 | :meth:`~unittest.TestCase.setUpClass()`. |
| 700 | (Contributed by Lisa Roach in :issue:`24412`.) |
| 701 | |
Brett Cannon | d64ee1a | 2018-09-21 15:27:26 -0700 | [diff] [blame] | 702 | venv |
| 703 | ---- |
| 704 | |
| 705 | * :mod:`venv` now includes an ``Activate.ps1`` script on all platforms for |
| 706 | activating virtual environments under PowerShell Core 6.1. |
| 707 | (Contributed by Brett Cannon in :issue:`32718`.) |
| 708 | |
Mark Dickinson | 7abb6c0 | 2019-04-26 15:56:15 +0900 | [diff] [blame] | 709 | weakref |
| 710 | ------- |
| 711 | |
| 712 | * The proxy objects returned by :func:`weakref.proxy` now support the matrix |
| 713 | multiplication operators ``@`` and ``@=`` in addition to the other |
| 714 | numeric operators. (Contributed by Mark Dickinson in :issue:`36669`.) |
| 715 | |
Christian Heimes | 17b1d5d | 2018-09-23 09:50:25 +0200 | [diff] [blame] | 716 | xml |
| 717 | --- |
| 718 | |
| 719 | * As mitigation against DTD and external entity retrieval, the |
Andrés Delfino | ca68261 | 2018-11-07 14:29:14 -0300 | [diff] [blame] | 720 | :mod:`xml.dom.minidom` and :mod:`xml.sax` modules no longer process |
Christian Heimes | 17b1d5d | 2018-09-23 09:50:25 +0200 | [diff] [blame] | 721 | external entities by default. |
| 722 | (Contributed by Christian Heimes in :issue:`17239`.) |
| 723 | |
Stefan Behnel | 4754168 | 2019-05-03 20:58:16 +0200 | [diff] [blame] | 724 | * The ``.find*()`` methods in the :mod:`xml.etree.ElementTree` module |
| 725 | support wildcard searches like ``{*}tag`` which ignores the namespace |
| 726 | and ``{namespace}*`` which returns all tags in the given namespace. |
| 727 | (Contributed by Stefan Behnel in :issue:`28238`.) |
| 728 | |
Stefan Behnel | e1d5dd6 | 2019-05-01 22:34:13 +0200 | [diff] [blame] | 729 | * The :mod:`xml.etree.ElementTree` module provides a new function |
| 730 | :func:`–xml.etree.ElementTree.canonicalize()` that implements C14N 2.0. |
| 731 | (Contributed by Stefan Behnel in :issue:`13611`.) |
| 732 | |
Stefan Behnel | e9a465f | 2019-05-10 10:25:13 +0200 | [diff] [blame] | 733 | * The target object of :class:`xml.etree.ElementTree.XMLParser` can |
| 734 | receive namespace declaration events through the new callback methods |
| 735 | ``start_ns()`` and ``end_ns()``. Additionally, the |
| 736 | :class:`xml.etree.ElementTree.TreeBuilder` target can be configured |
| 737 | to process events about comments and processing instructions to include |
| 738 | them in the generated tree. |
| 739 | (Contributed by Stefan Behnel in :issue:`36676` and :issue:`36673`.) |
Christian Heimes | 17b1d5d | 2018-09-23 09:50:25 +0200 | [diff] [blame] | 740 | |
Ned Deily | 07a1892 | 2018-01-31 18:12:38 -0500 | [diff] [blame] | 741 | Optimizations |
| 742 | ============= |
| 743 | |
Victor Stinner | 9daecf3 | 2019-01-16 00:02:35 +0100 | [diff] [blame] | 744 | * The :mod:`subprocess` module can now use the :func:`os.posix_spawn` function |
| 745 | in some cases for better performance. Currently, it is only used on macOS |
| 746 | and Linux (using glibc 2.24 or newer) if all these conditions are met: |
| 747 | |
| 748 | * *close_fds* is false; |
Victor Stinner | f6243ac | 2019-01-23 19:00:39 +0100 | [diff] [blame] | 749 | * *preexec_fn*, *pass_fds*, *cwd* and *start_new_session* parameters |
| 750 | are not set; |
Victor Stinner | 8c34956 | 2019-01-16 23:38:06 +0100 | [diff] [blame] | 751 | * the *executable* path contains a directory. |
Victor Stinner | 9daecf3 | 2019-01-16 00:02:35 +0100 | [diff] [blame] | 752 | |
Giampaolo Rodola | 4a172cc | 2018-06-12 23:04:50 +0200 | [diff] [blame] | 753 | * :func:`shutil.copyfile`, :func:`shutil.copy`, :func:`shutil.copy2`, |
| 754 | :func:`shutil.copytree` and :func:`shutil.move` use platform-specific |
Giampaolo Rodola | c7f02a9 | 2018-06-19 08:27:29 -0700 | [diff] [blame] | 755 | "fast-copy" syscalls on Linux, macOS and Solaris in order to copy the file |
| 756 | more efficiently. |
Giampaolo Rodola | 4a172cc | 2018-06-12 23:04:50 +0200 | [diff] [blame] | 757 | "fast-copy" means that the copying operation occurs within the kernel, |
| 758 | avoiding the use of userspace buffers in Python as in |
| 759 | "``outfd.write(infd.read())``". |
Giampaolo Rodola | c7f02a9 | 2018-06-19 08:27:29 -0700 | [diff] [blame] | 760 | On Windows :func:`shutil.copyfile` uses a bigger default buffer size (1 MiB |
| 761 | instead of 16 KiB) and a :func:`memoryview`-based variant of |
| 762 | :func:`shutil.copyfileobj` is used. |
| 763 | The speedup for copying a 512 MiB file within the same partition is about |
| 764 | +26% on Linux, +50% on macOS and +40% on Windows. Also, much less CPU cycles |
| 765 | are consumed. |
| 766 | See :ref:`shutil-platform-dependent-efficient-copy-operations` section. |
Mariatta | 16501b7 | 2018-12-06 21:59:42 -0800 | [diff] [blame] | 767 | (Contributed by Giampaolo Rodola' in :issue:`33671`.) |
Giampaolo Rodola | 4a172cc | 2018-06-12 23:04:50 +0200 | [diff] [blame] | 768 | |
Giampaolo Rodola | 19c46a4 | 2018-11-12 06:18:15 -0800 | [diff] [blame] | 769 | * :func:`shutil.copytree` uses :func:`os.scandir` function and all copy |
| 770 | functions depending from it use cached :func:`os.stat` values. The speedup |
| 771 | for copying a directory with 8000 files is around +9% on Linux, +20% on |
| 772 | Windows and +30% on a Windows SMB share. Also the number of :func:`os.stat` |
| 773 | syscalls is reduced by 38% making :func:`shutil.copytree` especially faster |
| 774 | on network filesystems. (Contributed by Giampaolo Rodola' in :issue:`33695`.) |
| 775 | |
Łukasz Langa | c51d8c9 | 2018-04-03 23:06:53 -0700 | [diff] [blame] | 776 | * The default protocol in the :mod:`pickle` module is now Protocol 4, |
| 777 | first introduced in Python 3.4. It offers better performance and smaller |
| 778 | size compared to Protocol 3 available since Python 3.0. |
Ned Deily | 07a1892 | 2018-01-31 18:12:38 -0500 | [diff] [blame] | 779 | |
INADA Naoki | d5c875b | 2018-07-11 17:42:49 +0900 | [diff] [blame] | 780 | * Removed one ``Py_ssize_t`` member from ``PyGC_Head``. All GC tracked |
| 781 | objects (e.g. tuple, list, dict) size is reduced 4 or 8 bytes. |
| 782 | (Contributed by Inada Naoki in :issue:`33597`) |
| 783 | |
Tal Einat | 5475253 | 2018-09-10 16:11:04 +0300 | [diff] [blame] | 784 | * :class:`uuid.UUID` now uses ``__slots__`` to reduce its memory footprint. |
Tal Einat | 5475253 | 2018-09-10 16:11:04 +0300 | [diff] [blame] | 785 | |
Raymond Hettinger | 63fa1cf | 2019-02-16 12:02:22 -0800 | [diff] [blame] | 786 | * Improved performance of :func:`operator.itemgetter` by 33%. Optimized |
| 787 | argument handling and added a fast path for the common case of a single |
| 788 | non-negative integer index into a tuple (which is the typical use case in |
| 789 | the standard library). (Contributed by Raymond Hettinger in |
| 790 | :issue:`35664`.) |
| 791 | |
| 792 | * Sped-up field lookups in :func:`collections.namedtuple`. They are now more |
| 793 | than two times faster, making them the fastest form of instance variable |
| 794 | lookup in Python. (Contributed by Raymond Hettinger, Pablo Galindo, and |
Joe Jevnik | f36f892 | 2019-02-21 16:00:40 -0500 | [diff] [blame] | 795 | Joe Jevnik, Serhiy Storchaka in :issue:`32492`.) |
Raymond Hettinger | 63fa1cf | 2019-02-16 12:02:22 -0800 | [diff] [blame] | 796 | |
Pablo Galindo | c61e229 | 2018-10-28 22:03:18 +0000 | [diff] [blame] | 797 | * The :class:`list` constructor does not overallocate the internal item buffer |
| 798 | if the input iterable has a known length (the input implements ``__len__``). |
Raymond Hettinger | e182318 | 2019-02-16 12:47:48 -0800 | [diff] [blame] | 799 | This makes the created list 12% smaller on average. (Contributed by |
| 800 | Raymond Hettinger and Pablo Galindo in :issue:`33234`.) |
Pablo Galindo | c61e229 | 2018-10-28 22:03:18 +0000 | [diff] [blame] | 801 | |
Stefan Behnel | d8b9e1f | 2019-02-20 18:29:24 +0100 | [diff] [blame] | 802 | * Doubled the speed of class variable writes. When a non-dunder attribute |
| 803 | was updated, there was an unnecessary call to update slots. |
| 804 | (Contributed by Stefan Behnel, Pablo Galindo Salgado, Raymond Hettinger, |
| 805 | Neil Schemenauer, and Serhiy Storchaka in :issue:`36012`.) |
| 806 | |
Serhiy Storchaka | 3191391 | 2019-03-14 10:32:22 +0200 | [diff] [blame] | 807 | * Reduced an overhead of converting arguments passed to many builtin functions |
| 808 | and methods. This sped up calling some simple builtin functions and |
| 809 | methods up to 20--50%. (Contributed by Serhiy Storchaka in :issue:`23867`, |
| 810 | :issue:`35582` and :issue:`36127`.) |
| 811 | |
Serhiy Storchaka | ceeef10 | 2018-06-15 11:09:43 +0300 | [diff] [blame] | 812 | |
Ned Deily | 07a1892 | 2018-01-31 18:12:38 -0500 | [diff] [blame] | 813 | Build and C API Changes |
| 814 | ======================= |
| 815 | |
Victor Stinner | bf4ac2d | 2019-01-22 17:39:03 +0100 | [diff] [blame] | 816 | * The :c:func:`PyByteArray_Init` and :c:func:`PyByteArray_Fini` functions have |
| 817 | been removed. They did nothing since Python 2.7.4 and Python 3.2.0, were |
| 818 | excluded from the limited API (stable ABI), and were not documented. |
| 819 | |
Serhiy Storchaka | ceeef10 | 2018-06-15 11:09:43 +0300 | [diff] [blame] | 820 | * The result of :c:func:`PyExceptionClass_Name` is now of type |
| 821 | ``const char *`` rather of ``char *``. |
| 822 | (Contributed by Serhiy Storchaka in :issue:`33818`.) |
Ned Deily | 07a1892 | 2018-01-31 18:12:38 -0500 | [diff] [blame] | 823 | |
Antoine Pitrou | 961d54c | 2018-07-16 19:03:03 +0200 | [diff] [blame] | 824 | * The duality of ``Modules/Setup.dist`` and ``Modules/Setup`` has been |
| 825 | removed. Previously, when updating the CPython source tree, one had |
| 826 | to manually copy ``Modules/Setup.dist`` (inside the source tree) to |
| 827 | ``Modules/Setup`` (inside the build tree) in order to reflect any changes |
| 828 | upstream. This was of a small benefit to packagers at the expense of |
| 829 | a frequent annoyance to developers following CPython development, as |
| 830 | forgetting to copy the file could produce build failures. |
| 831 | |
| 832 | Now the build system always reads from ``Modules/Setup`` inside the source |
| 833 | tree. People who want to customize that file are encouraged to maintain |
| 834 | their changes in a git fork of CPython or as patch files, as they would do |
| 835 | for any other change to the source tree. |
| 836 | |
| 837 | (Contributed by Antoine Pitrou in :issue:`32430`.) |
| 838 | |
Serhiy Storchaka | 6a44f6e | 2019-02-25 17:57:58 +0200 | [diff] [blame] | 839 | * Functions that convert Python number to C integer like |
| 840 | :c:func:`PyLong_AsLong` and argument parsing functions like |
| 841 | :c:func:`PyArg_ParseTuple` with integer converting format units like ``'i'`` |
| 842 | will now use the :meth:`~object.__index__` special method instead of |
| 843 | :meth:`~object.__int__`, if available. The deprecation warning will be |
| 844 | emitted for objects with the ``__int__()`` method but without the |
| 845 | ``__index__()`` method (like :class:`~decimal.Decimal` and |
| 846 | :class:`~fractions.Fraction`). :c:func:`PyNumber_Check` will now return |
| 847 | ``1`` for objects implementing ``__index__()``. |
| 848 | (Contributed by Serhiy Storchaka in :issue:`36048`.) |
| 849 | |
Eddie Elizondo | 364f0b0 | 2019-03-27 07:52:18 -0400 | [diff] [blame] | 850 | * Heap-allocated type objects will now increase their reference count |
| 851 | in :c:func:`PyObject_Init` (and its parallel macro ``PyObject_INIT``) |
| 852 | instead of in :c:func:`PyType_GenericAlloc`. Types that modify instance |
| 853 | allocation or deallocation may need to be adjusted. |
| 854 | (Contributed by Eddie Elizondo in :issue:`35810`.) |
| 855 | |
Ned Deily | 07a1892 | 2018-01-31 18:12:38 -0500 | [diff] [blame] | 856 | |
| 857 | Deprecated |
| 858 | ========== |
| 859 | |
Serhiy Storchaka | 02ec92f | 2018-07-24 12:03:34 +0300 | [diff] [blame] | 860 | * Deprecated methods ``getchildren()`` and ``getiterator()`` in |
| 861 | the :mod:`~xml.etree.ElementTree` module emit now a |
| 862 | :exc:`DeprecationWarning` instead of :exc:`PendingDeprecationWarning`. |
| 863 | They will be removed in Python 3.9. |
| 864 | (Contributed by Serhiy Storchaka in :issue:`29209`.) |
Ned Deily | 07a1892 | 2018-01-31 18:12:38 -0500 | [diff] [blame] | 865 | |
Elvis Pranskevichus | 22d2508 | 2018-07-30 11:42:43 +0100 | [diff] [blame] | 866 | * Passing an object that is not an instance of |
| 867 | :class:`concurrent.futures.ThreadPoolExecutor` to |
Yury Selivanov | 7c7605f | 2018-09-11 09:54:40 -0700 | [diff] [blame] | 868 | :meth:`asyncio.loop.set_default_executor()` is |
Elvis Pranskevichus | 22d2508 | 2018-07-30 11:42:43 +0100 | [diff] [blame] | 869 | deprecated and will be prohibited in Python 3.9. |
| 870 | (Contributed by Elvis Pranskevichus in :issue:`34075`.) |
| 871 | |
Berker Peksag | ef8861c | 2018-08-21 17:58:49 +0300 | [diff] [blame] | 872 | * The :meth:`__getitem__` methods of :class:`xml.dom.pulldom.DOMEventStream`, |
| 873 | :class:`wsgiref.util.FileWrapper` and :class:`fileinput.FileInput` have been |
| 874 | deprecated. |
| 875 | |
| 876 | Implementations of these methods have been ignoring their *index* parameter, |
| 877 | and returning the next item instead. |
| 878 | |
| 879 | (Contributed by Berker Peksag in :issue:`9372`.) |
| 880 | |
Raymond Hettinger | f7b57df | 2019-03-18 09:53:56 -0700 | [diff] [blame] | 881 | * The :class:`typing.NamedTuple` class has deprecated the ``_field_types`` |
| 882 | attribute in favor of the ``__annotations__`` attribute which has the same |
| 883 | information. (Contributed by Raymond Hettinger in :issue:`36320`.) |
| 884 | |
Serhiy Storchaka | 3f22811 | 2018-09-27 17:42:37 +0300 | [diff] [blame] | 885 | * :mod:`ast` classes ``Num``, ``Str``, ``Bytes``, ``NameConstant`` and |
| 886 | ``Ellipsis`` are considered deprecated and will be removed in future Python |
| 887 | versions. :class:`~ast.Constant` should be used instead. |
| 888 | (Contributed by Serhiy Storchaka in :issue:`32892`.) |
| 889 | |
Serhiy Storchaka | fec35c9 | 2018-10-27 08:00:41 +0300 | [diff] [blame] | 890 | * The following functions and methods are deprecated in the :mod:`gettext` |
| 891 | module: :func:`~gettext.lgettext`, :func:`~gettext.ldgettext`, |
| 892 | :func:`~gettext.lngettext` and :func:`~gettext.ldngettext`. |
| 893 | They return encoded bytes, and it's possible that you will get unexpected |
| 894 | Unicode-related exceptions if there are encoding problems with the |
| 895 | translated strings. It's much better to use alternatives which return |
| 896 | Unicode strings in Python 3. These functions have been broken for a long time. |
| 897 | |
| 898 | Function :func:`~gettext.bind_textdomain_codeset`, methods |
| 899 | :meth:`~gettext.NullTranslations.output_charset` and |
| 900 | :meth:`~gettext.NullTranslations.set_output_charset`, and the *codeset* |
| 901 | parameter of functions :func:`~gettext.translation` and |
| 902 | :func:`~gettext.install` are also deprecated, since they are only used for |
| 903 | for the ``l*gettext()`` functions. |
| 904 | |
| 905 | (Contributed by Serhiy Storchaka in :issue:`33710`.) |
| 906 | |
Dong-hee Na | 89669ff | 2019-01-17 21:14:45 +0900 | [diff] [blame] | 907 | * The :meth:`~threading.Thread.isAlive()` method of :class:`threading.Thread` has been deprecated. |
| 908 | (Contributed by Dong-hee Na in :issue:`35283`.) |
Ned Deily | 07a1892 | 2018-01-31 18:12:38 -0500 | [diff] [blame] | 909 | |
Serhiy Storchaka | 6a44f6e | 2019-02-25 17:57:58 +0200 | [diff] [blame] | 910 | * Many builtin and extension functions that take integer arguments will |
| 911 | now emit a deprecation warning for :class:`~decimal.Decimal`\ s, |
| 912 | :class:`~fractions.Fraction`\ s and any other objects that can be converted |
| 913 | to integers only with a loss (e.g. that have the :meth:`~object.__int__` |
| 914 | method but do not have the :meth:`~object.__index__` method). In future |
| 915 | version they will be errors. |
| 916 | (Contributed by Serhiy Storchaka in :issue:`36048`.) |
| 917 | |
Serhiy Storchaka | 42a139e | 2019-04-01 09:16:35 +0300 | [diff] [blame] | 918 | * Deprecated passing the following arguments as keyword arguments: |
| 919 | |
| 920 | - *func* in :func:`functools.partialmethod`, :func:`weakref.finalize`, |
| 921 | :meth:`profile.Profile.runcall`, :meth:`cProfile.Profile.runcall`, |
| 922 | :meth:`bdb.Bdb.runcall`, :meth:`trace.Trace.runfunc` and |
| 923 | :func:`curses.wrapper`. |
| 924 | - *function* in :func:`unittest.addModuleCleanup` and |
| 925 | :meth:`unittest.TestCase.addCleanup`. |
| 926 | - *fn* in the :meth:`~concurrent.futures.Executor.submit` method of |
| 927 | :class:`concurrent.futures.ThreadPoolExecutor` and |
| 928 | :class:`concurrent.futures.ProcessPoolExecutor`. |
| 929 | - *callback* in :meth:`contextlib.ExitStack.callback`, |
| 930 | :meth:`contextlib.AsyncExitStack.callback` and |
| 931 | :meth:`contextlib.AsyncExitStack.push_async_callback`. |
| 932 | - *c* and *typeid* in the :meth:`~multiprocessing.managers.Server.create` |
| 933 | method of :class:`multiprocessing.managers.Server` and |
| 934 | :class:`multiprocessing.managers.SharedMemoryServer`. |
| 935 | - *obj* in :func:`weakref.finalize`. |
| 936 | |
| 937 | In future releases of Python they will be :ref:`positional-only |
| 938 | <positional-only_parameter>`. |
| 939 | (Contributed by Serhiy Storchaka in :issue:`36492`.) |
| 940 | |
Serhiy Storchaka | 6a44f6e | 2019-02-25 17:57:58 +0200 | [diff] [blame] | 941 | |
Victor Stinner | 73104fa | 2018-11-29 09:58:20 +0100 | [diff] [blame] | 942 | API and Feature Removals |
| 943 | ======================== |
| 944 | |
| 945 | The following features and APIs have been removed from Python 3.8: |
| 946 | |
Victor Stinner | d7538dd | 2018-12-14 13:37:26 +0100 | [diff] [blame] | 947 | * The :mod:`macpath` module, deprecated in Python 3.7, has been removed. |
| 948 | (Contributed by Victor Stinner in :issue:`35471`.) |
| 949 | |
Victor Stinner | 73104fa | 2018-11-29 09:58:20 +0100 | [diff] [blame] | 950 | * The function :func:`platform.popen` has been removed, it was deprecated since |
| 951 | Python 3.3: use :func:`os.popen` instead. |
Ned Deily | 07a1892 | 2018-01-31 18:12:38 -0500 | [diff] [blame] | 952 | |
Matthias Bussonnier | b6a09ae | 2019-05-13 12:23:07 -0700 | [diff] [blame] | 953 | * The function :func:`time.clock` has been removed, it was deprecated since Python |
| 954 | 3.3: use :func:`time.perf_counter` or :func:`time.process_time` instead, depending |
| 955 | on your requirements, to have a well defined behavior. |
| 956 | |
Brett Cannon | a8c3424 | 2018-04-20 14:15:40 -0700 | [diff] [blame] | 957 | * The ``pyvenv`` script has been removed in favor of ``python3.8 -m venv`` |
| 958 | to help eliminate confusion as to what Python interpreter the ``pyvenv`` |
| 959 | script is tied to. (Contributed by Brett Cannon in :issue:`25427`.) |
Ned Deily | 07a1892 | 2018-01-31 18:12:38 -0500 | [diff] [blame] | 960 | |
INADA Naoki | 698865d | 2018-06-19 17:28:50 +0900 | [diff] [blame] | 961 | * ``parse_qs``, ``parse_qsl``, and ``escape`` are removed from :mod:`cgi` |
| 962 | module. They are deprecated from Python 3.2 or older. |
| 963 | |
INADA Naoki | 461a1c4 | 2018-06-28 17:10:36 +0900 | [diff] [blame] | 964 | * ``filemode`` function is removed from :mod:`tarfile` module. |
| 965 | It is not documented and deprecated since Python 3.3. |
INADA Naoki | 698865d | 2018-06-19 17:28:50 +0900 | [diff] [blame] | 966 | |
Serhiy Storchaka | 02ec92f | 2018-07-24 12:03:34 +0300 | [diff] [blame] | 967 | * The :class:`~xml.etree.ElementTree.XMLParser` constructor no longer accepts |
| 968 | the *html* argument. It never had effect and was deprecated in Python 3.4. |
| 969 | All other parameters are now :ref:`keyword-only <keyword-only_parameter>`. |
| 970 | (Contributed by Serhiy Storchaka in :issue:`29209`.) |
| 971 | |
| 972 | * Removed the ``doctype()`` method of :class:`~xml.etree.ElementTree.XMLParser`. |
| 973 | (Contributed by Serhiy Storchaka in :issue:`29209`.) |
| 974 | |
Inada Naoki | 6a16b18 | 2019-03-18 15:44:11 +0900 | [diff] [blame] | 975 | * "unicode_internal" codec is removed. |
| 976 | (Contributed by Inada Naoki in :issue:`36297`.) |
| 977 | |
Aviv Palivoda | e657624 | 2019-05-09 21:05:45 +0300 | [diff] [blame] | 978 | * The ``Cache`` and ``Statement`` objects of the :mod:`sqlite3` module are not |
| 979 | exposed to the user. |
| 980 | (Contributed by Aviv Palivoda in :issue:`30262`.) |
| 981 | |
Matthias Bussonnier | 1a3faf9 | 2019-05-20 13:44:11 -0700 | [diff] [blame] | 982 | * The ``bufsize`` keyword argument of :func:`fileinput.input` and |
| 983 | :func:`fileinput.FileInput` which was ignored and deprecated since Python 3.6 |
| 984 | has been removed. :issue:`36952` (Contributed by Matthias Bussonnier) |
| 985 | |
Matthias Bussonnier | 3880f26 | 2019-05-28 00:10:59 -0700 | [diff] [blame] | 986 | * The function :func:`sys.set_coroutine_wrapper` deprecated in Python 3.7 has |
| 987 | been removed; :func:`sys.get_coroutine_wrapper` now always return ``None``. |
| 988 | :issue:`36933` (Contributed by Matthias Bussonnier) |
| 989 | |
Ned Deily | 07a1892 | 2018-01-31 18:12:38 -0500 | [diff] [blame] | 990 | |
| 991 | Porting to Python 3.8 |
| 992 | ===================== |
| 993 | |
| 994 | This section lists previously described changes and other bugfixes |
| 995 | that may require changes to your code. |
| 996 | |
| 997 | |
Serhiy Storchaka | 07ca9af | 2018-02-04 10:53:48 +0200 | [diff] [blame] | 998 | Changes in Python behavior |
| 999 | -------------------------- |
| 1000 | |
| 1001 | * Yield expressions (both ``yield`` and ``yield from`` clauses) are now disallowed |
| 1002 | in comprehensions and generator expressions (aside from the iterable expression |
Serhiy Storchaka | 2b57c43 | 2018-12-19 08:09:46 +0200 | [diff] [blame] | 1003 | in the leftmost :keyword:`!for` clause). |
Serhiy Storchaka | 07ca9af | 2018-02-04 10:53:48 +0200 | [diff] [blame] | 1004 | (Contributed by Serhiy Storchaka in :issue:`10544`.) |
| 1005 | |
Serhiy Storchaka | 3bcbedc | 2019-01-18 07:47:48 +0200 | [diff] [blame] | 1006 | * The compiler now produces a :exc:`SyntaxWarning` when identity checks |
| 1007 | (``is`` and ``is not``) are used with certain types of literals |
| 1008 | (e.g. strings, ints). These can often work by accident in CPython, |
| 1009 | but are not guaranteed by the language spec. The warning advises users |
| 1010 | to use equality tests (``==`` and ``!=``) instead. |
| 1011 | (Contributed by Serhiy Storchaka in :issue:`34850`.) |
| 1012 | |
Serhiy Storchaka | 7a0630c | 2019-04-08 14:34:04 +0300 | [diff] [blame] | 1013 | * The CPython interpreter can swallow exceptions in some circumstances. |
| 1014 | In Python 3.8 this happens in less cases. In particular, exceptions |
| 1015 | raised when getting the attribute from the type dictionary are no longer |
| 1016 | ignored. (Contributed by Serhiy Storchaka in :issue:`35459`.) |
| 1017 | |
Serhiy Storchaka | 96aeaec | 2019-05-06 22:29:40 +0300 | [diff] [blame] | 1018 | * Removed ``__str__`` implementations from builtin types :class:`bool`, |
| 1019 | :class:`int`, :class:`float`, :class:`complex` and few classes from |
| 1020 | the standard library. They now inherit ``__str__()`` from :class:`object`. |
| 1021 | As result, defining the ``__repr__()`` method in the subclass of these |
| 1022 | classes will affect they string representation. |
| 1023 | (Contributed by Serhiy Storchaka in :issue:`36793`.) |
| 1024 | |
Michael Felt | 9d949f7 | 2019-04-12 16:15:32 +0200 | [diff] [blame] | 1025 | * On AIX, :attr:`sys.platform` doesn't contain the major version anymore. |
| 1026 | It is always ``'aix'``, instead of ``'aix3'`` .. ``'aix7'``. Since |
| 1027 | older Python versions include the version number, it is recommended to |
| 1028 | always use the ``sys.platform.startswith('aix')``. |
| 1029 | (Contributed by M. Felt in :issue:`36588`.) |
Serhiy Storchaka | 07ca9af | 2018-02-04 10:53:48 +0200 | [diff] [blame] | 1030 | |
Joannah Nanjekye | f781d20 | 2019-04-29 04:38:45 -0400 | [diff] [blame] | 1031 | * :c:func:`PyEval_AcquireLock` and :c:func:`PyEval_AcquireThread` now |
| 1032 | terminate the current thread if called while the interpreter is |
| 1033 | finalizing, making them consistent with :c:func:`PyEval_RestoreThread`, |
| 1034 | :c:func:`Py_END_ALLOW_THREADS`, and :c:func:`PyGILState_Ensure`. If this |
| 1035 | behaviour is not desired, guard the call by checking :c:func:`_Py_IsFinalizing` |
| 1036 | or :c:func:`sys.is_finalizing`. |
| 1037 | |
Serhiy Storchaka | 97f1ca1 | 2018-02-01 18:49:21 +0200 | [diff] [blame] | 1038 | Changes in the Python API |
| 1039 | ------------------------- |
| 1040 | |
Victor Stinner | d7befad | 2019-04-25 14:30:16 +0200 | [diff] [blame] | 1041 | * :class:`subprocess.Popen` can now use :func:`os.posix_spawn` in some cases |
| 1042 | for better performance. On Windows Subsystem for Linux and QEMU User |
| 1043 | Emulation, Popen constructor using :func:`os.posix_spawn` no longer raise an |
| 1044 | exception on errors like missing program, but the child process fails with a |
| 1045 | non-zero :attr:`~Popen.returncode`. |
| 1046 | |
Victor Stinner | 74125a6 | 2019-04-15 18:23:20 +0200 | [diff] [blame] | 1047 | * The :meth:`imap.IMAP4.logout` method no longer ignores silently arbitrary |
| 1048 | exceptions. |
| 1049 | |
Victor Stinner | 73104fa | 2018-11-29 09:58:20 +0100 | [diff] [blame] | 1050 | * The function :func:`platform.popen` has been removed, it was deprecated since |
| 1051 | Python 3.3: use :func:`os.popen` instead. |
| 1052 | |
Raymond Hettinger | fc06a19 | 2019-03-12 00:43:27 -0700 | [diff] [blame] | 1053 | * The :func:`statistics.mode` function no longer raises an exception |
| 1054 | when given multimodal data. Instead, it returns the first mode |
| 1055 | encountered in the input data. (Contributed by Raymond Hettinger |
| 1056 | in :issue:`35892`.) |
| 1057 | |
Serhiy Storchaka | 97f1ca1 | 2018-02-01 18:49:21 +0200 | [diff] [blame] | 1058 | * The :meth:`~tkinter.ttk.Treeview.selection` method of the |
| 1059 | :class:`tkinter.ttk.Treeview` class no longer takes arguments. Using it with |
| 1060 | arguments for changing the selection was deprecated in Python 3.6. Use |
| 1061 | specialized methods like :meth:`~tkinter.ttk.Treeview.selection_set` for |
| 1062 | changing the selection. (Contributed by Serhiy Storchaka in :issue:`31508`.) |
Serhiy Storchaka | 6c85efa5 | 2018-02-05 22:47:31 +0200 | [diff] [blame] | 1063 | |
Diego Rojas | 06e1e68 | 2019-03-16 18:44:56 -0500 | [diff] [blame] | 1064 | * The :meth:`writexml`, :meth:`toxml` and :meth:`toprettyxml` methods of the |
| 1065 | :mod:`xml.dom.minidom` module, and :mod:`xml.etree` now preserve the attribute |
| 1066 | order specified by the user. |
| 1067 | (Contributed by Diego Rojas and Raymond Hettinger in :issue:`34160`.) |
| 1068 | |
Serhiy Storchaka | 6c85efa5 | 2018-02-05 22:47:31 +0200 | [diff] [blame] | 1069 | * A :mod:`dbm.dumb` database opened with flags ``'r'`` is now read-only. |
| 1070 | :func:`dbm.dumb.open` with flags ``'r'`` and ``'w'`` no longer creates |
| 1071 | a database if it does not exist. |
| 1072 | (Contributed by Serhiy Storchaka in :issue:`32749`.) |
Serhiy Storchaka | 520b7ae | 2018-02-22 23:33:30 +0200 | [diff] [blame] | 1073 | |
Serhiy Storchaka | 02ec92f | 2018-07-24 12:03:34 +0300 | [diff] [blame] | 1074 | * The ``doctype()`` method defined in a subclass of |
| 1075 | :class:`~xml.etree.ElementTree.XMLParser` will no longer be called and will |
| 1076 | cause emitting a :exc:`RuntimeWarning` instead of a :exc:`DeprecationWarning`. |
| 1077 | Define the :meth:`doctype() <xml.etree.ElementTree.TreeBuilder.doctype>` |
| 1078 | method on a target for handling an XML doctype declaration. |
| 1079 | (Contributed by Serhiy Storchaka in :issue:`29209`.) |
| 1080 | |
Serhiy Storchaka | f5e7b19 | 2018-05-20 08:48:12 +0300 | [diff] [blame] | 1081 | * A :exc:`RuntimeError` is now raised when the custom metaclass doesn't |
| 1082 | provide the ``__classcell__`` entry in the namespace passed to |
| 1083 | ``type.__new__``. A :exc:`DeprecationWarning` was emitted in Python |
| 1084 | 3.6--3.7. (Contributed by Serhiy Storchaka in :issue:`23722`.) |
| 1085 | |
Scott Sanderson | cebe80b | 2018-06-07 05:46:42 -0400 | [diff] [blame] | 1086 | * The :class:`cProfile.Profile` class can now be used as a context |
| 1087 | manager. (Contributed by Scott Sanderson in :issue:`29235`.) |
Serhiy Storchaka | 520b7ae | 2018-02-22 23:33:30 +0200 | [diff] [blame] | 1088 | |
Giampaolo Rodola | c7f02a9 | 2018-06-19 08:27:29 -0700 | [diff] [blame] | 1089 | * :func:`shutil.copyfile`, :func:`shutil.copy`, :func:`shutil.copy2`, |
| 1090 | :func:`shutil.copytree` and :func:`shutil.move` use platform-specific |
| 1091 | "fast-copy" syscalls (see |
| 1092 | :ref:`shutil-platform-dependent-efficient-copy-operations` section). |
| 1093 | |
| 1094 | * :func:`shutil.copyfile` default buffer size on Windows was changed from |
| 1095 | 16 KiB to 1 MiB. |
| 1096 | |
INADA Naoki | d5c875b | 2018-07-11 17:42:49 +0900 | [diff] [blame] | 1097 | * ``PyGC_Head`` struct is changed completely. All code touched the |
| 1098 | struct member should be rewritten. (See :issue:`33597`) |
| 1099 | |
Eric Snow | be3b295 | 2019-02-23 11:35:52 -0700 | [diff] [blame] | 1100 | * The ``PyInterpreterState`` struct has been moved into the "internal" |
| 1101 | header files (specifically Include/internal/pycore_pystate.h). An |
| 1102 | opaque ``PyInterpreterState`` is still available as part of the public |
| 1103 | API (and stable ABI). The docs indicate that none of the struct's |
| 1104 | fields are public, so we hope no one has been using them. However, |
| 1105 | if you do rely on one or more of those private fields and have no |
| 1106 | alternative then please open a BPO issue. We'll work on helping |
| 1107 | you adjust (possibly including adding accessor functions to the |
| 1108 | public API). (See :issue:`35886`.) |
| 1109 | |
Alex Grönholm | cca4eec | 2018-08-09 00:06:47 +0300 | [diff] [blame] | 1110 | * Asyncio tasks can now be named, either by passing the ``name`` keyword |
| 1111 | argument to :func:`asyncio.create_task` or |
Yury Selivanov | 7c7605f | 2018-09-11 09:54:40 -0700 | [diff] [blame] | 1112 | the :meth:`~asyncio.loop.create_task` event loop method, or by |
Alex Grönholm | cca4eec | 2018-08-09 00:06:47 +0300 | [diff] [blame] | 1113 | calling the :meth:`~asyncio.Task.set_name` method on the task object. The |
| 1114 | task name is visible in the ``repr()`` output of :class:`asyncio.Task` and |
| 1115 | can also be retrieved using the :meth:`~asyncio.Task.get_name` method. |
| 1116 | |
Berker Peksag | e7d4b2f | 2018-08-22 21:21:05 +0300 | [diff] [blame] | 1117 | * The :meth:`mmap.flush() <mmap.mmap.flush>` method now returns ``None`` on |
| 1118 | success and raises an exception on error under all platforms. Previously, |
| 1119 | its behavior was platform-depended: a nonzero value was returned on success; |
| 1120 | zero was returned on error under Windows. A zero value was returned on |
| 1121 | success; an exception was raised on error under Unix. |
| 1122 | (Contributed by Berker Peksag in :issue:`2122`.) |
| 1123 | |
Pablo Galindo | fa221d8 | 2018-09-08 00:16:17 +0100 | [diff] [blame] | 1124 | * The function :func:`math.factorial` no longer accepts arguments that are not |
| 1125 | int-like. (Contributed by Pablo Galindo in :issue:`33083`.) |
| 1126 | |
Andrés Delfino | ca68261 | 2018-11-07 14:29:14 -0300 | [diff] [blame] | 1127 | * :mod:`xml.dom.minidom` and :mod:`xml.sax` modules no longer process |
Christian Heimes | 17b1d5d | 2018-09-23 09:50:25 +0200 | [diff] [blame] | 1128 | external entities by default. |
| 1129 | (Contributed by Christian Heimes in :issue:`17239`.) |
INADA Naoki | d5c875b | 2018-07-11 17:42:49 +0900 | [diff] [blame] | 1130 | |
Xiang Zhang | 4fb0b8b | 2018-12-12 20:46:55 +0800 | [diff] [blame] | 1131 | * Deleting a key from a read-only :mod:`dbm` database (:mod:`dbm.dumb`, |
| 1132 | :mod:`dbm.gnu` or :mod:`dbm.ndbm`) raises :attr:`error` (:exc:`dbm.dumb.error`, |
| 1133 | :exc:`dbm.gnu.error` or :exc:`dbm.ndbm.error`) instead of :exc:`KeyError`. |
| 1134 | (Contributed by Xiang Zhang in :issue:`33106`.) |
| 1135 | |
Steve Dower | 8ef864d | 2019-03-12 15:15:26 -0700 | [diff] [blame] | 1136 | * :func:`~os.path.expanduser` on Windows now prefers the :envvar:`USERPROFILE` |
| 1137 | environment variable and does not use :envvar:`HOME`, which is not normally |
| 1138 | set for regular user accounts. |
| 1139 | |
Steve Dower | 2438cdf | 2019-03-29 16:37:16 -0700 | [diff] [blame] | 1140 | .. _bpo-36085-whatsnew: |
| 1141 | |
| 1142 | * DLL dependencies for extension modules and DLLs loaded with :mod:`ctypes` on |
| 1143 | Windows are now resolved more securely. Only the system paths, the directory |
| 1144 | containing the DLL or PYD file, and directories added with |
| 1145 | :func:`~os.add_dll_directory` are searched for load-time dependencies. |
| 1146 | Specifically, :envvar:`PATH` and the current working directory are no longer |
| 1147 | used, and modifications to these will no longer have any effect on normal DLL |
| 1148 | resolution. If your application relies on these mechanisms, you should check |
| 1149 | for :func:`~os.add_dll_directory` and if it exists, use it to add your DLLs |
Steve Dower | 79da388 | 2019-03-30 20:58:17 -0700 | [diff] [blame] | 1150 | directory while loading your library. Note that Windows 7 users will need to |
| 1151 | ensure that Windows Update KB2533625 has been installed (this is also verified |
| 1152 | by the installer). |
Steve Dower | 2438cdf | 2019-03-29 16:37:16 -0700 | [diff] [blame] | 1153 | (See :issue:`36085`.) |
| 1154 | |
Pablo Galindo | f2cf1e3 | 2019-04-13 17:05:14 +0100 | [diff] [blame] | 1155 | * The header files and functions related to pgen have been removed after its |
| 1156 | replacement by a pure Python implementation. (Contributed by Pablo Galindo |
| 1157 | in :issue:`36623`.) |
| 1158 | |
Pablo Galindo | 5d23e28 | 2019-05-12 22:45:52 +0100 | [diff] [blame] | 1159 | * :class:`types.CodeType` has a new parameter in the second position of the |
| 1160 | constructor (*posonlyargcount*) to support positional-only arguments defined |
Victor Stinner | a9f05d6 | 2019-05-24 23:57:23 +0200 | [diff] [blame] | 1161 | in :pep:`570`. A new ``replace()`` method of :class:`types.CodeType` can be |
| 1162 | used to make the code future-proof. |
Pablo Galindo | 5d23e28 | 2019-05-12 22:45:52 +0100 | [diff] [blame] | 1163 | |
Xiang Zhang | 4fb0b8b | 2018-12-12 20:46:55 +0800 | [diff] [blame] | 1164 | |
Inada Naoki | d3c72a2 | 2019-03-23 21:04:40 +0900 | [diff] [blame] | 1165 | Changes in the C API |
| 1166 | -------------------- |
| 1167 | |
Victor Stinner | d5d9e81 | 2019-05-13 12:35:37 +0200 | [diff] [blame] | 1168 | * The :c:func:`PyEval_ReInitThreads` function has been removed from the C API. |
| 1169 | It should not be called explicitly: use :c:func:`PyOS_AfterFork_Child` |
| 1170 | instead. |
| 1171 | (Contributed by Victor Stinner in :issue:`36728`.) |
| 1172 | |
E. M. Bray | c994c8f | 2019-05-24 17:33:47 +0200 | [diff] [blame] | 1173 | * On Unix, C extensions are no longer linked to libpython except on Android |
| 1174 | and Cygwin. When Python is embedded, ``libpython`` must not be loaded with |
xdegaye | 254b309 | 2019-04-29 09:27:40 +0200 | [diff] [blame] | 1175 | ``RTLD_LOCAL``, but ``RTLD_GLOBAL`` instead. Previously, using |
E. M. Bray | c994c8f | 2019-05-24 17:33:47 +0200 | [diff] [blame] | 1176 | ``RTLD_LOCAL``, it was already not possible to load C extensions which |
| 1177 | were not linked to ``libpython``, like C extensions of the standard |
| 1178 | library built by the ``*shared*`` section of ``Modules/Setup``. |
Victor Stinner | 8c3ecc6 | 2019-04-25 20:13:10 +0200 | [diff] [blame] | 1179 | |
Inada Naoki | d3c72a2 | 2019-03-23 21:04:40 +0900 | [diff] [blame] | 1180 | * Use of ``#`` variants of formats in parsing or building value (e.g. |
| 1181 | :c:func:`PyArg_ParseTuple`, :c:func:`Py_BuildValue`, :c:func:`PyObject_CallFunction`, |
| 1182 | etc.) without ``PY_SSIZE_T_CLEAN`` defined raises ``DeprecationWarning`` now. |
| 1183 | It will be removed in 3.10 or 4.0. Read :ref:`arg-parsing` for detail. |
| 1184 | (Contributed by Inada Naoki in :issue:`36381`.) |
| 1185 | |
Eddie Elizondo | 364f0b0 | 2019-03-27 07:52:18 -0400 | [diff] [blame] | 1186 | * Instances of heap-allocated types (such as those created with |
| 1187 | :c:func:`PyType_FromSpec`) hold a reference to their type object. |
| 1188 | Increasing the reference count of these type objects has been moved from |
| 1189 | :c:func:`PyType_GenericAlloc` to the more low-level functions, |
| 1190 | :c:func:`PyObject_Init` and :c:func:`PyObject_INIT`. |
| 1191 | This makes types created through :c:func:`PyType_FromSpec` behave like |
| 1192 | other classes in managed code. |
| 1193 | |
| 1194 | Statically allocated types are not affected. |
| 1195 | |
| 1196 | For the vast majority of cases, there should be no side effect. |
| 1197 | However, types that manually increase the reference count after allocating |
| 1198 | an instance (perhaps to work around the bug) may now become immortal. |
| 1199 | To avoid this, these classes need to call Py_DECREF on the type object |
| 1200 | during instance deallocation. |
| 1201 | |
| 1202 | To correctly port these types into 3.8, please apply the following |
| 1203 | changes: |
| 1204 | |
| 1205 | * Remove :c:macro:`Py_INCREF` on the type object after allocating an |
| 1206 | instance - if any. |
| 1207 | This may happen after calling :c:func:`PyObject_New`, |
| 1208 | :c:func:`PyObject_NewVar`, :c:func:`PyObject_GC_New`, |
| 1209 | :c:func:`PyObject_GC_NewVar`, or any other custom allocator that uses |
| 1210 | :c:func:`PyObject_Init` or :c:func:`PyObject_INIT`. |
| 1211 | |
| 1212 | Example:: |
| 1213 | |
| 1214 | static foo_struct * |
| 1215 | foo_new(PyObject *type) { |
| 1216 | foo_struct *foo = PyObject_GC_New(foo_struct, (PyTypeObject *) type); |
| 1217 | if (foo == NULL) |
| 1218 | return NULL; |
| 1219 | #if PY_VERSION_HEX < 0x03080000 |
| 1220 | // Workaround for Python issue 35810; no longer necessary in Python 3.8 |
| 1221 | PY_INCREF(type) |
| 1222 | #endif |
| 1223 | return foo; |
| 1224 | } |
| 1225 | |
| 1226 | * Ensure that all custom ``tp_dealloc`` functions of heap-allocated types |
| 1227 | decrease the type's reference count. |
| 1228 | |
| 1229 | Example:: |
| 1230 | |
| 1231 | static void |
| 1232 | foo_dealloc(foo_struct *instance) { |
| 1233 | PyObject *type = Py_TYPE(instance); |
| 1234 | PyObject_GC_Del(instance); |
| 1235 | #if PY_VERSION_HEX >= 0x03080000 |
| 1236 | // This was not needed before Python 3.8 (Python issue 35810) |
| 1237 | Py_DECREF(type); |
| 1238 | #endif |
| 1239 | } |
| 1240 | |
| 1241 | (Contributed by Eddie Elizondo in :issue:`35810`.) |
| 1242 | |
Zackery Spytz | 3c8724f | 2019-05-28 09:16:33 -0600 | [diff] [blame^] | 1243 | * The :c:macro:`Py_DEPRECATED()` macro has been implemented for MSVC. |
| 1244 | The macro now must be placed before the symbol name. |
| 1245 | |
| 1246 | Example:: |
| 1247 | |
| 1248 | Py_DEPRECATED(3.8) PyAPI_FUNC(int) Py_OldFunction(void); |
| 1249 | |
| 1250 | (Contributed by Zackery Spytz in :issue:`33407`.) |
| 1251 | |
Eddie Elizondo | 364f0b0 | 2019-03-27 07:52:18 -0400 | [diff] [blame] | 1252 | |
Serhiy Storchaka | 520b7ae | 2018-02-22 23:33:30 +0200 | [diff] [blame] | 1253 | CPython bytecode changes |
| 1254 | ------------------------ |
| 1255 | |
| 1256 | * The interpreter loop has been simplified by moving the logic of unrolling |
| 1257 | the stack of blocks into the compiler. The compiler emits now explicit |
Serhiy Storchaka | 3f819ca | 2018-10-31 02:26:06 +0200 | [diff] [blame] | 1258 | instructions for adjusting the stack of values and calling the |
| 1259 | cleaning-up code for :keyword:`break`, :keyword:`continue` and |
| 1260 | :keyword:`return`. |
Serhiy Storchaka | 520b7ae | 2018-02-22 23:33:30 +0200 | [diff] [blame] | 1261 | |
| 1262 | Removed opcodes :opcode:`BREAK_LOOP`, :opcode:`CONTINUE_LOOP`, |
| 1263 | :opcode:`SETUP_LOOP` and :opcode:`SETUP_EXCEPT`. Added new opcodes |
| 1264 | :opcode:`ROT_FOUR`, :opcode:`BEGIN_FINALLY`, :opcode:`CALL_FINALLY` and |
| 1265 | :opcode:`POP_FINALLY`. Changed the behavior of :opcode:`END_FINALLY` |
| 1266 | and :opcode:`WITH_CLEANUP_START`. |
| 1267 | |
| 1268 | (Contributed by Mark Shannon, Antoine Pitrou and Serhiy Storchaka in |
| 1269 | :issue:`17611`.) |
Serhiy Storchaka | 702f8f3 | 2018-03-23 14:34:35 +0200 | [diff] [blame] | 1270 | |
| 1271 | * Added new opcode :opcode:`END_ASYNC_FOR` for handling exceptions raised |
| 1272 | when awaiting a next item in an :keyword:`async for` loop. |
| 1273 | (Contributed by Serhiy Storchaka in :issue:`33041`.) |
Raymond Hettinger | f75d59e | 2019-02-02 22:54:56 -0800 | [diff] [blame] | 1274 | |
| 1275 | |
| 1276 | Demos and Tools |
| 1277 | --------------- |
| 1278 | |
| 1279 | * Added a benchmark script for timing various ways to access variables: |
| 1280 | ``Tools/scripts/var_access_benchmark.py``. |
| 1281 | (Contributed by Raymond Hettinger in :issue:`35884`.) |