Issue #23731: Implement PEP 488.
The concept of .pyo files no longer exists. Now .pyc files have an
optional `opt-` tag which specifies if any extra optimizations beyond
the peepholer were applied.
diff --git a/Doc/c-api/import.rst b/Doc/c-api/import.rst
index 3641fc6..01e0deb 100644
--- a/Doc/c-api/import.rst
+++ b/Doc/c-api/import.rst
@@ -183,9 +183,9 @@
.. c:function:: long PyImport_GetMagicNumber()
- Return the magic number for Python bytecode files (a.k.a. :file:`.pyc` and
- :file:`.pyo` files). The magic number should be present in the first four bytes
- of the bytecode file, in little-endian byte order. Returns -1 on error.
+ Return the magic number for Python bytecode files (a.k.a. :file:`.pyc` file).
+ The magic number should be present in the first four bytes of the bytecode
+ file, in little-endian byte order. Returns -1 on error.
.. versionchanged:: 3.3
Return value of -1 upon failure.
diff --git a/Doc/distutils/apiref.rst b/Doc/distutils/apiref.rst
index 53d7527..087c671 100644
--- a/Doc/distutils/apiref.rst
+++ b/Doc/distutils/apiref.rst
@@ -1193,12 +1193,12 @@
.. function:: byte_compile(py_files[, optimize=0, force=0, prefix=None, base_dir=None, verbose=1, dry_run=0, direct=None])
- Byte-compile a collection of Python source files to either :file:`.pyc` or
- :file:`.pyo` files in a :file:`__pycache__` subdirectory (see :pep:`3147`).
+ Byte-compile a collection of Python source files to :file:`.pyc` files in a
+ :file:`__pycache__` subdirectory (see :pep:`3147` and :pep:`488`).
*py_files* is a list of files to compile; any files that don't end in
:file:`.py` are silently skipped. *optimize* must be one of the following:
- * ``0`` - don't optimize (generate :file:`.pyc`)
+ * ``0`` - don't optimize
* ``1`` - normal optimization (like ``python -O``)
* ``2`` - extra optimization (like ``python -OO``)
@@ -1222,10 +1222,13 @@
doing, leave it set to ``None``.
.. versionchanged:: 3.2.3
- Create ``.pyc`` or ``.pyo`` files with an :func:`import magic tag
+ Create ``.pyc`` files with an :func:`import magic tag
<imp.get_tag>` in their name, in a :file:`__pycache__` subdirectory
instead of files without tag in the current directory.
+ .. versionchanged: 3.5
+ Create ``.pyc`` files according to :pep:`488`.
+
.. function:: rfc822_escape(header)
diff --git a/Doc/distutils/introduction.rst b/Doc/distutils/introduction.rst
index 0ece646..8f46bd7 100644
--- a/Doc/distutils/introduction.rst
+++ b/Doc/distutils/introduction.rst
@@ -156,8 +156,8 @@
pure Python module
a module written in Python and contained in a single :file:`.py` file (and
- possibly associated :file:`.pyc` and/or :file:`.pyo` files). Sometimes referred
- to as a "pure module."
+ possibly associated :file:`.pyc` files). Sometimes referred to as a
+ "pure module."
extension module
a module written in the low-level language of the Python implementation: C/C++
@@ -210,5 +210,3 @@
the top-level directory of your source tree (or source distribution); the
directory where :file:`setup.py` exists. Generally :file:`setup.py` will be
run from this directory.
-
-
diff --git a/Doc/library/compileall.rst b/Doc/library/compileall.rst
index 430362e..57f4804 100644
--- a/Doc/library/compileall.rst
+++ b/Doc/library/compileall.rst
@@ -93,6 +93,10 @@
.. versionchanged:: 3.5
``-q`` option was changed to a multilevel value.
+.. versionchanged:: 3.5
+ ``-b`` will always produce a byte-code file ending in ``.pyc``, never
+ ``.pyo``.
+
There is no command-line option to control the optimization level used by the
:func:`compile` function, because the Python interpreter itself already
@@ -150,6 +154,10 @@
.. versionchanged:: 3.5
*quiet* parameter was changed to a multilevel value.
+ .. versionchanged:: 3.5
+ The *legacy* parameter only writes out ``.pyc`` files, not ``.pyo`` files
+ no matter what the value of *optimize* is.
+
.. function:: compile_file(fullname, ddir=None, force=False, rx=None, quiet=0, legacy=False, optimize=-1)
Compile the file with path *fullname*.
@@ -182,6 +190,10 @@
.. versionchanged:: 3.5
*quiet* parameter was changed to a multilevel value.
+ .. versionchanged:: 3.5
+ The *legacy* parameter only writes out ``.pyc`` files, not ``.pyo`` files
+ no matter what the value of *optimize* is.
+
.. function:: compile_path(skip_curdir=True, maxlevels=0, force=False, quiet=0, legacy=False, optimize=-1)
Byte-compile all the :file:`.py` files found along ``sys.path``. If
@@ -196,6 +208,10 @@
.. versionchanged:: 3.5
*quiet* parameter was changed to a multilevel value.
+ .. versionchanged:: 3.5
+ The *legacy* parameter only writes out ``.pyc`` files, not ``.pyo`` files
+ no matter what the value of *optimize* is.
+
To force a recompile of all the :file:`.py` files in the :file:`Lib/`
subdirectory and all its subdirectories::
diff --git a/Doc/library/imp.rst b/Doc/library/imp.rst
index c2dbdc5..5557346 100644
--- a/Doc/library/imp.rst
+++ b/Doc/library/imp.rst
@@ -203,11 +203,9 @@
value would be ``/foo/bar/__pycache__/baz.cpython-32.pyc`` for Python 3.2.
The ``cpython-32`` string comes from the current magic tag (see
:func:`get_tag`; if :attr:`sys.implementation.cache_tag` is not defined then
- :exc:`NotImplementedError` will be raised). The returned path will end in
- ``.pyc`` when ``__debug__`` is ``True`` or ``.pyo`` for an optimized Python
- (i.e. ``__debug__`` is ``False``). By passing in ``True`` or ``False`` for
- *debug_override* you can override the system's value for ``__debug__`` for
- extension selection.
+ :exc:`NotImplementedError` will be raised). By passing in ``True`` or
+ ``False`` for *debug_override* you can override the system's value for
+ ``__debug__``, leading to optimized bytecode.
*path* need not exist.
@@ -218,6 +216,9 @@
.. deprecated:: 3.4
Use :func:`importlib.util.cache_from_source` instead.
+ .. versionchanged:: 3.5
+ The *debug_override* parameter no longer creates a ``.pyo`` file.
+
.. function:: source_from_cache(path)
diff --git a/Doc/library/importlib.rst b/Doc/library/importlib.rst
index e61ac35..bd9c5eb 100644
--- a/Doc/library/importlib.rst
+++ b/Doc/library/importlib.rst
@@ -711,6 +711,9 @@
.. versionadded:: 3.3
+ .. deprecated:: 3.5
+ Use :attr:`BYTECODE_SUFFIXES` instead.
+
.. attribute:: OPTIMIZED_BYTECODE_SUFFIXES
A list of strings representing the file suffixes for optimized bytecode
@@ -718,14 +721,19 @@
.. versionadded:: 3.3
+ .. deprecated:: 3.5
+ Use :attr:`BYTECODE_SUFFIXES` instead.
+
.. attribute:: BYTECODE_SUFFIXES
A list of strings representing the recognized file suffixes for bytecode
- modules. Set to either :attr:`DEBUG_BYTECODE_SUFFIXES` or
- :attr:`OPTIMIZED_BYTECODE_SUFFIXES` based on whether ``__debug__`` is true.
+ modules (including the leading dot).
.. versionadded:: 3.3
+ .. versionchanged:: 3.5
+ The value is no longer dependent on ``__debug__``.
+
.. attribute:: EXTENSION_SUFFIXES
A list of strings representing the recognized file suffixes for
@@ -1074,23 +1082,37 @@
.. versionadded:: 3.4
-.. function:: cache_from_source(path, debug_override=None)
+.. function:: cache_from_source(path, debug_override=None, *, optimization=None)
- Return the :pep:`3147` path to the byte-compiled file associated with the
- source *path*. For example, if *path* is ``/foo/bar/baz.py`` the return
+ Return the :pep:`3147`/:pep:`488` path to the byte-compiled file associated
+ with the source *path*. For example, if *path* is ``/foo/bar/baz.py`` the return
value would be ``/foo/bar/__pycache__/baz.cpython-32.pyc`` for Python 3.2.
The ``cpython-32`` string comes from the current magic tag (see
:func:`get_tag`; if :attr:`sys.implementation.cache_tag` is not defined then
- :exc:`NotImplementedError` will be raised). The returned path will end in
- ``.pyc`` when ``__debug__`` is ``True`` or ``.pyo`` for an optimized Python
- (i.e. ``__debug__`` is ``False``). By passing in ``True`` or ``False`` for
- *debug_override* you can override the system's value for ``__debug__`` for
- extension selection.
+ :exc:`NotImplementedError` will be raised).
- *path* need not exist.
+ The *optimization* parameter is used to specify the optimization level of the
+ bytecode file. An empty string represents no optimization, so
+ ``/foo/bar/baz.py`` with an *optimization* of ``''`` will result in a
+ bytecode path of ``/foo/bar/__pycache__/baz.cpython-32.pyc``. ``None`` causes
+ the interpter's optimization level to be used. Any other value's string
+ representation being used, so ``/foo/bar/baz.py`` with an *optimization* of
+ ``2`` will lead to the bytecode path of
+ ``/foo/bar/__pycache__/baz.cpython-32.opt-2.pyc``. The string representation
+ of *optimization* can only be alphanumeric, else :exc:`ValueError` is raised.
+
+ The *debug_override* parameter is deprecated and can be used to override
+ the system's value for ``__debug__``. A ``True`` value is the equivalent of
+ setting *optimization* to the empty string. A ``False`` value is the same as
+ setting *optimization* to ``1``. If both *debug_override* an *optimization*
+ are not ``None`` then :exc:`TypeError` is raised.
.. versionadded:: 3.4
+ .. versionchanged ::3.5
+ The *optimization* parameter was added and the *debug_override* parameter
+ was deprecated.
+
.. function:: source_from_cache(path)
@@ -1098,7 +1120,7 @@
file path. For example, if *path* is
``/foo/bar/__pycache__/baz.cpython-32.pyc`` the returned path would be
``/foo/bar/baz.py``. *path* need not exist, however if it does not conform
- to :pep:`3147` format, a ``ValueError`` is raised. If
+ to :pep:`3147` or :pep`488` format, a ``ValueError`` is raised. If
:attr:`sys.implementation.cache_tag` is not defined,
:exc:`NotImplementedError` is raised.
diff --git a/Doc/library/py_compile.rst b/Doc/library/py_compile.rst
index bae8450..3647571 100644
--- a/Doc/library/py_compile.rst
+++ b/Doc/library/py_compile.rst
@@ -29,9 +29,9 @@
.. function:: compile(file, cfile=None, dfile=None, doraise=False, optimize=-1)
Compile a source file to byte-code and write out the byte-code cache file.
- The source code is loaded from the file name *file*. The byte-code is
- written to *cfile*, which defaults to the :PEP:`3147` path, ending in
- ``.pyc`` (``.pyo`` if optimization is enabled in the current interpreter).
+ The source code is loaded from the file name *file*. The byte-code is
+ written to *cfile*, which defaults to the :pep:`3147`/:pep`488` path, ending
+ in ``.pyc``.
For example, if *file* is ``/foo/bar/baz.py`` *cfile* will default to
``/foo/bar/__pycache__/baz.cpython-32.pyc`` for Python 3.2. If *dfile* is
specified, it is used as the name of the source file in error messages when
@@ -68,7 +68,7 @@
.. function:: main(args=None)
Compile several source files. The files named in *args* (or on the command
- line, if *args* is ``None``) are compiled and the resulting bytecode is
+ line, if *args* is ``None``) are compiled and the resulting byte-code is
cached in the normal manner. This function does not search a directory
structure to locate source files; it only compiles files named explicitly.
If ``'-'`` is the only parameter in args, the list of files is taken from
@@ -86,4 +86,3 @@
Module :mod:`compileall`
Utilities to compile all Python source files in a directory tree.
-
diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst
index fa125a0..3080b69 100644
--- a/Doc/library/sys.rst
+++ b/Doc/library/sys.rst
@@ -167,7 +167,7 @@
.. data:: dont_write_bytecode
- If this is true, Python won't try to write ``.pyc`` or ``.pyo`` files on the
+ If this is true, Python won't try to write ``.pyc`` files on the
import of source modules. This value is initially set to ``True`` or
``False`` depending on the :option:`-B` command line option and the
:envvar:`PYTHONDONTWRITEBYTECODE` environment variable, but you can set it
@@ -1231,4 +1231,3 @@
.. rubric:: Citations
.. [C99] ISO/IEC 9899:1999. "Programming languages -- C." A public draft of this standard is available at http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1256.pdf\ .
-
diff --git a/Doc/library/tracemalloc.rst b/Doc/library/tracemalloc.rst
index a04a432..13c81a7 100644
--- a/Doc/library/tracemalloc.rst
+++ b/Doc/library/tracemalloc.rst
@@ -363,7 +363,7 @@
Filter on traces of memory blocks.
See the :func:`fnmatch.fnmatch` function for the syntax of
- *filename_pattern*. The ``'.pyc'`` and ``'.pyo'`` file extensions are
+ *filename_pattern*. The ``'.pyc'`` file extension is
replaced with ``'.py'``.
Examples:
@@ -374,6 +374,10 @@
:mod:`tracemalloc` module
* ``Filter(False, "<unknown>")`` excludes empty tracebacks
+
+ .. versionchanged:: 3.5
+ The ``'.pyo'`` file extension is no longer replaced with ``'.py'``.
+
.. attribute:: inclusive
If *inclusive* is ``True`` (include), only trace memory blocks allocated
@@ -631,4 +635,3 @@
obj = Object()
File "test.py", line 12
tb = tracemalloc.get_object_traceback(f())
-
diff --git a/Doc/library/zipfile.rst b/Doc/library/zipfile.rst
index a15c461..6192cd0 100644
--- a/Doc/library/zipfile.rst
+++ b/Doc/library/zipfile.rst
@@ -405,8 +405,7 @@
archive.
If the *optimize* parameter to :class:`PyZipFile` was not given or ``-1``,
- the corresponding file is a :file:`\*.pyo` file if available, else a
- :file:`\*.pyc` file, compiling if necessary.
+ the corresponding file is a :file:`\*.pyc` file, compiling if necessary.
If the *optimize* parameter to :class:`PyZipFile` was ``0``, ``1`` or
``2``, only files with that optimization level (see :func:`compile`) are
@@ -569,4 +568,3 @@
.. attribute:: ZipInfo.file_size
Size of the uncompressed file.
-
diff --git a/Doc/library/zipimport.rst b/Doc/library/zipimport.rst
index 2cf508b..8a5d5d1 100644
--- a/Doc/library/zipimport.rst
+++ b/Doc/library/zipimport.rst
@@ -20,10 +20,10 @@
import from the :file:`lib/` subdirectory within the archive.
Any files may be present in the ZIP archive, but only files :file:`.py` and
-:file:`.py[co]` are available for import. ZIP import of dynamic modules
+:file:`.pyc` are available for import. ZIP import of dynamic modules
(:file:`.pyd`, :file:`.so`) is disallowed. Note that if an archive only contains
:file:`.py` files, Python will not attempt to modify the archive by adding the
-corresponding :file:`.pyc` or :file:`.pyo` file, meaning that if a ZIP archive
+corresponding :file:`.pyc` file, meaning that if a ZIP archive
doesn't contain :file:`.pyc` files, importing may be rather slow.
ZIP archives with an archive comment are currently not supported.
@@ -161,4 +161,3 @@
>>> import jwzthreading
>>> jwzthreading.__file__
'example.zip/jwzthreading.py'
-
diff --git a/Doc/reference/import.rst b/Doc/reference/import.rst
index 2d60e62..dec5f8f 100644
--- a/Doc/reference/import.rst
+++ b/Doc/reference/import.rst
@@ -646,7 +646,7 @@
The default set of path entry finders implement all the semantics for finding
modules on the file system, handling special file types such as Python source
-code (``.py`` files), Python byte code (``.pyc`` and ``.pyo`` files) and
+code (``.py`` files), Python byte code (``.pyc`` files) and
shared libraries (e.g. ``.so`` files). When supported by the :mod:`zipimport`
module in the standard library, the default path entry finders also handle
loading all of these file types (other than shared libraries) from zipfiles.
diff --git a/Doc/tutorial/modules.rst b/Doc/tutorial/modules.rst
index fd361ae..9ae64b0 100644
--- a/Doc/tutorial/modules.rst
+++ b/Doc/tutorial/modules.rst
@@ -216,15 +216,15 @@
statements, the ``-OO`` switch removes both assert statements and __doc__
strings. Since some programs may rely on having these available, you should
only use this option if you know what you're doing. "Optimized" modules have
- a .pyo rather than a .pyc suffix and are usually smaller. Future releases may
+ an ``opt-`` tag and are usually smaller. Future releases may
change the effects of optimization.
-* A program doesn't run any faster when it is read from a ``.pyc`` or ``.pyo``
+* A program doesn't run any faster when it is read from a ``.pyc``
file than when it is read from a ``.py`` file; the only thing that's faster
- about ``.pyc`` or ``.pyo`` files is the speed with which they are loaded.
+ about ``.pyc`` files is the speed with which they are loaded.
-* The module :mod:`compileall` can create .pyc files (or .pyo files when
- :option:`-O` is used) for all modules in a directory.
+* The module :mod:`compileall` can create .pyc files for all modules in a
+ directory.
* There is more detail on this process, including a flow chart of the
decisions, in PEP 3147.
@@ -548,4 +548,3 @@
.. [#] In fact function definitions are also 'statements' that are 'executed'; the
execution of a module-level function definition enters the function name in
the module's global symbol table.
-
diff --git a/Doc/using/cmdline.rst b/Doc/using/cmdline.rst
index f52afa0..81cd38a 100644
--- a/Doc/using/cmdline.rst
+++ b/Doc/using/cmdline.rst
@@ -199,7 +199,7 @@
.. cmdoption:: -B
- If given, Python won't try to write ``.pyc`` or ``.pyo`` files on the
+ If given, Python won't try to write ``.pyc``` files on the
import of source modules. See also :envvar:`PYTHONDONTWRITEBYTECODE`.
diff --git a/Doc/using/windows.rst b/Doc/using/windows.rst
index f194802..7113dfa 100644
--- a/Doc/using/windows.rst
+++ b/Doc/using/windows.rst
@@ -111,7 +111,7 @@
| | launcher is also installed. | |
+---------------------------+--------------------------------------+--------------------------+
| CompileAll | Compile all ``.py`` files to | 0 |
-| | ``.pyc`` and ``.pyo``. | |
+| | ``.pyc``. | |
+---------------------------+--------------------------------------+--------------------------+
| PrependPath | Add install and Scripts directories | 0 |
| | tho :envvar:`PATH` and ``.PY`` to | |
@@ -451,7 +451,7 @@
^^^^^^^^^^^^^^^^^^^^^^
The launcher should have been associated with Python files (i.e. ``.py``,
-``.pyw``, ``.pyc``, ``.pyo`` files) when it was installed. This means that
+``.pyw``, ``.pyc`` files) when it was installed. This means that
when you double-click on one of these files from Windows explorer the launcher
will be used, and therefore you can use the same facilities described above to
have the script specify the version which should be used.
@@ -796,5 +796,3 @@
:pep:`397` - Python launcher for Windows
The proposal for the launcher to be included in the Python distribution.
-
-
diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst
index 28914ef..5c21054 100644
--- a/Doc/whatsnew/3.5.rst
+++ b/Doc/whatsnew/3.5.rst
@@ -88,6 +88,8 @@
``surrogateescape`` error handler, instead of the ``strict`` error handler
(:issue:`19977`).
+* :pep:`488`, the elimination of ``.pyo`` files.
+
Significantly Improved Library Modules:
* None yet.
@@ -195,6 +197,24 @@
:pep:`486` -- Make the Python Launcher aware of virtual environments
+
+PEP 488: Elimination of PYO files
+---------------------------------
+
+:pep:`488` does away with the concept of ``.pyo`` files. This means that
+``.pyc`` files represent both unoptimized and optimized bytecode. To prevent
+the need to constantly regenerate bytecode files, ``.pyc`` files now have an
+optional ``opt-`` tag in their name when the bytecode is optimized. This has
+the side-effect of no more bytecode file name clashes when running under either
+``-O`` or ``-OO``, thus allowing unoptimized, ``-O``, and ``-OO`` bytecode files
+to all exist simultaneously. :func:`importlib.util.cache_from_source` has an
+updated API to help with this change.
+
+.. seealso::
+
+ :pep:`488` -- Elimination of PYO files
+
+
Other Language Changes
======================
@@ -535,7 +555,7 @@
``FindFirstFile``/``FindNextFile`` system calls. (Contributed by
Ben Hoyt with help from Victor Stinner in :issue:`23605`.)
-* Construction of ``bytes(int)`` (filled by zero bytes) is faster and use less
+* Construction of ``bytes(int)`` (filled by zero bytes) is faster and uses less
memory for large objects. ``calloc()`` is used instead of ``malloc()`` to
allocate memory for these objects.
@@ -630,6 +650,8 @@
3.4, and has now been removed.
(Contributed by Matt Chaput in :issue:`6623`.)
+* The concept of ``.pyo`` files has been removed.
+
* The JoinableQueue class in the provisional asyncio module was deprecated
in 3.4.4 and is now removed (:issue:`23464`).
@@ -759,6 +781,17 @@
*LegalChars* parameter of :func:`~http.cookies.Morsel.set` is deprecated and
is now ignored. (:issue:`2211`)
+* :pep:`488` has removed ``.pyo`` files from Python and introduced the optional
+ ``opt-`` tag in ``.pyc`` file names. The
+ :func:`importlib.util.cache_from_source` has gained an *optimization*
+ parameter to help control the ``opt-`` tag. Because of this, the
+ *debug_override* parameter of the function is now deprecated. `.pyo` files
+ are also no longer supported as a file argument to the Python interpreter and
+ thus serve no purpose when distributed on their own (i.e. sourcless code
+ distribution). Due to the fact that the magic number for bytecode has changed
+ in Python 3.5, all old `.pyo` files from previous versions of Python are
+ invalid regardless of this PEP.
+
Changes in the C API
--------------------