[3.8] bpo-38234: Backport init path config changes from master (GH-16423)
* bpo-38234: Py_SetPath() uses the program full path (GH-16357)
Py_SetPath() now sets sys.executable to the program full path
(Py_GetProgramFullPath()), rather than to the program name
(Py_GetProgramName()).
Fix also memory leaks in pathconfig_set_from_config().
(cherry picked from commit 1ce152a42eaa917d7763bce93f1e1ca72530d7ca)
* bpo-38234: Add tests for Python init path config (GH-16358)
(cherry picked from commit bb6bf7d342b4503a6227fd209fac934905b6a1aa)
* bpo-38234: test_embed: test pyvenv.cfg and pybuilddir.txt (GH-16366)
Add test_init_pybuilddir() and test_init_pyvenv_cfg() to test_embed
to test pyvenv.cfg and pybuilddir.txt configuration files.
Fix sysconfig._generate_posix_vars(): pybuilddir.txt uses UTF-8
encoding, not ASCII.
(cherry picked from commit 52ad33abbfb6637d74932617c7013bae0ccf6e32)
* bpo-38234: Cleanup getpath.c (GH-16367)
* search_for_prefix() directly calls reduce() if found is greater
than 0.
* Add calculate_pybuilddir() subfunction.
* search_for_prefix(): add path string buffer for readability.
* Fix some error handling code paths: release resources on error.
* calculate_read_pyenv(): rename tmpbuffer to filename.
* test.pythoninfo now also logs windows.dll_path
(cherry picked from commit 221fd84703c545408bbb4a6e0b58459651331f5c)
* bpo-38234: Fix test_embed pathconfig tests (GH-16390)
bpo-38234: On macOS and FreeBSD, the temporary directory can be
symbolic link. For example, /tmp can be a symbolic link to /var/tmp.
Call realpath() to resolve all symbolic links.
(cherry picked from commit 00508a7407d7d300b487532e2271534b20e378a7)
* bpo-38234: Add test_init_setpath_config() to test_embed (GH-16402)
* Add test_embed.test_init_setpath_config(): test Py_SetPath()
with PyConfig.
* test_init_setpath() and test_init_setpythonhome() no longer call
Py_SetProgramName(), but use the default program name.
* _PyPathConfig: isolated, site_import and base_executable
fields are now only available on Windows.
* If executable is set explicitly in the configuration, ignore
calculated base_executable: _PyConfig_InitPathConfig() copies
executable to base_executable.
* Complete path config documentation.
(cherry picked from commit 8bf39b606ef7b02c0279a80789f3c4824b0da5e9)
* bpo-38234: Complete init config documentation (GH-16404)
(cherry picked from commit 88feaecd46a8f427e30ef7ad8cfcddfe392a2402)
* bpo-38234: Fix test_embed.test_init_setpath_config() on FreeBSD (GH-16406)
Explicitly preinitializes with a Python preconfiguration to avoid
Py_SetPath() implicit preinitialization with a compat
preconfiguration.
Fix also test_init_setpath() and test_init_setpythonhome() on macOS:
use self.test_exe as the executable (and base_executable), rather
than shutil.which('python3').
(cherry picked from commit 49d99f01e6e51acec5ca57a02e857f0796bc418b)
* bpo-38234: Py_Initialize() sets global path configuration (GH-16421)
* Py_InitializeFromConfig() now writes PyConfig path configuration to
the global path configuration (_Py_path_config).
* Add test_embed.test_get_pathconfig().
* Fix typo in _PyWideStringList_Join().
(cherry picked from commit 12f2f177fc483723406d7917194e7f655a20631b)
diff --git a/Doc/c-api/init_config.rst b/Doc/c-api/init_config.rst
index bc24fa0..0c3c725 100644
--- a/Doc/c-api/init_config.rst
+++ b/Doc/c-api/init_config.rst
@@ -241,6 +241,7 @@
locale to decide if it should be coerced.
.. c:member:: int coerce_c_locale_warn
+
If non-zero, emit a warning if the C locale is coerced.
.. c:member:: int dev_mode
@@ -300,7 +301,7 @@
(:c:func:`PyPreConfig_InitPythonConfig`), if Python is initialized with
command line arguments, the command line arguments must also be passed to
preinitialize Python, since they have an effect on the pre-configuration
-like encodings. For example, the :option:`-X` ``utf8`` command line option
+like encodings. For example, the :option:`-X utf8 <-X>` command line option
enables the UTF-8 Mode.
``PyMem_SetAllocator()`` can be called after :c:func:`Py_PreInitialize` and
@@ -464,7 +465,7 @@
.. c:member:: int dev_mode
- Development mode: see :option:`-X` ``dev``.
+ Development mode: see :option:`-X dev <-X>`.
.. c:member:: int dump_refs
@@ -482,7 +483,7 @@
.. c:member:: int faulthandler
- If non-zero, call :func:`faulthandler.enable`.
+ If non-zero, call :func:`faulthandler.enable` at startup.
.. c:member:: wchar_t* filesystem_encoding
@@ -504,6 +505,9 @@
Python home directory.
+ Initialized from :envvar:`PYTHONHOME` environment variable value by
+ default.
+
.. c:member:: int import_time
If non-zero, profile import time.
@@ -561,7 +565,7 @@
:data:`sys.path`. If :c:member:`~PyConfig.module_search_paths_set` is
equal to 0, the :c:member:`~PyConfig.module_search_paths` is overridden
- by the function computing the :ref:`Path Configuration
+ by the function calculating the :ref:`Path Configuration
<init-path-config>`.
.. c:member:: int optimization_level
@@ -586,9 +590,9 @@
.. c:member:: int pathconfig_warnings
- If equal to 0, suppress warnings when computing the path configuration
- (Unix only, Windows does not log any warning). Otherwise, warnings are
- written into ``stderr``.
+ If equal to 0, suppress warnings when calculating the :ref:`Path
+ Configuration <init-path-config>` (Unix only, Windows does not log any
+ warning). Otherwise, warnings are written into ``stderr``.
.. c:member:: wchar_t* prefix
@@ -596,39 +600,46 @@
.. c:member:: wchar_t* program_name
- Program name.
+ Program name. Used to initialize :c:member:`~PyConfig.executable`, and in
+ early error messages.
.. c:member:: wchar_t* pycache_prefix
- ``.pyc`` cache prefix.
+ :data:`sys.pycache_prefix`: ``.pyc`` cache prefix.
+
+ If NULL, :data:`sys.pycache_prefix` is set to ``None``.
.. c:member:: int quiet
Quiet mode. For example, don't display the copyright and version messages
- even in interactive mode.
+ in interactive mode.
.. c:member:: wchar_t* run_command
- ``python3 -c COMMAND`` argument.
+ ``python3 -c COMMAND`` argument. Used by :c:func:`Py_RunMain`.
.. c:member:: wchar_t* run_filename
- ``python3 FILENAME`` argument.
+ ``python3 FILENAME`` argument. Used by :c:func:`Py_RunMain`.
.. c:member:: wchar_t* run_module
- ``python3 -m MODULE`` argument.
+ ``python3 -m MODULE`` argument. Used by :c:func:`Py_RunMain`.
.. c:member:: int show_alloc_count
Show allocation counts at exit?
+ Set to 1 by :option:`-X showalloccount <-X>` command line option.
+
Need a special Python build with ``COUNT_ALLOCS`` macro defined.
.. c:member:: int show_ref_count
Show total reference count at exit?
+ Set to 1 by :option:`-X showrefcount <-X>` command line option.
+
Need a debug build of Python (``Py_REF_DEBUG`` macro must be defined).
.. c:member:: int site_import
@@ -647,7 +658,7 @@
.. c:member:: int tracemalloc
- If non-zero, call :func:`tracemalloc.start`.
+ If non-zero, call :func:`tracemalloc.start` at startup.
.. c:member:: int use_environment
@@ -669,6 +680,9 @@
If non-zero, write ``.pyc`` files.
+ :data:`sys.dont_write_bytecode` is initialized to the inverted value of
+ :c:member:`~PyConfig.write_bytecode`.
+
.. c:member:: PyWideStringList xoptions
:data:`sys._xoptions`.
@@ -694,8 +708,8 @@
The caller is responsible to handle exceptions (error or exit) using
:c:func:`PyStatus_Exception` and :c:func:`Py_ExitStatusException`.
-``PyImport_FrozenModules``, ``PyImport_AppendInittab()`` or
-``PyImport_ExtendInittab()`` is used: they must be set or called after Python
+If ``PyImport_FrozenModules``, ``PyImport_AppendInittab()`` or
+``PyImport_ExtendInittab()`` are used, they must be set or called after Python
preinitialization and before the Python initialization.
Example setting the program name::
@@ -760,7 +774,7 @@
/* Append our custom search path to sys.path */
status = PyWideStringList_Append(&config.module_search_paths,
- L"/path/to/more/modules");
+ L"/path/to/more/modules");
if (PyStatus_Exception(status)) {
goto done;
}
@@ -791,9 +805,9 @@
application.
This configuration ignores global configuration variables, environments
-variables and command line arguments (:c:member:`PyConfig.argv` is not parsed).
-The C standard streams (ex: ``stdout``) and the LC_CTYPE locale are left
-unchanged by default.
+variables, command line arguments (:c:member:`PyConfig.argv` is not parsed)
+and user site directory. The C standard streams (ex: ``stdout``) and the
+LC_CTYPE locale are left unchanged. Signal handlers are not installed.
Configuration files are still used with this configuration. Set the
:ref:`Path Configuration <init-path-config>` ("output fields") to ignore these
@@ -864,29 +878,38 @@
:c:type:`PyConfig` contains multiple fields for the path configuration:
-* Path configuration input fields:
+* Path configuration inputs:
* :c:member:`PyConfig.home`
* :c:member:`PyConfig.pathconfig_warnings`
* :c:member:`PyConfig.program_name`
* :c:member:`PyConfig.pythonpath_env`
+ * current working directory: to get absolute paths
+ * ``PATH`` environment variable to get the program full path
+ (from :c:member:`PyConfig.program_name`)
+ * ``__PYVENV_LAUNCHER__`` environment variable
+ * (Windows only) Application paths in the registry under
+ "Software\Python\PythonCore\X.Y\PythonPath" of HKEY_CURRENT_USER and
+ HKEY_LOCAL_MACHINE (where X.Y is the Python version).
* Path configuration output fields:
+ * :c:member:`PyConfig.base_exec_prefix`
* :c:member:`PyConfig.base_executable`
+ * :c:member:`PyConfig.base_prefix`
* :c:member:`PyConfig.exec_prefix`
* :c:member:`PyConfig.executable`
- * :c:member:`PyConfig.prefix`
* :c:member:`PyConfig.module_search_paths_set`,
:c:member:`PyConfig.module_search_paths`
+ * :c:member:`PyConfig.prefix`
-If at least one "output field" is not set, Python computes the path
+If at least one "output field" is not set, Python calculates the path
configuration to fill unset fields. If
:c:member:`~PyConfig.module_search_paths_set` is equal to 0,
:c:member:`~PyConfig.module_search_paths` is overridden and
:c:member:`~PyConfig.module_search_paths_set` is set to 1.
-It is possible to completely ignore the function computing the default
+It is possible to completely ignore the function calculating the default
path configuration by setting explicitly all path configuration output
fields listed above. A string is considered as set even if it is non-empty.
``module_search_paths`` is considered as set if
@@ -894,7 +917,7 @@
configuration input fields are ignored as well.
Set :c:member:`~PyConfig.pathconfig_warnings` to 0 to suppress warnings when
-computing the path configuration (Unix only, Windows does not log any warning).
+calculating the path configuration (Unix only, Windows does not log any warning).
If :c:member:`~PyConfig.base_prefix` or :c:member:`~PyConfig.base_exec_prefix`
fields are not set, they inherit their value from :c:member:`~PyConfig.prefix`
@@ -961,7 +984,7 @@
* Builtin exceptions;
* Builtin and frozen modules;
* The :mod:`sys` module is only partially initialized
- (ex: :data:`sys.path` doesn't exist yet);
+ (ex: :data:`sys.path` doesn't exist yet).
* "Main" initialization phase, Python is fully initialized:
@@ -987,9 +1010,9 @@
not configured: the :ref:`Path Configuration <init-path-config>` is only
applied during the "Main" phase. It may allow to customize Python in Python to
override or tune the :ref:`Path Configuration <init-path-config>`, maybe
-install a custom sys.meta_path importer or an import hook, etc.
+install a custom :data:`sys.meta_path` importer or an import hook, etc.
-It may become possible to compute the :ref:`Path Configuration
+It may become possible to calculatin the :ref:`Path Configuration
<init-path-config>` in Python, after the Core phase and before the Main phase,
which is one of the :pep:`432` motivation.