bpo-40275: Add import_helper submodule in test.support (GH-20794)
diff --git a/Doc/library/test.rst b/Doc/library/test.rst
index 11d7484..a18197a 100644
--- a/Doc/library/test.rst
+++ b/Doc/library/test.rst
@@ -398,25 +398,6 @@
The :mod:`test.support` module defines the following functions:
-.. function:: forget(module_name)
-
- Remove the module named *module_name* from ``sys.modules`` and delete any
- byte-compiled files of the module.
-
-
-.. function:: unload(name)
-
- Delete *name* from ``sys.modules``.
-
-
-.. function:: make_legacy_pyc(source)
-
- Move a :pep:`3147`/:pep:`488` pyc file to its legacy pyc location and return the file
- system path to the legacy pyc file. The *source* value is the file system
- path to the source file. It does not need to exist, however the PEP
- 3147/488 pyc file must exist.
-
-
.. function:: is_resource_enabled(resource)
Return ``True`` if *resource* is enabled and available. The list of
@@ -889,67 +870,6 @@
Open *url*. If open fails, raises :exc:`TestFailed`.
-.. function:: import_module(name, deprecated=False, *, required_on())
-
- This function imports and returns the named module. Unlike a normal
- import, this function raises :exc:`unittest.SkipTest` if the module
- cannot be imported.
-
- Module and package deprecation messages are suppressed during this import
- if *deprecated* is ``True``. If a module is required on a platform but
- optional for others, set *required_on* to an iterable of platform prefixes
- which will be compared against :data:`sys.platform`.
-
- .. versionadded:: 3.1
-
-
-.. function:: import_fresh_module(name, fresh=(), blocked=(), deprecated=False)
-
- This function imports and returns a fresh copy of the named Python module
- by removing the named module from ``sys.modules`` before doing the import.
- Note that unlike :func:`reload`, the original module is not affected by
- this operation.
-
- *fresh* is an iterable of additional module names that are also removed
- from the ``sys.modules`` cache before doing the import.
-
- *blocked* is an iterable of module names that are replaced with ``None``
- in the module cache during the import to ensure that attempts to import
- them raise :exc:`ImportError`.
-
- The named module and any modules named in the *fresh* and *blocked*
- parameters are saved before starting the import and then reinserted into
- ``sys.modules`` when the fresh import is complete.
-
- Module and package deprecation messages are suppressed during this import
- if *deprecated* is ``True``.
-
- This function will raise :exc:`ImportError` if the named module cannot be
- imported.
-
- Example use::
-
- # Get copies of the warnings module for testing without affecting the
- # version being used by the rest of the test suite. One copy uses the
- # C implementation, the other is forced to use the pure Python fallback
- # implementation
- py_warnings = import_fresh_module('warnings', blocked=['_warnings'])
- c_warnings = import_fresh_module('warnings', fresh=['_warnings'])
-
- .. versionadded:: 3.1
-
-
-.. function:: modules_setup()
-
- Return a copy of :data:`sys.modules`.
-
-
-.. function:: modules_cleanup(oldmodules)
-
- Remove modules except for *oldmodules* and ``encodings`` in order to
- preserve internal cache.
-
-
.. function:: reap_children()
Use this at the end of ``test_main`` whenever sub-processes are started.
@@ -1113,29 +1033,6 @@
On both platforms, the old value is restored by :meth:`__exit__`.
-.. class:: CleanImport(*module_names)
-
- A context manager to force import to return a new module reference. This
- is useful for testing module-level behaviors, such as the emission of a
- DeprecationWarning on import. Example usage::
-
- with CleanImport('foo'):
- importlib.import_module('foo') # New reference.
-
-
-.. class:: DirsOnSysPath(*paths)
-
- A context manager to temporarily add directories to sys.path.
-
- This makes a copy of :data:`sys.path`, appends any directories given
- as positional arguments, then reverts :data:`sys.path` to the copied
- settings when the context ends.
-
- Note that *all* :data:`sys.path` modifications in the body of the
- context manager, including replacement of the object,
- will be reverted at the end of the block.
-
-
.. class:: SaveSignals()
Class to save and restore signal handlers registered by the Python signal
@@ -1646,3 +1543,119 @@
Call :func:`os.unlink` on *filename*. On Windows platforms, this is
wrapped with a wait loop that checks for the existence fo the file.
+
+
+:mod:`test.support.import_helper` --- Utilities for import tests
+================================================================
+
+.. module:: test.support.import_helper
+ :synopsis: Support for import tests.
+
+The :mod:`test.support.import_helper` module provides support for import tests.
+
+.. versionadded:: 3.10
+
+
+.. function:: forget(module_name)
+
+ Remove the module named *module_name* from ``sys.modules`` and delete any
+ byte-compiled files of the module.
+
+
+.. function:: import_fresh_module(name, fresh=(), blocked=(), deprecated=False)
+
+ This function imports and returns a fresh copy of the named Python module
+ by removing the named module from ``sys.modules`` before doing the import.
+ Note that unlike :func:`reload`, the original module is not affected by
+ this operation.
+
+ *fresh* is an iterable of additional module names that are also removed
+ from the ``sys.modules`` cache before doing the import.
+
+ *blocked* is an iterable of module names that are replaced with ``None``
+ in the module cache during the import to ensure that attempts to import
+ them raise :exc:`ImportError`.
+
+ The named module and any modules named in the *fresh* and *blocked*
+ parameters are saved before starting the import and then reinserted into
+ ``sys.modules`` when the fresh import is complete.
+
+ Module and package deprecation messages are suppressed during this import
+ if *deprecated* is ``True``.
+
+ This function will raise :exc:`ImportError` if the named module cannot be
+ imported.
+
+ Example use::
+
+ # Get copies of the warnings module for testing without affecting the
+ # version being used by the rest of the test suite. One copy uses the
+ # C implementation, the other is forced to use the pure Python fallback
+ # implementation
+ py_warnings = import_fresh_module('warnings', blocked=['_warnings'])
+ c_warnings = import_fresh_module('warnings', fresh=['_warnings'])
+
+ .. versionadded:: 3.1
+
+
+.. function:: import_module(name, deprecated=False, *, required_on())
+
+ This function imports and returns the named module. Unlike a normal
+ import, this function raises :exc:`unittest.SkipTest` if the module
+ cannot be imported.
+
+ Module and package deprecation messages are suppressed during this import
+ if *deprecated* is ``True``. If a module is required on a platform but
+ optional for others, set *required_on* to an iterable of platform prefixes
+ which will be compared against :data:`sys.platform`.
+
+ .. versionadded:: 3.1
+
+
+.. function:: modules_setup()
+
+ Return a copy of :data:`sys.modules`.
+
+
+.. function:: modules_cleanup(oldmodules)
+
+ Remove modules except for *oldmodules* and ``encodings`` in order to
+ preserve internal cache.
+
+
+.. function:: unload(name)
+
+ Delete *name* from ``sys.modules``.
+
+
+.. function:: make_legacy_pyc(source)
+
+ Move a :pep:`3147`/:pep:`488` pyc file to its legacy pyc location and return the file
+ system path to the legacy pyc file. The *source* value is the file system
+ path to the source file. It does not need to exist, however the PEP
+ 3147/488 pyc file must exist.
+
+
+.. class:: CleanImport(*module_names)
+
+ A context manager to force import to return a new module reference. This
+ is useful for testing module-level behaviors, such as the emission of a
+ DeprecationWarning on import. Example usage::
+
+ with CleanImport('foo'):
+ importlib.import_module('foo') # New reference.
+
+
+.. class:: DirsOnSysPath(*paths)
+
+ A context manager to temporarily add directories to sys.path.
+
+ This makes a copy of :data:`sys.path`, appends any directories given
+ as positional arguments, then reverts :data:`sys.path` to the copied
+ settings when the context ends.
+
+ Note that *all* :data:`sys.path` modifications in the body of the
+ context manager, including replacement of the object,
+ will be reverted at the end of the block.
+
+
diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py
index 83b2173..3778eed 100644
--- a/Lib/test/support/__init__.py
+++ b/Lib/test/support/__init__.py
@@ -8,8 +8,6 @@
import fnmatch
import functools
import glob
-import importlib
-import importlib.util
import os
import re
import stat
@@ -21,6 +19,11 @@
import unittest
import warnings
+from .import_helper import (
+ CleanImport, DirsOnSysPath, _ignore_deprecated_imports,
+ _save_and_block_module, _save_and_remove_module,
+ forget, import_fresh_module, import_module, make_legacy_pyc,
+ modules_cleanup, modules_setup, unload)
from .os_helper import (
FS_NONASCII, SAVEDCWD, TESTFN, TESTFN_NONASCII,
TESTFN_UNENCODABLE, TESTFN_UNDECODABLE,
@@ -39,10 +42,6 @@
"PIPE_MAX_SIZE", "verbose", "max_memuse", "use_resources", "failfast",
# exceptions
"Error", "TestFailed", "TestDidNotRun", "ResourceDenied",
- # imports
- "import_module", "import_fresh_module", "CleanImport",
- # modules
- "unload", "forget",
# io
"record_original_stdout", "get_original_stdout", "captured_stdout",
"captured_stdin", "captured_stderr",
@@ -132,22 +131,6 @@
and unexpected skips.
"""
-@contextlib.contextmanager
-def _ignore_deprecated_imports(ignore=True):
- """Context manager to suppress package and module deprecation
- warnings when importing them.
-
- If ignore is False, this context manager has no effect.
- """
- if ignore:
- with warnings.catch_warnings():
- warnings.filterwarnings("ignore", ".+ (module|package)",
- DeprecationWarning)
- yield
- else:
- yield
-
-
def ignore_warnings(*, category):
"""Decorator to suppress deprecation warnings.
@@ -164,52 +147,6 @@
return decorator
-def import_module(name, deprecated=False, *, required_on=()):
- """Import and return the module to be tested, raising SkipTest if
- it is not available.
-
- If deprecated is True, any module or package deprecation messages
- will be suppressed. If a module is required on a platform but optional for
- others, set required_on to an iterable of platform prefixes which will be
- compared against sys.platform.
- """
- with _ignore_deprecated_imports(deprecated):
- try:
- return importlib.import_module(name)
- except ImportError as msg:
- if sys.platform.startswith(tuple(required_on)):
- raise
- raise unittest.SkipTest(str(msg))
-
-
-def _save_and_remove_module(name, orig_modules):
- """Helper function to save and remove a module from sys.modules
-
- Raise ImportError if the module can't be imported.
- """
- # try to import the module and raise an error if it can't be imported
- if name not in sys.modules:
- __import__(name)
- del sys.modules[name]
- for modname in list(sys.modules):
- if modname == name or modname.startswith(name + '.'):
- orig_modules[modname] = sys.modules[modname]
- del sys.modules[modname]
-
-def _save_and_block_module(name, orig_modules):
- """Helper function to save and block a module in sys.modules
-
- Return True if the module was in sys.modules, False otherwise.
- """
- saved = True
- try:
- orig_modules[name] = sys.modules[name]
- except KeyError:
- saved = False
- sys.modules[name] = None
- return saved
-
-
def anticipate_failure(condition):
"""Decorator to mark a test that is known to be broken in some cases
@@ -240,56 +177,6 @@
return standard_tests
-def import_fresh_module(name, fresh=(), blocked=(), deprecated=False):
- """Import and return a module, deliberately bypassing sys.modules.
-
- This function imports and returns a fresh copy of the named Python module
- by removing the named module from sys.modules before doing the import.
- Note that unlike reload, the original module is not affected by
- this operation.
-
- *fresh* is an iterable of additional module names that are also removed
- from the sys.modules cache before doing the import.
-
- *blocked* is an iterable of module names that are replaced with None
- in the module cache during the import to ensure that attempts to import
- them raise ImportError.
-
- The named module and any modules named in the *fresh* and *blocked*
- parameters are saved before starting the import and then reinserted into
- sys.modules when the fresh import is complete.
-
- Module and package deprecation messages are suppressed during this import
- if *deprecated* is True.
-
- This function will raise ImportError if the named module cannot be
- imported.
- """
- # NOTE: test_heapq, test_json and test_warnings include extra sanity checks
- # to make sure that this utility function is working as expected
- with _ignore_deprecated_imports(deprecated):
- # Keep track of modules saved for later restoration as well
- # as those which just need a blocking entry removed
- orig_modules = {}
- names_to_remove = []
- _save_and_remove_module(name, orig_modules)
- try:
- for fresh_name in fresh:
- _save_and_remove_module(fresh_name, orig_modules)
- for blocked_name in blocked:
- if not _save_and_block_module(blocked_name, orig_modules):
- names_to_remove.append(blocked_name)
- fresh_module = importlib.import_module(name)
- except ImportError:
- fresh_module = None
- finally:
- for orig_name, module in orig_modules.items():
- sys.modules[orig_name] = module
- for name_to_remove in names_to_remove:
- del sys.modules[name_to_remove]
- return fresh_module
-
-
def get_attribute(obj, name):
"""Get an attribute, raising SkipTest if AttributeError is raised."""
try:
@@ -318,12 +205,6 @@
def get_original_stdout():
return _original_stdout or sys.stdout
-def unload(name):
- try:
- del sys.modules[name]
- except KeyError:
- pass
-
def _force_run(path, func, *args):
try:
@@ -336,34 +217,6 @@
return func(*args)
-def make_legacy_pyc(source):
- """Move a PEP 3147/488 pyc file to its legacy pyc location.
-
- :param source: The file system path to the source file. The source file
- does not need to exist, however the PEP 3147/488 pyc file must exist.
- :return: The file system path to the legacy pyc file.
- """
- pyc_file = importlib.util.cache_from_source(source)
- up_one = os.path.dirname(os.path.abspath(source))
- legacy_pyc = os.path.join(up_one, source + 'c')
- os.rename(pyc_file, legacy_pyc)
- return legacy_pyc
-
-def forget(modname):
- """'Forget' a module was ever imported.
-
- This removes the module from sys.modules and deletes any PEP 3147/488 or
- legacy .pyc files.
- """
- unload(modname)
- for dirname in sys.path:
- source = os.path.join(dirname, modname + '.py')
- # It doesn't matter if they exist or not, unlink all possible
- # combinations of PEP 3147/488 and legacy pyc files.
- unlink(source + 'c')
- for opt in ('', 1, 2):
- unlink(importlib.util.cache_from_source(source, optimization=opt))
-
# Check whether a gui is actually available
def _is_gui_available():
if hasattr(_is_gui_available, 'result'):
@@ -870,63 +723,6 @@
yield
-class CleanImport(object):
- """Context manager to force import to return a new module reference.
-
- This is useful for testing module-level behaviours, such as
- the emission of a DeprecationWarning on import.
-
- Use like this:
-
- with CleanImport("foo"):
- importlib.import_module("foo") # new reference
- """
-
- def __init__(self, *module_names):
- self.original_modules = sys.modules.copy()
- for module_name in module_names:
- if module_name in sys.modules:
- module = sys.modules[module_name]
- # It is possible that module_name is just an alias for
- # another module (e.g. stub for modules renamed in 3.x).
- # In that case, we also need delete the real module to clear
- # the import cache.
- if module.__name__ != module_name:
- del sys.modules[module.__name__]
- del sys.modules[module_name]
-
- def __enter__(self):
- return self
-
- def __exit__(self, *ignore_exc):
- sys.modules.update(self.original_modules)
-
-
-class DirsOnSysPath(object):
- """Context manager to temporarily add directories to sys.path.
-
- This makes a copy of sys.path, appends any directories given
- as positional arguments, then reverts sys.path to the copied
- settings when the context ends.
-
- Note that *all* sys.path modifications in the body of the
- context manager, including replacement of the object,
- will be reverted at the end of the block.
- """
-
- def __init__(self, *paths):
- self.original_value = sys.path[:]
- self.original_object = sys.path
- sys.path.extend(paths)
-
- def __enter__(self):
- return self
-
- def __exit__(self, *ignore_exc):
- sys.path = self.original_object
- sys.path[:] = self.original_value
-
-
class TransientResource(object):
"""Raise ResourceDenied if an exception is raised while the context manager
@@ -1553,24 +1349,6 @@
for line in msg.splitlines():
print(f"Warning -- {line}", file=sys.__stderr__, flush=True)
-def modules_setup():
- return sys.modules.copy(),
-
-def modules_cleanup(oldmodules):
- # Encoders/decoders are registered permanently within the internal
- # codec cache. If we destroy the corresponding modules their
- # globals will be set to None which will trip up the cached functions.
- encodings = [(k, v) for k, v in sys.modules.items()
- if k.startswith('encodings.')]
- sys.modules.clear()
- sys.modules.update(encodings)
- # XXX: This kind of problem can affect more than just encodings. In particular
- # extension modules (such as _ssl) don't cope with reloading properly.
- # Really, test modules should be cleaning out the test specific modules they
- # know they added (ala test_runpy) rather than relying on this function (as
- # test_importhooks and test_pkg do currently).
- # Implicitly imported *real* modules should be left alone (see issue 10556).
- sys.modules.update(oldmodules)
# Flag used by saved_test_environment of test.libregrtest.save_env,
# to check if a test modified the environment. The flag should be set to False
diff --git a/Lib/test/support/import_helper.py b/Lib/test/support/import_helper.py
new file mode 100644
index 0000000..5d1e940
--- /dev/null
+++ b/Lib/test/support/import_helper.py
@@ -0,0 +1,238 @@
+import contextlib
+import importlib
+import importlib.util
+import os
+import sys
+import unittest
+import warnings
+
+from .os_helper import unlink
+
+
+@contextlib.contextmanager
+def _ignore_deprecated_imports(ignore=True):
+ """Context manager to suppress package and module deprecation
+ warnings when importing them.
+
+ If ignore is False, this context manager has no effect.
+ """
+ if ignore:
+ with warnings.catch_warnings():
+ warnings.filterwarnings("ignore", ".+ (module|package)",
+ DeprecationWarning)
+ yield
+ else:
+ yield
+
+
+def unload(name):
+ try:
+ del sys.modules[name]
+ except KeyError:
+ pass
+
+
+def forget(modname):
+ """'Forget' a module was ever imported.
+
+ This removes the module from sys.modules and deletes any PEP 3147/488 or
+ legacy .pyc files.
+ """
+ unload(modname)
+ for dirname in sys.path:
+ source = os.path.join(dirname, modname + '.py')
+ # It doesn't matter if they exist or not, unlink all possible
+ # combinations of PEP 3147/488 and legacy pyc files.
+ unlink(source + 'c')
+ for opt in ('', 1, 2):
+ unlink(importlib.util.cache_from_source(source, optimization=opt))
+
+
+def make_legacy_pyc(source):
+ """Move a PEP 3147/488 pyc file to its legacy pyc location.
+
+ :param source: The file system path to the source file. The source file
+ does not need to exist, however the PEP 3147/488 pyc file must exist.
+ :return: The file system path to the legacy pyc file.
+ """
+ pyc_file = importlib.util.cache_from_source(source)
+ up_one = os.path.dirname(os.path.abspath(source))
+ legacy_pyc = os.path.join(up_one, source + 'c')
+ os.rename(pyc_file, legacy_pyc)
+ return legacy_pyc
+
+
+def import_module(name, deprecated=False, *, required_on=()):
+ """Import and return the module to be tested, raising SkipTest if
+ it is not available.
+
+ If deprecated is True, any module or package deprecation messages
+ will be suppressed. If a module is required on a platform but optional for
+ others, set required_on to an iterable of platform prefixes which will be
+ compared against sys.platform.
+ """
+ with _ignore_deprecated_imports(deprecated):
+ try:
+ return importlib.import_module(name)
+ except ImportError as msg:
+ if sys.platform.startswith(tuple(required_on)):
+ raise
+ raise unittest.SkipTest(str(msg))
+
+
+def _save_and_remove_module(name, orig_modules):
+ """Helper function to save and remove a module from sys.modules
+
+ Raise ImportError if the module can't be imported.
+ """
+ # try to import the module and raise an error if it can't be imported
+ if name not in sys.modules:
+ __import__(name)
+ del sys.modules[name]
+ for modname in list(sys.modules):
+ if modname == name or modname.startswith(name + '.'):
+ orig_modules[modname] = sys.modules[modname]
+ del sys.modules[modname]
+
+
+def _save_and_block_module(name, orig_modules):
+ """Helper function to save and block a module in sys.modules
+
+ Return True if the module was in sys.modules, False otherwise.
+ """
+ saved = True
+ try:
+ orig_modules[name] = sys.modules[name]
+ except KeyError:
+ saved = False
+ sys.modules[name] = None
+ return saved
+
+
+def import_fresh_module(name, fresh=(), blocked=(), deprecated=False):
+ """Import and return a module, deliberately bypassing sys.modules.
+
+ This function imports and returns a fresh copy of the named Python module
+ by removing the named module from sys.modules before doing the import.
+ Note that unlike reload, the original module is not affected by
+ this operation.
+
+ *fresh* is an iterable of additional module names that are also removed
+ from the sys.modules cache before doing the import.
+
+ *blocked* is an iterable of module names that are replaced with None
+ in the module cache during the import to ensure that attempts to import
+ them raise ImportError.
+
+ The named module and any modules named in the *fresh* and *blocked*
+ parameters are saved before starting the import and then reinserted into
+ sys.modules when the fresh import is complete.
+
+ Module and package deprecation messages are suppressed during this import
+ if *deprecated* is True.
+
+ This function will raise ImportError if the named module cannot be
+ imported.
+ """
+ # NOTE: test_heapq, test_json and test_warnings include extra sanity checks
+ # to make sure that this utility function is working as expected
+ with _ignore_deprecated_imports(deprecated):
+ # Keep track of modules saved for later restoration as well
+ # as those which just need a blocking entry removed
+ orig_modules = {}
+ names_to_remove = []
+ _save_and_remove_module(name, orig_modules)
+ try:
+ for fresh_name in fresh:
+ _save_and_remove_module(fresh_name, orig_modules)
+ for blocked_name in blocked:
+ if not _save_and_block_module(blocked_name, orig_modules):
+ names_to_remove.append(blocked_name)
+ fresh_module = importlib.import_module(name)
+ except ImportError:
+ fresh_module = None
+ finally:
+ for orig_name, module in orig_modules.items():
+ sys.modules[orig_name] = module
+ for name_to_remove in names_to_remove:
+ del sys.modules[name_to_remove]
+ return fresh_module
+
+
+class CleanImport(object):
+ """Context manager to force import to return a new module reference.
+
+ This is useful for testing module-level behaviours, such as
+ the emission of a DeprecationWarning on import.
+
+ Use like this:
+
+ with CleanImport("foo"):
+ importlib.import_module("foo") # new reference
+ """
+
+ def __init__(self, *module_names):
+ self.original_modules = sys.modules.copy()
+ for module_name in module_names:
+ if module_name in sys.modules:
+ module = sys.modules[module_name]
+ # It is possible that module_name is just an alias for
+ # another module (e.g. stub for modules renamed in 3.x).
+ # In that case, we also need delete the real module to clear
+ # the import cache.
+ if module.__name__ != module_name:
+ del sys.modules[module.__name__]
+ del sys.modules[module_name]
+
+ def __enter__(self):
+ return self
+
+ def __exit__(self, *ignore_exc):
+ sys.modules.update(self.original_modules)
+
+
+class DirsOnSysPath(object):
+ """Context manager to temporarily add directories to sys.path.
+
+ This makes a copy of sys.path, appends any directories given
+ as positional arguments, then reverts sys.path to the copied
+ settings when the context ends.
+
+ Note that *all* sys.path modifications in the body of the
+ context manager, including replacement of the object,
+ will be reverted at the end of the block.
+ """
+
+ def __init__(self, *paths):
+ self.original_value = sys.path[:]
+ self.original_object = sys.path
+ sys.path.extend(paths)
+
+ def __enter__(self):
+ return self
+
+ def __exit__(self, *ignore_exc):
+ sys.path = self.original_object
+ sys.path[:] = self.original_value
+
+
+def modules_setup():
+ return sys.modules.copy(),
+
+
+def modules_cleanup(oldmodules):
+ # Encoders/decoders are registered permanently within the internal
+ # codec cache. If we destroy the corresponding modules their
+ # globals will be set to None which will trip up the cached functions.
+ encodings = [(k, v) for k, v in sys.modules.items()
+ if k.startswith('encodings.')]
+ sys.modules.clear()
+ sys.modules.update(encodings)
+ # XXX: This kind of problem can affect more than just encodings.
+ # In particular extension modules (such as _ssl) don't cope
+ # with reloading properly. Really, test modules should be cleaning
+ # out the test specific modules they know they added (ala test_runpy)
+ # rather than relying on this function (as test_importhooks and test_pkg
+ # do currently). Implicitly imported *real* modules should be left alone
+ # (see issue 10556).
+ sys.modules.update(oldmodules)