Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1 | :mod:`pkgutil` --- Package extension utility |
| 2 | ============================================ |
| 3 | |
| 4 | .. module:: pkgutil |
Georg Brandl | f1f8d47 | 2010-10-15 16:35:46 +0000 | [diff] [blame] | 5 | :synopsis: Utilities for the import system. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 6 | |
Raymond Hettinger | 469271d | 2011-01-27 20:38:46 +0000 | [diff] [blame] | 7 | **Source code:** :source:`Lib/pkgutil.py` |
| 8 | |
| 9 | -------------- |
| 10 | |
Georg Brandl | f1f8d47 | 2010-10-15 16:35:46 +0000 | [diff] [blame] | 11 | This module provides utilities for the import system, in particular package |
| 12 | support. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 13 | |
Eric Snow | d5f9223 | 2016-09-07 18:37:17 -0700 | [diff] [blame] | 14 | .. class:: ModuleInfo(module_finder, name, ispkg) |
| 15 | |
| 16 | A namedtuple that holds a brief summary of a module's info. |
| 17 | |
Berker Peksag | 18a7d2b | 2016-09-08 23:36:25 +0300 | [diff] [blame] | 18 | .. versionadded:: 3.6 |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 19 | |
| 20 | .. function:: extend_path(path, name) |
| 21 | |
Georg Brandl | f1f8d47 | 2010-10-15 16:35:46 +0000 | [diff] [blame] | 22 | Extend the search path for the modules which comprise a package. Intended |
| 23 | use is to place the following code in a package's :file:`__init__.py`:: |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 24 | |
| 25 | from pkgutil import extend_path |
| 26 | __path__ = extend_path(__path__, __name__) |
| 27 | |
Georg Brandl | f1f8d47 | 2010-10-15 16:35:46 +0000 | [diff] [blame] | 28 | This will add to the package's ``__path__`` all subdirectories of directories |
| 29 | on ``sys.path`` named after the package. This is useful if one wants to |
| 30 | distribute different parts of a single logical package as multiple |
| 31 | directories. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 32 | |
Georg Brandl | f1f8d47 | 2010-10-15 16:35:46 +0000 | [diff] [blame] | 33 | It also looks for :file:`\*.pkg` files beginning where ``*`` matches the |
| 34 | *name* argument. This feature is similar to :file:`\*.pth` files (see the |
| 35 | :mod:`site` module for more information), except that it doesn't special-case |
| 36 | lines starting with ``import``. A :file:`\*.pkg` file is trusted at face |
| 37 | value: apart from checking for duplicates, all entries found in a |
| 38 | :file:`\*.pkg` file are added to the path, regardless of whether they exist |
| 39 | on the filesystem. (This is a feature.) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 40 | |
| 41 | If the input path is not a list (as is the case for frozen packages) it is |
| 42 | returned unchanged. The input path is not modified; an extended copy is |
| 43 | returned. Items are only appended to the copy at the end. |
| 44 | |
Georg Brandl | f1f8d47 | 2010-10-15 16:35:46 +0000 | [diff] [blame] | 45 | It is assumed that :data:`sys.path` is a sequence. Items of :data:`sys.path` |
| 46 | that are not strings referring to existing directories are ignored. Unicode |
| 47 | items on :data:`sys.path` that cause errors when used as filenames may cause |
| 48 | this function to raise an exception (in line with :func:`os.path.isdir` |
| 49 | behavior). |
| 50 | |
| 51 | |
| 52 | .. class:: ImpImporter(dirname=None) |
| 53 | |
Senthil Kumaran | 4672060 | 2016-09-05 17:11:51 -0700 | [diff] [blame] | 54 | :pep:`302` Finder that wraps Python's "classic" import algorithm. |
Georg Brandl | f1f8d47 | 2010-10-15 16:35:46 +0000 | [diff] [blame] | 55 | |
Senthil Kumaran | 4672060 | 2016-09-05 17:11:51 -0700 | [diff] [blame] | 56 | If *dirname* is a string, a :pep:`302` finder is created that searches that |
| 57 | directory. If *dirname* is ``None``, a :pep:`302` finder is created that |
Georg Brandl | f1f8d47 | 2010-10-15 16:35:46 +0000 | [diff] [blame] | 58 | searches the current :data:`sys.path`, plus any modules that are frozen or |
| 59 | built-in. |
| 60 | |
| 61 | Note that :class:`ImpImporter` does not currently support being used by |
| 62 | placement on :data:`sys.meta_path`. |
| 63 | |
Nick Coghlan | 85e729e | 2012-07-15 18:09:52 +1000 | [diff] [blame] | 64 | .. deprecated:: 3.3 |
| 65 | This emulation is no longer needed, as the standard import mechanism |
Georg Brandl | 29636ae | 2014-03-24 08:42:37 +0100 | [diff] [blame] | 66 | is now fully PEP 302 compliant and available in :mod:`importlib`. |
Nick Coghlan | 85e729e | 2012-07-15 18:09:52 +1000 | [diff] [blame] | 67 | |
Georg Brandl | f1f8d47 | 2010-10-15 16:35:46 +0000 | [diff] [blame] | 68 | |
| 69 | .. class:: ImpLoader(fullname, file, filename, etc) |
| 70 | |
Senthil Kumaran | 4672060 | 2016-09-05 17:11:51 -0700 | [diff] [blame] | 71 | :term:`Loader` that wraps Python's "classic" import algorithm. |
Georg Brandl | f1f8d47 | 2010-10-15 16:35:46 +0000 | [diff] [blame] | 72 | |
Nick Coghlan | 85e729e | 2012-07-15 18:09:52 +1000 | [diff] [blame] | 73 | .. deprecated:: 3.3 |
| 74 | This emulation is no longer needed, as the standard import mechanism |
Georg Brandl | 29636ae | 2014-03-24 08:42:37 +0100 | [diff] [blame] | 75 | is now fully PEP 302 compliant and available in :mod:`importlib`. |
Nick Coghlan | 85e729e | 2012-07-15 18:09:52 +1000 | [diff] [blame] | 76 | |
Georg Brandl | f1f8d47 | 2010-10-15 16:35:46 +0000 | [diff] [blame] | 77 | |
| 78 | .. function:: find_loader(fullname) |
| 79 | |
Senthil Kumaran | 4672060 | 2016-09-05 17:11:51 -0700 | [diff] [blame] | 80 | Retrieve a module :term:`loader` for the given *fullname*. |
Georg Brandl | f1f8d47 | 2010-10-15 16:35:46 +0000 | [diff] [blame] | 81 | |
Nick Coghlan | dc855b7 | 2014-03-04 20:39:42 +1000 | [diff] [blame] | 82 | This is a backwards compatibility wrapper around |
| 83 | :func:`importlib.util.find_spec` that converts most failures to |
| 84 | :exc:`ImportError` and only returns the loader rather than the full |
| 85 | :class:`ModuleSpec`. |
Nick Coghlan | 85e729e | 2012-07-15 18:09:52 +1000 | [diff] [blame] | 86 | |
| 87 | .. versionchanged:: 3.3 |
| 88 | Updated to be based directly on :mod:`importlib` rather than relying |
Nick Coghlan | 8ecf504 | 2012-07-15 21:19:18 +1000 | [diff] [blame] | 89 | on the package internal PEP 302 import emulation. |
Georg Brandl | f1f8d47 | 2010-10-15 16:35:46 +0000 | [diff] [blame] | 90 | |
Nick Coghlan | dc855b7 | 2014-03-04 20:39:42 +1000 | [diff] [blame] | 91 | .. versionchanged:: 3.4 |
| 92 | Updated to be based on :pep:`451` |
Georg Brandl | f1f8d47 | 2010-10-15 16:35:46 +0000 | [diff] [blame] | 93 | |
| 94 | .. function:: get_importer(path_item) |
| 95 | |
Senthil Kumaran | 4672060 | 2016-09-05 17:11:51 -0700 | [diff] [blame] | 96 | Retrieve a :term:`finder` for the given *path_item*. |
Georg Brandl | f1f8d47 | 2010-10-15 16:35:46 +0000 | [diff] [blame] | 97 | |
Senthil Kumaran | 4672060 | 2016-09-05 17:11:51 -0700 | [diff] [blame] | 98 | The returned finder is cached in :data:`sys.path_importer_cache` if it was |
Georg Brandl | f1f8d47 | 2010-10-15 16:35:46 +0000 | [diff] [blame] | 99 | newly created by a path hook. |
| 100 | |
Georg Brandl | f1f8d47 | 2010-10-15 16:35:46 +0000 | [diff] [blame] | 101 | The cache (or part of it) can be cleared manually if a rescan of |
| 102 | :data:`sys.path_hooks` is necessary. |
| 103 | |
Nick Coghlan | 85e729e | 2012-07-15 18:09:52 +1000 | [diff] [blame] | 104 | .. versionchanged:: 3.3 |
| 105 | Updated to be based directly on :mod:`importlib` rather than relying |
Nick Coghlan | 8ecf504 | 2012-07-15 21:19:18 +1000 | [diff] [blame] | 106 | on the package internal PEP 302 import emulation. |
Nick Coghlan | 85e729e | 2012-07-15 18:09:52 +1000 | [diff] [blame] | 107 | |
Georg Brandl | f1f8d47 | 2010-10-15 16:35:46 +0000 | [diff] [blame] | 108 | |
| 109 | .. function:: get_loader(module_or_name) |
| 110 | |
Senthil Kumaran | 4672060 | 2016-09-05 17:11:51 -0700 | [diff] [blame] | 111 | Get a :term:`loader` object for *module_or_name*. |
Georg Brandl | f1f8d47 | 2010-10-15 16:35:46 +0000 | [diff] [blame] | 112 | |
| 113 | If the module or package is accessible via the normal import mechanism, a |
| 114 | wrapper around the relevant part of that machinery is returned. Returns |
| 115 | ``None`` if the module cannot be found or imported. If the named module is |
| 116 | not already imported, its containing package (if any) is imported, in order |
| 117 | to establish the package ``__path__``. |
| 118 | |
Nick Coghlan | 85e729e | 2012-07-15 18:09:52 +1000 | [diff] [blame] | 119 | .. versionchanged:: 3.3 |
| 120 | Updated to be based directly on :mod:`importlib` rather than relying |
Nick Coghlan | 8ecf504 | 2012-07-15 21:19:18 +1000 | [diff] [blame] | 121 | on the package internal PEP 302 import emulation. |
Nick Coghlan | 85e729e | 2012-07-15 18:09:52 +1000 | [diff] [blame] | 122 | |
Nick Coghlan | dc855b7 | 2014-03-04 20:39:42 +1000 | [diff] [blame] | 123 | .. versionchanged:: 3.4 |
| 124 | Updated to be based on :pep:`451` |
| 125 | |
Georg Brandl | f1f8d47 | 2010-10-15 16:35:46 +0000 | [diff] [blame] | 126 | |
| 127 | .. function:: iter_importers(fullname='') |
| 128 | |
Senthil Kumaran | 4672060 | 2016-09-05 17:11:51 -0700 | [diff] [blame] | 129 | Yield :term:`finder` objects for the given module name. |
Georg Brandl | f1f8d47 | 2010-10-15 16:35:46 +0000 | [diff] [blame] | 130 | |
Senthil Kumaran | 4672060 | 2016-09-05 17:11:51 -0700 | [diff] [blame] | 131 | If fullname contains a '.', the finders will be for the package |
Nick Coghlan | 85e729e | 2012-07-15 18:09:52 +1000 | [diff] [blame] | 132 | containing fullname, otherwise they will be all registered top level |
Senthil Kumaran | 4672060 | 2016-09-05 17:11:51 -0700 | [diff] [blame] | 133 | finders (i.e. those on both sys.meta_path and sys.path_hooks). |
Georg Brandl | f1f8d47 | 2010-10-15 16:35:46 +0000 | [diff] [blame] | 134 | |
Nick Coghlan | 85e729e | 2012-07-15 18:09:52 +1000 | [diff] [blame] | 135 | If the named module is in a package, that package is imported as a side |
| 136 | effect of invoking this function. |
Georg Brandl | f1f8d47 | 2010-10-15 16:35:46 +0000 | [diff] [blame] | 137 | |
Senthil Kumaran | 4672060 | 2016-09-05 17:11:51 -0700 | [diff] [blame] | 138 | If no module name is specified, all top level finders are produced. |
Georg Brandl | f1f8d47 | 2010-10-15 16:35:46 +0000 | [diff] [blame] | 139 | |
Nick Coghlan | 85e729e | 2012-07-15 18:09:52 +1000 | [diff] [blame] | 140 | .. versionchanged:: 3.3 |
| 141 | Updated to be based directly on :mod:`importlib` rather than relying |
Nick Coghlan | 8ecf504 | 2012-07-15 21:19:18 +1000 | [diff] [blame] | 142 | on the package internal PEP 302 import emulation. |
Georg Brandl | f1f8d47 | 2010-10-15 16:35:46 +0000 | [diff] [blame] | 143 | |
| 144 | |
| 145 | .. function:: iter_modules(path=None, prefix='') |
| 146 | |
Eric Snow | d5f9223 | 2016-09-07 18:37:17 -0700 | [diff] [blame] | 147 | Yields :class:`ModuleInfo` for all submodules on *path*, or, if |
Martin Panter | f47a400 | 2016-05-15 00:13:04 +0000 | [diff] [blame] | 148 | *path* is ``None``, all top-level modules on ``sys.path``. |
Georg Brandl | f1f8d47 | 2010-10-15 16:35:46 +0000 | [diff] [blame] | 149 | |
| 150 | *path* should be either ``None`` or a list of paths to look for modules in. |
| 151 | |
| 152 | *prefix* is a string to output on the front of every module name on output. |
| 153 | |
Brett Cannon | 47b3239 | 2012-06-15 19:21:07 -0400 | [diff] [blame] | 154 | .. note:: |
Éric Araujo | fa5e6e4 | 2014-03-12 19:51:00 -0400 | [diff] [blame] | 155 | |
Nick Coghlan | 8ecf504 | 2012-07-15 21:19:18 +1000 | [diff] [blame] | 156 | Only works for a :term:`finder` which defines an ``iter_modules()`` |
| 157 | method. This interface is non-standard, so the module also provides |
| 158 | implementations for :class:`importlib.machinery.FileFinder` and |
| 159 | :class:`zipimport.zipimporter`. |
Brett Cannon | b194497 | 2012-07-09 14:10:23 -0400 | [diff] [blame] | 160 | |
| 161 | .. versionchanged:: 3.3 |
Nick Coghlan | 8ecf504 | 2012-07-15 21:19:18 +1000 | [diff] [blame] | 162 | Updated to be based directly on :mod:`importlib` rather than relying |
| 163 | on the package internal PEP 302 import emulation. |
Brett Cannon | 47b3239 | 2012-06-15 19:21:07 -0400 | [diff] [blame] | 164 | |
Georg Brandl | f1f8d47 | 2010-10-15 16:35:46 +0000 | [diff] [blame] | 165 | |
| 166 | .. function:: walk_packages(path=None, prefix='', onerror=None) |
| 167 | |
Eric Snow | d5f9223 | 2016-09-07 18:37:17 -0700 | [diff] [blame] | 168 | Yields :class:`ModuleInfo` for all modules recursively on |
Martin Panter | f47a400 | 2016-05-15 00:13:04 +0000 | [diff] [blame] | 169 | *path*, or, if *path* is ``None``, all accessible modules. |
Georg Brandl | f1f8d47 | 2010-10-15 16:35:46 +0000 | [diff] [blame] | 170 | |
| 171 | *path* should be either ``None`` or a list of paths to look for modules in. |
| 172 | |
| 173 | *prefix* is a string to output on the front of every module name on output. |
| 174 | |
| 175 | Note that this function must import all *packages* (*not* all modules!) on |
| 176 | the given *path*, in order to access the ``__path__`` attribute to find |
| 177 | submodules. |
| 178 | |
| 179 | *onerror* is a function which gets called with one argument (the name of the |
| 180 | package which was being imported) if any exception occurs while trying to |
| 181 | import a package. If no *onerror* function is supplied, :exc:`ImportError`\s |
| 182 | are caught and ignored, while all other exceptions are propagated, |
| 183 | terminating the search. |
| 184 | |
| 185 | Examples:: |
| 186 | |
| 187 | # list all modules python can access |
| 188 | walk_packages() |
| 189 | |
| 190 | # list all submodules of ctypes |
| 191 | walk_packages(ctypes.__path__, ctypes.__name__ + '.') |
| 192 | |
Brett Cannon | 47b3239 | 2012-06-15 19:21:07 -0400 | [diff] [blame] | 193 | .. note:: |
Éric Araujo | fa5e6e4 | 2014-03-12 19:51:00 -0400 | [diff] [blame] | 194 | |
Nick Coghlan | 8ecf504 | 2012-07-15 21:19:18 +1000 | [diff] [blame] | 195 | Only works for a :term:`finder` which defines an ``iter_modules()`` |
| 196 | method. This interface is non-standard, so the module also provides |
| 197 | implementations for :class:`importlib.machinery.FileFinder` and |
| 198 | :class:`zipimport.zipimporter`. |
Brett Cannon | b194497 | 2012-07-09 14:10:23 -0400 | [diff] [blame] | 199 | |
| 200 | .. versionchanged:: 3.3 |
Nick Coghlan | 8ecf504 | 2012-07-15 21:19:18 +1000 | [diff] [blame] | 201 | Updated to be based directly on :mod:`importlib` rather than relying |
| 202 | on the package internal PEP 302 import emulation. |
Brett Cannon | 47b3239 | 2012-06-15 19:21:07 -0400 | [diff] [blame] | 203 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 204 | |
Christian Heimes | dae2a89 | 2008-04-19 00:55:37 +0000 | [diff] [blame] | 205 | .. function:: get_data(package, resource) |
| 206 | |
| 207 | Get a resource from a package. |
| 208 | |
Senthil Kumaran | 1cea56b | 2016-10-13 22:58:47 -0700 | [diff] [blame] | 209 | This is a wrapper for the :term:`loader` |
| 210 | :meth:`get_data <importlib.abc.ResourceLoader.get_data>` API. The |
Georg Brandl | f1f8d47 | 2010-10-15 16:35:46 +0000 | [diff] [blame] | 211 | *package* argument should be the name of a package, in standard module format |
| 212 | (``foo.bar``). The *resource* argument should be in the form of a relative |
| 213 | filename, using ``/`` as the path separator. The parent directory name |
Christian Heimes | dae2a89 | 2008-04-19 00:55:37 +0000 | [diff] [blame] | 214 | ``..`` is not allowed, and nor is a rooted name (starting with a ``/``). |
| 215 | |
Georg Brandl | f1f8d47 | 2010-10-15 16:35:46 +0000 | [diff] [blame] | 216 | The function returns a binary string that is the contents of the specified |
| 217 | resource. |
Christian Heimes | dae2a89 | 2008-04-19 00:55:37 +0000 | [diff] [blame] | 218 | |
| 219 | For packages located in the filesystem, which have already been imported, |
| 220 | this is the rough equivalent of:: |
| 221 | |
Georg Brandl | f1f8d47 | 2010-10-15 16:35:46 +0000 | [diff] [blame] | 222 | d = os.path.dirname(sys.modules[package].__file__) |
| 223 | data = open(os.path.join(d, resource), 'rb').read() |
Christian Heimes | dae2a89 | 2008-04-19 00:55:37 +0000 | [diff] [blame] | 224 | |
Senthil Kumaran | 4672060 | 2016-09-05 17:11:51 -0700 | [diff] [blame] | 225 | If the package cannot be located or loaded, or it uses a :term:`loader` |
Senthil Kumaran | 1cea56b | 2016-10-13 22:58:47 -0700 | [diff] [blame] | 226 | which does not support :meth:`get_data <importlib.abc.ResourceLoader.get_data>`, |
Brett Cannon | bc538e3 | 2016-12-10 14:13:38 -0800 | [diff] [blame] | 227 | then ``None`` is returned. In particular, the :term:`loader` for |
| 228 | :term:`namespace packages <namespace package>` does not support |
| 229 | :meth:`get_data <importlib.abc.ResourceLoader.get_data>`. |