blob: 99bfeacbbc740711b525f39ee1ee7867df4b2fa3 [file] [log] [blame]
Serhiy Storchaka2b57c432018-12-19 08:09:46 +02001:mod:`!importlib` --- The implementation of :keyword:`!import`
2==============================================================
Brett Cannonafccd632009-01-20 02:21:27 +00003
4.. module:: importlib
Brett Cannon07fbd782014-02-06 09:46:08 -05005 :synopsis: The implementation of the import machinery.
Brett Cannonafccd632009-01-20 02:21:27 +00006
7.. moduleauthor:: Brett Cannon <brett@python.org>
8.. sectionauthor:: Brett Cannon <brett@python.org>
9
10.. versionadded:: 3.1
11
Terry Jan Reedyfa089b92016-06-11 15:02:54 -040012**Source code:** :source:`Lib/importlib/__init__.py`
13
14--------------
Brett Cannonafccd632009-01-20 02:21:27 +000015
16Introduction
17------------
18
Brett Cannon07fbd782014-02-06 09:46:08 -050019The purpose of the :mod:`importlib` package is two-fold. One is to provide the
Brett Cannonafccd632009-01-20 02:21:27 +000020implementation of the :keyword:`import` statement (and thus, by extension, the
21:func:`__import__` function) in Python source code. This provides an
Serhiy Storchaka2b57c432018-12-19 08:09:46 +020022implementation of :keyword:`!import` which is portable to any Python
Brett Cannon07fbd782014-02-06 09:46:08 -050023interpreter. This also provides an implementation which is easier to
Brett Cannonf23e3742010-06-27 23:57:46 +000024comprehend than one implemented in a programming language other than Python.
Brett Cannonafccd632009-01-20 02:21:27 +000025
Brett Cannonf23e3742010-06-27 23:57:46 +000026Two, the components to implement :keyword:`import` are exposed in this
Brett Cannonafccd632009-01-20 02:21:27 +000027package, making it easier for users to create their own custom objects (known
Brett Cannondebb98d2009-02-16 04:18:01 +000028generically as an :term:`importer`) to participate in the import process.
Brett Cannonafccd632009-01-20 02:21:27 +000029
30.. seealso::
31
32 :ref:`import`
33 The language reference for the :keyword:`import` statement.
34
Benjamin Peterson60dbed12017-09-05 16:24:39 -070035 `Packages specification <https://www.python.org/doc/essays/packages/>`__
Brett Cannonafccd632009-01-20 02:21:27 +000036 Original specification of packages. Some semantics have changed since
Georg Brandl375aec22011-01-15 17:03:02 +000037 the writing of this document (e.g. redirecting based on ``None``
Brett Cannonafccd632009-01-20 02:21:27 +000038 in :data:`sys.modules`).
39
40 The :func:`.__import__` function
Brett Cannon0e13c942010-06-29 18:26:11 +000041 The :keyword:`import` statement is syntactic sugar for this function.
Brett Cannonafccd632009-01-20 02:21:27 +000042
43 :pep:`235`
44 Import on Case-Insensitive Platforms
45
46 :pep:`263`
47 Defining Python Source Code Encodings
48
49 :pep:`302`
Brett Cannonf23e3742010-06-27 23:57:46 +000050 New Import Hooks
Brett Cannonafccd632009-01-20 02:21:27 +000051
52 :pep:`328`
53 Imports: Multi-Line and Absolute/Relative
54
55 :pep:`366`
56 Main module explicit relative imports
57
Eric Snow338502b2016-05-28 11:56:53 -060058 :pep:`420`
59 Implicit namespace packages
60
Brett Cannon07fbd782014-02-06 09:46:08 -050061 :pep:`451`
62 A ModuleSpec Type for the Import System
63
Nick Coghland5cacbb2015-05-23 22:24:10 +100064 :pep:`488`
65 Elimination of PYO files
66
67 :pep:`489`
68 Multi-phase extension module initialization
69
Benjamin Peterson42aa93b2017-12-09 10:26:52 -080070 :pep:`552`
71 Deterministic pycs
72
Brett Cannon8917d5e2010-01-13 19:21:00 +000073 :pep:`3120`
Brett Cannonafccd632009-01-20 02:21:27 +000074 Using UTF-8 as the Default Source Encoding
75
Brett Cannon30b7a902010-06-27 21:49:22 +000076 :pep:`3147`
77 PYC Repository Directories
78
Brett Cannonafccd632009-01-20 02:21:27 +000079
80Functions
81---------
82
Brett Cannoncb4996a2012-08-06 16:34:44 -040083.. function:: __import__(name, globals=None, locals=None, fromlist=(), level=0)
Brett Cannonafccd632009-01-20 02:21:27 +000084
Brett Cannonf23e3742010-06-27 23:57:46 +000085 An implementation of the built-in :func:`__import__` function.
Brett Cannonafccd632009-01-20 02:21:27 +000086
Brett Cannon3fa84222015-02-20 10:34:20 -050087 .. note::
88 Programmatic importing of modules should use :func:`import_module`
89 instead of this function.
90
Brett Cannonafccd632009-01-20 02:21:27 +000091.. function:: import_module(name, package=None)
92
Brett Cannon33418c82009-01-22 18:37:20 +000093 Import a module. The *name* argument specifies what module to
Brett Cannonafccd632009-01-20 02:21:27 +000094 import in absolute or relative terms
95 (e.g. either ``pkg.mod`` or ``..mod``). If the name is
Guido van Rossum09613542009-03-30 20:34:57 +000096 specified in relative terms, then the *package* argument must be set to
97 the name of the package which is to act as the anchor for resolving the
Brett Cannonafccd632009-01-20 02:21:27 +000098 package name (e.g. ``import_module('..mod', 'pkg.subpkg')`` will import
Brett Cannon2c318a12009-02-07 01:15:27 +000099 ``pkg.mod``).
Brett Cannon78246b62009-01-25 04:56:30 +0000100
Brett Cannon2c318a12009-02-07 01:15:27 +0000101 The :func:`import_module` function acts as a simplifying wrapper around
Brett Cannon9f4cb1c2009-04-01 23:26:47 +0000102 :func:`importlib.__import__`. This means all semantics of the function are
Brett Cannon3fa84222015-02-20 10:34:20 -0500103 derived from :func:`importlib.__import__`. The most important difference
104 between these two functions is that :func:`import_module` returns the
105 specified package or module (e.g. ``pkg.mod``), while :func:`__import__`
106 returns the top-level package or module (e.g. ``pkg``).
107
108 If you are dynamically importing a module that was created since the
109 interpreter began execution (e.g., created a Python source file), you may
110 need to call :func:`invalidate_caches` in order for the new module to be
111 noticed by the import system.
Guido van Rossum09613542009-03-30 20:34:57 +0000112
Brett Cannon98620d82013-12-13 13:57:41 -0500113 .. versionchanged:: 3.3
114 Parent packages are automatically imported.
115
Brett Cannonee78a2b2012-05-12 17:43:17 -0400116.. function:: find_loader(name, path=None)
117
118 Find the loader for a module, optionally within the specified *path*. If the
119 module is in :attr:`sys.modules`, then ``sys.modules[name].__loader__`` is
Brett Cannon32799232013-03-13 11:09:08 -0700120 returned (unless the loader would be ``None`` or is not set, in which case
Brett Cannonee78a2b2012-05-12 17:43:17 -0400121 :exc:`ValueError` is raised). Otherwise a search using :attr:`sys.meta_path`
122 is done. ``None`` is returned if no loader is found.
123
Benjamin Petersonf7e2ea22016-09-05 14:02:59 -0700124 A dotted name does not have its parents implicitly imported as that requires
Brett Cannon56b4ca72012-11-17 09:30:55 -0500125 loading them and that may not be desired. To properly import a submodule you
126 will need to import all parent packages of the submodule and use the correct
127 argument to *path*.
Brett Cannonee78a2b2012-05-12 17:43:17 -0400128
Brett Cannon32799232013-03-13 11:09:08 -0700129 .. versionadded:: 3.3
130
131 .. versionchanged:: 3.4
132 If ``__loader__`` is not set, raise :exc:`ValueError`, just like when the
133 attribute is set to ``None``.
134
Eric Snowca2d8542013-12-16 23:06:52 -0700135 .. deprecated:: 3.4
Eric Snow6029e082014-01-25 15:32:46 -0700136 Use :func:`importlib.util.find_spec` instead.
Eric Snowca2d8542013-12-16 23:06:52 -0700137
Antoine Pitrouc541f8e2012-02-20 01:48:16 +0100138.. function:: invalidate_caches()
139
Brett Cannonf4dc9202012-08-10 12:21:12 -0400140 Invalidate the internal caches of finders stored at
141 :data:`sys.meta_path`. If a finder implements ``invalidate_caches()`` then it
Brett Cannon4067aa22013-04-27 23:20:32 -0400142 will be called to perform the invalidation. This function should be called
143 if any modules are created/installed while your program is running to
144 guarantee all finders will notice the new module's existence.
Antoine Pitrouc541f8e2012-02-20 01:48:16 +0100145
146 .. versionadded:: 3.3
147
Brett Cannon3fe35e62013-06-14 15:04:26 -0400148.. function:: reload(module)
149
150 Reload a previously imported *module*. The argument must be a module object,
151 so it must have been successfully imported before. This is useful if you
152 have edited the module source file using an external editor and want to try
153 out the new version without leaving the Python interpreter. The return value
Brett Cannon8ad37862013-10-25 13:52:46 -0400154 is the module object (which can be different if re-importing causes a
155 different object to be placed in :data:`sys.modules`).
Brett Cannon3fe35e62013-06-14 15:04:26 -0400156
Brett Cannon8ad37862013-10-25 13:52:46 -0400157 When :func:`reload` is executed:
Brett Cannon3fe35e62013-06-14 15:04:26 -0400158
Larry Hastings3732ed22014-03-15 21:13:56 -0700159 * Python module's code is recompiled and the module-level code re-executed,
Brett Cannon3fe35e62013-06-14 15:04:26 -0400160 defining a new set of objects which are bound to names in the module's
161 dictionary by reusing the :term:`loader` which originally loaded the
162 module. The ``init`` function of extension modules is not called a second
163 time.
164
165 * As with all other objects in Python the old objects are only reclaimed
166 after their reference counts drop to zero.
167
168 * The names in the module namespace are updated to point to any new or
169 changed objects.
170
171 * Other references to the old objects (such as names external to the module) are
172 not rebound to refer to the new objects and must be updated in each namespace
173 where they occur if that is desired.
174
175 There are a number of other caveats:
176
Brett Cannon3fe35e62013-06-14 15:04:26 -0400177 When a module is reloaded, its dictionary (containing the module's global
178 variables) is retained. Redefinitions of names will override the old
179 definitions, so this is generally not a problem. If the new version of a
180 module does not define a name that was defined by the old version, the old
181 definition remains. This feature can be used to the module's advantage if it
182 maintains a global table or cache of objects --- with a :keyword:`try`
183 statement it can test for the table's presence and skip its initialization if
184 desired::
185
186 try:
187 cache
188 except NameError:
189 cache = {}
190
Robert Collins1ae28d22015-08-05 08:20:53 +1200191 It is generally not very useful to reload built-in or dynamically loaded
192 modules. Reloading :mod:`sys`, :mod:`__main__`, :mod:`builtins` and other
193 key modules is not recommended. In many cases extension modules are not
194 designed to be initialized more than once, and may fail in arbitrary ways
195 when reloaded.
Brett Cannon3fe35e62013-06-14 15:04:26 -0400196
197 If a module imports objects from another module using :keyword:`from` ...
198 :keyword:`import` ..., calling :func:`reload` for the other module does not
199 redefine the objects imported from it --- one way around this is to
Serhiy Storchaka2b57c432018-12-19 08:09:46 +0200200 re-execute the :keyword:`!from` statement, another is to use :keyword:`!import`
Brett Cannon3fe35e62013-06-14 15:04:26 -0400201 and qualified names (*module.name*) instead.
202
203 If a module instantiates instances of a class, reloading the module that
204 defines the class does not affect the method definitions of the instances ---
205 they continue to use the old class definition. The same is true for derived
206 classes.
207
208 .. versionadded:: 3.4
Garvit Khatri94987822017-05-25 03:49:50 +0530209 .. versionchanged:: 3.7
210 :exc:`ModuleNotFoundError` is raised when the module being reloaded lacks
211 a :class:`ModuleSpec`.
Brett Cannon3fe35e62013-06-14 15:04:26 -0400212
Brett Cannon78246b62009-01-25 04:56:30 +0000213
Brett Cannon2a922ed2009-03-09 03:35:50 +0000214:mod:`importlib.abc` -- Abstract base classes related to import
215---------------------------------------------------------------
216
217.. module:: importlib.abc
218 :synopsis: Abstract base classes related to import
219
Terry Jan Reedydcb6c882016-06-22 22:46:34 -0400220**Source code:** :source:`Lib/importlib/abc.py`
221
222--------------
223
224
Brett Cannon2a922ed2009-03-09 03:35:50 +0000225The :mod:`importlib.abc` module contains all of the core abstract base classes
226used by :keyword:`import`. Some subclasses of the core abstract base classes
227are also provided to help in implementing the core ABCs.
228
Andrew Svetlova8656542012-08-13 22:19:01 +0300229ABC hierarchy::
230
231 object
Brett Cannon1b799182012-08-17 14:08:24 -0400232 +-- Finder (deprecated)
Andrew Svetlova8656542012-08-13 22:19:01 +0300233 | +-- MetaPathFinder
234 | +-- PathEntryFinder
235 +-- Loader
236 +-- ResourceLoader --------+
237 +-- InspectLoader |
238 +-- ExecutionLoader --+
239 +-- FileLoader
240 +-- SourceLoader
Andrew Svetlova8656542012-08-13 22:19:01 +0300241
Brett Cannon2a922ed2009-03-09 03:35:50 +0000242
243.. class:: Finder
244
Brett Cannon1b799182012-08-17 14:08:24 -0400245 An abstract base class representing a :term:`finder`.
246
247 .. deprecated:: 3.3
248 Use :class:`MetaPathFinder` or :class:`PathEntryFinder` instead.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000249
Berker Peksag6e9d2e62015-12-08 12:14:50 +0200250 .. abstractmethod:: find_module(fullname, path=None)
Brett Cannonb46a1792012-02-27 18:15:42 -0500251
Xtreakc151f782018-06-16 10:38:31 +0530252 An abstract method for finding a :term:`loader` for the specified
Brett Cannonf4dc9202012-08-10 12:21:12 -0400253 module. Originally specified in :pep:`302`, this method was meant
254 for use in :data:`sys.meta_path` and in the path-based import subsystem.
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000255
Brett Cannon100883f2013-04-09 16:59:39 -0400256 .. versionchanged:: 3.4
257 Returns ``None`` when called instead of raising
258 :exc:`NotImplementedError`.
259
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000260
Brett Cannon077ef452012-08-02 17:50:06 -0400261.. class:: MetaPathFinder
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000262
Brett Cannonf4dc9202012-08-10 12:21:12 -0400263 An abstract base class representing a :term:`meta path finder`. For
264 compatibility, this is a subclass of :class:`Finder`.
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000265
266 .. versionadded:: 3.3
267
Eric Snowca2d8542013-12-16 23:06:52 -0700268 .. method:: find_spec(fullname, path, target=None)
269
270 An abstract method for finding a :term:`spec <module spec>` for
271 the specified module. If this is a top-level import, *path* will
272 be ``None``. Otherwise, this is a search for a subpackage or
273 module and *path* will be the value of :attr:`__path__` from the
274 parent package. If a spec cannot be found, ``None`` is returned.
275 When passed in, ``target`` is a module object that the finder may
Brett Cannona85e9272016-01-08 14:33:09 -0800276 use to make a more educated guess about what spec to return.
jdkandersson9cbb97b2019-09-11 02:06:22 +1000277 :func:`importlib.util.spec_from_loader` may be useful for implementing
278 concrete ``MetaPathFinders``.
Eric Snowca2d8542013-12-16 23:06:52 -0700279
280 .. versionadded:: 3.4
281
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000282 .. method:: find_module(fullname, path)
283
Eric Snowca2d8542013-12-16 23:06:52 -0700284 A legacy method for finding a :term:`loader` for the specified
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000285 module. If this is a top-level import, *path* will be ``None``.
Ezio Melotti1f67e802012-10-21 07:24:13 +0300286 Otherwise, this is a search for a subpackage or module and *path*
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000287 will be the value of :attr:`__path__` from the parent
288 package. If a loader cannot be found, ``None`` is returned.
289
Brett Cannon8d942292014-01-07 15:52:42 -0500290 If :meth:`find_spec` is defined, backwards-compatible functionality is
291 provided.
292
Brett Cannon100883f2013-04-09 16:59:39 -0400293 .. versionchanged:: 3.4
294 Returns ``None`` when called instead of raising
Brett Cannon8d942292014-01-07 15:52:42 -0500295 :exc:`NotImplementedError`. Can use :meth:`find_spec` to provide
296 functionality.
Brett Cannon100883f2013-04-09 16:59:39 -0400297
Eric Snowca2d8542013-12-16 23:06:52 -0700298 .. deprecated:: 3.4
299 Use :meth:`find_spec` instead.
300
Brett Cannonf4dc9202012-08-10 12:21:12 -0400301 .. method:: invalidate_caches()
302
303 An optional method which, when called, should invalidate any internal
Brett Cannona6e85812012-08-11 19:41:27 -0400304 cache used by the finder. Used by :func:`importlib.invalidate_caches`
305 when invalidating the caches of all finders on :data:`sys.meta_path`.
Brett Cannonf4dc9202012-08-10 12:21:12 -0400306
Brett Cannon100883f2013-04-09 16:59:39 -0400307 .. versionchanged:: 3.4
308 Returns ``None`` when called instead of ``NotImplemented``.
309
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000310
Brett Cannon077ef452012-08-02 17:50:06 -0400311.. class:: PathEntryFinder
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000312
Brett Cannonf4dc9202012-08-10 12:21:12 -0400313 An abstract base class representing a :term:`path entry finder`. Though
314 it bears some similarities to :class:`MetaPathFinder`, ``PathEntryFinder``
315 is meant for use only within the path-based import subsystem provided
316 by :class:`PathFinder`. This ABC is a subclass of :class:`Finder` for
Brett Cannon100883f2013-04-09 16:59:39 -0400317 compatibility reasons only.
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000318
319 .. versionadded:: 3.3
320
Eric Snowca2d8542013-12-16 23:06:52 -0700321 .. method:: find_spec(fullname, target=None)
322
323 An abstract method for finding a :term:`spec <module spec>` for
324 the specified module. The finder will search for the module only
325 within the :term:`path entry` to which it is assigned. If a spec
326 cannot be found, ``None`` is returned. When passed in, ``target``
327 is a module object that the finder may use to make a more educated
jdkandersson9cbb97b2019-09-11 02:06:22 +1000328 guess about what spec to return. :func:`importlib.util.spec_from_loader`
329 may be useful for implementing concrete ``PathEntryFinders``.
Eric Snowca2d8542013-12-16 23:06:52 -0700330
331 .. versionadded:: 3.4
332
Brett Cannon4067aa22013-04-27 23:20:32 -0400333 .. method:: find_loader(fullname)
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000334
Eric Snowca2d8542013-12-16 23:06:52 -0700335 A legacy method for finding a :term:`loader` for the specified
Brett Cannonf4dc9202012-08-10 12:21:12 -0400336 module. Returns a 2-tuple of ``(loader, portion)`` where ``portion``
337 is a sequence of file system locations contributing to part of a namespace
338 package. The loader may be ``None`` while specifying ``portion`` to
339 signify the contribution of the file system locations to a namespace
340 package. An empty list can be used for ``portion`` to signify the loader
Brett Cannon100883f2013-04-09 16:59:39 -0400341 is not part of a namespace package. If ``loader`` is ``None`` and
342 ``portion`` is the empty list then no loader or location for a namespace
343 package were found (i.e. failure to find anything for the module).
344
Brett Cannon8d942292014-01-07 15:52:42 -0500345 If :meth:`find_spec` is defined then backwards-compatible functionality is
346 provided.
347
Brett Cannon100883f2013-04-09 16:59:39 -0400348 .. versionchanged:: 3.4
349 Returns ``(None, [])`` instead of raising :exc:`NotImplementedError`.
Brett Cannon8d942292014-01-07 15:52:42 -0500350 Uses :meth:`find_spec` when available to provide functionality.
Brett Cannonf4dc9202012-08-10 12:21:12 -0400351
Eric Snowca2d8542013-12-16 23:06:52 -0700352 .. deprecated:: 3.4
353 Use :meth:`find_spec` instead.
354
Brett Cannon4067aa22013-04-27 23:20:32 -0400355 .. method:: find_module(fullname)
Brett Cannonf4dc9202012-08-10 12:21:12 -0400356
357 A concrete implementation of :meth:`Finder.find_module` which is
358 equivalent to ``self.find_loader(fullname)[0]``.
359
Eric Snowca2d8542013-12-16 23:06:52 -0700360 .. deprecated:: 3.4
361 Use :meth:`find_spec` instead.
362
Brett Cannonf4dc9202012-08-10 12:21:12 -0400363 .. method:: invalidate_caches()
364
365 An optional method which, when called, should invalidate any internal
Brett Cannona6e85812012-08-11 19:41:27 -0400366 cache used by the finder. Used by :meth:`PathFinder.invalidate_caches`
Brett Cannonf4dc9202012-08-10 12:21:12 -0400367 when invalidating the caches of all cached finders.
Brett Cannonb46a1792012-02-27 18:15:42 -0500368
Brett Cannon2a922ed2009-03-09 03:35:50 +0000369
370.. class:: Loader
371
372 An abstract base class for a :term:`loader`.
Guido van Rossum09613542009-03-30 20:34:57 +0000373 See :pep:`302` for the exact definition for a loader.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000374
Barry Warsaw997b8c12018-02-16 10:45:39 -0500375 Loaders that wish to support resource reading should implement a
376 ``get_resource_reader(fullname)`` method as specified by
377 :class:`importlib.abc.ResourceReader`.
Brett Cannonbca42182018-01-12 15:08:59 -0800378
379 .. versionchanged:: 3.7
380 Introduced the optional ``get_resource_reader()`` method.
381
Eric Snowca2d8542013-12-16 23:06:52 -0700382 .. method:: create_module(spec)
383
Brett Cannon02d84542015-01-09 11:39:21 -0500384 A method that returns the module object to use when
385 importing a module. This method may return ``None``,
386 indicating that default module creation semantics should take place.
Eric Snowca2d8542013-12-16 23:06:52 -0700387
388 .. versionadded:: 3.4
389
Brett Cannon02d84542015-01-09 11:39:21 -0500390 .. versionchanged:: 3.5
391 Starting in Python 3.6, this method will not be optional when
392 :meth:`exec_module` is defined.
393
Eric Snowca2d8542013-12-16 23:06:52 -0700394 .. method:: exec_module(module)
395
396 An abstract method that executes the module in its own namespace
397 when a module is imported or reloaded. The module should already
Brett Cannon696c35e2016-06-25 10:58:17 -0700398 be initialized when ``exec_module()`` is called. When this method exists,
399 :meth:`~importlib.abc.Loader.create_module` must be defined.
Eric Snowca2d8542013-12-16 23:06:52 -0700400
401 .. versionadded:: 3.4
402
Brett Cannon696c35e2016-06-25 10:58:17 -0700403 .. versionchanged:: 3.6
404 :meth:`~importlib.abc.Loader.create_module` must also be defined.
405
Brett Cannon9c751b72009-03-09 16:28:16 +0000406 .. method:: load_module(fullname)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000407
Eric Snowca2d8542013-12-16 23:06:52 -0700408 A legacy method for loading a module. If the module cannot be
Brett Cannon2a922ed2009-03-09 03:35:50 +0000409 loaded, :exc:`ImportError` is raised, otherwise the loaded module is
410 returned.
411
Guido van Rossum09613542009-03-30 20:34:57 +0000412 If the requested module already exists in :data:`sys.modules`, that
Brett Cannon2a922ed2009-03-09 03:35:50 +0000413 module should be used and reloaded.
Guido van Rossum09613542009-03-30 20:34:57 +0000414 Otherwise the loader should create a new module and insert it into
415 :data:`sys.modules` before any loading begins, to prevent recursion
416 from the import. If the loader inserted a module and the load fails, it
Brett Cannon2a922ed2009-03-09 03:35:50 +0000417 must be removed by the loader from :data:`sys.modules`; modules already
418 in :data:`sys.modules` before the loader began execution should be left
Eric Snowb523f842013-11-22 09:05:39 -0700419 alone (see :func:`importlib.util.module_for_loader`).
Brett Cannon2a922ed2009-03-09 03:35:50 +0000420
Guido van Rossum09613542009-03-30 20:34:57 +0000421 The loader should set several attributes on the module.
422 (Note that some of these attributes can change when a module is
Eric Snowb523f842013-11-22 09:05:39 -0700423 reloaded):
Brett Cannon2a922ed2009-03-09 03:35:50 +0000424
425 - :attr:`__name__`
426 The name of the module.
427
428 - :attr:`__file__`
429 The path to where the module data is stored (not set for built-in
430 modules).
431
Brett Cannon2cefb3c2013-05-25 11:26:11 -0400432 - :attr:`__cached__`
433 The path to where a compiled version of the module is/should be
434 stored (not set when the attribute would be inappropriate).
435
Brett Cannon2a922ed2009-03-09 03:35:50 +0000436 - :attr:`__path__`
Guido van Rossum09613542009-03-30 20:34:57 +0000437 A list of strings specifying the search path within a
Brett Cannon2a922ed2009-03-09 03:35:50 +0000438 package. This attribute is not set on modules.
439
440 - :attr:`__package__`
441 The parent package for the module/package. If the module is
442 top-level then it has a value of the empty string. The
Brett Cannon100883f2013-04-09 16:59:39 -0400443 :func:`importlib.util.module_for_loader` decorator can handle the
444 details for :attr:`__package__`.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000445
446 - :attr:`__loader__`
Brett Cannon100883f2013-04-09 16:59:39 -0400447 The loader used to load the module. The
448 :func:`importlib.util.module_for_loader` decorator can handle the
449 details for :attr:`__package__`.
450
Brett Cannon8d942292014-01-07 15:52:42 -0500451 When :meth:`exec_module` is available then backwards-compatible
452 functionality is provided.
453
Brett Cannon100883f2013-04-09 16:59:39 -0400454 .. versionchanged:: 3.4
455 Raise :exc:`ImportError` when called instead of
Brett Cannon8d942292014-01-07 15:52:42 -0500456 :exc:`NotImplementedError`. Functionality provided when
457 :meth:`exec_module` is available.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000458
Eric Snowca2d8542013-12-16 23:06:52 -0700459 .. deprecated:: 3.4
460 The recommended API for loading a module is :meth:`exec_module`
Brett Cannon02d84542015-01-09 11:39:21 -0500461 (and :meth:`create_module`). Loaders should implement
Eric Snowca2d8542013-12-16 23:06:52 -0700462 it instead of load_module(). The import machinery takes care of
463 all the other responsibilities of load_module() when exec_module()
464 is implemented.
465
Barry Warsawd7d21942012-07-29 16:36:17 -0400466 .. method:: module_repr(module)
467
Eric Snowca2d8542013-12-16 23:06:52 -0700468 A legacy method which when implemented calculates and returns the
Brett Cannon100883f2013-04-09 16:59:39 -0400469 given module's repr, as a string. The module type's default repr() will
470 use the result of this method as appropriate.
Barry Warsawd7d21942012-07-29 16:36:17 -0400471
Georg Brandl526575d2013-04-11 16:10:13 +0200472 .. versionadded:: 3.3
Barry Warsawd7d21942012-07-29 16:36:17 -0400473
Brett Cannon100883f2013-04-09 16:59:39 -0400474 .. versionchanged:: 3.4
Georg Brandl526575d2013-04-11 16:10:13 +0200475 Made optional instead of an abstractmethod.
Brett Cannon100883f2013-04-09 16:59:39 -0400476
Eric Snowca2d8542013-12-16 23:06:52 -0700477 .. deprecated:: 3.4
478 The import machinery now takes care of this automatically.
479
Brett Cannon2a922ed2009-03-09 03:35:50 +0000480
Brett Cannon4ac51502017-12-15 16:29:35 -0800481.. class:: ResourceReader
482
Jason R. Coombs7f7e7062020-05-08 19:20:26 -0400483 *Superseded by TraversableReader*
484
Brett Cannonbca42182018-01-12 15:08:59 -0800485 An :term:`abstract base class` to provide the ability to read
Brett Cannon4ac51502017-12-15 16:29:35 -0800486 *resources*.
487
488 From the perspective of this ABC, a *resource* is a binary
489 artifact that is shipped within a package. Typically this is
490 something like a data file that lives next to the ``__init__.py``
491 file of the package. The purpose of this class is to help abstract
492 out the accessing of such data files so that it does not matter if
493 the package and its data file(s) are stored in a e.g. zip file
494 versus on the file system.
495
496 For any of methods of this class, a *resource* argument is
Barry Warsawdeae6b42017-12-30 15:18:06 -0500497 expected to be a :term:`path-like object` which represents
Brett Cannon4ac51502017-12-15 16:29:35 -0800498 conceptually just a file name. This means that no subdirectory
499 paths should be included in the *resource* argument. This is
Brett Cannonbca42182018-01-12 15:08:59 -0800500 because the location of the package the reader is for, acts as the
501 "directory". Hence the metaphor for directories and file
Brett Cannon4ac51502017-12-15 16:29:35 -0800502 names is packages and resources, respectively. This is also why
503 instances of this class are expected to directly correlate to
504 a specific package (instead of potentially representing multiple
505 packages or a module).
506
Brett Cannonbca42182018-01-12 15:08:59 -0800507 Loaders that wish to support resource reading are expected to
aldwinaldwinb607d992019-07-04 08:58:45 +0800508 provide a method called ``get_resource_reader(fullname)`` which
Brett Cannonbca42182018-01-12 15:08:59 -0800509 returns an object implementing this ABC's interface. If the module
510 specified by fullname is not a package, this method should return
511 :const:`None`. An object compatible with this ABC should only be
512 returned when the specified module is a package.
513
Brett Cannon4ac51502017-12-15 16:29:35 -0800514 .. versionadded:: 3.7
515
516 .. abstractmethod:: open_resource(resource)
517
518 Returns an opened, :term:`file-like object` for binary reading
519 of the *resource*.
520
521 If the resource cannot be found, :exc:`FileNotFoundError` is
522 raised.
523
524 .. abstractmethod:: resource_path(resource)
525
526 Returns the file system path to the *resource*.
527
528 If the resource does not concretely exist on the file system,
529 raise :exc:`FileNotFoundError`.
530
531 .. abstractmethod:: is_resource(name)
532
533 Returns ``True`` if the named *name* is considered a resource.
534 :exc:`FileNotFoundError` is raised if *name* does not exist.
535
536 .. abstractmethod:: contents()
537
Brett Cannon3ab93652018-04-30 11:31:45 -0700538 Returns an :term:`iterable` of strings over the contents of
Brett Cannon4ac51502017-12-15 16:29:35 -0800539 the package. Do note that it is not required that all names
540 returned by the iterator be actual resources, e.g. it is
541 acceptable to return names for which :meth:`is_resource` would
542 be false.
543
544 Allowing non-resource names to be returned is to allow for
545 situations where how a package and its resources are stored
546 are known a priori and the non-resource names would be useful.
547 For instance, returning subdirectory names is allowed so that
548 when it is known that the package and resources are stored on
Brett Cannonbca42182018-01-12 15:08:59 -0800549 the file system then those subdirectory names can be used
550 directly.
Brett Cannon4ac51502017-12-15 16:29:35 -0800551
Brett Cannon3ab93652018-04-30 11:31:45 -0700552 The abstract method returns an iterable of no items.
Brett Cannon4ac51502017-12-15 16:29:35 -0800553
554
Brett Cannon2a922ed2009-03-09 03:35:50 +0000555.. class:: ResourceLoader
556
557 An abstract base class for a :term:`loader` which implements the optional
558 :pep:`302` protocol for loading arbitrary resources from the storage
559 back-end.
560
Brett Cannonbca42182018-01-12 15:08:59 -0800561 .. deprecated:: 3.7
562 This ABC is deprecated in favour of supporting resource loading
563 through :class:`importlib.abc.ResourceReader`.
564
Berker Peksag6e9d2e62015-12-08 12:14:50 +0200565 .. abstractmethod:: get_data(path)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000566
567 An abstract method to return the bytes for the data located at *path*.
Guido van Rossum09613542009-03-30 20:34:57 +0000568 Loaders that have a file-like storage back-end
Brett Cannon16248a42009-04-01 20:47:14 +0000569 that allows storing arbitrary data
Guido van Rossum09613542009-03-30 20:34:57 +0000570 can implement this abstract method to give direct access
Andrew Svetlov08af0002014-04-01 01:13:30 +0300571 to the data stored. :exc:`OSError` is to be raised if the *path* cannot
Brett Cannon2a922ed2009-03-09 03:35:50 +0000572 be found. The *path* is expected to be constructed using a module's
Brett Cannon16248a42009-04-01 20:47:14 +0000573 :attr:`__file__` attribute or an item from a package's :attr:`__path__`.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000574
Brett Cannon100883f2013-04-09 16:59:39 -0400575 .. versionchanged:: 3.4
Andrew Svetlov08af0002014-04-01 01:13:30 +0300576 Raises :exc:`OSError` instead of :exc:`NotImplementedError`.
Brett Cannon100883f2013-04-09 16:59:39 -0400577
Brett Cannon2a922ed2009-03-09 03:35:50 +0000578
579.. class:: InspectLoader
580
581 An abstract base class for a :term:`loader` which implements the optional
Guido van Rossum09613542009-03-30 20:34:57 +0000582 :pep:`302` protocol for loaders that inspect modules.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000583
Brett Cannona113ac52009-03-15 01:41:33 +0000584 .. method:: get_code(fullname)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000585
R David Murray0ae7ae12014-01-08 18:16:02 -0500586 Return the code object for a module, or ``None`` if the module does not
587 have a code object (as would be the case, for example, for a built-in
588 module). Raise an :exc:`ImportError` if loader cannot find the
589 requested module.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000590
Brett Cannon3b62ca82013-05-27 21:11:04 -0400591 .. note::
592 While the method has a default implementation, it is suggested that
593 it be overridden if possible for performance.
594
R David Murray1b00f252012-08-15 10:43:58 -0400595 .. index::
596 single: universal newlines; importlib.abc.InspectLoader.get_source method
597
Brett Cannon100883f2013-04-09 16:59:39 -0400598 .. versionchanged:: 3.4
Brett Cannon3b62ca82013-05-27 21:11:04 -0400599 No longer abstract and a concrete implementation is provided.
Brett Cannon100883f2013-04-09 16:59:39 -0400600
Berker Peksag6e9d2e62015-12-08 12:14:50 +0200601 .. abstractmethod:: get_source(fullname)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000602
603 An abstract method to return the source of a module. It is returned as
R David Murray1b00f252012-08-15 10:43:58 -0400604 a text string using :term:`universal newlines`, translating all
R David Murrayee0a9452012-08-15 11:05:36 -0400605 recognized line separators into ``'\n'`` characters. Returns ``None``
606 if no source is available (e.g. a built-in module). Raises
607 :exc:`ImportError` if the loader cannot find the module specified.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000608
Brett Cannon100883f2013-04-09 16:59:39 -0400609 .. versionchanged:: 3.4
610 Raises :exc:`ImportError` instead of :exc:`NotImplementedError`.
611
Brett Cannona113ac52009-03-15 01:41:33 +0000612 .. method:: is_package(fullname)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000613
Brett Cannona113ac52009-03-15 01:41:33 +0000614 An abstract method to return a true value if the module is a package, a
615 false value otherwise. :exc:`ImportError` is raised if the
616 :term:`loader` cannot find the module.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000617
Brett Cannon100883f2013-04-09 16:59:39 -0400618 .. versionchanged:: 3.4
619 Raises :exc:`ImportError` instead of :exc:`NotImplementedError`.
620
Brett Cannon6eaac132014-05-09 12:28:22 -0400621 .. staticmethod:: source_to_code(data, path='<string>')
Brett Cannon9ffe85e2013-05-26 16:45:10 -0400622
623 Create a code object from Python source.
624
625 The *data* argument can be whatever the :func:`compile` function
626 supports (i.e. string or bytes). The *path* argument should be
627 the "path" to where the source code originated from, which can be an
628 abstract concept (e.g. location in a zip file).
629
Brett Cannon6eaac132014-05-09 12:28:22 -0400630 With the subsequent code object one can execute it in a module by
631 running ``exec(code, module.__dict__)``.
632
Brett Cannon9ffe85e2013-05-26 16:45:10 -0400633 .. versionadded:: 3.4
634
Brett Cannon6eaac132014-05-09 12:28:22 -0400635 .. versionchanged:: 3.5
636 Made the method static.
637
Eric Snowca2d8542013-12-16 23:06:52 -0700638 .. method:: exec_module(module)
639
640 Implementation of :meth:`Loader.exec_module`.
641
642 .. versionadded:: 3.4
643
Brett Cannon0dbb4c72013-05-31 18:56:47 -0400644 .. method:: load_module(fullname)
645
Eric Snowca2d8542013-12-16 23:06:52 -0700646 Implementation of :meth:`Loader.load_module`.
647
648 .. deprecated:: 3.4
649 use :meth:`exec_module` instead.
Brett Cannon0dbb4c72013-05-31 18:56:47 -0400650
Brett Cannon2a922ed2009-03-09 03:35:50 +0000651
Brett Cannon69194272009-07-20 04:23:48 +0000652.. class:: ExecutionLoader
653
654 An abstract base class which inherits from :class:`InspectLoader` that,
Brett Cannon23460292009-07-20 22:59:00 +0000655 when implemented, helps a module to be executed as a script. The ABC
Brett Cannon69194272009-07-20 04:23:48 +0000656 represents an optional :pep:`302` protocol.
657
Berker Peksag6e9d2e62015-12-08 12:14:50 +0200658 .. abstractmethod:: get_filename(fullname)
Brett Cannon69194272009-07-20 04:23:48 +0000659
Brett Cannonf23e3742010-06-27 23:57:46 +0000660 An abstract method that is to return the value of :attr:`__file__` for
Brett Cannon69194272009-07-20 04:23:48 +0000661 the specified module. If no path is available, :exc:`ImportError` is
662 raised.
663
Brett Cannonf23e3742010-06-27 23:57:46 +0000664 If source code is available, then the method should return the path to
665 the source file, regardless of whether a bytecode was used to load the
666 module.
667
Brett Cannon100883f2013-04-09 16:59:39 -0400668 .. versionchanged:: 3.4
669 Raises :exc:`ImportError` instead of :exc:`NotImplementedError`.
670
Brett Cannonf23e3742010-06-27 23:57:46 +0000671
Brett Cannon938d44d2012-04-22 19:58:33 -0400672.. class:: FileLoader(fullname, path)
673
674 An abstract base class which inherits from :class:`ResourceLoader` and
Andrew Svetlova60de4f2013-02-17 16:55:58 +0200675 :class:`ExecutionLoader`, providing concrete implementations of
Brett Cannon938d44d2012-04-22 19:58:33 -0400676 :meth:`ResourceLoader.get_data` and :meth:`ExecutionLoader.get_filename`.
677
678 The *fullname* argument is a fully resolved name of the module the loader is
679 to handle. The *path* argument is the path to the file for the module.
680
681 .. versionadded:: 3.3
682
683 .. attribute:: name
684
685 The name of the module the loader can handle.
686
687 .. attribute:: path
688
689 Path to the file of the module.
690
Barry Warsawd7d21942012-07-29 16:36:17 -0400691 .. method:: load_module(fullname)
Brett Cannonc0499522012-05-11 14:48:41 -0400692
Barry Warsawd7d21942012-07-29 16:36:17 -0400693 Calls super's ``load_module()``.
Brett Cannonc0499522012-05-11 14:48:41 -0400694
Eric Snowca2d8542013-12-16 23:06:52 -0700695 .. deprecated:: 3.4
696 Use :meth:`Loader.exec_module` instead.
697
Berker Peksag6e9d2e62015-12-08 12:14:50 +0200698 .. abstractmethod:: get_filename(fullname)
Brett Cannon938d44d2012-04-22 19:58:33 -0400699
Barry Warsawd7d21942012-07-29 16:36:17 -0400700 Returns :attr:`path`.
Brett Cannon938d44d2012-04-22 19:58:33 -0400701
Berker Peksag6e9d2e62015-12-08 12:14:50 +0200702 .. abstractmethod:: get_data(path)
Brett Cannon938d44d2012-04-22 19:58:33 -0400703
Brett Cannon3b62ca82013-05-27 21:11:04 -0400704 Reads *path* as a binary file and returns the bytes from it.
Brett Cannon938d44d2012-04-22 19:58:33 -0400705
706
Brett Cannonf23e3742010-06-27 23:57:46 +0000707.. class:: SourceLoader
708
709 An abstract base class for implementing source (and optionally bytecode)
710 file loading. The class inherits from both :class:`ResourceLoader` and
711 :class:`ExecutionLoader`, requiring the implementation of:
712
713 * :meth:`ResourceLoader.get_data`
714 * :meth:`ExecutionLoader.get_filename`
Brett Cannon6dfbff32010-07-21 09:48:31 +0000715 Should only return the path to the source file; sourceless
Brett Cannona81d5272013-06-16 19:17:12 -0400716 loading is not supported.
Brett Cannonf23e3742010-06-27 23:57:46 +0000717
718 The abstract methods defined by this class are to add optional bytecode
Brett Cannon5650e4f2012-11-18 10:03:31 -0500719 file support. Not implementing these optional methods (or causing them to
720 raise :exc:`NotImplementedError`) causes the loader to
Brett Cannonf23e3742010-06-27 23:57:46 +0000721 only work with source code. Implementing the methods allows the loader to
722 work with source *and* bytecode files; it does not allow for *sourceless*
723 loading where only bytecode is provided. Bytecode files are an
724 optimization to speed up loading by removing the parsing step of Python's
725 compiler, and so no bytecode-specific API is exposed.
726
Brett Cannon773468f2012-08-02 17:35:34 -0400727 .. method:: path_stats(path)
Antoine Pitrou5136ac02012-01-13 18:52:16 +0100728
729 Optional abstract method which returns a :class:`dict` containing
Martin Pantereb995702016-07-28 01:11:04 +0000730 metadata about the specified path. Supported dictionary keys are:
Antoine Pitrou5136ac02012-01-13 18:52:16 +0100731
732 - ``'mtime'`` (mandatory): an integer or floating-point number
733 representing the modification time of the source code;
734 - ``'size'`` (optional): the size in bytes of the source code.
735
736 Any other keys in the dictionary are ignored, to allow for future
Andrew Svetlov08af0002014-04-01 01:13:30 +0300737 extensions. If the path cannot be handled, :exc:`OSError` is raised.
Antoine Pitrou5136ac02012-01-13 18:52:16 +0100738
739 .. versionadded:: 3.3
740
Brett Cannon100883f2013-04-09 16:59:39 -0400741 .. versionchanged:: 3.4
Andrew Svetlov08af0002014-04-01 01:13:30 +0300742 Raise :exc:`OSError` instead of :exc:`NotImplementedError`.
Brett Cannon100883f2013-04-09 16:59:39 -0400743
Brett Cannon773468f2012-08-02 17:35:34 -0400744 .. method:: path_mtime(path)
Brett Cannonf23e3742010-06-27 23:57:46 +0000745
746 Optional abstract method which returns the modification time for the
747 specified path.
748
Antoine Pitrou5136ac02012-01-13 18:52:16 +0100749 .. deprecated:: 3.3
750 This method is deprecated in favour of :meth:`path_stats`. You don't
751 have to implement it, but it is still available for compatibility
Andrew Svetlov08af0002014-04-01 01:13:30 +0300752 purposes. Raise :exc:`OSError` if the path cannot be handled.
Brett Cannon100883f2013-04-09 16:59:39 -0400753
Georg Brandldf48b972014-03-24 09:06:18 +0100754 .. versionchanged:: 3.4
Andrew Svetlov08af0002014-04-01 01:13:30 +0300755 Raise :exc:`OSError` instead of :exc:`NotImplementedError`.
Antoine Pitrou5136ac02012-01-13 18:52:16 +0100756
Brett Cannon773468f2012-08-02 17:35:34 -0400757 .. method:: set_data(path, data)
Brett Cannonf23e3742010-06-27 23:57:46 +0000758
759 Optional abstract method which writes the specified bytes to a file
Brett Cannon61b14252010-07-03 21:48:25 +0000760 path. Any intermediate directories which do not exist are to be created
761 automatically.
762
763 When writing to the path fails because the path is read-only
Brett Cannon2cefb3c2013-05-25 11:26:11 -0400764 (:attr:`errno.EACCES`/:exc:`PermissionError`), do not propagate the
765 exception.
Brett Cannonf23e3742010-06-27 23:57:46 +0000766
Brett Cannon100883f2013-04-09 16:59:39 -0400767 .. versionchanged:: 3.4
768 No longer raises :exc:`NotImplementedError` when called.
769
Brett Cannon773468f2012-08-02 17:35:34 -0400770 .. method:: get_code(fullname)
Brett Cannonf23e3742010-06-27 23:57:46 +0000771
772 Concrete implementation of :meth:`InspectLoader.get_code`.
773
Eric Snowca2d8542013-12-16 23:06:52 -0700774 .. method:: exec_module(module)
775
776 Concrete implementation of :meth:`Loader.exec_module`.
777
Andrés Delfino65b5ef02018-08-12 03:50:46 -0300778 .. versionadded:: 3.4
Eric Snowca2d8542013-12-16 23:06:52 -0700779
Brett Cannon773468f2012-08-02 17:35:34 -0400780 .. method:: load_module(fullname)
Brett Cannonf23e3742010-06-27 23:57:46 +0000781
Eric Snowca2d8542013-12-16 23:06:52 -0700782 Concrete implementation of :meth:`Loader.load_module`.
783
784 .. deprecated:: 3.4
785 Use :meth:`exec_module` instead.
Brett Cannonf23e3742010-06-27 23:57:46 +0000786
Brett Cannon773468f2012-08-02 17:35:34 -0400787 .. method:: get_source(fullname)
Brett Cannonf23e3742010-06-27 23:57:46 +0000788
789 Concrete implementation of :meth:`InspectLoader.get_source`.
790
Brett Cannon773468f2012-08-02 17:35:34 -0400791 .. method:: is_package(fullname)
Brett Cannonf23e3742010-06-27 23:57:46 +0000792
793 Concrete implementation of :meth:`InspectLoader.is_package`. A module
Brett Cannonea0b8232012-06-15 20:00:53 -0400794 is determined to be a package if its file path (as provided by
795 :meth:`ExecutionLoader.get_filename`) is a file named
796 ``__init__`` when the file extension is removed **and** the module name
797 itself does not end in ``__init__``.
Brett Cannonf23e3742010-06-27 23:57:46 +0000798
Brett Cannon69194272009-07-20 04:23:48 +0000799
Jason R. Coombs7f7e7062020-05-08 19:20:26 -0400800.. class:: Traversable
801
802 An object with a subset of pathlib.Path methods suitable for
803 traversing directories and opening files.
804
805 .. versionadded:: 3.9
806
807
808.. class:: TraversableReader
809
810 An abstract base class for resource readers capable of serving
811 the ``files`` interface. Subclasses ResourceReader and provides
812 concrete implementations of the ResourceReader's abstract
813 methods. Therefore, any loader supplying TraversableReader
814 also supplies ResourceReader.
815
816 Loaders that wish to support resource reading are expected to
817 implement this interface.
818
819 .. versionadded:: 3.9
820
821
Barry Warsawdeae6b42017-12-30 15:18:06 -0500822:mod:`importlib.resources` -- Resources
823---------------------------------------
824
825.. module:: importlib.resources
826 :synopsis: Package resource reading, opening, and access
827
828**Source code:** :source:`Lib/importlib/resources.py`
829
830--------------
831
832.. versionadded:: 3.7
833
834This module leverages Python's import system to provide access to *resources*
835within *packages*. If you can import a package, you can access resources
836within that package. Resources can be opened or read, in either binary or
837text mode.
838
839Resources are roughly akin to files inside directories, though it's important
840to keep in mind that this is just a metaphor. Resources and packages **do
841not** have to exist as physical files and directories on the file system.
842
Barry Warsaw997b8c12018-02-16 10:45:39 -0500843.. note::
844
845 This module provides functionality similar to `pkg_resources
846 <https://setuptools.readthedocs.io/en/latest/pkg_resources.html>`_ `Basic
847 Resource Access
848 <http://setuptools.readthedocs.io/en/latest/pkg_resources.html#basic-resource-access>`_
849 without the performance overhead of that package. This makes reading
850 resources included in packages easier, with more stable and consistent
851 semantics.
852
853 The standalone backport of this module provides more information
854 on `using importlib.resources
855 <http://importlib-resources.readthedocs.io/en/latest/using.html>`_ and
856 `migrating from pkg_resources to importlib.resources
857 <http://importlib-resources.readthedocs.io/en/latest/migration.html>`_.
858
859Loaders that wish to support resource reading should implement a
860``get_resource_reader(fullname)`` method as specified by
861:class:`importlib.abc.ResourceReader`.
Barry Warsawdeae6b42017-12-30 15:18:06 -0500862
863The following types are defined.
864
865.. data:: Package
866
867 The ``Package`` type is defined as ``Union[str, ModuleType]``. This means
868 that where the function describes accepting a ``Package``, you can pass in
869 either a string or a module. Module objects must have a resolvable
870 ``__spec__.submodule_search_locations`` that is not ``None``.
871
872.. data:: Resource
873
874 This type describes the resource names passed into the various functions
875 in this package. This is defined as ``Union[str, os.PathLike]``.
876
877
878The following functions are available.
879
Jason R. Coombs7f7e7062020-05-08 19:20:26 -0400880
881.. function:: files(package)
882
883 Returns an :class:`importlib.resources.abc.Traversable` object
884 representing the resource container for the package (think directory)
885 and its resources (think files). A Traversable may contain other
886 containers (think subdirectories).
887
888 *package* is either a name or a module object which conforms to the
889 ``Package`` requirements.
890
891 .. versionadded:: 3.9
892
Barry Warsawdeae6b42017-12-30 15:18:06 -0500893.. function:: open_binary(package, resource)
894
895 Open for binary reading the *resource* within *package*.
896
897 *package* is either a name or a module object which conforms to the
898 ``Package`` requirements. *resource* is the name of the resource to open
899 within *package*; it may not contain path separators and it may not have
900 sub-resources (i.e. it cannot be a directory). This function returns a
901 ``typing.BinaryIO`` instance, a binary I/O stream open for reading.
902
903
904.. function:: open_text(package, resource, encoding='utf-8', errors='strict')
905
906 Open for text reading the *resource* within *package*. By default, the
907 resource is opened for reading as UTF-8.
908
909 *package* is either a name or a module object which conforms to the
910 ``Package`` requirements. *resource* is the name of the resource to open
911 within *package*; it may not contain path separators and it may not have
912 sub-resources (i.e. it cannot be a directory). *encoding* and *errors*
913 have the same meaning as with built-in :func:`open`.
914
915 This function returns a ``typing.TextIO`` instance, a text I/O stream open
916 for reading.
917
918
919.. function:: read_binary(package, resource)
920
921 Read and return the contents of the *resource* within *package* as
922 ``bytes``.
923
924 *package* is either a name or a module object which conforms to the
925 ``Package`` requirements. *resource* is the name of the resource to open
926 within *package*; it may not contain path separators and it may not have
927 sub-resources (i.e. it cannot be a directory). This function returns the
928 contents of the resource as :class:`bytes`.
929
930
931.. function:: read_text(package, resource, encoding='utf-8', errors='strict')
932
933 Read and return the contents of *resource* within *package* as a ``str``.
934 By default, the contents are read as strict UTF-8.
935
936 *package* is either a name or a module object which conforms to the
937 ``Package`` requirements. *resource* is the name of the resource to open
938 within *package*; it may not contain path separators and it may not have
939 sub-resources (i.e. it cannot be a directory). *encoding* and *errors*
940 have the same meaning as with built-in :func:`open`. This function
941 returns the contents of the resource as :class:`str`.
942
943
944.. function:: path(package, resource)
945
946 Return the path to the *resource* as an actual file system path. This
947 function returns a context manager for use in a :keyword:`with` statement.
948 The context manager provides a :class:`pathlib.Path` object.
949
950 Exiting the context manager cleans up any temporary file created when the
951 resource needs to be extracted from e.g. a zip file.
952
953 *package* is either a name or a module object which conforms to the
954 ``Package`` requirements. *resource* is the name of the resource to open
955 within *package*; it may not contain path separators and it may not have
956 sub-resources (i.e. it cannot be a directory).
957
958
959.. function:: is_resource(package, name)
960
961 Return ``True`` if there is a resource named *name* in the package,
962 otherwise ``False``. Remember that directories are *not* resources!
963 *package* is either a name or a module object which conforms to the
964 ``Package`` requirements.
965
966
967.. function:: contents(package)
968
Brett Cannon3ab93652018-04-30 11:31:45 -0700969 Return an iterable over the named items within the package. The iterable
Barry Warsawdeae6b42017-12-30 15:18:06 -0500970 returns :class:`str` resources (e.g. files) and non-resources
Brett Cannon3ab93652018-04-30 11:31:45 -0700971 (e.g. directories). The iterable does not recurse into subdirectories.
Barry Warsawdeae6b42017-12-30 15:18:06 -0500972
973 *package* is either a name or a module object which conforms to the
974 ``Package`` requirements.
975
976
Brett Cannon78246b62009-01-25 04:56:30 +0000977:mod:`importlib.machinery` -- Importers and path hooks
978------------------------------------------------------
979
980.. module:: importlib.machinery
981 :synopsis: Importers and path hooks
982
Terry Jan Reedydcb6c882016-06-22 22:46:34 -0400983**Source code:** :source:`Lib/importlib/machinery.py`
984
985--------------
986
Brett Cannon78246b62009-01-25 04:56:30 +0000987This module contains the various objects that help :keyword:`import`
988find and load modules.
989
Brett Cannoncb66eb02012-05-11 12:58:42 -0400990.. attribute:: SOURCE_SUFFIXES
991
992 A list of strings representing the recognized file suffixes for source
993 modules.
994
995 .. versionadded:: 3.3
996
997.. attribute:: DEBUG_BYTECODE_SUFFIXES
998
999 A list of strings representing the file suffixes for non-optimized bytecode
1000 modules.
1001
1002 .. versionadded:: 3.3
1003
Brett Cannonf299abd2015-04-13 14:21:02 -04001004 .. deprecated:: 3.5
1005 Use :attr:`BYTECODE_SUFFIXES` instead.
1006
Brett Cannoncb66eb02012-05-11 12:58:42 -04001007.. attribute:: OPTIMIZED_BYTECODE_SUFFIXES
1008
1009 A list of strings representing the file suffixes for optimized bytecode
1010 modules.
1011
1012 .. versionadded:: 3.3
1013
Brett Cannonf299abd2015-04-13 14:21:02 -04001014 .. deprecated:: 3.5
1015 Use :attr:`BYTECODE_SUFFIXES` instead.
1016
Brett Cannoncb66eb02012-05-11 12:58:42 -04001017.. attribute:: BYTECODE_SUFFIXES
1018
1019 A list of strings representing the recognized file suffixes for bytecode
Brett Cannonf299abd2015-04-13 14:21:02 -04001020 modules (including the leading dot).
Brett Cannoncb66eb02012-05-11 12:58:42 -04001021
1022 .. versionadded:: 3.3
1023
Brett Cannonf299abd2015-04-13 14:21:02 -04001024 .. versionchanged:: 3.5
1025 The value is no longer dependent on ``__debug__``.
1026
Brett Cannoncb66eb02012-05-11 12:58:42 -04001027.. attribute:: EXTENSION_SUFFIXES
1028
Nick Coghlan76e07702012-07-18 23:14:57 +10001029 A list of strings representing the recognized file suffixes for
Brett Cannoncb66eb02012-05-11 12:58:42 -04001030 extension modules.
1031
1032 .. versionadded:: 3.3
1033
Nick Coghlanc5afd422012-07-18 23:59:08 +10001034.. function:: all_suffixes()
Nick Coghlan76e07702012-07-18 23:14:57 +10001035
1036 Returns a combined list of strings representing all file suffixes for
Nick Coghlanc5afd422012-07-18 23:59:08 +10001037 modules recognized by the standard import machinery. This is a
Nick Coghlan76e07702012-07-18 23:14:57 +10001038 helper for code which simply needs to know if a filesystem path
Nick Coghlanc5afd422012-07-18 23:59:08 +10001039 potentially refers to a module without needing any details on the kind
Martin Panterd21e0b52015-10-10 10:36:22 +00001040 of module (for example, :func:`inspect.getmodulename`).
Nick Coghlan76e07702012-07-18 23:14:57 +10001041
1042 .. versionadded:: 3.3
1043
1044
Brett Cannon78246b62009-01-25 04:56:30 +00001045.. class:: BuiltinImporter
1046
Brett Cannon2a922ed2009-03-09 03:35:50 +00001047 An :term:`importer` for built-in modules. All known built-in modules are
1048 listed in :data:`sys.builtin_module_names`. This class implements the
Nick Coghlan8a9080f2012-08-02 21:26:03 +10001049 :class:`importlib.abc.MetaPathFinder` and
1050 :class:`importlib.abc.InspectLoader` ABCs.
Brett Cannon78246b62009-01-25 04:56:30 +00001051
1052 Only class methods are defined by this class to alleviate the need for
1053 instantiation.
1054
Nick Coghland5cacbb2015-05-23 22:24:10 +10001055 .. versionchanged:: 3.5
1056 As part of :pep:`489`, the builtin importer now implements
1057 :meth:`Loader.create_module` and :meth:`Loader.exec_module`
Eric Snowca2d8542013-12-16 23:06:52 -07001058
Brett Cannon78246b62009-01-25 04:56:30 +00001059
1060.. class:: FrozenImporter
1061
Brett Cannon2a922ed2009-03-09 03:35:50 +00001062 An :term:`importer` for frozen modules. This class implements the
Nick Coghlan8a9080f2012-08-02 21:26:03 +10001063 :class:`importlib.abc.MetaPathFinder` and
1064 :class:`importlib.abc.InspectLoader` ABCs.
Brett Cannon78246b62009-01-25 04:56:30 +00001065
1066 Only class methods are defined by this class to alleviate the need for
1067 instantiation.
1068
Brett Cannon302e5a82020-03-25 11:57:47 -07001069 .. versionchanged:: 3.4
1070 Gained :meth:`~Loader.create_module` and :meth:`~Loader.exec_module`
1071 methods.
1072
Brett Cannondebb98d2009-02-16 04:18:01 +00001073
Nick Coghlanff794862012-08-02 21:45:24 +10001074.. class:: WindowsRegistryFinder
1075
1076 :term:`Finder` for modules declared in the Windows registry. This class
Himanshu Lakhara5cbb8412018-03-24 02:56:35 +05301077 implements the :class:`importlib.abc.MetaPathFinder` ABC.
Nick Coghlanff794862012-08-02 21:45:24 +10001078
1079 Only class methods are defined by this class to alleviate the need for
1080 instantiation.
1081
1082 .. versionadded:: 3.3
1083
Steve Dower20367422016-12-07 13:02:27 -08001084 .. deprecated:: 3.6
1085 Use :mod:`site` configuration instead. Future versions of Python may
1086 not enable this finder by default.
1087
Nick Coghlanff794862012-08-02 21:45:24 +10001088
Brett Cannondebb98d2009-02-16 04:18:01 +00001089.. class:: PathFinder
1090
Brett Cannon1b799182012-08-17 14:08:24 -04001091 A :term:`Finder` for :data:`sys.path` and package ``__path__`` attributes.
1092 This class implements the :class:`importlib.abc.MetaPathFinder` ABC.
Brett Cannondebb98d2009-02-16 04:18:01 +00001093
Brett Cannon1b799182012-08-17 14:08:24 -04001094 Only class methods are defined by this class to alleviate the need for
1095 instantiation.
Brett Cannondebb98d2009-02-16 04:18:01 +00001096
Eric Snowca2d8542013-12-16 23:06:52 -07001097 .. classmethod:: find_spec(fullname, path=None, target=None)
1098
1099 Class method that attempts to find a :term:`spec <module spec>`
1100 for the module specified by *fullname* on :data:`sys.path` or, if
1101 defined, on *path*. For each path entry that is searched,
1102 :data:`sys.path_importer_cache` is checked. If a non-false object
1103 is found then it is used as the :term:`path entry finder` to look
1104 for the module being searched for. If no entry is found in
1105 :data:`sys.path_importer_cache`, then :data:`sys.path_hooks` is
1106 searched for a finder for the path entry and, if found, is stored
1107 in :data:`sys.path_importer_cache` along with being queried about
1108 the module. If no finder is ever found then ``None`` is both
1109 stored in the cache and returned.
1110
1111 .. versionadded:: 3.4
1112
Brett Cannonb6e25562014-11-21 12:19:28 -05001113 .. versionchanged:: 3.5
1114 If the current working directory -- represented by an empty string --
1115 is no longer valid then ``None`` is returned but no value is cached
1116 in :data:`sys.path_importer_cache`.
1117
Brett Cannon1b799182012-08-17 14:08:24 -04001118 .. classmethod:: find_module(fullname, path=None)
Brett Cannondebb98d2009-02-16 04:18:01 +00001119
Eric Snowca2d8542013-12-16 23:06:52 -07001120 A legacy wrapper around :meth:`find_spec`.
1121
1122 .. deprecated:: 3.4
1123 Use :meth:`find_spec` instead.
Brett Cannond2e7b332009-02-17 02:45:03 +00001124
Brett Cannonf4dc9202012-08-10 12:21:12 -04001125 .. classmethod:: invalidate_caches()
1126
Eric Snowca2d8542013-12-16 23:06:52 -07001127 Calls :meth:`importlib.abc.PathEntryFinder.invalidate_caches` on all
Brett Cannon9e2be602018-04-06 16:10:18 -07001128 finders stored in :data:`sys.path_importer_cache` that define the method.
1129 Otherwise entries in :data:`sys.path_importer_cache` set to ``None`` are
1130 deleted.
1131
1132 .. versionchanged:: 3.7
1133 Entries of ``None`` in :data:`sys.path_importer_cache` are deleted.
Brett Cannonf4dc9202012-08-10 12:21:12 -04001134
Eric Snowca2d8542013-12-16 23:06:52 -07001135 .. versionchanged:: 3.4
1136 Calls objects in :data:`sys.path_hooks` with the current working
1137 directory for ``''`` (i.e. the empty string).
Brett Cannon27e27f72013-10-18 11:39:04 -04001138
Brett Cannond2e7b332009-02-17 02:45:03 +00001139
Brett Cannon938d44d2012-04-22 19:58:33 -04001140.. class:: FileFinder(path, \*loader_details)
1141
Nick Coghlan8a9080f2012-08-02 21:26:03 +10001142 A concrete implementation of :class:`importlib.abc.PathEntryFinder` which
1143 caches results from the file system.
Brett Cannon938d44d2012-04-22 19:58:33 -04001144
1145 The *path* argument is the directory for which the finder is in charge of
1146 searching.
1147
Brett Cannonac9f2f32012-08-10 13:47:54 -04001148 The *loader_details* argument is a variable number of 2-item tuples each
1149 containing a loader and a sequence of file suffixes the loader recognizes.
Brett Cannon29b2f172013-06-21 18:31:55 -04001150 The loaders are expected to be callables which accept two arguments of
1151 the module's name and the path to the file found.
Brett Cannon938d44d2012-04-22 19:58:33 -04001152
1153 The finder will cache the directory contents as necessary, making stat calls
1154 for each module search to verify the cache is not outdated. Because cache
1155 staleness relies upon the granularity of the operating system's state
1156 information of the file system, there is a potential race condition of
1157 searching for a module, creating a new file, and then searching for the
1158 module the new file represents. If the operations happen fast enough to fit
1159 within the granularity of stat calls, then the module search will fail. To
1160 prevent this from happening, when you create a module dynamically, make sure
1161 to call :func:`importlib.invalidate_caches`.
1162
1163 .. versionadded:: 3.3
1164
1165 .. attribute:: path
1166
1167 The path the finder will search in.
1168
Eric Snowca2d8542013-12-16 23:06:52 -07001169 .. method:: find_spec(fullname, target=None)
1170
1171 Attempt to find the spec to handle *fullname* within :attr:`path`.
1172
1173 .. versionadded:: 3.4
1174
Brett Cannon1d753822013-06-16 19:06:55 -04001175 .. method:: find_loader(fullname)
Brett Cannon938d44d2012-04-22 19:58:33 -04001176
1177 Attempt to find the loader to handle *fullname* within :attr:`path`.
1178
1179 .. method:: invalidate_caches()
1180
1181 Clear out the internal cache.
1182
1183 .. classmethod:: path_hook(\*loader_details)
1184
1185 A class method which returns a closure for use on :attr:`sys.path_hooks`.
1186 An instance of :class:`FileFinder` is returned by the closure using the
1187 path argument given to the closure directly and *loader_details*
1188 indirectly.
1189
1190 If the argument to the closure is not an existing directory,
1191 :exc:`ImportError` is raised.
1192
1193
1194.. class:: SourceFileLoader(fullname, path)
1195
1196 A concrete implementation of :class:`importlib.abc.SourceLoader` by
1197 subclassing :class:`importlib.abc.FileLoader` and providing some concrete
1198 implementations of other methods.
1199
1200 .. versionadded:: 3.3
1201
1202 .. attribute:: name
1203
1204 The name of the module that this loader will handle.
1205
1206 .. attribute:: path
1207
1208 The path to the source file.
1209
1210 .. method:: is_package(fullname)
1211
Serhiy Storchaka138ccbb2019-11-12 16:57:03 +02001212 Return ``True`` if :attr:`path` appears to be for a package.
Brett Cannon938d44d2012-04-22 19:58:33 -04001213
1214 .. method:: path_stats(path)
1215
1216 Concrete implementation of :meth:`importlib.abc.SourceLoader.path_stats`.
1217
1218 .. method:: set_data(path, data)
1219
1220 Concrete implementation of :meth:`importlib.abc.SourceLoader.set_data`.
1221
Brett Cannon062fcac2014-05-09 11:55:49 -04001222 .. method:: load_module(name=None)
1223
1224 Concrete implementation of :meth:`importlib.abc.Loader.load_module` where
1225 specifying the name of the module to load is optional.
1226
Brett Cannoneae30792015-12-28 17:55:27 -08001227 .. deprecated:: 3.6
1228
1229 Use :meth:`importlib.abc.Loader.exec_module` instead.
1230
Brett Cannon938d44d2012-04-22 19:58:33 -04001231
Marc-Andre Lemburg4fe29c92012-04-25 02:31:37 +02001232.. class:: SourcelessFileLoader(fullname, path)
Brett Cannon938d44d2012-04-22 19:58:33 -04001233
1234 A concrete implementation of :class:`importlib.abc.FileLoader` which can
1235 import bytecode files (i.e. no source code files exist).
1236
Marc-Andre Lemburg4fe29c92012-04-25 02:31:37 +02001237 Please note that direct use of bytecode files (and thus not source code
1238 files) inhibits your modules from being usable by all Python
1239 implementations or new versions of Python which change the bytecode
1240 format.
Brett Cannon938d44d2012-04-22 19:58:33 -04001241
1242 .. versionadded:: 3.3
1243
1244 .. attribute:: name
1245
1246 The name of the module the loader will handle.
1247
1248 .. attribute:: path
1249
1250 The path to the bytecode file.
1251
1252 .. method:: is_package(fullname)
1253
1254 Determines if the module is a package based on :attr:`path`.
1255
1256 .. method:: get_code(fullname)
1257
1258 Returns the code object for :attr:`name` created from :attr:`path`.
1259
1260 .. method:: get_source(fullname)
1261
1262 Returns ``None`` as bytecode files have no source when this loader is
1263 used.
1264
Brett Cannon062fcac2014-05-09 11:55:49 -04001265 .. method:: load_module(name=None)
1266
1267 Concrete implementation of :meth:`importlib.abc.Loader.load_module` where
1268 specifying the name of the module to load is optional.
1269
Brett Cannoneae30792015-12-28 17:55:27 -08001270 .. deprecated:: 3.6
1271
1272 Use :meth:`importlib.abc.Loader.exec_module` instead.
1273
Brett Cannon938d44d2012-04-22 19:58:33 -04001274
1275.. class:: ExtensionFileLoader(fullname, path)
1276
Eric Snow51794452013-10-03 12:08:55 -06001277 A concrete implementation of :class:`importlib.abc.ExecutionLoader` for
Brett Cannon938d44d2012-04-22 19:58:33 -04001278 extension modules.
1279
1280 The *fullname* argument specifies the name of the module the loader is to
1281 support. The *path* argument is the path to the extension module's file.
1282
1283 .. versionadded:: 3.3
1284
1285 .. attribute:: name
1286
1287 Name of the module the loader supports.
1288
1289 .. attribute:: path
1290
1291 Path to the extension module.
1292
Nick Coghland5cacbb2015-05-23 22:24:10 +10001293 .. method:: create_module(spec)
Brett Cannon938d44d2012-04-22 19:58:33 -04001294
Nick Coghland5cacbb2015-05-23 22:24:10 +10001295 Creates the module object from the given specification in accordance
1296 with :pep:`489`.
Brett Cannon938d44d2012-04-22 19:58:33 -04001297
Nick Coghland5cacbb2015-05-23 22:24:10 +10001298 .. versionadded:: 3.5
1299
1300 .. method:: exec_module(module)
1301
1302 Initializes the given module object in accordance with :pep:`489`.
1303
1304 .. versionadded:: 3.5
Eric Snowca2d8542013-12-16 23:06:52 -07001305
Brett Cannon938d44d2012-04-22 19:58:33 -04001306 .. method:: is_package(fullname)
1307
Brett Cannonac9f2f32012-08-10 13:47:54 -04001308 Returns ``True`` if the file path points to a package's ``__init__``
1309 module based on :attr:`EXTENSION_SUFFIXES`.
Brett Cannon938d44d2012-04-22 19:58:33 -04001310
1311 .. method:: get_code(fullname)
1312
1313 Returns ``None`` as extension modules lack a code object.
1314
1315 .. method:: get_source(fullname)
1316
1317 Returns ``None`` as extension modules do not have source code.
1318
Eric Snow51794452013-10-03 12:08:55 -06001319 .. method:: get_filename(fullname)
1320
1321 Returns :attr:`path`.
1322
Eric Snowdcd01b42013-10-04 20:35:34 -06001323 .. versionadded:: 3.4
1324
Brett Cannon938d44d2012-04-22 19:58:33 -04001325
Eric Snowb523f842013-11-22 09:05:39 -07001326.. class:: ModuleSpec(name, loader, *, origin=None, loader_state=None, is_package=None)
1327
Barry Warsaw191e3132017-10-17 15:52:38 -04001328 A specification for a module's import-system-related state. This is
1329 typically exposed as the module's ``__spec__`` attribute. In the
1330 descriptions below, the names in parentheses give the corresponding
1331 attribute available directly on the module object.
1332 E.g. ``module.__spec__.origin == module.__file__``. Note however that
1333 while the *values* are usually equivalent, they can differ since there is
1334 no synchronization between the two objects. Thus it is possible to update
1335 the module's ``__path__`` at runtime, and this will not be automatically
1336 reflected in ``__spec__.submodule_search_locations``.
Eric Snowb523f842013-11-22 09:05:39 -07001337
1338 .. versionadded:: 3.4
1339
1340 .. attribute:: name
1341
1342 (``__name__``)
1343
1344 A string for the fully-qualified name of the module.
1345
1346 .. attribute:: loader
1347
1348 (``__loader__``)
1349
1350 The loader to use for loading. For namespace packages this should be
Serhiy Storchakaecf41da2016-10-19 16:29:26 +03001351 set to ``None``.
Eric Snowb523f842013-11-22 09:05:39 -07001352
1353 .. attribute:: origin
1354
1355 (``__file__``)
1356
1357 Name of the place from which the module is loaded, e.g. "builtin" for
1358 built-in modules and the filename for modules loaded from source.
Serhiy Storchakaecf41da2016-10-19 16:29:26 +03001359 Normally "origin" should be set, but it may be ``None`` (the default)
Barry Warsawbbbcf862018-02-02 15:15:58 -05001360 which indicates it is unspecified (e.g. for namespace packages).
Eric Snowb523f842013-11-22 09:05:39 -07001361
1362 .. attribute:: submodule_search_locations
1363
1364 (``__path__``)
1365
Serhiy Storchakaecf41da2016-10-19 16:29:26 +03001366 List of strings for where to find submodules, if a package (``None``
Eric Snowb523f842013-11-22 09:05:39 -07001367 otherwise).
1368
1369 .. attribute:: loader_state
1370
1371 Container of extra module-specific data for use during loading (or
Serhiy Storchakaecf41da2016-10-19 16:29:26 +03001372 ``None``).
Eric Snowb523f842013-11-22 09:05:39 -07001373
1374 .. attribute:: cached
1375
1376 (``__cached__``)
1377
Serhiy Storchakaecf41da2016-10-19 16:29:26 +03001378 String for where the compiled module should be stored (or ``None``).
Eric Snowb523f842013-11-22 09:05:39 -07001379
1380 .. attribute:: parent
1381
1382 (``__package__``)
1383
1384 (Read-only) Fully-qualified name of the package to which the module
Serhiy Storchakaecf41da2016-10-19 16:29:26 +03001385 belongs as a submodule (or ``None``).
Eric Snowb523f842013-11-22 09:05:39 -07001386
1387 .. attribute:: has_location
1388
Eric Snowb282b3d2013-12-10 22:16:41 -07001389 Boolean indicating whether or not the module's "origin"
Eric Snowb523f842013-11-22 09:05:39 -07001390 attribute refers to a loadable location.
1391
Brett Cannond2e7b332009-02-17 02:45:03 +00001392:mod:`importlib.util` -- Utility code for importers
1393---------------------------------------------------
1394
1395.. module:: importlib.util
Brett Cannon75321e82012-03-02 11:58:25 -05001396 :synopsis: Utility code for importers
Brett Cannond2e7b332009-02-17 02:45:03 +00001397
Terry Jan Reedydcb6c882016-06-22 22:46:34 -04001398
1399**Source code:** :source:`Lib/importlib/util.py`
1400
1401--------------
1402
Brett Cannond2e7b332009-02-17 02:45:03 +00001403This module contains the various objects that help in the construction of
1404an :term:`importer`.
1405
Brett Cannon05a647d2013-06-14 19:02:34 -04001406.. attribute:: MAGIC_NUMBER
1407
1408 The bytes which represent the bytecode version number. If you need help with
1409 loading/writing bytecode then consider :class:`importlib.abc.SourceLoader`.
1410
1411 .. versionadded:: 3.4
1412
Brett Cannonf299abd2015-04-13 14:21:02 -04001413.. function:: cache_from_source(path, debug_override=None, *, optimization=None)
Brett Cannona3c96152013-06-14 22:26:30 -04001414
Brett Cannonf299abd2015-04-13 14:21:02 -04001415 Return the :pep:`3147`/:pep:`488` path to the byte-compiled file associated
1416 with the source *path*. For example, if *path* is ``/foo/bar/baz.py`` the return
Brett Cannona3c96152013-06-14 22:26:30 -04001417 value would be ``/foo/bar/__pycache__/baz.cpython-32.pyc`` for Python 3.2.
1418 The ``cpython-32`` string comes from the current magic tag (see
1419 :func:`get_tag`; if :attr:`sys.implementation.cache_tag` is not defined then
Brett Cannonf299abd2015-04-13 14:21:02 -04001420 :exc:`NotImplementedError` will be raised).
Brett Cannona3c96152013-06-14 22:26:30 -04001421
Brett Cannonf299abd2015-04-13 14:21:02 -04001422 The *optimization* parameter is used to specify the optimization level of the
1423 bytecode file. An empty string represents no optimization, so
1424 ``/foo/bar/baz.py`` with an *optimization* of ``''`` will result in a
1425 bytecode path of ``/foo/bar/__pycache__/baz.cpython-32.pyc``. ``None`` causes
Min ho Kim39d87b52019-08-31 06:21:19 +10001426 the interpreter's optimization level to be used. Any other value's string
1427 representation is used, so ``/foo/bar/baz.py`` with an *optimization* of
Brett Cannonf299abd2015-04-13 14:21:02 -04001428 ``2`` will lead to the bytecode path of
1429 ``/foo/bar/__pycache__/baz.cpython-32.opt-2.pyc``. The string representation
1430 of *optimization* can only be alphanumeric, else :exc:`ValueError` is raised.
1431
1432 The *debug_override* parameter is deprecated and can be used to override
1433 the system's value for ``__debug__``. A ``True`` value is the equivalent of
1434 setting *optimization* to the empty string. A ``False`` value is the same as
1435 setting *optimization* to ``1``. If both *debug_override* an *optimization*
1436 are not ``None`` then :exc:`TypeError` is raised.
Brett Cannona3c96152013-06-14 22:26:30 -04001437
1438 .. versionadded:: 3.4
1439
Berker Peksagfe5f6142016-01-30 19:30:06 +02001440 .. versionchanged:: 3.5
Brett Cannonf299abd2015-04-13 14:21:02 -04001441 The *optimization* parameter was added and the *debug_override* parameter
1442 was deprecated.
1443
Brett Cannon035a1002016-09-07 18:39:18 -07001444 .. versionchanged:: 3.6
1445 Accepts a :term:`path-like object`.
1446
Brett Cannona3c96152013-06-14 22:26:30 -04001447
1448.. function:: source_from_cache(path)
1449
1450 Given the *path* to a :pep:`3147` file name, return the associated source code
1451 file path. For example, if *path* is
1452 ``/foo/bar/__pycache__/baz.cpython-32.pyc`` the returned path would be
1453 ``/foo/bar/baz.py``. *path* need not exist, however if it does not conform
Stéphane Wirtele483f022018-10-26 12:52:11 +02001454 to :pep:`3147` or :pep:`488` format, a :exc:`ValueError` is raised. If
Brett Cannona3c96152013-06-14 22:26:30 -04001455 :attr:`sys.implementation.cache_tag` is not defined,
1456 :exc:`NotImplementedError` is raised.
1457
1458 .. versionadded:: 3.4
1459
Brett Cannon035a1002016-09-07 18:39:18 -07001460 .. versionchanged:: 3.6
1461 Accepts a :term:`path-like object`.
1462
Brett Cannonf24fecd2013-06-16 18:37:53 -04001463.. function:: decode_source(source_bytes)
1464
1465 Decode the given bytes representing source code and return it as a string
1466 with universal newlines (as required by
1467 :meth:`importlib.abc.InspectLoader.get_source`).
1468
1469 .. versionadded:: 3.4
1470
Brett Cannond200bf52012-05-13 13:45:09 -04001471.. function:: resolve_name(name, package)
1472
1473 Resolve a relative module name to an absolute one.
1474
1475 If **name** has no leading dots, then **name** is simply returned. This
1476 allows for usage such as
1477 ``importlib.util.resolve_name('sys', __package__)`` without doing a
1478 check to see if the **package** argument is needed.
1479
Ngalim Siregarc5fa4492019-08-03 12:46:02 +07001480 :exc:`ImportError` is raised if **name** is a relative module name but
1481 **package** is a false value (e.g. ``None`` or the empty string).
1482 :exc:`ImportError` is also raised a relative name would escape its containing
Brett Cannond200bf52012-05-13 13:45:09 -04001483 package (e.g. requesting ``..bacon`` from within the ``spam`` package).
1484
1485 .. versionadded:: 3.3
1486
Ngalim Siregarc5fa4492019-08-03 12:46:02 +07001487 .. versionchanged:: 3.9
1488 To improve consistency with import statements, raise
1489 :exc:`ImportError` instead of :exc:`ValueError` for invalid relative
1490 import attempts.
1491
Eric Snow6029e082014-01-25 15:32:46 -07001492.. function:: find_spec(name, package=None)
1493
1494 Find the :term:`spec <module spec>` for a module, optionally relative to
1495 the specified **package** name. If the module is in :attr:`sys.modules`,
1496 then ``sys.modules[name].__spec__`` is returned (unless the spec would be
1497 ``None`` or is not set, in which case :exc:`ValueError` is raised).
1498 Otherwise a search using :attr:`sys.meta_path` is done. ``None`` is
1499 returned if no spec is found.
1500
1501 If **name** is for a submodule (contains a dot), the parent module is
1502 automatically imported.
1503
1504 **name** and **package** work the same as for :func:`import_module`.
1505
1506 .. versionadded:: 3.4
1507
Milan Oberkirch8c3f05e2017-06-15 07:34:50 +10001508 .. versionchanged:: 3.7
1509 Raises :exc:`ModuleNotFoundError` instead of :exc:`AttributeError` if
1510 **package** is in fact not a package (i.e. lacks a :attr:`__path__`
1511 attribute).
1512
Brett Cannon2a17bde2014-05-30 14:55:29 -04001513.. function:: module_from_spec(spec)
1514
Brett Cannon696c35e2016-06-25 10:58:17 -07001515 Create a new module based on **spec** and
1516 :meth:`spec.loader.create_module <importlib.abc.Loader.create_module>`.
Brett Cannon2a17bde2014-05-30 14:55:29 -04001517
Brett Cannon696c35e2016-06-25 10:58:17 -07001518 If :meth:`spec.loader.create_module <importlib.abc.Loader.create_module>`
1519 does not return ``None``, then any pre-existing attributes will not be reset.
1520 Also, no :exc:`AttributeError` will be raised if triggered while accessing
1521 **spec** or setting an attribute on the module.
Brett Cannon2a17bde2014-05-30 14:55:29 -04001522
1523 This function is preferred over using :class:`types.ModuleType` to create a
1524 new module as **spec** is used to set as many import-controlled attributes on
1525 the module as possible.
1526
1527 .. versionadded:: 3.5
1528
Georg Brandl8a1caa22010-07-29 16:01:11 +00001529.. decorator:: module_for_loader
Brett Cannond2e7b332009-02-17 02:45:03 +00001530
Brett Cannona22faca2013-05-28 17:50:14 -04001531 A :term:`decorator` for :meth:`importlib.abc.Loader.load_module`
Guido van Rossum09613542009-03-30 20:34:57 +00001532 to handle selecting the proper
Brett Cannond2e7b332009-02-17 02:45:03 +00001533 module object to load with. The decorated method is expected to have a call
Brett Cannon2a922ed2009-03-09 03:35:50 +00001534 signature taking two positional arguments
1535 (e.g. ``load_module(self, module)``) for which the second argument
Guido van Rossum09613542009-03-30 20:34:57 +00001536 will be the module **object** to be used by the loader.
Brett Cannonefad00d2012-04-27 17:27:14 -04001537 Note that the decorator will not work on static methods because of the
1538 assumption of two arguments.
Brett Cannond2e7b332009-02-17 02:45:03 +00001539
Guido van Rossum09613542009-03-30 20:34:57 +00001540 The decorated method will take in the **name** of the module to be loaded
1541 as expected for a :term:`loader`. If the module is not found in
Brett Cannon3dc48d62013-05-28 18:35:54 -04001542 :data:`sys.modules` then a new one is constructed. Regardless of where the
1543 module came from, :attr:`__loader__` set to **self** and :attr:`__package__`
1544 is set based on what :meth:`importlib.abc.InspectLoader.is_package` returns
1545 (if available). These attributes are set unconditionally to support
1546 reloading.
Brett Cannonefad00d2012-04-27 17:27:14 -04001547
1548 If an exception is raised by the decorated method and a module was added to
Brett Cannona87e31c2013-09-13 16:52:19 -04001549 :data:`sys.modules`, then the module will be removed to prevent a partially
1550 initialized module from being in left in :data:`sys.modules`. If the module
1551 was already in :data:`sys.modules` then it is left alone.
Brett Cannond2e7b332009-02-17 02:45:03 +00001552
Brett Cannonefad00d2012-04-27 17:27:14 -04001553 .. versionchanged:: 3.3
Georg Brandl61063cc2012-06-24 22:48:30 +02001554 :attr:`__loader__` and :attr:`__package__` are automatically set
1555 (when possible).
Brett Cannon57b46f52009-03-02 14:38:26 +00001556
Brett Cannon3dc48d62013-05-28 18:35:54 -04001557 .. versionchanged:: 3.4
Brett Cannon0dbb4c72013-05-31 18:56:47 -04001558 Set :attr:`__name__`, :attr:`__loader__` :attr:`__package__`
1559 unconditionally to support reloading.
1560
1561 .. deprecated:: 3.4
Eric Snowb523f842013-11-22 09:05:39 -07001562 The import machinery now directly performs all the functionality
1563 provided by this function.
Brett Cannon3dc48d62013-05-28 18:35:54 -04001564
Georg Brandl8a1caa22010-07-29 16:01:11 +00001565.. decorator:: set_loader
Brett Cannon2cf03a82009-03-10 05:17:37 +00001566
Brett Cannona22faca2013-05-28 17:50:14 -04001567 A :term:`decorator` for :meth:`importlib.abc.Loader.load_module`
1568 to set the :attr:`__loader__`
1569 attribute on the returned module. If the attribute is already set the
1570 decorator does nothing. It is assumed that the first positional argument to
1571 the wrapped method (i.e. ``self``) is what :attr:`__loader__` should be set
1572 to.
Brett Cannon2cf03a82009-03-10 05:17:37 +00001573
Brett Cannon4802bec2013-03-13 10:41:36 -07001574 .. versionchanged:: 3.4
Brett Cannon4c14b5d2013-05-04 13:56:58 -04001575 Set ``__loader__`` if set to ``None``, as if the attribute does not
Brett Cannon4802bec2013-03-13 10:41:36 -07001576 exist.
1577
Eric Snowca2d8542013-12-16 23:06:52 -07001578 .. deprecated:: 3.4
1579 The import machinery takes care of this automatically.
1580
Georg Brandl8a1caa22010-07-29 16:01:11 +00001581.. decorator:: set_package
Brett Cannon57b46f52009-03-02 14:38:26 +00001582
Brett Cannon696c35e2016-06-25 10:58:17 -07001583 A :term:`decorator` for :meth:`importlib.abc.Loader.load_module` to set the
1584 :attr:`__package__` attribute on the returned module. If :attr:`__package__`
Brett Cannona22faca2013-05-28 17:50:14 -04001585 is set and has a value other than ``None`` it will not be changed.
Brett Cannon16248a42009-04-01 20:47:14 +00001586
Eric Snowca2d8542013-12-16 23:06:52 -07001587 .. deprecated:: 3.4
1588 The import machinery takes care of this automatically.
1589
Eric Snowb523f842013-11-22 09:05:39 -07001590.. function:: spec_from_loader(name, loader, *, origin=None, is_package=None)
1591
1592 A factory function for creating a :class:`ModuleSpec` instance based
1593 on a loader. The parameters have the same meaning as they do for
1594 ModuleSpec. The function uses available :term:`loader` APIs, such as
1595 :meth:`InspectLoader.is_package`, to fill in any missing
1596 information on the spec.
1597
1598 .. versionadded:: 3.4
1599
1600.. function:: spec_from_file_location(name, location, *, loader=None, submodule_search_locations=None)
1601
1602 A factory function for creating a :class:`ModuleSpec` instance based
1603 on the path to a file. Missing information will be filled in on the
1604 spec by making use of loader APIs and by the implication that the
1605 module will be file-based.
1606
1607 .. versionadded:: 3.4
Brett Cannona04dbe42014-04-04 13:53:38 -04001608
Brett Cannon035a1002016-09-07 18:39:18 -07001609 .. versionchanged:: 3.6
1610 Accepts a :term:`path-like object`.
1611
Benjamin Peterson42aa93b2017-12-09 10:26:52 -08001612.. function:: source_hash(source_bytes)
1613
1614 Return the hash of *source_bytes* as bytes. A hash-based ``.pyc`` file embeds
1615 the :func:`source_hash` of the corresponding source file's contents in its
1616 header.
1617
1618 .. versionadded:: 3.7
1619
Brett Cannona04dbe42014-04-04 13:53:38 -04001620.. class:: LazyLoader(loader)
1621
1622 A class which postpones the execution of the loader of a module until the
1623 module has an attribute accessed.
1624
1625 This class **only** works with loaders that define
Brett Cannon02d84542015-01-09 11:39:21 -05001626 :meth:`~importlib.abc.Loader.exec_module` as control over what module type
1627 is used for the module is required. For those same reasons, the loader's
Brett Cannon696c35e2016-06-25 10:58:17 -07001628 :meth:`~importlib.abc.Loader.create_module` method must return ``None`` or a
1629 type for which its ``__class__`` attribute can be mutated along with not
1630 using :term:`slots <__slots__>`. Finally, modules which substitute the object
1631 placed into :attr:`sys.modules` will not work as there is no way to properly
1632 replace the module references throughout the interpreter safely;
1633 :exc:`ValueError` is raised if such a substitution is detected.
Brett Cannona04dbe42014-04-04 13:53:38 -04001634
1635 .. note::
1636 For projects where startup time is critical, this class allows for
1637 potentially minimizing the cost of loading a module if it is never used.
1638 For projects where startup time is not essential then use of this class is
1639 **heavily** discouraged due to error messages created during loading being
1640 postponed and thus occurring out of context.
1641
1642 .. versionadded:: 3.5
1643
Brett Cannon696c35e2016-06-25 10:58:17 -07001644 .. versionchanged:: 3.6
1645 Began calling :meth:`~importlib.abc.Loader.create_module`, removing the
1646 compatibility warning for :class:`importlib.machinery.BuiltinImporter` and
1647 :class:`importlib.machinery.ExtensionFileLoader`.
1648
Brett Cannona04dbe42014-04-04 13:53:38 -04001649 .. classmethod:: factory(loader)
1650
1651 A static method which returns a callable that creates a lazy loader. This
1652 is meant to be used in situations where the loader is passed by class
1653 instead of by instance.
1654 ::
1655
1656 suffixes = importlib.machinery.SOURCE_SUFFIXES
1657 loader = importlib.machinery.SourceFileLoader
1658 lazy_loader = importlib.util.LazyLoader.factory(loader)
Brett Cannon3bf1d872016-01-22 14:03:27 -08001659 finder = importlib.machinery.FileFinder(path, (lazy_loader, suffixes))
Brett Cannona85e9272016-01-08 14:33:09 -08001660
1661.. _importlib-examples:
1662
1663Examples
1664--------
1665
Brett Cannon23763162016-09-08 10:12:47 -07001666Importing programmatically
1667''''''''''''''''''''''''''
1668
Brett Cannona85e9272016-01-08 14:33:09 -08001669To programmatically import a module, use :func:`importlib.import_module`.
1670::
1671
1672 import importlib
1673
1674 itertools = importlib.import_module('itertools')
1675
Brett Cannon23763162016-09-08 10:12:47 -07001676
1677Checking if a module can be imported
1678''''''''''''''''''''''''''''''''''''
1679
Brett Cannona85e9272016-01-08 14:33:09 -08001680If you need to find out if a module can be imported without actually doing the
1681import, then you should use :func:`importlib.util.find_spec`.
1682::
1683
1684 import importlib.util
1685 import sys
1686
1687 # For illustrative purposes.
1688 name = 'itertools'
1689
Brett Cannon08270642019-07-12 15:35:34 -07001690 if name in sys.modules:
1691 print(f"{name!r} already in sys.modules")
Tzu-ping Chung544fa152019-07-26 01:20:33 +08001692 elif (spec := importlib.util.find_spec(name)) is not None:
Brett Cannon59363132016-03-18 11:54:22 -07001693 # If you chose to perform the actual import ...
Brett Cannona85e9272016-01-08 14:33:09 -08001694 module = importlib.util.module_from_spec(spec)
Brett Cannona85e9272016-01-08 14:33:09 -08001695 sys.modules[name] = module
Brett Cannon08270642019-07-12 15:35:34 -07001696 spec.loader.exec_module(module)
1697 print(f"{name!r} has been imported")
Tzu-ping Chung544fa152019-07-26 01:20:33 +08001698 else:
1699 print(f"can't find the {name!r} module")
Brett Cannona85e9272016-01-08 14:33:09 -08001700
Brett Cannon23763162016-09-08 10:12:47 -07001701
1702Importing a source file directly
1703''''''''''''''''''''''''''''''''
1704
Brett Cannona85e9272016-01-08 14:33:09 -08001705To import a Python source file directly, use the following recipe
E Kawashima16c8a532018-11-06 07:41:17 +09001706(Python 3.5 and newer only)::
Brett Cannona85e9272016-01-08 14:33:09 -08001707
1708 import importlib.util
1709 import sys
1710
1711 # For illustrative purposes.
1712 import tokenize
1713 file_path = tokenize.__file__
1714 module_name = tokenize.__name__
1715
1716 spec = importlib.util.spec_from_file_location(module_name, file_path)
1717 module = importlib.util.module_from_spec(spec)
Brett Cannona85e9272016-01-08 14:33:09 -08001718 sys.modules[module_name] = module
Brett Cannon08270642019-07-12 15:35:34 -07001719 spec.loader.exec_module(module)
1720
Brett Cannona85e9272016-01-08 14:33:09 -08001721
Brett Cannon23763162016-09-08 10:12:47 -07001722
1723Setting up an importer
1724''''''''''''''''''''''
1725
Brett Cannon59363132016-03-18 11:54:22 -07001726For deep customizations of import, you typically want to implement an
1727:term:`importer`. This means managing both the :term:`finder` and :term:`loader`
1728side of things. For finders there are two flavours to choose from depending on
1729your needs: a :term:`meta path finder` or a :term:`path entry finder`. The
1730former is what you would put on :attr:`sys.meta_path` while the latter is what
1731you create using a :term:`path entry hook` on :attr:`sys.path_hooks` which works
1732with :attr:`sys.path` entries to potentially create a finder. This example will
1733show you how to register your own importers so that import will use them (for
1734creating an importer for yourself, read the documentation for the appropriate
1735classes defined within this package)::
1736
1737 import importlib.machinery
1738 import sys
1739
1740 # For illustrative purposes only.
1741 SpamMetaPathFinder = importlib.machinery.PathFinder
1742 SpamPathEntryFinder = importlib.machinery.FileFinder
1743 loader_details = (importlib.machinery.SourceFileLoader,
1744 importlib.machinery.SOURCE_SUFFIXES)
1745
1746 # Setting up a meta path finder.
1747 # Make sure to put the finder in the proper location in the list in terms of
1748 # priority.
1749 sys.meta_path.append(SpamMetaPathFinder)
1750
1751 # Setting up a path entry finder.
1752 # Make sure to put the path hook in the proper location in the list in terms
1753 # of priority.
1754 sys.path_hooks.append(SpamPathEntryFinder.path_hook(loader_details))
1755
Brett Cannon23763162016-09-08 10:12:47 -07001756
1757Approximating :func:`importlib.import_module`
1758'''''''''''''''''''''''''''''''''''''''''''''
1759
Brett Cannona85e9272016-01-08 14:33:09 -08001760Import itself is implemented in Python code, making it possible to
1761expose most of the import machinery through importlib. The following
1762helps illustrate the various APIs that importlib exposes by providing an
1763approximate implementation of
Brett Cannon59363132016-03-18 11:54:22 -07001764:func:`importlib.import_module` (Python 3.4 and newer for the importlib usage,
Brett Cannona85e9272016-01-08 14:33:09 -08001765Python 3.6 and newer for other parts of the code).
1766::
1767
1768 import importlib.util
1769 import sys
1770
1771 def import_module(name, package=None):
1772 """An approximate implementation of import."""
1773 absolute_name = importlib.util.resolve_name(name, package)
1774 try:
1775 return sys.modules[absolute_name]
1776 except KeyError:
1777 pass
1778
1779 path = None
1780 if '.' in absolute_name:
1781 parent_name, _, child_name = absolute_name.rpartition('.')
1782 parent_module = import_module(parent_name)
orlnub12378401f72018-10-25 02:32:26 +03001783 path = parent_module.__spec__.submodule_search_locations
Brett Cannona85e9272016-01-08 14:33:09 -08001784 for finder in sys.meta_path:
1785 spec = finder.find_spec(absolute_name, path)
1786 if spec is not None:
1787 break
1788 else:
Nick Coghlancee29b42019-01-17 20:41:29 +10001789 msg = f'No module named {absolute_name!r}'
1790 raise ModuleNotFoundError(msg, name=absolute_name)
Brett Cannon86a8be02016-02-20 18:47:09 -08001791 module = importlib.util.module_from_spec(spec)
Brett Cannona85e9272016-01-08 14:33:09 -08001792 sys.modules[absolute_name] = module
Brett Cannon08270642019-07-12 15:35:34 -07001793 spec.loader.exec_module(module)
Brett Cannona85e9272016-01-08 14:33:09 -08001794 if path is not None:
1795 setattr(parent_module, child_name, module)
1796 return module