blob: c80f4606f3d1c3e748152f6e12220f3903d2ff78 [file] [log] [blame]
Serhiy Storchaka29b0a262016-12-04 10:20:55 +02001:mod:`importlib` --- The implementation of :keyword:`import`
2============================================================
Brett Cannonafccd632009-01-20 02:21:27 +00003
4.. module:: importlib
Brett Cannon07fbd782014-02-06 09:46:08 -05005 :synopsis: The implementation of the import machinery.
Brett Cannonafccd632009-01-20 02:21:27 +00006
7.. moduleauthor:: Brett Cannon <brett@python.org>
8.. sectionauthor:: Brett Cannon <brett@python.org>
9
10.. versionadded:: 3.1
11
Terry Jan Reedyfa089b92016-06-11 15:02:54 -040012**Source code:** :source:`Lib/importlib/__init__.py`
13
14--------------
Brett Cannonafccd632009-01-20 02:21:27 +000015
16Introduction
17------------
18
Brett Cannon07fbd782014-02-06 09:46:08 -050019The purpose of the :mod:`importlib` package is two-fold. One is to provide the
Brett Cannonafccd632009-01-20 02:21:27 +000020implementation of the :keyword:`import` statement (and thus, by extension, the
21:func:`__import__` function) in Python source code. This provides an
Tarek Ziadé434caaa2009-05-14 12:48:09 +000022implementation of :keyword:`import` which is portable to any Python
Brett Cannon07fbd782014-02-06 09:46:08 -050023interpreter. This also provides an implementation which is easier to
Brett Cannonf23e3742010-06-27 23:57:46 +000024comprehend than one implemented in a programming language other than Python.
Brett Cannonafccd632009-01-20 02:21:27 +000025
Brett Cannonf23e3742010-06-27 23:57:46 +000026Two, the components to implement :keyword:`import` are exposed in this
Brett Cannonafccd632009-01-20 02:21:27 +000027package, making it easier for users to create their own custom objects (known
Brett Cannondebb98d2009-02-16 04:18:01 +000028generically as an :term:`importer`) to participate in the import process.
Brett Cannonafccd632009-01-20 02:21:27 +000029
30.. seealso::
31
32 :ref:`import`
33 The language reference for the :keyword:`import` statement.
34
Benjamin Peterson60dbed12017-09-05 16:24:39 -070035 `Packages specification <https://www.python.org/doc/essays/packages/>`__
Brett Cannonafccd632009-01-20 02:21:27 +000036 Original specification of packages. Some semantics have changed since
Georg Brandl375aec22011-01-15 17:03:02 +000037 the writing of this document (e.g. redirecting based on ``None``
Brett Cannonafccd632009-01-20 02:21:27 +000038 in :data:`sys.modules`).
39
40 The :func:`.__import__` function
Brett Cannon0e13c942010-06-29 18:26:11 +000041 The :keyword:`import` statement is syntactic sugar for this function.
Brett Cannonafccd632009-01-20 02:21:27 +000042
43 :pep:`235`
44 Import on Case-Insensitive Platforms
45
46 :pep:`263`
47 Defining Python Source Code Encodings
48
49 :pep:`302`
Brett Cannonf23e3742010-06-27 23:57:46 +000050 New Import Hooks
Brett Cannonafccd632009-01-20 02:21:27 +000051
52 :pep:`328`
53 Imports: Multi-Line and Absolute/Relative
54
55 :pep:`366`
56 Main module explicit relative imports
57
Eric Snow338502b2016-05-28 11:56:53 -060058 :pep:`420`
59 Implicit namespace packages
60
Brett Cannon07fbd782014-02-06 09:46:08 -050061 :pep:`451`
62 A ModuleSpec Type for the Import System
63
Nick Coghland5cacbb2015-05-23 22:24:10 +100064 :pep:`488`
65 Elimination of PYO files
66
67 :pep:`489`
68 Multi-phase extension module initialization
69
Benjamin Peterson42aa93b2017-12-09 10:26:52 -080070 :pep:`552`
71 Deterministic pycs
72
Brett Cannon8917d5e2010-01-13 19:21:00 +000073 :pep:`3120`
Brett Cannonafccd632009-01-20 02:21:27 +000074 Using UTF-8 as the Default Source Encoding
75
Brett Cannon30b7a902010-06-27 21:49:22 +000076 :pep:`3147`
77 PYC Repository Directories
78
Brett Cannonafccd632009-01-20 02:21:27 +000079
80Functions
81---------
82
Brett Cannoncb4996a2012-08-06 16:34:44 -040083.. function:: __import__(name, globals=None, locals=None, fromlist=(), level=0)
Brett Cannonafccd632009-01-20 02:21:27 +000084
Brett Cannonf23e3742010-06-27 23:57:46 +000085 An implementation of the built-in :func:`__import__` function.
Brett Cannonafccd632009-01-20 02:21:27 +000086
Brett Cannon3fa84222015-02-20 10:34:20 -050087 .. note::
88 Programmatic importing of modules should use :func:`import_module`
89 instead of this function.
90
Brett Cannonafccd632009-01-20 02:21:27 +000091.. function:: import_module(name, package=None)
92
Brett Cannon33418c82009-01-22 18:37:20 +000093 Import a module. The *name* argument specifies what module to
Brett Cannonafccd632009-01-20 02:21:27 +000094 import in absolute or relative terms
95 (e.g. either ``pkg.mod`` or ``..mod``). If the name is
Guido van Rossum09613542009-03-30 20:34:57 +000096 specified in relative terms, then the *package* argument must be set to
97 the name of the package which is to act as the anchor for resolving the
Brett Cannonafccd632009-01-20 02:21:27 +000098 package name (e.g. ``import_module('..mod', 'pkg.subpkg')`` will import
Brett Cannon2c318a12009-02-07 01:15:27 +000099 ``pkg.mod``).
Brett Cannon78246b62009-01-25 04:56:30 +0000100
Brett Cannon2c318a12009-02-07 01:15:27 +0000101 The :func:`import_module` function acts as a simplifying wrapper around
Brett Cannon9f4cb1c2009-04-01 23:26:47 +0000102 :func:`importlib.__import__`. This means all semantics of the function are
Brett Cannon3fa84222015-02-20 10:34:20 -0500103 derived from :func:`importlib.__import__`. The most important difference
104 between these two functions is that :func:`import_module` returns the
105 specified package or module (e.g. ``pkg.mod``), while :func:`__import__`
106 returns the top-level package or module (e.g. ``pkg``).
107
108 If you are dynamically importing a module that was created since the
109 interpreter began execution (e.g., created a Python source file), you may
110 need to call :func:`invalidate_caches` in order for the new module to be
111 noticed by the import system.
Guido van Rossum09613542009-03-30 20:34:57 +0000112
Brett Cannon98620d82013-12-13 13:57:41 -0500113 .. versionchanged:: 3.3
114 Parent packages are automatically imported.
115
Brett Cannonee78a2b2012-05-12 17:43:17 -0400116.. function:: find_loader(name, path=None)
117
118 Find the loader for a module, optionally within the specified *path*. If the
119 module is in :attr:`sys.modules`, then ``sys.modules[name].__loader__`` is
Brett Cannon32799232013-03-13 11:09:08 -0700120 returned (unless the loader would be ``None`` or is not set, in which case
Brett Cannonee78a2b2012-05-12 17:43:17 -0400121 :exc:`ValueError` is raised). Otherwise a search using :attr:`sys.meta_path`
122 is done. ``None`` is returned if no loader is found.
123
Benjamin Petersonf7e2ea22016-09-05 14:02:59 -0700124 A dotted name does not have its parents implicitly imported as that requires
Brett Cannon56b4ca72012-11-17 09:30:55 -0500125 loading them and that may not be desired. To properly import a submodule you
126 will need to import all parent packages of the submodule and use the correct
127 argument to *path*.
Brett Cannonee78a2b2012-05-12 17:43:17 -0400128
Brett Cannon32799232013-03-13 11:09:08 -0700129 .. versionadded:: 3.3
130
131 .. versionchanged:: 3.4
132 If ``__loader__`` is not set, raise :exc:`ValueError`, just like when the
133 attribute is set to ``None``.
134
Eric Snowca2d8542013-12-16 23:06:52 -0700135 .. deprecated:: 3.4
Eric Snow6029e082014-01-25 15:32:46 -0700136 Use :func:`importlib.util.find_spec` instead.
Eric Snowca2d8542013-12-16 23:06:52 -0700137
Antoine Pitrouc541f8e2012-02-20 01:48:16 +0100138.. function:: invalidate_caches()
139
Brett Cannonf4dc9202012-08-10 12:21:12 -0400140 Invalidate the internal caches of finders stored at
141 :data:`sys.meta_path`. If a finder implements ``invalidate_caches()`` then it
Brett Cannon4067aa22013-04-27 23:20:32 -0400142 will be called to perform the invalidation. This function should be called
143 if any modules are created/installed while your program is running to
144 guarantee all finders will notice the new module's existence.
Antoine Pitrouc541f8e2012-02-20 01:48:16 +0100145
146 .. versionadded:: 3.3
147
Brett Cannon3fe35e62013-06-14 15:04:26 -0400148.. function:: reload(module)
149
150 Reload a previously imported *module*. The argument must be a module object,
151 so it must have been successfully imported before. This is useful if you
152 have edited the module source file using an external editor and want to try
153 out the new version without leaving the Python interpreter. The return value
Brett Cannon8ad37862013-10-25 13:52:46 -0400154 is the module object (which can be different if re-importing causes a
155 different object to be placed in :data:`sys.modules`).
Brett Cannon3fe35e62013-06-14 15:04:26 -0400156
Brett Cannon8ad37862013-10-25 13:52:46 -0400157 When :func:`reload` is executed:
Brett Cannon3fe35e62013-06-14 15:04:26 -0400158
Larry Hastings3732ed22014-03-15 21:13:56 -0700159 * Python module's code is recompiled and the module-level code re-executed,
Brett Cannon3fe35e62013-06-14 15:04:26 -0400160 defining a new set of objects which are bound to names in the module's
161 dictionary by reusing the :term:`loader` which originally loaded the
162 module. The ``init`` function of extension modules is not called a second
163 time.
164
165 * As with all other objects in Python the old objects are only reclaimed
166 after their reference counts drop to zero.
167
168 * The names in the module namespace are updated to point to any new or
169 changed objects.
170
171 * Other references to the old objects (such as names external to the module) are
172 not rebound to refer to the new objects and must be updated in each namespace
173 where they occur if that is desired.
174
175 There are a number of other caveats:
176
Brett Cannon3fe35e62013-06-14 15:04:26 -0400177 When a module is reloaded, its dictionary (containing the module's global
178 variables) is retained. Redefinitions of names will override the old
179 definitions, so this is generally not a problem. If the new version of a
180 module does not define a name that was defined by the old version, the old
181 definition remains. This feature can be used to the module's advantage if it
182 maintains a global table or cache of objects --- with a :keyword:`try`
183 statement it can test for the table's presence and skip its initialization if
184 desired::
185
186 try:
187 cache
188 except NameError:
189 cache = {}
190
Robert Collins1ae28d22015-08-05 08:20:53 +1200191 It is generally not very useful to reload built-in or dynamically loaded
192 modules. Reloading :mod:`sys`, :mod:`__main__`, :mod:`builtins` and other
193 key modules is not recommended. In many cases extension modules are not
194 designed to be initialized more than once, and may fail in arbitrary ways
195 when reloaded.
Brett Cannon3fe35e62013-06-14 15:04:26 -0400196
197 If a module imports objects from another module using :keyword:`from` ...
198 :keyword:`import` ..., calling :func:`reload` for the other module does not
199 redefine the objects imported from it --- one way around this is to
200 re-execute the :keyword:`from` statement, another is to use :keyword:`import`
201 and qualified names (*module.name*) instead.
202
203 If a module instantiates instances of a class, reloading the module that
204 defines the class does not affect the method definitions of the instances ---
205 they continue to use the old class definition. The same is true for derived
206 classes.
207
208 .. versionadded:: 3.4
Garvit Khatri94987822017-05-25 03:49:50 +0530209 .. versionchanged:: 3.7
210 :exc:`ModuleNotFoundError` is raised when the module being reloaded lacks
211 a :class:`ModuleSpec`.
Brett Cannon3fe35e62013-06-14 15:04:26 -0400212
Brett Cannon78246b62009-01-25 04:56:30 +0000213
Brett Cannon2a922ed2009-03-09 03:35:50 +0000214:mod:`importlib.abc` -- Abstract base classes related to import
215---------------------------------------------------------------
216
217.. module:: importlib.abc
218 :synopsis: Abstract base classes related to import
219
Terry Jan Reedydcb6c882016-06-22 22:46:34 -0400220**Source code:** :source:`Lib/importlib/abc.py`
221
222--------------
223
224
Brett Cannon2a922ed2009-03-09 03:35:50 +0000225The :mod:`importlib.abc` module contains all of the core abstract base classes
226used by :keyword:`import`. Some subclasses of the core abstract base classes
227are also provided to help in implementing the core ABCs.
228
Andrew Svetlova8656542012-08-13 22:19:01 +0300229ABC hierarchy::
230
231 object
Brett Cannon1b799182012-08-17 14:08:24 -0400232 +-- Finder (deprecated)
Andrew Svetlova8656542012-08-13 22:19:01 +0300233 | +-- MetaPathFinder
234 | +-- PathEntryFinder
235 +-- Loader
236 +-- ResourceLoader --------+
237 +-- InspectLoader |
238 +-- ExecutionLoader --+
239 +-- FileLoader
240 +-- SourceLoader
Andrew Svetlova8656542012-08-13 22:19:01 +0300241
Brett Cannon2a922ed2009-03-09 03:35:50 +0000242
243.. class:: Finder
244
Brett Cannon1b799182012-08-17 14:08:24 -0400245 An abstract base class representing a :term:`finder`.
246
247 .. deprecated:: 3.3
248 Use :class:`MetaPathFinder` or :class:`PathEntryFinder` instead.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000249
Berker Peksag6e9d2e62015-12-08 12:14:50 +0200250 .. abstractmethod:: find_module(fullname, path=None)
Brett Cannonb46a1792012-02-27 18:15:42 -0500251
Brett Cannonf4dc9202012-08-10 12:21:12 -0400252 An abstact method for finding a :term:`loader` for the specified
253 module. Originally specified in :pep:`302`, this method was meant
254 for use in :data:`sys.meta_path` and in the path-based import subsystem.
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000255
Brett Cannon100883f2013-04-09 16:59:39 -0400256 .. versionchanged:: 3.4
257 Returns ``None`` when called instead of raising
258 :exc:`NotImplementedError`.
259
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000260
Brett Cannon077ef452012-08-02 17:50:06 -0400261.. class:: MetaPathFinder
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000262
Brett Cannonf4dc9202012-08-10 12:21:12 -0400263 An abstract base class representing a :term:`meta path finder`. For
264 compatibility, this is a subclass of :class:`Finder`.
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000265
266 .. versionadded:: 3.3
267
Eric Snowca2d8542013-12-16 23:06:52 -0700268 .. method:: find_spec(fullname, path, target=None)
269
270 An abstract method for finding a :term:`spec <module spec>` for
271 the specified module. If this is a top-level import, *path* will
272 be ``None``. Otherwise, this is a search for a subpackage or
273 module and *path* will be the value of :attr:`__path__` from the
274 parent package. If a spec cannot be found, ``None`` is returned.
275 When passed in, ``target`` is a module object that the finder may
Brett Cannona85e9272016-01-08 14:33:09 -0800276 use to make a more educated guess about what spec to return.
Eric Snowca2d8542013-12-16 23:06:52 -0700277
278 .. versionadded:: 3.4
279
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000280 .. method:: find_module(fullname, path)
281
Eric Snowca2d8542013-12-16 23:06:52 -0700282 A legacy method for finding a :term:`loader` for the specified
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000283 module. If this is a top-level import, *path* will be ``None``.
Ezio Melotti1f67e802012-10-21 07:24:13 +0300284 Otherwise, this is a search for a subpackage or module and *path*
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000285 will be the value of :attr:`__path__` from the parent
286 package. If a loader cannot be found, ``None`` is returned.
287
Brett Cannon8d942292014-01-07 15:52:42 -0500288 If :meth:`find_spec` is defined, backwards-compatible functionality is
289 provided.
290
Brett Cannon100883f2013-04-09 16:59:39 -0400291 .. versionchanged:: 3.4
292 Returns ``None`` when called instead of raising
Brett Cannon8d942292014-01-07 15:52:42 -0500293 :exc:`NotImplementedError`. Can use :meth:`find_spec` to provide
294 functionality.
Brett Cannon100883f2013-04-09 16:59:39 -0400295
Eric Snowca2d8542013-12-16 23:06:52 -0700296 .. deprecated:: 3.4
297 Use :meth:`find_spec` instead.
298
Brett Cannonf4dc9202012-08-10 12:21:12 -0400299 .. method:: invalidate_caches()
300
301 An optional method which, when called, should invalidate any internal
Brett Cannona6e85812012-08-11 19:41:27 -0400302 cache used by the finder. Used by :func:`importlib.invalidate_caches`
303 when invalidating the caches of all finders on :data:`sys.meta_path`.
Brett Cannonf4dc9202012-08-10 12:21:12 -0400304
Brett Cannon100883f2013-04-09 16:59:39 -0400305 .. versionchanged:: 3.4
306 Returns ``None`` when called instead of ``NotImplemented``.
307
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000308
Brett Cannon077ef452012-08-02 17:50:06 -0400309.. class:: PathEntryFinder
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000310
Brett Cannonf4dc9202012-08-10 12:21:12 -0400311 An abstract base class representing a :term:`path entry finder`. Though
312 it bears some similarities to :class:`MetaPathFinder`, ``PathEntryFinder``
313 is meant for use only within the path-based import subsystem provided
314 by :class:`PathFinder`. This ABC is a subclass of :class:`Finder` for
Brett Cannon100883f2013-04-09 16:59:39 -0400315 compatibility reasons only.
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000316
317 .. versionadded:: 3.3
318
Eric Snowca2d8542013-12-16 23:06:52 -0700319 .. method:: find_spec(fullname, target=None)
320
321 An abstract method for finding a :term:`spec <module spec>` for
322 the specified module. The finder will search for the module only
323 within the :term:`path entry` to which it is assigned. If a spec
324 cannot be found, ``None`` is returned. When passed in, ``target``
325 is a module object that the finder may use to make a more educated
Brett Cannona85e9272016-01-08 14:33:09 -0800326 guess about what spec to return.
Eric Snowca2d8542013-12-16 23:06:52 -0700327
328 .. versionadded:: 3.4
329
Brett Cannon4067aa22013-04-27 23:20:32 -0400330 .. method:: find_loader(fullname)
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000331
Eric Snowca2d8542013-12-16 23:06:52 -0700332 A legacy method for finding a :term:`loader` for the specified
Brett Cannonf4dc9202012-08-10 12:21:12 -0400333 module. Returns a 2-tuple of ``(loader, portion)`` where ``portion``
334 is a sequence of file system locations contributing to part of a namespace
335 package. The loader may be ``None`` while specifying ``portion`` to
336 signify the contribution of the file system locations to a namespace
337 package. An empty list can be used for ``portion`` to signify the loader
Brett Cannon100883f2013-04-09 16:59:39 -0400338 is not part of a namespace package. If ``loader`` is ``None`` and
339 ``portion`` is the empty list then no loader or location for a namespace
340 package were found (i.e. failure to find anything for the module).
341
Brett Cannon8d942292014-01-07 15:52:42 -0500342 If :meth:`find_spec` is defined then backwards-compatible functionality is
343 provided.
344
Brett Cannon100883f2013-04-09 16:59:39 -0400345 .. versionchanged:: 3.4
346 Returns ``(None, [])`` instead of raising :exc:`NotImplementedError`.
Brett Cannon8d942292014-01-07 15:52:42 -0500347 Uses :meth:`find_spec` when available to provide functionality.
Brett Cannonf4dc9202012-08-10 12:21:12 -0400348
Eric Snowca2d8542013-12-16 23:06:52 -0700349 .. deprecated:: 3.4
350 Use :meth:`find_spec` instead.
351
Brett Cannon4067aa22013-04-27 23:20:32 -0400352 .. method:: find_module(fullname)
Brett Cannonf4dc9202012-08-10 12:21:12 -0400353
354 A concrete implementation of :meth:`Finder.find_module` which is
355 equivalent to ``self.find_loader(fullname)[0]``.
356
Eric Snowca2d8542013-12-16 23:06:52 -0700357 .. deprecated:: 3.4
358 Use :meth:`find_spec` instead.
359
Brett Cannonf4dc9202012-08-10 12:21:12 -0400360 .. method:: invalidate_caches()
361
362 An optional method which, when called, should invalidate any internal
Brett Cannona6e85812012-08-11 19:41:27 -0400363 cache used by the finder. Used by :meth:`PathFinder.invalidate_caches`
Brett Cannonf4dc9202012-08-10 12:21:12 -0400364 when invalidating the caches of all cached finders.
Brett Cannonb46a1792012-02-27 18:15:42 -0500365
Brett Cannon2a922ed2009-03-09 03:35:50 +0000366
367.. class:: Loader
368
369 An abstract base class for a :term:`loader`.
Guido van Rossum09613542009-03-30 20:34:57 +0000370 See :pep:`302` for the exact definition for a loader.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000371
Brett Cannonbca42182018-01-12 15:08:59 -0800372 For loaders that wish to support resource reading, they should
373 implement a ``get_resource_reader(fullname)`` method as specified
374 by :class:`importlib.abc.ResourceReader`.
375
376 .. versionchanged:: 3.7
377 Introduced the optional ``get_resource_reader()`` method.
378
Eric Snowca2d8542013-12-16 23:06:52 -0700379 .. method:: create_module(spec)
380
Brett Cannon02d84542015-01-09 11:39:21 -0500381 A method that returns the module object to use when
382 importing a module. This method may return ``None``,
383 indicating that default module creation semantics should take place.
Eric Snowca2d8542013-12-16 23:06:52 -0700384
385 .. versionadded:: 3.4
386
Brett Cannon02d84542015-01-09 11:39:21 -0500387 .. versionchanged:: 3.5
388 Starting in Python 3.6, this method will not be optional when
389 :meth:`exec_module` is defined.
390
Eric Snowca2d8542013-12-16 23:06:52 -0700391 .. method:: exec_module(module)
392
393 An abstract method that executes the module in its own namespace
394 when a module is imported or reloaded. The module should already
Brett Cannon696c35e2016-06-25 10:58:17 -0700395 be initialized when ``exec_module()`` is called. When this method exists,
396 :meth:`~importlib.abc.Loader.create_module` must be defined.
Eric Snowca2d8542013-12-16 23:06:52 -0700397
398 .. versionadded:: 3.4
399
Brett Cannon696c35e2016-06-25 10:58:17 -0700400 .. versionchanged:: 3.6
401 :meth:`~importlib.abc.Loader.create_module` must also be defined.
402
Brett Cannon9c751b72009-03-09 16:28:16 +0000403 .. method:: load_module(fullname)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000404
Eric Snowca2d8542013-12-16 23:06:52 -0700405 A legacy method for loading a module. If the module cannot be
Brett Cannon2a922ed2009-03-09 03:35:50 +0000406 loaded, :exc:`ImportError` is raised, otherwise the loaded module is
407 returned.
408
Guido van Rossum09613542009-03-30 20:34:57 +0000409 If the requested module already exists in :data:`sys.modules`, that
Brett Cannon2a922ed2009-03-09 03:35:50 +0000410 module should be used and reloaded.
Guido van Rossum09613542009-03-30 20:34:57 +0000411 Otherwise the loader should create a new module and insert it into
412 :data:`sys.modules` before any loading begins, to prevent recursion
413 from the import. If the loader inserted a module and the load fails, it
Brett Cannon2a922ed2009-03-09 03:35:50 +0000414 must be removed by the loader from :data:`sys.modules`; modules already
415 in :data:`sys.modules` before the loader began execution should be left
Eric Snowb523f842013-11-22 09:05:39 -0700416 alone (see :func:`importlib.util.module_for_loader`).
Brett Cannon2a922ed2009-03-09 03:35:50 +0000417
Guido van Rossum09613542009-03-30 20:34:57 +0000418 The loader should set several attributes on the module.
419 (Note that some of these attributes can change when a module is
Eric Snowb523f842013-11-22 09:05:39 -0700420 reloaded):
Brett Cannon2a922ed2009-03-09 03:35:50 +0000421
422 - :attr:`__name__`
423 The name of the module.
424
425 - :attr:`__file__`
426 The path to where the module data is stored (not set for built-in
427 modules).
428
Brett Cannon2cefb3c2013-05-25 11:26:11 -0400429 - :attr:`__cached__`
430 The path to where a compiled version of the module is/should be
431 stored (not set when the attribute would be inappropriate).
432
Brett Cannon2a922ed2009-03-09 03:35:50 +0000433 - :attr:`__path__`
Guido van Rossum09613542009-03-30 20:34:57 +0000434 A list of strings specifying the search path within a
Brett Cannon2a922ed2009-03-09 03:35:50 +0000435 package. This attribute is not set on modules.
436
437 - :attr:`__package__`
438 The parent package for the module/package. If the module is
439 top-level then it has a value of the empty string. The
Brett Cannon100883f2013-04-09 16:59:39 -0400440 :func:`importlib.util.module_for_loader` decorator can handle the
441 details for :attr:`__package__`.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000442
443 - :attr:`__loader__`
Brett Cannon100883f2013-04-09 16:59:39 -0400444 The loader used to load the module. The
445 :func:`importlib.util.module_for_loader` decorator can handle the
446 details for :attr:`__package__`.
447
Brett Cannon8d942292014-01-07 15:52:42 -0500448 When :meth:`exec_module` is available then backwards-compatible
449 functionality is provided.
450
Brett Cannon100883f2013-04-09 16:59:39 -0400451 .. versionchanged:: 3.4
452 Raise :exc:`ImportError` when called instead of
Brett Cannon8d942292014-01-07 15:52:42 -0500453 :exc:`NotImplementedError`. Functionality provided when
454 :meth:`exec_module` is available.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000455
Eric Snowca2d8542013-12-16 23:06:52 -0700456 .. deprecated:: 3.4
457 The recommended API for loading a module is :meth:`exec_module`
Brett Cannon02d84542015-01-09 11:39:21 -0500458 (and :meth:`create_module`). Loaders should implement
Eric Snowca2d8542013-12-16 23:06:52 -0700459 it instead of load_module(). The import machinery takes care of
460 all the other responsibilities of load_module() when exec_module()
461 is implemented.
462
Barry Warsawd7d21942012-07-29 16:36:17 -0400463 .. method:: module_repr(module)
464
Eric Snowca2d8542013-12-16 23:06:52 -0700465 A legacy method which when implemented calculates and returns the
Brett Cannon100883f2013-04-09 16:59:39 -0400466 given module's repr, as a string. The module type's default repr() will
467 use the result of this method as appropriate.
Barry Warsawd7d21942012-07-29 16:36:17 -0400468
Georg Brandl526575d2013-04-11 16:10:13 +0200469 .. versionadded:: 3.3
Barry Warsawd7d21942012-07-29 16:36:17 -0400470
Brett Cannon100883f2013-04-09 16:59:39 -0400471 .. versionchanged:: 3.4
Georg Brandl526575d2013-04-11 16:10:13 +0200472 Made optional instead of an abstractmethod.
Brett Cannon100883f2013-04-09 16:59:39 -0400473
Eric Snowca2d8542013-12-16 23:06:52 -0700474 .. deprecated:: 3.4
475 The import machinery now takes care of this automatically.
476
Brett Cannon2a922ed2009-03-09 03:35:50 +0000477
Brett Cannon4ac51502017-12-15 16:29:35 -0800478.. class:: ResourceReader
479
Brett Cannonbca42182018-01-12 15:08:59 -0800480 An :term:`abstract base class` to provide the ability to read
Brett Cannon4ac51502017-12-15 16:29:35 -0800481 *resources*.
482
483 From the perspective of this ABC, a *resource* is a binary
484 artifact that is shipped within a package. Typically this is
485 something like a data file that lives next to the ``__init__.py``
486 file of the package. The purpose of this class is to help abstract
487 out the accessing of such data files so that it does not matter if
488 the package and its data file(s) are stored in a e.g. zip file
489 versus on the file system.
490
491 For any of methods of this class, a *resource* argument is
Barry Warsawdeae6b42017-12-30 15:18:06 -0500492 expected to be a :term:`path-like object` which represents
Brett Cannon4ac51502017-12-15 16:29:35 -0800493 conceptually just a file name. This means that no subdirectory
494 paths should be included in the *resource* argument. This is
Brett Cannonbca42182018-01-12 15:08:59 -0800495 because the location of the package the reader is for, acts as the
496 "directory". Hence the metaphor for directories and file
Brett Cannon4ac51502017-12-15 16:29:35 -0800497 names is packages and resources, respectively. This is also why
498 instances of this class are expected to directly correlate to
499 a specific package (instead of potentially representing multiple
500 packages or a module).
501
Brett Cannonbca42182018-01-12 15:08:59 -0800502 Loaders that wish to support resource reading are expected to
503 provide a method called ``get_resource_loader(fullname)`` which
504 returns an object implementing this ABC's interface. If the module
505 specified by fullname is not a package, this method should return
506 :const:`None`. An object compatible with this ABC should only be
507 returned when the specified module is a package.
508
Brett Cannon4ac51502017-12-15 16:29:35 -0800509 .. versionadded:: 3.7
510
511 .. abstractmethod:: open_resource(resource)
512
513 Returns an opened, :term:`file-like object` for binary reading
514 of the *resource*.
515
516 If the resource cannot be found, :exc:`FileNotFoundError` is
517 raised.
518
519 .. abstractmethod:: resource_path(resource)
520
521 Returns the file system path to the *resource*.
522
523 If the resource does not concretely exist on the file system,
524 raise :exc:`FileNotFoundError`.
525
526 .. abstractmethod:: is_resource(name)
527
528 Returns ``True`` if the named *name* is considered a resource.
529 :exc:`FileNotFoundError` is raised if *name* does not exist.
530
531 .. abstractmethod:: contents()
532
533 Returns an :term:`iterator` of strings over the contents of
534 the package. Do note that it is not required that all names
535 returned by the iterator be actual resources, e.g. it is
536 acceptable to return names for which :meth:`is_resource` would
537 be false.
538
539 Allowing non-resource names to be returned is to allow for
540 situations where how a package and its resources are stored
541 are known a priori and the non-resource names would be useful.
542 For instance, returning subdirectory names is allowed so that
543 when it is known that the package and resources are stored on
Brett Cannonbca42182018-01-12 15:08:59 -0800544 the file system then those subdirectory names can be used
545 directly.
Brett Cannon4ac51502017-12-15 16:29:35 -0800546
Brett Cannonbca42182018-01-12 15:08:59 -0800547 The abstract method returns an iterator of no items.
Brett Cannon4ac51502017-12-15 16:29:35 -0800548
549
Brett Cannon2a922ed2009-03-09 03:35:50 +0000550.. class:: ResourceLoader
551
552 An abstract base class for a :term:`loader` which implements the optional
553 :pep:`302` protocol for loading arbitrary resources from the storage
554 back-end.
555
Brett Cannonbca42182018-01-12 15:08:59 -0800556 .. deprecated:: 3.7
557 This ABC is deprecated in favour of supporting resource loading
558 through :class:`importlib.abc.ResourceReader`.
559
Berker Peksag6e9d2e62015-12-08 12:14:50 +0200560 .. abstractmethod:: get_data(path)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000561
562 An abstract method to return the bytes for the data located at *path*.
Guido van Rossum09613542009-03-30 20:34:57 +0000563 Loaders that have a file-like storage back-end
Brett Cannon16248a42009-04-01 20:47:14 +0000564 that allows storing arbitrary data
Guido van Rossum09613542009-03-30 20:34:57 +0000565 can implement this abstract method to give direct access
Andrew Svetlov08af0002014-04-01 01:13:30 +0300566 to the data stored. :exc:`OSError` is to be raised if the *path* cannot
Brett Cannon2a922ed2009-03-09 03:35:50 +0000567 be found. The *path* is expected to be constructed using a module's
Brett Cannon16248a42009-04-01 20:47:14 +0000568 :attr:`__file__` attribute or an item from a package's :attr:`__path__`.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000569
Brett Cannon100883f2013-04-09 16:59:39 -0400570 .. versionchanged:: 3.4
Andrew Svetlov08af0002014-04-01 01:13:30 +0300571 Raises :exc:`OSError` instead of :exc:`NotImplementedError`.
Brett Cannon100883f2013-04-09 16:59:39 -0400572
Brett Cannon2a922ed2009-03-09 03:35:50 +0000573
574.. class:: InspectLoader
575
576 An abstract base class for a :term:`loader` which implements the optional
Guido van Rossum09613542009-03-30 20:34:57 +0000577 :pep:`302` protocol for loaders that inspect modules.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000578
Brett Cannona113ac52009-03-15 01:41:33 +0000579 .. method:: get_code(fullname)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000580
R David Murray0ae7ae12014-01-08 18:16:02 -0500581 Return the code object for a module, or ``None`` if the module does not
582 have a code object (as would be the case, for example, for a built-in
583 module). Raise an :exc:`ImportError` if loader cannot find the
584 requested module.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000585
Brett Cannon3b62ca82013-05-27 21:11:04 -0400586 .. note::
587 While the method has a default implementation, it is suggested that
588 it be overridden if possible for performance.
589
R David Murray1b00f252012-08-15 10:43:58 -0400590 .. index::
591 single: universal newlines; importlib.abc.InspectLoader.get_source method
592
Brett Cannon100883f2013-04-09 16:59:39 -0400593 .. versionchanged:: 3.4
Brett Cannon3b62ca82013-05-27 21:11:04 -0400594 No longer abstract and a concrete implementation is provided.
Brett Cannon100883f2013-04-09 16:59:39 -0400595
Berker Peksag6e9d2e62015-12-08 12:14:50 +0200596 .. abstractmethod:: get_source(fullname)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000597
598 An abstract method to return the source of a module. It is returned as
R David Murray1b00f252012-08-15 10:43:58 -0400599 a text string using :term:`universal newlines`, translating all
R David Murrayee0a9452012-08-15 11:05:36 -0400600 recognized line separators into ``'\n'`` characters. Returns ``None``
601 if no source is available (e.g. a built-in module). Raises
602 :exc:`ImportError` if the loader cannot find the module specified.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000603
Brett Cannon100883f2013-04-09 16:59:39 -0400604 .. versionchanged:: 3.4
605 Raises :exc:`ImportError` instead of :exc:`NotImplementedError`.
606
Brett Cannona113ac52009-03-15 01:41:33 +0000607 .. method:: is_package(fullname)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000608
Brett Cannona113ac52009-03-15 01:41:33 +0000609 An abstract method to return a true value if the module is a package, a
610 false value otherwise. :exc:`ImportError` is raised if the
611 :term:`loader` cannot find the module.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000612
Brett Cannon100883f2013-04-09 16:59:39 -0400613 .. versionchanged:: 3.4
614 Raises :exc:`ImportError` instead of :exc:`NotImplementedError`.
615
Brett Cannon6eaac132014-05-09 12:28:22 -0400616 .. staticmethod:: source_to_code(data, path='<string>')
Brett Cannon9ffe85e2013-05-26 16:45:10 -0400617
618 Create a code object from Python source.
619
620 The *data* argument can be whatever the :func:`compile` function
621 supports (i.e. string or bytes). The *path* argument should be
622 the "path" to where the source code originated from, which can be an
623 abstract concept (e.g. location in a zip file).
624
Brett Cannon6eaac132014-05-09 12:28:22 -0400625 With the subsequent code object one can execute it in a module by
626 running ``exec(code, module.__dict__)``.
627
Brett Cannon9ffe85e2013-05-26 16:45:10 -0400628 .. versionadded:: 3.4
629
Brett Cannon6eaac132014-05-09 12:28:22 -0400630 .. versionchanged:: 3.5
631 Made the method static.
632
Eric Snowca2d8542013-12-16 23:06:52 -0700633 .. method:: exec_module(module)
634
635 Implementation of :meth:`Loader.exec_module`.
636
637 .. versionadded:: 3.4
638
Brett Cannon0dbb4c72013-05-31 18:56:47 -0400639 .. method:: load_module(fullname)
640
Eric Snowca2d8542013-12-16 23:06:52 -0700641 Implementation of :meth:`Loader.load_module`.
642
643 .. deprecated:: 3.4
644 use :meth:`exec_module` instead.
Brett Cannon0dbb4c72013-05-31 18:56:47 -0400645
Brett Cannon2a922ed2009-03-09 03:35:50 +0000646
Brett Cannon69194272009-07-20 04:23:48 +0000647.. class:: ExecutionLoader
648
649 An abstract base class which inherits from :class:`InspectLoader` that,
Brett Cannon23460292009-07-20 22:59:00 +0000650 when implemented, helps a module to be executed as a script. The ABC
Brett Cannon69194272009-07-20 04:23:48 +0000651 represents an optional :pep:`302` protocol.
652
Berker Peksag6e9d2e62015-12-08 12:14:50 +0200653 .. abstractmethod:: get_filename(fullname)
Brett Cannon69194272009-07-20 04:23:48 +0000654
Brett Cannonf23e3742010-06-27 23:57:46 +0000655 An abstract method that is to return the value of :attr:`__file__` for
Brett Cannon69194272009-07-20 04:23:48 +0000656 the specified module. If no path is available, :exc:`ImportError` is
657 raised.
658
Brett Cannonf23e3742010-06-27 23:57:46 +0000659 If source code is available, then the method should return the path to
660 the source file, regardless of whether a bytecode was used to load the
661 module.
662
Brett Cannon100883f2013-04-09 16:59:39 -0400663 .. versionchanged:: 3.4
664 Raises :exc:`ImportError` instead of :exc:`NotImplementedError`.
665
Brett Cannonf23e3742010-06-27 23:57:46 +0000666
Brett Cannon938d44d2012-04-22 19:58:33 -0400667.. class:: FileLoader(fullname, path)
668
669 An abstract base class which inherits from :class:`ResourceLoader` and
Andrew Svetlova60de4f2013-02-17 16:55:58 +0200670 :class:`ExecutionLoader`, providing concrete implementations of
Brett Cannon938d44d2012-04-22 19:58:33 -0400671 :meth:`ResourceLoader.get_data` and :meth:`ExecutionLoader.get_filename`.
672
673 The *fullname* argument is a fully resolved name of the module the loader is
674 to handle. The *path* argument is the path to the file for the module.
675
676 .. versionadded:: 3.3
677
678 .. attribute:: name
679
680 The name of the module the loader can handle.
681
682 .. attribute:: path
683
684 Path to the file of the module.
685
Barry Warsawd7d21942012-07-29 16:36:17 -0400686 .. method:: load_module(fullname)
Brett Cannonc0499522012-05-11 14:48:41 -0400687
Barry Warsawd7d21942012-07-29 16:36:17 -0400688 Calls super's ``load_module()``.
Brett Cannonc0499522012-05-11 14:48:41 -0400689
Eric Snowca2d8542013-12-16 23:06:52 -0700690 .. deprecated:: 3.4
691 Use :meth:`Loader.exec_module` instead.
692
Berker Peksag6e9d2e62015-12-08 12:14:50 +0200693 .. abstractmethod:: get_filename(fullname)
Brett Cannon938d44d2012-04-22 19:58:33 -0400694
Barry Warsawd7d21942012-07-29 16:36:17 -0400695 Returns :attr:`path`.
Brett Cannon938d44d2012-04-22 19:58:33 -0400696
Berker Peksag6e9d2e62015-12-08 12:14:50 +0200697 .. abstractmethod:: get_data(path)
Brett Cannon938d44d2012-04-22 19:58:33 -0400698
Brett Cannon3b62ca82013-05-27 21:11:04 -0400699 Reads *path* as a binary file and returns the bytes from it.
Brett Cannon938d44d2012-04-22 19:58:33 -0400700
701
Brett Cannonf23e3742010-06-27 23:57:46 +0000702.. class:: SourceLoader
703
704 An abstract base class for implementing source (and optionally bytecode)
705 file loading. The class inherits from both :class:`ResourceLoader` and
706 :class:`ExecutionLoader`, requiring the implementation of:
707
708 * :meth:`ResourceLoader.get_data`
709 * :meth:`ExecutionLoader.get_filename`
Brett Cannon6dfbff32010-07-21 09:48:31 +0000710 Should only return the path to the source file; sourceless
Brett Cannona81d5272013-06-16 19:17:12 -0400711 loading is not supported.
Brett Cannonf23e3742010-06-27 23:57:46 +0000712
713 The abstract methods defined by this class are to add optional bytecode
Brett Cannon5650e4f2012-11-18 10:03:31 -0500714 file support. Not implementing these optional methods (or causing them to
715 raise :exc:`NotImplementedError`) causes the loader to
Brett Cannonf23e3742010-06-27 23:57:46 +0000716 only work with source code. Implementing the methods allows the loader to
717 work with source *and* bytecode files; it does not allow for *sourceless*
718 loading where only bytecode is provided. Bytecode files are an
719 optimization to speed up loading by removing the parsing step of Python's
720 compiler, and so no bytecode-specific API is exposed.
721
Brett Cannon773468f2012-08-02 17:35:34 -0400722 .. method:: path_stats(path)
Antoine Pitrou5136ac02012-01-13 18:52:16 +0100723
724 Optional abstract method which returns a :class:`dict` containing
Martin Pantereb995702016-07-28 01:11:04 +0000725 metadata about the specified path. Supported dictionary keys are:
Antoine Pitrou5136ac02012-01-13 18:52:16 +0100726
727 - ``'mtime'`` (mandatory): an integer or floating-point number
728 representing the modification time of the source code;
729 - ``'size'`` (optional): the size in bytes of the source code.
730
731 Any other keys in the dictionary are ignored, to allow for future
Andrew Svetlov08af0002014-04-01 01:13:30 +0300732 extensions. If the path cannot be handled, :exc:`OSError` is raised.
Antoine Pitrou5136ac02012-01-13 18:52:16 +0100733
734 .. versionadded:: 3.3
735
Brett Cannon100883f2013-04-09 16:59:39 -0400736 .. versionchanged:: 3.4
Andrew Svetlov08af0002014-04-01 01:13:30 +0300737 Raise :exc:`OSError` instead of :exc:`NotImplementedError`.
Brett Cannon100883f2013-04-09 16:59:39 -0400738
Brett Cannon773468f2012-08-02 17:35:34 -0400739 .. method:: path_mtime(path)
Brett Cannonf23e3742010-06-27 23:57:46 +0000740
741 Optional abstract method which returns the modification time for the
742 specified path.
743
Antoine Pitrou5136ac02012-01-13 18:52:16 +0100744 .. deprecated:: 3.3
745 This method is deprecated in favour of :meth:`path_stats`. You don't
746 have to implement it, but it is still available for compatibility
Andrew Svetlov08af0002014-04-01 01:13:30 +0300747 purposes. Raise :exc:`OSError` if the path cannot be handled.
Brett Cannon100883f2013-04-09 16:59:39 -0400748
Georg Brandldf48b972014-03-24 09:06:18 +0100749 .. versionchanged:: 3.4
Andrew Svetlov08af0002014-04-01 01:13:30 +0300750 Raise :exc:`OSError` instead of :exc:`NotImplementedError`.
Antoine Pitrou5136ac02012-01-13 18:52:16 +0100751
Brett Cannon773468f2012-08-02 17:35:34 -0400752 .. method:: set_data(path, data)
Brett Cannonf23e3742010-06-27 23:57:46 +0000753
754 Optional abstract method which writes the specified bytes to a file
Brett Cannon61b14252010-07-03 21:48:25 +0000755 path. Any intermediate directories which do not exist are to be created
756 automatically.
757
758 When writing to the path fails because the path is read-only
Brett Cannon2cefb3c2013-05-25 11:26:11 -0400759 (:attr:`errno.EACCES`/:exc:`PermissionError`), do not propagate the
760 exception.
Brett Cannonf23e3742010-06-27 23:57:46 +0000761
Brett Cannon100883f2013-04-09 16:59:39 -0400762 .. versionchanged:: 3.4
763 No longer raises :exc:`NotImplementedError` when called.
764
Brett Cannon773468f2012-08-02 17:35:34 -0400765 .. method:: get_code(fullname)
Brett Cannonf23e3742010-06-27 23:57:46 +0000766
767 Concrete implementation of :meth:`InspectLoader.get_code`.
768
Eric Snowca2d8542013-12-16 23:06:52 -0700769 .. method:: exec_module(module)
770
771 Concrete implementation of :meth:`Loader.exec_module`.
772
773 .. versionadded:: 3.4
774
Brett Cannon773468f2012-08-02 17:35:34 -0400775 .. method:: load_module(fullname)
Brett Cannonf23e3742010-06-27 23:57:46 +0000776
Eric Snowca2d8542013-12-16 23:06:52 -0700777 Concrete implementation of :meth:`Loader.load_module`.
778
779 .. deprecated:: 3.4
780 Use :meth:`exec_module` instead.
Brett Cannonf23e3742010-06-27 23:57:46 +0000781
Brett Cannon773468f2012-08-02 17:35:34 -0400782 .. method:: get_source(fullname)
Brett Cannonf23e3742010-06-27 23:57:46 +0000783
784 Concrete implementation of :meth:`InspectLoader.get_source`.
785
Brett Cannon773468f2012-08-02 17:35:34 -0400786 .. method:: is_package(fullname)
Brett Cannonf23e3742010-06-27 23:57:46 +0000787
788 Concrete implementation of :meth:`InspectLoader.is_package`. A module
Brett Cannonea0b8232012-06-15 20:00:53 -0400789 is determined to be a package if its file path (as provided by
790 :meth:`ExecutionLoader.get_filename`) is a file named
791 ``__init__`` when the file extension is removed **and** the module name
792 itself does not end in ``__init__``.
Brett Cannonf23e3742010-06-27 23:57:46 +0000793
Brett Cannon69194272009-07-20 04:23:48 +0000794
Barry Warsawdeae6b42017-12-30 15:18:06 -0500795:mod:`importlib.resources` -- Resources
796---------------------------------------
797
798.. module:: importlib.resources
799 :synopsis: Package resource reading, opening, and access
800
801**Source code:** :source:`Lib/importlib/resources.py`
802
803--------------
804
805.. versionadded:: 3.7
806
807This module leverages Python's import system to provide access to *resources*
808within *packages*. If you can import a package, you can access resources
809within that package. Resources can be opened or read, in either binary or
810text mode.
811
812Resources are roughly akin to files inside directories, though it's important
813to keep in mind that this is just a metaphor. Resources and packages **do
814not** have to exist as physical files and directories on the file system.
815
816Loaders can support resources by implementing the :class:`ResourceReader`
817abstract base class.
818
819The following types are defined.
820
821.. data:: Package
822
823 The ``Package`` type is defined as ``Union[str, ModuleType]``. This means
824 that where the function describes accepting a ``Package``, you can pass in
825 either a string or a module. Module objects must have a resolvable
826 ``__spec__.submodule_search_locations`` that is not ``None``.
827
828.. data:: Resource
829
830 This type describes the resource names passed into the various functions
831 in this package. This is defined as ``Union[str, os.PathLike]``.
832
833
834The following functions are available.
835
836.. function:: open_binary(package, resource)
837
838 Open for binary reading the *resource* within *package*.
839
840 *package* is either a name or a module object which conforms to the
841 ``Package`` requirements. *resource* is the name of the resource to open
842 within *package*; it may not contain path separators and it may not have
843 sub-resources (i.e. it cannot be a directory). This function returns a
844 ``typing.BinaryIO`` instance, a binary I/O stream open for reading.
845
846
847.. function:: open_text(package, resource, encoding='utf-8', errors='strict')
848
849 Open for text reading the *resource* within *package*. By default, the
850 resource is opened for reading as UTF-8.
851
852 *package* is either a name or a module object which conforms to the
853 ``Package`` requirements. *resource* is the name of the resource to open
854 within *package*; it may not contain path separators and it may not have
855 sub-resources (i.e. it cannot be a directory). *encoding* and *errors*
856 have the same meaning as with built-in :func:`open`.
857
858 This function returns a ``typing.TextIO`` instance, a text I/O stream open
859 for reading.
860
861
862.. function:: read_binary(package, resource)
863
864 Read and return the contents of the *resource* within *package* as
865 ``bytes``.
866
867 *package* is either a name or a module object which conforms to the
868 ``Package`` requirements. *resource* is the name of the resource to open
869 within *package*; it may not contain path separators and it may not have
870 sub-resources (i.e. it cannot be a directory). This function returns the
871 contents of the resource as :class:`bytes`.
872
873
874.. function:: read_text(package, resource, encoding='utf-8', errors='strict')
875
876 Read and return the contents of *resource* within *package* as a ``str``.
877 By default, the contents are read as strict UTF-8.
878
879 *package* is either a name or a module object which conforms to the
880 ``Package`` requirements. *resource* is the name of the resource to open
881 within *package*; it may not contain path separators and it may not have
882 sub-resources (i.e. it cannot be a directory). *encoding* and *errors*
883 have the same meaning as with built-in :func:`open`. This function
884 returns the contents of the resource as :class:`str`.
885
886
887.. function:: path(package, resource)
888
889 Return the path to the *resource* as an actual file system path. This
890 function returns a context manager for use in a :keyword:`with` statement.
891 The context manager provides a :class:`pathlib.Path` object.
892
893 Exiting the context manager cleans up any temporary file created when the
894 resource needs to be extracted from e.g. a zip file.
895
896 *package* is either a name or a module object which conforms to the
897 ``Package`` requirements. *resource* is the name of the resource to open
898 within *package*; it may not contain path separators and it may not have
899 sub-resources (i.e. it cannot be a directory).
900
901
902.. function:: is_resource(package, name)
903
904 Return ``True`` if there is a resource named *name* in the package,
905 otherwise ``False``. Remember that directories are *not* resources!
906 *package* is either a name or a module object which conforms to the
907 ``Package`` requirements.
908
909
910.. function:: contents(package)
911
912 Return an iterator over the named items within the package. The iterator
913 returns :class:`str` resources (e.g. files) and non-resources
914 (e.g. directories). The iterator does not recurse into subdirectories.
915
916 *package* is either a name or a module object which conforms to the
917 ``Package`` requirements.
918
919
Brett Cannon78246b62009-01-25 04:56:30 +0000920:mod:`importlib.machinery` -- Importers and path hooks
921------------------------------------------------------
922
923.. module:: importlib.machinery
924 :synopsis: Importers and path hooks
925
Terry Jan Reedydcb6c882016-06-22 22:46:34 -0400926**Source code:** :source:`Lib/importlib/machinery.py`
927
928--------------
929
Brett Cannon78246b62009-01-25 04:56:30 +0000930This module contains the various objects that help :keyword:`import`
931find and load modules.
932
Brett Cannoncb66eb02012-05-11 12:58:42 -0400933.. attribute:: SOURCE_SUFFIXES
934
935 A list of strings representing the recognized file suffixes for source
936 modules.
937
938 .. versionadded:: 3.3
939
940.. attribute:: DEBUG_BYTECODE_SUFFIXES
941
942 A list of strings representing the file suffixes for non-optimized bytecode
943 modules.
944
945 .. versionadded:: 3.3
946
Brett Cannonf299abd2015-04-13 14:21:02 -0400947 .. deprecated:: 3.5
948 Use :attr:`BYTECODE_SUFFIXES` instead.
949
Brett Cannoncb66eb02012-05-11 12:58:42 -0400950.. attribute:: OPTIMIZED_BYTECODE_SUFFIXES
951
952 A list of strings representing the file suffixes for optimized bytecode
953 modules.
954
955 .. versionadded:: 3.3
956
Brett Cannonf299abd2015-04-13 14:21:02 -0400957 .. deprecated:: 3.5
958 Use :attr:`BYTECODE_SUFFIXES` instead.
959
Brett Cannoncb66eb02012-05-11 12:58:42 -0400960.. attribute:: BYTECODE_SUFFIXES
961
962 A list of strings representing the recognized file suffixes for bytecode
Brett Cannonf299abd2015-04-13 14:21:02 -0400963 modules (including the leading dot).
Brett Cannoncb66eb02012-05-11 12:58:42 -0400964
965 .. versionadded:: 3.3
966
Brett Cannonf299abd2015-04-13 14:21:02 -0400967 .. versionchanged:: 3.5
968 The value is no longer dependent on ``__debug__``.
969
Brett Cannoncb66eb02012-05-11 12:58:42 -0400970.. attribute:: EXTENSION_SUFFIXES
971
Nick Coghlan76e07702012-07-18 23:14:57 +1000972 A list of strings representing the recognized file suffixes for
Brett Cannoncb66eb02012-05-11 12:58:42 -0400973 extension modules.
974
975 .. versionadded:: 3.3
976
Nick Coghlanc5afd422012-07-18 23:59:08 +1000977.. function:: all_suffixes()
Nick Coghlan76e07702012-07-18 23:14:57 +1000978
979 Returns a combined list of strings representing all file suffixes for
Nick Coghlanc5afd422012-07-18 23:59:08 +1000980 modules recognized by the standard import machinery. This is a
Nick Coghlan76e07702012-07-18 23:14:57 +1000981 helper for code which simply needs to know if a filesystem path
Nick Coghlanc5afd422012-07-18 23:59:08 +1000982 potentially refers to a module without needing any details on the kind
Martin Panterd21e0b52015-10-10 10:36:22 +0000983 of module (for example, :func:`inspect.getmodulename`).
Nick Coghlan76e07702012-07-18 23:14:57 +1000984
985 .. versionadded:: 3.3
986
987
Brett Cannon78246b62009-01-25 04:56:30 +0000988.. class:: BuiltinImporter
989
Brett Cannon2a922ed2009-03-09 03:35:50 +0000990 An :term:`importer` for built-in modules. All known built-in modules are
991 listed in :data:`sys.builtin_module_names`. This class implements the
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000992 :class:`importlib.abc.MetaPathFinder` and
993 :class:`importlib.abc.InspectLoader` ABCs.
Brett Cannon78246b62009-01-25 04:56:30 +0000994
995 Only class methods are defined by this class to alleviate the need for
996 instantiation.
997
Nick Coghland5cacbb2015-05-23 22:24:10 +1000998 .. versionchanged:: 3.5
999 As part of :pep:`489`, the builtin importer now implements
1000 :meth:`Loader.create_module` and :meth:`Loader.exec_module`
Eric Snowca2d8542013-12-16 23:06:52 -07001001
Brett Cannon78246b62009-01-25 04:56:30 +00001002
1003.. class:: FrozenImporter
1004
Brett Cannon2a922ed2009-03-09 03:35:50 +00001005 An :term:`importer` for frozen modules. This class implements the
Nick Coghlan8a9080f2012-08-02 21:26:03 +10001006 :class:`importlib.abc.MetaPathFinder` and
1007 :class:`importlib.abc.InspectLoader` ABCs.
Brett Cannon78246b62009-01-25 04:56:30 +00001008
1009 Only class methods are defined by this class to alleviate the need for
1010 instantiation.
1011
Brett Cannondebb98d2009-02-16 04:18:01 +00001012
Nick Coghlanff794862012-08-02 21:45:24 +10001013.. class:: WindowsRegistryFinder
1014
1015 :term:`Finder` for modules declared in the Windows registry. This class
Nick Coghlan49417742012-08-02 23:03:58 +10001016 implements the :class:`importlib.abc.Finder` ABC.
Nick Coghlanff794862012-08-02 21:45:24 +10001017
1018 Only class methods are defined by this class to alleviate the need for
1019 instantiation.
1020
1021 .. versionadded:: 3.3
1022
Steve Dower20367422016-12-07 13:02:27 -08001023 .. deprecated:: 3.6
1024 Use :mod:`site` configuration instead. Future versions of Python may
1025 not enable this finder by default.
1026
Nick Coghlanff794862012-08-02 21:45:24 +10001027
Brett Cannondebb98d2009-02-16 04:18:01 +00001028.. class:: PathFinder
1029
Brett Cannon1b799182012-08-17 14:08:24 -04001030 A :term:`Finder` for :data:`sys.path` and package ``__path__`` attributes.
1031 This class implements the :class:`importlib.abc.MetaPathFinder` ABC.
Brett Cannondebb98d2009-02-16 04:18:01 +00001032
Brett Cannon1b799182012-08-17 14:08:24 -04001033 Only class methods are defined by this class to alleviate the need for
1034 instantiation.
Brett Cannondebb98d2009-02-16 04:18:01 +00001035
Eric Snowca2d8542013-12-16 23:06:52 -07001036 .. classmethod:: find_spec(fullname, path=None, target=None)
1037
1038 Class method that attempts to find a :term:`spec <module spec>`
1039 for the module specified by *fullname* on :data:`sys.path` or, if
1040 defined, on *path*. For each path entry that is searched,
1041 :data:`sys.path_importer_cache` is checked. If a non-false object
1042 is found then it is used as the :term:`path entry finder` to look
1043 for the module being searched for. If no entry is found in
1044 :data:`sys.path_importer_cache`, then :data:`sys.path_hooks` is
1045 searched for a finder for the path entry and, if found, is stored
1046 in :data:`sys.path_importer_cache` along with being queried about
1047 the module. If no finder is ever found then ``None`` is both
1048 stored in the cache and returned.
1049
1050 .. versionadded:: 3.4
1051
Brett Cannonb6e25562014-11-21 12:19:28 -05001052 .. versionchanged:: 3.5
1053 If the current working directory -- represented by an empty string --
1054 is no longer valid then ``None`` is returned but no value is cached
1055 in :data:`sys.path_importer_cache`.
1056
Brett Cannon1b799182012-08-17 14:08:24 -04001057 .. classmethod:: find_module(fullname, path=None)
Brett Cannondebb98d2009-02-16 04:18:01 +00001058
Eric Snowca2d8542013-12-16 23:06:52 -07001059 A legacy wrapper around :meth:`find_spec`.
1060
1061 .. deprecated:: 3.4
1062 Use :meth:`find_spec` instead.
Brett Cannond2e7b332009-02-17 02:45:03 +00001063
Brett Cannonf4dc9202012-08-10 12:21:12 -04001064 .. classmethod:: invalidate_caches()
1065
Eric Snowca2d8542013-12-16 23:06:52 -07001066 Calls :meth:`importlib.abc.PathEntryFinder.invalidate_caches` on all
1067 finders stored in :attr:`sys.path_importer_cache`.
Brett Cannonf4dc9202012-08-10 12:21:12 -04001068
Eric Snowca2d8542013-12-16 23:06:52 -07001069 .. versionchanged:: 3.4
1070 Calls objects in :data:`sys.path_hooks` with the current working
1071 directory for ``''`` (i.e. the empty string).
Brett Cannon27e27f72013-10-18 11:39:04 -04001072
Brett Cannond2e7b332009-02-17 02:45:03 +00001073
Brett Cannon938d44d2012-04-22 19:58:33 -04001074.. class:: FileFinder(path, \*loader_details)
1075
Nick Coghlan8a9080f2012-08-02 21:26:03 +10001076 A concrete implementation of :class:`importlib.abc.PathEntryFinder` which
1077 caches results from the file system.
Brett Cannon938d44d2012-04-22 19:58:33 -04001078
1079 The *path* argument is the directory for which the finder is in charge of
1080 searching.
1081
Brett Cannonac9f2f32012-08-10 13:47:54 -04001082 The *loader_details* argument is a variable number of 2-item tuples each
1083 containing a loader and a sequence of file suffixes the loader recognizes.
Brett Cannon29b2f172013-06-21 18:31:55 -04001084 The loaders are expected to be callables which accept two arguments of
1085 the module's name and the path to the file found.
Brett Cannon938d44d2012-04-22 19:58:33 -04001086
1087 The finder will cache the directory contents as necessary, making stat calls
1088 for each module search to verify the cache is not outdated. Because cache
1089 staleness relies upon the granularity of the operating system's state
1090 information of the file system, there is a potential race condition of
1091 searching for a module, creating a new file, and then searching for the
1092 module the new file represents. If the operations happen fast enough to fit
1093 within the granularity of stat calls, then the module search will fail. To
1094 prevent this from happening, when you create a module dynamically, make sure
1095 to call :func:`importlib.invalidate_caches`.
1096
1097 .. versionadded:: 3.3
1098
1099 .. attribute:: path
1100
1101 The path the finder will search in.
1102
Eric Snowca2d8542013-12-16 23:06:52 -07001103 .. method:: find_spec(fullname, target=None)
1104
1105 Attempt to find the spec to handle *fullname* within :attr:`path`.
1106
1107 .. versionadded:: 3.4
1108
Brett Cannon1d753822013-06-16 19:06:55 -04001109 .. method:: find_loader(fullname)
Brett Cannon938d44d2012-04-22 19:58:33 -04001110
1111 Attempt to find the loader to handle *fullname* within :attr:`path`.
1112
1113 .. method:: invalidate_caches()
1114
1115 Clear out the internal cache.
1116
1117 .. classmethod:: path_hook(\*loader_details)
1118
1119 A class method which returns a closure for use on :attr:`sys.path_hooks`.
1120 An instance of :class:`FileFinder` is returned by the closure using the
1121 path argument given to the closure directly and *loader_details*
1122 indirectly.
1123
1124 If the argument to the closure is not an existing directory,
1125 :exc:`ImportError` is raised.
1126
1127
1128.. class:: SourceFileLoader(fullname, path)
1129
1130 A concrete implementation of :class:`importlib.abc.SourceLoader` by
1131 subclassing :class:`importlib.abc.FileLoader` and providing some concrete
1132 implementations of other methods.
1133
1134 .. versionadded:: 3.3
1135
1136 .. attribute:: name
1137
1138 The name of the module that this loader will handle.
1139
1140 .. attribute:: path
1141
1142 The path to the source file.
1143
1144 .. method:: is_package(fullname)
1145
1146 Return true if :attr:`path` appears to be for a package.
1147
1148 .. method:: path_stats(path)
1149
1150 Concrete implementation of :meth:`importlib.abc.SourceLoader.path_stats`.
1151
1152 .. method:: set_data(path, data)
1153
1154 Concrete implementation of :meth:`importlib.abc.SourceLoader.set_data`.
1155
Brett Cannon062fcac2014-05-09 11:55:49 -04001156 .. method:: load_module(name=None)
1157
1158 Concrete implementation of :meth:`importlib.abc.Loader.load_module` where
1159 specifying the name of the module to load is optional.
1160
Brett Cannoneae30792015-12-28 17:55:27 -08001161 .. deprecated:: 3.6
1162
1163 Use :meth:`importlib.abc.Loader.exec_module` instead.
1164
Brett Cannon938d44d2012-04-22 19:58:33 -04001165
Marc-Andre Lemburg4fe29c92012-04-25 02:31:37 +02001166.. class:: SourcelessFileLoader(fullname, path)
Brett Cannon938d44d2012-04-22 19:58:33 -04001167
1168 A concrete implementation of :class:`importlib.abc.FileLoader` which can
1169 import bytecode files (i.e. no source code files exist).
1170
Marc-Andre Lemburg4fe29c92012-04-25 02:31:37 +02001171 Please note that direct use of bytecode files (and thus not source code
1172 files) inhibits your modules from being usable by all Python
1173 implementations or new versions of Python which change the bytecode
1174 format.
Brett Cannon938d44d2012-04-22 19:58:33 -04001175
1176 .. versionadded:: 3.3
1177
1178 .. attribute:: name
1179
1180 The name of the module the loader will handle.
1181
1182 .. attribute:: path
1183
1184 The path to the bytecode file.
1185
1186 .. method:: is_package(fullname)
1187
1188 Determines if the module is a package based on :attr:`path`.
1189
1190 .. method:: get_code(fullname)
1191
1192 Returns the code object for :attr:`name` created from :attr:`path`.
1193
1194 .. method:: get_source(fullname)
1195
1196 Returns ``None`` as bytecode files have no source when this loader is
1197 used.
1198
Brett Cannon062fcac2014-05-09 11:55:49 -04001199 .. method:: load_module(name=None)
1200
1201 Concrete implementation of :meth:`importlib.abc.Loader.load_module` where
1202 specifying the name of the module to load is optional.
1203
Brett Cannoneae30792015-12-28 17:55:27 -08001204 .. deprecated:: 3.6
1205
1206 Use :meth:`importlib.abc.Loader.exec_module` instead.
1207
Brett Cannon938d44d2012-04-22 19:58:33 -04001208
1209.. class:: ExtensionFileLoader(fullname, path)
1210
Eric Snow51794452013-10-03 12:08:55 -06001211 A concrete implementation of :class:`importlib.abc.ExecutionLoader` for
Brett Cannon938d44d2012-04-22 19:58:33 -04001212 extension modules.
1213
1214 The *fullname* argument specifies the name of the module the loader is to
1215 support. The *path* argument is the path to the extension module's file.
1216
1217 .. versionadded:: 3.3
1218
1219 .. attribute:: name
1220
1221 Name of the module the loader supports.
1222
1223 .. attribute:: path
1224
1225 Path to the extension module.
1226
Nick Coghland5cacbb2015-05-23 22:24:10 +10001227 .. method:: create_module(spec)
Brett Cannon938d44d2012-04-22 19:58:33 -04001228
Nick Coghland5cacbb2015-05-23 22:24:10 +10001229 Creates the module object from the given specification in accordance
1230 with :pep:`489`.
Brett Cannon938d44d2012-04-22 19:58:33 -04001231
Nick Coghland5cacbb2015-05-23 22:24:10 +10001232 .. versionadded:: 3.5
1233
1234 .. method:: exec_module(module)
1235
1236 Initializes the given module object in accordance with :pep:`489`.
1237
1238 .. versionadded:: 3.5
Eric Snowca2d8542013-12-16 23:06:52 -07001239
Brett Cannon938d44d2012-04-22 19:58:33 -04001240 .. method:: is_package(fullname)
1241
Brett Cannonac9f2f32012-08-10 13:47:54 -04001242 Returns ``True`` if the file path points to a package's ``__init__``
1243 module based on :attr:`EXTENSION_SUFFIXES`.
Brett Cannon938d44d2012-04-22 19:58:33 -04001244
1245 .. method:: get_code(fullname)
1246
1247 Returns ``None`` as extension modules lack a code object.
1248
1249 .. method:: get_source(fullname)
1250
1251 Returns ``None`` as extension modules do not have source code.
1252
Eric Snow51794452013-10-03 12:08:55 -06001253 .. method:: get_filename(fullname)
1254
1255 Returns :attr:`path`.
1256
Eric Snowdcd01b42013-10-04 20:35:34 -06001257 .. versionadded:: 3.4
1258
Brett Cannon938d44d2012-04-22 19:58:33 -04001259
Eric Snowb523f842013-11-22 09:05:39 -07001260.. class:: ModuleSpec(name, loader, *, origin=None, loader_state=None, is_package=None)
1261
Barry Warsaw191e3132017-10-17 15:52:38 -04001262 A specification for a module's import-system-related state. This is
1263 typically exposed as the module's ``__spec__`` attribute. In the
1264 descriptions below, the names in parentheses give the corresponding
1265 attribute available directly on the module object.
1266 E.g. ``module.__spec__.origin == module.__file__``. Note however that
1267 while the *values* are usually equivalent, they can differ since there is
1268 no synchronization between the two objects. Thus it is possible to update
1269 the module's ``__path__`` at runtime, and this will not be automatically
1270 reflected in ``__spec__.submodule_search_locations``.
Eric Snowb523f842013-11-22 09:05:39 -07001271
1272 .. versionadded:: 3.4
1273
1274 .. attribute:: name
1275
1276 (``__name__``)
1277
1278 A string for the fully-qualified name of the module.
1279
1280 .. attribute:: loader
1281
1282 (``__loader__``)
1283
1284 The loader to use for loading. For namespace packages this should be
Serhiy Storchakaecf41da2016-10-19 16:29:26 +03001285 set to ``None``.
Eric Snowb523f842013-11-22 09:05:39 -07001286
1287 .. attribute:: origin
1288
1289 (``__file__``)
1290
1291 Name of the place from which the module is loaded, e.g. "builtin" for
1292 built-in modules and the filename for modules loaded from source.
Serhiy Storchakaecf41da2016-10-19 16:29:26 +03001293 Normally "origin" should be set, but it may be ``None`` (the default)
Barry Warsawa23d30f2018-02-02 19:49:25 -05001294 which indicates it is unspecified (e.g. for namespace packages).
Eric Snowb523f842013-11-22 09:05:39 -07001295
1296 .. attribute:: submodule_search_locations
1297
1298 (``__path__``)
1299
Serhiy Storchakaecf41da2016-10-19 16:29:26 +03001300 List of strings for where to find submodules, if a package (``None``
Eric Snowb523f842013-11-22 09:05:39 -07001301 otherwise).
1302
1303 .. attribute:: loader_state
1304
1305 Container of extra module-specific data for use during loading (or
Serhiy Storchakaecf41da2016-10-19 16:29:26 +03001306 ``None``).
Eric Snowb523f842013-11-22 09:05:39 -07001307
1308 .. attribute:: cached
1309
1310 (``__cached__``)
1311
Serhiy Storchakaecf41da2016-10-19 16:29:26 +03001312 String for where the compiled module should be stored (or ``None``).
Eric Snowb523f842013-11-22 09:05:39 -07001313
1314 .. attribute:: parent
1315
1316 (``__package__``)
1317
1318 (Read-only) Fully-qualified name of the package to which the module
Serhiy Storchakaecf41da2016-10-19 16:29:26 +03001319 belongs as a submodule (or ``None``).
Eric Snowb523f842013-11-22 09:05:39 -07001320
1321 .. attribute:: has_location
1322
Eric Snowb282b3d2013-12-10 22:16:41 -07001323 Boolean indicating whether or not the module's "origin"
Eric Snowb523f842013-11-22 09:05:39 -07001324 attribute refers to a loadable location.
1325
Brett Cannond2e7b332009-02-17 02:45:03 +00001326:mod:`importlib.util` -- Utility code for importers
1327---------------------------------------------------
1328
1329.. module:: importlib.util
Brett Cannon75321e82012-03-02 11:58:25 -05001330 :synopsis: Utility code for importers
Brett Cannond2e7b332009-02-17 02:45:03 +00001331
Terry Jan Reedydcb6c882016-06-22 22:46:34 -04001332
1333**Source code:** :source:`Lib/importlib/util.py`
1334
1335--------------
1336
Brett Cannond2e7b332009-02-17 02:45:03 +00001337This module contains the various objects that help in the construction of
1338an :term:`importer`.
1339
Brett Cannon05a647d2013-06-14 19:02:34 -04001340.. attribute:: MAGIC_NUMBER
1341
1342 The bytes which represent the bytecode version number. If you need help with
1343 loading/writing bytecode then consider :class:`importlib.abc.SourceLoader`.
1344
1345 .. versionadded:: 3.4
1346
Brett Cannonf299abd2015-04-13 14:21:02 -04001347.. function:: cache_from_source(path, debug_override=None, *, optimization=None)
Brett Cannona3c96152013-06-14 22:26:30 -04001348
Brett Cannonf299abd2015-04-13 14:21:02 -04001349 Return the :pep:`3147`/:pep:`488` path to the byte-compiled file associated
1350 with the source *path*. For example, if *path* is ``/foo/bar/baz.py`` the return
Brett Cannona3c96152013-06-14 22:26:30 -04001351 value would be ``/foo/bar/__pycache__/baz.cpython-32.pyc`` for Python 3.2.
1352 The ``cpython-32`` string comes from the current magic tag (see
1353 :func:`get_tag`; if :attr:`sys.implementation.cache_tag` is not defined then
Brett Cannonf299abd2015-04-13 14:21:02 -04001354 :exc:`NotImplementedError` will be raised).
Brett Cannona3c96152013-06-14 22:26:30 -04001355
Brett Cannonf299abd2015-04-13 14:21:02 -04001356 The *optimization* parameter is used to specify the optimization level of the
1357 bytecode file. An empty string represents no optimization, so
1358 ``/foo/bar/baz.py`` with an *optimization* of ``''`` will result in a
1359 bytecode path of ``/foo/bar/__pycache__/baz.cpython-32.pyc``. ``None`` causes
1360 the interpter's optimization level to be used. Any other value's string
1361 representation being used, so ``/foo/bar/baz.py`` with an *optimization* of
1362 ``2`` will lead to the bytecode path of
1363 ``/foo/bar/__pycache__/baz.cpython-32.opt-2.pyc``. The string representation
1364 of *optimization* can only be alphanumeric, else :exc:`ValueError` is raised.
1365
1366 The *debug_override* parameter is deprecated and can be used to override
1367 the system's value for ``__debug__``. A ``True`` value is the equivalent of
1368 setting *optimization* to the empty string. A ``False`` value is the same as
1369 setting *optimization* to ``1``. If both *debug_override* an *optimization*
1370 are not ``None`` then :exc:`TypeError` is raised.
Brett Cannona3c96152013-06-14 22:26:30 -04001371
1372 .. versionadded:: 3.4
1373
Berker Peksagfe5f6142016-01-30 19:30:06 +02001374 .. versionchanged:: 3.5
Brett Cannonf299abd2015-04-13 14:21:02 -04001375 The *optimization* parameter was added and the *debug_override* parameter
1376 was deprecated.
1377
Brett Cannon035a1002016-09-07 18:39:18 -07001378 .. versionchanged:: 3.6
1379 Accepts a :term:`path-like object`.
1380
Brett Cannona3c96152013-06-14 22:26:30 -04001381
1382.. function:: source_from_cache(path)
1383
1384 Given the *path* to a :pep:`3147` file name, return the associated source code
1385 file path. For example, if *path* is
1386 ``/foo/bar/__pycache__/baz.cpython-32.pyc`` the returned path would be
1387 ``/foo/bar/baz.py``. *path* need not exist, however if it does not conform
Larry Hastings770ce202015-04-19 13:50:12 -07001388 to :pep:`3147` or :pep:`488` format, a ``ValueError`` is raised. If
Brett Cannona3c96152013-06-14 22:26:30 -04001389 :attr:`sys.implementation.cache_tag` is not defined,
1390 :exc:`NotImplementedError` is raised.
1391
1392 .. versionadded:: 3.4
1393
Brett Cannon035a1002016-09-07 18:39:18 -07001394 .. versionchanged:: 3.6
1395 Accepts a :term:`path-like object`.
1396
Brett Cannonf24fecd2013-06-16 18:37:53 -04001397.. function:: decode_source(source_bytes)
1398
1399 Decode the given bytes representing source code and return it as a string
1400 with universal newlines (as required by
1401 :meth:`importlib.abc.InspectLoader.get_source`).
1402
1403 .. versionadded:: 3.4
1404
Brett Cannond200bf52012-05-13 13:45:09 -04001405.. function:: resolve_name(name, package)
1406
1407 Resolve a relative module name to an absolute one.
1408
1409 If **name** has no leading dots, then **name** is simply returned. This
1410 allows for usage such as
1411 ``importlib.util.resolve_name('sys', __package__)`` without doing a
1412 check to see if the **package** argument is needed.
1413
1414 :exc:`ValueError` is raised if **name** is a relative module name but
1415 package is a false value (e.g. ``None`` or the empty string).
1416 :exc:`ValueError` is also raised a relative name would escape its containing
1417 package (e.g. requesting ``..bacon`` from within the ``spam`` package).
1418
1419 .. versionadded:: 3.3
1420
Eric Snow6029e082014-01-25 15:32:46 -07001421.. function:: find_spec(name, package=None)
1422
1423 Find the :term:`spec <module spec>` for a module, optionally relative to
1424 the specified **package** name. If the module is in :attr:`sys.modules`,
1425 then ``sys.modules[name].__spec__`` is returned (unless the spec would be
1426 ``None`` or is not set, in which case :exc:`ValueError` is raised).
1427 Otherwise a search using :attr:`sys.meta_path` is done. ``None`` is
1428 returned if no spec is found.
1429
1430 If **name** is for a submodule (contains a dot), the parent module is
1431 automatically imported.
1432
1433 **name** and **package** work the same as for :func:`import_module`.
1434
1435 .. versionadded:: 3.4
1436
Milan Oberkirch8c3f05e2017-06-15 07:34:50 +10001437 .. versionchanged:: 3.7
1438 Raises :exc:`ModuleNotFoundError` instead of :exc:`AttributeError` if
1439 **package** is in fact not a package (i.e. lacks a :attr:`__path__`
1440 attribute).
1441
Brett Cannon2a17bde2014-05-30 14:55:29 -04001442.. function:: module_from_spec(spec)
1443
Brett Cannon696c35e2016-06-25 10:58:17 -07001444 Create a new module based on **spec** and
1445 :meth:`spec.loader.create_module <importlib.abc.Loader.create_module>`.
Brett Cannon2a17bde2014-05-30 14:55:29 -04001446
Brett Cannon696c35e2016-06-25 10:58:17 -07001447 If :meth:`spec.loader.create_module <importlib.abc.Loader.create_module>`
1448 does not return ``None``, then any pre-existing attributes will not be reset.
1449 Also, no :exc:`AttributeError` will be raised if triggered while accessing
1450 **spec** or setting an attribute on the module.
Brett Cannon2a17bde2014-05-30 14:55:29 -04001451
1452 This function is preferred over using :class:`types.ModuleType` to create a
1453 new module as **spec** is used to set as many import-controlled attributes on
1454 the module as possible.
1455
1456 .. versionadded:: 3.5
1457
Georg Brandl8a1caa22010-07-29 16:01:11 +00001458.. decorator:: module_for_loader
Brett Cannond2e7b332009-02-17 02:45:03 +00001459
Brett Cannona22faca2013-05-28 17:50:14 -04001460 A :term:`decorator` for :meth:`importlib.abc.Loader.load_module`
Guido van Rossum09613542009-03-30 20:34:57 +00001461 to handle selecting the proper
Brett Cannond2e7b332009-02-17 02:45:03 +00001462 module object to load with. The decorated method is expected to have a call
Brett Cannon2a922ed2009-03-09 03:35:50 +00001463 signature taking two positional arguments
1464 (e.g. ``load_module(self, module)``) for which the second argument
Guido van Rossum09613542009-03-30 20:34:57 +00001465 will be the module **object** to be used by the loader.
Brett Cannonefad00d2012-04-27 17:27:14 -04001466 Note that the decorator will not work on static methods because of the
1467 assumption of two arguments.
Brett Cannond2e7b332009-02-17 02:45:03 +00001468
Guido van Rossum09613542009-03-30 20:34:57 +00001469 The decorated method will take in the **name** of the module to be loaded
1470 as expected for a :term:`loader`. If the module is not found in
Brett Cannon3dc48d62013-05-28 18:35:54 -04001471 :data:`sys.modules` then a new one is constructed. Regardless of where the
1472 module came from, :attr:`__loader__` set to **self** and :attr:`__package__`
1473 is set based on what :meth:`importlib.abc.InspectLoader.is_package` returns
1474 (if available). These attributes are set unconditionally to support
1475 reloading.
Brett Cannonefad00d2012-04-27 17:27:14 -04001476
1477 If an exception is raised by the decorated method and a module was added to
Brett Cannona87e31c2013-09-13 16:52:19 -04001478 :data:`sys.modules`, then the module will be removed to prevent a partially
1479 initialized module from being in left in :data:`sys.modules`. If the module
1480 was already in :data:`sys.modules` then it is left alone.
Brett Cannond2e7b332009-02-17 02:45:03 +00001481
Brett Cannonefad00d2012-04-27 17:27:14 -04001482 .. versionchanged:: 3.3
Georg Brandl61063cc2012-06-24 22:48:30 +02001483 :attr:`__loader__` and :attr:`__package__` are automatically set
1484 (when possible).
Brett Cannon57b46f52009-03-02 14:38:26 +00001485
Brett Cannon3dc48d62013-05-28 18:35:54 -04001486 .. versionchanged:: 3.4
Brett Cannon0dbb4c72013-05-31 18:56:47 -04001487 Set :attr:`__name__`, :attr:`__loader__` :attr:`__package__`
1488 unconditionally to support reloading.
1489
1490 .. deprecated:: 3.4
Eric Snowb523f842013-11-22 09:05:39 -07001491 The import machinery now directly performs all the functionality
1492 provided by this function.
Brett Cannon3dc48d62013-05-28 18:35:54 -04001493
Georg Brandl8a1caa22010-07-29 16:01:11 +00001494.. decorator:: set_loader
Brett Cannon2cf03a82009-03-10 05:17:37 +00001495
Brett Cannona22faca2013-05-28 17:50:14 -04001496 A :term:`decorator` for :meth:`importlib.abc.Loader.load_module`
1497 to set the :attr:`__loader__`
1498 attribute on the returned module. If the attribute is already set the
1499 decorator does nothing. It is assumed that the first positional argument to
1500 the wrapped method (i.e. ``self``) is what :attr:`__loader__` should be set
1501 to.
Brett Cannon2cf03a82009-03-10 05:17:37 +00001502
Brett Cannon4802bec2013-03-13 10:41:36 -07001503 .. versionchanged:: 3.4
Brett Cannon4c14b5d2013-05-04 13:56:58 -04001504 Set ``__loader__`` if set to ``None``, as if the attribute does not
Brett Cannon4802bec2013-03-13 10:41:36 -07001505 exist.
1506
Eric Snowca2d8542013-12-16 23:06:52 -07001507 .. deprecated:: 3.4
1508 The import machinery takes care of this automatically.
1509
Georg Brandl8a1caa22010-07-29 16:01:11 +00001510.. decorator:: set_package
Brett Cannon57b46f52009-03-02 14:38:26 +00001511
Brett Cannon696c35e2016-06-25 10:58:17 -07001512 A :term:`decorator` for :meth:`importlib.abc.Loader.load_module` to set the
1513 :attr:`__package__` attribute on the returned module. If :attr:`__package__`
Brett Cannona22faca2013-05-28 17:50:14 -04001514 is set and has a value other than ``None`` it will not be changed.
Brett Cannon16248a42009-04-01 20:47:14 +00001515
Eric Snowca2d8542013-12-16 23:06:52 -07001516 .. deprecated:: 3.4
1517 The import machinery takes care of this automatically.
1518
Eric Snowb523f842013-11-22 09:05:39 -07001519.. function:: spec_from_loader(name, loader, *, origin=None, is_package=None)
1520
1521 A factory function for creating a :class:`ModuleSpec` instance based
1522 on a loader. The parameters have the same meaning as they do for
1523 ModuleSpec. The function uses available :term:`loader` APIs, such as
1524 :meth:`InspectLoader.is_package`, to fill in any missing
1525 information on the spec.
1526
1527 .. versionadded:: 3.4
1528
1529.. function:: spec_from_file_location(name, location, *, loader=None, submodule_search_locations=None)
1530
1531 A factory function for creating a :class:`ModuleSpec` instance based
1532 on the path to a file. Missing information will be filled in on the
1533 spec by making use of loader APIs and by the implication that the
1534 module will be file-based.
1535
1536 .. versionadded:: 3.4
Brett Cannona04dbe42014-04-04 13:53:38 -04001537
Brett Cannon035a1002016-09-07 18:39:18 -07001538 .. versionchanged:: 3.6
1539 Accepts a :term:`path-like object`.
1540
Benjamin Peterson42aa93b2017-12-09 10:26:52 -08001541.. function:: source_hash(source_bytes)
1542
1543 Return the hash of *source_bytes* as bytes. A hash-based ``.pyc`` file embeds
1544 the :func:`source_hash` of the corresponding source file's contents in its
1545 header.
1546
1547 .. versionadded:: 3.7
1548
Brett Cannona04dbe42014-04-04 13:53:38 -04001549.. class:: LazyLoader(loader)
1550
1551 A class which postpones the execution of the loader of a module until the
1552 module has an attribute accessed.
1553
1554 This class **only** works with loaders that define
Brett Cannon02d84542015-01-09 11:39:21 -05001555 :meth:`~importlib.abc.Loader.exec_module` as control over what module type
1556 is used for the module is required. For those same reasons, the loader's
Brett Cannon696c35e2016-06-25 10:58:17 -07001557 :meth:`~importlib.abc.Loader.create_module` method must return ``None`` or a
1558 type for which its ``__class__`` attribute can be mutated along with not
1559 using :term:`slots <__slots__>`. Finally, modules which substitute the object
1560 placed into :attr:`sys.modules` will not work as there is no way to properly
1561 replace the module references throughout the interpreter safely;
1562 :exc:`ValueError` is raised if such a substitution is detected.
Brett Cannona04dbe42014-04-04 13:53:38 -04001563
1564 .. note::
1565 For projects where startup time is critical, this class allows for
1566 potentially minimizing the cost of loading a module if it is never used.
1567 For projects where startup time is not essential then use of this class is
1568 **heavily** discouraged due to error messages created during loading being
1569 postponed and thus occurring out of context.
1570
1571 .. versionadded:: 3.5
1572
Brett Cannon696c35e2016-06-25 10:58:17 -07001573 .. versionchanged:: 3.6
1574 Began calling :meth:`~importlib.abc.Loader.create_module`, removing the
1575 compatibility warning for :class:`importlib.machinery.BuiltinImporter` and
1576 :class:`importlib.machinery.ExtensionFileLoader`.
1577
Brett Cannona04dbe42014-04-04 13:53:38 -04001578 .. classmethod:: factory(loader)
1579
1580 A static method which returns a callable that creates a lazy loader. This
1581 is meant to be used in situations where the loader is passed by class
1582 instead of by instance.
1583 ::
1584
1585 suffixes = importlib.machinery.SOURCE_SUFFIXES
1586 loader = importlib.machinery.SourceFileLoader
1587 lazy_loader = importlib.util.LazyLoader.factory(loader)
Brett Cannon3bf1d872016-01-22 14:03:27 -08001588 finder = importlib.machinery.FileFinder(path, (lazy_loader, suffixes))
Brett Cannona85e9272016-01-08 14:33:09 -08001589
1590.. _importlib-examples:
1591
1592Examples
1593--------
1594
Brett Cannon23763162016-09-08 10:12:47 -07001595Importing programmatically
1596''''''''''''''''''''''''''
1597
Brett Cannona85e9272016-01-08 14:33:09 -08001598To programmatically import a module, use :func:`importlib.import_module`.
1599::
1600
1601 import importlib
1602
1603 itertools = importlib.import_module('itertools')
1604
Brett Cannon23763162016-09-08 10:12:47 -07001605
1606Checking if a module can be imported
1607''''''''''''''''''''''''''''''''''''
1608
Brett Cannona85e9272016-01-08 14:33:09 -08001609If you need to find out if a module can be imported without actually doing the
1610import, then you should use :func:`importlib.util.find_spec`.
1611::
1612
1613 import importlib.util
1614 import sys
1615
1616 # For illustrative purposes.
1617 name = 'itertools'
1618
1619 spec = importlib.util.find_spec(name)
1620 if spec is None:
1621 print("can't find the itertools module")
1622 else:
Brett Cannon59363132016-03-18 11:54:22 -07001623 # If you chose to perform the actual import ...
Brett Cannona85e9272016-01-08 14:33:09 -08001624 module = importlib.util.module_from_spec(spec)
1625 spec.loader.exec_module(module)
1626 # Adding the module to sys.modules is optional.
1627 sys.modules[name] = module
1628
Brett Cannon23763162016-09-08 10:12:47 -07001629
1630Importing a source file directly
1631''''''''''''''''''''''''''''''''
1632
Brett Cannona85e9272016-01-08 14:33:09 -08001633To import a Python source file directly, use the following recipe
1634(Python 3.4 and newer only)::
1635
1636 import importlib.util
1637 import sys
1638
1639 # For illustrative purposes.
1640 import tokenize
1641 file_path = tokenize.__file__
1642 module_name = tokenize.__name__
1643
1644 spec = importlib.util.spec_from_file_location(module_name, file_path)
1645 module = importlib.util.module_from_spec(spec)
1646 spec.loader.exec_module(module)
1647 # Optional; only necessary if you want to be able to import the module
1648 # by name later.
1649 sys.modules[module_name] = module
1650
Brett Cannon23763162016-09-08 10:12:47 -07001651
1652Setting up an importer
1653''''''''''''''''''''''
1654
Brett Cannon59363132016-03-18 11:54:22 -07001655For deep customizations of import, you typically want to implement an
1656:term:`importer`. This means managing both the :term:`finder` and :term:`loader`
1657side of things. For finders there are two flavours to choose from depending on
1658your needs: a :term:`meta path finder` or a :term:`path entry finder`. The
1659former is what you would put on :attr:`sys.meta_path` while the latter is what
1660you create using a :term:`path entry hook` on :attr:`sys.path_hooks` which works
1661with :attr:`sys.path` entries to potentially create a finder. This example will
1662show you how to register your own importers so that import will use them (for
1663creating an importer for yourself, read the documentation for the appropriate
1664classes defined within this package)::
1665
1666 import importlib.machinery
1667 import sys
1668
1669 # For illustrative purposes only.
1670 SpamMetaPathFinder = importlib.machinery.PathFinder
1671 SpamPathEntryFinder = importlib.machinery.FileFinder
1672 loader_details = (importlib.machinery.SourceFileLoader,
1673 importlib.machinery.SOURCE_SUFFIXES)
1674
1675 # Setting up a meta path finder.
1676 # Make sure to put the finder in the proper location in the list in terms of
1677 # priority.
1678 sys.meta_path.append(SpamMetaPathFinder)
1679
1680 # Setting up a path entry finder.
1681 # Make sure to put the path hook in the proper location in the list in terms
1682 # of priority.
1683 sys.path_hooks.append(SpamPathEntryFinder.path_hook(loader_details))
1684
Brett Cannon23763162016-09-08 10:12:47 -07001685
1686Approximating :func:`importlib.import_module`
1687'''''''''''''''''''''''''''''''''''''''''''''
1688
Brett Cannona85e9272016-01-08 14:33:09 -08001689Import itself is implemented in Python code, making it possible to
1690expose most of the import machinery through importlib. The following
1691helps illustrate the various APIs that importlib exposes by providing an
1692approximate implementation of
Brett Cannon59363132016-03-18 11:54:22 -07001693:func:`importlib.import_module` (Python 3.4 and newer for the importlib usage,
Brett Cannona85e9272016-01-08 14:33:09 -08001694Python 3.6 and newer for other parts of the code).
1695::
1696
1697 import importlib.util
1698 import sys
1699
1700 def import_module(name, package=None):
1701 """An approximate implementation of import."""
1702 absolute_name = importlib.util.resolve_name(name, package)
1703 try:
1704 return sys.modules[absolute_name]
1705 except KeyError:
1706 pass
1707
1708 path = None
1709 if '.' in absolute_name:
1710 parent_name, _, child_name = absolute_name.rpartition('.')
1711 parent_module = import_module(parent_name)
1712 path = parent_module.spec.submodule_search_locations
1713 for finder in sys.meta_path:
1714 spec = finder.find_spec(absolute_name, path)
1715 if spec is not None:
1716 break
1717 else:
1718 raise ImportError(f'No module named {absolute_name!r}')
Brett Cannon86a8be02016-02-20 18:47:09 -08001719 module = importlib.util.module_from_spec(spec)
Brett Cannona85e9272016-01-08 14:33:09 -08001720 spec.loader.exec_module(module)
1721 sys.modules[absolute_name] = module
1722 if path is not None:
1723 setattr(parent_module, child_name, module)
1724 return module