blob: 137721e4be13d25b0a3e6996038ec5fb255567b0 [file] [log] [blame]
Brett Cannonafccd632009-01-20 02:21:27 +00001:mod:`importlib` -- An implementation of :keyword:`import`
2==========================================================
3
4.. module:: importlib
5 :synopsis: An implementation of the import machinery.
6
7.. moduleauthor:: Brett Cannon <brett@python.org>
8.. sectionauthor:: Brett Cannon <brett@python.org>
9
10.. versionadded:: 3.1
11
12
13Introduction
14------------
15
16The purpose of the :mod:`importlib` package is two-fold. One is to provide an
17implementation of the :keyword:`import` statement (and thus, by extension, the
18:func:`__import__` function) in Python source code. This provides an
Tarek Ziadé434caaa2009-05-14 12:48:09 +000019implementation of :keyword:`import` which is portable to any Python
Brett Cannonafccd632009-01-20 02:21:27 +000020interpreter. This also provides a reference implementation which is easier to
Brett Cannonf23e3742010-06-27 23:57:46 +000021comprehend than one implemented in a programming language other than Python.
Brett Cannonafccd632009-01-20 02:21:27 +000022
Brett Cannonf23e3742010-06-27 23:57:46 +000023Two, the components to implement :keyword:`import` are exposed in this
Brett Cannonafccd632009-01-20 02:21:27 +000024package, making it easier for users to create their own custom objects (known
Brett Cannondebb98d2009-02-16 04:18:01 +000025generically as an :term:`importer`) to participate in the import process.
Brett Cannonf23e3742010-06-27 23:57:46 +000026Details on custom importers can be found in :pep:`302`.
Brett Cannonafccd632009-01-20 02:21:27 +000027
28.. seealso::
29
30 :ref:`import`
31 The language reference for the :keyword:`import` statement.
32
33 `Packages specification <http://www.python.org/doc/essays/packages.html>`__
34 Original specification of packages. Some semantics have changed since
Georg Brandl375aec22011-01-15 17:03:02 +000035 the writing of this document (e.g. redirecting based on ``None``
Brett Cannonafccd632009-01-20 02:21:27 +000036 in :data:`sys.modules`).
37
38 The :func:`.__import__` function
Brett Cannon0e13c942010-06-29 18:26:11 +000039 The :keyword:`import` statement is syntactic sugar for this function.
Brett Cannonafccd632009-01-20 02:21:27 +000040
41 :pep:`235`
42 Import on Case-Insensitive Platforms
43
44 :pep:`263`
45 Defining Python Source Code Encodings
46
47 :pep:`302`
Brett Cannonf23e3742010-06-27 23:57:46 +000048 New Import Hooks
Brett Cannonafccd632009-01-20 02:21:27 +000049
50 :pep:`328`
51 Imports: Multi-Line and Absolute/Relative
52
53 :pep:`366`
54 Main module explicit relative imports
55
Brett Cannon8917d5e2010-01-13 19:21:00 +000056 :pep:`3120`
Brett Cannonafccd632009-01-20 02:21:27 +000057 Using UTF-8 as the Default Source Encoding
58
Brett Cannon30b7a902010-06-27 21:49:22 +000059 :pep:`3147`
60 PYC Repository Directories
61
Brett Cannonafccd632009-01-20 02:21:27 +000062
63Functions
64---------
65
Brett Cannoncb4996a2012-08-06 16:34:44 -040066.. function:: __import__(name, globals=None, locals=None, fromlist=(), level=0)
Brett Cannonafccd632009-01-20 02:21:27 +000067
Brett Cannonf23e3742010-06-27 23:57:46 +000068 An implementation of the built-in :func:`__import__` function.
Brett Cannonafccd632009-01-20 02:21:27 +000069
70.. function:: import_module(name, package=None)
71
Brett Cannon33418c82009-01-22 18:37:20 +000072 Import a module. The *name* argument specifies what module to
Brett Cannonafccd632009-01-20 02:21:27 +000073 import in absolute or relative terms
74 (e.g. either ``pkg.mod`` or ``..mod``). If the name is
Guido van Rossum09613542009-03-30 20:34:57 +000075 specified in relative terms, then the *package* argument must be set to
76 the name of the package which is to act as the anchor for resolving the
Brett Cannonafccd632009-01-20 02:21:27 +000077 package name (e.g. ``import_module('..mod', 'pkg.subpkg')`` will import
Brett Cannon2c318a12009-02-07 01:15:27 +000078 ``pkg.mod``).
Brett Cannon78246b62009-01-25 04:56:30 +000079
Brett Cannon2c318a12009-02-07 01:15:27 +000080 The :func:`import_module` function acts as a simplifying wrapper around
Brett Cannon9f4cb1c2009-04-01 23:26:47 +000081 :func:`importlib.__import__`. This means all semantics of the function are
82 derived from :func:`importlib.__import__`, including requiring the package
83 from which an import is occurring to have been previously imported
84 (i.e., *package* must already be imported). The most important difference
85 is that :func:`import_module` returns the most nested package or module
86 that was imported (e.g. ``pkg.mod``), while :func:`__import__` returns the
Guido van Rossum09613542009-03-30 20:34:57 +000087 top-level package or module (e.g. ``pkg``).
88
Brett Cannonee78a2b2012-05-12 17:43:17 -040089.. function:: find_loader(name, path=None)
90
91 Find the loader for a module, optionally within the specified *path*. If the
92 module is in :attr:`sys.modules`, then ``sys.modules[name].__loader__`` is
Brett Cannon32799232013-03-13 11:09:08 -070093 returned (unless the loader would be ``None`` or is not set, in which case
Brett Cannonee78a2b2012-05-12 17:43:17 -040094 :exc:`ValueError` is raised). Otherwise a search using :attr:`sys.meta_path`
95 is done. ``None`` is returned if no loader is found.
96
Brett Cannon56b4ca72012-11-17 09:30:55 -050097 A dotted name does not have its parent's implicitly imported as that requires
98 loading them and that may not be desired. To properly import a submodule you
99 will need to import all parent packages of the submodule and use the correct
100 argument to *path*.
Brett Cannonee78a2b2012-05-12 17:43:17 -0400101
Brett Cannon32799232013-03-13 11:09:08 -0700102 .. versionadded:: 3.3
103
104 .. versionchanged:: 3.4
105 If ``__loader__`` is not set, raise :exc:`ValueError`, just like when the
106 attribute is set to ``None``.
107
Antoine Pitrouc541f8e2012-02-20 01:48:16 +0100108.. function:: invalidate_caches()
109
Brett Cannonf4dc9202012-08-10 12:21:12 -0400110 Invalidate the internal caches of finders stored at
111 :data:`sys.meta_path`. If a finder implements ``invalidate_caches()`` then it
Brett Cannon4067aa22013-04-27 23:20:32 -0400112 will be called to perform the invalidation. This function should be called
113 if any modules are created/installed while your program is running to
114 guarantee all finders will notice the new module's existence.
Antoine Pitrouc541f8e2012-02-20 01:48:16 +0100115
116 .. versionadded:: 3.3
117
Brett Cannon78246b62009-01-25 04:56:30 +0000118
Brett Cannon2a922ed2009-03-09 03:35:50 +0000119:mod:`importlib.abc` -- Abstract base classes related to import
120---------------------------------------------------------------
121
122.. module:: importlib.abc
123 :synopsis: Abstract base classes related to import
124
125The :mod:`importlib.abc` module contains all of the core abstract base classes
126used by :keyword:`import`. Some subclasses of the core abstract base classes
127are also provided to help in implementing the core ABCs.
128
Andrew Svetlova8656542012-08-13 22:19:01 +0300129ABC hierarchy::
130
131 object
Brett Cannon1b799182012-08-17 14:08:24 -0400132 +-- Finder (deprecated)
Andrew Svetlova8656542012-08-13 22:19:01 +0300133 | +-- MetaPathFinder
134 | +-- PathEntryFinder
135 +-- Loader
136 +-- ResourceLoader --------+
137 +-- InspectLoader |
138 +-- ExecutionLoader --+
139 +-- FileLoader
140 +-- SourceLoader
Andrew Svetlova8656542012-08-13 22:19:01 +0300141
Brett Cannon2a922ed2009-03-09 03:35:50 +0000142
143.. class:: Finder
144
Brett Cannon1b799182012-08-17 14:08:24 -0400145 An abstract base class representing a :term:`finder`.
146
147 .. deprecated:: 3.3
148 Use :class:`MetaPathFinder` or :class:`PathEntryFinder` instead.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000149
Brett Cannonf4dc9202012-08-10 12:21:12 -0400150 .. method:: find_module(fullname, path=None)
Brett Cannonb46a1792012-02-27 18:15:42 -0500151
Brett Cannonf4dc9202012-08-10 12:21:12 -0400152 An abstact method for finding a :term:`loader` for the specified
153 module. Originally specified in :pep:`302`, this method was meant
154 for use in :data:`sys.meta_path` and in the path-based import subsystem.
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000155
Brett Cannon100883f2013-04-09 16:59:39 -0400156 .. versionchanged:: 3.4
157 Returns ``None`` when called instead of raising
158 :exc:`NotImplementedError`.
159
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000160
Brett Cannon077ef452012-08-02 17:50:06 -0400161.. class:: MetaPathFinder
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000162
Brett Cannonf4dc9202012-08-10 12:21:12 -0400163 An abstract base class representing a :term:`meta path finder`. For
164 compatibility, this is a subclass of :class:`Finder`.
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000165
166 .. versionadded:: 3.3
167
168 .. method:: find_module(fullname, path)
169
170 An abstract method for finding a :term:`loader` for the specified
171 module. If this is a top-level import, *path* will be ``None``.
Ezio Melotti1f67e802012-10-21 07:24:13 +0300172 Otherwise, this is a search for a subpackage or module and *path*
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000173 will be the value of :attr:`__path__` from the parent
174 package. If a loader cannot be found, ``None`` is returned.
175
Brett Cannon100883f2013-04-09 16:59:39 -0400176 .. versionchanged:: 3.4
177 Returns ``None`` when called instead of raising
178 :exc:`NotImplementedError`.
179
Brett Cannonf4dc9202012-08-10 12:21:12 -0400180 .. method:: invalidate_caches()
181
182 An optional method which, when called, should invalidate any internal
Brett Cannona6e85812012-08-11 19:41:27 -0400183 cache used by the finder. Used by :func:`importlib.invalidate_caches`
184 when invalidating the caches of all finders on :data:`sys.meta_path`.
Brett Cannonf4dc9202012-08-10 12:21:12 -0400185
Brett Cannon100883f2013-04-09 16:59:39 -0400186 .. versionchanged:: 3.4
187 Returns ``None`` when called instead of ``NotImplemented``.
188
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000189
Brett Cannon077ef452012-08-02 17:50:06 -0400190.. class:: PathEntryFinder
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000191
Brett Cannonf4dc9202012-08-10 12:21:12 -0400192 An abstract base class representing a :term:`path entry finder`. Though
193 it bears some similarities to :class:`MetaPathFinder`, ``PathEntryFinder``
194 is meant for use only within the path-based import subsystem provided
195 by :class:`PathFinder`. This ABC is a subclass of :class:`Finder` for
Brett Cannon100883f2013-04-09 16:59:39 -0400196 compatibility reasons only.
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000197
198 .. versionadded:: 3.3
199
Brett Cannon4067aa22013-04-27 23:20:32 -0400200 .. method:: find_loader(fullname)
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000201
202 An abstract method for finding a :term:`loader` for the specified
Brett Cannonf4dc9202012-08-10 12:21:12 -0400203 module. Returns a 2-tuple of ``(loader, portion)`` where ``portion``
204 is a sequence of file system locations contributing to part of a namespace
205 package. The loader may be ``None`` while specifying ``portion`` to
206 signify the contribution of the file system locations to a namespace
207 package. An empty list can be used for ``portion`` to signify the loader
Brett Cannon100883f2013-04-09 16:59:39 -0400208 is not part of a namespace package. If ``loader`` is ``None`` and
209 ``portion`` is the empty list then no loader or location for a namespace
210 package were found (i.e. failure to find anything for the module).
211
212 .. versionchanged:: 3.4
213 Returns ``(None, [])`` instead of raising :exc:`NotImplementedError`.
Brett Cannonf4dc9202012-08-10 12:21:12 -0400214
Brett Cannon4067aa22013-04-27 23:20:32 -0400215 .. method:: find_module(fullname)
Brett Cannonf4dc9202012-08-10 12:21:12 -0400216
217 A concrete implementation of :meth:`Finder.find_module` which is
218 equivalent to ``self.find_loader(fullname)[0]``.
219
220 .. method:: invalidate_caches()
221
222 An optional method which, when called, should invalidate any internal
Brett Cannona6e85812012-08-11 19:41:27 -0400223 cache used by the finder. Used by :meth:`PathFinder.invalidate_caches`
Brett Cannonf4dc9202012-08-10 12:21:12 -0400224 when invalidating the caches of all cached finders.
Brett Cannonb46a1792012-02-27 18:15:42 -0500225
Brett Cannon2a922ed2009-03-09 03:35:50 +0000226
227.. class:: Loader
228
229 An abstract base class for a :term:`loader`.
Guido van Rossum09613542009-03-30 20:34:57 +0000230 See :pep:`302` for the exact definition for a loader.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000231
Brett Cannon9c751b72009-03-09 16:28:16 +0000232 .. method:: load_module(fullname)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000233
234 An abstract method for loading a module. If the module cannot be
235 loaded, :exc:`ImportError` is raised, otherwise the loaded module is
236 returned.
237
Guido van Rossum09613542009-03-30 20:34:57 +0000238 If the requested module already exists in :data:`sys.modules`, that
Brett Cannon2a922ed2009-03-09 03:35:50 +0000239 module should be used and reloaded.
Guido van Rossum09613542009-03-30 20:34:57 +0000240 Otherwise the loader should create a new module and insert it into
241 :data:`sys.modules` before any loading begins, to prevent recursion
242 from the import. If the loader inserted a module and the load fails, it
Brett Cannon2a922ed2009-03-09 03:35:50 +0000243 must be removed by the loader from :data:`sys.modules`; modules already
244 in :data:`sys.modules` before the loader began execution should be left
245 alone. The :func:`importlib.util.module_for_loader` decorator handles
246 all of these details.
247
Guido van Rossum09613542009-03-30 20:34:57 +0000248 The loader should set several attributes on the module.
249 (Note that some of these attributes can change when a module is
250 reloaded.)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000251
252 - :attr:`__name__`
253 The name of the module.
254
255 - :attr:`__file__`
256 The path to where the module data is stored (not set for built-in
257 modules).
258
Brett Cannon2cefb3c2013-05-25 11:26:11 -0400259 - :attr:`__cached__`
260 The path to where a compiled version of the module is/should be
261 stored (not set when the attribute would be inappropriate).
262
Brett Cannon2a922ed2009-03-09 03:35:50 +0000263 - :attr:`__path__`
Guido van Rossum09613542009-03-30 20:34:57 +0000264 A list of strings specifying the search path within a
Brett Cannon2a922ed2009-03-09 03:35:50 +0000265 package. This attribute is not set on modules.
266
267 - :attr:`__package__`
268 The parent package for the module/package. If the module is
269 top-level then it has a value of the empty string. The
Brett Cannon100883f2013-04-09 16:59:39 -0400270 :func:`importlib.util.module_for_loader` decorator can handle the
271 details for :attr:`__package__`.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000272
273 - :attr:`__loader__`
Brett Cannon100883f2013-04-09 16:59:39 -0400274 The loader used to load the module. The
275 :func:`importlib.util.module_for_loader` decorator can handle the
276 details for :attr:`__package__`.
277
278 .. versionchanged:: 3.4
279 Raise :exc:`ImportError` when called instead of
280 :exc:`NotImplementedError`.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000281
Barry Warsawd7d21942012-07-29 16:36:17 -0400282 .. method:: module_repr(module)
283
Brett Cannon100883f2013-04-09 16:59:39 -0400284 An optional method which when implemented calculates and returns the
285 given module's repr, as a string. The module type's default repr() will
286 use the result of this method as appropriate.
Barry Warsawd7d21942012-07-29 16:36:17 -0400287
Georg Brandl526575d2013-04-11 16:10:13 +0200288 .. versionadded:: 3.3
Barry Warsawd7d21942012-07-29 16:36:17 -0400289
Brett Cannon100883f2013-04-09 16:59:39 -0400290 .. versionchanged:: 3.4
Georg Brandl526575d2013-04-11 16:10:13 +0200291 Made optional instead of an abstractmethod.
Brett Cannon100883f2013-04-09 16:59:39 -0400292
Brett Cannon2a922ed2009-03-09 03:35:50 +0000293
294.. class:: ResourceLoader
295
296 An abstract base class for a :term:`loader` which implements the optional
297 :pep:`302` protocol for loading arbitrary resources from the storage
298 back-end.
299
Brett Cannon9c751b72009-03-09 16:28:16 +0000300 .. method:: get_data(path)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000301
302 An abstract method to return the bytes for the data located at *path*.
Guido van Rossum09613542009-03-30 20:34:57 +0000303 Loaders that have a file-like storage back-end
Brett Cannon16248a42009-04-01 20:47:14 +0000304 that allows storing arbitrary data
Guido van Rossum09613542009-03-30 20:34:57 +0000305 can implement this abstract method to give direct access
Brett Cannon2a922ed2009-03-09 03:35:50 +0000306 to the data stored. :exc:`IOError` is to be raised if the *path* cannot
307 be found. The *path* is expected to be constructed using a module's
Brett Cannon16248a42009-04-01 20:47:14 +0000308 :attr:`__file__` attribute or an item from a package's :attr:`__path__`.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000309
Brett Cannon100883f2013-04-09 16:59:39 -0400310 .. versionchanged:: 3.4
311 Raises :exc:`IOError` instead of :exc:`NotImplementedError`.
312
Brett Cannon2a922ed2009-03-09 03:35:50 +0000313
314.. class:: InspectLoader
315
316 An abstract base class for a :term:`loader` which implements the optional
Guido van Rossum09613542009-03-30 20:34:57 +0000317 :pep:`302` protocol for loaders that inspect modules.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000318
Brett Cannona113ac52009-03-15 01:41:33 +0000319 .. method:: get_code(fullname)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000320
Brett Cannon3b62ca82013-05-27 21:11:04 -0400321 Return the code object for a module.
322 ``None`` should be returned if the module does not have a code object
Brett Cannona113ac52009-03-15 01:41:33 +0000323 (e.g. built-in module). :exc:`ImportError` is raised if loader cannot
324 find the requested module.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000325
Brett Cannon3b62ca82013-05-27 21:11:04 -0400326 .. note::
327 While the method has a default implementation, it is suggested that
328 it be overridden if possible for performance.
329
R David Murray1b00f252012-08-15 10:43:58 -0400330 .. index::
331 single: universal newlines; importlib.abc.InspectLoader.get_source method
332
Brett Cannon100883f2013-04-09 16:59:39 -0400333 .. versionchanged:: 3.4
Brett Cannon3b62ca82013-05-27 21:11:04 -0400334 No longer abstract and a concrete implementation is provided.
Brett Cannon100883f2013-04-09 16:59:39 -0400335
Brett Cannon9c751b72009-03-09 16:28:16 +0000336 .. method:: get_source(fullname)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000337
338 An abstract method to return the source of a module. It is returned as
R David Murray1b00f252012-08-15 10:43:58 -0400339 a text string using :term:`universal newlines`, translating all
R David Murrayee0a9452012-08-15 11:05:36 -0400340 recognized line separators into ``'\n'`` characters. Returns ``None``
341 if no source is available (e.g. a built-in module). Raises
342 :exc:`ImportError` if the loader cannot find the module specified.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000343
Brett Cannon100883f2013-04-09 16:59:39 -0400344 .. versionchanged:: 3.4
345 Raises :exc:`ImportError` instead of :exc:`NotImplementedError`.
346
Brett Cannona113ac52009-03-15 01:41:33 +0000347 .. method:: is_package(fullname)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000348
Brett Cannona113ac52009-03-15 01:41:33 +0000349 An abstract method to return a true value if the module is a package, a
350 false value otherwise. :exc:`ImportError` is raised if the
351 :term:`loader` cannot find the module.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000352
Brett Cannon100883f2013-04-09 16:59:39 -0400353 .. versionchanged:: 3.4
354 Raises :exc:`ImportError` instead of :exc:`NotImplementedError`.
355
Brett Cannon9ffe85e2013-05-26 16:45:10 -0400356 .. method:: source_to_code(data, path='<string>')
357
358 Create a code object from Python source.
359
360 The *data* argument can be whatever the :func:`compile` function
361 supports (i.e. string or bytes). The *path* argument should be
362 the "path" to where the source code originated from, which can be an
363 abstract concept (e.g. location in a zip file).
364
365 .. versionadded:: 3.4
366
Brett Cannon2a922ed2009-03-09 03:35:50 +0000367
Brett Cannon69194272009-07-20 04:23:48 +0000368.. class:: ExecutionLoader
369
370 An abstract base class which inherits from :class:`InspectLoader` that,
Brett Cannon23460292009-07-20 22:59:00 +0000371 when implemented, helps a module to be executed as a script. The ABC
Brett Cannon69194272009-07-20 04:23:48 +0000372 represents an optional :pep:`302` protocol.
373
374 .. method:: get_filename(fullname)
375
Brett Cannonf23e3742010-06-27 23:57:46 +0000376 An abstract method that is to return the value of :attr:`__file__` for
Brett Cannon69194272009-07-20 04:23:48 +0000377 the specified module. If no path is available, :exc:`ImportError` is
378 raised.
379
Brett Cannonf23e3742010-06-27 23:57:46 +0000380 If source code is available, then the method should return the path to
381 the source file, regardless of whether a bytecode was used to load the
382 module.
383
Brett Cannon100883f2013-04-09 16:59:39 -0400384 .. versionchanged:: 3.4
385 Raises :exc:`ImportError` instead of :exc:`NotImplementedError`.
386
Brett Cannonf23e3742010-06-27 23:57:46 +0000387
Brett Cannon938d44d2012-04-22 19:58:33 -0400388.. class:: FileLoader(fullname, path)
389
390 An abstract base class which inherits from :class:`ResourceLoader` and
Andrew Svetlova60de4f2013-02-17 16:55:58 +0200391 :class:`ExecutionLoader`, providing concrete implementations of
Brett Cannon938d44d2012-04-22 19:58:33 -0400392 :meth:`ResourceLoader.get_data` and :meth:`ExecutionLoader.get_filename`.
393
394 The *fullname* argument is a fully resolved name of the module the loader is
395 to handle. The *path* argument is the path to the file for the module.
396
397 .. versionadded:: 3.3
398
399 .. attribute:: name
400
401 The name of the module the loader can handle.
402
403 .. attribute:: path
404
405 Path to the file of the module.
406
Barry Warsawd7d21942012-07-29 16:36:17 -0400407 .. method:: load_module(fullname)
Brett Cannonc0499522012-05-11 14:48:41 -0400408
Barry Warsawd7d21942012-07-29 16:36:17 -0400409 Calls super's ``load_module()``.
Brett Cannonc0499522012-05-11 14:48:41 -0400410
Brett Cannon938d44d2012-04-22 19:58:33 -0400411 .. method:: get_filename(fullname)
412
Barry Warsawd7d21942012-07-29 16:36:17 -0400413 Returns :attr:`path`.
Brett Cannon938d44d2012-04-22 19:58:33 -0400414
415 .. method:: get_data(path)
416
Brett Cannon3b62ca82013-05-27 21:11:04 -0400417 Reads *path* as a binary file and returns the bytes from it.
Brett Cannon938d44d2012-04-22 19:58:33 -0400418
419
Brett Cannonf23e3742010-06-27 23:57:46 +0000420.. class:: SourceLoader
421
422 An abstract base class for implementing source (and optionally bytecode)
423 file loading. The class inherits from both :class:`ResourceLoader` and
424 :class:`ExecutionLoader`, requiring the implementation of:
425
426 * :meth:`ResourceLoader.get_data`
427 * :meth:`ExecutionLoader.get_filename`
Brett Cannon6dfbff32010-07-21 09:48:31 +0000428 Should only return the path to the source file; sourceless
Brett Cannon5650e4f2012-11-18 10:03:31 -0500429 loading is not supported (see :class:`SourcelessLoader` if that
430 functionality is required)
Brett Cannonf23e3742010-06-27 23:57:46 +0000431
432 The abstract methods defined by this class are to add optional bytecode
Brett Cannon5650e4f2012-11-18 10:03:31 -0500433 file support. Not implementing these optional methods (or causing them to
434 raise :exc:`NotImplementedError`) causes the loader to
Brett Cannonf23e3742010-06-27 23:57:46 +0000435 only work with source code. Implementing the methods allows the loader to
436 work with source *and* bytecode files; it does not allow for *sourceless*
437 loading where only bytecode is provided. Bytecode files are an
438 optimization to speed up loading by removing the parsing step of Python's
439 compiler, and so no bytecode-specific API is exposed.
440
Brett Cannon773468f2012-08-02 17:35:34 -0400441 .. method:: path_stats(path)
Antoine Pitrou5136ac02012-01-13 18:52:16 +0100442
443 Optional abstract method which returns a :class:`dict` containing
444 metadata about the specifed path. Supported dictionary keys are:
445
446 - ``'mtime'`` (mandatory): an integer or floating-point number
447 representing the modification time of the source code;
448 - ``'size'`` (optional): the size in bytes of the source code.
449
450 Any other keys in the dictionary are ignored, to allow for future
Brett Cannon100883f2013-04-09 16:59:39 -0400451 extensions. If the path cannot be handled, :exc:`IOError` is raised.
Antoine Pitrou5136ac02012-01-13 18:52:16 +0100452
453 .. versionadded:: 3.3
454
Brett Cannon100883f2013-04-09 16:59:39 -0400455 .. versionchanged:: 3.4
456 Raise :exc:`IOError` instead of :exc:`NotImplementedError`.
457
Brett Cannon773468f2012-08-02 17:35:34 -0400458 .. method:: path_mtime(path)
Brett Cannonf23e3742010-06-27 23:57:46 +0000459
460 Optional abstract method which returns the modification time for the
461 specified path.
462
Antoine Pitrou5136ac02012-01-13 18:52:16 +0100463 .. deprecated:: 3.3
464 This method is deprecated in favour of :meth:`path_stats`. You don't
465 have to implement it, but it is still available for compatibility
Brett Cannon100883f2013-04-09 16:59:39 -0400466 purposes. Raise :exc:`IOError` if the path cannot be handled.
467
468 .. versionchanged:: 3.4
469 Raise :exc:`IOError` instead of :exc:`NotImplementedError`.
Antoine Pitrou5136ac02012-01-13 18:52:16 +0100470
Brett Cannon773468f2012-08-02 17:35:34 -0400471 .. method:: set_data(path, data)
Brett Cannonf23e3742010-06-27 23:57:46 +0000472
473 Optional abstract method which writes the specified bytes to a file
Brett Cannon61b14252010-07-03 21:48:25 +0000474 path. Any intermediate directories which do not exist are to be created
475 automatically.
476
477 When writing to the path fails because the path is read-only
Brett Cannon2cefb3c2013-05-25 11:26:11 -0400478 (:attr:`errno.EACCES`/:exc:`PermissionError`), do not propagate the
479 exception.
Brett Cannonf23e3742010-06-27 23:57:46 +0000480
Brett Cannon100883f2013-04-09 16:59:39 -0400481 .. versionchanged:: 3.4
482 No longer raises :exc:`NotImplementedError` when called.
483
Brett Cannon773468f2012-08-02 17:35:34 -0400484 .. method:: get_code(fullname)
Brett Cannonf23e3742010-06-27 23:57:46 +0000485
486 Concrete implementation of :meth:`InspectLoader.get_code`.
487
Brett Cannon773468f2012-08-02 17:35:34 -0400488 .. method:: load_module(fullname)
Brett Cannonf23e3742010-06-27 23:57:46 +0000489
490 Concrete implementation of :meth:`Loader.load_module`.
491
Brett Cannon773468f2012-08-02 17:35:34 -0400492 .. method:: get_source(fullname)
Brett Cannonf23e3742010-06-27 23:57:46 +0000493
494 Concrete implementation of :meth:`InspectLoader.get_source`.
495
Brett Cannon773468f2012-08-02 17:35:34 -0400496 .. method:: is_package(fullname)
Brett Cannonf23e3742010-06-27 23:57:46 +0000497
498 Concrete implementation of :meth:`InspectLoader.is_package`. A module
Brett Cannonea0b8232012-06-15 20:00:53 -0400499 is determined to be a package if its file path (as provided by
500 :meth:`ExecutionLoader.get_filename`) is a file named
501 ``__init__`` when the file extension is removed **and** the module name
502 itself does not end in ``__init__``.
Brett Cannonf23e3742010-06-27 23:57:46 +0000503
Brett Cannon69194272009-07-20 04:23:48 +0000504
Brett Cannon78246b62009-01-25 04:56:30 +0000505:mod:`importlib.machinery` -- Importers and path hooks
506------------------------------------------------------
507
508.. module:: importlib.machinery
509 :synopsis: Importers and path hooks
510
511This module contains the various objects that help :keyword:`import`
512find and load modules.
513
Brett Cannoncb66eb02012-05-11 12:58:42 -0400514.. attribute:: SOURCE_SUFFIXES
515
516 A list of strings representing the recognized file suffixes for source
517 modules.
518
519 .. versionadded:: 3.3
520
521.. attribute:: DEBUG_BYTECODE_SUFFIXES
522
523 A list of strings representing the file suffixes for non-optimized bytecode
524 modules.
525
526 .. versionadded:: 3.3
527
528.. attribute:: OPTIMIZED_BYTECODE_SUFFIXES
529
530 A list of strings representing the file suffixes for optimized bytecode
531 modules.
532
533 .. versionadded:: 3.3
534
535.. attribute:: BYTECODE_SUFFIXES
536
537 A list of strings representing the recognized file suffixes for bytecode
538 modules. Set to either :attr:`DEBUG_BYTECODE_SUFFIXES` or
539 :attr:`OPTIMIZED_BYTECODE_SUFFIXES` based on whether ``__debug__`` is true.
540
541 .. versionadded:: 3.3
542
543.. attribute:: EXTENSION_SUFFIXES
544
Nick Coghlan76e07702012-07-18 23:14:57 +1000545 A list of strings representing the recognized file suffixes for
Brett Cannoncb66eb02012-05-11 12:58:42 -0400546 extension modules.
547
548 .. versionadded:: 3.3
549
Nick Coghlanc5afd422012-07-18 23:59:08 +1000550.. function:: all_suffixes()
Nick Coghlan76e07702012-07-18 23:14:57 +1000551
552 Returns a combined list of strings representing all file suffixes for
Nick Coghlanc5afd422012-07-18 23:59:08 +1000553 modules recognized by the standard import machinery. This is a
Nick Coghlan76e07702012-07-18 23:14:57 +1000554 helper for code which simply needs to know if a filesystem path
Nick Coghlanc5afd422012-07-18 23:59:08 +1000555 potentially refers to a module without needing any details on the kind
556 of module (for example, :func:`inspect.getmodulename`)
Nick Coghlan76e07702012-07-18 23:14:57 +1000557
558 .. versionadded:: 3.3
559
560
Brett Cannon78246b62009-01-25 04:56:30 +0000561.. class:: BuiltinImporter
562
Brett Cannon2a922ed2009-03-09 03:35:50 +0000563 An :term:`importer` for built-in modules. All known built-in modules are
564 listed in :data:`sys.builtin_module_names`. This class implements the
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000565 :class:`importlib.abc.MetaPathFinder` and
566 :class:`importlib.abc.InspectLoader` ABCs.
Brett Cannon78246b62009-01-25 04:56:30 +0000567
568 Only class methods are defined by this class to alleviate the need for
569 instantiation.
570
Brett Cannon78246b62009-01-25 04:56:30 +0000571
572.. class:: FrozenImporter
573
Brett Cannon2a922ed2009-03-09 03:35:50 +0000574 An :term:`importer` for frozen modules. This class implements the
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000575 :class:`importlib.abc.MetaPathFinder` and
576 :class:`importlib.abc.InspectLoader` ABCs.
Brett Cannon78246b62009-01-25 04:56:30 +0000577
578 Only class methods are defined by this class to alleviate the need for
579 instantiation.
580
Brett Cannondebb98d2009-02-16 04:18:01 +0000581
Nick Coghlanff794862012-08-02 21:45:24 +1000582.. class:: WindowsRegistryFinder
583
584 :term:`Finder` for modules declared in the Windows registry. This class
Nick Coghlan49417742012-08-02 23:03:58 +1000585 implements the :class:`importlib.abc.Finder` ABC.
Nick Coghlanff794862012-08-02 21:45:24 +1000586
587 Only class methods are defined by this class to alleviate the need for
588 instantiation.
589
590 .. versionadded:: 3.3
591
592
Brett Cannondebb98d2009-02-16 04:18:01 +0000593.. class:: PathFinder
594
Brett Cannon1b799182012-08-17 14:08:24 -0400595 A :term:`Finder` for :data:`sys.path` and package ``__path__`` attributes.
596 This class implements the :class:`importlib.abc.MetaPathFinder` ABC.
Brett Cannondebb98d2009-02-16 04:18:01 +0000597
Brett Cannon1b799182012-08-17 14:08:24 -0400598 Only class methods are defined by this class to alleviate the need for
599 instantiation.
Brett Cannondebb98d2009-02-16 04:18:01 +0000600
Brett Cannon1b799182012-08-17 14:08:24 -0400601 .. classmethod:: find_module(fullname, path=None)
Brett Cannondebb98d2009-02-16 04:18:01 +0000602
Brett Cannon1b799182012-08-17 14:08:24 -0400603 Class method that attempts to find a :term:`loader` for the module
604 specified by *fullname* on :data:`sys.path` or, if defined, on
605 *path*. For each path entry that is searched,
606 :data:`sys.path_importer_cache` is checked. If a non-false object is
Brett Cannon2cefb3c2013-05-25 11:26:11 -0400607 found then it is used as the :term:`path entry finder` to look for the
608 module being searched for. If no entry is found in
Brett Cannon1b799182012-08-17 14:08:24 -0400609 :data:`sys.path_importer_cache`, then :data:`sys.path_hooks` is
610 searched for a finder for the path entry and, if found, is stored in
611 :data:`sys.path_importer_cache` along with being queried about the
612 module. If no finder is ever found then ``None`` is both stored in
613 the cache and returned.
Brett Cannond2e7b332009-02-17 02:45:03 +0000614
Brett Cannonf4dc9202012-08-10 12:21:12 -0400615 .. classmethod:: invalidate_caches()
616
Brett Cannon1b799182012-08-17 14:08:24 -0400617 Calls :meth:`importlib.abc.PathEntryFinder.invalidate_caches` on all
618 finders stored in :attr:`sys.path_importer_cache`.
Brett Cannonf4dc9202012-08-10 12:21:12 -0400619
Brett Cannond2e7b332009-02-17 02:45:03 +0000620
Brett Cannon938d44d2012-04-22 19:58:33 -0400621.. class:: FileFinder(path, \*loader_details)
622
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000623 A concrete implementation of :class:`importlib.abc.PathEntryFinder` which
624 caches results from the file system.
Brett Cannon938d44d2012-04-22 19:58:33 -0400625
626 The *path* argument is the directory for which the finder is in charge of
627 searching.
628
Brett Cannonac9f2f32012-08-10 13:47:54 -0400629 The *loader_details* argument is a variable number of 2-item tuples each
630 containing a loader and a sequence of file suffixes the loader recognizes.
Brett Cannon938d44d2012-04-22 19:58:33 -0400631
632 The finder will cache the directory contents as necessary, making stat calls
633 for each module search to verify the cache is not outdated. Because cache
634 staleness relies upon the granularity of the operating system's state
635 information of the file system, there is a potential race condition of
636 searching for a module, creating a new file, and then searching for the
637 module the new file represents. If the operations happen fast enough to fit
638 within the granularity of stat calls, then the module search will fail. To
639 prevent this from happening, when you create a module dynamically, make sure
640 to call :func:`importlib.invalidate_caches`.
641
642 .. versionadded:: 3.3
643
644 .. attribute:: path
645
646 The path the finder will search in.
647
648 .. method:: find_module(fullname)
649
650 Attempt to find the loader to handle *fullname* within :attr:`path`.
651
652 .. method:: invalidate_caches()
653
654 Clear out the internal cache.
655
656 .. classmethod:: path_hook(\*loader_details)
657
658 A class method which returns a closure for use on :attr:`sys.path_hooks`.
659 An instance of :class:`FileFinder` is returned by the closure using the
660 path argument given to the closure directly and *loader_details*
661 indirectly.
662
663 If the argument to the closure is not an existing directory,
664 :exc:`ImportError` is raised.
665
666
667.. class:: SourceFileLoader(fullname, path)
668
669 A concrete implementation of :class:`importlib.abc.SourceLoader` by
670 subclassing :class:`importlib.abc.FileLoader` and providing some concrete
671 implementations of other methods.
672
673 .. versionadded:: 3.3
674
675 .. attribute:: name
676
677 The name of the module that this loader will handle.
678
679 .. attribute:: path
680
681 The path to the source file.
682
683 .. method:: is_package(fullname)
684
685 Return true if :attr:`path` appears to be for a package.
686
687 .. method:: path_stats(path)
688
689 Concrete implementation of :meth:`importlib.abc.SourceLoader.path_stats`.
690
691 .. method:: set_data(path, data)
692
693 Concrete implementation of :meth:`importlib.abc.SourceLoader.set_data`.
694
Brett Cannon938d44d2012-04-22 19:58:33 -0400695
Marc-Andre Lemburg4fe29c92012-04-25 02:31:37 +0200696.. class:: SourcelessFileLoader(fullname, path)
Brett Cannon938d44d2012-04-22 19:58:33 -0400697
698 A concrete implementation of :class:`importlib.abc.FileLoader` which can
699 import bytecode files (i.e. no source code files exist).
700
Marc-Andre Lemburg4fe29c92012-04-25 02:31:37 +0200701 Please note that direct use of bytecode files (and thus not source code
702 files) inhibits your modules from being usable by all Python
703 implementations or new versions of Python which change the bytecode
704 format.
Brett Cannon938d44d2012-04-22 19:58:33 -0400705
706 .. versionadded:: 3.3
707
708 .. attribute:: name
709
710 The name of the module the loader will handle.
711
712 .. attribute:: path
713
714 The path to the bytecode file.
715
716 .. method:: is_package(fullname)
717
718 Determines if the module is a package based on :attr:`path`.
719
720 .. method:: get_code(fullname)
721
722 Returns the code object for :attr:`name` created from :attr:`path`.
723
724 .. method:: get_source(fullname)
725
726 Returns ``None`` as bytecode files have no source when this loader is
727 used.
728
Brett Cannon938d44d2012-04-22 19:58:33 -0400729
730.. class:: ExtensionFileLoader(fullname, path)
731
732 A concrete implementation of :class:`importlib.abc.InspectLoader` for
733 extension modules.
734
735 The *fullname* argument specifies the name of the module the loader is to
736 support. The *path* argument is the path to the extension module's file.
737
738 .. versionadded:: 3.3
739
740 .. attribute:: name
741
742 Name of the module the loader supports.
743
744 .. attribute:: path
745
746 Path to the extension module.
747
Barry Warsawd7d21942012-07-29 16:36:17 -0400748 .. method:: load_module(fullname)
Brett Cannon938d44d2012-04-22 19:58:33 -0400749
Brett Cannonc0499522012-05-11 14:48:41 -0400750 Loads the extension module if and only if *fullname* is the same as
751 :attr:`name` or is ``None``.
Brett Cannon938d44d2012-04-22 19:58:33 -0400752
753 .. method:: is_package(fullname)
754
Brett Cannonac9f2f32012-08-10 13:47:54 -0400755 Returns ``True`` if the file path points to a package's ``__init__``
756 module based on :attr:`EXTENSION_SUFFIXES`.
Brett Cannon938d44d2012-04-22 19:58:33 -0400757
758 .. method:: get_code(fullname)
759
760 Returns ``None`` as extension modules lack a code object.
761
762 .. method:: get_source(fullname)
763
764 Returns ``None`` as extension modules do not have source code.
765
766
Brett Cannond2e7b332009-02-17 02:45:03 +0000767:mod:`importlib.util` -- Utility code for importers
768---------------------------------------------------
769
770.. module:: importlib.util
Brett Cannon75321e82012-03-02 11:58:25 -0500771 :synopsis: Utility code for importers
Brett Cannond2e7b332009-02-17 02:45:03 +0000772
773This module contains the various objects that help in the construction of
774an :term:`importer`.
775
Brett Cannond200bf52012-05-13 13:45:09 -0400776.. function:: resolve_name(name, package)
777
778 Resolve a relative module name to an absolute one.
779
780 If **name** has no leading dots, then **name** is simply returned. This
781 allows for usage such as
782 ``importlib.util.resolve_name('sys', __package__)`` without doing a
783 check to see if the **package** argument is needed.
784
785 :exc:`ValueError` is raised if **name** is a relative module name but
786 package is a false value (e.g. ``None`` or the empty string).
787 :exc:`ValueError` is also raised a relative name would escape its containing
788 package (e.g. requesting ``..bacon`` from within the ``spam`` package).
789
790 .. versionadded:: 3.3
791
Brett Cannona3687f02013-05-28 17:29:34 -0400792.. class:: ModuleManager(name)
793
794 A :term:`context manager` which provides the module to load. The module will
795 either come from :attr:`sys.modules` in the case of reloading or a fresh
796 module if loading a new module. Proper cleanup of :attr:`sys.modules` occurs
797 if the module was new and an exception was raised.
798
799 .. versionadded:: 3.4
800
Georg Brandl8a1caa22010-07-29 16:01:11 +0000801.. decorator:: module_for_loader
Brett Cannond2e7b332009-02-17 02:45:03 +0000802
Brett Cannona22faca2013-05-28 17:50:14 -0400803 A :term:`decorator` for :meth:`importlib.abc.Loader.load_module`
Guido van Rossum09613542009-03-30 20:34:57 +0000804 to handle selecting the proper
Brett Cannond2e7b332009-02-17 02:45:03 +0000805 module object to load with. The decorated method is expected to have a call
Brett Cannon2a922ed2009-03-09 03:35:50 +0000806 signature taking two positional arguments
807 (e.g. ``load_module(self, module)``) for which the second argument
Guido van Rossum09613542009-03-30 20:34:57 +0000808 will be the module **object** to be used by the loader.
Brett Cannonefad00d2012-04-27 17:27:14 -0400809 Note that the decorator will not work on static methods because of the
810 assumption of two arguments.
Brett Cannond2e7b332009-02-17 02:45:03 +0000811
Guido van Rossum09613542009-03-30 20:34:57 +0000812 The decorated method will take in the **name** of the module to be loaded
813 as expected for a :term:`loader`. If the module is not found in
Brett Cannon3dc48d62013-05-28 18:35:54 -0400814 :data:`sys.modules` then a new one is constructed. Regardless of where the
815 module came from, :attr:`__loader__` set to **self** and :attr:`__package__`
816 is set based on what :meth:`importlib.abc.InspectLoader.is_package` returns
817 (if available). These attributes are set unconditionally to support
818 reloading.
Brett Cannonefad00d2012-04-27 17:27:14 -0400819
820 If an exception is raised by the decorated method and a module was added to
Brett Cannond2e7b332009-02-17 02:45:03 +0000821 :data:`sys.modules` it will be removed to prevent a partially initialized
Brett Cannon57b46f52009-03-02 14:38:26 +0000822 module from being in left in :data:`sys.modules`. If the module was already
823 in :data:`sys.modules` then it is left alone.
Brett Cannond2e7b332009-02-17 02:45:03 +0000824
Brett Cannona3687f02013-05-28 17:29:34 -0400825 .. note::
826 :class:`ModuleManager` subsumes the module management aspect of this
827 decorator.
828
Brett Cannonefad00d2012-04-27 17:27:14 -0400829 .. versionchanged:: 3.3
Georg Brandl61063cc2012-06-24 22:48:30 +0200830 :attr:`__loader__` and :attr:`__package__` are automatically set
831 (when possible).
Brett Cannon57b46f52009-03-02 14:38:26 +0000832
Brett Cannon3dc48d62013-05-28 18:35:54 -0400833 .. versionchanged:: 3.4
834 Set :attr:`__loader__` :attr:`__package__` unconditionally to support
835 reloading.
836
Georg Brandl8a1caa22010-07-29 16:01:11 +0000837.. decorator:: set_loader
Brett Cannon2cf03a82009-03-10 05:17:37 +0000838
Brett Cannona22faca2013-05-28 17:50:14 -0400839 A :term:`decorator` for :meth:`importlib.abc.Loader.load_module`
840 to set the :attr:`__loader__`
841 attribute on the returned module. If the attribute is already set the
842 decorator does nothing. It is assumed that the first positional argument to
843 the wrapped method (i.e. ``self``) is what :attr:`__loader__` should be set
844 to.
Brett Cannon2cf03a82009-03-10 05:17:37 +0000845
Brett Cannonefad00d2012-04-27 17:27:14 -0400846 .. note::
Brett Cannona22faca2013-05-28 17:50:14 -0400847 As this decorator sets :attr:`__loader__` after loading the module, it is
848 recommended to use :func:`module_for_loader` instead when appropriate.
Brett Cannonefad00d2012-04-27 17:27:14 -0400849
Brett Cannon4802bec2013-03-13 10:41:36 -0700850 .. versionchanged:: 3.4
Brett Cannon4c14b5d2013-05-04 13:56:58 -0400851 Set ``__loader__`` if set to ``None``, as if the attribute does not
Brett Cannon4802bec2013-03-13 10:41:36 -0700852 exist.
853
Georg Brandl8a1caa22010-07-29 16:01:11 +0000854.. decorator:: set_package
Brett Cannon57b46f52009-03-02 14:38:26 +0000855
Brett Cannona22faca2013-05-28 17:50:14 -0400856 A :term:`decorator` for :meth:`importlib.abc.Loader.load_module` to set the :attr:`__package__` attribute on the returned module. If :attr:`__package__`
857 is set and has a value other than ``None`` it will not be changed.
Brett Cannon16248a42009-04-01 20:47:14 +0000858
Brett Cannonefad00d2012-04-27 17:27:14 -0400859 .. note::
Brett Cannona22faca2013-05-28 17:50:14 -0400860 As this decorator sets :attr:`__package__` after loading the module, it is
Brett Cannon56dfc212013-05-28 18:40:31 -0400861 recommended to use :func:`module_for_loader` instead when appropriate.