blob: 9cb03be6e39003a73a5113cb7810e1b5d6d7fec2 [file] [log] [blame]
Brett Cannonafccd632009-01-20 02:21:27 +00001:mod:`importlib` -- An implementation of :keyword:`import`
2==========================================================
3
4.. module:: importlib
5 :synopsis: An implementation of the import machinery.
6
7.. moduleauthor:: Brett Cannon <brett@python.org>
8.. sectionauthor:: Brett Cannon <brett@python.org>
9
10.. versionadded:: 3.1
11
12
13Introduction
14------------
15
16The purpose of the :mod:`importlib` package is two-fold. One is to provide an
17implementation of the :keyword:`import` statement (and thus, by extension, the
18:func:`__import__` function) in Python source code. This provides an
Tarek Ziadé434caaa2009-05-14 12:48:09 +000019implementation of :keyword:`import` which is portable to any Python
Brett Cannonafccd632009-01-20 02:21:27 +000020interpreter. This also provides a reference implementation which is easier to
Brett Cannonf23e3742010-06-27 23:57:46 +000021comprehend than one implemented in a programming language other than Python.
Brett Cannonafccd632009-01-20 02:21:27 +000022
Brett Cannonf23e3742010-06-27 23:57:46 +000023Two, the components to implement :keyword:`import` are exposed in this
Brett Cannonafccd632009-01-20 02:21:27 +000024package, making it easier for users to create their own custom objects (known
Brett Cannondebb98d2009-02-16 04:18:01 +000025generically as an :term:`importer`) to participate in the import process.
Brett Cannonf23e3742010-06-27 23:57:46 +000026Details on custom importers can be found in :pep:`302`.
Brett Cannonafccd632009-01-20 02:21:27 +000027
28.. seealso::
29
30 :ref:`import`
31 The language reference for the :keyword:`import` statement.
32
33 `Packages specification <http://www.python.org/doc/essays/packages.html>`__
34 Original specification of packages. Some semantics have changed since
Georg Brandl375aec22011-01-15 17:03:02 +000035 the writing of this document (e.g. redirecting based on ``None``
Brett Cannonafccd632009-01-20 02:21:27 +000036 in :data:`sys.modules`).
37
38 The :func:`.__import__` function
Brett Cannon0e13c942010-06-29 18:26:11 +000039 The :keyword:`import` statement is syntactic sugar for this function.
Brett Cannonafccd632009-01-20 02:21:27 +000040
41 :pep:`235`
42 Import on Case-Insensitive Platforms
43
44 :pep:`263`
45 Defining Python Source Code Encodings
46
47 :pep:`302`
Brett Cannonf23e3742010-06-27 23:57:46 +000048 New Import Hooks
Brett Cannonafccd632009-01-20 02:21:27 +000049
50 :pep:`328`
51 Imports: Multi-Line and Absolute/Relative
52
53 :pep:`366`
54 Main module explicit relative imports
55
Brett Cannon8917d5e2010-01-13 19:21:00 +000056 :pep:`3120`
Brett Cannonafccd632009-01-20 02:21:27 +000057 Using UTF-8 as the Default Source Encoding
58
Brett Cannon30b7a902010-06-27 21:49:22 +000059 :pep:`3147`
60 PYC Repository Directories
61
Brett Cannonafccd632009-01-20 02:21:27 +000062
63Functions
64---------
65
Brett Cannoncb4996a2012-08-06 16:34:44 -040066.. function:: __import__(name, globals=None, locals=None, fromlist=(), level=0)
Brett Cannonafccd632009-01-20 02:21:27 +000067
Brett Cannonf23e3742010-06-27 23:57:46 +000068 An implementation of the built-in :func:`__import__` function.
Brett Cannonafccd632009-01-20 02:21:27 +000069
70.. function:: import_module(name, package=None)
71
Brett Cannon33418c82009-01-22 18:37:20 +000072 Import a module. The *name* argument specifies what module to
Brett Cannonafccd632009-01-20 02:21:27 +000073 import in absolute or relative terms
74 (e.g. either ``pkg.mod`` or ``..mod``). If the name is
Guido van Rossum09613542009-03-30 20:34:57 +000075 specified in relative terms, then the *package* argument must be set to
76 the name of the package which is to act as the anchor for resolving the
Brett Cannonafccd632009-01-20 02:21:27 +000077 package name (e.g. ``import_module('..mod', 'pkg.subpkg')`` will import
Brett Cannon2c318a12009-02-07 01:15:27 +000078 ``pkg.mod``).
Brett Cannon78246b62009-01-25 04:56:30 +000079
Brett Cannon2c318a12009-02-07 01:15:27 +000080 The :func:`import_module` function acts as a simplifying wrapper around
Brett Cannon9f4cb1c2009-04-01 23:26:47 +000081 :func:`importlib.__import__`. This means all semantics of the function are
82 derived from :func:`importlib.__import__`, including requiring the package
83 from which an import is occurring to have been previously imported
84 (i.e., *package* must already be imported). The most important difference
85 is that :func:`import_module` returns the most nested package or module
86 that was imported (e.g. ``pkg.mod``), while :func:`__import__` returns the
Guido van Rossum09613542009-03-30 20:34:57 +000087 top-level package or module (e.g. ``pkg``).
88
Brett Cannonee78a2b2012-05-12 17:43:17 -040089.. function:: find_loader(name, path=None)
90
91 Find the loader for a module, optionally within the specified *path*. If the
92 module is in :attr:`sys.modules`, then ``sys.modules[name].__loader__`` is
Brett Cannon32799232013-03-13 11:09:08 -070093 returned (unless the loader would be ``None`` or is not set, in which case
Brett Cannonee78a2b2012-05-12 17:43:17 -040094 :exc:`ValueError` is raised). Otherwise a search using :attr:`sys.meta_path`
95 is done. ``None`` is returned if no loader is found.
96
Brett Cannon56b4ca72012-11-17 09:30:55 -050097 A dotted name does not have its parent's implicitly imported as that requires
98 loading them and that may not be desired. To properly import a submodule you
99 will need to import all parent packages of the submodule and use the correct
100 argument to *path*.
Brett Cannonee78a2b2012-05-12 17:43:17 -0400101
Brett Cannon32799232013-03-13 11:09:08 -0700102 .. versionadded:: 3.3
103
104 .. versionchanged:: 3.4
105 If ``__loader__`` is not set, raise :exc:`ValueError`, just like when the
106 attribute is set to ``None``.
107
Antoine Pitrouc541f8e2012-02-20 01:48:16 +0100108.. function:: invalidate_caches()
109
Brett Cannonf4dc9202012-08-10 12:21:12 -0400110 Invalidate the internal caches of finders stored at
111 :data:`sys.meta_path`. If a finder implements ``invalidate_caches()`` then it
Brett Cannon4067aa22013-04-27 23:20:32 -0400112 will be called to perform the invalidation. This function should be called
113 if any modules are created/installed while your program is running to
114 guarantee all finders will notice the new module's existence.
Antoine Pitrouc541f8e2012-02-20 01:48:16 +0100115
116 .. versionadded:: 3.3
117
Brett Cannon78246b62009-01-25 04:56:30 +0000118
Brett Cannon2a922ed2009-03-09 03:35:50 +0000119:mod:`importlib.abc` -- Abstract base classes related to import
120---------------------------------------------------------------
121
122.. module:: importlib.abc
123 :synopsis: Abstract base classes related to import
124
125The :mod:`importlib.abc` module contains all of the core abstract base classes
126used by :keyword:`import`. Some subclasses of the core abstract base classes
127are also provided to help in implementing the core ABCs.
128
Andrew Svetlova8656542012-08-13 22:19:01 +0300129ABC hierarchy::
130
131 object
Brett Cannon1b799182012-08-17 14:08:24 -0400132 +-- Finder (deprecated)
Andrew Svetlova8656542012-08-13 22:19:01 +0300133 | +-- MetaPathFinder
134 | +-- PathEntryFinder
135 +-- Loader
136 +-- ResourceLoader --------+
137 +-- InspectLoader |
138 +-- ExecutionLoader --+
139 +-- FileLoader
140 +-- SourceLoader
Andrew Svetlova8656542012-08-13 22:19:01 +0300141
Brett Cannon2a922ed2009-03-09 03:35:50 +0000142
143.. class:: Finder
144
Brett Cannon1b799182012-08-17 14:08:24 -0400145 An abstract base class representing a :term:`finder`.
146
147 .. deprecated:: 3.3
148 Use :class:`MetaPathFinder` or :class:`PathEntryFinder` instead.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000149
Brett Cannonf4dc9202012-08-10 12:21:12 -0400150 .. method:: find_module(fullname, path=None)
Brett Cannonb46a1792012-02-27 18:15:42 -0500151
Brett Cannonf4dc9202012-08-10 12:21:12 -0400152 An abstact method for finding a :term:`loader` for the specified
153 module. Originally specified in :pep:`302`, this method was meant
154 for use in :data:`sys.meta_path` and in the path-based import subsystem.
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000155
Brett Cannon100883f2013-04-09 16:59:39 -0400156 .. versionchanged:: 3.4
157 Returns ``None`` when called instead of raising
158 :exc:`NotImplementedError`.
159
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000160
Brett Cannon077ef452012-08-02 17:50:06 -0400161.. class:: MetaPathFinder
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000162
Brett Cannonf4dc9202012-08-10 12:21:12 -0400163 An abstract base class representing a :term:`meta path finder`. For
164 compatibility, this is a subclass of :class:`Finder`.
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000165
166 .. versionadded:: 3.3
167
168 .. method:: find_module(fullname, path)
169
170 An abstract method for finding a :term:`loader` for the specified
171 module. If this is a top-level import, *path* will be ``None``.
Ezio Melotti1f67e802012-10-21 07:24:13 +0300172 Otherwise, this is a search for a subpackage or module and *path*
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000173 will be the value of :attr:`__path__` from the parent
174 package. If a loader cannot be found, ``None`` is returned.
175
Brett Cannon100883f2013-04-09 16:59:39 -0400176 .. versionchanged:: 3.4
177 Returns ``None`` when called instead of raising
178 :exc:`NotImplementedError`.
179
Brett Cannonf4dc9202012-08-10 12:21:12 -0400180 .. method:: invalidate_caches()
181
182 An optional method which, when called, should invalidate any internal
Brett Cannona6e85812012-08-11 19:41:27 -0400183 cache used by the finder. Used by :func:`importlib.invalidate_caches`
184 when invalidating the caches of all finders on :data:`sys.meta_path`.
Brett Cannonf4dc9202012-08-10 12:21:12 -0400185
Brett Cannon100883f2013-04-09 16:59:39 -0400186 .. versionchanged:: 3.4
187 Returns ``None`` when called instead of ``NotImplemented``.
188
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000189
Brett Cannon077ef452012-08-02 17:50:06 -0400190.. class:: PathEntryFinder
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000191
Brett Cannonf4dc9202012-08-10 12:21:12 -0400192 An abstract base class representing a :term:`path entry finder`. Though
193 it bears some similarities to :class:`MetaPathFinder`, ``PathEntryFinder``
194 is meant for use only within the path-based import subsystem provided
195 by :class:`PathFinder`. This ABC is a subclass of :class:`Finder` for
Brett Cannon100883f2013-04-09 16:59:39 -0400196 compatibility reasons only.
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000197
198 .. versionadded:: 3.3
199
Brett Cannon4067aa22013-04-27 23:20:32 -0400200 .. method:: find_loader(fullname)
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000201
202 An abstract method for finding a :term:`loader` for the specified
Brett Cannonf4dc9202012-08-10 12:21:12 -0400203 module. Returns a 2-tuple of ``(loader, portion)`` where ``portion``
204 is a sequence of file system locations contributing to part of a namespace
205 package. The loader may be ``None`` while specifying ``portion`` to
206 signify the contribution of the file system locations to a namespace
207 package. An empty list can be used for ``portion`` to signify the loader
Brett Cannon100883f2013-04-09 16:59:39 -0400208 is not part of a namespace package. If ``loader`` is ``None`` and
209 ``portion`` is the empty list then no loader or location for a namespace
210 package were found (i.e. failure to find anything for the module).
211
212 .. versionchanged:: 3.4
213 Returns ``(None, [])`` instead of raising :exc:`NotImplementedError`.
Brett Cannonf4dc9202012-08-10 12:21:12 -0400214
Brett Cannon4067aa22013-04-27 23:20:32 -0400215 .. method:: find_module(fullname)
Brett Cannonf4dc9202012-08-10 12:21:12 -0400216
217 A concrete implementation of :meth:`Finder.find_module` which is
218 equivalent to ``self.find_loader(fullname)[0]``.
219
220 .. method:: invalidate_caches()
221
222 An optional method which, when called, should invalidate any internal
Brett Cannona6e85812012-08-11 19:41:27 -0400223 cache used by the finder. Used by :meth:`PathFinder.invalidate_caches`
Brett Cannonf4dc9202012-08-10 12:21:12 -0400224 when invalidating the caches of all cached finders.
Brett Cannonb46a1792012-02-27 18:15:42 -0500225
Brett Cannon2a922ed2009-03-09 03:35:50 +0000226
227.. class:: Loader
228
229 An abstract base class for a :term:`loader`.
Guido van Rossum09613542009-03-30 20:34:57 +0000230 See :pep:`302` for the exact definition for a loader.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000231
Brett Cannon9c751b72009-03-09 16:28:16 +0000232 .. method:: load_module(fullname)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000233
234 An abstract method for loading a module. If the module cannot be
235 loaded, :exc:`ImportError` is raised, otherwise the loaded module is
236 returned.
237
Guido van Rossum09613542009-03-30 20:34:57 +0000238 If the requested module already exists in :data:`sys.modules`, that
Brett Cannon2a922ed2009-03-09 03:35:50 +0000239 module should be used and reloaded.
Guido van Rossum09613542009-03-30 20:34:57 +0000240 Otherwise the loader should create a new module and insert it into
241 :data:`sys.modules` before any loading begins, to prevent recursion
242 from the import. If the loader inserted a module and the load fails, it
Brett Cannon2a922ed2009-03-09 03:35:50 +0000243 must be removed by the loader from :data:`sys.modules`; modules already
244 in :data:`sys.modules` before the loader began execution should be left
245 alone. The :func:`importlib.util.module_for_loader` decorator handles
246 all of these details.
247
Guido van Rossum09613542009-03-30 20:34:57 +0000248 The loader should set several attributes on the module.
249 (Note that some of these attributes can change when a module is
250 reloaded.)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000251
252 - :attr:`__name__`
253 The name of the module.
254
255 - :attr:`__file__`
256 The path to where the module data is stored (not set for built-in
257 modules).
258
259 - :attr:`__path__`
Guido van Rossum09613542009-03-30 20:34:57 +0000260 A list of strings specifying the search path within a
Brett Cannon2a922ed2009-03-09 03:35:50 +0000261 package. This attribute is not set on modules.
262
263 - :attr:`__package__`
264 The parent package for the module/package. If the module is
265 top-level then it has a value of the empty string. The
Brett Cannon100883f2013-04-09 16:59:39 -0400266 :func:`importlib.util.module_for_loader` decorator can handle the
267 details for :attr:`__package__`.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000268
269 - :attr:`__loader__`
Brett Cannon100883f2013-04-09 16:59:39 -0400270 The loader used to load the module. The
271 :func:`importlib.util.module_for_loader` decorator can handle the
272 details for :attr:`__package__`.
273
274 .. versionchanged:: 3.4
275 Raise :exc:`ImportError` when called instead of
276 :exc:`NotImplementedError`.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000277
Barry Warsawd7d21942012-07-29 16:36:17 -0400278 .. method:: module_repr(module)
279
Brett Cannon100883f2013-04-09 16:59:39 -0400280 An optional method which when implemented calculates and returns the
281 given module's repr, as a string. The module type's default repr() will
282 use the result of this method as appropriate.
Barry Warsawd7d21942012-07-29 16:36:17 -0400283
Georg Brandl526575d2013-04-11 16:10:13 +0200284 .. versionadded:: 3.3
Barry Warsawd7d21942012-07-29 16:36:17 -0400285
Brett Cannon100883f2013-04-09 16:59:39 -0400286 .. versionchanged:: 3.4
Georg Brandl526575d2013-04-11 16:10:13 +0200287 Made optional instead of an abstractmethod.
Brett Cannon100883f2013-04-09 16:59:39 -0400288
Brett Cannon2a922ed2009-03-09 03:35:50 +0000289
290.. class:: ResourceLoader
291
292 An abstract base class for a :term:`loader` which implements the optional
293 :pep:`302` protocol for loading arbitrary resources from the storage
294 back-end.
295
Brett Cannon9c751b72009-03-09 16:28:16 +0000296 .. method:: get_data(path)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000297
298 An abstract method to return the bytes for the data located at *path*.
Guido van Rossum09613542009-03-30 20:34:57 +0000299 Loaders that have a file-like storage back-end
Brett Cannon16248a42009-04-01 20:47:14 +0000300 that allows storing arbitrary data
Guido van Rossum09613542009-03-30 20:34:57 +0000301 can implement this abstract method to give direct access
Brett Cannon2a922ed2009-03-09 03:35:50 +0000302 to the data stored. :exc:`IOError` is to be raised if the *path* cannot
303 be found. The *path* is expected to be constructed using a module's
Brett Cannon16248a42009-04-01 20:47:14 +0000304 :attr:`__file__` attribute or an item from a package's :attr:`__path__`.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000305
Brett Cannon100883f2013-04-09 16:59:39 -0400306 .. versionchanged:: 3.4
307 Raises :exc:`IOError` instead of :exc:`NotImplementedError`.
308
Brett Cannon2a922ed2009-03-09 03:35:50 +0000309
310.. class:: InspectLoader
311
312 An abstract base class for a :term:`loader` which implements the optional
Guido van Rossum09613542009-03-30 20:34:57 +0000313 :pep:`302` protocol for loaders that inspect modules.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000314
Brett Cannona113ac52009-03-15 01:41:33 +0000315 .. method:: get_code(fullname)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000316
Brett Cannona113ac52009-03-15 01:41:33 +0000317 An abstract method to return the :class:`code` object for a module.
Georg Brandl375aec22011-01-15 17:03:02 +0000318 ``None`` is returned if the module does not have a code object
Brett Cannona113ac52009-03-15 01:41:33 +0000319 (e.g. built-in module). :exc:`ImportError` is raised if loader cannot
320 find the requested module.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000321
R David Murray1b00f252012-08-15 10:43:58 -0400322 .. index::
323 single: universal newlines; importlib.abc.InspectLoader.get_source method
324
Brett Cannon100883f2013-04-09 16:59:39 -0400325 .. versionchanged:: 3.4
326 Raises :exc:`ImportError` instead of :exc:`NotImplementedError`.
327
Brett Cannon9c751b72009-03-09 16:28:16 +0000328 .. method:: get_source(fullname)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000329
330 An abstract method to return the source of a module. It is returned as
R David Murray1b00f252012-08-15 10:43:58 -0400331 a text string using :term:`universal newlines`, translating all
R David Murrayee0a9452012-08-15 11:05:36 -0400332 recognized line separators into ``'\n'`` characters. Returns ``None``
333 if no source is available (e.g. a built-in module). Raises
334 :exc:`ImportError` if the loader cannot find the module specified.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000335
Brett Cannon100883f2013-04-09 16:59:39 -0400336 .. versionchanged:: 3.4
337 Raises :exc:`ImportError` instead of :exc:`NotImplementedError`.
338
Brett Cannona113ac52009-03-15 01:41:33 +0000339 .. method:: is_package(fullname)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000340
Brett Cannona113ac52009-03-15 01:41:33 +0000341 An abstract method to return a true value if the module is a package, a
342 false value otherwise. :exc:`ImportError` is raised if the
343 :term:`loader` cannot find the module.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000344
Brett Cannon100883f2013-04-09 16:59:39 -0400345 .. versionchanged:: 3.4
346 Raises :exc:`ImportError` instead of :exc:`NotImplementedError`.
347
Brett Cannon2a922ed2009-03-09 03:35:50 +0000348
Brett Cannon69194272009-07-20 04:23:48 +0000349.. class:: ExecutionLoader
350
351 An abstract base class which inherits from :class:`InspectLoader` that,
Brett Cannon23460292009-07-20 22:59:00 +0000352 when implemented, helps a module to be executed as a script. The ABC
Brett Cannon69194272009-07-20 04:23:48 +0000353 represents an optional :pep:`302` protocol.
354
355 .. method:: get_filename(fullname)
356
Brett Cannonf23e3742010-06-27 23:57:46 +0000357 An abstract method that is to return the value of :attr:`__file__` for
Brett Cannon69194272009-07-20 04:23:48 +0000358 the specified module. If no path is available, :exc:`ImportError` is
359 raised.
360
Brett Cannonf23e3742010-06-27 23:57:46 +0000361 If source code is available, then the method should return the path to
362 the source file, regardless of whether a bytecode was used to load the
363 module.
364
Brett Cannon100883f2013-04-09 16:59:39 -0400365 .. versionchanged:: 3.4
366 Raises :exc:`ImportError` instead of :exc:`NotImplementedError`.
367
Brett Cannonf23e3742010-06-27 23:57:46 +0000368
Brett Cannon938d44d2012-04-22 19:58:33 -0400369.. class:: FileLoader(fullname, path)
370
371 An abstract base class which inherits from :class:`ResourceLoader` and
Andrew Svetlova60de4f2013-02-17 16:55:58 +0200372 :class:`ExecutionLoader`, providing concrete implementations of
Brett Cannon938d44d2012-04-22 19:58:33 -0400373 :meth:`ResourceLoader.get_data` and :meth:`ExecutionLoader.get_filename`.
374
375 The *fullname* argument is a fully resolved name of the module the loader is
376 to handle. The *path* argument is the path to the file for the module.
377
378 .. versionadded:: 3.3
379
380 .. attribute:: name
381
382 The name of the module the loader can handle.
383
384 .. attribute:: path
385
386 Path to the file of the module.
387
Barry Warsawd7d21942012-07-29 16:36:17 -0400388 .. method:: load_module(fullname)
Brett Cannonc0499522012-05-11 14:48:41 -0400389
Barry Warsawd7d21942012-07-29 16:36:17 -0400390 Calls super's ``load_module()``.
Brett Cannonc0499522012-05-11 14:48:41 -0400391
Brett Cannon938d44d2012-04-22 19:58:33 -0400392 .. method:: get_filename(fullname)
393
Barry Warsawd7d21942012-07-29 16:36:17 -0400394 Returns :attr:`path`.
Brett Cannon938d44d2012-04-22 19:58:33 -0400395
396 .. method:: get_data(path)
397
398 Returns the open, binary file for *path*.
399
400
Brett Cannonf23e3742010-06-27 23:57:46 +0000401.. class:: SourceLoader
402
403 An abstract base class for implementing source (and optionally bytecode)
404 file loading. The class inherits from both :class:`ResourceLoader` and
405 :class:`ExecutionLoader`, requiring the implementation of:
406
407 * :meth:`ResourceLoader.get_data`
408 * :meth:`ExecutionLoader.get_filename`
Brett Cannon6dfbff32010-07-21 09:48:31 +0000409 Should only return the path to the source file; sourceless
Brett Cannon5650e4f2012-11-18 10:03:31 -0500410 loading is not supported (see :class:`SourcelessLoader` if that
411 functionality is required)
Brett Cannonf23e3742010-06-27 23:57:46 +0000412
413 The abstract methods defined by this class are to add optional bytecode
Brett Cannon5650e4f2012-11-18 10:03:31 -0500414 file support. Not implementing these optional methods (or causing them to
415 raise :exc:`NotImplementedError`) causes the loader to
Brett Cannonf23e3742010-06-27 23:57:46 +0000416 only work with source code. Implementing the methods allows the loader to
417 work with source *and* bytecode files; it does not allow for *sourceless*
418 loading where only bytecode is provided. Bytecode files are an
419 optimization to speed up loading by removing the parsing step of Python's
420 compiler, and so no bytecode-specific API is exposed.
421
Brett Cannon773468f2012-08-02 17:35:34 -0400422 .. method:: path_stats(path)
Antoine Pitrou5136ac02012-01-13 18:52:16 +0100423
424 Optional abstract method which returns a :class:`dict` containing
425 metadata about the specifed path. Supported dictionary keys are:
426
427 - ``'mtime'`` (mandatory): an integer or floating-point number
428 representing the modification time of the source code;
429 - ``'size'`` (optional): the size in bytes of the source code.
430
431 Any other keys in the dictionary are ignored, to allow for future
Brett Cannon100883f2013-04-09 16:59:39 -0400432 extensions. If the path cannot be handled, :exc:`IOError` is raised.
Antoine Pitrou5136ac02012-01-13 18:52:16 +0100433
434 .. versionadded:: 3.3
435
Brett Cannon100883f2013-04-09 16:59:39 -0400436 .. versionchanged:: 3.4
437 Raise :exc:`IOError` instead of :exc:`NotImplementedError`.
438
Brett Cannon773468f2012-08-02 17:35:34 -0400439 .. method:: path_mtime(path)
Brett Cannonf23e3742010-06-27 23:57:46 +0000440
441 Optional abstract method which returns the modification time for the
442 specified path.
443
Antoine Pitrou5136ac02012-01-13 18:52:16 +0100444 .. deprecated:: 3.3
445 This method is deprecated in favour of :meth:`path_stats`. You don't
446 have to implement it, but it is still available for compatibility
Brett Cannon100883f2013-04-09 16:59:39 -0400447 purposes. Raise :exc:`IOError` if the path cannot be handled.
448
449 .. versionchanged:: 3.4
450 Raise :exc:`IOError` instead of :exc:`NotImplementedError`.
Antoine Pitrou5136ac02012-01-13 18:52:16 +0100451
Brett Cannon773468f2012-08-02 17:35:34 -0400452 .. method:: set_data(path, data)
Brett Cannonf23e3742010-06-27 23:57:46 +0000453
454 Optional abstract method which writes the specified bytes to a file
Brett Cannon61b14252010-07-03 21:48:25 +0000455 path. Any intermediate directories which do not exist are to be created
456 automatically.
457
458 When writing to the path fails because the path is read-only
459 (:attr:`errno.EACCES`), do not propagate the exception.
Brett Cannonf23e3742010-06-27 23:57:46 +0000460
Brett Cannon100883f2013-04-09 16:59:39 -0400461 .. versionchanged:: 3.4
462 No longer raises :exc:`NotImplementedError` when called.
463
Eric Snowa6cfb282012-12-04 23:43:43 -0800464 .. method:: source_to_code(data, path)
Brett Cannon5650e4f2012-11-18 10:03:31 -0500465
466 Create a code object from Python source.
467
468 The *data* argument can be whatever the :func:`compile` function
469 supports (i.e. string or bytes). The *path* argument should be
470 the "path" to where the source code originated from, which can be an
471 abstract concept (e.g. location in a zip file).
472
473 .. versionadded:: 3.4
474
Brett Cannon773468f2012-08-02 17:35:34 -0400475 .. method:: get_code(fullname)
Brett Cannonf23e3742010-06-27 23:57:46 +0000476
477 Concrete implementation of :meth:`InspectLoader.get_code`.
478
Brett Cannon773468f2012-08-02 17:35:34 -0400479 .. method:: load_module(fullname)
Brett Cannonf23e3742010-06-27 23:57:46 +0000480
481 Concrete implementation of :meth:`Loader.load_module`.
482
Brett Cannon773468f2012-08-02 17:35:34 -0400483 .. method:: get_source(fullname)
Brett Cannonf23e3742010-06-27 23:57:46 +0000484
485 Concrete implementation of :meth:`InspectLoader.get_source`.
486
Brett Cannon773468f2012-08-02 17:35:34 -0400487 .. method:: is_package(fullname)
Brett Cannonf23e3742010-06-27 23:57:46 +0000488
489 Concrete implementation of :meth:`InspectLoader.is_package`. A module
Brett Cannonea0b8232012-06-15 20:00:53 -0400490 is determined to be a package if its file path (as provided by
491 :meth:`ExecutionLoader.get_filename`) is a file named
492 ``__init__`` when the file extension is removed **and** the module name
493 itself does not end in ``__init__``.
Brett Cannonf23e3742010-06-27 23:57:46 +0000494
Brett Cannon69194272009-07-20 04:23:48 +0000495
Brett Cannon78246b62009-01-25 04:56:30 +0000496:mod:`importlib.machinery` -- Importers and path hooks
497------------------------------------------------------
498
499.. module:: importlib.machinery
500 :synopsis: Importers and path hooks
501
502This module contains the various objects that help :keyword:`import`
503find and load modules.
504
Brett Cannoncb66eb02012-05-11 12:58:42 -0400505.. attribute:: SOURCE_SUFFIXES
506
507 A list of strings representing the recognized file suffixes for source
508 modules.
509
510 .. versionadded:: 3.3
511
512.. attribute:: DEBUG_BYTECODE_SUFFIXES
513
514 A list of strings representing the file suffixes for non-optimized bytecode
515 modules.
516
517 .. versionadded:: 3.3
518
519.. attribute:: OPTIMIZED_BYTECODE_SUFFIXES
520
521 A list of strings representing the file suffixes for optimized bytecode
522 modules.
523
524 .. versionadded:: 3.3
525
526.. attribute:: BYTECODE_SUFFIXES
527
528 A list of strings representing the recognized file suffixes for bytecode
529 modules. Set to either :attr:`DEBUG_BYTECODE_SUFFIXES` or
530 :attr:`OPTIMIZED_BYTECODE_SUFFIXES` based on whether ``__debug__`` is true.
531
532 .. versionadded:: 3.3
533
534.. attribute:: EXTENSION_SUFFIXES
535
Nick Coghlan76e07702012-07-18 23:14:57 +1000536 A list of strings representing the recognized file suffixes for
Brett Cannoncb66eb02012-05-11 12:58:42 -0400537 extension modules.
538
539 .. versionadded:: 3.3
540
Nick Coghlanc5afd422012-07-18 23:59:08 +1000541.. function:: all_suffixes()
Nick Coghlan76e07702012-07-18 23:14:57 +1000542
543 Returns a combined list of strings representing all file suffixes for
Nick Coghlanc5afd422012-07-18 23:59:08 +1000544 modules recognized by the standard import machinery. This is a
Nick Coghlan76e07702012-07-18 23:14:57 +1000545 helper for code which simply needs to know if a filesystem path
Nick Coghlanc5afd422012-07-18 23:59:08 +1000546 potentially refers to a module without needing any details on the kind
547 of module (for example, :func:`inspect.getmodulename`)
Nick Coghlan76e07702012-07-18 23:14:57 +1000548
549 .. versionadded:: 3.3
550
551
Brett Cannon78246b62009-01-25 04:56:30 +0000552.. class:: BuiltinImporter
553
Brett Cannon2a922ed2009-03-09 03:35:50 +0000554 An :term:`importer` for built-in modules. All known built-in modules are
555 listed in :data:`sys.builtin_module_names`. This class implements the
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000556 :class:`importlib.abc.MetaPathFinder` and
557 :class:`importlib.abc.InspectLoader` ABCs.
Brett Cannon78246b62009-01-25 04:56:30 +0000558
559 Only class methods are defined by this class to alleviate the need for
560 instantiation.
561
Brett Cannon78246b62009-01-25 04:56:30 +0000562
563.. class:: FrozenImporter
564
Brett Cannon2a922ed2009-03-09 03:35:50 +0000565 An :term:`importer` for frozen modules. This class implements the
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000566 :class:`importlib.abc.MetaPathFinder` and
567 :class:`importlib.abc.InspectLoader` ABCs.
Brett Cannon78246b62009-01-25 04:56:30 +0000568
569 Only class methods are defined by this class to alleviate the need for
570 instantiation.
571
Brett Cannondebb98d2009-02-16 04:18:01 +0000572
Nick Coghlanff794862012-08-02 21:45:24 +1000573.. class:: WindowsRegistryFinder
574
575 :term:`Finder` for modules declared in the Windows registry. This class
Nick Coghlan49417742012-08-02 23:03:58 +1000576 implements the :class:`importlib.abc.Finder` ABC.
Nick Coghlanff794862012-08-02 21:45:24 +1000577
578 Only class methods are defined by this class to alleviate the need for
579 instantiation.
580
581 .. versionadded:: 3.3
582
583
Brett Cannondebb98d2009-02-16 04:18:01 +0000584.. class:: PathFinder
585
Brett Cannon1b799182012-08-17 14:08:24 -0400586 A :term:`Finder` for :data:`sys.path` and package ``__path__`` attributes.
587 This class implements the :class:`importlib.abc.MetaPathFinder` ABC.
Brett Cannondebb98d2009-02-16 04:18:01 +0000588
Brett Cannon1b799182012-08-17 14:08:24 -0400589 Only class methods are defined by this class to alleviate the need for
590 instantiation.
Brett Cannondebb98d2009-02-16 04:18:01 +0000591
Brett Cannon1b799182012-08-17 14:08:24 -0400592 .. classmethod:: find_module(fullname, path=None)
Brett Cannondebb98d2009-02-16 04:18:01 +0000593
Brett Cannon1b799182012-08-17 14:08:24 -0400594 Class method that attempts to find a :term:`loader` for the module
595 specified by *fullname* on :data:`sys.path` or, if defined, on
596 *path*. For each path entry that is searched,
597 :data:`sys.path_importer_cache` is checked. If a non-false object is
598 found then it is used as the :term:`finder` to look for the module
599 being searched for. If no entry is found in
600 :data:`sys.path_importer_cache`, then :data:`sys.path_hooks` is
601 searched for a finder for the path entry and, if found, is stored in
602 :data:`sys.path_importer_cache` along with being queried about the
603 module. If no finder is ever found then ``None`` is both stored in
604 the cache and returned.
Brett Cannond2e7b332009-02-17 02:45:03 +0000605
Brett Cannonf4dc9202012-08-10 12:21:12 -0400606 .. classmethod:: invalidate_caches()
607
Brett Cannon1b799182012-08-17 14:08:24 -0400608 Calls :meth:`importlib.abc.PathEntryFinder.invalidate_caches` on all
609 finders stored in :attr:`sys.path_importer_cache`.
Brett Cannonf4dc9202012-08-10 12:21:12 -0400610
Brett Cannond2e7b332009-02-17 02:45:03 +0000611
Brett Cannon938d44d2012-04-22 19:58:33 -0400612.. class:: FileFinder(path, \*loader_details)
613
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000614 A concrete implementation of :class:`importlib.abc.PathEntryFinder` which
615 caches results from the file system.
Brett Cannon938d44d2012-04-22 19:58:33 -0400616
617 The *path* argument is the directory for which the finder is in charge of
618 searching.
619
Brett Cannonac9f2f32012-08-10 13:47:54 -0400620 The *loader_details* argument is a variable number of 2-item tuples each
621 containing a loader and a sequence of file suffixes the loader recognizes.
Brett Cannon938d44d2012-04-22 19:58:33 -0400622
623 The finder will cache the directory contents as necessary, making stat calls
624 for each module search to verify the cache is not outdated. Because cache
625 staleness relies upon the granularity of the operating system's state
626 information of the file system, there is a potential race condition of
627 searching for a module, creating a new file, and then searching for the
628 module the new file represents. If the operations happen fast enough to fit
629 within the granularity of stat calls, then the module search will fail. To
630 prevent this from happening, when you create a module dynamically, make sure
631 to call :func:`importlib.invalidate_caches`.
632
633 .. versionadded:: 3.3
634
635 .. attribute:: path
636
637 The path the finder will search in.
638
639 .. method:: find_module(fullname)
640
641 Attempt to find the loader to handle *fullname* within :attr:`path`.
642
643 .. method:: invalidate_caches()
644
645 Clear out the internal cache.
646
647 .. classmethod:: path_hook(\*loader_details)
648
649 A class method which returns a closure for use on :attr:`sys.path_hooks`.
650 An instance of :class:`FileFinder` is returned by the closure using the
651 path argument given to the closure directly and *loader_details*
652 indirectly.
653
654 If the argument to the closure is not an existing directory,
655 :exc:`ImportError` is raised.
656
657
658.. class:: SourceFileLoader(fullname, path)
659
660 A concrete implementation of :class:`importlib.abc.SourceLoader` by
661 subclassing :class:`importlib.abc.FileLoader` and providing some concrete
662 implementations of other methods.
663
664 .. versionadded:: 3.3
665
666 .. attribute:: name
667
668 The name of the module that this loader will handle.
669
670 .. attribute:: path
671
672 The path to the source file.
673
674 .. method:: is_package(fullname)
675
676 Return true if :attr:`path` appears to be for a package.
677
678 .. method:: path_stats(path)
679
680 Concrete implementation of :meth:`importlib.abc.SourceLoader.path_stats`.
681
682 .. method:: set_data(path, data)
683
684 Concrete implementation of :meth:`importlib.abc.SourceLoader.set_data`.
685
Brett Cannon938d44d2012-04-22 19:58:33 -0400686
Marc-Andre Lemburg4fe29c92012-04-25 02:31:37 +0200687.. class:: SourcelessFileLoader(fullname, path)
Brett Cannon938d44d2012-04-22 19:58:33 -0400688
689 A concrete implementation of :class:`importlib.abc.FileLoader` which can
690 import bytecode files (i.e. no source code files exist).
691
Marc-Andre Lemburg4fe29c92012-04-25 02:31:37 +0200692 Please note that direct use of bytecode files (and thus not source code
693 files) inhibits your modules from being usable by all Python
694 implementations or new versions of Python which change the bytecode
695 format.
Brett Cannon938d44d2012-04-22 19:58:33 -0400696
697 .. versionadded:: 3.3
698
699 .. attribute:: name
700
701 The name of the module the loader will handle.
702
703 .. attribute:: path
704
705 The path to the bytecode file.
706
707 .. method:: is_package(fullname)
708
709 Determines if the module is a package based on :attr:`path`.
710
711 .. method:: get_code(fullname)
712
713 Returns the code object for :attr:`name` created from :attr:`path`.
714
715 .. method:: get_source(fullname)
716
717 Returns ``None`` as bytecode files have no source when this loader is
718 used.
719
Brett Cannon938d44d2012-04-22 19:58:33 -0400720
721.. class:: ExtensionFileLoader(fullname, path)
722
723 A concrete implementation of :class:`importlib.abc.InspectLoader` for
724 extension modules.
725
726 The *fullname* argument specifies the name of the module the loader is to
727 support. The *path* argument is the path to the extension module's file.
728
729 .. versionadded:: 3.3
730
731 .. attribute:: name
732
733 Name of the module the loader supports.
734
735 .. attribute:: path
736
737 Path to the extension module.
738
Barry Warsawd7d21942012-07-29 16:36:17 -0400739 .. method:: load_module(fullname)
Brett Cannon938d44d2012-04-22 19:58:33 -0400740
Brett Cannonc0499522012-05-11 14:48:41 -0400741 Loads the extension module if and only if *fullname* is the same as
742 :attr:`name` or is ``None``.
Brett Cannon938d44d2012-04-22 19:58:33 -0400743
744 .. method:: is_package(fullname)
745
Brett Cannonac9f2f32012-08-10 13:47:54 -0400746 Returns ``True`` if the file path points to a package's ``__init__``
747 module based on :attr:`EXTENSION_SUFFIXES`.
Brett Cannon938d44d2012-04-22 19:58:33 -0400748
749 .. method:: get_code(fullname)
750
751 Returns ``None`` as extension modules lack a code object.
752
753 .. method:: get_source(fullname)
754
755 Returns ``None`` as extension modules do not have source code.
756
757
Brett Cannond2e7b332009-02-17 02:45:03 +0000758:mod:`importlib.util` -- Utility code for importers
759---------------------------------------------------
760
761.. module:: importlib.util
Brett Cannon75321e82012-03-02 11:58:25 -0500762 :synopsis: Utility code for importers
Brett Cannond2e7b332009-02-17 02:45:03 +0000763
764This module contains the various objects that help in the construction of
765an :term:`importer`.
766
Brett Cannond200bf52012-05-13 13:45:09 -0400767.. function:: resolve_name(name, package)
768
769 Resolve a relative module name to an absolute one.
770
771 If **name** has no leading dots, then **name** is simply returned. This
772 allows for usage such as
773 ``importlib.util.resolve_name('sys', __package__)`` without doing a
774 check to see if the **package** argument is needed.
775
776 :exc:`ValueError` is raised if **name** is a relative module name but
777 package is a false value (e.g. ``None`` or the empty string).
778 :exc:`ValueError` is also raised a relative name would escape its containing
779 package (e.g. requesting ``..bacon`` from within the ``spam`` package).
780
781 .. versionadded:: 3.3
782
Georg Brandl8a1caa22010-07-29 16:01:11 +0000783.. decorator:: module_for_loader
Brett Cannond2e7b332009-02-17 02:45:03 +0000784
Guido van Rossum09613542009-03-30 20:34:57 +0000785 A :term:`decorator` for a :term:`loader` method,
786 to handle selecting the proper
Brett Cannond2e7b332009-02-17 02:45:03 +0000787 module object to load with. The decorated method is expected to have a call
Brett Cannon2a922ed2009-03-09 03:35:50 +0000788 signature taking two positional arguments
789 (e.g. ``load_module(self, module)``) for which the second argument
Guido van Rossum09613542009-03-30 20:34:57 +0000790 will be the module **object** to be used by the loader.
Brett Cannonefad00d2012-04-27 17:27:14 -0400791 Note that the decorator will not work on static methods because of the
792 assumption of two arguments.
Brett Cannond2e7b332009-02-17 02:45:03 +0000793
Guido van Rossum09613542009-03-30 20:34:57 +0000794 The decorated method will take in the **name** of the module to be loaded
795 as expected for a :term:`loader`. If the module is not found in
Brett Cannon57b46f52009-03-02 14:38:26 +0000796 :data:`sys.modules` then a new one is constructed with its
Brett Cannonefad00d2012-04-27 17:27:14 -0400797 :attr:`__name__` attribute set to **name**, :attr:`__loader__` set to
798 **self**, and :attr:`__package__` set if
799 :meth:`importlib.abc.InspectLoader.is_package` is defined for **self** and
800 does not raise :exc:`ImportError` for **name**. If a new module is not
801 needed then the module found in :data:`sys.modules` will be passed into the
802 method.
803
804 If an exception is raised by the decorated method and a module was added to
Brett Cannond2e7b332009-02-17 02:45:03 +0000805 :data:`sys.modules` it will be removed to prevent a partially initialized
Brett Cannon57b46f52009-03-02 14:38:26 +0000806 module from being in left in :data:`sys.modules`. If the module was already
807 in :data:`sys.modules` then it is left alone.
Brett Cannond2e7b332009-02-17 02:45:03 +0000808
Guido van Rossum09613542009-03-30 20:34:57 +0000809 Use of this decorator handles all the details of which module object a
Brett Cannonefad00d2012-04-27 17:27:14 -0400810 loader should initialize as specified by :pep:`302` as best as possible.
811
812 .. versionchanged:: 3.3
Georg Brandl61063cc2012-06-24 22:48:30 +0200813 :attr:`__loader__` and :attr:`__package__` are automatically set
814 (when possible).
Brett Cannon57b46f52009-03-02 14:38:26 +0000815
Georg Brandl8a1caa22010-07-29 16:01:11 +0000816.. decorator:: set_loader
Brett Cannon2cf03a82009-03-10 05:17:37 +0000817
Guido van Rossum09613542009-03-30 20:34:57 +0000818 A :term:`decorator` for a :term:`loader` method,
819 to set the :attr:`__loader__`
Brett Cannon2cf03a82009-03-10 05:17:37 +0000820 attribute on loaded modules. If the attribute is already set the decorator
821 does nothing. It is assumed that the first positional argument to the
Brett Cannon75321e82012-03-02 11:58:25 -0500822 wrapped method (i.e. ``self``) is what :attr:`__loader__` should be set to.
Brett Cannon2cf03a82009-03-10 05:17:37 +0000823
Brett Cannonefad00d2012-04-27 17:27:14 -0400824 .. note::
825
826 It is recommended that :func:`module_for_loader` be used over this
827 decorator as it subsumes this functionality.
828
Brett Cannon4802bec2013-03-13 10:41:36 -0700829 .. versionchanged:: 3.4
Brett Cannon4c14b5d2013-05-04 13:56:58 -0400830 Set ``__loader__`` if set to ``None``, as if the attribute does not
Brett Cannon4802bec2013-03-13 10:41:36 -0700831 exist.
832
Brett Cannonefad00d2012-04-27 17:27:14 -0400833
Georg Brandl8a1caa22010-07-29 16:01:11 +0000834.. decorator:: set_package
Brett Cannon57b46f52009-03-02 14:38:26 +0000835
836 A :term:`decorator` for a :term:`loader` to set the :attr:`__package__`
837 attribute on the module returned by the loader. If :attr:`__package__` is
Georg Brandl375aec22011-01-15 17:03:02 +0000838 set and has a value other than ``None`` it will not be changed.
Brett Cannon57b46f52009-03-02 14:38:26 +0000839 Note that the module returned by the loader is what has the attribute
840 set on and not the module found in :data:`sys.modules`.
Guido van Rossum09613542009-03-30 20:34:57 +0000841
Brett Cannon16248a42009-04-01 20:47:14 +0000842 Reliance on this decorator is discouraged when it is possible to set
Brett Cannon75321e82012-03-02 11:58:25 -0500843 :attr:`__package__` before importing. By
844 setting it beforehand the code for the module is executed with the
845 attribute set and thus can be used by global level code during
Brett Cannon16248a42009-04-01 20:47:14 +0000846 initialization.
847
Brett Cannonefad00d2012-04-27 17:27:14 -0400848 .. note::
849
850 It is recommended that :func:`module_for_loader` be used over this
851 decorator as it subsumes this functionality.