blob: b114dd53df810349c71a56e6c6c548933acc5020 [file] [log] [blame]
Brett Cannonafccd632009-01-20 02:21:27 +00001:mod:`importlib` -- An implementation of :keyword:`import`
2==========================================================
3
4.. module:: importlib
5 :synopsis: An implementation of the import machinery.
6
7.. moduleauthor:: Brett Cannon <brett@python.org>
8.. sectionauthor:: Brett Cannon <brett@python.org>
9
10.. versionadded:: 3.1
11
12
13Introduction
14------------
15
16The purpose of the :mod:`importlib` package is two-fold. One is to provide an
17implementation of the :keyword:`import` statement (and thus, by extension, the
18:func:`__import__` function) in Python source code. This provides an
Tarek Ziadé434caaa2009-05-14 12:48:09 +000019implementation of :keyword:`import` which is portable to any Python
Brett Cannonafccd632009-01-20 02:21:27 +000020interpreter. This also provides a reference implementation which is easier to
Brett Cannonf23e3742010-06-27 23:57:46 +000021comprehend than one implemented in a programming language other than Python.
Brett Cannonafccd632009-01-20 02:21:27 +000022
Brett Cannonf23e3742010-06-27 23:57:46 +000023Two, the components to implement :keyword:`import` are exposed in this
Brett Cannonafccd632009-01-20 02:21:27 +000024package, making it easier for users to create their own custom objects (known
Brett Cannondebb98d2009-02-16 04:18:01 +000025generically as an :term:`importer`) to participate in the import process.
Brett Cannonf23e3742010-06-27 23:57:46 +000026Details on custom importers can be found in :pep:`302`.
Brett Cannonafccd632009-01-20 02:21:27 +000027
28.. seealso::
29
30 :ref:`import`
31 The language reference for the :keyword:`import` statement.
32
33 `Packages specification <http://www.python.org/doc/essays/packages.html>`__
34 Original specification of packages. Some semantics have changed since
Georg Brandl375aec22011-01-15 17:03:02 +000035 the writing of this document (e.g. redirecting based on ``None``
Brett Cannonafccd632009-01-20 02:21:27 +000036 in :data:`sys.modules`).
37
38 The :func:`.__import__` function
Brett Cannon0e13c942010-06-29 18:26:11 +000039 The :keyword:`import` statement is syntactic sugar for this function.
Brett Cannonafccd632009-01-20 02:21:27 +000040
41 :pep:`235`
42 Import on Case-Insensitive Platforms
43
44 :pep:`263`
45 Defining Python Source Code Encodings
46
47 :pep:`302`
Brett Cannonf23e3742010-06-27 23:57:46 +000048 New Import Hooks
Brett Cannonafccd632009-01-20 02:21:27 +000049
50 :pep:`328`
51 Imports: Multi-Line and Absolute/Relative
52
53 :pep:`366`
54 Main module explicit relative imports
55
Brett Cannon8917d5e2010-01-13 19:21:00 +000056 :pep:`3120`
Brett Cannonafccd632009-01-20 02:21:27 +000057 Using UTF-8 as the Default Source Encoding
58
Brett Cannon30b7a902010-06-27 21:49:22 +000059 :pep:`3147`
60 PYC Repository Directories
61
Brett Cannonafccd632009-01-20 02:21:27 +000062
63Functions
64---------
65
Brett Cannon78246b62009-01-25 04:56:30 +000066.. function:: __import__(name, globals={}, locals={}, fromlist=list(), level=0)
Brett Cannonafccd632009-01-20 02:21:27 +000067
Brett Cannonf23e3742010-06-27 23:57:46 +000068 An implementation of the built-in :func:`__import__` function.
Brett Cannonafccd632009-01-20 02:21:27 +000069
70.. function:: import_module(name, package=None)
71
Brett Cannon33418c82009-01-22 18:37:20 +000072 Import a module. The *name* argument specifies what module to
Brett Cannonafccd632009-01-20 02:21:27 +000073 import in absolute or relative terms
74 (e.g. either ``pkg.mod`` or ``..mod``). If the name is
Guido van Rossum09613542009-03-30 20:34:57 +000075 specified in relative terms, then the *package* argument must be set to
76 the name of the package which is to act as the anchor for resolving the
Brett Cannonafccd632009-01-20 02:21:27 +000077 package name (e.g. ``import_module('..mod', 'pkg.subpkg')`` will import
Brett Cannon2c318a12009-02-07 01:15:27 +000078 ``pkg.mod``).
Brett Cannon78246b62009-01-25 04:56:30 +000079
Brett Cannon2c318a12009-02-07 01:15:27 +000080 The :func:`import_module` function acts as a simplifying wrapper around
Brett Cannon9f4cb1c2009-04-01 23:26:47 +000081 :func:`importlib.__import__`. This means all semantics of the function are
82 derived from :func:`importlib.__import__`, including requiring the package
83 from which an import is occurring to have been previously imported
84 (i.e., *package* must already be imported). The most important difference
85 is that :func:`import_module` returns the most nested package or module
86 that was imported (e.g. ``pkg.mod``), while :func:`__import__` returns the
Guido van Rossum09613542009-03-30 20:34:57 +000087 top-level package or module (e.g. ``pkg``).
88
Brett Cannonee78a2b2012-05-12 17:43:17 -040089.. function:: find_loader(name, path=None)
90
91 Find the loader for a module, optionally within the specified *path*. If the
92 module is in :attr:`sys.modules`, then ``sys.modules[name].__loader__`` is
93 returned (unless the loader would be ``None``, in which case
94 :exc:`ValueError` is raised). Otherwise a search using :attr:`sys.meta_path`
95 is done. ``None`` is returned if no loader is found.
96
97 A dotted name does not have its parent's implicitly imported. If that is
98 desired (although not nessarily required to find the loader, it will most
99 likely be needed if the loader actually is used to load the module), then
100 you will have to import the packages containing the module prior to calling
101 this function.
102
Antoine Pitrouc541f8e2012-02-20 01:48:16 +0100103.. function:: invalidate_caches()
104
Brett Cannonb46a1792012-02-27 18:15:42 -0500105 Invalidate the internal caches of the finders stored at
106 :data:`sys.path_importer_cache`. If a finder implements
107 :meth:`abc.Finder.invalidate_caches()` then it will be called to perform the
108 invalidation. This function may be needed if some modules are installed
109 while your program is running and you expect the program to notice the
110 changes.
Antoine Pitrouc541f8e2012-02-20 01:48:16 +0100111
112 .. versionadded:: 3.3
113
Brett Cannon78246b62009-01-25 04:56:30 +0000114
Brett Cannon2a922ed2009-03-09 03:35:50 +0000115:mod:`importlib.abc` -- Abstract base classes related to import
116---------------------------------------------------------------
117
118.. module:: importlib.abc
119 :synopsis: Abstract base classes related to import
120
121The :mod:`importlib.abc` module contains all of the core abstract base classes
122used by :keyword:`import`. Some subclasses of the core abstract base classes
123are also provided to help in implementing the core ABCs.
124
125
126.. class:: Finder
127
128 An abstract base class representing a :term:`finder`.
Guido van Rossum09613542009-03-30 20:34:57 +0000129 See :pep:`302` for the exact definition for a finder.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000130
Brett Cannon9c751b72009-03-09 16:28:16 +0000131 .. method:: find_module(fullname, path=None)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000132
133 An abstract method for finding a :term:`loader` for the specified
134 module. If the :term:`finder` is found on :data:`sys.meta_path` and the
Guido van Rossum09613542009-03-30 20:34:57 +0000135 module to be searched for is a subpackage or module then *path* will
136 be the value of :attr:`__path__` from the parent package. If a loader
Georg Brandl375aec22011-01-15 17:03:02 +0000137 cannot be found, ``None`` is returned.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000138
Brett Cannonb46a1792012-02-27 18:15:42 -0500139 .. method:: invalidate_caches()
140
141 An optional method which, when called, should invalidate any internal
142 cache used by the finder. Used by :func:`invalidate_caches()` when
143 invalidating the caches of all cached finders.
144
Brett Cannon2a922ed2009-03-09 03:35:50 +0000145
146.. class:: Loader
147
148 An abstract base class for a :term:`loader`.
Guido van Rossum09613542009-03-30 20:34:57 +0000149 See :pep:`302` for the exact definition for a loader.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000150
Brett Cannon9c751b72009-03-09 16:28:16 +0000151 .. method:: load_module(fullname)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000152
153 An abstract method for loading a module. If the module cannot be
154 loaded, :exc:`ImportError` is raised, otherwise the loaded module is
155 returned.
156
Guido van Rossum09613542009-03-30 20:34:57 +0000157 If the requested module already exists in :data:`sys.modules`, that
Brett Cannon2a922ed2009-03-09 03:35:50 +0000158 module should be used and reloaded.
Guido van Rossum09613542009-03-30 20:34:57 +0000159 Otherwise the loader should create a new module and insert it into
160 :data:`sys.modules` before any loading begins, to prevent recursion
161 from the import. If the loader inserted a module and the load fails, it
Brett Cannon2a922ed2009-03-09 03:35:50 +0000162 must be removed by the loader from :data:`sys.modules`; modules already
163 in :data:`sys.modules` before the loader began execution should be left
164 alone. The :func:`importlib.util.module_for_loader` decorator handles
165 all of these details.
166
Guido van Rossum09613542009-03-30 20:34:57 +0000167 The loader should set several attributes on the module.
168 (Note that some of these attributes can change when a module is
169 reloaded.)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000170
171 - :attr:`__name__`
172 The name of the module.
173
174 - :attr:`__file__`
175 The path to where the module data is stored (not set for built-in
176 modules).
177
178 - :attr:`__path__`
Guido van Rossum09613542009-03-30 20:34:57 +0000179 A list of strings specifying the search path within a
Brett Cannon2a922ed2009-03-09 03:35:50 +0000180 package. This attribute is not set on modules.
181
182 - :attr:`__package__`
183 The parent package for the module/package. If the module is
184 top-level then it has a value of the empty string. The
185 :func:`importlib.util.set_package` decorator can handle the details
186 for :attr:`__package__`.
187
188 - :attr:`__loader__`
Guido van Rossum09613542009-03-30 20:34:57 +0000189 The loader used to load the module.
190 (This is not set by the built-in import machinery,
191 but it should be set whenever a :term:`loader` is used.)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000192
193
194.. class:: ResourceLoader
195
196 An abstract base class for a :term:`loader` which implements the optional
197 :pep:`302` protocol for loading arbitrary resources from the storage
198 back-end.
199
Brett Cannon9c751b72009-03-09 16:28:16 +0000200 .. method:: get_data(path)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000201
202 An abstract method to return the bytes for the data located at *path*.
Guido van Rossum09613542009-03-30 20:34:57 +0000203 Loaders that have a file-like storage back-end
Brett Cannon16248a42009-04-01 20:47:14 +0000204 that allows storing arbitrary data
Guido van Rossum09613542009-03-30 20:34:57 +0000205 can implement this abstract method to give direct access
Brett Cannon2a922ed2009-03-09 03:35:50 +0000206 to the data stored. :exc:`IOError` is to be raised if the *path* cannot
207 be found. The *path* is expected to be constructed using a module's
Brett Cannon16248a42009-04-01 20:47:14 +0000208 :attr:`__file__` attribute or an item from a package's :attr:`__path__`.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000209
210
211.. class:: InspectLoader
212
213 An abstract base class for a :term:`loader` which implements the optional
Guido van Rossum09613542009-03-30 20:34:57 +0000214 :pep:`302` protocol for loaders that inspect modules.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000215
Brett Cannona113ac52009-03-15 01:41:33 +0000216 .. method:: get_code(fullname)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000217
Brett Cannona113ac52009-03-15 01:41:33 +0000218 An abstract method to return the :class:`code` object for a module.
Georg Brandl375aec22011-01-15 17:03:02 +0000219 ``None`` is returned if the module does not have a code object
Brett Cannona113ac52009-03-15 01:41:33 +0000220 (e.g. built-in module). :exc:`ImportError` is raised if loader cannot
221 find the requested module.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000222
Brett Cannon9c751b72009-03-09 16:28:16 +0000223 .. method:: get_source(fullname)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000224
225 An abstract method to return the source of a module. It is returned as
Georg Brandl375aec22011-01-15 17:03:02 +0000226 a text string with universal newlines. Returns ``None`` if no
Brett Cannon2a922ed2009-03-09 03:35:50 +0000227 source is available (e.g. a built-in module). Raises :exc:`ImportError`
228 if the loader cannot find the module specified.
229
Brett Cannona113ac52009-03-15 01:41:33 +0000230 .. method:: is_package(fullname)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000231
Brett Cannona113ac52009-03-15 01:41:33 +0000232 An abstract method to return a true value if the module is a package, a
233 false value otherwise. :exc:`ImportError` is raised if the
234 :term:`loader` cannot find the module.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000235
236
Brett Cannon69194272009-07-20 04:23:48 +0000237.. class:: ExecutionLoader
238
239 An abstract base class which inherits from :class:`InspectLoader` that,
Brett Cannon23460292009-07-20 22:59:00 +0000240 when implemented, helps a module to be executed as a script. The ABC
Brett Cannon69194272009-07-20 04:23:48 +0000241 represents an optional :pep:`302` protocol.
242
243 .. method:: get_filename(fullname)
244
Brett Cannonf23e3742010-06-27 23:57:46 +0000245 An abstract method that is to return the value of :attr:`__file__` for
Brett Cannon69194272009-07-20 04:23:48 +0000246 the specified module. If no path is available, :exc:`ImportError` is
247 raised.
248
Brett Cannonf23e3742010-06-27 23:57:46 +0000249 If source code is available, then the method should return the path to
250 the source file, regardless of whether a bytecode was used to load the
251 module.
252
253
Brett Cannon938d44d2012-04-22 19:58:33 -0400254.. class:: FileLoader(fullname, path)
255
256 An abstract base class which inherits from :class:`ResourceLoader` and
257 :class:`ExecutionLoader`, providing concreate implementations of
258 :meth:`ResourceLoader.get_data` and :meth:`ExecutionLoader.get_filename`.
259
260 The *fullname* argument is a fully resolved name of the module the loader is
261 to handle. The *path* argument is the path to the file for the module.
262
263 .. versionadded:: 3.3
264
265 .. attribute:: name
266
267 The name of the module the loader can handle.
268
269 .. attribute:: path
270
271 Path to the file of the module.
272
Brett Cannonc0499522012-05-11 14:48:41 -0400273 .. method:: load_module(fullname=None)
274
275 Calls
276 ``super().load_module(fullname if fullname is not None else self.name)``.
277
Brett Cannon938d44d2012-04-22 19:58:33 -0400278 .. method:: get_filename(fullname)
279
Brett Cannonc0499522012-05-11 14:48:41 -0400280 Returns :attr:`path` when ``fullname`` equals :attr:`name` or ``None``.
Brett Cannon938d44d2012-04-22 19:58:33 -0400281
282 .. method:: get_data(path)
283
284 Returns the open, binary file for *path*.
285
286
Brett Cannonf23e3742010-06-27 23:57:46 +0000287.. class:: SourceLoader
288
289 An abstract base class for implementing source (and optionally bytecode)
290 file loading. The class inherits from both :class:`ResourceLoader` and
291 :class:`ExecutionLoader`, requiring the implementation of:
292
293 * :meth:`ResourceLoader.get_data`
294 * :meth:`ExecutionLoader.get_filename`
Brett Cannon6dfbff32010-07-21 09:48:31 +0000295 Should only return the path to the source file; sourceless
Brett Cannonf23e3742010-06-27 23:57:46 +0000296 loading is not supported.
297
298 The abstract methods defined by this class are to add optional bytecode
299 file support. Not implementing these optional methods causes the loader to
300 only work with source code. Implementing the methods allows the loader to
301 work with source *and* bytecode files; it does not allow for *sourceless*
302 loading where only bytecode is provided. Bytecode files are an
303 optimization to speed up loading by removing the parsing step of Python's
304 compiler, and so no bytecode-specific API is exposed.
305
Antoine Pitrou5136ac02012-01-13 18:52:16 +0100306 .. method:: path_stats(self, path)
307
308 Optional abstract method which returns a :class:`dict` containing
309 metadata about the specifed path. Supported dictionary keys are:
310
311 - ``'mtime'`` (mandatory): an integer or floating-point number
312 representing the modification time of the source code;
313 - ``'size'`` (optional): the size in bytes of the source code.
314
315 Any other keys in the dictionary are ignored, to allow for future
316 extensions.
317
318 .. versionadded:: 3.3
319
Brett Cannonf23e3742010-06-27 23:57:46 +0000320 .. method:: path_mtime(self, path)
321
322 Optional abstract method which returns the modification time for the
323 specified path.
324
Antoine Pitrou5136ac02012-01-13 18:52:16 +0100325 .. deprecated:: 3.3
326 This method is deprecated in favour of :meth:`path_stats`. You don't
327 have to implement it, but it is still available for compatibility
328 purposes.
329
Brett Cannonf23e3742010-06-27 23:57:46 +0000330 .. method:: set_data(self, path, data)
331
332 Optional abstract method which writes the specified bytes to a file
Brett Cannon61b14252010-07-03 21:48:25 +0000333 path. Any intermediate directories which do not exist are to be created
334 automatically.
335
336 When writing to the path fails because the path is read-only
337 (:attr:`errno.EACCES`), do not propagate the exception.
Brett Cannonf23e3742010-06-27 23:57:46 +0000338
339 .. method:: get_code(self, fullname)
340
341 Concrete implementation of :meth:`InspectLoader.get_code`.
342
343 .. method:: load_module(self, fullname)
344
345 Concrete implementation of :meth:`Loader.load_module`.
346
347 .. method:: get_source(self, fullname)
348
349 Concrete implementation of :meth:`InspectLoader.get_source`.
350
351 .. method:: is_package(self, fullname)
352
353 Concrete implementation of :meth:`InspectLoader.is_package`. A module
Brett Cannonea0b8232012-06-15 20:00:53 -0400354 is determined to be a package if its file path (as provided by
355 :meth:`ExecutionLoader.get_filename`) is a file named
356 ``__init__`` when the file extension is removed **and** the module name
357 itself does not end in ``__init__``.
Brett Cannonf23e3742010-06-27 23:57:46 +0000358
Brett Cannon69194272009-07-20 04:23:48 +0000359
Brett Cannon2a922ed2009-03-09 03:35:50 +0000360.. class:: PyLoader
361
Brett Cannon69194272009-07-20 04:23:48 +0000362 An abstract base class inheriting from
Brett Cannonf23e3742010-06-27 23:57:46 +0000363 :class:`ExecutionLoader` and
364 :class:`ResourceLoader` designed to ease the loading of
Brett Cannon2a922ed2009-03-09 03:35:50 +0000365 Python source modules (bytecode is not handled; see
Brett Cannonf23e3742010-06-27 23:57:46 +0000366 :class:`SourceLoader` for a source/bytecode ABC). A subclass
Brett Cannon2a922ed2009-03-09 03:35:50 +0000367 implementing this ABC will only need to worry about exposing how the source
368 code is stored; all other details for loading Python source code will be
369 handled by the concrete implementations of key methods.
370
Brett Cannonf23e3742010-06-27 23:57:46 +0000371 .. deprecated:: 3.2
372 This class has been deprecated in favor of :class:`SourceLoader` and is
373 slated for removal in Python 3.4. See below for how to create a
Georg Brandl6faee4e2010-09-21 14:48:28 +0000374 subclass that is compatible with Python 3.1 onwards.
Brett Cannonf23e3742010-06-27 23:57:46 +0000375
376 If compatibility with Python 3.1 is required, then use the following idiom
377 to implement a subclass that will work with Python 3.1 onwards (make sure
378 to implement :meth:`ExecutionLoader.get_filename`)::
379
380 try:
381 from importlib.abc import SourceLoader
382 except ImportError:
383 from importlib.abc import PyLoader as SourceLoader
384
385
386 class CustomLoader(SourceLoader):
387 def get_filename(self, fullname):
388 """Return the path to the source file."""
389 # Implement ...
390
391 def source_path(self, fullname):
392 """Implement source_path in terms of get_filename."""
393 try:
394 return self.get_filename(fullname)
395 except ImportError:
396 return None
397
398 def is_package(self, fullname):
399 """Implement is_package by looking for an __init__ file
400 name as returned by get_filename."""
401 filename = os.path.basename(self.get_filename(fullname))
402 return os.path.splitext(filename)[0] == '__init__'
403
404
Brett Cannon9c751b72009-03-09 16:28:16 +0000405 .. method:: source_path(fullname)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000406
407 An abstract method that returns the path to the source code for a
Georg Brandl375aec22011-01-15 17:03:02 +0000408 module. Should return ``None`` if there is no source code.
Brett Cannon3e761a92009-12-10 00:24:21 +0000409 Raises :exc:`ImportError` if the loader knows it cannot handle the
410 module.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000411
Brett Cannon69194272009-07-20 04:23:48 +0000412 .. method:: get_filename(fullname)
413
414 A concrete implementation of
415 :meth:`importlib.abc.ExecutionLoader.get_filename` that
416 relies on :meth:`source_path`. If :meth:`source_path` returns
Georg Brandl375aec22011-01-15 17:03:02 +0000417 ``None``, then :exc:`ImportError` is raised.
Brett Cannon69194272009-07-20 04:23:48 +0000418
Brett Cannon9c751b72009-03-09 16:28:16 +0000419 .. method:: load_module(fullname)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000420
421 A concrete implementation of :meth:`importlib.abc.Loader.load_module`
Brett Cannonad876c72009-03-09 07:53:09 +0000422 that loads Python source code. All needed information comes from the
423 abstract methods required by this ABC. The only pertinent assumption
424 made by this method is that when loading a package
425 :attr:`__path__` is set to ``[os.path.dirname(__file__)]``.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000426
Brett Cannon9c751b72009-03-09 16:28:16 +0000427 .. method:: get_code(fullname)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000428
429 A concrete implementation of
430 :meth:`importlib.abc.InspectLoader.get_code` that creates code objects
Guido van Rossum09613542009-03-30 20:34:57 +0000431 from Python source code, by requesting the source code (using
Benjamin Peterson0089d752009-11-13 00:52:43 +0000432 :meth:`source_path` and :meth:`get_data`) and compiling it with the
433 built-in :func:`compile` function.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000434
Brett Cannond43b30b2009-03-10 03:29:23 +0000435 .. method:: get_source(fullname)
436
437 A concrete implementation of
438 :meth:`importlib.abc.InspectLoader.get_source`. Uses
Brett Cannon69194272009-07-20 04:23:48 +0000439 :meth:`importlib.abc.ResourceLoader.get_data` and :meth:`source_path`
440 to get the source code. It tries to guess the source encoding using
Guido van Rossum09613542009-03-30 20:34:57 +0000441 :func:`tokenize.detect_encoding`.
Brett Cannond43b30b2009-03-10 03:29:23 +0000442
Brett Cannon2a922ed2009-03-09 03:35:50 +0000443
444.. class:: PyPycLoader
445
Brett Cannonf23e3742010-06-27 23:57:46 +0000446 An abstract base class inheriting from :class:`PyLoader`.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000447 This ABC is meant to help in creating loaders that support both Python
448 source and bytecode.
449
Brett Cannonf23e3742010-06-27 23:57:46 +0000450 .. deprecated:: 3.2
451 This class has been deprecated in favor of :class:`SourceLoader` and to
452 properly support :pep:`3147`. If compatibility is required with
453 Python 3.1, implement both :class:`SourceLoader` and :class:`PyLoader`;
454 instructions on how to do so are included in the documentation for
455 :class:`PyLoader`. Do note that this solution will not support
456 sourceless/bytecode-only loading; only source *and* bytecode loading.
457
Brett Cannon1e331562012-07-02 14:35:34 -0400458 .. versionchanged:: 3.3
459 Updated to parse (but not use) the new source size field in bytecode
460 files when reading and to write out the field properly when writing.
461
Brett Cannon9c751b72009-03-09 16:28:16 +0000462 .. method:: source_mtime(fullname)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000463
464 An abstract method which returns the modification time for the source
465 code of the specified module. The modification time should be an
Georg Brandl375aec22011-01-15 17:03:02 +0000466 integer. If there is no source code, return ``None``. If the
Brett Cannon2a922ed2009-03-09 03:35:50 +0000467 module cannot be found then :exc:`ImportError` is raised.
468
Brett Cannon9c751b72009-03-09 16:28:16 +0000469 .. method:: bytecode_path(fullname)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000470
471 An abstract method which returns the path to the bytecode for the
Georg Brandl375aec22011-01-15 17:03:02 +0000472 specified module, if it exists. It returns ``None``
Guido van Rossum09613542009-03-30 20:34:57 +0000473 if no bytecode exists (yet).
Brett Cannon3e761a92009-12-10 00:24:21 +0000474 Raises :exc:`ImportError` if the loader knows it cannot handle the
475 module.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000476
Brett Cannon69194272009-07-20 04:23:48 +0000477 .. method:: get_filename(fullname)
478
479 A concrete implementation of
Brett Cannonf23e3742010-06-27 23:57:46 +0000480 :meth:`ExecutionLoader.get_filename` that relies on
481 :meth:`PyLoader.source_path` and :meth:`bytecode_path`.
Brett Cannon69194272009-07-20 04:23:48 +0000482 If :meth:`source_path` returns a path, then that value is returned.
483 Else if :meth:`bytecode_path` returns a path, that path will be
484 returned. If a path is not available from both methods,
485 :exc:`ImportError` is raised.
486
Brett Cannon9c751b72009-03-09 16:28:16 +0000487 .. method:: write_bytecode(fullname, bytecode)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000488
489 An abstract method which has the loader write *bytecode* for future
Georg Brandl375aec22011-01-15 17:03:02 +0000490 use. If the bytecode is written, return ``True``. Return
491 ``False`` if the bytecode could not be written. This method
Brett Cannon2a922ed2009-03-09 03:35:50 +0000492 should not be called if :data:`sys.dont_write_bytecode` is true.
Guido van Rossum09613542009-03-30 20:34:57 +0000493 The *bytecode* argument should be a bytes string or bytes array.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000494
495
Brett Cannon78246b62009-01-25 04:56:30 +0000496:mod:`importlib.machinery` -- Importers and path hooks
497------------------------------------------------------
498
499.. module:: importlib.machinery
500 :synopsis: Importers and path hooks
501
502This module contains the various objects that help :keyword:`import`
503find and load modules.
504
Brett Cannoncb66eb02012-05-11 12:58:42 -0400505.. attribute:: SOURCE_SUFFIXES
506
507 A list of strings representing the recognized file suffixes for source
508 modules.
509
510 .. versionadded:: 3.3
511
512.. attribute:: DEBUG_BYTECODE_SUFFIXES
513
514 A list of strings representing the file suffixes for non-optimized bytecode
515 modules.
516
517 .. versionadded:: 3.3
518
519.. attribute:: OPTIMIZED_BYTECODE_SUFFIXES
520
521 A list of strings representing the file suffixes for optimized bytecode
522 modules.
523
524 .. versionadded:: 3.3
525
526.. attribute:: BYTECODE_SUFFIXES
527
528 A list of strings representing the recognized file suffixes for bytecode
529 modules. Set to either :attr:`DEBUG_BYTECODE_SUFFIXES` or
530 :attr:`OPTIMIZED_BYTECODE_SUFFIXES` based on whether ``__debug__`` is true.
531
532 .. versionadded:: 3.3
533
534.. attribute:: EXTENSION_SUFFIXES
535
Nick Coghlan76e07702012-07-18 23:14:57 +1000536 A list of strings representing the recognized file suffixes for
Brett Cannoncb66eb02012-05-11 12:58:42 -0400537 extension modules.
538
539 .. versionadded:: 3.3
540
541
Nick Coghlanc5afd422012-07-18 23:59:08 +1000542.. function:: all_suffixes()
Nick Coghlan76e07702012-07-18 23:14:57 +1000543
544 Returns a combined list of strings representing all file suffixes for
Nick Coghlanc5afd422012-07-18 23:59:08 +1000545 modules recognized by the standard import machinery. This is a
Nick Coghlan76e07702012-07-18 23:14:57 +1000546 helper for code which simply needs to know if a filesystem path
Nick Coghlanc5afd422012-07-18 23:59:08 +1000547 potentially refers to a module without needing any details on the kind
548 of module (for example, :func:`inspect.getmodulename`)
Nick Coghlan76e07702012-07-18 23:14:57 +1000549
550 .. versionadded:: 3.3
551
552
Brett Cannon78246b62009-01-25 04:56:30 +0000553.. class:: BuiltinImporter
554
Brett Cannon2a922ed2009-03-09 03:35:50 +0000555 An :term:`importer` for built-in modules. All known built-in modules are
556 listed in :data:`sys.builtin_module_names`. This class implements the
Brett Cannona113ac52009-03-15 01:41:33 +0000557 :class:`importlib.abc.Finder` and :class:`importlib.abc.InspectLoader`
558 ABCs.
Brett Cannon78246b62009-01-25 04:56:30 +0000559
560 Only class methods are defined by this class to alleviate the need for
561 instantiation.
562
Brett Cannon78246b62009-01-25 04:56:30 +0000563
564.. class:: FrozenImporter
565
Brett Cannon2a922ed2009-03-09 03:35:50 +0000566 An :term:`importer` for frozen modules. This class implements the
Brett Cannon8d110132009-03-15 02:20:16 +0000567 :class:`importlib.abc.Finder` and :class:`importlib.abc.InspectLoader`
568 ABCs.
Brett Cannon78246b62009-01-25 04:56:30 +0000569
570 Only class methods are defined by this class to alleviate the need for
571 instantiation.
572
Brett Cannondebb98d2009-02-16 04:18:01 +0000573
574.. class:: PathFinder
575
Brett Cannon2a922ed2009-03-09 03:35:50 +0000576 :term:`Finder` for :data:`sys.path`. This class implements the
577 :class:`importlib.abc.Finder` ABC.
Brett Cannondebb98d2009-02-16 04:18:01 +0000578
579 This class does not perfectly mirror the semantics of :keyword:`import` in
580 terms of :data:`sys.path`. No implicit path hooks are assumed for
Brett Cannon83ac0132012-03-02 12:10:48 -0500581 simplification of the class and its semantics. This implies that when
582 ``None`` is found in :data:`sys.path_importer_cache` that it is simply
583 ignored instead of implying a default finder.
Brett Cannondebb98d2009-02-16 04:18:01 +0000584
Brett Cannon1b184d52009-11-07 08:22:58 +0000585 Only class methods are defined by this class to alleviate the need for
Brett Cannondebb98d2009-02-16 04:18:01 +0000586 instantiation.
587
588 .. classmethod:: find_module(fullname, path=None)
589
590 Class method that attempts to find a :term:`loader` for the module
Brett Cannon2a922ed2009-03-09 03:35:50 +0000591 specified by *fullname* on :data:`sys.path` or, if defined, on
Brett Cannondebb98d2009-02-16 04:18:01 +0000592 *path*. For each path entry that is searched,
Brett Cannon75321e82012-03-02 11:58:25 -0500593 :data:`sys.path_importer_cache` is checked. If a non-false object is
Brett Cannon2a922ed2009-03-09 03:35:50 +0000594 found then it is used as the :term:`finder` to look for the module
595 being searched for. If no entry is found in
Brett Cannondebb98d2009-02-16 04:18:01 +0000596 :data:`sys.path_importer_cache`, then :data:`sys.path_hooks` is
597 searched for a finder for the path entry and, if found, is stored in
598 :data:`sys.path_importer_cache` along with being queried about the
Georg Brandl375aec22011-01-15 17:03:02 +0000599 module. If no finder is ever found then ``None`` is returned.
Brett Cannond2e7b332009-02-17 02:45:03 +0000600
601
Brett Cannon938d44d2012-04-22 19:58:33 -0400602.. class:: FileFinder(path, \*loader_details)
603
604 A concrete implementation of :class:`importlib.abc.Finder` which caches
605 results from the file system.
606
607 The *path* argument is the directory for which the finder is in charge of
608 searching.
609
610 The *loader_details* argument is a variable number of 3-item tuples each
611 containing a loader, file suffixes the loader recognizes, and a boolean
612 representing whether the loader handles packages.
613
614 The finder will cache the directory contents as necessary, making stat calls
615 for each module search to verify the cache is not outdated. Because cache
616 staleness relies upon the granularity of the operating system's state
617 information of the file system, there is a potential race condition of
618 searching for a module, creating a new file, and then searching for the
619 module the new file represents. If the operations happen fast enough to fit
620 within the granularity of stat calls, then the module search will fail. To
621 prevent this from happening, when you create a module dynamically, make sure
622 to call :func:`importlib.invalidate_caches`.
623
624 .. versionadded:: 3.3
625
626 .. attribute:: path
627
628 The path the finder will search in.
629
630 .. method:: find_module(fullname)
631
632 Attempt to find the loader to handle *fullname* within :attr:`path`.
633
634 .. method:: invalidate_caches()
635
636 Clear out the internal cache.
637
638 .. classmethod:: path_hook(\*loader_details)
639
640 A class method which returns a closure for use on :attr:`sys.path_hooks`.
641 An instance of :class:`FileFinder` is returned by the closure using the
642 path argument given to the closure directly and *loader_details*
643 indirectly.
644
645 If the argument to the closure is not an existing directory,
646 :exc:`ImportError` is raised.
647
648
649.. class:: SourceFileLoader(fullname, path)
650
651 A concrete implementation of :class:`importlib.abc.SourceLoader` by
652 subclassing :class:`importlib.abc.FileLoader` and providing some concrete
653 implementations of other methods.
654
655 .. versionadded:: 3.3
656
657 .. attribute:: name
658
659 The name of the module that this loader will handle.
660
661 .. attribute:: path
662
663 The path to the source file.
664
665 .. method:: is_package(fullname)
666
667 Return true if :attr:`path` appears to be for a package.
668
669 .. method:: path_stats(path)
670
671 Concrete implementation of :meth:`importlib.abc.SourceLoader.path_stats`.
672
673 .. method:: set_data(path, data)
674
675 Concrete implementation of :meth:`importlib.abc.SourceLoader.set_data`.
676
Brett Cannon938d44d2012-04-22 19:58:33 -0400677
Marc-Andre Lemburg4fe29c92012-04-25 02:31:37 +0200678.. class:: SourcelessFileLoader(fullname, path)
Brett Cannon938d44d2012-04-22 19:58:33 -0400679
680 A concrete implementation of :class:`importlib.abc.FileLoader` which can
681 import bytecode files (i.e. no source code files exist).
682
Marc-Andre Lemburg4fe29c92012-04-25 02:31:37 +0200683 Please note that direct use of bytecode files (and thus not source code
684 files) inhibits your modules from being usable by all Python
685 implementations or new versions of Python which change the bytecode
686 format.
Brett Cannon938d44d2012-04-22 19:58:33 -0400687
688 .. versionadded:: 3.3
689
690 .. attribute:: name
691
692 The name of the module the loader will handle.
693
694 .. attribute:: path
695
696 The path to the bytecode file.
697
698 .. method:: is_package(fullname)
699
700 Determines if the module is a package based on :attr:`path`.
701
702 .. method:: get_code(fullname)
703
704 Returns the code object for :attr:`name` created from :attr:`path`.
705
706 .. method:: get_source(fullname)
707
708 Returns ``None`` as bytecode files have no source when this loader is
709 used.
710
Brett Cannon938d44d2012-04-22 19:58:33 -0400711
712.. class:: ExtensionFileLoader(fullname, path)
713
714 A concrete implementation of :class:`importlib.abc.InspectLoader` for
715 extension modules.
716
717 The *fullname* argument specifies the name of the module the loader is to
718 support. The *path* argument is the path to the extension module's file.
719
720 .. versionadded:: 3.3
721
722 .. attribute:: name
723
724 Name of the module the loader supports.
725
726 .. attribute:: path
727
728 Path to the extension module.
729
Brett Cannonc0499522012-05-11 14:48:41 -0400730 .. method:: load_module(fullname=None)
Brett Cannon938d44d2012-04-22 19:58:33 -0400731
Brett Cannonc0499522012-05-11 14:48:41 -0400732 Loads the extension module if and only if *fullname* is the same as
733 :attr:`name` or is ``None``.
Brett Cannon938d44d2012-04-22 19:58:33 -0400734
735 .. method:: is_package(fullname)
736
737 Returns ``False`` as extension modules can never be packages.
738
739 .. method:: get_code(fullname)
740
741 Returns ``None`` as extension modules lack a code object.
742
743 .. method:: get_source(fullname)
744
745 Returns ``None`` as extension modules do not have source code.
746
747
Brett Cannond2e7b332009-02-17 02:45:03 +0000748:mod:`importlib.util` -- Utility code for importers
749---------------------------------------------------
750
751.. module:: importlib.util
Brett Cannon75321e82012-03-02 11:58:25 -0500752 :synopsis: Utility code for importers
Brett Cannond2e7b332009-02-17 02:45:03 +0000753
754This module contains the various objects that help in the construction of
755an :term:`importer`.
756
Brett Cannond200bf52012-05-13 13:45:09 -0400757.. function:: resolve_name(name, package)
758
759 Resolve a relative module name to an absolute one.
760
761 If **name** has no leading dots, then **name** is simply returned. This
762 allows for usage such as
763 ``importlib.util.resolve_name('sys', __package__)`` without doing a
764 check to see if the **package** argument is needed.
765
766 :exc:`ValueError` is raised if **name** is a relative module name but
767 package is a false value (e.g. ``None`` or the empty string).
768 :exc:`ValueError` is also raised a relative name would escape its containing
769 package (e.g. requesting ``..bacon`` from within the ``spam`` package).
770
771 .. versionadded:: 3.3
772
Georg Brandl8a1caa22010-07-29 16:01:11 +0000773.. decorator:: module_for_loader
Brett Cannond2e7b332009-02-17 02:45:03 +0000774
Guido van Rossum09613542009-03-30 20:34:57 +0000775 A :term:`decorator` for a :term:`loader` method,
776 to handle selecting the proper
Brett Cannond2e7b332009-02-17 02:45:03 +0000777 module object to load with. The decorated method is expected to have a call
Brett Cannon2a922ed2009-03-09 03:35:50 +0000778 signature taking two positional arguments
779 (e.g. ``load_module(self, module)``) for which the second argument
Guido van Rossum09613542009-03-30 20:34:57 +0000780 will be the module **object** to be used by the loader.
Brett Cannonefad00d2012-04-27 17:27:14 -0400781 Note that the decorator will not work on static methods because of the
782 assumption of two arguments.
Brett Cannond2e7b332009-02-17 02:45:03 +0000783
Guido van Rossum09613542009-03-30 20:34:57 +0000784 The decorated method will take in the **name** of the module to be loaded
785 as expected for a :term:`loader`. If the module is not found in
Brett Cannon57b46f52009-03-02 14:38:26 +0000786 :data:`sys.modules` then a new one is constructed with its
Brett Cannonefad00d2012-04-27 17:27:14 -0400787 :attr:`__name__` attribute set to **name**, :attr:`__loader__` set to
788 **self**, and :attr:`__package__` set if
789 :meth:`importlib.abc.InspectLoader.is_package` is defined for **self** and
790 does not raise :exc:`ImportError` for **name**. If a new module is not
791 needed then the module found in :data:`sys.modules` will be passed into the
792 method.
793
794 If an exception is raised by the decorated method and a module was added to
Brett Cannond2e7b332009-02-17 02:45:03 +0000795 :data:`sys.modules` it will be removed to prevent a partially initialized
Brett Cannon57b46f52009-03-02 14:38:26 +0000796 module from being in left in :data:`sys.modules`. If the module was already
797 in :data:`sys.modules` then it is left alone.
Brett Cannond2e7b332009-02-17 02:45:03 +0000798
Guido van Rossum09613542009-03-30 20:34:57 +0000799 Use of this decorator handles all the details of which module object a
Brett Cannonefad00d2012-04-27 17:27:14 -0400800 loader should initialize as specified by :pep:`302` as best as possible.
801
802 .. versionchanged:: 3.3
Georg Brandl61063cc2012-06-24 22:48:30 +0200803 :attr:`__loader__` and :attr:`__package__` are automatically set
804 (when possible).
Brett Cannon57b46f52009-03-02 14:38:26 +0000805
Georg Brandl8a1caa22010-07-29 16:01:11 +0000806.. decorator:: set_loader
Brett Cannon2cf03a82009-03-10 05:17:37 +0000807
Guido van Rossum09613542009-03-30 20:34:57 +0000808 A :term:`decorator` for a :term:`loader` method,
809 to set the :attr:`__loader__`
Brett Cannon2cf03a82009-03-10 05:17:37 +0000810 attribute on loaded modules. If the attribute is already set the decorator
811 does nothing. It is assumed that the first positional argument to the
Brett Cannon75321e82012-03-02 11:58:25 -0500812 wrapped method (i.e. ``self``) is what :attr:`__loader__` should be set to.
Brett Cannon2cf03a82009-03-10 05:17:37 +0000813
Brett Cannonefad00d2012-04-27 17:27:14 -0400814 .. note::
815
816 It is recommended that :func:`module_for_loader` be used over this
817 decorator as it subsumes this functionality.
818
819
Georg Brandl8a1caa22010-07-29 16:01:11 +0000820.. decorator:: set_package
Brett Cannon57b46f52009-03-02 14:38:26 +0000821
822 A :term:`decorator` for a :term:`loader` to set the :attr:`__package__`
823 attribute on the module returned by the loader. If :attr:`__package__` is
Georg Brandl375aec22011-01-15 17:03:02 +0000824 set and has a value other than ``None`` it will not be changed.
Brett Cannon57b46f52009-03-02 14:38:26 +0000825 Note that the module returned by the loader is what has the attribute
826 set on and not the module found in :data:`sys.modules`.
Guido van Rossum09613542009-03-30 20:34:57 +0000827
Brett Cannon16248a42009-04-01 20:47:14 +0000828 Reliance on this decorator is discouraged when it is possible to set
Brett Cannon75321e82012-03-02 11:58:25 -0500829 :attr:`__package__` before importing. By
830 setting it beforehand the code for the module is executed with the
831 attribute set and thus can be used by global level code during
Brett Cannon16248a42009-04-01 20:47:14 +0000832 initialization.
833
Brett Cannonefad00d2012-04-27 17:27:14 -0400834 .. note::
835
836 It is recommended that :func:`module_for_loader` be used over this
837 decorator as it subsumes this functionality.