blob: 05b18d3a4b1e4ce5bcd8ebde9f8e0f697f90553d [file] [log] [blame]
Brett Cannonafccd632009-01-20 02:21:27 +00001:mod:`importlib` -- An implementation of :keyword:`import`
2==========================================================
3
4.. module:: importlib
5 :synopsis: An implementation of the import machinery.
6
7.. moduleauthor:: Brett Cannon <brett@python.org>
8.. sectionauthor:: Brett Cannon <brett@python.org>
9
10.. versionadded:: 3.1
11
12
13Introduction
14------------
15
16The purpose of the :mod:`importlib` package is two-fold. One is to provide an
17implementation of the :keyword:`import` statement (and thus, by extension, the
18:func:`__import__` function) in Python source code. This provides an
Tarek Ziadé434caaa2009-05-14 12:48:09 +000019implementation of :keyword:`import` which is portable to any Python
Brett Cannonafccd632009-01-20 02:21:27 +000020interpreter. This also provides a reference implementation which is easier to
Brett Cannonf23e3742010-06-27 23:57:46 +000021comprehend than one implemented in a programming language other than Python.
Brett Cannonafccd632009-01-20 02:21:27 +000022
Brett Cannonf23e3742010-06-27 23:57:46 +000023Two, the components to implement :keyword:`import` are exposed in this
Brett Cannonafccd632009-01-20 02:21:27 +000024package, making it easier for users to create their own custom objects (known
Brett Cannondebb98d2009-02-16 04:18:01 +000025generically as an :term:`importer`) to participate in the import process.
Brett Cannonf23e3742010-06-27 23:57:46 +000026Details on custom importers can be found in :pep:`302`.
Brett Cannonafccd632009-01-20 02:21:27 +000027
28.. seealso::
29
30 :ref:`import`
31 The language reference for the :keyword:`import` statement.
32
33 `Packages specification <http://www.python.org/doc/essays/packages.html>`__
34 Original specification of packages. Some semantics have changed since
Georg Brandl375aec22011-01-15 17:03:02 +000035 the writing of this document (e.g. redirecting based on ``None``
Brett Cannonafccd632009-01-20 02:21:27 +000036 in :data:`sys.modules`).
37
38 The :func:`.__import__` function
Brett Cannon0e13c942010-06-29 18:26:11 +000039 The :keyword:`import` statement is syntactic sugar for this function.
Brett Cannonafccd632009-01-20 02:21:27 +000040
41 :pep:`235`
42 Import on Case-Insensitive Platforms
43
44 :pep:`263`
45 Defining Python Source Code Encodings
46
47 :pep:`302`
Brett Cannonf23e3742010-06-27 23:57:46 +000048 New Import Hooks
Brett Cannonafccd632009-01-20 02:21:27 +000049
50 :pep:`328`
51 Imports: Multi-Line and Absolute/Relative
52
53 :pep:`366`
54 Main module explicit relative imports
55
Brett Cannon8917d5e2010-01-13 19:21:00 +000056 :pep:`3120`
Brett Cannonafccd632009-01-20 02:21:27 +000057 Using UTF-8 as the Default Source Encoding
58
Brett Cannon30b7a902010-06-27 21:49:22 +000059 :pep:`3147`
60 PYC Repository Directories
61
Brett Cannonafccd632009-01-20 02:21:27 +000062
63Functions
64---------
65
Brett Cannon78246b62009-01-25 04:56:30 +000066.. function:: __import__(name, globals={}, locals={}, fromlist=list(), level=0)
Brett Cannonafccd632009-01-20 02:21:27 +000067
Brett Cannonf23e3742010-06-27 23:57:46 +000068 An implementation of the built-in :func:`__import__` function.
Brett Cannonafccd632009-01-20 02:21:27 +000069
70.. function:: import_module(name, package=None)
71
Brett Cannon33418c82009-01-22 18:37:20 +000072 Import a module. The *name* argument specifies what module to
Brett Cannonafccd632009-01-20 02:21:27 +000073 import in absolute or relative terms
74 (e.g. either ``pkg.mod`` or ``..mod``). If the name is
Guido van Rossum09613542009-03-30 20:34:57 +000075 specified in relative terms, then the *package* argument must be set to
76 the name of the package which is to act as the anchor for resolving the
Brett Cannonafccd632009-01-20 02:21:27 +000077 package name (e.g. ``import_module('..mod', 'pkg.subpkg')`` will import
Brett Cannon2c318a12009-02-07 01:15:27 +000078 ``pkg.mod``).
Brett Cannon78246b62009-01-25 04:56:30 +000079
Brett Cannon2c318a12009-02-07 01:15:27 +000080 The :func:`import_module` function acts as a simplifying wrapper around
Brett Cannon9f4cb1c2009-04-01 23:26:47 +000081 :func:`importlib.__import__`. This means all semantics of the function are
82 derived from :func:`importlib.__import__`, including requiring the package
83 from which an import is occurring to have been previously imported
84 (i.e., *package* must already be imported). The most important difference
85 is that :func:`import_module` returns the most nested package or module
86 that was imported (e.g. ``pkg.mod``), while :func:`__import__` returns the
Guido van Rossum09613542009-03-30 20:34:57 +000087 top-level package or module (e.g. ``pkg``).
88
Brett Cannonee78a2b2012-05-12 17:43:17 -040089.. function:: find_loader(name, path=None)
90
91 Find the loader for a module, optionally within the specified *path*. If the
92 module is in :attr:`sys.modules`, then ``sys.modules[name].__loader__`` is
93 returned (unless the loader would be ``None``, in which case
94 :exc:`ValueError` is raised). Otherwise a search using :attr:`sys.meta_path`
95 is done. ``None`` is returned if no loader is found.
96
97 A dotted name does not have its parent's implicitly imported. If that is
98 desired (although not nessarily required to find the loader, it will most
99 likely be needed if the loader actually is used to load the module), then
100 you will have to import the packages containing the module prior to calling
101 this function.
102
Antoine Pitrouc541f8e2012-02-20 01:48:16 +0100103.. function:: invalidate_caches()
104
Brett Cannonb46a1792012-02-27 18:15:42 -0500105 Invalidate the internal caches of the finders stored at
106 :data:`sys.path_importer_cache`. If a finder implements
107 :meth:`abc.Finder.invalidate_caches()` then it will be called to perform the
108 invalidation. This function may be needed if some modules are installed
109 while your program is running and you expect the program to notice the
110 changes.
Antoine Pitrouc541f8e2012-02-20 01:48:16 +0100111
112 .. versionadded:: 3.3
113
Brett Cannon78246b62009-01-25 04:56:30 +0000114
Brett Cannon2a922ed2009-03-09 03:35:50 +0000115:mod:`importlib.abc` -- Abstract base classes related to import
116---------------------------------------------------------------
117
118.. module:: importlib.abc
119 :synopsis: Abstract base classes related to import
120
121The :mod:`importlib.abc` module contains all of the core abstract base classes
122used by :keyword:`import`. Some subclasses of the core abstract base classes
123are also provided to help in implementing the core ABCs.
124
125
126.. class:: Finder
127
128 An abstract base class representing a :term:`finder`.
Guido van Rossum09613542009-03-30 20:34:57 +0000129 See :pep:`302` for the exact definition for a finder.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000130
Barry Warsawd7d21942012-07-29 16:36:17 -0400131 .. method:: find_loader(self, fullname):
132
133 An abstract method for finding a :term:`loader` for the specified
134 module. Returns a 2-tuple of ``(loader, portion)`` where portion is a
135 sequence of file system locations contributing to part of a namespace
136 package. The sequence may be empty. When present, `find_loader()` is
137 preferred over `find_module()`.
138
139 .. versionadded: 3.3
140
Brett Cannon9c751b72009-03-09 16:28:16 +0000141 .. method:: find_module(fullname, path=None)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000142
143 An abstract method for finding a :term:`loader` for the specified
144 module. If the :term:`finder` is found on :data:`sys.meta_path` and the
Guido van Rossum09613542009-03-30 20:34:57 +0000145 module to be searched for is a subpackage or module then *path* will
146 be the value of :attr:`__path__` from the parent package. If a loader
Georg Brandl375aec22011-01-15 17:03:02 +0000147 cannot be found, ``None`` is returned.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000148
Brett Cannonb46a1792012-02-27 18:15:42 -0500149 .. method:: invalidate_caches()
150
151 An optional method which, when called, should invalidate any internal
152 cache used by the finder. Used by :func:`invalidate_caches()` when
153 invalidating the caches of all cached finders.
154
Brett Cannon2a922ed2009-03-09 03:35:50 +0000155
156.. class:: Loader
157
158 An abstract base class for a :term:`loader`.
Guido van Rossum09613542009-03-30 20:34:57 +0000159 See :pep:`302` for the exact definition for a loader.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000160
Brett Cannon9c751b72009-03-09 16:28:16 +0000161 .. method:: load_module(fullname)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000162
163 An abstract method for loading a module. If the module cannot be
164 loaded, :exc:`ImportError` is raised, otherwise the loaded module is
165 returned.
166
Guido van Rossum09613542009-03-30 20:34:57 +0000167 If the requested module already exists in :data:`sys.modules`, that
Brett Cannon2a922ed2009-03-09 03:35:50 +0000168 module should be used and reloaded.
Guido van Rossum09613542009-03-30 20:34:57 +0000169 Otherwise the loader should create a new module and insert it into
170 :data:`sys.modules` before any loading begins, to prevent recursion
171 from the import. If the loader inserted a module and the load fails, it
Brett Cannon2a922ed2009-03-09 03:35:50 +0000172 must be removed by the loader from :data:`sys.modules`; modules already
173 in :data:`sys.modules` before the loader began execution should be left
174 alone. The :func:`importlib.util.module_for_loader` decorator handles
175 all of these details.
176
Guido van Rossum09613542009-03-30 20:34:57 +0000177 The loader should set several attributes on the module.
178 (Note that some of these attributes can change when a module is
179 reloaded.)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000180
181 - :attr:`__name__`
182 The name of the module.
183
184 - :attr:`__file__`
185 The path to where the module data is stored (not set for built-in
186 modules).
187
188 - :attr:`__path__`
Guido van Rossum09613542009-03-30 20:34:57 +0000189 A list of strings specifying the search path within a
Brett Cannon2a922ed2009-03-09 03:35:50 +0000190 package. This attribute is not set on modules.
191
192 - :attr:`__package__`
193 The parent package for the module/package. If the module is
194 top-level then it has a value of the empty string. The
195 :func:`importlib.util.set_package` decorator can handle the details
196 for :attr:`__package__`.
197
198 - :attr:`__loader__`
Guido van Rossum09613542009-03-30 20:34:57 +0000199 The loader used to load the module.
200 (This is not set by the built-in import machinery,
201 but it should be set whenever a :term:`loader` is used.)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000202
Barry Warsawd7d21942012-07-29 16:36:17 -0400203 .. method:: module_repr(module)
204
205 An abstract method which when implemented calculates and returns the
206 given module's repr, as a string.
207
208 .. versionadded: 3.3
209
Brett Cannon2a922ed2009-03-09 03:35:50 +0000210
211.. class:: ResourceLoader
212
213 An abstract base class for a :term:`loader` which implements the optional
214 :pep:`302` protocol for loading arbitrary resources from the storage
215 back-end.
216
Brett Cannon9c751b72009-03-09 16:28:16 +0000217 .. method:: get_data(path)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000218
219 An abstract method to return the bytes for the data located at *path*.
Guido van Rossum09613542009-03-30 20:34:57 +0000220 Loaders that have a file-like storage back-end
Brett Cannon16248a42009-04-01 20:47:14 +0000221 that allows storing arbitrary data
Guido van Rossum09613542009-03-30 20:34:57 +0000222 can implement this abstract method to give direct access
Brett Cannon2a922ed2009-03-09 03:35:50 +0000223 to the data stored. :exc:`IOError` is to be raised if the *path* cannot
224 be found. The *path* is expected to be constructed using a module's
Brett Cannon16248a42009-04-01 20:47:14 +0000225 :attr:`__file__` attribute or an item from a package's :attr:`__path__`.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000226
227
228.. class:: InspectLoader
229
230 An abstract base class for a :term:`loader` which implements the optional
Guido van Rossum09613542009-03-30 20:34:57 +0000231 :pep:`302` protocol for loaders that inspect modules.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000232
Brett Cannona113ac52009-03-15 01:41:33 +0000233 .. method:: get_code(fullname)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000234
Brett Cannona113ac52009-03-15 01:41:33 +0000235 An abstract method to return the :class:`code` object for a module.
Georg Brandl375aec22011-01-15 17:03:02 +0000236 ``None`` is returned if the module does not have a code object
Brett Cannona113ac52009-03-15 01:41:33 +0000237 (e.g. built-in module). :exc:`ImportError` is raised if loader cannot
238 find the requested module.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000239
Brett Cannon9c751b72009-03-09 16:28:16 +0000240 .. method:: get_source(fullname)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000241
242 An abstract method to return the source of a module. It is returned as
Georg Brandl375aec22011-01-15 17:03:02 +0000243 a text string with universal newlines. Returns ``None`` if no
Brett Cannon2a922ed2009-03-09 03:35:50 +0000244 source is available (e.g. a built-in module). Raises :exc:`ImportError`
245 if the loader cannot find the module specified.
246
Brett Cannona113ac52009-03-15 01:41:33 +0000247 .. method:: is_package(fullname)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000248
Brett Cannona113ac52009-03-15 01:41:33 +0000249 An abstract method to return a true value if the module is a package, a
250 false value otherwise. :exc:`ImportError` is raised if the
251 :term:`loader` cannot find the module.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000252
253
Brett Cannon69194272009-07-20 04:23:48 +0000254.. class:: ExecutionLoader
255
256 An abstract base class which inherits from :class:`InspectLoader` that,
Brett Cannon23460292009-07-20 22:59:00 +0000257 when implemented, helps a module to be executed as a script. The ABC
Brett Cannon69194272009-07-20 04:23:48 +0000258 represents an optional :pep:`302` protocol.
259
260 .. method:: get_filename(fullname)
261
Brett Cannonf23e3742010-06-27 23:57:46 +0000262 An abstract method that is to return the value of :attr:`__file__` for
Brett Cannon69194272009-07-20 04:23:48 +0000263 the specified module. If no path is available, :exc:`ImportError` is
264 raised.
265
Brett Cannonf23e3742010-06-27 23:57:46 +0000266 If source code is available, then the method should return the path to
267 the source file, regardless of whether a bytecode was used to load the
268 module.
269
270
Brett Cannon938d44d2012-04-22 19:58:33 -0400271.. class:: FileLoader(fullname, path)
272
273 An abstract base class which inherits from :class:`ResourceLoader` and
274 :class:`ExecutionLoader`, providing concreate implementations of
275 :meth:`ResourceLoader.get_data` and :meth:`ExecutionLoader.get_filename`.
276
277 The *fullname* argument is a fully resolved name of the module the loader is
278 to handle. The *path* argument is the path to the file for the module.
279
280 .. versionadded:: 3.3
281
282 .. attribute:: name
283
284 The name of the module the loader can handle.
285
286 .. attribute:: path
287
288 Path to the file of the module.
289
Barry Warsawd7d21942012-07-29 16:36:17 -0400290 .. method:: load_module(fullname)
Brett Cannonc0499522012-05-11 14:48:41 -0400291
Barry Warsawd7d21942012-07-29 16:36:17 -0400292 Calls super's ``load_module()``.
Brett Cannonc0499522012-05-11 14:48:41 -0400293
Brett Cannon938d44d2012-04-22 19:58:33 -0400294 .. method:: get_filename(fullname)
295
Barry Warsawd7d21942012-07-29 16:36:17 -0400296 Returns :attr:`path`.
Brett Cannon938d44d2012-04-22 19:58:33 -0400297
298 .. method:: get_data(path)
299
300 Returns the open, binary file for *path*.
301
302
Brett Cannonf23e3742010-06-27 23:57:46 +0000303.. class:: SourceLoader
304
305 An abstract base class for implementing source (and optionally bytecode)
306 file loading. The class inherits from both :class:`ResourceLoader` and
307 :class:`ExecutionLoader`, requiring the implementation of:
308
309 * :meth:`ResourceLoader.get_data`
310 * :meth:`ExecutionLoader.get_filename`
Brett Cannon6dfbff32010-07-21 09:48:31 +0000311 Should only return the path to the source file; sourceless
Brett Cannonf23e3742010-06-27 23:57:46 +0000312 loading is not supported.
313
314 The abstract methods defined by this class are to add optional bytecode
315 file support. Not implementing these optional methods causes the loader to
316 only work with source code. Implementing the methods allows the loader to
317 work with source *and* bytecode files; it does not allow for *sourceless*
318 loading where only bytecode is provided. Bytecode files are an
319 optimization to speed up loading by removing the parsing step of Python's
320 compiler, and so no bytecode-specific API is exposed.
321
Antoine Pitrou5136ac02012-01-13 18:52:16 +0100322 .. method:: path_stats(self, path)
323
324 Optional abstract method which returns a :class:`dict` containing
325 metadata about the specifed path. Supported dictionary keys are:
326
327 - ``'mtime'`` (mandatory): an integer or floating-point number
328 representing the modification time of the source code;
329 - ``'size'`` (optional): the size in bytes of the source code.
330
331 Any other keys in the dictionary are ignored, to allow for future
332 extensions.
333
334 .. versionadded:: 3.3
335
Brett Cannonf23e3742010-06-27 23:57:46 +0000336 .. method:: path_mtime(self, path)
337
338 Optional abstract method which returns the modification time for the
339 specified path.
340
Antoine Pitrou5136ac02012-01-13 18:52:16 +0100341 .. deprecated:: 3.3
342 This method is deprecated in favour of :meth:`path_stats`. You don't
343 have to implement it, but it is still available for compatibility
344 purposes.
345
Brett Cannonf23e3742010-06-27 23:57:46 +0000346 .. method:: set_data(self, path, data)
347
348 Optional abstract method which writes the specified bytes to a file
Brett Cannon61b14252010-07-03 21:48:25 +0000349 path. Any intermediate directories which do not exist are to be created
350 automatically.
351
352 When writing to the path fails because the path is read-only
353 (:attr:`errno.EACCES`), do not propagate the exception.
Brett Cannonf23e3742010-06-27 23:57:46 +0000354
355 .. method:: get_code(self, fullname)
356
357 Concrete implementation of :meth:`InspectLoader.get_code`.
358
359 .. method:: load_module(self, fullname)
360
361 Concrete implementation of :meth:`Loader.load_module`.
362
363 .. method:: get_source(self, fullname)
364
365 Concrete implementation of :meth:`InspectLoader.get_source`.
366
367 .. method:: is_package(self, fullname)
368
369 Concrete implementation of :meth:`InspectLoader.is_package`. A module
Brett Cannonea0b8232012-06-15 20:00:53 -0400370 is determined to be a package if its file path (as provided by
371 :meth:`ExecutionLoader.get_filename`) is a file named
372 ``__init__`` when the file extension is removed **and** the module name
373 itself does not end in ``__init__``.
Brett Cannonf23e3742010-06-27 23:57:46 +0000374
Brett Cannon69194272009-07-20 04:23:48 +0000375
Brett Cannon2a922ed2009-03-09 03:35:50 +0000376.. class:: PyLoader
377
Brett Cannon69194272009-07-20 04:23:48 +0000378 An abstract base class inheriting from
Brett Cannonf23e3742010-06-27 23:57:46 +0000379 :class:`ExecutionLoader` and
380 :class:`ResourceLoader` designed to ease the loading of
Brett Cannon2a922ed2009-03-09 03:35:50 +0000381 Python source modules (bytecode is not handled; see
Brett Cannonf23e3742010-06-27 23:57:46 +0000382 :class:`SourceLoader` for a source/bytecode ABC). A subclass
Brett Cannon2a922ed2009-03-09 03:35:50 +0000383 implementing this ABC will only need to worry about exposing how the source
384 code is stored; all other details for loading Python source code will be
385 handled by the concrete implementations of key methods.
386
Brett Cannonf23e3742010-06-27 23:57:46 +0000387 .. deprecated:: 3.2
388 This class has been deprecated in favor of :class:`SourceLoader` and is
389 slated for removal in Python 3.4. See below for how to create a
Georg Brandl6faee4e2010-09-21 14:48:28 +0000390 subclass that is compatible with Python 3.1 onwards.
Brett Cannonf23e3742010-06-27 23:57:46 +0000391
392 If compatibility with Python 3.1 is required, then use the following idiom
393 to implement a subclass that will work with Python 3.1 onwards (make sure
394 to implement :meth:`ExecutionLoader.get_filename`)::
395
396 try:
397 from importlib.abc import SourceLoader
398 except ImportError:
399 from importlib.abc import PyLoader as SourceLoader
400
401
402 class CustomLoader(SourceLoader):
403 def get_filename(self, fullname):
404 """Return the path to the source file."""
405 # Implement ...
406
407 def source_path(self, fullname):
408 """Implement source_path in terms of get_filename."""
409 try:
410 return self.get_filename(fullname)
411 except ImportError:
412 return None
413
414 def is_package(self, fullname):
415 """Implement is_package by looking for an __init__ file
416 name as returned by get_filename."""
417 filename = os.path.basename(self.get_filename(fullname))
418 return os.path.splitext(filename)[0] == '__init__'
419
420
Brett Cannon9c751b72009-03-09 16:28:16 +0000421 .. method:: source_path(fullname)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000422
423 An abstract method that returns the path to the source code for a
Georg Brandl375aec22011-01-15 17:03:02 +0000424 module. Should return ``None`` if there is no source code.
Brett Cannon3e761a92009-12-10 00:24:21 +0000425 Raises :exc:`ImportError` if the loader knows it cannot handle the
426 module.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000427
Brett Cannon69194272009-07-20 04:23:48 +0000428 .. method:: get_filename(fullname)
429
430 A concrete implementation of
431 :meth:`importlib.abc.ExecutionLoader.get_filename` that
432 relies on :meth:`source_path`. If :meth:`source_path` returns
Georg Brandl375aec22011-01-15 17:03:02 +0000433 ``None``, then :exc:`ImportError` is raised.
Brett Cannon69194272009-07-20 04:23:48 +0000434
Brett Cannon9c751b72009-03-09 16:28:16 +0000435 .. method:: load_module(fullname)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000436
437 A concrete implementation of :meth:`importlib.abc.Loader.load_module`
Brett Cannonad876c72009-03-09 07:53:09 +0000438 that loads Python source code. All needed information comes from the
439 abstract methods required by this ABC. The only pertinent assumption
440 made by this method is that when loading a package
441 :attr:`__path__` is set to ``[os.path.dirname(__file__)]``.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000442
Brett Cannon9c751b72009-03-09 16:28:16 +0000443 .. method:: get_code(fullname)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000444
445 A concrete implementation of
446 :meth:`importlib.abc.InspectLoader.get_code` that creates code objects
Guido van Rossum09613542009-03-30 20:34:57 +0000447 from Python source code, by requesting the source code (using
Benjamin Peterson0089d752009-11-13 00:52:43 +0000448 :meth:`source_path` and :meth:`get_data`) and compiling it with the
449 built-in :func:`compile` function.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000450
Brett Cannond43b30b2009-03-10 03:29:23 +0000451 .. method:: get_source(fullname)
452
453 A concrete implementation of
454 :meth:`importlib.abc.InspectLoader.get_source`. Uses
Brett Cannon69194272009-07-20 04:23:48 +0000455 :meth:`importlib.abc.ResourceLoader.get_data` and :meth:`source_path`
456 to get the source code. It tries to guess the source encoding using
Guido van Rossum09613542009-03-30 20:34:57 +0000457 :func:`tokenize.detect_encoding`.
Brett Cannond43b30b2009-03-10 03:29:23 +0000458
Brett Cannon2a922ed2009-03-09 03:35:50 +0000459
460.. class:: PyPycLoader
461
Brett Cannonf23e3742010-06-27 23:57:46 +0000462 An abstract base class inheriting from :class:`PyLoader`.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000463 This ABC is meant to help in creating loaders that support both Python
464 source and bytecode.
465
Brett Cannonf23e3742010-06-27 23:57:46 +0000466 .. deprecated:: 3.2
467 This class has been deprecated in favor of :class:`SourceLoader` and to
468 properly support :pep:`3147`. If compatibility is required with
469 Python 3.1, implement both :class:`SourceLoader` and :class:`PyLoader`;
470 instructions on how to do so are included in the documentation for
471 :class:`PyLoader`. Do note that this solution will not support
472 sourceless/bytecode-only loading; only source *and* bytecode loading.
473
Brett Cannon1e331562012-07-02 14:35:34 -0400474 .. versionchanged:: 3.3
475 Updated to parse (but not use) the new source size field in bytecode
476 files when reading and to write out the field properly when writing.
477
Brett Cannon9c751b72009-03-09 16:28:16 +0000478 .. method:: source_mtime(fullname)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000479
480 An abstract method which returns the modification time for the source
481 code of the specified module. The modification time should be an
Georg Brandl375aec22011-01-15 17:03:02 +0000482 integer. If there is no source code, return ``None``. If the
Brett Cannon2a922ed2009-03-09 03:35:50 +0000483 module cannot be found then :exc:`ImportError` is raised.
484
Brett Cannon9c751b72009-03-09 16:28:16 +0000485 .. method:: bytecode_path(fullname)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000486
487 An abstract method which returns the path to the bytecode for the
Georg Brandl375aec22011-01-15 17:03:02 +0000488 specified module, if it exists. It returns ``None``
Guido van Rossum09613542009-03-30 20:34:57 +0000489 if no bytecode exists (yet).
Brett Cannon3e761a92009-12-10 00:24:21 +0000490 Raises :exc:`ImportError` if the loader knows it cannot handle the
491 module.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000492
Brett Cannon69194272009-07-20 04:23:48 +0000493 .. method:: get_filename(fullname)
494
495 A concrete implementation of
Brett Cannonf23e3742010-06-27 23:57:46 +0000496 :meth:`ExecutionLoader.get_filename` that relies on
497 :meth:`PyLoader.source_path` and :meth:`bytecode_path`.
Brett Cannon69194272009-07-20 04:23:48 +0000498 If :meth:`source_path` returns a path, then that value is returned.
499 Else if :meth:`bytecode_path` returns a path, that path will be
500 returned. If a path is not available from both methods,
501 :exc:`ImportError` is raised.
502
Brett Cannon9c751b72009-03-09 16:28:16 +0000503 .. method:: write_bytecode(fullname, bytecode)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000504
505 An abstract method which has the loader write *bytecode* for future
Georg Brandl375aec22011-01-15 17:03:02 +0000506 use. If the bytecode is written, return ``True``. Return
507 ``False`` if the bytecode could not be written. This method
Brett Cannon2a922ed2009-03-09 03:35:50 +0000508 should not be called if :data:`sys.dont_write_bytecode` is true.
Guido van Rossum09613542009-03-30 20:34:57 +0000509 The *bytecode* argument should be a bytes string or bytes array.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000510
511
Brett Cannon78246b62009-01-25 04:56:30 +0000512:mod:`importlib.machinery` -- Importers and path hooks
513------------------------------------------------------
514
515.. module:: importlib.machinery
516 :synopsis: Importers and path hooks
517
518This module contains the various objects that help :keyword:`import`
519find and load modules.
520
Brett Cannoncb66eb02012-05-11 12:58:42 -0400521.. attribute:: SOURCE_SUFFIXES
522
523 A list of strings representing the recognized file suffixes for source
524 modules.
525
526 .. versionadded:: 3.3
527
528.. attribute:: DEBUG_BYTECODE_SUFFIXES
529
530 A list of strings representing the file suffixes for non-optimized bytecode
531 modules.
532
533 .. versionadded:: 3.3
534
535.. attribute:: OPTIMIZED_BYTECODE_SUFFIXES
536
537 A list of strings representing the file suffixes for optimized bytecode
538 modules.
539
540 .. versionadded:: 3.3
541
542.. attribute:: BYTECODE_SUFFIXES
543
544 A list of strings representing the recognized file suffixes for bytecode
545 modules. Set to either :attr:`DEBUG_BYTECODE_SUFFIXES` or
546 :attr:`OPTIMIZED_BYTECODE_SUFFIXES` based on whether ``__debug__`` is true.
547
548 .. versionadded:: 3.3
549
550.. attribute:: EXTENSION_SUFFIXES
551
Nick Coghlan76e07702012-07-18 23:14:57 +1000552 A list of strings representing the recognized file suffixes for
Brett Cannoncb66eb02012-05-11 12:58:42 -0400553 extension modules.
554
555 .. versionadded:: 3.3
556
Nick Coghlanc5afd422012-07-18 23:59:08 +1000557.. function:: all_suffixes()
Nick Coghlan76e07702012-07-18 23:14:57 +1000558
559 Returns a combined list of strings representing all file suffixes for
Nick Coghlanc5afd422012-07-18 23:59:08 +1000560 modules recognized by the standard import machinery. This is a
Nick Coghlan76e07702012-07-18 23:14:57 +1000561 helper for code which simply needs to know if a filesystem path
Nick Coghlanc5afd422012-07-18 23:59:08 +1000562 potentially refers to a module without needing any details on the kind
563 of module (for example, :func:`inspect.getmodulename`)
Nick Coghlan76e07702012-07-18 23:14:57 +1000564
565 .. versionadded:: 3.3
566
567
Brett Cannon78246b62009-01-25 04:56:30 +0000568.. class:: BuiltinImporter
569
Brett Cannon2a922ed2009-03-09 03:35:50 +0000570 An :term:`importer` for built-in modules. All known built-in modules are
571 listed in :data:`sys.builtin_module_names`. This class implements the
Brett Cannona113ac52009-03-15 01:41:33 +0000572 :class:`importlib.abc.Finder` and :class:`importlib.abc.InspectLoader`
573 ABCs.
Brett Cannon78246b62009-01-25 04:56:30 +0000574
575 Only class methods are defined by this class to alleviate the need for
576 instantiation.
577
Brett Cannon78246b62009-01-25 04:56:30 +0000578
579.. class:: FrozenImporter
580
Brett Cannon2a922ed2009-03-09 03:35:50 +0000581 An :term:`importer` for frozen modules. This class implements the
Brett Cannon8d110132009-03-15 02:20:16 +0000582 :class:`importlib.abc.Finder` and :class:`importlib.abc.InspectLoader`
583 ABCs.
Brett Cannon78246b62009-01-25 04:56:30 +0000584
585 Only class methods are defined by this class to alleviate the need for
586 instantiation.
587
Brett Cannondebb98d2009-02-16 04:18:01 +0000588
589.. class:: PathFinder
590
Brett Cannon2a922ed2009-03-09 03:35:50 +0000591 :term:`Finder` for :data:`sys.path`. This class implements the
592 :class:`importlib.abc.Finder` ABC.
Brett Cannondebb98d2009-02-16 04:18:01 +0000593
594 This class does not perfectly mirror the semantics of :keyword:`import` in
595 terms of :data:`sys.path`. No implicit path hooks are assumed for
Brett Cannon83ac0132012-03-02 12:10:48 -0500596 simplification of the class and its semantics. This implies that when
597 ``None`` is found in :data:`sys.path_importer_cache` that it is simply
598 ignored instead of implying a default finder.
Brett Cannondebb98d2009-02-16 04:18:01 +0000599
Brett Cannon1b184d52009-11-07 08:22:58 +0000600 Only class methods are defined by this class to alleviate the need for
Brett Cannondebb98d2009-02-16 04:18:01 +0000601 instantiation.
602
603 .. classmethod:: find_module(fullname, path=None)
604
605 Class method that attempts to find a :term:`loader` for the module
Brett Cannon2a922ed2009-03-09 03:35:50 +0000606 specified by *fullname* on :data:`sys.path` or, if defined, on
Brett Cannondebb98d2009-02-16 04:18:01 +0000607 *path*. For each path entry that is searched,
Brett Cannon75321e82012-03-02 11:58:25 -0500608 :data:`sys.path_importer_cache` is checked. If a non-false object is
Brett Cannon2a922ed2009-03-09 03:35:50 +0000609 found then it is used as the :term:`finder` to look for the module
610 being searched for. If no entry is found in
Brett Cannondebb98d2009-02-16 04:18:01 +0000611 :data:`sys.path_importer_cache`, then :data:`sys.path_hooks` is
612 searched for a finder for the path entry and, if found, is stored in
613 :data:`sys.path_importer_cache` along with being queried about the
Georg Brandl375aec22011-01-15 17:03:02 +0000614 module. If no finder is ever found then ``None`` is returned.
Brett Cannond2e7b332009-02-17 02:45:03 +0000615
616
Brett Cannon938d44d2012-04-22 19:58:33 -0400617.. class:: FileFinder(path, \*loader_details)
618
619 A concrete implementation of :class:`importlib.abc.Finder` which caches
620 results from the file system.
621
622 The *path* argument is the directory for which the finder is in charge of
623 searching.
624
625 The *loader_details* argument is a variable number of 3-item tuples each
626 containing a loader, file suffixes the loader recognizes, and a boolean
627 representing whether the loader handles packages.
628
629 The finder will cache the directory contents as necessary, making stat calls
630 for each module search to verify the cache is not outdated. Because cache
631 staleness relies upon the granularity of the operating system's state
632 information of the file system, there is a potential race condition of
633 searching for a module, creating a new file, and then searching for the
634 module the new file represents. If the operations happen fast enough to fit
635 within the granularity of stat calls, then the module search will fail. To
636 prevent this from happening, when you create a module dynamically, make sure
637 to call :func:`importlib.invalidate_caches`.
638
639 .. versionadded:: 3.3
640
641 .. attribute:: path
642
643 The path the finder will search in.
644
645 .. method:: find_module(fullname)
646
647 Attempt to find the loader to handle *fullname* within :attr:`path`.
648
649 .. method:: invalidate_caches()
650
651 Clear out the internal cache.
652
653 .. classmethod:: path_hook(\*loader_details)
654
655 A class method which returns a closure for use on :attr:`sys.path_hooks`.
656 An instance of :class:`FileFinder` is returned by the closure using the
657 path argument given to the closure directly and *loader_details*
658 indirectly.
659
660 If the argument to the closure is not an existing directory,
661 :exc:`ImportError` is raised.
662
663
664.. class:: SourceFileLoader(fullname, path)
665
666 A concrete implementation of :class:`importlib.abc.SourceLoader` by
667 subclassing :class:`importlib.abc.FileLoader` and providing some concrete
668 implementations of other methods.
669
670 .. versionadded:: 3.3
671
672 .. attribute:: name
673
674 The name of the module that this loader will handle.
675
676 .. attribute:: path
677
678 The path to the source file.
679
680 .. method:: is_package(fullname)
681
682 Return true if :attr:`path` appears to be for a package.
683
684 .. method:: path_stats(path)
685
686 Concrete implementation of :meth:`importlib.abc.SourceLoader.path_stats`.
687
688 .. method:: set_data(path, data)
689
690 Concrete implementation of :meth:`importlib.abc.SourceLoader.set_data`.
691
Brett Cannon938d44d2012-04-22 19:58:33 -0400692
Marc-Andre Lemburg4fe29c92012-04-25 02:31:37 +0200693.. class:: SourcelessFileLoader(fullname, path)
Brett Cannon938d44d2012-04-22 19:58:33 -0400694
695 A concrete implementation of :class:`importlib.abc.FileLoader` which can
696 import bytecode files (i.e. no source code files exist).
697
Marc-Andre Lemburg4fe29c92012-04-25 02:31:37 +0200698 Please note that direct use of bytecode files (and thus not source code
699 files) inhibits your modules from being usable by all Python
700 implementations or new versions of Python which change the bytecode
701 format.
Brett Cannon938d44d2012-04-22 19:58:33 -0400702
703 .. versionadded:: 3.3
704
705 .. attribute:: name
706
707 The name of the module the loader will handle.
708
709 .. attribute:: path
710
711 The path to the bytecode file.
712
713 .. method:: is_package(fullname)
714
715 Determines if the module is a package based on :attr:`path`.
716
717 .. method:: get_code(fullname)
718
719 Returns the code object for :attr:`name` created from :attr:`path`.
720
721 .. method:: get_source(fullname)
722
723 Returns ``None`` as bytecode files have no source when this loader is
724 used.
725
Brett Cannon938d44d2012-04-22 19:58:33 -0400726
727.. class:: ExtensionFileLoader(fullname, path)
728
729 A concrete implementation of :class:`importlib.abc.InspectLoader` for
730 extension modules.
731
732 The *fullname* argument specifies the name of the module the loader is to
733 support. The *path* argument is the path to the extension module's file.
734
735 .. versionadded:: 3.3
736
737 .. attribute:: name
738
739 Name of the module the loader supports.
740
741 .. attribute:: path
742
743 Path to the extension module.
744
Barry Warsawd7d21942012-07-29 16:36:17 -0400745 .. method:: load_module(fullname)
Brett Cannon938d44d2012-04-22 19:58:33 -0400746
Brett Cannonc0499522012-05-11 14:48:41 -0400747 Loads the extension module if and only if *fullname* is the same as
748 :attr:`name` or is ``None``.
Brett Cannon938d44d2012-04-22 19:58:33 -0400749
750 .. method:: is_package(fullname)
751
752 Returns ``False`` as extension modules can never be packages.
753
754 .. method:: get_code(fullname)
755
756 Returns ``None`` as extension modules lack a code object.
757
758 .. method:: get_source(fullname)
759
760 Returns ``None`` as extension modules do not have source code.
761
762
Brett Cannond2e7b332009-02-17 02:45:03 +0000763:mod:`importlib.util` -- Utility code for importers
764---------------------------------------------------
765
766.. module:: importlib.util
Brett Cannon75321e82012-03-02 11:58:25 -0500767 :synopsis: Utility code for importers
Brett Cannond2e7b332009-02-17 02:45:03 +0000768
769This module contains the various objects that help in the construction of
770an :term:`importer`.
771
Brett Cannond200bf52012-05-13 13:45:09 -0400772.. function:: resolve_name(name, package)
773
774 Resolve a relative module name to an absolute one.
775
776 If **name** has no leading dots, then **name** is simply returned. This
777 allows for usage such as
778 ``importlib.util.resolve_name('sys', __package__)`` without doing a
779 check to see if the **package** argument is needed.
780
781 :exc:`ValueError` is raised if **name** is a relative module name but
782 package is a false value (e.g. ``None`` or the empty string).
783 :exc:`ValueError` is also raised a relative name would escape its containing
784 package (e.g. requesting ``..bacon`` from within the ``spam`` package).
785
786 .. versionadded:: 3.3
787
Georg Brandl8a1caa22010-07-29 16:01:11 +0000788.. decorator:: module_for_loader
Brett Cannond2e7b332009-02-17 02:45:03 +0000789
Guido van Rossum09613542009-03-30 20:34:57 +0000790 A :term:`decorator` for a :term:`loader` method,
791 to handle selecting the proper
Brett Cannond2e7b332009-02-17 02:45:03 +0000792 module object to load with. The decorated method is expected to have a call
Brett Cannon2a922ed2009-03-09 03:35:50 +0000793 signature taking two positional arguments
794 (e.g. ``load_module(self, module)``) for which the second argument
Guido van Rossum09613542009-03-30 20:34:57 +0000795 will be the module **object** to be used by the loader.
Brett Cannonefad00d2012-04-27 17:27:14 -0400796 Note that the decorator will not work on static methods because of the
797 assumption of two arguments.
Brett Cannond2e7b332009-02-17 02:45:03 +0000798
Guido van Rossum09613542009-03-30 20:34:57 +0000799 The decorated method will take in the **name** of the module to be loaded
800 as expected for a :term:`loader`. If the module is not found in
Brett Cannon57b46f52009-03-02 14:38:26 +0000801 :data:`sys.modules` then a new one is constructed with its
Brett Cannonefad00d2012-04-27 17:27:14 -0400802 :attr:`__name__` attribute set to **name**, :attr:`__loader__` set to
803 **self**, and :attr:`__package__` set if
804 :meth:`importlib.abc.InspectLoader.is_package` is defined for **self** and
805 does not raise :exc:`ImportError` for **name**. If a new module is not
806 needed then the module found in :data:`sys.modules` will be passed into the
807 method.
808
809 If an exception is raised by the decorated method and a module was added to
Brett Cannond2e7b332009-02-17 02:45:03 +0000810 :data:`sys.modules` it will be removed to prevent a partially initialized
Brett Cannon57b46f52009-03-02 14:38:26 +0000811 module from being in left in :data:`sys.modules`. If the module was already
812 in :data:`sys.modules` then it is left alone.
Brett Cannond2e7b332009-02-17 02:45:03 +0000813
Guido van Rossum09613542009-03-30 20:34:57 +0000814 Use of this decorator handles all the details of which module object a
Brett Cannonefad00d2012-04-27 17:27:14 -0400815 loader should initialize as specified by :pep:`302` as best as possible.
816
817 .. versionchanged:: 3.3
Georg Brandl61063cc2012-06-24 22:48:30 +0200818 :attr:`__loader__` and :attr:`__package__` are automatically set
819 (when possible).
Brett Cannon57b46f52009-03-02 14:38:26 +0000820
Georg Brandl8a1caa22010-07-29 16:01:11 +0000821.. decorator:: set_loader
Brett Cannon2cf03a82009-03-10 05:17:37 +0000822
Guido van Rossum09613542009-03-30 20:34:57 +0000823 A :term:`decorator` for a :term:`loader` method,
824 to set the :attr:`__loader__`
Brett Cannon2cf03a82009-03-10 05:17:37 +0000825 attribute on loaded modules. If the attribute is already set the decorator
826 does nothing. It is assumed that the first positional argument to the
Brett Cannon75321e82012-03-02 11:58:25 -0500827 wrapped method (i.e. ``self``) is what :attr:`__loader__` should be set to.
Brett Cannon2cf03a82009-03-10 05:17:37 +0000828
Brett Cannonefad00d2012-04-27 17:27:14 -0400829 .. note::
830
831 It is recommended that :func:`module_for_loader` be used over this
832 decorator as it subsumes this functionality.
833
834
Georg Brandl8a1caa22010-07-29 16:01:11 +0000835.. decorator:: set_package
Brett Cannon57b46f52009-03-02 14:38:26 +0000836
837 A :term:`decorator` for a :term:`loader` to set the :attr:`__package__`
838 attribute on the module returned by the loader. If :attr:`__package__` is
Georg Brandl375aec22011-01-15 17:03:02 +0000839 set and has a value other than ``None`` it will not be changed.
Brett Cannon57b46f52009-03-02 14:38:26 +0000840 Note that the module returned by the loader is what has the attribute
841 set on and not the module found in :data:`sys.modules`.
Guido van Rossum09613542009-03-30 20:34:57 +0000842
Brett Cannon16248a42009-04-01 20:47:14 +0000843 Reliance on this decorator is discouraged when it is possible to set
Brett Cannon75321e82012-03-02 11:58:25 -0500844 :attr:`__package__` before importing. By
845 setting it beforehand the code for the module is executed with the
846 attribute set and thus can be used by global level code during
Brett Cannon16248a42009-04-01 20:47:14 +0000847 initialization.
848
Brett Cannonefad00d2012-04-27 17:27:14 -0400849 .. note::
850
851 It is recommended that :func:`module_for_loader` be used over this
852 decorator as it subsumes this functionality.