blob: 06d997fdf066c73d342efcb818002a69a29f22a8 [file] [log] [blame]
Brett Cannonafccd632009-01-20 02:21:27 +00001:mod:`importlib` -- An implementation of :keyword:`import`
2==========================================================
3
4.. module:: importlib
5 :synopsis: An implementation of the import machinery.
6
7.. moduleauthor:: Brett Cannon <brett@python.org>
8.. sectionauthor:: Brett Cannon <brett@python.org>
9
10.. versionadded:: 3.1
11
12
13Introduction
14------------
15
16The purpose of the :mod:`importlib` package is two-fold. One is to provide an
17implementation of the :keyword:`import` statement (and thus, by extension, the
18:func:`__import__` function) in Python source code. This provides an
Tarek Ziadé434caaa2009-05-14 12:48:09 +000019implementation of :keyword:`import` which is portable to any Python
Brett Cannonafccd632009-01-20 02:21:27 +000020interpreter. This also provides a reference implementation which is easier to
Brett Cannonf23e3742010-06-27 23:57:46 +000021comprehend than one implemented in a programming language other than Python.
Brett Cannonafccd632009-01-20 02:21:27 +000022
Brett Cannonf23e3742010-06-27 23:57:46 +000023Two, the components to implement :keyword:`import` are exposed in this
Brett Cannonafccd632009-01-20 02:21:27 +000024package, making it easier for users to create their own custom objects (known
Brett Cannondebb98d2009-02-16 04:18:01 +000025generically as an :term:`importer`) to participate in the import process.
Brett Cannonf23e3742010-06-27 23:57:46 +000026Details on custom importers can be found in :pep:`302`.
Brett Cannonafccd632009-01-20 02:21:27 +000027
28.. seealso::
29
30 :ref:`import`
31 The language reference for the :keyword:`import` statement.
32
33 `Packages specification <http://www.python.org/doc/essays/packages.html>`__
34 Original specification of packages. Some semantics have changed since
Georg Brandl375aec22011-01-15 17:03:02 +000035 the writing of this document (e.g. redirecting based on ``None``
Brett Cannonafccd632009-01-20 02:21:27 +000036 in :data:`sys.modules`).
37
38 The :func:`.__import__` function
Brett Cannon0e13c942010-06-29 18:26:11 +000039 The :keyword:`import` statement is syntactic sugar for this function.
Brett Cannonafccd632009-01-20 02:21:27 +000040
41 :pep:`235`
42 Import on Case-Insensitive Platforms
43
44 :pep:`263`
45 Defining Python Source Code Encodings
46
47 :pep:`302`
Brett Cannonf23e3742010-06-27 23:57:46 +000048 New Import Hooks
Brett Cannonafccd632009-01-20 02:21:27 +000049
50 :pep:`328`
51 Imports: Multi-Line and Absolute/Relative
52
53 :pep:`366`
54 Main module explicit relative imports
55
Brett Cannon8917d5e2010-01-13 19:21:00 +000056 :pep:`3120`
Brett Cannonafccd632009-01-20 02:21:27 +000057 Using UTF-8 as the Default Source Encoding
58
Brett Cannon30b7a902010-06-27 21:49:22 +000059 :pep:`3147`
60 PYC Repository Directories
61
Brett Cannonafccd632009-01-20 02:21:27 +000062
63Functions
64---------
65
Brett Cannon78246b62009-01-25 04:56:30 +000066.. function:: __import__(name, globals={}, locals={}, fromlist=list(), level=0)
Brett Cannonafccd632009-01-20 02:21:27 +000067
Brett Cannonf23e3742010-06-27 23:57:46 +000068 An implementation of the built-in :func:`__import__` function.
Brett Cannonafccd632009-01-20 02:21:27 +000069
70.. function:: import_module(name, package=None)
71
Brett Cannon33418c82009-01-22 18:37:20 +000072 Import a module. The *name* argument specifies what module to
Brett Cannonafccd632009-01-20 02:21:27 +000073 import in absolute or relative terms
74 (e.g. either ``pkg.mod`` or ``..mod``). If the name is
Guido van Rossum09613542009-03-30 20:34:57 +000075 specified in relative terms, then the *package* argument must be set to
76 the name of the package which is to act as the anchor for resolving the
Brett Cannonafccd632009-01-20 02:21:27 +000077 package name (e.g. ``import_module('..mod', 'pkg.subpkg')`` will import
Brett Cannon2c318a12009-02-07 01:15:27 +000078 ``pkg.mod``).
Brett Cannon78246b62009-01-25 04:56:30 +000079
Brett Cannon2c318a12009-02-07 01:15:27 +000080 The :func:`import_module` function acts as a simplifying wrapper around
Brett Cannon9f4cb1c2009-04-01 23:26:47 +000081 :func:`importlib.__import__`. This means all semantics of the function are
82 derived from :func:`importlib.__import__`, including requiring the package
83 from which an import is occurring to have been previously imported
84 (i.e., *package* must already be imported). The most important difference
85 is that :func:`import_module` returns the most nested package or module
86 that was imported (e.g. ``pkg.mod``), while :func:`__import__` returns the
Guido van Rossum09613542009-03-30 20:34:57 +000087 top-level package or module (e.g. ``pkg``).
88
Brett Cannonee78a2b2012-05-12 17:43:17 -040089.. function:: find_loader(name, path=None)
90
91 Find the loader for a module, optionally within the specified *path*. If the
92 module is in :attr:`sys.modules`, then ``sys.modules[name].__loader__`` is
93 returned (unless the loader would be ``None``, in which case
94 :exc:`ValueError` is raised). Otherwise a search using :attr:`sys.meta_path`
95 is done. ``None`` is returned if no loader is found.
96
97 A dotted name does not have its parent's implicitly imported. If that is
98 desired (although not nessarily required to find the loader, it will most
99 likely be needed if the loader actually is used to load the module), then
100 you will have to import the packages containing the module prior to calling
101 this function.
102
Antoine Pitrouc541f8e2012-02-20 01:48:16 +0100103.. function:: invalidate_caches()
104
Brett Cannonb46a1792012-02-27 18:15:42 -0500105 Invalidate the internal caches of the finders stored at
106 :data:`sys.path_importer_cache`. If a finder implements
107 :meth:`abc.Finder.invalidate_caches()` then it will be called to perform the
108 invalidation. This function may be needed if some modules are installed
109 while your program is running and you expect the program to notice the
110 changes.
Antoine Pitrouc541f8e2012-02-20 01:48:16 +0100111
112 .. versionadded:: 3.3
113
Brett Cannon78246b62009-01-25 04:56:30 +0000114
Brett Cannon2a922ed2009-03-09 03:35:50 +0000115:mod:`importlib.abc` -- Abstract base classes related to import
116---------------------------------------------------------------
117
118.. module:: importlib.abc
119 :synopsis: Abstract base classes related to import
120
121The :mod:`importlib.abc` module contains all of the core abstract base classes
122used by :keyword:`import`. Some subclasses of the core abstract base classes
123are also provided to help in implementing the core ABCs.
124
125
126.. class:: Finder
127
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000128 An abstract base class representing a :term:`finder`. Finder
129 implementations should derive from (or register with) the more specific
130 :class:`MetaPathFinder` or :class:`PathEntryFinder` ABCs.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000131
Brett Cannonb46a1792012-02-27 18:15:42 -0500132 .. method:: invalidate_caches()
133
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000134 An optional method which, when called, should invalidate any internal
135 cache used by the finder. Used by :func:`invalidate_caches()` when
136 invalidating the caches of all cached finders.
137
138 .. versionchanged:: 3.3
139 The API signatures for meta path finders and path entry finders
140 were separated by PEP 420. Accordingly, the Finder ABC no
141 longer requires implementation of a ``find_module()`` method.
142
143
144.. class:: MetaPathFinder(Finder)
145
146 An abstract base class representing a :term:`meta path finder`.
147
148 .. versionadded:: 3.3
149
150 .. method:: find_module(fullname, path)
151
152 An abstract method for finding a :term:`loader` for the specified
153 module. If this is a top-level import, *path* will be ``None``.
154 Otheriwse, this is a search for a subpackage or module and *path*
155 will be the value of :attr:`__path__` from the parent
156 package. If a loader cannot be found, ``None`` is returned.
157
158
159.. class:: PathEntryFinder(Finder)
160
161 An abstract base class representing a :term:`path entry finder`.
162
163 .. versionadded:: 3.3
164
165 .. method:: find_loader(self, fullname):
166
167 An abstract method for finding a :term:`loader` for the specified
168 module. Returns a 2-tuple of ``(loader, portion)`` where portion is a
169 sequence of file system locations contributing to part of a namespace
170 package. The sequence may be empty.
Brett Cannonb46a1792012-02-27 18:15:42 -0500171
Brett Cannon2a922ed2009-03-09 03:35:50 +0000172
173.. class:: Loader
174
175 An abstract base class for a :term:`loader`.
Guido van Rossum09613542009-03-30 20:34:57 +0000176 See :pep:`302` for the exact definition for a loader.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000177
Brett Cannon9c751b72009-03-09 16:28:16 +0000178 .. method:: load_module(fullname)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000179
180 An abstract method for loading a module. If the module cannot be
181 loaded, :exc:`ImportError` is raised, otherwise the loaded module is
182 returned.
183
Guido van Rossum09613542009-03-30 20:34:57 +0000184 If the requested module already exists in :data:`sys.modules`, that
Brett Cannon2a922ed2009-03-09 03:35:50 +0000185 module should be used and reloaded.
Guido van Rossum09613542009-03-30 20:34:57 +0000186 Otherwise the loader should create a new module and insert it into
187 :data:`sys.modules` before any loading begins, to prevent recursion
188 from the import. If the loader inserted a module and the load fails, it
Brett Cannon2a922ed2009-03-09 03:35:50 +0000189 must be removed by the loader from :data:`sys.modules`; modules already
190 in :data:`sys.modules` before the loader began execution should be left
191 alone. The :func:`importlib.util.module_for_loader` decorator handles
192 all of these details.
193
Guido van Rossum09613542009-03-30 20:34:57 +0000194 The loader should set several attributes on the module.
195 (Note that some of these attributes can change when a module is
196 reloaded.)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000197
198 - :attr:`__name__`
199 The name of the module.
200
201 - :attr:`__file__`
202 The path to where the module data is stored (not set for built-in
203 modules).
204
205 - :attr:`__path__`
Guido van Rossum09613542009-03-30 20:34:57 +0000206 A list of strings specifying the search path within a
Brett Cannon2a922ed2009-03-09 03:35:50 +0000207 package. This attribute is not set on modules.
208
209 - :attr:`__package__`
210 The parent package for the module/package. If the module is
211 top-level then it has a value of the empty string. The
212 :func:`importlib.util.set_package` decorator can handle the details
213 for :attr:`__package__`.
214
215 - :attr:`__loader__`
Guido van Rossum09613542009-03-30 20:34:57 +0000216 The loader used to load the module.
217 (This is not set by the built-in import machinery,
218 but it should be set whenever a :term:`loader` is used.)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000219
Barry Warsawd7d21942012-07-29 16:36:17 -0400220 .. method:: module_repr(module)
221
222 An abstract method which when implemented calculates and returns the
223 given module's repr, as a string.
224
225 .. versionadded: 3.3
226
Brett Cannon2a922ed2009-03-09 03:35:50 +0000227
228.. class:: ResourceLoader
229
230 An abstract base class for a :term:`loader` which implements the optional
231 :pep:`302` protocol for loading arbitrary resources from the storage
232 back-end.
233
Brett Cannon9c751b72009-03-09 16:28:16 +0000234 .. method:: get_data(path)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000235
236 An abstract method to return the bytes for the data located at *path*.
Guido van Rossum09613542009-03-30 20:34:57 +0000237 Loaders that have a file-like storage back-end
Brett Cannon16248a42009-04-01 20:47:14 +0000238 that allows storing arbitrary data
Guido van Rossum09613542009-03-30 20:34:57 +0000239 can implement this abstract method to give direct access
Brett Cannon2a922ed2009-03-09 03:35:50 +0000240 to the data stored. :exc:`IOError` is to be raised if the *path* cannot
241 be found. The *path* is expected to be constructed using a module's
Brett Cannon16248a42009-04-01 20:47:14 +0000242 :attr:`__file__` attribute or an item from a package's :attr:`__path__`.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000243
244
245.. class:: InspectLoader
246
247 An abstract base class for a :term:`loader` which implements the optional
Guido van Rossum09613542009-03-30 20:34:57 +0000248 :pep:`302` protocol for loaders that inspect modules.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000249
Brett Cannona113ac52009-03-15 01:41:33 +0000250 .. method:: get_code(fullname)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000251
Brett Cannona113ac52009-03-15 01:41:33 +0000252 An abstract method to return the :class:`code` object for a module.
Georg Brandl375aec22011-01-15 17:03:02 +0000253 ``None`` is returned if the module does not have a code object
Brett Cannona113ac52009-03-15 01:41:33 +0000254 (e.g. built-in module). :exc:`ImportError` is raised if loader cannot
255 find the requested module.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000256
Brett Cannon9c751b72009-03-09 16:28:16 +0000257 .. method:: get_source(fullname)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000258
259 An abstract method to return the source of a module. It is returned as
Georg Brandl375aec22011-01-15 17:03:02 +0000260 a text string with universal newlines. Returns ``None`` if no
Brett Cannon2a922ed2009-03-09 03:35:50 +0000261 source is available (e.g. a built-in module). Raises :exc:`ImportError`
262 if the loader cannot find the module specified.
263
Brett Cannona113ac52009-03-15 01:41:33 +0000264 .. method:: is_package(fullname)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000265
Brett Cannona113ac52009-03-15 01:41:33 +0000266 An abstract method to return a true value if the module is a package, a
267 false value otherwise. :exc:`ImportError` is raised if the
268 :term:`loader` cannot find the module.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000269
270
Brett Cannon69194272009-07-20 04:23:48 +0000271.. class:: ExecutionLoader
272
273 An abstract base class which inherits from :class:`InspectLoader` that,
Brett Cannon23460292009-07-20 22:59:00 +0000274 when implemented, helps a module to be executed as a script. The ABC
Brett Cannon69194272009-07-20 04:23:48 +0000275 represents an optional :pep:`302` protocol.
276
277 .. method:: get_filename(fullname)
278
Brett Cannonf23e3742010-06-27 23:57:46 +0000279 An abstract method that is to return the value of :attr:`__file__` for
Brett Cannon69194272009-07-20 04:23:48 +0000280 the specified module. If no path is available, :exc:`ImportError` is
281 raised.
282
Brett Cannonf23e3742010-06-27 23:57:46 +0000283 If source code is available, then the method should return the path to
284 the source file, regardless of whether a bytecode was used to load the
285 module.
286
287
Brett Cannon938d44d2012-04-22 19:58:33 -0400288.. class:: FileLoader(fullname, path)
289
290 An abstract base class which inherits from :class:`ResourceLoader` and
291 :class:`ExecutionLoader`, providing concreate implementations of
292 :meth:`ResourceLoader.get_data` and :meth:`ExecutionLoader.get_filename`.
293
294 The *fullname* argument is a fully resolved name of the module the loader is
295 to handle. The *path* argument is the path to the file for the module.
296
297 .. versionadded:: 3.3
298
299 .. attribute:: name
300
301 The name of the module the loader can handle.
302
303 .. attribute:: path
304
305 Path to the file of the module.
306
Barry Warsawd7d21942012-07-29 16:36:17 -0400307 .. method:: load_module(fullname)
Brett Cannonc0499522012-05-11 14:48:41 -0400308
Barry Warsawd7d21942012-07-29 16:36:17 -0400309 Calls super's ``load_module()``.
Brett Cannonc0499522012-05-11 14:48:41 -0400310
Brett Cannon938d44d2012-04-22 19:58:33 -0400311 .. method:: get_filename(fullname)
312
Barry Warsawd7d21942012-07-29 16:36:17 -0400313 Returns :attr:`path`.
Brett Cannon938d44d2012-04-22 19:58:33 -0400314
315 .. method:: get_data(path)
316
317 Returns the open, binary file for *path*.
318
319
Brett Cannonf23e3742010-06-27 23:57:46 +0000320.. class:: SourceLoader
321
322 An abstract base class for implementing source (and optionally bytecode)
323 file loading. The class inherits from both :class:`ResourceLoader` and
324 :class:`ExecutionLoader`, requiring the implementation of:
325
326 * :meth:`ResourceLoader.get_data`
327 * :meth:`ExecutionLoader.get_filename`
Brett Cannon6dfbff32010-07-21 09:48:31 +0000328 Should only return the path to the source file; sourceless
Brett Cannonf23e3742010-06-27 23:57:46 +0000329 loading is not supported.
330
331 The abstract methods defined by this class are to add optional bytecode
332 file support. Not implementing these optional methods causes the loader to
333 only work with source code. Implementing the methods allows the loader to
334 work with source *and* bytecode files; it does not allow for *sourceless*
335 loading where only bytecode is provided. Bytecode files are an
336 optimization to speed up loading by removing the parsing step of Python's
337 compiler, and so no bytecode-specific API is exposed.
338
Antoine Pitrou5136ac02012-01-13 18:52:16 +0100339 .. method:: path_stats(self, path)
340
341 Optional abstract method which returns a :class:`dict` containing
342 metadata about the specifed path. Supported dictionary keys are:
343
344 - ``'mtime'`` (mandatory): an integer or floating-point number
345 representing the modification time of the source code;
346 - ``'size'`` (optional): the size in bytes of the source code.
347
348 Any other keys in the dictionary are ignored, to allow for future
349 extensions.
350
351 .. versionadded:: 3.3
352
Brett Cannonf23e3742010-06-27 23:57:46 +0000353 .. method:: path_mtime(self, path)
354
355 Optional abstract method which returns the modification time for the
356 specified path.
357
Antoine Pitrou5136ac02012-01-13 18:52:16 +0100358 .. deprecated:: 3.3
359 This method is deprecated in favour of :meth:`path_stats`. You don't
360 have to implement it, but it is still available for compatibility
361 purposes.
362
Brett Cannonf23e3742010-06-27 23:57:46 +0000363 .. method:: set_data(self, path, data)
364
365 Optional abstract method which writes the specified bytes to a file
Brett Cannon61b14252010-07-03 21:48:25 +0000366 path. Any intermediate directories which do not exist are to be created
367 automatically.
368
369 When writing to the path fails because the path is read-only
370 (:attr:`errno.EACCES`), do not propagate the exception.
Brett Cannonf23e3742010-06-27 23:57:46 +0000371
372 .. method:: get_code(self, fullname)
373
374 Concrete implementation of :meth:`InspectLoader.get_code`.
375
376 .. method:: load_module(self, fullname)
377
378 Concrete implementation of :meth:`Loader.load_module`.
379
380 .. method:: get_source(self, fullname)
381
382 Concrete implementation of :meth:`InspectLoader.get_source`.
383
384 .. method:: is_package(self, fullname)
385
386 Concrete implementation of :meth:`InspectLoader.is_package`. A module
Brett Cannonea0b8232012-06-15 20:00:53 -0400387 is determined to be a package if its file path (as provided by
388 :meth:`ExecutionLoader.get_filename`) is a file named
389 ``__init__`` when the file extension is removed **and** the module name
390 itself does not end in ``__init__``.
Brett Cannonf23e3742010-06-27 23:57:46 +0000391
Brett Cannon69194272009-07-20 04:23:48 +0000392
Brett Cannon2a922ed2009-03-09 03:35:50 +0000393.. class:: PyLoader
394
Brett Cannon69194272009-07-20 04:23:48 +0000395 An abstract base class inheriting from
Brett Cannonf23e3742010-06-27 23:57:46 +0000396 :class:`ExecutionLoader` and
397 :class:`ResourceLoader` designed to ease the loading of
Brett Cannon2a922ed2009-03-09 03:35:50 +0000398 Python source modules (bytecode is not handled; see
Brett Cannonf23e3742010-06-27 23:57:46 +0000399 :class:`SourceLoader` for a source/bytecode ABC). A subclass
Brett Cannon2a922ed2009-03-09 03:35:50 +0000400 implementing this ABC will only need to worry about exposing how the source
401 code is stored; all other details for loading Python source code will be
402 handled by the concrete implementations of key methods.
403
Brett Cannonf23e3742010-06-27 23:57:46 +0000404 .. deprecated:: 3.2
405 This class has been deprecated in favor of :class:`SourceLoader` and is
406 slated for removal in Python 3.4. See below for how to create a
Georg Brandl6faee4e2010-09-21 14:48:28 +0000407 subclass that is compatible with Python 3.1 onwards.
Brett Cannonf23e3742010-06-27 23:57:46 +0000408
409 If compatibility with Python 3.1 is required, then use the following idiom
410 to implement a subclass that will work with Python 3.1 onwards (make sure
411 to implement :meth:`ExecutionLoader.get_filename`)::
412
413 try:
414 from importlib.abc import SourceLoader
415 except ImportError:
416 from importlib.abc import PyLoader as SourceLoader
417
418
419 class CustomLoader(SourceLoader):
420 def get_filename(self, fullname):
421 """Return the path to the source file."""
422 # Implement ...
423
424 def source_path(self, fullname):
425 """Implement source_path in terms of get_filename."""
426 try:
427 return self.get_filename(fullname)
428 except ImportError:
429 return None
430
431 def is_package(self, fullname):
432 """Implement is_package by looking for an __init__ file
433 name as returned by get_filename."""
434 filename = os.path.basename(self.get_filename(fullname))
435 return os.path.splitext(filename)[0] == '__init__'
436
437
Brett Cannon9c751b72009-03-09 16:28:16 +0000438 .. method:: source_path(fullname)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000439
440 An abstract method that returns the path to the source code for a
Georg Brandl375aec22011-01-15 17:03:02 +0000441 module. Should return ``None`` if there is no source code.
Brett Cannon3e761a92009-12-10 00:24:21 +0000442 Raises :exc:`ImportError` if the loader knows it cannot handle the
443 module.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000444
Brett Cannon69194272009-07-20 04:23:48 +0000445 .. method:: get_filename(fullname)
446
447 A concrete implementation of
448 :meth:`importlib.abc.ExecutionLoader.get_filename` that
449 relies on :meth:`source_path`. If :meth:`source_path` returns
Georg Brandl375aec22011-01-15 17:03:02 +0000450 ``None``, then :exc:`ImportError` is raised.
Brett Cannon69194272009-07-20 04:23:48 +0000451
Brett Cannon9c751b72009-03-09 16:28:16 +0000452 .. method:: load_module(fullname)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000453
454 A concrete implementation of :meth:`importlib.abc.Loader.load_module`
Brett Cannonad876c72009-03-09 07:53:09 +0000455 that loads Python source code. All needed information comes from the
456 abstract methods required by this ABC. The only pertinent assumption
457 made by this method is that when loading a package
458 :attr:`__path__` is set to ``[os.path.dirname(__file__)]``.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000459
Brett Cannon9c751b72009-03-09 16:28:16 +0000460 .. method:: get_code(fullname)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000461
462 A concrete implementation of
463 :meth:`importlib.abc.InspectLoader.get_code` that creates code objects
Guido van Rossum09613542009-03-30 20:34:57 +0000464 from Python source code, by requesting the source code (using
Benjamin Peterson0089d752009-11-13 00:52:43 +0000465 :meth:`source_path` and :meth:`get_data`) and compiling it with the
466 built-in :func:`compile` function.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000467
Brett Cannond43b30b2009-03-10 03:29:23 +0000468 .. method:: get_source(fullname)
469
470 A concrete implementation of
471 :meth:`importlib.abc.InspectLoader.get_source`. Uses
Brett Cannon69194272009-07-20 04:23:48 +0000472 :meth:`importlib.abc.ResourceLoader.get_data` and :meth:`source_path`
473 to get the source code. It tries to guess the source encoding using
Guido van Rossum09613542009-03-30 20:34:57 +0000474 :func:`tokenize.detect_encoding`.
Brett Cannond43b30b2009-03-10 03:29:23 +0000475
Brett Cannon2a922ed2009-03-09 03:35:50 +0000476
477.. class:: PyPycLoader
478
Brett Cannonf23e3742010-06-27 23:57:46 +0000479 An abstract base class inheriting from :class:`PyLoader`.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000480 This ABC is meant to help in creating loaders that support both Python
481 source and bytecode.
482
Brett Cannonf23e3742010-06-27 23:57:46 +0000483 .. deprecated:: 3.2
484 This class has been deprecated in favor of :class:`SourceLoader` and to
485 properly support :pep:`3147`. If compatibility is required with
486 Python 3.1, implement both :class:`SourceLoader` and :class:`PyLoader`;
487 instructions on how to do so are included in the documentation for
488 :class:`PyLoader`. Do note that this solution will not support
489 sourceless/bytecode-only loading; only source *and* bytecode loading.
490
Brett Cannon1e331562012-07-02 14:35:34 -0400491 .. versionchanged:: 3.3
492 Updated to parse (but not use) the new source size field in bytecode
493 files when reading and to write out the field properly when writing.
494
Brett Cannon9c751b72009-03-09 16:28:16 +0000495 .. method:: source_mtime(fullname)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000496
497 An abstract method which returns the modification time for the source
498 code of the specified module. The modification time should be an
Georg Brandl375aec22011-01-15 17:03:02 +0000499 integer. If there is no source code, return ``None``. If the
Brett Cannon2a922ed2009-03-09 03:35:50 +0000500 module cannot be found then :exc:`ImportError` is raised.
501
Brett Cannon9c751b72009-03-09 16:28:16 +0000502 .. method:: bytecode_path(fullname)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000503
504 An abstract method which returns the path to the bytecode for the
Georg Brandl375aec22011-01-15 17:03:02 +0000505 specified module, if it exists. It returns ``None``
Guido van Rossum09613542009-03-30 20:34:57 +0000506 if no bytecode exists (yet).
Brett Cannon3e761a92009-12-10 00:24:21 +0000507 Raises :exc:`ImportError` if the loader knows it cannot handle the
508 module.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000509
Brett Cannon69194272009-07-20 04:23:48 +0000510 .. method:: get_filename(fullname)
511
512 A concrete implementation of
Brett Cannonf23e3742010-06-27 23:57:46 +0000513 :meth:`ExecutionLoader.get_filename` that relies on
514 :meth:`PyLoader.source_path` and :meth:`bytecode_path`.
Brett Cannon69194272009-07-20 04:23:48 +0000515 If :meth:`source_path` returns a path, then that value is returned.
516 Else if :meth:`bytecode_path` returns a path, that path will be
517 returned. If a path is not available from both methods,
518 :exc:`ImportError` is raised.
519
Brett Cannon9c751b72009-03-09 16:28:16 +0000520 .. method:: write_bytecode(fullname, bytecode)
Brett Cannon2a922ed2009-03-09 03:35:50 +0000521
522 An abstract method which has the loader write *bytecode* for future
Georg Brandl375aec22011-01-15 17:03:02 +0000523 use. If the bytecode is written, return ``True``. Return
524 ``False`` if the bytecode could not be written. This method
Brett Cannon2a922ed2009-03-09 03:35:50 +0000525 should not be called if :data:`sys.dont_write_bytecode` is true.
Guido van Rossum09613542009-03-30 20:34:57 +0000526 The *bytecode* argument should be a bytes string or bytes array.
Brett Cannon2a922ed2009-03-09 03:35:50 +0000527
528
Brett Cannon78246b62009-01-25 04:56:30 +0000529:mod:`importlib.machinery` -- Importers and path hooks
530------------------------------------------------------
531
532.. module:: importlib.machinery
533 :synopsis: Importers and path hooks
534
535This module contains the various objects that help :keyword:`import`
536find and load modules.
537
Brett Cannoncb66eb02012-05-11 12:58:42 -0400538.. attribute:: SOURCE_SUFFIXES
539
540 A list of strings representing the recognized file suffixes for source
541 modules.
542
543 .. versionadded:: 3.3
544
545.. attribute:: DEBUG_BYTECODE_SUFFIXES
546
547 A list of strings representing the file suffixes for non-optimized bytecode
548 modules.
549
550 .. versionadded:: 3.3
551
552.. attribute:: OPTIMIZED_BYTECODE_SUFFIXES
553
554 A list of strings representing the file suffixes for optimized bytecode
555 modules.
556
557 .. versionadded:: 3.3
558
559.. attribute:: BYTECODE_SUFFIXES
560
561 A list of strings representing the recognized file suffixes for bytecode
562 modules. Set to either :attr:`DEBUG_BYTECODE_SUFFIXES` or
563 :attr:`OPTIMIZED_BYTECODE_SUFFIXES` based on whether ``__debug__`` is true.
564
565 .. versionadded:: 3.3
566
567.. attribute:: EXTENSION_SUFFIXES
568
Nick Coghlan76e07702012-07-18 23:14:57 +1000569 A list of strings representing the recognized file suffixes for
Brett Cannoncb66eb02012-05-11 12:58:42 -0400570 extension modules.
571
572 .. versionadded:: 3.3
573
Nick Coghlanc5afd422012-07-18 23:59:08 +1000574.. function:: all_suffixes()
Nick Coghlan76e07702012-07-18 23:14:57 +1000575
576 Returns a combined list of strings representing all file suffixes for
Nick Coghlanc5afd422012-07-18 23:59:08 +1000577 modules recognized by the standard import machinery. This is a
Nick Coghlan76e07702012-07-18 23:14:57 +1000578 helper for code which simply needs to know if a filesystem path
Nick Coghlanc5afd422012-07-18 23:59:08 +1000579 potentially refers to a module without needing any details on the kind
580 of module (for example, :func:`inspect.getmodulename`)
Nick Coghlan76e07702012-07-18 23:14:57 +1000581
582 .. versionadded:: 3.3
583
584
Brett Cannon78246b62009-01-25 04:56:30 +0000585.. class:: BuiltinImporter
586
Brett Cannon2a922ed2009-03-09 03:35:50 +0000587 An :term:`importer` for built-in modules. All known built-in modules are
588 listed in :data:`sys.builtin_module_names`. This class implements the
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000589 :class:`importlib.abc.MetaPathFinder` and
590 :class:`importlib.abc.InspectLoader` ABCs.
Brett Cannon78246b62009-01-25 04:56:30 +0000591
592 Only class methods are defined by this class to alleviate the need for
593 instantiation.
594
Brett Cannon78246b62009-01-25 04:56:30 +0000595
596.. class:: FrozenImporter
597
Brett Cannon2a922ed2009-03-09 03:35:50 +0000598 An :term:`importer` for frozen modules. This class implements the
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000599 :class:`importlib.abc.MetaPathFinder` and
600 :class:`importlib.abc.InspectLoader` ABCs.
Brett Cannon78246b62009-01-25 04:56:30 +0000601
602 Only class methods are defined by this class to alleviate the need for
603 instantiation.
604
Brett Cannondebb98d2009-02-16 04:18:01 +0000605
Nick Coghlanff794862012-08-02 21:45:24 +1000606.. class:: WindowsRegistryFinder
607
608 :term:`Finder` for modules declared in the Windows registry. This class
609 implements the :class:`importlib.abc.MetaPathFinder` ABC.
610
611 Only class methods are defined by this class to alleviate the need for
612 instantiation.
613
614 .. versionadded:: 3.3
615
616
Brett Cannondebb98d2009-02-16 04:18:01 +0000617.. class:: PathFinder
618
Brett Cannon2a922ed2009-03-09 03:35:50 +0000619 :term:`Finder` for :data:`sys.path`. This class implements the
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000620 :class:`importlib.abc.MetaPathFinder` ABC.
Brett Cannondebb98d2009-02-16 04:18:01 +0000621
622 This class does not perfectly mirror the semantics of :keyword:`import` in
623 terms of :data:`sys.path`. No implicit path hooks are assumed for
Brett Cannon83ac0132012-03-02 12:10:48 -0500624 simplification of the class and its semantics. This implies that when
625 ``None`` is found in :data:`sys.path_importer_cache` that it is simply
626 ignored instead of implying a default finder.
Brett Cannondebb98d2009-02-16 04:18:01 +0000627
Brett Cannon1b184d52009-11-07 08:22:58 +0000628 Only class methods are defined by this class to alleviate the need for
Brett Cannondebb98d2009-02-16 04:18:01 +0000629 instantiation.
630
631 .. classmethod:: find_module(fullname, path=None)
632
633 Class method that attempts to find a :term:`loader` for the module
Brett Cannon2a922ed2009-03-09 03:35:50 +0000634 specified by *fullname* on :data:`sys.path` or, if defined, on
Brett Cannondebb98d2009-02-16 04:18:01 +0000635 *path*. For each path entry that is searched,
Brett Cannon75321e82012-03-02 11:58:25 -0500636 :data:`sys.path_importer_cache` is checked. If a non-false object is
Brett Cannon2a922ed2009-03-09 03:35:50 +0000637 found then it is used as the :term:`finder` to look for the module
638 being searched for. If no entry is found in
Brett Cannondebb98d2009-02-16 04:18:01 +0000639 :data:`sys.path_importer_cache`, then :data:`sys.path_hooks` is
640 searched for a finder for the path entry and, if found, is stored in
641 :data:`sys.path_importer_cache` along with being queried about the
Georg Brandl375aec22011-01-15 17:03:02 +0000642 module. If no finder is ever found then ``None`` is returned.
Brett Cannond2e7b332009-02-17 02:45:03 +0000643
644
Brett Cannon938d44d2012-04-22 19:58:33 -0400645.. class:: FileFinder(path, \*loader_details)
646
Nick Coghlan8a9080f2012-08-02 21:26:03 +1000647 A concrete implementation of :class:`importlib.abc.PathEntryFinder` which
648 caches results from the file system.
Brett Cannon938d44d2012-04-22 19:58:33 -0400649
650 The *path* argument is the directory for which the finder is in charge of
651 searching.
652
653 The *loader_details* argument is a variable number of 3-item tuples each
654 containing a loader, file suffixes the loader recognizes, and a boolean
655 representing whether the loader handles packages.
656
657 The finder will cache the directory contents as necessary, making stat calls
658 for each module search to verify the cache is not outdated. Because cache
659 staleness relies upon the granularity of the operating system's state
660 information of the file system, there is a potential race condition of
661 searching for a module, creating a new file, and then searching for the
662 module the new file represents. If the operations happen fast enough to fit
663 within the granularity of stat calls, then the module search will fail. To
664 prevent this from happening, when you create a module dynamically, make sure
665 to call :func:`importlib.invalidate_caches`.
666
667 .. versionadded:: 3.3
668
669 .. attribute:: path
670
671 The path the finder will search in.
672
673 .. method:: find_module(fullname)
674
675 Attempt to find the loader to handle *fullname* within :attr:`path`.
676
677 .. method:: invalidate_caches()
678
679 Clear out the internal cache.
680
681 .. classmethod:: path_hook(\*loader_details)
682
683 A class method which returns a closure for use on :attr:`sys.path_hooks`.
684 An instance of :class:`FileFinder` is returned by the closure using the
685 path argument given to the closure directly and *loader_details*
686 indirectly.
687
688 If the argument to the closure is not an existing directory,
689 :exc:`ImportError` is raised.
690
691
692.. class:: SourceFileLoader(fullname, path)
693
694 A concrete implementation of :class:`importlib.abc.SourceLoader` by
695 subclassing :class:`importlib.abc.FileLoader` and providing some concrete
696 implementations of other methods.
697
698 .. versionadded:: 3.3
699
700 .. attribute:: name
701
702 The name of the module that this loader will handle.
703
704 .. attribute:: path
705
706 The path to the source file.
707
708 .. method:: is_package(fullname)
709
710 Return true if :attr:`path` appears to be for a package.
711
712 .. method:: path_stats(path)
713
714 Concrete implementation of :meth:`importlib.abc.SourceLoader.path_stats`.
715
716 .. method:: set_data(path, data)
717
718 Concrete implementation of :meth:`importlib.abc.SourceLoader.set_data`.
719
Brett Cannon938d44d2012-04-22 19:58:33 -0400720
Marc-Andre Lemburg4fe29c92012-04-25 02:31:37 +0200721.. class:: SourcelessFileLoader(fullname, path)
Brett Cannon938d44d2012-04-22 19:58:33 -0400722
723 A concrete implementation of :class:`importlib.abc.FileLoader` which can
724 import bytecode files (i.e. no source code files exist).
725
Marc-Andre Lemburg4fe29c92012-04-25 02:31:37 +0200726 Please note that direct use of bytecode files (and thus not source code
727 files) inhibits your modules from being usable by all Python
728 implementations or new versions of Python which change the bytecode
729 format.
Brett Cannon938d44d2012-04-22 19:58:33 -0400730
731 .. versionadded:: 3.3
732
733 .. attribute:: name
734
735 The name of the module the loader will handle.
736
737 .. attribute:: path
738
739 The path to the bytecode file.
740
741 .. method:: is_package(fullname)
742
743 Determines if the module is a package based on :attr:`path`.
744
745 .. method:: get_code(fullname)
746
747 Returns the code object for :attr:`name` created from :attr:`path`.
748
749 .. method:: get_source(fullname)
750
751 Returns ``None`` as bytecode files have no source when this loader is
752 used.
753
Brett Cannon938d44d2012-04-22 19:58:33 -0400754
755.. class:: ExtensionFileLoader(fullname, path)
756
757 A concrete implementation of :class:`importlib.abc.InspectLoader` for
758 extension modules.
759
760 The *fullname* argument specifies the name of the module the loader is to
761 support. The *path* argument is the path to the extension module's file.
762
763 .. versionadded:: 3.3
764
765 .. attribute:: name
766
767 Name of the module the loader supports.
768
769 .. attribute:: path
770
771 Path to the extension module.
772
Barry Warsawd7d21942012-07-29 16:36:17 -0400773 .. method:: load_module(fullname)
Brett Cannon938d44d2012-04-22 19:58:33 -0400774
Brett Cannonc0499522012-05-11 14:48:41 -0400775 Loads the extension module if and only if *fullname* is the same as
776 :attr:`name` or is ``None``.
Brett Cannon938d44d2012-04-22 19:58:33 -0400777
778 .. method:: is_package(fullname)
779
780 Returns ``False`` as extension modules can never be packages.
781
782 .. method:: get_code(fullname)
783
784 Returns ``None`` as extension modules lack a code object.
785
786 .. method:: get_source(fullname)
787
788 Returns ``None`` as extension modules do not have source code.
789
790
Brett Cannond2e7b332009-02-17 02:45:03 +0000791:mod:`importlib.util` -- Utility code for importers
792---------------------------------------------------
793
794.. module:: importlib.util
Brett Cannon75321e82012-03-02 11:58:25 -0500795 :synopsis: Utility code for importers
Brett Cannond2e7b332009-02-17 02:45:03 +0000796
797This module contains the various objects that help in the construction of
798an :term:`importer`.
799
Brett Cannond200bf52012-05-13 13:45:09 -0400800.. function:: resolve_name(name, package)
801
802 Resolve a relative module name to an absolute one.
803
804 If **name** has no leading dots, then **name** is simply returned. This
805 allows for usage such as
806 ``importlib.util.resolve_name('sys', __package__)`` without doing a
807 check to see if the **package** argument is needed.
808
809 :exc:`ValueError` is raised if **name** is a relative module name but
810 package is a false value (e.g. ``None`` or the empty string).
811 :exc:`ValueError` is also raised a relative name would escape its containing
812 package (e.g. requesting ``..bacon`` from within the ``spam`` package).
813
814 .. versionadded:: 3.3
815
Georg Brandl8a1caa22010-07-29 16:01:11 +0000816.. decorator:: module_for_loader
Brett Cannond2e7b332009-02-17 02:45:03 +0000817
Guido van Rossum09613542009-03-30 20:34:57 +0000818 A :term:`decorator` for a :term:`loader` method,
819 to handle selecting the proper
Brett Cannond2e7b332009-02-17 02:45:03 +0000820 module object to load with. The decorated method is expected to have a call
Brett Cannon2a922ed2009-03-09 03:35:50 +0000821 signature taking two positional arguments
822 (e.g. ``load_module(self, module)``) for which the second argument
Guido van Rossum09613542009-03-30 20:34:57 +0000823 will be the module **object** to be used by the loader.
Brett Cannonefad00d2012-04-27 17:27:14 -0400824 Note that the decorator will not work on static methods because of the
825 assumption of two arguments.
Brett Cannond2e7b332009-02-17 02:45:03 +0000826
Guido van Rossum09613542009-03-30 20:34:57 +0000827 The decorated method will take in the **name** of the module to be loaded
828 as expected for a :term:`loader`. If the module is not found in
Brett Cannon57b46f52009-03-02 14:38:26 +0000829 :data:`sys.modules` then a new one is constructed with its
Brett Cannonefad00d2012-04-27 17:27:14 -0400830 :attr:`__name__` attribute set to **name**, :attr:`__loader__` set to
831 **self**, and :attr:`__package__` set if
832 :meth:`importlib.abc.InspectLoader.is_package` is defined for **self** and
833 does not raise :exc:`ImportError` for **name**. If a new module is not
834 needed then the module found in :data:`sys.modules` will be passed into the
835 method.
836
837 If an exception is raised by the decorated method and a module was added to
Brett Cannond2e7b332009-02-17 02:45:03 +0000838 :data:`sys.modules` it will be removed to prevent a partially initialized
Brett Cannon57b46f52009-03-02 14:38:26 +0000839 module from being in left in :data:`sys.modules`. If the module was already
840 in :data:`sys.modules` then it is left alone.
Brett Cannond2e7b332009-02-17 02:45:03 +0000841
Guido van Rossum09613542009-03-30 20:34:57 +0000842 Use of this decorator handles all the details of which module object a
Brett Cannonefad00d2012-04-27 17:27:14 -0400843 loader should initialize as specified by :pep:`302` as best as possible.
844
845 .. versionchanged:: 3.3
Georg Brandl61063cc2012-06-24 22:48:30 +0200846 :attr:`__loader__` and :attr:`__package__` are automatically set
847 (when possible).
Brett Cannon57b46f52009-03-02 14:38:26 +0000848
Georg Brandl8a1caa22010-07-29 16:01:11 +0000849.. decorator:: set_loader
Brett Cannon2cf03a82009-03-10 05:17:37 +0000850
Guido van Rossum09613542009-03-30 20:34:57 +0000851 A :term:`decorator` for a :term:`loader` method,
852 to set the :attr:`__loader__`
Brett Cannon2cf03a82009-03-10 05:17:37 +0000853 attribute on loaded modules. If the attribute is already set the decorator
854 does nothing. It is assumed that the first positional argument to the
Brett Cannon75321e82012-03-02 11:58:25 -0500855 wrapped method (i.e. ``self``) is what :attr:`__loader__` should be set to.
Brett Cannon2cf03a82009-03-10 05:17:37 +0000856
Brett Cannonefad00d2012-04-27 17:27:14 -0400857 .. note::
858
859 It is recommended that :func:`module_for_loader` be used over this
860 decorator as it subsumes this functionality.
861
862
Georg Brandl8a1caa22010-07-29 16:01:11 +0000863.. decorator:: set_package
Brett Cannon57b46f52009-03-02 14:38:26 +0000864
865 A :term:`decorator` for a :term:`loader` to set the :attr:`__package__`
866 attribute on the module returned by the loader. If :attr:`__package__` is
Georg Brandl375aec22011-01-15 17:03:02 +0000867 set and has a value other than ``None`` it will not be changed.
Brett Cannon57b46f52009-03-02 14:38:26 +0000868 Note that the module returned by the loader is what has the attribute
869 set on and not the module found in :data:`sys.modules`.
Guido van Rossum09613542009-03-30 20:34:57 +0000870
Brett Cannon16248a42009-04-01 20:47:14 +0000871 Reliance on this decorator is discouraged when it is possible to set
Brett Cannon75321e82012-03-02 11:58:25 -0500872 :attr:`__package__` before importing. By
873 setting it beforehand the code for the module is executed with the
874 attribute set and thus can be used by global level code during
Brett Cannon16248a42009-04-01 20:47:14 +0000875 initialization.
876
Brett Cannonefad00d2012-04-27 17:27:14 -0400877 .. note::
878
879 It is recommended that :func:`module_for_loader` be used over this
880 decorator as it subsumes this functionality.