Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1 | :mod:`imp` --- Access the :keyword:`import` internals |
| 2 | ===================================================== |
| 3 | |
| 4 | .. module:: imp |
| 5 | :synopsis: Access the implementation of the import statement. |
| 6 | |
| 7 | |
| 8 | .. index:: statement: import |
| 9 | |
| 10 | This module provides an interface to the mechanisms used to implement the |
| 11 | :keyword:`import` statement. It defines the following constants and functions: |
| 12 | |
| 13 | |
| 14 | .. function:: get_magic() |
| 15 | |
| 16 | .. index:: pair: file; byte-code |
| 17 | |
| 18 | Return the magic string value used to recognize byte-compiled code files |
| 19 | (:file:`.pyc` files). (This value may be different for each Python version.) |
| 20 | |
| 21 | |
| 22 | .. function:: get_suffixes() |
| 23 | |
Guido van Rossum | 04110fb | 2007-08-24 16:32:05 +0000 | [diff] [blame] | 24 | Return a list of 3-element tuples, each describing a particular type of |
| 25 | module. Each triple has the form ``(suffix, mode, type)``, where *suffix* is |
| 26 | a string to be appended to the module name to form the filename to search |
| 27 | for, *mode* is the mode string to pass to the built-in :func:`open` function |
| 28 | to open the file (this can be ``'r'`` for text files or ``'rb'`` for binary |
| 29 | files), and *type* is the file type, which has one of the values |
| 30 | :const:`PY_SOURCE`, :const:`PY_COMPILED`, or :const:`C_EXTENSION`, described |
| 31 | below. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 32 | |
Brett Cannon | cb66eb0 | 2012-05-11 12:58:42 -0400 | [diff] [blame] | 33 | .. deprecated:: 3.3 |
| 34 | Use the constants defined on :mod:`importlib.machinery` instead. |
| 35 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 36 | |
| 37 | .. function:: find_module(name[, path]) |
| 38 | |
Alexandre Vassalotti | 711ed4a | 2009-07-17 10:42:05 +0000 | [diff] [blame] | 39 | Try to find the module *name*. If *path* is omitted or ``None``, the list of |
| 40 | directory names given by ``sys.path`` is searched, but first a few special |
| 41 | places are searched: the function tries to find a built-in module with the |
| 42 | given name (:const:`C_BUILTIN`), then a frozen module (:const:`PY_FROZEN`), |
| 43 | and on some systems some other places are looked in as well (on Windows, it |
| 44 | looks in the registry which may point to a specific file). |
| 45 | |
| 46 | Otherwise, *path* must be a list of directory names; each directory is |
| 47 | searched for files with any of the suffixes returned by :func:`get_suffixes` |
| 48 | above. Invalid names in the list are silently ignored (but all list items |
| 49 | must be strings). |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 50 | |
Guido van Rossum | 04110fb | 2007-08-24 16:32:05 +0000 | [diff] [blame] | 51 | If search is successful, the return value is a 3-element tuple ``(file, |
| 52 | pathname, description)``: |
| 53 | |
Antoine Pitrou | 11cb961 | 2010-09-15 11:11:28 +0000 | [diff] [blame] | 54 | *file* is an open :term:`file object` positioned at the beginning, *pathname* |
| 55 | is the pathname of the file found, and *description* is a 3-element tuple as |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 56 | contained in the list returned by :func:`get_suffixes` describing the kind of |
Guido van Rossum | 04110fb | 2007-08-24 16:32:05 +0000 | [diff] [blame] | 57 | module found. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 58 | |
Guido van Rossum | 04110fb | 2007-08-24 16:32:05 +0000 | [diff] [blame] | 59 | If the module does not live in a file, the returned *file* is ``None``, |
| 60 | *pathname* is the empty string, and the *description* tuple contains empty |
| 61 | strings for its suffix and mode; the module type is indicated as given in |
| 62 | parentheses above. If the search is unsuccessful, :exc:`ImportError` is |
| 63 | raised. Other exceptions indicate problems with the arguments or |
| 64 | environment. |
| 65 | |
| 66 | If the module is a package, *file* is ``None``, *pathname* is the package |
| 67 | path and the last item in the *description* tuple is :const:`PKG_DIRECTORY`. |
| 68 | |
| 69 | This function does not handle hierarchical module names (names containing |
Senthil Kumaran | cc49790 | 2012-04-10 19:51:00 +0800 | [diff] [blame] | 70 | dots). In order to find *P.M*, that is, submodule *M* of package *P*, use |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 71 | :func:`find_module` and :func:`load_module` to find and load package *P*, and |
| 72 | then use :func:`find_module` with the *path* argument set to ``P.__path__``. |
| 73 | When *P* itself has a dotted name, apply this recipe recursively. |
| 74 | |
Brett Cannon | 62961dd | 2012-05-13 13:04:21 -0400 | [diff] [blame] | 75 | .. deprecated:: 3.3 |
| 76 | Use :func:`importlib.find_loader` instead. |
| 77 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 78 | |
Guido van Rossum | 04110fb | 2007-08-24 16:32:05 +0000 | [diff] [blame] | 79 | .. function:: load_module(name, file, pathname, description) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 80 | |
| 81 | Load a module that was previously found by :func:`find_module` (or by an |
| 82 | otherwise conducted search yielding compatible results). This function does |
| 83 | more than importing the module: if the module was already imported, it will |
Guido van Rossum | 04110fb | 2007-08-24 16:32:05 +0000 | [diff] [blame] | 84 | reload the module! The *name* argument indicates the full |
| 85 | module name (including the package name, if this is a submodule of a |
| 86 | package). The *file* argument is an open file, and *pathname* is the |
| 87 | corresponding file name; these can be ``None`` and ``''``, respectively, when |
| 88 | the module is a package or not being loaded from a file. The *description* |
| 89 | argument is a tuple, as would be returned by :func:`get_suffixes`, describing |
| 90 | what kind of module must be loaded. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 91 | |
Guido van Rossum | 04110fb | 2007-08-24 16:32:05 +0000 | [diff] [blame] | 92 | If the load is successful, the return value is the module object; otherwise, |
| 93 | an exception (usually :exc:`ImportError`) is raised. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 94 | |
Guido van Rossum | 04110fb | 2007-08-24 16:32:05 +0000 | [diff] [blame] | 95 | **Important:** the caller is responsible for closing the *file* argument, if |
| 96 | it was not ``None``, even when an exception is raised. This is best done |
| 97 | using a :keyword:`try` ... :keyword:`finally` statement. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 98 | |
Brett Cannon | 62961dd | 2012-05-13 13:04:21 -0400 | [diff] [blame] | 99 | .. deprecated:: 3.3 |
| 100 | Unneeded as loaders should be used to load modules and |
| 101 | :func:`find_module` is deprecated. |
| 102 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 103 | |
| 104 | .. function:: new_module(name) |
| 105 | |
| 106 | Return a new empty module object called *name*. This object is *not* inserted |
| 107 | in ``sys.modules``. |
| 108 | |
| 109 | |
Christian Heimes | 043d6f6 | 2008-01-07 17:19:16 +0000 | [diff] [blame] | 110 | .. function:: reload(module) |
| 111 | |
| 112 | Reload a previously imported *module*. The argument must be a module object, so |
| 113 | it must have been successfully imported before. This is useful if you have |
| 114 | edited the module source file using an external editor and want to try out the |
| 115 | new version without leaving the Python interpreter. The return value is the |
| 116 | module object (the same as the *module* argument). |
| 117 | |
| 118 | When ``reload(module)`` is executed: |
| 119 | |
| 120 | * Python modules' code is recompiled and the module-level code reexecuted, |
| 121 | defining a new set of objects which are bound to names in the module's |
| 122 | dictionary. The ``init`` function of extension modules is not called a second |
| 123 | time. |
| 124 | |
| 125 | * As with all other objects in Python the old objects are only reclaimed after |
| 126 | their reference counts drop to zero. |
| 127 | |
| 128 | * The names in the module namespace are updated to point to any new or changed |
| 129 | objects. |
| 130 | |
| 131 | * Other references to the old objects (such as names external to the module) are |
| 132 | not rebound to refer to the new objects and must be updated in each namespace |
| 133 | where they occur if that is desired. |
| 134 | |
| 135 | There are a number of other caveats: |
| 136 | |
| 137 | If a module is syntactically correct but its initialization fails, the first |
| 138 | :keyword:`import` statement for it does not bind its name locally, but does |
| 139 | store a (partially initialized) module object in ``sys.modules``. To reload the |
| 140 | module you must first :keyword:`import` it again (this will bind the name to the |
| 141 | partially initialized module object) before you can :func:`reload` it. |
| 142 | |
| 143 | When a module is reloaded, its dictionary (containing the module's global |
| 144 | variables) is retained. Redefinitions of names will override the old |
| 145 | definitions, so this is generally not a problem. If the new version of a module |
| 146 | does not define a name that was defined by the old version, the old definition |
| 147 | remains. This feature can be used to the module's advantage if it maintains a |
| 148 | global table or cache of objects --- with a :keyword:`try` statement it can test |
| 149 | for the table's presence and skip its initialization if desired:: |
| 150 | |
| 151 | try: |
| 152 | cache |
| 153 | except NameError: |
| 154 | cache = {} |
| 155 | |
| 156 | It is legal though generally not very useful to reload built-in or dynamically |
| 157 | loaded modules, except for :mod:`sys`, :mod:`__main__` and :mod:`__builtin__`. |
| 158 | In many cases, however, extension modules are not designed to be initialized |
| 159 | more than once, and may fail in arbitrary ways when reloaded. |
| 160 | |
| 161 | If a module imports objects from another module using :keyword:`from` ... |
| 162 | :keyword:`import` ..., calling :func:`reload` for the other module does not |
| 163 | redefine the objects imported from it --- one way around this is to re-execute |
| 164 | the :keyword:`from` statement, another is to use :keyword:`import` and qualified |
| 165 | names (*module*.*name*) instead. |
| 166 | |
| 167 | If a module instantiates instances of a class, reloading the module that defines |
| 168 | the class does not affect the method definitions of the instances --- they |
| 169 | continue to use the old class definition. The same is true for derived classes. |
| 170 | |
| 171 | |
Éric Araujo | 930df31 | 2010-12-16 06:28:48 +0000 | [diff] [blame] | 172 | The following functions are conveniences for handling :pep:`3147` byte-compiled |
| 173 | file paths. |
Barry Warsaw | 28a691b | 2010-04-17 00:19:56 +0000 | [diff] [blame] | 174 | |
| 175 | .. versionadded:: 3.2 |
| 176 | |
| 177 | .. function:: cache_from_source(path, debug_override=None) |
| 178 | |
Victor Stinner | 766ad36 | 2010-05-14 14:36:18 +0000 | [diff] [blame] | 179 | Return the :pep:`3147` path to the byte-compiled file associated with the |
Barry Warsaw | 28a691b | 2010-04-17 00:19:56 +0000 | [diff] [blame] | 180 | source *path*. For example, if *path* is ``/foo/bar/baz.py`` the return |
| 181 | value would be ``/foo/bar/__pycache__/baz.cpython-32.pyc`` for Python 3.2. |
| 182 | The ``cpython-32`` string comes from the current magic tag (see |
Brett Cannon | 19a2f59 | 2012-07-09 13:58:07 -0400 | [diff] [blame] | 183 | :func:`get_tag`; if :attr:`sys.implementation.cache_tag` is not defined then |
| 184 | :exc:`NotImplementedError` will be raised). The returned path will end in |
| 185 | ``.pyc`` when ``__debug__`` is True or ``.pyo`` for an optimized Python |
Barry Warsaw | 28a691b | 2010-04-17 00:19:56 +0000 | [diff] [blame] | 186 | (i.e. ``__debug__`` is False). By passing in True or False for |
| 187 | *debug_override* you can override the system's value for ``__debug__`` for |
| 188 | extension selection. |
| 189 | |
| 190 | *path* need not exist. |
| 191 | |
Brett Cannon | 19a2f59 | 2012-07-09 13:58:07 -0400 | [diff] [blame] | 192 | .. versionchanged:: 3.3 |
| 193 | If :attr:`sys.implementation.cache_tag` is ``None``, then |
| 194 | :exc:`NotImplementedError` is raised. |
| 195 | |
Benjamin Peterson | 0f4dd9a | 2010-09-13 01:31:57 +0000 | [diff] [blame] | 196 | |
Barry Warsaw | 28a691b | 2010-04-17 00:19:56 +0000 | [diff] [blame] | 197 | .. function:: source_from_cache(path) |
| 198 | |
Victor Stinner | 766ad36 | 2010-05-14 14:36:18 +0000 | [diff] [blame] | 199 | Given the *path* to a :pep:`3147` file name, return the associated source code |
Barry Warsaw | 28a691b | 2010-04-17 00:19:56 +0000 | [diff] [blame] | 200 | file path. For example, if *path* is |
| 201 | ``/foo/bar/__pycache__/baz.cpython-32.pyc`` the returned path would be |
| 202 | ``/foo/bar/baz.py``. *path* need not exist, however if it does not conform |
Brett Cannon | 19a2f59 | 2012-07-09 13:58:07 -0400 | [diff] [blame] | 203 | to :pep:`3147` format, a ``ValueError`` is raised. If |
| 204 | :attr:`sys.implementation.cache_tag` is not defined, |
| 205 | :exc:`NotImplementedError` is raised. |
| 206 | |
| 207 | .. versionchanged:: 3.3 |
| 208 | Raise :exc:`NotImplementedError` when |
| 209 | :attr:`sys.implementation.cache_tag` is not defined. |
Barry Warsaw | 28a691b | 2010-04-17 00:19:56 +0000 | [diff] [blame] | 210 | |
Benjamin Peterson | 0f4dd9a | 2010-09-13 01:31:57 +0000 | [diff] [blame] | 211 | |
Barry Warsaw | 28a691b | 2010-04-17 00:19:56 +0000 | [diff] [blame] | 212 | .. function:: get_tag() |
| 213 | |
Victor Stinner | 766ad36 | 2010-05-14 14:36:18 +0000 | [diff] [blame] | 214 | Return the :pep:`3147` magic tag string matching this version of Python's |
Barry Warsaw | 28a691b | 2010-04-17 00:19:56 +0000 | [diff] [blame] | 215 | magic number, as returned by :func:`get_magic`. |
| 216 | |
Brett Cannon | 19a2f59 | 2012-07-09 13:58:07 -0400 | [diff] [blame] | 217 | .. note:: |
| 218 | You may use :attr:`sys.implementation.cache_tag` directly starting |
| 219 | in Python 3.3. |
| 220 | |
Barry Warsaw | 28a691b | 2010-04-17 00:19:56 +0000 | [diff] [blame] | 221 | |
Antoine Pitrou | 6c6d3a2 | 2012-05-17 19:00:35 +0200 | [diff] [blame] | 222 | The following functions help interact with the import system's internal |
| 223 | locking mechanism. Locking semantics of imports are an implementation |
| 224 | detail which may vary from release to release. However, Python ensures |
| 225 | that circular imports work without any deadlocks. |
| 226 | |
Antoine Pitrou | 6c6d3a2 | 2012-05-17 19:00:35 +0200 | [diff] [blame] | 227 | |
| 228 | .. function:: lock_held() |
| 229 | |
| 230 | Return ``True`` if the global import lock is currently held, else |
| 231 | ``False``. On platforms without threads, always return ``False``. |
| 232 | |
| 233 | On platforms with threads, a thread executing an import first holds a |
| 234 | global import lock, then sets up a per-module lock for the rest of the |
| 235 | import. This blocks other threads from importing the same module until |
| 236 | the original import completes, preventing other threads from seeing |
| 237 | incomplete module objects constructed by the original thread. An |
| 238 | exception is made for circular imports, which by construction have to |
| 239 | expose an incomplete module object at some point. |
| 240 | |
Brett Cannon | d104eef | 2012-07-13 11:26:19 -0400 | [diff] [blame] | 241 | .. versionchanged:: 3.3 |
| 242 | The locking scheme has changed to per-module locks for |
| 243 | the most part. A global import lock is kept for some critical tasks, |
| 244 | such as initializing the per-module locks. |
| 245 | |
| 246 | |
Antoine Pitrou | 6c6d3a2 | 2012-05-17 19:00:35 +0200 | [diff] [blame] | 247 | .. function:: acquire_lock() |
| 248 | |
| 249 | Acquire the interpreter's global import lock for the current thread. |
| 250 | This lock should be used by import hooks to ensure thread-safety when |
| 251 | importing modules. |
| 252 | |
| 253 | Once a thread has acquired the import lock, the same thread may acquire it |
| 254 | again without blocking; the thread must release it once for each time it has |
| 255 | acquired it. |
| 256 | |
| 257 | On platforms without threads, this function does nothing. |
| 258 | |
Brett Cannon | d104eef | 2012-07-13 11:26:19 -0400 | [diff] [blame] | 259 | .. versionchanged:: 3.3 |
| 260 | The locking scheme has changed to per-module locks for |
| 261 | the most part. A global import lock is kept for some critical tasks, |
| 262 | such as initializing the per-module locks. |
| 263 | |
Antoine Pitrou | 6c6d3a2 | 2012-05-17 19:00:35 +0200 | [diff] [blame] | 264 | |
| 265 | .. function:: release_lock() |
| 266 | |
| 267 | Release the interpreter's global import lock. On platforms without |
| 268 | threads, this function does nothing. |
| 269 | |
Brett Cannon | d104eef | 2012-07-13 11:26:19 -0400 | [diff] [blame] | 270 | .. versionchanged:: 3.3 |
| 271 | The locking scheme has changed to per-module locks for |
| 272 | the most part. A global import lock is kept for some critical tasks, |
| 273 | such as initializing the per-module locks. |
| 274 | |
Antoine Pitrou | 6c6d3a2 | 2012-05-17 19:00:35 +0200 | [diff] [blame] | 275 | |
Barry Warsaw | 28a691b | 2010-04-17 00:19:56 +0000 | [diff] [blame] | 276 | The following constants with integer values, defined in this module, are used |
| 277 | to indicate the search result of :func:`find_module`. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 278 | |
| 279 | |
| 280 | .. data:: PY_SOURCE |
| 281 | |
| 282 | The module was found as a source file. |
| 283 | |
Brett Cannon | 0c59b03 | 2012-05-11 14:27:29 -0400 | [diff] [blame] | 284 | .. deprecated:: 3.3 |
| 285 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 286 | |
| 287 | .. data:: PY_COMPILED |
| 288 | |
| 289 | The module was found as a compiled code object file. |
| 290 | |
Brett Cannon | 0c59b03 | 2012-05-11 14:27:29 -0400 | [diff] [blame] | 291 | .. deprecated:: 3.3 |
| 292 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 293 | |
| 294 | .. data:: C_EXTENSION |
| 295 | |
| 296 | The module was found as dynamically loadable shared library. |
| 297 | |
Brett Cannon | 0c59b03 | 2012-05-11 14:27:29 -0400 | [diff] [blame] | 298 | .. deprecated:: 3.3 |
| 299 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 300 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 301 | .. data:: PKG_DIRECTORY |
| 302 | |
| 303 | The module was found as a package directory. |
| 304 | |
Brett Cannon | 0c59b03 | 2012-05-11 14:27:29 -0400 | [diff] [blame] | 305 | .. deprecated:: 3.3 |
| 306 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 307 | |
| 308 | .. data:: C_BUILTIN |
| 309 | |
| 310 | The module was found as a built-in module. |
| 311 | |
Brett Cannon | 0c59b03 | 2012-05-11 14:27:29 -0400 | [diff] [blame] | 312 | .. deprecated:: 3.3 |
| 313 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 314 | |
| 315 | .. data:: PY_FROZEN |
| 316 | |
R David Murray | 1623aff | 2012-03-18 20:50:03 -0400 | [diff] [blame] | 317 | The module was found as a frozen module. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 318 | |
Brett Cannon | 0c59b03 | 2012-05-11 14:27:29 -0400 | [diff] [blame] | 319 | .. deprecated:: 3.3 |
| 320 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 321 | |
| 322 | .. class:: NullImporter(path_string) |
| 323 | |
| 324 | The :class:`NullImporter` type is a :pep:`302` import hook that handles |
| 325 | non-directory path strings by failing to find any modules. Calling this type |
| 326 | with an existing directory or empty string raises :exc:`ImportError`. |
| 327 | Otherwise, a :class:`NullImporter` instance is returned. |
| 328 | |
Brett Cannon | 115309a | 2013-03-13 10:58:50 -0700 | [diff] [blame^] | 329 | Instances have only one method: |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 330 | |
| 331 | .. method:: NullImporter.find_module(fullname [, path]) |
| 332 | |
| 333 | This method always returns ``None``, indicating that the requested module could |
| 334 | not be found. |
| 335 | |
Brett Cannon | 115309a | 2013-03-13 10:58:50 -0700 | [diff] [blame^] | 336 | .. versionchanged:: 3.3 |
| 337 | ``None`` is inserted into ``sys.path_importer_cache`` instead of an |
| 338 | instance of :class:`NullImporter`. |
| 339 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 340 | |
| 341 | .. _examples-imp: |
| 342 | |
| 343 | Examples |
| 344 | -------- |
| 345 | |
| 346 | The following function emulates what was the standard import statement up to |
| 347 | Python 1.4 (no hierarchical module names). (This *implementation* wouldn't work |
| 348 | in that version, since :func:`find_module` has been extended and |
| 349 | :func:`load_module` has been added in 1.4.) :: |
| 350 | |
| 351 | import imp |
| 352 | import sys |
| 353 | |
| 354 | def __import__(name, globals=None, locals=None, fromlist=None): |
| 355 | # Fast path: see if the module has already been imported. |
| 356 | try: |
| 357 | return sys.modules[name] |
| 358 | except KeyError: |
| 359 | pass |
| 360 | |
| 361 | # If any of the following calls raises an exception, |
| 362 | # there's a problem we can't handle -- let the caller handle it. |
| 363 | |
| 364 | fp, pathname, description = imp.find_module(name) |
| 365 | |
| 366 | try: |
| 367 | return imp.load_module(name, fp, pathname, description) |
| 368 | finally: |
| 369 | # Since we may exit via an exception, close fp explicitly. |
| 370 | if fp: |
| 371 | fp.close() |