Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1 | .. _tut-modules: |
| 2 | |
| 3 | ******* |
| 4 | Modules |
| 5 | ******* |
| 6 | |
| 7 | If you quit from the Python interpreter and enter it again, the definitions you |
| 8 | have made (functions and variables) are lost. Therefore, if you want to write a |
| 9 | somewhat longer program, you are better off using a text editor to prepare the |
| 10 | input for the interpreter and running it with that file as input instead. This |
| 11 | is known as creating a *script*. As your program gets longer, you may want to |
| 12 | split it into several files for easier maintenance. You may also want to use a |
| 13 | handy function that you've written in several programs without copying its |
| 14 | definition into each program. |
| 15 | |
| 16 | To support this, Python has a way to put definitions in a file and use them in a |
| 17 | script or in an interactive instance of the interpreter. Such a file is called a |
| 18 | *module*; definitions from a module can be *imported* into other modules or into |
| 19 | the *main* module (the collection of variables that you have access to in a |
| 20 | script executed at the top level and in calculator mode). |
| 21 | |
| 22 | A module is a file containing Python definitions and statements. The file name |
| 23 | is the module name with the suffix :file:`.py` appended. Within a module, the |
| 24 | module's name (as a string) is available as the value of the global variable |
| 25 | ``__name__``. For instance, use your favorite text editor to create a file |
| 26 | called :file:`fibo.py` in the current directory with the following contents:: |
| 27 | |
| 28 | # Fibonacci numbers module |
| 29 | |
| 30 | def fib(n): # write Fibonacci series up to n |
| 31 | a, b = 0, 1 |
| 32 | while b < n: |
Guido van Rossum | 0616b79 | 2007-08-31 03:25:11 +0000 | [diff] [blame] | 33 | print(b, end=' ') |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 34 | a, b = b, a+b |
Georg Brandl | 11e18b0 | 2008-08-05 09:04:16 +0000 | [diff] [blame] | 35 | print() |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 36 | |
| 37 | def fib2(n): # return Fibonacci series up to n |
| 38 | result = [] |
| 39 | a, b = 0, 1 |
| 40 | while b < n: |
| 41 | result.append(b) |
| 42 | a, b = b, a+b |
| 43 | return result |
| 44 | |
| 45 | Now enter the Python interpreter and import this module with the following |
| 46 | command:: |
| 47 | |
| 48 | >>> import fibo |
| 49 | |
| 50 | This does not enter the names of the functions defined in ``fibo`` directly in |
| 51 | the current symbol table; it only enters the module name ``fibo`` there. Using |
| 52 | the module name you can access the functions:: |
| 53 | |
| 54 | >>> fibo.fib(1000) |
| 55 | 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987 |
| 56 | >>> fibo.fib2(100) |
| 57 | [1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89] |
| 58 | >>> fibo.__name__ |
| 59 | 'fibo' |
| 60 | |
| 61 | If you intend to use a function often you can assign it to a local name:: |
| 62 | |
| 63 | >>> fib = fibo.fib |
| 64 | >>> fib(500) |
| 65 | 1 1 2 3 5 8 13 21 34 55 89 144 233 377 |
| 66 | |
| 67 | |
| 68 | .. _tut-moremodules: |
| 69 | |
| 70 | More on Modules |
| 71 | =============== |
| 72 | |
| 73 | A module can contain executable statements as well as function definitions. |
| 74 | These statements are intended to initialize the module. They are executed only |
R David Murray | 25187e6 | 2013-04-21 16:58:36 -0400 | [diff] [blame] | 75 | the *first* time the module name is encountered in an import statement. [#]_ |
| 76 | (They are also run if the file is executed as a script.) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 77 | |
| 78 | Each module has its own private symbol table, which is used as the global symbol |
| 79 | table by all functions defined in the module. Thus, the author of a module can |
| 80 | use global variables in the module without worrying about accidental clashes |
| 81 | with a user's global variables. On the other hand, if you know what you are |
| 82 | doing you can touch a module's global variables with the same notation used to |
| 83 | refer to its functions, ``modname.itemname``. |
| 84 | |
| 85 | Modules can import other modules. It is customary but not required to place all |
| 86 | :keyword:`import` statements at the beginning of a module (or script, for that |
| 87 | matter). The imported module names are placed in the importing module's global |
| 88 | symbol table. |
| 89 | |
| 90 | There is a variant of the :keyword:`import` statement that imports names from a |
| 91 | module directly into the importing module's symbol table. For example:: |
| 92 | |
| 93 | >>> from fibo import fib, fib2 |
| 94 | >>> fib(500) |
| 95 | 1 1 2 3 5 8 13 21 34 55 89 144 233 377 |
| 96 | |
| 97 | This does not introduce the module name from which the imports are taken in the |
| 98 | local symbol table (so in the example, ``fibo`` is not defined). |
| 99 | |
| 100 | There is even a variant to import all names that a module defines:: |
| 101 | |
| 102 | >>> from fibo import * |
| 103 | >>> fib(500) |
| 104 | 1 1 2 3 5 8 13 21 34 55 89 144 233 377 |
| 105 | |
| 106 | This imports all names except those beginning with an underscore (``_``). |
Georg Brandl | 48310cd | 2009-01-03 21:18:54 +0000 | [diff] [blame] | 107 | In most cases Python programmers do not use this facility since it introduces |
| 108 | an unknown set of names into the interpreter, possibly hiding some things |
Guido van Rossum | 0616b79 | 2007-08-31 03:25:11 +0000 | [diff] [blame] | 109 | you have already defined. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 110 | |
Benjamin Peterson | 4ac9ce4 | 2009-10-04 14:49:41 +0000 | [diff] [blame] | 111 | Note that in general the practice of importing ``*`` from a module or package is |
| 112 | frowned upon, since it often causes poorly readable code. However, it is okay to |
| 113 | use it to save typing in interactive sessions. |
| 114 | |
Alexandre Vassalotti | 6461e10 | 2008-05-15 22:09:29 +0000 | [diff] [blame] | 115 | .. note:: |
| 116 | |
| 117 | For efficiency reasons, each module is only imported once per interpreter |
| 118 | session. Therefore, if you change your modules, you must restart the |
| 119 | interpreter -- or, if it's just one module you want to test interactively, |
Georg Brandl | abffe71 | 2008-12-15 08:28:37 +0000 | [diff] [blame] | 120 | use :func:`imp.reload`, e.g. ``import imp; imp.reload(modulename)``. |
Alexandre Vassalotti | 6461e10 | 2008-05-15 22:09:29 +0000 | [diff] [blame] | 121 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 122 | |
| 123 | .. _tut-modulesasscripts: |
| 124 | |
| 125 | Executing modules as scripts |
| 126 | ---------------------------- |
| 127 | |
| 128 | When you run a Python module with :: |
| 129 | |
| 130 | python fibo.py <arguments> |
| 131 | |
| 132 | the code in the module will be executed, just as if you imported it, but with |
| 133 | the ``__name__`` set to ``"__main__"``. That means that by adding this code at |
| 134 | the end of your module:: |
| 135 | |
| 136 | if __name__ == "__main__": |
| 137 | import sys |
| 138 | fib(int(sys.argv[1])) |
| 139 | |
| 140 | you can make the file usable as a script as well as an importable module, |
| 141 | because the code that parses the command line only runs if the module is |
| 142 | executed as the "main" file:: |
| 143 | |
| 144 | $ python fibo.py 50 |
| 145 | 1 1 2 3 5 8 13 21 34 |
| 146 | |
| 147 | If the module is imported, the code is not run:: |
| 148 | |
| 149 | >>> import fibo |
| 150 | >>> |
| 151 | |
| 152 | This is often used either to provide a convenient user interface to a module, or |
| 153 | for testing purposes (running the module as a script executes a test suite). |
| 154 | |
| 155 | |
| 156 | .. _tut-searchpath: |
| 157 | |
| 158 | The Module Search Path |
| 159 | ---------------------- |
| 160 | |
| 161 | .. index:: triple: module; search; path |
| 162 | |
Sandro Tosi | f0229aa | 2012-01-19 11:29:26 +0100 | [diff] [blame] | 163 | When a module named :mod:`spam` is imported, the interpreter first searches for |
| 164 | a built-in module with that name. If not found, it then searches for a file |
| 165 | named :file:`spam.py` in a list of directories given by the variable |
| 166 | :data:`sys.path`. :data:`sys.path` is initialized from these locations: |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 167 | |
Brett Cannon | f811bbf | 2014-02-06 09:22:51 -0500 | [diff] [blame] | 168 | * The directory containing the input script (or the current directory when no |
| 169 | file is specified). |
Sandro Tosi | f0229aa | 2012-01-19 11:29:26 +0100 | [diff] [blame] | 170 | * :envvar:`PYTHONPATH` (a list of directory names, with the same syntax as the |
| 171 | shell variable :envvar:`PATH`). |
Brett Cannon | f811bbf | 2014-02-06 09:22:51 -0500 | [diff] [blame] | 172 | * The installation-dependent default. |
| 173 | |
| 174 | .. note:: |
| 175 | On file systems which support symlinks, the directory containing the input |
| 176 | script is calculated after the symlink is followed. In other words the |
| 177 | directory containing the symlink is **not** added to the module search path. |
Sandro Tosi | f0229aa | 2012-01-19 11:29:26 +0100 | [diff] [blame] | 178 | |
| 179 | After initialization, Python programs can modify :data:`sys.path`. The |
| 180 | directory containing the script being run is placed at the beginning of the |
| 181 | search path, ahead of the standard library path. This means that scripts in that |
| 182 | directory will be loaded instead of modules of the same name in the library |
| 183 | directory. This is an error unless the replacement is intended. See section |
| 184 | :ref:`tut-standardmodules` for more information. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 185 | |
Guido van Rossum | 0616b79 | 2007-08-31 03:25:11 +0000 | [diff] [blame] | 186 | .. % |
| 187 | Do we need stuff on zip files etc. ? DUBOIS |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 188 | |
| 189 | "Compiled" Python files |
| 190 | ----------------------- |
| 191 | |
Georg Brandl | 5db7c54 | 2013-10-12 19:13:23 +0200 | [diff] [blame] | 192 | To speed up loading modules, Python caches the compiled version of each module |
Georg Brandl | 325a1c2 | 2013-10-27 09:16:01 +0100 | [diff] [blame] | 193 | in the ``__pycache__`` directory under the name :file:`module.{version}.pyc`, |
Georg Brandl | 5db7c54 | 2013-10-12 19:13:23 +0200 | [diff] [blame] | 194 | where the version encodes the format of the compiled file; it generally contains |
| 195 | the Python version number. For example, in CPython release 3.3 the compiled |
| 196 | version of spam.py would be cached as ``__pycache__/spam.cpython-33.pyc``. This |
| 197 | naming convention allows compiled modules from different releases and different |
| 198 | versions of Python to coexist. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 199 | |
Georg Brandl | 5db7c54 | 2013-10-12 19:13:23 +0200 | [diff] [blame] | 200 | Python checks the modification date of the source against the compiled version |
| 201 | to see if it's out of date and needs to be recompiled. This is a completely |
| 202 | automatic process. Also, the compiled modules are platform-independent, so the |
| 203 | same library can be shared among systems with different architectures. |
| 204 | |
| 205 | Python does not check the cache in two circumstances. First, it always |
| 206 | recompiles and does not store the result for the module that's loaded directly |
| 207 | from the command line. Second, it does not check the cache if there is no |
| 208 | source module. To support a non-source (compiled only) distribution, the |
| 209 | compiled module must be in the source directory, and there must not be a source |
| 210 | module. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 211 | |
| 212 | Some tips for experts: |
| 213 | |
Georg Brandl | 5db7c54 | 2013-10-12 19:13:23 +0200 | [diff] [blame] | 214 | * You can use the :option:`-O` or :option:`-OO` switches on the Python command |
| 215 | to reduce the size of a compiled module. The ``-O`` switch removes assert |
| 216 | statements, the ``-OO`` switch removes both assert statements and __doc__ |
| 217 | strings. Since some programs may rely on having these available, you should |
| 218 | only use this option if you know what you're doing. "Optimized" modules have |
| 219 | a .pyo rather than a .pyc suffix and are usually smaller. Future releases may |
| 220 | change the effects of optimization. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 221 | |
Georg Brandl | 5db7c54 | 2013-10-12 19:13:23 +0200 | [diff] [blame] | 222 | * A program doesn't run any faster when it is read from a ``.pyc`` or ``.pyo`` |
| 223 | file than when it is read from a ``.py`` file; the only thing that's faster |
| 224 | about ``.pyc`` or ``.pyo`` files is the speed with which they are loaded. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 225 | |
Georg Brandl | 5db7c54 | 2013-10-12 19:13:23 +0200 | [diff] [blame] | 226 | * The module :mod:`compileall` can create .pyc files (or .pyo files when |
| 227 | :option:`-O` is used) for all modules in a directory. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 228 | |
Georg Brandl | 5db7c54 | 2013-10-12 19:13:23 +0200 | [diff] [blame] | 229 | * There is more detail on this process, including a flow chart of the |
| 230 | decisions, in PEP 3147. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 231 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 232 | |
| 233 | .. _tut-standardmodules: |
| 234 | |
| 235 | Standard Modules |
| 236 | ================ |
| 237 | |
| 238 | .. index:: module: sys |
| 239 | |
| 240 | Python comes with a library of standard modules, described in a separate |
| 241 | document, the Python Library Reference ("Library Reference" hereafter). Some |
| 242 | modules are built into the interpreter; these provide access to operations that |
| 243 | are not part of the core of the language but are nevertheless built in, either |
| 244 | for efficiency or to provide access to operating system primitives such as |
| 245 | system calls. The set of such modules is a configuration option which also |
Sandro Tosi | da9df92 | 2012-08-04 19:42:24 +0200 | [diff] [blame] | 246 | depends on the underlying platform. For example, the :mod:`winreg` module is only |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 247 | provided on Windows systems. One particular module deserves some attention: |
| 248 | :mod:`sys`, which is built into every Python interpreter. The variables |
| 249 | ``sys.ps1`` and ``sys.ps2`` define the strings used as primary and secondary |
Christian Heimes | 5b5e81c | 2007-12-31 16:14:33 +0000 | [diff] [blame] | 250 | prompts:: |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 251 | |
| 252 | >>> import sys |
| 253 | >>> sys.ps1 |
| 254 | '>>> ' |
| 255 | >>> sys.ps2 |
| 256 | '... ' |
| 257 | >>> sys.ps1 = 'C> ' |
Guido van Rossum | 0616b79 | 2007-08-31 03:25:11 +0000 | [diff] [blame] | 258 | C> print('Yuck!') |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 259 | Yuck! |
| 260 | C> |
| 261 | |
| 262 | |
| 263 | These two variables are only defined if the interpreter is in interactive mode. |
| 264 | |
| 265 | The variable ``sys.path`` is a list of strings that determines the interpreter's |
| 266 | search path for modules. It is initialized to a default path taken from the |
| 267 | environment variable :envvar:`PYTHONPATH`, or from a built-in default if |
| 268 | :envvar:`PYTHONPATH` is not set. You can modify it using standard list |
| 269 | operations:: |
| 270 | |
| 271 | >>> import sys |
| 272 | >>> sys.path.append('/ufs/guido/lib/python') |
| 273 | |
| 274 | |
| 275 | .. _tut-dir: |
| 276 | |
| 277 | The :func:`dir` Function |
| 278 | ======================== |
| 279 | |
| 280 | The built-in function :func:`dir` is used to find out which names a module |
| 281 | defines. It returns a sorted list of strings:: |
| 282 | |
| 283 | >>> import fibo, sys |
| 284 | >>> dir(fibo) |
| 285 | ['__name__', 'fib', 'fib2'] |
Ezio Melotti | 52e8550 | 2012-11-17 12:50:14 +0200 | [diff] [blame] | 286 | >>> dir(sys) # doctest: +NORMALIZE_WHITESPACE |
Ezio Melotti | ac6ca3d | 2012-11-17 12:56:29 +0200 | [diff] [blame] | 287 | ['__displayhook__', '__doc__', '__excepthook__', '__loader__', '__name__', |
| 288 | '__package__', '__stderr__', '__stdin__', '__stdout__', |
| 289 | '_clear_type_cache', '_current_frames', '_debugmallocstats', '_getframe', |
| 290 | '_home', '_mercurial', '_xoptions', 'abiflags', 'api_version', 'argv', |
| 291 | 'base_exec_prefix', 'base_prefix', 'builtin_module_names', 'byteorder', |
| 292 | 'call_tracing', 'callstats', 'copyright', 'displayhook', |
| 293 | 'dont_write_bytecode', 'exc_info', 'excepthook', 'exec_prefix', |
| 294 | 'executable', 'exit', 'flags', 'float_info', 'float_repr_style', |
| 295 | 'getcheckinterval', 'getdefaultencoding', 'getdlopenflags', |
| 296 | 'getfilesystemencoding', 'getobjects', 'getprofile', 'getrecursionlimit', |
| 297 | 'getrefcount', 'getsizeof', 'getswitchinterval', 'gettotalrefcount', |
| 298 | 'gettrace', 'hash_info', 'hexversion', 'implementation', 'int_info', |
| 299 | 'intern', 'maxsize', 'maxunicode', 'meta_path', 'modules', 'path', |
| 300 | 'path_hooks', 'path_importer_cache', 'platform', 'prefix', 'ps1', |
| 301 | 'setcheckinterval', 'setdlopenflags', 'setprofile', 'setrecursionlimit', |
| 302 | 'setswitchinterval', 'settrace', 'stderr', 'stdin', 'stdout', |
| 303 | 'thread_info', 'version', 'version_info', 'warnoptions'] |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 304 | |
| 305 | Without arguments, :func:`dir` lists the names you have defined currently:: |
| 306 | |
| 307 | >>> a = [1, 2, 3, 4, 5] |
| 308 | >>> import fibo |
| 309 | >>> fib = fibo.fib |
| 310 | >>> dir() |
Ezio Melotti | 52e8550 | 2012-11-17 12:50:14 +0200 | [diff] [blame] | 311 | ['__builtins__', '__name__', 'a', 'fib', 'fibo', 'sys'] |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 312 | |
| 313 | Note that it lists all types of names: variables, modules, functions, etc. |
| 314 | |
Georg Brandl | 1a3284e | 2007-12-02 09:40:06 +0000 | [diff] [blame] | 315 | .. index:: module: builtins |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 316 | |
| 317 | :func:`dir` does not list the names of built-in functions and variables. If you |
| 318 | want a list of those, they are defined in the standard module |
Georg Brandl | 1a3284e | 2007-12-02 09:40:06 +0000 | [diff] [blame] | 319 | :mod:`builtins`:: |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 320 | |
Georg Brandl | 1a3284e | 2007-12-02 09:40:06 +0000 | [diff] [blame] | 321 | >>> import builtins |
Ezio Melotti | 52e8550 | 2012-11-17 12:50:14 +0200 | [diff] [blame] | 322 | >>> dir(builtins) # doctest: +NORMALIZE_WHITESPACE |
| 323 | ['ArithmeticError', 'AssertionError', 'AttributeError', 'BaseException', |
Ezio Melotti | 4a42ec5 | 2012-11-17 12:54:45 +0200 | [diff] [blame] | 324 | 'BlockingIOError', 'BrokenPipeError', 'BufferError', 'BytesWarning', |
| 325 | 'ChildProcessError', 'ConnectionAbortedError', 'ConnectionError', |
| 326 | 'ConnectionRefusedError', 'ConnectionResetError', 'DeprecationWarning', |
| 327 | 'EOFError', 'Ellipsis', 'EnvironmentError', 'Exception', 'False', |
| 328 | 'FileExistsError', 'FileNotFoundError', 'FloatingPointError', |
Ezio Melotti | 52e8550 | 2012-11-17 12:50:14 +0200 | [diff] [blame] | 329 | 'FutureWarning', 'GeneratorExit', 'IOError', 'ImportError', |
Ezio Melotti | 4a42ec5 | 2012-11-17 12:54:45 +0200 | [diff] [blame] | 330 | 'ImportWarning', 'IndentationError', 'IndexError', 'InterruptedError', |
| 331 | 'IsADirectoryError', 'KeyError', 'KeyboardInterrupt', 'LookupError', |
| 332 | 'MemoryError', 'NameError', 'None', 'NotADirectoryError', 'NotImplemented', |
| 333 | 'NotImplementedError', 'OSError', 'OverflowError', |
| 334 | 'PendingDeprecationWarning', 'PermissionError', 'ProcessLookupError', |
| 335 | 'ReferenceError', 'ResourceWarning', 'RuntimeError', 'RuntimeWarning', |
| 336 | 'StopIteration', 'SyntaxError', 'SyntaxWarning', 'SystemError', |
| 337 | 'SystemExit', 'TabError', 'TimeoutError', 'True', 'TypeError', |
| 338 | 'UnboundLocalError', 'UnicodeDecodeError', 'UnicodeEncodeError', |
| 339 | 'UnicodeError', 'UnicodeTranslateError', 'UnicodeWarning', 'UserWarning', |
| 340 | 'ValueError', 'Warning', 'ZeroDivisionError', '_', '__build_class__', |
| 341 | '__debug__', '__doc__', '__import__', '__name__', '__package__', 'abs', |
| 342 | 'all', 'any', 'ascii', 'bin', 'bool', 'bytearray', 'bytes', 'callable', |
| 343 | 'chr', 'classmethod', 'compile', 'complex', 'copyright', 'credits', |
| 344 | 'delattr', 'dict', 'dir', 'divmod', 'enumerate', 'eval', 'exec', 'exit', |
| 345 | 'filter', 'float', 'format', 'frozenset', 'getattr', 'globals', 'hasattr', |
| 346 | 'hash', 'help', 'hex', 'id', 'input', 'int', 'isinstance', 'issubclass', |
| 347 | 'iter', 'len', 'license', 'list', 'locals', 'map', 'max', 'memoryview', |
| 348 | 'min', 'next', 'object', 'oct', 'open', 'ord', 'pow', 'print', 'property', |
| 349 | 'quit', 'range', 'repr', 'reversed', 'round', 'set', 'setattr', 'slice', |
| 350 | 'sorted', 'staticmethod', 'str', 'sum', 'super', 'tuple', 'type', 'vars', |
| 351 | 'zip'] |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 352 | |
| 353 | .. _tut-packages: |
| 354 | |
| 355 | Packages |
| 356 | ======== |
| 357 | |
| 358 | Packages are a way of structuring Python's module namespace by using "dotted |
| 359 | module names". For example, the module name :mod:`A.B` designates a submodule |
| 360 | named ``B`` in a package named ``A``. Just like the use of modules saves the |
| 361 | authors of different modules from having to worry about each other's global |
| 362 | variable names, the use of dotted module names saves the authors of multi-module |
| 363 | packages like NumPy or the Python Imaging Library from having to worry about |
| 364 | each other's module names. |
| 365 | |
| 366 | Suppose you want to design a collection of modules (a "package") for the uniform |
| 367 | handling of sound files and sound data. There are many different sound file |
| 368 | formats (usually recognized by their extension, for example: :file:`.wav`, |
| 369 | :file:`.aiff`, :file:`.au`), so you may need to create and maintain a growing |
| 370 | collection of modules for the conversion between the various file formats. |
| 371 | There are also many different operations you might want to perform on sound data |
| 372 | (such as mixing, adding echo, applying an equalizer function, creating an |
| 373 | artificial stereo effect), so in addition you will be writing a never-ending |
| 374 | stream of modules to perform these operations. Here's a possible structure for |
Georg Brandl | 22a1fd7 | 2013-10-06 11:08:24 +0200 | [diff] [blame] | 375 | your package (expressed in terms of a hierarchical filesystem): |
| 376 | |
| 377 | .. code-block:: text |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 378 | |
| 379 | sound/ Top-level package |
| 380 | __init__.py Initialize the sound package |
| 381 | formats/ Subpackage for file format conversions |
| 382 | __init__.py |
| 383 | wavread.py |
| 384 | wavwrite.py |
| 385 | aiffread.py |
| 386 | aiffwrite.py |
| 387 | auread.py |
| 388 | auwrite.py |
| 389 | ... |
| 390 | effects/ Subpackage for sound effects |
| 391 | __init__.py |
| 392 | echo.py |
| 393 | surround.py |
| 394 | reverse.py |
| 395 | ... |
| 396 | filters/ Subpackage for filters |
| 397 | __init__.py |
| 398 | equalizer.py |
| 399 | vocoder.py |
| 400 | karaoke.py |
| 401 | ... |
| 402 | |
| 403 | When importing the package, Python searches through the directories on |
| 404 | ``sys.path`` looking for the package subdirectory. |
| 405 | |
| 406 | The :file:`__init__.py` files are required to make Python treat the directories |
| 407 | as containing packages; this is done to prevent directories with a common name, |
| 408 | such as ``string``, from unintentionally hiding valid modules that occur later |
| 409 | on the module search path. In the simplest case, :file:`__init__.py` can just be |
| 410 | an empty file, but it can also execute initialization code for the package or |
| 411 | set the ``__all__`` variable, described later. |
| 412 | |
| 413 | Users of the package can import individual modules from the package, for |
| 414 | example:: |
| 415 | |
| 416 | import sound.effects.echo |
| 417 | |
| 418 | This loads the submodule :mod:`sound.effects.echo`. It must be referenced with |
| 419 | its full name. :: |
| 420 | |
| 421 | sound.effects.echo.echofilter(input, output, delay=0.7, atten=4) |
| 422 | |
| 423 | An alternative way of importing the submodule is:: |
| 424 | |
| 425 | from sound.effects import echo |
| 426 | |
| 427 | This also loads the submodule :mod:`echo`, and makes it available without its |
| 428 | package prefix, so it can be used as follows:: |
| 429 | |
| 430 | echo.echofilter(input, output, delay=0.7, atten=4) |
| 431 | |
| 432 | Yet another variation is to import the desired function or variable directly:: |
| 433 | |
| 434 | from sound.effects.echo import echofilter |
| 435 | |
| 436 | Again, this loads the submodule :mod:`echo`, but this makes its function |
| 437 | :func:`echofilter` directly available:: |
| 438 | |
| 439 | echofilter(input, output, delay=0.7, atten=4) |
| 440 | |
| 441 | Note that when using ``from package import item``, the item can be either a |
| 442 | submodule (or subpackage) of the package, or some other name defined in the |
| 443 | package, like a function, class or variable. The ``import`` statement first |
| 444 | tests whether the item is defined in the package; if not, it assumes it is a |
| 445 | module and attempts to load it. If it fails to find it, an :exc:`ImportError` |
| 446 | exception is raised. |
| 447 | |
| 448 | Contrarily, when using syntax like ``import item.subitem.subsubitem``, each item |
| 449 | except for the last must be a package; the last item can be a module or a |
| 450 | package but can't be a class or function or variable defined in the previous |
| 451 | item. |
| 452 | |
| 453 | |
| 454 | .. _tut-pkg-import-star: |
| 455 | |
| 456 | Importing \* From a Package |
| 457 | --------------------------- |
| 458 | |
| 459 | .. index:: single: __all__ |
| 460 | |
| 461 | Now what happens when the user writes ``from sound.effects import *``? Ideally, |
| 462 | one would hope that this somehow goes out to the filesystem, finds which |
Benjamin Peterson | 4ac9ce4 | 2009-10-04 14:49:41 +0000 | [diff] [blame] | 463 | submodules are present in the package, and imports them all. This could take a |
| 464 | long time and importing sub-modules might have unwanted side-effects that should |
| 465 | only happen when the sub-module is explicitly imported. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 466 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 467 | The only solution is for the package author to provide an explicit index of the |
Alexandre Vassalotti | 6d3dfc3 | 2009-07-29 19:54:39 +0000 | [diff] [blame] | 468 | package. The :keyword:`import` statement uses the following convention: if a package's |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 469 | :file:`__init__.py` code defines a list named ``__all__``, it is taken to be the |
| 470 | list of module names that should be imported when ``from package import *`` is |
| 471 | encountered. It is up to the package author to keep this list up-to-date when a |
| 472 | new version of the package is released. Package authors may also decide not to |
| 473 | support it, if they don't see a use for importing \* from their package. For |
Georg Brandl | ac39add | 2013-10-06 19:21:14 +0200 | [diff] [blame] | 474 | example, the file :file:`sound/effects/__init__.py` could contain the following |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 475 | code:: |
| 476 | |
| 477 | __all__ = ["echo", "surround", "reverse"] |
| 478 | |
| 479 | This would mean that ``from sound.effects import *`` would import the three |
| 480 | named submodules of the :mod:`sound` package. |
| 481 | |
| 482 | If ``__all__`` is not defined, the statement ``from sound.effects import *`` |
| 483 | does *not* import all submodules from the package :mod:`sound.effects` into the |
| 484 | current namespace; it only ensures that the package :mod:`sound.effects` has |
| 485 | been imported (possibly running any initialization code in :file:`__init__.py`) |
| 486 | and then imports whatever names are defined in the package. This includes any |
| 487 | names defined (and submodules explicitly loaded) by :file:`__init__.py`. It |
| 488 | also includes any submodules of the package that were explicitly loaded by |
Alexandre Vassalotti | 6d3dfc3 | 2009-07-29 19:54:39 +0000 | [diff] [blame] | 489 | previous :keyword:`import` statements. Consider this code:: |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 490 | |
| 491 | import sound.effects.echo |
| 492 | import sound.effects.surround |
| 493 | from sound.effects import * |
| 494 | |
Alexandre Vassalotti | 6d3dfc3 | 2009-07-29 19:54:39 +0000 | [diff] [blame] | 495 | In this example, the :mod:`echo` and :mod:`surround` modules are imported in the |
| 496 | current namespace because they are defined in the :mod:`sound.effects` package |
| 497 | when the ``from...import`` statement is executed. (This also works when |
| 498 | ``__all__`` is defined.) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 499 | |
Benjamin Peterson | 4ac9ce4 | 2009-10-04 14:49:41 +0000 | [diff] [blame] | 500 | Although certain modules are designed to export only names that follow certain |
| 501 | patterns when you use ``import *``, it is still considered bad practise in |
| 502 | production code. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 503 | |
| 504 | Remember, there is nothing wrong with using ``from Package import |
| 505 | specific_submodule``! In fact, this is the recommended notation unless the |
| 506 | importing module needs to use submodules with the same name from different |
| 507 | packages. |
| 508 | |
| 509 | |
| 510 | Intra-package References |
| 511 | ------------------------ |
| 512 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 513 | When packages are structured into subpackages (as with the :mod:`sound` package |
| 514 | in the example), you can use absolute imports to refer to submodules of siblings |
| 515 | packages. For example, if the module :mod:`sound.filters.vocoder` needs to use |
| 516 | the :mod:`echo` module in the :mod:`sound.effects` package, it can use ``from |
| 517 | sound.effects import echo``. |
| 518 | |
Georg Brandl | e6bcc91 | 2008-05-12 18:05:20 +0000 | [diff] [blame] | 519 | You can also write relative imports, with the ``from module import name`` form |
| 520 | of import statement. These imports use leading dots to indicate the current and |
| 521 | parent packages involved in the relative import. From the :mod:`surround` |
| 522 | module for example, you might use:: |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 523 | |
| 524 | from . import echo |
| 525 | from .. import formats |
| 526 | from ..filters import equalizer |
| 527 | |
Georg Brandl | e6bcc91 | 2008-05-12 18:05:20 +0000 | [diff] [blame] | 528 | Note that relative imports are based on the name of the current module. Since |
| 529 | the name of the main module is always ``"__main__"``, modules intended for use |
| 530 | as the main module of a Python application must always use absolute imports. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 531 | |
| 532 | |
| 533 | Packages in Multiple Directories |
| 534 | -------------------------------- |
| 535 | |
| 536 | Packages support one more special attribute, :attr:`__path__`. This is |
| 537 | initialized to be a list containing the name of the directory holding the |
| 538 | package's :file:`__init__.py` before the code in that file is executed. This |
| 539 | variable can be modified; doing so affects future searches for modules and |
| 540 | subpackages contained in the package. |
| 541 | |
| 542 | While this feature is not often needed, it can be used to extend the set of |
| 543 | modules found in a package. |
| 544 | |
| 545 | |
| 546 | .. rubric:: Footnotes |
| 547 | |
| 548 | .. [#] In fact function definitions are also 'statements' that are 'executed'; the |
Georg Brandl | 5e2954e | 2013-04-14 11:47:46 +0200 | [diff] [blame] | 549 | execution of a module-level function definition enters the function name in |
| 550 | the module's global symbol table. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 551 | |