Éric Araujo | 3a9f58f | 2011-06-01 20:42:49 +0200 | [diff] [blame] | 1 | .. highlightlang:: none |
| 2 | |
| 3 | ==================================== |
Jason R. Coombs | a599489 | 2012-01-04 10:20:37 -0500 | [diff] [blame] | 4 | Installing Python projects: overview |
Éric Araujo | 3a9f58f | 2011-06-01 20:42:49 +0200 | [diff] [blame] | 5 | ==================================== |
| 6 | |
Éric Araujo | f8bebf8 | 2011-07-02 16:58:25 +0200 | [diff] [blame] | 7 | .. _packaging-install-intro: |
Éric Araujo | 3a9f58f | 2011-06-01 20:42:49 +0200 | [diff] [blame] | 8 | |
| 9 | Introduction |
| 10 | ============ |
| 11 | |
| 12 | Although Python's extensive standard library covers many programming needs, |
| 13 | there often comes a time when you need to add new functionality to your Python |
| 14 | installation in the form of third-party modules. This might be necessary to |
| 15 | support your own programming, or to support an application that you want to use |
| 16 | and that happens to be written in Python. |
| 17 | |
| 18 | In the past, there was little support for adding third-party modules to an |
| 19 | existing Python installation. With the introduction of the Python Distribution |
| 20 | Utilities (Distutils for short) in Python 2.0, this changed. However, not all |
| 21 | problems were solved; end-users had to rely on ``easy_install`` or |
| 22 | ``pip`` to download third-party modules from PyPI, uninstall distributions or do |
| 23 | other maintenance operations. Packaging is a more complete replacement for |
| 24 | Distutils, in the standard library, with a backport named Distutils2 available |
| 25 | for older Python versions. |
| 26 | |
| 27 | This document is aimed primarily at people who need to install third-party |
| 28 | Python modules: end-users and system administrators who just need to get some |
| 29 | Python application running, and existing Python programmers who want to add |
| 30 | new goodies to their toolbox. You don't need to know Python to read this |
| 31 | document; there will be some brief forays into using Python's interactive mode |
| 32 | to explore your installation, but that's it. If you're looking for information |
| 33 | on how to distribute your own Python modules so that others may use them, see |
| 34 | the :ref:`packaging-index` manual. |
| 35 | |
| 36 | |
| 37 | .. _packaging-trivial-install: |
| 38 | |
| 39 | Best case: trivial installation |
| 40 | ------------------------------- |
| 41 | |
| 42 | In the best case, someone will have prepared a special version of the module |
| 43 | distribution you want to install that is targeted specifically at your platform |
| 44 | and can be installed just like any other software on your platform. For example, |
| 45 | the module's developer might make an executable installer available for Windows |
| 46 | users, an RPM package for users of RPM-based Linux systems (Red Hat, SuSE, |
| 47 | Mandrake, and many others), a Debian package for users of Debian and derivative |
| 48 | systems, and so forth. |
| 49 | |
| 50 | In that case, you would use the standard system tools to download and install |
| 51 | the specific installer for your platform and its dependencies. |
| 52 | |
| 53 | Of course, things will not always be that easy. You might be interested in a |
| 54 | module whose distribution doesn't have an easy-to-use installer for your |
| 55 | platform. In that case, you'll have to start with the source distribution |
| 56 | released by the module's author/maintainer. Installing from a source |
| 57 | distribution is not too hard, as long as the modules are packaged in the |
| 58 | standard way. The bulk of this document addresses the building and installing |
| 59 | of modules from standard source distributions. |
| 60 | |
| 61 | |
| 62 | .. _packaging-distutils: |
| 63 | |
| 64 | The Python standard: Distutils |
| 65 | ------------------------------ |
| 66 | |
| 67 | If you download a source distribution of a module, it will be obvious whether |
| 68 | it was packaged and distributed using Distutils. First, the distribution's name |
| 69 | and version number will be featured prominently in the name of the downloaded |
| 70 | archive, e.g. :file:`foo-1.0.tar.gz` or :file:`widget-0.9.7.zip`. Next, the |
| 71 | archive will unpack into a similarly-named directory: :file:`foo-1.0` or |
| 72 | :file:`widget-0.9.7`. Additionally, the distribution may contain a |
| 73 | :file:`setup.cfg` file and a file named :file:`README.txt` ---or possibly just |
| 74 | :file:`README`--- explaining that building and installing the module |
| 75 | distribution is a simple matter of issuing the following command at your shell's |
| 76 | prompt:: |
| 77 | |
| 78 | python setup.py install |
| 79 | |
| 80 | Third-party projects have extended Distutils to work around its limitations or |
| 81 | add functionality. After some years of near-inactivity in Distutils, a new |
| 82 | maintainer has started to standardize good ideas in PEPs and implement them in a |
| 83 | new, improved version of Distutils, called Distutils2 or Packaging. |
| 84 | |
| 85 | |
| 86 | .. _packaging-new-standard: |
| 87 | |
| 88 | The new standard: Packaging |
| 89 | --------------------------- |
| 90 | |
| 91 | The rules described in the first paragraph above apply to Packaging-based |
| 92 | projects too: a source distribution will have a name like |
| 93 | :file:`widget-0.9.7.zip`. One of the main differences with Distutils is that |
| 94 | distributions no longer have a :file:`setup.py` script; it used to cause a |
| 95 | number of issues. Now there is a unique script installed with Python itself:: |
| 96 | |
| 97 | pysetup install widget-0.9.7.zip |
| 98 | |
| 99 | Running this command is enough to build and install projects (Python modules or |
| 100 | packages, scripts or whole applications), without even having to unpack the |
| 101 | archive. It is also compatible with Distutils-based distributions. |
| 102 | |
| 103 | Unless you have to perform non-standard installations or customize the build |
| 104 | process, you can stop reading this manual ---the above command is everything you |
| 105 | need to get out of it. |
| 106 | |
| 107 | With :program:`pysetup`, you won't even have to manually download a distribution |
| 108 | before installing it; see :ref:`packaging-pysetup`. |
| 109 | |
| 110 | |
| 111 | .. _packaging-standard-install: |
| 112 | |
| 113 | Standard build and install |
| 114 | ========================== |
| 115 | |
| 116 | As described in section :ref:`packaging-new-standard`, building and installing |
| 117 | a module distribution using Packaging usually comes down to one simple |
| 118 | command:: |
| 119 | |
| 120 | pysetup run install_dist |
| 121 | |
Éric Araujo | 24d6cc6 | 2011-08-19 03:19:36 +0200 | [diff] [blame] | 122 | This is a command that should be run in a terminal. On Windows, it is called a |
| 123 | command prompt and found in :menuselection:`Start --> Accessories`; Powershell |
| 124 | is a popular alternative. |
Éric Araujo | 3a9f58f | 2011-06-01 20:42:49 +0200 | [diff] [blame] | 125 | |
| 126 | |
| 127 | .. _packaging-platform-variations: |
| 128 | |
| 129 | Platform variations |
| 130 | ------------------- |
| 131 | |
| 132 | The setup command is meant to be run from the root directory of the source |
| 133 | distribution, i.e. the top-level subdirectory that the module source |
| 134 | distribution unpacks into. For example, if you've just downloaded a module |
| 135 | source distribution :file:`foo-1.0.tar.gz` onto a Unix system, the normal |
| 136 | steps to follow are these:: |
| 137 | |
| 138 | gunzip -c foo-1.0.tar.gz | tar xf - # unpacks into directory foo-1.0 |
| 139 | cd foo-1.0 |
| 140 | pysetup run install_dist |
| 141 | |
| 142 | On Windows, you'd probably download :file:`foo-1.0.zip`. If you downloaded the |
| 143 | archive file to :file:`C:\\Temp`, then it would unpack into |
| 144 | :file:`C:\\Temp\\foo-1.0`. To actually unpack the archive, you can use either |
| 145 | an archive manipulator with a graphical user interface (such as WinZip or 7-Zip) |
| 146 | or a command-line tool (such as :program:`unzip`, :program:`pkunzip` or, again, |
Éric Araujo | 24d6cc6 | 2011-08-19 03:19:36 +0200 | [diff] [blame] | 147 | :program:`7z`). Then, open a command prompt window and run:: |
Éric Araujo | 3a9f58f | 2011-06-01 20:42:49 +0200 | [diff] [blame] | 148 | |
| 149 | cd c:\Temp\foo-1.0 |
| 150 | pysetup run install_dist |
| 151 | |
| 152 | |
| 153 | .. _packaging-splitting-up: |
| 154 | |
| 155 | Splitting the job up |
| 156 | -------------------- |
| 157 | |
| 158 | Running ``pysetup run install_dist`` builds and installs all modules in one go. If you |
| 159 | prefer to work incrementally ---especially useful if you want to customize the |
| 160 | build process, or if things are going wrong--- you can use the setup script to |
| 161 | do one thing at a time. This is a valuable tool when different users will perform |
| 162 | separately the build and install steps. For example, you might want to build a |
| 163 | module distribution and hand it off to a system administrator for installation |
| 164 | (or do it yourself, but with super-user or admin privileges). |
| 165 | |
| 166 | For example, to build everything in one step and then install everything |
| 167 | in a second step, you aptly invoke two distinct Packaging commands:: |
| 168 | |
| 169 | pysetup run build |
| 170 | pysetup run install_dist |
| 171 | |
| 172 | If you do this, you will notice that invoking the :command:`install_dist` command |
| 173 | first runs the :command:`build` command, which ---in this case--- quickly |
| 174 | notices it can spare itself the work, since everything in the :file:`build` |
| 175 | directory is up-to-date. |
| 176 | |
| 177 | You may often ignore this ability to divide the process in steps if all you do |
| 178 | is installing modules downloaded from the Internet, but it's very handy for |
| 179 | more advanced tasks. If you find yourself in the need for distributing your own |
| 180 | Python modules and extensions, though, you'll most likely run many individual |
| 181 | Packaging commands. |
| 182 | |
| 183 | |
| 184 | .. _packaging-how-build-works: |
| 185 | |
| 186 | How building works |
| 187 | ------------------ |
| 188 | |
| 189 | As implied above, the :command:`build` command is responsible for collecting |
| 190 | and placing the files to be installed into a *build directory*. By default, |
| 191 | this is :file:`build`, under the distribution root. If you're excessively |
| 192 | concerned with speed, or want to keep the source tree pristine, you can specify |
| 193 | a different build directory with the :option:`--build-base` option. For example:: |
| 194 | |
| 195 | pysetup run build --build-base /tmp/pybuild/foo-1.0 |
| 196 | |
| 197 | (Or you could do this permanently with a directive in your system or personal |
| 198 | Packaging configuration file; see section :ref:`packaging-config-files`.) |
| 199 | In the usual case, however, all this is unnecessary. |
| 200 | |
| 201 | The build tree's default layout looks like so:: |
| 202 | |
| 203 | --- build/ --- lib/ |
| 204 | or |
| 205 | --- build/ --- lib.<plat>/ |
| 206 | temp.<plat>/ |
| 207 | |
| 208 | where ``<plat>`` expands to a brief description of the current OS/hardware |
| 209 | platform and Python version. The first form, with just a :file:`lib` directory, |
| 210 | is used for pure module distributions (module distributions that |
| 211 | include only pure Python modules). If a module distribution contains any |
| 212 | extensions (modules written in C/C++), then the second form, with two ``<plat>`` |
| 213 | directories, is used. In that case, the :file:`temp.{plat}` directory holds |
| 214 | temporary files generated during the compile/link process which are not intended |
| 215 | to be installed. In either case, the :file:`lib` (or :file:`lib.{plat}`) directory |
| 216 | contains all Python modules (pure Python and extensions) to be installed. |
| 217 | |
| 218 | In the future, more directories will be added to handle Python scripts, |
| 219 | documentation, binary executables, and whatever else is required to install |
| 220 | Python modules and applications. |
| 221 | |
| 222 | |
| 223 | .. _packaging-how-install-works: |
| 224 | |
| 225 | How installation works |
| 226 | ---------------------- |
| 227 | |
| 228 | After the :command:`build` command is run (whether explicitly or by the |
| 229 | :command:`install_dist` command on your behalf), the work of the :command:`install_dist` |
| 230 | command is relatively simple: all it has to do is copy the contents of |
| 231 | :file:`build/lib` (or :file:`build/lib.{plat}`) to the installation directory |
| 232 | of your choice. |
| 233 | |
| 234 | If you don't choose an installation directory ---i.e., if you just run |
| 235 | ``pysetup run install_dist``\ --- then the :command:`install_dist` command |
| 236 | installs to the standard location for third-party Python modules. This location |
| 237 | varies by platform and depending on how you built/installed Python itself. On |
| 238 | Unix (and Mac OS X, which is also Unix-based), it also depends on whether the |
| 239 | module distribution being installed is pure Python or contains extensions |
| 240 | ("non-pure"): |
| 241 | |
| 242 | +-----------------+-----------------------------------------------------+--------------------------------------------------+-------+ |
| 243 | | Platform | Standard installation location | Default value | Notes | |
| 244 | +=================+=====================================================+==================================================+=======+ |
| 245 | | Unix (pure) | :file:`{prefix}/lib/python{X.Y}/site-packages` | :file:`/usr/local/lib/python{X.Y}/site-packages` | \(1) | |
| 246 | +-----------------+-----------------------------------------------------+--------------------------------------------------+-------+ |
| 247 | | Unix (non-pure) | :file:`{exec-prefix}/lib/python{X.Y}/site-packages` | :file:`/usr/local/lib/python{X.Y}/site-packages` | \(1) | |
| 248 | +-----------------+-----------------------------------------------------+--------------------------------------------------+-------+ |
| 249 | | Windows | :file:`{prefix}\\Lib\\site-packages` | :file:`C:\\Python{XY}\\Lib\\site-packages` | \(2) | |
| 250 | +-----------------+-----------------------------------------------------+--------------------------------------------------+-------+ |
| 251 | |
| 252 | Notes: |
| 253 | |
| 254 | (1) |
| 255 | Most Linux distributions include Python as a standard part of the system, so |
| 256 | :file:`{prefix}` and :file:`{exec-prefix}` are usually both :file:`/usr` on |
| 257 | Linux. If you build Python yourself on Linux (or any Unix-like system), the |
| 258 | default :file:`{prefix}` and :file:`{exec-prefix}` are :file:`/usr/local`. |
| 259 | |
| 260 | (2) |
| 261 | The default installation directory on Windows was :file:`C:\\Program |
| 262 | Files\\Python` under Python 1.6a1, 1.5.2, and earlier. |
| 263 | |
| 264 | :file:`{prefix}` and :file:`{exec-prefix}` stand for the directories that Python |
| 265 | is installed to, and where it finds its libraries at run-time. They are always |
| 266 | the same under Windows, and very often the same under Unix and Mac OS X. You |
| 267 | can find out what your Python installation uses for :file:`{prefix}` and |
| 268 | :file:`{exec-prefix}` by running Python in interactive mode and typing a few |
| 269 | simple commands. |
| 270 | |
| 271 | .. TODO link to Doc/using instead of duplicating |
| 272 | |
| 273 | To start the interactive Python interpreter, you need to follow a slightly |
| 274 | different recipe for each platform. Under Unix, just type :command:`python` at |
| 275 | the shell prompt. Under Windows (assuming the Python executable is on your |
| 276 | :envvar:`PATH`, which is the usual case), you can choose :menuselection:`Start --> Run`, |
| 277 | type ``python`` and press ``enter``. Alternatively, you can simply execute |
Éric Araujo | 24d6cc6 | 2011-08-19 03:19:36 +0200 | [diff] [blame] | 278 | :command:`python` at a command prompt (:menuselection:`Start --> Accessories`) |
| 279 | or in Powershell. |
Éric Araujo | 3a9f58f | 2011-06-01 20:42:49 +0200 | [diff] [blame] | 280 | |
| 281 | Once the interpreter is started, you type Python code at the prompt. For |
| 282 | example, on my Linux system, I type the three Python statements shown below, |
| 283 | and get the output as shown, to find out my :file:`{prefix}` and :file:`{exec-prefix}`:: |
| 284 | |
| 285 | Python 3.3 (r32:88445, Apr 2 2011, 10:43:54) |
| 286 | Type "help", "copyright", "credits" or "license" for more information. |
| 287 | >>> import sys |
| 288 | >>> sys.prefix |
| 289 | '/usr' |
| 290 | >>> sys.exec_prefix |
| 291 | '/usr' |
| 292 | |
Éric Araujo | 7dc76fd | 2011-08-06 16:58:15 +0200 | [diff] [blame] | 293 | A few other placeholders are used in this document: :file:`{X.Y}` stands for the |
| 294 | version of Python, for example ``3.2``; :file:`{abiflags}` will be replaced by |
| 295 | the value of :data:`sys.abiflags` or the empty string for platforms which don't |
| 296 | define ABI flags; :file:`{distname}` will be replaced by the name of the module |
| 297 | distribution being installed. Dots and capitalization are important in the |
| 298 | paths; for example, a value that uses ``python3.2`` on UNIX will typically use |
| 299 | ``Python32`` on Windows. |
| 300 | |
Éric Araujo | 3a9f58f | 2011-06-01 20:42:49 +0200 | [diff] [blame] | 301 | If you don't want to install modules to the standard location, or if you don't |
| 302 | have permission to write there, then you need to read about alternate |
| 303 | installations in section :ref:`packaging-alt-install`. If you want to customize your |
| 304 | installation directories more heavily, see section :ref:`packaging-custom-install`. |
| 305 | |
| 306 | |
| 307 | .. _packaging-alt-install: |
| 308 | |
| 309 | Alternate installation |
| 310 | ====================== |
| 311 | |
| 312 | Often, it is necessary or desirable to install modules to a location other than |
| 313 | the standard location for third-party Python modules. For example, on a Unix |
| 314 | system you might not have permission to write to the standard third-party module |
| 315 | directory. Or you might wish to try out a module before making it a standard |
| 316 | part of your local Python installation. This is especially true when upgrading |
| 317 | a distribution already present: you want to make sure your existing base of |
| 318 | scripts still works with the new version before actually upgrading. |
| 319 | |
| 320 | The Packaging :command:`install_dist` command is designed to make installing module |
| 321 | distributions to an alternate location simple and painless. The basic idea is |
| 322 | that you supply a base directory for the installation, and the |
| 323 | :command:`install_dist` command picks a set of directories (called an *installation |
| 324 | scheme*) under this base directory in which to install files. The details |
| 325 | differ across platforms, so read whichever of the following sections applies to |
| 326 | you. |
| 327 | |
Éric Araujo | 7dc76fd | 2011-08-06 16:58:15 +0200 | [diff] [blame] | 328 | Note that the various alternate installation schemes are mutually exclusive: you |
| 329 | can pass ``--user``, or ``--home``, or ``--prefix`` and ``--exec-prefix``, or |
| 330 | ``--install-base`` and ``--install-platbase``, but you can't mix from these |
| 331 | groups. |
Éric Araujo | 3a9f58f | 2011-06-01 20:42:49 +0200 | [diff] [blame] | 332 | |
Éric Araujo | 7dc76fd | 2011-08-06 16:58:15 +0200 | [diff] [blame] | 333 | |
| 334 | .. _packaging-alt-install-user: |
| 335 | |
| 336 | Alternate installation: the user scheme |
| 337 | --------------------------------------- |
| 338 | |
| 339 | This scheme is designed to be the most convenient solution for users that don't |
| 340 | have write permission to the global site-packages directory or don't want to |
| 341 | install into it. It is enabled with a simple option:: |
| 342 | |
| 343 | pysetup run install_dist --user |
| 344 | |
| 345 | Files will be installed into subdirectories of :data:`site.USER_BASE` (written |
| 346 | as :file:`{userbase}` hereafter). This scheme installs pure Python modules and |
| 347 | extension modules in the same location (also known as :data:`site.USER_SITE`). |
| 348 | Here are the values for UNIX, including non-framework builds on Mac OS X: |
| 349 | |
| 350 | =============== =========================================================== |
| 351 | Type of file Installation directory |
| 352 | =============== =========================================================== |
| 353 | modules :file:`{userbase}/lib/python{X.Y}/site-packages` |
| 354 | scripts :file:`{userbase}/bin` |
| 355 | data :file:`{userbase}` |
| 356 | C headers :file:`{userbase}/include/python{X.Y}` |
| 357 | =============== =========================================================== |
| 358 | |
| 359 | Framework builds on Mac OS X use these paths: |
| 360 | |
| 361 | =============== =========================================================== |
| 362 | Type of file Installation directory |
| 363 | =============== =========================================================== |
| 364 | modules :file:`{userbase}/lib/python/site-packages` |
| 365 | scripts :file:`{userbase}/bin` |
| 366 | data :file:`{userbase}` |
| 367 | C headers :file:`{userbase}/include/python` |
| 368 | =============== =========================================================== |
| 369 | |
| 370 | And here are the values used on Windows: |
| 371 | |
| 372 | =============== =========================================================== |
| 373 | Type of file Installation directory |
| 374 | =============== =========================================================== |
| 375 | modules :file:`{userbase}\\Python{XY}\\site-packages` |
| 376 | scripts :file:`{userbase}\\Scripts` |
| 377 | data :file:`{userbase}` |
| 378 | C headers :file:`{userbase}\\Python{XY}\\Include` |
| 379 | =============== =========================================================== |
| 380 | |
| 381 | The advantage of using this scheme compared to the other ones described below is |
| 382 | that the user site-packages directory is under normal conditions always included |
| 383 | in :data:`sys.path` (see :mod:`site` for more information), which means that |
| 384 | there is no additional step to perform after running ``pysetup`` to finalize the |
| 385 | installation. |
| 386 | |
| 387 | The :command:`build_ext` command also has a ``--user`` option to add |
| 388 | :file:`{userbase}/include` to the compiler search path for header files and |
| 389 | :file:`{userbase}/lib` to the compiler search path for libraries as well as to |
| 390 | the runtime search path for shared C libraries (rpath). |
| 391 | |
| 392 | |
| 393 | .. _packaging-alt-install-home: |
Éric Araujo | 3a9f58f | 2011-06-01 20:42:49 +0200 | [diff] [blame] | 394 | |
| 395 | Alternate installation: the home scheme |
| 396 | --------------------------------------- |
| 397 | |
| 398 | The idea behind the "home scheme" is that you build and maintain a personal |
| 399 | stash of Python modules. This scheme's name is derived from the concept of a |
| 400 | "home" directory on Unix, since it's not unusual for a Unix user to make their |
| 401 | home directory have a layout similar to :file:`/usr/` or :file:`/usr/local/`. |
| 402 | In spite of its name's origin, this scheme can be used by anyone, regardless |
| 403 | of the operating system. |
| 404 | |
| 405 | Installing a new module distribution in this way is as simple as :: |
| 406 | |
| 407 | pysetup run install_dist --home <dir> |
| 408 | |
| 409 | where you can supply any directory you like for the :option:`--home` option. On |
| 410 | Unix, lazy typists can just type a tilde (``~``); the :command:`install_dist` command |
| 411 | will expand this to your home directory:: |
| 412 | |
| 413 | pysetup run install_dist --home ~ |
| 414 | |
Éric Araujo | 7dc76fd | 2011-08-06 16:58:15 +0200 | [diff] [blame] | 415 | To make Python find the distributions installed with this scheme, you may have |
| 416 | to :ref:`modify Python's search path <inst-search-path>` or edit |
| 417 | :mod:`sitecustomize` (see :mod:`site`) to call :func:`site.addsitedir` or edit |
| 418 | :data:`sys.path`. |
| 419 | |
Éric Araujo | 3a9f58f | 2011-06-01 20:42:49 +0200 | [diff] [blame] | 420 | The :option:`--home` option defines the base directory for the installation. |
| 421 | Under it, files are installed to the following directories: |
| 422 | |
Éric Araujo | 7dc76fd | 2011-08-06 16:58:15 +0200 | [diff] [blame] | 423 | =============== =========================================================== |
| 424 | Type of file Installation directory |
| 425 | =============== =========================================================== |
| 426 | modules :file:`{home}/lib/python` |
| 427 | scripts :file:`{home}/bin` |
| 428 | data :file:`{home}` |
| 429 | C headers :file:`{home}/include/python` |
| 430 | =============== =========================================================== |
| 431 | |
| 432 | (Mentally replace slashes with backslashes if you're on Windows.) |
Éric Araujo | 3a9f58f | 2011-06-01 20:42:49 +0200 | [diff] [blame] | 433 | |
| 434 | |
Éric Araujo | 7dc76fd | 2011-08-06 16:58:15 +0200 | [diff] [blame] | 435 | .. _packaging-alt-install-prefix-unix: |
Éric Araujo | 3a9f58f | 2011-06-01 20:42:49 +0200 | [diff] [blame] | 436 | |
| 437 | Alternate installation: Unix (the prefix scheme) |
| 438 | ------------------------------------------------ |
| 439 | |
| 440 | The "prefix scheme" is useful when you wish to use one Python installation to |
| 441 | run the build command, but install modules into the third-party module directory |
| 442 | of a different Python installation (or something that looks like a different |
| 443 | Python installation). If this sounds a trifle unusual, it is ---that's why the |
Éric Araujo | 7dc76fd | 2011-08-06 16:58:15 +0200 | [diff] [blame] | 444 | user and home schemes come before. However, there are at least two known cases |
| 445 | where the prefix scheme will be useful. |
Éric Araujo | 3a9f58f | 2011-06-01 20:42:49 +0200 | [diff] [blame] | 446 | |
| 447 | First, consider that many Linux distributions put Python in :file:`/usr`, rather |
| 448 | than the more traditional :file:`/usr/local`. This is entirely appropriate, |
| 449 | since in those cases Python is part of "the system" rather than a local add-on. |
| 450 | However, if you are installing Python modules from source, you probably want |
| 451 | them to go in :file:`/usr/local/lib/python2.{X}` rather than |
| 452 | :file:`/usr/lib/python2.{X}`. This can be done with :: |
| 453 | |
| 454 | pysetup run install_dist --prefix /usr/local |
| 455 | |
| 456 | Another possibility is a network filesystem where the name used to write to a |
| 457 | remote directory is different from the name used to read it: for example, the |
| 458 | Python interpreter accessed as :file:`/usr/local/bin/python` might search for |
| 459 | modules in :file:`/usr/local/lib/python2.{X}`, but those modules would have to |
| 460 | be installed to, say, :file:`/mnt/{@server}/export/lib/python2.{X}`. This could |
| 461 | be done with :: |
| 462 | |
| 463 | pysetup run install_dist --prefix=/mnt/@server/export |
| 464 | |
| 465 | In either case, the :option:`--prefix` option defines the installation base, and |
| 466 | the :option:`--exec-prefix` option defines the platform-specific installation |
| 467 | base, which is used for platform-specific files. (Currently, this just means |
| 468 | non-pure module distributions, but could be expanded to C libraries, binary |
| 469 | executables, etc.) If :option:`--exec-prefix` is not supplied, it defaults to |
| 470 | :option:`--prefix`. Files are installed as follows: |
| 471 | |
Éric Araujo | 7dc76fd | 2011-08-06 16:58:15 +0200 | [diff] [blame] | 472 | ================= ========================================================== |
| 473 | Type of file Installation directory |
| 474 | ================= ========================================================== |
| 475 | Python modules :file:`{prefix}/lib/python{X.Y}/site-packages` |
| 476 | extension modules :file:`{exec-prefix}/lib/python{X.Y}/site-packages` |
| 477 | scripts :file:`{prefix}/bin` |
| 478 | data :file:`{prefix}` |
| 479 | C headers :file:`{prefix}/include/python{X.Y}{abiflags}` |
| 480 | ================= ========================================================== |
| 481 | |
| 482 | .. XXX misses an entry for platinclude |
Éric Araujo | 3a9f58f | 2011-06-01 20:42:49 +0200 | [diff] [blame] | 483 | |
| 484 | There is no requirement that :option:`--prefix` or :option:`--exec-prefix` |
| 485 | actually point to an alternate Python installation; if the directories listed |
| 486 | above do not already exist, they are created at installation time. |
| 487 | |
| 488 | Incidentally, the real reason the prefix scheme is important is simply that a |
| 489 | standard Unix installation uses the prefix scheme, but with :option:`--prefix` |
| 490 | and :option:`--exec-prefix` supplied by Python itself as ``sys.prefix`` and |
| 491 | ``sys.exec_prefix``. Thus, you might think you'll never use the prefix scheme, |
| 492 | but every time you run ``pysetup run install_dist`` without any other |
| 493 | options, you're using it. |
| 494 | |
| 495 | Note that installing extensions to an alternate Python installation doesn't have |
| 496 | anything to do with how those extensions are built: in particular, extensions |
| 497 | will be compiled using the Python header files (:file:`Python.h` and friends) |
| 498 | installed with the Python interpreter used to run the build command. It is |
| 499 | therefore your responsibility to ensure compatibility between the interpreter |
| 500 | intended to run extensions installed in this way and the interpreter used to |
| 501 | build these same extensions. To avoid problems, it is best to make sure that |
| 502 | the two interpreters are the same version of Python (possibly different builds, |
| 503 | or possibly copies of the same build). (Of course, if your :option:`--prefix` |
| 504 | and :option:`--exec-prefix` don't even point to an alternate Python installation, |
| 505 | this is immaterial.) |
| 506 | |
| 507 | |
Éric Araujo | 7dc76fd | 2011-08-06 16:58:15 +0200 | [diff] [blame] | 508 | .. _packaging-alt-install-prefix-windows: |
Éric Araujo | 3a9f58f | 2011-06-01 20:42:49 +0200 | [diff] [blame] | 509 | |
| 510 | Alternate installation: Windows (the prefix scheme) |
| 511 | --------------------------------------------------- |
| 512 | |
| 513 | Windows has a different and vaguer notion of home directories than Unix, and |
| 514 | since its standard Python installation is simpler, the :option:`--prefix` option |
| 515 | has traditionally been used to install additional packages to arbitrary |
| 516 | locations. :: |
| 517 | |
| 518 | pysetup run install_dist --prefix "\Temp\Python" |
| 519 | |
| 520 | to install modules to the :file:`\\Temp\\Python` directory on the current drive. |
| 521 | |
| 522 | The installation base is defined by the :option:`--prefix` option; the |
Éric Araujo | 7dc76fd | 2011-08-06 16:58:15 +0200 | [diff] [blame] | 523 | :option:`--exec-prefix` option is not supported under Windows, which means that |
| 524 | pure Python modules and extension modules are installed into the same location. |
| 525 | Files are installed as follows: |
Éric Araujo | 3a9f58f | 2011-06-01 20:42:49 +0200 | [diff] [blame] | 526 | |
Éric Araujo | 7dc76fd | 2011-08-06 16:58:15 +0200 | [diff] [blame] | 527 | =============== ========================================================== |
| 528 | Type of file Installation directory |
| 529 | =============== ========================================================== |
| 530 | modules :file:`{prefix}\\Lib\\site-packages` |
| 531 | scripts :file:`{prefix}\\Scripts` |
| 532 | data :file:`{prefix}` |
| 533 | C headers :file:`{prefix}\\Include` |
| 534 | =============== ========================================================== |
Éric Araujo | 3a9f58f | 2011-06-01 20:42:49 +0200 | [diff] [blame] | 535 | |
| 536 | |
| 537 | .. _packaging-custom-install: |
| 538 | |
| 539 | Custom installation |
| 540 | =================== |
| 541 | |
| 542 | Sometimes, the alternate installation schemes described in section |
| 543 | :ref:`packaging-alt-install` just don't do what you want. You might want to tweak |
| 544 | just one or two directories while keeping everything under the same base |
| 545 | directory, or you might want to completely redefine the installation scheme. |
| 546 | In either case, you're creating a *custom installation scheme*. |
| 547 | |
Éric Araujo | 7dc76fd | 2011-08-06 16:58:15 +0200 | [diff] [blame] | 548 | To create a custom installation scheme, you start with one of the alternate |
| 549 | schemes and override some of the installation directories used for the various |
| 550 | types of files, using these options: |
| 551 | |
| 552 | ====================== ======================= |
| 553 | Type of file Override option |
| 554 | ====================== ======================= |
| 555 | Python modules ``--install-purelib`` |
| 556 | extension modules ``--install-platlib`` |
| 557 | all modules ``--install-lib`` |
| 558 | scripts ``--install-scripts`` |
| 559 | data ``--install-data`` |
| 560 | C headers ``--install-headers`` |
| 561 | ====================== ======================= |
| 562 | |
| 563 | These override options can be relative, absolute, |
Éric Araujo | 3a9f58f | 2011-06-01 20:42:49 +0200 | [diff] [blame] | 564 | or explicitly defined in terms of one of the installation base directories. |
| 565 | (There are two installation base directories, and they are normally the same |
| 566 | ---they only differ when you use the Unix "prefix scheme" and supply different |
Éric Araujo | 7dc76fd | 2011-08-06 16:58:15 +0200 | [diff] [blame] | 567 | ``--prefix`` and ``--exec-prefix`` options; using ``--install-lib`` will |
| 568 | override values computed or given for ``--install-purelib`` and |
| 569 | ``--install-platlib``, and is recommended for schemes that don't make a |
| 570 | difference between Python and extension modules.) |
Éric Araujo | 3a9f58f | 2011-06-01 20:42:49 +0200 | [diff] [blame] | 571 | |
| 572 | For example, say you're installing a module distribution to your home directory |
| 573 | under Unix, but you want scripts to go in :file:`~/scripts` rather than |
| 574 | :file:`~/bin`. As you might expect, you can override this directory with the |
| 575 | :option:`--install-scripts` option and, in this case, it makes most sense to supply |
| 576 | a relative path, which will be interpreted relative to the installation base |
| 577 | directory (in our example, your home directory):: |
| 578 | |
| 579 | pysetup run install_dist --home ~ --install-scripts scripts |
| 580 | |
| 581 | Another Unix example: suppose your Python installation was built and installed |
| 582 | with a prefix of :file:`/usr/local/python`. Thus, in a standard installation, |
| 583 | scripts will wind up in :file:`/usr/local/python/bin`. If you want them in |
| 584 | :file:`/usr/local/bin` instead, you would supply this absolute directory for |
| 585 | the :option:`--install-scripts` option:: |
| 586 | |
| 587 | pysetup run install_dist --install-scripts /usr/local/bin |
| 588 | |
| 589 | This command performs an installation using the "prefix scheme", where the |
| 590 | prefix is whatever your Python interpreter was installed with ---in this case, |
| 591 | :file:`/usr/local/python`. |
| 592 | |
| 593 | If you maintain Python on Windows, you might want third-party modules to live in |
| 594 | a subdirectory of :file:`{prefix}`, rather than right in :file:`{prefix}` |
| 595 | itself. This is almost as easy as customizing the script installation directory |
| 596 | ---you just have to remember that there are two types of modules to worry about, |
Éric Araujo | 7dc76fd | 2011-08-06 16:58:15 +0200 | [diff] [blame] | 597 | Python and extension modules, which can conveniently be both controlled by one |
| 598 | option:: |
Éric Araujo | 3a9f58f | 2011-06-01 20:42:49 +0200 | [diff] [blame] | 599 | |
Éric Araujo | 7dc76fd | 2011-08-06 16:58:15 +0200 | [diff] [blame] | 600 | pysetup run install_dist --install-lib Site |
Éric Araujo | 3a9f58f | 2011-06-01 20:42:49 +0200 | [diff] [blame] | 601 | |
| 602 | .. XXX Nothing is installed right under prefix in windows, is it?? |
| 603 | |
Éric Araujo | 7dc76fd | 2011-08-06 16:58:15 +0200 | [diff] [blame] | 604 | The specified installation directory is relative to :file:`{prefix}`. Of |
| 605 | course, you also have to ensure that this directory is in Python's module |
| 606 | search path, such as by putting a :file:`.pth` file in a site directory (see |
| 607 | :mod:`site`). See section :ref:`packaging-search-path` to find out how to modify |
| 608 | Python's search path. |
Éric Araujo | 3a9f58f | 2011-06-01 20:42:49 +0200 | [diff] [blame] | 609 | |
| 610 | If you want to define an entire installation scheme, you just have to supply all |
| 611 | of the installation directory options. Using relative paths is recommended here. |
| 612 | For example, if you want to maintain all Python module-related files under |
| 613 | :file:`python` in your home directory, and you want a separate directory for |
| 614 | each platform that you use your home directory from, you might define the |
| 615 | following installation scheme:: |
| 616 | |
| 617 | pysetup run install_dist --home ~ \ |
| 618 | --install-purelib python/lib \ |
| 619 | --install-platlib python/'lib.$PLAT' \ |
| 620 | --install-scripts python/scripts \ |
| 621 | --install-data python/data |
| 622 | |
| 623 | or, equivalently, :: |
| 624 | |
| 625 | pysetup run install_dist --home ~/python \ |
| 626 | --install-purelib lib \ |
| 627 | --install-platlib 'lib.$PLAT' \ |
| 628 | --install-scripts scripts \ |
| 629 | --install-data data |
| 630 | |
| 631 | ``$PLAT`` doesn't need to be defined as an environment variable ---it will also |
| 632 | be expanded by Packaging as it parses your command line options, just as it |
| 633 | does when parsing your configuration file(s). (More on that later.) |
| 634 | |
| 635 | Obviously, specifying the entire installation scheme every time you install a |
| 636 | new module distribution would be very tedious. To spare you all that work, you |
| 637 | can store it in a Packaging configuration file instead (see section |
| 638 | :ref:`packaging-config-files`), like so:: |
| 639 | |
| 640 | [install_dist] |
| 641 | install-base = $HOME |
| 642 | install-purelib = python/lib |
| 643 | install-platlib = python/lib.$PLAT |
| 644 | install-scripts = python/scripts |
| 645 | install-data = python/data |
| 646 | |
| 647 | or, equivalently, :: |
| 648 | |
| 649 | [install_dist] |
| 650 | install-base = $HOME/python |
| 651 | install-purelib = lib |
| 652 | install-platlib = lib.$PLAT |
| 653 | install-scripts = scripts |
| 654 | install-data = data |
| 655 | |
| 656 | Note that these two are *not* equivalent if you override their installation |
| 657 | base directory when running the setup script. For example, :: |
| 658 | |
| 659 | pysetup run install_dist --install-base /tmp |
| 660 | |
| 661 | would install pure modules to :file:`/tmp/python/lib` in the first case, and |
| 662 | to :file:`/tmp/lib` in the second case. (For the second case, you'd probably |
| 663 | want to supply an installation base of :file:`/tmp/python`.) |
| 664 | |
| 665 | You may have noticed the use of ``$HOME`` and ``$PLAT`` in the sample |
| 666 | configuration file. These are Packaging configuration variables, which |
| 667 | bear a strong resemblance to environment variables. In fact, you can use |
| 668 | environment variables in configuration files on platforms that have such a notion, but |
| 669 | Packaging additionally defines a few extra variables that may not be in your |
| 670 | environment, such as ``$PLAT``. Of course, on systems that don't have |
| 671 | environment variables, such as Mac OS 9, the configuration variables supplied by |
| 672 | the Packaging are the only ones you can use. See section :ref:`packaging-config-files` |
| 673 | for details. |
| 674 | |
| 675 | .. XXX which vars win out eventually in case of clash env or Packaging? |
| 676 | |
| 677 | .. XXX need some Windows examples---when would custom installation schemes be |
| 678 | needed on those platforms? |
| 679 | |
| 680 | |
| 681 | .. XXX Move this section to Doc/using |
| 682 | |
| 683 | .. _packaging-search-path: |
| 684 | |
| 685 | Modifying Python's search path |
| 686 | ------------------------------ |
| 687 | |
| 688 | When the Python interpreter executes an :keyword:`import` statement, it searches |
| 689 | for both Python code and extension modules along a search path. A default value |
| 690 | for this path is configured into the Python binary when the interpreter is built. |
| 691 | You can obtain the search path by importing the :mod:`sys` module and printing |
| 692 | the value of ``sys.path``. :: |
| 693 | |
| 694 | $ python |
| 695 | Python 2.2 (#11, Oct 3 2002, 13:31:27) |
| 696 | [GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-112)] on linux2 |
| 697 | Type "help", "copyright", "credits" or "license" for more information. |
| 698 | >>> import sys |
| 699 | >>> sys.path |
| 700 | ['', '/usr/local/lib/python2.3', '/usr/local/lib/python2.3/plat-linux2', |
| 701 | '/usr/local/lib/python2.3/lib-tk', '/usr/local/lib/python2.3/lib-dynload', |
| 702 | '/usr/local/lib/python2.3/site-packages'] |
| 703 | >>> |
| 704 | |
| 705 | The null string in ``sys.path`` represents the current working directory. |
| 706 | |
| 707 | The expected convention for locally installed packages is to put them in the |
| 708 | :file:`{...}/site-packages/` directory, but you may want to choose a different |
| 709 | location for some reason. For example, if your site kept by convention all web |
| 710 | server-related software under :file:`/www`. Add-on Python modules might then |
| 711 | belong in :file:`/www/python`, and in order to import them, this directory would |
| 712 | have to be added to ``sys.path``. There are several ways to solve this problem. |
| 713 | |
| 714 | The most convenient way is to add a path configuration file to a directory |
| 715 | that's already on Python's path, usually to the :file:`.../site-packages/` |
| 716 | directory. Path configuration files have an extension of :file:`.pth`, and each |
| 717 | line must contain a single path that will be appended to ``sys.path``. (Because |
| 718 | the new paths are appended to ``sys.path``, modules in the added directories |
| 719 | will not override standard modules. This means you can't use this mechanism for |
| 720 | installing fixed versions of standard modules.) |
| 721 | |
| 722 | Paths can be absolute or relative, in which case they're relative to the |
| 723 | directory containing the :file:`.pth` file. See the documentation of |
| 724 | the :mod:`site` module for more information. |
| 725 | |
| 726 | A slightly less convenient way is to edit the :file:`site.py` file in Python's |
| 727 | standard library, and modify ``sys.path``. :file:`site.py` is automatically |
| 728 | imported when the Python interpreter is executed, unless the :option:`-S` switch |
| 729 | is supplied to suppress this behaviour. So you could simply edit |
| 730 | :file:`site.py` and add two lines to it:: |
| 731 | |
| 732 | import sys |
| 733 | sys.path.append('/www/python/') |
| 734 | |
| 735 | However, if you reinstall the same major version of Python (perhaps when |
| 736 | upgrading from 3.3 to 3.3.1, for example) :file:`site.py` will be overwritten by |
| 737 | the stock version. You'd have to remember that it was modified and save a copy |
| 738 | before doing the installation. |
| 739 | |
| 740 | Alternatively, there are two environment variables that can modify ``sys.path``. |
| 741 | :envvar:`PYTHONHOME` sets an alternate value for the prefix of the Python |
| 742 | installation. For example, if :envvar:`PYTHONHOME` is set to ``/www/python``, |
| 743 | the search path will be set to ``['', '/www/python/lib/pythonX.Y/', |
| 744 | '/www/python/lib/pythonX.Y/plat-linux2', ...]``. |
| 745 | |
| 746 | The :envvar:`PYTHONPATH` variable can be set to a list of paths that will be |
| 747 | added to the beginning of ``sys.path``. For example, if :envvar:`PYTHONPATH` is |
| 748 | set to ``/www/python:/opt/py``, the search path will begin with |
| 749 | ``['/www/python', '/opt/py']``. (Note that directories must exist in order to |
| 750 | be added to ``sys.path``; the :mod:`site` module removes non-existent paths.) |
| 751 | |
| 752 | Finally, ``sys.path`` is just a regular Python list, so any Python application |
| 753 | can modify it by adding or removing entries. |
| 754 | |
| 755 | |
| 756 | .. _packaging-config-files: |
| 757 | |
| 758 | Configuration files for Packaging |
| 759 | ================================= |
| 760 | |
| 761 | As mentioned above, you can use configuration files to store personal or site |
| 762 | preferences for any option supported by any Packaging command. Depending on your |
| 763 | platform, you can use one of two or three possible configuration files. These |
| 764 | files will be read before parsing the command-line, so they take precedence over |
| 765 | default values. In turn, the command-line will override configuration files. |
| 766 | Lastly, if there are multiple configuration files, values from files read |
| 767 | earlier will be overridden by values from files read later. |
| 768 | |
| 769 | .. XXX "one of two or three possible..." seems wrong info. Below always 3 files |
| 770 | are indicated in the tables. |
| 771 | |
| 772 | |
| 773 | .. _packaging-config-filenames: |
| 774 | |
| 775 | Location and names of configuration files |
| 776 | ----------------------------------------- |
| 777 | |
| 778 | The name and location of the configuration files vary slightly across |
| 779 | platforms. On Unix and Mac OS X, these are the three configuration files listed |
| 780 | in the order they are processed: |
| 781 | |
| 782 | +--------------+----------------------------------------------------------+-------+ |
| 783 | | Type of file | Location and filename | Notes | |
| 784 | +==============+==========================================================+=======+ |
| 785 | | system | :file:`{prefix}/lib/python{ver}/packaging/packaging.cfg` | \(1) | |
| 786 | +--------------+----------------------------------------------------------+-------+ |
| 787 | | personal | :file:`$HOME/.pydistutils.cfg` | \(2) | |
| 788 | +--------------+----------------------------------------------------------+-------+ |
| 789 | | local | :file:`setup.cfg` | \(3) | |
| 790 | +--------------+----------------------------------------------------------+-------+ |
| 791 | |
| 792 | Similarly, the configuration files on Windows ---also listed in the order they |
| 793 | are processed--- are these: |
| 794 | |
| 795 | +--------------+-------------------------------------------------+-------+ |
| 796 | | Type of file | Location and filename | Notes | |
| 797 | +==============+=================================================+=======+ |
| 798 | | system | :file:`{prefix}\\Lib\\packaging\\packaging.cfg` | \(4) | |
| 799 | +--------------+-------------------------------------------------+-------+ |
| 800 | | personal | :file:`%HOME%\\pydistutils.cfg` | \(5) | |
| 801 | +--------------+-------------------------------------------------+-------+ |
| 802 | | local | :file:`setup.cfg` | \(3) | |
| 803 | +--------------+-------------------------------------------------+-------+ |
| 804 | |
| 805 | On all platforms, the *personal* file can be temporarily disabled by |
| 806 | means of the `--no-user-cfg` option. |
| 807 | |
| 808 | Notes: |
| 809 | |
| 810 | (1) |
| 811 | Strictly speaking, the system-wide configuration file lives in the directory |
| 812 | where Packaging is installed. |
| 813 | |
| 814 | (2) |
| 815 | On Unix, if the :envvar:`HOME` environment variable is not defined, the |
| 816 | user's home directory will be determined with the :func:`getpwuid` function |
| 817 | from the standard :mod:`pwd` module. Packaging uses the |
| 818 | :func:`os.path.expanduser` function to do this. |
| 819 | |
| 820 | (3) |
| 821 | I.e., in the current directory (usually the location of the setup script). |
| 822 | |
| 823 | (4) |
| 824 | (See also note (1).) Python's default installation prefix is |
| 825 | :file:`C:\\Python`, so the system configuration file is normally |
| 826 | :file:`C:\\Python\\Lib\\packaging\\packaging.cfg`. |
| 827 | |
| 828 | (5) |
| 829 | On Windows, if the :envvar:`HOME` environment variable is not defined, |
| 830 | :envvar:`USERPROFILE` then :envvar:`HOMEDRIVE` and :envvar:`HOMEPATH` will |
| 831 | be tried. Packaging uses the :func:`os.path.expanduser` function to do this. |
| 832 | |
| 833 | |
| 834 | .. _packaging-config-syntax: |
| 835 | |
| 836 | Syntax of configuration files |
| 837 | ----------------------------- |
| 838 | |
| 839 | All Packaging configuration files share the same syntax. Options defined in |
| 840 | them are grouped into sections, and each Packaging command gets its own section. |
| 841 | Additionally, there's a ``global`` section for options that affect every command. |
| 842 | Sections consist of one or more lines containing a single option specified as |
| 843 | ``option = value``. |
| 844 | |
Éric Araujo | 4d15546 | 2011-11-15 11:43:20 +0100 | [diff] [blame] | 845 | .. XXX use dry-run in the next example or use a pysetup option as example |
| 846 | |
Éric Araujo | 3a9f58f | 2011-06-01 20:42:49 +0200 | [diff] [blame] | 847 | For example, here's a complete configuration file that forces all commands to |
| 848 | run quietly by default:: |
| 849 | |
| 850 | [global] |
| 851 | verbose = 0 |
| 852 | |
| 853 | If this was the system configuration file, it would affect all processing |
| 854 | of any Python module distribution by any user on the current system. If it was |
| 855 | installed as your personal configuration file (on systems that support them), |
| 856 | it would affect only module distributions processed by you. Lastly, if it was |
| 857 | used as the :file:`setup.cfg` for a particular module distribution, it would |
| 858 | affect that distribution only. |
| 859 | |
| 860 | .. XXX "(on systems that support them)" seems wrong info |
| 861 | |
| 862 | If you wanted to, you could override the default "build base" directory and |
| 863 | make the :command:`build\*` commands always forcibly rebuild all files with |
| 864 | the following:: |
| 865 | |
| 866 | [build] |
| 867 | build-base = blib |
| 868 | force = 1 |
| 869 | |
| 870 | which corresponds to the command-line arguments:: |
| 871 | |
| 872 | pysetup run build --build-base blib --force |
| 873 | |
| 874 | except that including the :command:`build` command on the command-line means |
| 875 | that command will be run. Including a particular command in configuration files |
| 876 | has no such implication; it only means that if the command is run, the options |
| 877 | for it in the configuration file will apply. (This is also true if you run |
| 878 | other commands that derive values from it.) |
| 879 | |
| 880 | You can find out the complete list of options for any command using the |
| 881 | :option:`--help` option, e.g.:: |
| 882 | |
| 883 | pysetup run build --help |
| 884 | |
| 885 | and you can find out the complete list of global options by using |
| 886 | :option:`--help` without a command:: |
| 887 | |
| 888 | pysetup run --help |
| 889 | |
| 890 | See also the "Reference" section of the "Distributing Python Modules" manual. |
| 891 | |
| 892 | .. XXX no links to the relevant section exist. |
| 893 | |
| 894 | |
| 895 | .. _packaging-building-ext: |
| 896 | |
| 897 | Building extensions: tips and tricks |
| 898 | ==================================== |
| 899 | |
| 900 | Whenever possible, Packaging tries to use the configuration information made |
| 901 | available by the Python interpreter used to run `pysetup`. |
| 902 | For example, the same compiler and linker flags used to compile Python will also |
| 903 | be used for compiling extensions. Usually this will work well, but in |
| 904 | complicated situations this might be inappropriate. This section discusses how |
| 905 | to override the usual Packaging behaviour. |
| 906 | |
| 907 | |
| 908 | .. _packaging-tweak-flags: |
| 909 | |
| 910 | Tweaking compiler/linker flags |
| 911 | ------------------------------ |
| 912 | |
| 913 | Compiling a Python extension written in C or C++ will sometimes require |
| 914 | specifying custom flags for the compiler and linker in order to use a particular |
| 915 | library or produce a special kind of object code. This is especially true if the |
| 916 | extension hasn't been tested on your platform, or if you're trying to |
| 917 | cross-compile Python. |
| 918 | |
| 919 | .. TODO update to new setup.cfg |
| 920 | |
| 921 | In the most general case, the extension author might have foreseen that |
| 922 | compiling the extensions would be complicated, and provided a :file:`Setup` file |
| 923 | for you to edit. This will likely only be done if the module distribution |
| 924 | contains many separate extension modules, or if they often require elaborate |
| 925 | sets of compiler flags in order to work. |
| 926 | |
| 927 | A :file:`Setup` file, if present, is parsed in order to get a list of extensions |
| 928 | to build. Each line in a :file:`Setup` describes a single module. Lines have |
| 929 | the following structure:: |
| 930 | |
| 931 | module ... [sourcefile ...] [cpparg ...] [library ...] |
| 932 | |
| 933 | |
| 934 | Let's examine each of the fields in turn. |
| 935 | |
| 936 | * *module* is the name of the extension module to be built, and should be a |
| 937 | valid Python identifier. You can't just change this in order to rename a module |
| 938 | (edits to the source code would also be needed), so this should be left alone. |
| 939 | |
| 940 | * *sourcefile* is anything that's likely to be a source code file, at least |
| 941 | judging by the filename. Filenames ending in :file:`.c` are assumed to be |
| 942 | written in C, filenames ending in :file:`.C`, :file:`.cc`, and :file:`.c++` are |
| 943 | assumed to be C++, and filenames ending in :file:`.m` or :file:`.mm` are assumed |
| 944 | to be in Objective C. |
| 945 | |
| 946 | * *cpparg* is an argument for the C preprocessor, and is anything starting with |
| 947 | :option:`-I`, :option:`-D`, :option:`-U` or :option:`-C`. |
| 948 | |
| 949 | * *library* is anything ending in :file:`.a` or beginning with :option:`-l` or |
| 950 | :option:`-L`. |
| 951 | |
| 952 | If a particular platform requires a special library on your platform, you can |
| 953 | add it by editing the :file:`Setup` file and running ``pysetup run build``. |
| 954 | For example, if the module defined by the line :: |
| 955 | |
| 956 | foo foomodule.c |
| 957 | |
| 958 | must be linked with the math library :file:`libm.a` on your platform, simply add |
| 959 | :option:`-lm` to the line:: |
| 960 | |
| 961 | foo foomodule.c -lm |
| 962 | |
| 963 | Arbitrary switches intended for the compiler or the linker can be supplied with |
| 964 | the :option:`-Xcompiler` *arg* and :option:`-Xlinker` *arg* options:: |
| 965 | |
| 966 | foo foomodule.c -Xcompiler -o32 -Xlinker -shared -lm |
| 967 | |
| 968 | The next option after :option:`-Xcompiler` and :option:`-Xlinker` will be |
| 969 | appended to the proper command line, so in the above example the compiler will |
| 970 | be passed the :option:`-o32` option, and the linker will be passed |
| 971 | :option:`-shared`. If a compiler option requires an argument, you'll have to |
| 972 | supply multiple :option:`-Xcompiler` options; for example, to pass ``-x c++`` |
| 973 | the :file:`Setup` file would have to contain ``-Xcompiler -x -Xcompiler c++``. |
| 974 | |
| 975 | Compiler flags can also be supplied through setting the :envvar:`CFLAGS` |
| 976 | environment variable. If set, the contents of :envvar:`CFLAGS` will be added to |
| 977 | the compiler flags specified in the :file:`Setup` file. |
| 978 | |
| 979 | |
| 980 | .. _packaging-non-ms-compilers: |
| 981 | |
| 982 | Using non-Microsoft compilers on Windows |
| 983 | ---------------------------------------- |
| 984 | |
| 985 | .. sectionauthor:: Rene Liebscher <R.Liebscher@gmx.de> |
| 986 | |
| 987 | |
| 988 | |
| 989 | Borland/CodeGear C++ |
| 990 | ^^^^^^^^^^^^^^^^^^^^ |
| 991 | |
| 992 | This subsection describes the necessary steps to use Packaging with the Borland |
| 993 | C++ compiler version 5.5. First you have to know that Borland's object file |
| 994 | format (OMF) is different from the format used by the Python version you can |
| 995 | download from the Python or ActiveState Web site. (Python is built with |
| 996 | Microsoft Visual C++, which uses COFF as the object file format.) For this |
| 997 | reason, you have to convert Python's library :file:`python25.lib` into the |
| 998 | Borland format. You can do this as follows: |
| 999 | |
| 1000 | .. Should we mention that users have to create cfg-files for the compiler? |
| 1001 | .. see also http://community.borland.com/article/0,1410,21205,00.html |
| 1002 | |
| 1003 | :: |
| 1004 | |
| 1005 | coff2omf python25.lib python25_bcpp.lib |
| 1006 | |
| 1007 | The :file:`coff2omf` program comes with the Borland compiler. The file |
| 1008 | :file:`python25.lib` is in the :file:`Libs` directory of your Python |
| 1009 | installation. If your extension uses other libraries (zlib, ...) you have to |
| 1010 | convert them too. |
| 1011 | |
| 1012 | The converted files have to reside in the same directories as the normal |
| 1013 | libraries. |
| 1014 | |
| 1015 | How does Packaging manage to use these libraries with their changed names? If |
| 1016 | the extension needs a library (eg. :file:`foo`) Packaging checks first if it |
| 1017 | finds a library with suffix :file:`_bcpp` (eg. :file:`foo_bcpp.lib`) and then |
| 1018 | uses this library. In the case it doesn't find such a special library it uses |
| 1019 | the default name (:file:`foo.lib`.) [#]_ |
| 1020 | |
| 1021 | To let Packaging compile your extension with Borland, C++ you now have to |
| 1022 | type:: |
| 1023 | |
| 1024 | pysetup run build --compiler bcpp |
| 1025 | |
| 1026 | If you want to use the Borland C++ compiler as the default, you could specify |
| 1027 | this in your personal or system-wide configuration file for Packaging (see |
| 1028 | section :ref:`packaging-config-files`.) |
| 1029 | |
| 1030 | |
| 1031 | .. seealso:: |
| 1032 | |
| 1033 | `C++Builder Compiler <http://www.codegear.com/downloads/free/cppbuilder>`_ |
| 1034 | Information about the free C++ compiler from Borland, including links to the |
| 1035 | download pages. |
| 1036 | |
| 1037 | `Creating Python Extensions Using Borland's Free Compiler <http://www.cyberus.ca/~g_will/pyExtenDL.shtml>`_ |
| 1038 | Document describing how to use Borland's free command-line C++ compiler to build |
| 1039 | Python. |
| 1040 | |
| 1041 | |
| 1042 | GNU C / Cygwin / MinGW |
| 1043 | ^^^^^^^^^^^^^^^^^^^^^^ |
| 1044 | |
| 1045 | This section describes the necessary steps to use Packaging with the GNU C/C++ |
| 1046 | compilers in their Cygwin and MinGW distributions. [#]_ For a Python interpreter |
| 1047 | that was built with Cygwin, everything should work without any of these |
| 1048 | following steps. |
| 1049 | |
| 1050 | Not all extensions can be built with MinGW or Cygwin, but many can. Extensions |
| 1051 | most likely to not work are those that use C++ or depend on Microsoft Visual C |
| 1052 | extensions. |
| 1053 | |
| 1054 | To let Packaging compile your extension with Cygwin, you have to type:: |
| 1055 | |
| 1056 | pysetup run build --compiler=cygwin |
| 1057 | |
| 1058 | and for Cygwin in no-cygwin mode [#]_ or for MinGW, type:: |
| 1059 | |
| 1060 | pysetup run build --compiler=mingw32 |
| 1061 | |
| 1062 | If you want to use any of these options/compilers as default, you should |
| 1063 | consider writing it in your personal or system-wide configuration file for |
| 1064 | Packaging (see section :ref:`packaging-config-files`.) |
| 1065 | |
| 1066 | Older Versions of Python and MinGW |
| 1067 | """""""""""""""""""""""""""""""""" |
| 1068 | The following instructions only apply if you're using a version of Python |
| 1069 | inferior to 2.4.1 with a MinGW inferior to 3.0.0 (with |
| 1070 | :file:`binutils-2.13.90-20030111-1`). |
| 1071 | |
| 1072 | These compilers require some special libraries. This task is more complex than |
| 1073 | for Borland's C++, because there is no program to convert the library. First |
| 1074 | you have to create a list of symbols which the Python DLL exports. (You can find |
| 1075 | a good program for this task at |
| 1076 | http://www.emmestech.com/software/pexports-0.43/download_pexports.html). |
| 1077 | |
| 1078 | .. I don't understand what the next line means. --amk |
| 1079 | (inclusive the references on data structures.) |
| 1080 | |
| 1081 | :: |
| 1082 | |
| 1083 | pexports python25.dll > python25.def |
| 1084 | |
| 1085 | The location of an installed :file:`python25.dll` will depend on the |
| 1086 | installation options and the version and language of Windows. In a "just for |
| 1087 | me" installation, it will appear in the root of the installation directory. In |
| 1088 | a shared installation, it will be located in the system directory. |
| 1089 | |
| 1090 | Then you can create from these information an import library for gcc. :: |
| 1091 | |
| 1092 | /cygwin/bin/dlltool --dllname python25.dll --def python25.def --output-lib libpython25.a |
| 1093 | |
| 1094 | The resulting library has to be placed in the same directory as |
| 1095 | :file:`python25.lib`. (Should be the :file:`libs` directory under your Python |
| 1096 | installation directory.) |
| 1097 | |
| 1098 | If your extension uses other libraries (zlib,...) you might have to convert |
| 1099 | them too. The converted files have to reside in the same directories as the |
| 1100 | normal libraries do. |
| 1101 | |
| 1102 | |
| 1103 | .. seealso:: |
| 1104 | |
| 1105 | `Building Python modules on MS Windows platform with MinGW <http://www.zope.org/Members/als/tips/win32_mingw_modules>`_ |
| 1106 | Information about building the required libraries for the MinGW |
| 1107 | environment. |
| 1108 | |
| 1109 | |
| 1110 | .. rubric:: Footnotes |
| 1111 | |
| 1112 | .. [#] This also means you could replace all existing COFF-libraries with |
| 1113 | OMF-libraries of the same name. |
| 1114 | |
| 1115 | .. [#] Check http://sources.redhat.com/cygwin/ and http://www.mingw.org/ for |
| 1116 | more information. |
| 1117 | |
| 1118 | .. [#] Then you have no POSIX emulation available, but you also don't need |
| 1119 | :file:`cygwin1.dll`. |