| \documentclass{howto} |
| \usepackage{distutils} |
| |
| % TODO: |
| % Fill in XXX comments |
| |
| \title{Installing Python Modules} |
| |
| % The audience for this document includes people who don't know anything |
| % about Python and aren't about to learn the language just in order to |
| % install and maintain it for their users, i.e. system administrators. |
| % Thus, I have to be sure to explain the basics at some point: |
| % sys.path and PYTHONPATH at least. Should probably give pointers to |
| % other docs on "import site", PYTHONSTARTUP, PYTHONHOME, etc. |
| % |
| % Finally, it might be useful to include all the material from my "Care |
| % and Feeding of a Python Installation" talk in here somewhere. Yow! |
| |
| \input{boilerplate} |
| |
| \author{Greg Ward} |
| \authoraddress{ |
| \strong{Python Software Foundation}\\ |
| Email: \email{distutils-sig@python.org} |
| } |
| |
| \makeindex |
| |
| \begin{document} |
| |
| \maketitle |
| |
| \begin{abstract} |
| \noindent |
| This document describes the Python Distribution Utilities |
| (``Distutils'') from the end-user's point-of-view, describing how to |
| extend the capabilities of a standard Python installation by building |
| and installing third-party Python modules and extensions. |
| \end{abstract} |
| |
| %\begin{abstract} |
| %\noindent |
| %Abstract this! |
| %\end{abstract} |
| |
| |
| % The ugly "%begin{latexonly}" pseudo-environment supresses the table |
| % of contents for HTML generation. |
| % |
| %begin{latexonly} |
| \tableofcontents |
| %end{latexonly} |
| |
| |
| \section{Introduction} |
| \label{intro} |
| |
| Although Python's extensive standard library covers many programming |
| needs, there often comes a time when you need to add some new |
| functionality to your Python installation in the form of third-party |
| modules. This might be necessary to support your own programming, or to |
| support an application that you want to use and that happens to be |
| written in Python. |
| |
| In the past, there has been little support for adding third-party |
| modules to an existing Python installation. With the introduction of |
| the Python Distribution Utilities (Distutils for short) in Python 2.0, |
| this changed. |
| |
| This document is aimed primarily at the people who need to install |
| third-party Python modules: end-users and system administrators who just |
| need to get some Python application running, and existing Python |
| programmers who want to add some new goodies to their toolbox. You |
| don't need to know Python to read this document; there will be some |
| brief forays into using Python's interactive mode to explore your |
| installation, but that's it. If you're looking for information on how |
| to distribute your own Python modules so that others may use them, see |
| the \citetitle[../dist/dist.html]{Distributing Python Modules} manual. |
| |
| |
| \subsection{Best case: trivial installation} |
| \label{trivial-install} |
| |
| In the best case, someone will have prepared a special version of the |
| module distribution you want to install that is targeted specifically at |
| your platform and is installed just like any other software on your |
| platform. For example, the module developer might make an executable |
| installer available for Windows users, an RPM package for users of |
| RPM-based Linux systems (Red Hat, SuSE, Mandrake, and many others), a |
| Debian package for users of Debian-based Linux systems, and so forth. |
| |
| In that case, you would download the installer appropriate to your |
| platform and do the obvious thing with it: run it if it's an executable |
| installer, \code{rpm --install} it if it's an RPM, etc. You don't need |
| to run Python or a setup script, you don't need to compile |
| anything---you might not even need to read any instructions (although |
| it's always a good idea to do so anyways). |
| |
| Of course, things will not always be that easy. You might be interested |
| in a module distribution that doesn't have an easy-to-use installer for |
| your platform. In that case, you'll have to start with the source |
| distribution released by the module's author/maintainer. Installing |
| from a source distribution is not too hard, as long as the modules are |
| packaged in the standard way. The bulk of this document is about |
| building and installing modules from standard source distributions. |
| |
| |
| \subsection{The new standard: Distutils} |
| \label{new-standard} |
| |
| If you download a module source distribution, you can tell pretty |
| quickly if it was packaged and distributed in the standard way, i.e. |
| using the Distutils. First, the distribution's name and version number |
| will be featured prominently in the name of the downloaded archive, e.g. |
| \file{foo-1.0.tar.gz} or \file{widget-0.9.7.zip}. Next, the archive |
| will unpack into a similarly-named directory: \file{foo-1.0} or |
| \file{widget-0.9.7}. Additionally, the distribution will contain a |
| setup script \file{setup.py}, and a file named \file{README.txt} or possibly |
| just \file{README}, which should explain that building and installing the |
| module distribution is a simple matter of running |
| |
| \begin{verbatim} |
| python setup.py install |
| \end{verbatim} |
| |
| If all these things are true, then you already know how to build and |
| install the modules you've just downloaded: Run the command above. |
| Unless you need to install things in a non-standard way or customize the |
| build process, you don't really need this manual. Or rather, the above |
| command is everything you need to get out of this manual. |
| |
| |
| \section{Standard Build and Install} |
| \label{standard-install} |
| |
| As described in section~\ref{new-standard}, building and installing |
| a module distribution using the Distutils is usually one simple command: |
| |
| \begin{verbatim} |
| python setup.py install |
| \end{verbatim} |
| |
| On \UNIX, you'd run this command from a shell prompt; on Windows, you |
| have to open a command prompt window (``DOS box'') and do it there; on |
| Mac OS, things are a tad more complicated (see below). |
| |
| |
| \subsection{Platform variations} |
| \label{platform-variations} |
| |
| You should always run the setup command from the distribution root |
| directory, i.e. the top-level subdirectory that the module source |
| distribution unpacks into. For example, if you've just downloaded a |
| module source distribution \file{foo-1.0.tar.gz} onto a |
| \UNIX{} system, the normal thing to do is: |
| |
| \begin{verbatim} |
| gunzip -c foo-1.0.tar.gz | tar xf - # unpacks into directory foo-1.0 |
| cd foo-1.0 |
| python setup.py install |
| \end{verbatim} |
| |
| On Windows, you'd probably download \file{foo-1.0.zip}. If you |
| downloaded the archive file to \file{C:\textbackslash{}Temp}, then it |
| would unpack into \file{C:\textbackslash{}Temp\textbackslash{}foo-1.0}; |
| you can use either a archive manipulator with a graphical user interface |
| (such as WinZip) or a command-line tool (such as \program{unzip} or |
| \program{pkunzip}) to unpack the archive. Then, open a command prompt |
| window (``DOS box''), and run: |
| |
| \begin{verbatim} |
| cd c:\Temp\foo-1.0 |
| python setup.py install |
| \end{verbatim} |
| |
| On Mac OS 9, you double-click the \file{setup.py} script. It will bring |
| up a dialog where you can select the \command{install} command. Then |
| selecting the \command{run} button will install your distribution. |
| The dialog is built dynamically, so all commands and options for this |
| specific distribution are listed. |
| |
| \subsection{Splitting the job up} |
| \label{splitting-up} |
| |
| Running \code{setup.py install} builds and installs all modules in one |
| run. If you prefer to work incrementally---especially useful if you |
| want to customize the build process, or if things are going wrong---you |
| can use the setup script to do one thing at a time. This is |
| particularly helpful when the build and install will be done by |
| different users---for example, you might want to build a module distribution |
| and hand it off to a system administrator for installation (or do it |
| yourself, with super-user privileges). |
| |
| For example, you can build everything in one step, and then install |
| everything in a second step, by invoking the setup script twice: |
| |
| \begin{verbatim} |
| python setup.py build |
| python setup.py install |
| \end{verbatim} |
| |
| If you do this, you will notice that running the \command{install} |
| command first runs the \command{build} command, which---in this |
| case---quickly notices that it has nothing to do, since everything in |
| the \file{build} directory is up-to-date. |
| |
| You may not need this ability to break things down often if all you do |
| is install modules downloaded off the 'net, but it's very handy for more |
| advanced tasks. If you get into distributing your own Python modules |
| and extensions, you'll run lots of individual Distutils commands on |
| their own. |
| |
| |
| \subsection{How building works} |
| \label{how-build-works} |
| |
| As implied above, the \command{build} command is responsible for putting |
| the files to install into a \emph{build directory}. By default, this is |
| \file{build} under the distribution root; if you're excessively |
| concerned with speed, or want to keep the source tree pristine, you can |
| change the build directory with the \longprogramopt{build-base} option. |
| For example: |
| |
| \begin{verbatim} |
| python setup.py build --build-base=/tmp/pybuild/foo-1.0 |
| \end{verbatim} |
| |
| (Or you could do this permanently with a directive in your system or |
| personal Distutils configuration file; see |
| section~\ref{config-files}.) Normally, this isn't necessary. |
| |
| The default layout for the build tree is as follows: |
| |
| \begin{verbatim} |
| --- build/ --- lib/ |
| or |
| --- build/ --- lib.<plat>/ |
| temp.<plat>/ |
| \end{verbatim} |
| |
| where \code{<plat>} expands to a brief description of the current |
| OS/hardware platform and Python version. The first form, with just a |
| \file{lib} directory, is used for ``pure module distributions''---that |
| is, module distributions that include only pure Python modules. If a |
| module distribution contains any extensions (modules written in C/\Cpp), |
| then the second form, with two \code{<plat>} directories, is used. In |
| that case, the \file{temp.\filevar{plat}} directory holds temporary |
| files generated by the compile/link process that don't actually get |
| installed. In either case, the \file{lib} (or |
| \file{lib.\filevar{plat}}) directory contains all Python modules (pure |
| Python and extensions) that will be installed. |
| |
| In the future, more directories will be added to handle Python scripts, |
| documentation, binary executables, and whatever else is needed to handle |
| the job of installing Python modules and applications. |
| |
| |
| \subsection{How installation works} |
| \label{how-install-works} |
| |
| After the \command{build} command runs (whether you run it explicitly, |
| or the \command{install} command does it for you), the work of the |
| \command{install} command is relatively simple: all it has to do is copy |
| everything under \file{build/lib} (or \file{build/lib.\filevar{plat}}) |
| to your chosen installation directory. |
| |
| If you don't choose an installation directory---i.e., if you just run |
| \code{setup.py install}---then the \command{install} command installs to |
| the standard location for third-party Python modules. This location |
| varies by platform and by how you built/installed Python itself. On |
| \UNIX{} and Mac OS, it also depends on whether the module distribution |
| being installed is pure Python or contains extensions (``non-pure''): |
| \begin{tableiv}{l|l|l|c}{textrm}% |
| {Platform}{Standard installation location}{Default value}{Notes} |
| \lineiv{\UNIX{} (pure)} |
| {\filenq{\filevar{prefix}/lib/python2.0/site-packages}} |
| {\filenq{/usr/local/lib/python2.0/site-packages}} |
| {(1)} |
| \lineiv{\UNIX{} (non-pure)} |
| {\filenq{\filevar{exec-prefix}/lib/python2.0/site-packages}} |
| {\filenq{/usr/local/lib/python2.0/site-packages}} |
| {(1)} |
| \lineiv{Windows} |
| {\filenq{\filevar{prefix}}} |
| {\filenq{C:\textbackslash{}Python}} |
| {(2)} |
| \lineiv{Mac OS (pure)} |
| {\filenq{\filevar{prefix}:Lib:site-packages}} |
| {\filenq{Python:Lib:site-packages}} |
| {} |
| \lineiv{Mac OS (non-pure)} |
| {\filenq{\filevar{prefix}:Lib:site-packages}} |
| {\filenq{Python:Lib:site-packages}} |
| {} |
| \end{tableiv} |
| |
| \noindent Notes: |
| \begin{description} |
| \item[(1)] Most Linux distributions include Python as a standard part of |
| the system, so \filevar{prefix} and \filevar{exec-prefix} are usually |
| both \file{/usr} on Linux. If you build Python yourself on Linux (or |
| any \UNIX-like system), the default \filevar{prefix} and |
| \filevar{exec-prefix} are \file{/usr/local}. |
| \item[(2)] The default installation directory on Windows was |
| \file{C:\textbackslash{}Program Files\textbackslash{}Python} under |
| Python 1.6a1, 1.5.2, and earlier. |
| \end{description} |
| |
| \filevar{prefix} and \filevar{exec-prefix} stand for the directories |
| that Python is installed to, and where it finds its libraries at |
| run-time. They are always the same under Windows and Mac OS, and very |
| often the same under \UNIX. You can find out what your Python |
| installation uses for \filevar{prefix} and \filevar{exec-prefix} by |
| running Python in interactive mode and typing a few simple commands. |
| Under \UNIX, just type \code{python} at the shell prompt. Under |
| Windows, choose \menuselection{Start \sub Programs \sub Python |
| 2.1 \sub Python (command line)}. Under Mac OS 9, start \file{PythonInterpreter}. |
| Once the interpreter is started, you type Python code at the |
| prompt. For example, on my Linux system, I type the three Python |
| statements shown below, and get the output as shown, to find out my |
| \filevar{prefix} and \filevar{exec-prefix}: |
| |
| \begin{verbatim} |
| Python 1.5.2 (#1, Apr 18 1999, 16:03:16) [GCC pgcc-2.91.60 19981201 (egcs-1.1.1 on linux2 |
| Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam |
| >>> import sys |
| >>> sys.prefix |
| '/usr' |
| >>> sys.exec_prefix |
| '/usr' |
| \end{verbatim} |
| |
| If you don't want to install modules to the standard location, or if you |
| don't have permission to write there, then you need to read about |
| alternate installations in section~\ref{alt-install}. If you want to |
| customize your installation directories more heavily, see |
| section~\ref{custom-install} on custom installations. |
| |
| |
| % This rather nasty macro is used to generate the tables that describe |
| % each installation scheme. It's nasty because it takes two arguments |
| % for each "slot" in an installation scheme, there will soon be more |
| % than five of these slots, and TeX has a limit of 10 arguments to a |
| % macro. Uh-oh. |
| |
| \newcommand{\installscheme}[8] |
| {\begin{tableiii}{l|l|l}{textrm} |
| {Type of file} |
| {Installation Directory} |
| {Override option} |
| \lineiii{pure module distribution} |
| {\filevar{#1}\filenq{#2}} |
| {\longprogramopt{install-purelib}} |
| \lineiii{non-pure module distribution} |
| {\filevar{#3}\filenq{#4}} |
| {\longprogramopt{install-platlib}} |
| \lineiii{scripts} |
| {\filevar{#5}\filenq{#6}} |
| {\longprogramopt{install-scripts}} |
| \lineiii{data} |
| {\filevar{#7}\filenq{#8}} |
| {\longprogramopt{install-data}} |
| \end{tableiii}} |
| |
| |
| \section{Alternate Installation} |
| \label{alt-install} |
| |
| Often, it is necessary or desirable to install modules to a location |
| other than the standard location for third-party Python modules. For |
| example, on a \UNIX{} system you might not have permission to write to the |
| standard third-party module directory. Or you might wish to try out a |
| module before making it a standard part of your local Python |
| installation. This is especially true when upgrading a distribution |
| already present: you want to make sure your existing base of scripts |
| still works with the new version before actually upgrading. |
| |
| The Distutils \command{install} command is designed to make installing |
| module distributions to an alternate location simple and painless. The |
| basic idea is that you supply a base directory for the installation, and |
| the \command{install} command picks a set of directories (called an |
| \emph{installation scheme}) under this base directory in which to |
| install files. The details differ across platforms, so read whichever |
| of the following sections applies to you. |
| |
| |
| \subsection{Alternate installation: \UNIX{} (the home scheme)} |
| \label{alt-install-prefix} |
| |
| Under \UNIX, there are two ways to perform an alternate installation. |
| The ``prefix scheme'' is similar to how alternate installation works |
| under Windows and Mac OS, but is not necessarily the most useful way to |
| maintain a personal Python library. Hence, we document the more |
| convenient and commonly useful ``home scheme'' first. |
| |
| The idea behind the ``home scheme'' is that you build and maintain a |
| personal stash of Python modules, probably under your home directory. |
| Installing a new module distribution is as simple as |
| |
| \begin{verbatim} |
| python setup.py install --home=<dir> |
| \end{verbatim} |
| |
| where you can supply any directory you like for the \longprogramopt{home} |
| option. Lazy typists can just type a tilde (\code{\textasciitilde}); the |
| \command{install} command will expand this to your home directory: |
| |
| \begin{verbatim} |
| python setup.py install --home=~ |
| \end{verbatim} |
| |
| The \longprogramopt{home} option defines the installation base |
| directory. Files are installed to the following directories under the |
| installation base as follows: |
| \installscheme{home}{/lib/python} |
| {home}{/lib/python} |
| {home}{/bin} |
| {home}{/share} |
| |
| \subsection{Alternate installation: \UNIX{} (the prefix scheme)} |
| \label{alt-install-home} |
| |
| The ``prefix scheme'' is useful when you wish to use one Python |
| installation to perform the build/install (i.e., to run the setup |
| script), but install modules into the third-party module directory of a |
| different Python installation (or something that looks like a different |
| Python installation). If this sounds a trifle unusual, it is---that's |
| why the ``home scheme'' comes first. However, there are at least two |
| known cases where the prefix scheme will be useful. |
| |
| First, consider that many Linux distributions put Python in \file{/usr}, |
| rather than the more traditional \file{/usr/local}. This is entirely |
| appropriate, since in those cases Python is part of ``the system'' |
| rather than a local add-on. However, if you are installing Python |
| modules from source, you probably want them to go in |
| \file{/usr/local/lib/python2.\filevar{X}} rather than |
| \file{/usr/lib/python2.\filevar{X}}. This can be done with |
| |
| \begin{verbatim} |
| /usr/bin/python setup.py install --prefix=/usr/local |
| \end{verbatim} |
| |
| Another possibility is a network filesystem where the name used to write |
| to a remote directory is different from the name used to read it: for |
| example, the Python interpreter accessed as \file{/usr/local/bin/python} |
| might search for modules in \file{/usr/local/lib/python2.\filevar{X}}, |
| but those modules would have to be installed to, say, |
| \file{/mnt/\filevar{@server}/export/lib/python2.\filevar{X}}. This |
| could be done with |
| |
| \begin{verbatim} |
| /usr/local/bin/python setup.py install --prefix=/mnt/@server/export |
| \end{verbatim} |
| |
| In either case, the \longprogramopt{prefix} option defines the |
| installation base, and the \longprogramopt{exec-prefix} option defines |
| the platform-specific installation base, which is used for |
| platform-specific files. (Currently, this just means non-pure module |
| distributions, but could be expanded to C libraries, binary executables, |
| etc.) If \longprogramopt{exec-prefix} is not supplied, it defaults to |
| \longprogramopt{prefix}. Files are installed as follows: |
| |
| \installscheme{prefix}{/lib/python2.\filevar{X}/site-packages} |
| {exec-prefix}{/lib/python2.\filevar{X}/site-packages} |
| {prefix}{/bin} |
| {prefix}{/share} |
| |
| There is no requirement that \longprogramopt{prefix} or |
| \longprogramopt{exec-prefix} actually point to an alternate Python |
| installation; if the directories listed above do not already exist, they |
| are created at installation time. |
| |
| Incidentally, the real reason the prefix scheme is important is simply |
| that a standard \UNIX{} installation uses the prefix scheme, but with |
| \longprogramopt{prefix} and \longprogramopt{exec-prefix} supplied by |
| Python itself as \code{sys.prefix} and \code{sys.exec\_prefix}. Thus, |
| you might think you'll never use the prefix scheme, but every time you |
| run \code{python setup.py install} without any other options, you're |
| using it. |
| |
| Note that installing extensions to an alternate Python installation has |
| no effect on how those extensions are built: in particular, the Python |
| header files (\file{Python.h} and friends) installed with the Python |
| interpreter used to run the setup script will be used in compiling |
| extensions. It is your responsibility to ensure that the interpreter |
| used to run extensions installed in this way is compatible with the |
| interpreter used to build them. The best way to do this is to ensure |
| that the two interpreters are the same version of Python (possibly |
| different builds, or possibly copies of the same build). (Of course, if |
| your \longprogramopt{prefix} and \longprogramopt{exec-prefix} don't even |
| point to an alternate Python installation, this is immaterial.) |
| |
| |
| \subsection{Alternate installation: Windows} |
| \label{alt-install-windows} |
| |
| Since Windows has no conception of a user's home directory, and since |
| the standard Python installation under Windows is simpler than that |
| under \UNIX, there's no point in having separate \longprogramopt{prefix} |
| and \longprogramopt{home} options. Just use the \longprogramopt{prefix} |
| option to specify a base directory, e.g. |
| |
| \begin{verbatim} |
| python setup.py install --prefix="\Temp\Python" |
| \end{verbatim} |
| |
| to install modules to the |
| \file{\textbackslash{}Temp\textbackslash{}Python} directory on the |
| current drive. |
| |
| The installation base is defined by the \longprogramopt{prefix} option; |
| the \longprogramopt{exec-prefix} option is not supported under Windows. |
| Files are installed as follows: |
| \installscheme{prefix}{} |
| {prefix}{} |
| {prefix}{\textbackslash{}Scripts} |
| {prefix}{\textbackslash{}Data} |
| |
| |
| \subsection{Alternate installation: Mac OS 9} |
| \label{alt-install-macos} |
| |
| % XXX Mac OS X? |
| |
| Like Windows, Mac OS has no notion of home directories (or even of |
| users), and a fairly simple standard Python installation. Thus, only a |
| \longprogramopt{prefix} option is needed. It defines the installation |
| base, and files are installed under it as follows: |
| |
| \installscheme{prefix}{:Lib:site-packages} |
| {prefix}{:Lib:site-packages} |
| {prefix}{:Scripts} |
| {prefix}{:Data} |
| |
| See section~\ref{platform-variations} for information on supplying |
| command-line arguments to the setup script with MacPython. |
| |
| |
| \section{Custom Installation} |
| \label{custom-install} |
| |
| Sometimes, the alternate installation schemes described in |
| section~\ref{alt-install} just don't do what you want. You might |
| want to tweak just one or two directories while keeping everything under |
| the same base directory, or you might want to completely redefine the |
| installation scheme. In either case, you're creating a \emph{custom |
| installation scheme}. |
| |
| You probably noticed the column of ``override options'' in the tables |
| describing the alternate installation schemes above. Those options are |
| how you define a custom installation scheme. These override options can |
| be relative, absolute, or explicitly defined in terms of one of the |
| installation base directories. (There are two installation base |
| directories, and they are normally the same---they only differ when you |
| use the \UNIX{} ``prefix scheme'' and supply different |
| \longprogramopt{prefix} and \longprogramopt{exec-prefix} options.) |
| |
| For example, say you're installing a module distribution to your home |
| directory under \UNIX---but you want scripts to go in |
| \file{\textasciitilde/scripts} rather than \file{\textasciitilde/bin}. |
| As you might expect, you can override this directory with the |
| \longprogramopt{install-scripts} option; in this case, it makes most |
| sense to supply a relative path, which will be interpreted relative to |
| the installation base directory (your home directory, in this case): |
| |
| \begin{verbatim} |
| python setup.py install --home=~ --install-scripts=scripts |
| \end{verbatim} |
| |
| Another \UNIX{} example: suppose your Python installation was built and |
| installed with a prefix of \file{/usr/local/python}, so under a standard |
| installation scripts will wind up in \file{/usr/local/python/bin}. If |
| you want them in \file{/usr/local/bin} instead, you would supply this |
| absolute directory for the \longprogramopt{install-scripts} option: |
| |
| \begin{verbatim} |
| python setup.py install --install-scripts=/usr/local/bin |
| \end{verbatim} |
| |
| (This performs an installation using the ``prefix scheme,'' where the |
| prefix is whatever your Python interpreter was installed with--- |
| \file{/usr/local/python} in this case.) |
| |
| If you maintain Python on Windows, you might want third-party modules to |
| live in a subdirectory of \filevar{prefix}, rather than right in |
| \filevar{prefix} itself. This is almost as easy as customizing the |
| script installation directory---you just have to remember that there are |
| two types of modules to worry about, pure modules and non-pure modules |
| (i.e., modules from a non-pure distribution). For example: |
| |
| \begin{verbatim} |
| python setup.py install --install-purelib=Site --install-platlib=Site |
| \end{verbatim} |
| |
| The specified installation directories are relative to |
| \filevar{prefix}. Of course, you also have to ensure that these |
| directories are in Python's module search path, such as by putting a |
| \file{.pth} file in \filevar{prefix}. See section~\ref{search-path} |
| to find out how to modify Python's search path. |
| |
| If you want to define an entire installation scheme, you just have to |
| supply all of the installation directory options. The recommended way |
| to do this is to supply relative paths; for example, if you want to |
| maintain all Python module-related files under \file{python} in your |
| home directory, and you want a separate directory for each platform that |
| you use your home directory from, you might define the following |
| installation scheme: |
| |
| \begin{verbatim} |
| python setup.py install --home=~ \ |
| --install-purelib=python/lib \ |
| --install-platlib=python/lib.$PLAT \ |
| --install-scripts=python/scripts |
| --install-data=python/data |
| \end{verbatim} |
| % $ % -- bow to font-lock |
| |
| or, equivalently, |
| |
| \begin{verbatim} |
| python setup.py install --home=~/python \ |
| --install-purelib=lib \ |
| --install-platlib='lib.$PLAT' \ |
| --install-scripts=scripts |
| --install-data=data |
| \end{verbatim} |
| % $ % -- bow to font-lock |
| |
| \code{\$PLAT} is not (necessarily) an environment variable---it will be |
| expanded by the Distutils as it parses your command line options, just |
| as it does when parsing your configuration file(s). |
| |
| Obviously, specifying the entire installation scheme every time you |
| install a new module distribution would be very tedious. Thus, you can |
| put these options into your Distutils config file (see |
| section~\ref{config-files}): |
| |
| \begin{verbatim} |
| [install] |
| install-base=$HOME |
| install-purelib=python/lib |
| install-platlib=python/lib.$PLAT |
| install-scripts=python/scripts |
| install-data=python/data |
| \end{verbatim} |
| |
| or, equivalently, |
| |
| \begin{verbatim} |
| [install] |
| install-base=$HOME/python |
| install-purelib=lib |
| install-platlib=lib.$PLAT |
| install-scripts=scripts |
| install-data=data |
| \end{verbatim} |
| |
| Note that these two are \emph{not} equivalent if you supply a different |
| installation base directory when you run the setup script. For example, |
| |
| \begin{verbatim} |
| python setup.py --install-base=/tmp |
| \end{verbatim} |
| |
| would install pure modules to \filevar{/tmp/python/lib} in the first |
| case, and to \filevar{/tmp/lib} in the second case. (For the second |
| case, you probably want to supply an installation base of |
| \file{/tmp/python}.) |
| |
| You probably noticed the use of \code{\$HOME} and \code{\$PLAT} in the |
| sample configuration file input. These are Distutils configuration |
| variables, which bear a strong resemblance to environment variables. |
| In fact, you can use environment variables in config files on |
| platforms that have such a notion but the Distutils additionally |
| define a few extra variables that may not be in your environment, such |
| as \code{\$PLAT}. (And of course, on systems that don't have |
| environment variables, such as Mac OS 9, the configuration |
| variables supplied by the Distutils are the only ones you can use.) |
| See section~\ref{config-files} for details. |
| |
| % XXX need some Windows and Mac OS examples---when would custom |
| % installation schemes be needed on those platforms? |
| |
| |
| % XXX I'm not sure where this section should go. |
| \subsection{Modifying Python's Search Path} |
| \label{search-path} |
| |
| When the Python interpreter executes an \keyword{import} statement, it |
| searches for both Python code and extension modules along a search |
| path. A default value for the path is configured into the Python |
| binary when the interpreter is built. You can determine the path by |
| importing the \module{sys} module and printing the value of |
| \code{sys.path}. |
| |
| \begin{verbatim} |
| $ python |
| Python 2.2 (#11, Oct 3 2002, 13:31:27) |
| [GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-112)] on linux2 |
| Type ``help'', ``copyright'', ``credits'' or ``license'' for more information. |
| >>> import sys |
| >>> sys.path |
| ['', '/usr/local/lib/python2.3', '/usr/local/lib/python2.3/plat-linux2', |
| '/usr/local/lib/python2.3/lib-tk', '/usr/local/lib/python2.3/lib-dynload', |
| '/usr/local/lib/python2.3/site-packages'] |
| >>> |
| \end{verbatim} % $ <-- bow to font-lock |
| |
| The null string in \code{sys.path} represents the current working |
| directory. |
| |
| The expected convention for locally installed packages is to put them |
| in the \file{.../site-packages/} directory, but you may want to |
| install Python modules into some arbitrary directory. For example, |
| your site may have a convention of keeping all software related to the |
| web server under \file{/www}. Add-on Python modules might then belong |
| in \file{/www/python}, and in order to import them, this directory |
| must be added to \code{sys.path}. There are several different ways to |
| add the directory. |
| |
| The most convenient way is to add a path configuration file to a |
| directory that's already on Python's path, usually to the |
| \file{.../site-packages/} directory. Path configuration files have an |
| extension of \file{.pth}, and each line must contain a single path |
| that will be appended to \code{sys.path}. (Because the new paths are |
| appended to \code{sys.path}, modules in the added directories will not |
| override standard modules. This means you can't use this mechanism |
| for installing fixed versions of standard modules.) |
| |
| Paths can be absolute or relative, in which case they're relative to |
| the directory containing the \file{.pth} file. Any directories added |
| to the search path will be scanned in turn for \file{.pth} files. See |
| \citetitle[http://www.python.org/dev/doc/devel/lib/module-site.html]{the |
| documentation for the \module{site} module} for more information. |
| |
| A slightly less convenient way is to edit the \file{site.py} file in |
| Python's standard library, and modify \code{sys.path}. \file{site.py} |
| is automatically imported when the Python interpreter is executed, |
| unless the \programopt{-S} switch is supplied to suppress this |
| behaviour. So you could simply edit \file{site.py} and add two lines to it: |
| |
| \begin{verbatim} |
| import sys |
| sys.path.append('/www/python/') |
| \end{verbatim} |
| |
| However, if you reinstall the same major version of Python (perhaps |
| when upgrading from 2.2 to 2.2.2, for example) \file{site.py} will be |
| overwritten by the stock version. You'd have to remember that it was |
| modified and save a copy before doing the installation. |
| |
| There are two environment variables that can modify \code{sys.path}. |
| \envvar{PYTHONHOME} sets an alternate value for the prefix of the |
| Python installation. For example, if \envvar{PYTHONHOME} is set to |
| \samp{/www/python}, the search path will be set to \code{['', |
| '/www/python/lib/python2.2/', '/www/python/lib/python2.3/plat-linux2', |
| ...]}. |
| |
| The \envvar{PYTHONPATH} variable can be set to a list of paths that |
| will be added to the beginning of \code{sys.path}. For example, if |
| \envvar{PYTHONPATH} is set to \samp{/www/python:/opt/py}, the search |
| path will begin with \code{['/www/python', '/opt/py']}. (Note that |
| directories must exist in order to be added to \code{sys.path}; the |
| \module{site} module removes paths that don't exist.) |
| |
| Finally, \code{sys.path} is just a regular Python list, so any Python |
| application can modify it by adding or removing entries. |
| |
| |
| \section{Distutils Configuration Files} |
| \label{config-files} |
| |
| As mentioned above, you can use Distutils configuration files to record |
| personal or site preferences for any Distutils options. That is, any |
| option to any command can be stored in one of two or three (depending on |
| your platform) configuration files, which will be consulted before the |
| command-line is parsed. This means that configuration files will |
| override default values, and the command-line will in turn override |
| configuration files. Furthermore, if multiple configuration files |
| apply, values from ``earlier'' files are overridden by ``later'' files. |
| |
| |
| \subsection{Location and names of config files} |
| \label{config-filenames} |
| |
| The names and locations of the configuration files vary slightly across |
| platforms. On \UNIX, the three configuration files (in the order they |
| are processed) are: |
| \begin{tableiii}{l|l|c}{textrm} |
| {Type of file}{Location and filename}{Notes} |
| \lineiii{system}{\filenq{\filevar{prefix}/lib/python\filevar{ver}/distutils/distutils.cfg}}{(1)} |
| \lineiii{personal}{\filenq{\$HOME/.pydistutils.cfg}}{(2)} |
| \lineiii{local}{\filenq{setup.cfg}}{(3)} |
| \end{tableiii} |
| |
| On Windows, the configuration files are: |
| \begin{tableiii}{l|l|c}{textrm} |
| {Type of file}{Location and filename}{Notes} |
| \lineiii{system}{\filenq{\filevar{prefix}\textbackslash{}Lib\textbackslash{}distutils\textbackslash{}distutils.cfg}}{(4)} |
| \lineiii{personal}{\filenq{\%HOME\%\textbackslash{}pydistutils.cfg}}{(5)} |
| \lineiii{local}{\filenq{setup.cfg}}{(3)} |
| \end{tableiii} |
| |
| And on Mac OS, they are: |
| \begin{tableiii}{l|l|c}{textrm} |
| {Type of file}{Location and filename}{Notes} |
| \lineiii{system}{\filenq{\filevar{prefix}:Lib:distutils:distutils.cfg}}{(6)} |
| \lineiii{personal}{N/A}{} |
| \lineiii{local}{\filenq{setup.cfg}}{(3)} |
| \end{tableiii} |
| |
| \noindent Notes: |
| \begin{description} |
| \item[(1)] Strictly speaking, the system-wide configuration file lives |
| in the directory where the Distutils are installed; under Python 1.6 |
| and later on \UNIX, this is as shown. For Python 1.5.2, the Distutils |
| will normally be installed to |
| \file{\filevar{prefix}/lib/python1.5/site-packages/distutils}, |
| so the system configuration file should be put there under Python |
| 1.5.2. |
| \item[(2)] On \UNIX, if the \envvar{HOME} environment variable is not |
| defined, the user's home directory will be determined with the |
| \function{getpwuid()} function from the standard |
| \ulink{\module{pwd}}{../lib/module-pwd.html} module. |
| \item[(3)] I.e., in the current directory (usually the location of the |
| setup script). |
| \item[(4)] (See also note (1).) Under Python 1.6 and later, Python's |
| default ``installation prefix'' is \file{C:\textbackslash{}Python}, so |
| the system configuration file is normally |
| \file{C:\textbackslash{}Python\textbackslash{}Lib\textbackslash{}distutils\textbackslash{}distutils.cfg}. |
| Under Python 1.5.2, the default prefix was |
| \file{C:\textbackslash{}Program~Files\textbackslash{}Python}, and the |
| Distutils were not part of the standard library---so the system |
| configuration file would be |
| \file{C:\textbackslash{}Program~Files\textbackslash{}Python\textbackslash{}distutils\textbackslash{}distutils.cfg} |
| in a standard Python 1.5.2 installation under Windows. |
| \item[(5)] On Windows, if the \envvar{HOME} environment variable is not |
| defined, no personal configuration file will be found or used. (In |
| other words, the Distutils make no attempt to guess your home |
| directory on Windows.) |
| \item[(6)] (See also notes (1) and (4).) The default installation |
| prefix is just \file{Python:}, so under Python 1.6 and later this is |
| normally\file{Python:Lib:distutils:distutils.cfg}. |
| \end{description} |
| |
| |
| \subsection{Syntax of config files} |
| \label{config-syntax} |
| |
| The Distutils configuration files all have the same syntax. The config |
| files are grouped into sections. There is one section for each Distutils |
| command, plus a \code{global} section for global options that affect |
| every command. Each section consists of one option per line, specified |
| as \code{option=value}. |
| |
| For example, the following is a complete config file that just forces |
| all commands to run quietly by default: |
| |
| \begin{verbatim} |
| [global] |
| verbose=0 |
| \end{verbatim} |
| |
| If this is installed as the system config file, it will affect all |
| processing of any Python module distribution by any user on the current |
| system. If it is installed as your personal config file (on systems |
| that support them), it will affect only module distributions processed |
| by you. And if it is used as the \file{setup.cfg} for a particular |
| module distribution, it affects only that distribution. |
| |
| You could override the default ``build base'' directory and make the |
| \command{build*} commands always forcibly rebuild all files with the |
| following: |
| |
| \begin{verbatim} |
| [build] |
| build-base=blib |
| force=1 |
| \end{verbatim} |
| |
| which corresponds to the command-line arguments |
| |
| \begin{verbatim} |
| python setup.py build --build-base=blib --force |
| \end{verbatim} |
| |
| except that including the \command{build} command on the command-line |
| means that command will be run. Including a particular command in |
| config files has no such implication; it only means that if the command |
| is run, the options in the config file will apply. (Or if other |
| commands that derive values from it are run, they will use the values in |
| the config file.) |
| |
| You can find out the complete list of options for any command using the |
| \longprogramopt{help} option, e.g.: |
| |
| \begin{verbatim} |
| python setup.py build --help |
| \end{verbatim} |
| |
| and you can find out the complete list of global options by using |
| \longprogramopt{help} without a command: |
| |
| \begin{verbatim} |
| python setup.py --help |
| \end{verbatim} |
| |
| See also the ``Reference'' section of the ``Distributing Python |
| Modules'' manual. |
| |
| \section{Building Extensions: Tips and Tricks} |
| \label{building-ext} |
| |
| Whenever possible, the Distutils try to use the configuration |
| information made available by the Python interpreter used to run the |
| \file{setup.py} script. For example, the same compiler and linker |
| flags used to compile Python will also be used for compiling |
| extensions. Usually this will work well, but in complicated |
| situations this might be inappropriate. This section discusses how to |
| override the usual Distutils behaviour. |
| |
| \subsection{Tweaking compiler/linker flags} |
| \label{tweak-flags} |
| |
| Compiling a Python extension written in C or \Cpp will sometimes |
| require specifying custom flags for the compiler and linker in order |
| to use a particular library or produce a special kind of object code. |
| This is especially true if the extension hasn't been tested on your |
| platform, or if you're trying to cross-compile Python. |
| |
| In the most general case, the extension author might have foreseen |
| that compiling the extensions would be complicated, and provided a |
| \file{Setup} file for you to edit. This will likely only be done if |
| the module distribution contains many separate extension modules, or |
| if they often require elaborate sets of compiler flags in order to work. |
| |
| A \file{Setup} file, if present, is parsed in order to get a list of |
| extensions to build. Each line in a \file{Setup} describes a single |
| module. Lines have the following structure: |
| |
| \begin{alltt} |
| \var{module} ... [\var{sourcefile} ...] [\var{cpparg} ...] [\var{library} ...] |
| \end{alltt} |
| |
| Let's examine each of the fields in turn. |
| |
| \begin{itemize} |
| |
| \item \var{module} is the name of the extension module to be built, |
| and should be a valid Python identifier. You can't just change |
| this in order to rename a module (edits to the source code would |
| also be needed), so this should be left alone. |
| |
| \item \var{sourcefile} is anything that's likely to be a source code |
| file, at least judging by the filename. Filenames ending in |
| \file{.c} are assumed to be written in C, filenames ending in |
| \file{.C}, \file{.cc}, and \file{.c++} are assumed to be |
| \Cpp, and filenames ending in \file{.m} or \file{.mm} are |
| assumed to be in Objective C. |
| |
| \item \var{cpparg} is an argument for the C preprocessor, |
| and is anything starting with \programopt{-I}, \programopt{-D}, |
| \programopt{-U} or \programopt{-C}. |
| |
| \item \var{library} is anything ending in \file{.a} or beginning with |
| \programopt{-l} or \programopt{-L}. |
| \end{itemize} |
| |
| If a particular platform requires a special library on your platform, |
| you can add it by editing the \file{Setup} file and running |
| \code{python setup.py build}. For example, if the module defined by the line |
| |
| \begin{verbatim} |
| foo foomodule.c |
| \end{verbatim} |
| |
| must be linked with the math library \file{libm.a} on your platform, |
| simply add \programopt{-lm} to the line: |
| |
| \begin{verbatim} |
| foo foomodule.c -lm |
| \end{verbatim} |
| |
| Arbitrary switches intended for the compiler or the linker can be |
| supplied with the \programopt{-Xcompiler} \var{arg} and |
| \programopt{-Xlinker} \var{arg} options: |
| |
| \begin{verbatim} |
| foo foomodule.c -Xcompiler -o32 -Xlinker -shared -lm |
| \end{verbatim} |
| |
| The next option after \programopt{-Xcompiler} and |
| \programopt{-Xlinker} will be appended to the proper command line, so |
| in the above example the compiler will be passed the \programopt{-o32} |
| option, and the linker will be passed \programopt{-shared}. If a |
| compiler option requires an argument, you'll have to supply multiple |
| \programopt{-Xcompiler} options; for example, to pass \code{-x c++} the |
| \file{Setup} file would have to contain |
| \code{-Xcompiler -x -Xcompiler c++}. |
| |
| Compiler flags can also be supplied through setting the |
| \envvar{CFLAGS} environment variable. If set, the contents of |
| \envvar{CFLAGS} will be added to the compiler flags specified in the |
| \file{Setup} file. |
| |
| |
| \subsection{Using non-Microsoft compilers on Windows \label{non-ms-compilers}} |
| \sectionauthor{Rene Liebscher}{R.Liebscher@gmx.de} |
| |
| \subsubsection{Borland \Cpp} |
| |
| This subsection describes the necessary steps to use Distutils with the |
| Borland \Cpp{} compiler version 5.5. |
| %Should we mention that users have to create cfg-files for the compiler? |
| %see also http://community.borland.com/article/0,1410,21205,00.html |
| |
| First you have to know that Borland's object file format (OMF) is |
| different from the format used by the Python version you can download |
| from the Python or ActiveState Web site. (Python is built with |
| Microsoft Visual \Cpp, which uses COFF as the object file format.) |
| For this reason you have to convert Python's library |
| \file{python20.lib} into the Borland format. You can do this as |
| follows: |
| |
| \begin{verbatim} |
| coff2omf python20.lib python20_bcpp.lib |
| \end{verbatim} |
| |
| The \file{coff2omf} program comes with the Borland compiler. The file |
| \file{python20.lib} is in the \file{Libs} directory of your Python |
| installation. If your extension uses other libraries (zlib,...) you |
| have to convert them too. |
| |
| The converted files have to reside in the same directories as the |
| normal libraries. |
| |
| How does Distutils manage to use these libraries with their changed |
| names? If the extension needs a library (eg. \file{foo}) Distutils |
| checks first if it finds a library with suffix \file{_bcpp} |
| (eg. \file{foo_bcpp.lib}) and then uses this library. In the case it |
| doesn't find such a special library it uses the default name |
| (\file{foo.lib}.)\footnote{This also means you could replace all |
| existing COFF-libraries with OMF-libraries of the same name.} |
| |
| To let Distutils compile your extension with Borland \Cpp{} you now have |
| to type: |
| |
| \begin{verbatim} |
| python setup.py build --compiler=bcpp |
| \end{verbatim} |
| |
| If you want to use the Borland \Cpp{} compiler as the default, you |
| could specify this in your personal or system-wide configuration file |
| for Distutils (see section~\ref{config-files}.) |
| |
| \begin{seealso} |
| \seetitle[http://www.borland.com/bcppbuilder/freecompiler/] |
| {\Cpp{}Builder Compiler} |
| {Information about the free \Cpp{} compiler from Borland, |
| including links to the download pages.} |
| |
| \seetitle[http://www.cyberus.ca/\~{}g_will/pyExtenDL.shtml] |
| {Creating Python Extensions Using Borland's Free Compiler} |
| {Document describing how to use Borland's free command-line \Cpp |
| compiler to build Python.} |
| \end{seealso} |
| |
| |
| \subsubsection{GNU C / Cygwin / MinGW} |
| |
| This section describes the necessary steps to use Distutils with the |
| GNU C/\Cpp{} compilers in their Cygwin and MinGW |
| distributions.\footnote{Check |
| \url{http://sources.redhat.com/cygwin/} and |
| \url{http://www.mingw.org/} for more information} |
| For a Python interpreter that was built with Cygwin, everything should |
| work without any of these following steps. |
| |
| These compilers require some special libraries. |
| This task is more complex than for Borland's \Cpp, because there is no |
| program to convert the library. |
| % I don't understand what the next line means. --amk |
| % (inclusive the references on data structures.) |
| |
| First you have to create a list of symbols which the Python DLL exports. |
| (You can find a good program for this task at |
| \url{http://starship.python.net/crew/kernr/mingw32/Notes.html}, see at |
| PExports 0.42h there.) |
| |
| \begin{verbatim} |
| pexports python20.dll >python20.def |
| \end{verbatim} |
| |
| Then you can create from these information an import library for gcc. |
| |
| \begin{verbatim} |
| dlltool --dllname python20.dll --def python20.def --output-lib libpython20.a |
| \end{verbatim} |
| |
| The resulting library has to be placed in the same directory as |
| \file{python20.lib}. (Should be the \file{libs} directory under your |
| Python installation directory.) |
| |
| If your extension uses other libraries (zlib,...) you might |
| have to convert them too. |
| The converted files have to reside in the same directories as the normal |
| libraries do. |
| |
| To let Distutils compile your extension with Cygwin you now have to type |
| |
| \begin{verbatim} |
| python setup.py build --compiler=cygwin |
| \end{verbatim} |
| |
| and for Cygwin in no-cygwin mode\footnote{Then you have no |
| \POSIX{} emulation available, but you also don't need |
| \file{cygwin1.dll}.} or for MinGW type: |
| |
| \begin{verbatim} |
| python setup.py build --compiler=mingw32 |
| \end{verbatim} |
| |
| If you want to use any of these options/compilers as default, you should |
| consider to write it in your personal or system-wide configuration file |
| for Distutils (see section~\ref{config-files}.) |
| |
| \begin{seealso} |
| \seetitle[http://www.zope.org/Members/als/tips/win32_mingw_modules] |
| {Building Python modules on MS Windows platform with MinGW} |
| {Information about building the required libraries for the MinGW |
| environment.} |
| |
| \seeurl{http://pyopengl.sourceforge.net/ftp/win32-stuff/} |
| {Converted import libraries in Cygwin/MinGW and Borland format, |
| and a script to create the registry entries needed for Distutils |
| to locate the built Python.} |
| \end{seealso} |
| |
| |
| |
| \end{document} |