Doc/inst/inst.tex

\documentclass{howto}
\usepackage{ltxmarkup}
\usepackage{times}

\title{Installing Python Modules}

% The audience for this document includes people who don't know anything 
% about Python and aren't about to learn the language just in order to
% install and maintain it for their users, i.e. system administrators.
% Thus, I have to be sure to explain the basics at some point:
% sys.path and PYTHONPATH at least.  Should probably give pointers to
% other docs on "import site", PYTHONSTARTUP, PYTHONHOME, etc.
% 
% Also, I need to take into account that most modules out there don't
% (yet) use Distutils: briefly explain the old Makefile.pre.in
% convention (maybe move material from the E&E manual to here?), and
% explain where to copy .py and .so files manually if the distribution
% doesn't provide a mechanism for doing so.
%
% Finally, it might be useful to include all the material from my "Care
% and Feeding of a Python Installation" talk in here somewhere.  Yow!

% Hey wow, Guido didn't write this one either!
\author{Greg Ward}
\authoraddress{E-mail: \email{gward@python.net}}

% Should these be added to the standard Python doc tools?  (They'll be
% needed for my "Distributing Python Modules" guide, too.)
\newcommand{\command}[1]{\code{#1}}
\newcommand{\option}[1]{\textsf{\small{#1}}}
\newcommand{\filevar}[1]{{\textsl{\filenq{#1}}}}
\newcommand{\homefile}[1]{\file{\tilde/#1}}
\newcommand{\comingsoon}{\emph{Coming soon$\ \ldots$}}

% And how about these?  Very handy for writing pathnames (tilde for
% Unix, backslash for DOS/Windows).
\renewcommand{\tilde}{\raisebox{-0.5ex}{\symbol{126}}}
\newcommand{\bslash}{\symbol{92}}


\begin{document}

\maketitle

%\begin{abstract}
%\noindent
%Abstract this!
%\end{abstract}

\tableofcontents

\section{Introduction}
\label{sec:intro}

\comingsoon

\subsection{The new way: Distutils}
\label{sec:new-way}


\subsection{The old way (pure Python): whatever you feel like}
\label{sec:old-way-pure}


\subsection{The old way (extensions, \UNIX{} only): Makefile.pre.in}
\label{sec:old-way-ext}





\section{Normal Build and Install}
\label{sec:normal-install}

% This will cover:
%   * setup.py install        (the usual thing)
%   * setup.py build          (if you like doing things one-at-a-time)
%   * setup.py build install  (not necessary unless you need to supply
%                              build options--ref. next section)
%   * where things are installed, on Unix and Windows (Mac...?)
%   * simple custom install: "install --prefix=$HOME"
\comingsoon


\section{Custom Extension Building}
\label{sec:custom-ext}

% This will cover:
%   * normal extension build -- stress that it doesn't matter, you
%     do the same thing whether there are extensions or not
%   * what you might want to customize: compiler and compiler
%     flags (warn of the dangers); per-file compiler flags
%     (not handled yet!)
%   * when things go wrong: I don't know! (and I don't know what
%     to do, either!)
\comingsoon


\section{Custom Installation (\UNIX)}
\label{sec:custom-install-unix}

% XXX probably should banish mentions of Windows here to the
% separate "Non-standard installation (Windows)" section.

A \dfn{custom installation} is where you install modules to a location
that's not in Python's default module search path.  There are a couple
of reasons you might want to do this; the most typical is simply that
you don't have permission to write to the standard Python library
directory.  Or, even if you do have write permission to the standard
library, you might wish to install a module distribution into a
non-standard place for testing or experimentation.  (This is especially
useful when upgrading an existing module distribution: you might want to
make sure that your existing scripts continue to work as before, and
only then install the upgrade ``for real.'')

(XXX terminology: I keep saying ``standard Python library directory''
when I really mean ``the site-packages directory under the standard
Python library directory''.  Is there a better way?)

In any event, you can easily install to non-standard locations with a
couple of options to the \command{install} command:

\begin{tableii}{ll}{option}{Option}{Description}
  \lineii {install-lib}
          {install directory for modules from pure Python distributions}
  \lineii {install-platlib}
          {install directory for modules from distributions with extensions}
  \lineii {prefix}
          {override \code{sys.prefix}:
           point to an alternate Python installation}
  \lineii {exec-prefix}
          {override \code{sys.exec_prefix}:
           point to an alternate Python installation}
  \lineii {install-path}
          {extra sub-path to append to \option{install-lib} (for
           non-package-ized distributions)}
\end{tableii}

Of these, the most commonly used will probably be \option{install-lib}
and \option{install-platlib}: you use them to point module installation
right at a particular directory.  (You'll only need
\option{install-platlib} if you maintain a multi-platform installation,
which is often done on \UNIX{} networks with different architectures and
operating systems.)  The two prefix options are intended for the
somewhat arcane purpose of installing modules into a different Python
installation than the Python binary used to perform the installation.
The last, \option{install-path}, is mainly used for module developers to
ensure that their module will go into a directory of their own, but it
may occasionally be useful to you as a module installer.


\subsection{Directly specifying installation directories}
\label{sec:install-dirs}

The most common type of custom module installation is where you maintain
a personal stash of Python modules under your home directory, say in
\homefile{lib/python}.  If you only care about a single platform
there, then you only need to specify the \option{install-lib} option and 
can forget about \option{install-platlib}:
\begin{verbatim}
python setup.py install --install-lib=~/lib/python
\end{verbatim}
You can, of course, supply whatever directory you like in place of
\homefile{lib/python}.  More importantly, you can specify this
directory permanently in your personal configuration file (XXX
filename?):
\begin{verbatim}
[install]
install-lib=~/lib/python
\end{verbatim}
Note that use of shell-style tilde and environment variable expansion is
supported both on the command line and in configuration files.  (See
section~\ref{sec:config-files} for more information on configuration
files.)

Of course, in order for this personal Python library scheme to work, you
have to ensure that \homefile{lib/python} is in \code{sys.path} when you
run Python.  The easiest way to do this under \UNIX{} is to add it to
your \code{PYTHONPATH} environment variable when you login.  For
example, if you use a Bourne shell derivative such as bash, zsh, or ksh,
add the following to your \homefile{.profile} (or \homefile{.bashrc}, or
\homefile{.zshenv}, depending on your shell and personal preferences):
\begin{verbatim}
export PYTHONPATH=$HOME/lib/python
\end{verbatim}
If you use a csh-derivative such as tcsh, add the following to your
\homefile{.cshrc}:
\begin{verbatim}
setenv PYTHONPATH $HOME/lib/python
\end{verbatim}

If you use multiple platforms (architectures and/or operating systems)
from the same home directory, then you probably want to maintain a
multi-platform personal Python library.  One possible scheme is to put
platform-neutral (pure Python) distributions in \homefile{lib/python}
and platform-specific distributions (any that containe extension
modules) in \homefile{lib/python.\filevar{plat}}:
\begin{verbatim}
python setup.py install --install-lib=~/lib/python \
                        --install-lib-plat=~/lib/python.plat \
\end{verbatim}
On the command line, of course, you can just type in the current
platform in place of \filevar{plat}: \file{linux-x86},
\file{solaris-sparc}, \file{linux-alpha}, whatever.  That's not an
option in a configuration file, though---the same file has to cover all
platforms for which you maintain a personal Python library.  So the
Distutils provide a \code{PLAT} environment variable which will expand
to the current platform name:
\begin{verbatim}
[install]
install-lib=~/lib/python
install-platlib=~/lib/python.$PLAT
\end{verbatim}
(If \code{PLAT} is already defined in your environment, the Distutils
won't override it: that way you can maintain consistency with other
applications that look for a \code{PLAT} variable; this is especially
useful when you refer to \code{PLAT} in your login scripts, as explained
below.)

(XXX danger danger! this environment-variable-in-config-file thing is
frighteningly make-like: is there any way to avoid it?)

Again, you have to make sure that your personal Python library appears
in \code{sys.path}, and again the easiest way to do this is to set
\code{PYTHONPATH} in your login scripts.  This time, though, you have to 
be sure to set \emph{both} directories (platform-neutral and the current 
platform-specific directory).  For Bourne-shell derivatives:
\begin{verbatim}
export PYTHONPATH=$HOME/lib/python:$HOME/lib/python.$PLAT
\end{verbatim}
and for csh-derivatives:
\begin{verbatim}
setenv PYTHONPATH $HOME/lib/python:$HOME/lib/python.$PLAT
\end{verbatim}
Note that it is your responsibility to set the \code{PATH} environment
variable (unless your system administrator has kindly taken care of it
in the system-wide login scripts, which is a wise thing to do on
multi-platform networks).  One way to do this is with the \code{uname}
command:
\begin{verbatim}
export PLAT=`uname -sm | tr 'A-Z ' 'a-z-'`
\end{verbatim}
(XXX check that this works well on other Unices: on Linux, \code{-m}
becomes eg. \code{i586}, which is not the \emph{machine} but the
\emph{processor}.  Arggh!)

Of course, there are more reasons to do custom installation than
maintaining a personal Python library.  Even if you have write access to
the system-wide directories for third-party modules
(\file{\filevar{prefix}/lib/python1.\filevar{x}/site-packages} and
\file{\filevar{exec-prefix}/lib/python1.\filevar{x}/site-packages}), you
might want to install new module distributions---especially upgrades of
modules that are crucial to your local infrastructure---to a temporary
location, in order to test them before installing them ``for real.''
This is fundamentally no different from installing to your home
directory, except that you probably won't bother to set
\code{PYTHONPATH} permanently.  For example, to install a module
distribution to \file{/tmp/pylib}:
\begin{verbatim}
python setup.py install --install-lib=/tmp/pylib
\end{verbatim}
Then, of course, you'll want to run some script that depends on these
modules to make sure that they still work with your installed base of
code:
\begin{verbatim}
env PYTHONPATH=/tmp/pylib python /usr/local/bin/crucial_script ...
\end{verbatim}

Of course, you can do this temporary installation with separate
\option{install-lib} and \option{install-platlib} options.  If you're
doing this to a network-wide directory, not \file{/tmp}, this might be
essential.  As you might have guessed, it's not too hard:
\begin{verbatim}
python setup.py install --install-lib=/scratch/pylib \
                        --install-platlib=/scratch/pylib.plat
\end{verbatim}
and then, testing your crucial scripts on multiple platforms:
\begin{verbatim}
env PYTHONPATH=/scratch/pylib:/scratch/pylib.plat \
    python /usr/local/bin/crucial_script ...
\end{verbatim}

However you do the testing, once you're satisfied that the new version
doesn't break anything, you can install it to the system-wide
third-party module directory as usual:
\begin{verbatim}
python setup.py install
\end{verbatim}


\subsection{Indirect specification: prefix directories}
\label{sec:prefix-dirs}

Occasionally, you may want to install a module distribution


\section{Custom Installation (Windows)}
\label{sec:custom-install-windows}

\comingsoon


\section{Configuration Files}
\label{sec:config-files}

\comingsoon



\end{document}