blob: 60a42d18b5e8ce6af0136bdbe88ef8d2e1188980 [file] [log] [blame]
Greg Wardabc52162000-02-26 00:52:48 +00001\documentclass{howto}
2\usepackage{ltxmarkup}
Greg Wardfacb8db2000-04-09 04:32:40 +00003\usepackage{times}
Greg Ward16aafcd2000-04-09 04:06:44 +00004\usepackage{distutils}
Greg Wardabc52162000-02-26 00:52:48 +00005
Greg Ward16aafcd2000-04-09 04:06:44 +00006\title{Distributing Python Modules}
Greg Wardabc52162000-02-26 00:52:48 +00007
Greg Wardabc52162000-02-26 00:52:48 +00008\author{Greg Ward}
9\authoraddress{E-mail: \email{gward@python.net}}
10
Greg Ward16aafcd2000-04-09 04:06:44 +000011
Greg Wardabc52162000-02-26 00:52:48 +000012\begin{document}
13
Greg Wardfacb8db2000-04-09 04:32:40 +000014\maketitle
15\tableofcontents
Greg Ward16aafcd2000-04-09 04:06:44 +000016
17\section{Introduction}
Greg Warde78298a2000-04-28 17:12:24 +000018\label{intro}
Greg Ward16aafcd2000-04-09 04:06:44 +000019
20In the past, Python module developers have not had much infrastructure
21support for distributing modules, nor have Python users had much support
22for installing and maintaining third-party modules. With the
23introduction of the Python Distribution Utilities (Distutils for short)
24in Python 1.6, this situation should start to improve.
25
26This document only covers using the Distutils to distribute your Python
27modules. Using the Distutils does not tie you to Python 1.6, though:
28the Distutils work just fine with Python 1.5, and it is reasonable (and
29expected to become commonplace) to expect users of Python 1.5 to
30download and install the Distutils separately before they can install
31your modules. Python 1.6 users, of course, won't have to add anything
32to their Python installation in order to use the Distutils to install
33third-party modules.
34
35This document concentrates on the role of developer/distributor: if
36you're looking for information on installing Python modules, you should
37refer to the ``Installing Python Modules'' manual.
38
39
Greg Wardfacb8db2000-04-09 04:32:40 +000040\section{Concepts \& Terminology}
Greg Warde78298a2000-04-28 17:12:24 +000041\label{concepts}
Greg Ward16aafcd2000-04-09 04:06:44 +000042
43Using the Distutils is quite simple, both for module developers and for
44users/administrators installing third-party modules. As a developer,
45your responsibilites (apart from writing solid, well-documented and
46well-tested code, of course!) are:
47\begin{itemize}
48\item write a setup script (\file{setup.py} by convention)
49\item (optional) write a setup configuration file
50\item create a source distribution
51\item (optional) create one or more built (binary) distributions
52\end{itemize}
53Each of these tasks is covered in this document.
54
55Not all module developers have access to a multitude of platforms, so
56it's not always feasible to expect them to create a multitude of built
57distributions. It is hoped that a class of intermediaries, called
58\emph{packagers}, will arise to take address this need. Packagers will
59take source distributions released by module developers, build them on
60one or more platforms, and release the resulting built distributions.
61Thus, users on the most popular platforms will be able to install most
62popular Python module distributions in the most natural way for their
63platform, without having to run a single setup script or compile a line
64of code.
65
66
67\subsection{A simple example}
Greg Warde78298a2000-04-28 17:12:24 +000068\label{simple-example}
Greg Ward16aafcd2000-04-09 04:06:44 +000069
70The setup script is usually quite simple, although since it's written in
71Python, there are no arbitrary limits to what you can do. If all you
72want to do is distribute a module called \module{foo}, contained in a
73file \file{foo.py}, then you can get away with as little as this:
74\begin{verbatim}
75from distutils.core import setup
76setup (name = "foo",
77 version = "1.0",
78 py_modules = ["foo"])
79\end{verbatim}
80Some observations:
81\begin{itemize}
82\item all information that you supply to the Distutils is supplied as
Greg Wardfacb8db2000-04-09 04:32:40 +000083 keyword arguments to the \function{setup()} function
Greg Ward16aafcd2000-04-09 04:06:44 +000084\item those keyword arguments fall into two categories: package
85 meta-data (name, version number) and information about what's in the
86 package (list of pure modules, in this case)
87\item modules are specified by module name, not filename (the same will
88 hold true for packages and extensions)
89\item it's recommended that you supply a little more meta-data, in
90 particular your name, email address and a URL for the project
91\end{itemize}
92
93To create a source distribution for this module, you would run
94\begin{verbatim}
95python setup.py sdist
96\end{verbatim}
97which will create an archive file (e.g., tarball on Unix, zip file on
98Windows) containing your setup script, \file{setup.py}, and your module,
99\file{foo.py}. The archive file will be named \file{Foo-1.0.tar.gz} (or
100\file{.zip}), and will unpack into a directory \file{Foo-1.0}.
101
102If an end-user wishes to install your \module{foo} module, all she has
Greg Ward59d382e2000-05-26 01:04:47 +0000103to do is download \file{Foo-1.0.tar.gz} (or \file{.zip}), unpack it,
Greg Ward16aafcd2000-04-09 04:06:44 +0000104and---from the \file{Foo-1.0} directory---run
105\begin{verbatim}
106python setup.py install
107\end{verbatim}
108which will ultimately copy \file{foo.py} to the appropriate directory
109for third-party modules in their Python installation.
110
111This simple example demonstrates some fundamental concepts of the
112Distutils: first, both developers and installers have the same basic
113user interface, i.e. the setup script. The difference is which
114Distutils \emph{commands} they use: the \command{sdist} command is
115almost exclusively for module developers, while \command{install} is
116more often for installers (although most developers will want to install
117their own code occasionally).
118
119\XXX{only partially implemented}%
120If you want to make things really easy for your users, you can create
121one or more built distributions for them. For instance, if you are
122running on a Windows machine, and want to make things easy for other
123Windows users, you can create an executable installer (the most
124appropriate type of built distribution for this platform) with the
Greg Ward59d382e2000-05-26 01:04:47 +0000125\command{bdist\_wininst} command. For example:
Greg Ward16aafcd2000-04-09 04:06:44 +0000126\begin{verbatim}
Greg Ward59d382e2000-05-26 01:04:47 +0000127python setup.py bdist_wininst
Greg Ward16aafcd2000-04-09 04:06:44 +0000128\end{verbatim}
Greg Wardfacb8db2000-04-09 04:32:40 +0000129will create an executable installer, \file{Foo-1\_0.exe}, in the current
Greg Ward16aafcd2000-04-09 04:06:44 +0000130directory.
131
Greg Ward59d382e2000-05-26 01:04:47 +0000132(Another way to create executable installers for Windows is with the
133\command{bdist\_wise} command, which uses Wise---the commercial
134installer-generator used to create Python's own installer---to create
135the installer. Wise-based installers are more appropriate for large,
136industrial-strength applications that need the full capabilities of a
137``real'' installer. \command{bdist\_wininst} creates a self-extracting
138zip file with a minimal user interface, which is enough for small- to
139medium-sized module collections. You'll need to have version XXX of
140Wise installed on your system for the \command{bdist\_wise} to work;
141it's available from \url{http://foo/bar/baz}.)
142
143Other \command{bdist} commands exist for other platforms: for example,
144\command{bdist\_rpm} for RPM-based Linux systems, (\command{bdist\_deb})
145for Debian-based Linux systems, and so forth. See
146section~\ref{bdist-cmds} for details on all the \command{bdist}
147commands.
Greg Ward16aafcd2000-04-09 04:06:44 +0000148
149
150\subsection{General Python terminology}
Greg Warde78298a2000-04-28 17:12:24 +0000151\label{python-terms}
Greg Ward16aafcd2000-04-09 04:06:44 +0000152
153If you're reading this document, you probably have a good idea of what
154modules, extensions, and so forth are. Nevertheless, just to be sure
155that everyone is operating from a common starting point, we offer the
156following glossary of common Python terms:
157\begin{description}
158\item[module] the basic unit of code reusability in Python: a block of
159 code imported by some other code. There are three types of modules
160 that concern us here: pure Python modules, extension modules, and
161 packages.
162\item[pure Python module] a module written in Python and contained in a
163 single \file{.py} file (and possibly associated \file{.pyc} and/or
164 \file{.pyo} files). Sometimes referred to as a ``pure module.''
165\item[extension module] a module written in the low-level language of
166 the Python implemention: C/C++ for CPython, Java for JPython.
167 Typically contained in a single dynamically loadable pre-compiled
168 file, e.g. a shared object (\file{.so}) file for CPython extensions on
169 Unix, a DLL (given the \file{.pyd} extension) for CPython extensions
170 on Windows, or a Java class file for JPython extensions.
171\item[package] a module that contains other modules; typically contained
172 in a directory in the filesystem and distinguished from other
173 directories by the presence of a file \file{\_\_init\_\_.py}.
Greg Ward6153fa12000-05-26 02:24:28 +0000174\item[root package] the root of the hierarchy of packages. (This isn't
175 really a package, since it doesn't have an \file{\_\_init\_\_.py}
176 file. But we have to call it something.) The vast majority of the
177 standard library is in the root package, as are many small, standalone
178 third-party modules that don't belong to a larger module collection.
179 Unlike regular packages, modules in the root package can be found in
180 many directories: in fact, every directory listed in \code{sys.path}
181 can contribute modules to the root package.
Greg Ward16aafcd2000-04-09 04:06:44 +0000182\end{description}
183
184
185\subsection{Distutils-specific terminology}
Greg Warde78298a2000-04-28 17:12:24 +0000186\label{distutils-term}
Greg Ward16aafcd2000-04-09 04:06:44 +0000187
188The following terms apply more specifically to the domain of
189distributing Python modules using the Distutils:
190\begin{description}
191\item[module distribution] a collection of Python modules distributed
192 together as a single downloadable resource and meant to be installed
193 \emph{en masse}. Examples of some well-known module distributions are
194 Numeric Python, PyXML, PIL (the Python Imaging Library), or
195 mxDateTime. (This would be called a \emph{package}, except that term
Greg Ward59d382e2000-05-26 01:04:47 +0000196 is already taken in the Python context: a single module distribution
197 may contain zero, one, or many Python packages.)
Greg Ward16aafcd2000-04-09 04:06:44 +0000198\item[pure module distribution] a module distribution that contains only
199 pure Python modules and packages. Sometimes referred to as a ``pure
200 distribution.''
201\item[non-pure module distribution] a module distribution that contains
202 at least one extension module. Sometimes referred to as a ``non-pure
203 distribution.''
204\item[distribution root] the top-level directory of your source tree (or
205 source distribution); the directory where \file{setup.py} exists and
206 is run from
207\end{description}
208
209
210\section{Writing the Setup Script}
Greg Warde78298a2000-04-28 17:12:24 +0000211\label{setup-script}
Greg Ward16aafcd2000-04-09 04:06:44 +0000212
213The setup script is the centre of all activity in building,
214distributing, and installing modules using the Distutils. The main
215purpose of the setup script is to describe your module distribution to
Greg Wardd5767a52000-04-19 22:48:09 +0000216the Distutils, so that the various commands that operate on your modules
Greg Ward59d382e2000-05-26 01:04:47 +0000217do the right thing. As we saw in section~\ref{simple-example} above,
218the setup script consists mainly of a call to \function{setup()}, and
219all information supplied to the Distutils is supplied as keyword
Greg Wardfacb8db2000-04-09 04:32:40 +0000220arguments to \function{setup()}.
Greg Ward16aafcd2000-04-09 04:06:44 +0000221
222Here's a slightly more involved example, which we'll follow for the next
223couple of sections: the Distutils' own setup script. (Keep in mind that
224although the Distutils are included with Python 1.6, they also have an
225independent existence so that Python 1.5 users can use them to install
Greg Ward59d382e2000-05-26 01:04:47 +0000226other module distributions. The Distutils' own setup script is used to
227install the package into Python 1.5.)
Greg Ward16aafcd2000-04-09 04:06:44 +0000228
229\begin{verbatim}
230#!/usr/bin/env python
231
232from distutils.core import setup
233
234setup (name = "Distutils",
235 version = "1.0",
236 description = "Python Module Distribution Utilities",
237 author = "Greg Ward",
238 author_email = "gward@python.net",
239 url = "http://www.python.org/sigs/distutils-sig/",
240
241 packages = ['distutils', 'distutils.command'],
242 )
243\end{verbatim}
244There are only two differences between this and the trivial one-file
Greg Warde78298a2000-04-28 17:12:24 +0000245distribution presented in section~\ref{simple-example}: more
Greg Ward16aafcd2000-04-09 04:06:44 +0000246meta-data, and the specification of pure Python modules by package,
247rather than by module. This is important since the Distutils consist of
248a couple of dozen modules split into (so far) two packages; an explicit
249list of every module would be tedious to generate and difficult to
250maintain.
251
Greg Ward46b98e32000-04-14 01:53:36 +0000252Note that any pathnames (files or directories) supplied in the setup
253script should be written using the Unix convention, i.e.
254slash-separated. The Distutils will take care of converting this
Greg Ward59d382e2000-05-26 01:04:47 +0000255platform-neutral representation into whatever is appropriate on your
Greg Ward46b98e32000-04-14 01:53:36 +0000256current platform before actually using the pathname. This makes your
257setup script portable across operating systems, which of course is one
258of the major goals of the Distutils. In this spirit, all pathnames in
Greg Ward59d382e2000-05-26 01:04:47 +0000259this document are slash-separated (Mac OS programmers should keep in
260mind that the \emph{absence} of a leading slash indicates a relative
261path, the opposite of the Mac OS convention with colons).
Greg Ward46b98e32000-04-14 01:53:36 +0000262
Greg Ward16aafcd2000-04-09 04:06:44 +0000263
264\subsection{Package directories}
Greg Warde78298a2000-04-28 17:12:24 +0000265\label{package-dirs}
Greg Ward16aafcd2000-04-09 04:06:44 +0000266
267The \option{packages} option tells the Distutils to process (build,
268distribute, install, etc.) all pure Python modules found in each package
269mentioned in the \option{packages} list. In order to do this, of
270course, there has to be a correspondence between package names and
271directories in the filesystem. The default correspondence is the most
Greg Ward1ecc2512000-04-19 22:36:24 +0000272obvious one, i.e. package \module{distutils} is found in the directory
Greg Ward16aafcd2000-04-09 04:06:44 +0000273\file{distutils} relative to the distribution root. Thus, when you say
274\code{packages = ['foo']} in your setup script, you are promising that
275the Distutils will find a file \file{foo/\_\_init\_\_.py} (which might
276be spelled differently on your system, but you get the idea) relative to
277the directory where your setup script lives. (If you break this
278promise, the Distutils will issue a warning but process the broken
279package anyways.)
280
281If you use a different convention to lay out your source directory,
282that's no problem: you just have to supply the \option{package\_dir}
283option to tell the Distutils about your convention. For example, say
284you keep all Python source under \file{lib}, so that modules not in any
Greg Ward1ecc2512000-04-19 22:36:24 +0000285package are right in \file{lib}, modules in the \module{foo} package
Greg Ward16aafcd2000-04-09 04:06:44 +0000286are in \file{lib/foo}, and so forth. Then you would put
287\begin{verbatim}
288package_dir = {'': 'lib'}
289\end{verbatim}
290in your setup script. (The keys to this dictionary are package names,
291and an empty package name stands for the ``root package,'' i.e. no
292package at all. The values are directory names relative to your
293distribution root.) In this case, when you say
294\code{packages = ['foo']}, you are promising that the file
295\file{lib/foo/\_\_init\_\_.py} exists.
296
Greg Ward1ecc2512000-04-19 22:36:24 +0000297Another possible convention is to put the \module{foo} package right in
298\file{lib}, the \module{foo.bar} package in \file{lib/bar}, etc. This
Greg Ward16aafcd2000-04-09 04:06:44 +0000299would be written in the setup script as
300\begin{verbatim}
301package_dir = {'foo': 'lib'}
302\end{verbatim}
Greg Ward59d382e2000-05-26 01:04:47 +0000303A \code{\var{package}: \var{dir}} entry in the \option{package\_dir}
304dictionary implicitly applies to all packages below \var{package}, so
305the \module{foo.bar} case is automatically handled here. In this
306example, having \code{packages = ['foo', 'foo.bar']} tells the Distutils
307to look for \file{lib/\_\_init\_\_.py} and
308\file{lib/bar/\_\_init\_\_.py}. (Keep in mind that although
309\option{package\_dir} applies recursively, you must explicitly list all
310packages in \option{packages}: the Distutils will \emph{not} recursively
311scan your source tree looking for any directory with an
312\file{\_\_init\_\_.py} file.)
Greg Ward16aafcd2000-04-09 04:06:44 +0000313
314
315\subsection{Listing individual modules}
Greg Warde78298a2000-04-28 17:12:24 +0000316\label{listing-modules}
Greg Ward16aafcd2000-04-09 04:06:44 +0000317
318For a small module distribution, you might prefer to list all modules
319rather than listing packages---especially the case of a single module
320that goes in the ``root package'' (i.e., no package at all). This
Greg Warde78298a2000-04-28 17:12:24 +0000321simplest case was shown in section~\ref{simple-example}; here is a
Greg Ward16aafcd2000-04-09 04:06:44 +0000322slightly more involved example:
323\begin{verbatim}
324py_modules = ['mod1', 'pkg.mod2']
325\end{verbatim}
326This describes two modules, one of them in the ``root'' package, the
Greg Wardd5767a52000-04-19 22:48:09 +0000327other in the \module{pkg} package. Again, the default package/directory
328layout implies that these two modules can be found in \file{mod1.py} and
329\file{pkg/mod2.py}, and that \file{pkg/\_\_init\_\_.py} exists as well.
330And again, you can override the package/directory layout using the
Greg Ward59d382e2000-05-26 01:04:47 +0000331\option{package\_dir} option.
332
333
334\subsection{Describing extension modules}
335\label{sec:describing-extensions}
336
337\XXX{be sure to describe the whole \code{build\_info} dict, including
338 \code{extra\_compile\_args} and \code{extra\_link\_args}}
Greg Ward16aafcd2000-04-09 04:06:44 +0000339
340
341\section{Writing the Setup Configuration File}
Greg Warde78298a2000-04-28 17:12:24 +0000342\label{setup-config}
Greg Ward16aafcd2000-04-09 04:06:44 +0000343
344\XXX{not implemented yet!}
345
346Often, it's not possible to write down everything needed to build a
347distribution \emph{a priori}. You need to get some information from the
348user, or from the user's system, in order to proceed. For example, you
349might include an optional extension module that provides an interface to
350a particular C library. If that library is installed on the user's
351system, then you can build your optional extension---but you need to
352know where to find the header and library file. If it's not installed,
353you need to know this so you can omit your optional extension.
354
355The preferred way to do this, of course, would be for you to tell the
356Distutils which optional features (C libraries, system calls, external
357utilities, etc.) you're looking for, and it would inspect the user's
358system and try to find them. This functionality may appear in a future
359version of the Distutils, but it isn't there now. So, for the time
360being, we rely on the user building and installing your software to
361provide the necessary information. The vehicle for doing so is the
362setup configuration file, \file{setup.cfg}.
363
364\XXX{need more here!}
365
366
367\section{Creating a Source Distribution}
Greg Warde78298a2000-04-28 17:12:24 +0000368\label{source-dist}
Greg Ward16aafcd2000-04-09 04:06:44 +0000369
Greg Warde78298a2000-04-28 17:12:24 +0000370As shown in section~\ref{simple-example}, you use the
Greg Ward16aafcd2000-04-09 04:06:44 +0000371\command{sdist} command to create a source distribution. In the
372simplest case,
373\begin{verbatim}
374python setup.py sdist
375\end{verbatim}
376(assuming you haven't specified any \command{sdist} options in the setup
377script or config file), \command{sdist} creates the the archive of the
378default format for the current platform. The default formats are:
379\begin{tableii}{ll}{textrm}%
380 {Platform}{Default archive format for source distributions}
381 \lineii{Unix}{gzipped tar file (\file{.tar.gz})}
382 \lineii{Windows}{zip file}
383\end{tableii}
Greg Wardd5767a52000-04-19 22:48:09 +0000384You can specify as many formats as you like using the
385\longprogramopt{formats} option, for example:
Greg Ward16aafcd2000-04-09 04:06:44 +0000386\begin{verbatim}
387python setup.py sdist --formats=gztar,zip
388\end{verbatim}
389to create a gzipped tarball and a zip file. The available formats are:
Greg Ward46b98e32000-04-14 01:53:36 +0000390\begin{tableiii}{l|l|c}{code}%
391 {Format}{Description}{Notes}
392 \lineiii{zip}{zip file (\file{.zip})}{(1)}
393 \lineiii{gztar}{gzipped tar file (\file{.tar.gz})}{(2)}
394 \lineiii{ztar}{compressed tar file (\file{.tar.Z})}{}
395 \lineiii{tar}{tar file (\file{.tar})}{}
396\end{tableiii}
397
398\noindent Notes:
399\begin{description}
400\item[(1)] default on Windows
401\item[(2)] default on Unix
402\end{description}
Greg Ward16aafcd2000-04-09 04:06:44 +0000403
404
405\subsection{The manifest and manifest template}
Greg Warde78298a2000-04-28 17:12:24 +0000406\label{manifest}
Greg Ward16aafcd2000-04-09 04:06:44 +0000407
408Without any additional information, the \command{sdist} command puts a
409minimal set of files into the source distribution:
410\begin{itemize}
Greg Wardfacb8db2000-04-09 04:32:40 +0000411\item all Python source files implied by the \option{py\_modules} and
Greg Ward16aafcd2000-04-09 04:06:44 +0000412 \option{packages} options
Greg Wardfacb8db2000-04-09 04:32:40 +0000413\item all C source files mentioned in the \option{ext\_modules} or
Greg Ward16aafcd2000-04-09 04:06:44 +0000414 \option{libraries} options (\XXX{getting C library sources currently
Greg Wardfacb8db2000-04-09 04:32:40 +0000415 broken -- no get\_source\_files() method in build\_clib.py!})
Greg Ward16aafcd2000-04-09 04:06:44 +0000416\item anything that looks like a test script: \file{test/test*.py}
417 (currently, the Distutils don't do anything with test scripts except
418 include them in source distributions, but in the future there will be
419 a standard for testing Python module distributions)
420\item \file{README.txt} (or \file{README}) and \file{setup.py}
421\end{itemize}
422Sometimes this is enough, but usually you will want to specify
423additional files to distribute. The typical way to do this is to write
424a \emph{manifest template}, called \file{MANIFEST.in} by default. The
425\command{sdist} command processes this template and generates a manifest
426file, \file{MANIFEST}. (If you prefer, you can skip the manifest
427template and generate the manifest yourself: it just lists one file per
428line.)
429
430The manifest template has one command per line, where each command
431specifies a set of files to include or exclude from the source
432distribution. For an example, again we turn to the Distutils' own
433manifest template:
434\begin{verbatim}
435include *.txt
Greg Ward87da1ea2000-04-21 04:35:25 +0000436recursive-include examples *.txt *.py
Greg Ward16aafcd2000-04-09 04:06:44 +0000437prune examples/sample?/build
438\end{verbatim}
439The meanings should be fairly clear: include all files in the
440distribution root matching \code{*.txt}, all files anywhere under the
441\file{examples} directory matching \code{*.txt} or \code{*.py}, and
442exclude all directories matching \code{examples/sample?/build}. There
443are several other commands available in the manifest template
Greg Warde78298a2000-04-28 17:12:24 +0000444mini-language; see section~\ref{sdist-cmd}.
Greg Ward16aafcd2000-04-09 04:06:44 +0000445
446The order of commands in the manifest template very much matters:
447initially, we have the list of default files as described above, and
448each command in the template adds to or removes from that list of files.
449When we have fully processed the manifest template, we have our complete
450list of files. This list is written to the manifest for future
451reference, and then used to build the source distribution archive(s).
452
Greg Ward46b98e32000-04-14 01:53:36 +0000453Following the Distutils' own manifest template, let's trace how the
454\command{sdist} command will build the list of files to include in the
455Distutils source distribution:
Greg Ward16aafcd2000-04-09 04:06:44 +0000456\begin{enumerate}
457\item include all Python source files in the \file{distutils} and
458 \file{distutils/command} subdirectories (because packages
459 corresponding to those two directories were mentioned in the
460 \option{packages} option in the setup script)
461\item include \file{test/test*.py} (always included)
462\item include \file{README.txt} and \file{setup.py} (always included)
463\item include \file{*.txt} in the distribution root (this will find
464 \file{README.txt} a second time, but such redundancies are weeded out
465 later)
466\item in the sub-tree under \file{examples}, include anything matching
467 \file{*.txt}
468\item in the sub-tree under \file{examples}, include anything matching
469 \file{*.py}
470\item remove all files in the sub-trees starting at directories matching
471 \file{examples/sample?/build}---this may exclude files included by the
472 previous two steps, so it's important that the \code{prune} command in
473 the manifest template comes after the two \code{recursive-include}
474 commands
Greg Wardfacb8db2000-04-09 04:32:40 +0000475\end{enumerate}
Greg Ward16aafcd2000-04-09 04:06:44 +0000476
Greg Ward46b98e32000-04-14 01:53:36 +0000477Just like in the setup script, file and directory names in the manifest
478template should always be slash-separated; the Distutils will take care
479of converting them to the standard representation on your platform.
480That way, the manifest template is portable across operating systems.
481
Greg Ward16aafcd2000-04-09 04:06:44 +0000482
483\subsection{Manifest-related options}
Greg Warde78298a2000-04-28 17:12:24 +0000484\label{manifest-options}
Greg Ward16aafcd2000-04-09 04:06:44 +0000485
486The normal course of operations for the \command{sdist} command is as
487follows:
488\begin{itemize}
Greg Ward46b98e32000-04-14 01:53:36 +0000489\item if the manifest file, \file{MANIFEST} doesn't exist, read
490 \file{MANIFEST.in} and create the manifest
491\item if \file{MANIFEST.in} is more recent than \file{MANIFEST},
492 recreate \file{MANIFEST} by reading \file{MANIFEST.in}
Greg Ward16aafcd2000-04-09 04:06:44 +0000493\item use the list of files now in \file{MANIFEST} (either just
494 generated or read in) to create the source distribution archive(s)
495\end{itemize}
496There are a couple of options that modify this behaviour.
497
498First, you might want to force the manifest to be regenerated---for
499example, if you have added or removed files or directories that match an
500existing pattern in the manifest template, you should regenerate the
501manifest:
502\begin{verbatim}
503python setup.py sdist --force-manifest
504\end{verbatim}
505\XXX{this is stupid, but is there a better way to do it without
506 reprocessing MANIFEST.in every single bloody time?}
507
508Or, you might just want to (re)generate the manifest, but not create a
509source distribution:
510\begin{verbatim}
511python setup.py sdist --manifest-only
512\end{verbatim}
Greg Warda021aca2000-04-19 22:34:11 +0000513(\longprogramopt{manifest-only} implies \longprogramopt{force-manifest}.)
Greg Ward16aafcd2000-04-09 04:06:44 +0000514
515If you don't want to use the default file set, you can supply the
Greg Wardd5767a52000-04-19 22:48:09 +0000516\longprogramopt{no-defaults} option. If you use
517\longprogramopt{no-defaults} and don't supply a manifest template (or
518it's empty, or nothing matches the patterns in it), then your source
519distribution will be empty.
Greg Ward16aafcd2000-04-09 04:06:44 +0000520
521
522\section{Creating Built Distributions}
Greg Warde78298a2000-04-28 17:12:24 +0000523\label{built-dist}
Greg Ward16aafcd2000-04-09 04:06:44 +0000524
Greg Ward46b98e32000-04-14 01:53:36 +0000525A ``built distribution'' is what you're probably used to thinking of
526either as a ``binary package'' or an ``installer'' (depending on your
527background). It's not necessarily binary, though, because it might
528contain only Python source code and/or byte-code; and we don't call it a
529package, because that word is already spoken for in Python. (And
530``installer'' is a term specific to the Windows world. \XXX{do Mac
531 people use it?})
Greg Ward16aafcd2000-04-09 04:06:44 +0000532
Greg Ward46b98e32000-04-14 01:53:36 +0000533A built distribution is how you make life as easy as possible for
534installers of your module distribution: for users of RPM-based Linux
535systems, it's a binary RPM; for Windows users, it's an executable
536installer; for Debian-based Linux users, it's a Debian package; and so
537forth. Obviously, no one person will be able to create built
538distributions for every platform under the sun, so the Distutils is
539designed to enable module developers to concentrate on their
540specialty---writing code and creating source distributions---while an
541intermediary species of \emph{packager} springs up to turn source
542distributions into build distributions for as many platforms as there
543are packagers.
544
545Of course, the module developer could be his own packager; or the
546packager could be a volunteer ``out there'' somewhere who has access to
547a platform which the original developer does not; or it could be
548software periodically grabbing new source distributions and turning them
549into built distributions for as many platforms as the software has
550access to. Regardless of the nature of the beast, a packager uses the
551setup script and the \command{bdist} command family to generate built
552distributions.
553
554As a simple example, if I run the following command in the Distutils
555source tree:
556\begin{verbatim}
557python setup.py bdist
558\end{verbatim}
559then the Distutils builds my module distribution (the Distutils itself
560in this case), does a ``fake'' installation (also in the \file{build}
561directory), and creates the default type of built distribution for my
562platform. In Distutils 0.8, only two types of built distribution are
563supported: \code{gztar} (default on non-Linux Unix) and \code{zip}
564(default on Windows). Thus, the above command on a Unix system creates
565\file{Distutils-0.8.built-posix.tar.gz}; unpacking this tarball from
566Python's \filevar{prefix} directory installs the Distutils just as
567though you had downloaded the source distribution and run \code{python
568 setup.py install}. Obviously, for pure Python distributions, this
569isn't a huge win---but for non-pure distributions, which include
570extensions that would need to be compiled, it can mean the difference
571between someone being able to use your extensions or not.
572
573\XXX{filenames are inaccurate here!}
574
Greg Wardd5767a52000-04-19 22:48:09 +0000575The \command{bdist} command has a \longprogramopt{format} option,
576similar to the \command{sdist} command, that you can use to select which
577formats to generate: for example,
Greg Ward46b98e32000-04-14 01:53:36 +0000578\begin{verbatim}
579python setup.py bdist --format=zip
580\end{verbatim}
581would, when run on a Unix system, create
582\file{Distutils-0.8.built-posix.tar.gz}---again, this archive would be
583unpacked from Python's \filevar{prefix} directory to install the
584Distutils.
585
586The available formats for built distributions are:
587\begin{tableiii}{l|l|c}{code}%
588 {Format}{Description}{Notes}
589 \lineiii{zip}{zip file (\file{.zip})}{(1)}
590 \lineiii{gztar}{gzipped tar file (\file{.tar.gz})}{(2)}
591 \lineiii{ztar}{compressed tar file (\file{.tar.Z})}{}
592 \lineiii{tar}{tar file (\file{.tar})}{}
593 \lineiii{rpm}{RPM}{(3)}
594 \lineiii{srpm}{source RPM}{}
595 \lineiii{wise}{Wise installer for Windows}{}
596\end{tableiii}
597
598\noindent Notes:
599\begin{description}
600\item[(1)] default on Windows
601\item[(2)] default on Unix
602\item[(3)] not implemented yet; will be default on RPM-based Linux
603 systems
604\item[(5)] not implemented yet; will be default on Windows
605\end{description}
606
607You don't have to use the \command{bdist} command with the
Greg Wardd5767a52000-04-19 22:48:09 +0000608\longprogramopt{formats} option; you can also use the command that
609directly implements the format you're interested in. Many of these
Greg Ward46b98e32000-04-14 01:53:36 +0000610\command{bdist} ``sub-commands'' actually generate several similar
611formats; for instance, the \command{bdist\_dumb} command generates all
612the ``dumb'' archive formats (\code{tar}, \code{ztar}, \code{gztar}, and
613\code{zip}), and \command{bdist\_rpm} generates both binary and source
614RPMs. The \command{bdist} sub-commands, and the formats generated by
615each, are:
616\begin{tableii}{l|l}{command}%
617 {Command}{Formats}
618 \lineii{bdist\_dumb}{tar, ztar, gztar, zip}
619 \lineii{bdist\_rpm}{rpm, srpm}
620 \lineii{bdist\_wise}{wise}
621\end{tableii}
Greg Ward16aafcd2000-04-09 04:06:44 +0000622
623\section{Examples}
Greg Warde78298a2000-04-28 17:12:24 +0000624\label{examples}
Greg Ward16aafcd2000-04-09 04:06:44 +0000625
626
627\subsection{Pure Python distribution (by module)}
Greg Warde78298a2000-04-28 17:12:24 +0000628\label{pure-mod}
Greg Ward16aafcd2000-04-09 04:06:44 +0000629
630
631\subsection{Pure Python distribution (by package)}
Greg Warde78298a2000-04-28 17:12:24 +0000632\label{pure-pkg}
Greg Ward16aafcd2000-04-09 04:06:44 +0000633
634
635\subsection{Single extension module}
Greg Warde78298a2000-04-28 17:12:24 +0000636\label{single-ext}
Greg Ward16aafcd2000-04-09 04:06:44 +0000637
638
639\subsection{Multiple extension modules}
Greg Warde78298a2000-04-28 17:12:24 +0000640\label{multiple-ext}
Greg Ward16aafcd2000-04-09 04:06:44 +0000641
642
643\subsection{Putting it all together}
644
645
Greg Ward4a9e7222000-04-25 02:57:36 +0000646
647\section{Extending the Distutils}
Greg Warde78298a2000-04-28 17:12:24 +0000648\label{extending}
Greg Ward4a9e7222000-04-25 02:57:36 +0000649
650
651\subsection{Extending existing commands}
Greg Warde78298a2000-04-28 17:12:24 +0000652\label{extend-existing}
Greg Ward4a9e7222000-04-25 02:57:36 +0000653
654
655\subsection{Writing new commands}
Greg Warde78298a2000-04-28 17:12:24 +0000656\label{new-commands}
Greg Ward4a9e7222000-04-25 02:57:36 +0000657
658
659
Greg Ward16aafcd2000-04-09 04:06:44 +0000660\section{Reference}
Greg Warde78298a2000-04-28 17:12:24 +0000661\label{ref}
Greg Ward16aafcd2000-04-09 04:06:44 +0000662
663
Greg Wardfacb8db2000-04-09 04:32:40 +0000664\subsection{Building modules: the \protect\command{build} command family}
Greg Warde78298a2000-04-28 17:12:24 +0000665\label{build-cmds}
Greg Ward16aafcd2000-04-09 04:06:44 +0000666
Greg Wardfacb8db2000-04-09 04:32:40 +0000667\subsubsection{\protect\command{build}}
Greg Warde78298a2000-04-28 17:12:24 +0000668\label{build-cmd}
Greg Ward16aafcd2000-04-09 04:06:44 +0000669
Greg Wardfacb8db2000-04-09 04:32:40 +0000670\subsubsection{\protect\command{build\_py}}
Greg Warde78298a2000-04-28 17:12:24 +0000671\label{build-py-cmd}
Greg Ward16aafcd2000-04-09 04:06:44 +0000672
Greg Wardfacb8db2000-04-09 04:32:40 +0000673\subsubsection{\protect\command{build\_ext}}
Greg Warde78298a2000-04-28 17:12:24 +0000674\label{build-ext-cmd}
Greg Ward16aafcd2000-04-09 04:06:44 +0000675
Greg Wardfacb8db2000-04-09 04:32:40 +0000676\subsubsection{\protect\command{build\_clib}}
Greg Warde78298a2000-04-28 17:12:24 +0000677\label{build-clib-cmd}
Greg Ward16aafcd2000-04-09 04:06:44 +0000678
679
Greg Wardfacb8db2000-04-09 04:32:40 +0000680\subsection{Installing modules: the \protect\command{install} command family}
Greg Warde78298a2000-04-28 17:12:24 +0000681\label{install-cmd}
Greg Ward16aafcd2000-04-09 04:06:44 +0000682
Gregory P. Smith147e5f32000-05-12 00:58:18 +0000683The install command ensures that the build commands have been run and then
684runs the subcommands \command{install\_lib},
685\command{install\_data} and
686\command{install\_scripts}.
687
688\subsubsection{\protect\command{install\_lib}}
689\label{sec:install-lib-cmd}
690
691\subsubsection{\protect\command{install\_data}}
692\label{sec:install-data-cmd}
693This command installs all data files provided with the distribution.
694
695\subsubsection{\protect\command{install\_scripts}}
696\label{sec:install-scripts-cmd}
697This command installs all (Python) scripts in the distribution.
698
Greg Ward16aafcd2000-04-09 04:06:44 +0000699
Greg Wardfacb8db2000-04-09 04:32:40 +0000700\subsection{Cleaning up: the \protect\command{clean} command}
Greg Warde78298a2000-04-28 17:12:24 +0000701\label{clean-cmd}
Greg Ward16aafcd2000-04-09 04:06:44 +0000702
703
Greg Wardfacb8db2000-04-09 04:32:40 +0000704\subsection{Creating a source distribution: the \protect\command{sdist} command}
Greg Warde78298a2000-04-28 17:12:24 +0000705\label{sdist-cmd}
Greg Ward16aafcd2000-04-09 04:06:44 +0000706
707
708\XXX{fragment moved down from above: needs context!}
709The manifest template commands are:
710\begin{tableii}{ll}{command}{Command}{Description}
Greg Ward87da1ea2000-04-21 04:35:25 +0000711 \lineii{include \var{pat1} \var{pat2} ... }
712 {include all files matching any of the listed patterns}
713 \lineii{exclude \var{pat1} \var{pat2} ... }
714 {exclude all files matching any of the listed patterns}
715 \lineii{recursive-include \var{dir} \var{pat1} \var{pat2} ... }
716 {include all files under \var{dir} matching any of the listed patterns}
717 \lineii{recursive-exclude \var{dir} \var{pat1} \var{pat2} ...}
718 {exclude all files under \var{dir} matching any of the listed patterns}
719 \lineii{global-include \var{pat1} \var{pat2} ...}
720 {include all files anywhere in the source tree matching
721 any of the listed patterns}
722 \lineii{global-exclude \var{pat1} \var{pat2} ...}
723 {exclude all files anywhere in the source tree matching
724 any of the listed patterns}
Greg Ward16aafcd2000-04-09 04:06:44 +0000725 \lineii{prune \var{dir}}{exclude all files under \var{dir}}
726 \lineii{graft \var{dir}}{include all files under \var{dir}}
727\end{tableii}
728The patterns here are Unix-style ``glob'' patterns: \code{*} matches any
729sequence of regular filename characters, \code{?} matches any single
730regular filename character, and \code{[\var{range}]} matches any of the
731characters in \var{range} (e.g., \code{a-z}, \code{a-zA-Z},
Greg Wardfacb8db2000-04-09 04:32:40 +0000732\code{a-f0-9\_.}). The definition of ``regular filename character'' is
Greg Ward16aafcd2000-04-09 04:06:44 +0000733platform-specific: on Unix it is anything except slash; on Windows
734anything except backslash or colon; on Mac OS anything except colon.
735\XXX{Windows and Mac OS support not there yet}
736
737
Greg Wardd5767a52000-04-19 22:48:09 +0000738\subsection{Creating a ``built'' distribution: the
739 \protect\command{bdist} command family}
Greg Warde78298a2000-04-28 17:12:24 +0000740\label{bdist-cmds}
Greg Ward16aafcd2000-04-09 04:06:44 +0000741
742
Greg Wardfacb8db2000-04-09 04:32:40 +0000743\subsubsection{\protect\command{blib}}
Greg Ward16aafcd2000-04-09 04:06:44 +0000744
Greg Wardfacb8db2000-04-09 04:32:40 +0000745\subsubsection{\protect\command{blib\_dumb}}
Greg Ward16aafcd2000-04-09 04:06:44 +0000746
Greg Wardfacb8db2000-04-09 04:32:40 +0000747\subsubsection{\protect\command{blib\_rpm}}
Greg Ward16aafcd2000-04-09 04:06:44 +0000748
Greg Wardfacb8db2000-04-09 04:32:40 +0000749\subsubsection{\protect\command{blib\_wise}}
Greg Ward16aafcd2000-04-09 04:06:44 +0000750
751
752
753
754
755
756
757
Greg Wardabc52162000-02-26 00:52:48 +0000758\end{document}