Greg Ward | abc5216 | 2000-02-26 00:52:48 +0000 | [diff] [blame] | 1 | \documentclass{howto} |
| 2 | \usepackage{ltxmarkup} |
Greg Ward | facb8db | 2000-04-09 04:32:40 +0000 | [diff] [blame] | 3 | \usepackage{times} |
Greg Ward | 16aafcd | 2000-04-09 04:06:44 +0000 | [diff] [blame] | 4 | \usepackage{distutils} |
Greg Ward | abc5216 | 2000-02-26 00:52:48 +0000 | [diff] [blame] | 5 | |
Greg Ward | 16aafcd | 2000-04-09 04:06:44 +0000 | [diff] [blame] | 6 | \title{Distributing Python Modules} |
Greg Ward | abc5216 | 2000-02-26 00:52:48 +0000 | [diff] [blame] | 7 | |
Greg Ward | abc5216 | 2000-02-26 00:52:48 +0000 | [diff] [blame] | 8 | \author{Greg Ward} |
| 9 | \authoraddress{E-mail: \email{gward@python.net}} |
| 10 | |
Greg Ward | 16aafcd | 2000-04-09 04:06:44 +0000 | [diff] [blame] | 11 | |
Greg Ward | 16aafcd | 2000-04-09 04:06:44 +0000 | [diff] [blame] | 12 | |
| 13 | |
Greg Ward | abc5216 | 2000-02-26 00:52:48 +0000 | [diff] [blame] | 14 | \begin{document} |
| 15 | |
Greg Ward | facb8db | 2000-04-09 04:32:40 +0000 | [diff] [blame] | 16 | \maketitle |
| 17 | \tableofcontents |
Greg Ward | 16aafcd | 2000-04-09 04:06:44 +0000 | [diff] [blame] | 18 | |
| 19 | \section{Introduction} |
| 20 | \label{sec:intro} |
| 21 | |
| 22 | In the past, Python module developers have not had much infrastructure |
| 23 | support for distributing modules, nor have Python users had much support |
| 24 | for installing and maintaining third-party modules. With the |
| 25 | introduction of the Python Distribution Utilities (Distutils for short) |
| 26 | in Python 1.6, this situation should start to improve. |
| 27 | |
| 28 | This document only covers using the Distutils to distribute your Python |
| 29 | modules. Using the Distutils does not tie you to Python 1.6, though: |
| 30 | the Distutils work just fine with Python 1.5, and it is reasonable (and |
| 31 | expected to become commonplace) to expect users of Python 1.5 to |
| 32 | download and install the Distutils separately before they can install |
| 33 | your modules. Python 1.6 users, of course, won't have to add anything |
| 34 | to their Python installation in order to use the Distutils to install |
| 35 | third-party modules. |
| 36 | |
| 37 | This document concentrates on the role of developer/distributor: if |
| 38 | you're looking for information on installing Python modules, you should |
| 39 | refer to the ``Installing Python Modules'' manual. |
| 40 | |
| 41 | |
Greg Ward | facb8db | 2000-04-09 04:32:40 +0000 | [diff] [blame] | 42 | \section{Concepts \& Terminology} |
Greg Ward | 16aafcd | 2000-04-09 04:06:44 +0000 | [diff] [blame] | 43 | \label{sec:concepts} |
| 44 | |
| 45 | Using the Distutils is quite simple, both for module developers and for |
| 46 | users/administrators installing third-party modules. As a developer, |
| 47 | your responsibilites (apart from writing solid, well-documented and |
| 48 | well-tested code, of course!) are: |
| 49 | \begin{itemize} |
| 50 | \item write a setup script (\file{setup.py} by convention) |
| 51 | \item (optional) write a setup configuration file |
| 52 | \item create a source distribution |
| 53 | \item (optional) create one or more built (binary) distributions |
| 54 | \end{itemize} |
| 55 | Each of these tasks is covered in this document. |
| 56 | |
| 57 | Not all module developers have access to a multitude of platforms, so |
| 58 | it's not always feasible to expect them to create a multitude of built |
| 59 | distributions. It is hoped that a class of intermediaries, called |
| 60 | \emph{packagers}, will arise to take address this need. Packagers will |
| 61 | take source distributions released by module developers, build them on |
| 62 | one or more platforms, and release the resulting built distributions. |
| 63 | Thus, users on the most popular platforms will be able to install most |
| 64 | popular Python module distributions in the most natural way for their |
| 65 | platform, without having to run a single setup script or compile a line |
| 66 | of code. |
| 67 | |
| 68 | |
| 69 | \subsection{A simple example} |
| 70 | \label{sec:simple-example} |
| 71 | |
| 72 | The setup script is usually quite simple, although since it's written in |
| 73 | Python, there are no arbitrary limits to what you can do. If all you |
| 74 | want to do is distribute a module called \module{foo}, contained in a |
| 75 | file \file{foo.py}, then you can get away with as little as this: |
| 76 | \begin{verbatim} |
| 77 | from distutils.core import setup |
| 78 | setup (name = "foo", |
| 79 | version = "1.0", |
| 80 | py_modules = ["foo"]) |
| 81 | \end{verbatim} |
| 82 | Some observations: |
| 83 | \begin{itemize} |
| 84 | \item all information that you supply to the Distutils is supplied as |
Greg Ward | facb8db | 2000-04-09 04:32:40 +0000 | [diff] [blame] | 85 | keyword arguments to the \function{setup()} function |
Greg Ward | 16aafcd | 2000-04-09 04:06:44 +0000 | [diff] [blame] | 86 | \item those keyword arguments fall into two categories: package |
| 87 | meta-data (name, version number) and information about what's in the |
| 88 | package (list of pure modules, in this case) |
| 89 | \item modules are specified by module name, not filename (the same will |
| 90 | hold true for packages and extensions) |
| 91 | \item it's recommended that you supply a little more meta-data, in |
| 92 | particular your name, email address and a URL for the project |
| 93 | \end{itemize} |
| 94 | |
| 95 | To create a source distribution for this module, you would run |
| 96 | \begin{verbatim} |
| 97 | python setup.py sdist |
| 98 | \end{verbatim} |
| 99 | which will create an archive file (e.g., tarball on Unix, zip file on |
| 100 | Windows) containing your setup script, \file{setup.py}, and your module, |
| 101 | \file{foo.py}. The archive file will be named \file{Foo-1.0.tar.gz} (or |
| 102 | \file{.zip}), and will unpack into a directory \file{Foo-1.0}. |
| 103 | |
| 104 | If an end-user wishes to install your \module{foo} module, all she has |
| 105 | to do is download \file{Foo-1.0.tar.gz}) (or \file{.zip}), unpack it, |
| 106 | and---from the \file{Foo-1.0} directory---run |
| 107 | \begin{verbatim} |
| 108 | python setup.py install |
| 109 | \end{verbatim} |
| 110 | which will ultimately copy \file{foo.py} to the appropriate directory |
| 111 | for third-party modules in their Python installation. |
| 112 | |
| 113 | This simple example demonstrates some fundamental concepts of the |
| 114 | Distutils: first, both developers and installers have the same basic |
| 115 | user interface, i.e. the setup script. The difference is which |
| 116 | Distutils \emph{commands} they use: the \command{sdist} command is |
| 117 | almost exclusively for module developers, while \command{install} is |
| 118 | more often for installers (although most developers will want to install |
| 119 | their own code occasionally). |
| 120 | |
| 121 | \XXX{only partially implemented}% |
| 122 | If you want to make things really easy for your users, you can create |
| 123 | one or more built distributions for them. For instance, if you are |
| 124 | running on a Windows machine, and want to make things easy for other |
| 125 | Windows users, you can create an executable installer (the most |
| 126 | appropriate type of built distribution for this platform) with the |
| 127 | \command{bdist\_wise} command. (Wise is the installation tool used to |
| 128 | create Windows installers for Python itself, so we have adopted it for |
| 129 | use by any Python module distribution. You'll need to have version XXX |
| 130 | of Wise installed on your system for the \command{bdist\_wise} to work; |
| 131 | it's available from \url{http://foo/bar/baz}. For example: |
| 132 | \begin{verbatim} |
| 133 | python setup.py bdist_wise |
| 134 | \end{verbatim} |
Greg Ward | facb8db | 2000-04-09 04:32:40 +0000 | [diff] [blame] | 135 | will create an executable installer, \file{Foo-1\_0.exe}, in the current |
Greg Ward | 16aafcd | 2000-04-09 04:06:44 +0000 | [diff] [blame] | 136 | directory. |
| 137 | |
| 138 | \XXX{not implemented yet} |
| 139 | Other \command{bdist\_*} commands exist for RPM-based Linux systems |
| 140 | (\command{bdist\_rpm}), Debian-based Linux systems |
| 141 | (\command{bdist\_deb}), ... |
| 142 | |
| 143 | |
| 144 | \subsection{General Python terminology} |
| 145 | \label{sec:python-terms} |
| 146 | |
| 147 | If you're reading this document, you probably have a good idea of what |
| 148 | modules, extensions, and so forth are. Nevertheless, just to be sure |
| 149 | that everyone is operating from a common starting point, we offer the |
| 150 | following glossary of common Python terms: |
| 151 | \begin{description} |
| 152 | \item[module] the basic unit of code reusability in Python: a block of |
| 153 | code imported by some other code. There are three types of modules |
| 154 | that concern us here: pure Python modules, extension modules, and |
| 155 | packages. |
| 156 | \item[pure Python module] a module written in Python and contained in a |
| 157 | single \file{.py} file (and possibly associated \file{.pyc} and/or |
| 158 | \file{.pyo} files). Sometimes referred to as a ``pure module.'' |
| 159 | \item[extension module] a module written in the low-level language of |
| 160 | the Python implemention: C/C++ for CPython, Java for JPython. |
| 161 | Typically contained in a single dynamically loadable pre-compiled |
| 162 | file, e.g. a shared object (\file{.so}) file for CPython extensions on |
| 163 | Unix, a DLL (given the \file{.pyd} extension) for CPython extensions |
| 164 | on Windows, or a Java class file for JPython extensions. |
| 165 | \item[package] a module that contains other modules; typically contained |
| 166 | in a directory in the filesystem and distinguished from other |
| 167 | directories by the presence of a file \file{\_\_init\_\_.py}. |
| 168 | \end{description} |
| 169 | |
| 170 | |
| 171 | \subsection{Distutils-specific terminology} |
| 172 | \label{sec:distutils-term} |
| 173 | |
| 174 | The following terms apply more specifically to the domain of |
| 175 | distributing Python modules using the Distutils: |
| 176 | \begin{description} |
| 177 | \item[module distribution] a collection of Python modules distributed |
| 178 | together as a single downloadable resource and meant to be installed |
| 179 | \emph{en masse}. Examples of some well-known module distributions are |
| 180 | Numeric Python, PyXML, PIL (the Python Imaging Library), or |
| 181 | mxDateTime. (This would be called a \emph{package}, except that term |
| 182 | is already spoken for in the Python context: a single module |
| 183 | distribution may contain zero, one, or many Python packages.) |
| 184 | \item[pure module distribution] a module distribution that contains only |
| 185 | pure Python modules and packages. Sometimes referred to as a ``pure |
| 186 | distribution.'' |
| 187 | \item[non-pure module distribution] a module distribution that contains |
| 188 | at least one extension module. Sometimes referred to as a ``non-pure |
| 189 | distribution.'' |
| 190 | \item[distribution root] the top-level directory of your source tree (or |
| 191 | source distribution); the directory where \file{setup.py} exists and |
| 192 | is run from |
| 193 | \end{description} |
| 194 | |
| 195 | |
| 196 | \section{Writing the Setup Script} |
| 197 | \label{sec:setup-script} |
| 198 | |
| 199 | The setup script is the centre of all activity in building, |
| 200 | distributing, and installing modules using the Distutils. The main |
| 201 | purpose of the setup script is to describe your module distribution to |
| 202 | the Distutils, so that the various commands that operate on your modules |
| 203 | do the right thing. As we saw in section~\ref{sec:simple-example} |
Greg Ward | facb8db | 2000-04-09 04:32:40 +0000 | [diff] [blame] | 204 | above, the setup script consists mainly of a call to \function{setup()}, and |
Greg Ward | 16aafcd | 2000-04-09 04:06:44 +0000 | [diff] [blame] | 205 | all information supplied to the Distutils is suppled as keyword |
Greg Ward | facb8db | 2000-04-09 04:32:40 +0000 | [diff] [blame] | 206 | arguments to \function{setup()}. |
Greg Ward | 16aafcd | 2000-04-09 04:06:44 +0000 | [diff] [blame] | 207 | |
| 208 | Here's a slightly more involved example, which we'll follow for the next |
| 209 | couple of sections: the Distutils' own setup script. (Keep in mind that |
| 210 | although the Distutils are included with Python 1.6, they also have an |
| 211 | independent existence so that Python 1.5 users can use them to install |
| 212 | other module distributions.) |
| 213 | |
| 214 | \begin{verbatim} |
| 215 | #!/usr/bin/env python |
| 216 | |
| 217 | from distutils.core import setup |
| 218 | |
| 219 | setup (name = "Distutils", |
| 220 | version = "1.0", |
| 221 | description = "Python Module Distribution Utilities", |
| 222 | author = "Greg Ward", |
| 223 | author_email = "gward@python.net", |
| 224 | url = "http://www.python.org/sigs/distutils-sig/", |
| 225 | |
| 226 | packages = ['distutils', 'distutils.command'], |
| 227 | ) |
| 228 | \end{verbatim} |
| 229 | There are only two differences between this and the trivial one-file |
| 230 | distribution presented in section~\ref{sec:simple-example}: more |
| 231 | meta-data, and the specification of pure Python modules by package, |
| 232 | rather than by module. This is important since the Distutils consist of |
| 233 | a couple of dozen modules split into (so far) two packages; an explicit |
| 234 | list of every module would be tedious to generate and difficult to |
| 235 | maintain. |
| 236 | |
Greg Ward | 46b98e3 | 2000-04-14 01:53:36 +0000 | [diff] [blame] | 237 | Note that any pathnames (files or directories) supplied in the setup |
| 238 | script should be written using the Unix convention, i.e. |
| 239 | slash-separated. The Distutils will take care of converting this |
| 240 | platform-neutral representation to whatever is appropriate on your |
| 241 | current platform before actually using the pathname. This makes your |
| 242 | setup script portable across operating systems, which of course is one |
| 243 | of the major goals of the Distutils. In this spirit, all pathnames in |
| 244 | this document are slash-separated (Mac OS users should keep in mind that |
| 245 | the \emph{absence} of a leading slash indicates a relative directory, |
| 246 | the opposite of the Mac OS convention with colons). |
| 247 | |
Greg Ward | 16aafcd | 2000-04-09 04:06:44 +0000 | [diff] [blame] | 248 | |
| 249 | \subsection{Package directories} |
| 250 | \label{sec:package-dirs} |
| 251 | |
| 252 | The \option{packages} option tells the Distutils to process (build, |
| 253 | distribute, install, etc.) all pure Python modules found in each package |
| 254 | mentioned in the \option{packages} list. In order to do this, of |
| 255 | course, there has to be a correspondence between package names and |
| 256 | directories in the filesystem. The default correspondence is the most |
| 257 | obvious one, i.e. package \package{distutils} is found in the directory |
| 258 | \file{distutils} relative to the distribution root. Thus, when you say |
| 259 | \code{packages = ['foo']} in your setup script, you are promising that |
| 260 | the Distutils will find a file \file{foo/\_\_init\_\_.py} (which might |
| 261 | be spelled differently on your system, but you get the idea) relative to |
| 262 | the directory where your setup script lives. (If you break this |
| 263 | promise, the Distutils will issue a warning but process the broken |
| 264 | package anyways.) |
| 265 | |
| 266 | If you use a different convention to lay out your source directory, |
| 267 | that's no problem: you just have to supply the \option{package\_dir} |
| 268 | option to tell the Distutils about your convention. For example, say |
| 269 | you keep all Python source under \file{lib}, so that modules not in any |
| 270 | package are right in \file{lib}, modules in the \package{foo} package |
| 271 | are in \file{lib/foo}, and so forth. Then you would put |
| 272 | \begin{verbatim} |
| 273 | package_dir = {'': 'lib'} |
| 274 | \end{verbatim} |
| 275 | in your setup script. (The keys to this dictionary are package names, |
| 276 | and an empty package name stands for the ``root package,'' i.e. no |
| 277 | package at all. The values are directory names relative to your |
| 278 | distribution root.) In this case, when you say |
| 279 | \code{packages = ['foo']}, you are promising that the file |
| 280 | \file{lib/foo/\_\_init\_\_.py} exists. |
| 281 | |
| 282 | Another possible convention is to put the \package{foo} package right in |
| 283 | \file{lib}, the \package{foo.bar} package in \file{lib/bar}, etc. This |
| 284 | would be written in the setup script as |
| 285 | \begin{verbatim} |
| 286 | package_dir = {'foo': 'lib'} |
| 287 | \end{verbatim} |
| 288 | Note that a \code{\var{package}: \var{dir}} entry in the |
| 289 | \option{package\_dir} option implicitly applies to all packages below |
| 290 | \var{package}, so the \package{foo.bar} case is automatically handled |
| 291 | here. In this example, having \code{packages = ['foo', 'foo.bar']} |
Greg Ward | facb8db | 2000-04-09 04:32:40 +0000 | [diff] [blame] | 292 | tells the Distutils to look for \file{lib/\_\_init\_\_.py} and |
| 293 | \file{lib/bar/\_\_init\_\_.py}. |
Greg Ward | 16aafcd | 2000-04-09 04:06:44 +0000 | [diff] [blame] | 294 | |
| 295 | |
| 296 | \subsection{Listing individual modules} |
| 297 | \label{sec:listing-modules} |
| 298 | |
| 299 | For a small module distribution, you might prefer to list all modules |
| 300 | rather than listing packages---especially the case of a single module |
| 301 | that goes in the ``root package'' (i.e., no package at all). This |
| 302 | simplest case was shown in section~\ref{sec:simple-example}; here is a |
| 303 | slightly more involved example: |
| 304 | \begin{verbatim} |
| 305 | py_modules = ['mod1', 'pkg.mod2'] |
| 306 | \end{verbatim} |
| 307 | This describes two modules, one of them in the ``root'' package, the |
| 308 | other in the \package{pkg} package. Again, the default |
| 309 | package/directory layout implies that these two modules can be found in |
Greg Ward | facb8db | 2000-04-09 04:32:40 +0000 | [diff] [blame] | 310 | \file{mod1.py} and \file{pkg/mod2.py}, and that \file{pkg/\_\_init\_\_.py} |
Greg Ward | 16aafcd | 2000-04-09 04:06:44 +0000 | [diff] [blame] | 311 | exists as well. And again, you can override the package/directory |
| 312 | layout using the \option{package\_dir} option. \XXX{not sure if this is |
| 313 | actually true---must check!} |
| 314 | |
| 315 | |
| 316 | \section{Writing the Setup Configuration File} |
| 317 | \label{sec:setup-config} |
| 318 | |
| 319 | \XXX{not implemented yet!} |
| 320 | |
| 321 | Often, it's not possible to write down everything needed to build a |
| 322 | distribution \emph{a priori}. You need to get some information from the |
| 323 | user, or from the user's system, in order to proceed. For example, you |
| 324 | might include an optional extension module that provides an interface to |
| 325 | a particular C library. If that library is installed on the user's |
| 326 | system, then you can build your optional extension---but you need to |
| 327 | know where to find the header and library file. If it's not installed, |
| 328 | you need to know this so you can omit your optional extension. |
| 329 | |
| 330 | The preferred way to do this, of course, would be for you to tell the |
| 331 | Distutils which optional features (C libraries, system calls, external |
| 332 | utilities, etc.) you're looking for, and it would inspect the user's |
| 333 | system and try to find them. This functionality may appear in a future |
| 334 | version of the Distutils, but it isn't there now. So, for the time |
| 335 | being, we rely on the user building and installing your software to |
| 336 | provide the necessary information. The vehicle for doing so is the |
| 337 | setup configuration file, \file{setup.cfg}. |
| 338 | |
| 339 | \XXX{need more here!} |
| 340 | |
| 341 | |
| 342 | \section{Creating a Source Distribution} |
| 343 | \label{sec:source-dist} |
| 344 | |
| 345 | As shown in section~\ref{sec:simple-example}, you use the |
| 346 | \command{sdist} command to create a source distribution. In the |
| 347 | simplest case, |
| 348 | \begin{verbatim} |
| 349 | python setup.py sdist |
| 350 | \end{verbatim} |
| 351 | (assuming you haven't specified any \command{sdist} options in the setup |
| 352 | script or config file), \command{sdist} creates the the archive of the |
| 353 | default format for the current platform. The default formats are: |
| 354 | \begin{tableii}{ll}{textrm}% |
| 355 | {Platform}{Default archive format for source distributions} |
| 356 | \lineii{Unix}{gzipped tar file (\file{.tar.gz})} |
| 357 | \lineii{Windows}{zip file} |
| 358 | \end{tableii} |
| 359 | You can specify as many formats as you like using the \option{--formats} |
| 360 | option, for example: |
| 361 | \begin{verbatim} |
| 362 | python setup.py sdist --formats=gztar,zip |
| 363 | \end{verbatim} |
| 364 | to create a gzipped tarball and a zip file. The available formats are: |
Greg Ward | 46b98e3 | 2000-04-14 01:53:36 +0000 | [diff] [blame] | 365 | \begin{tableiii}{l|l|c}{code}% |
| 366 | {Format}{Description}{Notes} |
| 367 | \lineiii{zip}{zip file (\file{.zip})}{(1)} |
| 368 | \lineiii{gztar}{gzipped tar file (\file{.tar.gz})}{(2)} |
| 369 | \lineiii{ztar}{compressed tar file (\file{.tar.Z})}{} |
| 370 | \lineiii{tar}{tar file (\file{.tar})}{} |
| 371 | \end{tableiii} |
| 372 | |
| 373 | \noindent Notes: |
| 374 | \begin{description} |
| 375 | \item[(1)] default on Windows |
| 376 | \item[(2)] default on Unix |
| 377 | \end{description} |
Greg Ward | 16aafcd | 2000-04-09 04:06:44 +0000 | [diff] [blame] | 378 | |
| 379 | |
| 380 | \subsection{The manifest and manifest template} |
| 381 | \label{sec:manifest} |
| 382 | |
| 383 | Without any additional information, the \command{sdist} command puts a |
| 384 | minimal set of files into the source distribution: |
| 385 | \begin{itemize} |
Greg Ward | facb8db | 2000-04-09 04:32:40 +0000 | [diff] [blame] | 386 | \item all Python source files implied by the \option{py\_modules} and |
Greg Ward | 16aafcd | 2000-04-09 04:06:44 +0000 | [diff] [blame] | 387 | \option{packages} options |
Greg Ward | facb8db | 2000-04-09 04:32:40 +0000 | [diff] [blame] | 388 | \item all C source files mentioned in the \option{ext\_modules} or |
Greg Ward | 16aafcd | 2000-04-09 04:06:44 +0000 | [diff] [blame] | 389 | \option{libraries} options (\XXX{getting C library sources currently |
Greg Ward | facb8db | 2000-04-09 04:32:40 +0000 | [diff] [blame] | 390 | broken -- no get\_source\_files() method in build\_clib.py!}) |
Greg Ward | 16aafcd | 2000-04-09 04:06:44 +0000 | [diff] [blame] | 391 | \item anything that looks like a test script: \file{test/test*.py} |
| 392 | (currently, the Distutils don't do anything with test scripts except |
| 393 | include them in source distributions, but in the future there will be |
| 394 | a standard for testing Python module distributions) |
| 395 | \item \file{README.txt} (or \file{README}) and \file{setup.py} |
| 396 | \end{itemize} |
| 397 | Sometimes this is enough, but usually you will want to specify |
| 398 | additional files to distribute. The typical way to do this is to write |
| 399 | a \emph{manifest template}, called \file{MANIFEST.in} by default. The |
| 400 | \command{sdist} command processes this template and generates a manifest |
| 401 | file, \file{MANIFEST}. (If you prefer, you can skip the manifest |
| 402 | template and generate the manifest yourself: it just lists one file per |
| 403 | line.) |
| 404 | |
| 405 | The manifest template has one command per line, where each command |
| 406 | specifies a set of files to include or exclude from the source |
| 407 | distribution. For an example, again we turn to the Distutils' own |
| 408 | manifest template: |
| 409 | \begin{verbatim} |
| 410 | include *.txt |
| 411 | recursive-include examples *.txt |
| 412 | recursive-include examples *.py |
| 413 | prune examples/sample?/build |
| 414 | \end{verbatim} |
| 415 | The meanings should be fairly clear: include all files in the |
| 416 | distribution root matching \code{*.txt}, all files anywhere under the |
| 417 | \file{examples} directory matching \code{*.txt} or \code{*.py}, and |
| 418 | exclude all directories matching \code{examples/sample?/build}. There |
| 419 | are several other commands available in the manifest template |
| 420 | mini-language; see section~\ref{sec:sdist-cmd}. |
| 421 | |
| 422 | The order of commands in the manifest template very much matters: |
| 423 | initially, we have the list of default files as described above, and |
| 424 | each command in the template adds to or removes from that list of files. |
| 425 | When we have fully processed the manifest template, we have our complete |
| 426 | list of files. This list is written to the manifest for future |
| 427 | reference, and then used to build the source distribution archive(s). |
| 428 | |
Greg Ward | 46b98e3 | 2000-04-14 01:53:36 +0000 | [diff] [blame] | 429 | Following the Distutils' own manifest template, let's trace how the |
| 430 | \command{sdist} command will build the list of files to include in the |
| 431 | Distutils source distribution: |
Greg Ward | 16aafcd | 2000-04-09 04:06:44 +0000 | [diff] [blame] | 432 | \begin{enumerate} |
| 433 | \item include all Python source files in the \file{distutils} and |
| 434 | \file{distutils/command} subdirectories (because packages |
| 435 | corresponding to those two directories were mentioned in the |
| 436 | \option{packages} option in the setup script) |
| 437 | \item include \file{test/test*.py} (always included) |
| 438 | \item include \file{README.txt} and \file{setup.py} (always included) |
| 439 | \item include \file{*.txt} in the distribution root (this will find |
| 440 | \file{README.txt} a second time, but such redundancies are weeded out |
| 441 | later) |
| 442 | \item in the sub-tree under \file{examples}, include anything matching |
| 443 | \file{*.txt} |
| 444 | \item in the sub-tree under \file{examples}, include anything matching |
| 445 | \file{*.py} |
| 446 | \item remove all files in the sub-trees starting at directories matching |
| 447 | \file{examples/sample?/build}---this may exclude files included by the |
| 448 | previous two steps, so it's important that the \code{prune} command in |
| 449 | the manifest template comes after the two \code{recursive-include} |
| 450 | commands |
Greg Ward | facb8db | 2000-04-09 04:32:40 +0000 | [diff] [blame] | 451 | \end{enumerate} |
Greg Ward | 16aafcd | 2000-04-09 04:06:44 +0000 | [diff] [blame] | 452 | |
Greg Ward | 46b98e3 | 2000-04-14 01:53:36 +0000 | [diff] [blame] | 453 | Just like in the setup script, file and directory names in the manifest |
| 454 | template should always be slash-separated; the Distutils will take care |
| 455 | of converting them to the standard representation on your platform. |
| 456 | That way, the manifest template is portable across operating systems. |
| 457 | |
Greg Ward | 16aafcd | 2000-04-09 04:06:44 +0000 | [diff] [blame] | 458 | |
| 459 | \subsection{Manifest-related options} |
| 460 | \label{sec:manifest-options} |
| 461 | |
| 462 | The normal course of operations for the \command{sdist} command is as |
| 463 | follows: |
| 464 | \begin{itemize} |
Greg Ward | 46b98e3 | 2000-04-14 01:53:36 +0000 | [diff] [blame] | 465 | \item if the manifest file, \file{MANIFEST} doesn't exist, read |
| 466 | \file{MANIFEST.in} and create the manifest |
| 467 | \item if \file{MANIFEST.in} is more recent than \file{MANIFEST}, |
| 468 | recreate \file{MANIFEST} by reading \file{MANIFEST.in} |
Greg Ward | 16aafcd | 2000-04-09 04:06:44 +0000 | [diff] [blame] | 469 | \item use the list of files now in \file{MANIFEST} (either just |
| 470 | generated or read in) to create the source distribution archive(s) |
| 471 | \end{itemize} |
| 472 | There are a couple of options that modify this behaviour. |
| 473 | |
| 474 | First, you might want to force the manifest to be regenerated---for |
| 475 | example, if you have added or removed files or directories that match an |
| 476 | existing pattern in the manifest template, you should regenerate the |
| 477 | manifest: |
| 478 | \begin{verbatim} |
| 479 | python setup.py sdist --force-manifest |
| 480 | \end{verbatim} |
| 481 | \XXX{this is stupid, but is there a better way to do it without |
| 482 | reprocessing MANIFEST.in every single bloody time?} |
| 483 | |
| 484 | Or, you might just want to (re)generate the manifest, but not create a |
| 485 | source distribution: |
| 486 | \begin{verbatim} |
| 487 | python setup.py sdist --manifest-only |
| 488 | \end{verbatim} |
| 489 | (\option{--manifest-only} implies \option{--force-manifest}.) |
| 490 | |
| 491 | If you don't want to use the default file set, you can supply the |
| 492 | \option{--no-defaults} option. If you use \option{--no-defaults} and |
| 493 | don't supply a manifest template (or it's empty, or nothing matches the |
| 494 | patterns in it), then your source distribution will be empty. |
| 495 | |
| 496 | |
| 497 | \section{Creating Built Distributions} |
| 498 | \label{sec:built-dist} |
| 499 | |
Greg Ward | 46b98e3 | 2000-04-14 01:53:36 +0000 | [diff] [blame] | 500 | A ``built distribution'' is what you're probably used to thinking of |
| 501 | either as a ``binary package'' or an ``installer'' (depending on your |
| 502 | background). It's not necessarily binary, though, because it might |
| 503 | contain only Python source code and/or byte-code; and we don't call it a |
| 504 | package, because that word is already spoken for in Python. (And |
| 505 | ``installer'' is a term specific to the Windows world. \XXX{do Mac |
| 506 | people use it?}) |
Greg Ward | 16aafcd | 2000-04-09 04:06:44 +0000 | [diff] [blame] | 507 | |
Greg Ward | 46b98e3 | 2000-04-14 01:53:36 +0000 | [diff] [blame] | 508 | A built distribution is how you make life as easy as possible for |
| 509 | installers of your module distribution: for users of RPM-based Linux |
| 510 | systems, it's a binary RPM; for Windows users, it's an executable |
| 511 | installer; for Debian-based Linux users, it's a Debian package; and so |
| 512 | forth. Obviously, no one person will be able to create built |
| 513 | distributions for every platform under the sun, so the Distutils is |
| 514 | designed to enable module developers to concentrate on their |
| 515 | specialty---writing code and creating source distributions---while an |
| 516 | intermediary species of \emph{packager} springs up to turn source |
| 517 | distributions into build distributions for as many platforms as there |
| 518 | are packagers. |
| 519 | |
| 520 | Of course, the module developer could be his own packager; or the |
| 521 | packager could be a volunteer ``out there'' somewhere who has access to |
| 522 | a platform which the original developer does not; or it could be |
| 523 | software periodically grabbing new source distributions and turning them |
| 524 | into built distributions for as many platforms as the software has |
| 525 | access to. Regardless of the nature of the beast, a packager uses the |
| 526 | setup script and the \command{bdist} command family to generate built |
| 527 | distributions. |
| 528 | |
| 529 | As a simple example, if I run the following command in the Distutils |
| 530 | source tree: |
| 531 | \begin{verbatim} |
| 532 | python setup.py bdist |
| 533 | \end{verbatim} |
| 534 | then the Distutils builds my module distribution (the Distutils itself |
| 535 | in this case), does a ``fake'' installation (also in the \file{build} |
| 536 | directory), and creates the default type of built distribution for my |
| 537 | platform. In Distutils 0.8, only two types of built distribution are |
| 538 | supported: \code{gztar} (default on non-Linux Unix) and \code{zip} |
| 539 | (default on Windows). Thus, the above command on a Unix system creates |
| 540 | \file{Distutils-0.8.built-posix.tar.gz}; unpacking this tarball from |
| 541 | Python's \filevar{prefix} directory installs the Distutils just as |
| 542 | though you had downloaded the source distribution and run \code{python |
| 543 | setup.py install}. Obviously, for pure Python distributions, this |
| 544 | isn't a huge win---but for non-pure distributions, which include |
| 545 | extensions that would need to be compiled, it can mean the difference |
| 546 | between someone being able to use your extensions or not. |
| 547 | |
| 548 | \XXX{filenames are inaccurate here!} |
| 549 | |
| 550 | The \command{bdist} command has a \option{--format} option, similar to |
| 551 | the \command{sdist} command, that you can use to select which formats to |
| 552 | generate: for example, |
| 553 | \begin{verbatim} |
| 554 | python setup.py bdist --format=zip |
| 555 | \end{verbatim} |
| 556 | would, when run on a Unix system, create |
| 557 | \file{Distutils-0.8.built-posix.tar.gz}---again, this archive would be |
| 558 | unpacked from Python's \filevar{prefix} directory to install the |
| 559 | Distutils. |
| 560 | |
| 561 | The available formats for built distributions are: |
| 562 | \begin{tableiii}{l|l|c}{code}% |
| 563 | {Format}{Description}{Notes} |
| 564 | \lineiii{zip}{zip file (\file{.zip})}{(1)} |
| 565 | \lineiii{gztar}{gzipped tar file (\file{.tar.gz})}{(2)} |
| 566 | \lineiii{ztar}{compressed tar file (\file{.tar.Z})}{} |
| 567 | \lineiii{tar}{tar file (\file{.tar})}{} |
| 568 | \lineiii{rpm}{RPM}{(3)} |
| 569 | \lineiii{srpm}{source RPM}{} |
| 570 | \lineiii{wise}{Wise installer for Windows}{} |
| 571 | \end{tableiii} |
| 572 | |
| 573 | \noindent Notes: |
| 574 | \begin{description} |
| 575 | \item[(1)] default on Windows |
| 576 | \item[(2)] default on Unix |
| 577 | \item[(3)] not implemented yet; will be default on RPM-based Linux |
| 578 | systems |
| 579 | \item[(5)] not implemented yet; will be default on Windows |
| 580 | \end{description} |
| 581 | |
| 582 | You don't have to use the \command{bdist} command with the |
| 583 | \option{--formats} option; you can also use the command that directly |
| 584 | implements the format you're interested in. Many of these |
| 585 | \command{bdist} ``sub-commands'' actually generate several similar |
| 586 | formats; for instance, the \command{bdist\_dumb} command generates all |
| 587 | the ``dumb'' archive formats (\code{tar}, \code{ztar}, \code{gztar}, and |
| 588 | \code{zip}), and \command{bdist\_rpm} generates both binary and source |
| 589 | RPMs. The \command{bdist} sub-commands, and the formats generated by |
| 590 | each, are: |
| 591 | \begin{tableii}{l|l}{command}% |
| 592 | {Command}{Formats} |
| 593 | \lineii{bdist\_dumb}{tar, ztar, gztar, zip} |
| 594 | \lineii{bdist\_rpm}{rpm, srpm} |
| 595 | \lineii{bdist\_wise}{wise} |
| 596 | \end{tableii} |
Greg Ward | 16aafcd | 2000-04-09 04:06:44 +0000 | [diff] [blame] | 597 | |
| 598 | \section{Examples} |
| 599 | \label{sec:examples} |
| 600 | |
| 601 | |
| 602 | \subsection{Pure Python distribution (by module)} |
| 603 | \label{sec:pure-mod} |
| 604 | |
| 605 | |
| 606 | \subsection{Pure Python distribution (by package)} |
| 607 | \label{sec:pure-pkg} |
| 608 | |
| 609 | |
| 610 | \subsection{Single extension module} |
| 611 | \label{sec:single-ext} |
| 612 | |
| 613 | |
| 614 | \subsection{Multiple extension modules} |
| 615 | \label{sec:multiple-ext} |
| 616 | |
| 617 | |
| 618 | \subsection{Putting it all together} |
| 619 | |
| 620 | |
| 621 | \section{Reference} |
| 622 | \label{sec:ref} |
| 623 | |
| 624 | |
Greg Ward | facb8db | 2000-04-09 04:32:40 +0000 | [diff] [blame] | 625 | \subsection{Building modules: the \protect\command{build} command family} |
Greg Ward | 16aafcd | 2000-04-09 04:06:44 +0000 | [diff] [blame] | 626 | \label{sec:build-cmds} |
| 627 | |
Greg Ward | facb8db | 2000-04-09 04:32:40 +0000 | [diff] [blame] | 628 | \subsubsection{\protect\command{build}} |
Greg Ward | 16aafcd | 2000-04-09 04:06:44 +0000 | [diff] [blame] | 629 | \label{sec:build-cmd} |
| 630 | |
Greg Ward | facb8db | 2000-04-09 04:32:40 +0000 | [diff] [blame] | 631 | \subsubsection{\protect\command{build\_py}} |
Greg Ward | 16aafcd | 2000-04-09 04:06:44 +0000 | [diff] [blame] | 632 | \label{sec:build-py-cmd} |
| 633 | |
Greg Ward | facb8db | 2000-04-09 04:32:40 +0000 | [diff] [blame] | 634 | \subsubsection{\protect\command{build\_ext}} |
Greg Ward | 16aafcd | 2000-04-09 04:06:44 +0000 | [diff] [blame] | 635 | \label{sec:build-ext-cmd} |
| 636 | |
Greg Ward | facb8db | 2000-04-09 04:32:40 +0000 | [diff] [blame] | 637 | \subsubsection{\protect\command{build\_clib}} |
Greg Ward | 16aafcd | 2000-04-09 04:06:44 +0000 | [diff] [blame] | 638 | \label{sec:build-clib-cmd} |
| 639 | |
| 640 | |
Greg Ward | facb8db | 2000-04-09 04:32:40 +0000 | [diff] [blame] | 641 | \subsection{Installing modules: the \protect\command{install} command family} |
Greg Ward | 16aafcd | 2000-04-09 04:06:44 +0000 | [diff] [blame] | 642 | \label{sec:install-cmd} |
| 643 | |
| 644 | |
Greg Ward | facb8db | 2000-04-09 04:32:40 +0000 | [diff] [blame] | 645 | \subsection{Cleaning up: the \protect\command{clean} command} |
Greg Ward | 16aafcd | 2000-04-09 04:06:44 +0000 | [diff] [blame] | 646 | \label{sec:clean-cmd} |
| 647 | |
| 648 | |
Greg Ward | facb8db | 2000-04-09 04:32:40 +0000 | [diff] [blame] | 649 | \subsection{Creating a source distribution: the \protect\command{sdist} command} |
Greg Ward | 16aafcd | 2000-04-09 04:06:44 +0000 | [diff] [blame] | 650 | \label{sec:sdist-cmd} |
| 651 | |
| 652 | |
| 653 | \XXX{fragment moved down from above: needs context!} |
| 654 | The manifest template commands are: |
| 655 | \begin{tableii}{ll}{command}{Command}{Description} |
| 656 | \lineii{include \var{pat}}{include all files matching \var{pat}} |
| 657 | \lineii{exclude \var{pat}}{exclude all files matching \var{pat}} |
| 658 | \lineii{recursive-include \var{dir} \var{pat}} |
| 659 | {include all files under \var{dir} matching \var{pat}} |
| 660 | \lineii{recursive-exclude \var{dir} \var{pat}} |
| 661 | {exclude all files under \var{dir} matching \var{pat}} |
| 662 | \lineii{global-include \var{pat}} |
| 663 | {include all files anywhere in the source tree matching \var{pat}} |
| 664 | \lineii{global-exclude \var{pat}} |
| 665 | {exclude all files anywhere in the source tree matching \var{pat}} |
| 666 | \lineii{prune \var{dir}}{exclude all files under \var{dir}} |
| 667 | \lineii{graft \var{dir}}{include all files under \var{dir}} |
| 668 | \end{tableii} |
| 669 | The patterns here are Unix-style ``glob'' patterns: \code{*} matches any |
| 670 | sequence of regular filename characters, \code{?} matches any single |
| 671 | regular filename character, and \code{[\var{range}]} matches any of the |
| 672 | characters in \var{range} (e.g., \code{a-z}, \code{a-zA-Z}, |
Greg Ward | facb8db | 2000-04-09 04:32:40 +0000 | [diff] [blame] | 673 | \code{a-f0-9\_.}). The definition of ``regular filename character'' is |
Greg Ward | 16aafcd | 2000-04-09 04:06:44 +0000 | [diff] [blame] | 674 | platform-specific: on Unix it is anything except slash; on Windows |
| 675 | anything except backslash or colon; on Mac OS anything except colon. |
| 676 | \XXX{Windows and Mac OS support not there yet} |
| 677 | |
| 678 | |
Greg Ward | facb8db | 2000-04-09 04:32:40 +0000 | [diff] [blame] | 679 | \subsection{Creating a ``built'' distribution: the \protect\command{bdist} command |
Greg Ward | 16aafcd | 2000-04-09 04:06:44 +0000 | [diff] [blame] | 680 | family} |
| 681 | \label{sec:bdist-cmds} |
| 682 | |
| 683 | |
Greg Ward | facb8db | 2000-04-09 04:32:40 +0000 | [diff] [blame] | 684 | \subsubsection{\protect\command{blib}} |
Greg Ward | 16aafcd | 2000-04-09 04:06:44 +0000 | [diff] [blame] | 685 | |
Greg Ward | facb8db | 2000-04-09 04:32:40 +0000 | [diff] [blame] | 686 | \subsubsection{\protect\command{blib\_dumb}} |
Greg Ward | 16aafcd | 2000-04-09 04:06:44 +0000 | [diff] [blame] | 687 | |
Greg Ward | facb8db | 2000-04-09 04:32:40 +0000 | [diff] [blame] | 688 | \subsubsection{\protect\command{blib\_rpm}} |
Greg Ward | 16aafcd | 2000-04-09 04:06:44 +0000 | [diff] [blame] | 689 | |
Greg Ward | facb8db | 2000-04-09 04:32:40 +0000 | [diff] [blame] | 690 | \subsubsection{\protect\command{blib\_wise}} |
Greg Ward | 16aafcd | 2000-04-09 04:06:44 +0000 | [diff] [blame] | 691 | |
| 692 | |
| 693 | |
| 694 | |
| 695 | |
| 696 | |
| 697 | |
| 698 | |
Greg Ward | abc5216 | 2000-02-26 00:52:48 +0000 | [diff] [blame] | 699 | \end{document} |