| Nan Zhang | 8539a2a | 2018-05-15 14:00:05 -0700 | [diff] [blame] | 1 | ================================================== |
| 2 | Building and Distributing Packages with Setuptools |
| 3 | ================================================== |
| 4 | |
| 5 | ``Setuptools`` is a collection of enhancements to the Python ``distutils`` |
| 6 | that allow developers to more easily build and |
| 7 | distribute Python packages, especially ones that have dependencies on other |
| 8 | packages. |
| 9 | |
| 10 | Packages built and distributed using ``setuptools`` look to the user like |
| 11 | ordinary Python packages based on the ``distutils``. Your users don't need to |
| 12 | install or even know about setuptools in order to use them, and you don't |
| 13 | have to include the entire setuptools package in your distributions. By |
| 14 | including just a single `bootstrap module`_ (a 12K .py file), your package will |
| 15 | automatically download and install ``setuptools`` if the user is building your |
| 16 | package from source and doesn't have a suitable version already installed. |
| 17 | |
| 18 | .. _bootstrap module: https://bootstrap.pypa.io/ez_setup.py |
| 19 | |
| 20 | Feature Highlights: |
| 21 | |
| 22 | * Automatically find/download/install/upgrade dependencies at build time using |
| 23 | the `EasyInstall tool <easy_install.html>`_, |
| 24 | which supports downloading via HTTP, FTP, Subversion, and SourceForge, and |
| 25 | automatically scans web pages linked from PyPI to find download links. (It's |
| 26 | the closest thing to CPAN currently available for Python.) |
| 27 | |
| 28 | * Create `Python Eggs <http://peak.telecommunity.com/DevCenter/PythonEggs>`_ - |
| 29 | a single-file importable distribution format |
| 30 | |
| 31 | * Enhanced support for accessing data files hosted in zipped packages. |
| 32 | |
| 33 | * Automatically include all packages in your source tree, without listing them |
| 34 | individually in setup.py |
| 35 | |
| 36 | * Automatically include all relevant files in your source distributions, |
| 37 | without needing to create a ``MANIFEST.in`` file, and without having to force |
| 38 | regeneration of the ``MANIFEST`` file when your source tree changes. |
| 39 | |
| 40 | * Automatically generate wrapper scripts or Windows (console and GUI) .exe |
| 41 | files for any number of "main" functions in your project. (Note: this is not |
| 42 | a py2exe replacement; the .exe files rely on the local Python installation.) |
| 43 | |
| 44 | * Transparent Pyrex support, so that your setup.py can list ``.pyx`` files and |
| 45 | still work even when the end-user doesn't have Pyrex installed (as long as |
| 46 | you include the Pyrex-generated C in your source distribution) |
| 47 | |
| 48 | * Command aliases - create project-specific, per-user, or site-wide shortcut |
| 49 | names for commonly used commands and options |
| 50 | |
| 51 | * PyPI upload support - upload your source distributions and eggs to PyPI |
| 52 | |
| 53 | * Deploy your project in "development mode", such that it's available on |
| 54 | ``sys.path``, yet can still be edited directly from its source checkout. |
| 55 | |
| 56 | * Easily extend the distutils with new commands or ``setup()`` arguments, and |
| 57 | distribute/reuse your extensions for multiple projects, without copying code. |
| 58 | |
| 59 | * Create extensible applications and frameworks that automatically discover |
| 60 | extensions, using simple "entry points" declared in a project's setup script. |
| 61 | |
| 62 | .. contents:: **Table of Contents** |
| 63 | |
| 64 | .. _ez_setup.py: `bootstrap module`_ |
| 65 | |
| 66 | |
| 67 | ----------------- |
| 68 | Developer's Guide |
| 69 | ----------------- |
| 70 | |
| 71 | |
| 72 | Installing ``setuptools`` |
| 73 | ========================= |
| 74 | |
| 75 | Please follow the `EasyInstall Installation Instructions`_ to install the |
| 76 | current stable version of setuptools. In particular, be sure to read the |
| 77 | section on `Custom Installation Locations`_ if you are installing anywhere |
| 78 | other than Python's ``site-packages`` directory. |
| 79 | |
| 80 | .. _EasyInstall Installation Instructions: easy_install.html#installation-instructions |
| 81 | |
| 82 | .. _Custom Installation Locations: easy_install.html#custom-installation-locations |
| 83 | |
| 84 | If you want the current in-development version of setuptools, you should first |
| 85 | install a stable version, and then run:: |
| 86 | |
| 87 | ez_setup.py setuptools==dev |
| 88 | |
| 89 | This will download and install the latest development (i.e. unstable) version |
| 90 | of setuptools from the Python Subversion sandbox. |
| 91 | |
| 92 | |
| 93 | Basic Use |
| 94 | ========= |
| 95 | |
| 96 | For basic use of setuptools, just import things from setuptools instead of |
| 97 | the distutils. Here's a minimal setup script using setuptools:: |
| 98 | |
| 99 | from setuptools import setup, find_packages |
| 100 | setup( |
| 101 | name="HelloWorld", |
| 102 | version="0.1", |
| 103 | packages=find_packages(), |
| 104 | ) |
| 105 | |
| 106 | As you can see, it doesn't take much to use setuptools in a project. |
| 107 | Run that script in your project folder, alongside the Python packages |
| 108 | you have developed. |
| 109 | |
| 110 | Invoke that script to produce eggs, upload to |
| 111 | PyPI, and automatically include all packages in the directory where the |
| 112 | setup.py lives. See the `Command Reference`_ section below to see what |
| 113 | commands you can give to this setup script. For example, |
| 114 | to produce a source distribution, simply invoke:: |
| 115 | |
| 116 | python setup.py sdist |
| 117 | |
| 118 | Of course, before you release your project to PyPI, you'll want to add a bit |
| 119 | more information to your setup script to help people find or learn about your |
| 120 | project. And maybe your project will have grown by then to include a few |
| 121 | dependencies, and perhaps some data files and scripts:: |
| 122 | |
| 123 | from setuptools import setup, find_packages |
| 124 | setup( |
| 125 | name="HelloWorld", |
| 126 | version="0.1", |
| 127 | packages=find_packages(), |
| 128 | scripts=['say_hello.py'], |
| 129 | |
| 130 | # Project uses reStructuredText, so ensure that the docutils get |
| 131 | # installed or upgraded on the target machine |
| 132 | install_requires=['docutils>=0.3'], |
| 133 | |
| 134 | package_data={ |
| 135 | # If any package contains *.txt or *.rst files, include them: |
| 136 | '': ['*.txt', '*.rst'], |
| 137 | # And include any *.msg files found in the 'hello' package, too: |
| 138 | 'hello': ['*.msg'], |
| 139 | }, |
| 140 | |
| 141 | # metadata for upload to PyPI |
| 142 | author="Me", |
| 143 | author_email="me@example.com", |
| 144 | description="This is an Example Package", |
| 145 | license="PSF", |
| 146 | keywords="hello world example examples", |
| 147 | url="http://example.com/HelloWorld/", # project home page, if any |
| 148 | project_urls={ |
| 149 | "Bug Tracker": "https://bugs.example.com/HelloWorld/", |
| 150 | "Documentation": "https://docs.example.com/HelloWorld/", |
| 151 | "Source Code": "https://code.example.com/HelloWorld/", |
| 152 | } |
| 153 | |
| 154 | # could also include long_description, download_url, classifiers, etc. |
| 155 | ) |
| 156 | |
| 157 | In the sections that follow, we'll explain what most of these ``setup()`` |
| 158 | arguments do (except for the metadata ones), and the various ways you might use |
| 159 | them in your own project(s). |
| 160 | |
| 161 | |
| 162 | Specifying Your Project's Version |
| 163 | --------------------------------- |
| 164 | |
| 165 | Setuptools can work well with most versioning schemes; there are, however, a |
| 166 | few special things to watch out for, in order to ensure that setuptools and |
| 167 | EasyInstall can always tell what version of your package is newer than another |
| 168 | version. Knowing these things will also help you correctly specify what |
| 169 | versions of other projects your project depends on. |
| 170 | |
| 171 | A version consists of an alternating series of release numbers and pre-release |
| 172 | or post-release tags. A release number is a series of digits punctuated by |
| 173 | dots, such as ``2.4`` or ``0.5``. Each series of digits is treated |
| 174 | numerically, so releases ``2.1`` and ``2.1.0`` are different ways to spell the |
| 175 | same release number, denoting the first subrelease of release 2. But ``2.10`` |
| 176 | is the *tenth* subrelease of release 2, and so is a different and newer release |
| 177 | from ``2.1`` or ``2.1.0``. Leading zeros within a series of digits are also |
| 178 | ignored, so ``2.01`` is the same as ``2.1``, and different from ``2.0.1``. |
| 179 | |
| 180 | Following a release number, you can have either a pre-release or post-release |
| 181 | tag. Pre-release tags make a version be considered *older* than the version |
| 182 | they are appended to. So, revision ``2.4`` is *newer* than revision ``2.4c1``, |
| 183 | which in turn is newer than ``2.4b1`` or ``2.4a1``. Postrelease tags make |
| 184 | a version be considered *newer* than the version they are appended to. So, |
| 185 | revisions like ``2.4-1`` and ``2.4pl3`` are newer than ``2.4``, but are *older* |
| 186 | than ``2.4.1`` (which has a higher release number). |
| 187 | |
| 188 | A pre-release tag is a series of letters that are alphabetically before |
| 189 | "final". Some examples of prerelease tags would include ``alpha``, ``beta``, |
| 190 | ``a``, ``c``, ``dev``, and so on. You do not have to place a dot or dash |
| 191 | before the prerelease tag if it's immediately after a number, but it's okay to |
| 192 | do so if you prefer. Thus, ``2.4c1`` and ``2.4.c1`` and ``2.4-c1`` all |
| 193 | represent release candidate 1 of version ``2.4``, and are treated as identical |
| 194 | by setuptools. |
| 195 | |
| 196 | In addition, there are three special prerelease tags that are treated as if |
| 197 | they were the letter ``c``: ``pre``, ``preview``, and ``rc``. So, version |
| 198 | ``2.4rc1``, ``2.4pre1`` and ``2.4preview1`` are all the exact same version as |
| 199 | ``2.4c1``, and are treated as identical by setuptools. |
| 200 | |
| 201 | A post-release tag is either a series of letters that are alphabetically |
| 202 | greater than or equal to "final", or a dash (``-``). Post-release tags are |
| 203 | generally used to separate patch numbers, port numbers, build numbers, revision |
| 204 | numbers, or date stamps from the release number. For example, the version |
| 205 | ``2.4-r1263`` might denote Subversion revision 1263 of a post-release patch of |
| 206 | version ``2.4``. Or you might use ``2.4-20051127`` to denote a date-stamped |
| 207 | post-release. |
| 208 | |
| 209 | Notice that after each pre or post-release tag, you are free to place another |
| 210 | release number, followed again by more pre- or post-release tags. For example, |
| 211 | ``0.6a9.dev-r41475`` could denote Subversion revision 41475 of the in- |
| 212 | development version of the ninth alpha of release 0.6. Notice that ``dev`` is |
| 213 | a pre-release tag, so this version is a *lower* version number than ``0.6a9``, |
| 214 | which would be the actual ninth alpha of release 0.6. But the ``-r41475`` is |
| 215 | a post-release tag, so this version is *newer* than ``0.6a9.dev``. |
| 216 | |
| 217 | For the most part, setuptools' interpretation of version numbers is intuitive, |
| 218 | but here are a few tips that will keep you out of trouble in the corner cases: |
| 219 | |
| 220 | * Don't stick adjoining pre-release tags together without a dot or number |
| 221 | between them. Version ``1.9adev`` is the ``adev`` prerelease of ``1.9``, |
| 222 | *not* a development pre-release of ``1.9a``. Use ``.dev`` instead, as in |
| 223 | ``1.9a.dev``, or separate the prerelease tags with a number, as in |
| 224 | ``1.9a0dev``. ``1.9a.dev``, ``1.9a0dev``, and even ``1.9.a.dev`` are |
| 225 | identical versions from setuptools' point of view, so you can use whatever |
| 226 | scheme you prefer. |
| 227 | |
| 228 | * If you want to be certain that your chosen numbering scheme works the way |
| 229 | you think it will, you can use the ``pkg_resources.parse_version()`` function |
| 230 | to compare different version numbers:: |
| 231 | |
| 232 | >>> from pkg_resources import parse_version |
| 233 | >>> parse_version('1.9.a.dev') == parse_version('1.9a0dev') |
| 234 | True |
| 235 | >>> parse_version('2.1-rc2') < parse_version('2.1') |
| 236 | True |
| 237 | >>> parse_version('0.6a9dev-r41475') < parse_version('0.6a9') |
| 238 | True |
| 239 | |
| 240 | Once you've decided on a version numbering scheme for your project, you can |
| 241 | have setuptools automatically tag your in-development releases with various |
| 242 | pre- or post-release tags. See the following sections for more details: |
| 243 | |
| 244 | * `Tagging and "Daily Build" or "Snapshot" Releases`_ |
| 245 | * `Managing "Continuous Releases" Using Subversion`_ |
| 246 | * The `egg_info`_ command |
| 247 | |
| 248 | |
| 249 | New and Changed ``setup()`` Keywords |
| 250 | ==================================== |
| 251 | |
| 252 | The following keyword arguments to ``setup()`` are added or changed by |
| 253 | ``setuptools``. All of them are optional; you do not have to supply them |
| 254 | unless you need the associated ``setuptools`` feature. |
| 255 | |
| 256 | ``include_package_data`` |
| 257 | If set to ``True``, this tells ``setuptools`` to automatically include any |
| 258 | data files it finds inside your package directories that are specified by |
| 259 | your ``MANIFEST.in`` file. For more information, see the section below on |
| 260 | `Including Data Files`_. |
| 261 | |
| 262 | ``exclude_package_data`` |
| 263 | A dictionary mapping package names to lists of glob patterns that should |
| 264 | be *excluded* from your package directories. You can use this to trim back |
| 265 | any excess files included by ``include_package_data``. For a complete |
| 266 | description and examples, see the section below on `Including Data Files`_. |
| 267 | |
| 268 | ``package_data`` |
| 269 | A dictionary mapping package names to lists of glob patterns. For a |
| 270 | complete description and examples, see the section below on `Including |
| 271 | Data Files`_. You do not need to use this option if you are using |
| 272 | ``include_package_data``, unless you need to add e.g. files that are |
| 273 | generated by your setup script and build process. (And are therefore not |
| 274 | in source control or are files that you don't want to include in your |
| 275 | source distribution.) |
| 276 | |
| 277 | ``zip_safe`` |
| 278 | A boolean (True or False) flag specifying whether the project can be |
| 279 | safely installed and run from a zip file. If this argument is not |
| 280 | supplied, the ``bdist_egg`` command will have to analyze all of your |
| 281 | project's contents for possible problems each time it builds an egg. |
| 282 | |
| 283 | ``install_requires`` |
| 284 | A string or list of strings specifying what other distributions need to |
| 285 | be installed when this one is. See the section below on `Declaring |
| 286 | Dependencies`_ for details and examples of the format of this argument. |
| 287 | |
| 288 | ``entry_points`` |
| 289 | A dictionary mapping entry point group names to strings or lists of strings |
| 290 | defining the entry points. Entry points are used to support dynamic |
| 291 | discovery of services or plugins provided by a project. See `Dynamic |
| 292 | Discovery of Services and Plugins`_ for details and examples of the format |
| 293 | of this argument. In addition, this keyword is used to support `Automatic |
| 294 | Script Creation`_. |
| 295 | |
| 296 | ``extras_require`` |
| 297 | A dictionary mapping names of "extras" (optional features of your project) |
| 298 | to strings or lists of strings specifying what other distributions must be |
| 299 | installed to support those features. See the section below on `Declaring |
| 300 | Dependencies`_ for details and examples of the format of this argument. |
| 301 | |
| 302 | ``python_requires`` |
| 303 | A string corresponding to a version specifier (as defined in PEP 440) for |
| 304 | the Python version, used to specify the Requires-Python defined in PEP 345. |
| 305 | |
| 306 | ``setup_requires`` |
| 307 | A string or list of strings specifying what other distributions need to |
| 308 | be present in order for the *setup script* to run. ``setuptools`` will |
| 309 | attempt to obtain these (even going so far as to download them using |
| 310 | ``EasyInstall``) before processing the rest of the setup script or commands. |
| 311 | This argument is needed if you are using distutils extensions as part of |
| 312 | your build process; for example, extensions that process setup() arguments |
| 313 | and turn them into EGG-INFO metadata files. |
| 314 | |
| 315 | (Note: projects listed in ``setup_requires`` will NOT be automatically |
| 316 | installed on the system where the setup script is being run. They are |
| 317 | simply downloaded to the ./.eggs directory if they're not locally available |
| 318 | already. If you want them to be installed, as well as being available |
| 319 | when the setup script is run, you should add them to ``install_requires`` |
| 320 | **and** ``setup_requires``.) |
| 321 | |
| 322 | ``dependency_links`` |
| 323 | A list of strings naming URLs to be searched when satisfying dependencies. |
| 324 | These links will be used if needed to install packages specified by |
| 325 | ``setup_requires`` or ``tests_require``. They will also be written into |
| 326 | the egg's metadata for use by tools like EasyInstall to use when installing |
| 327 | an ``.egg`` file. |
| 328 | |
| 329 | ``namespace_packages`` |
| 330 | A list of strings naming the project's "namespace packages". A namespace |
| 331 | package is a package that may be split across multiple project |
| 332 | distributions. For example, Zope 3's ``zope`` package is a namespace |
| 333 | package, because subpackages like ``zope.interface`` and ``zope.publisher`` |
| 334 | may be distributed separately. The egg runtime system can automatically |
| 335 | merge such subpackages into a single parent package at runtime, as long |
| 336 | as you declare them in each project that contains any subpackages of the |
| 337 | namespace package, and as long as the namespace package's ``__init__.py`` |
| 338 | does not contain any code other than a namespace declaration. See the |
| 339 | section below on `Namespace Packages`_ for more information. |
| 340 | |
| 341 | ``test_suite`` |
| 342 | A string naming a ``unittest.TestCase`` subclass (or a package or module |
| 343 | containing one or more of them, or a method of such a subclass), or naming |
| 344 | a function that can be called with no arguments and returns a |
| 345 | ``unittest.TestSuite``. If the named suite is a module, and the module |
| 346 | has an ``additional_tests()`` function, it is called and the results are |
| 347 | added to the tests to be run. If the named suite is a package, any |
| 348 | submodules and subpackages are recursively added to the overall test suite. |
| 349 | |
| 350 | Specifying this argument enables use of the `test`_ command to run the |
| 351 | specified test suite, e.g. via ``setup.py test``. See the section on the |
| 352 | `test`_ command below for more details. |
| 353 | |
| 354 | ``tests_require`` |
| 355 | If your project's tests need one or more additional packages besides those |
| 356 | needed to install it, you can use this option to specify them. It should |
| 357 | be a string or list of strings specifying what other distributions need to |
| 358 | be present for the package's tests to run. When you run the ``test`` |
| 359 | command, ``setuptools`` will attempt to obtain these (even going |
| 360 | so far as to download them using ``EasyInstall``). Note that these |
| 361 | required projects will *not* be installed on the system where the tests |
| 362 | are run, but only downloaded to the project's setup directory if they're |
| 363 | not already installed locally. |
| 364 | |
| 365 | .. _test_loader: |
| 366 | |
| 367 | ``test_loader`` |
| 368 | If you would like to use a different way of finding tests to run than what |
| 369 | setuptools normally uses, you can specify a module name and class name in |
| 370 | this argument. The named class must be instantiable with no arguments, and |
| 371 | its instances must support the ``loadTestsFromNames()`` method as defined |
| 372 | in the Python ``unittest`` module's ``TestLoader`` class. Setuptools will |
| 373 | pass only one test "name" in the `names` argument: the value supplied for |
| 374 | the ``test_suite`` argument. The loader you specify may interpret this |
| 375 | string in any way it likes, as there are no restrictions on what may be |
| 376 | contained in a ``test_suite`` string. |
| 377 | |
| 378 | The module name and class name must be separated by a ``:``. The default |
| 379 | value of this argument is ``"setuptools.command.test:ScanningLoader"``. If |
| 380 | you want to use the default ``unittest`` behavior, you can specify |
| 381 | ``"unittest:TestLoader"`` as your ``test_loader`` argument instead. This |
| 382 | will prevent automatic scanning of submodules and subpackages. |
| 383 | |
| 384 | The module and class you specify here may be contained in another package, |
| 385 | as long as you use the ``tests_require`` option to ensure that the package |
| 386 | containing the loader class is available when the ``test`` command is run. |
| 387 | |
| 388 | ``eager_resources`` |
| 389 | A list of strings naming resources that should be extracted together, if |
| 390 | any of them is needed, or if any C extensions included in the project are |
| 391 | imported. This argument is only useful if the project will be installed as |
| 392 | a zipfile, and there is a need to have all of the listed resources be |
| 393 | extracted to the filesystem *as a unit*. Resources listed here |
| 394 | should be '/'-separated paths, relative to the source root, so to list a |
| 395 | resource ``foo.png`` in package ``bar.baz``, you would include the string |
| 396 | ``bar/baz/foo.png`` in this argument. |
| 397 | |
| 398 | If you only need to obtain resources one at a time, or you don't have any C |
| 399 | extensions that access other files in the project (such as data files or |
| 400 | shared libraries), you probably do NOT need this argument and shouldn't |
| 401 | mess with it. For more details on how this argument works, see the section |
| 402 | below on `Automatic Resource Extraction`_. |
| 403 | |
| 404 | ``use_2to3`` |
| 405 | Convert the source code from Python 2 to Python 3 with 2to3 during the |
| 406 | build process. See :doc:`python3` for more details. |
| 407 | |
| 408 | ``convert_2to3_doctests`` |
| 409 | List of doctest source files that need to be converted with 2to3. |
| 410 | See :doc:`python3` for more details. |
| 411 | |
| 412 | ``use_2to3_fixers`` |
| 413 | A list of modules to search for additional fixers to be used during |
| 414 | the 2to3 conversion. See :doc:`python3` for more details. |
| 415 | |
| 416 | ``project_urls`` |
| 417 | An arbitrary map of URL names to hyperlinks, allowing more extensible |
| 418 | documentation of where various resources can be found than the simple |
| 419 | ``url`` and ``download_url`` options provide. |
| 420 | |
| 421 | |
| 422 | Using ``find_packages()`` |
| 423 | ------------------------- |
| 424 | |
| 425 | For simple projects, it's usually easy enough to manually add packages to |
| 426 | the ``packages`` argument of ``setup()``. However, for very large projects |
| 427 | (Twisted, PEAK, Zope, Chandler, etc.), it can be a big burden to keep the |
| 428 | package list updated. That's what ``setuptools.find_packages()`` is for. |
| 429 | |
| 430 | ``find_packages()`` takes a source directory and two lists of package name |
| 431 | patterns to exclude and include. If omitted, the source directory defaults to |
| 432 | the same |
| 433 | directory as the setup script. Some projects use a ``src`` or ``lib`` |
| 434 | directory as the root of their source tree, and those projects would of course |
| 435 | use ``"src"`` or ``"lib"`` as the first argument to ``find_packages()``. (And |
| 436 | such projects also need something like ``package_dir={'':'src'}`` in their |
| 437 | ``setup()`` arguments, but that's just a normal distutils thing.) |
| 438 | |
| 439 | Anyway, ``find_packages()`` walks the target directory, filtering by inclusion |
| 440 | patterns, and finds Python packages (any directory). Packages are only |
| 441 | recognized if they include an ``__init__.py`` file. Finally, exclusion |
| 442 | patterns are applied to remove matching packages. |
| 443 | |
| 444 | Inclusion and exclusion patterns are package names, optionally including |
| 445 | wildcards. For |
| 446 | example, ``find_packages(exclude=["*.tests"])`` will exclude all packages whose |
| 447 | last name part is ``tests``. Or, ``find_packages(exclude=["*.tests", |
| 448 | "*.tests.*"])`` will also exclude any subpackages of packages named ``tests``, |
| 449 | but it still won't exclude a top-level ``tests`` package or the children |
| 450 | thereof. In fact, if you really want no ``tests`` packages at all, you'll need |
| 451 | something like this:: |
| 452 | |
| 453 | find_packages(exclude=["*.tests", "*.tests.*", "tests.*", "tests"]) |
| 454 | |
| 455 | in order to cover all the bases. Really, the exclusion patterns are intended |
| 456 | to cover simpler use cases than this, like excluding a single, specified |
| 457 | package and its subpackages. |
| 458 | |
| 459 | Regardless of the parameters, the ``find_packages()`` |
| 460 | function returns a list of package names suitable for use as the ``packages`` |
| 461 | argument to ``setup()``, and so is usually the easiest way to set that |
| 462 | argument in your setup script. Especially since it frees you from having to |
| 463 | remember to modify your setup script whenever your project grows additional |
| 464 | top-level packages or subpackages. |
| 465 | |
| 466 | |
| 467 | Automatic Script Creation |
| 468 | ========================= |
| 469 | |
| 470 | Packaging and installing scripts can be a bit awkward with the distutils. For |
| 471 | one thing, there's no easy way to have a script's filename match local |
| 472 | conventions on both Windows and POSIX platforms. For another, you often have |
| 473 | to create a separate file just for the "main" script, when your actual "main" |
| 474 | is a function in a module somewhere. And even in Python 2.4, using the ``-m`` |
| 475 | option only works for actual ``.py`` files that aren't installed in a package. |
| 476 | |
| 477 | ``setuptools`` fixes all of these problems by automatically generating scripts |
| 478 | for you with the correct extension, and on Windows it will even create an |
| 479 | ``.exe`` file so that users don't have to change their ``PATHEXT`` settings. |
| 480 | The way to use this feature is to define "entry points" in your setup script |
| 481 | that indicate what function the generated script should import and run. For |
| 482 | example, to create two console scripts called ``foo`` and ``bar``, and a GUI |
| 483 | script called ``baz``, you might do something like this:: |
| 484 | |
| 485 | setup( |
| 486 | # other arguments here... |
| 487 | entry_points={ |
| 488 | 'console_scripts': [ |
| 489 | 'foo = my_package.some_module:main_func', |
| 490 | 'bar = other_module:some_func', |
| 491 | ], |
| 492 | 'gui_scripts': [ |
| 493 | 'baz = my_package_gui:start_func', |
| 494 | ] |
| 495 | } |
| 496 | ) |
| 497 | |
| 498 | When this project is installed on non-Windows platforms (using "setup.py |
| 499 | install", "setup.py develop", or by using EasyInstall), a set of ``foo``, |
| 500 | ``bar``, and ``baz`` scripts will be installed that import ``main_func`` and |
| 501 | ``some_func`` from the specified modules. The functions you specify are called |
| 502 | with no arguments, and their return value is passed to ``sys.exit()``, so you |
| 503 | can return an errorlevel or message to print to stderr. |
| 504 | |
| 505 | On Windows, a set of ``foo.exe``, ``bar.exe``, and ``baz.exe`` launchers are |
| 506 | created, alongside a set of ``foo.py``, ``bar.py``, and ``baz.pyw`` files. The |
| 507 | ``.exe`` wrappers find and execute the right version of Python to run the |
| 508 | ``.py`` or ``.pyw`` file. |
| 509 | |
| 510 | You may define as many "console script" and "gui script" entry points as you |
| 511 | like, and each one can optionally specify "extras" that it depends on, that |
| 512 | will be added to ``sys.path`` when the script is run. For more information on |
| 513 | "extras", see the section below on `Declaring Extras`_. For more information |
| 514 | on "entry points" in general, see the section below on `Dynamic Discovery of |
| 515 | Services and Plugins`_. |
| 516 | |
| 517 | |
| 518 | "Eggsecutable" Scripts |
| 519 | ---------------------- |
| 520 | |
| 521 | Occasionally, there are situations where it's desirable to make an ``.egg`` |
| 522 | file directly executable. You can do this by including an entry point such |
| 523 | as the following:: |
| 524 | |
| 525 | setup( |
| 526 | # other arguments here... |
| 527 | entry_points={ |
| 528 | 'setuptools.installation': [ |
| 529 | 'eggsecutable = my_package.some_module:main_func', |
| 530 | ] |
| 531 | } |
| 532 | ) |
| 533 | |
| 534 | Any eggs built from the above setup script will include a short executable |
| 535 | prelude that imports and calls ``main_func()`` from ``my_package.some_module``. |
| 536 | The prelude can be run on Unix-like platforms (including Mac and Linux) by |
| 537 | invoking the egg with ``/bin/sh``, or by enabling execute permissions on the |
| 538 | ``.egg`` file. For the executable prelude to run, the appropriate version of |
| 539 | Python must be available via the ``PATH`` environment variable, under its |
| 540 | "long" name. That is, if the egg is built for Python 2.3, there must be a |
| 541 | ``python2.3`` executable present in a directory on ``PATH``. |
| 542 | |
| 543 | This feature is primarily intended to support ez_setup the installation of |
| 544 | setuptools itself on non-Windows platforms, but may also be useful for other |
| 545 | projects as well. |
| 546 | |
| 547 | IMPORTANT NOTE: Eggs with an "eggsecutable" header cannot be renamed, or |
| 548 | invoked via symlinks. They *must* be invoked using their original filename, in |
| 549 | order to ensure that, once running, ``pkg_resources`` will know what project |
| 550 | and version is in use. The header script will check this and exit with an |
| 551 | error if the ``.egg`` file has been renamed or is invoked via a symlink that |
| 552 | changes its base name. |
| 553 | |
| 554 | |
| 555 | Declaring Dependencies |
| 556 | ====================== |
| 557 | |
| 558 | ``setuptools`` supports automatically installing dependencies when a package is |
| 559 | installed, and including information about dependencies in Python Eggs (so that |
| 560 | package management tools like EasyInstall can use the information). |
| 561 | |
| 562 | ``setuptools`` and ``pkg_resources`` use a common syntax for specifying a |
| 563 | project's required dependencies. This syntax consists of a project's PyPI |
| 564 | name, optionally followed by a comma-separated list of "extras" in square |
| 565 | brackets, optionally followed by a comma-separated list of version |
| 566 | specifiers. A version specifier is one of the operators ``<``, ``>``, ``<=``, |
| 567 | ``>=``, ``==`` or ``!=``, followed by a version identifier. Tokens may be |
| 568 | separated by whitespace, but any whitespace or nonstandard characters within a |
| 569 | project name or version identifier must be replaced with ``-``. |
| 570 | |
| 571 | Version specifiers for a given project are internally sorted into ascending |
| 572 | version order, and used to establish what ranges of versions are acceptable. |
| 573 | Adjacent redundant conditions are also consolidated (e.g. ``">1, >2"`` becomes |
| 574 | ``">2"``, and ``"<2,<3"`` becomes ``"<2"``). ``"!="`` versions are excised from |
| 575 | the ranges they fall within. A project's version is then checked for |
| 576 | membership in the resulting ranges. (Note that providing conflicting conditions |
| 577 | for the same version (e.g. "<2,>=2" or "==2,!=2") is meaningless and may |
| 578 | therefore produce bizarre results.) |
| 579 | |
| 580 | Here are some example requirement specifiers:: |
| 581 | |
| 582 | docutils >= 0.3 |
| 583 | |
| 584 | # comment lines and \ continuations are allowed in requirement strings |
| 585 | BazSpam ==1.1, ==1.2, ==1.3, ==1.4, ==1.5, \ |
| 586 | ==1.6, ==1.7 # and so are line-end comments |
| 587 | |
| 588 | PEAK[FastCGI, reST]>=0.5a4 |
| 589 | |
| 590 | setuptools==0.5a7 |
| 591 | |
| 592 | The simplest way to include requirement specifiers is to use the |
| 593 | ``install_requires`` argument to ``setup()``. It takes a string or list of |
| 594 | strings containing requirement specifiers. If you include more than one |
| 595 | requirement in a string, each requirement must begin on a new line. |
| 596 | |
| 597 | This has three effects: |
| 598 | |
| 599 | 1. When your project is installed, either by using EasyInstall, ``setup.py |
| 600 | install``, or ``setup.py develop``, all of the dependencies not already |
| 601 | installed will be located (via PyPI), downloaded, built (if necessary), |
| 602 | and installed. |
| 603 | |
| 604 | 2. Any scripts in your project will be installed with wrappers that verify |
| 605 | the availability of the specified dependencies at runtime, and ensure that |
| 606 | the correct versions are added to ``sys.path`` (e.g. if multiple versions |
| 607 | have been installed). |
| 608 | |
| 609 | 3. Python Egg distributions will include a metadata file listing the |
| 610 | dependencies. |
| 611 | |
| 612 | Note, by the way, that if you declare your dependencies in ``setup.py``, you do |
| 613 | *not* need to use the ``require()`` function in your scripts or modules, as |
| 614 | long as you either install the project or use ``setup.py develop`` to do |
| 615 | development work on it. (See `"Development Mode"`_ below for more details on |
| 616 | using ``setup.py develop``.) |
| 617 | |
| 618 | |
| 619 | Dependencies that aren't in PyPI |
| 620 | -------------------------------- |
| 621 | |
| 622 | If your project depends on packages that aren't registered in PyPI, you may |
| 623 | still be able to depend on them, as long as they are available for download |
| 624 | as: |
| 625 | |
| 626 | - an egg, in the standard distutils ``sdist`` format, |
| 627 | - a single ``.py`` file, or |
| 628 | - a VCS repository (Subversion, Mercurial, or Git). |
| 629 | |
| 630 | You just need to add some URLs to the ``dependency_links`` argument to |
| 631 | ``setup()``. |
| 632 | |
| 633 | The URLs must be either: |
| 634 | |
| 635 | 1. direct download URLs, |
| 636 | 2. the URLs of web pages that contain direct download links, or |
| 637 | 3. the repository's URL |
| 638 | |
| 639 | In general, it's better to link to web pages, because it is usually less |
| 640 | complex to update a web page than to release a new version of your project. |
| 641 | You can also use a SourceForge ``showfiles.php`` link in the case where a |
| 642 | package you depend on is distributed via SourceForge. |
| 643 | |
| 644 | If you depend on a package that's distributed as a single ``.py`` file, you |
| 645 | must include an ``"#egg=project-version"`` suffix to the URL, to give a project |
| 646 | name and version number. (Be sure to escape any dashes in the name or version |
| 647 | by replacing them with underscores.) EasyInstall will recognize this suffix |
| 648 | and automatically create a trivial ``setup.py`` to wrap the single ``.py`` file |
| 649 | as an egg. |
| 650 | |
| 651 | In the case of a VCS checkout, you should also append ``#egg=project-version`` |
| 652 | in order to identify for what package that checkout should be used. You can |
| 653 | append ``@REV`` to the URL's path (before the fragment) to specify a revision. |
| 654 | Additionally, you can also force the VCS being used by prepending the URL with |
| 655 | a certain prefix. Currently available are: |
| 656 | |
| 657 | - ``svn+URL`` for Subversion, |
| 658 | - ``git+URL`` for Git, and |
| 659 | - ``hg+URL`` for Mercurial |
| 660 | |
| 661 | A more complete example would be: |
| 662 | |
| 663 | ``vcs+proto://host/path@revision#egg=project-version`` |
| 664 | |
| 665 | Be careful with the version. It should match the one inside the project files. |
| 666 | If you want to disregard the version, you have to omit it both in the |
| 667 | ``requires`` and in the URL's fragment. |
| 668 | |
| 669 | This will do a checkout (or a clone, in Git and Mercurial parlance) to a |
| 670 | temporary folder and run ``setup.py bdist_egg``. |
| 671 | |
| 672 | The ``dependency_links`` option takes the form of a list of URL strings. For |
| 673 | example, the below will cause EasyInstall to search the specified page for |
| 674 | eggs or source distributions, if the package's dependencies aren't already |
| 675 | installed:: |
| 676 | |
| 677 | setup( |
| 678 | ... |
| 679 | dependency_links=[ |
| 680 | "http://peak.telecommunity.com/snapshots/" |
| 681 | ], |
| 682 | ) |
| 683 | |
| 684 | |
| 685 | .. _Declaring Extras: |
| 686 | |
| 687 | |
| 688 | Declaring "Extras" (optional features with their own dependencies) |
| 689 | ------------------------------------------------------------------ |
| 690 | |
| 691 | Sometimes a project has "recommended" dependencies, that are not required for |
| 692 | all uses of the project. For example, a project might offer optional PDF |
| 693 | output if ReportLab is installed, and reStructuredText support if docutils is |
| 694 | installed. These optional features are called "extras", and setuptools allows |
| 695 | you to define their requirements as well. In this way, other projects that |
| 696 | require these optional features can force the additional requirements to be |
| 697 | installed, by naming the desired extras in their ``install_requires``. |
| 698 | |
| 699 | For example, let's say that Project A offers optional PDF and reST support:: |
| 700 | |
| 701 | setup( |
| 702 | name="Project-A", |
| 703 | ... |
| 704 | extras_require={ |
| 705 | 'PDF': ["ReportLab>=1.2", "RXP"], |
| 706 | 'reST': ["docutils>=0.3"], |
| 707 | } |
| 708 | ) |
| 709 | |
| 710 | As you can see, the ``extras_require`` argument takes a dictionary mapping |
| 711 | names of "extra" features, to strings or lists of strings describing those |
| 712 | features' requirements. These requirements will *not* be automatically |
| 713 | installed unless another package depends on them (directly or indirectly) by |
| 714 | including the desired "extras" in square brackets after the associated project |
| 715 | name. (Or if the extras were listed in a requirement spec on the EasyInstall |
| 716 | command line.) |
| 717 | |
| 718 | Extras can be used by a project's `entry points`_ to specify dynamic |
| 719 | dependencies. For example, if Project A includes a "rst2pdf" script, it might |
| 720 | declare it like this, so that the "PDF" requirements are only resolved if the |
| 721 | "rst2pdf" script is run:: |
| 722 | |
| 723 | setup( |
| 724 | name="Project-A", |
| 725 | ... |
| 726 | entry_points={ |
| 727 | 'console_scripts': [ |
| 728 | 'rst2pdf = project_a.tools.pdfgen [PDF]', |
| 729 | 'rst2html = project_a.tools.htmlgen', |
| 730 | # more script entry points ... |
| 731 | ], |
| 732 | } |
| 733 | ) |
| 734 | |
| 735 | Projects can also use another project's extras when specifying dependencies. |
| 736 | For example, if project B needs "project A" with PDF support installed, it |
| 737 | might declare the dependency like this:: |
| 738 | |
| 739 | setup( |
| 740 | name="Project-B", |
| 741 | install_requires=["Project-A[PDF]"], |
| 742 | ... |
| 743 | ) |
| 744 | |
| 745 | This will cause ReportLab to be installed along with project A, if project B is |
| 746 | installed -- even if project A was already installed. In this way, a project |
| 747 | can encapsulate groups of optional "downstream dependencies" under a feature |
| 748 | name, so that packages that depend on it don't have to know what the downstream |
| 749 | dependencies are. If a later version of Project A builds in PDF support and |
| 750 | no longer needs ReportLab, or if it ends up needing other dependencies besides |
| 751 | ReportLab in order to provide PDF support, Project B's setup information does |
| 752 | not need to change, but the right packages will still be installed if needed. |
| 753 | |
| 754 | Note, by the way, that if a project ends up not needing any other packages to |
| 755 | support a feature, it should keep an empty requirements list for that feature |
| 756 | in its ``extras_require`` argument, so that packages depending on that feature |
| 757 | don't break (due to an invalid feature name). For example, if Project A above |
| 758 | builds in PDF support and no longer needs ReportLab, it could change its |
| 759 | setup to this:: |
| 760 | |
| 761 | setup( |
| 762 | name="Project-A", |
| 763 | ... |
| 764 | extras_require={ |
| 765 | 'PDF': [], |
| 766 | 'reST': ["docutils>=0.3"], |
| 767 | } |
| 768 | ) |
| 769 | |
| 770 | so that Package B doesn't have to remove the ``[PDF]`` from its requirement |
| 771 | specifier. |
| 772 | |
| 773 | |
| 774 | .. _Platform Specific Dependencies: |
| 775 | |
| 776 | |
| 777 | Declaring platform specific dependencies |
| 778 | ---------------------------------------- |
| 779 | |
| 780 | Sometimes a project might require a dependency to run on a specific platform. |
| 781 | This could to a package that back ports a module so that it can be used in |
| 782 | older python versions. Or it could be a package that is required to run on a |
| 783 | specific operating system. This will allow a project to work on multiple |
| 784 | different platforms without installing dependencies that are not required for |
| 785 | a platform that is installing the project. |
| 786 | |
| 787 | For example, here is a project that uses the ``enum`` module and ``pywin32``:: |
| 788 | |
| 789 | setup( |
| 790 | name="Project", |
| 791 | ... |
| 792 | install_requires=[ |
| 793 | 'enum34;python_version<"3.4"', |
| 794 | 'pywin32 >= 1.0;platform_system=="Windows"' |
| 795 | ] |
| 796 | ) |
| 797 | |
| 798 | Since the ``enum`` module was added in Python 3.4, it should only be installed |
| 799 | if the python version is earlier. Since ``pywin32`` will only be used on |
| 800 | windows, it should only be installed when the operating system is Windows. |
| 801 | Specifying version requirements for the dependencies is supported as normal. |
| 802 | |
| 803 | The environmental markers that may be used for testing platform types are |
| 804 | detailed in `PEP 508`_. |
| 805 | |
| 806 | .. _PEP 508: https://www.python.org/dev/peps/pep-0508/ |
| 807 | |
| 808 | Including Data Files |
| 809 | ==================== |
| 810 | |
| 811 | The distutils have traditionally allowed installation of "data files", which |
| 812 | are placed in a platform-specific location. However, the most common use case |
| 813 | for data files distributed with a package is for use *by* the package, usually |
| 814 | by including the data files in the package directory. |
| 815 | |
| 816 | Setuptools offers three ways to specify data files to be included in your |
| 817 | packages. First, you can simply use the ``include_package_data`` keyword, |
| 818 | e.g.:: |
| 819 | |
| 820 | from setuptools import setup, find_packages |
| 821 | setup( |
| 822 | ... |
| 823 | include_package_data=True |
| 824 | ) |
| 825 | |
| 826 | This tells setuptools to install any data files it finds in your packages. |
| 827 | The data files must be specified via the distutils' ``MANIFEST.in`` file. |
| 828 | (They can also be tracked by a revision control system, using an appropriate |
| 829 | plugin. See the section below on `Adding Support for Revision Control |
| 830 | Systems`_ for information on how to write such plugins.) |
| 831 | |
| 832 | If you want finer-grained control over what files are included (for example, |
| 833 | if you have documentation files in your package directories and want to exclude |
| 834 | them from installation), then you can also use the ``package_data`` keyword, |
| 835 | e.g.:: |
| 836 | |
| 837 | from setuptools import setup, find_packages |
| 838 | setup( |
| 839 | ... |
| 840 | package_data={ |
| 841 | # If any package contains *.txt or *.rst files, include them: |
| 842 | '': ['*.txt', '*.rst'], |
| 843 | # And include any *.msg files found in the 'hello' package, too: |
| 844 | 'hello': ['*.msg'], |
| 845 | } |
| 846 | ) |
| 847 | |
| 848 | The ``package_data`` argument is a dictionary that maps from package names to |
| 849 | lists of glob patterns. The globs may include subdirectory names, if the data |
| 850 | files are contained in a subdirectory of the package. For example, if the |
| 851 | package tree looks like this:: |
| 852 | |
| 853 | setup.py |
| 854 | src/ |
| 855 | mypkg/ |
| 856 | __init__.py |
| 857 | mypkg.txt |
| 858 | data/ |
| 859 | somefile.dat |
| 860 | otherdata.dat |
| 861 | |
| 862 | The setuptools setup file might look like this:: |
| 863 | |
| 864 | from setuptools import setup, find_packages |
| 865 | setup( |
| 866 | ... |
| 867 | packages=find_packages('src'), # include all packages under src |
| 868 | package_dir={'':'src'}, # tell distutils packages are under src |
| 869 | |
| 870 | package_data={ |
| 871 | # If any package contains *.txt files, include them: |
| 872 | '': ['*.txt'], |
| 873 | # And include any *.dat files found in the 'data' subdirectory |
| 874 | # of the 'mypkg' package, also: |
| 875 | 'mypkg': ['data/*.dat'], |
| 876 | } |
| 877 | ) |
| 878 | |
| 879 | Notice that if you list patterns in ``package_data`` under the empty string, |
| 880 | these patterns are used to find files in every package, even ones that also |
| 881 | have their own patterns listed. Thus, in the above example, the ``mypkg.txt`` |
| 882 | file gets included even though it's not listed in the patterns for ``mypkg``. |
| 883 | |
| 884 | Also notice that if you use paths, you *must* use a forward slash (``/``) as |
| 885 | the path separator, even if you are on Windows. Setuptools automatically |
| 886 | converts slashes to appropriate platform-specific separators at build time. |
| 887 | |
| 888 | If datafiles are contained in a subdirectory of a package that isn't a package |
| 889 | itself (no ``__init__.py``), then the subdirectory names (or ``*``) are required |
| 890 | in the ``package_data`` argument (as shown above with ``'data/*.dat'``). |
| 891 | |
| 892 | When building an ``sdist``, the datafiles are also drawn from the |
| 893 | ``package_name.egg-info/SOURCES.txt`` file, so make sure that this is removed if |
| 894 | the ``setup.py`` ``package_data`` list is updated before calling ``setup.py``. |
| 895 | |
| 896 | (Note: although the ``package_data`` argument was previously only available in |
| 897 | ``setuptools``, it was also added to the Python ``distutils`` package as of |
| 898 | Python 2.4; there is `some documentation for the feature`__ available on the |
| 899 | python.org website. If using the setuptools-specific ``include_package_data`` |
| 900 | argument, files specified by ``package_data`` will *not* be automatically |
| 901 | added to the manifest unless they are listed in the MANIFEST.in file.) |
| 902 | |
| 903 | __ http://docs.python.org/dist/node11.html |
| 904 | |
| 905 | Sometimes, the ``include_package_data`` or ``package_data`` options alone |
| 906 | aren't sufficient to precisely define what files you want included. For |
| 907 | example, you may want to include package README files in your revision control |
| 908 | system and source distributions, but exclude them from being installed. So, |
| 909 | setuptools offers an ``exclude_package_data`` option as well, that allows you |
| 910 | to do things like this:: |
| 911 | |
| 912 | from setuptools import setup, find_packages |
| 913 | setup( |
| 914 | ... |
| 915 | packages=find_packages('src'), # include all packages under src |
| 916 | package_dir={'':'src'}, # tell distutils packages are under src |
| 917 | |
| 918 | include_package_data=True, # include everything in source control |
| 919 | |
| 920 | # ...but exclude README.txt from all packages |
| 921 | exclude_package_data={'': ['README.txt']}, |
| 922 | ) |
| 923 | |
| 924 | The ``exclude_package_data`` option is a dictionary mapping package names to |
| 925 | lists of wildcard patterns, just like the ``package_data`` option. And, just |
| 926 | as with that option, a key of ``''`` will apply the given pattern(s) to all |
| 927 | packages. However, any files that match these patterns will be *excluded* |
| 928 | from installation, even if they were listed in ``package_data`` or were |
| 929 | included as a result of using ``include_package_data``. |
| 930 | |
| 931 | In summary, the three options allow you to: |
| 932 | |
| 933 | ``include_package_data`` |
| 934 | Accept all data files and directories matched by ``MANIFEST.in``. |
| 935 | |
| 936 | ``package_data`` |
| 937 | Specify additional patterns to match files that may or may |
| 938 | not be matched by ``MANIFEST.in`` or found in source control. |
| 939 | |
| 940 | ``exclude_package_data`` |
| 941 | Specify patterns for data files and directories that should *not* be |
| 942 | included when a package is installed, even if they would otherwise have |
| 943 | been included due to the use of the preceding options. |
| 944 | |
| 945 | NOTE: Due to the way the distutils build process works, a data file that you |
| 946 | include in your project and then stop including may be "orphaned" in your |
| 947 | project's build directories, requiring you to run ``setup.py clean --all`` to |
| 948 | fully remove them. This may also be important for your users and contributors |
| 949 | if they track intermediate revisions of your project using Subversion; be sure |
| 950 | to let them know when you make changes that remove files from inclusion so they |
| 951 | can run ``setup.py clean --all``. |
| 952 | |
| 953 | |
| 954 | Accessing Data Files at Runtime |
| 955 | ------------------------------- |
| 956 | |
| 957 | Typically, existing programs manipulate a package's ``__file__`` attribute in |
| 958 | order to find the location of data files. However, this manipulation isn't |
| 959 | compatible with PEP 302-based import hooks, including importing from zip files |
| 960 | and Python Eggs. It is strongly recommended that, if you are using data files, |
| 961 | you should use the :ref:`ResourceManager API` of ``pkg_resources`` to access |
| 962 | them. The ``pkg_resources`` module is distributed as part of setuptools, so if |
| 963 | you're using setuptools to distribute your package, there is no reason not to |
| 964 | use its resource management API. See also `Accessing Package Resources`_ for |
| 965 | a quick example of converting code that uses ``__file__`` to use |
| 966 | ``pkg_resources`` instead. |
| 967 | |
| 968 | .. _Accessing Package Resources: http://peak.telecommunity.com/DevCenter/PythonEggs#accessing-package-resources |
| 969 | |
| 970 | |
| 971 | Non-Package Data Files |
| 972 | ---------------------- |
| 973 | |
| 974 | The ``distutils`` normally install general "data files" to a platform-specific |
| 975 | location (e.g. ``/usr/share``). This feature intended to be used for things |
| 976 | like documentation, example configuration files, and the like. ``setuptools`` |
| 977 | does not install these data files in a separate location, however. They are |
| 978 | bundled inside the egg file or directory, alongside the Python modules and |
| 979 | packages. The data files can also be accessed using the :ref:`ResourceManager |
| 980 | API`, by specifying a ``Requirement`` instead of a package name:: |
| 981 | |
| 982 | from pkg_resources import Requirement, resource_filename |
| 983 | filename = resource_filename(Requirement.parse("MyProject"),"sample.conf") |
| 984 | |
| 985 | The above code will obtain the filename of the "sample.conf" file in the data |
| 986 | root of the "MyProject" distribution. |
| 987 | |
| 988 | Note, by the way, that this encapsulation of data files means that you can't |
| 989 | actually install data files to some arbitrary location on a user's machine; |
| 990 | this is a feature, not a bug. You can always include a script in your |
| 991 | distribution that extracts and copies your the documentation or data files to |
| 992 | a user-specified location, at their discretion. If you put related data files |
| 993 | in a single directory, you can use ``resource_filename()`` with the directory |
| 994 | name to get a filesystem directory that then can be copied with the ``shutil`` |
| 995 | module. (Even if your package is installed as a zipfile, calling |
| 996 | ``resource_filename()`` on a directory will return an actual filesystem |
| 997 | directory, whose contents will be that entire subtree of your distribution.) |
| 998 | |
| 999 | (Of course, if you're writing a new package, you can just as easily place your |
| 1000 | data files or directories inside one of your packages, rather than using the |
| 1001 | distutils' approach. However, if you're updating an existing application, it |
| 1002 | may be simpler not to change the way it currently specifies these data files.) |
| 1003 | |
| 1004 | |
| 1005 | Automatic Resource Extraction |
| 1006 | ----------------------------- |
| 1007 | |
| 1008 | If you are using tools that expect your resources to be "real" files, or your |
| 1009 | project includes non-extension native libraries or other files that your C |
| 1010 | extensions expect to be able to access, you may need to list those files in |
| 1011 | the ``eager_resources`` argument to ``setup()``, so that the files will be |
| 1012 | extracted together, whenever a C extension in the project is imported. |
| 1013 | |
| 1014 | This is especially important if your project includes shared libraries *other* |
| 1015 | than distutils-built C extensions, and those shared libraries use file |
| 1016 | extensions other than ``.dll``, ``.so``, or ``.dylib``, which are the |
| 1017 | extensions that setuptools 0.6a8 and higher automatically detects as shared |
| 1018 | libraries and adds to the ``native_libs.txt`` file for you. Any shared |
| 1019 | libraries whose names do not end with one of those extensions should be listed |
| 1020 | as ``eager_resources``, because they need to be present in the filesystem when |
| 1021 | he C extensions that link to them are used. |
| 1022 | |
| 1023 | The ``pkg_resources`` runtime for compressed packages will automatically |
| 1024 | extract *all* C extensions and ``eager_resources`` at the same time, whenever |
| 1025 | *any* C extension or eager resource is requested via the ``resource_filename()`` |
| 1026 | API. (C extensions are imported using ``resource_filename()`` internally.) |
| 1027 | This ensures that C extensions will see all of the "real" files that they |
| 1028 | expect to see. |
| 1029 | |
| 1030 | Note also that you can list directory resource names in ``eager_resources`` as |
| 1031 | well, in which case the directory's contents (including subdirectories) will be |
| 1032 | extracted whenever any C extension or eager resource is requested. |
| 1033 | |
| 1034 | Please note that if you're not sure whether you need to use this argument, you |
| 1035 | don't! It's really intended to support projects with lots of non-Python |
| 1036 | dependencies and as a last resort for crufty projects that can't otherwise |
| 1037 | handle being compressed. If your package is pure Python, Python plus data |
| 1038 | files, or Python plus C, you really don't need this. You've got to be using |
| 1039 | either C or an external program that needs "real" files in your project before |
| 1040 | there's any possibility of ``eager_resources`` being relevant to your project. |
| 1041 | |
| 1042 | |
| 1043 | Extensible Applications and Frameworks |
| 1044 | ====================================== |
| 1045 | |
| 1046 | |
| 1047 | .. _Entry Points: |
| 1048 | |
| 1049 | Dynamic Discovery of Services and Plugins |
| 1050 | ----------------------------------------- |
| 1051 | |
| 1052 | ``setuptools`` supports creating libraries that "plug in" to extensible |
| 1053 | applications and frameworks, by letting you register "entry points" in your |
| 1054 | project that can be imported by the application or framework. |
| 1055 | |
| 1056 | For example, suppose that a blogging tool wants to support plugins |
| 1057 | that provide translation for various file types to the blog's output format. |
| 1058 | The framework might define an "entry point group" called ``blogtool.parsers``, |
| 1059 | and then allow plugins to register entry points for the file extensions they |
| 1060 | support. |
| 1061 | |
| 1062 | This would allow people to create distributions that contain one or more |
| 1063 | parsers for different file types, and then the blogging tool would be able to |
| 1064 | find the parsers at runtime by looking up an entry point for the file |
| 1065 | extension (or mime type, or however it wants to). |
| 1066 | |
| 1067 | Note that if the blogging tool includes parsers for certain file formats, it |
| 1068 | can register these as entry points in its own setup script, which means it |
| 1069 | doesn't have to special-case its built-in formats. They can just be treated |
| 1070 | the same as any other plugin's entry points would be. |
| 1071 | |
| 1072 | If you're creating a project that plugs in to an existing application or |
| 1073 | framework, you'll need to know what entry points or entry point groups are |
| 1074 | defined by that application or framework. Then, you can register entry points |
| 1075 | in your setup script. Here are a few examples of ways you might register an |
| 1076 | ``.rst`` file parser entry point in the ``blogtool.parsers`` entry point group, |
| 1077 | for our hypothetical blogging tool:: |
| 1078 | |
| 1079 | setup( |
| 1080 | # ... |
| 1081 | entry_points={'blogtool.parsers': '.rst = some_module:SomeClass'} |
| 1082 | ) |
| 1083 | |
| 1084 | setup( |
| 1085 | # ... |
| 1086 | entry_points={'blogtool.parsers': ['.rst = some_module:a_func']} |
| 1087 | ) |
| 1088 | |
| 1089 | setup( |
| 1090 | # ... |
| 1091 | entry_points=""" |
| 1092 | [blogtool.parsers] |
| 1093 | .rst = some.nested.module:SomeClass.some_classmethod [reST] |
| 1094 | """, |
| 1095 | extras_require=dict(reST="Docutils>=0.3.5") |
| 1096 | ) |
| 1097 | |
| 1098 | The ``entry_points`` argument to ``setup()`` accepts either a string with |
| 1099 | ``.ini``-style sections, or a dictionary mapping entry point group names to |
| 1100 | either strings or lists of strings containing entry point specifiers. An |
| 1101 | entry point specifier consists of a name and value, separated by an ``=`` |
| 1102 | sign. The value consists of a dotted module name, optionally followed by a |
| 1103 | ``:`` and a dotted identifier naming an object within the module. It can |
| 1104 | also include a bracketed list of "extras" that are required for the entry |
| 1105 | point to be used. When the invoking application or framework requests loading |
| 1106 | of an entry point, any requirements implied by the associated extras will be |
| 1107 | passed to ``pkg_resources.require()``, so that an appropriate error message |
| 1108 | can be displayed if the needed package(s) are missing. (Of course, the |
| 1109 | invoking app or framework can ignore such errors if it wants to make an entry |
| 1110 | point optional if a requirement isn't installed.) |
| 1111 | |
| 1112 | |
| 1113 | Defining Additional Metadata |
| 1114 | ---------------------------- |
| 1115 | |
| 1116 | Some extensible applications and frameworks may need to define their own kinds |
| 1117 | of metadata to include in eggs, which they can then access using the |
| 1118 | ``pkg_resources`` metadata APIs. Ordinarily, this is done by having plugin |
| 1119 | developers include additional files in their ``ProjectName.egg-info`` |
| 1120 | directory. However, since it can be tedious to create such files by hand, you |
| 1121 | may want to create a distutils extension that will create the necessary files |
| 1122 | from arguments to ``setup()``, in much the same way that ``setuptools`` does |
| 1123 | for many of the ``setup()`` arguments it adds. See the section below on |
| 1124 | `Creating distutils Extensions`_ for more details, especially the subsection on |
| 1125 | `Adding new EGG-INFO Files`_. |
| 1126 | |
| 1127 | |
| 1128 | "Development Mode" |
| 1129 | ================== |
| 1130 | |
| 1131 | Under normal circumstances, the ``distutils`` assume that you are going to |
| 1132 | build a distribution of your project, not use it in its "raw" or "unbuilt" |
| 1133 | form. If you were to use the ``distutils`` that way, you would have to rebuild |
| 1134 | and reinstall your project every time you made a change to it during |
| 1135 | development. |
| 1136 | |
| 1137 | Another problem that sometimes comes up with the ``distutils`` is that you may |
| 1138 | need to do development on two related projects at the same time. You may need |
| 1139 | to put both projects' packages in the same directory to run them, but need to |
| 1140 | keep them separate for revision control purposes. How can you do this? |
| 1141 | |
| 1142 | Setuptools allows you to deploy your projects for use in a common directory or |
| 1143 | staging area, but without copying any files. Thus, you can edit each project's |
| 1144 | code in its checkout directory, and only need to run build commands when you |
| 1145 | change a project's C extensions or similarly compiled files. You can even |
| 1146 | deploy a project into another project's checkout directory, if that's your |
| 1147 | preferred way of working (as opposed to using a common independent staging area |
| 1148 | or the site-packages directory). |
| 1149 | |
| 1150 | To do this, use the ``setup.py develop`` command. It works very similarly to |
| 1151 | ``setup.py install`` or the EasyInstall tool, except that it doesn't actually |
| 1152 | install anything. Instead, it creates a special ``.egg-link`` file in the |
| 1153 | deployment directory, that links to your project's source code. And, if your |
| 1154 | deployment directory is Python's ``site-packages`` directory, it will also |
| 1155 | update the ``easy-install.pth`` file to include your project's source code, |
| 1156 | thereby making it available on ``sys.path`` for all programs using that Python |
| 1157 | installation. |
| 1158 | |
| 1159 | If you have enabled the ``use_2to3`` flag, then of course the ``.egg-link`` |
| 1160 | will not link directly to your source code when run under Python 3, since |
| 1161 | that source code would be made for Python 2 and not work under Python 3. |
| 1162 | Instead the ``setup.py develop`` will build Python 3 code under the ``build`` |
| 1163 | directory, and link there. This means that after doing code changes you will |
| 1164 | have to run ``setup.py build`` before these changes are picked up by your |
| 1165 | Python 3 installation. |
| 1166 | |
| 1167 | In addition, the ``develop`` command creates wrapper scripts in the target |
| 1168 | script directory that will run your in-development scripts after ensuring that |
| 1169 | all your ``install_requires`` packages are available on ``sys.path``. |
| 1170 | |
| 1171 | You can deploy the same project to multiple staging areas, e.g. if you have |
| 1172 | multiple projects on the same machine that are sharing the same project you're |
| 1173 | doing development work. |
| 1174 | |
| 1175 | When you're done with a given development task, you can remove the project |
| 1176 | source from a staging area using ``setup.py develop --uninstall``, specifying |
| 1177 | the desired staging area if it's not the default. |
| 1178 | |
| 1179 | There are several options to control the precise behavior of the ``develop`` |
| 1180 | command; see the section on the `develop`_ command below for more details. |
| 1181 | |
| 1182 | Note that you can also apply setuptools commands to non-setuptools projects, |
| 1183 | using commands like this:: |
| 1184 | |
| 1185 | python -c "import setuptools; execfile('setup.py')" develop |
| 1186 | |
| 1187 | That is, you can simply list the normal setup commands and options following |
| 1188 | the quoted part. |
| 1189 | |
| 1190 | |
| 1191 | Distributing a ``setuptools``-based project |
| 1192 | =========================================== |
| 1193 | |
| 1194 | Using ``setuptools``... Without bundling it! |
| 1195 | --------------------------------------------- |
| 1196 | |
| 1197 | .. warning:: **ez_setup** is deprecated in favor of PIP with **PEP-518** support. |
| 1198 | |
| 1199 | Your users might not have ``setuptools`` installed on their machines, or even |
| 1200 | if they do, it might not be the right version. Fixing this is easy; just |
| 1201 | download `ez_setup.py`_, and put it in the same directory as your ``setup.py`` |
| 1202 | script. (Be sure to add it to your revision control system, too.) Then add |
| 1203 | these two lines to the very top of your setup script, before the script imports |
| 1204 | anything from setuptools: |
| 1205 | |
| 1206 | .. code-block:: python |
| 1207 | |
| 1208 | import ez_setup |
| 1209 | ez_setup.use_setuptools() |
| 1210 | |
| 1211 | That's it. The ``ez_setup`` module will automatically download a matching |
| 1212 | version of ``setuptools`` from PyPI, if it isn't present on the target system. |
| 1213 | Whenever you install an updated version of setuptools, you should also update |
| 1214 | your projects' ``ez_setup.py`` files, so that a matching version gets installed |
| 1215 | on the target machine(s). |
| 1216 | |
| 1217 | By the way, setuptools supports the new PyPI "upload" command, so you can use |
| 1218 | ``setup.py sdist upload`` or ``setup.py bdist_egg upload`` to upload your |
| 1219 | source or egg distributions respectively. Your project's current version must |
| 1220 | be registered with PyPI first, of course; you can use ``setup.py register`` to |
| 1221 | do that. Or you can do it all in one step, e.g. ``setup.py register sdist |
| 1222 | bdist_egg upload`` will register the package, build source and egg |
| 1223 | distributions, and then upload them both to PyPI, where they'll be easily |
| 1224 | found by other projects that depend on them. |
| 1225 | |
| 1226 | (By the way, if you need to distribute a specific version of ``setuptools``, |
| 1227 | you can specify the exact version and base download URL as parameters to the |
| 1228 | ``use_setuptools()`` function. See the function's docstring for details.) |
| 1229 | |
| 1230 | |
| 1231 | What Your Users Should Know |
| 1232 | --------------------------- |
| 1233 | |
| 1234 | In general, a setuptools-based project looks just like any distutils-based |
| 1235 | project -- as long as your users have an internet connection and are installing |
| 1236 | to ``site-packages``, that is. But for some users, these conditions don't |
| 1237 | apply, and they may become frustrated if this is their first encounter with |
| 1238 | a setuptools-based project. To keep these users happy, you should review the |
| 1239 | following topics in your project's installation instructions, if they are |
| 1240 | relevant to your project and your target audience isn't already familiar with |
| 1241 | setuptools and ``easy_install``. |
| 1242 | |
| 1243 | Network Access |
| 1244 | If your project is using ``ez_setup``, you should inform users of the |
| 1245 | need to either have network access, or to preinstall the correct version of |
| 1246 | setuptools using the `EasyInstall installation instructions`_. Those |
| 1247 | instructions also have tips for dealing with firewalls as well as how to |
| 1248 | manually download and install setuptools. |
| 1249 | |
| 1250 | Custom Installation Locations |
| 1251 | You should inform your users that if they are installing your project to |
| 1252 | somewhere other than the main ``site-packages`` directory, they should |
| 1253 | first install setuptools using the instructions for `Custom Installation |
| 1254 | Locations`_, before installing your project. |
| 1255 | |
| 1256 | Your Project's Dependencies |
| 1257 | If your project depends on other projects that may need to be downloaded |
| 1258 | from PyPI or elsewhere, you should list them in your installation |
| 1259 | instructions, or tell users how to find out what they are. While most |
| 1260 | users will not need this information, any users who don't have unrestricted |
| 1261 | internet access may have to find, download, and install the other projects |
| 1262 | manually. (Note, however, that they must still install those projects |
| 1263 | using ``easy_install``, or your project will not know they are installed, |
| 1264 | and your setup script will try to download them again.) |
| 1265 | |
| 1266 | If you want to be especially friendly to users with limited network access, |
| 1267 | you may wish to build eggs for your project and its dependencies, making |
| 1268 | them all available for download from your site, or at least create a page |
| 1269 | with links to all of the needed eggs. In this way, users with limited |
| 1270 | network access can manually download all the eggs to a single directory, |
| 1271 | then use the ``-f`` option of ``easy_install`` to specify the directory |
| 1272 | to find eggs in. Users who have full network access can just use ``-f`` |
| 1273 | with the URL of your download page, and ``easy_install`` will find all the |
| 1274 | needed eggs using your links directly. This is also useful when your |
| 1275 | target audience isn't able to compile packages (e.g. most Windows users) |
| 1276 | and your package or some of its dependencies include C code. |
| 1277 | |
| 1278 | Revision Control System Users and Co-Developers |
| 1279 | Users and co-developers who are tracking your in-development code using |
| 1280 | a revision control system should probably read this manual's sections |
| 1281 | regarding such development. Alternately, you may wish to create a |
| 1282 | quick-reference guide containing the tips from this manual that apply to |
| 1283 | your particular situation. For example, if you recommend that people use |
| 1284 | ``setup.py develop`` when tracking your in-development code, you should let |
| 1285 | them know that this needs to be run after every update or commit. |
| 1286 | |
| 1287 | Similarly, if you remove modules or data files from your project, you |
| 1288 | should remind them to run ``setup.py clean --all`` and delete any obsolete |
| 1289 | ``.pyc`` or ``.pyo``. (This tip applies to the distutils in general, not |
| 1290 | just setuptools, but not everybody knows about them; be kind to your users |
| 1291 | by spelling out your project's best practices rather than leaving them |
| 1292 | guessing.) |
| 1293 | |
| 1294 | Creating System Packages |
| 1295 | Some users want to manage all Python packages using a single package |
| 1296 | manager, and sometimes that package manager isn't ``easy_install``! |
| 1297 | Setuptools currently supports ``bdist_rpm``, ``bdist_wininst``, and |
| 1298 | ``bdist_dumb`` formats for system packaging. If a user has a locally- |
| 1299 | installed "bdist" packaging tool that internally uses the distutils |
| 1300 | ``install`` command, it should be able to work with ``setuptools``. Some |
| 1301 | examples of "bdist" formats that this should work with include the |
| 1302 | ``bdist_nsi`` and ``bdist_msi`` formats for Windows. |
| 1303 | |
| 1304 | However, packaging tools that build binary distributions by running |
| 1305 | ``setup.py install`` on the command line or as a subprocess will require |
| 1306 | modification to work with setuptools. They should use the |
| 1307 | ``--single-version-externally-managed`` option to the ``install`` command, |
| 1308 | combined with the standard ``--root`` or ``--record`` options. |
| 1309 | See the `install command`_ documentation below for more details. The |
| 1310 | ``bdist_deb`` command is an example of a command that currently requires |
| 1311 | this kind of patching to work with setuptools. |
| 1312 | |
| 1313 | If you or your users have a problem building a usable system package for |
| 1314 | your project, please report the problem via the mailing list so that |
| 1315 | either the "bdist" tool in question or setuptools can be modified to |
| 1316 | resolve the issue. |
| 1317 | |
| 1318 | |
| 1319 | Setting the ``zip_safe`` flag |
| 1320 | ----------------------------- |
| 1321 | |
| 1322 | For some use cases (such as bundling as part of a larger application), Python |
| 1323 | packages may be run directly from a zip file. |
| 1324 | Not all packages, however, are capable of running in compressed form, because |
| 1325 | they may expect to be able to access either source code or data files as |
| 1326 | normal operating system files. So, ``setuptools`` can install your project |
| 1327 | as a zipfile or a directory, and its default choice is determined by the |
| 1328 | project's ``zip_safe`` flag. |
| 1329 | |
| 1330 | You can pass a True or False value for the ``zip_safe`` argument to the |
| 1331 | ``setup()`` function, or you can omit it. If you omit it, the ``bdist_egg`` |
| 1332 | command will analyze your project's contents to see if it can detect any |
| 1333 | conditions that would prevent it from working in a zipfile. It will output |
| 1334 | notices to the console about any such conditions that it finds. |
| 1335 | |
| 1336 | Currently, this analysis is extremely conservative: it will consider the |
| 1337 | project unsafe if it contains any C extensions or datafiles whatsoever. This |
| 1338 | does *not* mean that the project can't or won't work as a zipfile! It just |
| 1339 | means that the ``bdist_egg`` authors aren't yet comfortable asserting that |
| 1340 | the project *will* work. If the project contains no C or data files, and does |
| 1341 | no ``__file__`` or ``__path__`` introspection or source code manipulation, then |
| 1342 | there is an extremely solid chance the project will work when installed as a |
| 1343 | zipfile. (And if the project uses ``pkg_resources`` for all its data file |
| 1344 | access, then C extensions and other data files shouldn't be a problem at all. |
| 1345 | See the `Accessing Data Files at Runtime`_ section above for more information.) |
| 1346 | |
| 1347 | However, if ``bdist_egg`` can't be *sure* that your package will work, but |
| 1348 | you've checked over all the warnings it issued, and you are either satisfied it |
| 1349 | *will* work (or if you want to try it for yourself), then you should set |
| 1350 | ``zip_safe`` to ``True`` in your ``setup()`` call. If it turns out that it |
| 1351 | doesn't work, you can always change it to ``False``, which will force |
| 1352 | ``setuptools`` to install your project as a directory rather than as a zipfile. |
| 1353 | |
| 1354 | Of course, the end-user can still override either decision, if they are using |
| 1355 | EasyInstall to install your package. And, if you want to override for testing |
| 1356 | purposes, you can just run ``setup.py easy_install --zip-ok .`` or ``setup.py |
| 1357 | easy_install --always-unzip .`` in your project directory. to install the |
| 1358 | package as a zipfile or directory, respectively. |
| 1359 | |
| 1360 | In the future, as we gain more experience with different packages and become |
| 1361 | more satisfied with the robustness of the ``pkg_resources`` runtime, the |
| 1362 | "zip safety" analysis may become less conservative. However, we strongly |
| 1363 | recommend that you determine for yourself whether your project functions |
| 1364 | correctly when installed as a zipfile, correct any problems if you can, and |
| 1365 | then make an explicit declaration of ``True`` or ``False`` for the ``zip_safe`` |
| 1366 | flag, so that it will not be necessary for ``bdist_egg`` or ``EasyInstall`` to |
| 1367 | try to guess whether your project can work as a zipfile. |
| 1368 | |
| 1369 | |
| 1370 | Namespace Packages |
| 1371 | ------------------ |
| 1372 | |
| 1373 | Sometimes, a large package is more useful if distributed as a collection of |
| 1374 | smaller eggs. However, Python does not normally allow the contents of a |
| 1375 | package to be retrieved from more than one location. "Namespace packages" |
| 1376 | are a solution for this problem. When you declare a package to be a namespace |
| 1377 | package, it means that the package has no meaningful contents in its |
| 1378 | ``__init__.py``, and that it is merely a container for modules and subpackages. |
| 1379 | |
| 1380 | The ``pkg_resources`` runtime will then automatically ensure that the contents |
| 1381 | of namespace packages that are spread over multiple eggs or directories are |
| 1382 | combined into a single "virtual" package. |
| 1383 | |
| 1384 | The ``namespace_packages`` argument to ``setup()`` lets you declare your |
| 1385 | project's namespace packages, so that they will be included in your project's |
| 1386 | metadata. The argument should list the namespace packages that the egg |
| 1387 | participates in. For example, the ZopeInterface project might do this:: |
| 1388 | |
| 1389 | setup( |
| 1390 | # ... |
| 1391 | namespace_packages=['zope'] |
| 1392 | ) |
| 1393 | |
| 1394 | because it contains a ``zope.interface`` package that lives in the ``zope`` |
| 1395 | namespace package. Similarly, a project for a standalone ``zope.publisher`` |
| 1396 | would also declare the ``zope`` namespace package. When these projects are |
| 1397 | installed and used, Python will see them both as part of a "virtual" ``zope`` |
| 1398 | package, even though they will be installed in different locations. |
| 1399 | |
| 1400 | Namespace packages don't have to be top-level packages. For example, Zope 3's |
| 1401 | ``zope.app`` package is a namespace package, and in the future PEAK's |
| 1402 | ``peak.util`` package will be too. |
| 1403 | |
| 1404 | Note, by the way, that your project's source tree must include the namespace |
| 1405 | packages' ``__init__.py`` files (and the ``__init__.py`` of any parent |
| 1406 | packages), in a normal Python package layout. These ``__init__.py`` files |
| 1407 | *must* contain the line:: |
| 1408 | |
| 1409 | __import__('pkg_resources').declare_namespace(__name__) |
| 1410 | |
| 1411 | This code ensures that the namespace package machinery is operating and that |
| 1412 | the current package is registered as a namespace package. |
| 1413 | |
| 1414 | You must NOT include any other code and data in a namespace package's |
| 1415 | ``__init__.py``. Even though it may appear to work during development, or when |
| 1416 | projects are installed as ``.egg`` files, it will not work when the projects |
| 1417 | are installed using "system" packaging tools -- in such cases the |
| 1418 | ``__init__.py`` files will not be installed, let alone executed. |
| 1419 | |
| 1420 | You must include the ``declare_namespace()`` line in the ``__init__.py`` of |
| 1421 | *every* project that has contents for the namespace package in question, in |
| 1422 | order to ensure that the namespace will be declared regardless of which |
| 1423 | project's copy of ``__init__.py`` is loaded first. If the first loaded |
| 1424 | ``__init__.py`` doesn't declare it, it will never *be* declared, because no |
| 1425 | other copies will ever be loaded! |
| 1426 | |
| 1427 | |
| 1428 | TRANSITIONAL NOTE |
| 1429 | ~~~~~~~~~~~~~~~~~ |
| 1430 | |
| 1431 | Setuptools automatically calls ``declare_namespace()`` for you at runtime, |
| 1432 | but future versions may *not*. This is because the automatic declaration |
| 1433 | feature has some negative side effects, such as needing to import all namespace |
| 1434 | packages during the initialization of the ``pkg_resources`` runtime, and also |
| 1435 | the need for ``pkg_resources`` to be explicitly imported before any namespace |
| 1436 | packages work at all. In some future releases, you'll be responsible |
| 1437 | for including your own declaration lines, and the automatic declaration feature |
| 1438 | will be dropped to get rid of the negative side effects. |
| 1439 | |
| 1440 | During the remainder of the current development cycle, therefore, setuptools |
| 1441 | will warn you about missing ``declare_namespace()`` calls in your |
| 1442 | ``__init__.py`` files, and you should correct these as soon as possible |
| 1443 | before the compatibility support is removed. |
| 1444 | Namespace packages without declaration lines will not work |
| 1445 | correctly once a user has upgraded to a later version, so it's important that |
| 1446 | you make this change now in order to avoid having your code break in the field. |
| 1447 | Our apologies for the inconvenience, and thank you for your patience. |
| 1448 | |
| 1449 | |
| 1450 | |
| 1451 | Tagging and "Daily Build" or "Snapshot" Releases |
| 1452 | ------------------------------------------------ |
| 1453 | |
| 1454 | When a set of related projects are under development, it may be important to |
| 1455 | track finer-grained version increments than you would normally use for e.g. |
| 1456 | "stable" releases. While stable releases might be measured in dotted numbers |
| 1457 | with alpha/beta/etc. status codes, development versions of a project often |
| 1458 | need to be tracked by revision or build number or even build date. This is |
| 1459 | especially true when projects in development need to refer to one another, and |
| 1460 | therefore may literally need an up-to-the-minute version of something! |
| 1461 | |
| 1462 | To support these scenarios, ``setuptools`` allows you to "tag" your source and |
| 1463 | egg distributions by adding one or more of the following to the project's |
| 1464 | "official" version identifier: |
| 1465 | |
| 1466 | * A manually-specified pre-release tag, such as "build" or "dev", or a |
| 1467 | manually-specified post-release tag, such as a build or revision number |
| 1468 | (``--tag-build=STRING, -bSTRING``) |
| 1469 | |
| 1470 | * An 8-character representation of the build date (``--tag-date, -d``), as |
| 1471 | a postrelease tag |
| 1472 | |
| 1473 | You can add these tags by adding ``egg_info`` and the desired options to |
| 1474 | the command line ahead of the ``sdist`` or ``bdist`` commands that you want |
| 1475 | to generate a daily build or snapshot for. See the section below on the |
| 1476 | `egg_info`_ command for more details. |
| 1477 | |
| 1478 | (Also, before you release your project, be sure to see the section above on |
| 1479 | `Specifying Your Project's Version`_ for more information about how pre- and |
| 1480 | post-release tags affect how setuptools and EasyInstall interpret version |
| 1481 | numbers. This is important in order to make sure that dependency processing |
| 1482 | tools will know which versions of your project are newer than others.) |
| 1483 | |
| 1484 | Finally, if you are creating builds frequently, and either building them in a |
| 1485 | downloadable location or are copying them to a distribution server, you should |
| 1486 | probably also check out the `rotate`_ command, which lets you automatically |
| 1487 | delete all but the N most-recently-modified distributions matching a glob |
| 1488 | pattern. So, you can use a command line like:: |
| 1489 | |
| 1490 | setup.py egg_info -rbDEV bdist_egg rotate -m.egg -k3 |
| 1491 | |
| 1492 | to build an egg whose version info includes 'DEV-rNNNN' (where NNNN is the |
| 1493 | most recent Subversion revision that affected the source tree), and then |
| 1494 | delete any egg files from the distribution directory except for the three |
| 1495 | that were built most recently. |
| 1496 | |
| 1497 | If you have to manage automated builds for multiple packages, each with |
| 1498 | different tagging and rotation policies, you may also want to check out the |
| 1499 | `alias`_ command, which would let each package define an alias like ``daily`` |
| 1500 | that would perform the necessary tag, build, and rotate commands. Then, a |
| 1501 | simpler script or cron job could just run ``setup.py daily`` in each project |
| 1502 | directory. (And, you could also define sitewide or per-user default versions |
| 1503 | of the ``daily`` alias, so that projects that didn't define their own would |
| 1504 | use the appropriate defaults.) |
| 1505 | |
| 1506 | |
| 1507 | Generating Source Distributions |
| 1508 | ------------------------------- |
| 1509 | |
| 1510 | ``setuptools`` enhances the distutils' default algorithm for source file |
| 1511 | selection with pluggable endpoints for looking up files to include. If you are |
| 1512 | using a revision control system, and your source distributions only need to |
| 1513 | include files that you're tracking in revision control, use a corresponding |
| 1514 | plugin instead of writing a ``MANIFEST.in`` file. See the section below on |
| 1515 | `Adding Support for Revision Control Systems`_ for information on plugins. |
| 1516 | |
| 1517 | If you need to include automatically generated files, or files that are kept in |
| 1518 | an unsupported revision control system, you'll need to create a ``MANIFEST.in`` |
| 1519 | file to specify any files that the default file location algorithm doesn't |
| 1520 | catch. See the distutils documentation for more information on the format of |
| 1521 | the ``MANIFEST.in`` file. |
| 1522 | |
| 1523 | But, be sure to ignore any part of the distutils documentation that deals with |
| 1524 | ``MANIFEST`` or how it's generated from ``MANIFEST.in``; setuptools shields you |
| 1525 | from these issues and doesn't work the same way in any case. Unlike the |
| 1526 | distutils, setuptools regenerates the source distribution manifest file |
| 1527 | every time you build a source distribution, and it builds it inside the |
| 1528 | project's ``.egg-info`` directory, out of the way of your main project |
| 1529 | directory. You therefore need not worry about whether it is up-to-date or not. |
| 1530 | |
| 1531 | Indeed, because setuptools' approach to determining the contents of a source |
| 1532 | distribution is so much simpler, its ``sdist`` command omits nearly all of |
| 1533 | the options that the distutils' more complex ``sdist`` process requires. For |
| 1534 | all practical purposes, you'll probably use only the ``--formats`` option, if |
| 1535 | you use any option at all. |
| 1536 | |
| 1537 | |
| 1538 | Making your package available for EasyInstall |
| 1539 | --------------------------------------------- |
| 1540 | |
| 1541 | If you use the ``register`` command (``setup.py register``) to register your |
| 1542 | package with PyPI, that's most of the battle right there. (See the |
| 1543 | `docs for the register command`_ for more details.) |
| 1544 | |
| 1545 | .. _docs for the register command: http://docs.python.org/dist/package-index.html |
| 1546 | |
| 1547 | If you also use the `upload`_ command to upload actual distributions of your |
| 1548 | package, that's even better, because EasyInstall will be able to find and |
| 1549 | download them directly from your project's PyPI page. |
| 1550 | |
| 1551 | However, there may be reasons why you don't want to upload distributions to |
| 1552 | PyPI, and just want your existing distributions (or perhaps a Subversion |
| 1553 | checkout) to be used instead. |
| 1554 | |
| 1555 | So here's what you need to do before running the ``register`` command. There |
| 1556 | are three ``setup()`` arguments that affect EasyInstall: |
| 1557 | |
| 1558 | ``url`` and ``download_url`` |
| 1559 | These become links on your project's PyPI page. EasyInstall will examine |
| 1560 | them to see if they link to a package ("primary links"), or whether they are |
| 1561 | HTML pages. If they're HTML pages, EasyInstall scans all HREF's on the |
| 1562 | page for primary links |
| 1563 | |
| 1564 | ``long_description`` |
| 1565 | EasyInstall will check any URLs contained in this argument to see if they |
| 1566 | are primary links. |
| 1567 | |
| 1568 | A URL is considered a "primary link" if it is a link to a .tar.gz, .tgz, .zip, |
| 1569 | .egg, .egg.zip, .tar.bz2, or .exe file, or if it has an ``#egg=project`` or |
| 1570 | ``#egg=project-version`` fragment identifier attached to it. EasyInstall |
| 1571 | attempts to determine a project name and optional version number from the text |
| 1572 | of a primary link *without* downloading it. When it has found all the primary |
| 1573 | links, EasyInstall will select the best match based on requested version, |
| 1574 | platform compatibility, and other criteria. |
| 1575 | |
| 1576 | So, if your ``url`` or ``download_url`` point either directly to a downloadable |
| 1577 | source distribution, or to HTML page(s) that have direct links to such, then |
| 1578 | EasyInstall will be able to locate downloads automatically. If you want to |
| 1579 | make Subversion checkouts available, then you should create links with either |
| 1580 | ``#egg=project`` or ``#egg=project-version`` added to the URL. You should |
| 1581 | replace ``project`` and ``version`` with the values they would have in an egg |
| 1582 | filename. (Be sure to actually generate an egg and then use the initial part |
| 1583 | of the filename, rather than trying to guess what the escaped form of the |
| 1584 | project name and version number will be.) |
| 1585 | |
| 1586 | Note that Subversion checkout links are of lower precedence than other kinds |
| 1587 | of distributions, so EasyInstall will not select a Subversion checkout for |
| 1588 | downloading unless it has a version included in the ``#egg=`` suffix, and |
| 1589 | it's a higher version than EasyInstall has seen in any other links for your |
| 1590 | project. |
| 1591 | |
| 1592 | As a result, it's a common practice to use mark checkout URLs with a version of |
| 1593 | "dev" (i.e., ``#egg=projectname-dev``), so that users can do something like |
| 1594 | this:: |
| 1595 | |
| 1596 | easy_install --editable projectname==dev |
| 1597 | |
| 1598 | in order to check out the in-development version of ``projectname``. |
| 1599 | |
| 1600 | |
| 1601 | Making "Official" (Non-Snapshot) Releases |
| 1602 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 1603 | |
| 1604 | When you make an official release, creating source or binary distributions, |
| 1605 | you will need to override the tag settings from ``setup.cfg``, so that you |
| 1606 | don't end up registering versions like ``foobar-0.7a1.dev-r34832``. This is |
| 1607 | easy to do if you are developing on the trunk and using tags or branches for |
| 1608 | your releases - just make the change to ``setup.cfg`` after branching or |
| 1609 | tagging the release, so the trunk will still produce development snapshots. |
| 1610 | |
| 1611 | Alternately, if you are not branching for releases, you can override the |
| 1612 | default version options on the command line, using something like:: |
| 1613 | |
| 1614 | python setup.py egg_info -Db "" sdist bdist_egg register upload |
| 1615 | |
| 1616 | The first part of this command (``egg_info -Db ""``) will override the |
| 1617 | configured tag information, before creating source and binary eggs, registering |
| 1618 | the project with PyPI, and uploading the files. Thus, these commands will use |
| 1619 | the plain version from your ``setup.py``, without adding the build designation |
| 1620 | string. |
| 1621 | |
| 1622 | Of course, if you will be doing this a lot, you may wish to create a personal |
| 1623 | alias for this operation, e.g.:: |
| 1624 | |
| 1625 | python setup.py alias -u release egg_info -Db "" |
| 1626 | |
| 1627 | You can then use it like this:: |
| 1628 | |
| 1629 | python setup.py release sdist bdist_egg register upload |
| 1630 | |
| 1631 | Or of course you can create more elaborate aliases that do all of the above. |
| 1632 | See the sections below on the `egg_info`_ and `alias`_ commands for more ideas. |
| 1633 | |
| 1634 | |
| 1635 | |
| 1636 | Distributing Extensions compiled with Pyrex |
| 1637 | ------------------------------------------- |
| 1638 | |
| 1639 | ``setuptools`` includes transparent support for building Pyrex extensions, as |
| 1640 | long as you define your extensions using ``setuptools.Extension``, *not* |
| 1641 | ``distutils.Extension``. You must also not import anything from Pyrex in |
| 1642 | your setup script. |
| 1643 | |
| 1644 | If you follow these rules, you can safely list ``.pyx`` files as the source |
| 1645 | of your ``Extension`` objects in the setup script. ``setuptools`` will detect |
| 1646 | at build time whether Pyrex is installed or not. If it is, then ``setuptools`` |
| 1647 | will use it. If not, then ``setuptools`` will silently change the |
| 1648 | ``Extension`` objects to refer to the ``.c`` counterparts of the ``.pyx`` |
| 1649 | files, so that the normal distutils C compilation process will occur. |
| 1650 | |
| 1651 | Of course, for this to work, your source distributions must include the C |
| 1652 | code generated by Pyrex, as well as your original ``.pyx`` files. This means |
| 1653 | that you will probably want to include current ``.c`` files in your revision |
| 1654 | control system, rebuilding them whenever you check changes in for the ``.pyx`` |
| 1655 | source files. This will ensure that people tracking your project in a revision |
| 1656 | control system will be able to build it even if they don't have Pyrex |
| 1657 | installed, and that your source releases will be similarly usable with or |
| 1658 | without Pyrex. |
| 1659 | |
| 1660 | |
| 1661 | ----------------- |
| 1662 | Command Reference |
| 1663 | ----------------- |
| 1664 | |
| 1665 | .. _alias: |
| 1666 | |
| 1667 | ``alias`` - Define shortcuts for commonly used commands |
| 1668 | ======================================================= |
| 1669 | |
| 1670 | Sometimes, you need to use the same commands over and over, but you can't |
| 1671 | necessarily set them as defaults. For example, if you produce both development |
| 1672 | snapshot releases and "stable" releases of a project, you may want to put |
| 1673 | the distributions in different places, or use different ``egg_info`` tagging |
| 1674 | options, etc. In these cases, it doesn't make sense to set the options in |
| 1675 | a distutils configuration file, because the values of the options changed based |
| 1676 | on what you're trying to do. |
| 1677 | |
| 1678 | Setuptools therefore allows you to define "aliases" - shortcut names for |
| 1679 | an arbitrary string of commands and options, using ``setup.py alias aliasname |
| 1680 | expansion``, where aliasname is the name of the new alias, and the remainder of |
| 1681 | the command line supplies its expansion. For example, this command defines |
| 1682 | a sitewide alias called "daily", that sets various ``egg_info`` tagging |
| 1683 | options:: |
| 1684 | |
| 1685 | setup.py alias --global-config daily egg_info --tag-build=development |
| 1686 | |
| 1687 | Once the alias is defined, it can then be used with other setup commands, |
| 1688 | e.g.:: |
| 1689 | |
| 1690 | setup.py daily bdist_egg # generate a daily-build .egg file |
| 1691 | setup.py daily sdist # generate a daily-build source distro |
| 1692 | setup.py daily sdist bdist_egg # generate both |
| 1693 | |
| 1694 | The above commands are interpreted as if the word ``daily`` were replaced with |
| 1695 | ``egg_info --tag-build=development``. |
| 1696 | |
| 1697 | Note that setuptools will expand each alias *at most once* in a given command |
| 1698 | line. This serves two purposes. First, if you accidentally create an alias |
| 1699 | loop, it will have no effect; you'll instead get an error message about an |
| 1700 | unknown command. Second, it allows you to define an alias for a command, that |
| 1701 | uses that command. For example, this (project-local) alias:: |
| 1702 | |
| 1703 | setup.py alias bdist_egg bdist_egg rotate -k1 -m.egg |
| 1704 | |
| 1705 | redefines the ``bdist_egg`` command so that it always runs the ``rotate`` |
| 1706 | command afterwards to delete all but the newest egg file. It doesn't loop |
| 1707 | indefinitely on ``bdist_egg`` because the alias is only expanded once when |
| 1708 | used. |
| 1709 | |
| 1710 | You can remove a defined alias with the ``--remove`` (or ``-r``) option, e.g.:: |
| 1711 | |
| 1712 | setup.py alias --global-config --remove daily |
| 1713 | |
| 1714 | would delete the "daily" alias we defined above. |
| 1715 | |
| 1716 | Aliases can be defined on a project-specific, per-user, or sitewide basis. The |
| 1717 | default is to define or remove a project-specific alias, but you can use any of |
| 1718 | the `configuration file options`_ (listed under the `saveopts`_ command, below) |
| 1719 | to determine which distutils configuration file an aliases will be added to |
| 1720 | (or removed from). |
| 1721 | |
| 1722 | Note that if you omit the "expansion" argument to the ``alias`` command, |
| 1723 | you'll get output showing that alias' current definition (and what |
| 1724 | configuration file it's defined in). If you omit the alias name as well, |
| 1725 | you'll get a listing of all current aliases along with their configuration |
| 1726 | file locations. |
| 1727 | |
| 1728 | |
| 1729 | ``bdist_egg`` - Create a Python Egg for the project |
| 1730 | =================================================== |
| 1731 | |
| 1732 | This command generates a Python Egg (``.egg`` file) for the project. Python |
| 1733 | Eggs are the preferred binary distribution format for EasyInstall, because they |
| 1734 | are cross-platform (for "pure" packages), directly importable, and contain |
| 1735 | project metadata including scripts and information about the project's |
| 1736 | dependencies. They can be simply downloaded and added to ``sys.path`` |
| 1737 | directly, or they can be placed in a directory on ``sys.path`` and then |
| 1738 | automatically discovered by the egg runtime system. |
| 1739 | |
| 1740 | This command runs the `egg_info`_ command (if it hasn't already run) to update |
| 1741 | the project's metadata (``.egg-info``) directory. If you have added any extra |
| 1742 | metadata files to the ``.egg-info`` directory, those files will be included in |
| 1743 | the new egg file's metadata directory, for use by the egg runtime system or by |
| 1744 | any applications or frameworks that use that metadata. |
| 1745 | |
| 1746 | You won't usually need to specify any special options for this command; just |
| 1747 | use ``bdist_egg`` and you're done. But there are a few options that may |
| 1748 | be occasionally useful: |
| 1749 | |
| 1750 | ``--dist-dir=DIR, -d DIR`` |
| 1751 | Set the directory where the ``.egg`` file will be placed. If you don't |
| 1752 | supply this, then the ``--dist-dir`` setting of the ``bdist`` command |
| 1753 | will be used, which is usually a directory named ``dist`` in the project |
| 1754 | directory. |
| 1755 | |
| 1756 | ``--plat-name=PLATFORM, -p PLATFORM`` |
| 1757 | Set the platform name string that will be embedded in the egg's filename |
| 1758 | (assuming the egg contains C extensions). This can be used to override |
| 1759 | the distutils default platform name with something more meaningful. Keep |
| 1760 | in mind, however, that the egg runtime system expects to see eggs with |
| 1761 | distutils platform names, so it may ignore or reject eggs with non-standard |
| 1762 | platform names. Similarly, the EasyInstall program may ignore them when |
| 1763 | searching web pages for download links. However, if you are |
| 1764 | cross-compiling or doing some other unusual things, you might find a use |
| 1765 | for this option. |
| 1766 | |
| 1767 | ``--exclude-source-files`` |
| 1768 | Don't include any modules' ``.py`` files in the egg, just compiled Python, |
| 1769 | C, and data files. (Note that this doesn't affect any ``.py`` files in the |
| 1770 | EGG-INFO directory or its subdirectories, since for example there may be |
| 1771 | scripts with a ``.py`` extension which must still be retained.) We don't |
| 1772 | recommend that you use this option except for packages that are being |
| 1773 | bundled for proprietary end-user applications, or for "embedded" scenarios |
| 1774 | where space is at an absolute premium. On the other hand, if your package |
| 1775 | is going to be installed and used in compressed form, you might as well |
| 1776 | exclude the source because Python's ``traceback`` module doesn't currently |
| 1777 | understand how to display zipped source code anyway, or how to deal with |
| 1778 | files that are in a different place from where their code was compiled. |
| 1779 | |
| 1780 | There are also some options you will probably never need, but which are there |
| 1781 | because they were copied from similar ``bdist`` commands used as an example for |
| 1782 | creating this one. They may be useful for testing and debugging, however, |
| 1783 | which is why we kept them: |
| 1784 | |
| 1785 | ``--keep-temp, -k`` |
| 1786 | Keep the contents of the ``--bdist-dir`` tree around after creating the |
| 1787 | ``.egg`` file. |
| 1788 | |
| 1789 | ``--bdist-dir=DIR, -b DIR`` |
| 1790 | Set the temporary directory for creating the distribution. The entire |
| 1791 | contents of this directory are zipped to create the ``.egg`` file, after |
| 1792 | running various installation commands to copy the package's modules, data, |
| 1793 | and extensions here. |
| 1794 | |
| 1795 | ``--skip-build`` |
| 1796 | Skip doing any "build" commands; just go straight to the |
| 1797 | install-and-compress phases. |
| 1798 | |
| 1799 | |
| 1800 | .. _develop: |
| 1801 | |
| 1802 | ``develop`` - Deploy the project source in "Development Mode" |
| 1803 | ============================================================= |
| 1804 | |
| 1805 | This command allows you to deploy your project's source for use in one or more |
| 1806 | "staging areas" where it will be available for importing. This deployment is |
| 1807 | done in such a way that changes to the project source are immediately available |
| 1808 | in the staging area(s), without needing to run a build or install step after |
| 1809 | each change. |
| 1810 | |
| 1811 | The ``develop`` command works by creating an ``.egg-link`` file (named for the |
| 1812 | project) in the given staging area. If the staging area is Python's |
| 1813 | ``site-packages`` directory, it also updates an ``easy-install.pth`` file so |
| 1814 | that the project is on ``sys.path`` by default for all programs run using that |
| 1815 | Python installation. |
| 1816 | |
| 1817 | The ``develop`` command also installs wrapper scripts in the staging area (or |
| 1818 | a separate directory, as specified) that will ensure the project's dependencies |
| 1819 | are available on ``sys.path`` before running the project's source scripts. |
| 1820 | And, it ensures that any missing project dependencies are available in the |
| 1821 | staging area, by downloading and installing them if necessary. |
| 1822 | |
| 1823 | Last, but not least, the ``develop`` command invokes the ``build_ext -i`` |
| 1824 | command to ensure any C extensions in the project have been built and are |
| 1825 | up-to-date, and the ``egg_info`` command to ensure the project's metadata is |
| 1826 | updated (so that the runtime and wrappers know what the project's dependencies |
| 1827 | are). If you make any changes to the project's setup script or C extensions, |
| 1828 | you should rerun the ``develop`` command against all relevant staging areas to |
| 1829 | keep the project's scripts, metadata and extensions up-to-date. Most other |
| 1830 | kinds of changes to your project should not require any build operations or |
| 1831 | rerunning ``develop``, but keep in mind that even minor changes to the setup |
| 1832 | script (e.g. changing an entry point definition) require you to re-run the |
| 1833 | ``develop`` or ``test`` commands to keep the distribution updated. |
| 1834 | |
| 1835 | Here are some of the options that the ``develop`` command accepts. Note that |
| 1836 | they affect the project's dependencies as well as the project itself, so if you |
| 1837 | have dependencies that need to be installed and you use ``--exclude-scripts`` |
| 1838 | (for example), the dependencies' scripts will not be installed either! For |
| 1839 | this reason, you may want to use EasyInstall to install the project's |
| 1840 | dependencies before using the ``develop`` command, if you need finer control |
| 1841 | over the installation options for dependencies. |
| 1842 | |
| 1843 | ``--uninstall, -u`` |
| 1844 | Un-deploy the current project. You may use the ``--install-dir`` or ``-d`` |
| 1845 | option to designate the staging area. The created ``.egg-link`` file will |
| 1846 | be removed, if present and it is still pointing to the project directory. |
| 1847 | The project directory will be removed from ``easy-install.pth`` if the |
| 1848 | staging area is Python's ``site-packages`` directory. |
| 1849 | |
| 1850 | Note that this option currently does *not* uninstall script wrappers! You |
| 1851 | must uninstall them yourself, or overwrite them by using EasyInstall to |
| 1852 | activate a different version of the package. You can also avoid installing |
| 1853 | script wrappers in the first place, if you use the ``--exclude-scripts`` |
| 1854 | (aka ``-x``) option when you run ``develop`` to deploy the project. |
| 1855 | |
| 1856 | ``--multi-version, -m`` |
| 1857 | "Multi-version" mode. Specifying this option prevents ``develop`` from |
| 1858 | adding an ``easy-install.pth`` entry for the project(s) being deployed, and |
| 1859 | if an entry for any version of a project already exists, the entry will be |
| 1860 | removed upon successful deployment. In multi-version mode, no specific |
| 1861 | version of the package is available for importing, unless you use |
| 1862 | ``pkg_resources.require()`` to put it on ``sys.path``, or you are running |
| 1863 | a wrapper script generated by ``setuptools`` or EasyInstall. (In which |
| 1864 | case the wrapper script calls ``require()`` for you.) |
| 1865 | |
| 1866 | Note that if you install to a directory other than ``site-packages``, |
| 1867 | this option is automatically in effect, because ``.pth`` files can only be |
| 1868 | used in ``site-packages`` (at least in Python 2.3 and 2.4). So, if you use |
| 1869 | the ``--install-dir`` or ``-d`` option (or they are set via configuration |
| 1870 | file(s)) your project and its dependencies will be deployed in multi- |
| 1871 | version mode. |
| 1872 | |
| 1873 | ``--install-dir=DIR, -d DIR`` |
| 1874 | Set the installation directory (staging area). If this option is not |
| 1875 | directly specified on the command line or in a distutils configuration |
| 1876 | file, the distutils default installation location is used. Normally, this |
| 1877 | will be the ``site-packages`` directory, but if you are using distutils |
| 1878 | configuration files, setting things like ``prefix`` or ``install_lib``, |
| 1879 | then those settings are taken into account when computing the default |
| 1880 | staging area. |
| 1881 | |
| 1882 | ``--script-dir=DIR, -s DIR`` |
| 1883 | Set the script installation directory. If you don't supply this option |
| 1884 | (via the command line or a configuration file), but you *have* supplied |
| 1885 | an ``--install-dir`` (via command line or config file), then this option |
| 1886 | defaults to the same directory, so that the scripts will be able to find |
| 1887 | their associated package installation. Otherwise, this setting defaults |
| 1888 | to the location where the distutils would normally install scripts, taking |
| 1889 | any distutils configuration file settings into account. |
| 1890 | |
| 1891 | ``--exclude-scripts, -x`` |
| 1892 | Don't deploy script wrappers. This is useful if you don't want to disturb |
| 1893 | existing versions of the scripts in the staging area. |
| 1894 | |
| 1895 | ``--always-copy, -a`` |
| 1896 | Copy all needed distributions to the staging area, even if they |
| 1897 | are already present in another directory on ``sys.path``. By default, if |
| 1898 | a requirement can be met using a distribution that is already available in |
| 1899 | a directory on ``sys.path``, it will not be copied to the staging area. |
| 1900 | |
| 1901 | ``--egg-path=DIR`` |
| 1902 | Force the generated ``.egg-link`` file to use a specified relative path |
| 1903 | to the source directory. This can be useful in circumstances where your |
| 1904 | installation directory is being shared by code running under multiple |
| 1905 | platforms (e.g. Mac and Windows) which have different absolute locations |
| 1906 | for the code under development, but the same *relative* locations with |
| 1907 | respect to the installation directory. If you use this option when |
| 1908 | installing, you must supply the same relative path when uninstalling. |
| 1909 | |
| 1910 | In addition to the above options, the ``develop`` command also accepts all of |
| 1911 | the same options accepted by ``easy_install``. If you've configured any |
| 1912 | ``easy_install`` settings in your ``setup.cfg`` (or other distutils config |
| 1913 | files), the ``develop`` command will use them as defaults, unless you override |
| 1914 | them in a ``[develop]`` section or on the command line. |
| 1915 | |
| 1916 | |
| 1917 | ``easy_install`` - Find and install packages |
| 1918 | ============================================ |
| 1919 | |
| 1920 | This command runs the `EasyInstall tool |
| 1921 | <easy_install.html>`_ for you. It is exactly |
| 1922 | equivalent to running the ``easy_install`` command. All command line arguments |
| 1923 | following this command are consumed and not processed further by the distutils, |
| 1924 | so this must be the last command listed on the command line. Please see |
| 1925 | the EasyInstall documentation for the options reference and usage examples. |
| 1926 | Normally, there is no reason to use this command via the command line, as you |
| 1927 | can just use ``easy_install`` directly. It's only listed here so that you know |
| 1928 | it's a distutils command, which means that you can: |
| 1929 | |
| 1930 | * create command aliases that use it, |
| 1931 | * create distutils extensions that invoke it as a subcommand, and |
| 1932 | * configure options for it in your ``setup.cfg`` or other distutils config |
| 1933 | files. |
| 1934 | |
| 1935 | |
| 1936 | .. _egg_info: |
| 1937 | |
| 1938 | ``egg_info`` - Create egg metadata and set build tags |
| 1939 | ===================================================== |
| 1940 | |
| 1941 | This command performs two operations: it updates a project's ``.egg-info`` |
| 1942 | metadata directory (used by the ``bdist_egg``, ``develop``, and ``test`` |
| 1943 | commands), and it allows you to temporarily change a project's version string, |
| 1944 | to support "daily builds" or "snapshot" releases. It is run automatically by |
| 1945 | the ``sdist``, ``bdist_egg``, ``develop``, ``register``, and ``test`` commands |
| 1946 | in order to update the project's metadata, but you can also specify it |
| 1947 | explicitly in order to temporarily change the project's version string while |
| 1948 | executing other commands. (It also generates the``.egg-info/SOURCES.txt`` |
| 1949 | manifest file, which is used when you are building source distributions.) |
| 1950 | |
| 1951 | In addition to writing the core egg metadata defined by ``setuptools`` and |
| 1952 | required by ``pkg_resources``, this command can be extended to write other |
| 1953 | metadata files as well, by defining entry points in the ``egg_info.writers`` |
| 1954 | group. See the section on `Adding new EGG-INFO Files`_ below for more details. |
| 1955 | Note that using additional metadata writers may require you to include a |
| 1956 | ``setup_requires`` argument to ``setup()`` in order to ensure that the desired |
| 1957 | writers are available on ``sys.path``. |
| 1958 | |
| 1959 | |
| 1960 | Release Tagging Options |
| 1961 | ----------------------- |
| 1962 | |
| 1963 | The following options can be used to modify the project's version string for |
| 1964 | all remaining commands on the setup command line. The options are processed |
| 1965 | in the order shown, so if you use more than one, the requested tags will be |
| 1966 | added in the following order: |
| 1967 | |
| 1968 | ``--tag-build=NAME, -b NAME`` |
| 1969 | Append NAME to the project's version string. Due to the way setuptools |
| 1970 | processes "pre-release" version suffixes beginning with the letters "a" |
| 1971 | through "e" (like "alpha", "beta", and "candidate"), you will usually want |
| 1972 | to use a tag like ".build" or ".dev", as this will cause the version number |
| 1973 | to be considered *lower* than the project's default version. (If you |
| 1974 | want to make the version number *higher* than the default version, you can |
| 1975 | always leave off --tag-build and then use one or both of the following |
| 1976 | options.) |
| 1977 | |
| 1978 | If you have a default build tag set in your ``setup.cfg``, you can suppress |
| 1979 | it on the command line using ``-b ""`` or ``--tag-build=""`` as an argument |
| 1980 | to the ``egg_info`` command. |
| 1981 | |
| 1982 | ``--tag-date, -d`` |
| 1983 | Add a date stamp of the form "-YYYYMMDD" (e.g. "-20050528") to the |
| 1984 | project's version number. |
| 1985 | |
| 1986 | ``--no-date, -D`` |
| 1987 | Don't include a date stamp in the version number. This option is included |
| 1988 | so you can override a default setting in ``setup.cfg``. |
| 1989 | |
| 1990 | |
| 1991 | (Note: Because these options modify the version number used for source and |
| 1992 | binary distributions of your project, you should first make sure that you know |
| 1993 | how the resulting version numbers will be interpreted by automated tools |
| 1994 | like EasyInstall. See the section above on `Specifying Your Project's |
| 1995 | Version`_ for an explanation of pre- and post-release tags, as well as tips on |
| 1996 | how to choose and verify a versioning scheme for your your project.) |
| 1997 | |
| 1998 | For advanced uses, there is one other option that can be set, to change the |
| 1999 | location of the project's ``.egg-info`` directory. Commands that need to find |
| 2000 | the project's source directory or metadata should get it from this setting: |
| 2001 | |
| 2002 | |
| 2003 | Other ``egg_info`` Options |
| 2004 | -------------------------- |
| 2005 | |
| 2006 | ``--egg-base=SOURCEDIR, -e SOURCEDIR`` |
| 2007 | Specify the directory that should contain the .egg-info directory. This |
| 2008 | should normally be the root of your project's source tree (which is not |
| 2009 | necessarily the same as your project directory; some projects use a ``src`` |
| 2010 | or ``lib`` subdirectory as the source root). You should not normally need |
| 2011 | to specify this directory, as it is normally determined from the |
| 2012 | ``package_dir`` argument to the ``setup()`` function, if any. If there is |
| 2013 | no ``package_dir`` set, this option defaults to the current directory. |
| 2014 | |
| 2015 | |
| 2016 | ``egg_info`` Examples |
| 2017 | --------------------- |
| 2018 | |
| 2019 | Creating a dated "nightly build" snapshot egg:: |
| 2020 | |
| 2021 | python setup.py egg_info --tag-date --tag-build=DEV bdist_egg |
| 2022 | |
| 2023 | Creating and uploading a release with no version tags, even if some default |
| 2024 | tags are specified in ``setup.cfg``:: |
| 2025 | |
| 2026 | python setup.py egg_info -RDb "" sdist bdist_egg register upload |
| 2027 | |
| 2028 | (Notice that ``egg_info`` must always appear on the command line *before* any |
| 2029 | commands that you want the version changes to apply to.) |
| 2030 | |
| 2031 | |
| 2032 | .. _install command: |
| 2033 | |
| 2034 | ``install`` - Run ``easy_install`` or old-style installation |
| 2035 | ============================================================ |
| 2036 | |
| 2037 | The setuptools ``install`` command is basically a shortcut to run the |
| 2038 | ``easy_install`` command on the current project. However, for convenience |
| 2039 | in creating "system packages" of setuptools-based projects, you can also |
| 2040 | use this option: |
| 2041 | |
| 2042 | ``--single-version-externally-managed`` |
| 2043 | This boolean option tells the ``install`` command to perform an "old style" |
| 2044 | installation, with the addition of an ``.egg-info`` directory so that the |
| 2045 | installed project will still have its metadata available and operate |
| 2046 | normally. If you use this option, you *must* also specify the ``--root`` |
| 2047 | or ``--record`` options (or both), because otherwise you will have no way |
| 2048 | to identify and remove the installed files. |
| 2049 | |
| 2050 | This option is automatically in effect when ``install`` is invoked by another |
| 2051 | distutils command, so that commands like ``bdist_wininst`` and ``bdist_rpm`` |
| 2052 | will create system packages of eggs. It is also automatically in effect if |
| 2053 | you specify the ``--root`` option. |
| 2054 | |
| 2055 | |
| 2056 | ``install_egg_info`` - Install an ``.egg-info`` directory in ``site-packages`` |
| 2057 | ============================================================================== |
| 2058 | |
| 2059 | Setuptools runs this command as part of ``install`` operations that use the |
| 2060 | ``--single-version-externally-managed`` options. You should not invoke it |
| 2061 | directly; it is documented here for completeness and so that distutils |
| 2062 | extensions such as system package builders can make use of it. This command |
| 2063 | has only one option: |
| 2064 | |
| 2065 | ``--install-dir=DIR, -d DIR`` |
| 2066 | The parent directory where the ``.egg-info`` directory will be placed. |
| 2067 | Defaults to the same as the ``--install-dir`` option specified for the |
| 2068 | ``install_lib`` command, which is usually the system ``site-packages`` |
| 2069 | directory. |
| 2070 | |
| 2071 | This command assumes that the ``egg_info`` command has been given valid options |
| 2072 | via the command line or ``setup.cfg``, as it will invoke the ``egg_info`` |
| 2073 | command and use its options to locate the project's source ``.egg-info`` |
| 2074 | directory. |
| 2075 | |
| 2076 | |
| 2077 | .. _rotate: |
| 2078 | |
| 2079 | ``rotate`` - Delete outdated distribution files |
| 2080 | =============================================== |
| 2081 | |
| 2082 | As you develop new versions of your project, your distribution (``dist``) |
| 2083 | directory will gradually fill up with older source and/or binary distribution |
| 2084 | files. The ``rotate`` command lets you automatically clean these up, keeping |
| 2085 | only the N most-recently modified files matching a given pattern. |
| 2086 | |
| 2087 | ``--match=PATTERNLIST, -m PATTERNLIST`` |
| 2088 | Comma-separated list of glob patterns to match. This option is *required*. |
| 2089 | The project name and ``-*`` is prepended to the supplied patterns, in order |
| 2090 | to match only distributions belonging to the current project (in case you |
| 2091 | have a shared distribution directory for multiple projects). Typically, |
| 2092 | you will use a glob pattern like ``.zip`` or ``.egg`` to match files of |
| 2093 | the specified type. Note that each supplied pattern is treated as a |
| 2094 | distinct group of files for purposes of selecting files to delete. |
| 2095 | |
| 2096 | ``--keep=COUNT, -k COUNT`` |
| 2097 | Number of matching distributions to keep. For each group of files |
| 2098 | identified by a pattern specified with the ``--match`` option, delete all |
| 2099 | but the COUNT most-recently-modified files in that group. This option is |
| 2100 | *required*. |
| 2101 | |
| 2102 | ``--dist-dir=DIR, -d DIR`` |
| 2103 | Directory where the distributions are. This defaults to the value of the |
| 2104 | ``bdist`` command's ``--dist-dir`` option, which will usually be the |
| 2105 | project's ``dist`` subdirectory. |
| 2106 | |
| 2107 | **Example 1**: Delete all .tar.gz files from the distribution directory, except |
| 2108 | for the 3 most recently modified ones:: |
| 2109 | |
| 2110 | setup.py rotate --match=.tar.gz --keep=3 |
| 2111 | |
| 2112 | **Example 2**: Delete all Python 2.3 or Python 2.4 eggs from the distribution |
| 2113 | directory, except the most recently modified one for each Python version:: |
| 2114 | |
| 2115 | setup.py rotate --match=-py2.3*.egg,-py2.4*.egg --keep=1 |
| 2116 | |
| 2117 | |
| 2118 | .. _saveopts: |
| 2119 | |
| 2120 | ``saveopts`` - Save used options to a configuration file |
| 2121 | ======================================================== |
| 2122 | |
| 2123 | Finding and editing ``distutils`` configuration files can be a pain, especially |
| 2124 | since you also have to translate the configuration options from command-line |
| 2125 | form to the proper configuration file format. You can avoid these hassles by |
| 2126 | using the ``saveopts`` command. Just add it to the command line to save the |
| 2127 | options you used. For example, this command builds the project using |
| 2128 | the ``mingw32`` C compiler, then saves the --compiler setting as the default |
| 2129 | for future builds (even those run implicitly by the ``install`` command):: |
| 2130 | |
| 2131 | setup.py build --compiler=mingw32 saveopts |
| 2132 | |
| 2133 | The ``saveopts`` command saves all options for every command specified on the |
| 2134 | command line to the project's local ``setup.cfg`` file, unless you use one of |
| 2135 | the `configuration file options`_ to change where the options are saved. For |
| 2136 | example, this command does the same as above, but saves the compiler setting |
| 2137 | to the site-wide (global) distutils configuration:: |
| 2138 | |
| 2139 | setup.py build --compiler=mingw32 saveopts -g |
| 2140 | |
| 2141 | Note that it doesn't matter where you place the ``saveopts`` command on the |
| 2142 | command line; it will still save all the options specified for all commands. |
| 2143 | For example, this is another valid way to spell the last example:: |
| 2144 | |
| 2145 | setup.py saveopts -g build --compiler=mingw32 |
| 2146 | |
| 2147 | Note, however, that all of the commands specified are always run, regardless of |
| 2148 | where ``saveopts`` is placed on the command line. |
| 2149 | |
| 2150 | |
| 2151 | Configuration File Options |
| 2152 | -------------------------- |
| 2153 | |
| 2154 | Normally, settings such as options and aliases are saved to the project's |
| 2155 | local ``setup.cfg`` file. But you can override this and save them to the |
| 2156 | global or per-user configuration files, or to a manually-specified filename. |
| 2157 | |
| 2158 | ``--global-config, -g`` |
| 2159 | Save settings to the global ``distutils.cfg`` file inside the ``distutils`` |
| 2160 | package directory. You must have write access to that directory to use |
| 2161 | this option. You also can't combine this option with ``-u`` or ``-f``. |
| 2162 | |
| 2163 | ``--user-config, -u`` |
| 2164 | Save settings to the current user's ``~/.pydistutils.cfg`` (POSIX) or |
| 2165 | ``$HOME/pydistutils.cfg`` (Windows) file. You can't combine this option |
| 2166 | with ``-g`` or ``-f``. |
| 2167 | |
| 2168 | ``--filename=FILENAME, -f FILENAME`` |
| 2169 | Save settings to the specified configuration file to use. You can't |
| 2170 | combine this option with ``-g`` or ``-u``. Note that if you specify a |
| 2171 | non-standard filename, the ``distutils`` and ``setuptools`` will not |
| 2172 | use the file's contents. This option is mainly included for use in |
| 2173 | testing. |
| 2174 | |
| 2175 | These options are used by other ``setuptools`` commands that modify |
| 2176 | configuration files, such as the `alias`_ and `setopt`_ commands. |
| 2177 | |
| 2178 | |
| 2179 | .. _setopt: |
| 2180 | |
| 2181 | ``setopt`` - Set a distutils or setuptools option in a config file |
| 2182 | ================================================================== |
| 2183 | |
| 2184 | This command is mainly for use by scripts, but it can also be used as a quick |
| 2185 | and dirty way to change a distutils configuration option without having to |
| 2186 | remember what file the options are in and then open an editor. |
| 2187 | |
| 2188 | **Example 1**. Set the default C compiler to ``mingw32`` (using long option |
| 2189 | names):: |
| 2190 | |
| 2191 | setup.py setopt --command=build --option=compiler --set-value=mingw32 |
| 2192 | |
| 2193 | **Example 2**. Remove any setting for the distutils default package |
| 2194 | installation directory (short option names):: |
| 2195 | |
| 2196 | setup.py setopt -c install -o install_lib -r |
| 2197 | |
| 2198 | |
| 2199 | Options for the ``setopt`` command: |
| 2200 | |
| 2201 | ``--command=COMMAND, -c COMMAND`` |
| 2202 | Command to set the option for. This option is required. |
| 2203 | |
| 2204 | ``--option=OPTION, -o OPTION`` |
| 2205 | The name of the option to set. This option is required. |
| 2206 | |
| 2207 | ``--set-value=VALUE, -s VALUE`` |
| 2208 | The value to set the option to. Not needed if ``-r`` or ``--remove`` is |
| 2209 | set. |
| 2210 | |
| 2211 | ``--remove, -r`` |
| 2212 | Remove (unset) the option, instead of setting it. |
| 2213 | |
| 2214 | In addition to the above options, you may use any of the `configuration file |
| 2215 | options`_ (listed under the `saveopts`_ command, above) to determine which |
| 2216 | distutils configuration file the option will be added to (or removed from). |
| 2217 | |
| 2218 | |
| 2219 | .. _test: |
| 2220 | |
| 2221 | ``test`` - Build package and run a unittest suite |
| 2222 | ================================================= |
| 2223 | |
| 2224 | When doing test-driven development, or running automated builds that need |
| 2225 | testing before they are deployed for downloading or use, it's often useful |
| 2226 | to be able to run a project's unit tests without actually deploying the project |
| 2227 | anywhere, even using the ``develop`` command. The ``test`` command runs a |
| 2228 | project's unit tests without actually deploying it, by temporarily putting the |
| 2229 | project's source on ``sys.path``, after first running ``build_ext -i`` and |
| 2230 | ``egg_info`` to ensure that any C extensions and project metadata are |
| 2231 | up-to-date. |
| 2232 | |
| 2233 | To use this command, your project's tests must be wrapped in a ``unittest`` |
| 2234 | test suite by either a function, a ``TestCase`` class or method, or a module |
| 2235 | or package containing ``TestCase`` classes. If the named suite is a module, |
| 2236 | and the module has an ``additional_tests()`` function, it is called and the |
| 2237 | result (which must be a ``unittest.TestSuite``) is added to the tests to be |
| 2238 | run. If the named suite is a package, any submodules and subpackages are |
| 2239 | recursively added to the overall test suite. (Note: if your project specifies |
| 2240 | a ``test_loader``, the rules for processing the chosen ``test_suite`` may |
| 2241 | differ; see the `test_loader`_ documentation for more details.) |
| 2242 | |
| 2243 | Note that many test systems including ``doctest`` support wrapping their |
| 2244 | non-``unittest`` tests in ``TestSuite`` objects. So, if you are using a test |
| 2245 | package that does not support this, we suggest you encourage its developers to |
| 2246 | implement test suite support, as this is a convenient and standard way to |
| 2247 | aggregate a collection of tests to be run under a common test harness. |
| 2248 | |
| 2249 | By default, tests will be run in the "verbose" mode of the ``unittest`` |
| 2250 | package's text test runner, but you can get the "quiet" mode (just dots) if |
| 2251 | you supply the ``-q`` or ``--quiet`` option, either as a global option to |
| 2252 | the setup script (e.g. ``setup.py -q test``) or as an option for the ``test`` |
| 2253 | command itself (e.g. ``setup.py test -q``). There is one other option |
| 2254 | available: |
| 2255 | |
| 2256 | ``--test-suite=NAME, -s NAME`` |
| 2257 | Specify the test suite (or module, class, or method) to be run |
| 2258 | (e.g. ``some_module.test_suite``). The default for this option can be |
| 2259 | set by giving a ``test_suite`` argument to the ``setup()`` function, e.g.:: |
| 2260 | |
| 2261 | setup( |
| 2262 | # ... |
| 2263 | test_suite="my_package.tests.test_all" |
| 2264 | ) |
| 2265 | |
| 2266 | If you did not set a ``test_suite`` in your ``setup()`` call, and do not |
| 2267 | provide a ``--test-suite`` option, an error will occur. |
| 2268 | |
| 2269 | |
| 2270 | .. _upload: |
| 2271 | |
| 2272 | ``upload`` - Upload source and/or egg distributions to PyPI |
| 2273 | =========================================================== |
| 2274 | |
| 2275 | The ``upload`` command is implemented and `documented |
| 2276 | <https://docs.python.org/3.1/distutils/uploading.html>`_ |
| 2277 | in distutils. |
| 2278 | |
| 2279 | Setuptools augments the ``upload`` command with support |
| 2280 | for `keyring <https://pypi.org/project/keyring/>`_, |
| 2281 | allowing the password to be stored in a secure |
| 2282 | location and not in plaintext in the .pypirc file. To use |
| 2283 | keyring, first install keyring and set the password for |
| 2284 | the relevant repository, e.g.:: |
| 2285 | |
| 2286 | python -m keyring set <repository> <username> |
| 2287 | Password for '<username>' in '<repository>': ******** |
| 2288 | |
| 2289 | Then, in .pypirc, set the repository configuration as normal, |
| 2290 | but omit the password. Thereafter, uploads will use the |
| 2291 | password from the keyring. |
| 2292 | |
| 2293 | New in 20.1: Added keyring support. |
| 2294 | |
| 2295 | |
| 2296 | ----------------------------------------- |
| 2297 | Configuring setup() using setup.cfg files |
| 2298 | ----------------------------------------- |
| 2299 | |
| 2300 | .. note:: New in 30.3.0 (8 Dec 2016). |
| 2301 | |
| 2302 | .. important:: |
| 2303 | A ``setup.py`` file containing a ``setup()`` function call is still |
| 2304 | required even if your configuration resides in ``setup.cfg``. |
| 2305 | |
| 2306 | ``Setuptools`` allows using configuration files (usually :file:`setup.cfg`) |
| 2307 | to define a package’s metadata and other options that are normally supplied |
| 2308 | to the ``setup()`` function. |
| 2309 | |
| 2310 | This approach not only allows automation scenarios but also reduces |
| 2311 | boilerplate code in some cases. |
| 2312 | |
| 2313 | .. note:: |
| 2314 | |
| 2315 | This implementation has limited compatibility with the distutils2-like |
| 2316 | ``setup.cfg`` sections used by the ``pbr`` and ``d2to1`` packages. |
| 2317 | |
| 2318 | Namely: only metadata-related keys from ``metadata`` section are supported |
| 2319 | (except for ``description-file``); keys from ``files``, ``entry_points`` |
| 2320 | and ``backwards_compat`` are not supported. |
| 2321 | |
| 2322 | |
| 2323 | .. code-block:: ini |
| 2324 | |
| 2325 | [metadata] |
| 2326 | name = my_package |
| 2327 | version = attr: src.VERSION |
| 2328 | description = My package description |
| 2329 | long_description = file: README.rst, CHANGELOG.rst, LICENSE.rst |
| 2330 | keywords = one, two |
| 2331 | license = BSD 3-Clause License |
| 2332 | classifiers = |
| 2333 | Framework :: Django |
| 2334 | Programming Language :: Python :: 3 |
| 2335 | Programming Language :: Python :: 3.5 |
| 2336 | |
| 2337 | [options] |
| 2338 | zip_safe = False |
| 2339 | include_package_data = True |
| 2340 | packages = find: |
| 2341 | scripts = |
| 2342 | bin/first.py |
| 2343 | bin/second.py |
| 2344 | |
| 2345 | [options.package_data] |
| 2346 | * = *.txt, *.rst |
| 2347 | hello = *.msg |
| 2348 | |
| 2349 | [options.extras_require] |
| 2350 | pdf = ReportLab>=1.2; RXP |
| 2351 | rest = docutils>=0.3; pack ==1.1, ==1.3 |
| 2352 | |
| 2353 | [options.packages.find] |
| 2354 | exclude = |
| 2355 | src.subpackage1 |
| 2356 | src.subpackage2 |
| 2357 | |
| 2358 | |
| 2359 | Metadata and options are set in the config sections of the same name. |
| 2360 | |
| 2361 | * Keys are the same as the keyword arguments one provides to the ``setup()`` |
| 2362 | function. |
| 2363 | |
| 2364 | * Complex values can be written comma-separated or placed one per line |
| 2365 | in *dangling* config values. The following are equivalent: |
| 2366 | |
| 2367 | .. code-block:: ini |
| 2368 | |
| 2369 | [metadata] |
| 2370 | keywords = one, two |
| 2371 | |
| 2372 | [metadata] |
| 2373 | keywords = |
| 2374 | one |
| 2375 | two |
| 2376 | |
| 2377 | * In some cases, complex values can be provided in dedicated subsections for |
| 2378 | clarity. |
| 2379 | |
| 2380 | * Some keys allow ``file:``, ``attr:``, and ``find:`` directives in order to |
| 2381 | cover common usecases. |
| 2382 | |
| 2383 | * Unknown keys are ignored. |
| 2384 | |
| 2385 | |
| 2386 | Specifying values |
| 2387 | ================= |
| 2388 | |
| 2389 | Some values are treated as simple strings, some allow more logic. |
| 2390 | |
| 2391 | Type names used below: |
| 2392 | |
| 2393 | * ``str`` - simple string |
| 2394 | * ``list-comma`` - dangling list or string of comma-separated values |
| 2395 | * ``list-semi`` - dangling list or string of semicolon-separated values |
| 2396 | * ``bool`` - ``True`` is 1, yes, true |
| 2397 | * ``dict`` - list-comma where keys are separated from values by ``=`` |
| 2398 | * ``section`` - values are read from a dedicated (sub)section |
| 2399 | |
| 2400 | |
| 2401 | Special directives: |
| 2402 | |
| 2403 | * ``attr:`` - Value is read from a module attribute. ``attr:`` supports |
| 2404 | callables and iterables; unsupported types are cast using ``str()``. |
| 2405 | * ``file:`` - Value is read from a list of files and then concatenated |
| 2406 | |
| 2407 | |
| 2408 | .. note:: |
| 2409 | The ``file:`` directive is sandboxed and won't reach anything outside |
| 2410 | the directory containing ``setup.py``. |
| 2411 | |
| 2412 | |
| 2413 | Metadata |
| 2414 | -------- |
| 2415 | |
| 2416 | .. note:: |
| 2417 | The aliases given below are supported for compatibility reasons, |
| 2418 | but their use is not advised. |
| 2419 | |
| 2420 | ============================== ================= ===== |
| 2421 | Key Aliases Type |
| 2422 | ============================== ================= ===== |
| 2423 | name str |
| 2424 | version attr:, str |
| 2425 | url home-page str |
| 2426 | download_url download-url str |
| 2427 | project_urls dict |
| 2428 | author str |
| 2429 | author_email author-email str |
| 2430 | maintainer str |
| 2431 | maintainer_email maintainer-email str |
| 2432 | classifiers classifier file:, list-comma |
| 2433 | license file:, str |
| 2434 | description summary file:, str |
| 2435 | long_description long-description file:, str |
| 2436 | long_description_content_type str |
| 2437 | keywords list-comma |
| 2438 | platforms platform list-comma |
| 2439 | provides list-comma |
| 2440 | requires list-comma |
| 2441 | obsoletes list-comma |
| 2442 | ============================== ================= ===== |
| 2443 | |
| 2444 | |
| 2445 | Options |
| 2446 | ------- |
| 2447 | |
| 2448 | ======================= ===== |
| 2449 | Key Type |
| 2450 | ======================= ===== |
| 2451 | zip_safe bool |
| 2452 | setup_requires list-semi |
| 2453 | install_requires list-semi |
| 2454 | extras_require section |
| 2455 | python_requires str |
| 2456 | entry_points file:, section |
| 2457 | use_2to3 bool |
| 2458 | use_2to3_fixers list-comma |
| 2459 | use_2to3_exclude_fixers list-comma |
| 2460 | convert_2to3_doctests list-comma |
| 2461 | scripts list-comma |
| 2462 | eager_resources list-comma |
| 2463 | dependency_links list-comma |
| 2464 | tests_require list-semi |
| 2465 | include_package_data bool |
| 2466 | packages find:, list-comma |
| 2467 | package_dir dict |
| 2468 | package_data section |
| 2469 | exclude_package_data section |
| 2470 | namespace_packages list-comma |
| 2471 | py_modules list-comma |
| 2472 | ======================= ===== |
| 2473 | |
| 2474 | .. note:: |
| 2475 | |
| 2476 | **packages** - The ``find:`` directive can be further configured |
| 2477 | in a dedicated subsection ``options.packages.find``. This subsection |
| 2478 | accepts the same keys as the `setuptools.find` function: |
| 2479 | ``where``, ``include``, and ``exclude``. |
| 2480 | |
| 2481 | |
| 2482 | Configuration API |
| 2483 | ================= |
| 2484 | |
| 2485 | Some automation tools may wish to access data from a configuration file. |
| 2486 | |
| 2487 | ``Setuptools`` exposes a ``read_configuration()`` function for |
| 2488 | parsing ``metadata`` and ``options`` sections into a dictionary. |
| 2489 | |
| 2490 | |
| 2491 | .. code-block:: python |
| 2492 | |
| 2493 | from setuptools.config import read_configuration |
| 2494 | |
| 2495 | conf_dict = read_configuration('/home/user/dev/package/setup.cfg') |
| 2496 | |
| 2497 | |
| 2498 | By default, ``read_configuration()`` will read only the file provided |
| 2499 | in the first argument. To include values from other configuration files |
| 2500 | which could be in various places, set the ``find_others`` keyword argument |
| 2501 | to ``True``. |
| 2502 | |
| 2503 | If you have only a configuration file but not the whole package, you can still |
| 2504 | try to get data out of it with the help of the ``ignore_option_errors`` keyword |
| 2505 | argument. When it is set to ``True``, all options with errors possibly produced |
| 2506 | by directives, such as ``attr:`` and others, will be silently ignored. |
| 2507 | As a consequence, the resulting dictionary will include no such options. |
| 2508 | |
| 2509 | |
| 2510 | -------------------------------- |
| 2511 | Extending and Reusing Setuptools |
| 2512 | -------------------------------- |
| 2513 | |
| 2514 | Creating ``distutils`` Extensions |
| 2515 | ================================= |
| 2516 | |
| 2517 | It can be hard to add new commands or setup arguments to the distutils. But |
| 2518 | the ``setuptools`` package makes it a bit easier, by allowing you to distribute |
| 2519 | a distutils extension as a separate project, and then have projects that need |
| 2520 | the extension just refer to it in their ``setup_requires`` argument. |
| 2521 | |
| 2522 | With ``setuptools``, your distutils extension projects can hook in new |
| 2523 | commands and ``setup()`` arguments just by defining "entry points". These |
| 2524 | are mappings from command or argument names to a specification of where to |
| 2525 | import a handler from. (See the section on `Dynamic Discovery of Services and |
| 2526 | Plugins`_ above for some more background on entry points.) |
| 2527 | |
| 2528 | |
| 2529 | Adding Commands |
| 2530 | --------------- |
| 2531 | |
| 2532 | You can add new ``setup`` commands by defining entry points in the |
| 2533 | ``distutils.commands`` group. For example, if you wanted to add a ``foo`` |
| 2534 | command, you might add something like this to your distutils extension |
| 2535 | project's setup script:: |
| 2536 | |
| 2537 | setup( |
| 2538 | # ... |
| 2539 | entry_points={ |
| 2540 | "distutils.commands": [ |
| 2541 | "foo = mypackage.some_module:foo", |
| 2542 | ], |
| 2543 | }, |
| 2544 | ) |
| 2545 | |
| 2546 | (Assuming, of course, that the ``foo`` class in ``mypackage.some_module`` is |
| 2547 | a ``setuptools.Command`` subclass.) |
| 2548 | |
| 2549 | Once a project containing such entry points has been activated on ``sys.path``, |
| 2550 | (e.g. by running "install" or "develop" with a site-packages installation |
| 2551 | directory) the command(s) will be available to any ``setuptools``-based setup |
| 2552 | scripts. It is not necessary to use the ``--command-packages`` option or |
| 2553 | to monkeypatch the ``distutils.command`` package to install your commands; |
| 2554 | ``setuptools`` automatically adds a wrapper to the distutils to search for |
| 2555 | entry points in the active distributions on ``sys.path``. In fact, this is |
| 2556 | how setuptools' own commands are installed: the setuptools project's setup |
| 2557 | script defines entry points for them! |
| 2558 | |
| 2559 | |
| 2560 | Adding ``setup()`` Arguments |
| 2561 | ---------------------------- |
| 2562 | |
| 2563 | Sometimes, your commands may need additional arguments to the ``setup()`` |
| 2564 | call. You can enable this by defining entry points in the |
| 2565 | ``distutils.setup_keywords`` group. For example, if you wanted a ``setup()`` |
| 2566 | argument called ``bar_baz``, you might add something like this to your |
| 2567 | distutils extension project's setup script:: |
| 2568 | |
| 2569 | setup( |
| 2570 | # ... |
| 2571 | entry_points={ |
| 2572 | "distutils.commands": [ |
| 2573 | "foo = mypackage.some_module:foo", |
| 2574 | ], |
| 2575 | "distutils.setup_keywords": [ |
| 2576 | "bar_baz = mypackage.some_module:validate_bar_baz", |
| 2577 | ], |
| 2578 | }, |
| 2579 | ) |
| 2580 | |
| 2581 | The idea here is that the entry point defines a function that will be called |
| 2582 | to validate the ``setup()`` argument, if it's supplied. The ``Distribution`` |
| 2583 | object will have the initial value of the attribute set to ``None``, and the |
| 2584 | validation function will only be called if the ``setup()`` call sets it to |
| 2585 | a non-None value. Here's an example validation function:: |
| 2586 | |
| 2587 | def assert_bool(dist, attr, value): |
| 2588 | """Verify that value is True, False, 0, or 1""" |
| 2589 | if bool(value) != value: |
| 2590 | raise DistutilsSetupError( |
| 2591 | "%r must be a boolean value (got %r)" % (attr,value) |
| 2592 | ) |
| 2593 | |
| 2594 | Your function should accept three arguments: the ``Distribution`` object, |
| 2595 | the attribute name, and the attribute value. It should raise a |
| 2596 | ``DistutilsSetupError`` (from the ``distutils.errors`` module) if the argument |
| 2597 | is invalid. Remember, your function will only be called with non-None values, |
| 2598 | and the default value of arguments defined this way is always None. So, your |
| 2599 | commands should always be prepared for the possibility that the attribute will |
| 2600 | be ``None`` when they access it later. |
| 2601 | |
| 2602 | If more than one active distribution defines an entry point for the same |
| 2603 | ``setup()`` argument, *all* of them will be called. This allows multiple |
| 2604 | distutils extensions to define a common argument, as long as they agree on |
| 2605 | what values of that argument are valid. |
| 2606 | |
| 2607 | Also note that as with commands, it is not necessary to subclass or monkeypatch |
| 2608 | the distutils ``Distribution`` class in order to add your arguments; it is |
| 2609 | sufficient to define the entry points in your extension, as long as any setup |
| 2610 | script using your extension lists your project in its ``setup_requires`` |
| 2611 | argument. |
| 2612 | |
| 2613 | |
| 2614 | Adding new EGG-INFO Files |
| 2615 | ------------------------- |
| 2616 | |
| 2617 | Some extensible applications or frameworks may want to allow third parties to |
| 2618 | develop plugins with application or framework-specific metadata included in |
| 2619 | the plugins' EGG-INFO directory, for easy access via the ``pkg_resources`` |
| 2620 | metadata API. The easiest way to allow this is to create a distutils extension |
| 2621 | to be used from the plugin projects' setup scripts (via ``setup_requires``) |
| 2622 | that defines a new setup keyword, and then uses that data to write an EGG-INFO |
| 2623 | file when the ``egg_info`` command is run. |
| 2624 | |
| 2625 | The ``egg_info`` command looks for extension points in an ``egg_info.writers`` |
| 2626 | group, and calls them to write the files. Here's a simple example of a |
| 2627 | distutils extension defining a setup argument ``foo_bar``, which is a list of |
| 2628 | lines that will be written to ``foo_bar.txt`` in the EGG-INFO directory of any |
| 2629 | project that uses the argument:: |
| 2630 | |
| 2631 | setup( |
| 2632 | # ... |
| 2633 | entry_points={ |
| 2634 | "distutils.setup_keywords": [ |
| 2635 | "foo_bar = setuptools.dist:assert_string_list", |
| 2636 | ], |
| 2637 | "egg_info.writers": [ |
| 2638 | "foo_bar.txt = setuptools.command.egg_info:write_arg", |
| 2639 | ], |
| 2640 | }, |
| 2641 | ) |
| 2642 | |
| 2643 | This simple example makes use of two utility functions defined by setuptools |
| 2644 | for its own use: a routine to validate that a setup keyword is a sequence of |
| 2645 | strings, and another one that looks up a setup argument and writes it to |
| 2646 | a file. Here's what the writer utility looks like:: |
| 2647 | |
| 2648 | def write_arg(cmd, basename, filename): |
| 2649 | argname = os.path.splitext(basename)[0] |
| 2650 | value = getattr(cmd.distribution, argname, None) |
| 2651 | if value is not None: |
| 2652 | value = '\n'.join(value) + '\n' |
| 2653 | cmd.write_or_delete_file(argname, filename, value) |
| 2654 | |
| 2655 | As you can see, ``egg_info.writers`` entry points must be a function taking |
| 2656 | three arguments: a ``egg_info`` command instance, the basename of the file to |
| 2657 | write (e.g. ``foo_bar.txt``), and the actual full filename that should be |
| 2658 | written to. |
| 2659 | |
| 2660 | In general, writer functions should honor the command object's ``dry_run`` |
| 2661 | setting when writing files, and use the ``distutils.log`` object to do any |
| 2662 | console output. The easiest way to conform to this requirement is to use |
| 2663 | the ``cmd`` object's ``write_file()``, ``delete_file()``, and |
| 2664 | ``write_or_delete_file()`` methods exclusively for your file operations. See |
| 2665 | those methods' docstrings for more details. |
| 2666 | |
| 2667 | |
| 2668 | Adding Support for Revision Control Systems |
| 2669 | ------------------------------------------------- |
| 2670 | |
| 2671 | If the files you want to include in the source distribution are tracked using |
| 2672 | Git, Mercurial or SVN, you can use the following packages to achieve that: |
| 2673 | |
| 2674 | - Git and Mercurial: `setuptools_scm <https://pypi.org/project/setuptools_scm/>`_ |
| 2675 | - SVN: `setuptools_svn <https://pypi.org/project/setuptools_svn/>`_ |
| 2676 | |
| 2677 | If you would like to create a plugin for ``setuptools`` to find files tracked |
| 2678 | by another revision control system, you can do so by adding an entry point to |
| 2679 | the ``setuptools.file_finders`` group. The entry point should be a function |
| 2680 | accepting a single directory name, and should yield all the filenames within |
| 2681 | that directory (and any subdirectories thereof) that are under revision |
| 2682 | control. |
| 2683 | |
| 2684 | For example, if you were going to create a plugin for a revision control system |
| 2685 | called "foobar", you would write a function something like this: |
| 2686 | |
| 2687 | .. code-block:: python |
| 2688 | |
| 2689 | def find_files_for_foobar(dirname): |
| 2690 | # loop to yield paths that start with `dirname` |
| 2691 | |
| 2692 | And you would register it in a setup script using something like this:: |
| 2693 | |
| 2694 | entry_points={ |
| 2695 | "setuptools.file_finders": [ |
| 2696 | "foobar = my_foobar_module:find_files_for_foobar", |
| 2697 | ] |
| 2698 | } |
| 2699 | |
| 2700 | Then, anyone who wants to use your plugin can simply install it, and their |
| 2701 | local setuptools installation will be able to find the necessary files. |
| 2702 | |
| 2703 | It is not necessary to distribute source control plugins with projects that |
| 2704 | simply use the other source control system, or to specify the plugins in |
| 2705 | ``setup_requires``. When you create a source distribution with the ``sdist`` |
| 2706 | command, setuptools automatically records what files were found in the |
| 2707 | ``SOURCES.txt`` file. That way, recipients of source distributions don't need |
| 2708 | to have revision control at all. However, if someone is working on a package |
| 2709 | by checking out with that system, they will need the same plugin(s) that the |
| 2710 | original author is using. |
| 2711 | |
| 2712 | A few important points for writing revision control file finders: |
| 2713 | |
| 2714 | * Your finder function MUST return relative paths, created by appending to the |
| 2715 | passed-in directory name. Absolute paths are NOT allowed, nor are relative |
| 2716 | paths that reference a parent directory of the passed-in directory. |
| 2717 | |
| 2718 | * Your finder function MUST accept an empty string as the directory name, |
| 2719 | meaning the current directory. You MUST NOT convert this to a dot; just |
| 2720 | yield relative paths. So, yielding a subdirectory named ``some/dir`` under |
| 2721 | the current directory should NOT be rendered as ``./some/dir`` or |
| 2722 | ``/somewhere/some/dir``, but *always* as simply ``some/dir`` |
| 2723 | |
| 2724 | * Your finder function SHOULD NOT raise any errors, and SHOULD deal gracefully |
| 2725 | with the absence of needed programs (i.e., ones belonging to the revision |
| 2726 | control system itself. It *may*, however, use ``distutils.log.warn()`` to |
| 2727 | inform the user of the missing program(s). |
| 2728 | |
| 2729 | |
| 2730 | Subclassing ``Command`` |
| 2731 | ----------------------- |
| 2732 | |
| 2733 | Sorry, this section isn't written yet, and neither is a lot of what's below |
| 2734 | this point. |
| 2735 | |
| 2736 | XXX |
| 2737 | |
| 2738 | |
| 2739 | Reusing ``setuptools`` Code |
| 2740 | =========================== |
| 2741 | |
| 2742 | ``ez_setup`` |
| 2743 | ------------ |
| 2744 | |
| 2745 | XXX |
| 2746 | |
| 2747 | |
| 2748 | ``setuptools.archive_util`` |
| 2749 | --------------------------- |
| 2750 | |
| 2751 | XXX |
| 2752 | |
| 2753 | |
| 2754 | ``setuptools.sandbox`` |
| 2755 | ---------------------- |
| 2756 | |
| 2757 | XXX |
| 2758 | |
| 2759 | |
| 2760 | ``setuptools.package_index`` |
| 2761 | ---------------------------- |
| 2762 | |
| 2763 | XXX |
| 2764 | |
| 2765 | |
| 2766 | Mailing List and Bug Tracker |
| 2767 | ============================ |
| 2768 | |
| 2769 | Please use the `distutils-sig mailing list`_ for questions and discussion about |
| 2770 | setuptools, and the `setuptools bug tracker`_ ONLY for issues you have |
| 2771 | confirmed via the list are actual bugs, and which you have reduced to a minimal |
| 2772 | set of steps to reproduce. |
| 2773 | |
| 2774 | .. _distutils-sig mailing list: http://mail.python.org/pipermail/distutils-sig/ |
| 2775 | .. _setuptools bug tracker: https://github.com/pypa/setuptools/ |