feat: setup.py redesign and helpers (#2433)
* feat: setup.py redesign and helpers
* refactor: simpler design with two outputs
* refactor: helper file update and Windows support
* fix: review points from @YannickJadoul
* refactor: fixes to naming and more docs
* feat: more customization points
* feat: add entry point pybind11-config
* refactor: Try Extension-focused method
* refactor: rename alt/inplace to global
* fix: allow usage with git modules, better docs
* feat: global as an extra (@YannickJadoul's suggestion)
* feat: single version location
* fix: remove the requirement that setuptools must be imported first
* fix: some review points from @wjacob
* fix: use .in, add procedure to docs
* refactor: avoid monkeypatch copy
* docs: minor typos corrected
* fix: minor points from @YannickJadoul
* fix: typo on Windows C++ mode
* fix: MSVC 15 update 3+ have c++14 flag
See <https://docs.microsoft.com/en-us/cpp/build/reference/std-specify-language-standard-version?view=vs-2019>
* docs: discuss making SDists by hand
* ci: use pep517.build instead of manual setup.py
* refactor: more comments from @YannickJadoul
* docs: updates from @ktbarrett
* fix: change to newly recommended tool instead of pep517.build
This was intended as a proof of concept; build seems to be the correct replacement.
See https://github.com/pypa/pep517/pull/83
* docs: updates from @wjakob
* refactor: dual version locations
* docs: typo spotted by @wjakob
diff --git a/docs/basics.rst b/docs/basics.rst
index 6bb5f98..71440c9 100644
--- a/docs/basics.rst
+++ b/docs/basics.rst
@@ -11,11 +11,11 @@
Compiling the test cases
========================
-Linux/MacOS
+Linux/macOS
-----------
On Linux you'll need to install the **python-dev** or **python3-dev** packages as
-well as **cmake**. On Mac OS, the included python version works out of the box,
+well as **cmake**. On macOS, the included python version works out of the box,
but **cmake** must still be installed.
After installing the prerequisites, run
@@ -138,7 +138,7 @@
$ c++ -O3 -Wall -shared -std=c++11 -fPIC `python3 -m pybind11 --includes` example.cpp -o example`python3-config --extension-suffix`
-For more details on the required compiler flags on Linux and MacOS, see
+For more details on the required compiler flags on Linux and macOS, see
:ref:`building_manually`. For complete cross-platform compilation instructions,
refer to the :ref:`compiling` page.
diff --git a/docs/changelog.rst b/docs/changelog.rst
index 448192f..8f7d272 100644
--- a/docs/changelog.rst
+++ b/docs/changelog.rst
@@ -43,6 +43,25 @@
* ``py::memoryview`` update and documentation.
`#2223 <https://github.com/pybind/pybind11/pull/2223>`_
+* The Python package was reworked to be more powerful and useful.
+ `#2433 <https://github.com/pybind/pybind11/pull/2433>`_
+
+ * :ref:`build-setuptools` is easier thanks to a new
+ ``pybind11.setup_helpers`` module, which provides utilities to use
+ setuptools with pybind11. It can be used via PEP 518, ``setup_requires``,
+ or by directly copying ``setup_helpers.py`` into your project.
+
+ * CMake configuration files are now included in the Python package. Use
+ ``pybind11.get_cmake_dir()`` or ``python -m pybind11 --cmakedir`` to get
+ the directory with the CMake configuration files, or include the
+ site-packages location in your ``CMAKE_MODULE_PATH``. Or you can use the
+ new ``pybind11[global]`` extra when you install ``pybind11``, which
+ installs the CMake files and headers into your base environment in the
+ standard location
+
+ * ``pybind11-config`` is another way to write ``python -m pybind11`` if you
+ have your PATH set up.
+
* Minimum CMake required increased to 3.4.
`#2338 <https://github.com/pybind/pybind11/pull/2338>`_ and
`#2370 <https://github.com/pybind/pybind11/pull/2370>`_
diff --git a/docs/compiling.rst b/docs/compiling.rst
index b924b85..cbf14a4 100644
--- a/docs/compiling.rst
+++ b/docs/compiling.rst
@@ -3,6 +3,8 @@
Build systems
#############
+.. _build-setuptools:
+
Building with setuptools
========================
@@ -13,6 +15,135 @@
.. [python_example] https://github.com/pybind/python_example
+A helper file is provided with pybind11 that can simplify usage with setuptools.
+
+To use pybind11 inside your ``setup.py``, you have to have some system to
+ensure that ``pybind11`` is installed when you build your package. There are
+four possible ways to do this, and pybind11 supports all four: You can ask all
+users to install pybind11 beforehand (bad), you can use
+:ref:`setup_helpers-pep518` (good, but very new and requires Pip 10),
+:ref:`setup_helpers-setup_requires` (discouraged by Python packagers now that
+PEP 518 is available, but it still works everywhere), or you can
+:ref:`setup_helpers-copy-manually` (always works but you have to manually sync
+your copy to get updates).
+
+An example of a ``setup.py`` using pybind11's helpers:
+
+.. code-block:: python
+
+ from setuptools import setup
+ from pybind11.setup_helpers import Pybind11Extension
+
+ ext_modules = [
+ Pybind11Extension(
+ "python_example",
+ ["src/main.cpp"],
+ ),
+ ]
+
+ setup(
+ ...,
+ ext_modules=ext_modules
+ )
+
+If you want to do an automatic search for the highest supported C++ standard,
+that is supported via a ``build_ext`` command override; it will only affect
+``Pybind11Extensions``:
+
+.. code-block:: python
+
+ from setuptools import setup
+ from pybind11.setup_helpers import Pybind11Extension, build_ext
+
+ ext_modules = [
+ Pybind11Extension(
+ "python_example",
+ ["src/main.cpp"],
+ ),
+ ]
+
+ setup(
+ ...,
+ cmdclass={"build_ext": build_ext},
+ ext_modules=ext_modules
+ )
+
+.. _setup_helpers-pep518:
+
+PEP 518 requirements (Pip 10+ required)
+---------------------------------------
+
+If you use `PEP 518's <https://www.python.org/dev/peps/pep-0518/>`_
+``pyproject.toml`` file, you can ensure that ``pybind11`` is available during
+the compilation of your project. When this file exists, Pip will make a new
+virtual environment, download just the packages listed here in ``requires=``,
+and build a wheel (binary Python package). It will then throw away the
+environment, and install your wheel.
+
+Your ``pyproject.toml`` file will likely look something like this:
+
+.. code-block:: toml
+
+ [build-system]
+ requires = ["setuptools", "wheel", "pybind11==2.6.0"]
+ build-backend = "setuptools.build_meta"
+
+.. note::
+
+ The main drawback to this method is that a `PEP 517`_ compliant build tool,
+ such as Pip 10+, is required for this approach to work; older versions of
+ Pip completely ignore this file. If you distribute binaries (called wheels
+ in Python) using something like `cibuildwheel`_, remember that ``setup.py``
+ and ``pyproject.toml`` are not even contained in the wheel, so this high
+ Pip requirement is only for source builds, and will not affect users of
+ your binary wheels.
+
+.. _PEP 517: https://www.python.org/dev/peps/pep-0517/
+.. _cibuildwheel: https://cibuildwheel.readthedocs.io
+
+.. _setup_helpers-setup_requires:
+
+Classic ``setup_requires``
+--------------------------
+
+If you want to support old versions of Pip with the classic
+``setup_requires=["pybind11"]`` keyword argument to setup, which triggers a
+two-phase ``setup.py`` run, then you will need to use something like this to
+ensure the first pass works (which has not yet installed the ``setup_requires``
+packages, since it can't install something it does not know about):
+
+.. code-block:: python
+
+ try:
+ from pybind11.setup_helpers import Pybind11Extension
+ except ImportError:
+ from setuptools import Extension as Pybind11Extension
+
+
+It doesn't matter that the Extension class is not the enhanced subclass for the
+first pass run; and the second pass will have the ``setup_requires``
+requirements.
+
+This is obviously more of a hack than the PEP 518 method, but it supports
+ancient versions of Pip.
+
+.. _setup_helpers-copy-manually:
+
+Copy manually
+-------------
+
+You can also copy ``setup_helpers.py`` directly to your project; it was
+designed to be usable standalone, like the old example ``setup.py``. You can
+set ``include_pybind11=False`` to skip including the pybind11 package headers,
+so you can use it with git submodules and a specific git version. If you use
+this, you will need to import from a local file in ``setup.py`` and ensure the
+helper file is part of your MANIFEST.
+
+
+.. versionchanged:: 2.6
+
+ Added ``setup_helpers`` file.
+
Building with cppimport
========================
@@ -367,7 +498,7 @@
on the distribution; in the latter case, the module extension can be manually
set to ``.so``.
-On Mac OS: the build command is almost the same but it also requires passing
+On macOS: the build command is almost the same but it also requires passing
the ``-undefined dynamic_lookup`` flag so as to ignore missing symbols when
building the module:
diff --git a/docs/upgrade.rst b/docs/upgrade.rst
index 7320cfc..e4d329b 100644
--- a/docs/upgrade.rst
+++ b/docs/upgrade.rst
@@ -28,6 +28,13 @@
be replaced by ``PYBIND11_OVERRIDE*`` and ``get_override``. In the future, the
old macros may be deprecated and removed.
+The ``pybind11`` package on PyPI no longer fills the wheel "headers" slot - if
+you were using the headers from this slot, they are available by requesting the
+``global`` extra, that is, ``pip install "pybind11[global]"``. (Most users will
+be unaffected, as the ``pybind11/include`` location is reported by ``python -m
+pybind11 --includes`` and ``pybind11.get_include()`` is still correct and has
+not changed since 2.5).
+
CMake support:
--------------
@@ -66,11 +73,22 @@
* Using ``find_package(Python COMPONENTS Interpreter Development)`` before
pybind11 will cause pybind11 to use the new Python mechanisms instead of its
- own custom search, based on a patched version of classic
- FindPythonInterp/FindPythonLibs. In the future, this may become the default.
+ own custom search, based on a patched version of classic ``FindPythonInterp``
+ / ``FindPythonLibs``. In the future, this may become the default.
+v2.5
+====
+
+The Python package now includes the headers as data in the package itself, as
+well as in the "headers" wheel slot. ``pybind11 --includes`` and
+``pybind11.get_include()`` report the new location, which is always correct
+regardless of how pybind11 was installed, making the old ``user=`` argument
+meaningless. If you are not using the function to get the location already, you
+are encouraged to switch to the package location.
+
+
v2.2
====