a bit of work on the new documentation structure
diff --git a/docs/advanced/cast/chrono.rst b/docs/advanced/cast/chrono.rst
index 1213e2c..6d4a5ee 100644
--- a/docs/advanced/cast/chrono.rst
+++ b/docs/advanced/cast/chrono.rst
@@ -39,7 +39,7 @@
If it is a typedef of the system clock, python will get datetime objects, but if
it is a different clock they will be timedelta objects.
-Conversions Provided
+Provided conversions
--------------------
.. rubric:: C++ to Python
diff --git a/docs/advanced/cast/index.rst b/docs/advanced/cast/index.rst
index a0e4d56..36586af 100644
--- a/docs/advanced/cast/index.rst
+++ b/docs/advanced/cast/index.rst
@@ -1,148 +1,41 @@
Type conversions
################
-There are 3 mechanisms that pybind11 uses to move data between C++ and Python.
-We'll take a quick look at each one to get an overview of what's happening.
+Apart from enabling cross-language function calls, a fundamental problem
+that a binding tool like pybind11 must address is to provide access to
+native Python types in C++ and vice versa. There are three fundamentally
+different ways to do this—which approach is preferable for a particular type
+depends on the situation at hand.
-.. rubric:: 1. Native type in C++, wrapper in Python
+1. Use a native C++ type everywhere. In this case, the type must be wrapped
+ using pybind11-generated bindings so that Python can interact with it.
-Exposing a custom C++ type using :class:`py::class_` was covered in detail in
-the :doc:`/classes` section. There, the underlying data structure is always the
-original C++ class while the :class:`py::class_` wrapper provides a Python
-interface. Internally, when an object like this is sent from C++ to Python,
-pybind11 will just add the outer wrapper layer over the native C++ object.
-Getting it back from Python is just a matter of peeling off the wrapper.
+2. Use a native Python type everywhere. It will need to be wrapped so that
+ C++ functions can interact with it.
-.. rubric:: 2. Wrapper in C++, native type in Python
+3. Use a native C++ type on the C++ side and a native Python type on the
+ Python side. pybind11 refers to this as a *type conversion*.
-This is the exact opposite situation. Now, we have a type which is native to
-Python, like a ``tuple`` or a ``list``. One way to get this data into C++ is
-with the :class:`py::object` family of wrappers. These are explained in more
-detail in the :doc:`/advanced/pycpp/object` section. We'll just give a quick
-example here:
+ Type conversions are the most "natural" option in the sense that native
+ (non-wrapped) types are used everywhere. The main downside is that a copy
+ of the data must be made on every Python ↔ C++ transition: this is
+ needed since the C++ and Python versions of the same type generally won't
+ have the same memory layout.
-.. code-block:: cpp
+ pybind11 can perform many kinds of conversions automatically. An overview
+ is provided in the table ":ref:`conversion_table`".
- void print_list(py::list my_list) {
- for (auto item : my_list)
- std::cout << item << " ";
- }
-
-.. code-block:: pycon
-
- >>> print_list([1, 2, 3])
- 1 2 3
-
-The Python ``list`` is not converted in any way -- it's just wrapped in a C++
-:class:`py::list` class. At its core it's still a Python object. Copying a
-:class:`py::list` will do the usual reference-counting like in Python.
-Returning the object to Python will just remove the thin wrapper.
-
-.. rubric:: 3. Converting between native C++ and Python types
-
-In the previous two cases we had a native type in one language and a wrapper in
-the other. Now, we have native types on both sides and we convert between them.
-
-.. code-block:: cpp
-
- void print_vector(const std::vector<int> &v) {
- for (auto item : v)
- std::cout << item << "\n";
- }
-
-.. code-block:: pycon
-
- >>> print_vector([1, 2, 3])
- 1 2 3
-
-In this case, pybind11 will construct a new ``std::vector<int>`` and copy each
-element from the Python ``list``. The newly constructed object will be passed
-to ``print_vector``. The same thing happens in the other direction: a new
-``list`` is made to match the value returned from C++.
-
-Lots of these conversions are supported out of the box, as shown in the table
-below. They are very convenient, but keep in mind that these conversions are
-fundamentally based on copying data. This is perfectly fine for small immutable
-types but it may become quite expensive for large data structures. This can be
-avoided by overriding the automatic conversion with a custom wrapper (i.e. the
-above-mentioned approach 1). This requires some manual effort and more details
-are available in the :ref:`opaque` section.
-
-.. rubric:: Supported automatic conversions
+The following subsections discuss the differences between these options in more
+detail. The main focus in this section is on type conversions, which represent
+the last case of the above list.
.. toctree::
:maxdepth: 1
+ overview
stl
functional
chrono
eigen
custom
-The following basic data types are supported out of the box (some may require
-an additional extension header to be included). To pass other data structures
-as arguments and return values, refer to the section on binding :ref:`classes`.
-
-+---------------------------------+--------------------------+-------------------------------+
-| Data type | Description | Header file |
-+=================================+==========================+===============================+
-| ``int8_t``, ``uint8_t`` | 8-bit integers | :file:`pybind11/pybind11.h` |
-+---------------------------------+--------------------------+-------------------------------+
-| ``int16_t``, ``uint16_t`` | 16-bit integers | :file:`pybind11/pybind11.h` |
-+---------------------------------+--------------------------+-------------------------------+
-| ``int32_t``, ``uint32_t`` | 32-bit integers | :file:`pybind11/pybind11.h` |
-+---------------------------------+--------------------------+-------------------------------+
-| ``int64_t``, ``uint64_t`` | 64-bit integers | :file:`pybind11/pybind11.h` |
-+---------------------------------+--------------------------+-------------------------------+
-| ``ssize_t``, ``size_t`` | Platform-dependent size | :file:`pybind11/pybind11.h` |
-+---------------------------------+--------------------------+-------------------------------+
-| ``float``, ``double`` | Floating point types | :file:`pybind11/pybind11.h` |
-+---------------------------------+--------------------------+-------------------------------+
-| ``bool`` | Two-state Boolean type | :file:`pybind11/pybind11.h` |
-+---------------------------------+--------------------------+-------------------------------+
-| ``char`` | Character literal | :file:`pybind11/pybind11.h` |
-+---------------------------------+--------------------------+-------------------------------+
-| ``wchar_t`` | Wide character literal | :file:`pybind11/pybind11.h` |
-+---------------------------------+--------------------------+-------------------------------+
-| ``const char *`` | UTF-8 string literal | :file:`pybind11/pybind11.h` |
-+---------------------------------+--------------------------+-------------------------------+
-| ``const wchar_t *`` | Wide string literal | :file:`pybind11/pybind11.h` |
-+---------------------------------+--------------------------+-------------------------------+
-| ``std::string`` | STL dynamic UTF-8 string | :file:`pybind11/pybind11.h` |
-+---------------------------------+--------------------------+-------------------------------+
-| ``std::wstring`` | STL dynamic wide string | :file:`pybind11/pybind11.h` |
-+---------------------------------+--------------------------+-------------------------------+
-| ``std::pair<T1, T2>`` | Pair of two custom types | :file:`pybind11/pybind11.h` |
-+---------------------------------+--------------------------+-------------------------------+
-| ``std::tuple<...>`` | Arbitrary tuple of types | :file:`pybind11/pybind11.h` |
-+---------------------------------+--------------------------+-------------------------------+
-| ``std::reference_wrapper<...>`` | Reference type wrapper | :file:`pybind11/pybind11.h` |
-+---------------------------------+--------------------------+-------------------------------+
-| ``std::complex<T>`` | Complex numbers | :file:`pybind11/complex.h` |
-+---------------------------------+--------------------------+-------------------------------+
-| ``std::array<T, Size>`` | STL static array | :file:`pybind11/stl.h` |
-+---------------------------------+--------------------------+-------------------------------+
-| ``std::vector<T>`` | STL dynamic array | :file:`pybind11/stl.h` |
-+---------------------------------+--------------------------+-------------------------------+
-| ``std::list<T>`` | STL linked list | :file:`pybind11/stl.h` |
-+---------------------------------+--------------------------+-------------------------------+
-| ``std::map<T1, T2>`` | STL ordered map | :file:`pybind11/stl.h` |
-+---------------------------------+--------------------------+-------------------------------+
-| ``std::unordered_map<T1, T2>`` | STL unordered map | :file:`pybind11/stl.h` |
-+---------------------------------+--------------------------+-------------------------------+
-| ``std::set<T>`` | STL ordered set | :file:`pybind11/stl.h` |
-+---------------------------------+--------------------------+-------------------------------+
-| ``std::unordered_set<T>`` | STL unordered set | :file:`pybind11/stl.h` |
-+---------------------------------+--------------------------+-------------------------------+
-| ``std::function<...>`` | STL polymorphic function | :file:`pybind11/functional.h` |
-+---------------------------------+--------------------------+-------------------------------+
-| ``std::chrono::duration<...>`` | STL time duration | :file:`pybind11/chrono.h` |
-+---------------------------------+--------------------------+-------------------------------+
-| ``std::chrono::time_point<...>``| STL date/time | :file:`pybind11/chrono.h` |
-+---------------------------------+--------------------------+-------------------------------+
-| ``Eigen::Matrix<...>`` | Eigen: dense matrix | :file:`pybind11/eigen.h` |
-+---------------------------------+--------------------------+-------------------------------+
-| ``Eigen::Map<...>`` | Eigen: mapped memory | :file:`pybind11/eigen.h` |
-+---------------------------------+--------------------------+-------------------------------+
-| ``Eigen::SparseMatrix<...>`` | Eigen: sparse matrix | :file:`pybind11/eigen.h` |
-+---------------------------------+--------------------------+-------------------------------+
diff --git a/docs/advanced/cast/overview.rst b/docs/advanced/cast/overview.rst
new file mode 100644
index 0000000..4df1bd1
--- /dev/null
+++ b/docs/advanced/cast/overview.rst
@@ -0,0 +1,140 @@
+Overview
+########
+
+.. rubric:: 1. Native type in C++, wrapper in Python
+
+Exposing a custom C++ type using :class:`py::class_` was covered in detail
+in the :doc:`/classes` section. There, the underlying data structure is
+always the original C++ class while the :class:`py::class_` wrapper provides
+a Python interface. Internally, when an object like this is sent from C++ to
+Python, pybind11 will just add the outer wrapper layer over the native C++
+object. Getting it back from Python is just a matter of peeling off the
+wrapper.
+
+.. rubric:: 2. Wrapper in C++, native type in Python
+
+This is the exact opposite situation. Now, we have a type which is native to
+Python, like a ``tuple`` or a ``list``. One way to get this data into C++ is
+with the :class:`py::object` family of wrappers. These are explained in more
+detail in the :doc:`/advanced/pycpp/object` section. We'll just give a quick
+example here:
+
+.. code-block:: cpp
+
+ void print_list(py::list my_list) {
+ for (auto item : my_list)
+ std::cout << item << " ";
+ }
+
+.. code-block:: pycon
+
+ >>> print_list([1, 2, 3])
+ 1 2 3
+
+The Python ``list`` is not converted in any way -- it's just wrapped in a C++
+:class:`py::list` class. At its core it's still a Python object. Copying a
+:class:`py::list` will do the usual reference-counting like in Python.
+Returning the object to Python will just remove the thin wrapper.
+
+.. rubric:: 3. Converting between native C++ and Python types
+
+In the previous two cases we had a native type in one language and a wrapper in
+the other. Now, we have native types on both sides and we convert between them.
+
+.. code-block:: cpp
+
+ void print_vector(const std::vector<int> &v) {
+ for (auto item : v)
+ std::cout << item << "\n";
+ }
+
+.. code-block:: pycon
+
+ >>> print_vector([1, 2, 3])
+ 1 2 3
+
+In this case, pybind11 will construct a new ``std::vector<int>`` and copy each
+element from the Python ``list``. The newly constructed object will be passed
+to ``print_vector``. The same thing happens in the other direction: a new
+``list`` is made to match the value returned from C++.
+
+Lots of these conversions are supported out of the box, as shown in the table
+below. They are very convenient, but keep in mind that these conversions are
+fundamentally based on copying data. This is perfectly fine for small immutable
+types but it may become quite expensive for large data structures. This can be
+avoided by overriding the automatic conversion with a custom wrapper (i.e. the
+above-mentioned approach 1). This requires some manual effort and more details
+are available in the :ref:`opaque` section.
+
+.. _conversion_table:
+
+List of all builtin conversions
+-------------------------------
+
+The following basic data types are supported out of the box (some may require
+an additional extension header to be included). To pass other data structures
+as arguments and return values, refer to the section on binding :ref:`classes`.
+
++---------------------------------+--------------------------+-------------------------------+
+| Data type | Description | Header file |
++=================================+==========================+===============================+
+| ``int8_t``, ``uint8_t`` | 8-bit integers | :file:`pybind11/pybind11.h` |
++---------------------------------+--------------------------+-------------------------------+
+| ``int16_t``, ``uint16_t`` | 16-bit integers | :file:`pybind11/pybind11.h` |
++---------------------------------+--------------------------+-------------------------------+
+| ``int32_t``, ``uint32_t`` | 32-bit integers | :file:`pybind11/pybind11.h` |
++---------------------------------+--------------------------+-------------------------------+
+| ``int64_t``, ``uint64_t`` | 64-bit integers | :file:`pybind11/pybind11.h` |
++---------------------------------+--------------------------+-------------------------------+
+| ``ssize_t``, ``size_t`` | Platform-dependent size | :file:`pybind11/pybind11.h` |
++---------------------------------+--------------------------+-------------------------------+
+| ``float``, ``double`` | Floating point types | :file:`pybind11/pybind11.h` |
++---------------------------------+--------------------------+-------------------------------+
+| ``bool`` | Two-state Boolean type | :file:`pybind11/pybind11.h` |
++---------------------------------+--------------------------+-------------------------------+
+| ``char`` | Character literal | :file:`pybind11/pybind11.h` |
++---------------------------------+--------------------------+-------------------------------+
+| ``wchar_t`` | Wide character literal | :file:`pybind11/pybind11.h` |
++---------------------------------+--------------------------+-------------------------------+
+| ``const char *`` | UTF-8 string literal | :file:`pybind11/pybind11.h` |
++---------------------------------+--------------------------+-------------------------------+
+| ``const wchar_t *`` | Wide string literal | :file:`pybind11/pybind11.h` |
++---------------------------------+--------------------------+-------------------------------+
+| ``std::string`` | STL dynamic UTF-8 string | :file:`pybind11/pybind11.h` |
++---------------------------------+--------------------------+-------------------------------+
+| ``std::wstring`` | STL dynamic wide string | :file:`pybind11/pybind11.h` |
++---------------------------------+--------------------------+-------------------------------+
+| ``std::pair<T1, T2>`` | Pair of two custom types | :file:`pybind11/pybind11.h` |
++---------------------------------+--------------------------+-------------------------------+
+| ``std::tuple<...>`` | Arbitrary tuple of types | :file:`pybind11/pybind11.h` |
++---------------------------------+--------------------------+-------------------------------+
+| ``std::reference_wrapper<...>`` | Reference type wrapper | :file:`pybind11/pybind11.h` |
++---------------------------------+--------------------------+-------------------------------+
+| ``std::complex<T>`` | Complex numbers | :file:`pybind11/complex.h` |
++---------------------------------+--------------------------+-------------------------------+
+| ``std::array<T, Size>`` | STL static array | :file:`pybind11/stl.h` |
++---------------------------------+--------------------------+-------------------------------+
+| ``std::vector<T>`` | STL dynamic array | :file:`pybind11/stl.h` |
++---------------------------------+--------------------------+-------------------------------+
+| ``std::list<T>`` | STL linked list | :file:`pybind11/stl.h` |
++---------------------------------+--------------------------+-------------------------------+
+| ``std::map<T1, T2>`` | STL ordered map | :file:`pybind11/stl.h` |
++---------------------------------+--------------------------+-------------------------------+
+| ``std::unordered_map<T1, T2>`` | STL unordered map | :file:`pybind11/stl.h` |
++---------------------------------+--------------------------+-------------------------------+
+| ``std::set<T>`` | STL ordered set | :file:`pybind11/stl.h` |
++---------------------------------+--------------------------+-------------------------------+
+| ``std::unordered_set<T>`` | STL unordered set | :file:`pybind11/stl.h` |
++---------------------------------+--------------------------+-------------------------------+
+| ``std::function<...>`` | STL polymorphic function | :file:`pybind11/functional.h` |
++---------------------------------+--------------------------+-------------------------------+
+| ``std::chrono::duration<...>`` | STL time duration | :file:`pybind11/chrono.h` |
++---------------------------------+--------------------------+-------------------------------+
+| ``std::chrono::time_point<...>``| STL date/time | :file:`pybind11/chrono.h` |
++---------------------------------+--------------------------+-------------------------------+
+| ``Eigen::Matrix<...>`` | Eigen: dense matrix | :file:`pybind11/eigen.h` |
++---------------------------------+--------------------------+-------------------------------+
+| ``Eigen::Map<...>`` | Eigen: mapped memory | :file:`pybind11/eigen.h` |
++---------------------------------+--------------------------+-------------------------------+
+| ``Eigen::SparseMatrix<...>`` | Eigen: sparse matrix | :file:`pybind11/eigen.h` |
++---------------------------------+--------------------------+-------------------------------+
diff --git a/docs/advanced/functions.rst b/docs/advanced/functions.rst
index 5c697b1..d9886e2 100644
--- a/docs/advanced/functions.rst
+++ b/docs/advanced/functions.rst
@@ -9,48 +9,62 @@
Return value policies
=====================
-Python and C++ use wildly different ways of managing the memory and lifetime of
-objects managed by them. This can lead to issues when creating bindings for
-functions that return a non-trivial type. Just by looking at the type
-information, it is not clear whether Python should take charge of the returned
-value and eventually free its resources, or if this is handled on the C++ side.
-For this reason, pybind11 provides a several `return value policy` annotations
-that can be passed to the :func:`module::def` and :func:`class_::def`
-functions. The default policy is :enum:`return_value_policy::automatic`.
+Python and C++ use fundamentally different ways of managing the memory and
+lifetime of objects managed by them. This can lead to issues when creating
+bindings for functions that return a non-trivial type. Just by looking at the
+type information, it is not clear whether Python should take charge of the
+returned value and eventually free its resources, or if this is handled on the
+C++ side. For this reason, pybind11 provides a several `return value policy`
+annotations that can be passed to the :func:`module::def` and
+:func:`class_::def` functions. The default policy is
+:enum:`return_value_policy::automatic`.
-Return value policies can also be applied to properties, in which case the
-arguments must be passed through the :class:`cpp_function` constructor:
+Return value policies are tricky, and it's very important to get them right.
+Just to illustrate what can go wrong, consider the following simple example:
.. code-block:: cpp
- class_<MyClass>(m, "MyClass")
- def_property("data"
- py::cpp_function(&MyClass::getData, py::return_value_policy::copy),
- py::cpp_function(&MyClass::setData)
- );
+ /* Function declaration */
+ Data *get_data() { return _data; /* (pointer to a static data structure) */ }
+ ...
-The following table provides an overview of the available return value policies:
+ /* Binding code */
+ m.def("get_data", &get_data); // <-- KABOOM, will cause crash when called from Python
+
+What's going on here? When ``get_data()`` is called from Python, the return
+value (a native C++ type) must be wrapped to turn it into a usable Python type.
+In this case, the default return value policy (:enum:`return_value_policy::automatic`)
+causes pybind11 to assume ownership of the static ``_data`` instance.
+
+When Python's garbage collector eventually deletes the Python
+wrapper, pybind11 will also attempt to delete the C++ instance (via ``operator
+delete()``) due to the implied ownership. At this point, the entire application
+will come crashing down, though errors could also be more subtle and involve
+silent data corruption.
+
+In the above example, the policy :enum:`return_value_policy::reference` should have
+been specified so that the global data instance is only *referenced* without any
+implied transfer of ownership, i.e.:
+
+.. code-block:: cpp
+
+ m.def("get_data", &get_data, return_value_policy::reference);
+
+On the other hand, this is not the right policy for many other situations,
+where ignoring ownership could lead to resource leaks.
+As a developer using pybind11, it's important to be familiar with the different
+return value policies, including which situation calls for which one of them.
+The following table provides an overview of available policies:
.. tabularcolumns:: |p{0.5\textwidth}|p{0.45\textwidth}|
+--------------------------------------------------+----------------------------------------------------------------------------+
| Return value policy | Description |
+==================================================+============================================================================+
-| :enum:`return_value_policy::automatic` | This is the default return value policy, which falls back to the policy |
-| | :enum:`return_value_policy::take_ownership` when the return value is a |
-| | pointer. Otherwise, it uses :enum:`return_value::move` or |
-| | :enum:`return_value::copy` for rvalue and lvalue references, respectively. |
-| | See below for a description of what all of these different policies do. |
-+--------------------------------------------------+----------------------------------------------------------------------------+
-| :enum:`return_value_policy::automatic_reference` | As above, but use policy :enum:`return_value_policy::reference` when the |
-| | return value is a pointer. This is the default conversion policy for |
-| | function arguments when calling Python functions manually from C++ code |
-| | (i.e. via handle::operator()). You probably won't need to use this. |
-+--------------------------------------------------+----------------------------------------------------------------------------+
| :enum:`return_value_policy::take_ownership` | Reference an existing object (i.e. do not create a new copy) and take |
| | ownership. Python will call the destructor and delete operator when the |
| | object's reference count reaches zero. Undefined behavior ensues when the |
-| | C++ side does the same. |
+| | C++ side does the same, or when the data was not dynamically allocated. |
+--------------------------------------------------+----------------------------------------------------------------------------+
| :enum:`return_value_policy::copy` | Create a new copy of the returned object, which will be owned by Python. |
| | This policy is comparably safe because the lifetimes of the two instances |
@@ -74,6 +88,28 @@
| | return value is referenced by Python. This is the default policy for |
| | property getters created via ``def_property``, ``def_readwrite``, etc. |
+--------------------------------------------------+----------------------------------------------------------------------------+
+| :enum:`return_value_policy::automatic` | This is the default return value policy, which falls back to the policy |
+| | :enum:`return_value_policy::take_ownership` when the return value is a |
+| | pointer. Otherwise, it uses :enum:`return_value::move` or |
+| | :enum:`return_value::copy` for rvalue and lvalue references, respectively. |
+| | See above for a description of what all of these different policies do. |
++--------------------------------------------------+----------------------------------------------------------------------------+
+| :enum:`return_value_policy::automatic_reference` | As above, but use policy :enum:`return_value_policy::reference` when the |
+| | return value is a pointer. This is the default conversion policy for |
+| | function arguments when calling Python functions manually from C++ code |
+| | (i.e. via handle::operator()). You probably won't need to use this. |
++--------------------------------------------------+----------------------------------------------------------------------------+
+
+Return value policies can also be applied to properties, in which case the
+arguments must be passed through the :class:`cpp_function` constructor:
+
+.. code-block:: cpp
+
+ class_<MyClass>(m, "MyClass")
+ def_property("data"
+ py::cpp_function(&MyClass::getData, py::return_value_policy::copy),
+ py::cpp_function(&MyClass::setData)
+ );
.. warning::
@@ -82,12 +118,14 @@
non-determinism and segmentation faults, hence it is worth spending the
time to understand all the different options in the table above.
-One important aspect of the above policies is that they only apply to instances
-which pybind11 has *not* seen before, in which case the policy clarifies
-essential questions about the return value's lifetime and ownership. When
-pybind11 knows the instance already (as identified by its type and address in
-memory), it will return the existing Python object wrapper rather than creating
-a new copy.
+.. note::
+
+ One important aspect of the above policies is that they only apply to
+ instances which pybind11 has *not* seen before, in which case the policy
+ clarifies essential questions about the return value's lifetime and
+ ownership. When pybind11 knows the instance already (as identified by its
+ type and address in memory), it will return the existing Python object
+ wrapper rather than creating a new copy.
.. note::