Dean Moldovan | 67b52d8 | 2016-10-16 19:12:43 +0200 | [diff] [blame] | 1 | Miscellaneous |
| 2 | ############# |
| 3 | |
| 4 | .. _macro_notes: |
| 5 | |
| 6 | General notes regarding convenience macros |
| 7 | ========================================== |
| 8 | |
| 9 | pybind11 provides a few convenience macros such as |
| 10 | :func:`PYBIND11_MAKE_OPAQUE` and :func:`PYBIND11_DECLARE_HOLDER_TYPE`, and |
| 11 | ``PYBIND11_OVERLOAD_*``. Since these are "just" macros that are evaluated |
| 12 | in the preprocessor (which has no concept of types), they *will* get confused |
| 13 | by commas in a template argument such as ``PYBIND11_OVERLOAD(MyReturnValue<T1, |
| 14 | T2>, myFunc)``. In this case, the preprocessor assumes that the comma indicates |
| 15 | the beginning of the next parameter. Use a ``typedef`` to bind the template to |
| 16 | another name and use it in the macro to avoid this problem. |
| 17 | |
| 18 | |
| 19 | Global Interpreter Lock (GIL) |
| 20 | ============================= |
| 21 | |
Dustin Spicuzza | 18d7df5 | 2017-01-31 10:55:16 -0500 | [diff] [blame] | 22 | When calling a C++ function from Python, the GIL is always held. |
Dean Moldovan | 67b52d8 | 2016-10-16 19:12:43 +0200 | [diff] [blame] | 23 | The classes :class:`gil_scoped_release` and :class:`gil_scoped_acquire` can be |
| 24 | used to acquire and release the global interpreter lock in the body of a C++ |
| 25 | function call. In this way, long-running C++ code can be parallelized using |
| 26 | multiple Python threads. Taking :ref:`overriding_virtuals` as an example, this |
| 27 | could be realized as follows (important changes highlighted): |
| 28 | |
| 29 | .. code-block:: cpp |
| 30 | :emphasize-lines: 8,9,33,34 |
| 31 | |
| 32 | class PyAnimal : public Animal { |
| 33 | public: |
| 34 | /* Inherit the constructors */ |
| 35 | using Animal::Animal; |
| 36 | |
| 37 | /* Trampoline (need one for each virtual function) */ |
| 38 | std::string go(int n_times) { |
| 39 | /* Acquire GIL before calling Python code */ |
| 40 | py::gil_scoped_acquire acquire; |
| 41 | |
| 42 | PYBIND11_OVERLOAD_PURE( |
| 43 | std::string, /* Return type */ |
| 44 | Animal, /* Parent class */ |
| 45 | go, /* Name of function */ |
| 46 | n_times /* Argument(s) */ |
| 47 | ); |
| 48 | } |
| 49 | }; |
| 50 | |
| 51 | PYBIND11_PLUGIN(example) { |
| 52 | py::module m("example", "pybind11 example plugin"); |
| 53 | |
| 54 | py::class_<Animal, PyAnimal> animal(m, "Animal"); |
| 55 | animal |
| 56 | .def(py::init<>()) |
| 57 | .def("go", &Animal::go); |
| 58 | |
| 59 | py::class_<Dog>(m, "Dog", animal) |
| 60 | .def(py::init<>()); |
| 61 | |
| 62 | m.def("call_go", [](Animal *animal) -> std::string { |
| 63 | /* Release GIL before calling into (potentially long-running) C++ code */ |
| 64 | py::gil_scoped_release release; |
| 65 | return call_go(animal); |
| 66 | }); |
| 67 | |
| 68 | return m.ptr(); |
| 69 | } |
| 70 | |
| 71 | |
| 72 | Binding sequence data types, iterators, the slicing protocol, etc. |
| 73 | ================================================================== |
| 74 | |
| 75 | Please refer to the supplemental example for details. |
| 76 | |
| 77 | .. seealso:: |
| 78 | |
| 79 | The file :file:`tests/test_sequences_and_iterators.cpp` contains a |
| 80 | complete example that shows how to bind a sequence data type, including |
| 81 | length queries (``__len__``), iterators (``__iter__``), the slicing |
| 82 | protocol and other kinds of useful operations. |
| 83 | |
| 84 | |
| 85 | Partitioning code over multiple extension modules |
| 86 | ================================================= |
| 87 | |
| 88 | It's straightforward to split binding code over multiple extension modules, |
| 89 | while referencing types that are declared elsewhere. Everything "just" works |
| 90 | without any special precautions. One exception to this rule occurs when |
| 91 | extending a type declared in another extension module. Recall the basic example |
| 92 | from Section :ref:`inheritance`. |
| 93 | |
| 94 | .. code-block:: cpp |
| 95 | |
| 96 | py::class_<Pet> pet(m, "Pet"); |
| 97 | pet.def(py::init<const std::string &>()) |
| 98 | .def_readwrite("name", &Pet::name); |
| 99 | |
| 100 | py::class_<Dog>(m, "Dog", pet /* <- specify parent */) |
| 101 | .def(py::init<const std::string &>()) |
| 102 | .def("bark", &Dog::bark); |
| 103 | |
| 104 | Suppose now that ``Pet`` bindings are defined in a module named ``basic``, |
| 105 | whereas the ``Dog`` bindings are defined somewhere else. The challenge is of |
| 106 | course that the variable ``pet`` is not available anymore though it is needed |
| 107 | to indicate the inheritance relationship to the constructor of ``class_<Dog>``. |
| 108 | However, it can be acquired as follows: |
| 109 | |
| 110 | .. code-block:: cpp |
| 111 | |
| 112 | py::object pet = (py::object) py::module::import("basic").attr("Pet"); |
| 113 | |
| 114 | py::class_<Dog>(m, "Dog", pet) |
| 115 | .def(py::init<const std::string &>()) |
| 116 | .def("bark", &Dog::bark); |
| 117 | |
| 118 | Alternatively, you can specify the base class as a template parameter option to |
| 119 | ``class_``, which performs an automated lookup of the corresponding Python |
| 120 | type. Like the above code, however, this also requires invoking the ``import`` |
| 121 | function once to ensure that the pybind11 binding code of the module ``basic`` |
| 122 | has been executed: |
| 123 | |
| 124 | .. code-block:: cpp |
| 125 | |
| 126 | py::module::import("basic"); |
| 127 | |
| 128 | py::class_<Dog, Pet>(m, "Dog") |
| 129 | .def(py::init<const std::string &>()) |
| 130 | .def("bark", &Dog::bark); |
| 131 | |
| 132 | Naturally, both methods will fail when there are cyclic dependencies. |
| 133 | |
| 134 | Note that compiling code which has its default symbol visibility set to |
| 135 | *hidden* (e.g. via the command line flag ``-fvisibility=hidden`` on GCC/Clang) can interfere with the |
| 136 | ability to access types defined in another extension module. Workarounds |
| 137 | include changing the global symbol visibility (not recommended, because it will |
| 138 | lead unnecessarily large binaries) or manually exporting types that are |
| 139 | accessed by multiple extension modules: |
| 140 | |
| 141 | .. code-block:: cpp |
| 142 | |
| 143 | #ifdef _WIN32 |
| 144 | # define EXPORT_TYPE __declspec(dllexport) |
| 145 | #else |
| 146 | # define EXPORT_TYPE __attribute__ ((visibility("default"))) |
| 147 | #endif |
| 148 | |
| 149 | class EXPORT_TYPE Dog : public Animal { |
| 150 | ... |
| 151 | }; |
| 152 | |
Ivan Smirnov | f95fda0 | 2016-10-31 21:40:11 +0000 | [diff] [blame] | 153 | Note also that it is possible (although would rarely be required) to share arbitrary |
| 154 | C++ objects between extension modules at runtime. Internal library data is shared |
| 155 | between modules using capsule machinery [#f6]_ which can be also utilized for |
| 156 | storing, modifying and accessing user-defined data. Note that an extension module |
| 157 | will "see" other extensions' data if and only if they were built with the same |
| 158 | pybind11 version. Consider the following example: |
| 159 | |
| 160 | .. code-block:: cpp |
| 161 | |
| 162 | auto data = (MyData *) py::get_shared_data("mydata"); |
| 163 | if (!data) |
| 164 | data = (MyData *) py::set_shared_data("mydata", new MyData(42)); |
| 165 | |
| 166 | If the above snippet was used in several separately compiled extension modules, |
| 167 | the first one to be imported would create a ``MyData`` instance and associate |
| 168 | a ``"mydata"`` key with a pointer to it. Extensions that are imported later |
| 169 | would be then able to access the data behind the same pointer. |
| 170 | |
| 171 | .. [#f6] https://docs.python.org/3/extending/extending.html#using-capsules |
Dean Moldovan | 67b52d8 | 2016-10-16 19:12:43 +0200 | [diff] [blame] | 172 | |
Wenzel Jakob | b16421e | 2017-03-22 22:04:00 +0100 | [diff] [blame^] | 173 | Module Destructors |
| 174 | ================== |
| 175 | |
| 176 | pybind11 does not provide an explicit mechanism to invoke cleanup code at |
| 177 | module destruction time. In rare cases where such functionality is required, it |
| 178 | is possible to emulate it using Python capsules with a destruction callback. |
| 179 | |
| 180 | .. code-block:: cpp |
| 181 | |
| 182 | auto cleanup_callback = []() { |
| 183 | // perform cleanup here -- this function is called with the GIL held |
| 184 | }; |
| 185 | |
| 186 | m.add_object("_cleanup", py::capsule(cleanup_callback)); |
Dean Moldovan | 67b52d8 | 2016-10-16 19:12:43 +0200 | [diff] [blame] | 187 | |
| 188 | Generating documentation using Sphinx |
| 189 | ===================================== |
| 190 | |
| 191 | Sphinx [#f4]_ has the ability to inspect the signatures and documentation |
| 192 | strings in pybind11-based extension modules to automatically generate beautiful |
| 193 | documentation in a variety formats. The python_example repository [#f5]_ contains a |
| 194 | simple example repository which uses this approach. |
| 195 | |
| 196 | There are two potential gotchas when using this approach: first, make sure that |
| 197 | the resulting strings do not contain any :kbd:`TAB` characters, which break the |
| 198 | docstring parsing routines. You may want to use C++11 raw string literals, |
| 199 | which are convenient for multi-line comments. Conveniently, any excess |
| 200 | indentation will be automatically be removed by Sphinx. However, for this to |
| 201 | work, it is important that all lines are indented consistently, i.e.: |
| 202 | |
| 203 | .. code-block:: cpp |
| 204 | |
| 205 | // ok |
| 206 | m.def("foo", &foo, R"mydelimiter( |
| 207 | The foo function |
| 208 | |
| 209 | Parameters |
| 210 | ---------- |
| 211 | )mydelimiter"); |
| 212 | |
| 213 | // *not ok* |
| 214 | m.def("foo", &foo, R"mydelimiter(The foo function |
| 215 | |
| 216 | Parameters |
| 217 | ---------- |
| 218 | )mydelimiter"); |
| 219 | |
Alexander Stukowski | 9a110e6 | 2016-11-15 12:38:05 +0100 | [diff] [blame] | 220 | By default, pybind11 automatically generates and prepends a signature to the docstring of a function |
| 221 | registered with ``module::def()`` and ``class_::def()``. Sometimes this |
| 222 | behavior is not desirable, because you want to provide your own signature or remove |
| 223 | the docstring completely to exclude the function from the Sphinx documentation. |
| 224 | The class ``options`` allows you to selectively suppress auto-generated signatures: |
| 225 | |
| 226 | .. code-block:: cpp |
| 227 | |
| 228 | PYBIND11_PLUGIN(example) { |
| 229 | py::module m("example", "pybind11 example plugin"); |
| 230 | |
| 231 | py::options options; |
| 232 | options.disable_function_signatures(); |
| 233 | |
| 234 | m.def("add", [](int a, int b) { return a + b; }, "A function which adds two numbers"); |
| 235 | |
| 236 | return m.ptr(); |
| 237 | } |
| 238 | |
| 239 | Note that changes to the settings affect only function bindings created during the |
| 240 | lifetime of the ``options`` instance. When it goes out of scope at the end of the module's init function, |
| 241 | the default settings are restored to prevent unwanted side effects. |
| 242 | |
Dean Moldovan | 67b52d8 | 2016-10-16 19:12:43 +0200 | [diff] [blame] | 243 | .. [#f4] http://www.sphinx-doc.org |
| 244 | .. [#f5] http://github.com/pybind/python_example |