Wenzel Jakob | 28f98aa | 2015-10-13 02:57:16 +0200 | [diff] [blame] | 1 | .. _advanced: |
| 2 | |
| 3 | Advanced topics |
| 4 | ############### |
| 5 | |
Wenzel Jakob | 9329669 | 2015-10-13 23:21:54 +0200 | [diff] [blame] | 6 | For brevity, the rest of this chapter assumes that the following two lines are |
| 7 | present: |
| 8 | |
| 9 | .. code-block:: cpp |
| 10 | |
Wenzel Jakob | 8f4eb00 | 2015-10-15 18:13:33 +0200 | [diff] [blame] | 11 | #include <pybind11/pybind11.h> |
Wenzel Jakob | 9329669 | 2015-10-13 23:21:54 +0200 | [diff] [blame] | 12 | |
Wenzel Jakob | 10e62e1 | 2015-10-15 22:46:07 +0200 | [diff] [blame] | 13 | namespace py = pybind11; |
Wenzel Jakob | 9329669 | 2015-10-13 23:21:54 +0200 | [diff] [blame] | 14 | |
Wenzel Jakob | 28f98aa | 2015-10-13 02:57:16 +0200 | [diff] [blame] | 15 | Operator overloading |
| 16 | ==================== |
| 17 | |
Wenzel Jakob | 9329669 | 2015-10-13 23:21:54 +0200 | [diff] [blame] | 18 | Suppose that we're given the following ``Vector2`` class with a vector addition |
| 19 | and scalar multiplication operation, all implemented using overloaded operators |
| 20 | in C++. |
| 21 | |
| 22 | .. code-block:: cpp |
| 23 | |
| 24 | class Vector2 { |
| 25 | public: |
| 26 | Vector2(float x, float y) : x(x), y(y) { } |
| 27 | |
| 28 | std::string toString() const { return "[" + std::to_string(x) + ", " + std::to_string(y) + "]"; } |
| 29 | |
| 30 | Vector2 operator+(const Vector2 &v) const { return Vector2(x + v.x, y + v.y); } |
| 31 | Vector2 operator*(float value) const { return Vector2(x * value, y * value); } |
| 32 | Vector2& operator+=(const Vector2 &v) { x += v.x; y += v.y; return *this; } |
| 33 | Vector2& operator*=(float v) { x *= v; y *= v; return *this; } |
| 34 | |
| 35 | friend Vector2 operator*(float f, const Vector2 &v) { return Vector2(f * v.x, f * v.y); } |
| 36 | |
| 37 | private: |
| 38 | float x, y; |
| 39 | }; |
| 40 | |
| 41 | The following snippet shows how the above operators can be conveniently exposed |
| 42 | to Python. |
| 43 | |
| 44 | .. code-block:: cpp |
| 45 | |
Wenzel Jakob | 8f4eb00 | 2015-10-15 18:13:33 +0200 | [diff] [blame] | 46 | #include <pybind11/operators.h> |
Wenzel Jakob | 9329669 | 2015-10-13 23:21:54 +0200 | [diff] [blame] | 47 | |
Wenzel Jakob | b1b7140 | 2015-10-18 16:48:30 +0200 | [diff] [blame] | 48 | PYBIND11_PLUGIN(example) { |
Wenzel Jakob | 8f4eb00 | 2015-10-15 18:13:33 +0200 | [diff] [blame] | 49 | py::module m("example", "pybind11 example plugin"); |
Wenzel Jakob | 9329669 | 2015-10-13 23:21:54 +0200 | [diff] [blame] | 50 | |
| 51 | py::class_<Vector2>(m, "Vector2") |
| 52 | .def(py::init<float, float>()) |
| 53 | .def(py::self + py::self) |
| 54 | .def(py::self += py::self) |
| 55 | .def(py::self *= float()) |
| 56 | .def(float() * py::self) |
| 57 | .def("__repr__", &Vector2::toString); |
| 58 | |
| 59 | return m.ptr(); |
| 60 | } |
| 61 | |
| 62 | Note that a line like |
| 63 | |
| 64 | .. code-block:: cpp |
| 65 | |
| 66 | .def(py::self * float()) |
| 67 | |
| 68 | is really just short hand notation for |
| 69 | |
| 70 | .. code-block:: cpp |
| 71 | |
| 72 | .def("__mul__", [](const Vector2 &a, float b) { |
| 73 | return a * b; |
| 74 | }) |
| 75 | |
| 76 | This can be useful for exposing additional operators that don't exist on the |
| 77 | C++ side, or to perform other types of customization. |
| 78 | |
| 79 | .. note:: |
| 80 | |
| 81 | To use the more convenient ``py::self`` notation, the additional |
Wenzel Jakob | 8f4eb00 | 2015-10-15 18:13:33 +0200 | [diff] [blame] | 82 | header file :file:`pybind11/operators.h` must be included. |
Wenzel Jakob | 9329669 | 2015-10-13 23:21:54 +0200 | [diff] [blame] | 83 | |
| 84 | .. seealso:: |
| 85 | |
| 86 | The file :file:`example/example3.cpp` contains a complete example that |
| 87 | demonstrates how to work with overloaded operators in more detail. |
| 88 | |
| 89 | Callbacks and passing anonymous functions |
| 90 | ========================================= |
| 91 | |
| 92 | The C++11 standard brought lambda functions and the generic polymorphic |
| 93 | function wrapper ``std::function<>`` to the C++ programming language, which |
| 94 | enable powerful new ways of working with functions. Lambda functions come in |
| 95 | two flavors: stateless lambda function resemble classic function pointers that |
| 96 | link to an anonymous piece of code, while stateful lambda functions |
| 97 | additionally depend on captured variables that are stored in an anonymous |
| 98 | *lambda closure object*. |
| 99 | |
| 100 | Here is a simple example of a C++ function that takes an arbitrary function |
| 101 | (stateful or stateless) with signature ``int -> int`` as an argument and runs |
| 102 | it with the value 10. |
| 103 | |
| 104 | .. code-block:: cpp |
| 105 | |
| 106 | int func_arg(const std::function<int(int)> &f) { |
| 107 | return f(10); |
| 108 | } |
| 109 | |
| 110 | The example below is more involved: it takes a function of signature ``int -> int`` |
| 111 | and returns another function of the same kind. The return value is a stateful |
| 112 | lambda function, which stores the value ``f`` in the capture object and adds 1 to |
| 113 | its return value upon execution. |
| 114 | |
| 115 | .. code-block:: cpp |
| 116 | |
| 117 | std::function<int(int)> func_ret(const std::function<int(int)> &f) { |
| 118 | return [f](int i) { |
| 119 | return f(i) + 1; |
| 120 | }; |
| 121 | } |
| 122 | |
Wenzel Jakob | 8f4eb00 | 2015-10-15 18:13:33 +0200 | [diff] [blame] | 123 | After including the extra header file :file:`pybind11/functional.h`, it is almost |
Wenzel Jakob | 9329669 | 2015-10-13 23:21:54 +0200 | [diff] [blame] | 124 | trivial to generate binding code for both of these functions. |
| 125 | |
| 126 | .. code-block:: cpp |
| 127 | |
Wenzel Jakob | 8f4eb00 | 2015-10-15 18:13:33 +0200 | [diff] [blame] | 128 | #include <pybind11/functional.h> |
Wenzel Jakob | 9329669 | 2015-10-13 23:21:54 +0200 | [diff] [blame] | 129 | |
Wenzel Jakob | b1b7140 | 2015-10-18 16:48:30 +0200 | [diff] [blame] | 130 | PYBIND11_PLUGIN(example) { |
Wenzel Jakob | 8f4eb00 | 2015-10-15 18:13:33 +0200 | [diff] [blame] | 131 | py::module m("example", "pybind11 example plugin"); |
Wenzel Jakob | 9329669 | 2015-10-13 23:21:54 +0200 | [diff] [blame] | 132 | |
| 133 | m.def("func_arg", &func_arg); |
| 134 | m.def("func_ret", &func_ret); |
| 135 | |
| 136 | return m.ptr(); |
| 137 | } |
| 138 | |
| 139 | The following interactive session shows how to call them from Python. |
| 140 | |
| 141 | .. code-block:: python |
| 142 | |
| 143 | $ python |
| 144 | >>> import example |
| 145 | >>> def square(i): |
| 146 | ... return i * i |
| 147 | ... |
| 148 | >>> example.func_arg(square) |
| 149 | 100L |
| 150 | >>> square_plus_1 = example.func_ret(square) |
| 151 | >>> square_plus_1(4) |
| 152 | 17L |
| 153 | >>> |
| 154 | |
| 155 | .. note:: |
| 156 | |
| 157 | This functionality is very useful when generating bindings for callbacks in |
| 158 | C++ libraries (e.g. a graphical user interface library). |
| 159 | |
| 160 | The file :file:`example/example5.cpp` contains a complete example that |
| 161 | demonstrates how to work with callbacks and anonymous functions in more detail. |
| 162 | |
Wenzel Jakob | a4175d6 | 2015-11-17 08:30:34 +0100 | [diff] [blame] | 163 | .. warning:: |
| 164 | |
| 165 | Keep in mind that passing a function from C++ to Python (or vice versa) |
| 166 | will instantiate a piece of wrapper code that translates function |
| 167 | invocations between the two languages. Copying the same function back and |
| 168 | forth between Python and C++ many times in a row will cause these wrappers |
| 169 | to accumulate, which can decrease performance. |
| 170 | |
Wenzel Jakob | 28f98aa | 2015-10-13 02:57:16 +0200 | [diff] [blame] | 171 | Overriding virtual functions in Python |
| 172 | ====================================== |
| 173 | |
Wenzel Jakob | 9329669 | 2015-10-13 23:21:54 +0200 | [diff] [blame] | 174 | Suppose that a C++ class or interface has a virtual function that we'd like to |
| 175 | to override from within Python (we'll focus on the class ``Animal``; ``Dog`` is |
| 176 | given as a specific example of how one would do this with traditional C++ |
| 177 | code). |
| 178 | |
| 179 | .. code-block:: cpp |
| 180 | |
| 181 | class Animal { |
| 182 | public: |
| 183 | virtual ~Animal() { } |
| 184 | virtual std::string go(int n_times) = 0; |
| 185 | }; |
| 186 | |
| 187 | class Dog : public Animal { |
| 188 | public: |
| 189 | std::string go(int n_times) { |
| 190 | std::string result; |
| 191 | for (int i=0; i<n_times; ++i) |
| 192 | result += "woof! "; |
| 193 | return result; |
| 194 | } |
| 195 | }; |
| 196 | |
| 197 | Let's also suppose that we are given a plain function which calls the |
| 198 | function ``go()`` on an arbitrary ``Animal`` instance. |
| 199 | |
| 200 | .. code-block:: cpp |
| 201 | |
| 202 | std::string call_go(Animal *animal) { |
| 203 | return animal->go(3); |
| 204 | } |
| 205 | |
| 206 | Normally, the binding code for these classes would look as follows: |
| 207 | |
| 208 | .. code-block:: cpp |
| 209 | |
Wenzel Jakob | b1b7140 | 2015-10-18 16:48:30 +0200 | [diff] [blame] | 210 | PYBIND11_PLUGIN(example) { |
Wenzel Jakob | 8f4eb00 | 2015-10-15 18:13:33 +0200 | [diff] [blame] | 211 | py::module m("example", "pybind11 example plugin"); |
Wenzel Jakob | 9329669 | 2015-10-13 23:21:54 +0200 | [diff] [blame] | 212 | |
| 213 | py::class_<Animal> animal(m, "Animal"); |
| 214 | animal |
| 215 | .def("go", &Animal::go); |
| 216 | |
| 217 | py::class_<Dog>(m, "Dog", animal) |
| 218 | .def(py::init<>()); |
| 219 | |
| 220 | m.def("call_go", &call_go); |
| 221 | |
| 222 | return m.ptr(); |
| 223 | } |
| 224 | |
| 225 | However, these bindings are impossible to extend: ``Animal`` is not |
| 226 | constructible, and we clearly require some kind of "trampoline" that |
| 227 | redirects virtual calls back to Python. |
| 228 | |
| 229 | Defining a new type of ``Animal`` from within Python is possible but requires a |
| 230 | helper class that is defined as follows: |
| 231 | |
| 232 | .. code-block:: cpp |
| 233 | |
| 234 | class PyAnimal : public Animal { |
| 235 | public: |
| 236 | /* Inherit the constructors */ |
| 237 | using Animal::Animal; |
| 238 | |
| 239 | /* Trampoline (need one for each virtual function) */ |
| 240 | std::string go(int n_times) { |
Wenzel Jakob | b1b7140 | 2015-10-18 16:48:30 +0200 | [diff] [blame] | 241 | PYBIND11_OVERLOAD_PURE( |
Wenzel Jakob | 9329669 | 2015-10-13 23:21:54 +0200 | [diff] [blame] | 242 | std::string, /* Return type */ |
| 243 | Animal, /* Parent class */ |
| 244 | go, /* Name of function */ |
| 245 | n_times /* Argument(s) */ |
| 246 | ); |
| 247 | } |
| 248 | }; |
| 249 | |
Wenzel Jakob | b1b7140 | 2015-10-18 16:48:30 +0200 | [diff] [blame] | 250 | The macro :func:`PYBIND11_OVERLOAD_PURE` should be used for pure virtual |
| 251 | functions, and :func:`PYBIND11_OVERLOAD` should be used for functions which have |
Wenzel Jakob | 9329669 | 2015-10-13 23:21:54 +0200 | [diff] [blame] | 252 | a default implementation. The binding code also needs a few minor adaptations |
| 253 | (highlighted): |
| 254 | |
| 255 | .. code-block:: cpp |
| 256 | :emphasize-lines: 4,6,7 |
| 257 | |
Wenzel Jakob | b1b7140 | 2015-10-18 16:48:30 +0200 | [diff] [blame] | 258 | PYBIND11_PLUGIN(example) { |
Wenzel Jakob | 8f4eb00 | 2015-10-15 18:13:33 +0200 | [diff] [blame] | 259 | py::module m("example", "pybind11 example plugin"); |
Wenzel Jakob | 9329669 | 2015-10-13 23:21:54 +0200 | [diff] [blame] | 260 | |
| 261 | py::class_<PyAnimal> animal(m, "Animal"); |
| 262 | animal |
| 263 | .alias<Animal>() |
| 264 | .def(py::init<>()) |
| 265 | .def("go", &Animal::go); |
| 266 | |
| 267 | py::class_<Dog>(m, "Dog", animal) |
| 268 | .def(py::init<>()); |
| 269 | |
| 270 | m.def("call_go", &call_go); |
| 271 | |
| 272 | return m.ptr(); |
| 273 | } |
| 274 | |
| 275 | Importantly, the trampoline helper class is used as the template argument to |
| 276 | :class:`class_`, and a call to :func:`class_::alias` informs the binding |
| 277 | generator that this is merely an alias for the underlying type ``Animal``. |
| 278 | Following this, we are able to define a constructor as usual. |
| 279 | |
| 280 | The Python session below shows how to override ``Animal::go`` and invoke it via |
| 281 | a virtual method call. |
| 282 | |
| 283 | .. code-block:: cpp |
| 284 | |
| 285 | >>> from example import * |
| 286 | >>> d = Dog() |
| 287 | >>> call_go(d) |
| 288 | u'woof! woof! woof! ' |
| 289 | >>> class Cat(Animal): |
| 290 | ... def go(self, n_times): |
| 291 | ... return "meow! " * n_times |
| 292 | ... |
| 293 | >>> c = Cat() |
| 294 | >>> call_go(c) |
| 295 | u'meow! meow! meow! ' |
| 296 | |
| 297 | .. seealso:: |
| 298 | |
| 299 | The file :file:`example/example12.cpp` contains a complete example that |
| 300 | demonstrates how to override virtual functions using pybind11 in more |
| 301 | detail. |
| 302 | |
Wenzel Jakob | ecdd868 | 2015-12-07 18:17:58 +0100 | [diff] [blame] | 303 | |
| 304 | Global Interpreter Lock (GIL) |
| 305 | ============================= |
| 306 | |
| 307 | The classes :class:`gil_scoped_release` and :class:`gil_scoped_acquire` can be |
| 308 | used to acquire and release the global interpreter lock in the body of a C++ |
| 309 | function call. In this way, long-running C++ code can be parallelized using |
| 310 | multiple Python threads. Taking the previous section as an example, this could |
| 311 | be realized as follows (important changes highlighted): |
| 312 | |
| 313 | .. code-block:: cpp |
| 314 | :emphasize-lines: 8,9,33,34 |
| 315 | |
| 316 | class PyAnimal : public Animal { |
| 317 | public: |
| 318 | /* Inherit the constructors */ |
| 319 | using Animal::Animal; |
| 320 | |
| 321 | /* Trampoline (need one for each virtual function) */ |
| 322 | std::string go(int n_times) { |
| 323 | /* Acquire GIL before calling Python code */ |
| 324 | gil_scoped_acquire acquire; |
| 325 | |
| 326 | PYBIND11_OVERLOAD_PURE( |
| 327 | std::string, /* Return type */ |
| 328 | Animal, /* Parent class */ |
| 329 | go, /* Name of function */ |
| 330 | n_times /* Argument(s) */ |
| 331 | ); |
| 332 | } |
| 333 | }; |
| 334 | |
| 335 | PYBIND11_PLUGIN(example) { |
| 336 | py::module m("example", "pybind11 example plugin"); |
| 337 | |
| 338 | py::class_<PyAnimal> animal(m, "Animal"); |
| 339 | animal |
| 340 | .alias<Animal>() |
| 341 | .def(py::init<>()) |
| 342 | .def("go", &Animal::go); |
| 343 | |
| 344 | py::class_<Dog>(m, "Dog", animal) |
| 345 | .def(py::init<>()); |
| 346 | |
| 347 | m.def("call_go", [](Animal *animal) -> std::string { |
| 348 | /* Release GIL before calling into (potentially long-running) C++ code */ |
| 349 | gil_scoped_release release; |
| 350 | return call_go(animal); |
| 351 | }); |
| 352 | |
| 353 | return m.ptr(); |
| 354 | } |
| 355 | |
Wenzel Jakob | 9329669 | 2015-10-13 23:21:54 +0200 | [diff] [blame] | 356 | Passing STL data structures |
Wenzel Jakob | 28f98aa | 2015-10-13 02:57:16 +0200 | [diff] [blame] | 357 | =========================== |
| 358 | |
Wenzel Jakob | 8f4eb00 | 2015-10-15 18:13:33 +0200 | [diff] [blame] | 359 | When including the additional header file :file:`pybind11/stl.h`, conversions |
Wenzel Jakob | 9329669 | 2015-10-13 23:21:54 +0200 | [diff] [blame] | 360 | between ``std::vector<>`` and ``std::map<>`` and the Python ``list`` and |
| 361 | ``dict`` data structures are automatically enabled. The types ``std::pair<>`` |
| 362 | and ``std::tuple<>`` are already supported out of the box with just the core |
Wenzel Jakob | 8f4eb00 | 2015-10-15 18:13:33 +0200 | [diff] [blame] | 363 | :file:`pybind11/pybind11.h` header. |
Wenzel Jakob | 9329669 | 2015-10-13 23:21:54 +0200 | [diff] [blame] | 364 | |
| 365 | .. note:: |
| 366 | |
| 367 | Arbitrary nesting of any of these types is explicitly permitted. |
| 368 | |
| 369 | .. seealso:: |
| 370 | |
| 371 | The file :file:`example/example2.cpp` contains a complete example that |
| 372 | demonstrates how to pass STL data types in more detail. |
| 373 | |
| 374 | Binding sequence data types, the slicing protocol, etc. |
| 375 | ======================================================= |
| 376 | |
| 377 | Please refer to the supplemental example for details. |
| 378 | |
| 379 | .. seealso:: |
| 380 | |
| 381 | The file :file:`example/example6.cpp` contains a complete example that |
| 382 | shows how to bind a sequence data type, including length queries |
| 383 | (``__len__``), iterators (``__iter__``), the slicing protocol and other |
| 384 | kinds of useful operations. |
| 385 | |
Wenzel Jakob | 28f98aa | 2015-10-13 02:57:16 +0200 | [diff] [blame] | 386 | Return value policies |
| 387 | ===================== |
| 388 | |
Wenzel Jakob | 9329669 | 2015-10-13 23:21:54 +0200 | [diff] [blame] | 389 | Python and C++ use wildly different ways of managing the memory and lifetime of |
| 390 | objects managed by them. This can lead to issues when creating bindings for |
| 391 | functions that return a non-trivial type. Just by looking at the type |
| 392 | information, it is not clear whether Python should take charge of the returned |
| 393 | value and eventually free its resources, or if this is handled on the C++ side. |
| 394 | For this reason, pybind11 provides a several `return value policy` annotations |
| 395 | that can be passed to the :func:`module::def` and :func:`class_::def` |
| 396 | functions. The default policy is :enum:`return_value_policy::automatic``. |
Wenzel Jakob | 28f98aa | 2015-10-13 02:57:16 +0200 | [diff] [blame] | 397 | |
Wenzel Jakob | 9329669 | 2015-10-13 23:21:54 +0200 | [diff] [blame] | 398 | |
| 399 | +--------------------------------------------------+---------------------------------------------------------------------------+ |
| 400 | | Return value policy | Description | |
| 401 | +==================================================+===========================================================================+ |
| 402 | | :enum:`return_value_policy::automatic` | Automatic: copy objects returned as values and take ownership of | |
| 403 | | | objects returned as pointers | |
| 404 | +--------------------------------------------------+---------------------------------------------------------------------------+ |
| 405 | | :enum:`return_value_policy::copy` | Create a new copy of the returned object, which will be owned by Python | |
| 406 | +--------------------------------------------------+---------------------------------------------------------------------------+ |
| 407 | | :enum:`return_value_policy::take_ownership` | Reference the existing object and take ownership. Python will call | |
| 408 | | | the destructor and delete operator when the reference count reaches zero | |
| 409 | +--------------------------------------------------+---------------------------------------------------------------------------+ |
| 410 | | :enum:`return_value_policy::reference` | Reference the object, but do not take ownership and defer responsibility | |
| 411 | | | for deleting it to C++ (dangerous when C++ code at some point decides to | |
| 412 | | | delete it while Python still has a nonzero reference count) | |
| 413 | +--------------------------------------------------+---------------------------------------------------------------------------+ |
| 414 | | :enum:`return_value_policy::reference_internal` | Reference the object, but do not take ownership. The object is considered | |
| 415 | | | be owned by the C++ instance whose method or property returned it. The | |
| 416 | | | Python object will increase the reference count of this 'parent' by 1 | |
| 417 | | | to ensure that it won't be deallocated while Python is using the 'child' | |
| 418 | +--------------------------------------------------+---------------------------------------------------------------------------+ |
| 419 | |
| 420 | .. warning:: |
| 421 | |
| 422 | Code with invalid call policies might access unitialized memory and free |
| 423 | data structures multiple times, which can lead to hard-to-debug |
| 424 | non-determinism and segmentation faults, hence it is worth spending the |
| 425 | time to understand all the different options above. |
| 426 | |
| 427 | See below for an example that uses the |
| 428 | :enum:`return_value_policy::reference_internal` policy. |
| 429 | |
| 430 | .. code-block:: cpp |
| 431 | |
| 432 | class Example { |
| 433 | public: |
| 434 | Internal &get_internal() { return internal; } |
| 435 | private: |
| 436 | Internal internal; |
| 437 | }; |
| 438 | |
Wenzel Jakob | b1b7140 | 2015-10-18 16:48:30 +0200 | [diff] [blame] | 439 | PYBIND11_PLUGIN(example) { |
Wenzel Jakob | 8f4eb00 | 2015-10-15 18:13:33 +0200 | [diff] [blame] | 440 | py::module m("example", "pybind11 example plugin"); |
Wenzel Jakob | 9329669 | 2015-10-13 23:21:54 +0200 | [diff] [blame] | 441 | |
| 442 | py::class_<Example>(m, "Example") |
| 443 | .def(py::init<>()) |
| 444 | .def("get_internal", &Example::get_internal, "Return the internal data", py::return_value_policy::reference_internal) |
| 445 | |
| 446 | return m.ptr(); |
| 447 | } |
| 448 | |
| 449 | Implicit type conversions |
| 450 | ========================= |
| 451 | |
| 452 | Suppose that instances of two types ``A`` and ``B`` are used in a project, and |
| 453 | that an ``A`` can easily be converted into a an instance of type ``B`` (examples of this |
| 454 | could be a fixed and an arbitrary precision number type). |
| 455 | |
| 456 | .. code-block:: cpp |
| 457 | |
| 458 | py::class_<A>(m, "A") |
| 459 | /// ... members ... |
| 460 | |
| 461 | py::class_<B>(m, "B") |
| 462 | .def(py::init<A>()) |
| 463 | /// ... members ... |
| 464 | |
| 465 | m.def("func", |
| 466 | [](const B &) { /* .... */ } |
| 467 | ); |
| 468 | |
| 469 | To invoke the function ``func`` using a variable ``a`` containing an ``A`` |
| 470 | instance, we'd have to write ``func(B(a))`` in Python. On the other hand, C++ |
| 471 | will automatically apply an implicit type conversion, which makes it possible |
| 472 | to directly write ``func(a)``. |
| 473 | |
| 474 | In this situation (i.e. where ``B`` has a constructor that converts from |
| 475 | ``A``), the following statement enables similar implicit conversions on the |
| 476 | Python side: |
| 477 | |
| 478 | .. code-block:: cpp |
| 479 | |
| 480 | py::implicitly_convertible<A, B>(); |
| 481 | |
| 482 | Smart pointers |
| 483 | ============== |
| 484 | |
| 485 | The binding generator for classes (:class:`class_`) takes an optional second |
| 486 | template type, which denotes a special *holder* type that is used to manage |
| 487 | references to the object. When wrapping a type named ``Type``, the default |
| 488 | value of this template parameter is ``std::unique_ptr<Type>``, which means that |
| 489 | the object is deallocated when Python's reference count goes to zero. |
| 490 | |
Wenzel Jakob | 1853b65 | 2015-10-18 15:38:50 +0200 | [diff] [blame] | 491 | It is possible to switch to other types of reference counting wrappers or smart |
| 492 | pointers, which is useful in codebases that rely on them. For instance, the |
| 493 | following snippet causes ``std::shared_ptr`` to be used instead. |
Wenzel Jakob | 9329669 | 2015-10-13 23:21:54 +0200 | [diff] [blame] | 494 | |
| 495 | .. code-block:: cpp |
| 496 | |
| 497 | py::class_<Example, std::shared_ptr<Example>> obj(m, "Example"); |
| 498 | |
Wenzel Jakob | 1853b65 | 2015-10-18 15:38:50 +0200 | [diff] [blame] | 499 | To enable transparent conversions for functions that take shared pointers as an |
| 500 | argument or that return them, a macro invocation similar to the following must |
| 501 | be declared at the top level before any binding code: |
| 502 | |
| 503 | .. code-block:: cpp |
| 504 | |
Wenzel Jakob | b1b7140 | 2015-10-18 16:48:30 +0200 | [diff] [blame] | 505 | PYBIND11_DECLARE_HOLDER_TYPE(T, std::shared_ptr<T>); |
Wenzel Jakob | 1853b65 | 2015-10-18 15:38:50 +0200 | [diff] [blame] | 506 | |
Wenzel Jakob | 9329669 | 2015-10-13 23:21:54 +0200 | [diff] [blame] | 507 | .. seealso:: |
| 508 | |
| 509 | The file :file:`example/example8.cpp` contains a complete example that |
Wenzel Jakob | 1853b65 | 2015-10-18 15:38:50 +0200 | [diff] [blame] | 510 | demonstrates how to work with custom reference-counting holder types in |
| 511 | more detail. |
Wenzel Jakob | 9329669 | 2015-10-13 23:21:54 +0200 | [diff] [blame] | 512 | |
Wenzel Jakob | 6e213c9 | 2015-11-24 23:05:58 +0100 | [diff] [blame] | 513 | .. warning:: |
| 514 | |
| 515 | To ensure correct reference counting among Python and C++, the use of |
| 516 | ``std::shared_ptr<T>`` as a holder type requires that ``T`` inherits from |
| 517 | ``std::enable_shared_from_this<T>`` (see cppreference_ for details). |
| 518 | |
| 519 | .. _cppreference: http://en.cppreference.com/w/cpp/memory/enable_shared_from_this |
| 520 | |
Wenzel Jakob | 9329669 | 2015-10-13 23:21:54 +0200 | [diff] [blame] | 521 | .. _custom_constructors: |
| 522 | |
| 523 | Custom constructors |
| 524 | =================== |
| 525 | |
| 526 | The syntax for binding constructors was previously introduced, but it only |
| 527 | works when a constructor with the given parameters actually exists on the C++ |
| 528 | side. To extend this to more general cases, let's take a look at what actually |
| 529 | happens under the hood: the following statement |
| 530 | |
| 531 | .. code-block:: cpp |
| 532 | |
| 533 | py::class_<Example>(m, "Example") |
| 534 | .def(py::init<int>()); |
| 535 | |
| 536 | is short hand notation for |
| 537 | |
| 538 | .. code-block:: cpp |
| 539 | |
| 540 | py::class_<Example>(m, "Example") |
| 541 | .def("__init__", |
| 542 | [](Example &instance, int arg) { |
| 543 | new (&instance) Example(arg); |
| 544 | } |
| 545 | ); |
| 546 | |
| 547 | In other words, :func:`init` creates an anonymous function that invokes an |
| 548 | in-place constructor. Memory allocation etc. is already take care of beforehand |
| 549 | within pybind11. |
| 550 | |
| 551 | Catching and throwing exceptions |
| 552 | ================================ |
| 553 | |
| 554 | When C++ code invoked from Python throws an ``std::exception``, it is |
| 555 | automatically converted into a Python ``Exception``. pybind11 defines multiple |
| 556 | special exception classes that will map to different types of Python |
| 557 | exceptions: |
| 558 | |
| 559 | +----------------------------+------------------------------+ |
| 560 | | C++ exception type | Python exception type | |
| 561 | +============================+==============================+ |
| 562 | | :class:`std::exception` | ``Exception`` | |
| 563 | +----------------------------+------------------------------+ |
| 564 | | :class:`stop_iteration` | ``StopIteration`` (used to | |
| 565 | | | implement custom iterators) | |
| 566 | +----------------------------+------------------------------+ |
| 567 | | :class:`index_error` | ``IndexError`` (used to | |
| 568 | | | indicate out of bounds | |
| 569 | | | accesses in ``__getitem__``, | |
| 570 | | | ``__setitem__``, etc.) | |
| 571 | +----------------------------+------------------------------+ |
| 572 | | :class:`error_already_set` | Indicates that the Python | |
| 573 | | | exception flag has already | |
| 574 | | | been initialized. | |
| 575 | +----------------------------+------------------------------+ |
| 576 | |
| 577 | When a Python function invoked from C++ throws an exception, it is converted |
| 578 | into a C++ exception of type :class:`error_already_set` whose string payload |
| 579 | contains a textual summary. |
| 580 | |
| 581 | There is also a special exception :class:`cast_error` that is thrown by |
| 582 | :func:`handle::call` when the input arguments cannot be converted to Python |
| 583 | objects. |
Wenzel Jakob | 28f98aa | 2015-10-13 02:57:16 +0200 | [diff] [blame] | 584 | |
| 585 | Buffer protocol |
| 586 | =============== |
| 587 | |
| 588 | Python supports an extremely general and convenient approach for exchanging |
| 589 | data between plugin libraries. Types can expose a buffer view which provides |
| 590 | fast direct access to the raw internal representation. Suppose we want to bind |
| 591 | the following simplistic Matrix class: |
| 592 | |
| 593 | .. code-block:: cpp |
| 594 | |
| 595 | class Matrix { |
| 596 | public: |
| 597 | Matrix(size_t rows, size_t cols) : m_rows(rows), m_cols(cols) { |
| 598 | m_data = new float[rows*cols]; |
| 599 | } |
| 600 | float *data() { return m_data; } |
| 601 | size_t rows() const { return m_rows; } |
| 602 | size_t cols() const { return m_cols; } |
| 603 | private: |
| 604 | size_t m_rows, m_cols; |
| 605 | float *m_data; |
| 606 | }; |
| 607 | |
| 608 | The following binding code exposes the ``Matrix`` contents as a buffer object, |
| 609 | making it possible to cast Matrixes into NumPy arrays. It is even possible to |
| 610 | completely avoid copy operations with Python expressions like |
| 611 | ``np.array(matrix_instance, copy = False)``. |
| 612 | |
| 613 | .. code-block:: cpp |
| 614 | |
| 615 | py::class_<Matrix>(m, "Matrix") |
| 616 | .def_buffer([](Matrix &m) -> py::buffer_info { |
| 617 | return py::buffer_info( |
| 618 | m.data(), /* Pointer to buffer */ |
| 619 | sizeof(float), /* Size of one scalar */ |
| 620 | py::format_descriptor<float>::value(), /* Python struct-style format descriptor */ |
| 621 | 2, /* Number of dimensions */ |
| 622 | { m.rows(), m.cols() }, /* Buffer dimensions */ |
| 623 | { sizeof(float) * m.rows(), /* Strides (in bytes) for each index */ |
| 624 | sizeof(float) } |
| 625 | ); |
| 626 | }); |
| 627 | |
| 628 | The snippet above binds a lambda function, which can create ``py::buffer_info`` |
| 629 | description records on demand describing a given matrix. The contents of |
| 630 | ``py::buffer_info`` mirror the Python buffer protocol specification. |
| 631 | |
| 632 | .. code-block:: cpp |
| 633 | |
| 634 | struct buffer_info { |
| 635 | void *ptr; |
| 636 | size_t itemsize; |
| 637 | std::string format; |
| 638 | int ndim; |
| 639 | std::vector<size_t> shape; |
| 640 | std::vector<size_t> strides; |
| 641 | }; |
| 642 | |
| 643 | To create a C++ function that can take a Python buffer object as an argument, |
| 644 | simply use the type ``py::buffer`` as one of its arguments. Buffers can exist |
| 645 | in a great variety of configurations, hence some safety checks are usually |
| 646 | necessary in the function body. Below, you can see an basic example on how to |
| 647 | define a custom constructor for the Eigen double precision matrix |
| 648 | (``Eigen::MatrixXd``) type, which supports initialization from compatible |
| 649 | buffer |
| 650 | objects (e.g. a NumPy matrix). |
| 651 | |
| 652 | .. code-block:: cpp |
| 653 | |
| 654 | py::class_<Eigen::MatrixXd>(m, "MatrixXd") |
| 655 | .def("__init__", [](Eigen::MatrixXd &m, py::buffer b) { |
| 656 | /* Request a buffer descriptor from Python */ |
| 657 | py::buffer_info info = b.request(); |
| 658 | |
| 659 | /* Some sanity checks ... */ |
| 660 | if (info.format != py::format_descriptor<double>::value()) |
| 661 | throw std::runtime_error("Incompatible format: expected a double array!"); |
| 662 | |
| 663 | if (info.ndim != 2) |
| 664 | throw std::runtime_error("Incompatible buffer dimension!"); |
| 665 | |
| 666 | if (info.strides[0] == sizeof(double)) { |
| 667 | /* Buffer has the right layout -- directly copy. */ |
| 668 | new (&m) Eigen::MatrixXd(info.shape[0], info.shape[1]); |
| 669 | memcpy(m.data(), info.ptr, sizeof(double) * m.size()); |
| 670 | } else { |
| 671 | /* Oops -- the buffer is transposed */ |
| 672 | new (&m) Eigen::MatrixXd(info.shape[1], info.shape[0]); |
| 673 | memcpy(m.data(), info.ptr, sizeof(double) * m.size()); |
| 674 | m.transposeInPlace(); |
| 675 | } |
| 676 | }); |
| 677 | |
Wenzel Jakob | 9329669 | 2015-10-13 23:21:54 +0200 | [diff] [blame] | 678 | .. seealso:: |
| 679 | |
| 680 | The file :file:`example/example7.cpp` contains a complete example that |
| 681 | demonstrates using the buffer protocol with pybind11 in more detail. |
| 682 | |
Wenzel Jakob | 28f98aa | 2015-10-13 02:57:16 +0200 | [diff] [blame] | 683 | NumPy support |
| 684 | ============= |
| 685 | |
| 686 | By exchanging ``py::buffer`` with ``py::array`` in the above snippet, we can |
| 687 | restrict the function so that it only accepts NumPy arrays (rather than any |
| 688 | type of Python object satisfying the buffer object protocol). |
| 689 | |
| 690 | In many situations, we want to define a function which only accepts a NumPy |
Wenzel Jakob | 9329669 | 2015-10-13 23:21:54 +0200 | [diff] [blame] | 691 | array of a certain data type. This is possible via the ``py::array_t<T>`` |
Wenzel Jakob | 28f98aa | 2015-10-13 02:57:16 +0200 | [diff] [blame] | 692 | template. For instance, the following function requires the argument to be a |
| 693 | dense array of doubles in C-style ordering. |
| 694 | |
| 695 | .. code-block:: cpp |
| 696 | |
Wenzel Jakob | 9329669 | 2015-10-13 23:21:54 +0200 | [diff] [blame] | 697 | void f(py::array_t<double> array); |
Wenzel Jakob | 28f98aa | 2015-10-13 02:57:16 +0200 | [diff] [blame] | 698 | |
| 699 | When it is invoked with a different type (e.g. an integer), the binding code |
| 700 | will attempt to cast the input into a NumPy array of the requested type. |
Wenzel Jakob | 8f4eb00 | 2015-10-15 18:13:33 +0200 | [diff] [blame] | 701 | Note that this feature requires the ``pybind11/numpy.h`` header to be included. |
Wenzel Jakob | 28f98aa | 2015-10-13 02:57:16 +0200 | [diff] [blame] | 702 | |
| 703 | Vectorizing functions |
| 704 | ===================== |
| 705 | |
| 706 | Suppose we want to bind a function with the following signature to Python so |
| 707 | that it can process arbitrary NumPy array arguments (vectors, matrices, general |
| 708 | N-D arrays) in addition to its normal arguments: |
| 709 | |
| 710 | .. code-block:: cpp |
| 711 | |
| 712 | double my_func(int x, float y, double z); |
| 713 | |
Wenzel Jakob | 8f4eb00 | 2015-10-15 18:13:33 +0200 | [diff] [blame] | 714 | After including the ``pybind11/numpy.h`` header, this is extremely simple: |
Wenzel Jakob | 28f98aa | 2015-10-13 02:57:16 +0200 | [diff] [blame] | 715 | |
| 716 | .. code-block:: cpp |
| 717 | |
| 718 | m.def("vectorized_func", py::vectorize(my_func)); |
| 719 | |
| 720 | Invoking the function like below causes 4 calls to be made to ``my_func`` with |
| 721 | each of the the array elements. The result is returned as a NumPy array of type |
| 722 | ``numpy.dtype.float64``. |
| 723 | |
| 724 | .. code-block:: python |
| 725 | |
| 726 | >>> x = np.array([[1, 3],[5, 7]]) |
| 727 | >>> y = np.array([[2, 4],[6, 8]]) |
| 728 | >>> z = 3 |
| 729 | >>> result = vectorized_func(x, y, z) |
| 730 | |
| 731 | The scalar argument ``z`` is transparently replicated 4 times. The input |
| 732 | arrays ``x`` and ``y`` are automatically converted into the right types (they |
| 733 | are of type ``numpy.dtype.int64`` but need to be ``numpy.dtype.int32`` and |
| 734 | ``numpy.dtype.float32``, respectively) |
| 735 | |
| 736 | Sometimes we might want to explitly exclude an argument from the vectorization |
| 737 | because it makes little sense to wrap it in a NumPy array. For instance, |
| 738 | suppose the function signature was |
| 739 | |
| 740 | .. code-block:: cpp |
| 741 | |
| 742 | double my_func(int x, float y, my_custom_type *z); |
| 743 | |
| 744 | This can be done with a stateful Lambda closure: |
| 745 | |
| 746 | .. code-block:: cpp |
| 747 | |
| 748 | // Vectorize a lambda function with a capture object (e.g. to exclude some arguments from the vectorization) |
| 749 | m.def("vectorized_func", |
Wenzel Jakob | 9329669 | 2015-10-13 23:21:54 +0200 | [diff] [blame] | 750 | [](py::array_t<int> x, py::array_t<float> y, my_custom_type *z) { |
Wenzel Jakob | 28f98aa | 2015-10-13 02:57:16 +0200 | [diff] [blame] | 751 | auto stateful_closure = [z](int x, float y) { return my_func(x, y, z); }; |
| 752 | return py::vectorize(stateful_closure)(x, y); |
| 753 | } |
| 754 | ); |
| 755 | |
Wenzel Jakob | 9329669 | 2015-10-13 23:21:54 +0200 | [diff] [blame] | 756 | .. seealso:: |
Wenzel Jakob | 28f98aa | 2015-10-13 02:57:16 +0200 | [diff] [blame] | 757 | |
Wenzel Jakob | 9329669 | 2015-10-13 23:21:54 +0200 | [diff] [blame] | 758 | The file :file:`example/example10.cpp` contains a complete example that |
| 759 | demonstrates using :func:`vectorize` in more detail. |
Wenzel Jakob | 28f98aa | 2015-10-13 02:57:16 +0200 | [diff] [blame] | 760 | |
Wenzel Jakob | 9329669 | 2015-10-13 23:21:54 +0200 | [diff] [blame] | 761 | Functions taking Python objects as arguments |
| 762 | ============================================ |
Wenzel Jakob | 28f98aa | 2015-10-13 02:57:16 +0200 | [diff] [blame] | 763 | |
Wenzel Jakob | 9329669 | 2015-10-13 23:21:54 +0200 | [diff] [blame] | 764 | pybind11 exposes all major Python types using thin C++ wrapper classes. These |
| 765 | wrapper classes can also be used as parameters of functions in bindings, which |
| 766 | makes it possible to directly work with native Python types on the C++ side. |
| 767 | For instance, the following statement iterates over a Python ``dict``: |
Wenzel Jakob | 28f98aa | 2015-10-13 02:57:16 +0200 | [diff] [blame] | 768 | |
Wenzel Jakob | 9329669 | 2015-10-13 23:21:54 +0200 | [diff] [blame] | 769 | .. code-block:: cpp |
| 770 | |
| 771 | void print_dict(py::dict dict) { |
| 772 | /* Easily interact with Python types */ |
| 773 | for (auto item : dict) |
| 774 | std::cout << "key=" << item.first << ", " |
| 775 | << "value=" << item.second << std::endl; |
| 776 | } |
| 777 | |
| 778 | Available types include :class:`handle`, :class:`object`, :class:`bool_`, |
| 779 | :class:`int_`, :class:`float_`, :class:`str`, :class:`tuple`, :class:`list`, |
| 780 | :class:`dict`, :class:`slice`, :class:`capsule`, :class:`function`, |
| 781 | :class:`buffer`, :class:`array`, and :class:`array_t`. |
| 782 | |
Wenzel Jakob | 436b731 | 2015-10-20 01:04:30 +0200 | [diff] [blame] | 783 | In this kind of mixed code, it is often necessary to convert arbitrary C++ |
| 784 | types to Python, which can be done using :func:`cast`: |
| 785 | |
| 786 | .. code-block:: cpp |
| 787 | |
| 788 | MyClass *cls = ..; |
| 789 | py::object obj = py::cast(cls); |
| 790 | |
| 791 | The reverse direction uses the following syntax: |
| 792 | |
| 793 | .. code-block:: cpp |
| 794 | |
| 795 | py::object obj = ...; |
| 796 | MyClass *cls = obj.cast<MyClass *>(); |
| 797 | |
| 798 | When conversion fails, both directions throw the exception :class:`cast_error`. |
| 799 | |
Wenzel Jakob | 9329669 | 2015-10-13 23:21:54 +0200 | [diff] [blame] | 800 | .. seealso:: |
| 801 | |
| 802 | The file :file:`example/example2.cpp` contains a complete example that |
| 803 | demonstrates passing native Python types in more detail. |