blob: f8ed303c90b21823336add000cd57afb009aa770 [file] [log] [blame]
Wenzel Jakob28f98aa2015-10-13 02:57:16 +02001.. _advanced:
2
3Advanced topics
4###############
5
Wenzel Jakob93296692015-10-13 23:21:54 +02006For brevity, the rest of this chapter assumes that the following two lines are
7present:
8
9.. code-block:: cpp
10
Wenzel Jakob8f4eb002015-10-15 18:13:33 +020011 #include <pybind11/pybind11.h>
Wenzel Jakob93296692015-10-13 23:21:54 +020012
Wenzel Jakob10e62e12015-10-15 22:46:07 +020013 namespace py = pybind11;
Wenzel Jakob93296692015-10-13 23:21:54 +020014
Wenzel Jakob28f98aa2015-10-13 02:57:16 +020015Operator overloading
16====================
17
Wenzel Jakob93296692015-10-13 23:21:54 +020018Suppose that we're given the following ``Vector2`` class with a vector addition
19and scalar multiplication operation, all implemented using overloaded operators
20in C++.
21
22.. code-block:: cpp
23
24 class Vector2 {
25 public:
26 Vector2(float x, float y) : x(x), y(y) { }
27
28 std::string toString() const { return "[" + std::to_string(x) + ", " + std::to_string(y) + "]"; }
29
30 Vector2 operator+(const Vector2 &v) const { return Vector2(x + v.x, y + v.y); }
31 Vector2 operator*(float value) const { return Vector2(x * value, y * value); }
32 Vector2& operator+=(const Vector2 &v) { x += v.x; y += v.y; return *this; }
33 Vector2& operator*=(float v) { x *= v; y *= v; return *this; }
34
35 friend Vector2 operator*(float f, const Vector2 &v) { return Vector2(f * v.x, f * v.y); }
36
37 private:
38 float x, y;
39 };
40
41The following snippet shows how the above operators can be conveniently exposed
42to Python.
43
44.. code-block:: cpp
45
Wenzel Jakob8f4eb002015-10-15 18:13:33 +020046 #include <pybind11/operators.h>
Wenzel Jakob93296692015-10-13 23:21:54 +020047
Wenzel Jakobb1b71402015-10-18 16:48:30 +020048 PYBIND11_PLUGIN(example) {
Wenzel Jakob8f4eb002015-10-15 18:13:33 +020049 py::module m("example", "pybind11 example plugin");
Wenzel Jakob93296692015-10-13 23:21:54 +020050
51 py::class_<Vector2>(m, "Vector2")
52 .def(py::init<float, float>())
53 .def(py::self + py::self)
54 .def(py::self += py::self)
55 .def(py::self *= float())
56 .def(float() * py::self)
57 .def("__repr__", &Vector2::toString);
58
59 return m.ptr();
60 }
61
62Note that a line like
63
64.. code-block:: cpp
65
66 .def(py::self * float())
67
68is really just short hand notation for
69
70.. code-block:: cpp
71
72 .def("__mul__", [](const Vector2 &a, float b) {
73 return a * b;
74 })
75
76This can be useful for exposing additional operators that don't exist on the
77C++ side, or to perform other types of customization.
78
79.. note::
80
81 To use the more convenient ``py::self`` notation, the additional
Wenzel Jakob8f4eb002015-10-15 18:13:33 +020082 header file :file:`pybind11/operators.h` must be included.
Wenzel Jakob93296692015-10-13 23:21:54 +020083
84.. seealso::
85
86 The file :file:`example/example3.cpp` contains a complete example that
87 demonstrates how to work with overloaded operators in more detail.
88
89Callbacks and passing anonymous functions
90=========================================
91
92The C++11 standard brought lambda functions and the generic polymorphic
93function wrapper ``std::function<>`` to the C++ programming language, which
94enable powerful new ways of working with functions. Lambda functions come in
95two flavors: stateless lambda function resemble classic function pointers that
96link to an anonymous piece of code, while stateful lambda functions
97additionally depend on captured variables that are stored in an anonymous
98*lambda closure object*.
99
100Here is a simple example of a C++ function that takes an arbitrary function
101(stateful or stateless) with signature ``int -> int`` as an argument and runs
102it with the value 10.
103
104.. code-block:: cpp
105
106 int func_arg(const std::function<int(int)> &f) {
107 return f(10);
108 }
109
110The example below is more involved: it takes a function of signature ``int -> int``
111and returns another function of the same kind. The return value is a stateful
112lambda function, which stores the value ``f`` in the capture object and adds 1 to
113its return value upon execution.
114
115.. code-block:: cpp
116
117 std::function<int(int)> func_ret(const std::function<int(int)> &f) {
118 return [f](int i) {
119 return f(i) + 1;
120 };
121 }
122
Wenzel Jakob8f4eb002015-10-15 18:13:33 +0200123After including the extra header file :file:`pybind11/functional.h`, it is almost
Wenzel Jakob93296692015-10-13 23:21:54 +0200124trivial to generate binding code for both of these functions.
125
126.. code-block:: cpp
127
Wenzel Jakob8f4eb002015-10-15 18:13:33 +0200128 #include <pybind11/functional.h>
Wenzel Jakob93296692015-10-13 23:21:54 +0200129
Wenzel Jakobb1b71402015-10-18 16:48:30 +0200130 PYBIND11_PLUGIN(example) {
Wenzel Jakob8f4eb002015-10-15 18:13:33 +0200131 py::module m("example", "pybind11 example plugin");
Wenzel Jakob93296692015-10-13 23:21:54 +0200132
133 m.def("func_arg", &func_arg);
134 m.def("func_ret", &func_ret);
135
136 return m.ptr();
137 }
138
139The following interactive session shows how to call them from Python.
140
141.. code-block:: python
142
143 $ python
144 >>> import example
145 >>> def square(i):
146 ... return i * i
147 ...
148 >>> example.func_arg(square)
149 100L
150 >>> square_plus_1 = example.func_ret(square)
151 >>> square_plus_1(4)
152 17L
153 >>>
154
155.. note::
156
157 This functionality is very useful when generating bindings for callbacks in
158 C++ libraries (e.g. a graphical user interface library).
159
160 The file :file:`example/example5.cpp` contains a complete example that
161 demonstrates how to work with callbacks and anonymous functions in more detail.
162
Wenzel Jakoba4175d62015-11-17 08:30:34 +0100163.. warning::
164
165 Keep in mind that passing a function from C++ to Python (or vice versa)
166 will instantiate a piece of wrapper code that translates function
167 invocations between the two languages. Copying the same function back and
168 forth between Python and C++ many times in a row will cause these wrappers
169 to accumulate, which can decrease performance.
170
Wenzel Jakob28f98aa2015-10-13 02:57:16 +0200171Overriding virtual functions in Python
172======================================
173
Wenzel Jakob93296692015-10-13 23:21:54 +0200174Suppose that a C++ class or interface has a virtual function that we'd like to
175to override from within Python (we'll focus on the class ``Animal``; ``Dog`` is
176given as a specific example of how one would do this with traditional C++
177code).
178
179.. code-block:: cpp
180
181 class Animal {
182 public:
183 virtual ~Animal() { }
184 virtual std::string go(int n_times) = 0;
185 };
186
187 class Dog : public Animal {
188 public:
189 std::string go(int n_times) {
190 std::string result;
191 for (int i=0; i<n_times; ++i)
192 result += "woof! ";
193 return result;
194 }
195 };
196
197Let's also suppose that we are given a plain function which calls the
198function ``go()`` on an arbitrary ``Animal`` instance.
199
200.. code-block:: cpp
201
202 std::string call_go(Animal *animal) {
203 return animal->go(3);
204 }
205
206Normally, the binding code for these classes would look as follows:
207
208.. code-block:: cpp
209
Wenzel Jakobb1b71402015-10-18 16:48:30 +0200210 PYBIND11_PLUGIN(example) {
Wenzel Jakob8f4eb002015-10-15 18:13:33 +0200211 py::module m("example", "pybind11 example plugin");
Wenzel Jakob93296692015-10-13 23:21:54 +0200212
213 py::class_<Animal> animal(m, "Animal");
214 animal
215 .def("go", &Animal::go);
216
217 py::class_<Dog>(m, "Dog", animal)
218 .def(py::init<>());
219
220 m.def("call_go", &call_go);
221
222 return m.ptr();
223 }
224
225However, these bindings are impossible to extend: ``Animal`` is not
226constructible, and we clearly require some kind of "trampoline" that
227redirects virtual calls back to Python.
228
229Defining a new type of ``Animal`` from within Python is possible but requires a
230helper class that is defined as follows:
231
232.. code-block:: cpp
233
234 class PyAnimal : public Animal {
235 public:
236 /* Inherit the constructors */
237 using Animal::Animal;
238
239 /* Trampoline (need one for each virtual function) */
240 std::string go(int n_times) {
Wenzel Jakobb1b71402015-10-18 16:48:30 +0200241 PYBIND11_OVERLOAD_PURE(
Wenzel Jakob93296692015-10-13 23:21:54 +0200242 std::string, /* Return type */
243 Animal, /* Parent class */
244 go, /* Name of function */
245 n_times /* Argument(s) */
246 );
247 }
248 };
249
Wenzel Jakobb1b71402015-10-18 16:48:30 +0200250The macro :func:`PYBIND11_OVERLOAD_PURE` should be used for pure virtual
251functions, and :func:`PYBIND11_OVERLOAD` should be used for functions which have
Wenzel Jakob93296692015-10-13 23:21:54 +0200252a default implementation. The binding code also needs a few minor adaptations
253(highlighted):
254
255.. code-block:: cpp
256 :emphasize-lines: 4,6,7
257
Wenzel Jakobb1b71402015-10-18 16:48:30 +0200258 PYBIND11_PLUGIN(example) {
Wenzel Jakob8f4eb002015-10-15 18:13:33 +0200259 py::module m("example", "pybind11 example plugin");
Wenzel Jakob93296692015-10-13 23:21:54 +0200260
261 py::class_<PyAnimal> animal(m, "Animal");
262 animal
263 .alias<Animal>()
264 .def(py::init<>())
265 .def("go", &Animal::go);
266
267 py::class_<Dog>(m, "Dog", animal)
268 .def(py::init<>());
269
270 m.def("call_go", &call_go);
271
272 return m.ptr();
273 }
274
275Importantly, the trampoline helper class is used as the template argument to
276:class:`class_`, and a call to :func:`class_::alias` informs the binding
277generator that this is merely an alias for the underlying type ``Animal``.
278Following this, we are able to define a constructor as usual.
279
280The Python session below shows how to override ``Animal::go`` and invoke it via
281a virtual method call.
282
283.. code-block:: cpp
284
285 >>> from example import *
286 >>> d = Dog()
287 >>> call_go(d)
288 u'woof! woof! woof! '
289 >>> class Cat(Animal):
290 ... def go(self, n_times):
291 ... return "meow! " * n_times
292 ...
293 >>> c = Cat()
294 >>> call_go(c)
295 u'meow! meow! meow! '
296
297.. seealso::
298
299 The file :file:`example/example12.cpp` contains a complete example that
300 demonstrates how to override virtual functions using pybind11 in more
301 detail.
302
Wenzel Jakobecdd8682015-12-07 18:17:58 +0100303
304Global Interpreter Lock (GIL)
305=============================
306
307The classes :class:`gil_scoped_release` and :class:`gil_scoped_acquire` can be
308used to acquire and release the global interpreter lock in the body of a C++
309function call. In this way, long-running C++ code can be parallelized using
310multiple Python threads. Taking the previous section as an example, this could
311be realized as follows (important changes highlighted):
312
313.. code-block:: cpp
314 :emphasize-lines: 8,9,33,34
315
316 class PyAnimal : public Animal {
317 public:
318 /* Inherit the constructors */
319 using Animal::Animal;
320
321 /* Trampoline (need one for each virtual function) */
322 std::string go(int n_times) {
323 /* Acquire GIL before calling Python code */
Wenzel Jakoba4caa852015-12-14 12:39:02 +0100324 py::gil_scoped_acquire acquire;
Wenzel Jakobecdd8682015-12-07 18:17:58 +0100325
326 PYBIND11_OVERLOAD_PURE(
327 std::string, /* Return type */
328 Animal, /* Parent class */
329 go, /* Name of function */
330 n_times /* Argument(s) */
331 );
332 }
333 };
334
335 PYBIND11_PLUGIN(example) {
336 py::module m("example", "pybind11 example plugin");
337
338 py::class_<PyAnimal> animal(m, "Animal");
339 animal
340 .alias<Animal>()
341 .def(py::init<>())
342 .def("go", &Animal::go);
343
344 py::class_<Dog>(m, "Dog", animal)
345 .def(py::init<>());
346
347 m.def("call_go", [](Animal *animal) -> std::string {
348 /* Release GIL before calling into (potentially long-running) C++ code */
Wenzel Jakoba4caa852015-12-14 12:39:02 +0100349 py::gil_scoped_release release;
Wenzel Jakobecdd8682015-12-07 18:17:58 +0100350 return call_go(animal);
351 });
352
353 return m.ptr();
354 }
355
Wenzel Jakob93296692015-10-13 23:21:54 +0200356Passing STL data structures
Wenzel Jakob28f98aa2015-10-13 02:57:16 +0200357===========================
358
Wenzel Jakob8f4eb002015-10-15 18:13:33 +0200359When including the additional header file :file:`pybind11/stl.h`, conversions
Jared Casper6be9e2f2015-12-15 15:56:14 -0800360between ``std::vector<>``, ``std::set<>``, and ``std::map<>`` and the Python
Wenzel Jakob44db04f2015-12-14 12:40:45 +0100361``list``, ``set`` and ``dict`` data structures are automatically enabled. The
362types ``std::pair<>`` and ``std::tuple<>`` are already supported out of the box
363with just the core :file:`pybind11/pybind11.h` header.
Wenzel Jakob93296692015-10-13 23:21:54 +0200364
365.. note::
366
Wenzel Jakob44db04f2015-12-14 12:40:45 +0100367 Arbitrary nesting of any of these types is supported.
Wenzel Jakob93296692015-10-13 23:21:54 +0200368
369.. seealso::
370
371 The file :file:`example/example2.cpp` contains a complete example that
372 demonstrates how to pass STL data types in more detail.
373
374Binding sequence data types, the slicing protocol, etc.
375=======================================================
376
377Please refer to the supplemental example for details.
378
379.. seealso::
380
381 The file :file:`example/example6.cpp` contains a complete example that
382 shows how to bind a sequence data type, including length queries
383 (``__len__``), iterators (``__iter__``), the slicing protocol and other
384 kinds of useful operations.
385
Wenzel Jakob28f98aa2015-10-13 02:57:16 +0200386Return value policies
387=====================
388
Wenzel Jakob93296692015-10-13 23:21:54 +0200389Python and C++ use wildly different ways of managing the memory and lifetime of
390objects managed by them. This can lead to issues when creating bindings for
391functions that return a non-trivial type. Just by looking at the type
392information, it is not clear whether Python should take charge of the returned
393value and eventually free its resources, or if this is handled on the C++ side.
394For this reason, pybind11 provides a several `return value policy` annotations
395that can be passed to the :func:`module::def` and :func:`class_::def`
Wenzel Jakob61d67f02015-12-14 12:53:06 +0100396functions. The default policy is :enum:`return_value_policy::automatic`.
Wenzel Jakob28f98aa2015-10-13 02:57:16 +0200397
Wenzel Jakob93296692015-10-13 23:21:54 +0200398
399+--------------------------------------------------+---------------------------------------------------------------------------+
400| Return value policy | Description |
401+==================================================+===========================================================================+
402| :enum:`return_value_policy::automatic` | Automatic: copy objects returned as values and take ownership of |
403| | objects returned as pointers |
404+--------------------------------------------------+---------------------------------------------------------------------------+
405| :enum:`return_value_policy::copy` | Create a new copy of the returned object, which will be owned by Python |
406+--------------------------------------------------+---------------------------------------------------------------------------+
407| :enum:`return_value_policy::take_ownership` | Reference the existing object and take ownership. Python will call |
408| | the destructor and delete operator when the reference count reaches zero |
409+--------------------------------------------------+---------------------------------------------------------------------------+
410| :enum:`return_value_policy::reference` | Reference the object, but do not take ownership and defer responsibility |
411| | for deleting it to C++ (dangerous when C++ code at some point decides to |
412| | delete it while Python still has a nonzero reference count) |
413+--------------------------------------------------+---------------------------------------------------------------------------+
414| :enum:`return_value_policy::reference_internal` | Reference the object, but do not take ownership. The object is considered |
415| | be owned by the C++ instance whose method or property returned it. The |
416| | Python object will increase the reference count of this 'parent' by 1 |
417| | to ensure that it won't be deallocated while Python is using the 'child' |
418+--------------------------------------------------+---------------------------------------------------------------------------+
419
420.. warning::
421
422 Code with invalid call policies might access unitialized memory and free
423 data structures multiple times, which can lead to hard-to-debug
424 non-determinism and segmentation faults, hence it is worth spending the
425 time to understand all the different options above.
426
427See below for an example that uses the
428:enum:`return_value_policy::reference_internal` policy.
429
430.. code-block:: cpp
431
432 class Example {
433 public:
434 Internal &get_internal() { return internal; }
435 private:
436 Internal internal;
437 };
438
Wenzel Jakobb1b71402015-10-18 16:48:30 +0200439 PYBIND11_PLUGIN(example) {
Wenzel Jakob8f4eb002015-10-15 18:13:33 +0200440 py::module m("example", "pybind11 example plugin");
Wenzel Jakob93296692015-10-13 23:21:54 +0200441
442 py::class_<Example>(m, "Example")
443 .def(py::init<>())
444 .def("get_internal", &Example::get_internal, "Return the internal data", py::return_value_policy::reference_internal)
445
446 return m.ptr();
447 }
448
449Implicit type conversions
450=========================
451
452Suppose that instances of two types ``A`` and ``B`` are used in a project, and
453that an ``A`` can easily be converted into a an instance of type ``B`` (examples of this
454could be a fixed and an arbitrary precision number type).
455
456.. code-block:: cpp
457
458 py::class_<A>(m, "A")
459 /// ... members ...
460
461 py::class_<B>(m, "B")
462 .def(py::init<A>())
463 /// ... members ...
464
465 m.def("func",
466 [](const B &) { /* .... */ }
467 );
468
469To invoke the function ``func`` using a variable ``a`` containing an ``A``
470instance, we'd have to write ``func(B(a))`` in Python. On the other hand, C++
471will automatically apply an implicit type conversion, which makes it possible
472to directly write ``func(a)``.
473
474In this situation (i.e. where ``B`` has a constructor that converts from
475``A``), the following statement enables similar implicit conversions on the
476Python side:
477
478.. code-block:: cpp
479
480 py::implicitly_convertible<A, B>();
481
482Smart pointers
483==============
484
485The binding generator for classes (:class:`class_`) takes an optional second
486template type, which denotes a special *holder* type that is used to manage
487references to the object. When wrapping a type named ``Type``, the default
488value of this template parameter is ``std::unique_ptr<Type>``, which means that
489the object is deallocated when Python's reference count goes to zero.
490
Wenzel Jakob1853b652015-10-18 15:38:50 +0200491It is possible to switch to other types of reference counting wrappers or smart
492pointers, which is useful in codebases that rely on them. For instance, the
493following snippet causes ``std::shared_ptr`` to be used instead.
Wenzel Jakob93296692015-10-13 23:21:54 +0200494
495.. code-block:: cpp
496
Wenzel Jakob5ef12192015-12-15 17:07:35 +0100497 /// Type declaration
498 class Example : public std::enable_shared_from_this<Example> /* <- important, see below */ {
499 // ...
500 };
501
502 /// .... code within PYBIND11_PLUGIN declaration .....
503 py::class_<Example, std::shared_ptr<Example> /* <- important */> obj(m, "Example");
Wenzel Jakob93296692015-10-13 23:21:54 +0200504
Wenzel Jakob1853b652015-10-18 15:38:50 +0200505To enable transparent conversions for functions that take shared pointers as an
Wenzel Jakob5ef12192015-12-15 17:07:35 +0100506argument or that return them, a macro invocation similar to the following must
Wenzel Jakob1853b652015-10-18 15:38:50 +0200507be declared at the top level before any binding code:
508
509.. code-block:: cpp
510
Wenzel Jakobb1b71402015-10-18 16:48:30 +0200511 PYBIND11_DECLARE_HOLDER_TYPE(T, std::shared_ptr<T>);
Wenzel Jakob1853b652015-10-18 15:38:50 +0200512
Wenzel Jakob61d67f02015-12-14 12:53:06 +0100513.. warning::
514
515 The first argument of :func:`PYBIND11_DECLARE_HOLDER_TYPE` should be a
516 placeholder name that is used as a template parameter of the second
517 argument. Thus, feel free to use any identifier, but use it consistently on
518 both sides; also, don't use the name of a type that already exists in your
519 codebase.
520
Wenzel Jakob6e213c92015-11-24 23:05:58 +0100521.. warning::
522
523 To ensure correct reference counting among Python and C++, the use of
524 ``std::shared_ptr<T>`` as a holder type requires that ``T`` inherits from
525 ``std::enable_shared_from_this<T>`` (see cppreference_ for details).
526
Wenzel Jakob5ef12192015-12-15 17:07:35 +0100527If you encounter issues (failure to compile, ``bad_weak_ptr`` exceptions),
528please check that you really did all three steps:
529
5301. invoking the ``PYBIND11_DECLARE_HOLDER_TYPE`` macro in every file that
531 contains pybind11 code and uses your chosen smart pointer type.
532
5332. specifying the holder types to ``class_``.
534
5353. extending from ``std::enable_shared_from_this`` when using
536 ``std::shared_ptr``.
537
Wenzel Jakob6e213c92015-11-24 23:05:58 +0100538.. _cppreference: http://en.cppreference.com/w/cpp/memory/enable_shared_from_this
539
Wenzel Jakob5ef12192015-12-15 17:07:35 +0100540.. seealso::
541
542 The file :file:`example/example8.cpp` contains a complete example that
543 demonstrates how to work with custom reference-counting holder types in
544 more detail.
545
Wenzel Jakob93296692015-10-13 23:21:54 +0200546.. _custom_constructors:
547
548Custom constructors
549===================
550
551The syntax for binding constructors was previously introduced, but it only
552works when a constructor with the given parameters actually exists on the C++
553side. To extend this to more general cases, let's take a look at what actually
554happens under the hood: the following statement
555
556.. code-block:: cpp
557
558 py::class_<Example>(m, "Example")
559 .def(py::init<int>());
560
561is short hand notation for
562
563.. code-block:: cpp
564
565 py::class_<Example>(m, "Example")
566 .def("__init__",
567 [](Example &instance, int arg) {
568 new (&instance) Example(arg);
569 }
570 );
571
572In other words, :func:`init` creates an anonymous function that invokes an
573in-place constructor. Memory allocation etc. is already take care of beforehand
574within pybind11.
575
576Catching and throwing exceptions
577================================
578
579When C++ code invoked from Python throws an ``std::exception``, it is
580automatically converted into a Python ``Exception``. pybind11 defines multiple
581special exception classes that will map to different types of Python
582exceptions:
583
584+----------------------------+------------------------------+
585| C++ exception type | Python exception type |
586+============================+==============================+
587| :class:`std::exception` | ``Exception`` |
588+----------------------------+------------------------------+
589| :class:`stop_iteration` | ``StopIteration`` (used to |
590| | implement custom iterators) |
591+----------------------------+------------------------------+
592| :class:`index_error` | ``IndexError`` (used to |
593| | indicate out of bounds |
594| | accesses in ``__getitem__``, |
595| | ``__setitem__``, etc.) |
596+----------------------------+------------------------------+
597| :class:`error_already_set` | Indicates that the Python |
598| | exception flag has already |
599| | been initialized. |
600+----------------------------+------------------------------+
601
602When a Python function invoked from C++ throws an exception, it is converted
603into a C++ exception of type :class:`error_already_set` whose string payload
604contains a textual summary.
605
606There is also a special exception :class:`cast_error` that is thrown by
607:func:`handle::call` when the input arguments cannot be converted to Python
608objects.
Wenzel Jakob28f98aa2015-10-13 02:57:16 +0200609
610Buffer protocol
611===============
612
613Python supports an extremely general and convenient approach for exchanging
614data between plugin libraries. Types can expose a buffer view which provides
615fast direct access to the raw internal representation. Suppose we want to bind
616the following simplistic Matrix class:
617
618.. code-block:: cpp
619
620 class Matrix {
621 public:
622 Matrix(size_t rows, size_t cols) : m_rows(rows), m_cols(cols) {
623 m_data = new float[rows*cols];
624 }
625 float *data() { return m_data; }
626 size_t rows() const { return m_rows; }
627 size_t cols() const { return m_cols; }
628 private:
629 size_t m_rows, m_cols;
630 float *m_data;
631 };
632
633The following binding code exposes the ``Matrix`` contents as a buffer object,
634making it possible to cast Matrixes into NumPy arrays. It is even possible to
635completely avoid copy operations with Python expressions like
636``np.array(matrix_instance, copy = False)``.
637
638.. code-block:: cpp
639
640 py::class_<Matrix>(m, "Matrix")
641 .def_buffer([](Matrix &m) -> py::buffer_info {
642 return py::buffer_info(
643 m.data(), /* Pointer to buffer */
644 sizeof(float), /* Size of one scalar */
645 py::format_descriptor<float>::value(), /* Python struct-style format descriptor */
646 2, /* Number of dimensions */
647 { m.rows(), m.cols() }, /* Buffer dimensions */
648 { sizeof(float) * m.rows(), /* Strides (in bytes) for each index */
649 sizeof(float) }
650 );
651 });
652
653The snippet above binds a lambda function, which can create ``py::buffer_info``
654description records on demand describing a given matrix. The contents of
655``py::buffer_info`` mirror the Python buffer protocol specification.
656
657.. code-block:: cpp
658
659 struct buffer_info {
660 void *ptr;
661 size_t itemsize;
662 std::string format;
663 int ndim;
664 std::vector<size_t> shape;
665 std::vector<size_t> strides;
666 };
667
668To create a C++ function that can take a Python buffer object as an argument,
669simply use the type ``py::buffer`` as one of its arguments. Buffers can exist
670in a great variety of configurations, hence some safety checks are usually
671necessary in the function body. Below, you can see an basic example on how to
672define a custom constructor for the Eigen double precision matrix
673(``Eigen::MatrixXd``) type, which supports initialization from compatible
674buffer
675objects (e.g. a NumPy matrix).
676
677.. code-block:: cpp
678
679 py::class_<Eigen::MatrixXd>(m, "MatrixXd")
680 .def("__init__", [](Eigen::MatrixXd &m, py::buffer b) {
681 /* Request a buffer descriptor from Python */
682 py::buffer_info info = b.request();
683
684 /* Some sanity checks ... */
685 if (info.format != py::format_descriptor<double>::value())
686 throw std::runtime_error("Incompatible format: expected a double array!");
687
688 if (info.ndim != 2)
689 throw std::runtime_error("Incompatible buffer dimension!");
690
691 if (info.strides[0] == sizeof(double)) {
692 /* Buffer has the right layout -- directly copy. */
693 new (&m) Eigen::MatrixXd(info.shape[0], info.shape[1]);
694 memcpy(m.data(), info.ptr, sizeof(double) * m.size());
695 } else {
696 /* Oops -- the buffer is transposed */
697 new (&m) Eigen::MatrixXd(info.shape[1], info.shape[0]);
698 memcpy(m.data(), info.ptr, sizeof(double) * m.size());
699 m.transposeInPlace();
700 }
701 });
702
Wenzel Jakob93296692015-10-13 23:21:54 +0200703.. seealso::
704
705 The file :file:`example/example7.cpp` contains a complete example that
706 demonstrates using the buffer protocol with pybind11 in more detail.
707
Wenzel Jakob28f98aa2015-10-13 02:57:16 +0200708NumPy support
709=============
710
711By exchanging ``py::buffer`` with ``py::array`` in the above snippet, we can
712restrict the function so that it only accepts NumPy arrays (rather than any
713type of Python object satisfying the buffer object protocol).
714
715In many situations, we want to define a function which only accepts a NumPy
Wenzel Jakob93296692015-10-13 23:21:54 +0200716array of a certain data type. This is possible via the ``py::array_t<T>``
Wenzel Jakob28f98aa2015-10-13 02:57:16 +0200717template. For instance, the following function requires the argument to be a
718dense array of doubles in C-style ordering.
719
720.. code-block:: cpp
721
Wenzel Jakob93296692015-10-13 23:21:54 +0200722 void f(py::array_t<double> array);
Wenzel Jakob28f98aa2015-10-13 02:57:16 +0200723
724When it is invoked with a different type (e.g. an integer), the binding code
725will attempt to cast the input into a NumPy array of the requested type.
Wenzel Jakob8f4eb002015-10-15 18:13:33 +0200726Note that this feature requires the ``pybind11/numpy.h`` header to be included.
Wenzel Jakob28f98aa2015-10-13 02:57:16 +0200727
728Vectorizing functions
729=====================
730
731Suppose we want to bind a function with the following signature to Python so
732that it can process arbitrary NumPy array arguments (vectors, matrices, general
733N-D arrays) in addition to its normal arguments:
734
735.. code-block:: cpp
736
737 double my_func(int x, float y, double z);
738
Wenzel Jakob8f4eb002015-10-15 18:13:33 +0200739After including the ``pybind11/numpy.h`` header, this is extremely simple:
Wenzel Jakob28f98aa2015-10-13 02:57:16 +0200740
741.. code-block:: cpp
742
743 m.def("vectorized_func", py::vectorize(my_func));
744
745Invoking the function like below causes 4 calls to be made to ``my_func`` with
746each of the the array elements. The result is returned as a NumPy array of type
747``numpy.dtype.float64``.
748
749.. code-block:: python
750
751 >>> x = np.array([[1, 3],[5, 7]])
752 >>> y = np.array([[2, 4],[6, 8]])
753 >>> z = 3
754 >>> result = vectorized_func(x, y, z)
755
756The scalar argument ``z`` is transparently replicated 4 times. The input
757arrays ``x`` and ``y`` are automatically converted into the right types (they
758are of type ``numpy.dtype.int64`` but need to be ``numpy.dtype.int32`` and
759``numpy.dtype.float32``, respectively)
760
761Sometimes we might want to explitly exclude an argument from the vectorization
762because it makes little sense to wrap it in a NumPy array. For instance,
763suppose the function signature was
764
765.. code-block:: cpp
766
767 double my_func(int x, float y, my_custom_type *z);
768
769This can be done with a stateful Lambda closure:
770
771.. code-block:: cpp
772
773 // Vectorize a lambda function with a capture object (e.g. to exclude some arguments from the vectorization)
774 m.def("vectorized_func",
Wenzel Jakob93296692015-10-13 23:21:54 +0200775 [](py::array_t<int> x, py::array_t<float> y, my_custom_type *z) {
Wenzel Jakob28f98aa2015-10-13 02:57:16 +0200776 auto stateful_closure = [z](int x, float y) { return my_func(x, y, z); };
777 return py::vectorize(stateful_closure)(x, y);
778 }
779 );
780
Wenzel Jakob93296692015-10-13 23:21:54 +0200781.. seealso::
Wenzel Jakob28f98aa2015-10-13 02:57:16 +0200782
Wenzel Jakob93296692015-10-13 23:21:54 +0200783 The file :file:`example/example10.cpp` contains a complete example that
784 demonstrates using :func:`vectorize` in more detail.
Wenzel Jakob28f98aa2015-10-13 02:57:16 +0200785
Wenzel Jakob93296692015-10-13 23:21:54 +0200786Functions taking Python objects as arguments
787============================================
Wenzel Jakob28f98aa2015-10-13 02:57:16 +0200788
Wenzel Jakob93296692015-10-13 23:21:54 +0200789pybind11 exposes all major Python types using thin C++ wrapper classes. These
790wrapper classes can also be used as parameters of functions in bindings, which
791makes it possible to directly work with native Python types on the C++ side.
792For instance, the following statement iterates over a Python ``dict``:
Wenzel Jakob28f98aa2015-10-13 02:57:16 +0200793
Wenzel Jakob93296692015-10-13 23:21:54 +0200794.. code-block:: cpp
795
796 void print_dict(py::dict dict) {
797 /* Easily interact with Python types */
798 for (auto item : dict)
799 std::cout << "key=" << item.first << ", "
800 << "value=" << item.second << std::endl;
801 }
802
803Available types include :class:`handle`, :class:`object`, :class:`bool_`,
804:class:`int_`, :class:`float_`, :class:`str`, :class:`tuple`, :class:`list`,
805:class:`dict`, :class:`slice`, :class:`capsule`, :class:`function`,
806:class:`buffer`, :class:`array`, and :class:`array_t`.
807
Wenzel Jakob436b7312015-10-20 01:04:30 +0200808In this kind of mixed code, it is often necessary to convert arbitrary C++
809types to Python, which can be done using :func:`cast`:
810
811.. code-block:: cpp
812
813 MyClass *cls = ..;
814 py::object obj = py::cast(cls);
815
816The reverse direction uses the following syntax:
817
818.. code-block:: cpp
819
820 py::object obj = ...;
821 MyClass *cls = obj.cast<MyClass *>();
822
823When conversion fails, both directions throw the exception :class:`cast_error`.
824
Wenzel Jakob93296692015-10-13 23:21:54 +0200825.. seealso::
826
827 The file :file:`example/example2.cpp` contains a complete example that
828 demonstrates passing native Python types in more detail.
Wenzel Jakob2ac50442016-01-17 22:36:35 +0100829
830Default arguments revisited
831===========================
832
833The section on :ref:`default_args` previously discussed basic usage of default
834arguments using pybind11. One noteworthy aspect of their implementation is that
835default arguments are converted to Python objects right at declaration time.
836Consider the following example:
837
838.. code-block:: cpp
839
840 py::class_<MyClass>("MyClass")
841 .def("myFunction", py::arg("arg") = SomeType(123));
842
843In this case, pybind11 must already be set up to deal with values of the type
844``SomeType`` (via a prior instantiation of ``py::class_<SomeType>``), or an
845exception will be thrown.
846
847Another aspect worth highlighting is that the "preview" of the default argument
848in the function signature is generated using the object's ``__repr__`` method.
849If not available, the signature may not be very helpful, e.g.:
850
851.. code-block:: python
852
853 FUNCTIONS
854 ...
855 | myFunction(...)
856 | Signature : (MyClass, arg : SomeType = <SomeType object at 0x101b7b080>) -> None
857 ...
858
859The first way of addressing this is by defining ``SomeType.__repr__``.
860Alternatively, it is possible to specify the human-readable preview of the
861default argument manually using the ``arg_t`` notation:
862
863.. code-block:: cpp
864
865 py::class_<MyClass>("MyClass")
866 .def("myFunction", py::arg_t<SomeType>("arg", SomeType(123), "SomeType(123)"));
867