Wenzel Jakob | 5cd3311 | 2015-10-20 00:58:59 +0200 | [diff] [blame] | 1 | Benchmark |
| 2 | ========= |
| 3 | |
| 4 | The following is the result of a synthetic benchmark comparing both compilation |
| 5 | time and module size of pybind11 against Boost.Python. |
| 6 | |
| 7 | A python script (see the ``docs/benchmark.py`` file) was used to generate a |
| 8 | set of dummy classes whose count increases for each successive benchmark |
| 9 | (between 1 and 512 classes in powers of two). Each class has four methods with |
| 10 | a randomly generated signature with a return value and four arguments. (There |
| 11 | was no particular reason for this setup other than the desire to generate many |
| 12 | unique function signatures whose count could be controlled in a simple way.) |
| 13 | |
| 14 | Here is an example of the binding code for one class: |
| 15 | |
| 16 | .. code-block:: cpp |
| 17 | |
| 18 | ... |
| 19 | class cl034 { |
| 20 | public: |
| 21 | cl279 *fn_000(cl084 *, cl057 *, cl065 *, cl042 *); |
| 22 | cl025 *fn_001(cl098 *, cl262 *, cl414 *, cl121 *); |
| 23 | cl085 *fn_002(cl445 *, cl297 *, cl145 *, cl421 *); |
| 24 | cl470 *fn_003(cl200 *, cl323 *, cl332 *, cl492 *); |
| 25 | }; |
| 26 | ... |
| 27 | |
| 28 | PYBIND11_PLUGIN(example) { |
| 29 | py::module m("example"); |
| 30 | ... |
| 31 | py::class_<cl034>(m, "cl034") |
| 32 | .def("fn_000", &cl034::fn_000) |
| 33 | .def("fn_001", &cl034::fn_001) |
| 34 | .def("fn_002", &cl034::fn_002) |
| 35 | .def("fn_003", &cl034::fn_003) |
| 36 | ... |
| 37 | return m.ptr(); |
| 38 | } |
| 39 | |
| 40 | The Boost.Python version looks almost identical except that a return value |
| 41 | policy had to be specified as an argument to ``def()``. For both libraries, |
| 42 | compilation was done with |
| 43 | |
| 44 | .. code-block:: bash |
| 45 | |
| 46 | Apple LLVM version 7.0.0 (clang-700.0.72) |
| 47 | |
| 48 | and the following compilation flags |
| 49 | |
| 50 | .. code-block:: bash |
| 51 | |
| 52 | g++ -Os -shared -rdynamic -undefined dynamic_lookup -fvisibility=hidden -std=c++11 |
| 53 | |
| 54 | The following log-log plot shows how the compilation time grows for an |
| 55 | increasing number of class and function declarations. pybind11 includes fewer |
| 56 | headers, which initially leads to shorter compilation times, but the |
| 57 | performance is ultimately very similar (pybind11 is 1 second faster for the |
| 58 | largest file, which is less than 1% of the total compilation time). |
| 59 | |
| 60 | .. image:: pybind11_vs_boost_python1.svg |
| 61 | |
| 62 | Differences between the two libraries become more pronounced when considering |
| 63 | the file size of the generated Python plugin. Note that the plot below does not |
| 64 | include the size of the Boost.Python shared library, hence Boost actually has a |
| 65 | slight advantage. |
| 66 | |
| 67 | .. image:: pybind11_vs_boost_python2.svg |
| 68 | |
| 69 | Despite this, the libraries procuced by Boost.Python for more than a few |
| 70 | functions are consistently larger by a factor of 1.75. |