Dean Moldovan | 67b52d8 | 2016-10-16 19:12:43 +0200 | [diff] [blame] | 1 | Exceptions |
| 2 | ########## |
| 3 | |
jbarlow83 | b886369 | 2020-08-22 15:11:09 -0700 | [diff] [blame] | 4 | Built-in C++ to Python exception translation |
| 5 | ============================================ |
Dean Moldovan | 67b52d8 | 2016-10-16 19:12:43 +0200 | [diff] [blame] | 6 | |
jbarlow83 | b886369 | 2020-08-22 15:11:09 -0700 | [diff] [blame] | 7 | When Python calls C++ code through pybind11, pybind11 provides a C++ exception handler |
| 8 | that will trap C++ exceptions, translate them to the corresponding Python exception, |
| 9 | and raise them so that Python code can handle them. |
| 10 | |
| 11 | pybind11 defines translations for ``std::exception`` and its standard |
| 12 | subclasses, and several special exception classes that translate to specific |
| 13 | Python exceptions. Note that these are not actually Python exceptions, so they |
| 14 | cannot be examined using the Python C API. Instead, they are pure C++ objects |
| 15 | that pybind11 will translate the corresponding Python exception when they arrive |
| 16 | at its exception handler. |
Dean Moldovan | 67b52d8 | 2016-10-16 19:12:43 +0200 | [diff] [blame] | 17 | |
| 18 | .. tabularcolumns:: |p{0.5\textwidth}|p{0.45\textwidth}| |
| 19 | |
Antony Lee | 8fbb559 | 2018-03-11 16:15:56 -0700 | [diff] [blame] | 20 | +--------------------------------------+--------------------------------------+ |
jbarlow83 | b886369 | 2020-08-22 15:11:09 -0700 | [diff] [blame] | 21 | | Exception thrown by C++ | Translated to Python exception type | |
Antony Lee | 8fbb559 | 2018-03-11 16:15:56 -0700 | [diff] [blame] | 22 | +======================================+======================================+ |
| 23 | | :class:`std::exception` | ``RuntimeError`` | |
| 24 | +--------------------------------------+--------------------------------------+ |
| 25 | | :class:`std::bad_alloc` | ``MemoryError`` | |
| 26 | +--------------------------------------+--------------------------------------+ |
| 27 | | :class:`std::domain_error` | ``ValueError`` | |
| 28 | +--------------------------------------+--------------------------------------+ |
| 29 | | :class:`std::invalid_argument` | ``ValueError`` | |
| 30 | +--------------------------------------+--------------------------------------+ |
| 31 | | :class:`std::length_error` | ``ValueError`` | |
| 32 | +--------------------------------------+--------------------------------------+ |
Wenzel Jakob | 51ca6b0 | 2019-06-11 22:47:37 +0200 | [diff] [blame] | 33 | | :class:`std::out_of_range` | ``IndexError`` | |
Antony Lee | 8fbb559 | 2018-03-11 16:15:56 -0700 | [diff] [blame] | 34 | +--------------------------------------+--------------------------------------+ |
| 35 | | :class:`std::range_error` | ``ValueError`` | |
| 36 | +--------------------------------------+--------------------------------------+ |
Francesco Biscani | deb3cb2 | 2019-11-14 08:56:58 +0100 | [diff] [blame] | 37 | | :class:`std::overflow_error` | ``OverflowError`` | |
| 38 | +--------------------------------------+--------------------------------------+ |
Antony Lee | 8fbb559 | 2018-03-11 16:15:56 -0700 | [diff] [blame] | 39 | | :class:`pybind11::stop_iteration` | ``StopIteration`` (used to implement | |
| 40 | | | custom iterators) | |
| 41 | +--------------------------------------+--------------------------------------+ |
| 42 | | :class:`pybind11::index_error` | ``IndexError`` (used to indicate out | |
| 43 | | | of bounds access in ``__getitem__``, | |
| 44 | | | ``__setitem__``, etc.) | |
| 45 | +--------------------------------------+--------------------------------------+ |
| 46 | | :class:`pybind11::value_error` | ``ValueError`` (used to indicate | |
| 47 | | | wrong value passed in | |
| 48 | | | ``container.remove(...)``) | |
| 49 | +--------------------------------------+--------------------------------------+ |
| 50 | | :class:`pybind11::key_error` | ``KeyError`` (used to indicate out | |
| 51 | | | of bounds access in ``__getitem__``, | |
| 52 | | | ``__setitem__`` in dict-like | |
| 53 | | | objects, etc.) | |
| 54 | +--------------------------------------+--------------------------------------+ |
Dean Moldovan | 67b52d8 | 2016-10-16 19:12:43 +0200 | [diff] [blame] | 55 | |
jbarlow83 | b886369 | 2020-08-22 15:11:09 -0700 | [diff] [blame] | 56 | Exception translation is not bidirectional. That is, *catching* the C++ |
| 57 | exceptions defined above above will not trap exceptions that originate from |
| 58 | Python. For that, catch :class:`pybind11::error_already_set`. See :ref:`below |
| 59 | <handling_python_exceptions_cpp>` for further details. |
Dean Moldovan | 67b52d8 | 2016-10-16 19:12:43 +0200 | [diff] [blame] | 60 | |
| 61 | There is also a special exception :class:`cast_error` that is thrown by |
| 62 | :func:`handle::call` when the input arguments cannot be converted to Python |
| 63 | objects. |
| 64 | |
| 65 | Registering custom translators |
| 66 | ============================== |
| 67 | |
| 68 | If the default exception conversion policy described above is insufficient, |
| 69 | pybind11 also provides support for registering custom exception translators. |
| 70 | To register a simple exception conversion that translates a C++ exception into |
| 71 | a new Python exception using the C++ exception's ``what()`` method, a helper |
| 72 | function is available: |
| 73 | |
| 74 | .. code-block:: cpp |
| 75 | |
| 76 | py::register_exception<CppExp>(module, "PyExp"); |
| 77 | |
| 78 | This call creates a Python exception class with the name ``PyExp`` in the given |
| 79 | module and automatically converts any encountered exceptions of type ``CppExp`` |
| 80 | into Python exceptions of type ``PyExp``. |
| 81 | |
michalsustr | 3bd0d7a | 2020-09-06 05:35:53 -0600 | [diff] [blame] | 82 | It is possible to specify base class for the exception using the third |
Yannick Jadoul | 16f199f | 2020-09-15 16:24:39 +0200 | [diff] [blame^] | 83 | parameter, a `handle`: |
michalsustr | 3bd0d7a | 2020-09-06 05:35:53 -0600 | [diff] [blame] | 84 | |
| 85 | .. code-block:: cpp |
| 86 | |
| 87 | py::register_exception<CppExp>(module, "PyExp", PyExc_RuntimeError); |
| 88 | |
| 89 | Then `PyExp` can be caught both as `PyExp` and `RuntimeError`. |
| 90 | |
| 91 | The class objects of the built-in Python exceptions are listed in the Python |
| 92 | documentation on `Standard Exceptions <https://docs.python.org/3/c-api/exceptions.html#standard-exceptions>`_. |
| 93 | The default base class is `PyExc_Exception`. |
| 94 | |
Dean Moldovan | 67b52d8 | 2016-10-16 19:12:43 +0200 | [diff] [blame] | 95 | When more advanced exception translation is needed, the function |
| 96 | ``py::register_exception_translator(translator)`` can be used to register |
| 97 | functions that can translate arbitrary exception types (and which may include |
| 98 | additional logic to do so). The function takes a stateless callable (e.g. a |
| 99 | function pointer or a lambda function without captured variables) with the call |
| 100 | signature ``void(std::exception_ptr)``. |
| 101 | |
| 102 | When a C++ exception is thrown, the registered exception translators are tried |
| 103 | in reverse order of registration (i.e. the last registered translator gets the |
| 104 | first shot at handling the exception). |
| 105 | |
| 106 | Inside the translator, ``std::rethrow_exception`` should be used within |
| 107 | a try block to re-throw the exception. One or more catch clauses to catch |
| 108 | the appropriate exceptions should then be used with each clause using |
| 109 | ``PyErr_SetString`` to set a Python exception or ``ex(string)`` to set |
| 110 | the python exception to a custom exception type (see below). |
| 111 | |
| 112 | To declare a custom Python exception type, declare a ``py::exception`` variable |
| 113 | and use this in the associated exception translator (note: it is often useful |
| 114 | to make this a static declaration when using it inside a lambda expression |
| 115 | without requiring capturing). |
| 116 | |
Dean Moldovan | 67b52d8 | 2016-10-16 19:12:43 +0200 | [diff] [blame] | 117 | The following example demonstrates this for a hypothetical exception classes |
| 118 | ``MyCustomException`` and ``OtherException``: the first is translated to a |
| 119 | custom python exception ``MyCustomError``, while the second is translated to a |
| 120 | standard python RuntimeError: |
| 121 | |
| 122 | .. code-block:: cpp |
| 123 | |
| 124 | static py::exception<MyCustomException> exc(m, "MyCustomError"); |
| 125 | py::register_exception_translator([](std::exception_ptr p) { |
| 126 | try { |
| 127 | if (p) std::rethrow_exception(p); |
| 128 | } catch (const MyCustomException &e) { |
| 129 | exc(e.what()); |
| 130 | } catch (const OtherException &e) { |
| 131 | PyErr_SetString(PyExc_RuntimeError, e.what()); |
| 132 | } |
| 133 | }); |
| 134 | |
| 135 | Multiple exceptions can be handled by a single translator, as shown in the |
| 136 | example above. If the exception is not caught by the current translator, the |
| 137 | previously registered one gets a chance. |
| 138 | |
| 139 | If none of the registered exception translators is able to handle the |
| 140 | exception, it is handled by the default converter as described in the previous |
| 141 | section. |
| 142 | |
| 143 | .. seealso:: |
| 144 | |
| 145 | The file :file:`tests/test_exceptions.cpp` contains examples |
| 146 | of various custom exception translators and custom exception types. |
| 147 | |
| 148 | .. note:: |
| 149 | |
jbarlow83 | b886369 | 2020-08-22 15:11:09 -0700 | [diff] [blame] | 150 | Call either ``PyErr_SetString`` or a custom exception's call |
Dean Moldovan | 67b52d8 | 2016-10-16 19:12:43 +0200 | [diff] [blame] | 151 | operator (``exc(string)``) for every exception caught in a custom exception |
| 152 | translator. Failure to do so will cause Python to crash with ``SystemError: |
| 153 | error return without exception set``. |
| 154 | |
| 155 | Exceptions that you do not plan to handle should simply not be caught, or |
luz.paz | 28cb676 | 2018-01-09 12:30:19 -0500 | [diff] [blame] | 156 | may be explicitly (re-)thrown to delegate it to the other, |
Dean Moldovan | 67b52d8 | 2016-10-16 19:12:43 +0200 | [diff] [blame] | 157 | previously-declared existing exception translators. |
James R. Barlow | 3618bea | 2020-08-08 03:07:14 -0700 | [diff] [blame] | 158 | |
jbarlow83 | b886369 | 2020-08-22 15:11:09 -0700 | [diff] [blame] | 159 | .. _handling_python_exceptions_cpp: |
| 160 | |
| 161 | Handling exceptions from Python in C++ |
| 162 | ====================================== |
| 163 | |
| 164 | When C++ calls Python functions, such as in a callback function or when |
| 165 | manipulating Python objects, and Python raises an ``Exception``, pybind11 |
| 166 | converts the Python exception into a C++ exception of type |
| 167 | :class:`pybind11::error_already_set` whose payload contains a C++ string textual |
| 168 | summary and the actual Python exception. ``error_already_set`` is used to |
| 169 | propagate Python exception back to Python (or possibly, handle them in C++). |
| 170 | |
| 171 | .. tabularcolumns:: |p{0.5\textwidth}|p{0.45\textwidth}| |
| 172 | |
| 173 | +--------------------------------------+--------------------------------------+ |
| 174 | | Exception raised in Python | Thrown as C++ exception type | |
| 175 | +======================================+======================================+ |
| 176 | | Any Python ``Exception`` | :class:`pybind11::error_already_set` | |
| 177 | +--------------------------------------+--------------------------------------+ |
| 178 | |
| 179 | For example: |
| 180 | |
| 181 | .. code-block:: cpp |
| 182 | |
| 183 | try { |
| 184 | // open("missing.txt", "r") |
| 185 | auto file = py::module::import("io").attr("open")("missing.txt", "r"); |
| 186 | auto text = file.attr("read")(); |
| 187 | file.attr("close")(); |
| 188 | } catch (py::error_already_set &e) { |
| 189 | if (e.matches(PyExc_FileNotFoundError)) { |
| 190 | py::print("missing.txt not found"); |
| 191 | } else if (e.match(PyExc_PermissionError)) { |
| 192 | py::print("missing.txt found but not accessible"); |
| 193 | } else { |
| 194 | throw; |
| 195 | } |
| 196 | } |
| 197 | |
| 198 | Note that C++ to Python exception translation does not apply here, since that is |
| 199 | a method for translating C++ exceptions to Python, not vice versa. The error raised |
| 200 | from Python is always ``error_already_set``. |
| 201 | |
| 202 | This example illustrates this behavior: |
| 203 | |
| 204 | .. code-block:: cpp |
| 205 | |
| 206 | try { |
| 207 | py::eval("raise ValueError('The Ring')"); |
| 208 | } catch (py::value_error &boromir) { |
| 209 | // Boromir never gets the ring |
| 210 | assert(false); |
| 211 | } catch (py::error_already_set &frodo) { |
| 212 | // Frodo gets the ring |
| 213 | py::print("I will take the ring"); |
| 214 | } |
| 215 | |
| 216 | try { |
| 217 | // py::value_error is a request for pybind11 to raise a Python exception |
| 218 | throw py::value_error("The ball"); |
| 219 | } catch (py::error_already_set &cat) { |
| 220 | // cat won't catch the ball since |
| 221 | // py::value_error is not a Python exception |
| 222 | assert(false); |
| 223 | } catch (py::value_error &dog) { |
| 224 | // dog will catch the ball |
| 225 | py::print("Run Spot run"); |
| 226 | throw; // Throw it again (pybind11 will raise ValueError) |
| 227 | } |
| 228 | |
| 229 | Handling errors from the Python C API |
| 230 | ===================================== |
| 231 | |
| 232 | Where possible, use :ref:`pybind11 wrappers <wrappers>` instead of calling |
| 233 | the Python C API directly. When calling the Python C API directly, in |
| 234 | addition to manually managing reference counts, one must follow the pybind11 |
| 235 | error protocol, which is outlined here. |
| 236 | |
| 237 | After calling the Python C API, if Python returns an error, |
| 238 | ``throw py::error_already_set();``, which allows pybind11 to deal with the |
| 239 | exception and pass it back to the Python interpreter. This includes calls to |
| 240 | the error setting functions such as ``PyErr_SetString``. |
| 241 | |
| 242 | .. code-block:: cpp |
| 243 | |
| 244 | PyErr_SetString(PyExc_TypeError, "C API type error demo"); |
| 245 | throw py::error_already_set(); |
| 246 | |
| 247 | // But it would be easier to simply... |
| 248 | throw py::type_error("pybind11 wrapper type error"); |
| 249 | |
| 250 | Alternately, to ignore the error, call `PyErr_Clear |
| 251 | <https://docs.python.org/3/c-api/exceptions.html#c.PyErr_Clear>`_. |
| 252 | |
| 253 | Any Python error must be thrown or cleared, or Python/pybind11 will be left in |
| 254 | an invalid state. |
| 255 | |
James R. Barlow | 3618bea | 2020-08-08 03:07:14 -0700 | [diff] [blame] | 256 | .. _unraisable_exceptions: |
| 257 | |
| 258 | Handling unraisable exceptions |
| 259 | ============================== |
| 260 | |
| 261 | If a Python function invoked from a C++ destructor or any function marked |
| 262 | ``noexcept(true)`` (collectively, "noexcept functions") throws an exception, there |
jbarlow83 | b886369 | 2020-08-22 15:11:09 -0700 | [diff] [blame] | 263 | is no way to propagate the exception, as such functions may not throw. |
| 264 | Should they throw or fail to catch any exceptions in their call graph, |
| 265 | the C++ runtime calls ``std::terminate()`` to abort immediately. |
James R. Barlow | 3618bea | 2020-08-08 03:07:14 -0700 | [diff] [blame] | 266 | |
jbarlow83 | b886369 | 2020-08-22 15:11:09 -0700 | [diff] [blame] | 267 | Similarly, Python exceptions raised in a class's ``__del__`` method do not |
| 268 | propagate, but are logged by Python as an unraisable error. In Python 3.8+, a |
| 269 | `system hook is triggered |
| 270 | <https://docs.python.org/3/library/sys.html#sys.unraisablehook>`_ |
| 271 | and an auditing event is logged. |
James R. Barlow | 3618bea | 2020-08-08 03:07:14 -0700 | [diff] [blame] | 272 | |
| 273 | Any noexcept function should have a try-catch block that traps |
jbarlow83 | b886369 | 2020-08-22 15:11:09 -0700 | [diff] [blame] | 274 | class:`error_already_set` (or any other exception that can occur). Note that |
| 275 | pybind11 wrappers around Python exceptions such as |
| 276 | :class:`pybind11::value_error` are *not* Python exceptions; they are C++ |
| 277 | exceptions that pybind11 catches and converts to Python exceptions. Noexcept |
| 278 | functions cannot propagate these exceptions either. A useful approach is to |
| 279 | convert them to Python exceptions and then ``discard_as_unraisable`` as shown |
| 280 | below. |
James R. Barlow | 3618bea | 2020-08-08 03:07:14 -0700 | [diff] [blame] | 281 | |
| 282 | .. code-block:: cpp |
| 283 | |
| 284 | void nonthrowing_func() noexcept(true) { |
| 285 | try { |
| 286 | // ... |
| 287 | } catch (py::error_already_set &eas) { |
| 288 | // Discard the Python error using Python APIs, using the C++ magic |
| 289 | // variable __func__. Python already knows the type and value and of the |
| 290 | // exception object. |
| 291 | eas.discard_as_unraisable(__func__); |
| 292 | } catch (const std::exception &e) { |
| 293 | // Log and discard C++ exceptions. |
James R. Barlow | 3618bea | 2020-08-08 03:07:14 -0700 | [diff] [blame] | 294 | third_party::log(e); |
| 295 | } |
| 296 | } |
Henry Schreiner | a6887b6 | 2020-08-19 14:53:59 -0400 | [diff] [blame] | 297 | |
| 298 | .. versionadded:: 2.6 |