Jonathan Ballet | c7b7ba7 | 2011-07-16 15:29:47 +0900 | [diff] [blame] | 1 | .. _internals: |
| 2 | |
| 3 | Internals |
| 4 | ========= |
| 5 | |
| 6 | We ran into three main problems developing this: Exceptions, callbacks and |
| 7 | accessing socket methods. This is what this chapter is about. |
| 8 | |
| 9 | |
| 10 | .. _exceptions: |
| 11 | |
| 12 | Exceptions |
| 13 | ---------- |
| 14 | |
| 15 | We realized early that most of the exceptions would be raised by the I/O |
| 16 | functions of OpenSSL, so it felt natural to mimic OpenSSL's error code system, |
| 17 | translating them into Python exceptions. This naturally gives us the exceptions |
| 18 | :py:exc:`.SSL.ZeroReturnError`, :py:exc:`.SSL.WantReadError`, |
| 19 | :py:exc:`.SSL.WantWriteError`, :py:exc:`.SSL.WantX509LookupError` and |
| 20 | :py:exc:`.SSL.SysCallError`. |
| 21 | |
| 22 | For more information about this, see section :ref:`openssl-ssl`. |
| 23 | |
| 24 | |
| 25 | .. _callbacks: |
| 26 | |
| 27 | Callbacks |
| 28 | --------- |
| 29 | |
| 30 | There are a number of problems with callbacks. First of all, OpenSSL is written |
| 31 | as a C library, it's not meant to have Python callbacks, so a way around that |
| 32 | is needed. Another problem is thread support. A lot of the OpenSSL I/O |
| 33 | functions can block if the socket is in blocking mode, and then you want other |
| 34 | Python threads to be able to do other things. The real trouble is if you've |
| 35 | released the global CPython interpreter lock to do a potentially blocking |
| 36 | operation, and the operation calls a callback. Then we must take the GIL back, |
| 37 | since calling Python APIs without holding it is not allowed. |
| 38 | |
| 39 | There are two solutions to the first problem, both of which are necessary. The |
| 40 | first solution to use is if the C callback allows ''userdata'' to be passed to |
| 41 | it (an arbitrary pointer normally). This is great! We can set our Python |
| 42 | function object as the real userdata and emulate userdata for the Python |
| 43 | function in another way. The other solution can be used if an object with an |
| 44 | ''app_data'' system always is passed to the callback. For example, the SSL |
| 45 | object in OpenSSL has app_data functions and in e.g. the verification |
| 46 | callbacks, you can retrieve the related SSL object. What we do is to set our |
| 47 | wrapper :py:class:`.Connection` object as app_data for the SSL object, and we can |
| 48 | easily find the Python callback. |
| 49 | |
| 50 | The other problem is solved using thread local variables. Whenever the GIL is |
| 51 | released before calling into an OpenSSL API, the PyThreadState pointer returned |
| 52 | by :c:func:`PyEval_SaveState` is stored in a global thread local variable |
| 53 | (using Python's own TLS API, :c:func:`PyThread_set_key_value`). When it is |
| 54 | necessary to re-acquire the GIL, either after the OpenSSL API returns or in a C |
| 55 | callback invoked by that OpenSSL API, the value of the thread local variable is |
Jonathan Ballet | 6381da3 | 2011-07-20 16:43:38 +0900 | [diff] [blame] | 56 | retrieved (:c:func:`PyThread_get_key_value`) and used to re-acquire the GIL. |
Jonathan Ballet | c7b7ba7 | 2011-07-16 15:29:47 +0900 | [diff] [blame] | 57 | This allows Python threads to execute while OpenSSL APIs are running and allows |
| 58 | use of any particular pyOpenSSL object from any Python thread, since there is |
| 59 | no per-thread state associated with any of these objects and since OpenSSL is |
| 60 | threadsafe (as long as properly initialized, as pyOpenSSL initializes it). |
| 61 | |
| 62 | |
| 63 | .. _socket-methods: |
| 64 | |
| 65 | Accessing Socket Methods |
| 66 | ------------------------ |
| 67 | |
| 68 | We quickly saw the benefit of wrapping socket methods in the |
| 69 | :py:class:`.SSL.Connection` class, for an easy transition into using SSL. The |
| 70 | problem here is that the :py:mod:`socket` module lacks a C API, and all the |
| 71 | methods are declared static. One approach would be to have :py:mod:`.OpenSSL` as |
| 72 | a submodule to the :py:mod:`socket` module, placing all the code in |
| 73 | ``socketmodule.c``, but this is obviously not a good solution, since you |
| 74 | might not want to import tonnes of extra stuff you're not going to use when |
| 75 | importing the :py:mod:`socket` module. The other approach is to somehow get a |
| 76 | pointer to the method to be called, either the C function, or a callable Python |
| 77 | object. This is not really a good solution either, since there's a lot of |
| 78 | lookups involved. |
| 79 | |
| 80 | The way it works is that you have to supply a :py:class:`socket`- **like** transport |
| 81 | object to the :py:class:`.SSL.Connection`. The only requirement of this object is |
| 82 | that it has a :py:meth:`fileno()` method that returns a file descriptor that's |
| 83 | valid at the C level (i.e. you can use the system calls read and write). If you |
| 84 | want to use the :py:meth:`connect()` or :py:meth:`accept()` methods of the |
| 85 | :py:class:`.SSL.Connection` object, the transport object has to supply such |
| 86 | methods too. Apart from them, any method lookups in the :py:class:`.SSL.Connection` |
| 87 | object that fail are passed on to the underlying transport object. |
| 88 | |
| 89 | Future changes might be to allow Python-level transport objects, that instead |
| 90 | of having :py:meth:`fileno()` methods, have :py:meth:`read()` and :py:meth:`write()` |
| 91 | methods, so more advanced features of Python can be used. This would probably |
| 92 | entail some sort of OpenSSL **BIOs**, but converting Python strings back and |
| 93 | forth is expensive, so this shouldn't be used unless necessary. Other nice |
| 94 | things would be to be able to pass in different transport objects for reading |
| 95 | and writing, but then the :py:meth:`fileno()` method of :py:class:`.SSL.Connection` |
| 96 | becomes virtually useless. Also, should the method resolution be used on the |
| 97 | read-transport or the write-transport? |