Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1 | .. highlightlang:: c |
| 2 | |
| 3 | |
| 4 | .. _extending-intro: |
| 5 | |
| 6 | ****************************** |
| 7 | Extending Python with C or C++ |
| 8 | ****************************** |
| 9 | |
| 10 | It is quite easy to add new built-in modules to Python, if you know how to |
| 11 | program in C. Such :dfn:`extension modules` can do two things that can't be |
| 12 | done directly in Python: they can implement new built-in object types, and they |
| 13 | can call C library functions and system calls. |
| 14 | |
| 15 | To support extensions, the Python API (Application Programmers Interface) |
| 16 | defines a set of functions, macros and variables that provide access to most |
| 17 | aspects of the Python run-time system. The Python API is incorporated in a C |
| 18 | source file by including the header ``"Python.h"``. |
| 19 | |
| 20 | The compilation of an extension module depends on its intended use as well as on |
| 21 | your system setup; details are given in later chapters. |
| 22 | |
Brett Cannon | 7f98a6c | 2009-09-17 03:39:33 +0000 | [diff] [blame] | 23 | Do note that if your use case is calling C library functions or system calls, |
| 24 | you should consider using the :mod:`ctypes` module rather than writing custom |
| 25 | C code. Not only does :mod:`ctypes` let you write Python code to interface |
| 26 | with C code, but it is more portable between implementations of Python than |
| 27 | writing and compiling an extension module which typically ties you to CPython. |
| 28 | |
| 29 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 30 | |
| 31 | .. _extending-simpleexample: |
| 32 | |
| 33 | A Simple Example |
| 34 | ================ |
| 35 | |
| 36 | Let's create an extension module called ``spam`` (the favorite food of Monty |
| 37 | Python fans...) and let's say we want to create a Python interface to the C |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 38 | library function :c:func:`system`. [#]_ This function takes a null-terminated |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 39 | character string as argument and returns an integer. We want this function to |
| 40 | be callable from Python as follows:: |
| 41 | |
| 42 | >>> import spam |
| 43 | >>> status = spam.system("ls -l") |
| 44 | |
| 45 | Begin by creating a file :file:`spammodule.c`. (Historically, if a module is |
| 46 | called ``spam``, the C file containing its implementation is called |
| 47 | :file:`spammodule.c`; if the module name is very long, like ``spammify``, the |
| 48 | module name can be just :file:`spammify.c`.) |
| 49 | |
| 50 | The first line of our file can be:: |
| 51 | |
| 52 | #include <Python.h> |
| 53 | |
| 54 | which pulls in the Python API (you can add a comment describing the purpose of |
| 55 | the module and a copyright notice if you like). |
| 56 | |
Georg Brandl | e720c0a | 2009-04-27 16:20:50 +0000 | [diff] [blame] | 57 | .. note:: |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 58 | |
| 59 | Since Python may define some pre-processor definitions which affect the standard |
| 60 | headers on some systems, you *must* include :file:`Python.h` before any standard |
| 61 | headers are included. |
| 62 | |
| 63 | All user-visible symbols defined by :file:`Python.h` have a prefix of ``Py`` or |
| 64 | ``PY``, except those defined in standard header files. For convenience, and |
| 65 | since they are used extensively by the Python interpreter, ``"Python.h"`` |
| 66 | includes a few standard header files: ``<stdio.h>``, ``<string.h>``, |
| 67 | ``<errno.h>``, and ``<stdlib.h>``. If the latter header file does not exist on |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 68 | your system, it declares the functions :c:func:`malloc`, :c:func:`free` and |
| 69 | :c:func:`realloc` directly. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 70 | |
| 71 | The next thing we add to our module file is the C function that will be called |
| 72 | when the Python expression ``spam.system(string)`` is evaluated (we'll see |
| 73 | shortly how it ends up being called):: |
| 74 | |
| 75 | static PyObject * |
| 76 | spam_system(PyObject *self, PyObject *args) |
| 77 | { |
| 78 | const char *command; |
| 79 | int sts; |
| 80 | |
| 81 | if (!PyArg_ParseTuple(args, "s", &command)) |
| 82 | return NULL; |
| 83 | sts = system(command); |
Georg Brandl | c877a7c | 2010-11-26 11:55:48 +0000 | [diff] [blame] | 84 | return PyLong_FromLong(sts); |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 85 | } |
| 86 | |
| 87 | There is a straightforward translation from the argument list in Python (for |
| 88 | example, the single expression ``"ls -l"``) to the arguments passed to the C |
| 89 | function. The C function always has two arguments, conventionally named *self* |
| 90 | and *args*. |
| 91 | |
Georg Brandl | 21dc5ba | 2009-07-11 10:43:08 +0000 | [diff] [blame] | 92 | The *self* argument points to the module object for module-level functions; |
| 93 | for a method it would point to the object instance. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 94 | |
| 95 | The *args* argument will be a pointer to a Python tuple object containing the |
| 96 | arguments. Each item of the tuple corresponds to an argument in the call's |
| 97 | argument list. The arguments are Python objects --- in order to do anything |
| 98 | with them in our C function we have to convert them to C values. The function |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 99 | :c:func:`PyArg_ParseTuple` in the Python API checks the argument types and |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 100 | converts them to C values. It uses a template string to determine the required |
| 101 | types of the arguments as well as the types of the C variables into which to |
| 102 | store the converted values. More about this later. |
| 103 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 104 | :c:func:`PyArg_ParseTuple` returns true (nonzero) if all arguments have the right |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 105 | type and its components have been stored in the variables whose addresses are |
| 106 | passed. It returns false (zero) if an invalid argument list was passed. In the |
| 107 | latter case it also raises an appropriate exception so the calling function can |
| 108 | return *NULL* immediately (as we saw in the example). |
| 109 | |
| 110 | |
| 111 | .. _extending-errors: |
| 112 | |
| 113 | Intermezzo: Errors and Exceptions |
| 114 | ================================= |
| 115 | |
| 116 | An important convention throughout the Python interpreter is the following: when |
| 117 | a function fails, it should set an exception condition and return an error value |
| 118 | (usually a *NULL* pointer). Exceptions are stored in a static global variable |
| 119 | inside the interpreter; if this variable is *NULL* no exception has occurred. A |
| 120 | second global variable stores the "associated value" of the exception (the |
| 121 | second argument to :keyword:`raise`). A third variable contains the stack |
| 122 | traceback in case the error originated in Python code. These three variables |
| 123 | are the C equivalents of the result in Python of :meth:`sys.exc_info` (see the |
| 124 | section on module :mod:`sys` in the Python Library Reference). It is important |
| 125 | to know about them to understand how errors are passed around. |
| 126 | |
| 127 | The Python API defines a number of functions to set various types of exceptions. |
| 128 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 129 | The most common one is :c:func:`PyErr_SetString`. Its arguments are an exception |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 130 | object and a C string. The exception object is usually a predefined object like |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 131 | :c:data:`PyExc_ZeroDivisionError`. The C string indicates the cause of the error |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 132 | and is converted to a Python string object and stored as the "associated value" |
| 133 | of the exception. |
| 134 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 135 | Another useful function is :c:func:`PyErr_SetFromErrno`, which only takes an |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 136 | exception argument and constructs the associated value by inspection of the |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 137 | global variable :c:data:`errno`. The most general function is |
| 138 | :c:func:`PyErr_SetObject`, which takes two object arguments, the exception and |
| 139 | its associated value. You don't need to :c:func:`Py_INCREF` the objects passed |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 140 | to any of these functions. |
| 141 | |
| 142 | You can test non-destructively whether an exception has been set with |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 143 | :c:func:`PyErr_Occurred`. This returns the current exception object, or *NULL* |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 144 | if no exception has occurred. You normally don't need to call |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 145 | :c:func:`PyErr_Occurred` to see whether an error occurred in a function call, |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 146 | since you should be able to tell from the return value. |
| 147 | |
| 148 | When a function *f* that calls another function *g* detects that the latter |
| 149 | fails, *f* should itself return an error value (usually *NULL* or ``-1``). It |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 150 | should *not* call one of the :c:func:`PyErr_\*` functions --- one has already |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 151 | been called by *g*. *f*'s caller is then supposed to also return an error |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 152 | indication to *its* caller, again *without* calling :c:func:`PyErr_\*`, and so on |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 153 | --- the most detailed cause of the error was already reported by the function |
| 154 | that first detected it. Once the error reaches the Python interpreter's main |
| 155 | loop, this aborts the currently executing Python code and tries to find an |
| 156 | exception handler specified by the Python programmer. |
| 157 | |
| 158 | (There are situations where a module can actually give a more detailed error |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 159 | message by calling another :c:func:`PyErr_\*` function, and in such cases it is |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 160 | fine to do so. As a general rule, however, this is not necessary, and can cause |
| 161 | information about the cause of the error to be lost: most operations can fail |
| 162 | for a variety of reasons.) |
| 163 | |
| 164 | To ignore an exception set by a function call that failed, the exception |
Georg Brandl | 682d7e0 | 2010-10-06 10:26:05 +0000 | [diff] [blame] | 165 | condition must be cleared explicitly by calling :c:func:`PyErr_Clear`. The only |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 166 | time C code should call :c:func:`PyErr_Clear` is if it doesn't want to pass the |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 167 | error on to the interpreter but wants to handle it completely by itself |
| 168 | (possibly by trying something else, or pretending nothing went wrong). |
| 169 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 170 | Every failing :c:func:`malloc` call must be turned into an exception --- the |
| 171 | direct caller of :c:func:`malloc` (or :c:func:`realloc`) must call |
| 172 | :c:func:`PyErr_NoMemory` and return a failure indicator itself. All the |
| 173 | object-creating functions (for example, :c:func:`PyLong_FromLong`) already do |
| 174 | this, so this note is only relevant to those who call :c:func:`malloc` directly. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 175 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 176 | Also note that, with the important exception of :c:func:`PyArg_ParseTuple` and |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 177 | friends, functions that return an integer status usually return a positive value |
| 178 | or zero for success and ``-1`` for failure, like Unix system calls. |
| 179 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 180 | Finally, be careful to clean up garbage (by making :c:func:`Py_XDECREF` or |
| 181 | :c:func:`Py_DECREF` calls for objects you have already created) when you return |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 182 | an error indicator! |
| 183 | |
| 184 | The choice of which exception to raise is entirely yours. There are predeclared |
| 185 | C objects corresponding to all built-in Python exceptions, such as |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 186 | :c:data:`PyExc_ZeroDivisionError`, which you can use directly. Of course, you |
| 187 | should choose exceptions wisely --- don't use :c:data:`PyExc_TypeError` to mean |
| 188 | that a file couldn't be opened (that should probably be :c:data:`PyExc_IOError`). |
| 189 | If something's wrong with the argument list, the :c:func:`PyArg_ParseTuple` |
| 190 | function usually raises :c:data:`PyExc_TypeError`. If you have an argument whose |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 191 | value must be in a particular range or must satisfy other conditions, |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 192 | :c:data:`PyExc_ValueError` is appropriate. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 193 | |
| 194 | You can also define a new exception that is unique to your module. For this, you |
| 195 | usually declare a static object variable at the beginning of your file:: |
| 196 | |
| 197 | static PyObject *SpamError; |
| 198 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 199 | and initialize it in your module's initialization function (:c:func:`PyInit_spam`) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 200 | with an exception object (leaving out the error checking for now):: |
| 201 | |
| 202 | PyMODINIT_FUNC |
Martin v. Löwis | 1a21451 | 2008-06-11 05:26:20 +0000 | [diff] [blame] | 203 | PyInit_spam(void) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 204 | { |
| 205 | PyObject *m; |
| 206 | |
Martin v. Löwis | 1a21451 | 2008-06-11 05:26:20 +0000 | [diff] [blame] | 207 | m = PyModule_Create(&spammodule); |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 208 | if (m == NULL) |
Martin v. Löwis | 1a21451 | 2008-06-11 05:26:20 +0000 | [diff] [blame] | 209 | return NULL; |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 210 | |
| 211 | SpamError = PyErr_NewException("spam.error", NULL, NULL); |
| 212 | Py_INCREF(SpamError); |
| 213 | PyModule_AddObject(m, "error", SpamError); |
Martin v. Löwis | 1a21451 | 2008-06-11 05:26:20 +0000 | [diff] [blame] | 214 | return m; |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 215 | } |
| 216 | |
| 217 | Note that the Python name for the exception object is :exc:`spam.error`. The |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 218 | :c:func:`PyErr_NewException` function may create a class with the base class |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 219 | being :exc:`Exception` (unless another class is passed in instead of *NULL*), |
| 220 | described in :ref:`bltin-exceptions`. |
| 221 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 222 | Note also that the :c:data:`SpamError` variable retains a reference to the newly |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 223 | created exception class; this is intentional! Since the exception could be |
| 224 | removed from the module by external code, an owned reference to the class is |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 225 | needed to ensure that it will not be discarded, causing :c:data:`SpamError` to |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 226 | become a dangling pointer. Should it become a dangling pointer, C code which |
| 227 | raises the exception could cause a core dump or other unintended side effects. |
| 228 | |
Georg Brandl | 9c491c9 | 2010-08-02 20:21:21 +0000 | [diff] [blame] | 229 | We discuss the use of ``PyMODINIT_FUNC`` as a function return type later in this |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 230 | sample. |
| 231 | |
Georg Brandl | 9c491c9 | 2010-08-02 20:21:21 +0000 | [diff] [blame] | 232 | The :exc:`spam.error` exception can be raised in your extension module using a |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 233 | call to :c:func:`PyErr_SetString` as shown below:: |
Georg Brandl | 9c491c9 | 2010-08-02 20:21:21 +0000 | [diff] [blame] | 234 | |
| 235 | static PyObject * |
| 236 | spam_system(PyObject *self, PyObject *args) |
| 237 | { |
| 238 | const char *command; |
| 239 | int sts; |
| 240 | |
| 241 | if (!PyArg_ParseTuple(args, "s", &command)) |
| 242 | return NULL; |
| 243 | sts = system(command); |
| 244 | if (sts < 0) { |
| 245 | PyErr_SetString(SpamError, "System command failed"); |
| 246 | return NULL; |
| 247 | } |
| 248 | return PyLong_FromLong(sts); |
| 249 | } |
| 250 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 251 | |
| 252 | .. _backtoexample: |
| 253 | |
| 254 | Back to the Example |
| 255 | =================== |
| 256 | |
| 257 | Going back to our example function, you should now be able to understand this |
| 258 | statement:: |
| 259 | |
| 260 | if (!PyArg_ParseTuple(args, "s", &command)) |
| 261 | return NULL; |
| 262 | |
| 263 | It returns *NULL* (the error indicator for functions returning object pointers) |
| 264 | if an error is detected in the argument list, relying on the exception set by |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 265 | :c:func:`PyArg_ParseTuple`. Otherwise the string value of the argument has been |
| 266 | copied to the local variable :c:data:`command`. This is a pointer assignment and |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 267 | you are not supposed to modify the string to which it points (so in Standard C, |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 268 | the variable :c:data:`command` should properly be declared as ``const char |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 269 | *command``). |
| 270 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 271 | The next statement is a call to the Unix function :c:func:`system`, passing it |
| 272 | the string we just got from :c:func:`PyArg_ParseTuple`:: |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 273 | |
| 274 | sts = system(command); |
| 275 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 276 | Our :func:`spam.system` function must return the value of :c:data:`sts` as a |
Georg Brandl | c877a7c | 2010-11-26 11:55:48 +0000 | [diff] [blame] | 277 | Python object. This is done using the function :c:func:`PyLong_FromLong`. :: |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 278 | |
Georg Brandl | c877a7c | 2010-11-26 11:55:48 +0000 | [diff] [blame] | 279 | return PyLong_FromLong(sts); |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 280 | |
| 281 | In this case, it will return an integer object. (Yes, even integers are objects |
| 282 | on the heap in Python!) |
| 283 | |
| 284 | If you have a C function that returns no useful argument (a function returning |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 285 | :c:type:`void`), the corresponding Python function must return ``None``. You |
| 286 | need this idiom to do so (which is implemented by the :c:macro:`Py_RETURN_NONE` |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 287 | macro):: |
| 288 | |
| 289 | Py_INCREF(Py_None); |
| 290 | return Py_None; |
| 291 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 292 | :c:data:`Py_None` is the C name for the special Python object ``None``. It is a |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 293 | genuine Python object rather than a *NULL* pointer, which means "error" in most |
| 294 | contexts, as we have seen. |
| 295 | |
| 296 | |
| 297 | .. _methodtable: |
| 298 | |
| 299 | The Module's Method Table and Initialization Function |
| 300 | ===================================================== |
| 301 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 302 | I promised to show how :c:func:`spam_system` is called from Python programs. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 303 | First, we need to list its name and address in a "method table":: |
| 304 | |
| 305 | static PyMethodDef SpamMethods[] = { |
| 306 | ... |
| 307 | {"system", spam_system, METH_VARARGS, |
| 308 | "Execute a shell command."}, |
| 309 | ... |
| 310 | {NULL, NULL, 0, NULL} /* Sentinel */ |
| 311 | }; |
| 312 | |
| 313 | Note the third entry (``METH_VARARGS``). This is a flag telling the interpreter |
| 314 | the calling convention to be used for the C function. It should normally always |
| 315 | be ``METH_VARARGS`` or ``METH_VARARGS | METH_KEYWORDS``; a value of ``0`` means |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 316 | that an obsolete variant of :c:func:`PyArg_ParseTuple` is used. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 317 | |
| 318 | When using only ``METH_VARARGS``, the function should expect the Python-level |
| 319 | parameters to be passed in as a tuple acceptable for parsing via |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 320 | :c:func:`PyArg_ParseTuple`; more information on this function is provided below. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 321 | |
| 322 | The :const:`METH_KEYWORDS` bit may be set in the third field if keyword |
| 323 | arguments should be passed to the function. In this case, the C function should |
Eli Bendersky | 44fb613 | 2012-02-11 10:27:31 +0200 | [diff] [blame] | 324 | accept a third ``PyObject *`` parameter which will be a dictionary of keywords. |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 325 | Use :c:func:`PyArg_ParseTupleAndKeywords` to parse the arguments to such a |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 326 | function. |
| 327 | |
Martin v. Löwis | 1a21451 | 2008-06-11 05:26:20 +0000 | [diff] [blame] | 328 | The method table must be referenced in the module definition structure:: |
| 329 | |
Benjamin Peterson | 3851d12 | 2008-10-20 21:04:06 +0000 | [diff] [blame] | 330 | static struct PyModuleDef spammodule = { |
Martin v. Löwis | 1a21451 | 2008-06-11 05:26:20 +0000 | [diff] [blame] | 331 | PyModuleDef_HEAD_INIT, |
| 332 | "spam", /* name of module */ |
| 333 | spam_doc, /* module documentation, may be NULL */ |
| 334 | -1, /* size of per-interpreter state of the module, |
| 335 | or -1 if the module keeps state in global variables. */ |
| 336 | SpamMethods |
| 337 | }; |
| 338 | |
| 339 | This structure, in turn, must be passed to the interpreter in the module's |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 340 | initialization function. The initialization function must be named |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 341 | :c:func:`PyInit_name`, where *name* is the name of the module, and should be the |
Christian Heimes | 5b5e81c | 2007-12-31 16:14:33 +0000 | [diff] [blame] | 342 | only non-\ ``static`` item defined in the module file:: |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 343 | |
| 344 | PyMODINIT_FUNC |
Martin v. Löwis | 1a21451 | 2008-06-11 05:26:20 +0000 | [diff] [blame] | 345 | PyInit_spam(void) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 346 | { |
Martin v. Löwis | 1a21451 | 2008-06-11 05:26:20 +0000 | [diff] [blame] | 347 | return PyModule_Create(&spammodule); |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 348 | } |
| 349 | |
Benjamin Peterson | 71e30a0 | 2008-12-24 16:27:25 +0000 | [diff] [blame] | 350 | Note that PyMODINIT_FUNC declares the function as ``PyObject *`` return type, |
| 351 | declares any special linkage declarations required by the platform, and for C++ |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 352 | declares the function as ``extern "C"``. |
| 353 | |
| 354 | When the Python program imports module :mod:`spam` for the first time, |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 355 | :c:func:`PyInit_spam` is called. (See below for comments about embedding Python.) |
| 356 | It calls :c:func:`PyModule_Create`, which returns a module object, and |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 357 | inserts built-in function objects into the newly created module based upon the |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 358 | table (an array of :c:type:`PyMethodDef` structures) found in the module definition. |
| 359 | :c:func:`PyModule_Create` returns a pointer to the module object |
Martin v. Löwis | 1a21451 | 2008-06-11 05:26:20 +0000 | [diff] [blame] | 360 | that it creates. It may abort with a fatal error for |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 361 | certain errors, or return *NULL* if the module could not be initialized |
Martin v. Löwis | 1a21451 | 2008-06-11 05:26:20 +0000 | [diff] [blame] | 362 | satisfactorily. The init function must return the module object to its caller, |
| 363 | so that it then gets inserted into ``sys.modules``. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 364 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 365 | When embedding Python, the :c:func:`PyInit_spam` function is not called |
| 366 | automatically unless there's an entry in the :c:data:`PyImport_Inittab` table. |
| 367 | To add the module to the initialization table, use :c:func:`PyImport_AppendInittab`, |
Martin v. Löwis | 1a21451 | 2008-06-11 05:26:20 +0000 | [diff] [blame] | 368 | optionally followed by an import of the module:: |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 369 | |
| 370 | int |
| 371 | main(int argc, char *argv[]) |
| 372 | { |
Victor Stinner | 25e014b | 2014-08-01 12:28:49 +0200 | [diff] [blame] | 373 | wchar_t *program = Py_DecodeLocale(argv[0], NULL); |
| 374 | if (program == NULL) { |
| 375 | fprintf(stderr, "Fatal error: cannot decode argv[0]\n"); |
| 376 | exit(1); |
| 377 | } |
| 378 | |
Georg Brandl | c4a55fc | 2010-02-06 18:46:57 +0000 | [diff] [blame] | 379 | /* Add a built-in module, before Py_Initialize */ |
Martin v. Löwis | 1a21451 | 2008-06-11 05:26:20 +0000 | [diff] [blame] | 380 | PyImport_AppendInittab("spam", PyInit_spam); |
| 381 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 382 | /* Pass argv[0] to the Python interpreter */ |
Victor Stinner | 25e014b | 2014-08-01 12:28:49 +0200 | [diff] [blame] | 383 | Py_SetProgramName(program); |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 384 | |
| 385 | /* Initialize the Python interpreter. Required. */ |
| 386 | Py_Initialize(); |
| 387 | |
Martin v. Löwis | 1a21451 | 2008-06-11 05:26:20 +0000 | [diff] [blame] | 388 | /* Optionally import the module; alternatively, |
| 389 | import can be deferred until the embedded script |
| 390 | imports it. */ |
| 391 | PyImport_ImportModule("spam"); |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 392 | |
Georg Brandl | 49c6fc9 | 2013-10-06 13:14:10 +0200 | [diff] [blame] | 393 | ... |
| 394 | |
Victor Stinner | 25e014b | 2014-08-01 12:28:49 +0200 | [diff] [blame] | 395 | PyMem_RawFree(program); |
| 396 | return 0; |
| 397 | } |
| 398 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 399 | .. note:: |
| 400 | |
| 401 | Removing entries from ``sys.modules`` or importing compiled modules into |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 402 | multiple interpreters within a process (or following a :c:func:`fork` without an |
| 403 | intervening :c:func:`exec`) can create problems for some extension modules. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 404 | Extension module authors should exercise caution when initializing internal data |
| 405 | structures. |
| 406 | |
| 407 | A more substantial example module is included in the Python source distribution |
| 408 | as :file:`Modules/xxmodule.c`. This file may be used as a template or simply |
Benjamin Peterson | 2614cda | 2010-03-21 22:36:19 +0000 | [diff] [blame] | 409 | read as an example. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 410 | |
| 411 | |
| 412 | .. _compilation: |
| 413 | |
| 414 | Compilation and Linkage |
| 415 | ======================= |
| 416 | |
| 417 | There are two more things to do before you can use your new extension: compiling |
| 418 | and linking it with the Python system. If you use dynamic loading, the details |
| 419 | may depend on the style of dynamic loading your system uses; see the chapters |
| 420 | about building extension modules (chapter :ref:`building`) and additional |
| 421 | information that pertains only to building on Windows (chapter |
| 422 | :ref:`building-on-windows`) for more information about this. |
| 423 | |
| 424 | If you can't use dynamic loading, or if you want to make your module a permanent |
| 425 | part of the Python interpreter, you will have to change the configuration setup |
| 426 | and rebuild the interpreter. Luckily, this is very simple on Unix: just place |
| 427 | your file (:file:`spammodule.c` for example) in the :file:`Modules/` directory |
| 428 | of an unpacked source distribution, add a line to the file |
| 429 | :file:`Modules/Setup.local` describing your file:: |
| 430 | |
| 431 | spam spammodule.o |
| 432 | |
| 433 | and rebuild the interpreter by running :program:`make` in the toplevel |
| 434 | directory. You can also run :program:`make` in the :file:`Modules/` |
| 435 | subdirectory, but then you must first rebuild :file:`Makefile` there by running |
| 436 | ':program:`make` Makefile'. (This is necessary each time you change the |
| 437 | :file:`Setup` file.) |
| 438 | |
| 439 | If your module requires additional libraries to link with, these can be listed |
| 440 | on the line in the configuration file as well, for instance:: |
| 441 | |
| 442 | spam spammodule.o -lX11 |
| 443 | |
| 444 | |
| 445 | .. _callingpython: |
| 446 | |
| 447 | Calling Python Functions from C |
| 448 | =============================== |
| 449 | |
| 450 | So far we have concentrated on making C functions callable from Python. The |
| 451 | reverse is also useful: calling Python functions from C. This is especially the |
| 452 | case for libraries that support so-called "callback" functions. If a C |
| 453 | interface makes use of callbacks, the equivalent Python often needs to provide a |
| 454 | callback mechanism to the Python programmer; the implementation will require |
| 455 | calling the Python callback functions from a C callback. Other uses are also |
| 456 | imaginable. |
| 457 | |
| 458 | Fortunately, the Python interpreter is easily called recursively, and there is a |
| 459 | standard interface to call a Python function. (I won't dwell on how to call the |
| 460 | Python parser with a particular string as input --- if you're interested, have a |
| 461 | look at the implementation of the :option:`-c` command line option in |
Georg Brandl | 22291c5 | 2007-09-06 14:49:02 +0000 | [diff] [blame] | 462 | :file:`Modules/main.c` from the Python source code.) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 463 | |
| 464 | Calling a Python function is easy. First, the Python program must somehow pass |
| 465 | you the Python function object. You should provide a function (or some other |
| 466 | interface) to do this. When this function is called, save a pointer to the |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 467 | Python function object (be careful to :c:func:`Py_INCREF` it!) in a global |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 468 | variable --- or wherever you see fit. For example, the following function might |
| 469 | be part of a module definition:: |
| 470 | |
| 471 | static PyObject *my_callback = NULL; |
| 472 | |
| 473 | static PyObject * |
| 474 | my_set_callback(PyObject *dummy, PyObject *args) |
| 475 | { |
| 476 | PyObject *result = NULL; |
| 477 | PyObject *temp; |
| 478 | |
| 479 | if (PyArg_ParseTuple(args, "O:set_callback", &temp)) { |
| 480 | if (!PyCallable_Check(temp)) { |
| 481 | PyErr_SetString(PyExc_TypeError, "parameter must be callable"); |
| 482 | return NULL; |
| 483 | } |
| 484 | Py_XINCREF(temp); /* Add a reference to new callback */ |
| 485 | Py_XDECREF(my_callback); /* Dispose of previous callback */ |
| 486 | my_callback = temp; /* Remember new callback */ |
| 487 | /* Boilerplate to return "None" */ |
| 488 | Py_INCREF(Py_None); |
| 489 | result = Py_None; |
| 490 | } |
| 491 | return result; |
| 492 | } |
| 493 | |
| 494 | This function must be registered with the interpreter using the |
| 495 | :const:`METH_VARARGS` flag; this is described in section :ref:`methodtable`. The |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 496 | :c:func:`PyArg_ParseTuple` function and its arguments are documented in section |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 497 | :ref:`parsetuple`. |
| 498 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 499 | The macros :c:func:`Py_XINCREF` and :c:func:`Py_XDECREF` increment/decrement the |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 500 | reference count of an object and are safe in the presence of *NULL* pointers |
| 501 | (but note that *temp* will not be *NULL* in this context). More info on them |
| 502 | in section :ref:`refcounts`. |
| 503 | |
Benjamin Peterson | d23f822 | 2009-04-05 19:13:16 +0000 | [diff] [blame] | 504 | .. index:: single: PyObject_CallObject() |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 505 | |
| 506 | Later, when it is time to call the function, you call the C function |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 507 | :c:func:`PyObject_CallObject`. This function has two arguments, both pointers to |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 508 | arbitrary Python objects: the Python function, and the argument list. The |
| 509 | argument list must always be a tuple object, whose length is the number of |
Georg Brandl | 48310cd | 2009-01-03 21:18:54 +0000 | [diff] [blame] | 510 | arguments. To call the Python function with no arguments, pass in NULL, or |
Christian Heimes | d8654cf | 2007-12-02 15:22:16 +0000 | [diff] [blame] | 511 | an empty tuple; to call it with one argument, pass a singleton tuple. |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 512 | :c:func:`Py_BuildValue` returns a tuple when its format string consists of zero |
Christian Heimes | d8654cf | 2007-12-02 15:22:16 +0000 | [diff] [blame] | 513 | or more format codes between parentheses. For example:: |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 514 | |
| 515 | int arg; |
| 516 | PyObject *arglist; |
| 517 | PyObject *result; |
| 518 | ... |
| 519 | arg = 123; |
| 520 | ... |
| 521 | /* Time to call the callback */ |
| 522 | arglist = Py_BuildValue("(i)", arg); |
Benjamin Peterson | d23f822 | 2009-04-05 19:13:16 +0000 | [diff] [blame] | 523 | result = PyObject_CallObject(my_callback, arglist); |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 524 | Py_DECREF(arglist); |
| 525 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 526 | :c:func:`PyObject_CallObject` returns a Python object pointer: this is the return |
| 527 | value of the Python function. :c:func:`PyObject_CallObject` is |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 528 | "reference-count-neutral" with respect to its arguments. In the example a new |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 529 | tuple was created to serve as the argument list, which is :c:func:`Py_DECREF`\ |
Georg Brandl | 337672b | 2013-10-06 11:02:38 +0200 | [diff] [blame] | 530 | -ed immediately after the :c:func:`PyObject_CallObject` call. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 531 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 532 | The return value of :c:func:`PyObject_CallObject` is "new": either it is a brand |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 533 | new object, or it is an existing object whose reference count has been |
| 534 | incremented. So, unless you want to save it in a global variable, you should |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 535 | somehow :c:func:`Py_DECREF` the result, even (especially!) if you are not |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 536 | interested in its value. |
| 537 | |
| 538 | Before you do this, however, it is important to check that the return value |
| 539 | isn't *NULL*. If it is, the Python function terminated by raising an exception. |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 540 | If the C code that called :c:func:`PyObject_CallObject` is called from Python, it |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 541 | should now return an error indication to its Python caller, so the interpreter |
| 542 | can print a stack trace, or the calling Python code can handle the exception. |
| 543 | If this is not possible or desirable, the exception should be cleared by calling |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 544 | :c:func:`PyErr_Clear`. For example:: |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 545 | |
| 546 | if (result == NULL) |
| 547 | return NULL; /* Pass error back */ |
| 548 | ...use result... |
Georg Brandl | 48310cd | 2009-01-03 21:18:54 +0000 | [diff] [blame] | 549 | Py_DECREF(result); |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 550 | |
| 551 | Depending on the desired interface to the Python callback function, you may also |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 552 | have to provide an argument list to :c:func:`PyObject_CallObject`. In some cases |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 553 | the argument list is also provided by the Python program, through the same |
| 554 | interface that specified the callback function. It can then be saved and used |
| 555 | in the same manner as the function object. In other cases, you may have to |
| 556 | construct a new tuple to pass as the argument list. The simplest way to do this |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 557 | is to call :c:func:`Py_BuildValue`. For example, if you want to pass an integral |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 558 | event code, you might use the following code:: |
| 559 | |
| 560 | PyObject *arglist; |
| 561 | ... |
| 562 | arglist = Py_BuildValue("(l)", eventcode); |
Benjamin Peterson | d23f822 | 2009-04-05 19:13:16 +0000 | [diff] [blame] | 563 | result = PyObject_CallObject(my_callback, arglist); |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 564 | Py_DECREF(arglist); |
| 565 | if (result == NULL) |
| 566 | return NULL; /* Pass error back */ |
| 567 | /* Here maybe use the result */ |
| 568 | Py_DECREF(result); |
| 569 | |
| 570 | Note the placement of ``Py_DECREF(arglist)`` immediately after the call, before |
Christian Heimes | d8654cf | 2007-12-02 15:22:16 +0000 | [diff] [blame] | 571 | the error check! Also note that strictly speaking this code is not complete: |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 572 | :c:func:`Py_BuildValue` may run out of memory, and this should be checked. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 573 | |
Georg Brandl | 48310cd | 2009-01-03 21:18:54 +0000 | [diff] [blame] | 574 | You may also call a function with keyword arguments by using |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 575 | :c:func:`PyObject_Call`, which supports arguments and keyword arguments. As in |
| 576 | the above example, we use :c:func:`Py_BuildValue` to construct the dictionary. :: |
Christian Heimes | d8654cf | 2007-12-02 15:22:16 +0000 | [diff] [blame] | 577 | |
| 578 | PyObject *dict; |
| 579 | ... |
| 580 | dict = Py_BuildValue("{s:i}", "name", val); |
Benjamin Peterson | d23f822 | 2009-04-05 19:13:16 +0000 | [diff] [blame] | 581 | result = PyObject_Call(my_callback, NULL, dict); |
Christian Heimes | d8654cf | 2007-12-02 15:22:16 +0000 | [diff] [blame] | 582 | Py_DECREF(dict); |
| 583 | if (result == NULL) |
| 584 | return NULL; /* Pass error back */ |
| 585 | /* Here maybe use the result */ |
| 586 | Py_DECREF(result); |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 587 | |
Benjamin Peterson | d23f822 | 2009-04-05 19:13:16 +0000 | [diff] [blame] | 588 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 589 | .. _parsetuple: |
| 590 | |
| 591 | Extracting Parameters in Extension Functions |
| 592 | ============================================ |
| 593 | |
| 594 | .. index:: single: PyArg_ParseTuple() |
| 595 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 596 | The :c:func:`PyArg_ParseTuple` function is declared as follows:: |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 597 | |
| 598 | int PyArg_ParseTuple(PyObject *arg, char *format, ...); |
| 599 | |
| 600 | The *arg* argument must be a tuple object containing an argument list passed |
| 601 | from Python to a C function. The *format* argument must be a format string, |
| 602 | whose syntax is explained in :ref:`arg-parsing` in the Python/C API Reference |
| 603 | Manual. The remaining arguments must be addresses of variables whose type is |
| 604 | determined by the format string. |
| 605 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 606 | Note that while :c:func:`PyArg_ParseTuple` checks that the Python arguments have |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 607 | the required types, it cannot check the validity of the addresses of C variables |
| 608 | passed to the call: if you make mistakes there, your code will probably crash or |
| 609 | at least overwrite random bits in memory. So be careful! |
| 610 | |
| 611 | Note that any Python object references which are provided to the caller are |
| 612 | *borrowed* references; do not decrement their reference count! |
| 613 | |
| 614 | Some example calls:: |
| 615 | |
Gregory P. Smith | 02c3b5c | 2008-11-23 23:49:16 +0000 | [diff] [blame] | 616 | #define PY_SSIZE_T_CLEAN /* Make "s#" use Py_ssize_t rather than int. */ |
| 617 | #include <Python.h> |
| 618 | |
| 619 | :: |
| 620 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 621 | int ok; |
| 622 | int i, j; |
| 623 | long k, l; |
| 624 | const char *s; |
Gregory P. Smith | 02c3b5c | 2008-11-23 23:49:16 +0000 | [diff] [blame] | 625 | Py_ssize_t size; |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 626 | |
| 627 | ok = PyArg_ParseTuple(args, ""); /* No arguments */ |
| 628 | /* Python call: f() */ |
| 629 | |
| 630 | :: |
| 631 | |
| 632 | ok = PyArg_ParseTuple(args, "s", &s); /* A string */ |
| 633 | /* Possible Python call: f('whoops!') */ |
| 634 | |
| 635 | :: |
| 636 | |
| 637 | ok = PyArg_ParseTuple(args, "lls", &k, &l, &s); /* Two longs and a string */ |
| 638 | /* Possible Python call: f(1, 2, 'three') */ |
| 639 | |
| 640 | :: |
| 641 | |
| 642 | ok = PyArg_ParseTuple(args, "(ii)s#", &i, &j, &s, &size); |
| 643 | /* A pair of ints and a string, whose size is also returned */ |
| 644 | /* Possible Python call: f((1, 2), 'three') */ |
| 645 | |
| 646 | :: |
| 647 | |
| 648 | { |
| 649 | const char *file; |
| 650 | const char *mode = "r"; |
| 651 | int bufsize = 0; |
| 652 | ok = PyArg_ParseTuple(args, "s|si", &file, &mode, &bufsize); |
| 653 | /* A string, and optionally another string and an integer */ |
| 654 | /* Possible Python calls: |
| 655 | f('spam') |
| 656 | f('spam', 'w') |
| 657 | f('spam', 'wb', 100000) */ |
| 658 | } |
| 659 | |
| 660 | :: |
| 661 | |
| 662 | { |
| 663 | int left, top, right, bottom, h, v; |
| 664 | ok = PyArg_ParseTuple(args, "((ii)(ii))(ii)", |
| 665 | &left, &top, &right, &bottom, &h, &v); |
| 666 | /* A rectangle and a point */ |
| 667 | /* Possible Python call: |
| 668 | f(((0, 0), (400, 300)), (10, 10)) */ |
| 669 | } |
| 670 | |
| 671 | :: |
| 672 | |
| 673 | { |
| 674 | Py_complex c; |
| 675 | ok = PyArg_ParseTuple(args, "D:myfunction", &c); |
| 676 | /* a complex, also providing a function name for errors */ |
| 677 | /* Possible Python call: myfunction(1+2j) */ |
| 678 | } |
| 679 | |
| 680 | |
| 681 | .. _parsetupleandkeywords: |
| 682 | |
| 683 | Keyword Parameters for Extension Functions |
| 684 | ========================================== |
| 685 | |
| 686 | .. index:: single: PyArg_ParseTupleAndKeywords() |
| 687 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 688 | The :c:func:`PyArg_ParseTupleAndKeywords` function is declared as follows:: |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 689 | |
| 690 | int PyArg_ParseTupleAndKeywords(PyObject *arg, PyObject *kwdict, |
| 691 | char *format, char *kwlist[], ...); |
| 692 | |
| 693 | The *arg* and *format* parameters are identical to those of the |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 694 | :c:func:`PyArg_ParseTuple` function. The *kwdict* parameter is the dictionary of |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 695 | keywords received as the third parameter from the Python runtime. The *kwlist* |
| 696 | parameter is a *NULL*-terminated list of strings which identify the parameters; |
| 697 | the names are matched with the type information from *format* from left to |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 698 | right. On success, :c:func:`PyArg_ParseTupleAndKeywords` returns true, otherwise |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 699 | it returns false and raises an appropriate exception. |
| 700 | |
| 701 | .. note:: |
| 702 | |
| 703 | Nested tuples cannot be parsed when using keyword arguments! Keyword parameters |
| 704 | passed in which are not present in the *kwlist* will cause :exc:`TypeError` to |
| 705 | be raised. |
| 706 | |
| 707 | .. index:: single: Philbrick, Geoff |
| 708 | |
| 709 | Here is an example module which uses keywords, based on an example by Geoff |
Christian Heimes | 5b5e81c | 2007-12-31 16:14:33 +0000 | [diff] [blame] | 710 | Philbrick (philbrick@hks.com):: |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 711 | |
| 712 | #include "Python.h" |
| 713 | |
| 714 | static PyObject * |
| 715 | keywdarg_parrot(PyObject *self, PyObject *args, PyObject *keywds) |
Georg Brandl | 48310cd | 2009-01-03 21:18:54 +0000 | [diff] [blame] | 716 | { |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 717 | int voltage; |
| 718 | char *state = "a stiff"; |
| 719 | char *action = "voom"; |
| 720 | char *type = "Norwegian Blue"; |
| 721 | |
| 722 | static char *kwlist[] = {"voltage", "state", "action", "type", NULL}; |
| 723 | |
Georg Brandl | 48310cd | 2009-01-03 21:18:54 +0000 | [diff] [blame] | 724 | if (!PyArg_ParseTupleAndKeywords(args, keywds, "i|sss", kwlist, |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 725 | &voltage, &state, &action, &type)) |
Georg Brandl | 48310cd | 2009-01-03 21:18:54 +0000 | [diff] [blame] | 726 | return NULL; |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 727 | |
Georg Brandl | 48310cd | 2009-01-03 21:18:54 +0000 | [diff] [blame] | 728 | printf("-- This parrot wouldn't %s if you put %i Volts through it.\n", |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 729 | action, voltage); |
| 730 | printf("-- Lovely plumage, the %s -- It's %s!\n", type, state); |
| 731 | |
Georg Brandl | a072de1 | 2013-10-06 20:46:08 +0200 | [diff] [blame] | 732 | Py_RETURN_NONE; |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 733 | } |
| 734 | |
| 735 | static PyMethodDef keywdarg_methods[] = { |
| 736 | /* The cast of the function is necessary since PyCFunction values |
| 737 | * only take two PyObject* parameters, and keywdarg_parrot() takes |
| 738 | * three. |
| 739 | */ |
| 740 | {"parrot", (PyCFunction)keywdarg_parrot, METH_VARARGS | METH_KEYWORDS, |
| 741 | "Print a lovely skit to standard output."}, |
| 742 | {NULL, NULL, 0, NULL} /* sentinel */ |
| 743 | }; |
| 744 | |
Eli Bendersky | 8f77349 | 2012-08-15 14:49:49 +0300 | [diff] [blame] | 745 | static struct PyModuleDef keywdargmodule = { |
| 746 | PyModuleDef_HEAD_INIT, |
| 747 | "keywdarg", |
| 748 | NULL, |
| 749 | -1, |
| 750 | keywdarg_methods |
| 751 | }; |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 752 | |
Eli Bendersky | 8f77349 | 2012-08-15 14:49:49 +0300 | [diff] [blame] | 753 | PyMODINIT_FUNC |
| 754 | PyInit_keywdarg(void) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 755 | { |
Eli Bendersky | 8f77349 | 2012-08-15 14:49:49 +0300 | [diff] [blame] | 756 | return PyModule_Create(&keywdargmodule); |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 757 | } |
| 758 | |
| 759 | |
| 760 | .. _buildvalue: |
| 761 | |
| 762 | Building Arbitrary Values |
| 763 | ========================= |
| 764 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 765 | This function is the counterpart to :c:func:`PyArg_ParseTuple`. It is declared |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 766 | as follows:: |
| 767 | |
| 768 | PyObject *Py_BuildValue(char *format, ...); |
| 769 | |
| 770 | It recognizes a set of format units similar to the ones recognized by |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 771 | :c:func:`PyArg_ParseTuple`, but the arguments (which are input to the function, |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 772 | not output) must not be pointers, just values. It returns a new Python object, |
| 773 | suitable for returning from a C function called from Python. |
| 774 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 775 | One difference with :c:func:`PyArg_ParseTuple`: while the latter requires its |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 776 | first argument to be a tuple (since Python argument lists are always represented |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 777 | as tuples internally), :c:func:`Py_BuildValue` does not always build a tuple. It |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 778 | builds a tuple only if its format string contains two or more format units. If |
| 779 | the format string is empty, it returns ``None``; if it contains exactly one |
| 780 | format unit, it returns whatever object is described by that format unit. To |
| 781 | force it to return a tuple of size 0 or one, parenthesize the format string. |
| 782 | |
| 783 | Examples (to the left the call, to the right the resulting Python value):: |
| 784 | |
| 785 | Py_BuildValue("") None |
| 786 | Py_BuildValue("i", 123) 123 |
| 787 | Py_BuildValue("iii", 123, 456, 789) (123, 456, 789) |
| 788 | Py_BuildValue("s", "hello") 'hello' |
| 789 | Py_BuildValue("y", "hello") b'hello' |
| 790 | Py_BuildValue("ss", "hello", "world") ('hello', 'world') |
| 791 | Py_BuildValue("s#", "hello", 4) 'hell' |
| 792 | Py_BuildValue("y#", "hello", 4) b'hell' |
| 793 | Py_BuildValue("()") () |
| 794 | Py_BuildValue("(i)", 123) (123,) |
| 795 | Py_BuildValue("(ii)", 123, 456) (123, 456) |
| 796 | Py_BuildValue("(i,i)", 123, 456) (123, 456) |
| 797 | Py_BuildValue("[i,i]", 123, 456) [123, 456] |
| 798 | Py_BuildValue("{s:i,s:i}", |
| 799 | "abc", 123, "def", 456) {'abc': 123, 'def': 456} |
| 800 | Py_BuildValue("((ii)(ii)) (ii)", |
| 801 | 1, 2, 3, 4, 5, 6) (((1, 2), (3, 4)), (5, 6)) |
| 802 | |
| 803 | |
| 804 | .. _refcounts: |
| 805 | |
| 806 | Reference Counts |
| 807 | ================ |
| 808 | |
| 809 | In languages like C or C++, the programmer is responsible for dynamic allocation |
| 810 | and deallocation of memory on the heap. In C, this is done using the functions |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 811 | :c:func:`malloc` and :c:func:`free`. In C++, the operators ``new`` and |
Christian Heimes | 5b5e81c | 2007-12-31 16:14:33 +0000 | [diff] [blame] | 812 | ``delete`` are used with essentially the same meaning and we'll restrict |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 813 | the following discussion to the C case. |
| 814 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 815 | Every block of memory allocated with :c:func:`malloc` should eventually be |
| 816 | returned to the pool of available memory by exactly one call to :c:func:`free`. |
| 817 | It is important to call :c:func:`free` at the right time. If a block's address |
| 818 | is forgotten but :c:func:`free` is not called for it, the memory it occupies |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 819 | cannot be reused until the program terminates. This is called a :dfn:`memory |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 820 | leak`. On the other hand, if a program calls :c:func:`free` for a block and then |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 821 | continues to use the block, it creates a conflict with re-use of the block |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 822 | through another :c:func:`malloc` call. This is called :dfn:`using freed memory`. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 823 | It has the same bad consequences as referencing uninitialized data --- core |
| 824 | dumps, wrong results, mysterious crashes. |
| 825 | |
| 826 | Common causes of memory leaks are unusual paths through the code. For instance, |
| 827 | a function may allocate a block of memory, do some calculation, and then free |
| 828 | the block again. Now a change in the requirements for the function may add a |
| 829 | test to the calculation that detects an error condition and can return |
| 830 | prematurely from the function. It's easy to forget to free the allocated memory |
| 831 | block when taking this premature exit, especially when it is added later to the |
| 832 | code. Such leaks, once introduced, often go undetected for a long time: the |
| 833 | error exit is taken only in a small fraction of all calls, and most modern |
| 834 | machines have plenty of virtual memory, so the leak only becomes apparent in a |
| 835 | long-running process that uses the leaking function frequently. Therefore, it's |
| 836 | important to prevent leaks from happening by having a coding convention or |
| 837 | strategy that minimizes this kind of errors. |
| 838 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 839 | Since Python makes heavy use of :c:func:`malloc` and :c:func:`free`, it needs a |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 840 | strategy to avoid memory leaks as well as the use of freed memory. The chosen |
| 841 | method is called :dfn:`reference counting`. The principle is simple: every |
| 842 | object contains a counter, which is incremented when a reference to the object |
| 843 | is stored somewhere, and which is decremented when a reference to it is deleted. |
| 844 | When the counter reaches zero, the last reference to the object has been deleted |
| 845 | and the object is freed. |
| 846 | |
| 847 | An alternative strategy is called :dfn:`automatic garbage collection`. |
| 848 | (Sometimes, reference counting is also referred to as a garbage collection |
| 849 | strategy, hence my use of "automatic" to distinguish the two.) The big |
| 850 | advantage of automatic garbage collection is that the user doesn't need to call |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 851 | :c:func:`free` explicitly. (Another claimed advantage is an improvement in speed |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 852 | or memory usage --- this is no hard fact however.) The disadvantage is that for |
| 853 | C, there is no truly portable automatic garbage collector, while reference |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 854 | counting can be implemented portably (as long as the functions :c:func:`malloc` |
| 855 | and :c:func:`free` are available --- which the C Standard guarantees). Maybe some |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 856 | day a sufficiently portable automatic garbage collector will be available for C. |
| 857 | Until then, we'll have to live with reference counts. |
| 858 | |
| 859 | While Python uses the traditional reference counting implementation, it also |
| 860 | offers a cycle detector that works to detect reference cycles. This allows |
| 861 | applications to not worry about creating direct or indirect circular references; |
| 862 | these are the weakness of garbage collection implemented using only reference |
| 863 | counting. Reference cycles consist of objects which contain (possibly indirect) |
| 864 | references to themselves, so that each object in the cycle has a reference count |
| 865 | which is non-zero. Typical reference counting implementations are not able to |
| 866 | reclaim the memory belonging to any objects in a reference cycle, or referenced |
| 867 | from the objects in the cycle, even though there are no further references to |
| 868 | the cycle itself. |
| 869 | |
Georg Brandl | a4c8c47 | 2014-10-31 10:38:49 +0100 | [diff] [blame] | 870 | The cycle detector is able to detect garbage cycles and can reclaim them. |
| 871 | The :mod:`gc` module exposes a way to run the detector (the |
Serhiy Storchaka | 0b68a2d | 2013-10-09 13:26:17 +0300 | [diff] [blame] | 872 | :func:`~gc.collect` function), as well as configuration |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 873 | interfaces and the ability to disable the detector at runtime. The cycle |
| 874 | detector is considered an optional component; though it is included by default, |
| 875 | it can be disabled at build time using the :option:`--without-cycle-gc` option |
Georg Brandl | f694518 | 2008-02-01 11:56:49 +0000 | [diff] [blame] | 876 | to the :program:`configure` script on Unix platforms (including Mac OS X). If |
| 877 | the cycle detector is disabled in this way, the :mod:`gc` module will not be |
| 878 | available. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 879 | |
| 880 | |
| 881 | .. _refcountsinpython: |
| 882 | |
| 883 | Reference Counting in Python |
| 884 | ---------------------------- |
| 885 | |
| 886 | There are two macros, ``Py_INCREF(x)`` and ``Py_DECREF(x)``, which handle the |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 887 | incrementing and decrementing of the reference count. :c:func:`Py_DECREF` also |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 888 | frees the object when the count reaches zero. For flexibility, it doesn't call |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 889 | :c:func:`free` directly --- rather, it makes a call through a function pointer in |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 890 | the object's :dfn:`type object`. For this purpose (and others), every object |
| 891 | also contains a pointer to its type object. |
| 892 | |
| 893 | The big question now remains: when to use ``Py_INCREF(x)`` and ``Py_DECREF(x)``? |
| 894 | Let's first introduce some terms. Nobody "owns" an object; however, you can |
| 895 | :dfn:`own a reference` to an object. An object's reference count is now defined |
| 896 | as the number of owned references to it. The owner of a reference is |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 897 | responsible for calling :c:func:`Py_DECREF` when the reference is no longer |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 898 | needed. Ownership of a reference can be transferred. There are three ways to |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 899 | dispose of an owned reference: pass it on, store it, or call :c:func:`Py_DECREF`. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 900 | Forgetting to dispose of an owned reference creates a memory leak. |
| 901 | |
| 902 | It is also possible to :dfn:`borrow` [#]_ a reference to an object. The |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 903 | borrower of a reference should not call :c:func:`Py_DECREF`. The borrower must |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 904 | not hold on to the object longer than the owner from which it was borrowed. |
| 905 | Using a borrowed reference after the owner has disposed of it risks using freed |
| 906 | memory and should be avoided completely. [#]_ |
| 907 | |
| 908 | The advantage of borrowing over owning a reference is that you don't need to |
| 909 | take care of disposing of the reference on all possible paths through the code |
| 910 | --- in other words, with a borrowed reference you don't run the risk of leaking |
Benjamin Peterson | 6ebe78f | 2008-12-21 00:06:59 +0000 | [diff] [blame] | 911 | when a premature exit is taken. The disadvantage of borrowing over owning is |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 912 | that there are some subtle situations where in seemingly correct code a borrowed |
| 913 | reference can be used after the owner from which it was borrowed has in fact |
| 914 | disposed of it. |
| 915 | |
| 916 | A borrowed reference can be changed into an owned reference by calling |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 917 | :c:func:`Py_INCREF`. This does not affect the status of the owner from which the |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 918 | reference was borrowed --- it creates a new owned reference, and gives full |
| 919 | owner responsibilities (the new owner must dispose of the reference properly, as |
| 920 | well as the previous owner). |
| 921 | |
| 922 | |
| 923 | .. _ownershiprules: |
| 924 | |
| 925 | Ownership Rules |
| 926 | --------------- |
| 927 | |
| 928 | Whenever an object reference is passed into or out of a function, it is part of |
| 929 | the function's interface specification whether ownership is transferred with the |
| 930 | reference or not. |
| 931 | |
| 932 | Most functions that return a reference to an object pass on ownership with the |
| 933 | reference. In particular, all functions whose function it is to create a new |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 934 | object, such as :c:func:`PyLong_FromLong` and :c:func:`Py_BuildValue`, pass |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 935 | ownership to the receiver. Even if the object is not actually new, you still |
| 936 | receive ownership of a new reference to that object. For instance, |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 937 | :c:func:`PyLong_FromLong` maintains a cache of popular values and can return a |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 938 | reference to a cached item. |
| 939 | |
| 940 | Many functions that extract objects from other objects also transfer ownership |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 941 | with the reference, for instance :c:func:`PyObject_GetAttrString`. The picture |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 942 | is less clear, here, however, since a few common routines are exceptions: |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 943 | :c:func:`PyTuple_GetItem`, :c:func:`PyList_GetItem`, :c:func:`PyDict_GetItem`, and |
| 944 | :c:func:`PyDict_GetItemString` all return references that you borrow from the |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 945 | tuple, list or dictionary. |
| 946 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 947 | The function :c:func:`PyImport_AddModule` also returns a borrowed reference, even |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 948 | though it may actually create the object it returns: this is possible because an |
| 949 | owned reference to the object is stored in ``sys.modules``. |
| 950 | |
| 951 | When you pass an object reference into another function, in general, the |
| 952 | function borrows the reference from you --- if it needs to store it, it will use |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 953 | :c:func:`Py_INCREF` to become an independent owner. There are exactly two |
| 954 | important exceptions to this rule: :c:func:`PyTuple_SetItem` and |
| 955 | :c:func:`PyList_SetItem`. These functions take over ownership of the item passed |
| 956 | to them --- even if they fail! (Note that :c:func:`PyDict_SetItem` and friends |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 957 | don't take over ownership --- they are "normal.") |
| 958 | |
| 959 | When a C function is called from Python, it borrows references to its arguments |
| 960 | from the caller. The caller owns a reference to the object, so the borrowed |
| 961 | reference's lifetime is guaranteed until the function returns. Only when such a |
| 962 | borrowed reference must be stored or passed on, it must be turned into an owned |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 963 | reference by calling :c:func:`Py_INCREF`. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 964 | |
| 965 | The object reference returned from a C function that is called from Python must |
| 966 | be an owned reference --- ownership is transferred from the function to its |
| 967 | caller. |
| 968 | |
| 969 | |
| 970 | .. _thinice: |
| 971 | |
| 972 | Thin Ice |
| 973 | -------- |
| 974 | |
| 975 | There are a few situations where seemingly harmless use of a borrowed reference |
| 976 | can lead to problems. These all have to do with implicit invocations of the |
| 977 | interpreter, which can cause the owner of a reference to dispose of it. |
| 978 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 979 | The first and most important case to know about is using :c:func:`Py_DECREF` on |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 980 | an unrelated object while borrowing a reference to a list item. For instance:: |
| 981 | |
| 982 | void |
| 983 | bug(PyObject *list) |
| 984 | { |
| 985 | PyObject *item = PyList_GetItem(list, 0); |
| 986 | |
Georg Brandl | 9914dd3 | 2007-12-02 23:08:39 +0000 | [diff] [blame] | 987 | PyList_SetItem(list, 1, PyLong_FromLong(0L)); |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 988 | PyObject_Print(item, stdout, 0); /* BUG! */ |
| 989 | } |
| 990 | |
| 991 | This function first borrows a reference to ``list[0]``, then replaces |
| 992 | ``list[1]`` with the value ``0``, and finally prints the borrowed reference. |
| 993 | Looks harmless, right? But it's not! |
| 994 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 995 | Let's follow the control flow into :c:func:`PyList_SetItem`. The list owns |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 996 | references to all its items, so when item 1 is replaced, it has to dispose of |
| 997 | the original item 1. Now let's suppose the original item 1 was an instance of a |
| 998 | user-defined class, and let's further suppose that the class defined a |
| 999 | :meth:`__del__` method. If this class instance has a reference count of 1, |
| 1000 | disposing of it will call its :meth:`__del__` method. |
| 1001 | |
| 1002 | Since it is written in Python, the :meth:`__del__` method can execute arbitrary |
| 1003 | Python code. Could it perhaps do something to invalidate the reference to |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 1004 | ``item`` in :c:func:`bug`? You bet! Assuming that the list passed into |
| 1005 | :c:func:`bug` is accessible to the :meth:`__del__` method, it could execute a |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1006 | statement to the effect of ``del list[0]``, and assuming this was the last |
| 1007 | reference to that object, it would free the memory associated with it, thereby |
| 1008 | invalidating ``item``. |
| 1009 | |
| 1010 | The solution, once you know the source of the problem, is easy: temporarily |
| 1011 | increment the reference count. The correct version of the function reads:: |
| 1012 | |
| 1013 | void |
| 1014 | no_bug(PyObject *list) |
| 1015 | { |
| 1016 | PyObject *item = PyList_GetItem(list, 0); |
| 1017 | |
| 1018 | Py_INCREF(item); |
Georg Brandl | 9914dd3 | 2007-12-02 23:08:39 +0000 | [diff] [blame] | 1019 | PyList_SetItem(list, 1, PyLong_FromLong(0L)); |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1020 | PyObject_Print(item, stdout, 0); |
| 1021 | Py_DECREF(item); |
| 1022 | } |
| 1023 | |
| 1024 | This is a true story. An older version of Python contained variants of this bug |
| 1025 | and someone spent a considerable amount of time in a C debugger to figure out |
| 1026 | why his :meth:`__del__` methods would fail... |
| 1027 | |
| 1028 | The second case of problems with a borrowed reference is a variant involving |
| 1029 | threads. Normally, multiple threads in the Python interpreter can't get in each |
| 1030 | other's way, because there is a global lock protecting Python's entire object |
| 1031 | space. However, it is possible to temporarily release this lock using the macro |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 1032 | :c:macro:`Py_BEGIN_ALLOW_THREADS`, and to re-acquire it using |
| 1033 | :c:macro:`Py_END_ALLOW_THREADS`. This is common around blocking I/O calls, to |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1034 | let other threads use the processor while waiting for the I/O to complete. |
| 1035 | Obviously, the following function has the same problem as the previous one:: |
| 1036 | |
| 1037 | void |
| 1038 | bug(PyObject *list) |
| 1039 | { |
| 1040 | PyObject *item = PyList_GetItem(list, 0); |
| 1041 | Py_BEGIN_ALLOW_THREADS |
| 1042 | ...some blocking I/O call... |
| 1043 | Py_END_ALLOW_THREADS |
| 1044 | PyObject_Print(item, stdout, 0); /* BUG! */ |
| 1045 | } |
| 1046 | |
| 1047 | |
| 1048 | .. _nullpointers: |
| 1049 | |
| 1050 | NULL Pointers |
| 1051 | ------------- |
| 1052 | |
| 1053 | In general, functions that take object references as arguments do not expect you |
| 1054 | to pass them *NULL* pointers, and will dump core (or cause later core dumps) if |
| 1055 | you do so. Functions that return object references generally return *NULL* only |
| 1056 | to indicate that an exception occurred. The reason for not testing for *NULL* |
| 1057 | arguments is that functions often pass the objects they receive on to other |
| 1058 | function --- if each function were to test for *NULL*, there would be a lot of |
| 1059 | redundant tests and the code would run more slowly. |
| 1060 | |
| 1061 | It is better to test for *NULL* only at the "source:" when a pointer that may be |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 1062 | *NULL* is received, for example, from :c:func:`malloc` or from a function that |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1063 | may raise an exception. |
| 1064 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 1065 | The macros :c:func:`Py_INCREF` and :c:func:`Py_DECREF` do not check for *NULL* |
| 1066 | pointers --- however, their variants :c:func:`Py_XINCREF` and :c:func:`Py_XDECREF` |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1067 | do. |
| 1068 | |
| 1069 | The macros for checking for a particular object type (``Pytype_Check()``) don't |
| 1070 | check for *NULL* pointers --- again, there is much code that calls several of |
| 1071 | these in a row to test an object against various different expected types, and |
| 1072 | this would generate redundant tests. There are no variants with *NULL* |
| 1073 | checking. |
| 1074 | |
| 1075 | The C function calling mechanism guarantees that the argument list passed to C |
| 1076 | functions (``args`` in the examples) is never *NULL* --- in fact it guarantees |
| 1077 | that it is always a tuple. [#]_ |
| 1078 | |
| 1079 | It is a severe error to ever let a *NULL* pointer "escape" to the Python user. |
| 1080 | |
Christian Heimes | 5b5e81c | 2007-12-31 16:14:33 +0000 | [diff] [blame] | 1081 | .. Frank Stajano: |
| 1082 | A pedagogically buggy example, along the lines of the previous listing, would |
| 1083 | be helpful here -- showing in more concrete terms what sort of actions could |
| 1084 | cause the problem. I can't very well imagine it from the description. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1085 | |
| 1086 | |
| 1087 | .. _cplusplus: |
| 1088 | |
| 1089 | Writing Extensions in C++ |
| 1090 | ========================= |
| 1091 | |
| 1092 | It is possible to write extension modules in C++. Some restrictions apply. If |
| 1093 | the main program (the Python interpreter) is compiled and linked by the C |
| 1094 | compiler, global or static objects with constructors cannot be used. This is |
| 1095 | not a problem if the main program is linked by the C++ compiler. Functions that |
| 1096 | will be called by the Python interpreter (in particular, module initialization |
| 1097 | functions) have to be declared using ``extern "C"``. It is unnecessary to |
| 1098 | enclose the Python header files in ``extern "C" {...}`` --- they use this form |
| 1099 | already if the symbol ``__cplusplus`` is defined (all recent C++ compilers |
| 1100 | define this symbol). |
| 1101 | |
| 1102 | |
Benjamin Peterson | b173f78 | 2009-05-05 22:31:58 +0000 | [diff] [blame] | 1103 | .. _using-capsules: |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1104 | |
| 1105 | Providing a C API for an Extension Module |
| 1106 | ========================================= |
| 1107 | |
| 1108 | .. sectionauthor:: Konrad Hinsen <hinsen@cnrs-orleans.fr> |
| 1109 | |
| 1110 | |
| 1111 | Many extension modules just provide new functions and types to be used from |
| 1112 | Python, but sometimes the code in an extension module can be useful for other |
| 1113 | extension modules. For example, an extension module could implement a type |
| 1114 | "collection" which works like lists without order. Just like the standard Python |
| 1115 | list type has a C API which permits extension modules to create and manipulate |
| 1116 | lists, this new collection type should have a set of C functions for direct |
| 1117 | manipulation from other extension modules. |
| 1118 | |
| 1119 | At first sight this seems easy: just write the functions (without declaring them |
Christian Heimes | 5b5e81c | 2007-12-31 16:14:33 +0000 | [diff] [blame] | 1120 | ``static``, of course), provide an appropriate header file, and document |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1121 | the C API. And in fact this would work if all extension modules were always |
| 1122 | linked statically with the Python interpreter. When modules are used as shared |
| 1123 | libraries, however, the symbols defined in one module may not be visible to |
| 1124 | another module. The details of visibility depend on the operating system; some |
| 1125 | systems use one global namespace for the Python interpreter and all extension |
| 1126 | modules (Windows, for example), whereas others require an explicit list of |
| 1127 | imported symbols at module link time (AIX is one example), or offer a choice of |
| 1128 | different strategies (most Unices). And even if symbols are globally visible, |
| 1129 | the module whose functions one wishes to call might not have been loaded yet! |
| 1130 | |
| 1131 | Portability therefore requires not to make any assumptions about symbol |
| 1132 | visibility. This means that all symbols in extension modules should be declared |
Christian Heimes | 5b5e81c | 2007-12-31 16:14:33 +0000 | [diff] [blame] | 1133 | ``static``, except for the module's initialization function, in order to |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1134 | avoid name clashes with other extension modules (as discussed in section |
| 1135 | :ref:`methodtable`). And it means that symbols that *should* be accessible from |
| 1136 | other extension modules must be exported in a different way. |
| 1137 | |
| 1138 | Python provides a special mechanism to pass C-level information (pointers) from |
Benjamin Peterson | b173f78 | 2009-05-05 22:31:58 +0000 | [diff] [blame] | 1139 | one extension module to another one: Capsules. A Capsule is a Python data type |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 1140 | which stores a pointer (:c:type:`void \*`). Capsules can only be created and |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1141 | accessed via their C API, but they can be passed around like any other Python |
| 1142 | object. In particular, they can be assigned to a name in an extension module's |
| 1143 | namespace. Other extension modules can then import this module, retrieve the |
Benjamin Peterson | b173f78 | 2009-05-05 22:31:58 +0000 | [diff] [blame] | 1144 | value of this name, and then retrieve the pointer from the Capsule. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1145 | |
Benjamin Peterson | b173f78 | 2009-05-05 22:31:58 +0000 | [diff] [blame] | 1146 | There are many ways in which Capsules can be used to export the C API of an |
| 1147 | extension module. Each function could get its own Capsule, or all C API pointers |
| 1148 | could be stored in an array whose address is published in a Capsule. And the |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1149 | various tasks of storing and retrieving the pointers can be distributed in |
| 1150 | different ways between the module providing the code and the client modules. |
| 1151 | |
Benjamin Peterson | b173f78 | 2009-05-05 22:31:58 +0000 | [diff] [blame] | 1152 | Whichever method you choose, it's important to name your Capsules properly. |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 1153 | The function :c:func:`PyCapsule_New` takes a name parameter |
| 1154 | (:c:type:`const char \*`); you're permitted to pass in a *NULL* name, but |
Benjamin Peterson | b173f78 | 2009-05-05 22:31:58 +0000 | [diff] [blame] | 1155 | we strongly encourage you to specify a name. Properly named Capsules provide |
| 1156 | a degree of runtime type-safety; there is no feasible way to tell one unnamed |
| 1157 | Capsule from another. |
| 1158 | |
| 1159 | In particular, Capsules used to expose C APIs should be given a name following |
| 1160 | this convention:: |
| 1161 | |
| 1162 | modulename.attributename |
| 1163 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 1164 | The convenience function :c:func:`PyCapsule_Import` makes it easy to |
Benjamin Peterson | b173f78 | 2009-05-05 22:31:58 +0000 | [diff] [blame] | 1165 | load a C API provided via a Capsule, but only if the Capsule's name |
| 1166 | matches this convention. This behavior gives C API users a high degree |
| 1167 | of certainty that the Capsule they load contains the correct C API. |
| 1168 | |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1169 | The following example demonstrates an approach that puts most of the burden on |
| 1170 | the writer of the exporting module, which is appropriate for commonly used |
| 1171 | library modules. It stores all C API pointers (just one in the example!) in an |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 1172 | array of :c:type:`void` pointers which becomes the value of a Capsule. The header |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1173 | file corresponding to the module provides a macro that takes care of importing |
| 1174 | the module and retrieving its C API pointers; client modules only have to call |
| 1175 | this macro before accessing the C API. |
| 1176 | |
| 1177 | The exporting module is a modification of the :mod:`spam` module from section |
| 1178 | :ref:`extending-simpleexample`. The function :func:`spam.system` does not call |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 1179 | the C library function :c:func:`system` directly, but a function |
| 1180 | :c:func:`PySpam_System`, which would of course do something more complicated in |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1181 | reality (such as adding "spam" to every command). This function |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 1182 | :c:func:`PySpam_System` is also exported to other extension modules. |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1183 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 1184 | The function :c:func:`PySpam_System` is a plain C function, declared |
Christian Heimes | 5b5e81c | 2007-12-31 16:14:33 +0000 | [diff] [blame] | 1185 | ``static`` like everything else:: |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1186 | |
| 1187 | static int |
| 1188 | PySpam_System(const char *command) |
| 1189 | { |
| 1190 | return system(command); |
| 1191 | } |
| 1192 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 1193 | The function :c:func:`spam_system` is modified in a trivial way:: |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1194 | |
| 1195 | static PyObject * |
| 1196 | spam_system(PyObject *self, PyObject *args) |
| 1197 | { |
| 1198 | const char *command; |
| 1199 | int sts; |
| 1200 | |
| 1201 | if (!PyArg_ParseTuple(args, "s", &command)) |
| 1202 | return NULL; |
| 1203 | sts = PySpam_System(command); |
Georg Brandl | c877a7c | 2010-11-26 11:55:48 +0000 | [diff] [blame] | 1204 | return PyLong_FromLong(sts); |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1205 | } |
| 1206 | |
| 1207 | In the beginning of the module, right after the line :: |
| 1208 | |
| 1209 | #include "Python.h" |
| 1210 | |
| 1211 | two more lines must be added:: |
| 1212 | |
| 1213 | #define SPAM_MODULE |
| 1214 | #include "spammodule.h" |
| 1215 | |
| 1216 | The ``#define`` is used to tell the header file that it is being included in the |
| 1217 | exporting module, not a client module. Finally, the module's initialization |
| 1218 | function must take care of initializing the C API pointer array:: |
| 1219 | |
| 1220 | PyMODINIT_FUNC |
Martin v. Löwis | 1a21451 | 2008-06-11 05:26:20 +0000 | [diff] [blame] | 1221 | PyInit_spam(void) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1222 | { |
| 1223 | PyObject *m; |
| 1224 | static void *PySpam_API[PySpam_API_pointers]; |
| 1225 | PyObject *c_api_object; |
| 1226 | |
Martin v. Löwis | 1a21451 | 2008-06-11 05:26:20 +0000 | [diff] [blame] | 1227 | m = PyModule_Create(&spammodule); |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1228 | if (m == NULL) |
Martin v. Löwis | 1a21451 | 2008-06-11 05:26:20 +0000 | [diff] [blame] | 1229 | return NULL; |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1230 | |
| 1231 | /* Initialize the C API pointer array */ |
| 1232 | PySpam_API[PySpam_System_NUM] = (void *)PySpam_System; |
| 1233 | |
Benjamin Peterson | b173f78 | 2009-05-05 22:31:58 +0000 | [diff] [blame] | 1234 | /* Create a Capsule containing the API pointer array's address */ |
| 1235 | c_api_object = PyCapsule_New((void *)PySpam_API, "spam._C_API", NULL); |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1236 | |
| 1237 | if (c_api_object != NULL) |
| 1238 | PyModule_AddObject(m, "_C_API", c_api_object); |
Martin v. Löwis | 1a21451 | 2008-06-11 05:26:20 +0000 | [diff] [blame] | 1239 | return m; |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1240 | } |
| 1241 | |
Christian Heimes | 5b5e81c | 2007-12-31 16:14:33 +0000 | [diff] [blame] | 1242 | Note that ``PySpam_API`` is declared ``static``; otherwise the pointer |
Martin v. Löwis | 1a21451 | 2008-06-11 05:26:20 +0000 | [diff] [blame] | 1243 | array would disappear when :func:`PyInit_spam` terminates! |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1244 | |
| 1245 | The bulk of the work is in the header file :file:`spammodule.h`, which looks |
| 1246 | like this:: |
| 1247 | |
| 1248 | #ifndef Py_SPAMMODULE_H |
| 1249 | #define Py_SPAMMODULE_H |
| 1250 | #ifdef __cplusplus |
| 1251 | extern "C" { |
| 1252 | #endif |
| 1253 | |
| 1254 | /* Header file for spammodule */ |
| 1255 | |
| 1256 | /* C API functions */ |
| 1257 | #define PySpam_System_NUM 0 |
| 1258 | #define PySpam_System_RETURN int |
| 1259 | #define PySpam_System_PROTO (const char *command) |
| 1260 | |
| 1261 | /* Total number of C API pointers */ |
| 1262 | #define PySpam_API_pointers 1 |
| 1263 | |
| 1264 | |
| 1265 | #ifdef SPAM_MODULE |
| 1266 | /* This section is used when compiling spammodule.c */ |
| 1267 | |
| 1268 | static PySpam_System_RETURN PySpam_System PySpam_System_PROTO; |
| 1269 | |
| 1270 | #else |
| 1271 | /* This section is used in modules that use spammodule's API */ |
| 1272 | |
| 1273 | static void **PySpam_API; |
| 1274 | |
| 1275 | #define PySpam_System \ |
| 1276 | (*(PySpam_System_RETURN (*)PySpam_System_PROTO) PySpam_API[PySpam_System_NUM]) |
| 1277 | |
Benjamin Peterson | b173f78 | 2009-05-05 22:31:58 +0000 | [diff] [blame] | 1278 | /* Return -1 on error, 0 on success. |
| 1279 | * PyCapsule_Import will set an exception if there's an error. |
| 1280 | */ |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1281 | static int |
| 1282 | import_spam(void) |
| 1283 | { |
Benjamin Peterson | b173f78 | 2009-05-05 22:31:58 +0000 | [diff] [blame] | 1284 | PySpam_API = (void **)PyCapsule_Import("spam._C_API", 0); |
| 1285 | return (PySpam_API != NULL) ? 0 : -1; |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1286 | } |
| 1287 | |
| 1288 | #endif |
| 1289 | |
| 1290 | #ifdef __cplusplus |
| 1291 | } |
| 1292 | #endif |
| 1293 | |
| 1294 | #endif /* !defined(Py_SPAMMODULE_H) */ |
| 1295 | |
| 1296 | All that a client module must do in order to have access to the function |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 1297 | :c:func:`PySpam_System` is to call the function (or rather macro) |
| 1298 | :c:func:`import_spam` in its initialization function:: |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1299 | |
| 1300 | PyMODINIT_FUNC |
Benjamin Peterson | 7c43524 | 2009-03-24 01:40:39 +0000 | [diff] [blame] | 1301 | PyInit_client(void) |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1302 | { |
| 1303 | PyObject *m; |
| 1304 | |
Georg Brandl | 2115176 | 2009-03-31 15:52:41 +0000 | [diff] [blame] | 1305 | m = PyModule_Create(&clientmodule); |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1306 | if (m == NULL) |
Georg Brandl | 2115176 | 2009-03-31 15:52:41 +0000 | [diff] [blame] | 1307 | return NULL; |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1308 | if (import_spam() < 0) |
Georg Brandl | 2115176 | 2009-03-31 15:52:41 +0000 | [diff] [blame] | 1309 | return NULL; |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1310 | /* additional initialization can happen here */ |
Georg Brandl | 2115176 | 2009-03-31 15:52:41 +0000 | [diff] [blame] | 1311 | return m; |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1312 | } |
| 1313 | |
| 1314 | The main disadvantage of this approach is that the file :file:`spammodule.h` is |
| 1315 | rather complicated. However, the basic structure is the same for each function |
| 1316 | that is exported, so it has to be learned only once. |
| 1317 | |
Benjamin Peterson | b173f78 | 2009-05-05 22:31:58 +0000 | [diff] [blame] | 1318 | Finally it should be mentioned that Capsules offer additional functionality, |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1319 | which is especially useful for memory allocation and deallocation of the pointer |
Benjamin Peterson | b173f78 | 2009-05-05 22:31:58 +0000 | [diff] [blame] | 1320 | stored in a Capsule. The details are described in the Python/C API Reference |
| 1321 | Manual in the section :ref:`capsules` and in the implementation of Capsules (files |
| 1322 | :file:`Include/pycapsule.h` and :file:`Objects/pycapsule.c` in the Python source |
Georg Brandl | 116aa62 | 2007-08-15 14:28:22 +0000 | [diff] [blame] | 1323 | code distribution). |
| 1324 | |
| 1325 | .. rubric:: Footnotes |
| 1326 | |
| 1327 | .. [#] An interface for this function already exists in the standard module :mod:`os` |
| 1328 | --- it was chosen as a simple and straightforward example. |
| 1329 | |
| 1330 | .. [#] The metaphor of "borrowing" a reference is not completely correct: the owner |
| 1331 | still has a copy of the reference. |
| 1332 | |
| 1333 | .. [#] Checking that the reference count is at least 1 **does not work** --- the |
| 1334 | reference count itself could be in freed memory and may thus be reused for |
| 1335 | another object! |
| 1336 | |
| 1337 | .. [#] These guarantees don't hold when you use the "old" style calling convention --- |
| 1338 | this is still found in much existing code. |
| 1339 | |