blob: 83d1078de8cc31da24143ba0018399f48ad18799 [file] [log] [blame]
Guido van Rossum6938f061994-08-01 12:22:53 +00001\documentstyle[twoside,11pt,myformat]{report}
Guido van Rossum7a2dba21993-11-05 14:45:11 +00002
Guido van Rossum5049bcb1995-03-13 16:55:23 +00003% XXX PM Modulator
4
Guido van Rossum6938f061994-08-01 12:22:53 +00005\title{Extending and Embedding the Python Interpreter}
Guido van Rossum7a2dba21993-11-05 14:45:11 +00006
Guido van Rossum16cd7f91994-10-06 10:29:26 +00007\input{boilerplate}
Guido van Rossum83eb9621993-11-23 16:28:45 +00008
Guido van Rossum7a2dba21993-11-05 14:45:11 +00009% Tell \index to actually write the .idx file
10\makeindex
11
12\begin{document}
13
14\pagenumbering{roman}
15
16\maketitle
17
Guido van Rossum16cd7f91994-10-06 10:29:26 +000018\input{copyright}
19
Guido van Rossum7a2dba21993-11-05 14:45:11 +000020\begin{abstract}
21
22\noindent
Guido van Rossumb92112d1995-03-20 14:24:09 +000023Python is an interpreted, object-oriented programming language. This
24document describes how to write modules in C or \Cpp{} to extend the
25Python interpreter with new modules. Those modules can define new
26functions but also new object types and their methods. The document
27also describes how to embed the Python interpreter in another
28application, for use as an extension language. Finally, it shows how
29to compile and link extension modules so that they can be loaded
30dynamically (at run time) into the interpreter, if the underlying
31operating system supports this feature.
32
33This document assumes basic knowledge about Python. For an informal
34introduction to the language, see the Python Tutorial. The Python
35Reference Manual gives a more formal definition of the language. The
36Python Library Reference documents the existing object types,
37functions and modules (both built-in and written in Python) that give
38the language its wide application range.
Guido van Rossum7a2dba21993-11-05 14:45:11 +000039
40\end{abstract}
41
42\pagebreak
43
44{
45\parskip = 0mm
46\tableofcontents
47}
48
49\pagebreak
50
51\pagenumbering{arabic}
52
Guido van Rossumdb65a6c1993-11-05 17:11:16 +000053
Guido van Rossum16d6e711994-08-08 12:30:22 +000054\chapter{Extending Python with C or \Cpp{} code}
Guido van Rossum7a2dba21993-11-05 14:45:11 +000055
Guido van Rossum6f0132f1993-11-19 13:13:22 +000056
57\section{Introduction}
58
Guido van Rossumb92112d1995-03-20 14:24:09 +000059It is quite easy to add new built-in modules to Python, if you know
60how to program in C. Such \dfn{extension modules} can do two things
61that can't be done directly in Python: they can implement new built-in
62object types, and they can call C library functions and system calls.
Guido van Rossum6938f061994-08-01 12:22:53 +000063
Guido van Rossum5049bcb1995-03-13 16:55:23 +000064To support extensions, the Python API (Application Programmers
Guido van Rossumb92112d1995-03-20 14:24:09 +000065Interface) defines a set of functions, macros and variables that
66provide access to most aspects of the Python run-time system. The
67Python API is incorporated in a C source file by including the header
68\code{"Python.h"}.
Guido van Rossum6938f061994-08-01 12:22:53 +000069
Guido van Rossumb92112d1995-03-20 14:24:09 +000070The compilation of an extension module depends on its intended use as
71well as on your system setup; details are given in a later section.
Guido van Rossum6938f061994-08-01 12:22:53 +000072
Guido van Rossum7a2dba21993-11-05 14:45:11 +000073
Guido van Rossum5049bcb1995-03-13 16:55:23 +000074\section{A Simple Example}
Guido van Rossum7a2dba21993-11-05 14:45:11 +000075
Guido van Rossumb92112d1995-03-20 14:24:09 +000076Let's create an extension module called \samp{spam} (the favorite food
77of Monty Python fans...) and let's say we want to create a Python
78interface to the C library function \code{system()}.\footnote{An
79interface for this function already exists in the standard module
80\code{os} --- it was chosen as a simple and straightfoward example.}
81This function takes a null-terminated character string as argument and
82returns an integer. We want this function to be callable from Python
83as follows:
84
85\begin{verbatim}
86 >>> import spam
87 >>> status = spam.system("ls -l")
88\end{verbatim}
89
90Begin by creating a file \samp{spammodule.c}. (In general, if a
91module is called \samp{spam}, the C file containing its implementation
92is called \file{spammodule.c}; if the module name is very long, like
93\samp{spammify}, the module name can be just \file{spammify.c}.)
94
95The first line of our file can be:
Guido van Rossum7a2dba21993-11-05 14:45:11 +000096
97\begin{verbatim}
Guido van Rossum5049bcb1995-03-13 16:55:23 +000098 #include "Python.h"
99\end{verbatim}
100
101which pulls in the Python API (you can add a comment describing the
102purpose of the module and a copyright notice if you like).
103
Guido van Rossumb92112d1995-03-20 14:24:09 +0000104All user-visible symbols defined by \code{"Python.h"} have a prefix of
105\samp{Py} or \samp{PY}, except those defined in standard header files.
106For convenience, and since they are used extensively by the Python
107interpreter, \code{"Python.h"} includes a few standard header files:
108\code{<stdio.h>}, \code{<string.h>}, \code{<errno.h>}, and
109\code{<stdlib.h>}. If the latter header file does not exist on your
110system, it declares the functions \code{malloc()}, \code{free()} and
111\code{realloc()} directly.
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000112
113The next thing we add to our module file is the C function that will
114be called when the Python expression \samp{spam.system(\var{string})}
Guido van Rossumb92112d1995-03-20 14:24:09 +0000115is evaluated (we'll see shortly how it ends up being called):
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000116
117\begin{verbatim}
118 static PyObject *
119 spam_system(self, args)
120 PyObject *self;
121 PyObject *args;
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000122 {
123 char *command;
124 int sts;
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000125 if (!PyArg_ParseTuple(args, "s", &command))
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000126 return NULL;
127 sts = system(command);
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000128 return Py_BuildValue("i", sts);
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000129 }
130\end{verbatim}
131
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000132There is a straightforward translation from the argument list in
Guido van Rossumb92112d1995-03-20 14:24:09 +0000133Python (e.g.\ the single expression \code{"ls -l"}) to the arguments
134passed to the C function. The C function always has two arguments,
135conventionally named \var{self} and \var{args}.
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000136
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000137The \var{self} argument is only used when the C function implements a
Guido van Rossumb92112d1995-03-20 14:24:09 +0000138builtin method. This will be discussed later. In the example,
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000139\var{self} will always be a \code{NULL} pointer, since we are defining
140a function, not a method. (This is done so that the interpreter
141doesn't have to understand two different types of C functions.)
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000142
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000143The \var{args} argument will be a pointer to a Python tuple object
Guido van Rossumb92112d1995-03-20 14:24:09 +0000144containing the arguments. Each item of the tuple corresponds to an
145argument in the call's argument list. The arguments are Python
146objects -- in order to do anything with them in our C function we have
147to convert them to C values. The function \code{PyArg_ParseTuple()}
148in the Python API checks the argument types and converts them to C
149values. It uses a template string to determine the required types of
150the arguments as well as the types of the C variables into which to
151store the converted values. More about this later.
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000152
Guido van Rossumb92112d1995-03-20 14:24:09 +0000153\code{PyArg_ParseTuple()} returns true (nonzero) if all arguments have
154the right type and its components have been stored in the variables
155whose addresses are passed. It returns false (zero) if an invalid
156argument list was passed. In the latter case it also raises an
157appropriate exception by so the calling function can return
158\code{NULL} immediately (as we saw in the example).
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000159
160
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000161\section{Intermezzo: Errors and Exceptions}
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000162
163An important convention throughout the Python interpreter is the
164following: when a function fails, it should set an exception condition
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000165and return an error value (usually a \code{NULL} pointer). Exceptions
Guido van Rossumb92112d1995-03-20 14:24:09 +0000166are stored in a static global variable inside the interpreter; if this
167variable is \code{NULL} no exception has occurred. A second global
168variable stores the ``associated value'' of the exception (the second
169argument to \code{raise}). A third variable contains the stack
170traceback in case the error originated in Python code. These three
171variables are the C equivalents of the Python variables
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000172\code{sys.exc_type}, \code{sys.exc_value} and \code{sys.exc_traceback}
Guido van Rossumb92112d1995-03-20 14:24:09 +0000173(see the section on module \code{sys} in the Library Reference
174Manual). It is important to know about them to understand how errors
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000175are passed around.
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000176
Guido van Rossumb92112d1995-03-20 14:24:09 +0000177The Python API defines a number of functions to set various types of
178exceptions.
179
180The most common one is \code{PyErr_SetString()}. Its arguments are an
181exception object and a C string. The exception object is usually a
182predefined object like \code{PyExc_ZeroDivisionError}. The C string
183indicates the cause of the error and is converted to a Python string
184object and stored as the ``associated value'' of the exception.
185
186Another useful function is \code{PyErr_SetFromErrno()}, which only
187takes an exception argument and constructs the associated value by
188inspection of the (\UNIX{}) global variable \code{errno}. The most
189general function is \code{PyErr_SetObject()}, which takes two object
190arguments, the exception and its associated value. You don't need to
191\code{Py_INCREF()} the objects passed to any of these functions.
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000192
193You can test non-destructively whether an exception has been set with
Guido van Rossumb92112d1995-03-20 14:24:09 +0000194\code{PyErr_Occurred()}. This returns the current exception object,
195or \code{NULL} if no exception has occurred. You normally don't need
196to call \code{PyErr_Occurred()} to see whether an error occurred in a
197function call, since you should be able to tell from the return value.
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000198
Guido van Rossumd16ddb61996-12-13 02:38:17 +0000199When a function \var{f} that calls another function \var{g} detects
Guido van Rossumb92112d1995-03-20 14:24:09 +0000200that the latter fails, \var{f} should itself return an error value
201(e.g. \code{NULL} or \code{-1}). It should \emph{not} call one of the
202\code{PyErr_*()} functions --- one has already been called by \var{g}.
203\var{f}'s caller is then supposed to also return an error indication
204to \emph{its} caller, again \emph{without} calling \code{PyErr_*()},
205and so on --- the most detailed cause of the error was already
206reported by the function that first detected it. Once the error
207reaches the Python interpreter's main loop, this aborts the currently
208executing Python code and tries to find an exception handler specified
209by the Python programmer.
Guido van Rossum6938f061994-08-01 12:22:53 +0000210
211(There are situations where a module can actually give a more detailed
Guido van Rossumb92112d1995-03-20 14:24:09 +0000212error message by calling another \code{PyErr_*()} function, and in
213such cases it is fine to do so. As a general rule, however, this is
214not necessary, and can cause information about the cause of the error
215to be lost: most operations can fail for a variety of reasons.)
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000216
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000217To ignore an exception set by a function call that failed, the exception
218condition must be cleared explicitly by calling \code{PyErr_Clear()}.
219The only time C code should call \code{PyErr_Clear()} is if it doesn't
220want to pass the error on to the interpreter but wants to handle it
221completely by itself (e.g. by trying something else or pretending
222nothing happened).
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000223
Guido van Rossumb92112d1995-03-20 14:24:09 +0000224Note that a failing \code{malloc()} call must be turned into an
Guido van Rossumdb65a6c1993-11-05 17:11:16 +0000225exception --- the direct caller of \code{malloc()} (or
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000226\code{realloc()}) must call \code{PyErr_NoMemory()} and return a
227failure indicator itself. All the object-creating functions
228(\code{PyInt_FromLong()} etc.) already do this, so only if you call
Guido van Rossumdb65a6c1993-11-05 17:11:16 +0000229\code{malloc()} directly this note is of importance.
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000230
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000231Also note that, with the important exception of
Guido van Rossumb92112d1995-03-20 14:24:09 +0000232\code{PyArg_ParseTuple()} and friends, functions that return an
233integer status usually return a positive value or zero for success and
234\code{-1} for failure, like \UNIX{} system calls.
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000235
Guido van Rossumb92112d1995-03-20 14:24:09 +0000236Finally, be careful to clean up garbage (by making \code{Py_XDECREF()}
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000237or \code{Py_DECREF()} calls for objects you have already created) when
Guido van Rossumb92112d1995-03-20 14:24:09 +0000238you return an error indicator!
Guido van Rossum6938f061994-08-01 12:22:53 +0000239
240The choice of which exception to raise is entirely yours. There are
241predeclared C objects corresponding to all built-in Python exceptions,
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000242e.g. \code{PyExc_ZeroDevisionError} which you can use directly. Of
Guido van Rossumb92112d1995-03-20 14:24:09 +0000243course, you should choose exceptions wisely --- don't use
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000244\code{PyExc_TypeError} to mean that a file couldn't be opened (that
245should probably be \code{PyExc_IOError}). If something's wrong with
246the argument list, the \code{PyArg_ParseTuple()} function usually
247raises \code{PyExc_TypeError}. If you have an argument whose value
248which must be in a particular range or must satisfy other conditions,
249\code{PyExc_ValueError} is appropriate.
Guido van Rossum6938f061994-08-01 12:22:53 +0000250
251You can also define a new exception that is unique to your module.
252For this, you usually declare a static object variable at the
253beginning of your file, e.g.
254
255\begin{verbatim}
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000256 static PyObject *SpamError;
Guido van Rossum6938f061994-08-01 12:22:53 +0000257\end{verbatim}
258
259and initialize it in your module's initialization function
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000260(\code{initspam()}) with a string object, e.g. (leaving out the error
Guido van Rossumb92112d1995-03-20 14:24:09 +0000261checking for now):
Guido van Rossum6938f061994-08-01 12:22:53 +0000262
263\begin{verbatim}
264 void
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000265 initspam()
Guido van Rossum6938f061994-08-01 12:22:53 +0000266 {
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000267 PyObject *m, *d;
Guido van Rossumb92112d1995-03-20 14:24:09 +0000268 m = Py_InitModule("spam", SpamMethods);
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000269 d = PyModule_GetDict(m);
270 SpamError = PyString_FromString("spam.error");
271 PyDict_SetItemString(d, "error", SpamError);
Guido van Rossum6938f061994-08-01 12:22:53 +0000272 }
273\end{verbatim}
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000274
Guido van Rossumb92112d1995-03-20 14:24:09 +0000275Note that the Python name for the exception object is
276\code{spam.error}. It is conventional for module and exception names
277to be spelled in lower case. It is also conventional that the
278\emph{value} of the exception object is the same as its name, e.g.\
279the string \code{"spam.error"}.
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000280
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000281
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000282\section{Back to the Example}
283
284Going back to our example function, you should now be able to
285understand this statement:
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000286
287\begin{verbatim}
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000288 if (!PyArg_ParseTuple(args, "s", &command))
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000289 return NULL;
290\end{verbatim}
291
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000292It returns \code{NULL} (the error indicator for functions returning
293object pointers) if an error is detected in the argument list, relying
294on the exception set by \code{PyArg_ParseTuple()}. Otherwise the
295string value of the argument has been copied to the local variable
296\code{command}. This is a pointer assignment and you are not supposed
Guido van Rossumb92112d1995-03-20 14:24:09 +0000297to modify the string to which it points (so in Standard C, the variable
298\code{command} should properly be declared as \samp{const char
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000299*command}).
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000300
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000301The next statement is a call to the \UNIX{} function \code{system()},
302passing it the string we just got from \code{PyArg_ParseTuple()}:
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000303
304\begin{verbatim}
305 sts = system(command);
306\end{verbatim}
307
Guido van Rossumd16ddb61996-12-13 02:38:17 +0000308Our \code{spam.system()} function must return the value of \code{sts}
Guido van Rossumb92112d1995-03-20 14:24:09 +0000309as a Python object. This is done using the function
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000310\code{Py_BuildValue()}, which is something like the inverse of
311\code{PyArg_ParseTuple()}: it takes a format string and an arbitrary
312number of C values, and returns a new Python object. More info on
313\code{Py_BuildValue()} is given later.
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000314
315\begin{verbatim}
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000316 return Py_BuildValue("i", sts);
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000317\end{verbatim}
318
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000319In this case, it will return an integer object. (Yes, even integers
320are objects on the heap in Python!)
Guido van Rossum6938f061994-08-01 12:22:53 +0000321
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000322If you have a C function that returns no useful argument (a function
323returning \code{void}), the corresponding Python function must return
324\code{None}. You need this idiom to do so:
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000325
326\begin{verbatim}
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000327 Py_INCREF(Py_None);
328 return Py_None;
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000329\end{verbatim}
330
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000331\code{Py_None} is the C name for the special Python object
332\code{None}. It is a genuine Python object (not a \code{NULL}
Guido van Rossumb92112d1995-03-20 14:24:09 +0000333pointer, which means ``error'' in most contexts, as we have seen).
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000334
335
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000336\section{The Module's Method Table and Initialization Function}
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000337
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000338I promised to show how \code{spam_system()} is called from Python
339programs. First, we need to list its name and address in a ``method
340table'':
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000341
342\begin{verbatim}
Guido van Rossumb92112d1995-03-20 14:24:09 +0000343 static PyMethodDef SpamMethods[] = {
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000344 ...
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000345 {"system", spam_system, 1},
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000346 ...
347 {NULL, NULL} /* Sentinel */
348 };
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000349\end{verbatim}
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000350
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000351Note the third entry (\samp{1}). This is a flag telling the
352interpreter the calling convention to be used for the C function. It
353should normally always be \samp{1}; a value of \samp{0} means that an
354obsolete variant of \code{PyArg_ParseTuple()} is used.
355
356The method table must be passed to the interpreter in the module's
357initialization function (which should be the only non-\code{static}
358item defined in the module file):
359
360\begin{verbatim}
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000361 void
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000362 initspam()
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000363 {
Guido van Rossumb92112d1995-03-20 14:24:09 +0000364 (void) Py_InitModule("spam", SpamMethods);
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000365 }
366\end{verbatim}
367
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000368When the Python program imports module \code{spam} for the first time,
369\code{initspam()} is called. It calls \code{Py_InitModule()}, which
370creates a ``module object'' (which is inserted in the dictionary
371\code{sys.modules} under the key \code{"spam"}), and inserts built-in
372function objects into the newly created module based upon the table
373(an array of \code{PyMethodDef} structures) that was passed as its
374second argument. \code{Py_InitModule()} returns a pointer to the
Guido van Rossum6938f061994-08-01 12:22:53 +0000375module object that it creates (which is unused here). It aborts with
376a fatal error if the module could not be initialized satisfactorily,
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000377so the caller doesn't need to check for errors.
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000378
379
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000380\section{Compilation and Linkage}
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000381
Guido van Rossumb92112d1995-03-20 14:24:09 +0000382There are two more things to do before you can use your new extension:
383compiling and linking it with the Python system. If you use dynamic
384loading, the details depend on the style of dynamic loading your
385system uses; see the chapter on Dynamic Loading for more info about
386this.
Guido van Rossum6938f061994-08-01 12:22:53 +0000387
388If you can't use dynamic loading, or if you want to make your module a
389permanent part of the Python interpreter, you will have to change the
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000390configuration setup and rebuild the interpreter. Luckily, this is
391very simple: just place your file (\file{spammodule.c} for example) in
392the \file{Modules} directory, add a line to the file
393\file{Modules/Setup} describing your file:
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000394
395\begin{verbatim}
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000396 spam spammodule.o
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000397\end{verbatim}
398
Guido van Rossum6938f061994-08-01 12:22:53 +0000399and rebuild the interpreter by running \code{make} in the toplevel
400directory. You can also run \code{make} in the \file{Modules}
401subdirectory, but then you must first rebuilt the \file{Makefile}
402there by running \code{make Makefile}. (This is necessary each time
403you change the \file{Setup} file.)
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000404
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000405If your module requires additional libraries to link with, these can
406be listed on the line in the \file{Setup} file as well, for instance:
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000407
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000408\begin{verbatim}
409 spam spammodule.o -lX11
410\end{verbatim}
411
412
413\section{Calling Python Functions From C}
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000414
Guido van Rossum6938f061994-08-01 12:22:53 +0000415So far we have concentrated on making C functions callable from
416Python. The reverse is also useful: calling Python functions from C.
417This is especially the case for libraries that support so-called
Guido van Rossumb92112d1995-03-20 14:24:09 +0000418``callback'' functions. If a C interface makes use of callbacks, the
Guido van Rossum6938f061994-08-01 12:22:53 +0000419equivalent Python often needs to provide a callback mechanism to the
420Python programmer; the implementation will require calling the Python
421callback functions from a C callback. Other uses are also imaginable.
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000422
423Fortunately, the Python interpreter is easily called recursively, and
Guido van Rossum6938f061994-08-01 12:22:53 +0000424there is a standard interface to call a Python function. (I won't
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000425dwell on how to call the Python parser with a particular string as
Guido van Rossumdb65a6c1993-11-05 17:11:16 +0000426input --- if you're interested, have a look at the implementation of
Guido van Rossum6938f061994-08-01 12:22:53 +0000427the \samp{-c} command line option in \file{Python/pythonmain.c}.)
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000428
429Calling a Python function is easy. First, the Python program must
430somehow pass you the Python function object. You should provide a
431function (or some other interface) to do this. When this function is
432called, save a pointer to the Python function object (be careful to
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000433\code{Py_INCREF()} it!) in a global variable --- or whereever you see fit.
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000434For example, the following function might be part of a module
435definition:
436
437\begin{verbatim}
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000438 static PyObject *my_callback = NULL;
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000439
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000440 static PyObject *
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000441 my_set_callback(dummy, arg)
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000442 PyObject *dummy, *arg;
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000443 {
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000444 Py_XDECREF(my_callback); /* Dispose of previous callback */
445 Py_XINCREF(arg); /* Add a reference to new callback */
446 my_callback = arg; /* Remember new callback */
447 /* Boilerplate to return "None" */
448 Py_INCREF(Py_None);
449 return Py_None;
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000450 }
451\end{verbatim}
452
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000453The macros \code{Py_XINCREF()} and \code{Py_XDECREF()} increment/decrement
Guido van Rossum6938f061994-08-01 12:22:53 +0000454the reference count of an object and are safe in the presence of
455\code{NULL} pointers. More info on them in the section on Reference
456Counts below.
457
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000458Later, when it is time to call the function, you call the C function
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000459\code{PyEval_CallObject()}. This function has two arguments, both
460pointers to arbitrary Python objects: the Python function, and the
461argument list. The argument list must always be a tuple object, whose
462length is the number of arguments. To call the Python function with
463no arguments, pass an empty tuple; to call it with one argument, pass
464a singleton tuple. \code{Py_BuildValue()} returns a tuple when its
465format string consists of zero or more format codes between
466parentheses. For example:
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000467
468\begin{verbatim}
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000469 int arg;
470 PyObject *arglist;
471 PyObject *result;
472 ...
473 arg = 123;
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000474 ...
475 /* Time to call the callback */
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000476 arglist = Py_BuildValue("(i)", arg);
477 result = PyEval_CallObject(my_callback, arglist);
478 Py_DECREF(arglist);
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000479\end{verbatim}
480
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000481\code{PyEval_CallObject()} returns a Python object pointer: this is
482the return value of the Python function. \code{PyEval_CallObject()} is
Guido van Rossumb92112d1995-03-20 14:24:09 +0000483``reference-count-neutral'' with respect to its arguments. In the
Guido van Rossum6938f061994-08-01 12:22:53 +0000484example a new tuple was created to serve as the argument list, which
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000485is \code{Py_DECREF()}-ed immediately after the call.
Guido van Rossum6938f061994-08-01 12:22:53 +0000486
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000487The return value of \code{PyEval_CallObject()} is ``new'': either it
488is a brand new object, or it is an existing object whose reference
489count has been incremented. So, unless you want to save it in a
490global variable, you should somehow \code{Py_DECREF()} the result,
491even (especially!) if you are not interested in its value.
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000492
493Before you do this, however, it is important to check that the return
Guido van Rossum6938f061994-08-01 12:22:53 +0000494value isn't \code{NULL}. If it is, the Python function terminated by raising
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000495an exception. If the C code that called \code{PyEval_CallObject()} is
Guido van Rossumdb65a6c1993-11-05 17:11:16 +0000496called from Python, it should now return an error indication to its
497Python caller, so the interpreter can print a stack trace, or the
498calling Python code can handle the exception. If this is not possible
499or desirable, the exception should be cleared by calling
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000500\code{PyErr_Clear()}. For example:
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000501
502\begin{verbatim}
503 if (result == NULL)
504 return NULL; /* Pass error back */
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000505 ...use result...
506 Py_DECREF(result);
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000507\end{verbatim}
508
509Depending on the desired interface to the Python callback function,
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000510you may also have to provide an argument list to \code{PyEval_CallObject()}.
Guido van Rossum6938f061994-08-01 12:22:53 +0000511In some cases the argument list is also provided by the Python
512program, through the same interface that specified the callback
513function. It can then be saved and used in the same manner as the
514function object. In other cases, you may have to construct a new
515tuple to pass as the argument list. The simplest way to do this is to
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000516call \code{Py_BuildValue()}. For example, if you want to pass an integral
Guido van Rossum6938f061994-08-01 12:22:53 +0000517event code, you might use the following code:
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000518
519\begin{verbatim}
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000520 PyObject *arglist;
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000521 ...
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000522 arglist = Py_BuildValue("(l)", eventcode);
523 result = PyEval_CallObject(my_callback, arglist);
524 Py_DECREF(arglist);
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000525 if (result == NULL)
526 return NULL; /* Pass error back */
527 /* Here maybe use the result */
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000528 Py_DECREF(result);
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000529\end{verbatim}
530
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000531Note the placement of \code{Py_DECREF(argument)} immediately after the call,
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000532before the error check! Also note that strictly spoken this code is
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000533not complete: \code{Py_BuildValue()} may run out of memory, and this should
Guido van Rossum6938f061994-08-01 12:22:53 +0000534be checked.
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000535
536
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000537\section{Format Strings for {\tt PyArg_ParseTuple()}}
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000538
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000539The \code{PyArg_ParseTuple()} function is declared as follows:
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000540
541\begin{verbatim}
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000542 int PyArg_ParseTuple(PyObject *arg, char *format, ...);
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000543\end{verbatim}
544
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000545The \var{arg} argument must be a tuple object containing an argument
546list passed from Python to a C function. The \var{format} argument
547must be a format string, whose syntax is explained below. The
548remaining arguments must be addresses of variables whose type is
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000549determined by the format string. For the conversion to succeed, the
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000550\var{arg} object must match the format and the format must be
551exhausted.
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000552
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000553Note that while \code{PyArg_ParseTuple()} checks that the Python
554arguments have the required types, it cannot check the validity of the
555addresses of C variables passed to the call: if you make mistakes
556there, your code will probably crash or at least overwrite random bits
557in memory. So be careful!
558
559A format string consists of zero or more ``format units''. A format
560unit describes one Python object; it is usually a single character or
561a parenthesized sequence of format units. With a few exceptions, a
562format unit that is not a parenthesized sequence normally corresponds
563to a single address argument to \code{PyArg_ParseTuple()}. In the
564following description, the quoted form is the format unit; the entry
565in (round) parentheses is the Python object type that matches the
566format unit; and the entry in [square] brackets is the type of the C
567variable(s) whose address should be passed. (Use the \samp{\&}
568operator to pass a variable's address.)
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000569
Guido van Rossumdb65a6c1993-11-05 17:11:16 +0000570\begin{description}
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000571
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000572\item[\samp{s} (string) [char *]]
573Convert a Python string to a C pointer to a character string. You
574must not provide storage for the string itself; a pointer to an
575existing string is stored into the character pointer variable whose
576address you pass. The C string is null-terminated. The Python string
577must not contain embedded null bytes; if it does, a \code{TypeError}
578exception is raised.
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000579
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000580\item[\samp{s\#} (string) {[char *, int]}]
581This variant on \code{'s'} stores into two C variables, the first one
582a pointer to a character string, the second one its length. In this
583case the Python string may contain embedded null bytes.
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000584
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000585\item[\samp{z} (string or \code{None}) {[char *]}]
586Like \samp{s}, but the Python object may also be \code{None}, in which
587case the C pointer is set to \code{NULL}.
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000588
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000589\item[\samp{z\#} (string or \code{None}) {[char *, int]}]
590This is to \code{'s\#'} as \code{'z'} is to \code{'s'}.
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000591
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000592\item[\samp{b} (integer) {[char]}]
593Convert a Python integer to a tiny int, stored in a C \code{char}.
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000594
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000595\item[\samp{h} (integer) {[short int]}]
596Convert a Python integer to a C \code{short int}.
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000597
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000598\item[\samp{i} (integer) {[int]}]
599Convert a Python integer to a plain C \code{int}.
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000600
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000601\item[\samp{l} (integer) {[long int]}]
602Convert a Python integer to a C \code{long int}.
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000603
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000604\item[\samp{c} (string of length 1) {[char]}]
605Convert a Python character, represented as a string of length 1, to a
606C \code{char}.
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000607
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000608\item[\samp{f} (float) {[float]}]
609Convert a Python floating point number to a C \code{float}.
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000610
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000611\item[\samp{d} (float) {[double]}]
612Convert a Python floating point number to a C \code{double}.
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000613
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000614\item[\samp{O} (object) {[PyObject *]}]
615Store a Python object (without any conversion) in a C object pointer.
616The C program thus receives the actual object that was passed. The
617object's reference count is not increased. The pointer stored is not
618\code{NULL}.
619
620\item[\samp{O!} (object) {[\var{typeobject}, PyObject *]}]
621Store a Python object in a C object pointer. This is similar to
622\samp{O}, but takes two C arguments: the first is the address of a
623Python type object, the second is the address of the C variable (of
624type \code{PyObject *}) into which the object pointer is stored.
625If the Python object does not have the required type, a
626\code{TypeError} exception is raised.
627
628\item[\samp{O\&} (object) {[\var{converter}, \var{anything}]}]
629Convert a Python object to a C variable through a \var{converter}
630function. This takes two arguments: the first is a function, the
631second is the address of a C variable (of arbitrary type), converted
632to \code{void *}. The \var{converter} function in turn is called as
633follows:
634
635\code{\var{status} = \var{converter}(\var{object}, \var{address});}
636
637where \var{object} is the Python object to be converted and
638\var{address} is the \code{void *} argument that was passed to
639\code{PyArg_ConvertTuple()}. The returned \var{status} should be
640\code{1} for a successful conversion and \code{0} if the conversion
641has failed. When the conversion fails, the \var{converter} function
642should raise an exception.
643
644\item[\samp{S} (string) {[PyStringObject *]}]
645Like \samp{O} but raises a \code{TypeError} exception that the object
646is a string object. The C variable may also be declared as
647\code{PyObject *}.
648
649\item[\samp{(\var{items})} (tuple) {[\var{matching-items}]}]
650The object must be a Python tuple whose length is the number of format
651units in \var{items}. The C arguments must correspond to the
652individual format units in \var{items}. Format units for tuples may
653be nested.
Guido van Rossumdb65a6c1993-11-05 17:11:16 +0000654
655\end{description}
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000656
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000657It is possible to pass Python long integers where integers are
658requested; however no proper range checking is done -- the most
659significant bits are silently truncated when the receiving field is
660too small to receive the value (actually, the semantics are inherited
661from downcasts in C --- your milage may vary).
662
663A few other characters have a meaning in a format string. These may
664not occur inside nested parentheses. They are:
665
666\begin{description}
667
668\item[\samp{|}]
669Indicates that the remaining arguments in the Python argument list are
670optional. The C variables corresponding to optional arguments should
671be initialized to their default value --- when an optional argument is
672not specified, the \code{PyArg_ParseTuple} does not touch the contents
673of the corresponding C variable(s).
674
675\item[\samp{:}]
676The list of format units ends here; the string after the colon is used
677as the function name in error messages (the ``associated value'' of
678the exceptions that \code{PyArg_ParseTuple} raises).
679
680\item[\samp{;}]
681The list of format units ends here; the string after the colon is used
682as the error message \emph{instead} of the default error message.
683Clearly, \samp{:} and \samp{;} mutually exclude each other.
684
685\end{description}
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000686
687Some example calls:
688
689\begin{verbatim}
690 int ok;
691 int i, j;
692 long k, l;
693 char *s;
694 int size;
695
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000696 ok = PyArg_ParseTuple(args, ""); /* No arguments */
Guido van Rossum6938f061994-08-01 12:22:53 +0000697 /* Python call: f() */
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000698
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000699 ok = PyArg_ParseTuple(args, "s", &s); /* A string */
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000700 /* Possible Python call: f('whoops!') */
701
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000702 ok = PyArg_ParseTuple(args, "lls", &k, &l, &s); /* Two longs and a string */
Guido van Rossum6938f061994-08-01 12:22:53 +0000703 /* Possible Python call: f(1, 2, 'three') */
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000704
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000705 ok = PyArg_ParseTuple(args, "(ii)s#", &i, &j, &s, &size);
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000706 /* A pair of ints and a string, whose size is also returned */
707 /* Possible Python call: f(1, 2, 'three') */
708
709 {
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000710 char *file;
711 char *mode = "r";
712 int bufsize = 0;
713 ok = PyArg_ParseTuple(args, "s|si", &file, &mode, &bufsize);
714 /* A string, and optionally another string and an integer */
715 /* Possible Python calls:
716 f('spam')
717 f('spam', 'w')
718 f('spam', 'wb', 100000) */
719 }
720
721 {
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000722 int left, top, right, bottom, h, v;
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000723 ok = PyArg_ParseTuple(args, "((ii)(ii))(ii)",
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000724 &left, &top, &right, &bottom, &h, &v);
725 /* A rectangle and a point */
726 /* Possible Python call:
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000727 f(((0, 0), (400, 300)), (10, 10)) */
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000728 }
729\end{verbatim}
730
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000731
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000732\section{The {\tt Py_BuildValue()} Function}
733
734This function is the counterpart to \code{PyArg_ParseTuple()}. It is
735declared as follows:
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000736
737\begin{verbatim}
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000738 PyObject *Py_BuildValue(char *format, ...);
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000739\end{verbatim}
740
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000741It recognizes a set of format units similar to the ones recognized by
742\code{PyArg_ParseTuple()}, but the arguments (which are input to the
743function, not output) must not be pointers, just values. It returns a
744new Python object, suitable for returning from a C function called
745from Python.
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000746
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000747One difference with \code{PyArg_ParseTuple()}: while the latter
748requires its first argument to be a tuple (since Python argument lists
749are always represented as tuples internally), \code{BuildValue()} does
750not always build a tuple. It builds a tuple only if its format string
751contains two or more format units. If the format string is empty, it
752returns \code{None}; if it contains exactly one format unit, it
753returns whatever object is described by that format unit. To force it
754to return a tuple of size 0 or one, parenthesize the format string.
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000755
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000756In the following description, the quoted form is the format unit; the
757entry in (round) parentheses is the Python object type that the format
758unit will return; and the entry in [square] brackets is the type of
759the C value(s) to be passed.
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000760
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000761The characters space, tab, colon and comma are ignored in format
762strings (but not within format units such as \samp{s\#}). This can be
763used to make long format strings a tad more readable.
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000764
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000765\begin{description}
766
767\item[\samp{s} (string) {[char *]}]
768Convert a null-terminated C string to a Python object. If the C
769string pointer is \code{NULL}, \code{None} is returned.
770
771\item[\samp{s\#} (string) {[char *, int]}]
772Convert a C string and its length to a Python object. If the C string
773pointer is \code{NULL}, the length is ignored and \code{None} is
774returned.
775
776\item[\samp{z} (string or \code{None}) {[char *]}]
777Same as \samp{s}.
778
779\item[\samp{z\#} (string or \code{None}) {[char *, int]}]
780Same as \samp{s\#}.
781
782\item[\samp{i} (integer) {[int]}]
783Convert a plain C \code{int} to a Python integer object.
784
785\item[\samp{b} (integer) {[char]}]
786Same as \samp{i}.
787
788\item[\samp{h} (integer) {[short int]}]
789Same as \samp{i}.
790
791\item[\samp{l} (integer) {[long int]}]
792Convert a C \code{long int} to a Python integer object.
793
794\item[\samp{c} (string of length 1) {[char]}]
795Convert a C \code{int} representing a character to a Python string of
796length 1.
797
798\item[\samp{d} (float) {[double]}]
799Convert a C \code{double} to a Python floating point number.
800
801\item[\samp{f} (float) {[float]}]
802Same as \samp{d}.
803
804\item[\samp{O} (object) {[PyObject *]}]
805Pass a Python object untouched (except for its reference count, which
806is incremented by one). If the object passed in is a \code{NULL}
807pointer, it is assumed that this was caused because the call producing
808the argument found an error and set an exception. Therefore,
809\code{Py_BuildValue()} will return \code{NULL} but won't raise an
810exception. If no exception has been raised yet,
811\code{PyExc_SystemError} is set.
812
813\item[\samp{S} (object) {[PyObject *]}]
814Same as \samp{O}.
815
816\item[\samp{O\&} (object) {[\var{converter}, \var{anything}]}]
817Convert \var{anything} to a Python object through a \var{converter}
818function. The function is called with \var{anything} (which should be
819compatible with \code{void *}) as its argument and should return a
820``new'' Python object, or \code{NULL} if an error occurred.
821
822\item[\samp{(\var{items})} (tuple) {[\var{matching-items}]}]
823Convert a sequence of C values to a Python tuple with the same number
824of items.
825
826\item[\samp{[\var{items}]} (list) {[\var{matching-items}]}]
827Convert a sequence of C values to a Python list with the same number
828of items.
829
830\item[\samp{\{\var{items}\}} (dictionary) {[\var{matching-items}]}]
831Convert a sequence of C values to a Python dictionary. Each pair of
832consecutive C values adds one item to the dictionary, serving as key
833and value, respectively.
834
835\end{description}
836
837If there is an error in the format string, the
838\code{PyExc_SystemError} exception is raised and \code{NULL} returned.
839
840Examples (to the left the call, to the right the resulting Python value):
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000841
842\begin{verbatim}
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000843 Py_BuildValue("") None
844 Py_BuildValue("i", 123) 123
Guido van Rossumf23e0fe1995-03-18 11:04:29 +0000845 Py_BuildValue("iii", 123, 456, 789) (123, 456, 789)
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000846 Py_BuildValue("s", "hello") 'hello'
847 Py_BuildValue("ss", "hello", "world") ('hello', 'world')
848 Py_BuildValue("s#", "hello", 4) 'hell'
849 Py_BuildValue("()") ()
850 Py_BuildValue("(i)", 123) (123,)
851 Py_BuildValue("(ii)", 123, 456) (123, 456)
852 Py_BuildValue("(i,i)", 123, 456) (123, 456)
853 Py_BuildValue("[i,i]", 123, 456) [123, 456]
Guido van Rossumf23e0fe1995-03-18 11:04:29 +0000854 Py_BuildValue("{s:i,s:i}",
855 "abc", 123, "def", 456) {'abc': 123, 'def': 456}
856 Py_BuildValue("((ii)(ii)) (ii)",
857 1, 2, 3, 4, 5, 6) (((1, 2), (3, 4)), (5, 6))
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000858\end{verbatim}
859
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000860
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000861\section{Reference Counts}
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000862
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000863\subsection{Introduction}
Guido van Rossum7a2dba21993-11-05 14:45:11 +0000864
Guido van Rossum5049bcb1995-03-13 16:55:23 +0000865In languages like C or \Cpp{}, the programmer is responsible for
866dynamic allocation and deallocation of memory on the heap. In C, this
867is done using the functions \code{malloc()} and \code{free()}. In
868\Cpp{}, the operators \code{new} and \code{delete} are used with
869essentially the same meaning; they are actually implemented using
870\code{malloc()} and \code{free()}, so we'll restrict the following
871discussion to the latter.
872
873Every block of memory allocated with \code{malloc()} should eventually
874be returned to the pool of available memory by exactly one call to
875\code{free()}. It is important to call \code{free()} at the right
876time. If a block's address is forgotten but \code{free()} is not
877called for it, the memory it occupies cannot be reused until the
878program terminates. This is called a \dfn{memory leak}. On the other
879hand, if a program calls \code{free()} for a block and then continues
880to use the block, it creates a conflict with re-use of the block
881through another \code{malloc()} call. This is called \dfn{using freed
882memory} has the same bad consequences as referencing uninitialized
883data --- core dumps, wrong results, mysterious crashes.
884
885Common causes of memory leaks are unusual paths through the code. For
886instance, a function may allocate a block of memory, do some
887calculation, and then free the block again. Now a change in the
888requirements for the function may add a test to the calculation that
889detects an error condition and can return prematurely from the
890function. It's easy to forget to free the allocated memory block when
891taking this premature exit, especially when it is added later to the
892code. Such leaks, once introduced, often go undetected for a long
893time: the error exit is taken only in a small fraction of all calls,
894and most modern machines have plenty of virtual memory, so the leak
895only becomes apparent in a long-running process that uses the leaking
896function frequently. Therefore, it's important to prevent leaks from
897happening by having a coding convention or strategy that minimizes
898this kind of errors.
899
900Since Python makes heavy use of \code{malloc()} and \code{free()}, it
901needs a strategy to avoid memory leaks as well as the use of freed
902memory. The chosen method is called \dfn{reference counting}. The
903principle is simple: every object contains a counter, which is
904incremented when a reference to the object is stored somewhere, and
905which is decremented when a reference to it is deleted. When the
906counter reaches zero, the last reference to the object has been
907deleted and the object is freed.
908
909An alternative strategy is called \dfn{automatic garbage collection}.
910(Sometimes, reference counting is also referred to as a garbage
911collection strategy, hence my use of ``automatic'' to distinguish the
912two.) The big advantage of automatic garbage collection is that the
913user doesn't need to call \code{free()} explicitly. (Another claimed
914advantage is an improvement in speed or memory usage --- this is no
915hard fact however.) The disadvantage is that for C, there is no
916truly portable automatic garbage collector, while reference counting
917can be implemented portably (as long as the functions \code{malloc()}
918and \code{free()} are available --- which the C Standard guarantees).
919Maybe some day a sufficiently portable automatic garbage collector
920will be available for C. Until then, we'll have to live with
921reference counts.
922
923\subsection{Reference Counting in Python}
924
925There are two macros, \code{Py_INCREF(x)} and \code{Py_DECREF(x)},
926which handle the incrementing and decrementing of the reference count.
927\code{Py_DECREF()} also frees the object when the count reaches zero.
928For flexibility, it doesn't call \code{free()} directly --- rather, it
929makes a call through a function pointer in the object's \dfn{type
930object}. For this purpose (and others), every object also contains a
931pointer to its type object.
932
933The big question now remains: when to use \code{Py_INCREF(x)} and
934\code{Py_DECREF(x)}? Let's first introduce some terms. Nobody
935``owns'' an object; however, you can \dfn{own a reference} to an
936object. An object's reference count is now defined as the number of
937owned references to it. The owner of a reference is responsible for
938calling \code{Py_DECREF()} when the reference is no longer needed.
939Ownership of a reference can be transferred. There are three ways to
940dispose of an owned reference: pass it on, store it, or call
941\code{Py_DECREF()}. Forgetting to dispose of an owned reference creates
942a memory leak.
943
944It is also possible to \dfn{borrow}\footnote{The metaphor of
945``borrowing'' a reference is not completely correct: the owner still
946has a copy of the reference.} a reference to an object. The borrower
947of a reference should not call \code{Py_DECREF()}. The borrower must
948not hold on to the object longer than the owner from which it was
949borrowed. Using a borrowed reference after the owner has disposed of
950it risks using freed memory and should be avoided
951completely.\footnote{Checking that the reference count is at least 1
952\strong{does not work} --- the reference count itself could be in
953freed memory and may thus be reused for another object!}
954
955The advantage of borrowing over owning a reference is that you don't
956need to take care of disposing of the reference on all possible paths
957through the code --- in other words, with a borrowed reference you
958don't run the risk of leaking when a premature exit is taken. The
959disadvantage of borrowing over leaking is that there are some subtle
960situations where in seemingly correct code a borrowed reference can be
961used after the owner from which it was borrowed has in fact disposed
962of it.
963
964A borrowed reference can be changed into an owned reference by calling
965\code{Py_INCREF()}. This does not affect the status of the owner from
966which the reference was borrowed --- it creates a new owned reference,
967and gives full owner responsibilities (i.e., the new owner must
968dispose of the reference properly, as well as the previous owner).
969
970\subsection{Ownership Rules}
971
972Whenever an object reference is passed into or out of a function, it
973is part of the function's interface specification whether ownership is
974transferred with the reference or not.
975
976Most functions that return a reference to an object pass on ownership
977with the reference. In particular, all functions whose function it is
978to create a new object, e.g.\ \code{PyInt_FromLong()} and
979\code{Py_BuildValue()}, pass ownership to the receiver. Even if in
980fact, in some cases, you don't receive a reference to a brand new
981object, you still receive ownership of the reference. For instance,
982\code{PyInt_FromLong()} maintains a cache of popular values and can
983return a reference to a cached item.
984
985Many functions that extract objects from other objects also transfer
986ownership with the reference, for instance
987\code{PyObject_GetAttrString()}. The picture is less clear, here,
988however, since a few common routines are exceptions:
989\code{PyTuple_GetItem()}, \code{PyList_GetItem()} and
990\code{PyDict_GetItem()} (and \code{PyDict_GetItemString()}) all return
991references that you borrow from the tuple, list or dictionary.
992
993The function \code{PyImport_AddModule()} also returns a borrowed
994reference, even though it may actually create the object it returns:
995this is possible because an owned reference to the object is stored in
996\code{sys.modules}.
997
998When you pass an object reference into another function, in general,
999the function borrows the reference from you --- if it needs to store
1000it, it will use \code{Py_INCREF()} to become an independent owner.
1001There are exactly two important exceptions to this rule:
1002\code{PyTuple_SetItem()} and \code{PyList_SetItem()}. These functions
1003take over ownership of the item passed to them --- even if they fail!
1004(Note that \code{PyDict_SetItem()} and friends don't take over
1005ownership --- they are ``normal''.)
1006
1007When a C function is called from Python, it borrows references to its
1008arguments from the caller. The caller owns a reference to the object,
1009so the borrowed reference's lifetime is guaranteed until the function
1010returns. Only when such a borrowed reference must be stored or passed
1011on, it must be turned into an owned reference by calling
1012\code{Py_INCREF()}.
1013
1014The object reference returned from a C function that is called from
1015Python must be an owned reference --- ownership is tranferred from the
1016function to its caller.
1017
1018\subsection{Thin Ice}
1019
1020There are a few situations where seemingly harmless use of a borrowed
1021reference can lead to problems. These all have to do with implicit
1022invocations of the interpreter, which can cause the owner of a
1023reference to dispose of it.
1024
1025The first and most important case to know about is using
1026\code{Py_DECREF()} on an unrelated object while borrowing a reference
1027to a list item. For instance:
Guido van Rossum7a2dba21993-11-05 14:45:11 +00001028
1029\begin{verbatim}
Guido van Rossum5049bcb1995-03-13 16:55:23 +00001030bug(PyObject *list) {
1031 PyObject *item = PyList_GetItem(list, 0);
1032 PyList_SetItem(list, 1, PyInt_FromLong(0L));
1033 PyObject_Print(item, stdout, 0); /* BUG! */
1034}
Guido van Rossum7a2dba21993-11-05 14:45:11 +00001035\end{verbatim}
1036
Guido van Rossum5049bcb1995-03-13 16:55:23 +00001037This function first borrows a reference to \code{list[0]}, then
1038replaces \code{list[1]} with the value \code{0}, and finally prints
1039the borrowed reference. Looks harmless, right? But it's not!
1040
1041Let's follow the control flow into \code{PyList_SetItem()}. The list
1042owns references to all its items, so when item 1 is replaced, it has
1043to dispose of the original item 1. Now let's suppose the original
1044item 1 was an instance of a user-defined class, and let's further
1045suppose that the class defined a \code{__del__()} method. If this
1046class instance has a reference count of 1, disposing of it will call
1047its \code{__del__()} method.
1048
1049Since it is written in Python, the \code{__del__()} method can execute
1050arbitrary Python code. Could it perhaps do something to invalidate
1051the reference to \code{item} in \code{bug()}? You bet! Assuming that
1052the list passed into \code{bug()} is accessible to the
1053\code{__del__()} method, it could execute a statement to the effect of
1054\code{del list[0]}, and assuming this was the last reference to that
1055object, it would free the memory associated with it, thereby
1056invalidating \code{item}.
1057
1058The solution, once you know the source of the problem, is easy:
1059temporarily increment the reference count. The correct version of the
1060function reads:
1061
1062\begin{verbatim}
1063no_bug(PyObject *list) {
1064 PyObject *item = PyList_GetItem(list, 0);
1065 Py_INCREF(item);
1066 PyList_SetItem(list, 1, PyInt_FromLong(0L));
1067 PyObject_Print(item, stdout, 0);
1068 Py_DECREF(item);
1069}
1070\end{verbatim}
1071
1072This is a true story. An older version of Python contained variants
1073of this bug and someone spent a considerable amount of time in a C
1074debugger to figure out why his \code{__del__()} methods would fail...
1075
1076The second case of problems with a borrowed reference is a variant
1077involving threads. Normally, multiple threads in the Python
1078interpreter can't get in each other's way, because there is a global
1079lock protecting Python's entire object space. However, it is possible
1080to temporarily release this lock using the macro
1081\code{Py_BEGIN_ALLOW_THREADS}, and to re-acquire it using
1082\code{Py_END_ALLOW_THREADS}. This is common around blocking I/O
1083calls, to let other threads use the CPU while waiting for the I/O to
1084complete. Obviously, the following function has the same problem as
1085the previous one:
1086
1087\begin{verbatim}
1088bug(PyObject *list) {
1089 PyObject *item = PyList_GetItem(list, 0);
1090 Py_BEGIN_ALLOW_THREADS
1091 ...some blocking I/O call...
1092 Py_END_ALLOW_THREADS
1093 PyObject_Print(item, stdout, 0); /* BUG! */
1094}
1095\end{verbatim}
1096
1097\subsection{NULL Pointers}
1098
1099In general, functions that take object references as arguments don't
1100expect you to pass them \code{NULL} pointers, and will dump core (or
1101cause later core dumps) if you do so. Functions that return object
1102references generally return \code{NULL} only to indicate that an
1103exception occurred. The reason for not testing for \code{NULL}
1104arguments is that functions often pass the objects they receive on to
1105other function --- if each function were to test for \code{NULL},
1106there would be a lot of redundant tests and the code would run slower.
1107
1108It is better to test for \code{NULL} only at the ``source'', i.e.\
1109when a pointer that may be \code{NULL} is received, e.g.\ from
1110\code{malloc()} or from a function that may raise an exception.
1111
1112The macros \code{Py_INCREF()} and \code{Py_DECREF()}
1113don't check for \code{NULL} pointers --- however, their variants
1114\code{Py_XINCREF()} and \code{Py_XDECREF()} do.
1115
1116The macros for checking for a particular object type
1117(\code{Py\var{type}_Check()}) don't check for \code{NULL} pointers ---
1118again, there is much code that calls several of these in a row to test
1119an object against various different expected types, and this would
1120generate redundant tests. There are no variants with \code{NULL}
1121checking.
1122
1123The C function calling mechanism guarantees that the argument list
1124passed to C functions (\code{args} in the examples) is never
1125\code{NULL} --- in fact it guarantees that it is always a tuple.%
1126\footnote{These guarantees don't hold when you use the ``old'' style
1127calling convention --- this is still found in much existing code.}
1128
1129It is a severe error to ever let a \code{NULL} pointer ``escape'' to
1130the Python user.
Guido van Rossumdb65a6c1993-11-05 17:11:16 +00001131
Guido van Rossum7a2dba21993-11-05 14:45:11 +00001132
Guido van Rossum5049bcb1995-03-13 16:55:23 +00001133\section{Writing Extensions in \Cpp{}}
Guido van Rossumdb65a6c1993-11-05 17:11:16 +00001134
Guido van Rossum16d6e711994-08-08 12:30:22 +00001135It is possible to write extension modules in \Cpp{}. Some restrictions
Guido van Rossumed39cd01995-10-08 00:17:19 +00001136apply. If the main program (the Python interpreter) is compiled and
Guido van Rossumdb65a6c1993-11-05 17:11:16 +00001137linked by the C compiler, global or static objects with constructors
Guido van Rossumed39cd01995-10-08 00:17:19 +00001138cannot be used. This is not a problem if the main program is linked
1139by the \Cpp{} compiler. All functions that will be called directly or
Guido van Rossumdb65a6c1993-11-05 17:11:16 +00001140indirectly (i.e. via function pointers) by the Python interpreter will
1141have to be declared using \code{extern "C"}; this applies to all
Guido van Rossumb92112d1995-03-20 14:24:09 +00001142``methods'' as well as to the module's initialization function.
Guido van Rossumdb65a6c1993-11-05 17:11:16 +00001143It is unnecessary to enclose the Python header files in
Guido van Rossum5049bcb1995-03-13 16:55:23 +00001144\code{extern "C" \{...\}} --- they use this form already if the symbol
1145\samp{__cplusplus} is defined (all recent C++ compilers define this
1146symbol).
Guido van Rossum7a2dba21993-11-05 14:45:11 +00001147
1148\chapter{Embedding Python in another application}
1149
1150Embedding Python is similar to extending it, but not quite. The
1151difference is that when you extend Python, the main program of the
Guido van Rossum16d6e711994-08-08 12:30:22 +00001152application is still the Python interpreter, while if you embed
Guido van Rossumdb65a6c1993-11-05 17:11:16 +00001153Python, the main program may have nothing to do with Python ---
Guido van Rossum7a2dba21993-11-05 14:45:11 +00001154instead, some parts of the application occasionally call the Python
1155interpreter to run some Python code.
1156
1157So if you are embedding Python, you are providing your own main
1158program. One of the things this main program has to do is initialize
1159the Python interpreter. At the very least, you have to call the
Guido van Rossum5049bcb1995-03-13 16:55:23 +00001160function \code{Py_Initialize()}. There are optional calls to pass command
Guido van Rossumdb65a6c1993-11-05 17:11:16 +00001161line arguments to Python. Then later you can call the interpreter
1162from any part of the application.
Guido van Rossum7a2dba21993-11-05 14:45:11 +00001163
1164There are several different ways to call the interpreter: you can pass
Guido van Rossum5049bcb1995-03-13 16:55:23 +00001165a string containing Python statements to \code{PyRun_SimpleString()},
1166or you can pass a stdio file pointer and a file name (for
1167identification in error messages only) to \code{PyRun_SimpleFile()}. You
1168can also call the lower-level operations described in the previous
1169chapters to construct and use Python objects.
Guido van Rossum7a2dba21993-11-05 14:45:11 +00001170
1171A simple demo of embedding Python can be found in the directory
Guido van Rossum6938f061994-08-01 12:22:53 +00001172\file{Demo/embed}.
Guido van Rossumdb65a6c1993-11-05 17:11:16 +00001173
Guido van Rossum7a2dba21993-11-05 14:45:11 +00001174
Guido van Rossum16d6e711994-08-08 12:30:22 +00001175\section{Embedding Python in \Cpp{}}
Guido van Rossum7a2dba21993-11-05 14:45:11 +00001176
Guido van Rossum16d6e711994-08-08 12:30:22 +00001177It is also possible to embed Python in a \Cpp{} program; precisely how this
1178is done will depend on the details of the \Cpp{} system used; in general you
1179will need to write the main program in \Cpp{}, and use the \Cpp{} compiler
1180to compile and link your program. There is no need to recompile Python
1181itself using \Cpp{}.
Guido van Rossum7a2dba21993-11-05 14:45:11 +00001182
Guido van Rossum6f0132f1993-11-19 13:13:22 +00001183
1184\chapter{Dynamic Loading}
1185
Guido van Rossum6938f061994-08-01 12:22:53 +00001186On most modern systems it is possible to configure Python to support
1187dynamic loading of extension modules implemented in C. When shared
1188libraries are used dynamic loading is configured automatically;
1189otherwise you have to select it as a build option (see below). Once
1190configured, dynamic loading is trivial to use: when a Python program
Guido van Rossum5049bcb1995-03-13 16:55:23 +00001191executes \code{import spam}, the search for modules tries to find a
1192file \file{spammodule.o} (\file{spammodule.so} when using shared
Guido van Rossum6938f061994-08-01 12:22:53 +00001193libraries) in the module search path, and if one is found, it is
1194loaded into the executing binary and executed. Once loaded, the
1195module acts just like a built-in extension module.
Guido van Rossum6f0132f1993-11-19 13:13:22 +00001196
Guido van Rossumb92112d1995-03-20 14:24:09 +00001197The advantages of dynamic loading are twofold: the ``core'' Python
Guido van Rossum6f0132f1993-11-19 13:13:22 +00001198binary gets smaller, and users can extend Python with their own
1199modules implemented in C without having to build and maintain their
1200own copy of the Python interpreter. There are also disadvantages:
1201dynamic loading isn't available on all systems (this just means that
1202on some systems you have to use static loading), and dynamically
1203loading a module that was compiled for a different version of Python
Guido van Rossum6938f061994-08-01 12:22:53 +00001204(e.g. with a different representation of objects) may dump core.
Guido van Rossum6f0132f1993-11-19 13:13:22 +00001205
1206
Guido van Rossum5049bcb1995-03-13 16:55:23 +00001207\section{Configuring and Building the Interpreter for Dynamic Loading}
Guido van Rossum6f0132f1993-11-19 13:13:22 +00001208
Guido van Rossum6938f061994-08-01 12:22:53 +00001209There are three styles of dynamic loading: one using shared libraries,
1210one using SGI IRIX 4 dynamic loading, and one using GNU dynamic
1211loading.
Guido van Rossum6f0132f1993-11-19 13:13:22 +00001212
Guido van Rossum5049bcb1995-03-13 16:55:23 +00001213\subsection{Shared Libraries}
Guido van Rossum6f0132f1993-11-19 13:13:22 +00001214
Guido van Rossum16d6e711994-08-08 12:30:22 +00001215The following systems support dynamic loading using shared libraries:
Guido van Rossum6938f061994-08-01 12:22:53 +00001216SunOS 4; Solaris 2; SGI IRIX 5 (but not SGI IRIX 4!); and probably all
1217systems derived from SVR4, or at least those SVR4 derivatives that
1218support shared libraries (are there any that don't?).
Guido van Rossum6f0132f1993-11-19 13:13:22 +00001219
Guido van Rossum6938f061994-08-01 12:22:53 +00001220You don't need to do anything to configure dynamic loading on these
1221systems --- the \file{configure} detects the presence of the
1222\file{<dlfcn.h>} header file and automatically configures dynamic
1223loading.
Guido van Rossum6f0132f1993-11-19 13:13:22 +00001224
Guido van Rossum5049bcb1995-03-13 16:55:23 +00001225\subsection{SGI IRIX 4 Dynamic Loading}
Guido van Rossum6f0132f1993-11-19 13:13:22 +00001226
Guido van Rossum6938f061994-08-01 12:22:53 +00001227Only SGI IRIX 4 supports dynamic loading of modules using SGI dynamic
1228loading. (SGI IRIX 5 might also support it but it is inferior to
1229using shared libraries so there is no reason to; a small test didn't
1230work right away so I gave up trying to support it.)
Guido van Rossum6f0132f1993-11-19 13:13:22 +00001231
Guido van Rossum6938f061994-08-01 12:22:53 +00001232Before you build Python, you first need to fetch and build the \code{dl}
1233package written by Jack Jansen. This is available by anonymous ftp
1234from host \file{ftp.cwi.nl}, directory \file{pub/dynload}, file
1235\file{dl-1.6.tar.Z}. (The version number may change.) Follow the
1236instructions in the package's \file{README} file to build it.
1237
1238Once you have built \code{dl}, you can configure Python to use it. To
1239this end, you run the \file{configure} script with the option
1240\code{--with-dl=\var{directory}} where \var{directory} is the absolute
1241pathname of the \code{dl} directory.
1242
1243Now build and install Python as you normally would (see the
1244\file{README} file in the toplevel Python directory.)
1245
Guido van Rossum5049bcb1995-03-13 16:55:23 +00001246\subsection{GNU Dynamic Loading}
Guido van Rossum6938f061994-08-01 12:22:53 +00001247
1248GNU dynamic loading supports (according to its \file{README} file) the
1249following hardware and software combinations: VAX (Ultrix), Sun 3
1250(SunOS 3.4 and 4.0), Sparc (SunOS 4.0), Sequent Symmetry (Dynix), and
1251Atari ST. There is no reason to use it on a Sparc; I haven't seen a
1252Sun 3 for years so I don't know if these have shared libraries or not.
1253
1254You need to fetch and build two packages. One is GNU DLD 3.2.3,
1255available by anonymous ftp from host \file{ftp.cwi.nl}, directory
1256\file{pub/dynload}, file \file{dld-3.2.3.tar.Z}. (As far as I know,
1257no further development on GNU DLD is being done.) The other is an
1258emulation of Jack Jansen's \code{dl} package that I wrote on top of
1259GNU DLD 3.2.3. This is available from the same host and directory,
1260file dl-dld-1.1.tar.Z. (The version number may change --- but I doubt
1261it will.) Follow the instructions in each package's \file{README}
1262file to configure build them.
1263
1264Now configure Python. Run the \file{configure} script with the option
1265\code{--with-dl-dld=\var{dl-directory},\var{dld-directory}} where
1266\var{dl-directory} is the absolute pathname of the directory where you
1267have built the \file{dl-dld} package, and \var{dld-directory} is that
1268of the GNU DLD package. The Python interpreter you build hereafter
1269will support GNU dynamic loading.
Guido van Rossum6f0132f1993-11-19 13:13:22 +00001270
1271
Guido van Rossum5049bcb1995-03-13 16:55:23 +00001272\section{Building a Dynamically Loadable Module}
Guido van Rossum6f0132f1993-11-19 13:13:22 +00001273
Guido van Rossum6938f061994-08-01 12:22:53 +00001274Since there are three styles of dynamic loading, there are also three
1275groups of instructions for building a dynamically loadable module.
1276Instructions common for all three styles are given first. Assuming
Guido van Rossum5049bcb1995-03-13 16:55:23 +00001277your module is called \code{spam}, the source filename must be
1278\file{spammodule.c}, so the object name is \file{spammodule.o}. The
Guido van Rossum6938f061994-08-01 12:22:53 +00001279module must be written as a normal Python extension module (as
1280described earlier).
Guido van Rossum6f0132f1993-11-19 13:13:22 +00001281
Guido van Rossum6938f061994-08-01 12:22:53 +00001282Note that in all cases you will have to create your own Makefile that
1283compiles your module file(s). This Makefile will have to pass two
1284\samp{-I} arguments to the C compiler which will make it find the
1285Python header files. If the Make variable \var{PYTHONTOP} points to
1286the toplevel Python directory, your \var{CFLAGS} Make variable should
1287contain the options \samp{-I\$(PYTHONTOP) -I\$(PYTHONTOP)/Include}.
1288(Most header files are in the \file{Include} subdirectory, but the
Guido van Rossum305ed111996-08-19 22:59:46 +00001289\file{config.h} header lives in the toplevel directory.)
Guido van Rossum6f0132f1993-11-19 13:13:22 +00001290
Guido van Rossum6f0132f1993-11-19 13:13:22 +00001291
Guido van Rossum5049bcb1995-03-13 16:55:23 +00001292\subsection{Shared Libraries}
Guido van Rossum6f0132f1993-11-19 13:13:22 +00001293
Guido van Rossum6938f061994-08-01 12:22:53 +00001294You must link the \samp{.o} file to produce a shared library. This is
1295done using a special invocation of the \UNIX{} loader/linker, {\em
1296ld}(1). Unfortunately the invocation differs slightly per system.
Guido van Rossum6f0132f1993-11-19 13:13:22 +00001297
Guido van Rossum6938f061994-08-01 12:22:53 +00001298On SunOS 4, use
Guido van Rossum6f0132f1993-11-19 13:13:22 +00001299\begin{verbatim}
Guido van Rossum5049bcb1995-03-13 16:55:23 +00001300 ld spammodule.o -o spammodule.so
Guido van Rossum6f0132f1993-11-19 13:13:22 +00001301\end{verbatim}
1302
Guido van Rossum6938f061994-08-01 12:22:53 +00001303On Solaris 2, use
1304\begin{verbatim}
Guido van Rossum5049bcb1995-03-13 16:55:23 +00001305 ld -G spammodule.o -o spammodule.so
Guido van Rossum6938f061994-08-01 12:22:53 +00001306\end{verbatim}
Guido van Rossum6f0132f1993-11-19 13:13:22 +00001307
Guido van Rossum6938f061994-08-01 12:22:53 +00001308On SGI IRIX 5, use
1309\begin{verbatim}
Guido van Rossum5049bcb1995-03-13 16:55:23 +00001310 ld -shared spammodule.o -o spammodule.so
Guido van Rossum6938f061994-08-01 12:22:53 +00001311\end{verbatim}
1312
Guido van Rossumb92112d1995-03-20 14:24:09 +00001313On other systems, consult the manual page for \code{ld}(1) to find what
Guido van Rossum6938f061994-08-01 12:22:53 +00001314flags, if any, must be used.
1315
1316If your extension module uses system libraries that haven't already
1317been linked with Python (e.g. a windowing system), these must be
Guido van Rossumb92112d1995-03-20 14:24:09 +00001318passed to the \code{ld} command as \samp{-l} options after the
Guido van Rossum6938f061994-08-01 12:22:53 +00001319\samp{.o} file.
1320
Guido van Rossum5049bcb1995-03-13 16:55:23 +00001321The resulting file \file{spammodule.so} must be copied into a directory
Guido van Rossum6938f061994-08-01 12:22:53 +00001322along the Python module search path.
Guido van Rossum6f0132f1993-11-19 13:13:22 +00001323
1324
Guido van Rossum5049bcb1995-03-13 16:55:23 +00001325\subsection{SGI IRIX 4 Dynamic Loading}
Guido van Rossum6f0132f1993-11-19 13:13:22 +00001326
Guido van Rossum7ec59571995-04-07 15:35:33 +00001327{\bf IMPORTANT:} You must compile your extension module with the
Guido van Rossum6938f061994-08-01 12:22:53 +00001328additional C flag \samp{-G0} (or \samp{-G 0}). This instruct the
1329assembler to generate position-independent code.
Guido van Rossum6f0132f1993-11-19 13:13:22 +00001330
Guido van Rossum5049bcb1995-03-13 16:55:23 +00001331You don't need to link the resulting \file{spammodule.o} file; just
Guido van Rossum6938f061994-08-01 12:22:53 +00001332copy it into a directory along the Python module search path.
1333
1334The first time your extension is loaded, it takes some extra time and
1335a few messages may be printed. This creates a file
Guido van Rossum5049bcb1995-03-13 16:55:23 +00001336\file{spammodule.ld} which is an image that can be loaded quickly into
Guido van Rossum6938f061994-08-01 12:22:53 +00001337the Python interpreter process. When a new Python interpreter is
1338installed, the \code{dl} package detects this and rebuilds
Guido van Rossum5049bcb1995-03-13 16:55:23 +00001339\file{spammodule.ld}. The file \file{spammodule.ld} is placed in the
1340directory where \file{spammodule.o} was found, unless this directory is
Guido van Rossum6938f061994-08-01 12:22:53 +00001341unwritable; in that case it is placed in a temporary
1342directory.\footnote{Check the manual page of the \code{dl} package for
1343details.}
1344
1345If your extension modules uses additional system libraries, you must
Guido van Rossum5049bcb1995-03-13 16:55:23 +00001346create a file \file{spammodule.libs} in the same directory as the
1347\file{spammodule.o}. This file should contain one or more lines with
Guido van Rossum6938f061994-08-01 12:22:53 +00001348whitespace-separated options that will be passed to the linker ---
1349normally only \samp{-l} options or absolute pathnames of libraries
1350(\samp{.a} files) should be used.
Guido van Rossum6f0132f1993-11-19 13:13:22 +00001351
1352
Guido van Rossum5049bcb1995-03-13 16:55:23 +00001353\subsection{GNU Dynamic Loading}
Guido van Rossum6f0132f1993-11-19 13:13:22 +00001354
Guido van Rossum5049bcb1995-03-13 16:55:23 +00001355Just copy \file{spammodule.o} into a directory along the Python module
Guido van Rossum6938f061994-08-01 12:22:53 +00001356search path.
Guido van Rossum6f0132f1993-11-19 13:13:22 +00001357
Guido van Rossum6938f061994-08-01 12:22:53 +00001358If your extension modules uses additional system libraries, you must
Guido van Rossum5049bcb1995-03-13 16:55:23 +00001359create a file \file{spammodule.libs} in the same directory as the
1360\file{spammodule.o}. This file should contain one or more lines with
Guido van Rossum6938f061994-08-01 12:22:53 +00001361whitespace-separated absolute pathnames of libraries (\samp{.a}
1362files). No \samp{-l} options can be used.
Guido van Rossum6f0132f1993-11-19 13:13:22 +00001363
1364
Guido van Rossum267e80d1996-08-09 21:01:07 +00001365\input{extref}
1366
Guido van Rossum7a2dba21993-11-05 14:45:11 +00001367\input{ext.ind}
1368
1369\end{document}