blob: 4942b1a46980c5a2ec99260eece4ecba333cefa0 [file] [log] [blame]
Georg Brandl116aa622007-08-15 14:28:22 +00001.. highlightlang:: c
2
3
4.. _api-intro:
5
6************
7Introduction
8************
9
10The Application Programmer's Interface to Python gives C and C++ programmers
11access to the Python interpreter at a variety of levels. The API is equally
12usable from C++, but for brevity it is generally referred to as the Python/C
13API. There are two fundamentally different reasons for using the Python/C API.
14The first reason is to write *extension modules* for specific purposes; these
15are C modules that extend the Python interpreter. This is probably the most
16common use. The second reason is to use Python as a component in a larger
17application; this technique is generally referred to as :dfn:`embedding` Python
18in an application.
19
Barry Warsawb2e57942017-09-14 18:13:16 -070020Writing an extension module is a relatively well-understood process, where a
21"cookbook" approach works well. There are several tools that automate the
22process to some extent. While people have embedded Python in other
23applications since its early existence, the process of embedding Python is
24less straightforward than writing an extension.
Georg Brandl116aa622007-08-15 14:28:22 +000025
26Many API functions are useful independent of whether you're embedding or
27extending Python; moreover, most applications that embed Python will need to
28provide a custom extension as well, so it's probably a good idea to become
29familiar with writing an extension before attempting to embed Python in a real
30application.
31
32
Barry Warsawb2e57942017-09-14 18:13:16 -070033Coding standards
34================
35
36If you're writing C code for inclusion in CPython, you **must** follow the
37guidelines and standards defined in :PEP:`7`. These guidelines apply
38regardless of the version of Python you are contributing to. Following these
39conventions is not necessary for your own third party extension modules,
40unless you eventually expect to contribute them to Python.
41
42
Georg Brandl116aa622007-08-15 14:28:22 +000043.. _api-includes:
44
45Include Files
46=============
47
48All function, type and macro definitions needed to use the Python/C API are
49included in your code by the following line::
50
51 #include "Python.h"
52
53This implies inclusion of the following standard headers: ``<stdio.h>``,
Georg Brandl4f13d612010-11-23 18:14:57 +000054``<string.h>``, ``<errno.h>``, ``<limits.h>``, ``<assert.h>`` and ``<stdlib.h>``
55(if available).
Georg Brandl116aa622007-08-15 14:28:22 +000056
Georg Brandle720c0a2009-04-27 16:20:50 +000057.. note::
Georg Brandl116aa622007-08-15 14:28:22 +000058
59 Since Python may define some pre-processor definitions which affect the standard
60 headers on some systems, you *must* include :file:`Python.h` before any standard
61 headers are included.
62
63All user visible names defined by Python.h (except those defined by the included
64standard headers) have one of the prefixes ``Py`` or ``_Py``. Names beginning
65with ``_Py`` are for internal use by the Python implementation and should not be
66used by extension writers. Structure member names do not have a reserved prefix.
67
68**Important:** user code should never define names that begin with ``Py`` or
69``_Py``. This confuses the reader, and jeopardizes the portability of the user
70code to future Python versions, which may define additional names beginning with
71one of these prefixes.
72
73The header files are typically installed with Python. On Unix, these are
74located in the directories :file:`{prefix}/include/pythonversion/` and
75:file:`{exec_prefix}/include/pythonversion/`, where :envvar:`prefix` and
76:envvar:`exec_prefix` are defined by the corresponding parameters to Python's
Serhiy Storchaka885bdc42016-02-11 13:10:36 +020077:program:`configure` script and *version* is
78``'%d.%d' % sys.version_info[:2]``. On Windows, the headers are installed
79in :file:`{prefix}/include`, where :envvar:`prefix` is the installation
80directory specified to the installer.
Georg Brandl116aa622007-08-15 14:28:22 +000081
82To include the headers, place both directories (if different) on your compiler's
83search path for includes. Do *not* place the parent directories on the search
84path and then use ``#include <pythonX.Y/Python.h>``; this will break on
85multi-platform builds since the platform independent headers under
86:envvar:`prefix` include the platform specific headers from
87:envvar:`exec_prefix`.
88
89C++ users should note that though the API is defined entirely using C, the
90header files do properly declare the entry points to be ``extern "C"``, so there
91is no need to do anything special to use the API from C++.
92
93
Barry Warsawb2e57942017-09-14 18:13:16 -070094Useful macros
95=============
96
97Several useful macros are defined in the Python header files. Many are
98defined closer to where they are useful (e.g. :c:macro:`Py_RETURN_NONE`).
99Others of a more general utility are defined here. This is not necessarily a
100complete listing.
101
102.. c:macro:: Py_UNREACHABLE()
103
104 Use this when you have a code path that you do not expect to be reached.
105 For example, in the ``default:`` clause in a ``switch`` statement for which
106 all possible values are covered in ``case`` statements. Use this in places
107 where you might be tempted to put an ``assert(0)`` or ``abort()`` call.
108
109.. c:macro:: Py_ABS(x)
110
111 Return the absolute value of ``x``.
112
113.. c:macro:: Py_MIN(x, y)
114
115 Return the minimum value between ``x`` and ``y``.
116
117.. c:macro:: Py_MAX(x, y)
118
119 Return the maximum value between ``x`` and ``y``.
120
121.. c:macro:: Py_STRINGIFY(x)
122
123 Convert ``x`` to a C string. E.g. ``Py_STRINGIFY(123)`` returns
124 ``"123"``.
125
126.. c:macro:: Py_MEMBER_SIZE(type, member)
127
128 Return the size of a structure (``type``) ``member`` in bytes.
129
130.. c:macro:: Py_CHARMASK(c)
131
132 Argument must be a character or an integer in the range [-128, 127] or [0,
133 255]. This macro returns ``c`` cast to an ``unsigned char``.
134
135
Georg Brandl116aa622007-08-15 14:28:22 +0000136.. _api-objects:
137
138Objects, Types and Reference Counts
139===================================
140
141.. index:: object: type
142
143Most Python/C API functions have one or more arguments as well as a return value
Georg Brandl60203b42010-10-06 10:11:56 +0000144of type :c:type:`PyObject\*`. This type is a pointer to an opaque data type
Georg Brandl116aa622007-08-15 14:28:22 +0000145representing an arbitrary Python object. Since all Python object types are
146treated the same way by the Python language in most situations (e.g.,
147assignments, scope rules, and argument passing), it is only fitting that they
148should be represented by a single C type. Almost all Python objects live on the
149heap: you never declare an automatic or static variable of type
Georg Brandl60203b42010-10-06 10:11:56 +0000150:c:type:`PyObject`, only pointer variables of type :c:type:`PyObject\*` can be
Georg Brandl116aa622007-08-15 14:28:22 +0000151declared. The sole exception are the type objects; since these must never be
Georg Brandl60203b42010-10-06 10:11:56 +0000152deallocated, they are typically static :c:type:`PyTypeObject` objects.
Georg Brandl116aa622007-08-15 14:28:22 +0000153
154All Python objects (even Python integers) have a :dfn:`type` and a
155:dfn:`reference count`. An object's type determines what kind of object it is
156(e.g., an integer, a list, or a user-defined function; there are many more as
157explained in :ref:`types`). For each of the well-known types there is a macro
158to check whether an object is of that type; for instance, ``PyList_Check(a)`` is
159true if (and only if) the object pointed to by *a* is a Python list.
160
161
162.. _api-refcounts:
163
164Reference Counts
165----------------
166
167The reference count is important because today's computers have a finite (and
168often severely limited) memory size; it counts how many different places there
169are that have a reference to an object. Such a place could be another object,
170or a global (or static) C variable, or a local variable in some C function.
171When an object's reference count becomes zero, the object is deallocated. If
172it contains references to other objects, their reference count is decremented.
173Those other objects may be deallocated in turn, if this decrement makes their
174reference count become zero, and so on. (There's an obvious problem with
175objects that reference each other here; for now, the solution is "don't do
176that.")
177
178.. index::
179 single: Py_INCREF()
180 single: Py_DECREF()
181
182Reference counts are always manipulated explicitly. The normal way is to use
Georg Brandl60203b42010-10-06 10:11:56 +0000183the macro :c:func:`Py_INCREF` to increment an object's reference count by one,
184and :c:func:`Py_DECREF` to decrement it by one. The :c:func:`Py_DECREF` macro
Georg Brandl116aa622007-08-15 14:28:22 +0000185is considerably more complex than the incref one, since it must check whether
186the reference count becomes zero and then cause the object's deallocator to be
187called. The deallocator is a function pointer contained in the object's type
188structure. The type-specific deallocator takes care of decrementing the
189reference counts for other objects contained in the object if this is a compound
190object type, such as a list, as well as performing any additional finalization
191that's needed. There's no chance that the reference count can overflow; at
192least as many bits are used to hold the reference count as there are distinct
Christian Heimesdd15f6c2008-03-16 00:07:10 +0000193memory locations in virtual memory (assuming ``sizeof(Py_ssize_t) >= sizeof(void*)``).
Georg Brandl116aa622007-08-15 14:28:22 +0000194Thus, the reference count increment is a simple operation.
195
196It is not necessary to increment an object's reference count for every local
197variable that contains a pointer to an object. In theory, the object's
198reference count goes up by one when the variable is made to point to it and it
199goes down by one when the variable goes out of scope. However, these two
200cancel each other out, so at the end the reference count hasn't changed. The
201only real reason to use the reference count is to prevent the object from being
202deallocated as long as our variable is pointing to it. If we know that there
203is at least one other reference to the object that lives at least as long as
204our variable, there is no need to increment the reference count temporarily.
205An important situation where this arises is in objects that are passed as
206arguments to C functions in an extension module that are called from Python;
207the call mechanism guarantees to hold a reference to every argument for the
208duration of the call.
209
210However, a common pitfall is to extract an object from a list and hold on to it
211for a while without incrementing its reference count. Some other operation might
212conceivably remove the object from the list, decrementing its reference count
213and possible deallocating it. The real danger is that innocent-looking
214operations may invoke arbitrary Python code which could do this; there is a code
Georg Brandl60203b42010-10-06 10:11:56 +0000215path which allows control to flow back to the user from a :c:func:`Py_DECREF`, so
Georg Brandl116aa622007-08-15 14:28:22 +0000216almost any operation is potentially dangerous.
217
218A safe approach is to always use the generic operations (functions whose name
219begins with ``PyObject_``, ``PyNumber_``, ``PySequence_`` or ``PyMapping_``).
220These operations always increment the reference count of the object they return.
Georg Brandl60203b42010-10-06 10:11:56 +0000221This leaves the caller with the responsibility to call :c:func:`Py_DECREF` when
Georg Brandl116aa622007-08-15 14:28:22 +0000222they are done with the result; this soon becomes second nature.
223
224
225.. _api-refcountdetails:
226
227Reference Count Details
228^^^^^^^^^^^^^^^^^^^^^^^
229
230The reference count behavior of functions in the Python/C API is best explained
231in terms of *ownership of references*. Ownership pertains to references, never
232to objects (objects are not owned: they are always shared). "Owning a
233reference" means being responsible for calling Py_DECREF on it when the
234reference is no longer needed. Ownership can also be transferred, meaning that
235the code that receives ownership of the reference then becomes responsible for
Georg Brandl60203b42010-10-06 10:11:56 +0000236eventually decref'ing it by calling :c:func:`Py_DECREF` or :c:func:`Py_XDECREF`
Georg Brandl116aa622007-08-15 14:28:22 +0000237when it's no longer needed---or passing on this responsibility (usually to its
238caller). When a function passes ownership of a reference on to its caller, the
239caller is said to receive a *new* reference. When no ownership is transferred,
240the caller is said to *borrow* the reference. Nothing needs to be done for a
241borrowed reference.
242
Benjamin Petersonad3d5c22009-02-26 03:38:59 +0000243Conversely, when a calling function passes in a reference to an object, there
Georg Brandl116aa622007-08-15 14:28:22 +0000244are two possibilities: the function *steals* a reference to the object, or it
245does not. *Stealing a reference* means that when you pass a reference to a
246function, that function assumes that it now owns that reference, and you are not
247responsible for it any longer.
248
249.. index::
250 single: PyList_SetItem()
251 single: PyTuple_SetItem()
252
253Few functions steal references; the two notable exceptions are
Georg Brandl60203b42010-10-06 10:11:56 +0000254:c:func:`PyList_SetItem` and :c:func:`PyTuple_SetItem`, which steal a reference
Georg Brandl116aa622007-08-15 14:28:22 +0000255to the item (but not to the tuple or list into which the item is put!). These
256functions were designed to steal a reference because of a common idiom for
257populating a tuple or list with newly created objects; for example, the code to
258create the tuple ``(1, 2, "three")`` could look like this (forgetting about
259error handling for the moment; a better way to code this is shown below)::
260
261 PyObject *t;
262
263 t = PyTuple_New(3);
Georg Brandld019fe22007-12-08 18:58:51 +0000264 PyTuple_SetItem(t, 0, PyLong_FromLong(1L));
265 PyTuple_SetItem(t, 1, PyLong_FromLong(2L));
Gregory P. Smith4b52ae82013-03-22 13:43:30 -0700266 PyTuple_SetItem(t, 2, PyUnicode_FromString("three"));
Georg Brandl116aa622007-08-15 14:28:22 +0000267
Georg Brandl60203b42010-10-06 10:11:56 +0000268Here, :c:func:`PyLong_FromLong` returns a new reference which is immediately
269stolen by :c:func:`PyTuple_SetItem`. When you want to keep using an object
270although the reference to it will be stolen, use :c:func:`Py_INCREF` to grab
Georg Brandl116aa622007-08-15 14:28:22 +0000271another reference before calling the reference-stealing function.
272
Georg Brandl60203b42010-10-06 10:11:56 +0000273Incidentally, :c:func:`PyTuple_SetItem` is the *only* way to set tuple items;
274:c:func:`PySequence_SetItem` and :c:func:`PyObject_SetItem` refuse to do this
Georg Brandl116aa622007-08-15 14:28:22 +0000275since tuples are an immutable data type. You should only use
Georg Brandl60203b42010-10-06 10:11:56 +0000276:c:func:`PyTuple_SetItem` for tuples that you are creating yourself.
Georg Brandl116aa622007-08-15 14:28:22 +0000277
Georg Brandl60203b42010-10-06 10:11:56 +0000278Equivalent code for populating a list can be written using :c:func:`PyList_New`
279and :c:func:`PyList_SetItem`.
Georg Brandl116aa622007-08-15 14:28:22 +0000280
281However, in practice, you will rarely use these ways of creating and populating
Georg Brandl60203b42010-10-06 10:11:56 +0000282a tuple or list. There's a generic function, :c:func:`Py_BuildValue`, that can
Georg Brandl116aa622007-08-15 14:28:22 +0000283create most common objects from C values, directed by a :dfn:`format string`.
284For example, the above two blocks of code could be replaced by the following
285(which also takes care of the error checking)::
286
287 PyObject *tuple, *list;
288
289 tuple = Py_BuildValue("(iis)", 1, 2, "three");
290 list = Py_BuildValue("[iis]", 1, 2, "three");
291
Georg Brandl60203b42010-10-06 10:11:56 +0000292It is much more common to use :c:func:`PyObject_SetItem` and friends with items
Georg Brandl116aa622007-08-15 14:28:22 +0000293whose references you are only borrowing, like arguments that were passed in to
294the function you are writing. In that case, their behaviour regarding reference
295counts is much saner, since you don't have to increment a reference count so you
296can give a reference away ("have it be stolen"). For example, this function
297sets all items of a list (actually, any mutable sequence) to a given item::
298
299 int
300 set_all(PyObject *target, PyObject *item)
301 {
Antoine Pitrou04707c02012-01-27 14:07:29 +0100302 Py_ssize_t i, n;
Georg Brandl116aa622007-08-15 14:28:22 +0000303
304 n = PyObject_Length(target);
305 if (n < 0)
306 return -1;
307 for (i = 0; i < n; i++) {
Antoine Pitrou04707c02012-01-27 14:07:29 +0100308 PyObject *index = PyLong_FromSsize_t(i);
Georg Brandl116aa622007-08-15 14:28:22 +0000309 if (!index)
310 return -1;
Antoine Pitrou04707c02012-01-27 14:07:29 +0100311 if (PyObject_SetItem(target, index, item) < 0) {
312 Py_DECREF(index);
Georg Brandl116aa622007-08-15 14:28:22 +0000313 return -1;
Antoine Pitrou04707c02012-01-27 14:07:29 +0100314 }
Georg Brandl116aa622007-08-15 14:28:22 +0000315 Py_DECREF(index);
316 }
317 return 0;
318 }
319
320.. index:: single: set_all()
321
322The situation is slightly different for function return values. While passing
323a reference to most functions does not change your ownership responsibilities
324for that reference, many functions that return a reference to an object give
325you ownership of the reference. The reason is simple: in many cases, the
326returned object is created on the fly, and the reference you get is the only
327reference to the object. Therefore, the generic functions that return object
Georg Brandl60203b42010-10-06 10:11:56 +0000328references, like :c:func:`PyObject_GetItem` and :c:func:`PySequence_GetItem`,
Georg Brandl116aa622007-08-15 14:28:22 +0000329always return a new reference (the caller becomes the owner of the reference).
330
331It is important to realize that whether you own a reference returned by a
332function depends on which function you call only --- *the plumage* (the type of
333the object passed as an argument to the function) *doesn't enter into it!*
Georg Brandl60203b42010-10-06 10:11:56 +0000334Thus, if you extract an item from a list using :c:func:`PyList_GetItem`, you
Georg Brandl116aa622007-08-15 14:28:22 +0000335don't own the reference --- but if you obtain the same item from the same list
Georg Brandl60203b42010-10-06 10:11:56 +0000336using :c:func:`PySequence_GetItem` (which happens to take exactly the same
Georg Brandl116aa622007-08-15 14:28:22 +0000337arguments), you do own a reference to the returned object.
338
339.. index::
340 single: PyList_GetItem()
341 single: PySequence_GetItem()
342
343Here is an example of how you could write a function that computes the sum of
Georg Brandl60203b42010-10-06 10:11:56 +0000344the items in a list of integers; once using :c:func:`PyList_GetItem`, and once
345using :c:func:`PySequence_GetItem`. ::
Georg Brandl116aa622007-08-15 14:28:22 +0000346
347 long
348 sum_list(PyObject *list)
349 {
Antoine Pitrou04707c02012-01-27 14:07:29 +0100350 Py_ssize_t i, n;
351 long total = 0, value;
Georg Brandl116aa622007-08-15 14:28:22 +0000352 PyObject *item;
353
354 n = PyList_Size(list);
355 if (n < 0)
356 return -1; /* Not a list */
357 for (i = 0; i < n; i++) {
358 item = PyList_GetItem(list, i); /* Can't fail */
Georg Brandld019fe22007-12-08 18:58:51 +0000359 if (!PyLong_Check(item)) continue; /* Skip non-integers */
Antoine Pitrou04707c02012-01-27 14:07:29 +0100360 value = PyLong_AsLong(item);
361 if (value == -1 && PyErr_Occurred())
362 /* Integer too big to fit in a C long, bail out */
363 return -1;
364 total += value;
Georg Brandl116aa622007-08-15 14:28:22 +0000365 }
366 return total;
367 }
368
369.. index:: single: sum_list()
370
371::
372
373 long
374 sum_sequence(PyObject *sequence)
375 {
Antoine Pitrou04707c02012-01-27 14:07:29 +0100376 Py_ssize_t i, n;
377 long total = 0, value;
Georg Brandl116aa622007-08-15 14:28:22 +0000378 PyObject *item;
379 n = PySequence_Length(sequence);
380 if (n < 0)
381 return -1; /* Has no length */
382 for (i = 0; i < n; i++) {
383 item = PySequence_GetItem(sequence, i);
384 if (item == NULL)
385 return -1; /* Not a sequence, or other failure */
Antoine Pitrou04707c02012-01-27 14:07:29 +0100386 if (PyLong_Check(item)) {
387 value = PyLong_AsLong(item);
388 Py_DECREF(item);
389 if (value == -1 && PyErr_Occurred())
390 /* Integer too big to fit in a C long, bail out */
391 return -1;
392 total += value;
393 }
394 else {
395 Py_DECREF(item); /* Discard reference ownership */
396 }
Georg Brandl116aa622007-08-15 14:28:22 +0000397 }
398 return total;
399 }
400
401.. index:: single: sum_sequence()
402
403
404.. _api-types:
405
406Types
407-----
408
409There are few other data types that play a significant role in the Python/C
Georg Brandl60203b42010-10-06 10:11:56 +0000410API; most are simple C types such as :c:type:`int`, :c:type:`long`,
411:c:type:`double` and :c:type:`char\*`. A few structure types are used to
Georg Brandl116aa622007-08-15 14:28:22 +0000412describe static tables used to list the functions exported by a module or the
413data attributes of a new object type, and another is used to describe the value
414of a complex number. These will be discussed together with the functions that
415use them.
416
417
418.. _api-exceptions:
419
420Exceptions
421==========
422
423The Python programmer only needs to deal with exceptions if specific error
424handling is required; unhandled exceptions are automatically propagated to the
425caller, then to the caller's caller, and so on, until they reach the top-level
426interpreter, where they are reported to the user accompanied by a stack
427traceback.
428
429.. index:: single: PyErr_Occurred()
430
Georg Brandldd909db2010-10-17 06:32:59 +0000431For C programmers, however, error checking always has to be explicit. All
432functions in the Python/C API can raise exceptions, unless an explicit claim is
433made otherwise in a function's documentation. In general, when a function
434encounters an error, it sets an exception, discards any object references that
435it owns, and returns an error indicator. If not documented otherwise, this
436indicator is either *NULL* or ``-1``, depending on the function's return type.
437A few functions return a Boolean true/false result, with false indicating an
438error. Very few functions return no explicit error indicator or have an
439ambiguous return value, and require explicit testing for errors with
440:c:func:`PyErr_Occurred`. These exceptions are always explicitly documented.
Georg Brandl116aa622007-08-15 14:28:22 +0000441
442.. index::
443 single: PyErr_SetString()
444 single: PyErr_Clear()
445
446Exception state is maintained in per-thread storage (this is equivalent to
447using global storage in an unthreaded application). A thread can be in one of
448two states: an exception has occurred, or not. The function
Georg Brandl60203b42010-10-06 10:11:56 +0000449:c:func:`PyErr_Occurred` can be used to check for this: it returns a borrowed
Georg Brandl116aa622007-08-15 14:28:22 +0000450reference to the exception type object when an exception has occurred, and
451*NULL* otherwise. There are a number of functions to set the exception state:
Georg Brandl60203b42010-10-06 10:11:56 +0000452:c:func:`PyErr_SetString` is the most common (though not the most general)
453function to set the exception state, and :c:func:`PyErr_Clear` clears the
Georg Brandl116aa622007-08-15 14:28:22 +0000454exception state.
455
456The full exception state consists of three objects (all of which can be
457*NULL*): the exception type, the corresponding exception value, and the
458traceback. These have the same meanings as the Python result of
459``sys.exc_info()``; however, they are not the same: the Python objects represent
460the last exception being handled by a Python :keyword:`try` ...
461:keyword:`except` statement, while the C level exception state only exists while
462an exception is being passed on between C functions until it reaches the Python
463bytecode interpreter's main loop, which takes care of transferring it to
464``sys.exc_info()`` and friends.
465
466.. index:: single: exc_info() (in module sys)
467
468Note that starting with Python 1.5, the preferred, thread-safe way to access the
469exception state from Python code is to call the function :func:`sys.exc_info`,
470which returns the per-thread exception state for Python code. Also, the
471semantics of both ways to access the exception state have changed so that a
472function which catches an exception will save and restore its thread's exception
473state so as to preserve the exception state of its caller. This prevents common
474bugs in exception handling code caused by an innocent-looking function
475overwriting the exception being handled; it also reduces the often unwanted
476lifetime extension for objects that are referenced by the stack frames in the
477traceback.
478
479As a general principle, a function that calls another function to perform some
480task should check whether the called function raised an exception, and if so,
481pass the exception state on to its caller. It should discard any object
482references that it owns, and return an error indicator, but it should *not* set
483another exception --- that would overwrite the exception that was just raised,
484and lose important information about the exact cause of the error.
485
486.. index:: single: sum_sequence()
487
488A simple example of detecting exceptions and passing them on is shown in the
Terry Jan Reedy65e69b32013-03-11 17:23:46 -0400489:c:func:`sum_sequence` example above. It so happens that this example doesn't
Georg Brandl116aa622007-08-15 14:28:22 +0000490need to clean up any owned references when it detects an error. The following
491example function shows some error cleanup. First, to remind you why you like
492Python, we show the equivalent Python code::
493
494 def incr_item(dict, key):
495 try:
496 item = dict[key]
497 except KeyError:
498 item = 0
499 dict[key] = item + 1
500
501.. index:: single: incr_item()
502
503Here is the corresponding C code, in all its glory::
504
505 int
506 incr_item(PyObject *dict, PyObject *key)
507 {
508 /* Objects all initialized to NULL for Py_XDECREF */
509 PyObject *item = NULL, *const_one = NULL, *incremented_item = NULL;
510 int rv = -1; /* Return value initialized to -1 (failure) */
511
512 item = PyObject_GetItem(dict, key);
513 if (item == NULL) {
514 /* Handle KeyError only: */
515 if (!PyErr_ExceptionMatches(PyExc_KeyError))
516 goto error;
517
518 /* Clear the error and use zero: */
519 PyErr_Clear();
Georg Brandld019fe22007-12-08 18:58:51 +0000520 item = PyLong_FromLong(0L);
Georg Brandl116aa622007-08-15 14:28:22 +0000521 if (item == NULL)
522 goto error;
523 }
Georg Brandld019fe22007-12-08 18:58:51 +0000524 const_one = PyLong_FromLong(1L);
Georg Brandl116aa622007-08-15 14:28:22 +0000525 if (const_one == NULL)
526 goto error;
527
528 incremented_item = PyNumber_Add(item, const_one);
529 if (incremented_item == NULL)
530 goto error;
531
532 if (PyObject_SetItem(dict, key, incremented_item) < 0)
533 goto error;
534 rv = 0; /* Success */
535 /* Continue with cleanup code */
536
537 error:
538 /* Cleanup code, shared by success and failure path */
539
540 /* Use Py_XDECREF() to ignore NULL references */
541 Py_XDECREF(item);
542 Py_XDECREF(const_one);
543 Py_XDECREF(incremented_item);
544
545 return rv; /* -1 for error, 0 for success */
546 }
547
548.. index:: single: incr_item()
549
550.. index::
551 single: PyErr_ExceptionMatches()
552 single: PyErr_Clear()
553 single: Py_XDECREF()
554
Christian Heimes5b5e81c2007-12-31 16:14:33 +0000555This example represents an endorsed use of the ``goto`` statement in C!
Georg Brandl60203b42010-10-06 10:11:56 +0000556It illustrates the use of :c:func:`PyErr_ExceptionMatches` and
557:c:func:`PyErr_Clear` to handle specific exceptions, and the use of
558:c:func:`Py_XDECREF` to dispose of owned references that may be *NULL* (note the
559``'X'`` in the name; :c:func:`Py_DECREF` would crash when confronted with a
Georg Brandl116aa622007-08-15 14:28:22 +0000560*NULL* reference). It is important that the variables used to hold owned
561references are initialized to *NULL* for this to work; likewise, the proposed
562return value is initialized to ``-1`` (failure) and only set to success after
563the final call made is successful.
564
565
566.. _api-embedding:
567
568Embedding Python
569================
570
571The one important task that only embedders (as opposed to extension writers) of
572the Python interpreter have to worry about is the initialization, and possibly
573the finalization, of the Python interpreter. Most functionality of the
574interpreter can only be used after the interpreter has been initialized.
575
576.. index::
577 single: Py_Initialize()
Georg Brandl1a3284e2007-12-02 09:40:06 +0000578 module: builtins
Georg Brandl116aa622007-08-15 14:28:22 +0000579 module: __main__
580 module: sys
Georg Brandl116aa622007-08-15 14:28:22 +0000581 triple: module; search; path
582 single: path (in module sys)
583
Georg Brandl60203b42010-10-06 10:11:56 +0000584The basic initialization function is :c:func:`Py_Initialize`. This initializes
Georg Brandl116aa622007-08-15 14:28:22 +0000585the table of loaded modules, and creates the fundamental modules
Éric Araujo8b8f2ec2011-03-26 07:22:01 +0100586:mod:`builtins`, :mod:`__main__`, and :mod:`sys`. It also
Georg Brandl116aa622007-08-15 14:28:22 +0000587initializes the module search path (``sys.path``).
588
Benjamin Peterson2ebf8ce2010-06-27 21:48:35 +0000589.. index:: single: PySys_SetArgvEx()
Georg Brandl116aa622007-08-15 14:28:22 +0000590
Georg Brandl60203b42010-10-06 10:11:56 +0000591:c:func:`Py_Initialize` does not set the "script argument list" (``sys.argv``).
Benjamin Peterson2ebf8ce2010-06-27 21:48:35 +0000592If this variable is needed by Python code that will be executed later, it must
593be set explicitly with a call to ``PySys_SetArgvEx(argc, argv, updatepath)``
Georg Brandl60203b42010-10-06 10:11:56 +0000594after the call to :c:func:`Py_Initialize`.
Georg Brandl116aa622007-08-15 14:28:22 +0000595
596On most systems (in particular, on Unix and Windows, although the details are
Georg Brandl60203b42010-10-06 10:11:56 +0000597slightly different), :c:func:`Py_Initialize` calculates the module search path
Georg Brandl116aa622007-08-15 14:28:22 +0000598based upon its best guess for the location of the standard Python interpreter
599executable, assuming that the Python library is found in a fixed location
600relative to the Python interpreter executable. In particular, it looks for a
601directory named :file:`lib/python{X.Y}` relative to the parent directory
602where the executable named :file:`python` is found on the shell command search
603path (the environment variable :envvar:`PATH`).
604
605For instance, if the Python executable is found in
606:file:`/usr/local/bin/python`, it will assume that the libraries are in
607:file:`/usr/local/lib/python{X.Y}`. (In fact, this particular path is also
608the "fallback" location, used when no executable file named :file:`python` is
609found along :envvar:`PATH`.) The user can override this behavior by setting the
610environment variable :envvar:`PYTHONHOME`, or insert additional directories in
611front of the standard path by setting :envvar:`PYTHONPATH`.
612
613.. index::
614 single: Py_SetProgramName()
615 single: Py_GetPath()
616 single: Py_GetPrefix()
617 single: Py_GetExecPrefix()
618 single: Py_GetProgramFullPath()
619
620The embedding application can steer the search by calling
Georg Brandl60203b42010-10-06 10:11:56 +0000621``Py_SetProgramName(file)`` *before* calling :c:func:`Py_Initialize`. Note that
Georg Brandl116aa622007-08-15 14:28:22 +0000622:envvar:`PYTHONHOME` still overrides this and :envvar:`PYTHONPATH` is still
623inserted in front of the standard path. An application that requires total
Georg Brandl60203b42010-10-06 10:11:56 +0000624control has to provide its own implementation of :c:func:`Py_GetPath`,
625:c:func:`Py_GetPrefix`, :c:func:`Py_GetExecPrefix`, and
626:c:func:`Py_GetProgramFullPath` (all defined in :file:`Modules/getpath.c`).
Georg Brandl116aa622007-08-15 14:28:22 +0000627
628.. index:: single: Py_IsInitialized()
629
630Sometimes, it is desirable to "uninitialize" Python. For instance, the
631application may want to start over (make another call to
Georg Brandl60203b42010-10-06 10:11:56 +0000632:c:func:`Py_Initialize`) or the application is simply done with its use of
Georg Brandl116aa622007-08-15 14:28:22 +0000633Python and wants to free memory allocated by Python. This can be accomplished
Martin Panterb4ce1fc2015-11-30 03:18:29 +0000634by calling :c:func:`Py_FinalizeEx`. The function :c:func:`Py_IsInitialized` returns
Georg Brandl116aa622007-08-15 14:28:22 +0000635true if Python is currently in the initialized state. More information about
Martin Panterb4ce1fc2015-11-30 03:18:29 +0000636these functions is given in a later chapter. Notice that :c:func:`Py_FinalizeEx`
Georg Brandl116aa622007-08-15 14:28:22 +0000637does *not* free all memory allocated by the Python interpreter, e.g. memory
638allocated by extension modules currently cannot be released.
639
640
641.. _api-debugging:
642
643Debugging Builds
644================
645
646Python can be built with several macros to enable extra checks of the
647interpreter and extension modules. These checks tend to add a large amount of
648overhead to the runtime so they are not enabled by default.
649
650A full list of the various types of debugging builds is in the file
651:file:`Misc/SpecialBuilds.txt` in the Python source distribution. Builds are
652available that support tracing of reference counts, debugging the memory
653allocator, or low-level profiling of the main interpreter loop. Only the most
654frequently-used builds will be described in the remainder of this section.
655
Georg Brandl60203b42010-10-06 10:11:56 +0000656Compiling the interpreter with the :c:macro:`Py_DEBUG` macro defined produces
657what is generally meant by "a debug build" of Python. :c:macro:`Py_DEBUG` is
Éric Araujod2f8cec2011-06-08 05:29:39 +0200658enabled in the Unix build by adding ``--with-pydebug`` to the
659:file:`./configure` command. It is also implied by the presence of the
Georg Brandl60203b42010-10-06 10:11:56 +0000660not-Python-specific :c:macro:`_DEBUG` macro. When :c:macro:`Py_DEBUG` is enabled
Georg Brandl116aa622007-08-15 14:28:22 +0000661in the Unix build, compiler optimization is disabled.
662
663In addition to the reference count debugging described below, the following
664extra checks are performed:
665
666* Extra checks are added to the object allocator.
667
668* Extra checks are added to the parser and compiler.
669
670* Downcasts from wide types to narrow types are checked for loss of information.
671
672* A number of assertions are added to the dictionary and set implementations.
673 In addition, the set object acquires a :meth:`test_c_api` method.
674
675* Sanity checks of the input arguments are added to frame creation.
676
Mark Dickinsonbf5c6a92009-01-17 10:21:23 +0000677* The storage for ints is initialized with a known invalid pattern to catch
Georg Brandl116aa622007-08-15 14:28:22 +0000678 reference to uninitialized digits.
679
680* Low-level tracing and extra exception checking are added to the runtime
681 virtual machine.
682
683* Extra checks are added to the memory arena implementation.
684
685* Extra debugging is added to the thread module.
686
687There may be additional checks not mentioned here.
688
Georg Brandl60203b42010-10-06 10:11:56 +0000689Defining :c:macro:`Py_TRACE_REFS` enables reference tracing. When defined, a
Georg Brandl116aa622007-08-15 14:28:22 +0000690circular doubly linked list of active objects is maintained by adding two extra
Georg Brandl60203b42010-10-06 10:11:56 +0000691fields to every :c:type:`PyObject`. Total allocations are tracked as well. Upon
Georg Brandl116aa622007-08-15 14:28:22 +0000692exit, all existing references are printed. (In interactive mode this happens
Georg Brandl60203b42010-10-06 10:11:56 +0000693after every statement run by the interpreter.) Implied by :c:macro:`Py_DEBUG`.
Georg Brandl116aa622007-08-15 14:28:22 +0000694
695Please refer to :file:`Misc/SpecialBuilds.txt` in the Python source distribution
696for more detailed information.
697