blob: e03732218b8d91c3f4080aef1b97dedbee0387c1 [file] [log] [blame]
Tim Peters8b078f92002-04-28 04:11:46 +00001/* The PyObject_ memory family: high-level object memory interfaces.
2 See pymem.h for the low-level PyMem_ family.
3*/
Guido van Rossumf70e43a1991-02-19 12:39:46 +00004
Fred Drake3cf4d2b2000-07-09 00:55:06 +00005#ifndef Py_OBJIMPL_H
6#define Py_OBJIMPL_H
Peter Schneider-Kamp25f68942000-07-31 22:19:30 +00007
8#include "pymem.h"
9
Fred Drake3cf4d2b2000-07-09 00:55:06 +000010#ifdef __cplusplus
11extern "C" {
12#endif
13
Tim Peters8b078f92002-04-28 04:11:46 +000014/* BEWARE:
15
16 Each interface exports both functions and macros. Extension modules should
17 use the functions, to ensure binary compatibility across Python versions.
18 Because the Python implementation is free to change internal details, and
19 the macros may (or may not) expose details for speed, if you do use the
20 macros you must recompile your extensions with each Python release.
21
22 Never mix calls to PyObject_ memory functions with calls to the platform
23 malloc/realloc/ calloc/free, or with calls to PyMem_.
24*/
25
Guido van Rossum85a5fbb1990-10-14 12:07:46 +000026/*
Guido van Rossumb18618d2000-05-03 23:44:39 +000027Functions and macros for modules that implement new object types.
Guido van Rossum85a5fbb1990-10-14 12:07:46 +000028
Tim Peters8b078f92002-04-28 04:11:46 +000029 - PyObject_New(type, typeobj) allocates memory for a new object of the given
30 type, and initializes part of it. 'type' must be the C structure type used
31 to represent the object, and 'typeobj' the address of the corresponding
32 type object. Reference count and type pointer are filled in; the rest of
33 the bytes of the object are *undefined*! The resulting expression type is
34 'type *'. The size of the object is determined by the tp_basicsize field
35 of the type object.
Guido van Rossum85a5fbb1990-10-14 12:07:46 +000036
Tim Peters8b078f92002-04-28 04:11:46 +000037 - PyObject_NewVar(type, typeobj, n) is similar but allocates a variable-size
38 object with room for n items. In addition to the refcount and type pointer
39 fields, this also fills in the ob_size field.
Guido van Rossum85a5fbb1990-10-14 12:07:46 +000040
Tim Peters8b078f92002-04-28 04:11:46 +000041 - PyObject_Del(op) releases the memory allocated for an object. It does not
42 run a destructor -- it only frees the memory. PyObject_Free is identical.
Guido van Rossumb18618d2000-05-03 23:44:39 +000043
Tim Peters8b078f92002-04-28 04:11:46 +000044 - PyObject_Init(op, typeobj) and PyObject_InitVar(op, typeobj, n) don't
45 allocate memory. Instead of a 'type' parameter, they take a pointer to a
46 new object (allocated by an arbitrary allocator), and initialize its object
47 header fields.
Guido van Rossumb18618d2000-05-03 23:44:39 +000048
Tim Peters8b078f92002-04-28 04:11:46 +000049Note that objects created with PyObject_{New, NewVar} are allocated using the
50specialized Python allocator (implemented in obmalloc.c), if WITH_PYMALLOC is
51enabled. In addition, a special debugging allocator is used if PYMALLOC_DEBUG
52is also #defined.
Guido van Rossumb18618d2000-05-03 23:44:39 +000053
Tim Peters8b078f92002-04-28 04:11:46 +000054In case a specific form of memory management is needed (for example, if you
55must use the platform malloc heap(s), or shared memory, or C++ local storage or
56operator new), you must first allocate the object with your custom allocator,
57then pass its pointer to PyObject_{Init, InitVar} for filling in its Python-
58specific fields: reference count, type pointer, possibly others. You should
59be aware that Python no control over these objects because they don't
60cooperate with the Python memory manager. Such objects may not be eligible
61for automatic garbage collection and you have to make sure that they are
62released accordingly whenever their destructor gets called (cf. the specific
Guido van Rossumb18618d2000-05-03 23:44:39 +000063form of memory management you're using).
64
Tim Peters8b078f92002-04-28 04:11:46 +000065Unless you have specific memory management requirements, use
66PyObject_{New, NewVar, Del}.
67*/
Guido van Rossumb18618d2000-05-03 23:44:39 +000068
Tim Peters6d483d32001-10-06 21:27:34 +000069/*
Guido van Rossumb18618d2000-05-03 23:44:39 +000070 * Raw object memory interface
71 * ===========================
72 */
73
Tim Peterse9e74522002-04-12 05:21:34 +000074/* Functions to call the same malloc/realloc/free as used by Python's
75 object allocator. If WITH_PYMALLOC is enabled, these may differ from
76 the platform malloc/realloc/free. The Python object allocator is
77 designed for fast, cache-conscious allocation of many "small" objects,
Tim Peters8b078f92002-04-28 04:11:46 +000078 and with low hidden memory overhead.
79
80 PyObject_Malloc(0) returns a unique non-NULL pointer if possible.
81
82 PyObject_Realloc(NULL, n) acts like PyObject_Malloc(n).
83 PyObject_Realloc(p != NULL, 0) does not return NULL, or free the memory
84 at p.
85
86 Returned pointers must be checked for NULL explicitly; no action is
87 performed on failure other than to return NULL (no warning it printed, no
88 exception is set, etc).
89
90 For allocating objects, use PyObject_{New, NewVar} instead whenever
91 possible. The PyObject_{Malloc, Realloc, Free} family is exposed
92 so that you can exploit Python's small-block allocator for non-object
93 uses. If you must use these routines to allocate object memory, make sure
94 the object gets initialized via PyObject_{Init, InitVar} after obtaining
95 the raw memory.
96*/
Thomas Wouters334fb892000-07-25 12:56:38 +000097extern DL_IMPORT(void *) PyObject_Malloc(size_t);
98extern DL_IMPORT(void *) PyObject_Realloc(void *, size_t);
99extern DL_IMPORT(void) PyObject_Free(void *);
Guido van Rossumb18618d2000-05-03 23:44:39 +0000100
Neil Schemenauer3e7b8932002-04-12 02:38:45 +0000101
Guido van Rossumb18618d2000-05-03 23:44:39 +0000102/* Macros */
Neil Schemenauer3e7b8932002-04-12 02:38:45 +0000103#ifdef WITH_PYMALLOC
104#ifdef PYMALLOC_DEBUG
105DL_IMPORT(void *) _PyObject_DebugMalloc(size_t nbytes);
106DL_IMPORT(void *) _PyObject_DebugRealloc(void *p, size_t nbytes);
107DL_IMPORT(void) _PyObject_DebugFree(void *p);
108DL_IMPORT(void) _PyObject_DebugDumpAddress(const void *p);
109DL_IMPORT(void) _PyObject_DebugCheckAddress(const void *p);
Tim Peters0e871182002-04-13 08:29:14 +0000110DL_IMPORT(void) _PyObject_DebugMallocStats(void);
Tim Peterse9e74522002-04-12 05:21:34 +0000111#define PyObject_MALLOC _PyObject_DebugMalloc
112#define PyObject_Malloc _PyObject_DebugMalloc
113#define PyObject_REALLOC _PyObject_DebugRealloc
114#define PyObject_Realloc _PyObject_DebugRealloc
115#define PyObject_FREE _PyObject_DebugFree
116#define PyObject_Free _PyObject_DebugFree
Neil Schemenauer3e7b8932002-04-12 02:38:45 +0000117
118#else /* WITH_PYMALLOC && ! PYMALLOC_DEBUG */
119#define PyObject_MALLOC PyObject_Malloc
120#define PyObject_REALLOC PyObject_Realloc
121#define PyObject_FREE PyObject_Free
122#endif
123
124#else /* ! WITH_PYMALLOC */
125#define PyObject_MALLOC PyMem_MALLOC
126#define PyObject_REALLOC PyMem_REALLOC
Tim Peters8b078f92002-04-28 04:11:46 +0000127/* This is an odd one! For backward compatability with old extensions, the
128 PyMem "release memory" functions have to invoke the object allocator's
129 free() function. When pymalloc isn't enabled, that leaves us using
130 the platform free(). */
131#define PyObject_FREE free
132
Neil Schemenauer3e7b8932002-04-12 02:38:45 +0000133#endif /* WITH_PYMALLOC */
134
Tim Peters8b078f92002-04-28 04:11:46 +0000135#define PyObject_Del PyObject_Free
136#define PyObject_DEL PyObject_FREE
Neil Schemenauer3e7b8932002-04-12 02:38:45 +0000137
138/* for source compatibility with 2.2 */
Tim Peters8b078f92002-04-28 04:11:46 +0000139#define _PyObject_Del PyObject_Free
Guido van Rossumb18618d2000-05-03 23:44:39 +0000140
141/*
142 * Generic object allocator interface
143 * ==================================
144 */
145
146/* Functions */
Fred Drake3cf4d2b2000-07-09 00:55:06 +0000147extern DL_IMPORT(PyObject *) PyObject_Init(PyObject *, PyTypeObject *);
148extern DL_IMPORT(PyVarObject *) PyObject_InitVar(PyVarObject *,
149 PyTypeObject *, int);
150extern DL_IMPORT(PyObject *) _PyObject_New(PyTypeObject *);
151extern DL_IMPORT(PyVarObject *) _PyObject_NewVar(PyTypeObject *, int);
Guido van Rossum85a5fbb1990-10-14 12:07:46 +0000152
Guido van Rossumb18618d2000-05-03 23:44:39 +0000153#define PyObject_New(type, typeobj) \
154 ( (type *) _PyObject_New(typeobj) )
155#define PyObject_NewVar(type, typeobj, n) \
156 ( (type *) _PyObject_NewVar((typeobj), (n)) )
Guido van Rossuma3309961993-07-28 09:05:47 +0000157
Andrew M. Kuchling1582a3a2000-08-16 12:27:23 +0000158/* Macros trading binary compatibility for speed. See also pymem.h.
Guido van Rossumb18618d2000-05-03 23:44:39 +0000159 Note that these macros expect non-NULL object pointers.*/
160#define PyObject_INIT(op, typeobj) \
Fred Drake4e262a92001-03-22 18:26:47 +0000161 ( (op)->ob_type = (typeobj), _Py_NewReference((PyObject *)(op)), (op) )
Guido van Rossumb18618d2000-05-03 23:44:39 +0000162#define PyObject_INIT_VAR(op, typeobj, size) \
163 ( (op)->ob_size = (size), PyObject_INIT((op), (typeobj)) )
Guido van Rossum5a849141996-07-21 02:23:54 +0000164
Guido van Rossumb18618d2000-05-03 23:44:39 +0000165#define _PyObject_SIZE(typeobj) ( (typeobj)->tp_basicsize )
Tim Peters6d483d32001-10-06 21:27:34 +0000166
Tim Petersf2a67da2001-10-07 03:54:51 +0000167/* _PyObject_VAR_SIZE returns the number of bytes (as size_t) allocated for a
168 vrbl-size object with nitems items, exclusive of gc overhead (if any). The
169 value is rounded up to the closest multiple of sizeof(void *), in order to
170 ensure that pointer fields at the end of the object are correctly aligned
171 for the platform (this is of special importance for subclasses of, e.g.,
172 str or long, so that pointers can be stored after the embedded data).
Tim Peters6d483d32001-10-06 21:27:34 +0000173
Tim Petersf2a67da2001-10-07 03:54:51 +0000174 Note that there's no memory wastage in doing this, as malloc has to
175 return (at worst) pointer-aligned memory anyway.
Tim Peters6d483d32001-10-06 21:27:34 +0000176*/
Tim Petersf2a67da2001-10-07 03:54:51 +0000177#if ((SIZEOF_VOID_P - 1) & SIZEOF_VOID_P) != 0
178# error "_PyObject_VAR_SIZE requires SIZEOF_VOID_P be a power of 2"
179#endif
180
181#define _PyObject_VAR_SIZE(typeobj, nitems) \
182 (size_t) \
183 ( ( (typeobj)->tp_basicsize + \
184 (nitems)*(typeobj)->tp_itemsize + \
185 (SIZEOF_VOID_P - 1) \
186 ) & ~(SIZEOF_VOID_P - 1) \
187 )
Guido van Rossum5a849141996-07-21 02:23:54 +0000188
Guido van Rossumb18618d2000-05-03 23:44:39 +0000189#define PyObject_NEW(type, typeobj) \
190( (type *) PyObject_Init( \
191 (PyObject *) PyObject_MALLOC( _PyObject_SIZE(typeobj) ), (typeobj)) )
Tim Peters6d483d32001-10-06 21:27:34 +0000192
Tim Petersf2a67da2001-10-07 03:54:51 +0000193#define PyObject_NEW_VAR(type, typeobj, n) \
194( (type *) PyObject_InitVar( \
195 (PyVarObject *) PyObject_MALLOC(_PyObject_VAR_SIZE((typeobj),(n)) ),\
196 (typeobj), (n)) )
Guido van Rossumb18618d2000-05-03 23:44:39 +0000197
198/* This example code implements an object constructor with a custom
199 allocator, where PyObject_New is inlined, and shows the important
200 distinction between two steps (at least):
201 1) the actual allocation of the object storage;
202 2) the initialization of the Python specific fields
203 in this storage with PyObject_{Init, InitVar}.
204
205 PyObject *
206 YourObject_New(...)
207 {
208 PyObject *op;
209
210 op = (PyObject *) Your_Allocator(_PyObject_SIZE(YourTypeStruct));
211 if (op == NULL)
212 return PyErr_NoMemory();
213
Tim Peters8b078f92002-04-28 04:11:46 +0000214 PyObject_Init(op, &YourTypeStruct);
Guido van Rossumb18618d2000-05-03 23:44:39 +0000215
216 op->ob_field = value;
217 ...
218 return op;
219 }
220
221 Note that in C++, the use of the new operator usually implies that
222 the 1st step is performed automatically for you, so in a C++ class
Tim Peters8b078f92002-04-28 04:11:46 +0000223 constructor you would start directly with PyObject_Init/InitVar
224*/
Guido van Rossum5a849141996-07-21 02:23:54 +0000225
Jeremy Hyltonc5007aa2000-06-30 05:02:53 +0000226/*
227 * Garbage Collection Support
228 * ==========================
Guido van Rossum048eb752001-10-02 21:24:57 +0000229 *
230 * Some of the functions and macros below are always defined; when
231 * WITH_CYCLE_GC is undefined, they simply don't do anything different
232 * than their non-GC counterparts.
Jeremy Hyltonc5007aa2000-06-30 05:02:53 +0000233 */
Jeremy Hyltond08b4c42000-06-23 19:37:02 +0000234
Neil Schemenauer74b5ade2001-08-29 23:49:28 +0000235/* Test if a type has a GC head */
236#define PyType_IS_GC(t) PyType_HasFeature((t), Py_TPFLAGS_HAVE_GC)
Jeremy Hyltond08b4c42000-06-23 19:37:02 +0000237
Neil Schemenauer74b5ade2001-08-29 23:49:28 +0000238/* Test if an object has a GC head */
Guido van Rossum048eb752001-10-02 21:24:57 +0000239#define PyObject_IS_GC(o) (PyType_IS_GC((o)->ob_type) && \
240 ((o)->ob_type->tp_is_gc == NULL || (o)->ob_type->tp_is_gc(o)))
Jeremy Hyltonc5007aa2000-06-30 05:02:53 +0000241
Neil Schemenauer74b5ade2001-08-29 23:49:28 +0000242extern DL_IMPORT(PyVarObject *) _PyObject_GC_Resize(PyVarObject *, int);
Neil Schemenauer74b5ade2001-08-29 23:49:28 +0000243#define PyObject_GC_Resize(type, op, n) \
244 ( (type *) _PyObject_GC_Resize((PyVarObject *)(op), (n)) )
245
Neil Schemenauer3e7b8932002-04-12 02:38:45 +0000246/* for source compatibility with 2.2 */
247#define _PyObject_GC_Del PyObject_GC_Del
Neil Schemenauer74b5ade2001-08-29 23:49:28 +0000248
Guido van Rossum048eb752001-10-02 21:24:57 +0000249#ifdef WITH_CYCLE_GC
250
Tim Peters8b078f92002-04-28 04:11:46 +0000251/* GC information is stored BEFORE the object structure. */
Tim Peters9e4ca102001-10-11 18:31:31 +0000252typedef union _gc_head {
253 struct {
254 union _gc_head *gc_next; /* not NULL if object is tracked */
255 union _gc_head *gc_prev;
256 int gc_refs;
257 } gc;
Tim Peters5e67cde2002-02-28 19:38:51 +0000258 long double dummy; /* force worst-case alignment */
Neil Schemenauer74b5ade2001-08-29 23:49:28 +0000259} PyGC_Head;
260
Neil Schemenauerb1094f02002-05-04 05:36:06 +0000261extern PyGC_Head *_PyGC_generation0;
Neil Schemenauer74b5ade2001-08-29 23:49:28 +0000262
Neil Schemenaueref997232002-03-28 21:06:16 +0000263#define _Py_AS_GC(o) ((PyGC_Head *)(o)-1)
264
Neil Schemenauer74b5ade2001-08-29 23:49:28 +0000265/* Tell the GC to track this object. NB: While the object is tracked the
266 * collector it must be safe to call the ob_traverse method. */
267#define _PyObject_GC_TRACK(o) do { \
Neil Schemenaueref997232002-03-28 21:06:16 +0000268 PyGC_Head *g = _Py_AS_GC(o); \
Tim Peters9e4ca102001-10-11 18:31:31 +0000269 if (g->gc.gc_next != NULL) \
Neil Schemenauer74b5ade2001-08-29 23:49:28 +0000270 Py_FatalError("GC object already in linked list"); \
Neil Schemenauerb1094f02002-05-04 05:36:06 +0000271 g->gc.gc_next = _PyGC_generation0; \
272 g->gc.gc_prev = _PyGC_generation0->gc.gc_prev; \
Tim Peters9e4ca102001-10-11 18:31:31 +0000273 g->gc.gc_prev->gc.gc_next = g; \
Neil Schemenauerb1094f02002-05-04 05:36:06 +0000274 _PyGC_generation0->gc.gc_prev = g; \
Neil Schemenauer74b5ade2001-08-29 23:49:28 +0000275 } while (0);
276
277/* Tell the GC to stop tracking this object. */
278#define _PyObject_GC_UNTRACK(o) do { \
Neil Schemenaueref997232002-03-28 21:06:16 +0000279 PyGC_Head *g = _Py_AS_GC(o); \
Tim Peters9e4ca102001-10-11 18:31:31 +0000280 g->gc.gc_prev->gc.gc_next = g->gc.gc_next; \
281 g->gc.gc_next->gc.gc_prev = g->gc.gc_prev; \
282 g->gc.gc_next = NULL; \
Neil Schemenauer74b5ade2001-08-29 23:49:28 +0000283 } while (0);
284
Neil Schemenauer3e7b8932002-04-12 02:38:45 +0000285extern DL_IMPORT(PyObject *) _PyObject_GC_Malloc(size_t);
286extern DL_IMPORT(PyObject *) _PyObject_GC_New(PyTypeObject *);
287extern DL_IMPORT(PyVarObject *) _PyObject_GC_NewVar(PyTypeObject *, int);
288extern DL_IMPORT(void) PyObject_GC_Track(void *);
289extern DL_IMPORT(void) PyObject_GC_UnTrack(void *);
290extern DL_IMPORT(void) PyObject_GC_Del(void *);
Neil Schemenauer74b5ade2001-08-29 23:49:28 +0000291
292#define PyObject_GC_New(type, typeobj) \
293 ( (type *) _PyObject_GC_New(typeobj) )
294#define PyObject_GC_NewVar(type, typeobj, n) \
295 ( (type *) _PyObject_GC_NewVar((typeobj), (n)) )
Neil Schemenauer3e7b8932002-04-12 02:38:45 +0000296
Neil Schemenauer74b5ade2001-08-29 23:49:28 +0000297
298#else /* !WITH_CYCLE_GC */
299
Tim Peters8b078f92002-04-28 04:11:46 +0000300#define _PyObject_GC_Malloc PyObject_Malloc
301#define PyObject_GC_New PyObject_New
302#define PyObject_GC_NewVar PyObject_NewVar
303#define PyObject_GC_Del PyObject_Del
Neil Schemenauer49417e72001-09-03 15:44:48 +0000304#define _PyObject_GC_TRACK(op)
305#define _PyObject_GC_UNTRACK(op)
Neil Schemenauer74b5ade2001-08-29 23:49:28 +0000306#define PyObject_GC_Track(op)
307#define PyObject_GC_UnTrack(op)
308
309#endif
310
311/* This is here for the sake of backwards compatibility. Extensions that
312 * use the old GC API will still compile but the objects will not be
313 * tracked by the GC. */
Jeremy Hyltonc5007aa2000-06-30 05:02:53 +0000314#define PyGC_HEAD_SIZE 0
315#define PyObject_GC_Init(op)
316#define PyObject_GC_Fini(op)
317#define PyObject_AS_GC(op) (op)
318#define PyObject_FROM_GC(op) (op)
Tim Peters6d6c1a32001-08-02 04:15:00 +0000319
Jeremy Hyltond08b4c42000-06-23 19:37:02 +0000320
Fred Drake41deb1e2001-02-01 05:27:45 +0000321/* Test if a type supports weak references */
Fred Drake033f3122001-02-02 18:17:30 +0000322#define PyType_SUPPORTS_WEAKREFS(t) \
323 (PyType_HasFeature((t), Py_TPFLAGS_HAVE_WEAKREFS) \
324 && ((t)->tp_weaklistoffset > 0))
Fred Drake41deb1e2001-02-01 05:27:45 +0000325
326#define PyObject_GET_WEAKREFS_LISTPTR(o) \
327 ((PyObject **) (((char *) (o)) + (o)->ob_type->tp_weaklistoffset))
328
Guido van Rossuma3309961993-07-28 09:05:47 +0000329#ifdef __cplusplus
330}
331#endif
332#endif /* !Py_OBJIMPL_H */