blob: 5707e500eef13c96ae9dd1f0ee047c814fa046d3 [file] [log] [blame]
Tim Peters8b078f92002-04-28 04:11:46 +00001/* The PyObject_ memory family: high-level object memory interfaces.
2 See pymem.h for the low-level PyMem_ family.
3*/
Guido van Rossumf70e43a1991-02-19 12:39:46 +00004
Fred Drake3cf4d2b2000-07-09 00:55:06 +00005#ifndef Py_OBJIMPL_H
6#define Py_OBJIMPL_H
Peter Schneider-Kamp25f68942000-07-31 22:19:30 +00007
8#include "pymem.h"
9
Fred Drake3cf4d2b2000-07-09 00:55:06 +000010#ifdef __cplusplus
11extern "C" {
12#endif
13
Tim Peters8b078f92002-04-28 04:11:46 +000014/* BEWARE:
15
16 Each interface exports both functions and macros. Extension modules should
17 use the functions, to ensure binary compatibility across Python versions.
18 Because the Python implementation is free to change internal details, and
19 the macros may (or may not) expose details for speed, if you do use the
20 macros you must recompile your extensions with each Python release.
21
22 Never mix calls to PyObject_ memory functions with calls to the platform
23 malloc/realloc/ calloc/free, or with calls to PyMem_.
24*/
25
Guido van Rossum85a5fbb1990-10-14 12:07:46 +000026/*
Guido van Rossumb18618d2000-05-03 23:44:39 +000027Functions and macros for modules that implement new object types.
Guido van Rossum85a5fbb1990-10-14 12:07:46 +000028
Tim Peters8b078f92002-04-28 04:11:46 +000029 - PyObject_New(type, typeobj) allocates memory for a new object of the given
30 type, and initializes part of it. 'type' must be the C structure type used
31 to represent the object, and 'typeobj' the address of the corresponding
32 type object. Reference count and type pointer are filled in; the rest of
33 the bytes of the object are *undefined*! The resulting expression type is
34 'type *'. The size of the object is determined by the tp_basicsize field
35 of the type object.
Guido van Rossum85a5fbb1990-10-14 12:07:46 +000036
Tim Peters8b078f92002-04-28 04:11:46 +000037 - PyObject_NewVar(type, typeobj, n) is similar but allocates a variable-size
38 object with room for n items. In addition to the refcount and type pointer
39 fields, this also fills in the ob_size field.
Guido van Rossum85a5fbb1990-10-14 12:07:46 +000040
Tim Peters8b078f92002-04-28 04:11:46 +000041 - PyObject_Del(op) releases the memory allocated for an object. It does not
42 run a destructor -- it only frees the memory. PyObject_Free is identical.
Guido van Rossumb18618d2000-05-03 23:44:39 +000043
Tim Peters8b078f92002-04-28 04:11:46 +000044 - PyObject_Init(op, typeobj) and PyObject_InitVar(op, typeobj, n) don't
45 allocate memory. Instead of a 'type' parameter, they take a pointer to a
46 new object (allocated by an arbitrary allocator), and initialize its object
47 header fields.
Guido van Rossumb18618d2000-05-03 23:44:39 +000048
Tim Peters8b078f92002-04-28 04:11:46 +000049Note that objects created with PyObject_{New, NewVar} are allocated using the
50specialized Python allocator (implemented in obmalloc.c), if WITH_PYMALLOC is
51enabled. In addition, a special debugging allocator is used if PYMALLOC_DEBUG
52is also #defined.
Guido van Rossumb18618d2000-05-03 23:44:39 +000053
Tim Peters8b078f92002-04-28 04:11:46 +000054In case a specific form of memory management is needed (for example, if you
55must use the platform malloc heap(s), or shared memory, or C++ local storage or
56operator new), you must first allocate the object with your custom allocator,
57then pass its pointer to PyObject_{Init, InitVar} for filling in its Python-
58specific fields: reference count, type pointer, possibly others. You should
59be aware that Python no control over these objects because they don't
60cooperate with the Python memory manager. Such objects may not be eligible
61for automatic garbage collection and you have to make sure that they are
62released accordingly whenever their destructor gets called (cf. the specific
Guido van Rossumb18618d2000-05-03 23:44:39 +000063form of memory management you're using).
64
Tim Peters8b078f92002-04-28 04:11:46 +000065Unless you have specific memory management requirements, use
66PyObject_{New, NewVar, Del}.
67*/
Guido van Rossumb18618d2000-05-03 23:44:39 +000068
Tim Peters6d483d32001-10-06 21:27:34 +000069/*
Guido van Rossumb18618d2000-05-03 23:44:39 +000070 * Raw object memory interface
71 * ===========================
72 */
73
Tim Peterse9e74522002-04-12 05:21:34 +000074/* Functions to call the same malloc/realloc/free as used by Python's
75 object allocator. If WITH_PYMALLOC is enabled, these may differ from
76 the platform malloc/realloc/free. The Python object allocator is
77 designed for fast, cache-conscious allocation of many "small" objects,
Tim Peters8b078f92002-04-28 04:11:46 +000078 and with low hidden memory overhead.
79
80 PyObject_Malloc(0) returns a unique non-NULL pointer if possible.
81
82 PyObject_Realloc(NULL, n) acts like PyObject_Malloc(n).
83 PyObject_Realloc(p != NULL, 0) does not return NULL, or free the memory
84 at p.
85
86 Returned pointers must be checked for NULL explicitly; no action is
87 performed on failure other than to return NULL (no warning it printed, no
88 exception is set, etc).
89
90 For allocating objects, use PyObject_{New, NewVar} instead whenever
91 possible. The PyObject_{Malloc, Realloc, Free} family is exposed
92 so that you can exploit Python's small-block allocator for non-object
93 uses. If you must use these routines to allocate object memory, make sure
94 the object gets initialized via PyObject_{Init, InitVar} after obtaining
95 the raw memory.
96*/
Mark Hammond91a681d2002-08-12 07:21:58 +000097PyAPI_FUNC(void *) PyObject_Malloc(size_t);
98PyAPI_FUNC(void *) PyObject_Realloc(void *, size_t);
99PyAPI_FUNC(void) PyObject_Free(void *);
Guido van Rossumb18618d2000-05-03 23:44:39 +0000100
Neil Schemenauer3e7b8932002-04-12 02:38:45 +0000101
Guido van Rossumb18618d2000-05-03 23:44:39 +0000102/* Macros */
Neil Schemenauer3e7b8932002-04-12 02:38:45 +0000103#ifdef WITH_PYMALLOC
104#ifdef PYMALLOC_DEBUG
Mark Hammond91a681d2002-08-12 07:21:58 +0000105PyAPI_FUNC(void *) _PyObject_DebugMalloc(size_t nbytes);
106PyAPI_FUNC(void *) _PyObject_DebugRealloc(void *p, size_t nbytes);
107PyAPI_FUNC(void) _PyObject_DebugFree(void *p);
108PyAPI_FUNC(void) _PyObject_DebugDumpAddress(const void *p);
109PyAPI_FUNC(void) _PyObject_DebugCheckAddress(const void *p);
110PyAPI_FUNC(void) _PyObject_DebugMallocStats(void);
Tim Peterse9e74522002-04-12 05:21:34 +0000111#define PyObject_MALLOC _PyObject_DebugMalloc
112#define PyObject_Malloc _PyObject_DebugMalloc
113#define PyObject_REALLOC _PyObject_DebugRealloc
114#define PyObject_Realloc _PyObject_DebugRealloc
115#define PyObject_FREE _PyObject_DebugFree
116#define PyObject_Free _PyObject_DebugFree
Neil Schemenauer3e7b8932002-04-12 02:38:45 +0000117
118#else /* WITH_PYMALLOC && ! PYMALLOC_DEBUG */
119#define PyObject_MALLOC PyObject_Malloc
120#define PyObject_REALLOC PyObject_Realloc
121#define PyObject_FREE PyObject_Free
122#endif
123
124#else /* ! WITH_PYMALLOC */
125#define PyObject_MALLOC PyMem_MALLOC
126#define PyObject_REALLOC PyMem_REALLOC
Tim Peters8b078f92002-04-28 04:11:46 +0000127/* This is an odd one! For backward compatability with old extensions, the
128 PyMem "release memory" functions have to invoke the object allocator's
129 free() function. When pymalloc isn't enabled, that leaves us using
130 the platform free(). */
131#define PyObject_FREE free
132
Neil Schemenauer3e7b8932002-04-12 02:38:45 +0000133#endif /* WITH_PYMALLOC */
134
Tim Peters8b078f92002-04-28 04:11:46 +0000135#define PyObject_Del PyObject_Free
136#define PyObject_DEL PyObject_FREE
Neil Schemenauer3e7b8932002-04-12 02:38:45 +0000137
138/* for source compatibility with 2.2 */
Tim Peters8b078f92002-04-28 04:11:46 +0000139#define _PyObject_Del PyObject_Free
Guido van Rossumb18618d2000-05-03 23:44:39 +0000140
141/*
142 * Generic object allocator interface
143 * ==================================
144 */
145
146/* Functions */
Mark Hammond91a681d2002-08-12 07:21:58 +0000147PyAPI_FUNC(PyObject *) PyObject_Init(PyObject *, PyTypeObject *);
148PyAPI_FUNC(PyVarObject *) PyObject_InitVar(PyVarObject *,
Fred Drake3cf4d2b2000-07-09 00:55:06 +0000149 PyTypeObject *, int);
Mark Hammond91a681d2002-08-12 07:21:58 +0000150PyAPI_FUNC(PyObject *) _PyObject_New(PyTypeObject *);
151PyAPI_FUNC(PyVarObject *) _PyObject_NewVar(PyTypeObject *, int);
Guido van Rossum85a5fbb1990-10-14 12:07:46 +0000152
Guido van Rossumb18618d2000-05-03 23:44:39 +0000153#define PyObject_New(type, typeobj) \
154 ( (type *) _PyObject_New(typeobj) )
155#define PyObject_NewVar(type, typeobj, n) \
156 ( (type *) _PyObject_NewVar((typeobj), (n)) )
Guido van Rossuma3309961993-07-28 09:05:47 +0000157
Andrew M. Kuchling1582a3a2000-08-16 12:27:23 +0000158/* Macros trading binary compatibility for speed. See also pymem.h.
Guido van Rossumb18618d2000-05-03 23:44:39 +0000159 Note that these macros expect non-NULL object pointers.*/
160#define PyObject_INIT(op, typeobj) \
Fred Drake4e262a92001-03-22 18:26:47 +0000161 ( (op)->ob_type = (typeobj), _Py_NewReference((PyObject *)(op)), (op) )
Guido van Rossumb18618d2000-05-03 23:44:39 +0000162#define PyObject_INIT_VAR(op, typeobj, size) \
163 ( (op)->ob_size = (size), PyObject_INIT((op), (typeobj)) )
Guido van Rossum5a849141996-07-21 02:23:54 +0000164
Guido van Rossumb18618d2000-05-03 23:44:39 +0000165#define _PyObject_SIZE(typeobj) ( (typeobj)->tp_basicsize )
Tim Peters6d483d32001-10-06 21:27:34 +0000166
Tim Petersf2a67da2001-10-07 03:54:51 +0000167/* _PyObject_VAR_SIZE returns the number of bytes (as size_t) allocated for a
168 vrbl-size object with nitems items, exclusive of gc overhead (if any). The
169 value is rounded up to the closest multiple of sizeof(void *), in order to
170 ensure that pointer fields at the end of the object are correctly aligned
171 for the platform (this is of special importance for subclasses of, e.g.,
172 str or long, so that pointers can be stored after the embedded data).
Tim Peters6d483d32001-10-06 21:27:34 +0000173
Tim Petersf2a67da2001-10-07 03:54:51 +0000174 Note that there's no memory wastage in doing this, as malloc has to
175 return (at worst) pointer-aligned memory anyway.
Tim Peters6d483d32001-10-06 21:27:34 +0000176*/
Tim Petersf2a67da2001-10-07 03:54:51 +0000177#if ((SIZEOF_VOID_P - 1) & SIZEOF_VOID_P) != 0
178# error "_PyObject_VAR_SIZE requires SIZEOF_VOID_P be a power of 2"
179#endif
180
181#define _PyObject_VAR_SIZE(typeobj, nitems) \
182 (size_t) \
183 ( ( (typeobj)->tp_basicsize + \
184 (nitems)*(typeobj)->tp_itemsize + \
185 (SIZEOF_VOID_P - 1) \
186 ) & ~(SIZEOF_VOID_P - 1) \
187 )
Guido van Rossum5a849141996-07-21 02:23:54 +0000188
Guido van Rossumb18618d2000-05-03 23:44:39 +0000189#define PyObject_NEW(type, typeobj) \
190( (type *) PyObject_Init( \
191 (PyObject *) PyObject_MALLOC( _PyObject_SIZE(typeobj) ), (typeobj)) )
Tim Peters6d483d32001-10-06 21:27:34 +0000192
Tim Petersf2a67da2001-10-07 03:54:51 +0000193#define PyObject_NEW_VAR(type, typeobj, n) \
194( (type *) PyObject_InitVar( \
195 (PyVarObject *) PyObject_MALLOC(_PyObject_VAR_SIZE((typeobj),(n)) ),\
196 (typeobj), (n)) )
Guido van Rossumb18618d2000-05-03 23:44:39 +0000197
198/* This example code implements an object constructor with a custom
199 allocator, where PyObject_New is inlined, and shows the important
200 distinction between two steps (at least):
201 1) the actual allocation of the object storage;
202 2) the initialization of the Python specific fields
203 in this storage with PyObject_{Init, InitVar}.
204
205 PyObject *
206 YourObject_New(...)
207 {
208 PyObject *op;
209
210 op = (PyObject *) Your_Allocator(_PyObject_SIZE(YourTypeStruct));
211 if (op == NULL)
212 return PyErr_NoMemory();
213
Tim Peters8b078f92002-04-28 04:11:46 +0000214 PyObject_Init(op, &YourTypeStruct);
Guido van Rossumb18618d2000-05-03 23:44:39 +0000215
216 op->ob_field = value;
217 ...
218 return op;
219 }
220
221 Note that in C++, the use of the new operator usually implies that
222 the 1st step is performed automatically for you, so in a C++ class
Tim Peters8b078f92002-04-28 04:11:46 +0000223 constructor you would start directly with PyObject_Init/InitVar
224*/
Guido van Rossum5a849141996-07-21 02:23:54 +0000225
Jeremy Hyltonc5007aa2000-06-30 05:02:53 +0000226/*
227 * Garbage Collection Support
228 * ==========================
229 */
Jeremy Hyltond08b4c42000-06-23 19:37:02 +0000230
Guido van Rossume13ddc92003-04-17 17:29:22 +0000231/* C equivalent of gc.collect(). */
232long PyGC_Collect(void);
233
Neil Schemenauer74b5ade2001-08-29 23:49:28 +0000234/* Test if a type has a GC head */
235#define PyType_IS_GC(t) PyType_HasFeature((t), Py_TPFLAGS_HAVE_GC)
Jeremy Hyltond08b4c42000-06-23 19:37:02 +0000236
Neil Schemenauer74b5ade2001-08-29 23:49:28 +0000237/* Test if an object has a GC head */
Guido van Rossum048eb752001-10-02 21:24:57 +0000238#define PyObject_IS_GC(o) (PyType_IS_GC((o)->ob_type) && \
239 ((o)->ob_type->tp_is_gc == NULL || (o)->ob_type->tp_is_gc(o)))
Jeremy Hyltonc5007aa2000-06-30 05:02:53 +0000240
Mark Hammond91a681d2002-08-12 07:21:58 +0000241PyAPI_FUNC(PyVarObject *) _PyObject_GC_Resize(PyVarObject *, int);
Neil Schemenauer74b5ade2001-08-29 23:49:28 +0000242#define PyObject_GC_Resize(type, op, n) \
243 ( (type *) _PyObject_GC_Resize((PyVarObject *)(op), (n)) )
244
Neil Schemenauer3e7b8932002-04-12 02:38:45 +0000245/* for source compatibility with 2.2 */
246#define _PyObject_GC_Del PyObject_GC_Del
Neil Schemenauer74b5ade2001-08-29 23:49:28 +0000247
Tim Peters8b078f92002-04-28 04:11:46 +0000248/* GC information is stored BEFORE the object structure. */
Tim Peters9e4ca102001-10-11 18:31:31 +0000249typedef union _gc_head {
250 struct {
Tim Peters6fc13d92002-07-02 18:12:35 +0000251 union _gc_head *gc_next;
Tim Peters9e4ca102001-10-11 18:31:31 +0000252 union _gc_head *gc_prev;
253 int gc_refs;
254 } gc;
Tim Peters5e67cde2002-02-28 19:38:51 +0000255 long double dummy; /* force worst-case alignment */
Neil Schemenauer74b5ade2001-08-29 23:49:28 +0000256} PyGC_Head;
257
Neil Schemenauerb1094f02002-05-04 05:36:06 +0000258extern PyGC_Head *_PyGC_generation0;
Neil Schemenauer74b5ade2001-08-29 23:49:28 +0000259
Neil Schemenaueref997232002-03-28 21:06:16 +0000260#define _Py_AS_GC(o) ((PyGC_Head *)(o)-1)
261
Tim Petersea405632002-07-02 00:52:30 +0000262#define _PyGC_REFS_UNTRACKED (-2)
263#define _PyGC_REFS_REACHABLE (-3)
264#define _PyGC_REFS_TENTATIVELY_UNREACHABLE (-4)
265
Neil Schemenauer74b5ade2001-08-29 23:49:28 +0000266/* Tell the GC to track this object. NB: While the object is tracked the
267 * collector it must be safe to call the ob_traverse method. */
268#define _PyObject_GC_TRACK(o) do { \
Neil Schemenaueref997232002-03-28 21:06:16 +0000269 PyGC_Head *g = _Py_AS_GC(o); \
Tim Petersea405632002-07-02 00:52:30 +0000270 if (g->gc.gc_refs != _PyGC_REFS_UNTRACKED) \
271 Py_FatalError("GC object already tracked"); \
Tim Petersea405632002-07-02 00:52:30 +0000272 g->gc.gc_refs = _PyGC_REFS_REACHABLE; \
Neil Schemenauerb1094f02002-05-04 05:36:06 +0000273 g->gc.gc_next = _PyGC_generation0; \
274 g->gc.gc_prev = _PyGC_generation0->gc.gc_prev; \
Tim Peters9e4ca102001-10-11 18:31:31 +0000275 g->gc.gc_prev->gc.gc_next = g; \
Neil Schemenauerb1094f02002-05-04 05:36:06 +0000276 _PyGC_generation0->gc.gc_prev = g; \
Neil Schemenauer74b5ade2001-08-29 23:49:28 +0000277 } while (0);
278
Tim Peters6fc13d92002-07-02 18:12:35 +0000279/* Tell the GC to stop tracking this object.
280 * gc_next doesn't need to be set to NULL, but doing so is a good
281 * way to provoke memory errors if calling code is confused.
282 */
Neil Schemenauer74b5ade2001-08-29 23:49:28 +0000283#define _PyObject_GC_UNTRACK(o) do { \
Neil Schemenaueref997232002-03-28 21:06:16 +0000284 PyGC_Head *g = _Py_AS_GC(o); \
Tim Petersea405632002-07-02 00:52:30 +0000285 assert(g->gc.gc_refs != _PyGC_REFS_UNTRACKED); \
286 g->gc.gc_refs = _PyGC_REFS_UNTRACKED; \
Tim Peters9e4ca102001-10-11 18:31:31 +0000287 g->gc.gc_prev->gc.gc_next = g->gc.gc_next; \
288 g->gc.gc_next->gc.gc_prev = g->gc.gc_prev; \
289 g->gc.gc_next = NULL; \
Neil Schemenauer74b5ade2001-08-29 23:49:28 +0000290 } while (0);
291
Mark Hammond91a681d2002-08-12 07:21:58 +0000292PyAPI_FUNC(PyObject *) _PyObject_GC_Malloc(size_t);
293PyAPI_FUNC(PyObject *) _PyObject_GC_New(PyTypeObject *);
294PyAPI_FUNC(PyVarObject *) _PyObject_GC_NewVar(PyTypeObject *, int);
295PyAPI_FUNC(void) PyObject_GC_Track(void *);
296PyAPI_FUNC(void) PyObject_GC_UnTrack(void *);
297PyAPI_FUNC(void) PyObject_GC_Del(void *);
Neil Schemenauer74b5ade2001-08-29 23:49:28 +0000298
299#define PyObject_GC_New(type, typeobj) \
300 ( (type *) _PyObject_GC_New(typeobj) )
301#define PyObject_GC_NewVar(type, typeobj, n) \
302 ( (type *) _PyObject_GC_NewVar((typeobj), (n)) )
Neil Schemenauer3e7b8932002-04-12 02:38:45 +0000303
Neil Schemenauer74b5ade2001-08-29 23:49:28 +0000304
Tim Peterseda29302004-07-15 04:05:59 +0000305/* Utility macro to help write tp_traverse functions.
306 * To use this macro, the tp_traverse function must name its arguments
307 * "visit" and "arg". This is intended to keep tp_traverse functions
308 * looking as much alike as possible.
309 */
Jim Fultonaa6389e2004-07-14 19:08:17 +0000310#define Py_VISIT(op) \
311 do { \
312 if (op) { \
313 int vret = visit((op), arg); \
314 if (vret) \
315 return vret; \
316 } \
317 } while (0)
318
Neil Schemenauer74b5ade2001-08-29 23:49:28 +0000319/* This is here for the sake of backwards compatibility. Extensions that
320 * use the old GC API will still compile but the objects will not be
321 * tracked by the GC. */
Jeremy Hyltonc5007aa2000-06-30 05:02:53 +0000322#define PyGC_HEAD_SIZE 0
323#define PyObject_GC_Init(op)
324#define PyObject_GC_Fini(op)
325#define PyObject_AS_GC(op) (op)
326#define PyObject_FROM_GC(op) (op)
Tim Peters6d6c1a32001-08-02 04:15:00 +0000327
Jeremy Hyltond08b4c42000-06-23 19:37:02 +0000328
Fred Drake41deb1e2001-02-01 05:27:45 +0000329/* Test if a type supports weak references */
Fred Drake033f3122001-02-02 18:17:30 +0000330#define PyType_SUPPORTS_WEAKREFS(t) \
331 (PyType_HasFeature((t), Py_TPFLAGS_HAVE_WEAKREFS) \
332 && ((t)->tp_weaklistoffset > 0))
Fred Drake41deb1e2001-02-01 05:27:45 +0000333
334#define PyObject_GET_WEAKREFS_LISTPTR(o) \
335 ((PyObject **) (((char *) (o)) + (o)->ob_type->tp_weaklistoffset))
336
Guido van Rossuma3309961993-07-28 09:05:47 +0000337#ifdef __cplusplus
338}
339#endif
340#endif /* !Py_OBJIMPL_H */