Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 1 | .. highlightlang:: c |
| 2 | |
Chris Jerdonek | 5fae0e5 | 2012-11-20 17:45:51 -0800 | [diff] [blame] | 3 | .. index:: |
| 4 | single: buffer protocol |
| 5 | single: buffer interface; (see buffer protocol) |
| 6 | single: buffer object; (see buffer protocol) |
| 7 | |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 8 | .. _bufferobjects: |
| 9 | |
Antoine Pitrou | f7ba2fa | 2010-09-28 23:39:41 +0000 | [diff] [blame] | 10 | Buffer Protocol |
| 11 | --------------- |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 12 | |
| 13 | .. sectionauthor:: Greg Stein <gstein@lyra.org> |
Benjamin Peterson | 9d0ced3 | 2008-09-16 02:24:31 +0000 | [diff] [blame] | 14 | .. sectionauthor:: Benjamin Peterson |
Stefan Krah | 9a2d99e | 2012-02-25 12:24:21 +0100 | [diff] [blame] | 15 | .. sectionauthor:: Stefan Krah |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 16 | |
| 17 | |
Antoine Pitrou | 8abc935 | 2010-12-12 19:59:47 +0000 | [diff] [blame] | 18 | Certain objects available in Python wrap access to an underlying memory |
| 19 | array or *buffer*. Such objects include the built-in :class:`bytes` and |
| 20 | :class:`bytearray`, and some extension types like :class:`array.array`. |
| 21 | Third-party libraries may define their own types for special purposes, such |
| 22 | as image processing or numeric analysis. |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 23 | |
Antoine Pitrou | 8abc935 | 2010-12-12 19:59:47 +0000 | [diff] [blame] | 24 | While each of these types have their own semantics, they share the common |
| 25 | characteristic of being backed by a possibly large memory buffer. It is |
Stefan Krah | 9a2d99e | 2012-02-25 12:24:21 +0100 | [diff] [blame] | 26 | then desirable, in some situations, to access that buffer directly and |
Antoine Pitrou | 8abc935 | 2010-12-12 19:59:47 +0000 | [diff] [blame] | 27 | without intermediate copying. |
| 28 | |
Chris Jerdonek | 5fae0e5 | 2012-11-20 17:45:51 -0800 | [diff] [blame] | 29 | Python provides such a facility at the C level in the form of the :ref:`buffer |
| 30 | protocol <bufferobjects>`. This protocol has two sides: |
Antoine Pitrou | 8abc935 | 2010-12-12 19:59:47 +0000 | [diff] [blame] | 31 | |
| 32 | .. index:: single: PyBufferProcs |
| 33 | |
| 34 | - on the producer side, a type can export a "buffer interface" which allows |
| 35 | objects of that type to expose information about their underlying buffer. |
| 36 | This interface is described in the section :ref:`buffer-structs`; |
| 37 | |
| 38 | - on the consumer side, several means are available to obtain a pointer to |
| 39 | the raw underlying data of an object (for example a method parameter). |
| 40 | |
| 41 | Simple objects such as :class:`bytes` and :class:`bytearray` expose their |
| 42 | underlying buffer in byte-oriented form. Other forms are possible; for example, |
| 43 | the elements exposed by a :class:`array.array` can be multi-byte values. |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 44 | |
Antoine Pitrou | 99a00a4 | 2010-09-28 23:04:04 +0000 | [diff] [blame] | 45 | An example consumer of the buffer interface is the :meth:`~io.BufferedIOBase.write` |
| 46 | method of file objects: any object that can export a series of bytes through |
| 47 | the buffer interface can be written to a file. While :meth:`write` only |
| 48 | needs read-only access to the internal contents of the object passed to it, |
| 49 | other methods such as :meth:`~io.BufferedIOBase.readinto` need write access |
| 50 | to the contents of their argument. The buffer interface allows objects to |
| 51 | selectively allow or reject exporting of read-write and read-only buffers. |
| 52 | |
| 53 | There are two ways for a consumer of the buffer interface to acquire a buffer |
| 54 | over a target object: |
| 55 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 56 | * call :c:func:`PyObject_GetBuffer` with the right parameters; |
Antoine Pitrou | 99a00a4 | 2010-09-28 23:04:04 +0000 | [diff] [blame] | 57 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 58 | * call :c:func:`PyArg_ParseTuple` (or one of its siblings) with one of the |
Antoine Pitrou | 99a00a4 | 2010-09-28 23:04:04 +0000 | [diff] [blame] | 59 | ``y*``, ``w*`` or ``s*`` :ref:`format codes <arg-parsing>`. |
| 60 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 61 | In both cases, :c:func:`PyBuffer_Release` must be called when the buffer |
Antoine Pitrou | 99a00a4 | 2010-09-28 23:04:04 +0000 | [diff] [blame] | 62 | isn't needed anymore. Failure to do so could lead to various issues such as |
| 63 | resource leaks. |
| 64 | |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 65 | |
Stefan Krah | 9a2d99e | 2012-02-25 12:24:21 +0100 | [diff] [blame] | 66 | .. _buffer-structure: |
| 67 | |
| 68 | Buffer structure |
| 69 | ================ |
Antoine Pitrou | 99a00a4 | 2010-09-28 23:04:04 +0000 | [diff] [blame] | 70 | |
Antoine Pitrou | f7ba2fa | 2010-09-28 23:39:41 +0000 | [diff] [blame] | 71 | Buffer structures (or simply "buffers") are useful as a way to expose the |
| 72 | binary data from another object to the Python programmer. They can also be |
| 73 | used as a zero-copy slicing mechanism. Using their ability to reference a |
| 74 | block of memory, it is possible to expose any data to the Python programmer |
| 75 | quite easily. The memory could be a large, constant array in a C extension, |
| 76 | it could be a raw block of memory for manipulation before passing to an |
| 77 | operating system library, or it could be used to pass around structured data |
| 78 | in its native, in-memory format. |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 79 | |
Antoine Pitrou | f7ba2fa | 2010-09-28 23:39:41 +0000 | [diff] [blame] | 80 | Contrary to most data types exposed by the Python interpreter, buffers |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 81 | are not :c:type:`PyObject` pointers but rather simple C structures. This |
Antoine Pitrou | 99a00a4 | 2010-09-28 23:04:04 +0000 | [diff] [blame] | 82 | allows them to be created and copied very simply. When a generic wrapper |
Antoine Pitrou | c663b58 | 2010-09-28 23:59:51 +0000 | [diff] [blame] | 83 | around a buffer is needed, a :ref:`memoryview <memoryview-objects>` object |
Antoine Pitrou | 99a00a4 | 2010-09-28 23:04:04 +0000 | [diff] [blame] | 84 | can be created. |
| 85 | |
Stefan Krah | abd887d | 2012-03-06 14:55:06 +0100 | [diff] [blame] | 86 | For short instructions how to write an exporting object, see |
| 87 | :ref:`Buffer Object Structures <buffer-structs>`. For obtaining |
| 88 | a buffer, see :c:func:`PyObject_GetBuffer`. |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 89 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 90 | .. c:type:: Py_buffer |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 91 | |
Stefan Krah | 9a2d99e | 2012-02-25 12:24:21 +0100 | [diff] [blame] | 92 | .. c:member:: void \*obj |
Benjamin Peterson | 9d0ced3 | 2008-09-16 02:24:31 +0000 | [diff] [blame] | 93 | |
Stefan Krah | abd887d | 2012-03-06 14:55:06 +0100 | [diff] [blame] | 94 | A new reference to the exporting object. The reference is owned by |
| 95 | the consumer and automatically decremented and set to *NULL* by |
| 96 | :c:func:`PyBuffer_Release`. The field is the equivalent of the return |
| 97 | value of any standard C-API function. |
Stefan Krah | 9a2d99e | 2012-02-25 12:24:21 +0100 | [diff] [blame] | 98 | |
Stefan Krah | abd887d | 2012-03-06 14:55:06 +0100 | [diff] [blame] | 99 | As a special case, for *temporary* buffers that are wrapped by |
| 100 | :c:func:`PyMemoryView_FromBuffer` or :c:func:`PyBuffer_FillInfo` |
| 101 | this field is *NULL*. In general, exporting objects MUST NOT |
| 102 | use this scheme. |
Stefan Krah | 9a2d99e | 2012-02-25 12:24:21 +0100 | [diff] [blame] | 103 | |
| 104 | .. c:member:: void \*buf |
| 105 | |
| 106 | A pointer to the start of the logical structure described by the buffer |
| 107 | fields. This can be any location within the underlying physical memory |
| 108 | block of the exporter. For example, with negative :c:member:`~Py_buffer.strides` |
| 109 | the value may point to the end of the memory block. |
| 110 | |
| 111 | For contiguous arrays, the value points to the beginning of the memory |
| 112 | block. |
Benjamin Peterson | 9d0ced3 | 2008-09-16 02:24:31 +0000 | [diff] [blame] | 113 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 114 | .. c:member:: Py_ssize_t len |
Benjamin Peterson | 9d0ced3 | 2008-09-16 02:24:31 +0000 | [diff] [blame] | 115 | |
Stefan Krah | 9a2d99e | 2012-02-25 12:24:21 +0100 | [diff] [blame] | 116 | ``product(shape) * itemsize``. For contiguous arrays, this is the length |
| 117 | of the underlying memory block. For non-contiguous arrays, it is the length |
| 118 | that the logical structure would have if it were copied to a contiguous |
| 119 | representation. |
| 120 | |
| 121 | Accessing ``((char *)buf)[0] up to ((char *)buf)[len-1]`` is only valid |
| 122 | if the buffer has been obtained by a request that guarantees contiguity. In |
| 123 | most cases such a request will be :c:macro:`PyBUF_SIMPLE` or :c:macro:`PyBUF_WRITABLE`. |
Benjamin Peterson | 9d0ced3 | 2008-09-16 02:24:31 +0000 | [diff] [blame] | 124 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 125 | .. c:member:: int readonly |
Benjamin Peterson | 9d0ced3 | 2008-09-16 02:24:31 +0000 | [diff] [blame] | 126 | |
Stefan Krah | 9a2d99e | 2012-02-25 12:24:21 +0100 | [diff] [blame] | 127 | An indicator of whether the buffer is read-only. This field is controlled |
| 128 | by the :c:macro:`PyBUF_WRITABLE` flag. |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 129 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 130 | .. c:member:: Py_ssize_t itemsize |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 131 | |
Stefan Krah | 9a2d99e | 2012-02-25 12:24:21 +0100 | [diff] [blame] | 132 | Item size in bytes of a single element. Same as the value of :func:`struct.calcsize` |
| 133 | called on non-NULL :c:member:`~Py_buffer.format` values. |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 134 | |
Stefan Krah | 9a2d99e | 2012-02-25 12:24:21 +0100 | [diff] [blame] | 135 | Important exception: If a consumer requests a buffer without the |
| 136 | :c:macro:`PyBUF_FORMAT` flag, :c:member:`~Py_Buffer.format` will |
| 137 | be set to *NULL*, but :c:member:`~Py_buffer.itemsize` still has |
| 138 | the value for the original format. |
| 139 | |
| 140 | If :c:member:`~Py_Buffer.shape` is present, the equality |
| 141 | ``product(shape) * itemsize == len`` still holds and the consumer |
| 142 | can use :c:member:`~Py_buffer.itemsize` to navigate the buffer. |
| 143 | |
| 144 | If :c:member:`~Py_Buffer.shape` is *NULL* as a result of a :c:macro:`PyBUF_SIMPLE` |
| 145 | or a :c:macro:`PyBUF_WRITABLE` request, the consumer must disregard |
| 146 | :c:member:`~Py_buffer.itemsize` and assume ``itemsize == 1``. |
| 147 | |
| 148 | .. c:member:: const char \*format |
| 149 | |
| 150 | A *NUL* terminated string in :mod:`struct` module style syntax describing |
| 151 | the contents of a single item. If this is *NULL*, ``"B"`` (unsigned bytes) |
| 152 | is assumed. |
| 153 | |
| 154 | This field is controlled by the :c:macro:`PyBUF_FORMAT` flag. |
| 155 | |
| 156 | .. c:member:: int ndim |
| 157 | |
| 158 | The number of dimensions the memory represents as an n-dimensional array. |
| 159 | If it is 0, :c:member:`~Py_Buffer.buf` points to a single item representing |
| 160 | a scalar. In this case, :c:member:`~Py_buffer.shape`, :c:member:`~Py_buffer.strides` |
| 161 | and :c:member:`~Py_buffer.suboffsets` MUST be *NULL*. |
| 162 | |
| 163 | The macro :c:macro:`PyBUF_MAX_NDIM` limits the maximum number of dimensions |
| 164 | to 64. Exporters MUST respect this limit, consumers of multi-dimensional |
| 165 | buffers SHOULD be able to handle up to :c:macro:`PyBUF_MAX_NDIM` dimensions. |
| 166 | |
| 167 | .. c:member:: Py_ssize_t \*shape |
| 168 | |
| 169 | An array of :c:type:`Py_ssize_t` of length :c:member:`~Py_buffer.ndim` |
| 170 | indicating the shape of the memory as an n-dimensional array. Note that |
| 171 | ``shape[0] * ... * shape[ndim-1] * itemsize`` MUST be equal to |
| 172 | :c:member:`~Py_buffer.len`. |
| 173 | |
| 174 | Shape values are restricted to ``shape[n] >= 0``. The case |
| 175 | ``shape[n] == 0`` requires special attention. See `complex arrays`_ |
| 176 | for further information. |
| 177 | |
| 178 | The shape array is read-only for the consumer. |
| 179 | |
| 180 | .. c:member:: Py_ssize_t \*strides |
| 181 | |
| 182 | An array of :c:type:`Py_ssize_t` of length :c:member:`~Py_buffer.ndim` |
| 183 | giving the number of bytes to skip to get to a new element in each |
| 184 | dimension. |
| 185 | |
| 186 | Stride values can be any integer. For regular arrays, strides are |
| 187 | usually positive, but a consumer MUST be able to handle the case |
| 188 | ``strides[n] <= 0``. See `complex arrays`_ for further information. |
| 189 | |
| 190 | The strides array is read-only for the consumer. |
| 191 | |
| 192 | .. c:member:: Py_ssize_t \*suboffsets |
| 193 | |
| 194 | An array of :c:type:`Py_ssize_t` of length :c:member:`~Py_buffer.ndim`. |
| 195 | If ``suboffsets[n] >= 0``, the values stored along the nth dimension are |
| 196 | pointers and the suboffset value dictates how many bytes to add to each |
| 197 | pointer after de-referencing. A suboffset value that is negative |
| 198 | indicates that no de-referencing should occur (striding in a contiguous |
| 199 | memory block). |
| 200 | |
| 201 | This type of array representation is used by the Python Imaging Library |
| 202 | (PIL). See `complex arrays`_ for further information how to access elements |
| 203 | of such an array. |
| 204 | |
| 205 | The suboffsets array is read-only for the consumer. |
| 206 | |
| 207 | .. c:member:: void \*internal |
Benjamin Peterson | 9d0ced3 | 2008-09-16 02:24:31 +0000 | [diff] [blame] | 208 | |
| 209 | This is for use internally by the exporting object. For example, this |
| 210 | might be re-cast as an integer by the exporter and used to store flags |
| 211 | about whether or not the shape, strides, and suboffsets arrays must be |
Stefan Krah | 9a2d99e | 2012-02-25 12:24:21 +0100 | [diff] [blame] | 212 | freed when the buffer is released. The consumer MUST NOT alter this |
Benjamin Peterson | 9d0ced3 | 2008-09-16 02:24:31 +0000 | [diff] [blame] | 213 | value. |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 214 | |
Stefan Krah | 9a2d99e | 2012-02-25 12:24:21 +0100 | [diff] [blame] | 215 | .. _buffer-request-types: |
| 216 | |
| 217 | Buffer request types |
| 218 | ==================== |
| 219 | |
| 220 | Buffers are usually obtained by sending a buffer request to an exporting |
| 221 | object via :c:func:`PyObject_GetBuffer`. Since the complexity of the logical |
| 222 | structure of the memory can vary drastically, the consumer uses the *flags* |
| 223 | argument to specify the exact buffer type it can handle. |
| 224 | |
| 225 | All :c:data:`Py_buffer` fields are unambiguously defined by the request |
| 226 | type. |
| 227 | |
| 228 | request-independent fields |
| 229 | ~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 230 | The following fields are not influenced by *flags* and must always be filled in |
| 231 | with the correct values: :c:member:`~Py_buffer.obj`, :c:member:`~Py_buffer.buf`, |
| 232 | :c:member:`~Py_buffer.len`, :c:member:`~Py_buffer.itemsize`, :c:member:`~Py_buffer.ndim`. |
| 233 | |
| 234 | |
| 235 | readonly, format |
| 236 | ~~~~~~~~~~~~~~~~ |
| 237 | |
| 238 | .. c:macro:: PyBUF_WRITABLE |
| 239 | |
| 240 | Controls the :c:member:`~Py_buffer.readonly` field. If set, the exporter |
| 241 | MUST provide a writable buffer or else report failure. Otherwise, the |
| 242 | exporter MAY provide either a read-only or writable buffer, but the choice |
| 243 | MUST be consistent for all consumers. |
| 244 | |
| 245 | .. c:macro:: PyBUF_FORMAT |
| 246 | |
| 247 | Controls the :c:member:`~Py_buffer.format` field. If set, this field MUST |
| 248 | be filled in correctly. Otherwise, this field MUST be *NULL*. |
| 249 | |
| 250 | |
| 251 | :c:macro:`PyBUF_WRITABLE` can be \|'d to any of the flags in the next section. |
| 252 | Since :c:macro:`PyBUF_SIMPLE` is defined as 0, :c:macro:`PyBUF_WRITABLE` |
| 253 | can be used as a stand-alone flag to request a simple writable buffer. |
| 254 | |
| 255 | :c:macro:`PyBUF_FORMAT` can be \|'d to any of the flags except :c:macro:`PyBUF_SIMPLE`. |
| 256 | The latter already implies format ``B`` (unsigned bytes). |
| 257 | |
| 258 | |
| 259 | shape, strides, suboffsets |
| 260 | ~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 261 | |
| 262 | The flags that control the logical structure of the memory are listed |
| 263 | in decreasing order of complexity. Note that each flag contains all bits |
| 264 | of the flags below it. |
| 265 | |
Georg Brandl | 44ea77b | 2013-03-28 13:28:44 +0100 | [diff] [blame] | 266 | .. tabularcolumns:: |p{0.35\linewidth}|l|l|l| |
Stefan Krah | 9a2d99e | 2012-02-25 12:24:21 +0100 | [diff] [blame] | 267 | |
| 268 | +-----------------------------+-------+---------+------------+ |
| 269 | | Request | shape | strides | suboffsets | |
| 270 | +=============================+=======+=========+============+ |
| 271 | | .. c:macro:: PyBUF_INDIRECT | yes | yes | if needed | |
| 272 | +-----------------------------+-------+---------+------------+ |
| 273 | | .. c:macro:: PyBUF_STRIDES | yes | yes | NULL | |
| 274 | +-----------------------------+-------+---------+------------+ |
| 275 | | .. c:macro:: PyBUF_ND | yes | NULL | NULL | |
| 276 | +-----------------------------+-------+---------+------------+ |
| 277 | | .. c:macro:: PyBUF_SIMPLE | NULL | NULL | NULL | |
| 278 | +-----------------------------+-------+---------+------------+ |
| 279 | |
| 280 | |
| 281 | contiguity requests |
| 282 | ~~~~~~~~~~~~~~~~~~~ |
| 283 | |
| 284 | C or Fortran contiguity can be explicitly requested, with and without stride |
| 285 | information. Without stride information, the buffer must be C-contiguous. |
| 286 | |
Georg Brandl | 44ea77b | 2013-03-28 13:28:44 +0100 | [diff] [blame] | 287 | .. tabularcolumns:: |p{0.35\linewidth}|l|l|l|l| |
| 288 | |
Stefan Krah | 9a2d99e | 2012-02-25 12:24:21 +0100 | [diff] [blame] | 289 | +-----------------------------------+-------+---------+------------+--------+ |
| 290 | | Request | shape | strides | suboffsets | contig | |
| 291 | +===================================+=======+=========+============+========+ |
| 292 | | .. c:macro:: PyBUF_C_CONTIGUOUS | yes | yes | NULL | C | |
| 293 | +-----------------------------------+-------+---------+------------+--------+ |
| 294 | | .. c:macro:: PyBUF_F_CONTIGUOUS | yes | yes | NULL | F | |
| 295 | +-----------------------------------+-------+---------+------------+--------+ |
| 296 | | .. c:macro:: PyBUF_ANY_CONTIGUOUS | yes | yes | NULL | C or F | |
| 297 | +-----------------------------------+-------+---------+------------+--------+ |
| 298 | | .. c:macro:: PyBUF_ND | yes | NULL | NULL | C | |
| 299 | +-----------------------------------+-------+---------+------------+--------+ |
| 300 | |
| 301 | |
| 302 | compound requests |
| 303 | ~~~~~~~~~~~~~~~~~ |
| 304 | |
| 305 | All possible requests are fully defined by some combination of the flags in |
| 306 | the previous section. For convenience, the buffer protocol provides frequently |
| 307 | used combinations as single flags. |
| 308 | |
| 309 | In the following table *U* stands for undefined contiguity. The consumer would |
| 310 | have to call :c:func:`PyBuffer_IsContiguous` to determine contiguity. |
| 311 | |
Georg Brandl | 44ea77b | 2013-03-28 13:28:44 +0100 | [diff] [blame] | 312 | .. tabularcolumns:: |p{0.35\linewidth}|l|l|l|l|l|l| |
Stefan Krah | 9a2d99e | 2012-02-25 12:24:21 +0100 | [diff] [blame] | 313 | |
| 314 | +-------------------------------+-------+---------+------------+--------+----------+--------+ |
| 315 | | Request | shape | strides | suboffsets | contig | readonly | format | |
| 316 | +===============================+=======+=========+============+========+==========+========+ |
| 317 | | .. c:macro:: PyBUF_FULL | yes | yes | if needed | U | 0 | yes | |
| 318 | +-------------------------------+-------+---------+------------+--------+----------+--------+ |
| 319 | | .. c:macro:: PyBUF_FULL_RO | yes | yes | if needed | U | 1 or 0 | yes | |
| 320 | +-------------------------------+-------+---------+------------+--------+----------+--------+ |
| 321 | | .. c:macro:: PyBUF_RECORDS | yes | yes | NULL | U | 0 | yes | |
| 322 | +-------------------------------+-------+---------+------------+--------+----------+--------+ |
| 323 | | .. c:macro:: PyBUF_RECORDS_RO | yes | yes | NULL | U | 1 or 0 | yes | |
| 324 | +-------------------------------+-------+---------+------------+--------+----------+--------+ |
| 325 | | .. c:macro:: PyBUF_STRIDED | yes | yes | NULL | U | 0 | NULL | |
| 326 | +-------------------------------+-------+---------+------------+--------+----------+--------+ |
| 327 | | .. c:macro:: PyBUF_STRIDED_RO | yes | yes | NULL | U | 1 or 0 | NULL | |
| 328 | +-------------------------------+-------+---------+------------+--------+----------+--------+ |
| 329 | | .. c:macro:: PyBUF_CONTIG | yes | NULL | NULL | C | 0 | NULL | |
| 330 | +-------------------------------+-------+---------+------------+--------+----------+--------+ |
| 331 | | .. c:macro:: PyBUF_CONTIG_RO | yes | NULL | NULL | C | 1 or 0 | NULL | |
| 332 | +-------------------------------+-------+---------+------------+--------+----------+--------+ |
| 333 | |
| 334 | |
| 335 | Complex arrays |
| 336 | ============== |
| 337 | |
| 338 | NumPy-style: shape and strides |
| 339 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 340 | |
| 341 | The logical structure of NumPy-style arrays is defined by :c:member:`~Py_buffer.itemsize`, |
| 342 | :c:member:`~Py_buffer.ndim`, :c:member:`~Py_buffer.shape` and :c:member:`~Py_buffer.strides`. |
| 343 | |
| 344 | If ``ndim == 0``, the memory location pointed to by :c:member:`~Py_buffer.buf` is |
| 345 | interpreted as a scalar of size :c:member:`~Py_buffer.itemsize`. In that case, |
| 346 | both :c:member:`~Py_buffer.shape` and :c:member:`~Py_buffer.strides` are *NULL*. |
| 347 | |
| 348 | If :c:member:`~Py_buffer.strides` is *NULL*, the array is interpreted as |
| 349 | a standard n-dimensional C-array. Otherwise, the consumer must access an |
| 350 | n-dimensional array as follows: |
| 351 | |
| 352 | ``ptr = (char *)buf + indices[0] * strides[0] + ... + indices[n-1] * strides[n-1]`` |
| 353 | ``item = *((typeof(item) *)ptr);`` |
| 354 | |
| 355 | |
| 356 | As noted above, :c:member:`~Py_buffer.buf` can point to any location within |
| 357 | the actual memory block. An exporter can check the validity of a buffer with |
| 358 | this function: |
| 359 | |
| 360 | .. code-block:: python |
| 361 | |
| 362 | def verify_structure(memlen, itemsize, ndim, shape, strides, offset): |
| 363 | """Verify that the parameters represent a valid array within |
| 364 | the bounds of the allocated memory: |
| 365 | char *mem: start of the physical memory block |
| 366 | memlen: length of the physical memory block |
| 367 | offset: (char *)buf - mem |
| 368 | """ |
| 369 | if offset % itemsize: |
| 370 | return False |
| 371 | if offset < 0 or offset+itemsize > memlen: |
| 372 | return False |
| 373 | if any(v % itemsize for v in strides): |
| 374 | return False |
| 375 | |
| 376 | if ndim <= 0: |
| 377 | return ndim == 0 and not shape and not strides |
| 378 | if 0 in shape: |
| 379 | return True |
| 380 | |
| 381 | imin = sum(strides[j]*(shape[j]-1) for j in range(ndim) |
| 382 | if strides[j] <= 0) |
| 383 | imax = sum(strides[j]*(shape[j]-1) for j in range(ndim) |
| 384 | if strides[j] > 0) |
| 385 | |
| 386 | return 0 <= offset+imin and offset+imax+itemsize <= memlen |
| 387 | |
| 388 | |
| 389 | PIL-style: shape, strides and suboffsets |
| 390 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 391 | |
| 392 | In addition to the regular items, PIL-style arrays can contain pointers |
| 393 | that must be followed in order to get to the next element in a dimension. |
| 394 | For example, the regular three-dimensional C-array ``char v[2][2][3]`` can |
| 395 | also be viewed as an array of 2 pointers to 2 two-dimensional arrays: |
| 396 | ``char (*v[2])[2][3]``. In suboffsets representation, those two pointers |
| 397 | can be embedded at the start of :c:member:`~Py_buffer.buf`, pointing |
| 398 | to two ``char x[2][3]`` arrays that can be located anywhere in memory. |
| 399 | |
| 400 | |
| 401 | Here is a function that returns a pointer to the element in an N-D array |
| 402 | pointed to by an N-dimensional index when there are both non-NULL strides |
| 403 | and suboffsets:: |
| 404 | |
| 405 | void *get_item_pointer(int ndim, void *buf, Py_ssize_t *strides, |
| 406 | Py_ssize_t *suboffsets, Py_ssize_t *indices) { |
| 407 | char *pointer = (char*)buf; |
| 408 | int i; |
| 409 | for (i = 0; i < ndim; i++) { |
| 410 | pointer += strides[i] * indices[i]; |
| 411 | if (suboffsets[i] >=0 ) { |
| 412 | pointer = *((char**)pointer) + suboffsets[i]; |
| 413 | } |
| 414 | } |
| 415 | return (void*)pointer; |
| 416 | } |
| 417 | |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 418 | |
Antoine Pitrou | c663b58 | 2010-09-28 23:59:51 +0000 | [diff] [blame] | 419 | Buffer-related functions |
Benjamin Peterson | 9d0ced3 | 2008-09-16 02:24:31 +0000 | [diff] [blame] | 420 | ======================== |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 421 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 422 | .. c:function:: int PyObject_CheckBuffer(PyObject *obj) |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 423 | |
Antoine Pitrou | 99a00a4 | 2010-09-28 23:04:04 +0000 | [diff] [blame] | 424 | Return 1 if *obj* supports the buffer interface otherwise 0. When 1 is |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 425 | returned, it doesn't guarantee that :c:func:`PyObject_GetBuffer` will |
Antoine Pitrou | 99a00a4 | 2010-09-28 23:04:04 +0000 | [diff] [blame] | 426 | succeed. |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 427 | |
| 428 | |
Stefan Krah | 9a2d99e | 2012-02-25 12:24:21 +0100 | [diff] [blame] | 429 | .. c:function:: int PyObject_GetBuffer(PyObject *exporter, Py_buffer *view, int flags) |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 430 | |
Stefan Krah | 9a2d99e | 2012-02-25 12:24:21 +0100 | [diff] [blame] | 431 | Send a request to *exporter* to fill in *view* as specified by *flags*. |
| 432 | If the exporter cannot provide a buffer of the exact type, it MUST raise |
| 433 | :c:data:`PyExc_BufferError`, set :c:member:`view->obj` to *NULL* and |
| 434 | return -1. |
Benjamin Peterson | 9d0ced3 | 2008-09-16 02:24:31 +0000 | [diff] [blame] | 435 | |
Stefan Krah | 9a2d99e | 2012-02-25 12:24:21 +0100 | [diff] [blame] | 436 | On success, fill in *view*, set :c:member:`view->obj` to a new reference |
Stefan Krah | abd887d | 2012-03-06 14:55:06 +0100 | [diff] [blame] | 437 | to *exporter* and return 0. In the case of chained buffer providers |
| 438 | that redirect requests to a single object, :c:member:`view->obj` MAY |
| 439 | refer to this object instead of *exporter* (See :ref:`Buffer Object Structures <buffer-structs>`). |
Benjamin Peterson | 9d0ced3 | 2008-09-16 02:24:31 +0000 | [diff] [blame] | 440 | |
Stefan Krah | 9a2d99e | 2012-02-25 12:24:21 +0100 | [diff] [blame] | 441 | Successful calls to :c:func:`PyObject_GetBuffer` must be paired with calls |
| 442 | to :c:func:`PyBuffer_Release`, similar to :c:func:`malloc` and :c:func:`free`. |
| 443 | Thus, after the consumer is done with the buffer, :c:func:`PyBuffer_Release` |
| 444 | must be called exactly once. |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 445 | |
| 446 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 447 | .. c:function:: void PyBuffer_Release(Py_buffer *view) |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 448 | |
Stefan Krah | 9a2d99e | 2012-02-25 12:24:21 +0100 | [diff] [blame] | 449 | Release the buffer *view* and decrement the reference count for |
| 450 | :c:member:`view->obj`. This function MUST be called when the buffer |
| 451 | is no longer being used, otherwise reference leaks may occur. |
| 452 | |
| 453 | It is an error to call this function on a buffer that was not obtained via |
| 454 | :c:func:`PyObject_GetBuffer`. |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 455 | |
| 456 | |
Georg Brandl | 60203b4 | 2010-10-06 10:11:56 +0000 | [diff] [blame] | 457 | .. c:function:: Py_ssize_t PyBuffer_SizeFromFormat(const char *) |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 458 | |
Stefan Krah | 9a2d99e | 2012-02-25 12:24:21 +0100 | [diff] [blame] | 459 | Return the implied :c:data:`~Py_buffer.itemsize` from :c:data:`~Py_buffer.format`. |
| 460 | This function is not yet implemented. |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 461 | |
| 462 | |
Stefan Krah | 9a2d99e | 2012-02-25 12:24:21 +0100 | [diff] [blame] | 463 | .. c:function:: int PyBuffer_IsContiguous(Py_buffer *view, char order) |
Georg Brandl | 54a3faa | 2008-01-20 09:30:57 +0000 | [diff] [blame] | 464 | |
Stefan Krah | 9a2d99e | 2012-02-25 12:24:21 +0100 | [diff] [blame] | 465 | Return 1 if the memory defined by the *view* is C-style (*order* is |
| 466 | ``'C'``) or Fortran-style (*order* is ``'F'``) contiguous or either one |
| 467 | (*order* is ``'A'``). Return 0 otherwise. |
Benjamin Peterson | 9d0ced3 | 2008-09-16 02:24:31 +0000 | [diff] [blame] | 468 | |
| 469 | |
Stefan Krah | 9a2d99e | 2012-02-25 12:24:21 +0100 | [diff] [blame] | 470 | .. c:function:: void PyBuffer_FillContiguousStrides(int ndim, Py_ssize_t *shape, Py_ssize_t *strides, Py_ssize_t itemsize, char order) |
Benjamin Peterson | 9d0ced3 | 2008-09-16 02:24:31 +0000 | [diff] [blame] | 471 | |
| 472 | Fill the *strides* array with byte-strides of a contiguous (C-style if |
Stefan Krah | 9a2d99e | 2012-02-25 12:24:21 +0100 | [diff] [blame] | 473 | *order* is ``'C'`` or Fortran-style if *order* is ``'F'``) array of the |
Benjamin Peterson | 9d0ced3 | 2008-09-16 02:24:31 +0000 | [diff] [blame] | 474 | given shape with the given number of bytes per element. |
| 475 | |
| 476 | |
Stefan Krah | 9a2d99e | 2012-02-25 12:24:21 +0100 | [diff] [blame] | 477 | .. c:function:: int PyBuffer_FillInfo(Py_buffer *view, PyObject *exporter, void *buf, Py_ssize_t len, int readonly, int flags) |
Benjamin Peterson | 9d0ced3 | 2008-09-16 02:24:31 +0000 | [diff] [blame] | 478 | |
Stefan Krah | 9a2d99e | 2012-02-25 12:24:21 +0100 | [diff] [blame] | 479 | Handle buffer requests for an exporter that wants to expose *buf* of size *len* |
| 480 | with writability set according to *readonly*. *buf* is interpreted as a sequence |
| 481 | of unsigned bytes. |
| 482 | |
| 483 | The *flags* argument indicates the request type. This function always fills in |
| 484 | *view* as specified by flags, unless *buf* has been designated as read-only |
| 485 | and :c:macro:`PyBUF_WRITABLE` is set in *flags*. |
| 486 | |
| 487 | On success, set :c:member:`view->obj` to a new reference to *exporter* and |
| 488 | return 0. Otherwise, raise :c:data:`PyExc_BufferError`, set |
| 489 | :c:member:`view->obj` to *NULL* and return -1; |
| 490 | |
| 491 | If this function is used as part of a :ref:`getbufferproc <buffer-structs>`, |
| 492 | *exporter* MUST be set to the exporting object. Otherwise, *exporter* MUST |
| 493 | be NULL. |
| 494 | |
| 495 | |
Benjamin Peterson | 9d0ced3 | 2008-09-16 02:24:31 +0000 | [diff] [blame] | 496 | |