blob: f3cd261f132cb93e9034a526cc8e185a852e4c70 [file] [log] [blame]
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +00001:mod:`io` --- Core tools for working with streams
2=================================================
3
4.. module:: io
5 :synopsis: Core tools for working with streams.
6.. moduleauthor:: Guido van Rossum <guido@python.org>
7.. moduleauthor:: Mike Verdone <mike.verdone@gmail.com>
8.. moduleauthor:: Mark Russell <mark.russell@zen.co.uk>
Antoine Pitrouc9062ca2009-10-01 17:08:03 +00009.. moduleauthor:: Antoine Pitrou <solipsis@pitrou.net>
10.. moduleauthor:: Amaury Forgeot d'Arc <amauryfa@gmail.com>
11.. moduleauthor:: Benjamin Peterson <benjamin@python.org>
Benjamin Peterson51a37032009-01-11 19:48:15 +000012.. sectionauthor:: Benjamin Peterson <benjamin@python.org>
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +000013
Benjamin Peterson183515c2010-08-28 20:56:50 +000014.. versionadded:: 2.6
15
Antoine Pitrouc9062ca2009-10-01 17:08:03 +000016The :mod:`io` module provides the Python interfaces to stream handling.
17Under Python 2.x, this is proposed as an alternative to the built-in
18:class:`file` object, but in Python 3.x it is the default interface to
19access files and streams.
20
21.. note::
22
23 Since this module has been designed primarily for Python 3.x, you have to
24 be aware that all uses of "bytes" in this document refer to the
25 :class:`str` type (of which :class:`bytes` is an alias), and all uses
26 of "text" refer to the :class:`unicode` type. Furthermore, those two
27 types are not interchangeable in the :mod:`io` APIs.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +000028
29At the top of the I/O hierarchy is the abstract base class :class:`IOBase`. It
30defines the basic interface to a stream. Note, however, that there is no
Mark Dickinson3e4caeb2009-02-21 20:27:01 +000031separation between reading and writing to streams; implementations are allowed
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +000032to throw an :exc:`IOError` if they do not support a given operation.
33
34Extending :class:`IOBase` is :class:`RawIOBase` which deals simply with the
35reading and writing of raw bytes to a stream. :class:`FileIO` subclasses
Benjamin Petersonad9f6292008-04-21 11:57:40 +000036:class:`RawIOBase` to provide an interface to files in the machine's
37file system.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +000038
39:class:`BufferedIOBase` deals with buffering on a raw byte stream
40(:class:`RawIOBase`). Its subclasses, :class:`BufferedWriter`,
41:class:`BufferedReader`, and :class:`BufferedRWPair` buffer streams that are
Benjamin Petersonad9f6292008-04-21 11:57:40 +000042readable, writable, and both readable and writable.
43:class:`BufferedRandom` provides a buffered interface to random access
44streams. :class:`BytesIO` is a simple stream of in-memory bytes.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +000045
Benjamin Petersonad9f6292008-04-21 11:57:40 +000046Another :class:`IOBase` subclass, :class:`TextIOBase`, deals with
47streams whose bytes represent text, and handles encoding and decoding
Antoine Pitrouc9062ca2009-10-01 17:08:03 +000048from and to :class:`unicode` strings. :class:`TextIOWrapper`, which extends
49it, is a buffered text interface to a buffered raw stream
Benjamin Petersonad9f6292008-04-21 11:57:40 +000050(:class:`BufferedIOBase`). Finally, :class:`StringIO` is an in-memory
Antoine Pitrouc9062ca2009-10-01 17:08:03 +000051stream for unicode text.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +000052
53Argument names are not part of the specification, and only the arguments of
Georg Brandl9fa61bb2009-07-26 14:19:57 +000054:func:`.open` are intended to be used as keyword arguments.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +000055
56
57Module Interface
58----------------
59
60.. data:: DEFAULT_BUFFER_SIZE
61
62 An int containing the default buffer size used by the module's buffered I/O
Georg Brandl9fa61bb2009-07-26 14:19:57 +000063 classes. :func:`.open` uses the file's blksize (as obtained by
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +000064 :func:`os.stat`) if possible.
65
Benjamin Petersona9bd6d52010-04-27 21:01:54 +000066.. function:: open(file, mode='r', buffering=-1, encoding=None, errors=None, newline=None, closefd=True)
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +000067
Antoine Pitrouc9062ca2009-10-01 17:08:03 +000068 Open *file* and return a corresponding stream. If the file cannot be opened,
69 an :exc:`IOError` is raised.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +000070
Georg Brandlb917af82010-08-01 21:33:42 +000071 *file* is either a string giving the pathname (absolute or
72 relative to the current working directory) of the file to be opened or
73 an integer file descriptor of the file to be wrapped. (If a file descriptor
74 is given, it is closed when the returned I/O object is closed, unless
75 *closefd* is set to ``False``.)
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +000076
77 *mode* is an optional string that specifies the mode in which the file is
78 opened. It defaults to ``'r'`` which means open for reading in text mode.
79 Other common values are ``'w'`` for writing (truncating the file if it
80 already exists), and ``'a'`` for appending (which on *some* Unix systems,
81 means that *all* writes append to the end of the file regardless of the
82 current seek position). In text mode, if *encoding* is not specified the
83 encoding used is platform dependent. (For reading and writing raw bytes use
84 binary mode and leave *encoding* unspecified.) The available modes are:
85
86 ========= ===============================================================
87 Character Meaning
88 --------- ---------------------------------------------------------------
89 ``'r'`` open for reading (default)
90 ``'w'`` open for writing, truncating the file first
91 ``'a'`` open for writing, appending to the end of the file if it exists
92 ``'b'`` binary mode
93 ``'t'`` text mode (default)
94 ``'+'`` open a disk file for updating (reading and writing)
Benjamin Petersonad9f6292008-04-21 11:57:40 +000095 ``'U'`` universal newline mode (for backwards compatibility; should
96 not be used in new code)
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +000097 ========= ===============================================================
98
99 The default mode is ``'rt'`` (open for reading text). For binary random
100 access, the mode ``'w+b'`` opens and truncates the file to 0 bytes, while
101 ``'r+b'`` opens the file without truncation.
102
103 Python distinguishes between files opened in binary and text modes, even when
104 the underlying operating system doesn't. Files opened in binary mode
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000105 (including ``'b'`` in the *mode* argument) return contents as :class:`bytes`
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000106 objects without any decoding. In text mode (the default, or when ``'t'`` is
Benjamin Petersonad9f6292008-04-21 11:57:40 +0000107 included in the *mode* argument), the contents of the file are returned as
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000108 :class:`unicode` strings, the bytes having been first decoded using a
109 platform-dependent encoding or using the specified *encoding* if given.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000110
Antoine Pitroue812d292009-12-19 21:01:10 +0000111 *buffering* is an optional integer used to set the buffering policy.
112 Pass 0 to switch buffering off (only allowed in binary mode), 1 to select
113 line buffering (only usable in text mode), and an integer > 1 to indicate
114 the size of a fixed-size chunk buffer. When no *buffering* argument is
115 given, the default buffering policy works as follows:
116
117 * Binary files are buffered in fixed-size chunks; the size of the buffer
118 is chosen using a heuristic trying to determine the underlying device's
119 "block size" and falling back on :attr:`DEFAULT_BUFFER_SIZE`.
120 On many systems, the buffer will typically be 4096 or 8192 bytes long.
121
122 * "Interactive" text files (files for which :meth:`isatty` returns True)
123 use line buffering. Other text files use the policy described above
124 for binary files.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000125
126 *encoding* is the name of the encoding used to decode or encode the file.
127 This should only be used in text mode. The default encoding is platform
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000128 dependent (whatever :func:`locale.getpreferredencoding` returns), but any
129 encoding supported by Python can be used. See the :mod:`codecs` module for
130 the list of supported encodings.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000131
Benjamin Peterson53be57e2008-04-19 19:34:05 +0000132 *errors* is an optional string that specifies how encoding and decoding
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000133 errors are to be handled--this cannot be used in binary mode. Pass
134 ``'strict'`` to raise a :exc:`ValueError` exception if there is an encoding
135 error (the default of ``None`` has the same effect), or pass ``'ignore'`` to
136 ignore errors. (Note that ignoring encoding errors can lead to data loss.)
137 ``'replace'`` causes a replacement marker (such as ``'?'``) to be inserted
138 where there is malformed data. When writing, ``'xmlcharrefreplace'``
139 (replace with the appropriate XML character reference) or
140 ``'backslashreplace'`` (replace with backslashed escape sequences) can be
141 used. Any other error handling name that has been registered with
142 :func:`codecs.register_error` is also valid.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000143
144 *newline* controls how universal newlines works (it only applies to text
145 mode). It can be ``None``, ``''``, ``'\n'``, ``'\r'``, and ``'\r\n'``. It
146 works as follows:
147
148 * On input, if *newline* is ``None``, universal newlines mode is enabled.
149 Lines in the input can end in ``'\n'``, ``'\r'``, or ``'\r\n'``, and these
150 are translated into ``'\n'`` before being returned to the caller. If it is
151 ``''``, universal newline mode is enabled, but line endings are returned to
152 the caller untranslated. If it has any of the other legal values, input
153 lines are only terminated by the given string, and the line ending is
154 returned to the caller untranslated.
155
156 * On output, if *newline* is ``None``, any ``'\n'`` characters written are
157 translated to the system default line separator, :data:`os.linesep`. If
158 *newline* is ``''``, no translation takes place. If *newline* is any of
159 the other legal values, any ``'\n'`` characters written are translated to
160 the given string.
161
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000162 If *closefd* is ``False`` and a file descriptor rather than a filename was
163 given, the underlying file descriptor will be kept open when the file is
164 closed. If a filename is given *closefd* has no effect and must be ``True``
165 (the default).
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000166
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000167 The type of file object returned by the :func:`.open` function depends on the
168 mode. When :func:`.open` is used to open a file in a text mode (``'w'``,
169 ``'r'``, ``'wt'``, ``'rt'``, etc.), it returns a subclass of
170 :class:`TextIOBase` (specifically :class:`TextIOWrapper`). When used to open
171 a file in a binary mode with buffering, the returned class is a subclass of
172 :class:`BufferedIOBase`. The exact class varies: in read binary mode, it
173 returns a :class:`BufferedReader`; in write binary and append binary modes,
174 it returns a :class:`BufferedWriter`, and in read/write mode, it returns a
175 :class:`BufferedRandom`. When buffering is disabled, the raw stream, a
176 subclass of :class:`RawIOBase`, :class:`FileIO`, is returned.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000177
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000178 It is also possible to use an :class:`unicode` or :class:`bytes` string
179 as a file for both reading and writing. For :class:`unicode` strings
180 :class:`StringIO` can be used like a file opened in text mode,
181 and for :class:`bytes` a :class:`BytesIO` can be used like a
Benjamin Petersonad9f6292008-04-21 11:57:40 +0000182 file opened in a binary mode.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000183
184
185.. exception:: BlockingIOError
186
187 Error raised when blocking would occur on a non-blocking stream. It inherits
188 :exc:`IOError`.
189
190 In addition to those of :exc:`IOError`, :exc:`BlockingIOError` has one
191 attribute:
192
193 .. attribute:: characters_written
194
195 An integer containing the number of characters written to the stream
196 before it blocked.
197
198
199.. exception:: UnsupportedOperation
200
201 An exception inheriting :exc:`IOError` and :exc:`ValueError` that is raised
202 when an unsupported operation is called on a stream.
203
204
205I/O Base Classes
206----------------
207
208.. class:: IOBase
209
210 The abstract base class for all I/O classes, acting on streams of bytes.
211 There is no public constructor.
212
Benjamin Petersonad9f6292008-04-21 11:57:40 +0000213 This class provides empty abstract implementations for many methods
214 that derived classes can override selectively; the default
215 implementations represent a file that cannot be read, written or
216 seeked.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000217
218 Even though :class:`IOBase` does not declare :meth:`read`, :meth:`readinto`,
219 or :meth:`write` because their signatures will vary, implementations and
220 clients should consider those methods part of the interface. Also,
221 implementations may raise a :exc:`IOError` when operations they do not
222 support are called.
223
224 The basic type used for binary data read from or written to a file is
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000225 :class:`bytes` (also known as :class:`str`). :class:`bytearray`\s are
226 accepted too, and in some cases (such as :class:`readinto`) required.
227 Text I/O classes work with :class:`unicode` data.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000228
229 Note that calling any method (even inquiries) on a closed stream is
230 undefined. Implementations may raise :exc:`IOError` in this case.
231
232 IOBase (and its subclasses) support the iterator protocol, meaning that an
233 :class:`IOBase` object can be iterated over yielding the lines in a stream.
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000234 Lines are defined slightly differently depending on whether the stream is
235 a binary stream (yielding :class:`bytes`), or a text stream (yielding
236 :class:`unicode` strings). See :meth:`readline` below.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000237
Benjamin Petersonad9f6292008-04-21 11:57:40 +0000238 IOBase is also a context manager and therefore supports the
239 :keyword:`with` statement. In this example, *file* is closed after the
240 :keyword:`with` statement's suite is finished---even if an exception occurs::
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000241
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000242 with io.open('spam.txt', 'w') as file:
243 file.write(u'Spam and eggs!')
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000244
Benjamin Petersonad9f6292008-04-21 11:57:40 +0000245 :class:`IOBase` provides these data attributes and methods:
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000246
247 .. method:: close()
248
Amaury Forgeot d'Arc32265652008-11-20 23:34:31 +0000249 Flush and close this stream. This method has no effect if the file is
Georg Brandlc62ef8b2009-01-03 20:55:06 +0000250 already closed. Once the file is closed, any operation on the file
Georg Brandl64879522010-05-10 21:51:33 +0000251 (e.g. reading or writing) will raise a :exc:`ValueError`.
Antoine Pitrou689405e2010-04-28 19:57:33 +0000252
253 As a convenience, it is allowed to call this method more than once;
254 only the first call, however, will have an effect.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000255
256 .. attribute:: closed
257
258 True if the stream is closed.
259
260 .. method:: fileno()
261
Benjamin Peterson3c399d12008-04-22 02:16:03 +0000262 Return the underlying file descriptor (an integer) of the stream if it
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000263 exists. An :exc:`IOError` is raised if the IO object does not use a file
264 descriptor.
265
266 .. method:: flush()
267
Benjamin Peterson53be57e2008-04-19 19:34:05 +0000268 Flush the write buffers of the stream if applicable. This does nothing
269 for read-only and non-blocking streams.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000270
271 .. method:: isatty()
272
Benjamin Peterson3c399d12008-04-22 02:16:03 +0000273 Return ``True`` if the stream is interactive (i.e., connected to
Benjamin Petersonad9f6292008-04-21 11:57:40 +0000274 a terminal/tty device).
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000275
276 .. method:: readable()
277
Benjamin Peterson3c399d12008-04-22 02:16:03 +0000278 Return ``True`` if the stream can be read from. If False, :meth:`read`
279 will raise :exc:`IOError`.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000280
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000281 .. method:: readline(limit=-1)
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000282
Benjamin Peterson3c399d12008-04-22 02:16:03 +0000283 Read and return one line from the stream. If *limit* is specified, at
284 most *limit* bytes will be read.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000285
286 The line terminator is always ``b'\n'`` for binary files; for text files,
Georg Brandl9fa61bb2009-07-26 14:19:57 +0000287 the *newlines* argument to :func:`.open` can be used to select the line
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000288 terminator(s) recognized.
289
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000290 .. method:: readlines(hint=-1)
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000291
Benjamin Peterson3c399d12008-04-22 02:16:03 +0000292 Read and return a list of lines from the stream. *hint* can be specified
293 to control the number of lines read: no more lines will be read if the
294 total size (in bytes/characters) of all lines so far exceeds *hint*.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000295
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000296 .. method:: seek(offset, whence=SEEK_SET)
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000297
Benjamin Petersonad9f6292008-04-21 11:57:40 +0000298 Change the stream position to the given byte *offset*. *offset* is
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000299 interpreted relative to the position indicated by *whence*. Values for
300 *whence* are:
301
Georg Brandl88ed8f22009-04-01 21:00:55 +0000302 * :data:`SEEK_SET` or ``0`` -- start of the stream (the default);
303 *offset* should be zero or positive
304 * :data:`SEEK_CUR` or ``1`` -- current stream position; *offset* may
305 be negative
306 * :data:`SEEK_END` or ``2`` -- end of the stream; *offset* is usually
307 negative
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000308
Benjamin Peterson3c399d12008-04-22 02:16:03 +0000309 Return the new absolute position.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000310
Georg Brandl88ed8f22009-04-01 21:00:55 +0000311 .. versionadded:: 2.7
312 The ``SEEK_*`` constants
313
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000314 .. method:: seekable()
315
Benjamin Peterson3c399d12008-04-22 02:16:03 +0000316 Return ``True`` if the stream supports random access. If ``False``,
317 :meth:`seek`, :meth:`tell` and :meth:`truncate` will raise :exc:`IOError`.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000318
319 .. method:: tell()
320
Benjamin Peterson3c399d12008-04-22 02:16:03 +0000321 Return the current stream position.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000322
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000323 .. method:: truncate(size=None)
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000324
Antoine Pitrou9fe61992010-05-29 12:06:13 +0000325 Resize the stream to the given *size* in bytes (or the current position
326 if *size* is not specified). The current stream position isn't changed.
327 This resizing can extend or reduce the current file size. In case of
328 extension, the contents of the new file area depend on the platform
329 (on most systems, additional bytes are zero-filled, on Windows they're
330 undetermined). The new file size is returned.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000331
332 .. method:: writable()
333
Benjamin Peterson3c399d12008-04-22 02:16:03 +0000334 Return ``True`` if the stream supports writing. If ``False``,
Benjamin Petersonad9f6292008-04-21 11:57:40 +0000335 :meth:`write` and :meth:`truncate` will raise :exc:`IOError`.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000336
337 .. method:: writelines(lines)
338
Benjamin Peterson3c399d12008-04-22 02:16:03 +0000339 Write a list of lines to the stream. Line separators are not added, so it
340 is usual for each of the lines provided to have a line separator at the
341 end.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000342
343
344.. class:: RawIOBase
345
346 Base class for raw binary I/O. It inherits :class:`IOBase`. There is no
347 public constructor.
348
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000349 Raw binary I/O typically provides low-level access to an underlying OS
350 device or API, and does not try to encapsulate it in high-level primitives
351 (this is left to Buffered I/O and Text I/O, described later in this page).
352
Benjamin Petersonad9f6292008-04-21 11:57:40 +0000353 In addition to the attributes and methods from :class:`IOBase`,
354 RawIOBase provides the following methods:
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000355
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000356 .. method:: read(n=-1)
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000357
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000358 Read up to *n* bytes from the object and return them. As a convenience,
359 if *n* is unspecified or -1, :meth:`readall` is called. Otherwise,
360 only one system call is ever made. Fewer than *n* bytes may be
361 returned if the operating system call returns fewer than *n* bytes.
362
363 If 0 bytes are returned, and *n* was not 0, this indicates end of file.
364 If the object is in non-blocking mode and no bytes are available,
365 ``None`` is returned.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000366
367 .. method:: readall()
368
Benjamin Peterson3c399d12008-04-22 02:16:03 +0000369 Read and return all the bytes from the stream until EOF, using multiple
370 calls to the stream if necessary.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000371
372 .. method:: readinto(b)
373
Benjamin Peterson3c399d12008-04-22 02:16:03 +0000374 Read up to len(b) bytes into bytearray *b* and return the number of bytes
375 read.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000376
377 .. method:: write(b)
378
Benjamin Peterson3c399d12008-04-22 02:16:03 +0000379 Write the given bytes or bytearray object, *b*, to the underlying raw
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000380 stream and return the number of bytes written. This can be less than
381 ``len(b)``, depending on specifics of the underlying raw stream, and
382 especially if it is in non-blocking mode. ``None`` is returned if the
383 raw stream is set not to block and no single byte could be readily
384 written to it.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000385
386
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000387.. class:: BufferedIOBase
388
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000389 Base class for binary streams that support some kind of buffering.
390 It inherits :class:`IOBase`. There is no public constructor.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000391
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000392 The main difference with :class:`RawIOBase` is that methods :meth:`read`,
393 :meth:`readinto` and :meth:`write` will try (respectively) to read as much
394 input as requested or to consume all given output, at the expense of
395 making perhaps more than one system call.
396
397 In addition, those methods can raise :exc:`BlockingIOError` if the
398 underlying raw stream is in non-blocking mode and cannot take or give
399 enough data; unlike their :class:`RawIOBase` counterparts, they will
400 never return ``None``.
401
402 Besides, the :meth:`read` method does not have a default
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000403 implementation that defers to :meth:`readinto`.
404
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000405 A typical :class:`BufferedIOBase` implementation should not inherit from a
406 :class:`RawIOBase` implementation, but wrap one, like
407 :class:`BufferedWriter` and :class:`BufferedReader` do.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000408
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000409 :class:`BufferedIOBase` provides or overrides these members in addition to
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000410 those from :class:`IOBase`:
411
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000412 .. attribute:: raw
413
414 The underlying raw stream (a :class:`RawIOBase` instance) that
415 :class:`BufferedIOBase` deals with. This is not part of the
416 :class:`BufferedIOBase` API and may not exist on some implementations.
417
418 .. method:: detach()
419
420 Separate the underlying raw stream from the buffer and return it.
421
422 After the raw stream has been detached, the buffer is in an unusable
423 state.
424
425 Some buffers, like :class:`BytesIO`, do not have the concept of a single
426 raw stream to return from this method. They raise
427 :exc:`UnsupportedOperation`.
428
429 .. versionadded:: 2.7
430
431 .. method:: read(n=-1)
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000432
Benjamin Peterson3c399d12008-04-22 02:16:03 +0000433 Read and return up to *n* bytes. If the argument is omitted, ``None``, or
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000434 negative, data is read and returned until EOF is reached. An empty bytes
435 object is returned if the stream is already at EOF.
436
437 If the argument is positive, and the underlying raw stream is not
438 interactive, multiple raw reads may be issued to satisfy the byte count
439 (unless EOF is reached first). But for interactive raw streams, at most
440 one raw read will be issued, and a short result does not imply that EOF is
441 imminent.
442
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000443 A :exc:`BlockingIOError` is raised if the underlying raw stream is in
444 non blocking-mode, and has no data available at the moment.
445
446 .. method:: read1(n=-1)
447
448 Read and return up to *n* bytes, with at most one call to the underlying
449 raw stream's :meth:`~RawIOBase.read` method. This can be useful if you
450 are implementing your own buffering on top of a :class:`BufferedIOBase`
451 object.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000452
453 .. method:: readinto(b)
454
Benjamin Peterson3c399d12008-04-22 02:16:03 +0000455 Read up to len(b) bytes into bytearray *b* and return the number of bytes
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000456 read.
457
458 Like :meth:`read`, multiple reads may be issued to the underlying raw
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000459 stream, unless the latter is 'interactive'.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000460
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000461 A :exc:`BlockingIOError` is raised if the underlying raw stream is in
462 non blocking-mode, and has no data available at the moment.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000463
464 .. method:: write(b)
465
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000466 Write the given bytes or bytearray object, *b* and return the number
467 of bytes written (never less than ``len(b)``, since if the write fails
468 an :exc:`IOError` will be raised). Depending on the actual
469 implementation, these bytes may be readily written to the underlying
470 stream, or held in a buffer for performance and latency reasons.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000471
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000472 When in non-blocking mode, a :exc:`BlockingIOError` is raised if the
473 data needed to be written to the raw stream but it couldn't accept
474 all the data without blocking.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000475
476
Benjamin Petersonb6c7beb2009-01-19 16:17:54 +0000477Raw File I/O
478------------
479
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000480.. class:: FileIO(name, mode='r', closefd=True)
Benjamin Petersonb6c7beb2009-01-19 16:17:54 +0000481
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000482 :class:`FileIO` represents an OS-level file containing bytes data.
483 It implements the :class:`RawIOBase` interface (and therefore the
484 :class:`IOBase` interface, too).
485
486 The *name* can be one of two things:
487
488 * a string representing the path to the file which will be opened;
489 * an integer representing the number of an existing OS-level file descriptor
490 to which the resulting :class:`FileIO` object will give access.
Benjamin Petersonb6c7beb2009-01-19 16:17:54 +0000491
492 The *mode* can be ``'r'``, ``'w'`` or ``'a'`` for reading (default), writing,
493 or appending. The file will be created if it doesn't exist when opened for
494 writing or appending; it will be truncated when opened for writing. Add a
495 ``'+'`` to the mode to allow simultaneous reading and writing.
496
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000497 The :meth:`read` (when called with a positive argument), :meth:`readinto`
498 and :meth:`write` methods on this class will only make one system call.
499
Benjamin Petersonb6c7beb2009-01-19 16:17:54 +0000500 In addition to the attributes and methods from :class:`IOBase` and
501 :class:`RawIOBase`, :class:`FileIO` provides the following data
502 attributes and methods:
503
504 .. attribute:: mode
505
506 The mode as given in the constructor.
507
508 .. attribute:: name
509
510 The file name. This is the file descriptor of the file when no name is
511 given in the constructor.
512
Benjamin Petersonb6c7beb2009-01-19 16:17:54 +0000513
514Buffered Streams
515----------------
516
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000517In many situations, buffered I/O streams will provide higher performance
518(bandwidth and latency) than raw I/O streams. Their API is also more usable.
519
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000520.. class:: BytesIO([initial_bytes])
521
522 A stream implementation using an in-memory bytes buffer. It inherits
523 :class:`BufferedIOBase`.
524
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000525 The argument *initial_bytes* is an optional initial :class:`bytes`.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000526
527 :class:`BytesIO` provides or overrides these methods in addition to those
528 from :class:`BufferedIOBase` and :class:`IOBase`:
529
530 .. method:: getvalue()
531
Benjamin Peterson3c399d12008-04-22 02:16:03 +0000532 Return ``bytes`` containing the entire contents of the buffer.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000533
534 .. method:: read1()
535
Benjamin Peterson53be57e2008-04-19 19:34:05 +0000536 In :class:`BytesIO`, this is the same as :meth:`read`.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000537
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000538
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000539.. class:: BufferedReader(raw, buffer_size=DEFAULT_BUFFER_SIZE)
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000540
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000541 A buffer providing higher-level access to a readable, sequential
542 :class:`RawIOBase` object. It inherits :class:`BufferedIOBase`.
543 When reading data from this object, a larger amount of data may be
544 requested from the underlying raw stream, and kept in an internal buffer.
545 The buffered data can then be returned directly on subsequent reads.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000546
547 The constructor creates a :class:`BufferedReader` for the given readable
548 *raw* stream and *buffer_size*. If *buffer_size* is omitted,
549 :data:`DEFAULT_BUFFER_SIZE` is used.
550
551 :class:`BufferedReader` provides or overrides these methods in addition to
552 those from :class:`BufferedIOBase` and :class:`IOBase`:
553
554 .. method:: peek([n])
555
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000556 Return bytes from the stream without advancing the position. At most one
557 single read on the raw stream is done to satisfy the call. The number of
558 bytes returned may be less or more than requested.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000559
560 .. method:: read([n])
561
Benjamin Peterson3c399d12008-04-22 02:16:03 +0000562 Read and return *n* bytes, or if *n* is not given or negative, until EOF
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000563 or if the read call would block in non-blocking mode.
564
565 .. method:: read1(n)
566
Benjamin Peterson3c399d12008-04-22 02:16:03 +0000567 Read and return up to *n* bytes with only one call on the raw stream. If
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000568 at least one byte is buffered, only buffered bytes are returned.
569 Otherwise, one raw stream read call is made.
570
571
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000572.. class:: BufferedWriter(raw, buffer_size=DEFAULT_BUFFER_SIZE)
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000573
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000574 A buffer providing higher-level access to a writeable, sequential
575 :class:`RawIOBase` object. It inherits :class:`BufferedIOBase`.
576 When writing to this object, data is normally held into an internal
577 buffer. The buffer will be written out to the underlying :class:`RawIOBase`
578 object under various conditions, including:
579
580 * when the buffer gets too small for all pending data;
581 * when :meth:`flush()` is called;
582 * when a :meth:`seek()` is requested (for :class:`BufferedRandom` objects);
583 * when the :class:`BufferedWriter` object is closed or destroyed.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000584
585 The constructor creates a :class:`BufferedWriter` for the given writeable
586 *raw* stream. If the *buffer_size* is not given, it defaults to
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000587 :data:`DEFAULT_BUFFER_SIZE`.
588
589 A third argument, *max_buffer_size*, is supported, but unused and deprecated.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000590
591 :class:`BufferedWriter` provides or overrides these methods in addition to
592 those from :class:`BufferedIOBase` and :class:`IOBase`:
593
594 .. method:: flush()
595
596 Force bytes held in the buffer into the raw stream. A
Benjamin Petersonad9f6292008-04-21 11:57:40 +0000597 :exc:`BlockingIOError` should be raised if the raw stream blocks.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000598
599 .. method:: write(b)
600
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000601 Write the bytes or bytearray object, *b* and return the number of bytes
602 written. When in non-blocking mode, a :exc:`BlockingIOError` is raised
603 if the buffer needs to be written out but the raw stream blocks.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000604
605
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000606.. class:: BufferedRWPair(reader, writer, buffer_size=DEFAULT_BUFFER_SIZE)
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000607
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000608 A buffered I/O object giving a combined, higher-level access to two
609 sequential :class:`RawIOBase` objects: one readable, the other writeable.
610 It is useful for pairs of unidirectional communication channels
611 (pipes, for instance). It inherits :class:`BufferedIOBase`.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000612
613 *reader* and *writer* are :class:`RawIOBase` objects that are readable and
614 writeable respectively. If the *buffer_size* is omitted it defaults to
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000615 :data:`DEFAULT_BUFFER_SIZE`.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000616
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000617 A fourth argument, *max_buffer_size*, is supported, but unused and
618 deprecated.
619
620 :class:`BufferedRWPair` implements all of :class:`BufferedIOBase`\'s methods
621 except for :meth:`~BufferedIOBase.detach`, which raises
622 :exc:`UnsupportedOperation`.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000623
624
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000625.. class:: BufferedRandom(raw, buffer_size=DEFAULT_BUFFER_SIZE)
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000626
627 A buffered interface to random access streams. It inherits
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000628 :class:`BufferedReader` and :class:`BufferedWriter`, and further supports
629 :meth:`seek` and :meth:`tell` functionality.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000630
631 The constructor creates a reader and writer for a seekable raw stream, given
632 in the first argument. If the *buffer_size* is omitted it defaults to
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000633 :data:`DEFAULT_BUFFER_SIZE`.
634
635 A third argument, *max_buffer_size*, is supported, but unused and deprecated.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000636
637 :class:`BufferedRandom` is capable of anything :class:`BufferedReader` or
638 :class:`BufferedWriter` can do.
639
640
641Text I/O
642--------
643
644.. class:: TextIOBase
645
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000646 Base class for text streams. This class provides an unicode character
647 and line based interface to stream I/O. There is no :meth:`readinto`
648 method because Python's :class:`unicode` strings are immutable.
649 It inherits :class:`IOBase`. There is no public constructor.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000650
Benjamin Petersonad9f6292008-04-21 11:57:40 +0000651 :class:`TextIOBase` provides or overrides these data attributes and
652 methods in addition to those from :class:`IOBase`:
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000653
654 .. attribute:: encoding
655
Benjamin Petersonad9f6292008-04-21 11:57:40 +0000656 The name of the encoding used to decode the stream's bytes into
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000657 strings, and to encode strings into bytes.
658
Antoine Pitrou19690592009-06-12 20:14:08 +0000659 .. attribute:: errors
660
661 The error setting of the decoder or encoder.
662
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000663 .. attribute:: newlines
664
Benjamin Petersonad9f6292008-04-21 11:57:40 +0000665 A string, a tuple of strings, or ``None``, indicating the newlines
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000666 translated so far. Depending on the implementation and the initial
667 constructor flags, this may not be available.
668
669 .. attribute:: buffer
670
671 The underlying binary buffer (a :class:`BufferedIOBase` instance) that
672 :class:`TextIOBase` deals with. This is not part of the
673 :class:`TextIOBase` API and may not exist on some implementations.
674
675 .. method:: detach()
676
677 Separate the underlying binary buffer from the :class:`TextIOBase` and
678 return it.
679
680 After the underlying buffer has been detached, the :class:`TextIOBase` is
681 in an unusable state.
682
683 Some :class:`TextIOBase` implementations, like :class:`StringIO`, may not
684 have the concept of an underlying buffer and calling this method will
685 raise :exc:`UnsupportedOperation`.
686
687 .. versionadded:: 2.7
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000688
689 .. method:: read(n)
690
Benjamin Peterson3c399d12008-04-22 02:16:03 +0000691 Read and return at most *n* characters from the stream as a single
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000692 :class:`unicode`. If *n* is negative or ``None``, reads until EOF.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000693
694 .. method:: readline()
695
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000696 Read until newline or EOF and return a single ``unicode``. If the
697 stream is already at EOF, an empty string is returned.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000698
699 .. method:: write(s)
700
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000701 Write the :class:`unicode` string *s* to the stream and return the
702 number of characters written.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000703
704
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000705.. class:: TextIOWrapper(buffer, encoding=None, errors=None, newline=None, line_buffering=False)
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000706
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000707 A buffered text stream over a :class:`BufferedIOBase` binary stream.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000708 It inherits :class:`TextIOBase`.
709
710 *encoding* gives the name of the encoding that the stream will be decoded or
711 encoded with. It defaults to :func:`locale.getpreferredencoding`.
712
Benjamin Peterson53be57e2008-04-19 19:34:05 +0000713 *errors* is an optional string that specifies how encoding and decoding
714 errors are to be handled. Pass ``'strict'`` to raise a :exc:`ValueError`
715 exception if there is an encoding error (the default of ``None`` has the same
716 effect), or pass ``'ignore'`` to ignore errors. (Note that ignoring encoding
717 errors can lead to data loss.) ``'replace'`` causes a replacement marker
Benjamin Petersona7d09032008-04-19 19:47:34 +0000718 (such as ``'?'``) to be inserted where there is malformed data. When
719 writing, ``'xmlcharrefreplace'`` (replace with the appropriate XML character
720 reference) or ``'backslashreplace'`` (replace with backslashed escape
721 sequences) can be used. Any other error handling name that has been
722 registered with :func:`codecs.register_error` is also valid.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000723
724 *newline* can be ``None``, ``''``, ``'\n'``, ``'\r'``, or ``'\r\n'``. It
725 controls the handling of line endings. If it is ``None``, universal newlines
726 is enabled. With this enabled, on input, the lines endings ``'\n'``,
727 ``'\r'``, or ``'\r\n'`` are translated to ``'\n'`` before being returned to
728 the caller. Conversely, on output, ``'\n'`` is translated to the system
Mark Dickinson3e4caeb2009-02-21 20:27:01 +0000729 default line separator, :data:`os.linesep`. If *newline* is any other of its
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000730 legal values, that newline becomes the newline when the file is read and it
731 is returned untranslated. On output, ``'\n'`` is converted to the *newline*.
732
733 If *line_buffering* is ``True``, :meth:`flush` is implied when a call to
734 write contains a newline character.
735
Antoine Pitrou19690592009-06-12 20:14:08 +0000736 :class:`TextIOWrapper` provides one attribute in addition to those of
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000737 :class:`TextIOBase` and its parents:
738
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000739 .. attribute:: line_buffering
740
741 Whether line buffering is enabled.
Georg Brandlc62ef8b2009-01-03 20:55:06 +0000742
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000743
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000744.. class:: StringIO(initial_value=u'', newline=None)
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000745
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000746 An in-memory stream for unicode text. It inherits :class:`TextIOWrapper`.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000747
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000748 The initial value of the buffer (an empty unicode string by default) can
749 be set by providing *initial_value*. The *newline* argument works like
750 that of :class:`TextIOWrapper`. The default is to do no newline
751 translation.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000752
Benjamin Petersonad9f6292008-04-21 11:57:40 +0000753 :class:`StringIO` provides this method in addition to those from
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000754 :class:`TextIOWrapper` and its parents:
755
756 .. method:: getvalue()
757
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000758 Return a ``unicode`` containing the entire contents of the buffer at any
759 time before the :class:`StringIO` object's :meth:`close` method is
760 called.
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000761
Antoine Pitrouc9062ca2009-10-01 17:08:03 +0000762 Example usage::
763
764 import io
765
766 output = io.StringIO()
767 output.write(u'First line.\n')
768 output.write(u'Second line.\n')
769
770 # Retrieve file contents -- this will be
771 # u'First line.\nSecond line.\n'
772 contents = output.getvalue()
773
774 # Close object and discard memory buffer --
775 # .getvalue() will now raise an exception.
776 output.close()
Benjamin Peterson7bb4d2d2008-04-13 02:01:27 +0000777
778.. class:: IncrementalNewlineDecoder
779
780 A helper codec that decodes newlines for universal newlines mode. It
781 inherits :class:`codecs.IncrementalDecoder`.
782