blob: 9c6b72237177d19e760b3c42d685711ce4b7c200 [file] [log] [blame]
Georg Brandl116aa622007-08-15 14:28:22 +00001:mod:`gzip` --- Support for :program:`gzip` files
2=================================================
3
4.. module:: gzip
5 :synopsis: Interfaces for gzip compression and decompression using file objects.
6
Raymond Hettinger469271d2011-01-27 20:38:46 +00007**Source code:** :source:`Lib/gzip.py`
8
9--------------
10
Christian Heimesbbe741d2008-03-28 10:53:29 +000011This module provides a simple interface to compress and decompress files just
12like the GNU programs :program:`gzip` and :program:`gunzip` would.
Georg Brandl116aa622007-08-15 14:28:22 +000013
Georg Brandl1f01deb2009-01-03 22:47:39 +000014The data compression is provided by the :mod:`zlib` module.
Christian Heimesbbe741d2008-03-28 10:53:29 +000015
Nadeem Vawda7e126202012-05-06 15:04:01 +020016The :mod:`gzip` module provides the :class:`GzipFile` class, as well as the
Nadeem Vawda68721012012-06-04 23:21:38 +020017:func:`.open`, :func:`compress` and :func:`decompress` convenience functions.
18The :class:`GzipFile` class reads and writes :program:`gzip`\ -format files,
19automatically compressing or decompressing the data so that it looks like an
20ordinary :term:`file object`.
Christian Heimesbbe741d2008-03-28 10:53:29 +000021
22Note that additional file formats which can be decompressed by the
23:program:`gzip` and :program:`gunzip` programs, such as those produced by
24:program:`compress` and :program:`pack`, are not supported by this module.
Georg Brandl116aa622007-08-15 14:28:22 +000025
Georg Brandl116aa622007-08-15 14:28:22 +000026The module defines the following items:
27
28
Nadeem Vawda7e126202012-05-06 15:04:01 +020029.. function:: open(filename, mode='rb', compresslevel=9, encoding=None, errors=None, newline=None)
30
Nadeem Vawda68721012012-06-04 23:21:38 +020031 Open a gzip-compressed file in binary or text mode, returning a :term:`file
32 object`.
Nadeem Vawda7e126202012-05-06 15:04:01 +020033
Nadeem Vawda68721012012-06-04 23:21:38 +020034 The *filename* argument can be an actual filename (a :class:`str` or
35 :class:`bytes` object), or an existing file object to read from or write to.
Nadeem Vawda7e126202012-05-06 15:04:01 +020036
37 The *mode* argument can be any of ``'r'``, ``'rb'``, ``'a'``, ``'ab'``,
Nadeem Vawdaee1be992013-10-19 00:11:13 +020038 ``'w'``, ``'wb'``, ``'x'`` or ``'xb'`` for binary mode, or ``'rt'``,
39 ``'at'``, ``'wt'``, or ``'xt'`` for text mode. The default is ``'rb'``.
Nadeem Vawda7e126202012-05-06 15:04:01 +020040
Nadeem Vawda6ff262e2012-11-11 14:14:47 +010041 The *compresslevel* argument is an integer from 0 to 9, as for the
Nadeem Vawda7e126202012-05-06 15:04:01 +020042 :class:`GzipFile` constructor.
43
44 For binary mode, this function is equivalent to the :class:`GzipFile`
45 constructor: ``GzipFile(filename, mode, compresslevel)``. In this case, the
46 *encoding*, *errors* and *newline* arguments must not be provided.
47
48 For text mode, a :class:`GzipFile` object is created, and wrapped in an
49 :class:`io.TextIOWrapper` instance with the specified encoding, error
50 handling behavior, and line ending(s).
51
52 .. versionchanged:: 3.3
Nadeem Vawda68721012012-06-04 23:21:38 +020053 Added support for *filename* being a file object, support for text mode,
54 and the *encoding*, *errors* and *newline* arguments.
Nadeem Vawda7e126202012-05-06 15:04:01 +020055
Nadeem Vawdaee1be992013-10-19 00:11:13 +020056 .. versionchanged:: 3.4
57 Added support for the ``'x'``, ``'xb'`` and ``'xt'`` modes.
58
Berker Peksag03020cf2016-10-02 13:47:58 +030059 .. versionchanged:: 3.6
60 Accepts a :term:`path-like object`.
Nadeem Vawda7e126202012-05-06 15:04:01 +020061
Georg Brandl036490d2009-05-17 13:00:36 +000062.. class:: GzipFile(filename=None, mode=None, compresslevel=9, fileobj=None, mtime=None)
Georg Brandl116aa622007-08-15 14:28:22 +000063
Antoine Pitrouc3ed2e72010-09-29 10:49:46 +000064 Constructor for the :class:`GzipFile` class, which simulates most of the
65 methods of a :term:`file object`, with the exception of the :meth:`truncate`
66 method. At least one of *fileobj* and *filename* must be given a non-trivial
67 value.
Georg Brandl116aa622007-08-15 14:28:22 +000068
Serhiy Storchakad65c9492015-11-02 14:10:23 +020069 The new class instance is based on *fileobj*, which can be a regular file, an
Serhiy Storchakae79be872013-08-17 00:09:55 +030070 :class:`io.BytesIO` object, or any other object which simulates a file. It
Georg Brandl116aa622007-08-15 14:28:22 +000071 defaults to ``None``, in which case *filename* is opened to provide a file
72 object.
73
74 When *fileobj* is not ``None``, the *filename* argument is only used to be
Georg Brandlf27bfd82013-10-06 12:33:20 +020075 included in the :program:`gzip` file header, which may include the original
Georg Brandl116aa622007-08-15 14:28:22 +000076 filename of the uncompressed file. It defaults to the filename of *fileobj*, if
77 discernible; otherwise, it defaults to the empty string, and in this case the
78 original filename is not included in the header.
79
80 The *mode* argument can be any of ``'r'``, ``'rb'``, ``'a'``, ``'ab'``, ``'w'``,
Nadeem Vawdaee1be992013-10-19 00:11:13 +020081 ``'wb'``, ``'x'``, or ``'xb'``, depending on whether the file will be read or
82 written. The default is the mode of *fileobj* if discernible; otherwise, the
83 default is ``'rb'``.
Nadeem Vawda30d94b72012-02-11 23:45:10 +020084
Nadeem Vawda7e126202012-05-06 15:04:01 +020085 Note that the file is always opened in binary mode. To open a compressed file
Nadeem Vawda68721012012-06-04 23:21:38 +020086 in text mode, use :func:`.open` (or wrap your :class:`GzipFile` with an
Nadeem Vawda7e126202012-05-06 15:04:01 +020087 :class:`io.TextIOWrapper`).
Georg Brandl116aa622007-08-15 14:28:22 +000088
Nadeem Vawda19e568d2012-11-11 14:04:14 +010089 The *compresslevel* argument is an integer from ``0`` to ``9`` controlling
90 the level of compression; ``1`` is fastest and produces the least
91 compression, and ``9`` is slowest and produces the most compression. ``0``
92 is no compression. The default is ``9``.
Georg Brandl116aa622007-08-15 14:28:22 +000093
Antoine Pitrou42db3ef2009-01-04 21:37:59 +000094 The *mtime* argument is an optional numeric timestamp to be written to
Antoine Pitrou2dbc6e62015-04-11 00:31:01 +020095 the last modification time field in the stream when compressing. It
96 should only be provided in compression mode. If omitted or ``None``, the
97 current time is used. See the :attr:`mtime` attribute for more details.
Antoine Pitrou42db3ef2009-01-04 21:37:59 +000098
Georg Brandl116aa622007-08-15 14:28:22 +000099 Calling a :class:`GzipFile` object's :meth:`close` method does not close
100 *fileobj*, since you might wish to append more material after the compressed
Martin Panter7462b6492015-11-02 03:37:02 +0000101 data. This also allows you to pass an :class:`io.BytesIO` object opened for
Georg Brandl116aa622007-08-15 14:28:22 +0000102 writing as *fileobj*, and retrieve the resulting memory buffer using the
Antoine Pitroue5768cf2010-09-23 16:45:17 +0000103 :class:`io.BytesIO` object's :meth:`~io.BytesIO.getvalue` method.
Georg Brandl116aa622007-08-15 14:28:22 +0000104
Antoine Pitrouc3ed2e72010-09-29 10:49:46 +0000105 :class:`GzipFile` supports the :class:`io.BufferedIOBase` interface,
106 including iteration and the :keyword:`with` statement. Only the
107 :meth:`truncate` method isn't implemented.
Benjamin Petersone0124bd2009-03-09 21:04:33 +0000108
Antoine Pitrou2dbc6e62015-04-11 00:31:01 +0200109 :class:`GzipFile` also provides the following method and attribute:
Antoine Pitrou7b998e92010-10-04 21:55:14 +0000110
Antoine Pitrou2dbc6e62015-04-11 00:31:01 +0200111 .. method:: peek(n)
Antoine Pitrou7b998e92010-10-04 21:55:14 +0000112
113 Read *n* uncompressed bytes without advancing the file position.
114 At most one single read on the compressed stream is done to satisfy
115 the call. The number of bytes returned may be more or less than
116 requested.
117
Nadeem Vawda69761042013-12-08 19:47:22 +0100118 .. note:: While calling :meth:`peek` does not change the file position of
119 the :class:`GzipFile`, it may change the position of the underlying
120 file object (e.g. if the :class:`GzipFile` was constructed with the
121 *fileobj* parameter).
122
Antoine Pitrou7b998e92010-10-04 21:55:14 +0000123 .. versionadded:: 3.2
124
Antoine Pitrou2dbc6e62015-04-11 00:31:01 +0200125 .. attribute:: mtime
126
127 When decompressing, the value of the last modification time field in
128 the most recently read header may be read from this attribute, as an
129 integer. The initial value before reading any headers is ``None``.
130
131 All :program:`gzip` compressed streams are required to contain this
132 timestamp field. Some programs, such as :program:`gunzip`\ , make use
133 of the timestamp. The format is the same as the return value of
134 :func:`time.time` and the :attr:`~os.stat_result.st_mtime` attribute of
135 the object returned by :func:`os.stat`.
136
Benjamin Peterson10745a92009-03-09 21:08:47 +0000137 .. versionchanged:: 3.1
Georg Brandlffb94ae2013-10-06 19:02:08 +0200138 Support for the :keyword:`with` statement was added, along with the
Antoine Pitrou2dbc6e62015-04-11 00:31:01 +0200139 *mtime* constructor argument and :attr:`mtime` attribute.
Benjamin Petersone0124bd2009-03-09 21:04:33 +0000140
Antoine Pitrou8e33fd72010-01-13 14:37:26 +0000141 .. versionchanged:: 3.2
Georg Brandlffb94ae2013-10-06 19:02:08 +0200142 Support for zero-padded and unseekable files was added.
Antoine Pitrou7b969842010-09-23 16:22:51 +0000143
Antoine Pitrou6b4be362011-04-04 21:09:05 +0200144 .. versionchanged:: 3.3
145 The :meth:`io.BufferedIOBase.read1` method is now implemented.
146
Nadeem Vawdaee1be992013-10-19 00:11:13 +0200147 .. versionchanged:: 3.4
148 Added support for the ``'x'`` and ``'xb'`` modes.
149
Serhiy Storchakabca63b32015-03-23 14:59:48 +0200150 .. versionchanged:: 3.5
151 Added support for writing arbitrary
152 :term:`bytes-like objects <bytes-like object>`.
Antoine Pitrou2dbc6e62015-04-11 00:31:01 +0200153 The :meth:`~io.BufferedIOBase.read` method now accepts an argument of
154 ``None``.
Serhiy Storchakabca63b32015-03-23 14:59:48 +0200155
Berker Peksag03020cf2016-10-02 13:47:58 +0300156 .. versionchanged:: 3.6
157 Accepts a :term:`path-like object`.
158
Georg Brandl116aa622007-08-15 14:28:22 +0000159
Antoine Pitrou79c5ef12010-08-17 21:10:05 +0000160.. function:: compress(data, compresslevel=9)
161
162 Compress the *data*, returning a :class:`bytes` object containing
163 the compressed data. *compresslevel* has the same meaning as in
164 the :class:`GzipFile` constructor above.
165
Antoine Pitroucdfe1c52010-08-17 21:15:00 +0000166 .. versionadded:: 3.2
167
Antoine Pitrou79c5ef12010-08-17 21:10:05 +0000168.. function:: decompress(data)
169
170 Decompress the *data*, returning a :class:`bytes` object containing the
171 uncompressed data.
172
Antoine Pitroucdfe1c52010-08-17 21:15:00 +0000173 .. versionadded:: 3.2
174
Georg Brandl116aa622007-08-15 14:28:22 +0000175
Christian Heimesbbe741d2008-03-28 10:53:29 +0000176.. _gzip-usage-examples:
177
178Examples of usage
179-----------------
180
181Example of how to read a compressed file::
182
183 import gzip
Antoine Pitroubf1a0182010-08-17 21:11:49 +0000184 with gzip.open('/home/joe/file.txt.gz', 'rb') as f:
185 file_content = f.read()
Christian Heimesbbe741d2008-03-28 10:53:29 +0000186
187Example of how to create a compressed GZIP file::
188
189 import gzip
Antoine Pitroubf1a0182010-08-17 21:11:49 +0000190 content = b"Lots of content here"
191 with gzip.open('/home/joe/file.txt.gz', 'wb') as f:
192 f.write(content)
Christian Heimesbbe741d2008-03-28 10:53:29 +0000193
194Example of how to GZIP compress an existing file::
195
196 import gzip
Andrew Kuchlingf887a612015-04-14 11:44:40 -0400197 import shutil
Antoine Pitroubf1a0182010-08-17 21:11:49 +0000198 with open('/home/joe/file.txt', 'rb') as f_in:
Éric Araujof5be0902010-08-17 21:24:05 +0000199 with gzip.open('/home/joe/file.txt.gz', 'wb') as f_out:
Andrew Kuchlingf887a612015-04-14 11:44:40 -0400200 shutil.copyfileobj(f_in, f_out)
Christian Heimesbbe741d2008-03-28 10:53:29 +0000201
Antoine Pitrou79c5ef12010-08-17 21:10:05 +0000202Example of how to GZIP compress a binary string::
203
204 import gzip
205 s_in = b"Lots of content here"
206 s_out = gzip.compress(s_in)
Christian Heimesbbe741d2008-03-28 10:53:29 +0000207
Georg Brandl116aa622007-08-15 14:28:22 +0000208.. seealso::
209
210 Module :mod:`zlib`
211 The basic data compression module needed to support the :program:`gzip` file
212 format.
213