Issue #6715: Add module for compression using the LZMA algorithm.
diff --git a/Doc/library/archiving.rst b/Doc/library/archiving.rst
index f2d08ad..b7d400c 100644
--- a/Doc/library/archiving.rst
+++ b/Doc/library/archiving.rst
@@ -5,7 +5,8 @@
******************************
The modules described in this chapter support data compression with the zlib,
-gzip, and bzip2 algorithms, and the creation of ZIP- and tar-format archives.
+gzip, bzip2 and lzma algorithms, and the creation of ZIP- and tar-format
+archives.
.. toctree::
@@ -13,5 +14,6 @@
zlib.rst
gzip.rst
bz2.rst
+ lzma.rst
zipfile.rst
tarfile.rst
diff --git a/Doc/library/bz2.rst b/Doc/library/bz2.rst
index 87f2cf3..dbf938a 100644
--- a/Doc/library/bz2.rst
+++ b/Doc/library/bz2.rst
@@ -12,7 +12,7 @@
This module provides a comprehensive interface for compressing and
decompressing data using the bzip2 compression algorithm.
-For related file formats, see the :mod:`gzip`, :mod:`zipfile`, and
+For related file formats, see the :mod:`gzip`, :mod:`lzma`, :mod:`zipfile`, and
:mod:`tarfile` modules.
The :mod:`bz2` module contains:
diff --git a/Doc/library/gzip.rst b/Doc/library/gzip.rst
index a5e08c9..4491acc 100644
--- a/Doc/library/gzip.rst
+++ b/Doc/library/gzip.rst
@@ -21,7 +21,7 @@
:program:`gzip` and :program:`gunzip` programs, such as those produced by
:program:`compress` and :program:`pack`, are not supported by this module.
-For other archive formats, see the :mod:`bz2`, :mod:`zipfile`, and
+For related file formats, see the :mod:`bz2`, :mod:`lzma`, :mod:`zipfile`, and
:mod:`tarfile` modules.
The module defines the following items:
diff --git a/Doc/library/lzma.rst b/Doc/library/lzma.rst
new file mode 100644
index 0000000..4e6db15
--- /dev/null
+++ b/Doc/library/lzma.rst
@@ -0,0 +1,344 @@
+:mod:`lzma` --- Compression using the LZMA algorithm
+====================================================
+
+.. module:: lzma
+ :synopsis: A Python wrapper for the liblzma compression library.
+.. moduleauthor:: Nadeem Vawda <nadeem.vawda@gmail.com>
+.. sectionauthor:: Nadeem Vawda <nadeem.vawda@gmail.com>
+
+.. versionadded:: 3.3
+
+
+This module provides classes and convenience functions for compressing and
+decompressing data using the LZMA compression algorithm. Also included is a file
+interface supporting the ``.xz`` and legacy ``.lzma`` file formats used by the
+:program:`xz` utility, as well as raw compressed streams.
+
+For related file formats, see the :mod:`bz2`, :mod:`gzip`, :mod:`zipfile`, and
+:mod:`tarfile` modules.
+
+The interface provided by this module is very similar to that of the :mod:`bz2`
+module. However, note that :class:`LZMAFile` is *not* thread-safe, unlike
+:class:`bz2.BZ2File`, so if you need to use a single :class:`LZMAFile` instance
+from multiple threads, it is necessary to protect it with a lock.
+
+
+.. exception:: LZMAError
+
+ This exception is raised when an error occurs during compression or
+ decompression, or while initializing the compressor/decompressor state.
+
+
+Reading and writing compressed files
+------------------------------------
+
+.. class:: LZMAFile(filename=None, mode="r", fileobj=None, format=None, check=-1, preset=None, filters=None)
+
+ Open an LZMA-compressed file.
+
+ An :class:`LZMAFile` can wrap an existing :term:`file object` (given by
+ *fileobj*), or operate directly on a named file (named by *filename*).
+ Exactly one of these two parameters should be provided. If *fileobj* is
+ provided, it is not closed when the :class:`LZMAFile` is closed.
+
+ The *mode* argument can be either ``"r"`` for reading (default), ``"w"`` for
+ overwriting, or ``"a"`` for appending. If *fileobj* is provided, a mode of
+ ``"w"`` does not truncate the file, and is instead equivalent to ``"a"``.
+
+ When opening a file for reading, the input file may be the concatenation of
+ multiple separate compressed streams. These are transparently decoded as a
+ single logical stream.
+
+ When opening a file for reading, the *format* and *filters* arguments have
+ the same meanings as for :class:`LZMADecompressor`. In this case, the *check*
+ and *preset* arguments should not be used.
+
+ When opening a file for writing, the *format*, *check*, *preset* and
+ *filters* arguments have the same meanings as for :class:`LZMACompressor`.
+
+ :class:`LZMAFile` supports all the members specified by
+ :class:`io.BufferedIOBase`, except for :meth:`detach` and :meth:`truncate`.
+ Iteration and the :keyword:`with` statement are supported.
+
+ The following method is also provided:
+
+ .. method:: peek(size=-1)
+
+ Return buffered data without advancing the file position. At least one
+ byte of data will be returned, unless EOF has been reached. The exact
+ number of bytes returned is unspecified (the *size* argument is ignored).
+
+
+Compressing and decompressing data in memory
+--------------------------------------------
+
+.. class:: LZMACompressor(format=FORMAT_XZ, check=-1, preset=None, filters=None)
+
+ Create a compressor object, which can be used to compress data incrementally.
+
+ For a more convenient way of compressing a single chunk of data, see
+ :func:`compress`.
+
+ The *format* argument specifies what container format should be used.
+ Possible values are:
+
+ * :const:`FORMAT_XZ`: The ``.xz`` container format.
+ This is the default format.
+
+ * :const:`FORMAT_ALONE`: The legacy ``.lzma`` container format.
+ This format is more limited than ``.xz`` -- it does not support integrity
+ checks or multiple filters.
+
+ * :const:`FORMAT_RAW`: A raw data stream, not using any container format.
+ This format specifier does not support integrity checks, and requires that
+ you always specify a custom filter chain (for both compression and
+ decompression). Additionally, data compressed in this manner cannot be
+ decompressed using :const:`FORMAT_AUTO` (see :class:`LZMADecompressor`).
+
+ The *check* argument specifies the type of integrity check to include in the
+ compressed data. This check is used when decompressing, to ensure that the
+ data has not been corrupted. Possible values are:
+
+ * :const:`CHECK_NONE`: No integrity check.
+ This is the default (and the only acceptable value) for
+ :const:`FORMAT_ALONE` and :const:`FORMAT_RAW`.
+
+ * :const:`CHECK_CRC32`: 32-bit Cyclic Redundancy Check.
+
+ * :const:`CHECK_CRC64`: 64-bit Cyclic Redundancy Check.
+ This is the default for :const:`FORMAT_XZ`.
+
+ * :const:`CHECK_SHA256`: 256-bit Secure Hash Algorithm.
+
+ If the specified check is not supported, an :class:`LZMAError` is raised.
+
+ The compression settings can be specified either as a preset compression
+ level (with the *preset* argument), or in detail as a custom filter chain
+ (with the *filters* argument).
+
+ The *preset* argument (if provided) should be an integer between ``0`` and
+ ``9`` (inclusive), optionally OR-ed with the constant
+ :const:`PRESET_EXTREME`. If neither *preset* nor *filters* are given, the
+ default behavior is to use :const:`PRESET_DEFAULT` (preset level ``6``).
+ Higher presets produce smaller output, but make compression more CPU- and
+ memory-intensive, and also increase the memory required for decompression.
+
+ The *filters* argument (if provided) should be a filter chain specifier.
+ See :ref:`filter-chain-specs` for details.
+
+ .. method:: compress(data)
+
+ Compress *data* (a :class:`bytes` object), returning a :class:`bytes`
+ object containing compressed data for at least part of the input. Some of
+ *data* may be buffered internally, for use in later calls to
+ :meth:`compress` and :meth:`flush`. The returned data should be
+ concatenated with the output of any previous calls to :meth:`compress`.
+
+ .. method:: flush()
+
+ Finish the compression process, returning a :class:`bytes` object
+ containing any data stored in the compressor's internal buffers.
+
+ The compressor cannot be used after this method has been called.
+
+
+.. class:: LZMADecompressor(format=FORMAT_AUTO, memlimit=None, filters=None)
+
+ Create a decompressor object, which can be used to decompress data
+ incrementally.
+
+ For a more convenient way of decompressing an entire compressed stream at
+ once, see :func:`decompress`.
+
+ The *format* argument specifies the container format that should be used. The
+ default is :const:`FORMAT_AUTO`, which can decompress both ``.xz`` and
+ ``.lzma`` files. Other possible values are :const:`FORMAT_XZ`,
+ :const:`FORMAT_ALONE`, and :const:`FORMAT_RAW`.
+
+ The *memlimit* argument specifies a limit (in bytes) on the amount of memory
+ that the decompressor can use. When this argument is used, decompression will
+ fail with an :class:`LZMAError` if it is not possible to decompress the input
+ within the given memory limit.
+
+ The *filters* argument specifies the filter chain that was used to create
+ the stream being decompressed. This argument is required if *format* is
+ :const:`FORMAT_RAW`, but should not be used for other formats.
+ See :ref:`filter-chain-specs` for more information about filter chains.
+
+ .. note::
+ This class does not transparently handle inputs containing multiple
+ compressed streams, unlike :func:`decompress` and :class:`LZMAFile`. To
+ decompress a multi-stream input with :class:`LZMADecompressor`, you must
+ create a new decompressor for each stream.
+
+ .. method:: decompress(data)
+
+ Decompress *data* (a :class:`bytes` object), returning a :class:`bytes`
+ object containing the decompressed data for at least part of the input.
+ Some of *data* may be buffered internally, for use in later calls to
+ :meth:`decompress`. The returned data should be concatenated with the
+ output of any previous calls to :meth:`decompress`.
+
+ .. attribute:: check
+
+ The ID of the integrity check used by the input stream. This may be
+ :const:`CHECK_UNKNOWN` until enough of the input has been decoded to
+ determine what integrity check it uses.
+
+ .. attribute:: eof
+
+ True if the end-of-stream marker has been reached.
+
+ .. attribute:: unused_data
+
+ Data found after the end of the compressed stream.
+
+ Before the end of the stream is reached, this will be ``b""``.
+
+
+.. function:: compress(data, format=FORMAT_XZ, check=-1, preset=None, filters=None)
+
+ Compress *data* (a :class:`bytes` object), returning the compressed data as a
+ :class:`bytes` object.
+
+ See :class:`LZMACompressor` above for a description of the *format*, *check*,
+ *preset* and *filters* arguments.
+
+
+.. function:: decompress(data, format=FORMAT_AUTO, memlimit=None, filters=None)
+
+ Decompress *data* (a :class:`bytes` object), returning the uncompressed data
+ as a :class:`bytes` object.
+
+ If *data* is the concatenation of multiple distinct compressed streams,
+ decompress all of these streams, and return the concatenation of the results.
+
+ See :class:`LZMADecompressor` above for a description of the *format*,
+ *memlimit* and *filters* arguments.
+
+
+Miscellaneous
+-------------
+
+.. function:: check_is_supported(check)
+
+ Returns true if the given integrity check is supported on this system.
+
+ :const:`CHECK_NONE` and :const:`CHECK_CRC32` are always supported.
+ :const:`CHECK_CRC64` and :const:`CHECK_SHA256` may be unavailable if you are
+ using a version of :program:`liblzma` that was compiled with a limited
+ feature set.
+
+
+.. _filter-chain-specs:
+
+Specifying custom filter chains
+-------------------------------
+
+A filter chain specifier is a sequence of dictionaries, where each dictionary
+contains the ID and options for a single filter. Each dictionary must contain
+the key ``"id"``, and may contain additional keys to specify filter-dependent
+options. Valid filter IDs are as follows:
+
+* Compression filters:
+ * :const:`FILTER_LZMA1` (for use with :const:`FORMAT_ALONE`)
+ * :const:`FILTER_LZMA2` (for use with :const:`FORMAT_XZ` and :const:`FORMAT_RAW`)
+
+* Delta filter:
+ * :const:`FILTER_DELTA`
+
+* Branch-Call-Jump (BCJ) filters:
+ * :const:`FILTER_X86`
+ * :const:`FILTER_IA64`
+ * :const:`FILTER_ARM`
+ * :const:`FILTER_ARMTHUMB`
+ * :const:`FILTER_POWERPC`
+ * :const:`FILTER_SPARC`
+
+A filter chain can consist of up to 4 filters, and cannot be empty. The last
+filter in the chain must be a compression filter, and any other filters must be
+delta or BCJ filters.
+
+Compression filters support the following options (specified as additional
+entries in the dictionary representing the filter):
+
+ * ``preset``: A compression preset to use as a source of default values for
+ options that are not specified explicitly.
+ * ``dict_size``: Dictionary size in bytes. This should be between 4KiB and
+ 1.5GiB (inclusive).
+ * ``lc``: Number of literal context bits.
+ * ``lp``: Number of literal position bits. The sum ``lc + lp`` must be at
+ most 4.
+ * ``pb``: Number of position bits; must be at most 4.
+ * ``mode``: :const:`MODE_FAST` or :const:`MODE_NORMAL`.
+ * ``nice_len``: What should be considered a "nice length" for a match.
+ This should be 273 or less.
+ * ``mf``: What match finder to use -- :const:`MF_HC3`, :const:`MF_HC4`,
+ :const:`MF_BT2`, :const:`MF_BT3`, or :const:`MF_BT4`.
+ * ``depth``: Maximum search depth used by match finder. 0 (default) means to
+ select automatically based on other filter options.
+
+The delta filter stores the differences between bytes, producing more repetitive
+input for the compressor in certain circumstances. It only supports a single
+The delta filter supports only one option, ``dist``. This indicates the distance
+between bytes to be subtracted. The default is 1, i.e. take the differences
+between adjacent bytes.
+
+The BCJ filters are intended to be applied to machine code. They convert
+relative branches, calls and jumps in the code to use absolute addressing, with
+the aim of increasing the redundancy that can be exploited by the compressor.
+These filters support one option, ``start_offset``. This specifies the address
+that should be mapped to the beginning of the input data. The default is 0.
+
+
+Examples
+--------
+
+Reading in a compressed file::
+
+ import lzma
+ with lzma.LZMAFile("file.xz") as f:
+ file_content = f.read()
+
+Creating a compressed file::
+
+ import lzma
+ data = b"Insert Data Here"
+ with lzma.LZMAFile("file.xz", "w") as f:
+ f.write(data)
+
+Compressing data in memory::
+
+ import lzma
+ data_in = b"Insert Data Here"
+ data_out = lzma.compress(data_in)
+
+Incremental compression::
+
+ import lzma
+ lzc = lzma.LZMACompressor()
+ out1 = lzc.compress(b"Some data\n")
+ out2 = lzc.compress(b"Another piece of data\n")
+ out3 = lzc.compress(b"Even more data\n")
+ out4 = lzc.flush()
+ # Concatenate all the partial results:
+ result = b"".join([out1, out2, out3, out4])
+
+Writing compressed data to an already-open file::
+
+ import lzma
+ with open("file.xz", "wb") as f:
+ f.write(b"This data will not be compressed\n")
+ with lzma.LZMAFile(fileobj=f, mode="w") as lzf:
+ lzf.write(b"This *will* be compressed\n")
+ f.write(b"Not compressed\n")
+
+Creating a compressed file using a custom filter chain::
+
+ import lzma
+ my_filters = [
+ {"id": lzma.FILTER_DELTA, "dist": 5},
+ {"id": lzma.FILTER_LZMA2, "preset": 7 | lzma.PRESET_EXTREME},
+ ]
+ with lzma.LZMAFile("file.xz", "w", filters=my_filters) as f:
+ f.write(b"blah blah blah")
diff --git a/Doc/library/zipfile.rst b/Doc/library/zipfile.rst
index 6f84bcc..c3e13ed 100644
--- a/Doc/library/zipfile.rst
+++ b/Doc/library/zipfile.rst
@@ -23,7 +23,7 @@
create an encrypted file. Decryption is extremely slow as it is
implemented in native Python rather than C.
-For other archive formats, see the :mod:`bz2`, :mod:`gzip`, and
+For related file formats, see the :mod:`bz2`, :mod:`gzip`, :mod:`lzma`, and
:mod:`tarfile` modules.
The module defines the following items:
diff --git a/Doc/library/zlib.rst b/Doc/library/zlib.rst
index 9f8eb26..7ad3624 100644
--- a/Doc/library/zlib.rst
+++ b/Doc/library/zlib.rst
@@ -18,8 +18,8 @@
consult the zlib manual at http://www.zlib.net/manual.html for authoritative
information.
-For reading and writing ``.gz`` files see the :mod:`gzip` module. For
-other related file formats, see the :mod:`bz2`, :mod:`zipfile`, and
+For reading and writing ``.gz`` files see the :mod:`gzip` module. For other
+related file formats, see the :mod:`bz2`, :mod:`lzma`, :mod:`zipfile`, and
:mod:`tarfile` modules.
The available exception and functions in this module are:
diff --git a/Lib/lzma.py b/Lib/lzma.py
new file mode 100644
index 0000000..780c666
--- /dev/null
+++ b/Lib/lzma.py
@@ -0,0 +1,398 @@
+"""Interface to the liblzma compression library.
+
+This module provides a class for reading and writing compressed files,
+classes for incremental (de)compression, and convenience functions for
+one-shot (de)compression.
+
+These classes and functions support both the XZ and legacy LZMA
+container formats, as well as raw compressed data streams.
+"""
+
+__all__ = [
+ "CHECK_NONE", "CHECK_CRC32", "CHECK_CRC64", "CHECK_SHA256",
+ "CHECK_ID_MAX", "CHECK_UNKNOWN",
+ "FILTER_LZMA1", "FILTER_LZMA2", "FILTER_DELTA", "FILTER_X86", "FILTER_IA64",
+ "FILTER_ARM", "FILTER_ARMTHUMB", "FILTER_POWERPC", "FILTER_SPARC",
+ "FORMAT_AUTO", "FORMAT_XZ", "FORMAT_ALONE", "FORMAT_RAW",
+ "MF_HC3", "MF_HC4", "MF_BT2", "MF_BT3", "MF_BT4",
+ "MODE_FAST", "MODE_NORMAL", "PRESET_DEFAULT", "PRESET_EXTREME",
+
+ "LZMACompressor", "LZMADecompressor", "LZMAFile", "LZMAError",
+ "compress", "decompress", "check_is_supported",
+]
+
+import io
+from _lzma import *
+
+
+_MODE_CLOSED = 0
+_MODE_READ = 1
+_MODE_READ_EOF = 2
+_MODE_WRITE = 3
+
+_BUFFER_SIZE = 8192
+
+
+class LZMAFile(io.BufferedIOBase):
+
+ """A file object providing transparent LZMA (de)compression.
+
+ An LZMAFile can act as a wrapper for an existing file object, or
+ refer directly to a named file on disk.
+
+ Note that LZMAFile provides a *binary* file interface - data read
+ is returned as bytes, and data to be written must be given as bytes.
+ """
+
+ def __init__(self, filename=None, mode="r", *,
+ fileobj=None, format=None, check=-1,
+ preset=None, filters=None):
+ """Open an LZMA-compressed file.
+
+ If filename is given, open the named file. Otherwise, operate on
+ the file object given by fileobj. Exactly one of these two
+ parameters should be provided.
+
+ mode can be "r" for reading (default), "w" for (over)writing, or
+ "a" for appending.
+
+ format specifies the container format to use for the file.
+ If mode is "r", this defaults to FORMAT_AUTO. Otherwise, the
+ default is FORMAT_XZ.
+
+ check specifies the integrity check to use. This argument can
+ only be used when opening a file for writing. For FORMAT_XZ,
+ the default is CHECK_CRC64. FORMAT_ALONE and FORMAT_RAW do not
+ support integrity checks - for these formats, check must be
+ omitted, or be CHECK_NONE.
+
+ When opening a file for reading, the *preset* argument is not
+ meaningful, and should be omitted. The *filters* argument should
+ also be omitted, except when format is FORMAT_RAW (in which case
+ it is required).
+
+ When opening a file for writing, the settings used by the
+ compressor can be specified either as a preset compression
+ level (with the *preset* argument), or in detail as a custom
+ filter chain (with the *filters* argument). For FORMAT_XZ and
+ FORMAT_ALONE, the default is to use the PRESET_DEFAULT preset
+ level. For FORMAT_RAW, the caller must always specify a filter
+ chain; the raw compressor does not support preset compression
+ levels.
+
+ preset (if provided) should be an integer in the range 0-9,
+ optionally OR-ed with the constant PRESET_EXTREME.
+
+ filters (if provided) should be a sequence of dicts. Each dict
+ should have an entry for "id" indicating ID of the filter, plus
+ additional entries for options to the filter.
+ """
+ self._fp = None
+ self._closefp = False
+ self._mode = _MODE_CLOSED
+ self._pos = 0
+ self._size = -1
+
+ if mode == "r":
+ if check != -1:
+ raise ValueError("Cannot specify an integrity check "
+ "when opening a file for reading")
+ if preset is not None:
+ raise ValueError("Cannot specify a preset compression "
+ "level when opening a file for reading")
+ if format is None:
+ format = FORMAT_AUTO
+ mode_code = _MODE_READ
+ # Save the args to pass to the LZMADecompressor initializer.
+ # If the file contains multiple compressed streams, each
+ # stream will need a separate decompressor object.
+ self._init_args = {"format":format, "filters":filters}
+ self._decompressor = LZMADecompressor(**self._init_args)
+ self._buffer = None
+ elif mode in ("w", "a"):
+ if format is None:
+ format = FORMAT_XZ
+ mode_code = _MODE_WRITE
+ self._compressor = LZMACompressor(format=format, check=check,
+ preset=preset, filters=filters)
+ else:
+ raise ValueError("Invalid mode: {!r}".format(mode))
+
+ if filename is not None and fileobj is None:
+ mode += "b"
+ self._fp = open(filename, mode)
+ self._closefp = True
+ self._mode = mode_code
+ elif fileobj is not None and filename is None:
+ self._fp = fileobj
+ self._mode = mode_code
+ else:
+ raise ValueError("Must give exactly one of filename and fileobj")
+
+ def close(self):
+ """Flush and close the file.
+
+ May be called more than once without error. Once the file is
+ closed, any other operation on it will raise a ValueError.
+ """
+ if self._mode == _MODE_CLOSED:
+ return
+ try:
+ if self._mode in (_MODE_READ, _MODE_READ_EOF):
+ self._decompressor = None
+ self._buffer = None
+ elif self._mode == _MODE_WRITE:
+ self._fp.write(self._compressor.flush())
+ self._compressor = None
+ finally:
+ try:
+ if self._closefp:
+ self._fp.close()
+ finally:
+ self._fp = None
+ self._closefp = False
+ self._mode = _MODE_CLOSED
+
+ @property
+ def closed(self):
+ """True if this file is closed."""
+ return self._mode == _MODE_CLOSED
+
+ def fileno(self):
+ """Return the file descriptor for the underlying file."""
+ self._check_not_closed()
+ return self._fp.fileno()
+
+ def seekable(self):
+ """Return whether the file supports seeking."""
+ return self.readable()
+
+ def readable(self):
+ """Return whether the file was opened for reading."""
+ self._check_not_closed()
+ return self._mode in (_MODE_READ, _MODE_READ_EOF)
+
+ def writable(self):
+ """Return whether the file was opened for writing."""
+ self._check_not_closed()
+ return self._mode == _MODE_WRITE
+
+ # Mode-checking helper functions.
+
+ def _check_not_closed(self):
+ if self.closed:
+ raise ValueError("I/O operation on closed file")
+
+ def _check_can_read(self):
+ if not self.readable():
+ raise io.UnsupportedOperation("File not open for reading")
+
+ def _check_can_write(self):
+ if not self.writable():
+ raise io.UnsupportedOperation("File not open for writing")
+
+ def _check_can_seek(self):
+ if not self.seekable():
+ raise io.UnsupportedOperation("Seeking is only supported "
+ "on files open for reading")
+
+ # Fill the readahead buffer if it is empty. Returns False on EOF.
+ def _fill_buffer(self):
+ if self._buffer:
+ return True
+
+ if self._decompressor.unused_data:
+ rawblock = self._decompressor.unused_data
+ else:
+ rawblock = self._fp.read(_BUFFER_SIZE)
+
+ if not rawblock:
+ if self._decompressor.eof:
+ self._mode = _MODE_READ_EOF
+ self._size = self._pos
+ return False
+ else:
+ raise EOFError("Compressed file ended before the "
+ "end-of-stream marker was reached")
+
+ # Continue to next stream.
+ if self._decompressor.eof:
+ self._decompressor = LZMADecompressor(**self._init_args)
+
+ self._buffer = self._decompressor.decompress(rawblock)
+ return True
+
+ # Read data until EOF.
+ # If return_data is false, consume the data without returning it.
+ def _read_all(self, return_data=True):
+ blocks = []
+ while self._fill_buffer():
+ if return_data:
+ blocks.append(self._buffer)
+ self._pos += len(self._buffer)
+ self._buffer = None
+ if return_data:
+ return b"".join(blocks)
+
+ # Read a block of up to n bytes.
+ # If return_data is false, consume the data without returning it.
+ def _read_block(self, n, return_data=True):
+ blocks = []
+ while n > 0 and self._fill_buffer():
+ if n < len(self._buffer):
+ data = self._buffer[:n]
+ self._buffer = self._buffer[n:]
+ else:
+ data = self._buffer
+ self._buffer = None
+ if return_data:
+ blocks.append(data)
+ self._pos += len(data)
+ n -= len(data)
+ if return_data:
+ return b"".join(blocks)
+
+ def peek(self, size=-1):
+ """Return buffered data without advancing the file position.
+
+ Always returns at least one byte of data, unless at EOF.
+ The exact number of bytes returned is unspecified.
+ """
+ self._check_can_read()
+ if self._mode == _MODE_READ_EOF or not self._fill_buffer():
+ return b""
+ return self._buffer
+
+ def read(self, size=-1):
+ """Read up to size uncompressed bytes from the file.
+
+ If size is negative or omitted, read until EOF is reached.
+ Returns b"" if the file is already at EOF.
+ """
+ self._check_can_read()
+ if self._mode == _MODE_READ_EOF or size == 0:
+ return b""
+ elif size < 0:
+ return self._read_all()
+ else:
+ return self._read_block(size)
+
+ def read1(self, size=-1):
+ """Read up to size uncompressed bytes with at most one read
+ from the underlying stream.
+
+ Returns b"" if the file is at EOF.
+ """
+ self._check_can_read()
+ if (size == 0 or self._mode == _MODE_READ_EOF or
+ not self._fill_buffer()):
+ return b""
+ if 0 < size < len(self._buffer):
+ data = self._buffer[:size]
+ self._buffer = self._buffer[size:]
+ else:
+ data = self._buffer
+ self._buffer = None
+ self._pos += len(data)
+ return data
+
+ def write(self, data):
+ """Write a bytes object to the file.
+
+ Returns the number of uncompressed bytes written, which is
+ always len(data). Note that due to buffering, the file on disk
+ may not reflect the data written until close() is called.
+ """
+ self._check_can_write()
+ compressed = self._compressor.compress(data)
+ self._fp.write(compressed)
+ self._pos += len(data)
+ return len(data)
+
+ # Rewind the file to the beginning of the data stream.
+ def _rewind(self):
+ self._fp.seek(0, 0)
+ self._mode = _MODE_READ
+ self._pos = 0
+ self._decompressor = LZMADecompressor(**self._init_args)
+ self._buffer = None
+
+ def seek(self, offset, whence=0):
+ """Change the file position.
+
+ The new position is specified by offset, relative to the
+ position indicated by whence. Possible values for whence are:
+
+ 0: start of stream (default): offset must not be negative
+ 1: current stream position
+ 2: end of stream; offset must not be positive
+
+ Returns the new file position.
+
+ Note that seeking is emulated, sp depending on the parameters,
+ this operation may be extremely slow.
+ """
+ self._check_can_seek()
+
+ # Recalculate offset as an absolute file position.
+ if whence == 0:
+ pass
+ elif whence == 1:
+ offset = self._pos + offset
+ elif whence == 2:
+ # Seeking relative to EOF - we need to know the file's size.
+ if self._size < 0:
+ self._read_all(return_data=False)
+ offset = self._size + offset
+ else:
+ raise ValueError("Invalid value for whence: {}".format(whence))
+
+ # Make it so that offset is the number of bytes to skip forward.
+ if offset < self._pos:
+ self._rewind()
+ else:
+ offset -= self._pos
+
+ # Read and discard data until we reach the desired position.
+ if self._mode != _MODE_READ_EOF:
+ self._read_block(offset, return_data=False)
+
+ return self._pos
+
+ def tell(self):
+ """Return the current file position."""
+ self._check_not_closed()
+ return self._pos
+
+
+def compress(data, format=FORMAT_XZ, check=-1, preset=None, filters=None):
+ """Compress a block of data.
+
+ Refer to LZMACompressor's docstring for a description of the
+ optional arguments *format*, *check*, *preset* and *filters*.
+
+ For incremental compression, use an LZMACompressor object instead.
+ """
+ comp = LZMACompressor(format, check, preset, filters)
+ return comp.compress(data) + comp.flush()
+
+
+def decompress(data, format=FORMAT_AUTO, memlimit=None, filters=None):
+ """Decompress a block of data.
+
+ Refer to LZMADecompressor's docstring for a description of the
+ optional arguments *format*, *check* and *filters*.
+
+ For incremental decompression, use a LZMADecompressor object instead.
+ """
+ results = []
+ while True:
+ decomp = LZMADecompressor(format, memlimit, filters)
+ results.append(decomp.decompress(data))
+ if not decomp.eof:
+ raise LZMAError("Compressed data ended before the "
+ "end-of-stream marker was reached")
+ if not decomp.unused_data:
+ return b"".join(results)
+ # There is unused data left over. Proceed to next stream.
+ data = decomp.unused_data
diff --git a/Lib/test/test_lzma.py b/Lib/test/test_lzma.py
new file mode 100644
index 0000000..8d3df92
--- /dev/null
+++ b/Lib/test/test_lzma.py
@@ -0,0 +1,1335 @@
+from io import BytesIO, UnsupportedOperation
+import os
+import random
+import unittest
+
+from test.support import (
+ _4G, TESTFN, import_module, bigmemtest, run_unittest, unlink
+)
+
+lzma = import_module("lzma")
+from lzma import LZMACompressor, LZMADecompressor, LZMAError, LZMAFile
+
+
+class CompressorDecompressorTestCase(unittest.TestCase):
+
+ # Test error cases.
+
+ def test_simple_bad_args(self):
+ self.assertRaises(TypeError, LZMACompressor, [])
+ self.assertRaises(TypeError, LZMACompressor, format=3.45)
+ self.assertRaises(TypeError, LZMACompressor, check="")
+ self.assertRaises(TypeError, LZMACompressor, preset="asdf")
+ self.assertRaises(TypeError, LZMACompressor, filters=3)
+ # Can't specify FORMAT_AUTO when compressing.
+ self.assertRaises(ValueError, LZMACompressor, format=lzma.FORMAT_AUTO)
+ # Can't specify a preset and a custom filter chain at the same time.
+ with self.assertRaises(ValueError):
+ LZMACompressor(preset=7, filters=[{"id": lzma.FILTER_LZMA2}])
+
+ self.assertRaises(TypeError, LZMADecompressor, ())
+ self.assertRaises(TypeError, LZMADecompressor, memlimit=b"qw")
+ with self.assertRaises(TypeError):
+ LZMADecompressor(lzma.FORMAT_RAW, filters="zzz")
+ # Cannot specify a memory limit with FILTER_RAW.
+ with self.assertRaises(ValueError):
+ LZMADecompressor(lzma.FORMAT_RAW, memlimit=0x1000000)
+ # Can only specify a custom filter chain with FILTER_RAW.
+ self.assertRaises(ValueError, LZMADecompressor, filters=FILTERS_RAW_1)
+ with self.assertRaises(ValueError):
+ LZMADecompressor(format=lzma.FORMAT_XZ, filters=FILTERS_RAW_1)
+ with self.assertRaises(ValueError):
+ LZMADecompressor(format=lzma.FORMAT_ALONE, filters=FILTERS_RAW_1)
+
+ lzc = LZMACompressor()
+ self.assertRaises(TypeError, lzc.compress)
+ self.assertRaises(TypeError, lzc.compress, b"foo", b"bar")
+ self.assertRaises(TypeError, lzc.flush, b"blah")
+ empty = lzc.flush()
+ self.assertRaises(ValueError, lzc.compress, b"quux")
+ self.assertRaises(ValueError, lzc.flush)
+
+ lzd = LZMADecompressor()
+ self.assertRaises(TypeError, lzd.decompress)
+ self.assertRaises(TypeError, lzd.decompress, b"foo", b"bar")
+ lzd.decompress(empty)
+ self.assertRaises(EOFError, lzd.decompress, b"quux")
+
+ def test_bad_filter_spec(self):
+ self.assertRaises(TypeError, LZMACompressor, filters=[b"wobsite"])
+ self.assertRaises(ValueError, LZMACompressor, filters=[{"xyzzy": 3}])
+ self.assertRaises(ValueError, LZMACompressor, filters=[{"id": 98765}])
+ with self.assertRaises(ValueError):
+ LZMACompressor(filters=[{"id": lzma.FILTER_LZMA2, "foo": 0}])
+ with self.assertRaises(ValueError):
+ LZMACompressor(filters=[{"id": lzma.FILTER_DELTA, "foo": 0}])
+ with self.assertRaises(ValueError):
+ LZMACompressor(filters=[{"id": lzma.FILTER_X86, "foo": 0}])
+
+ def test_decompressor_after_eof(self):
+ lzd = LZMADecompressor()
+ lzd.decompress(COMPRESSED_XZ)
+ self.assertRaises(EOFError, lzd.decompress, b"nyan")
+
+ def test_decompressor_memlimit(self):
+ lzd = LZMADecompressor(memlimit=1024)
+ self.assertRaises(LZMAError, lzd.decompress, COMPRESSED_XZ)
+
+ lzd = LZMADecompressor(lzma.FORMAT_XZ, memlimit=1024)
+ self.assertRaises(LZMAError, lzd.decompress, COMPRESSED_XZ)
+
+ lzd = LZMADecompressor(lzma.FORMAT_ALONE, memlimit=1024)
+ self.assertRaises(LZMAError, lzd.decompress, COMPRESSED_ALONE)
+
+ # Test LZMADecompressor on known-good input data.
+
+ def _test_decompressor(self, lzd, data, check, unused_data=b""):
+ self.assertFalse(lzd.eof)
+ out = lzd.decompress(data)
+ self.assertEqual(out, INPUT)
+ self.assertEqual(lzd.check, check)
+ self.assertTrue(lzd.eof)
+ self.assertEqual(lzd.unused_data, unused_data)
+
+ def test_decompressor_auto(self):
+ lzd = LZMADecompressor()
+ self._test_decompressor(lzd, COMPRESSED_XZ, lzma.CHECK_CRC64)
+
+ lzd = LZMADecompressor()
+ self._test_decompressor(lzd, COMPRESSED_ALONE, lzma.CHECK_NONE)
+
+ def test_decompressor_xz(self):
+ lzd = LZMADecompressor(lzma.FORMAT_XZ)
+ self._test_decompressor(lzd, COMPRESSED_XZ, lzma.CHECK_CRC64)
+
+ def test_decompressor_alone(self):
+ lzd = LZMADecompressor(lzma.FORMAT_ALONE)
+ self._test_decompressor(lzd, COMPRESSED_ALONE, lzma.CHECK_NONE)
+
+ def test_decompressor_raw_1(self):
+ lzd = LZMADecompressor(lzma.FORMAT_RAW, filters=FILTERS_RAW_1)
+ self._test_decompressor(lzd, COMPRESSED_RAW_1, lzma.CHECK_NONE)
+
+ def test_decompressor_raw_2(self):
+ lzd = LZMADecompressor(lzma.FORMAT_RAW, filters=FILTERS_RAW_2)
+ self._test_decompressor(lzd, COMPRESSED_RAW_2, lzma.CHECK_NONE)
+
+ def test_decompressor_raw_3(self):
+ lzd = LZMADecompressor(lzma.FORMAT_RAW, filters=FILTERS_RAW_3)
+ self._test_decompressor(lzd, COMPRESSED_RAW_3, lzma.CHECK_NONE)
+
+ def test_decompressor_raw_4(self):
+ lzd = LZMADecompressor(lzma.FORMAT_RAW, filters=FILTERS_RAW_4)
+ self._test_decompressor(lzd, COMPRESSED_RAW_4, lzma.CHECK_NONE)
+
+ def test_decompressor_chunks(self):
+ lzd = LZMADecompressor()
+ out = []
+ for i in range(0, len(COMPRESSED_XZ), 10):
+ self.assertFalse(lzd.eof)
+ out.append(lzd.decompress(COMPRESSED_XZ[i:i+10]))
+ out = b"".join(out)
+ self.assertEqual(out, INPUT)
+ self.assertEqual(lzd.check, lzma.CHECK_CRC64)
+ self.assertTrue(lzd.eof)
+ self.assertEqual(lzd.unused_data, b"")
+
+ def test_decompressor_unused_data(self):
+ lzd = LZMADecompressor()
+ extra = b"fooblibar"
+ self._test_decompressor(lzd, COMPRESSED_XZ + extra, lzma.CHECK_CRC64,
+ unused_data=extra)
+
+ def test_decompressor_bad_input(self):
+ lzd = LZMADecompressor()
+ self.assertRaises(LZMAError, lzd.decompress, COMPRESSED_RAW_1)
+
+ lzd = LZMADecompressor(lzma.FORMAT_XZ)
+ self.assertRaises(LZMAError, lzd.decompress, COMPRESSED_ALONE)
+
+ lzd = LZMADecompressor(lzma.FORMAT_ALONE)
+ self.assertRaises(LZMAError, lzd.decompress, COMPRESSED_XZ)
+
+ lzd = LZMADecompressor(lzma.FORMAT_RAW, filters=FILTERS_RAW_1)
+ self.assertRaises(LZMAError, lzd.decompress, COMPRESSED_XZ)
+
+ # Test that LZMACompressor->LZMADecompressor preserves the input data.
+
+ def test_roundtrip_xz(self):
+ lzc = LZMACompressor()
+ cdata = lzc.compress(INPUT) + lzc.flush()
+ lzd = LZMADecompressor()
+ self._test_decompressor(lzd, cdata, lzma.CHECK_CRC64)
+
+ def test_roundtrip_alone(self):
+ lzc = LZMACompressor(lzma.FORMAT_ALONE)
+ cdata = lzc.compress(INPUT) + lzc.flush()
+ lzd = LZMADecompressor()
+ self._test_decompressor(lzd, cdata, lzma.CHECK_NONE)
+
+ def test_roundtrip_raw(self):
+ lzc = LZMACompressor(lzma.FORMAT_RAW, filters=FILTERS_RAW_4)
+ cdata = lzc.compress(INPUT) + lzc.flush()
+ lzd = LZMADecompressor(lzma.FORMAT_RAW, filters=FILTERS_RAW_4)
+ self._test_decompressor(lzd, cdata, lzma.CHECK_NONE)
+
+ def test_roundtrip_chunks(self):
+ lzc = LZMACompressor()
+ cdata = []
+ for i in range(0, len(INPUT), 10):
+ cdata.append(lzc.compress(INPUT[i:i+10]))
+ cdata.append(lzc.flush())
+ cdata = b"".join(cdata)
+ lzd = LZMADecompressor()
+ self._test_decompressor(lzd, cdata, lzma.CHECK_CRC64)
+
+ # LZMADecompressor intentionally does not handle concatenated streams.
+
+ def test_decompressor_multistream(self):
+ lzd = LZMADecompressor()
+ self._test_decompressor(lzd, COMPRESSED_XZ + COMPRESSED_ALONE,
+ lzma.CHECK_CRC64, unused_data=COMPRESSED_ALONE)
+
+ # Test with inputs larger than 4GiB.
+
+ @bigmemtest(size=_4G + 100, memuse=2)
+ def test_compressor_bigmem(self, size):
+ lzc = LZMACompressor()
+ cdata = lzc.compress(b"x" * size) + lzc.flush()
+ ddata = lzma.decompress(cdata)
+ try:
+ self.assertEqual(len(ddata), size)
+ self.assertEqual(len(ddata.strip(b"x")), 0)
+ finally:
+ ddata = None
+
+ @bigmemtest(size=_4G + 100, memuse=3)
+ def test_decompressor_bigmem(self, size):
+ lzd = LZMADecompressor()
+ blocksize = 10 * 1024 * 1024
+ block = random.getrandbits(blocksize * 8).to_bytes(blocksize, "little")
+ try:
+ input = block * (size // blocksize + 1)
+ cdata = lzma.compress(input)
+ ddata = lzd.decompress(cdata)
+ self.assertEqual(ddata, input)
+ finally:
+ input = cdata = ddata = None
+
+
+class CompressDecompressFunctionTestCase(unittest.TestCase):
+
+ # Test error cases:
+
+ def test_bad_args(self):
+ self.assertRaises(TypeError, lzma.compress)
+ self.assertRaises(TypeError, lzma.compress, [])
+ self.assertRaises(TypeError, lzma.compress, b"", format="xz")
+ self.assertRaises(TypeError, lzma.compress, b"", check="none")
+ self.assertRaises(TypeError, lzma.compress, b"", preset="blah")
+ self.assertRaises(TypeError, lzma.compress, b"", filters=1024)
+ # Can't specify a preset and a custom filter chain at the same time.
+ with self.assertRaises(ValueError):
+ lzma.compress(b"", preset=3, filters=[{"id": lzma.FILTER_LZMA2}])
+
+ self.assertRaises(TypeError, lzma.decompress)
+ self.assertRaises(TypeError, lzma.decompress, [])
+ self.assertRaises(TypeError, lzma.decompress, b"", format="lzma")
+ self.assertRaises(TypeError, lzma.decompress, b"", memlimit=7.3e9)
+ with self.assertRaises(TypeError):
+ lzma.decompress(b"", format=lzma.FORMAT_RAW, filters={})
+ # Cannot specify a memory limit with FILTER_RAW.
+ with self.assertRaises(ValueError):
+ lzma.decompress(b"", format=lzma.FORMAT_RAW, memlimit=0x1000000)
+ # Can only specify a custom filter chain with FILTER_RAW.
+ with self.assertRaises(ValueError):
+ lzma.decompress(b"", filters=FILTERS_RAW_1)
+ with self.assertRaises(ValueError):
+ lzma.decompress(b"", format=lzma.FORMAT_XZ, filters=FILTERS_RAW_1)
+ with self.assertRaises(ValueError):
+ lzma.decompress(
+ b"", format=lzma.FORMAT_ALONE, filters=FILTERS_RAW_1)
+
+ def test_decompress_memlimit(self):
+ with self.assertRaises(LZMAError):
+ lzma.decompress(COMPRESSED_XZ, memlimit=1024)
+ with self.assertRaises(LZMAError):
+ lzma.decompress(
+ COMPRESSED_XZ, format=lzma.FORMAT_XZ, memlimit=1024)
+ with self.assertRaises(LZMAError):
+ lzma.decompress(
+ COMPRESSED_ALONE, format=lzma.FORMAT_ALONE, memlimit=1024)
+
+ # Test LZMADecompressor on known-good input data.
+
+ def test_decompress_good_input(self):
+ ddata = lzma.decompress(COMPRESSED_XZ)
+ self.assertEqual(ddata, INPUT)
+
+ ddata = lzma.decompress(COMPRESSED_ALONE)
+ self.assertEqual(ddata, INPUT)
+
+ ddata = lzma.decompress(COMPRESSED_XZ, lzma.FORMAT_XZ)
+ self.assertEqual(ddata, INPUT)
+
+ ddata = lzma.decompress(COMPRESSED_ALONE, lzma.FORMAT_ALONE)
+ self.assertEqual(ddata, INPUT)
+
+ ddata = lzma.decompress(
+ COMPRESSED_RAW_1, lzma.FORMAT_RAW, filters=FILTERS_RAW_1)
+ self.assertEqual(ddata, INPUT)
+
+ ddata = lzma.decompress(
+ COMPRESSED_RAW_2, lzma.FORMAT_RAW, filters=FILTERS_RAW_2)
+ self.assertEqual(ddata, INPUT)
+
+ ddata = lzma.decompress(
+ COMPRESSED_RAW_3, lzma.FORMAT_RAW, filters=FILTERS_RAW_3)
+ self.assertEqual(ddata, INPUT)
+
+ ddata = lzma.decompress(
+ COMPRESSED_RAW_4, lzma.FORMAT_RAW, filters=FILTERS_RAW_4)
+ self.assertEqual(ddata, INPUT)
+
+ def test_decompress_incomplete_input(self):
+ self.assertRaises(LZMAError, lzma.decompress, COMPRESSED_XZ[:128])
+ self.assertRaises(LZMAError, lzma.decompress, COMPRESSED_ALONE[:128])
+ self.assertRaises(LZMAError, lzma.decompress, COMPRESSED_RAW_1[:128],
+ format=lzma.FORMAT_RAW, filters=FILTERS_RAW_1)
+ self.assertRaises(LZMAError, lzma.decompress, COMPRESSED_RAW_2[:128],
+ format=lzma.FORMAT_RAW, filters=FILTERS_RAW_2)
+ self.assertRaises(LZMAError, lzma.decompress, COMPRESSED_RAW_3[:128],
+ format=lzma.FORMAT_RAW, filters=FILTERS_RAW_3)
+ self.assertRaises(LZMAError, lzma.decompress, COMPRESSED_RAW_4[:128],
+ format=lzma.FORMAT_RAW, filters=FILTERS_RAW_4)
+
+ def test_decompress_bad_input(self):
+ with self.assertRaises(LZMAError):
+ lzma.decompress(COMPRESSED_RAW_1)
+ with self.assertRaises(LZMAError):
+ lzma.decompress(COMPRESSED_ALONE, format=lzma.FORMAT_XZ)
+ with self.assertRaises(LZMAError):
+ lzma.decompress(COMPRESSED_XZ, format=lzma.FORMAT_ALONE)
+ with self.assertRaises(LZMAError):
+ lzma.decompress(COMPRESSED_XZ, format=lzma.FORMAT_RAW,
+ filters=FILTERS_RAW_1)
+
+ # Test that compress()->decompress() preserves the input data.
+
+ def test_roundtrip(self):
+ cdata = lzma.compress(INPUT)
+ ddata = lzma.decompress(cdata)
+ self.assertEqual(ddata, INPUT)
+
+ cdata = lzma.compress(INPUT, lzma.FORMAT_XZ)
+ ddata = lzma.decompress(cdata)
+ self.assertEqual(ddata, INPUT)
+
+ cdata = lzma.compress(INPUT, lzma.FORMAT_ALONE)
+ ddata = lzma.decompress(cdata)
+ self.assertEqual(ddata, INPUT)
+
+ cdata = lzma.compress(INPUT, lzma.FORMAT_RAW, filters=FILTERS_RAW_4)
+ ddata = lzma.decompress(cdata, lzma.FORMAT_RAW, filters=FILTERS_RAW_4)
+ self.assertEqual(ddata, INPUT)
+
+ # Unlike LZMADecompressor, decompress() *does* handle concatenated streams.
+
+ def test_decompress_multistream(self):
+ ddata = lzma.decompress(COMPRESSED_XZ + COMPRESSED_ALONE)
+ self.assertEqual(ddata, INPUT * 2)
+
+
+class TempFile:
+ """Context manager - creates a file, and deletes it on __exit__."""
+
+ def __init__(self, filename, data=b""):
+ self.filename = filename
+ self.data = data
+
+ def __enter__(self):
+ with open(self.filename, "wb") as f:
+ f.write(self.data)
+
+ def __exit__(self, *args):
+ unlink(self.filename)
+
+
+class FileTestCase(unittest.TestCase):
+
+ def test_init(self):
+ with LZMAFile(fileobj=BytesIO(COMPRESSED_XZ)) as f:
+ pass
+ with LZMAFile(fileobj=BytesIO(), mode="w") as f:
+ pass
+ with LZMAFile(fileobj=BytesIO(), mode="a") as f:
+ pass
+
+ def test_init_with_filename(self):
+ with TempFile(TESTFN, COMPRESSED_XZ):
+ with LZMAFile(TESTFN) as f:
+ pass
+ with LZMAFile(TESTFN, "w") as f:
+ pass
+ with LZMAFile(TESTFN, "a") as f:
+ pass
+
+ def test_init_bad_mode(self):
+ with self.assertRaises(ValueError):
+ LZMAFile(fileobj=BytesIO(COMPRESSED_XZ), mode=(3, "x"))
+ with self.assertRaises(ValueError):
+ LZMAFile(fileobj=BytesIO(COMPRESSED_XZ), mode="")
+ with self.assertRaises(ValueError):
+ LZMAFile(fileobj=BytesIO(COMPRESSED_XZ), mode="x")
+ with self.assertRaises(ValueError):
+ LZMAFile(fileobj=BytesIO(COMPRESSED_XZ), mode="rb")
+ with self.assertRaises(ValueError):
+ LZMAFile(fileobj=BytesIO(COMPRESSED_XZ), mode="r+")
+ with self.assertRaises(ValueError):
+ LZMAFile(fileobj=BytesIO(COMPRESSED_XZ), mode="wb")
+ with self.assertRaises(ValueError):
+ LZMAFile(fileobj=BytesIO(COMPRESSED_XZ), mode="w+")
+ with self.assertRaises(ValueError):
+ LZMAFile(fileobj=BytesIO(COMPRESSED_XZ), mode="rw")
+
+ def test_init_bad_check(self):
+ with self.assertRaises(TypeError):
+ LZMAFile(fileobj=BytesIO(), mode="w", check=b"asd")
+ # CHECK_UNKNOWN and anything above CHECK_ID_MAX should be invalid.
+ with self.assertRaises(LZMAError):
+ LZMAFile(fileobj=BytesIO(), mode="w", check=lzma.CHECK_UNKNOWN)
+ with self.assertRaises(LZMAError):
+ LZMAFile(fileobj=BytesIO(), mode="w", check=lzma.CHECK_ID_MAX + 3)
+ # Cannot specify a check with mode="r".
+ with self.assertRaises(ValueError):
+ LZMAFile(fileobj=BytesIO(COMPRESSED_XZ), check=lzma.CHECK_NONE)
+ with self.assertRaises(ValueError):
+ LZMAFile(fileobj=BytesIO(COMPRESSED_XZ), check=lzma.CHECK_CRC32)
+ with self.assertRaises(ValueError):
+ LZMAFile(fileobj=BytesIO(COMPRESSED_XZ), check=lzma.CHECK_CRC64)
+ with self.assertRaises(ValueError):
+ LZMAFile(fileobj=BytesIO(COMPRESSED_XZ), check=lzma.CHECK_SHA256)
+ with self.assertRaises(ValueError):
+ LZMAFile(fileobj=BytesIO(COMPRESSED_XZ), check=lzma.CHECK_UNKNOWN)
+
+ def test_init_bad_preset(self):
+ with self.assertRaises(TypeError):
+ LZMAFile(fileobj=BytesIO(), mode="w", preset=4.39)
+ with self.assertRaises(LZMAError):
+ LZMAFile(fileobj=BytesIO(), mode="w", preset=10)
+ with self.assertRaises(LZMAError):
+ LZMAFile(fileobj=BytesIO(), mode="w", preset=23)
+ with self.assertRaises(OverflowError):
+ LZMAFile(fileobj=BytesIO(), mode="w", preset=-1)
+ with self.assertRaises(OverflowError):
+ LZMAFile(fileobj=BytesIO(), mode="w", preset=-7)
+ with self.assertRaises(TypeError):
+ LZMAFile(fileobj=BytesIO(), mode="w", preset="foo")
+ # Cannot specify a preset with mode="r".
+ with self.assertRaises(ValueError):
+ LZMAFile(fileobj=BytesIO(COMPRESSED_XZ), preset=3)
+
+ def test_init_bad_filter_spec(self):
+ with self.assertRaises(TypeError):
+ LZMAFile(fileobj=BytesIO(), mode="w", filters=[b"wobsite"])
+ with self.assertRaises(ValueError):
+ LZMAFile(fileobj=BytesIO(), mode="w", filters=[{"xyzzy": 3}])
+ with self.assertRaises(ValueError):
+ LZMAFile(fileobj=BytesIO(), mode="w", filters=[{"id": 98765}])
+ with self.assertRaises(ValueError):
+ LZMAFile(fileobj=BytesIO(), mode="w",
+ filters=[{"id": lzma.FILTER_LZMA2, "foo": 0}])
+ with self.assertRaises(ValueError):
+ LZMAFile(fileobj=BytesIO(), mode="w",
+ filters=[{"id": lzma.FILTER_DELTA, "foo": 0}])
+ with self.assertRaises(ValueError):
+ LZMAFile(fileobj=BytesIO(), mode="w",
+ filters=[{"id": lzma.FILTER_X86, "foo": 0}])
+
+ def test_init_with_preset_and_filters(self):
+ with self.assertRaises(ValueError):
+ LZMAFile(fileobj=BytesIO(), mode="w", format=lzma.FORMAT_RAW,
+ preset=6, filters=FILTERS_RAW_1)
+
+ def test_init_with_filename_and_fileobj(self):
+ with self.assertRaises(ValueError):
+ LZMAFile("/dev/null", fileobj=BytesIO())
+
+ def test_close(self):
+ with BytesIO(COMPRESSED_XZ) as src:
+ f = LZMAFile(fileobj=src)
+ f.close()
+ # LZMAFile.close() should not close the underlying file object.
+ self.assertFalse(src.closed)
+ # Try closing an already-closed LZMAFile.
+ f.close()
+ self.assertFalse(src.closed)
+
+ # Test with a real file on disk, opened directly by LZMAFile.
+ with TempFile(TESTFN, COMPRESSED_XZ):
+ f = LZMAFile(TESTFN)
+ fp = f._fp
+ f.close()
+ # Here, LZMAFile.close() *should* close the underlying file object.
+ self.assertTrue(fp.closed)
+ # Try closing an already-closed LZMAFile.
+ f.close()
+
+ def test_closed(self):
+ f = LZMAFile(fileobj=BytesIO(COMPRESSED_XZ))
+ try:
+ self.assertFalse(f.closed)
+ f.read()
+ self.assertFalse(f.closed)
+ finally:
+ f.close()
+ self.assertTrue(f.closed)
+
+ f = LZMAFile(fileobj=BytesIO(), mode="w")
+ try:
+ self.assertFalse(f.closed)
+ finally:
+ f.close()
+ self.assertTrue(f.closed)
+
+ def test_fileno(self):
+ f = LZMAFile(fileobj=BytesIO(COMPRESSED_XZ))
+ try:
+ self.assertRaises(UnsupportedOperation, f.fileno)
+ finally:
+ f.close()
+ self.assertRaises(ValueError, f.fileno)
+ with TempFile(TESTFN, COMPRESSED_XZ):
+ f = LZMAFile(TESTFN)
+ try:
+ self.assertEqual(f.fileno(), f._fp.fileno())
+ self.assertIsInstance(f.fileno(), int)
+ finally:
+ f.close()
+ self.assertRaises(ValueError, f.fileno)
+
+ def test_seekable(self):
+ f = LZMAFile(fileobj=BytesIO(COMPRESSED_XZ))
+ try:
+ self.assertTrue(f.seekable())
+ f.read()
+ self.assertTrue(f.seekable())
+ finally:
+ f.close()
+ self.assertRaises(ValueError, f.seekable)
+
+ f = LZMAFile(fileobj=BytesIO(), mode="w")
+ try:
+ self.assertFalse(f.seekable())
+ finally:
+ f.close()
+ self.assertRaises(ValueError, f.seekable)
+
+ def test_readable(self):
+ f = LZMAFile(fileobj=BytesIO(COMPRESSED_XZ))
+ try:
+ self.assertTrue(f.readable())
+ f.read()
+ self.assertTrue(f.readable())
+ finally:
+ f.close()
+ self.assertRaises(ValueError, f.readable)
+
+ f = LZMAFile(fileobj=BytesIO(), mode="w")
+ try:
+ self.assertFalse(f.readable())
+ finally:
+ f.close()
+ self.assertRaises(ValueError, f.readable)
+
+ def test_writable(self):
+ f = LZMAFile(fileobj=BytesIO(COMPRESSED_XZ))
+ try:
+ self.assertFalse(f.writable())
+ f.read()
+ self.assertFalse(f.writable())
+ finally:
+ f.close()
+ self.assertRaises(ValueError, f.writable)
+
+ f = LZMAFile(fileobj=BytesIO(), mode="w")
+ try:
+ self.assertTrue(f.writable())
+ finally:
+ f.close()
+ self.assertRaises(ValueError, f.writable)
+
+ def test_read(self):
+ with LZMAFile(fileobj=BytesIO(COMPRESSED_XZ)) as f:
+ self.assertEqual(f.read(), INPUT)
+ self.assertEqual(f.read(), b"")
+ with LZMAFile(fileobj=BytesIO(COMPRESSED_ALONE)) as f:
+ self.assertEqual(f.read(), INPUT)
+ with LZMAFile(fileobj=BytesIO(COMPRESSED_XZ),
+ format=lzma.FORMAT_XZ) as f:
+ self.assertEqual(f.read(), INPUT)
+ self.assertEqual(f.read(), b"")
+ with LZMAFile(fileobj=BytesIO(COMPRESSED_ALONE),
+ format=lzma.FORMAT_ALONE) as f:
+ self.assertEqual(f.read(), INPUT)
+ self.assertEqual(f.read(), b"")
+ with LZMAFile(fileobj=BytesIO(COMPRESSED_RAW_1),
+ format=lzma.FORMAT_RAW, filters=FILTERS_RAW_1) as f:
+ self.assertEqual(f.read(), INPUT)
+ self.assertEqual(f.read(), b"")
+ with LZMAFile(fileobj=BytesIO(COMPRESSED_RAW_2),
+ format=lzma.FORMAT_RAW, filters=FILTERS_RAW_2) as f:
+ self.assertEqual(f.read(), INPUT)
+ self.assertEqual(f.read(), b"")
+ with LZMAFile(fileobj=BytesIO(COMPRESSED_RAW_3),
+ format=lzma.FORMAT_RAW, filters=FILTERS_RAW_3) as f:
+ self.assertEqual(f.read(), INPUT)
+ self.assertEqual(f.read(), b"")
+ with LZMAFile(fileobj=BytesIO(COMPRESSED_RAW_4),
+ format=lzma.FORMAT_RAW, filters=FILTERS_RAW_4) as f:
+ self.assertEqual(f.read(), INPUT)
+ self.assertEqual(f.read(), b"")
+
+ def test_read_0(self):
+ with LZMAFile(fileobj=BytesIO(COMPRESSED_XZ)) as f:
+ self.assertEqual(f.read(0), b"")
+ with LZMAFile(fileobj=BytesIO(COMPRESSED_ALONE)) as f:
+ self.assertEqual(f.read(0), b"")
+ with LZMAFile(fileobj=BytesIO(COMPRESSED_XZ),
+ format=lzma.FORMAT_XZ) as f:
+ self.assertEqual(f.read(0), b"")
+ with LZMAFile(fileobj=BytesIO(COMPRESSED_ALONE),
+ format=lzma.FORMAT_ALONE) as f:
+ self.assertEqual(f.read(0), b"")
+
+ def test_read_10(self):
+ with LZMAFile(fileobj=BytesIO(COMPRESSED_XZ)) as f:
+ chunks = []
+ while True:
+ result = f.read(10)
+ if not result:
+ break
+ self.assertLessEqual(len(result), 10)
+ chunks.append(result)
+ self.assertEqual(b"".join(chunks), INPUT)
+
+ def test_read_multistream(self):
+ with LZMAFile(fileobj=BytesIO(COMPRESSED_XZ * 5)) as f:
+ self.assertEqual(f.read(), INPUT * 5)
+ with LZMAFile(fileobj=BytesIO(COMPRESSED_XZ + COMPRESSED_ALONE)) as f:
+ self.assertEqual(f.read(), INPUT * 2)
+ with LZMAFile(fileobj=BytesIO(COMPRESSED_RAW_3 * 4),
+ format=lzma.FORMAT_RAW, filters=FILTERS_RAW_3) as f:
+ self.assertEqual(f.read(), INPUT * 4)
+
+ def test_read_multistream_buffer_size_aligned(self):
+ # Test the case where a stream boundary coincides with the end
+ # of the raw read buffer.
+ saved_buffer_size = lzma._BUFFER_SIZE
+ lzma._BUFFER_SIZE = len(COMPRESSED_XZ)
+ try:
+ with LZMAFile(fileobj=BytesIO(COMPRESSED_XZ * 5)) as f:
+ self.assertEqual(f.read(), INPUT * 5)
+ finally:
+ lzma._BUFFER_SIZE = saved_buffer_size
+
+ def test_read_from_file(self):
+ with TempFile(TESTFN, COMPRESSED_XZ):
+ with LZMAFile(TESTFN) as f:
+ self.assertEqual(f.read(), INPUT)
+ self.assertEqual(f.read(), b"")
+
+ def test_read_incomplete(self):
+ with LZMAFile(fileobj=BytesIO(COMPRESSED_XZ[:128])) as f:
+ self.assertRaises(EOFError, f.read)
+
+ def test_read_bad_args(self):
+ f = LZMAFile(fileobj=BytesIO(COMPRESSED_XZ))
+ f.close()
+ self.assertRaises(ValueError, f.read)
+ with LZMAFile(fileobj=BytesIO(), mode="w") as f:
+ self.assertRaises(ValueError, f.read)
+ with LZMAFile(fileobj=BytesIO(COMPRESSED_XZ)) as f:
+ self.assertRaises(TypeError, f.read, None)
+
+ def test_read1(self):
+ with LZMAFile(fileobj=BytesIO(COMPRESSED_XZ)) as f:
+ blocks = []
+ while True:
+ result = f.read1()
+ if not result:
+ break
+ blocks.append(result)
+ self.assertEqual(b"".join(blocks), INPUT)
+ self.assertEqual(f.read1(), b"")
+
+ def test_read1_0(self):
+ with LZMAFile(fileobj=BytesIO(COMPRESSED_XZ)) as f:
+ self.assertEqual(f.read1(0), b"")
+
+ def test_read1_10(self):
+ with LZMAFile(fileobj=BytesIO(COMPRESSED_XZ)) as f:
+ blocks = []
+ while True:
+ result = f.read1(10)
+ if not result:
+ break
+ blocks.append(result)
+ self.assertEqual(b"".join(blocks), INPUT)
+ self.assertEqual(f.read1(), b"")
+
+ def test_read1_multistream(self):
+ with LZMAFile(fileobj=BytesIO(COMPRESSED_XZ * 5)) as f:
+ blocks = []
+ while True:
+ result = f.read1()
+ if not result:
+ break
+ blocks.append(result)
+ self.assertEqual(b"".join(blocks), INPUT * 5)
+ self.assertEqual(f.read1(), b"")
+
+ def test_read1_bad_args(self):
+ f = LZMAFile(fileobj=BytesIO(COMPRESSED_XZ))
+ f.close()
+ self.assertRaises(ValueError, f.read1)
+ with LZMAFile(fileobj=BytesIO(), mode="w") as f:
+ self.assertRaises(ValueError, f.read1)
+ with LZMAFile(fileobj=BytesIO(COMPRESSED_XZ)) as f:
+ self.assertRaises(TypeError, f.read1, None)
+
+ def test_peek(self):
+ with LZMAFile(fileobj=BytesIO(COMPRESSED_XZ)) as f:
+ result = f.peek()
+ self.assertGreater(len(result), 0)
+ self.assertTrue(INPUT.startswith(result))
+ self.assertEqual(f.read(), INPUT)
+ with LZMAFile(fileobj=BytesIO(COMPRESSED_XZ)) as f:
+ result = f.peek(10)
+ self.assertGreater(len(result), 0)
+ self.assertTrue(INPUT.startswith(result))
+ self.assertEqual(f.read(), INPUT)
+
+ def test_peek_bad_args(self):
+ with LZMAFile(fileobj=BytesIO(), mode="w") as f:
+ self.assertRaises(ValueError, f.peek)
+
+ def test_iterator(self):
+ with BytesIO(INPUT) as f:
+ lines = f.readlines()
+ with LZMAFile(fileobj=BytesIO(COMPRESSED_XZ)) as f:
+ self.assertListEqual(list(iter(f)), lines)
+ with LZMAFile(fileobj=BytesIO(COMPRESSED_ALONE)) as f:
+ self.assertListEqual(list(iter(f)), lines)
+ with LZMAFile(fileobj=BytesIO(COMPRESSED_XZ),
+ format=lzma.FORMAT_XZ) as f:
+ self.assertListEqual(list(iter(f)), lines)
+ with LZMAFile(fileobj=BytesIO(COMPRESSED_ALONE),
+ format=lzma.FORMAT_ALONE) as f:
+ self.assertListEqual(list(iter(f)), lines)
+ with LZMAFile(fileobj=BytesIO(COMPRESSED_RAW_2),
+ format=lzma.FORMAT_RAW, filters=FILTERS_RAW_2) as f:
+ self.assertListEqual(list(iter(f)), lines)
+
+ def test_readline(self):
+ with BytesIO(INPUT) as f:
+ lines = f.readlines()
+ with LZMAFile(fileobj=BytesIO(COMPRESSED_XZ)) as f:
+ for line in lines:
+ self.assertEqual(f.readline(), line)
+
+ def test_readlines(self):
+ with BytesIO(INPUT) as f:
+ lines = f.readlines()
+ with LZMAFile(fileobj=BytesIO(COMPRESSED_XZ)) as f:
+ self.assertListEqual(f.readlines(), lines)
+
+ def test_write(self):
+ with BytesIO() as dst:
+ with LZMAFile(fileobj=dst, mode="w") as f:
+ f.write(INPUT)
+ expected = lzma.compress(INPUT)
+ self.assertEqual(dst.getvalue(), expected)
+ with BytesIO() as dst:
+ with LZMAFile(fileobj=dst, mode="w", format=lzma.FORMAT_XZ) as f:
+ f.write(INPUT)
+ expected = lzma.compress(INPUT, format=lzma.FORMAT_XZ)
+ self.assertEqual(dst.getvalue(), expected)
+ with BytesIO() as dst:
+ with LZMAFile(fileobj=dst, mode="w", format=lzma.FORMAT_ALONE) as f:
+ f.write(INPUT)
+ expected = lzma.compress(INPUT, format=lzma.FORMAT_ALONE)
+ self.assertEqual(dst.getvalue(), expected)
+ with BytesIO() as dst:
+ with LZMAFile(fileobj=dst, mode="w", format=lzma.FORMAT_RAW,
+ filters=FILTERS_RAW_2) as f:
+ f.write(INPUT)
+ expected = lzma.compress(INPUT, format=lzma.FORMAT_RAW,
+ filters=FILTERS_RAW_2)
+ self.assertEqual(dst.getvalue(), expected)
+
+ def test_write_10(self):
+ with BytesIO() as dst:
+ with LZMAFile(fileobj=dst, mode="w") as f:
+ for start in range(0, len(INPUT), 10):
+ f.write(INPUT[start:start+10])
+ expected = lzma.compress(INPUT)
+ self.assertEqual(dst.getvalue(), expected)
+
+ def test_write_append(self):
+ part1 = INPUT[:1024]
+ part2 = INPUT[1024:1536]
+ part3 = INPUT[1536:]
+ expected = b"".join(lzma.compress(x) for x in (part1, part2, part3))
+ with BytesIO() as dst:
+ with LZMAFile(fileobj=dst, mode="w") as f:
+ f.write(part1)
+ with LZMAFile(fileobj=dst, mode="a") as f:
+ f.write(part2)
+ with LZMAFile(fileobj=dst, mode="a") as f:
+ f.write(part3)
+ self.assertEqual(dst.getvalue(), expected)
+
+ def test_write_to_file(self):
+ try:
+ with LZMAFile(TESTFN, "w") as f:
+ f.write(INPUT)
+ expected = lzma.compress(INPUT)
+ with open(TESTFN, "rb") as f:
+ self.assertEqual(f.read(), expected)
+ finally:
+ unlink(TESTFN)
+
+ def test_write_append_to_file(self):
+ part1 = INPUT[:1024]
+ part2 = INPUT[1024:1536]
+ part3 = INPUT[1536:]
+ expected = b"".join(lzma.compress(x) for x in (part1, part2, part3))
+ try:
+ with LZMAFile(TESTFN, "w") as f:
+ f.write(part1)
+ with LZMAFile(TESTFN, "a") as f:
+ f.write(part2)
+ with LZMAFile(TESTFN, "a") as f:
+ f.write(part3)
+ with open(TESTFN, "rb") as f:
+ self.assertEqual(f.read(), expected)
+ finally:
+ unlink(TESTFN)
+
+ def test_write_bad_args(self):
+ f = LZMAFile(fileobj=BytesIO(), mode="w")
+ f.close()
+ self.assertRaises(ValueError, f.write, b"foo")
+ with LZMAFile(fileobj=BytesIO(COMPRESSED_XZ), mode="r") as f:
+ self.assertRaises(ValueError, f.write, b"bar")
+ with LZMAFile(fileobj=BytesIO(), mode="w") as f:
+ self.assertRaises(TypeError, f.write, None)
+ self.assertRaises(TypeError, f.write, "text")
+ self.assertRaises(TypeError, f.write, 789)
+
+ def test_writelines(self):
+ with BytesIO(INPUT) as f:
+ lines = f.readlines()
+ with BytesIO() as dst:
+ with LZMAFile(fileobj=dst, mode="w") as f:
+ f.writelines(lines)
+ expected = lzma.compress(INPUT)
+ self.assertEqual(dst.getvalue(), expected)
+
+ def test_seek_forward(self):
+ with LZMAFile(fileobj=BytesIO(COMPRESSED_XZ)) as f:
+ f.seek(555)
+ self.assertEqual(f.read(), INPUT[555:])
+
+ def test_seek_forward_across_streams(self):
+ with LZMAFile(fileobj=BytesIO(COMPRESSED_XZ * 2)) as f:
+ f.seek(len(INPUT) + 123)
+ self.assertEqual(f.read(), INPUT[123:])
+
+ def test_seek_forward_relative_to_current(self):
+ with LZMAFile(fileobj=BytesIO(COMPRESSED_XZ)) as f:
+ f.read(100)
+ f.seek(1236, 1)
+ self.assertEqual(f.read(), INPUT[1336:])
+
+ def test_seek_forward_relative_to_end(self):
+ with LZMAFile(fileobj=BytesIO(COMPRESSED_XZ)) as f:
+ f.seek(-555, 2)
+ self.assertEqual(f.read(), INPUT[-555:])
+
+ def test_seek_backward(self):
+ with LZMAFile(fileobj=BytesIO(COMPRESSED_XZ)) as f:
+ f.read(1001)
+ f.seek(211)
+ self.assertEqual(f.read(), INPUT[211:])
+
+ def test_seek_backward_across_streams(self):
+ with LZMAFile(fileobj=BytesIO(COMPRESSED_XZ * 2)) as f:
+ f.read(len(INPUT) + 333)
+ f.seek(737)
+ self.assertEqual(f.read(), INPUT[737:] + INPUT)
+
+ def test_seek_backward_relative_to_end(self):
+ with LZMAFile(fileobj=BytesIO(COMPRESSED_XZ)) as f:
+ f.seek(-150, 2)
+ self.assertEqual(f.read(), INPUT[-150:])
+
+ def test_seek_past_end(self):
+ with LZMAFile(fileobj=BytesIO(COMPRESSED_XZ)) as f:
+ f.seek(len(INPUT) + 9001)
+ self.assertEqual(f.tell(), len(INPUT))
+ self.assertEqual(f.read(), b"")
+
+ def test_seek_past_start(self):
+ with LZMAFile(fileobj=BytesIO(COMPRESSED_XZ)) as f:
+ f.seek(-88)
+ self.assertEqual(f.tell(), 0)
+ self.assertEqual(f.read(), INPUT)
+
+ def test_seek_bad_args(self):
+ f = LZMAFile(fileobj=BytesIO(COMPRESSED_XZ))
+ f.close()
+ self.assertRaises(ValueError, f.seek, 0)
+ with LZMAFile(fileobj=BytesIO(), mode="w") as f:
+ self.assertRaises(ValueError, f.seek, 0)
+ with LZMAFile(fileobj=BytesIO(COMPRESSED_XZ)) as f:
+ self.assertRaises(ValueError, f.seek, 0, 3)
+ self.assertRaises(ValueError, f.seek, 9, ())
+ self.assertRaises(TypeError, f.seek, None)
+ self.assertRaises(TypeError, f.seek, b"derp")
+
+ def test_tell(self):
+ with LZMAFile(fileobj=BytesIO(COMPRESSED_XZ)) as f:
+ pos = 0
+ while True:
+ self.assertEqual(f.tell(), pos)
+ result = f.read(183)
+ if not result:
+ break
+ pos += len(result)
+ self.assertEqual(f.tell(), len(INPUT))
+ with LZMAFile(fileobj=BytesIO(), mode="w") as f:
+ for pos in range(0, len(INPUT), 144):
+ self.assertEqual(f.tell(), pos)
+ f.write(INPUT[pos:pos+144])
+ self.assertEqual(f.tell(), len(INPUT))
+
+ def test_tell_bad_args(self):
+ f = LZMAFile(fileobj=BytesIO(COMPRESSED_XZ))
+ f.close()
+ self.assertRaises(ValueError, f.tell)
+
+
+class MiscellaneousTestCase(unittest.TestCase):
+
+ def test_is_check_supported(self):
+ # CHECK_NONE and CHECK_CRC32 should always be supported,
+ # regardless of the options liblzma was compiled with.
+ self.assertTrue(lzma.check_is_supported(lzma.CHECK_NONE))
+ self.assertTrue(lzma.check_is_supported(lzma.CHECK_CRC32))
+
+ # The .xz format spec cannot store check IDs above this value.
+ self.assertFalse(lzma.check_is_supported(lzma.CHECK_ID_MAX + 1))
+
+ # This value should not be a valid check ID.
+ self.assertFalse(lzma.check_is_supported(lzma.CHECK_UNKNOWN))
+
+
+# Test data:
+
+INPUT = b"""
+LAERTES
+
+ O, fear me not.
+ I stay too long: but here my father comes.
+
+ Enter POLONIUS
+
+ A double blessing is a double grace,
+ Occasion smiles upon a second leave.
+
+LORD POLONIUS
+
+ Yet here, Laertes! aboard, aboard, for shame!
+ The wind sits in the shoulder of your sail,
+ And you are stay'd for. There; my blessing with thee!
+ And these few precepts in thy memory
+ See thou character. Give thy thoughts no tongue,
+ Nor any unproportioned thought his act.
+ Be thou familiar, but by no means vulgar.
+ Those friends thou hast, and their adoption tried,
+ Grapple them to thy soul with hoops of steel;
+ But do not dull thy palm with entertainment
+ Of each new-hatch'd, unfledged comrade. Beware
+ Of entrance to a quarrel, but being in,
+ Bear't that the opposed may beware of thee.
+ Give every man thy ear, but few thy voice;
+ Take each man's censure, but reserve thy judgment.
+ Costly thy habit as thy purse can buy,
+ But not express'd in fancy; rich, not gaudy;
+ For the apparel oft proclaims the man,
+ And they in France of the best rank and station
+ Are of a most select and generous chief in that.
+ Neither a borrower nor a lender be;
+ For loan oft loses both itself and friend,
+ And borrowing dulls the edge of husbandry.
+ This above all: to thine ownself be true,
+ And it must follow, as the night the day,
+ Thou canst not then be false to any man.
+ Farewell: my blessing season this in thee!
+
+LAERTES
+
+ Most humbly do I take my leave, my lord.
+
+LORD POLONIUS
+
+ The time invites you; go; your servants tend.
+
+LAERTES
+
+ Farewell, Ophelia; and remember well
+ What I have said to you.
+
+OPHELIA
+
+ 'Tis in my memory lock'd,
+ And you yourself shall keep the key of it.
+
+LAERTES
+
+ Farewell.
+"""
+
+COMPRESSED_XZ = (
+ b"\xfd7zXZ\x00\x00\x04\xe6\xd6\xb4F\x02\x00!\x01\x16\x00\x00\x00t/\xe5\xa3"
+ b"\xe0\x07\x80\x03\xdf]\x00\x05\x14\x07bX\x19\xcd\xddn\x98\x15\xe4\xb4\x9d"
+ b"o\x1d\xc4\xe5\n\x03\xcc2h\xc7\\\x86\xff\xf8\xe2\xfc\xe7\xd9\xfe6\xb8("
+ b"\xa8wd\xc2\"u.n\x1e\xc3\xf2\x8e\x8d\x8f\x02\x17/\xa6=\xf0\xa2\xdf/M\x89"
+ b"\xbe\xde\xa7\x1cz\x18-]\xd5\xef\x13\x8frZ\x15\x80\x8c\xf8\x8do\xfa\x12"
+ b"\x9b#z/\xef\xf0\xfaF\x01\x82\xa3M\x8e\xa1t\xca6 BF$\xe5Q\xa4\x98\xee\xde"
+ b"l\xe8\x7f\xf0\x9d,bn\x0b\x13\xd4\xa8\x81\xe4N\xc8\x86\x153\xf5x2\xa2O"
+ b"\x13@Q\xa1\x00/\xa5\xd0O\x97\xdco\xae\xf7z\xc4\xcdS\xb6t<\x16\xf2\x9cI#"
+ b"\x89ud\xc66Y\xd9\xee\xe6\xce\x12]\xe5\xf0\xaa\x96-Pe\xade:\x04\t\x1b\xf7"
+ b"\xdb7\n\x86\x1fp\xc8J\xba\xf4\xf0V\xa9\xdc\xf0\x02%G\xf9\xdf=?\x15\x1b"
+ b"\xe1(\xce\x82=\xd6I\xac3\x12\x0cR\xb7\xae\r\xb1i\x03\x95\x01\xbd\xbe\xfa"
+ b"\x02s\x01P\x9d\x96X\xb12j\xc8L\xa8\x84b\xf6\xc3\xd4c-H\x93oJl\xd0iQ\xe4k"
+ b"\x84\x0b\xc1\xb7\xbc\xb1\x17\x88\xb1\xca?@\xf6\x07\xea\xe6x\xf1H12P\x0f"
+ b"\x8a\xc9\xeauw\xe3\xbe\xaai\xa9W\xd0\x80\xcd#cb5\x99\xd8]\xa9d\x0c\xbd"
+ b"\xa2\xdcWl\xedUG\xbf\x89yF\xf77\x81v\xbd5\x98\xbeh8\x18W\x08\xf0\x1b\x99"
+ b"5:\x1a?rD\x96\xa1\x04\x0f\xae\xba\x85\xeb\x9d5@\xf5\x83\xd37\x83\x8ac"
+ b"\x06\xd4\x97i\xcdt\x16S\x82k\xf6K\x01vy\x88\x91\x9b6T\xdae\r\xfd]:k\xbal"
+ b"\xa9\xbba\xc34\xf9r\xeb}r\xdb\xc7\xdb*\x8f\x03z\xdc8h\xcc\xc9\xd3\xbcl"
+ b"\xa5-\xcb\xeaK\xa2\xc5\x15\xc0\xe3\xc1\x86Z\xfb\xebL\xe13\xcf\x9c\xe3"
+ b"\x1d\xc9\xed\xc2\x06\xcc\xce!\x92\xe5\xfe\x9c^\xa59w \x9bP\xa3PK\x08d"
+ b"\xf9\xe2Z}\xa7\xbf\xed\xeb%$\x0c\x82\xb8/\xb0\x01\xa9&,\xf7qh{Q\x96)\xf2"
+ b"q\x96\xc3\x80\xb4\x12\xb0\xba\xe6o\xf4!\xb4[\xd4\x8aw\x10\xf7t\x0c\xb3"
+ b"\xd9\xd5\xc3`^\x81\x11??\\\xa4\x99\x85R\xd4\x8e\x83\xc9\x1eX\xbfa\xf1"
+ b"\xac\xb0\xea\xea\xd7\xd0\xab\x18\xe2\xf2\xed\xe1\xb7\xc9\x18\xcbS\xe4>"
+ b"\xc9\x95H\xe8\xcb\t\r%\xeb\xc7$.o\xf1\xf3R\x17\x1db\xbb\xd8U\xa5^\xccS"
+ b"\x16\x01\x87\xf3/\x93\xd1\xf0v\xc0r\xd7\xcc\xa2Gkz\xca\x80\x0e\xfd\xd0"
+ b"\x8b\xbb\xd2Ix\xb3\x1ey\xca-0\xe3z^\xd6\xd6\x8f_\xf1\x9dP\x9fi\xa7\xd1"
+ b"\xe8\x90\x84\xdc\xbf\xcdky\x8e\xdc\x81\x7f\xa3\xb2+\xbf\x04\xef\xd8\\"
+ b"\xc4\xdf\xe1\xb0\x01\xe9\x93\xe3Y\xf1\x1dY\xe8h\x81\xcf\xf1w\xcc\xb4\xef"
+ b" \x8b|\x04\xea\x83ej\xbe\x1f\xd4z\x9c`\xd3\x1a\x92A\x06\xe5\x8f\xa9\x13"
+ b"\t\x9e=\xfa\x1c\xe5_\x9f%v\x1bo\x11ZO\xd8\xf4\t\xddM\x16-\x04\xfc\x18<\""
+ b"CM\xddg~b\xf6\xef\x8e\x0c\xd0\xde|\xa0'\x8a\x0c\xd6x\xae!J\xa6F\x88\x15u"
+ b"\x008\x17\xbc7y\xb3\xd8u\xac_\x85\x8d\xe7\xc1@\x9c\xecqc\xa3#\xad\xf1"
+ b"\x935\xb5)_\r\xec3]\x0fo]5\xd0my\x07\x9b\xee\x81\xb5\x0f\xcfK+\x00\xc0"
+ b"\xe4b\x10\xe4\x0c\x1a \x9b\xe0\x97t\xf6\xa1\x9e\x850\xba\x0c\x9a\x8d\xc8"
+ b"\x8f\x07\xd7\xae\xc8\xf9+i\xdc\xb9k\xb0>f\x19\xb8\r\xa8\xf8\x1f$\xa5{p"
+ b"\xc6\x880\xce\xdb\xcf\xca_\x86\xac\x88h6\x8bZ%'\xd0\n\xbf\x0f\x9c\"\xba"
+ b"\xe5\x86\x9f\x0f7X=mNX[\xcc\x19FU\xc9\x860\xbc\x90a+* \xae_$\x03\x1e\xd3"
+ b"\xcd_\xa0\x9c\xde\xaf46q\xa5\xc9\x92\xd7\xca\xe3`\x9d\x85}\xb4\xff\xb3"
+ b"\x83\xfb\xb6\xca\xae`\x0bw\x7f\xfc\xd8\xacVe\x19\xc8\x17\x0bZ\xad\x88"
+ b"\xeb#\x97\x03\x13\xb1d\x0f{\x0c\x04w\x07\r\x97\xbd\xd6\xc1\xc3B:\x95\x08"
+ b"^\x10V\xaeaH\x02\xd9\xe3\n\\\x01X\xf6\x9c\x8a\x06u#%\xbe*\xa1\x18v\x85"
+ b"\xec!\t4\x00\x00\x00\x00Vj?uLU\xf3\xa6\x00\x01\xfb\x07\x81\x0f\x00\x00tw"
+ b"\x99P\xb1\xc4g\xfb\x02\x00\x00\x00\x00\x04YZ"
+)
+
+COMPRESSED_ALONE = (
+ b"]\x00\x00\x80\x00\xff\xff\xff\xff\xff\xff\xff\xff\x00\x05\x14\x07bX\x19"
+ b"\xcd\xddn\x98\x15\xe4\xb4\x9do\x1d\xc4\xe5\n\x03\xcc2h\xc7\\\x86\xff\xf8"
+ b"\xe2\xfc\xe7\xd9\xfe6\xb8(\xa8wd\xc2\"u.n\x1e\xc3\xf2\x8e\x8d\x8f\x02"
+ b"\x17/\xa6=\xf0\xa2\xdf/M\x89\xbe\xde\xa7\x1cz\x18-]\xd5\xef\x13\x8frZ"
+ b"\x15\x80\x8c\xf8\x8do\xfa\x12\x9b#z/\xef\xf0\xfaF\x01\x82\xa3M\x8e\xa1t"
+ b"\xca6 BF$\xe5Q\xa4\x98\xee\xdel\xe8\x7f\xf0\x9d,bn\x0b\x13\xd4\xa8\x81"
+ b"\xe4N\xc8\x86\x153\xf5x2\xa2O\x13@Q\xa1\x00/\xa5\xd0O\x97\xdco\xae\xf7z"
+ b"\xc4\xcdS\xb6t<\x16\xf2\x9cI#\x89ud\xc66Y\xd9\xee\xe6\xce\x12]\xe5\xf0"
+ b"\xaa\x96-Pe\xade:\x04\t\x1b\xf7\xdb7\n\x86\x1fp\xc8J\xba\xf4\xf0V\xa9"
+ b"\xdc\xf0\x02%G\xf9\xdf=?\x15\x1b\xe1(\xce\x82=\xd6I\xac3\x12\x0cR\xb7"
+ b"\xae\r\xb1i\x03\x95\x01\xbd\xbe\xfa\x02s\x01P\x9d\x96X\xb12j\xc8L\xa8"
+ b"\x84b\xf8\x1epl\xeajr\xd1=\t\x03\xdd\x13\x1b3!E\xf9vV\xdaF\xf3\xd7\xb4"
+ b"\x0c\xa9P~\xec\xdeE\xe37\xf6\x1d\xc6\xbb\xddc%\xb6\x0fI\x07\xf0;\xaf\xe7"
+ b"\xa0\x8b\xa7Z\x99(\xe9\xe2\xf0o\x18>`\xe1\xaa\xa8\xd9\xa1\xb2}\xe7\x8d"
+ b"\x834T\xb6\xef\xc1\xde\xe3\x98\xbcD\x03MA@\xd8\xed\xdc\xc8\x93\x03\x1a"
+ b"\x93\x0b\x7f\x94\x12\x0b\x02Sa\x18\xc9\xc5\x9bTJE}\xf6\xc8g\x17#ZV\x01"
+ b"\xc9\x9dc\x83\x0e>0\x16\x90S\xb8/\x03y_\x18\xfa(\xd7\x0br\xa2\xb0\xba?"
+ b"\x8c\xe6\x83@\x84\xdf\x02:\xc5z\x9e\xa6\x84\xc9\xf5BeyX\x83\x1a\xf1 :\t"
+ b"\xf7\x19\xfexD\\&G\xf3\x85Y\xa2J\xf9\x0bv{\x89\xf6\xe7)A\xaf\x04o\x00"
+ b"\x075\xd3\xe0\x7f\x97\x98F\x0f?v\x93\xedVtTf\xb5\x97\x83\xed\x19\xd7\x1a"
+ b"'k\xd7\xd9\xc5\\Y\xd1\xdc\x07\x15|w\xbc\xacd\x87\x08d\xec\xa7\xf6\x82"
+ b"\xfc\xb3\x93\xeb\xb9 \x8d\xbc ,\xb3X\xb0\xd2s\xd7\xd1\xffv\x05\xdf}\xa2"
+ b"\x96\xfb%\n\xdf\xa2\x7f\x08.\xa16\n\xe0\x19\x93\x7fh\n\x1c\x8c\x0f \x11"
+ b"\xc6Bl\x95\x19U}\xe4s\xb5\x10H\xea\x86pB\xe88\x95\xbe\x8cZ\xdb\xe4\x94A"
+ b"\x92\xb9;z\xaa\xa7{\x1c5!\xc0\xaf\xc1A\xf9\xda\xf0$\xb0\x02qg\xc8\xc7/|"
+ b"\xafr\x99^\x91\x88\xbf\x03\xd9=\xd7n\xda6{>8\n\xc7:\xa9'\xba.\x0b\xe2"
+ b"\xb5\x1d\x0e\n\x9a\x8e\x06\x8f:\xdd\x82'[\xc3\"wD$\xa7w\xecq\x8c,1\x93"
+ b"\xd0,\xae2w\x93\x12$Jd\x19mg\x02\x93\x9cA\x95\x9d&\xca8i\x9c\xb0;\xe7NQ"
+ b"\x1frh\x8beL;\xb0m\xee\x07Q\x9b\xc6\xd8\x03\xb5\xdeN\xd4\xfe\x98\xd0\xdc"
+ b"\x1a[\x04\xde\x1a\xf6\x91j\xf8EOli\x8eB^\x1d\x82\x07\xb2\xb5R]\xb7\xd7"
+ b"\xe9\xa6\xc3.\xfb\xf0-\xb4e\x9b\xde\x03\x88\xc6\xc1iN\x0e\x84wbQ\xdf~"
+ b"\xe9\xa4\x884\x96kM\xbc)T\xf3\x89\x97\x0f\x143\xe7)\xa0\xb3B\x00\xa8\xaf"
+ b"\x82^\xcb\xc7..\xdb\xc7\t\x9dH\xee5\xe9#\xe6NV\x94\xcb$Kk\xe3\x7f\r\xe3t"
+ b"\x12\xcf'\xefR\x8b\xf42\xcf-LH\xac\xe5\x1f0~?SO\xeb\xc1E\x1a\x1c]\xf2"
+ b"\xc4<\x11\x02\x10Z0a*?\xe4r\xff\xfb\xff\xf6\x14nG\xead^\xd6\xef8\xb6uEI"
+ b"\x99\nV\xe2\xb3\x95\x8e\x83\xf6i!\xb5&1F\xb1DP\xf4 SO3D!w\x99_G\x7f+\x90"
+ b".\xab\xbb]\x91>\xc9#h;\x0f5J\x91K\xf4^-[\x9e\x8a\\\x94\xca\xaf\xf6\x19"
+ b"\xd4\xa1\x9b\xc4\xb8p\xa1\xae\x15\xe9r\x84\xe0\xcar.l []\x8b\xaf+0\xf2g"
+ b"\x01aKY\xdfI\xcf,\n\xe8\xf0\xe7V\x80_#\xb2\xf2\xa9\x06\x8c>w\xe2W,\xf4"
+ b"\x8c\r\xf963\xf5J\xcc2\x05=kT\xeaUti\xe5_\xce\x1b\xfa\x8dl\x02h\xef\xa8"
+ b"\xfbf\x7f\xff\xf0\x19\xeax"
+)
+
+FILTERS_RAW_1 = [{"id": lzma.FILTER_LZMA2, "preset": 3}]
+COMPRESSED_RAW_1 = (
+ b"\xe0\x07\x80\x03\xfd]\x00\x05\x14\x07bX\x19\xcd\xddn\x96cyq\xa1\xdd\xee"
+ b"\xf8\xfam\xe3'\x88\xd3\xff\xe4\x9e \xceQ\x91\xa4\x14I\xf6\xb9\x9dVL8\x15"
+ b"_\x0e\x12\xc3\xeb\xbc\xa5\xcd\nW\x1d$=R;\x1d\xf8k8\t\xb1{\xd4\xc5+\x9d"
+ b"\x87c\xe5\xef\x98\xb4\xd7S3\xcd\xcc\xd2\xed\xa4\x0em\xe5\xf4\xdd\xd0b"
+ b"\xbe4*\xaa\x0b\xc5\x08\x10\x85+\x81.\x17\xaf9\xc9b\xeaZrA\xe20\x7fs\"r"
+ b"\xdaG\x81\xde\x90cu\xa5\xdb\xa9.A\x08l\xb0<\xf6\x03\xddOi\xd0\xc5\xb4"
+ b"\xec\xecg4t6\"\xa6\xb8o\xb5?\x18^}\xb6}\x03[:\xeb\x03\xa9\n[\x89l\x19g"
+ b"\x16\xc82\xed\x0b\xfb\x86n\xa2\x857@\x93\xcd6T\xc3u\xb0\t\xf9\x1b\x918"
+ b"\xfc[\x1b\x1e4\xb3\x14\x06PCV\xa8\"\xf5\x81x~\xe9\xb5N\x9cK\x9f\xc6\xc3%"
+ b"\xc8k:{6\xe7\xf7\xbd\x05\x02\xb4\xc4\xc3\xd3\xfd\xc3\xa8\\\xfc@\xb1F_"
+ b"\xc8\x90\xd9sU\x98\xad8\x05\x07\xde7J\x8bM\xd0\xb3;X\xec\x87\xef\xae\xb3"
+ b"eO,\xb1z,d\x11y\xeejlB\x02\x1d\xf28\x1f#\x896\xce\x0b\xf0\xf5\xa9PK\x0f"
+ b"\xb3\x13P\xd8\x88\xd2\xa1\x08\x04C?\xdb\x94_\x9a\"\xe9\xe3e\x1d\xde\x9b"
+ b"\xa1\xe8>H\x98\x10;\xc5\x03#\xb5\x9d4\x01\xe7\xc5\xba%v\xa49\x97A\xe0\""
+ b"\x8c\xc22\xe3i\xc1\x9d\xab3\xdf\xbe\xfdDm7\x1b\x9d\xab\xb5\x15o:J\x92"
+ b"\xdb\x816\x17\xc2O\x99\x1b\x0e\x8d\xf3\tQ\xed\x8e\x95S/\x16M\xb2S\x04"
+ b"\x0f\xc3J\xc6\xc7\xe4\xcb\xc5\xf4\xe7d\x14\xe4=^B\xfb\xd3E\xd3\x1e\xcd"
+ b"\x91\xa5\xd0G\x8f.\xf6\xf9\x0bb&\xd9\x9f\xc2\xfdj\xa2\x9e\xc4\\\x0e\x1dC"
+ b"v\xe8\xd2\x8a?^H\xec\xae\xeb>\xfe\xb8\xab\xd4IqY\x8c\xd4K7\x11\xf4D\xd0W"
+ b"\xa5\xbe\xeaO\xbf\xd0\x04\xfdl\x10\xae5\xd4U\x19\x06\xf9{\xaa\xe0\x81"
+ b"\x0f\xcf\xa3k{\x95\xbd\x19\xa2\xf8\xe4\xa3\x08O*\xf1\xf1B-\xc7(\x0eR\xfd"
+ b"@E\x9f\xd3\x1e:\xfdV\xb7\x04Y\x94\xeb]\x83\xc4\xa5\xd7\xc0gX\x98\xcf\x0f"
+ b"\xcd3\x00]n\x17\xec\xbd\xa3Y\x86\xc5\xf3u\xf6*\xbdT\xedA$A\xd9A\xe7\x98"
+ b"\xef\x14\x02\x9a\xfdiw\xec\xa0\x87\x11\xd9%\xc5\xeb\x8a=\xae\xc0\xc4\xc6"
+ b"D\x80\x8f\xa8\xd1\xbbq\xb2\xc0\xa0\xf5Cqp\xeeL\xe3\xe5\xdc \x84\"\xe9"
+ b"\x80t\x83\x05\xba\xf1\xc5~\x93\xc9\xf0\x01c\xceix\x9d\xed\xc5)l\x16)\xd1"
+ b"\x03@l\x04\x7f\x87\xa5yn\x1b\x01D\xaa:\xd2\x96\xb4\xb3?\xb0\xf9\xce\x07"
+ b"\xeb\x81\x00\xe4\xc3\xf5%_\xae\xd4\xf9\xeb\xe2\rh\xb2#\xd67Q\x16D\x82hn"
+ b"\xd1\xa3_?q\xf0\xe2\xac\xf317\x9e\xd0_\x83|\xf1\xca\xb7\x95S\xabW\x12"
+ b"\xff\xddt\xf69L\x01\xf2|\xdaW\xda\xees\x98L\x18\xb8_\xe8$\x82\xea\xd6"
+ b"\xd1F\xd4\x0b\xcdk\x01vf\x88h\xc3\xae\xb91\xc7Q\x9f\xa5G\xd9\xcc\x1f\xe3"
+ b"5\xb1\xdcy\x7fI\x8bcw\x8e\x10rIp\x02:\x19p_\xc8v\xcea\"\xc1\xd9\x91\x03"
+ b"\xbfe\xbe\xa6\xb3\xa8\x14\x18\xc3\xabH*m}\xc2\xc1\x9a}>l%\xce\x84\x99"
+ b"\xb3d\xaf\xd3\x82\x15\xdf\xc1\xfc5fOg\x9b\xfc\x8e^&\t@\xce\x9f\x06J\xb8"
+ b"\xb5\x86\x1d\xda{\x9f\xae\xb0\xff\x02\x81r\x92z\x8cM\xb7ho\xc9^\x9c\xb6"
+ b"\x9c\xae\xd1\xc9\xf4\xdfU7\xd6\\!\xea\x0b\x94k\xb9Ud~\x98\xe7\x86\x8az"
+ b"\x10;\xe3\x1d\xe5PG\xf8\xa4\x12\x05w\x98^\xc4\xb1\xbb\xfb\xcf\xe0\x7f"
+ b"\x033Sf\x0c \xb1\xf6@\x94\xe5\xa3\xb2\xa7\x10\x9a\xc0\x14\xc3s\xb5xRD"
+ b"\xf4`W\xd9\xe5\xd3\xcf\x91\rTZ-X\xbe\xbf\xb5\xe2\xee|\x1a\xbf\xfb\x08"
+ b"\x91\xe1\xfc\x9a\x18\xa3\x8b\xd6^\x89\xf5[\xef\x87\xd1\x06\x1c7\xd6\xa2"
+ b"\t\tQ5/@S\xc05\xd2VhAK\x03VC\r\x9b\x93\xd6M\xf1xO\xaaO\xed\xb9<\x0c\xdae"
+ b"*\xd0\x07Hk6\x9fG+\xa1)\xcd\x9cl\x87\xdb\xe1\xe7\xefK}\x875\xab\xa0\x19u"
+ b"\xf6*F\xb32\x00\x00\x00"
+)
+
+FILTERS_RAW_2 = [{"id": lzma.FILTER_DELTA, "dist": 2},
+ {"id": lzma.FILTER_LZMA2,
+ "preset": lzma.PRESET_DEFAULT | lzma.PRESET_EXTREME}]
+COMPRESSED_RAW_2 = (
+ b"\xe0\x07\x80\x05\x91]\x00\x05\x14\x06-\xd4\xa8d?\xef\xbe\xafH\xee\x042"
+ b"\xcb.\xb5g\x8f\xfb\x14\xab\xa5\x9f\x025z\xa4\xdd\xd8\t[}W\xf8\x0c\x1dmH"
+ b"\xfa\x05\xfcg\xba\xe5\x01Q\x0b\x83R\xb6A\x885\xc0\xba\xee\n\x1cv~\xde:o"
+ b"\x06:J\xa7\x11Cc\xea\xf7\xe5*o\xf7\x83\\l\xbdE\x19\x1f\r\xa8\x10\xb42"
+ b"\x0caU{\xd7\xb8w\xdc\xbe\x1b\xfc8\xb4\xcc\xd38\\\xf6\x13\xf6\xe7\x98\xfa"
+ b"\xc7[\x17_9\x86%\xa8\xf8\xaa\xb8\x8dfs#\x1e=\xed<\x92\x10\\t\xff\x86\xfb"
+ b"=\x9e7\x18\x1dft\\\xb5\x01\x95Q\xc5\x19\xb38\xe0\xd4\xaa\x07\xc3\x7f\xd8"
+ b"\xa2\x00>-\xd3\x8e\xa1#\xfa\x83ArAm\xdbJ~\x93\xa3B\x82\xe0\xc7\xcc(\x08`"
+ b"WK\xad\x1b\x94kaj\x04 \xde\xfc\xe1\xed\xb0\x82\x91\xefS\x84%\x86\xfbi"
+ b"\x99X\xf1B\xe7\x90;E\xfde\x98\xda\xca\xd6T\xb4bg\xa4\n\x9aj\xd1\x83\x9e]"
+ b"\"\x7fM\xb5\x0fr\xd2\\\xa5j~P\x10GH\xbfN*Z\x10.\x81\tpE\x8a\x08\xbe1\xbd"
+ b"\xcd\xa9\xe1\x8d\x1f\x04\xf9\x0eH\xb9\xae\xd6\xc3\xc1\xa5\xa9\x95P\xdc~"
+ b"\xff\x01\x930\xa9\x04\xf6\x03\xfe\xb5JK\xc3]\xdd9\xb1\xd3\xd7F\xf5\xd1"
+ b"\x1e\xa0\x1c_\xed[\x0c\xae\xd4\x8b\x946\xeb\xbf\xbb\xe3$kS{\xb5\x80,f:Sj"
+ b"\x0f\x08z\x1c\xf5\xe8\xe6\xae\x98\xb0Q~r\x0f\xb0\x05?\xb6\x90\x19\x02&"
+ b"\xcb\x80\t\xc4\xea\x9c|x\xce\x10\x9c\xc5|\xcbdhh+\x0c'\xc5\x81\xc33\xb5"
+ b"\x14q\xd6\xc5\xe3`Z#\xdc\x8a\xab\xdd\xea\x08\xc2I\xe7\x02l{\xec\x196\x06"
+ b"\x91\x8d\xdc\xd5\xb3x\xe1hz%\xd1\xf8\xa5\xdd\x98!\x8c\x1c\xc1\x17RUa\xbb"
+ b"\x95\x0f\xe4X\xea1\x0c\xf1=R\xbe\xc60\xe3\xa4\x9a\x90bd\x97$]B\x01\xdd"
+ b"\x1f\xe3h2c\x1e\xa0L`4\xc6x\xa3Z\x8a\r\x14]T^\xd8\x89\x1b\x92\r;\xedY"
+ b"\x0c\xef\x8d9z\xf3o\xb6)f\xa9]$n\rp\x93\xd0\x10\xa4\x08\xb8\xb2\x8b\xb6"
+ b"\x8f\x80\xae;\xdcQ\xf1\xfa\x9a\x06\x8e\xa5\x0e\x8cK\x9c @\xaa:UcX\n!\xc6"
+ b"\x02\x12\xcb\x1b\"=\x16.\x1f\x176\xf2g=\xe1Wn\xe9\xe1\xd4\xf1O\xad\x15"
+ b"\x86\xe9\xa3T\xaf\xa9\xd7D\xb5\xd1W3pnt\x11\xc7VOj\xb7M\xc4i\xa1\xf1$3"
+ b"\xbb\xdc\x8af\xb0\xc5Y\r\xd1\xfb\xf2\xe7K\xe6\xc5hwO\xfe\x8c2^&\x07\xd5"
+ b"\x1fV\x19\xfd\r\x14\xd2i=yZ\xe6o\xaf\xc6\xb6\x92\x9d\xc4\r\xb3\xafw\xac%"
+ b"\xcfc\x1a\xf1`]\xf2\x1a\x9e\x808\xedm\xedQ\xb2\xfe\xe4h`[q\xae\xe0\x0f"
+ b"\xba0g\xb6\"N\xc3\xfb\xcfR\x11\xc5\x18)(\xc40\\\xa3\x02\xd9G!\xce\x1b"
+ b"\xc1\x96x\xb5\xc8z\x1f\x01\xb4\xaf\xde\xc2\xcd\x07\xe7H\xb3y\xa8M\n\\A\t"
+ b"ar\xddM\x8b\x9a\xea\x84\x9b!\xf1\x8d\xb1\xf1~\x1e\r\xa5H\xba\xf1\x84o"
+ b"\xda\x87\x01h\xe9\xa2\xbe\xbeqN\x9d\x84\x0b!WG\xda\xa1\xa5A\xb7\xc7`j"
+ b"\x15\xf2\xe9\xdd?\x015B\xd2~E\x06\x11\xe0\x91!\x05^\x80\xdd\xa8y\x15}"
+ b"\xa1)\xb1)\x81\x18\xf4\xf4\xf8\xc0\xefD\xe3\xdb2f\x1e\x12\xabu\xc9\x97"
+ b"\xcd\x1e\xa7\x0c\x02x4_6\x03\xc4$t\xf39\x94\x1d=\xcb\xbfv\\\xf5\xa3\x1d"
+ b"\x9d8jk\x95\x13)ff\xf9n\xc4\xa9\xe3\x01\xb8\xda\xfb\xab\xdfM\x99\xfb\x05"
+ b"\xe0\xe9\xb0I\xf4E\xab\xe2\x15\xa3\x035\xe7\xdeT\xee\x82p\xb4\x88\xd3"
+ b"\x893\x9c/\xc0\xd6\x8fou;\xf6\x95PR\xa9\xb2\xc1\xefFj\xe2\xa7$\xf7h\xf1"
+ b"\xdfK(\xc9c\xba7\xe8\xe3)\xdd\xb2,\x83\xfb\x84\x18.y\x18Qi\x88\xf8`h-"
+ b"\xef\xd5\xed\x8c\t\xd8\xc3^\x0f\x00\xb7\xd0[!\xafM\x9b\xd7.\x07\xd8\xfb"
+ b"\xd9\xe2-S+\xaa8,\xa0\x03\x1b \xea\xa8\x00\xc3\xab~\xd0$e\xa5\x7f\xf7"
+ b"\x95P]\x12\x19i\xd9\x7fo\x0c\xd8g^\rE\xa5\x80\x18\xc5\x01\x80\xaek`\xff~"
+ b"\xb6y\xe7+\xe5\x11^D\xa7\x85\x18\"!\xd6\xd2\xa7\xf4\x1eT\xdb\x02\xe15"
+ b"\x02Y\xbc\x174Z\xe7\x9cH\x1c\xbf\x0f\xc6\xe9f]\xcf\x8cx\xbc\xe5\x15\x94"
+ b"\xfc3\xbc\xa7TUH\xf1\x84\x1b\xf7\xa9y\xc07\x84\xf8X\xd8\xef\xfc \x1c\xd8"
+ b"( /\xf2\xb7\xec\xc1\\\x8c\xf6\x95\xa1\x03J\x83vP8\xe1\xe3\xbb~\xc24kA"
+ b"\x98y\xa1\xf2P\xe9\x9d\xc9J\xf8N\x99\xb4\xceaO\xde\x16\x1e\xc2\x19\xa7"
+ b"\x03\xd2\xe0\x8f:\x15\xf3\x84\x9e\xee\xe6e\xb8\x02q\xc7AC\x1emw\xfd\t"
+ b"\x9a\x1eu\xc1\xa9\xcaCwUP\x00\xa5\xf78L4w!\x91L2 \x87\xd0\xf2\x06\x81j"
+ b"\x80;\x03V\x06\x87\x92\xcb\x90lv@E\x8d\x8d\xa5\xa6\xe7Z[\xdf\xd6E\x03`>"
+ b"\x8f\xde\xa1bZ\x84\xd0\xa9`\x05\x0e{\x80;\xe3\xbef\x8d\x1d\xebk1.\xe3"
+ b"\xe9N\x15\xf7\xd4(\xfa\xbb\x15\xbdu\xf7\x7f\x86\xae!\x03L\x1d\xb5\xc1"
+ b"\xb9\x11\xdb\xd0\x93\xe4\x02\xe1\xd2\xcbBjc_\xe8}d\xdb\xc3\xa0Y\xbe\xc9/"
+ b"\x95\x01\xa3,\xe6bl@\x01\xdbp\xc2\xce\x14\x168\xc2q\xe3uH\x89X\xa4\xa9"
+ b"\x19\x1d\xc1}\x7fOX\x19\x9f\xdd\xbe\x85\x83\xff\x96\x1ee\x82O`CF=K\xeb$I"
+ b"\x17_\xefX\x8bJ'v\xde\x1f+\xd9.v\xf8Tv\x17\xf2\x9f5\x19\xe1\xb9\x91\xa8S"
+ b"\x86\xbd\x1a\"(\xa5x\x8dC\x03X\x81\x91\xa8\x11\xc4pS\x13\xbc\xf2'J\xae!"
+ b"\xef\xef\x84G\t\x8d\xc4\x10\x132\x00oS\x9e\xe0\xe4d\x8f\xb8y\xac\xa6\x9f"
+ b",\xb8f\x87\r\xdf\x9eE\x0f\xe1\xd0\\L\x00\xb2\xe1h\x84\xef}\x98\xa8\x11"
+ b"\xccW#\\\x83\x7fo\xbbz\x8f\x00"
+)
+
+FILTERS_RAW_3 = [{"id": lzma.FILTER_IA64, "start_offset": 0x100},
+ {"id": lzma.FILTER_LZMA2}]
+COMPRESSED_RAW_3 = (
+ b"\xe0\x07\x80\x03\xdf]\x00\x05\x14\x07bX\x19\xcd\xddn\x98\x15\xe4\xb4\x9d"
+ b"o\x1d\xc4\xe5\n\x03\xcc2h\xc7\\\x86\xff\xf8\xe2\xfc\xe7\xd9\xfe6\xb8("
+ b"\xa8wd\xc2\"u.n\x1e\xc3\xf2\x8e\x8d\x8f\x02\x17/\xa6=\xf0\xa2\xdf/M\x89"
+ b"\xbe\xde\xa7\x1cz\x18-]\xd5\xef\x13\x8frZ\x15\x80\x8c\xf8\x8do\xfa\x12"
+ b"\x9b#z/\xef\xf0\xfaF\x01\x82\xa3M\x8e\xa1t\xca6 BF$\xe5Q\xa4\x98\xee\xde"
+ b"l\xe8\x7f\xf0\x9d,bn\x0b\x13\xd4\xa8\x81\xe4N\xc8\x86\x153\xf5x2\xa2O"
+ b"\x13@Q\xa1\x00/\xa5\xd0O\x97\xdco\xae\xf7z\xc4\xcdS\xb6t<\x16\xf2\x9cI#"
+ b"\x89ud\xc66Y\xd9\xee\xe6\xce\x12]\xe5\xf0\xaa\x96-Pe\xade:\x04\t\x1b\xf7"
+ b"\xdb7\n\x86\x1fp\xc8J\xba\xf4\xf0V\xa9\xdc\xf0\x02%G\xf9\xdf=?\x15\x1b"
+ b"\xe1(\xce\x82=\xd6I\xac3\x12\x0cR\xb7\xae\r\xb1i\x03\x95\x01\xbd\xbe\xfa"
+ b"\x02s\x01P\x9d\x96X\xb12j\xc8L\xa8\x84b\xf6\xc3\xd4c-H\x93oJl\xd0iQ\xe4k"
+ b"\x84\x0b\xc1\xb7\xbc\xb1\x17\x88\xb1\xca?@\xf6\x07\xea\xe6x\xf1H12P\x0f"
+ b"\x8a\xc9\xeauw\xe3\xbe\xaai\xa9W\xd0\x80\xcd#cb5\x99\xd8]\xa9d\x0c\xbd"
+ b"\xa2\xdcWl\xedUG\xbf\x89yF\xf77\x81v\xbd5\x98\xbeh8\x18W\x08\xf0\x1b\x99"
+ b"5:\x1a?rD\x96\xa1\x04\x0f\xae\xba\x85\xeb\x9d5@\xf5\x83\xd37\x83\x8ac"
+ b"\x06\xd4\x97i\xcdt\x16S\x82k\xf6K\x01vy\x88\x91\x9b6T\xdae\r\xfd]:k\xbal"
+ b"\xa9\xbba\xc34\xf9r\xeb}r\xdb\xc7\xdb*\x8f\x03z\xdc8h\xcc\xc9\xd3\xbcl"
+ b"\xa5-\xcb\xeaK\xa2\xc5\x15\xc0\xe3\xc1\x86Z\xfb\xebL\xe13\xcf\x9c\xe3"
+ b"\x1d\xc9\xed\xc2\x06\xcc\xce!\x92\xe5\xfe\x9c^\xa59w \x9bP\xa3PK\x08d"
+ b"\xf9\xe2Z}\xa7\xbf\xed\xeb%$\x0c\x82\xb8/\xb0\x01\xa9&,\xf7qh{Q\x96)\xf2"
+ b"q\x96\xc3\x80\xb4\x12\xb0\xba\xe6o\xf4!\xb4[\xd4\x8aw\x10\xf7t\x0c\xb3"
+ b"\xd9\xd5\xc3`^\x81\x11??\\\xa4\x99\x85R\xd4\x8e\x83\xc9\x1eX\xbfa\xf1"
+ b"\xac\xb0\xea\xea\xd7\xd0\xab\x18\xe2\xf2\xed\xe1\xb7\xc9\x18\xcbS\xe4>"
+ b"\xc9\x95H\xe8\xcb\t\r%\xeb\xc7$.o\xf1\xf3R\x17\x1db\xbb\xd8U\xa5^\xccS"
+ b"\x16\x01\x87\xf3/\x93\xd1\xf0v\xc0r\xd7\xcc\xa2Gkz\xca\x80\x0e\xfd\xd0"
+ b"\x8b\xbb\xd2Ix\xb3\x1ey\xca-0\xe3z^\xd6\xd6\x8f_\xf1\x9dP\x9fi\xa7\xd1"
+ b"\xe8\x90\x84\xdc\xbf\xcdky\x8e\xdc\x81\x7f\xa3\xb2+\xbf\x04\xef\xd8\\"
+ b"\xc4\xdf\xe1\xb0\x01\xe9\x93\xe3Y\xf1\x1dY\xe8h\x81\xcf\xf1w\xcc\xb4\xef"
+ b" \x8b|\x04\xea\x83ej\xbe\x1f\xd4z\x9c`\xd3\x1a\x92A\x06\xe5\x8f\xa9\x13"
+ b"\t\x9e=\xfa\x1c\xe5_\x9f%v\x1bo\x11ZO\xd8\xf4\t\xddM\x16-\x04\xfc\x18<\""
+ b"CM\xddg~b\xf6\xef\x8e\x0c\xd0\xde|\xa0'\x8a\x0c\xd6x\xae!J\xa6F\x88\x15u"
+ b"\x008\x17\xbc7y\xb3\xd8u\xac_\x85\x8d\xe7\xc1@\x9c\xecqc\xa3#\xad\xf1"
+ b"\x935\xb5)_\r\xec3]\x0fo]5\xd0my\x07\x9b\xee\x81\xb5\x0f\xcfK+\x00\xc0"
+ b"\xe4b\x10\xe4\x0c\x1a \x9b\xe0\x97t\xf6\xa1\x9e\x850\xba\x0c\x9a\x8d\xc8"
+ b"\x8f\x07\xd7\xae\xc8\xf9+i\xdc\xb9k\xb0>f\x19\xb8\r\xa8\xf8\x1f$\xa5{p"
+ b"\xc6\x880\xce\xdb\xcf\xca_\x86\xac\x88h6\x8bZ%'\xd0\n\xbf\x0f\x9c\"\xba"
+ b"\xe5\x86\x9f\x0f7X=mNX[\xcc\x19FU\xc9\x860\xbc\x90a+* \xae_$\x03\x1e\xd3"
+ b"\xcd_\xa0\x9c\xde\xaf46q\xa5\xc9\x92\xd7\xca\xe3`\x9d\x85}\xb4\xff\xb3"
+ b"\x83\xfb\xb6\xca\xae`\x0bw\x7f\xfc\xd8\xacVe\x19\xc8\x17\x0bZ\xad\x88"
+ b"\xeb#\x97\x03\x13\xb1d\x0f{\x0c\x04w\x07\r\x97\xbd\xd6\xc1\xc3B:\x95\x08"
+ b"^\x10V\xaeaH\x02\xd9\xe3\n\\\x01X\xf6\x9c\x8a\x06u#%\xbe*\xa1\x18v\x85"
+ b"\xec!\t4\x00\x00\x00"
+)
+
+FILTERS_RAW_4 = [{"id": lzma.FILTER_DELTA, "dist": 4},
+ {"id": lzma.FILTER_X86, "start_offset": 0x40},
+ {"id": lzma.FILTER_LZMA2, "preset": 4, "lc": 2}]
+COMPRESSED_RAW_4 = (
+ b"\xe0\x07\x80\x06\x0e\\\x00\x05\x14\x07bW\xaah\xdd\x10\xdc'\xd6\x90,\xc6v"
+ b"Jq \x14l\xb7\x83xB\x0b\x97f=&fx\xba\n>Tn\xbf\x8f\xfb\x1dF\xca\xc3v_\xca?"
+ b"\xfbV<\x92#\xd4w\xa6\x8a\xeb\xf6\x03\xc0\x01\x94\xd8\x9e\x13\x12\x98\xd1"
+ b"*\xfa]c\xe8\x1e~\xaf\xb5]Eg\xfb\x9e\x01\"8\xb2\x90\x06=~\xe9\x91W\xcd"
+ b"\xecD\x12\xc7\xfa\xe1\x91\x06\xc7\x99\xb9\xe3\x901\x87\x19u\x0f\x869\xff"
+ b"\xc1\xb0hw|\xb0\xdcl\xcck\xb16o7\x85\xee{Y_b\xbf\xbc$\xf3=\x8d\x8bw\xe5Z"
+ b"\x08@\xc4kmE\xad\xfb\xf6*\xd8\xad\xa1\xfb\xc5{\xdej,)\x1emB\x1f<\xaeca"
+ b"\x80(\xee\x07 \xdf\xe9\xf8\xeb\x0e-\x97\x86\x90c\xf9\xea'B\xf7`\xd7\xb0"
+ b"\x92\xbd\xa0\x82]\xbd\x0e\x0eB\x19\xdc\x96\xc6\x19\xd86D\xf0\xd5\x831"
+ b"\x03\xb7\x1c\xf7&5\x1a\x8f PZ&j\xf8\x98\x1bo\xcc\x86\x9bS\xd3\xa5\xcdu"
+ b"\xf9$\xcc\x97o\xe5V~\xfb\x97\xb5\x0b\x17\x9c\xfdxW\x10\xfep4\x80\xdaHDY"
+ b"\xfa)\xfet\xb5\"\xd4\xd3F\x81\xf4\x13\x1f\xec\xdf\xa5\x13\xfc\"\x91x\xb7"
+ b"\x99\xce\xc8\x92\n\xeb[\x10l*Y\xd8\xb1@\x06\xc8o\x8d7r\xebu\xfd5\x0e\x7f"
+ b"\xf1$U{\t}\x1fQ\xcfxN\x9d\x9fXX\xe9`\x83\xc1\x06\xf4\x87v-f\x11\xdb/\\"
+ b"\x06\xff\xd7)B\xf3g\x06\x88#2\x1eB244\x7f4q\t\xc893?mPX\x95\xa6a\xfb)d"
+ b"\x9b\xfc\x98\x9aj\x04\xae\x9b\x9d\x19w\xba\xf92\xfaA\x11\\\x17\x97C3\xa4"
+ b"\xbc!\x88\xcdo[\xec:\x030\x91.\x85\xe0@\\4\x16\x12\x9d\xcaJv\x97\xb04"
+ b"\xack\xcbkf\xa3ss\xfc\x16^\x8ce\x85a\xa5=&\xecr\xb3p\xd1E\xd5\x80y\xc7"
+ b"\xda\xf6\xfek\xbcT\xbfH\xee\x15o\xc5\x8c\x830\xec\x1d\x01\xae\x0c-e\\"
+ b"\x91\x90\x94\xb2\xf8\x88\x91\xe8\x0b\xae\xa7>\x98\xf6\x9ck\xd2\xc6\x08"
+ b"\xe6\xab\t\x98\xf2!\xa0\x8c^\xacqA\x99<\x1cEG\x97\xc8\xf1\xb6\xb9\x82"
+ b"\x8d\xf7\x08s\x98a\xff\xe3\xcc\x92\x0e\xd2\xb6U\xd7\xd9\x86\x7fa\xe5\x1c"
+ b"\x8dTG@\t\x1e\x0e7*\xfc\xde\xbc]6N\xf7\xf1\x84\x9e\x9f\xcf\xe9\x1e\xb5'"
+ b"\xf4<\xdf\x99sq\xd0\x9d\xbd\x99\x0b\xb4%p4\xbf{\xbb\x8a\xd2\x0b\xbc=M"
+ b"\x94H:\xf5\xa8\xd6\xa4\xc90\xc2D\xb9\xd3\xa8\xb0S\x87 `\xa2\xeb\xf3W\xce"
+ b" 7\xf9N#\r\xe6\xbe\t\x9d\xe7\x811\xf9\x10\xc1\xc2\x14\xf6\xfc\xcba\xb7"
+ b"\xb1\x7f\x95l\xe4\tjA\xec:\x10\xe5\xfe\xc2\\=D\xe2\x0c\x0b3]\xf7\xc1\xf7"
+ b"\xbceZ\xb1A\xea\x16\xe5\xfddgFQ\xed\xaf\x04\xa3\xd3\xf8\xa2q\x19B\xd4r"
+ b"\xc5\x0c\x9a\x14\x94\xea\x91\xc4o\xe4\xbb\xb4\x99\xf4@\xd1\xe6\x0c\xe3"
+ b"\xc6d\xa0Q\n\xf2/\xd8\xb8S5\x8a\x18:\xb5g\xac\x95D\xce\x17\x07\xd4z\xda"
+ b"\x90\xe65\x07\x19H!\t\xfdu\x16\x8e\x0eR\x19\xf4\x8cl\x0c\xf9Q\xf1\x80"
+ b"\xe3\xbf\xd7O\xf8\x8c\x18\x0b\x9c\xf1\x1fb\xe1\tR\xb2\xf1\xe1A\xea \xcf-"
+ b"IGE\xf1\x14\x98$\x83\x15\xc9\xd8j\xbf\x19\x0f\xd5\xd1\xaa\xb3\xf3\xa5I2s"
+ b"\x8d\x145\xca\xd5\xd93\x9c\xb8D0\xe6\xaa%\xd0\xc0P}JO^h\x8e\x08\xadlV."
+ b"\x18\x88\x13\x05o\xb0\x07\xeaw\xe0\xb6\xa4\xd5*\xe4r\xef\x07G+\xc1\xbei["
+ b"w\xe8\xab@_\xef\x15y\xe5\x12\xc9W\x1b.\xad\x85-\xc2\xf7\xe3mU6g\x8eSA"
+ b"\x01(\xd3\xdb\x16\x13=\xde\x92\xf9,D\xb8\x8a\xb2\xb4\xc9\xc3\xefnE\xe8\\"
+ b"\xa6\xe2Y\xd2\xcf\xcb\x8c\xb6\xd5\xe9\x1d\x1e\x9a\x8b~\xe2\xa6\rE\x84uV"
+ b"\xed\xc6\x99\xddm<\x10[\x0fu\x1f\xc1\x1d1\n\xcfw\xb2%!\xf0[\xce\x87\x83B"
+ b"\x08\xaa,\x08%d\xcef\x94\"\xd9g.\xc83\xcbXY+4\xec\x85qA\n\x1d=9\xf0*\xb1"
+ b"\x1f/\xf3s\xd61b\x7f@\xfb\x9d\xe3FQ\\\xbd\x82\x1e\x00\xf3\xce\xd3\xe1"
+ b"\xca,E\xfd7[\xab\xb6\xb7\xac!mA}\xbd\x9d3R5\x9cF\xabH\xeb\x92)cc\x13\xd0"
+ b"\xbd\xee\xe9n{\x1dIJB\xa5\xeb\x11\xe8`w&`\x8b}@Oxe\t\x8a\x07\x02\x95\xf2"
+ b"\xed\xda|\xb1e\xbe\xaa\xbbg\x19@\xe1Y\x878\x84\x0f\x8c\xe3\xc98\xf2\x9e"
+ b"\xd5N\xb5J\xef\xab!\xe2\x8dq\xe1\xe5q\xc5\xee\x11W\xb7\xe4k*\x027\xa0"
+ b"\xa3J\xf4\xd8m\xd0q\x94\xcb\x07\n:\xb6`.\xe4\x9c\x15+\xc0)\xde\x80X\xd4"
+ b"\xcfQm\x01\xc2cP\x1cA\x85'\xc9\xac\x8b\xe6\xb2)\xe6\x84t\x1c\x92\xe4Z"
+ b"\x1cR\xb0\x9e\x96\xd1\xfb\x1c\xa6\x8b\xcb`\x10\x12]\xf2gR\x9bFT\xe0\xc8H"
+ b"S\xfb\xac<\x04\xc7\xc1\xe8\xedP\xf4\x16\xdb\xc0\xd7e\xc2\x17J^\x1f\xab"
+ b"\xff[\x08\x19\xb4\xf5\xfb\x19\xb4\x04\xe5c~']\xcb\xc2A\xec\x90\xd0\xed"
+ b"\x06,\xc5K{\x86\x03\xb1\xcdMx\xdeQ\x8c3\xf9\x8a\xea=\x89\xaba\xd2\xc89a"
+ b"\xd72\xf0\xc3\x19\x8a\xdfs\xd4\xfd\xbb\x81b\xeaE\"\xd8\xf4d\x0cD\xf7IJ!"
+ b"\xe5d\xbbG\xe9\xcam\xaa\x0f_r\x95\x91NBq\xcaP\xce\xa7\xa9\xb5\x10\x94eP!"
+ b"|\x856\xcd\xbfIir\xb8e\x9bjP\x97q\xabwS7\x1a\x0ehM\xe7\xca\x86?\xdeP}y~"
+ b"\x0f\x95I\xfc\x13\xe1<Q\x1b\x868\x1d\x11\xdf\x94\xf4\x82>r\xa9k\x88\xcb"
+ b"\xfd\xc3v\xe2\xb9\x8a\x02\x8eq\x92I\xf8\xf6\xf1\x03s\x9b\xb8\xe3\"\xe3"
+ b"\xa9\xa5>D\xb8\x96;\xe7\x92\xd133\xe8\xdd'e\xc9.\xdc;\x17\x1f\xf5H\x13q"
+ b"\xa4W\x0c\xdb~\x98\x01\xeb\xdf\xe32\x13\x0f\xddx\n6\xa0\t\x10\xb6\xbb"
+ b"\xb0\xc3\x18\xb6;\x9fj[\xd9\xd5\xc9\x06\x8a\x87\xcd\xe5\xee\xfc\x9c-%@"
+ b"\xee\xe0\xeb\xd2\xe3\xe8\xfb\xc0\x122\\\xc7\xaf\xc2\xa1Oth\xb3\x8f\x82"
+ b"\xb3\x18\xa8\x07\xd5\xee_\xbe\xe0\x1cA\x1e_\r\x9a\xb0\x17W&\xa2D\x91\x94"
+ b"\x1a\xb2\xef\xf2\xdc\x85;X\xb0,\xeb>-7S\xe5\xca\x07)\x1fp\x7f\xcaQBL\xca"
+ b"\xf3\xb9d\xfc\xb5su\xb0\xc8\x95\x90\xeb*)\xa0v\xe4\x9a{FW\xf4l\xde\xcdj"
+ b"\x00"
+)
+
+
+def test_main():
+ run_unittest(
+ CompressorDecompressorTestCase,
+ CompressDecompressFunctionTestCase,
+ FileTestCase,
+ MiscellaneousTestCase,
+ )
+
+if __name__ == "__main__":
+ test_main()
diff --git a/Misc/NEWS b/Misc/NEWS
index 5132f8c..f01b7e2 100644
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -399,6 +399,8 @@
Library
-------
+- Issue #6715: Add a module 'lzma' for compression using the LZMA algorithm.
+
- Issue #13487: Make inspect.getmodule robust against changes done to
sys.modules while it is iterating over it.
diff --git a/Modules/_lzmamodule.c b/Modules/_lzmamodule.c
new file mode 100644
index 0000000..7371517
--- /dev/null
+++ b/Modules/_lzmamodule.c
@@ -0,0 +1,1106 @@
+/* _lzma - Low-level Python interface to liblzma. */
+
+#define PY_SSIZE_T_CLEAN
+
+#include "Python.h"
+#include "structmember.h"
+#ifdef WITH_THREAD
+#include "pythread.h"
+#endif
+
+#include <stdarg.h>
+#include <string.h>
+
+#include <lzma.h>
+
+
+#ifndef PY_LONG_LONG
+#error "This module requires PY_LONG_LONG to be defined"
+#endif
+
+
+#ifdef WITH_THREAD
+#define ACQUIRE_LOCK(obj) do { \
+ if (!PyThread_acquire_lock((obj)->lock, 0)) { \
+ Py_BEGIN_ALLOW_THREADS \
+ PyThread_acquire_lock((obj)->lock, 1); \
+ Py_END_ALLOW_THREADS \
+ } } while (0)
+#define RELEASE_LOCK(obj) PyThread_release_lock((obj)->lock)
+#else
+#define ACQUIRE_LOCK(obj)
+#define RELEASE_LOCK(obj)
+#endif
+
+
+/* Container formats: */
+enum {
+ FORMAT_AUTO,
+ FORMAT_XZ,
+ FORMAT_ALONE,
+ FORMAT_RAW,
+};
+
+#define LZMA_CHECK_UNKNOWN (LZMA_CHECK_ID_MAX + 1)
+
+
+typedef struct {
+ PyObject_HEAD
+ lzma_stream lzs;
+ int flushed;
+#ifdef WITH_THREAD
+ PyThread_type_lock lock;
+#endif
+} Compressor;
+
+typedef struct {
+ PyObject_HEAD
+ lzma_stream lzs;
+ int check;
+ char eof;
+ PyObject *unused_data;
+#ifdef WITH_THREAD
+ PyThread_type_lock lock;
+#endif
+} Decompressor;
+
+/* LZMAError class object. */
+static PyObject *Error;
+
+/* An empty tuple, used by the filter specifier parsing code. */
+static PyObject *empty_tuple;
+
+
+/* Helper functions. */
+
+static int
+catch_lzma_error(lzma_ret lzret)
+{
+ switch (lzret) {
+ case LZMA_OK:
+ case LZMA_GET_CHECK:
+ case LZMA_NO_CHECK:
+ case LZMA_STREAM_END:
+ return 0;
+ case LZMA_UNSUPPORTED_CHECK:
+ PyErr_SetString(Error, "Unsupported integrity check");
+ return 1;
+ case LZMA_MEM_ERROR:
+ PyErr_NoMemory();
+ return 1;
+ case LZMA_MEMLIMIT_ERROR:
+ PyErr_SetString(Error, "Memory usage limit exceeded");
+ return 1;
+ case LZMA_FORMAT_ERROR:
+ PyErr_SetString(Error, "Input format not supported by decoder");
+ return 1;
+ case LZMA_OPTIONS_ERROR:
+ PyErr_SetString(Error, "Invalid or unsupported options");
+ return 1;
+ case LZMA_DATA_ERROR:
+ PyErr_SetString(Error, "Corrupt input data");
+ return 1;
+ case LZMA_BUF_ERROR:
+ PyErr_SetString(Error, "Insufficient buffer space");
+ return 1;
+ case LZMA_PROG_ERROR:
+ PyErr_SetString(Error, "Internal error");
+ return 1;
+ default:
+ PyErr_Format(Error, "Unrecognized error from liblzma: %d", lzret);
+ return 1;
+ }
+}
+
+#if BUFSIZ < 8192
+#define INITIAL_BUFFER_SIZE 8192
+#else
+#define INITIAL_BUFFER_SIZE BUFSIZ
+#endif
+
+static int
+grow_buffer(PyObject **buf)
+{
+ size_t size = PyBytes_GET_SIZE(*buf);
+ return _PyBytes_Resize(buf, size + (size >> 3) + 6);
+}
+
+
+/* Some custom type conversions for PyArg_ParseTupleAndKeywords(),
+ since the predefined conversion specifiers do not suit our needs:
+
+ uint32_t - the "I" (unsigned int) specifier is the right size, but
+ silently ignores overflows on conversion.
+
+ lzma_mode and lzma_match_finder - these are enumeration types, and
+ so the size of each is implementation-defined. Worse, different
+ enum types can be of different sizes within the same program, so
+ to be strictly correct, we need to define two separate converters.
+ */
+
+#define INT_TYPE_CONVERTER_FUNC(TYPE, FUNCNAME) \
+ static int \
+ FUNCNAME(PyObject *obj, void *ptr) \
+ { \
+ unsigned long val; \
+ \
+ val = PyLong_AsUnsignedLong(obj); \
+ if (PyErr_Occurred()) \
+ return 0; \
+ if ((unsigned long)(TYPE)val != val) { \
+ PyErr_SetString(PyExc_OverflowError, \
+ "Value too large for " #TYPE " type"); \
+ return 0; \
+ } \
+ *(TYPE *)ptr = val; \
+ return 1; \
+ }
+
+INT_TYPE_CONVERTER_FUNC(uint32_t, uint32_converter)
+INT_TYPE_CONVERTER_FUNC(lzma_mode, lzma_mode_converter)
+INT_TYPE_CONVERTER_FUNC(lzma_match_finder, lzma_mf_converter)
+
+#undef INT_TYPE_CONVERTER_FUNC
+
+
+/* Filter specifier parsing functions. */
+
+static void *
+parse_filter_spec_lzma(PyObject *spec)
+{
+ static char *optnames[] = {"id", "preset", "dict_size", "lc", "lp",
+ "pb", "mode", "nice_len", "mf", "depth", NULL};
+ PyObject *id;
+ PyObject *preset_obj;
+ uint32_t preset = LZMA_PRESET_DEFAULT;
+ lzma_options_lzma *options;
+
+ /* First, fill in default values for all the options using a preset.
+ Then, override the defaults with any values given by the caller. */
+
+ preset_obj = PyMapping_GetItemString(spec, "preset");
+ if (preset_obj == NULL) {
+ if (PyErr_ExceptionMatches(PyExc_KeyError))
+ PyErr_Clear();
+ else
+ return NULL;
+ } else {
+ int ok = uint32_converter(preset_obj, &preset);
+ Py_DECREF(preset_obj);
+ if (!ok)
+ return NULL;
+ }
+
+ options = (lzma_options_lzma *)PyMem_Malloc(sizeof *options);
+ if (options == NULL)
+ return PyErr_NoMemory();
+ memset(options, 0, sizeof *options);
+
+ if (lzma_lzma_preset(options, preset)) {
+ PyMem_Free(options);
+ PyErr_Format(Error, "lzma_lzma_preset() failed for preset %#x", preset);
+ return NULL;
+ }
+
+ if (!PyArg_ParseTupleAndKeywords(empty_tuple, spec,
+ "|OOO&O&O&O&O&O&O&O&", optnames,
+ &id, &preset_obj,
+ uint32_converter, &options->dict_size,
+ uint32_converter, &options->lc,
+ uint32_converter, &options->lp,
+ uint32_converter, &options->pb,
+ lzma_mode_converter, &options->mode,
+ uint32_converter, &options->nice_len,
+ lzma_mf_converter, &options->mf,
+ uint32_converter, &options->depth)) {
+ PyErr_SetString(PyExc_ValueError,
+ "Invalid filter specifier for LZMA filter");
+ PyMem_Free(options);
+ options = NULL;
+ }
+ return options;
+}
+
+static void *
+parse_filter_spec_delta(PyObject *spec)
+{
+ static char *optnames[] = {"id", "dist", NULL};
+ PyObject *id;
+ uint32_t dist = 1;
+ lzma_options_delta *options;
+
+ if (!PyArg_ParseTupleAndKeywords(empty_tuple, spec, "|OO&", optnames,
+ &id, uint32_converter, &dist)) {
+ PyErr_SetString(PyExc_ValueError,
+ "Invalid filter specifier for delta filter");
+ return NULL;
+ }
+
+ options = (lzma_options_delta *)PyMem_Malloc(sizeof *options);
+ if (options == NULL)
+ return PyErr_NoMemory();
+ memset(options, 0, sizeof *options);
+ options->type = LZMA_DELTA_TYPE_BYTE;
+ options->dist = dist;
+ return options;
+}
+
+static void *
+parse_filter_spec_bcj(PyObject *spec)
+{
+ static char *optnames[] = {"id", "start_offset", NULL};
+ PyObject *id;
+ uint32_t start_offset = 0;
+ lzma_options_bcj *options;
+
+ if (!PyArg_ParseTupleAndKeywords(empty_tuple, spec, "|OO&", optnames,
+ &id, uint32_converter, &start_offset)) {
+ PyErr_SetString(PyExc_ValueError,
+ "Invalid filter specifier for BCJ filter");
+ return NULL;
+ }
+
+ options = (lzma_options_bcj *)PyMem_Malloc(sizeof *options);
+ if (options == NULL)
+ return PyErr_NoMemory();
+ memset(options, 0, sizeof *options);
+ options->start_offset = start_offset;
+ return options;
+}
+
+static void *
+parse_filter_spec(lzma_filter *f, PyObject *spec)
+{
+ PyObject *id_obj;
+
+ if (!PyMapping_Check(spec)) {
+ PyErr_SetString(PyExc_TypeError,
+ "Filter specifier must be a dict or dict-like object");
+ return NULL;
+ }
+ id_obj = PyMapping_GetItemString(spec, "id");
+ if (id_obj == NULL) {
+ if (PyErr_ExceptionMatches(PyExc_KeyError))
+ PyErr_SetString(PyExc_ValueError,
+ "Filter specifier must have an \"id\" entry");
+ return NULL;
+ }
+ f->id = PyLong_AsUnsignedLongLong(id_obj);
+ Py_DECREF(id_obj);
+ if (PyErr_Occurred())
+ return NULL;
+
+ switch (f->id) {
+ case LZMA_FILTER_LZMA1:
+ case LZMA_FILTER_LZMA2:
+ f->options = parse_filter_spec_lzma(spec);
+ return f->options;
+ case LZMA_FILTER_DELTA:
+ f->options = parse_filter_spec_delta(spec);
+ return f->options;
+ case LZMA_FILTER_X86:
+ case LZMA_FILTER_POWERPC:
+ case LZMA_FILTER_IA64:
+ case LZMA_FILTER_ARM:
+ case LZMA_FILTER_ARMTHUMB:
+ case LZMA_FILTER_SPARC:
+ f->options = parse_filter_spec_bcj(spec);
+ return f->options;
+ default:
+ PyErr_Format(PyExc_ValueError, "Invalid filter ID: %llu", f->id);
+ return NULL;
+ }
+}
+
+static void
+free_filter_chain(lzma_filter filters[])
+{
+ int i;
+
+ for (i = 0; filters[i].id != LZMA_VLI_UNKNOWN; i++)
+ PyMem_Free(filters[i].options);
+}
+
+static int
+parse_filter_chain_spec(lzma_filter filters[], PyObject *filterspecs)
+{
+ Py_ssize_t i, num_filters;
+
+ num_filters = PySequence_Length(filterspecs);
+ if (num_filters == -1)
+ return -1;
+ if (num_filters > LZMA_FILTERS_MAX) {
+ PyErr_Format(PyExc_ValueError,
+ "Too many filters - liblzma supports a maximum of %d",
+ LZMA_FILTERS_MAX);
+ return -1;
+ }
+
+ for (i = 0; i < num_filters; i++) {
+ int ok = 1;
+ PyObject *spec = PySequence_GetItem(filterspecs, i);
+ if (spec == NULL || parse_filter_spec(&filters[i], spec) == NULL)
+ ok = 0;
+ Py_XDECREF(spec);
+ if (!ok) {
+ filters[i].id = LZMA_VLI_UNKNOWN;
+ free_filter_chain(filters);
+ return -1;
+ }
+ }
+ filters[num_filters].id = LZMA_VLI_UNKNOWN;
+ return 0;
+}
+
+
+/* LZMACompressor class. */
+
+static PyObject *
+compress(Compressor *c, uint8_t *data, size_t len, lzma_action action)
+{
+ size_t data_size = 0;
+ PyObject *result;
+
+ result = PyBytes_FromStringAndSize(NULL, INITIAL_BUFFER_SIZE);
+ if (result == NULL)
+ return NULL;
+ c->lzs.next_in = data;
+ c->lzs.avail_in = len;
+ c->lzs.next_out = (uint8_t *)PyBytes_AS_STRING(result);
+ c->lzs.avail_out = PyBytes_GET_SIZE(result);
+ for (;;) {
+ lzma_ret lzret;
+
+ Py_BEGIN_ALLOW_THREADS
+ lzret = lzma_code(&c->lzs, action);
+ data_size = (char *)c->lzs.next_out - PyBytes_AS_STRING(result);
+ Py_END_ALLOW_THREADS
+ if (catch_lzma_error(lzret))
+ goto error;
+ if ((action == LZMA_RUN && c->lzs.avail_in == 0) ||
+ (action == LZMA_FINISH && lzret == LZMA_STREAM_END)) {
+ break;
+ } else if (c->lzs.avail_out == 0) {
+ if (grow_buffer(&result) == -1)
+ goto error;
+ c->lzs.next_out = (uint8_t *)PyBytes_AS_STRING(result) + data_size;
+ c->lzs.avail_out = PyBytes_GET_SIZE(result) - data_size;
+ }
+ }
+ if (data_size != PyBytes_GET_SIZE(result))
+ if (_PyBytes_Resize(&result, data_size) == -1)
+ goto error;
+ return result;
+
+error:
+ Py_XDECREF(result);
+ return NULL;
+}
+
+PyDoc_STRVAR(Compressor_compress_doc,
+"compress(data) -> bytes\n"
+"\n"
+"Provide data to the compressor object. Returns a chunk of\n"
+"compressed data if possible, or b\"\" otherwise.\n"
+"\n"
+"When you have finished providing data to the compressor, call the\n"
+"flush() method to finish the conversion process.\n");
+
+static PyObject *
+Compressor_compress(Compressor *self, PyObject *args)
+{
+ Py_buffer buffer;
+ PyObject *result = NULL;
+
+ if (!PyArg_ParseTuple(args, "y*:compress", &buffer))
+ return NULL;
+
+ ACQUIRE_LOCK(self);
+ if (self->flushed)
+ PyErr_SetString(PyExc_ValueError, "Compressor has been flushed");
+ else
+ result = compress(self, buffer.buf, buffer.len, LZMA_RUN);
+ RELEASE_LOCK(self);
+ PyBuffer_Release(&buffer);
+ return result;
+}
+
+PyDoc_STRVAR(Compressor_flush_doc,
+"flush() -> bytes\n"
+"\n"
+"Finish the compression process. Returns the compressed data left\n"
+"in internal buffers.\n"
+"\n"
+"The compressor object cannot be used after this method is called.\n");
+
+static PyObject *
+Compressor_flush(Compressor *self, PyObject *noargs)
+{
+ PyObject *result = NULL;
+
+ ACQUIRE_LOCK(self);
+ if (self->flushed) {
+ PyErr_SetString(PyExc_ValueError, "Repeated call to flush()");
+ } else {
+ self->flushed = 1;
+ result = compress(self, NULL, 0, LZMA_FINISH);
+ }
+ RELEASE_LOCK(self);
+ return result;
+}
+
+static int
+Compressor_init_xz(lzma_stream *lzs, int check, uint32_t preset,
+ PyObject *filterspecs)
+{
+ lzma_ret lzret;
+
+ if (filterspecs == Py_None) {
+ lzret = lzma_easy_encoder(lzs, preset, check);
+ } else {
+ lzma_filter filters[LZMA_FILTERS_MAX + 1];
+
+ if (parse_filter_chain_spec(filters, filterspecs) == -1)
+ return -1;
+ lzret = lzma_stream_encoder(lzs, filters, check);
+ free_filter_chain(filters);
+ }
+ if (catch_lzma_error(lzret))
+ return -1;
+ else
+ return 0;
+}
+
+static int
+Compressor_init_alone(lzma_stream *lzs, uint32_t preset, PyObject *filterspecs)
+{
+ lzma_ret lzret;
+
+ if (filterspecs == Py_None) {
+ lzma_options_lzma options;
+
+ if (lzma_lzma_preset(&options, preset)) {
+ PyErr_Format(Error, "Invalid compression preset: %#x", preset);
+ return -1;
+ }
+ lzret = lzma_alone_encoder(lzs, &options);
+ } else {
+ lzma_filter filters[LZMA_FILTERS_MAX + 1];
+
+ if (parse_filter_chain_spec(filters, filterspecs) == -1)
+ return -1;
+ if (filters[0].id == LZMA_FILTER_LZMA1 &&
+ filters[1].id == LZMA_VLI_UNKNOWN) {
+ lzret = lzma_alone_encoder(lzs, filters[0].options);
+ } else {
+ PyErr_SetString(PyExc_ValueError,
+ "Invalid filter chain for FORMAT_ALONE - "
+ "must be a single LZMA1 filter");
+ lzret = LZMA_PROG_ERROR;
+ }
+ free_filter_chain(filters);
+ }
+ if (PyErr_Occurred() || catch_lzma_error(lzret))
+ return -1;
+ else
+ return 0;
+}
+
+static int
+Compressor_init_raw(lzma_stream *lzs, PyObject *filterspecs)
+{
+ lzma_filter filters[LZMA_FILTERS_MAX + 1];
+ lzma_ret lzret;
+
+ if (filterspecs == Py_None) {
+ PyErr_SetString(PyExc_ValueError,
+ "Must specify filters for FORMAT_RAW");
+ return -1;
+ }
+ if (parse_filter_chain_spec(filters, filterspecs) == -1)
+ return -1;
+ lzret = lzma_raw_encoder(lzs, filters);
+ free_filter_chain(filters);
+ if (catch_lzma_error(lzret))
+ return -1;
+ else
+ return 0;
+}
+
+static int
+Compressor_init(Compressor *self, PyObject *args, PyObject *kwargs)
+{
+ static char *arg_names[] = {"format", "check", "preset", "filters", NULL};
+ int format = FORMAT_XZ;
+ int check = -1;
+ uint32_t preset = LZMA_PRESET_DEFAULT;
+ PyObject *preset_obj = Py_None;
+ PyObject *filterspecs = Py_None;
+
+ if (!PyArg_ParseTupleAndKeywords(args, kwargs,
+ "|iiOO:LZMACompressor", arg_names,
+ &format, &check, &preset_obj,
+ &filterspecs))
+ return -1;
+
+ if (format != FORMAT_XZ && check != -1 && check != LZMA_CHECK_NONE) {
+ PyErr_SetString(PyExc_ValueError,
+ "Integrity checks are only supported by FORMAT_XZ");
+ return -1;
+ }
+
+ if (preset_obj != Py_None && filterspecs != Py_None) {
+ PyErr_SetString(PyExc_ValueError,
+ "Cannot specify both preset and filter chain");
+ return -1;
+ }
+
+ if (preset_obj != Py_None)
+ if (!uint32_converter(preset_obj, &preset))
+ return -1;
+
+#ifdef WITH_THREAD
+ self->lock = PyThread_allocate_lock();
+ if (self->lock == NULL) {
+ PyErr_SetString(PyExc_MemoryError, "Unable to allocate lock");
+ return -1;
+ }
+#endif
+
+ self->flushed = 0;
+ switch (format) {
+ case FORMAT_XZ:
+ if (check == -1)
+ check = LZMA_CHECK_CRC64;
+ if (Compressor_init_xz(&self->lzs, check, preset, filterspecs) != 0)
+ break;
+ return 0;
+
+ case FORMAT_ALONE:
+ if (Compressor_init_alone(&self->lzs, preset, filterspecs) != 0)
+ break;
+ return 0;
+
+ case FORMAT_RAW:
+ if (Compressor_init_raw(&self->lzs, filterspecs) != 0)
+ break;
+ return 0;
+
+ default:
+ PyErr_Format(PyExc_ValueError,
+ "Invalid container format: %d", format);
+ break;
+ }
+
+#ifdef WITH_THREAD
+ PyThread_free_lock(self->lock);
+ self->lock = NULL;
+#endif
+ return -1;
+}
+
+static void
+Compressor_dealloc(Compressor *self)
+{
+ lzma_end(&self->lzs);
+#ifdef WITH_THREAD
+ if (self->lock != NULL)
+ PyThread_free_lock(self->lock);
+#endif
+ Py_TYPE(self)->tp_free((PyObject *)self);
+}
+
+static PyMethodDef Compressor_methods[] = {
+ {"compress", (PyCFunction)Compressor_compress, METH_VARARGS,
+ Compressor_compress_doc},
+ {"flush", (PyCFunction)Compressor_flush, METH_NOARGS,
+ Compressor_flush_doc},
+ {NULL}
+};
+
+PyDoc_STRVAR(Compressor_doc,
+"LZMACompressor(format=FORMAT_XZ, check=-1, preset=None, filters=None)\n"
+"\n"
+"Create a compressor object for compressing data incrementally.\n"
+"\n"
+"format specifies the container format to use for the output. This can\n"
+"be FORMAT_XZ (default), FORMAT_ALONE, or FORMAT_RAW.\n"
+"\n"
+"check specifies the integrity check to use. For FORMAT_XZ, the default\n"
+"is CHECK_CRC64. FORMAT_ALONE and FORMAT_RAW do not suport integrity\n"
+"checks; for these formats, check must be omitted, or be CHECK_NONE.\n"
+"\n"
+"The settings used by the compressor can be specified either as a\n"
+"preset compression level (with the 'preset' argument), or in detail\n"
+"as a custom filter chain (with the 'filters' argument). For FORMAT_XZ\n"
+"and FORMAT_ALONE, the default is to use the PRESET_DEFAULT preset\n"
+"level. For FORMAT_RAW, the caller must always specify a filter chain;\n"
+"the raw compressor does not support preset compression levels.\n"
+"\n"
+"preset (if provided) should be an integer in the range 0-9, optionally\n"
+"OR-ed with the constant PRESET_EXTREME.\n"
+"\n"
+"filters (if provided) should be a sequence of dicts. Each dict should\n"
+"have an entry for \"id\" indicating the ID of the filter, plus\n"
+"additional entries for options to the filter.\n"
+"\n"
+"For one-shot compression, use the compress() function instead.\n");
+
+static PyTypeObject Compressor_type = {
+ PyVarObject_HEAD_INIT(NULL, 0)
+ "_lzma.LZMACompressor", /* tp_name */
+ sizeof(Compressor), /* tp_basicsize */
+ 0, /* tp_itemsize */
+ (destructor)Compressor_dealloc, /* tp_dealloc */
+ 0, /* tp_print */
+ 0, /* tp_getattr */
+ 0, /* tp_setattr */
+ 0, /* tp_reserved */
+ 0, /* tp_repr */
+ 0, /* tp_as_number */
+ 0, /* tp_as_sequence */
+ 0, /* tp_as_mapping */
+ 0, /* tp_hash */
+ 0, /* tp_call */
+ 0, /* tp_str */
+ 0, /* tp_getattro */
+ 0, /* tp_setattro */
+ 0, /* tp_as_buffer */
+ Py_TPFLAGS_DEFAULT, /* tp_flags */
+ Compressor_doc, /* tp_doc */
+ 0, /* tp_traverse */
+ 0, /* tp_clear */
+ 0, /* tp_richcompare */
+ 0, /* tp_weaklistoffset */
+ 0, /* tp_iter */
+ 0, /* tp_iternext */
+ Compressor_methods, /* tp_methods */
+ 0, /* tp_members */
+ 0, /* tp_getset */
+ 0, /* tp_base */
+ 0, /* tp_dict */
+ 0, /* tp_descr_get */
+ 0, /* tp_descr_set */
+ 0, /* tp_dictoffset */
+ (initproc)Compressor_init, /* tp_init */
+ 0, /* tp_alloc */
+ PyType_GenericNew, /* tp_new */
+};
+
+
+/* LZMADecompressor class. */
+
+static PyObject *
+decompress(Decompressor *d, uint8_t *data, size_t len)
+{
+ size_t data_size = 0;
+ PyObject *result;
+
+ result = PyBytes_FromStringAndSize(NULL, INITIAL_BUFFER_SIZE);
+ if (result == NULL)
+ return NULL;
+ d->lzs.next_in = data;
+ d->lzs.avail_in = len;
+ d->lzs.next_out = (uint8_t *)PyBytes_AS_STRING(result);
+ d->lzs.avail_out = PyBytes_GET_SIZE(result);
+ for (;;) {
+ lzma_ret lzret;
+
+ Py_BEGIN_ALLOW_THREADS
+ lzret = lzma_code(&d->lzs, LZMA_RUN);
+ data_size = (char *)d->lzs.next_out - PyBytes_AS_STRING(result);
+ Py_END_ALLOW_THREADS
+ if (catch_lzma_error(lzret))
+ goto error;
+ if (lzret == LZMA_GET_CHECK || lzret == LZMA_NO_CHECK)
+ d->check = lzma_get_check(&d->lzs);
+ if (lzret == LZMA_STREAM_END) {
+ d->eof = 1;
+ if (d->lzs.avail_in > 0) {
+ Py_CLEAR(d->unused_data);
+ d->unused_data = PyBytes_FromStringAndSize(
+ (char *)d->lzs.next_in, d->lzs.avail_in);
+ if (d->unused_data == NULL)
+ goto error;
+ }
+ break;
+ } else if (d->lzs.avail_in == 0) {
+ break;
+ } else if (d->lzs.avail_out == 0) {
+ if (grow_buffer(&result) == -1)
+ goto error;
+ d->lzs.next_out = (uint8_t *)PyBytes_AS_STRING(result) + data_size;
+ d->lzs.avail_out = PyBytes_GET_SIZE(result) - data_size;
+ }
+ }
+ if (data_size != PyBytes_GET_SIZE(result))
+ if (_PyBytes_Resize(&result, data_size) == -1)
+ goto error;
+ return result;
+
+error:
+ Py_XDECREF(result);
+ return NULL;
+}
+
+PyDoc_STRVAR(Decompressor_decompress_doc,
+"decompress(data) -> bytes\n"
+"\n"
+"Provide data to the decompressor object. Returns a chunk of\n"
+"decompressed data if possible, or b\"\" otherwise.\n"
+"\n"
+"Attempting to decompress data after the end of the stream is\n"
+"reached raises an EOFError. Any data found after the end of the\n"
+"stream is ignored, and saved in the unused_data attribute.\n");
+
+static PyObject *
+Decompressor_decompress(Decompressor *self, PyObject *args)
+{
+ Py_buffer buffer;
+ PyObject *result = NULL;
+
+ if (!PyArg_ParseTuple(args, "y*:decompress", &buffer))
+ return NULL;
+
+ ACQUIRE_LOCK(self);
+ if (self->eof)
+ PyErr_SetString(PyExc_EOFError, "Already at end of stream");
+ else
+ result = decompress(self, buffer.buf, buffer.len);
+ RELEASE_LOCK(self);
+ PyBuffer_Release(&buffer);
+ return result;
+}
+
+static int
+Decompressor_init_raw(lzma_stream *lzs, PyObject *filterspecs)
+{
+ lzma_filter filters[LZMA_FILTERS_MAX + 1];
+ lzma_ret lzret;
+
+ if (parse_filter_chain_spec(filters, filterspecs) == -1)
+ return -1;
+ lzret = lzma_raw_decoder(lzs, filters);
+ free_filter_chain(filters);
+ if (catch_lzma_error(lzret))
+ return -1;
+ else
+ return 0;
+}
+
+static int
+Decompressor_init(Decompressor *self, PyObject *args, PyObject *kwargs)
+{
+ static char *arg_names[] = {"format", "memlimit", "filters", NULL};
+ const uint32_t decoder_flags = LZMA_TELL_ANY_CHECK | LZMA_TELL_NO_CHECK;
+ int format = FORMAT_AUTO;
+ uint64_t memlimit = UINT64_MAX;
+ PyObject *memlimit_obj = Py_None;
+ PyObject *filterspecs = Py_None;
+ lzma_ret lzret;
+
+ if (!PyArg_ParseTupleAndKeywords(args, kwargs,
+ "|iOO:LZMADecompressor", arg_names,
+ &format, &memlimit_obj, &filterspecs))
+ return -1;
+
+ if (memlimit_obj != Py_None) {
+ if (format == FORMAT_RAW) {
+ PyErr_SetString(PyExc_ValueError,
+ "Cannot specify memory limit with FORMAT_RAW");
+ return -1;
+ }
+ memlimit = PyLong_AsUnsignedLongLong(memlimit_obj);
+ if (PyErr_Occurred())
+ return -1;
+ }
+
+ if (format == FORMAT_RAW && filterspecs == Py_None) {
+ PyErr_SetString(PyExc_ValueError,
+ "Must specify filters for FORMAT_RAW");
+ return -1;
+ } else if (format != FORMAT_RAW && filterspecs != Py_None) {
+ PyErr_SetString(PyExc_ValueError,
+ "Cannot specify filters except with FORMAT_RAW");
+ return -1;
+ }
+
+#ifdef WITH_THREAD
+ self->lock = PyThread_allocate_lock();
+ if (self->lock == NULL) {
+ PyErr_SetString(PyExc_MemoryError, "Unable to allocate lock");
+ return -1;
+ }
+#endif
+
+ self->check = LZMA_CHECK_UNKNOWN;
+ self->unused_data = PyBytes_FromStringAndSize(NULL, 0);
+ if (self->unused_data == NULL)
+ goto error;
+
+ switch (format) {
+ case FORMAT_AUTO:
+ lzret = lzma_auto_decoder(&self->lzs, memlimit, decoder_flags);
+ if (catch_lzma_error(lzret))
+ break;
+ return 0;
+
+ case FORMAT_XZ:
+ lzret = lzma_stream_decoder(&self->lzs, memlimit, decoder_flags);
+ if (catch_lzma_error(lzret))
+ break;
+ return 0;
+
+ case FORMAT_ALONE:
+ self->check = LZMA_CHECK_NONE;
+ lzret = lzma_alone_decoder(&self->lzs, memlimit);
+ if (catch_lzma_error(lzret))
+ break;
+ return 0;
+
+ case FORMAT_RAW:
+ self->check = LZMA_CHECK_NONE;
+ if (Decompressor_init_raw(&self->lzs, filterspecs) == -1)
+ break;
+ return 0;
+
+ default:
+ PyErr_Format(PyExc_ValueError,
+ "Invalid container format: %d", format);
+ break;
+ }
+
+error:
+ Py_CLEAR(self->unused_data);
+#ifdef WITH_THREAD
+ PyThread_free_lock(self->lock);
+ self->lock = NULL;
+#endif
+ return -1;
+}
+
+static void
+Decompressor_dealloc(Decompressor *self)
+{
+ lzma_end(&self->lzs);
+ Py_CLEAR(self->unused_data);
+#ifdef WITH_THREAD
+ if (self->lock != NULL)
+ PyThread_free_lock(self->lock);
+#endif
+ Py_TYPE(self)->tp_free((PyObject *)self);
+}
+
+static PyMethodDef Decompressor_methods[] = {
+ {"decompress", (PyCFunction)Decompressor_decompress, METH_VARARGS,
+ Decompressor_decompress_doc},
+ {NULL}
+};
+
+PyDoc_STRVAR(Decompressor_check_doc,
+"ID of the integrity check used by the input stream.");
+
+PyDoc_STRVAR(Decompressor_eof_doc,
+"True if the end-of-stream marker has been reached.");
+
+PyDoc_STRVAR(Decompressor_unused_data_doc,
+"Data found after the end of the compressed stream.");
+
+static PyMemberDef Decompressor_members[] = {
+ {"check", T_INT, offsetof(Decompressor, check), READONLY,
+ Decompressor_check_doc},
+ {"eof", T_BOOL, offsetof(Decompressor, eof), READONLY,
+ Decompressor_eof_doc},
+ {"unused_data", T_OBJECT_EX, offsetof(Decompressor, unused_data), READONLY,
+ Decompressor_unused_data_doc},
+ {NULL}
+};
+
+PyDoc_STRVAR(Decompressor_doc,
+"LZMADecompressor(format=FORMAT_AUTO, memlimit=None, filters=None)\n"
+"\n"
+"Create a decompressor object for decompressing data incrementally.\n"
+"\n"
+"format specifies the container format of the input stream. If this is\n"
+"FORMAT_AUTO (the default), the decompressor will automatically detect\n"
+"whether the input is FORMAT_XZ or FORMAT_ALONE. Streams created with\n"
+"FORMAT_RAW cannot be autodetected.\n"
+"\n"
+"memlimit can be specified to limit the amount of memory used by the\n"
+"decompressor. This will cause decompression to fail if the input\n"
+"cannot be decompressed within the given limit.\n"
+"\n"
+"filters specifies a custom filter chain. This argument is required for\n"
+"FORMAT_RAW, and not accepted with any other format. When provided,\n"
+"this should be a sequence of dicts, each indicating the ID and options\n"
+"for a single filter.\n"
+"\n"
+"For one-shot decompression, use the decompress() function instead.\n");
+
+static PyTypeObject Decompressor_type = {
+ PyVarObject_HEAD_INIT(NULL, 0)
+ "_lzma.LZMADecompressor", /* tp_name */
+ sizeof(Decompressor), /* tp_basicsize */
+ 0, /* tp_itemsize */
+ (destructor)Decompressor_dealloc, /* tp_dealloc */
+ 0, /* tp_print */
+ 0, /* tp_getattr */
+ 0, /* tp_setattr */
+ 0, /* tp_reserved */
+ 0, /* tp_repr */
+ 0, /* tp_as_number */
+ 0, /* tp_as_sequence */
+ 0, /* tp_as_mapping */
+ 0, /* tp_hash */
+ 0, /* tp_call */
+ 0, /* tp_str */
+ 0, /* tp_getattro */
+ 0, /* tp_setattro */
+ 0, /* tp_as_buffer */
+ Py_TPFLAGS_DEFAULT, /* tp_flags */
+ Decompressor_doc, /* tp_doc */
+ 0, /* tp_traverse */
+ 0, /* tp_clear */
+ 0, /* tp_richcompare */
+ 0, /* tp_weaklistoffset */
+ 0, /* tp_iter */
+ 0, /* tp_iternext */
+ Decompressor_methods, /* tp_methods */
+ Decompressor_members, /* tp_members */
+ 0, /* tp_getset */
+ 0, /* tp_base */
+ 0, /* tp_dict */
+ 0, /* tp_descr_get */
+ 0, /* tp_descr_set */
+ 0, /* tp_dictoffset */
+ (initproc)Decompressor_init, /* tp_init */
+ 0, /* tp_alloc */
+ PyType_GenericNew, /* tp_new */
+};
+
+
+/* Module-level functions. */
+
+PyDoc_STRVAR(check_is_supported_doc,
+"check_is_supported(check_id) -> bool\n"
+"\n"
+"Test whether the given integrity check is supported.\n"
+"\n"
+"Always returns True for CHECK_NONE and CHECK_CRC32.\n");
+
+static PyObject *
+check_is_supported(PyObject *self, PyObject *args)
+{
+ int check_id;
+
+ if (!PyArg_ParseTuple(args, "i:check_is_supported", &check_id))
+ return NULL;
+
+ return PyBool_FromLong(lzma_check_is_supported(check_id));
+}
+
+
+/* Module initialization. */
+
+static PyMethodDef module_methods[] = {
+ {"check_is_supported", (PyCFunction)check_is_supported,
+ METH_VARARGS, check_is_supported_doc},
+ {NULL}
+};
+
+static PyModuleDef _lzmamodule = {
+ PyModuleDef_HEAD_INIT,
+ "_lzma",
+ NULL,
+ -1,
+ module_methods,
+ NULL,
+ NULL,
+ NULL,
+ NULL,
+};
+
+/* Some of our constants are more than 32 bits wide, so PyModule_AddIntConstant
+ would not work correctly on platforms with 32-bit longs. */
+static int
+module_add_int_constant(PyObject *m, const char *name, PY_LONG_LONG value)
+{
+ PyObject *o = PyLong_FromLongLong(value);
+ if (o == NULL)
+ return -1;
+ if (PyModule_AddObject(m, name, o) == 0)
+ return 0;
+ Py_DECREF(o);
+ return -1;
+}
+
+#define ADD_INT_PREFIX_MACRO(m, macro) \
+ module_add_int_constant(m, #macro, LZMA_ ## macro)
+
+PyMODINIT_FUNC
+PyInit__lzma(void)
+{
+ PyObject *m;
+
+ empty_tuple = PyTuple_New(0);
+ if (empty_tuple == NULL)
+ return NULL;
+
+ m = PyModule_Create(&_lzmamodule);
+ if (m == NULL)
+ return NULL;
+
+ if (PyModule_AddIntMacro(m, FORMAT_AUTO) == -1 ||
+ PyModule_AddIntMacro(m, FORMAT_XZ) == -1 ||
+ PyModule_AddIntMacro(m, FORMAT_ALONE) == -1 ||
+ PyModule_AddIntMacro(m, FORMAT_RAW) == -1 ||
+ ADD_INT_PREFIX_MACRO(m, CHECK_NONE) == -1 ||
+ ADD_INT_PREFIX_MACRO(m, CHECK_CRC32) == -1 ||
+ ADD_INT_PREFIX_MACRO(m, CHECK_CRC64) == -1 ||
+ ADD_INT_PREFIX_MACRO(m, CHECK_SHA256) == -1 ||
+ ADD_INT_PREFIX_MACRO(m, CHECK_ID_MAX) == -1 ||
+ ADD_INT_PREFIX_MACRO(m, CHECK_UNKNOWN) == -1 ||
+ ADD_INT_PREFIX_MACRO(m, FILTER_LZMA1) == -1 ||
+ ADD_INT_PREFIX_MACRO(m, FILTER_LZMA2) == -1 ||
+ ADD_INT_PREFIX_MACRO(m, FILTER_DELTA) == -1 ||
+ ADD_INT_PREFIX_MACRO(m, FILTER_X86) == -1 ||
+ ADD_INT_PREFIX_MACRO(m, FILTER_IA64) == -1 ||
+ ADD_INT_PREFIX_MACRO(m, FILTER_ARM) == -1 ||
+ ADD_INT_PREFIX_MACRO(m, FILTER_ARMTHUMB) == -1 ||
+ ADD_INT_PREFIX_MACRO(m, FILTER_SPARC) == -1 ||
+ ADD_INT_PREFIX_MACRO(m, FILTER_POWERPC) == -1 ||
+ ADD_INT_PREFIX_MACRO(m, MF_HC3) == -1 ||
+ ADD_INT_PREFIX_MACRO(m, MF_HC4) == -1 ||
+ ADD_INT_PREFIX_MACRO(m, MF_BT2) == -1 ||
+ ADD_INT_PREFIX_MACRO(m, MF_BT3) == -1 ||
+ ADD_INT_PREFIX_MACRO(m, MF_BT4) == -1 ||
+ ADD_INT_PREFIX_MACRO(m, MODE_FAST) == -1 ||
+ ADD_INT_PREFIX_MACRO(m, MODE_NORMAL) == -1 ||
+ ADD_INT_PREFIX_MACRO(m, PRESET_DEFAULT) == -1 ||
+ ADD_INT_PREFIX_MACRO(m, PRESET_EXTREME) == -1)
+ return NULL;
+
+ Error = PyErr_NewExceptionWithDoc(
+ "_lzma.LZMAError", "Call to liblzma failed.", NULL, NULL);
+ if (Error == NULL)
+ return NULL;
+ Py_INCREF(Error);
+ if (PyModule_AddObject(m, "LZMAError", Error) == -1)
+ return NULL;
+
+ if (PyType_Ready(&Compressor_type) == -1)
+ return NULL;
+ Py_INCREF(&Compressor_type);
+ if (PyModule_AddObject(m, "LZMACompressor",
+ (PyObject *)&Compressor_type) == -1)
+ return NULL;
+
+ if (PyType_Ready(&Decompressor_type) == -1)
+ return NULL;
+ Py_INCREF(&Decompressor_type);
+ if (PyModule_AddObject(m, "LZMADecompressor",
+ (PyObject *)&Decompressor_type) == -1)
+ return NULL;
+
+ return m;
+}
diff --git a/PCbuild/_lzma.vcproj b/PCbuild/_lzma.vcproj
new file mode 100644
index 0000000..10f3722
--- /dev/null
+++ b/PCbuild/_lzma.vcproj
@@ -0,0 +1,537 @@
+<?xml version="1.0" encoding="Windows-1252"?>
+<VisualStudioProject
+ ProjectType="Visual C++"
+ Version="9,00"
+ Name="_lzma"
+ ProjectGUID="{F9D71780-F393-11E0-BE50-0800200C9A66}"
+ RootNamespace="lzma"
+ Keyword="Win32Proj"
+ TargetFrameworkVersion="196613"
+ >
+ <Platforms>
+ <Platform
+ Name="Win32"
+ />
+ <Platform
+ Name="x64"
+ />
+ </Platforms>
+ <ToolFiles>
+ </ToolFiles>
+ <Configurations>
+ <Configuration
+ Name="Debug|Win32"
+ ConfigurationType="2"
+ InheritedPropertySheets=".\pyd_d.vsprops"
+ CharacterSet="0"
+ >
+ <Tool
+ Name="VCPreBuildEventTool"
+ />
+ <Tool
+ Name="VCCustomBuildTool"
+ />
+ <Tool
+ Name="VCXMLDataGeneratorTool"
+ />
+ <Tool
+ Name="VCWebServiceProxyGeneratorTool"
+ />
+ <Tool
+ Name="VCMIDLTool"
+ />
+ <Tool
+ Name="VCCLCompilerTool"
+ AdditionalIncludeDirectories="$(lzmaDir)\include"
+ PreprocessorDefinitions="WIN32;_FILE_OFFSET_BITS=64;_CRT_SECURE_NO_DEPRECATE;_CRT_NONSTDC_NO_DEPRECATE;LZMA_API_STATIC"
+ />
+ <Tool
+ Name="VCManagedResourceCompilerTool"
+ />
+ <Tool
+ Name="VCResourceCompilerTool"
+ />
+ <Tool
+ Name="VCPreLinkEventTool"
+ />
+ <Tool
+ Name="VCLinkerTool"
+ AdditionalDependencies="$(lzmaDir)\bin_i486\liblzma.a"
+ />
+ <Tool
+ Name="VCALinkTool"
+ />
+ <Tool
+ Name="VCManifestTool"
+ />
+ <Tool
+ Name="VCXDCMakeTool"
+ />
+ <Tool
+ Name="VCBscMakeTool"
+ />
+ <Tool
+ Name="VCFxCopTool"
+ />
+ <Tool
+ Name="VCAppVerifierTool"
+ />
+ <Tool
+ Name="VCPostBuildEventTool"
+ />
+ </Configuration>
+ <Configuration
+ Name="Debug|x64"
+ ConfigurationType="2"
+ InheritedPropertySheets=".\pyd_d.vsprops;.\x64.vsprops"
+ CharacterSet="0"
+ >
+ <Tool
+ Name="VCPreBuildEventTool"
+ />
+ <Tool
+ Name="VCCustomBuildTool"
+ />
+ <Tool
+ Name="VCXMLDataGeneratorTool"
+ />
+ <Tool
+ Name="VCWebServiceProxyGeneratorTool"
+ />
+ <Tool
+ Name="VCMIDLTool"
+ TargetEnvironment="3"
+ />
+ <Tool
+ Name="VCCLCompilerTool"
+ AdditionalIncludeDirectories="$(lzmaDir)\include"
+ PreprocessorDefinitions="WIN32;_FILE_OFFSET_BITS=64;_CRT_SECURE_NO_DEPRECATE;_CRT_NONSTDC_NO_DEPRECATE;LZMA_API_STATIC"
+ />
+ <Tool
+ Name="VCManagedResourceCompilerTool"
+ />
+ <Tool
+ Name="VCResourceCompilerTool"
+ />
+ <Tool
+ Name="VCPreLinkEventTool"
+ />
+ <Tool
+ Name="VCLinkerTool"
+ AdditionalDependencies="$(lzmaDir)\bin_x86-64\liblzma.a"
+ />
+ <Tool
+ Name="VCALinkTool"
+ />
+ <Tool
+ Name="VCManifestTool"
+ />
+ <Tool
+ Name="VCXDCMakeTool"
+ />
+ <Tool
+ Name="VCBscMakeTool"
+ />
+ <Tool
+ Name="VCFxCopTool"
+ />
+ <Tool
+ Name="VCAppVerifierTool"
+ />
+ <Tool
+ Name="VCPostBuildEventTool"
+ />
+ </Configuration>
+ <Configuration
+ Name="Release|Win32"
+ ConfigurationType="2"
+ InheritedPropertySheets=".\pyd.vsprops"
+ CharacterSet="0"
+ WholeProgramOptimization="1"
+ >
+ <Tool
+ Name="VCPreBuildEventTool"
+ />
+ <Tool
+ Name="VCCustomBuildTool"
+ />
+ <Tool
+ Name="VCXMLDataGeneratorTool"
+ />
+ <Tool
+ Name="VCWebServiceProxyGeneratorTool"
+ />
+ <Tool
+ Name="VCMIDLTool"
+ />
+ <Tool
+ Name="VCCLCompilerTool"
+ AdditionalIncludeDirectories="$(lzmaDir)\include"
+ PreprocessorDefinitions="WIN32;_FILE_OFFSET_BITS=64;_CRT_SECURE_NO_DEPRECATE;_CRT_NONSTDC_NO_DEPRECATE;LZMA_API_STATIC"
+ />
+ <Tool
+ Name="VCManagedResourceCompilerTool"
+ />
+ <Tool
+ Name="VCResourceCompilerTool"
+ />
+ <Tool
+ Name="VCPreLinkEventTool"
+ />
+ <Tool
+ Name="VCLinkerTool"
+ AdditionalDependencies="$(lzmaDir)\bin_i486\liblzma.a"
+ />
+ <Tool
+ Name="VCALinkTool"
+ />
+ <Tool
+ Name="VCManifestTool"
+ />
+ <Tool
+ Name="VCXDCMakeTool"
+ />
+ <Tool
+ Name="VCBscMakeTool"
+ />
+ <Tool
+ Name="VCFxCopTool"
+ />
+ <Tool
+ Name="VCAppVerifierTool"
+ />
+ <Tool
+ Name="VCPostBuildEventTool"
+ />
+ </Configuration>
+ <Configuration
+ Name="Release|x64"
+ ConfigurationType="2"
+ InheritedPropertySheets=".\pyd.vsprops;.\x64.vsprops"
+ CharacterSet="0"
+ WholeProgramOptimization="1"
+ >
+ <Tool
+ Name="VCPreBuildEventTool"
+ />
+ <Tool
+ Name="VCCustomBuildTool"
+ />
+ <Tool
+ Name="VCXMLDataGeneratorTool"
+ />
+ <Tool
+ Name="VCWebServiceProxyGeneratorTool"
+ />
+ <Tool
+ Name="VCMIDLTool"
+ TargetEnvironment="3"
+ />
+ <Tool
+ Name="VCCLCompilerTool"
+ AdditionalIncludeDirectories="$(lzmaDir)\include"
+ PreprocessorDefinitions="WIN32;_FILE_OFFSET_BITS=64;_CRT_SECURE_NO_DEPRECATE;_CRT_NONSTDC_NO_DEPRECATE;LZMA_API_STATIC"
+ />
+ <Tool
+ Name="VCManagedResourceCompilerTool"
+ />
+ <Tool
+ Name="VCResourceCompilerTool"
+ />
+ <Tool
+ Name="VCPreLinkEventTool"
+ />
+ <Tool
+ Name="VCLinkerTool"
+ AdditionalDependencies="$(lzmaDir)\bin_x86-64\liblzma.a"
+ />
+ <Tool
+ Name="VCALinkTool"
+ />
+ <Tool
+ Name="VCManifestTool"
+ />
+ <Tool
+ Name="VCXDCMakeTool"
+ />
+ <Tool
+ Name="VCBscMakeTool"
+ />
+ <Tool
+ Name="VCFxCopTool"
+ />
+ <Tool
+ Name="VCAppVerifierTool"
+ />
+ <Tool
+ Name="VCPostBuildEventTool"
+ />
+ </Configuration>
+ <Configuration
+ Name="PGInstrument|Win32"
+ ConfigurationType="2"
+ InheritedPropertySheets=".\pyd.vsprops;.\pginstrument.vsprops"
+ CharacterSet="0"
+ WholeProgramOptimization="1"
+ >
+ <Tool
+ Name="VCPreBuildEventTool"
+ />
+ <Tool
+ Name="VCCustomBuildTool"
+ />
+ <Tool
+ Name="VCXMLDataGeneratorTool"
+ />
+ <Tool
+ Name="VCWebServiceProxyGeneratorTool"
+ />
+ <Tool
+ Name="VCMIDLTool"
+ />
+ <Tool
+ Name="VCCLCompilerTool"
+ AdditionalIncludeDirectories="$(lzmaDir)\include"
+ PreprocessorDefinitions="WIN32;_FILE_OFFSET_BITS=64;_CRT_SECURE_NO_DEPRECATE;_CRT_NONSTDC_NO_DEPRECATE;LZMA_API_STATIC"
+ />
+ <Tool
+ Name="VCManagedResourceCompilerTool"
+ />
+ <Tool
+ Name="VCResourceCompilerTool"
+ />
+ <Tool
+ Name="VCPreLinkEventTool"
+ />
+ <Tool
+ Name="VCLinkerTool"
+ AdditionalDependencies="$(lzmaDir)\bin_i486\liblzma.a"
+ />
+ <Tool
+ Name="VCALinkTool"
+ />
+ <Tool
+ Name="VCManifestTool"
+ />
+ <Tool
+ Name="VCXDCMakeTool"
+ />
+ <Tool
+ Name="VCBscMakeTool"
+ />
+ <Tool
+ Name="VCFxCopTool"
+ />
+ <Tool
+ Name="VCAppVerifierTool"
+ />
+ <Tool
+ Name="VCPostBuildEventTool"
+ />
+ </Configuration>
+ <Configuration
+ Name="PGInstrument|x64"
+ ConfigurationType="2"
+ InheritedPropertySheets=".\pyd.vsprops;.\x64.vsprops;.\pginstrument.vsprops"
+ CharacterSet="0"
+ WholeProgramOptimization="1"
+ >
+ <Tool
+ Name="VCPreBuildEventTool"
+ />
+ <Tool
+ Name="VCCustomBuildTool"
+ />
+ <Tool
+ Name="VCXMLDataGeneratorTool"
+ />
+ <Tool
+ Name="VCWebServiceProxyGeneratorTool"
+ />
+ <Tool
+ Name="VCMIDLTool"
+ TargetEnvironment="3"
+ />
+ <Tool
+ Name="VCCLCompilerTool"
+ AdditionalIncludeDirectories="$(lzmaDir)\include"
+ PreprocessorDefinitions="WIN32;_FILE_OFFSET_BITS=64;_CRT_SECURE_NO_DEPRECATE;_CRT_NONSTDC_NO_DEPRECATE;LZMA_API_STATIC"
+ />
+ <Tool
+ Name="VCManagedResourceCompilerTool"
+ />
+ <Tool
+ Name="VCResourceCompilerTool"
+ />
+ <Tool
+ Name="VCPreLinkEventTool"
+ />
+ <Tool
+ Name="VCLinkerTool"
+ AdditionalDependencies="$(lzmaDir)\bin_x86-64\liblzma.a"
+ TargetMachine="17"
+ />
+ <Tool
+ Name="VCALinkTool"
+ />
+ <Tool
+ Name="VCManifestTool"
+ />
+ <Tool
+ Name="VCXDCMakeTool"
+ />
+ <Tool
+ Name="VCBscMakeTool"
+ />
+ <Tool
+ Name="VCFxCopTool"
+ />
+ <Tool
+ Name="VCAppVerifierTool"
+ />
+ <Tool
+ Name="VCPostBuildEventTool"
+ />
+ </Configuration>
+ <Configuration
+ Name="PGUpdate|Win32"
+ ConfigurationType="2"
+ InheritedPropertySheets=".\pyd.vsprops;.\pgupdate.vsprops"
+ CharacterSet="0"
+ WholeProgramOptimization="1"
+ >
+ <Tool
+ Name="VCPreBuildEventTool"
+ />
+ <Tool
+ Name="VCCustomBuildTool"
+ />
+ <Tool
+ Name="VCXMLDataGeneratorTool"
+ />
+ <Tool
+ Name="VCWebServiceProxyGeneratorTool"
+ />
+ <Tool
+ Name="VCMIDLTool"
+ />
+ <Tool
+ Name="VCCLCompilerTool"
+ AdditionalIncludeDirectories="$(lzmaDir)\include"
+ PreprocessorDefinitions="WIN32;_FILE_OFFSET_BITS=64;_CRT_SECURE_NO_DEPRECATE;_CRT_NONSTDC_NO_DEPRECATE;LZMA_API_STATIC"
+ />
+ <Tool
+ Name="VCManagedResourceCompilerTool"
+ />
+ <Tool
+ Name="VCResourceCompilerTool"
+ />
+ <Tool
+ Name="VCPreLinkEventTool"
+ />
+ <Tool
+ Name="VCLinkerTool"
+ AdditionalDependencies="$(lzmaDir)\bin_i486\liblzma.a"
+ />
+ <Tool
+ Name="VCALinkTool"
+ />
+ <Tool
+ Name="VCManifestTool"
+ />
+ <Tool
+ Name="VCXDCMakeTool"
+ />
+ <Tool
+ Name="VCBscMakeTool"
+ />
+ <Tool
+ Name="VCFxCopTool"
+ />
+ <Tool
+ Name="VCAppVerifierTool"
+ />
+ <Tool
+ Name="VCPostBuildEventTool"
+ />
+ </Configuration>
+ <Configuration
+ Name="PGUpdate|x64"
+ ConfigurationType="2"
+ InheritedPropertySheets=".\pyd.vsprops;.\x64.vsprops;.\pgupdate.vsprops"
+ CharacterSet="0"
+ WholeProgramOptimization="1"
+ >
+ <Tool
+ Name="VCPreBuildEventTool"
+ />
+ <Tool
+ Name="VCCustomBuildTool"
+ />
+ <Tool
+ Name="VCXMLDataGeneratorTool"
+ />
+ <Tool
+ Name="VCWebServiceProxyGeneratorTool"
+ />
+ <Tool
+ Name="VCMIDLTool"
+ TargetEnvironment="3"
+ />
+ <Tool
+ Name="VCCLCompilerTool"
+ AdditionalIncludeDirectories="$(lzmaDir)\include"
+ PreprocessorDefinitions="WIN32;_FILE_OFFSET_BITS=64;_CRT_SECURE_NO_DEPRECATE;_CRT_NONSTDC_NO_DEPRECATE;LZMA_API_STATIC"
+ />
+ <Tool
+ Name="VCManagedResourceCompilerTool"
+ />
+ <Tool
+ Name="VCResourceCompilerTool"
+ />
+ <Tool
+ Name="VCPreLinkEventTool"
+ />
+ <Tool
+ Name="VCLinkerTool"
+ AdditionalDependencies="$(lzmaDir)\bin_x86-64\liblzma.a"
+ TargetMachine="17"
+ />
+ <Tool
+ Name="VCALinkTool"
+ />
+ <Tool
+ Name="VCManifestTool"
+ />
+ <Tool
+ Name="VCXDCMakeTool"
+ />
+ <Tool
+ Name="VCBscMakeTool"
+ />
+ <Tool
+ Name="VCFxCopTool"
+ />
+ <Tool
+ Name="VCAppVerifierTool"
+ />
+ <Tool
+ Name="VCPostBuildEventTool"
+ />
+ </Configuration>
+ </Configurations>
+ <References>
+ </References>
+ <Files>
+ <Filter
+ Name="Source Files"
+ >
+ <File
+ RelativePath="..\Modules\_lzmamodule.c"
+ >
+ </File>
+ </Filter>
+ </Files>
+ <Globals>
+ </Globals>
+</VisualStudioProject>
diff --git a/PCbuild/pcbuild.sln b/PCbuild/pcbuild.sln
index ed3a7a0..9efb6d9 100644
--- a/PCbuild/pcbuild.sln
+++ b/PCbuild/pcbuild.sln
@@ -92,6 +92,11 @@
{CF7AC3D1-E2DF-41D2-BEA6-1E2556CDEA26} = {CF7AC3D1-E2DF-41D2-BEA6-1E2556CDEA26}
EndProjectSection
EndProject
+Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "_lzma", "_lzma.vcproj", "{F9D71780-F393-11E0-BE50-0800200C9A66}"
+ ProjectSection(ProjectDependencies) = postProject
+ {CF7AC3D1-E2DF-41D2-BEA6-1E2556CDEA26} = {CF7AC3D1-E2DF-41D2-BEA6-1E2556CDEA26}
+ EndProjectSection
+EndProject
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "select", "select.vcproj", "{18CAE28C-B454-46C1-87A0-493D91D97F03}"
ProjectSection(ProjectDependencies) = postProject
{CF7AC3D1-E2DF-41D2-BEA6-1E2556CDEA26} = {CF7AC3D1-E2DF-41D2-BEA6-1E2556CDEA26}
@@ -421,6 +426,22 @@
{73FCD2BD-F133-46B7-8EC1-144CD82A59D5}.Release|Win32.Build.0 = Release|Win32
{73FCD2BD-F133-46B7-8EC1-144CD82A59D5}.Release|x64.ActiveCfg = Release|x64
{73FCD2BD-F133-46B7-8EC1-144CD82A59D5}.Release|x64.Build.0 = Release|x64
+ {F9D71780-F393-11E0-BE50-0800200C9A66}.Debug|Win32.ActiveCfg = Debug|Win32
+ {F9D71780-F393-11E0-BE50-0800200C9A66}.Debug|Win32.Build.0 = Debug|Win32
+ {F9D71780-F393-11E0-BE50-0800200C9A66}.Debug|x64.ActiveCfg = Debug|x64
+ {F9D71780-F393-11E0-BE50-0800200C9A66}.Debug|x64.Build.0 = Debug|x64
+ {F9D71780-F393-11E0-BE50-0800200C9A66}.PGInstrument|Win32.ActiveCfg = PGInstrument|Win32
+ {F9D71780-F393-11E0-BE50-0800200C9A66}.PGInstrument|Win32.Build.0 = PGInstrument|Win32
+ {F9D71780-F393-11E0-BE50-0800200C9A66}.PGInstrument|x64.ActiveCfg = PGInstrument|x64
+ {F9D71780-F393-11E0-BE50-0800200C9A66}.PGInstrument|x64.Build.0 = PGInstrument|x64
+ {F9D71780-F393-11E0-BE50-0800200C9A66}.PGUpdate|Win32.ActiveCfg = PGUpdate|Win32
+ {F9D71780-F393-11E0-BE50-0800200C9A66}.PGUpdate|Win32.Build.0 = PGUpdate|Win32
+ {F9D71780-F393-11E0-BE50-0800200C9A66}.PGUpdate|x64.ActiveCfg = PGUpdate|x64
+ {F9D71780-F393-11E0-BE50-0800200C9A66}.PGUpdate|x64.Build.0 = PGUpdate|x64
+ {F9D71780-F393-11E0-BE50-0800200C9A66}.Release|Win32.ActiveCfg = Release|Win32
+ {F9D71780-F393-11E0-BE50-0800200C9A66}.Release|Win32.Build.0 = Release|Win32
+ {F9D71780-F393-11E0-BE50-0800200C9A66}.Release|x64.ActiveCfg = Release|x64
+ {F9D71780-F393-11E0-BE50-0800200C9A66}.Release|x64.Build.0 = Release|x64
{18CAE28C-B454-46C1-87A0-493D91D97F03}.Debug|Win32.ActiveCfg = Debug|Win32
{18CAE28C-B454-46C1-87A0-493D91D97F03}.Debug|Win32.Build.0 = Debug|Win32
{18CAE28C-B454-46C1-87A0-493D91D97F03}.Debug|x64.ActiveCfg = Debug|x64
diff --git a/PCbuild/pyproject.vsprops b/PCbuild/pyproject.vsprops
index 4eab942..441f95f 100644
--- a/PCbuild/pyproject.vsprops
+++ b/PCbuild/pyproject.vsprops
@@ -57,6 +57,10 @@
Value="$(externalsDir)\bzip2-1.0.5"
/>
<UserMacro
+ Name="lzmaDir"
+ Value="$(externalsDir)\xz-5.0.3"
+ />
+ <UserMacro
Name="opensslDir"
Value="$(externalsDir)\openssl-1.0.0a"
/>
diff --git a/PCbuild/readme.txt b/PCbuild/readme.txt
index 11ab127..f47d555 100644
--- a/PCbuild/readme.txt
+++ b/PCbuild/readme.txt
@@ -133,6 +133,12 @@
All of this managed to build libbz2.lib in
bzip2-1.0.5\$platform-$configuration\, which the Python project links in.
+_lzma
+ Python wrapper for the liblzma compression library.
+
+ Download the pre-built Windows binaries from http://tukaani.org/xz/, and
+ extract to ..\xz-5.0.3. If you are using a more recent version of liblzma,
+ it will be necessary to rename the directory from xz-<VERSION> to xz-5.0.3.
_ssl
Python wrapper for the secure sockets library.
diff --git a/Tools/buildbot/external-common.bat b/Tools/buildbot/external-common.bat
index 1ff282e..244d5f5 100644
--- a/Tools/buildbot/external-common.bat
+++ b/Tools/buildbot/external-common.bat
@@ -41,3 +41,8 @@
rd /s/q sqlite-source-3.6.21
svn export http://svn.python.org/projects/external/sqlite-3.7.4
)
+
+@rem lzma
+if not exist xz-5.0.3 (
+ svn export http://svn.python.org/projects/external/xz-5.0.3
+)
diff --git a/setup.py b/setup.py
index fff4361..39aff67 100644
--- a/setup.py
+++ b/setup.py
@@ -1279,6 +1279,13 @@
else:
missing.append('_bz2')
+ # LZMA compression support.
+ if self.compiler.find_library_file(lib_dirs, 'lzma'):
+ exts.append( Extension('_lzma', ['_lzmamodule.c'],
+ libraries = ['lzma']) )
+ else:
+ missing.append('_lzma')
+
# Interface to the Expat XML parser
#
# Expat was written by James Clark and is now maintained by a group of