blob: b8ff8783e49babb3df369b0bc1c390075bf9c670 [file] [log] [blame] [view]
Yann Collet4b100f42015-10-30 15:49:48 +01001 **Zstd**, short for Zstandard, is a fast lossless compression algorithm, targeting real-time compression scenarios at zlib-level compression ratio.
Yann Collet4856a002015-01-24 01:58:16 +01002
3It is provided as a BSD-license package, hosted on Github.
4
5|Branch |Status |
6|------------|---------|
7|master | [![Build Status](https://travis-ci.org/Cyan4973/zstd.svg?branch=master)](https://travis-ci.org/Cyan4973/zstd) |
8|dev | [![Build Status](https://travis-ci.org/Cyan4973/zstd.svg?branch=dev)](https://travis-ci.org/Cyan4973/zstd) |
9
Yann Colletacd222c2015-11-06 12:39:39 +010010For a taste of its performance, here are a few benchmark numbers from a number of compression codecs suitable for real-time. The test was completed on a Core i7-5600U @ 2.6 GHz, using m^2's [fsbench 0.14.3](http://encode.ru/threads/1371-Filesystem-benchmark?p=34029&viewfull=1#post34029) compiled with gcc 4.8.4, on the [Silesia compression corpus](http://sun.aei.polsl.pl/~sdeor/index.php?page=silesia).
Yann Collet4856a002015-01-24 01:58:16 +010011
Yann Collet4b100f42015-10-30 15:49:48 +010012|Name | Ratio | C.speed | D.speed |
13|-----------------|-------|--------:|--------:|
14| | | MB/s | MB/s |
15| **zstd 0.3** |**2.858**|**280**| **670** |
Yann Collet66d22b82015-11-02 02:36:10 +010016| [zlib] 1.2.8 -1 | 2.730 | 70 | 300 |
Yann Collet4b100f42015-10-30 15:49:48 +010017| QuickLZ 1.5.1b6 | 2.237 | 370 | 415 |
18| LZO 2.06 | 2.106 | 400 | 580 |
19| [LZ4] r131 | 2.101 | 450 | 2100 |
20| Snappy 1.1.0 | 2.091 | 330 | 1100 |
21| LZF 3.6 | 2.077 | 200 | 560 |
Yann Collet56213d82015-08-07 20:15:27 +010022
Yann Collet66d22b82015-11-02 02:36:10 +010023[zlib]:http://www.zlib.net/
Yann Collet56213d82015-08-07 20:15:27 +010024[LZ4]:http://www.lz4.org/
Yann Collet4856a002015-01-24 01:58:16 +010025
Yann Colletacd222c2015-11-06 12:39:39 +010026Zstd can also offer stronger compression ratio at the cost of compression speed. Speed / Ratio trade-off is configurable by small increment, to fit different situations. Note however that decompression speed is preserved and remain roughly the same at all settings, a property shared by most LZ compression algorithms, such as [zlib]. The following test is run on a Core i7-3930K CPU @ 4.5GHz, using [lzbench], an open-source in-memory benchmark by inikep compiled with gcc 5.2.1, on the [Silesia compression corpus](http://sun.aei.polsl.pl/~sdeor/index.php?page=silesia).
Yann Collet4856a002015-01-24 01:58:16 +010027
Yann Collet66d22b82015-11-02 02:36:10 +010028[lzbench]:https://github.com/inikep/lzbench
Yann Collet7671f392015-11-02 12:17:39 +010029
30Compression Speed vs Ratio | Decompression Speed
Yann Collet8d8d59e2015-11-02 02:44:43 +010031---------------------------|--------------------
Yann Collet7671f392015-11-02 12:17:39 +010032![Compression Speed vs Ratio](images/CSpeed.png "Compression Speed vs Ratio") | ![Decompression Speed](images/DSpeed.png "Decompression Speed")
Yann Collet8d8d59e2015-11-02 02:44:43 +010033
Yann Collet4856a002015-01-24 01:58:16 +010034
Yann Colletacd222c2015-11-06 12:39:39 +010035Zstd entropy stage is provided by [Huff0 and FSE, from Finite State Entropy library](https://github.com/Cyan4973/FiniteStateEntropy).
Yann Colletf506c8b2015-01-24 10:31:47 +010036
Yann Colletacd222c2015-11-06 12:39:39 +010037Its memory requirement can be configured to fit into low-memory hardware configurations, or servers handling multiple connections/contexts in parallel.
Yann Colletdb9559f2015-08-24 00:14:40 +010038
Yann Collet66d22b82015-11-02 02:36:10 +010039Zstd has not yet reached "stable format" status. It doesn't guarantee yet that its current compressed format will remain stable and supported in future versions. During this period, it can still change to adapt new optimizations still being investigated. "Stable Format" is projected sometimes early 2016.
40
41That being said, the library is now fairly robust, able to withstand hazards situations, including invalid inputs. The library reliability has been tested using [Fuzz Testing](https://en.wikipedia.org/wiki/Fuzz_testing), with both [internal tools](programs/fuzzer.c) and [external ones](http://lcamtuf.coredump.cx/afl). Therefore, it seems now safe to test Zstandard even within production environments.
Yann Collet4856a002015-01-24 01:58:16 +010042
Yann Collet56213d82015-08-07 20:15:27 +010043### Branch Policy
44The "dev" branch is the one where all contributions will be merged before reaching "master". If you plan to propose a patch, please commit into the "dev" branch or its own feature branch. Direct commit to "master" are not permitted.