| commit | aa074051d6d2c9057897af4ab0b14388aaa93ef2 | [log] [tgz] |
|---|---|---|
| author | Yann Collet <yann.collet.73@gmail.com> | Fri Oct 30 11:21:50 2015 +0100 |
| committer | Yann Collet <yann.collet.73@gmail.com> | Fri Oct 30 11:21:50 2015 +0100 |
| tree | c3c7d4c5feddbabfa6c900fae3882fa830f85316 | |
| parent | fc04bcc6d73d67c1f93c92eeb214db322b303ea5 [diff] |
decoding support for legacy v0.2 format
- lib/Makefile[diff]
- lib/error.h[diff]
- lib/fse.h[diff]
- lib/fse_static.h[diff]
- lib/huff0.c[diff]
- lib/huff0.h[diff]
- lib/huff0_static.h[diff]
- lib/legacy/zstd_legacy.h[Added - diff]
- lib/legacy/zstd_v01.c[diff]
- lib/legacy/zstd_v02.c[Added - diff]
- lib/legacy/zstd_v02.h[Added - diff]
- lib/zstd.c[diff]
- lib/zstd.h[diff]
- lib/zstd_static.h[diff]
- programs/Makefile[diff]
- programs/fileio.c[diff]
- programs/legacy/fileio_legacy.c[Added - diff]
- programs/legacy/fileio_legacy.h[Added - diff]
- programs/paramgrill.c[diff]
Zstd, short for Zstandard, is a fast lossless compression algorithm, targeting real-time compression scenarios, and featuring zlib-level compression ratio.
It is provided as a BSD-license package, hosted on Github.
| Branch | Status |
|---|---|
| master |  |
| dev |  |
For a taste of its performance, here are a few benchmark numbers from a number of compression codecs suitable for real-time. The test was completed on a Core i7-5600U @ 2.6 GHz, using fsbench 0.14.3, an open-source benchmark program by m^2.
| Name | Ratio | C.speed | D.speed |
|---|---|---|---|
| MB/s | MB/s | ||
| zstd 0.2 | 2.871 | 255 | 670 |
| zlib 1.2.8 -1 | 2.730 | 70 | 300 |
| QuickLZ 1.5.1b6 | 2.237 | 370 | 415 |
| LZO 2.06 | 2.106 | 400 | 580 |
| LZ4 r131 | 2.101 | 450 | 2100 |
| Snappy 1.1.0 | 2.091 | 330 | 1100 |
| LZF 3.6 | 2.077 | 200 | 560 |
Zstd strong feature is its very high decompression speed, at more than >600 MB/s per core. Obviously, your exact mileage will vary depending on target system.
Zstd compression speed will be configurable to fit different situations. The first available version is the fast one, at ~250 MB/s per core, which is suitable for a few real-time scenarios. But similar to LZ4, zstd can offer derivatives trading compression time for compression ratio, keeping decompression properties intact. "Offline compression", where compression time is of little importance because the content is only compressed once and decompressed many times, will likely prefer this setup.
Note that high compression derivatives still have to be developed. It's a complex area which will require time and benefit from contributions.
Another property zstd is developed for is configurable memory requirement, with the objective to fit into low-memory configurations, or servers handling many connections in parallel.
Zstd entropy stage is provided by Huff0 and FSE, from Finite State Entrop library.
Zstd has not yet reached "stable" status. Specifically, it doesn't guarantee yet that its current compressed format will remain stable and supported in future versions. It may still change to adapt further optimizations still being investigated. However, the library starts to be pretty robust, able to withstand hazards situations, including invalid input. The library reliability has been tested using Fuzz Testing, using both internal tools and external ones. Therefore, you can now safely test zstd, even within production environments.
"Stable Format" is projected sometimes early 2016.
Branch Policy
The "dev" branch is the one where all contributions will be merged before reaching "master". If you plan to propose a patch, please commit into the "dev" branch or its own feature branch. Direct commit to "master" are not permitted.