| \" |
| \" zstd.1: This is a manual page for 'zstd' program. This file is part of the |
| \" zstd <http://www.zstd.net/> project. |
| \" Author: Yann Collet |
| \" |
| |
| \" No hyphenation |
| .hy 0 |
| .nr HY 0 |
| |
| .TH zstd "1" "2015-08-22" "zstd" "User Commands" |
| .SH NAME |
| \fBzstd, unzstd, zstdcat\fR - Compress or decompress .zst files |
| |
| .SH SYNOPSIS |
| .TP 5 |
| \fBzstd\fR [\fBOPTIONS\fR] [-|INPUT-FILE] [-o <OUTPUT-FILE>] |
| .PP |
| .B unzstd |
| is equivalent to |
| .BR "zstd \-d" |
| .br |
| .B zstdcat |
| is equivalent to |
| .BR "zstd \-dcf" |
| .br |
| |
| .SH DESCRIPTION |
| .PP |
| \fBzstd\fR is a fast lossless compression algorithm |
| and data compression tool, |
| with command line syntax similar to \fB gzip (1) \fR and \fB xz (1) \fR . |
| It is based on the \fBLZ77\fR family, with further FSE & huff0 entropy stages. |
| \fBzstd\fR offers highly configurable compression speed, |
| with fast modes at > 200 MB/s per core, |
| and strong modes nearing lzma compression ratios. |
| It also features a very fast decoder, with speeds > 500 MB/s per core. |
| |
| \fBzstd\fR command line syntax is generally similar to gzip, |
| but features the following differences : |
| - Source files are preserved by default. |
| It's possible to remove them automatically by using \fB--rm\fR command. |
| - When compressing a single file, \fBzstd\fR displays progress notifications and result summary by default. |
| Use \fB-q\fR to turn them off |
| |
| .PP |
| .B zstd |
| compresses or decompresses each |
| .I file |
| according to the selected operation mode. |
| If no |
| .I files |
| are given or |
| .I file |
| is |
| .BR \- , |
| .B zstd |
| reads from standard input and writes the processed data |
| to standard output. |
| .B zstd |
| will refuse (display an error and skip the |
| .IR file ) |
| to write compressed data to standard output if it is a terminal. |
| Similarly, |
| .B zstd |
| will refuse to read compressed data |
| from standard input if it is a terminal. |
| |
| .PP |
| Unless |
| .B \-\-stdout |
| or |
| .B \-o |
| is specified, |
| .I files |
| are written to a new file whose name is derived from the source |
| .I file |
| name: |
| .IP \(bu 3 |
| When compressing, the suffix |
| .B .zst |
| is appended to the source filename to get the target filename. |
| .IP \(bu 3 |
| When decompressing, the |
| .B .zst |
| suffix is removed from the filename to get the target filename. |
| |
| .SS "Concatenation with .zst files" |
| It is possible to concatenate |
| .B .zst |
| files as is. |
| .B zstd |
| will decompress such files as if they were a single |
| .B .zst |
| file. |
| |
| |
| |
| .SH OPTIONS |
| |
| . |
| .SS "Integer suffixes and special values" |
| In most places where an integer argument is expected, |
| an optional suffix is supported to easily indicate large integers. |
| There must be no space between the integer and the suffix. |
| .TP |
| .B KiB |
| Multiply the integer by 1,024 (2^10). |
| .BR Ki , |
| .BR K , |
| and |
| .B KB |
| are accepted as synonyms for |
| .BR KiB . |
| .TP |
| .B MiB |
| Multiply the integer by 1,048,576 (2^20). |
| .BR Mi , |
| .BR M , |
| and |
| .B MB |
| are accepted as synonyms for |
| .BR MiB . |
| |
| . |
| .SS "Operation mode" |
| If multiple operation mode options are given, |
| the last one takes effect. |
| .TP |
| .BR \-z ", " \-\-compress |
| Compress. |
| This is the default operation mode when no operation mode option |
| is specified and no other operation mode is implied from |
| the command name (for example, |
| .B unzstd |
| implies |
| .BR \-\-decompress ). |
| .TP |
| .BR \-d ", " \-\-decompress ", " \-\-uncompress |
| Decompress. |
| .TP |
| .BR \-t ", " \-\-test |
| Test the integrity of compressed |
| .IR files . |
| This option is equivalent to |
| .B "\-\-decompress \-\-stdout" |
| except that the decompressed data is discarded instead of being |
| written to standard output. |
| No files are created or removed. |
| .TP |
| .B \-b# |
| benchmark file(s) using compression level # |
| .TP |
| .B \--train FILEs |
| use FILEs as training set to create a dictionary. The training set should contain a lot of small files (> 100). |
| |
| . |
| .SS "Operation modifiers" |
| .TP |
| .B \-# |
| # compression level [1-19] (default:3) |
| .TP |
| .BR \--ultra |
| unlocks high compression levels 20+ (maximum 22), using a lot more memory. |
| Note that decompression will also require more memory when using these levels. |
| .TP |
| .B \-D file |
| use `file` as Dictionary to compress or decompress FILE(s) |
| .TP |
| .BR \--no-dictID |
| do not store dictionary ID within frame header (dictionary compression). |
| The decoder will have to rely on implicit knowledge about which dictionary to use, |
| it won't be able to check if it's correct. |
| .TP |
| .B \-o file |
| save result into `file` (only possible with a single INPUT-FILE) |
| .TP |
| .BR \-f ", " --force |
| overwrite output without prompting |
| .TP |
| .BR \-c ", " --stdout |
| force write to standard output, even if it is the console |
| .TP |
| .BR \--[no-]sparse |
| enable / disable sparse FS support, to make files with many zeroes smaller on disk. |
| Creating sparse files may save disk space and speed up the decompression |
| by reducing the amount of disk I/O. |
| default : enabled when output is into a file, and disabled when output is stdout. |
| This setting overrides default and can force sparse mode over stdout. |
| .TP |
| .BR \--rm |
| remove source file(s) after successful compression or decompression |
| .TP |
| .BR \-k ", " --keep |
| keep source file(s) after successful compression or decompression. |
| This is the default behavior. |
| .TP |
| .BR \-r |
| operate recursively on directories |
| .TP |
| .BR \-h/\-H ", " --help |
| display help/long help and exit |
| .TP |
| .BR \-V ", " --version |
| display Version number and exit |
| .TP |
| .BR \-v ", " --verbose |
| verbose mode |
| .TP |
| .BR \-q ", " --quiet |
| suppress warnings, interactivity and notifications. |
| specify twice to suppress errors too. |
| .TP |
| .BR \-C ", " --[no-]check |
| add integrity check computed from uncompressed data (default : enabled) |
| .TP |
| .BR \-t ", " --test |
| Test the integrity of compressed files. This option is equivalent to \fB--decompress --stdout > /dev/null\fR. |
| No files are created or removed. |
| .TP |
| .BR -- |
| All arguments after -- are treated as files |
| |
| |
| .SH DICTIONARY BUILDER |
| .PP |
| \fBzstd\fR offers \fIdictionary\fR compression, useful for very small files and messages. |
| It's possible to train \fBzstd\fR with some samples, the result of which is saved into a file called `dictionary`. |
| Then during compression and decompression, make reference to the same dictionary. |
| It will improve compression ratio of small files. |
| Typical gains range from ~10% (at 64KB) to x5 better (at <1KB). |
| .TP |
| .B \--train FILEs |
| use FILEs as training set to create a dictionary. The training set should contain a lot of small files (> 100), |
| and weight typically 100x the target dictionary size (for example, 10 MB for a 100 KB dictionary) |
| .TP |
| .B \-o file |
| dictionary saved into `file` (default: dictionary) |
| .TP |
| .B \--maxdict # |
| limit dictionary to specified size (default : 112640) |
| .TP |
| .B \--dictID # |
| A dictionary ID is a locally unique ID that a decoder can use to verify it is using the right dictionary. |
| By default, zstd will create a 4-bytes random number ID. |
| It's possible to give a precise number instead. |
| Short numbers have an advantage : an ID < 256 will only need 1 byte in the compressed frame header, |
| and an ID < 65536 will only need 2 bytes. This compares favorably to 4 bytes default. |
| However, it's up to the dictionary manager to not assign twice the same ID to 2 different dictionaries. |
| .TP |
| .B \-s# |
| dictionary selectivity level (default: 9) |
| the smaller the value, the denser the dictionary, improving its efficiency but reducing its possible maximum size. |
| .TP |
| .B \--cover=k=#,d=# |
| Use alternate dictionary builder algorithm named cover with parameters \fIk\fR and \fId\fR with \fId\fR <= \fIk\fR. |
| Selects segments of size \fIk\fR with the highest score to put in the dictionary. |
| The score of a segment is computed by the sum of the frequencies of all the subsegments of of size \fId\fR. |
| Generally \fId\fR should be in the range [6, 24]. |
| Good values for \fIk\fR vary widely based on the input data, but a safe range is [32, 2048]. |
| Example: \fB--train --cover=k=64,d=8 FILEs\fR. |
| .TP |
| .B \--optimize-cover[=steps=#,k=#,d=#] |
| If \fIsteps\fR is not specified, the default value of 32 is used. |
| If \fIk\fR is not specified, \fIsteps\fR values in [16, 2048] are checked for each value of \fId\fR. |
| If \fId\fR is not specified, the values checked are [6, 8, ..., 16]. |
| |
| Runs the cover dictionary builder for each parameter set saves the optimal parameters and dictionary. |
| Prints the optimal parameters and writes the optimal dictionary to the output file. |
| Supports multithreading if \fBzstd\fR is compiled with threading support. |
| |
| The parameter \fIk\fR is more sensitve than \fId\fR, and is faster to optimize over. |
| Suggested use is to run with a \fIsteps\fR <= 32 with neither \fIk\fR nor \fId\fR set. |
| Once it completes, use the value of \fId\fR it selects with a higher \fIsteps\fR (in the range [256, 1024]). |
| \fBzstd --train --optimize-cover FILEs |
| \fBzstd --train --optimize-cover=d=d,steps=512 FILEs |
| .TP |
| |
| .SH BENCHMARK |
| .TP |
| .B \-b# |
| benchmark file(s) using compression level # |
| .TP |
| .B \-e# |
| benchmark file(s) using multiple compression levels, from -b# to -e# (included). |
| .TP |
| .B \-i# |
| minimum evaluation time, in seconds (default : 3s), benchmark mode only |
| .TP |
| .B \-B# |
| cut file into independent blocks of size # (default: no block) |
| .B \--priority=rt |
| set process priority to real-time |
| |
| .SH ADVANCED COMPRESSION OPTIONS |
| .TP |
| .B \--zstd[=\fIoptions\fR] |
| .PD |
| \fBzstd\fR provides 22 predefined compression levels. The selected or default predefined compression level can be changed with advanced compression options. |
| The \fIoptions\fR are provided as a comma-separated list. You may specify only the \fIoptions\fR you want to change and the rest will be taken from the selected or default compression level. |
| The list of available \fIoptions\fR: |
| .RS |
| |
| .TP |
| .BI strategy= strat |
| .PD 0 |
| .TP |
| .BI strat= strat |
| .PD |
| Specify a strategy used by a match finder. |
| .IP "" |
| There are 8 strategies numbered from 0 to 7, from faster to stronger: |
| 0=ZSTD_fast, 1=ZSTD_dfast, 2=ZSTD_greedy, 3=ZSTD_lazy, 4=ZSTD_lazy2, 5=ZSTD_btlazy2, 6=ZSTD_btopt, 7=ZSTD_btopt2. |
| .IP "" |
| |
| .TP |
| .BI windowLog= wlog |
| .PD 0 |
| .TP |
| .BI wlog= wlog |
| .PD |
| Specify the maximum number of bits for a match distance. |
| .IP "" |
| The higher number of bits increases the chance to find a match what usually improves compression ratio. |
| It also increases memory requirements for compressor and decompressor. |
| .IP "" |
| The minimum \fIwlog\fR is 10 (1 KiB) and the maximum is 25 (32 MiB) for 32-bit compilation and 27 (128 MiB) for 64-bit compilation. |
| .IP "" |
| |
| .TP |
| .BI hashLog= hlog |
| .PD 0 |
| .TP |
| .BI hlog= hlog |
| .PD |
| Specify the maximum number of bits for a hash table. |
| .IP "" |
| The bigger hash table causes less collisions what usually make compression faster but requires more memory during compression. |
| .IP "" |
| The minimum \fIhlog\fR is 6 (64 B) and the maximum is 25 (32 MiB) for 32-bit compilation and 27 (128 MiB) for 64-bit compilation. |
| |
| .TP |
| .BI chainLog= clog |
| .PD 0 |
| .TP |
| .BI clog= clog |
| .PD |
| Specify the maximum number of bits for a hash chain or a binary tree. |
| .IP "" |
| The higher number of bits increases the chance to find a match what usually improves compression ratio. |
| It also slows down compression speed and increases memory requirements for compression. |
| This option is ignored for the ZSTD_fast strategy. |
| .IP "" |
| The minimum \fIclog\fR is 6 (64 B) and the maximum is 26 (64 MiB) for 32-bit compilation and 28 (256 MiB) for 64-bit compilation. |
| .IP "" |
| |
| .TP |
| .BI searchLog= slog |
| .PD 0 |
| .TP |
| .BI slog= slog |
| .PD |
| Specify the maximum number of searches in a hash chain or a binary tree using logarithmic scale. |
| .IP "" |
| The bigger number of searches increases the chance to find a match what usually improves compression ratio but decreases compression speed. |
| .IP "" |
| The minimum \fIslog\fR is 1 and the maximum is 24 for 32-bit compilation and 26 for 64-bit compilation. |
| .IP "" |
| |
| .TP |
| .BI searchLength= slen |
| .PD 0 |
| .TP |
| .BI slen= slen |
| .PD |
| Specify the minimum searched length of a match in a hash table. |
| .IP "" |
| The bigger search length usually decreases compression ratio but improves decompression speed. |
| .IP "" |
| The minimum \fIslen\fR is 3 and the maximum is 7. |
| .IP "" |
| |
| .TP |
| .BI targetLength= tlen |
| .PD 0 |
| .TP |
| .BI tlen= tlen |
| .PD |
| Specify the minimum match length that causes a match finder to interrupt searching of better matches. |
| .IP "" |
| The bigger minimum match length usually improves compression ratio but decreases compression speed. |
| This option is used only with ZSTD_btopt and ZSTD_btopt2 strategies. |
| .IP "" |
| The minimum \fItlen\fR is 4 and the maximum is 999. |
| .IP "" |
| |
| .PP |
| .B An example |
| .br |
| The following parameters sets advanced compression options to predefined level 19 for files bigger than 256 KB: |
| .IP "" |
| \fB--zstd=\fRwindowLog=23,chainLog=23,hashLog=22,searchLog=6,searchLength=3,targetLength=48,strategy=6 |
| |
| .SH BUGS |
| Report bugs at:- https://github.com/facebook/zstd/issues |
| |
| .SH AUTHOR |
| Yann Collet |