blob: 6ac12ff0f5e03a0f69a09e56ef4c94b4afbd7d97 [file] [log] [blame]
Kostya Serebryany79677382015-03-31 21:39:38 +00001========================================================
Kostya Serebryany35ce8632015-03-30 23:05:30 +00002LibFuzzer -- a library for coverage-guided fuzz testing.
3========================================================
Kostya Serebryany79677382015-03-31 21:39:38 +00004.. contents::
5 :local:
Kostya Serebryanyd11dc172016-03-12 02:56:25 +00006 :depth: 1
Kostya Serebryany79677382015-03-31 21:39:38 +00007
8Introduction
9============
Kostya Serebryany35ce8632015-03-30 23:05:30 +000010
Kostya Serebryanyd11dc172016-03-12 02:56:25 +000011libFuzzer -- library for in-process evolutionary fuzzing of other libraries.
Kostya Serebryany35ce8632015-03-30 23:05:30 +000012
Kostya Serebryanyd11dc172016-03-12 02:56:25 +000013The typical workflow looks like the following.
14First, implement a fuzzing target function, like this::
15
16 // fuzz_target.cc
17 extern "C" int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size) {
18 DoSomethingInterestingWithMyAPI(Data, Size);
Kostya Serebryany9e1a2382016-03-29 23:07:36 +000019 return 0; // Non-zero return values are reserved for future use.
Kostya Serebryanyd11dc172016-03-12 02:56:25 +000020 }
21
22Next, build the Fuzzer library as a static archive. Note that libFuzzer contains the `main()` function::
23
24 svn co http://llvm.org/svn/llvm-project/llvm/trunk/lib/Fuzzer
25 clang++ -c -g -O2 -std=c++11 Fuzzer/*.cpp -IFuzzer
26 ar ruv libFuzzer.a Fuzzer*.o
27
28Then build the target function and the library you are going to test.
29You should use SanitizerCoverage_ and one of ASan, MSan, or UBSan.
30Link it with `libFuzzer.a`::
31
32 clang -fsanitize-coverage=edge -fsanitize=address your_lib.cc fuzz_target.cc libFuzzer.a -o my_fuzzer
33
34Create a directory with the initial "seed" samlpes.
35For some input types libFuzzer will work just fine w/o any seeds,
36but for complex inputs this step is very important::
37
38 mkdir CORPUS_DIR
39 cp /some/input/samples/* CORPUS_DIR
40
41Finally, run the fuzzer on the `CORPUS_DIR`::
42
43 ./my_fuzzer CORPUS_DIR # -max_len=1000 -jobs=20 -more_lags=...
Kostya Serebryany35ce8632015-03-30 23:05:30 +000044
45
Kostya Serebryanyd11dc172016-03-12 02:56:25 +000046As new interesting test cases are discovered they will be added to the corpus.
47If a bug is discovered by the sanitizer (ASan, etc) it will be reported as usual and the reproducer
48will be written to disk.
49Each Fuzzer process is single-threaded (unless the library starts its own
50threads). You can run the libFuzzer on the same corpus in multiple processes
51in parallel (use the flags `-jobs=N` and `-workers=N`).
52
53libFuzzer is similar in concept to AFL_,
54but uses in-process Fuzzing, which is more fragile and restrictive, but
Kostya Serebryany35ce8632015-03-30 23:05:30 +000055potentially much faster as it has no overhead for process start-up.
Kostya Serebryany79677382015-03-31 21:39:38 +000056It uses LLVM's SanitizerCoverage_ instrumentation to get in-process
57coverage-feedback
Kostya Serebryany35ce8632015-03-30 23:05:30 +000058
Kostya Serebryany9e1a2382016-03-29 23:07:36 +000059The code resides in the LLVM repository,
60requires the fresh Clang compiler to build
Kostya Serebryany79677382015-03-31 21:39:38 +000061and is used to fuzz various parts of LLVM,
62but the Fuzzer itself does not (and should not) depend on any
63part of LLVM and can be used for other projects w/o requiring the rest of LLVM.
Kostya Serebryany35ce8632015-03-30 23:05:30 +000064
Kostya Serebryany9e1a2382016-03-29 23:07:36 +000065Fresh Clang
66-----------
67
68If you don't know where to get the fresh Clang binaries and don't want to build
69it from trunk (why wouldn't you?) you may grab the fresh Clang binaries
70maintained by the Chromium developers::
71
72 mkdir TMP_CLANG
73 cd TMP_CLANG
74 git clone https://chromium.googlesource.com/chromium/src/tools/clang
75 cd ..
76 TMP_CLANG/clang/scripts/update.py
77
78This will install a reasonably fresh and well tested clang binaries as
79`third_party/llvm-build/Release+Asserts/bin/clang`
80
Kostya Serebryanyd11dc172016-03-12 02:56:25 +000081Usage
82=====
83To run fuzzing pass 0 or more directories. New samples will be written into `dir1`, other directories will be read once during startup.::
Kostya Serebryanybfbe7fc2016-02-02 03:03:47 +000084
85./fuzzer [-flag1=val1 [-flag2=val2 ...] ] [dir1 [dir2 ...] ]
86
87To run individual tests without fuzzing pass 1 or more files::
88
89./fuzzer [-flag1=val1 [-flag2=val2 ...] ] file1 [file2 ...]
90
Kostya Serebryany2adfa3b2015-05-20 21:03:03 +000091The most important flags are::
92
93 seed 0 Random seed. If 0, seed is generated.
94 runs -1 Number of individual test runs (-1 for infinite runs).
Kostya Serebryany64d24572016-03-12 01:57:04 +000095 max_len 0 Maximum length of the test input. If 0, libFuzzer tries to guess a good value based on the corpus and reports it.
Kostya Serebryany316b5712015-05-26 20:57:47 +000096 timeout 1200 Timeout in seconds (if positive). If one unit runs more than this number of seconds the process will abort.
Kostya Serebryany54a63632016-01-29 23:30:07 +000097 timeout_exitcode 77 Unless abort_on_timeout is set, use this exitcode on timeout.
Kostya Serebryanyb85db172015-10-02 20:47:55 +000098 max_total_time 0 If positive, indicates the maximal total time in seconds to run the fuzzer.
Kostya Serebryany2adfa3b2015-05-20 21:03:03 +000099 help 0 Print help.
Kostya Serebryany9cc3b0d2015-10-24 01:16:40 +0000100 merge 0 If 1, the 2-nd, 3-rd, etc corpora will be merged into the 1-st corpus. Only interesting units will be taken.
Kostya Serebryany2adfa3b2015-05-20 21:03:03 +0000101 jobs 0 Number of jobs to run. If jobs >= 1 we spawn this number of jobs in separate worker processes with stdout/stderr redirected to fuzz-JOB.log.
102 workers 0 Number of simultaneous worker processes to run the jobs. If zero, "min(jobs,NumberOfCpuCores()/2)" is used.
Kostya Serebryanyb17e2982015-07-31 21:48:10 +0000103 use_traces 0 Experimental: use instruction traces
Kostya Serebryanybc7c0ad2015-08-11 01:44:42 +0000104 only_ascii 0 If 1, generate only ASCII (isprint+isspace) inputs.
Kostya Serebryanybd5d1cd2015-10-09 03:57:59 +0000105 artifact_prefix "" Write fuzzing artifacts (crash, timeout, or slow inputs) as $(artifact_prefix)file
Kostya Serebryany2d0ef142015-11-25 21:40:46 +0000106 exact_artifact_path "" Write the single artifact on failure (crash, timeout) as $(exact_artifact_path). This overrides -artifact_prefix and will not use checksum in the file name. Do not use the same path for several parallel processes.
Kostya Serebryany3c767db2016-02-27 05:45:12 +0000107 print_final_stats 0 If 1, print statistics at exit.
Kostya Serebryany9e1a2382016-03-29 23:07:36 +0000108 close_fd_mask 0 If 1, close stdout at startup; if 2, close stderr; if 3, close both. Be careful, this will also close e.g. asan's stderr/stdout.
Kostya Serebryany2adfa3b2015-05-20 21:03:03 +0000109
110For the full list of flags run the fuzzer binary with ``-help=1``.
111
Kostya Serebryany79677382015-03-31 21:39:38 +0000112Usage examples
113==============
Kostya Serebryanyd11dc172016-03-12 02:56:25 +0000114.. contents::
115 :local:
116 :depth: 1
Kostya Serebryany79677382015-03-31 21:39:38 +0000117
118Toy example
119-----------
120
121A simple function that does something interesting if it receives the input "HI!"::
122
123 cat << EOF >> test_fuzzer.cc
Kostya Serebryany1c80b9d2015-11-26 00:12:57 +0000124 #include <stdint.h>
125 #include <stddef.h>
126 extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) {
Kostya Serebryany79677382015-03-31 21:39:38 +0000127 if (size > 0 && data[0] == 'H')
128 if (size > 1 && data[1] == 'I')
129 if (size > 2 && data[2] == '!')
130 __builtin_trap();
Kostya Serebryany20bb5e72015-10-02 23:34:06 +0000131 return 0;
Kostya Serebryany79677382015-03-31 21:39:38 +0000132 }
133 EOF
Kostya Serebryanyabca88e2016-03-12 03:05:37 +0000134 # Build test_fuzzer.cc with asan and link against libFuzzer.a
135 clang++ -fsanitize=address -fsanitize-coverage=edge test_fuzzer.cc libFuzzer.a
Kostya Serebryany79677382015-03-31 21:39:38 +0000136 # Run the fuzzer with no corpus.
137 ./a.out
138
Kostya Serebryanyabca88e2016-03-12 03:05:37 +0000139You should get an error pretty quickly::
140
141 #0 READ units: 1 exec/s: 0
142 #1 INITED cov: 3 units: 1 exec/s: 0
143 #2 NEW cov: 5 units: 2 exec/s: 0 L: 64 MS: 0
144 #19237 NEW cov: 9 units: 3 exec/s: 0 L: 64 MS: 0
145 #20595 NEW cov: 10 units: 4 exec/s: 0 L: 1 MS: 4 ChangeASCIIInt-ShuffleBytes-ChangeByte-CrossOver-
146 #34574 NEW cov: 13 units: 5 exec/s: 0 L: 2 MS: 3 ShuffleBytes-CrossOver-ChangeBit-
147 #34807 NEW cov: 15 units: 6 exec/s: 0 L: 3 MS: 1 CrossOver-
148 ==31511== ERROR: libFuzzer: deadly signal
149 ...
150 artifact_prefix='./'; Test unit written to ./crash-b13e8756b13a00cf168300179061fb4b91fefbed
151
Kostya Serebryany79677382015-03-31 21:39:38 +0000152
153PCRE2
154-----
155
Kostya Serebryanyabca88e2016-03-12 03:05:37 +0000156Here we show how to use libFuzzer on something real, yet simple: pcre2_::
Kostya Serebryany79677382015-03-31 21:39:38 +0000157
Alexey Samsonov21a33812015-05-07 23:33:24 +0000158 COV_FLAGS=" -fsanitize-coverage=edge,indirect-calls,8bit-counters"
Kostya Serebryany79677382015-03-31 21:39:38 +0000159 # Get PCRE2
160 svn co svn://vcs.exim.org/pcre2/code/trunk pcre
Kostya Serebryany79677382015-03-31 21:39:38 +0000161 # Build PCRE2 with AddressSanitizer and coverage.
162 (cd pcre; ./autogen.sh; CC="clang -fsanitize=address $COV_FLAGS" ./configure --prefix=`pwd`/../inst && make -j && make install)
Kostya Serebryanyabca88e2016-03-12 03:05:37 +0000163 # Build the fuzzing target function that does something interesting with PCRE2.
Kostya Serebryany79677382015-03-31 21:39:38 +0000164 cat << EOF > pcre_fuzzer.cc
165 #include <string.h>
Kostya Serebryany1c80b9d2015-11-26 00:12:57 +0000166 #include <stdint.h>
Kostya Serebryany79677382015-03-31 21:39:38 +0000167 #include "pcre2posix.h"
Kostya Serebryany1c80b9d2015-11-26 00:12:57 +0000168 extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) {
Kostya Serebryany20bb5e72015-10-02 23:34:06 +0000169 if (size < 1) return 0;
Kostya Serebryany79677382015-03-31 21:39:38 +0000170 char *str = new char[size+1];
171 memcpy(str, data, size);
172 str[size] = 0;
173 regex_t preg;
174 if (0 == regcomp(&preg, str, 0)) {
175 regexec(&preg, str, 0, 0, 0);
176 regfree(&preg);
177 }
178 delete [] str;
Kostya Serebryany20bb5e72015-10-02 23:34:06 +0000179 return 0;
Kostya Serebryany79677382015-03-31 21:39:38 +0000180 }
181 EOF
182 clang++ -g -fsanitize=address $COV_FLAGS -c -std=c++11 -I inst/include/ pcre_fuzzer.cc
183 # Link.
Kostya Serebryanyabca88e2016-03-12 03:05:37 +0000184 clang++ -g -fsanitize=address -Wl,--whole-archive inst/lib/*.a -Wl,-no-whole-archive libFuzzer.a pcre_fuzzer.o -o pcre_fuzzer
Kostya Serebryany79677382015-03-31 21:39:38 +0000185
186This will give you a binary of the fuzzer, called ``pcre_fuzzer``.
187Now, create a directory that will hold the test corpus::
188
189 mkdir -p CORPUS
190
191For simple input languages like regular expressions this is all you need.
192For more complicated inputs populate the directory with some input samples.
193Now run the fuzzer with the corpus dir as the only parameter::
194
195 ./pcre_fuzzer ./CORPUS
196
197You will see output like this::
198
199 Seed: 1876794929
200 #0 READ cov 0 bits 0 units 1 exec/s 0
201 #1 pulse cov 3 bits 0 units 1 exec/s 0
202 #1 INITED cov 3 bits 0 units 1 exec/s 0
203 #2 pulse cov 208 bits 0 units 1 exec/s 0
204 #2 NEW cov 208 bits 0 units 2 exec/s 0 L: 64
205 #3 NEW cov 217 bits 0 units 3 exec/s 0 L: 63
206 #4 pulse cov 217 bits 0 units 3 exec/s 0
207
208* The ``Seed:`` line shows you the current random seed (you can change it with ``-seed=N`` flag).
209* The ``READ`` line shows you how many input files were read (since you passed an empty dir there were inputs, but one dummy input was synthesised).
210* The ``INITED`` line shows you that how many inputs will be fuzzed.
211* The ``NEW`` lines appear with the fuzzer finds a new interesting input, which is saved to the CORPUS dir. If multiple corpus dirs are given, the first one is used.
212* The ``pulse`` lines appear periodically to show the current status.
213
214Now, interrupt the fuzzer and run it again the same way. You will see::
215
216 Seed: 1879995378
217 #0 READ cov 0 bits 0 units 564 exec/s 0
218 #1 pulse cov 502 bits 0 units 564 exec/s 0
219 ...
220 #512 pulse cov 2933 bits 0 units 564 exec/s 512
221 #564 INITED cov 2991 bits 0 units 344 exec/s 564
222 #1024 pulse cov 2991 bits 0 units 344 exec/s 1024
223 #1455 NEW cov 2995 bits 0 units 345 exec/s 1455 L: 49
224
225This time you were running the fuzzer with a non-empty input corpus (564 items).
226As the first step, the fuzzer minimized the set to produce 344 interesting items (the ``INITED`` line)
227
228You may run ``N`` independent fuzzer jobs in parallel on ``M`` CPUs::
229
230 N=100; M=4; ./pcre_fuzzer ./CORPUS -jobs=$N -workers=$M
231
Kostya Serebryany9690fcf2015-05-12 18:51:57 +0000232By default (``-reload=1``) the fuzzer processes will periodically scan the CORPUS directory
233and reload any new tests. This way the test inputs found by one process will be picked up
234by all others.
Kostya Serebryany79677382015-03-31 21:39:38 +0000235
Kostya Serebryany9690fcf2015-05-12 18:51:57 +0000236If ``-workers=$M`` is not supplied, ``min($N,NumberOfCpuCore/2)`` will be used.
Kostya Serebryany79677382015-03-31 21:39:38 +0000237
Kostya Serebryany5e593a42015-04-08 06:16:11 +0000238Heartbleed
239----------
240Remember Heartbleed_?
241As it was recently `shown <https://blog.hboeck.de/archives/868-How-Heartbleed-couldve-been-found.html>`_,
242fuzzing with AddressSanitizer can find Heartbleed. Indeed, here are the step-by-step instructions
243to find Heartbleed with LibFuzzer::
244
245 wget https://www.openssl.org/source/openssl-1.0.1f.tar.gz
246 tar xf openssl-1.0.1f.tar.gz
Alexey Samsonov21a33812015-05-07 23:33:24 +0000247 COV_FLAGS="-fsanitize-coverage=edge,indirect-calls" # -fsanitize-coverage=8bit-counters
Kostya Serebryany5e593a42015-04-08 06:16:11 +0000248 (cd openssl-1.0.1f/ && ./config &&
249 make -j 32 CC="clang -g -fsanitize=address $COV_FLAGS")
250 # Get and build LibFuzzer
251 svn co http://llvm.org/svn/llvm-project/llvm/trunk/lib/Fuzzer
252 clang -c -g -O2 -std=c++11 Fuzzer/*.cpp -IFuzzer
253 # Get examples of key/pem files.
254 git clone https://github.com/hannob/selftls
255 cp selftls/server* . -v
256 cat << EOF > handshake-fuzz.cc
257 #include <openssl/ssl.h>
258 #include <openssl/err.h>
259 #include <assert.h>
Kostya Serebryany1c80b9d2015-11-26 00:12:57 +0000260 #include <stdint.h>
261 #include <stddef.h>
262
Kostya Serebryany5e593a42015-04-08 06:16:11 +0000263 SSL_CTX *sctx;
264 int Init() {
265 SSL_library_init();
266 SSL_load_error_strings();
267 ERR_load_BIO_strings();
268 OpenSSL_add_all_algorithms();
269 assert (sctx = SSL_CTX_new(TLSv1_method()));
270 assert (SSL_CTX_use_certificate_file(sctx, "server.pem", SSL_FILETYPE_PEM));
271 assert (SSL_CTX_use_PrivateKey_file(sctx, "server.key", SSL_FILETYPE_PEM));
272 return 0;
273 }
Kostya Serebryany1c80b9d2015-11-26 00:12:57 +0000274 extern "C" int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size) {
Kostya Serebryany5e593a42015-04-08 06:16:11 +0000275 static int unused = Init();
276 SSL *server = SSL_new(sctx);
277 BIO *sinbio = BIO_new(BIO_s_mem());
278 BIO *soutbio = BIO_new(BIO_s_mem());
279 SSL_set_bio(server, sinbio, soutbio);
280 SSL_set_accept_state(server);
281 BIO_write(sinbio, Data, Size);
282 SSL_do_handshake(server);
283 SSL_free(server);
Kostya Serebryany20bb5e72015-10-02 23:34:06 +0000284 return 0;
Kostya Serebryany5e593a42015-04-08 06:16:11 +0000285 }
286 EOF
Mehdi Amini30618f92015-09-17 15:59:52 +0000287 # Build the fuzzer.
Kostya Serebryany5e593a42015-04-08 06:16:11 +0000288 clang++ -g handshake-fuzz.cc -fsanitize=address \
289 openssl-1.0.1f/libssl.a openssl-1.0.1f/libcrypto.a Fuzzer*.o
290 # Run 20 independent fuzzer jobs.
291 ./a.out -jobs=20 -workers=20
292
293Voila::
294
295 #1048576 pulse cov 3424 bits 0 units 9 exec/s 24385
296 =================================================================
297 ==17488==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x629000004748 at pc 0x00000048c979 bp 0x7fffe3e864f0 sp 0x7fffe3e85ca8
298 READ of size 60731 at 0x629000004748 thread T0
299 #0 0x48c978 in __asan_memcpy
300 #1 0x4db504 in tls1_process_heartbeat openssl-1.0.1f/ssl/t1_lib.c:2586:3
301 #2 0x580be3 in ssl3_read_bytes openssl-1.0.1f/ssl/s3_pkt.c:1092:4
302
Kostya Serebryany1c80b9d2015-11-26 00:12:57 +0000303Note: a `similar fuzzer <https://boringssl.googlesource.com/boringssl/+/HEAD/FUZZING.md>`_
304is now a part of the boringssl source tree.
305
Kostya Serebryany043ab1c2015-04-01 21:33:20 +0000306Advanced features
307=================
Kostya Serebryanyd11dc172016-03-12 02:56:25 +0000308.. contents::
309 :local:
310 :depth: 1
Kostya Serebryany043ab1c2015-04-01 21:33:20 +0000311
Kostya Serebryany7d211662015-09-04 00:12:11 +0000312Dictionaries
313------------
Kostya Serebryany7d211662015-09-04 00:12:11 +0000314LibFuzzer supports user-supplied dictionaries with input language keywords
315or other interesting byte sequences (e.g. multi-byte magic values).
316Use ``-dict=DICTIONARY_FILE``. For some input languages using a dictionary
317may significantly improve the search speed.
318The dictionary syntax is similar to that used by AFL_ for its ``-x`` option::
319
320 # Lines starting with '#' and empty lines are ignored.
321
322 # Adds "blah" (w/o quotes) to the dictionary.
323 kw1="blah"
324 # Use \\ for backslash and \" for quotes.
325 kw2="\"ac\\dc\""
326 # Use \xAB for hex values
327 kw3="\xF7\xF8"
328 # the name of the keyword followed by '=' may be omitted:
329 "foo\x0Abar"
330
Kostya Serebryanyb17e2982015-07-31 21:48:10 +0000331Data-flow-guided fuzzing
332------------------------
333
334*EXPERIMENTAL*.
335With an additional compiler flag ``-fsanitize-coverage=trace-cmp`` (see SanitizerCoverageTraceDataFlow_)
336and extra run-time flag ``-use_traces=1`` the fuzzer will try to apply *data-flow-guided fuzzing*.
337That is, the fuzzer will record the inputs to comparison instructions, switch statements,
Kostya Serebryany7f4227d2015-08-05 18:23:01 +0000338and several libc functions (``memcmp``, ``strcmp``, ``strncmp``, etc).
Kostya Serebryanyb17e2982015-07-31 21:48:10 +0000339It will later use those recorded inputs during mutations.
340
341This mode can be combined with DataFlowSanitizer_ to achieve better sensitivity.
342
Kostya Serebryany6bd016b2015-04-10 05:44:43 +0000343AFL compatibility
344-----------------
Kostya Serebryany9e1a2382016-03-29 23:07:36 +0000345LibFuzzer can be used together with AFL_ on the same test corpus.
Kostya Serebryany6bd016b2015-04-10 05:44:43 +0000346Both fuzzers expect the test corpus to reside in a directory, one file per input.
Kostya Serebryany9e1a2382016-03-29 23:07:36 +0000347You can run both fuzzers on the same corpus, one after another::
Kostya Serebryany6bd016b2015-04-10 05:44:43 +0000348
Kostya Serebryany9e1a2382016-03-29 23:07:36 +0000349 ./afl-fuzz -i testcase_dir -o findings_dir /path/to/program @@
Kostya Serebryany6bd016b2015-04-10 05:44:43 +0000350 ./llvm-fuzz testcase_dir findings_dir # Will write new tests to testcase_dir
351
352Periodically restart both fuzzers so that they can use each other's findings.
Kostya Serebryany9e1a2382016-03-29 23:07:36 +0000353Currently, there is no simple way to run both fuzzing engines in parallel while sharing the same corpus dir.
Kostya Serebryany79677382015-03-31 21:39:38 +0000354
Kostya Serebryanycd073d52015-04-10 06:32:29 +0000355How good is my fuzzer?
356----------------------
357
Kostya Serebryany566bc5a2015-05-06 22:19:00 +0000358Once you implement your target function ``LLVMFuzzerTestOneInput`` and fuzz it to death,
Kostya Serebryanycd073d52015-04-10 06:32:29 +0000359you will want to know whether the function or the corpus can be improved further.
360One easy to use metric is, of course, code coverage.
361You can get the coverage for your corpus like this::
362
Kostya Serebryany7ead9262016-03-12 03:11:27 +0000363 ASAN_OPTIONS=coverage=1 ./fuzzer CORPUS_DIR -runs=0
Kostya Serebryanycd073d52015-04-10 06:32:29 +0000364
365This will run all the tests in the CORPUS_DIR but will not generate any new tests
366and dump covered PCs to disk before exiting.
367Then you can subtract the set of covered PCs from the set of all instrumented PCs in the binary,
368see SanitizerCoverage_ for details.
369
Kostya Serebryany926b9bd2015-05-22 22:43:05 +0000370User-supplied mutators
371----------------------
372
373LibFuzzer allows to use custom (user-supplied) mutators,
374see FuzzerInterface.h_
375
Kostya Serebryanyaca76962016-01-16 01:23:12 +0000376Startup initialization
377----------------------
378If the library being tested needs to be initialized, there are several options.
379
380The simplest way is to have a statically initialized global object::
381
382 static bool Initialized = DoInitialization();
383
384Alternatively, you may define an optional init function and it will receive
385the program arguments that you can read and modify::
386
387 extern "C" int LLVMFuzzerInitialize(int *argc, char ***argv) {
388 ReadAndMaybeModify(argc, argv);
389 return 0;
390 }
391
Kostya Serebryanyaca76962016-01-16 01:23:12 +0000392Try to avoid initialization inside the target function itself as
393it will skew the coverage data. Don't do this::
394
395 extern "C" int LLVMFuzzerTestOneInput(...) {
396 static bool initialized = false;
397 if (!initialized) {
398 ...
399 }
400 }
401
Kostya Serebryany9e1a2382016-03-29 23:07:36 +0000402Leaks
403-----
404
405When running libFuzzer with AddressSanitizer_ the latter will be able to report
406memory leaks, but only when the process exits, so if you suspect memory leaks
407in your target you should run libFuzzer with `-runs=N` or `-max_total_time=N`.
408If a leak is reported at the end, you will not get the reproducer from libFuzzer.
409You will need to re-run the target on every file in the corpus separately to
410find which one causes the leak.
411
412If your target has massive leaks you will eventually run out of RAM.
413To protect your machine from OOM death you may use
414e.g. `ASAN_OPTIONS=hard_rss_limit_mb=2000` (with AddressSanitizer_).
415
416In future libFuzzer may support finding/reporting leaks better than this, stay tuned.
417
Kostya Serebryany79677382015-03-31 21:39:38 +0000418Fuzzing components of LLVM
419==========================
Kostya Serebryanyd11dc172016-03-12 02:56:25 +0000420.. contents::
421 :local:
422 :depth: 1
Kostya Serebryany35ce8632015-03-30 23:05:30 +0000423
424clang-format-fuzzer
425-------------------
426The inputs are random pieces of C++-like text.
427
428Build (make sure to use fresh clang as the host compiler)::
429
430 cmake -GNinja -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DLLVM_USE_SANITIZER=Address -DLLVM_USE_SANITIZE_COVERAGE=YES -DCMAKE_BUILD_TYPE=Release /path/to/llvm
431 ninja clang-format-fuzzer
432 mkdir CORPUS_DIR
433 ./bin/clang-format-fuzzer CORPUS_DIR
434
435Optionally build other kinds of binaries (asan+Debug, msan, ubsan, etc).
436
Kostya Serebryany79677382015-03-31 21:39:38 +0000437Tracking bug: https://llvm.org/bugs/show_bug.cgi?id=23052
Kostya Serebryany35ce8632015-03-30 23:05:30 +0000438
Kostya Serebryany79677382015-03-31 21:39:38 +0000439clang-fuzzer
440------------
Kostya Serebryany35ce8632015-03-30 23:05:30 +0000441
Kostya Serebryany866e0d12015-09-02 22:44:46 +0000442The behavior is very similar to ``clang-format-fuzzer``.
Kostya Serebryany79677382015-03-31 21:39:38 +0000443
444Tracking bug: https://llvm.org/bugs/show_bug.cgi?id=23057
Kostya Serebryany35ce8632015-03-30 23:05:30 +0000445
Kostya Serebryanyb98e3272015-08-31 18:57:24 +0000446llvm-as-fuzzer
447--------------
448
449Tracking bug: https://llvm.org/bugs/show_bug.cgi?id=24639
450
Daniel Sanders5151b202015-09-18 10:47:45 +0000451llvm-mc-fuzzer
452--------------
453
454This tool fuzzes the MC layer. Currently it is only able to fuzz the
455disassembler but it is hoped that assembly, and round-trip verification will be
456added in future.
457
458When run in dissassembly mode, the inputs are opcodes to be disassembled. The
459fuzzer will consume as many instructions as possible and will stop when it
460finds an invalid instruction or runs out of data.
461
Daniel Sanders4fe1c8b2015-09-26 17:09:01 +0000462Please note that the command line interface differs slightly from that of other
463fuzzers. The fuzzer arguments should follow ``--fuzzer-args`` and should have
464a single dash, while other arguments control the operation mode and target in a
465similar manner to ``llvm-mc`` and should have two dashes. For example::
Daniel Sanders5151b202015-09-18 10:47:45 +0000466
Daniel Sanders4fe1c8b2015-09-26 17:09:01 +0000467 llvm-mc-fuzzer --triple=aarch64-linux-gnu --disassemble --fuzzer-args -max_len=4 -jobs=10
Daniel Sanders5151b202015-09-18 10:47:45 +0000468
Kostya Serebryanyfb2f3312015-05-13 22:42:28 +0000469Buildbot
470--------
471
472We have a buildbot that runs the above fuzzers for LLVM components
47324/7/365 at http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-fuzzer .
474
Kostya Serebryany35ce8632015-03-30 23:05:30 +0000475FAQ
476=========================
477
Kostya Serebryany241fb612016-03-12 03:23:02 +0000478Q. Why libFuzzer does not use any of the LLVM support?
479------------------------------------------------------
Kostya Serebryany35ce8632015-03-30 23:05:30 +0000480
481There are two reasons.
482
483First, we want this library to be used outside of the LLVM w/o users having to
484build the rest of LLVM. This may sound unconvincing for many LLVM folks,
485but in practice the need for building the whole LLVM frightens many potential
486users -- and we want more users to use this code.
487
488Second, there is a subtle technical reason not to rely on the rest of LLVM, or
489any other large body of code (maybe not even STL). When coverage instrumentation
490is enabled, it will also instrument the LLVM support code which will blow up the
491coverage set of the process (since the fuzzer is in-process). In other words, by
492using more external dependencies we will slow down the fuzzer while the main
493reason for it to exist is extreme speed.
494
495Q. What about Windows then? The Fuzzer contains code that does not build on Windows.
496------------------------------------------------------------------------------------
497
Kostya Serebryany241fb612016-03-12 03:23:02 +0000498Volunteers are welcome.
Kostya Serebryany35ce8632015-03-30 23:05:30 +0000499
500Q. When this Fuzzer is not a good solution for a problem?
501---------------------------------------------------------
502
503* If the test inputs are validated by the target library and the validator
Kostya Serebryany241fb612016-03-12 03:23:02 +0000504 asserts/crashes on invalid inputs, in-process fuzzing is not applicable.
Kostya Serebryany35ce8632015-03-30 23:05:30 +0000505* Bugs in the target library may accumulate w/o being detected. E.g. a memory
506 corruption that goes undetected at first and then leads to a crash while
507 testing another input. This is why it is highly recommended to run this
508 in-process fuzzer with all sanitizers to detect most bugs on the spot.
509* It is harder to protect the in-process fuzzer from excessive memory
510 consumption and infinite loops in the target library (still possible).
511* The target library should not have significant global state that is not
512 reset between the runs.
513* Many interesting target libs are not designed in a way that supports
514 the in-process fuzzer interface (e.g. require a file path instead of a
515 byte array).
516* If a single test run takes a considerable fraction of a second (or
517 more) the speed benefit from the in-process fuzzer is negligible.
518* If the target library runs persistent threads (that outlive
519 execution of one test) the fuzzing results will be unreliable.
520
521Q. So, what exactly this Fuzzer is good for?
522--------------------------------------------
523
524This Fuzzer might be a good choice for testing libraries that have relatively
Kostya Serebryany241fb612016-03-12 03:23:02 +0000525small inputs, each input takes < 10ms to run, and the library code is not expected
Kostya Serebryany35ce8632015-03-30 23:05:30 +0000526to crash on invalid inputs.
Kostya Serebryany241fb612016-03-12 03:23:02 +0000527Examples: regular expression matchers, text or binary format parsers, compression,
528network, crypto.
Kostya Serebryany35ce8632015-03-30 23:05:30 +0000529
Kostya Serebryanyfab4fba2015-08-11 01:53:45 +0000530Trophies
531========
532* GLIBC: https://sourceware.org/glibc/wiki/FuzzingLibc
Kostya Serebryanyfdf44182015-08-11 04:16:37 +0000533
Kostya Serebryanyfab4fba2015-08-11 01:53:45 +0000534* MUSL LIBC:
Kostya Serebryanyfdf44182015-08-11 04:16:37 +0000535
536 * http://git.musl-libc.org/cgit/musl/commit/?id=39dfd58417ef642307d90306e1c7e50aaec5a35c
537 * http://www.openwall.com/lists/oss-security/2015/03/30/3
538
Kostya Serebryany928eb332015-10-12 18:15:42 +0000539* `pugixml <https://github.com/zeux/pugixml/issues/39>`_
Kostya Serebryanyfdf44182015-08-11 04:16:37 +0000540
Kostya Serebryany45dac2a2015-10-10 02:14:18 +0000541* PCRE: Search for "LLVM fuzzer" in http://vcs.pcre.org/pcre2/code/trunk/ChangeLog?view=markup;
Kostya Serebryany928eb332015-10-12 18:15:42 +0000542 also in `bugzilla <https://bugs.exim.org/buglist.cgi?bug_status=__all__&content=libfuzzer&no_redirect=1&order=Importance&product=PCRE&query_format=specific>`_
Kostya Serebryanyfdf44182015-08-11 04:16:37 +0000543
Kostya Serebryany928eb332015-10-12 18:15:42 +0000544* `ICU <http://bugs.icu-project.org/trac/ticket/11838>`_
Kostya Serebryanyed483772015-08-11 20:34:48 +0000545
Kostya Serebryany928eb332015-10-12 18:15:42 +0000546* `Freetype <https://savannah.nongnu.org/search/?words=LibFuzzer&type_of_search=bugs&Search=Search&exact=1#options>`_
Kostya Serebryany62921282015-09-11 16:34:14 +0000547
Kostya Serebryany928eb332015-10-12 18:15:42 +0000548* `Harfbuzz <https://github.com/behdad/harfbuzz/issues/139>`_
549
Kostya Serebryany240a1592015-11-11 05:25:24 +0000550* `SQLite <http://www3.sqlite.org/cgi/src/info/088009efdd56160b>`_
Kostya Serebryany65e71262015-11-11 05:20:55 +0000551
Kostya Serebryany12fa3b52015-11-13 02:44:16 +0000552* `Python <http://bugs.python.org/issue25388>`_
553
Kostya Serebryany2cf90822016-03-19 01:05:33 +0000554* OpenSSL/BoringSSL: `[1] <https://boringssl.googlesource.com/boringssl/+/cb852981cd61733a7a1ae4fd8755b7ff950e857d>`_ `[2] <https://openssl.org/news/secadv/20160301.txt>`_ `[3] <https://boringssl.googlesource.com/boringssl/+/2b07fa4b22198ac02e0cee8f37f3337c3dba91bc>`_ `[4] <https://boringssl.googlesource.com/boringssl/+/6b6e0b20893e2be0e68af605a60ffa2cbb0ffa64>`_
Kostya Serebryany064a6722015-12-05 02:23:49 +0000555
Kostya Serebryany928eb332015-10-12 18:15:42 +0000556* `Libxml2
557 <https://bugzilla.gnome.org/buglist.cgi?bug_status=__all__&content=libFuzzer&list_id=68957&order=Importance&product=libxml2&query_format=specific>`_
Kostya Serebryany45dac2a2015-10-10 02:14:18 +0000558
Kostya Serebryany240a1592015-11-11 05:25:24 +0000559* `Linux Kernel's BPF verifier <https://github.com/iovisor/bpf-fuzzer>`_
Kostya Serebryany62921282015-09-11 16:34:14 +0000560
Kostya Serebryany240a1592015-11-11 05:25:24 +0000561* LLVM: `Clang <https://llvm.org/bugs/show_bug.cgi?id=23057>`_, `Clang-format <https://llvm.org/bugs/show_bug.cgi?id=23052>`_, `libc++ <https://llvm.org/bugs/show_bug.cgi?id=24411>`_, `llvm-as <https://llvm.org/bugs/show_bug.cgi?id=24639>`_, Disassembler: http://reviews.llvm.org/rL247405, http://reviews.llvm.org/rL247414, http://reviews.llvm.org/rL247416, http://reviews.llvm.org/rL247417, http://reviews.llvm.org/rL247420, http://reviews.llvm.org/rL247422.
Kostya Serebryanyfab4fba2015-08-11 01:53:45 +0000562
Kostya Serebryany79677382015-03-31 21:39:38 +0000563.. _pcre2: http://www.pcre.org/
564
565.. _AFL: http://lcamtuf.coredump.cx/afl/
566
Alexey Samsonov675e5392015-04-27 22:50:06 +0000567.. _SanitizerCoverage: http://clang.llvm.org/docs/SanitizerCoverage.html
Kostya Serebryanyb17e2982015-07-31 21:48:10 +0000568.. _SanitizerCoverageTraceDataFlow: http://clang.llvm.org/docs/SanitizerCoverage.html#tracing-data-flow
569.. _DataFlowSanitizer: http://clang.llvm.org/docs/DataFlowSanitizer.html
Kostya Serebryany9e1a2382016-03-29 23:07:36 +0000570.. _AddressSanitizer: http://clang.llvm.org/docs/AddressSanitizer.html
Kostya Serebryany5e593a42015-04-08 06:16:11 +0000571
572.. _Heartbleed: http://en.wikipedia.org/wiki/Heartbleed
Kostya Serebryany926b9bd2015-05-22 22:43:05 +0000573
574.. _FuzzerInterface.h: https://github.com/llvm-mirror/llvm/blob/master/lib/Fuzzer/FuzzerInterface.h