| Sean Purcell | 2500dcf | 2017-03-09 13:59:26 -0800 | [diff] [blame] | 1 | Testing |
| 2 | ======= |
| 3 | |
| 4 | Zstandard CI testing is split up into three sections: |
| 5 | short, medium, and long tests. |
| 6 | |
| 7 | Short Tests |
| 8 | ----------- |
| 9 | Short tests run on CircleCI for new commits on every branch and pull request. |
| 10 | They consist of the following tests: |
| 11 | - Compilation on all supported targets (x86, x86_64, ARM, AArch64, PowerPC, and PowerPC64) |
| 12 | - Compilation on various versions of gcc, clang, and g++ |
| 13 | - `tests/playTests.sh` on x86_64, without the tests on long data (CLI tests) |
| Bimba Shrestha | 6a4258a | 2020-02-05 16:55:00 -0800 | [diff] [blame] | 14 | - Small tests (`tests/legacy.c`, `tests/longmatch.c`) on x64_64 |
| Sean Purcell | 2500dcf | 2017-03-09 13:59:26 -0800 | [diff] [blame] | 15 | |
| 16 | Medium Tests |
| 17 | ------------ |
| 18 | Medium tests run on every commit and pull request to `dev` branch, on TravisCI. |
| 19 | They consist of the following tests: |
| 20 | - The following tests run with UBsan and Asan on x86_64 and x86, as well as with |
| 21 | Msan on x86_64 |
| Taylor Braun-Jones | 12e421b | 2020-01-30 10:31:00 -0500 | [diff] [blame] | 22 | - `tests/playTests.sh --test-large-data` |
| Sean Purcell | 2500dcf | 2017-03-09 13:59:26 -0800 | [diff] [blame] | 23 | - Fuzzer tests: `tests/fuzzer.c`, `tests/zstreamtest.c`, and `tests/decodecorpus.c` |
| 24 | - `tests/zstreamtest.c` under Tsan (streaming mode, including multithreaded mode) |
| 25 | - Valgrind Test (`make -C tests valgrindTest`) (testing CLI and fuzzer under valgrind) |
| 26 | - Fuzzer tests (see above) on ARM, AArch64, PowerPC, and PowerPC64 |
| 27 | |
| 28 | Long Tests |
| 29 | ---------- |
| 30 | Long tests run on all commits to `master` branch, |
| 31 | and once a day on the current version of `dev` branch, |
| 32 | on TravisCI. |
| 33 | They consist of the following tests: |
| 34 | - Entire test suite (including fuzzers and some other specialized tests) on: |
| 35 | - x86_64 and x86 with UBsan and Asan |
| 36 | - x86_64 with Msan |
| 37 | - ARM, AArch64, PowerPC, and PowerPC64 |
| 38 | - Streaming mode fuzzer with Tsan (for the `zstdmt` testing) |
| 39 | - ZlibWrapper tests, including under valgrind |
| 40 | - Versions test (ensuring `zstd` can decode files from all previous versions) |
| 41 | - `pzstd` with asan and tsan, as well as in 32-bits mode |
| 42 | - Testing `zstd` with legacy mode off |
| 43 | - Testing `zbuff` (old streaming API) |
| Ryan Schmidt | b567ce9 | 2018-06-09 14:31:17 -0500 | [diff] [blame] | 44 | - Entire test suite and make install on macOS |