| # Benchmark Tools |
| |
| ## compare_bench.py |
| |
| The `compare_bench.py` utility which can be used to compare the result of benchmarks. |
| The program is invoked like: |
| |
| ``` bash |
| $ compare_bench.py <old-benchmark> <new-benchmark> [benchmark options]... |
| ``` |
| |
| Where `<old-benchmark>` and `<new-benchmark>` either specify a benchmark executable file, or a JSON output file. The type of the input file is automatically detected. If a benchmark executable is specified then the benchmark is run to obtain the results. Otherwise the results are simply loaded from the output file. |
| |
| The sample output using the JSON test files under `Inputs/` gives: |
| |
| ``` bash |
| $ ./compare_bench.py ./gbench/Inputs/test1_run1.json ./gbench/Inputs/test1_run2.json |
| Comparing ./gbench/Inputs/test1_run1.json to ./gbench/Inputs/test1_run2.json |
| Benchmark Time CPU |
| ---------------------------------------------- |
| BM_SameTimes +0.00 +0.00 |
| BM_2xFaster -0.50 -0.50 |
| BM_2xSlower +1.00 +1.00 |
| BM_10PercentFaster -0.10 -0.10 |
| BM_10PercentSlower +0.10 +0.10 |
| ``` |
| |
| When a benchmark executable is run, the raw output from the benchmark is printed in real time to stdout. The sample output using `benchmark/basic_test` for both arguments looks like: |
| |
| ``` |
| ./compare_bench.py test/basic_test test/basic_test --benchmark_filter=BM_empty.* |
| RUNNING: test/basic_test --benchmark_filter=BM_empty.* |
| Run on (4 X 4228.32 MHz CPU s) |
| 2016-08-02 19:21:33 |
| Benchmark Time CPU Iterations |
| -------------------------------------------------------------------- |
| BM_empty 9 ns 9 ns 79545455 |
| BM_empty/threads:4 4 ns 9 ns 75268816 |
| BM_empty_stop_start 8 ns 8 ns 83333333 |
| BM_empty_stop_start/threads:4 3 ns 8 ns 83333332 |
| RUNNING: test/basic_test --benchmark_filter=BM_empty.* |
| Run on (4 X 4228.32 MHz CPU s) |
| 2016-08-02 19:21:35 |
| Benchmark Time CPU Iterations |
| -------------------------------------------------------------------- |
| BM_empty 9 ns 9 ns 76086957 |
| BM_empty/threads:4 4 ns 9 ns 76086956 |
| BM_empty_stop_start 8 ns 8 ns 87500000 |
| BM_empty_stop_start/threads:4 3 ns 8 ns 88607596 |
| Comparing test/basic_test to test/basic_test |
| Benchmark Time CPU |
| --------------------------------------------------------- |
| BM_empty +0.00 +0.00 |
| BM_empty/threads:4 +0.00 +0.00 |
| BM_empty_stop_start +0.00 +0.00 |
| BM_empty_stop_start/threads:4 +0.00 +0.00 |
| ``` |
| |
| Obviously this example doesn't give any useful output, but it's intended to show the output format when 'compare_bench.py' needs to run benchmarks. |