Try to reduce variance in angle_perftests.
This change does a few things:
- make perf test runner script print % variation instead of stddev
This makes it a bit more clear how much variance there is.
- stabilize CPU in the render perf tests
Setting a thread affinity and priority should stop from switching cores
during the run. Hopefully can prevent background noise from changing
the test results.
- warm up the benchmark with a few iterations
This should hopefully make the test results a bit more stable.
- output a new normalized perf result value
The new result is normalized against the number of iterations. So it
should hopefully be stable even if the number of iterations is changed.
- increases the iteration count in the draw call perf tests.
These tests were completely dominated by SwapBuffers time. Increasing
the iterations per step means we actually are bottlenecked on CPU time
instead.
Bug: angleproject:2923
Change-Id: I5ee347cf93df239ac33b83dc5effe4c21e066736
Reviewed-on: https://chromium-review.googlesource.com/c/1303679
Commit-Queue: Jamie Madill <jmadill@chromium.org>
Reviewed-by: Shahbaz Youssefi <syoussefi@chromium.org>
Reviewed-by: Yuly Novikov <ynovikov@chromium.org>
diff --git a/src/tests/perf_tests/CompilerPerf.cpp b/src/tests/perf_tests/CompilerPerf.cpp
index 1018d4f..a29a23b 100644
--- a/src/tests/perf_tests/CompilerPerf.cpp
+++ b/src/tests/perf_tests/CompilerPerf.cpp
@@ -162,6 +162,8 @@
const char *kTrickyESSL300Id = "TrickyESSL300";
+constexpr int kNumIterationsPerStep = 10;
+
struct CompilerPerfParameters final : public angle::CompilerParameters
{
CompilerPerfParameters(ShShaderOutput output,
@@ -206,7 +208,8 @@
sh::TCompiler *mTranslator;
};
-CompilerPerfTest::CompilerPerfTest() : ANGLEPerfTest("CompilerPerf", GetParam().testId)
+CompilerPerfTest::CompilerPerfTest()
+ : ANGLEPerfTest("CompilerPerf", GetParam().testId, kNumIterationsPerStep)
{
}
@@ -250,8 +253,6 @@
ShCompileOptions compileOptions = SH_OBJECT_CODE | SH_VARIABLES |
SH_INITIALIZE_UNINITIALIZED_LOCALS | SH_INIT_OUTPUT_VARIABLES;
- const int kNumIterationsPerStep = 10;
-
#if !defined(NDEBUG)
// Make sure that compilation succeeds and print the info log if it doesn't in debug mode.
if (!mTranslator->compile(shaderStrings, 1, compileOptions))