ci/deqp: Switch to a new dEQP runner written in Rust.
I found the C++ runner hard to develop on, and we had stability issues and
outstanding feature needs that made me want something I felt good about
hacking on. Thus, Rewrite It In Rust of the deqp runner.
The new runner includes:
- Skip lists don't reshuffle the test list.
- Known-flake handling without resorting to skip lists (fixing our main CI
reliability issue on a3xx right now).
- Per-thread Vulkan shader caches should speed up VK CI runtime.
- Tracking of crashes separate from fails (so we can see progress on that
front).
- Logging of deqp stderr spam (particularly assertion failures!) in the CI
log.
- Integrated QPA filtering so we don't have bash perf issues for it.
- Logging of what caselist to go look at for a given error report (in red,
so it's easier to find in your CI log).
- The code is 1/3 unit tests, and easy to extend for more coverage.
- Non-LAVA CI runs create a failures.csv in artifacts that you can check
in as your deqp-*-fails.txt file.
- Test runtime is included in results.csv so you can debug how to speed up
your CI job.
- Pretty summary at the end of the run of slow/flaky/failed tests.
Since this is a new runner with a different RNG, the test groups are
shuffled one more time. This seems to result in some panfrost T720
stability issues (See its new deqp-panfrost-t720-flakes.txt), and one new
flake in freedreno a630.
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7434>
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 2164850..73e1034 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -352,13 +352,13 @@
x86_test-gl:
extends: .use-x86_test-base
variables:
- FDO_DISTRIBUTION_TAG: &x86_test-gl "2020-11-05-kmod"
+ FDO_DISTRIBUTION_TAG: &x86_test-gl "2020-11-05-deqp-runner"
# Debian 10 based x86 test image for VK
x86_test-vk:
extends: .use-x86_test-base
variables:
- FDO_DISTRIBUTION_TAG: &x86_test-vk "2020-11-05-kmod"
+ FDO_DISTRIBUTION_TAG: &x86_test-vk "2020-11-05-deqp-runner"
# Debian 9 based x86 build image (old LLVM)
x86_build_old:
@@ -414,7 +414,7 @@
extends:
- .use-arm_test-base
variables:
- FDO_DISTRIBUTION_TAG: &arm64_test "2020-09-28-deqp-surfaceless-fix"
+ FDO_DISTRIBUTION_TAG: &arm64_test "2020-11-03-deqp-runner"
.use-arm64_test:
variables:
@@ -1122,6 +1122,7 @@
variables:
DEQP_EXPECTED_FAILS: deqp-softpipe-fails.txt
DEQP_SKIPS: deqp-softpipe-skips.txt
+ DEQP_FLAKES: deqp-softpipe-flakes.txt
GALLIUM_DRIVER: "softpipe"
DEQP_EXPECTED_RENDERER: softpipe