tradefed_test: inaccurate test results => fail.
Looking through the logs of past 3 months, all the instances of
this warning was really lacking some test cases. Let's try making
them a failure for now.
BUG=none
TEST=normal run for some tests not affected (I could not reproduced the
case that hits this code path, though.)
Change-Id: Ib42ddf08ef20282fb7e04a87810a37bace7a9824
Reviewed-on: https://chromium-review.googlesource.com/1223715
Commit-Ready: Kazuhiro Inaba <kinaba@chromium.org>
Tested-by: Kazuhiro Inaba <kinaba@chromium.org>
Reviewed-by: Ilja H. Friedel <ihf@chromium.org>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
diff --git a/server/cros/tradefed_test.py b/server/cros/tradefed_test.py
index 2454371..6ac4a0d 100644
--- a/server/cros/tradefed_test.py
+++ b/server/cros/tradefed_test.py
@@ -1144,6 +1144,11 @@
# Check if all the tests passed.
if failed <= waived and all_done:
+ if not all(accurate):
+ raise error.TestFail(
+ 'Failed: Not all tests were executed. After %d '
+ 'retries passing %d tests, waived=%d. %s' % (
+ steps, passed, waived, self.summary))
# TODO(ihf): Make this error.TestPass('...') once
# available.
if steps > 0 and self._warn_on_test_retry:
@@ -1151,11 +1156,6 @@
'Passed: after %d retries passing %d tests, '
'waived=%d. %s' % (steps, passed, waived,
self.summary))
- if not all(accurate):
- raise error.TestWarn(
- 'Passed: after %d retries passing %d tests, '
- 'waived=%d. Tests may not be accurate. %s' % (
- steps, passed, waived, self.summary))
return
if session_id == None: