[autotest] run_suite returns new codes INFRA_FAILURE and SUITE_TIMEOUT
Previously run_suite returns:
0. OK
1. ERROR: tests failed
2. WARNING: tests warned, none failed
Lab is down
Once this cl lands, it will return:
0 - OK: Suite finished successfully
1 - ERROR: Test(s) failed, or hits its own timeout
2 - WARNING: Test(s) rasied a wanring, none failed/timeout.
3 - INFRA_FAILURE: Infrastructure related issues, e.g.
* Lab is down
* Too many duts (defined as a constant) in repair failed status
* Suite job issues, like bug in dynamic suite,
user aborted the suite, lose a drone/all devservers/rpc server,
0 tests ran, etc.
4 - SUITE_TIMEOUT: Suite timed out, some tests ran,
none failed by the time the suite job was aborted. This will cover,
but not limited to, the following cases:
* A devserver failure that manifests as a timeout
* No DUTs available midway through a suite
* Provision/Reset/Cleanup took longer time than expected for new image
* A regression in scheduler tick time.
BUG=chromium:371122
TEST=unittest; locally call run_suite to trigger different
return codes;Run "run_suite -m" against past 500 jobs, verify the return
code.
Change-Id: Ib9d9baa77866a4dfdf8d8160eb155a5a01738505
Reviewed-on: https://chromium-review.googlesource.com/198802
Tested-by: Fang Deng <fdeng@chromium.org>
Reviewed-by: Prashanth B <beeps@chromium.org>
Commit-Queue: Fang Deng <fdeng@chromium.org>
2 files changed