Add MultiNode TradeFed tests for --shards and with session retry
This builds upon the previous test setup and runner for TradeFed
(CTS/VTS) with the following major features and changes:
(1) Allow for "sharded" tests accross multiple DUTs (speedup test suite
execution)
(2) Use session retry for stabilizing final failure counts
(3) Don't require DUT root access in any of the involved operations
(4) Trying to be failure tolerant: Continue as long as at least one
device is available. Re-establish lost connection if possible.
(5) Drop support for VTS (temporary solution, it was not tested at all)
(6) Add support for GTS and STS
Details related to the points above:
(1) Implemented using the LAVA MultiNode API. There should be one
"master" job that executes the TradeFed runner and an arbitrary
number of 'workers' that host additional DUTs and make them
accessible via adb tcpip. Continuous messaging between master and
worker is used to recover lost devices when needed.
(3) Migrated features requiring root to non-root equivalents.
(5) No testing was done with VTS, therefore it was partly dropped for
now.
(6) Apart from adjusted paths and command names, no changes were
required compared to CTS.
This change would ideally be a replacement of the existing TradeFed
setup, or at least share more code with it. Remaining issues on that way
are:
- (5) - no VTS testing done. This relates to behavior of --retry,
--shards, and related shell output
- MultiNode environment expected by this setup. The runner could be
further generalized to be usable from a regular (non-MultiNode) test
submission, so that it would in a basic case not introduce any
overhead compared to the existing runner.
Change-Id: Idef4a5a9aac1f3cd8fc2aa1e609f544ee15ae528
Depends-On: I23f22344b9bd758d3898d4345204157cecd7d624
Depends-On: Ie011cb2cd899dac938066ca4eee7652b83ac38d4
Depends-On: I638d6a2cf44a5569569703308172e3056030783f
Depends-On: I912ab5168bbe1b4fc0a3a8112db5cbc94a812b7c
Signed-off-by: Karsten Tausche <karsten@fairphone.com>
6 files changed