Generate LAVA job definitions to test a software build

generate-lava-jobs aims at generating ready-to-use LAVA job definitions to execute build test suites and obtain final results without operator intervention. LAVA (Linaro Automated Validation Architecture) is "an automated testing system [...] which delivers the device automation and result collection" that we use to validate software builds on many device configurations.

Job definitions for the following Google test suites are generated:

CTS, the Compatibility Test Suite;
GTS, the Google Mobile Services Test Suite;
STS, the Security Test Suite.

These suites are automated testing harnesses to ensure compatibility and correct behavior of Android software builds. More information is available on the CTS page. GTS and STS are not publicly available.

Since all test suites mentioned above are based on the Trade Federation continuous test framework (TradeFed), they share a common template (templates/multinode-tradefed-xts.yaml) that is configured per test suite.

Additionally, following Fairphone-internal test suites are supported:

Sanity Check, a quick build validation test;
Upgrade Path Test Suite, that validates (OTA) upgrade paths using Sanity Check.

Requirements

Python 3

How to use

This assumes that you have a lavacli identity set up; see below for details.

$ virtualenv -p python3 .venv
$ . ./.venv/bin/activate
$ pip install -r requirements.txt
$ ./generate-lava-jobs --output-directory <OUTPUT_DIR> <BUILD_ID>
$ lavacli -i <IDENTITY> jobs submit <OUTPUT_DIR>/*

Setting up a lavacli identity

LAVA implements an XML-RPC API for controlling the LAVA scheduler, fetching results and more. lavacli provides command line access to many of these API functions, including submitting jobs.

Set up API access for lavacli:

Assuming your LAVA host address is https://lavahost/ and your user name is lavauser.
Create an API authentication token in LAVA on https://lavahost/api/tokens/.
Add an identity to lavacli for your user and LAVA instance. <IDENTITY> is an arbitrary name for you to locally identify your setup.
```
 $ lavacli identities add --uri http://lavahost/RPC2 --username lavauser --token <TOKEN> <IDENTITY>
```