| Getting started |
| =============== |
| git clone https://git.linaro.org/qa/test-definitions.git |
| cd ./test-definitions |
| . ./automated/bin/setenv.sh |
| test-runner -h |
| |
| Running test |
| ============ |
| Both test script and corresponding test definition files provided in the same |
| folder that named with the test case name. |
| |
| Test script can be run independently, you may need to specify parameters |
| manually as needed. |
| |
| Test definition file in YAML format created for each test to |
| - Define default test parameters |
| - Run test with local test-runner, which supports to run a set of tests |
| defined in agenda file, customized parameters are supported as well. |
| - Run test in LAVA(Linaro Automation and Validation Architecture) |
| |
| Running test script |
| ------------------ |
| linux: |
| cd ./automated/linux/smoke/ |
| ./smoke.sh |
| # skip package installation. |
| ./smoke.sh -s true |
| |
| android: |
| cd ./automated/android/dd-wr-speed/ |
| ./dd-wr-speed.sh |
| # Specify SN when more than one device connected. |
| ./dd-wr-speed.sh -s "serial_no" |
| # Specify other params. |
| ./dd-wr-speed.sh -i "10" -p "/dev/block/mmcblk1p1" |
| |
| Using test-runner |
| ---------------- |
| single test run: |
| test-runner -d ./automated/linux/smoke/smoke.yaml |
| # skip package install |
| test-runner -d ./automated/linux/smoke/smoke.yaml -s |
| |
| running test plan: |
| # Run a set of tests defined in agenda file. |
| test-runner -p ./plans/linux-example.yaml |
| # Apply test plan overlay to skip, amend or add tests. |
| test-runner -p ./plans/linux-example.yaml -O test-plan-overlay-example.yaml |
| |
| Collecting result |
| ================= |
| Please find test results in the following locations. |
| - ${OUTPUT} and ${RESULT_FILE} for each test are defined in the test case. |
| - test-runner parse stdout.log and save result to ${test_path} |
| - test-runner combines results of executed tests and save them in |
| ${OUTPUT}/results.csv, |
| - ${OUTPUT} path of test-runner can be modified with '-o' |
| |
| Contributing |
| ============ |
| |
| Changes need to be able to pass the following check, which will check files in |
| the most recent commit: |
| |
| python3 validate.py -g -s SC1091 |
| |
| To develop locally, there are Dockerfiles in test/ that can be used to simulate |
| target environments. The easiest way to use is to run `test.sh |
| [debian|centos]`. test.sh will run validate.py, and then build the Docker |
| environment specified, run plans/linux-example.yaml, and then drop into a bash |
| shell inside the container so that things like /root/output can be inspected. |
| It is not (yet) a pass/fail test; merely a development helper and validation |
| environment. |