CrosVM Continuous Integration

Crosvm has a complex set of dependencies and requirements on the host machine to successfully build and run test cases. To allow for consistent testing in our continuous integration system (kokoro) and reproduction of those tests locally, we provide docker containers containing the build toolchain and a VM for testing.

How to run crosvm tests

Setting up the source

Since crosvm is part of chromiumos, and uses a couple of it's projects as dependencies, you need a standard chromiumos checkout as described by the ChromiumOS Developer Guide.

To reduce the number of repositories to download, you can use the -g crosvm argument on repo init. This will be significantly faster:

In summary:

$ repo init -u https://chromium.googlesource.com/chromiumos/manifest.git --repo-url https://chromium.googlesource.com/external/repo.git -g crosvm
$ repo sync -j4
$ cd src/platform/crosvm

Installing Podman (or Docker)

See Podman Installation for instructions on how to install podman.

For Googlers, see go/dont-install-docker for special instructions on how to set up podman.

If you already have docker installed, that will do as well. However podman is recommended as it will not run containers with root privileges.

Running all tests

To run all tests, just run:

./test_all

This will run all tests using the x86 and aarch64 builder containers. What does this do?

1. It will start ./ci/[aarch64_]builder --vm.

   The builder will build ChromeOS dependencies from your local repo checkout. If you make modifications to these dependencies (e.g. minijail, tpm2, cras) these will be included in tests.

   Then it will start a VM for running tests in the background. The VM is booting while the next step is running.

2. Then it will call ./run_tests inside the builder

   The script will pick which tests to execute and where. Simple tests can be executed directly, other tests require privileged access to devices and will be loaded into the VM to execute.

   Each test will in the end be executed by a call to cargo test -p crate_name.

Intermediate build data is stored in a scratch directory at ./target/ci/ to allow for faster subsequent calls (Note: If running with docker, these files will be owned by root).

Fast, iterative test runs

For faster iteration time, you can directly invoke some of these steps directly:

To only run x86 tests: ./ci/[aarch64_]builder --vm ./run_tests.

To run a simple test (e.g. the tempfile crate) that does not need the vm: ./ci/[aarch64_]builder cargo test -p tempfile.

Or run a single test (e.g. kvm_sys) inside the vm: ./ci/[aarch64*]builder --vm cargo test -p kvm_sys.

Since the VM (especially the fully emulated aarch64 VM) can be slow to boot, you can start an interactive shell and run commands from there as usual. All cargo tests will be executed inside the VM, without the need to restart the VM between test calls.

host$ ./ci/aarch64_builder --vm
crosvm-aarch64$ ./run_tests
crosvm-aarch64$ cargo test -p kvm_sys
...

Reproducing Kokoro locally

Kokoro uses the same builders and the same run_tests script to run tests. However, to keep the build stable, it syncs the chromiumos checkout to the fixed manifest found at ./ci/kokoro/manifest.xml.

To run tests using the same manifest, as well as the same build process that Kokoro uses, you can run: ./ci/kokoro/simulate_all.

Implementation Overview

Directories:

• ci/build_environment: Contains tooling for building the dependencies of crosvm.
• ci/crosvm_aarch64_builder: An x86 docker image to cross-compile for aarch64 and test with user-space emulation.
• ci/crosvm_base: Docker image shared by crosvm_builder and crosvm_aarch64_builder
• ci/crosvm_builder: A native docker image for building and testing crosvm
• ci/crosvm_test_vm: Dockerfile to build the VM included in the builder containers.
• ci/kokoro: Configuration files and build scripts used by Kokoro to run crosvm tests.

Scripts:

• ci/aarch64_builder: Script to start the crosvm_aarch64_builder container
• ci/builder: Script to start the crosvm_builder container
• ci/run_container.sh: Implementation behind the above scripts.
• test_runner.py: Implementation behind the ./test_all script.

Building and uploading a new version of builders

The docker images for all builders can be built with make and uploaded with make upload. Of course you need to have docker push permissions for gcr.io/crosvm-packages/ for the upload to work.