LAVA: Add basic master and worker host definition

This creates very basic installations of a LAVA master and LAVA workers.
It assumes that both are in the same internal network. The setup also
prepares for MultiNode jobs by deploying the LAVA coordinator.

Issue: INFRA-18
Change-Id: I9cb8a6f22af1ce1e76928bb709f8965ac230822b
Test: ansible-lint lava-master.yaml
Test: ansible-lint lava-workers.yaml
21 files changed
tree: 57a6e722aad19850b8b6af9f1306b093f8d41855
  1. group_vars/
  2. inventories/
  3. roles/
  4. staging/
  5. testing/
  6. .gitignore
  7. .gitmodules
  8. artifactorial.yaml
  9. hiccup-server-proxy.yaml
  10. hiccup-server.yaml
  11. lava-master.yaml
  12. lava-workers.yaml
  13. README.md
  14. requirements-dev.txt
  15. requirements.txt
README.md

Ansible Playbooks

Be sure to init the submodule after cloning the repository:

$ git submodule init && git submodule update

Further you need to clone the ansible-playbooks-secrets repository at the same level as the ansible playbooks repository in your directory tree:

$ git clone ssh://$USER@review.fairphone.software:29418/Infra/ansible-playbooks-secrets

To use the playbooks install the ansible pip package.

$ virtualenv .venv/ansible -p python3
$ source .venv/ansible/bin/activate
$ pip3 install -r requirements.txt

Development

Run the Ansible linter on changed playbooks before submitting patches. Besides basic validation checks, it also checks for best practices for Ansible playbook development.

$ pip3 install -r requirements-dev.txt
$ ansible-lint <playbook-name.yaml>

Hiccup server proxy deployment

The Hiccup server runs behind a proxy that is taking care of the SSL of the requests. All requests arrive at the proxy and are then forwarded to the actual Hiccup server which handles only HTTP requests. The playbook takes care of the certificate generation using letsencrypt and configures the nginx proxy server.

To deploy the proxy, ansible connects to both the proxy and the Hiccup server, but the connection to the Hiccup server is only used to gather network facts about it.

To run the playbook execute:

$ ansible-playbook hiccup-server-proxy.yaml -i inventories/hiccup-server-proxy.ini

By default, this would attempt to connect to the servers using your current user. If a different user has the credentials for accessing the servers, you can specify it using the -u 'REMOTE_USER' option.

Staging proxy server

A staging server is currently running on ag.fairphone.com, its public domain name is hiccup.fairphone.software. By default, it uses the staging environment of letsencrypt so it does not run into rate limits on certificate issue and renewal.

To deploy it, it is only necessary to use the staging inventory file and to override some of the default variables for the hiccup-server-proxy playbook:

$ ansible-playbook hiccup-server-proxy.yaml \
    -i staging/inventories/hiccup-server-proxy.ini \
    -e @staging/group_vars/hiccup-server-proxy.yml

Hiccup server deployment

For deploying the Hiccup server, two machines will be configured: One for the PosgreSQL database and another for the actual Hiccup API server.

To run the playbook execute:

$ ansible-playbook hiccup-server.yaml \
    -i inventories/hiccup-server.ini \
    --ask-vault-pass

You can override all variables defined in group_vars/hiccup-server.yml and group_vars/hiccup-server-db.yml by using command line arguments. For example, if you would want to run the playbook but checkout Hiccup server code from <another-branch> instead of the default master branch:

$ ansible-playbook hiccup-server.yaml \
    -i inventories/hiccup-server.ini \
    --extra-vars "hiccup_server_revision=<another-branch>" \
    --ask-vault-pass

Staging server

To deploy the Hiccup server in the container running on the staging server ag.fairphone.com, use the staging inventory file and the staging vars file:

$ ansible-playbook hiccup-server.yaml \
    -i staging/inventories/hiccup-server.ini \
    -e @staging/group_vars/hiccup-server.yml \
    --ask-vault-pass

Testing

For testing purposes with virtual machines, a different inventory file can be specified using the -i option when running the playbook. Examples for inventory files for testing can be found in the testing/inventories subdirectory.

To use the testing inventory execute:

$ ansible-playbook hiccup-server.yaml \
    -i testing/inventories/hiccup-server.ini \
    --ask-vault-pass

The example testing configuration works for virtual machines which are spun up using vagrant. They can be accessed through forwarded ssh ports.