| commit | 40a5a4a33df9dc5bb8aec137b9ea1ae98d39e77e | [log] [tgz] |
|---|---|---|
| author | Joel Galenson <jgalenson@google.com> | Thu May 20 09:50:44 2021 -0700 |
| committer | Joel Galenson <jgalenson@google.com> | Tue May 25 09:34:03 2021 -0700 |
| tree | 97d3981227798334efe10bdbc22e5158d50181f4 | |
| parent | ca72ecc4e7506bbbfcaa9f171dc9ded5634e35c3 [diff] |
Allow rolling crate upgrades back. Currently, if an error occurs after we overwrite the previous version of the crate with a new one, we can end up in either of two bad states. If the error is hidden (e.g., when generating TEST_MAPPING), the updater proceeds as normal, which can result in uploading an invalid CL (although it does often mark it as Verified - 1). Worse, if the error is visible (e.g., git commit fails), the bad files are left in the current directory, which if they contain a build error, can cause all future upgrades to fail. We fix this by keeping the previous version of the crate around (a comment in the code suggests that it did this, but it did not). If the updater encounters an error, it rolls back to that saved copy. We have to be careful not to try to rollback if we haven't swapped to the new crate yet, however. Note that this is currently only implemented for crates. The GithubArchiveUpdater has similar code and could probably do the same thing; the GitUpdater would do it differently. Test: Run with and without errors. Change-Id: Ia48660ad72cb3ea1689d92f880ff033e9c35aed9
external updater is a tool to automatically updates libraries in external/ .
Check updates for a library, or verify METADATA is valid:
tools/external_updater/updater.sh check ${LIBNAME}
Check updates for all libraries in external/:
tools/external_updater/updater.sh check --all
Update a library:
tools/external_updater/updater.sh update ${LIBNAME}
LIBNAME can be the path to a library under external/. E.g. kotlinc, or python/cpython3.
To use this tool, a METADATA file must present at the root of the repository. The full definition can be found here. Or see example here
The most important part in the file is a list of urls. external_updater will go through all urls and uses the first supported url.
If type of a URL is set to GIT, the URL must be a git upstream (the one you can use with git clone). And the version field must be either a version tag, or SHA. The tool will find the latest version tag or sha based on it.
When upgrade, the tool will simply run git merge tag/sha.
IMPORTANT: It is suggested to set up a upstream-master branch to replicate upstream. Because most users don't have the privilege to upload changes not authored by themselves. This can be done by filing a bug to componentid:99104.
If the version is a SHA, the tool will always try to upgrade to the top of upstream. As long as there is any new change upstream, local library will be treated as stale.
If the version is not a SHA, the tool will try to parse the version to get a numbered version. Currently the supported version format is:
<prefix><version_number><suffix>
version_number part can be numbers separated by . or - or _.
If you have project where this isn't working, file a bug so we can take a look.
It is suggested to verify all local changes when upgrading. This can be done easily in Gerrit, by comparing parent2 and the patchset.
If the url type is ARCHIVE, and the url is from GitHub, external_updater can upgrade a library based on GitHub releases.
If you have the choice between archives and git tags, choose tags. Because that makes it easier to manage local changes.
The tool will query GitHub to get the latest release from:
https://github.com/user/proj/releases/latest
If the tag of latest release is not equal to version in METADATA file, a new version is found. The tool will download the tarball and overwrite the library with it.
If there are multiple archives in one GitHub release, the one most similar to previous (from METADATA) will be used.
After upgrade, files not present in the new tarball will be removed. But we explicitly keep files famous in Android tree. See here.
If more files need to be reserved, a post_update.sh can be created to copy these files over. See example.
Local patches can be kept as patches/*.diff. They will be applied after upgrade. example
There is some support to automatically check updates for all external libraries every hour, send email and change. Currently this is done by running the following script on a desktop machine.
#!/bin/bash cd /src/aosp while true do repo abandon tmp_auto_upgrade repo forall -c git checkout . repo forall -c git clean -xdf repo sync -c source build/envsetup.sh lunch aosp_arm-eng mmma tools/external_updater out/soong/host/linux-x86/bin/external_updater_notifier \ --history ~/updater/history \ --recipients=android_external_lib_updates@google.com \ --generate_change \ --all date echo "Sleeping..." sleep 3600 done