[autotest] Sync suite job keyvals to shards.
As we roll out shard, we need a way to communicate suite-level
metadata to shards. One way to do it is to store suite-level
metadata as job keyvals and sync them to shard, which is what
this CL is about.
Currently, only the keyval 'suite_min_duts' is needed by a shard.
In the future, we can pipe more keyvals to shards as needed like
file_bugs and retry if we were to move bug filing and retry
out of dynamic suites.
CQ-DEPEND=CL:254270
DEPLOY=apache,shard_client
BUG=chromium:432653
TEST=Setup a testing cluster with on master and one shard.
python
>> import common
>> from autotest_lib.server import frontend
>> afe = frontend.AFE(server='localhost:8001')
>> afe.run('shard_heartbeat', shard_hostname='localhost:8004')
Confirm the returned packet has 'suite_keyvals'
TEST=Setup a testing cluster with on master and one shard.
- Create a suite job and confirm suite keyvals are synced to shard.
- Confirm keyvals known to shard won't get synced multiple times.
TEST=unittest
TEST=Run dummy suite with run_suite, confirm everything still works.
Change-Id: I5d7816c2b3f6dfa374a297c09d597573f762c374
Reviewed-on: https://chromium-review.googlesource.com/254230
Tested-by: Fang Deng <fdeng@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Fang Deng <fdeng@chromium.org>
diff --git a/frontend/afe/rpc_utils.py b/frontend/afe/rpc_utils.py
index 5d37be2..6804c32 100644
--- a/frontend/afe/rpc_utils.py
+++ b/frontend/afe/rpc_utils.py
@@ -986,12 +986,15 @@
@param known_job_ids: List of ids of jobs the shard already has.
@param known_host_ids: List of ids of hosts the shard already has.
- @returns: Tuple of two lists for hosts and jobs: (hosts, jobs).
+ @returns: Tuple of three lists for hosts, jobs, and suite job keyvals:
+ (hosts, jobs, suite_job_keyvals).
"""
hosts = models.Host.assign_to_shard(shard, known_host_ids)
jobs = models.Job.assign_to_shard(shard, known_job_ids)
-
- return hosts, jobs
+ parent_job_ids = [job.parent_job_id for job in jobs]
+ suite_job_keyvals = models.JobKeyval.objects.filter(
+ job_id__in=parent_job_ids)
+ return hosts, jobs, suite_job_keyvals
def _persist_records_with_type_sent_from_shard(