[autotest] Optimize shard queries
The django filters are slow as it translates
to a sql that searches through entire afe_host_queue_entries
table in order to exclude completed jobs, non-aborted known jobs
and active jobs.
This CL optimizes the quries.
DEPLOY=apache
BUG=chromium:490485
TEST=1. Test in puppy lab. Shard sync jobs and frontend jobs correctly.
Aborting jobs works.
2. manually execute the resulting SQL and verify return value.
3. python
>>> shard =
models.Shard.objects.filter(hostname='chromeos-server14.mtv.corp.google.com')[0]
>>> q = models.Job.objects.filter(dependency_labels=shard.labels.all(),
hostqueueentry__isnull=False)
>>> q = models.Job._add_filters_for_shard_assignment(q, [333])
The above query takes 30 secs in prod.
4. unittest
With the raw sql, it takes 5 secs.
Change-Id: I95297f26a7f81b48e1ad18f7247b0e3d19751a9d
Reviewed-on: https://chromium-review.googlesource.com/272516
Tested-by: Fang Deng <fdeng@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Fang Deng <fdeng@chromium.org>
1 file changed