Autotest: Add timeout logic for Try-Jobs.

Changed the reimager/job_status logic to monitor the Try-Job it's
waiting on and if a certain amount of time passes, it will time it
out and abort it.

Also adapted the suite job automatic abort on aborted Try-Job logic,
to have it decide whether or not to continue depending on what
percentage of Host Queue Entries (i.e. number of devices being imaged)
was successful. This is done through a new public method in job_status.

By default, the timeout time (4 hours) is controlled by a value in the
global config but can be passed in by the suite control file.

BUG=chromium-os:36175
TEST=Existing unit tests and ensuring that if not enough jobs are
     completed, it aborts and if enough are done it continues.

Change-Id: I3ebd1653ba2b196e027e72768b33e02d87ec9673
Reviewed-on: https://gerrit.chromium.org/gerrit/39125
Commit-Ready: Simran Basi <sbasi@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Tested-by: Simran Basi <sbasi@chromium.org>
6 files changed