[autotest] Send an email if a drone is hitting process limit

Add a setting 'max_processes_warning_threshold' to global_config.ini.
If the raitio of active processes to max processes goes over
the threshold, DroneManager will send a email to
chromeos-lab-infrastructure@.

To prevent it from spamming our mailing list, only one email
will be sent for each drone within 24 hours.

BUG=chromium:277184
TEST=Test locally and confirm email is sent out when
the threshold is reached.
DEPLOY=scheduler

Change-Id: Id4a883ff6c26e9bba384974c255a0ce0f3cb4056
Reviewed-on: https://chromium-review.googlesource.com/168147
Reviewed-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Alex Miller <milleral@chromium.org>
Tested-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Fang Deng <fdeng@chromium.org>
3 files changed