Gitiles
Code Review Sign In
gerrit-public.fairphone.software / platform / external / autotest / 4245a73ec638b602bd4546637b41c74f39158e14 / scheduler / monitor_db.py
  1. 456d3c1 Upstream Autotest merge. by Dale Curtis · 13 years ago
  2. 30cb8eb Fix --image support so it only runs on normal jobs. by Dale Curtis · 13 years ago
  3. aa51336 Host scheduler refactoring. Move HostScheduler out of monitor_db. by Dale Curtis · 14 years ago
  4. a82dc35 Merge remote branch 'autotest-upstream/master' into try-box1 by Eric Li · 14 years ago
  5. 861b2d5 Merge remote branch 'cros/upstream' into master by Eric Li · 14 years ago
  6. 5a8c6ad Add support for an --image flag to atest. by Paul Pendlebury · 14 years ago
  7. e0493a4 Merge remote branch 'cros/upstream' into tempbranch by Eric Li · 14 years ago
  8. 6f27d4f Merge remote branch 'cros/upstream' into tempbranch3 by Eric Li · 14 years ago
  9. 517d95a Revert "Merge remote branch 'cros/upstream' into tempbranch2" by Benson Leung · 14 years ago
  10. 25fc6d1 Merge remote branch 'cros/upstream' into tempbranch2 by Eric Li · 14 years ago
  11. c7d387e by jamesren · 14 years ago
  12. 17cadd6 by jamesren · 14 years ago
  13. b8f3f35 by jamesren · 14 years ago
  14. e7c65cb by jamesren · 14 years ago
  15. 42318f7 by jamesren · 14 years ago
  16. dd77e01 by jamesren · 14 years ago
  17. 2566356 Fix process counting for SelfThrottledPostJobTask. Would previously by jamesren · 14 years ago
  18. 76fcf19 Add ability to associate drone sets with jobs. This restricts a job to by jamesren · 14 years ago
  19. b7c5d27 monitor_db.py: Fix SyntaxWarning by lmr · 14 years ago
  20. 47bd737 Set hostless queue entries to STARTING upon scheduling the agent. This by jamesren · 15 years ago
  21. e0cbc91 Add support to autoserv for a --control-filename parameter, to allow users to by mbligh · 15 years ago
  22. dd85524 Abstract out common models used in the frontend's models.py so that django is not required to interact with non Django portions of the code. by jamesren · 15 years ago
  23. e21bf41 Minor fix to new metahost handlers code in scheduler to ensure handlers get a tick every cycle, even if there are no queued metahost jobs. by jamesren · 15 years ago
  24. c44ae99 Refactor scheduler models into a separate module, scheduler_models. This module doesn't depend on monitor_db, only the other way around. The separation and isolation of dependencies should help us organize the scheduler code a bit better. by jamesren · 15 years ago
  25. 883492a First iteration of pluggable metahost handlers. This change adds the basic framework and moves the default, label-based metahost assignment code into a handler. It includes some refactorings to the basic scheduling code to make things a bit cleaner. by jamesren · 15 years ago
  26. cc92936 Basic support for "summary results" -- articifial test results that are explicitly recorded by a server-side control file or code that it calls. This CL just adds the record_summary() method to the server_job object. It lacks any special parser support or TKO DB changes, those will come later. by showard · 15 years ago
  27. 7e67b43 New code for performing explicit joins with custom join conditions. by showard · 15 years ago
  28. 4076c63 In scheduler check for existence of results before trying to write the .archiver_failed file. by showard · 15 years ago
  29. c1a98d1 Support for job keyvals by showard · 15 years ago
  30. e1575b5 When the archiver fails for any reason, write a .archiver_failed file to the results dir. by showard · 15 years ago
  31. 948eb30 Construct an absolute path to the archiving control file when running the Archiving stage. Using a relative path was just silly and lazy and prone to breakage. by showard · 15 years ago
  32. 64a9595 When using Django models from a script, make the current user default to an actual database user named "autotest_system". This allows for simpler, more consistent code. by showard · 15 years ago
  33. 4608b00 Add a new Archiving stage to the scheduler, which runs after Parsing. This stage is responsible for copying results to the results server in a drone setup, a task currently performed directly by the scheduler, and allows for site-specific archiving functionality, replacing the site_parse functionality. It does this by running autoserv with a special control file (scheduler/archive_results.control.srv), which loads and runs code from the new scheduler.archive_results module. The implementation was mostly straightfoward, as the archiving stage is fully analogous to the parser stage. I did make a couple of refactorings: by mbligh · 15 years ago
  34. eab66ce Rename the tables in the databases, by prefixing the app name. This is by showard · 15 years ago
  35. 402934a Clear the Django connection query log after each tick. This was a major memory leak. by showard · 15 years ago
  36. f13a9e2 Add periodic CPython garbage collector statistics logging to aid in by showard · 15 years ago
  37. 493beaa fix a bug with pre-job keyvals, introduced in recent refactorings, and added new test to check it by showard · 15 years ago
  38. a9545c0 backend support for hostless jobs by showard · 15 years ago
  39. 2ca64c9 * add a couple simple test cases to the scheduler functional test for metahosts by showard · 15 years ago
  40. d119565 Make drone_manager track running processes counts using only the information passed in from the scheduler. Currently it also uses process counts derived from "ps", but that is an unreliable source of information. This improves accuracy and consistency and gives us full control over the process. by showard · 15 years ago
  41. b21b8c8 Fix handling of database reconnects in the scheduler by enhancing the "django" database_connection backend and having the scheduler use it. This eliminates the duplicate connection that the scheduler was setting up -- now it uses only a single connection (the Django one). by showard · 15 years ago
  42. d07a5f3 The check for enough pending hosts after the delay to wait for others to by showard · 15 years ago
  43. 418785b Some improvements to process tracking in the scheduler. by showard · 15 years ago
  44. 9bb960b Support restricting access to drones by user. Administrators can put lines like by showard · 15 years ago
  45. e60e44e Special tasks show "Failed" as their status instead of "Completed" if by showard · 15 years ago
  46. 65db393 * impose prioritization on SpecialTasks based on task type: Repair, then Cleanup, then Verify. remove prioritization of STs with queue entry over those without. this leads to more sane ordering of execution in certain unusual contexts -- the added functional test cases illustrate a few (in some cases, it's not just more sane, it eliminates bugs as well). by showard · 15 years ago
  47. 7b2d7cb We never considered the handling of DO_NOT_VERIFY hosts in certain situations. This adds handling of those cases to the scheduler and adds tests to the scheduler functional test. by showard · 15 years ago
  48. 37757f3 Change "unrecovered active host queue entries" to be a more accurate by showard · 15 years ago
  49. 8375ce0 Fix unindexable object error raised on the error path within by showard · 15 years ago
  50. b890045 In scheduler recovery, allow Running HQEs with no process. The tick code already handles them fine (by re-executing Autoserv), but the recovery code was explicitly disallowing them. With this change, it turns out there's only one status that's not allowed to go unrecovered -- Verifying -- so I changed the code to reflect that and I made the failure conditions more accurate. by showard · 15 years ago
  51. 5682407 Added more logging, and fixed logging in HostQueueEntry.set_status() by showard · 15 years ago
  52. 0db3d43 Recheck queue entry status in Dispatcher._get_unassigned_entries() by showard · 15 years ago
  53. d201482 When a delayed call task finishes waiting for extra hosts to enter by showard · 15 years ago
  54. dae680a Ignore microsecond differences in datetimes when checking existing in by showard · 15 years ago
  55. e55955f Rewrite a conditional that was very confusing to me. by showard · 15 years ago
  56. f85a0b7 Explicitly release pidfiles after we're done with them. This does it in a kind of lazy way, but it should work just fine. Also extended the new scheduler functional test with a few more cases and added a test to check pidfile release under these various cases. In the process, I changed how some of the code works to allow the tests to more cleanly express their intentions. by showard · 15 years ago
  57. ec6a3b9 Make the pidfile timeout in the scheduler configurable. Raise the by showard · 15 years ago
  58. 0c5c18d Changed error message to be more useful by showard · 15 years ago
  59. 828fc4c Make assertion in _choose_group_to_run non-fatal and log an error by showard · 15 years ago
  60. db50276 Write host keyvals for all verify/cleanup/repair tasks. by showard · 15 years ago
  61. 775300b Cleanups on hosts marked DO_NOT_VERIFY should continue to run as if they by showard · 15 years ago
  62. 8cc058f Make scheduler more stateless. Agents are now scheduled only by the by showard · 15 years ago
  63. 8de3713 Renamed process_is_alive to program_is_alive. by showard · 15 years ago
  64. cdaeae8 Fixed bug where scheduler would crash if the autoserv process is lost by showard · 15 years ago
  65. 4ac4754 Don't mark HQEs as Failed before the GatherLogsTask and the by showard · 15 years ago
  66. 549afad Added pid file checks to monitor_db and monitor_db_babysitter, so that by showard · 15 years ago
  67. 70a294f Don't expect aborted "Pending" entries to be recovered. They'll be immediately picked up by _find_aborting() so they don't need to be recovered. by showard · 15 years ago
  68. 58721a8 One-off fix to address the issue where a scheduler shutdown immediately by showard · 15 years ago
  69. 6bba3d1 Don't assert if we were unable to load the pidfile in num_tests_failed. by showard · 15 years ago
  70. e8e3707 Treat unrecoverable host queue entries as a fatal error. Their existance by showard · 15 years ago
  71. 6d1c143 Fix scheduler's handling of jobs when the PID file can't be found. by showard · 15 years ago
  72. 708b352 Do not go through a DelayedCallTask on atomic group jobs when all Hosts by showard · 15 years ago
  73. 9b6ec50 Turn an assertion into a more useful error message. by showard · 15 years ago
  74. 5fa9e11 By default, only warn when orphaned autoservs are found by mbligh · 15 years ago
  75. 6fbdb80 Change print msg to logging.error(msg) so that we actually get the error in the scheduler log about the scheduler not being enalbed. by mbligh · 15 years ago
  76. c6a5687 Remove an assertion error that was preventing recovered atomic group by showard · 15 years ago
  77. f4a2e50 log aborts in the scheduler more explicitly by showard · 15 years ago
  78. a5288b4 Upgrade from Django 0.96 to Django 1.0.2. by showard · 15 years ago
  79. b000a8d Added logging and email code to help track down a bug (asynchronous jobs are by showard · 15 years ago
  80. 6af73ad "Recover" HQEs in "Starting" status by requeuing them. This is what it used to do, but it was lost in the new recovery code. This restores legacy bahavior until we implement proper recovery. by showard · 15 years ago
  81. 6878e8b Never kill processes in scheduler recovery. Instead, consider it an error if any unrecovered orphan process exists. Since we recover special tasks now, we should recover all processes, so if we find any extra, that means something went wrong and it's not safe to continue. by showard · 15 years ago
  82. a640b2d Fix scheduler bug with aborting a pre-job task. Scheduler was by showard · 15 years ago
  83. 8ac6f2a When a SpecialAgentTask is passed an existing SpecialTask, set the _working_directory upon object construction. It was previously set in prolog(), but recovery agents don't run prolog, but they still need _working_directory sometimes (i.e. when a RepairTask fails). by showard · 15 years ago
  84. cfd4a7e With the new SpecialTask recovery code, a RepairTask can be passed a queue entry that was previously requeued. So make sure the task leaves the HQE alone in that case. by showard · 15 years ago
  85. b6681aa SpecialAgentTasks can be aborted if they're tied to a job that gets aborted while they're active. In that case, we still need to update the SpecialTask entry to mark it as complete. by showard · 15 years ago
  86. ed2afea make SpecialTasks recoverable. this involves quite a few changes. by showard · 15 years ago
  87. 6157c63 Make the scheduler robust to finding a HostQueueEntry with more than one by showard · 15 years ago
  88. 2fe3f1d Enter all Verify/Cleanup/Repair tasks into the special_tasks table. Also by showard · 15 years ago
  89. e7d9c60 Make the job executiontag available in both the server and client side job by mbligh · 15 years ago
  90. e9c6936 Pass --verbose flag for verify/repair/cleanup. Since we currently log these via piped console output, we want verbose output. by showard · 15 years ago
  91. b562645 ensure hosts get cleaned up even in the rare but possible case that a QueueTask finds no process at all by showard · 15 years ago
  92. 2924b0a Ensure one-time-hosts aren't in the Everyone ACL, and make the scheduler ignore this. by showard · 15 years ago
  93. cbe6f94 add a log message to the scheduler thats useful for debugging atomic groups by showard · 15 years ago
  94. af8b4ca Fix _atomic_and_has_started() to check *only* for states that are a by showard · 15 years ago
  95. 08356c1 Do not call .set_host if the host is already set. by showard · 15 years ago
  96. 043c62a Ensure all entry points get the import-time logging logic executed before other autotest imports. by showard · 15 years ago
  97. 136e6dc Make scheduler and babysitter use the new logging_manager system. by showard · 15 years ago
  98. 6d7b2ff Redesign the reverify hosts feature. Host status is no longer changed by showard · 15 years ago
  99. 7718256 Have the scheduler wait a configurable amount of time before starting by showard · 15 years ago
  100. 5613c66 Add an option to global config to disable to the scheduler isn't accidentally started on drones. by showard · 15 years ago