Add more runtime options.

Changed HeapGCThreads to be split into two different options:
-XX:ParallelGCThreads: Which specifies how many threads the GC may
use when the mutators are suspended.

-XX:ConcGCThreads: Which specifies how many threads the GC may use
when the mutators are running.

Added runtime options to specify long pause / long GC thresholds:
-XX:LongPauseThreshold (default 5ms)
-XX:LongGCThreshold (default 100ms)
These thresholds were previously constants, but are now runtime
options. If we exceed either of the thresholds, we print the GC
message.

Added a new runtime option: -XX:IgnoreMaxFootprint which makes it
that the GC only does GC when the number of bytes allocated hits
the growth limit. This causes GC to occur much less frequently and
can be useful to measure how much of an impact GC has on performance.

Changed the GC behaviour to use only one thread when we do not care
about pauses to prevent jank that can be caused by 2 simultaneous GC
on different processes fighting for CPU time.

Added thread pool functionality for changing the maximum number of
active workers.

Fixed an accounting error where we didn't count large objects in the
total freed.

Bug: 9986416

Change-Id: I86afa358d93dcd3780e18ac5d85bdb1a130cb7e7
diff --git a/runtime/thread_pool.cc b/runtime/thread_pool.cc
index 39d30bb2..674ab9d 100644
--- a/runtime/thread_pool.cc
+++ b/runtime/thread_pool.cc
@@ -81,7 +81,8 @@
     start_time_(0),
     total_wait_time_(0),
     // Add one since the caller of constructor waits on the barrier too.
-    creation_barier_(num_threads + 1) {
+    creation_barier_(num_threads + 1),
+    max_active_workers_(num_threads) {
   Thread* self = Thread::Current();
   while (GetThreadCount() < num_threads) {
     const std::string name = StringPrintf("Thread pool worker %zu", GetThreadCount());
@@ -91,6 +92,12 @@
   creation_barier_.Wait(self);
 }
 
+void ThreadPool::SetMaxActiveWorkers(size_t threads) {
+  MutexLock mu(Thread::Current(), task_queue_lock_);
+  CHECK_LE(threads, GetThreadCount());
+  max_active_workers_ = threads;
+}
+
 ThreadPool::~ThreadPool() {
   {
     Thread* self = Thread::Current();
@@ -121,12 +128,18 @@
 Task* ThreadPool::GetTask(Thread* self) {
   MutexLock mu(self, task_queue_lock_);
   while (!IsShuttingDown()) {
-    Task* task = TryGetTaskLocked(self);
-    if (task != NULL) {
-      return task;
+    const size_t thread_count = GetThreadCount();
+    // Ensure that we don't use more threads than the maximum active workers.
+    const size_t active_threads = thread_count - waiting_count_;
+    // <= since self is considered an active worker.
+    if (active_threads <= max_active_workers_) {
+      Task* task = TryGetTaskLocked(self);
+      if (task != NULL) {
+        return task;
+      }
     }
 
-    waiting_count_++;
+    ++waiting_count_;
     if (waiting_count_ == GetThreadCount() && tasks_.empty()) {
       // We may be done, lets broadcast to the completion condition.
       completion_condition_.Broadcast(self);