Fix vr flinger deadlock and race condition

While investigating hangs when transitioning from 2d --> vr and back I
found a deadlock and race condition in the vr flinger pause/resume
handling. This CL should fix both issues.

Unfortunately there's still another deadlock related to multiple threads
trying to suspend/resume vr flinger, but considering how vr flinger is
currently used by surface flinger we shouldn't ever hit that scenario in
practice.

Bug: None

Test: I was able to reliably get a hang when starting/stopping vr
launcher a few times, but with this CL applied and another CL to remove
calls to SetPowerMode() applied (that CL will be submitted separately),
I no longer see any hangs.

Change-Id: Ie842bf9fb00e4e2937769ed7e1e2ec9cc47861f7
(cherry picked from commit cf921a3919d68d8c8d1b8be39e03a372f6346f57)
diff --git a/libs/vr/libvrflinger/hardware_composer.h b/libs/vr/libvrflinger/hardware_composer.h
index cfe8c84..567af5c 100644
--- a/libs/vr/libvrflinger/hardware_composer.h
+++ b/libs/vr/libvrflinger/hardware_composer.h
@@ -191,7 +191,6 @@
 
   bool Suspend();
   bool Resume();
-  bool IsSuspended() const { return pause_post_thread_; }
 
   // Get the HMD display metrics for the current display.
   DisplayMetrics GetHmdDisplayMetrics() const;
@@ -263,7 +262,7 @@
   void PostThread();
 
   int ReadWaitPPState();
-  int BlockUntilVSync();
+  int BlockUntilVSync(/*out*/ bool* suspend_requested);
   int ReadVSyncTimestamp(int64_t* timestamp);
   int WaitForVSync(int64_t* timestamp);
   int SleepUntil(int64_t wakeup_timestamp);
@@ -303,8 +302,6 @@
 
   void HandlePendingScreenshots();
 
-  void PausePostThread();
-
   // Hardware composer HAL device.
   std::unique_ptr<Hwc2::Composer> hwc2_hidl_;
   sp<ComposerCallback> callbacks_;
@@ -347,16 +344,35 @@
   // Handler to hook vsync events outside of this class.
   VSyncCallback vsync_callback_;
 
-  // Thread and condition for managing the layer posting thread. This thread
-  // wakes up a short time before vsync to hand buffers to post processing and
-  // the results to hardware composer.
+  // The layer posting thread. This thread wakes up a short time before vsync to
+  // hand buffers to post processing and the results to hardware composer.
   std::thread post_thread_;
 
+  enum class PostThreadState {
+    // post_thread_state_ starts off paused. When suspending, the control thread
+    // will block until post_thread_state_ == kPaused, indicating the post
+    // thread has completed the transition to paused (most importantly: no more
+    // hardware composer calls).
+    kPaused,
+    // post_thread_state_ is set to kRunning by the control thread (either
+    // surface flinger's main thread or the vr flinger dispatcher thread). The
+    // post thread blocks until post_thread_state_ == kRunning.
+    kRunning,
+    // Set by the control thread to indicate the post thread should pause. The
+    // post thread will change post_thread_state_ from kPauseRequested to
+    // kPaused when it stops.
+    kPauseRequested
+  };
   // Control variables to control the state of the post thread
+  PostThreadState post_thread_state_;
+  // Used to wake the post thread up while it's waiting for vsync, for faster
+  // transition to the paused state.
   pdx::LocalHandle terminate_post_thread_event_fd_;
-  bool pause_post_thread_;
-  std::mutex thread_pause_mutex_;
-  std::condition_variable thread_pause_semaphore_;
+  // post_thread_state_mutex_ should be held before checking or modifying
+  // post_thread_state_.
+  std::mutex post_thread_state_mutex_;
+  // Used to communicate between the control thread and the post thread.
+  std::condition_variable post_thread_state_cond_var_;
 
   // Backlight LED brightness sysfs node.
   pdx::LocalHandle backlight_brightness_fd_;