Docs: HWC2 updates, new section+subsections
Removing Framebuffer bullet
Making OpenGL 3.x drivers optional
Fixing horrendous typo in nav
Adding Clay's feedback
Bug: 28419158
Change-Id: I19ce49d22f7bdb54229363580d9075f71037ef9c
diff --git a/src/devices/graphics/implement-hwc.jd b/src/devices/graphics/implement-hwc.jd
new file mode 100644
index 0000000..63387d9
--- /dev/null
+++ b/src/devices/graphics/implement-hwc.jd
@@ -0,0 +1,318 @@
+page.title=Implementing the Hardware Composer HAL
+@jd:body
+
+<!--
+ Copyright 2016 The Android Open Source Project
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+<div id="qv-wrapper">
+ <div id="qv">
+ <h2>In this document</h2>
+ <ol id="auto-toc">
+ </ol>
+ </div>
+</div>
+
+
+<p>The Hardware Composer HAL (HWC) is used by SurfaceFlinger to composite
+surfaces to the screen. The HWC abstracts objects such as overlays and 2D
+blitters and helps offload some work that would normally be done with OpenGL.</p>
+
+<p>Android 7.0 includes a new version of HWC (HWC2) used by SurfaceFlinger to
+talk to specialized window composition hardware. SurfaceFlinger contains a
+fallback path that uses the 3D graphics processor (GPU) to perform the task of
+window composition, but this path is not ideal for a couple of reasons:</p>
+
+<ul>
+ <li>Typically, GPUs are not optimized for this use case and may use more power
+ than necessary to perform composition.</li>
+ <li>Any time SurfaceFlinger is using the GPU for composition is time that
+ applications cannot use the processor for their own rendering, so it is
+ preferable to use specialized hardware for composition instead of the GPU
+ whenever possible.</li>
+</ul>
+
+<h2 id="guidance">General guidance</h2>
+
+<p>As the physical display hardware behind the Hardware Composer abstraction
+layer can vary from device to device, it's difficult to give recommendations on
+specific features. In general, use the following guidance:</p>
+
+<ul>
+ <li>The HWC should support at least four overlays (status bar, system bar,
+ application, and wallpaper/background).</li>
+ <li>Layers can be bigger than the screen, so the HWC should be able to handle
+ layers that are larger than the display (for example, a wallpaper).</li>
+ <li>Pre-multiplied per-pixel alpha blending and per-plane alpha blending
+ should be supported at the same time.</li>
+ <li>The HWC should be able to consume the same buffers the GPU, camera, and
+ video decoder are producing, so supporting some of the following
+ properties is helpful:
+ <ul>
+ <li>RGBA packing order</li>
+ <li>YUV formats</li>
+ <li>Tiling, swizzling, and stride properties</li>
+ </ul>
+ <li>To support protected content, a hardware path for protected video playback
+ must be present.</li>
+ </ul>
+
+<p>The general recommendation is to implement a non-operational HWC first; after
+the structure is complete, implement a simple algorithm to delegate composition
+to the HWC (for example, delegate only the first three or four surfaces to the
+overlay hardware of the HWC).</p>
+
+<p>Focus on optimization, such as intelligently selecting the surfaces to send
+to the overlay hardware that maximizes the load taken off of the GPU. Another
+optimization is to detect whether the screen is updating; if it isn't, delegate
+composition to OpenGL instead of the HWC to save power. When the screen updates
+again, continue to offload composition to the HWC.</p>
+
+<p>Prepare for common use cases, such as:</p>
+
+<ul>
+ <li>Full-screen games in portrait and landscape mode</li>
+ <li>Full-screen video with closed captioning and playback control</li>
+ <li>The home screen (compositing the status bar, system bar, application
+ window, and live wallpapers)</li>
+ <li>Protected video playback</li>
+ <li>Multiple display support</li>
+</ul>
+
+<p>These use cases should address regular, predictable uses rather than edge
+cases that are rarely encountered (otherwise, optimizations will have little
+benefit). Implementations must balance two competing goals: animation smoothness
+and interaction latency.</p>
+
+
+<h2 id="interface_activities">HWC2 interface activities</h2>
+
+<p>HWC2 provides a few primitives (layer, display) to represent composition work
+and its interaction with the display hardware.</p>
+<p>A <em>layer</em> is the most important unit of composition; every layer has a
+set of properties that define how it interacts with other layers. Property
+categories include the following:</p>
+
+<ul>
+<li><strong>Positional</strong>. Defines where the layer appears on its display.
+Includes information such as the positions of a layer's edges and its <em>Z
+order</em> relative to other layers (whether it should be in front of or behind
+other layers).</li>
+<li><strong>Content</strong>. Defines how content displayed on the layer should
+be presented within the bounds defined by the positional properties. Includes
+information such as crop (to expand a portion of the content to fill the bounds
+of the layer) and transform (to show rotated or flipped content).</li>
+<li><strong>Composition</strong>. Defines how the layer should be composited
+with other layers. Includes information such as blending mode and a layer-wide
+alpha value for
+<a href="https://en.wikipedia.org/wiki/Alpha_compositing#Alpha_blending">alpha
+compositing</a>.</li>
+<li><strong>Optimization</strong>. Provides information not strictly necessary
+to correctly composite the layer, but which can be used by the HWC device to
+optimize how it performs composition. Includes information such as the visible
+region of the layer and which portion of the layer has been updated since the
+previous frame.</li>
+</ul>
+
+<p>A <em>display</em> is another important unit of composition. Every layer can
+be present only on one display. A system can have multiple displays, and
+displays can be added or removed during normal system operations. This
+addition/removal can come at the request of the HWC device (typically in
+response to an external display being plugged into or removed from the device,
+called <em>hotplugging</em>), or at the request of the client, which permits the
+creation of <em>virtual displays</em> whose contents are rendered into an
+off-screen buffer instead of to a physical display.</p>
+<p>HWC2 provides functions to determine the properties of a given display, to
+switch between different configurations (e.g., 4k or 1080p resolution) and color
+modes (e.g., native color or true sRGB), and to turn the display on, off, or
+into a low-power mode if supported.</p>
+<p>In addition to layers and displays, HWC2 also provides control over the
+hardware vertical sync (VSYNC) signal along with a callback into the client to
+notify it of when a vsync event has occurred.</p>
+
+<h3 id="func_pointers">Function pointers</h3>
+<p>In this section and in HWC2 header comments, HWC interface functions are
+referred to by lowerCamelCase names that do not actually exist in the interface
+as named fields. Instead, almost every function is loaded by requesting a
+function pointer using <code>getFunction</code> provided by
+<code>hwc2_device_t</code>. For example, the function <code>createLayer</code>
+is a function pointer of type <code>HWC2_PFN_CREATE_LAYER</code>, which is
+returned when the enumerated value <code>HWC2_FUNCTION_CREATE_LAYER</code> is
+passed into <code>getFunction</code>.</p>
+<p>For detailed documentation on functions (including functions required for
+every HWC2 implementation), refer to the HWC2 header.</p>
+
+<h3 id="layer_display_handles">Layer and display handles</h3>
+<p>Layers and displays are manipulated by opaque handles.</p>
+<p>When SurfaceFlinger wants to create a new layer, it calls the
+<code>createLayer</code> function, which then returns an opaque handle of type
+<code>hwc2_layer_t</code>. From that point on, any time SurfaceFlinger wants to
+modify a property of that layer, it passes that <code>hwc2_layer_t</code> value
+into the appropriate modification function, along with any other information
+needed to make the modification. The <code>hwc2_layer_t</code> type was made
+large enough to be able to hold either a pointer or an index, and it will be
+treated as opaque by SurfaceFlinger to provide HWC implementers maximum
+flexibility.</p>
+<p>Most of the above also applies to display handles, though handles are created
+differently depending on whether they are hotplugged (where the handle is passed
+through the hotplug callback) or requested by the client as a virtual display
+(where the handle is returned from <code>createVirtualDisplay</code>).</p>
+
+<h2 id="display_comp_ops">Display composition operations</h2>
+<p>Once per hardware vsync, SurfaceFlinger wakes if it has new content to
+composite. This new content could be new image buffers from applications or just
+a change in the properties of one or more layers. When it wakes, it performs the
+following steps:</p>
+
+<ol>
+<li>Apply transactions, if present. Includes changes in the properties of layers
+specified by the window manager but not changes in the contents of layers (i.e.,
+graphic buffers from applications).</li>
+<li>Latch new graphic buffers (acquire their handles from their respective
+applications), if present.</li>
+<li>If step 1 or 2 resulted in a change to the display contents, perform a new
+composition (described below).</li>
+</ol>
+
+<p>Steps 1 and 2 have some nuances (such as deferred transactions and
+presentation timestamps) that are outside the scope of this section. However,
+step 3 involves the HWC interface and is detailed below.</p>
+<p>At the beginning of the composition process, SurfaceFlinger will create and
+destroy layers or modify layer state as applicable. It will also update the
+layers with their current contents, using calls such as
+<code>setLayerBuffer</code> or <code>setLayerColor</code>. After all layers have
+been updated, it will call <code>validateDisplay</code>, which tells the device
+to examine the state of the various layers and determine how composition will
+proceed. By default, SurfaceFlinger usually attempts to configure every layer
+such that it will be composited by the device, though there may be some
+circumstances where it will mandate that it be composited by the client.</p>
+<p>After the call to <code>validateDisplay</code>, SurfaceFlinger will follow up
+with a call to <code>getChangedCompositionTypes</code> to see if the device
+wants any of the layers' composition types changed before performing the actual
+composition. SurfaceFlinger may choose to:</p>
+
+<ul>
+<li>Change some of the layer composition types and re-validate the display.</li>
+</ul>
+
+<blockquote><strong><em>OR</strong></em></blockquote>
+
+<ul>
+<li>Call <code>acceptDisplayChanges</code>, which has the same effect as
+changing the composition types as requested by the device and re-validating
+without actually having to call <code>validateDisplay</code> again.</li>
+</ul>
+
+<p>In practice, SurfaceFlinger always takes the latter path (calling
+<code>acceptDisplayChanges</code>) though this behavior may change in the
+future.</p>
+<p>At this point, the behavior differs depending on whether any of the layers
+have been marked for client composition. If any (or all) layers have been marked
+for client composition, SurfaceFlinger will now composite all of those layers
+into the client target buffer. This buffer will be provided to the device using
+the <code>setClientTarget</code> call so that it may be either displayed
+directly on the screen or further composited with layers that have not been
+marked for client composition. If no layers have been marked for client
+composition, then the client composition step is bypassed.</p>
+<p>Finally, after all of the state has been validated and client composition has
+been performed if needed, SurfaceFlinger will call <code>presentDisplay</code>.
+This is the HWC device's cue to complete the composition process and display the
+final result.</p>
+
+<h2 id="multiple_displays">Multiple displays in Android N</h2>
+<p>While the HWC2 interface is quite flexible when it comes to the number of
+displays in the system, the rest of the Android framework is not yet as
+flexible. When designing a HWC2 implementation intended for use on Android N,
+there are some additional restrictions not present in the HWC definition itself:
+</p>
+
+<ul>
+<li>It is assumed that there is exactly one <em>primary</em> display; that is,
+that there is one physical display that will be hotplugged immediately during
+the initialization of the device (specifically after the hotplug callback is
+registered).</li>
+<li>In addition to the primary display, exactly one <em>external</em> display
+may be hotplugged during normal operation of the device.</li>
+</ul>
+
+<p>While the SurfaceFlinger operations described above are performed per-display
+(eventual goal is to be able to composite displays independently of each other),
+they are currently performed sequentially for all active displays, even if only
+the contents of one display are updated.</p>
+<p>For example, if only the external display is updated, the sequence is:</p>
+
+<pre>
+// Update state for internal display
+// Update state for external display
+validateDisplay(<internal display>)
+validateDisplay(<external display>)
+presentDisplay(<internal display>)
+presentDisplay(<external display>)
+</pre>
+
+
+<h2 id="sync_fences">Synchronization fences</h2>
+<p>Synchronization (sync) fences are a crucial aspect of the Android graphics
+system. Fences allow CPU work to proceed independently from concurrent GPU work,
+blocking only when there is a true dependency.</p>
+<p>For example, when an application submits a buffer that is being produced on
+the GPU, it will also submit a fence object; this fence signals only when the
+GPU has finished writing into the buffer. Since the only part of the system that
+truly needs the GPU write to have finished is the display hardware (the hardware
+abstracted by the HWC HAL), the graphics pipeline is able to pass this fence
+along with the buffer through SurfaceFlinger to the HWC device. Only immediately
+before that buffer would be displayed does the device need to actually check
+that the fence has signaled.</p>
+<p>Sync fences are integrated tightly into HWC2 and organized in the following
+categories:</p>
+
+<ol>
+<li>Acquire fences are passed along with input buffers to the
+<code>setLayerBuffer</code> and <code>setClientTarget</code> calls. These
+represent a pending write into the buffer and must signal before the HWC client
+or device attempts to read from the associated buffer to perform composition.
+</li>
+<li>Release fences are retrieved after the call to <code>presentDisplay</code>
+using the <code>getReleaseFences</code> call and are passed back to the
+application along with buffers that will be replaced during the next
+composition. These represent a pending read from the buffer, and must signal
+before the application attempts to write new contents into the buffer.</li>
+<li>Retire fences are returned, one per frame, as part of the call to
+<code>presentDisplay</code> and represent when the composition of this frame
+has completed, or alternately, when the composition result of the prior frame is
+no longer needed. For physical displays, this is when the current frame appears
+on the screen and can also be interpreted as the time after which it is safe to
+write to the client target buffer again (if applicable). For virtual displays,
+this is the time when it is safe to read from the output buffer.</li>
+</ol>
+
+<h3 id="hwc2_changes">Changes in HWC2</h3>
+<p>The meaning of sync fences in HWC 2.0 has changed significantly relative to
+previous versions of the HAL.</p>
+<p>In HWC v1.x, the release and retire fences were speculative. A release fence
+for a buffer or a retire fence for the display retrieved in frame N would not
+signal any sooner than frame N + 1. In other words, the meaning of the fence
+was "the content of the buffer you provided for frame N is no longer needed."
+This is speculative because in theory SurfaceFlinger may not run again after
+frame N for an indeterminate period of time, which would leave those fences
+unsignaled for the same period.</p>
+<p>In HWC 2.0, release and retire fences are non-speculative. A release or
+retire fence retrieved in frame N will signal as soon as the content of the
+associated buffers replaces the contents of the buffers from frame N - 1, or in
+other words, the meaning of the fence is "the content of the buffer you provided
+for frame N has now replaced the previous content." This is non-speculative,
+since this fence should signal shortly after <code>presentDisplay</code> is
+called as soon as the hardware presents this frame's content.</p>
+<p>For implementation details, refer to the HWC2 header.</p>