src/devices/graphics/architecture.jd

page.title=Graphics architecture
@jd:body

<!--
    Copyright 2014 The Android Open Source Project

    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.
    You may obtain a copy of the License at

        http://www.apache.org/licenses/LICENSE-2.0

    Unless required by applicable law or agreed to in writing, software
    distributed under the License is distributed on an "AS IS" BASIS,
    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    See the License for the specific language governing permissions and
    limitations under the License.
-->
<div id="qv-wrapper">
  <div id="qv">
    <h2>In this document</h2>
    <ol id="auto-toc">
    </ol>
  </div>
</div>


<p><em>What every developer should know about Surface, SurfaceHolder,
EGLSurface, SurfaceView, GLSurfaceView, SurfaceTexture, TextureView,
SurfaceFlinger, and Vulkan.</em></p>

<p>This page describes essential elements of the Android system-level graphics
architecture and how they are used by the application framework and multimedia
system. The focus is on how buffers of graphical data move through the system.
If you've ever wondered why SurfaceView and TextureView behave the way they do,
or how Surface and EGLSurface interact, you are in the correct place.</p>

<p>Some familiarity with Android devices and application development is assumed.
You don't need detailed knowledge of the app framework and very few API calls
are mentioned, but the material doesn't overlap with other public
documentation. The goal is to provide details on the significant events
involved in rendering a frame for output to help you make informed choices
when designing an application. To achieve this, we work from the bottom up,
describing how the UI classes work rather than how they can be used.</p>

<p>This section includes several pages covering everything from background
material to HAL details to use cases. It starts with an explanation of Android
graphics buffers, describes the composition and display mechanism, then proceeds
to the higher-level mechanisms that supply the compositor with data. We
recommend reading pages in the order listed below rather than skipping to a
topic that sounds interesting.</p>

<h2 id=low_level>Low-level components</h2>

<ul>
<li><a href="{@docRoot}devices/graphics/arch-bq-gralloc.html">BufferQueue and
gralloc</a>. BufferQueue connects something that generates buffers of graphical
data (the <em>producer</em>) to something that accepts the data for display or
further processing (the <em>consumer</em>). Buffer allocations are performed
through the <em>gralloc</em> memory allocator implemented through a
vendor-specific HAL interface.</li>

<li><a href="{@docRoot}devices/graphics/arch-sf-hwc.html">SurfaceFlinger,
Hardware Composer, and virtual displays</a>. SurfaceFlinger accepts buffers of
data from multiple sources, composites them, and sends them to the display. The
Hardware Composer HAL (HWC) determines the most efficient way to composite
buffers with the available hardware, and virtual displays make composited output
available within the system (recording the screen or sending the screen over a
network).</li>

<li><a href="{@docRoot}devices/graphics/arch-sh.html">Surface, Canvas, and
SurfaceHolder</a>. A Surface produces a buffer queue that is often consumed by
SurfaceFlinger. When rendering onto a Surface, the result ends up in a buffer
that gets shipped to the consumer. Canvas APIs provide a software implementation
(with hardware-acceleration support) for drawing directly on a Surface
(low-level alternative to OpenGL ES). Anything having to do with a View involves
a SurfaceHolder, whose APIs enable getting and setting Surface parameters such
as size and format.</li>

<li><a href="{@docRoot}devices/graphics/arch-egl-opengl.html">EGLSurface and
OpenGL ES</a>. OpenGL ES (GLES) defines a graphics-rendering API designed to be
combined with EGL, a library that knows how to create and access windows through
the operating system (to draw textured polygons, use GLES calls; to put
rendering on the screen, use EGL calls). This page also covers ANativeWindow,
the C/C++ equivalent of the Java Surface class used to create an EGL window
surface from native code.</li>

<li><a href="{@docRoot}devices/graphics/arch-vulkan.html">Vulkan</a>. Vulkan is
a low-overhead, cross-platform API for high-performance 3D graphics. Like OpenGL
ES, Vulkan provides tools for creating high-quality, real-time graphics in
applications. Vulkan advantages include reductions in CPU overhead and support
for the <a href="https://www.khronos.org/spir">SPIR-V Binary Intermediate</a>
language.</li>

</ul>

<h2 id=high_level>High-level components</h2>

<ul>
<li><a href="{@docRoot}devices/graphics/arch-sv.html">SurfaceView and
GLSurfaceView</a>. SurfaceView combines a Surface and a View. SurfaceView's View
components are composited by SurfaceFlinger (and not the app), enabling
rendering from a separate thread/process and isolation from app UI rendering.
GLSurfaceView provides helper classes to manage EGL contexts, inter-thread
communication, and interaction with the Activity lifecycle (but is not required
to use GLES).</li>

<li><a href="{@docRoot}devices/graphics/arch-st.html">SurfaceTexture</a>.
SurfaceTexture combines a Surface and GLES texture to create a BufferQueue for
which your app is the consumer. When a producer queues a new buffer, it notifies
your app, which in turn releases the previously-held buffer, acquires the new
buffer from the queue, and makes EGL calls to make the buffer available to GLES
as an external texture. Android 7.0 adds support for secure texture video
playback enabling GPU post-processing of protected video content.</li>

<li><a href="{@docRoot}devices/graphics/arch-tv.html">TextureView</a>.
TextureView combines a View with a SurfaceTexture. TextureView wraps a
SurfaceTexture and takes responsibility for responding to callbacks and
acquiring new buffers. When drawing, TextureView uses the contents of the most
recently received buffer as its data source, rendering wherever and however the
View state indicates it should. View composition is always performed with GLES,
meaning updates to contents may cause other View elements to redraw as well.</li>
</ul>