docs/html/ndk/guides/audio/output-latency.jd

page.title=Audio Output Latency
@jd:body

<div id="qv-wrapper">
    <div id="qv">
      <h2>On this page</h2>

      <ol>
        <li><a href="#prereq">Prerequisites</a></li>
        <li><a href="#low-lat-track">Obtain a Low-Latency Track</a></li>
        <li><a href="#buffer-size">Use the Optimal Buffer Size When Enqueuing Audio Data</a></li>
        <li><a href="#warmup-lat">Avoid Warmup Latency</a></li>
      </ol>
      <h2>Also read</h2>

      <ol>
        <li><a href="https://source.android.com/devices/audio/latency_app.html" class="external-link">
        Audio Latency for App Developers</a></li>
        <li><a href="https://source.android.com/devices/audio/latency_contrib.html" class="external-link">
        Contributors to Audio Latency</a></li>
        <li><a href="https://source.android.com/devices/audio/latency_measure.html" class="external-link">
        Measuring Audio Latency</a></li>
        <li><a href="https://source.android.com/devices/audio/warmup.html" class="external-link">
        Audio Warmup</a></li>
        <li><a href="https://en.wikipedia.org/wiki/Latency_%28audio%29" class="external-link">
        Latency (audio)</a></li>
        <li><a href="https://en.wikipedia.org/wiki/Round-trip_delay_time" class="external-link">
        Round-trip delay time</a></li>
      </ol>
    </div>
  </div>

<a href="https://www.youtube.com/watch?v=PnDK17zP9BI" class="notice-developers-video">
<div>
    <h3>Video</h3>
    <p>Audio latency: buffer sizes</p>
</div>
</a>

<a href="https://www.youtube.com/watch?v=92fgcUNCHic" class="notice-developers-video">
<div>
    <h3>Video</h3>
    <p>Building great multi-media experiences on Android</p>
</div>
</a>

<p>This page describes how to develop your audio app for low-latency output and how to avoid
warmup latency.</p>

<h2 id="prereq">Prerequisites</h2>

<p>Low-latency audio is currently only supported when using Android's implementation of the
OpenSL ES™ API specification, and the Android NDK:
</p>

<ol>
  <li>Download and install the <a href="{@docRoot}tools/sdk/ndk/index.html">Android NDK</a>.</li>
  <li>Read the <a href="{@docRoot}ndk/guides/audio/opensl-for-android.html">OpenSL ES
  documentation</a>.
</ol>

<h2 id="low-lat-track">Obtain a Low-Latency Track</h2>

<p>Latency is the time it takes for a signal to travel through a system.  These are the common
types of latency related to audio apps:

<ul>
  <li><strong>Audio output latency</strong> is the time between an audio sample being generated by an
app and the sample being played through the headphone jack or built-in speaker.</li>

  <li><strong>Audio input latency</strong> is the time between an audio signal being received by a
device’s audio input, such as the microphone, and that same audio data being available to an
app.</li>

  <li><strong>Round-trip latency</strong> is the sum of input latency, app processing time, and
  output latency.</li>

  <li><strong>Touch latency</strong> is the time between a user touching the screen and that
touch event being received by an app.</li>
</ul>

<p>It is difficult to test audio output latency in isolation since it requires knowing exactly
when the first sample is sent into the audio path (although this can be done using a
<a href="https://source.android.com/devices/audio/testing_circuit.html" class="external-link">
light testing circuit</a> and an oscilloscope). If you know the round-trip audio latency, you can
use the rough rule of thumb: <strong>audio output latency is half the round-trip audio latency
over paths without signal processing</strong>.
</p>

<p>To obtain the lowest latency, you must supply audio data that matches the device's optimal
sample rate and buffer size. For more information, see
<a href="https://source.android.com/devices/audio/latency_design.html" class="external-link">
Design For Reduced Latency</a>.</p>

<h3>Obtain the optimal sample rate</h3>

<p>In Java, you can obtain the optimal sample rate from AudioManager as shown in the following
code example:</p>

<pre>
AudioManager am = (AudioManager) getSystemService(Context.AUDIO_SERVICE);
String frameRate = am.getProperty(AudioManager.PROPERTY_OUTPUT_SAMPLE_RATE);
int frameRateInt = Integer.parseInt(frameRate);
if (frameRateInt == 0) frameRateInt = 44100; // Use a default value if property not found
</pre>

<p class="note">
  <strong>Note:</strong> The sample rate refers to the rate of each stream. If your source audio
  has two channels (stereo), then you will have one stream playing a pair of samples (frame) at
  <a href="{@docRoot}reference/android/media/AudioManager.html#PROPERTY_OUTPUT_SAMPLE_RATE">
  PROPERTY_OUTPUT_SAMPLE_RATE</a>.
</p>

<h3>Use the optimal sample rate when creating your audio player</h3>

<p>Once you have the optimal sample output rate, you can supply it when creating your player
using OpenSL ES:</p>

<pre>
// create buffer queue audio player
void Java_com_example_audio_generatetone_MainActivity_createBufferQueueAudioPlayer
        (JNIEnv* env, jclass clazz, jint sampleRate, jint framesPerBuffer)
{
   ...
   // specify the audio source format
   SLDataFormat_PCM format_pcm;
   format_pcm.numChannels = 2;
   format_pcm.samplesPerSec = (SLuint32) sampleRate * 1000;
   ...
}
</pre>

<p class="note">
  <strong>Note:</strong> {@code samplesPerSec} refers to the <em>sample rate per channel in
  millihertz</em> (1 Hz = 1000 mHz).
</p>

<h3>Avoid adding output interfaces that involve signal processing</h3>

<p>Only these interfaces are supported by the fast mixer:</p>

<ul>
  <li>SL_IID_ANDROIDSIMPLEBUFFERQUEUE</li>
  <li>SL_IID_VOLUME</li>
  <li>SL_IID_MUTESOLO</li>
</ul>

<p>These interfaces are not allowed because they involve signal processing and will cause
your request for a fast-track to be rejected:</p>

<ul>
  <li>SL_IID_BASSBOOST</li>
  <li>SL_IID_EFFECTSEND</li>
  <li>SL_IID_ENVIRONMENTALREVERB</li>
  <li>SL_IID_EQUALIZER</li>
  <li>SL_IID_PLAYBACKRATE</li>
  <li>SL_IID_PRESETREVERB</li>
  <li>SL_IID_VIRTUALIZER</li>
  <li>SL_IID_ANDROIDEFFECT</li>
  <li>SL_IID_ANDROIDEFFECTSEND</li>
</ul>

<p>When you create your player, make sure you only add <em>fast</em> interfaces, as shown in
the following example:</p>

<pre>
const SLInterfaceID interface_ids[2] = { SL_IID_ANDROIDSIMPLEBUFFERQUEUE, SL_IID_VOLUME };
</pre>

<h3>Verify you're using a low-latency track</h3>

<p>Complete these steps to verify that you have successfully obtained a low-latency track:</p>

<ol>
  <li>Launch your app and then run the following command:</li>

<pre>
adb shell ps | grep your_app_name
</pre>

  <li>Make a note of your app's process ID.</li>

  <li>Now, play some audio from your app. You have approximately three seconds to run the
following command from the terminal:</li>

<pre>
adb shell dumpsys media.audio_flinger
</pre>

  <li>Scan for your process ID. If you see an <em>F</em> in the <em>Name</em> column, it's on a
low-latency track (the F stands for <em>fast track</em>).</li>

</ol>

<h3>Measure round-trip latency</h3>

<p>You can measure round-trip audio latency by creating an app that generates an audio signal,
listens for that signal, and measures the time between sending it and receiving it.
Alternatively, you can install this
<a href="https://play.google.com/store/apps/details?id=org.drrickorang.loopback" class="external-link">
latency testing app</a>. This performs a round-trip latency test using the
<a href="https://source.android.com/devices/audio/latency_measure.html#larsenTest" class="external-link">
Larsen test</a>. You can also
<a href="https://github.com/gkasten/drrickorang/tree/master/LoopbackApp" class="external-link">
view the source code</a> for the latency testing app.</p>

<p>Since the lowest latency is achieved over audio paths with minimal signal processing, you may
also want to use an
<a href="https://source.android.com/devices/audio/latency_measure.html#loopback" class="external-link">
Audio Loopback Dongle</a>, which allows the test to be run over the headset connector.</p>

<p>The lowest possible round-trip audio latency varies greatly depending on device model and
Android build. You can measure it yourself using the latency testing app and loopback
dongle. When creating apps for <em>Nexus devices</em>, you can also use the
<a href="https://source.android.com/devices/audio/latency_measurements.html" class="external-link">
published measurements</a>.</p>

<p>You can also get a rough idea of audio performance by testing whether the device reports
support for the
<a href="http://developer.android.com/reference/android/content/pm/PackageManager.html#FEATURE_AUDIO_LOW_LATENCY">
low_latency</a> and
<a href="http://developer.android.com/reference/android/content/pm/PackageManager.html#FEATURE_AUDIO_PRO">
pro</a> hardware features.</p>

<h3>Review the CDD and audio latency</h3>

<p>The Android Compatibility Definition Document (CDD) enumerates the hardware and software
requirements of a compatible Android device.
See <a href="https://source.android.com/compatibility/" class="external-link">
Android Compatibility</a> for more information on the overall compatibility program, and
<a href="https://static.googleusercontent.com/media/source.android.com/en//compatibility/android-cdd.pdf" class="external-link">
CDD</a> for the actual CDD document.</p>

<p>In the CDD, round-trip latency is specified as 20&nbsp;ms or lower (even though musicians
generally require 10&nbsp;ms). This is because there are important use cases that are enabled by
20&nbsp;ms.</p>

<p>There is currently no API to determine audio latency over any path on an Android device at
runtime. You can, however, use the following hardware feature flags to find out whether the
device makes any guarantees for latency:</p>

<ul>
  <li><a href="http://developer.android.com/reference/android/content/pm/PackageManager.html#FEATURE_AUDIO_LOW_LATENCY">
android.hardware.audio.low_latency</a> indicates a continuous output latency of 45&nbsp;ms or
less.</li>

  <li><a href="http://developer.android.com/reference/android/content/pm/PackageManager.html#FEATURE_AUDIO_PRO">
android.hardware.audio.pro</a> indicates a continuous round-trip latency of 20&nbsp;ms or
less.</li>
</ul>

<p>The criteria for reporting these flags is defined in the CDD in sections <em>5.6 Audio
Latency</em> and <em>5.10 Professional Audio</em>.</p>

<p>Here’s how to check for these features in Java:</p>

<pre>
boolean hasLowLatencyFeature =
    getPackageManager().hasSystemFeature(PackageManager.FEATURE_AUDIO_LOW_LATENCY);

boolean hasProFeature =
    getPackageManager().hasSystemFeature(PackageManager.FEATURE_AUDIO_PRO);
</pre>

<p>Regarding the relationship of audio features, the {@code android.hardware.audio.low_latency}
feature is a prerequisite for {@code android.hardware.audio.pro}. A device can implement
{@code android.hardware.audio.low_latency} and not {@code android.hardware.audio.pro}, but not
vice-versa.</p>

<h2 id="buffer-size">Use the Optimal Buffer Size When Enqueuing Audio Data</h2>

<p>You can obtain the optimal buffer size in a similar way to the optimal frame rate, using the
AudioManager API:</p>

<pre>
AudioManager am = (AudioManager) getSystemService(Context.AUDIO_SERVICE);
String framesPerBuffer = am.getProperty(AudioManager.PROPERTY_OUTPUT_FRAMES_PER_BUFFER);
int framesPerBufferInt = Integer.parseInt(framesPerBuffer);
if (framesPerBufferInt == 0) framesPerBufferInt = 256; // Use default
</pre>

<p>The
<a href="{@docRoot}reference/android/media/AudioManager.html#PROPERTY_OUTPUT_FRAMES_PER_BUFFER">
PROPERTY_OUTPUT_FRAMES_PER_BUFFER</a> property indicates the number of audio frames
that the HAL (Hardware Abstraction Layer) buffer can hold. You should construct your audio
buffers so that they contain an exact multiple of this number. If you use the correct number
of audio frames, your callbacks occur at regular intervals, which reduces jitter.</p>

<p>It is important to use the API to determine buffer size rather than using a hardcoded value,
 because HAL buffer sizes differ across devices and across Android builds.</p>

<h2 id="warmup-lat">Avoid Warmup Latency</h2>

<p>When you enqueue audio data for the first time, it takes a small, but still significant,
amount of time for the device audio circuit to warm up. To avoid this warmup latency, you should
enqueue buffers of audio data containing silence, as shown in the following code example:</p>

<pre>
#define CHANNELS 1
static short* silenceBuffer;
int numSamples = frames * CHANNELS;
silenceBuffer = malloc(sizeof(*silenceBuffer) * numSamples);
    for (i = 0; i < numSamples; i++) {
        silenceBuffer[i] = 0;
    }
</pre>

<p>At the point when audio should be produced, you can switch to enqueuing buffers containing
real audio data.</p>