Glenn Kasten | c0c342b | 2013-04-22 13:49:43 -0700 | [diff] [blame] | 1 | page.title=Design For Reduced Latency |
| 2 | @jd:body |
| 3 | |
Clay Murphy | bc92aea | 2014-10-16 10:13:18 -0700 | [diff] [blame] | 4 | <!-- |
| 5 | Copyright 2013 The Android Open Source Project |
| 6 | |
| 7 | Licensed under the Apache License, Version 2.0 (the "License"); |
| 8 | you may not use this file except in compliance with the License. |
| 9 | You may obtain a copy of the License at |
| 10 | |
| 11 | http://www.apache.org/licenses/LICENSE-2.0 |
| 12 | |
| 13 | Unless required by applicable law or agreed to in writing, software |
| 14 | distributed under the License is distributed on an "AS IS" BASIS, |
| 15 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
| 16 | See the License for the specific language governing permissions and |
| 17 | limitations under the License. |
| 18 | --> |
Glenn Kasten | c0c342b | 2013-04-22 13:49:43 -0700 | [diff] [blame] | 19 | <div id="qv-wrapper"> |
| 20 | <div id="qv"> |
| 21 | <h2>In this document</h2> |
| 22 | <ol id="auto-toc"> |
| 23 | </ol> |
| 24 | </div> |
| 25 | </div> |
| 26 | |
| 27 | <p> |
Clay Murphy | c28f237 | 2013-09-25 16:13:40 -0700 | [diff] [blame] | 28 | The Android 4.1 release introduced internal framework changes for |
Glenn Kasten | c0c342b | 2013-04-22 13:49:43 -0700 | [diff] [blame] | 29 | a lower latency audio output path. There were no public client API |
| 30 | or HAL API changes. This document describes the initial design, |
| 31 | which is expected to evolve over time. |
| 32 | Having a good understanding of this design should help device OEM and |
Clay Murphy | c28f237 | 2013-09-25 16:13:40 -0700 | [diff] [blame] | 33 | SoC vendors implement the design correctly on their particular devices |
Glenn Kasten | c0c342b | 2013-04-22 13:49:43 -0700 | [diff] [blame] | 34 | and chipsets. This article is not intended for application developers. |
| 35 | </p> |
| 36 | |
| 37 | <h2 id="trackCreation">Track Creation</h2> |
| 38 | |
| 39 | <p> |
| 40 | The client can optionally set bit <code>AUDIO_OUTPUT_FLAG_FAST</code> in the |
| 41 | <code>audio_output_flags_t</code> parameter of AudioTrack C++ constructor or |
| 42 | <code>AudioTrack::set()</code>. Currently the only clients that do so are: |
| 43 | </p> |
| 44 | |
| 45 | <ul> |
| 46 | <li>OpenSL ES</li> |
| 47 | <li>SoundPool</li> |
| 48 | <li>ToneGenerator</li> |
| 49 | </ul> |
| 50 | |
| 51 | <p> |
| 52 | The AudioTrack C++ implementation reviews the <code>AUDIO_OUTPUT_FLAG_FAST</code> |
| 53 | request and may optionally deny the request at client level. If it |
| 54 | decides to pass the request on, it does so using <code>TRACK_FAST</code> bit of |
| 55 | the <code>track_flags_t</code> parameter of the <code>IAudioTrack</code> factory method |
| 56 | <code>IAudioFlinger::createTrack()</code>. |
| 57 | </p> |
| 58 | |
| 59 | <p> |
Clay Murphy | c28f237 | 2013-09-25 16:13:40 -0700 | [diff] [blame] | 60 | The AudioFlinger audio server reviews the <code>TRACK_FAST</code> request and may |
Glenn Kasten | c0c342b | 2013-04-22 13:49:43 -0700 | [diff] [blame] | 61 | optionally deny the request at server level. It informs the client |
| 62 | whether or not the request was accepted, via bit <code>CBLK_FAST</code> of the |
| 63 | shared memory control block. |
| 64 | </p> |
| 65 | |
| 66 | <p> |
| 67 | The factors that impact the decision include: |
| 68 | </p> |
| 69 | |
| 70 | <ul> |
| 71 | <li>presence of a fast mixer thread for this output (see below)</li> |
| 72 | <li>track sample rate</li> |
| 73 | <li>presence of a client thread to execute callback handlers for this track</li> |
| 74 | <li>track buffer size</li> |
| 75 | <li>available fast track slots (see below)</li> |
| 76 | </ul> |
| 77 | |
| 78 | <p> |
Clay Murphy | c28f237 | 2013-09-25 16:13:40 -0700 | [diff] [blame] | 79 | If the client's request was accepted, it is called a "fast track." |
| 80 | Otherwise it's called a "normal track." |
Glenn Kasten | c0c342b | 2013-04-22 13:49:43 -0700 | [diff] [blame] | 81 | </p> |
| 82 | |
| 83 | <h2 id="mixerThreads">Mixer Threads</h2> |
| 84 | |
| 85 | <p> |
| 86 | At the time AudioFlinger creates a normal mixer thread, it decides |
| 87 | whether or not to also create a fast mixer thread. Both the normal |
| 88 | mixer and fast mixer are not associated with a particular track, |
| 89 | but rather with a set of tracks. There is always a normal mixer |
| 90 | thread. The fast mixer thread, if it exists, is subservient to the |
| 91 | normal mixer thread and acts under its control. |
| 92 | </p> |
| 93 | |
| 94 | <h3 id="fastMixer">Fast mixer</h3> |
| 95 | |
| 96 | <h4>Features</h4> |
| 97 | |
| 98 | <p> |
| 99 | The fast mixer thread provides these features: |
| 100 | </p> |
| 101 | |
| 102 | <ul> |
| 103 | <li>mixing of the normal mixer's sub-mix and up to 7 client fast tracks</li> |
| 104 | <li>Per track attenuation</li> |
| 105 | </ul> |
| 106 | |
| 107 | <p> |
| 108 | Omitted features: |
| 109 | </p> |
| 110 | |
| 111 | <ul> |
| 112 | <li>Per track sample rate conversion</li> |
| 113 | <li>Per track effects</li> |
| 114 | <li>Per mix effects</li> |
| 115 | </ul> |
| 116 | |
| 117 | <h4>Period</h4> |
| 118 | |
| 119 | <p> |
Clay Murphy | c28f237 | 2013-09-25 16:13:40 -0700 | [diff] [blame] | 120 | The fast mixer runs periodically, with a recommended period of two |
| 121 | to three milliseconds (ms), or slightly higher if needed for scheduling stability. |
Glenn Kasten | c0c342b | 2013-04-22 13:49:43 -0700 | [diff] [blame] | 122 | This number was chosen so that, accounting for the complete |
| 123 | buffer pipeline, the total latency is on the order of 10 ms. Smaller |
| 124 | values are possible but may result in increased power consumption |
| 125 | and chance of glitches depending on CPU scheduling predictability. |
| 126 | Larger values are possible, up to 20 ms, but result in degraded |
| 127 | total latency and so should be avoided. |
| 128 | </p> |
| 129 | |
| 130 | <h4>Scheduling</h4> |
| 131 | |
| 132 | <p> |
| 133 | The fast mixer runs at elevated <code>SCHED_FIFO</code> priority. It needs very |
| 134 | little CPU time, but must run often and with low scheduling jitter. |
| 135 | Running too late will result in glitches due to underrun. Running |
| 136 | too early will result in glitches due to pulling from a fast track |
| 137 | before the track has provided data. |
| 138 | </p> |
| 139 | |
| 140 | <h4>Blocking</h4> |
| 141 | |
| 142 | <p> |
| 143 | Ideally the fast mixer thread never blocks, other than at HAL |
| 144 | <code>write()</code>. Other occurrences of blocking within the fast mixer are |
| 145 | considered bugs. In particular, mutexes are avoided. |
| 146 | </p> |
| 147 | |
| 148 | <h4>Relationship to other components</h4> |
| 149 | |
| 150 | <p> |
| 151 | The fast mixer has little direct interaction with clients. In |
| 152 | particular, it does not see binder-level operations, but it does |
| 153 | access the client's shared memory control block. |
| 154 | </p> |
| 155 | |
| 156 | <p> |
| 157 | The fast mixer receives commands from the normal mixer via a state queue. |
| 158 | </p> |
| 159 | |
| 160 | <p> |
| 161 | Other than pulling track data, interaction with clients is via the normal mixer. |
| 162 | </p> |
| 163 | |
| 164 | <p> |
| 165 | The fast mixer's primary sink is the audio HAL. |
| 166 | </p> |
| 167 | |
| 168 | <h3 id="normalMixer">Normal mixer</h3> |
| 169 | |
| 170 | <h4>Features</h4> |
| 171 | |
| 172 | <p> |
| 173 | All features are enabled: |
| 174 | </p> |
| 175 | |
| 176 | <ul> |
| 177 | <li>Up to 32 tracks</li> |
| 178 | <li>Per track attenuation</li> |
| 179 | <li>Per track sample rate conversion</li> |
| 180 | <li>Effects processing</li> |
| 181 | </ul> |
| 182 | |
| 183 | <h4>Period</h4> |
| 184 | |
| 185 | <p> |
| 186 | The period is computed to be the first integral multiple of the |
Clay Murphy | c28f237 | 2013-09-25 16:13:40 -0700 | [diff] [blame] | 187 | fast mixer period that is >= 20 ms. |
Glenn Kasten | c0c342b | 2013-04-22 13:49:43 -0700 | [diff] [blame] | 188 | </p> |
| 189 | |
| 190 | <h4>Scheduling</h4> |
| 191 | |
| 192 | <p> |
| 193 | The normal mixer runs at elevated <code>SCHED_OTHER</code> priority. |
| 194 | </p> |
| 195 | |
| 196 | <h4>Blocking</h4> |
| 197 | |
| 198 | <p> |
| 199 | The normal mixer is permitted to block, and often does so at various |
| 200 | mutexes as well as at a blocking pipe to write its sub-mix. |
| 201 | </p> |
| 202 | |
| 203 | <h4>Relationship to other components</h4> |
| 204 | |
| 205 | <p> |
| 206 | The normal mixer interacts extensively with the outside world, |
| 207 | including binder threads, audio policy manager, fast mixer thread, |
| 208 | and client tracks. |
| 209 | </p> |
| 210 | |
| 211 | <p> |
| 212 | The normal mixer's sink is a blocking pipe to the fast mixer's track 0. |
| 213 | </p> |
| 214 | |
| 215 | <h2 id="flags">Flags</h2> |
| 216 | |
| 217 | <p> |
| 218 | <code>AUDIO_OUTPUT_FLAG_FAST</code> bit is a hint. There's no guarantee the |
| 219 | request will be fulfilled. |
| 220 | </p> |
| 221 | |
| 222 | <p> |
| 223 | <code>AUDIO_OUTPUT_FLAG_FAST</code> is a client-level concept. It does not appear |
| 224 | in server. |
| 225 | </p> |
| 226 | |
| 227 | <p> |
| 228 | <code>TRACK_FAST</code> is a client -> server concept. |
| 229 | </p> |
| 230 | |