Glenn Kasten | c0c342b | 2013-04-22 13:49:43 -0700 | [diff] [blame^] | 1 | page.title=Design For Reduced Latency |
| 2 | @jd:body |
| 3 | |
| 4 | <div id="qv-wrapper"> |
| 5 | <div id="qv"> |
| 6 | <h2>In this document</h2> |
| 7 | <ol id="auto-toc"> |
| 8 | </ol> |
| 9 | </div> |
| 10 | </div> |
| 11 | |
| 12 | <p> |
| 13 | Android 4.1 (Jelly Bean) release introduced internal framework changes for |
| 14 | a lower latency audio output path. There were no public client API |
| 15 | or HAL API changes. This document describes the initial design, |
| 16 | which is expected to evolve over time. |
| 17 | Having a good understanding of this design should help device OEM and |
| 18 | SoC vendors to implement the design correctly on their particular devices |
| 19 | and chipsets. This article is not intended for application developers. |
| 20 | </p> |
| 21 | |
| 22 | <h2 id="trackCreation">Track Creation</h2> |
| 23 | |
| 24 | <p> |
| 25 | The client can optionally set bit <code>AUDIO_OUTPUT_FLAG_FAST</code> in the |
| 26 | <code>audio_output_flags_t</code> parameter of AudioTrack C++ constructor or |
| 27 | <code>AudioTrack::set()</code>. Currently the only clients that do so are: |
| 28 | </p> |
| 29 | |
| 30 | <ul> |
| 31 | <li>OpenSL ES</li> |
| 32 | <li>SoundPool</li> |
| 33 | <li>ToneGenerator</li> |
| 34 | </ul> |
| 35 | |
| 36 | <p> |
| 37 | The AudioTrack C++ implementation reviews the <code>AUDIO_OUTPUT_FLAG_FAST</code> |
| 38 | request and may optionally deny the request at client level. If it |
| 39 | decides to pass the request on, it does so using <code>TRACK_FAST</code> bit of |
| 40 | the <code>track_flags_t</code> parameter of the <code>IAudioTrack</code> factory method |
| 41 | <code>IAudioFlinger::createTrack()</code>. |
| 42 | </p> |
| 43 | |
| 44 | <p> |
| 45 | AudioFlinger (server) reviews the <code>TRACK_FAST</code> request and may |
| 46 | optionally deny the request at server level. It informs the client |
| 47 | whether or not the request was accepted, via bit <code>CBLK_FAST</code> of the |
| 48 | shared memory control block. |
| 49 | </p> |
| 50 | |
| 51 | <p> |
| 52 | The factors that impact the decision include: |
| 53 | </p> |
| 54 | |
| 55 | <ul> |
| 56 | <li>presence of a fast mixer thread for this output (see below)</li> |
| 57 | <li>track sample rate</li> |
| 58 | <li>presence of a client thread to execute callback handlers for this track</li> |
| 59 | <li>track buffer size</li> |
| 60 | <li>available fast track slots (see below)</li> |
| 61 | </ul> |
| 62 | |
| 63 | <p> |
| 64 | If the client's request was accepted, it is called a "fast track", |
| 65 | otherwise it's called a "normal track". |
| 66 | </p> |
| 67 | |
| 68 | <h2 id="mixerThreads">Mixer Threads</h2> |
| 69 | |
| 70 | <p> |
| 71 | At the time AudioFlinger creates a normal mixer thread, it decides |
| 72 | whether or not to also create a fast mixer thread. Both the normal |
| 73 | mixer and fast mixer are not associated with a particular track, |
| 74 | but rather with a set of tracks. There is always a normal mixer |
| 75 | thread. The fast mixer thread, if it exists, is subservient to the |
| 76 | normal mixer thread and acts under its control. |
| 77 | </p> |
| 78 | |
| 79 | <h3 id="fastMixer">Fast mixer</h3> |
| 80 | |
| 81 | <h4>Features</h4> |
| 82 | |
| 83 | <p> |
| 84 | The fast mixer thread provides these features: |
| 85 | </p> |
| 86 | |
| 87 | <ul> |
| 88 | <li>mixing of the normal mixer's sub-mix and up to 7 client fast tracks</li> |
| 89 | <li>Per track attenuation</li> |
| 90 | </ul> |
| 91 | |
| 92 | <p> |
| 93 | Omitted features: |
| 94 | </p> |
| 95 | |
| 96 | <ul> |
| 97 | <li>Per track sample rate conversion</li> |
| 98 | <li>Per track effects</li> |
| 99 | <li>Per mix effects</li> |
| 100 | </ul> |
| 101 | |
| 102 | <h4>Period</h4> |
| 103 | |
| 104 | <p> |
| 105 | The fast mixer runs periodically, with a recommended period of 2 |
| 106 | to 3 ms, or slightly higher if needed for scheduling stability. |
| 107 | This number was chosen so that, accounting for the complete |
| 108 | buffer pipeline, the total latency is on the order of 10 ms. Smaller |
| 109 | values are possible but may result in increased power consumption |
| 110 | and chance of glitches depending on CPU scheduling predictability. |
| 111 | Larger values are possible, up to 20 ms, but result in degraded |
| 112 | total latency and so should be avoided. |
| 113 | </p> |
| 114 | |
| 115 | <h4>Scheduling</h4> |
| 116 | |
| 117 | <p> |
| 118 | The fast mixer runs at elevated <code>SCHED_FIFO</code> priority. It needs very |
| 119 | little CPU time, but must run often and with low scheduling jitter. |
| 120 | Running too late will result in glitches due to underrun. Running |
| 121 | too early will result in glitches due to pulling from a fast track |
| 122 | before the track has provided data. |
| 123 | </p> |
| 124 | |
| 125 | <h4>Blocking</h4> |
| 126 | |
| 127 | <p> |
| 128 | Ideally the fast mixer thread never blocks, other than at HAL |
| 129 | <code>write()</code>. Other occurrences of blocking within the fast mixer are |
| 130 | considered bugs. In particular, mutexes are avoided. |
| 131 | </p> |
| 132 | |
| 133 | <h4>Relationship to other components</h4> |
| 134 | |
| 135 | <p> |
| 136 | The fast mixer has little direct interaction with clients. In |
| 137 | particular, it does not see binder-level operations, but it does |
| 138 | access the client's shared memory control block. |
| 139 | </p> |
| 140 | |
| 141 | <p> |
| 142 | The fast mixer receives commands from the normal mixer via a state queue. |
| 143 | </p> |
| 144 | |
| 145 | <p> |
| 146 | Other than pulling track data, interaction with clients is via the normal mixer. |
| 147 | </p> |
| 148 | |
| 149 | <p> |
| 150 | The fast mixer's primary sink is the audio HAL. |
| 151 | </p> |
| 152 | |
| 153 | <h3 id="normalMixer">Normal mixer</h3> |
| 154 | |
| 155 | <h4>Features</h4> |
| 156 | |
| 157 | <p> |
| 158 | All features are enabled: |
| 159 | </p> |
| 160 | |
| 161 | <ul> |
| 162 | <li>Up to 32 tracks</li> |
| 163 | <li>Per track attenuation</li> |
| 164 | <li>Per track sample rate conversion</li> |
| 165 | <li>Effects processing</li> |
| 166 | </ul> |
| 167 | |
| 168 | <h4>Period</h4> |
| 169 | |
| 170 | <p> |
| 171 | The period is computed to be the first integral multiple of the |
| 172 | fast mixer period that is >= 20 milliseconds. |
| 173 | </p> |
| 174 | |
| 175 | <h4>Scheduling</h4> |
| 176 | |
| 177 | <p> |
| 178 | The normal mixer runs at elevated <code>SCHED_OTHER</code> priority. |
| 179 | </p> |
| 180 | |
| 181 | <h4>Blocking</h4> |
| 182 | |
| 183 | <p> |
| 184 | The normal mixer is permitted to block, and often does so at various |
| 185 | mutexes as well as at a blocking pipe to write its sub-mix. |
| 186 | </p> |
| 187 | |
| 188 | <h4>Relationship to other components</h4> |
| 189 | |
| 190 | <p> |
| 191 | The normal mixer interacts extensively with the outside world, |
| 192 | including binder threads, audio policy manager, fast mixer thread, |
| 193 | and client tracks. |
| 194 | </p> |
| 195 | |
| 196 | <p> |
| 197 | The normal mixer's sink is a blocking pipe to the fast mixer's track 0. |
| 198 | </p> |
| 199 | |
| 200 | <h2 id="flags">Flags</h2> |
| 201 | |
| 202 | <p> |
| 203 | <code>AUDIO_OUTPUT_FLAG_FAST</code> bit is a hint. There's no guarantee the |
| 204 | request will be fulfilled. |
| 205 | </p> |
| 206 | |
| 207 | <p> |
| 208 | <code>AUDIO_OUTPUT_FLAG_FAST</code> is a client-level concept. It does not appear |
| 209 | in server. |
| 210 | </p> |
| 211 | |
| 212 | <p> |
| 213 | <code>TRACK_FAST</code> is a client -> server concept. |
| 214 | </p> |
| 215 | |