Jamie Gennis | 9c183f2 | 2012-12-03 16:44:16 -0800 | [diff] [blame] | 1 | Flatland is a benchmark for measuring GPU performance in various 2D UI |
| 2 | rendering and window compositing scenarios. It is designed to be used early |
| 3 | in the device development process to evaluate GPU hardware (e.g. for SoC |
| 4 | selection). It uses OpenGL ES 2.0, gralloc, and the Android explicit |
| 5 | synchronization framework, so it can only be run on devices with drivers |
| 6 | supporting those HALs. |
| 7 | |
| 8 | |
| 9 | Preparing a Device |
| 10 | |
| 11 | Because it's measuring hardware performance, flatland should be run in as |
| 12 | consistent and static an environment as possible. The display should be |
| 13 | turned off and background services should be stopped before running the |
| 14 | benchmark. Running 'adb shell stop' after turning off the display is probably |
| 15 | sufficient for this, but if there are device- specific background services |
| 16 | that consume much CPU cycles, memory bandwidth, or might otherwise interfere |
| 17 | with GPU rendering, those should be stopped as well (and ideally they'd be |
| 18 | fixed or eliminated for production devices). |
| 19 | |
| 20 | Additionally, all relevant hardware clocks should be locked at a particular |
| 21 | frequency when running flatland. At a minimum this includes the CPU, GPU, and |
| 22 | memory bus clocks. Running flatland with dynamic clocking essentially |
| 23 | measures the behavior of the dynamic clocking algorithm under a fairly |
| 24 | unrealistic workload, and will likely result in unstable and useless results. |
| 25 | |
| 26 | If running the benchmark with the clocks locked causes thermal issues, the -s |
| 27 | command line option can be used to insert a sleep (specified in milliseconds) |
| 28 | in between each benchmark sample run. Regardless of the scenario being |
| 29 | measured, each sample measurement runs for between 50 and 200 ms, so a sleep |
| 30 | time between 10 and 50 ms should address most thermal problems. |
| 31 | |
| 32 | |
| 33 | Interpreting the Output |
| 34 | |
| 35 | The output of flatland should look something like this: |
| 36 | |
| 37 | cmdline: flatland |
| 38 | Scenario | Resolution | Time (ms) |
| 39 | 16:10 Single Static Window | 1280 x 800 | fast |
| 40 | 16:10 Single Static Window | 2560 x 1600 | 5.368 |
| 41 | 16:10 Single Static Window | 3840 x 2400 | 11.979 |
| 42 | 16:10 App -> Home Transition | 1280 x 800 | 4.069 |
| 43 | 16:10 App -> Home Transition | 2560 x 1600 | 15.911 |
| 44 | 16:10 App -> Home Transition | 3840 x 2400 | 38.795 |
| 45 | 16:10 SurfaceView -> Home Transition | 1280 x 800 | 5.387 |
| 46 | 16:10 SurfaceView -> Home Transition | 2560 x 1600 | 21.147 |
| 47 | 16:10 SurfaceView -> Home Transition | 3840 x 2400 | slow |
| 48 | |
| 49 | The first column is simply a description of the scenario that's being |
| 50 | simulated. The second column indicates the resolution at which the scenario |
| 51 | was measured. The third column is the measured benchmark result. It |
| 52 | indicates the expected time in milliseconds that a single frame of the |
| 53 | scenario takes to complete. |
| 54 | |
| 55 | The third column may also contain one of three other values: |
| 56 | |
| 57 | fast - This indicates that frames of the scenario completed too fast to be |
| 58 | reliably benchmarked. This corresponds to a frame time less than 3 ms. |
| 59 | Rather than spending time trying (and likely failing) to get a stable |
| 60 | result, the scenario was skipped. |
| 61 | |
| 62 | slow - This indicates that frames of the scenario took too long to |
| 63 | complete. This corresponds to a frame time over 50 ms. Rather than |
| 64 | simulating a scenario that is obviously impractical on this device, the |
| 65 | scenario was skipped. |
| 66 | |
| 67 | varies - This indicates that the scenario was measured, but it did not |
| 68 | yield a stable result. Occasionally this happens with an otherwise stable |
| 69 | scenario. In this case, simply rerunning flatland should yield a valid |
| 70 | result. If a scenario repeatedly results in a 'varies' output, that |
| 71 | probably indicates that something is wrong with the environment in which |
| 72 | flatland is being run. Check that the hardware clock frequencies are |
| 73 | locked and that no heavy-weight services / daemons are running in the |
| 74 | background. |