Arthur Eubanks | 263d674 | 2017-12-18 13:46:59 -0800 | [diff] [blame] | 1 | ActivityManagerPerfTests |
| 2 | |
| 3 | Performance tests for various ActivityManager components, e.g. Services, Broadcasts |
Arthur Eubanks | 2dd6779 | 2018-02-07 15:55:10 -0800 | [diff] [blame] | 4 | * These are only for tests that don't require a target package to test against |
| 5 | * Self-contained perf tests should go in frameworks/base/apct-tests/perftests |
Arthur Eubanks | 263d674 | 2017-12-18 13:46:59 -0800 | [diff] [blame] | 6 | |
Arthur Eubanks | 2dd6779 | 2018-02-07 15:55:10 -0800 | [diff] [blame] | 7 | Command to run tests |
| 8 | * atest .../frameworks/base/tests/ActivityManagerPerfTests/tests/ |
| 9 | * Command currently not working: b/71859981 |
Arthur Eubanks | 263d674 | 2017-12-18 13:46:59 -0800 | [diff] [blame] | 10 | * m ActivityManagerPerfTests ActivityManagerPerfTestsTestApp && \ |
Arthur Eubanks | 2dd6779 | 2018-02-07 15:55:10 -0800 | [diff] [blame] | 11 | adb install "$OUT"/data/app/ActivityManagerPerfTests/ActivityManagerPerfTests.apk && \ |
| 12 | adb install "$OUT"/data/app/ActivityManagerPerfTestsTestApp/ActivityManagerPerfTestsTestApp.apk && \ |
Arthur Eubanks | 263d674 | 2017-12-18 13:46:59 -0800 | [diff] [blame] | 13 | adb shell am instrument -w \ |
| 14 | com.android.frameworks.perftests.amtests/android.support.test.runner.AndroidJUnitRunner |
| 15 | |
| 16 | Overview |
| 17 | * The numbers we are trying to measure are end-to-end numbers |
| 18 | * For example, the time it takes from sending an Intent to start a Service |
| 19 | to the time the Service runs its callbacks |
| 20 | * System.nanoTime() is monotonic and consistent between processes, so we use that for measuring time |
Arthur Eubanks | 263d674 | 2017-12-18 13:46:59 -0800 | [diff] [blame] | 21 | * If the test app is involved, it will measure the time and send it back to the instrumentation test |
Arthur Eubanks | 2dd6779 | 2018-02-07 15:55:10 -0800 | [diff] [blame] | 22 | * The time is sent back through a Binder interface in the Intent with the help of Utils.sendTime() |
Arthur Eubanks | 263d674 | 2017-12-18 13:46:59 -0800 | [diff] [blame] | 23 | * Each sent time is tagged with an id since there can be multiple events that send back a time |
Arthur Eubanks | 2dd6779 | 2018-02-07 15:55:10 -0800 | [diff] [blame] | 24 | * Each test will run multiple times to account for variation in test runs |
Arthur Eubanks | 263d674 | 2017-12-18 13:46:59 -0800 | [diff] [blame] | 25 | |
| 26 | Structure |
| 27 | * tests |
| 28 | * Instrumentation test which runs the various performance tests and reports the results |
Arthur Eubanks | 263d674 | 2017-12-18 13:46:59 -0800 | [diff] [blame] | 29 | * test-app |
| 30 | * Target package which contains the Services, BroadcastReceivers, etc. to test against |
| 31 | * Sends the time it measures back to the test package |
Arthur Eubanks | 263d674 | 2017-12-18 13:46:59 -0800 | [diff] [blame] | 32 | * utils |
| 33 | * Utilities that both the instrumentation test and test app can use |
Arthur Eubanks | 2dd6779 | 2018-02-07 15:55:10 -0800 | [diff] [blame] | 34 | |
| 35 | Adding tests |
| 36 | * Example |
| 37 | * Look at tests/src/com/android/frameworks/perftests/am/BroadcastPerfTest and |
| 38 | test-app/src/com/android/frameworks/perftests/amteststestapp/TestBroadcastReceiver |
| 39 | for simple examples using this framework |
| 40 | * Steps |
| 41 | * Add any components you will test against in the target package under |
| 42 | test-app/src/com/android/frameworks/perftests/amteststestapp/ |
| 43 | * Add the test class under tests/src/com/android/frameworks/perftests/am/tests/ |
| 44 | * The class should extend BasePerfTest |
| 45 | * Each test should call runPerfFunction() returning the elapsed time for a single iteration |
| 46 | * The test has access to a Context through mContext |
| 47 | * If you are measuring the time elapsed of something that either starts or ends in the target |
| 48 | package |
| 49 | * The target package can report the time it measures through an ITimeReceiverCallback passed |
| 50 | through an Intent through Utils.sendTime(intent, "tag") |
| 51 | (or however a Binder needs to be passed to the target package) |
| 52 | * The instrumentation test can collect that time by calling getReceivedTimeNs("tag") and |
| 53 | calculate the elapsed time |
| 54 | * Each timestamp sent to the instrumentation test is tagged with a tag since multiple timestamps |
| 55 | can be reported in an iteration |
| 56 | * If the target package should be running before your test logic starts, add startTargetPackage(); |
| 57 | at the beginning of the iteration |
| 58 | * Reporting |
| 59 | * Look at go/am-perf for how to add new tests to dashboards and receive notification on regression |