| { |
| "cells": [ |
| { |
| "cell_type": "markdown", |
| "metadata": {}, |
| "source": [ |
| "# EAS Testing - Recents Fling on Android" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "metadata": {}, |
| "source": [ |
| "The goal of this experiment is to collect frame statistics while swiping up and down tabs of recently opened applications on a Nexus N5X running Android with an EAS kernel. This process is name **Recents Fling**. The Analysis phase will consist in comparing EAS with other schedulers, that is comparing *sched* governor with:\n", |
| "\n", |
| " - interactive\n", |
| " - performance\n", |
| " - powersave\n", |
| " - ondemand\n", |
| " \n", |
| "For this experiment it is recommended to open many applications so that we can swipe over more recently opened applications." |
| ] |
| }, |
| { |
| "cell_type": "code", |
| "execution_count": 1, |
| "metadata": { |
| "collapsed": false |
| }, |
| "outputs": [], |
| "source": [ |
| "import logging\n", |
| "reload(logging)\n", |
| "log_fmt = '%(asctime)-9s %(levelname)-8s: %(message)s'\n", |
| "logging.basicConfig(format=log_fmt)\n", |
| "\n", |
| "# Change to info once the notebook runs ok\n", |
| "logging.getLogger().setLevel(logging.INFO)" |
| ] |
| }, |
| { |
| "cell_type": "code", |
| "execution_count": 2, |
| "metadata": { |
| "collapsed": false |
| }, |
| "outputs": [ |
| { |
| "name": "stdout", |
| "output_type": "stream", |
| "text": [ |
| "Populating the interactive namespace from numpy and matplotlib\n" |
| ] |
| } |
| ], |
| "source": [ |
| "%pylab inline\n", |
| "\n", |
| "import copy\n", |
| "import os\n", |
| "from time import sleep\n", |
| "\n", |
| "# Support to access the remote target\n", |
| "import devlib\n", |
| "from env import TestEnv\n", |
| "\n", |
| "from devlib.utils.android import adb_command\n", |
| "\n", |
| "# Support for trace events analysis\n", |
| "from trace import Trace\n", |
| "from trace_analysis import TraceAnalysis\n", |
| "\n", |
| "# Suport for FTrace events parsing and visualization\n", |
| "import trappy" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "metadata": {}, |
| "source": [ |
| "# Test Environment set up" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "metadata": {}, |
| "source": [ |
| "In case more than one Android device are conencted to the host, you must specify the ID of the device you want to target in `my_target_conf`. Run `adb devices` on your host to get the ID." |
| ] |
| }, |
| { |
| "cell_type": "code", |
| "execution_count": 3, |
| "metadata": { |
| "collapsed": false |
| }, |
| "outputs": [], |
| "source": [ |
| "# Setup a target configuration\n", |
| "my_target_conf = {\n", |
| " \n", |
| " # Target platform and board\n", |
| " \"platform\" : 'android',\n", |
| "\n", |
| " # Add target support\n", |
| " \"board\" : 'n5x',\n", |
| "\n", |
| " # Device ID\n", |
| " #\"device\" : \"00b1346f0878ccb1\",\n", |
| " \n", |
| " # Define devlib modules to load\n", |
| " \"modules\" : [\n", |
| " 'cpufreq' # enable CPUFreq support\n", |
| " ],\n", |
| "}" |
| ] |
| }, |
| { |
| "cell_type": "code", |
| "execution_count": 4, |
| "metadata": { |
| "collapsed": false, |
| "scrolled": false |
| }, |
| "outputs": [], |
| "source": [ |
| "my_tests_conf = {\n", |
| "\n", |
| " # Folder where all the results will be collected\n", |
| " \"results_dir\" : \"Android_RecentsFling\",\n", |
| "\n", |
| " # Platform configurations to test\n", |
| " \"confs\" : [\n", |
| " {\n", |
| " \"tag\" : \"recentsfling\",\n", |
| " \"flags\" : \"ftrace\", # Enable FTrace events\n", |
| " \"sched_features\" : \"ENERGY_AWARE\", # enable EAS\n", |
| " },\n", |
| " ],\n", |
| " \n", |
| " # FTrace events to collect for all the tests configuration which have\n", |
| " # the \"ftrace\" flag enabled\n", |
| " \"ftrace\" : {\n", |
| " \"events\" : [\n", |
| " \"sched_switch\",\n", |
| " \"sched_load_avg_cpu\",\n", |
| " \"cpu_frequency\",\n", |
| " \"cpu_capacity\"\n", |
| " ],\n", |
| " \"buffsize\" : 10 * 1024,\n", |
| " },\n", |
| " \n", |
| " # Tools required by the experiments\n", |
| " \"tools\" : [ 'trace-cmd' ],\n", |
| "}" |
| ] |
| }, |
| { |
| "cell_type": "code", |
| "execution_count": 5, |
| "metadata": { |
| "collapsed": false, |
| "scrolled": true |
| }, |
| "outputs": [ |
| { |
| "name": "stderr", |
| "output_type": "stream", |
| "text": [ |
| "2016-04-12 15:13:56,738 INFO : Target - Using base path: /home/pippo/work/lisa\n", |
| "2016-04-12 15:13:56,740 INFO : Target - Loading custom (inline) target configuration\n", |
| "2016-04-12 15:13:56,740 INFO : Target - Loading custom (inline) test configuration\n", |
| "2016-04-12 15:13:56,741 INFO : Target - Devlib modules to load: ['bl', 'cpufreq']\n", |
| "2016-04-12 15:13:56,742 INFO : Target - Connecting Android target [DEFAULT]\n", |
| "2016-04-12 15:13:57,210 INFO : Target - Initializing target workdir:\n", |
| "2016-04-12 15:13:57,211 INFO : Target - /data/local/tmp/devlib-target\n", |
| "2016-04-12 15:13:59,578 INFO : Target - Topology:\n", |
| "2016-04-12 15:13:59,579 INFO : Target - [[0, 1, 2, 3], [4, 5]]\n", |
| "2016-04-12 15:14:00,205 INFO : FTrace - Enabled tracepoints:\n", |
| "2016-04-12 15:14:00,206 INFO : FTrace - sched_switch\n", |
| "2016-04-12 15:14:00,207 INFO : FTrace - sched_load_avg_cpu\n", |
| "2016-04-12 15:14:00,208 INFO : FTrace - cpu_frequency\n", |
| "2016-04-12 15:14:00,208 INFO : FTrace - cpu_capacity\n", |
| "2016-04-12 15:14:00,209 INFO : TestEnv - Set results folder to:\n", |
| "2016-04-12 15:14:00,210 INFO : TestEnv - /home/pippo/work/lisa/results/Android_RecentsFling\n", |
| "2016-04-12 15:14:00,211 INFO : TestEnv - Experiment results available also in:\n", |
| "2016-04-12 15:14:00,211 INFO : TestEnv - /home/pippo/work/lisa/results_latest\n" |
| ] |
| } |
| ], |
| "source": [ |
| "# Initialize a test environment using:\n", |
| "# the provided target configuration (my_target_conf)\n", |
| "# the provided test configuration (my_test_conf)\n", |
| "te = TestEnv(target_conf=my_target_conf, test_conf=my_tests_conf)\n", |
| "target = te.target" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "metadata": {}, |
| "source": [ |
| "# Support Functions" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "metadata": {}, |
| "source": [ |
| "This set of support functions will help us running the benchmark using different CPUFreq governors." |
| ] |
| }, |
| { |
| "cell_type": "code", |
| "execution_count": 6, |
| "metadata": { |
| "collapsed": true |
| }, |
| "outputs": [], |
| "source": [ |
| "def set_performance():\n", |
| " target.cpufreq.set_all_governors('performance')\n", |
| "\n", |
| "def set_powersave():\n", |
| " target.cpufreq.set_all_governors('powersave')\n", |
| "\n", |
| "def set_interactive():\n", |
| " target.cpufreq.set_all_governors('interactive')\n", |
| "\n", |
| "def set_sched():\n", |
| " target.cpufreq.set_all_governors('sched')\n", |
| "\n", |
| "def set_ondemand():\n", |
| " target.cpufreq.set_all_governors('ondemand')\n", |
| " \n", |
| " for cpu in target.list_online_cpus():\n", |
| " tunables = target.cpufreq.get_governor_tunables(cpu)\n", |
| " target.cpufreq.set_governor_tunables(\n", |
| " cpu,\n", |
| " 'ondemand',\n", |
| " **{'sampling_rate' : tunables['sampling_rate_min']}\n", |
| " )" |
| ] |
| }, |
| { |
| "cell_type": "code", |
| "execution_count": 7, |
| "metadata": { |
| "collapsed": false |
| }, |
| "outputs": [], |
| "source": [ |
| "# CPUFreq configurations to test\n", |
| "confs = {\n", |
| " 'performance' : {\n", |
| " 'label' : 'prf',\n", |
| " 'set' : set_performance,\n", |
| " },\n", |
| " #'powersave' : {\n", |
| " # 'label' : 'pws',\n", |
| " # 'set' : set_powersave,\n", |
| " #},\n", |
| " 'interactive' : {\n", |
| " 'label' : 'int',\n", |
| " 'set' : set_interactive,\n", |
| " },\n", |
| " #'sched' : {\n", |
| " # 'label' : 'sch',\n", |
| " # 'set' : set_sched,\n", |
| " #},\n", |
| " #'ondemand' : {\n", |
| " # 'label' : 'odm',\n", |
| " # 'set' : set_ondemand,\n", |
| " #}\n", |
| "}\n", |
| "\n", |
| "# The set of results for each comparison test\n", |
| "results = {}" |
| ] |
| }, |
| { |
| "cell_type": "code", |
| "execution_count": 8, |
| "metadata": { |
| "collapsed": true |
| }, |
| "outputs": [], |
| "source": [ |
| "def recentsfling_run(exp_dir):\n", |
| " # Open Recents on the target device\n", |
| " target.execute('input keyevent KEYCODE_APP_SWITCH')\n", |
| " # Allow the activity to start\n", |
| " sleep(5)\n", |
| " # Reset framestats collection\n", |
| " target.execute('dumpsys gfxinfo --reset')\n", |
| " \n", |
| " logging.info('Start Swiping Recents')\n", |
| " for i in range(5):\n", |
| " # Simulate two fast UP and DOWN swipes\n", |
| " target.execute('input swipe 500 1700 500 500 100')\n", |
| " target.execute('input swipe 500 500 500 1700 100')\n", |
| " # Simulate two slow UP and DOWN swipes\n", |
| " target.execute('input swipe 500 1700 500 500 1000')\n", |
| " target.execute('input swipe 500 500 500 1700 1000')\n", |
| " logging.info('Swiping Recents Completed')\n", |
| " \n", |
| " # Get frame stats\n", |
| " framestats_file = os.path.join(exp_dir, \"framestats.txt\")\n", |
| " adb_command(target.adb_name, 'shell dumpsys gfxinfo com.android.systemui > {}'.format(framestats_file))\n", |
| " \n", |
| " # Close Recents\n", |
| " target.execute('input keyevent KEYCODE_BACK')\n", |
| "\n", |
| " return framestats_file" |
| ] |
| }, |
| { |
| "cell_type": "code", |
| "execution_count": 9, |
| "metadata": { |
| "collapsed": false |
| }, |
| "outputs": [], |
| "source": [ |
| "def experiment(governor, exp_dir):\n", |
| " os.system('mkdir -p {}'.format(exp_dir));\n", |
| "\n", |
| " logging.info('------------------------')\n", |
| " logging.info('Run workload using %s governor', governor)\n", |
| " confs[governor]['set']()\n", |
| " \n", |
| " # Start FTrace\n", |
| " te.ftrace.start()\n", |
| " \n", |
| " ### Run the benchmark ###\n", |
| " framestats_file = recentsfling_run(exp_dir)\n", |
| " \n", |
| " # Stop FTrace\n", |
| " te.ftrace.stop() \n", |
| "\n", |
| " # Collect and keep track of the trace\n", |
| " trace_file = os.path.join(exp_dir, 'trace.dat')\n", |
| " te.ftrace.get_trace(trace_file)\n", |
| " \n", |
| " # Parse trace\n", |
| " tr = Trace(te.platform, exp_dir,\n", |
| " events=my_tests_conf['ftrace']['events'])\n", |
| " ta = TraceAnalysis(tr, te.platform)\n", |
| " \n", |
| " # return all the experiment data\n", |
| " return {\n", |
| " 'dir' : exp_dir,\n", |
| " 'framestats_file' : framestats_file,\n", |
| " 'trace' : trace_file,\n", |
| " 'ftrace' : tr.ftrace,\n", |
| " 'ta' : ta\n", |
| " }" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "metadata": {}, |
| "source": [ |
| "# Run experiments and collect traces" |
| ] |
| }, |
| { |
| "cell_type": "code", |
| "execution_count": 10, |
| "metadata": { |
| "collapsed": false, |
| "scrolled": true |
| }, |
| "outputs": [ |
| { |
| "name": "stderr", |
| "output_type": "stream", |
| "text": [ |
| "2016-04-12 15:14:07,112 INFO : ------------------------\n", |
| "2016-04-12 15:14:07,113 INFO : Run workload using performance governor\n", |
| "2016-04-12 15:14:14,452 INFO : Start Swiping Recents\n", |
| "2016-04-12 15:14:38,443 INFO : Swiping Recents Completed\n", |
| "2016-04-12 15:14:44,128 INFO : Parsing FTrace format...\n", |
| "2016-04-12 15:14:51,841 INFO : Collected events spans a 30.689 [s] time interval\n", |
| "2016-04-12 15:14:51,842 INFO : Set plots time range to (0.000000, 30.688793)[s]\n", |
| "2016-04-12 15:14:52,761 INFO : ------------------------\n", |
| "2016-04-12 15:14:52,761 INFO : Run workload using interactive governor\n", |
| "2016-04-12 15:15:00,321 INFO : Start Swiping Recents\n", |
| "2016-04-12 15:15:25,884 INFO : Swiping Recents Completed\n", |
| "2016-04-12 15:15:33,608 INFO : Parsing FTrace format...\n", |
| "2016-04-12 15:15:41,310 INFO : Collected events spans a 32.681 [s] time interval\n", |
| "2016-04-12 15:15:41,310 INFO : Set plots time range to (0.000000, 32.681148)[s]\n" |
| ] |
| } |
| ], |
| "source": [ |
| "# Unlock device screen (assume no password required)\n", |
| "target.execute('input keyevent 82')\n", |
| "\n", |
| "# Run the benchmark in all the configured governors\n", |
| "for governor in confs:\n", |
| " test_dir = os.path.join(te.res_dir, governor)\n", |
| " res = experiment(governor, test_dir)\n", |
| " results[governor] = copy.deepcopy(res)" |
| ] |
| }, |
| { |
| "cell_type": "markdown", |
| "metadata": { |
| "collapsed": false |
| }, |
| "source": [ |
| "# UI Performance Analysis" |
| ] |
| }, |
| { |
| "cell_type": "code", |
| "execution_count": 11, |
| "metadata": { |
| "collapsed": false |
| }, |
| "outputs": [ |
| { |
| "name": "stdout", |
| "output_type": "stream", |
| "text": [ |
| "Frame Statistics for PERFORMANCE governor\n", |
| "Stats since: 26524169207ns\n", |
| "Total frames rendered: 10425\n", |
| "Janky frames: 1175 (11.27%)\n", |
| "90th percentile: 18ms\n", |
| "95th percentile: 24ms\n", |
| "99th percentile: 40ms\n", |
| "\n", |
| "Frame Statistics for INTERACTIVE governor\n", |
| "Stats since: 26524169207ns\n", |
| "Total frames rendered: 11673\n", |
| "Janky frames: 1265 (10.84%)\n", |
| "90th percentile: 17ms\n", |
| "95th percentile: 24ms\n", |
| "99th percentile: 40ms\n", |
| "\n" |
| ] |
| } |
| ], |
| "source": [ |
| "for governor in confs:\n", |
| " framestats_file = results[governor]['framestats_file']\n", |
| " print \"Frame Statistics for {} governor\".format(governor.upper())\n", |
| " !sed '/Stats since/,/99th/!d;/99th/q' $framestats_file\n", |
| " print \"\"" |
| ] |
| } |
| ], |
| "metadata": { |
| "kernelspec": { |
| "display_name": "Python 2", |
| "language": "python", |
| "name": "python2" |
| }, |
| "language_info": { |
| "codemirror_mode": { |
| "name": "ipython", |
| "version": 2 |
| }, |
| "file_extension": ".py", |
| "mimetype": "text/x-python", |
| "name": "python", |
| "nbconvert_exporter": "python", |
| "pygments_lexer": "ipython2", |
| "version": "2.7.6" |
| }, |
| "toc": { |
| "toc_cell": false, |
| "toc_number_sections": true, |
| "toc_threshold": 6, |
| "toc_window_display": false |
| } |
| }, |
| "nbformat": 4, |
| "nbformat_minor": 0 |
| } |