commit | 28f423de2270a79edc30317d39fbb6e7b3d90f8c | [log] [tgz] |
---|---|---|
author | Patrick Bellasi <patrick.bellasi@arm.com> | Mon Nov 16 16:03:37 2015 +0000 |
committer | Patrick Bellasi <patrick.bellasi@arm.com> | Fri Jan 22 11:29:21 2016 +0000 |
tree | 04d4d2e09246b3a07f70439f51dbcdafc69435d0 | |
parent | 680ad8b9c70754064b686e1fcea6db022bc5e152 [diff] |
libs/utils/perf_analysis: demote some logging to DEBUG level Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>
This project provides a collection of tools to support regression testing and interactive analysis of workload behavior. Its main goal is to support Linux kernel developer to assess the impact of modification in core parts of the kernel. The main focus is on scheduler and power management frameworks, however the toolkit is generic enough to accommodate other usages.
The toolkit is based on a set of core libraries:
TODO: add a figure which represent the toolkit structure
This notes assumes an installation from scratch on a freshly installed Debian system.
$ sudo apt-get install build-essential autoconf automake libtool
$ sudo apt-get install python-matplotlib python-numpy
$ sudo apt-get install python-pip python-dev
$ sudo pip install trappy bart-py devlib
TODO: add notes on how to clone and initialize the repostiory
The toolkit provides these resources:
A collection of "assets" required to run examples and tests: . |-- assets | `-- mp3-short.json
A script to setup the test environment.
|-- init_env
A collection of IPython Notebooks grouped by topic:
|-- ipynb | |-- devlib | |-- ipyserver_start | |-- ipyserver_stop | |-- nohup.out | |-- sched_dvfs | |-- utils | `-- wlgen
A set of support libraries which allows to easily develop new IPython notebooks and tests:
|-- libs | |-- __init__.py | |-- utils | `-- wlgen
A JSON based configuration for the target device to use for running the tests:
|-- target.config
A collection of regression tests grouped by topic:
|-- tests | `-- eas
A collection of binary tools, pre-compiled for different architectures, which are required to run tests on the target device:
`-- tools |-- arm64 |-- armeabi |-- plots.py |-- report.py |-- scripts `-- x86
This section provide a quick start guide on understanding the toolkit by guiding the user thought a set of example usages.
IPython notebook is a web based interactive python programming interface. This toolkit provides a set of IPython notebooks ready to use. To use one of these notebooks you first need to start a simple IPython server.
# Enter the ipynb folder $ cd ipynb # Start the server $ ./ipyserver_start lo
This will start the server and open the index page in a new browser tab. If the index is not automatically loaded in your browser, visit the link reported by the server startup script.
The index page is an HTML representation of the local ipynb folder. From that page we can access all the IPython notebooks provided by the toolkit.
Usually notebooks and tests make use of the TestEnv class to initialize and access a remote target device. To get an overall view of the functionalities exposed by this class you can have a look at this notebook: utils/testenv_example.ipynb
RT-App is a configurable synthetic workload generator used by many tests and notebooks to run experiments on a target. The toolkit provides a python API to easy the definition of RT-App based workloads and their execution on a target.
This notebook: wlgen/simple_rtapp.ipynb is a complete example of experiment setup, execution and data collection.
Specifically it shows how to:
One of the main aims of this toolkit is to become a repository for regression tests on scheduler and power management behavior. A common pattern on defining new test cases is to start from an IPython notebook to design an experiment and compute metrics of interest. The notebook can than be converted into a self contained test to run in batch mode.
An example of such a notebook is: sched_dvfs/smoke_test.ipynb
In this notebook the toolkit API is more extensively used to defined an experiment which allows to:
The notebook compare three different CPUFreq governor: "performance", "sched" and "ondemand". New configurations are easy to add. For each configuration the notebook generate plots and tabular reports regarding working frequencies and energy consumption.
This notebook is a good example of usage of the toolkit to define a new set of experiments which can than be transformed into a standalone regression test.