Marc Bonnici | 60e69fc | 2017-08-18 17:23:18 +0100 | [diff] [blame] | 1 | Derived Measurements |
| 2 | ===================== |
| 3 | |
| 4 | |
| 5 | The ``DerivedMeasurements`` API provides a consistent way of performing post |
| 6 | processing on a provided :class:`MeasurementCsv` file. |
| 7 | |
| 8 | Example |
| 9 | ------- |
| 10 | |
| 11 | The following example shows how to use an implementation of a |
Sergei Trofimov | dd26b43 | 2017-09-07 11:13:34 +0100 | [diff] [blame] | 12 | :class:`DerivedMeasurement` to obtain a list of calculated ``DerivedMetric``'s. |
Marc Bonnici | 60e69fc | 2017-08-18 17:23:18 +0100 | [diff] [blame] | 13 | |
| 14 | .. code-block:: ipython |
| 15 | |
| 16 | # Import the relevant derived measurement module |
| 17 | # in this example the derived energy module is used. |
| 18 | In [1]: from devlib import DerivedEnergyMeasurements |
| 19 | |
| 20 | # Obtain a MeasurementCsv file from an instrument or create from |
| 21 | # existing .csv file. In this example an existing csv file is used which was |
| 22 | # created with a sampling rate of 100Hz |
| 23 | In [2]: from devlib import MeasurementsCsv |
| 24 | In [3]: measurement_csv = MeasurementsCsv('/example/measurements.csv', sample_rate_hz=100) |
| 25 | |
| 26 | # Process the file and obtain a list of the derived measurements |
| 27 | In [4]: derived_measurements = DerivedEnergyMeasurements.process(measurement_csv) |
| 28 | |
| 29 | In [5]: derived_measurements |
| 30 | Out[5]: [device_energy: 239.1854075 joules, device_power: 5.5494089227 watts] |
| 31 | |
| 32 | API |
| 33 | --- |
| 34 | |
| 35 | Derived Measurements |
| 36 | ~~~~~~~~~~~~~~~~~~~~ |
| 37 | |
Sergei Trofimov | adf25f9 | 2017-09-07 14:19:24 +0100 | [diff] [blame] | 38 | .. class:: DerivedMeasurements |
Marc Bonnici | 60e69fc | 2017-08-18 17:23:18 +0100 | [diff] [blame] | 39 | |
Sergei Trofimov | adf25f9 | 2017-09-07 14:19:24 +0100 | [diff] [blame] | 40 | The ``DerivedMeasurements`` class provides an API for post-processing |
| 41 | instrument output offline (i.e. without a connection to the target device) to |
| 42 | generate additional metrics. |
Marc Bonnici | 60e69fc | 2017-08-18 17:23:18 +0100 | [diff] [blame] | 43 | |
| 44 | .. method:: DerivedMeasurements.process(measurement_csv) |
| 45 | |
Sergei Trofimov | adf25f9 | 2017-09-07 14:19:24 +0100 | [diff] [blame] | 46 | Process a :class:`MeasurementsCsv`, returning a list of |
| 47 | :class:`DerivedMetric` and/or :class:`MeasurementsCsv` objects that have been |
| 48 | derived from the input. The exact nature and ordering of the list memebers |
| 49 | is specific to indivial 'class'`DerivedMeasurements` implementations. |
| 50 | |
| 51 | .. method:: DerivedMeasurements.process_raw(\*args) |
| 52 | |
| 53 | Process raw output from an instrument, returnin a list :class:`DerivedMetric` |
| 54 | and/or :class:`MeasurementsCsv` objects that have been derived from the |
| 55 | input. The exact nature and ordering of the list memebers is specific to |
| 56 | indivial 'class'`DerivedMeasurements` implewmentations. |
| 57 | |
| 58 | The arguents to this method should be paths to raw output files generated by |
| 59 | an instrument. The number and order of expected arguments is specific to |
| 60 | particular implmentations. |
Marc Bonnici | 60e69fc | 2017-08-18 17:23:18 +0100 | [diff] [blame] | 61 | |
| 62 | |
Sergei Trofimov | dd26b43 | 2017-09-07 11:13:34 +0100 | [diff] [blame] | 63 | Derived Metric |
| 64 | ~~~~~~~~~~~~~~ |
| 65 | |
| 66 | .. class:: DerivedMetric |
| 67 | |
| 68 | Represents a metric derived from previously collected ``Measurement``s. |
| 69 | Unlike, a ``Measurement``, this was not measured directly from the target. |
| 70 | |
| 71 | |
| 72 | .. attribute:: DerivedMetric.name |
| 73 | |
| 74 | The name of the derived metric. This uniquely defines a metric -- two |
| 75 | ``DerivedMetric`` objects with the same ``name`` represent to instances of |
| 76 | the same metric (e.g. computed from two different inputs). |
| 77 | |
| 78 | .. attribute:: DerivedMetric.value |
| 79 | |
| 80 | The ``numeric`` value of the metric that has been computed for a particular |
| 81 | input. |
| 82 | |
| 83 | .. attribute:: DerivedMetric.measurement_type |
| 84 | |
| 85 | The ``MeasurementType`` of the metric. This indicates which conceptual |
| 86 | category the metric falls into, its units, and conversions to other |
| 87 | measurement types. |
| 88 | |
| 89 | .. attribute:: DerivedMetric.units |
| 90 | |
| 91 | The units in which the metric's value is expressed. |
| 92 | |
Marc Bonnici | 60e69fc | 2017-08-18 17:23:18 +0100 | [diff] [blame] | 93 | |
| 94 | Available Derived Measurements |
| 95 | ------------------------------- |
Marc Bonnici | 60e69fc | 2017-08-18 17:23:18 +0100 | [diff] [blame] | 96 | |
Sergei Trofimov | adf25f9 | 2017-09-07 14:19:24 +0100 | [diff] [blame] | 97 | .. note:: If a method of the API is not documented for a particular |
| 98 | implementation, that means that it s not overriden by that |
| 99 | implementation. It is still safe to call it -- an empty list will be |
| 100 | returned. |
Marc Bonnici | 60e69fc | 2017-08-18 17:23:18 +0100 | [diff] [blame] | 101 | |
Sergei Trofimov | adf25f9 | 2017-09-07 14:19:24 +0100 | [diff] [blame] | 102 | Energy |
| 103 | ~~~~~~ |
| 104 | |
| 105 | .. class:: DerivedEnergyMeasurements |
| 106 | |
| 107 | The ``DerivedEnergyMeasurements`` class is used to calculate average power and |
| 108 | cumulative energy for each site if the required data is present. |
| 109 | |
| 110 | The calculation of cumulative energy can occur in 3 ways. If a |
| 111 | ``site`` contains ``energy`` results, the first and last measurements are extracted |
| 112 | and the delta calculated. If not, a ``timestamp`` channel will be used to calculate |
| 113 | the energy from the power channel, failing back to using the sample rate attribute |
| 114 | of the :class:`MeasurementCsv` file if timestamps are not available. If neither |
| 115 | timestamps or a sample rate are available then an error will be raised. |
Marc Bonnici | 60e69fc | 2017-08-18 17:23:18 +0100 | [diff] [blame] | 116 | |
| 117 | |
| 118 | .. method:: DerivedEnergyMeasurements.process(measurement_csv) |
| 119 | |
Sergei Trofimov | adf25f9 | 2017-09-07 14:19:24 +0100 | [diff] [blame] | 120 | This will return total cumulative energy for each energy channel, and the |
| 121 | average power for each power channel in the input CSV. The output will contain |
| 122 | all energy metrics followed by power metrics. The ordering of both will match |
| 123 | the ordering of channels in the input. The metrics will by named based on the |
| 124 | sites of the coresponding channels according to the following patters: |
| 125 | ``"<site>_total_energy"`` and ``"<site>_average_power"``. |
Marc Bonnici | 60e69fc | 2017-08-18 17:23:18 +0100 | [diff] [blame] | 126 | |
Sergei Trofimov | f692315 | 2017-09-07 17:36:18 +0100 | [diff] [blame] | 127 | |
| 128 | FPS / Rendering |
| 129 | ~~~~~~~~~~~~~~~ |
| 130 | |
| 131 | .. class:: DerivedGfxInfoStats(drop_threshold=5, suffix='-fps', filename=None, outdir=None) |
| 132 | |
| 133 | Produces FPS (frames-per-second) and other dervied statistics from |
| 134 | :class:`GfxInfoFramesInstrument` output. This takes several optional |
| 135 | parameters in creation: |
| 136 | |
| 137 | :param drop_threshold: FPS in an application, such as a game, which this |
| 138 | processor is primarily targeted at, cannot reasonably |
| 139 | drop to a very low value. This is specified to this |
| 140 | threhold. If an FPS for a frame is computed to be |
| 141 | lower than this treshold, it will be dropped on the |
| 142 | assumption that frame rednering was suspended by the |
| 143 | system (e.g. when idling), or there was some sort of |
| 144 | error, and therefore this should be used in |
| 145 | performance calculations. defaults to ``5``. |
| 146 | :param suffix: The name of the gerated per-frame FPS csv file will be |
| 147 | derived from the input frames csv file by appending this |
| 148 | suffix. This cannot be specified at the same time as |
| 149 | a ``filename``. |
| 150 | :param filename: As an alternative to the suffix, a complete file name for |
| 151 | FPS csv can be specified. This cannot be used at the same |
| 152 | time as the ``suffix``. |
| 153 | :param outdir: By default, the FPS csv file will be placed in the same |
| 154 | directory as the input frames csv file. This can be changed |
| 155 | by specifying an alternate directory here |
| 156 | |
| 157 | .. warning:: Specifying both ``filename`` and ``oudir`` will mean that exactly |
| 158 | the same file will be used for FPS output on each invocation of |
| 159 | ``process()`` (even for different inputs) resulting in previous |
| 160 | results being overwritten. |
| 161 | |
| 162 | .. method:: DerivedGfxInfoStats.process(measurement_csv) |
| 163 | |
| 164 | Process the fames csv generated by :class:`GfxInfoFramesInstrument` and |
| 165 | returns a list containing exactly three entries: :class:`DerivedMetric`\ s |
| 166 | ``fps`` and ``total_frames``, followed by a :class:`MeasurentCsv` containing |
| 167 | per-frame FPSs values. |
| 168 | |
| 169 | .. method:: DerivedGfxInfoStats.process_raw(gfxinfo_frame_raw_file) |
| 170 | |
| 171 | As input, this takes a single argument, which should be the path to the raw |
| 172 | output file of :class:`GfxInfoFramesInstrument`. The returns stats |
| 173 | accumulated by gfxinfo. At the time of wrinting, the stats (in order) are: |
| 174 | ``janks``, ``janks_pc`` (percentage of all frames), |
| 175 | ``render_time_50th_ptile`` (50th percentile, or median, for time to render a |
| 176 | frame), ``render_time_90th_ptile``, ``render_time_95th_ptile``, |
| 177 | ``render_time_99th_ptile``, ``missed_vsync``, ``hight_input_latency``, |
| 178 | ``slow_ui_thread``, ``slow_bitmap_uploads``, ``slow_issue_draw_commands``. |
| 179 | Please see the `gfxinfo documentation`_ for details. |
| 180 | |
| 181 | .. _gfxinfo documentation: https://developer.android.com/training/testing/performance.html |
Sergei Trofimov | 9f66632 | 2017-09-13 11:45:55 +0100 | [diff] [blame] | 182 | |
| 183 | |
| 184 | .. class:: DerivedSurfaceFlingerStats(drop_threshold=5, suffix='-fps', filename=None, outdir=None) |
| 185 | |
| 186 | Produces FPS (frames-per-second) and other dervied statistics from |
| 187 | :class:`SurfaceFlingerFramesInstrument` output. This takes several optional |
| 188 | parameters in creation: |
| 189 | |
| 190 | :param drop_threshold: FPS in an application, such as a game, which this |
| 191 | processor is primarily targeted at, cannot reasonably |
| 192 | drop to a very low value. This is specified to this |
| 193 | threhold. If an FPS for a frame is computed to be |
| 194 | lower than this treshold, it will be dropped on the |
| 195 | assumption that frame rednering was suspended by the |
| 196 | system (e.g. when idling), or there was some sort of |
| 197 | error, and therefore this should be used in |
| 198 | performance calculations. defaults to ``5``. |
| 199 | :param suffix: The name of the gerated per-frame FPS csv file will be |
| 200 | derived from the input frames csv file by appending this |
| 201 | suffix. This cannot be specified at the same time as |
| 202 | a ``filename``. |
| 203 | :param filename: As an alternative to the suffix, a complete file name for |
| 204 | FPS csv can be specified. This cannot be used at the same |
| 205 | time as the ``suffix``. |
| 206 | :param outdir: By default, the FPS csv file will be placed in the same |
| 207 | directory as the input frames csv file. This can be changed |
| 208 | by specifying an alternate directory here |
| 209 | |
| 210 | .. warning:: Specifying both ``filename`` and ``oudir`` will mean that exactly |
| 211 | the same file will be used for FPS output on each invocation of |
| 212 | ``process()`` (even for different inputs) resulting in previous |
| 213 | results being overwritten. |
| 214 | |
| 215 | .. method:: DerivedSurfaceFlingerStats.process(measurement_csv) |
| 216 | |
| 217 | Process the fames csv generated by :class:`SurfaceFlingerFramesInstrument` and |
| 218 | returns a list containing exactly three entries: :class:`DerivedMetric`\ s |
| 219 | ``fps`` and ``total_frames``, followed by a :class:`MeasurentCsv` containing |
| 220 | per-frame FPSs values, followed by ``janks`` ``janks_pc``, and |
| 221 | ``missed_vsync`` metrics. |