doc/derived_measurements.rst

Derived Measurements
=====================


The ``DerivedMeasurements`` API provides a consistent way of performing post
processing on a provided :class:`MeasurementCsv` file.

Example
-------

The following example shows how to use an implementation of a
:class:`DerivedMeasurement` to obtain a list of calculated ``DerivedMetric``'s.

.. code-block:: ipython

    # Import the relevant derived measurement module
    # in this example the derived energy module is used.
    In [1]: from devlib import DerivedEnergyMeasurements

    # Obtain a MeasurementCsv file from an instrument or create from
    # existing .csv file. In this example an existing csv file is used which was
    # created with a sampling rate of 100Hz
    In [2]: from devlib import MeasurementsCsv
    In [3]: measurement_csv = MeasurementsCsv('/example/measurements.csv', sample_rate_hz=100)

    # Process the file and obtain a list of the derived measurements
    In [4]: derived_measurements = DerivedEnergyMeasurements.process(measurement_csv)

    In [5]: derived_measurements
    Out[5]: [device_energy: 239.1854075 joules, device_power: 5.5494089227 watts]

API
---

Derived Measurements
~~~~~~~~~~~~~~~~~~~~

.. class:: DerivedMeasurements

   The ``DerivedMeasurements`` class provides an API for post-processing
   instrument output offline (i.e. without a connection to the target device) to
   generate additional metrics.

.. method:: DerivedMeasurements.process(measurement_csv)

   Process a :class:`MeasurementsCsv`, returning  a list of
   :class:`DerivedMetric` and/or :class:`MeasurementsCsv` objects that have been
   derived from the input. The exact nature and ordering of the list memebers
   is specific to indivial 'class'`DerivedMeasurements` implementations.

.. method:: DerivedMeasurements.process_raw(\*args)

   Process raw output from an instrument, returnin a list :class:`DerivedMetric`
   and/or :class:`MeasurementsCsv` objects that have been derived from the
   input. The exact nature and ordering of the list memebers is specific to
   indivial 'class'`DerivedMeasurements` implewmentations.

   The arguents to this method should be paths to raw output files generated by
   an instrument. The number and order of expected arguments is specific to
   particular implmentations.


Derived Metric
~~~~~~~~~~~~~~

.. class:: DerivedMetric

  Represents a metric derived from previously collected ``Measurement``s.
  Unlike, a ``Measurement``, this was not measured directly from the target.


.. attribute:: DerivedMetric.name

   The name of the derived metric. This uniquely defines a metric -- two
   ``DerivedMetric`` objects with the same ``name`` represent to instances of
   the same metric (e.g. computed from two different inputs).

.. attribute:: DerivedMetric.value

   The ``numeric`` value of the metric that has been computed for a particular
   input.

.. attribute:: DerivedMetric.measurement_type

   The ``MeasurementType`` of the metric. This indicates which conceptual
   category the metric falls into, its units, and conversions to other
   measurement types.

.. attribute:: DerivedMetric.units

   The units in which the metric's value is expressed.


Available Derived Measurements
-------------------------------

.. note:: If a method of the API is not documented for a particular
          implementation, that means that it s not overriden by that
          implementation. It is still safe to call it -- an empty list will be
          returned.

Energy
~~~~~~

.. class:: DerivedEnergyMeasurements

   The ``DerivedEnergyMeasurements`` class is used to calculate average power and
   cumulative energy for each site if the required data is present.

   The calculation of cumulative energy can occur in 3 ways. If a
   ``site`` contains ``energy`` results, the first and last measurements are extracted
   and the delta calculated. If not, a ``timestamp`` channel will be used to calculate
   the energy from the power channel, failing back to using the sample rate attribute
   of the :class:`MeasurementCsv` file if timestamps are not available. If neither
   timestamps or a sample rate are available then an error will be raised.


.. method:: DerivedEnergyMeasurements.process(measurement_csv)

   This will return total cumulative energy for each energy channel, and the
   average power for each power channel in the input CSV. The output will contain
   all energy metrics followed by power metrics. The ordering of both will match
   the ordering of channels in the input. The metrics will by named based on the
   sites of the coresponding channels according to the following patters:
   ``"<site>_total_energy"`` and ``"<site>_average_power"``.


FPS / Rendering
~~~~~~~~~~~~~~~

.. class:: DerivedGfxInfoStats(drop_threshold=5, suffix='-fps', filename=None, outdir=None)

   Produces FPS (frames-per-second) and other dervied statistics from
   :class:`GfxInfoFramesInstrument` output. This takes several optional
   parameters in creation:

   :param drop_threshold: FPS in an application, such as a game, which this
                          processor is primarily targeted at, cannot reasonably
                          drop to a very low value. This is specified to this
                          threhold. If an FPS for a frame is computed to be
                          lower than this treshold, it will be dropped on the
                          assumption that frame rednering was suspended by the
                          system (e.g. when idling), or there was some sort of
                          error, and therefore this should be used in
                          performance calculations. defaults to ``5``.
   :param  suffix: The name of the gerated per-frame FPS csv file will be
                   derived from the input frames csv file by appending this
                   suffix. This cannot be specified at the same time as
                   a ``filename``.
   :param filename: As an alternative to the suffix, a complete file name for
                    FPS csv can be specified. This cannot be used at the same
                    time as the ``suffix``.
   :param outdir: By default, the FPS csv file will be placed in the same
                  directory as the input frames csv file. This can be changed
                  by specifying an alternate directory here

   .. warning:: Specifying both ``filename`` and ``oudir`` will mean that exactly
                the same file will be used for FPS output on each invocation of
                ``process()`` (even for different inputs) resulting in previous
                results being overwritten.

.. method:: DerivedGfxInfoStats.process(measurement_csv)

   Process the fames csv generated by :class:`GfxInfoFramesInstrument` and
   returns a list containing exactly three entries: :class:`DerivedMetric`\ s
   ``fps`` and ``total_frames``, followed by a :class:`MeasurentCsv` containing
   per-frame FPSs values.

.. method:: DerivedGfxInfoStats.process_raw(gfxinfo_frame_raw_file)

   As input, this takes a single argument, which should be the path to the raw
   output file of  :class:`GfxInfoFramesInstrument`. The returns stats
   accumulated by gfxinfo. At the time of wrinting, the stats (in order) are:
   ``janks``, ``janks_pc`` (percentage of all frames),
   ``render_time_50th_ptile`` (50th percentile, or median, for time to render a
   frame), ``render_time_90th_ptile``, ``render_time_95th_ptile``,
   ``render_time_99th_ptile``, ``missed_vsync``, ``hight_input_latency``,
   ``slow_ui_thread``, ``slow_bitmap_uploads``, ``slow_issue_draw_commands``.
   Please see the `gfxinfo documentation`_ for details.

.. _gfxinfo documentation: https://developer.android.com/training/testing/performance.html


.. class:: DerivedSurfaceFlingerStats(drop_threshold=5, suffix='-fps', filename=None, outdir=None)

   Produces FPS (frames-per-second) and other dervied statistics from
   :class:`SurfaceFlingerFramesInstrument` output. This takes several optional
   parameters in creation:

   :param drop_threshold: FPS in an application, such as a game, which this
                          processor is primarily targeted at, cannot reasonably
                          drop to a very low value. This is specified to this
                          threhold. If an FPS for a frame is computed to be
                          lower than this treshold, it will be dropped on the
                          assumption that frame rednering was suspended by the
                          system (e.g. when idling), or there was some sort of
                          error, and therefore this should be used in
                          performance calculations. defaults to ``5``.
   :param  suffix: The name of the gerated per-frame FPS csv file will be
                   derived from the input frames csv file by appending this
                   suffix. This cannot be specified at the same time as
                   a ``filename``.
   :param filename: As an alternative to the suffix, a complete file name for
                    FPS csv can be specified. This cannot be used at the same
                    time as the ``suffix``.
   :param outdir: By default, the FPS csv file will be placed in the same
                  directory as the input frames csv file. This can be changed
                  by specifying an alternate directory here

   .. warning:: Specifying both ``filename`` and ``oudir`` will mean that exactly
                the same file will be used for FPS output on each invocation of
                ``process()`` (even for different inputs) resulting in previous
                results being overwritten.

.. method:: DerivedSurfaceFlingerStats.process(measurement_csv)

   Process the fames csv generated by :class:`SurfaceFlingerFramesInstrument` and
   returns a list containing exactly three entries: :class:`DerivedMetric`\ s
   ``fps`` and ``total_frames``, followed by a :class:`MeasurentCsv` containing
   per-frame FPSs values, followed by ``janks`` ``janks_pc``, and
   ``missed_vsync`` metrics.