| <!-- HTML header for doxygen 1.8.9.1--> |
| <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> |
| <html xmlns="http://www.w3.org/1999/xhtml"> |
| <head> |
| <meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/> |
| <meta http-equiv="X-UA-Compatible" content="IE=9"/> |
| <meta name="generator" content="Doxygen 1.8.11"/> |
| <meta name="robots" content="NOINDEX, NOFOLLOW" /> <!-- Prevent indexing by search engines --> |
| <title>Compute Library: Validation and benchmarks tests</title> |
| <link href="tabs.css" rel="stylesheet" type="text/css"/> |
| <script type="text/javascript" src="jquery.js"></script> |
| <script type="text/javascript" src="dynsections.js"></script> |
| <link href="navtree.css" rel="stylesheet" type="text/css"/> |
| <script type="text/javascript" src="resize.js"></script> |
| <script type="text/javascript" src="navtreedata.js"></script> |
| <script type="text/javascript" src="navtree.js"></script> |
| <script type="text/javascript"> |
| $(document).ready(initResizable); |
| $(window).load(resizeHeight); |
| </script> |
| <link href="search/search.css" rel="stylesheet" type="text/css"/> |
| <script type="text/javascript" src="search/searchdata.js"></script> |
| <script type="text/javascript" src="search/search.js"></script> |
| <script type="text/javascript"> |
| $(document).ready(function() { init_search(); }); |
| </script> |
| <script type="text/x-mathjax-config"> |
| MathJax.Hub.Config({ |
| extensions: ["tex2jax.js"], |
| jax: ["input/TeX","output/HTML-CSS"], |
| }); |
| </script><script type="text/javascript" src="http://cdn.mathjax.org/mathjax/latest/MathJax.js"></script> |
| <link href="doxygen.css" rel="stylesheet" type="text/css" /> |
| </head> |
| <body> |
| <div id="top"><!-- do not remove this div, it is closed by doxygen! --> |
| <div id="titlearea"> |
| <table cellspacing="0" cellpadding="0"> |
| <tbody> |
| <tr style="height: 56px;"> |
| <td style="padding-left: 0.5em;"> |
| <div id="projectname">Compute Library |
|  <span id="projectnumber">v17.06</span> |
| </div> |
| </td> |
| </tr> |
| </tbody> |
| </table> |
| </div> |
| <!-- end header part --> |
| <!-- Generated by Doxygen 1.8.11 --> |
| <script type="text/javascript"> |
| var searchBox = new SearchBox("searchBox", "search",false,'Search'); |
| </script> |
| <div id="navrow1" class="tabs"> |
| <ul class="tablist"> |
| <li><a href="index.xhtml"><span>Main Page</span></a></li> |
| <li class="current"><a href="pages.xhtml"><span>Related Pages</span></a></li> |
| <li><a href="namespaces.xhtml"><span>Namespaces</span></a></li> |
| <li><a href="annotated.xhtml"><span>Data Structures</span></a></li> |
| <li><a href="files.xhtml"><span>Files</span></a></li> |
| <li> |
| <div id="MSearchBox" class="MSearchBoxInactive"> |
| <span class="left"> |
| <img id="MSearchSelect" src="search/mag_sel.png" |
| onmouseover="return searchBox.OnSearchSelectShow()" |
| onmouseout="return searchBox.OnSearchSelectHide()" |
| alt=""/> |
| <input type="text" id="MSearchField" value="Search" accesskey="S" |
| onfocus="searchBox.OnSearchFieldFocus(true)" |
| onblur="searchBox.OnSearchFieldFocus(false)" |
| onkeyup="searchBox.OnSearchFieldChange(event)"/> |
| </span><span class="right"> |
| <a id="MSearchClose" href="javascript:searchBox.CloseResultsWindow()"><img id="MSearchCloseImg" border="0" src="search/close.png" alt=""/></a> |
| </span> |
| </div> |
| </li> |
| </ul> |
| </div> |
| </div><!-- top --> |
| <div id="side-nav" class="ui-resizable side-nav-resizable"> |
| <div id="nav-tree"> |
| <div id="nav-tree-contents"> |
| <div id="nav-sync" class="sync"></div> |
| </div> |
| </div> |
| <div id="splitbar" style="-moz-user-select:none;" |
| class="ui-resizable-handle"> |
| </div> |
| </div> |
| <script type="text/javascript"> |
| $(document).ready(function(){initNavTree('tests.xhtml','');}); |
| </script> |
| <div id="doc-content"> |
| <!-- window showing the filter options --> |
| <div id="MSearchSelectWindow" |
| onmouseover="return searchBox.OnSearchSelectShow()" |
| onmouseout="return searchBox.OnSearchSelectHide()" |
| onkeydown="return searchBox.OnSearchSelectKey(event)"> |
| </div> |
| |
| <!-- iframe showing the search results (closed by default) --> |
| <div id="MSearchResultsWindow"> |
| <iframe src="javascript:void(0)" frameborder="0" |
| name="MSearchResults" id="MSearchResults"> |
| </iframe> |
| </div> |
| |
| <div class="header"> |
| <div class="headertitle"> |
| <div class="title">Validation and benchmarks tests </div> </div> |
| </div><!--header--> |
| <div class="contents"> |
| <div class="toc"><h3>Table of Contents</h3> |
| <ul><li class="level1"><a href="#building_test_dependencies">Building dependencies</a><ul><li class="level2"><a href="#building_boost">Building Boost</a></li> |
| <li class="level2"><a href="#building_google_benchmark">Building Google Benchmark</a></li> |
| </ul> |
| </li> |
| <li class="level1"><a href="#tests_running_tests">Running tests</a><ul><li class="level2"><a href="#tests_running_tests_benchmarking">Benchmarking</a><ul><li class="level3"><a href="#tests_running_tests_benchmarking_filter">Filter tests</a></li> |
| <li class="level3"><a href="#tests_running_tests_benchmarking_runtime">Runtime</a></li> |
| <li class="level3"><a href="#tests_running_tests_benchmarking_verbosity">Verbosity</a></li> |
| </ul> |
| </li> |
| <li class="level2"><a href="#tests_running_tests_validation">Validation</a><ul><li class="level3"><a href="#tests_running_tests_validation_filter">Filter tests</a></li> |
| <li class="level3"><a href="#tests_running_tests_validation_verbosity">Verbosity</a></li> |
| </ul> |
| </li> |
| </ul> |
| </li> |
| </ul> |
| </div> |
| <div class="textblock"><h1><a class="anchor" id="building_test_dependencies"></a> |
| Building dependencies</h1> |
| <p>The tests currently make use of Boost (Test and Program options) for validation and Google Benchmark for performance runs. Below are instructions about how to build these 3rd party libraries.</p> |
| <dl class="section note"><dt>Note</dt><dd>By default the build of the validation and benchmark tests is disabled, to enable it use <code>validation_tests=1</code> and <code>benchmark_tests=1</code></dd></dl> |
| <h2><a class="anchor" id="building_boost"></a> |
| Building Boost</h2> |
| <p>First follow the instructions from the Boost library on how to setup the Boost build system (<a href="http://www.boost.org/doc/libs/1_64_0/more/getting_started/index.html">http://www.boost.org/doc/libs/1_64_0/more/getting_started/index.html</a>). Afterwards the required libraries can be build with: </p><pre class="fragment">./b2 --with-program_options --with-test link=static \ |
| define=BOOST_TEST_ALTERNATIVE_INIT_API |
| </pre><p>Additionally, depending on your environment, it might be necessary to specify the <code>toolset=</code> option to choose the right compiler. Moreover, <code>address-model=32</code> can be used to force building for 32bit and <code>target-os=android</code> must be specified to build for Android.</p> |
| <p>After executing the build command the libraries <code>libboost_program_options.a</code> and <code>libboost_unit_test_framework.a</code> can be found in <code>./stage/lib</code>.</p> |
| <h2><a class="anchor" id="building_google_benchmark"></a> |
| Building Google Benchmark</h2> |
| <p>Instructions on how to build Google Benchmark using CMake can be found in their repository: <a href="https://github.com/google/benchmark">https://github.com/google/benchmark</a>. For example, building for Android 32bit can be achieved via </p><pre class="fragment">cmake -DCMAKE_BUILD_TYPE=Release \ |
| -DCMAKE_CXX_COMPILER=arm-linux-androideabi-clang++ \ |
| -DBENCHMARK_ENABLE_LTO=false -DBENCHMARK_ENABLE_TESTING=false .. |
| </pre><p>The library required by the compute library is <code>libbenchmark.a</code>.</p> |
| <h1><a class="anchor" id="tests_running_tests"></a> |
| Running tests</h1> |
| <h2><a class="anchor" id="tests_running_tests_benchmarking"></a> |
| Benchmarking</h2> |
| <h3><a class="anchor" id="tests_running_tests_benchmarking_filter"></a> |
| Filter tests</h3> |
| <p>All tests can be run by invoking </p><pre class="fragment">./arm_compute_benchmark -- ./data |
| </pre><p>where <code>./data</code> contains the assets needed by the tests.</p> |
| <p>If only a subset of the tests has to be executed the <code>--benchmark_filter</code> option takes a regular expression to select matching tests. </p><pre class="fragment">./arm_compute_benchmark --benchmark_filter=neon_bitwise_and ./data |
| </pre><p>All available tests can be displayed with the <code>--benchmark_list_tests</code> switch. </p><pre class="fragment">./arm_compute_benchmark --benchmark_list_tests ./data |
| </pre><h3><a class="anchor" id="tests_running_tests_benchmarking_runtime"></a> |
| Runtime</h3> |
| <p>By default every test is run multiple <em>iterations</em> until a minimum time is reached. The minimum time (in seconds) can be controlled with the <code>--benchmark_min_time</code> flag. However, each test might have a hard coded value for the number of iterations or minimum execution time. In that case the command line argument is ignored for those specific tests. Additionally it is possible to specify multiple <em>repetitions</em> (<code>--benchmark_repetitions</code>) which will run each test multiple times (including the iterations). The average and standard deviation for all repetitions is automatically computed and reported.</p> |
| <h3><a class="anchor" id="tests_running_tests_benchmarking_verbosity"></a> |
| Verbosity</h3> |
| <p>The verbosity of the test output can be controlled via the <code>--v</code> flag. Though it should hardly ever be necessary.</p> |
| <h2><a class="anchor" id="tests_running_tests_validation"></a> |
| Validation</h2> |
| <h3><a class="anchor" id="tests_running_tests_validation_filter"></a> |
| Filter tests</h3> |
| <p>All tests can be run by invoking </p><pre class="fragment">./arm_compute_validation -- ./data |
| </pre><p>where <code>./data</code> contains the assets needed by the tests.</p> |
| <p>As running all tests can take a lot of time the suite is split into "precommit" and "nightly" tests. The precommit tests will be fast to execute but still cover the most important features. In contrast the nightly tests offer more extensive coverage but take longer. The different subsets can be selected from the command line as follows: </p><pre class="fragment">./arm_compute_validation -t @precommit -- ./data |
| ./arm_compute_validation -t @nightly -- ./data |
| </pre><p>Additionally it is possible to select specific suites or tests: </p><pre class="fragment">./arm_compute_validation -t CL -- ./data |
| ./arm_compute_validation -t NEON/BitwiseAnd/RunSmall/_0 -- ./data |
| </pre><p>All available tests can be displayed with the <code>--list_content</code> switch. </p><pre class="fragment">./arm_compute_validation --list_content -- ./data |
| </pre><p>For a complete list of possible selectors please see: <a href="http://www.boost.org/doc/libs/1_64_0/libs/test/doc/html/boost_test/runtime_config/test_unit_filtering.html">http://www.boost.org/doc/libs/1_64_0/libs/test/doc/html/boost_test/runtime_config/test_unit_filtering.html</a></p> |
| <h3><a class="anchor" id="tests_running_tests_validation_verbosity"></a> |
| Verbosity</h3> |
| <p>There are two separate flags to control the verbosity of the test output. <code>--report_level</code> controls the verbosity of the summary produced after all tests have been executed. <code>--log_level</code> controls the verbosity of the information generated during the execution of tests. All available settings can be found in the Boost documentation for <a href="http://www.boost.org/doc/libs/1_64_0/libs/test/doc/html/boost_test/utf_reference/rt_param_reference/report_level.html">–report_level</a> and <a href="http://www.boost.org/doc/libs/1_64_0/libs/test/doc/html/boost_test/utf_reference/rt_param_reference/log_level.html">–log_level</a>, respectively. </p> |
| </div></div><!-- contents --> |
| </div><!-- doc-content --> |
| <!-- start footer part --> |
| <div id="nav-path" class="navpath"><!-- id is needed for treeview function! --> |
| <ul> |
| <li class="footer">Generated on Fri Jun 23 2017 15:44:34 for Compute Library by |
| <a href="http://www.doxygen.org/index.html"> |
| <img class="footer" src="doxygen.png" alt="doxygen"/></a> 1.8.11 </li> |
| </ul> |
| </div> |
| </body> |
| </html> |