Measuring the performance of CanvasKit using Puppeteer and Chrome.
Run npm ci to install the dependencies need to run the tests. In //modules/canvaskit, run make release to build the canvaskit that will be used. With modifications to the Makefile, other builds (e.g. make profile) can be used as well.
If needed, one can download the lottie-samples and/or skp assets from CIPD using the sk tool:
sk asset download lottie-samples ~/Downloads/lottie-samples sk asset download skps ~/Downloads/skps
The actual location that these assets can be downloaded to is not important - the Makefile assumes them to be in Downloads, but that can be modified by the local user.
We have a harness for running benchmarks. Benchmark code snippets can be added to canvas_perf.js. The harness itself is the canvas_perf.html and benchmark.js. It will run the "test" portion of the code on multiple frames and gather data.
To run the benchmarks, run make perf_js. By default, this will use the most recent release build of canvaskit done locally. If you want to only run one or a few, modify the canvas_perf.js file by changing the relevent tests.push to onlytests.push and then run make perf_js.
On the CI, the results from these tests are uploaded to Perf. For example: https://perf.skia.org/e/?queries=test%3Dcanvas_drawOval We include metrics such as the 90th, 95th, and 99th percentile frame, average frame time, median frame time, and standard deviation. There are three types of measurements: without_flush_ms is the measurement of the test() function; with_flush_ms is the measurement of test() and the subsequent flush() call; total_frame_ms is the frame-to-frame time. Frame-to-frame is important to measure because it accounts for any work the GPU needs to do, even after CanvasKit flushes.
There is a harness that gathers data about rendering 600 frames of a skottie animation, cycling through it in a similar fashion to how it would be displayed to a user (e.g. as it is on skottie.skia.org).
To test it locally with a specific skottie animation, feel free to modify the Makefile to adjust the input_lottie argument and then run make frames. The harness itself is skottie-frames.html and benchmark.js.
On the CI, the results from these tests are uploaded to Perf. For example: https://perf.skia.org/e/?queries=test%3Dlego_loader We include metrics such as the first 5 frame times, average frame times, 90th, 95th and 99th percentile frame time.
There is a harness that repeatedly will draw an SKP and measure various metrics. This is handled by skottie-frames.html and benchmark.js. As before, feel free to modify the Makefile (the input_skp argument) and run make skp.
On the CI, the results from these tests are uploaded to Perf. For example: https://perf.skia.org/e/?queries=binary%3DCanvasKit%26test%3Ddesk_chalkboard.skp