Jean-Luc Brouillet | b636349 | 2021-04-19 01:11:17 -0700 | [diff] [blame] | 1 | # RenderScript Intrinsics Replacement Toolkit - v0.8 BETA |
| 2 | |
| 3 | This Toolkit provides a collection of high-performance image manipulation functions |
| 4 | like blur, blend, and resize. It can be used as a stand-alone replacement for most |
| 5 | of the deprecated RenderScript Intrinsics functions. |
| 6 | |
| 7 | The Toolkit provides ten image manipulation functions: |
| 8 | * blend, |
| 9 | * blur, |
| 10 | * color matrix, |
| 11 | * convolve, |
| 12 | * histogram and histogramDot, |
| 13 | * LUT (lookup table) and LUT 3D, |
| 14 | * resize, and |
| 15 | * YUV to RGB. |
| 16 | |
| 17 | The Toolkit provides a C++ and a Java/Kotlin interface. It is packaged as an Android |
| 18 | library that you can add to your project. |
| 19 | |
| 20 | These functions execute multithreaded on the CPU. They take advantage of Neon/AdvSimd |
| 21 | on Arm processors and SSE on Intel's. |
| 22 | |
Xusong Wang | 9a70eae | 2021-10-12 17:49:56 -0700 | [diff] [blame] | 23 | Compared to the RenderScript Intrinsics, this Toolkit is simpler to use and twice as fast |
Jean-Luc Brouillet | b636349 | 2021-04-19 01:11:17 -0700 | [diff] [blame] | 24 | when executing on the CPU. However RenderScript Intrinsics allow more flexibility for |
| 25 | the type of allocations supported. This toolkit does not support allocations of floats; |
| 26 | most the functions support ByteArrays and Bitmaps. |
| 27 | |
| 28 | You should instantiate the Toolkit once and reuse it throughout your application. |
| 29 | On instantiation, the Toolkit creates a thread pool that's used for processing all the functions. |
| 30 | You can limit the number of poolThreads used by the Toolkit via the constructor. The poolThreads |
| 31 | are destroyed once the Toolkit is destroyed, after any pending work is done. |
| 32 | |
| 33 | This library is thread safe. You can call methods from different poolThreads. The functions will |
| 34 | execute sequentially. |
| 35 | |
| 36 | |
| 37 | ## Future improvement ideas: |
| 38 | |
| 39 | * Turn the Java version of the Toolkit into a singleton, to reduce the chance that someone inadventarly |
| 40 | create multiple threadpools. |
| 41 | |
| 42 | * Support ByteBuffer. It should be straightforward to use GetDirectBufferAddress in JniEntryPoints.cpp. |
| 43 | See https://developer.android.com/training/articles/perf-jni and jni_helper.h. |
| 44 | |
| 45 | * The RenderScript Intrinsics support floats for colorMatrix, convolve, and resize. The Toolkit does not. |
| 46 | |
| 47 | * Allow in place update of buffers, or writing to an existing byte array. |
| 48 | |
| 49 | * For Blur, we could have a version that accepts a mask. This is commonly used for background |
| 50 | blurring. We should allow the mask to be smaller than the original, since neural networks models |
| 51 | that do segmentation are downscaled. |
| 52 | |
| 53 | * Allow yuvToRgb to have a Restriction. |
| 54 | |
| 55 | * Add support for YUV_420_888, the YUV format favored by Camera2. Allow various strides to be specified. |
| 56 | |
| 57 | * When passing a Restriction, it would be nice to say "Create a smaller output". |
| 58 | The original RenderScript does not allow that. It's not that useful when outputing new buffers as |
| 59 | our Java library does. |
| 60 | |
| 61 | * For Resize, Restriction working on input buffer would be more useful but that's not RenderScript. |
| 62 | |
| 63 | * Integrate and test with imageprocessing_jb. Do the same for [github/renderscript-samples/](https://github.com/android/renderscript-samples/tree/main/RenderScriptIntrinsic) |
| 64 | |
| 65 | * Allow Bitmaps with rowSize != width * vectorSize. We could do this also for ByteArray. |
| 66 | |
| 67 | - In TaskProcessor.cpp, the code below is fine and clean, but probably a bit inefficient. |
| 68 | When this wakes up another thread, it may have to immediately go back to sleep, since we still hold the lock. |
| 69 | It could instead set a need_to_notify flag and test that after releasing the lock (both places). |
| 70 | That might avoid some context switches. |
| 71 | ```cpp |
| 72 | if (mTilesInProcess == 0 && mTilesNotYetStarted == 0) { |
| 73 | mWorkIsFinished.notify_one(); |
| 74 | ``` |
| 75 | |
| 76 | * When compiled as part of Android, librenderscript_toolkit.so is 101,456 bytes. When compiled by Android Studio as part of an .aar, it's 387K. Figure out why and slim it down. |