Limit dotProduct value to 1.0f, so acosf would not return NaN.
Due to precision loss of float math, we sometimes get 1.000001f for
dotProduct. This causes NaN result from acosf() and floor() funcs.
At the moment, this does not cause any problems on ARM, as casting
NaN to int results in 0. On mips however (possibly on x86), such cast
gives INT_MAX, so crash occurs when trying to use the resulting value.
Change-Id: I8e0285a0306a65b8469d9f4885c19665066fc4c8
diff --git a/libs/hwui/ShadowTessellator.cpp b/libs/hwui/ShadowTessellator.cpp
index 93d4b31..c1b0f7f 100644
--- a/libs/hwui/ShadowTessellator.cpp
+++ b/libs/hwui/ShadowTessellator.cpp
@@ -20,6 +20,7 @@
#include <math.h>
#include <utils/Log.h>
#include <utils/Trace.h>
+#include <utils/MathUtils.h>
#include "AmbientShadow.h"
#include "Caches.h"
@@ -264,6 +265,8 @@
// acos( ) --- [0, M_PI]
// floor(...) --- [0, EXTRA_VERTEX_PER_PI]
float dotProduct = vector1.dot(vector2);
+ // make sure that dotProduct value is in acsof input range [-1, 1]
+ dotProduct = MathUtils::clamp(dotProduct, -1.0f, 1.0f);
// TODO: Use look up table for the dotProduct to extraVerticesNumber
// computation, if needed.
float angle = acosf(dotProduct);