Return correct error from keymaster0engine for large RSA input
Also, ensure that we always put some error on the OpenSSL error queue
whenever a wrapped keymaster0 operation fails. Higher layers will look
a the last entry on the queue and use it to determine what error code to
return. Not putting any error on the queue means that those higher
layers will get whatever error was last enqueued, making the result
effectively random. Non-determinism bad.
(cherry-picked from commit 22d2355b7edc470949c163e47ba8e837a1a87f47)
Bug: 25337630
Change-Id: I701ab735dd089f5258b2252f543906d9f3baa7a2
diff --git a/openssl_err.cpp b/openssl_err.cpp
index 51a29d9..df2920b 100644
--- a/openssl_err.cpp
+++ b/openssl_err.cpp
@@ -49,7 +49,8 @@
int reason = ERR_GET_REASON(error);
switch (ERR_GET_LIB(error)) {
-
+ case ERR_LIB_USER:
+ return static_cast<keymaster_error_t>(reason);
case ERR_LIB_EVP:
return TranslateEvpError(reason);
#if defined(OPENSSL_IS_BORINGSSL)