bpo-41894: Fix UnicodeDecodeError while loading native module (GH-22466)
When running in a non-UTF-8 locale, if an error occurs while importing a
native Python module (say because a dependent share library is missing),
the error message string returned may contain non-ASCII code points
causing a UnicodeDecodeError.
PyUnicode_DecodeFSDefault is used for buffers which may contain
filesystem paths. For consistency with os.strerror(),
PyUnicode_DecodeLocale is used for buffers which contain system error
messages. While the shortname parameter is always encoded in ASCII
according to PEP 489, it is left decoded using PyUnicode_FromString to
minimize the changes and since it should not affect the decoding (albeit
_potentially_ slower).
In dynload_hpux, since the error buffer contains a message generated
from a static ASCII string and the module filesystem path,
PyUnicode_DecodeFSDefault is used instead of PyUnicode_DecodeLocale as
is used elsewhere.
* bpo-41894: Fix bugs in dynload error msg handling
For both dynload_aix and dynload_hpux, properly handle the possibility
that decoding strings may return NULL and when such an error happens,
properly decrement any previously decoded strings and return early.
In addition, in dynload_aix, ensure that we pass the decoded string
*object* pathname_ob to PyErr_SetImportError instead of the original
pathname buffer.
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
(cherry picked from commit 2d2af320d94afc6561e8f8adf174c9d3fd9065bc)
Co-authored-by: Kevin Adler <kadler@us.ibm.com>
diff --git a/Python/dynload_hpux.c b/Python/dynload_hpux.c
index 4b964a6..e36d608 100644
--- a/Python/dynload_hpux.c
+++ b/Python/dynload_hpux.c
@@ -36,9 +36,20 @@
char buf[256];
PyOS_snprintf(buf, sizeof(buf), "Failed to load %.200s",
pathname);
- PyObject *buf_ob = PyUnicode_FromString(buf);
+ PyObject *buf_ob = PyUnicode_DecodeFSDefault(buf);
+ if (buf_ob == NULL)
+ return NULL;
PyObject *shortname_ob = PyUnicode_FromString(shortname);
- PyObject *pathname_ob = PyUnicode_FromString(pathname);
+ if (shortname_ob == NULL) {
+ Py_DECREF(buf_ob);
+ return NULL;
+ }
+ PyObject *pathname_ob = PyUnicode_DecodeFSDefault(pathname);
+ if (pathname_ob == NULL) {
+ Py_DECREF(buf_ob);
+ Py_DECREF(shortname_ob);
+ return NULL;
+ }
PyErr_SetImportError(buf_ob, shortname_ob, pathname_ob);
Py_DECREF(buf_ob);
Py_DECREF(shortname_ob);