closes bpo-31650: PEP 552 (Deterministic pycs) implementation (#4575)
Python now supports checking bytecode cache up-to-dateness with a hash of the
source contents rather than volatile source metadata. See the PEP for details.
While a fairly straightforward idea, quite a lot of code had to be modified due
to the pervasiveness of pyc implementation details in the codebase. Changes in
this commit include:
- The core changes to importlib to understand how to read, validate, and
regenerate hash-based pycs.
- Support for generating hash-based pycs in py_compile and compileall.
- Modifications to our siphash implementation to support passing a custom
key. We then expose it to importlib through _imp.
- Updates to all places in the interpreter, standard library, and tests that
manually generate or parse pyc files to grok the new format.
- Support in the interpreter command line code for long options like
--check-hash-based-pycs.
- Tests and documentation for all of the above.
diff --git a/Include/internal/hash.h b/Include/internal/hash.h
new file mode 100644
index 0000000..e14b80a
--- /dev/null
+++ b/Include/internal/hash.h
@@ -0,0 +1,6 @@
+#ifndef Py_INTERNAL_HASH_H
+#define Py_INTERNAL_HASH_H
+
+uint64_t _Py_KeyedHash(uint64_t, const char *, Py_ssize_t);
+
+#endif
diff --git a/Include/internal/import.h b/Include/internal/import.h
new file mode 100644
index 0000000..4746e75
--- /dev/null
+++ b/Include/internal/import.h
@@ -0,0 +1,6 @@
+#ifndef Py_INTERNAL_IMPORT_H
+#define Py_INTERNAL_IMPORT_H
+
+extern const char *_Py_CheckHashBasedPycsMode;
+
+#endif
diff --git a/Include/pygetopt.h b/Include/pygetopt.h
index 962720c..3e3fe36 100644
--- a/Include/pygetopt.h
+++ b/Include/pygetopt.h
@@ -12,7 +12,14 @@
PyAPI_FUNC(void) _PyOS_ResetGetOpt(void);
-PyAPI_FUNC(int) _PyOS_GetOpt(int argc, wchar_t **argv, wchar_t *optstring);
+typedef struct {
+ const wchar_t *name;
+ int has_arg;
+ int val;
+} _PyOS_LongOption;
+
+PyAPI_FUNC(int) _PyOS_GetOpt(int argc, wchar_t **argv, wchar_t *optstring,
+ const _PyOS_LongOption *longopts, int *longindex);
#endif /* !Py_LIMITED_API */
#ifdef __cplusplus