bpo-42827: Fix crash on SyntaxError in multiline expressions (GH-24140)
When trying to extract the error line for the error message there
are two distinct cases:
1. The input comes from a file, which means that we can extract the
error line by using `PyErr_ProgramTextObject` and which we already
do.
2. The input does not come from a file, at which point we need to get
the source code from the tokenizer:
* If the tokenizer's current line number is the same with the line
of the error, we get the line from `tok->buf` and we're ready.
* Else, we can extract the error line from the source code in the
following two ways:
* If the input comes from a string we have all the input
in `tok->str` and we can extract the error line from it.
* If the input comes from stdin, i.e. the interactive prompt, we
do not have access to the previous line. That's why a new
field `tok->stdin_content` is added which holds the whole input for the
current (multiline) statement or expression. We can then extract the
error line from `tok->stdin_content` like we do in the string case above.
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>diff --git a/Lib/test/test_exceptions.py b/Lib/test/test_exceptions.py
index 8644223..eb70d7b 100644
--- a/Lib/test/test_exceptions.py
+++ b/Lib/test/test_exceptions.py
@@ -209,6 +209,9 @@ def testSyntaxErrorOffset(self):
check('x = "a', 1, 7)
check('lambda x: x = 2', 1, 1)
check('f{a + b + c}', 1, 2)
+ check('[file for str(file) in []\n])', 1, 11)
+ check('[\nfile\nfor str(file)\nin\n[]\n]', 3, 5)
+ check('[file for\n str(file) in []]', 2, 2)
# Errors thrown by compile.c
check('class foo:return 1', 1, 11)