bpo-40847: Consider a line with only a LINECONT a blank line (GH-20769)
A line with only a line continuation character should be considered
a blank line at tokenizer level so that only a single NEWLINE token
gets emitted. The old parser was working around the issue, but the
new parser threw a `SyntaxError` for valid input. For example,
an empty line following a line continuation character was interpreted
as a `SyntaxError`.
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
diff --git a/Parser/tokenizer.c b/Parser/tokenizer.c
index cebfadc..d461e4e 100644
--- a/Parser/tokenizer.c
+++ b/Parser/tokenizer.c
@@ -1203,8 +1203,9 @@
}
}
tok_backup(tok, c);
- if (c == '#' || c == '\n') {
+ if (c == '#' || c == '\n' || c == '\\') {
/* Lines with only whitespace and/or comments
+ and/or a line continuation character
shouldn't affect the indentation and are
not passed to the parser as NEWLINE tokens,
except *totally* empty lines in interactive