#3367 from Kristjan Valur Jonsson:
If a PyTokenizer_FromString() is called with an empty string, the
tokenizer's line_start member never gets initialized. Later, it is
compared with the token pointer 'a' in parsetok.c:193 and that behavior
can result in undefined behavior.
diff --git a/Parser/tokenizer.c b/Parser/tokenizer.c
index 1d0a4aa..73ff92a 100644
--- a/Parser/tokenizer.c
+++ b/Parser/tokenizer.c
@@ -1117,7 +1117,7 @@
register int c;
int blankline;
- *p_start = *p_end = NULL;
+ tok->line_start = *p_start = *p_end = NULL;
nextline:
tok->start = NULL;
blankline = 0;