Repair longstanding damage introduced by the AST merge:

- Get rid of pgen.c and tokenizer_pgen.c.

- Include tokenizer.c.

All the -uall tests pass on my WinXP VC7.1 box now.
1 file changed