commit | d3afadaa4908df544e0181c11199e59b1bfb5c37 | [log] [tgz] |
---|---|---|
author | Benjamin Peterson <benjamin@python.org> | Fri Oct 09 21:43:09 2009 +0000 |
committer | Benjamin Peterson <benjamin@python.org> | Fri Oct 09 21:43:09 2009 +0000 |
tree | 5b214ec4a85f64411b50dd40499bf9a7691d4a5f | |
parent | ffc08fcad6d91a50224914e94eae6505b2e55548 [diff] |
normalize latin-1 and utf-8 variant encodings like the builtin tokenizer does