| commit | c56b17bd8c7a3fd03859822246633d2c9586f8bd | [log] [tgz] |
|---|---|---|
| author | Thomas Kluyver <takowl@gmail.com> | Tue Jun 05 19:26:39 2018 +0200 |
| committer | Carol Willing <carolcode@willingconsulting.com> | Tue Jun 05 10:26:39 2018 -0700 |
| tree | 346fb8b3a6614679232792b3f46398b33e5f3c0e | |
| parent | c2745d2d05546d76f655ab450eb23d1af39e0b1c [diff] |
bpo-12486: Document tokenize.generate_tokens() as public API (#6957) * Document tokenize.generate_tokens() * Add news file * Add test for generate_tokens * Document behaviour around ENCODING token * Add generate_tokens to __all__