Misc/NEWS.d/next/Library/2018-05-17-22-14-58.bpo-12486.HBeh62.rst

:func:`tokenize.generate_tokens` is now documented as a public API to
tokenize unicode strings. It was previously present but undocumented.