Issue #2134: Clarify token.OP handling rationale in tokenize documentation.
diff --git a/Doc/library/tokenize.rst b/Doc/library/tokenize.rst
index 30677ea..7075035 100644
--- a/Doc/library/tokenize.rst
+++ b/Doc/library/tokenize.rst
@@ -15,6 +15,12 @@
well, making it useful for implementing "pretty-printers," including colorizers
for on-screen displays.
+To simplify token stream handling, all :ref:`operators` and :ref:`delimiters`
+tokens are returned using the generic :data:`token.OP` token type. The exact
+type can be determined by checking the token ``string`` field on the
+:term:`named tuple` returned from :func:`tokenize.tokenize` for the character
+sequence that identifies a specific operator token.
+
The primary entry point is a :term:`generator`:
.. function:: generate_tokens(readline)