Merged revisions 85572-85573,85606,85609-85622,85624,85626-85627,85629,85631,85633,85635-85636,85638-85639,85641-85642 via svnmerge from
svn+ssh://svn.python.org/python/branches/py3k
........
r85572 | georg.brandl | 2010-10-16 20:51:05 +0200 (Sa, 16 Okt 2010) | 1 line
#10122: typo fix.
........
r85573 | georg.brandl | 2010-10-16 20:53:08 +0200 (Sa, 16 Okt 2010) | 1 line
#10124: typo fix.
........
r85606 | georg.brandl | 2010-10-17 08:32:59 +0200 (So, 17 Okt 2010) | 1 line
#10058: tweak wording about exception returns.
........
r85609 | georg.brandl | 2010-10-17 11:19:03 +0200 (So, 17 Okt 2010) | 1 line
#8556: use less confusing mapping key in example.
........
r85610 | georg.brandl | 2010-10-17 11:23:05 +0200 (So, 17 Okt 2010) | 1 line
#8686: remove potentially confusing wording that does not add any value.
........
r85611 | georg.brandl | 2010-10-17 11:33:24 +0200 (So, 17 Okt 2010) | 1 line
#8811: small fixes to sqlite3 docs.
........
r85612 | georg.brandl | 2010-10-17 11:37:54 +0200 (So, 17 Okt 2010) | 1 line
#8855: add shelve security warning.
........
r85613 | georg.brandl | 2010-10-17 11:43:35 +0200 (So, 17 Okt 2010) | 1 line
Fix hmac docs: it takes and returns bytes, except for hexdigest().
........
r85614 | georg.brandl | 2010-10-17 11:46:11 +0200 (So, 17 Okt 2010) | 1 line
#8968: add actual name of token constants.
........
r85615 | georg.brandl | 2010-10-17 12:05:13 +0200 (So, 17 Okt 2010) | 1 line
#459007: merge info from PC/getpathp.c and using/windows.rst to document the forming of sys.path under Windows.
........
r85616 | georg.brandl | 2010-10-17 12:07:29 +0200 (So, 17 Okt 2010) | 1 line
Fix copy-paste error in example.
........
r85617 | georg.brandl | 2010-10-17 12:09:06 +0200 (So, 17 Okt 2010) | 1 line
#5212: md5 weaknesses do not affect hmac, so remove the note about that.
........
r85618 | georg.brandl | 2010-10-17 12:14:38 +0200 (So, 17 Okt 2010) | 1 line
#9086: correct wrong terminology about linking with pythonXY.dll.
........
r85619 | georg.brandl | 2010-10-17 12:15:50 +0200 (So, 17 Okt 2010) | 1 line
Make file names consistent.
........
r85620 | georg.brandl | 2010-10-17 12:22:28 +0200 (So, 17 Okt 2010) | 1 line
Remove second parser module example; it referred to non-readily-available example files, and this kind of discovery is much better done with the AST nowadays anyway.
........
r85621 | georg.brandl | 2010-10-17 12:24:54 +0200 (So, 17 Okt 2010) | 1 line
#9105: move pickle warning to a bit more prominent location.
........
r85622 | georg.brandl | 2010-10-17 12:28:04 +0200 (So, 17 Okt 2010) | 1 line
#9112: document error() and exit() methods of ArgumentParser.
........
r85624 | georg.brandl | 2010-10-17 12:34:28 +0200 (So, 17 Okt 2010) | 1 line
Some markup and style fixes in argparse docs.
........
r85626 | georg.brandl | 2010-10-17 12:38:20 +0200 (So, 17 Okt 2010) | 1 line
#9117: fix syntax for class definition.
........
r85627 | georg.brandl | 2010-10-17 12:44:11 +0200 (So, 17 Okt 2010) | 1 line
#9138: reword introduction to classes in Python.
........
r85629 | georg.brandl | 2010-10-17 12:51:45 +0200 (So, 17 Okt 2010) | 1 line
#5962: clarify sys.exit() vs. threads.
........
r85631 | georg.brandl | 2010-10-17 12:53:54 +0200 (So, 17 Okt 2010) | 1 line
Fix capitalization.
........
r85633 | georg.brandl | 2010-10-17 12:59:41 +0200 (So, 17 Okt 2010) | 1 line
#9204: remove mentions of removed types in the types module.
........
r85635 | georg.brandl | 2010-10-17 13:03:22 +0200 (So, 17 Okt 2010) | 1 line
#5121: fix claims about default values leading to segfaults.
........
r85636 | georg.brandl | 2010-10-17 13:06:14 +0200 (So, 17 Okt 2010) | 1 line
#9237: document sys.call_tracing().
........
r85638 | georg.brandl | 2010-10-17 13:13:37 +0200 (So, 17 Okt 2010) | 1 line
Port changes to pickle docs apparently lost in py3k.
........
r85639 | georg.brandl | 2010-10-17 13:23:56 +0200 (So, 17 Okt 2010) | 1 line
Make twisted example a bit more logical.
........
r85641 | georg.brandl | 2010-10-17 13:29:07 +0200 (So, 17 Okt 2010) | 1 line
Fix documentation of dis.opmap direction.
........
r85642 | georg.brandl | 2010-10-17 13:36:28 +0200 (So, 17 Okt 2010) | 1 line
#9730: fix example.
........
diff --git a/Doc/library/token.rst b/Doc/library/token.rst
index bc5c19e..991762f 100644
--- a/Doc/library/token.rst
+++ b/Doc/library/token.rst
@@ -12,8 +12,8 @@
the language grammar. The specific numeric values which the names map to may
change between Python versions.
-This module also provides one data object and some functions. The functions
-mirror definitions in the Python C header files.
+The module also provides a mapping from numeric codes to names and some
+functions. The functions mirror definitions in the Python C header files.
.. data:: tok_name
@@ -38,6 +38,65 @@
Return true if *x* is the marker indicating the end of input.
+The token constants are:
+
+.. data:: ENDMARKER
+ NAME
+ NUMBER
+ STRING
+ NEWLINE
+ INDENT
+ DEDENT
+ LPAR
+ RPAR
+ LSQB
+ RSQB
+ COLON
+ COMMA
+ SEMI
+ PLUS
+ MINUS
+ STAR
+ SLASH
+ VBAR
+ AMPER
+ LESS
+ GREATER
+ EQUAL
+ DOT
+ PERCENT
+ BACKQUOTE
+ LBRACE
+ RBRACE
+ EQEQUAL
+ NOTEQUAL
+ LESSEQUAL
+ GREATEREQUAL
+ TILDE
+ CIRCUMFLEX
+ LEFTSHIFT
+ RIGHTSHIFT
+ DOUBLESTAR
+ PLUSEQUAL
+ MINEQUAL
+ STAREQUAL
+ SLASHEQUAL
+ PERCENTEQUAL
+ AMPEREQUAL
+ VBAREQUAL
+ CIRCUMFLEXEQUAL
+ LEFTSHIFTEQUAL
+ RIGHTSHIFTEQUAL
+ DOUBLESTAREQUAL
+ DOUBLESLASH
+ DOUBLESLASHEQUAL
+ AT
+ OP
+ ERRORTOKEN
+ N_TOKENS
+ NT_OFFSET
+
+
.. seealso::
Module :mod:`parser`