Gitiles
Code Review Sign In
gerrit-public.fairphone.software / platform / external / python / cpython3 / 9912b395c38b7e4e07e5b169d29afab6fad16374 / Lib / tokenize.py
  1. 10a99b0 Issue #13150: The tokenize module doesn't compile large regular expressions at startup anymore. by Antoine Pitrou · 13 years ago
  2. 14c0f03 Issue #12943: python -m tokenize support has been added to tokenize. by Meador Inge · 13 years ago
  3. f304278 Issue #11074: Make 'tokenize' so it can be reloaded. by Brett Cannon · 14 years ago
  4. b9d10d0 Issue #10386: Added __all__ to token module; this simplifies importing by Alexander Belopolsky · 14 years ago
  5. 58c0752 Issue #10335: Add tokenize.open(), detect the file encoding using by Victor Stinner · 14 years ago
  6. a0e7940 A little bit more readable repr method. by Raymond Hettinger · 14 years ago
  7. 3fb79c7 Experiment: Let collections.namedtuple() do the work. This should work now that _collections is pre-built. The buildbots will tell us shortly. by Raymond Hettinger · 14 years ago
  8. 6c60d09 Improve the repr for the TokenInfo named tuple. by Raymond Hettinger · 14 years ago
  9. 43e4ea1 Remove unused import, fix typo and rewrap docstrings. by Florent Xicluna · 14 years ago
  10. 33856de handle names starting with non-ascii characters correctly #9712 by Benjamin Peterson · 14 years ago
  11. 1613ed8 fix for files with coding cookies and BOMs by Benjamin Peterson · 15 years ago
  12. 689a558 in tokenize.detect_encoding(), return utf-8-sig when a BOM is found by Benjamin Peterson · 15 years ago
  13. 81dd8b9 use some more itertools magic to make '' be yielded after readline is done by Benjamin Peterson · 15 years ago
  14. 21db77e simply by using itertools.chain() by Benjamin Peterson · 15 years ago
  15. a0dfa82 Merged revisions 75149,75260-75263,75265-75267,75292,75300,75376,75405,75429-75433,75437,75445,75501,75551,75572,75589-75591,75657,75742,75868,75952-75957,76057,76105,76139,76143,76162,76223 via svnmerge from by Benjamin Peterson · 15 years ago
  16. d3afada normalize latin-1 and utf-8 variant encodings like the builtin tokenizer does by Benjamin Peterson · 15 years ago
  17. aa17a7f Remove dependency on the collections module. by Raymond Hettinger · 16 years ago
  18. a48db39 Issue #5857: tokenize.tokenize() now returns named tuples. by Raymond Hettinger · 16 years ago
  19. 9b8d24b reuse tokenize.detect_encoding in linecache instead of a custom solution by Benjamin Peterson · 16 years ago
  20. 433f32c raise a SyntaxError in detect_encoding() when a codec lookup fails like the builtin parser #4021 by Benjamin Peterson · 16 years ago
  21. fd03645 #2834: Change re module semantics, so that str and bytes mixing is forbidden, by Antoine Pitrou · 16 years ago
  22. 0fe1438 use the more idomatic (and Py3k faster) while True by Benjamin Peterson · 16 years ago
  23. ba4af49 Merged revisions 61964-61979 via svnmerge from by Christian Heimes · 17 years ago
  24. 428de65 - Issue #719888: Updated tokenize to use a bytes API. generate_tokens has been by Trent Nelson · 17 years ago
  25. fceab5a Merged revisions 60080-60089,60091-60093 via svnmerge from by Georg Brandl · 17 years ago
  26. 4fe72f9 Patch 1420 by Ron Adam. by Guido van Rossum · 17 years ago
  27. ce36ad8 Raise statement normalization in Lib/. by Collin Winter · 17 years ago
  28. cd16bf6 Merged revisions 55817-55961 via svnmerge from by Guido van Rossum · 17 years ago
  29. 1bc535d Merged revisions 55328-55341 via svnmerge from by Guido van Rossum · 17 years ago
  30. a18af4e PEP 3114: rename .next() to .__next__() and add next() builtin. by Georg Brandl · 18 years ago
  31. 650f0d0 Hide list comp variables and support set comprehensions by Nick Coghlan · 18 years ago
  32. dde0028 Make ELLIPSIS a separate token. This makes it a syntax error to write ". . ." for Ellipsis. by Georg Brandl · 18 years ago
  33. be19ed7 Fix most trivially-findable print statements. by Guido van Rossum · 18 years ago
  34. c150536 PEP 3107 - Function Annotations thanks to Tony Lownds by Neal Norwitz · 18 years ago
  35. 89f507f Four months of trunk changes (including a few releases...) by Thomas Wouters · 18 years ago
  36. cf588f6 Remove support for backticks from the grammar and compiler. by Brett Cannon · 18 years ago
  37. b053cd8 Killed the <> operator. You must now use !=. by Guido van Rossum · 18 years ago
  38. 00ee7ba Merge current trunk into p3yk. This includes the PyNumber_Index API change, by Thomas Wouters · 18 years ago
  39. 49fd7fa Merge p3yk branch with the trunk up to revision 45595. This breaks a fair by Thomas Wouters · 19 years ago
  40. da99d1c SF bug #1224621: tokenize module does not detect inconsistent dedents by Raymond Hettinger · 19 years ago
  41. 68c0453 Add untokenize() function to allow full round-trip tokenization. by Raymond Hettinger · 19 years ago
  42. c2a5a63 PEP-0318, @decorator-style. In Guido's words: by Anthony Baxter · 20 years ago
  43. 68468eb Get rid of many apply() calls. by Guido van Rossum · 22 years ago
  44. 78a7aee SF 633560: tokenize.__all__ needs "generate_tokens" by Raymond Hettinger · 22 years ago
  45. 9d6897a Speed up the most egregious "if token in (long tuple)" cases by using by Guido van Rossum · 22 years ago
  46. 8ac1495 Whitespace normalization. by Tim Peters · 22 years ago
  47. d1fa3db Added docstrings excerpted from Python Library Reference. Closes patch 556161. by Raymond Hettinger · 22 years ago
  48. 496563a Remove some now-obsolete generator future statements. by Tim Peters · 23 years ago
  49. e98d16e Cleanup x so it is not left in module by Neal Norwitz · 23 years ago
  50. d507dab SF patch #455966: Allow leading 0 in float/imag literals. by Tim Peters · 23 years ago
  51. 96204f5 Add new tokens // and //=, in support of PEP 238. by Guido van Rossum · 23 years ago
  52. 79e75e1 Use string.ascii_letters instead of string.letters (SF bug #226706). by Fred Drake · 23 years ago
  53. b09f7ed Preliminary support for "from __future__ import generators" to enable by Guido van Rossum · 23 years ago
  54. 4efb6e9 Turns out Neil didn't intend for *all* of his gen-branch work to get committed. by Tim Peters · 23 years ago
  55. 5ca576e Merging the gen-branch into the main line, at Guido's direction. Yay! by Tim Peters · 23 years ago
  56. 28c62bb Provide a StopTokenizing exception for conveniently exiting the loop. by Ka-Ping Yee · 24 years ago
  57. 4f64c13 Better __credits__. by Ka-Ping Yee · 24 years ago
  58. 244c593 Add __author__ and __credits__ variables. by Ka-Ping Yee · 24 years ago
  59. 40fc160 final round of __all__ lists (I hope) - skipped urllib2 because Moshe may be by Skip Montanaro · 24 years ago
  60. b08b2d3 String method conversion. by Eric S. Raymond · 24 years ago
  61. 1ff08b1 Add tokenizer support and tests for u'', U"", uR'', Ur"", etc. by Ka-Ping Yee · 24 years ago
  62. b90f89a Whitespace normalization. by Tim Peters · 24 years ago
  63. de49583 Possible fix for Skip's bug 116136 (sre recursion limit hit in tokenize.py). by Tim Peters · 24 years ago
  64. e1519a1 Update for augmented assignment, tested & approved by Guido. by Thomas Wouters · 24 years ago
  65. 9b8d801 Convert some old-style string exceptions to class exceptions. by Fred Drake · 24 years ago
  66. a90c78b Differentiate between NEWLINE token (an official newline) and NL token by Guido van Rossum · 27 years ago
  67. fefc922 New, fixed version with proper r"..." and R"..." support from Ka-Ping. by Guido van Rossum · 27 years ago
  68. 3b63177 Redone (by Ka-Ping) using the new re module, and adding recognition by Guido van Rossum · 27 years ago
  69. 2b1566b Correct typo in last line (test program invocation). by Guido van Rossum · 27 years ago
  70. de65527 Ping's latest. Fixes triple quoted strings ending in odd by Guido van Rossum · 28 years ago
  71. 1aec323 Ka-Ping's muich improved version of March 26, 1997: by Guido van Rossum · 28 years ago
  72. b5dc5e3 Added support for imaginary constants (e.g. 0j, 1j, 1.0j). by Guido van Rossum · 28 years ago
  73. b51eaa1 Fixed doc string, added __version__, fixed 1 bug. by Guido van Rossum · 28 years ago
  74. fc6f533 Ka-Ping's version. by Guido van Rossum · 28 years ago
  75. b31c7f7 * test_select.py: (some) tests for built-in select module by Guido van Rossum · 31 years ago
  76. 10d10ff Change the order in which Floatnumber and Intnumber are tried by Guido van Rossum · 33 years ago
  77. 4d8e859 Initial revision by Guido van Rossum · 33 years ago