Guido van Rossum | 7627c0d | 2000-03-31 14:58:54 +0000 | [diff] [blame] | 1 | # |
| 2 | # Secret Labs' Regular Expression Engine |
Guido van Rossum | 7627c0d | 2000-03-31 14:58:54 +0000 | [diff] [blame] | 3 | # |
| 4 | # re-compatible interface for the sre matching engine |
| 5 | # |
Fredrik Lundh | 770617b | 2001-01-14 15:06:11 +0000 | [diff] [blame] | 6 | # Copyright (c) 1998-2001 by Secret Labs AB. All rights reserved. |
Guido van Rossum | 7627c0d | 2000-03-31 14:58:54 +0000 | [diff] [blame] | 7 | # |
Fredrik Lundh | 29c4ba9 | 2000-08-01 18:20:07 +0000 | [diff] [blame] | 8 | # This version of the SRE library can be redistributed under CNRI's |
| 9 | # Python 1.6 license. For any other use, please contact Secret Labs |
| 10 | # AB (info@pythonware.com). |
| 11 | # |
Guido van Rossum | 7627c0d | 2000-03-31 14:58:54 +0000 | [diff] [blame] | 12 | # Portions of this engine have been developed in cooperation with |
Fredrik Lundh | 29c4ba9 | 2000-08-01 18:20:07 +0000 | [diff] [blame] | 13 | # CNRI. Hewlett-Packard provided funding for 1.6 integration and |
Guido van Rossum | 7627c0d | 2000-03-31 14:58:54 +0000 | [diff] [blame] | 14 | # other compatibility work. |
| 15 | # |
| 16 | |
Fred Drake | 9f5b822 | 2001-09-04 19:20:06 +0000 | [diff] [blame] | 17 | r"""Support for regular expressions (RE). |
Fred Drake | b8f2274 | 2001-09-04 19:10:20 +0000 | [diff] [blame] | 18 | |
| 19 | This module provides regular expression matching operations similar to |
Fredrik Lundh | dac5849 | 2001-10-21 21:48:30 +0000 | [diff] [blame] | 20 | those found in Perl. It supports both 8-bit and Unicode strings; both |
| 21 | the pattern and the strings being processed can contain null bytes and |
| 22 | characters outside the US ASCII range. |
Fred Drake | b8f2274 | 2001-09-04 19:10:20 +0000 | [diff] [blame] | 23 | |
Fredrik Lundh | dac5849 | 2001-10-21 21:48:30 +0000 | [diff] [blame] | 24 | Regular expressions can contain both special and ordinary characters. |
| 25 | Most ordinary characters, like "A", "a", or "0", are the simplest |
| 26 | regular expressions; they simply match themselves. You can |
Fred Drake | b8f2274 | 2001-09-04 19:10:20 +0000 | [diff] [blame] | 27 | concatenate ordinary characters, so last matches the string 'last'. |
| 28 | |
| 29 | The special characters are: |
| 30 | "." Matches any character except a newline. |
| 31 | "^" Matches the start of the string. |
Georg Brandl | 86def6c | 2008-01-21 20:36:10 +0000 | [diff] [blame] | 32 | "$" Matches the end of the string or just before the newline at |
| 33 | the end of the string. |
Fred Drake | b8f2274 | 2001-09-04 19:10:20 +0000 | [diff] [blame] | 34 | "*" Matches 0 or more (greedy) repetitions of the preceding RE. |
| 35 | Greedy means that it will match as many repetitions as possible. |
| 36 | "+" Matches 1 or more (greedy) repetitions of the preceding RE. |
| 37 | "?" Matches 0 or 1 (greedy) of the preceding RE. |
| 38 | *?,+?,?? Non-greedy versions of the previous three special characters. |
| 39 | {m,n} Matches from m to n repetitions of the preceding RE. |
| 40 | {m,n}? Non-greedy version of the above. |
Christian Heimes | af98da1 | 2008-01-27 15:18:18 +0000 | [diff] [blame] | 41 | "\\" Either escapes special characters or signals a special sequence. |
Fred Drake | b8f2274 | 2001-09-04 19:10:20 +0000 | [diff] [blame] | 42 | [] Indicates a set of characters. |
| 43 | A "^" as the first character indicates a complementing set. |
| 44 | "|" A|B, creates an RE that will match either A or B. |
| 45 | (...) Matches the RE inside the parentheses. |
| 46 | The contents can be retrieved or matched later in the string. |
Fredrik Lundh | dac5849 | 2001-10-21 21:48:30 +0000 | [diff] [blame] | 47 | (?iLmsux) Set the I, L, M, S, U, or X flag for the RE (see below). |
Fred Drake | b8f2274 | 2001-09-04 19:10:20 +0000 | [diff] [blame] | 48 | (?:...) Non-grouping version of regular parentheses. |
| 49 | (?P<name>...) The substring matched by the group is accessible by name. |
| 50 | (?P=name) Matches the text matched earlier by the group named name. |
| 51 | (?#...) A comment; ignored. |
| 52 | (?=...) Matches if ... matches next, but doesn't consume the string. |
| 53 | (?!...) Matches if ... doesn't match next. |
Christian Heimes | af98da1 | 2008-01-27 15:18:18 +0000 | [diff] [blame] | 54 | (?<=...) Matches if preceded by ... (must be fixed length). |
| 55 | (?<!...) Matches if not preceded by ... (must be fixed length). |
| 56 | (?(id/name)yes|no) Matches yes pattern if the group with id/name matched, |
| 57 | the (optional) no pattern otherwise. |
Fred Drake | b8f2274 | 2001-09-04 19:10:20 +0000 | [diff] [blame] | 58 | |
| 59 | The special sequences consist of "\\" and a character from the list |
Fredrik Lundh | dac5849 | 2001-10-21 21:48:30 +0000 | [diff] [blame] | 60 | below. If the ordinary character is not on the list, then the |
Fred Drake | b8f2274 | 2001-09-04 19:10:20 +0000 | [diff] [blame] | 61 | resulting RE will match the second character. |
Fred Drake | 9f5b822 | 2001-09-04 19:20:06 +0000 | [diff] [blame] | 62 | \number Matches the contents of the group of the same number. |
| 63 | \A Matches only at the start of the string. |
| 64 | \Z Matches only at the end of the string. |
| 65 | \b Matches the empty string, but only at the start or end of a word. |
| 66 | \B Matches the empty string, but not at the start or end of a word. |
| 67 | \d Matches any decimal digit; equivalent to the set [0-9]. |
| 68 | \D Matches any non-digit character; equivalent to the set [^0-9]. |
| 69 | \s Matches any whitespace character; equivalent to [ \t\n\r\f\v]. |
| 70 | \S Matches any non-whitespace character; equiv. to [^ \t\n\r\f\v]. |
| 71 | \w Matches any alphanumeric character; equivalent to [a-zA-Z0-9_]. |
Fred Drake | b8f2274 | 2001-09-04 19:10:20 +0000 | [diff] [blame] | 72 | With LOCALE, it will match the set [0-9_] plus characters defined |
| 73 | as letters for the current locale. |
Fred Drake | 9f5b822 | 2001-09-04 19:20:06 +0000 | [diff] [blame] | 74 | \W Matches the complement of \w. |
| 75 | \\ Matches a literal backslash. |
Fred Drake | b8f2274 | 2001-09-04 19:10:20 +0000 | [diff] [blame] | 76 | |
| 77 | This module exports the following functions: |
| 78 | match Match a regular expression pattern to the beginning of a string. |
| 79 | search Search a string for the presence of a pattern. |
| 80 | sub Substitute occurrences of a pattern found in a string. |
| 81 | subn Same as sub, but also return the number of substitutions made. |
| 82 | split Split a string by the occurrences of a pattern. |
| 83 | findall Find all occurrences of a pattern in a string. |
Christian Heimes | af98da1 | 2008-01-27 15:18:18 +0000 | [diff] [blame] | 84 | finditer Return an iterator yielding a match object for each match. |
Fred Drake | b8f2274 | 2001-09-04 19:10:20 +0000 | [diff] [blame] | 85 | compile Compile a pattern into a RegexObject. |
| 86 | purge Clear the regular expression cache. |
Fred Drake | b8f2274 | 2001-09-04 19:10:20 +0000 | [diff] [blame] | 87 | escape Backslash all non-alphanumerics in a string. |
| 88 | |
| 89 | Some of the functions in this module takes flags as optional parameters: |
| 90 | I IGNORECASE Perform case-insensitive matching. |
| 91 | L LOCALE Make \w, \W, \b, \B, dependent on the current locale. |
Georg Brandl | 86def6c | 2008-01-21 20:36:10 +0000 | [diff] [blame] | 92 | M MULTILINE "^" matches the beginning of lines (after a newline) |
| 93 | as well as the string. |
| 94 | "$" matches the end of lines (before a newline) as well |
| 95 | as the end of the string. |
Fred Drake | b8f2274 | 2001-09-04 19:10:20 +0000 | [diff] [blame] | 96 | S DOTALL "." matches any character at all, including the newline. |
| 97 | X VERBOSE Ignore whitespace and comments for nicer looking RE's. |
Fredrik Lundh | 21009b9 | 2001-09-18 18:47:09 +0000 | [diff] [blame] | 98 | U UNICODE Make \w, \W, \b, \B, dependent on the Unicode locale. |
Fred Drake | b8f2274 | 2001-09-04 19:10:20 +0000 | [diff] [blame] | 99 | |
| 100 | This module also defines an exception 'error'. |
| 101 | |
| 102 | """ |
Fredrik Lundh | 21009b9 | 2001-09-18 18:47:09 +0000 | [diff] [blame] | 103 | |
Fredrik Lundh | 703ce81 | 2001-10-24 22:16:30 +0000 | [diff] [blame] | 104 | import sys |
Guido van Rossum | 7627c0d | 2000-03-31 14:58:54 +0000 | [diff] [blame] | 105 | import sre_compile |
Fredrik Lundh | 436c3d58 | 2000-06-29 08:58:44 +0000 | [diff] [blame] | 106 | import sre_parse |
Guido van Rossum | 7627c0d | 2000-03-31 14:58:54 +0000 | [diff] [blame] | 107 | |
Fredrik Lundh | f2989b2 | 2001-02-18 12:05:16 +0000 | [diff] [blame] | 108 | # public symbols |
| 109 | __all__ = [ "match", "search", "sub", "subn", "split", "findall", |
| 110 | "compile", "purge", "template", "escape", "I", "L", "M", "S", "X", |
| 111 | "U", "IGNORECASE", "LOCALE", "MULTILINE", "DOTALL", "VERBOSE", |
| 112 | "UNICODE", "error" ] |
| 113 | |
Fredrik Lundh | bec95b9 | 2001-10-21 16:47:57 +0000 | [diff] [blame] | 114 | __version__ = "2.2.1" |
Fredrik Lundh | b25e1ad | 2001-03-22 15:50:10 +0000 | [diff] [blame] | 115 | |
Jeremy Hylton | b1aa195 | 2000-06-01 17:39:12 +0000 | [diff] [blame] | 116 | # flags |
Fredrik Lundh | 770617b | 2001-01-14 15:06:11 +0000 | [diff] [blame] | 117 | I = IGNORECASE = sre_compile.SRE_FLAG_IGNORECASE # ignore case |
| 118 | L = LOCALE = sre_compile.SRE_FLAG_LOCALE # assume current 8-bit locale |
| 119 | U = UNICODE = sre_compile.SRE_FLAG_UNICODE # assume unicode locale |
| 120 | M = MULTILINE = sre_compile.SRE_FLAG_MULTILINE # make anchors look for newline |
| 121 | S = DOTALL = sre_compile.SRE_FLAG_DOTALL # make dot match newline |
| 122 | X = VERBOSE = sre_compile.SRE_FLAG_VERBOSE # ignore whitespace and comments |
Jeremy Hylton | b1aa195 | 2000-06-01 17:39:12 +0000 | [diff] [blame] | 123 | |
Fredrik Lundh | 770617b | 2001-01-14 15:06:11 +0000 | [diff] [blame] | 124 | # sre extensions (experimental, don't rely on these) |
| 125 | T = TEMPLATE = sre_compile.SRE_FLAG_TEMPLATE # disable backtracking |
| 126 | DEBUG = sre_compile.SRE_FLAG_DEBUG # dump pattern after compilation |
Fredrik Lundh | 436c3d58 | 2000-06-29 08:58:44 +0000 | [diff] [blame] | 127 | |
| 128 | # sre exception |
Fredrik Lundh | be2211e | 2000-06-29 16:57:40 +0000 | [diff] [blame] | 129 | error = sre_compile.error |
Fredrik Lundh | 436c3d58 | 2000-06-29 08:58:44 +0000 | [diff] [blame] | 130 | |
Guido van Rossum | 7627c0d | 2000-03-31 14:58:54 +0000 | [diff] [blame] | 131 | # -------------------------------------------------------------------- |
| 132 | # public interface |
| 133 | |
Guido van Rossum | 7627c0d | 2000-03-31 14:58:54 +0000 | [diff] [blame] | 134 | def match(pattern, string, flags=0): |
Fredrik Lundh | 770617b | 2001-01-14 15:06:11 +0000 | [diff] [blame] | 135 | """Try to apply the pattern at the start of the string, returning |
| 136 | a match object, or None if no match was found.""" |
Jeremy Hylton | b1aa195 | 2000-06-01 17:39:12 +0000 | [diff] [blame] | 137 | return _compile(pattern, flags).match(string) |
Guido van Rossum | 7627c0d | 2000-03-31 14:58:54 +0000 | [diff] [blame] | 138 | |
| 139 | def search(pattern, string, flags=0): |
Fredrik Lundh | 770617b | 2001-01-14 15:06:11 +0000 | [diff] [blame] | 140 | """Scan through string looking for a match to the pattern, returning |
| 141 | a match object, or None if no match was found.""" |
Jeremy Hylton | b1aa195 | 2000-06-01 17:39:12 +0000 | [diff] [blame] | 142 | return _compile(pattern, flags).search(string) |
Guido van Rossum | 7627c0d | 2000-03-31 14:58:54 +0000 | [diff] [blame] | 143 | |
Jeremy Hylton | b1aa195 | 2000-06-01 17:39:12 +0000 | [diff] [blame] | 144 | def sub(pattern, repl, string, count=0): |
Fredrik Lundh | 770617b | 2001-01-14 15:06:11 +0000 | [diff] [blame] | 145 | """Return the string obtained by replacing the leftmost |
| 146 | non-overlapping occurrences of the pattern in string by the |
Andrew M. Kuchling | 43ab0cd | 2004-08-07 17:41:54 +0000 | [diff] [blame] | 147 | replacement repl. repl can be either a string or a callable; |
| 148 | if a callable, it's passed the match object and must return |
| 149 | a replacement string to be used.""" |
Fredrik Lundh | 7898c3e | 2000-08-07 20:59:04 +0000 | [diff] [blame] | 150 | return _compile(pattern, 0).sub(repl, string, count) |
Jeremy Hylton | b1aa195 | 2000-06-01 17:39:12 +0000 | [diff] [blame] | 151 | |
| 152 | def subn(pattern, repl, string, count=0): |
Fredrik Lundh | 770617b | 2001-01-14 15:06:11 +0000 | [diff] [blame] | 153 | """Return a 2-tuple containing (new_string, number). |
| 154 | new_string is the string obtained by replacing the leftmost |
| 155 | non-overlapping occurrences of the pattern in the source |
| 156 | string by the replacement repl. number is the number of |
Andrew M. Kuchling | 43ab0cd | 2004-08-07 17:41:54 +0000 | [diff] [blame] | 157 | substitutions that were made. repl can be either a string or a |
| 158 | callable; if a callable, it's passed the match object and must |
| 159 | return a replacement string to be used.""" |
Fredrik Lundh | 7898c3e | 2000-08-07 20:59:04 +0000 | [diff] [blame] | 160 | return _compile(pattern, 0).subn(repl, string, count) |
Jeremy Hylton | b1aa195 | 2000-06-01 17:39:12 +0000 | [diff] [blame] | 161 | |
| 162 | def split(pattern, string, maxsplit=0): |
Fredrik Lundh | 770617b | 2001-01-14 15:06:11 +0000 | [diff] [blame] | 163 | """Split the source string by the occurrences of the pattern, |
| 164 | returning a list containing the resulting substrings.""" |
Fredrik Lundh | 7898c3e | 2000-08-07 20:59:04 +0000 | [diff] [blame] | 165 | return _compile(pattern, 0).split(string, maxsplit) |
Jeremy Hylton | b1aa195 | 2000-06-01 17:39:12 +0000 | [diff] [blame] | 166 | |
Raymond Hettinger | 596ba4d | 2004-09-24 03:41:05 +0000 | [diff] [blame] | 167 | def findall(pattern, string, flags=0): |
Fredrik Lundh | 770617b | 2001-01-14 15:06:11 +0000 | [diff] [blame] | 168 | """Return a list of all non-overlapping matches in the string. |
| 169 | |
| 170 | If one or more groups are present in the pattern, return a |
| 171 | list of groups; this will be a list of tuples if the pattern |
| 172 | has more than one group. |
| 173 | |
| 174 | Empty matches are included in the result.""" |
Raymond Hettinger | 596ba4d | 2004-09-24 03:41:05 +0000 | [diff] [blame] | 175 | return _compile(pattern, flags).findall(string) |
Jeremy Hylton | b1aa195 | 2000-06-01 17:39:12 +0000 | [diff] [blame] | 176 | |
Fredrik Lundh | 703ce81 | 2001-10-24 22:16:30 +0000 | [diff] [blame] | 177 | if sys.hexversion >= 0x02020000: |
Guido van Rossum | 0d97655 | 2002-10-14 12:22:17 +0000 | [diff] [blame] | 178 | __all__.append("finditer") |
Raymond Hettinger | 596ba4d | 2004-09-24 03:41:05 +0000 | [diff] [blame] | 179 | def finditer(pattern, string, flags=0): |
Fredrik Lundh | b7747e2 | 2001-10-28 20:15:40 +0000 | [diff] [blame] | 180 | """Return an iterator over all non-overlapping matches in the |
| 181 | string. For each match, the iterator returns a match object. |
Fredrik Lundh | 703ce81 | 2001-10-24 22:16:30 +0000 | [diff] [blame] | 182 | |
| 183 | Empty matches are included in the result.""" |
Raymond Hettinger | 596ba4d | 2004-09-24 03:41:05 +0000 | [diff] [blame] | 184 | return _compile(pattern, flags).finditer(string) |
Fredrik Lundh | 703ce81 | 2001-10-24 22:16:30 +0000 | [diff] [blame] | 185 | |
Jeremy Hylton | b1aa195 | 2000-06-01 17:39:12 +0000 | [diff] [blame] | 186 | def compile(pattern, flags=0): |
Fredrik Lundh | 770617b | 2001-01-14 15:06:11 +0000 | [diff] [blame] | 187 | "Compile a regular expression pattern, returning a pattern object." |
Jeremy Hylton | b1aa195 | 2000-06-01 17:39:12 +0000 | [diff] [blame] | 188 | return _compile(pattern, flags) |
| 189 | |
Fredrik Lundh | 8a3ebf8 | 2000-07-23 21:46:17 +0000 | [diff] [blame] | 190 | def purge(): |
Fredrik Lundh | 770617b | 2001-01-14 15:06:11 +0000 | [diff] [blame] | 191 | "Clear the regular expression cache" |
Fredrik Lundh | 8a3ebf8 | 2000-07-23 21:46:17 +0000 | [diff] [blame] | 192 | _cache.clear() |
Fredrik Lundh | b25e1ad | 2001-03-22 15:50:10 +0000 | [diff] [blame] | 193 | _cache_repl.clear() |
Fredrik Lundh | 8a3ebf8 | 2000-07-23 21:46:17 +0000 | [diff] [blame] | 194 | |
Fredrik Lundh | 436c3d58 | 2000-06-29 08:58:44 +0000 | [diff] [blame] | 195 | def template(pattern, flags=0): |
Fredrik Lundh | 770617b | 2001-01-14 15:06:11 +0000 | [diff] [blame] | 196 | "Compile a template pattern, returning a pattern object" |
Fredrik Lundh | 436c3d58 | 2000-06-29 08:58:44 +0000 | [diff] [blame] | 197 | return _compile(pattern, flags|T) |
| 198 | |
Raymond Hettinger | ebb7f30 | 2005-09-12 22:50:37 +0000 | [diff] [blame] | 199 | _alphanum = {} |
| 200 | for c in 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890': |
| 201 | _alphanum[c] = 1 |
| 202 | del c |
| 203 | |
Jeremy Hylton | b1aa195 | 2000-06-01 17:39:12 +0000 | [diff] [blame] | 204 | def escape(pattern): |
Fredrik Lundh | 770617b | 2001-01-14 15:06:11 +0000 | [diff] [blame] | 205 | "Escape all non-alphanumeric characters in pattern." |
Jeremy Hylton | b1aa195 | 2000-06-01 17:39:12 +0000 | [diff] [blame] | 206 | s = list(pattern) |
Raymond Hettinger | ebb7f30 | 2005-09-12 22:50:37 +0000 | [diff] [blame] | 207 | alphanum = _alphanum |
Jeremy Hylton | b1aa195 | 2000-06-01 17:39:12 +0000 | [diff] [blame] | 208 | for i in range(len(pattern)): |
| 209 | c = pattern[i] |
Raymond Hettinger | ebb7f30 | 2005-09-12 22:50:37 +0000 | [diff] [blame] | 210 | if c not in alphanum: |
Jeremy Hylton | b1aa195 | 2000-06-01 17:39:12 +0000 | [diff] [blame] | 211 | if c == "\000": |
| 212 | s[i] = "\\000" |
| 213 | else: |
| 214 | s[i] = "\\" + c |
Barry Warsaw | 8bee761 | 2004-08-25 02:22:30 +0000 | [diff] [blame] | 215 | return pattern[:0].join(s) |
Guido van Rossum | 7627c0d | 2000-03-31 14:58:54 +0000 | [diff] [blame] | 216 | |
| 217 | # -------------------------------------------------------------------- |
Jeremy Hylton | b1aa195 | 2000-06-01 17:39:12 +0000 | [diff] [blame] | 218 | # internals |
Guido van Rossum | 7627c0d | 2000-03-31 14:58:54 +0000 | [diff] [blame] | 219 | |
Jeremy Hylton | b1aa195 | 2000-06-01 17:39:12 +0000 | [diff] [blame] | 220 | _cache = {} |
Fredrik Lundh | b25e1ad | 2001-03-22 15:50:10 +0000 | [diff] [blame] | 221 | _cache_repl = {} |
| 222 | |
Fredrik Lundh | 397a654 | 2001-10-18 19:30:16 +0000 | [diff] [blame] | 223 | _pattern_type = type(sre_compile.compile("", 0)) |
| 224 | |
Jeremy Hylton | b1aa195 | 2000-06-01 17:39:12 +0000 | [diff] [blame] | 225 | _MAXCACHE = 100 |
Guido van Rossum | 7627c0d | 2000-03-31 14:58:54 +0000 | [diff] [blame] | 226 | |
Fredrik Lundh | 7898c3e | 2000-08-07 20:59:04 +0000 | [diff] [blame] | 227 | def _compile(*key): |
Jeremy Hylton | b1aa195 | 2000-06-01 17:39:12 +0000 | [diff] [blame] | 228 | # internal: compile pattern |
Hye-Shik Chang | 0f5bf1e | 2004-04-20 21:11:11 +0000 | [diff] [blame] | 229 | cachekey = (type(key[0]),) + key |
| 230 | p = _cache.get(cachekey) |
Fredrik Lundh | 7898c3e | 2000-08-07 20:59:04 +0000 | [diff] [blame] | 231 | if p is not None: |
| 232 | return p |
| 233 | pattern, flags = key |
Just van Rossum | 12723ba | 2003-07-02 20:03:04 +0000 | [diff] [blame] | 234 | if isinstance(pattern, _pattern_type): |
Christian Heimes | 5fb7c2a | 2007-12-24 08:52:31 +0000 | [diff] [blame] | 235 | if flags: |
| 236 | raise ValueError('Cannot process flags argument with a compiled pattern') |
Jeremy Hylton | b1aa195 | 2000-06-01 17:39:12 +0000 | [diff] [blame] | 237 | return pattern |
Just van Rossum | 7490250 | 2003-07-02 21:37:16 +0000 | [diff] [blame] | 238 | if not sre_compile.isstring(pattern): |
Collin Winter | ce36ad8 | 2007-08-30 01:19:48 +0000 | [diff] [blame] | 239 | raise TypeError("first argument must be string or compiled pattern") |
| 240 | p = sre_compile.compile(pattern, flags) |
Jeremy Hylton | b1aa195 | 2000-06-01 17:39:12 +0000 | [diff] [blame] | 241 | if len(_cache) >= _MAXCACHE: |
| 242 | _cache.clear() |
Hye-Shik Chang | 0f5bf1e | 2004-04-20 21:11:11 +0000 | [diff] [blame] | 243 | _cache[cachekey] = p |
Jeremy Hylton | b1aa195 | 2000-06-01 17:39:12 +0000 | [diff] [blame] | 244 | return p |
| 245 | |
Fredrik Lundh | b25e1ad | 2001-03-22 15:50:10 +0000 | [diff] [blame] | 246 | def _compile_repl(*key): |
| 247 | # internal: compile replacement pattern |
| 248 | p = _cache_repl.get(key) |
| 249 | if p is not None: |
| 250 | return p |
| 251 | repl, pattern = key |
Collin Winter | ce36ad8 | 2007-08-30 01:19:48 +0000 | [diff] [blame] | 252 | p = sre_parse.parse_template(repl, pattern) |
Fredrik Lundh | b25e1ad | 2001-03-22 15:50:10 +0000 | [diff] [blame] | 253 | if len(_cache_repl) >= _MAXCACHE: |
| 254 | _cache_repl.clear() |
| 255 | _cache_repl[key] = p |
| 256 | return p |
| 257 | |
Fredrik Lundh | 5644b7f | 2000-09-21 17:03:25 +0000 | [diff] [blame] | 258 | def _expand(pattern, match, template): |
| 259 | # internal: match.expand implementation hook |
| 260 | template = sre_parse.parse_template(template, pattern) |
| 261 | return sre_parse.expand_template(template, match) |
| 262 | |
Fredrik Lundh | bec95b9 | 2001-10-21 16:47:57 +0000 | [diff] [blame] | 263 | def _subx(pattern, template): |
| 264 | # internal: pattern.sub/subn implementation helper |
Fredrik Lundh | dac5849 | 2001-10-21 21:48:30 +0000 | [diff] [blame] | 265 | template = _compile_repl(template, pattern) |
| 266 | if not template[0] and len(template[1]) == 1: |
Fredrik Lundh | bec95b9 | 2001-10-21 16:47:57 +0000 | [diff] [blame] | 267 | # literal replacement |
Fredrik Lundh | dac5849 | 2001-10-21 21:48:30 +0000 | [diff] [blame] | 268 | return template[1][0] |
| 269 | def filter(match, template=template): |
| 270 | return sre_parse.expand_template(template, match) |
| 271 | return filter |
Fredrik Lundh | 0640e11 | 2000-06-30 13:55:15 +0000 | [diff] [blame] | 272 | |
| 273 | # register myself for pickling |
| 274 | |
Alexandre Vassalotti | f7fa63d | 2008-05-11 08:55:36 +0000 | [diff] [blame] | 275 | import copyreg |
Fredrik Lundh | 0640e11 | 2000-06-30 13:55:15 +0000 | [diff] [blame] | 276 | |
| 277 | def _pickle(p): |
| 278 | return _compile, (p.pattern, p.flags) |
| 279 | |
Alexandre Vassalotti | f7fa63d | 2008-05-11 08:55:36 +0000 | [diff] [blame] | 280 | copyreg.pickle(_pattern_type, _pickle, _compile) |
Fredrik Lundh | 7cafe4d | 2000-07-02 17:33:27 +0000 | [diff] [blame] | 281 | |
| 282 | # -------------------------------------------------------------------- |
| 283 | # experimental stuff (see python-dev discussions for details) |
| 284 | |
| 285 | class Scanner: |
Fredrik Lundh | 1296a8d | 2001-10-21 18:04:11 +0000 | [diff] [blame] | 286 | def __init__(self, lexicon, flags=0): |
Fredrik Lundh | 29c4ba9 | 2000-08-01 18:20:07 +0000 | [diff] [blame] | 287 | from sre_constants import BRANCH, SUBPATTERN |
Fredrik Lundh | 7cafe4d | 2000-07-02 17:33:27 +0000 | [diff] [blame] | 288 | self.lexicon = lexicon |
Fredrik Lundh | 8a3ebf8 | 2000-07-23 21:46:17 +0000 | [diff] [blame] | 289 | # combine phrases into a compound pattern |
Fredrik Lundh | 7cafe4d | 2000-07-02 17:33:27 +0000 | [diff] [blame] | 290 | p = [] |
Fredrik Lundh | 8a3ebf8 | 2000-07-23 21:46:17 +0000 | [diff] [blame] | 291 | s = sre_parse.Pattern() |
Fredrik Lundh | 1296a8d | 2001-10-21 18:04:11 +0000 | [diff] [blame] | 292 | s.flags = flags |
Fredrik Lundh | 7cafe4d | 2000-07-02 17:33:27 +0000 | [diff] [blame] | 293 | for phrase, action in lexicon: |
Fredrik Lundh | 8a3ebf8 | 2000-07-23 21:46:17 +0000 | [diff] [blame] | 294 | p.append(sre_parse.SubPattern(s, [ |
Fredrik Lundh | 1296a8d | 2001-10-21 18:04:11 +0000 | [diff] [blame] | 295 | (SUBPATTERN, (len(p)+1, sre_parse.parse(phrase, flags))), |
Fredrik Lundh | 8a3ebf8 | 2000-07-23 21:46:17 +0000 | [diff] [blame] | 296 | ])) |
Christian Heimes | 2202f87 | 2008-02-06 14:31:34 +0000 | [diff] [blame] | 297 | s.groups = len(p)+1 |
Fredrik Lundh | 8a3ebf8 | 2000-07-23 21:46:17 +0000 | [diff] [blame] | 298 | p = sre_parse.SubPattern(s, [(BRANCH, (None, p))]) |
Fredrik Lundh | 8a3ebf8 | 2000-07-23 21:46:17 +0000 | [diff] [blame] | 299 | self.scanner = sre_compile.compile(p) |
Fredrik Lundh | 7cafe4d | 2000-07-02 17:33:27 +0000 | [diff] [blame] | 300 | def scan(self, string): |
| 301 | result = [] |
| 302 | append = result.append |
Fredrik Lundh | 1296a8d | 2001-10-21 18:04:11 +0000 | [diff] [blame] | 303 | match = self.scanner.scanner(string).match |
Fredrik Lundh | 7cafe4d | 2000-07-02 17:33:27 +0000 | [diff] [blame] | 304 | i = 0 |
| 305 | while 1: |
Fredrik Lundh | 1296a8d | 2001-10-21 18:04:11 +0000 | [diff] [blame] | 306 | m = match() |
Fredrik Lundh | 7cafe4d | 2000-07-02 17:33:27 +0000 | [diff] [blame] | 307 | if not m: |
| 308 | break |
| 309 | j = m.end() |
| 310 | if i == j: |
| 311 | break |
Fredrik Lundh | 1296a8d | 2001-10-21 18:04:11 +0000 | [diff] [blame] | 312 | action = self.lexicon[m.lastindex-1][1] |
Guido van Rossum | d59da4b | 2007-05-22 18:11:13 +0000 | [diff] [blame] | 313 | if hasattr(action, '__call__'): |
Fredrik Lundh | 770617b | 2001-01-14 15:06:11 +0000 | [diff] [blame] | 314 | self.match = m |
Fredrik Lundh | 7cafe4d | 2000-07-02 17:33:27 +0000 | [diff] [blame] | 315 | action = action(self, m.group()) |
| 316 | if action is not None: |
| 317 | append(action) |
| 318 | i = j |
| 319 | return result, string[i:] |