blob: ea6563bde04ddb3700cc788943ef51676b239e67 [file] [log] [blame]
Georg Brandl116aa622007-08-15 14:28:22 +00001
2:mod:`re` --- Regular expression operations
3===========================================
4
5.. module:: re
6 :synopsis: Regular expression operations.
7.. moduleauthor:: Fredrik Lundh <fredrik@pythonware.com>
8.. sectionauthor:: Andrew M. Kuchling <amk@amk.ca>
9
10
Georg Brandl116aa622007-08-15 14:28:22 +000011This module provides regular expression matching operations similar to
Georg Brandled2a1db2009-06-08 07:48:27 +000012those found in Perl.
Antoine Pitroufd036452008-08-19 17:56:33 +000013
14Both patterns and strings to be searched can be Unicode strings as well as
158-bit strings. However, Unicode strings and 8-bit strings cannot be mixed:
16that is, you cannot match an Unicode string with a byte pattern or
Georg Brandlae2dbe22009-03-13 19:04:40 +000017vice-versa; similarly, when asking for a substitution, the replacement
Antoine Pitroufd036452008-08-19 17:56:33 +000018string must be of the same type as both the pattern and the search string.
Georg Brandl116aa622007-08-15 14:28:22 +000019
20Regular expressions use the backslash character (``'\'``) to indicate
21special forms or to allow special characters to be used without invoking
22their special meaning. This collides with Python's usage of the same
23character for the same purpose in string literals; for example, to match
24a literal backslash, one might have to write ``'\\\\'`` as the pattern
25string, because the regular expression must be ``\\``, and each
26backslash must be expressed as ``\\`` inside a regular Python string
27literal.
28
29The solution is to use Python's raw string notation for regular expression
30patterns; backslashes are not handled in any special way in a string literal
31prefixed with ``'r'``. So ``r"\n"`` is a two-character string containing
32``'\'`` and ``'n'``, while ``"\n"`` is a one-character string containing a
Georg Brandl9afde1c2007-11-01 20:32:30 +000033newline. Usually patterns will be expressed in Python code using this raw
34string notation.
Georg Brandl116aa622007-08-15 14:28:22 +000035
Christian Heimesb9eccbf2007-12-05 20:18:38 +000036It is important to note that most regular expression operations are available as
37module-level functions and :class:`RegexObject` methods. The functions are
38shortcuts that don't require you to compile a regex object first, but miss some
39fine-tuning parameters.
40
Georg Brandl116aa622007-08-15 14:28:22 +000041.. seealso::
42
43 Mastering Regular Expressions
44 Book on regular expressions by Jeffrey Friedl, published by O'Reilly. The
Georg Brandl9afde1c2007-11-01 20:32:30 +000045 second edition of the book no longer covers Python at all, but the first
Georg Brandl116aa622007-08-15 14:28:22 +000046 edition covered writing good regular expression patterns in great detail.
47
48
49.. _re-syntax:
50
51Regular Expression Syntax
52-------------------------
53
54A regular expression (or RE) specifies a set of strings that matches it; the
55functions in this module let you check if a particular string matches a given
56regular expression (or if a given regular expression matches a particular
57string, which comes down to the same thing).
58
59Regular expressions can be concatenated to form new regular expressions; if *A*
60and *B* are both regular expressions, then *AB* is also a regular expression.
61In general, if a string *p* matches *A* and another string *q* matches *B*, the
62string *pq* will match AB. This holds unless *A* or *B* contain low precedence
63operations; boundary conditions between *A* and *B*; or have numbered group
64references. Thus, complex expressions can easily be constructed from simpler
65primitive expressions like the ones described here. For details of the theory
66and implementation of regular expressions, consult the Friedl book referenced
67above, or almost any textbook about compiler construction.
68
69A brief explanation of the format of regular expressions follows. For further
Christian Heimes2202f872008-02-06 14:31:34 +000070information and a gentler presentation, consult the :ref:`regex-howto`.
Georg Brandl116aa622007-08-15 14:28:22 +000071
72Regular expressions can contain both special and ordinary characters. Most
73ordinary characters, like ``'A'``, ``'a'``, or ``'0'``, are the simplest regular
74expressions; they simply match themselves. You can concatenate ordinary
75characters, so ``last`` matches the string ``'last'``. (In the rest of this
76section, we'll write RE's in ``this special style``, usually without quotes, and
77strings to be matched ``'in single quotes'``.)
78
79Some characters, like ``'|'`` or ``'('``, are special. Special
80characters either stand for classes of ordinary characters, or affect
81how the regular expressions around them are interpreted. Regular
82expression pattern strings may not contain null bytes, but can specify
83the null byte using the ``\number`` notation, e.g., ``'\x00'``.
84
85
86The special characters are:
87
Georg Brandl116aa622007-08-15 14:28:22 +000088``'.'``
89 (Dot.) In the default mode, this matches any character except a newline. If
90 the :const:`DOTALL` flag has been specified, this matches any character
91 including a newline.
92
93``'^'``
94 (Caret.) Matches the start of the string, and in :const:`MULTILINE` mode also
95 matches immediately after each newline.
96
97``'$'``
98 Matches the end of the string or just before the newline at the end of the
99 string, and in :const:`MULTILINE` mode also matches before a newline. ``foo``
100 matches both 'foo' and 'foobar', while the regular expression ``foo$`` matches
101 only 'foo'. More interestingly, searching for ``foo.$`` in ``'foo1\nfoo2\n'``
Christian Heimes25bb7832008-01-11 16:17:00 +0000102 matches 'foo2' normally, but 'foo1' in :const:`MULTILINE` mode; searching for
103 a single ``$`` in ``'foo\n'`` will find two (empty) matches: one just before
104 the newline, and one at the end of the string.
Georg Brandl116aa622007-08-15 14:28:22 +0000105
106``'*'``
107 Causes the resulting RE to match 0 or more repetitions of the preceding RE, as
108 many repetitions as are possible. ``ab*`` will match 'a', 'ab', or 'a' followed
109 by any number of 'b's.
110
111``'+'``
112 Causes the resulting RE to match 1 or more repetitions of the preceding RE.
113 ``ab+`` will match 'a' followed by any non-zero number of 'b's; it will not
114 match just 'a'.
115
116``'?'``
117 Causes the resulting RE to match 0 or 1 repetitions of the preceding RE.
118 ``ab?`` will match either 'a' or 'ab'.
119
120``*?``, ``+?``, ``??``
121 The ``'*'``, ``'+'``, and ``'?'`` qualifiers are all :dfn:`greedy`; they match
122 as much text as possible. Sometimes this behaviour isn't desired; if the RE
123 ``<.*>`` is matched against ``'<H1>title</H1>'``, it will match the entire
124 string, and not just ``'<H1>'``. Adding ``'?'`` after the qualifier makes it
125 perform the match in :dfn:`non-greedy` or :dfn:`minimal` fashion; as *few*
126 characters as possible will be matched. Using ``.*?`` in the previous
127 expression will match only ``'<H1>'``.
128
129``{m}``
130 Specifies that exactly *m* copies of the previous RE should be matched; fewer
131 matches cause the entire RE not to match. For example, ``a{6}`` will match
132 exactly six ``'a'`` characters, but not five.
133
134``{m,n}``
135 Causes the resulting RE to match from *m* to *n* repetitions of the preceding
136 RE, attempting to match as many repetitions as possible. For example,
137 ``a{3,5}`` will match from 3 to 5 ``'a'`` characters. Omitting *m* specifies a
138 lower bound of zero, and omitting *n* specifies an infinite upper bound. As an
139 example, ``a{4,}b`` will match ``aaaab`` or a thousand ``'a'`` characters
140 followed by a ``b``, but not ``aaab``. The comma may not be omitted or the
141 modifier would be confused with the previously described form.
142
143``{m,n}?``
144 Causes the resulting RE to match from *m* to *n* repetitions of the preceding
145 RE, attempting to match as *few* repetitions as possible. This is the
146 non-greedy version of the previous qualifier. For example, on the
147 6-character string ``'aaaaaa'``, ``a{3,5}`` will match 5 ``'a'`` characters,
148 while ``a{3,5}?`` will only match 3 characters.
149
150``'\'``
151 Either escapes special characters (permitting you to match characters like
152 ``'*'``, ``'?'``, and so forth), or signals a special sequence; special
153 sequences are discussed below.
154
155 If you're not using a raw string to express the pattern, remember that Python
156 also uses the backslash as an escape sequence in string literals; if the escape
157 sequence isn't recognized by Python's parser, the backslash and subsequent
158 character are included in the resulting string. However, if Python would
159 recognize the resulting sequence, the backslash should be repeated twice. This
160 is complicated and hard to understand, so it's highly recommended that you use
161 raw strings for all but the simplest expressions.
162
163``[]``
164 Used to indicate a set of characters. Characters can be listed individually, or
165 a range of characters can be indicated by giving two characters and separating
166 them by a ``'-'``. Special characters are not active inside sets. For example,
167 ``[akm$]`` will match any of the characters ``'a'``, ``'k'``,
168 ``'m'``, or ``'$'``; ``[a-z]`` will match any lowercase letter, and
169 ``[a-zA-Z0-9]`` matches any letter or digit. Character classes such
170 as ``\w`` or ``\S`` (defined below) are also acceptable inside a
Mark Summerfield86765342008-08-20 07:40:18 +0000171 range, although the characters they match depends on whether
172 :const:`ASCII` or :const:`LOCALE` mode is in force. If you want to
173 include a ``']'`` or a ``'-'`` inside a set, precede it with a
174 backslash, or place it as the first character. The pattern ``[]]``
175 will match ``']'``, for example.
Georg Brandl116aa622007-08-15 14:28:22 +0000176
177 You can match the characters not within a range by :dfn:`complementing` the set.
178 This is indicated by including a ``'^'`` as the first character of the set;
179 ``'^'`` elsewhere will simply match the ``'^'`` character. For example,
180 ``[^5]`` will match any character except ``'5'``, and ``[^^]`` will match any
181 character except ``'^'``.
182
Mark Summerfield9e670c22008-05-31 13:05:34 +0000183 Note that inside ``[]`` the special forms and special characters lose
184 their meanings and only the syntaxes described here are valid. For
185 example, ``+``, ``*``, ``(``, ``)``, and so on are treated as
186 literals inside ``[]``, and backreferences cannot be used inside
187 ``[]``.
188
Georg Brandl116aa622007-08-15 14:28:22 +0000189``'|'``
190 ``A|B``, where A and B can be arbitrary REs, creates a regular expression that
191 will match either A or B. An arbitrary number of REs can be separated by the
192 ``'|'`` in this way. This can be used inside groups (see below) as well. As
193 the target string is scanned, REs separated by ``'|'`` are tried from left to
194 right. When one pattern completely matches, that branch is accepted. This means
195 that once ``A`` matches, ``B`` will not be tested further, even if it would
196 produce a longer overall match. In other words, the ``'|'`` operator is never
197 greedy. To match a literal ``'|'``, use ``\|``, or enclose it inside a
198 character class, as in ``[|]``.
199
200``(...)``
201 Matches whatever regular expression is inside the parentheses, and indicates the
202 start and end of a group; the contents of a group can be retrieved after a match
203 has been performed, and can be matched later in the string with the ``\number``
204 special sequence, described below. To match the literals ``'('`` or ``')'``,
205 use ``\(`` or ``\)``, or enclose them inside a character class: ``[(] [)]``.
206
207``(?...)``
208 This is an extension notation (a ``'?'`` following a ``'('`` is not meaningful
209 otherwise). The first character after the ``'?'`` determines what the meaning
210 and further syntax of the construct is. Extensions usually do not create a new
211 group; ``(?P<name>...)`` is the only exception to this rule. Following are the
212 currently supported extensions.
213
Antoine Pitroufd036452008-08-19 17:56:33 +0000214``(?aiLmsux)``
215 (One or more letters from the set ``'a'``, ``'i'``, ``'L'``, ``'m'``,
216 ``'s'``, ``'u'``, ``'x'``.) The group matches the empty string; the
Andrew M. Kuchling1c50e862009-06-01 00:11:36 +0000217 letters set the corresponding flags: :const:`re.A` (ASCII-only matching),
Antoine Pitroufd036452008-08-19 17:56:33 +0000218 :const:`re.I` (ignore case), :const:`re.L` (locale dependent),
Georg Brandl48310cd2009-01-03 21:18:54 +0000219 :const:`re.M` (multi-line), :const:`re.S` (dot matches all),
Georg Brandl116aa622007-08-15 14:28:22 +0000220 and :const:`re.X` (verbose), for the entire regular expression. (The
221 flags are described in :ref:`contents-of-module-re`.) This
222 is useful if you wish to include the flags as part of the regular
223 expression, instead of passing a *flag* argument to the
Georg Brandlf346ac02009-07-26 15:03:49 +0000224 :func:`re.compile` function.
Georg Brandl116aa622007-08-15 14:28:22 +0000225
226 Note that the ``(?x)`` flag changes how the expression is parsed. It should be
227 used first in the expression string, or after one or more whitespace characters.
228 If there are non-whitespace characters before the flag, the results are
229 undefined.
230
231``(?:...)``
232 A non-grouping version of regular parentheses. Matches whatever regular
233 expression is inside the parentheses, but the substring matched by the group
234 *cannot* be retrieved after performing a match or referenced later in the
235 pattern.
236
237``(?P<name>...)``
238 Similar to regular parentheses, but the substring matched by the group is
Benjamin Petersond23f8222009-04-05 19:13:16 +0000239 accessible within the rest of the regular expression via the symbolic group
240 name *name*. Group names must be valid Python identifiers, and each group
241 name must be defined only once within a regular expression. A symbolic group
242 is also a numbered group, just as if the group were not named. So the group
243 named ``id`` in the example below can also be referenced as the numbered group
244 ``1``.
Georg Brandl116aa622007-08-15 14:28:22 +0000245
246 For example, if the pattern is ``(?P<id>[a-zA-Z_]\w*)``, the group can be
247 referenced by its name in arguments to methods of match objects, such as
Benjamin Petersond23f8222009-04-05 19:13:16 +0000248 ``m.group('id')`` or ``m.end('id')``, and also by name in the regular
249 expression itself (using ``(?P=id)``) and replacement text given to
250 ``.sub()`` (using ``\g<id>``).
Georg Brandl116aa622007-08-15 14:28:22 +0000251
252``(?P=name)``
253 Matches whatever text was matched by the earlier group named *name*.
254
255``(?#...)``
256 A comment; the contents of the parentheses are simply ignored.
257
258``(?=...)``
259 Matches if ``...`` matches next, but doesn't consume any of the string. This is
260 called a lookahead assertion. For example, ``Isaac (?=Asimov)`` will match
261 ``'Isaac '`` only if it's followed by ``'Asimov'``.
262
263``(?!...)``
264 Matches if ``...`` doesn't match next. This is a negative lookahead assertion.
265 For example, ``Isaac (?!Asimov)`` will match ``'Isaac '`` only if it's *not*
266 followed by ``'Asimov'``.
267
268``(?<=...)``
269 Matches if the current position in the string is preceded by a match for ``...``
270 that ends at the current position. This is called a :dfn:`positive lookbehind
271 assertion`. ``(?<=abc)def`` will find a match in ``abcdef``, since the
272 lookbehind will back up 3 characters and check if the contained pattern matches.
273 The contained pattern must only match strings of some fixed length, meaning that
274 ``abc`` or ``a|b`` are allowed, but ``a*`` and ``a{3,4}`` are not. Note that
275 patterns which start with positive lookbehind assertions will never match at the
276 beginning of the string being searched; you will most likely want to use the
Christian Heimesfe337bf2008-03-23 21:54:12 +0000277 :func:`search` function rather than the :func:`match` function:
Georg Brandl116aa622007-08-15 14:28:22 +0000278
279 >>> import re
280 >>> m = re.search('(?<=abc)def', 'abcdef')
281 >>> m.group(0)
282 'def'
283
Christian Heimesfe337bf2008-03-23 21:54:12 +0000284 This example looks for a word following a hyphen:
Georg Brandl116aa622007-08-15 14:28:22 +0000285
286 >>> m = re.search('(?<=-)\w+', 'spam-egg')
287 >>> m.group(0)
288 'egg'
289
290``(?<!...)``
291 Matches if the current position in the string is not preceded by a match for
292 ``...``. This is called a :dfn:`negative lookbehind assertion`. Similar to
293 positive lookbehind assertions, the contained pattern must only match strings of
294 some fixed length. Patterns which start with negative lookbehind assertions may
295 match at the beginning of the string being searched.
296
297``(?(id/name)yes-pattern|no-pattern)``
298 Will try to match with ``yes-pattern`` if the group with given *id* or *name*
299 exists, and with ``no-pattern`` if it doesn't. ``no-pattern`` is optional and
300 can be omitted. For example, ``(<)?(\w+@\w+(?:\.\w+)+)(?(1)>)`` is a poor email
301 matching pattern, which will match with ``'<user@host.com>'`` as well as
302 ``'user@host.com'``, but not with ``'<user@host.com'``.
303
Georg Brandl116aa622007-08-15 14:28:22 +0000304
305The special sequences consist of ``'\'`` and a character from the list below.
306If the ordinary character is not on the list, then the resulting RE will match
307the second character. For example, ``\$`` matches the character ``'$'``.
308
Georg Brandl116aa622007-08-15 14:28:22 +0000309``\number``
310 Matches the contents of the group of the same number. Groups are numbered
311 starting from 1. For example, ``(.+) \1`` matches ``'the the'`` or ``'55 55'``,
312 but not ``'the end'`` (note the space after the group). This special sequence
313 can only be used to match one of the first 99 groups. If the first digit of
314 *number* is 0, or *number* is 3 octal digits long, it will not be interpreted as
315 a group match, but as the character with octal value *number*. Inside the
316 ``'['`` and ``']'`` of a character class, all numeric escapes are treated as
317 characters.
318
319``\A``
320 Matches only at the start of the string.
321
322``\b``
Mark Summerfield6c4f6172008-08-20 07:34:41 +0000323 Matches the empty string, but only at the beginning or end of a word.
324 A word is defined as a sequence of Unicode alphanumeric or underscore
325 characters, so the end of a word is indicated by whitespace or a
326 non-alphanumeric, non-underscore Unicode character. Note that
327 formally, ``\b`` is defined as the boundary between a ``\w`` and a
328 ``\W`` character (or vice versa). By default Unicode alphanumerics
329 are the ones used, but this can be changed by using the :const:`ASCII`
330 flag. Inside a character range, ``\b`` represents the backspace
331 character, for compatibility with Python's string literals.
Georg Brandl116aa622007-08-15 14:28:22 +0000332
333``\B``
334 Matches the empty string, but only when it is *not* at the beginning or end of a
Mark Summerfield6c4f6172008-08-20 07:34:41 +0000335 word. This is just the opposite of ``\b``, so word characters are
336 Unicode alphanumerics or the underscore, although this can be changed
337 by using the :const:`ASCII` flag.
Georg Brandl116aa622007-08-15 14:28:22 +0000338
339``\d``
Antoine Pitroufd036452008-08-19 17:56:33 +0000340 For Unicode (str) patterns:
Mark Dickinson1f268282009-07-28 17:22:36 +0000341 Matches any Unicode decimal digit (that is, any character in
342 Unicode character category [Nd]). This includes ``[0-9]``, and
343 also many other digit characters. If the :const:`ASCII` flag is
344 used only ``[0-9]`` is matched (but the flag affects the entire
345 regular expression, so in such cases using an explicit ``[0-9]``
346 may be a better choice).
Antoine Pitroufd036452008-08-19 17:56:33 +0000347 For 8-bit (bytes) patterns:
Mark Summerfield6c4f6172008-08-20 07:34:41 +0000348 Matches any decimal digit; this is equivalent to ``[0-9]``.
Georg Brandl116aa622007-08-15 14:28:22 +0000349
350``\D``
Mark Summerfield6c4f6172008-08-20 07:34:41 +0000351 Matches any character which is not a Unicode decimal digit. This is
352 the opposite of ``\d``. If the :const:`ASCII` flag is used this
353 becomes the equivalent of ``[^0-9]`` (but the flag affects the entire
354 regular expression, so in such cases using an explicit ``[^0-9]`` may
355 be a better choice).
Georg Brandl116aa622007-08-15 14:28:22 +0000356
357``\s``
Antoine Pitroufd036452008-08-19 17:56:33 +0000358 For Unicode (str) patterns:
Mark Summerfield6c4f6172008-08-20 07:34:41 +0000359 Matches Unicode whitespace characters (which includes
360 ``[ \t\n\r\f\v]``, and also many other characters, for example the
361 non-breaking spaces mandated by typography rules in many
362 languages). If the :const:`ASCII` flag is used, only
363 ``[ \t\n\r\f\v]`` is matched (but the flag affects the entire
364 regular expression, so in such cases using an explicit
365 ``[ \t\n\r\f\v]`` may be a better choice).
366
Antoine Pitroufd036452008-08-19 17:56:33 +0000367 For 8-bit (bytes) patterns:
368 Matches characters considered whitespace in the ASCII character set;
Mark Summerfield6c4f6172008-08-20 07:34:41 +0000369 this is equivalent to ``[ \t\n\r\f\v]``.
Georg Brandl116aa622007-08-15 14:28:22 +0000370
371``\S``
Mark Summerfield6c4f6172008-08-20 07:34:41 +0000372 Matches any character which is not a Unicode whitespace character. This is
373 the opposite of ``\s``. If the :const:`ASCII` flag is used this
374 becomes the equivalent of ``[^ \t\n\r\f\v]`` (but the flag affects the entire
375 regular expression, so in such cases using an explicit ``[^ \t\n\r\f\v]`` may
376 be a better choice).
Georg Brandl116aa622007-08-15 14:28:22 +0000377
378``\w``
Antoine Pitroufd036452008-08-19 17:56:33 +0000379 For Unicode (str) patterns:
Mark Summerfield6c4f6172008-08-20 07:34:41 +0000380 Matches Unicode word characters; this includes most characters
381 that can be part of a word in any language, as well as numbers and
382 the underscore. If the :const:`ASCII` flag is used, only
383 ``[a-zA-Z0-9_]`` is matched (but the flag affects the entire
384 regular expression, so in such cases using an explicit
385 ``[a-zA-Z0-9_]`` may be a better choice).
Antoine Pitroufd036452008-08-19 17:56:33 +0000386 For 8-bit (bytes) patterns:
387 Matches characters considered alphanumeric in the ASCII character set;
Mark Summerfield6c4f6172008-08-20 07:34:41 +0000388 this is equivalent to ``[a-zA-Z0-9_]``.
Georg Brandl116aa622007-08-15 14:28:22 +0000389
390``\W``
Mark Summerfield6c4f6172008-08-20 07:34:41 +0000391 Matches any character which is not a Unicode word character. This is
392 the opposite of ``\w``. If the :const:`ASCII` flag is used this
393 becomes the equivalent of ``[^a-zA-Z0-9_]`` (but the flag affects the
394 entire regular expression, so in such cases using an explicit
395 ``[^a-zA-Z0-9_]`` may be a better choice).
Georg Brandl116aa622007-08-15 14:28:22 +0000396
397``\Z``
398 Matches only at the end of the string.
399
400Most of the standard escapes supported by Python string literals are also
401accepted by the regular expression parser::
402
403 \a \b \f \n
404 \r \t \v \x
405 \\
406
407Octal escapes are included in a limited form: If the first digit is a 0, or if
408there are three octal digits, it is considered an octal escape. Otherwise, it is
409a group reference. As for string literals, octal escapes are always at most
410three digits in length.
411
Georg Brandl116aa622007-08-15 14:28:22 +0000412
413.. _matching-searching:
414
415Matching vs Searching
416---------------------
417
418.. sectionauthor:: Fred L. Drake, Jr. <fdrake@acm.org>
419
420
421Python offers two different primitive operations based on regular expressions:
Guido van Rossum04110fb2007-08-24 16:32:05 +0000422**match** checks for a match only at the beginning of the string, while
423**search** checks for a match anywhere in the string (this is what Perl does
424by default).
Georg Brandl116aa622007-08-15 14:28:22 +0000425
Guido van Rossum04110fb2007-08-24 16:32:05 +0000426Note that match may differ from search even when using a regular expression
427beginning with ``'^'``: ``'^'`` matches only at the start of the string, or in
Georg Brandl116aa622007-08-15 14:28:22 +0000428:const:`MULTILINE` mode also immediately following a newline. The "match"
429operation succeeds only if the pattern matches at the start of the string
430regardless of mode, or at the starting position given by the optional *pos*
Christian Heimesfe337bf2008-03-23 21:54:12 +0000431argument regardless of whether a newline precedes it.
Georg Brandl116aa622007-08-15 14:28:22 +0000432
Christian Heimesfe337bf2008-03-23 21:54:12 +0000433 >>> re.match("c", "abcdef") # No match
434 >>> re.search("c", "abcdef") # Match
435 <_sre.SRE_Match object at ...>
Georg Brandl116aa622007-08-15 14:28:22 +0000436
437
438.. _contents-of-module-re:
439
440Module Contents
441---------------
442
443The module defines several functions, constants, and an exception. Some of the
444functions are simplified versions of the full featured methods for compiled
445regular expressions. Most non-trivial applications always use the compiled
446form.
447
448
449.. function:: compile(pattern[, flags])
450
Georg Brandl9afde1c2007-11-01 20:32:30 +0000451 Compile a regular expression pattern into a regular expression object, which
452 can be used for matching using its :func:`match` and :func:`search` methods,
Georg Brandl116aa622007-08-15 14:28:22 +0000453 described below.
454
455 The expression's behaviour can be modified by specifying a *flags* value.
456 Values can be any of the following variables, combined using bitwise OR (the
457 ``|`` operator).
458
459 The sequence ::
460
Gregory P. Smith4221c742009-03-02 05:04:04 +0000461 prog = re.compile(pattern)
462 result = prog.match(string)
Georg Brandl116aa622007-08-15 14:28:22 +0000463
464 is equivalent to ::
465
Gregory P. Smith4221c742009-03-02 05:04:04 +0000466 result = re.match(pattern, string)
Georg Brandl116aa622007-08-15 14:28:22 +0000467
Georg Brandlf346ac02009-07-26 15:03:49 +0000468 but using :func:`re.compile` and saving the resulting regular expression
469 object for reuse is more efficient when the expression will be used several
470 times in a single program.
Georg Brandl116aa622007-08-15 14:28:22 +0000471
Gregory P. Smith4221c742009-03-02 05:04:04 +0000472 .. note::
473
474 The compiled versions of the most recent patterns passed to
475 :func:`re.match`, :func:`re.search` or :func:`re.compile` are cached, so
476 programs that use only a few regular expressions at a time needn't worry
477 about compiling regular expressions.
Georg Brandl116aa622007-08-15 14:28:22 +0000478
479
Antoine Pitroufd036452008-08-19 17:56:33 +0000480.. data:: A
481 ASCII
482
Georg Brandl4049ce02009-06-08 07:49:54 +0000483 Make ``\w``, ``\W``, ``\b``, ``\B``, ``\d``, ``\D``, ``\s`` and ``\S``
484 perform ASCII-only matching instead of full Unicode matching. This is only
485 meaningful for Unicode patterns, and is ignored for byte patterns.
Antoine Pitroufd036452008-08-19 17:56:33 +0000486
Mark Summerfield6c4f6172008-08-20 07:34:41 +0000487 Note that for backward compatibility, the :const:`re.U` flag still
488 exists (as well as its synonym :const:`re.UNICODE` and its embedded
489 counterpart ``(?u)``), but these are redundant in Python 3.0 since
490 matches are Unicode by default for strings (and Unicode matching
491 isn't allowed for bytes).
Georg Brandl48310cd2009-01-03 21:18:54 +0000492
Antoine Pitroufd036452008-08-19 17:56:33 +0000493
Georg Brandl116aa622007-08-15 14:28:22 +0000494.. data:: I
495 IGNORECASE
496
497 Perform case-insensitive matching; expressions like ``[A-Z]`` will match
Mark Summerfield86765342008-08-20 07:40:18 +0000498 lowercase letters, too. This is not affected by the current locale
499 and works for Unicode characters as expected.
Georg Brandl116aa622007-08-15 14:28:22 +0000500
501
502.. data:: L
503 LOCALE
504
Georg Brandl9afde1c2007-11-01 20:32:30 +0000505 Make ``\w``, ``\W``, ``\b``, ``\B``, ``\s`` and ``\S`` dependent on the
Antoine Pitroufd036452008-08-19 17:56:33 +0000506 current locale. The use of this flag is discouraged as the locale mechanism
507 is very unreliable, and it only handles one "culture" at a time anyway;
508 you should use Unicode matching instead, which is the default in Python 3.0
509 for Unicode (str) patterns.
Georg Brandl116aa622007-08-15 14:28:22 +0000510
511
512.. data:: M
513 MULTILINE
514
515 When specified, the pattern character ``'^'`` matches at the beginning of the
516 string and at the beginning of each line (immediately following each newline);
517 and the pattern character ``'$'`` matches at the end of the string and at the
518 end of each line (immediately preceding each newline). By default, ``'^'``
519 matches only at the beginning of the string, and ``'$'`` only at the end of the
520 string and immediately before the newline (if any) at the end of the string.
521
522
523.. data:: S
524 DOTALL
525
526 Make the ``'.'`` special character match any character at all, including a
527 newline; without this flag, ``'.'`` will match anything *except* a newline.
528
529
Georg Brandl116aa622007-08-15 14:28:22 +0000530.. data:: X
531 VERBOSE
532
533 This flag allows you to write regular expressions that look nicer. Whitespace
534 within the pattern is ignored, except when in a character class or preceded by
535 an unescaped backslash, and, when a line contains a ``'#'`` neither in a
536 character class or preceded by an unescaped backslash, all characters from the
537 leftmost such ``'#'`` through the end of the line are ignored.
538
Christian Heimesb9eccbf2007-12-05 20:18:38 +0000539 That means that the two following regular expression objects that match a
540 decimal number are functionally equal::
Georg Brandl81ac1ce2007-08-31 17:17:17 +0000541
Christian Heimesb9eccbf2007-12-05 20:18:38 +0000542 a = re.compile(r"""\d + # the integral part
543 \. # the decimal point
544 \d * # some fractional digits""", re.X)
545 b = re.compile(r"\d+\.\d*")
Georg Brandl116aa622007-08-15 14:28:22 +0000546
547
Antoine Pitroufd036452008-08-19 17:56:33 +0000548
549
Georg Brandl116aa622007-08-15 14:28:22 +0000550.. function:: search(pattern, string[, flags])
551
552 Scan through *string* looking for a location where the regular expression
553 *pattern* produces a match, and return a corresponding :class:`MatchObject`
554 instance. Return ``None`` if no position in the string matches the pattern; note
555 that this is different from finding a zero-length match at some point in the
556 string.
557
558
559.. function:: match(pattern, string[, flags])
560
561 If zero or more characters at the beginning of *string* match the regular
562 expression *pattern*, return a corresponding :class:`MatchObject` instance.
563 Return ``None`` if the string does not match the pattern; note that this is
564 different from a zero-length match.
565
566 .. note::
567
Georg Brandlf346ac02009-07-26 15:03:49 +0000568 If you want to locate a match anywhere in *string*, use :func:`search`
Christian Heimesb9eccbf2007-12-05 20:18:38 +0000569 instead.
Georg Brandl116aa622007-08-15 14:28:22 +0000570
571
Gregory P. Smithccc5ae72009-03-02 05:21:55 +0000572.. function:: split(pattern, string[, maxsplit=0, flags=0])
Georg Brandl116aa622007-08-15 14:28:22 +0000573
574 Split *string* by the occurrences of *pattern*. If capturing parentheses are
575 used in *pattern*, then the text of all groups in the pattern are also returned
576 as part of the resulting list. If *maxsplit* is nonzero, at most *maxsplit*
577 splits occur, and the remainder of the string is returned as the final element
Georg Brandl96473892008-03-06 07:09:43 +0000578 of the list. ::
Georg Brandl116aa622007-08-15 14:28:22 +0000579
580 >>> re.split('\W+', 'Words, words, words.')
581 ['Words', 'words', 'words', '']
582 >>> re.split('(\W+)', 'Words, words, words.')
583 ['Words', ', ', 'words', ', ', 'words', '.', '']
584 >>> re.split('\W+', 'Words, words, words.', 1)
585 ['Words', 'words, words.']
Gregory P. Smithccc5ae72009-03-02 05:21:55 +0000586 >>> re.split('[a-f]+', '0a3B9', flags=re.IGNORECASE)
587 ['0', '3', '9']
Georg Brandl116aa622007-08-15 14:28:22 +0000588
Christian Heimesdd15f6c2008-03-16 00:07:10 +0000589 If there are capturing groups in the separator and it matches at the start of
590 the string, the result will start with an empty string. The same holds for
Christian Heimesfe337bf2008-03-23 21:54:12 +0000591 the end of the string:
Christian Heimesdd15f6c2008-03-16 00:07:10 +0000592
593 >>> re.split('(\W+)', '...words, words...')
594 ['', '...', 'words', ', ', 'words', '...', '']
595
596 That way, separator components are always found at the same relative
597 indices within the result list (e.g., if there's one capturing group
598 in the separator, the 0th, the 2nd and so forth).
599
Thomas Wouters89d996e2007-09-08 17:39:28 +0000600 Note that *split* will never split a string on an empty pattern match.
Christian Heimesfe337bf2008-03-23 21:54:12 +0000601 For example:
Thomas Wouters89d996e2007-09-08 17:39:28 +0000602
603 >>> re.split('x*', 'foo')
604 ['foo']
605 >>> re.split("(?m)^$", "foo\n\nbar\n")
606 ['foo\n\nbar\n']
Georg Brandl116aa622007-08-15 14:28:22 +0000607
Jeroen Ruigrok van der Wervenb70ccc32009-04-27 08:07:12 +0000608 .. versionchanged:: 3.1
Gregory P. Smithccc5ae72009-03-02 05:21:55 +0000609 Added the optional flags argument.
610
Christian Heimesdd15f6c2008-03-16 00:07:10 +0000611
Georg Brandl116aa622007-08-15 14:28:22 +0000612.. function:: findall(pattern, string[, flags])
613
Georg Brandl9afde1c2007-11-01 20:32:30 +0000614 Return all non-overlapping matches of *pattern* in *string*, as a list of
Georg Brandl3dbca812008-07-23 16:10:53 +0000615 strings. The *string* is scanned left-to-right, and matches are returned in
616 the order found. If one or more groups are present in the pattern, return a
617 list of groups; this will be a list of tuples if the pattern has more than
618 one group. Empty matches are included in the result unless they touch the
619 beginning of another match.
Georg Brandl116aa622007-08-15 14:28:22 +0000620
Georg Brandl116aa622007-08-15 14:28:22 +0000621
622.. function:: finditer(pattern, string[, flags])
623
Georg Brandl9afde1c2007-11-01 20:32:30 +0000624 Return an :term:`iterator` yielding :class:`MatchObject` instances over all
Georg Brandl3dbca812008-07-23 16:10:53 +0000625 non-overlapping matches for the RE *pattern* in *string*. The *string* is
626 scanned left-to-right, and matches are returned in the order found. Empty
627 matches are included in the result unless they touch the beginning of another
628 match.
Georg Brandl116aa622007-08-15 14:28:22 +0000629
Georg Brandl116aa622007-08-15 14:28:22 +0000630
Gregory P. Smithccc5ae72009-03-02 05:21:55 +0000631.. function:: sub(pattern, repl, string[, count, flags])
Georg Brandl116aa622007-08-15 14:28:22 +0000632
633 Return the string obtained by replacing the leftmost non-overlapping occurrences
634 of *pattern* in *string* by the replacement *repl*. If the pattern isn't found,
635 *string* is returned unchanged. *repl* can be a string or a function; if it is
636 a string, any backslash escapes in it are processed. That is, ``\n`` is
637 converted to a single newline character, ``\r`` is converted to a linefeed, and
638 so forth. Unknown escapes such as ``\j`` are left alone. Backreferences, such
639 as ``\6``, are replaced with the substring matched by group 6 in the pattern.
Christian Heimesfe337bf2008-03-23 21:54:12 +0000640 For example:
Georg Brandl116aa622007-08-15 14:28:22 +0000641
642 >>> re.sub(r'def\s+([a-zA-Z_][a-zA-Z_0-9]*)\s*\(\s*\):',
643 ... r'static PyObject*\npy_\1(void)\n{',
644 ... 'def myfunc():')
645 'static PyObject*\npy_myfunc(void)\n{'
646
647 If *repl* is a function, it is called for every non-overlapping occurrence of
648 *pattern*. The function takes a single match object argument, and returns the
Christian Heimesfe337bf2008-03-23 21:54:12 +0000649 replacement string. For example:
Georg Brandl116aa622007-08-15 14:28:22 +0000650
651 >>> def dashrepl(matchobj):
652 ... if matchobj.group(0) == '-': return ' '
653 ... else: return '-'
654 >>> re.sub('-{1,2}', dashrepl, 'pro----gram-files')
655 'pro--gram files'
Gregory P. Smithccc5ae72009-03-02 05:21:55 +0000656 >>> re.sub(r'\sAND\s', ' & ', 'Baked Beans And Spam', flags=re.IGNORECASE)
657 'Baked Beans & Spam'
Georg Brandl116aa622007-08-15 14:28:22 +0000658
Georg Brandl1b5ab452009-08-13 07:56:35 +0000659 The pattern may be a string or an RE object.
Georg Brandl116aa622007-08-15 14:28:22 +0000660
661 The optional argument *count* is the maximum number of pattern occurrences to be
662 replaced; *count* must be a non-negative integer. If omitted or zero, all
663 occurrences will be replaced. Empty matches for the pattern are replaced only
664 when not adjacent to a previous match, so ``sub('x*', '-', 'abc')`` returns
665 ``'-a-b-c-'``.
666
667 In addition to character escapes and backreferences as described above,
668 ``\g<name>`` will use the substring matched by the group named ``name``, as
669 defined by the ``(?P<name>...)`` syntax. ``\g<number>`` uses the corresponding
670 group number; ``\g<2>`` is therefore equivalent to ``\2``, but isn't ambiguous
671 in a replacement such as ``\g<2>0``. ``\20`` would be interpreted as a
672 reference to group 20, not a reference to group 2 followed by the literal
673 character ``'0'``. The backreference ``\g<0>`` substitutes in the entire
674 substring matched by the RE.
675
Jeroen Ruigrok van der Wervenb70ccc32009-04-27 08:07:12 +0000676 .. versionchanged:: 3.1
Gregory P. Smithccc5ae72009-03-02 05:21:55 +0000677 Added the optional flags argument.
Georg Brandl116aa622007-08-15 14:28:22 +0000678
Gregory P. Smithccc5ae72009-03-02 05:21:55 +0000679
680.. function:: subn(pattern, repl, string[, count, flags])
Georg Brandl116aa622007-08-15 14:28:22 +0000681
682 Perform the same operation as :func:`sub`, but return a tuple ``(new_string,
683 number_of_subs_made)``.
684
Jeroen Ruigrok van der Wervenb70ccc32009-04-27 08:07:12 +0000685 .. versionchanged:: 3.1
Gregory P. Smithccc5ae72009-03-02 05:21:55 +0000686 Added the optional flags argument.
687
Georg Brandl116aa622007-08-15 14:28:22 +0000688
689.. function:: escape(string)
690
691 Return *string* with all non-alphanumerics backslashed; this is useful if you
692 want to match an arbitrary literal string that may have regular expression
693 metacharacters in it.
694
695
696.. exception:: error
697
698 Exception raised when a string passed to one of the functions here is not a
699 valid regular expression (for example, it might contain unmatched parentheses)
700 or when some other error occurs during compilation or matching. It is never an
701 error if a string contains no match for a pattern.
702
703
704.. _re-objects:
705
706Regular Expression Objects
707--------------------------
708
709Compiled regular expression objects support the following methods and
710attributes:
711
712
713.. method:: RegexObject.match(string[, pos[, endpos]])
714
715 If zero or more characters at the beginning of *string* match this regular
716 expression, return a corresponding :class:`MatchObject` instance. Return
717 ``None`` if the string does not match the pattern; note that this is different
718 from a zero-length match.
719
720 .. note::
721
Georg Brandlf346ac02009-07-26 15:03:49 +0000722 If you want to locate a match anywhere in *string*, use
723 :meth:`~RegexObject.search` instead.
Georg Brandl116aa622007-08-15 14:28:22 +0000724
725 The optional second parameter *pos* gives an index in the string where the
726 search is to start; it defaults to ``0``. This is not completely equivalent to
727 slicing the string; the ``'^'`` pattern character matches at the real beginning
728 of the string and at positions just after a newline, but not necessarily at the
729 index where the search is to start.
730
731 The optional parameter *endpos* limits how far the string will be searched; it
732 will be as if the string is *endpos* characters long, so only the characters
733 from *pos* to ``endpos - 1`` will be searched for a match. If *endpos* is less
734 than *pos*, no match will be found, otherwise, if *rx* is a compiled regular
735 expression object, ``rx.match(string, 0, 50)`` is equivalent to
Christian Heimesfe337bf2008-03-23 21:54:12 +0000736 ``rx.match(string[:50], 0)``.
Christian Heimesb9eccbf2007-12-05 20:18:38 +0000737
738 >>> pattern = re.compile("o")
739 >>> pattern.match("dog") # No match as "o" is not at the start of "dog."
740 >>> pattern.match("dog", 1) # Match as "o" is the 2nd character of "dog".
Christian Heimesfe337bf2008-03-23 21:54:12 +0000741 <_sre.SRE_Match object at ...>
Georg Brandl116aa622007-08-15 14:28:22 +0000742
743
744.. method:: RegexObject.search(string[, pos[, endpos]])
745
746 Scan through *string* looking for a location where this regular expression
747 produces a match, and return a corresponding :class:`MatchObject` instance.
748 Return ``None`` if no position in the string matches the pattern; note that this
749 is different from finding a zero-length match at some point in the string.
750
751 The optional *pos* and *endpos* parameters have the same meaning as for the
Georg Brandlf346ac02009-07-26 15:03:49 +0000752 :meth:`~RegexObject.match` method.
Georg Brandl116aa622007-08-15 14:28:22 +0000753
754
755.. method:: RegexObject.split(string[, maxsplit=0])
756
757 Identical to the :func:`split` function, using the compiled pattern.
758
759
760.. method:: RegexObject.findall(string[, pos[, endpos]])
761
762 Identical to the :func:`findall` function, using the compiled pattern.
763
764
765.. method:: RegexObject.finditer(string[, pos[, endpos]])
766
767 Identical to the :func:`finditer` function, using the compiled pattern.
768
769
770.. method:: RegexObject.sub(repl, string[, count=0])
771
772 Identical to the :func:`sub` function, using the compiled pattern.
773
774
775.. method:: RegexObject.subn(repl, string[, count=0])
776
777 Identical to the :func:`subn` function, using the compiled pattern.
778
779
780.. attribute:: RegexObject.flags
781
782 The flags argument used when the RE object was compiled, or ``0`` if no flags
783 were provided.
784
785
Georg Brandlaf265f42008-12-07 15:06:20 +0000786.. attribute:: RegexObject.groups
787
788 The number of capturing groups in the pattern.
789
790
Georg Brandl116aa622007-08-15 14:28:22 +0000791.. attribute:: RegexObject.groupindex
792
793 A dictionary mapping any symbolic group names defined by ``(?P<id>)`` to group
794 numbers. The dictionary is empty if no symbolic groups were used in the
795 pattern.
796
797
798.. attribute:: RegexObject.pattern
799
800 The pattern string from which the RE object was compiled.
801
802
803.. _match-objects:
804
805Match Objects
806-------------
807
Georg Brandl9afde1c2007-11-01 20:32:30 +0000808Match objects always have a boolean value of :const:`True`, so that you can test
809whether e.g. :func:`match` resulted in a match with a simple if statement. They
810support the following methods and attributes:
Georg Brandl116aa622007-08-15 14:28:22 +0000811
812
813.. method:: MatchObject.expand(template)
814
815 Return the string obtained by doing backslash substitution on the template
Georg Brandlf346ac02009-07-26 15:03:49 +0000816 string *template*, as done by the :meth:`~RegexObject.sub` method. Escapes
817 such as ``\n`` are converted to the appropriate characters, and numeric
818 backreferences (``\1``, ``\2``) and named backreferences (``\g<1>``,
819 ``\g<name>``) are replaced by the contents of the corresponding group.
Georg Brandl116aa622007-08-15 14:28:22 +0000820
821
822.. method:: MatchObject.group([group1, ...])
823
824 Returns one or more subgroups of the match. If there is a single argument, the
825 result is a single string; if there are multiple arguments, the result is a
826 tuple with one item per argument. Without arguments, *group1* defaults to zero
827 (the whole match is returned). If a *groupN* argument is zero, the corresponding
828 return value is the entire matching string; if it is in the inclusive range
829 [1..99], it is the string matching the corresponding parenthesized group. If a
830 group number is negative or larger than the number of groups defined in the
831 pattern, an :exc:`IndexError` exception is raised. If a group is contained in a
832 part of the pattern that did not match, the corresponding result is ``None``.
833 If a group is contained in a part of the pattern that matched multiple times,
Christian Heimesfe337bf2008-03-23 21:54:12 +0000834 the last match is returned.
Christian Heimesb9eccbf2007-12-05 20:18:38 +0000835
836 >>> m = re.match(r"(\w+) (\w+)", "Isaac Newton, physicist")
Christian Heimesfe337bf2008-03-23 21:54:12 +0000837 >>> m.group(0) # The entire match
838 'Isaac Newton'
839 >>> m.group(1) # The first parenthesized subgroup.
840 'Isaac'
841 >>> m.group(2) # The second parenthesized subgroup.
842 'Newton'
843 >>> m.group(1, 2) # Multiple arguments give us a tuple.
844 ('Isaac', 'Newton')
Georg Brandl116aa622007-08-15 14:28:22 +0000845
846 If the regular expression uses the ``(?P<name>...)`` syntax, the *groupN*
847 arguments may also be strings identifying groups by their group name. If a
848 string argument is not used as a group name in the pattern, an :exc:`IndexError`
849 exception is raised.
850
Christian Heimesfe337bf2008-03-23 21:54:12 +0000851 A moderately complicated example:
Georg Brandl116aa622007-08-15 14:28:22 +0000852
Christian Heimesb9eccbf2007-12-05 20:18:38 +0000853 >>> m = re.match(r"(?P<first_name>\w+) (?P<last_name>\w+)", "Malcom Reynolds")
854 >>> m.group('first_name')
855 'Malcom'
856 >>> m.group('last_name')
857 'Reynolds'
Georg Brandl116aa622007-08-15 14:28:22 +0000858
Christian Heimesfe337bf2008-03-23 21:54:12 +0000859 Named groups can also be referred to by their index:
Christian Heimesb9eccbf2007-12-05 20:18:38 +0000860
861 >>> m.group(1)
862 'Malcom'
863 >>> m.group(2)
864 'Reynolds'
865
Christian Heimesfe337bf2008-03-23 21:54:12 +0000866 If a group matches multiple times, only the last match is accessible:
867
Christian Heimesb9eccbf2007-12-05 20:18:38 +0000868 >>> m = re.match(r"(..)+", "a1b2c3") # Matches 3 times.
869 >>> m.group(1) # Returns only the last match.
870 'c3'
Georg Brandl116aa622007-08-15 14:28:22 +0000871
872
873.. method:: MatchObject.groups([default])
874
875 Return a tuple containing all the subgroups of the match, from 1 up to however
876 many groups are in the pattern. The *default* argument is used for groups that
Georg Brandl96473892008-03-06 07:09:43 +0000877 did not participate in the match; it defaults to ``None``.
Georg Brandl116aa622007-08-15 14:28:22 +0000878
Christian Heimesfe337bf2008-03-23 21:54:12 +0000879 For example:
Christian Heimesb9eccbf2007-12-05 20:18:38 +0000880
881 >>> m = re.match(r"(\d+)\.(\d+)", "24.1632")
882 >>> m.groups()
883 ('24', '1632')
884
885 If we make the decimal place and everything after it optional, not all groups
886 might participate in the match. These groups will default to ``None`` unless
Christian Heimesfe337bf2008-03-23 21:54:12 +0000887 the *default* argument is given:
Christian Heimesb9eccbf2007-12-05 20:18:38 +0000888
889 >>> m = re.match(r"(\d+)\.?(\d+)?", "24")
Christian Heimesfe337bf2008-03-23 21:54:12 +0000890 >>> m.groups() # Second group defaults to None.
891 ('24', None)
892 >>> m.groups('0') # Now, the second group defaults to '0'.
893 ('24', '0')
Christian Heimesb9eccbf2007-12-05 20:18:38 +0000894
Georg Brandl116aa622007-08-15 14:28:22 +0000895
896.. method:: MatchObject.groupdict([default])
897
898 Return a dictionary containing all the *named* subgroups of the match, keyed by
899 the subgroup name. The *default* argument is used for groups that did not
Christian Heimesfe337bf2008-03-23 21:54:12 +0000900 participate in the match; it defaults to ``None``. For example:
Christian Heimesb9eccbf2007-12-05 20:18:38 +0000901
902 >>> m = re.match(r"(?P<first_name>\w+) (?P<last_name>\w+)", "Malcom Reynolds")
903 >>> m.groupdict()
904 {'first_name': 'Malcom', 'last_name': 'Reynolds'}
Georg Brandl116aa622007-08-15 14:28:22 +0000905
906
907.. method:: MatchObject.start([group])
908 MatchObject.end([group])
909
910 Return the indices of the start and end of the substring matched by *group*;
911 *group* defaults to zero (meaning the whole matched substring). Return ``-1`` if
912 *group* exists but did not contribute to the match. For a match object *m*, and
913 a group *g* that did contribute to the match, the substring matched by group *g*
914 (equivalent to ``m.group(g)``) is ::
915
916 m.string[m.start(g):m.end(g)]
917
918 Note that ``m.start(group)`` will equal ``m.end(group)`` if *group* matched a
919 null string. For example, after ``m = re.search('b(c?)', 'cba')``,
920 ``m.start(0)`` is 1, ``m.end(0)`` is 2, ``m.start(1)`` and ``m.end(1)`` are both
921 2, and ``m.start(2)`` raises an :exc:`IndexError` exception.
922
Christian Heimesfe337bf2008-03-23 21:54:12 +0000923 An example that will remove *remove_this* from email addresses:
Christian Heimesb9eccbf2007-12-05 20:18:38 +0000924
925 >>> email = "tony@tiremove_thisger.net"
926 >>> m = re.search("remove_this", email)
927 >>> email[:m.start()] + email[m.end():]
928 'tony@tiger.net'
929
Georg Brandl116aa622007-08-15 14:28:22 +0000930
931.. method:: MatchObject.span([group])
932
933 For :class:`MatchObject` *m*, return the 2-tuple ``(m.start(group),
934 m.end(group))``. Note that if *group* did not contribute to the match, this is
Christian Heimesb9eccbf2007-12-05 20:18:38 +0000935 ``(-1, -1)``. *group* defaults to zero, the entire match.
Georg Brandl116aa622007-08-15 14:28:22 +0000936
937
938.. attribute:: MatchObject.pos
939
Georg Brandlf346ac02009-07-26 15:03:49 +0000940 The value of *pos* which was passed to the :meth:`~RegexObject.search` or
941 :meth:`~RegexObject.match` method of the :class:`RegexObject`. This is the
942 index into the string at which the RE engine started looking for a match.
Georg Brandl116aa622007-08-15 14:28:22 +0000943
944
945.. attribute:: MatchObject.endpos
946
Georg Brandlf346ac02009-07-26 15:03:49 +0000947 The value of *endpos* which was passed to the :meth:`~RegexObject.search` or
948 :meth:`~RegexObject.match` method of the :class:`RegexObject`. This is the
949 index into the string beyond which the RE engine will not go.
Georg Brandl116aa622007-08-15 14:28:22 +0000950
951
952.. attribute:: MatchObject.lastindex
953
954 The integer index of the last matched capturing group, or ``None`` if no group
955 was matched at all. For example, the expressions ``(a)b``, ``((a)(b))``, and
956 ``((ab))`` will have ``lastindex == 1`` if applied to the string ``'ab'``, while
957 the expression ``(a)(b)`` will have ``lastindex == 2``, if applied to the same
958 string.
959
960
961.. attribute:: MatchObject.lastgroup
962
963 The name of the last matched capturing group, or ``None`` if the group didn't
964 have a name, or if no group was matched at all.
965
966
967.. attribute:: MatchObject.re
968
Georg Brandlf346ac02009-07-26 15:03:49 +0000969 The regular expression object whose :meth:`~RegexObject.match` or
970 :meth:`~RegexObject.search` method produced this :class:`MatchObject`
971 instance.
Georg Brandl116aa622007-08-15 14:28:22 +0000972
973
974.. attribute:: MatchObject.string
975
Georg Brandlf346ac02009-07-26 15:03:49 +0000976 The string passed to :meth:`~RegexObject.match` or
977 :meth:`~RegexObject.search`.
Georg Brandl116aa622007-08-15 14:28:22 +0000978
979
980Examples
981--------
982
Christian Heimesb9eccbf2007-12-05 20:18:38 +0000983
984Checking For a Pair
985^^^^^^^^^^^^^^^^^^^
986
987In this example, we'll use the following helper function to display match
Christian Heimesfe337bf2008-03-23 21:54:12 +0000988objects a little more gracefully:
989
990.. testcode::
Christian Heimesb9eccbf2007-12-05 20:18:38 +0000991
992 def displaymatch(match):
993 if match is None:
994 return None
995 return '<Match: %r, groups=%r>' % (match.group(), match.groups())
996
997Suppose you are writing a poker program where a player's hand is represented as
998a 5-character string with each character representing a card, "a" for ace, "k"
999for king, "q" for queen, j for jack, "0" for 10, and "1" through "9"
1000representing the card with that value.
1001
Christian Heimesfe337bf2008-03-23 21:54:12 +00001002To see if a given string is a valid hand, one could do the following:
Christian Heimesb9eccbf2007-12-05 20:18:38 +00001003
Christian Heimesfe337bf2008-03-23 21:54:12 +00001004 >>> valid = re.compile(r"[0-9akqj]{5}$")
Christian Heimesb9eccbf2007-12-05 20:18:38 +00001005 >>> displaymatch(valid.match("ak05q")) # Valid.
Christian Heimesfe337bf2008-03-23 21:54:12 +00001006 "<Match: 'ak05q', groups=()>"
Christian Heimesb9eccbf2007-12-05 20:18:38 +00001007 >>> displaymatch(valid.match("ak05e")) # Invalid.
1008 >>> displaymatch(valid.match("ak0")) # Invalid.
1009 >>> displaymatch(valid.match("727ak")) # Valid.
Christian Heimesfe337bf2008-03-23 21:54:12 +00001010 "<Match: '727ak', groups=()>"
Christian Heimesb9eccbf2007-12-05 20:18:38 +00001011
1012That last hand, ``"727ak"``, contained a pair, or two of the same valued cards.
Christian Heimesfe337bf2008-03-23 21:54:12 +00001013To match this with a regular expression, one could use backreferences as such:
Christian Heimesb9eccbf2007-12-05 20:18:38 +00001014
1015 >>> pair = re.compile(r".*(.).*\1")
1016 >>> displaymatch(pair.match("717ak")) # Pair of 7s.
Christian Heimesfe337bf2008-03-23 21:54:12 +00001017 "<Match: '717', groups=('7',)>"
Christian Heimesb9eccbf2007-12-05 20:18:38 +00001018 >>> displaymatch(pair.match("718ak")) # No pairs.
1019 >>> displaymatch(pair.match("354aa")) # Pair of aces.
Christian Heimesfe337bf2008-03-23 21:54:12 +00001020 "<Match: '354aa', groups=('a',)>"
Christian Heimesb9eccbf2007-12-05 20:18:38 +00001021
Georg Brandlf346ac02009-07-26 15:03:49 +00001022To find out what card the pair consists of, one could use the
1023:meth:`~MatchObject.group` method of :class:`MatchObject` in the following
1024manner:
Christian Heimesfe337bf2008-03-23 21:54:12 +00001025
1026.. doctest::
Christian Heimesb9eccbf2007-12-05 20:18:38 +00001027
1028 >>> pair.match("717ak").group(1)
1029 '7'
Georg Brandl48310cd2009-01-03 21:18:54 +00001030
Christian Heimesb9eccbf2007-12-05 20:18:38 +00001031 # Error because re.match() returns None, which doesn't have a group() method:
1032 >>> pair.match("718ak").group(1)
1033 Traceback (most recent call last):
1034 File "<pyshell#23>", line 1, in <module>
1035 re.match(r".*(.).*\1", "718ak").group(1)
1036 AttributeError: 'NoneType' object has no attribute 'group'
Georg Brandl48310cd2009-01-03 21:18:54 +00001037
Christian Heimesb9eccbf2007-12-05 20:18:38 +00001038 >>> pair.match("354aa").group(1)
1039 'a'
1040
1041
1042Simulating scanf()
1043^^^^^^^^^^^^^^^^^^
Georg Brandl116aa622007-08-15 14:28:22 +00001044
1045.. index:: single: scanf()
1046
1047Python does not currently have an equivalent to :cfunc:`scanf`. Regular
1048expressions are generally more powerful, though also more verbose, than
1049:cfunc:`scanf` format strings. The table below offers some more-or-less
1050equivalent mappings between :cfunc:`scanf` format tokens and regular
1051expressions.
1052
1053+--------------------------------+---------------------------------------------+
1054| :cfunc:`scanf` Token | Regular Expression |
1055+================================+=============================================+
1056| ``%c`` | ``.`` |
1057+--------------------------------+---------------------------------------------+
1058| ``%5c`` | ``.{5}`` |
1059+--------------------------------+---------------------------------------------+
1060| ``%d`` | ``[-+]?\d+`` |
1061+--------------------------------+---------------------------------------------+
1062| ``%e``, ``%E``, ``%f``, ``%g`` | ``[-+]?(\d+(\.\d*)?|\.\d+)([eE][-+]?\d+)?`` |
1063+--------------------------------+---------------------------------------------+
1064| ``%i`` | ``[-+]?(0[xX][\dA-Fa-f]+|0[0-7]*|\d+)`` |
1065+--------------------------------+---------------------------------------------+
1066| ``%o`` | ``0[0-7]*`` |
1067+--------------------------------+---------------------------------------------+
1068| ``%s`` | ``\S+`` |
1069+--------------------------------+---------------------------------------------+
1070| ``%u`` | ``\d+`` |
1071+--------------------------------+---------------------------------------------+
1072| ``%x``, ``%X`` | ``0[xX][\dA-Fa-f]+`` |
1073+--------------------------------+---------------------------------------------+
1074
1075To extract the filename and numbers from a string like ::
1076
1077 /usr/sbin/sendmail - 0 errors, 4 warnings
1078
1079you would use a :cfunc:`scanf` format like ::
1080
1081 %s - %d errors, %d warnings
1082
1083The equivalent regular expression would be ::
1084
1085 (\S+) - (\d+) errors, (\d+) warnings
1086
Christian Heimesb9eccbf2007-12-05 20:18:38 +00001087
1088Avoiding recursion
1089^^^^^^^^^^^^^^^^^^
Georg Brandl116aa622007-08-15 14:28:22 +00001090
1091If you create regular expressions that require the engine to perform a lot of
1092recursion, you may encounter a :exc:`RuntimeError` exception with the message
1093``maximum recursion limit`` exceeded. For example, ::
1094
Georg Brandl116aa622007-08-15 14:28:22 +00001095 >>> s = 'Begin ' + 1000*'a very long string ' + 'end'
1096 >>> re.match('Begin (\w| )*? end', s).end()
1097 Traceback (most recent call last):
1098 File "<stdin>", line 1, in ?
1099 File "/usr/local/lib/python2.5/re.py", line 132, in match
1100 return _compile(pattern, flags).match(string)
1101 RuntimeError: maximum recursion limit exceeded
1102
1103You can often restructure your regular expression to avoid recursion.
1104
Georg Brandle6bcc912008-05-12 18:05:20 +00001105Simple uses of the ``*?`` pattern are special-cased to avoid recursion. Thus,
1106the above regular expression can avoid recursion by being recast as ``Begin
1107[a-zA-Z0-9_ ]*?end``. As a further benefit, such regular expressions will run
1108faster than their recursive equivalents.
Georg Brandl116aa622007-08-15 14:28:22 +00001109
Christian Heimesb9eccbf2007-12-05 20:18:38 +00001110
1111search() vs. match()
1112^^^^^^^^^^^^^^^^^^^^
1113
1114In a nutshell, :func:`match` only attempts to match a pattern at the beginning
1115of a string where :func:`search` will match a pattern anywhere in a string.
Christian Heimesfe337bf2008-03-23 21:54:12 +00001116For example:
Christian Heimesb9eccbf2007-12-05 20:18:38 +00001117
1118 >>> re.match("o", "dog") # No match as "o" is not the first letter of "dog".
1119 >>> re.search("o", "dog") # Match as search() looks everywhere in the string.
Christian Heimesfe337bf2008-03-23 21:54:12 +00001120 <_sre.SRE_Match object at ...>
Christian Heimesb9eccbf2007-12-05 20:18:38 +00001121
1122.. note::
1123
Christian Heimesfe337bf2008-03-23 21:54:12 +00001124 The following applies only to regular expression objects like those created
1125 with ``re.compile("pattern")``, not the primitives ``re.match(pattern,
1126 string)`` or ``re.search(pattern, string)``.
Christian Heimesb9eccbf2007-12-05 20:18:38 +00001127
1128:func:`match` has an optional second parameter that gives an index in the string
Benjamin Petersonf07d0022009-03-21 17:31:58 +00001129where the search is to start::
Christian Heimesb9eccbf2007-12-05 20:18:38 +00001130
1131 >>> pattern = re.compile("o")
1132 >>> pattern.match("dog") # No match as "o" is not at the start of "dog."
Christian Heimesfe337bf2008-03-23 21:54:12 +00001133
Christian Heimesb9eccbf2007-12-05 20:18:38 +00001134 # Equivalent to the above expression as 0 is the default starting index:
1135 >>> pattern.match("dog", 0)
Christian Heimesfe337bf2008-03-23 21:54:12 +00001136
Christian Heimesb9eccbf2007-12-05 20:18:38 +00001137 # Match as "o" is the 2nd character of "dog" (index 0 is the first):
1138 >>> pattern.match("dog", 1)
Christian Heimesfe337bf2008-03-23 21:54:12 +00001139 <_sre.SRE_Match object at ...>
Christian Heimesb9eccbf2007-12-05 20:18:38 +00001140 >>> pattern.match("dog", 2) # No match as "o" is not the 3rd character of "dog."
1141
1142
1143Making a Phonebook
1144^^^^^^^^^^^^^^^^^^
1145
Georg Brandl48310cd2009-01-03 21:18:54 +00001146:func:`split` splits a string into a list delimited by the passed pattern. The
Christian Heimesb9eccbf2007-12-05 20:18:38 +00001147method is invaluable for converting textual data into data structures that can be
1148easily read and modified by Python as demonstrated in the following example that
1149creates a phonebook.
1150
Christian Heimes255f53b2007-12-08 15:33:56 +00001151First, here is the input. Normally it may come from a file, here we are using
Christian Heimesfe337bf2008-03-23 21:54:12 +00001152triple-quoted string syntax:
Christian Heimesb9eccbf2007-12-05 20:18:38 +00001153
Christian Heimes255f53b2007-12-08 15:33:56 +00001154 >>> input = """Ross McFluff: 834.345.1254 155 Elm Street
Georg Brandl48310cd2009-01-03 21:18:54 +00001155 ...
Christian Heimesfe337bf2008-03-23 21:54:12 +00001156 ... Ronald Heathmore: 892.345.3428 436 Finley Avenue
1157 ... Frank Burger: 925.541.7625 662 South Dogwood Way
1158 ...
1159 ...
1160 ... Heather Albrecht: 548.326.4584 919 Park Place"""
Christian Heimes255f53b2007-12-08 15:33:56 +00001161
1162The entries are separated by one or more newlines. Now we convert the string
Christian Heimesfe337bf2008-03-23 21:54:12 +00001163into a list with each nonempty line having its own entry:
1164
1165.. doctest::
1166 :options: +NORMALIZE_WHITESPACE
Christian Heimes255f53b2007-12-08 15:33:56 +00001167
1168 >>> entries = re.split("\n+", input)
Christian Heimesb9eccbf2007-12-05 20:18:38 +00001169 >>> entries
Christian Heimesfe337bf2008-03-23 21:54:12 +00001170 ['Ross McFluff: 834.345.1254 155 Elm Street',
1171 'Ronald Heathmore: 892.345.3428 436 Finley Avenue',
1172 'Frank Burger: 925.541.7625 662 South Dogwood Way',
1173 'Heather Albrecht: 548.326.4584 919 Park Place']
Christian Heimesb9eccbf2007-12-05 20:18:38 +00001174
1175Finally, split each entry into a list with first name, last name, telephone
Christian Heimesc3f30c42008-02-22 16:37:40 +00001176number, and address. We use the ``maxsplit`` parameter of :func:`split`
Christian Heimesfe337bf2008-03-23 21:54:12 +00001177because the address has spaces, our splitting pattern, in it:
1178
1179.. doctest::
1180 :options: +NORMALIZE_WHITESPACE
Christian Heimesb9eccbf2007-12-05 20:18:38 +00001181
Christian Heimes255f53b2007-12-08 15:33:56 +00001182 >>> [re.split(":? ", entry, 3) for entry in entries]
Christian Heimesb9eccbf2007-12-05 20:18:38 +00001183 [['Ross', 'McFluff', '834.345.1254', '155 Elm Street'],
1184 ['Ronald', 'Heathmore', '892.345.3428', '436 Finley Avenue'],
1185 ['Frank', 'Burger', '925.541.7625', '662 South Dogwood Way'],
1186 ['Heather', 'Albrecht', '548.326.4584', '919 Park Place']]
1187
Christian Heimes255f53b2007-12-08 15:33:56 +00001188The ``:?`` pattern matches the colon after the last name, so that it does not
Christian Heimesc3f30c42008-02-22 16:37:40 +00001189occur in the result list. With a ``maxsplit`` of ``4``, we could separate the
Christian Heimesfe337bf2008-03-23 21:54:12 +00001190house number from the street name:
1191
1192.. doctest::
1193 :options: +NORMALIZE_WHITESPACE
Christian Heimesb9eccbf2007-12-05 20:18:38 +00001194
Christian Heimes255f53b2007-12-08 15:33:56 +00001195 >>> [re.split(":? ", entry, 4) for entry in entries]
Christian Heimesb9eccbf2007-12-05 20:18:38 +00001196 [['Ross', 'McFluff', '834.345.1254', '155', 'Elm Street'],
1197 ['Ronald', 'Heathmore', '892.345.3428', '436', 'Finley Avenue'],
1198 ['Frank', 'Burger', '925.541.7625', '662', 'South Dogwood Way'],
1199 ['Heather', 'Albrecht', '548.326.4584', '919', 'Park Place']]
1200
1201
1202Text Munging
1203^^^^^^^^^^^^
1204
1205:func:`sub` replaces every occurrence of a pattern with a string or the
1206result of a function. This example demonstrates using :func:`sub` with
1207a function to "munge" text, or randomize the order of all the characters
1208in each word of a sentence except for the first and last characters::
1209
1210 >>> def repl(m):
1211 ... inner_word = list(m.group(2))
1212 ... random.shuffle(inner_word)
1213 ... return m.group(1) + "".join(inner_word) + m.group(3)
1214 >>> text = "Professor Abdolmalek, please report your absences promptly."
1215 >>> re.sub("(\w)(\w+)(\w)", repl, text)
1216 'Poefsrosr Aealmlobdk, pslaee reorpt your abnseces plmrptoy.'
1217 >>> re.sub("(\w)(\w+)(\w)", repl, text)
1218 'Pofsroser Aodlambelk, plasee reoprt yuor asnebces potlmrpy.'
1219
1220
1221Finding all Adverbs
1222^^^^^^^^^^^^^^^^^^^
1223
Christian Heimesc3f30c42008-02-22 16:37:40 +00001224:func:`findall` matches *all* occurrences of a pattern, not just the first
Christian Heimesb9eccbf2007-12-05 20:18:38 +00001225one as :func:`search` does. For example, if one was a writer and wanted to
1226find all of the adverbs in some text, he or she might use :func:`findall` in
Christian Heimesfe337bf2008-03-23 21:54:12 +00001227the following manner:
Christian Heimesb9eccbf2007-12-05 20:18:38 +00001228
1229 >>> text = "He was carefully disguised but captured quickly by police."
1230 >>> re.findall(r"\w+ly", text)
1231 ['carefully', 'quickly']
1232
1233
1234Finding all Adverbs and their Positions
1235^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1236
1237If one wants more information about all matches of a pattern than the matched
1238text, :func:`finditer` is useful as it provides instances of
1239:class:`MatchObject` instead of strings. Continuing with the previous example,
1240if one was a writer who wanted to find all of the adverbs *and their positions*
Christian Heimesfe337bf2008-03-23 21:54:12 +00001241in some text, he or she would use :func:`finditer` in the following manner:
Christian Heimesb9eccbf2007-12-05 20:18:38 +00001242
1243 >>> text = "He was carefully disguised but captured quickly by police."
1244 >>> for m in re.finditer(r"\w+ly", text):
Christian Heimesfe337bf2008-03-23 21:54:12 +00001245 ... print('%02d-%02d: %s' % (m.start(), m.end(), m.group(0)))
Christian Heimesb9eccbf2007-12-05 20:18:38 +00001246 07-16: carefully
1247 40-47: quickly
1248
1249
1250Raw String Notation
1251^^^^^^^^^^^^^^^^^^^
1252
1253Raw string notation (``r"text"``) keeps regular expressions sane. Without it,
1254every backslash (``'\'``) in a regular expression would have to be prefixed with
1255another one to escape it. For example, the two following lines of code are
Christian Heimesfe337bf2008-03-23 21:54:12 +00001256functionally identical:
Christian Heimesb9eccbf2007-12-05 20:18:38 +00001257
1258 >>> re.match(r"\W(.)\1\W", " ff ")
Christian Heimesfe337bf2008-03-23 21:54:12 +00001259 <_sre.SRE_Match object at ...>
Christian Heimesb9eccbf2007-12-05 20:18:38 +00001260 >>> re.match("\\W(.)\\1\\W", " ff ")
Christian Heimesfe337bf2008-03-23 21:54:12 +00001261 <_sre.SRE_Match object at ...>
Christian Heimesb9eccbf2007-12-05 20:18:38 +00001262
1263When one wants to match a literal backslash, it must be escaped in the regular
1264expression. With raw string notation, this means ``r"\\"``. Without raw string
1265notation, one must use ``"\\\\"``, making the following lines of code
Christian Heimesfe337bf2008-03-23 21:54:12 +00001266functionally identical:
Christian Heimesb9eccbf2007-12-05 20:18:38 +00001267
1268 >>> re.match(r"\\", r"\\")
Christian Heimesfe337bf2008-03-23 21:54:12 +00001269 <_sre.SRE_Match object at ...>
Christian Heimesb9eccbf2007-12-05 20:18:38 +00001270 >>> re.match("\\\\", r"\\")
Christian Heimesfe337bf2008-03-23 21:54:12 +00001271 <_sre.SRE_Match object at ...>