| Elliott Hughes | 5b80804 | 2021-10-01 10:56:10 -0700 | [diff] [blame^] | 1 | <html> |
| 2 | <head> |
| 3 | <title>pcre2 specification</title> |
| 4 | </head> |
| 5 | <body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB"> |
| 6 | <h1>pcre2 man page</h1> |
| 7 | <p> |
| 8 | Return to the <a href="index.html">PCRE2 index page</a>. |
| 9 | </p> |
| 10 | <p> |
| 11 | This page is part of the PCRE2 HTML documentation. It was generated |
| 12 | automatically from the original man page. If there is any nonsense in it, |
| 13 | please consult the man page, in case the conversion went wrong. |
| 14 | <br> |
| 15 | <ul> |
| 16 | <li><a name="TOC1" href="#SEC1">INTRODUCTION</a> |
| 17 | <li><a name="TOC2" href="#SEC2">SECURITY CONSIDERATIONS</a> |
| 18 | <li><a name="TOC3" href="#SEC3">USER DOCUMENTATION</a> |
| 19 | <li><a name="TOC4" href="#SEC4">AUTHOR</a> |
| 20 | <li><a name="TOC5" href="#SEC5">REVISION</a> |
| 21 | </ul> |
| 22 | <br><a name="SEC1" href="#TOC1">INTRODUCTION</a><br> |
| 23 | <P> |
| 24 | PCRE2 is the name used for a revised API for the PCRE library, which is a set |
| 25 | of functions, written in C, that implement regular expression pattern matching |
| 26 | using the same syntax and semantics as Perl, with just a few differences. After |
| 27 | nearly two decades, the limitations of the original API were making development |
| 28 | increasingly difficult. The new API is more extensible, and it was simplified |
| 29 | by abolishing the separate "study" optimizing function; in PCRE2, patterns are |
| 30 | automatically optimized where possible. Since forking from PCRE1, the code has |
| 31 | been extensively refactored and new features introduced. The old library is now |
| 32 | obsolete and is no longer maintained. |
| 33 | </P> |
| 34 | <P> |
| 35 | As well as Perl-style regular expression patterns, some features that appeared |
| 36 | in Python and the original PCRE before they appeared in Perl are available |
| 37 | using the Python syntax. There is also some support for one or two .NET and |
| 38 | Oniguruma syntax items, and there are options for requesting some minor changes |
| 39 | that give better ECMAScript (aka JavaScript) compatibility. |
| 40 | </P> |
| 41 | <P> |
| 42 | The source code for PCRE2 can be compiled to support strings of 8-bit, 16-bit, |
| 43 | or 32-bit code units, which means that up to three separate libraries may be |
| 44 | installed, one for each code unit size. The size of code unit is not related to |
| 45 | the bit size of the underlying hardware. In a 64-bit environment that also |
| 46 | supports 32-bit applications, versions of PCRE2 that are compiled in both |
| 47 | 64-bit and 32-bit modes may be needed. |
| 48 | </P> |
| 49 | <P> |
| 50 | The original work to extend PCRE to 16-bit and 32-bit code units was done by |
| 51 | Zoltan Herczeg and Christian Persch, respectively. In all three cases, strings |
| 52 | can be interpreted either as one character per code unit, or as UTF-encoded |
| 53 | Unicode, with support for Unicode general category properties. Unicode support |
| 54 | is optional at build time (but is the default). However, processing strings as |
| 55 | UTF code units must be enabled explicitly at run time. The version of Unicode |
| 56 | in use can be discovered by running |
| 57 | <pre> |
| 58 | pcre2test -C |
| 59 | </PRE> |
| 60 | </P> |
| 61 | <P> |
| 62 | The three libraries contain identical sets of functions, with names ending in |
| 63 | _8, _16, or _32, respectively (for example, <b>pcre2_compile_8()</b>). However, |
| 64 | by defining PCRE2_CODE_UNIT_WIDTH to be 8, 16, or 32, a program that uses just |
| 65 | one code unit width can be written using generic names such as |
| 66 | <b>pcre2_compile()</b>, and the documentation is written assuming that this is |
| 67 | the case. |
| 68 | </P> |
| 69 | <P> |
| 70 | In addition to the Perl-compatible matching function, PCRE2 contains an |
| 71 | alternative function that matches the same compiled patterns in a different |
| 72 | way. In certain circumstances, the alternative function has some advantages. |
| 73 | For a discussion of the two matching algorithms, see the |
| 74 | <a href="pcre2matching.html"><b>pcre2matching</b></a> |
| 75 | page. |
| 76 | </P> |
| 77 | <P> |
| 78 | Details of exactly which Perl regular expression features are and are not |
| 79 | supported by PCRE2 are given in separate documents. See the |
| 80 | <a href="pcre2pattern.html"><b>pcre2pattern</b></a> |
| 81 | and |
| 82 | <a href="pcre2compat.html"><b>pcre2compat</b></a> |
| 83 | pages. There is a syntax summary in the |
| 84 | <a href="pcre2syntax.html"><b>pcre2syntax</b></a> |
| 85 | page. |
| 86 | </P> |
| 87 | <P> |
| 88 | Some features of PCRE2 can be included, excluded, or changed when the library |
| 89 | is built. The |
| 90 | <a href="pcre2_config.html"><b>pcre2_config()</b></a> |
| 91 | function makes it possible for a client to discover which features are |
| 92 | available. The features themselves are described in the |
| 93 | <a href="pcre2build.html"><b>pcre2build</b></a> |
| 94 | page. Documentation about building PCRE2 for various operating systems can be |
| 95 | found in the |
| 96 | <a href="README.txt"><b>README</b></a> |
| 97 | and |
| 98 | <a href="NON-AUTOTOOLS-BUILD.txt"><b>NON-AUTOTOOLS_BUILD</b></a> |
| 99 | files in the source distribution. |
| 100 | </P> |
| 101 | <P> |
| 102 | The libraries contains a number of undocumented internal functions and data |
| 103 | tables that are used by more than one of the exported external functions, but |
| 104 | which are not intended for use by external callers. Their names all begin with |
| 105 | "_pcre2", which hopefully will not provoke any name clashes. In some |
| 106 | environments, it is possible to control which external symbols are exported |
| 107 | when a shared library is built, and in these cases the undocumented symbols are |
| 108 | not exported. |
| 109 | </P> |
| 110 | <br><a name="SEC2" href="#TOC1">SECURITY CONSIDERATIONS</a><br> |
| 111 | <P> |
| 112 | If you are using PCRE2 in a non-UTF application that permits users to supply |
| 113 | arbitrary patterns for compilation, you should be aware of a feature that |
| 114 | allows users to turn on UTF support from within a pattern. For example, an |
| 115 | 8-bit pattern that begins with "(*UTF)" turns on UTF-8 mode, which interprets |
| 116 | patterns and subjects as strings of UTF-8 code units instead of individual |
| 117 | 8-bit characters. This causes both the pattern and any data against which it is |
| 118 | matched to be checked for UTF-8 validity. If the data string is very long, such |
| 119 | a check might use sufficiently many resources as to cause your application to |
| 120 | lose performance. |
| 121 | </P> |
| 122 | <P> |
| 123 | One way of guarding against this possibility is to use the |
| 124 | <b>pcre2_pattern_info()</b> function to check the compiled pattern's options for |
| 125 | PCRE2_UTF. Alternatively, you can set the PCRE2_NEVER_UTF option when calling |
| 126 | <b>pcre2_compile()</b>. This causes a compile time error if the pattern contains |
| 127 | a UTF-setting sequence. |
| 128 | </P> |
| 129 | <P> |
| 130 | The use of Unicode properties for character types such as \d can also be |
| 131 | enabled from within the pattern, by specifying "(*UCP)". This feature can be |
| 132 | disallowed by setting the PCRE2_NEVER_UCP option. |
| 133 | </P> |
| 134 | <P> |
| 135 | If your application is one that supports UTF, be aware that validity checking |
| 136 | can take time. If the same data string is to be matched many times, you can use |
| 137 | the PCRE2_NO_UTF_CHECK option for the second and subsequent matches to avoid |
| 138 | running redundant checks. |
| 139 | </P> |
| 140 | <P> |
| 141 | The use of the \C escape sequence in a UTF-8 or UTF-16 pattern can lead to |
| 142 | problems, because it may leave the current matching point in the middle of a |
| 143 | multi-code-unit character. The PCRE2_NEVER_BACKSLASH_C option can be used by an |
| 144 | application to lock out the use of \C, causing a compile-time error if it is |
| 145 | encountered. It is also possible to build PCRE2 with the use of \C permanently |
| 146 | disabled. |
| 147 | </P> |
| 148 | <P> |
| 149 | Another way that performance can be hit is by running a pattern that has a very |
| 150 | large search tree against a string that will never match. Nested unlimited |
| 151 | repeats in a pattern are a common example. PCRE2 provides some protection |
| 152 | against this: see the <b>pcre2_set_match_limit()</b> function in the |
| 153 | <a href="pcre2api.html"><b>pcre2api</b></a> |
| 154 | page. There is a similar function called <b>pcre2_set_depth_limit()</b> that can |
| 155 | be used to restrict the amount of memory that is used. |
| 156 | </P> |
| 157 | <br><a name="SEC3" href="#TOC1">USER DOCUMENTATION</a><br> |
| 158 | <P> |
| 159 | The user documentation for PCRE2 comprises a number of different sections. In |
| 160 | the "man" format, each of these is a separate "man page". In the HTML format, |
| 161 | each is a separate page, linked from the index page. In the plain text format, |
| 162 | the descriptions of the <b>pcre2grep</b> and <b>pcre2test</b> programs are in |
| 163 | files called <b>pcre2grep.txt</b> and <b>pcre2test.txt</b>, respectively. The |
| 164 | remaining sections, except for the <b>pcre2demo</b> section (which is a program |
| 165 | listing), and the short pages for individual functions, are concatenated in |
| 166 | <b>pcre2.txt</b>, for ease of searching. The sections are as follows: |
| 167 | <pre> |
| 168 | pcre2 this document |
| 169 | pcre2-config show PCRE2 installation configuration information |
| 170 | pcre2api details of PCRE2's native C API |
| 171 | pcre2build building PCRE2 |
| 172 | pcre2callout details of the pattern callout feature |
| 173 | pcre2compat discussion of Perl compatibility |
| 174 | pcre2convert details of pattern conversion functions |
| 175 | pcre2demo a demonstration C program that uses PCRE2 |
| 176 | pcre2grep description of the <b>pcre2grep</b> command (8-bit only) |
| 177 | pcre2jit discussion of just-in-time optimization support |
| 178 | pcre2limits details of size and other limits |
| 179 | pcre2matching discussion of the two matching algorithms |
| 180 | pcre2partial details of the partial matching facility |
| 181 | pcre2pattern syntax and semantics of supported regular expression patterns |
| 182 | pcre2perform discussion of performance issues |
| 183 | pcre2posix the POSIX-compatible C API for the 8-bit library |
| 184 | pcre2sample discussion of the pcre2demo program |
| 185 | pcre2serialize details of pattern serialization |
| 186 | pcre2syntax quick syntax reference |
| 187 | pcre2test description of the <b>pcre2test</b> command |
| 188 | pcre2unicode discussion of Unicode and UTF support |
| 189 | </pre> |
| 190 | In the "man" and HTML formats, there is also a short page for each C library |
| 191 | function, listing its arguments and results. |
| 192 | </P> |
| 193 | <br><a name="SEC4" href="#TOC1">AUTHOR</a><br> |
| 194 | <P> |
| 195 | Philip Hazel |
| 196 | <br> |
| 197 | Retired from University Computing Service |
| 198 | <br> |
| 199 | Cambridge, England. |
| 200 | <br> |
| 201 | </P> |
| 202 | <P> |
| 203 | Putting an actual email address here is a spam magnet. If you want to email me, |
| 204 | use my two names separated by a dot at gmail.com. |
| 205 | </P> |
| 206 | <br><a name="SEC5" href="#TOC1">REVISION</a><br> |
| 207 | <P> |
| 208 | Last updated: 27 August 2021 |
| 209 | <br> |
| 210 | Copyright © 1997-2021 University of Cambridge. |
| 211 | <br> |
| 212 | <p> |
| 213 | Return to the <a href="index.html">PCRE2 index page</a>. |
| 214 | </p> |