Janis Danisevskis | 112c9cc | 2016-03-31 13:35:25 +0100 | [diff] [blame] | 1 | Building PCRE2 without using autotools |
| 2 | -------------------------------------- |
| 3 | |
Janis Danisevskis | 112c9cc | 2016-03-31 13:35:25 +0100 | [diff] [blame] | 4 | This document contains the following sections: |
| 5 | |
| 6 | General |
| 7 | Generic instructions for the PCRE2 C library |
| 8 | Stack size in Windows environments |
| 9 | Linking programs in Windows environments |
| 10 | Calling conventions in Windows environments |
| 11 | Comments about Win32 builds |
| 12 | Building PCRE2 on Windows with CMake |
| 13 | Testing with RunTest.bat |
| 14 | Building PCRE2 on native z/OS and z/VM |
| 15 | |
| 16 | |
| 17 | GENERAL |
| 18 | |
| 19 | The basic PCRE2 library consists entirely of code written in Standard C, and so |
| 20 | should compile successfully on any system that has a Standard C compiler and |
| 21 | library. |
| 22 | |
| 23 | The PCRE2 distribution includes a "configure" file for use by the |
| 24 | configure/make (autotools) build system, as found in many Unix-like |
| 25 | environments. The README file contains information about the options for |
| 26 | "configure". |
| 27 | |
| 28 | There is also support for CMake, which some users prefer, especially in Windows |
| 29 | environments, though it can also be run in Unix-like environments. See the |
| 30 | section entitled "Building PCRE2 on Windows with CMake" below. |
| 31 | |
| 32 | Versions of src/config.h and src/pcre2.h are distributed in the PCRE2 tarballs |
| 33 | under the names src/config.h.generic and src/pcre2.h.generic. These are |
| 34 | provided for those who build PCRE2 without using "configure" or CMake. If you |
| 35 | use "configure" or CMake, the .generic versions are not used. |
| 36 | |
| 37 | |
| 38 | GENERIC INSTRUCTIONS FOR THE PCRE2 C LIBRARY |
| 39 | |
| 40 | The following are generic instructions for building the PCRE2 C library "by |
| 41 | hand". If you are going to use CMake, this section does not apply to you; you |
| 42 | can skip ahead to the CMake section. |
| 43 | |
| 44 | (1) Copy or rename the file src/config.h.generic as src/config.h, and edit the |
| 45 | macro settings that it contains to whatever is appropriate for your |
| 46 | environment. In particular, you can alter the definition of the NEWLINE |
| 47 | macro to specify what character(s) you want to be interpreted as line |
Elliott Hughes | 9bc971b | 2018-07-27 13:23:14 -0700 | [diff] [blame^] | 48 | terminators by default. |
Janis Danisevskis | 112c9cc | 2016-03-31 13:35:25 +0100 | [diff] [blame] | 49 | |
| 50 | When you compile any of the PCRE2 modules, you must specify |
| 51 | -DHAVE_CONFIG_H to your compiler so that src/config.h is included in the |
| 52 | sources. |
| 53 | |
| 54 | An alternative approach is not to edit src/config.h, but to use -D on the |
| 55 | compiler command line to make any changes that you need to the |
| 56 | configuration options. In this case -DHAVE_CONFIG_H must not be set. |
| 57 | |
| 58 | NOTE: There have been occasions when the way in which certain parameters |
| 59 | in src/config.h are used has changed between releases. (In the |
| 60 | configure/make world, this is handled automatically.) When upgrading to a |
| 61 | new release, you are strongly advised to review src/config.h.generic |
| 62 | before re-using what you had previously. |
| 63 | |
| 64 | (2) Copy or rename the file src/pcre2.h.generic as src/pcre2.h. |
| 65 | |
| 66 | (3) EITHER: |
| 67 | Copy or rename file src/pcre2_chartables.c.dist as |
| 68 | src/pcre2_chartables.c. |
| 69 | |
| 70 | OR: |
| 71 | Compile src/dftables.c as a stand-alone program (using -DHAVE_CONFIG_H |
| 72 | if you have set up src/config.h), and then run it with the single |
| 73 | argument "src/pcre2_chartables.c". This generates a set of standard |
| 74 | character tables and writes them to that file. The tables are generated |
| 75 | using the default C locale for your system. If you want to use a locale |
| 76 | that is specified by LC_xxx environment variables, add the -L option to |
| 77 | the dftables command. You must use this method if you are building on a |
| 78 | system that uses EBCDIC code. |
| 79 | |
| 80 | The tables in src/pcre2_chartables.c are defaults. The caller of PCRE2 can |
| 81 | specify alternative tables at run time. |
| 82 | |
| 83 | (4) For an 8-bit library, compile the following source files from the src |
| 84 | directory, setting -DPCRE2_CODE_UNIT_WIDTH=8 as a compiler option. Also |
| 85 | set -DHAVE_CONFIG_H if you have set up src/config.h with your |
| 86 | configuration, or else use other -D settings to change the configuration |
| 87 | as required. |
| 88 | |
| 89 | pcre2_auto_possess.c |
| 90 | pcre2_chartables.c |
| 91 | pcre2_compile.c |
| 92 | pcre2_config.c |
| 93 | pcre2_context.c |
Elliott Hughes | 9bc971b | 2018-07-27 13:23:14 -0700 | [diff] [blame^] | 94 | pcre2_convert.c |
Janis Danisevskis | 112c9cc | 2016-03-31 13:35:25 +0100 | [diff] [blame] | 95 | pcre2_dfa_match.c |
| 96 | pcre2_error.c |
Elliott Hughes | 9bc971b | 2018-07-27 13:23:14 -0700 | [diff] [blame^] | 97 | pcre2_extuni.c |
Janis Danisevskis | 112c9cc | 2016-03-31 13:35:25 +0100 | [diff] [blame] | 98 | pcre2_find_bracket.c |
| 99 | pcre2_jit_compile.c |
| 100 | pcre2_maketables.c |
| 101 | pcre2_match.c |
| 102 | pcre2_match_data.c |
| 103 | pcre2_newline.c |
| 104 | pcre2_ord2utf.c |
| 105 | pcre2_pattern_info.c |
| 106 | pcre2_serialize.c |
| 107 | pcre2_string_utils.c |
| 108 | pcre2_study.c |
| 109 | pcre2_substitute.c |
| 110 | pcre2_substring.c |
| 111 | pcre2_tables.c |
| 112 | pcre2_ucd.c |
| 113 | pcre2_valid_utf.c |
| 114 | pcre2_xclass.c |
| 115 | |
| 116 | Make sure that you include -I. in the compiler command (or equivalent for |
| 117 | an unusual compiler) so that all included PCRE2 header files are first |
| 118 | sought in the src directory under the current directory. Otherwise you run |
| 119 | the risk of picking up a previously-installed file from somewhere else. |
| 120 | |
| 121 | Note that you must compile pcre2_jit_compile.c, even if you have not |
| 122 | defined SUPPORT_JIT in src/config.h, because when JIT support is not |
| 123 | configured, dummy functions are compiled. When JIT support IS configured, |
Elliott Hughes | 9bc971b | 2018-07-27 13:23:14 -0700 | [diff] [blame^] | 124 | pcre2_jit_compile.c #includes other files from the sljit subdirectory, |
| 125 | all of whose names begin with "sljit". It also #includes |
| 126 | src/pcre2_jit_match.c and src/pcre2_jit_misc.c, so you should not compile |
| 127 | these yourself. |
| 128 | |
| 129 | Not also that the pcre2_fuzzsupport.c file contains special code that is |
| 130 | useful to those who want to run fuzzing tests on the PCRE2 library. Unless |
| 131 | you are doing that, you can ignore it. |
Janis Danisevskis | 112c9cc | 2016-03-31 13:35:25 +0100 | [diff] [blame] | 132 | |
| 133 | (5) Now link all the compiled code into an object library in whichever form |
| 134 | your system keeps such libraries. This is the basic PCRE2 C 8-bit library. |
| 135 | If your system has static and shared libraries, you may have to do this |
| 136 | once for each type. |
| 137 | |
| 138 | (6) If you want to build a 16-bit library or 32-bit library (as well as, or |
| 139 | instead of the 8-bit library) just supply 16 or 32 as the value of |
| 140 | -DPCRE2_CODE_UNIT_WIDTH when you are compiling. |
| 141 | |
| 142 | (7) If you want to build the POSIX wrapper functions (which apply only to the |
| 143 | 8-bit library), ensure that you have the src/pcre2posix.h file and then |
| 144 | compile src/pcre2posix.c. Link the result (on its own) as the pcre2posix |
| 145 | library. |
| 146 | |
| 147 | (8) The pcre2test program can be linked with any combination of the 8-bit, |
| 148 | 16-bit and 32-bit libraries (depending on what you selected in |
| 149 | src/config.h). Compile src/pcre2test.c; don't forget -DHAVE_CONFIG_H if |
| 150 | necessary, but do NOT define PCRE2_CODE_UNIT_WIDTH. Then link with the |
| 151 | appropriate library/ies. If you compiled an 8-bit library, pcre2test also |
| 152 | needs the pcre2posix wrapper library. |
| 153 | |
| 154 | (9) Run pcre2test on the testinput files in the testdata directory, and check |
| 155 | that the output matches the corresponding testoutput files. There are |
| 156 | comments about what each test does in the section entitled "Testing PCRE2" |
| 157 | in the README file. If you compiled more than one of the 8-bit, 16-bit and |
| 158 | 32-bit libraries, you need to run pcre2test with the -16 option to do |
| 159 | 16-bit tests and with the -32 option to do 32-bit tests. |
| 160 | |
| 161 | Some tests are relevant only when certain build-time options are selected. |
| 162 | For example, test 4 is for Unicode support, and will not run if you have |
| 163 | built PCRE2 without it. See the comments at the start of each testinput |
| 164 | file. If you have a suitable Unix-like shell, the RunTest script will run |
| 165 | the appropriate tests for you. The command "RunTest list" will output a |
| 166 | list of all the tests. |
| 167 | |
| 168 | Note that the supplied files are in Unix format, with just LF characters |
| 169 | as line terminators. You may need to edit them to change this if your |
| 170 | system uses a different convention. |
| 171 | |
| 172 | (10) If you have built PCRE2 with SUPPORT_JIT, the JIT features can be tested |
| 173 | by running pcre2test with the -jit option. This is done automatically by |
| 174 | the RunTest script. You might also like to build and run the freestanding |
| 175 | JIT test program, src/pcre2_jit_test.c. |
| 176 | |
| 177 | (11) If you want to use the pcre2grep command, compile and link |
| 178 | src/pcre2grep.c; it uses only the basic 8-bit PCRE2 library (it does not |
Elliott Hughes | 9bc971b | 2018-07-27 13:23:14 -0700 | [diff] [blame^] | 179 | need the pcre2posix library). If you have built the PCRE2 library with JIT |
| 180 | support by defining SUPPORT_JIT in src/config.h, you can also define |
| 181 | SUPPORT_PCRE2GREP_JIT, which causes pcre2grep to make use of JIT (unless |
| 182 | it is run with --no-jit). If you define SUPPORT_PCRE2GREP_JIT without |
| 183 | defining SUPPORT_JIT, pcre2grep does not try to make use of JIT. |
Janis Danisevskis | 112c9cc | 2016-03-31 13:35:25 +0100 | [diff] [blame] | 184 | |
| 185 | |
| 186 | STACK SIZE IN WINDOWS ENVIRONMENTS |
| 187 | |
Elliott Hughes | 9bc971b | 2018-07-27 13:23:14 -0700 | [diff] [blame^] | 188 | Prior to release 10.30 the default system stack size of 1Mb in some Windows |
| 189 | environments caused issues with some tests. This should no longer be the case |
| 190 | for 10.30 and later releases. |
Janis Danisevskis | 112c9cc | 2016-03-31 13:35:25 +0100 | [diff] [blame] | 191 | |
| 192 | |
| 193 | LINKING PROGRAMS IN WINDOWS ENVIRONMENTS |
| 194 | |
| 195 | If you want to statically link a program against a PCRE2 library in the form of |
| 196 | a non-dll .a file, you must define PCRE2_STATIC before including src/pcre2.h. |
| 197 | |
| 198 | |
| 199 | CALLING CONVENTIONS IN WINDOWS ENVIRONMENTS |
| 200 | |
| 201 | It is possible to compile programs to use different calling conventions using |
| 202 | MSVC. Search the web for "calling conventions" for more information. To make it |
| 203 | easier to change the calling convention for the exported functions in the |
| 204 | PCRE2 library, the macro PCRE2_CALL_CONVENTION is present in all the external |
| 205 | definitions. It can be set externally when compiling (e.g. in CFLAGS). If it is |
| 206 | not set, it defaults to empty; the default calling convention is then used |
| 207 | (which is what is wanted most of the time). |
| 208 | |
| 209 | |
| 210 | COMMENTS ABOUT WIN32 BUILDS (see also "BUILDING PCRE2 ON WINDOWS WITH CMAKE") |
| 211 | |
| 212 | There are two ways of building PCRE2 using the "configure, make, make install" |
| 213 | paradigm on Windows systems: using MinGW or using Cygwin. These are not at all |
| 214 | the same thing; they are completely different from each other. There is also |
| 215 | support for building using CMake, which some users find a more straightforward |
| 216 | way of building PCRE2 under Windows. |
| 217 | |
| 218 | The MinGW home page (http://www.mingw.org/) says this: |
| 219 | |
| 220 | MinGW: A collection of freely available and freely distributable Windows |
| 221 | specific header files and import libraries combined with GNU toolsets that |
| 222 | allow one to produce native Windows programs that do not rely on any |
| 223 | 3rd-party C runtime DLLs. |
| 224 | |
| 225 | The Cygwin home page (http://www.cygwin.com/) says this: |
| 226 | |
| 227 | Cygwin is a Linux-like environment for Windows. It consists of two parts: |
| 228 | |
| 229 | . A DLL (cygwin1.dll) which acts as a Linux API emulation layer providing |
| 230 | substantial Linux API functionality |
| 231 | |
| 232 | . A collection of tools which provide Linux look and feel. |
| 233 | |
| 234 | On both MinGW and Cygwin, PCRE2 should build correctly using: |
| 235 | |
| 236 | ./configure && make && make install |
| 237 | |
| 238 | This should create two libraries called libpcre2-8 and libpcre2-posix. These |
| 239 | are independent libraries: when you link with libpcre2-posix you must also link |
| 240 | with libpcre2-8, which contains the basic functions. |
| 241 | |
| 242 | Using Cygwin's compiler generates libraries and executables that depend on |
| 243 | cygwin1.dll. If a library that is generated this way is distributed, |
| 244 | cygwin1.dll has to be distributed as well. Since cygwin1.dll is under the GPL |
| 245 | licence, this forces not only PCRE2 to be under the GPL, but also the entire |
| 246 | application. A distributor who wants to keep their own code proprietary must |
| 247 | purchase an appropriate Cygwin licence. |
| 248 | |
| 249 | MinGW has no such restrictions. The MinGW compiler generates a library or |
| 250 | executable that can run standalone on Windows without any third party dll or |
| 251 | licensing issues. |
| 252 | |
| 253 | But there is more complication: |
| 254 | |
| 255 | If a Cygwin user uses the -mno-cygwin Cygwin gcc flag, what that really does is |
| 256 | to tell Cygwin's gcc to use the MinGW gcc. Cygwin's gcc is only acting as a |
| 257 | front end to MinGW's gcc (if you install Cygwin's gcc, you get both Cygwin's |
| 258 | gcc and MinGW's gcc). So, a user can: |
| 259 | |
| 260 | . Build native binaries by using MinGW or by getting Cygwin and using |
| 261 | -mno-cygwin. |
| 262 | |
| 263 | . Build binaries that depend on cygwin1.dll by using Cygwin with the normal |
| 264 | compiler flags. |
| 265 | |
| 266 | The test files that are supplied with PCRE2 are in UNIX format, with LF |
| 267 | characters as line terminators. Unless your PCRE2 library uses a default |
| 268 | newline option that includes LF as a valid newline, it may be necessary to |
| 269 | change the line terminators in the test files to get some of the tests to work. |
| 270 | |
| 271 | |
| 272 | BUILDING PCRE2 ON WINDOWS WITH CMAKE |
| 273 | |
| 274 | CMake is an alternative configuration facility that can be used instead of |
| 275 | "configure". CMake creates project files (make files, solution files, etc.) |
| 276 | tailored to numerous development environments, including Visual Studio, |
| 277 | Borland, Msys, MinGW, NMake, and Unix. If possible, use short paths with no |
| 278 | spaces in the names for your CMake installation and your PCRE2 source and build |
| 279 | directories. |
| 280 | |
| 281 | The following instructions were contributed by a PCRE1 user, but they should |
| 282 | also work for PCRE2. If they are not followed exactly, errors may occur. In the |
| 283 | event that errors do occur, it is recommended that you delete the CMake cache |
| 284 | before attempting to repeat the CMake build process. In the CMake GUI, the |
| 285 | cache can be deleted by selecting "File > Delete Cache". |
| 286 | |
| 287 | 1. Install the latest CMake version available from http://www.cmake.org/, and |
| 288 | ensure that cmake\bin is on your path. |
| 289 | |
| 290 | 2. Unzip (retaining folder structure) the PCRE2 source tree into a source |
| 291 | directory such as C:\pcre2. You should ensure your local date and time |
| 292 | is not earlier than the file dates in your source dir if the release is |
| 293 | very new. |
| 294 | |
| 295 | 3. Create a new, empty build directory, preferably a subdirectory of the |
| 296 | source dir. For example, C:\pcre2\pcre2-xx\build. |
| 297 | |
| 298 | 4. Run cmake-gui from the Shell envirornment of your build tool, for example, |
| 299 | Msys for Msys/MinGW or Visual Studio Command Prompt for VC/VC++. Do not try |
| 300 | to start Cmake from the Windows Start menu, as this can lead to errors. |
| 301 | |
| 302 | 5. Enter C:\pcre2\pcre2-xx and C:\pcre2\pcre2-xx\build for the source and |
| 303 | build directories, respectively. |
| 304 | |
| 305 | 6. Hit the "Configure" button. |
| 306 | |
| 307 | 7. Select the particular IDE / build tool that you are using (Visual |
| 308 | Studio, MSYS makefiles, MinGW makefiles, etc.) |
| 309 | |
| 310 | 8. The GUI will then list several configuration options. This is where |
| 311 | you can disable Unicode support or select other PCRE2 optional features. |
| 312 | |
| 313 | 9. Hit "Configure" again. The adjacent "Generate" button should now be |
| 314 | active. |
| 315 | |
| 316 | 10. Hit "Generate". |
| 317 | |
| 318 | 11. The build directory should now contain a usable build system, be it a |
| 319 | solution file for Visual Studio, makefiles for MinGW, etc. Exit from |
| 320 | cmake-gui and use the generated build system with your compiler or IDE. |
| 321 | E.g., for MinGW you can run "make", or for Visual Studio, open the PCRE2 |
| 322 | solution, select the desired configuration (Debug, or Release, etc.) and |
| 323 | build the ALL_BUILD project. |
| 324 | |
| 325 | 12. If during configuration with cmake-gui you've elected to build the test |
| 326 | programs, you can execute them by building the test project. E.g., for |
| 327 | MinGW: "make test"; for Visual Studio build the RUN_TESTS project. The |
| 328 | most recent build configuration is targeted by the tests. A summary of |
| 329 | test results is presented. Complete test output is subsequently |
| 330 | available for review in Testing\Temporary under your build dir. |
| 331 | |
| 332 | |
| 333 | TESTING WITH RUNTEST.BAT |
| 334 | |
| 335 | If configured with CMake, building the test project ("make test" or building |
| 336 | ALL_TESTS in Visual Studio) creates (and runs) pcre2_test.bat (and depending |
| 337 | on your configuration options, possibly other test programs) in the build |
| 338 | directory. The pcre2_test.bat script runs RunTest.bat with correct source and |
| 339 | exe paths. |
| 340 | |
| 341 | For manual testing with RunTest.bat, provided the build dir is a subdirectory |
| 342 | of the source directory: Open command shell window. Chdir to the location |
| 343 | of your pcre2test.exe and pcre2grep.exe programs. Call RunTest.bat with |
| 344 | "..\RunTest.Bat" or "..\..\RunTest.bat" as appropriate. |
| 345 | |
| 346 | To run only a particular test with RunTest.Bat provide a test number argument. |
| 347 | |
| 348 | Otherwise: |
| 349 | |
| 350 | 1. Copy RunTest.bat into the directory where pcre2test.exe and pcre2grep.exe |
| 351 | have been created. |
| 352 | |
| 353 | 2. Edit RunTest.bat to indentify the full or relative location of |
| 354 | the pcre2 source (wherein which the testdata folder resides), e.g.: |
| 355 | |
| 356 | set srcdir=C:\pcre2\pcre2-10.00 |
| 357 | |
| 358 | 3. In a Windows command environment, chdir to the location of your bat and |
| 359 | exe programs. |
| 360 | |
| 361 | 4. Run RunTest.bat. Test outputs will automatically be compared to expected |
| 362 | results, and discrepancies will be identified in the console output. |
| 363 | |
| 364 | To independently test the just-in-time compiler, run pcre2_jit_test.exe. |
| 365 | |
| 366 | |
| 367 | BUILDING PCRE2 ON NATIVE Z/OS AND Z/VM |
| 368 | |
| 369 | z/OS and z/VM are operating systems for mainframe computers, produced by IBM. |
| 370 | The character code used is EBCDIC, not ASCII or Unicode. In z/OS, UNIX APIs and |
| 371 | applications can be supported through UNIX System Services, and in such an |
Elliott Hughes | 9bc971b | 2018-07-27 13:23:14 -0700 | [diff] [blame^] | 372 | environment it should be possible to build PCRE2 in the same way as in other |
| 373 | systems, with the EBCDIC related configuration settings, but it is not known if |
| 374 | anybody has tried this. |
Janis Danisevskis | 112c9cc | 2016-03-31 13:35:25 +0100 | [diff] [blame] | 375 | |
Elliott Hughes | 9bc971b | 2018-07-27 13:23:14 -0700 | [diff] [blame^] | 376 | In native z/OS (without UNIX System Services) and in z/VM, special ports are |
| 377 | required. For details, please see file 939 on this web site: |
Janis Danisevskis | 112c9cc | 2016-03-31 13:35:25 +0100 | [diff] [blame] | 378 | |
Elliott Hughes | 9bc971b | 2018-07-27 13:23:14 -0700 | [diff] [blame^] | 379 | http://www.cbttape.org |
Janis Danisevskis | 112c9cc | 2016-03-31 13:35:25 +0100 | [diff] [blame] | 380 | |
Elliott Hughes | 9bc971b | 2018-07-27 13:23:14 -0700 | [diff] [blame^] | 381 | Everything in that location, source and executable, is in EBCDIC and native |
| 382 | z/OS file formats. The port provides an API for LE languages such as COBOL and |
| 383 | for the z/OS and z/VM versions of the Rexx languages. |
Janis Danisevskis | 112c9cc | 2016-03-31 13:35:25 +0100 | [diff] [blame] | 384 | |
Elliott Hughes | 9bc971b | 2018-07-27 13:23:14 -0700 | [diff] [blame^] | 385 | =============================== |
| 386 | Last Updated: 13 September 2017 |
| 387 | =============================== |