1PCRE2TEST(1) General Commands Manual PCRE2TEST(1)
2
3
4
6 pcre2test - a program for testing Perl-compatible regular expressions.
7
9
10 pcre2test [options] [input file [output file]]
11
12 pcre2test is a test program for the PCRE2 regular expression libraries,
13 but it can also be used for experimenting with regular expressions.
14 This document describes the features of the test program; for details
15 of the regular expressions themselves, see the pcre2pattern documenta‐
16 tion. For details of the PCRE2 library function calls and their
17 options, see the pcre2api documentation.
18
19 The input for pcre2test is a sequence of regular expression patterns
20 and subject strings to be matched. There are also command lines for
21 setting defaults and controlling some special actions. The output shows
22 the result of each match attempt. Modifiers on external or internal
23 command lines, the patterns, and the subject lines specify PCRE2 func‐
24 tion options, control how the subject is processed, and what output is
25 produced.
26
27 As the original fairly simple PCRE library evolved, it acquired many
28 different features, and as a result, the original pcretest program
29 ended up with a lot of options in a messy, arcane syntax for testing
30 all the features. The move to the new PCRE2 API provided an opportunity
31 to re-implement the test program as pcre2test, with a cleaner modifier
32 syntax. Nevertheless, there are still many obscure modifiers, some of
33 which are specifically designed for use in conjunction with the test
34 script and data files that are distributed as part of PCRE2. All the
35 modifiers are documented here, some without much justification, but
36 many of them are unlikely to be of use except when testing the
37 libraries.
38
40
41 Different versions of the PCRE2 library can be built to support charac‐
42 ter strings that are encoded in 8-bit, 16-bit, or 32-bit code units.
43 One, two, or all three of these libraries may be simultaneously
44 installed. The pcre2test program can be used to test all the libraries.
45 However, its own input and output are always in 8-bit format. When
46 testing the 16-bit or 32-bit libraries, patterns and subject strings
47 are converted to 16-bit or 32-bit format before being passed to the
48 library functions. Results are converted back to 8-bit code units for
49 output.
50
51 In the rest of this document, the names of library functions and struc‐
52 tures are given in generic form, for example, pcre_compile(). The
53 actual names used in the libraries have a suffix _8, _16, or _32, as
54 appropriate.
55
57
58 Input to pcre2test is processed line by line, either by calling the C
59 library's fgets() function, or via the libreadline library. In some
60 Windows environments character 26 (hex 1A) causes an immediate end of
61 file, and no further data is read, so this character should be avoided
62 unless you really want that action.
63
64 The input is processed using using C's string functions, so must not
65 contain binary zeros, even though in Unix-like environments, fgets()
66 treats any bytes other than newline as data characters. An error is
67 generated if a binary zero is encountered. By default subject lines are
68 processed for backslash escapes, which makes it possible to include any
69 data value in strings that are passed to the library for matching. For
70 patterns, there is a facility for specifying some or all of the 8-bit
71 input characters as hexadecimal pairs, which makes it possible to
72 include binary zeros.
73
74 Input for the 16-bit and 32-bit libraries
75
76 When testing the 16-bit or 32-bit libraries, there is a need to be able
77 to generate character code points greater than 255 in the strings that
78 are passed to the library. For subject lines, backslash escapes can be
79 used. In addition, when the utf modifier (see "Setting compilation
80 options" below) is set, the pattern and any following subject lines are
81 interpreted as UTF-8 strings and translated to UTF-16 or UTF-32 as
82 appropriate.
83
84 For non-UTF testing of wide characters, the utf8_input modifier can be
85 used. This is mutually exclusive with utf, and is allowed only in
86 16-bit or 32-bit mode. It causes the pattern and following subject
87 lines to be treated as UTF-8 according to the original definition (RFC
88 2279), which allows for character values up to 0x7fffffff. Each charac‐
89 ter is placed in one 16-bit or 32-bit code unit (in the 16-bit case,
90 values greater than 0xffff cause an error to occur).
91
92 UTF-8 (in its original definition) is not capable of encoding values
93 greater than 0x7fffffff, but such values can be handled by the 32-bit
94 library. When testing this library in non-UTF mode with utf8_input set,
95 if any character is preceded by the byte 0xff (which is an invalid byte
96 in UTF-8) 0x80000000 is added to the character's value. This is the
97 only way of passing such code points in a pattern string. For subject
98 strings, using an escape sequence is preferable.
99
101
102 -8 If the 8-bit library has been built, this option causes it to
103 be used (this is the default). If the 8-bit library has not
104 been built, this option causes an error.
105
106 -16 If the 16-bit library has been built, this option causes it
107 to be used. If only the 16-bit library has been built, this
108 is the default. If the 16-bit library has not been built,
109 this option causes an error.
110
111 -32 If the 32-bit library has been built, this option causes it
112 to be used. If only the 32-bit library has been built, this
113 is the default. If the 32-bit library has not been built,
114 this option causes an error.
115
116 -ac Behave as if each pattern has the auto_callout modifier, that
117 is, insert automatic callouts into every pattern that is com‐
118 piled.
119
120 -AC As for -ac, but in addition behave as if each subject line
121 has the callout_extra modifier, that is, show additional
122 information from callouts.
123
124 -b Behave as if each pattern has the fullbincode modifier; the
125 full internal binary form of the pattern is output after com‐
126 pilation.
127
128 -C Output the version number of the PCRE2 library, and all
129 available information about the optional features that are
130 included, and then exit with zero exit code. All other
131 options are ignored. If both -C and -LM are present, which‐
132 ever is first is recognized.
133
134 -C option Output information about a specific build-time option, then
135 exit. This functionality is intended for use in scripts such
136 as RunTest. The following options output the value and set
137 the exit code as indicated:
138
139 ebcdic-nl the code for LF (= NL) in an EBCDIC environment:
140 0x15 or 0x25
141 0 if used in an ASCII environment
142 exit code is always 0
143 linksize the configured internal link size (2, 3, or 4)
144 exit code is set to the link size
145 newline the default newline setting:
146 CR, LF, CRLF, ANYCRLF, ANY, or NUL
147 exit code is always 0
148 bsr the default setting for what \R matches:
149 ANYCRLF or ANY
150 exit code is always 0
151
152 The following options output 1 for true or 0 for false, and
153 set the exit code to the same value:
154
155 backslash-C \C is supported (not locked out)
156 ebcdic compiled for an EBCDIC environment
157 jit just-in-time support is available
158 pcre2-16 the 16-bit library was built
159 pcre2-32 the 32-bit library was built
160 pcre2-8 the 8-bit library was built
161 unicode Unicode support is available
162
163 If an unknown option is given, an error message is output;
164 the exit code is 0.
165
166 -d Behave as if each pattern has the debug modifier; the inter‐
167 nal form and information about the compiled pattern is output
168 after compilation; -d is equivalent to -b -i.
169
170 -dfa Behave as if each subject line has the dfa modifier; matching
171 is done using the pcre2_dfa_match() function instead of the
172 default pcre2_match().
173
174 -error number[,number,...]
175 Call pcre2_get_error_message() for each of the error numbers
176 in the comma-separated list, display the resulting messages
177 on the standard output, then exit with zero exit code. The
178 numbers may be positive or negative. This is a convenience
179 facility for PCRE2 maintainers.
180
181 -help Output a brief summary these options and then exit.
182
183 -i Behave as if each pattern has the info modifier; information
184 about the compiled pattern is given after compilation.
185
186 -jit Behave as if each pattern line has the jit modifier; after
187 successful compilation, each pattern is passed to the just-
188 in-time compiler, if available.
189
190 -jitverify
191 Behave as if each pattern line has the jitverify modifier;
192 after successful compilation, each pattern is passed to the
193 just-in-time compiler, if available, and the use of JIT is
194 verified.
195
196 -LM List modifiers: write a list of available pattern and subject
197 modifiers to the standard output, then exit with zero exit
198 code. All other options are ignored. If both -C and -LM are
199 present, whichever is first is recognized.
200
201 -pattern modifier-list
202 Behave as if each pattern line contains the given modifiers.
203
204 -q Do not output the version number of pcre2test at the start of
205 execution.
206
207 -S size On Unix-like systems, set the size of the run-time stack to
208 size mebibytes (units of 1024*1024 bytes).
209
210 -subject modifier-list
211 Behave as if each subject line contains the given modifiers.
212
213 -t Run each compile and match many times with a timer, and out‐
214 put the resulting times per compile or match. When JIT is
215 used, separate times are given for the initial compile and
216 the JIT compile. You can control the number of iterations
217 that are used for timing by following -t with a number (as a
218 separate item on the command line). For example, "-t 1000"
219 iterates 1000 times. The default is to iterate 500,000 times.
220
221 -tm This is like -t except that it times only the matching phase,
222 not the compile phase.
223
224 -T -TM These behave like -t and -tm, but in addition, at the end of
225 a run, the total times for all compiles and matches are out‐
226 put.
227
228 -version Output the PCRE2 version number and then exit.
229
231
232 If pcre2test is given two filename arguments, it reads from the first
233 and writes to the second. If the first name is "-", input is taken from
234 the standard input. If pcre2test is given only one argument, it reads
235 from that file and writes to stdout. Otherwise, it reads from stdin and
236 writes to stdout.
237
238 When pcre2test is built, a configuration option can specify that it
239 should be linked with the libreadline or libedit library. When this is
240 done, if the input is from a terminal, it is read using the readline()
241 function. This provides line-editing and history facilities. The output
242 from the -help option states whether or not readline() will be used.
243
244 The program handles any number of tests, each of which consists of a
245 set of input lines. Each set starts with a regular expression pattern,
246 followed by any number of subject lines to be matched against that pat‐
247 tern. In between sets of test data, command lines that begin with # may
248 appear. This file format, with some restrictions, can also be processed
249 by the perltest.sh script that is distributed with PCRE2 as a means of
250 checking that the behaviour of PCRE2 and Perl is the same. For a speci‐
251 fication of perltest.sh, see the comments near its beginning.
252
253 When the input is a terminal, pcre2test prompts for each line of input,
254 using "re>" to prompt for regular expression patterns, and "data>" to
255 prompt for subject lines. Command lines starting with # can be entered
256 only in response to the "re>" prompt.
257
258 Each subject line is matched separately and independently. If you want
259 to do multi-line matches, you have to use the \n escape sequence (or \r
260 or \r\n, etc., depending on the newline setting) in a single line of
261 input to encode the newline sequences. There is no limit on the length
262 of subject lines; the input buffer is automatically extended if it is
263 too small. There are replication features that makes it possible to
264 generate long repetitive pattern or subject lines without having to
265 supply them explicitly.
266
267 An empty line or the end of the file signals the end of the subject
268 lines for a test, at which point a new pattern or command line is
269 expected if there is still input to be read.
270
272
273 In between sets of test data, a line that begins with # is interpreted
274 as a command line. If the first character is followed by white space or
275 an exclamation mark, the line is treated as a comment, and ignored.
276 Otherwise, the following commands are recognized:
277
278 #forbid_utf
279
280 Subsequent patterns automatically have the PCRE2_NEVER_UTF and
281 PCRE2_NEVER_UCP options set, which locks out the use of the PCRE2_UTF
282 and PCRE2_UCP options and the use of (*UTF) and (*UCP) at the start of
283 patterns. This command also forces an error if a subsequent pattern
284 contains any occurrences of \P, \p, or \X, which are still supported
285 when PCRE2_UTF is not set, but which require Unicode property support
286 to be included in the library.
287
288 This is a trigger guard that is used in test files to ensure that UTF
289 or Unicode property tests are not accidentally added to files that are
290 used when Unicode support is not included in the library. Setting
291 PCRE2_NEVER_UTF and PCRE2_NEVER_UCP as a default can also be obtained
292 by the use of #pattern; the difference is that #forbid_utf cannot be
293 unset, and the automatic options are not displayed in pattern informa‐
294 tion, to avoid cluttering up test output.
295
296 #load <filename>
297
298 This command is used to load a set of precompiled patterns from a file,
299 as described in the section entitled "Saving and restoring compiled
300 patterns" below.
301
302 #newline_default [<newline-list>]
303
304 When PCRE2 is built, a default newline convention can be specified.
305 This determines which characters and/or character pairs are recognized
306 as indicating a newline in a pattern or subject string. The default can
307 be overridden when a pattern is compiled. The standard test files con‐
308 tain tests of various newline conventions, but the majority of the
309 tests expect a single linefeed to be recognized as a newline by
310 default. Without special action the tests would fail when PCRE2 is com‐
311 piled with either CR or CRLF as the default newline.
312
313 The #newline_default command specifies a list of newline types that are
314 acceptable as the default. The types must be one of CR, LF, CRLF, ANY‐
315 CRLF, ANY, or NUL (in upper or lower case), for example:
316
317 #newline_default LF Any anyCRLF
318
319 If the default newline is in the list, this command has no effect. Oth‐
320 erwise, except when testing the POSIX API, a newline modifier that
321 specifies the first newline convention in the list (LF in the above
322 example) is added to any pattern that does not already have a newline
323 modifier. If the newline list is empty, the feature is turned off. This
324 command is present in a number of the standard test input files.
325
326 When the POSIX API is being tested there is no way to override the
327 default newline convention, though it is possible to set the newline
328 convention from within the pattern. A warning is given if the posix or
329 posix_nosub modifier is used when #newline_default would set a default
330 for the non-POSIX API.
331
332 #pattern <modifier-list>
333
334 This command sets a default modifier list that applies to all subse‐
335 quent patterns. Modifiers on a pattern can change these settings.
336
337 #perltest
338
339 The appearance of this line causes all subsequent modifier settings to
340 be checked for compatibility with the perltest.sh script, which is used
341 to confirm that Perl gives the same results as PCRE2. Also, apart from
342 comment lines, #pattern commands, and #subject commands that set or
343 unset "mark", no command lines are permitted, because they and many of
344 the modifiers are specific to pcre2test, and should not be used in test
345 files that are also processed by perltest.sh. The #perltest command
346 helps detect tests that are accidentally put in the wrong file.
347
348 #pop [<modifiers>]
349 #popcopy [<modifiers>]
350
351 These commands are used to manipulate the stack of compiled patterns,
352 as described in the section entitled "Saving and restoring compiled
353 patterns" below.
354
355 #save <filename>
356
357 This command is used to save a set of compiled patterns to a file, as
358 described in the section entitled "Saving and restoring compiled pat‐
359 terns" below.
360
361 #subject <modifier-list>
362
363 This command sets a default modifier list that applies to all subse‐
364 quent subject lines. Modifiers on a subject line can change these set‐
365 tings.
366
368
369 Modifier lists are used with both pattern and subject lines. Items in a
370 list are separated by commas followed by optional white space. Trailing
371 whitespace in a modifier list is ignored. Some modifiers may be given
372 for both patterns and subject lines, whereas others are valid only for
373 one or the other. Each modifier has a long name, for example
374 "anchored", and some of them must be followed by an equals sign and a
375 value, for example, "offset=12". Values cannot contain comma charac‐
376 ters, but may contain spaces. Modifiers that do not take values may be
377 preceded by a minus sign to turn off a previous setting.
378
379 A few of the more common modifiers can also be specified as single let‐
380 ters, for example "i" for "caseless". In documentation, following the
381 Perl convention, these are written with a slash ("the /i modifier") for
382 clarity. Abbreviated modifiers must all be concatenated in the first
383 item of a modifier list. If the first item is not recognized as a long
384 modifier name, it is interpreted as a sequence of these abbreviations.
385 For example:
386
387 /abc/ig,newline=cr,jit=3
388
389 This is a pattern line whose modifier list starts with two one-letter
390 modifiers (/i and /g). The lower-case abbreviated modifiers are the
391 same as used in Perl.
392
394
395 A pattern line must start with one of the following characters (common
396 symbols, excluding pattern meta-characters):
397
398 / ! " ' ` - = _ : ; , % & @ ~
399
400 This is interpreted as the pattern's delimiter. A regular expression
401 may be continued over several input lines, in which case the newline
402 characters are included within it. It is possible to include the delim‐
403 iter within the pattern by escaping it with a backslash, for example
404
405 /abc\/def/
406
407 If you do this, the escape and the delimiter form part of the pattern,
408 but since the delimiters are all non-alphanumeric, this does not affect
409 its interpretation. If the terminating delimiter is immediately fol‐
410 lowed by a backslash, for example,
411
412 /abc/\
413
414 then a backslash is added to the end of the pattern. This is done to
415 provide a way of testing the error condition that arises if a pattern
416 finishes with a backslash, because
417
418 /abc\/
419
420 is interpreted as the first line of a pattern that starts with "abc/",
421 causing pcre2test to read the next line as a continuation of the regu‐
422 lar expression.
423
424 A pattern can be followed by a modifier list (details below).
425
427
428 Before each subject line is passed to pcre2_match() or
429 pcre2_dfa_match(), leading and trailing white space is removed, and the
430 line is scanned for backslash escapes, unless the subject_literal modi‐
431 fier was set for the pattern. The following provide a means of encoding
432 non-printing characters in a visible way:
433
434 \a alarm (BEL, \x07)
435 \b backspace (\x08)
436 \e escape (\x27)
437 \f form feed (\x0c)
438 \n newline (\x0a)
439 \r carriage return (\x0d)
440 \t tab (\x09)
441 \v vertical tab (\x0b)
442 \nnn octal character (up to 3 octal digits); always
443 a byte unless > 255 in UTF-8 or 16-bit or 32-bit mode
444 \o{dd...} octal character (any number of octal digits}
445 \xhh hexadecimal byte (up to 2 hex digits)
446 \x{hh...} hexadecimal character (any number of hex digits)
447
448 The use of \x{hh...} is not dependent on the use of the utf modifier on
449 the pattern. It is recognized always. There may be any number of hexa‐
450 decimal digits inside the braces; invalid values provoke error mes‐
451 sages.
452
453 Note that \xhh specifies one byte rather than one character in UTF-8
454 mode; this makes it possible to construct invalid UTF-8 sequences for
455 testing purposes. On the other hand, \x{hh} is interpreted as a UTF-8
456 character in UTF-8 mode, generating more than one byte if the value is
457 greater than 127. When testing the 8-bit library not in UTF-8 mode,
458 \x{hh} generates one byte for values less than 256, and causes an error
459 for greater values.
460
461 In UTF-16 mode, all 4-digit \x{hhhh} values are accepted. This makes it
462 possible to construct invalid UTF-16 sequences for testing purposes.
463
464 In UTF-32 mode, all 4- to 8-digit \x{...} values are accepted. This
465 makes it possible to construct invalid UTF-32 sequences for testing
466 purposes.
467
468 There is a special backslash sequence that specifies replication of one
469 or more characters:
470
471 \[<characters>]{<count>}
472
473 This makes it possible to test long strings without having to provide
474 them as part of the file. For example:
475
476 \[abc]{4}
477
478 is converted to "abcabcabcabc". This feature does not support nesting.
479 To include a closing square bracket in the characters, code it as \x5D.
480
481 A backslash followed by an equals sign marks the end of the subject
482 string and the start of a modifier list. For example:
483
484 abc\=notbol,notempty
485
486 If the subject string is empty and \= is followed by whitespace, the
487 line is treated as a comment line, and is not used for matching. For
488 example:
489
490 \= This is a comment.
491 abc\= This is an invalid modifier list.
492
493 A backslash followed by any other non-alphanumeric character just
494 escapes that character. A backslash followed by anything else causes an
495 error. However, if the very last character in the line is a backslash
496 (and there is no modifier list), it is ignored. This gives a way of
497 passing an empty line as data, since a real empty line terminates the
498 data input.
499
500 If the subject_literal modifier is set for a pattern, all subject lines
501 that follow are treated as literals, with no special treatment of back‐
502 slashes. No replication is possible, and any subject modifiers must be
503 set as defaults by a #subject command.
504
506
507 There are several types of modifier that can appear in pattern lines.
508 Except where noted below, they may also be used in #pattern commands. A
509 pattern's modifier list can add to or override default modifiers that
510 were set by a previous #pattern command.
511
512 Setting compilation options
513
514 The following modifiers set options for pcre2_compile(). Most of them
515 set bits in the options argument of that function, but those whose
516 names start with PCRE2_EXTRA are additional options that are set in the
517 compile context. For the main options, there are some single-letter
518 abbreviations that are the same as Perl options. There is special han‐
519 dling for /x: if a second x is present, PCRE2_EXTENDED is converted
520 into PCRE2_EXTENDED_MORE as in Perl. A third appearance adds
521 PCRE2_EXTENDED as well, though this makes no difference to the way
522 pcre2_compile() behaves. See pcre2api for a description of the effects
523 of these options.
524
525 allow_empty_class set PCRE2_ALLOW_EMPTY_CLASS
526 allow_surrogate_escapes set PCRE2_EXTRA_ALLOW_SURROGATE_ESCAPES
527 alt_bsux set PCRE2_ALT_BSUX
528 alt_circumflex set PCRE2_ALT_CIRCUMFLEX
529 alt_verbnames set PCRE2_ALT_VERBNAMES
530 anchored set PCRE2_ANCHORED
531 auto_callout set PCRE2_AUTO_CALLOUT
532 bad_escape_is_literal set PCRE2_EXTRA_BAD_ESCAPE_IS_LITERAL
533 /i caseless set PCRE2_CASELESS
534 dollar_endonly set PCRE2_DOLLAR_ENDONLY
535 /s dotall set PCRE2_DOTALL
536 dupnames set PCRE2_DUPNAMES
537 endanchored set PCRE2_ENDANCHORED
538 escaped_cr_is_lf set PCRE2_EXTRA_ESCAPED_CR_IS_LF
539 /x extended set PCRE2_EXTENDED
540 /xx extended_more set PCRE2_EXTENDED_MORE
541 extra_alt_bsux set PCRE2_EXTRA_ALT_BSUX
542 firstline set PCRE2_FIRSTLINE
543 literal set PCRE2_LITERAL
544 match_line set PCRE2_EXTRA_MATCH_LINE
545 match_unset_backref set PCRE2_MATCH_UNSET_BACKREF
546 match_word set PCRE2_EXTRA_MATCH_WORD
547 /m multiline set PCRE2_MULTILINE
548 never_backslash_c set PCRE2_NEVER_BACKSLASH_C
549 never_ucp set PCRE2_NEVER_UCP
550 never_utf set PCRE2_NEVER_UTF
551 /n no_auto_capture set PCRE2_NO_AUTO_CAPTURE
552 no_auto_possess set PCRE2_NO_AUTO_POSSESS
553 no_dotstar_anchor set PCRE2_NO_DOTSTAR_ANCHOR
554 no_start_optimize set PCRE2_NO_START_OPTIMIZE
555 no_utf_check set PCRE2_NO_UTF_CHECK
556 ucp set PCRE2_UCP
557 ungreedy set PCRE2_UNGREEDY
558 use_offset_limit set PCRE2_USE_OFFSET_LIMIT
559 utf set PCRE2_UTF
560
561 As well as turning on the PCRE2_UTF option, the utf modifier causes all
562 non-printing characters in output strings to be printed using the
563 \x{hh...} notation. Otherwise, those less than 0x100 are output in hex
564 without the curly brackets. Setting utf in 16-bit or 32-bit mode also
565 causes pattern and subject strings to be translated to UTF-16 or
566 UTF-32, respectively, before being passed to library functions.
567
568 Setting compilation controls
569
570 The following modifiers affect the compilation process or request
571 information about the pattern. There are single-letter abbreviations
572 for some that are heavily used in the test files.
573
574 bsr=[anycrlf|unicode] specify \R handling
575 /B bincode show binary code without lengths
576 callout_info show callout information
577 convert=<options> request foreign pattern conversion
578 convert_glob_escape=c set glob escape character
579 convert_glob_separator=c set glob separator character
580 convert_length set convert buffer length
581 debug same as info,fullbincode
582 framesize show matching frame size
583 fullbincode show binary code with lengths
584 /I info show info about compiled pattern
585 hex unquoted characters are hexadecimal
586 jit[=<number>] use JIT
587 jitfast use JIT fast path
588 jitverify verify JIT use
589 locale=<name> use this locale
590 max_pattern_length=<n> set the maximum pattern length
591 memory show memory used
592 newline=<type> set newline type
593 null_context compile with a NULL context
594 parens_nest_limit=<n> set maximum parentheses depth
595 posix use the POSIX API
596 posix_nosub use the POSIX API with REG_NOSUB
597 push push compiled pattern onto the stack
598 pushcopy push a copy onto the stack
599 stackguard=<number> test the stackguard feature
600 subject_literal treat all subject lines as literal
601 tables=[0|1|2] select internal tables
602 use_length do not zero-terminate the pattern
603 utf8_input treat input as UTF-8
604
605 The effects of these modifiers are described in the following sections.
606
607 Newline and \R handling
608
609 The bsr modifier specifies what \R in a pattern should match. If it is
610 set to "anycrlf", \R matches CR, LF, or CRLF only. If it is set to
611 "unicode", \R matches any Unicode newline sequence. The default can be
612 specified when PCRE2 is built; if it is not, the default is set to Uni‐
613 code.
614
615 The newline modifier specifies which characters are to be interpreted
616 as newlines, both in the pattern and in subject lines. The type must be
617 one of CR, LF, CRLF, ANYCRLF, ANY, or NUL (in upper or lower case).
618
619 Information about a pattern
620
621 The debug modifier is a shorthand for info,fullbincode, requesting all
622 available information.
623
624 The bincode modifier causes a representation of the compiled code to be
625 output after compilation. This information does not contain length and
626 offset values, which ensures that the same output is generated for dif‐
627 ferent internal link sizes and different code unit widths. By using
628 bincode, the same regression tests can be used in different environ‐
629 ments.
630
631 The fullbincode modifier, by contrast, does include length and offset
632 values. This is used in a few special tests that run only for specific
633 code unit widths and link sizes, and is also useful for one-off tests.
634
635 The info modifier requests information about the compiled pattern
636 (whether it is anchored, has a fixed first character, and so on). The
637 information is obtained from the pcre2_pattern_info() function. Here
638 are some typical examples:
639
640 re> /(?i)(^a|^b)/m,info
641 Capture group count = 1
642 Compile options: multiline
643 Overall options: caseless multiline
644 First code unit at start or follows newline
645 Subject length lower bound = 1
646
647 re> /(?i)abc/info
648 Capture group count = 0
649 Compile options: <none>
650 Overall options: caseless
651 First code unit = 'a' (caseless)
652 Last code unit = 'c' (caseless)
653 Subject length lower bound = 3
654
655 "Compile options" are those specified by modifiers; "overall options"
656 have added options that are taken or deduced from the pattern. If both
657 sets of options are the same, just a single "options" line is output;
658 if there are no options, the line is omitted. "First code unit" is
659 where any match must start; if there is more than one they are listed
660 as "starting code units". "Last code unit" is the last literal code
661 unit that must be present in any match. This is not necessarily the
662 last character. These lines are omitted if no starting or ending code
663 units are recorded.
664
665 The framesize modifier shows the size, in bytes, of the storage frames
666 used by pcre2_match() for handling backtracking. The size depends on
667 the number of capturing parentheses in the pattern.
668
669 The callout_info modifier requests information about all the callouts
670 in the pattern. A list of them is output at the end of any other infor‐
671 mation that is requested. For each callout, either its number or string
672 is given, followed by the item that follows it in the pattern.
673
674 Passing a NULL context
675
676 Normally, pcre2test passes a context block to pcre2_compile(). If the
677 null_context modifier is set, however, NULL is passed. This is for
678 testing that pcre2_compile() behaves correctly in this case (it uses
679 default values).
680
681 Specifying pattern characters in hexadecimal
682
683 The hex modifier specifies that the characters of the pattern, except
684 for substrings enclosed in single or double quotes, are to be inter‐
685 preted as pairs of hexadecimal digits. This feature is provided as a
686 way of creating patterns that contain binary zeros and other non-print‐
687 ing characters. White space is permitted between pairs of digits. For
688 example, this pattern contains three characters:
689
690 /ab 32 59/hex
691
692 Parts of such a pattern are taken literally if quoted. This pattern
693 contains nine characters, only two of which are specified in hexadeci‐
694 mal:
695
696 /ab "literal" 32/hex
697
698 Either single or double quotes may be used. There is no way of includ‐
699 ing the delimiter within a substring. The hex and expand modifiers are
700 mutually exclusive.
701
702 Specifying the pattern's length
703
704 By default, patterns are passed to the compiling functions as zero-ter‐
705 minated strings but can be passed by length instead of being zero-ter‐
706 minated. The use_length modifier causes this to happen. Using a length
707 happens automatically (whether or not use_length is set) when hex is
708 set, because patterns specified in hexadecimal may contain binary
709 zeros.
710
711 If hex or use_length is used with the POSIX wrapper API (see "Using the
712 POSIX wrapper API" below), the REG_PEND extension is used to pass the
713 pattern's length.
714
715 Specifying wide characters in 16-bit and 32-bit modes
716
717 In 16-bit and 32-bit modes, all input is automatically treated as UTF-8
718 and translated to UTF-16 or UTF-32 when the utf modifier is set. For
719 testing the 16-bit and 32-bit libraries in non-UTF mode, the utf8_input
720 modifier can be used. It is mutually exclusive with utf. Input lines
721 are interpreted as UTF-8 as a means of specifying wide characters. More
722 details are given in "Input encoding" above.
723
724 Generating long repetitive patterns
725
726 Some tests use long patterns that are very repetitive. Instead of cre‐
727 ating a very long input line for such a pattern, you can use a special
728 repetition feature, similar to the one described for subject lines
729 above. If the expand modifier is present on a pattern, parts of the
730 pattern that have the form
731
732 \[<characters>]{<count>}
733
734 are expanded before the pattern is passed to pcre2_compile(). For exam‐
735 ple, \[AB]{6000} is expanded to "ABAB..." 6000 times. This construction
736 cannot be nested. An initial "\[" sequence is recognized only if "]{"
737 followed by decimal digits and "}" is found later in the pattern. If
738 not, the characters remain in the pattern unaltered. The expand and hex
739 modifiers are mutually exclusive.
740
741 If part of an expanded pattern looks like an expansion, but is really
742 part of the actual pattern, unwanted expansion can be avoided by giving
743 two values in the quantifier. For example, \[AB]{6000,6000} is not rec‐
744 ognized as an expansion item.
745
746 If the info modifier is set on an expanded pattern, the result of the
747 expansion is included in the information that is output.
748
749 JIT compilation
750
751 Just-in-time (JIT) compiling is a heavyweight optimization that can
752 greatly speed up pattern matching. See the pcre2jit documentation for
753 details. JIT compiling happens, optionally, after a pattern has been
754 successfully compiled into an internal form. The JIT compiler converts
755 this to optimized machine code. It needs to know whether the match-time
756 options PCRE2_PARTIAL_HARD and PCRE2_PARTIAL_SOFT are going to be used,
757 because different code is generated for the different cases. See the
758 partial modifier in "Subject Modifiers" below for details of how these
759 options are specified for each match attempt.
760
761 JIT compilation is requested by the jit pattern modifier, which may
762 optionally be followed by an equals sign and a number in the range 0 to
763 7. The three bits that make up the number specify which of the three
764 JIT operating modes are to be compiled:
765
766 1 compile JIT code for non-partial matching
767 2 compile JIT code for soft partial matching
768 4 compile JIT code for hard partial matching
769
770 The possible values for the jit modifier are therefore:
771
772 0 disable JIT
773 1 normal matching only
774 2 soft partial matching only
775 3 normal and soft partial matching
776 4 hard partial matching only
777 6 soft and hard partial matching only
778 7 all three modes
779
780 If no number is given, 7 is assumed. The phrase "partial matching"
781 means a call to pcre2_match() with either the PCRE2_PARTIAL_SOFT or the
782 PCRE2_PARTIAL_HARD option set. Note that such a call may return a com‐
783 plete match; the options enable the possibility of a partial match, but
784 do not require it. Note also that if you request JIT compilation only
785 for partial matching (for example, jit=2) but do not set the partial
786 modifier on a subject line, that match will not use JIT code because
787 none was compiled for non-partial matching.
788
789 If JIT compilation is successful, the compiled JIT code will automati‐
790 cally be used when an appropriate type of match is run, except when
791 incompatible run-time options are specified. For more details, see the
792 pcre2jit documentation. See also the jitstack modifier below for a way
793 of setting the size of the JIT stack.
794
795 If the jitfast modifier is specified, matching is done using the JIT
796 "fast path" interface, pcre2_jit_match(), which skips some of the san‐
797 ity checks that are done by pcre2_match(), and of course does not work
798 when JIT is not supported. If jitfast is specified without jit, jit=7
799 is assumed.
800
801 If the jitverify modifier is specified, information about the compiled
802 pattern shows whether JIT compilation was or was not successful. If
803 jitverify is specified without jit, jit=7 is assumed. If JIT compila‐
804 tion is successful when jitverify is set, the text "(JIT)" is added to
805 the first output line after a match or non match when JIT-compiled code
806 was actually used in the match.
807
808 Setting a locale
809
810 The locale modifier must specify the name of a locale, for example:
811
812 /pattern/locale=fr_FR
813
814 The given locale is set, pcre2_maketables() is called to build a set of
815 character tables for the locale, and this is then passed to pcre2_com‐
816 pile() when compiling the regular expression. The same tables are used
817 when matching the following subject lines. The locale modifier applies
818 only to the pattern on which it appears, but can be given in a #pattern
819 command if a default is needed. Setting a locale and alternate charac‐
820 ter tables are mutually exclusive.
821
822 Showing pattern memory
823
824 The memory modifier causes the size in bytes of the memory used to hold
825 the compiled pattern to be output. This does not include the size of
826 the pcre2_code block; it is just the actual compiled data. If the pat‐
827 tern is subsequently passed to the JIT compiler, the size of the JIT
828 compiled code is also output. Here is an example:
829
830 re> /a(b)c/jit,memory
831 Memory allocation (code space): 21
832 Memory allocation (JIT code): 1910
833
834
835 Limiting nested parentheses
836
837 The parens_nest_limit modifier sets a limit on the depth of nested
838 parentheses in a pattern. Breaching the limit causes a compilation
839 error. The default for the library is set when PCRE2 is built, but
840 pcre2test sets its own default of 220, which is required for running
841 the standard test suite.
842
843 Limiting the pattern length
844
845 The max_pattern_length modifier sets a limit, in code units, to the
846 length of pattern that pcre2_compile() will accept. Breaching the limit
847 causes a compilation error. The default is the largest number a
848 PCRE2_SIZE variable can hold (essentially unlimited).
849
850 Using the POSIX wrapper API
851
852 The posix and posix_nosub modifiers cause pcre2test to call PCRE2 via
853 the POSIX wrapper API rather than its native API. When posix_nosub is
854 used, the POSIX option REG_NOSUB is passed to regcomp(). The POSIX
855 wrapper supports only the 8-bit library. Note that it does not imply
856 POSIX matching semantics; for more detail see the pcre2posix documenta‐
857 tion. The following pattern modifiers set options for the regcomp()
858 function:
859
860 caseless REG_ICASE
861 multiline REG_NEWLINE
862 dotall REG_DOTALL )
863 ungreedy REG_UNGREEDY ) These options are not part of
864 ucp REG_UCP ) the POSIX standard
865 utf REG_UTF8 )
866
867 The regerror_buffsize modifier specifies a size for the error buffer
868 that is passed to regerror() in the event of a compilation error. For
869 example:
870
871 /abc/posix,regerror_buffsize=20
872
873 This provides a means of testing the behaviour of regerror() when the
874 buffer is too small for the error message. If this modifier has not
875 been set, a large buffer is used.
876
877 The aftertext and allaftertext subject modifiers work as described
878 below. All other modifiers are either ignored, with a warning message,
879 or cause an error.
880
881 The pattern is passed to regcomp() as a zero-terminated string by
882 default, but if the use_length or hex modifiers are set, the REG_PEND
883 extension is used to pass it by length.
884
885 Testing the stack guard feature
886
887 The stackguard modifier is used to test the use of pcre2_set_com‐
888 pile_recursion_guard(), a function that is provided to enable stack
889 availability to be checked during compilation (see the pcre2api docu‐
890 mentation for details). If the number specified by the modifier is
891 greater than zero, pcre2_set_compile_recursion_guard() is called to set
892 up callback from pcre2_compile() to a local function. The argument it
893 receives is the current nesting parenthesis depth; if this is greater
894 than the value given by the modifier, non-zero is returned, causing the
895 compilation to be aborted.
896
897 Using alternative character tables
898
899 The value specified for the tables modifier must be one of the digits
900 0, 1, or 2. It causes a specific set of built-in character tables to be
901 passed to pcre2_compile(). This is used in the PCRE2 tests to check be‐
902 haviour with different character tables. The digit specifies the tables
903 as follows:
904
905 0 do not pass any special character tables
906 1 the default ASCII tables, as distributed in
907 pcre2_chartables.c.dist
908 2 a set of tables defining ISO 8859 characters
909
910 In table 2, some characters whose codes are greater than 128 are iden‐
911 tified as letters, digits, spaces, etc. Setting alternate character
912 tables and a locale are mutually exclusive.
913
914 Setting certain match controls
915
916 The following modifiers are really subject modifiers, and are described
917 under "Subject Modifiers" below. However, they may be included in a
918 pattern's modifier list, in which case they are applied to every sub‐
919 ject line that is processed with that pattern. These modifiers do not
920 affect the compilation process.
921
922 aftertext show text after match
923 allaftertext show text after captures
924 allcaptures show all captures
925 allvector show the entire ovector
926 allusedtext show all consulted text
927 altglobal alternative global matching
928 /g global global matching
929 jitstack=<n> set size of JIT stack
930 mark show mark values
931 replace=<string> specify a replacement string
932 startchar show starting character when relevant
933 substitute_callout use substitution callouts
934 substitute_extended use PCRE2_SUBSTITUTE_EXTENDED
935 substitute_skip=<n> skip substitution number n
936 substitute_overflow_length use PCRE2_SUBSTITUTE_OVERFLOW_LENGTH
937 substitute_stop=<n> skip substitution number n and greater
938 substitute_unknown_unset use PCRE2_SUBSTITUTE_UNKNOWN_UNSET
939 substitute_unset_empty use PCRE2_SUBSTITUTE_UNSET_EMPTY
940
941 These modifiers may not appear in a #pattern command. If you want them
942 as defaults, set them in a #subject command.
943
944 Specifying literal subject lines
945
946 If the subject_literal modifier is present on a pattern, all the sub‐
947 ject lines that it matches are taken as literal strings, with no inter‐
948 pretation of backslashes. It is not possible to set subject modifiers
949 on such lines, but any that are set as defaults by a #subject command
950 are recognized.
951
952 Saving a compiled pattern
953
954 When a pattern with the push modifier is successfully compiled, it is
955 pushed onto a stack of compiled patterns, and pcre2test expects the
956 next line to contain a new pattern (or a command) instead of a subject
957 line. This facility is used when saving compiled patterns to a file, as
958 described in the section entitled "Saving and restoring compiled pat‐
959 terns" below. If pushcopy is used instead of push, a copy of the com‐
960 piled pattern is stacked, leaving the original as current, ready to
961 match the following input lines. This provides a way of testing the
962 pcre2_code_copy() function. The push and pushcopy modifiers are
963 incompatible with compilation modifiers such as global that act at
964 match time. Any that are specified are ignored (for the stacked copy),
965 with a warning message, except for replace, which causes an error. Note
966 that jitverify, which is allowed, does not carry through to any subse‐
967 quent matching that uses a stacked pattern.
968
969 Testing foreign pattern conversion
970
971 The experimental foreign pattern conversion functions in PCRE2 can be
972 tested by setting the convert modifier. Its argument is a colon-sepa‐
973 rated list of options, which set the equivalent option for the
974 pcre2_pattern_convert() function:
975
976 glob PCRE2_CONVERT_GLOB
977 glob_no_starstar PCRE2_CONVERT_GLOB_NO_STARSTAR
978 glob_no_wild_separator PCRE2_CONVERT_GLOB_NO_WILD_SEPARATOR
979 posix_basic PCRE2_CONVERT_POSIX_BASIC
980 posix_extended PCRE2_CONVERT_POSIX_EXTENDED
981 unset Unset all options
982
983 The "unset" value is useful for turning off a default that has been set
984 by a #pattern command. When one of these options is set, the input pat‐
985 tern is passed to pcre2_pattern_convert(). If the conversion is suc‐
986 cessful, the result is reflected in the output and then passed to
987 pcre2_compile(). The normal utf and no_utf_check options, if set, cause
988 the PCRE2_CONVERT_UTF and PCRE2_CONVERT_NO_UTF_CHECK options to be
989 passed to pcre2_pattern_convert().
990
991 By default, the conversion function is allowed to allocate a buffer for
992 its output. However, if the convert_length modifier is set to a value
993 greater than zero, pcre2test passes a buffer of the given length. This
994 makes it possible to test the length check.
995
996 The convert_glob_escape and convert_glob_separator modifiers can be
997 used to specify the escape and separator characters for glob process‐
998 ing, overriding the defaults, which are operating-system dependent.
999
1001
1002 The modifiers that can appear in subject lines and the #subject command
1003 are of two types.
1004
1005 Setting match options
1006
1007 The following modifiers set options for pcre2_match() or
1008 pcre2_dfa_match(). See pcreapi for a description of their effects.
1009
1010 anchored set PCRE2_ANCHORED
1011 endanchored set PCRE2_ENDANCHORED
1012 dfa_restart set PCRE2_DFA_RESTART
1013 dfa_shortest set PCRE2_DFA_SHORTEST
1014 no_jit set PCRE2_NO_JIT
1015 no_utf_check set PCRE2_NO_UTF_CHECK
1016 notbol set PCRE2_NOTBOL
1017 notempty set PCRE2_NOTEMPTY
1018 notempty_atstart set PCRE2_NOTEMPTY_ATSTART
1019 noteol set PCRE2_NOTEOL
1020 partial_hard (or ph) set PCRE2_PARTIAL_HARD
1021 partial_soft (or ps) set PCRE2_PARTIAL_SOFT
1022
1023 The partial matching modifiers are provided with abbreviations because
1024 they appear frequently in tests.
1025
1026 If the posix or posix_nosub modifier was present on the pattern, caus‐
1027 ing the POSIX wrapper API to be used, the only option-setting modifiers
1028 that have any effect are notbol, notempty, and noteol, causing REG_NOT‐
1029 BOL, REG_NOTEMPTY, and REG_NOTEOL, respectively, to be passed to
1030 regexec(). The other modifiers are ignored, with a warning message.
1031
1032 There is one additional modifier that can be used with the POSIX wrap‐
1033 per. It is ignored (with a warning) if used for non-POSIX matching.
1034
1035 posix_startend=<n>[:<m>]
1036
1037 This causes the subject string to be passed to regexec() using the
1038 REG_STARTEND option, which uses offsets to specify which part of the
1039 string is searched. If only one number is given, the end offset is
1040 passed as the end of the subject string. For more detail of REG_STAR‐
1041 TEND, see the pcre2posix documentation. If the subject string contains
1042 binary zeros (coded as escapes such as \x{00} because pcre2test does
1043 not support actual binary zeros in its input), you must use posix_star‐
1044 tend to specify its length.
1045
1046 Setting match controls
1047
1048 The following modifiers affect the matching process or request addi‐
1049 tional information. Some of them may also be specified on a pattern
1050 line (see above), in which case they apply to every subject line that
1051 is matched against that pattern.
1052
1053 aftertext show text after match
1054 allaftertext show text after captures
1055 allcaptures show all captures
1056 allvector show the entire ovector
1057 allusedtext show all consulted text (non-JIT only)
1058 altglobal alternative global matching
1059 callout_capture show captures at callout time
1060 callout_data=<n> set a value to pass via callouts
1061 callout_error=<n>[:<m>] control callout error
1062 callout_extra show extra callout information
1063 callout_fail=<n>[:<m>] control callout failure
1064 callout_no_where do not show position of a callout
1065 callout_none do not supply a callout function
1066 copy=<number or name> copy captured substring
1067 depth_limit=<n> set a depth limit
1068 dfa use pcre2_dfa_match()
1069 find_limits find match and depth limits
1070 get=<number or name> extract captured substring
1071 getall extract all captured substrings
1072 /g global global matching
1073 heap_limit=<n> set a limit on heap memory (Kbytes)
1074 jitstack=<n> set size of JIT stack
1075 mark show mark values
1076 match_limit=<n> set a match limit
1077 memory show heap memory usage
1078 null_context match with a NULL context
1079 offset=<n> set starting offset
1080 offset_limit=<n> set offset limit
1081 ovector=<n> set size of output vector
1082 recursion_limit=<n> obsolete synonym for depth_limit
1083 replace=<string> specify a replacement string
1084 startchar show startchar when relevant
1085 startoffset=<n> same as offset=<n>
1086 substitute_callout use substitution callouts
1087 substitute_extedded use PCRE2_SUBSTITUTE_EXTENDED
1088 substitute_skip=<n> skip substitution number n
1089 substitute_overflow_length use PCRE2_SUBSTITUTE_OVERFLOW_LENGTH
1090 substitute_stop=<n> skip substitution number n and greater
1091 substitute_unknown_unset use PCRE2_SUBSTITUTE_UNKNOWN_UNSET
1092 substitute_unset_empty use PCRE2_SUBSTITUTE_UNSET_EMPTY
1093 zero_terminate pass the subject as zero-terminated
1094
1095 The effects of these modifiers are described in the following sections.
1096 When matching via the POSIX wrapper API, the aftertext, allaftertext,
1097 and ovector subject modifiers work as described below. All other modi‐
1098 fiers are either ignored, with a warning message, or cause an error.
1099
1100 Showing more text
1101
1102 The aftertext modifier requests that as well as outputting the part of
1103 the subject string that matched the entire pattern, pcre2test should in
1104 addition output the remainder of the subject string. This is useful for
1105 tests where the subject contains multiple copies of the same substring.
1106 The allaftertext modifier requests the same action for captured sub‐
1107 strings as well as the main matched substring. In each case the remain‐
1108 der is output on the following line with a plus character following the
1109 capture number.
1110
1111 The allusedtext modifier requests that all the text that was consulted
1112 during a successful pattern match by the interpreter should be shown.
1113 This feature is not supported for JIT matching, and if requested with
1114 JIT it is ignored (with a warning message). Setting this modifier
1115 affects the output if there is a lookbehind at the start of a match, or
1116 a lookahead at the end, or if \K is used in the pattern. Characters
1117 that precede or follow the start and end of the actual match are indi‐
1118 cated in the output by '<' or '>' characters underneath them. Here is
1119 an example:
1120
1121 re> /(?<=pqr)abc(?=xyz)/
1122 data> 123pqrabcxyz456\=allusedtext
1123 0: pqrabcxyz
1124 <<< >>>
1125
1126 This shows that the matched string is "abc", with the preceding and
1127 following strings "pqr" and "xyz" having been consulted during the
1128 match (when processing the assertions).
1129
1130 The startchar modifier requests that the starting character for the
1131 match be indicated, if it is different to the start of the matched
1132 string. The only time when this occurs is when \K has been processed as
1133 part of the match. In this situation, the output for the matched string
1134 is displayed from the starting character instead of from the match
1135 point, with circumflex characters under the earlier characters. For
1136 example:
1137
1138 re> /abc\Kxyz/
1139 data> abcxyz\=startchar
1140 0: abcxyz
1141 ^^^
1142
1143 Unlike allusedtext, the startchar modifier can be used with JIT. How‐
1144 ever, these two modifiers are mutually exclusive.
1145
1146 Showing the value of all capture groups
1147
1148 The allcaptures modifier requests that the values of all potential cap‐
1149 tured parentheses be output after a match. By default, only those up to
1150 the highest one actually used in the match are output (corresponding to
1151 the return code from pcre2_match()). Groups that did not take part in
1152 the match are output as "<unset>". This modifier is not relevant for
1153 DFA matching (which does no capturing) and does not apply when replace
1154 is specified; it is ignored, with a warning message, if present.
1155
1156 Showing the entire ovector, for all outcomes
1157
1158 The allvector modifier requests that the entire ovector be shown, what‐
1159 ever the outcome of the match. Compare allcaptures, which shows only up
1160 to the maximum number of capture groups for the pattern, and then only
1161 for a successful complete non-DFA match. This modifier, which acts
1162 after any match result, and also for DFA matching, provides a means of
1163 checking that there are no unexpected modifications to ovector fields.
1164 Before each match attempt, the ovector is filled with a special value,
1165 and if this is found in both elements of a capturing pair,
1166 "<unchanged>" is output. After a successful match, this applies to all
1167 groups after the maximum capture group for the pattern. In other cases
1168 it applies to the entire ovector. After a partial match, the first two
1169 elements are the only ones that should be set. After a DFA match, the
1170 amount of ovector that is used depends on the number of matches that
1171 were found.
1172
1173 Testing pattern callouts
1174
1175 A callout function is supplied when pcre2test calls the library match‐
1176 ing functions, unless callout_none is specified. Its behaviour can be
1177 controlled by various modifiers listed above whose names begin with
1178 callout_. Details are given in the section entitled "Callouts" below.
1179 Testing callouts from pcre2_substitute() is decribed separately in
1180 "Testing the substitution function" below.
1181
1182 Finding all matches in a string
1183
1184 Searching for all possible matches within a subject can be requested by
1185 the global or altglobal modifier. After finding a match, the matching
1186 function is called again to search the remainder of the subject. The
1187 difference between global and altglobal is that the former uses the
1188 start_offset argument to pcre2_match() or pcre2_dfa_match() to start
1189 searching at a new point within the entire string (which is what Perl
1190 does), whereas the latter passes over a shortened subject. This makes a
1191 difference to the matching process if the pattern begins with a lookbe‐
1192 hind assertion (including \b or \B).
1193
1194 If an empty string is matched, the next match is done with the
1195 PCRE2_NOTEMPTY_ATSTART and PCRE2_ANCHORED flags set, in order to search
1196 for another, non-empty, match at the same point in the subject. If this
1197 match fails, the start offset is advanced, and the normal match is
1198 retried. This imitates the way Perl handles such cases when using the
1199 /g modifier or the split() function. Normally, the start offset is
1200 advanced by one character, but if the newline convention recognizes
1201 CRLF as a newline, and the current character is CR followed by LF, an
1202 advance of two characters occurs.
1203
1204 Testing substring extraction functions
1205
1206 The copy and get modifiers can be used to test the pcre2_sub‐
1207 string_copy_xxx() and pcre2_substring_get_xxx() functions. They can be
1208 given more than once, and each can specify a capture group name or num‐
1209 ber, for example:
1210
1211 abcd\=copy=1,copy=3,get=G1
1212
1213 If the #subject command is used to set default copy and/or get lists,
1214 these can be unset by specifying a negative number to cancel all num‐
1215 bered groups and an empty name to cancel all named groups.
1216
1217 The getall modifier tests pcre2_substring_list_get(), which extracts
1218 all captured substrings.
1219
1220 If the subject line is successfully matched, the substrings extracted
1221 by the convenience functions are output with C, G, or L after the
1222 string number instead of a colon. This is in addition to the normal
1223 full list. The string length (that is, the return from the extraction
1224 function) is given in parentheses after each substring, followed by the
1225 name when the extraction was by name.
1226
1227 Testing the substitution function
1228
1229 If the replace modifier is set, the pcre2_substitute() function is
1230 called instead of one of the matching functions. Note that replacement
1231 strings cannot contain commas, because a comma signifies the end of a
1232 modifier. This is not thought to be an issue in a test program.
1233
1234 Unlike subject strings, pcre2test does not process replacement strings
1235 for escape sequences. In UTF mode, a replacement string is checked to
1236 see if it is a valid UTF-8 string. If so, it is correctly converted to
1237 a UTF string of the appropriate code unit width. If it is not a valid
1238 UTF-8 string, the individual code units are copied directly. This pro‐
1239 vides a means of passing an invalid UTF-8 string for testing purposes.
1240
1241 The following modifiers set options (in additional to the normal match
1242 options) for pcre2_substitute():
1243
1244 global PCRE2_SUBSTITUTE_GLOBAL
1245 substitute_extended PCRE2_SUBSTITUTE_EXTENDED
1246 substitute_overflow_length PCRE2_SUBSTITUTE_OVERFLOW_LENGTH
1247 substitute_unknown_unset PCRE2_SUBSTITUTE_UNKNOWN_UNSET
1248 substitute_unset_empty PCRE2_SUBSTITUTE_UNSET_EMPTY
1249
1250
1251 After a successful substitution, the modified string is output, pre‐
1252 ceded by the number of replacements. This may be zero if there were no
1253 matches. Here is a simple example of a substitution test:
1254
1255 /abc/replace=xxx
1256 =abc=abc=
1257 1: =xxx=abc=
1258 =abc=abc=\=global
1259 2: =xxx=xxx=
1260
1261 Subject and replacement strings should be kept relatively short (fewer
1262 than 256 characters) for substitution tests, as fixed-size buffers are
1263 used. To make it easy to test for buffer overflow, if the replacement
1264 string starts with a number in square brackets, that number is passed
1265 to pcre2_substitute() as the size of the output buffer, with the
1266 replacement string starting at the next character. Here is an example
1267 that tests the edge case:
1268
1269 /abc/
1270 123abc123\=replace=[10]XYZ
1271 1: 123XYZ123
1272 123abc123\=replace=[9]XYZ
1273 Failed: error -47: no more memory
1274
1275 The default action of pcre2_substitute() is to return
1276 PCRE2_ERROR_NOMEMORY when the output buffer is too small. However, if
1277 the PCRE2_SUBSTITUTE_OVERFLOW_LENGTH option is set (by using the sub‐
1278 stitute_overflow_length modifier), pcre2_substitute() continues to go
1279 through the motions of matching and substituting (but not doing any
1280 callouts), in order to compute the size of buffer that is required.
1281 When this happens, pcre2test shows the required buffer length (which
1282 includes space for the trailing zero) as part of the error message. For
1283 example:
1284
1285 /abc/substitute_overflow_length
1286 123abc123\=replace=[9]XYZ
1287 Failed: error -47: no more memory: 10 code units are needed
1288
1289 A replacement string is ignored with POSIX and DFA matching. Specifying
1290 partial matching provokes an error return ("bad option value") from
1291 pcre2_substitute().
1292
1293 Testing substitute callouts
1294
1295 If the substitute_callout modifier is set, a substitution callout func‐
1296 tion is set up. The null_context modifier must not be set, because the
1297 address of the callout function is passed in a match context. When the
1298 callout function is called (after each substitution), details of the
1299 the input and output strings are output. For example:
1300
1301 /abc/g,replace=<$0>,substitute_callout
1302 abcdefabcpqr
1303 1(1) Old 0 3 "abc" New 0 5 "<abc>"
1304 2(1) Old 6 9 "abc" New 8 13 "<abc>"
1305 2: <abc>def<abc>pqr
1306
1307 The first number on each callout line is the count of matches. The
1308 parenthesized number is the number of pairs that are set in the ovector
1309 (that is, one more than the number of capturing groups that were set).
1310 Then are listed the offsets of the old substring, its contents, and the
1311 same for the replacement.
1312
1313 By default, the substitution callout function returns zero, which
1314 accepts the replacement and causes matching to continue if /g was used.
1315 Two further modifiers can be used to test other return values. If sub‐
1316 stitute_skip is set to a value greater than zero the callout function
1317 returns +1 for the match of that number, and similarly substitute_stop
1318 returns -1. These cause the replacement to be rejected, and -1 causes
1319 no further matching to take place. If either of them are set, substi‐
1320 tute_callout is assumed. For example:
1321
1322 /abc/g,replace=<$0>,substitute_skip=1
1323 abcdefabcpqr
1324 1(1) Old 0 3 "abc" New 0 5 "<abc> SKIPPED"
1325 2(1) Old 6 9 "abc" New 6 11 "<abc>"
1326 2: abcdef<abc>pqr
1327 abcdefabcpqr\=substitute_stop=1
1328 1(1) Old 0 3 "abc" New 0 5 "<abc> STOPPED"
1329 1: abcdefabcpqr
1330
1331 If both are set for the same number, stop takes precedence. Only a sin‐
1332 gle skip or stop is supported, which is sufficient for testing that the
1333 feature works.
1334
1335 Setting the JIT stack size
1336
1337 The jitstack modifier provides a way of setting the maximum stack size
1338 that is used by the just-in-time optimization code. It is ignored if
1339 JIT optimization is not being used. The value is a number of kibibytes
1340 (units of 1024 bytes). Setting zero reverts to the default of 32KiB.
1341 Providing a stack that is larger than the default is necessary only for
1342 very complicated patterns. If jitstack is set non-zero on a subject
1343 line it overrides any value that was set on the pattern.
1344
1345 Setting heap, match, and depth limits
1346
1347 The heap_limit, match_limit, and depth_limit modifiers set the appro‐
1348 priate limits in the match context. These values are ignored when the
1349 find_limits modifier is specified.
1350
1351 Finding minimum limits
1352
1353 If the find_limits modifier is present on a subject line, pcre2test
1354 calls the relevant matching function several times, setting different
1355 values in the match context via pcre2_set_heap_limit(),
1356 pcre2_set_match_limit(), or pcre2_set_depth_limit() until it finds the
1357 minimum values for each parameter that allows the match to complete
1358 without error. If JIT is being used, only the match limit is relevant.
1359
1360 When using this modifier, the pattern should not contain any limit set‐
1361 tings such as (*LIMIT_MATCH=...) within it. If such a setting is
1362 present and is lower than the minimum matching value, the minimum value
1363 cannot be found because pcre2_set_match_limit() etc. are only able to
1364 reduce the value of an in-pattern limit; they cannot increase it.
1365
1366 For non-DFA matching, the minimum depth_limit number is a measure of
1367 how much nested backtracking happens (that is, how deeply the pattern's
1368 tree is searched). In the case of DFA matching, depth_limit controls
1369 the depth of recursive calls of the internal function that is used for
1370 handling pattern recursion, lookaround assertions, and atomic groups.
1371
1372 For non-DFA matching, the match_limit number is a measure of the amount
1373 of backtracking that takes place, and learning the minimum value can be
1374 instructive. For most simple matches, the number is quite small, but
1375 for patterns with very large numbers of matching possibilities, it can
1376 become large very quickly with increasing length of subject string. In
1377 the case of DFA matching, match_limit controls the total number of
1378 calls, both recursive and non-recursive, to the internal matching func‐
1379 tion, thus controlling the overall amount of computing resource that is
1380 used.
1381
1382 For both kinds of matching, the heap_limit number, which is in
1383 kibibytes (units of 1024 bytes), limits the amount of heap memory used
1384 for matching. A value of zero disables the use of any heap memory; many
1385 simple pattern matches can be done without using the heap, so zero is
1386 not an unreasonable setting.
1387
1388 Showing MARK names
1389
1390
1391 The mark modifier causes the names from backtracking control verbs that
1392 are returned from calls to pcre2_match() to be displayed. If a mark is
1393 returned for a match, non-match, or partial match, pcre2test shows it.
1394 For a match, it is on a line by itself, tagged with "MK:". Otherwise,
1395 it is added to the non-match message.
1396
1397 Showing memory usage
1398
1399 The memory modifier causes pcre2test to log the sizes of all heap mem‐
1400 ory allocation and freeing calls that occur during a call to
1401 pcre2_match() or pcre2_dfa_match(). These occur only when a match
1402 requires a bigger vector than the default for remembering backtracking
1403 points (pcre2_match()) or for internal workspace (pcre2_dfa_match()).
1404 In many cases there will be no heap memory used and therefore no addi‐
1405 tional output. No heap memory is allocated during matching with JIT, so
1406 in that case the memory modifier never has any effect. For this modi‐
1407 fier to work, the null_context modifier must not be set on both the
1408 pattern and the subject, though it can be set on one or the other.
1409
1410 Setting a starting offset
1411
1412 The offset modifier sets an offset in the subject string at which
1413 matching starts. Its value is a number of code units, not characters.
1414
1415 Setting an offset limit
1416
1417 The offset_limit modifier sets a limit for unanchored matches. If a
1418 match cannot be found starting at or before this offset in the subject,
1419 a "no match" return is given. The data value is a number of code units,
1420 not characters. When this modifier is used, the use_offset_limit modi‐
1421 fier must have been set for the pattern; if not, an error is generated.
1422
1423 Setting the size of the output vector
1424
1425 The ovector modifier applies only to the subject line in which it
1426 appears, though of course it can also be used to set a default in a
1427 #subject command. It specifies the number of pairs of offsets that are
1428 available for storing matching information. The default is 15.
1429
1430 A value of zero is useful when testing the POSIX API because it causes
1431 regexec() to be called with a NULL capture vector. When not testing the
1432 POSIX API, a value of zero is used to cause pcre2_match_data_cre‐
1433 ate_from_pattern() to be called, in order to create a match block of
1434 exactly the right size for the pattern. (It is not possible to create a
1435 match block with a zero-length ovector; there is always at least one
1436 pair of offsets.)
1437
1438 Passing the subject as zero-terminated
1439
1440 By default, the subject string is passed to a native API matching func‐
1441 tion with its correct length. In order to test the facility for passing
1442 a zero-terminated string, the zero_terminate modifier is provided. It
1443 causes the length to be passed as PCRE2_ZERO_TERMINATED. When matching
1444 via the POSIX interface, this modifier is ignored, with a warning.
1445
1446 When testing pcre2_substitute(), this modifier also has the effect of
1447 passing the replacement string as zero-terminated.
1448
1449 Passing a NULL context
1450
1451 Normally, pcre2test passes a context block to pcre2_match(),
1452 pcre2_dfa_match(), pcre2_jit_match() or pcre2_substitute(). If the
1453 null_context modifier is set, however, NULL is passed. This is for
1454 testing that the matching and substitution functions behave correctly
1455 in this case (they use default values). This modifier cannot be used
1456 with the find_limits or substitute_callout modifiers.
1457
1459
1460 By default, pcre2test uses the standard PCRE2 matching function,
1461 pcre2_match() to match each subject line. PCRE2 also supports an alter‐
1462 native matching function, pcre2_dfa_match(), which operates in a dif‐
1463 ferent way, and has some restrictions. The differences between the two
1464 functions are described in the pcre2matching documentation.
1465
1466 If the dfa modifier is set, the alternative matching function is used.
1467 This function finds all possible matches at a given point in the sub‐
1468 ject. If, however, the dfa_shortest modifier is set, processing stops
1469 after the first match is found. This is always the shortest possible
1470 match.
1471
1473
1474 This section describes the output when the normal matching function,
1475 pcre2_match(), is being used.
1476
1477 When a match succeeds, pcre2test outputs the list of captured sub‐
1478 strings, starting with number 0 for the string that matched the whole
1479 pattern. Otherwise, it outputs "No match" when the return is
1480 PCRE2_ERROR_NOMATCH, or "Partial match:" followed by the partially
1481 matching substring when the return is PCRE2_ERROR_PARTIAL. (Note that
1482 this is the entire substring that was inspected during the partial
1483 match; it may include characters before the actual match start if a
1484 lookbehind assertion, \K, \b, or \B was involved.)
1485
1486 For any other return, pcre2test outputs the PCRE2 negative error number
1487 and a short descriptive phrase. If the error is a failed UTF string
1488 check, the code unit offset of the start of the failing character is
1489 also output. Here is an example of an interactive pcre2test run.
1490
1491 $ pcre2test
1492 PCRE2 version 10.22 2016-07-29
1493
1494 re> /^abc(\d+)/
1495 data> abc123
1496 0: abc123
1497 1: 123
1498 data> xyz
1499 No match
1500
1501 Unset capturing substrings that are not followed by one that is set are
1502 not shown by pcre2test unless the allcaptures modifier is specified. In
1503 the following example, there are two capturing substrings, but when the
1504 first data line is matched, the second, unset substring is not shown.
1505 An "internal" unset substring is shown as "<unset>", as for the second
1506 data line.
1507
1508 re> /(a)|(b)/
1509 data> a
1510 0: a
1511 1: a
1512 data> b
1513 0: b
1514 1: <unset>
1515 2: b
1516
1517 If the strings contain any non-printing characters, they are output as
1518 \xhh escapes if the value is less than 256 and UTF mode is not set.
1519 Otherwise they are output as \x{hh...} escapes. See below for the defi‐
1520 nition of non-printing characters. If the aftertext modifier is set,
1521 the output for substring 0 is followed by the the rest of the subject
1522 string, identified by "0+" like this:
1523
1524 re> /cat/aftertext
1525 data> cataract
1526 0: cat
1527 0+ aract
1528
1529 If global matching is requested, the results of successive matching
1530 attempts are output in sequence, like this:
1531
1532 re> /\Bi(\w\w)/g
1533 data> Mississippi
1534 0: iss
1535 1: ss
1536 0: iss
1537 1: ss
1538 0: ipp
1539 1: pp
1540
1541 "No match" is output only if the first match attempt fails. Here is an
1542 example of a failure message (the offset 4 that is specified by the
1543 offset modifier is past the end of the subject string):
1544
1545 re> /xyz/
1546 data> xyz\=offset=4
1547 Error -24 (bad offset value)
1548
1549 Note that whereas patterns can be continued over several lines (a plain
1550 ">" prompt is used for continuations), subject lines may not. However
1551 newlines can be included in a subject by means of the \n escape (or \r,
1552 \r\n, etc., depending on the newline sequence setting).
1553
1555
1556 When the alternative matching function, pcre2_dfa_match(), is used, the
1557 output consists of a list of all the matches that start at the first
1558 point in the subject where there is at least one match. For example:
1559
1560 re> /(tang|tangerine|tan)/
1561 data> yellow tangerine\=dfa
1562 0: tangerine
1563 1: tang
1564 2: tan
1565
1566 Using the normal matching function on this data finds only "tang". The
1567 longest matching string is always given first (and numbered zero).
1568 After a PCRE2_ERROR_PARTIAL return, the output is "Partial match:",
1569 followed by the partially matching substring. Note that this is the
1570 entire substring that was inspected during the partial match; it may
1571 include characters before the actual match start if a lookbehind asser‐
1572 tion, \b, or \B was involved. (\K is not supported for DFA matching.)
1573
1574 If global matching is requested, the search for further matches resumes
1575 at the end of the longest match. For example:
1576
1577 re> /(tang|tangerine|tan)/g
1578 data> yellow tangerine and tangy sultana\=dfa
1579 0: tangerine
1580 1: tang
1581 2: tan
1582 0: tang
1583 1: tan
1584 0: tan
1585
1586 The alternative matching function does not support substring capture,
1587 so the modifiers that are concerned with captured substrings are not
1588 relevant.
1589
1591
1592 When the alternative matching function has given the PCRE2_ERROR_PAR‐
1593 TIAL return, indicating that the subject partially matched the pattern,
1594 you can restart the match with additional subject data by means of the
1595 dfa_restart modifier. For example:
1596
1597 re> /^\d?\d(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\d\d$/
1598 data> 23ja\=P,dfa
1599 Partial match: 23ja
1600 data> n05\=dfa,dfa_restart
1601 0: n05
1602
1603 For further information about partial matching, see the pcre2partial
1604 documentation.
1605
1607
1608 If the pattern contains any callout requests, pcre2test's callout func‐
1609 tion is called during matching unless callout_none is specified. This
1610 works with both matching functions, and with JIT, though there are some
1611 differences in behaviour. The output for callouts with numerical argu‐
1612 ments and those with string arguments is slightly different.
1613
1614 Callouts with numerical arguments
1615
1616 By default, the callout function displays the callout number, the start
1617 and current positions in the subject text at the callout time, and the
1618 next pattern item to be tested. For example:
1619
1620 --->pqrabcdef
1621 0 ^ ^ \d
1622
1623 This output indicates that callout number 0 occurred for a match
1624 attempt starting at the fourth character of the subject string, when
1625 the pointer was at the seventh character, and when the next pattern
1626 item was \d. Just one circumflex is output if the start and current
1627 positions are the same, or if the current position precedes the start
1628 position, which can happen if the callout is in a lookbehind assertion.
1629
1630 Callouts numbered 255 are assumed to be automatic callouts, inserted as
1631 a result of the auto_callout pattern modifier. In this case, instead of
1632 showing the callout number, the offset in the pattern, preceded by a
1633 plus, is output. For example:
1634
1635 re> /\d?[A-E]\*/auto_callout
1636 data> E*
1637 --->E*
1638 +0 ^ \d?
1639 +3 ^ [A-E]
1640 +8 ^^ \*
1641 +10 ^ ^
1642 0: E*
1643
1644 If a pattern contains (*MARK) items, an additional line is output when‐
1645 ever a change of latest mark is passed to the callout function. For
1646 example:
1647
1648 re> /a(*MARK:X)bc/auto_callout
1649 data> abc
1650 --->abc
1651 +0 ^ a
1652 +1 ^^ (*MARK:X)
1653 +10 ^^ b
1654 Latest Mark: X
1655 +11 ^ ^ c
1656 +12 ^ ^
1657 0: abc
1658
1659 The mark changes between matching "a" and "b", but stays the same for
1660 the rest of the match, so nothing more is output. If, as a result of
1661 backtracking, the mark reverts to being unset, the text "<unset>" is
1662 output.
1663
1664 Callouts with string arguments
1665
1666 The output for a callout with a string argument is similar, except that
1667 instead of outputting a callout number before the position indicators,
1668 the callout string and its offset in the pattern string are output
1669 before the reflection of the subject string, and the subject string is
1670 reflected for each callout. For example:
1671
1672 re> /^ab(?C'first')cd(?C"second")ef/
1673 data> abcdefg
1674 Callout (7): 'first'
1675 --->abcdefg
1676 ^ ^ c
1677 Callout (20): "second"
1678 --->abcdefg
1679 ^ ^ e
1680 0: abcdef
1681
1682
1683 Callout modifiers
1684
1685 The callout function in pcre2test returns zero (carry on matching) by
1686 default, but you can use a callout_fail modifier in a subject line to
1687 change this and other parameters of the callout (see below).
1688
1689 If the callout_capture modifier is set, the current captured groups are
1690 output when a callout occurs. This is useful only for non-DFA matching,
1691 as pcre2_dfa_match() does not support capturing, so no captures are
1692 ever shown.
1693
1694 The normal callout output, showing the callout number or pattern offset
1695 (as described above) is suppressed if the callout_no_where modifier is
1696 set.
1697
1698 When using the interpretive matching function pcre2_match() without
1699 JIT, setting the callout_extra modifier causes additional output from
1700 pcre2test's callout function to be generated. For the first callout in
1701 a match attempt at a new starting position in the subject, "New match
1702 attempt" is output. If there has been a backtrack since the last call‐
1703 out (or start of matching if this is the first callout), "Backtrack" is
1704 output, followed by "No other matching paths" if the backtrack ended
1705 the previous match attempt. For example:
1706
1707 re> /(a+)b/auto_callout,no_start_optimize,no_auto_possess
1708 data> aac\=callout_extra
1709 New match attempt
1710 --->aac
1711 +0 ^ (
1712 +1 ^ a+
1713 +3 ^ ^ )
1714 +4 ^ ^ b
1715 Backtrack
1716 --->aac
1717 +3 ^^ )
1718 +4 ^^ b
1719 Backtrack
1720 No other matching paths
1721 New match attempt
1722 --->aac
1723 +0 ^ (
1724 +1 ^ a+
1725 +3 ^^ )
1726 +4 ^^ b
1727 Backtrack
1728 No other matching paths
1729 New match attempt
1730 --->aac
1731 +0 ^ (
1732 +1 ^ a+
1733 Backtrack
1734 No other matching paths
1735 New match attempt
1736 --->aac
1737 +0 ^ (
1738 +1 ^ a+
1739 No match
1740
1741 Notice that various optimizations must be turned off if you want all
1742 possible matching paths to be scanned. If no_start_optimize is not
1743 used, there is an immediate "no match", without any callouts, because
1744 the starting optimization fails to find "b" in the subject, which it
1745 knows must be present for any match. If no_auto_possess is not used,
1746 the "a+" item is turned into "a++", which reduces the number of back‐
1747 tracks.
1748
1749 The callout_extra modifier has no effect if used with the DFA matching
1750 function, or with JIT.
1751
1752 Return values from callouts
1753
1754 The default return from the callout function is zero, which allows
1755 matching to continue. The callout_fail modifier can be given one or two
1756 numbers. If there is only one number, 1 is returned instead of 0 (caus‐
1757 ing matching to backtrack) when a callout of that number is reached. If
1758 two numbers (<n>:<m>) are given, 1 is returned when callout <n> is
1759 reached and there have been at least <m> callouts. The callout_error
1760 modifier is similar, except that PCRE2_ERROR_CALLOUT is returned, caus‐
1761 ing the entire matching process to be aborted. If both these modifiers
1762 are set for the same callout number, callout_error takes precedence.
1763 Note that callouts with string arguments are always given the number
1764 zero.
1765
1766 The callout_data modifier can be given an unsigned or a negative num‐
1767 ber. This is set as the "user data" that is passed to the matching
1768 function, and passed back when the callout function is invoked. Any
1769 value other than zero is used as a return from pcre2test's callout
1770 function.
1771
1772 Inserting callouts can be helpful when using pcre2test to check compli‐
1773 cated regular expressions. For further information about callouts, see
1774 the pcre2callout documentation.
1775
1777
1778 When pcre2test is outputting text in the compiled version of a pattern,
1779 bytes other than 32-126 are always treated as non-printing characters
1780 and are therefore shown as hex escapes.
1781
1782 When pcre2test is outputting text that is a matched part of a subject
1783 string, it behaves in the same way, unless a different locale has been
1784 set for the pattern (using the locale modifier). In this case, the
1785 isprint() function is used to distinguish printing and non-printing
1786 characters.
1787
1789
1790 It is possible to save compiled patterns on disc or elsewhere, and
1791 reload them later, subject to a number of restrictions. JIT data cannot
1792 be saved. The host on which the patterns are reloaded must be running
1793 the same version of PCRE2, with the same code unit width, and must also
1794 have the same endianness, pointer width and PCRE2_SIZE type. Before
1795 compiled patterns can be saved they must be serialized, that is, con‐
1796 verted to a stream of bytes. A single byte stream may contain any num‐
1797 ber of compiled patterns, but they must all use the same character
1798 tables. A single copy of the tables is included in the byte stream (its
1799 size is 1088 bytes).
1800
1801 The functions whose names begin with pcre2_serialize_ are used for
1802 serializing and de-serializing. They are described in the pcre2serial‐
1803 ize documentation. In this section we describe the features of
1804 pcre2test that can be used to test these functions.
1805
1806 Note that "serialization" in PCRE2 does not convert compiled patterns
1807 to an abstract format like Java or .NET. It just makes a reloadable
1808 byte code stream. Hence the restrictions on reloading mentioned above.
1809
1810 In pcre2test, when a pattern with push modifier is successfully com‐
1811 piled, it is pushed onto a stack of compiled patterns, and pcre2test
1812 expects the next line to contain a new pattern (or command) instead of
1813 a subject line. By contrast, the pushcopy modifier causes a copy of the
1814 compiled pattern to be stacked, leaving the original available for
1815 immediate matching. By using push and/or pushcopy, a number of patterns
1816 can be compiled and retained. These modifiers are incompatible with
1817 posix, and control modifiers that act at match time are ignored (with a
1818 message) for the stacked patterns. The jitverify modifier applies only
1819 at compile time.
1820
1821 The command
1822
1823 #save <filename>
1824
1825 causes all the stacked patterns to be serialized and the result written
1826 to the named file. Afterwards, all the stacked patterns are freed. The
1827 command
1828
1829 #load <filename>
1830
1831 reads the data in the file, and then arranges for it to be de-serial‐
1832 ized, with the resulting compiled patterns added to the pattern stack.
1833 The pattern on the top of the stack can be retrieved by the #pop com‐
1834 mand, which must be followed by lines of subjects that are to be
1835 matched with the pattern, terminated as usual by an empty line or end
1836 of file. This command may be followed by a modifier list containing
1837 only control modifiers that act after a pattern has been compiled. In
1838 particular, hex, posix, posix_nosub, push, and pushcopy are not
1839 allowed, nor are any option-setting modifiers. The JIT modifiers are,
1840 however permitted. Here is an example that saves and reloads two pat‐
1841 terns.
1842
1843 /abc/push
1844 /xyz/push
1845 #save tempfile
1846 #load tempfile
1847 #pop info
1848 xyz
1849
1850 #pop jit,bincode
1851 abc
1852
1853 If jitverify is used with #pop, it does not automatically imply jit,
1854 which is different behaviour from when it is used on a pattern.
1855
1856 The #popcopy command is analagous to the pushcopy modifier in that it
1857 makes current a copy of the topmost stack pattern, leaving the original
1858 still on the stack.
1859
1861
1862 pcre2(3), pcre2api(3), pcre2callout(3), pcre2jit, pcre2matching(3),
1863 pcre2partial(d), pcre2pattern(3), pcre2serialize(3).
1864
1866
1867 Philip Hazel
1868 University Computing Service
1869 Cambridge, England.
1870
1872
1873 Last updated: 11 March 2019
1874 Copyright (c) 1997-2019 University of Cambridge.
1875
1876
1877
1878PCRE 10.33 11 March 2019 PCRE2TEST(1)