1PCRE2TEST(1) General Commands Manual PCRE2TEST(1)
2
6 pcre2test - a program for testing Perl-compatible regular expressions.
7
9 pcre2test [options] [input file [output file]]
11 pcre2test is a test program for the PCRE2 regular expression libraries,
13 but it can also be used for experimenting with regular expressions.
14 This document describes the features of the test program; for details
15 of the regular expressions themselves, see the pcre2pattern documenta‐
16 tion. For details of the PCRE2 library function calls and their op‐
17 tions, see the pcre2api documentation.
18 The input for pcre2test is a sequence of regular expression patterns
20 and subject strings to be matched. There are also command lines for
21 setting defaults and controlling some special actions. The output shows
22 the result of each match attempt. Modifiers on external or internal
23 command lines, the patterns, and the subject lines specify PCRE2 func‐
24 tion options, control how the subject is processed, and what output is
25 produced.
26 There are many obscure modifiers, some of which are specifically de‐
28 signed for use in conjunction with the test script and data files that
29 are distributed as part of PCRE2. All the modifiers are documented
30 here, some without much justification, but many of them are unlikely to
31 be of use except when testing the libraries.
32
34 Different versions of the PCRE2 library can be built to support charac‐
36 ter strings that are encoded in 8-bit, 16-bit, or 32-bit code units.
37 One, two, or all three of these libraries may be simultaneously in‐
38 stalled. The pcre2test program can be used to test all the libraries.
39 However, its own input and output are always in 8-bit format. When
40 testing the 16-bit or 32-bit libraries, patterns and subject strings
41 are converted to 16-bit or 32-bit format before being passed to the li‐
42 brary functions. Results are converted back to 8-bit code units for
43 output.
44 In the rest of this document, the names of library functions and struc‐
46 tures are given in generic form, for example, pcre2_compile(). The ac‐
47 tual names used in the libraries have a suffix _8, _16, or _32, as ap‐
48 propriate.
49
51 Input to pcre2test is processed line by line, either by calling the C
53 library's fgets() function, or via the libreadline or libedit library.
54 In some Windows environments character 26 (hex 1A) causes an immediate
55 end of file, and no further data is read, so this character should be
56 avoided unless you really want that action.
57 The input is processed using using C's string functions, so must not
59 contain binary zeros, even though in Unix-like environments, fgets()
60 treats any bytes other than newline as data characters. An error is
61 generated if a binary zero is encountered. By default subject lines are
62 processed for backslash escapes, which makes it possible to include any
63 data value in strings that are passed to the library for matching. For
64 patterns, there is a facility for specifying some or all of the 8-bit
65 input characters as hexadecimal pairs, which makes it possible to in‐
66 clude binary zeros.
67 Input for the 16-bit and 32-bit libraries
69 When testing the 16-bit or 32-bit libraries, there is a need to be able
71 to generate character code points greater than 255 in the strings that
72 are passed to the library. For subject lines, backslash escapes can be
73 used. In addition, when the utf modifier (see "Setting compilation op‐
74 tions" below) is set, the pattern and any following subject lines are
75 interpreted as UTF-8 strings and translated to UTF-16 or UTF-32 as ap‐
76 propriate.
77 For non-UTF testing of wide characters, the utf8_input modifier can be
79 used. This is mutually exclusive with utf, and is allowed only in
80 16-bit or 32-bit mode. It causes the pattern and following subject
81 lines to be treated as UTF-8 according to the original definition (RFC
82 2279), which allows for character values up to 0x7fffffff. Each charac‐
83 ter is placed in one 16-bit or 32-bit code unit (in the 16-bit case,
84 values greater than 0xffff cause an error to occur).
85 UTF-8 (in its original definition) is not capable of encoding values
87 greater than 0x7fffffff, but such values can be handled by the 32-bit
88 library. When testing this library in non-UTF mode with utf8_input set,
89 if any character is preceded by the byte 0xff (which is an invalid byte
90 in UTF-8) 0x80000000 is added to the character's value. This is the
91 only way of passing such code points in a pattern string. For subject
92 strings, using an escape sequence is preferable.
93
95 -8 If the 8-bit library has been built, this option causes it to
97 be used (this is the default). If the 8-bit library has not
98 been built, this option causes an error.
99 -16 If the 16-bit library has been built, this option causes it
101 to be used. If only the 16-bit library has been built, this
102 is the default. If the 16-bit library has not been built,
103 this option causes an error.
104 -32 If the 32-bit library has been built, this option causes it
106 to be used. If only the 32-bit library has been built, this
107 is the default. If the 32-bit library has not been built,
108 this option causes an error.
109 -ac Behave as if each pattern has the auto_callout modifier, that
111 is, insert automatic callouts into every pattern that is com‐
112 piled.
113 -AC As for -ac, but in addition behave as if each subject line
115 has the callout_extra modifier, that is, show additional in‐
116 formation from callouts.
117 -b Behave as if each pattern has the fullbincode modifier; the
119 full internal binary form of the pattern is output after com‐
120 pilation.
121 -C Output the version number of the PCRE2 library, and all
123 available information about the optional features that are
124 included, and then exit with zero exit code. All other op‐
125 tions are ignored. If both -C and -LM are present, whichever
126 is first is recognized.
127 -C option Output information about a specific build-time option, then
129 exit. This functionality is intended for use in scripts such
130 as RunTest. The following options output the value and set
131 the exit code as indicated:
132 ebcdic-nl the code for LF (= NL) in an EBCDIC environment:
134 0x15 or 0x25
135 0 if used in an ASCII environment
136 exit code is always 0
137 linksize the configured internal link size (2, 3, or 4)
138 exit code is set to the link size
139 newline the default newline setting:
140 CR, LF, CRLF, ANYCRLF, ANY, or NUL
141 exit code is always 0
142 bsr the default setting for what \R matches:
143 ANYCRLF or ANY
144 exit code is always 0
145 The following options output 1 for true or 0 for false, and
147 set the exit code to the same value:
148 backslash-C \C is supported (not locked out)
150 ebcdic compiled for an EBCDIC environment
151 jit just-in-time support is available
152 pcre2-16 the 16-bit library was built
153 pcre2-32 the 32-bit library was built
154 pcre2-8 the 8-bit library was built
155 unicode Unicode support is available
156 If an unknown option is given, an error message is output;
158 the exit code is 0.
159 -d Behave as if each pattern has the debug modifier; the inter‐
161 nal form and information about the compiled pattern is output
162 after compilation; -d is equivalent to -b -i.
163 -dfa Behave as if each subject line has the dfa modifier; matching
165 is done using the pcre2_dfa_match() function instead of the
166 default pcre2_match().
167 -error number[,number,...]
169 Call pcre2_get_error_message() for each of the error numbers
170 in the comma-separated list, display the resulting messages
171 on the standard output, then exit with zero exit code. The
172 numbers may be positive or negative. This is a convenience
173 facility for PCRE2 maintainers.
174 -help Output a brief summary these options and then exit.
176 -i Behave as if each pattern has the info modifier; information
178 about the compiled pattern is given after compilation.
179 -jit Behave as if each pattern line has the jit modifier; after
181 successful compilation, each pattern is passed to the just-
182 in-time compiler, if available.
183 -jitfast Behave as if each pattern line has the jitfast modifier; af‐
185 ter successful compilation, each pattern is passed to the
186 just-in-time compiler, if available, and each subject line is
187 passed directly to the JIT matcher via its "fast path".
188 -jitverify
190 Behave as if each pattern line has the jitverify modifier;
191 after successful compilation, each pattern is passed to the
192 just-in-time compiler, if available, and the use of JIT for
193 matching is verified.
194 -LM List modifiers: write a list of available pattern and subject
196 modifiers to the standard output, then exit with zero exit
197 code. All other options are ignored. If both -C and any -Lx
198 options are present, whichever is first is recognized.
199 -LP List properties: write a list of recognized Unicode proper‐
201 ties to the standard output, then exit with zero exit code.
202 All other options are ignored. If both -C and any -Lx options
203 are present, whichever is first is recognized.
204 -LS List scripts: write a list of recognized Unicode script names
206 to the standard output, then exit with zero exit code. All
207 other options are ignored. If both -C and any -Lx options are
208 present, whichever is first is recognized.
209 -pattern modifier-list
211 Behave as if each pattern line contains the given modifiers.
212 -q Do not output the version number of pcre2test at the start of
214 execution.
215 -S size On Unix-like systems, set the size of the run-time stack to
217 size mebibytes (units of 1024*1024 bytes).
218 -subject modifier-list
220 Behave as if each subject line contains the given modifiers.
221 -t Run each compile and match many times with a timer, and out‐
223 put the resulting times per compile or match. When JIT is
224 used, separate times are given for the initial compile and
225 the JIT compile. You can control the number of iterations
226 that are used for timing by following -t with a number (as a
227 separate item on the command line). For example, "-t 1000"
228 iterates 1000 times. The default is to iterate 500,000 times.
229 -tm This is like -t except that it times only the matching phase,
231 not the compile phase.
232 -T -TM These behave like -t and -tm, but in addition, at the end of
234 a run, the total times for all compiles and matches are out‐
235 put.
236 -version Output the PCRE2 version number and then exit.
238
240 If pcre2test is given two filename arguments, it reads from the first
242 and writes to the second. If the first name is "-", input is taken from
243 the standard input. If pcre2test is given only one argument, it reads
244 from that file and writes to stdout. Otherwise, it reads from stdin and
245 writes to stdout.
246 When pcre2test is built, a configuration option can specify that it
248 should be linked with the libreadline or libedit library. When this is
249 done, if the input is from a terminal, it is read using the readline()
250 function. This provides line-editing and history facilities. The output
251 from the -help option states whether or not readline() will be used.
252 The program handles any number of tests, each of which consists of a
254 set of input lines. Each set starts with a regular expression pattern,
255 followed by any number of subject lines to be matched against that pat‐
256 tern. In between sets of test data, command lines that begin with # may
257 appear. This file format, with some restrictions, can also be processed
258 by the perltest.sh script that is distributed with PCRE2 as a means of
259 checking that the behaviour of PCRE2 and Perl is the same. For a speci‐
260 fication of perltest.sh, see the comments near its beginning. See also
261 the #perltest command below.
262 When the input is a terminal, pcre2test prompts for each line of input,
264 using "re>" to prompt for regular expression patterns, and "data>" to
265 prompt for subject lines. Command lines starting with # can be entered
266 only in response to the "re>" prompt.
267 Each subject line is matched separately and independently. If you want
269 to do multi-line matches, you have to use the \n escape sequence (or \r
270 or \r\n, etc., depending on the newline setting) in a single line of
271 input to encode the newline sequences. There is no limit on the length
272 of subject lines; the input buffer is automatically extended if it is
273 too small. There are replication features that makes it possible to
274 generate long repetitive pattern or subject lines without having to
275 supply them explicitly.
276 An empty line or the end of the file signals the end of the subject
278 lines for a test, at which point a new pattern or command line is ex‐
279 pected if there is still input to be read.
280
282 In between sets of test data, a line that begins with # is interpreted
284 as a command line. If the first character is followed by white space or
285 an exclamation mark, the line is treated as a comment, and ignored.
286 Otherwise, the following commands are recognized:
287 #forbid_utf
289 Subsequent patterns automatically have the PCRE2_NEVER_UTF and
291 PCRE2_NEVER_UCP options set, which locks out the use of the PCRE2_UTF
292 and PCRE2_UCP options and the use of (*UTF) and (*UCP) at the start of
293 patterns. This command also forces an error if a subsequent pattern
294 contains any occurrences of \P, \p, or \X, which are still supported
295 when PCRE2_UTF is not set, but which require Unicode property support
296 to be included in the library.
297 This is a trigger guard that is used in test files to ensure that UTF
299 or Unicode property tests are not accidentally added to files that are
300 used when Unicode support is not included in the library. Setting
301 PCRE2_NEVER_UTF and PCRE2_NEVER_UCP as a default can also be obtained
302 by the use of #pattern; the difference is that #forbid_utf cannot be
303 unset, and the automatic options are not displayed in pattern informa‐
304 tion, to avoid cluttering up test output.
305 #load <filename>
307 This command is used to load a set of precompiled patterns from a file,
309 as described in the section entitled "Saving and restoring compiled
310 patterns" below.
311 #loadtables <filename>
313 This command is used to load a set of binary character tables that can
315 be accessed by the tables=3 qualifier. Such tables can be created by
316 the pcre2_dftables program with the -b option.
317 #newline_default [<newline-list>]
319 When PCRE2 is built, a default newline convention can be specified.
321 This determines which characters and/or character pairs are recognized
322 as indicating a newline in a pattern or subject string. The default can
323 be overridden when a pattern is compiled. The standard test files con‐
324 tain tests of various newline conventions, but the majority of the
325 tests expect a single linefeed to be recognized as a newline by de‐
326 fault. Without special action the tests would fail when PCRE2 is com‐
327 piled with either CR or CRLF as the default newline.
328 The #newline_default command specifies a list of newline types that are
330 acceptable as the default. The types must be one of CR, LF, CRLF, ANY‐
331 CRLF, ANY, or NUL (in upper or lower case), for example:
332 #newline_default LF Any anyCRLF
334 If the default newline is in the list, this command has no effect. Oth‐
336 erwise, except when testing the POSIX API, a newline modifier that
337 specifies the first newline convention in the list (LF in the above ex‐
338 ample) is added to any pattern that does not already have a newline
339 modifier. If the newline list is empty, the feature is turned off. This
340 command is present in a number of the standard test input files.
341 When the POSIX API is being tested there is no way to override the de‐
343 fault newline convention, though it is possible to set the newline con‐
344 vention from within the pattern. A warning is given if the posix or
345 posix_nosub modifier is used when #newline_default would set a default
346 for the non-POSIX API.
347 #pattern <modifier-list>
349 This command sets a default modifier list that applies to all subse‐
351 quent patterns. Modifiers on a pattern can change these settings.
352 #perltest
354 This line is used in test files that can also be processed by perl‐
356 test.sh to confirm that Perl gives the same results as PCRE2. Subse‐
357 quent tests are checked for the use of pcre2test features that are in‐
358 compatible with the perltest.sh script.
359 Patterns must use '/' as their delimiter, and only certain modifiers
361 are supported. Comment lines, #pattern commands, and #subject commands
362 that set or unset "mark" are recognized and acted on. The #perltest,
363 #forbid_utf, and #newline_default commands, which are needed in the
364 relevant pcre2test files, are silently ignored. All other command lines
365 are ignored, but give a warning message. The #perltest command helps
366 detect tests that are accidentally put in the wrong file or use the
367 wrong delimiter. For more details of the perltest.sh script see the
368 comments it contains.
369 #pop [<modifiers>]
371 #popcopy [<modifiers>]
372 These commands are used to manipulate the stack of compiled patterns,
374 as described in the section entitled "Saving and restoring compiled
375 patterns" below.
376 #save <filename>
378 This command is used to save a set of compiled patterns to a file, as
380 described in the section entitled "Saving and restoring compiled pat‐
381 terns" below.
382 #subject <modifier-list>
384 This command sets a default modifier list that applies to all subse‐
386 quent subject lines. Modifiers on a subject line can change these set‐
387 tings.
388
390 Modifier lists are used with both pattern and subject lines. Items in a
392 list are separated by commas followed by optional white space. Trailing
393 whitespace in a modifier list is ignored. Some modifiers may be given
394 for both patterns and subject lines, whereas others are valid only for
395 one or the other. Each modifier has a long name, for example "an‐
396 chored", and some of them must be followed by an equals sign and a
397 value, for example, "offset=12". Values cannot contain comma charac‐
398 ters, but may contain spaces. Modifiers that do not take values may be
399 preceded by a minus sign to turn off a previous setting.
400 A few of the more common modifiers can also be specified as single let‐
402 ters, for example "i" for "caseless". In documentation, following the
403 Perl convention, these are written with a slash ("the /i modifier") for
404 clarity. Abbreviated modifiers must all be concatenated in the first
405 item of a modifier list. If the first item is not recognized as a long
406 modifier name, it is interpreted as a sequence of these abbreviations.
407 For example:
408 /abc/ig,newline=cr,jit=3
410 This is a pattern line whose modifier list starts with two one-letter
412 modifiers (/i and /g). The lower-case abbreviated modifiers are the
413 same as used in Perl.
414
416 A pattern line must start with one of the following characters (common
418 symbols, excluding pattern meta-characters):
419 / ! " ' ` - = _ : ; , % & @ ~
421 This is interpreted as the pattern's delimiter. A regular expression
423 may be continued over several input lines, in which case the newline
424 characters are included within it. It is possible to include the delim‐
425 iter as a literal within the pattern by escaping it with a backslash,
426 for example
427 /abc\/def/
429 If you do this, the escape and the delimiter form part of the pattern,
431 but since the delimiters are all non-alphanumeric, the inclusion of the
432 backslash does not affect the pattern's interpretation. Note, however,
433 that this trick does not work within \Q...\E literal bracketing because
434 the backslash will itself be interpreted as a literal. If the terminat‐
435 ing delimiter is immediately followed by a backslash, for example,
436 /abc/\
438 then a backslash is added to the end of the pattern. This is done to
440 provide a way of testing the error condition that arises if a pattern
441 finishes with a backslash, because
442 /abc\/
444 is interpreted as the first line of a pattern that starts with "abc/",
446 causing pcre2test to read the next line as a continuation of the regu‐
447 lar expression.
448 A pattern can be followed by a modifier list (details below).
450
452 Before each subject line is passed to pcre2_match(), pcre2_dfa_match(),
454 or pcre2_jit_match(), leading and trailing white space is removed, and
455 the line is scanned for backslash escapes, unless the subject_literal
456 modifier was set for the pattern. The following provide a means of en‐
457 coding non-printing characters in a visible way:
458 \a alarm (BEL, \x07)
460 \b backspace (\x08)
461 \e escape (\x27)
462 \f form feed (\x0c)
463 \n newline (\x0a)
464 \r carriage return (\x0d)
465 \t tab (\x09)
466 \v vertical tab (\x0b)
467 \nnn octal character (up to 3 octal digits); always
468 a byte unless > 255 in UTF-8 or 16-bit or 32-bit mode
469 \o{dd...} octal character (any number of octal digits}
470 \xhh hexadecimal byte (up to 2 hex digits)
471 \x{hh...} hexadecimal character (any number of hex digits)
472 The use of \x{hh...} is not dependent on the use of the utf modifier on
474 the pattern. It is recognized always. There may be any number of hexa‐
475 decimal digits inside the braces; invalid values provoke error mes‐
476 sages.
477 Note that \xhh specifies one byte rather than one character in UTF-8
479 mode; this makes it possible to construct invalid UTF-8 sequences for
480 testing purposes. On the other hand, \x{hh} is interpreted as a UTF-8
481 character in UTF-8 mode, generating more than one byte if the value is
482 greater than 127. When testing the 8-bit library not in UTF-8 mode,
483 \x{hh} generates one byte for values less than 256, and causes an error
484 for greater values.
485 In UTF-16 mode, all 4-digit \x{hhhh} values are accepted. This makes it
487 possible to construct invalid UTF-16 sequences for testing purposes.
488 In UTF-32 mode, all 4- to 8-digit \x{...} values are accepted. This
490 makes it possible to construct invalid UTF-32 sequences for testing
491 purposes.
492 There is a special backslash sequence that specifies replication of one
494 or more characters:
495 \[<characters>]{<count>}
497 This makes it possible to test long strings without having to provide
499 them as part of the file. For example:
500 \[abc]{4}
502 is converted to "abcabcabcabc". This feature does not support nesting.
504 To include a closing square bracket in the characters, code it as \x5D.
505 A backslash followed by an equals sign marks the end of the subject
507 string and the start of a modifier list. For example:
508 abc\=notbol,notempty
510 If the subject string is empty and \= is followed by whitespace, the
512 line is treated as a comment line, and is not used for matching. For
513 example:
514 \= This is a comment.
516 abc\= This is an invalid modifier list.
517 A backslash followed by any other non-alphanumeric character just es‐
519 capes that character. A backslash followed by anything else causes an
520 error. However, if the very last character in the line is a backslash
521 (and there is no modifier list), it is ignored. This gives a way of
522 passing an empty line as data, since a real empty line terminates the
523 data input.
524 If the subject_literal modifier is set for a pattern, all subject lines
526 that follow are treated as literals, with no special treatment of back‐
527 slashes. No replication is possible, and any subject modifiers must be
528 set as defaults by a #subject command.
529
531 There are several types of modifier that can appear in pattern lines.
533 Except where noted below, they may also be used in #pattern commands. A
534 pattern's modifier list can add to or override default modifiers that
535 were set by a previous #pattern command.
536 Setting compilation options
538 The following modifiers set options for pcre2_compile(). Most of them
540 set bits in the options argument of that function, but those whose
541 names start with PCRE2_EXTRA are additional options that are set in the
542 compile context. For the main options, there are some single-letter ab‐
543 breviations that are the same as Perl options. There is special han‐
544 dling for /x: if a second x is present, PCRE2_EXTENDED is converted
545 into PCRE2_EXTENDED_MORE as in Perl. A third appearance adds PCRE2_EX‐
546 TENDED as well, though this makes no difference to the way pcre2_com‐
547 pile() behaves. See pcre2api for a description of the effects of these
548 options.
549 allow_empty_class set PCRE2_ALLOW_EMPTY_CLASS
551 allow_lookaround_bsk set PCRE2_EXTRA_ALLOW_LOOKAROUND_BSK
552 allow_surrogate_escapes set PCRE2_EXTRA_ALLOW_SURROGATE_ESCAPES
553 alt_bsux set PCRE2_ALT_BSUX
554 alt_circumflex set PCRE2_ALT_CIRCUMFLEX
555 alt_verbnames set PCRE2_ALT_VERBNAMES
556 anchored set PCRE2_ANCHORED
557 auto_callout set PCRE2_AUTO_CALLOUT
558 bad_escape_is_literal set PCRE2_EXTRA_BAD_ESCAPE_IS_LITERAL
559 /i caseless set PCRE2_CASELESS
560 dollar_endonly set PCRE2_DOLLAR_ENDONLY
561 /s dotall set PCRE2_DOTALL
562 dupnames set PCRE2_DUPNAMES
563 endanchored set PCRE2_ENDANCHORED
564 escaped_cr_is_lf set PCRE2_EXTRA_ESCAPED_CR_IS_LF
565 /x extended set PCRE2_EXTENDED
566 /xx extended_more set PCRE2_EXTENDED_MORE
567 extra_alt_bsux set PCRE2_EXTRA_ALT_BSUX
568 firstline set PCRE2_FIRSTLINE
569 literal set PCRE2_LITERAL
570 match_line set PCRE2_EXTRA_MATCH_LINE
571 match_invalid_utf set PCRE2_MATCH_INVALID_UTF
572 match_unset_backref set PCRE2_MATCH_UNSET_BACKREF
573 match_word set PCRE2_EXTRA_MATCH_WORD
574 /m multiline set PCRE2_MULTILINE
575 never_backslash_c set PCRE2_NEVER_BACKSLASH_C
576 never_ucp set PCRE2_NEVER_UCP
577 never_utf set PCRE2_NEVER_UTF
578 /n no_auto_capture set PCRE2_NO_AUTO_CAPTURE
579 no_auto_possess set PCRE2_NO_AUTO_POSSESS
580 no_dotstar_anchor set PCRE2_NO_DOTSTAR_ANCHOR
581 no_start_optimize set PCRE2_NO_START_OPTIMIZE
582 no_utf_check set PCRE2_NO_UTF_CHECK
583 ucp set PCRE2_UCP
584 ungreedy set PCRE2_UNGREEDY
585 use_offset_limit set PCRE2_USE_OFFSET_LIMIT
586 utf set PCRE2_UTF
587 As well as turning on the PCRE2_UTF option, the utf modifier causes all
589 non-printing characters in output strings to be printed using the
590 \x{hh...} notation. Otherwise, those less than 0x100 are output in hex
591 without the curly brackets. Setting utf in 16-bit or 32-bit mode also
592 causes pattern and subject strings to be translated to UTF-16 or
593 UTF-32, respectively, before being passed to library functions.
594 Setting compilation controls
596 The following modifiers affect the compilation process or request in‐
598 formation about the pattern. There are single-letter abbreviations for
599 some that are heavily used in the test files.
600 bsr=[anycrlf|unicode] specify \R handling
602 /B bincode show binary code without lengths
603 callout_info show callout information
604 convert=<options> request foreign pattern conversion
605 convert_glob_escape=c set glob escape character
606 convert_glob_separator=c set glob separator character
607 convert_length set convert buffer length
608 debug same as info,fullbincode
609 framesize show matching frame size
610 fullbincode show binary code with lengths
611 /I info show info about compiled pattern
612 hex unquoted characters are hexadecimal
613 jit[=<number>] use JIT
614 jitfast use JIT fast path
615 jitverify verify JIT use
616 locale=<name> use this locale
617 max_pattern_length=<n> set the maximum pattern length
618 memory show memory used
619 newline=<type> set newline type
620 null_context compile with a NULL context
621 parens_nest_limit=<n> set maximum parentheses depth
622 posix use the POSIX API
623 posix_nosub use the POSIX API with REG_NOSUB
624 push push compiled pattern onto the stack
625 pushcopy push a copy onto the stack
626 stackguard=<number> test the stackguard feature
627 subject_literal treat all subject lines as literal
628 tables=[0|1|2|3] select internal tables
629 use_length do not zero-terminate the pattern
630 utf8_input treat input as UTF-8
631 The effects of these modifiers are described in the following sections.
633 Newline and \R handling
635 The bsr modifier specifies what \R in a pattern should match. If it is
637 set to "anycrlf", \R matches CR, LF, or CRLF only. If it is set to
638 "unicode", \R matches any Unicode newline sequence. The default can be
639 specified when PCRE2 is built; if it is not, the default is set to Uni‐
640 code.
641 The newline modifier specifies which characters are to be interpreted
643 as newlines, both in the pattern and in subject lines. The type must be
644 one of CR, LF, CRLF, ANYCRLF, ANY, or NUL (in upper or lower case).
645 Information about a pattern
647 The debug modifier is a shorthand for info,fullbincode, requesting all
649 available information.
650 The bincode modifier causes a representation of the compiled code to be
652 output after compilation. This information does not contain length and
653 offset values, which ensures that the same output is generated for dif‐
654 ferent internal link sizes and different code unit widths. By using
655 bincode, the same regression tests can be used in different environ‐
656 ments.
657 The fullbincode modifier, by contrast, does include length and offset
659 values. This is used in a few special tests that run only for specific
660 code unit widths and link sizes, and is also useful for one-off tests.
661 The info modifier requests information about the compiled pattern
663 (whether it is anchored, has a fixed first character, and so on). The
664 information is obtained from the pcre2_pattern_info() function. Here
665 are some typical examples:
666 re> /(?i)(^a|^b)/m,info
668 Capture group count = 1
669 Compile options: multiline
670 Overall options: caseless multiline
671 First code unit at start or follows newline
672 Subject length lower bound = 1
673 re> /(?i)abc/info
675 Capture group count = 0
676 Compile options: <none>
677 Overall options: caseless
678 First code unit = 'a' (caseless)
679 Last code unit = 'c' (caseless)
680 Subject length lower bound = 3
681 "Compile options" are those specified by modifiers; "overall options"
683 have added options that are taken or deduced from the pattern. If both
684 sets of options are the same, just a single "options" line is output;
685 if there are no options, the line is omitted. "First code unit" is
686 where any match must start; if there is more than one they are listed
687 as "starting code units". "Last code unit" is the last literal code
688 unit that must be present in any match. This is not necessarily the
689 last character. These lines are omitted if no starting or ending code
690 units are recorded. The subject length line is omitted when
691 no_start_optimize is set because the minimum length is not calculated
692 when it can never be used.
693 The framesize modifier shows the size, in bytes, of the storage frames
695 used by pcre2_match() for handling backtracking. The size depends on
696 the number of capturing parentheses in the pattern.
697 The callout_info modifier requests information about all the callouts
699 in the pattern. A list of them is output at the end of any other infor‐
700 mation that is requested. For each callout, either its number or string
701 is given, followed by the item that follows it in the pattern.
702 Passing a NULL context
704 Normally, pcre2test passes a context block to pcre2_compile(). If the
706 null_context modifier is set, however, NULL is passed. This is for
707 testing that pcre2_compile() behaves correctly in this case (it uses
708 default values).
709 Specifying pattern characters in hexadecimal
711 The hex modifier specifies that the characters of the pattern, except
713 for substrings enclosed in single or double quotes, are to be inter‐
714 preted as pairs of hexadecimal digits. This feature is provided as a
715 way of creating patterns that contain binary zeros and other non-print‐
716 ing characters. White space is permitted between pairs of digits. For
717 example, this pattern contains three characters:
718 /ab 32 59/hex
720 Parts of such a pattern are taken literally if quoted. This pattern
722 contains nine characters, only two of which are specified in hexadeci‐
723 mal:
724 /ab "literal" 32/hex
726 Either single or double quotes may be used. There is no way of includ‐
728 ing the delimiter within a substring. The hex and expand modifiers are
729 mutually exclusive.
730 Specifying the pattern's length
732 By default, patterns are passed to the compiling functions as zero-ter‐
734 minated strings but can be passed by length instead of being zero-ter‐
735 minated. The use_length modifier causes this to happen. Using a length
736 happens automatically (whether or not use_length is set) when hex is
737 set, because patterns specified in hexadecimal may contain binary ze‐
738 ros.
739 If hex or use_length is used with the POSIX wrapper API (see "Using the
741 POSIX wrapper API" below), the REG_PEND extension is used to pass the
742 pattern's length.
743 Specifying wide characters in 16-bit and 32-bit modes
745 In 16-bit and 32-bit modes, all input is automatically treated as UTF-8
747 and translated to UTF-16 or UTF-32 when the utf modifier is set. For
748 testing the 16-bit and 32-bit libraries in non-UTF mode, the utf8_input
749 modifier can be used. It is mutually exclusive with utf. Input lines
750 are interpreted as UTF-8 as a means of specifying wide characters. More
751 details are given in "Input encoding" above.
752 Generating long repetitive patterns
754 Some tests use long patterns that are very repetitive. Instead of cre‐
756 ating a very long input line for such a pattern, you can use a special
757 repetition feature, similar to the one described for subject lines
758 above. If the expand modifier is present on a pattern, parts of the
759 pattern that have the form
760 \[<characters>]{<count>}
762 are expanded before the pattern is passed to pcre2_compile(). For exam‐
764 ple, \[AB]{6000} is expanded to "ABAB..." 6000 times. This construction
765 cannot be nested. An initial "\[" sequence is recognized only if "]{"
766 followed by decimal digits and "}" is found later in the pattern. If
767 not, the characters remain in the pattern unaltered. The expand and hex
768 modifiers are mutually exclusive.
769 If part of an expanded pattern looks like an expansion, but is really
771 part of the actual pattern, unwanted expansion can be avoided by giving
772 two values in the quantifier. For example, \[AB]{6000,6000} is not rec‐
773 ognized as an expansion item.
774 If the info modifier is set on an expanded pattern, the result of the
776 expansion is included in the information that is output.
777 JIT compilation
779 Just-in-time (JIT) compiling is a heavyweight optimization that can
781 greatly speed up pattern matching. See the pcre2jit documentation for
782 details. JIT compiling happens, optionally, after a pattern has been
783 successfully compiled into an internal form. The JIT compiler converts
784 this to optimized machine code. It needs to know whether the match-time
785 options PCRE2_PARTIAL_HARD and PCRE2_PARTIAL_SOFT are going to be used,
786 because different code is generated for the different cases. See the
787 partial modifier in "Subject Modifiers" below for details of how these
788 options are specified for each match attempt.
789 JIT compilation is requested by the jit pattern modifier, which may op‐
791 tionally be followed by an equals sign and a number in the range 0 to
792 7. The three bits that make up the number specify which of the three
793 JIT operating modes are to be compiled:
794 1 compile JIT code for non-partial matching
796 2 compile JIT code for soft partial matching
797 4 compile JIT code for hard partial matching
798 The possible values for the jit modifier are therefore:
800 0 disable JIT
802 1 normal matching only
803 2 soft partial matching only
804 3 normal and soft partial matching
805 4 hard partial matching only
806 6 soft and hard partial matching only
807 7 all three modes
808 If no number is given, 7 is assumed. The phrase "partial matching"
810 means a call to pcre2_match() with either the PCRE2_PARTIAL_SOFT or the
811 PCRE2_PARTIAL_HARD option set. Note that such a call may return a com‐
812 plete match; the options enable the possibility of a partial match, but
813 do not require it. Note also that if you request JIT compilation only
814 for partial matching (for example, jit=2) but do not set the partial
815 modifier on a subject line, that match will not use JIT code because
816 none was compiled for non-partial matching.
817 If JIT compilation is successful, the compiled JIT code will automati‐
819 cally be used when an appropriate type of match is run, except when in‐
820 compatible run-time options are specified. For more details, see the
821 pcre2jit documentation. See also the jitstack modifier below for a way
822 of setting the size of the JIT stack.
823 If the jitfast modifier is specified, matching is done using the JIT
825 "fast path" interface, pcre2_jit_match(), which skips some of the san‐
826 ity checks that are done by pcre2_match(), and of course does not work
827 when JIT is not supported. If jitfast is specified without jit, jit=7
828 is assumed.
829 If the jitverify modifier is specified, information about the compiled
831 pattern shows whether JIT compilation was or was not successful. If
832 jitverify is specified without jit, jit=7 is assumed. If JIT compila‐
833 tion is successful when jitverify is set, the text "(JIT)" is added to
834 the first output line after a match or non match when JIT-compiled code
835 was actually used in the match.
836 Setting a locale
838 The locale modifier must specify the name of a locale, for example:
840 /pattern/locale=fr_FR
842 The given locale is set, pcre2_maketables() is called to build a set of
844 character tables for the locale, and this is then passed to pcre2_com‐
845 pile() when compiling the regular expression. The same tables are used
846 when matching the following subject lines. The locale modifier applies
847 only to the pattern on which it appears, but can be given in a #pattern
848 command if a default is needed. Setting a locale and alternate charac‐
849 ter tables are mutually exclusive.
850 Showing pattern memory
852 The memory modifier causes the size in bytes of the memory used to hold
854 the compiled pattern to be output. This does not include the size of
855 the pcre2_code block; it is just the actual compiled data. If the pat‐
856 tern is subsequently passed to the JIT compiler, the size of the JIT
857 compiled code is also output. Here is an example:
858 re> /a(b)c/jit,memory
860 Memory allocation (code space): 21
861 Memory allocation (JIT code): 1910
862 Limiting nested parentheses
865 The parens_nest_limit modifier sets a limit on the depth of nested
867 parentheses in a pattern. Breaching the limit causes a compilation er‐
868 ror. The default for the library is set when PCRE2 is built, but
869 pcre2test sets its own default of 220, which is required for running
870 the standard test suite.
871 Limiting the pattern length
873 The max_pattern_length modifier sets a limit, in code units, to the
875 length of pattern that pcre2_compile() will accept. Breaching the limit
876 causes a compilation error. The default is the largest number a
877 PCRE2_SIZE variable can hold (essentially unlimited).
878 Using the POSIX wrapper API
880 The posix and posix_nosub modifiers cause pcre2test to call PCRE2 via
882 the POSIX wrapper API rather than its native API. When posix_nosub is
883 used, the POSIX option REG_NOSUB is passed to regcomp(). The POSIX
884 wrapper supports only the 8-bit library. Note that it does not imply
885 POSIX matching semantics; for more detail see the pcre2posix documenta‐
886 tion. The following pattern modifiers set options for the regcomp()
887 function:
888 caseless REG_ICASE
890 multiline REG_NEWLINE
891 dotall REG_DOTALL )
892 ungreedy REG_UNGREEDY ) These options are not part of
893 ucp REG_UCP ) the POSIX standard
894 utf REG_UTF8 )
895 The regerror_buffsize modifier specifies a size for the error buffer
897 that is passed to regerror() in the event of a compilation error. For
898 example:
899 /abc/posix,regerror_buffsize=20
901 This provides a means of testing the behaviour of regerror() when the
903 buffer is too small for the error message. If this modifier has not
904 been set, a large buffer is used.
905 The aftertext and allaftertext subject modifiers work as described be‐
907 low. All other modifiers are either ignored, with a warning message, or
908 cause an error.
909 The pattern is passed to regcomp() as a zero-terminated string by de‐
911 fault, but if the use_length or hex modifiers are set, the REG_PEND ex‐
912 tension is used to pass it by length.
913 Testing the stack guard feature
915 The stackguard modifier is used to test the use of pcre2_set_com‐
917 pile_recursion_guard(), a function that is provided to enable stack
918 availability to be checked during compilation (see the pcre2api docu‐
919 mentation for details). If the number specified by the modifier is
920 greater than zero, pcre2_set_compile_recursion_guard() is called to set
921 up callback from pcre2_compile() to a local function. The argument it
922 receives is the current nesting parenthesis depth; if this is greater
923 than the value given by the modifier, non-zero is returned, causing the
924 compilation to be aborted.
925 Using alternative character tables
927 The value specified for the tables modifier must be one of the digits
929 0, 1, 2, or 3. It causes a specific set of built-in character tables to
930 be passed to pcre2_compile(). This is used in the PCRE2 tests to check
931 behaviour with different character tables. The digit specifies the ta‐
932 bles as follows:
933 0 do not pass any special character tables
935 1 the default ASCII tables, as distributed in
936 pcre2_chartables.c.dist
937 2 a set of tables defining ISO 8859 characters
938 3 a set of tables loaded by the #loadtables command
939 In tables 2, some characters whose codes are greater than 128 are iden‐
941 tified as letters, digits, spaces, etc. Tables 3 can be used only after
942 a #loadtables command has loaded them from a binary file. Setting al‐
943 ternate character tables and a locale are mutually exclusive.
944 Setting certain match controls
946 The following modifiers are really subject modifiers, and are described
948 under "Subject Modifiers" below. However, they may be included in a
949 pattern's modifier list, in which case they are applied to every sub‐
950 ject line that is processed with that pattern. These modifiers do not
951 affect the compilation process.
952 aftertext show text after match
954 allaftertext show text after captures
955 allcaptures show all captures
956 allvector show the entire ovector
957 allusedtext show all consulted text
958 altglobal alternative global matching
959 /g global global matching
960 jitstack=<n> set size of JIT stack
961 mark show mark values
962 replace=<string> specify a replacement string
963 startchar show starting character when relevant
964 substitute_callout use substitution callouts
965 substitute_extended use PCRE2_SUBSTITUTE_EXTENDED
966 substitute_literal use PCRE2_SUBSTITUTE_LITERAL
967 substitute_matched use PCRE2_SUBSTITUTE_MATCHED
968 substitute_overflow_length use PCRE2_SUBSTITUTE_OVERFLOW_LENGTH
969 substitute_replacement_only use PCRE2_SUBSTITUTE_REPLACEMENT_ONLY
970 substitute_skip=<n> skip substitution <n>
971 substitute_stop=<n> skip substitution <n> and following
972 substitute_unknown_unset use PCRE2_SUBSTITUTE_UNKNOWN_UNSET
973 substitute_unset_empty use PCRE2_SUBSTITUTE_UNSET_EMPTY
974 These modifiers may not appear in a #pattern command. If you want them
976 as defaults, set them in a #subject command.
977 Specifying literal subject lines
979 If the subject_literal modifier is present on a pattern, all the sub‐
981 ject lines that it matches are taken as literal strings, with no inter‐
982 pretation of backslashes. It is not possible to set subject modifiers
983 on such lines, but any that are set as defaults by a #subject command
984 are recognized.
985 Saving a compiled pattern
987 When a pattern with the push modifier is successfully compiled, it is
989 pushed onto a stack of compiled patterns, and pcre2test expects the
990 next line to contain a new pattern (or a command) instead of a subject
991 line. This facility is used when saving compiled patterns to a file, as
992 described in the section entitled "Saving and restoring compiled pat‐
993 terns" below. If pushcopy is used instead of push, a copy of the com‐
994 piled pattern is stacked, leaving the original as current, ready to
995 match the following input lines. This provides a way of testing the
996 pcre2_code_copy() function. The push and pushcopy modifiers are in‐
997 compatible with compilation modifiers such as global that act at match
998 time. Any that are specified are ignored (for the stacked copy), with a
999 warning message, except for replace, which causes an error. Note that
1000 jitverify, which is allowed, does not carry through to any subsequent
1001 matching that uses a stacked pattern.
1002 Testing foreign pattern conversion
1004 The experimental foreign pattern conversion functions in PCRE2 can be
1006 tested by setting the convert modifier. Its argument is a colon-sepa‐
1007 rated list of options, which set the equivalent option for the
1008 pcre2_pattern_convert() function:
1009 glob PCRE2_CONVERT_GLOB
1011 glob_no_starstar PCRE2_CONVERT_GLOB_NO_STARSTAR
1012 glob_no_wild_separator PCRE2_CONVERT_GLOB_NO_WILD_SEPARATOR
1013 posix_basic PCRE2_CONVERT_POSIX_BASIC
1014 posix_extended PCRE2_CONVERT_POSIX_EXTENDED
1015 unset Unset all options
1016 The "unset" value is useful for turning off a default that has been set
1018 by a #pattern command. When one of these options is set, the input pat‐
1019 tern is passed to pcre2_pattern_convert(). If the conversion is suc‐
1020 cessful, the result is reflected in the output and then passed to
1021 pcre2_compile(). The normal utf and no_utf_check options, if set, cause
1022 the PCRE2_CONVERT_UTF and PCRE2_CONVERT_NO_UTF_CHECK options to be
1023 passed to pcre2_pattern_convert().
1024 By default, the conversion function is allowed to allocate a buffer for
1026 its output. However, if the convert_length modifier is set to a value
1027 greater than zero, pcre2test passes a buffer of the given length. This
1028 makes it possible to test the length check.
1029 The convert_glob_escape and convert_glob_separator modifiers can be
1031 used to specify the escape and separator characters for glob process‐
1032 ing, overriding the defaults, which are operating-system dependent.
1033
1035 The modifiers that can appear in subject lines and the #subject command
1037 are of two types.
1038 Setting match options
1040 The following modifiers set options for pcre2_match() or
1042 pcre2_dfa_match(). See pcreapi for a description of their effects.
1043 anchored set PCRE2_ANCHORED
1045 endanchored set PCRE2_ENDANCHORED
1046 dfa_restart set PCRE2_DFA_RESTART
1047 dfa_shortest set PCRE2_DFA_SHORTEST
1048 no_jit set PCRE2_NO_JIT
1049 no_utf_check set PCRE2_NO_UTF_CHECK
1050 notbol set PCRE2_NOTBOL
1051 notempty set PCRE2_NOTEMPTY
1052 notempty_atstart set PCRE2_NOTEMPTY_ATSTART
1053 noteol set PCRE2_NOTEOL
1054 partial_hard (or ph) set PCRE2_PARTIAL_HARD
1055 partial_soft (or ps) set PCRE2_PARTIAL_SOFT
1056 The partial matching modifiers are provided with abbreviations because
1058 they appear frequently in tests.
1059 If the posix or posix_nosub modifier was present on the pattern, caus‐
1061 ing the POSIX wrapper API to be used, the only option-setting modifiers
1062 that have any effect are notbol, notempty, and noteol, causing REG_NOT‐
1063 BOL, REG_NOTEMPTY, and REG_NOTEOL, respectively, to be passed to
1064 regexec(). The other modifiers are ignored, with a warning message.
1065 There is one additional modifier that can be used with the POSIX wrap‐
1067 per. It is ignored (with a warning) if used for non-POSIX matching.
1068 posix_startend=<n>[:<m>]
1070 This causes the subject string to be passed to regexec() using the
1072 REG_STARTEND option, which uses offsets to specify which part of the
1073 string is searched. If only one number is given, the end offset is
1074 passed as the end of the subject string. For more detail of REG_STAR‐
1075 TEND, see the pcre2posix documentation. If the subject string contains
1076 binary zeros (coded as escapes such as \x{00} because pcre2test does
1077 not support actual binary zeros in its input), you must use posix_star‐
1078 tend to specify its length.
1079 Setting match controls
1081 The following modifiers affect the matching process or request addi‐
1083 tional information. Some of them may also be specified on a pattern
1084 line (see above), in which case they apply to every subject line that
1085 is matched against that pattern, but can be overridden by modifiers on
1086 the subject.
1087 aftertext show text after match
1089 allaftertext show text after captures
1090 allcaptures show all captures
1091 allvector show the entire ovector
1092 allusedtext show all consulted text (non-JIT only)
1093 altglobal alternative global matching
1094 callout_capture show captures at callout time
1095 callout_data=<n> set a value to pass via callouts
1096 callout_error=<n>[:<m>] control callout error
1097 callout_extra show extra callout information
1098 callout_fail=<n>[:<m>] control callout failure
1099 callout_no_where do not show position of a callout
1100 callout_none do not supply a callout function
1101 copy=<number or name> copy captured substring
1102 depth_limit=<n> set a depth limit
1103 dfa use pcre2_dfa_match()
1104 find_limits find heap, match and depth limits
1105 find_limits_noheap find match and depth limits
1106 get=<number or name> extract captured substring
1107 getall extract all captured substrings
1108 /g global global matching
1109 heap_limit=<n> set a limit on heap memory (Kbytes)
1110 jitstack=<n> set size of JIT stack
1111 mark show mark values
1112 match_limit=<n> set a match limit
1113 memory show heap memory usage
1114 null_context match with a NULL context
1115 null_replacement substitute with NULL replacement
1116 null_subject match with NULL subject
1117 offset=<n> set starting offset
1118 offset_limit=<n> set offset limit
1119 ovector=<n> set size of output vector
1120 recursion_limit=<n> obsolete synonym for depth_limit
1121 replace=<string> specify a replacement string
1122 startchar show startchar when relevant
1123 startoffset=<n> same as offset=<n>
1124 substitute_callout use substitution callouts
1125 substitute_extedded use PCRE2_SUBSTITUTE_EXTENDED
1126 substitute_literal use PCRE2_SUBSTITUTE_LITERAL
1127 substitute_matched use PCRE2_SUBSTITUTE_MATCHED
1128 substitute_overflow_length use PCRE2_SUBSTITUTE_OVERFLOW_LENGTH
1129 substitute_replacement_only use PCRE2_SUBSTITUTE_REPLACEMENT_ONLY
1130 substitute_skip=<n> skip substitution number n
1131 substitute_stop=<n> skip substitution number n and greater
1132 substitute_unknown_unset use PCRE2_SUBSTITUTE_UNKNOWN_UNSET
1133 substitute_unset_empty use PCRE2_SUBSTITUTE_UNSET_EMPTY
1134 zero_terminate pass the subject as zero-terminated
1135 The effects of these modifiers are described in the following sections.
1137 When matching via the POSIX wrapper API, the aftertext, allaftertext,
1138 and ovector subject modifiers work as described below. All other modi‐
1139 fiers are either ignored, with a warning message, or cause an error.
1140 Showing more text
1142 The aftertext modifier requests that as well as outputting the part of
1144 the subject string that matched the entire pattern, pcre2test should in
1145 addition output the remainder of the subject string. This is useful for
1146 tests where the subject contains multiple copies of the same substring.
1147 The allaftertext modifier requests the same action for captured sub‐
1148 strings as well as the main matched substring. In each case the remain‐
1149 der is output on the following line with a plus character following the
1150 capture number.
1151 The allusedtext modifier requests that all the text that was consulted
1153 during a successful pattern match by the interpreter should be shown,
1154 for both full and partial matches. This feature is not supported for
1155 JIT matching, and if requested with JIT it is ignored (with a warning
1156 message). Setting this modifier affects the output if there is a look‐
1157 behind at the start of a match, or, for a complete match, a lookahead
1158 at the end, or if \K is used in the pattern. Characters that precede or
1159 follow the start and end of the actual match are indicated in the out‐
1160 put by '<' or '>' characters underneath them. Here is an example:
1161 re> /(?<=pqr)abc(?=xyz)/
1163 data> 123pqrabcxyz456\=allusedtext
1164 0: pqrabcxyz
1165 <<< >>>
1166 data> 123pqrabcxy\=ph,allusedtext
1167 Partial match: pqrabcxy
1168 <<<
1169 The first, complete match shows that the matched string is "abc", with
1171 the preceding and following strings "pqr" and "xyz" having been con‐
1172 sulted during the match (when processing the assertions). The partial
1173 match can indicate only the preceding string.
1174 The startchar modifier requests that the starting character for the
1176 match be indicated, if it is different to the start of the matched
1177 string. The only time when this occurs is when \K has been processed as
1178 part of the match. In this situation, the output for the matched string
1179 is displayed from the starting character instead of from the match
1180 point, with circumflex characters under the earlier characters. For ex‐
1181 ample:
1182 re> /abc\Kxyz/
1184 data> abcxyz\=startchar
1185 0: abcxyz
1186 ^^^
1187 Unlike allusedtext, the startchar modifier can be used with JIT. How‐
1189 ever, these two modifiers are mutually exclusive.
1190 Showing the value of all capture groups
1192 The allcaptures modifier requests that the values of all potential cap‐
1194 tured parentheses be output after a match. By default, only those up to
1195 the highest one actually used in the match are output (corresponding to
1196 the return code from pcre2_match()). Groups that did not take part in
1197 the match are output as "<unset>". This modifier is not relevant for
1198 DFA matching (which does no capturing) and does not apply when replace
1199 is specified; it is ignored, with a warning message, if present.
1200 Showing the entire ovector, for all outcomes
1202 The allvector modifier requests that the entire ovector be shown, what‐
1204 ever the outcome of the match. Compare allcaptures, which shows only up
1205 to the maximum number of capture groups for the pattern, and then only
1206 for a successful complete non-DFA match. This modifier, which acts af‐
1207 ter any match result, and also for DFA matching, provides a means of
1208 checking that there are no unexpected modifications to ovector fields.
1209 Before each match attempt, the ovector is filled with a special value,
1210 and if this is found in both elements of a capturing pair, "<un‐
1211 changed>" is output. After a successful match, this applies to all
1212 groups after the maximum capture group for the pattern. In other cases
1213 it applies to the entire ovector. After a partial match, the first two
1214 elements are the only ones that should be set. After a DFA match, the
1215 amount of ovector that is used depends on the number of matches that
1216 were found.
1217 Testing pattern callouts
1219 A callout function is supplied when pcre2test calls the library match‐
1221 ing functions, unless callout_none is specified. Its behaviour can be
1222 controlled by various modifiers listed above whose names begin with
1223 callout_. Details are given in the section entitled "Callouts" below.
1224 Testing callouts from pcre2_substitute() is described separately in
1225 "Testing the substitution function" below.
1226 Finding all matches in a string
1228 Searching for all possible matches within a subject can be requested by
1230 the global or altglobal modifier. After finding a match, the matching
1231 function is called again to search the remainder of the subject. The
1232 difference between global and altglobal is that the former uses the
1233 start_offset argument to pcre2_match() or pcre2_dfa_match() to start
1234 searching at a new point within the entire string (which is what Perl
1235 does), whereas the latter passes over a shortened subject. This makes a
1236 difference to the matching process if the pattern begins with a lookbe‐
1237 hind assertion (including \b or \B).
1238 If an empty string is matched, the next match is done with the
1240 PCRE2_NOTEMPTY_ATSTART and PCRE2_ANCHORED flags set, in order to search
1241 for another, non-empty, match at the same point in the subject. If this
1242 match fails, the start offset is advanced, and the normal match is re‐
1243 tried. This imitates the way Perl handles such cases when using the /g
1244 modifier or the split() function. Normally, the start offset is ad‐
1245 vanced by one character, but if the newline convention recognizes CRLF
1246 as a newline, and the current character is CR followed by LF, an ad‐
1247 vance of two characters occurs.
1248 Testing substring extraction functions
1250 The copy and get modifiers can be used to test the pcre2_sub‐
1252 string_copy_xxx() and pcre2_substring_get_xxx() functions. They can be
1253 given more than once, and each can specify a capture group name or num‐
1254 ber, for example:
1255 abcd\=copy=1,copy=3,get=G1
1257 If the #subject command is used to set default copy and/or get lists,
1259 these can be unset by specifying a negative number to cancel all num‐
1260 bered groups and an empty name to cancel all named groups.
1261 The getall modifier tests pcre2_substring_list_get(), which extracts
1263 all captured substrings.
1264 If the subject line is successfully matched, the substrings extracted
1266 by the convenience functions are output with C, G, or L after the
1267 string number instead of a colon. This is in addition to the normal
1268 full list. The string length (that is, the return from the extraction
1269 function) is given in parentheses after each substring, followed by the
1270 name when the extraction was by name.
1271 Testing the substitution function
1273 If the replace modifier is set, the pcre2_substitute() function is
1275 called instead of one of the matching functions (or after one call of
1276 pcre2_match() in the case of PCRE2_SUBSTITUTE_MATCHED). Note that re‐
1277 placement strings cannot contain commas, because a comma signifies the
1278 end of a modifier. This is not thought to be an issue in a test pro‐
1279 gram.
1280 Specifying a completely empty replacement string disables this modi‐
1282 fier. However, it is possible to specify an empty replacement by pro‐
1283 viding a buffer length, as described below, for an otherwise empty re‐
1284 placement.
1285 Unlike subject strings, pcre2test does not process replacement strings
1287 for escape sequences. In UTF mode, a replacement string is checked to
1288 see if it is a valid UTF-8 string. If so, it is correctly converted to
1289 a UTF string of the appropriate code unit width. If it is not a valid
1290 UTF-8 string, the individual code units are copied directly. This pro‐
1291 vides a means of passing an invalid UTF-8 string for testing purposes.
1292 The following modifiers set options (in additional to the normal match
1294 options) for pcre2_substitute():
1295 global PCRE2_SUBSTITUTE_GLOBAL
1297 substitute_extended PCRE2_SUBSTITUTE_EXTENDED
1298 substitute_literal PCRE2_SUBSTITUTE_LITERAL
1299 substitute_matched PCRE2_SUBSTITUTE_MATCHED
1300 substitute_overflow_length PCRE2_SUBSTITUTE_OVERFLOW_LENGTH
1301 substitute_replacement_only PCRE2_SUBSTITUTE_REPLACEMENT_ONLY
1302 substitute_unknown_unset PCRE2_SUBSTITUTE_UNKNOWN_UNSET
1303 substitute_unset_empty PCRE2_SUBSTITUTE_UNSET_EMPTY
1304 See the pcre2api documentation for details of these options.
1306 After a successful substitution, the modified string is output, pre‐
1308 ceded by the number of replacements. This may be zero if there were no
1309 matches. Here is a simple example of a substitution test:
1310 /abc/replace=xxx
1312 =abc=abc=
1313 1: =xxx=abc=
1314 =abc=abc=\=global
1315 2: =xxx=xxx=
1316 Subject and replacement strings should be kept relatively short (fewer
1318 than 256 characters) for substitution tests, as fixed-size buffers are
1319 used. To make it easy to test for buffer overflow, if the replacement
1320 string starts with a number in square brackets, that number is passed
1321 to pcre2_substitute() as the size of the output buffer, with the re‐
1322 placement string starting at the next character. Here is an example
1323 that tests the edge case:
1324 /abc/
1326 123abc123\=replace=[10]XYZ
1327 1: 123XYZ123
1328 123abc123\=replace=[9]XYZ
1329 Failed: error -47: no more memory
1330 The default action of pcre2_substitute() is to return PCRE2_ER‐
1332 ROR_NOMEMORY when the output buffer is too small. However, if the
1333 PCRE2_SUBSTITUTE_OVERFLOW_LENGTH option is set (by using the substi‐
1334 tute_overflow_length modifier), pcre2_substitute() continues to go
1335 through the motions of matching and substituting (but not doing any
1336 callouts), in order to compute the size of buffer that is required.
1337 When this happens, pcre2test shows the required buffer length (which
1338 includes space for the trailing zero) as part of the error message. For
1339 example:
1340 /abc/substitute_overflow_length
1342 123abc123\=replace=[9]XYZ
1343 Failed: error -47: no more memory: 10 code units are needed
1344 A replacement string is ignored with POSIX and DFA matching. Specifying
1346 partial matching provokes an error return ("bad option value") from
1347 pcre2_substitute().
1348 Testing substitute callouts
1350 If the substitute_callout modifier is set, a substitution callout func‐
1352 tion is set up. The null_context modifier must not be set, because the
1353 address of the callout function is passed in a match context. When the
1354 callout function is called (after each substitution), details of the
1355 the input and output strings are output. For example:
1356 /abc/g,replace=<$0>,substitute_callout
1358 abcdefabcpqr
1359 1(1) Old 0 3 "abc" New 0 5 "<abc>"
1360 2(1) Old 6 9 "abc" New 8 13 "<abc>"
1361 2: <abc>def<abc>pqr
1362 The first number on each callout line is the count of matches. The
1364 parenthesized number is the number of pairs that are set in the ovector
1365 (that is, one more than the number of capturing groups that were set).
1366 Then are listed the offsets of the old substring, its contents, and the
1367 same for the replacement.
1368 By default, the substitution callout function returns zero, which ac‐
1370 cepts the replacement and causes matching to continue if /g was used.
1371 Two further modifiers can be used to test other return values. If sub‐
1372 stitute_skip is set to a value greater than zero the callout function
1373 returns +1 for the match of that number, and similarly substitute_stop
1374 returns -1. These cause the replacement to be rejected, and -1 causes
1375 no further matching to take place. If either of them are set, substi‐
1376 tute_callout is assumed. For example:
1377 /abc/g,replace=<$0>,substitute_skip=1
1379 abcdefabcpqr
1380 1(1) Old 0 3 "abc" New 0 5 "<abc> SKIPPED"
1381 2(1) Old 6 9 "abc" New 6 11 "<abc>"
1382 2: abcdef<abc>pqr
1383 abcdefabcpqr\=substitute_stop=1
1384 1(1) Old 0 3 "abc" New 0 5 "<abc> STOPPED"
1385 1: abcdefabcpqr
1386 If both are set for the same number, stop takes precedence. Only a sin‐
1388 gle skip or stop is supported, which is sufficient for testing that the
1389 feature works.
1390 Setting the JIT stack size
1392 The jitstack modifier provides a way of setting the maximum stack size
1394 that is used by the just-in-time optimization code. It is ignored if
1395 JIT optimization is not being used. The value is a number of kibibytes
1396 (units of 1024 bytes). Setting zero reverts to the default of 32KiB.
1397 Providing a stack that is larger than the default is necessary only for
1398 very complicated patterns. If jitstack is set non-zero on a subject
1399 line it overrides any value that was set on the pattern.
1400 Setting heap, match, and depth limits
1402 The heap_limit, match_limit, and depth_limit modifiers set the appro‐
1404 priate limits in the match context. These values are ignored when the
1405 find_limits or find_limits_noheap modifier is specified.
1406 Finding minimum limits
1408 If the find_limits modifier is present on a subject line, pcre2test
1410 calls the relevant matching function several times, setting different
1411 values in the match context via pcre2_set_heap_limit(),
1412 pcre2_set_match_limit(), or pcre2_set_depth_limit() until it finds the
1413 smallest value for each parameter that allows the match to complete
1414 without a "limit exceeded" error. The match itself may succeed or fail.
1415 An alternative modifier, find_limits_noheap, omits the heap limit. This
1416 is used in the standard tests, because the minimum heap limit varies
1417 between systems. If JIT is being used, only the match limit is rele‐
1418 vant, and the other two are automatically omitted.
1419 When using this modifier, the pattern should not contain any limit set‐
1421 tings such as (*LIMIT_MATCH=...) within it. If such a setting is
1422 present and is lower than the minimum matching value, the minimum value
1423 cannot be found because pcre2_set_match_limit() etc. are only able to
1424 reduce the value of an in-pattern limit; they cannot increase it.
1425 For non-DFA matching, the minimum depth_limit number is a measure of
1427 how much nested backtracking happens (that is, how deeply the pattern's
1428 tree is searched). In the case of DFA matching, depth_limit controls
1429 the depth of recursive calls of the internal function that is used for
1430 handling pattern recursion, lookaround assertions, and atomic groups.
1431 For non-DFA matching, the match_limit number is a measure of the amount
1433 of backtracking that takes place, and learning the minimum value can be
1434 instructive. For most simple matches, the number is quite small, but
1435 for patterns with very large numbers of matching possibilities, it can
1436 become large very quickly with increasing length of subject string. In
1437 the case of DFA matching, match_limit controls the total number of
1438 calls, both recursive and non-recursive, to the internal matching func‐
1439 tion, thus controlling the overall amount of computing resource that is
1440 used.
1441 For both kinds of matching, the heap_limit number, which is in
1443 kibibytes (units of 1024 bytes), limits the amount of heap memory used
1444 for matching.
1445 Showing MARK names
1447 The mark modifier causes the names from backtracking control verbs that
1450 are returned from calls to pcre2_match() to be displayed. If a mark is
1451 returned for a match, non-match, or partial match, pcre2test shows it.
1452 For a match, it is on a line by itself, tagged with "MK:". Otherwise,
1453 it is added to the non-match message.
1454 Showing memory usage
1456 The memory modifier causes pcre2test to log the sizes of all heap mem‐
1458 ory allocation and freeing calls that occur during a call to
1459 pcre2_match() or pcre2_dfa_match(). In the latter case, heap memory is
1460 used only when a match requires more internal workspace that the de‐
1461 fault allocation on the stack, so in many cases there will be no out‐
1462 put. No heap memory is allocated during matching with JIT. For this
1463 modifier to work, the null_context modifier must not be set on both the
1464 pattern and the subject, though it can be set on one or the other.
1465 Setting a starting offset
1467 The offset modifier sets an offset in the subject string at which
1469 matching starts. Its value is a number of code units, not characters.
1470 Setting an offset limit
1472 The offset_limit modifier sets a limit for unanchored matches. If a
1474 match cannot be found starting at or before this offset in the subject,
1475 a "no match" return is given. The data value is a number of code units,
1476 not characters. When this modifier is used, the use_offset_limit modi‐
1477 fier must have been set for the pattern; if not, an error is generated.
1478 Setting the size of the output vector
1480 The ovector modifier applies only to the subject line in which it ap‐
1482 pears, though of course it can also be used to set a default in a #sub‐
1483 ject command. It specifies the number of pairs of offsets that are
1484 available for storing matching information. The default is 15.
1485 A value of zero is useful when testing the POSIX API because it causes
1487 regexec() to be called with a NULL capture vector. When not testing the
1488 POSIX API, a value of zero is used to cause pcre2_match_data_cre‐
1489 ate_from_pattern() to be called, in order to create a match block of
1490 exactly the right size for the pattern. (It is not possible to create a
1491 match block with a zero-length ovector; there is always at least one
1492 pair of offsets.)
1493 Passing the subject as zero-terminated
1495 By default, the subject string is passed to a native API matching func‐
1497 tion with its correct length. In order to test the facility for passing
1498 a zero-terminated string, the zero_terminate modifier is provided. It
1499 causes the length to be passed as PCRE2_ZERO_TERMINATED. When matching
1500 via the POSIX interface, this modifier is ignored, with a warning.
1501 When testing pcre2_substitute(), this modifier also has the effect of
1503 passing the replacement string as zero-terminated.
1504 Passing a NULL context, subject, or replacement
1506 Normally, pcre2test passes a context block to pcre2_match(),
1508 pcre2_dfa_match(), pcre2_jit_match() or pcre2_substitute(). If the
1509 null_context modifier is set, however, NULL is passed. This is for
1510 testing that the matching and substitution functions behave correctly
1511 in this case (they use default values). This modifier cannot be used
1512 with the find_limits, find_limits_noheap, or substitute_callout modi‐
1513 fiers.
1514 Similarly, for testing purposes, if the null_subject or null_replace‐
1516 ment modifier is set, the subject or replacement string pointers are
1517 passed as NULL, respectively, to the relevant functions.
1518
1520 By default, pcre2test uses the standard PCRE2 matching function,
1522 pcre2_match() to match each subject line. PCRE2 also supports an alter‐
1523 native matching function, pcre2_dfa_match(), which operates in a dif‐
1524 ferent way, and has some restrictions. The differences between the two
1525 functions are described in the pcre2matching documentation.
1526 If the dfa modifier is set, the alternative matching function is used.
1528 This function finds all possible matches at a given point in the sub‐
1529 ject. If, however, the dfa_shortest modifier is set, processing stops
1530 after the first match is found. This is always the shortest possible
1531 match.
1532
1534 This section describes the output when the normal matching function,
1536 pcre2_match(), is being used.
1537 When a match succeeds, pcre2test outputs the list of captured sub‐
1539 strings, starting with number 0 for the string that matched the whole
1540 pattern. Otherwise, it outputs "No match" when the return is PCRE2_ER‐
1541 ROR_NOMATCH, or "Partial match:" followed by the partially matching
1542 substring when the return is PCRE2_ERROR_PARTIAL. (Note that this is
1543 the entire substring that was inspected during the partial match; it
1544 may include characters before the actual match start if a lookbehind
1545 assertion, \K, \b, or \B was involved.)
1546 For any other return, pcre2test outputs the PCRE2 negative error number
1548 and a short descriptive phrase. If the error is a failed UTF string
1549 check, the code unit offset of the start of the failing character is
1550 also output. Here is an example of an interactive pcre2test run.
1551 $ pcre2test
1553 PCRE2 version 10.22 2016-07-29
1554 re> /^abc(\d+)/
1556 data> abc123
1557 0: abc123
1558 1: 123
1559 data> xyz
1560 No match
1561 Unset capturing substrings that are not followed by one that is set are
1563 not shown by pcre2test unless the allcaptures modifier is specified. In
1564 the following example, there are two capturing substrings, but when the
1565 first data line is matched, the second, unset substring is not shown.
1566 An "internal" unset substring is shown as "<unset>", as for the second
1567 data line.
1568 re> /(a)|(b)/
1570 data> a
1571 0: a
1572 1: a
1573 data> b
1574 0: b
1575 1: <unset>
1576 2: b
1577 If the strings contain any non-printing characters, they are output as
1579 \xhh escapes if the value is less than 256 and UTF mode is not set.
1580 Otherwise they are output as \x{hh...} escapes. See below for the defi‐
1581 nition of non-printing characters. If the aftertext modifier is set,
1582 the output for substring 0 is followed by the the rest of the subject
1583 string, identified by "0+" like this:
1584 re> /cat/aftertext
1586 data> cataract
1587 0: cat
1588 0+ aract
1589 If global matching is requested, the results of successive matching at‐
1591 tempts are output in sequence, like this:
1592 re> /\Bi(\w\w)/g
1594 data> Mississippi
1595 0: iss
1596 1: ss
1597 0: iss
1598 1: ss
1599 0: ipp
1600 1: pp
1601 "No match" is output only if the first match attempt fails. Here is an
1603 example of a failure message (the offset 4 that is specified by the
1604 offset modifier is past the end of the subject string):
1605 re> /xyz/
1607 data> xyz\=offset=4
1608 Error -24 (bad offset value)
1609 Note that whereas patterns can be continued over several lines (a plain
1611 ">" prompt is used for continuations), subject lines may not. However
1612 newlines can be included in a subject by means of the \n escape (or \r,
1613 \r\n, etc., depending on the newline sequence setting).
1614
1616 When the alternative matching function, pcre2_dfa_match(), is used, the
1618 output consists of a list of all the matches that start at the first
1619 point in the subject where there is at least one match. For example:
1620 re> /(tang|tangerine|tan)/
1622 data> yellow tangerine\=dfa
1623 0: tangerine
1624 1: tang
1625 2: tan
1626 Using the normal matching function on this data finds only "tang". The
1628 longest matching string is always given first (and numbered zero). Af‐
1629 ter a PCRE2_ERROR_PARTIAL return, the output is "Partial match:", fol‐
1630 lowed by the partially matching substring. Note that this is the entire
1631 substring that was inspected during the partial match; it may include
1632 characters before the actual match start if a lookbehind assertion, \b,
1633 or \B was involved. (\K is not supported for DFA matching.)
1634 If global matching is requested, the search for further matches resumes
1636 at the end of the longest match. For example:
1637 re> /(tang|tangerine|tan)/g
1639 data> yellow tangerine and tangy sultana\=dfa
1640 0: tangerine
1641 1: tang
1642 2: tan
1643 0: tang
1644 1: tan
1645 0: tan
1646 The alternative matching function does not support substring capture,
1648 so the modifiers that are concerned with captured substrings are not
1649 relevant.
1650
1652 When the alternative matching function has given the PCRE2_ERROR_PAR‐
1654 TIAL return, indicating that the subject partially matched the pattern,
1655 you can restart the match with additional subject data by means of the
1656 dfa_restart modifier. For example:
1657 re> /^\d?\d(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\d\d$/
1659 data> 23ja\=ps,dfa
1660 Partial match: 23ja
1661 data> n05\=dfa,dfa_restart
1662 0: n05
1663 For further information about partial matching, see the pcre2partial
1665 documentation.
1666
1668 If the pattern contains any callout requests, pcre2test's callout func‐
1670 tion is called during matching unless callout_none is specified. This
1671 works with both matching functions, and with JIT, though there are some
1672 differences in behaviour. The output for callouts with numerical argu‐
1673 ments and those with string arguments is slightly different.
1674 Callouts with numerical arguments
1676 By default, the callout function displays the callout number, the start
1678 and current positions in the subject text at the callout time, and the
1679 next pattern item to be tested. For example:
1680 --->pqrabcdef
1682 0 ^ ^ \d
1683 This output indicates that callout number 0 occurred for a match at‐
1685 tempt starting at the fourth character of the subject string, when the
1686 pointer was at the seventh character, and when the next pattern item
1687 was \d. Just one circumflex is output if the start and current posi‐
1688 tions are the same, or if the current position precedes the start posi‐
1689 tion, which can happen if the callout is in a lookbehind assertion.
1690 Callouts numbered 255 are assumed to be automatic callouts, inserted as
1692 a result of the auto_callout pattern modifier. In this case, instead of
1693 showing the callout number, the offset in the pattern, preceded by a
1694 plus, is output. For example:
1695 re> /\d?[A-E]\*/auto_callout
1697 data> E*
1698 --->E*
1699 +0 ^ \d?
1700 +3 ^ [A-E]
1701 +8 ^^ \*
1702 +10 ^ ^
1703 0: E*
1704 If a pattern contains (*MARK) items, an additional line is output when‐
1706 ever a change of latest mark is passed to the callout function. For ex‐
1707 ample:
1708 re> /a(*MARK:X)bc/auto_callout
1710 data> abc
1711 --->abc
1712 +0 ^ a
1713 +1 ^^ (*MARK:X)
1714 +10 ^^ b
1715 Latest Mark: X
1716 +11 ^ ^ c
1717 +12 ^ ^
1718 0: abc
1719 The mark changes between matching "a" and "b", but stays the same for
1721 the rest of the match, so nothing more is output. If, as a result of
1722 backtracking, the mark reverts to being unset, the text "<unset>" is
1723 output.
1724 Callouts with string arguments
1726 The output for a callout with a string argument is similar, except that
1728 instead of outputting a callout number before the position indicators,
1729 the callout string and its offset in the pattern string are output be‐
1730 fore the reflection of the subject string, and the subject string is
1731 reflected for each callout. For example:
1732 re> /^ab(?C'first')cd(?C"second")ef/
1734 data> abcdefg
1735 Callout (7): 'first'
1736 --->abcdefg
1737 ^ ^ c
1738 Callout (20): "second"
1739 --->abcdefg
1740 ^ ^ e
1741 0: abcdef
1742 Callout modifiers
1745 The callout function in pcre2test returns zero (carry on matching) by
1747 default, but you can use a callout_fail modifier in a subject line to
1748 change this and other parameters of the callout (see below).
1749 If the callout_capture modifier is set, the current captured groups are
1751 output when a callout occurs. This is useful only for non-DFA matching,
1752 as pcre2_dfa_match() does not support capturing, so no captures are
1753 ever shown.
1754 The normal callout output, showing the callout number or pattern offset
1756 (as described above) is suppressed if the callout_no_where modifier is
1757 set.
1758 When using the interpretive matching function pcre2_match() without
1760 JIT, setting the callout_extra modifier causes additional output from
1761 pcre2test's callout function to be generated. For the first callout in
1762 a match attempt at a new starting position in the subject, "New match
1763 attempt" is output. If there has been a backtrack since the last call‐
1764 out (or start of matching if this is the first callout), "Backtrack" is
1765 output, followed by "No other matching paths" if the backtrack ended
1766 the previous match attempt. For example:
1767 re> /(a+)b/auto_callout,no_start_optimize,no_auto_possess
1769 data> aac\=callout_extra
1770 New match attempt
1771 --->aac
1772 +0 ^ (
1773 +1 ^ a+
1774 +3 ^ ^ )
1775 +4 ^ ^ b
1776 Backtrack
1777 --->aac
1778 +3 ^^ )
1779 +4 ^^ b
1780 Backtrack
1781 No other matching paths
1782 New match attempt
1783 --->aac
1784 +0 ^ (
1785 +1 ^ a+
1786 +3 ^^ )
1787 +4 ^^ b
1788 Backtrack
1789 No other matching paths
1790 New match attempt
1791 --->aac
1792 +0 ^ (
1793 +1 ^ a+
1794 Backtrack
1795 No other matching paths
1796 New match attempt
1797 --->aac
1798 +0 ^ (
1799 +1 ^ a+
1800 No match
1801 Notice that various optimizations must be turned off if you want all
1803 possible matching paths to be scanned. If no_start_optimize is not
1804 used, there is an immediate "no match", without any callouts, because
1805 the starting optimization fails to find "b" in the subject, which it
1806 knows must be present for any match. If no_auto_possess is not used,
1807 the "a+" item is turned into "a++", which reduces the number of back‐
1808 tracks.
1809 The callout_extra modifier has no effect if used with the DFA matching
1811 function, or with JIT.
1812 Return values from callouts
1814 The default return from the callout function is zero, which allows
1816 matching to continue. The callout_fail modifier can be given one or two
1817 numbers. If there is only one number, 1 is returned instead of 0 (caus‐
1818 ing matching to backtrack) when a callout of that number is reached. If
1819 two numbers (<n>:<m>) are given, 1 is returned when callout <n> is
1820 reached and there have been at least <m> callouts. The callout_error
1821 modifier is similar, except that PCRE2_ERROR_CALLOUT is returned, caus‐
1822 ing the entire matching process to be aborted. If both these modifiers
1823 are set for the same callout number, callout_error takes precedence.
1824 Note that callouts with string arguments are always given the number
1825 zero.
1826 The callout_data modifier can be given an unsigned or a negative num‐
1828 ber. This is set as the "user data" that is passed to the matching
1829 function, and passed back when the callout function is invoked. Any
1830 value other than zero is used as a return from pcre2test's callout
1831 function.
1832 Inserting callouts can be helpful when using pcre2test to check compli‐
1834 cated regular expressions. For further information about callouts, see
1835 the pcre2callout documentation.
1836
1838 When pcre2test is outputting text in the compiled version of a pattern,
1840 bytes other than 32-126 are always treated as non-printing characters
1841 and are therefore shown as hex escapes.
1842 When pcre2test is outputting text that is a matched part of a subject
1844 string, it behaves in the same way, unless a different locale has been
1845 set for the pattern (using the locale modifier). In this case, the is‐
1846 print() function is used to distinguish printing and non-printing char‐
1847 acters.
1848
1850 It is possible to save compiled patterns on disc or elsewhere, and
1852 reload them later, subject to a number of restrictions. JIT data cannot
1853 be saved. The host on which the patterns are reloaded must be running
1854 the same version of PCRE2, with the same code unit width, and must also
1855 have the same endianness, pointer width and PCRE2_SIZE type. Before
1856 compiled patterns can be saved they must be serialized, that is, con‐
1857 verted to a stream of bytes. A single byte stream may contain any num‐
1858 ber of compiled patterns, but they must all use the same character ta‐
1859 bles. A single copy of the tables is included in the byte stream (its
1860 size is 1088 bytes).
1861 The functions whose names begin with pcre2_serialize_ are used for se‐
1863 rializing and de-serializing. They are described in the pcre2serialize
1864 documentation. In this section we describe the features of pcre2test
1865 that can be used to test these functions.
1866 Note that "serialization" in PCRE2 does not convert compiled patterns
1868 to an abstract format like Java or .NET. It just makes a reloadable
1869 byte code stream. Hence the restrictions on reloading mentioned above.
1870 In pcre2test, when a pattern with push modifier is successfully com‐
1872 piled, it is pushed onto a stack of compiled patterns, and pcre2test
1873 expects the next line to contain a new pattern (or command) instead of
1874 a subject line. By contrast, the pushcopy modifier causes a copy of the
1875 compiled pattern to be stacked, leaving the original available for im‐
1876 mediate matching. By using push and/or pushcopy, a number of patterns
1877 can be compiled and retained. These modifiers are incompatible with
1878 posix, and control modifiers that act at match time are ignored (with a
1879 message) for the stacked patterns. The jitverify modifier applies only
1880 at compile time.
1881 The command
1883 #save <filename>
1885 causes all the stacked patterns to be serialized and the result written
1887 to the named file. Afterwards, all the stacked patterns are freed. The
1888 command
1889 #load <filename>
1891 reads the data in the file, and then arranges for it to be de-serial‐
1893 ized, with the resulting compiled patterns added to the pattern stack.
1894 The pattern on the top of the stack can be retrieved by the #pop com‐
1895 mand, which must be followed by lines of subjects that are to be
1896 matched with the pattern, terminated as usual by an empty line or end
1897 of file. This command may be followed by a modifier list containing
1898 only control modifiers that act after a pattern has been compiled. In
1899 particular, hex, posix, posix_nosub, push, and pushcopy are not al‐
1900 lowed, nor are any option-setting modifiers. The JIT modifiers are,
1901 however permitted. Here is an example that saves and reloads two pat‐
1902 terns.
1903 /abc/push
1905 /xyz/push
1906 #save tempfile
1907 #load tempfile
1908 #pop info
1909 xyz
1910 #pop jit,bincode
1912 abc
1913 If jitverify is used with #pop, it does not automatically imply jit,
1915 which is different behaviour from when it is used on a pattern.
1916 The #popcopy command is analogous to the pushcopy modifier in that it
1918 makes current a copy of the topmost stack pattern, leaving the original
1919 still on the stack.
1920
1922 pcre2(3), pcre2api(3), pcre2callout(3), pcre2jit, pcre2matching(3),
1924 pcre2partial(d), pcre2pattern(3), pcre2serialize(3).
1925
1927 Philip Hazel
1929 Retired from University Computing Service
1930 Cambridge, England.
1931
1933 Last updated: 27 July 2022
1935 Copyright (c) 1997-2022 University of Cambridge.
1936PCRE 10.41 27 July 2022 PCRE2TEST(1)