1PCRE2TEST(1)                General Commands Manual               PCRE2TEST(1)
2
3
4

NAME

6       pcre2test - a program for testing Perl-compatible regular expressions.
7

SYNOPSIS

9
10       pcre2test [options] [input file [output file]]
11
12       pcre2test is a test program for the PCRE2 regular expression libraries,
13       but it can also be used for  experimenting  with  regular  expressions.
14       This  document  describes the features of the test program; for details
15       of the regular expressions themselves, see the pcre2pattern  documenta‐
16       tion.  For  details  of  the  PCRE2  library  function  calls and their
17       options, see the pcre2api documentation.
18
19       The input for pcre2test is a sequence of  regular  expression  patterns
20       and  subject  strings  to  be matched. There are also command lines for
21       setting defaults and controlling some special actions. The output shows
22       the  result  of  each  match attempt. Modifiers on external or internal
23       command lines, the patterns, and the subject lines specify PCRE2  func‐
24       tion  options, control how the subject is processed, and what output is
25       produced.
26
27       As the original fairly simple PCRE library evolved,  it  acquired  many
28       different  features,  and  as  a  result, the original pcretest program
29       ended up with a lot of options in a messy, arcane  syntax  for  testing
30       all the features. The move to the new PCRE2 API provided an opportunity
31       to re-implement the test program as pcre2test, with a cleaner  modifier
32       syntax.  Nevertheless,  there are still many obscure modifiers, some of
33       which are specifically designed for use in conjunction  with  the  test
34       script  and  data  files that are distributed as part of PCRE2. All the
35       modifiers are documented here, some  without  much  justification,  but
36       many  of  them  are  unlikely  to  be  of  use  except when testing the
37       libraries.
38

PCRE2's 8-BIT, 16-BIT AND 32-BIT LIBRARIES

40
41       Different versions of the PCRE2 library can be built to support charac‐
42       ter  strings  that  are encoded in 8-bit, 16-bit, or 32-bit code units.
43       One, two, or  all  three  of  these  libraries  may  be  simultaneously
44       installed. The pcre2test program can be used to test all the libraries.
45       However, its own input and output are  always  in  8-bit  format.  When
46       testing  the  16-bit  or 32-bit libraries, patterns and subject strings
47       are converted to 16-bit or 32-bit format before  being  passed  to  the
48       library  functions.  Results are converted back to 8-bit code units for
49       output.
50
51       In the rest of this document, the names of library functions and struc‐
52       tures  are  given  in  generic  form,  for example, pcre_compile(). The
53       actual names used in the libraries have a suffix _8, _16,  or  _32,  as
54       appropriate.
55

INPUT ENCODING

57
58       Input  to  pcre2test is processed line by line, either by calling the C
59       library's fgets() function, or via the  libreadline  library.  In  some
60       Windows  environments  character 26 (hex 1A) causes an immediate end of
61       file, and no further data is read, so this character should be  avoided
62       unless you really want that action.
63
64       The  input  is  processed using using C's string functions, so must not
65       contain binary zeros, even though in  Unix-like  environments,  fgets()
66       treats  any  bytes  other  than newline as data characters. An error is
67       generated if a binary zero is encountered. By default subject lines are
68       processed for backslash escapes, which makes it possible to include any
69       data value in strings that are passed to the library for matching.  For
70       patterns,  there  is a facility for specifying some or all of the 8-bit
71       input characters as hexadecimal  pairs,  which  makes  it  possible  to
72       include binary zeros.
73
74   Input for the 16-bit and 32-bit libraries
75
76       When testing the 16-bit or 32-bit libraries, there is a need to be able
77       to generate character code points greater than 255 in the strings  that
78       are  passed to the library. For subject lines, backslash escapes can be
79       used. In addition, when the  utf  modifier  (see  "Setting  compilation
80       options" below) is set, the pattern and any following subject lines are
81       interpreted as UTF-8 strings and translated  to  UTF-16  or  UTF-32  as
82       appropriate.
83
84       For  non-UTF testing of wide characters, the utf8_input modifier can be
85       used. This is mutually exclusive with  utf,  and  is  allowed  only  in
86       16-bit  or  32-bit  mode.  It  causes the pattern and following subject
87       lines to be treated as UTF-8 according to the original definition  (RFC
88       2279), which allows for character values up to 0x7fffffff. Each charac‐
89       ter is placed in one 16-bit or 32-bit code unit (in  the  16-bit  case,
90       values greater than 0xffff cause an error to occur).
91
92       UTF-8  (in  its  original definition) is not capable of encoding values
93       greater than 0x7fffffff, but such values can be handled by  the  32-bit
94       library. When testing this library in non-UTF mode with utf8_input set,
95       if any character is preceded by the byte 0xff (which is an invalid byte
96       in  UTF-8)  0x80000000  is  added to the character's value. This is the
97       only way of passing such code points in a pattern string.  For  subject
98       strings, using an escape sequence is preferable.
99

COMMAND LINE OPTIONS

101
102       -8        If the 8-bit library has been built, this option causes it to
103                 be used (this is the default). If the 8-bit library  has  not
104                 been built, this option causes an error.
105
106       -16       If  the  16-bit library has been built, this option causes it
107                 to be used. If only the 16-bit library has been  built,  this
108                 is  the  default.  If  the 16-bit library has not been built,
109                 this option causes an error.
110
111       -32       If the 32-bit library has been built, this option  causes  it
112                 to  be  used. If only the 32-bit library has been built, this
113                 is the default. If the 32-bit library  has  not  been  built,
114                 this option causes an error.
115
116       -ac       Behave as if each pattern has the auto_callout modifier, that
117                 is, insert automatic callouts into every pattern that is com‐
118                 piled.
119
120       -AC       As  for  -ac,  but in addition behave as if each subject line
121                 has the callout_extra  modifier,  that  is,  show  additional
122                 information from callouts.
123
124       -b        Behave  as  if each pattern has the fullbincode modifier; the
125                 full internal binary form of the pattern is output after com‐
126                 pilation.
127
128       -C        Output  the  version  number  of  the  PCRE2 library, and all
129                 available information about the optional  features  that  are
130                 included,  and  then  exit  with  zero  exit  code. All other
131                 options are ignored. If both -C and -LM are  present,  which‐
132                 ever is first is recognized.
133
134       -C option Output  information  about a specific build-time option, then
135                 exit. This functionality is intended for use in scripts  such
136                 as  RunTest.  The  following options output the value and set
137                 the exit code as indicated:
138
139                   ebcdic-nl  the code for LF (= NL) in an EBCDIC environment:
140                                0x15 or 0x25
141                                0 if used in an ASCII environment
142                                exit code is always 0
143                   linksize   the configured internal link size (2, 3, or 4)
144                                exit code is set to the link size
145                   newline    the default newline setting:
146                                CR, LF, CRLF, ANYCRLF, ANY, or NUL
147                                exit code is always 0
148                   bsr        the default setting for what \R matches:
149                                ANYCRLF or ANY
150                                exit code is always 0
151
152                 The following options output 1 for true or 0 for  false,  and
153                 set the exit code to the same value:
154
155                   backslash-C  \C is supported (not locked out)
156                   ebcdic       compiled for an EBCDIC environment
157                   jit          just-in-time support is available
158                   pcre2-16     the 16-bit library was built
159                   pcre2-32     the 32-bit library was built
160                   pcre2-8      the 8-bit library was built
161                   unicode      Unicode support is available
162
163                 If  an  unknown  option is given, an error message is output;
164                 the exit code is 0.
165
166       -d        Behave as if each pattern has the debug modifier; the  inter‐
167                 nal form and information about the compiled pattern is output
168                 after compilation; -d is equivalent to -b -i.
169
170       -dfa      Behave as if each subject line has the dfa modifier; matching
171                 is  done  using the pcre2_dfa_match() function instead of the
172                 default pcre2_match().
173
174       -error number[,number,...]
175                 Call pcre2_get_error_message() for each of the error  numbers
176                 in  the  comma-separated list, display the resulting messages
177                 on the standard output, then exit with zero  exit  code.  The
178                 numbers  may  be  positive or negative. This is a convenience
179                 facility for PCRE2 maintainers.
180
181       -help     Output a brief summary these options and then exit.
182
183       -i        Behave as if each pattern has the info modifier;  information
184                 about the compiled pattern is given after compilation.
185
186       -jit      Behave  as  if  each pattern line has the jit modifier; after
187                 successful compilation, each pattern is passed to  the  just-
188                 in-time compiler, if available.
189
190       -jitfast  Behave  as  if  each  pattern  line has the jitfast modifier;
191                 after successful compilation, each pattern is passed  to  the
192                 just-in-time compiler, if available, and each subject line is
193                 passed directly to the JIT matcher via its "fast path".
194
195       -jitverify
196                 Behave as if each pattern line has  the  jitverify  modifier;
197                 after  successful  compilation, each pattern is passed to the
198                 just-in-time compiler, if available, and the use of  JIT  for
199                 matching is verified.
200
201       -LM       List modifiers: write a list of available pattern and subject
202                 modifiers to the standard output, then exit  with  zero  exit
203                 code.  All other options are ignored.  If both -C and -LM are
204                 present, whichever is first is recognized.
205
206       -pattern modifier-list
207                 Behave as if each pattern line contains the given modifiers.
208
209       -q        Do not output the version number of pcre2test at the start of
210                 execution.
211
212       -S size   On  Unix-like  systems, set the size of the run-time stack to
213                 size mebibytes (units of 1024*1024 bytes).
214
215       -subject modifier-list
216                 Behave as if each subject line contains the given modifiers.
217
218       -t        Run each compile and match many times with a timer, and  out‐
219                 put  the  resulting  times  per compile or match. When JIT is
220                 used, separate times are given for the  initial  compile  and
221                 the  JIT  compile.  You  can control the number of iterations
222                 that are used for timing by following -t with a number (as  a
223                 separate  item  on  the command line). For example, "-t 1000"
224                 iterates 1000 times. The default is to iterate 500,000 times.
225
226       -tm       This is like -t except that it times only the matching phase,
227                 not the compile phase.
228
229       -T -TM    These  behave like -t and -tm, but in addition, at the end of
230                 a run, the total times for all compiles and matches are  out‐
231                 put.
232
233       -version  Output the PCRE2 version number and then exit.
234

DESCRIPTION

236
237       If  pcre2test  is given two filename arguments, it reads from the first
238       and writes to the second. If the first name is "-", input is taken from
239       the  standard  input. If pcre2test is given only one argument, it reads
240       from that file and writes to stdout. Otherwise, it reads from stdin and
241       writes to stdout.
242
243       When  pcre2test  is  built,  a configuration option can specify that it
244       should be linked with the libreadline or libedit library. When this  is
245       done,  if the input is from a terminal, it is read using the readline()
246       function. This provides line-editing and history facilities. The output
247       from the -help option states whether or not readline() will be used.
248
249       The  program  handles  any number of tests, each of which consists of a
250       set of input lines. Each set starts with a regular expression  pattern,
251       followed by any number of subject lines to be matched against that pat‐
252       tern. In between sets of test data, command lines that begin with # may
253       appear. This file format, with some restrictions, can also be processed
254       by the perltest.sh script that is distributed with PCRE2 as a means  of
255       checking that the behaviour of PCRE2 and Perl is the same. For a speci‐
256       fication of perltest.sh, see the comments near its beginning.
257
258       When the input is a terminal, pcre2test prompts for each line of input,
259       using  "re>"  to prompt for regular expression patterns, and "data>" to
260       prompt for subject lines. Command lines starting with # can be  entered
261       only in response to the "re>" prompt.
262
263       Each  subject line is matched separately and independently. If you want
264       to do multi-line matches, you have to use the \n escape sequence (or \r
265       or  \r\n,  etc.,  depending on the newline setting) in a single line of
266       input to encode the newline sequences. There is no limit on the  length
267       of  subject  lines; the input buffer is automatically extended if it is
268       too small. There are replication features that  makes  it  possible  to
269       generate  long  repetitive  pattern  or subject lines without having to
270       supply them explicitly.
271
272       An empty line or the end of the file signals the  end  of  the  subject
273       lines  for  a  test,  at  which  point a new pattern or command line is
274       expected if there is still input to be read.
275

COMMAND LINES

277
278       In between sets of test data, a line that begins with # is  interpreted
279       as a command line. If the first character is followed by white space or
280       an exclamation mark, the line is treated as  a  comment,  and  ignored.
281       Otherwise, the following commands are recognized:
282
283         #forbid_utf
284
285       Subsequent   patterns   automatically   have  the  PCRE2_NEVER_UTF  and
286       PCRE2_NEVER_UCP options set, which locks out the use of  the  PCRE2_UTF
287       and  PCRE2_UCP options and the use of (*UTF) and (*UCP) at the start of
288       patterns. This command also forces an error  if  a  subsequent  pattern
289       contains  any  occurrences  of \P, \p, or \X, which are still supported
290       when PCRE2_UTF is not set, but which require Unicode  property  support
291       to be included in the library.
292
293       This  is  a trigger guard that is used in test files to ensure that UTF
294       or Unicode property tests are not accidentally added to files that  are
295       used  when  Unicode  support  is  not  included in the library. Setting
296       PCRE2_NEVER_UTF and PCRE2_NEVER_UCP as a default can also  be  obtained
297       by  the  use  of #pattern; the difference is that #forbid_utf cannot be
298       unset, and the automatic options are not displayed in pattern  informa‐
299       tion, to avoid cluttering up test output.
300
301         #load <filename>
302
303       This command is used to load a set of precompiled patterns from a file,
304       as described in the section entitled  "Saving  and  restoring  compiled
305       patterns" below.
306
307         #newline_default [<newline-list>]
308
309       When  PCRE2  is  built,  a default newline convention can be specified.
310       This determines which characters and/or character pairs are  recognized
311       as indicating a newline in a pattern or subject string. The default can
312       be overridden when a pattern is compiled. The standard test files  con‐
313       tain  tests  of  various  newline  conventions, but the majority of the
314       tests expect a single  linefeed  to  be  recognized  as  a  newline  by
315       default. Without special action the tests would fail when PCRE2 is com‐
316       piled with either CR or CRLF as the default newline.
317
318       The #newline_default command specifies a list of newline types that are
319       acceptable  as the default. The types must be one of CR, LF, CRLF, ANY‐
320       CRLF, ANY, or NUL (in upper or lower case), for example:
321
322         #newline_default LF Any anyCRLF
323
324       If the default newline is in the list, this command has no effect. Oth‐
325       erwise,  except  when  testing  the  POSIX API, a newline modifier that
326       specifies the first newline convention in the list  (LF  in  the  above
327       example)  is  added to any pattern that does not already have a newline
328       modifier. If the newline list is empty, the feature is turned off. This
329       command is present in a number of the standard test input files.
330
331       When  the  POSIX  API  is  being tested there is no way to override the
332       default newline convention, though it is possible to  set  the  newline
333       convention  from within the pattern. A warning is given if the posix or
334       posix_nosub modifier is used when #newline_default would set a  default
335       for the non-POSIX API.
336
337         #pattern <modifier-list>
338
339       This  command  sets  a default modifier list that applies to all subse‐
340       quent patterns. Modifiers on a pattern can change these settings.
341
342         #perltest
343
344       The appearance of this line causes all subsequent modifier settings  to
345       be checked for compatibility with the perltest.sh script, which is used
346       to confirm that Perl gives the same results as PCRE2. Also, apart  from
347       comment  lines,  #pattern  commands,  and #subject commands that set or
348       unset "mark", no command lines are permitted, because they and many  of
349       the modifiers are specific to pcre2test, and should not be used in test
350       files that are also processed by  perltest.sh.  The  #perltest  command
351       helps detect tests that are accidentally put in the wrong file.
352
353         #pop [<modifiers>]
354         #popcopy [<modifiers>]
355
356       These  commands  are used to manipulate the stack of compiled patterns,
357       as described in the section entitled  "Saving  and  restoring  compiled
358       patterns" below.
359
360         #save <filename>
361
362       This  command  is used to save a set of compiled patterns to a file, as
363       described in the section entitled "Saving and restoring  compiled  pat‐
364       terns" below.
365
366         #subject <modifier-list>
367
368       This  command  sets  a default modifier list that applies to all subse‐
369       quent subject lines. Modifiers on a subject line can change these  set‐
370       tings.
371

MODIFIER SYNTAX

373
374       Modifier lists are used with both pattern and subject lines. Items in a
375       list are separated by commas followed by optional white space. Trailing
376       whitespace  in  a modifier list is ignored. Some modifiers may be given
377       for both patterns and subject lines, whereas others are valid only  for
378       one  or  the  other.  Each  modifier  has  a  long  name,  for  example
379       "anchored", and some of them must be followed by an equals sign  and  a
380       value,  for  example,  "offset=12". Values cannot contain comma charac‐
381       ters, but may contain spaces. Modifiers that do not take values may  be
382       preceded by a minus sign to turn off a previous setting.
383
384       A few of the more common modifiers can also be specified as single let‐
385       ters, for example "i" for "caseless". In documentation,  following  the
386       Perl convention, these are written with a slash ("the /i modifier") for
387       clarity. Abbreviated modifiers must all be concatenated  in  the  first
388       item  of a modifier list. If the first item is not recognized as a long
389       modifier name, it is interpreted as a sequence of these  abbreviations.
390       For example:
391
392         /abc/ig,newline=cr,jit=3
393
394       This  is  a pattern line whose modifier list starts with two one-letter
395       modifiers (/i and /g). The lower-case  abbreviated  modifiers  are  the
396       same as used in Perl.
397

PATTERN SYNTAX

399
400       A  pattern line must start with one of the following characters (common
401       symbols, excluding pattern meta-characters):
402
403         / ! " ' ` - = _ : ; , % & @ ~
404
405       This is interpreted as the pattern's delimiter.  A  regular  expression
406       may  be  continued  over several input lines, in which case the newline
407       characters are included within it. It is possible to include the delim‐
408       iter within the pattern by escaping it with a backslash, for example
409
410         /abc\/def/
411
412       If  you do this, the escape and the delimiter form part of the pattern,
413       but since the delimiters are all non-alphanumeric, this does not affect
414       its  interpretation.  If  the terminating delimiter is immediately fol‐
415       lowed by a backslash, for example,
416
417         /abc/\
418
419       then a backslash is added to the end of the pattern. This  is  done  to
420       provide  a  way of testing the error condition that arises if a pattern
421       finishes with a backslash, because
422
423         /abc\/
424
425       is interpreted as the first line of a pattern that starts with  "abc/",
426       causing  pcre2test to read the next line as a continuation of the regu‐
427       lar expression.
428
429       A pattern can be followed by a modifier list (details below).
430

SUBJECT LINE SYNTAX

432
433       Before   each   subject   line   is   passed   to   pcre2_match()    or
434       pcre2_dfa_match(), leading and trailing white space is removed, and the
435       line is scanned for backslash escapes, unless the subject_literal modi‐
436       fier was set for the pattern. The following provide a means of encoding
437       non-printing characters in a visible way:
438
439         \a         alarm (BEL, \x07)
440         \b         backspace (\x08)
441         \e         escape (\x27)
442         \f         form feed (\x0c)
443         \n         newline (\x0a)
444         \r         carriage return (\x0d)
445         \t         tab (\x09)
446         \v         vertical tab (\x0b)
447         \nnn       octal character (up to 3 octal digits); always
448                      a byte unless > 255 in UTF-8 or 16-bit or 32-bit mode
449         \o{dd...}  octal character (any number of octal digits}
450         \xhh       hexadecimal byte (up to 2 hex digits)
451         \x{hh...}  hexadecimal character (any number of hex digits)
452
453       The use of \x{hh...} is not dependent on the use of the utf modifier on
454       the  pattern. It is recognized always. There may be any number of hexa‐
455       decimal digits inside the braces; invalid  values  provoke  error  mes‐
456       sages.
457
458       Note  that  \xhh  specifies one byte rather than one character in UTF-8
459       mode; this makes it possible to construct invalid UTF-8  sequences  for
460       testing  purposes.  On the other hand, \x{hh} is interpreted as a UTF-8
461       character in UTF-8 mode, generating more than one byte if the value  is
462       greater  than  127.   When testing the 8-bit library not in UTF-8 mode,
463       \x{hh} generates one byte for values less than 256, and causes an error
464       for greater values.
465
466       In UTF-16 mode, all 4-digit \x{hhhh} values are accepted. This makes it
467       possible to construct invalid UTF-16 sequences for testing purposes.
468
469       In UTF-32 mode, all 4- to 8-digit \x{...}  values  are  accepted.  This
470       makes  it  possible  to  construct invalid UTF-32 sequences for testing
471       purposes.
472
473       There is a special backslash sequence that specifies replication of one
474       or more characters:
475
476         \[<characters>]{<count>}
477
478       This  makes  it possible to test long strings without having to provide
479       them as part of the file. For example:
480
481         \[abc]{4}
482
483       is converted to "abcabcabcabc". This feature does not support  nesting.
484       To include a closing square bracket in the characters, code it as \x5D.
485
486       A  backslash  followed  by  an equals sign marks the end of the subject
487       string and the start of a modifier list. For example:
488
489         abc\=notbol,notempty
490
491       If the subject string is empty and \= is followed  by  whitespace,  the
492       line  is  treated  as a comment line, and is not used for matching. For
493       example:
494
495         \= This is a comment.
496         abc\= This is an invalid modifier list.
497
498       A backslash followed  by  any  other  non-alphanumeric  character  just
499       escapes that character. A backslash followed by anything else causes an
500       error. However, if the very last character in the line is  a  backslash
501       (and  there  is  no  modifier list), it is ignored. This gives a way of
502       passing an empty line as data, since a real empty line  terminates  the
503       data input.
504
505       If the subject_literal modifier is set for a pattern, all subject lines
506       that follow are treated as literals, with no special treatment of back‐
507       slashes.  No replication is possible, and any subject modifiers must be
508       set as defaults by a #subject command.
509

PATTERN MODIFIERS

511
512       There are several types of modifier that can appear in  pattern  lines.
513       Except where noted below, they may also be used in #pattern commands. A
514       pattern's modifier list can add to or override default  modifiers  that
515       were set by a previous #pattern command.
516
517   Setting compilation options
518
519       The  following  modifiers set options for pcre2_compile(). Most of them
520       set bits in the options argument of  that  function,  but  those  whose
521       names start with PCRE2_EXTRA are additional options that are set in the
522       compile context. For the main options,  there  are  some  single-letter
523       abbreviations  that are the same as Perl options. There is special han‐
524       dling for /x: if a second x is  present,  PCRE2_EXTENDED  is  converted
525       into   PCRE2_EXTENDED_MORE   as   in  Perl.  A  third  appearance  adds
526       PCRE2_EXTENDED as well, though this makes  no  difference  to  the  way
527       pcre2_compile()  behaves. See pcre2api for a description of the effects
528       of these options.
529
530             allow_empty_class         set PCRE2_ALLOW_EMPTY_CLASS
531             allow_surrogate_escapes   set PCRE2_EXTRA_ALLOW_SURROGATE_ESCAPES
532             alt_bsux                  set PCRE2_ALT_BSUX
533             alt_circumflex            set PCRE2_ALT_CIRCUMFLEX
534             alt_verbnames             set PCRE2_ALT_VERBNAMES
535             anchored                  set PCRE2_ANCHORED
536             auto_callout              set PCRE2_AUTO_CALLOUT
537             bad_escape_is_literal     set PCRE2_EXTRA_BAD_ESCAPE_IS_LITERAL
538         /i  caseless                  set PCRE2_CASELESS
539             dollar_endonly            set PCRE2_DOLLAR_ENDONLY
540         /s  dotall                    set PCRE2_DOTALL
541             dupnames                  set PCRE2_DUPNAMES
542             endanchored               set PCRE2_ENDANCHORED
543             escaped_cr_is_lf          set PCRE2_EXTRA_ESCAPED_CR_IS_LF
544         /x  extended                  set PCRE2_EXTENDED
545         /xx extended_more             set PCRE2_EXTENDED_MORE
546             extra_alt_bsux            set PCRE2_EXTRA_ALT_BSUX
547             firstline                 set PCRE2_FIRSTLINE
548             literal                   set PCRE2_LITERAL
549             match_line                set PCRE2_EXTRA_MATCH_LINE
550             match_invalid_utf         set PCRE2_MATCH_INVALID_UTF
551             match_unset_backref       set PCRE2_MATCH_UNSET_BACKREF
552             match_word                set PCRE2_EXTRA_MATCH_WORD
553         /m  multiline                 set PCRE2_MULTILINE
554             never_backslash_c         set PCRE2_NEVER_BACKSLASH_C
555             never_ucp                 set PCRE2_NEVER_UCP
556             never_utf                 set PCRE2_NEVER_UTF
557         /n  no_auto_capture           set PCRE2_NO_AUTO_CAPTURE
558             no_auto_possess           set PCRE2_NO_AUTO_POSSESS
559             no_dotstar_anchor         set PCRE2_NO_DOTSTAR_ANCHOR
560             no_start_optimize         set PCRE2_NO_START_OPTIMIZE
561             no_utf_check              set PCRE2_NO_UTF_CHECK
562             ucp                       set PCRE2_UCP
563             ungreedy                  set PCRE2_UNGREEDY
564             use_offset_limit          set PCRE2_USE_OFFSET_LIMIT
565             utf                       set PCRE2_UTF
566
567       As well as turning on the PCRE2_UTF option, the utf modifier causes all
568       non-printing  characters  in  output  strings  to  be printed using the
569       \x{hh...} notation. Otherwise, those less than 0x100 are output in  hex
570       without  the  curly brackets. Setting utf in 16-bit or 32-bit mode also
571       causes pattern and subject  strings  to  be  translated  to  UTF-16  or
572       UTF-32, respectively, before being passed to library functions.
573
574   Setting compilation controls
575
576       The  following  modifiers  affect  the  compilation  process or request
577       information about the pattern. There  are  single-letter  abbreviations
578       for some that are heavily used in the test files.
579
580             bsr=[anycrlf|unicode]     specify \R handling
581         /B  bincode                   show binary code without lengths
582             callout_info              show callout information
583             convert=<options>         request foreign pattern conversion
584             convert_glob_escape=c     set glob escape character
585             convert_glob_separator=c  set glob separator character
586             convert_length            set convert buffer length
587             debug                     same as info,fullbincode
588             framesize                 show matching frame size
589             fullbincode               show binary code with lengths
590         /I  info                      show info about compiled pattern
591             hex                       unquoted characters are hexadecimal
592             jit[=<number>]            use JIT
593             jitfast                   use JIT fast path
594             jitverify                 verify JIT use
595             locale=<name>             use this locale
596             max_pattern_length=<n>    set the maximum pattern length
597             memory                    show memory used
598             newline=<type>            set newline type
599             null_context              compile with a NULL context
600             parens_nest_limit=<n>     set maximum parentheses depth
601             posix                     use the POSIX API
602             posix_nosub               use the POSIX API with REG_NOSUB
603             push                      push compiled pattern onto the stack
604             pushcopy                  push a copy onto the stack
605             stackguard=<number>       test the stackguard feature
606             subject_literal           treat all subject lines as literal
607             tables=[0|1|2]            select internal tables
608             use_length                do not zero-terminate the pattern
609             utf8_input                treat input as UTF-8
610
611       The effects of these modifiers are described in the following sections.
612
613   Newline and \R handling
614
615       The  bsr modifier specifies what \R in a pattern should match. If it is
616       set to "anycrlf", \R matches CR, LF, or CRLF only.  If  it  is  set  to
617       "unicode",  \R matches any Unicode newline sequence. The default can be
618       specified when PCRE2 is built; if it is not, the default is set to Uni‐
619       code.
620
621       The  newline  modifier specifies which characters are to be interpreted
622       as newlines, both in the pattern and in subject lines. The type must be
623       one of CR, LF, CRLF, ANYCRLF, ANY, or NUL (in upper or lower case).
624
625   Information about a pattern
626
627       The  debug modifier is a shorthand for info,fullbincode, requesting all
628       available information.
629
630       The bincode modifier causes a representation of the compiled code to be
631       output  after compilation. This information does not contain length and
632       offset values, which ensures that the same output is generated for dif‐
633       ferent  internal  link  sizes  and different code unit widths. By using
634       bincode, the same regression tests can be used  in  different  environ‐
635       ments.
636
637       The  fullbincode  modifier, by contrast, does include length and offset
638       values. This is used in a few special tests that run only for  specific
639       code unit widths and link sizes, and is also useful for one-off tests.
640
641       The  info  modifier  requests  information  about  the compiled pattern
642       (whether it is anchored, has a fixed first character, and so  on).  The
643       information  is  obtained  from the pcre2_pattern_info() function. Here
644       are some typical examples:
645
646           re> /(?i)(^a|^b)/m,info
647         Capture group count = 1
648         Compile options: multiline
649         Overall options: caseless multiline
650         First code unit at start or follows newline
651         Subject length lower bound = 1
652
653           re> /(?i)abc/info
654         Capture group count = 0
655         Compile options: <none>
656         Overall options: caseless
657         First code unit = 'a' (caseless)
658         Last code unit = 'c' (caseless)
659         Subject length lower bound = 3
660
661       "Compile options" are those specified by modifiers;  "overall  options"
662       have  added options that are taken or deduced from the pattern. If both
663       sets of options are the same, just a single "options" line  is  output;
664       if  there  are  no  options,  the line is omitted. "First code unit" is
665       where any match must start; if there is more than one they  are  listed
666       as  "starting  code  units".  "Last code unit" is the last literal code
667       unit that must be present in any match. This  is  not  necessarily  the
668       last  character.  These lines are omitted if no starting or ending code
669       units  are  recorded.  The  subject  length  line   is   omitted   when
670       no_start_optimize  is  set because the minimum length is not calculated
671       when it can never be used.
672
673       The framesize modifier shows the size, in bytes, of the storage  frames
674       used  by  pcre2_match()  for handling backtracking. The size depends on
675       the number of capturing parentheses in the pattern.
676
677       The callout_info modifier requests information about all  the  callouts
678       in the pattern. A list of them is output at the end of any other infor‐
679       mation that is requested. For each callout, either its number or string
680       is given, followed by the item that follows it in the pattern.
681
682   Passing a NULL context
683
684       Normally,  pcre2test  passes a context block to pcre2_compile(). If the
685       null_context modifier is set, however, NULL  is  passed.  This  is  for
686       testing  that  pcre2_compile()  behaves correctly in this case (it uses
687       default values).
688
689   Specifying pattern characters in hexadecimal
690
691       The hex modifier specifies that the characters of the  pattern,  except
692       for  substrings  enclosed  in single or double quotes, are to be inter‐
693       preted as pairs of hexadecimal digits. This feature is  provided  as  a
694       way of creating patterns that contain binary zeros and other non-print‐
695       ing characters. White space is permitted between pairs of  digits.  For
696       example, this pattern contains three characters:
697
698         /ab 32 59/hex
699
700       Parts  of  such  a  pattern are taken literally if quoted. This pattern
701       contains nine characters, only two of which are specified in  hexadeci‐
702       mal:
703
704         /ab "literal" 32/hex
705
706       Either  single or double quotes may be used. There is no way of includ‐
707       ing the delimiter within a substring. The hex and expand modifiers  are
708       mutually exclusive.
709
710   Specifying the pattern's length
711
712       By default, patterns are passed to the compiling functions as zero-ter‐
713       minated strings but can be passed by length instead of being  zero-ter‐
714       minated.  The use_length modifier causes this to happen. Using a length
715       happens automatically (whether or not use_length is set)  when  hex  is
716       set,  because  patterns  specified  in  hexadecimal  may contain binary
717       zeros.
718
719       If hex or use_length is used with the POSIX wrapper API (see "Using the
720       POSIX  wrapper  API" below), the REG_PEND extension is used to pass the
721       pattern's length.
722
723   Specifying wide characters in 16-bit and 32-bit modes
724
725       In 16-bit and 32-bit modes, all input is automatically treated as UTF-8
726       and  translated  to  UTF-16 or UTF-32 when the utf modifier is set. For
727       testing the 16-bit and 32-bit libraries in non-UTF mode, the utf8_input
728       modifier  can  be  used. It is mutually exclusive with utf. Input lines
729       are interpreted as UTF-8 as a means of specifying wide characters. More
730       details are given in "Input encoding" above.
731
732   Generating long repetitive patterns
733
734       Some  tests use long patterns that are very repetitive. Instead of cre‐
735       ating a very long input line for such a pattern, you can use a  special
736       repetition  feature,  similar  to  the  one described for subject lines
737       above. If the expand modifier is present on a  pattern,  parts  of  the
738       pattern that have the form
739
740         \[<characters>]{<count>}
741
742       are expanded before the pattern is passed to pcre2_compile(). For exam‐
743       ple, \[AB]{6000} is expanded to "ABAB..." 6000 times. This construction
744       cannot  be  nested. An initial "\[" sequence is recognized only if "]{"
745       followed by decimal digits and "}" is found later in  the  pattern.  If
746       not, the characters remain in the pattern unaltered. The expand and hex
747       modifiers are mutually exclusive.
748
749       If part of an expanded pattern looks like an expansion, but  is  really
750       part of the actual pattern, unwanted expansion can be avoided by giving
751       two values in the quantifier. For example, \[AB]{6000,6000} is not rec‐
752       ognized as an expansion item.
753
754       If  the  info modifier is set on an expanded pattern, the result of the
755       expansion is included in the information that is output.
756
757   JIT compilation
758
759       Just-in-time (JIT) compiling is a  heavyweight  optimization  that  can
760       greatly  speed  up pattern matching. See the pcre2jit documentation for
761       details. JIT compiling happens, optionally, after a  pattern  has  been
762       successfully  compiled into an internal form. The JIT compiler converts
763       this to optimized machine code. It needs to know whether the match-time
764       options PCRE2_PARTIAL_HARD and PCRE2_PARTIAL_SOFT are going to be used,
765       because different code is generated for the different  cases.  See  the
766       partial  modifier in "Subject Modifiers" below for details of how these
767       options are specified for each match attempt.
768
769       JIT compilation is requested by the jit  pattern  modifier,  which  may
770       optionally be followed by an equals sign and a number in the range 0 to
771       7.  The three bits that make up the number specify which of  the  three
772       JIT operating modes are to be compiled:
773
774         1  compile JIT code for non-partial matching
775         2  compile JIT code for soft partial matching
776         4  compile JIT code for hard partial matching
777
778       The possible values for the jit modifier are therefore:
779
780         0  disable JIT
781         1  normal matching only
782         2  soft partial matching only
783         3  normal and soft partial matching
784         4  hard partial matching only
785         6  soft and hard partial matching only
786         7  all three modes
787
788       If  no  number  is  given,  7 is assumed. The phrase "partial matching"
789       means a call to pcre2_match() with either the PCRE2_PARTIAL_SOFT or the
790       PCRE2_PARTIAL_HARD  option set. Note that such a call may return a com‐
791       plete match; the options enable the possibility of a partial match, but
792       do  not  require it. Note also that if you request JIT compilation only
793       for partial matching (for example, jit=2) but do not  set  the  partial
794       modifier  on  a  subject line, that match will not use JIT code because
795       none was compiled for non-partial matching.
796
797       If JIT compilation is successful, the compiled JIT code will  automati‐
798       cally  be  used  when  an appropriate type of match is run, except when
799       incompatible run-time options are specified. For more details, see  the
800       pcre2jit  documentation. See also the jitstack modifier below for a way
801       of setting the size of the JIT stack.
802
803       If the jitfast modifier is specified, matching is done  using  the  JIT
804       "fast  path" interface, pcre2_jit_match(), which skips some of the san‐
805       ity checks that are done by pcre2_match(), and of course does not  work
806       when  JIT  is not supported. If jitfast is specified without jit, jit=7
807       is assumed.
808
809       If the jitverify modifier is specified, information about the  compiled
810       pattern  shows  whether  JIT  compilation was or was not successful. If
811       jitverify is specified without jit, jit=7 is assumed. If  JIT  compila‐
812       tion  is successful when jitverify is set, the text "(JIT)" is added to
813       the first output line after a match or non match when JIT-compiled code
814       was actually used in the match.
815
816   Setting a locale
817
818       The locale modifier must specify the name of a locale, for example:
819
820         /pattern/locale=fr_FR
821
822       The given locale is set, pcre2_maketables() is called to build a set of
823       character tables for the locale, and this is then passed to  pcre2_com‐
824       pile()  when compiling the regular expression. The same tables are used
825       when matching the following subject lines. The locale modifier  applies
826       only to the pattern on which it appears, but can be given in a #pattern
827       command if a default is needed. Setting a locale and alternate  charac‐
828       ter tables are mutually exclusive.
829
830   Showing pattern memory
831
832       The memory modifier causes the size in bytes of the memory used to hold
833       the compiled pattern to be output. This does not include  the  size  of
834       the  pcre2_code block; it is just the actual compiled data. If the pat‐
835       tern is subsequently passed to the JIT compiler, the size  of  the  JIT
836       compiled code is also output. Here is an example:
837
838           re> /a(b)c/jit,memory
839         Memory allocation (code space): 21
840         Memory allocation (JIT code): 1910
841
842
843   Limiting nested parentheses
844
845       The  parens_nest_limit  modifier  sets  a  limit on the depth of nested
846       parentheses in a pattern. Breaching  the  limit  causes  a  compilation
847       error.   The  default  for  the library is set when PCRE2 is built, but
848       pcre2test sets its own default of 220, which is  required  for  running
849       the standard test suite.
850
851   Limiting the pattern length
852
853       The  max_pattern_length  modifier  sets  a limit, in code units, to the
854       length of pattern that pcre2_compile() will accept. Breaching the limit
855       causes  a  compilation  error.  The  default  is  the  largest number a
856       PCRE2_SIZE variable can hold (essentially unlimited).
857
858   Using the POSIX wrapper API
859
860       The posix and posix_nosub modifiers cause pcre2test to call  PCRE2  via
861       the  POSIX  wrapper API rather than its native API. When posix_nosub is
862       used, the POSIX option REG_NOSUB is  passed  to  regcomp().  The  POSIX
863       wrapper  supports  only  the 8-bit library. Note that it does not imply
864       POSIX matching semantics; for more detail see the pcre2posix documenta‐
865       tion.  The  following  pattern  modifiers set options for the regcomp()
866       function:
867
868         caseless           REG_ICASE
869         multiline          REG_NEWLINE
870         dotall             REG_DOTALL     )
871         ungreedy           REG_UNGREEDY   ) These options are not part of
872         ucp                REG_UCP        )   the POSIX standard
873         utf                REG_UTF8       )
874
875       The regerror_buffsize modifier specifies a size for  the  error  buffer
876       that  is  passed to regerror() in the event of a compilation error. For
877       example:
878
879         /abc/posix,regerror_buffsize=20
880
881       This provides a means of testing the behaviour of regerror()  when  the
882       buffer  is  too  small  for the error message. If this modifier has not
883       been set, a large buffer is used.
884
885       The aftertext and allaftertext  subject  modifiers  work  as  described
886       below.  All other modifiers are either ignored, with a warning message,
887       or cause an error.
888
889       The pattern is passed to  regcomp()  as  a  zero-terminated  string  by
890       default,  but  if the use_length or hex modifiers are set, the REG_PEND
891       extension is used to pass it by length.
892
893   Testing the stack guard feature
894
895       The stackguard modifier is used  to  test  the  use  of  pcre2_set_com‐
896       pile_recursion_guard(),  a  function  that  is provided to enable stack
897       availability to be checked during compilation (see the  pcre2api  docu‐
898       mentation  for  details).  If  the  number specified by the modifier is
899       greater than zero, pcre2_set_compile_recursion_guard() is called to set
900       up  callback  from pcre2_compile() to a local function. The argument it
901       receives is the current nesting parenthesis depth; if this  is  greater
902       than the value given by the modifier, non-zero is returned, causing the
903       compilation to be aborted.
904
905   Using alternative character tables
906
907       The value specified for the tables modifier must be one of  the  digits
908       0, 1, or 2. It causes a specific set of built-in character tables to be
909       passed to pcre2_compile(). This is used in the PCRE2 tests to check be‐
910       haviour with different character tables. The digit specifies the tables
911       as follows:
912
913         0   do not pass any special character tables
914         1   the default ASCII tables, as distributed in
915               pcre2_chartables.c.dist
916         2   a set of tables defining ISO 8859 characters
917
918       In table 2, some characters whose codes are greater than 128 are  iden‐
919       tified  as  letters,  digits,  spaces, etc. Setting alternate character
920       tables and a locale are mutually exclusive.
921
922   Setting certain match controls
923
924       The following modifiers are really subject modifiers, and are described
925       under  "Subject  Modifiers"  below.  However, they may be included in a
926       pattern's modifier list, in which case they are applied to  every  sub‐
927       ject  line  that is processed with that pattern. These modifiers do not
928       affect the compilation process.
929
930             aftertext                  show text after match
931             allaftertext               show text after captures
932             allcaptures                show all captures
933             allvector                  show the entire ovector
934             allusedtext                show all consulted text
935             altglobal                  alternative global matching
936         /g  global                     global matching
937             jitstack=<n>               set size of JIT stack
938             mark                       show mark values
939             replace=<string>           specify a replacement string
940             startchar                  show starting character when relevant
941             substitute_callout         use substitution callouts
942             substitute_extended        use PCRE2_SUBSTITUTE_EXTENDED
943             substitute_skip=<n>        skip substitution number n
944             substitute_overflow_length use PCRE2_SUBSTITUTE_OVERFLOW_LENGTH
945             substitute_stop=<n>        skip substitution number n and greater
946             substitute_unknown_unset   use PCRE2_SUBSTITUTE_UNKNOWN_UNSET
947             substitute_unset_empty     use PCRE2_SUBSTITUTE_UNSET_EMPTY
948
949       These modifiers may not appear in a #pattern command. If you want  them
950       as defaults, set them in a #subject command.
951
952   Specifying literal subject lines
953
954       If  the  subject_literal modifier is present on a pattern, all the sub‐
955       ject lines that it matches are taken as literal strings, with no inter‐
956       pretation  of  backslashes. It is not possible to set subject modifiers
957       on such lines, but any that are set as defaults by a  #subject  command
958       are recognized.
959
960   Saving a compiled pattern
961
962       When  a  pattern with the push modifier is successfully compiled, it is
963       pushed onto a stack of compiled patterns,  and  pcre2test  expects  the
964       next  line to contain a new pattern (or a command) instead of a subject
965       line. This facility is used when saving compiled patterns to a file, as
966       described  in  the section entitled "Saving and restoring compiled pat‐
967       terns" below.  If pushcopy is used instead of push, a copy of the  com‐
968       piled  pattern  is  stacked,  leaving the original as current, ready to
969       match the following input lines. This provides a  way  of  testing  the
970       pcre2_code_copy()  function.   The  push  and  pushcopy   modifiers are
971       incompatible with compilation modifiers such  as  global  that  act  at
972       match  time. Any that are specified are ignored (for the stacked copy),
973       with a warning message, except for replace, which causes an error. Note
974       that  jitverify, which is allowed, does not carry through to any subse‐
975       quent matching that uses a stacked pattern.
976
977   Testing foreign pattern conversion
978
979       The experimental foreign pattern conversion functions in PCRE2  can  be
980       tested  by  setting the convert modifier. Its argument is a colon-sepa‐
981       rated list  of  options,  which  set  the  equivalent  option  for  the
982       pcre2_pattern_convert() function:
983
984         glob                    PCRE2_CONVERT_GLOB
985         glob_no_starstar        PCRE2_CONVERT_GLOB_NO_STARSTAR
986         glob_no_wild_separator  PCRE2_CONVERT_GLOB_NO_WILD_SEPARATOR
987         posix_basic             PCRE2_CONVERT_POSIX_BASIC
988         posix_extended          PCRE2_CONVERT_POSIX_EXTENDED
989         unset                   Unset all options
990
991       The "unset" value is useful for turning off a default that has been set
992       by a #pattern command. When one of these options is set, the input pat‐
993       tern  is  passed  to pcre2_pattern_convert(). If the conversion is suc‐
994       cessful, the result is reflected in  the  output  and  then  passed  to
995       pcre2_compile(). The normal utf and no_utf_check options, if set, cause
996       the PCRE2_CONVERT_UTF  and  PCRE2_CONVERT_NO_UTF_CHECK  options  to  be
997       passed to pcre2_pattern_convert().
998
999       By default, the conversion function is allowed to allocate a buffer for
1000       its output. However, if the convert_length modifier is set to  a  value
1001       greater  than zero, pcre2test passes a buffer of the given length. This
1002       makes it possible to test the length check.
1003
1004       The convert_glob_escape and  convert_glob_separator  modifiers  can  be
1005       used  to  specify the escape and separator characters for glob process‐
1006       ing, overriding the defaults, which are operating-system dependent.
1007

SUBJECT MODIFIERS

1009
1010       The modifiers that can appear in subject lines and the #subject command
1011       are of two types.
1012
1013   Setting match options
1014
1015       The    following   modifiers   set   options   for   pcre2_match()   or
1016       pcre2_dfa_match(). See pcreapi for a description of their effects.
1017
1018             anchored                  set PCRE2_ANCHORED
1019             endanchored               set PCRE2_ENDANCHORED
1020             dfa_restart               set PCRE2_DFA_RESTART
1021             dfa_shortest              set PCRE2_DFA_SHORTEST
1022             no_jit                    set PCRE2_NO_JIT
1023             no_utf_check              set PCRE2_NO_UTF_CHECK
1024             notbol                    set PCRE2_NOTBOL
1025             notempty                  set PCRE2_NOTEMPTY
1026             notempty_atstart          set PCRE2_NOTEMPTY_ATSTART
1027             noteol                    set PCRE2_NOTEOL
1028             partial_hard (or ph)      set PCRE2_PARTIAL_HARD
1029             partial_soft (or ps)      set PCRE2_PARTIAL_SOFT
1030
1031       The partial matching modifiers are provided with abbreviations  because
1032       they appear frequently in tests.
1033
1034       If  the posix or posix_nosub modifier was present on the pattern, caus‐
1035       ing the POSIX wrapper API to be used, the only option-setting modifiers
1036       that have any effect are notbol, notempty, and noteol, causing REG_NOT‐
1037       BOL, REG_NOTEMPTY,  and  REG_NOTEOL,  respectively,  to  be  passed  to
1038       regexec(). The other modifiers are ignored, with a warning message.
1039
1040       There  is one additional modifier that can be used with the POSIX wrap‐
1041       per. It is ignored (with a warning) if used for non-POSIX matching.
1042
1043             posix_startend=<n>[:<m>]
1044
1045       This causes the subject string to be  passed  to  regexec()  using  the
1046       REG_STARTEND  option,  which  uses offsets to specify which part of the
1047       string is searched. If only one number is  given,  the  end  offset  is
1048       passed  as  the end of the subject string. For more detail of REG_STAR‐
1049       TEND, see the pcre2posix documentation. If the subject string  contains
1050       binary  zeros  (coded  as escapes such as \x{00} because pcre2test does
1051       not support actual binary zeros in its input), you must use posix_star‐
1052       tend to specify its length.
1053
1054   Setting match controls
1055
1056       The  following  modifiers  affect the matching process or request addi‐
1057       tional information. Some of them may also be  specified  on  a  pattern
1058       line  (see  above), in which case they apply to every subject line that
1059       is matched against that pattern.
1060
1061             aftertext                  show text after match
1062             allaftertext               show text after captures
1063             allcaptures                show all captures
1064             allvector                  show the entire ovector
1065             allusedtext                show all consulted text (non-JIT only)
1066             altglobal                  alternative global matching
1067             callout_capture            show captures at callout time
1068             callout_data=<n>           set a value to pass via callouts
1069             callout_error=<n>[:<m>]    control callout error
1070             callout_extra              show extra callout information
1071             callout_fail=<n>[:<m>]     control callout failure
1072             callout_no_where           do not show position of a callout
1073             callout_none               do not supply a callout function
1074             copy=<number or name>      copy captured substring
1075             depth_limit=<n>            set a depth limit
1076             dfa                        use pcre2_dfa_match()
1077             find_limits                find match and depth limits
1078             get=<number or name>       extract captured substring
1079             getall                     extract all captured substrings
1080         /g  global                     global matching
1081             heap_limit=<n>             set a limit on heap memory (Kbytes)
1082             jitstack=<n>               set size of JIT stack
1083             mark                       show mark values
1084             match_limit=<n>            set a match limit
1085             memory                     show heap memory usage
1086             null_context               match with a NULL context
1087             offset=<n>                 set starting offset
1088             offset_limit=<n>           set offset limit
1089             ovector=<n>                set size of output vector
1090             recursion_limit=<n>        obsolete synonym for depth_limit
1091             replace=<string>           specify a replacement string
1092             startchar                  show startchar when relevant
1093             startoffset=<n>            same as offset=<n>
1094             substitute_callout         use substitution callouts
1095             substitute_extedded        use PCRE2_SUBSTITUTE_EXTENDED
1096             substitute_skip=<n>        skip substitution number n
1097             substitute_overflow_length use PCRE2_SUBSTITUTE_OVERFLOW_LENGTH
1098             substitute_stop=<n>        skip substitution number n and greater
1099             substitute_unknown_unset   use PCRE2_SUBSTITUTE_UNKNOWN_UNSET
1100             substitute_unset_empty     use PCRE2_SUBSTITUTE_UNSET_EMPTY
1101             zero_terminate             pass the subject as zero-terminated
1102
1103       The effects of these modifiers are described in the following sections.
1104       When  matching  via the POSIX wrapper API, the aftertext, allaftertext,
1105       and ovector subject modifiers work as described below. All other  modi‐
1106       fiers are either ignored, with a warning message, or cause an error.
1107
1108   Showing more text
1109
1110       The  aftertext modifier requests that as well as outputting the part of
1111       the subject string that matched the entire pattern, pcre2test should in
1112       addition output the remainder of the subject string. This is useful for
1113       tests where the subject contains multiple copies of the same substring.
1114       The  allaftertext  modifier  requests the same action for captured sub‐
1115       strings as well as the main matched substring. In each case the remain‐
1116       der is output on the following line with a plus character following the
1117       capture number.
1118
1119       The allusedtext modifier requests that all the text that was  consulted
1120       during  a  successful pattern match by the interpreter should be shown,
1121       for both full and partial matches. This feature is  not  supported  for
1122       JIT  matching,  and if requested with JIT it is ignored (with a warning
1123       message). Setting this modifier affects the output if there is a  look‐
1124       behind  at  the start of a match, or, for a complete match, a lookahead
1125       at the end, or if \K is used in the pattern. Characters that precede or
1126       follow  the start and end of the actual match are indicated in the out‐
1127       put by '<' or '>' characters underneath them.  Here is an example:
1128
1129           re> /(?<=pqr)abc(?=xyz)/
1130         data> 123pqrabcxyz456\=allusedtext
1131          0: pqrabcxyz
1132             <<<   >>>
1133         data> 123pqrabcxy\=ph,allusedtext
1134         Partial match: pqrabcxy
1135                        <<<
1136
1137       The first, complete match shows that the matched string is "abc",  with
1138       the  preceding  and  following strings "pqr" and "xyz" having been con‐
1139       sulted during the match (when processing the assertions).  The  partial
1140       match can indicate only the preceding string.
1141
1142       The  startchar  modifier  requests  that the starting character for the
1143       match be indicated, if it is different to  the  start  of  the  matched
1144       string. The only time when this occurs is when \K has been processed as
1145       part of the match. In this situation, the output for the matched string
1146       is  displayed  from  the  starting  character instead of from the match
1147       point, with circumflex characters under  the  earlier  characters.  For
1148       example:
1149
1150           re> /abc\Kxyz/
1151         data> abcxyz\=startchar
1152          0: abcxyz
1153             ^^^
1154
1155       Unlike  allusedtext, the startchar modifier can be used with JIT.  How‐
1156       ever, these two modifiers are mutually exclusive.
1157
1158   Showing the value of all capture groups
1159
1160       The allcaptures modifier requests that the values of all potential cap‐
1161       tured parentheses be output after a match. By default, only those up to
1162       the highest one actually used in the match are output (corresponding to
1163       the  return  code from pcre2_match()). Groups that did not take part in
1164       the match are output as "<unset>". This modifier is  not  relevant  for
1165       DFA  matching (which does no capturing) and does not apply when replace
1166       is specified; it is ignored, with a warning message, if present.
1167
1168   Showing the entire ovector, for all outcomes
1169
1170       The allvector modifier requests that the entire ovector be shown, what‐
1171       ever the outcome of the match. Compare allcaptures, which shows only up
1172       to the maximum number of capture groups for the pattern, and then  only
1173       for  a  successful  complete  non-DFA  match. This modifier, which acts
1174       after any match result, and also for DFA matching, provides a means  of
1175       checking  that there are no unexpected modifications to ovector fields.
1176       Before each match attempt, the ovector is filled with a special  value,
1177       and   if   this  is  found  in  both  elements  of  a  capturing  pair,
1178       "<unchanged>" is output. After a successful match, this applies to  all
1179       groups  after the maximum capture group for the pattern. In other cases
1180       it applies to the entire ovector. After a partial match, the first  two
1181       elements  are  the only ones that should be set. After a DFA match, the
1182       amount of ovector that is used depends on the number  of  matches  that
1183       were found.
1184
1185   Testing pattern callouts
1186
1187       A  callout function is supplied when pcre2test calls the library match‐
1188       ing functions, unless callout_none is specified. Its behaviour  can  be
1189       controlled  by  various  modifiers  listed above whose names begin with
1190       callout_. Details are given in the section entitled  "Callouts"  below.
1191       Testing  callouts  from  pcre2_substitute()  is  decribed separately in
1192       "Testing the substitution function" below.
1193
1194   Finding all matches in a string
1195
1196       Searching for all possible matches within a subject can be requested by
1197       the  global  or altglobal modifier. After finding a match, the matching
1198       function is called again to search the remainder of  the  subject.  The
1199       difference  between  global  and  altglobal is that the former uses the
1200       start_offset argument to pcre2_match() or  pcre2_dfa_match()  to  start
1201       searching  at  a new point within the entire string (which is what Perl
1202       does), whereas the latter passes over a shortened subject. This makes a
1203       difference to the matching process if the pattern begins with a lookbe‐
1204       hind assertion (including \b or \B).
1205
1206       If an empty string  is  matched,  the  next  match  is  done  with  the
1207       PCRE2_NOTEMPTY_ATSTART and PCRE2_ANCHORED flags set, in order to search
1208       for another, non-empty, match at the same point in the subject. If this
1209       match  fails,  the  start  offset  is advanced, and the normal match is
1210       retried. This imitates the way Perl handles such cases when  using  the
1211       /g  modifier  or  the  split()  function. Normally, the start offset is
1212       advanced by one character, but if  the  newline  convention  recognizes
1213       CRLF  as  a newline, and the current character is CR followed by LF, an
1214       advance of two characters occurs.
1215
1216   Testing substring extraction functions
1217
1218       The copy  and  get  modifiers  can  be  used  to  test  the  pcre2_sub‐
1219       string_copy_xxx() and pcre2_substring_get_xxx() functions.  They can be
1220       given more than once, and each can specify a capture group name or num‐
1221       ber, for example:
1222
1223          abcd\=copy=1,copy=3,get=G1
1224
1225       If  the  #subject command is used to set default copy and/or get lists,
1226       these can be unset by specifying a negative number to cancel  all  num‐
1227       bered groups and an empty name to cancel all named groups.
1228
1229       The  getall  modifier  tests pcre2_substring_list_get(), which extracts
1230       all captured substrings.
1231
1232       If the subject line is successfully matched, the  substrings  extracted
1233       by  the  convenience  functions  are  output  with C, G, or L after the
1234       string number instead of a colon. This is in  addition  to  the  normal
1235       full  list.  The string length (that is, the return from the extraction
1236       function) is given in parentheses after each substring, followed by the
1237       name when the extraction was by name.
1238
1239   Testing the substitution function
1240
1241       If  the  replace  modifier  is  set, the pcre2_substitute() function is
1242       called instead of one of the matching functions. Note that  replacement
1243       strings  cannot  contain commas, because a comma signifies the end of a
1244       modifier. This is not thought to be an issue in a test program.
1245
1246       Unlike subject strings, pcre2test does not process replacement  strings
1247       for  escape  sequences. In UTF mode, a replacement string is checked to
1248       see if it is a valid UTF-8 string. If so, it is correctly converted  to
1249       a  UTF  string of the appropriate code unit width. If it is not a valid
1250       UTF-8 string, the individual code units are copied directly. This  pro‐
1251       vides a means of passing an invalid UTF-8 string for testing purposes.
1252
1253       The  following modifiers set options (in additional to the normal match
1254       options) for pcre2_substitute():
1255
1256         global                      PCRE2_SUBSTITUTE_GLOBAL
1257         substitute_extended         PCRE2_SUBSTITUTE_EXTENDED
1258         substitute_overflow_length  PCRE2_SUBSTITUTE_OVERFLOW_LENGTH
1259         substitute_unknown_unset    PCRE2_SUBSTITUTE_UNKNOWN_UNSET
1260         substitute_unset_empty      PCRE2_SUBSTITUTE_UNSET_EMPTY
1261
1262
1263       After a successful substitution, the modified string  is  output,  pre‐
1264       ceded  by the number of replacements. This may be zero if there were no
1265       matches. Here is a simple example of a substitution test:
1266
1267         /abc/replace=xxx
1268             =abc=abc=
1269          1: =xxx=abc=
1270             =abc=abc=\=global
1271          2: =xxx=xxx=
1272
1273       Subject and replacement strings should be kept relatively short  (fewer
1274       than  256 characters) for substitution tests, as fixed-size buffers are
1275       used. To make it easy to test for buffer overflow, if  the  replacement
1276       string  starts  with a number in square brackets, that number is passed
1277       to pcre2_substitute() as the  size  of  the  output  buffer,  with  the
1278       replacement  string  starting at the next character. Here is an example
1279       that tests the edge case:
1280
1281         /abc/
1282             123abc123\=replace=[10]XYZ
1283          1: 123XYZ123
1284             123abc123\=replace=[9]XYZ
1285         Failed: error -47: no more memory
1286
1287       The   default   action   of    pcre2_substitute()    is    to    return
1288       PCRE2_ERROR_NOMEMORY  when  the output buffer is too small. However, if
1289       the PCRE2_SUBSTITUTE_OVERFLOW_LENGTH option is set (by using  the  sub‐
1290       stitute_overflow_length  modifier),  pcre2_substitute() continues to go
1291       through the motions of matching and substituting  (but  not  doing  any
1292       callouts),  in  order  to  compute the size of buffer that is required.
1293       When this happens, pcre2test shows the required  buffer  length  (which
1294       includes space for the trailing zero) as part of the error message. For
1295       example:
1296
1297         /abc/substitute_overflow_length
1298             123abc123\=replace=[9]XYZ
1299         Failed: error -47: no more memory: 10 code units are needed
1300
1301       A replacement string is ignored with POSIX and DFA matching. Specifying
1302       partial  matching  provokes  an  error return ("bad option value") from
1303       pcre2_substitute().
1304
1305   Testing substitute callouts
1306
1307       If the substitute_callout modifier is set, a substitution callout func‐
1308       tion  is set up. The null_context modifier must not be set, because the
1309       address of the callout function is passed in a match context. When  the
1310       callout  function  is  called (after each substitution), details of the
1311       the input and output strings are output. For example:
1312
1313         /abc/g,replace=<$0>,substitute_callout
1314             abcdefabcpqr
1315          1(1) Old 0 3 "abc" New 0 5 "<abc>"
1316          2(1) Old 6 9 "abc" New 8 13 "<abc>"
1317          2: <abc>def<abc>pqr
1318
1319       The first number on each callout line is  the  count  of  matches.  The
1320       parenthesized number is the number of pairs that are set in the ovector
1321       (that is, one more than the number of capturing groups that were  set).
1322       Then are listed the offsets of the old substring, its contents, and the
1323       same for the replacement.
1324
1325       By default, the  substitution  callout  function  returns  zero,  which
1326       accepts the replacement and causes matching to continue if /g was used.
1327       Two further modifiers can be used to test other return values. If  sub‐
1328       stitute_skip  is  set to a value greater than zero the callout function
1329       returns +1 for the match of that number, and similarly  substitute_stop
1330       returns  -1.  These cause the replacement to be rejected, and -1 causes
1331       no further matching to take place. If either of them are  set,  substi‐
1332       tute_callout is assumed. For example:
1333
1334         /abc/g,replace=<$0>,substitute_skip=1
1335             abcdefabcpqr
1336          1(1) Old 0 3 "abc" New 0 5 "<abc> SKIPPED"
1337          2(1) Old 6 9 "abc" New 6 11 "<abc>"
1338          2: abcdef<abc>pqr
1339             abcdefabcpqr\=substitute_stop=1
1340          1(1) Old 0 3 "abc" New 0 5 "<abc> STOPPED"
1341          1: abcdefabcpqr
1342
1343       If both are set for the same number, stop takes precedence. Only a sin‐
1344       gle skip or stop is supported, which is sufficient for testing that the
1345       feature works.
1346
1347   Setting the JIT stack size
1348
1349       The  jitstack modifier provides a way of setting the maximum stack size
1350       that is used by the just-in-time optimization code. It  is  ignored  if
1351       JIT  optimization is not being used. The value is a number of kibibytes
1352       (units of 1024 bytes). Setting zero reverts to the  default  of  32KiB.
1353       Providing a stack that is larger than the default is necessary only for
1354       very complicated patterns. If jitstack is set  non-zero  on  a  subject
1355       line it overrides any value that was set on the pattern.
1356
1357   Setting heap, match, and depth limits
1358
1359       The  heap_limit,  match_limit, and depth_limit modifiers set the appro‐
1360       priate limits in the match context. These values are ignored  when  the
1361       find_limits modifier is specified.
1362
1363   Finding minimum limits
1364
1365       If  the  find_limits  modifier  is present on a subject line, pcre2test
1366       calls the relevant matching function several times,  setting  different
1367       values    in    the    match    context   via   pcre2_set_heap_limit(),
1368       pcre2_set_match_limit(), or pcre2_set_depth_limit() until it finds  the
1369       minimum  values  for  each  parameter that allows the match to complete
1370       without error. If JIT is being used, only the match limit is relevant.
1371
1372       When using this modifier, the pattern should not contain any limit set‐
1373       tings  such  as  (*LIMIT_MATCH=...)  within  it.  If  such a setting is
1374       present and is lower than the minimum matching value, the minimum value
1375       cannot  be  found because pcre2_set_match_limit() etc. are only able to
1376       reduce the value of an in-pattern limit; they cannot increase it.
1377
1378       For non-DFA matching, the minimum depth_limit number is  a  measure  of
1379       how much nested backtracking happens (that is, how deeply the pattern's
1380       tree is searched). In the case of DFA  matching,  depth_limit  controls
1381       the  depth of recursive calls of the internal function that is used for
1382       handling pattern recursion, lookaround assertions, and atomic groups.
1383
1384       For non-DFA matching, the match_limit number is a measure of the amount
1385       of backtracking that takes place, and learning the minimum value can be
1386       instructive. For most simple matches, the number is  quite  small,  but
1387       for  patterns with very large numbers of matching possibilities, it can
1388       become large very quickly with increasing length of subject string.  In
1389       the  case  of  DFA  matching,  match_limit controls the total number of
1390       calls, both recursive and non-recursive, to the internal matching func‐
1391       tion, thus controlling the overall amount of computing resource that is
1392       used.
1393
1394       For both  kinds  of  matching,  the  heap_limit  number,  which  is  in
1395       kibibytes  (units of 1024 bytes), limits the amount of heap memory used
1396       for matching. A value of zero disables the use of any heap memory; many
1397       simple  pattern  matches can be done without using the heap, so zero is
1398       not an unreasonable setting.
1399
1400   Showing MARK names
1401
1402
1403       The mark modifier causes the names from backtracking control verbs that
1404       are  returned from calls to pcre2_match() to be displayed. If a mark is
1405       returned for a match, non-match, or partial match, pcre2test shows  it.
1406       For  a  match, it is on a line by itself, tagged with "MK:". Otherwise,
1407       it is added to the non-match message.
1408
1409   Showing memory usage
1410
1411       The memory modifier causes pcre2test to log the sizes of all heap  mem‐
1412       ory   allocation  and  freeing  calls  that  occur  during  a  call  to
1413       pcre2_match() or pcre2_dfa_match().  These  occur  only  when  a  match
1414       requires  a bigger vector than the default for remembering backtracking
1415       points (pcre2_match()) or for internal  workspace  (pcre2_dfa_match()).
1416       In  many cases there will be no heap memory used and therefore no addi‐
1417       tional output. No heap memory is allocated during matching with JIT, so
1418       in  that  case the memory modifier never has any effect. For this modi‐
1419       fier to work, the null_context modifier must not be  set  on  both  the
1420       pattern and the subject, though it can be set on one or the other.
1421
1422   Setting a starting offset
1423
1424       The  offset  modifier  sets  an  offset  in the subject string at which
1425       matching starts. Its value is a number of code units, not characters.
1426
1427   Setting an offset limit
1428
1429       The offset_limit modifier sets a limit for  unanchored  matches.  If  a
1430       match cannot be found starting at or before this offset in the subject,
1431       a "no match" return is given. The data value is a number of code units,
1432       not  characters. When this modifier is used, the use_offset_limit modi‐
1433       fier must have been set for the pattern; if not, an error is generated.
1434
1435   Setting the size of the output vector
1436
1437       The ovector modifier applies only to  the  subject  line  in  which  it
1438       appears,  though  of  course  it can also be used to set a default in a
1439       #subject command. It specifies the number of pairs of offsets that  are
1440       available for storing matching information. The default is 15.
1441
1442       A  value of zero is useful when testing the POSIX API because it causes
1443       regexec() to be called with a NULL capture vector. When not testing the
1444       POSIX  API,  a  value  of  zero  is used to cause pcre2_match_data_cre‐
1445       ate_from_pattern() to be called, in order to create a  match  block  of
1446       exactly the right size for the pattern. (It is not possible to create a
1447       match block with a zero-length ovector; there is always  at  least  one
1448       pair of offsets.)
1449
1450   Passing the subject as zero-terminated
1451
1452       By default, the subject string is passed to a native API matching func‐
1453       tion with its correct length. In order to test the facility for passing
1454       a  zero-terminated  string, the zero_terminate modifier is provided. It
1455       causes the length to be passed as PCRE2_ZERO_TERMINATED. When  matching
1456       via the POSIX interface, this modifier is ignored, with a warning.
1457
1458       When  testing  pcre2_substitute(), this modifier also has the effect of
1459       passing the replacement string as zero-terminated.
1460
1461   Passing a NULL context
1462
1463       Normally,  pcre2test  passes  a   context   block   to   pcre2_match(),
1464       pcre2_dfa_match(),  pcre2_jit_match()  or  pcre2_substitute().   If the
1465       null_context modifier is set, however, NULL  is  passed.  This  is  for
1466       testing  that  the matching and substitution functions behave correctly
1467       in this case (they use default values). This modifier  cannot  be  used
1468       with the find_limits or substitute_callout modifiers.
1469

THE ALTERNATIVE MATCHING FUNCTION

1471
1472       By  default,  pcre2test  uses  the  standard  PCRE2  matching function,
1473       pcre2_match() to match each subject line. PCRE2 also supports an alter‐
1474       native  matching  function, pcre2_dfa_match(), which operates in a dif‐
1475       ferent way, and has some restrictions. The differences between the  two
1476       functions are described in the pcre2matching documentation.
1477
1478       If  the dfa modifier is set, the alternative matching function is used.
1479       This function finds all possible matches at a given point in  the  sub‐
1480       ject.  If,  however, the dfa_shortest modifier is set, processing stops
1481       after the first match is found. This is always  the  shortest  possible
1482       match.
1483

DEFAULT OUTPUT FROM pcre2test

1485
1486       This  section  describes  the output when the normal matching function,
1487       pcre2_match(), is being used.
1488
1489       When a match succeeds, pcre2test outputs  the  list  of  captured  sub‐
1490       strings,  starting  with number 0 for the string that matched the whole
1491       pattern.   Otherwise,  it  outputs  "No  match"  when  the  return   is
1492       PCRE2_ERROR_NOMATCH,  or  "Partial  match:"  followed  by the partially
1493       matching substring when the return is PCRE2_ERROR_PARTIAL.  (Note  that
1494       this  is  the  entire  substring  that was inspected during the partial
1495       match; it may include characters before the actual  match  start  if  a
1496       lookbehind assertion, \K, \b, or \B was involved.)
1497
1498       For any other return, pcre2test outputs the PCRE2 negative error number
1499       and a short descriptive phrase. If the error is  a  failed  UTF  string
1500       check,  the  code  unit offset of the start of the failing character is
1501       also output. Here is an example of an interactive pcre2test run.
1502
1503         $ pcre2test
1504         PCRE2 version 10.22 2016-07-29
1505
1506           re> /^abc(\d+)/
1507         data> abc123
1508          0: abc123
1509          1: 123
1510         data> xyz
1511         No match
1512
1513       Unset capturing substrings that are not followed by one that is set are
1514       not shown by pcre2test unless the allcaptures modifier is specified. In
1515       the following example, there are two capturing substrings, but when the
1516       first  data  line is matched, the second, unset substring is not shown.
1517       An "internal" unset substring is shown as "<unset>", as for the  second
1518       data line.
1519
1520           re> /(a)|(b)/
1521         data> a
1522          0: a
1523          1: a
1524         data> b
1525          0: b
1526          1: <unset>
1527          2: b
1528
1529       If  the strings contain any non-printing characters, they are output as
1530       \xhh escapes if the value is less than 256 and UTF  mode  is  not  set.
1531       Otherwise they are output as \x{hh...} escapes. See below for the defi‐
1532       nition of non-printing characters. If the aftertext  modifier  is  set,
1533       the  output  for substring 0 is followed by the the rest of the subject
1534       string, identified by "0+" like this:
1535
1536           re> /cat/aftertext
1537         data> cataract
1538          0: cat
1539          0+ aract
1540
1541       If global matching is requested, the  results  of  successive  matching
1542       attempts are output in sequence, like this:
1543
1544           re> /\Bi(\w\w)/g
1545         data> Mississippi
1546          0: iss
1547          1: ss
1548          0: iss
1549          1: ss
1550          0: ipp
1551          1: pp
1552
1553       "No  match" is output only if the first match attempt fails. Here is an
1554       example of a failure message (the offset 4 that  is  specified  by  the
1555       offset modifier is past the end of the subject string):
1556
1557           re> /xyz/
1558         data> xyz\=offset=4
1559         Error -24 (bad offset value)
1560
1561       Note that whereas patterns can be continued over several lines (a plain
1562       ">" prompt is used for continuations), subject lines may  not.  However
1563       newlines can be included in a subject by means of the \n escape (or \r,
1564       \r\n, etc., depending on the newline sequence setting).
1565

OUTPUT FROM THE ALTERNATIVE MATCHING FUNCTION

1567
1568       When the alternative matching function, pcre2_dfa_match(), is used, the
1569       output  consists  of  a list of all the matches that start at the first
1570       point in the subject where there is at least one match. For example:
1571
1572           re> /(tang|tangerine|tan)/
1573         data> yellow tangerine\=dfa
1574          0: tangerine
1575          1: tang
1576          2: tan
1577
1578       Using the normal matching function on this data finds only "tang".  The
1579       longest  matching  string  is  always  given first (and numbered zero).
1580       After a PCRE2_ERROR_PARTIAL return, the  output  is  "Partial  match:",
1581       followed  by  the  partially  matching substring. Note that this is the
1582       entire substring that was inspected during the partial  match;  it  may
1583       include characters before the actual match start if a lookbehind asser‐
1584       tion, \b, or \B was involved. (\K is not supported for DFA matching.)
1585
1586       If global matching is requested, the search for further matches resumes
1587       at the end of the longest match. For example:
1588
1589           re> /(tang|tangerine|tan)/g
1590         data> yellow tangerine and tangy sultana\=dfa
1591          0: tangerine
1592          1: tang
1593          2: tan
1594          0: tang
1595          1: tan
1596          0: tan
1597
1598       The  alternative  matching function does not support substring capture,
1599       so the modifiers that are concerned with captured  substrings  are  not
1600       relevant.
1601

RESTARTING AFTER A PARTIAL MATCH

1603
1604       When  the  alternative matching function has given the PCRE2_ERROR_PAR‐
1605       TIAL return, indicating that the subject partially matched the pattern,
1606       you  can restart the match with additional subject data by means of the
1607       dfa_restart modifier. For example:
1608
1609           re> /^\d?\d(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\d\d$/
1610         data> 23ja\=ps,dfa
1611         Partial match: 23ja
1612         data> n05\=dfa,dfa_restart
1613          0: n05
1614
1615       For further information about partial matching,  see  the  pcre2partial
1616       documentation.
1617

CALLOUTS

1619
1620       If the pattern contains any callout requests, pcre2test's callout func‐
1621       tion is called during matching unless callout_none is  specified.  This
1622       works with both matching functions, and with JIT, though there are some
1623       differences in behaviour. The output for callouts with numerical  argu‐
1624       ments and those with string arguments is slightly different.
1625
1626   Callouts with numerical arguments
1627
1628       By default, the callout function displays the callout number, the start
1629       and current positions in the subject text at the callout time, and  the
1630       next pattern item to be tested. For example:
1631
1632         --->pqrabcdef
1633           0    ^  ^     \d
1634
1635       This  output  indicates  that  callout  number  0  occurred for a match
1636       attempt starting at the fourth character of the  subject  string,  when
1637       the  pointer  was  at  the seventh character, and when the next pattern
1638       item was \d. Just one circumflex is output if  the  start  and  current
1639       positions  are  the same, or if the current position precedes the start
1640       position, which can happen if the callout is in a lookbehind assertion.
1641
1642       Callouts numbered 255 are assumed to be automatic callouts, inserted as
1643       a result of the auto_callout pattern modifier. In this case, instead of
1644       showing the callout number, the offset in the pattern,  preceded  by  a
1645       plus, is output. For example:
1646
1647           re> /\d?[A-E]\*/auto_callout
1648         data> E*
1649         --->E*
1650          +0 ^      \d?
1651          +3 ^      [A-E]
1652          +8 ^^     \*
1653         +10 ^ ^
1654          0: E*
1655
1656       If a pattern contains (*MARK) items, an additional line is output when‐
1657       ever a change of latest mark is passed to  the  callout  function.  For
1658       example:
1659
1660           re> /a(*MARK:X)bc/auto_callout
1661         data> abc
1662         --->abc
1663          +0 ^       a
1664          +1 ^^      (*MARK:X)
1665         +10 ^^      b
1666         Latest Mark: X
1667         +11 ^ ^     c
1668         +12 ^  ^
1669          0: abc
1670
1671       The  mark  changes between matching "a" and "b", but stays the same for
1672       the rest of the match, so nothing more is output. If, as  a  result  of
1673       backtracking,  the  mark  reverts to being unset, the text "<unset>" is
1674       output.
1675
1676   Callouts with string arguments
1677
1678       The output for a callout with a string argument is similar, except that
1679       instead  of outputting a callout number before the position indicators,
1680       the callout string and its offset in  the  pattern  string  are  output
1681       before  the reflection of the subject string, and the subject string is
1682       reflected for each callout. For example:
1683
1684           re> /^ab(?C'first')cd(?C"second")ef/
1685         data> abcdefg
1686         Callout (7): 'first'
1687         --->abcdefg
1688             ^ ^         c
1689         Callout (20): "second"
1690         --->abcdefg
1691             ^   ^       e
1692          0: abcdef
1693
1694
1695   Callout modifiers
1696
1697       The callout function in pcre2test returns zero (carry on  matching)  by
1698       default,  but  you can use a callout_fail modifier in a subject line to
1699       change this and other parameters of the callout (see below).
1700
1701       If the callout_capture modifier is set, the current captured groups are
1702       output when a callout occurs. This is useful only for non-DFA matching,
1703       as pcre2_dfa_match() does not support capturing,  so  no  captures  are
1704       ever shown.
1705
1706       The normal callout output, showing the callout number or pattern offset
1707       (as described above) is suppressed if the callout_no_where modifier  is
1708       set.
1709
1710       When  using  the  interpretive  matching function pcre2_match() without
1711       JIT, setting the callout_extra modifier causes additional  output  from
1712       pcre2test's  callout function to be generated. For the first callout in
1713       a match attempt at a new starting position in the subject,  "New  match
1714       attempt"  is output. If there has been a backtrack since the last call‐
1715       out (or start of matching if this is the first callout), "Backtrack" is
1716       output,  followed  by  "No other matching paths" if the backtrack ended
1717       the previous match attempt. For example:
1718
1719          re> /(a+)b/auto_callout,no_start_optimize,no_auto_possess
1720         data> aac\=callout_extra
1721         New match attempt
1722         --->aac
1723          +0 ^       (
1724          +1 ^       a+
1725          +3 ^ ^     )
1726          +4 ^ ^     b
1727         Backtrack
1728         --->aac
1729          +3 ^^      )
1730          +4 ^^      b
1731         Backtrack
1732         No other matching paths
1733         New match attempt
1734         --->aac
1735          +0  ^      (
1736          +1  ^      a+
1737          +3  ^^     )
1738          +4  ^^     b
1739         Backtrack
1740         No other matching paths
1741         New match attempt
1742         --->aac
1743          +0   ^     (
1744          +1   ^     a+
1745         Backtrack
1746         No other matching paths
1747         New match attempt
1748         --->aac
1749          +0    ^    (
1750          +1    ^    a+
1751         No match
1752
1753       Notice that various optimizations must be turned off if  you  want  all
1754       possible  matching  paths  to  be  scanned. If no_start_optimize is not
1755       used, there is an immediate "no match", without any  callouts,  because
1756       the  starting  optimization  fails to find "b" in the subject, which it
1757       knows must be present for any match. If no_auto_possess  is  not  used,
1758       the  "a+"  item is turned into "a++", which reduces the number of back‐
1759       tracks.
1760
1761       The callout_extra modifier has no effect if used with the DFA  matching
1762       function, or with JIT.
1763
1764   Return values from callouts
1765
1766       The  default  return  from  the  callout function is zero, which allows
1767       matching to continue. The callout_fail modifier can be given one or two
1768       numbers. If there is only one number, 1 is returned instead of 0 (caus‐
1769       ing matching to backtrack) when a callout of that number is reached. If
1770       two  numbers  (<n>:<m>)  are  given,  1 is returned when callout <n> is
1771       reached and there have been at least <m>  callouts.  The  callout_error
1772       modifier is similar, except that PCRE2_ERROR_CALLOUT is returned, caus‐
1773       ing the entire matching process to be aborted. If both these  modifiers
1774       are  set  for  the same callout number, callout_error takes precedence.
1775       Note that callouts with string arguments are always  given  the  number
1776       zero.
1777
1778       The  callout_data  modifier can be given an unsigned or a negative num‐
1779       ber.  This is set as the "user data" that is  passed  to  the  matching
1780       function,  and  passed  back  when the callout function is invoked. Any
1781       value other than zero is used as  a  return  from  pcre2test's  callout
1782       function.
1783
1784       Inserting callouts can be helpful when using pcre2test to check compli‐
1785       cated regular expressions. For further information about callouts,  see
1786       the pcre2callout documentation.
1787

NON-PRINTING CHARACTERS

1789
1790       When pcre2test is outputting text in the compiled version of a pattern,
1791       bytes other than 32-126 are always treated as  non-printing  characters
1792       and are therefore shown as hex escapes.
1793
1794       When  pcre2test  is outputting text that is a matched part of a subject
1795       string, it behaves in the same way, unless a different locale has  been
1796       set  for  the  pattern  (using  the locale modifier). In this case, the
1797       isprint() function is used to  distinguish  printing  and  non-printing
1798       characters.
1799

SAVING AND RESTORING COMPILED PATTERNS

1801
1802       It  is  possible  to  save  compiled patterns on disc or elsewhere, and
1803       reload them later, subject to a number of restrictions. JIT data cannot
1804       be  saved.  The host on which the patterns are reloaded must be running
1805       the same version of PCRE2, with the same code unit width, and must also
1806       have  the  same  endianness,  pointer width and PCRE2_SIZE type. Before
1807       compiled patterns can be saved they must be serialized, that  is,  con‐
1808       verted  to a stream of bytes. A single byte stream may contain any num‐
1809       ber of compiled patterns, but they must  all  use  the  same  character
1810       tables. A single copy of the tables is included in the byte stream (its
1811       size is 1088 bytes).
1812
1813       The functions whose names begin  with  pcre2_serialize_  are  used  for
1814       serializing  and de-serializing. They are described in the pcre2serial‐
1815       ize  documentation.  In  this  section  we  describe  the  features  of
1816       pcre2test that can be used to test these functions.
1817
1818       Note  that  "serialization" in PCRE2 does not convert compiled patterns
1819       to an abstract format like Java or .NET. It  just  makes  a  reloadable
1820       byte code stream.  Hence the restrictions on reloading mentioned above.
1821
1822       In  pcre2test,  when  a pattern with push modifier is successfully com‐
1823       piled, it is pushed onto a stack of compiled  patterns,  and  pcre2test
1824       expects  the next line to contain a new pattern (or command) instead of
1825       a subject line. By contrast, the pushcopy modifier causes a copy of the
1826       compiled  pattern  to  be  stacked,  leaving the original available for
1827       immediate matching. By using push and/or pushcopy, a number of patterns
1828       can  be  compiled  and  retained. These modifiers are incompatible with
1829       posix, and control modifiers that act at match time are ignored (with a
1830       message)  for the stacked patterns. The jitverify modifier applies only
1831       at compile time.
1832
1833       The command
1834
1835         #save <filename>
1836
1837       causes all the stacked patterns to be serialized and the result written
1838       to  the named file. Afterwards, all the stacked patterns are freed. The
1839       command
1840
1841         #load <filename>
1842
1843       reads the data in the file, and then arranges for it to  be  de-serial‐
1844       ized,  with the resulting compiled patterns added to the pattern stack.
1845       The pattern on the top of the stack can be retrieved by the  #pop  com‐
1846       mand,  which  must  be  followed  by  lines  of subjects that are to be
1847       matched with the pattern, terminated as usual by an empty line  or  end
1848       of  file.  This  command  may be followed by a modifier list containing
1849       only control modifiers that act after a pattern has been  compiled.  In
1850       particular,  hex,  posix,  posix_nosub,  push,  and  pushcopy  are  not
1851       allowed, nor are any option-setting modifiers.  The JIT modifiers  are,
1852       however  permitted.  Here is an example that saves and reloads two pat‐
1853       terns.
1854
1855         /abc/push
1856         /xyz/push
1857         #save tempfile
1858         #load tempfile
1859         #pop info
1860         xyz
1861
1862         #pop jit,bincode
1863         abc
1864
1865       If jitverify is used with #pop, it does not  automatically  imply  jit,
1866       which is different behaviour from when it is used on a pattern.
1867
1868       The  #popcopy  command is analagous to the pushcopy modifier in that it
1869       makes current a copy of the topmost stack pattern, leaving the original
1870       still on the stack.
1871

SEE ALSO

1873
1874       pcre2(3),  pcre2api(3),  pcre2callout(3),  pcre2jit,  pcre2matching(3),
1875       pcre2partial(d), pcre2pattern(3), pcre2serialize(3).
1876

AUTHOR

1878
1879       Philip Hazel
1880       University Computing Service
1881       Cambridge, England.
1882

REVISION

1884
1885       Last updated: 30 July 2019
1886       Copyright (c) 1997-2019 University of Cambridge.
1887
1888
1889
1890PCRE 10.34                       30 July 2019                     PCRE2TEST(1)
Impressum