1PCRE2TEST(1)                General Commands Manual               PCRE2TEST(1)
2
3
4

NAME

6       pcre2test - a program for testing Perl-compatible regular expressions.
7

SYNOPSIS

9
10       pcre2test [options] [input file [output file]]
11
12       pcre2test is a test program for the PCRE2 regular expression libraries,
13       but it can also be used for  experimenting  with  regular  expressions.
14       This  document  describes the features of the test program; for details
15       of the regular expressions themselves, see the pcre2pattern  documenta‐
16       tion.  For  details  of  the  PCRE2  library  function  calls and their
17       options, see the pcre2api documentation.
18
19       The input for pcre2test is a sequence of  regular  expression  patterns
20       and  subject  strings  to  be matched. There are also command lines for
21       setting defaults and controlling some special actions. The output shows
22       the  result  of  each  match attempt. Modifiers on external or internal
23       command lines, the patterns, and the subject lines specify PCRE2  func‐
24       tion  options, control how the subject is processed, and what output is
25       produced.
26
27       As the original fairly simple PCRE library evolved,  it  acquired  many
28       different  features,  and  as  a  result, the original pcretest program
29       ended up with a lot of options in a messy, arcane  syntax  for  testing
30       all the features. The move to the new PCRE2 API provided an opportunity
31       to re-implement the test program as pcre2test, with a cleaner  modifier
32       syntax.  Nevertheless,  there are still many obscure modifiers, some of
33       which are specifically designed for use in conjunction  with  the  test
34       script  and  data  files that are distributed as part of PCRE2. All the
35       modifiers are documented here, some  without  much  justification,  but
36       many  of  them  are  unlikely  to  be  of  use  except when testing the
37       libraries.
38

PCRE2's 8-BIT, 16-BIT AND 32-BIT LIBRARIES

40
41       Different versions of the PCRE2 library can be built to support charac‐
42       ter  strings  that  are encoded in 8-bit, 16-bit, or 32-bit code units.
43       One, two, or  all  three  of  these  libraries  may  be  simultaneously
44       installed. The pcre2test program can be used to test all the libraries.
45       However, its own input and output are  always  in  8-bit  format.  When
46       testing  the  16-bit  or 32-bit libraries, patterns and subject strings
47       are converted to 16-bit or 32-bit format before  being  passed  to  the
48       library  functions.  Results are converted back to 8-bit code units for
49       output.
50
51       In the rest of this document, the names of library functions and struc‐
52       tures  are  given  in  generic  form,  for example, pcre_compile(). The
53       actual names used in the libraries have a suffix _8, _16,  or  _32,  as
54       appropriate.
55

INPUT ENCODING

57
58       Input  to  pcre2test is processed line by line, either by calling the C
59       library's fgets() function, or via the  libreadline  library.  In  some
60       Windows  environments  character 26 (hex 1A) causes an immediate end of
61       file, and no further data is read, so this character should be  avoided
62       unless you really want that action.
63
64       The  input  is  processed using using C's string functions, so must not
65       contain binary zeros, even though in  Unix-like  environments,  fgets()
66       treats  any  bytes  other  than newline as data characters. An error is
67       generated if a binary zero is encountered. By default subject lines are
68       processed for backslash escapes, which makes it possible to include any
69       data value in strings that are passed to the library for matching.  For
70       patterns,  there  is a facility for specifying some or all of the 8-bit
71       input characters as hexadecimal  pairs,  which  makes  it  possible  to
72       include binary zeros.
73
74   Input for the 16-bit and 32-bit libraries
75
76       When testing the 16-bit or 32-bit libraries, there is a need to be able
77       to generate character code points greater than 255 in the strings  that
78       are  passed to the library. For subject lines, backslash escapes can be
79       used. In addition, when the  utf  modifier  (see  "Setting  compilation
80       options" below) is set, the pattern and any following subject lines are
81       interpreted as UTF-8 strings and translated  to  UTF-16  or  UTF-32  as
82       appropriate.
83
84       For  non-UTF testing of wide characters, the utf8_input modifier can be
85       used. This is mutually exclusive with  utf,  and  is  allowed  only  in
86       16-bit  or  32-bit  mode.  It  causes the pattern and following subject
87       lines to be treated as UTF-8 according to the original definition  (RFC
88       2279), which allows for character values up to 0x7fffffff. Each charac‐
89       ter is placed in one 16-bit or 32-bit code unit (in  the  16-bit  case,
90       values greater than 0xffff cause an error to occur).
91
92       UTF-8  (in  its  original definition) is not capable of encoding values
93       greater than 0x7fffffff, but such values can be handled by  the  32-bit
94       library. When testing this library in non-UTF mode with utf8_input set,
95       if any character is preceded by the byte 0xff (which is an invalid byte
96       in  UTF-8)  0x80000000  is  added to the character's value. This is the
97       only way of passing such code points in a pattern string.  For  subject
98       strings, using an escape sequence is preferable.
99

COMMAND LINE OPTIONS

101
102       -8        If the 8-bit library has been built, this option causes it to
103                 be used (this is the default). If the 8-bit library  has  not
104                 been built, this option causes an error.
105
106       -16       If  the  16-bit library has been built, this option causes it
107                 to be used. If only the 16-bit library has been  built,  this
108                 is  the  default.  If  the 16-bit library has not been built,
109                 this option causes an error.
110
111       -32       If the 32-bit library has been built, this option  causes  it
112                 to  be  used. If only the 32-bit library has been built, this
113                 is the default. If the 32-bit library  has  not  been  built,
114                 this option causes an error.
115
116       -ac       Behave as if each pattern has the auto_callout modifier, that
117                 is, insert automatic callouts into every pattern that is com‐
118                 piled.
119
120       -AC       As  for  -ac,  but in addition behave as if each subject line
121                 has the callout_extra  modifier,  that  is,  show  additional
122                 information from callouts.
123
124       -b        Behave  as  if each pattern has the fullbincode modifier; the
125                 full internal binary form of the pattern is output after com‐
126                 pilation.
127
128       -C        Output  the  version  number  of  the  PCRE2 library, and all
129                 available information about the optional  features  that  are
130                 included,  and  then  exit  with  zero  exit  code. All other
131                 options are ignored. If both -C and -LM are  present,  which‐
132                 ever is first is recognized.
133
134       -C option Output  information  about a specific build-time option, then
135                 exit. This functionality is intended for use in scripts  such
136                 as  RunTest.  The  following options output the value and set
137                 the exit code as indicated:
138
139                   ebcdic-nl  the code for LF (= NL) in an EBCDIC environment:
140                                0x15 or 0x25
141                                0 if used in an ASCII environment
142                                exit code is always 0
143                   linksize   the configured internal link size (2, 3, or 4)
144                                exit code is set to the link size
145                   newline    the default newline setting:
146                                CR, LF, CRLF, ANYCRLF, ANY, or NUL
147                                exit code is always 0
148                   bsr        the default setting for what \R matches:
149                                ANYCRLF or ANY
150                                exit code is always 0
151
152                 The following options output 1 for true or 0 for  false,  and
153                 set the exit code to the same value:
154
155                   backslash-C  \C is supported (not locked out)
156                   ebcdic       compiled for an EBCDIC environment
157                   jit          just-in-time support is available
158                   pcre2-16     the 16-bit library was built
159                   pcre2-32     the 32-bit library was built
160                   pcre2-8      the 8-bit library was built
161                   unicode      Unicode support is available
162
163                 If  an  unknown  option is given, an error message is output;
164                 the exit code is 0.
165
166       -d        Behave as if each pattern has the debug modifier; the  inter‐
167                 nal form and information about the compiled pattern is output
168                 after compilation; -d is equivalent to -b -i.
169
170       -dfa      Behave as if each subject line has the dfa modifier; matching
171                 is  done  using the pcre2_dfa_match() function instead of the
172                 default pcre2_match().
173
174       -error number[,number,...]
175                 Call pcre2_get_error_message() for each of the error  numbers
176                 in  the  comma-separated list, display the resulting messages
177                 on the standard output, then exit with zero  exit  code.  The
178                 numbers  may  be  positive or negative. This is a convenience
179                 facility for PCRE2 maintainers.
180
181       -help     Output a brief summary these options and then exit.
182
183       -i        Behave as if each pattern has the info modifier;  information
184                 about the compiled pattern is given after compilation.
185
186       -jit      Behave  as  if  each pattern line has the jit modifier; after
187                 successful compilation, each pattern is passed to  the  just-
188                 in-time compiler, if available.
189
190       -jitverify
191                 Behave  as  if  each pattern line has the jitverify modifier;
192                 after successful compilation, each pattern is passed  to  the
193                 just-in-time  compiler,  if  available, and the use of JIT is
194                 verified.
195
196       -LM       List modifiers: write a list of available pattern and subject
197                 modifiers  to  the  standard output, then exit with zero exit
198                 code. All other options are ignored.  If both -C and -LM  are
199                 present, whichever is first is recognized.
200
201       -pattern modifier-list
202                 Behave as if each pattern line contains the given modifiers.
203
204       -q        Do not output the version number of pcre2test at the start of
205                 execution.
206
207       -S size   On Unix-like systems, set the size of the run-time  stack  to
208                 size mebibytes (units of 1024*1024 bytes).
209
210       -subject modifier-list
211                 Behave as if each subject line contains the given modifiers.
212
213       -t        Run  each compile and match many times with a timer, and out‐
214                 put the resulting times per compile or  match.  When  JIT  is
215                 used,  separate  times  are given for the initial compile and
216                 the JIT compile. You can control  the  number  of  iterations
217                 that  are used for timing by following -t with a number (as a
218                 separate item on the command line). For  example,  "-t  1000"
219                 iterates 1000 times. The default is to iterate 500,000 times.
220
221       -tm       This is like -t except that it times only the matching phase,
222                 not the compile phase.
223
224       -T -TM    These behave like -t and -tm, but in addition, at the end  of
225                 a  run, the total times for all compiles and matches are out‐
226                 put.
227
228       -version  Output the PCRE2 version number and then exit.
229

DESCRIPTION

231
232       If pcre2test is given two filename arguments, it reads from  the  first
233       and writes to the second. If the first name is "-", input is taken from
234       the standard input. If pcre2test is given only one argument,  it  reads
235       from that file and writes to stdout. Otherwise, it reads from stdin and
236       writes to stdout.
237
238       When pcre2test is built, a configuration option  can  specify  that  it
239       should  be linked with the libreadline or libedit library. When this is
240       done, if the input is from a terminal, it is read using the  readline()
241       function. This provides line-editing and history facilities. The output
242       from the -help option states whether or not readline() will be used.
243
244       The program handles any number of tests, each of which  consists  of  a
245       set  of input lines. Each set starts with a regular expression pattern,
246       followed by any number of subject lines to be matched against that pat‐
247       tern. In between sets of test data, command lines that begin with # may
248       appear. This file format, with some restrictions, can also be processed
249       by  the perltest.sh script that is distributed with PCRE2 as a means of
250       checking that the behaviour of PCRE2 and Perl is the same. For a speci‐
251       fication of perltest.sh, see the comments near its beginning.
252
253       When the input is a terminal, pcre2test prompts for each line of input,
254       using "re>" to prompt for regular expression patterns, and  "data>"  to
255       prompt  for subject lines. Command lines starting with # can be entered
256       only in response to the "re>" prompt.
257
258       Each subject line is matched separately and independently. If you  want
259       to do multi-line matches, you have to use the \n escape sequence (or \r
260       or \r\n, etc., depending on the newline setting) in a  single  line  of
261       input  to encode the newline sequences. There is no limit on the length
262       of subject lines; the input buffer is automatically extended if  it  is
263       too  small.  There  are  replication features that makes it possible to
264       generate long repetitive pattern or subject  lines  without  having  to
265       supply them explicitly.
266
267       An  empty  line  or  the end of the file signals the end of the subject
268       lines for a test, at which point a  new  pattern  or  command  line  is
269       expected if there is still input to be read.
270

COMMAND LINES

272
273       In  between sets of test data, a line that begins with # is interpreted
274       as a command line. If the first character is followed by white space or
275       an  exclamation  mark,  the  line is treated as a comment, and ignored.
276       Otherwise, the following commands are recognized:
277
278         #forbid_utf
279
280       Subsequent  patterns  automatically  have   the   PCRE2_NEVER_UTF   and
281       PCRE2_NEVER_UCP  options  set, which locks out the use of the PCRE2_UTF
282       and PCRE2_UCP options and the use of (*UTF) and (*UCP) at the start  of
283       patterns.  This  command  also  forces an error if a subsequent pattern
284       contains any occurrences of \P, \p, or \X, which  are  still  supported
285       when  PCRE2_UTF  is not set, but which require Unicode property support
286       to be included in the library.
287
288       This is a trigger guard that is used in test files to ensure  that  UTF
289       or  Unicode property tests are not accidentally added to files that are
290       used when Unicode support is  not  included  in  the  library.  Setting
291       PCRE2_NEVER_UTF  and  PCRE2_NEVER_UCP as a default can also be obtained
292       by the use of #pattern; the difference is that  #forbid_utf  cannot  be
293       unset,  and the automatic options are not displayed in pattern informa‐
294       tion, to avoid cluttering up test output.
295
296         #load <filename>
297
298       This command is used to load a set of precompiled patterns from a file,
299       as  described  in  the  section entitled "Saving and restoring compiled
300       patterns" below.
301
302         #newline_default [<newline-list>]
303
304       When PCRE2 is built, a default newline  convention  can  be  specified.
305       This  determines which characters and/or character pairs are recognized
306       as indicating a newline in a pattern or subject string. The default can
307       be  overridden when a pattern is compiled. The standard test files con‐
308       tain tests of various newline conventions,  but  the  majority  of  the
309       tests  expect  a  single  linefeed  to  be  recognized  as a newline by
310       default. Without special action the tests would fail when PCRE2 is com‐
311       piled with either CR or CRLF as the default newline.
312
313       The #newline_default command specifies a list of newline types that are
314       acceptable as the default. The types must be one of CR, LF, CRLF,  ANY‐
315       CRLF, ANY, or NUL (in upper or lower case), for example:
316
317         #newline_default LF Any anyCRLF
318
319       If the default newline is in the list, this command has no effect. Oth‐
320       erwise, except when testing the POSIX  API,  a  newline  modifier  that
321       specifies  the  first  newline  convention in the list (LF in the above
322       example) is added to any pattern that does not already have  a  newline
323       modifier. If the newline list is empty, the feature is turned off. This
324       command is present in a number of the standard test input files.
325
326       When the POSIX API is being tested there is  no  way  to  override  the
327       default  newline  convention,  though it is possible to set the newline
328       convention from within the pattern. A warning is given if the posix  or
329       posix_nosub  modifier is used when #newline_default would set a default
330       for the non-POSIX API.
331
332         #pattern <modifier-list>
333
334       This command sets a default modifier list that applies  to  all  subse‐
335       quent patterns. Modifiers on a pattern can change these settings.
336
337         #perltest
338
339       The  appearance of this line causes all subsequent modifier settings to
340       be checked for compatibility with the perltest.sh script, which is used
341       to  confirm that Perl gives the same results as PCRE2. Also, apart from
342       comment lines, #pattern commands, and #subject  commands  that  set  or
343       unset  "mark", no command lines are permitted, because they and many of
344       the modifiers are specific to pcre2test, and should not be used in test
345       files  that  are  also  processed by perltest.sh. The #perltest command
346       helps detect tests that are accidentally put in the wrong file.
347
348         #pop [<modifiers>]
349         #popcopy [<modifiers>]
350
351       These commands are used to manipulate the stack of  compiled  patterns,
352       as  described  in  the  section entitled "Saving and restoring compiled
353       patterns" below.
354
355         #save <filename>
356
357       This command is used to save a set of compiled patterns to a  file,  as
358       described  in  the section entitled "Saving and restoring compiled pat‐
359       terns" below.
360
361         #subject <modifier-list>
362
363       This command sets a default modifier list that applies  to  all  subse‐
364       quent  subject lines. Modifiers on a subject line can change these set‐
365       tings.
366

MODIFIER SYNTAX

368
369       Modifier lists are used with both pattern and subject lines. Items in a
370       list are separated by commas followed by optional white space. Trailing
371       whitespace in a modifier list is ignored. Some modifiers may  be  given
372       for  both patterns and subject lines, whereas others are valid only for
373       one  or  the  other.  Each  modifier  has  a  long  name,  for  example
374       "anchored",  and  some of them must be followed by an equals sign and a
375       value, for example, "offset=12". Values cannot  contain  comma  charac‐
376       ters,  but may contain spaces. Modifiers that do not take values may be
377       preceded by a minus sign to turn off a previous setting.
378
379       A few of the more common modifiers can also be specified as single let‐
380       ters,  for  example "i" for "caseless". In documentation, following the
381       Perl convention, these are written with a slash ("the /i modifier") for
382       clarity.  Abbreviated  modifiers  must all be concatenated in the first
383       item of a modifier list. If the first item is not recognized as a  long
384       modifier  name, it is interpreted as a sequence of these abbreviations.
385       For example:
386
387         /abc/ig,newline=cr,jit=3
388
389       This is a pattern line whose modifier list starts with  two  one-letter
390       modifiers  (/i  and  /g).  The lower-case abbreviated modifiers are the
391       same as used in Perl.
392

PATTERN SYNTAX

394
395       A pattern line must start with one of the following characters  (common
396       symbols, excluding pattern meta-characters):
397
398         / ! " ' ` - = _ : ; , % & @ ~
399
400       This  is  interpreted  as the pattern's delimiter. A regular expression
401       may be continued over several input lines, in which  case  the  newline
402       characters are included within it. It is possible to include the delim‐
403       iter within the pattern by escaping it with a backslash, for example
404
405         /abc\/def/
406
407       If you do this, the escape and the delimiter form part of the  pattern,
408       but since the delimiters are all non-alphanumeric, this does not affect
409       its interpretation. If the terminating delimiter  is  immediately  fol‐
410       lowed by a backslash, for example,
411
412         /abc/\
413
414       then  a  backslash  is added to the end of the pattern. This is done to
415       provide a way of testing the error condition that arises if  a  pattern
416       finishes with a backslash, because
417
418         /abc\/
419
420       is  interpreted as the first line of a pattern that starts with "abc/",
421       causing pcre2test to read the next line as a continuation of the  regu‐
422       lar expression.
423
424       A pattern can be followed by a modifier list (details below).
425

SUBJECT LINE SYNTAX

427
428       Before    each   subject   line   is   passed   to   pcre2_match()   or
429       pcre2_dfa_match(), leading and trailing white space is removed, and the
430       line is scanned for backslash escapes, unless the subject_literal modi‐
431       fier was set for the pattern. The following provide a means of encoding
432       non-printing characters in a visible way:
433
434         \a         alarm (BEL, \x07)
435         \b         backspace (\x08)
436         \e         escape (\x27)
437         \f         form feed (\x0c)
438         \n         newline (\x0a)
439         \r         carriage return (\x0d)
440         \t         tab (\x09)
441         \v         vertical tab (\x0b)
442         \nnn       octal character (up to 3 octal digits); always
443                      a byte unless > 255 in UTF-8 or 16-bit or 32-bit mode
444         \o{dd...}  octal character (any number of octal digits}
445         \xhh       hexadecimal byte (up to 2 hex digits)
446         \x{hh...}  hexadecimal character (any number of hex digits)
447
448       The use of \x{hh...} is not dependent on the use of the utf modifier on
449       the pattern. It is recognized always. There may be any number of  hexa‐
450       decimal  digits  inside  the  braces; invalid values provoke error mes‐
451       sages.
452
453       Note that \xhh specifies one byte rather than one  character  in  UTF-8
454       mode;  this  makes it possible to construct invalid UTF-8 sequences for
455       testing purposes. On the other hand, \x{hh} is interpreted as  a  UTF-8
456       character  in UTF-8 mode, generating more than one byte if the value is
457       greater than 127.  When testing the 8-bit library not  in  UTF-8  mode,
458       \x{hh} generates one byte for values less than 256, and causes an error
459       for greater values.
460
461       In UTF-16 mode, all 4-digit \x{hhhh} values are accepted. This makes it
462       possible to construct invalid UTF-16 sequences for testing purposes.
463
464       In  UTF-32  mode,  all  4- to 8-digit \x{...} values are accepted. This
465       makes it possible to construct invalid  UTF-32  sequences  for  testing
466       purposes.
467
468       There is a special backslash sequence that specifies replication of one
469       or more characters:
470
471         \[<characters>]{<count>}
472
473       This makes it possible to test long strings without having  to  provide
474       them as part of the file. For example:
475
476         \[abc]{4}
477
478       is  converted to "abcabcabcabc". This feature does not support nesting.
479       To include a closing square bracket in the characters, code it as \x5D.
480
481       A backslash followed by an equals sign marks the  end  of  the  subject
482       string and the start of a modifier list. For example:
483
484         abc\=notbol,notempty
485
486       If  the  subject  string is empty and \= is followed by whitespace, the
487       line is treated as a comment line, and is not used  for  matching.  For
488       example:
489
490         \= This is a comment.
491         abc\= This is an invalid modifier list.
492
493       A  backslash  followed  by  any  other  non-alphanumeric character just
494       escapes that character. A backslash followed by anything else causes an
495       error.  However,  if the very last character in the line is a backslash
496       (and there is no modifier list), it is ignored. This  gives  a  way  of
497       passing  an  empty line as data, since a real empty line terminates the
498       data input.
499
500       If the subject_literal modifier is set for a pattern, all subject lines
501       that follow are treated as literals, with no special treatment of back‐
502       slashes.  No replication is possible, and any subject modifiers must be
503       set as defaults by a #subject command.
504

PATTERN MODIFIERS

506
507       There  are  several types of modifier that can appear in pattern lines.
508       Except where noted below, they may also be used in #pattern commands. A
509       pattern's  modifier  list can add to or override default modifiers that
510       were set by a previous #pattern command.
511
512   Setting compilation options
513
514       The following modifiers set options for pcre2_compile(). Most  of  them
515       set  bits  in  the  options  argument of that function, but those whose
516       names start with PCRE2_EXTRA are additional options that are set in the
517       compile  context.  For  the  main options, there are some single-letter
518       abbreviations that are the same as Perl options. There is special  han‐
519       dling  for  /x:  if  a second x is present, PCRE2_EXTENDED is converted
520       into  PCRE2_EXTENDED_MORE  as  in  Perl.  A   third   appearance   adds
521       PCRE2_EXTENDED  as  well,  though  this  makes no difference to the way
522       pcre2_compile() behaves. See pcre2api for a description of the  effects
523       of these options.
524
525             allow_empty_class         set PCRE2_ALLOW_EMPTY_CLASS
526             allow_surrogate_escapes   set PCRE2_EXTRA_ALLOW_SURROGATE_ESCAPES
527             alt_bsux                  set PCRE2_ALT_BSUX
528             alt_circumflex            set PCRE2_ALT_CIRCUMFLEX
529             alt_verbnames             set PCRE2_ALT_VERBNAMES
530             anchored                  set PCRE2_ANCHORED
531             auto_callout              set PCRE2_AUTO_CALLOUT
532             bad_escape_is_literal     set PCRE2_EXTRA_BAD_ESCAPE_IS_LITERAL
533         /i  caseless                  set PCRE2_CASELESS
534             dollar_endonly            set PCRE2_DOLLAR_ENDONLY
535         /s  dotall                    set PCRE2_DOTALL
536             dupnames                  set PCRE2_DUPNAMES
537             endanchored               set PCRE2_ENDANCHORED
538             escaped_cr_is_lf          set PCRE2_EXTRA_ESCAPED_CR_IS_LF
539         /x  extended                  set PCRE2_EXTENDED
540         /xx extended_more             set PCRE2_EXTENDED_MORE
541             extra_alt_bsux            set PCRE2_EXTRA_ALT_BSUX
542             firstline                 set PCRE2_FIRSTLINE
543             literal                   set PCRE2_LITERAL
544             match_line                set PCRE2_EXTRA_MATCH_LINE
545             match_unset_backref       set PCRE2_MATCH_UNSET_BACKREF
546             match_word                set PCRE2_EXTRA_MATCH_WORD
547         /m  multiline                 set PCRE2_MULTILINE
548             never_backslash_c         set PCRE2_NEVER_BACKSLASH_C
549             never_ucp                 set PCRE2_NEVER_UCP
550             never_utf                 set PCRE2_NEVER_UTF
551         /n  no_auto_capture           set PCRE2_NO_AUTO_CAPTURE
552             no_auto_possess           set PCRE2_NO_AUTO_POSSESS
553             no_dotstar_anchor         set PCRE2_NO_DOTSTAR_ANCHOR
554             no_start_optimize         set PCRE2_NO_START_OPTIMIZE
555             no_utf_check              set PCRE2_NO_UTF_CHECK
556             ucp                       set PCRE2_UCP
557             ungreedy                  set PCRE2_UNGREEDY
558             use_offset_limit          set PCRE2_USE_OFFSET_LIMIT
559             utf                       set PCRE2_UTF
560
561       As well as turning on the PCRE2_UTF option, the utf modifier causes all
562       non-printing characters in output  strings  to  be  printed  using  the
563       \x{hh...}  notation. Otherwise, those less than 0x100 are output in hex
564       without the curly brackets. Setting utf in 16-bit or 32-bit  mode  also
565       causes  pattern  and  subject  strings  to  be  translated to UTF-16 or
566       UTF-32, respectively, before being passed to library functions.
567
568   Setting compilation controls
569
570       The following modifiers  affect  the  compilation  process  or  request
571       information  about  the  pattern. There are single-letter abbreviations
572       for some that are heavily used in the test files.
573
574             bsr=[anycrlf|unicode]     specify \R handling
575         /B  bincode                   show binary code without lengths
576             callout_info              show callout information
577             convert=<options>         request foreign pattern conversion
578             convert_glob_escape=c     set glob escape character
579             convert_glob_separator=c  set glob separator character
580             convert_length            set convert buffer length
581             debug                     same as info,fullbincode
582             framesize                 show matching frame size
583             fullbincode               show binary code with lengths
584         /I  info                      show info about compiled pattern
585             hex                       unquoted characters are hexadecimal
586             jit[=<number>]            use JIT
587             jitfast                   use JIT fast path
588             jitverify                 verify JIT use
589             locale=<name>             use this locale
590             max_pattern_length=<n>    set the maximum pattern length
591             memory                    show memory used
592             newline=<type>            set newline type
593             null_context              compile with a NULL context
594             parens_nest_limit=<n>     set maximum parentheses depth
595             posix                     use the POSIX API
596             posix_nosub               use the POSIX API with REG_NOSUB
597             push                      push compiled pattern onto the stack
598             pushcopy                  push a copy onto the stack
599             stackguard=<number>       test the stackguard feature
600             subject_literal           treat all subject lines as literal
601             tables=[0|1|2]            select internal tables
602             use_length                do not zero-terminate the pattern
603             utf8_input                treat input as UTF-8
604
605       The effects of these modifiers are described in the following sections.
606
607   Newline and \R handling
608
609       The bsr modifier specifies what \R in a pattern should match. If it  is
610       set  to  "anycrlf",  \R  matches  CR, LF, or CRLF only. If it is set to
611       "unicode", \R matches any Unicode newline sequence. The default can  be
612       specified when PCRE2 is built; if it is not, the default is set to Uni‐
613       code.
614
615       The newline modifier specifies which characters are to  be  interpreted
616       as newlines, both in the pattern and in subject lines. The type must be
617       one of CR, LF, CRLF, ANYCRLF, ANY, or NUL (in upper or lower case).
618
619   Information about a pattern
620
621       The debug modifier is a shorthand for info,fullbincode, requesting  all
622       available information.
623
624       The bincode modifier causes a representation of the compiled code to be
625       output after compilation. This information does not contain length  and
626       offset values, which ensures that the same output is generated for dif‐
627       ferent internal link sizes and different code  unit  widths.  By  using
628       bincode,  the  same  regression tests can be used in different environ‐
629       ments.
630
631       The fullbincode modifier, by contrast, does include length  and  offset
632       values.  This is used in a few special tests that run only for specific
633       code unit widths and link sizes, and is also useful for one-off tests.
634
635       The info modifier  requests  information  about  the  compiled  pattern
636       (whether  it  is anchored, has a fixed first character, and so on). The
637       information is obtained from the  pcre2_pattern_info()  function.  Here
638       are some typical examples:
639
640           re> /(?i)(^a|^b)/m,info
641         Capture group count = 1
642         Compile options: multiline
643         Overall options: caseless multiline
644         First code unit at start or follows newline
645         Subject length lower bound = 1
646
647           re> /(?i)abc/info
648         Capture group count = 0
649         Compile options: <none>
650         Overall options: caseless
651         First code unit = 'a' (caseless)
652         Last code unit = 'c' (caseless)
653         Subject length lower bound = 3
654
655       "Compile  options"  are those specified by modifiers; "overall options"
656       have added options that are taken or deduced from the pattern. If  both
657       sets  of  options are the same, just a single "options" line is output;
658       if there are no options, the line is  omitted.  "First  code  unit"  is
659       where  any  match must start; if there is more than one they are listed
660       as "starting code units". "Last code unit" is  the  last  literal  code
661       unit  that  must  be  present in any match. This is not necessarily the
662       last character. These lines are omitted if no starting or  ending  code
663       units are recorded.
664
665       The  framesize modifier shows the size, in bytes, of the storage frames
666       used by pcre2_match() for handling backtracking. The  size  depends  on
667       the number of capturing parentheses in the pattern.
668
669       The  callout_info  modifier requests information about all the callouts
670       in the pattern. A list of them is output at the end of any other infor‐
671       mation that is requested. For each callout, either its number or string
672       is given, followed by the item that follows it in the pattern.
673
674   Passing a NULL context
675
676       Normally, pcre2test passes a context block to pcre2_compile().  If  the
677       null_context  modifier  is  set,  however,  NULL is passed. This is for
678       testing that pcre2_compile() behaves correctly in this  case  (it  uses
679       default values).
680
681   Specifying pattern characters in hexadecimal
682
683       The  hex  modifier specifies that the characters of the pattern, except
684       for substrings enclosed in single or double quotes, are  to  be  inter‐
685       preted  as  pairs  of hexadecimal digits. This feature is provided as a
686       way of creating patterns that contain binary zeros and other non-print‐
687       ing  characters.  White space is permitted between pairs of digits. For
688       example, this pattern contains three characters:
689
690         /ab 32 59/hex
691
692       Parts of such a pattern are taken literally  if  quoted.  This  pattern
693       contains  nine characters, only two of which are specified in hexadeci‐
694       mal:
695
696         /ab "literal" 32/hex
697
698       Either single or double quotes may be used. There is no way of  includ‐
699       ing  the delimiter within a substring. The hex and expand modifiers are
700       mutually exclusive.
701
702   Specifying the pattern's length
703
704       By default, patterns are passed to the compiling functions as zero-ter‐
705       minated  strings but can be passed by length instead of being zero-ter‐
706       minated. The use_length modifier causes this to happen. Using a  length
707       happens  automatically  (whether  or not use_length is set) when hex is
708       set, because patterns  specified  in  hexadecimal  may  contain  binary
709       zeros.
710
711       If hex or use_length is used with the POSIX wrapper API (see "Using the
712       POSIX wrapper API" below), the REG_PEND extension is used to  pass  the
713       pattern's length.
714
715   Specifying wide characters in 16-bit and 32-bit modes
716
717       In 16-bit and 32-bit modes, all input is automatically treated as UTF-8
718       and translated to UTF-16 or UTF-32 when the utf modifier  is  set.  For
719       testing the 16-bit and 32-bit libraries in non-UTF mode, the utf8_input
720       modifier can be used. It is mutually exclusive with  utf.  Input  lines
721       are interpreted as UTF-8 as a means of specifying wide characters. More
722       details are given in "Input encoding" above.
723
724   Generating long repetitive patterns
725
726       Some tests use long patterns that are very repetitive. Instead of  cre‐
727       ating  a very long input line for such a pattern, you can use a special
728       repetition feature, similar to the  one  described  for  subject  lines
729       above.  If  the  expand  modifier is present on a pattern, parts of the
730       pattern that have the form
731
732         \[<characters>]{<count>}
733
734       are expanded before the pattern is passed to pcre2_compile(). For exam‐
735       ple, \[AB]{6000} is expanded to "ABAB..." 6000 times. This construction
736       cannot be nested. An initial "\[" sequence is recognized only  if  "]{"
737       followed  by  decimal  digits and "}" is found later in the pattern. If
738       not, the characters remain in the pattern unaltered. The expand and hex
739       modifiers are mutually exclusive.
740
741       If  part  of an expanded pattern looks like an expansion, but is really
742       part of the actual pattern, unwanted expansion can be avoided by giving
743       two values in the quantifier. For example, \[AB]{6000,6000} is not rec‐
744       ognized as an expansion item.
745
746       If the info modifier is set on an expanded pattern, the result  of  the
747       expansion is included in the information that is output.
748
749   JIT compilation
750
751       Just-in-time  (JIT)  compiling  is  a heavyweight optimization that can
752       greatly speed up pattern matching. See the pcre2jit  documentation  for
753       details.  JIT  compiling  happens, optionally, after a pattern has been
754       successfully compiled into an internal form. The JIT compiler  converts
755       this to optimized machine code. It needs to know whether the match-time
756       options PCRE2_PARTIAL_HARD and PCRE2_PARTIAL_SOFT are going to be used,
757       because  different  code  is generated for the different cases. See the
758       partial modifier in "Subject Modifiers" below for details of how  these
759       options are specified for each match attempt.
760
761       JIT  compilation  is  requested  by the jit pattern modifier, which may
762       optionally be followed by an equals sign and a number in the range 0 to
763       7.   The  three bits that make up the number specify which of the three
764       JIT operating modes are to be compiled:
765
766         1  compile JIT code for non-partial matching
767         2  compile JIT code for soft partial matching
768         4  compile JIT code for hard partial matching
769
770       The possible values for the jit modifier are therefore:
771
772         0  disable JIT
773         1  normal matching only
774         2  soft partial matching only
775         3  normal and soft partial matching
776         4  hard partial matching only
777         6  soft and hard partial matching only
778         7  all three modes
779
780       If no number is given, 7 is  assumed.  The  phrase  "partial  matching"
781       means a call to pcre2_match() with either the PCRE2_PARTIAL_SOFT or the
782       PCRE2_PARTIAL_HARD option set. Note that such a call may return a  com‐
783       plete match; the options enable the possibility of a partial match, but
784       do not require it. Note also that if you request JIT  compilation  only
785       for  partial  matching  (for example, jit=2) but do not set the partial
786       modifier on a subject line, that match will not use  JIT  code  because
787       none was compiled for non-partial matching.
788
789       If  JIT compilation is successful, the compiled JIT code will automati‐
790       cally be used when an appropriate type of match  is  run,  except  when
791       incompatible  run-time options are specified. For more details, see the
792       pcre2jit documentation. See also the jitstack modifier below for a  way
793       of setting the size of the JIT stack.
794
795       If  the  jitfast  modifier is specified, matching is done using the JIT
796       "fast path" interface, pcre2_jit_match(), which skips some of the  san‐
797       ity  checks that are done by pcre2_match(), and of course does not work
798       when JIT is not supported. If jitfast is specified without  jit,  jit=7
799       is assumed.
800
801       If  the jitverify modifier is specified, information about the compiled
802       pattern shows whether JIT compilation was or  was  not  successful.  If
803       jitverify  is  specified without jit, jit=7 is assumed. If JIT compila‐
804       tion is successful when jitverify is set, the text "(JIT)" is added  to
805       the first output line after a match or non match when JIT-compiled code
806       was actually used in the match.
807
808   Setting a locale
809
810       The locale modifier must specify the name of a locale, for example:
811
812         /pattern/locale=fr_FR
813
814       The given locale is set, pcre2_maketables() is called to build a set of
815       character  tables for the locale, and this is then passed to pcre2_com‐
816       pile() when compiling the regular expression. The same tables are  used
817       when  matching the following subject lines. The locale modifier applies
818       only to the pattern on which it appears, but can be given in a #pattern
819       command  if a default is needed. Setting a locale and alternate charac‐
820       ter tables are mutually exclusive.
821
822   Showing pattern memory
823
824       The memory modifier causes the size in bytes of the memory used to hold
825       the  compiled  pattern  to be output. This does not include the size of
826       the pcre2_code block; it is just the actual compiled data. If the  pat‐
827       tern  is  subsequently  passed to the JIT compiler, the size of the JIT
828       compiled code is also output. Here is an example:
829
830           re> /a(b)c/jit,memory
831         Memory allocation (code space): 21
832         Memory allocation (JIT code): 1910
833
834
835   Limiting nested parentheses
836
837       The parens_nest_limit modifier sets a limit  on  the  depth  of  nested
838       parentheses  in  a  pattern.  Breaching  the limit causes a compilation
839       error.  The default for the library is set when  PCRE2  is  built,  but
840       pcre2test  sets  its  own default of 220, which is required for running
841       the standard test suite.
842
843   Limiting the pattern length
844
845       The max_pattern_length modifier sets a limit, in  code  units,  to  the
846       length of pattern that pcre2_compile() will accept. Breaching the limit
847       causes a compilation  error.  The  default  is  the  largest  number  a
848       PCRE2_SIZE variable can hold (essentially unlimited).
849
850   Using the POSIX wrapper API
851
852       The  posix  and posix_nosub modifiers cause pcre2test to call PCRE2 via
853       the POSIX wrapper API rather than its native API. When  posix_nosub  is
854       used,  the  POSIX  option  REG_NOSUB  is passed to regcomp(). The POSIX
855       wrapper supports only the 8-bit library. Note that it  does  not  imply
856       POSIX matching semantics; for more detail see the pcre2posix documenta‐
857       tion. The following pattern modifiers set  options  for  the  regcomp()
858       function:
859
860         caseless           REG_ICASE
861         multiline          REG_NEWLINE
862         dotall             REG_DOTALL     )
863         ungreedy           REG_UNGREEDY   ) These options are not part of
864         ucp                REG_UCP        )   the POSIX standard
865         utf                REG_UTF8       )
866
867       The  regerror_buffsize  modifier  specifies a size for the error buffer
868       that is passed to regerror() in the event of a compilation  error.  For
869       example:
870
871         /abc/posix,regerror_buffsize=20
872
873       This  provides  a means of testing the behaviour of regerror() when the
874       buffer is too small for the error message. If  this  modifier  has  not
875       been set, a large buffer is used.
876
877       The  aftertext  and  allaftertext  subject  modifiers work as described
878       below. All other modifiers are either ignored, with a warning  message,
879       or cause an error.
880
881       The  pattern  is  passed  to  regcomp()  as a zero-terminated string by
882       default, but if the use_length or hex modifiers are set,  the  REG_PEND
883       extension is used to pass it by length.
884
885   Testing the stack guard feature
886
887       The  stackguard  modifier  is  used  to  test the use of pcre2_set_com‐
888       pile_recursion_guard(), a function that is  provided  to  enable  stack
889       availability  to  be checked during compilation (see the pcre2api docu‐
890       mentation for details). If the number  specified  by  the  modifier  is
891       greater than zero, pcre2_set_compile_recursion_guard() is called to set
892       up callback from pcre2_compile() to a local function. The  argument  it
893       receives  is  the current nesting parenthesis depth; if this is greater
894       than the value given by the modifier, non-zero is returned, causing the
895       compilation to be aborted.
896
897   Using alternative character tables
898
899       The  value  specified for the tables modifier must be one of the digits
900       0, 1, or 2. It causes a specific set of built-in character tables to be
901       passed to pcre2_compile(). This is used in the PCRE2 tests to check be‐
902       haviour with different character tables. The digit specifies the tables
903       as follows:
904
905         0   do not pass any special character tables
906         1   the default ASCII tables, as distributed in
907               pcre2_chartables.c.dist
908         2   a set of tables defining ISO 8859 characters
909
910       In  table 2, some characters whose codes are greater than 128 are iden‐
911       tified as letters, digits, spaces,  etc.  Setting  alternate  character
912       tables and a locale are mutually exclusive.
913
914   Setting certain match controls
915
916       The following modifiers are really subject modifiers, and are described
917       under "Subject Modifiers" below. However, they may  be  included  in  a
918       pattern's  modifier  list, in which case they are applied to every sub‐
919       ject line that is processed with that pattern. These modifiers  do  not
920       affect the compilation process.
921
922             aftertext                  show text after match
923             allaftertext               show text after captures
924             allcaptures                show all captures
925             allvector                  show the entire ovector
926             allusedtext                show all consulted text
927             altglobal                  alternative global matching
928         /g  global                     global matching
929             jitstack=<n>               set size of JIT stack
930             mark                       show mark values
931             replace=<string>           specify a replacement string
932             startchar                  show starting character when relevant
933             substitute_callout         use substitution callouts
934             substitute_extended        use PCRE2_SUBSTITUTE_EXTENDED
935             substitute_skip=<n>        skip substitution number n
936             substitute_overflow_length use PCRE2_SUBSTITUTE_OVERFLOW_LENGTH
937             substitute_stop=<n>        skip substitution number n and greater
938             substitute_unknown_unset   use PCRE2_SUBSTITUTE_UNKNOWN_UNSET
939             substitute_unset_empty     use PCRE2_SUBSTITUTE_UNSET_EMPTY
940
941       These  modifiers may not appear in a #pattern command. If you want them
942       as defaults, set them in a #subject command.
943
944   Specifying literal subject lines
945
946       If the subject_literal modifier is present on a pattern, all  the  sub‐
947       ject lines that it matches are taken as literal strings, with no inter‐
948       pretation of backslashes. It is not possible to set  subject  modifiers
949       on  such  lines, but any that are set as defaults by a #subject command
950       are recognized.
951
952   Saving a compiled pattern
953
954       When a pattern with the push modifier is successfully compiled,  it  is
955       pushed  onto  a  stack  of compiled patterns, and pcre2test expects the
956       next line to contain a new pattern (or a command) instead of a  subject
957       line. This facility is used when saving compiled patterns to a file, as
958       described in the section entitled "Saving and restoring  compiled  pat‐
959       terns"  below.  If pushcopy is used instead of push, a copy of the com‐
960       piled pattern is stacked, leaving the original  as  current,  ready  to
961       match  the  following  input  lines. This provides a way of testing the
962       pcre2_code_copy() function.   The  push  and  pushcopy   modifiers  are
963       incompatible  with  compilation  modifiers  such  as global that act at
964       match time. Any that are specified are ignored (for the stacked  copy),
965       with a warning message, except for replace, which causes an error. Note
966       that jitverify, which is allowed, does not carry through to any  subse‐
967       quent matching that uses a stacked pattern.
968
969   Testing foreign pattern conversion
970
971       The  experimental  foreign pattern conversion functions in PCRE2 can be
972       tested by setting the convert modifier. Its argument is  a  colon-sepa‐
973       rated  list  of  options,  which  set  the  equivalent  option  for the
974       pcre2_pattern_convert() function:
975
976         glob                    PCRE2_CONVERT_GLOB
977         glob_no_starstar        PCRE2_CONVERT_GLOB_NO_STARSTAR
978         glob_no_wild_separator  PCRE2_CONVERT_GLOB_NO_WILD_SEPARATOR
979         posix_basic             PCRE2_CONVERT_POSIX_BASIC
980         posix_extended          PCRE2_CONVERT_POSIX_EXTENDED
981         unset                   Unset all options
982
983       The "unset" value is useful for turning off a default that has been set
984       by a #pattern command. When one of these options is set, the input pat‐
985       tern is passed to pcre2_pattern_convert(). If the  conversion  is  suc‐
986       cessful,  the  result  is  reflected  in  the output and then passed to
987       pcre2_compile(). The normal utf and no_utf_check options, if set, cause
988       the  PCRE2_CONVERT_UTF  and  PCRE2_CONVERT_NO_UTF_CHECK  options  to be
989       passed to pcre2_pattern_convert().
990
991       By default, the conversion function is allowed to allocate a buffer for
992       its  output.  However, if the convert_length modifier is set to a value
993       greater than zero, pcre2test passes a buffer of the given length.  This
994       makes it possible to test the length check.
995
996       The  convert_glob_escape  and  convert_glob_separator  modifiers can be
997       used to specify the escape and separator characters for  glob  process‐
998       ing, overriding the defaults, which are operating-system dependent.
999

SUBJECT MODIFIERS

1001
1002       The modifiers that can appear in subject lines and the #subject command
1003       are of two types.
1004
1005   Setting match options
1006
1007       The   following   modifiers   set   options   for   pcre2_match()    or
1008       pcre2_dfa_match(). See pcreapi for a description of their effects.
1009
1010             anchored                  set PCRE2_ANCHORED
1011             endanchored               set PCRE2_ENDANCHORED
1012             dfa_restart               set PCRE2_DFA_RESTART
1013             dfa_shortest              set PCRE2_DFA_SHORTEST
1014             no_jit                    set PCRE2_NO_JIT
1015             no_utf_check              set PCRE2_NO_UTF_CHECK
1016             notbol                    set PCRE2_NOTBOL
1017             notempty                  set PCRE2_NOTEMPTY
1018             notempty_atstart          set PCRE2_NOTEMPTY_ATSTART
1019             noteol                    set PCRE2_NOTEOL
1020             partial_hard (or ph)      set PCRE2_PARTIAL_HARD
1021             partial_soft (or ps)      set PCRE2_PARTIAL_SOFT
1022
1023       The  partial matching modifiers are provided with abbreviations because
1024       they appear frequently in tests.
1025
1026       If the posix or posix_nosub modifier was present on the pattern,  caus‐
1027       ing the POSIX wrapper API to be used, the only option-setting modifiers
1028       that have any effect are notbol, notempty, and noteol, causing REG_NOT‐
1029       BOL,  REG_NOTEMPTY,  and  REG_NOTEOL,  respectively,  to  be  passed to
1030       regexec(). The other modifiers are ignored, with a warning message.
1031
1032       There is one additional modifier that can be used with the POSIX  wrap‐
1033       per. It is ignored (with a warning) if used for non-POSIX matching.
1034
1035             posix_startend=<n>[:<m>]
1036
1037       This  causes  the  subject  string  to be passed to regexec() using the
1038       REG_STARTEND option, which uses offsets to specify which  part  of  the
1039       string  is  searched.  If  only  one number is given, the end offset is
1040       passed as the end of the subject string. For more detail  of  REG_STAR‐
1041       TEND,  see the pcre2posix documentation. If the subject string contains
1042       binary zeros (coded as escapes such as \x{00}  because  pcre2test  does
1043       not support actual binary zeros in its input), you must use posix_star‐
1044       tend to specify its length.
1045
1046   Setting match controls
1047
1048       The following modifiers affect the matching process  or  request  addi‐
1049       tional  information.  Some  of  them may also be specified on a pattern
1050       line (see above), in which case they apply to every subject  line  that
1051       is matched against that pattern.
1052
1053             aftertext                  show text after match
1054             allaftertext               show text after captures
1055             allcaptures                show all captures
1056             allvector                  show the entire ovector
1057             allusedtext                show all consulted text (non-JIT only)
1058             altglobal                  alternative global matching
1059             callout_capture            show captures at callout time
1060             callout_data=<n>           set a value to pass via callouts
1061             callout_error=<n>[:<m>]    control callout error
1062             callout_extra              show extra callout information
1063             callout_fail=<n>[:<m>]     control callout failure
1064             callout_no_where           do not show position of a callout
1065             callout_none               do not supply a callout function
1066             copy=<number or name>      copy captured substring
1067             depth_limit=<n>            set a depth limit
1068             dfa                        use pcre2_dfa_match()
1069             find_limits                find match and depth limits
1070             get=<number or name>       extract captured substring
1071             getall                     extract all captured substrings
1072         /g  global                     global matching
1073             heap_limit=<n>             set a limit on heap memory (Kbytes)
1074             jitstack=<n>               set size of JIT stack
1075             mark                       show mark values
1076             match_limit=<n>            set a match limit
1077             memory                     show heap memory usage
1078             null_context               match with a NULL context
1079             offset=<n>                 set starting offset
1080             offset_limit=<n>           set offset limit
1081             ovector=<n>                set size of output vector
1082             recursion_limit=<n>        obsolete synonym for depth_limit
1083             replace=<string>           specify a replacement string
1084             startchar                  show startchar when relevant
1085             startoffset=<n>            same as offset=<n>
1086             substitute_callout         use substitution callouts
1087             substitute_extedded        use PCRE2_SUBSTITUTE_EXTENDED
1088             substitute_skip=<n>        skip substitution number n
1089             substitute_overflow_length use PCRE2_SUBSTITUTE_OVERFLOW_LENGTH
1090             substitute_stop=<n>        skip substitution number n and greater
1091             substitute_unknown_unset   use PCRE2_SUBSTITUTE_UNKNOWN_UNSET
1092             substitute_unset_empty     use PCRE2_SUBSTITUTE_UNSET_EMPTY
1093             zero_terminate             pass the subject as zero-terminated
1094
1095       The effects of these modifiers are described in the following sections.
1096       When matching via the POSIX wrapper API, the  aftertext,  allaftertext,
1097       and  ovector subject modifiers work as described below. All other modi‐
1098       fiers are either ignored, with a warning message, or cause an error.
1099
1100   Showing more text
1101
1102       The aftertext modifier requests that as well as outputting the part  of
1103       the subject string that matched the entire pattern, pcre2test should in
1104       addition output the remainder of the subject string. This is useful for
1105       tests where the subject contains multiple copies of the same substring.
1106       The allaftertext modifier requests the same action  for  captured  sub‐
1107       strings as well as the main matched substring. In each case the remain‐
1108       der is output on the following line with a plus character following the
1109       capture number.
1110
1111       The  allusedtext modifier requests that all the text that was consulted
1112       during a successful pattern match by the interpreter should  be  shown.
1113       This  feature  is not supported for JIT matching, and if requested with
1114       JIT it is ignored (with  a  warning  message).  Setting  this  modifier
1115       affects the output if there is a lookbehind at the start of a match, or
1116       a lookahead at the end, or if \K is used  in  the  pattern.  Characters
1117       that  precede or follow the start and end of the actual match are indi‐
1118       cated in the output by '<' or '>' characters underneath them.  Here  is
1119       an example:
1120
1121           re> /(?<=pqr)abc(?=xyz)/
1122         data> 123pqrabcxyz456\=allusedtext
1123          0: pqrabcxyz
1124             <<<   >>>
1125
1126       This  shows  that  the  matched string is "abc", with the preceding and
1127       following strings "pqr" and "xyz"  having  been  consulted  during  the
1128       match (when processing the assertions).
1129
1130       The  startchar  modifier  requests  that the starting character for the
1131       match be indicated, if it is different to  the  start  of  the  matched
1132       string. The only time when this occurs is when \K has been processed as
1133       part of the match. In this situation, the output for the matched string
1134       is  displayed  from  the  starting  character instead of from the match
1135       point, with circumflex characters under  the  earlier  characters.  For
1136       example:
1137
1138           re> /abc\Kxyz/
1139         data> abcxyz\=startchar
1140          0: abcxyz
1141             ^^^
1142
1143       Unlike  allusedtext, the startchar modifier can be used with JIT.  How‐
1144       ever, these two modifiers are mutually exclusive.
1145
1146   Showing the value of all capture groups
1147
1148       The allcaptures modifier requests that the values of all potential cap‐
1149       tured parentheses be output after a match. By default, only those up to
1150       the highest one actually used in the match are output (corresponding to
1151       the  return  code from pcre2_match()). Groups that did not take part in
1152       the match are output as "<unset>". This modifier is  not  relevant  for
1153       DFA  matching (which does no capturing) and does not apply when replace
1154       is specified; it is ignored, with a warning message, if present.
1155
1156   Showing the entire ovector, for all outcomes
1157
1158       The allvector modifier requests that the entire ovector be shown, what‐
1159       ever the outcome of the match. Compare allcaptures, which shows only up
1160       to the maximum number of capture groups for the pattern, and then  only
1161       for  a  successful  complete  non-DFA  match. This modifier, which acts
1162       after any match result, and also for DFA matching, provides a means  of
1163       checking  that there are no unexpected modifications to ovector fields.
1164       Before each match attempt, the ovector is filled with a special  value,
1165       and   if   this  is  found  in  both  elements  of  a  capturing  pair,
1166       "<unchanged>" is output. After a successful match, this applies to  all
1167       groups  after the maximum capture group for the pattern. In other cases
1168       it applies to the entire ovector. After a partial match, the first  two
1169       elements  are  the only ones that should be set. After a DFA match, the
1170       amount of ovector that is used depends on the number  of  matches  that
1171       were found.
1172
1173   Testing pattern callouts
1174
1175       A  callout function is supplied when pcre2test calls the library match‐
1176       ing functions, unless callout_none is specified. Its behaviour  can  be
1177       controlled  by  various  modifiers  listed above whose names begin with
1178       callout_. Details are given in the section entitled  "Callouts"  below.
1179       Testing  callouts  from  pcre2_substitute()  is  decribed separately in
1180       "Testing the substitution function" below.
1181
1182   Finding all matches in a string
1183
1184       Searching for all possible matches within a subject can be requested by
1185       the  global  or altglobal modifier. After finding a match, the matching
1186       function is called again to search the remainder of  the  subject.  The
1187       difference  between  global  and  altglobal is that the former uses the
1188       start_offset argument to pcre2_match() or  pcre2_dfa_match()  to  start
1189       searching  at  a new point within the entire string (which is what Perl
1190       does), whereas the latter passes over a shortened subject. This makes a
1191       difference to the matching process if the pattern begins with a lookbe‐
1192       hind assertion (including \b or \B).
1193
1194       If an empty string  is  matched,  the  next  match  is  done  with  the
1195       PCRE2_NOTEMPTY_ATSTART and PCRE2_ANCHORED flags set, in order to search
1196       for another, non-empty, match at the same point in the subject. If this
1197       match  fails,  the  start  offset  is advanced, and the normal match is
1198       retried. This imitates the way Perl handles such cases when  using  the
1199       /g  modifier  or  the  split()  function. Normally, the start offset is
1200       advanced by one character, but if  the  newline  convention  recognizes
1201       CRLF  as  a newline, and the current character is CR followed by LF, an
1202       advance of two characters occurs.
1203
1204   Testing substring extraction functions
1205
1206       The copy  and  get  modifiers  can  be  used  to  test  the  pcre2_sub‐
1207       string_copy_xxx() and pcre2_substring_get_xxx() functions.  They can be
1208       given more than once, and each can specify a capture group name or num‐
1209       ber, for example:
1210
1211          abcd\=copy=1,copy=3,get=G1
1212
1213       If  the  #subject command is used to set default copy and/or get lists,
1214       these can be unset by specifying a negative number to cancel  all  num‐
1215       bered groups and an empty name to cancel all named groups.
1216
1217       The  getall  modifier  tests pcre2_substring_list_get(), which extracts
1218       all captured substrings.
1219
1220       If the subject line is successfully matched, the  substrings  extracted
1221       by  the  convenience  functions  are  output  with C, G, or L after the
1222       string number instead of a colon. This is in  addition  to  the  normal
1223       full  list.  The string length (that is, the return from the extraction
1224       function) is given in parentheses after each substring, followed by the
1225       name when the extraction was by name.
1226
1227   Testing the substitution function
1228
1229       If  the  replace  modifier  is  set, the pcre2_substitute() function is
1230       called instead of one of the matching functions. Note that  replacement
1231       strings  cannot  contain commas, because a comma signifies the end of a
1232       modifier. This is not thought to be an issue in a test program.
1233
1234       Unlike subject strings, pcre2test does not process replacement  strings
1235       for  escape  sequences. In UTF mode, a replacement string is checked to
1236       see if it is a valid UTF-8 string. If so, it is correctly converted  to
1237       a  UTF  string of the appropriate code unit width. If it is not a valid
1238       UTF-8 string, the individual code units are copied directly. This  pro‐
1239       vides a means of passing an invalid UTF-8 string for testing purposes.
1240
1241       The  following modifiers set options (in additional to the normal match
1242       options) for pcre2_substitute():
1243
1244         global                      PCRE2_SUBSTITUTE_GLOBAL
1245         substitute_extended         PCRE2_SUBSTITUTE_EXTENDED
1246         substitute_overflow_length  PCRE2_SUBSTITUTE_OVERFLOW_LENGTH
1247         substitute_unknown_unset    PCRE2_SUBSTITUTE_UNKNOWN_UNSET
1248         substitute_unset_empty      PCRE2_SUBSTITUTE_UNSET_EMPTY
1249
1250
1251       After a successful substitution, the modified string  is  output,  pre‐
1252       ceded  by the number of replacements. This may be zero if there were no
1253       matches. Here is a simple example of a substitution test:
1254
1255         /abc/replace=xxx
1256             =abc=abc=
1257          1: =xxx=abc=
1258             =abc=abc=\=global
1259          2: =xxx=xxx=
1260
1261       Subject and replacement strings should be kept relatively short  (fewer
1262       than  256 characters) for substitution tests, as fixed-size buffers are
1263       used. To make it easy to test for buffer overflow, if  the  replacement
1264       string  starts  with a number in square brackets, that number is passed
1265       to pcre2_substitute() as the  size  of  the  output  buffer,  with  the
1266       replacement  string  starting at the next character. Here is an example
1267       that tests the edge case:
1268
1269         /abc/
1270             123abc123\=replace=[10]XYZ
1271          1: 123XYZ123
1272             123abc123\=replace=[9]XYZ
1273         Failed: error -47: no more memory
1274
1275       The   default   action   of    pcre2_substitute()    is    to    return
1276       PCRE2_ERROR_NOMEMORY  when  the output buffer is too small. However, if
1277       the PCRE2_SUBSTITUTE_OVERFLOW_LENGTH option is set (by using  the  sub‐
1278       stitute_overflow_length  modifier),  pcre2_substitute() continues to go
1279       through the motions of matching and substituting  (but  not  doing  any
1280       callouts),  in  order  to  compute the size of buffer that is required.
1281       When this happens, pcre2test shows the required  buffer  length  (which
1282       includes space for the trailing zero) as part of the error message. For
1283       example:
1284
1285         /abc/substitute_overflow_length
1286             123abc123\=replace=[9]XYZ
1287         Failed: error -47: no more memory: 10 code units are needed
1288
1289       A replacement string is ignored with POSIX and DFA matching. Specifying
1290       partial  matching  provokes  an  error return ("bad option value") from
1291       pcre2_substitute().
1292
1293   Testing substitute callouts
1294
1295       If the substitute_callout modifier is set, a substitution callout func‐
1296       tion  is set up. The null_context modifier must not be set, because the
1297       address of the callout function is passed in a match context. When  the
1298       callout  function  is  called (after each substitution), details of the
1299       the input and output strings are output. For example:
1300
1301         /abc/g,replace=<$0>,substitute_callout
1302             abcdefabcpqr
1303          1(1) Old 0 3 "abc" New 0 5 "<abc>"
1304          2(1) Old 6 9 "abc" New 8 13 "<abc>"
1305          2: <abc>def<abc>pqr
1306
1307       The first number on each callout line is  the  count  of  matches.  The
1308       parenthesized number is the number of pairs that are set in the ovector
1309       (that is, one more than the number of capturing groups that were  set).
1310       Then are listed the offsets of the old substring, its contents, and the
1311       same for the replacement.
1312
1313       By default, the  substitution  callout  function  returns  zero,  which
1314       accepts the replacement and causes matching to continue if /g was used.
1315       Two further modifiers can be used to test other return values. If  sub‐
1316       stitute_skip  is  set to a value greater than zero the callout function
1317       returns +1 for the match of that number, and similarly  substitute_stop
1318       returns  -1.  These cause the replacement to be rejected, and -1 causes
1319       no further matching to take place. If either of them are  set,  substi‐
1320       tute_callout is assumed. For example:
1321
1322         /abc/g,replace=<$0>,substitute_skip=1
1323             abcdefabcpqr
1324          1(1) Old 0 3 "abc" New 0 5 "<abc> SKIPPED"
1325          2(1) Old 6 9 "abc" New 6 11 "<abc>"
1326          2: abcdef<abc>pqr
1327             abcdefabcpqr\=substitute_stop=1
1328          1(1) Old 0 3 "abc" New 0 5 "<abc> STOPPED"
1329          1: abcdefabcpqr
1330
1331       If both are set for the same number, stop takes precedence. Only a sin‐
1332       gle skip or stop is supported, which is sufficient for testing that the
1333       feature works.
1334
1335   Setting the JIT stack size
1336
1337       The  jitstack modifier provides a way of setting the maximum stack size
1338       that is used by the just-in-time optimization code. It  is  ignored  if
1339       JIT  optimization is not being used. The value is a number of kibibytes
1340       (units of 1024 bytes). Setting zero reverts to the  default  of  32KiB.
1341       Providing a stack that is larger than the default is necessary only for
1342       very complicated patterns. If jitstack is set  non-zero  on  a  subject
1343       line it overrides any value that was set on the pattern.
1344
1345   Setting heap, match, and depth limits
1346
1347       The  heap_limit,  match_limit, and depth_limit modifiers set the appro‐
1348       priate limits in the match context. These values are ignored  when  the
1349       find_limits modifier is specified.
1350
1351   Finding minimum limits
1352
1353       If  the  find_limits  modifier  is present on a subject line, pcre2test
1354       calls the relevant matching function several times,  setting  different
1355       values    in    the    match    context   via   pcre2_set_heap_limit(),
1356       pcre2_set_match_limit(), or pcre2_set_depth_limit() until it finds  the
1357       minimum  values  for  each  parameter that allows the match to complete
1358       without error. If JIT is being used, only the match limit is relevant.
1359
1360       When using this modifier, the pattern should not contain any limit set‐
1361       tings  such  as  (*LIMIT_MATCH=...)  within  it.  If  such a setting is
1362       present and is lower than the minimum matching value, the minimum value
1363       cannot  be  found because pcre2_set_match_limit() etc. are only able to
1364       reduce the value of an in-pattern limit; they cannot increase it.
1365
1366       For non-DFA matching, the minimum depth_limit number is  a  measure  of
1367       how much nested backtracking happens (that is, how deeply the pattern's
1368       tree is searched). In the case of DFA  matching,  depth_limit  controls
1369       the  depth of recursive calls of the internal function that is used for
1370       handling pattern recursion, lookaround assertions, and atomic groups.
1371
1372       For non-DFA matching, the match_limit number is a measure of the amount
1373       of backtracking that takes place, and learning the minimum value can be
1374       instructive. For most simple matches, the number is  quite  small,  but
1375       for  patterns with very large numbers of matching possibilities, it can
1376       become large very quickly with increasing length of subject string.  In
1377       the  case  of  DFA  matching,  match_limit controls the total number of
1378       calls, both recursive and non-recursive, to the internal matching func‐
1379       tion, thus controlling the overall amount of computing resource that is
1380       used.
1381
1382       For both  kinds  of  matching,  the  heap_limit  number,  which  is  in
1383       kibibytes  (units of 1024 bytes), limits the amount of heap memory used
1384       for matching. A value of zero disables the use of any heap memory; many
1385       simple  pattern  matches can be done without using the heap, so zero is
1386       not an unreasonable setting.
1387
1388   Showing MARK names
1389
1390
1391       The mark modifier causes the names from backtracking control verbs that
1392       are  returned from calls to pcre2_match() to be displayed. If a mark is
1393       returned for a match, non-match, or partial match, pcre2test shows  it.
1394       For  a  match, it is on a line by itself, tagged with "MK:". Otherwise,
1395       it is added to the non-match message.
1396
1397   Showing memory usage
1398
1399       The memory modifier causes pcre2test to log the sizes of all heap  mem‐
1400       ory   allocation  and  freeing  calls  that  occur  during  a  call  to
1401       pcre2_match() or pcre2_dfa_match().  These  occur  only  when  a  match
1402       requires  a bigger vector than the default for remembering backtracking
1403       points (pcre2_match()) or for internal  workspace  (pcre2_dfa_match()).
1404       In  many cases there will be no heap memory used and therefore no addi‐
1405       tional output. No heap memory is allocated during matching with JIT, so
1406       in  that  case the memory modifier never has any effect. For this modi‐
1407       fier to work, the null_context modifier must not be  set  on  both  the
1408       pattern and the subject, though it can be set on one or the other.
1409
1410   Setting a starting offset
1411
1412       The  offset  modifier  sets  an  offset  in the subject string at which
1413       matching starts. Its value is a number of code units, not characters.
1414
1415   Setting an offset limit
1416
1417       The offset_limit modifier sets a limit for  unanchored  matches.  If  a
1418       match cannot be found starting at or before this offset in the subject,
1419       a "no match" return is given. The data value is a number of code units,
1420       not  characters. When this modifier is used, the use_offset_limit modi‐
1421       fier must have been set for the pattern; if not, an error is generated.
1422
1423   Setting the size of the output vector
1424
1425       The ovector modifier applies only to  the  subject  line  in  which  it
1426       appears,  though  of  course  it can also be used to set a default in a
1427       #subject command. It specifies the number of pairs of offsets that  are
1428       available for storing matching information. The default is 15.
1429
1430       A  value of zero is useful when testing the POSIX API because it causes
1431       regexec() to be called with a NULL capture vector. When not testing the
1432       POSIX  API,  a  value  of  zero  is used to cause pcre2_match_data_cre‐
1433       ate_from_pattern() to be called, in order to create a  match  block  of
1434       exactly the right size for the pattern. (It is not possible to create a
1435       match block with a zero-length ovector; there is always  at  least  one
1436       pair of offsets.)
1437
1438   Passing the subject as zero-terminated
1439
1440       By default, the subject string is passed to a native API matching func‐
1441       tion with its correct length. In order to test the facility for passing
1442       a  zero-terminated  string, the zero_terminate modifier is provided. It
1443       causes the length to be passed as PCRE2_ZERO_TERMINATED. When  matching
1444       via the POSIX interface, this modifier is ignored, with a warning.
1445
1446       When  testing  pcre2_substitute(), this modifier also has the effect of
1447       passing the replacement string as zero-terminated.
1448
1449   Passing a NULL context
1450
1451       Normally,  pcre2test  passes  a   context   block   to   pcre2_match(),
1452       pcre2_dfa_match(),  pcre2_jit_match()  or  pcre2_substitute().   If the
1453       null_context modifier is set, however, NULL  is  passed.  This  is  for
1454       testing  that  the matching and substitution functions behave correctly
1455       in this case (they use default values). This modifier  cannot  be  used
1456       with the find_limits or substitute_callout modifiers.
1457

THE ALTERNATIVE MATCHING FUNCTION

1459
1460       By  default,  pcre2test  uses  the  standard  PCRE2  matching function,
1461       pcre2_match() to match each subject line. PCRE2 also supports an alter‐
1462       native  matching  function, pcre2_dfa_match(), which operates in a dif‐
1463       ferent way, and has some restrictions. The differences between the  two
1464       functions are described in the pcre2matching documentation.
1465
1466       If  the dfa modifier is set, the alternative matching function is used.
1467       This function finds all possible matches at a given point in  the  sub‐
1468       ject.  If,  however, the dfa_shortest modifier is set, processing stops
1469       after the first match is found. This is always  the  shortest  possible
1470       match.
1471

DEFAULT OUTPUT FROM pcre2test

1473
1474       This  section  describes  the output when the normal matching function,
1475       pcre2_match(), is being used.
1476
1477       When a match succeeds, pcre2test outputs  the  list  of  captured  sub‐
1478       strings,  starting  with number 0 for the string that matched the whole
1479       pattern.   Otherwise,  it  outputs  "No  match"  when  the  return   is
1480       PCRE2_ERROR_NOMATCH,  or  "Partial  match:"  followed  by the partially
1481       matching substring when the return is PCRE2_ERROR_PARTIAL.  (Note  that
1482       this  is  the  entire  substring  that was inspected during the partial
1483       match; it may include characters before the actual  match  start  if  a
1484       lookbehind assertion, \K, \b, or \B was involved.)
1485
1486       For any other return, pcre2test outputs the PCRE2 negative error number
1487       and a short descriptive phrase. If the error is  a  failed  UTF  string
1488       check,  the  code  unit offset of the start of the failing character is
1489       also output. Here is an example of an interactive pcre2test run.
1490
1491         $ pcre2test
1492         PCRE2 version 10.22 2016-07-29
1493
1494           re> /^abc(\d+)/
1495         data> abc123
1496          0: abc123
1497          1: 123
1498         data> xyz
1499         No match
1500
1501       Unset capturing substrings that are not followed by one that is set are
1502       not shown by pcre2test unless the allcaptures modifier is specified. In
1503       the following example, there are two capturing substrings, but when the
1504       first  data  line is matched, the second, unset substring is not shown.
1505       An "internal" unset substring is shown as "<unset>", as for the  second
1506       data line.
1507
1508           re> /(a)|(b)/
1509         data> a
1510          0: a
1511          1: a
1512         data> b
1513          0: b
1514          1: <unset>
1515          2: b
1516
1517       If  the strings contain any non-printing characters, they are output as
1518       \xhh escapes if the value is less than 256 and UTF  mode  is  not  set.
1519       Otherwise they are output as \x{hh...} escapes. See below for the defi‐
1520       nition of non-printing characters. If the aftertext  modifier  is  set,
1521       the  output  for substring 0 is followed by the the rest of the subject
1522       string, identified by "0+" like this:
1523
1524           re> /cat/aftertext
1525         data> cataract
1526          0: cat
1527          0+ aract
1528
1529       If global matching is requested, the  results  of  successive  matching
1530       attempts are output in sequence, like this:
1531
1532           re> /\Bi(\w\w)/g
1533         data> Mississippi
1534          0: iss
1535          1: ss
1536          0: iss
1537          1: ss
1538          0: ipp
1539          1: pp
1540
1541       "No  match" is output only if the first match attempt fails. Here is an
1542       example of a failure message (the offset 4 that  is  specified  by  the
1543       offset modifier is past the end of the subject string):
1544
1545           re> /xyz/
1546         data> xyz\=offset=4
1547         Error -24 (bad offset value)
1548
1549       Note that whereas patterns can be continued over several lines (a plain
1550       ">" prompt is used for continuations), subject lines may  not.  However
1551       newlines can be included in a subject by means of the \n escape (or \r,
1552       \r\n, etc., depending on the newline sequence setting).
1553

OUTPUT FROM THE ALTERNATIVE MATCHING FUNCTION

1555
1556       When the alternative matching function, pcre2_dfa_match(), is used, the
1557       output  consists  of  a list of all the matches that start at the first
1558       point in the subject where there is at least one match. For example:
1559
1560           re> /(tang|tangerine|tan)/
1561         data> yellow tangerine\=dfa
1562          0: tangerine
1563          1: tang
1564          2: tan
1565
1566       Using the normal matching function on this data finds only "tang".  The
1567       longest  matching  string  is  always  given first (and numbered zero).
1568       After a PCRE2_ERROR_PARTIAL return, the  output  is  "Partial  match:",
1569       followed  by  the  partially  matching substring. Note that this is the
1570       entire substring that was inspected during the partial  match;  it  may
1571       include characters before the actual match start if a lookbehind asser‐
1572       tion, \b, or \B was involved. (\K is not supported for DFA matching.)
1573
1574       If global matching is requested, the search for further matches resumes
1575       at the end of the longest match. For example:
1576
1577           re> /(tang|tangerine|tan)/g
1578         data> yellow tangerine and tangy sultana\=dfa
1579          0: tangerine
1580          1: tang
1581          2: tan
1582          0: tang
1583          1: tan
1584          0: tan
1585
1586       The  alternative  matching function does not support substring capture,
1587       so the modifiers that are concerned with captured  substrings  are  not
1588       relevant.
1589

RESTARTING AFTER A PARTIAL MATCH

1591
1592       When  the  alternative matching function has given the PCRE2_ERROR_PAR‐
1593       TIAL return, indicating that the subject partially matched the pattern,
1594       you  can restart the match with additional subject data by means of the
1595       dfa_restart modifier. For example:
1596
1597           re> /^\d?\d(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\d\d$/
1598         data> 23ja\=P,dfa
1599         Partial match: 23ja
1600         data> n05\=dfa,dfa_restart
1601          0: n05
1602
1603       For further information about partial matching,  see  the  pcre2partial
1604       documentation.
1605

CALLOUTS

1607
1608       If the pattern contains any callout requests, pcre2test's callout func‐
1609       tion is called during matching unless callout_none is  specified.  This
1610       works with both matching functions, and with JIT, though there are some
1611       differences in behaviour. The output for callouts with numerical  argu‐
1612       ments and those with string arguments is slightly different.
1613
1614   Callouts with numerical arguments
1615
1616       By default, the callout function displays the callout number, the start
1617       and current positions in the subject text at the callout time, and  the
1618       next pattern item to be tested. For example:
1619
1620         --->pqrabcdef
1621           0    ^  ^     \d
1622
1623       This  output  indicates  that  callout  number  0  occurred for a match
1624       attempt starting at the fourth character of the  subject  string,  when
1625       the  pointer  was  at  the seventh character, and when the next pattern
1626       item was \d. Just one circumflex is output if  the  start  and  current
1627       positions  are  the same, or if the current position precedes the start
1628       position, which can happen if the callout is in a lookbehind assertion.
1629
1630       Callouts numbered 255 are assumed to be automatic callouts, inserted as
1631       a result of the auto_callout pattern modifier. In this case, instead of
1632       showing the callout number, the offset in the pattern,  preceded  by  a
1633       plus, is output. For example:
1634
1635           re> /\d?[A-E]\*/auto_callout
1636         data> E*
1637         --->E*
1638          +0 ^      \d?
1639          +3 ^      [A-E]
1640          +8 ^^     \*
1641         +10 ^ ^
1642          0: E*
1643
1644       If a pattern contains (*MARK) items, an additional line is output when‐
1645       ever a change of latest mark is passed to  the  callout  function.  For
1646       example:
1647
1648           re> /a(*MARK:X)bc/auto_callout
1649         data> abc
1650         --->abc
1651          +0 ^       a
1652          +1 ^^      (*MARK:X)
1653         +10 ^^      b
1654         Latest Mark: X
1655         +11 ^ ^     c
1656         +12 ^  ^
1657          0: abc
1658
1659       The  mark  changes between matching "a" and "b", but stays the same for
1660       the rest of the match, so nothing more is output. If, as  a  result  of
1661       backtracking,  the  mark  reverts to being unset, the text "<unset>" is
1662       output.
1663
1664   Callouts with string arguments
1665
1666       The output for a callout with a string argument is similar, except that
1667       instead  of outputting a callout number before the position indicators,
1668       the callout string and its offset in  the  pattern  string  are  output
1669       before  the reflection of the subject string, and the subject string is
1670       reflected for each callout. For example:
1671
1672           re> /^ab(?C'first')cd(?C"second")ef/
1673         data> abcdefg
1674         Callout (7): 'first'
1675         --->abcdefg
1676             ^ ^         c
1677         Callout (20): "second"
1678         --->abcdefg
1679             ^   ^       e
1680          0: abcdef
1681
1682
1683   Callout modifiers
1684
1685       The callout function in pcre2test returns zero (carry on  matching)  by
1686       default,  but  you can use a callout_fail modifier in a subject line to
1687       change this and other parameters of the callout (see below).
1688
1689       If the callout_capture modifier is set, the current captured groups are
1690       output when a callout occurs. This is useful only for non-DFA matching,
1691       as pcre2_dfa_match() does not support capturing,  so  no  captures  are
1692       ever shown.
1693
1694       The normal callout output, showing the callout number or pattern offset
1695       (as described above) is suppressed if the callout_no_where modifier  is
1696       set.
1697
1698       When  using  the  interpretive  matching function pcre2_match() without
1699       JIT, setting the callout_extra modifier causes additional  output  from
1700       pcre2test's  callout function to be generated. For the first callout in
1701       a match attempt at a new starting position in the subject,  "New  match
1702       attempt"  is output. If there has been a backtrack since the last call‐
1703       out (or start of matching if this is the first callout), "Backtrack" is
1704       output,  followed  by  "No other matching paths" if the backtrack ended
1705       the previous match attempt. For example:
1706
1707          re> /(a+)b/auto_callout,no_start_optimize,no_auto_possess
1708         data> aac\=callout_extra
1709         New match attempt
1710         --->aac
1711          +0 ^       (
1712          +1 ^       a+
1713          +3 ^ ^     )
1714          +4 ^ ^     b
1715         Backtrack
1716         --->aac
1717          +3 ^^      )
1718          +4 ^^      b
1719         Backtrack
1720         No other matching paths
1721         New match attempt
1722         --->aac
1723          +0  ^      (
1724          +1  ^      a+
1725          +3  ^^     )
1726          +4  ^^     b
1727         Backtrack
1728         No other matching paths
1729         New match attempt
1730         --->aac
1731          +0   ^     (
1732          +1   ^     a+
1733         Backtrack
1734         No other matching paths
1735         New match attempt
1736         --->aac
1737          +0    ^    (
1738          +1    ^    a+
1739         No match
1740
1741       Notice that various optimizations must be turned off if  you  want  all
1742       possible  matching  paths  to  be  scanned. If no_start_optimize is not
1743       used, there is an immediate "no match", without any  callouts,  because
1744       the  starting  optimization  fails to find "b" in the subject, which it
1745       knows must be present for any match. If no_auto_possess  is  not  used,
1746       the  "a+"  item is turned into "a++", which reduces the number of back‐
1747       tracks.
1748
1749       The callout_extra modifier has no effect if used with the DFA  matching
1750       function, or with JIT.
1751
1752   Return values from callouts
1753
1754       The  default  return  from  the  callout function is zero, which allows
1755       matching to continue. The callout_fail modifier can be given one or two
1756       numbers. If there is only one number, 1 is returned instead of 0 (caus‐
1757       ing matching to backtrack) when a callout of that number is reached. If
1758       two  numbers  (<n>:<m>)  are  given,  1 is returned when callout <n> is
1759       reached and there have been at least <m>  callouts.  The  callout_error
1760       modifier is similar, except that PCRE2_ERROR_CALLOUT is returned, caus‐
1761       ing the entire matching process to be aborted. If both these  modifiers
1762       are  set  for  the same callout number, callout_error takes precedence.
1763       Note that callouts with string arguments are always  given  the  number
1764       zero.
1765
1766       The  callout_data  modifier can be given an unsigned or a negative num‐
1767       ber.  This is set as the "user data" that is  passed  to  the  matching
1768       function,  and  passed  back  when the callout function is invoked. Any
1769       value other than zero is used as  a  return  from  pcre2test's  callout
1770       function.
1771
1772       Inserting callouts can be helpful when using pcre2test to check compli‐
1773       cated regular expressions. For further information about callouts,  see
1774       the pcre2callout documentation.
1775

NON-PRINTING CHARACTERS

1777
1778       When pcre2test is outputting text in the compiled version of a pattern,
1779       bytes other than 32-126 are always treated as  non-printing  characters
1780       and are therefore shown as hex escapes.
1781
1782       When  pcre2test  is outputting text that is a matched part of a subject
1783       string, it behaves in the same way, unless a different locale has  been
1784       set  for  the  pattern  (using  the locale modifier). In this case, the
1785       isprint() function is used to  distinguish  printing  and  non-printing
1786       characters.
1787

SAVING AND RESTORING COMPILED PATTERNS

1789
1790       It  is  possible  to  save  compiled patterns on disc or elsewhere, and
1791       reload them later, subject to a number of restrictions. JIT data cannot
1792       be  saved.  The host on which the patterns are reloaded must be running
1793       the same version of PCRE2, with the same code unit width, and must also
1794       have  the  same  endianness,  pointer width and PCRE2_SIZE type. Before
1795       compiled patterns can be saved they must be serialized, that  is,  con‐
1796       verted  to a stream of bytes. A single byte stream may contain any num‐
1797       ber of compiled patterns, but they must  all  use  the  same  character
1798       tables. A single copy of the tables is included in the byte stream (its
1799       size is 1088 bytes).
1800
1801       The functions whose names begin  with  pcre2_serialize_  are  used  for
1802       serializing  and de-serializing. They are described in the pcre2serial‐
1803       ize  documentation.  In  this  section  we  describe  the  features  of
1804       pcre2test that can be used to test these functions.
1805
1806       Note  that  "serialization" in PCRE2 does not convert compiled patterns
1807       to an abstract format like Java or .NET. It  just  makes  a  reloadable
1808       byte code stream.  Hence the restrictions on reloading mentioned above.
1809
1810       In  pcre2test,  when  a pattern with push modifier is successfully com‐
1811       piled, it is pushed onto a stack of compiled  patterns,  and  pcre2test
1812       expects  the next line to contain a new pattern (or command) instead of
1813       a subject line. By contrast, the pushcopy modifier causes a copy of the
1814       compiled  pattern  to  be  stacked,  leaving the original available for
1815       immediate matching. By using push and/or pushcopy, a number of patterns
1816       can  be  compiled  and  retained. These modifiers are incompatible with
1817       posix, and control modifiers that act at match time are ignored (with a
1818       message)  for the stacked patterns. The jitverify modifier applies only
1819       at compile time.
1820
1821       The command
1822
1823         #save <filename>
1824
1825       causes all the stacked patterns to be serialized and the result written
1826       to  the named file. Afterwards, all the stacked patterns are freed. The
1827       command
1828
1829         #load <filename>
1830
1831       reads the data in the file, and then arranges for it to  be  de-serial‐
1832       ized,  with the resulting compiled patterns added to the pattern stack.
1833       The pattern on the top of the stack can be retrieved by the  #pop  com‐
1834       mand,  which  must  be  followed  by  lines  of subjects that are to be
1835       matched with the pattern, terminated as usual by an empty line  or  end
1836       of  file.  This  command  may be followed by a modifier list containing
1837       only control modifiers that act after a pattern has been  compiled.  In
1838       particular,  hex,  posix,  posix_nosub,  push,  and  pushcopy  are  not
1839       allowed, nor are any option-setting modifiers.  The JIT modifiers  are,
1840       however  permitted.  Here is an example that saves and reloads two pat‐
1841       terns.
1842
1843         /abc/push
1844         /xyz/push
1845         #save tempfile
1846         #load tempfile
1847         #pop info
1848         xyz
1849
1850         #pop jit,bincode
1851         abc
1852
1853       If jitverify is used with #pop, it does not  automatically  imply  jit,
1854       which is different behaviour from when it is used on a pattern.
1855
1856       The  #popcopy  command is analagous to the pushcopy modifier in that it
1857       makes current a copy of the topmost stack pattern, leaving the original
1858       still on the stack.
1859

SEE ALSO

1861
1862       pcre2(3),  pcre2api(3),  pcre2callout(3),  pcre2jit,  pcre2matching(3),
1863       pcre2partial(d), pcre2pattern(3), pcre2serialize(3).
1864

AUTHOR

1866
1867       Philip Hazel
1868       University Computing Service
1869       Cambridge, England.
1870

REVISION

1872
1873       Last updated: 11 March 2019
1874       Copyright (c) 1997-2019 University of Cambridge.
1875
1876
1877
1878PCRE 10.33                       11 March 2019                    PCRE2TEST(1)
Impressum