1Text::Balanced(3) User Contributed Perl Documentation Text::Balanced(3)
2
6 Text::Balanced - Extract delimited text sequences from strings.
7
9 use Text::Balanced qw (
10 extract_delimited
11 extract_bracketed
12 extract_quotelike
13 extract_codeblock
14 extract_variable
15 extract_tagged
16 extract_multiple
17 gen_delimited_pat
18 gen_extract_tagged
19 );
20 # Extract the initial substring of $text that is delimited by
22 # two (unescaped) instances of the first character in $delim.
23 ($extracted, $remainder) = extract_delimited($text,$delim);
25 # Extract the initial substring of $text that is bracketed
27 # with a delimiter(s) specified by $delim (where the string
28 # in $delim contains one or more of '(){}[]<>').
29 ($extracted, $remainder) = extract_bracketed($text,$delim);
31 # Extract the initial substring of $text that is bounded by
33 # an XML tag.
34 ($extracted, $remainder) = extract_tagged($text);
36 # Extract the initial substring of $text that is bounded by
38 # a C<BEGIN>...C<END> pair. Don't allow nested C<BEGIN> tags
39 ($extracted, $remainder) =
41 extract_tagged($text,"BEGIN","END",undef,{bad=>["BEGIN"]});
42 # Extract the initial substring of $text that represents a
44 # Perl "quote or quote-like operation"
45 ($extracted, $remainder) = extract_quotelike($text);
47 # Extract the initial substring of $text that represents a block
49 # of Perl code, bracketed by any of character(s) specified by $delim
50 # (where the string $delim contains one or more of '(){}[]<>').
51 ($extracted, $remainder) = extract_codeblock($text,$delim);
53 # Extract the initial substrings of $text that would be extracted by
55 # one or more sequential applications of the specified functions
56 # or regular expressions
57 @extracted = extract_multiple($text,
59 [ \&extract_bracketed,
60 \&extract_quotelike,
61 \&some_other_extractor_sub,
62 qr/[xyz]*/,
63 'literal',
64 ]);
65 # Create a string representing an optimized pattern (a la Friedl)
67 # that matches a substring delimited by any of the specified characters
68 # (in this case: any type of quote or a slash)
69 $patstring = gen_delimited_pat(q{'"`/});
71 # Generate a reference to an anonymous sub that is just like extract_tagged
73 # but pre-compiled and optimized for a specific pair of tags, and
74 # consequently much faster (i.e. 3 times faster). It uses qr// for better
75 # performance on repeated calls.
76 $extract_head = gen_extract_tagged('<HEAD>','</HEAD>');
78 ($extracted, $remainder) = $extract_head->($text);
79
81 The various "extract_..." subroutines may be used to extract a
82 delimited substring, possibly after skipping a specified prefix string.
83 By default, that prefix is optional whitespace ("/\s*/"), but you can
84 change it to whatever you wish (see below).
85 The substring to be extracted must appear at the current "pos" location
87 of the string's variable (or at index zero, if no "pos" position is
88 defined). In other words, the "extract_..." subroutines don't extract
89 the first occurrence of a substring anywhere in a string (like an
90 unanchored regex would). Rather, they extract an occurrence of the
91 substring appearing immediately at the current matching position in the
92 string (like a "\G"-anchored regex would).
93 General Behaviour in List Contexts
95 In a list context, all the subroutines return a list, the first three
96 elements of which are always:
97 [0] The extracted string, including the specified delimiters. If the
99 extraction fails "undef" is returned.
100 [1] The remainder of the input string (i.e. the characters after the
102 extracted string). On failure, the entire string is returned.
103 [2] The skipped prefix (i.e. the characters before the extracted
105 string). On failure, "undef" is returned.
106 Note that in a list context, the contents of the original input text
108 (the first argument) are not modified in any way.
109 However, if the input text was passed in a variable, that variable's
111 "pos" value is updated to point at the first character after the
112 extracted text. That means that in a list context the various
113 subroutines can be used much like regular expressions. For example:
114 while ( $next = (extract_quotelike($text))[0] )
116 {
117 # process next quote-like (in $next)
118 }
119 General Behaviour in Scalar and Void Contexts
121 In a scalar context, the extracted string is returned, having first
122 been removed from the input text. Thus, the following code also
123 processes each quote-like operation, but actually removes them from
124 $text:
125 while ( $next = extract_quotelike($text) )
127 {
128 # process next quote-like (in $next)
129 }
130 Note that if the input text is a read-only string (i.e. a literal), no
132 attempt is made to remove the extracted text.
133 In a void context the behaviour of the extraction subroutines is
135 exactly the same as in a scalar context, except (of course) that the
136 extracted substring is not returned.
137 A Note About Prefixes
139 Prefix patterns are matched without any trailing modifiers ("/gimsox"
140 etc.) This can bite you if you're expecting a prefix specification
141 like '.*?(?=<H1>)' to skip everything up to the first <H1> tag. Such a
142 prefix pattern will only succeed if the <H1> tag is on the current
143 line, since . normally doesn't match newlines.
144 To overcome this limitation, you need to turn on /s matching within the
146 prefix pattern, using the "(?s)" directive: '(?s).*?(?=<H1>)'
147 Functions
149 "extract_delimited"
150 The "extract_delimited" function formalizes the common idiom of
151 extracting a single-character-delimited substring from the start of
152 a string. For example, to extract a single-quote delimited string,
153 the following code is typically used:
154 ($remainder = $text) =~ s/\A('(\\.|[^'])*')//s;
156 $extracted = $1;
157 but with "extract_delimited" it can be simplified to:
159 ($extracted,$remainder) = extract_delimited($text, "'");
161 "extract_delimited" takes up to four scalars (the input text, the
163 delimiters, a prefix pattern to be skipped, and any escape
164 characters) and extracts the initial substring of the text that is
165 appropriately delimited. If the delimiter string has multiple
166 characters, the first one encountered in the text is taken to
167 delimit the substring. The third argument specifies a prefix
168 pattern that is to be skipped (but must be present!) before the
169 substring is extracted. The final argument specifies the escape
170 character to be used for each delimiter.
171 All arguments are optional. If the escape characters are not
173 specified, every delimiter is escaped with a backslash ("\"). If
174 the prefix is not specified, the pattern '\s*' - optional
175 whitespace - is used. If the delimiter set is also not specified,
176 the set "/["'`]/" is used. If the text to be processed is not
177 specified either, $_ is used.
178 In list context, "extract_delimited" returns a array of three
180 elements, the extracted substring (including the surrounding
181 delimiters), the remainder of the text, and the skipped prefix (if
182 any). If a suitable delimited substring is not found, the first
183 element of the array is the empty string, the second is the
184 complete original text, and the prefix returned in the third
185 element is an empty string.
186 In a scalar context, just the extracted substring is returned. In a
188 void context, the extracted substring (and any prefix) are simply
189 removed from the beginning of the first argument.
190 Examples:
192 # Remove a single-quoted substring from the very beginning of $text:
194 $substring = extract_delimited($text, "'", '');
196 # Remove a single-quoted Pascalish substring (i.e. one in which
198 # doubling the quote character escapes it) from the very
199 # beginning of $text:
200 $substring = extract_delimited($text, "'", '', "'");
202 # Extract a single- or double- quoted substring from the
204 # beginning of $text, optionally after some whitespace
205 # (note the list context to protect $text from modification):
206 ($substring) = extract_delimited $text, q{"'};
208 # Delete the substring delimited by the first '/' in $text:
210 $text = join '', (extract_delimited($text,'/','[^/]*')[2,1];
212 Note that this last example is not the same as deleting the first
214 quote-like pattern. For instance, if $text contained the string:
215 "if ('./cmd' =~ m/$UNIXCMD/s) { $cmd = $1; }"
217 then after the deletion it would contain:
219 "if ('.$UNIXCMD/s) { $cmd = $1; }"
221 not:
223 "if ('./cmd' =~ ms) { $cmd = $1; }"
225 See "extract_quotelike" for a (partial) solution to this problem.
227 "extract_bracketed"
229 Like "extract_delimited", the "extract_bracketed" function takes up
230 to three optional scalar arguments: a string to extract from, a
231 delimiter specifier, and a prefix pattern. As before, a missing
232 prefix defaults to optional whitespace and a missing text defaults
233 to $_. However, a missing delimiter specifier defaults to
234 '{}()[]<>' (see below).
235 "extract_bracketed" extracts a balanced-bracket-delimited substring
237 (using any one (or more) of the user-specified delimiter brackets:
238 '(..)', '{..}', '[..]', or '<..>'). Optionally it will also respect
239 quoted unbalanced brackets (see below).
240 A "delimiter bracket" is a bracket in list of delimiters passed as
242 "extract_bracketed"'s second argument. Delimiter brackets are
243 specified by giving either the left or right (or both!) versions of
244 the required bracket(s). Note that the order in which two or more
245 delimiter brackets are specified is not significant.
246 A "balanced-bracket-delimited substring" is a substring bounded by
248 matched brackets, such that any other (left or right) delimiter
249 bracket within the substring is also matched by an opposite (right
250 or left) delimiter bracket at the same level of nesting. Any type
251 of bracket not in the delimiter list is treated as an ordinary
252 character.
253 In other words, each type of bracket specified as a delimiter must
255 be balanced and correctly nested within the substring, and any
256 other kind of ("non-delimiter") bracket in the substring is
257 ignored.
258 For example, given the string:
260 $text = "{ an '[irregularly :-(] {} parenthesized >:-)' string }";
262 then a call to "extract_bracketed" in a list context:
264 @result = extract_bracketed( $text, '{}' );
266 would return:
268 ( "{ an '[irregularly :-(] {} parenthesized >:-)' string }" , "" , "" )
270 since both sets of '{..}' brackets are properly nested and evenly
272 balanced. (In a scalar context just the first element of the array
273 would be returned. In a void context, $text would be replaced by an
274 empty string.)
275 Likewise the call in:
277 @result = extract_bracketed( $text, '{[' );
279 would return the same result, since all sets of both types of
281 specified delimiter brackets are correctly nested and balanced.
282 However, the call in:
284 @result = extract_bracketed( $text, '{([<' );
286 would fail, returning:
288 ( undef , "{ an '[irregularly :-(] {} parenthesized >:-)' string }" );
290 because the embedded pairs of '(..)'s and '[..]'s are "cross-
292 nested" and the embedded '>' is unbalanced. (In a scalar context,
293 this call would return an empty string. In a void context, $text
294 would be unchanged.)
295 Note that the embedded single-quotes in the string don't help in
297 this case, since they have not been specified as acceptable
298 delimiters and are therefore treated as non-delimiter characters
299 (and ignored).
300 However, if a particular species of quote character is included in
302 the delimiter specification, then that type of quote will be
303 correctly handled. for example, if $text is:
304 $text = '<A HREF=">>>>">link</A>';
306 then
308 @result = extract_bracketed( $text, '<">' );
310 returns:
312 ( '<A HREF=">>>>">', 'link</A>', "" )
314 as expected. Without the specification of """ as an embedded
316 quoter:
317 @result = extract_bracketed( $text, '<>' );
319 the result would be:
321 ( '<A HREF=">', '>>>">link</A>', "" )
323 In addition to the quote delimiters "'", """, and "`", full Perl
325 quote-like quoting (i.e. q{string}, qq{string}, etc) can be
326 specified by including the letter 'q' as a delimiter. Hence:
327 @result = extract_bracketed( $text, '<q>' );
329 would correctly match something like this:
331 $text = '<leftop: conj /and/ conj>';
333 See also: "extract_quotelike" and "extract_codeblock".
335 "extract_variable"
337 "extract_variable" extracts any valid Perl variable or variable-
338 involved expression, including scalars, arrays, hashes, array
339 accesses, hash look-ups, method calls through objects, subroutine
340 calls through subroutine references, etc.
341 The subroutine takes up to two optional arguments:
343 1. A string to be processed ($_ if the string is omitted or
345 "undef")
346 2. A string specifying a pattern to be matched as a prefix (which
348 is to be skipped). If omitted, optional whitespace is skipped.
349 On success in a list context, an array of 3 elements is returned.
351 The elements are:
352 [0] the extracted variable, or variablish expression
354 [1] the remainder of the input text,
356 [2] the prefix substring (if any),
358 On failure, all of these values (except the remaining text) are
360 "undef".
361 In a scalar context, "extract_variable" returns just the complete
363 substring that matched a variablish expression. "undef" is returned
364 on failure. In addition, the original input text has the returned
365 substring (and any prefix) removed from it.
366 In a void context, the input text just has the matched substring
368 (and any specified prefix) removed.
369 "extract_tagged"
371 "extract_tagged" extracts and segments text between (balanced)
372 specified tags.
373 The subroutine takes up to five optional arguments:
375 1. A string to be processed ($_ if the string is omitted or
377 "undef")
378 2. A string specifying a pattern (i.e. regex) to be matched as the
380 opening tag. If the pattern string is omitted (or "undef")
381 then a pattern that matches any standard XML tag is used.
382 3. A string specifying a pattern to be matched at the closing tag.
384 If the pattern string is omitted (or "undef") then the closing
385 tag is constructed by inserting a "/" after any leading bracket
386 characters in the actual opening tag that was matched (not the
387 pattern that matched the tag). For example, if the opening tag
388 pattern is specified as '{{\w+}}' and actually matched the
389 opening tag "{{DATA}}", then the constructed closing tag would
390 be "{{/DATA}}".
391 4. A string specifying a pattern to be matched as a prefix (which
393 is to be skipped). If omitted, optional whitespace is skipped.
394 5. A hash reference containing various parsing options (see below)
396 The various options that can be specified are:
398 "reject => $listref"
400 The list reference contains one or more strings specifying
401 patterns that must not appear within the tagged text.
402 For example, to extract an HTML link (which should not contain
404 nested links) use:
405 extract_tagged($text, '<A>', '</A>', undef, {reject => ['<A>']} );
407 "ignore => $listref"
409 The list reference contains one or more strings specifying
410 patterns that are not to be treated as nested tags within the
411 tagged text (even if they would match the start tag pattern).
412 For example, to extract an arbitrary XML tag, but ignore
414 "empty" elements:
415 extract_tagged($text, undef, undef, undef, {ignore => ['<[^>]*/>']} );
417 (also see "gen_delimited_pat" below).
419 "fail => $str"
421 The "fail" option indicates the action to be taken if a
422 matching end tag is not encountered (i.e. before the end of the
423 string or some "reject" pattern matches). By default, a failure
424 to match a closing tag causes "extract_tagged" to immediately
425 fail.
426 However, if the string value associated with <reject> is "MAX",
428 then "extract_tagged" returns the complete text up to the point
429 of failure. If the string is "PARA", "extract_tagged" returns
430 only the first paragraph after the tag (up to the first line
431 that is either empty or contains only whitespace characters).
432 If the string is "", the default behaviour (i.e. failure) is
433 reinstated.
434 For example, suppose the start tag "/para" introduces a
436 paragraph, which then continues until the next "/endpara" tag
437 or until another "/para" tag is encountered:
438 $text = "/para line 1\n\nline 3\n/para line 4";
440 extract_tagged($text, '/para', '/endpara', undef,
442 {reject => '/para', fail => MAX );
443 # EXTRACTED: "/para line 1\n\nline 3\n"
445 Suppose instead, that if no matching "/endpara" tag is found,
447 the "/para" tag refers only to the immediately following
448 paragraph:
449 $text = "/para line 1\n\nline 3\n/para line 4";
451 extract_tagged($text, '/para', '/endpara', undef,
453 {reject => '/para', fail => MAX );
454 # EXTRACTED: "/para line 1\n"
456 Note that the specified "fail" behaviour applies to nested tags
458 as well.
459 On success in a list context, an array of 6 elements is returned.
461 The elements are:
462 [0] the extracted tagged substring (including the outermost tags),
464 [1] the remainder of the input text,
466 [2] the prefix substring (if any),
468 [3] the opening tag
470 [4] the text between the opening and closing tags
472 [5] the closing tag (or "" if no closing tag was found)
474 On failure, all of these values (except the remaining text) are
476 "undef".
477 In a scalar context, "extract_tagged" returns just the complete
479 substring that matched a tagged text (including the start and end
480 tags). "undef" is returned on failure. In addition, the original
481 input text has the returned substring (and any prefix) removed from
482 it.
483 In a void context, the input text just has the matched substring
485 (and any specified prefix) removed.
486 "gen_extract_tagged"
488 "gen_extract_tagged" generates a new anonymous subroutine which
489 extracts text between (balanced) specified tags. In other words, it
490 generates a function identical in function to "extract_tagged".
491 The difference between "extract_tagged" and the anonymous
493 subroutines generated by "gen_extract_tagged", is that those
494 generated subroutines:
495 • do not have to reparse tag specification or parsing options
497 every time they are called (whereas "extract_tagged" has to
498 effectively rebuild its tag parser on every call);
499 • make use of the new qr// construct to pre-compile the regexes
501 they use (whereas "extract_tagged" uses standard string
502 variable interpolation to create tag-matching patterns).
503 The subroutine takes up to four optional arguments (the same set as
505 "extract_tagged" except for the string to be processed). It returns
506 a reference to a subroutine which in turn takes a single argument
507 (the text to be extracted from).
508 In other words, the implementation of "extract_tagged" is exactly
510 equivalent to:
511 sub extract_tagged
513 {
514 my $text = shift;
515 $extractor = gen_extract_tagged(@_);
516 return $extractor->($text);
517 }
518 (although "extract_tagged" is not currently implemented that way).
520 Using "gen_extract_tagged" to create extraction functions for
522 specific tags is a good idea if those functions are going to be
523 called more than once, since their performance is typically twice
524 as good as the more general-purpose "extract_tagged".
525 "extract_quotelike"
527 "extract_quotelike" attempts to recognize, extract, and segment any
528 one of the various Perl quotes and quotelike operators (see
529 perlop(3)) Nested backslashed delimiters, embedded balanced bracket
530 delimiters (for the quotelike operators), and trailing modifiers
531 are all caught. For example, in:
532 extract_quotelike 'q # an octothorpe: \# (not the end of the q!) #'
534 extract_quotelike ' "You said, \"Use sed\"." '
536 extract_quotelike ' s{([A-Z]{1,8}\.[A-Z]{3})} /\L$1\E/; '
538 extract_quotelike ' tr/\\\/\\\\/\\\//ds; '
540 the full Perl quotelike operations are all extracted correctly.
542 Note too that, when using the /x modifier on a regex, any comment
544 containing the current pattern delimiter will cause the regex to be
545 immediately terminated. In other words:
546 'm /
548 (?i) # CASE INSENSITIVE
549 [a-z_] # LEADING ALPHABETIC/UNDERSCORE
550 [a-z0-9]* # FOLLOWED BY ANY NUMBER OF ALPHANUMERICS
551 /x'
552 will be extracted as if it were:
554 'm /
556 (?i) # CASE INSENSITIVE
557 [a-z_] # LEADING ALPHABETIC/'
558 This behaviour is identical to that of the actual compiler.
560 "extract_quotelike" takes two arguments: the text to be processed
562 and a prefix to be matched at the very beginning of the text. If no
563 prefix is specified, optional whitespace is the default. If no text
564 is given, $_ is used.
565 In a list context, an array of 11 elements is returned. The
567 elements are:
568 [0] the extracted quotelike substring (including trailing
570 modifiers),
571 [1] the remainder of the input text,
573 [2] the prefix substring (if any),
575 [3] the name of the quotelike operator (if any),
577 [4] the left delimiter of the first block of the operation,
579 [5] the text of the first block of the operation (that is, the
581 contents of a quote, the regex of a match or substitution or
582 the target list of a translation),
583 [6] the right delimiter of the first block of the operation,
585 [7] the left delimiter of the second block of the operation (that
587 is, if it is a "s", "tr", or "y"),
588 [8] the text of the second block of the operation (that is, the
590 replacement of a substitution or the translation list of a
591 translation),
592 [9] the right delimiter of the second block of the operation (if
594 any),
595 [10]
597 the trailing modifiers on the operation (if any).
598 For each of the fields marked "(if any)" the default value on
600 success is an empty string. On failure, all of these values
601 (except the remaining text) are "undef".
602 In a scalar context, "extract_quotelike" returns just the complete
604 substring that matched a quotelike operation (or "undef" on
605 failure). In a scalar or void context, the input text has the same
606 substring (and any specified prefix) removed.
607 Examples:
609 # Remove the first quotelike literal that appears in text
611 $quotelike = extract_quotelike($text,'.*?');
613 # Replace one or more leading whitespace-separated quotelike
615 # literals in $_ with "<QLL>"
616 do { $_ = join '<QLL>', (extract_quotelike)[2,1] } until $@;
618 # Isolate the search pattern in a quotelike operation from $text
621 ($op,$pat) = (extract_quotelike $text)[3,5];
623 if ($op =~ /[ms]/)
624 {
625 print "search pattern: $pat\n";
626 }
627 else
628 {
629 print "$op is not a pattern matching operation\n";
630 }
631 "extract_quotelike"
633 "extract_quotelike" can successfully extract "here documents" from
634 an input string, but with an important caveat in list contexts.
635 Unlike other types of quote-like literals, a here document is
637 rarely a contiguous substring. For example, a typical piece of code
638 using here document might look like this:
639 <<'EOMSG' || die;
641 This is the message.
642 EOMSG
643 exit;
644 Given this as an input string in a scalar context,
646 "extract_quotelike" would correctly return the string
647 "<<'EOMSG'\nThis is the message.\nEOMSG", leaving the string " ||
648 die;\nexit;" in the original variable. In other words, the two
649 separate pieces of the here document are successfully extracted and
650 concatenated.
651 In a list context, "extract_quotelike" would return the list
653 [0] "<<'EOMSG'\nThis is the message.\nEOMSG\n" (i.e. the full
655 extracted here document, including fore and aft delimiters),
656 [1] " || die;\nexit;" (i.e. the remainder of the input text,
658 concatenated),
659 [2] "" (i.e. the prefix substring -- trivial in this case),
661 [3] "<<" (i.e. the "name" of the quotelike operator)
663 [4] "'EOMSG'" (i.e. the left delimiter of the here document,
665 including any quotes),
666 [5] "This is the message.\n" (i.e. the text of the here document),
668 [6] "EOMSG" (i.e. the right delimiter of the here document),
670 [7..10]
672 "" (a here document has no second left delimiter, second text,
673 second right delimiter, or trailing modifiers).
674 However, the matching position of the input variable would be set
676 to "exit;" (i.e. after the closing delimiter of the here document),
677 which would cause the earlier " || die;\nexit;" to be skipped in
678 any sequence of code fragment extractions.
679 To avoid this problem, when it encounters a here document whilst
681 extracting from a modifiable string, "extract_quotelike" silently
682 rearranges the string to an equivalent piece of Perl:
683 <<'EOMSG'
685 This is the message.
686 EOMSG
687 || die;
688 exit;
689 in which the here document is contiguous. It still leaves the
691 matching position after the here document, but now the rest of the
692 line on which the here document starts is not skipped.
693 To prevent <extract_quotelike> from mucking about with the input in
695 this way (this is the only case where a list-context
696 "extract_quotelike" does so), you can pass the input variable as an
697 interpolated literal:
698 $quotelike = extract_quotelike("$var");
700 "extract_codeblock"
702 "extract_codeblock" attempts to recognize and extract a balanced
703 bracket delimited substring that may contain unbalanced brackets
704 inside Perl quotes or quotelike operations. That is,
705 "extract_codeblock" is like a combination of "extract_bracketed"
706 and "extract_quotelike".
707 "extract_codeblock" takes the same initial three parameters as
709 "extract_bracketed": a text to process, a set of delimiter brackets
710 to look for, and a prefix to match first. It also takes an optional
711 fourth parameter, which allows the outermost delimiter brackets to
712 be specified separately (see below), and a fifth parameter used
713 only by Parse::RecDescent.
714 Omitting the first argument (input text) means process $_ instead.
716 Omitting the second argument (delimiter brackets) indicates that
717 only '{' is to be used. Omitting the third argument (prefix
718 argument) implies optional whitespace at the start. Omitting the
719 fourth argument (outermost delimiter brackets) indicates that the
720 value of the second argument is to be used for the outermost
721 delimiters.
722 Once the prefix and the outermost opening delimiter bracket have
724 been recognized, code blocks are extracted by stepping through the
725 input text and trying the following alternatives in sequence:
726 1. Try and match a closing delimiter bracket. If the bracket was
728 the same species as the last opening bracket, return the
729 substring to that point. If the bracket was mismatched, return
730 an error.
731 2. Try to match a quote or quotelike operator. If found, call
733 "extract_quotelike" to eat it. If "extract_quotelike" fails,
734 return the error it returned. Otherwise go back to step 1.
735 3. Try to match an opening delimiter bracket. If found, call
737 "extract_codeblock" recursively to eat the embedded block. If
738 the recursive call fails, return an error. Otherwise, go back
739 to step 1.
740 4. Unconditionally match a bareword or any other single character,
742 and then go back to step 1.
743 Examples:
745 # Find a while loop in the text
747 if ($text =~ s/.*?while\s*\{/{/)
749 {
750 $loop = "while " . extract_codeblock($text);
751 }
752 # Remove the first round-bracketed list (which may include
754 # round- or curly-bracketed code blocks or quotelike operators)
755 extract_codeblock $text, "(){}", '[^(]*';
757 The ability to specify a different outermost delimiter bracket is
759 useful in some circumstances. For example, in the Parse::RecDescent
760 module, parser actions which are to be performed only on a
761 successful parse are specified using a "<defer:...>" directive. For
762 example:
763 sentence: subject verb object
765 <defer: {$::theVerb = $item{verb}} >
766 Parse::RecDescent uses "extract_codeblock($text, '{}<>')" to
768 extract the code within the "<defer:...>" directive, but there's a
769 problem.
770 A deferred action like this:
772 <defer: {if ($count>10) {$count--}} >
774 will be incorrectly parsed as:
776 <defer: {if ($count>
778 because the "less than" operator is interpreted as a closing
780 delimiter.
781 But, by extracting the directive using
783 "extract_codeblock($text, '{}', undef, '<>')" the '>' character is
784 only treated as a delimited at the outermost level of the code
785 block, so the directive is parsed correctly.
786 "extract_multiple"
788 The "extract_multiple" subroutine takes a string to be processed
789 and a list of extractors (subroutines or regular expressions) to
790 apply to that string.
791 In an array context "extract_multiple" returns an array of
793 substrings of the original string, as extracted by the specified
794 extractors. In a scalar context, "extract_multiple" returns the
795 first substring successfully extracted from the original string. In
796 both scalar and void contexts the original string has the first
797 successfully extracted substring removed from it. In all contexts
798 "extract_multiple" starts at the current "pos" of the string, and
799 sets that "pos" appropriately after it matches.
800 Hence, the aim of a call to "extract_multiple" in a list context is
802 to split the processed string into as many non-overlapping fields
803 as possible, by repeatedly applying each of the specified
804 extractors to the remainder of the string. Thus "extract_multiple"
805 is a generalized form of Perl's "split" subroutine.
806 The subroutine takes up to four optional arguments:
808 1. A string to be processed ($_ if the string is omitted or
810 "undef")
811 2. A reference to a list of subroutine references and/or qr//
813 objects and/or literal strings and/or hash references,
814 specifying the extractors to be used to split the string. If
815 this argument is omitted (or "undef") the list:
816 [
818 sub { extract_variable($_[0], '') },
819 sub { extract_quotelike($_[0],'') },
820 sub { extract_codeblock($_[0],'{}','') },
821 ]
822 is used.
824 3. An number specifying the maximum number of fields to return. If
826 this argument is omitted (or "undef"), split continues as long
827 as possible.
828 If the third argument is N, then extraction continues until N
830 fields have been successfully extracted, or until the string
831 has been completely processed.
832 Note that in scalar and void contexts the value of this
834 argument is automatically reset to 1 (under "-w", a warning is
835 issued if the argument has to be reset).
836 4. A value indicating whether unmatched substrings (see below)
838 within the text should be skipped or returned as fields. If the
839 value is true, such substrings are skipped. Otherwise, they are
840 returned.
841 The extraction process works by applying each extractor in sequence
843 to the text string.
844 If the extractor is a subroutine it is called in a list context and
846 is expected to return a list of a single element, namely the
847 extracted text. It may optionally also return two further
848 arguments: a string representing the text left after extraction
849 (like $' for a pattern match), and a string representing any prefix
850 skipped before the extraction (like $` in a pattern match). Note
851 that this is designed to facilitate the use of other Text::Balanced
852 subroutines with "extract_multiple". Note too that the value
853 returned by an extractor subroutine need not bear any relationship
854 to the corresponding substring of the original text (see examples
855 below).
856 If the extractor is a precompiled regular expression or a string,
858 it is matched against the text in a scalar context with a leading
859 '\G' and the gc modifiers enabled. The extracted value is either $1
860 if that variable is defined after the match, or else the complete
861 match (i.e. $&).
862 If the extractor is a hash reference, it must contain exactly one
864 element. The value of that element is one of the above extractor
865 types (subroutine reference, regular expression, or string). The
866 key of that element is the name of a class into which the
867 successful return value of the extractor will be blessed.
868 If an extractor returns a defined value, that value is immediately
870 treated as the next extracted field and pushed onto the list of
871 fields. If the extractor was specified in a hash reference, the
872 field is also blessed into the appropriate class,
873 If the extractor fails to match (in the case of a regex extractor),
875 or returns an empty list or an undefined value (in the case of a
876 subroutine extractor), it is assumed to have failed to extract. If
877 none of the extractor subroutines succeeds, then one character is
878 extracted from the start of the text and the extraction subroutines
879 reapplied. Characters which are thus removed are accumulated and
880 eventually become the next field (unless the fourth argument is
881 true, in which case they are discarded).
882 For example, the following extracts substrings that are valid Perl
884 variables:
885 @fields = extract_multiple($text,
887 [ sub { extract_variable($_[0]) } ],
888 undef, 1);
889 This example separates a text into fields which are quote
891 delimited, curly bracketed, and anything else. The delimited and
892 bracketed parts are also blessed to identify them (the "anything
893 else" is unblessed):
894 @fields = extract_multiple($text,
896 [
897 { Delim => sub { extract_delimited($_[0],q{'"}) } },
898 { Brack => sub { extract_bracketed($_[0],'{}') } },
899 ]);
900 This call extracts the next single substring that is a valid Perl
902 quotelike operator (and removes it from $text):
903 $quotelike = extract_multiple($text,
905 [
906 sub { extract_quotelike($_[0]) },
907 ], undef, 1);
908 Finally, here is yet another way to do comma-separated value
910 parsing:
911 $csv_text = "a,'x b',c";
913 @fields = extract_multiple($csv_text,
914 [
915 sub { extract_delimited($_[0],q{'"}) },
916 qr/([^,]+)/,
917 ],
918 undef,1);
919 # @fields is now ('a', "'x b'", 'c')
920 The list in the second argument means: "Try and extract a ' or "
922 delimited string, otherwise extract anything up to a comma...".
923 The undef third argument means: "...as many times as possible...",
924 and the true value in the fourth argument means "...discarding
925 anything else that appears (i.e. the commas)".
926 If you wanted the commas preserved as separate fields (i.e. like
928 split does if your split pattern has capturing parentheses), you
929 would just make the last parameter undefined (or remove it).
930 "gen_delimited_pat"
932 The "gen_delimited_pat" subroutine takes a single (string) argument
933 and builds a Friedl-style optimized regex that matches a string
934 delimited by any one of the characters in the single argument. For
935 example:
936 gen_delimited_pat(q{'"})
938 returns the regex:
940 (?:\"(?:\\\"|(?!\").)*\"|\'(?:\\\'|(?!\').)*\')
942 Note that the specified delimiters are automatically quotemeta'd.
944 A typical use of "gen_delimited_pat" would be to build special
946 purpose tags for "extract_tagged". For example, to properly ignore
947 "empty" XML elements (which might contain quoted strings):
948 my $empty_tag = '<(' . gen_delimited_pat(q{'"}) . '|.)+/>';
950 extract_tagged($text, undef, undef, undef, {ignore => [$empty_tag]} );
952 "gen_delimited_pat" may also be called with an optional second
954 argument, which specifies the "escape" character(s) to be used for
955 each delimiter. For example to match a Pascal-style string (where
956 ' is the delimiter and '' is a literal ' within the string):
957 gen_delimited_pat(q{'},q{'});
959 Different escape characters can be specified for different
961 delimiters. For example, to specify that '/' is the escape for
962 single quotes and '%' is the escape for double quotes:
963 gen_delimited_pat(q{'"},q{/%});
965 If more delimiters than escape chars are specified, the last escape
967 char is used for the remaining delimiters. If no escape char is
968 specified for a given specified delimiter, '\' is used.
969 "delimited_pat"
971 Note that "gen_delimited_pat" was previously called
972 "delimited_pat". That name may still be used, but is now
973 deprecated.
974
976 In a list context, all the functions return "(undef,$original_text)" on
977 failure. In a scalar context, failure is indicated by returning "undef"
978 (in this case the input text is not modified in any way).
979 In addition, on failure in any context, the $@ variable is set.
981 Accessing "$@->{error}" returns one of the error diagnostics listed
982 below. Accessing "$@->{pos}" returns the offset into the original
983 string at which the error was detected (although not necessarily where
984 it occurred!) Printing $@ directly produces the error message, with
985 the offset appended. On success, the $@ variable is guaranteed to be
986 "undef".
987 The available diagnostics are:
989 "Did not find a suitable bracket: "%s""
991 The delimiter provided to "extract_bracketed" was not one of
992 '()[]<>{}'.
993 "Did not find prefix: /%s/"
995 A non-optional prefix was specified but wasn't found at the start
996 of the text.
997 "Did not find opening bracket after prefix: "%s""
999 "extract_bracketed" or "extract_codeblock" was expecting a
1000 particular kind of bracket at the start of the text, and didn't
1001 find it.
1002 "No quotelike operator found after prefix: "%s""
1004 "extract_quotelike" didn't find one of the quotelike operators "q",
1005 "qq", "qw", "qx", "s", "tr" or "y" at the start of the substring it
1006 was extracting.
1007 "Unmatched closing bracket: "%c""
1009 "extract_bracketed", "extract_quotelike" or "extract_codeblock"
1010 encountered a closing bracket where none was expected.
1011 "Unmatched opening bracket(s): "%s""
1013 "extract_bracketed", "extract_quotelike" or "extract_codeblock" ran
1014 out of characters in the text before closing one or more levels of
1015 nested brackets.
1016 "Unmatched embedded quote (%s)"
1018 "extract_bracketed" attempted to match an embedded quoted
1019 substring, but failed to find a closing quote to match it.
1020 "Did not find closing delimiter to match '%s'"
1022 "extract_quotelike" was unable to find a closing delimiter to match
1023 the one that opened the quote-like operation.
1024 "Mismatched closing bracket: expected "%c" but found "%s""
1026 "extract_bracketed", "extract_quotelike" or "extract_codeblock"
1027 found a valid bracket delimiter, but it was the wrong species. This
1028 usually indicates a nesting error, but may indicate incorrect
1029 quoting or escaping.
1030 "No block delimiter found after quotelike "%s""
1032 "extract_quotelike" or "extract_codeblock" found one of the
1033 quotelike operators "q", "qq", "qw", "qx", "s", "tr" or "y" without
1034 a suitable block after it.
1035 "Did not find leading dereferencer"
1037 "extract_variable" was expecting one of '$', '@', or '%' at the
1038 start of a variable, but didn't find any of them.
1039 "Bad identifier after dereferencer"
1041 "extract_variable" found a '$', '@', or '%' indicating a variable,
1042 but that character was not followed by a legal Perl identifier.
1043 "Did not find expected opening bracket at %s"
1045 "extract_codeblock" failed to find any of the outermost opening
1046 brackets that were specified.
1047 "Improperly nested codeblock at %s"
1049 A nested code block was found that started with a delimiter that
1050 was specified as being only to be used as an outermost bracket.
1051 "Missing second block for quotelike "%s""
1053 "extract_codeblock" or "extract_quotelike" found one of the
1054 quotelike operators "s", "tr" or "y" followed by only one block.
1055 "No match found for opening bracket"
1057 "extract_codeblock" failed to find a closing bracket to match the
1058 outermost opening bracket.
1059 "Did not find opening tag: /%s/"
1061 "extract_tagged" did not find a suitable opening tag (after any
1062 specified prefix was removed).
1063 "Unable to construct closing tag to match: /%s/"
1065 "extract_tagged" matched the specified opening tag and tried to
1066 modify the matched text to produce a matching closing tag (because
1067 none was specified). It failed to generate the closing tag, almost
1068 certainly because the opening tag did not start with a bracket of
1069 some kind.
1070 "Found invalid nested tag: %s"
1072 "extract_tagged" found a nested tag that appeared in the "reject"
1073 list (and the failure mode was not "MAX" or "PARA").
1074 "Found unbalanced nested tag: %s"
1076 "extract_tagged" found a nested opening tag that was not matched by
1077 a corresponding nested closing tag (and the failure mode was not
1078 "MAX" or "PARA").
1079 "Did not find closing tag"
1081 "extract_tagged" reached the end of the text without finding a
1082 closing tag to match the original opening tag (and the failure mode
1083 was not "MAX" or "PARA").
1084
1086 The following symbols are, or can be, exported by this module:
1087 Default Exports
1089 None.
1090 Optional Exports
1092 "extract_delimited", "extract_bracketed", "extract_quotelike",
1093 "extract_codeblock", "extract_variable", "extract_tagged",
1094 "extract_multiple", "gen_delimited_pat", "gen_extract_tagged",
1095 "delimited_pat".
1096 Export Tags
1098 ":ALL"
1099 "extract_delimited", "extract_bracketed", "extract_quotelike",
1100 "extract_codeblock", "extract_variable", "extract_tagged",
1101 "extract_multiple", "gen_delimited_pat", "gen_extract_tagged",
1102 "delimited_pat".
1103
1105 See
1106 <https://rt.cpan.org/Dist/Display.html?Status=Active&Queue=Text-Balanced>.
1107
1109 Patches, bug reports, suggestions or any other feedback is welcome.
1110 Patches can be sent as GitHub pull requests at
1112 <https://github.com/steve-m-hay/Text-Balanced/pulls>.
1113 Bug reports and suggestions can be made on the CPAN Request Tracker at
1115 <https://rt.cpan.org/Public/Bug/Report.html?Queue=Text-Balanced>.
1116 Currently active requests on the CPAN Request Tracker can be viewed at
1118 <https://rt.cpan.org/Public/Dist/Display.html?Status=Active;Queue=Text-Balanced>.
1119 Please test this distribution. See CPAN Testers Reports at
1121 <https://www.cpantesters.org/> for details of how to get involved.
1122 Previous test results on CPAN Testers Reports can be viewed at
1124 <https://www.cpantesters.org/distro/T/Text-Balanced.html>.
1125 Please rate this distribution on CPAN Ratings at
1127 <https://cpanratings.perl.org/rate/?distribution=Text-Balanced>.
1128
1130 The latest version of this module is available from CPAN (see "CPAN" in
1131 perlmodlib for details) at
1132 <https://metacpan.org/release/Text-Balanced> or
1134 <https://www.cpan.org/authors/id/S/SH/SHAY/> or
1136 <https://www.cpan.org/modules/by-module/Text/>.
1138 The latest source code is available from GitHub at
1140 <https://github.com/steve-m-hay/Text-Balanced>.
1141
1143 See the INSTALL file.
1144
1146 Damian Conway <damian@conway.org <mailto:damian@conway.org>>.
1147 Steve Hay <shay@cpan.org <mailto:shay@cpan.org>> is now maintaining
1149 Text::Balanced as of version 2.03.
1150
1152 Copyright (C) 1997-2001 Damian Conway. All rights reserved.
1153 Copyright (C) 2009 Adam Kennedy.
1155 Copyright (C) 2015, 2020, 2022 Steve Hay and other contributors. All
1157 rights reserved.
1158
1160 This module is free software; you can redistribute it and/or modify it
1161 under the same terms as Perl itself, i.e. under the terms of either the
1162 GNU General Public License or the Artistic License, as specified in the
1163 LICENCE file.
1164
1166 Version 2.06
1167
1169 05 Jun 2022
1170
1172 See the Changes file.
1173perl v5.36.0 2022-07-22 Text::Balanced(3)