1Math::Expression::EvaluUasteorr:C:oLnetxreirb(u3t)ed PerMlatDho:c:uEmxepnrteastsiioonn::Evaluator::Lexer(3)
2
3
4

NAME

6       Math::Expression::Evaluator::Lexer - Simple Lexer
7

SYNOPSIS

9           use Math::Expression::Evaluator::Lexer qw(lex);
10           # suppose you want to parse simple math expressions
11           my @input_tokens = (
12               ['Int',             qr/[+-]?\d+/ ],
13               ['Op',              qr{[+-/*]}   ],
14               ['Brace_Open',      qr/\(/       ],
15               ['Brace_Close',     qr/\)/       ],
16               ['Whitespace',      qr/\s+/, sub { return; }],
17                );
18           my $text = "-12 * (3+4)";
19           my $out_tokens = lex($text, \@input_tokens);
20           for (@$out_tokens){
21               my ($name, $text, $pos) = @$_;
22               print "Found Token $name: $text (string pos: $pos)\n";
23           }
24

DESCRIPTION

26       Math::Expression::Evaluator::Lexer is a simple lexer that breaks up a
27       text into tokens, depending on the input tokens you provide
28

METHODS

30       lex
31         The only exported routine is lex, which expects input text as its
32         first argument and a array ref to list of input tokens.
33
34         Each input token consists of a token name (which you can choose
35         freely), a regex which matches the desired token, and optionally a
36         reference to a functions that takes the matched token text as its
37         argument. The token text is replaced by the return value of that
38         function. If the function returns undef, that token will not be
39         included in the list of output tokens. The regex should either fail
40         or match at least one character; zero-width matches utterly confuse
41         the lexer, and are disallowed.
42
43         lex() returns an array ref to a list of output tokens, each output
44         token is a reference to a list which contains the token name, the
45         matched text, the string position (in characters, counted from the
46         start of the input string, zero based) and the line number.
47
48         Note that "lex()" puts parentheses around the entire regex, so if you
49         want to use backreferences, the numbering of the capturing group is
50         changed.
51

53       Copyright (C) 2007 by Moritz Lenz, <http://moritz.faui2k3.org>,
54       moritz@faui2k3.org.
55
56       This Program and its Documentation is free software. You may distribute
57       it under the same terms as perl itself.
58
59       However all code examples are to be public domain, so you can use it in
60       any way you want to.
61
62
63
64perl v5.32.0                      2020-07-M2a8th::Expression::Evaluator::Lexer(3)
Impressum