1Hardware::Vhdl::Lexer(3U)ser Contributed Perl DocumentatiHoanrdware::Vhdl::Lexer(3)
2
6 Hardware::Vhdl::Lexer - Split VHDL code into lexical tokens
7
9 use Hardware::Vhdl::Lexer;
10 # Open the file to get the VHDL code from
12 my $fh;
13 open $fh, '<', 'device_behav.vhd' || die $!
14 # Create the Lexer object
16 my $lexer = Hardware::Vhdl::Lexer->new({ linesource => $fh });
17 # Dump all the tokens
19 my ($token, $type);
20 while( (($token, $type) = $lexer->get_next_token) && defined $token) {
21 print "# type = '$type' token='$token'\n";
22 }
23
25 "Hardware::Vhdl::Lexer" splits VHDL code into lexical tokens. To use
26 it, you need to first create a lexer object, passing in something which
27 will supply chunks of VHDL code to the lexer. Repeated calls to the
28 "get_next_token" method of the lexer will then return VHDL tokens (in
29 scalar context) or a token type code and the token (in list context).
30 "get_next_token" returns undef when there are no more tokens to be
31 read.
32 NB: in this documentation I refer to "lines" of VHDL code and "line"
34 sources etc., but in fact the chunks of code don't have to be broken up
35 at line-ends - they can be broken anywhere that isn't in the middle of
36 a token. New-line characters just happen to be a simple and safe way
37 to split up a file. You don't even have to split up the VHDL at all,
38 you can pass in the whole thing as the first and only "line".
39
41 new({ linesource => <source> [, nhistory => N] })
42 Note that from version 1.0 of this module the arguments must now be
44 given as a hash reference rather than a hash, so the curly brackets
45 above are required.
46 The linesource argument is required: it defines where the VHDL source
48 code will be taken from (see below).
49 The optional nhistory argument sets how many "code" tokens (see the
51 "get_next_token" method) will be remembered for access by the "history"
52 method.
53 new({ linesource => $filehandle_reference [, nhistory => N] })
55 To read from a file, pass in the filehandle reference like this:
56 use Hardware::Vhdl::Lexer;
58 my $fh;
59 open $fh, '<', $filename || die $!;
60 my $lexer = Hardware::Vhdl::Lexer->new({ linesource => $fh });
61 new({ linesource => \@array_of_lines [, nhistory => N] })
63 new({ linesource => \$scalar_containing_vhdl [, nhistory => N] })
64 To read VHDL source that is already in program memory, the
65 linesource argument can be a reference to either an array of lines
66 or a single string which can have embedded newlines.
67 new({ linesource => $object_with_get_next_line_method [, nhistory => N]
69 })
70 The linesource argument can be an object with a "get_next_line"
71 method. This method must return undef when there are no more lines
72 to read.
73 new({ linesource => \&subroutine_that_returns_lines [, nhistory => N]
75 })
76 If none of the above input methods suits your needs, you can give a
77 subroutine reference and wrap whatever code you need to get the
78 VHDL source. When called, this subroutine must return each line of
79 source code in turn, and then return undef when there are no more
80 lines.
81
83 get_linesource()
84 Returns the linesource argument passed into the constructor.
85 Before version 1.0 of this module, this method was called
86 "linesource()".
87 "get_next_token()"
89 In scalar context, returns the next VHDL token.
90 In list context, returns a token type code and the token
92 Nothing is removed from the source code: if you concatenate all the
94 tokens returned by "get_next_token()", you will get the same result
95 as if you concatenate all the strings returned by the linesource
96 object.
97 The token type codes are 1 or 2-character strings. When the codes
99 are 2 characters, the first character gives the general class of
100 the token and the second indicates its type more specifically. The
101 first character will be 'w' for whitespace, 'r' for comments
102 (remarks) or 'c' for code. It should be possible to remove all
103 comment tokens, and change whitespace tokens for different
104 whitespace, and always end up with functionally equivalent code.
105 The token type codes are:
107 wn Whitespace:Newline. This could be any of \012, \015, \015\012
109 or \012\015.
110 ws Whitespace:Spaces. A group of whitespace characters which
112 match the /s regexp pattern but which do not include any
113 carriage-return or linefeed characters.
114 r Remark. The token will start with two dashes and include the
116 remainder of the source code line, not including any newline
117 characters. The next token will either be a newline or undef.
118 cs Code:String literal. The lexer accepts multi-line strings,
120 even though the VHDL specification does not allow them.
121 cc Code:Character literal.
123 cb Code:Bit_vector literal. For example, "B"001_1010"" or
125 "O"7720"" or "H"A7_DEAD"".
126 cn Code:Numeric literal. This could be a specified-base literal
128 like "8#7720#" or a simple integer or floating-point value.
129 ci Code:Identifier or keyword. For example, "package" or
131 "my_signal_23" or "/extended identifier$%!/"..
132 cp Code:Punctuation. A 1 or 2-character group of punctuation
134 symbols that is part of VHDL syntax. For example, '<=' is
135 returned as a single 'cp' token, as is '&', but '#' would be
136 returned as an unexpected character (see below).
137 cu Unexpected character. Any character in the source that does
139 not match any of the above definitions, and cannot be part of
140 valid VHDL code. Note that prior to version 1.0 of this
141 module, these would be returned with the 'cp' token type code.
142 history(N)
144 Returns previous code tokens. N must not be larger than the
145 nhistory argument passed to the constructor. history(0) will
146 return the text of the last token returned by "get_next_token"
147 whose type started with a 'c', history(1) will return the code
148 token before that, and so on.
149
151 Michael Attenborough, "<michael.attenborough at physics.org>"
152
154 This module requires the following modules to be available:
155 Carp: any version Class::Std: any version Readonly: version 1.03 or
157 later
158
160 "Argument to Hardware::Vhdl::Lexer->new() must be hash reference"
161 Have you remembered to put curly brackets around the argument list?
162 Pre-1.0 versions of this module used to take the arguments to new()
163 as a direct hash, but version 1.0 onwards need a hash reference.
164 This means that the curly brackets need to be added when migrating
165 from pre-1.0 to 1.0 or later.
166 # Old style (argument list is hash) - doesn't work any more
168 my $lexer = Hardware::Vhdl::Lexer->new( linesource => $fh );
169 # New style (argument is a hash ref) - do it this way now
171 my $lexer = Hardware::Vhdl::Lexer->new({ linesource => $fh });
172 "Hardware::Vhdl::Lexer constructor requires a linesource to be
174 specified"
175 The 'linesource' argument to Hardware::Vhdl::Lexer->new() is
176 required, and it is a fatal error not to provide one.
177 "Hardware::Vhdl::Lexer->new 'linesource' parameter is not of a valid
179 type (it is not a reference)"
180 The linesource parameter needs to be a reference to something. If
181 your VHDL code to be passed is in a scalar string, you need to pass
182 in a reference to the string, not the string itself.
183 "Hardware::Vhdl::Lexer->new 'linesource' parameter is not of a valid
185 type (type is '<type>')"
186 The linesource parameter that you have passed to new() does not
187 appear to be a reference to a scalar, a list, a filehandle, a
188 subroutine or an object with a get_next_line method. You have
189 passed a reference to something (otherwise you would see the
190 previous message) and the error message will tell you what it
191 appears to be a reference to.
192 "Internal error (token failed to match anything)"
194 This is a "this should never happen" type of error, and is a sign
195 that I have included a bug. If you ever see this error, I would
196 appreciate a bug report describing how to reproduce the error.
197
199 Please report any bugs or feature requests to "bug-hardware-vhdl-lexer
200 at rt.cpan.org", or through the web interface at
201 <http://rt.cpan.org/NoAuth/ReportBug.html?Queue=Hardware-Vhdl-Lexer>.
202 I will be notified, and then you'll automatically be notified of
203 progress on your bug as I make changes.
204
206 You can find documentation for this module with the perldoc command.
207 perldoc Hardware::Vhdl::Lexer
209 You can also look for information at:
211 • AnnoCPAN: Annotated CPAN documentation
213 <http://annocpan.org/dist/Hardware-Vhdl-Lexer>
215 • CPAN Ratings
217 <http://cpanratings.perl.org/d/Hardware-Vhdl-Lexer>
219 • RT: CPAN's request tracker
221 <http://rt.cpan.org/NoAuth/Bugs.html?Dist=Hardware-Vhdl-Lexer>
223 • Search CPAN
225 <http://search.cpan.org/dist/Hardware-Vhdl-Lexer>
227
229 Copyright 2006 Michael Attenborough, all rights reserved.
230 This program is free software; you can redistribute it and/or modify it
232 under the same terms as Perl itself.
233perl v5.36.0 2022-07-22 Hardware::Vhdl::Lexer(3)