1grammar::me::cpu(n) Grammar operations and usage grammar::me::cpu(n)
2______________________________________________________________________________
6
8 grammar::me::cpu - Virtual machine implementation II for parsing token
9 streams
10
12 package require Tcl 8.4
13 package require grammar::me::cpu ?0.2?
15 ::grammar::me::cpu meName matchcode
17 meName option ?arg arg ...?
19 meName lc location
21 meName tok ?from ?to??
23 meName pc state
25 meName iseof state
27 meName at state
29 meName cc state
31 meName sv
33 meName ok
35 meName error
37 meName lstk state
39 meName astk state
41 meName mstk state
43 meName estk state
45 meName rstk state
47 meName nc state
49 meName ast
51 meName halted
53 meName code
55 meName eof
57 meName put tok lex line col
59 meName putstring string lvar cvar
61 meName run ?n?
63 meName pull nextcmd
65 meName reset
67 meName destroy
69______________________________________________________________________________
71
73 This package provides an implementation of the ME virtual machine.
74 Please go and read the document grammar::me_intro first if you do not
75 know what a ME virtual machine is.
76 This implementation provides an object-based API and the machines are
78 not truly tied to Tcl. A C implementation of the same API is quite pos‐
79 sible.
80 Internally the package actually uses the value-based machine manipula‐
82 tion commands as provided by the package grammar::me::cpu::core to per‐
83 form its duties.
84
86 CLASS API
87 The package directly provides only a single command for the construc‐
88 tion of ME virtual machines.
89 ::grammar::me::cpu meName matchcode
91 The command creates a new ME machine object with an associated
92 global Tcl command whose name is meName. This command may be
93 used to invoke various operations on the machine. It has the
94 following general form:
95 meName option ?arg arg ...?
97 Option and the args determine the exact behavior of the
98 command.
99 The argument matchcode contains the match instructions the machine has
101 to execute while parsing the input stream. Please read section MATCH
102 CODE REPRESENTATION of the documentation for the package gram‐
103 mar::me::cpu::core for the specification of the structure of this
104 value.
105 The tokmap argument taken by the implementation provided by the package
107 grammar::me::tcl is here hidden inside of the match instructions and
108 therefore not needed.
109 OBJECT API
111 All ME virtual machine objects created by the class command specified
112 in section CLASS API support the methods listed below.
113 The machines provided by this package provide methods for operation in
115 both push- and pull-styles. Push-style means that tokens are pushed
116 into the machine state when they arrive, triggering further execution
117 until they are consumed. In other words, this allows the machine to be
118 suspended and resumed at will and an arbitrary number of times, the
119 quasi-parallel operation of several machines, and the operation as part
120 of the event loop.
121 meName lc location
123 This method converts the location of a token given as offset in
124 the input stream into the associated line number and column in‐
125 dex. The result of the command is a 2-element list containing
126 the two values, in the order mentioned in the previous sentence.
127 This allows higher levels to convert the location information
128 found in the error status and the generated AST into more human
129 readable data.
130 Note that the command is not able to convert locations which
132 have not been reached by the machine yet. In other words, if the
133 machine has read 7 tokens the command is able to convert the
134 offsets 0 to 6, but nothing beyond that. This also shows that it
135 is not possible to convert offsets which refer to locations be‐
136 fore the beginning of the stream.
137 meName tok ?from ?to??
139 This method returns a Tcl list containing the part of the input
140 stream between the locations from and to (both inclusive). If to
141 is not specified it will default to the value of from. If from
142 is not specified either the whole input stream is returned.
143 Each element of the returned list is a list of four elements,
145 the token, its associated lexeme, line number, and column index,
146 in this order. This command places the same restrictions on its
147 location arguments as the method lc.
148 meName pc state
150 This method takes the state value of a ME virtual machine and
151 returns the current value of the stored program counter.
152 meName iseof state
154 This method takes the state value of a ME virtual machine and
155 returns the current value of the stored eof flag.
156 meName at state
158 This method takes the state value of a ME virtual machine and
159 returns the current location in the input stream.
160 meName cc state
162 This method takes the state value of a ME virtual machine and
163 returns the current token.
164 meName sv
166 This command returns the current semantic value SV stored in the
167 machine. This is an abstract syntax tree as specified in the
168 document grammar::me_ast, section AST VALUES.
169 meName ok
171 This method returns the current match status OK.
172 meName error
174 This method returns the current error status ER.
175 meName lstk state
177 This method takes the state value of a ME virtual machine and
178 returns the location stack.
179 meName astk state
181 This method takes the state value of a ME virtual machine and
182 returns the AST stack.
183 meName mstk state
185 This method takes the state value of a ME virtual machine and
186 returns the AST marker stack.
187 meName estk state
189 This method takes the state value of a ME virtual machine and
190 returns the error stack.
191 meName rstk state
193 This method takes the state value of a ME virtual machine and
194 returns the subroutine return stack.
195 meName nc state
197 This method takes the state value of a ME virtual machine and
198 returns the nonterminal match cache as a dictionary.
199 meName ast
201 This method returns the current top entry of the AST stack AS.
202 This is an abstract syntax tree as specified in the document
203 grammar::me_ast, section AST VALUES.
204 meName halted
206 This method returns a boolean value telling the caller whether
207 the engine has halted execution or not. Halt means that no fur‐
208 ther matching is possible, and the information retrieved via the
209 other method is final. Attempts to run the engine will be ig‐
210 nored, until a reset is made.
211 meName code
213 This method returns the code information used to construct the
214 object. In other words, the match program executed by the ma‐
215 chine.
216 meName eof
218 This method adds an end of file marker to the end of the input
219 stream. This signals the machine that the current contents of
220 the input queue are the final parts of the input and nothing
221 will come after. Attempts to put more characters into the queue
222 will fail.
223 meName put tok lex line col
225 This method adds the token tok to the end of the input stream,
226 with associated lexeme data lex and line/column information.
227 meName putstring string lvar cvar
229 This method adds each individual character in the string as a
230 token to the end of the input stream, from first to last. The
231 lexemes will be empty and the line/col information is computed
232 based on the characters encountered and the data in the vari‐
233 ables lvar and cvar.
234 meName run ?n?
236 This methods causes the engine to execute match instructions un‐
237 til either
238 • n instructions have been executed, or
240 • a halt instruction was executed, or
242 • the input queue is empty and the code is asking for more
244 tokens to process.
245 If no limit n was set only the last two conditions are checked for.
247 meName pull nextcmd
249 This method implements pull-style operation of the machine. It
250 causes it to execute match instructions until either a halt in‐
251 struction is reached, or the command prefix nextcmd ceases to
252 deliver more tokens.
253 The command prefix nextcmd represents the input stream of char‐
255 acters and is invoked by the machine whenever the a new charac‐
256 ter from the stream is required. The instruction for handling
257 this is ict_advance. The callback has to return either the
258 empty list, or a list of 4 elements containing the token, its
259 lexeme attribute, and its location as line number and column in‐
260 dex, in this order. The empty list is the signal that the end
261 of the input stream has been reached. The lexeme attribute is
262 stored in the terminal cache, but otherwise not used by the ma‐
263 chine.
264 The end of the input stream for this method does not imply that
266 method eof is called for the machine as a whole. By avoiding
267 this and still asking for an explicit call of the method it is
268 possible to mix push- and pull-style operation during the life‐
269 time of the machine.
270 meName reset
272 This method resets the machine to its initial state, discarding
273 any state it may have.
274 meName destroy
276 This method deletes the object and releases all resurces it
277 claimed.
278
280 This document, and the package it describes, will undoubtedly contain
281 bugs and other problems. Please report such in the category grammar_me
282 of the Tcllib Trackers [http://core.tcl.tk/tcllib/reportlist]. Please
283 also report any ideas for enhancements you may have for either package
284 and/or documentation.
285 When proposing code changes, please provide unified diffs, i.e the out‐
287 put of diff -u.
288 Note further that attachments are strongly preferred over inlined
290 patches. Attachments can be made by going to the Edit form of the
291 ticket immediately after its creation, and then using the left-most
292 button in the secondary navigation bar.
293
295 grammar, parsing, virtual machine
296
298 Grammars and finite automata
299
301 Copyright (c) 2005-2006 Andreas Kupries <andreas_kupries@users.sourceforge.net>
302tcllib 0.2 grammar::me::cpu(n)