1pt::peg::to::container(n) Parser Tools pt::peg::to::container(n)
2______________________________________________________________________________
6
8 pt::peg::to::container - PEG Conversion. Write CONTAINER format
9
11 package require Tcl 8.5
12 package require pt::peg::to::container ?1?
14 package require pt::peg
16 package require text::write
18 package require char
20 pt::peg::to::container reset
22 pt::peg::to::container configure
24 pt::peg::to::container configure option
26 pt::peg::to::container configure option value...
28 pt::peg::to::container convert serial
30______________________________________________________________________________
32
34 Are you lost ? Do you have trouble understanding this document ? In
35 that case please read the overview provided by the Introduction to
36 Parser Tools. This document is the entrypoint to the whole system the
37 current package is a part of.
38 This package implements the converter from parsing expression grammars
40 to CONTAINER markup.
41 It resides in the Export section of the Core Layer of Parser Tools, and
43 can be used either directly with the other packages of this layer, or
44 indirectly through the export manager provided by pt::peg::export. The
45 latter is intented for use in untrusted environments and done through
46 the corresponding export plugin pt::peg::export::container sitting be‐
47 tween converter and export manager.
48 IMAGE: arch_core_eplugins
50
52 The API provided by this package satisfies the specification of the
53 Converter API found in the Parser Tools Export API specification.
54 pt::peg::to::container reset
56 This command resets the configuration of the package to its de‐
57 fault settings.
58 pt::peg::to::container configure
60 This command returns a dictionary containing the current config‐
61 uration of the package.
62 pt::peg::to::container configure option
64 This command returns the current value of the specified configu‐
65 ration option of the package. For the set of legal options,
66 please read the section Options.
67 pt::peg::to::container configure option value...
69 This command sets the given configuration options of the pack‐
70 age, to the specified values. For the set of legal options,
71 please read the section Options.
72 pt::peg::to::container convert serial
74 This command takes the canonical serialization of a parsing ex‐
75 pression grammar, as specified in section PEG serialization for‐
76 mat, and contained in serial, and generates CONTAINER markup en‐
77 coding the grammar, per the current package configuration. The
78 created string is then returned as the result of the command.
79
81 The converter to the CONTAINER format recognizes the following options
82 and changes its behaviour as they specify.
83 -file string
85 The value of this option is the name of the file or other entity
86 from which the grammar came, for which the command is run. The
87 default value is unknown.
88 -name string
90 The value of this option is the name of the grammar we are pro‐
91 cessing. The default value is a_pe_grammar.
92 -user string
94 The value of this option is the name of the user for which the
95 command is run. The default value is unknown.
96 -mode bulk|incremental
98 The value of this option controls which methods of pt::peg::con‐
99 tainer instances are used to specify the grammar, i.e. preload
100 it into the container. There are two legal values, as listed be‐
101 low. The default is bulk.
102 bulk In this mode the methods start, add, modes, and rules are
104 used to specify the grammar in a bulk manner, i.e. as a
105 set of nonterminal symbols, and two dictionaries mapping
106 from the symbols to their semantic modes and parsing ex‐
107 pressions.
108 This mode is the default.
110 incremental
112 In this mode the methods start, add, mode, and rule are
113 used to specify the grammar piecemal, with each nontermi‐
114 nal having its own block of defining commands.
115 -template string
117 The value of this option is a string into which to put the gen‐
118 erated code and the other configuration settings. The various
119 locations for user-data are expected to be specified with the
120 placeholders listed below. The default value is "@code@".
121 @user@ To be replaced with the value of the option -user.
123 @format@
125 To be replaced with the the constant CONTAINER.
126 @file@ To be replaced with the value of the option -file.
128 @name@ To be replaced with the value of the option -name.
130 @mode@ To be replaced with the value of the option -mode.
132 @code@ To be replaced with the generated code.
134
136 The container format is another form of describing parsing expression
137 grammars. While data in this format is executable it does not consti‐
138 tute a parser for the grammar. It always has to be used in conjunction
139 with the package pt::peg::interp, a grammar interpreter.
140 The format represents grammars by a snit::type, i.e. class, whose in‐
142 stances are API-compatible to the instances of the pt::peg::container
143 package, and which are preloaded with the grammar in question.
144 It has no direct formal specification beyond what was said above.
146 EXAMPLE
148 Assuming the following PEG for simple mathematical expressions
149 PEG calculator (Expression)
151 Digit <- '0'/'1'/'2'/'3'/'4'/'5'/'6'/'7'/'8'/'9' ;
152 Sign <- '-' / '+' ;
153 Number <- Sign? Digit+ ;
154 Expression <- Term (AddOp Term)* ;
155 MulOp <- '*' / '/' ;
156 Term <- Factor (MulOp Factor)* ;
157 AddOp <- '+'/'-' ;
158 Factor <- '(' Expression ')' / Number ;
159 END;
160 one possible CONTAINER serialization for it is
163 snit::type a_pe_grammar {
165 constructor {} {
166 install myg using pt::peg::container ${selfns}::G
167 $myg start {n Expression}
168 $myg add AddOp Digit Expression Factor MulOp Number Sign Term
169 $myg modes {
170 AddOp value
171 Digit value
172 Expression value
173 Factor value
174 MulOp value
175 Number value
176 Sign value
177 Term value
178 }
179 $myg rules {
180 AddOp {/ {t -} {t +}}
181 Digit {/ {t 0} {t 1} {t 2} {t 3} {t 4} {t 5} {t 6} {t 7} {t 8} {t 9}}
182 Expression {/ {x {t \50} {n Expression} {t \51}} {x {n Factor} {* {x {n MulOp} {n Factor}}}}}
183 Factor {x {n Term} {* {x {n AddOp} {n Term}}}}
184 MulOp {/ {t *} {t /}}
185 Number {x {? {n Sign}} {+ {n Digit}}}
186 Sign {/ {t -} {t +}}
187 Term {n Number}
188 }
189 return
190 }
191 component myg
193 delegate method * to myg
194 }
195
198 Here we specify the format used by the Parser Tools to serialize Pars‐
199 ing Expression Grammars as immutable values for transport, comparison,
200 etc.
201 We distinguish between regular and canonical serializations. While a
203 PEG may have more than one regular serialization only exactly one of
204 them will be canonical.
205 regular serialization
207 [1] The serialization of any PEG is a nested Tcl dictionary.
209 [2] This dictionary holds a single key, pt::grammar::peg, and
211 its value. This value holds the contents of the grammar.
212 [3] The contents of the grammar are a Tcl dictionary holding
214 the set of nonterminal symbols and the starting expres‐
215 sion. The relevant keys and their values are
216 rules The value is a Tcl dictionary whose keys are the
218 names of the nonterminal symbols known to the
219 grammar.
220 [1] Each nonterminal symbol may occur only
222 once.
223 [2] The empty string is not a legal nonterminal
225 symbol.
226 [3] The value for each symbol is a Tcl dictio‐
228 nary itself. The relevant keys and their
229 values in this dictionary are
230 is The value is the serialization of
232 the parsing expression describing
233 the symbols sentennial structure, as
234 specified in the section PE serial‐
235 ization format.
236 mode The value can be one of three values
238 specifying how a parser should han‐
239 dle the semantic value produced by
240 the symbol.
241 value The semantic value of the
243 nonterminal symbol is an ab‐
244 stract syntax tree consisting
245 of a single node node for the
246 nonterminal itself, which has
247 the ASTs of the symbol's
248 right hand side as its chil‐
249 dren.
250 leaf The semantic value of the
252 nonterminal symbol is an ab‐
253 stract syntax tree consisting
254 of a single node node for the
255 nonterminal, without any
256 children. Any ASTs generated
257 by the symbol's right hand
258 side are discarded.
259 void The nonterminal has no seman‐
261 tic value. Any ASTs generated
262 by the symbol's right hand
263 side are discarded (as well).
264 start The value is the serialization of the start pars‐
266 ing expression of the grammar, as specified in the
267 section PE serialization format.
268 [4] The terminal symbols of the grammar are specified implic‐
270 itly as the set of all terminal symbols used in the start
271 expression and on the RHS of the grammar rules.
272 canonical serialization
274 The canonical serialization of a grammar has the format as spec‐
275 ified in the previous item, and then additionally satisfies the
276 constraints below, which make it unique among all the possible
277 serializations of this grammar.
278 [1] The keys found in all the nested Tcl dictionaries are
280 sorted in ascending dictionary order, as generated by
281 Tcl's builtin command lsort -increasing -dict.
282 [2] The string representation of the value is the canonical
284 representation of a Tcl dictionary. I.e. it does not con‐
285 tain superfluous whitespace.
286 EXAMPLE
288 Assuming the following PEG for simple mathematical expressions
289 PEG calculator (Expression)
291 Digit <- '0'/'1'/'2'/'3'/'4'/'5'/'6'/'7'/'8'/'9' ;
292 Sign <- '-' / '+' ;
293 Number <- Sign? Digit+ ;
294 Expression <- Term (AddOp Term)* ;
295 MulOp <- '*' / '/' ;
296 Term <- Factor (MulOp Factor)* ;
297 AddOp <- '+'/'-' ;
298 Factor <- '(' Expression ')' / Number ;
299 END;
300 then its canonical serialization (except for whitespace) is
303 pt::grammar::peg {
305 rules {
306 AddOp {is {/ {t -} {t +}} mode value}
307 Digit {is {/ {t 0} {t 1} {t 2} {t 3} {t 4} {t 5} {t 6} {t 7} {t 8} {t 9}} mode value}
308 Expression {is {x {n Term} {* {x {n AddOp} {n Term}}}} mode value}
309 Factor {is {/ {x {t (} {n Expression} {t )}} {n Number}} mode value}
310 MulOp {is {/ {t *} {t /}} mode value}
311 Number {is {x {? {n Sign}} {+ {n Digit}}} mode value}
312 Sign {is {/ {t -} {t +}} mode value}
313 Term {is {x {n Factor} {* {x {n MulOp} {n Factor}}}} mode value}
314 }
315 start {n Expression}
316 }
317
320 Here we specify the format used by the Parser Tools to serialize Pars‐
321 ing Expressions as immutable values for transport, comparison, etc.
322 We distinguish between regular and canonical serializations. While a
324 parsing expression may have more than one regular serialization only
325 exactly one of them will be canonical.
326 Regular serialization
328 Atomic Parsing Expressions
330 [1] The string epsilon is an atomic parsing expres‐
332 sion. It matches the empty string.
333 [2] The string dot is an atomic parsing expression. It
335 matches any character.
336 [3] The string alnum is an atomic parsing expression.
338 It matches any Unicode alphabet or digit charac‐
339 ter. This is a custom extension of PEs based on
340 Tcl's builtin command string is.
341 [4] The string alpha is an atomic parsing expression.
343 It matches any Unicode alphabet character. This is
344 a custom extension of PEs based on Tcl's builtin
345 command string is.
346 [5] The string ascii is an atomic parsing expression.
348 It matches any Unicode character below U0080. This
349 is a custom extension of PEs based on Tcl's
350 builtin command string is.
351 [6] The string control is an atomic parsing expres‐
353 sion. It matches any Unicode control character.
354 This is a custom extension of PEs based on Tcl's
355 builtin command string is.
356 [7] The string digit is an atomic parsing expression.
358 It matches any Unicode digit character. Note that
359 this includes characters outside of the [0..9]
360 range. This is a custom extension of PEs based on
361 Tcl's builtin command string is.
362 [8] The string graph is an atomic parsing expression.
364 It matches any Unicode printing character, except
365 for space. This is a custom extension of PEs based
366 on Tcl's builtin command string is.
367 [9] The string lower is an atomic parsing expression.
369 It matches any Unicode lower-case alphabet charac‐
370 ter. This is a custom extension of PEs based on
371 Tcl's builtin command string is.
372 [10] The string print is an atomic parsing expression.
374 It matches any Unicode printing character, includ‐
375 ing space. This is a custom extension of PEs based
376 on Tcl's builtin command string is.
377 [11] The string punct is an atomic parsing expression.
379 It matches any Unicode punctuation character. This
380 is a custom extension of PEs based on Tcl's
381 builtin command string is.
382 [12] The string space is an atomic parsing expression.
384 It matches any Unicode space character. This is a
385 custom extension of PEs based on Tcl's builtin
386 command string is.
387 [13] The string upper is an atomic parsing expression.
389 It matches any Unicode upper-case alphabet charac‐
390 ter. This is a custom extension of PEs based on
391 Tcl's builtin command string is.
392 [14] The string wordchar is an atomic parsing expres‐
394 sion. It matches any Unicode word character. This
395 is any alphanumeric character (see alnum), and any
396 connector punctuation characters (e.g. under‐
397 score). This is a custom extension of PEs based on
398 Tcl's builtin command string is.
399 [15] The string xdigit is an atomic parsing expression.
401 It matches any hexadecimal digit character. This
402 is a custom extension of PEs based on Tcl's
403 builtin command string is.
404 [16] The string ddigit is an atomic parsing expression.
406 It matches any decimal digit character. This is a
407 custom extension of PEs based on Tcl's builtin
408 command regexp.
409 [17] The expression [list t x] is an atomic parsing ex‐
411 pression. It matches the terminal string x.
412 [18] The expression [list n A] is an atomic parsing ex‐
414 pression. It matches the nonterminal A.
415 Combined Parsing Expressions
417 [1] For parsing expressions e1, e2, ... the result of
419 [list / e1 e2 ... ] is a parsing expression as
420 well. This is the ordered choice, aka prioritized
421 choice.
422 [2] For parsing expressions e1, e2, ... the result of
424 [list x e1 e2 ... ] is a parsing expression as
425 well. This is the sequence.
426 [3] For a parsing expression e the result of [list *
428 e] is a parsing expression as well. This is the
429 kleene closure, describing zero or more repeti‐
430 tions.
431 [4] For a parsing expression e the result of [list +
433 e] is a parsing expression as well. This is the
434 positive kleene closure, describing one or more
435 repetitions.
436 [5] For a parsing expression e the result of [list &
438 e] is a parsing expression as well. This is the
439 and lookahead predicate.
440 [6] For a parsing expression e the result of [list !
442 e] is a parsing expression as well. This is the
443 not lookahead predicate.
444 [7] For a parsing expression e the result of [list ?
446 e] is a parsing expression as well. This is the
447 optional input.
448 Canonical serialization
450 The canonical serialization of a parsing expression has the for‐
451 mat as specified in the previous item, and then additionally
452 satisfies the constraints below, which make it unique among all
453 the possible serializations of this parsing expression.
454 [1] The string representation of the value is the canonical
456 representation of a pure Tcl list. I.e. it does not con‐
457 tain superfluous whitespace.
458 [2] Terminals are not encoded as ranges (where start and end
460 of the range are identical).
461 EXAMPLE
463 Assuming the parsing expression shown on the right-hand side of the
464 rule
465 Expression <- Term (AddOp Term)*
467 then its canonical serialization (except for whitespace) is
470 {x {n Term} {* {x {n AddOp} {n Term}}}}
472
475 This document, and the package it describes, will undoubtedly contain
476 bugs and other problems. Please report such in the category pt of the
477 Tcllib Trackers [http://core.tcl.tk/tcllib/reportlist]. Please also
478 report any ideas for enhancements you may have for either package
479 and/or documentation.
480 When proposing code changes, please provide unified diffs, i.e the out‐
482 put of diff -u.
483 Note further that attachments are strongly preferred over inlined
485 patches. Attachments can be made by going to the Edit form of the
486 ticket immediately after its creation, and then using the left-most
487 button in the secondary navigation bar.
488
490 CONTAINER, EBNF, LL(k), PEG, TDPL, context-free languages, conversion,
491 expression, format conversion, grammar, matching, parser, parsing ex‐
492 pression, parsing expression grammar, push down automaton, recursive
493 descent, serialization, state, top-down parsing languages, transducer
494
496 Parsing and Grammars
497
499 Copyright (c) 2009 Andreas Kupries <andreas_kupries@users.sourceforge.net>
500tcllib 1 pt::peg::to::container(n)