1pt::pgen(n) Parser Tools pt::pgen(n)
2______________________________________________________________________________
6
8 pt::pgen - Parser Generator
9
11 package require Tcl 8.5
12 package require pt::pgen ?1.1?
14 ::pt::pgen inputformat text resultformat ?options...?
16______________________________________________________________________________
18
20 Are you lost ? Do you have trouble understanding this document ? In
21 that case please read the overview provided by the Introduction to
22 Parser Tools. This document is the entrypoint to the whole system the
23 current package is a part of.
24 This package provides a command implementing a parser generator taking
26 parsing expression grammars as input.
27 It is the implementation of method generate of pt, the Parser Tools
29 Application.
30 As such the intended audience of this document are people wishing to
32 modify and/or extend this part of pt's functionality. Users of pt on
33 the other hand are hereby refered to the applications' manpage, i.e.
34 Parser Tools Application.
35 It resides in the User Package Layer of Parser Tools.
37 IMAGE: arch_user_pkg
39
41 ::pt::pgen inputformat text resultformat ?options...?
42 This command takes the parsing expression grammar in text (in
43 the format specified by inputformat), and returns the same gram‐
44 mar in the format resultformat as the result of the command.
45 The two known input formats are peg and json. Introductions to
47 them, including their formal specifications, can be found in the
48 PEG Language Tutorial and The JSON Grammar Exchange Format. The
49 packages used to parse these formats are
50 peg pt::peg::from::peg
52 json pt::peg::from::json
54 On the output side the known formats, and the packages used to generate
56 them are
57 c pt::peg::to::cparam
59 container
61 pt::peg::to::container
62 critcl pt::peg::to::cparam + pt::cparam::configuration::critcl
64 json pt::peg::to::json
66 oo pt::peg::to::tclparam + pt::tclparam::configura‐
68 tion::tcloo
69 peg pt::peg::to::peg
71 snit pt::peg::to::tclparam + pt::tclparam::configuration::snit
73 The options supported by each of these formats are documented
75 with their respective packages.
76
78 In this section we are working a complete example, starting with a PEG
79 grammar and ending with running the parser generated from it over some
80 input, following the outline shown in the figure below:
81 IMAGE: flow
83 Our grammar, assumed to the stored in the file "calculator.peg" is
85 PEG calculator (Expression)
88 Digit <- '0'/'1'/'2'/'3'/'4'/'5'/'6'/'7'/'8'/'9' ;
89 Sign <- '-' / '+' ;
90 Number <- Sign? Digit+ ;
91 Expression <- Term (AddOp Term)* ;
92 MulOp <- '*' / '/' ;
93 Term <- Factor (MulOp Factor)* ;
94 AddOp <- '+'/'-' ;
95 Factor <- '(' Expression ')' / Number ;
96 END;
97 From this we create a snit-based parser using the script "gen"
99 package require Tcl 8.5
102 package require fileutil
103 package require pt::pgen
104 lassign $argv name
106 set grammar [fileutil::cat $name.peg]
107 set pclass [pt::pgen peg $gr snit -class $name -file $name.peg -name $name]
108 fileutil::writeFile $name.tcl $pclass
109 exit 0
110 calling it like
112 tclsh8.5 gen calculator
114 which leaves us with the parser package and class written to the file
115 "calculator.tcl". Assuming that this package is then properly
116 installed in a place where Tcl can find it we can now use this class
117 via a script like
118 package require calculator
121 lassign $argv input
123 set channel [open $input r]
124 set parser [calculator]
126 set ast [$parser parse $channel]
127 $parser destroy
128 close $channel
129 ... now process the returned abstract syntax tree ...
131 where the abstract syntax tree stored in the variable will look like
133 set ast {Expression 0 4
135 {Factor 0 4
136 {Term 0 2
137 {Number 0 2
138 {Digit 0 0}
139 {Digit 1 1}
140 {Digit 2 2}
141 }
142 }
143 {AddOp 3 3}
144 {Term 4 4
145 {Number 4 4
146 {Digit 4 4}
147 }
148 }
149 }
150 }
151 assuming that the input file and channel contained the text
154 120+5
156 A more graphical representation of the tree would be
157 .nf +- Digit 0 0 | 1 | | +- Term 0 2 --- Number 0 2 -+-
159 Digit 1 1 | 2 | | | |
160 +- Digit 2 2 | 0 | | Expression
161 0 4 --- Factor 0 4 -+----------------------------- AddOp 3 3 | + |
162 | +- Term 4 4 --- Number 4 4 --- Digit 4 4 | 5 .fi
163 Regardless, at this point it is the user's responsibility to work with
165 the tree to reach whatever goal she desires. I.e. analyze it, transform
166 it, etc. The package pt::ast should be of help here, providing commands
167 to walk such ASTs structures in various ways.
168 One important thing to note is that the parsers used here return a data
170 structure representing the structure of the input per the grammar
171 underlying the parser. There are no callbacks during the parsing
172 process, i.e. no parsing actions, as most other parsers will have.
173 Going back to the last snippet of code, the execution of the parser for
175 some input, note how the parser instance follows the specified Parser
176 API.
177
179 This document, and the package it describes, will undoubtedly contain
180 bugs and other problems. Please report such in the category pt of the
181 Tcllib Trackers [http://core.tcl.tk/tcllib/reportlist]. Please also
182 report any ideas for enhancements you may have for either package
183 and/or documentation.
184 When proposing code changes, please provide unified diffs, i.e the out‐
186 put of diff -u.
187 Note further that attachments are strongly preferred over inlined
189 patches. Attachments can be made by going to the Edit form of the
190 ticket immediately after its creation, and then using the left-most
191 button in the secondary navigation bar.
192
194 EBNF, LL(k), PEG, TDPL, context-free languages, expression, grammar,
195 matching, parser, parsing expression, parsing expression grammar, push
196 down automaton, recursive descent, state, top-down parsing languages,
197 transducer
198
200 Parsing and Grammars
201
203 Copyright (c) 2009 Andreas Kupries <andreas_kupries@users.sourceforge.net>
204tcllib 1.1 pt::pgen(n)