1doctools::idx::parse(n) Documentation tools doctools::idx::parse(n)
2______________________________________________________________________________
6
8 doctools::idx::parse - Parsing text in docidx format
9
11 package require doctools::idx::parse ?0.1?
12 package require Tcl 8.4
14 package require doctools::idx::structure
16 package require doctools::msgcat
18 package require doctools::tcl::parse
20 package require fileutil
22 package require logger
24 package require snit
26 package require struct::list
28 package require struct::stack
30 ::doctools::idx::parse text text
32 ::doctools::idx::parse file path
34 ::doctools::idx::parse includes
36 ::doctools::idx::parse include add path
38 ::doctools::idx::parse include remove path
40 ::doctools::idx::parse include clear
42 ::doctools::idx::parse vars
44 ::doctools::idx::parse var set name value
46 ::doctools::idx::parse var unset name
48 ::doctools::idx::parse var clear ?pattern?
50______________________________________________________________________________
52
54 This package provides commands to parse text written in the docidx
55 markup language and convert it into the canonical serialization of the
56 keyword index encoded in the text. See the section Keyword index seri‐
57 alization format for specification of their format.
58 This is an internal package of doctools, for use by the higher level
60 packages handling docidx documents.
61
63 ::doctools::idx::parse text text
64 The command takes the string contained in text and parses it
65 under the assumption that it contains a document written using
66 the docidx markup language. An error is thrown if this assump‐
67 tion is found to be false. The format of these errors is
68 described in section Parse errors.
69 When successful the command returns the canonical serialization
71 of the keyword index which was encoded in the text. See the
72 section Keyword index serialization format for specification of
73 that format.
74 ::doctools::idx::parse file path
76 The same as text, except that the text to parse is read from the
77 file specified by path.
78 ::doctools::idx::parse includes
80 This method returns the current list of search paths used when
81 looking for include files.
82 ::doctools::idx::parse include add path
84 This method adds the path to the list of paths searched when
85 looking for an include file. The call is ignored if the path is
86 already in the list of paths. The method returns the empty
87 string as its result.
88 ::doctools::idx::parse include remove path
90 This method removes the path from the list of paths searched
91 when looking for an include file. The call is ignored if the
92 path is not contained in the list of paths. The method returns
93 the empty string as its result.
94 ::doctools::idx::parse include clear
96 This method clears the list of search paths for include files.
97 ::doctools::idx::parse vars
99 This method returns a dictionary containing the current set of
100 predefined variables known to the vset markup command during
101 processing.
102 ::doctools::idx::parse var set name value
104 This method adds the variable name to the set of predefined
105 variables known to the vset markup command during processing,
106 and gives it the specified value. The method returns the empty
107 string as its result.
108 ::doctools::idx::parse var unset name
110 This method removes the variable name from the set of predefined
111 variables known to the vset markup command during processing.
112 The method returns the empty string as its result.
113 ::doctools::idx::parse var clear ?pattern?
115 This method removes all variables matching the pattern from the
116 set of predefined variables known to the vset markup command
117 during processing. The method returns the empty string as its
118 result.
119 The pattern matching is done with string match, and the default
121 pattern used when none is specified, is *.
122
124 The format of the parse error messages thrown when encountering viola‐
125 tions of the docidx markup syntax is human readable and not intended
126 for processing by machines. As such it is not documented.
127 However, the errorCode attached to the message is machine-readable and
129 has the following format:
130 [1] The error code will be a list, each element describing a single
132 error found in the input. The list has at least one element,
133 possibly more.
134 [2] Each error element will be a list containing six strings
136 describing an error in detail. The strings will be
137 [1] The path of the file the error occurred in. This may be
139 empty.
140 [2] The range of the token the error was found at. This range
142 is a two-element list containing the offset of the first
143 and last character in the range, counted from the begin‐
144 ning of the input (file). Offsets are counted from zero.
145 [3] The line the first character after the error is on.
147 Lines are counted from one.
148 [4] The column the first character after the error is at.
150 Columns are counted from zero.
151 [5] The message code of the error. This value can be used as
153 argument to msgcat::mc to obtain a localized error mes‐
154 sage, assuming that the application had a suitable call
155 of doctools::msgcat::init to initialize the necessary
156 message catalogs (See package doctools::msgcat).
157 [6] A list of details for the error, like the markup command
159 involved. In the case of message code docidx/include/syn‐
160 tax this value is the set of errors found in the included
161 file, using the format described here.
162
164 The docidx format for keyword indices, also called the docidx markup
165 language, is too large to be covered in single section. The interested
166 reader should start with the document
167 [1] docidx language introduction
169 and then proceed from there to the formal specifications, i.e. the doc‐
171 uments
172 [1] docidx language syntax and
174 [2] docidx language command reference.
176 to get a thorough understanding of the language.
178
180 Here we specify the format used by the doctools v2 packages to serial‐
181 ize keyword indices as immutable values for transport, comparison, etc.
182 We distinguish between regular and canonical serializations. While a
184 keyword index may have more than one regular serialization only exactly
185 one of them will be canonical.
186 regular serialization
188 [1] An index serialization is a nested Tcl dictionary.
190 [2] This dictionary holds a single key, doctools::idx, and
192 its value. This value holds the contents of the index.
193 [3] The contents of the index are a Tcl dictionary holding
195 the title of the index, a label, and the keywords and
196 references. The relevant keys and their values are
197 title The value is a string containing the title of the
199 index.
200 label The value is a string containing a label for the
202 index.
203 keywords
205 The value is a Tcl dictionary, using the keywords
206 known to the index as keys. The associated values
207 are lists containing the identifiers of the refer‐
208 ences associated with that particular keyword.
209 Any reference identifier used in these lists has
211 to exist as a key in the references dictionary,
212 see the next item for its definition.
213 references
215 The value is a Tcl dictionary, using the identi‐
216 fiers for the references known to the index as
217 keys. The associated values are 2-element lists
218 containing the type and label of the reference, in
219 this order.
220 Any key here has to be associated with at least
222 one keyword, i.e. occur in at least one of the
223 reference lists which are the values in the key‐
224 words dictionary, see previous item for its defi‐
225 nition.
226 [4] The type of a reference can be one of two values,
228 manpage
230 The identifier of the reference is interpreted as
231 symbolic file name, referring to one of the docu‐
232 ments the index was made for.
233 url The identifier of the reference is interpreted as
235 an url, referring to some external location, like
236 a website, etc.
237 canonical serialization
239 The canonical serialization of a keyword index has the format as
240 specified in the previous item, and then additionally satisfies
241 the constraints below, which make it unique among all the possi‐
242 ble serializations of the keyword index.
243 [1] The keys found in all the nested Tcl dictionaries are
245 sorted in ascending dictionary order, as generated by
246 Tcl's builtin command lsort -increasing -dict.
247 [2] The references listed for each keyword of the index, if
249 any, are listed in ascending dictionary order of their
250 labels, as generated by Tcl's builtin command lsort
251 -increasing -dict.
252
254 This document, and the package it describes, will undoubtedly contain
255 bugs and other problems. Please report such in the category doctools
256 of the Tcllib Trackers [http://core.tcl.tk/tcllib/reportlist]. Please
257 also report any ideas for enhancements you may have for either package
258 and/or documentation.
259 When proposing code changes, please provide unified diffs, i.e the out‐
261 put of diff -u.
262 Note further that attachments are strongly preferred over inlined
264 patches. Attachments can be made by going to the Edit form of the
265 ticket immediately after its creation, and then using the left-most
266 button in the secondary navigation bar.
267
269 docidx, doctools, lexer, parser
270
272 Documentation tools
273
275 Copyright (c) 2009 Andreas Kupries <andreas_kupries@users.sourceforge.net>
276tcllib 1 doctools::idx::parse(n)