1BT_POST_PROCESSING(1) btparse BT_POST_PROCESSING(1)
2
6 bt_post_processing - post-processing of BibTeX strings, values, and
7 entries
8
10 void bt_postprocess_string (char * s,
11 btshort options)
12 char * bt_postprocess_value (AST * value,
14 btshort options,
15 boolean replace);
16 char * bt_postprocess_field (AST * field,
18 btshort options,
19 boolean replace);
20 void bt_postprocess_entry (AST * entry,
22 btshort options);
23
25 When btparse parses a BibTeX entry, it initially stores the results in
26 an abstract syntax tree (AST), in a form exactly mirroring the parsed
27 data. For example, the entry
28 @Article{Jones:1997a,
30 AuThOr = "Bob Jones" # and # "Jim Smith ",
31 TITLE = "Feeding Habits of
32 the Common Cockroach",
33 JoUrNaL = j_ent,
34 YEAR = 1997
35 }
36 would parse to an AST that could be represented as follows:
38 (entry,"Article")
40 (key,"Jones:1997a")
41 (field,"AuThOr")
42 (string,"Bob Jones")
43 (macro,"and")
44 (string,"Jim Smith ")
45 (field,"TITLE")
46 (string,"Feeding Habits of the Common Cockroach")
47 (field,"JoUrNaL")
48 (macro,"j_ent")
49 (field,"YEAR")
50 (number,"1997")
51 The advantage of this form is that all the important information in the
53 entry is readily available by traversing the tree using the functions
54 described in bt_traversal. This obvious problem is that the data is a
55 little too raw to be immediately useful: entry types and field names
56 are inconsistently capitalized, strings are full of unwanted
57 whitespace, field values not reduced to single strings, and so forth.
58 All of these problems are addressed by btparse's post-processing
60 functions, described here. Normally, you won't have to call these
61 functions---the library does the Right Thing for you after parsing each
62 entry, and you can customize what exactly the Right Thing is for your
63 application. (For instance, you can tell it to expand macros, but not
64 to concatenate substrings together.) However, it's conceivable that
65 you might wish to move the post-processing into your own code and out
66 of the library's control. More likely, you could have strings that
67 come from something other than BibTeX files that you would like to have
68 treated as BibTeX strings; for that situation, the post-processing
69 functions are essential. Finally, you might just be curious about what
70 exactly happens to your data after it's parsed. If so, you've come to
71 the right place for excruciatingly detailed explanations.
72
74 btparse offers four points of entry to its post-processing code. Of
75 these, probably only the first and last---for processing individual
76 strings and whole entries---will be commonly used.
77 Post-processing entry points
79 To understand why four entry points are offered, an explanation of the
80 sample AST shown above will help. First of all, the whole entry is
81 represented by the "(entry,"Article")" node; this node has the entry
82 key and all its field/value pairs as children. Entry nodes are
83 returned by bt_parse_entry() and bt_parse_entry_s() (see bt_input) as
84 well as bt_next_entry() (which traverses a list of entries returned
85 from bt_parse_file()---see bt_traversal). Whole entries may be post-
86 processed with bt_postprocess_entry().
87 You may also need to post-process a single field, or just the value
89 associated with it. (The difference is that processing the field can
90 change the field name---e.g. to lowercase---in addition to the field
91 value.) The "(field,"AuThOr")" node above is an example of a field
92 sub-AST, and "(string,"Bob Jones")" is the first node in the list of
93 simple values representing that field's value. (Recall that a field
94 value is, in general, a list of simple values.) Field nodes are
95 returned by bt_next_field(), value nodes by bt_next_value(). The
96 former may be passed to bt_postprocess_field() for post-processing, the
97 latter to bt_postprocess_value().
98 Finally, individual strings may wander into your program from many
100 places other than a btparse AST. For that reason,
101 bt_postprocess_string() is available for post-processing arbitrary
102 strings.
103 Post-processing options
105 All of the post-processing routines have an "options" parameter, which
106 you can use to fine-tune the post-processing. (This is just like the
107 per-metatype string-processing options that you can set before parsing
108 entries; see bt_set_stringopts() in bt_input.) Like elsewhere in the
109 library, "options" is a bitmap constructed by or'ing together various
110 predefined constants. These constants and their effects are documented
111 in "String processing option macros" in btparse.
112 bt_postprocess_string ()
114 void bt_postprocess_string (char * s,
115 btshort options)
116 Post-processes an individual string, "s", which is modified in
118 place. The only post-processing option that makes sense on
119 individual strings is whether to collapse whitespace according to
120 the BibTeX rules; thus, if "options & BTO_COLLAPSE" is false, this
121 function has no effect. (Although it makes a complete pass over
122 the string anyways. This is for future expansion.)
123 The exact rules for collapsing whitespace are simple: non-space
125 whitespace characters (tabs and newlines mainly) are converted to
126 space, any strings of more than one space within are collapsed to a
127 single space, and any leading or trailing spaces are deleted.
128 (Ensuring that all whitespace is spaces is actually done by
129 btparse's lexical scanner, so strings in btparse ASTs will never
130 have whitespace apart from space. Likewise, any strings passed to
131 bt_postprocess_string() should not contain non-space whitespace
132 characters.)
133 bt_postprocess_value ()
135 char * bt_postprocess_value (AST * value,
136 btshort options,
137 boolean replace);
138 Post-processes a single field value, which is the head of a list of
140 simple values as returned by bt_next_value(). All of the relevant
141 string-processing options come into play here: conversion of
142 numbers to strings ("BTO_CONVERT"), macro expansion ("BTO_EXPAND"),
143 collapsing of whitespace ("BTO_COLLAPSE"), and string pasting
144 ("BTO_PASTE"). Since pasting substrings together without first
145 expanding macros and converting numbers would be nonsensical,
146 attempting to do so is a fatal error.
147 If "replace" is true, then the list headed by "value" will be
149 replaced by a list representing the processed value. That is, if
150 string pasting is turned on ("options & BTO_PASTE" is true), then
151 this list will be collapsed to a single node containing the single
152 string that results from pasting together all the substrings. If
153 string pasting is not on, then each node in the list will be left
154 intact, but will have its text replaced by processed text.
155 If "replace" is false, then a new string will be built on the fly
157 and returned by the function. Note that if pasting is not on in
158 this case, you will only get the last string in the list. (It
159 doesn't really make a lot of sense to post-process a value without
160 pasting unless you're replacing it with the new value, though.)
161 Returns the string that resulted from processing the whole value,
163 which only makes sense if pasting was on or there was only one
164 value in the list. If a multiple-value list was processed without
165 pasting, the last string in the list is returned (after
166 processing).
167 Consider what might be done to the value of the "author" field in
169 the above example, which is the concatenation of a string, a macro,
170 and another string. Assume that the macro "and" expands to " and
171 ", and that the variable "value" points to the sub-AST for this
172 value. The original sub-AST corresponding to this value is
173 (string,"Bob Jones")
175 (macro,"and")
176 (string,"Jim Smith ")
177 To fully process this value in-place, you would call
179 bt_postprocess_value (value, BTO_FULL, TRUE);
181 ("BTO_FULL" is just the combination of all possible string-
183 processing options:
184 "BTO_CONVERT|BTO_EXPAND|BTO_PASTE|BTO_COLLAPSE".) This would
185 convert the value to a single-element list,
186 (string,"Bob Jones and Jim Smith")
188 and return the fully-processed string "Bob Jones and Jim Smith".
190 Note that the "and" macro has been expanded, interpolated between
191 the two literal strings, everything pasted together, and finally
192 whitespace collapsed. (Collapsing whitespace before concatenating
193 the strings would be a bad idea.)
194 Let's say you'd rather preserve the list nature of the value, while
196 expanding macros and converting any numbers to strings. (This
197 conversion is trivial: it just changes the type of the node from
198 "BTAST_NUMBER" to "BTAST_STRING". "Number" values are always
199 stored as a string of digits, just as they appear in the file.)
200 This would be done with the call
201 bt_postprocess_value
203 (value, BTO_CONVERT|BTO_EXPAND|BTO_COLLAPSE,TRUE);
204 which would change the list to
206 (string,"Bob Jones")
208 (string,"and")
209 (string,"Jim Smith")
210 Note that whitespace is collapsed here before any concatenation can
212 be done; this is probably a bad idea. But you can do it if you
213 wish. (If you get any ideas about cooking up your own value post-
214 processing scheme by doing it in little steps like this, take a
215 look at the source to bt_postprocess_value(); it should dissuade
216 you from such a venture.)
217 bt_postprocess_field ()
219 char * bt_postprocess_field (AST * field,
220 btshort options,
221 boolean replace);
222 This is little more than a front-end to bt_postprocess_value(); the
224 only difference is that you pass it a "field" AST node (eg. the
225 "(field,"AuThOr")" in the above example), and that it transforms
226 the field name in addition to its value. In particular, the field
227 name is forced to lowercase; this behaviour is (currently) not
228 optional.
229 Returns the string returned by bt_postprocess_value().
231 bt_postprocess_entry ()
233 void bt_postprocess_entry (AST * entry,
234 btshort options);
235 Post-processes all values in an entry. If "entry" points to the
237 AST for a "regular" or "macro definition" entry, then the values
238 are just what you'd expect: everything on the right-hand side of a
239 field or macro "assignment." You can also post-process comment and
240 preamble entries, though. Comment entries are essentially one big
241 string, so only whitespace collapsing makes sense on them.
242 Preambles may have multiple strings pasted together, so all the
243 string-processing options apply to them. (And there's nothing to
244 prevent you from using macros in a preamble.)
245btparse, version 0.89 2023-01-29 BT_POST_PROCESSING(1)