1LIBUCL(3) LIBUCL(3)
2
6 ucl_parser_new, ucl_parser_register_macro, ucl_parser_register_vari‐
7 able, ucl_parser_add_chunk, ucl_parser_add_string, ucl_parser_add_file,
8 ucl_parser_get_object, ucl_parser_get_error, ucl_parser_free, ucl_pub‐
9 key_add, ucl_parser_set_filevars - universal configuration library
10 parser and utility functions
11
13 UCL library (libucl, -lucl)
14
16 #include <ucl.h>
17
19 Libucl is a parser and C API to parse and generate ucl objects. Libucl
20 consist of several groups of functions:
21 Parser functions
23 Used to parse ucl files and provide interface to extract ucl object.
24 Currently, libucl can parse only full ucl documents, for instance, it
25 is impossible to parse a part of document and therefore it is impossi‐
26 ble to use libucl as a streaming parser. In future, this limitation
27 can be removed.
28 Emitting functions
30 Convert ucl objects to some textual or binary representation. Cur‐
31 rently, libucl supports the following exports:
32 • JSON - valid json format (can possibly lose some original data, such
34 as implicit arrays)
35 • Config - human-readable configuration format (lossless)
37 • YAML - embedded yaml format (has the same limitations as json output)
39 Conversion functions
41 Help to convert ucl objects to C types. These functions are used to
42 convert ucl_object_t to C primitive types, such as numbers, strings or
43 boolean values.
44 Generation functions
46 Allow creation of ucl objects from C types and creating of complex ucl
47 objects, such as hashes or arrays from primitive ucl objects, such as
48 numbers or strings.
49 Iteration functions
51 Iterate over ucl complex objects or over a chain of values, for example
52 when a key in an object has multiple values (that can be treated as im‐
53 plicit array or implicit consolidation).
54 Validation functions
56 Validation functions are used to validate some object obj using
57 json-schema compatible object schema. Both input and schema must be
58 UCL objects to perform validation.
59 Utility functions
61 Provide basic utilities to manage ucl objects: creating, removing, re‐
62 taining and releasing reference count and so on.
63
65 Parser functions operates with struct ucl_parser.
66 ucl_parser_new
68 struct ucl_parser* ucl_parser_new (int flags);
69 Creates new parser with the specified flags:
71 • UCL_PARSER_KEY_LOWERCASE - lowercase keys parsed
73 • UCL_PARSER_ZEROCOPY - try to use zero-copy mode when reading files
75 (in zero-copy mode text chunk being parsed without copying strings so
76 it should exist till any object parsed is used)
77 • UCL_PARSER_NO_TIME - treat time values as strings without parsing
79 them as floats
80 ucl_parser_register_macro
82 void ucl_parser_register_macro (struct ucl_parser *parser,
83 const char *macro, ucl_macro_handler handler, void* ud);
84 Register new macro with name .macro parsed by handler handler that ac‐
86 cepts opaque data pointer ud. Macro handler should be of the following
87 type:
88 bool (*ucl_macro_handler) (const unsigned char *data,
90 size_t len, void* ud);`
91 Handler function accepts macro text data of length len and the opaque
93 pointer ud. If macro is parsed successfully the handler should return
94 true. false indicates parsing failure and the parser can be termi‐
95 nated.
96 ucl_parser_register_variable
98 void ucl_parser_register_variable (struct ucl_parser *parser,
99 const char *var, const char *value);
100 Register new variable $var that should be replaced by the parser to the
102 value string.
103 ucl_parser_add_chunk
105 bool ucl_parser_add_chunk (struct ucl_parser *parser,
106 const unsigned char *data, size_t len);
107 Add new text chunk with data of length len to the parser. At the mo‐
109 ment, libucl parser is not a streamlined parser and chunk must contain
110 the valid ucl object. For example, this object should be valid:
111 { "var": "value" }
113 while this one won't be parsed correctly:
115 { "var":
117 This limitation may possible be removed in future.
119 ucl_parser_add_string
121 bool ucl_parser_add_string (struct ucl_parser *parser,
122 const char *data, size_t len);
123 This function acts exactly like ucl_parser_add_chunk does but if len
125 argument is zero, then the string data must be zero-terminated and the
126 actual length is calculated up to \0 character.
127 ucl_parser_add_file
129 bool ucl_parser_add_file (struct ucl_parser *parser,
130 const char *filename);
131 Load file filename and parse it with the specified parser. This func‐
133 tion uses mmap call to load file, therefore, it should not be shrunk
134 during parsing. Otherwise, libucl can cause memory corruption and ter‐
135 minate the calling application. This function is also used by the in‐
136 ternal handler of include macro, hence, this macro has the same limita‐
137 tion.
138 ucl_parser_get_object
140 ucl_object_t* ucl_parser_get_object (struct ucl_parser *parser);
141 If the ucl data has been parsed correctly this function returns the top
143 object for the parser. Otherwise, this function returns the NULL
144 pointer. The reference count for ucl object returned is increased by
145 one, therefore, a caller should decrease reference by using ucl_ob‐
146 ject_unref to free object after usage.
147 ucl_parser_get_error
149 const char *ucl_parser_get_error(struct ucl_parser *parser);
150 Returns the constant error string for the parser object. If no error
152 occurred during parsing a NULL object is returned. A caller should not
153 try to free or modify this string.
154 ucl_parser_free
156 void ucl_parser_free (struct ucl_parser *parser);
157 Frees memory occupied by the parser object. The reference count for
159 top object is decreased as well, however if the function
160 ucl_parser_get_object was called previously then the top object won't
161 be freed.
162 ucl_pubkey_add
164 bool ucl_pubkey_add (struct ucl_parser *parser,
165 const unsigned char *key, size_t len);
166 This function adds a public key from text blob key of length len to the
168 parser object. This public key should be in the PEM format and can be
169 used by .includes macro for checking signatures of files included.
170 Openssl support should be enabled to make this function working. If a
171 key cannot be added (e.g. due to format error) or openssl was not
172 linked to libucl then this function returns false.
173 ucl_parser_set_filevars
175 bool ucl_parser_set_filevars (struct ucl_parser *parser,
176 const char *filename, bool need_expand);
177 Add the standard file variables to the parser based on the filename
179 specified:
180 • $FILENAME - a filename of ucl input
182 • $CURDIR - a current directory of the input
184 For example, if a filename param is ../something.conf then the vari‐
186 ables will have the following values:
187 • $FILENAME - "../something.conf"
189 • $CURDIR - ".."
191 if need_expand parameter is true then all relative paths are expanded
193 using realpath call. In this example if .. is /etc/dir then variables
194 will have these values:
195 • $FILENAME - "/etc/something.conf"
197 • $CURDIR - "/etc"
199 Parser usage example
201 The following example loads, parses and extracts ucl object from stdin
202 using libucl parser functions (the length of input is limited to 8K):
203 char inbuf[8192];
205 struct ucl_parser *parser = NULL;
206 int ret = 0, r = 0;
207 ucl_object_t *obj = NULL;
208 FILE *in;
209 in = stdin;
211 parser = ucl_parser_new (0);
212 while (!feof (in) && r < (int)sizeof (inbuf)) {
213 r += fread (inbuf + r, 1, sizeof (inbuf) - r, in);
214 }
215 ucl_parser_add_chunk (parser, inbuf, r);
216 fclose (in);
217 if (ucl_parser_get_error (parser)) {
219 printf ("Error occurred: %s\n", ucl_parser_get_error (parser));
220 ret = 1;
221 }
222 else {
223 obj = ucl_parser_get_object (parser);
224 }
225 if (parser != NULL) {
227 ucl_parser_free (parser);
228 }
229 if (obj != NULL) {
230 ucl_object_unref (obj);
231 }
232 return ret;
233
235 Libucl can transform UCL objects to a number of tectual formats:
236 • configuration (UCL_EMIT_CONFIG) - nginx like human readable configu‐
238 ration file where implicit arrays are transformed to the duplicate
239 keys
240 • compact json: UCL_EMIT_JSON_COMPACT - single line valid json without
242 spaces
243 • formatted json: UCL_EMIT_JSON - pretty formatted JSON with newlines
245 and spaces
246 • compact yaml: UCL_EMIT_YAML - compact YAML output
248 Moreover, libucl API allows to select a custom set of emitting func‐
250 tions allowing efficient and zero-copy output of libucl objects.
251 Libucl uses the following structure to support this feature:
252 struct ucl_emitter_functions {
254 /** Append a single character */
255 int (*ucl_emitter_append_character) (unsigned char c, size_t nchars, void *ud);
256 /** Append a string of a specified length */
257 int (*ucl_emitter_append_len) (unsigned const char *str, size_t len, void *ud);
258 /** Append a 64 bit integer */
259 int (*ucl_emitter_append_int) (int64_t elt, void *ud);
260 /** Append floating point element */
261 int (*ucl_emitter_append_double) (double elt, void *ud);
262 /** Opaque userdata pointer */
263 void *ud;
264 };
265 This structure defines the following callbacks:
267 • ucl_emitter_append_character - a function that is called to append
269 nchars characters equal to c
270 • ucl_emitter_append_len - used to append a string of length len start‐
272 ing from pointer str
273 • ucl_emitter_append_int - this function applies to integer numbers
275 • ucl_emitter_append_double - this function is intended to output
277 floating point variable
278 The set of these functions could be used to output text formats of UCL
280 objects to different structures or streams.
281 Libucl provides the following functions for emitting UCL objects:
283 ucl_object_emit
285 unsigned char *ucl_object_emit (const ucl_object_t *obj, enum ucl_emitter emit_type);
286 Allocate a string that is suitable to fit the underlying UCL object obj
288 and fill it with the textual representation of the object obj according
289 to style emit_type. The caller should free the returned string after
290 using.
291 ucl_object_emit_full
293 bool ucl_object_emit_full (const ucl_object_t *obj, enum ucl_emitter emit_type,
294 struct ucl_emitter_functions *emitter);
295 This function is similar to the previous with the exception that it ac‐
297 cepts the additional argument emitter that defines the concrete set of
298 output functions. This emit function could be useful for custom struc‐
299 tures or streams emitters (including C++ ones, for example).
300
302 Conversion functions are used to convert UCL objects to primitive
303 types, such as strings, numbers, or boolean values. There are two
304 types of conversion functions:
305 • safe: try to convert an ucl object to a primitive type and fail if
307 such a conversion is not possible
308 • unsafe: return primitive type without additional checks, if the ob‐
310 ject cannot be converted then some reasonable default is returned
311 (NULL for strings and 0 for numbers)
312 Also there is a single ucl_object_tostring_forced function that con‐
314 verts any UCL object (including compound types - arrays and objects) to
315 a string representation. For objects, arrays, booleans and numeric
316 types this function performs emitting to a compact json format actu‐
317 ally.
318 Here is a list of all conversion functions:
320 • ucl_object_toint - returns int64_t of UCL object
322 • ucl_object_todouble - returns double of UCL object
324 • ucl_object_toboolean - returns bool of UCL object
326 • ucl_object_tostring - returns const char * of UCL object (this string
328 is NULL terminated)
329 • ucl_object_tolstring - returns const char * and size_t len of UCL ob‐
331 ject (string does not need to be NULL terminated)
332 • ucl_object_tostring_forced - returns string representation of any UCL
334 object
335 Strings returned by these pointers are associated with the UCL object
337 and exist over its lifetime. A caller should not free this memory.
338
340 It is possible to generate UCL objects from C primitive types. More‐
341 over, libucl allows creation and modifying complex UCL objects, such as
342 arrays or associative objects.
343 ucl_object_new
345 ucl_object_t * ucl_object_new (void)
346 Creates new object of type UCL_NULL. This object should be released by
348 caller.
349 ucl_object_typed_new
351 ucl_object_t * ucl_object_typed_new (unsigned int type)
352 Create an object of a specified type: - UCL_OBJECT - UCL object -
354 key/value pairs - UCL_ARRAY - UCL array - UCL_INT - integer number -
355 UCL_FLOAT - floating point number - UCL_STRING - NULL terminated string
356 - UCL_BOOLEAN - boolean value - UCL_TIME - time value (floating point
357 number of seconds) - UCL_USERDATA - opaque userdata pointer (may be
358 used in macros) - UCL_NULL - null value
359 This object should be released by caller.
361 Primitive objects generation
363 Libucl provides the functions similar to inverse conversion functions
364 called with the specific C type: - ucl_object_fromint - converts
365 int64_t to UCL object - ucl_object_fromdouble - converts double to UCL
366 object - ucl_object_fromboolean - converts bool to UCL object - ucl_ob‐
367 ject_fromstring - converts const char * to UCL object (this string
368 should be NULL terminated) - ucl_object_fromlstring - converts
369 const char * and size_t len to UCL object (string does not need to be
370 NULL terminated)
371 Also there is a function to generate UCL object from a string perform‐
373 ing various parsing or conversion operations called ucl_object_from‐
374 string_common.
375 ucl_object_fromstring_common
377 ucl_object_t * ucl_object_fromstring_common (const char *str,
378 size_t len, enum ucl_string_flags flags)
379 This function is used to convert a string str of size len to a UCL ob‐
381 ject applying flags conversions. If len is equal to zero then a str is
382 assumed as NULL-terminated. This function supports the following flags
383 (a set of flags can be specified using logical OR operation):
384 • UCL_STRING_ESCAPE - perform JSON escape
386 • UCL_STRING_TRIM - trim leading and trailing whitespaces
388 • UCL_STRING_PARSE_BOOLEAN - parse passed string and detect boolean
390 • UCL_STRING_PARSE_INT - parse passed string and detect integer number
392 • UCL_STRING_PARSE_DOUBLE - parse passed string and detect integer or
394 float number
395 • UCL_STRING_PARSE_TIME - parse time values as floating point numbers
397 • UCL_STRING_PARSE_NUMBER - parse passed string and detect number (both
399 float, integer and time types)
400 • UCL_STRING_PARSE - parse passed string (and detect booleans, numbers
402 and time values)
403 • UCL_STRING_PARSE_BYTES - assume that numeric multipliers are in bytes
405 notation, for example 10k means 10*1024 and not 10*1000 as assumed
406 without this flag
407 If parsing operations fail then the resulting UCL object will be a
409 UCL_STRING. A caller should always check the type of the returned ob‐
410 ject and release it after using.
411
413 Iteration are used to iterate over UCL compound types: arrays and ob‐
414 jects. Moreover, iterations could be performed over the keys with mul‐
415 tiple values (implicit arrays). There are two types of iterators API:
416 old and unsafe one via ucl_iterate_object and the proposed interface of
417 safe iterators.
418 ucl_iterate_object
420 const ucl_object_t* ucl_iterate_object (const ucl_object_t *obj,
421 ucl_object_iter_t *iter, bool expand_values);
422 This function accepts opaque iterator pointer iter. In the first call
424 this iterator must be initialized to NULL. Iterator is changed by this
425 function call. ucl_iterate_object returns the next UCL object in the
426 compound object obj or NULL if all objects have been iterated. The
427 reference count of the object returned is not increased, so a caller
428 should not unref the object or modify its content (e.g. by inserting
429 to another compound object). The object obj should not be changed dur‐
430 ing the iteration process as well. expand_values flag speicifies
431 whether ucl_iterate_object should expand keys with multiple values.
432 The general rule is that if you need to iterate through the object or
433 explicit array, then you always need to set this flag to true. How‐
434 ever, if you get some key in the object and want to extract all its
435 values then you should set expand_values to false. Mixing of iteration
436 types is not permitted since the iterator is set according to the iter‐
437 ation type and cannot be reused. Here is an example of iteration over
438 the objects using libucl API (assuming that top is UCL_OBJECT in this
439 example):
440 ucl_object_iter_t it = NULL, it_obj = NULL;
442 const ucl_object_t *cur, *tmp;
443 /* Iterate over the object */
445 while ((obj = ucl_iterate_object (top, &it, true))) {
446 printf ("key: \"%s\"\n", ucl_object_key (obj));
447 /* Iterate over the values of a key */
448 while ((cur = ucl_iterate_object (obj, &it_obj, false))) {
449 printf ("value: \"%s\"\n",
450 ucl_object_tostring_forced (cur));
451 }
452 }
453 Safe iterators API
455 Safe iterators are defined to clarify iterating over UCL objects and
456 simplify flattening of UCL objects in non-trivial cases. For example,
457 if there is an implicit array that contains another array and a boolean
458 value it is extremely unclear how to iterate over such an object. Safe
459 iterators are desinged to define two sorts of iteration:
460 1. Iteration over complex objects with expanding all values
462 2. Iteration over complex objects without expanding of values
464 The following example demonstrates the difference between these two
466 types of iteration:
467 key = 1;
469 key = [2, 3, 4];
470 Iteration with expansion:
472 1, 2, 3, 4
474 Iteration without expansion:
476 1, [2, 3, 4]
478 UCL defines the following functions to manage safe iterators:
480 • ucl_object_iterate_new - creates new safe iterator.
482 • ucl_object_iterate_reset - resets iterator to a new object.
484 • ucl_object_iterate_safe - safely iterate the object inside iterator.
486 Note: function may allocate and free memory during its operation.
487 Therefore it returns NULL either while trying to access item after
488 the last one or when exception (such as memory allocation failure)
489 happens.
490 • ucl_object_iter_chk_excpn - check if the last call to ucl_object_it‐
492 erate_safe ended up in unrecoverable exception (e.g. ENOMEM).
493 • ucl_object_iterate_free - free memory associated with the safe itera‐
495 tor.
496 Please note that unlike unsafe iterators, safe iterators must be ex‐
498 plicitly initialized and freed. An assert is likely generated if you
499 use uninitialized or NULL iterator in all safe iterators functions.
500 ucl_object_iter_t it;
502 const ucl_object_t *cur;
503 it = ucl_object_iterate_new (obj);
505 while ((cur = ucl_object_iterate_safe (it, true)) != NULL) {
507 /* Do something */
508 }
509 /* Check error condition */
510 if (ucl_object_iter_chk_excpn (it)) {
511 ucl_object_iterate_free (it);
512 exit (1);
513 }
514 /* Switch to another object */
516 it = ucl_object_iterate_reset (it, another_obj);
517 while ((cur = ucl_object_iterate_safe (it, true)) != NULL) {
519 /* Do something else */
520 }
521 /* Check error condition */
522 if (ucl_object_iter_chk_excpn (it)) {
523 ucl_object_iterate_free (it);
524 exit (1);
525 }
526 ucl_object_iterate_free (it);
528
530 Currently, there is only one validation function called ucl_object_val‐
531 idate. It performs validation of object using the specified schema.
532 This function is defined as following:
533 ucl_object_validate
535 bool ucl_object_validate (const ucl_object_t *schema,
536 const ucl_object_t *obj, struct ucl_schema_error *err);
537 This function uses ucl object schema, that must be valid in terms of
539 json-schema draft v4, to validate input object obj. If this function
540 returns true then validation procedure has been succeed. Otherwise,
541 false is returned and err is set to a specific value. If a caller sets
542 err to NULL then this function does not set any error just returning
543 false. Error is the structure defined as following:
544 struct ucl_schema_error {
546 enum ucl_schema_error_code code; /* error code */
547 char msg[128]; /* error message */
548 ucl_object_t *obj; /* object where error occurred */
549 };
550 Caller may use code field to get a numeric error code:
552 enum ucl_schema_error_code {
554 UCL_SCHEMA_OK = 0, /* no error */
555 UCL_SCHEMA_TYPE_MISMATCH, /* type of object is incorrect */
556 UCL_SCHEMA_INVALID_SCHEMA, /* schema is invalid */
557 UCL_SCHEMA_MISSING_PROPERTY,/* missing properties */
558 UCL_SCHEMA_CONSTRAINT, /* constraint found */
559 UCL_SCHEMA_MISSING_DEPENDENCY, /* missing dependency */
560 UCL_SCHEMA_UNKNOWN /* generic error */
561 };
562 msg is a string description of an error and obj is an object where er‐
564 ror has occurred. Error object is not allocated by libucl, so there is
565 no need to free it after validation (a static object should thus be
566 used).
567
569 Vsevolod Stakhov <vsevolod@highsecure.ru>.
570Libucl manual 27 December, 2014 LIBUCL(3)