1LIBUCL(3)                                                            LIBUCL(3)
2
3
4

NAME

6       ucl_parser_new,   ucl_parser_register_macro,  ucl_parser_register_vari‐
7       able, ucl_parser_add_chunk, ucl_parser_add_string, ucl_parser_add_file,
8       ucl_parser_get_object,  ucl_parser_get_error, ucl_parser_free, ucl_pub‐
9       key_add,  ucl_parser_set_filevars  -  universal  configuration  library
10       parser and utility functions
11

LIBRARY

13       UCL library (libucl, -lucl)
14

SYNOPSIS

16       #include <ucl.h>
17

DESCRIPTION

19       Libucl is a parser and C API to parse and generate ucl objects.  Libucl
20       consist of several groups of functions:
21
22   Parser functions
23       Used to parse ucl files and provide interface to  extract  ucl  object.
24       Currently,  libucl  can parse only full ucl documents, for instance, it
25       is impossible to parse a part of document and therefore it is  impossi‐
26       ble  to  use  libucl as a streaming parser.  In future, this limitation
27       can be removed.
28
29   Emitting functions
30       Convert ucl objects to some textual  or  binary  representation.   Cur‐
31       rently, libucl supports the following exports:
32
33       • JSON  - valid json format (can possibly lose some original data, such
34         as implicit arrays)
35
36       • Config - human-readable configuration format (lossless)
37
38       • YAML - embedded yaml format (has the same limitations as json output)
39
40   Conversion functions
41       Help to convert ucl objects to C types.  These functions  are  used  to
42       convert  ucl_object_t to C primitive types, such as numbers, strings or
43       boolean values.
44
45   Generation functions
46       Allow creation of ucl objects from C types and creating of complex  ucl
47       objects,  such  as hashes or arrays from primitive ucl objects, such as
48       numbers or strings.
49
50   Iteration functions
51       Iterate over ucl complex objects or over a chain of values, for example
52       when a key in an object has multiple values (that can be treated as im‐
53       plicit array or implicit consolidation).
54
55   Validation functions
56       Validation functions  are  used  to  validate  some  object  obj  using
57       json-schema  compatible  object  schema.  Both input and schema must be
58       UCL objects to perform validation.
59
60   Utility functions
61       Provide basic utilities to manage ucl objects: creating, removing,  re‐
62       taining and releasing reference count and so on.
63

PARSER FUNCTIONS

65       Parser functions operates with struct ucl_parser.
66
67   ucl_parser_new
68              struct ucl_parser* ucl_parser_new (int flags);
69
70       Creates new parser with the specified flags:
71
72       • UCL_PARSER_KEY_LOWERCASE - lowercase keys parsed
73
74       • UCL_PARSER_ZEROCOPY  -  try  to use zero-copy mode when reading files
75         (in zero-copy mode text chunk being parsed without copying strings so
76         it should exist till any object parsed is used)
77
78       • UCL_PARSER_NO_TIME  -  treat  time  values as strings without parsing
79         them as floats
80
81   ucl_parser_register_macro
82              void ucl_parser_register_macro (struct ucl_parser *parser,
83                  const char *macro, ucl_macro_handler handler, void* ud);
84
85       Register new macro with name .macro parsed by handler handler that  ac‐
86       cepts opaque data pointer ud.  Macro handler should be of the following
87       type:
88
89              bool (*ucl_macro_handler) (const unsigned char *data,
90                  size_t len, void* ud);`
91
92       Handler function accepts macro text data of length len and  the  opaque
93       pointer  ud.  If macro is parsed successfully the handler should return
94       true.  false indicates parsing failure and the  parser  can  be  termi‐
95       nated.
96
97   ucl_parser_register_variable
98              void ucl_parser_register_variable (struct ucl_parser *parser,
99                  const char *var, const char *value);
100
101       Register new variable $var that should be replaced by the parser to the
102       value string.
103
104   ucl_parser_add_chunk
105              bool ucl_parser_add_chunk (struct ucl_parser *parser,
106                  const unsigned char *data, size_t len);
107
108       Add new text chunk with data of length len to the parser.  At  the  mo‐
109       ment,  libucl parser is not a streamlined parser and chunk must contain
110       the valid ucl object.  For example, this object should be valid:
111
112              { "var": "value" }
113
114       while this one won't be parsed correctly:
115
116              { "var":
117
118       This limitation may possible be removed in future.
119
120   ucl_parser_add_string
121              bool ucl_parser_add_string (struct ucl_parser *parser,
122                  const char *data, size_t len);
123
124       This function acts exactly like ucl_parser_add_chunk does  but  if  len
125       argument  is zero, then the string data must be zero-terminated and the
126       actual length is calculated up to \0 character.
127
128   ucl_parser_add_file
129              bool ucl_parser_add_file (struct ucl_parser *parser,
130                  const char *filename);
131
132       Load file filename and parse it with the specified parser.  This  func‐
133       tion  uses  mmap  call to load file, therefore, it should not be shrunk
134       during parsing.  Otherwise, libucl can cause memory corruption and ter‐
135       minate  the calling application.  This function is also used by the in‐
136       ternal handler of include macro, hence, this macro has the same limita‐
137       tion.
138
139   ucl_parser_get_object
140              ucl_object_t* ucl_parser_get_object (struct ucl_parser *parser);
141
142       If the ucl data has been parsed correctly this function returns the top
143       object for the parser.   Otherwise,  this  function  returns  the  NULL
144       pointer.   The  reference count for ucl object returned is increased by
145       one, therefore, a caller should decrease  reference  by  using  ucl_ob‐
146       ject_unref to free object after usage.
147
148   ucl_parser_get_error
149              const char *ucl_parser_get_error(struct ucl_parser *parser);
150
151       Returns  the  constant error string for the parser object.  If no error
152       occurred during parsing a NULL object is returned.  A caller should not
153       try to free or modify this string.
154
155   ucl_parser_free
156              void ucl_parser_free (struct ucl_parser *parser);
157
158       Frees  memory  occupied  by the parser object.  The reference count for
159       top  object  is  decreased   as   well,   however   if   the   function
160       ucl_parser_get_object  was  called previously then the top object won't
161       be freed.
162
163   ucl_pubkey_add
164              bool ucl_pubkey_add (struct ucl_parser *parser,
165                  const unsigned char *key, size_t len);
166
167       This function adds a public key from text blob key of length len to the
168       parser  object.  This public key should be in the PEM format and can be
169       used by .includes macro for  checking  signatures  of  files  included.
170       Openssl  support should be enabled to make this function working.  If a
171       key cannot be added (e.g.  due to format  error)  or  openssl  was  not
172       linked to libucl then this function returns false.
173
174   ucl_parser_set_filevars
175              bool ucl_parser_set_filevars (struct ucl_parser *parser,
176                  const char *filename, bool need_expand);
177
178       Add  the  standard  file  variables to the parser based on the filename
179       specified:
180
181       • $FILENAME - a filename of ucl input
182
183       • $CURDIR - a current directory of the input
184
185       For example, if a filename param is ../something.conf  then  the  vari‐
186       ables will have the following values:
187
188       • $FILENAME - "../something.conf"
189
190       • $CURDIR - ".."
191
192       if  need_expand  parameter is true then all relative paths are expanded
193       using realpath call.  In this example if .. is /etc/dir then  variables
194       will have these values:
195
196       • $FILENAME - "/etc/something.conf"
197
198       • $CURDIR - "/etc"
199
200   Parser usage example
201       The  following example loads, parses and extracts ucl object from stdin
202       using libucl parser functions (the length of input is limited to 8K):
203
204              char inbuf[8192];
205              struct ucl_parser *parser = NULL;
206              int ret = 0, r = 0;
207              ucl_object_t *obj = NULL;
208              FILE *in;
209
210              in = stdin;
211              parser = ucl_parser_new (0);
212              while (!feof (in) && r < (int)sizeof (inbuf)) {
213                  r += fread (inbuf + r, 1, sizeof (inbuf) - r, in);
214              }
215              ucl_parser_add_chunk (parser, inbuf, r);
216              fclose (in);
217
218              if (ucl_parser_get_error (parser)) {
219                  printf ("Error occurred: %s\n", ucl_parser_get_error (parser));
220                  ret = 1;
221              }
222              else {
223                  obj = ucl_parser_get_object (parser);
224              }
225
226              if (parser != NULL) {
227                  ucl_parser_free (parser);
228              }
229              if (obj != NULL) {
230                  ucl_object_unref (obj);
231              }
232              return ret;
233

EMITTING FUNCTIONS

235       Libucl can transform UCL objects to a number of tectual formats:
236
237       • configuration (UCL_EMIT_CONFIG) - nginx like human readable  configu‐
238         ration  file  where  implicit arrays are transformed to the duplicate
239         keys
240
241       • compact json: UCL_EMIT_JSON_COMPACT - single line valid json  without
242         spaces
243
244       • formatted  json:  UCL_EMIT_JSON - pretty formatted JSON with newlines
245         and spaces
246
247       • compact yaml: UCL_EMIT_YAML - compact YAML output
248
249       Moreover, libucl API allows to select a custom set  of  emitting  func‐
250       tions  allowing  efficient  and  zero-copy  output  of  libucl objects.
251       Libucl uses the following structure to support this feature:
252
253              struct ucl_emitter_functions {
254                  /** Append a single character */
255                  int (*ucl_emitter_append_character) (unsigned char c, size_t nchars, void *ud);
256                  /** Append a string of a specified length */
257                  int (*ucl_emitter_append_len) (unsigned const char *str, size_t len, void *ud);
258                  /** Append a 64 bit integer */
259                  int (*ucl_emitter_append_int) (int64_t elt, void *ud);
260                  /** Append floating point element */
261                  int (*ucl_emitter_append_double) (double elt, void *ud);
262                  /** Opaque userdata pointer */
263                  void *ud;
264              };
265
266       This structure defines the following callbacks:
267
268       • ucl_emitter_append_character - a function that is  called  to  append
269         nchars characters equal to c
270
271       • ucl_emitter_append_len - used to append a string of length len start‐
272         ing from pointer str
273
274       • ucl_emitter_append_int - this function applies to integer numbers
275
276       • ucl_emitter_append_double -  this  function  is  intended  to  output
277         floating point variable
278
279       The  set of these functions could be used to output text formats of UCL
280       objects to different structures or streams.
281
282       Libucl provides the following functions for emitting UCL objects:
283
284   ucl_object_emit
285              unsigned char *ucl_object_emit (const ucl_object_t *obj, enum ucl_emitter emit_type);
286
287       Allocate a string that is suitable to fit the underlying UCL object obj
288       and fill it with the textual representation of the object obj according
289       to style emit_type.  The caller should free the returned  string  after
290       using.
291
292   ucl_object_emit_full
293              bool ucl_object_emit_full (const ucl_object_t *obj, enum ucl_emitter emit_type,
294                      struct ucl_emitter_functions *emitter);
295
296       This function is similar to the previous with the exception that it ac‐
297       cepts the additional argument emitter that defines the concrete set  of
298       output functions.  This emit function could be useful for custom struc‐
299       tures or streams emitters (including C++ ones, for example).
300

CONVERSION FUNCTIONS

302       Conversion functions are used  to  convert  UCL  objects  to  primitive
303       types,  such  as  strings,  numbers,  or boolean values.  There are two
304       types of conversion functions:
305
306       • safe: try to convert an ucl object to a primitive type  and  fail  if
307         such a conversion is not possible
308
309       • unsafe:  return  primitive type without additional checks, if the ob‐
310         ject cannot be converted then some  reasonable  default  is  returned
311         (NULL for strings and 0 for numbers)
312
313       Also  there  is  a single ucl_object_tostring_forced function that con‐
314       verts any UCL object (including compound types - arrays and objects) to
315       a  string  representation.   For  objects, arrays, booleans and numeric
316       types this function performs emitting to a compact  json  format  actu‐
317       ally.
318
319       Here is a list of all conversion functions:
320
321       • ucl_object_toint - returns int64_t of UCL object
322
323       • ucl_object_todouble - returns double of UCL object
324
325       • ucl_object_toboolean - returns bool of UCL object
326
327       • ucl_object_tostring - returns const char * of UCL object (this string
328         is NULL terminated)
329
330       • ucl_object_tolstring - returns const char * and size_t len of UCL ob‐
331         ject (string does not need to be NULL terminated)
332
333       • ucl_object_tostring_forced - returns string representation of any UCL
334         object
335
336       Strings returned by these pointers are associated with the  UCL  object
337       and exist over its lifetime.  A caller should not free this memory.
338

GENERATION FUNCTIONS

340       It  is  possible to generate UCL objects from C primitive types.  More‐
341       over, libucl allows creation and modifying complex UCL objects, such as
342       arrays or associative objects.
343
344   ucl_object_new
345              ucl_object_t * ucl_object_new (void)
346
347       Creates new object of type UCL_NULL.  This object should be released by
348       caller.
349
350   ucl_object_typed_new
351              ucl_object_t * ucl_object_typed_new (unsigned int type)
352
353       Create an object of a specified type:  -  UCL_OBJECT  -  UCL  object  -
354       key/value  pairs  -  UCL_ARRAY - UCL array - UCL_INT - integer number -
355       UCL_FLOAT - floating point number - UCL_STRING - NULL terminated string
356       -  UCL_BOOLEAN  - boolean value - UCL_TIME - time value (floating point
357       number of seconds) - UCL_USERDATA - opaque  userdata  pointer  (may  be
358       used in macros) - UCL_NULL - null value
359
360       This object should be released by caller.
361
362   Primitive objects generation
363       Libucl  provides  the functions similar to inverse conversion functions
364       called with the  specific  C  type:  -  ucl_object_fromint  -  converts
365       int64_t  to UCL object - ucl_object_fromdouble - converts double to UCL
366       object - ucl_object_fromboolean - converts bool to UCL object - ucl_ob‐
367       ject_fromstring  -  converts  const char *  to  UCL object (this string
368       should  be  NULL  terminated)  -  ucl_object_fromlstring   -   converts
369       const char *  and  size_t len to UCL object (string does not need to be
370       NULL terminated)
371
372       Also there is a function to generate UCL object from a string  perform‐
373       ing  various  parsing  or conversion operations called ucl_object_from‐
374       string_common.
375
376   ucl_object_fromstring_common
377              ucl_object_t * ucl_object_fromstring_common (const char *str,
378                  size_t len, enum ucl_string_flags flags)
379
380       This function is used to convert a string str of size len to a UCL  ob‐
381       ject applying flags conversions.  If len is equal to zero then a str is
382       assumed as NULL-terminated.  This function supports the following flags
383       (a set of flags can be specified using logical OR operation):
384
385       • UCL_STRING_ESCAPE - perform JSON escape
386
387       • UCL_STRING_TRIM - trim leading and trailing whitespaces
388
389       • UCL_STRING_PARSE_BOOLEAN - parse passed string and detect boolean
390
391       • UCL_STRING_PARSE_INT - parse passed string and detect integer number
392
393       • UCL_STRING_PARSE_DOUBLE  -  parse passed string and detect integer or
394         float number
395
396       • UCL_STRING_PARSE_TIME - parse time values as floating point numbers
397
398       • UCL_STRING_PARSE_NUMBER - parse passed string and detect number (both
399         float, integer and time types)
400
401       • UCL_STRING_PARSE  - parse passed string (and detect booleans, numbers
402         and time values)
403
404       • UCL_STRING_PARSE_BYTES - assume that numeric multipliers are in bytes
405         notation,  for  example  10k means 10*1024 and not 10*1000 as assumed
406         without this flag
407
408       If parsing operations fail then the resulting  UCL  object  will  be  a
409       UCL_STRING.   A caller should always check the type of the returned ob‐
410       ject and release it after using.
411

ITERATION FUNCTIONS

413       Iteration are used to iterate over UCL compound types: arrays  and  ob‐
414       jects.  Moreover, iterations could be performed over the keys with mul‐
415       tiple values (implicit arrays).  There are two types of iterators  API:
416       old and unsafe one via ucl_iterate_object and the proposed interface of
417       safe iterators.
418
419   ucl_iterate_object
420              const ucl_object_t* ucl_iterate_object (const ucl_object_t *obj,
421                  ucl_object_iter_t *iter, bool expand_values);
422
423       This function accepts opaque iterator pointer iter.  In the first  call
424       this iterator must be initialized to NULL.  Iterator is changed by this
425       function call.  ucl_iterate_object returns the next UCL object  in  the
426       compound  object  obj  or  NULL if all objects have been iterated.  The
427       reference count of the object returned is not increased,  so  a  caller
428       should  not  unref the object or modify its content (e.g.  by inserting
429       to another compound object).  The object obj should not be changed dur‐
430       ing  the  iteration  process  as  well.   expand_values flag speicifies
431       whether ucl_iterate_object should expand  keys  with  multiple  values.
432       The  general  rule is that if you need to iterate through the object or
433       explicit array, then you always need to set this flag  to  true.   How‐
434       ever,  if  you  get  some key in the object and want to extract all its
435       values then you should set expand_values to false.  Mixing of iteration
436       types is not permitted since the iterator is set according to the iter‐
437       ation type and cannot be reused.  Here is an example of iteration  over
438       the  objects  using libucl API (assuming that top is UCL_OBJECT in this
439       example):
440
441              ucl_object_iter_t it = NULL, it_obj = NULL;
442              const ucl_object_t *cur, *tmp;
443
444              /* Iterate over the object */
445              while ((obj = ucl_iterate_object (top, &it, true))) {
446                  printf ("key: \"%s\"\n", ucl_object_key (obj));
447                  /* Iterate over the values of a key */
448                  while ((cur = ucl_iterate_object (obj, &it_obj, false))) {
449                      printf ("value: \"%s\"\n",
450                          ucl_object_tostring_forced (cur));
451                  }
452              }
453
454   Safe iterators API
455       Safe iterators are defined to clarify iterating over  UCL  objects  and
456       simplify  flattening of UCL objects in non-trivial cases.  For example,
457       if there is an implicit array that contains another array and a boolean
458       value it is extremely unclear how to iterate over such an object.  Safe
459       iterators are desinged to define two sorts of iteration:
460
461       1. Iteration over complex objects with expanding all values
462
463       2. Iteration over complex objects without expanding of values
464
465       The following example demonstrates the  difference  between  these  two
466       types of iteration:
467
468              key = 1;
469              key = [2, 3, 4];
470
471              Iteration with expansion:
472
473              1, 2, 3, 4
474
475              Iteration without expansion:
476
477              1, [2, 3, 4]
478
479       UCL defines the following functions to manage safe iterators:
480
481       • ucl_object_iterate_new - creates new safe iterator.
482
483       • ucl_object_iterate_reset - resets iterator to a new object.
484
485       • ucl_object_iterate_safe  - safely iterate the object inside iterator.
486         Note: function may allocate and free  memory  during  its  operation.
487         Therefore  it  returns  NULL either while trying to access item after
488         the last one or when exception (such as  memory  allocation  failure)
489         happens.
490
491       • ucl_object_iter_chk_excpn  - check if the last call to ucl_object_it‐
492         erate_safe ended up in unrecoverable exception (e.g. ENOMEM).
493
494       • ucl_object_iterate_free - free memory associated with the safe itera‐
495         tor.
496
497       Please  note  that  unlike unsafe iterators, safe iterators must be ex‐
498       plicitly initialized and freed.  An assert is likely generated  if  you
499       use uninitialized or NULL iterator in all safe iterators functions.
500
501              ucl_object_iter_t it;
502              const ucl_object_t *cur;
503
504              it = ucl_object_iterate_new (obj);
505
506              while ((cur = ucl_object_iterate_safe (it, true)) != NULL) {
507                  /* Do something */
508              }
509              /* Check error condition */
510              if (ucl_object_iter_chk_excpn (it)) {
511                  ucl_object_iterate_free (it);
512                  exit (1);
513              }
514
515              /* Switch to another object */
516              it = ucl_object_iterate_reset (it, another_obj);
517
518              while ((cur = ucl_object_iterate_safe (it, true)) != NULL) {
519                  /* Do something else */
520              }
521              /* Check error condition */
522              if (ucl_object_iter_chk_excpn (it)) {
523                  ucl_object_iterate_free (it);
524                  exit (1);
525              }
526
527              ucl_object_iterate_free (it);
528

VALIDATION FUNCTIONS

530       Currently, there is only one validation function called ucl_object_val‐
531       idate.  It performs validation of object using  the  specified  schema.
532       This function is defined as following:
533
534   ucl_object_validate
535              bool ucl_object_validate (const ucl_object_t *schema,
536                  const ucl_object_t *obj, struct ucl_schema_error *err);
537
538       This  function  uses  ucl object schema, that must be valid in terms of
539       json-schema draft v4, to validate input object obj.  If  this  function
540       returns  true  then  validation procedure has been succeed.  Otherwise,
541       false is returned and err is set to a specific value.  If a caller sets
542       err  to  NULL  then this function does not set any error just returning
543       false.  Error is the structure defined as following:
544
545              struct ucl_schema_error {
546                  enum ucl_schema_error_code code;    /* error code */
547                  char msg[128];              /* error message */
548                  ucl_object_t *obj;          /* object where error occurred */
549              };
550
551       Caller may use code field to get a numeric error code:
552
553              enum ucl_schema_error_code {
554                  UCL_SCHEMA_OK = 0,          /* no error */
555                  UCL_SCHEMA_TYPE_MISMATCH,   /* type of object is incorrect */
556                  UCL_SCHEMA_INVALID_SCHEMA,  /* schema is invalid */
557                  UCL_SCHEMA_MISSING_PROPERTY,/* missing properties */
558                  UCL_SCHEMA_CONSTRAINT,      /* constraint found */
559                  UCL_SCHEMA_MISSING_DEPENDENCY, /* missing dependency */
560                  UCL_SCHEMA_UNKNOWN          /* generic error */
561              };
562
563       msg is a string description of an error and obj is an object where  er‐
564       ror has occurred.  Error object is not allocated by libucl, so there is
565       no need to free it after validation (a static  object  should  thus  be
566       used).
567

AUTHORS

569       Vsevolod Stakhov <vsevolod@highsecure.ru>.
570
571
572
573Libucl manual                  27 December, 2014                     LIBUCL(3)
Impressum