1LIBCBOR(1) libcbor LIBCBOR(1)
2
6 libcbor - libcbor Documentation
7 Documentation for version 0.7.0, updated on Jul 22, 2021.
9
11 libcbor is a C library for parsing and generating CBOR, the gen‐
12 eral-purpose schema-less binary data format.
13 Main features
15 • Complete RFC conformance [1]
17 • Robust C99 implementation
19 • Layered architecture offers both control and convenience
21 • Flexible memory management
23 • No shared global state - threading friendly [2]
25 • Proper handling of UTF-8
27 • Full support for streams & incremental processing
29 • Extensive documentation and test suite
31 • No runtime dependencies, small footprint
33 [1] See rfc_conformance
35 [2] With the exception of custom memory allocators (see api/item_ref‐
37 erence_counting)
38
40 Getting started
41 Pre-built Linux packages are distributed from the libcbor website.
42 OS X users can use Homebrew:
44 brew tap pjk/libcbor
46 brew install libcbor
47 For other platforms, you will need to compile it from source.
49 Building & installing libcbor
51 Prerequisites:
52 • C99 compiler
54 • CMake 2.8 or newer (might also be called cmakesetup, cmake-gui
56 or ccmake depending on the installed version and system)
57 • C build system CMake can target (make, Apple Xcode, MinGW,
59 ...)
60 NOTE:
62 As of May 2015, not even the 2015 release candidate of Visual Studio
63 supports C99. While CMake will be happy to generate a VS solution
64 that you can play with, libcbor currently cannot be compiled using
65 the MSVC toolchain. ICC, GCC under Cygwin, and MinGW's GCC will all
66 work. The MinGW build process is described below.
67 Configuration options
69 A handful of configuration flags can be passed to cmake. The following
71 table lists libcbor compile-time directives and several important
72 generic flags.
73 ┌─────────────────┬──────────────────┬──────────────────┬──────────────────┐
75 │Option │ Meaning │ Default │ Possible values │
76 ├─────────────────┼──────────────────┼──────────────────┼──────────────────┤
77 │CMAKE_C_COMPILER │ C compiler to │ cc │ gcc, clang, │
78 │ │ use │ │ clang-3.5, ... │
79 ├─────────────────┼──────────────────┼──────────────────┼──────────────────┤
80 │CMAKE_IN‐ │ Installation │ System-dependent │ /usr/local/lib, │
81 │STALL_PREFIX │ prefix │ │ ... │
82 ├─────────────────┼──────────────────┼──────────────────┼──────────────────┤
83 │HUGE_FUZZ │ Fuzz test with │ OFF │ ON, OFF │
84 │ │ 8GB of data │ │ │
85 ├─────────────────┼──────────────────┼──────────────────┼──────────────────┤
86 │SANE_MALLOC │ Assume malloc │ OFF │ ON, OFF │
87 │ │ will refuse un‐ │ │ │
88 │ │ reasonable allo‐ │ │ │
89 │ │ cations │ │ │
90 ├─────────────────┼──────────────────┼──────────────────┼──────────────────┤
91 │COVERAGE │ Generate test │ OFF │ ON, OFF │
92 │ │ coverage instru‐ │ │ │
93 │ │ mentation │ │ │
94 ├─────────────────┼──────────────────┼──────────────────┼──────────────────┤
95 │WITH_TESTS │ Build unit tests │ OFF │ ON, OFF │
96 │ │ (see develop‐ │ │ │
97 │ │ ment) │ │ │
98 └─────────────────┴──────────────────┴──────────────────┴──────────────────┘
99 The following configuration options will also be defined as macros[#]_
101 in <cbor/common.h> and can therefore be used in client code:
102 ┌────────────────────┬──────────────────┬─────────┬─────────────────┐
104 │Option │ Meaning │ Default │ Possible values │
105 ├────────────────────┼──────────────────┼─────────┼─────────────────┤
106 │CBOR_CUSTOM_AL‐ │ Enable custom │ OFF │ ON, OFF │
107 │LOC │ allocator sup‐ │ │ │
108 │ │ port │ │ │
109 ├────────────────────┼──────────────────┼─────────┼─────────────────┤
110 │CBOR_PRETTY_PRINTER │ Include a │ ON │ ON, OFF │
111 │ │ pretty-printing │ │ │
112 │ │ routine │ │ │
113 ├────────────────────┼──────────────────┼─────────┼─────────────────┤
114 │CBOR_BUFFER_GROWTH │ Factor for buf‐ │ 2 │ Decimals > 1 │
115 │ │ fer growth & │ │ │
116 │ │ shrinking │ │ │
117 └────────────────────┴──────────────────┴─────────┴─────────────────┘
118 [1] ON & OFF will be translated to 1 and 0 using cmakedefine.
120 If you want to pass other custom configuration options, please re‐
122 fer to http://www.cmake.org/Wiki/CMake_Useful_Variables.
123 Building using make
125 CMake will generate a Makefile and other configuration files for
127 the build. As a rule of thumb, you should configure the build out‐
128 side of the source tree in order to keep different configurations
129 isolated. If you are unsure where to execute the build, just use a
130 temporary directory:
131 cd $(mktemp -d /tmp/cbor_build.XXXX)
133 Now, assuming you are in the directory where you want to build, execute
135 the following to configure the build and run make
136 cmake -DCMAKE_BUILD_TYPE=Release path_to_libcbor_dir
138 make cbor cbor_shared
139 Both the shared (libcbor.so) and the static (libcbor.a) libraries
141 should now be in the src subdirectory.
142 In order to install the libcbor headers and libraries, the usual
144 make install
146 is what your're looking for. Root permissions are required on most sys‐
148 tems when using the default installation prefix.
149 Portability
151 libcbor is highly portable and works on both little- and big-endian
153 systems regardless of the operating system. After building on an exotic
154 platform, you might wish to verify the result by running the test
155 suite. If you encounter any problems, please report them to the issue
156 tracker.
157 libcbor is known to successfully work on ARM Android devices.
159 Cross-compilation is possible with arm-linux-gnueabi-gcc.
160 Linking with libcbor
162 If you include and linker paths include the directories to which libc‐
163 bor has been installed, compiling programs that uses libcbor requires
164 no extra considerations.
165 You can verify that everything has been set up properly by creating a
167 file with the following contents
168 #include <cbor.h>
170 #include <stdio.h>
171 int main(int argc, char * argv[])
173 {
174 printf("Hello from libcbor %s\n", CBOR_VERSION);
175 }
176 and compiling it
178 cc hello_cbor.c -lcbor -o hello_cbor
180 libcbor also comes with pkg-config support. If you install libcbor with
182 a custom prefix, you can use pkg-config to resolve the headers and ob‐
183 jects:
184 cc $(pkg-config --cflags libcbor) hello_cbor.c $(pkg-config --libs libcbor) -o hello_cbor
186 A note on linkage
188 libcbor is primarily intended to be linked statically. The shared li‐
190 brary versioning scheme generally follows SemVer, but is irregular for
191 the 0.X.Y development branch for historical reasons. The following ver‐
192 sion identifiers are used as a part of the SONAME (Linux) or the dylib
193 "Compatibility version" (OS X):
194 • 0.Y for the 0.Y.Z branch. Patches are backwards compatible, minor
196 releases are generally not and require re-compilation of any de‐
197 pendent code.
198 • X for the X.Y.Z stable versions starting 1.X.Y. All minor release
200 of the major version are backwards compatible.
201 WARNING:
203 Please note that releases up to and including v0.6.0 may export mis‐
204 leading .so/.dylib version number.
205 MinGW build instructions
207 Prerequisites:
208 • MinGW
210 • CMake GUI
212 First of all, create a folder that will be used for the output. For
214 this demonstration, we will use cbor_out. Start CMake and select the
215 source path and the destination folder. [image]
216 Then hit the 'Configure' button. You will be prompted to select the
218 build system: [image]
219 Choose MinGW and confirm.
221 NOTE:
223 If you select Visual Studio at this point, a MSVC project will be
224 generated for you. This is useful if you just want to browse through
225 the source code.
226 You can then adjust the build options. The defaults will work just
228 fine. Hit 'Generate' when you are done. [image]
229 You can then adjust the build options. The defaults will work just
231 fine. Hit 'Generate' when you are done.
232 Open the shell, navigate to the output directory, and run mingw32-make
234 cbor cbor_shared. [image]
235 libcbor will be built and your .dll should be ready at this point [im‐
237 age]
238 Feel free to also try building and running some of the examples, e.g.
240 mingw32-make sort [image]
241 Troubleshooting
243 cbor.h not found: The headers directory is probably not in your include
244 path. First, verify the installation location by checking the installa‐
245 tion log. If you used make, it will look something like
246 ...
248 -- Installing: /usr/local/include/cbor
249 -- Installing: /usr/local/include/cbor/callbacks.h
250 -- Installing: /usr/local/include/cbor/encoding.h
251 ...
252 Make sure that CMAKE_INSTALL_PREFIX (if you provided it) was correct.
254 Including the path path during compilation should suffice, e.g.:
255 cc -I/usr/local/include hello_cbor.c -lcbor -o hello_cbor
257 cannot find -lcbor during linking: Most likely the same problem as be‐
259 fore. Include the installation directory in the linker shared path us‐
260 ing -R, e.g.:
261 cc -Wl,-rpath,/usr/local/lib -lcbor -o hello_cbor
263 shared library missing during execution: Verify the linkage using ldd,
265 otool, or similar and adjust the compilation directives accordingly:
266 ⇒ ldd hello_cbor
268 linux-vdso.so.1 => (0x00007ffe85585000)
269 libcbor.so => /usr/local/lib/libcbor.so (0x00007f9af69da000)
270 libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f9af65eb000)
271 /lib64/ld-linux-x86-64.so.2 (0x00007f9af6be9000)
272 compilation failed: If your compiler supports C99 yet the compilation
274 has failed, please report the issue to the issue tracker.
275 Usage & preliminaries
277 Version information
278 libcbor exports its version using three self-explanatory macros:
279 • CBOR_MAJOR_VERSION
281 • CBOR_MINOR_VERSION
283 • CBOR_PATCH_VERSION
285 The CBOR_VERSION is a string concatenating these three identifiers into
287 one (e.g. 0.2.0).
288 In order to simplify version comparisons, the version is also exported
290 as
291 #define CBOR_HEX_VERSION ((CBOR_MAJOR_VERSION << 16) | (CBOR_MINOR_VERSION << 8) | CBOR_PATCH_VERSION)
293 Since macros are difficult to work with through FFIs, the same informa‐
295 tion is also available through three uint8_t constants, namely
296 • cbor_major_version
298 • cbor_minor_version
300 • cbor_patch_version
302 Headers to include
304 The cbor.h header includes all the symbols. If, for any reason, you
305 don't want to include all the exported symbols, feel free to use just
306 some of the cbor/*.h headers:
307 • cbor/arrays.h - api/type_4
309 • cbor/bytestrings.h - api/type_2
311 • cbor/callbacks.h - Callbacks used for streaming/decoding
313 • cbor/common.h - Common utilities - always transitively included
315 • cbor/data.h - Data types definitions - always transitively in‐
317 cluded
318 • cbor/encoding.h - Streaming encoders for streaming/encoding
320 • cbor/floats_ctrls.h - api/type_7
322 • cbor/ints.h - api/type_0_1
324 • cbor/maps.h - api/type_5
326 • cbor/serialization.h - High level serialization such as cbor_seri‐
328 alize()
329 • cbor/streaming.h - Home of cbor_stream_decode()
331 • cbor/strings.h - api/type_3
333 • cbor/tags.h - api/type_6
335 Using libcbor
337 If you want to get more familiar with CBOR, we recommend the cbor.io
338 website. Once you get the grasp of what is it CBOR does, the examples
339 (located in the examples directory) should give you a good feel of the
340 API. The API documentation should then provide with all the information
341 you may need.
342 Creating and serializing items
344 #include "cbor.h"
346 #include <stdio.h>
347 int main(int argc, char * argv[])
349 {
350 /* Preallocate the map structure */
351 cbor_item_t * root = cbor_new_definite_map(2);
352 /* Add the content */
353 cbor_map_add(root, (struct cbor_pair) {
354 .key = cbor_move(cbor_build_string("Is CBOR awesome?")),
355 .value = cbor_move(cbor_build_bool(true))
356 });
357 cbor_map_add(root, (struct cbor_pair) {
358 .key = cbor_move(cbor_build_uint8(42)),
359 .value = cbor_move(cbor_build_string("Is the answer"))
360 });
361 /* Output: `length` bytes of data in the `buffer` */
362 unsigned char * buffer;
363 size_t buffer_size, length = cbor_serialize_alloc(root, &buffer, &buffer_size);
364 fwrite(buffer, 1, length, stdout);
366 free(buffer);
367 fflush(stdout);
369 cbor_decref(&root);
370 }
371 Reading serialized data
373 #include "cbor.h"
375 #include <stdio.h>
376 /*
378 * Reads data from a file. Example usage:
379 * $ ./examples/readfile examples/data/nested_array.cbor
380 */
381 int main(int argc, char * argv[])
383 {
384 FILE * f = fopen(argv[1], "rb");
385 fseek(f, 0, SEEK_END);
386 size_t length = (size_t)ftell(f);
387 fseek(f, 0, SEEK_SET);
388 unsigned char * buffer = malloc(length);
389 fread(buffer, length, 1, f);
390 /* Assuming `buffer` contains `info.st_size` bytes of input data */
392 struct cbor_load_result result;
393 cbor_item_t * item = cbor_load(buffer, length, &result);
394 /* Pretty-print the result */
395 cbor_describe(item, stdout);
396 fflush(stdout);
397 /* Deallocate the result */
398 cbor_decref(&item);
399 fclose(f);
401 }
402 Using the streaming parser
404 #include "cbor.h"
406 #include <stdio.h>
407 #include <string.h>
408 /*
410 * Illustrates how one might skim through a map (which is assumed to have
411 * string keys and values only), looking for the value of a specific key
412 *
413 * Use the examples/data/map.cbor input to test this.
414 */
415 const char * key = "a secret key";
417 bool key_found = false;
418 void find_string(void * _ctx, cbor_data buffer, size_t len)
420 {
421 if (key_found) {
422 printf("Found the value: %*s\n", (int) len, buffer);
423 key_found = false;
424 } else if (len == strlen(key)) {
425 key_found = (memcmp(key, buffer, len) == 0);
426 }
427 }
428 int main(int argc, char * argv[])
430 {
431 FILE * f = fopen(argv[1], "rb");
432 fseek(f, 0, SEEK_END);
433 size_t length = (size_t)ftell(f);
434 fseek(f, 0, SEEK_SET);
435 unsigned char * buffer = malloc(length);
436 fread(buffer, length, 1, f);
437 struct cbor_callbacks callbacks = cbor_empty_callbacks;
439 struct cbor_decoder_result decode_result;
440 size_t bytes_read = 0;
441 callbacks.string = find_string;
442 while (bytes_read < length) {
443 decode_result = cbor_stream_decode(buffer + bytes_read,
444 length - bytes_read,
445 &callbacks, NULL);
446 bytes_read += decode_result.read;
447 }
448 fclose(f);
450 }
451 API
453 The data API is centered around cbor_item_t, a generic handle for any
454 CBOR item. There are functions to
455 • create items,
457 • set items' data,
459 • parse serialized data into items,
461 • manage, move, and links item together.
463 The single most important thing to keep in mind is: cbor_item_t is an
465 opaque type and should only be manipulated using the appropriate func‐
466 tions! Think of it as an object.
467 The libcbor API closely follows the semantics outlined by CBOR stan‐
469 dard. This part of the documentation provides a short overview of the
470 CBOR constructs, as well as a general introduction to the libcbor API.
471 Remaining reference can be found in the following files structured by
472 data types.
473 The API is designed to allow both very tight control & flexibility and
475 general convenience with sane defaults. [1] For example, client with
476 very specific requirements (constrained environment, custom application
477 protocol built on top of CBOR, etc.) may choose to take full control
478 (and responsibility) of memory and data structures management by inter‐
479 acting directly with the decoder. Other clients might want to take con‐
480 trol of specific aspects (streamed collections, hash maps storage), but
481 leave other responsibilities to libcbor. More general clients might
482 prefer to be abstracted away from all aforementioned details and only
483 be presented complete data structures.
484 libcbor provides
486 • stateless encoders and decoders
488 • encoding and decoding drivers, routines that coordinate encod‐
490 ing and decoding of complex structures
491 • data structures to represent and transform CBOR structures
493 • routines for building and manipulating these structures
495 • utilities for inspection and debugging
497 Types of items
499 Every cbor_item_t has a cbor_type associated with it - these constants
500 correspond to the types specified by the CBOR standard:
501 enum cbor_type
503 Specifies the Major type of cbor_item_t.
504 Values:
506 enumerator CBOR_TYPE_UINT
508 0 - positive integers
509 enumerator CBOR_TYPE_NEGINT
511 1 - negative integers
512 enumerator CBOR_TYPE_BYTESTRING
514 2 - byte strings
515 enumerator CBOR_TYPE_STRING
517 3 - strings
518 enumerator CBOR_TYPE_ARRAY
520 4 - arrays
521 enumerator CBOR_TYPE_MAP
523 5 - maps
524 enumerator CBOR_TYPE_TAG
526 6 - tags
527 enumerator CBOR_TYPE_FLOAT_CTRL
529 7 - decimals and special values (true, false, nil, ...)
530 To find out the type of an item, one can use
532 cbor_type cbor_typeof(const cbor_item_t *item)
534 Get the type of the item.
535 param item[borrow]
537 return The type
540 Please note the distinction between functions like cbor_isa_uint() and
542 cbor_is_int(). The following functions work solely with the major type
543 value.
544 Binary queries
546 Alternatively, there are functions to query each particular type.
547 WARNING:
549 Passing an invalid cbor_item_t reference to any of these functions
550 results in undefined behavior.
551 bool cbor_isa_uint(const cbor_item_t *item)
553 Does the item have the appropriate major type?
554 param item[borrow]
556 the item
557 return Is the item an CBOR_TYPE_UINT?
559 bool cbor_isa_negint(const cbor_item_t *item)
561 Does the item have the appropriate major type?
562 param item[borrow]
564 the item
565 return Is the item a CBOR_TYPE_NEGINT?
567 bool cbor_isa_bytestring(const cbor_item_t *item)
569 Does the item have the appropriate major type?
570 param item[borrow]
572 the item
573 return Is the item a CBOR_TYPE_BYTESTRING?
575 bool cbor_isa_string(const cbor_item_t *item)
577 Does the item have the appropriate major type?
578 param item[borrow]
580 the item
581 return Is the item a CBOR_TYPE_STRING?
583 bool cbor_isa_array(const cbor_item_t *item)
585 Does the item have the appropriate major type?
586 param item[borrow]
588 the item
589 return Is the item an CBOR_TYPE_ARRAY?
591 bool cbor_isa_map(const cbor_item_t *item)
593 Does the item have the appropriate major type?
594 param item[borrow]
596 the item
597 return Is the item a CBOR_TYPE_MAP?
599 bool cbor_isa_tag(const cbor_item_t *item)
601 Does the item have the appropriate major type?
602 param item[borrow]
604 the item
605 return Is the item a CBOR_TYPE_TAG?
607 bool cbor_isa_float_ctrl(const cbor_item_t *item)
609 Does the item have the appropriate major type?
610 param item[borrow]
612 the item
613 return Is the item a CBOR_TYPE_FLOAT_CTRL?
615 Logical queries
617 These functions provide information about the item type from a more
618 high-level perspective
619 bool cbor_is_int(const cbor_item_t *item)
621 Is the item an integer, either positive or negative?
622 param item[borrow]
624 the item
625 return Is the item an integer, either positive or negative?
627 bool cbor_is_float(const cbor_item_t *item)
629 Is the item an a floating point number?
630 param item[borrow]
632 the item
633 return Is the item a floating point number?
635 bool cbor_is_bool(const cbor_item_t *item)
637 Is the item an a boolean?
638 param item[borrow]
640 the item
641 return Is the item a boolean?
643 bool cbor_is_null(const cbor_item_t *item)
645 Does this item represent null
646 null pointer will most likely result in a crash.
648 WARNING:
652 This is in no way related to the value of the pointer. Pass‐
653 ing a
654 param item[borrow]
656 the item
657 return Is the item (CBOR logical) null?
659 bool cbor_is_undef(const cbor_item_t *item)
661 Does this item represent undefined
662 C.
665 WARNING:
669 Care must be taken to distinguish nulls and undefined values
670 in
671 param item[borrow]
673 the item
674 return Is the item (CBOR logical) undefined?
676 Memory management and reference counting
678 Due to the nature of its domain, libcbor will need to work with heap
679 memory. The stateless decoder and encoder don't allocate any memory.
680 If you have specific requirements, you should consider rolling your own
682 driver for the stateless API.
683 Using custom allocator
685 libcbor gives you with the ability to provide your own implementations
686 of malloc, realloc, and free. This can be useful if you are using a
687 custom allocator throughout your application, or if you want to imple‐
688 ment custom policies (e.g. tighter restrictions on the amount of allo‐
689 cated memory).
690 In order to use this feature, libcbor has to be compiled with the ap‐
692 propriate flags. You can verify the configuration using the CBOR_CUS‐
693 TOM_ALLOC macro. A simple usage might be as follows:
694 #if CBOR_CUSTOM_ALLOC
696 cbor_set_allocs(malloc, realloc, free);
697 #else
698 #error "libcbor built with support for custom allocation is required"
699 #endif
700 void cbor_set_allocs(_cbor_malloc_t custom_malloc, _cbor_realloc_t cus‐
702 tom_realloc, _cbor_free_t custom_free)
703 Sets the memory management routines to use.
704 Only available when CBOR_CUSTOM_ALLOC is truthy
706 used accordingly. Changing the memory handlers while allocated
708 items exist will result in a free/malloc mismatch. This function
709 is not thread safe with respect to both itself and all the other
710 libcbor functions that work with the heap. .. note:: realloc
711 implementation must correctly support NULL reallocation (see
712 e.g. http://en.cppreference.com/w/c/memory/realloc)
713 WARNING:
716 This function modifies the global state and should therefore
717 be
718 param custom_malloc
720 malloc implementation
721 param custom_realloc
723 realloc implementation
724 param custom_free
726 free implementation
727 Reference counting
729 As CBOR items may require complex cleanups at the end of their life‐
730 time, there is a reference counting mechanism in place. This also en‐
731 ables very simple GC when integrating libcbor into managed environment.
732 Every item starts its life (by either explicit creation, or as a result
733 of parsing) with reference count set to 1. When the refcount reaches
734 zero, it will be destroyed.
735 Items containing nested items will be destroyed recursively - refcount
737 of every nested item will be decreased by one.
738 The destruction is synchronous and renders any pointers to items with
740 refcount zero invalid immediately after calling the cbor_decref().
741 cbor_item_t *cbor_incref(cbor_item_t *item)
743 Increases the reference count by one.
744 No dependent items are affected.
746 param item[incref]
748 item the item
749 return the input reference
751 void cbor_decref(cbor_item_t **item)
753 Decreases the reference count by one, deallocating the item if
754 needed.
755 In case the item is deallocated, the reference count of any de‐
757 pendent items is adjusted accordingly in a recursive manner.
758 param item[take]
760 the item. Set to NULL if deallocated
761 void cbor_intermediate_decref(cbor_item_t *item)
763 Decreases the reference count by one, deallocating the item if
764 needed.
765 Convenience wrapper for cbor_decref when its set-to-null behav‐
767 ior is not needed
768 param item[take]
770 the item
771 size_t cbor_refcount(const cbor_item_t *item)
773 Get the reference count.
774 WARNING:
776 This does not account for transitive references.
777 param item[borrow]
779 the item
780 return the reference count
782 cbor_item_t *cbor_move(cbor_item_t *item)
784 Provides CPP-like move construct.
785 Decreases the reference count by one, but does not deallocate
787 the item even if its refcount reaches zero. This is useful for
788 passing intermediate values to functions that increase reference
789 count. Should only be used with functions that incref their ar‐
790 guments.
791 count afterwards, the memory will be leaked.
793 WARNING:
796 If the item is moved without correctly increasing the refer‐
797 ence
798 param item[take]
800 the item
801 return the item with reference count decreased by one
803 cbor_item_t *cbor_copy(cbor_item_t *item)
805 Deep copy of an item.
806 All the reference counts in the new structure are set to one.
808 param item[borrow]
810 item to copy
811 return new CBOR deep copy
813 Decoding
815 The following diagram illustrates the relationship among different
816 parts of libcbor from the decoding standpoint.
817 ┌──────────────────────────────────────────────────────────────────────────────────────────────┐
819 │ │
820 │ Client application │
821 │ │
822 │ ┌────────────────────────────────────────────┘
823 │ │ ↕
824 │ │ ┌──────────────────────────────────────────┐
825 │ │ │ │
826 │ │ │ Manipulation routines │
827 │ │ │ │
828 │ ┌─────────────────────────────────────┘ └──────────────────────────────────────────┘
829 │ │ ↑ ↑ ↑ ↑
830 │ │ │ │ ┌─────────────╫──────────┬───────────────────┴─┐
831 │ │ │ CDS │ ║ │ │
832 │ │ │ │ PDS ║ PDS PDS
833 │ │ ↓ ↓ ↓ ↓ ↓ ↓
834 │ │ ┌─────────────────┐ ┌────────────────────┐ ┌────────────────────────────┐
835 │ │ │ │ │ │ │ │
836 │ │ │ Custom driver │ ↔ │ Streaming driver │ ↔ │ Default driver │ ↔ CD
837 │ │ │ │ │ │ │ │
838 └───────────┘ └─────────────────┘ └────────────────────┘ └────────────────────────────┘
839 ↕ ↕ ↕ ↕
840 ┌──────────────────────────────────────────────────────────────────────────────────────────────┐
841 │ │
842 │ Stateless event─driven decoder │
843 │ │
844 └──────────────────────────────────────────────────────────────────────────────────────────────┘
845 (PSD = Provided Data Structures, CDS = Custom Data Structures)
847 This section will deal with the API that is labeled as the "Default
849 driver" in the diagram. That is, routines that decode complete libcbor
850 data items
851 cbor_item_t *cbor_load(cbor_data source, size_t source_size, struct
853 cbor_load_result *result)
854 Loads data item from a buffer.
855 param source
857 The buffer
858 param source_size
860 param result[out]
863 Result indicator. CBOR_ERR_NONE on success
864 return new CBOR item or NULL on failure. In that case, result
866 contains location and description of the error.
867 Associated data structures
869 enum cbor_error_code
870 Possible decoding errors.
871 Values:
873 enumerator CBOR_ERR_NONE
875 enumerator CBOR_ERR_NOTENOUGHDATA
877 enumerator CBOR_ERR_NODATA
879 enumerator CBOR_ERR_MALFORMATED
881 enumerator CBOR_ERR_MEMERROR
883 Memory error - item allocation failed.
884 Is it too big for your allocator?
886 enumerator CBOR_ERR_SYNTAXERROR
888 Stack parsing algorithm failed.
889 struct cbor_load_result
891 High-level decoding result.
892 Public Members
894 struct cbor_error error
896 Error indicator.
897 size_t read
899 Number of bytes read.
900 struct cbor_error
902 High-level decoding error.
903 Public Members
905 size_t position
907 Aproximate position.
908 cbor_error_code code
910 Description.
911 Encoding
913 The easiest way to encode data items is using the cbor_serialize() or
914 cbor_serialize_alloc() functions:
915 size_t cbor_serialize(const cbor_item_t *item, cbor_mutable_data buf‐
917 fer, size_t buffer_size)
918 Serialize the given item.
919 param item[borrow]
921 A data item
922 param buffer
924 Buffer to serialize to
925 param buffer_size
927 Size of the buffer
928 return Length of the result. 0 on failure.
930 size_t cbor_serialize_alloc(const cbor_item_t *item, cbor_mutable_data
932 *buffer, size_t *buffer_size)
933 Serialize the given item, allocating buffers as needed.
934 appropriate free implementation.
936 WARNING:
939 It is your responsibility to free the buffer using an
940 param item[borrow]
942 A data item
943 param buffer[out]
945 Buffer containing the result
946 param buffer_size[out]
948 Size of the buffer
949 return Length of the result. 0 on failure, in which case buffer
951 is NULL.
952 Type-specific serializers
954 In case you know the type of the item you want to serialize beforehand,
955 you can use one of the type-specific serializers.
956 NOTE:
958 Unless compiled in debug mode, these do not verify the type. Passing
959 an incorrect item will result in an undefined behavior.
960 size_t cbor_serialize_uint(const cbor_item_t *item, cbor_mutable_data
962 buffer, size_t buffer_size)
963 Serialize an uint.
964 param item[borrow]
966 A uint
967 param buffer
969 Buffer to serialize to
970 param buffer_size
972 Size of the buffer
973 return Length of the result. 0 on failure.
975 size_t cbor_serialize_negint(const cbor_item_t *item, cbor_mutable_data
977 buffer, size_t buffer_size)
978 Serialize a negint.
979 param item[borrow]
981 A neging
982 param buffer
984 Buffer to serialize to
985 param buffer_size
987 Size of the buffer
988 return Length of the result. 0 on failure.
990 size_t cbor_serialize_bytestring(const cbor_item_t *item, cbor_muta‐
992 ble_data buffer, size_t buffer_size)
993 Serialize a bytestring.
994 param item[borrow]
996 A bytestring
997 param buffer
999 Buffer to serialize to
1000 param buffer_size
1002 Size of the buffer
1003 return Length of the result. 0 on failure.
1005 size_t cbor_serialize_string(const cbor_item_t *item, cbor_mutable_data
1007 buffer, size_t buffer_size)
1008 Serialize a string.
1009 param item[borrow]
1011 A string
1012 param buffer
1014 Buffer to serialize to
1015 param buffer_size
1017 Size of the buffer
1018 return Length of the result. 0 on failure.
1020 size_t cbor_serialize_array(const cbor_item_t *item, cbor_mutable_data
1022 buffer, size_t buffer_size)
1023 Serialize an array.
1024 param item[borrow]
1026 An array
1027 param buffer
1029 Buffer to serialize to
1030 param buffer_size
1032 Size of the buffer
1033 return Length of the result. 0 on failure.
1035 size_t cbor_serialize_map(const cbor_item_t *item, cbor_mutable_data
1037 buffer, size_t buffer_size)
1038 Serialize a map.
1039 param item[borrow]
1041 A map
1042 param buffer
1044 Buffer to serialize to
1045 param buffer_size
1047 Size of the buffer
1048 return Length of the result. 0 on failure.
1050 size_t cbor_serialize_tag(const cbor_item_t *item, cbor_mutable_data
1052 buffer, size_t buffer_size)
1053 Serialize a tag.
1054 param item[borrow]
1056 A tag
1057 param buffer
1059 Buffer to serialize to
1060 param buffer_size
1062 Size of the buffer
1063 return Length of the result. 0 on failure.
1065 size_t cbor_serialize_float_ctrl(const cbor_item_t *item, cbor_muta‐
1067 ble_data buffer, size_t buffer_size)
1068 Serialize a.
1069 param item[borrow]
1071 A float or ctrl
1072 param buffer
1074 Buffer to serialize to
1075 param buffer_size
1077 Size of the buffer
1078 return Length of the result. 0 on failure.
1080 Types 0 & 1 – Positive and negative integers
1082 CBOR has two types of integers – positive (which may be effectively re‐
1083 garded as unsigned), and negative. There are four possible widths for
1084 an integer – 1, 2, 4, or 8 bytes. These are represented by
1085 enum cbor_int_width
1087 Possible widths of CBOR_TYPE_UINT items.
1088 Values:
1090 enumerator CBOR_INT_8
1092 enumerator CBOR_INT_16
1094 enumerator CBOR_INT_32
1096 enumerator CBOR_INT_64
1098 Type 0 - positive integers
1100 ┌────────────────────────┬────────────────────────────┐
1101 │Corresponding cbor_type │ CBOR_TYPE_UINT │
1102 ├────────────────────────┼────────────────────────────┤
1103 │Number of allocations │ One per lifetime │
1104 ├────────────────────────┼────────────────────────────┤
1105 │Storage requirements │ sizeof(cbor_item_t) + │
1106 │ │ sizeof(uint*_t) │
1107 └────────────────────────┴────────────────────────────┘
1108 Note: once a positive integer has been created, its width cannot be
1110 changed.
1111 Type 1 - negative integers
1113 ┌────────────────────────┬────────────────────────────┐
1114 │Corresponding cbor_type │ CBOR_TYPE_NEGINT │
1115 ├────────────────────────┼────────────────────────────┤
1116 │Number of allocations │ One per lifetime │
1117 ├────────────────────────┼────────────────────────────┤
1118 │Storage requirements │ sizeof(cbor_item_t) + │
1119 │ │ sizeof(uint*_t) │
1120 └────────────────────────┴────────────────────────────┘
1121 Note: once a positive integer has been created, its width cannot be
1123 changed.
1124 Type 0 & 1
1126 Due to their largely similar semantics, the following functions can be
1127 used for both Type 0 and Type 1 items. One can convert between them
1128 freely using the conversion functions.
1129 Actual Type of the integer can be checked using item types API.
1131 An integer item is created with one of the four widths. Because inte‐
1133 gers' storage is bundled together with the handle, the width cannot be
1134 changed over its lifetime.
1135 WARNING:
1137 Due to the fact that CBOR negative integers represent integers in
1138 the range [-1, -2^N], cbor_set_uint API is somewhat counter-intu‐
1139 itive as the resulting logical value is 1 less. This behavior is
1140 necessary in order to permit uniform manipulation with the full
1141 range of permitted values. For example, the following snippet
1142 cbor_item_t * item = cbor_new_int8();
1144 cbor_mark_negint(item);
1145 cbor_set_uint8(0);
1146 will produce an item with the logical value of -1. There is, how‐
1148 ever, an upside to this as well: There is only one representation of
1149 zero.
1150 Building new items
1152 cbor_item_t *cbor_build_uint8(uint8_t value)
1153 Constructs a new positive integer.
1154 param value
1156 the value to use
1157 return new positive integer or NULL on memory allocation failure
1159 cbor_item_t *cbor_build_uint16(uint16_t value)
1161 Constructs a new positive integer.
1162 param value
1164 the value to use
1165 return new positive integer or NULL on memory allocation failure
1167 cbor_item_t *cbor_build_uint32(uint32_t value)
1169 Constructs a new positive integer.
1170 param value
1172 the value to use
1173 return new positive integer or NULL on memory allocation failure
1175 cbor_item_t *cbor_build_uint64(uint64_t value)
1177 Constructs a new positive integer.
1178 param value
1180 the value to use
1181 return new positive integer or NULL on memory allocation failure
1183 Retrieving values
1185 uint8_t cbor_get_uint8(const cbor_item_t *item)
1186 Extracts the integer value.
1187 param item[borrow]
1189 positive or negative integer
1190 return the value
1192 uint16_t cbor_get_uint16(const cbor_item_t *item)
1194 Extracts the integer value.
1195 param item[borrow]
1197 positive or negative integer
1198 return the value
1200 uint32_t cbor_get_uint32(const cbor_item_t *item)
1202 Extracts the integer value.
1203 param item[borrow]
1205 positive or negative integer
1206 return the value
1208 uint64_t cbor_get_uint64(const cbor_item_t *item)
1210 Extracts the integer value.
1211 param item[borrow]
1213 positive or negative integer
1214 return the value
1216 Setting values
1218 void cbor_set_uint8(cbor_item_t *item, uint8_t value)
1219 Assigns the integer value.
1220 param item[borrow]
1222 positive or negative integer item
1223 param value
1225 the value to assign. For negative integer, the logical
1226 value is -value - 1
1227 void cbor_set_uint16(cbor_item_t *item, uint16_t value)
1229 Assigns the integer value.
1230 param item[borrow]
1232 positive or negative integer item
1233 param value
1235 the value to assign. For negative integer, the logical
1236 value is -value - 1
1237 void cbor_set_uint32(cbor_item_t *item, uint32_t value)
1239 Assigns the integer value.
1240 param item[borrow]
1242 positive or negative integer item
1243 param value
1245 the value to assign. For negative integer, the logical
1246 value is -value - 1
1247 void cbor_set_uint64(cbor_item_t *item, uint64_t value)
1249 Assigns the integer value.
1250 param item[borrow]
1252 positive or negative integer item
1253 param value
1255 the value to assign. For negative integer, the logical
1256 value is -value - 1
1257 Dealing with width
1259 cbor_int_width cbor_int_get_width(const cbor_item_t *item)
1260 Queries the integer width.
1261 param item[borrow]
1263 positive or negative integer item
1264 return the width
1266 Dealing with signedness
1268 void cbor_mark_uint(cbor_item_t *item)
1269 Marks the integer item as a positive integer.
1270 The data value is not changed
1272 param item[borrow]
1274 positive or negative integer item
1275 void cbor_mark_negint(cbor_item_t *item)
1277 Marks the integer item as a negative integer.
1278 The data value is not changed
1280 param item[borrow]
1282 positive or negative integer item
1283 Creating new items
1285 cbor_item_t *cbor_new_int8()
1286 Allocates new integer with 1B width.
1287 The width cannot be changed once allocated
1289 return new positive integer or NULL on memory allocation fail‐
1291 ure. The value is not initialized
1292 cbor_item_t *cbor_new_int16()
1294 Allocates new integer with 2B width.
1295 The width cannot be changed once allocated
1297 return new positive integer or NULL on memory allocation fail‐
1299 ure. The value is not initialized
1300 cbor_item_t *cbor_new_int32()
1302 Allocates new integer with 4B width.
1303 The width cannot be changed once allocated
1305 return new positive integer or NULL on memory allocation fail‐
1307 ure. The value is not initialized
1308 cbor_item_t *cbor_new_int64()
1310 Allocates new integer with 8B width.
1311 The width cannot be changed once allocated
1313 return new positive integer or NULL on memory allocation fail‐
1315 ure. The value is not initialized
1316 Type 2 – Byte strings
1318 CBOR byte strings are just (ordered) series of bytes without further
1319 interpretation (unless there is a tag). Byte string's length may or may
1320 not be known during encoding. These two kinds of byte strings can be
1321 distinguished using cbor_bytestring_is_definite() and
1322 cbor_bytestring_is_indefinite() respectively.
1323 In case a byte string is indefinite, it is encoded as a series of defi‐
1325 nite byte strings. These are called "chunks". For example, the encoded
1326 item
1327 0xf5 Start indefinite byte string
1329 0x41 Byte string (1B long)
1330 0x00
1331 0x41 Byte string (1B long)
1332 0xff
1333 0xff "Break" control token
1334 represents two bytes, 0x00 and 0xff. This on one hand enables streaming
1336 messages even before they are fully generated, but on the other hand it
1337 adds more complexity to the client code.
1338 ┌───────────────────────────┬────────────────────────────┐
1340 │Corresponding cbor_type │ CBOR_TYPE_BYTESTRING │
1341 ├───────────────────────────┼────────────────────────────┤
1342 │Number of allocations │ One plus any manipulations │
1343 │(definite) │ with the data │
1344 ├───────────────────────────┼────────────────────────────┤
1345 │Number of allocations (in‐ │ One plus logarithmically │
1346 │definite) │ many reallocations rela‐ │
1347 │ │ tive to chunk count │
1348 ├───────────────────────────┼────────────────────────────┤
1349 │Storage requirements (def‐ │ sizeof(cbor_item_t) + │
1350 │inite) │ length(handle) │
1351 ├───────────────────────────┼────────────────────────────┤
1352 │Storage requirements (in‐ │ sizeof(cbor_item_t) * (1 + │
1353 │definite) │ chunk_count) + chunks │
1354 └───────────────────────────┴────────────────────────────┘
1355 Streaming indefinite byte strings
1357 Please refer to /streaming.
1358 Getting metadata
1360 size_t cbor_bytestring_length(const cbor_item_t *item)
1361 Returns the length of the binary data.
1362 For definite byte strings only
1364 param item[borrow]
1366 a definite bytestring
1367 return length of the binary data. Zero if no chunk has been at‐
1369 tached yet
1370 bool cbor_bytestring_is_definite(const cbor_item_t *item)
1372 Is the byte string definite?
1373 param item[borrow]
1375 a byte string
1376 return Is the byte string definite?
1378 bool cbor_bytestring_is_indefinite(const cbor_item_t *item)
1380 Is the byte string indefinite?
1381 param item[borrow]
1383 a byte string
1384 return Is the byte string indefinite?
1386 size_t cbor_bytestring_chunk_count(const cbor_item_t *item)
1388 Get the number of chunks this string consist of.
1389 param item[borrow]
1391 A indefinite bytestring
1392 return The chunk count. 0 for freshly created items.
1394 Reading data
1396 cbor_mutable_data cbor_bytestring_handle(const cbor_item_t *item)
1397 Get the handle to the binary data.
1398 Definite items only. Modifying the data is allowed. In that
1400 case, the caller takes responsibility for the effect on items
1401 this item might be a part of
1402 param item[borrow]
1404 A definite byte string
1405 return The address of the binary data. NULL if no data have been
1407 assigned yet.
1408 cbor_item_t **cbor_bytestring_chunks_handle(const cbor_item_t *item)
1410 Get the handle to the array of chunks.
1411 Manipulations with the memory block (e.g. sorting it) are al‐
1413 lowed, but the validity and the number of chunks must be re‐
1414 tained.
1415 param item[borrow]
1417 A indefinite byte string
1418 return array of cbor_bytestring_chunk_count definite bytestrings
1420 Creating new items
1422 cbor_item_t *cbor_new_definite_bytestring()
1423 Creates a new definite byte string.
1424 The handle is initialized to NULL and length to 0
1426 return new definite bytestring. NULL on malloc failure.
1428 cbor_item_t *cbor_new_indefinite_bytestring()
1430 Creates a new indefinite byte string.
1431 The chunks array is initialized to NULL and chunkcount to 0
1433 return new indefinite bytestring. NULL on malloc failure.
1435 Building items
1437 cbor_item_t *cbor_build_bytestring(cbor_data handle, size_t length)
1438 Creates a new byte string and initializes it.
1439 The handle will be copied to a newly allocated block
1441 param handle
1443 Block of binary data
1444 param length
1446 Length of data
1447 return A new byte string with content handle. NULL on malloc
1449 failure.
1450 Manipulating existing items
1452 void cbor_bytestring_set_handle(cbor_item_t *item, cbor_mutable_data
1453 data, size_t length)
1454 Set the handle to the binary data.
1455 param item[borrow]
1457 A definite byte string
1458 param data
1460 The memory block. The caller gives up the ownership of
1461 the block. libcbor will deallocate it when appropriate
1462 using its free function
1463 param length
1465 Length of the data block
1466 bool cbor_bytestring_add_chunk(cbor_item_t *item, cbor_item_t *chunk)
1468 Appends a chunk to the bytestring.
1469 Indefinite byte strings only.
1471 May realloc the chunk storage.
1473 param item[borrow]
1475 An indefinite byte string
1476 param item[incref]
1478 A definite byte string
1479 return true on success, false on realloc failure. In that case,
1481 the refcount of chunk is not increased and the item is
1482 left intact.
1483 Type 3 – UTF-8 strings
1485 CBOR strings work in much the same ways as type_2.
1486 ┌───────────────────────────┬────────────────────────────┐
1488 │Corresponding cbor_type │ CBOR_TYPE_STRING │
1489 ├───────────────────────────┼────────────────────────────┤
1490 │Number of allocations │ One plus any manipulations │
1491 │(definite) │ with the data │
1492 ├───────────────────────────┼────────────────────────────┤
1493 │Number of allocations (in‐ │ One plus logarithmically │
1494 │definite) │ many reallocations rela‐ │
1495 │ │ tive to chunk count │
1496 ├───────────────────────────┼────────────────────────────┤
1497 │Storage requirements (def‐ │ sizeof(cbor_item_t) + │
1498 │inite) │ length(handle) │
1499 ├───────────────────────────┼────────────────────────────┤
1500 │Storage requirements (in‐ │ sizeof(cbor_item_t) * (1 + │
1501 │definite) │ chunk_count) + chunks │
1502 └───────────────────────────┴────────────────────────────┘
1503 Streaming indefinite strings
1505 Please refer to /streaming.
1506 UTF-8 encoding validation
1508 libcbor considers UTF-8 encoding validity to be a part of the
1509 well-formedness notion of CBOR and therefore invalid UTF-8 strings will
1510 be rejected by the parser. Strings created by the user are not checked.
1511 Getting metadata
1513 size_t cbor_string_length(const cbor_item_t *item)
1514 Returns the length of the underlying string.
1515 For definite strings only
1517 param item[borrow]
1519 a definite string
1520 return length of the string. Zero if no chunk has been attached
1522 yet
1523 bool cbor_string_is_definite(const cbor_item_t *item)
1525 Is the string definite?
1526 param item[borrow]
1528 a string
1529 return Is the string definite?
1531 bool cbor_string_is_indefinite(const cbor_item_t *item)
1533 Is the string indefinite?
1534 param item[borrow]
1536 a string
1537 return Is the string indefinite?
1539 size_t cbor_string_chunk_count(const cbor_item_t *item)
1541 Get the number of chunks this string consist of.
1542 param item[borrow]
1544 A indefinite string
1545 return The chunk count. 0 for freshly created items.
1547 Reading data
1549 cbor_mutable_data cbor_string_handle(const cbor_item_t *item)
1550 Get the handle to the underlying string.
1551 Definite items only. Modifying the data is allowed. In that
1553 case, the caller takes responsibility for the effect on items
1554 this item might be a part of
1555 param item[borrow]
1557 A definite string
1558 return The address of the underlying string. NULL if no data
1560 have been assigned yet.
1561 cbor_item_t **cbor_string_chunks_handle(const cbor_item_t *item)
1563 Get the handle to the array of chunks.
1564 Manipulations with the memory block (e.g. sorting it) are al‐
1566 lowed, but the validity and the number of chunks must be re‐
1567 tained.
1568 param item[borrow]
1570 A indefinite string
1571 return array of cbor_string_chunk_count definite strings
1573 Creating new items
1575 cbor_item_t *cbor_new_definite_string()
1576 Creates a new definite string.
1577 The handle is initialized to NULL and length to 0
1579 return new definite string. NULL on malloc failure.
1581 cbor_item_t *cbor_new_indefinite_string()
1583 Creates a new indefinite string.
1584 The chunks array is initialized to NULL and chunkcount to 0
1586 return new indefinite string. NULL on malloc failure.
1588 Building items
1590 cbor_item_t *cbor_build_string(const char *val)
1591 Creates a new string and initializes it.
1592 The val will be copied to a newly allocated block
1594 param val
1596 A null-terminated UTF-8 string
1597 return A new string with content handle. NULL on malloc failure.
1599 Manipulating existing items
1601 void cbor_string_set_handle(cbor_item_t *item, cbor_mutable_data data,
1602 size_t length)
1603 Set the handle to the underlying string.
1604 mistake. Lifetime of the string will expire when it goes out of
1606 scope and the CBOR item will be left inconsistent.
1607 WARNING:
1610 Using a pointer to a stack allocated constant is a common
1611 param item[borrow]
1613 A definite string
1614 param data
1616 The memory block. The caller gives up the ownership of
1617 the block. libcbor will deallocate it when appropriate
1618 using its free function
1619 param length
1621 Length of the data block
1622 bool cbor_string_add_chunk(cbor_item_t *item, cbor_item_t *chunk)
1624 Appends a chunk to the string.
1625 Indefinite strings only.
1627 May realloc the chunk storage.
1629 param item[borrow]
1631 An indefinite string
1632 param item[incref]
1634 A definite string
1635 return true on success. false on realloc failure. In that case,
1637 the refcount of chunk is not increased and the item is
1638 left intact.
1639 Type 4 – Arrays
1641 CBOR arrays, just like byte strings and strings, can be encoded either
1642 as definite, or as indefinite.
1643 ┌───────────────────────────┬────────────────────────────┐
1645 │Corresponding cbor_type │ CBOR_TYPE_ARRAY │
1646 ├───────────────────────────┼────────────────────────────┤
1647 │Number of allocations │ Two plus any manipulations │
1648 │(definite) │ with the data │
1649 ├───────────────────────────┼────────────────────────────┤
1650 │Number of allocations (in‐ │ Two plus logarithmically │
1651 │definite) │ many reallocations rela‐ │
1652 │ │ tive to additions │
1653 ├───────────────────────────┼────────────────────────────┤
1654 │Storage requirements (def‐ │ (sizeof(cbor_item_t) + 1) │
1655 │inite) │ * size │
1656 ├───────────────────────────┼────────────────────────────┤
1657 │Storage requirements (in‐ │ <= sizeof(cbor_item_t) + │
1658 │definite) │ sizeof(cbor_item_t) * size │
1659 │ │ * BUFFER_GROWTH │
1660 └───────────────────────────┴────────────────────────────┘
1661 Examples
1663 0x9f Start indefinite array
1664 0x01 Unsigned integer 1
1665 0xff "Break" control token
1666 0x9f Start array, 1B length follows
1668 0x20 Unsigned integer 32
1669 ... 32 items follow
1670 Streaming indefinite arrays
1672 Please refer to /streaming.
1673 Getting metadata
1675 size_t cbor_array_size(const cbor_item_t *item)
1676 Get the number of members.
1677 param item[borrow]
1679 An array
1680 return The number of members
1682 size_t cbor_array_allocated(const cbor_item_t *item)
1684 Get the size of the allocated storage.
1685 param item[borrow]
1687 An array
1688 return The size of the allocated storage (number of items)
1690 bool cbor_array_is_definite(const cbor_item_t *item)
1692 Is the array definite?
1693 param item[borrow]
1695 An array
1696 return Is the array definite?
1698 bool cbor_array_is_indefinite(const cbor_item_t *item)
1700 Is the array indefinite?
1701 param item[borrow]
1703 An array
1704 return Is the array indefinite?
1706 Reading data
1708 cbor_item_t **cbor_array_handle(const cbor_item_t *item)
1709 Get the array contents.
1710 The items may be reordered and modified as long as references
1712 remain consistent.
1713 param item[borrow]
1715 An array
1716 return cbor_array_size items
1718 cbor_item_t *cbor_array_get(const cbor_item_t *item, size_t index)
1720 Get item by index.
1721 param item[borrow]
1723 An array
1724 param index
1726 The index
1727 return incref The item, or NULL in case of boundary violation
1729 Creating new items
1731 cbor_item_t *cbor_new_definite_array(size_t size)
1732 Create new definite array.
1733 param size
1735 Number of slots to preallocate
1736 return new array or NULL upon malloc failure
1738 cbor_item_t *cbor_new_indefinite_array()
1740 Create new indefinite array.
1741 return new array or NULL upon malloc failure
1743 Modifying items
1745 bool cbor_array_push(cbor_item_t *array, cbor_item_t *pushee)
1746 Append to the end.
1747 For indefinite items, storage may be realloacted. For definite
1749 items, only the preallocated capacity is available.
1750 param array[borrow]
1752 An array
1753 param pushee[incref]
1755 The item to push
1756 return true on success, false on failure
1758 bool cbor_array_replace(cbor_item_t *item, size_t index, cbor_item_t
1760 *value)
1761 Replace item at an index.
1762 The item being replace will be cbor_decref 'ed.
1764 param item[borrow]
1766 An array
1767 param value[incref]
1769 The item to assign
1770 param index
1772 The index, first item is 0.
1773 return true on success, false on allocation failure.
1775 bool cbor_array_set(cbor_item_t *item, size_t index, cbor_item_t
1777 *value)
1778 Set item by index.
1779 Creating arrays with holes is not possible
1781 param item[borrow]
1783 An array
1784 param value[incref]
1786 The item to assign
1787 param index
1789 The index, first item is 0.
1790 return true on success, false on allocation failure.
1792 Type 5 – Maps
1794 CBOR maps are the plain old associate hash maps known from JSON and
1795 many other formats and languages, with one exception: any CBOR data
1796 item can be a key, not just strings. This is somewhat unusual and you,
1797 as an application developer, should keep that in mind.
1798 Maps can be either definite or indefinite, in much the same way as
1800 type_4.
1801 ┌───────────────────────────┬────────────────────────────┐
1803 │Corresponding cbor_type │ CBOR_TYPE_MAP │
1804 ├───────────────────────────┼────────────────────────────┤
1805 │Number of allocations │ Two plus any manipulations │
1806 │(definite) │ with the data │
1807 ├───────────────────────────┼────────────────────────────┤
1808 │Number of allocations (in‐ │ Two plus logarithmically │
1809 │definite) │ many reallocations rela‐ │
1810 │ │ tive to additions │
1811 ├───────────────────────────┼────────────────────────────┤
1812 │Storage requirements (def‐ │ sizeof(cbor_pair) * size + │
1813 │inite) │ sizeof(cbor_item_t) │
1814 ├───────────────────────────┼────────────────────────────┤
1815 │Storage requirements (in‐ │ <= sizeof(cbor_item_t) + │
1816 │definite) │ sizeof(cbor_pair) * size * │
1817 │ │ BUFFER_GROWTH │
1818 └───────────────────────────┴────────────────────────────┘
1819 Streaming maps
1821 Please refer to /streaming.
1822 Getting metadata
1824 size_t cbor_map_size(const cbor_item_t *item)
1825 Get the number of pairs.
1826 param item[borrow]
1828 A map
1829 return The number of pairs
1831 size_t cbor_map_allocated(const cbor_item_t *item)
1833 Get the size of the allocated storage.
1834 param item[borrow]
1836 A map
1837 return Allocated storage size (as the number of cbor_pair items)
1839 bool cbor_map_is_definite(const cbor_item_t *item)
1841 Is this map definite?
1842 param item[borrow]
1844 A map
1845 return Is this map definite?
1847 bool cbor_map_is_indefinite(const cbor_item_t *item)
1849 Is this map indefinite?
1850 param item[borrow]
1852 A map
1853 return Is this map indefinite?
1855 Reading data
1857 struct cbor_pair *cbor_map_handle(const cbor_item_t *item)
1858 Get the pairs storage.
1859 param item[borrow]
1861 A map
1862 return Array of cbor_map_size pairs. Manipulation is possible as
1864 long as references remain valid.
1865 Creating new items
1867 cbor_item_t *cbor_new_definite_map(size_t size)
1868 Create a new definite map.
1869 param size
1871 The number of slots to preallocate
1872 return new definite map. NULL on malloc failure.
1874 cbor_item_t *cbor_new_indefinite_map()
1876 Create a new indefinite map.
1877 param size
1879 The number of slots to preallocate
1880 return new definite map. NULL on malloc failure.
1882 Modifying items
1884 bool cbor_map_add(cbor_item_t *item, struct cbor_pair pair)
1885 Add a pair to the map.
1886 For definite maps, items can only be added to the preallocated
1888 space. For indefinite maps, the storage will be expanded as
1889 needed
1890 param item[borrow]
1892 A map
1893 param pair[incref]
1895 The key-value pair to add (incref is member-wise)
1896 return true on success, false if either reallocation failed or
1898 the preallcoated storage is full
1899 Type 6 – Semantic tags
1901 Tag are additional metadata that can be used to extend or specialize
1902 the meaning or interpretation of the other data items.
1903 For example, one might tag an array of numbers to communicate that it
1905 should be interpreted as a vector.
1906 Please consult the official IANA repository of CBOR tags before invent‐
1908 ing new ones.
1909 ┌────────────────────────┬────────────────────────────┐
1911 │Corresponding cbor_type │ CBOR_TYPE_TAG │
1912 ├────────────────────────┼────────────────────────────┤
1913 │Number of allocations │ One plus any manipulations │
1914 │ │ with the data realloca‐ │
1915 │ │ tions relative to chunk │
1916 │ │ count │
1917 ├────────────────────────┼────────────────────────────┤
1918 │Storage requirements │ sizeof(cbor_item_t) + the │
1919 │ │ tagged item │
1920 └────────────────────────┴────────────────────────────┘
1921 cbor_item_t *cbor_new_tag(uint64_t value)
1923 Create a new tag.
1924 param value
1926 The tag value. Please consult the tag repository
1927 return new tag. Item reference is NULL. Returns NULL upon memory
1929 allocation failure
1930 cbor_item_t *cbor_tag_item(const cbor_item_t *item)
1932 Get the tagged item.
1933 param item[borrow]
1935 A tag
1936 return incref the tagged item
1938 uint64_t cbor_tag_value(const cbor_item_t *item)
1940 Get tag value.
1941 param item[borrow]
1943 A tag
1944 return The tag value. Please consult the tag repository
1946 void cbor_tag_set_item(cbor_item_t *item, cbor_item_t *tagged_item)
1948 Set the tagged item.
1949 param item[borrow]
1951 A tag
1952 param tagged_item[incref]
1954 The item to tag
1955 Type 7 – Floats & control tokens
1957 This type combines two completely unrelated types of items -- floating
1958 point numbers and special values such as true, false, null, etc. We re‐
1959 fer to these special values as 'control values' or 'ctrls' for short
1960 throughout the code.
1961 Just like integers, they have different possible width (resulting in
1963 different value ranges and precisions).
1964 enum cbor_float_width
1966 Possible widths of CBOR_TYPE_FLOAT_CTRL items.
1967 Values:
1969 enumerator CBOR_FLOAT_0
1971 Internal use - ctrl and special values.
1972 enumerator CBOR_FLOAT_16
1974 Half float.
1975 enumerator CBOR_FLOAT_32
1977 Single float.
1978 enumerator CBOR_FLOAT_64
1980 Double.
1981 ┌────────────────────────┬────────────────────────────┐
1983 │Corresponding cbor_type │ CBOR_TYPE_FLOAT_CTRL │
1984 ├────────────────────────┼────────────────────────────┤
1985 │Number of allocations │ One per lifetime │
1986 ├────────────────────────┼────────────────────────────┤
1987 │Storage requirements │ sizeof(cbor_item_t) + │
1988 │ │ 1/4/8 │
1989 └────────────────────────┴────────────────────────────┘
1990 Getting metadata
1992 bool cbor_float_ctrl_is_ctrl(const cbor_item_t *item)
1993 Is this a ctrl value?
1994 param item[borrow]
1996 A float or ctrl item
1997 return Is this a ctrl value?
1999 cbor_float_width cbor_float_get_width(const cbor_item_t *item)
2001 Get the float width.
2002 param item[borrow]
2004 A float or ctrl item
2005 return The width.
2007 Reading data
2009 float cbor_float_get_float2(const cbor_item_t *item)
2010 Get a half precision float.
2011 The item must have the corresponding width
2013 param [borrow]
2015 A half precision float
2016 return half precision value
2018 float cbor_float_get_float4(const cbor_item_t *item)
2020 Get a single precision float.
2021 The item must have the corresponding width
2023 param [borrow]
2025 A signle precision float
2026 return single precision value
2028 double cbor_float_get_float8(const cbor_item_t *item)
2030 Get a double precision float.
2031 The item must have the corresponding width
2033 param [borrow]
2035 A double precision float
2036 return double precision value
2038 double cbor_float_get_float(const cbor_item_t *item)
2040 Get the float value represented as double.
2041 Can be used regardless of the width.
2043 param [borrow]
2045 Any float
2046 return double precision value
2048 uint8_t cbor_ctrl_value(const cbor_item_t *item)
2050 Reads the control value.
2051 param item[borrow]
2053 A ctrl item
2054 return the simple value
2056 bool cbor_get_bool(const cbor_item_t *item)
2058 Get value from a boolean ctrl item.
2059 param item[borrow]
2061 A ctrl item
2062 return boolean value
2064 Creating new items
2066 cbor_item_t *cbor_new_ctrl()
2067 Constructs a new ctrl item.
2068 The width cannot be changed once the item is created
2070 return new 1B ctrl or NULL upon memory allocation failure
2072 cbor_item_t *cbor_new_float2()
2074 Constructs a new float item.
2075 The width cannot be changed once the item is created
2077 return new 2B float or NULL upon memory allocation failure
2079 cbor_item_t *cbor_new_float4()
2081 Constructs a new float item.
2082 The width cannot be changed once the item is created
2084 return new 4B float or NULL upon memory allocation failure
2086 cbor_item_t *cbor_new_float8()
2088 Constructs a new float item.
2089 The width cannot be changed once the item is created
2091 return new 8B float or NULL upon memory allocation failure
2093 cbor_item_t *cbor_new_null()
2095 Constructs new null ctrl item.
2096 return new null ctrl item or NULL upon memory allocation failure
2098 cbor_item_t *cbor_new_undef()
2100 Constructs new undef ctrl item.
2101 return new undef ctrl item or NULL upon memory allocation fail‐
2103 ure
2104 Building items
2106 cbor_item_t *cbor_build_bool(bool value)
2107 Constructs new boolean ctrl item.
2108 param value
2110 The value to use
2111 return new boolen ctrl item or NULL upon memory allocation fail‐
2113 ure
2114 cbor_item_t *cbor_build_ctrl(uint8_t value)
2116 Constructs a ctrl item.
2117 param value
2119 the value to use
2120 return new ctrl item or NULL upon memory allocation failure
2122 cbor_item_t *cbor_build_float2(float value)
2124 Constructs a new float.
2125 param value
2127 the value to use
2128 return new float
2130 cbor_item_t *cbor_build_float4(float value)
2132 Constructs a new float.
2133 param value
2135 the value to use
2136 return new float or NULL upon memory allocation failure
2138 cbor_item_t *cbor_build_float8(double value)
2140 Constructs a new float.
2141 param value
2143 the value to use
2144 return new float or NULL upon memory allocation failure
2146 Manipulating existing items
2148 void cbor_set_ctrl(cbor_item_t *item, uint8_t value)
2149 Assign a control value.
2150 invalid value using this mechanism. Please consult the standard
2152 before use.
2153 WARNING:
2156 It is possible to produce an invalid CBOR value by assigning
2157 a
2158 param item[borrow]
2160 A ctrl item
2161 param value
2163 The simple value to assign. Please consult the standard
2164 for allowed values
2165 void cbor_set_bool(cbor_item_t *item, bool value)
2167 Assign a boolean value to a boolean ctrl item.
2168 param item[borrow]
2170 A ctrl item
2171 param value
2173 The simple value to assign.
2174 void cbor_set_float2(cbor_item_t *item, float value)
2176 Assigns a float value.
2177 param item[borrow]
2179 A half precision float
2180 param value
2182 The value to assign
2183 void cbor_set_float4(cbor_item_t *item, float value)
2185 Assigns a float value.
2186 param item[borrow]
2188 A single precision float
2189 param value
2191 The value to assign
2192 void cbor_set_float8(cbor_item_t *item, double value)
2194 Assigns a float value.
2195 param item[borrow]
2197 A double precision float
2198 param value
2200 The value to assign
2201 Half floats
2203 CBOR supports two bytes wide ("half-precision") floats which are not
2204 supported by the C language. libcbor represents them using float
2205 <https://en.cppreference.com/w/c/language/type> values throughout the
2206 API, which has important implications when manipulating these values.
2207 In particular, if a user uses some of the manipulation APIs (e.g.
2209 cbor_set_float2(), cbor_new_float2()) to introduce a value that doesn't
2210 have an exect half-float representation, the encoding semantics are
2211 given by cbor_encode_half() as follows:
2212 size_t cbor_encode_half(float value, unsigned char *buffer, size_t buf‐
2214 fer_size)
2215 Encodes a half-precision float.
2216 Since there is no native representation or semantics for half
2218 floats in the language, we use single-precision floats, as every
2219 value that can be expressed as a half-float can also be ex‐
2220 pressed as a float.
2221 This however means that not all floats passed to this function
2223 can be unambiguously encoded. The behavior is as follows:.INDENT
2224 7.0
2225 • Infinity, NaN are preserved
2227 • Zero is preserved
2229 • Denormalized numbers keep their sign bit and 10 most significant bit
2231 of the significand
2232 • All other numbers.INDENT 2.0
2234 • If the logical value of the exponent is < -24, the output is zero
2236 • If the logical value of the exponent is between -23 and -14, the out‐
2238 put is cut off to represent the 'magnitude' of the input, by which we
2239 mean (-1)^{signbit} x 1.0e{exponent}. The value in the significand is
2240 lost.
2241 • In all other cases, the sign bit, the exponent, and 10 most signifi‐
2243 cant bits of the significand are kept
2244 param value
2248 param buffer
2251 Target buffer
2252 param buffer_size
2254 Available space in the buffer
2255 return number of bytes written
2257 [1] http://softwareengineering.vazexqi.com/files/pattern.html
2259 Streaming & indefinite items
2261 CBOR strings, byte strings, arrays, and maps can be encoded as indefi‐
2262 nite, meaning their length or size is not specified. Instead, they are
2263 divided into chunks (strings, byte strings), or explicitly terminated
2264 (arrays, maps).
2265 This is one of the most important (and due to poor implementations, un‐
2267 derutilized) features of CBOR. It enables low-overhead streaming just
2268 about anywhere without dealing with channels or pub/sub mechanism. It
2269 is, however, important to recognize that CBOR streaming is not a sub‐
2270 stitute for Websockets [1] and similar technologies.
2271 [1] RFC 6455
2273 Decoding
2275 Another way to decode data using libcbor is to specify a callbacks that
2276 will be invoked when upon finding certain items in the input. This ser‐
2277 vice is provided by
2278 struct cbor_decoder_result cbor_stream_decode(cbor_data buffer, size_t
2280 buffer_size, const struct cbor_callbacks *callbacks, void *context)
2281 Stateless decoder.
2282 Will try parsing the buffer and will invoke the appropriate
2284 callback on success. Decodes one item at a time. No memory allo‐
2285 cations occur.
2286 param buffer
2288 Input buffer
2289 param buffer_size
2291 Length of the buffer
2292 param callbacks
2294 The callback bundle
2295 param context
2297 An arbitrary pointer to allow for maintaining context.
2298 To get started, you might want to have a look at the simple streaming
2300 example:
2301 #include "cbor.h"
2303 #include <stdio.h>
2304 #include <string.h>
2305 /*
2307 * Illustrates how one might skim through a map (which is assumed to have
2308 * string keys and values only), looking for the value of a specific key
2309 *
2310 * Use the examples/data/map.cbor input to test this.
2311 */
2312 const char * key = "a secret key";
2314 bool key_found = false;
2315 void find_string(void * _ctx, cbor_data buffer, size_t len)
2317 {
2318 if (key_found) {
2319 printf("Found the value: %*s\n", (int) len, buffer);
2320 key_found = false;
2321 } else if (len == strlen(key)) {
2322 key_found = (memcmp(key, buffer, len) == 0);
2323 }
2324 }
2325 int main(int argc, char * argv[])
2327 {
2328 FILE * f = fopen(argv[1], "rb");
2329 fseek(f, 0, SEEK_END);
2330 size_t length = (size_t)ftell(f);
2331 fseek(f, 0, SEEK_SET);
2332 unsigned char * buffer = malloc(length);
2333 fread(buffer, length, 1, f);
2334 struct cbor_callbacks callbacks = cbor_empty_callbacks;
2336 struct cbor_decoder_result decode_result;
2337 size_t bytes_read = 0;
2338 callbacks.string = find_string;
2339 while (bytes_read < length) {
2340 decode_result = cbor_stream_decode(buffer + bytes_read,
2341 length - bytes_read,
2342 &callbacks, NULL);
2343 bytes_read += decode_result.read;
2344 }
2345 free(buffer);
2347 fclose(f);
2348 }
2349 The callbacks are defined by
2351 struct cbor_callbacks
2353 Callback bundle passed to the decoder.
2354 Public Members
2356 cbor_int8_callback uint8
2358 Unsigned int.
2359 cbor_int16_callback uint16
2361 Unsigned int.
2362 cbor_int32_callback uint32
2364 Unsigned int.
2365 cbor_int64_callback uint64
2367 Unsigned int.
2368 cbor_int64_callback negint64
2370 Negative int.
2371 cbor_int32_callback negint32
2373 Negative int.
2374 cbor_int16_callback negint16
2376 Negative int.
2377 cbor_int8_callback negint8
2379 Negative int.
2380 cbor_simple_callback byte_string_start
2382 Definite byte string.
2383 cbor_string_callback byte_string
2385 Indefinite byte string start.
2386 cbor_string_callback string
2388 Definite string.
2389 cbor_simple_callback string_start
2391 Indefinite string start.
2392 cbor_simple_callback indef_array_start
2394 Definite array.
2395 cbor_collection_callback array_start
2397 Indefinite array.
2398 cbor_simple_callback indef_map_start
2400 Definite map.
2401 cbor_collection_callback map_start
2403 Indefinite map.
2404 cbor_int64_callback tag
2406 Tags.
2407 cbor_float_callback float2
2409 Half float.
2410 cbor_float_callback float4
2412 Single float.
2413 cbor_double_callback float8
2415 Double float.
2416 cbor_simple_callback undefined
2418 Undef.
2419 cbor_simple_callback null
2421 Null.
2422 cbor_bool_callback boolean
2424 Bool.
2425 cbor_simple_callback indef_break
2427 Indefinite item break.
2428 When building custom sets of callbacks, feel free to start from
2430 const struct cbor_callbacks cbor_empty_callbacks
2432 Dummy callback bundle - does nothing.
2433 Related structures
2435 enum cbor_decoder_status
2436 Streaming decoder result - status.
2437 Values:
2439 enumerator CBOR_DECODER_FINISHED
2441 OK, finished.
2442 enumerator CBOR_DECODER_NEDATA
2444 Not enough data - mismatch with MTB.
2445 enumerator CBOR_DECODER_EBUFFER
2447 Buffer manipulation problem.
2448 enumerator CBOR_DECODER_ERROR
2450 Malformed or reserved MTB/value.
2451 struct cbor_decoder_result
2453 Streaming decoder result.
2454 Public Members
2456 size_t read
2458 Bytes read.
2459 enum cbor_decoder_status status
2461 The result.
2462 size_t required
2464 When status == CBOR_DECODER_NEDATA, the minimum number of
2465 bytes required to continue parsing.
2466 Callback types definition
2468 typedef void (*cbor_int8_callback)(void*, uint8_t)
2469 Callback prototype.
2470 typedef void (*cbor_int16_callback)(void*, uint16_t)
2472 Callback prototype.
2473 typedef void (*cbor_int32_callback)(void*, uint32_t)
2475 Callback prototype.
2476 typedef void (*cbor_int64_callback)(void*, uint64_t)
2478 Callback prototype.
2479 typedef void (*cbor_simple_callback)(void*)
2481 Callback prototype.
2482 typedef void (*cbor_string_callback)(void*, cbor_data, size_t)
2484 Callback prototype.
2485 typedef void (*cbor_collection_callback)(void*, size_t)
2487 Callback prototype.
2488 typedef void (*cbor_float_callback)(void*, float)
2490 Callback prototype.
2491 typedef void (*cbor_double_callback)(void*, double)
2493 Callback prototype.
2494 typedef void (*cbor_bool_callback)(void*, bool)
2496 Callback prototype.
2497 Encoding
2499 TODO
2500 Tests
2502 Unit tests
2503 There is a comprehensive test suite employing CMocka. You can run all
2504 of them using ctest in the build directory. Individual tests are them‐
2505 selves runnable. Please refer to CTest documentation for detailed in‐
2506 formation on how to specify particular subset of tests.
2507 Testing for memory leaks
2509 Every release is tested for memory correctness. You can run these tests
2510 by passing the -T memcheck flag to ctest. [1]
2511 [1] Project should be configured with -DCMAKE_BUILD_TYPE=Debug to ob‐
2513 tain meaningful description of location of the leak. You might
2514 also need --dsymutil=yes on OS X.
2515 Code coverage
2517 Every release is inspected using GCOV/LCOV. Platform-independent code
2518 should be fully covered by the test suite. Simply run
2519 make coverage
2521 or alternatively run lcov by hand using
2523 lcov --capture --directory . --output-file coverage.info
2525 genhtml coverage.info --output-directory out
2526 Fuzz testing
2528 Every release is tested using a fuzz test. In this test, a huge buffer
2529 filled with random data is passed to the decoder. We require that it
2530 either succeeds or fail with a sensible error, without leaking any mem‐
2531 ory. This is intended to simulate real-world situations where data re‐
2532 ceived from the network are CBOR-decoded before any further processing.
2533 RFC conformance
2535 libcbor is, generally speaking, very faithful implementation of RFC
2536 7049. There are, however, some limitations imposed by technical con‐
2537 straints.
2538 Bytestring length
2540 There is no explicit limitation of indefinite length byte strings. [1]
2541 libcbor will not handle byte strings with more chunks than the maximum
2542 value of size_t. On any sane platform, such string would not fit in the
2543 memory anyway. It is, however, possible to process arbitrarily long
2544 strings and byte strings using the streaming decoder.
2545 [1] http://tools.ietf.org/html/rfc7049#section-2.2.2
2547 Half-precision IEEE 754 floats
2549 As of C99 and even C11, there is no standard implementation for 2 bytes
2550 floats. libcbor packs them as a float <https://en.cpprefer‐
2551 ence.com/w/c/language/type>. When encoding, libcbor selects the appro‐
2552 priate wire representation based on metadata and the actual value. This
2553 applies both to canonical and normal mode.
2554 For more information on half-float serialization, please refer to the
2556 section on api_type_7_hard_floats.
2557 Internal mechanics
2559 Internal workings of libcbor are mostly derived from the specification.
2560 The purpose of this document is to describe technical choices made dur‐
2561 ing design & implementation and to explicate the reasoning behind those
2562 choices.
2563 Terminology
2565 ┌────┬─────────────────────┬────────────────────────────────────────────────┐
2566 │MTB │ Major Type Byte │ http://tools.ietf.org/html/rfc7049#section-2.1 │
2567 ├────┼─────────────────────┼────────────────────────────────────────────────┤
2568 │DST │ Dynamically Sized │ Type whose storage requirements cannot be de‐ │
2569 │ │ Type │ termined │
2570 │ │ │ │
2571 │ │ │ during compilation (originated in the Rust │
2572 │ │ │ community) │
2573 └────┴─────────────────────┴────────────────────────────────────────────────┘
2574 Conventions
2576 API symbols start with cbor_ or CBOR_ prefix, internal symbols have
2577 _cbor_ or _CBOR_ prefix.
2578 Inspiration & related projects
2580 Most of the API is largely modelled after existing JSON libraries, in‐
2581 cluding
2582 • Jansson
2584 • json-c
2586 • Gnome's JsonGlib
2588 and also borrowing from
2590 • msgpack-c
2592 • Google Protocol Buffers.
2594 General notes on the API design
2596 The API design has two main driving priciples:
2597 1. Let the client manage the memory as much as possible
2599 2. Behave exactly as specified by the standard
2601 Combining these two principles in practice turns out to be quite diffi‐
2603 cult. Indefinite-length strings, arrays, and maps require client to
2604 handle every fixed-size chunk explicitly in order to
2605 • ensure the client never runs out of memory due to libcbor
2607 • use realloc() sparsely and predictably [1]
2609 • provide strong guarantees about its usage (to prevent la‐
2611 tency spikes)
2612 • provide APIs to avoid realloc() altogether
2614 • allow proper handling of (streamed) data bigger than available
2616 memory
2617 [1] Reasonable handling of DSTs requires reallocation if the API is
2619 to remain sane.
2620 Coding style
2622 This code loosely follows the Linux kernel coding style. Tabs are tabs,
2623 and they are 4 characters wide.
2624 Memory layout
2626 CBOR is very dynamic in the sense that it contains many data elements
2627 of variable length, sometimes even indefinite length. This section de‐
2628 scribes internal representation of all CBOR data types.
2629 Generally speaking, data items consist of three parts:
2631 • a generic handle,
2633 • the associated metadata,
2635 • and the actual data
2637 type cbor_item_t
2639 Represents the item. Used as an opaque type
2640 cbor_type type
2642 Type discriminator
2643 size_t refcount
2645 Reference counter. Used by cbor_decref(), cbor_incref()
2646 union cbor_item_metadata metadata
2648 Union discriminated by type. Contains type-specific meta‐
2649 data
2650 unsigned char *data
2652 Contains pointer to the actual data. Small, fixed size
2653 items (api/type_0_1, api/type_6, api/type_7) are allo‐
2654 cated as a single memory block.
2655 Consider the following snippet
2657 cbor_item_t * item = cbor_new_int8();
2659 then the memory is laid out as follows
2661 +-----------+---------------+---------------+-----------------------------------++-----------+
2663 | | | | || |
2664 | type | refcount | metadata | data || uint8_t |
2665 | | | | (= item + sizeof(cbor_item_t)) || |
2666 +-----------+---------------+---------------+-----------------------------------++-----------+
2667 ^ ^
2668 | |
2669 +--- item +--- item->data
2670 Dynamically sized types (api/type_2, api/type_3,
2672 api/type_4, api/type_5) may store handle and data in sep‐
2673 arate locations. This enables creating large items (e.g
2674 byte strings) without realloc() or copying large blocks
2675 of memory. One simply attaches the correct pointer to the
2676 handle.
2677 type cbor_item_metadata
2679 Union type of the following members, based on the item type:
2680 struct _cbor_int_metadata int_metadata
2682 Used both by both api/type_0_1
2683 struct _cbor_bytestring_metadata bytestring_metadata
2685 struct _cbor_string_metadata string_metadata
2687 struct _cbor_array_metadata array_metadata
2689 struct _cbor_map_metadata map_metadata
2691 struct _cbor_tag_metadata tag_metadata
2693 struct _cbor_float_ctrl_metadata float_ctrl_metadata
2695 Decoding
2697 As outlined in api, there decoding is based on the streaming decoder
2698 Essentially, the decoder is a custom set of callbacks for the streaming
2699 decoder.
2700 Changelog
2702 Next
2703 0.7.0 (2020-04-25)
2704 •
2705 Fix bad encoding of NaN half-floats [[Fixes #53]](‐
2707 https://github.com/PJK/libcbor/issues/53) (discovered by
2708 [BSipos-RKF](https://github.com/BSipos-RKF))
2709 • Warning: Previous versions encoded NaNs as 0xf9e700 instead
2711 of 0xf97e00; if you rely on the broken behavior, this will
2712 be a breaking change
2713 • Fix potentially bad encoding of negative half-float with exponent <
2715 -14 [[Fixes #112]](https://github.com/PJK/libcbor/issues/112) (dis‐
2716 covered by [yami36](https://github.com/yami36))
2717 •
2719 BREAKING: Improved bool support [[Fixes #63]](‐
2721 https://github.com/PJK/libcbor/issues/63)
2722 • Rename cbor_ctrl_is_bool to cbor_get_bool and fix the behav‐
2724 ior
2725 • Add cbor_set_bool
2727 • Fix memory_allocation_test breaking the build without CBOR_CUSTOM_AL‐
2729 LOC [[Fixes #128]](https://github.com/PJK/libcbor/issues/128) (by
2730 [panlinux](https://github.com/panlinux))
2731 • [Fix a potential build issue where cJSON includes may be
2733 misconfigured](https://github.com/PJK/libcbor/pull/132)
2734 •
2736 Breaking: [Add a limit on the size of the decoding context stack](‐
2738 https://github.com/PJK/libcbor/pull/138) (by [James-ZHANG](‐
2739 https://github.com/James-ZHANG))
2740 • If your usecase requires parsing very deeply nested struc‐
2742 tures, you might need to increase the default 2k limit via
2743 CBOR_MAX_STACK_SIZE
2744 •
2746 Enable LTO/IPO based on [CheckIPOSupported](‐
2748 https://cmake.org/cmake/help/latest/module/CheckIPOSup‐
2749 ported.html#module:CheckIPOSupported) [[#143]](‐
2750 https://github.com/PJK/libcbor/pull/143) (by [xanderlent](‐
2751 https://github.com/xanderlent))
2752 • If you rely on LTO being enabled and use CMake version older
2754 than 3.9, you will need to re-enable it manually or upgrade
2755 your CMake
2756 0.6.1 (2020-03-26)
2758 •
2759 [Fix bad shared library version number](‐
2761 https://github.com/PJK/libcbor/pull/131)
2762 • Warning: Shared library built from the 0.6.0 release is er‐
2764 roneously marked as version "0.6.0", which makes it incom‐
2765 patible with future releases including the v0.6.X line even
2766 though they may be compatible API/ABI-wise. Refer to the
2767 documentation for the new SO versioning scheme.
2768 0.6.0 (2020-03-15)
2770 •
2771 Correctly set .so version [[Fixes #52]](‐
2773 https://github.com/PJK/libcbor/issues/52).
2774 • Warning: All previous releases will be identified as 0.0 by
2776 the linker.
2777 • Fix & prevent heap overflow error in example code [[#74]](‐
2779 https://github.com/PJK/libcbor/pull/74) [[#76]](‐
2780 https://github.com/PJK/libcbor/pull/76) (by @nevun)
2781 • Correctly set OSX dynamic library version [[Fixes #75]](‐
2783 https://github.com/PJK/libcbor/issues/75)
2784 • [Fix misplaced 0xFF bytes in maps possibly causing memory
2786 corruption](https://github.com/PJK/libcbor/pull/82)
2787 • BREAKING: Fix handling & cleanup of failed memory allocation in con‐
2789 structor and builder helper functions [[Fixes #84]](‐
2790 https://github.com/PJK/libcbor/issues/84) - All cbor_new_* and
2791 cbor_build_* functions will now explicitly return NULL when memory
2792 allocation fails - It is up to the client to handle such cases
2793 • Globally enforced code style [[Fixes #83]](‐
2795 https://github.com/PJK/libcbor/issues/83)
2796 • Fix issue possible memory corruption bug on repeated
2798 cbor_(byte)string_add_chunk calls with intermittently failing realloc
2799 calls
2800 • Fix possibly misaligned reads and writes when endian.h is uses or
2802 when running on a big-endian machine [[Fixes #99](‐
2803 https://github.com/PJK/libcbor/issues/99), [#100](‐
2804 https://github.com/PJK/libcbor/issues/100)]
2805 • [Improved CI setup with Travis-native arm64 support](‐
2807 https://github.com/PJK/libcbor/pull/116)
2808 • [Docs migrated to Sphinx 2.4 and Python3](‐
2810 https://github.com/PJK/libcbor/pull/117)
2811 0.5.0 (2017-02-06)
2813 • Remove cmocka from the subtree (always rely on system or user-pro‐
2814 vided version)
2815 • Windows CI
2817 • Only build tests if explicitly enabled (-DWITH_TESTS=ON)
2819 • Fixed static header declarations (by cedric-d)
2821 • Improved documentation (by Michael Richardson)
2823 • Improved examples/readfile.c
2825 • Reworked (re)allocation to handle huge inputs and overflows in size_t
2827 [[Fixes #16]](https://github.com/PJK/libcbor/issues/16)
2828 • Improvements to C++ linkage (corrected cbor_empty_callbacks, fixed
2830 restrict pointers) (by Dennis Bijwaard)
2831 • Fixed Linux installation directory depending on architecture [[Fixes
2833 #34]](https://github.com/PJK/libcbor/issues/34) (by jvymazal)
2834 • Improved 32-bit support [[Fixes #35]](‐
2836 https://github.com/PJK/libcbor/issues/35)
2837 • Fixed MSVC compatibility [[Fixes #31]](‐
2839 https://github.com/PJK/libcbor/issues/31)
2840 • Fixed and improved half-float encoding [[Fixes #5](‐
2842 https://github.com/PJK/libcbor/issues/5), [#11](‐
2843 https://github.com/PJK/libcbor/issues/11)]
2844 0.4.0 (2015-12-25)
2846 Breaks build & header compatibility due to:
2847 • Improved build configuration and feature check macros
2849 • Endianess configuration fixes (by Erwin Kroon and David Grigsby)
2851 • pkg-config compatibility (by Vincent Bernat)
2853 • enable use of versioned SONAME (by Vincent Bernat)
2855 • better fuzzer (wasn't random until now, ooops)
2857 0.3.1 (2015-05-21)
2859 • documentation and comments improvements, mostly for the API reference
2860 0.3.0 (2015-05-21)
2862 • Fixes, polishing, niceties across the code base
2863 • Updated examples
2865 • cbor_copy
2867 • cbor_build_negint8, 16, 32, 64, matching asserts
2869 • cbor_build_stringn
2871 • cbor_build_tag
2873 • cbor_build_float2, ...
2875 0.2.1 (2015-05-17)
2877 • C99 support
2878 0.2.0 (2015-05-17)
2880 • cbor_ctrl_bool -> cbor_ctrl_is_bool
2881 • Added cbor_array_allocated & map equivalent
2883 • Overhauled endianess conversion - ARM now works as expected
2885 • 'sort.c' example added
2887 • Significantly improved and doxyfied documentation
2889 0.1.0 (2015-05-06)
2891 The initial release, yay!
2892 Development
2894 Vision and principles
2895 Consistency and coherence are one of the key characteristics of good
2896 software. While the reality is never black and white, it is important
2897 libcbor contributors are working towards the same high-level goal. This
2898 document attempts to set out the basic principles of libcbor and the
2899 rationale behind them. If you are contributing to libcbor or looking to
2900 evaluate whether libcbor is the right choice for your project, it might
2901 be worthwhile to skim through the section below.
2902 Mission statement
2904 libcbor is the compact, full-featured, and safe CBOR library that works
2905 everywhere.
2906 Goals
2908 RFC-conformance and full feature support
2909 Anything the standard allows, libcbor can do.
2910 Why? Because conformance and interoperability is the point of defining
2912 standards. Clients expect the support to be feature-complete and there
2913 is no significant complexity reduction that can be achieved by slightly
2914 cutting corners, which means that the incremental cost of full RFC sup‐
2915 port is comparatively small over "almost-conformance" seen in many al‐
2916 ternatives.
2917 Safety
2919 Untrusted bytes from the network are the typical input.
2920 Why? Because it is the client expectation. Vast majority of security
2922 vulnerabilities are violations of contracts -- in other words, bugs --
2923 anyway.
2924 Self-containment
2926 libcbor has no runtime dependencies.
2927 Why? Because any constraint imposed on libcbor has to be enforced tran‐
2929 sitively, which is difficult and leads to incompatibilities and distri‐
2930 bution issues, especially in IoT applications.
2931 Portability
2933 If you can compile C for it, libcbor will work there.
2934 Why? Lowest-common-denominator solution for system-level and IoT soft‐
2936 ware was the original niche of libcbor. Users who rely on libcbor ex‐
2937 pect future updates to work on their target platform.
2938 Stable and predictable API
2940 libcbor will not break without a warning.
2941 Why? Industry-standard versioning is a basic requirement for produc‐
2943 tion-quality software. This is especially relevant in IoT environments
2944 where updates may be costly.
2945 Performance
2947 libcbor is fast and resource-efficient by design
2948 Why? Because the main maintainer is an avid hater of slow bloated soft‐
2950 ware. Who wouldn't want more bang per their electricity buck?
2951 Non-goals
2953 • Convenience -- libcbor only provides the minimum surface to make
2954 it usable
2955 • FFI/SWIG/interop support -- libcbor is primarily a C library for C
2957 clients
2958 • One-off usecases support -- although there are primitives to re‐
2960 use, the basic assumption is that most clients want most of CBOR
2961 features
2962 Development dependencies
2964 • CMocka (testing)
2965 • Python and pip (Sphinx platform)
2967 • Doxygen
2969 • Sphinx (documentation)
2971 • There are some Ruby scripts in misc
2973 • Valgrind (memory correctness & profiling)
2975 • GCOV/LCOV (test coverage)
2977 • clang-format
2979 Installing sphinx
2981 pip install sphinx
2982 pip install sphinx_rtd_theme
2983 pip install breathe
2984 pip install https://github.com/lepture/python-livereload/archive/master.zip
2985 pip install sphinx-autobuild
2986 Further instructions on configuring advanced features can be found at
2988 http://read-the-docs.readthedocs.org/en/latest/install.html.
2989 Live preview of docs
2991 cd doc
2992 make livehtml
2993 Set up git hooks
2995 A catch-all git hook that runs clang-format and automatically refreshes
2996 the GH pages contents located in docs can be symlinked:
2997 ln -sf $(pwd)/misc/hooks/pre-commit .git/hooks
2999 Testing and code coverage
3001 Please refer to tests
3002
3004 Pavel Kalvoda
3005
3007 2021 - 2020, Pavel Kalvoda
30080.7 Jul 22, 2021 LIBCBOR(1)