1libnbd(3) LIBNBD libnbd(3)
2
6 libnbd - network block device (NBD) client library in userspace
7
9 #include <libnbd.h>
10 struct nbd_handle *nbd;
12 char buf[512];
13 if ((nbd = nbd_create ()) == NULL ||
15 nbd_connect_tcp (nbd, "server.example.com", "nbd") == -1 ||
16 nbd_pread (nbd, buf, sizeof buf, 0, 0) == -1)
17 fprintf (stderr, "%s\n", nbd_get_error ());
18 nbd_close (nbd);
19 exit (EXIT_FAILURE);
20 }
21 nbd_close (nbd);
22 cc prog.c -o prog -lnbd
24 or:
25 cc prog.c -o prog `pkg-config libnbd --cflags --libs`
26
28 Network Block Device (NBD) is a network protocol for accessing block
29 devices over the network. Block devices are hard disks and things that
30 behave like hard disks such as disk images and virtual machines.
31 Libnbd is a client library for the NBD protocol which can access most
33 of the features of NBD while being simple to use and powerful.
34 This manual page gives an overview of libnbd, using C as an example,
36 but the library is available from other programming languages.
37 nbd_create(3), nbd_pread(3), etc.
39 Each manual page covers one function from the C API in detail.
40 There is a full list in section "C API" below.
41 libnbd-ocaml(3)
43 Using the API from OCaml.
44 libnbd-golang(3)
46 Using the API from Go.
47 nbdsh(1)
49 Using the NBD shell (nbdsh) for command line and Python scripting.
50
52 To use the API at all you must first open a handle by calling
53 nbd_create(3) (or its equivalent in other languages):
54 struct nbd_handle *nbd;
56 nbd = nbd_create ();
58 This creates and returns a handle, which is associated with one
60 connection to an NBD server, initially not connected.
61 Each handle is a complex state machine which can be in states such as
63 created, connected to a remote server, handshaking, idle and ready to
64 issue commands, or busy sending or receiving commands.
65 Handles have a name used in debugging messages. The name is normally
67 generated ("nbd1", "nbd2" etc) but you can set a friendly name with
68 nbd_set_handle_name(3). Also there is a private field in the handle
69 for use by the application, see nbd_set_private_data(3).
70 When you have finished with the handle you must call nbd_close(3) which
72 closes the underlying socket (if necessary) and frees up all associated
73 resources.
74
76 There are two levels of API available. A simple high level synchronous
77 API lets you give the handle high level instructions like “connect to
78 the server”, “read a block”, “write a block”, etc. Each of these
79 functions will run to completion, blocking the current thread before
80 returning. A more complicated low level non-blocking asynchronous API
81 is also available where you can integrate with poll(2) or another main
82 loop.
83 You can freely mix the two APIs on the same handle. You can also call
85 APIs on a single handle from multiple threads. Single API calls on the
86 handle are atomic — they either take a lock on the handle while they
87 run or are careful to access handle fields atomically.
88 Libnbd does not create its own threads.
90
92 This is the simplest way to use the API, with the possible drawback
93 that each libnbd function blocks until it is finished.
94 Create a handle and connect to the server:
96 struct nbd_handle *nbd;
98 nbd = nbd_create ();
100 if (!nbd) {
101 fprintf (stderr, "%s\n", nbd_get_error ());
102 nbd_close (nbd);
103 exit (EXIT_FAILURE);
104 }
105 if (nbd_connect_tcp (nbd, "server.example.com", "nbd") == -1) {
106 fprintf (stderr, "%s\n", nbd_get_error ());
107 nbd_close (nbd);
108 exit (EXIT_FAILURE);
109 }
110 Read the first sector (512 bytes) from the NBD export:
112 char buf[512];
114 if (nbd_pread (nbd, buf, sizeof buf, 0, 0) == -1) {
116 fprintf (stderr, "%s\n", nbd_get_error ());
117 nbd_close (nbd);
118 exit (EXIT_FAILURE);
119 }
120 Close the handle:
122 nbd_close (nbd);
124 You can call the high level API from multiple threads, but each libnbd
126 API call takes a lock on the handle and so commands will not run in
127 parallel.
128
130 The low level API is useful if you want to use libnbd in non-blocking
131 code; or if you want to issue commands in parallel from multiple
132 threads; or if you need more control especially over having multiple
133 commands in-flight on a single connection.
134 To use the low level API you will need to integrate with poll(2) or
136 another “main loop” such as the GLib main event loop.
137 Issuing asynchronous commands
139 Use the "nbd_aio_*" variants to issue commands asynchronously (without
140 waiting for the command to complete before returning). For example the
141 asynchronous variant of nbd_pread(3) is:
142 int64_t cookie;
144 cookie = nbd_aio_pread (nbd, buf, sizeof buf,
146 NBD_NULL_COMPLETION, 0);
147 if (cookie == -1) {
148 fprintf (stderr, "%s\n", nbd_get_error ());
149 nbd_close (nbd);
150 exit (EXIT_FAILURE);
151 }
152 There are several things to note here:
154 • This only starts the command. The command is still in flight when
156 the call returns.
157 • A buffer ("buf") has been assigned to collect the result of the
159 read, but it is not guaranteed to be filled with data until the
160 command has completed (see examples below). The buffer must not be
161 freed until the command has finished running.
162 • You can issue multiple commands on the same handle at the same
164 time.
165 • A cookie is returned which identifies this command in subsequent
167 calls. The cookie is unique (per libnbd handle) and ≥ 1.
168 • You may register a function which is called when the command
170 completes, see "Completion callbacks" below. In this case we have
171 specified a null completion callback.
172 Socket and direction
174 Each libnbd handle has an associated socket (once it has started
175 connecting). You can read the file descriptor of the socket using:
176 int fd = nbd_aio_get_fd (nbd);
178 The socket is non-blocking. Between calls into libnbd it is in the
180 "would block" condition. You can find out if libnbd is expecting to
181 read or write from the socket next by calling:
182 int dir = nbd_aio_get_direction (nbd);
184 which returns one of "LIBNBD_AIO_DIRECTION_READ",
186 "LIBNBD_AIO_DIRECTION_WRITE" or "LIBNBD_AIO_DIRECTION_BOTH" (=
187 "READ|WRITE"). And so to set up the next call to poll(2) or other main
188 loop you must translate this to "POLLIN", "POLLOUT" or "POLLIN|POLLOUT"
189 (or whatever mechanism your main loop uses).
190 Notifying libnbd when an event happens
192 When you detect (eg. using poll(2)) that a read or write event has
193 happened on the socket, you must then tell libnbd about it. You have
194 to check the direction again (since it may have been changed by another
195 thread), and notify libnbd:
196 int r = 0;
198 dir = nbd_aio_get_direction (nbd);
200 if ((dir & LIBNBD_AIO_DIRECTION_READ) &&
202 a_read_event_occurred ())
203 r = nbd_aio_notify_read (nbd);
204 else if ((dir & LIBNBD_AIO_DIRECTION_WRITE) &&
205 a_write_event_occurred ())
206 r = nbd_aio_notify_write (nbd);
207 if (r == -1) {
209 fprintf (stderr, "%s\n", nbd_get_error ());
210 // ...
211 }
212 The notify calls move the state machine along, reading and writing from
214 the socket possibly multiple times, until the socket would block again,
215 at which point they return control to the caller.
216 Simple implementation with nbd_poll(3)
218 In fact if you want to use poll(2) on a single handle, a simple
219 implementation has already been written called nbd_poll(3). It is also
220 useful to examine how this is implemented (lib/poll.c in the libnbd
221 source code) because that will tell you how to integrate libnbd with
222 more complex main loops.
223 Some examples of using nbd_poll(3) follow.
225 As with the high level API, it all starts by creating a handle:
227 struct nbd_handle *nbd;
229 nbd = nbd_create ();
231 if (nbd == NULL) {
232 fprintf (stderr, "%s\n", nbd_get_error ());
233 nbd_close (nbd);
234 exit (EXIT_FAILURE);
235 }
236 To connect to the server asynchronously, we start the connection using
238 nbd_aio_connect(3) and then enter our main loop to check for events
239 until the connection becomes ready:
240 int fd;
242 struct sockaddr_un addr;
243 socklen_t len;
244 /* some code to set up addr,
246 then ... */
247 if (nbd_aio_connect (nbd, &addr, len) == -1) {
248 fprintf (stderr, "%s\n", nbd_get_error ());
249 nbd_close (nbd);
250 exit (EXIT_FAILURE);
251 }
252 while (! nbd_aio_is_ready (nbd)) {
253 if (nbd_poll (nbd, -1) == -1) {
254 fprintf (stderr, "%s\n", nbd_get_error ());
255 nbd_close (nbd);
256 exit (EXIT_FAILURE);
257 }
258 }
259 To read data asynchronously, start an asynchronous read command, which
261 returns a 64 bit command cookie, and enter the main loop until the
262 command has completed:
263 int64_t cookie;
265 char buf[512];
266 cookie = nbd_aio_pread (nbd, buf, sizeof buf, offset,
268 NBD_NULL_COMPLETION, 0);
269 if (cookie == -1) {
270 fprintf (stderr, "%s\n", nbd_get_error ());
271 nbd_close (nbd);
272 exit (EXIT_FAILURE);
273 }
274 while (! nbd_aio_command_completed (nbd, cookie)) {
275 if (nbd_poll (nbd, -1) == -1) {
276 fprintf (stderr, "%s\n", nbd_get_error ());
277 nbd_close (nbd);
278 exit (EXIT_FAILURE);
279 }
280 }
281 For almost all high level synchronous calls (eg. nbd_pread(3)) there is
283 a low level asynchronous equivalent (eg. nbd_aio_pread(3)) for starting
284 a command.
285 glib2 integration
287 See
288 https://gitlab.com/nbdkit/libnbd/blob/master/examples/glib-main-loop.c
289 libev integration
291 See https://gitlab.com/nbdkit/libnbd/blob/master/examples/copy-libev.c
292
294 When any API call returns an error ("-1" or "NULL" depending on the
295 API), an error message and sometimes an errno value are available. You
296 can retrieve the error message and/or errno of the most recently failed
297 call using nbd_get_error(3) and nbd_get_errno(3). For example:
298 if (nbd_connect_tcp (nbd, "remote", "nbd") == -1) {
300 fprintf (stderr,
301 "failed to connect to remote server: %s (errno = %d)\n",
302 nbd_get_error (), nbd_get_errno ());
303 }
304 These functions use thread-local storage to return the most recent
306 error in the current thread. This is why you don't need to pass the
307 handle to these calls. They even work if nbd_create(3) returns "NULL"
308 when there is no handle at all.
309 For this reason you cannot call them from a different thread. You
311 should call them immediately after the failed API call, from the same
312 thread. Furthermore the error string returned by nbd_get_error(3) is
313 only valid until the next libnbd API call in the current thread, so if
314 you need to keep the string you must copy it (eg. using strdup(3)).
315 Errno
317 For some errors, a system call error number (see errno(3)) is
318 available. You can find the error number by calling nbd_get_errno(3).
319 It works the same way as nbd_get_error(3) with respect to threads.
320 Even when a call returns an error, nbd_get_errno(3) might return 0.
322 This does not mean there was no error. It means no additional errno
323 information is available for this error.
324 The error number is often the raw error returned by a system call that
326 failed.
327 It can also be used to indicate special conditions. The most common
329 cases are:
330 "EINVAL"
332 Invalid parameters or state for the current libnbd call.
333 "ENOTSUP"
335 The libnbd call is not available in this build of libnbd (eg. when
336 using a TLS API if the library was compiled without TLS support).
337 "ENOMEM"
339 The library ran out of memory while performing some operation.
340 "ERANGE"
342 A request is too large, for example if you try to read too many
343 bytes in a single nbd_pread(3) call.
344
346 Libnbd can print lots of debugging messages, useful if you have a
347 problem with the library. Either enable debugging after creating the
348 handle:
349 nbd = nbd_create ();
351 nbd_set_debug (nbd, true);
352 or set the "LIBNBD_DEBUG=1" environment variable which will enable
354 debugging by default on all new handles.
355 Debugging messages are sent to stderr by default, but you can redirect
357 them to a logging system using nbd_set_debug_callback(3).
358
360 There are several ways to connect to NBD servers, and you can even run
361 a server from libnbd. Normally you would connect to a server which is
362 already running, over a local Unix domain socket or a remote TCP
363 connection. The high level API calls are:
364 nbd_connect_unix (nbd, "socket");
366 nbd_connect_tcp (nbd, "localhost", "nbd");
367 For nbd_connect_tcp(3) the third parameter is the port name or number,
369 which can either be a name from /etc/services or the port number as a
370 string (eg. "10809").
371 Connecting to an NBD URI
373 libnbd supports the NBD URI specification. The format of URIs is
374 documented in nbd_connect_uri(3).
375 You can connect to a URI as in these examples (using the high level
377 API):
378 nbd_connect_uri (nbd, "nbd://example.com/");
380 nbd_connect_uri (nbd, "nbds+unix:///export?socket=/tmp/nbd.sock");
382 This feature is implemented by calling other libnbd APIs to set up the
384 export name, TLS parameters, and finally connect over a Unix domain
385 socket or TCP.
386 URI support is an optional feature of the library, requiring libxml2 at
388 compile time. The nbd_connect_uri(3) and nbd_aio_connect_uri(3) calls
389 will raise an error (with nbd_get_errno(3) returning "ENOTSUP") if it
390 was not built with this feature, and you can also test for it
391 explicitly using nbd_supports_uri(3).
392 Connecting to a subprocess
394 Some NBD servers — notably nbdkit(1) with the -s parameter, and
395 nbd-server(1) with the port parameter set to 0 — can also accept a
396 single NBD connection on stdin/stdout. You can run these servers as a
397 subprocess of your main program using nbd_connect_command(3). This
398 example creates a 1G writable RAM disk:
399 char *argv[] = { "nbdkit", "-s", "--exit-with-parent",
401 "memory", "1G", NULL };
402 nbd_connect_command (nbd, argv);
403 When the handle is closed the nbdkit subprocess is killed, which in
405 this case means the RAM disk is discarded, so this is useful for
406 testing.
407 Connecting to a subprocess using systemd socket activation
409 Some NBD servers — notably nbdkit(1) and qemu-nbd(1) — support systemd
410 socket activation allowing libnbd to pass a socket to the subprocess.
411 This works very similarly to nbd_connect_command(3) described above,
412 but you must use nbd_connect_systemd_socket_activation(3) instead.
413 Connecting to any socket
415 If none of the other nbd_connect* methods are suitable you can create a
416 connected socket yourself and pass it to nbd_connect_socket(3).
417 One use for this is in fuzzing where we use socketpair(2) to create the
419 socket, then fork, then have the test harness in the child process
420 connected to libnbd over the socket pair (see:
421 https://gitlab.com/nbdkit/libnbd/-/blob/master/fuzzing/libnbd-fuzz-wrapper.c).
422 Another use is to connect libnbd to an address family that it does not
424 support natively, such as XDP or IB.
425
427 By default, when beginning a connection, libnbd will handle all
428 negotiation with the server, using only the configuration (eg.
429 nbd_set_export_name(3) or nbd_add_meta_context(3)) that was requested
430 before the connection attempt; this phase continues until
431 nbd_aio_is_connecting(3) no longer returns true, at which point, either
432 data commands are ready to use or else the connection has failed with
433 an error.
434 But there are scenarios in which it is useful to also control the
436 handshaking commands sent during negotiation, such as asking the server
437 for a list of available exports prior to selecting which one to use.
438 This is done by calling nbd_set_opt_mode(3) before connecting; then
439 after requesting a connection, the state machine will pause at
440 nbd_aio_is_negotiating(3) at any point that the user can decide which
441 handshake command to send next. Note that the negotiation state is
442 only reachable from newstyle servers; older servers cannot negotiate
443 and will progress all the way to the ready state.
444 When the negotiating state is reached, you can initiate option commands
446 such as nbd_opt_list(3) or their asynchronous equivalents, as well as
447 alter configuration such as export name that previously had to be set
448 before connection. Since the NBD protocol does not allow parallel
449 negotiating commands, no cookie is involved, and you can track
450 completion of each command when the state is no longer
451 nbd_aio_is_connecting(3). If nbd_opt_go(3) fails but the connection is
452 still live, you will be back in negotiation state, where you can
453 request a different export name and try again. Exiting the negotiation
454 state is only possible with a successful nbd_opt_go(3) which moves to
455 the data phase, or nbd_opt_abort(3) which performs a clean shutdown of
456 the connection by skipping the data phase.
457
459 It is possible for NBD servers to serve different content on different
460 “exports”. For this you must pass the right export name to the server.
461 Call this API before connecting:
462 nbd_set_export_name (nbd, "export");
464 Note that there are some servers (like nbdkit(1) ≤ 1.14) which ignore
466 this, and other servers (like qemu-nbd(8)) which require it to be set
467 correctly but cannot serve different content.
468 These APIs are also available after a successful nbd_opt_info(3) during
470 the negotiation phase, if you used nbd_set_opt_mode(3) prior to
471 connecting.
472 Flag calls
474 After connecting the server will send back a set of flags describing
475 the export, such as whether it is writable and if it can support flush
476 to permanent storage. These flags can be accessed from libnbd using
477 APIs such as:
478 int is_read_only = nbd_is_read_only (nbd);
480 int can_flush = nbd_can_flush (nbd);
481 Flag calls are: nbd_can_cache(3), nbd_can_df(3), nbd_can_fast_zero(3),
483 nbd_can_flush(3), nbd_can_fua(3), nbd_can_meta_context(3),
484 nbd_can_multi_conn(3), nbd_can_trim(3), nbd_can_zero(3),
485 nbd_is_read_only(3), nbd_is_rotational(3).
486 Size of the export
488 To get the size of the export in bytes, use nbd_get_size(3):
489 int64_t size = nbd_get_size (nbd);
491
493 You can read and write data from the NBD server using nbd_pread(3) and
494 nbd_pwrite(3) or their asynchronous equivalents.
495 All data commands support a "flags" argument (mandatory in C, but
497 optional in languages where it can default to 0). For convenience, the
498 constant "LIBNBD_CMD_FLAG_MASK" is defined with the set of flags
499 currently recognized by libnbd, where future NBD protocol extensions
500 may result in additional flags being supported; but in general,
501 specific data commands only accept a subset of known flags.
502 Libnbd defaults to performing some client-side sanity checking in each
504 of its data commands; for example, attempts to write to a server that
505 has advertised a read-only connection are rejected. It is possible to
506 override aspects of this checking by using nbd_set_strict_mode(3).
507 Some servers also support:
509 trim/discard
511 If nbd_can_trim(3) returns true, nbd_trim(3) can be used to “punch
512 holes” in the backing storage of the disk on the server. Normally
513 (although not in every case) the holes read back as zeroes but take
514 up no space.
515 zeroing
517 If nbd_can_zero(3) returns true, nbd_zero(3) can be used to
518 efficiently zero parts of the disk without having to send large
519 amounts of zero bytes over the network (as would be necessary if
520 using nbd_pwrite(3)).
521 This is slightly different from trimming because the backing
523 storage is still allocated. For some storage types this can make
524 future writes more efficient and/or less likely to fail because of
525 out of space errors.
526 flushing
528 Some servers can commit data to permanent storage and tell you that
529 this has happened reliably. There are two export flags associated
530 with this: nbd_can_flush(3) and nbd_can_fua(3).
531 The nbd_flush(3) call (available if nbd_can_flush(3) returns true)
533 flushes all pending writes to disk and does not complete until that
534 operation has finished. It is similar to using sync(2) on POSIX
535 systems.
536 A more efficient way to achieve this is to set the flag
538 "LIBNBD_CMD_FLAG_FUA" on write-like calls (like write, trim and
539 zero). This flag means the call will not complete until committed
540 to permanent storage, but it does not involve flushing the entire
541 disk.
542 prefetching
544 Some servers can prefetch data, making subsequent reads faster.
545 The nbd_cache(3) call (available if nbd_can_cache(3) returns true)
546 is used to prefetch.
547 block status
549 Some servers are able to provide information about the various
550 extents within the image, via the notion of one or more meta
551 contexts. The most common meta context is "base:allocation"
552 (available in libnbd.h as "LIBNBD_CONTEXT_BASE_ALLOCATION"), which
553 can be used to learn which portions of a file are allocated or read
554 as zero. Other contexts may be available; for example, qemu-nbd(8)
555 can expose a meta context "qemu:dirty-bitmap:NAME" for tracking
556 which portions of a file are tracked by a qcow2 dirty bitmap.
557 In order to utilize block status, the client must call
559 nbd_add_meta_context(3) prior to connecting, for each meta context
560 in which it is interested, then check nbd_can_meta_context(3) after
561 connection to see which contexts the server actually supports. If
562 a context is supported, the client can then use nbd_block_status(3)
563 with a callback function that will receive an array of 32-bit
564 integer pairs describing consecutive extents within a context. In
565 each pair, the first integer is the length of the extent, the
566 second is a bitmask description of that extent (for the
567 "base:allocation" context, the bitmask may include
568 "LIBNBD_STATE_HOLE" for unallocated portions of the file, and/or
569 "LIBNBD_STATE_ZERO" for portions of the file known to read as
570 zero).
571 There is a full example of requesting meta context and using block
573 status available at
574 https://gitlab.com/nbdkit/libnbd/blob/master/interop/dirty-bitmap.c
575
577 Issuing multiple in-flight requests
578 NBD servers which properly implement the specification can handle
579 multiple data requests in flight over the same connection at the same
580 time. Libnbd supports this when using the low level API.
581 To use it you simply issue more requests as needed (eg. using calls
583 like nbd_aio_pread(3), nbd_aio_pwrite(3)) without waiting for previous
584 commands to complete. You need to be careful that requests in flight
585 do not overlap with disk offsets of other write-like commands in flight
586 — an overlapping read may see indeterminate data, and an overlapping
587 write may even cause disk corruption where the resulting disk contents
588 do not match either of the two writes.
589 Each request is identified by a unique 64 bit cookie (assigned by
591 libnbd), allowing libnbd and callers to match replies to requests.
592 Replies may arrive out of order. A request that is rejected client-
593 side for failing a sanity check (such as attempting to write to a read-
594 only server, see nbd_set_strict_mode(3)) will fail rather than
595 returning a cookie, although closure cleanup is still performed.
596 Although in theory you can have an indefinite number of requests in
598 flight at the same time, in practice it's a good idea to limit them to
599 some number. Libnbd will queue commands in the handle even if it
600 cannot write them to the server, so this limit is largely to prevent a
601 backlog of commands from consuming too much memory. It is suggested to
602 start with a limit of 64 requests in flight (per NBD connection), and
603 measure how adjusting the limit up and down affects performance for
604 your local configuration.
605 There is a full example using multiple in-flight requests available at
607 https://gitlab.com/nbdkit/libnbd/blob/master/examples/threaded-reads-and-writes.c
608 Multi-conn
610 Some NBD servers advertise “multi-conn” which means that it is safe to
611 make multiple connections to the server and load-balance commands
612 across all of the connections.
613 To do this you should open a single connection first and test for this
615 feature using nbd_can_multi_conn(3). Without error handling it would
616 look like this:
617 struct nbd_handle *nbd[4];
619 size_t i;
620 bool supports_multi_conn;
621 nbd[0] = nbd_create ();
623 nbd_connect_tcp (nbd[0], "server", "10809");
624 supports_multi_conn = nbd_can_multi_conn (nbd[0]) > 0;
625 If multi-conn is supported then you can open further connections:
627 if (supports_multi_conn) {
629 for (i = 1; i <= 3; ++i) {
630 nbd[i] = nbd_create ();
631 nbd_connect_tcp (nbd[i], "server", "10809");
632 }
633 }
634 If you are issuing multiple in-flight requests (see above) and limiting
636 the number, then the limit should be applied to each individual NBD
637 connection.
638
640 The NBD protocol and libnbd supports TLS (sometimes incorrectly called
641 “SSL”) for encryption of the data stream and authentication of clients
642 and servers. Libnbd defaults to TLS disabled for maximum
643 interoperability. To enable it on a handle you must call
644 nbd_set_tls(3) before connecting.
645 To allow TLS, but fall back to unencrypted:
647 nbd_set_tls (nbd, LIBNBD_TLS_ALLOW);
649 Use nbd_get_tls_negotiated(3) to find out if TLS negotiation was
651 successful. Avoid "LIBNBD_TLS_ALLOW" if man-in-the-middle attacks are
652 a concern.
653 The most secure mode is to require TLS and fail to connect if the
655 server does not support it:
656 nbd_set_tls (nbd, LIBNBD_TLS_REQUIRE);
658 It may also be necessary to verify that the server’s identity is
660 correct. For some servers it may be necessary to verify to the server
661 that the client is permitted to connect. This can be done using either
662 X.509 certificates, or TLS Pre-Shared Keys (PSK). Certificates are
663 more secure. PSK is far more convenient, but you must have an existing
664 secure channel to distribute the keys.
665 Setting up X.509 using system certificate authorities (CAs)
667 This is the default if you don’t call any other "nbd_set_tls_*"
668 functions. In this case the server must have a public (eg. HTTPS)
669 certificate which can be verified against the CAs registered on your
670 system (eg. under /etc/pki).
671 To disable server name verification — which opens you up to a potential
673 Man-In-The-Middle (MITM) attack — use:
674 nbd_set_tls_verify_peer (nbd, false);
676 Setting up an X.509 certificate authority (CA)
678 You can set up your own CA and register clients and servers with it,
679 issuing client and server certificates which will reliably authenticate
680 your clients and servers to each other.
681 Doing this is described in detail in the nbdkit-tls(1) manual. The
683 only differences for libnbd are:
684 • Non-root certificates must be placed in "$HOME/.pki/libnbd/" or
686 "$HOME/.config/pki/libnbd/"
687 • Libnbd reads client-cert.pem and client-key.pem (instead of
689 server-cert.pem and server-key.pem).
690 Once you have set up the directory containing the certificates, call:
692 nbd_set_tls_certificates (nbd, "/path/to/directory");
694 Setting up Pre-Shared Keys (PSK)
696 TLS Pre-Shared Keys are a much more convenient method of setting up
697 TLS, and more appropriate for NBD, but you should have an existing
698 secure method available to distribute the keys. They are therefore
699 ideal if you want to set up an NBD service as an adjunct to an existing
700 secure REST API.
701 Use psktool(1) to create a file of "username:key" pairs:
703 psktool -u username -p keys.psk
705 and pass this path to libnbd:
707 nbd_set_tls_psk_file (nbd, "keys.psk");
709 If necessary you may need to set the client username (otherwise libnbd
711 will use your login name):
712 nbd_set_tls_username (nbd, "username");
714
716 Some libnbd calls take callbacks (eg. nbd_set_debug_callback(3),
717 nbd_aio_pread(3)). Libnbd can call these functions while processing.
718 In the C API these libnbd calls take a structure which contains the
720 function pointer and an optional opaque "void *user_data" pointer:
721 nbd_aio_pread (nbd, buf, sizeof buf, offset,
723 (nbd_completion_callback) { .callback = my_fn,
724 .user_data = my_data },
725 0);
726 For optional callbacks, if you don't want the callback, either set
728 ".callback" to "NULL" or use the equivalent macros (such as
729 "NBD_NULL_COMPLETION") defined in "libnbd.h":
730 nbd_aio_pread (nbd, buf, sizeof buf, offset,
732 NBD_NULL_COMPLETION, 0);
733 From other languages the structure and opaque pointer are not needed
735 because you can use closures to achieve the same effect.
736 Callback lifetimes
738 You can associate an optional free function with callbacks. Libnbd
739 will call this function when the callback will not be called again by
740 libnbd, including in the case where the API fails.
741 This can be used to free associated "user_data". For example:
743 void *my_data = malloc (...);
745 nbd_aio_pread_structured (nbd, buf, sizeof buf, offset,
747 (nbd_chunk_callback) { .callback = my_fn,
748 .user_data = my_data,
749 .free = free },
750 NBD_NULL_CALLBACK(completion),
751 0);
752 will call free(3) on "my_data" after the last time that the
754 "chunk.callback = my_fn" function is called.
755 The free function is only accessible in the C API as it is not needed
757 in garbage collected programming languages.
758 Callbacks with ".callback=NULL" and ".free!=NULL"
760 It is possible to register a callback like this:
761 ...
763 (nbd_completion_callback) { .callback = NULL,
764 .user_data = my_data,
765 .free = free },
766 ...
767 The meaning of this is that the callback is never called, but the free
769 function is still called after the last time the callback would have
770 been called. This is useful for applying generic freeing actions when
771 asynchronous commands are retired.
772 Callbacks and locking
774 The callbacks are invoked at a point where the libnbd lock is held; as
775 such, it is unsafe for the callback to call any "nbd_*" APIs on the
776 same nbd object, as it would cause deadlock.
777 Completion callbacks
779 All of the low-level commands have a completion callback variant that
780 registers a callback function used right before the command is marked
781 complete.
782 When the completion callback returns 1, the command is automatically
784 retired (there is no need to call nbd_aio_command_completed(3)); for
785 any other return value, the command still needs to be retired.
786 Callbacks with "int *error" parameter
788 Some of the high-level commands (nbd_pread_structured(3),
789 nbd_block_status(3)) involve the use of a callback function invoked by
790 the state machine at appropriate points in the server's reply before
791 the overall command is complete. These callback functions, along with
792 all of the completion callbacks, include a parameter "error" containing
793 the value of any error detected so far; if the callback function fails,
794 it should assign back into "error" and return "-1" to change the
795 resulting error of the overall command. Assignments into "error" are
796 ignored for any other return value; similarly, assigning 0 into "error"
797 does not have an effect.
798
800 On most systems, C programs that use libnbd can be compiled like this:
801 cc prog.c -o prog -lnbd
803 To detect if the libnbd library and header file is installed, the
805 preferred method is to use pkg-config(1) or pkgconf(1):
806 pkg-config libnbd --exists || fail libnbd is required
808 In case the library or header file are not installed in the usual
810 system locations, you can compile your program like this, using pkg-
811 config to detect the proper location of libnbd:
812 cc prog.c -o prog `pkg-config libnbd --cflags --libs`
814 To compile an external project against a built copy of the libnbd
816 source tree which hasn't been installed, see the ./run script.
817 Autoconf projects
819 External projects which use autoconf and need to check if libnbd is
820 installed should use the "PKG_CHECK_MODULES" macro in configure.ac like
821 this:
822 PKG_CHECK_MODULES([LIBNBD], [libnbd])
824 This will define "@LIBNBD_CFLAGS@" and "@LIBNBD_LIBS@" which you will
826 need to add to your Makefile.am.
827 CMake projects
829 For CMake projects use:
830 find_package(PkgConfig REQUIRED)
832 pkg_check_modules(LIBNBD REQUIRED libnbd)
833 target_link_libraries(prog ${LIBNBD_LIBRARIES})
834 target_include_directories(prog PUBLIC ${LIBNBD_INCLUDE_DIRS})
835 target_compile_options(prog PUBLIC ${LIBNBD_CFLAGS_OTHER})
836 Meson projects
838 For meson projects use:
839 nbd_dep = dependency('libnbd')
841 executable('prog', 'prog.c', dependencies : [nbd_dep])
842
844 "HOME"
845 Used in some situations to find TLS certificates. See
846 nbd_set_tls_certificates(3).
847 "LIBNBD_DEBUG"
849 If this is set to the exact string 1 when the handle is created
850 then debugging is enabled. See "DEBUGGING MESSAGES" above.
851 "LOGNAME"
853 The default TLS username. See nbd_set_tls_username(3).
854
856 C API
857 nbd_add_meta_context(3), nbd_aio_block_status(3), nbd_aio_cache(3),
858 nbd_aio_command_completed(3), nbd_aio_connect(3),
859 nbd_aio_connect_command(3), nbd_aio_connect_socket(3),
860 nbd_aio_connect_systemd_socket_activation(3), nbd_aio_connect_tcp(3),
861 nbd_aio_connect_unix(3), nbd_aio_connect_uri(3),
862 nbd_aio_connect_vsock(3), nbd_aio_disconnect(3), nbd_aio_flush(3),
863 nbd_aio_get_direction(3), nbd_aio_get_fd(3), nbd_aio_in_flight(3),
864 nbd_aio_is_closed(3), nbd_aio_is_connecting(3), nbd_aio_is_created(3),
865 nbd_aio_is_dead(3), nbd_aio_is_negotiating(3),
866 nbd_aio_is_processing(3), nbd_aio_is_ready(3), nbd_aio_notify_read(3),
867 nbd_aio_notify_write(3), nbd_aio_opt_abort(3), nbd_aio_opt_go(3),
868 nbd_aio_opt_info(3), nbd_aio_opt_list(3),
869 nbd_aio_opt_list_meta_context(3), nbd_aio_peek_command_completed(3),
870 nbd_aio_pread(3), nbd_aio_pread_structured(3), nbd_aio_pwrite(3),
871 nbd_aio_trim(3), nbd_aio_zero(3), nbd_block_status(3), nbd_cache(3),
872 nbd_can_cache(3), nbd_can_df(3), nbd_can_fast_zero(3),
873 nbd_can_flush(3), nbd_can_fua(3), nbd_can_meta_context(3),
874 nbd_can_multi_conn(3), nbd_can_trim(3), nbd_can_zero(3),
875 nbd_clear_debug_callback(3), nbd_clear_meta_contexts(3), nbd_close(3),
876 nbd_connect_command(3), nbd_connect_socket(3),
877 nbd_connect_systemd_socket_activation(3), nbd_connect_tcp(3),
878 nbd_connect_unix(3), nbd_connect_uri(3), nbd_connect_vsock(3),
879 nbd_connection_state(3), nbd_create(3), nbd_flush(3),
880 nbd_get_block_size(3), nbd_get_canonical_export_name(3),
881 nbd_get_debug(3), nbd_get_errno(3), nbd_get_error(3),
882 nbd_get_export_description(3), nbd_get_export_name(3),
883 nbd_get_full_info(3), nbd_get_handle_name(3),
884 nbd_get_handshake_flags(3), nbd_get_meta_context(3),
885 nbd_get_nr_meta_contexts(3), nbd_get_opt_mode(3),
886 nbd_get_package_name(3), nbd_get_private_data(3), nbd_get_protocol(3),
887 nbd_get_request_structured_replies(3), nbd_get_size(3),
888 nbd_get_strict_mode(3), nbd_get_structured_replies_negotiated(3),
889 nbd_get_tls(3), nbd_get_tls_negotiated(3), nbd_get_tls_username(3),
890 nbd_get_tls_verify_peer(3), nbd_get_uri(3), nbd_get_version(3),
891 nbd_is_read_only(3), nbd_is_rotational(3), nbd_kill_subprocess(3),
892 nbd_opt_abort(3), nbd_opt_go(3), nbd_opt_info(3), nbd_opt_list(3),
893 nbd_opt_list_meta_context(3), nbd_poll(3), nbd_pread(3),
894 nbd_pread_structured(3), nbd_pwrite(3), nbd_set_debug(3),
895 nbd_set_debug_callback(3), nbd_set_export_name(3),
896 nbd_set_full_info(3), nbd_set_handle_name(3),
897 nbd_set_handshake_flags(3), nbd_set_opt_mode(3),
898 nbd_set_private_data(3), nbd_set_request_structured_replies(3),
899 nbd_set_strict_mode(3), nbd_set_tls(3), nbd_set_tls_certificates(3),
900 nbd_set_tls_psk_file(3), nbd_set_tls_username(3),
901 nbd_set_tls_verify_peer(3), nbd_set_uri_allow_local_file(3),
902 nbd_set_uri_allow_tls(3), nbd_set_uri_allow_transports(3),
903 nbd_shutdown(3), nbd_supports_tls(3), nbd_supports_uri(3), nbd_trim(3),
904 nbd_zero(3).
905 Servers
907 nbdkit(1), nbd-server(1), qemu-nbd(8).
908 Encryption tools
910 certtool(1), nbdkit-tls(1), psktool(1).
911 Standards
913 https://github.com/NetworkBlockDevice/nbd/blob/master/doc/proto.md,
914 https://github.com/NetworkBlockDevice/nbd/blob/master/doc/uri.md.
915 Other
917 libnbd-release-notes-1.6(1), libnbd-release-notes-1.4(1),
918 libnbd-release-notes-1.2(1), libnbd-security(3), nbdcopy(1),
919 nbdfuse(1), nbdinfo(1), nbdsh(1), qemu(1).
920
922 Eric Blake
923 Richard W.M. Jones
925
927 Copyright (C) 2019-2021 Red Hat Inc.
928
930 This library is free software; you can redistribute it and/or modify it
931 under the terms of the GNU Lesser General Public License as published
932 by the Free Software Foundation; either version 2 of the License, or
933 (at your option) any later version.
934 This library is distributed in the hope that it will be useful, but
936 WITHOUT ANY WARRANTY; without even the implied warranty of
937 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
938 Lesser General Public License for more details.
939 You should have received a copy of the GNU Lesser General Public
941 License along with this library; if not, write to the Free Software
942 Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
943 02110-1301 USA
944libnbd-1.7.12 2021-05-29 libnbd(3)