1libnbd(3) LIBNBD libnbd(3)
2
6 libnbd - network block device (NBD) client library in userspace
7
9 #include <libnbd.h>
10 struct nbd_handle *nbd;
12 char buf[512];
13 if ((nbd = nbd_create ()) == NULL ||
15 nbd_connect_tcp (nbd, "server.example.com", "nbd") == -1 ||
16 nbd_pread (nbd, buf, sizeof buf, 0, 0) == -1)
17 fprintf (stderr, "%s\n", nbd_get_error ());
18 nbd_close (nbd);
19 exit (EXIT_FAILURE);
20 }
21 nbd_close (nbd);
22 cc prog.c -o prog -lnbd
24 or:
25 cc prog.c -o prog `pkg-config libnbd --cflags --libs`
26
28 Network Block Device (NBD) is a network protocol for accessing block
29 devices over the network. Block devices are hard disks and things that
30 behave like hard disks such as disk images and virtual machines.
31 Libnbd is a client library for the NBD protocol which can access most
33 of the features of NBD while being simple to use and powerful.
34 This manual page gives an overview of libnbd, using C as an example,
36 but the library is available from other programming languages.
37 nbd_create(3), nbd_pread(3), etc.
39 Each manual page covers one function from the C API in detail.
40 There is a full list in section "C API" below.
41 libnbd-ocaml(3)
43 Using the API from OCaml.
44 libnbd-golang(3)
46 Using the API from Go.
47 nbdsh(1)
49 Using the NBD shell (nbdsh) for command line and Python scripting.
50
52 To use the API at all you must first open a handle by calling
53 nbd_create(3) (or its equivalent in other languages):
54 struct nbd_handle *nbd;
56 nbd = nbd_create ();
58 This creates and returns a handle, which is associated with one
60 connection to an NBD server, initially not connected.
61 Each handle is a complex state machine which can be in states such as
63 created, connected to a remote server, handshaking, idle and ready to
64 issue commands, or busy sending or receiving commands.
65 Handles have a name used in debugging messages. The name is normally
67 generated ("nbd1", "nbd2" etc) but you can set a friendly name with
68 nbd_set_handle_name(3). Also there is a private field in the handle
69 for use by the application, see nbd_set_private_data(3).
70 When you have finished with the handle you must call nbd_close(3) which
72 closes the underlying socket (if necessary) and frees up all associated
73 resources.
74
76 There are two levels of API available. A simple high level synchronous
77 API lets you give the handle high level instructions like “connect to
78 the server”, “read a block”, “write a block”, etc. Each of these
79 functions will run to completion, blocking the current thread before
80 returning. A more complicated low level non-blocking asynchronous API
81 is also available where you can integrate with poll(2) or another main
82 loop.
83 You can freely mix the two APIs on the same handle. You can also call
85 APIs on a single handle from multiple threads. Single API calls on the
86 handle are atomic — they either take a lock on the handle while they
87 run or are careful to access handle fields atomically.
88 Libnbd does not create its own threads.
90
92 This is the simplest way to use the API, with the possible drawback
93 that each libnbd function blocks until it is finished.
94 Create a handle and connect to the server:
96 struct nbd_handle *nbd;
98 nbd = nbd_create ();
100 if (!nbd) {
101 fprintf (stderr, "%s\n", nbd_get_error ());
102 nbd_close (nbd);
103 exit (EXIT_FAILURE);
104 }
105 if (nbd_connect_tcp (nbd, "server.example.com", "nbd") == -1) {
106 fprintf (stderr, "%s\n", nbd_get_error ());
107 nbd_close (nbd);
108 exit (EXIT_FAILURE);
109 }
110 Read the first sector (512 bytes) from the NBD export:
112 char buf[512];
114 if (nbd_pread (nbd, buf, sizeof buf, 0, 0) == -1) {
116 fprintf (stderr, "%s\n", nbd_get_error ());
117 nbd_close (nbd);
118 exit (EXIT_FAILURE);
119 }
120 Close the handle:
122 nbd_close (nbd);
124 You can call the high level API from multiple threads, but each libnbd
126 API call takes a lock on the handle and so commands will not run in
127 parallel.
128
130 The low level API is useful if you want to use libnbd in non-blocking
131 code; or if you want to issue commands in parallel from multiple
132 threads; or if you need more control especially over having multiple
133 commands in-flight on a single connection.
134 To use the low level API you will need to integrate with poll(2) or
136 another “main loop” such as the GLib main event loop.
137 Issuing asynchronous commands
139 Use the "nbd_aio_*" variants to issue commands asynchronously (without
140 waiting for the command to complete before returning). For example the
141 asynchronous variant of nbd_pread(3) is:
142 int64_t cookie;
144 cookie = nbd_aio_pread (nbd, buf, sizeof buf,
146 NBD_NULL_COMPLETION, 0);
147 if (cookie == -1) {
148 fprintf (stderr, "%s\n", nbd_get_error ());
149 nbd_close (nbd);
150 exit (EXIT_FAILURE);
151 }
152 There are several things to note here:
154 • This only starts the command. The command is still in flight when
156 the call returns.
157 • A buffer ("buf") has been assigned to collect the result of the
159 read, but it is not guaranteed to be filled with data until the
160 command has completed (see examples below). The buffer must not be
161 freed until the command has finished running.
162 • You can issue multiple commands on the same handle at the same
164 time.
165 • A cookie is returned which identifies this command in subsequent
167 calls. The cookie is unique (per libnbd handle) and ≥ 1.
168 • You may register a function which is called when the command
170 completes, see "Completion callbacks" below. In this case we have
171 specified a null completion callback.
172 Socket and direction
174 Each libnbd handle has an associated socket (once it has started
175 connecting). You can read the file descriptor of the socket using:
176 int fd = nbd_aio_get_fd (nbd);
178 The socket is non-blocking. Between calls into libnbd it is in the
180 "would block" condition. You can find out if libnbd is expecting to
181 read or write from the socket next by calling:
182 int dir = nbd_aio_get_direction (nbd);
184 which returns one of "LIBNBD_AIO_DIRECTION_READ",
186 "LIBNBD_AIO_DIRECTION_WRITE" or "LIBNBD_AIO_DIRECTION_BOTH" (=
187 "READ|WRITE"). And so to set up the next call to poll(2) or other main
188 loop you must translate this to "POLLIN", "POLLOUT" or "POLLIN|POLLOUT"
189 (or whatever mechanism your main loop uses).
190 Notifying libnbd when an event happens
192 When you detect (eg. using poll(2)) that a read or write event has
193 happened on the socket, you must then tell libnbd about it. You have
194 to check the direction again (since it may have been changed by another
195 thread), and notify libnbd:
196 int r = 0;
198 dir = nbd_aio_get_direction (nbd);
200 if ((dir & LIBNBD_AIO_DIRECTION_READ) &&
202 a_read_event_occurred ())
203 r = nbd_aio_notify_read (nbd);
204 else if ((dir & LIBNBD_AIO_DIRECTION_WRITE) &&
205 a_write_event_occurred ())
206 r = nbd_aio_notify_write (nbd);
207 if (r == -1) {
209 fprintf (stderr, "%s\n", nbd_get_error ());
210 // ...
211 }
212 The notify calls move the state machine along, reading and writing from
214 the socket possibly multiple times, until the socket would block again,
215 at which point they return control to the caller.
216 Simple implementation with nbd_poll(3)
218 In fact if you want to use poll(2) on a single handle, a simple
219 implementation has already been written called nbd_poll(3). It is also
220 useful to examine how this is implemented (lib/poll.c in the libnbd
221 source code) because that will tell you how to integrate libnbd with
222 more complex main loops.
223 Some examples of using nbd_poll(3) follow.
225 As with the high level API, it all starts by creating a handle:
227 struct nbd_handle *nbd;
229 nbd = nbd_create ();
231 if (nbd == NULL) {
232 fprintf (stderr, "%s\n", nbd_get_error ());
233 nbd_close (nbd);
234 exit (EXIT_FAILURE);
235 }
236 To connect to the server asynchronously, we start the connection using
238 nbd_aio_connect(3) and then enter our main loop to check for events
239 until the connection becomes ready:
240 int fd;
242 struct sockaddr_un addr;
243 socklen_t len;
244 /* some code to set up addr,
246 then ... */
247 if (nbd_aio_connect (nbd, &addr, len) == -1) {
248 fprintf (stderr, "%s\n", nbd_get_error ());
249 nbd_close (nbd);
250 exit (EXIT_FAILURE);
251 }
252 while (! nbd_aio_is_ready (nbd)) {
253 if (nbd_poll (nbd, -1) == -1) {
254 fprintf (stderr, "%s\n", nbd_get_error ());
255 nbd_close (nbd);
256 exit (EXIT_FAILURE);
257 }
258 }
259 To read data asynchronously, start an asynchronous read command, which
261 returns a 64 bit command cookie, and enter the main loop until the
262 command has completed:
263 int64_t cookie;
265 char buf[512];
266 cookie = nbd_aio_pread (nbd, buf, sizeof buf, offset,
268 NBD_NULL_COMPLETION, 0);
269 if (cookie == -1) {
270 fprintf (stderr, "%s\n", nbd_get_error ());
271 nbd_close (nbd);
272 exit (EXIT_FAILURE);
273 }
274 while (! nbd_aio_command_completed (nbd, cookie)) {
275 if (nbd_poll (nbd, -1) == -1) {
276 fprintf (stderr, "%s\n", nbd_get_error ());
277 nbd_close (nbd);
278 exit (EXIT_FAILURE);
279 }
280 }
281 For almost all high level synchronous calls (eg. nbd_pread(3)) there is
283 a low level asynchronous equivalent (eg. nbd_aio_pread(3)) for starting
284 a command.
285 glib2 integration
287 See
288 https://gitlab.com/nbdkit/libnbd/blob/master/examples/glib-main-loop.c
289 libev integration
291 See https://gitlab.com/nbdkit/libnbd/blob/master/examples/copy-libev.c
292
294 When any API call returns an error ("-1" or "NULL" depending on the
295 API), an error message and sometimes an errno value are available. You
296 can retrieve the error message and/or errno of the most recently failed
297 call using nbd_get_error(3) and nbd_get_errno(3). For example:
298 if (nbd_connect_tcp (nbd, "remote", "nbd") == -1) {
300 fprintf (stderr,
301 "failed to connect to remote server: %s (errno = %d)\n",
302 nbd_get_error (), nbd_get_errno ());
303 }
304 These functions use thread-local storage to return the most recent
306 error in the current thread. This is why you don't need to pass the
307 handle to these calls. They even work if nbd_create(3) returns "NULL"
308 when there is no handle at all.
309 For this reason you cannot call them from a different thread. You
311 should call them immediately after the failed API call, from the same
312 thread. Furthermore the error string returned by nbd_get_error(3) is
313 only valid until the next libnbd API call in the current thread, so if
314 you need to keep the string you must copy it (eg. using strdup(3)).
315 Errno
317 For some errors, a system call error number (see errno(3)) is
318 available. You can find the error number by calling nbd_get_errno(3).
319 It works the same way as nbd_get_error(3) with respect to threads.
320 Even when a call returns an error, nbd_get_errno(3) might return 0.
322 This does not mean there was no error. It means no additional errno
323 information is available for this error.
324 The error number is often the raw error returned by a system call that
326 failed.
327 It can also be used to indicate special conditions. The most common
329 cases are:
330 "EINVAL"
332 Invalid parameters or state for the current libnbd call. (This can
333 also indicate that requests are not aligned to "Block size
334 constraints").
335 "ENOTSUP"
337 The libnbd call is not available in this build of libnbd (eg. when
338 using a TLS API if the library was compiled without TLS support).
339 "ENOMEM"
341 The library ran out of memory while performing some operation.
342 "ERANGE"
344 A request is too large, for example if you try to read too many
345 bytes in a single nbd_pread(3) call.
346
348 Libnbd can print lots of debugging messages, useful if you have a
349 problem with the library. Either enable debugging after creating the
350 handle:
351 nbd = nbd_create ();
353 nbd_set_debug (nbd, true);
354 or set the "LIBNBD_DEBUG=1" environment variable which will enable
356 debugging by default on all new handles.
357 Debugging messages are sent to stderr by default, but you can redirect
359 them to a logging system using nbd_set_debug_callback(3).
360
362 There are several ways to connect to NBD servers, and you can even run
363 a server from libnbd. Normally you would connect to a server which is
364 already running, over a local Unix domain socket or a remote TCP
365 connection. The high level API calls are:
366 nbd_connect_unix (nbd, "socket");
368 nbd_connect_tcp (nbd, "localhost", "nbd");
369 For nbd_connect_tcp(3) the third parameter is the port name or number,
371 which can either be a name from /etc/services or the port number as a
372 string (eg. "10809").
373 Connecting to an NBD URI
375 libnbd supports the NBD URI specification. The format of URIs is
376 documented in nbd_connect_uri(3).
377 You can connect to a URI as in these examples (using the high level
379 API):
380 nbd_connect_uri (nbd, "nbd://example.com/");
382 nbd_connect_uri (nbd, "nbds+unix:///export?socket=/tmp/nbd.sock");
384 This feature is implemented by calling other libnbd APIs to set up the
386 export name, TLS parameters, and finally connect over a Unix domain
387 socket or TCP.
388 URI support is an optional feature of the library, requiring libxml2 at
390 compile time. The nbd_connect_uri(3) and nbd_aio_connect_uri(3) calls
391 will raise an error (with nbd_get_errno(3) returning "ENOTSUP") if it
392 was not built with this feature, and you can also test for it
393 explicitly using nbd_supports_uri(3).
394 Connecting to a subprocess
396 Some NBD servers — notably nbdkit(1) with the -s parameter, and
397 nbd-server(1) with the port parameter set to 0 — can also accept a
398 single NBD connection on stdin/stdout. You can run these servers as a
399 subprocess of your main program using nbd_connect_command(3). This
400 example creates a 1G writable RAM disk:
401 char *argv[] = { "nbdkit", "-s", "--exit-with-parent",
403 "memory", "1G", NULL };
404 nbd_connect_command (nbd, argv);
405 When the handle is closed the nbdkit subprocess is killed, which in
407 this case means the RAM disk is discarded, so this is useful for
408 testing.
409 Connecting to a subprocess using systemd socket activation
411 Some NBD servers — notably nbdkit(1) and qemu-nbd(1) — support systemd
412 socket activation allowing libnbd to pass a socket to the subprocess.
413 This works very similarly to nbd_connect_command(3) described above,
414 but you must use nbd_connect_systemd_socket_activation(3) instead.
415 Connecting to any socket
417 If none of the other nbd_connect* methods are suitable you can create a
418 connected socket yourself and pass it to nbd_connect_socket(3).
419 One use for this is in fuzzing where we use socketpair(2) to create the
421 socket, then fork, then have the test harness in the child process
422 connected to libnbd over the socket pair (see:
423 https://gitlab.com/nbdkit/libnbd/-/blob/master/fuzzing/libnbd-fuzz-wrapper.c).
424 Another use is to connect libnbd to an address family that it does not
426 support natively, such as XDP or IB.
427
429 By default, when beginning a connection, libnbd will handle all
430 negotiation with the server, using only the configuration (eg.
431 nbd_set_export_name(3) or nbd_add_meta_context(3)) that was requested
432 before the connection attempt; this phase continues until
433 nbd_aio_is_connecting(3) no longer returns true, at which point, either
434 data commands are ready to use or else the connection has failed with
435 an error.
436 But there are scenarios in which it is useful to also control the
438 handshaking commands sent during negotiation, such as asking the server
439 for a list of available exports prior to selecting which one to use.
440 This is done by calling nbd_set_opt_mode(3) before connecting; then
441 after requesting a connection, the state machine will pause at
442 nbd_aio_is_negotiating(3) at any point that the user can decide which
443 handshake command to send next. Note that the negotiation state is
444 only reachable from newstyle servers; older servers cannot negotiate
445 and will progress all the way to the ready state.
446 When the negotiating state is reached, you can initiate option commands
448 such as nbd_opt_list(3) or their asynchronous equivalents, as well as
449 alter configuration such as export name that previously had to be set
450 before connection. Since the NBD protocol does not allow parallel
451 negotiating commands, no cookie is involved, and you can track
452 completion of each command when the state is no longer
453 nbd_aio_is_connecting(3). If nbd_opt_go(3) fails but the connection is
454 still live, you will be back in negotiation state, where you can
455 request a different export name and try again. Exiting the negotiation
456 state is only possible with a successful nbd_opt_go(3) which moves to
457 the data phase, or nbd_opt_abort(3) which performs a clean shutdown of
458 the connection by skipping the data phase.
459
461 It is possible for NBD servers to serve different content on different
462 “exports”. For this you must pass the right export name to the server.
463 Call this API before connecting:
464 nbd_set_export_name (nbd, "export");
466 Note that there are some servers (like nbdkit(1) ≤ 1.14) which ignore
468 this, and other servers (like qemu-nbd(8)) which require it to be set
469 correctly but cannot serve different content.
470 These APIs are also available after a successful nbd_opt_info(3) during
472 the negotiation phase, if you used nbd_set_opt_mode(3) prior to
473 connecting.
474 Flag calls
476 After connecting the server will send back a set of flags describing
477 the export, such as whether it is writable and if it can support flush
478 to permanent storage. These flags can be accessed from libnbd using
479 APIs such as:
480 int is_read_only = nbd_is_read_only (nbd);
482 int can_flush = nbd_can_flush (nbd);
483 Flag calls are: nbd_can_cache(3), nbd_can_df(3), nbd_can_fast_zero(3),
485 nbd_can_flush(3), nbd_can_fua(3), nbd_can_meta_context(3),
486 nbd_can_multi_conn(3), nbd_can_trim(3), nbd_can_zero(3),
487 nbd_is_read_only(3), nbd_is_rotational(3).
488 Size of the export
490 To get the size of the export in bytes, use nbd_get_size(3):
491 int64_t size = nbd_get_size (nbd);
493 Block size constraints
495 Some NBD servers cannot handle requests at any byte boundary. They
496 might, for example, require all requests to be aligned to 512 byte
497 sectors.
498 Also some servers advertise a preferred block size. This is not a
500 requirement, but is the minimum block size that can be accessed
501 efficiently (usually without triggering expensive read-modify-write
502 cycles inside the server).
503 These are referred to as block size constraints and can be queried by
505 calling nbd_get_block_size(3). Pay attention in particular to the
506 "LIBNBD_SIZE_MINIMUM" constraint as some servers will fail requests
507 which are smaller or not aligned to this block size with "EINVAL"
508 ("Invalid argument") errors.
509
511 You can read and write data from the NBD server using nbd_pread(3) and
512 nbd_pwrite(3) or their asynchronous equivalents.
513 All data commands support a "flags" argument (mandatory in C, but
515 optional in languages where it can default to 0). For convenience, the
516 constant "LIBNBD_CMD_FLAG_MASK" is defined with the set of flags
517 currently recognized by libnbd, where future NBD protocol extensions
518 may result in additional flags being supported; but in general,
519 specific data commands only accept a subset of known flags.
520 Libnbd defaults to performing some client-side sanity checking in each
522 of its data commands; for example, attempts to write to a server that
523 has advertised a read-only connection are rejected. It is possible to
524 override aspects of this checking by using nbd_set_strict_mode(3).
525 Some servers also support:
527 trim/discard
529 If nbd_can_trim(3) returns true, nbd_trim(3) can be used to “punch
530 holes” in the backing storage of the disk on the server. Normally
531 (although not in every case) the holes read back as zeroes but take
532 up no space.
533 zeroing
535 If nbd_can_zero(3) returns true, nbd_zero(3) can be used to
536 efficiently zero parts of the disk without having to send large
537 amounts of zero bytes over the network (as would be necessary if
538 using nbd_pwrite(3)).
539 This is slightly different from trimming because the backing
541 storage is still allocated. For some storage types this can make
542 future writes more efficient and/or less likely to fail because of
543 out of space errors.
544 flushing
546 Some servers can commit data to permanent storage and tell you that
547 this has happened reliably. There are two export flags associated
548 with this: nbd_can_flush(3) and nbd_can_fua(3).
549 The nbd_flush(3) call (available if nbd_can_flush(3) returns true)
551 flushes all pending writes to disk and does not complete until that
552 operation has finished. It is similar to using sync(2) on POSIX
553 systems.
554 A more efficient way to achieve this is to set the flag
556 "LIBNBD_CMD_FLAG_FUA" on write-like calls (like write, trim and
557 zero). This flag means the call will not complete until committed
558 to permanent storage, but it does not involve flushing the entire
559 disk.
560 prefetching
562 Some servers can prefetch data, making subsequent reads faster.
563 The nbd_cache(3) call (available if nbd_can_cache(3) returns true)
564 is used to prefetch.
565 block status
567 Some servers are able to provide information about the various
568 extents within the image, via the notion of one or more meta
569 contexts. The most common meta context is "base:allocation"
570 (available in libnbd.h as "LIBNBD_CONTEXT_BASE_ALLOCATION"), which
571 can be used to learn which portions of a file are allocated or read
572 as zero. Other contexts may be available; for example, qemu-nbd(8)
573 can expose a meta context "qemu:dirty-bitmap:NAME" for tracking
574 which portions of a file are tracked by a qcow2 dirty bitmap.
575 In order to utilize block status, the client must call
577 nbd_add_meta_context(3) prior to connecting, for each meta context
578 in which it is interested, then check nbd_can_meta_context(3) after
579 connection to see which contexts the server actually supports. If
580 a context is supported, the client can then use nbd_block_status(3)
581 with a callback function that will receive an array of 32-bit
582 integer pairs describing consecutive extents within a context. In
583 each pair, the first integer is the length of the extent, the
584 second is a bitmask description of that extent (for the
585 "base:allocation" context, the bitmask may include
586 "LIBNBD_STATE_HOLE" for unallocated portions of the file, and/or
587 "LIBNBD_STATE_ZERO" for portions of the file known to read as
588 zero).
589 There is a full example of requesting meta context and using block
591 status available at
592 https://gitlab.com/nbdkit/libnbd/blob/master/interop/dirty-bitmap.c
593
595 Issuing multiple in-flight requests
596 NBD servers which properly implement the specification can handle
597 multiple data requests in flight over the same connection at the same
598 time. Libnbd supports this when using the low level API.
599 To use it you simply issue more requests as needed (eg. using calls
601 like nbd_aio_pread(3), nbd_aio_pwrite(3)) without waiting for previous
602 commands to complete. You need to be careful that requests in flight
603 do not overlap with disk offsets of other write-like commands in flight
604 — an overlapping read may see indeterminate data, and an overlapping
605 write may even cause disk corruption where the resulting disk contents
606 do not match either of the two writes.
607 Each request is identified by a unique 64 bit cookie (assigned by
609 libnbd), allowing libnbd and callers to match replies to requests.
610 Replies may arrive out of order. A request that is rejected client-
611 side for failing a sanity check (such as attempting to write to a read-
612 only server, see nbd_set_strict_mode(3)) will fail rather than
613 returning a cookie, although closure cleanup is still performed.
614 Although in theory you can have an indefinite number of requests in
616 flight at the same time, in practice it's a good idea to limit them to
617 some number. Libnbd will queue commands in the handle even if it
618 cannot write them to the server, so this limit is largely to prevent a
619 backlog of commands from consuming too much memory. It is suggested to
620 start with a limit of 64 requests in flight (per NBD connection), and
621 measure how adjusting the limit up and down affects performance for
622 your local configuration.
623 There is a full example using multiple in-flight requests available at
625 https://gitlab.com/nbdkit/libnbd/blob/master/examples/threaded-reads-and-writes.c
626 Multi-conn
628 Some NBD servers advertise “multi-conn” which means that it is safe to
629 make multiple connections to the server and load-balance commands
630 across all of the connections.
631 To do this you should open a single connection first and test for this
633 feature using nbd_can_multi_conn(3). Without error handling it would
634 look like this:
635 struct nbd_handle *nbd[4];
637 size_t i;
638 bool supports_multi_conn;
639 nbd[0] = nbd_create ();
641 nbd_connect_tcp (nbd[0], "server", "10809");
642 supports_multi_conn = nbd_can_multi_conn (nbd[0]) > 0;
643 If multi-conn is supported then you can open further connections:
645 if (supports_multi_conn) {
647 for (i = 1; i <= 3; ++i) {
648 nbd[i] = nbd_create ();
649 nbd_connect_tcp (nbd[i], "server", "10809");
650 }
651 }
652 If you are issuing multiple in-flight requests (see above) and limiting
654 the number, then the limit should be applied to each individual NBD
655 connection.
656
658 The NBD protocol and libnbd supports TLS (sometimes incorrectly called
659 “SSL”) for encryption of the data stream and authentication of clients
660 and servers. Libnbd defaults to TLS disabled for maximum
661 interoperability. To enable it on a handle you must call
662 nbd_set_tls(3) before connecting.
663 To allow TLS, but fall back to unencrypted:
665 nbd_set_tls (nbd, LIBNBD_TLS_ALLOW);
667 Use nbd_get_tls_negotiated(3) to find out if TLS negotiation was
669 successful. Avoid "LIBNBD_TLS_ALLOW" if man-in-the-middle attacks are
670 a concern.
671 The most secure mode is to require TLS and fail to connect if the
673 server does not support it:
674 nbd_set_tls (nbd, LIBNBD_TLS_REQUIRE);
676 It may also be necessary to verify that the server’s identity is
678 correct. For some servers it may be necessary to verify to the server
679 that the client is permitted to connect. This can be done using either
680 X.509 certificates, or TLS Pre-Shared Keys (PSK). Certificates are
681 more secure. PSK is far more convenient, but you must have an existing
682 secure channel to distribute the keys.
683 Setting up X.509 using system certificate authorities (CAs)
685 This is the default if you don’t call any other "nbd_set_tls_*"
686 functions. In this case the server must have a public (eg. HTTPS)
687 certificate which can be verified against the CAs registered on your
688 system (eg. under /etc/pki).
689 To disable server name verification — which opens you up to a potential
691 Man-In-The-Middle (MITM) attack — use:
692 nbd_set_tls_verify_peer (nbd, false);
694 Setting up an X.509 certificate authority (CA)
696 You can set up your own CA and register clients and servers with it,
697 issuing client and server certificates which will reliably authenticate
698 your clients and servers to each other.
699 Doing this is described in detail in the nbdkit-tls(1) manual. The
701 only differences for libnbd are:
702 • Non-root certificates must be placed in "$HOME/.pki/libnbd/" or
704 "$HOME/.config/pki/libnbd/"
705 • Libnbd reads client-cert.pem and client-key.pem (instead of
707 server-cert.pem and server-key.pem).
708 Once you have set up the directory containing the certificates, call:
710 nbd_set_tls_certificates (nbd, "/path/to/directory");
712 Setting up Pre-Shared Keys (PSK)
714 TLS Pre-Shared Keys are a much more convenient method of setting up
715 TLS, and more appropriate for NBD, but you should have an existing
716 secure method available to distribute the keys. They are therefore
717 ideal if you want to set up an NBD service as an adjunct to an existing
718 secure REST API.
719 Use psktool(1) to create a file of "username:key" pairs:
721 psktool -u username -p keys.psk
723 and pass this path to libnbd:
725 nbd_set_tls_psk_file (nbd, "keys.psk");
727 If necessary you may need to set the client username (otherwise libnbd
729 will use your login name):
730 nbd_set_tls_username (nbd, "username");
732
734 Some libnbd calls take callbacks (eg. nbd_set_debug_callback(3),
735 nbd_aio_pread(3)). Libnbd can call these functions while processing.
736 In the C API these libnbd calls take a structure which contains the
738 function pointer and an optional opaque "void *user_data" pointer:
739 nbd_aio_pread (nbd, buf, sizeof buf, offset,
741 (nbd_completion_callback) { .callback = my_fn,
742 .user_data = my_data },
743 0);
744 For optional callbacks, if you don't want the callback, either set
746 ".callback" to "NULL" or use the equivalent macros (such as
747 "NBD_NULL_COMPLETION") defined in "libnbd.h":
748 nbd_aio_pread (nbd, buf, sizeof buf, offset,
750 NBD_NULL_COMPLETION, 0);
751 From other languages the structure and opaque pointer are not needed
753 because you can use closures to achieve the same effect.
754 Callback lifetimes
756 You can associate an optional free function with callbacks. Libnbd
757 will call this function when the callback will not be called again by
758 libnbd, including in the case where the API fails.
759 This can be used to free associated "user_data". For example:
761 void *my_data = malloc (...);
763 nbd_aio_pread_structured (nbd, buf, sizeof buf, offset,
765 (nbd_chunk_callback) { .callback = my_fn,
766 .user_data = my_data,
767 .free = free },
768 NBD_NULL_COMPLETION,
769 0);
770 will call free(3) once on "my_data" after the point where it is known
772 that the "chunk.callback = my_fn" function can no longer be called,
773 regardless of how many times "my_fn" was actually called. If both a
774 mid-command and completion callback are supplied, the functions will be
775 reached in this order: mid-function callbacks, completion callback,
776 mid-function free, and finally completion free.
777 The free function is only accessible in the C API as it is not needed
779 in garbage collected programming languages.
780 Callbacks with ".callback=NULL" and ".free!=NULL"
782 It is possible to register a callback like this:
783 ...
785 (nbd_completion_callback) { .callback = NULL,
786 .user_data = my_data,
787 .free = free },
788 ...
789 The meaning of this is that the callback is never called, but the free
791 function is still called after the last time the callback would have
792 been called. This is useful for applying generic freeing actions when
793 asynchronous commands are retired.
794 Callbacks and locking
796 The callbacks are invoked at a point where the libnbd lock is held; as
797 such, it is unsafe for the callback to call any "nbd_*" APIs on the
798 same nbd object, as it would cause deadlock.
799 Completion callbacks
801 All of the asychronous commands have an optional completion callback
802 function that is used right before the command is marked complete,
803 after any mid-command callbacks have finished, and before any free
804 functions.
805 When the completion callback returns 1, the command is automatically
807 retired (there is no need to call nbd_aio_command_completed(3)); for
808 any other return value, the command still needs to be manually retired
809 (otherwise, the command will tie up resources until nbd_close(3) is
810 eventually reached).
811 Callbacks with "int *error" parameter
813 Some of the high-level commands (nbd_pread_structured(3),
814 nbd_block_status(3)) involve the use of a callback function invoked by
815 the state machine at appropriate points in the server's reply before
816 the overall command is complete. These callback functions, along with
817 all of the completion callbacks, include a parameter "error" which is a
818 pointer containing the value of any error detected so far. If a
819 callback function fails and wants to change the resulting error of the
820 overall command visible later in the API sequence, it should assign
821 back into "error" and return "-1" in the C API. Assignments into
822 "error" are ignored for any other return value; similarly, assigning 0
823 into "error" does not have an effect. In other language bindings,
824 reporting callback errors is generally done by raising an exception
825 rather than by return value.
826 Note that a mid-command callback might never be reached, such as if
828 libnbd detects that the command was invalid to send (see
829 nbd_set_strict_mode(3)) or if the server reports a failure that
830 concludes the command. It is safe for a mid-command callback to ignore
831 non-zero "error": all the other parameters to the mid-command callback
832 will still be valid (corresponding to the current portion of the
833 server's reply), and the overall command will still fail (at the
834 completion callback or nbd_aio_command_completed(3) for an asynchronous
835 command, or as the result of the overall synchronous command).
836 Returing "-1" from a mid-command callback does not prevent that
837 callback from being reached again, if the server sends more mid-command
838 replies that warrant another use of that callback. A mid-command
839 callback may be reached more times than expected if the server is non-
840 compliant.
841 On the other hand, if a completion callback is supplied (only possible
843 with asynchronous commands), it will always be reached exactly once,
844 and the completion callback must not ignore the value pointed to by
845 "error". In particular, the content of a buffer passed to
846 nbd_aio_pread(3) or nbd_aio_pread_structured(3) is undefined if *error
847 is non-zero on entry to the completion callback. It is recommended
848 that if you choose to use automatic command retirement (where the
849 completion callback returns 1 to avoid needing to call
850 nbd_aio_command_completed(3) later), your completion function should
851 return 1 on all control paths, even when handling errors (note that
852 with automatic retirement, assigning into "error" is pointless as there
853 is no later API to see that value).
854
856 On most systems, C programs that use libnbd can be compiled like this:
857 cc prog.c -o prog -lnbd
859 To detect if the libnbd library and header file is installed, the
861 preferred method is to use pkg-config(1) or pkgconf(1):
862 pkg-config libnbd --exists || fail libnbd is required
864 In case the library or header file are not installed in the usual
866 system locations, you can compile your program like this, using pkg-
867 config to detect the proper location of libnbd:
868 cc prog.c -o prog `pkg-config libnbd --cflags --libs`
870 To compile an external project against a built copy of the libnbd
872 source tree which hasn't been installed, see the ./run script.
873 Autoconf projects
875 External projects which use autoconf and need to check if libnbd is
876 installed should use the "PKG_CHECK_MODULES" macro in configure.ac like
877 this:
878 PKG_CHECK_MODULES([LIBNBD], [libnbd])
880 This will define "@LIBNBD_CFLAGS@" and "@LIBNBD_LIBS@" which you will
882 need to add to your Makefile.am.
883 CMake projects
885 For CMake projects use:
886 find_package(PkgConfig REQUIRED)
888 pkg_check_modules(LIBNBD REQUIRED libnbd)
889 target_link_libraries(prog ${LIBNBD_LIBRARIES})
890 target_include_directories(prog PUBLIC ${LIBNBD_INCLUDE_DIRS})
891 target_compile_options(prog PUBLIC ${LIBNBD_CFLAGS_OTHER})
892 Meson projects
894 For meson projects use:
895 nbd_dep = dependency('libnbd')
897 executable('prog', 'prog.c', dependencies : [nbd_dep])
898
900 "HOME"
901 Used in some situations to find TLS certificates. See
902 nbd_set_tls_certificates(3).
903 "LIBNBD_DEBUG"
905 If this is set to the exact string 1 when the handle is created
906 then debugging is enabled. See "DEBUGGING MESSAGES" above.
907 "LOGNAME"
909 The default TLS username. See nbd_set_tls_username(3).
910
912 C API
913 nbd_add_meta_context(3), nbd_aio_block_status(3), nbd_aio_cache(3),
914 nbd_aio_command_completed(3), nbd_aio_connect(3),
915 nbd_aio_connect_command(3), nbd_aio_connect_socket(3),
916 nbd_aio_connect_systemd_socket_activation(3), nbd_aio_connect_tcp(3),
917 nbd_aio_connect_unix(3), nbd_aio_connect_uri(3),
918 nbd_aio_connect_vsock(3), nbd_aio_disconnect(3), nbd_aio_flush(3),
919 nbd_aio_get_direction(3), nbd_aio_get_fd(3), nbd_aio_in_flight(3),
920 nbd_aio_is_closed(3), nbd_aio_is_connecting(3), nbd_aio_is_created(3),
921 nbd_aio_is_dead(3), nbd_aio_is_negotiating(3),
922 nbd_aio_is_processing(3), nbd_aio_is_ready(3), nbd_aio_notify_read(3),
923 nbd_aio_notify_write(3), nbd_aio_opt_abort(3), nbd_aio_opt_go(3),
924 nbd_aio_opt_info(3), nbd_aio_opt_list(3),
925 nbd_aio_opt_list_meta_context(3), nbd_aio_peek_command_completed(3),
926 nbd_aio_pread(3), nbd_aio_pread_structured(3), nbd_aio_pwrite(3),
927 nbd_aio_trim(3), nbd_aio_zero(3), nbd_block_status(3), nbd_cache(3),
928 nbd_can_cache(3), nbd_can_df(3), nbd_can_fast_zero(3),
929 nbd_can_flush(3), nbd_can_fua(3), nbd_can_meta_context(3),
930 nbd_can_multi_conn(3), nbd_can_trim(3), nbd_can_zero(3),
931 nbd_clear_debug_callback(3), nbd_clear_meta_contexts(3), nbd_close(3),
932 nbd_connect_command(3), nbd_connect_socket(3),
933 nbd_connect_systemd_socket_activation(3), nbd_connect_tcp(3),
934 nbd_connect_unix(3), nbd_connect_uri(3), nbd_connect_vsock(3),
935 nbd_connection_state(3), nbd_create(3), nbd_flush(3),
936 nbd_get_block_size(3), nbd_get_canonical_export_name(3),
937 nbd_get_debug(3), nbd_get_errno(3), nbd_get_error(3),
938 nbd_get_export_description(3), nbd_get_export_name(3),
939 nbd_get_full_info(3), nbd_get_handle_name(3),
940 nbd_get_handshake_flags(3), nbd_get_meta_context(3),
941 nbd_get_nr_meta_contexts(3), nbd_get_opt_mode(3),
942 nbd_get_package_name(3), nbd_get_pread_initialize(3),
943 nbd_get_private_data(3), nbd_get_protocol(3),
944 nbd_get_request_block_size(3), nbd_get_request_structured_replies(3),
945 nbd_get_size(3), nbd_get_strict_mode(3),
946 nbd_get_structured_replies_negotiated(3), nbd_get_tls(3),
947 nbd_get_tls_negotiated(3), nbd_get_tls_username(3),
948 nbd_get_tls_verify_peer(3), nbd_get_uri(3), nbd_get_version(3),
949 nbd_is_read_only(3), nbd_is_rotational(3), nbd_kill_subprocess(3),
950 nbd_opt_abort(3), nbd_opt_go(3), nbd_opt_info(3), nbd_opt_list(3),
951 nbd_opt_list_meta_context(3), nbd_poll(3), nbd_pread(3),
952 nbd_pread_structured(3), nbd_pwrite(3), nbd_set_debug(3),
953 nbd_set_debug_callback(3), nbd_set_export_name(3),
954 nbd_set_full_info(3), nbd_set_handle_name(3),
955 nbd_set_handshake_flags(3), nbd_set_opt_mode(3),
956 nbd_set_pread_initialize(3), nbd_set_private_data(3),
957 nbd_set_request_block_size(3), nbd_set_request_structured_replies(3),
958 nbd_set_strict_mode(3), nbd_set_tls(3), nbd_set_tls_certificates(3),
959 nbd_set_tls_psk_file(3), nbd_set_tls_username(3),
960 nbd_set_tls_verify_peer(3), nbd_set_uri_allow_local_file(3),
961 nbd_set_uri_allow_tls(3), nbd_set_uri_allow_transports(3),
962 nbd_shutdown(3), nbd_supports_tls(3), nbd_supports_uri(3), nbd_trim(3),
963 nbd_zero(3).
964 Servers
966 nbdkit(1), nbd-server(1), qemu-nbd(8).
967 Encryption tools
969 certtool(1), nbdkit-tls(1), psktool(1).
970 Standards
972 https://github.com/NetworkBlockDevice/nbd/blob/master/doc/proto.md,
973 https://github.com/NetworkBlockDevice/nbd/blob/master/doc/uri.md.
974 Release notes
976 libnbd-release-notes-1.12(1), libnbd-release-notes-1.10(1),
977 libnbd-release-notes-1.8(1), libnbd-release-notes-1.6(1),
978 libnbd-release-notes-1.4(1), libnbd-release-notes-1.2(1).
979 Other
981 libnbd-security(3), nbdcopy(1), nbdfuse(1), nbdinfo(1), nbdsh(1),
982 qemu(1).
983
985 Eric Blake
986 Richard W.M. Jones
988
990 Copyright (C) 2019-2021 Red Hat Inc.
991
993 This library is free software; you can redistribute it and/or modify it
994 under the terms of the GNU Lesser General Public License as published
995 by the Free Software Foundation; either version 2 of the License, or
996 (at your option) any later version.
997 This library is distributed in the hope that it will be useful, but
999 WITHOUT ANY WARRANTY; without even the implied warranty of
1000 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
1001 Lesser General Public License for more details.
1002 You should have received a copy of the GNU Lesser General Public
1004 License along with this library; if not, write to the Free Software
1005 Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
1006 02110-1301 USA
1007libnbd-1.12.5 2022-07-10 libnbd(3)