1libnbd(3) LIBNBD libnbd(3)
2
6 libnbd - network block device (NBD) client library in userspace
7
9 #include <libnbd.h>
10 struct nbd_handle *nbd;
12 char buf[512];
13 if ((nbd = nbd_create ()) == NULL ||
15 nbd_connect_tcp (nbd, "server.example.com", "nbd") == -1 ||
16 nbd_pread (nbd, buf, sizeof buf, 0, 0) == -1)
17 fprintf (stderr, "%s\n", nbd_get_error ());
18 nbd_close (nbd);
19 exit (EXIT_FAILURE);
20 }
21 nbd_close (nbd);
22 cc prog.c -o prog -lnbd
24 or:
25 cc prog.c -o prog `pkg-config libnbd --cflags --libs`
26
28 Network Block Device (NBD) is a network protocol for accessing block
29 devices over the network. Block devices are hard disks and things that
30 behave like hard disks such as disk images and virtual machines.
31 Libnbd is a client library for the NBD protocol which can access most
33 of the features of NBD while being simple to use and powerful.
34 This manual page gives an overview of libnbd, using C as an example,
36 but the library is available from other programming languages.
37 nbd_create(3), nbd_pread(3), etc.
39 Each manual page covers one function from the C API in detail.
40 There is a full list in section "C API" below.
41 libnbd-ocaml(3)
43 Using the API from OCaml.
44 libnbd-golang(3)
46 Using the API from Go.
47 nbdsh(1)
49 Using the NBD shell (nbdsh) for command line and Python scripting.
50
52 To use the API at all you must first open a handle by calling
53 nbd_create(3) (or its equivalent in other languages):
54 struct nbd_handle *nbd;
56 nbd = nbd_create ();
58 This creates and returns a handle, which is associated with one
60 connection to an NBD server, initially not connected.
61 Each handle is a complex state machine which can be in states such as
63 created, connected to a remote server, handshaking, idle and ready to
64 issue commands, or busy sending or receiving commands.
65 Handles have a name used in debugging messages. The name is normally
67 generated ("nbd1", "nbd2" etc) but you can set a friendly name with
68 nbd_set_handle_name(3). Also there is a private field in the handle
69 for use by the application, see nbd_set_private_data(3).
70 When you have finished with the handle you must call nbd_close(3) which
72 closes the underlying socket (if necessary) and frees up all associated
73 resources.
74
76 There are two levels of API available. A simple high level synchronous
77 API lets you give the handle high level instructions like “connect to
78 the server”, “read a block”, “write a block”, etc. Each of these
79 functions will run to completion, blocking the current thread before
80 returning. A more complicated low level non-blocking asynchronous API
81 is also available where you can integrate with poll(2) or another main
82 loop.
83 You can freely mix the two APIs on the same handle. You can also call
85 APIs on a single handle from multiple threads. Single API calls on the
86 handle are atomic — they either take a lock on the handle while they
87 run or are careful to access handle fields atomically.
88 Libnbd does not create its own threads.
90
92 This is the simplest way to use the API, with the possible drawback
93 that each libnbd function blocks until it is finished.
94 Create a handle and connect to the server:
96 struct nbd_handle *nbd;
98 nbd = nbd_create ();
100 if (!nbd) {
101 fprintf (stderr, "%s\n", nbd_get_error ());
102 nbd_close (nbd);
103 exit (EXIT_FAILURE);
104 }
105 if (nbd_connect_tcp (nbd, "server.example.com", "nbd") == -1) {
106 fprintf (stderr, "%s\n", nbd_get_error ());
107 nbd_close (nbd);
108 exit (EXIT_FAILURE);
109 }
110 Read the first sector (512 bytes) from the NBD export:
112 char buf[512];
114 if (nbd_pread (nbd, buf, sizeof buf, 0, 0) == -1) {
116 fprintf (stderr, "%s\n", nbd_get_error ());
117 nbd_close (nbd);
118 exit (EXIT_FAILURE);
119 }
120 Close the handle:
122 nbd_close (nbd);
124 You can call the high level API from multiple threads, but each libnbd
126 API call takes a lock on the handle and so commands will not run in
127 parallel.
128
130 The low level API is useful if you want to use libnbd in non-blocking
131 code; or if you want to issue commands in parallel from multiple
132 threads; or if you need more control especially over having multiple
133 commands in-flight on a single connection.
134 To use the low level API you will need to integrate with poll(2) or
136 another “main loop” such as the GLib main event loop.
137 Issuing asynchronous commands
139 Use the "nbd_aio_*" variants to issue commands asynchronously (without
140 waiting for the command to complete before returning). For example the
141 asynchronous variant of nbd_pread(3) is:
142 int64_t cookie;
144 cookie = nbd_aio_pread (nbd, buf, sizeof buf,
146 NBD_NULL_COMPLETION, 0);
147 if (cookie == -1) {
148 fprintf (stderr, "%s\n", nbd_get_error ());
149 nbd_close (nbd);
150 exit (EXIT_FAILURE);
151 }
152 There are several things to note here:
154 • This only starts the command. The command is still in flight when
156 the call returns.
157 • A buffer ("buf") has been assigned to collect the result of the
159 read, but it is not guaranteed to be filled with data until the
160 command has completed (see examples below). The buffer must not be
161 freed until the command has finished running.
162 • You can issue multiple commands on the same handle at the same
164 time.
165 • A cookie is returned which identifies this command in subsequent
167 calls. The cookie is unique (per libnbd handle) and ≥ 1.
168 • You may register a function which is called when the command
170 completes, see "Completion callbacks" below. In this case we have
171 specified a null completion callback.
172 Socket and direction
174 Each libnbd handle has an associated socket (once it has started
175 connecting). You can read the file descriptor of the socket using:
176 int fd = nbd_aio_get_fd (nbd);
178 The socket is non-blocking. Between calls into libnbd it is in the
180 "would block" condition. You can find out if libnbd is expecting to
181 read or write from the socket next by calling:
182 int dir = nbd_aio_get_direction (nbd);
184 which returns one of "LIBNBD_AIO_DIRECTION_READ",
186 "LIBNBD_AIO_DIRECTION_WRITE" or "LIBNBD_AIO_DIRECTION_BOTH" (=
187 "READ|WRITE"). And so to set up the next call to poll(2) or other main
188 loop you must translate this to "POLLIN", "POLLOUT" or "POLLIN|POLLOUT"
189 (or whatever mechanism your main loop uses).
190 Notifying libnbd when an event happens
192 When you detect (eg. using poll(2)) that a read or write event has
193 happened on the socket, you must then tell libnbd about it. You have
194 to check the direction again (since it may have been changed by another
195 thread), and notify libnbd:
196 int r = 0;
198 dir = nbd_aio_get_direction (nbd);
200 if ((dir & LIBNBD_AIO_DIRECTION_READ) &&
202 a_read_event_occurred ())
203 r = nbd_aio_notify_read (nbd);
204 else if ((dir & LIBNBD_AIO_DIRECTION_WRITE) &&
205 a_write_event_occurred ())
206 r = nbd_aio_notify_write (nbd);
207 if (r == -1) {
209 fprintf (stderr, "%s\n", nbd_get_error ());
210 // ...
211 }
212 The notify calls move the state machine along, reading and writing from
214 the socket possibly multiple times, until the socket would block again,
215 at which point they return control to the caller.
216 Simple implementation with nbd_poll(3)
218 In fact if you want to use poll(2) on a single handle, a simple
219 implementation has already been written called nbd_poll(3). It is also
220 useful to examine how this is implemented (lib/poll.c in the libnbd
221 source code) because that will tell you how to integrate libnbd with
222 more complex main loops.
223 Some examples of using nbd_poll(3) follow.
225 As with the high level API, it all starts by creating a handle:
227 struct nbd_handle *nbd;
229 nbd = nbd_create ();
231 if (nbd == NULL) {
232 fprintf (stderr, "%s\n", nbd_get_error ());
233 nbd_close (nbd);
234 exit (EXIT_FAILURE);
235 }
236 To connect to the server asynchronously, we start the connection using
238 nbd_aio_connect(3) and then enter our main loop to check for events
239 until the connection becomes ready:
240 int fd;
242 struct sockaddr_un addr;
243 socklen_t len;
244 /* some code to set up addr,
246 then ... */
247 if (nbd_aio_connect (nbd, &addr, len) == -1) {
248 fprintf (stderr, "%s\n", nbd_get_error ());
249 nbd_close (nbd);
250 exit (EXIT_FAILURE);
251 }
252 while (! nbd_aio_is_ready (nbd)) {
253 if (nbd_poll (nbd, -1) == -1) {
254 fprintf (stderr, "%s\n", nbd_get_error ());
255 nbd_close (nbd);
256 exit (EXIT_FAILURE);
257 }
258 }
259 To read data asynchronously, start an asynchronous read command, which
261 returns a 64 bit command cookie, and enter the main loop until the
262 command has completed:
263 int64_t cookie;
265 char buf[512];
266 cookie = nbd_aio_pread (nbd, buf, sizeof buf, offset,
268 NBD_NULL_COMPLETION, 0);
269 if (cookie == -1) {
270 fprintf (stderr, "%s\n", nbd_get_error ());
271 nbd_close (nbd);
272 exit (EXIT_FAILURE);
273 }
274 while (! nbd_aio_command_completed (nbd, cookie)) {
275 if (nbd_poll (nbd, -1) == -1) {
276 fprintf (stderr, "%s\n", nbd_get_error ());
277 nbd_close (nbd);
278 exit (EXIT_FAILURE);
279 }
280 }
281 For almost all high level synchronous calls (eg. nbd_pread(3)) there is
283 a low level asynchronous equivalent (eg. nbd_aio_pread(3)) for starting
284 a command.
285 glib2 integration
287 See
288 https://gitlab.com/nbdkit/libnbd/blob/master/examples/glib-main-loop.c
289 libev integration
291 See https://gitlab.com/nbdkit/libnbd/blob/master/examples/copy-libev.c
292
294 When any API call returns an error ("-1" or "NULL" depending on the
295 API), an error message and sometimes an errno value are available. You
296 can retrieve the error message and/or errno of the most recently failed
297 call using nbd_get_error(3) and nbd_get_errno(3). For example:
298 if (nbd_connect_tcp (nbd, "remote", "nbd") == -1) {
300 fprintf (stderr,
301 "failed to connect to remote server: %s (errno = %d)\n",
302 nbd_get_error (), nbd_get_errno ());
303 }
304 These functions use thread-local storage to return the most recent
306 error in the current thread. This is why you don't need to pass the
307 handle to these calls. They even work if nbd_create(3) returns "NULL"
308 when there is no handle at all.
309 For this reason you cannot call them from a different thread. You
311 should call them immediately after the failed API call, from the same
312 thread. Furthermore the error string returned by nbd_get_error(3) is
313 only valid until the next libnbd API call in the current thread, so if
314 you need to keep the string you must copy it (eg. using strdup(3)).
315 Errno
317 For some errors, a system call error number (see errno(3)) is
318 available. You can find the error number by calling nbd_get_errno(3).
319 It works the same way as nbd_get_error(3) with respect to threads.
320 Even when a call returns an error, nbd_get_errno(3) might return 0.
322 This does not mean there was no error. It means no additional errno
323 information is available for this error.
324 The error number is often the raw error returned by a system call that
326 failed.
327 It can also be used to indicate special conditions. The most common
329 cases are:
330 "EINVAL"
332 Invalid parameters or state for the current libnbd call. (This can
333 also indicate that requests are not aligned to "Block size
334 constraints").
335 "ENOTSUP"
337 The libnbd call is not available in this build of libnbd (eg. when
338 using a TLS API if the library was compiled without TLS support).
339 "ENOMEM"
341 The library ran out of memory while performing some operation.
342 "ERANGE"
344 A request is too large, for example if you try to read too many
345 bytes in a single nbd_pread(3) call.
346 "EFAULT"
348 A pointer parameter was "NULL" when it should be non-NULL. See the
349 section below.
350 Non-NULL parameters
352 Almost all libnbd functions when called from C take one or more pointer
353 parameters that must not be "NULL". For example, the handle parameter,
354 strings and buffers should usually not be "NULL".
355 If a "NULL" is passed as one of these parameters, libnbd attempts to
357 return an error with nbd_get_errno(3) returning "EFAULT".
358 However it may cause other compiler-related warnings and even undefined
360 behaviour, so you should try to avoid this programming mistake.
361
363 Libnbd can print lots of debugging messages, useful if you have a
364 problem with the library. Either enable debugging after creating the
365 handle:
366 nbd = nbd_create ();
368 nbd_set_debug (nbd, true);
369 or set the "LIBNBD_DEBUG=1" environment variable which will enable
371 debugging by default on all new handles.
372 Debugging messages are sent to stderr by default, but you can redirect
374 them to a logging system using nbd_set_debug_callback(3).
375
377 There are several ways to connect to NBD servers, and you can even run
378 a server from libnbd. Normally you would connect to a server which is
379 already running, over a local Unix domain socket or a remote TCP
380 connection. The high level API calls are:
381 nbd_connect_unix (nbd, "socket");
383 nbd_connect_tcp (nbd, "localhost", "nbd");
384 For nbd_connect_tcp(3) the third parameter is the port name or number,
386 which can either be a name from /etc/services or the port number as a
387 string (eg. "10809").
388 Connecting to an NBD URI
390 libnbd supports the NBD URI specification. The format of URIs is
391 documented in nbd_connect_uri(3).
392 You can connect to a URI as in these examples (using the high level
394 API):
395 nbd_connect_uri (nbd, "nbd://example.com/");
397 nbd_connect_uri (nbd, "nbds+unix:///export?socket=/tmp/nbd.sock");
399 This feature is implemented by calling other libnbd APIs to set up the
401 export name, TLS parameters, and finally connect over a Unix domain
402 socket or TCP.
403 URI support is an optional feature of the library, requiring libxml2 at
405 compile time. The nbd_connect_uri(3) and nbd_aio_connect_uri(3) calls
406 will raise an error (with nbd_get_errno(3) returning "ENOTSUP") if it
407 was not built with this feature, and you can also test for it
408 explicitly using nbd_supports_uri(3).
409 Connecting to a subprocess
411 Some NBD servers — notably nbdkit(1) with the -s parameter, and
412 nbd-server(1) with the port parameter set to 0 — can also accept a
413 single NBD connection on stdin/stdout. You can run these servers as a
414 subprocess of your main program using nbd_connect_command(3). This
415 example creates a 1G writable RAM disk:
416 char *argv[] = { "nbdkit", "-s", "--exit-with-parent",
418 "memory", "1G", NULL };
419 nbd_connect_command (nbd, argv);
420 When the handle is closed the nbdkit subprocess is killed, which in
422 this case means the RAM disk is discarded, so this is useful for
423 testing.
424 Connecting to a subprocess using systemd socket activation
426 Some NBD servers — notably nbdkit(1) and qemu-nbd(1) — support systemd
427 socket activation allowing libnbd to pass a socket to the subprocess.
428 This works very similarly to nbd_connect_command(3) described above,
429 but you must use nbd_connect_systemd_socket_activation(3) instead.
430 Connecting to any socket
432 If none of the other nbd_connect* methods are suitable you can create a
433 connected socket yourself and pass it to nbd_connect_socket(3).
434 One use for this is in fuzzing where we use socketpair(2) to create the
436 socket, then fork, then have the test harness in the child process
437 connected to libnbd over the socket pair (see:
438 https://gitlab.com/nbdkit/libnbd/-/blob/master/fuzzing/libnbd-fuzz-wrapper.c).
439 Another use is to connect libnbd to an address family that it does not
441 support natively, such as XDP or IB.
442
444 By default, when beginning a connection, libnbd will handle all
445 negotiation with the server, using only the configuration (eg.
446 nbd_set_export_name(3) or nbd_add_meta_context(3)) that was requested
447 before the connection attempt; this phase continues until
448 nbd_aio_is_connecting(3) no longer returns true, at which point, either
449 data commands are ready to use or else the connection has failed with
450 an error.
451 But there are scenarios in which it is useful to also control the
453 handshaking commands sent during negotiation, such as asking the server
454 for a list of available exports prior to selecting which one to use.
455 This is done by calling nbd_set_opt_mode(3) before connecting; then
456 after requesting a connection, the state machine will pause at
457 nbd_aio_is_negotiating(3) at any point that the user can decide which
458 handshake command to send next. Note that the negotiation state is
459 only reachable from newstyle servers; older servers cannot negotiate
460 and will progress all the way to the ready state.
461 When the negotiating state is reached, you can initiate option commands
463 such as nbd_opt_list(3) or their asynchronous equivalents, as well as
464 alter configuration such as export name that previously had to be set
465 before connection. Since the NBD protocol does not allow parallel
466 negotiating commands, no cookie is involved, and you can track
467 completion of each command when the state is no longer
468 nbd_aio_is_connecting(3). If nbd_opt_go(3) fails but the connection is
469 still live, you will be back in negotiation state, where you can
470 request a different export name and try again. Exiting the negotiation
471 state is only possible with a successful nbd_opt_go(3) which moves to
472 the data phase, or nbd_opt_abort(3) which performs a clean shutdown of
473 the connection by skipping the data phase.
474
476 It is possible for NBD servers to serve different content on different
477 “exports”. For this you must pass the right export name to the server.
478 Call this API before connecting:
479 nbd_set_export_name (nbd, "export");
481 Note that there are some servers (like nbdkit(1) ≤ 1.14) which ignore
483 this, and other servers (like qemu-nbd(8)) which require it to be set
484 correctly but cannot serve different content.
485 These APIs are also available after a successful nbd_opt_info(3) during
487 the negotiation phase, if you used nbd_set_opt_mode(3) prior to
488 connecting.
489 Flag calls
491 After connecting the server will send back a set of flags describing
492 the export, such as whether it is writable and if it can support flush
493 to permanent storage. These flags can be accessed from libnbd using
494 APIs such as:
495 int is_read_only = nbd_is_read_only (nbd);
497 int can_flush = nbd_can_flush (nbd);
498 Flag calls are: nbd_can_cache(3), nbd_can_df(3), nbd_can_fast_zero(3),
500 nbd_can_flush(3), nbd_can_fua(3), nbd_can_meta_context(3),
501 nbd_can_multi_conn(3), nbd_can_trim(3), nbd_can_zero(3),
502 nbd_is_read_only(3), nbd_is_rotational(3).
503 Size of the export
505 To get the size of the export in bytes, use nbd_get_size(3):
506 int64_t size = nbd_get_size (nbd);
508 Block size constraints
510 Some NBD servers cannot handle requests at any byte boundary. They
511 might, for example, require all requests to be aligned to 512 byte
512 sectors.
513 Also some servers advertise a preferred block size. This is not a
515 requirement, but is the minimum block size that can be accessed
516 efficiently (usually without triggering expensive read-modify-write
517 cycles inside the server).
518 These are referred to as block size constraints and can be queried by
520 calling nbd_get_block_size(3). Pay attention in particular to the
521 "LIBNBD_SIZE_MINIMUM" constraint as some servers will fail requests
522 which are smaller or not aligned to this block size with "EINVAL"
523 ("Invalid argument") errors.
524
526 You can read and write data from the NBD server using nbd_pread(3) and
527 nbd_pwrite(3) or their asynchronous equivalents.
528 All data commands support a "flags" argument (mandatory in C, but
530 optional in languages where it can default to 0). For convenience, the
531 constant "LIBNBD_CMD_FLAG_MASK" is defined with the set of flags
532 currently recognized by libnbd, where future NBD protocol extensions
533 may result in additional flags being supported; but in general,
534 specific data commands only accept a subset of known flags.
535 Libnbd defaults to performing some client-side sanity checking in each
537 of its data commands; for example, attempts to write to a server that
538 has advertised a read-only connection are rejected. It is possible to
539 override aspects of this checking by using nbd_set_strict_mode(3).
540 Some servers also support:
542 trim/discard
544 If nbd_can_trim(3) returns true, nbd_trim(3) can be used to “punch
545 holes” in the backing storage of the disk on the server. Normally
546 (although not in every case) the holes read back as zeroes but take
547 up no space.
548 zeroing
550 If nbd_can_zero(3) returns true, nbd_zero(3) can be used to
551 efficiently zero parts of the disk without having to send large
552 amounts of zero bytes over the network (as would be necessary if
553 using nbd_pwrite(3)).
554 This is slightly different from trimming because the backing
556 storage is still allocated. For some storage types this can make
557 future writes more efficient and/or less likely to fail because of
558 out of space errors.
559 flushing
561 Some servers can commit data to permanent storage and tell you that
562 this has happened reliably. There are two export flags associated
563 with this: nbd_can_flush(3) and nbd_can_fua(3).
564 The nbd_flush(3) call (available if nbd_can_flush(3) returns true)
566 flushes all pending writes to disk and does not complete until that
567 operation has finished. It is similar to using sync(2) on POSIX
568 systems.
569 A more efficient way to achieve this is to set the flag
571 "LIBNBD_CMD_FLAG_FUA" on write-like calls (like write, trim and
572 zero). This flag means the call will not complete until committed
573 to permanent storage, but it does not involve flushing the entire
574 disk.
575 prefetching
577 Some servers can prefetch data, making subsequent reads faster.
578 The nbd_cache(3) call (available if nbd_can_cache(3) returns true)
579 is used to prefetch.
580 block status
582 Some servers are able to provide information about the various
583 extents within the image, via the notion of one or more meta
584 contexts. The most common meta context is "base:allocation"
585 (available in libnbd.h as "LIBNBD_CONTEXT_BASE_ALLOCATION"), which
586 can be used to learn which portions of a file are allocated or read
587 as zero. Other contexts may be available; for example, qemu-nbd(8)
588 can expose a meta context "qemu:dirty-bitmap:NAME" for tracking
589 which portions of a file are tracked by a qcow2 dirty bitmap.
590 In order to utilize block status, the client must call
592 nbd_add_meta_context(3) prior to connecting, for each meta context
593 in which it is interested, then check nbd_can_meta_context(3) after
594 connection to see which contexts the server actually supports. If
595 a context is supported, the client can then use nbd_block_status(3)
596 with a callback function that will receive an array of 32-bit
597 integer pairs describing consecutive extents within a context. In
598 each pair, the first integer is the length of the extent, the
599 second is a bitmask description of that extent (for the
600 "base:allocation" context, the bitmask may include
601 "LIBNBD_STATE_HOLE" for unallocated portions of the file, and/or
602 "LIBNBD_STATE_ZERO" for portions of the file known to read as
603 zero).
604 There is a full example of requesting meta context and using block
606 status available at
607 https://gitlab.com/nbdkit/libnbd/blob/master/interop/dirty-bitmap.c
608
610 Issuing multiple in-flight requests
611 NBD servers which properly implement the specification can handle
612 multiple data requests in flight over the same connection at the same
613 time. Libnbd supports this when using the low level API.
614 To use it you simply issue more requests as needed (eg. using calls
616 like nbd_aio_pread(3), nbd_aio_pwrite(3)) without waiting for previous
617 commands to complete. You need to be careful that requests in flight
618 do not overlap with disk offsets of other write-like commands in flight
619 — an overlapping read may see indeterminate data, and an overlapping
620 write may even cause disk corruption where the resulting disk contents
621 do not match either of the two writes.
622 Each request is identified by a unique 64 bit cookie (assigned by
624 libnbd), allowing libnbd and callers to match replies to requests.
625 Replies may arrive out of order. A request that is rejected client-
626 side for failing a sanity check (such as attempting to write to a read-
627 only server, see nbd_set_strict_mode(3)) will fail rather than
628 returning a cookie, although closure cleanup is still performed.
629 Although in theory you can have an indefinite number of requests in
631 flight at the same time, in practice it's a good idea to limit them to
632 some number. Libnbd will queue commands in the handle even if it
633 cannot write them to the server, so this limit is largely to prevent a
634 backlog of commands from consuming too much memory. It is suggested to
635 start with a limit of 64 requests in flight (per NBD connection), and
636 measure how adjusting the limit up and down affects performance for
637 your local configuration.
638 There is a full example using multiple in-flight requests available at
640 https://gitlab.com/nbdkit/libnbd/blob/master/examples/threaded-reads-and-writes.c
641 Multi-conn
643 Some NBD servers advertise “multi-conn” which means that it is safe to
644 make multiple connections to the server and load-balance commands
645 across all of the connections.
646 To do this you should open a single connection first and test for this
648 feature using nbd_can_multi_conn(3). Without error handling it would
649 look like this:
650 struct nbd_handle *nbd[4];
652 size_t i;
653 bool supports_multi_conn;
654 nbd[0] = nbd_create ();
656 nbd_connect_tcp (nbd[0], "server", "10809");
657 supports_multi_conn = nbd_can_multi_conn (nbd[0]) > 0;
658 If multi-conn is supported then you can open further connections:
660 if (supports_multi_conn) {
662 for (i = 1; i <= 3; ++i) {
663 nbd[i] = nbd_create ();
664 nbd_connect_tcp (nbd[i], "server", "10809");
665 }
666 }
667 If you are issuing multiple in-flight requests (see above) and limiting
669 the number, then the limit should be applied to each individual NBD
670 connection.
671
673 The NBD protocol and libnbd supports TLS (sometimes incorrectly called
674 “SSL”) for encryption of the data stream and authentication of clients
675 and servers. Libnbd defaults to TLS disabled for maximum
676 interoperability. To enable it on a handle you must call
677 nbd_set_tls(3) before connecting.
678 To allow TLS, but fall back to unencrypted:
680 nbd_set_tls (nbd, LIBNBD_TLS_ALLOW);
682 Use nbd_get_tls_negotiated(3) to find out if TLS negotiation was
684 successful. Avoid "LIBNBD_TLS_ALLOW" if man-in-the-middle attacks are
685 a concern.
686 The most secure mode is to require TLS and fail to connect if the
688 server does not support it:
689 nbd_set_tls (nbd, LIBNBD_TLS_REQUIRE);
691 It may also be necessary to verify that the server’s identity is
693 correct. For some servers it may be necessary to verify to the server
694 that the client is permitted to connect. This can be done using either
695 X.509 certificates, or TLS Pre-Shared Keys (PSK). Certificates are
696 more secure. PSK is far more convenient, but you must have an existing
697 secure channel to distribute the keys.
698 Setting up X.509 using system certificate authorities (CAs)
700 This is the default if you don’t call any other "nbd_set_tls_*"
701 functions. In this case the server must have a public (eg. HTTPS)
702 certificate which can be verified against the CAs registered on your
703 system (eg. under /etc/pki).
704 To disable server name verification — which opens you up to a potential
706 Man-In-The-Middle (MITM) attack — use:
707 nbd_set_tls_verify_peer (nbd, false);
709 Setting up an X.509 certificate authority (CA)
711 You can set up your own CA and register clients and servers with it,
712 issuing client and server certificates which will reliably authenticate
713 your clients and servers to each other.
714 Doing this is described in detail in the nbdkit-tls(1) manual. The
716 only differences for libnbd are:
717 • Non-root certificates must be placed in "$HOME/.pki/libnbd/" or
719 "$HOME/.config/pki/libnbd/"
720 • Libnbd reads client-cert.pem and client-key.pem (instead of
722 server-cert.pem and server-key.pem).
723 Once you have set up the directory containing the certificates, call:
725 nbd_set_tls_certificates (nbd, "/path/to/directory");
727 Setting up Pre-Shared Keys (PSK)
729 TLS Pre-Shared Keys are a much more convenient method of setting up
730 TLS, and more appropriate for NBD, but you should have an existing
731 secure method available to distribute the keys. They are therefore
732 ideal if you want to set up an NBD service as an adjunct to an existing
733 secure REST API.
734 Use psktool(1) to create a file of "username:key" pairs:
736 psktool -u username -p keys.psk
738 and pass this path to libnbd:
740 nbd_set_tls_psk_file (nbd, "keys.psk");
742 If necessary you may need to set the client username (otherwise libnbd
744 will use your login name):
745 nbd_set_tls_username (nbd, "username");
747
749 Some libnbd calls take callbacks (eg. nbd_set_debug_callback(3),
750 nbd_aio_pread(3)). Libnbd can call these functions while processing.
751 In the C API these libnbd calls take a structure which contains the
753 function pointer and an optional opaque "void *user_data" pointer:
754 nbd_aio_pread (nbd, buf, sizeof buf, offset,
756 (nbd_completion_callback) { .callback = my_fn,
757 .user_data = my_data },
758 0);
759 For optional callbacks, if you don't want the callback, either set
761 ".callback" to "NULL" or use the equivalent macros (such as
762 "NBD_NULL_COMPLETION") defined in "libnbd.h":
763 nbd_aio_pread (nbd, buf, sizeof buf, offset,
765 NBD_NULL_COMPLETION, 0);
766 From other languages the structure and opaque pointer are not needed
768 because you can use closures to achieve the same effect.
769 Callback lifetimes
771 You can associate an optional free function with callbacks. Libnbd
772 will call this function when the callback will not be called again by
773 libnbd, including in the case where the API fails.
774 This can be used to free associated "user_data". For example:
776 void *my_data = malloc (...);
778 nbd_aio_pread_structured (nbd, buf, sizeof buf, offset,
780 (nbd_chunk_callback) { .callback = my_fn,
781 .user_data = my_data,
782 .free = free },
783 NBD_NULL_COMPLETION,
784 0);
785 will call free(3) once on "my_data" after the point where it is known
787 that the "chunk.callback = my_fn" function can no longer be called,
788 regardless of how many times "my_fn" was actually called. If both a
789 mid-command and completion callback are supplied, the functions will be
790 reached in this order: mid-function callbacks, completion callback,
791 mid-function free, and finally completion free.
792 The free function is only accessible in the C API as it is not needed
794 in garbage collected programming languages.
795 Callbacks with ".callback=NULL" and ".free!=NULL"
797 It is possible to register a callback like this:
798 ...
800 (nbd_completion_callback) { .callback = NULL,
801 .user_data = my_data,
802 .free = free },
803 ...
804 The meaning of this is that the callback is never called, but the free
806 function is still called after the last time the callback would have
807 been called. This is useful for applying generic freeing actions when
808 asynchronous commands are retired.
809 Callbacks and locking
811 The callbacks are invoked at a point where the libnbd lock is held; as
812 such, it is unsafe for the callback to call any "nbd_*" APIs on the
813 same nbd object, as it would cause deadlock.
814 Completion callbacks
816 All of the asychronous commands have an optional completion callback
817 function that is used right before the command is marked complete,
818 after any mid-command callbacks have finished, and before any free
819 functions.
820 When the completion callback returns 1, the command is automatically
822 retired (there is no need to call nbd_aio_command_completed(3)); for
823 any other return value, the command still needs to be manually retired
824 (otherwise, the command will tie up resources until nbd_close(3) is
825 eventually reached).
826 Callbacks with "int *error" parameter
828 Some of the high-level commands (nbd_pread_structured(3),
829 nbd_block_status(3)) involve the use of a callback function invoked by
830 the state machine at appropriate points in the server's reply before
831 the overall command is complete. These callback functions, along with
832 all of the completion callbacks, include a parameter "error" which is a
833 pointer containing the value of any error detected so far. If a
834 callback function fails and wants to change the resulting error of the
835 overall command visible later in the API sequence, it should assign
836 back into "error" and return "-1" in the C API. Assignments into
837 "error" are ignored for any other return value; similarly, assigning 0
838 into "error" does not have an effect. In other language bindings,
839 reporting callback errors is generally done by raising an exception
840 rather than by return value.
841 Note that a mid-command callback might never be reached, such as if
843 libnbd detects that the command was invalid to send (see
844 nbd_set_strict_mode(3)) or if the server reports a failure that
845 concludes the command. It is safe for a mid-command callback to ignore
846 non-zero "error": all the other parameters to the mid-command callback
847 will still be valid (corresponding to the current portion of the
848 server's reply), and the overall command will still fail (at the
849 completion callback or nbd_aio_command_completed(3) for an asynchronous
850 command, or as the result of the overall synchronous command).
851 Returing "-1" from a mid-command callback does not prevent that
852 callback from being reached again, if the server sends more mid-command
853 replies that warrant another use of that callback. A mid-command
854 callback may be reached more times than expected if the server is non-
855 compliant.
856 On the other hand, if a completion callback is supplied (only possible
858 with asynchronous commands), it will always be reached exactly once,
859 and the completion callback must not ignore the value pointed to by
860 "error". In particular, the content of a buffer passed to
861 nbd_aio_pread(3) or nbd_aio_pread_structured(3) is undefined if *error
862 is non-zero on entry to the completion callback. It is recommended
863 that if you choose to use automatic command retirement (where the
864 completion callback returns 1 to avoid needing to call
865 nbd_aio_command_completed(3) later), your completion function should
866 return 1 on all control paths, even when handling errors (note that
867 with automatic retirement, assigning into "error" is pointless as there
868 is no later API to see that value).
869
871 Libnbd does not install signal handlers. It attempts to disable
872 "SIGPIPE" when writing to the NBD socket using the "MSG_NOSIGNAL" flag
873 of send(2), or the "SO_NOSIGPIPE" socket option, on platforms that
874 support those.
875 On some old Linux or newer non-Linux platforms the main program may
877 wish to register a signal handler to ignore SIGPIPE:
878 signal (SIGPIPE, SIG_IGN);
880
882 On most systems, C programs that use libnbd can be compiled like this:
883 cc prog.c -o prog -lnbd
885 To detect if the libnbd library and header file is installed, the
887 preferred method is to use pkg-config(1) or pkgconf(1):
888 pkg-config libnbd --exists || fail libnbd is required
890 In case the library or header file are not installed in the usual
892 system locations, you can compile your program like this, using pkg-
893 config to detect the proper location of libnbd:
894 cc prog.c -o prog `pkg-config libnbd --cflags --libs`
896 To compile an external project against a built copy of the libnbd
898 source tree which hasn't been installed, see the ./run script.
899 Autoconf projects
901 External projects which use autoconf and need to check if libnbd is
902 installed should use the "PKG_CHECK_MODULES" macro in configure.ac like
903 this:
904 PKG_CHECK_MODULES([LIBNBD], [libnbd])
906 This will define "@LIBNBD_CFLAGS@" and "@LIBNBD_LIBS@" which you will
908 need to add to your Makefile.am.
909 CMake projects
911 For CMake projects use:
912 find_package(PkgConfig REQUIRED)
914 pkg_check_modules(LIBNBD REQUIRED libnbd)
915 target_link_libraries(prog ${LIBNBD_LIBRARIES})
916 target_include_directories(prog PUBLIC ${LIBNBD_INCLUDE_DIRS})
917 target_compile_options(prog PUBLIC ${LIBNBD_CFLAGS_OTHER})
918 Meson projects
920 For meson projects use:
921 nbd_dep = dependency('libnbd')
923 executable('prog', 'prog.c', dependencies : [nbd_dep])
924
926 "HOME"
927 Used in some situations to find TLS certificates. See
928 nbd_set_tls_certificates(3).
929 "LIBNBD_DEBUG"
931 If this is set to the exact string 1 when the handle is created
932 then debugging is enabled. See "DEBUGGING MESSAGES" above.
933 "LOGNAME"
935 The default TLS username. See nbd_set_tls_username(3).
936
938 C API
939 nbd_add_meta_context(3), nbd_aio_block_status(3), nbd_aio_cache(3),
940 nbd_aio_command_completed(3), nbd_aio_connect(3),
941 nbd_aio_connect_command(3), nbd_aio_connect_socket(3),
942 nbd_aio_connect_systemd_socket_activation(3), nbd_aio_connect_tcp(3),
943 nbd_aio_connect_unix(3), nbd_aio_connect_uri(3),
944 nbd_aio_connect_vsock(3), nbd_aio_disconnect(3), nbd_aio_flush(3),
945 nbd_aio_get_direction(3), nbd_aio_get_fd(3), nbd_aio_in_flight(3),
946 nbd_aio_is_closed(3), nbd_aio_is_connecting(3), nbd_aio_is_created(3),
947 nbd_aio_is_dead(3), nbd_aio_is_negotiating(3),
948 nbd_aio_is_processing(3), nbd_aio_is_ready(3), nbd_aio_notify_read(3),
949 nbd_aio_notify_write(3), nbd_aio_opt_abort(3), nbd_aio_opt_go(3),
950 nbd_aio_opt_info(3), nbd_aio_opt_list(3),
951 nbd_aio_opt_list_meta_context(3), nbd_aio_peek_command_completed(3),
952 nbd_aio_pread(3), nbd_aio_pread_structured(3), nbd_aio_pwrite(3),
953 nbd_aio_trim(3), nbd_aio_zero(3), nbd_block_status(3), nbd_cache(3),
954 nbd_can_cache(3), nbd_can_df(3), nbd_can_fast_zero(3),
955 nbd_can_flush(3), nbd_can_fua(3), nbd_can_meta_context(3),
956 nbd_can_multi_conn(3), nbd_can_trim(3), nbd_can_zero(3),
957 nbd_clear_debug_callback(3), nbd_clear_meta_contexts(3), nbd_close(3),
958 nbd_connect_command(3), nbd_connect_socket(3),
959 nbd_connect_systemd_socket_activation(3), nbd_connect_tcp(3),
960 nbd_connect_unix(3), nbd_connect_uri(3), nbd_connect_vsock(3),
961 nbd_connection_state(3), nbd_create(3), nbd_flush(3),
962 nbd_get_block_size(3), nbd_get_canonical_export_name(3),
963 nbd_get_debug(3), nbd_get_errno(3), nbd_get_error(3),
964 nbd_get_export_description(3), nbd_get_export_name(3),
965 nbd_get_full_info(3), nbd_get_handle_name(3),
966 nbd_get_handshake_flags(3), nbd_get_meta_context(3),
967 nbd_get_nr_meta_contexts(3), nbd_get_opt_mode(3),
968 nbd_get_package_name(3), nbd_get_pread_initialize(3),
969 nbd_get_private_data(3), nbd_get_protocol(3),
970 nbd_get_request_block_size(3), nbd_get_request_structured_replies(3),
971 nbd_get_size(3), nbd_get_strict_mode(3),
972 nbd_get_structured_replies_negotiated(3), nbd_get_tls(3),
973 nbd_get_tls_negotiated(3), nbd_get_tls_username(3),
974 nbd_get_tls_verify_peer(3), nbd_get_uri(3), nbd_get_version(3),
975 nbd_is_read_only(3), nbd_is_rotational(3), nbd_kill_subprocess(3),
976 nbd_opt_abort(3), nbd_opt_go(3), nbd_opt_info(3), nbd_opt_list(3),
977 nbd_opt_list_meta_context(3), nbd_poll(3), nbd_pread(3),
978 nbd_pread_structured(3), nbd_pwrite(3), nbd_set_debug(3),
979 nbd_set_debug_callback(3), nbd_set_export_name(3),
980 nbd_set_full_info(3), nbd_set_handle_name(3),
981 nbd_set_handshake_flags(3), nbd_set_opt_mode(3),
982 nbd_set_pread_initialize(3), nbd_set_private_data(3),
983 nbd_set_request_block_size(3), nbd_set_request_structured_replies(3),
984 nbd_set_strict_mode(3), nbd_set_tls(3), nbd_set_tls_certificates(3),
985 nbd_set_tls_psk_file(3), nbd_set_tls_username(3),
986 nbd_set_tls_verify_peer(3), nbd_set_uri_allow_local_file(3),
987 nbd_set_uri_allow_tls(3), nbd_set_uri_allow_transports(3),
988 nbd_shutdown(3), nbd_supports_tls(3), nbd_supports_uri(3), nbd_trim(3),
989 nbd_zero(3).
990 Servers
992 nbdkit(1), nbd-server(1), qemu-nbd(8).
993 Encryption tools
995 certtool(1), nbdkit-tls(1), psktool(1).
996 Standards
998 https://github.com/NetworkBlockDevice/nbd/blob/master/doc/proto.md,
999 https://github.com/NetworkBlockDevice/nbd/blob/master/doc/uri.md.
1000 Release notes
1002 libnbd-release-notes-1.14(1), libnbd-release-notes-1.12(1),
1003 libnbd-release-notes-1.10(1), libnbd-release-notes-1.8(1),
1004 libnbd-release-notes-1.6(1), libnbd-release-notes-1.4(1),
1005 libnbd-release-notes-1.2(1).
1006 Other
1008 libnbd-security(3), nbdcopy(1), nbddump(1), nbdfuse(1), nbdinfo(1),
1009 nbdsh(1), qemu(1).
1010
1012 Eric Blake
1013 Richard W.M. Jones
1015
1017 Copyright (C) 2019-2022 Red Hat Inc.
1018
1020 This library is free software; you can redistribute it and/or modify it
1021 under the terms of the GNU Lesser General Public License as published
1022 by the Free Software Foundation; either version 2 of the License, or
1023 (at your option) any later version.
1024 This library is distributed in the hope that it will be useful, but
1026 WITHOUT ANY WARRANTY; without even the implied warranty of
1027 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
1028 Lesser General Public License for more details.
1029 You should have received a copy of the GNU Lesser General Public
1031 License along with this library; if not, write to the Free Software
1032 Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
1033 02110-1301 USA
1034libnbd-1.14.2 2023-01-03 libnbd(3)