1libnbd(3)                           LIBNBD                           libnbd(3)
2
3
4

NAME

6       libnbd - network block device (NBD) client library in userspace
7

SYNOPSIS

9        #include <libnbd.h>
10
11        struct nbd_handle *nbd;
12        char buf[512];
13
14        if ((nbd = nbd_create ()) == NULL ||
15            nbd_connect_tcp (nbd, "server.example.com", "nbd") == -1 ||
16            nbd_pread (nbd, buf, sizeof buf, 0, 0) == -1)
17          fprintf (stderr, "%s\n", nbd_get_error ());
18          nbd_close (nbd);
19          exit (EXIT_FAILURE);
20        }
21        nbd_close (nbd);
22
23        cc prog.c -o prog -lnbd
24       or:
25        cc prog.c -o prog `pkg-config libnbd --cflags --libs`
26

DESCRIPTION

28       Network Block Device (NBD) is a network protocol for accessing block
29       devices over the network.  Block devices are hard disks and things that
30       behave like hard disks such as disk images and virtual machines.
31
32       Libnbd is a client library for the NBD protocol which can access most
33       of the features of NBD while being simple to use and powerful.
34
35       This manual page gives an overview of libnbd, using C as an example,
36       but the library is available from other programming languages.
37
38       nbd_create(3), nbd_pread(3), etc.
39           Each manual page covers one function from the C API in detail.
40           There is a full list in section "C API" below.
41
42       libnbd-ocaml(3)
43           Using the API from OCaml.
44
45       libnbd-golang(3)
46           Using the API from Go.
47
48       nbdsh(1)
49           Using the NBD shell (nbdsh) for command line and Python scripting.
50

HANDLES

52       To use the API at all you must first open a handle by calling
53       nbd_create(3) (or its equivalent in other languages):
54
55        struct nbd_handle *nbd;
56
57        nbd = nbd_create ();
58
59       This creates and returns a handle, which is associated with one
60       connection to an NBD server, initially not connected.
61
62       Each handle is a complex state machine which can be in states such as
63       created, connected to a remote server, handshaking, idle and ready to
64       issue commands, or busy sending or receiving commands.
65
66       Handles have a name used in debugging messages.  The name is normally
67       generated ("nbd1", "nbd2" etc) but you can set a friendly name with
68       nbd_set_handle_name(3).  Also there is a private field in the handle
69       for use by the application, see nbd_set_private_data(3).
70
71       When you have finished with the handle you must call nbd_close(3) which
72       closes the underlying socket (if necessary) and frees up all associated
73       resources.
74

SYNCHRONOUS VS ASYNCHRONOUS API

76       There are two levels of API available.  A simple high level synchronous
77       API lets you give the handle high level instructions like “connect to
78       the server”, “read a block”, “write a block”, etc.  Each of these
79       functions will run to completion, blocking the current thread before
80       returning.  A more complicated low level non-blocking asynchronous API
81       is also available where you can integrate with poll(2) or another main
82       loop.
83
84       You can freely mix the two APIs on the same handle.  You can also call
85       APIs on a single handle from multiple threads.  Single API calls on the
86       handle are atomic — they either take a lock on the handle while they
87       run or are careful to access handle fields atomically.
88
89       Libnbd does not create its own threads.
90

USING THE SYNCHRONOUS (“HIGH LEVEL”) API

92       This is the simplest way to use the API, with the possible drawback
93       that each libnbd function blocks until it is finished.
94
95       Create a handle and connect to the server:
96
97        struct nbd_handle *nbd;
98
99        nbd = nbd_create ();
100        if (!nbd) {
101          fprintf (stderr, "%s\n", nbd_get_error ());
102          nbd_close (nbd);
103          exit (EXIT_FAILURE);
104        }
105        if (nbd_connect_tcp (nbd, "server.example.com", "nbd") == -1) {
106          fprintf (stderr, "%s\n", nbd_get_error ());
107          nbd_close (nbd);
108          exit (EXIT_FAILURE);
109        }
110
111       Read the first sector (512 bytes) from the NBD export:
112
113        char buf[512];
114
115        if (nbd_pread (nbd, buf, sizeof buf, 0, 0) == -1) {
116          fprintf (stderr, "%s\n", nbd_get_error ());
117          nbd_close (nbd);
118          exit (EXIT_FAILURE);
119        }
120
121       Close the handle:
122
123        nbd_close (nbd);
124
125       You can call the high level API from multiple threads, but each libnbd
126       API call takes a lock on the handle and so commands will not run in
127       parallel.
128

USING THE ASYNCHRONOUS (“LOW LEVEL”) API

130       The low level API is useful if you want to use libnbd in non-blocking
131       code; or if you want to issue commands in parallel from multiple
132       threads; or if you need more control especially over having multiple
133       commands in-flight on a single connection.
134
135       To use the low level API you will need to integrate with poll(2) or
136       another “main loop” such as the GLib main event loop.
137
138   Issuing asynchronous commands
139       Use the "nbd_aio_*" variants to issue commands asynchronously (without
140       waiting for the command to complete before returning).  For example the
141       asynchronous variant of nbd_pread(3) is:
142
143        int64_t cookie;
144
145        cookie = nbd_aio_pread (nbd, buf, sizeof buf,
146                                NBD_NULL_COMPLETION, 0);
147        if (cookie == -1) {
148          fprintf (stderr, "%s\n", nbd_get_error ());
149          nbd_close (nbd);
150          exit (EXIT_FAILURE);
151        }
152
153       There are several things to note here:
154
155       •   This only starts the command.  The command is still in flight when
156           the call returns.
157
158       •   A buffer ("buf") has been assigned to collect the result of the
159           read, but it is not guaranteed to be filled with data until the
160           command has completed (see examples below).  The buffer must not be
161           freed until the command has finished running.
162
163       •   You can issue multiple commands on the same handle at the same
164           time.
165
166       •   A cookie is returned which identifies this command in subsequent
167           calls.  The cookie is unique (per libnbd handle) and ≥ 1.
168
169       •   You may register a function which is called when the command
170           completes, see "Completion callbacks" below.  In this case we have
171           specified a null completion callback.
172
173   Socket and direction
174       Each libnbd handle has an associated socket (once it has started
175       connecting).  You can read the file descriptor of the socket using:
176
177        int fd = nbd_aio_get_fd (nbd);
178
179       The socket is non-blocking.  Between calls into libnbd it is in the
180       "would block" condition.  You can find out if libnbd is expecting to
181       read or write from the socket next by calling:
182
183        int dir = nbd_aio_get_direction (nbd);
184
185       which returns one of "LIBNBD_AIO_DIRECTION_READ",
186       "LIBNBD_AIO_DIRECTION_WRITE" or "LIBNBD_AIO_DIRECTION_BOTH" (=
187       "READ|WRITE").  And so to set up the next call to poll(2) or other main
188       loop you must translate this to "POLLIN", "POLLOUT" or "POLLIN|POLLOUT"
189       (or whatever mechanism your main loop uses).
190
191   Notifying libnbd when an event happens
192       When you detect (eg. using poll(2)) that a read or write event has
193       happened on the socket, you must then tell libnbd about it.  You have
194       to check the direction again (since it may have been changed by another
195       thread), and notify libnbd:
196
197        int r = 0;
198
199        dir = nbd_aio_get_direction (nbd);
200
201        if ((dir & LIBNBD_AIO_DIRECTION_READ) &&
202                        a_read_event_occurred ())
203          r = nbd_aio_notify_read (nbd);
204        else if ((dir & LIBNBD_AIO_DIRECTION_WRITE) &&
205                        a_write_event_occurred ())
206          r = nbd_aio_notify_write (nbd);
207
208        if (r == -1) {
209          fprintf (stderr, "%s\n", nbd_get_error ());
210          // ...
211        }
212
213       The notify calls move the state machine along, reading and writing from
214       the socket possibly multiple times, until the socket would block again,
215       at which point they return control to the caller.
216
217   Simple implementation with nbd_poll(3)
218       In fact if you want to use poll(2) on a single handle, a simple
219       implementation has already been written called nbd_poll(3).  It is also
220       useful to examine how this is implemented (lib/poll.c in the libnbd
221       source code) because that will tell you how to integrate libnbd with
222       more complex main loops.
223
224       Some examples of using nbd_poll(3) follow.
225
226       As with the high level API, it all starts by creating a handle:
227
228        struct nbd_handle *nbd;
229
230        nbd = nbd_create ();
231        if (nbd == NULL) {
232          fprintf (stderr, "%s\n", nbd_get_error ());
233          nbd_close (nbd);
234          exit (EXIT_FAILURE);
235        }
236
237       To connect to the server asynchronously, we start the connection using
238       nbd_aio_connect(3) and then enter our main loop to check for events
239       until the connection becomes ready:
240
241        int fd;
242        struct sockaddr_un addr;
243        socklen_t len;
244
245        /* some code to set up addr,
246           then ... */
247        if (nbd_aio_connect (nbd, &addr, len) == -1) {
248          fprintf (stderr, "%s\n", nbd_get_error ());
249          nbd_close (nbd);
250          exit (EXIT_FAILURE);
251        }
252        while (! nbd_aio_is_ready (nbd)) {
253          if (nbd_poll (nbd, -1) == -1) {
254            fprintf (stderr, "%s\n", nbd_get_error ());
255            nbd_close (nbd);
256            exit (EXIT_FAILURE);
257          }
258        }
259
260       To read data asynchronously, start an asynchronous read command, which
261       returns a 64 bit command cookie, and enter the main loop until the
262       command has completed:
263
264        int64_t cookie;
265        char buf[512];
266
267        cookie = nbd_aio_pread (nbd, buf, sizeof buf, offset,
268                                NBD_NULL_COMPLETION, 0);
269        if (cookie == -1) {
270          fprintf (stderr, "%s\n", nbd_get_error ());
271          nbd_close (nbd);
272          exit (EXIT_FAILURE);
273        }
274        while (! nbd_aio_command_completed (nbd, cookie)) {
275          if (nbd_poll (nbd, -1) == -1) {
276            fprintf (stderr, "%s\n", nbd_get_error ());
277            nbd_close (nbd);
278            exit (EXIT_FAILURE);
279          }
280        }
281
282       For almost all high level synchronous calls (eg. nbd_pread(3)) there is
283       a low level asynchronous equivalent (eg. nbd_aio_pread(3)) for starting
284       a command.
285
286   glib2 integration
287       See
288       https://gitlab.com/nbdkit/libnbd/blob/master/examples/glib-main-loop.c
289
290   libev integration
291       See https://gitlab.com/nbdkit/libnbd/blob/master/examples/copy-libev.c
292

ERROR HANDLING

294       When any API call returns an error ("-1" or "NULL" depending on the
295       API), an error message and sometimes an errno value are available.  You
296       can retrieve the error message and/or errno of the most recently failed
297       call using nbd_get_error(3) and nbd_get_errno(3).  For example:
298
299        if (nbd_connect_tcp (nbd, "remote", "nbd") == -1) {
300          fprintf (stderr,
301                   "failed to connect to remote server: %s (errno = %d)\n",
302                   nbd_get_error (), nbd_get_errno ());
303        }
304
305       These functions use thread-local storage to return the most recent
306       error in the current thread.  This is why you don't need to pass the
307       handle to these calls.  They even work if nbd_create(3) returns "NULL"
308       when there is no handle at all.
309
310       For this reason you cannot call them from a different thread.  You
311       should call them immediately after the failed API call, from the same
312       thread.  Furthermore the error string returned by nbd_get_error(3) is
313       only valid until the next libnbd API call in the current thread, so if
314       you need to keep the string you must copy it (eg. using strdup(3)).
315
316   Errno
317       For some errors, a system call error number (see errno(3)) is
318       available.  You can find the error number by calling nbd_get_errno(3).
319       It works the same way as nbd_get_error(3) with respect to threads.
320
321       Even when a call returns an error, nbd_get_errno(3) might return 0.
322       This does not mean there was no error.  It means no additional errno
323       information is available for this error.
324
325       The error number is often the raw error returned by a system call that
326       failed.
327
328       It can also be used to indicate special conditions.  The most common
329       cases are:
330
331       "EINVAL"
332           Invalid parameters or state for the current libnbd call.
333
334       "ENOTSUP"
335           The libnbd call is not available in this build of libnbd (eg. when
336           using a TLS API if the library was compiled without TLS support).
337
338       "ENOMEM"
339           The library ran out of memory while performing some operation.
340
341       "ERANGE"
342           A request is too large, for example if you try to read too many
343           bytes in a single nbd_pread(3) call.
344

DEBUGGING MESSAGES

346       Libnbd can print lots of debugging messages, useful if you have a
347       problem with the library.  Either enable debugging after creating the
348       handle:
349
350        nbd = nbd_create ();
351        nbd_set_debug (nbd, true);
352
353       or set the "LIBNBD_DEBUG=1" environment variable which will enable
354       debugging by default on all new handles.
355
356       Debugging messages are sent to stderr by default, but you can redirect
357       them to a logging system using nbd_set_debug_callback(3).
358

CONNECTING TO LOCAL OR REMOTE NBD SERVERS

360       There are several ways to connect to NBD servers, and you can even run
361       a server from libnbd.  Normally you would connect to a server which is
362       already running, over a local Unix domain socket or a remote TCP
363       connection.  The high level API calls are:
364
365        nbd_connect_unix (nbd, "socket");
366        nbd_connect_tcp (nbd, "localhost", "nbd");
367
368       For nbd_connect_tcp(3) the third parameter is the port name or number,
369       which can either be a name from /etc/services or the port number as a
370       string (eg. "10809").
371
372   Connecting to an NBD URI
373       libnbd supports the NBD URI specification.  The format of URIs is
374       documented in nbd_connect_uri(3).
375
376       You can connect to a URI as in these examples (using the high level
377       API):
378
379        nbd_connect_uri (nbd, "nbd://example.com/");
380
381        nbd_connect_uri (nbd, "nbds+unix:///export?socket=/tmp/nbd.sock");
382
383       This feature is implemented by calling other libnbd APIs to set up the
384       export name, TLS parameters, and finally connect over a Unix domain
385       socket or TCP.
386
387       URI support is an optional feature of the library, requiring libxml2 at
388       compile time.  The nbd_connect_uri(3) and nbd_aio_connect_uri(3) calls
389       will raise an error (with nbd_get_errno(3) returning "ENOTSUP") if it
390       was not built with this feature, and you can also test for it
391       explicitly using nbd_supports_uri(3).
392
393   Connecting to a subprocess
394       Some NBD servers — notably nbdkit(1) with the -s parameter, and
395       nbd-server(1) with the port parameter set to 0 — can also accept a
396       single NBD connection on stdin/stdout.  You can run these servers as a
397       subprocess of your main program using nbd_connect_command(3).  This
398       example creates a 1G writable RAM disk:
399
400        char *argv[] = { "nbdkit", "-s", "--exit-with-parent",
401                                   "memory", "1G", NULL };
402        nbd_connect_command (nbd, argv);
403
404       When the handle is closed the nbdkit subprocess is killed, which in
405       this case means the RAM disk is discarded, so this is useful for
406       testing.
407
408   Connecting to a subprocess using systemd socket activation
409       Some NBD servers — notably nbdkit(1) and qemu-nbd(1) — support systemd
410       socket activation allowing libnbd to pass a socket to the subprocess.
411       This works very similarly to nbd_connect_command(3) described above,
412       but you must use nbd_connect_systemd_socket_activation(3) instead.
413
414   Connecting to any socket
415       If none of the other nbd_connect* methods are suitable you can create a
416       connected socket yourself and pass it to nbd_connect_socket(3).
417
418       One use for this is in fuzzing where we use socketpair(2) to create the
419       socket, then fork, then have the test harness in the child process
420       connected to libnbd over the socket pair (see:
421       https://gitlab.com/nbdkit/libnbd/-/blob/master/fuzzing/libnbd-fuzz-wrapper.c).
422
423       Another use is to connect libnbd to an address family that it does not
424       support natively, such as XDP or IB.
425

CONTROLLING NEGOTIATION

427       By default, when beginning a connection, libnbd will handle all
428       negotiation with the server, using only the configuration (eg.
429       nbd_set_export_name(3) or nbd_add_meta_context(3)) that was requested
430       before the connection attempt; this phase continues until
431       nbd_aio_is_connecting(3) no longer returns true, at which point, either
432       data commands are ready to use or else the connection has failed with
433       an error.
434
435       But there are scenarios in which it is useful to also control the
436       handshaking commands sent during negotiation, such as asking the server
437       for a list of available exports prior to selecting which one to use.
438       This is done by calling nbd_set_opt_mode(3) before connecting; then
439       after requesting a connection, the state machine will pause at
440       nbd_aio_is_negotiating(3) at any point that the user can decide which
441       handshake command to send next.  Note that the negotiation state is
442       only reachable from newstyle servers; older servers cannot negotiate
443       and will progress all the way to the ready state.
444
445       When the negotiating state is reached, you can initiate option commands
446       such as nbd_opt_list(3) or their asynchronous equivalents, as well as
447       alter configuration such as export name that previously had to be set
448       before connection.  Since the NBD protocol does not allow parallel
449       negotiating commands, no cookie is involved, and you can track
450       completion of each command when the state is no longer
451       nbd_aio_is_connecting(3).  If nbd_opt_go(3) fails but the connection is
452       still live, you will be back in negotiation state, where you can
453       request a different export name and try again.  Exiting the negotiation
454       state is only possible with a successful nbd_opt_go(3) which moves to
455       the data phase, or nbd_opt_abort(3) which performs a clean shutdown of
456       the connection by skipping the data phase.
457

EXPORTS AND FLAGS

459       It is possible for NBD servers to serve different content on different
460       “exports”.  For this you must pass the right export name to the server.
461       Call this API before connecting:
462
463        nbd_set_export_name (nbd, "export");
464
465       Note that there are some servers (like nbdkit(1) ≤ 1.14) which ignore
466       this, and other servers (like qemu-nbd(8)) which require it to be set
467       correctly but cannot serve different content.
468
469       These APIs are also available after a successful nbd_opt_info(3) during
470       the negotiation phase, if you used nbd_set_opt_mode(3) prior to
471       connecting.
472
473   Flag calls
474       After connecting the server will send back a set of flags describing
475       the export, such as whether it is writable and if it can support flush
476       to permanent storage.  These flags can be accessed from libnbd using
477       APIs such as:
478
479        int is_read_only = nbd_is_read_only (nbd);
480        int can_flush = nbd_can_flush (nbd);
481
482       Flag calls are: nbd_can_cache(3), nbd_can_df(3), nbd_can_fast_zero(3),
483       nbd_can_flush(3), nbd_can_fua(3), nbd_can_meta_context(3),
484       nbd_can_multi_conn(3), nbd_can_trim(3), nbd_can_zero(3),
485       nbd_is_read_only(3), nbd_is_rotational(3).
486
487   Size of the export
488       To get the size of the export in bytes, use nbd_get_size(3):
489
490        int64_t size = nbd_get_size (nbd);
491

DATA COMMANDS

493       You can read and write data from the NBD server using nbd_pread(3) and
494       nbd_pwrite(3) or their asynchronous equivalents.
495
496       All data commands support a "flags" argument (mandatory in C, but
497       optional in languages where it can default to 0).  For convenience, the
498       constant "LIBNBD_CMD_FLAG_MASK" is defined with the set of flags
499       currently recognized by libnbd, where future NBD protocol extensions
500       may result in additional flags being supported; but in general,
501       specific data commands only accept a subset of known flags.
502
503       Libnbd defaults to performing some client-side sanity checking in each
504       of its data commands; for example, attempts to write to a server that
505       has advertised a read-only connection are rejected.  It is possible to
506       override aspects of this checking by using nbd_set_strict_mode(3).
507
508       Some servers also support:
509
510       trim/discard
511           If nbd_can_trim(3) returns true, nbd_trim(3) can be used to “punch
512           holes” in the backing storage of the disk on the server.  Normally
513           (although not in every case) the holes read back as zeroes but take
514           up no space.
515
516       zeroing
517           If nbd_can_zero(3) returns true, nbd_zero(3) can be used to
518           efficiently zero parts of the disk without having to send large
519           amounts of zero bytes over the network (as would be necessary if
520           using nbd_pwrite(3)).
521
522           This is slightly different from trimming because the backing
523           storage is still allocated.  For some storage types this can make
524           future writes more efficient and/or less likely to fail because of
525           out of space errors.
526
527       flushing
528           Some servers can commit data to permanent storage and tell you that
529           this has happened reliably.  There are two export flags associated
530           with this: nbd_can_flush(3) and nbd_can_fua(3).
531
532           The nbd_flush(3) call (available if nbd_can_flush(3) returns true)
533           flushes all pending writes to disk and does not complete until that
534           operation has finished.  It is similar to using sync(2) on POSIX
535           systems.
536
537           A more efficient way to achieve this is to set the flag
538           "LIBNBD_CMD_FLAG_FUA" on write-like calls (like write, trim and
539           zero).  This flag means the call will not complete until committed
540           to permanent storage, but it does not involve flushing the entire
541           disk.
542
543       prefetching
544           Some servers can prefetch data, making subsequent reads faster.
545           The nbd_cache(3) call (available if nbd_can_cache(3) returns true)
546           is used to prefetch.
547
548       block status
549           Some servers are able to provide information about the various
550           extents within the image, via the notion of one or more meta
551           contexts.  The most common meta context is "base:allocation"
552           (available in libnbd.h as "LIBNBD_CONTEXT_BASE_ALLOCATION"), which
553           can be used to learn which portions of a file are allocated or read
554           as zero.  Other contexts may be available; for example, qemu-nbd(8)
555           can expose a meta context "qemu:dirty-bitmap:NAME" for tracking
556           which portions of a file are tracked by a qcow2 dirty bitmap.
557
558           In order to utilize block status, the client must call
559           nbd_add_meta_context(3) prior to connecting, for each meta context
560           in which it is interested, then check nbd_can_meta_context(3) after
561           connection to see which contexts the server actually supports.  If
562           a context is supported, the client can then use nbd_block_status(3)
563           with a callback function that will receive an array of 32-bit
564           integer pairs describing consecutive extents within a context.  In
565           each pair, the first integer is the length of the extent, the
566           second is a bitmask description of that extent (for the
567           "base:allocation" context, the bitmask may include
568           "LIBNBD_STATE_HOLE" for unallocated portions of the file, and/or
569           "LIBNBD_STATE_ZERO" for portions of the file known to read as
570           zero).
571
572           There is a full example of requesting meta context and using block
573           status available at
574           https://gitlab.com/nbdkit/libnbd/blob/master/interop/dirty-bitmap.c
575

PERFORMANCE

577   Issuing multiple in-flight requests
578       NBD servers which properly implement the specification can handle
579       multiple data requests in flight over the same connection at the same
580       time.  Libnbd supports this when using the low level API.
581
582       To use it you simply issue more requests as needed (eg. using calls
583       like nbd_aio_pread(3), nbd_aio_pwrite(3)) without waiting for previous
584       commands to complete.  You need to be careful that requests in flight
585       do not overlap with disk offsets of other write-like commands in flight
586       — an overlapping read may see indeterminate data, and an overlapping
587       write may even cause disk corruption where the resulting disk contents
588       do not match either of the two writes.
589
590       Each request is identified by a unique 64 bit cookie (assigned by
591       libnbd), allowing libnbd and callers to match replies to requests.
592       Replies may arrive out of order.  A request that is rejected client-
593       side for failing a sanity check (such as attempting to write to a read-
594       only server, see nbd_set_strict_mode(3)) will fail rather than
595       returning a cookie, although closure cleanup is still performed.
596
597       Although in theory you can have an indefinite number of requests in
598       flight at the same time, in practice it's a good idea to limit them to
599       some number.  Libnbd will queue commands in the handle even if it
600       cannot write them to the server, so this limit is largely to prevent a
601       backlog of commands from consuming too much memory.  It is suggested to
602       start with a limit of 64 requests in flight (per NBD connection), and
603       measure how adjusting the limit up and down affects performance for
604       your local configuration.
605
606       There is a full example using multiple in-flight requests available at
607       https://gitlab.com/nbdkit/libnbd/blob/master/examples/threaded-reads-and-writes.c
608
609   Multi-conn
610       Some NBD servers advertise “multi-conn” which means that it is safe to
611       make multiple connections to the server and load-balance commands
612       across all of the connections.
613
614       To do this you should open a single connection first and test for this
615       feature using nbd_can_multi_conn(3).  Without error handling it would
616       look like this:
617
618        struct nbd_handle *nbd[4];
619        size_t i;
620        bool supports_multi_conn;
621
622        nbd[0] = nbd_create ();
623        nbd_connect_tcp (nbd[0], "server", "10809");
624        supports_multi_conn = nbd_can_multi_conn (nbd[0]) > 0;
625
626       If multi-conn is supported then you can open further connections:
627
628        if (supports_multi_conn) {
629          for (i = 1; i <= 3; ++i) {
630            nbd[i] = nbd_create ();
631            nbd_connect_tcp (nbd[i], "server", "10809");
632          }
633        }
634
635       If you are issuing multiple in-flight requests (see above) and limiting
636       the number, then the limit should be applied to each individual NBD
637       connection.
638

ENCRYPTION AND AUTHENTICATION

640       The NBD protocol and libnbd supports TLS (sometimes incorrectly called
641       “SSL”) for encryption of the data stream and authentication of clients
642       and servers.  Libnbd defaults to TLS disabled for maximum
643       interoperability.  To enable it on a handle you must call
644       nbd_set_tls(3) before connecting.
645
646       To allow TLS, but fall back to unencrypted:
647
648        nbd_set_tls (nbd, LIBNBD_TLS_ALLOW);
649
650       Use nbd_get_tls_negotiated(3) to find out if TLS negotiation was
651       successful.  Avoid "LIBNBD_TLS_ALLOW" if man-in-the-middle attacks are
652       a concern.
653
654       The most secure mode is to require TLS and fail to connect if the
655       server does not support it:
656
657        nbd_set_tls (nbd, LIBNBD_TLS_REQUIRE);
658
659       It may also be necessary to verify that the server’s identity is
660       correct.  For some servers it may be necessary to verify to the server
661       that the client is permitted to connect.  This can be done using either
662       X.509 certificates, or TLS Pre-Shared Keys (PSK).  Certificates are
663       more secure.  PSK is far more convenient, but you must have an existing
664       secure channel to distribute the keys.
665
666   Setting up X.509 using system certificate authorities (CAs)
667       This is the default if you don’t call any other "nbd_set_tls_*"
668       functions.  In this case the server must have a public (eg. HTTPS)
669       certificate which can be verified against the CAs registered on your
670       system (eg. under /etc/pki).
671
672       To disable server name verification — which opens you up to a potential
673       Man-In-The-Middle (MITM) attack — use:
674
675        nbd_set_tls_verify_peer (nbd, false);
676
677   Setting up an X.509 certificate authority (CA)
678       You can set up your own CA and register clients and servers with it,
679       issuing client and server certificates which will reliably authenticate
680       your clients and servers to each other.
681
682       Doing this is described in detail in the nbdkit-tls(1) manual.  The
683       only differences for libnbd are:
684
685       •   Non-root certificates must be placed in "$HOME/.pki/libnbd/" or
686           "$HOME/.config/pki/libnbd/"
687
688       •   Libnbd reads client-cert.pem and client-key.pem (instead of
689           server-cert.pem and server-key.pem).
690
691       Once you have set up the directory containing the certificates, call:
692
693        nbd_set_tls_certificates (nbd, "/path/to/directory");
694
695   Setting up Pre-Shared Keys (PSK)
696       TLS Pre-Shared Keys are a much more convenient method of setting up
697       TLS, and more appropriate for NBD, but you should have an existing
698       secure method available to distribute the keys.  They are therefore
699       ideal if you want to set up an NBD service as an adjunct to an existing
700       secure REST API.
701
702       Use psktool(1) to create a file of "username:key" pairs:
703
704        psktool -u username -p keys.psk
705
706       and pass this path to libnbd:
707
708        nbd_set_tls_psk_file (nbd, "keys.psk");
709
710       If necessary you may need to set the client username (otherwise libnbd
711       will use your login name):
712
713        nbd_set_tls_username (nbd, "username");
714

CALLBACKS

716       Some libnbd calls take callbacks (eg. nbd_set_debug_callback(3),
717       nbd_aio_pread(3)).  Libnbd can call these functions while processing.
718
719       In the C API these libnbd calls take a structure which contains the
720       function pointer and an optional opaque "void *user_data" pointer:
721
722        nbd_aio_pread (nbd, buf, sizeof buf, offset,
723                       (nbd_completion_callback) { .callback = my_fn,
724                                                   .user_data = my_data },
725                       0);
726
727       For optional callbacks, if you don't want the callback, either set
728       ".callback" to "NULL" or use the equivalent macros (such as
729       "NBD_NULL_COMPLETION") defined in "libnbd.h":
730
731        nbd_aio_pread (nbd, buf, sizeof buf, offset,
732                       NBD_NULL_COMPLETION, 0);
733
734       From other languages the structure and opaque pointer are not needed
735       because you can use closures to achieve the same effect.
736
737   Callback lifetimes
738       You can associate an optional free function with callbacks.  Libnbd
739       will call this function when the callback will not be called again by
740       libnbd, including in the case where the API fails.
741
742       This can be used to free associated "user_data".  For example:
743
744        void *my_data = malloc (...);
745
746        nbd_aio_pread_structured (nbd, buf, sizeof buf, offset,
747                       (nbd_chunk_callback) { .callback = my_fn,
748                                              .user_data = my_data,
749                                              .free = free },
750                       NBD_NULL_CALLBACK(completion),
751                       0);
752
753       will call free(3) on "my_data" after the last time that the
754       "chunk.callback = my_fn" function is called.
755
756       The free function is only accessible in the C API as it is not needed
757       in garbage collected programming languages.
758
759   Callbacks with ".callback=NULL" and ".free!=NULL"
760       It is possible to register a callback like this:
761
762         ...
763           (nbd_completion_callback) { .callback = NULL,
764                                       .user_data = my_data,
765                                       .free = free },
766         ...
767
768       The meaning of this is that the callback is never called, but the free
769       function is still called after the last time the callback would have
770       been called.  This is useful for applying generic freeing actions when
771       asynchronous commands are retired.
772
773   Callbacks and locking
774       The callbacks are invoked at a point where the libnbd lock is held; as
775       such, it is unsafe for the callback to call any "nbd_*" APIs on the
776       same nbd object, as it would cause deadlock.
777
778   Completion callbacks
779       All of the low-level commands have a completion callback variant that
780       registers a callback function used right before the command is marked
781       complete.
782
783       When the completion callback returns 1, the command is automatically
784       retired (there is no need to call nbd_aio_command_completed(3)); for
785       any other return value, the command still needs to be retired.
786
787   Callbacks with "int *error" parameter
788       Some of the high-level commands (nbd_pread_structured(3),
789       nbd_block_status(3)) involve the use of a callback function invoked by
790       the state machine at appropriate points in the server's reply before
791       the overall command is complete.  These callback functions, along with
792       all of the completion callbacks, include a parameter "error" containing
793       the value of any error detected so far; if the callback function fails,
794       it should assign back into "error" and return "-1" to change the
795       resulting error of the overall command.  Assignments into "error" are
796       ignored for any other return value; similarly, assigning 0 into "error"
797       does not have an effect.
798

COMPILING YOUR PROGRAM

800       On most systems, C programs that use libnbd can be compiled like this:
801
802        cc prog.c -o prog -lnbd
803
804       To detect if the libnbd library and header file is installed, the
805       preferred method is to use pkg-config(1) or pkgconf(1):
806
807        pkg-config libnbd --exists || fail libnbd is required
808
809       In case the library or header file are not installed in the usual
810       system locations, you can compile your program like this, using pkg-
811       config to detect the proper location of libnbd:
812
813        cc prog.c -o prog `pkg-config libnbd --cflags --libs`
814
815       To compile an external project against a built copy of the libnbd
816       source tree which hasn't been installed, see the ./run script.
817
818   Autoconf projects
819       External projects which use autoconf and need to check if libnbd is
820       installed should use the "PKG_CHECK_MODULES" macro in configure.ac like
821       this:
822
823        PKG_CHECK_MODULES([LIBNBD], [libnbd])
824
825       This will define "@LIBNBD_CFLAGS@" and "@LIBNBD_LIBS@" which you will
826       need to add to your Makefile.am.
827
828   CMake projects
829       For CMake projects use:
830
831        find_package(PkgConfig REQUIRED)
832        pkg_check_modules(LIBNBD REQUIRED libnbd)
833        target_link_libraries(prog ${LIBNBD_LIBRARIES})
834        target_include_directories(prog PUBLIC ${LIBNBD_INCLUDE_DIRS})
835        target_compile_options(prog PUBLIC ${LIBNBD_CFLAGS_OTHER})
836
837   Meson projects
838       For meson projects use:
839
840        nbd_dep = dependency('libnbd')
841        executable('prog', 'prog.c', dependencies : [nbd_dep])
842

ENVIRONMENT VARIABLES

844       "HOME"
845           Used in some situations to find TLS certificates.  See
846           nbd_set_tls_certificates(3).
847
848       "LIBNBD_DEBUG"
849           If this is set to the exact string 1 when the handle is created
850           then debugging is enabled.  See "DEBUGGING MESSAGES" above.
851
852       "LOGNAME"
853           The default TLS username.  See nbd_set_tls_username(3).
854

SEE ALSO

856   C API
857       nbd_add_meta_context(3), nbd_aio_block_status(3), nbd_aio_cache(3),
858       nbd_aio_command_completed(3), nbd_aio_connect(3),
859       nbd_aio_connect_command(3), nbd_aio_connect_socket(3),
860       nbd_aio_connect_systemd_socket_activation(3), nbd_aio_connect_tcp(3),
861       nbd_aio_connect_unix(3), nbd_aio_connect_uri(3),
862       nbd_aio_connect_vsock(3), nbd_aio_disconnect(3), nbd_aio_flush(3),
863       nbd_aio_get_direction(3), nbd_aio_get_fd(3), nbd_aio_in_flight(3),
864       nbd_aio_is_closed(3), nbd_aio_is_connecting(3), nbd_aio_is_created(3),
865       nbd_aio_is_dead(3), nbd_aio_is_negotiating(3),
866       nbd_aio_is_processing(3), nbd_aio_is_ready(3), nbd_aio_notify_read(3),
867       nbd_aio_notify_write(3), nbd_aio_opt_abort(3), nbd_aio_opt_go(3),
868       nbd_aio_opt_info(3), nbd_aio_opt_list(3),
869       nbd_aio_opt_list_meta_context(3), nbd_aio_peek_command_completed(3),
870       nbd_aio_pread(3), nbd_aio_pread_structured(3), nbd_aio_pwrite(3),
871       nbd_aio_trim(3), nbd_aio_zero(3), nbd_block_status(3), nbd_cache(3),
872       nbd_can_cache(3), nbd_can_df(3), nbd_can_fast_zero(3),
873       nbd_can_flush(3), nbd_can_fua(3), nbd_can_meta_context(3),
874       nbd_can_multi_conn(3), nbd_can_trim(3), nbd_can_zero(3),
875       nbd_clear_debug_callback(3), nbd_clear_meta_contexts(3), nbd_close(3),
876       nbd_connect_command(3), nbd_connect_socket(3),
877       nbd_connect_systemd_socket_activation(3), nbd_connect_tcp(3),
878       nbd_connect_unix(3), nbd_connect_uri(3), nbd_connect_vsock(3),
879       nbd_connection_state(3), nbd_create(3), nbd_flush(3),
880       nbd_get_block_size(3), nbd_get_canonical_export_name(3),
881       nbd_get_debug(3), nbd_get_errno(3), nbd_get_error(3),
882       nbd_get_export_description(3), nbd_get_export_name(3),
883       nbd_get_full_info(3), nbd_get_handle_name(3),
884       nbd_get_handshake_flags(3), nbd_get_meta_context(3),
885       nbd_get_nr_meta_contexts(3), nbd_get_opt_mode(3),
886       nbd_get_package_name(3), nbd_get_private_data(3), nbd_get_protocol(3),
887       nbd_get_request_structured_replies(3), nbd_get_size(3),
888       nbd_get_strict_mode(3), nbd_get_structured_replies_negotiated(3),
889       nbd_get_tls(3), nbd_get_tls_negotiated(3), nbd_get_tls_username(3),
890       nbd_get_tls_verify_peer(3), nbd_get_uri(3), nbd_get_version(3),
891       nbd_is_read_only(3), nbd_is_rotational(3), nbd_kill_subprocess(3),
892       nbd_opt_abort(3), nbd_opt_go(3), nbd_opt_info(3), nbd_opt_list(3),
893       nbd_opt_list_meta_context(3), nbd_poll(3), nbd_pread(3),
894       nbd_pread_structured(3), nbd_pwrite(3), nbd_set_debug(3),
895       nbd_set_debug_callback(3), nbd_set_export_name(3),
896       nbd_set_full_info(3), nbd_set_handle_name(3),
897       nbd_set_handshake_flags(3), nbd_set_opt_mode(3),
898       nbd_set_private_data(3), nbd_set_request_structured_replies(3),
899       nbd_set_strict_mode(3), nbd_set_tls(3), nbd_set_tls_certificates(3),
900       nbd_set_tls_psk_file(3), nbd_set_tls_username(3),
901       nbd_set_tls_verify_peer(3), nbd_set_uri_allow_local_file(3),
902       nbd_set_uri_allow_tls(3), nbd_set_uri_allow_transports(3),
903       nbd_shutdown(3), nbd_supports_tls(3), nbd_supports_uri(3), nbd_trim(3),
904       nbd_zero(3).
905
906   Servers
907       nbdkit(1), nbd-server(1), qemu-nbd(8).
908
909   Encryption tools
910       certtool(1), nbdkit-tls(1), psktool(1).
911
912   Standards
913       https://github.com/NetworkBlockDevice/nbd/blob/master/doc/proto.md,
914       https://github.com/NetworkBlockDevice/nbd/blob/master/doc/uri.md.
915
916   Release notes
917       libnbd-release-notes-1.8(1), libnbd-release-notes-1.6(1),
918       libnbd-release-notes-1.4(1), libnbd-release-notes-1.2(1).
919
920   Other
921       libnbd-security(3), nbdcopy(1), nbdfuse(1), nbdinfo(1), nbdsh(1),
922       qemu(1).
923

AUTHORS

925       Eric Blake
926
927       Richard W.M. Jones
928
930       Copyright (C) 2019-2021 Red Hat Inc.
931

LICENSE

933       This library is free software; you can redistribute it and/or modify it
934       under the terms of the GNU Lesser General Public License as published
935       by the Free Software Foundation; either version 2 of the License, or
936       (at your option) any later version.
937
938       This library is distributed in the hope that it will be useful, but
939       WITHOUT ANY WARRANTY; without even the implied warranty of
940       MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
941       Lesser General Public License for more details.
942
943       You should have received a copy of the GNU Lesser General Public
944       License along with this library; if not, write to the Free Software
945       Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
946       02110-1301 USA
947
948
949
950libnbd-1.10.1                     2021-10-25                         libnbd(3)
Impressum