1nbdkit(1) NBDKIT nbdkit(1)
2
6 nbdkit - toolkit for creating NBD servers
7
9 nbdkit [-D|--debug PLUGIN|FILTER|nbdkit.FLAG=N]
10 [-e|--exportname EXPORTNAME] [--exit-with-parent]
11 [--filter FILTER ...] [-f|--foreground]
12 [-g|--group GROUP] [-i|--ipaddr IPADDR]
13 [--log stderr|syslog|null]
14 [-n|--newstyle] [--mask-handshake MASK] [--no-sr] [-o|--oldstyle]
15 [-P|--pidfile PIDFILE]
16 [-p|--port PORT] [-r|--readonly]
17 [--run CMD] [-s|--single] [--selinux-label LABEL] [--swap]
18 [-t|--threads THREADS]
19 [--tls off|on|require]
20 [--tls-certificates /path/to/certificates]
21 [--tls-psk /path/to/pskfile] [--tls-verify-peer]
22 [-U|--unix SOCKET] [-u|--user USER]
23 [-v|--verbose] [-V|--version] [--vsock]
24 PLUGIN [[KEY=]VALUE [KEY=VALUE [...]]]
25 nbdkit --dump-config
27 nbdkit PLUGIN --dump-plugin
29 nbdkit --help
31
33 Network Block Device (NBD) is a network protocol for accessing block
34 devices over the network. Block devices are hard disks and things that
35 behave like hard disks such as disk images and virtual machines.
36 nbdkit is both a toolkit for creating NBD servers from “unconventional”
38 sources, and the name of an NBD server. nbdkit ships with many plugins
39 for performing common tasks like serving local files.
40 Plugins and filters
42 nbdkit is different from other NBD servers because you can easily
43 create new Network Block Device sources by writing a few glue
44 functions, possibly in C, or perhaps in a high level language like Perl
45 or Python. The liberal licensing of nbdkit is meant to allow you to
46 link nbdkit with proprietary libraries or to include nbdkit in
47 proprietary code.
48 If you want to write your own nbdkit plugin you should read
50 nbdkit-plugin(3).
51 nbdkit also has a concept of filters which can be layered on top of
53 plugins. Several filters are provided with nbdkit and if you want to
54 write your own you should read nbdkit-filter(3).
55
57 Basic file serving
58 · Serve file disk.img on port 10809 using nbdkit-file-plugin(1), and
59 connect to it using guestfish(1):
60 nbdkit file disk.img
62 guestfish --rw --format=raw -a nbd://localhost
63 · Serve file disk.img on port 10809, requiring clients to use
65 encrypted (TLS) connections:
66 nbdkit --tls=require file disk.img
68 Other nbdkit plugins
70 · Create a 1MB disk with one empty partition entirely on the command
71 line using nbdkit-data-plugin(1):
72 nbdkit data size=1M \
74 data="@0x1b8 0xf8 0x21 0xdc 0xeb 0 0 0 0
75 2 0 0x83 0x20 0x20 0 1 0 0 0 0xff 0x7
76 @0x1fe 0x55 0xaa"
77 · Forward an NBD connection to a remote server over HTTPS or SSH
79 using nbdkit-curl-plugin(1) or nbdkit-ssh-plugin(1):
80 nbdkit -r curl https://example.com/disk.img
82 nbdkit ssh host=example.com /var/tmp/disk.img
84 · Create a RAM disk using nbdkit-memory-plugin(1):
86 nbdkit memory 64M
88 · Create a floppy disk image containing files from a local directory
90 using nbdkit-floppy-plugin(1):
91 nbdkit floppy dir/
93 Combining plugins and filters
95 · Serve only the first partition from compressed disk image
96 disk.img.xz, combining nbdkit-partition-filter(1),
97 nbdkit-xz-filter(1) and nbdkit-file-plugin(1).
98 nbdkit --filter=partition --filter=xz file disk.img.xz partition=1
100 To understand this command line:
102 plugin name and plugin parameter
104 │
105 ┌───────┴──────┐
106 │ │
107 nbdkit --filter=partition --filter=xz file disk.img.xz partition=1
108 │ │ │
109 └──────────────┴────┬─────────────────────┘
110 │
111 filters and filter parameter
112 · Create a scratch, empty nbdkit device and inject errors and delays,
114 for testing clients, using nbdkit-memory-plugin(1),
115 nbdkit-error-filter(1) and nbdkit-delay-filter(1):
116 nbdkit --filter=error --filter=delay memory 100M \
118 error-rate=10% rdelay=1 wdelay=1
119 Writing plugins in shell script
121 · Write a simple, custom plugin in shell script using
122 nbdkit-sh-plugin(3):
123 nbdkit sh - <<'EOF'
125 case "$1" in
126 get_size) echo 1M ;;
127 pread) dd if=/dev/zero count=$3 iflag=count_bytes ;;
128 *) exit 2 ;;
129 esac
130 EOF
131 · The same example as above can be written entirely on the command
133 line using nbdkit-eval-plugin(1):
134 nbdkit eval get_size='echo 1M' \
136 pread='dd if=/dev/zero count=$3 iflag=count_bytes'
137 Display information
139 Display information about nbdkit or a specific plugin:
140 nbdkit --help
142 nbdkit --version
143 nbdkit --dump-config
144 nbdkit example1 --help
145 nbdkit example1 --dump-plugin
146
148 --help
149 Display brief command line usage information and exit.
150 -D PLUGIN.FLAG=N
152 -D FILTER.FLAG=N
153 --debug PLUGIN.FLAG=N
154 --debug FILTER.FLAG=N
155 Set the plugin or filter Debug Flag called "FLAG" to the integer
156 value "N". See "Debug Flags" in nbdkit-plugin(3).
157 -D nbdkit.FLAG=N
159 --debug nbdkit.FLAG=N
160 Set the nbdkit server Debug Flag called "FLAG" to the integer value
161 "N". See "SERVER DEBUG FLAGS" below.
162 --dump-config
164 Dump out the compile-time configuration values and exit. See
165 nbdkit-probing(1).
166 --dump-plugin
168 Dump out information about the plugin and exit. See
169 nbdkit-probing(1).
170 --exit-with-parent
172 If the parent process exits, we exit. This can be used to avoid
173 complicated cleanup or orphaned nbdkit processes. There are some
174 important caveats with this, see "EXIT WITH PARENT" in
175 nbdkit-captive(1).
176 An alternative to this is "CAPTIVE NBDKIT" in nbdkit-captive(1).
178 This option implies --foreground.
180 -e EXPORTNAME
182 --export EXPORTNAME
183 --export-name EXPORTNAME
184 --exportname EXPORTNAME
185 Set the exportname.
186 If not set, exportname "" (empty string) is used. Exportnames are
188 not allowed with the oldstyle protocol.
189 -f
191 --foreground
192 --no-fork
193 Don't fork into the background.
194 --filter FILTER
196 Add a filter before the plugin. This option may be given one or
197 more times to stack filters in front of the plugin. They are
198 processed in the order they appear on the command line. See
199 "FILTERS" and nbdkit-filter(3).
200 -g GROUP
202 --group GROUP
203 Change group to "GROUP" after starting up. A group name or numeric
204 group ID can be used.
205 The server needs sufficient permissions to be able to do this.
207 Normally this would mean starting the server up as root.
208 See also -u.
210 -i IPADDR
212 --ip-addr IPADDR
213 --ipaddr IPADDR
214 Listen on the specified interface. The default is to listen on all
215 interfaces. See also -p.
216 --log=stderr
218 --log=syslog
219 --log=null
220 Send error messages to standard error (--log=stderr), or to the
221 system log (--log=syslog), or discard them completely (--log=null,
222 not recommended for normal use).
223 The default is to send error messages to stderr, unless nbdkit
225 forks into the background in which case they are sent to syslog.
226 For more details see "LOGGING" in nbdkit-service(1).
228 -n
230 --new-style
231 --newstyle
232 Use the newstyle NBD protocol. This is the default in nbdkit ≥
233 1.3. In earlier versions the default was oldstyle. See
234 nbdkit-protocol(1).
235 --no-sr
237 Do not advertise structured replies. A client must request
238 structured replies to take advantage of block status and potential
239 sparse reads; however, as structured reads are not a mandatory part
240 of the newstyle NBD protocol, this option can be used to debug
241 client fallbacks for dealing with older servers. See
242 nbdkit-protocol(1).
243 -o
245 --old-style
246 --oldstyle
247 Use the oldstyle NBD protocol. This was the default in nbdkit ≤
248 1.2, but now the default is newstyle. Note this is incompatible
249 with newer features such as export names and TLS. See
250 nbdkit-protocol(1).
251 -P PIDFILE
253 --pid-file PIDFILE
254 --pidfile PIDFILE
255 Write "PIDFILE" (containing the process ID of the server) after
256 nbdkit becomes ready to accept connections.
257 If the file already exists, it is overwritten. nbdkit does not
259 delete the file when it exits.
260 -p PORT
262 --port PORT
263 Change the TCP/IP port number on which nbdkit serves requests. The
264 default is 10809. See also -i.
265 -r
267 --read-only
268 --readonly
269 The export will be read-only. If a client writes, then it will get
270 an error.
271 Note that some plugins inherently don't support writes. With those
273 plugins the -r option is added implicitly.
274 nbdkit-cow-filter(1) can be placed over read-only plugins to
276 provide copy-on-write (or "snapshot") functionality. If you are
277 using qemu as a client then it also supports snapshots.
278 --run CMD
280 Run nbdkit as a captive subprocess of "CMD". When "CMD" exits,
281 nbdkit is killed. See "CAPTIVE NBDKIT" in nbdkit-captive(1).
282 This option implies --foreground.
284 -s
286 --single
287 --stdin
288 Don't fork. Handle a single NBD connection on stdin/stdout. After
289 stdin closes, the server exits.
290 You can use this option to run nbdkit from inetd or similar
292 superservers; or just for testing; or if you want to run nbdkit in
293 a non-conventional way. Note that if you want to run nbdkit from
294 systemd, then it may be better to use "SOCKET ACTIVATION" in
295 nbdkit-service(1) instead of this option.
296 This option implies --foreground.
298 --selinux-label SOCKET-LABEL
300 Apply the SELinux label "SOCKET-LABEL" to the nbdkit listening
301 socket.
302 The common — perhaps only — use of this option is to allow libvirt
304 guests which are using SELinux and sVirt confinement to access
305 nbdkit Unix domain sockets:
306 nbdkit --selinux-label system_u:object_r:svirt_t:s0 ...
308 --swap
310 Specifies that the NBD device will be used as swap space loop
311 mounted on the same machine which is running nbdkit. To avoid
312 deadlocks this locks the whole nbdkit process into memory using
313 mlockall(2). This may require additional permissions, such as
314 starting the server as root or raising the "RLIMIT_MEMLOCK"
315 (ulimit(1) -l) limit on the process.
316 -t THREADS
318 --threads THREADS
319 Set the number of threads to be used per connection, which in turn
320 controls the number of outstanding requests that can be processed
321 at once. Only matters for plugins with thread_model=parallel
322 (where it defaults to 16). To force serialized behavior (useful if
323 the client is not prepared for out-of-order responses), set this to
324 1.
325 --tls=off
327 --tls=on
328 --tls=require
329 Disable, enable or require TLS (authentication and encryption
330 support). See nbdkit-tls(1).
331 --tls-certificates /path/to/certificates
333 Set the path to the TLS certificates directory. If not specified,
334 some built-in paths are checked. See nbdkit-tls(1) for more
335 details.
336 --tls-psk /path/to/pskfile
338 Set the path to the pre-shared keys (PSK) file. If used, this
339 overrides certificate authentication. There is no built-in path.
340 See nbdkit-tls(1) for more details.
341 --tls-verify-peer
343 Enables TLS client certificate verification. The default is not to
344 check the client's certificate.
345 -U SOCKET
347 --unix SOCKET
348 -U -
349 --unix -
350 Accept connections on the Unix domain socket "SOCKET" (which is a
351 path).
352 nbdkit creates this socket, but it will probably have incorrect
354 permissions (too permissive). If it is a problem that some
355 unauthorized user could connect to this socket between the time
356 that nbdkit starts up and the authorized user connects, then put
357 the socket into a directory that has restrictive permissions.
358 nbdkit does not delete the socket file when it exits. The caller
360 should delete the socket file after use (else if you try to start
361 nbdkit up again you will get an "Address already in use" error).
362 If the socket name is - then nbdkit generates a randomly named
364 private socket. This is useful with "CAPTIVE NBDKIT" in
365 nbdkit-captive(1).
366 -u USER
368 --user USER
369 Change user to "USER" after starting up. A user name or numeric
370 user ID can be used.
371 The server needs sufficient permissions to be able to do this.
373 Normally this would mean starting the server up as root.
374 See also -g.
376 -v
378 --verbose
379 Enable verbose messages.
380 It's a good idea to use -f as well so the process does not fork
382 into the background (but not required).
383 -V
385 --version
386 Print the version number of nbdkit and exit.
387 --vsock
389 Use the AF_VSOCK protocol (instead of TCP/IP). You must use this
390 in conjunction with -p/--port. See "AF_VSOCK" in
391 nbdkit-service(1).
392
394 You can give the full path to the plugin, like this:
395 nbdkit $libdir/nbdkit/plugins/nbdkit-file-plugin.so [...]
397 but it is usually more convenient to use this equivalent syntax:
399 nbdkit file [...]
401 $libdir is set at compile time. To print it out, do:
403 nbdkit --dump-config
405
407 After specifying the plugin name you can (optionally, it depends on the
408 plugin) give plugin configuration on the command line in the form of
409 "key=value". For example:
410 nbdkit file file=disk.img
412 To list all the options supported by a plugin, do:
414 nbdkit --help file
416 To dump information about a plugin, do:
418 nbdkit file --dump-plugin
420 Magic parameters
422 Some plugins declare a special "magic config key". This is a key which
423 is assumed if no "key=" part is present. For example:
424 nbdkit file disk.img
426 is assumed to be "file=disk.img" because the file plugin declares
428 "file" as its magic config key. There can be ambiguity in the parsing
429 of magic config keys if the value might look like a "key=value". If
430 there could be ambiguity then modify the value, eg. by prefixing it
431 with "./"
432 There is also a special exception for plugins which do not declare a
434 magic config key, but where the first plugin argument does not contain
435 an '=' character: it is assumed to be "script=value". This is used by
436 scripting language plugins:
437 nbdkit perl foo.pl [args...]
439 has the same meaning as:
441 nbdkit perl script=foo.pl [args...]
443 Shebang scripts
445 You can use "#!" to run nbdkit plugins written in most scripting
446 languages. The file should be executable. For example:
447 #!/usr/sbin/nbdkit perl
449 sub open {
450 # etc
451 }
452 (see nbdkit-perl-plugin(3) for a full example).
454
456 As well as enabling or disabling debugging in the server using
457 --verbose you can control extra debugging in the server using the "-D
458 nbdkit.*" flags listed in this section. Note these flags are an
459 internal implementation detail of the server and may be changed or
460 removed at any time in the future.
461 -D nbdkit.backend.controlpath=0
463 -D nbdkit.backend.controlpath=1
464 -D nbdkit.backend.datapath=0
465 -D nbdkit.backend.datapath=1
466 These flags control the verbosity of nbdkit backend debugging
467 messages (the ones which show every request processed by the
468 server). The default for both settings is 1 (normal debugging) but
469 you can set them to 0 to suppress these messages.
470 "-D nbdkit.backend.datapath=0" is the more useful setting which
472 lets you suppress messages about pread, pwrite, zero, trim, etc.
473 commands. When transferring large amounts of data these messages
474 are numerous and not usually very interesting.
475 "-D nbdkit.backend.controlpath=0" suppresses the non-datapath
477 commands (config, open, close, can_write, etc.)
478
480 nbdkit responds to the following signals:
481 "SIGINT"
483 "SIGQUIT"
484 "SIGTERM"
485 The server exits cleanly.
486 "SIGPIPE"
488 This signal is ignored.
489
491 "LISTEN_FDS"
492 "LISTEN_PID"
493 If present in the environment when nbdkit starts up, these trigger
494 "SOCKET ACTIVATION" in nbdkit-service(1).
495
497 Other topics
498 nbdkit-captive(1) — Run nbdkit under another process and have it
499 reliably cleaned up.
500 nbdkit-loop(1) — Use nbdkit with the Linux kernel client to create loop
502 devices and loop mounts.
503 nbdkit-probing(1) — How to probe for nbdkit configuration and plugins.
505 nbdkit-protocol(1) — Which parts of the NBD protocol nbdkit supports.
507 nbdkit-security(1) — Lists past security issues in nbdkit.
509 nbdkit-service(1) — Running nbdkit as a service, and systemd socket
511 activation.
512 nbdkit-tls(1) — Authentication and encryption of NBD connections
514 (sometimes incorrectly called "SSL").
515 Plugins
517 nbdkit-curl-plugin(1), nbdkit-data-plugin(1), nbdkit-eval-plugin(1),
518 nbdkit-example1-plugin(1), nbdkit-example2-plugin(1),
519 nbdkit-example3-plugin(1), nbdkit-example4-plugin(1),
520 nbdkit-ext2-plugin(1), nbdkit-file-plugin(1), nbdkit-floppy-plugin(1),
521 nbdkit-full-plugin(1), nbdkit-guestfs-plugin(1), nbdkit-gzip-plugin(1),
522 nbdkit-info-plugin(1), nbdkit-iso-plugin(1), nbdkit-libvirt-plugin(1),
523 nbdkit-linuxdisk-plugin(1), nbdkit-memory-plugin(1),
524 nbdkit-nbd-plugin(1), nbdkit-null-plugin(1),
525 nbdkit-partitioning-plugin(1), nbdkit-pattern-plugin(1),
526 nbdkit-random-plugin(1), nbdkit-split-plugin(1), nbdkit-ssh-plugin(1),
527 nbdkit-streaming-plugin(1), nbdkit-tar-plugin(1),
528 nbdkit-vddk-plugin(1), nbdkit-zero-plugin(1) ; nbdkit-lua-plugin(3),
529 nbdkit-ocaml-plugin(3), nbdkit-perl-plugin(3), nbdkit-python-plugin(3),
530 nbdkit-ruby-plugin(3), nbdkit-rust-plugin(3), nbdkit-sh-plugin(3),
531 nbdkit-tcl-plugin(3) .
532 Filters
534 nbdkit-blocksize-filter(1), nbdkit-cache-filter(1),
535 nbdkit-cacheextents-filter(1), nbdkit-cow-filter(1),
536 nbdkit-delay-filter(1), nbdkit-error-filter(1), nbdkit-ext2-filter(1),
537 nbdkit-extentlist-filter(1), nbdkit-fua-filter(1), nbdkit-ip-filter(1),
538 nbdkit-log-filter(1), nbdkit-nocache-filter(1),
539 nbdkit-noextents-filter(1), nbdkit-nofilter-filter(1),
540 nbdkit-noparallel-filter(1), nbdkit-nozero-filter(1),
541 nbdkit-offset-filter(1), nbdkit-partition-filter(1),
542 nbdkit-rate-filter(1), nbdkit-readahead-filter(1),
543 nbdkit-retry-filter(1), nbdkit-stats-filter(1),
544 nbdkit-truncate-filter(1), nbdkit-xz-filter(1) .
545 For developers
547 nbdkit-plugin(3), nbdkit-filter(3).
548 Writing plugins in other programming languages
550 nbdkit-lua-plugin(3), nbdkit-ocaml-plugin(3), nbdkit-perl-plugin(3),
551 nbdkit-python-plugin(3), nbdkit-ruby-plugin(3), nbdkit-rust-plugin(3),
552 nbdkit-sh-plugin(3), nbdkit-tcl-plugin(3) .
553 Release notes for previous releases of nbdkit
555 nbdkit-release-notes-1.4(1), nbdkit-release-notes-1.6(1),
556 nbdkit-release-notes-1.8(1), nbdkit-release-notes-1.10(1),
557 nbdkit-release-notes-1.12(1), nbdkit-release-notes-1.14(1),
558 nbdkit-release-notes-1.16(1), nbdkit-release-notes-1.18(1).
559 NBD clients
561 guestfish(1), libnbd(3), nbd-client(1), nbdfuse(1), nbdsh(1), qemu(1).
562 nbdkit links
564 http://github.com/libguestfs/nbdkit — Source code.
565 Other NBD servers
567 qemu-nbd(1), nbd-server(1), https://bitbucket.org/hirofuchi/xnbd.
568 Documentation for the NBD protocol
570 https://github.com/NetworkBlockDevice/nbd/blob/master/doc/proto.md,
571 https://nbd.sourceforge.io/.
572 Similar protocols
574 https://en.wikipedia.org/wiki/iSCSI,
575 https://en.wikipedia.org/wiki/ATA_over_Ethernet,
576 https://en.wikipedia.org/wiki/Fibre_Channel_over_Ethernet.
577 Other manual pages of interest
579 gnutls_priority_init(3), qemu-img(1), psktool(1), systemd.socket(5).
580
582 Eric Blake
583 Richard W.M. Jones
585 Yann E. MORIN
587 Nir Soffer
589 Pino Toscano
591
593 Copyright (C) 2013-2019 Red Hat Inc.
594
596 Redistribution and use in source and binary forms, with or without
597 modification, are permitted provided that the following conditions are
598 met:
599 · Redistributions of source code must retain the above copyright
601 notice, this list of conditions and the following disclaimer.
602 · Redistributions in binary form must reproduce the above copyright
604 notice, this list of conditions and the following disclaimer in the
605 documentation and/or other materials provided with the
606 distribution.
607 · Neither the name of Red Hat nor the names of its contributors may
609 be used to endorse or promote products derived from this software
610 without specific prior written permission.
611 THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND ANY
613 EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
614 IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
615 PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR CONTRIBUTORS BE
616 LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
617 CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
618 SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
619 BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
620 WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
621 OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
622 ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
623nbdkit-1.18.4 2020-04-16 nbdkit(1)