1nbdkit(1) NBDKIT nbdkit(1)
2
6 nbdkit - toolkit for creating NBD servers
7
9 nbdkit [-D|--debug PLUGIN|FILTER|nbdkit.FLAG=N]
10 [-e|--exportname EXPORTNAME] [--exit-with-parent]
11 [--filter FILTER ...] [-f|--foreground]
12 [-g|--group GROUP] [-i|--ipaddr IPADDR]
13 [--log stderr|syslog|null]
14 [-n|--newstyle] [--mask-handshake MASK] [--no-sr] [-o|--oldstyle]
15 [-P|--pidfile PIDFILE]
16 [-p|--port PORT] [-r|--readonly]
17 [--run CMD] [-s|--single] [--selinux-label LABEL] [--swap]
18 [-t|--threads THREADS]
19 [--tls off|on|require]
20 [--tls-certificates /path/to/certificates]
21 [--tls-psk /path/to/pskfile] [--tls-verify-peer]
22 [-U|--unix SOCKET] [-u|--user USER]
23 [-v|--verbose] [-V|--version] [--vsock]
24 PLUGIN [[KEY=]VALUE [KEY=VALUE [...]]]
25 nbdkit --dump-config
27 nbdkit PLUGIN --dump-plugin
29 nbdkit --help
31
33 Network Block Device (NBD) is a network protocol for accessing block
34 devices over the network. Block devices are hard disks and things that
35 behave like hard disks such as disk images and virtual machines.
36 nbdkit is both a toolkit for creating NBD servers from “unconventional”
38 sources, and the name of an NBD server. nbdkit ships with many plugins
39 for performing common tasks like serving local files.
40 Plugins and filters
42 nbdkit is different from other NBD servers because you can easily
43 create new Network Block Device sources by writing a few glue
44 functions, possibly in C, or perhaps in a high level language like Perl
45 or Python. The liberal licensing of nbdkit is meant to allow you to
46 link nbdkit with proprietary libraries or to include nbdkit in
47 proprietary code.
48 If you want to write your own nbdkit plugin you should read
50 nbdkit-plugin(3).
51 nbdkit also has a concept of filters which can be layered on top of
53 plugins. Several filters are provided with nbdkit and if you want to
54 write your own you should read nbdkit-filter(3).
55
57 Basic file serving
58 • Serve file disk.img on port 10809 using nbdkit-file-plugin(1), and
59 connect to it using guestfish(1):
60 nbdkit file disk.img
62 guestfish --rw --format=raw -a nbd://localhost
63 • Serve file disk.img on port 10809, requiring clients to use
65 encrypted (TLS) connections:
66 nbdkit --tls=require file disk.img
68 Other nbdkit plugins
70 • Create a small disk containing test patterns using
71 nbdkit-data-plugin(1):
72 nbdkit data ' ( 0x55 0xAA )*2048 '
74 • Forward an NBD connection to a remote server over HTTPS or SSH
76 using nbdkit-curl-plugin(1) or nbdkit-ssh-plugin(1):
77 nbdkit -r curl https://example.com/disk.img
79 nbdkit ssh host=example.com /var/tmp/disk.img
81 • Create a sparse 1 terabyte RAM disk using nbdkit-memory-plugin(1)
83 and use it as a loop device (nbdkit-loop(1)):
84 nbdkit memory 1T
86 nbd-client -b 512 localhost /dev/nbd0
87 • Create a floppy disk image containing files from a local directory
89 using nbdkit-floppy-plugin(1):
90 nbdkit floppy dir/
92 Combining plugins and filters
94 • Serve only the first partition from compressed disk image
95 disk.img.xz, combining nbdkit-partition-filter(1),
96 nbdkit-xz-filter(1) and nbdkit-file-plugin(1).
97 nbdkit --filter=partition --filter=xz file disk.img.xz partition=1
99 To understand this command line:
101 plugin name and plugin parameter
103 │
104 ┌───────┴──────┐
105 │ │
106 nbdkit --filter=partition --filter=xz file disk.img.xz partition=1
107 │ │ │
108 └──────────────┴────┬─────────────────────┘
109 │
110 filters and filter parameter
111 • Create a scratch, empty nbdkit device and inject errors and delays,
113 for testing clients, using nbdkit-memory-plugin(1),
114 nbdkit-error-filter(1) and nbdkit-delay-filter(1):
115 nbdkit --filter=error --filter=delay memory 100M \
117 error-rate=10% rdelay=1 wdelay=1
118 Writing plugins in shell script
120 • Write a simple, custom plugin in shell script using
121 nbdkit-sh-plugin(3):
122 nbdkit sh - <<'EOF'
124 case "$1" in
125 get_size) echo 1M ;;
126 pread) dd if=/dev/zero count=$3 iflag=count_bytes ;;
127 *) exit 2 ;;
128 esac
129 EOF
130 • The same example as above can be written entirely on the command
132 line using nbdkit-eval-plugin(1):
133 nbdkit eval get_size='echo 1M' \
135 pread='dd if=/dev/zero count=$3 iflag=count_bytes'
136 Display information
138 Display information about nbdkit or a specific plugin:
139 nbdkit --help
141 nbdkit --version
142 nbdkit --dump-config
143 nbdkit example1 --help
144 nbdkit example1 --dump-plugin
145
147 --help
148 Display brief command line usage information and exit.
149 -D PLUGIN.FLAG=N
151 -D FILTER.FLAG=N
152 --debug PLUGIN.FLAG=N
153 --debug FILTER.FLAG=N
154 Set the plugin or filter Debug Flag called "FLAG" to the integer
155 value "N". See "Debug Flags" in nbdkit-plugin(3).
156 -D nbdkit.FLAG=N
158 --debug nbdkit.FLAG=N
159 (nbdkit ≥ 1.18)
160 Set the nbdkit server Debug Flag called "FLAG" to the integer value
162 "N". See "SERVER DEBUG FLAGS" below.
163 --dump-config
165 Dump out the compile-time configuration values and exit. See
166 nbdkit-probing(1).
167 --dump-plugin
169 Dump out information about the plugin and exit. See
170 nbdkit-probing(1).
171 --exit-with-parent
173 If the parent process exits, we exit. This can be used to avoid
174 complicated cleanup or orphaned nbdkit processes. There are some
175 important caveats with this, see "EXIT WITH PARENT" in
176 nbdkit-captive(1).
177 An alternative to this is "CAPTIVE NBDKIT" in nbdkit-captive(1).
179 This option implies --foreground.
181 -e EXPORTNAME
183 --export EXPORTNAME
184 --export-name EXPORTNAME
185 --exportname EXPORTNAME
186 Set a preferred exportname to expose in the shell environment
187 created during --run. The use of this option without --run has no
188 effect. This option does not change what nbdkit advertises as a
189 server, but can aid in writing a captive client that wants to
190 access particular content from a plugin that differentiates content
191 based on the client's choice of export name.
192 If not set, the --run environment is set to access the default
194 exportname "" (empty string).
195 -f
197 --foreground
198 --no-fork
199 Don't fork into the background.
200 --filter FILTER
202 Add a filter before the plugin. This option may be given one or
203 more times to stack filters in front of the plugin. They are
204 processed in the order they appear on the command line. See
205 "FILTERS" and nbdkit-filter(3).
206 -g GROUP
208 --group GROUP
209 Change group to "GROUP" after starting up. A group name or numeric
210 group ID can be used.
211 The server needs sufficient permissions to be able to do this.
213 Normally this would mean starting the server up as root.
214 See also -u.
216 -i IPADDR
218 --ip-addr IPADDR
219 --ipaddr IPADDR
220 Listen on the specified interface. The default is to listen on all
221 interfaces. See also -p.
222 --log=stderr
224 --log=syslog
225 --log=null
226 Send error messages to standard error (--log=stderr), or to the
227 system log (--log=syslog), or discard them completely (--log=null,
228 not recommended for normal use).
229 The default is to send error messages to stderr, unless nbdkit
231 forks into the background in which case they are sent to syslog.
232 For more details see "LOGGING" in nbdkit-service(1).
234 -n
236 --new-style
237 --newstyle
238 Use the newstyle NBD protocol. This is the default in nbdkit ≥
239 1.3. In earlier versions the default was oldstyle. See
240 nbdkit-protocol(1).
241 --no-sr
243 Do not advertise structured replies. A client must request
244 structured replies to take advantage of block status and potential
245 sparse reads; however, as structured reads are not a mandatory part
246 of the newstyle NBD protocol, this option can be used to debug
247 client fallbacks for dealing with older servers. See
248 nbdkit-protocol(1).
249 -o
251 --old-style
252 --oldstyle
253 Use the oldstyle NBD protocol. This was the default in nbdkit ≤
254 1.2, but now the default is newstyle. Note this is incompatible
255 with newer features such as export names and TLS. See
256 nbdkit-protocol(1).
257 -P PIDFILE
259 --pid-file PIDFILE
260 --pidfile PIDFILE
261 Write "PIDFILE" (containing the process ID of the server) after
262 nbdkit becomes ready to accept connections.
263 If the file already exists, it is overwritten. nbdkit does not
265 delete the file when it exits.
266 -p PORT
268 --port PORT
269 Change the TCP/IP port number on which nbdkit serves requests. The
270 default is 10809. See also -i.
271 -r
273 --read-only
274 --readonly
275 The export will be read-only. If a client writes, then it will get
276 an error.
277 Note that some plugins inherently don't support writes. With those
279 plugins the -r option is added implicitly.
280 nbdkit-cow-filter(1) can be placed over read-only plugins to
282 provide copy-on-write (or "snapshot") functionality. If you are
283 using qemu as a client then it also supports snapshots.
284 --run CMD
286 Run nbdkit as a captive subprocess of "CMD". When "CMD" exits,
287 nbdkit is killed. See "CAPTIVE NBDKIT" in nbdkit-captive(1).
288 This option implies --foreground.
290 -s
292 --single
293 --stdin
294 Don't fork. Handle a single NBD connection on stdin/stdout. After
295 stdin closes, the server exits.
296 You can use this option to run nbdkit from inetd or similar
298 superservers; or just for testing; or if you want to run nbdkit in
299 a non-conventional way. Note that if you want to run nbdkit from
300 systemd, then it may be better to use "SOCKET ACTIVATION" in
301 nbdkit-service(1) instead of this option.
302 This option implies --foreground.
304 --selinux-label SOCKET-LABEL
306 Apply the SELinux label "SOCKET-LABEL" to the nbdkit listening
307 socket.
308 The common — perhaps only — use of this option is to allow libvirt
310 guests which are using SELinux and sVirt confinement to access
311 nbdkit Unix domain sockets:
312 nbdkit --selinux-label system_u:object_r:svirt_t:s0 ...
314 --swap
316 (nbdkit ≥ 1.18)
317 Specifies that the NBD device will be used as swap space loop
319 mounted on the same machine which is running nbdkit. To avoid
320 deadlocks this locks the whole nbdkit process into memory using
321 mlockall(2). This may require additional permissions, such as
322 starting the server as root or raising the "RLIMIT_MEMLOCK"
323 (ulimit(1) -l) limit on the process.
324 -t THREADS
326 --threads THREADS
327 Set the number of threads to be used per connection, which in turn
328 controls the number of outstanding requests that can be processed
329 at once. Only matters for plugins with thread_model=parallel
330 (where it defaults to 16). To force serialized behavior (useful if
331 the client is not prepared for out-of-order responses), set this to
332 1.
333 --tls=off
335 --tls=on
336 --tls=require
337 Disable, enable or require TLS (authentication and encryption
338 support). See nbdkit-tls(1).
339 --tls-certificates /path/to/certificates
341 Set the path to the TLS certificates directory. If not specified,
342 some built-in paths are checked. See nbdkit-tls(1) for more
343 details.
344 --tls-psk /path/to/pskfile
346 Set the path to the pre-shared keys (PSK) file. If used, this
347 overrides certificate authentication. There is no built-in path.
348 See nbdkit-tls(1) for more details.
349 --tls-verify-peer
351 Enables TLS client certificate verification. The default is not to
352 check the client's certificate.
353 -U SOCKET
355 --unix SOCKET
356 -U -
357 --unix -
358 Accept connections on the Unix domain socket "SOCKET" (which is a
359 path).
360 nbdkit creates this socket, but it will probably have incorrect
362 permissions (too permissive). If it is a problem that some
363 unauthorized user could connect to this socket between the time
364 that nbdkit starts up and the authorized user connects, then put
365 the socket into a directory that has restrictive permissions.
366 nbdkit does not delete the socket file when it exits. The caller
368 should delete the socket file after use (else if you try to start
369 nbdkit up again you will get an "Address already in use" error).
370 If the socket name is - then nbdkit generates a randomly named
372 private socket. This is useful with "CAPTIVE NBDKIT" in
373 nbdkit-captive(1).
374 -u USER
376 --user USER
377 Change user to "USER" after starting up. A user name or numeric
378 user ID can be used.
379 The server needs sufficient permissions to be able to do this.
381 Normally this would mean starting the server up as root.
382 See also -g.
384 -v
386 --verbose
387 Enable verbose messages.
388 It's a good idea to use -f as well so the process does not fork
390 into the background (but not required).
391 -V
393 --version
394 Print the version number of nbdkit and exit.
395 The --dump-config option provides separate major and minor numbers
397 and may be easier to parse from shell scripts.
398 --vsock
400 (nbdkit ≥ 1.16)
401 Use the AF_VSOCK protocol (instead of TCP/IP). You must use this
403 in conjunction with -p/--port. See "AF_VSOCK" in
404 nbdkit-service(1).
405
407 You can give the full path to the plugin, like this:
408 nbdkit $libdir/nbdkit/plugins/nbdkit-file-plugin.so [...]
410 but it is usually more convenient to use this equivalent syntax:
412 nbdkit file [...]
414 $libdir is set at compile time. To print it out, do:
416 nbdkit --dump-config
418
420 After specifying the plugin name you can (optionally, it depends on the
421 plugin) give plugin configuration on the command line in the form of
422 "key=value". For example:
423 nbdkit file file=disk.img
425 To list all the options supported by a plugin, do:
427 nbdkit --help file
429 To dump information about a plugin, do:
431 nbdkit file --dump-plugin
433 Magic parameters
435 Some plugins declare a special "magic config key". This is a key which
436 is assumed if no "key=" part is present. For example:
437 nbdkit file disk.img
439 is assumed to be "file=disk.img" because the file plugin declares
441 "file" as its magic config key. There can be ambiguity in the parsing
442 of magic config keys if the value might look like a "key=value". If
443 there could be ambiguity then modify the value, eg. by prefixing it
444 with "./"
445 There is also a special exception for plugins which do not declare a
447 magic config key, but where the first plugin argument does not contain
448 an '=' character: it is assumed to be "script=value". This is used by
449 scripting language plugins:
450 nbdkit perl foo.pl [args...]
452 has the same meaning as:
454 nbdkit perl script=foo.pl [args...]
456 Shebang scripts
458 You can use "#!" to run nbdkit plugins written in most scripting
459 languages. The file should be executable. For example:
460 #!/usr/sbin/nbdkit perl
462 sub open {
463 # etc
464 }
465 (see nbdkit-perl-plugin(3) for a full example).
467
469 As well as enabling or disabling debugging in the server using
470 --verbose you can control extra debugging in the server using the "-D
471 nbdkit.*" flags listed in this section. Note these flags are an
472 internal implementation detail of the server and may be changed or
473 removed at any time in the future.
474 -D nbdkit.backend.controlpath=0
476 -D nbdkit.backend.controlpath=1
477 -D nbdkit.backend.datapath=0
478 -D nbdkit.backend.datapath=1
479 These flags control the verbosity of nbdkit backend debugging
480 messages (the ones which show every request processed by the
481 server). The default for both settings is 1 (normal debugging) but
482 you can set them to 0 to suppress these messages.
483 "-D nbdkit.backend.datapath=0" is the more useful setting which
485 lets you suppress messages about pread, pwrite, zero, trim, etc.
486 commands. When transferring large amounts of data these messages
487 are numerous and not usually very interesting.
488 "-D nbdkit.backend.controlpath=0" suppresses the non-datapath
490 commands (config, open, close, can_write, etc.)
491 -D nbdkit.tls.log=N
493 Enable TLS logging. "N" can be in the range 0 (no logging) to 99.
494 See gnutls_global_set_log_level(3).
495 -D nbdkit.tls.session=1
497 Print additional information about the TLS session, such as the
498 type of authentication and encryption, and client certificate
499 information.
500
502 nbdkit responds to the following signals:
503 "SIGINT"
505 "SIGQUIT"
506 "SIGTERM"
507 The server exits cleanly.
508 "SIGPIPE"
510 This signal is ignored.
511
513 "LISTEN_FDS"
514 "LISTEN_PID"
515 If present in the environment when nbdkit starts up, these trigger
516 "SOCKET ACTIVATION" in nbdkit-service(1).
517
519 Other topics
520 nbdkit-captive(1) — Run nbdkit under another process and have it
521 reliably cleaned up.
522 nbdkit-client(1) — How to mount NBD filesystems on a client machine.
524 nbdkit-loop(1) — Use nbdkit with the Linux kernel client to create loop
526 devices and loop mounts.
527 nbdkit-probing(1) — How to probe for nbdkit configuration and plugins.
529 nbdkit-protocol(1) — Which parts of the NBD protocol nbdkit supports.
531 nbdkit-security(1) — Lists past security issues in nbdkit.
533 nbdkit-service(1) — Running nbdkit as a service, and systemd socket
535 activation.
536 nbdkit-tls(1) — Authentication and encryption of NBD connections
538 (sometimes incorrectly called "SSL").
539 Plugins
541 nbdkit-cdi-plugin(1), nbdkit-curl-plugin(1), nbdkit-data-plugin(1),
542 nbdkit-eval-plugin(1), nbdkit-example1-plugin(1),
543 nbdkit-example2-plugin(1), nbdkit-example3-plugin(1),
544 nbdkit-example4-plugin(1), nbdkit-file-plugin(1),
545 nbdkit-floppy-plugin(1), nbdkit-full-plugin(1),
546 nbdkit-guestfs-plugin(1), nbdkit-info-plugin(1), nbdkit-iso-plugin(1),
547 nbdkit-libvirt-plugin(1), nbdkit-linuxdisk-plugin(1),
548 nbdkit-memory-plugin(1), nbdkit-nbd-plugin(1), nbdkit-null-plugin(1),
549 nbdkit-ondemand-plugin(1), nbdkit-partitioning-plugin(1),
550 nbdkit-pattern-plugin(1), nbdkit-random-plugin(1), nbdkit-S3-plugin(1),
551 nbdkit-sparse-random-plugin(1), nbdkit-split-plugin(1),
552 nbdkit-ssh-plugin(1), nbdkit-streaming-plugin(1),
553 nbdkit-tmpdisk-plugin(1), nbdkit-torrent-plugin(1),
554 nbdkit-vddk-plugin(1), nbdkit-zero-plugin(1) ; nbdkit-cc-plugin(3),
555 nbdkit-golang-plugin(3), nbdkit-lua-plugin(3), nbdkit-ocaml-plugin(3),
556 nbdkit-perl-plugin(3), nbdkit-python-plugin(3), nbdkit-ruby-plugin(3),
557 nbdkit-rust-plugin(3), nbdkit-sh-plugin(3), nbdkit-tcl-plugin(3) .
558 Filters
560 nbdkit-blocksize-filter(1), nbdkit-cache-filter(1),
561 nbdkit-cacheextents-filter(1), nbdkit-checkwrite-filter(1),
562 nbdkit-cow-filter(1), nbdkit-ddrescue-filter(1),
563 nbdkit-delay-filter(1), nbdkit-error-filter(1),
564 nbdkit-exitlast-filter(1), nbdkit-exitwhen-filter(1),
565 nbdkit-exportname-filter(1), nbdkit-ext2-filter(1),
566 nbdkit-extentlist-filter(1), nbdkit-fua-filter(1),
567 nbdkit-gzip-filter(1), nbdkit-ip-filter(1), nbdkit-limit-filter(1),
568 nbdkit-log-filter(1), nbdkit-multi-conn-filter(1),
569 nbdkit-nocache-filter(1), nbdkit-noextents-filter(1),
570 nbdkit-nofilter-filter(1), nbdkit-noparallel-filter(1),
571 nbdkit-nozero-filter(1), nbdkit-offset-filter(1),
572 nbdkit-partition-filter(1), nbdkit-pause-filter(1),
573 nbdkit-rate-filter(1), nbdkit-readahead-filter(1),
574 nbdkit-retry-filter(1), nbdkit-stats-filter(1), nbdkit-swab-filter(1),
575 nbdkit-tar-filter(1), nbdkit-tls-fallback-filter(1),
576 nbdkit-truncate-filter(1), nbdkit-xz-filter(1) .
577 For developers
579 nbdkit-plugin(3), nbdkit-filter(3).
580 Writing plugins in other programming languages
582 nbdkit-cc-plugin(3), nbdkit-golang-plugin(3), nbdkit-lua-plugin(3),
583 nbdkit-ocaml-plugin(3), nbdkit-perl-plugin(3), nbdkit-python-plugin(3),
584 nbdkit-ruby-plugin(3), nbdkit-rust-plugin(3), nbdkit-sh-plugin(3),
585 nbdkit-tcl-plugin(3) .
586 Release notes for previous releases of nbdkit
588 nbdkit-release-notes-1.4(1), nbdkit-release-notes-1.6(1),
589 nbdkit-release-notes-1.8(1), nbdkit-release-notes-1.10(1),
590 nbdkit-release-notes-1.12(1), nbdkit-release-notes-1.14(1),
591 nbdkit-release-notes-1.16(1), nbdkit-release-notes-1.18(1),
592 nbdkit-release-notes-1.20(1), nbdkit-release-notes-1.22(1),
593 nbdkit-release-notes-1.24(1).
594 NBD clients
596 guestfish(1), libnbd(3), nbd-client(1), nbdcopy(1), nbdfuse(1),
597 nbdinfo(1), nbdsh(1), qemu(1).
598 nbdkit links
600 http://gitlab.com/nbdkit/nbdkit — Source code.
601 Other NBD servers
603 qemu-nbd(1), nbd-server(1), https://bitbucket.org/hirofuchi/xnbd.
604 Documentation for the NBD protocol
606 https://github.com/NetworkBlockDevice/nbd/blob/master/doc/proto.md,
607 https://nbd.sourceforge.io/.
608 Similar protocols
610 https://en.wikipedia.org/wiki/iSCSI,
611 https://en.wikipedia.org/wiki/ATA_over_Ethernet,
612 https://en.wikipedia.org/wiki/Fibre_Channel_over_Ethernet.
613 Other manual pages of interest
615 gnutls_priority_init(3), qemu-img(1), psktool(1), systemd.socket(5).
616
618 Eric Blake
619 Richard W.M. Jones
621 Yann E. MORIN
623 Nir Soffer
625 Pino Toscano
627
629 Copyright (C) 2013-2020 Red Hat Inc.
630
632 Redistribution and use in source and binary forms, with or without
633 modification, are permitted provided that the following conditions are
634 met:
635 • Redistributions of source code must retain the above copyright
637 notice, this list of conditions and the following disclaimer.
638 • Redistributions in binary form must reproduce the above copyright
640 notice, this list of conditions and the following disclaimer in the
641 documentation and/or other materials provided with the
642 distribution.
643 • Neither the name of Red Hat nor the names of its contributors may
645 be used to endorse or promote products derived from this software
646 without specific prior written permission.
647 THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND ANY
649 EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
650 IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
651 PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR CONTRIBUTORS BE
652 LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
653 CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
654 SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
655 BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
656 WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
657 OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
658 ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
659nbdkit-1.25.8 2021-05-25 nbdkit(1)