1nbdkit-sh-plugin(3) NBDKIT nbdkit-sh-plugin(3)
2
6 nbdkit-sh-plugin - nbdkit shell, script or executable plugin
7
9 nbdkit sh /path/to/script [arguments...]
10 nbdkit sh - <<'EOF'
12 ... shell script ...
13 EOF
14
16 "nbdkit-sh-plugin" allows you to write plugins for nbdkit(1) using
17 arbitrary scripting languages, including shells like bash(1), dash(1),
18 csh(1), zsh(1) etc., other scripting environments, or any executable.
19 Note if you want to use an established scripting language like Perl or
21 Python, then nbdkit has specific plugins to handle those languages and
22 those will be more efficient (see nbdkit(1) for a complete list).
23 To use shell script fragments from the nbdkit command line (rather than
25 a separate script) see nbdkit-eval-plugin(1).
26 If you have been given an nbdkit sh plugin
28 Assuming you have a shell script which is an nbdkit plugin, you run it
29 like this:
30 nbdkit sh /path/to/script
32 You may have to add further "key=value" arguments to the command line.
34 The script must be executable ("chmod +x").
35 Inline shell scripts
37 It is also possible to write a shell script plugin "inline" using "-"
38 as the name of the script, like this:
39 nbdkit sh - <<'EOF'
41 case "$1" in
42 get_size) echo 1M ;;
43 pread) dd if=/dev/zero count=$3 iflag=count_bytes ;;
44 *) exit 2 ;;
45 esac
46 EOF
47 By default the inline script runs under /bin/sh. You can add a shebang
49 ("#!") to use other scripting languages. Of course, reading an inline
50 script from stdin is incompatible with the -s (--single) mode of nbdkit
51 that connects a client on stdin.
52
54 For an example plugin written in Bash, see:
55 https://gitlab.com/nbdkit/nbdkit/blob/master/plugins/sh/example.sh
56 Broadly speaking, nbdkit shell plugins work like C ones, so you should
58 read nbdkit-plugin(3) first.
59 Programming model
61 This plugin has a simple programming model: For every plugin method
62 that needs to be called, the external script is invoked with parameters
63 describing the method and its arguments. The first parameter is always
64 the method name. For example:
65 /path/to/script config file disk.img
67 │ │ │
68 │ │ └─ value ($3)
69 │ └── key ($2)
70 method ($1)
71 /path/to/script pread <handle> <count> <offset>
73 │ │ │ │
74 │ │ │ └─ offset in bytes ($4)
75 │ │ └── request size in bytes ($3)
76 method ($1) └── handle ($2) ─ see "Handles" below
77 Scripts should ignore extra parameters that they don't understand since
79 we may add new parameters in future.
80 Exit codes
82 The script should exit with specific exit codes:
83 0 The method was executed successfully.
85 1 and 16-255
87 There was an error. The script may print on stderr an errno name,
88 optionally followed by whitespace and a message, for example:
89 echo 'ENOSPC Out of space' >&2
91 exit 1
92 or if you don't need the log message:
94 echo ENOSPC >&2
96 exit 1
97 If the script doesn't print anything or the output cannot be parsed
99 then nbdkit assumes error "EIO". Note that output to stderr is
100 ignored if the command succeeds, so it is acceptable to output a
101 potential error message prefix prior to attempting a command which
102 will add further details if a failure occurs.
103 2 The requested method is not supported by the script.
105 3 For methods which return booleans, this code indicates false.
107 4 and 5
109 Triggers a call to the C function "nbdkit_shutdown", which requests
110 an asynchronous exit of the nbdkit server (disconnecting all
111 clients). The client will usually get a response before shutdown
112 is complete (although this is racy); so once the shutdown is
113 requested, code 4 then behaves like code 0 (stderr is ignored, and
114 the server tries to return success), and code 5 behaves like code 1
115 (the server tries to return an error to the client parsed from
116 stderr, although a missing error defaults to "ESHUTDOWN" instead of
117 "EIO").
118 6 Triggers a call to the C function "nbdkit_disconnect" with "force"
120 set to true, which requests an abrupt disconnect of the current
121 client. The contents of stderr are irrelevant with this status,
122 since the client will not get a response.
123 7 and 8
125 Triggers a call to the C function "nbdkit_disconnect" with "force"
126 set to false, which requests a soft disconnect of the current
127 client (future client requests are rejected with "ESHUTDOWN"
128 without calling into the plugin, but current requests may
129 complete). Since the client will likely get the response to this
130 command, code 7 then behaves like code 0 (stderr is ignored, and
131 the server tries to return success), and code 8 behaves like code 1
132 (the server tries to return an error to the client parsed from
133 stderr, although a missing error defaults to "ESHUTDOWN" instead of
134 "EIO").
135 9-15
137 These exit codes are reserved for future use. Note that versions
138 of nbdkit < 1.34 documented that codes 8 through 15 behaved like
139 code 1; although it is unlikely that many scripts relied on this
140 similarity in practice.
141 In nbdkit > 1.34, it is possible to probe whether additional exit codes
143 have been assigned meaning, by looking for the line max_known_status=
144 in the output of nbdkit --dump-plugin sh. If this line is not present,
145 exit codes 4 and above behave like status 1.
146 Temporary directory
148 A fresh script is invoked for each method call (ie. scripts are
149 stateless), so if the script needs to store state it has to store it
150 somewhere in the filesystem in a format and location which is left up
151 to the author of the script.
152 However nbdkit helps by creating a randomly named, empty directory for
154 the script. This directory persists for the lifetime of nbdkit and is
155 deleted when nbdkit exits. The name of the directory is passed to each
156 script invocation in the $tmpdir environment variable.
157 Handles
159 Handles are arbitrary strings, but it is best to limit them to short
160 alphanumeric strings.
161 Per-connection state
163 The temporary directory described above can be used for state for the
165 lifetime of the nbdkit instance (across multiple connections). If you
166 want to store state per connection then one way to do it is to create a
167 randomly named subdirectory under the temporary directory:
168 case "$1" in
170 ...
171 open)
172 mktemp -d $tmpdir/handle-XXXXXX ;;
173 The handle will be the subdirectory name, returned to the script as $2
175 in all connected calls (eg. "pread", "get_size"). You can delete the
176 subdirectory explicitly in "close":
177 case "$1" in
179 ...
180 close)
181 rm -rf "$2" ;;
182 or rely on nbdkit deleting the whole temporary directory including all
184 per-handle subdirectories when it exits.
185 Performance
187 This plugin has to fork on every request, so performance will never be
188 great. For best performance, consider using the nbdkit-plugin(3) API
189 directly. Having said that, if you have a sh plugin and want to
190 improve performance then the following tips may help:
191 Relax the thread model.
193 The default "thread_model" is "serialize_all_requests" meaning that
194 two instances of the script can never be running at the same time.
195 This is safe but slow. If your script is safe to be called in
196 parallel, set this to "parallel".
197 Implement the "zero" method.
199 If the "zero" method is not implemented then nbdkit will fall back
200 to using "pwrite" which is considerably slower because nbdkit has
201 to send blocks of zeroes to the script.
202 You don't have to write shell scripts.
204 This plugin can run any external binary, not only shell scripts.
205 You should get more performance by rewriting the shell script as a
206 program in a compiled language.
207 Methods
209 This just documents the arguments to the script corresponding to each
210 plugin method, and any way that they differ from the C callbacks. In
211 all other respects they work the same way as the C callbacks, so you
212 should go and read nbdkit-plugin(3).
213 "load"
215 /path/to/script load
216 "unload"
218 /path/to/script unload
219 This is called just before nbdkit exits. Errors from this method
221 are ignored.
222 "dump_plugin"
224 /path/to/script dump_plugin
225 "config"
227 /path/to/script config <key> <value>
228 "config_complete"
230 /path/to/script config_complete
231 "magic_config_key"
233 /path/to/script magic_config_key
234 If a magic config key is needed, this should echo it to stdout.
236 See "Magic parameters" in nbdkit(1).
237 "thread_model"
239 /path/to/script thread_model
240 On success this should print the desired thread model of the
242 script, one of "serialize_connections", "serialize_all_requests",
243 "serialize_requests", or "parallel".
244 This method is not required; if omitted, then the plugin will be
246 executed under the safe "serialize_all_requests" model. However,
247 this means that this method must be provided if you want to use the
248 "parallel" or "serialize_requests" model. Even then your request
249 may be restricted for other reasons; look for "thread_model" in the
250 output of "nbdkit --dump-plugin sh script" to see what actually
251 gets selected.
252 If an error occurs, the script should output an error message and
254 exit with status 1; unrecognized output is ignored.
255 "get_ready"
257 /path/to/script get_ready
258 "after_fork"
260 /path/to/script after_fork
261 "preconnect"
263 /path/to/script preconnect <readonly>
264 "list_exports"
266 /path/to/script list_exports <readonly> <tls>
267 The "readonly" and "tls" parameters will be "true" or "false".
269 The first line of output informs nbdkit how to parse the rest of
271 the output, the remaining lines then supply the inputs of the C
272 "nbdkit_add_export" function (see nbdkit-plugin(3)), as follows:
273 NAMES
275 The remaining output provides one export name per line, and no
276 export will be given a description. For convenience, this form
277 is also assumed if the first output line does not match one of
278 the recognized parse modes.
279 INTERLEAVED
281 The remaining output provides pairs of lines, the first line
282 being an export name, and the second the corresponding
283 description.
284 NAMES+DESCRIPTIONS
286 The number of remaining lines is counted, with the first half
287 being used as export names, and the second half providing
288 descriptions to pair with names from the first half.
289 An example of using this form to list files in the current
291 directory, followed by their ls(1) long description, would be:
292 echo NAMES+DESCRIPTIONS
294 ls
295 ls -l
296 Note that other output modes might be introduced in the future; in
298 particular, none of the existing modes allow a literal newline in
299 an export name or description, although this could be possible
300 under a new mode supporting escape sequences.
301 This method is not required; if it is absent, the list of exports
303 advertised by nbdkit will be the single name result of
304 "default_export" and no description.
305 "default_export"
307 /path/to/script default_export <readonly> <tls>
308 The "readonly" and "tls" parameters will be "true" or "false".
310 On success this should print a name on stdout to use in place of
312 the default export "", then exit with code 0. For convenience, the
313 output can be any of the list forms recognized by "list_exports",
314 in which case the first listed export name is used, and where an
315 empty list uses "". Given the current set of recognized export
316 lists, it is not possible for the resulting name to include a
317 newline.
318 This method is not required; if it is absent, the default export
320 name will be the empty string, "".
321 "open"
323 /path/to/script open <readonly> <exportname> <tls>
324 The "readonly" parameter will be "true" or "false". The
326 "exportname" parameter, if present, is the export name passed to
327 the server from the client. The "tls" parameter, if present, will
328 be "true" or "false" depending on whether the client is using TLS.
329 On success this should print the handle (any string) on stdout and
331 exit with code 0. If the handle ends with a newline character then
332 the newline is removed.
333 Unlike C plugins, this method is not required. If omitted then the
335 handle will be "" (empty string).
336 "close"
338 /path/to/script close <handle>
339 "export_description"
341 /path/to/script export_description <handle>
342 The script should print a human-readable description of the disk
344 image on stdout. If the description ends with a newline character
345 then the newline is removed.
346 This method is not required; if it is absent, no export description
348 will be provided to the client.
349 "get_size"
351 /path/to/script get_size <handle>
352 The script should print the size of the disk image on stdout. You
354 can print the size in bytes, or use any format understood by
355 "nbdkit_parse_size" such as "1M" (see "PARSING SIZE PARAMETERS" in
356 nbdkit-plugin(3)).
357 This method is required.
359 "block_size"
361 /path/to/script block_size <handle>
362 This script should print three numbers on stdout, separated by
364 whitespace. These are (in order) the minimum block size, the
365 preferred block size, and the maximum block size. You can print
366 the sizes in bytes or use any format understood by
367 "nbdkit_parse_size" such as "1M" (see "PARSING SIZE PARAMETERS" in
368 nbdkit-plugin(3)).
369 "can_write"
371 "can_flush"
372 "can_trim"
373 "can_zero"
374 "can_extents"
375 Unlike in other languages, you must provide the "can_*" methods
376 otherwise they are assumed to all return false and your "pwrite",
377 "flush", "trim", "zero" and "extents" methods will never be called.
378 The reason for this is obscure: In other languages we can detect if
379 (eg) a "pwrite" method is defined and synthesize an appropriate
380 response if no actual "can_write" method is defined. However
381 detecting if methods are present without running them is not
382 possible with this plugin.
383 /path/to/script can_write <handle>
385 /path/to/script can_flush <handle>
386 /path/to/script can_trim <handle>
387 /path/to/script can_zero <handle>
388 /path/to/script can_extents <handle>
389 The script should exit with code 0 for true or code 3 for false.
391 "is_rotational"
393 "can_fast_zero"
394 /path/to/script is_rotational <handle>
395 /path/to/script can_fast_zero <handle>
396 The script should exit with code 0 for true or code 3 for false.
398 "can_fua"
400 "can_cache"
401 /path/to/script can_fua <handle>
402 /path/to/script can_cache <handle>
403 These control Forced Unit Access (FUA) and caching behaviour of the
405 core server.
406 Unlike the other "can_*" callbacks, these two are not a boolean.
408 They must print either "none", "emulate" or "native" to stdout.
409 The meaning of these is described in nbdkit-plugin(3).
410 Furthermore, you must provide a "can_cache" method if you desire
411 the "cache" callback to be utilized, similar to the reasoning
412 behind requiring "can_write" to utilize "pwrite".
413 "can_multi_conn"
415 /path/to/script can_multi_conn <handle>
416 The script should exit with code 0 for true or code 3 for false.
418 "pread"
420 /path/to/script pread <handle> <count> <offset>
421 The script should print the requested binary data on stdout.
423 Exactly "count" bytes must be printed.
424 This method is required.
426 "pwrite"
428 /path/to/script pwrite <handle> <count> <offset> <flags>
429 The script should read the binary data to be written from stdin.
431 The "flags" parameter can be an empty string or "fua". In the
433 future, a comma-separated list of flags may be present.
434 Unlike in other languages, if you provide a "pwrite" method you
436 must also provide a "can_write" method which exits with code 0
437 (true).
438 "flush"
440 /path/to/script flush <handle>
441 Unlike in other languages, if you provide a "flush" method you must
443 also provide a "can_flush" method which exits with code 0 (true).
444 "trim"
446 /path/to/script trim <handle> <count> <offset> <flags>
447 The "flags" parameter can be an empty string or "fua". In the
449 future, a comma-separated list of flags may be present.
450 Unlike in other languages, if you provide a "trim" method you must
452 also provide a "can_trim" method which exits with code 0 (true).
453 "zero"
455 /path/to/script zero <handle> <count> <offset> <flags>
456 The "flags" parameter can be an empty string or a comma-separated
458 list of the flags: "fua", "may_trim", and "fast" (eg. "", "fua",
459 "fua,may_trim,fast" are some of the 8 possible values).
460 Unlike in other languages, if you provide a "zero" method you must
462 also provide a "can_zero" method which exits with code 0 (true).
463 To trigger a fallback to "pwrite" on a normal zero request, or to
465 respond quickly to the "fast" flag that a specific zero request is
466 no faster than a corresponding write, the script must output
467 "ENOTSUP" or "EOPNOTSUPP" to stderr (possibly followed by a
468 description of the problem) before exiting with code 1 (failure).
469 "extents"
471 /path/to/script extents <handle> <count> <offset> <flags>
472 The "flags" parameter can be an empty string or "req_one".
474 This must print, one per line on stdout, a list of one or more
476 extents in the format:
477 offset length type
479 which correspond to the inputs of the C "nbdkit_add_extent"
481 function (see nbdkit-plugin(3)). The "offset" and "length" fields
482 may use any format understood by "nbdkit_parse_size". The optional
483 "type" field may be an integer, missing (same as 0), or a comma-
484 separated list of the words "hole" and "zero". An example of a
485 valid set of extents covering a "10M" disk where the first megabyte
486 only is allocated data:
487 0 1M
489 1M 9M hole,zero
490 Unlike in other languages, if you provide an "extents" method you
492 must also provide a "can_extents" method which exits with code 0
493 (true).
494 "cache"
496 /path/to/script cache <handle> <count> <offset>
497 Unlike in other languages, if you provide a "cache" method you must
499 also provide a "can_cache" method which prints "native" and exits
500 with code 0 (true).
501 Missing callbacks
503 Missing: "name", "version", "longname", "description", "config_help"
504 These are not yet supported.
505
507 $plugindir/nbdkit-sh-plugin.so
508 The plugin.
509 Use "nbdkit --dump-config" to find the location of $plugindir.
511
513 "nbdkit-sh-plugin" first appeared in nbdkit 1.8.
514
516 nbdkit(1), nbdkit-plugin(3), nbdkit-eval-plugin(1),
517 nbdkit-cc-plugin(1).
518
520 Richard W.M. Jones
521
523 Copyright Red Hat
524
526 Redistribution and use in source and binary forms, with or without
527 modification, are permitted provided that the following conditions are
528 met:
529 • Redistributions of source code must retain the above copyright
531 notice, this list of conditions and the following disclaimer.
532 • Redistributions in binary form must reproduce the above copyright
534 notice, this list of conditions and the following disclaimer in the
535 documentation and/or other materials provided with the
536 distribution.
537 • Neither the name of Red Hat nor the names of its contributors may
539 be used to endorse or promote products derived from this software
540 without specific prior written permission.
541 THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND ANY
543 EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
544 IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
545 PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR CONTRIBUTORS BE
546 LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
547 CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
548 SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
549 BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
550 WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
551 OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
552 ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
553nbdkit-1.34.4 2023-09-26 nbdkit-sh-plugin(3)