1nbdkit-sh-plugin(3) NBDKIT nbdkit-sh-plugin(3)
2
6 nbdkit-sh-plugin - nbdkit shell, script or executable plugin
7
9 nbdkit sh /path/to/script [arguments...]
10 nbdkit sh - <<'EOF'
12 ... shell script ...
13 EOF
14
16 "nbdkit-sh-plugin" allows you to write plugins for nbdkit(1) using
17 arbitrary scripting languages, including shells like bash(1), dash(1),
18 csh(1), zsh(1) etc., other scripting environments, or any executable.
19 Note if you want to use an established scripting language like Perl or
21 Python, then nbdkit has specific plugins to handle those languages and
22 those will be more efficient (see nbdkit(1) for a complete list).
23 To use shell script fragments from the nbdkit command line (rather than
25 a separate script) see nbdkit-eval-plugin(1).
26 If you have been given an nbdkit sh plugin
28 Assuming you have a shell script which is an nbdkit plugin, you run it
29 like this:
30 nbdkit sh /path/to/script
32 You may have to add further "key=value" arguments to the command line.
34 The script must be executable ("chmod +x").
35 Inline shell scripts
37 It is also possible to write a shell script plugin "inline" using "-"
38 as the name of the script, like this:
39 nbdkit sh - <<'EOF'
41 case "$1" in
42 get_size) echo 1M ;;
43 pread) dd if=/dev/zero count=$3 iflag=count_bytes ;;
44 *) exit 2 ;;
45 esac
46 EOF
47 By default the inline script runs under /bin/sh. You can add a shebang
49 ("#!") to use other scripting languages. Of course, reading an inline
50 script from stdin is incompatible with the -s (--single) mode of nbdkit
51 that connects a client on stdin.
52
54 For an example plugin written in Bash, see:
55 https://gitlab.com/nbdkit/nbdkit/blob/master/plugins/sh/example.sh
56 Broadly speaking, nbdkit shell plugins work like C ones, so you should
58 read nbdkit-plugin(3) first.
59 Programming model
61 This plugin has a simple programming model: For every plugin method
62 that needs to be called, the external script is invoked with parameters
63 describing the method and its arguments. The first parameter is always
64 the method name. For example:
65 /path/to/script config file disk.img
67 │ │ │
68 │ │ └─ value ($3)
69 │ └── key ($2)
70 method ($1)
71 /path/to/script pread <handle> <count> <offset>
73 │ │ │ │
74 │ │ │ └─ offset in bytes ($4)
75 │ │ └── request size in bytes ($3)
76 method ($1) └── handle ($2) ─ see "Handles" below
77 Scripts should ignore extra parameters that they don't understand since
79 we may add new parameters in future.
80 Exit codes
82 The script should exit with specific exit codes:
83 0 The method was executed successfully.
85 1 and 8-127
87 There was an error. The script may print on stderr an errno name,
88 optionally followed by whitespace and a message, for example:
89 echo 'ENOSPC Out of space' >&2
91 exit 1
92 or if you don't need the log message:
94 echo ENOSPC >&2
96 exit 1
97 If the script doesn't print anything or the output cannot be parsed
99 then nbdkit assumes error "EIO". Note that output to stderr is
100 ignored if the command succeeds, so it is acceptable to output a
101 potential error message prefix prior to attempting a command which
102 will add further details if a failure occurs.
103 2 The requested method is not supported by the script.
105 3 For methods which return booleans, this code indicates false.
107 4, 5, 6, 7
109 These exit codes are reserved for future use.
110 In nbdkit > 1.34, it is possible to probe whether additional exit codes
112 have been assigned meaning, by looking for the line max_known_status=
113 in the output of nbdkit --dump-plugin sh. If this line is not present,
114 exit codes 4 and above behave like status 1.
115 Temporary directory
117 A fresh script is invoked for each method call (ie. scripts are
118 stateless), so if the script needs to store state it has to store it
119 somewhere in the filesystem in a format and location which is left up
120 to the author of the script.
121 However nbdkit helps by creating a randomly named, empty directory for
123 the script. This directory persists for the lifetime of nbdkit and is
124 deleted when nbdkit exits. The name of the directory is passed to each
125 script invocation in the $tmpdir environment variable.
126 Handles
128 Handles are arbitrary strings, but it is best to limit them to short
129 alphanumeric strings.
130 Per-connection state
132 The temporary directory described above can be used for state for the
134 lifetime of the nbdkit instance (across multiple connections). If you
135 want to store state per connection then one way to do it is to create a
136 randomly named subdirectory under the temporary directory:
137 case "$1" in
139 ...
140 open)
141 mktemp -d $tmpdir/handle-XXXXXX ;;
142 The handle will be the subdirectory name, returned to the script as $2
144 in all connected calls (eg. "pread", "get_size"). You can delete the
145 subdirectory explicitly in "close":
146 case "$1" in
148 ...
149 close)
150 rm -rf "$2" ;;
151 or rely on nbdkit deleting the whole temporary directory including all
153 per-handle subdirectories when it exits.
154 Performance
156 This plugin has to fork on every request, so performance will never be
157 great. For best performance, consider using the nbdkit-plugin(3) API
158 directly. Having said that, if you have a sh plugin and want to
159 improve performance then the following tips may help:
160 Relax the thread model.
162 The default "thread_model" is "serialize_all_requests" meaning that
163 two instances of the script can never be running at the same time.
164 This is safe but slow. If your script is safe to be called in
165 parallel, set this to "parallel".
166 Implement the "zero" method.
168 If the "zero" method is not implemented then nbdkit will fall back
169 to using "pwrite" which is considerably slower because nbdkit has
170 to send blocks of zeroes to the script.
171 You don't have to write shell scripts.
173 This plugin can run any external binary, not only shell scripts.
174 You should get more performance by rewriting the shell script as a
175 program in a compiled language.
176 Methods
178 This just documents the arguments to the script corresponding to each
179 plugin method, and any way that they differ from the C callbacks. In
180 all other respects they work the same way as the C callbacks, so you
181 should go and read nbdkit-plugin(3).
182 "load"
184 /path/to/script load
185 "unload"
187 /path/to/script unload
188 This is called just before nbdkit exits. Errors from this method
190 are ignored.
191 "dump_plugin"
193 /path/to/script dump_plugin
194 "config"
196 /path/to/script config <key> <value>
197 "config_complete"
199 /path/to/script config_complete
200 "magic_config_key"
202 /path/to/script magic_config_key
203 If a magic config key is needed, this should echo it to stdout.
205 See "Magic parameters" in nbdkit(1).
206 "thread_model"
208 /path/to/script thread_model
209 On success this should print the desired thread model of the
211 script, one of "serialize_connections", "serialize_all_requests",
212 "serialize_requests", or "parallel".
213 This method is not required; if omitted, then the plugin will be
215 executed under the safe "serialize_all_requests" model. However,
216 this means that this method must be provided if you want to use the
217 "parallel" or "serialize_requests" model. Even then your request
218 may be restricted for other reasons; look for "thread_model" in the
219 output of "nbdkit --dump-plugin sh script" to see what actually
220 gets selected.
221 If an error occurs, the script should output an error message and
223 exit with status 1; unrecognized output is ignored.
224 "get_ready"
226 /path/to/script get_ready
227 "after_fork"
229 /path/to/script after_fork
230 "preconnect"
232 /path/to/script preconnect <readonly>
233 "list_exports"
235 /path/to/script list_exports <readonly> <tls>
236 The "readonly" and "tls" parameters will be "true" or "false".
238 The first line of output informs nbdkit how to parse the rest of
240 the output, the remaining lines then supply the inputs of the C
241 "nbdkit_add_export" function (see nbdkit-plugin(3)), as follows:
242 NAMES
244 The remaining output provides one export name per line, and no
245 export will be given a description. For convenience, this form
246 is also assumed if the first output line does not match one of
247 the recognized parse modes.
248 INTERLEAVED
250 The remaining output provides pairs of lines, the first line
251 being an export name, and the second the corresponding
252 description.
253 NAMES+DESCRIPTIONS
255 The number of remaining lines is counted, with the first half
256 being used as export names, and the second half providing
257 descriptions to pair with names from the first half.
258 An example of using this form to list files in the current
260 directory, followed by their ls(1) long description, would be:
261 echo NAMES+DESCRIPTIONS
263 ls
264 ls -l
265 Note that other output modes might be introduced in the future; in
267 particular, none of the existing modes allow a literal newline in
268 an export name or description, although this could be possible
269 under a new mode supporting escape sequences.
270 This method is not required; if it is absent, the list of exports
272 advertised by nbdkit will be the single name result of
273 "default_export" and no description.
274 "default_export"
276 /path/to/script default_export <readonly> <tls>
277 The "readonly" and "tls" parameters will be "true" or "false".
279 On success this should print a name on stdout to use in place of
281 the default export "", then exit with code 0. For convenience, the
282 output can be any of the list forms recognized by "list_exports",
283 in which case the first listed export name is used, and where an
284 empty list uses "". Given the current set of recognized export
285 lists, it is not possible for the resulting name to include a
286 newline.
287 This method is not required; if it is absent, the default export
289 name will be the empty string, "".
290 "open"
292 /path/to/script open <readonly> <exportname> <tls>
293 The "readonly" parameter will be "true" or "false". The
295 "exportname" parameter, if present, is the export name passed to
296 the server from the client. The "tls" parameter, if present, will
297 be "true" or "false" depending on whether the client is using TLS.
298 On success this should print the handle (any string) on stdout and
300 exit with code 0. If the handle ends with a newline character then
301 the newline is removed.
302 Unlike C plugins, this method is not required. If omitted then the
304 handle will be "" (empty string).
305 "close"
307 /path/to/script close <handle>
308 "export_description"
310 /path/to/script export_description <handle>
311 The script should print a human-readable description of the disk
313 image on stdout. If the description ends with a newline character
314 then the newline is removed.
315 This method is not required; if it is absent, no export description
317 will be provided to the client.
318 "get_size"
320 /path/to/script get_size <handle>
321 The script should print the size of the disk image on stdout. You
323 can print the size in bytes, or use any format understood by
324 "nbdkit_parse_size" such as "1M" (see "PARSING SIZE PARAMETERS" in
325 nbdkit-plugin(3)).
326 This method is required.
328 "block_size"
330 /path/to/script block_size <handle>
331 This script should print three numbers on stdout, separated by
333 whitespace. These are (in order) the minimum block size, the
334 preferred block size, and the maximum block size. You can print
335 the sizes in bytes or use any format understood by
336 "nbdkit_parse_size" such as "1M" (see "PARSING SIZE PARAMETERS" in
337 nbdkit-plugin(3)).
338 "can_write"
340 "can_flush"
341 "can_trim"
342 "can_zero"
343 "can_extents"
344 Unlike in other languages, you must provide the "can_*" methods
345 otherwise they are assumed to all return false and your "pwrite",
346 "flush", "trim", "zero" and "extents" methods will never be called.
347 The reason for this is obscure: In other languages we can detect if
348 (eg) a "pwrite" method is defined and synthesize an appropriate
349 response if no actual "can_write" method is defined. However
350 detecting if methods are present without running them is not
351 possible with this plugin.
352 /path/to/script can_write <handle>
354 /path/to/script can_flush <handle>
355 /path/to/script can_trim <handle>
356 /path/to/script can_zero <handle>
357 /path/to/script can_extents <handle>
358 The script should exit with code 0 for true or code 3 for false.
360 "is_rotational"
362 "can_fast_zero"
363 /path/to/script is_rotational <handle>
364 /path/to/script can_fast_zero <handle>
365 The script should exit with code 0 for true or code 3 for false.
367 "can_fua"
369 "can_cache"
370 /path/to/script can_fua <handle>
371 /path/to/script can_cache <handle>
372 These control Forced Unit Access (FUA) and caching behaviour of the
374 core server.
375 Unlike the other "can_*" callbacks, these two are not a boolean.
377 They must print either "none", "emulate" or "native" to stdout.
378 The meaning of these is described in nbdkit-plugin(3).
379 Furthermore, you must provide a "can_cache" method if you desire
380 the "cache" callback to be utilized, similar to the reasoning
381 behind requiring "can_write" to utilize "pwrite".
382 "can_multi_conn"
384 /path/to/script can_multi_conn <handle>
385 The script should exit with code 0 for true or code 3 for false.
387 "pread"
389 /path/to/script pread <handle> <count> <offset>
390 The script should print the requested binary data on stdout.
392 Exactly "count" bytes must be printed.
393 This method is required.
395 "pwrite"
397 /path/to/script pwrite <handle> <count> <offset> <flags>
398 The script should read the binary data to be written from stdin.
400 The "flags" parameter can be an empty string or "fua". In the
402 future, a comma-separated list of flags may be present.
403 Unlike in other languages, if you provide a "pwrite" method you
405 must also provide a "can_write" method which exits with code 0
406 (true).
407 "flush"
409 /path/to/script flush <handle>
410 Unlike in other languages, if you provide a "flush" method you must
412 also provide a "can_flush" method which exits with code 0 (true).
413 "trim"
415 /path/to/script trim <handle> <count> <offset> <flags>
416 The "flags" parameter can be an empty string or "fua". In the
418 future, a comma-separated list of flags may be present.
419 Unlike in other languages, if you provide a "trim" method you must
421 also provide a "can_trim" method which exits with code 0 (true).
422 "zero"
424 /path/to/script zero <handle> <count> <offset> <flags>
425 The "flags" parameter can be an empty string or a comma-separated
427 list of the flags: "fua", "may_trim", and "fast" (eg. "", "fua",
428 "fua,may_trim,fast" are some of the 8 possible values).
429 Unlike in other languages, if you provide a "zero" method you must
431 also provide a "can_zero" method which exits with code 0 (true).
432 To trigger a fallback to <pwrite> on a normal zero request, or to
434 respond quickly to the "fast" flag that a specific zero request is
435 no faster than a corresponding write, the script must output
436 "ENOTSUP" or "EOPNOTSUPP" to stderr (possibly followed by a
437 description of the problem) before exiting with code 1 (failure).
438 "extents"
440 /path/to/script extents <handle> <count> <offset> <flags>
441 The "flags" parameter can be an empty string or "req_one".
443 This must print, one per line on stdout, a list of one or more
445 extents in the format:
446 offset length type
448 which correspond to the inputs of the C "nbdkit_add_extent"
450 function (see nbdkit-plugin(3)). The "offset" and "length" fields
451 may use any format understood by "nbdkit_parse_size". The optional
452 "type" field may be an integer, missing (same as 0), or a comma-
453 separated list of the words "hole" and "zero". An example of a
454 valid set of extents covering a "10M" disk where the first megabyte
455 only is allocated data:
456 0 1M
458 1M 9M hole,zero
459 Unlike in other languages, if you provide an "extents" method you
461 must also provide a "can_extents" method which exits with code 0
462 (true).
463 "cache"
465 /path/to/script cache <handle> <count> <offset>
466 Unlike in other languages, if you provide a "cache" method you must
468 also provide a "can_cache" method which prints "native" and exits
469 with code 0 (true).
470 Missing callbacks
472 Missing: "name", "version", "longname", "description", "config_help"
473 These are not yet supported.
474
476 $plugindir/nbdkit-sh-plugin.so
477 The plugin.
478 Use "nbdkit --dump-config" to find the location of $plugindir.
480
482 "nbdkit-sh-plugin" first appeared in nbdkit 1.8.
483
485 nbdkit(1), nbdkit-plugin(3), nbdkit-eval-plugin(1),
486 nbdkit-cc-plugin(1).
487
489 Richard W.M. Jones
490
492 Copyright (C) 2018-2020 Red Hat Inc.
493
495 Redistribution and use in source and binary forms, with or without
496 modification, are permitted provided that the following conditions are
497 met:
498 • Redistributions of source code must retain the above copyright
500 notice, this list of conditions and the following disclaimer.
501 • Redistributions in binary form must reproduce the above copyright
503 notice, this list of conditions and the following disclaimer in the
504 documentation and/or other materials provided with the
505 distribution.
506 • Neither the name of Red Hat nor the names of its contributors may
508 be used to endorse or promote products derived from this software
509 without specific prior written permission.
510 THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND ANY
512 EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
513 IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
514 PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR CONTRIBUTORS BE
515 LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
516 CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
517 SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
518 BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
519 WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
520 OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
521 ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
522nbdkit-1.32.5 2023-01-03 nbdkit-sh-plugin(3)