1nbdkit-sh-plugin(3) NBDKIT nbdkit-sh-plugin(3)
2
6 nbdkit-sh-plugin - nbdkit shell, script or executable plugin
7
9 nbdkit sh /path/to/script [arguments...]
10 nbdkit sh - <<'EOF'
12 ... shell script ...
13 EOF
14
16 "nbdkit-sh-plugin" allows you to write plugins for nbdkit(1) using
17 arbitrary scripting languages, including shells like bash(1), dash(1),
18 csh(1), zsh(1) etc., other scripting environments, or any executable.
19 Note if you want to use an established scripting language like Perl or
21 Python, then nbdkit has specific plugins to handle those languages and
22 those will be more efficient (see nbdkit(1) for a complete list).
23 To use shell script fragments from the nbdkit command line (rather than
25 a separate script) see nbdkit-eval-plugin(1).
26 If you have been given an nbdkit sh plugin
28 Assuming you have a shell script which is an nbdkit plugin, you run it
29 like this:
30 nbdkit sh /path/to/script
32 You may have to add further "key=value" arguments to the command line.
34 The script must be executable ("chmod +x").
35 Inline shell scripts
37 It is also possible to write a shell script plugin "inline" using "-"
38 as the name of the script, like this:
39 nbdkit sh - <<'EOF'
41 case "$1" in
42 get_size) echo 1M ;;
43 pread) dd if=/dev/zero count=$3 iflag=count_bytes ;;
44 *) exit 2 ;;
45 esac
46 EOF
47 By default the inline script runs under /bin/sh. You can add a shebang
49 ("#!") to use other scripting languages. Of course, reading an inline
50 script from stdin is incompatible with the -s (--single) mode of nbdkit
51 that connects a client on stdin.
52
54 For an example plugin written in Bash, see:
55 https://gitlab.com/nbdkit/nbdkit/blob/master/plugins/sh/example.sh
56 Broadly speaking, nbdkit shell plugins work like C ones, so you should
58 read nbdkit-plugin(3) first.
59 Programming model
61 This plugin has a simple programming model: For every plugin method
62 that needs to be called, the external script is invoked with parameters
63 describing the method and its arguments. The first parameter is always
64 the method name. For example:
65 /path/to/script config file disk.img
67 │ │ │
68 │ │ └─ value ($3)
69 │ └── key ($2)
70 method ($1)
71 /path/to/script pread <handle> <count> <offset>
73 │ │ │ │
74 │ │ │ └─ offset in bytes ($4)
75 │ │ └── request size in bytes ($3)
76 method ($1) └── handle ($2) ─ see "Handles" below
77 Scripts should ignore extra parameters that they don't understand since
79 we may add new parameters in future.
80 Exit codes
82 The script should exit with specific exit codes:
83 0 The method was executed successfully.
85 1 and 8-127
87 There was an error. The script may print on stderr an errno name,
88 optionally followed by whitespace and a message, for example:
89 echo 'ENOSPC Out of space' >&2
91 exit 1
92 or if you don't need the log message:
94 echo ENOSPC >&2
96 exit 1
97 If the script doesn't print anything or the output cannot be parsed
99 then nbdkit assumes error "EIO". Note that output to stderr is
100 ignored if the command succeeds, so it is acceptable to output a
101 potential error message prefix prior to attempting a command which
102 will add further details if a failure occurs.
103 2 The requested method is not supported by the script.
105 3 For methods which return booleans, this code indicates false.
107 4, 5, 6, 7
109 These exit codes are reserved for future use.
110 Temporary directory
112 A fresh script is invoked for each method call (ie. scripts are
113 stateless), so if the script needs to store state it has to store it
114 somewhere in the filesystem in a format and location which is left up
115 to the author of the script.
116 However nbdkit helps by creating a randomly named, empty directory for
118 the script. This directory persists for the lifetime of nbdkit and is
119 deleted when nbdkit exits. The name of the directory is passed to each
120 script invocation in the $tmpdir environment variable.
121 Handles
123 Handles are arbitrary strings, but it is best to limit them to short
124 alphanumeric strings.
125 Per-connection state
127 The temporary directory described above can be used for state for the
129 lifetime of the nbdkit instance (across multiple connections). If you
130 want to store state per connection then one way to do it is to create a
131 randomly named subdirectory under the temporary directory:
132 case "$1" in
134 ...
135 open)
136 mktemp -d $tmpdir/handle-XXXXXX ;;
137 The handle will be the subdirectory name, returned to the script as $2
139 in all connected calls (eg. "pread", "get_size"). You can delete the
140 subdirectory explicitly in "close":
141 case "$1" in
143 ...
144 close)
145 rm -rf "$2" ;;
146 or rely on nbdkit deleting the whole temporary directory including all
148 per-handle subdirectories when it exits.
149 Performance
151 This plugin has to fork on every request, so performance will never be
152 great. For best performance, consider using the nbdkit-plugin(3) API
153 directly. Having said that, if you have a sh plugin and want to
154 improve performance then the following tips may help:
155 Relax the thread model.
157 The default "thread_model" is "serialize_all_requests" meaning that
158 two instances of the script can never be running at the same time.
159 This is safe but slow. If your script is safe to be called in
160 parallel, set this to "parallel".
161 Implement the "zero" method.
163 If the "zero" method is not implemented then nbdkit will fall back
164 to using "pwrite" which is considerably slower because nbdkit has
165 to send blocks of zeroes to the script.
166 You don't have to write shell scripts.
168 This plugin can run any external binary, not only shell scripts.
169 You should get more performance by rewriting the shell script as a
170 program in a compiled language.
171 Methods
173 This just documents the arguments to the script corresponding to each
174 plugin method, and any way that they differ from the C callbacks. In
175 all other respects they work the same way as the C callbacks, so you
176 should go and read nbdkit-plugin(3).
177 "load"
179 /path/to/script load
180 "unload"
182 /path/to/script unload
183 This is called just before nbdkit exits. Errors from this method
185 are ignored.
186 "dump_plugin"
188 /path/to/script dump_plugin
189 "config"
191 /path/to/script config <key> <value>
192 "config_complete"
194 /path/to/script config_complete
195 "magic_config_key"
197 /path/to/script magic_config_key
198 If a magic config key is needed, this should echo it to stdout.
200 See "Magic parameters" in nbdkit(1).
201 "thread_model"
203 /path/to/script thread_model
204 On success this should print the desired thread model of the
206 script, one of "serialize_connections", "serialize_all_requests",
207 "serialize_requests", or "parallel".
208 This method is not required; if omitted, then the plugin will be
210 executed under the safe "serialize_all_requests" model. However,
211 this means that this method must be provided if you want to use the
212 "parallel" or "serialize_requests" model. Even then your request
213 may be restricted for other reasons; look for "thread_model" in the
214 output of "nbdkit --dump-plugin sh script" to see what actually
215 gets selected.
216 If an error occurs, the script should output an error message and
218 exit with status 1; unrecognized output is ignored.
219 "get_ready"
221 /path/to/script get_ready
222 "after_fork"
224 /path/to/script after_fork
225 "preconnect"
227 /path/to/script preconnect <readonly>
228 "list_exports"
230 /path/to/script list_exports <readonly> <tls>
231 The "readonly" and "tls" parameters will be "true" or "false".
233 The first line of output informs nbdkit how to parse the rest of
235 the output, the remaining lines then supply the inputs of the C
236 "nbdkit_add_export" function (see nbdkit-plugin(3)), as follows:
237 NAMES
239 The remaining output provides one export name per line, and no
240 export will be given a description. For convenience, this form
241 is also assumed if the first output line does not match one of
242 the recognized parse modes.
243 INTERLEAVED
245 The remaining output provides pairs of lines, the first line
246 being an export name, and the second the corresponding
247 description.
248 NAMES+DESCRIPTIONS
250 The number of remaining lines is counted, with the first half
251 being used as export names, and the second half providing
252 descriptions to pair with names from the first half.
253 An example of using this form to list files in the current
255 directory, followed by their ls(1) long description, would be:
256 echo NAMES+DESCRIPTIONS
258 ls
259 ls -l
260 Note that other output modes might be introduced in the future; in
262 particular, none of the existing modes allow a literal newline in
263 an export name or description, although this could be possible
264 under a new mode supporting escape sequences.
265 This method is not required; if it is absent, the list of exports
267 advertised by nbdkit will be the single name result of
268 "default_export" and no description.
269 "default_export"
271 /path/to/script default_export <readonly> <tls>
272 The "readonly" and "tls" parameters will be "true" or "false".
274 On success this should print a name on stdout to use in place of
276 the default export "", then exit with code 0. For convenience, the
277 output can be any of the list forms recognized by "list_exports",
278 in which case the first listed export name is used, and where an
279 empty list uses "". Given the current set of recognized export
280 lists, it is not possible for the resulting name to include a
281 newline.
282 This method is not required; if it is absent, the default export
284 name will be the empty string, "".
285 "open"
287 /path/to/script open <readonly> <exportname> <tls>
288 The "readonly" parameter will be "true" or "false". The
290 "exportname" parameter, if present, is the export name passed to
291 the server from the client. The "tls" parameter, if present, will
292 be "true" or "false" depending on whether the client is using TLS.
293 On success this should print the handle (any string) on stdout and
295 exit with code 0. If the handle ends with a newline character then
296 the newline is removed.
297 Unlike C plugins, this method is not required. If omitted then the
299 handle will be "" (empty string).
300 "close"
302 /path/to/script close <handle>
303 "export_description"
305 /path/to/script export_description <handle>
306 The script should print a human-readable description of the disk
308 image on stdout. If the description ends with a newline character
309 then the newline is removed.
310 This method is not required; if it is absent, no export description
312 will be provided to the client.
313 "get_size"
315 /path/to/script get_size <handle>
316 The script should print the size of the disk image on stdout. You
318 can print the size in bytes, or use any format understood by
319 "nbdkit_parse_size" such as "1M" (see "PARSING SIZE PARAMETERS" in
320 nbdkit-plugin(3)).
321 This method is required.
323 "block_size"
325 /path/to/script block_size <handle>
326 This script should print three numbers on stdout, separated by
328 whitespace. These are (in order) the minimum block size, the
329 preferred block size, and the maximum block size. You can print
330 the sizes in bytes or use any format understood by
331 "nbdkit_parse_size" such as "1M" (see "PARSING SIZE PARAMETERS" in
332 nbdkit-plugin(3)).
333 "can_write"
335 "can_flush"
336 "can_trim"
337 "can_zero"
338 "can_extents"
339 Unlike in other languages, you must provide the "can_*" methods
340 otherwise they are assumed to all return false and your "pwrite",
341 "flush", "trim", "zero" and "extents" methods will never be called.
342 The reason for this is obscure: In other languages we can detect if
343 (eg) a "pwrite" method is defined and synthesize an appropriate
344 response if no actual "can_write" method is defined. However
345 detecting if methods are present without running them is not
346 possible with this plugin.
347 /path/to/script can_write <handle>
349 /path/to/script can_flush <handle>
350 /path/to/script can_trim <handle>
351 /path/to/script can_zero <handle>
352 /path/to/script can_extents <handle>
353 The script should exit with code 0 for true or code 3 for false.
355 "is_rotational"
357 "can_fast_zero"
358 /path/to/script is_rotational <handle>
359 /path/to/script can_fast_zero <handle>
360 The script should exit with code 0 for true or code 3 for false.
362 "can_fua"
364 "can_cache"
365 /path/to/script can_fua <handle>
366 /path/to/script can_cache <handle>
367 These control Forced Unit Access (FUA) and caching behaviour of the
369 core server.
370 Unlike the other "can_*" callbacks, these two are not a boolean.
372 They must print either "none", "emulate" or "native" to stdout.
373 The meaning of these is described in nbdkit-plugin(3).
374 Furthermore, you must provide a "can_cache" method if you desire
375 the "cache" callback to be utilized, similar to the reasoning
376 behind requiring "can_write" to utilize "pwrite".
377 "can_multi_conn"
379 /path/to/script can_multi_conn <handle>
380 The script should exit with code 0 for true or code 3 for false.
382 "pread"
384 /path/to/script pread <handle> <count> <offset>
385 The script should print the requested binary data on stdout.
387 Exactly "count" bytes must be printed.
388 This method is required.
390 "pwrite"
392 /path/to/script pwrite <handle> <count> <offset> <flags>
393 The script should read the binary data to be written from stdin.
395 The "flags" parameter can be an empty string or "fua". In the
397 future, a comma-separated list of flags may be present.
398 Unlike in other languages, if you provide a "pwrite" method you
400 must also provide a "can_write" method which exits with code 0
401 (true).
402 "flush"
404 /path/to/script flush <handle>
405 Unlike in other languages, if you provide a "flush" method you must
407 also provide a "can_flush" method which exits with code 0 (true).
408 "trim"
410 /path/to/script trim <handle> <count> <offset> <flags>
411 The "flags" parameter can be an empty string or "fua". In the
413 future, a comma-separated list of flags may be present.
414 Unlike in other languages, if you provide a "trim" method you must
416 also provide a "can_trim" method which exits with code 0 (true).
417 "zero"
419 /path/to/script zero <handle> <count> <offset> <flags>
420 The "flags" parameter can be an empty string or a comma-separated
422 list of the flags: "fua", "may_trim", and "fast" (eg. "", "fua",
423 "fua,may_trim,fast" are some of the 8 possible values).
424 Unlike in other languages, if you provide a "zero" method you must
426 also provide a "can_zero" method which exits with code 0 (true).
427 To trigger a fallback to <pwrite> on a normal zero request, or to
429 respond quickly to the "fast" flag that a specific zero request is
430 no faster than a corresponding write, the script must output
431 "ENOTSUP" or "EOPNOTSUPP" to stderr (possibly followed by a
432 description of the problem) before exiting with code 1 (failure).
433 "extents"
435 /path/to/script extents <handle> <count> <offset> <flags>
436 The "flags" parameter can be an empty string or "req_one".
438 This must print, one per line on stdout, a list of one or more
440 extents in the format:
441 offset length type
443 which correspond to the inputs of the C "nbdkit_add_extent"
445 function (see nbdkit-plugin(3)). The "offset" and "length" fields
446 may use any format understood by "nbdkit_parse_size". The optional
447 "type" field may be an integer, missing (same as 0), or a comma-
448 separated list of the words "hole" and "zero". An example of a
449 valid set of extents covering a "10M" disk where the first megabyte
450 only is allocated data:
451 0 1M
453 1M 9M hole,zero
454 Unlike in other languages, if you provide an "extents" method you
456 must also provide a "can_extents" method which exits with code 0
457 (true).
458 "cache"
460 /path/to/script cache <handle> <count> <offset>
461 Unlike in other languages, if you provide a "cache" method you must
463 also provide a "can_cache" method which prints "native" and exits
464 with code 0 (true).
465 Missing callbacks
467 Missing: "name", "version", "longname", "description", "config_help"
468 These are not yet supported.
469
471 $plugindir/nbdkit-sh-plugin.so
472 The plugin.
473 Use "nbdkit --dump-config" to find the location of $plugindir.
475
477 "nbdkit-sh-plugin" first appeared in nbdkit 1.8.
478
480 nbdkit(1), nbdkit-plugin(3), nbdkit-eval-plugin(1),
481 nbdkit-cc-plugin(1).
482
484 Richard W.M. Jones
485
487 Copyright (C) 2018-2020 Red Hat Inc.
488
490 Redistribution and use in source and binary forms, with or without
491 modification, are permitted provided that the following conditions are
492 met:
493 • Redistributions of source code must retain the above copyright
495 notice, this list of conditions and the following disclaimer.
496 • Redistributions in binary form must reproduce the above copyright
498 notice, this list of conditions and the following disclaimer in the
499 documentation and/or other materials provided with the
500 distribution.
501 • Neither the name of Red Hat nor the names of its contributors may
503 be used to endorse or promote products derived from this software
504 without specific prior written permission.
505 THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND ANY
507 EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
508 IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
509 PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR CONTRIBUTORS BE
510 LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
511 CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
512 SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
513 BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
514 WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
515 OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
516 ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
517nbdkit-1.30.7 2022-07-10 nbdkit-sh-plugin(3)