1nbdkit-sh-plugin(3)                 NBDKIT                 nbdkit-sh-plugin(3)
2
3
4

NAME

6       nbdkit-sh-plugin - nbdkit shell, script or executable plugin
7

SYNOPSIS

9        nbdkit sh /path/to/script [arguments...]
10
11        nbdkit sh - <<'EOF'
12        ... shell script ...
13        EOF
14

DESCRIPTION

16       "nbdkit-sh-plugin" allows you to write plugins for nbdkit(1) using
17       arbitrary scripting languages, including shells like bash(1), dash(1),
18       csh(1), zsh(1) etc., other scripting environments, or any executable.
19
20       Note if you want to use an established scripting language like Perl or
21       Python, then nbdkit has specific plugins to handle those languages and
22       those will be more efficient (see nbdkit(1) for a complete list).
23
24       To use shell script fragments from the nbdkit command line (rather than
25       a separate script) see nbdkit-eval-plugin(1).
26
27   If you have been given an nbdkit sh plugin
28       Assuming you have a shell script which is an nbdkit plugin, you run it
29       like this:
30
31        nbdkit sh /path/to/script
32
33       You may have to add further "key=value" arguments to the command line.
34       The script must be executable ("chmod +x").
35
36   Inline shell scripts
37       It is also possible to write a shell script plugin "inline" using "-"
38       as the name of the script, like this:
39
40        nbdkit sh - <<'EOF'
41          case "$1" in
42            get_size) echo 1M ;;
43            pread) dd if=/dev/zero count=$3 iflag=count_bytes ;;
44            *) exit 2 ;;
45          esac
46        EOF
47
48       By default the inline script runs under /bin/sh.  You can add a shebang
49       ("#!") to use other scripting languages.  Of course, reading an inline
50       script from stdin is incompatible with the -s (--single) mode of nbdkit
51       that connects a client on stdin.
52

WRITING AN NBDKIT SH PLUGIN

54       For an example plugin written in Bash, see:
55       https://gitlab.com/nbdkit/nbdkit/blob/master/plugins/sh/example.sh
56
57       Broadly speaking, nbdkit shell plugins work like C ones, so you should
58       read nbdkit-plugin(3) first.
59
60   Programming model
61       This plugin has a simple programming model: For every plugin method
62       that needs to be called, the external script is invoked with parameters
63       describing the method and its arguments.  The first parameter is always
64       the method name.  For example:
65
66        /path/to/script config file disk.img
67                          │      │   │
68                          │      │   └─ value ($3)
69                          │      └── key ($2)
70                      method ($1)
71
72        /path/to/script pread <handle> <count> <offset>
73                          │       │       │       │
74                          │       │       │       └─ offset in bytes ($4)
75                          │       │       └── request size in bytes ($3)
76                      method ($1) └── handle ($2) ─ see "Handles" below
77
78       Scripts should ignore extra parameters that they don't understand since
79       we may add new parameters in future.
80
81   Exit codes
82       The script should exit with specific exit codes:
83
84       0   The method was executed successfully.
85
86       1 and 16-255
87           There was an error.  The script may print on stderr an errno name,
88           optionally followed by whitespace and a message, for example:
89
90            echo 'ENOSPC Out of space' >&2
91            exit 1
92
93           or if you don't need the log message:
94
95            echo ENOSPC >&2
96            exit 1
97
98           If the script doesn't print anything or the output cannot be parsed
99           then nbdkit assumes error "EIO".  Note that output to stderr is
100           ignored if the command succeeds, so it is acceptable to output a
101           potential error message prefix prior to attempting a command which
102           will add further details if a failure occurs.
103
104       2   The requested method is not supported by the script.
105
106       3   For methods which return booleans, this code indicates false.
107
108       4 and 5
109           Triggers a call to the C function "nbdkit_shutdown", which requests
110           an asynchronous exit of the nbdkit server (disconnecting all
111           clients).  The client will usually get a response before shutdown
112           is complete (although this is racy); so once the shutdown is
113           requested, code 4 then behaves like code 0 (stderr is ignored, and
114           the server tries to return success), and code 5 behaves like code 1
115           (the server tries to return an error to the client parsed from
116           stderr, although a missing error defaults to "ESHUTDOWN" instead of
117           "EIO").
118
119       6   Triggers a call to the C function "nbdkit_disconnect" with "force"
120           set to true, which requests an abrupt disconnect of the current
121           client.  The contents of stderr are irrelevant with this status,
122           since the client will not get a response.
123
124       7 and 8
125           Triggers a call to the C function "nbdkit_disconnect" with "force"
126           set to false, which requests a soft disconnect of the current
127           client (future client requests are rejected with "ESHUTDOWN"
128           without calling into the plugin, but current requests may
129           complete).  Since the client will likely get the response to this
130           command, code 7 then behaves like code 0 (stderr is ignored, and
131           the server tries to return success), and code 8 behaves like code 1
132           (the server tries to return an error to the client parsed from
133           stderr, although a missing error defaults to "ESHUTDOWN" instead of
134           "EIO").
135
136       9-15
137           These exit codes are reserved for future use.  Note that versions
138           of nbdkit < 1.34 documented that codes 8 through 15 behaved like
139           code 1; although it is unlikely that many scripts relied on this
140           similarity in practice.
141
142       In nbdkit > 1.34, it is possible to probe whether additional exit codes
143       have been assigned meaning, by looking for the line max_known_status=
144       in the output of nbdkit --dump-plugin sh.  If this line is not present,
145       exit codes 4 and above behave like status 1.
146
147   Temporary directory
148       A fresh script is invoked for each method call (ie. scripts are
149       stateless), so if the script needs to store state it has to store it
150       somewhere in the filesystem in a format and location which is left up
151       to the author of the script.
152
153       However nbdkit helps by creating a randomly named, empty directory for
154       the script.  This directory persists for the lifetime of nbdkit and is
155       deleted when nbdkit exits.  The name of the directory is passed to each
156       script invocation in the $tmpdir environment variable.
157
158   Handles
159       Handles are arbitrary strings, but it is best to limit them to short
160       alphanumeric strings.
161
162       Per-connection state
163
164       The temporary directory described above can be used for state for the
165       lifetime of the nbdkit instance (across multiple connections).  If you
166       want to store state per connection then one way to do it is to create a
167       randomly named subdirectory under the temporary directory:
168
169        case "$1" in
170          ...
171          open)
172            mktemp -d $tmpdir/handle-XXXXXX ;;
173
174       The handle will be the subdirectory name, returned to the script as $2
175       in all connected calls (eg. "pread", "get_size").  You can delete the
176       subdirectory explicitly in "close":
177
178        case "$1" in
179          ...
180          close)
181            rm -rf "$2" ;;
182
183       or rely on nbdkit deleting the whole temporary directory including all
184       per-handle subdirectories when it exits.
185
186   Performance
187       This plugin has to fork on every request, so performance will never be
188       great.  For best performance, consider using the nbdkit-plugin(3) API
189       directly.  Having said that, if you have a sh plugin and want to
190       improve performance then the following tips may help:
191
192       Relax the thread model.
193           The default "thread_model" is "serialize_all_requests" meaning that
194           two instances of the script can never be running at the same time.
195           This is safe but slow.  If your script is safe to be called in
196           parallel, set this to "parallel".
197
198       Implement the "zero" method.
199           If the "zero" method is not implemented then nbdkit will fall back
200           to using "pwrite" which is considerably slower because nbdkit has
201           to send blocks of zeroes to the script.
202
203       You don't have to write shell scripts.
204           This plugin can run any external binary, not only shell scripts.
205           You should get more performance by rewriting the shell script as a
206           program in a compiled language.
207
208   Methods
209       This just documents the arguments to the script corresponding to each
210       plugin method, and any way that they differ from the C callbacks.  In
211       all other respects they work the same way as the C callbacks, so you
212       should go and read nbdkit-plugin(3).
213
214       "load"
215            /path/to/script load
216
217       "unload"
218            /path/to/script unload
219
220           This is called just before nbdkit exits.  Errors from this method
221           are ignored.
222
223       "dump_plugin"
224            /path/to/script dump_plugin
225
226       "config"
227            /path/to/script config <key> <value>
228
229       "config_complete"
230            /path/to/script config_complete
231
232       "magic_config_key"
233            /path/to/script magic_config_key
234
235           If a magic config key is needed, this should echo it to stdout.
236           See "Magic parameters" in nbdkit(1).
237
238       "thread_model"
239            /path/to/script thread_model
240
241           On success this should print the desired thread model of the
242           script, one of "serialize_connections", "serialize_all_requests",
243           "serialize_requests", or "parallel".
244
245           This method is not required; if omitted, then the plugin will be
246           executed under the safe "serialize_all_requests" model.  However,
247           this means that this method must be provided if you want to use the
248           "parallel" or "serialize_requests" model.  Even then your request
249           may be restricted for other reasons; look for "thread_model" in the
250           output of "nbdkit --dump-plugin sh script" to see what actually
251           gets selected.
252
253           If an error occurs, the script should output an error message and
254           exit with status 1; unrecognized output is ignored.
255
256       "get_ready"
257            /path/to/script get_ready
258
259       "after_fork"
260            /path/to/script after_fork
261
262       "preconnect"
263            /path/to/script preconnect <readonly>
264
265       "list_exports"
266            /path/to/script list_exports <readonly> <tls>
267
268           The "readonly" and "tls" parameters will be "true" or "false".
269
270           The first line of output informs nbdkit how to parse the rest of
271           the output, the remaining lines then supply the inputs of the C
272           "nbdkit_add_export" function (see nbdkit-plugin(3)), as follows:
273
274           NAMES
275               The remaining output provides one export name per line, and no
276               export will be given a description.  For convenience, this form
277               is also assumed if the first output line does not match one of
278               the recognized parse modes.
279
280           INTERLEAVED
281               The remaining output provides pairs of lines, the first line
282               being an export name, and the second the corresponding
283               description.
284
285           NAMES+DESCRIPTIONS
286               The number of remaining lines is counted, with the first half
287               being used as export names, and the second half providing
288               descriptions to pair with names from the first half.
289
290               An example of using this form to list files in the current
291               directory, followed by their ls(1) long description, would be:
292
293                echo NAMES+DESCRIPTIONS
294                ls
295                ls -l
296
297           Note that other output modes might be introduced in the future; in
298           particular, none of the existing modes allow a literal newline in
299           an export name or description, although this could be possible
300           under a new mode supporting escape sequences.
301
302           This method is not required; if it is absent, the list of exports
303           advertised by nbdkit will be the single name result of
304           "default_export" and no description.
305
306       "default_export"
307            /path/to/script default_export <readonly> <tls>
308
309           The "readonly" and "tls" parameters will be "true" or "false".
310
311           On success this should print a name on stdout to use in place of
312           the default export "", then exit with code 0.  For convenience, the
313           output can be any of the list forms recognized by "list_exports",
314           in which case the first listed export name is used, and where an
315           empty list uses "".  Given the current set of recognized export
316           lists, it is not possible for the resulting name to include a
317           newline.
318
319           This method is not required; if it is absent, the default export
320           name will be the empty string, "".
321
322       "open"
323            /path/to/script open <readonly> <exportname> <tls>
324
325           The "readonly" parameter will be "true" or "false".  The
326           "exportname" parameter, if present, is the export name passed to
327           the server from the client.  The "tls" parameter, if present, will
328           be "true" or "false" depending on whether the client is using TLS.
329
330           On success this should print the handle (any string) on stdout and
331           exit with code 0.  If the handle ends with a newline character then
332           the newline is removed.
333
334           Unlike C plugins, this method is not required.  If omitted then the
335           handle will be "" (empty string).
336
337       "close"
338            /path/to/script close <handle>
339
340       "export_description"
341            /path/to/script export_description <handle>
342
343           The script should print a human-readable description of the disk
344           image on stdout.  If the description ends with a newline character
345           then the newline is removed.
346
347           This method is not required; if it is absent, no export description
348           will be provided to the client.
349
350       "get_size"
351            /path/to/script get_size <handle>
352
353           The script should print the size of the disk image on stdout.  You
354           can print the size in bytes, or use any format understood by
355           "nbdkit_parse_size" such as "1M" (see "PARSING SIZE PARAMETERS" in
356           nbdkit-plugin(3)).
357
358           This method is required.
359
360       "block_size"
361            /path/to/script block_size <handle>
362
363           This script should print three numbers on stdout, separated by
364           whitespace.  These are (in order) the minimum block size, the
365           preferred block size, and the maximum block size.  You can print
366           the sizes in bytes or use any format understood by
367           "nbdkit_parse_size" such as "1M" (see "PARSING SIZE PARAMETERS" in
368           nbdkit-plugin(3)).
369
370       "can_write"
371       "can_flush"
372       "can_trim"
373       "can_zero"
374       "can_extents"
375           Unlike in other languages, you must provide the "can_*" methods
376           otherwise they are assumed to all return false and your "pwrite",
377           "flush", "trim", "zero" and "extents" methods will never be called.
378           The reason for this is obscure: In other languages we can detect if
379           (eg) a "pwrite" method is defined and synthesize an appropriate
380           response if no actual "can_write" method is defined.  However
381           detecting if methods are present without running them is not
382           possible with this plugin.
383
384            /path/to/script can_write <handle>
385            /path/to/script can_flush <handle>
386            /path/to/script can_trim <handle>
387            /path/to/script can_zero <handle>
388            /path/to/script can_extents <handle>
389
390           The script should exit with code 0 for true or code 3 for false.
391
392       "is_rotational"
393       "can_fast_zero"
394            /path/to/script is_rotational <handle>
395            /path/to/script can_fast_zero <handle>
396
397           The script should exit with code 0 for true or code 3 for false.
398
399       "can_fua"
400       "can_cache"
401            /path/to/script can_fua <handle>
402            /path/to/script can_cache <handle>
403
404           These control Forced Unit Access (FUA) and caching behaviour of the
405           core server.
406
407           Unlike the other "can_*" callbacks, these two are not a boolean.
408           They must print either "none", "emulate" or "native" to stdout.
409           The meaning of these is described in nbdkit-plugin(3).
410           Furthermore, you must provide a "can_cache" method if you desire
411           the "cache" callback to be utilized, similar to the reasoning
412           behind requiring "can_write" to utilize "pwrite".
413
414       "can_multi_conn"
415            /path/to/script can_multi_conn <handle>
416
417           The script should exit with code 0 for true or code 3 for false.
418
419       "pread"
420            /path/to/script pread <handle> <count> <offset>
421
422           The script should print the requested binary data on stdout.
423           Exactly "count" bytes must be printed.
424
425           This method is required.
426
427       "pwrite"
428            /path/to/script pwrite <handle> <count> <offset> <flags>
429
430           The script should read the binary data to be written from stdin.
431
432           The "flags" parameter can be an empty string or "fua".  In the
433           future, a comma-separated list of flags may be present.
434
435           Unlike in other languages, if you provide a "pwrite" method you
436           must also provide a "can_write" method which exits with code 0
437           (true).
438
439       "flush"
440            /path/to/script flush <handle>
441
442           Unlike in other languages, if you provide a "flush" method you must
443           also provide a "can_flush" method which exits with code 0 (true).
444
445       "trim"
446            /path/to/script trim <handle> <count> <offset> <flags>
447
448           The "flags" parameter can be an empty string or "fua".  In the
449           future, a comma-separated list of flags may be present.
450
451           Unlike in other languages, if you provide a "trim" method you must
452           also provide a "can_trim" method which exits with code 0 (true).
453
454       "zero"
455            /path/to/script zero <handle> <count> <offset> <flags>
456
457           The "flags" parameter can be an empty string or a comma-separated
458           list of the flags: "fua", "may_trim", and "fast" (eg. "", "fua",
459           "fua,may_trim,fast" are some of the 8 possible values).
460
461           Unlike in other languages, if you provide a "zero" method you must
462           also provide a "can_zero" method which exits with code 0 (true).
463
464           To trigger a fallback to "pwrite" on a normal zero request, or to
465           respond quickly to the "fast" flag that a specific zero request is
466           no faster than a corresponding write, the script must output
467           "ENOTSUP" or "EOPNOTSUPP" to stderr (possibly followed by a
468           description of the problem) before exiting with code 1 (failure).
469
470       "extents"
471            /path/to/script extents <handle> <count> <offset> <flags>
472
473           The "flags" parameter can be an empty string or "req_one".
474
475           This must print, one per line on stdout, a list of one or more
476           extents in the format:
477
478            offset length type
479
480           which correspond to the inputs of the C "nbdkit_add_extent"
481           function (see nbdkit-plugin(3)).  The "offset" and "length" fields
482           may use any format understood by "nbdkit_parse_size".  The optional
483           "type" field may be an integer, missing (same as 0), or a comma-
484           separated list of the words "hole" and "zero".  An example of a
485           valid set of extents covering a "10M" disk where the first megabyte
486           only is allocated data:
487
488            0  1M
489            1M 9M  hole,zero
490
491           Unlike in other languages, if you provide an "extents" method you
492           must also provide a "can_extents" method which exits with code 0
493           (true).
494
495       "cache"
496            /path/to/script cache <handle> <count> <offset>
497
498           Unlike in other languages, if you provide a "cache" method you must
499           also provide a "can_cache" method which prints "native" and exits
500           with code 0 (true).
501
502   Missing callbacks
503       Missing: "name", "version", "longname", "description", "config_help"
504           These are not yet supported.
505

FILES

507       $plugindir/nbdkit-sh-plugin.so
508           The plugin.
509
510           Use "nbdkit --dump-config" to find the location of $plugindir.
511

VERSION

513       "nbdkit-sh-plugin" first appeared in nbdkit 1.8.
514

SEE ALSO

516       nbdkit(1), nbdkit-plugin(3), nbdkit-eval-plugin(1),
517       nbdkit-cc-plugin(1).
518

AUTHORS

520       Richard W.M. Jones
521
523       Copyright Red Hat
524

LICENSE

526       Redistribution and use in source and binary forms, with or without
527       modification, are permitted provided that the following conditions are
528       met:
529
530       •   Redistributions of source code must retain the above copyright
531           notice, this list of conditions and the following disclaimer.
532
533       •   Redistributions in binary form must reproduce the above copyright
534           notice, this list of conditions and the following disclaimer in the
535           documentation and/or other materials provided with the
536           distribution.
537
538       •   Neither the name of Red Hat nor the names of its contributors may
539           be used to endorse or promote products derived from this software
540           without specific prior written permission.
541
542       THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND ANY
543       EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
544       IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
545       PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR CONTRIBUTORS BE
546       LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
547       CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
548       SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
549       BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
550       WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
551       OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
552       ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
553
554
555
556nbdkit-1.36.2                     2023-11-26               nbdkit-sh-plugin(3)
Impressum