nbdkit-data-plugin(1)

1nbdkit-data-plugin(1)               NBDKIT               nbdkit-data-plugin(1)
2
3
4

NAME

6       nbdkit-data-plugin - nbdkit plugin for serving data from the command
7       line
8

SYNOPSIS

10        nbdkit data [data=]'0 1 2 3 @0x1fe 0x55 0xaa'
11                    [size=SIZE] [allocator=sparse|...]
12
13        nbdkit data base64='aGVsbG8gbmJka2l0IHVzZXI='
14                    [size=SIZE] [allocator=sparse|...]
15
16        nbdkit data raw='binary_data'
17                    [size=SIZE] [allocator=sparse|...]
18

DESCRIPTION

20       "nbdkit-data-plugin" is a plugin for nbdkit(1) which serves a small
21       amount of data specified directly on the command line.  The plugin gets
22       its name from the "data:" URI scheme used by web browsers.  This is
23       mainly useful for testing NBD clients.
24
25       You can serve data read-only using the -r flag, or read-write.  Any
26       writes are thrown away when nbdkit exits.
27
28       Most operating systems have command line size limits which are quite a
29       lot smaller than any desirable disk image, so specifying a large, fully
30       populated disk image on the command line would not be possible.
31       However you can specify a small amount of data at the beginning of the
32       image, possibly followed by zeroes (using the "size" parameter to pad
33       the image to the full size), or use the "data" parameter creatively to
34       make mostly sparse disk images.
35
36       The "size" parameter can specify any virtual size up to the maximum
37       supported by nbdkit (2⁶³-1 bytes).
38

EXAMPLES

40   Create small disks filled with test patterns
41        nbdkit data ' ( 0x55 0xAA )*2048 '
42        nbdkit data ' ( "Hello" )*2000 ' size=8192
43
44       The first command creates a disk containing 4096 bytes filled with the
45       repeating bytes 0x55 0xAA.  The second command repeats
46       "HelloHelloHello...", truncating the disk to exactly 8192 bytes.
47
48       See also nbdkit-pattern-plugin(3).
49
50   Create a 1 MB disk with one empty MBR-formatted partition
51        nbdkit data '
52          @0x1b8 178 190 207 221 0 0 0 0 2 0 131 32 32 0 1 0 0 0 255 7
53          @0x1fe 85 170
54          ' size=1M
55
56       This example was created by running:
57
58        $ rm -f disk
59        $ truncate -s 1M disk
60        $ echo start=1 | sfdisk disk
61        Device Boot Start   End Sectors    Size Id Type
62        disk1           1  2047    2047 1023.5K 83 Linux
63        $ ./disk2data.pl disk
64
65       The "disk2data.pl" script is provided in the nbdkit sources
66       (https://gitlab.com/nbdkit/nbdkit/blob/master/plugins/data/disk2data.pl).
67
68       See also nbdkit-partitioning-plugin(1).
69
70   Create a disk image with sector-aligned data
71        nbdkit data ' <file1 @^512 <file2 @^512 <file3 @^512 '
72
73       Local binary files file1, file2 and file3 are copied into the disk
74       image.  Regardless of the size of these files, they will all be aligned
75       to 512-byte sector boundaries.  Furthermore because of the final
76       alignment operation ("@^512") the total size of the disk will also be
77       rounded to a whole number of sectors.
78
79   Create a disk with the same random data in each sector
80        nbdkit data ' </dev/urandom[:512]*16 '
81
82       The expression "</dev/urandom[:512]" reads 512 bytes (one sector) of
83       randomness from the system.  The same random data is repeated over 16
84       sectors.
85
86   Create a 1 MB disk with some nonsense data at the beginning
87        nbdkit data base64=MTIz size=1M
88
89       The above command serves the bytes "0x31 0x32 0x33" (which is the
90       base64 decoding of "MTIz"), followed by 1M - 3 bytes of zeroes.
91
92   "Hello, world" using this plugin
93        $ nbdkit data raw='Hello, world!' --run 'nbdcopy "$uri" - | cat'
94        Hello, world!
95
96       This works by creating a disk containing the string "Hello, world!".
97       nbdcopy(1) connects to the server using an NBD URI ("$uri") and copies
98       the disk to stdout ("-").  The extra cat(1) is needed because nbdcopy
99       refuses to write raw disk data to a terminal.
100

PARAMETERS

102       Exactly one of the "data", "base64" or "raw" parameters must be
103       supplied.
104
105       [data=]DATA
106           Specify the disk data using a simple compact format.  See "DATA
107           FORMAT" below.
108
109           "data=" is a magic config key and may be omitted in most cases.
110           See "Magic parameters" in nbdkit(1).
111
112       base64=BASE64
113           The "base64" parameter can be used to supply binary data encoded in
114           base64 on the command line.
115
116           This is only supported if nbdkit was compiled with GnuTLS ≥ 3.6.0.
117           You can find out by checking if:
118
119            $ nbdkit data --dump-plugin
120
121           contains:
122
123            data_base64=yes
124
125       raw=BINARY
126           The "raw" parameter can be used to supply raw binary data directly
127           on the command line.
128
129           It is usually quite difficult to do this unless you are running
130           nbdkit from another program (see nbdkit-captive(1)).  One
131           particular problem is that the data must not contain zero bytes
132           (ie. "\0") since those will be processed in C to mean the end of
133           the string.  In almost all cases it is better to use base64
134           encoding or the custom "data" format.
135
136       size=SIZE
137           The data is truncated or extended to the size specified.
138
139           This parameter is optional: If omitted the size is defined by the
140           size of the "data", "raw" or "base64" parameter.
141
142       allocator=sparse
143       allocator=malloc[,mlock=true]
144       allocator=zstd
145           (nbdkit ≥ 1.22)
146
147           Select the backend allocation strategy.  See "ALLOCATORS" in
148           nbdkit-memory-plugin(1).  The default is sparse.
149

DATA FORMAT

151       The "data" parameter lets you specify small disk images in a simple,
152       compact format.  It is a string containing a list of bytes which are
153       written into the disk image sequentially.  You can move the virtual
154       offset where bytes are written using @offset.
155
156       For example:
157
158        nbdkit data '0 1 2 3 @0x1fe 0x55 0xaa'
159
160       creates a 0x200 = 512 byte (1 sector) image containing the four bytes
161       "0 1 2 3" at the start, and the two bytes "0x55 0xaa" at the end of the
162       sector, with the remaining 506 bytes in the middle being all zeroes.
163       In this example the size (512 bytes) is implied by the data.  But you
164       could additionally use the "size" parameter to either truncate or
165       extend (with zeroes) the disk image.
166
167       Whitespace between fields in the string is ignored.
168
169       Fields in the string can be:
170
171       BYTE
172           Write a byte at the current offset and advance the offset by 1.
173           The byte may be specified as either decimal, octal (prefixed by 0)
174           or hexadecimal (prefixed by "0x").  To add repeated bytes use the
175           "*" operator (eg. "0xFF*16").
176
177       @OFFSET
178           Moves the current offset to "OFFSET".  The offset may be specified
179           as either decimal, octal (prefixed by 0) or hexadecimal (prefixed
180           by "0x").  Offset @0 is the first byte of the disk.
181
182       @+N
183       @-N (nbdkit ≥ 1.22)
184
185           Add or subtract "N" from the current offset.
186
187       @^ALIGNMENT
188           (nbdkit ≥ 1.22)
189
190           If the current offset is not a multiple of "ALIGNMENT" then the
191           offset is moved forward to the next multiple.  The next byte
192           written will be aligned to "ALIGNMENT".
193
194       <FILE
195           (nbdkit ≥ 1.8)
196
197           Read the contents of binary FILE into the disk image at the current
198           offset.  The offset is incremented by the size of the file.  The
199           filename can be a relative or absolute path, but cannot contain
200           whitespace in the name.
201
202       <(SCRIPT)
203           (nbdkit ≥ 1.24, not Windows)
204
205           Substitute the output of the shell script or external program as a
206           binary blob and advance the offset by the length in bytes of the
207           output.  You can use this to create more complex test patterns.
208           For example this produces a 32K disk image with an incrementing
209           test pattern in groups of 4 bytes:
210
211            nbdkit data ' <( i=0
212                             while :; do
213                                 printf "%04d" $((i++))
214                             done )[:32768] '
215
216           The script may contain "(" and ")" characters, but they must be in
217           matching pairs.  A script can produce a finite amount of output; or
218           (as in the example) an infinite amount which must be truncated
219           using the "[:len]" slice operator.
220
221           Scripts must be idempotent, producing the same output each time
222           they are run.  This is because optimizations might change the order
223           of evaluation or number of times the script is called and you could
224           get different output in a future version of nbdkit.
225
226       "STRING"
227           (nbdkit ≥ 1.22)
228
229           Write a string into the image at the current offset and advance the
230           offset by the length of the string.  To include special characters
231           in the string you can escape them in the same way as C strings (eg.
232           a double quote character within the string should be written "\"").
233           Be careful with shell quoting around the whole data parameter.
234
235       ( ... )
236           (nbdkit ≥ 1.24)
237
238           Group a set of expressions into a single expression.
239
240           "( ... )" recursively creates a new data parser so any of the above
241           operators can appear inside, including nested "( ... )".  Note that
242           offsets and alignments within the subpattern are relative to the
243           start of the subpattern, not relative to the final disk image.
244
245       expression * N
246           (nbdkit ≥ 1.24)
247
248           Repeat the expression "N" times.  The offset is incremented by the
249           length of the expression × N.  For example to create a repeating
250           pattern of 0x55, 0xAA for 512 (2×256) bytes do:
251
252            nbdkit data '( 0x55 0xAA ) * 256'
253
254       expression [N:M]
255           (nbdkit ≥ 1.24)
256
257           Take a slice of the expression.  Slices are [start:end+1] where
258           start and end are the first and last byte offsets of the expression
259           desired.  Either or both may be omitted.  [:len] means to take the
260           first len bytes.  [start:] means to take bytes from offset start to
261           the end of the expression.
262
263       expression -> \NAME
264       \NAME
265           (nbdkit ≥ 1.24)
266
267           Assign an expression to a name which can be used later.  Names can
268           be used in the current scope (or any scopes nested within the
269           current scope), but disappear at the end of the current scope.
270           Names start with a backslash character followed by one or more
271           alphanumeric, dash and underscore.  For example this makes two
272           identical sectors both containing a boot signature at the end:
273
274            nbdkit data ' ( 0x55 0xAA ) -> \boot-signature
275                          ( @0x1fe \boot-signature ) -> \sector
276                          \sector \sector '
277
278       $VAR
279           (nbdkit ≥ 1.24)
280
281           Substitute command line parameters or environment variables.  The
282           variable is written in the same language as the "data" parameter,
283           and when substituted it creates a nested scope like "( ... )"
284           expressions.  These are all equivalent:
285
286            nbdkit data '$pattern*16' pattern='0x55 0xAA'
287
288            export pattern='0x55 0xAA'
289            nbdkit data '$pattern*16'
290
291            nbdkit data '( 0x55 0xAA )*16'
292
293       # COMMENT
294           (nbdkit ≥ 1.24)
295
296           "#" begins a comment stretching to the end of the current line.
297
298   disk2data.pl script
299       This script can convert from small disk images into the data format
300       described above.
301
302       It is provided in the nbdkit sources.  See
303       https://gitlab.com/nbdkit/nbdkit/blob/master/plugins/data/disk2data.pl
304

FILES

306       $plugindir/nbdkit-data-plugin.so
307           The plugin.
308
309           Use "nbdkit --dump-config" to find the location of $plugindir.
310

VERSION

312       "nbdkit-data-plugin" first appeared in nbdkit 1.6.
313

AUTHORS

324       Richard W.M. Jones
325

COPYRIGHT

327       Copyright (C) 2018-2020 Red Hat Inc.
328

LICENSE

330       Redistribution and use in source and binary forms, with or without
331       modification, are permitted provided that the following conditions are
332       met:
333
334       •   Redistributions of source code must retain the above copyright
335           notice, this list of conditions and the following disclaimer.
336
337       •   Redistributions in binary form must reproduce the above copyright
338           notice, this list of conditions and the following disclaimer in the
339           documentation and/or other materials provided with the
340           distribution.
341
342       •   Neither the name of Red Hat nor the names of its contributors may
343           be used to endorse or promote products derived from this software
344           without specific prior written permission.
345
346       THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND ANY
347       EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
348       IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
349       PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR CONTRIBUTORS BE
350       LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
351       CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
352       SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
353       BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
354       WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
355       OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
356       ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
357
358
359
360nbdkit-1.25.8                     2021-05-25             nbdkit-data-plugin(1)