1nbdkit-data-plugin(1) NBDKIT nbdkit-data-plugin(1)
2
6 nbdkit-data-plugin - nbdkit plugin for serving data from the command
7 line
8
10 nbdkit data [data=]'0 1 2 3 @0x1fe 0x55 0xaa'
11 [size=SIZE] [allocator=sparse|malloc|zstd]
12 nbdkit data base64='aGVsbG8gbmJka2l0IHVzZXI='
14 [size=SIZE] [allocator=sparse|malloc|zstd]
15 nbdkit data raw='binary_data'
17 [size=SIZE] [allocator=sparse|malloc|zstd]
18
20 "nbdkit-data-plugin" is a plugin for nbdkit(1) which serves a small
21 amount of data specified directly on the command line. The plugin gets
22 its name from the "data:" URI scheme used by web browsers. This is
23 mainly useful for testing NBD clients.
24 You can serve data read-only using the -r flag, or read-write. Any
26 writes are thrown away when nbdkit exits.
27 Most operating systems have command line size limits which are quite a
29 lot smaller than any desirable disk image, so specifying a large, fully
30 populated disk image on the command line would not be possible.
31 However you can specify a small amount of data at the beginning of the
32 image, possibly followed by zeroes (using the "size" parameter to pad
33 the image to the full size), or use the "data" parameter creatively to
34 make mostly sparse disk images.
35 The "size" parameter can specify any virtual size up to the maximum
37 supported by nbdkit (2⁶³-1 bytes).
38
40 Create small disks filled with test patterns
41 nbdkit data ' ( 0x55 0xAA )*2048 '
42 nbdkit data ' ( "Hello" )*2000 ' size=8192
43 The first command creates a disk containing 4096 bytes filled with the
45 repeating bytes 0x55 0xAA. The second command repeats
46 "HelloHelloHello...", truncating the disk to exactly 8192 bytes.
47 See also nbdkit-pattern-plugin(3).
49 Create a 1 MB disk with MBR-format partition table
51 nbdkit data '
52 @0x1be # MBR first partition entry
53 0 # Partition status
54 0 2 0 # CHS start
55 0x83 # Partition type (Linux)
56 0x20 0x20 0 # CHS last sector
57 le32:1 # LBA first sector
58 le32:0x7ff # LBA number of sectors
59 @0x1fe # Boot signature
60 0x55 0xaa
61 ' size=1M
62 A more flexible way to create partitions is to use
64 nbdkit-partitioning-plugin(1). To create a data string from an
65 existing disk use the "disk2data.pl" script provided in the nbdkit
66 sources
67 (https://gitlab.com/nbdkit/nbdkit/blob/master/plugins/data/disk2data.pl).
68 Create a disk image with sector-aligned data
70 nbdkit data ' <file1 @^512 <file2 @^512 <file3 @^512 '
71 Local binary files file1, file2 and file3 are copied into the disk
73 image. Regardless of the size of these files, they will all be aligned
74 to 512-byte sector boundaries. Furthermore because of the final
75 alignment operation ("@^512") the total size of the disk will also be
76 rounded to a whole number of sectors.
77 Create a disk with the same random data in each sector
79 nbdkit data ' </dev/urandom[:512]*16 '
80 The expression "</dev/urandom[:512]" reads 512 bytes (one sector) of
82 randomness from the system. The same random data is repeated over 16
83 sectors.
84 Create a 1 MB disk with some nonsense data at the beginning
86 nbdkit data base64=MTIz size=1M
87 The above command serves the bytes "0x31 0x32 0x33" (which is the
89 base64 decoding of "MTIz"), followed by 1M - 3 bytes of zeroes.
90 "Hello, world" using this plugin
92 $ nbdkit data raw='Hello, world!' --run 'nbdcopy "$uri" - | cat'
93 Hello, world!
94 This works by creating a disk containing the string "Hello, world!".
96 nbdcopy(1) connects to the server using an NBD URI ("$uri") and copies
97 the disk to stdout ("-"). The extra cat(1) is needed because nbdcopy
98 refuses to write raw disk data to a terminal.
99
101 Exactly one of the "data", "base64" or "raw" parameters must be
102 supplied.
103 [data=]DATA
105 Specify the disk data using a simple compact format. See "DATA
106 FORMAT" below.
107 "data=" is a magic config key and may be omitted in most cases.
109 See "Magic parameters" in nbdkit(1).
110 base64=BASE64
112 The "base64" parameter can be used to supply binary data encoded in
113 base64 on the command line.
114 This is only supported if nbdkit was compiled with GnuTLS ≥ 3.6.0.
116 You can find out by checking if:
117 $ nbdkit data --dump-plugin
119 contains:
121 data_base64=yes
123 raw=BINARY
125 The "raw" parameter can be used to supply raw binary data directly
126 on the command line.
127 It is usually quite difficult to do this unless you are running
129 nbdkit from another program (see nbdkit-captive(1)). One
130 particular problem is that the data must not contain zero bytes
131 (ie. "\0") since those will be processed in C to mean the end of
132 the string. In almost all cases it is better to use base64
133 encoding or the custom "data" format.
134 size=SIZE
136 The data is truncated or extended to the size specified.
137 This parameter is optional: If omitted the size is defined by the
139 size of the "data", "raw" or "base64" parameter.
140 allocator=sparse
142 allocator=malloc[,mlock=true]
143 allocator=zstd
144 (nbdkit ≥ 1.22)
145 Select the backend allocation strategy. See "ALLOCATORS" in
147 nbdkit-memory-plugin(1). The default is sparse.
148
150 The "data" parameter lets you specify small disk images in a simple,
151 compact format. It is a string containing a list of bytes which are
152 written into the disk image sequentially. You can move the virtual
153 offset where bytes are written using @offset.
154 nbdkit data '0 1 2 3 @0x1fe 0x55 0xaa'
156 creates:
158 total size 0x200 = 512 bytes (1 sector)
160 ┌──────┬──────┬──────┬──────┬───────── ── ── ───┬──────┬──────┐
161 │ 0 │ 1 │ 2 │ 3 │ 0 0 ... 0 │ 0x55 │ 0xaa │
162 └──────┴──────┴──────┴──────┴───────── ── ── ───┴──────┴──────┘
163 ↑
164 offset 0x1fe
165 In this example the size is implied by the data. But you could also
167 use the "size" parameter to either truncate or extend (with zeroes) the
168 disk image. Another way to write the same disk would be this, where we
169 align the offset to the end of the sector and move back 2 bytes to
170 write the signature:
171 nbdkit data '0 1 2 3 @^0x200 @-2 le16:0xaa55'
173 Fields in the string can be:
175 BYTE
177 Write a byte at the current offset and advance the offset by 1.
178 The byte may be specified as either decimal, octal (prefixed by 0)
179 or hexadecimal (prefixed by "0x"). To add repeated bytes use the
180 "*" operator (eg. "0xFF*16").
181 le16:WORD
183 be16:WORD
184 le32:WORD
185 be32:WORD
186 le64:WORD
187 be64:WORD
188 (nbdkit ≥ 1.28)
189 Write a word expressed in little endian (le) or big endian (be) of
191 length 16/32/64 bits and advance the offset by 2/4/8. The word may
192 be specified in decimal, octal or hexadecimal. For example:
193 nbdkit data ' be32:0x1 '
195 generates the 4 byte sequence "0 0 0 1".
197 @OFFSET
199 Moves the current offset to "OFFSET". The offset may be specified
200 as either decimal, octal (prefixed by 0) or hexadecimal (prefixed
201 by "0x"). Offset @0 is the first byte of the disk.
202 @+N
204 @-N (nbdkit ≥ 1.22)
205 Add or subtract "N" from the current offset.
207 @^ALIGNMENT
209 (nbdkit ≥ 1.22)
210 If the current offset is not a multiple of "ALIGNMENT" then the
212 offset is moved forward to the next multiple. The next byte
213 written will be aligned to "ALIGNMENT".
214 <FILE
216 (nbdkit ≥ 1.8)
217 Read the contents of binary FILE into the disk image at the current
219 offset. The offset is incremented by the size of the file. The
220 filename can be a relative or absolute path, but cannot contain
221 whitespace in the name.
222 <(SCRIPT)
224 (nbdkit ≥ 1.24, not Windows)
225 Substitute the output of the shell script or external program as a
227 binary blob and advance the offset by the length in bytes of the
228 output. You can use this to create more complex test patterns.
229 For example this produces a 32K disk image with an incrementing
230 test pattern in groups of 4 bytes:
231 nbdkit data ' <( i=0
233 while :; do
234 printf "%04d" $i; i=$((i+1))
235 done )[:32768] '
236 The script may contain "(" and ")" characters, but they must be in
238 matching pairs. A script can produce a finite amount of output; or
239 (as in the example) an infinite amount which must be truncated
240 using the "[:len]" slice operator.
241 Scripts must be idempotent, producing the same output each time
243 they are run. This is because optimizations might change the order
244 of evaluation or number of times the script is called and you could
245 get different output in a future version of nbdkit.
246 Note that the script is passed to /bin/sh. On some platforms like
248 Debian this might not be a full-featured shell.
249 "STRING"
251 (nbdkit ≥ 1.22)
252 Write a string into the image at the current offset and advance the
254 offset by the length of the string. To include special characters
255 in the string you can escape them in the same way as C strings (eg.
256 a double quote character within the string should be written "\"").
257 Be careful with shell quoting around the whole data parameter.
258 ( ... )
260 (nbdkit ≥ 1.24)
261 Group a set of expressions into a single expression.
263 "( ... )" recursively creates a new data parser so any expression
265 can appear inside, including nested "( ... )". Note that offsets
266 and alignments within the subpattern are relative to the start of
267 the subpattern, not relative to the final disk image.
268 expression * N
270 (nbdkit ≥ 1.24)
271 Repeat the expression "N" times. The offset is incremented by the
273 length of the expression × N. For example to create a repeating
274 pattern of 0x55, 0xAA for 512 (2×256) bytes do:
275 nbdkit data '( 0x55 0xAA ) * 256'
277 expression [N:M]
279 (nbdkit ≥ 1.24)
280 Take a slice of the expression. Slices are [start:end+1] where
282 start and end are the first and last byte offsets of the expression
283 desired. Either or both may be omitted. [:len] means to take the
284 first len bytes. [start:] means to take bytes from offset start to
285 the end of the expression.
286 expression -> \NAME
288 \NAME
289 (nbdkit ≥ 1.24)
290 Assign an expression to a name which can be used later. Names can
292 be used in the current scope (or any scopes nested within the
293 current scope), but disappear at the end of the current scope.
294 Names start with a backslash character followed by one or more
295 alphanumeric, dash and underscore. For example this makes two
296 identical sectors both containing a boot signature at the end:
297 nbdkit data ' ( 0x55 0xAA ) -> \boot-signature
299 ( @0x1fe \boot-signature ) -> \sector
300 \sector \sector '
301 $VAR
303 (nbdkit ≥ 1.24)
304 Substitute command line parameters or environment variables. The
306 variable is written in the same language as the "data" parameter,
307 and when substituted it creates a nested scope like "( ... )"
308 expressions. These are all equivalent:
309 nbdkit data '$pattern*16' pattern='0x55 0xAA'
311 export pattern='0x55 0xAA'
313 nbdkit data '$pattern*16'
314 nbdkit data '( 0x55 0xAA )*16'
316 # COMMENT
318 (nbdkit ≥ 1.24)
319 "#" begins a comment stretching to the end of the current line.
321 disk2data.pl script
323 This script can convert from small disk images into the data format
324 described above.
325 It is provided in the nbdkit sources. See
327 https://gitlab.com/nbdkit/nbdkit/blob/master/plugins/data/disk2data.pl
328
330 $plugindir/nbdkit-data-plugin.so
331 The plugin.
332 Use "nbdkit --dump-config" to find the location of $plugindir.
334
336 "nbdkit-data-plugin" first appeared in nbdkit 1.6.
337
339 nbdkit(1), nbdkit-captive(1), nbdkit-plugin(3), nbdkit-info-plugin(1),
340 nbdkit-memory-plugin(1), nbdkit-null-plugin(1), nbdkit-ones-plugin(1),
341 nbdkit-partitioning-plugin(1), nbdkit-pattern-plugin(1),
342 nbdkit-random-plugin(1), nbdkit-sparse-random-plugin(1),
343 nbdkit-tmpdisk-plugin(1), nbdkit-zero-plugin(1),
344 https://gitlab.com/nbdkit/nbdkit/blob/master/plugins/data/disk2data.pl,
345 https://en.wikipedia.org/wiki/Base64.
346
348 Richard W.M. Jones
349
351 Copyright Red Hat
352
354 Redistribution and use in source and binary forms, with or without
355 modification, are permitted provided that the following conditions are
356 met:
357 • Redistributions of source code must retain the above copyright
359 notice, this list of conditions and the following disclaimer.
360 • Redistributions in binary form must reproduce the above copyright
362 notice, this list of conditions and the following disclaimer in the
363 documentation and/or other materials provided with the
364 distribution.
365 • Neither the name of Red Hat nor the names of its contributors may
367 be used to endorse or promote products derived from this software
368 without specific prior written permission.
369 THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND ANY
371 EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
372 IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
373 PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR CONTRIBUTORS BE
374 LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
375 CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
376 SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
377 BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
378 WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
379 OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
380 ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
381nbdkit-1.36.2 2023-11-26 nbdkit-data-plugin(1)