1nbdkit-filter(3) nbdkit nbdkit-filter(3)
2
6 nbdkit-filter - How to write nbdkit filters
7
9 #include <nbdkit-filter.h>
10 #define THREAD_MODEL NBDKIT_THREAD_MODEL_PARALLEL
12 static int
14 myfilter_config (nbdkit_next_config *next, void *nxdata,
15 const char *key, const char *value)
16 {
17 if (strcmp (key, "myparameter") == 0) {
18 // ...
19 return 0;
20 }
21 else {
22 // pass through to next filter or plugin
23 return next (nxdata, key, value);
24 }
25 }
26 static struct nbdkit_filter filter = {
28 .name = "filter",
29 .config = myfilter_config,
30 /* etc */
31 };
32 NBDKIT_REGISTER_FILTER(filter)
34 When this has been compiled to a shared library, do:
36 nbdkit [--args ...] --filter=./myfilter.so plugin [key=value ...]
38 When debugging, use the -fv options:
40 nbdkit -fv --filter=./myfilter.so plugin [key=value ...]
42
44 One or more nbdkit filters can be placed in front of an nbdkit plugin
45 to modify the behaviour of the plugin. This manual page describes how
46 to create an nbdkit filter.
47 Filters can be used for example to limit requests to an offset/limit,
49 add copy-on-write support, or inject delays or errors (for testing).
50 Different filters can be stacked:
52 NBD ┌─────────┐ ┌─────────┐ ┌────────┐
54 client ───▶│ filter1 │───▶│ filter2 │── ─ ─ ──▶│ plugin │
55 request └─────────┘ └─────────┘ └────────┘
56 Each filter intercepts plugin functions (see nbdkit-plugin(3)) and can
58 call the next filter or plugin in the chain, modifying parameters,
59 calling before the filter function, in the middle or after. Filters
60 may even short-cut the chain. As an example, to process its own
61 parameters the filter can intercept the ".config" method:
62 static int
64 myfilter_config (nbdkit_next_config *next, void *nxdata,
65 const char *key, const char *value)
66 {
67 if (strcmp (key, "myparameter") == 0) {
68 // ...
69 // here you would handle this key, value
70 // ...
71 return 0;
72 }
73 else {
74 // pass through to next filter or plugin
75 return next (nxdata, key, value);
76 }
77 }
78 static struct nbdkit_filter filter = {
80 // ...
81 .config = myfilter_config,
82 // ...
83 };
84 The call to "next (nxdata, ...)" calls the ".config" method of the next
86 filter or plugin in the chain. In the example above any instances of
87 "myparameter=..." on the command line would not be seen by the plugin.
88 To see example filters, take a look at the source of nbdkit, in the
90 "filters" directory.
91 Filters must be written in C.
93 Unlike the plugin API where we provide a stable ABI guarantee, filters
95 only have a best effort ABI. The reason for this is because if we
96 added new methods then an old filter would not see them and so they
97 would be passed unmodified through the filter, and in some cases that
98 leads to data corruption. You should therefore not expect to
99 distribute filters separately from nbdkit, unlike plugins.
100
102 All filters should start by including this header file:
103 #include <nbdkit-filter.h>
105
107 All filters must define a thread model. See "THREADS" in
108 nbdkit-plugin(3) for a discussion of thread models.
109 The final thread model is the smallest (ie. most serialized) out of all
111 the filters and the plugin. Filters cannot alter the thread model to
112 make it larger (more parallel).
113 If possible filters should be be written to handle fully parallel
115 requests ("NBDKIT_THREAD_MODEL_PARALLEL", even multiple requests issued
116 in parallel on the same connection). This ensures that they don't slow
117 down other filters or plugins.
118
120 All filters must define and register one "struct nbdkit_filter", which
121 contains the name of the filter and pointers to plugin methods that the
122 filter wants to intercept.
123 static struct nbdkit_filter filter = {
125 .name = "filter",
126 .longname = "My Filter",
127 .description = "This is my great filter for nbdkit",
128 .config = myfilter_config,
129 /* etc */
130 };
131 NBDKIT_REGISTER_FILTER(filter)
133 The ".name" field is the name of the filter. This is the only field
135 which is required.
136
138 nbdkit-filter.h defines three function types ("nbdkit_next_config",
139 "nbdkit_next_config_complete", "nbdkit_next_open") and a structure
140 called "struct nbdkit_next_ops". These abstract the next plugin or
141 filter in the chain. There is also an opaque pointer "nxdata" which
142 must be passed along when calling these functions.
143 The filter’s ".config", ".config_complete" and ".open" methods may only
145 call the next ".config", ".config_complete" and ".open" method in the
146 chain (optionally for ".config").
147 The filter’s ".close" method is called when an old connection closed,
149 and this has no "next" parameter because it cannot be short-circuited.
150 The filter’s other methods like ".prepare", ".get_size", ".pread" etc ―
152 always called in the context of a connection ― are passed a pointer to
153 "struct nbdkit_next_ops" which contains a subset of the plugin methods
154 that can be called during a connection. It is possible for a filter to
155 issue (for example) extra read calls in response to a single ".pwrite"
156 call.
157 You can modify parameters when you call the "next" function. However
159 be careful when modifying strings because for some methods (eg.
160 ".config") the plugin may save the string pointer that you pass along.
161 So you may have to ensure that the string is not freed for the lifetime
162 of the server.
163 Note that if your filter registers a callback but in that callback it
165 doesn't call the "next" function then the corresponding method in the
166 plugin will never be called. In particular, your ".open" method, if
167 you have one, must call the ".next" method.
168
170 "struct nbdkit_filter" has some static fields describing the filter and
171 optional callback functions which can be used to intercept plugin
172 methods.
173 ".name"
175 const char *name;
176 This field (a string) is required, and must contain only ASCII
178 alphanumeric characters and be unique amongst all filters.
179 ".version"
181 const char *version;
182 Filters may optionally set a version string which is displayed in help
184 and debugging output.
185 ".longname"
187 const char *longname;
188 An optional free text name of the filter. This field is used in error
190 messages.
191 ".description"
193 const char *description;
194 An optional multi-line description of the filter.
196 ".load"
198 void load (void);
199 This is called once just after the filter is loaded into memory. You
201 can use this to perform any global initialization needed by the filter.
202 ".unload"
204 void unload (void);
205 This may be called once just before the filter is unloaded from memory.
207 Note that it's not guaranteed that ".unload" will always be called (eg.
208 the server might be killed or segfault), so you should try to make the
209 filter as robust as possible by not requiring cleanup. See also
210 "SHUTDOWN" in nbdkit-plugin(3).
211 ".config"
213 int (*config) (nbdkit_next_config *next, void *nxdata,
214 const char *key, const char *value);
215 This intercepts the plugin ".config" method and can be used by the
217 filter to parse its own command line parameters. You should try to
218 make sure that command line parameter keys that the filter uses do not
219 conflict with ones that could be used by a plugin.
220 If there is an error, ".config" should call "nbdkit_error" with an
222 error message and return "-1".
223 ".config_complete"
225 int (*config_complete) (nbdkit_next_config_complete *next, void *nxdata);
226 This intercepts the plugin ".config_complete" method and can be used to
228 ensure that all parameters needed by the filter were supplied on the
229 command line.
230 If there is an error, ".config_complete" should call "nbdkit_error"
232 with an error message and return "-1".
233 ".config_help"
235 const char *config_help;
236 This optional multi-line help message should summarize any "key=value"
238 parameters that it takes. It does not need to repeat what already
239 appears in ".description".
240 If the filter doesn't take any config parameters you should probably
242 omit this.
243 ".open"
245 void * (*open) (nbdkit_next_open *next, void *nxdata,
246 int readonly);
247 This is called when a new client connection is opened and can be used
249 to allocate any per-connection data structures needed by the filter.
250 The handle (which is not the same as the plugin handle) is passed back
251 to other filter callbacks and could be freed in the ".close" callback.
252 Note that the handle is completely opaque to nbdkit, but it must not be
254 NULL.
255 If there is an error, ".open" should call "nbdkit_error" with an error
257 message and return "NULL".
258 ".close"
260 void (*close) (void *handle);
261 This is called when the client closes the connection. It should clean
263 up any per-connection resources used by the filter.
264 ".prepare"
266 ".finalize"
267 int (*prepare) (struct nbdkit_next_ops *next_ops, void *nxdata,
268 void *handle);
269 int (*finalize) (struct nbdkit_next_ops *next_ops, void *nxdata,
270 void *handle);
271 These two methods can be used to perform any necessary operations just
273 after opening the connection (".prepare") or just before closing the
274 connection (".finalize").
275 For example if you need to scan the underlying disk to check for a
277 partition table, you could do it in your ".prepare" method (calling the
278 plugin's ".pread" method via "next_ops"). Or if you need to cleanly
279 update superblock data in the image on close you can do it in your
280 ".finalize" method (calling the plugin's ".pwrite" method). Doing
281 these things in the filter's ".open" or ".close" method is not
282 possible.
283 If there is an error, both callbacks should call "nbdkit_error" with an
285 error message and return "-1".
286 ".get_size"
288 int64_t (*get_size) (struct nbdkit_next_ops *next_ops, void *nxdata,
289 void *handle);
290 This intercepts the plugin ".get_size" method and can be used to read
292 or modify the apparent size of the block device that the NBD client
293 will see.
294 The returned size must be ≥ 0. If there is an error, ".get_size"
296 should call "nbdkit_error" with an error message and return "-1".
297 ".can_write"
299 ".can_flush"
300 ".is_rotational"
301 ".can_trim"
302 int (*can_write) (struct nbdkit_next_ops *next_ops, void *nxdata,
303 void *handle);
304 int (*can_flush) (struct nbdkit_next_ops *next_ops, void *nxdata,
305 void *handle);
306 int (*is_rotational) (struct nbdkit_next_ops *next_ops,
307 void *nxdata,
308 void *handle);
309 int (*can_trim) (struct nbdkit_next_ops *next_ops, void *nxdata,
310 void *handle);
311 These intercept the corresponding plugin methods.
313 If there is an error, the callback should call "nbdkit_error" with an
315 error message and return "-1".
316 ".pread"
318 int (*pread) (struct nbdkit_next_ops *next_ops, void *nxdata,
319 void *handle, void *buf, uint32_t count, uint64_t offset);
320 This intercepts the plugin ".pread" method and can be used to read or
322 modify data read by the plugin.
323 If there is an error (including a short read which couldn't be
325 recovered from), ".pread" should call "nbdkit_error" with an error
326 message and set "errno", then return "-1".
327 ".pwrite"
329 int (*pwrite) (struct nbdkit_next_ops *next_ops, void *nxdata,
330 void *handle,
331 const void *buf, uint32_t count, uint64_t offset);
332 This intercepts the plugin ".pwrite" method and can be used to modify
334 data written by the plugin.
335 If there is an error (including a short write which couldn't be
337 recovered from), ".pwrite" should call "nbdkit_error" with an error
338 message and set "errno", then return "-1".
339 ".flush"
341 int (*flush) (struct nbdkit_next_ops *next_ops, void *nxdata,
342 void *handle);
343 This intercepts the plugin ".flush" method and can be used to modify
345 flush requests.
346 If there is an error, ".flush" should call "nbdkit_error" with an error
348 message and set "errno", then return "-1".
349 ".trim"
351 int (*trim) (struct nbdkit_next_ops *next_ops, void *nxdata,
352 void *handle, uint32_t count, uint64_t offset);
353 This intercepts the plugin ".trim" method and can be used to modify
355 trim requests.
356 If there is an error, ".trim" should call "nbdkit_error" with an error
358 message and set "errno", then return "-1".
359 ".zero"
361 int (*zero) (struct nbdkit_next_ops *next_ops, void *nxdata,
362 void *handle, uint32_t count, uint64_t offset, int may_trim);
363 This intercepts the plugin ".zero" method and can be used to modify
365 zero requests.
366 If there is an error, ".zero" should call "nbdkit_error" with an error
368 message and set "errno", then return "-1".
369
371 Run the server with -f and -v options so it doesn't fork and you can
372 see debugging information:
373 nbdkit -fv --filter=./myfilter.so plugin [key=value [key=value [...]]]
375 To print debugging information from within the filter, call
377 "nbdkit_debug", which has the following prototype and works like
378 printf(3):
379 void nbdkit_debug (const char *fs, ...);
381 void nbdkit_vdebug (const char *fs, va_list args);
382 For convenience, "nbdkit_debug" preserves the value of "errno". Note
384 that "nbdkit_debug" only prints things when the server is in verbose
385 mode (-v option).
386
388 The filter is a "*.so" file and possibly a manual page. You can of
389 course install the filter "*.so" file wherever you want, and users will
390 be able to use it by running:
391 nbdkit --filter=/path/to/filter.so plugin [args]
393 However if the shared library has a name of the form
395 "nbdkit-name-filter.so" and if the library is installed in the
396 $filterdir directory, then users can be run it by only typing:
397 nbdkit --filter=name plugin [args]
399 The location of the $filterdir directory is set when nbdkit is compiled
401 and can be found by doing:
402 nbdkit --dump-config
404 If using the pkg-config/pkgconf system then you can also find the
406 filter directory at compile time by doing:
407 pkgconf nbdkit --variable=filterdir
409
411 nbdkit provides a pkg-config/pkgconf file called "nbdkit.pc" which
412 should be installed on the correct path when the nbdkit development
413 environment is installed. You can use this in autoconf configure.ac
414 scripts to test for the development environment:
415 PKG_CHECK_MODULES([NBDKIT], [nbdkit >= 1.2.3])
417 The above will fail unless nbdkit ≥ 1.2.3 and the header file is
419 installed, and will set "NBDKIT_CFLAGS" and "NBDKIT_LIBS" appropriately
420 for compiling filters.
421 You can also run pkg-config/pkgconf directly, for example:
423 if ! pkgconf nbdkit --exists; then
425 echo "you must install the nbdkit development environment"
426 exit 1
427 fi
428 You can also substitute the filterdir variable by doing:
430 PKG_CHECK_VAR([NBDKIT_FILTERDIR], [nbdkit], [filterdir])
432 which defines "$(NBDKIT_FILTERDIR)" in automake-generated Makefiles.
434
436 nbdkit(1), nbdkit-plugin(3).
437 Filters:
439 nbdkit-cache-filter(1), nbdkit-cow-filter(1), nbdkit-delay-filter(1),
441 nbdkit-offset-filter(1), nbdkit-partition-filter(1).
442
444 Richard W.M. Jones
445
447 Copyright (C) 2013-2018 Red Hat Inc.
448
450 Redistribution and use in source and binary forms, with or without
451 modification, are permitted provided that the following conditions are
452 met:
453 · Redistributions of source code must retain the above copyright
455 notice, this list of conditions and the following disclaimer.
456 · Redistributions in binary form must reproduce the above copyright
458 notice, this list of conditions and the following disclaimer in the
459 documentation and/or other materials provided with the
460 distribution.
461 · Neither the name of Red Hat nor the names of its contributors may
463 be used to endorse or promote products derived from this software
464 without specific prior written permission.
465 THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND ANY
467 EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
468 IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
469 PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR CONTRIBUTORS BE
470 LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
471 CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
472 SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
473 BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
474 WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
475 OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
476 ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
477nbdkit 2018-07-25 nbdkit-filter(3)