1nbdkit-filter(3) NBDKIT nbdkit-filter(3)
2
6 nbdkit-filter - how to write nbdkit filters
7
9 #include <nbdkit-filter.h>
10 static int
12 myfilter_config (nbdkit_next_config *next, void *nxdata,
13 const char *key, const char *value)
14 {
15 if (strcmp (key, "myparameter") == 0) {
16 // ...
17 return 0;
18 }
19 else {
20 // pass through to next filter or plugin
21 return next (nxdata, key, value);
22 }
23 }
24 static struct nbdkit_filter filter = {
26 .name = "filter",
27 .config = myfilter_config,
28 /* etc */
29 };
30 NBDKIT_REGISTER_FILTER(filter)
32 When this has been compiled to a shared library, do:
34 nbdkit [--args ...] --filter=./myfilter.so plugin [key=value ...]
36 When debugging, use the -fv options:
38 nbdkit -fv --filter=./myfilter.so plugin [key=value ...]
40
42 One or more nbdkit filters can be placed in front of an nbdkit plugin
43 to modify the behaviour of the plugin. This manual page describes how
44 to create an nbdkit filter.
45 Filters can be used for example to limit requests to an offset/limit,
47 add copy-on-write support, or inject delays or errors (for testing).
48 Different filters can be stacked:
50 NBD ┌─────────┐ ┌─────────┐ ┌────────┐
52 client ───▶│ filter1 │───▶│ filter2 │── ─ ─ ──▶│ plugin │
53 request └─────────┘ └─────────┘ └────────┘
54 Each filter intercepts plugin functions (see nbdkit-plugin(3)) and can
56 call the next filter or plugin in the chain, modifying parameters,
57 calling before the filter function, in the middle or after. Filters
58 may even short-cut the chain. As an example, to process its own
59 parameters the filter can intercept the ".config" method:
60 static int
62 myfilter_config (nbdkit_next_config *next, void *nxdata,
63 const char *key, const char *value)
64 {
65 if (strcmp (key, "myparameter") == 0) {
66 // ...
67 // here you would handle this key, value
68 // ...
69 return 0;
70 }
71 else {
72 // pass through to next filter or plugin
73 return next (nxdata, key, value);
74 }
75 }
76 static struct nbdkit_filter filter = {
78 // ...
79 .config = myfilter_config,
80 // ...
81 };
82 The call to "next (nxdata, ...)" calls the ".config" method of the next
84 filter or plugin in the chain. In the example above any instances of
85 "myparameter=..." on the command line would not be seen by the plugin.
86 To see example filters:
88 https://gitlab.com/nbdkit/nbdkit/tree/master/filters
89 Filters must be written in C.
91 Unlike plugins, where we provide a stable ABI guarantee that permits
93 operation across version differences, filters can only be run with the
94 same version of nbdkit that they were compiled with. The reason for
95 this is two-fold: the filter API includes access to struct
96 nbdkit_next_ops that is likely to change if new callbacks are added
97 (old nbdkit cannot safely run new filters that access new methods); and
98 if we added new methods then an old filter would not see them and so
99 they would be passed unmodified through the filter, and in some cases
100 that leads to data corruption (new nbdkit cannot safely run old filters
101 unaware of new methods). Therefore, unlike plugins, you should not
102 expect to distribute filters separately from nbdkit.
103
105 All filters should start by including this header file.
106
108 All filters must define and register one "struct nbdkit_filter", which
109 contains the name of the filter and pointers to plugin methods that the
110 filter wants to intercept.
111 static struct nbdkit_filter filter = {
113 .name = "filter",
114 .longname = "My Filter",
115 .description = "This is my great filter for nbdkit",
116 .config = myfilter_config,
117 /* etc */
118 };
119 NBDKIT_REGISTER_FILTER(filter)
121 The ".name" field is the name of the filter. This is the only field
123 which is required.
124
126 nbdkit-filter.h defines some function types ("nbdkit_next_config",
127 "nbdkit_next_config_complete", "nbdkit_next_preconnect",
128 "nbdkit_next_list_exports", "nbdkit_next_default_export",
129 "nbdkit_next_open") and a structure called "struct nbdkit_next_ops".
130 These abstract the next plugin or filter in the chain. There is also
131 an opaque pointer "backend", "context" or "nxdata" which must be passed
132 along when calling these functions. The value of "backend" is stable
133 between ".after_fork", ".preconnect", ".list_exports", and
134 ".default_export", and can also be obtained by using
135 "nbdkit_context_get_backend" on the "context" parameter to ".open".
136 Meanwhile, if the filter does not use "nbdkit_context_set_next", the
138 value of "next" passed to ".prepare" has a stable lifetime that lasts
139 to the corresponding ".finalize", with all intermediate functions (such
140 as ".pread") receiving the same value for convenience. Functions where
141 "nxdata" is not reused are ".config", ".config_complete", and
142 ".get_ready", which are all called during initialization outside any
143 connections. The value of "backend" passed to ".after_fork" also
144 occurs without connections, but is shared with ".preconnect",
145 ".list_exports", and ".default_export", and can also be obtained from
146 the "context" passed to ".open", and has a lifetime that lasts to
147 ".cleanup" for use by "nbdkit_next_context_open". In turn, the value
148 of "context" passed to ".open" has a lifetime that lasts until the
149 matching ".close" for use by "nbdkit_context_get_backend" and
150 "nbdkit_context_set_next".
151 Next config, open and close
153 The filter’s ".config", ".config_complete", ".get_ready",
154 ".after_fork", ".preconnect", ".list_exports", ".default_export" and
155 ".open" methods may only call the next ".config", ".config_complete",
156 ".get_ready", ".after_fork", ".preconnect", ".list_exports",
157 ".default_export" and ".open" method in the chain (optionally for
158 ".config" and ".open").
159 The filter’s ".close" method is called when an old connection closed,
161 and this has no "next" parameter because it cannot be short-circuited.
162 "nbdkit_next"
164 The filter generally needs to call into the underlying plugin, which is
165 done via a pointer to "struct nbdkit_next_ops", also available as the
166 typedef "nbdkit_next". The most common behavior is to create a next
167 context per connection by calling the "next_open" parameter during
168 ".open", at which point the next context will be automatically provided
169 to the filter’s other methods like ".prepare", ".get_size", ".pread"
170 etc. The "nbdkit_next" struct contains a comparable set of accessors
171 to plugin methods that can be called during a connection. When using
172 automatic registration, the "next" parameter is stable between
173 ".prepare" and ".finalize", and nbdkit automatically prepares,
174 finalizes, and closes the next context at the right point in the filter
175 connection lifecycle.
176 Alternatively, the filter can manage plugin contexts manually, whether
178 to multiplex multiple client connections through a single context into
179 the plugin, or to open multiple plugin contexts to perform retries or
180 otherwise service a single client connection more efficiently. In this
181 mode of operation, the filter uses "nbdkit_next_context_open" to open a
182 plugin context using the "backend" parameter passed to ".after_fork",
183 ".preconnect", ".list_exports", ".default_export", or obtained from
184 using "nbdkit_context_get_backend" on the "context" parameter to
185 ".open". The resulting next context has a lifecycle under manual
186 control, where the filter must use "next->prepare (next)" before using
187 any other function pointers within the next context, and must reclaim
188 the memory using "next->finalize (next)" and
189 "nbdkit_next_context_close" when done. A filter using manual lifecycle
190 management may use "nbdkit_context_set_next" to associate the next
191 context into the current connection, which lets nbdkit then pass that
192 context as the "next" parameter to future connection-related functions
193 like ".pread" and take over lifecycle responsibility.
194 "nbdkit_context_get_backend"
196 "nbdkit_next_context_open"
198 "nbdkit_next_context_close"
200 "nbdkit_context_set_next"
202 nbdkit_backend *nbdkit_context_get_backend (nbdkit_context *context);
204 Obtains the backend pointer from the "context" parameter to ".open",
206 matching the backend pointer available to ".after_fork", ".preconnect",
207 ".list_exports", and ".default_export". This backend pointer has a
208 stable lifetime from the time of ".after_fork" until ".cleanup".
209 nbdkit_next *nbdkit_next_context_open (nbdkit_backend *backend,
211 int readonly, const char *exportname,
212 int shared);
213 This function attempts to open a new context into the plugin in
215 relation to the filter's current "backend". The "readonly" and
216 "exportname" parameters behave the same as documented in ".open". The
217 resulting context will be under the filter's manual lifecycle control
218 unless the filter associates it into the connection with
219 "nbdkit_context_set_next". The filter should be careful to not violate
220 any threading model restrictions of the plugin if it opens more than
221 one context.
222 If "shared" is false, this function must be called while servicing an
224 existing client connection, and the new context will share the same
225 connection details (export name, tls status, and shorter interned
226 string lifetimes) as the current connection, and thus should not be
227 used after the client connection ends. Conversely, if "shared" is
228 true, this function may be called outside of a current client
229 connection (such as during ".after_fork"), and the resulting context
230 may be freely shared among multiple client connections. In shared
231 mode, it will not be possible for the plugin to differentiate content
232 based on the client export name, the result of the plugin calling
233 nbdkit_is_tls() will depend solely whether --tls=require was on the
234 command line, the lifetime of interned strings (via
235 "nbdkit_strdup_intern" and friends) lasts for the life of the filter,
236 and the filter must take care to not expose potentially-secure
237 information from the backend to an insecure client.
238 void nbdkit_next_context_close (nbdkit_next *next);
240 This function closes a context into the plugin. If the context has
242 previously been prepared, it should first be finalized before using
243 this function. This function does not need to be called for a plugin
244 context that has been associated with the filter connection via
245 "nbdkit_context_set_next" prior to the ".close" callback.
246 nbdkit_next *nbdkit_context_set_next (nbdkit_context *context,
248 nbdkit_next *next);
249 This function associates a plugin context with the filter's current
251 connection context, given by the "context" parameter to ".open". Once
252 associated, this plugin context will be given as the "next" parameter
253 to all other connection-specific callbacks. If associated during
254 ".open", nbdkit will take care of preparing the context prior to
255 ".prepare"; if still associated before ".finalize", nbdkit will take
256 care of finalizing the context, and also for closing it. A filter may
257 also pass "NULL" for "next", to remove any association; if no plugin
258 context is associated with the connection, then filter callbacks such
259 as ".pread" will receive "NULL" for their "next" parameter.
260 This function returns the previous context that had been associated
262 with the connection prior to switching the association to "next"; this
263 result will be "NULL" if there was no previous association. The filter
264 assumes manual responsibility for any remaining lifecycle functions
265 that must be called on the returned context.
266 Using "nbdkit_next"
268 Regardless of whether the plugin context is managed automatically or
269 manually, it is possible for a filter to issue (for example) extra
270 "next->pread" calls in response to a single ".pwrite" call.
271 The "next" parameter serves two purposes: it serves as the struct to
273 access the pointers to all the plugin connection functions, and it
274 serves as the opaque data that must be passed as the first parameter to
275 those functions. For example, calling the plugin's can_flush
276 functionality would be done via
277 next->can_flush (next)
279 Note that the semantics of the functions in "struct nbdkit_next_ops"
281 are slightly different from what a plugin implements: for example, when
282 a plugin's ".pread" returns -1 on error, the error value to advertise
283 to the client is implicit (via the plugin calling "nbdkit_set_error" or
284 setting "errno"), whereas "next->pread" exposes this via an explicit
285 parameter, allowing a filter to learn or modify this error if desired.
286 Use of "next->prepare" and "next->finalize" is only needed when
288 manually managing the plugin context lifetime.
289 Other considerations
291 You can modify parameters when you call the "next" function. However
292 be careful when modifying strings because for some methods (eg.
293 ".config") the plugin may save the string pointer that you pass along.
294 So you may have to ensure that the string is not freed for the lifetime
295 of the server; you may find "nbdkit_strdup_intern" helpful for avoiding
296 a memory leak while still obeying lifecycle constraints.
297 Note that if your filter registers a callback but in that callback it
299 doesn't call the "next" function then the corresponding method in the
300 plugin will never be called. In particular, your ".open" method, if
301 you have one, must call the "next" method if you want the underlying
302 plugin to be available to all further "nbdkit_next" use.
303
305 "struct nbdkit_filter" has some static fields describing the filter and
306 optional callback functions which can be used to intercept plugin
307 methods.
308 ".name"
310 const char *name;
311 This field (a string) is required, and must contain only ASCII
313 alphanumeric characters or non-leading dashes, and be unique amongst
314 all filters.
315 ".longname"
317 const char *longname;
318 An optional free text name of the filter. This field is used in error
320 messages.
321 ".description"
323 const char *description;
324 An optional multi-line description of the filter.
326 ".load"
328 void load (void);
329 This is called once just after the filter is loaded into memory. You
331 can use this to perform any global initialization needed by the filter.
332 ".unload"
334 void unload (void);
335 This may be called once just before the filter is unloaded from memory.
337 Note that it's not guaranteed that ".unload" will always be called (eg.
338 the server might be killed or segfault), so you should try to make the
339 filter as robust as possible by not requiring cleanup. See also
340 "SHUTDOWN" in nbdkit-plugin(3).
341 ".config"
343 int (*config) (nbdkit_next_config *next, void *nxdata,
344 const char *key, const char *value);
345 This intercepts the plugin ".config" method and can be used by the
347 filter to parse its own command line parameters. You should try to
348 make sure that command line parameter keys that the filter uses do not
349 conflict with ones that could be used by a plugin.
350 If there is an error, ".config" should call "nbdkit_error" with an
352 error message and return -1.
353 ".config_complete"
355 int (*config_complete) (nbdkit_next_config_complete *next, void *nxdata);
356 This intercepts the plugin ".config_complete" method and can be used to
358 ensure that all parameters needed by the filter were supplied on the
359 command line.
360 If there is an error, ".config_complete" should call "nbdkit_error"
362 with an error message and return -1.
363 ".config_help"
365 const char *config_help;
366 This optional multi-line help message should summarize any "key=value"
368 parameters that it takes. It does not need to repeat what already
369 appears in ".description".
370 If the filter doesn't take any config parameters you should probably
372 omit this.
373 ".thread_model"
375 int (*thread_model) (void);
376 Filters may tighten (but not relax) the thread model of the plugin, by
378 defining this callback. Note that while plugins use a compile-time
379 definition of "THREAD_MODEL", filters do not need to declare a model at
380 compile time; instead, this callback is called after ".config_complete"
381 and before any connections are created. See "THREADS" in
382 nbdkit-plugin(3) for a discussion of thread models.
383 The final thread model used by nbdkit is the smallest (ie. most
385 serialized) out of all the filters and the plugin, and applies for all
386 connections. Requests for a model larger than permitted by the plugin
387 are silently ignored. It is acceptable for decisions made during
388 ".config" and ".config_complete" to determine which model to request.
389 This callback is optional; if it is not present, the filter must be
391 written to handle fully parallel requests, including when multiple
392 requests are issued in parallel on the same connection, similar to a
393 plugin requesting "NBDKIT_THREAD_MODEL_PARALLEL". This ensures the
394 filter doesn't slow down other filters or plugins.
395 If there is an error, ".thread_model" should call "nbdkit_error" with
397 an error message and return -1.
398 ".dump_plugin"
400 void (*dump_plugin) (void);
401 This optional callback is called when the
403 "nbdkit null --filter=filtername --dump-plugin" command is used. It
404 should print any additional informative "key=value" fields to stdout as
405 needed. Prefixing the keys with the name of the filter will avoid
406 conflicts.
407 ".get_ready"
409 int (*get_ready) (int thread_model);
410 This optional callback is reached if the plugin ".get_ready" method
412 succeeded (if the plugin failed, nbdkit has already exited), and can be
413 used by the filter to get ready to serve requests.
414 The "thread_model" parameter informs the filter about the final thread
416 model chosen by nbdkit after considering the results of ".thread_model"
417 of all filters in the chain after ".config_complete".
418 If there is an error, ".get_ready" should call "nbdkit_error" with an
420 error message and return -1.
421 ".after_fork"
423 int (*after_fork) (nbdkit_backend *backend);
424 This optional callback is reached after the plugin ".after_fork" method
426 has succeeded (if the plugin failed, nbdkit has already exited), and
427 can be used by the filter to start background threads. The "backend"
428 parameter is valid until ".cleanup", for creating manual contexts into
429 the backend with "nbdkit_next_context_open".
430 If there is an error, ".after_fork" should call "nbdkit_error" with an
432 error message and return -1.
433 ".cleanup"
435 int (cleanup) (nbdkit_backend *backend);
436 This optional callback is reached once after all client connections
438 have been closed, but before the underlying plugin ".cleanup" or any
439 ".unload" callbacks. It can be used by the filter to gracefully close
440 any background threads created during ".after_fork", as well as close
441 any manual contexts into "backend" previously opened with
442 "nbdkit_next_context_open".
443 Note that it's not guaranteed that ".cleanup" will always be called
445 (eg. the server might be killed or segfault), so you should try to make
446 the filter as robust as possible by not requiring cleanup. See also
447 "SHUTDOWN" in nbdkit-plugin(3).
448 ".preconnect"
450 int (*preconnect) (nbdkit_next_preconnect *next, nbdkit_backend *nxdata,
451 int readonly);
452 This intercepts the plugin ".preconnect" method and can be used to
454 filter access to the server.
455 If there is an error, ".preconnect" should call "nbdkit_error" with an
457 error message and return -1.
458 ".list_exports"
460 int (*list_exports) (nbdkit_next_list_exports *next, nbdkit_backend *nxdata,
461 int readonly, int is_tls,
462 struct nbdkit_exports *exports);
463 This intercepts the plugin ".list_exports" method and can be used to
465 filter which exports are advertised.
466 The "readonly" parameter matches what is passed to <.preconnect> and
468 ".open", and may be changed by the filter when calling into the plugin.
469 The "is_tls" parameter informs the filter whether TLS negotiation has
470 been completed by the client, but is not passed on to "next" because it
471 cannot be altered.
472 It is possible for filters to transform the exports list received back
474 from the layer below. Without error checking it would look like this:
475 myfilter_list_exports (...)
477 {
478 size_t i;
479 struct nbdkit_exports *exports2;
480 struct nbdkit_export e;
481 char *name, *desc;
482 exports2 = nbdkit_exports_new ();
484 next_list_exports (nxdata, readonly, exports);
485 for (i = 0; i < nbdkit_exports_count (exports2); ++i) {
486 e = nbdkit_get_export (exports2, i);
487 name = adjust (e.name);
488 desc = adjust (e.desc);
489 nbdkit_add_export (exports, name, desc);
490 free (name);
491 free (desc);
492 }
493 nbdkit_exports_free (exports2);
494 }
495 If there is an error, ".list_exports" should call "nbdkit_error" with
497 an error message and return -1.
498 Allocating and freeing nbdkit_exports list
500 Two functions are provided to filters only for allocating and freeing
502 the list:
503 struct nbdkit_exports *nbdkit_exports_new (void);
505 Allocates and returns a new, empty exports list.
507 On error this function can return "NULL". In this case it calls
509 "nbdkit_error" as required. "errno" will be set to a suitable value.
510 void nbdkit_exports_free (struct nbdkit_exports *);
512 Frees an existing exports list.
514 Iterating over nbdkit_exports list
516 Two functions are provided to filters only to iterate over the exports
518 in order:
519 size_t nbdkit_exports_count (const struct nbdkit_exports *);
521 Returns the number of exports in the list.
523 struct nbdkit_export {
525 char *name;
526 char *description;
527 };
528 const struct nbdkit_export nbdkit_get_export (const struct nbdkit_exports *,
529 size_t i);
530 Returns a copy of the "i"'th export.
532 ".default_export"
534 const char *default_export (nbdkit_next_default_export *next,
535 nbdkit_backend *nxdata,
536 int readonly, int is_tls)
537 This intercepts the plugin ".default_export" method and can be used to
539 alter the canonical export name used in place of the default "".
540 The "readonly" parameter matches what is passed to <.preconnect> and
542 ".open", and may be changed by the filter when calling into the plugin.
543 The "is_tls" parameter informs the filter whether TLS negotiation has
544 been completed by the client, but is not passed on to "next" because it
545 cannot be altered.
546 ".open"
548 void * (*open) (nbdkit_next_open *next, nbdkit_context *context,
549 int readonly, const char *exportname, int is_tls);
550 This is called when a new client connection is opened and can be used
552 to allocate any per-connection data structures needed by the filter.
553 The handle (which is not the same as the plugin handle) is passed back
554 to other filter callbacks and could be freed in the ".close" callback.
555 Note that the handle is completely opaque to nbdkit, but it must not be
557 NULL. If you don't need to use a handle, return
558 "NBDKIT_HANDLE_NOT_NEEDED" which is a static non-NULL pointer.
559 If there is an error, ".open" should call "nbdkit_error" with an error
561 message and return "NULL".
562 This callback is optional, but if provided, it should call "next",
564 passing "readonly" and "exportname" possibly modified according to how
565 the filter plans to use the plugin ("is_tls" is not passed, because a
566 filter cannot modify it). Typically, the filter passes the same values
567 as it received, or passes readonly=true to provide a writable layer on
568 top of a read-only backend. However, it is also acceptable to attempt
569 write access to the plugin even if this filter is readonly, such as
570 when a file system mounted read-only still requires write access to the
571 underlying device in case a journal needs to be replayed for
572 consistency as part of the mounting process.
573 The "exportname" string is only guaranteed to be available during the
575 call (different than the lifetime for the return of
576 "nbdkit_export_name" used by plugins). If the filter needs to use it
577 (other than immediately passing it down to the next layer) it must take
578 a copy, although "nbdkit_strdup_intern" is useful for this task. The
579 "exportname" and "is_tls" parameters are provided so that filters do
580 not need to use the plugin-only interfaces of "nbdkit_export_name" and
581 "nbdkit_is_tls".
582 The filter should generally call "next" as its first step, to allocate
584 from the plugin outwards, so that ".close" running from the outer
585 filter to the plugin will be in reverse. Skipping a call to "next" is
586 acceptable if the filter will not access "nbdkit_next" during any of
587 the remaining callbacks reached on the same connection. The "next"
588 function is provided for convenience; the same functionality can be
589 obtained manually (other than error checking) by using the following:
590 nbdkit_context_set_next (context, nbdkit_next_context_open
592 (nbdkit_context_get_backend (context), readonly, exportname, false));
593 The value of "context" in this call has a lifetime that lasts until the
595 counterpart ".close", and it is this value that may be passed to
596 "nbdkit_context_get_backend" to obtain the "backend" parameter used to
597 open a plugin context with "nbdkit_next_context_open", as well as the
598 "context" parameter used to associate a plugin context into the current
599 connection with "nbdkit_context_set_next".
600 ".close"
602 void (*close) (void *handle);
603 This is called when the client closes the connection. It should clean
605 up any per-connection resources used by the filter. It is called
606 beginning with the outermost filter and ending with the plugin (the
607 opposite order of ".open" if all filters call "next" first), although
608 this order technically does not matter since the callback cannot report
609 failures or access the underlying plugin.
610 ".prepare"
612 ".finalize"
613 int (*prepare) (nbdkit_next *next, void *handle, int readonly);
614 int (*finalize) (nbdkit_next *next, void *handle);
615 These two methods can be used to perform any necessary operations just
617 after opening the connection (".prepare") or just before closing the
618 connection (".finalize").
619 For example if you need to scan the underlying disk to check for a
621 partition table, you could do it in your ".prepare" method (calling the
622 plugin's ".get_size" and ".pread" methods via "next"). Or if you need
623 to cleanly update superblock data in the image on close you can do it
624 in your ".finalize" method (calling the plugin's ".pwrite" method).
625 Doing these things in the filter's ".open" or ".close" method is not
626 possible without using manual context lifecycle management.
627 For ".prepare", the value of "readonly" is the same as was passed to
629 ".open", declaring how this filter will be used.
630 Note that nbdkit performs sanity checking on requests made to the
632 underlying plugin; for example, "next->pread" cannot be called on a
633 given connection unless "next->get_size" has first been called at least
634 once in the same connection (to ensure the read requests are in
635 bounds), and "next->pwrite" further requires an earlier successful call
636 to "next->can_write". In many filters, these prerequisites will be
637 automatically called during the client negotiation phase, but there are
638 cases where a filter overrides query functions or makes I/O calls into
639 the plugin before handshaking is complete, where the filter needs to
640 make those prerequisite calls manually during ".prepare".
641 While there are "next->prepare" and "next->finalize" functions, these
643 are different from other filter methods, in that any plugin context
644 associated with the current connection (via the "next" parameter to
645 ".open", or via "nbdkit_context_set_next", is prepared and finalized
646 automatically by nbdkit, so they are only used during manual lifecycle
647 management. Prepare methods are called starting with the filter
648 closest to the plugin and proceeding outwards (matching the order of
649 ".open" if all filters call "next" before doing anything locally), and
650 only when an outer filter did not skip the "next" call during ".open".
651 Finalize methods are called in the reverse order of prepare methods,
652 with the outermost filter first (and matching the order of ".close"),
653 and only if the prepare method succeeded.
654 If there is an error, both callbacks should call "nbdkit_error" with an
656 error message and return -1. An error in ".prepare" is reported to the
657 client, but leaves the connection open (a client may try again with a
658 different export name, for example); while an error in ".finalize"
659 forces the client to disconnect.
660 ".get_size"
662 int64_t (*get_size) (nbdkit_next *next, void *handle);
663 This intercepts the plugin ".get_size" method and can be used to read
665 or modify the apparent size of the block device that the NBD client
666 will see.
667 The returned size must be ≥ 0. If there is an error, ".get_size"
669 should call "nbdkit_error" with an error message and return -1. This
670 function is only called once per connection and cached by nbdkit.
671 Similarly, repeated calls to "next->get_size" will return a cached
672 value.
673 ".export_description"
675 const char *export_description (nbdkit_next *next, void *handle);
676 This intercepts the plugin ".export_description" method and can be used
678 to read or modify the export description that the NBD client will see.
679 ".block_size"
681 int block_size (nbdkit_next *next, void *handle, uint32_t *minimum,
682 uint32_t *preferred, uint32_t *maximum);
683 This intercepts the plugin ".block_size" method and can be used to read
685 or modify the block size constraints that the NBD client will see.
686 ".can_write"
688 ".can_flush"
689 ".is_rotational"
690 ".can_trim"
691 ".can_zero"
692 ".can_fast_zero"
693 ".can_extents"
694 ".can_fua"
695 ".can_multi_conn"
696 ".can_cache"
697 int (*can_write) (nbdkit_next *next, void *handle);
698 int (*can_flush) (nbdkit_next *next, void *handle);
699 int (*is_rotational) (nbdkit_next *next, void *handle);
700 int (*can_trim) (nbdkit_next *next, void *handle);
701 int (*can_zero) (nbdkit_next *next, void *handle);
702 int (*can_fast_zero) (nbdkit_next *next, void *handle);
703 int (*can_extents) (nbdkit_next *next, void *handle);
704 int (*can_fua) (nbdkit_next *next, void *handle);
705 int (*can_multi_conn) (nbdkit_next *next, void *handle);
706 int (*can_cache) (nbdkit_next *next, void *handle);
707 These intercept the corresponding plugin methods, and control feature
709 bits advertised to the client.
710 Of note, the semantics of ".can_zero" callback in the filter are
712 slightly different from the plugin, and must be one of three success
713 values visible only to filters:
714 "NBDKIT_ZERO_NONE"
716 Completely suppress advertisement of write zero support (this can
717 only be done from filters, not plugins).
718 "NBDKIT_ZERO_EMULATE"
720 Inform nbdkit that write zeroes should immediately fall back to
721 ".pwrite" emulation without trying ".zero" (this value is returned
722 by "next->can_zero" if the plugin returned false in its
723 ".can_zero").
724 "NBDKIT_ZERO_NATIVE"
726 Inform nbdkit that write zeroes should attempt to use ".zero",
727 although it may still fall back to ".pwrite" emulation for
728 "ENOTSUP" or "EOPNOTSUPP" failures (this value is returned by
729 "next->can_zero" if the plugin returned true in its ".can_zero").
730 Remember that most of the feature check functions return merely a
732 boolean success value, while ".can_zero", ".can_fua" and ".can_cache"
733 have three success values.
734 The difference between ".can_fua" values may affect choices made in the
736 filter: when splitting a write request that requested FUA from the
737 client, if "next->can_fua" returns "NBDKIT_FUA_NATIVE", then the filter
738 should pass the FUA flag on to each sub-request; while if it is known
739 that FUA is emulated by a flush because of a return of
740 "NBDKIT_FUA_EMULATE", it is more efficient to only flush once after all
741 sub-requests have completed (often by passing "NBDKIT_FLAG_FUA" on to
742 only the final sub-request, or by dropping the flag and ending with a
743 direct call to "next->flush").
744 If there is an error, the callback should call "nbdkit_error" with an
746 error message and return -1. These functions are called at most once
747 per connection and cached by nbdkit. Similarly, repeated calls to any
748 of the "nbdkit_next" counterparts will return a cached value; by
749 calling into the plugin during ".prepare", you can ensure that later
750 use of the cached values during data commands like <.pwrite> will not
751 fail.
752 ".pread"
754 int (*pread) (nbdkit_next *next,
755 void *handle, void *buf, uint32_t count, uint64_t offset,
756 uint32_t flags, int *err);
757 This intercepts the plugin ".pread" method and can be used to read or
759 modify data read by the plugin.
760 The parameter "flags" exists in case of future NBD protocol extensions;
762 at this time, it will be 0 on input, and the filter should not pass any
763 flags to "next->pread".
764 If there is an error (including a short read which couldn't be
766 recovered from), ".pread" should call "nbdkit_error" with an error
767 message and return -1 with "err" set to the positive errno value to
768 return to the client.
769 ".pwrite"
771 int (*pwrite) (nbdkit_next *next,
772 void *handle,
773 const void *buf, uint32_t count, uint64_t offset,
774 uint32_t flags, int *err);
775 This intercepts the plugin ".pwrite" method and can be used to modify
777 data written by the plugin.
778 This function will not be called if ".can_write" returned false; in
780 turn, the filter should not call "next->pwrite" if "next->can_write"
781 did not return true.
782 The parameter "flags" may include "NBDKIT_FLAG_FUA" on input based on
784 the result of ".can_fua". In turn, the filter should only pass
785 "NBDKIT_FLAG_FUA" on to "next->pwrite" if "next->can_fua" returned a
786 positive value.
787 If there is an error (including a short write which couldn't be
789 recovered from), ".pwrite" should call "nbdkit_error" with an error
790 message and return -1 with "err" set to the positive errno value to
791 return to the client.
792 ".flush"
794 int (*flush) (nbdkit_next *next,
795 void *handle, uint32_t flags, int *err);
796 This intercepts the plugin ".flush" method and can be used to modify
798 flush requests.
799 This function will not be called if ".can_flush" returned false; in
801 turn, the filter should not call "next->flush" if "next->can_flush" did
802 not return true.
803 The parameter "flags" exists in case of future NBD protocol extensions;
805 at this time, it will be 0 on input, and the filter should not pass any
806 flags to "next->flush".
807 If there is an error, ".flush" should call "nbdkit_error" with an error
809 message and return -1 with "err" set to the positive errno value to
810 return to the client.
811 ".trim"
813 int (*trim) (nbdkit_next *next,
814 void *handle, uint32_t count, uint64_t offset,
815 uint32_t flags, int *err);
816 This intercepts the plugin ".trim" method and can be used to modify
818 trim requests.
819 This function will not be called if ".can_trim" returned false; in
821 turn, the filter should not call "next->trim" if "next->can_trim" did
822 not return true.
823 The parameter "flags" may include "NBDKIT_FLAG_FUA" on input based on
825 the result of ".can_fua". In turn, the filter should only pass
826 "NBDKIT_FLAG_FUA" on to "next->trim" if "next->can_fua" returned a
827 positive value.
828 If there is an error, ".trim" should call "nbdkit_error" with an error
830 message and return -1 with "err" set to the positive errno value to
831 return to the client.
832 ".zero"
834 int (*zero) (nbdkit_next *next,
835 void *handle, uint32_t count, uint64_t offset, uint32_t flags,
836 int *err);
837 This intercepts the plugin ".zero" method and can be used to modify
839 zero requests.
840 This function will not be called if ".can_zero" returned
842 "NBDKIT_ZERO_NONE" or "NBDKIT_ZERO_EMULATE"; in turn, the filter should
843 not call "next->zero" if "next->can_zero" returned "NBDKIT_ZERO_NONE".
844 On input, the parameter "flags" may include "NBDKIT_FLAG_MAY_TRIM"
846 unconditionally, "NBDKIT_FLAG_FUA" based on the result of ".can_fua",
847 and "NBDKIT_FLAG_FAST_ZERO" based on the result of ".can_fast_zero".
848 In turn, the filter may pass "NBDKIT_FLAG_MAY_TRIM" unconditionally,
849 but should only pass "NBDKIT_FLAG_FUA" or "NBDKIT_FLAG_FAST_ZERO" on to
850 "next->zero" if the corresponding "next->can_fua" or
851 "next->can_fast_zero" returned a positive value.
852 Note that unlike the plugin ".zero" which is permitted to fail with
854 "ENOTSUP" or "EOPNOTSUPP" to force a fallback to ".pwrite", the
855 function "next->zero" will not fail with "err" set to "ENOTSUP" or
856 "EOPNOTSUPP" unless "NBDKIT_FLAG_FAST_ZERO" was used, because otherwise
857 the fallback has already taken place.
858 If there is an error, ".zero" should call "nbdkit_error" with an error
860 message and return -1 with "err" set to the positive errno value to
861 return to the client. The filter should not fail with "ENOTSUP" or
862 "EOPNOTSUPP" unless "flags" includes "NBDKIT_FLAG_FAST_ZERO" (while
863 plugins have automatic fallback to ".pwrite", filters do not).
864 ".extents"
866 int (*extents) (nbdkit_next *next,
867 void *handle, uint32_t count, uint64_t offset, uint32_t flags,
868 struct nbdkit_extents *extents,
869 int *err);
870 This intercepts the plugin ".extents" method and can be used to modify
872 extent requests.
873 This function will not be called if ".can_extents" returned false; in
875 turn, the filter should not call "next->extents" if "next->can_extents"
876 did not return true.
877 It is possible for filters to transform the extents list received back
879 from the layer below. Without error checking it would look like this:
880 myfilter_extents (..., uint32_t count, uint64_t offset, ...)
882 {
883 size_t i;
884 struct nbdkit_extents *extents2;
885 struct nbdkit_extent e;
886 int64_t size;
887 size = next->get_size (next);
889 extents2 = nbdkit_extents_new (offset + shift, size);
890 next->extents (next, count, offset + shift, flags, extents2, err);
891 for (i = 0; i < nbdkit_extents_count (extents2); ++i) {
892 e = nbdkit_get_extent (extents2, i);
893 e.offset -= shift;
894 nbdkit_add_extent (extents, e.offset, e.length, e.type);
895 }
896 nbdkit_extents_free (extents2);
897 }
898 If there is an error, ".extents" should call "nbdkit_error" with an
900 error message and return -1 with "err" set to the positive errno value
901 to return to the client.
902 Allocating and freeing nbdkit_extents list
904 Two functions are provided to filters only for allocating and freeing
906 the map:
907 struct nbdkit_extents *nbdkit_extents_new (uint64_t start, uint64_t end);
909 Allocates and returns a new, empty extents list. The "start" parameter
911 is the start of the range described in the list, and the "end"
912 parameter is the offset of the byte beyond the end. Normally you would
913 pass in "offset" as the start and the size of the plugin as the end,
914 but for filters which adjust offsets, they should pass in the adjusted
915 offset.
916 On error this function can return "NULL". In this case it calls
918 "nbdkit_error" and/or "nbdkit_set_error" as required. "errno" will be
919 set to a suitable value.
920 void nbdkit_extents_free (struct nbdkit_extents *);
922 Frees an existing extents list.
924 Iterating over nbdkit_extents list
926 Two functions are provided to filters only to iterate over the extents
928 in order:
929 size_t nbdkit_extents_count (const struct nbdkit_extents *);
931 Returns the number of extents in the list.
933 struct nbdkit_extent {
935 uint64_t offset;
936 uint64_t length;
937 uint32_t type;
938 };
939 struct nbdkit_extent nbdkit_get_extent (const struct nbdkit_extents *,
940 size_t i);
941 Returns a copy of the "i"'th extent.
943 Reading the full extents from the plugin
945 A convenience function is provided to filters only which makes one or
947 more requests to the underlying plugin until we have a full set of
948 extents covering the region "[offset..offset+count-1]".
949 struct nbdkit_extents *nbdkit_extents_full (
951 nbdkit_next *next,
952 uint32_t count, uint64_t offset,
953 uint32_t flags, int *err);
954 Note this allocates a new "struct nbdkit_extents" which the caller must
956 free. "flags" is passed through to the underlying plugin, but
957 "NBDKIT_FLAG_REQ_ONE" is removed from the set of flags so that the
958 plugin returns as much information as possible (this is usually what
959 you want).
960 On error this function can return "NULL". In this case it calls
962 "nbdkit_error" and/or "nbdkit_set_error" as required. *err will be set
963 to a suitable value.
964 Enforcing alignment of an nbdkit_extents list
966 A convenience function is provided to filters only which makes it
968 easier to ensure that the client only encounters aligned extents.
969 int nbdkit_extents_aligned (nbdkit_next *next,
971 uint32_t count, uint64_t offset,
972 uint32_t flags, uint32_t align,
973 struct nbdkit_extents *extents, int *err);
974 Calls "next->extents" as needed until at least "align" bytes are
976 obtained, where "align" is a power of 2. Anywhere the underlying
977 plugin returns differing extents within "align" bytes, this function
978 treats that portion of the disk as a single extent with zero and sparse
979 status bits determined by the intersection of all underlying extents.
980 It is an error to call this function with "count" or "offset" that is
981 not already aligned.
982 ".cache"
984 int (*cache) (nbdkit_next *next,
985 void *handle, uint32_t count, uint64_t offset,
986 uint32_t flags, int *err);
987 This intercepts the plugin ".cache" method and can be used to modify
989 cache requests.
990 This function will not be called if ".can_cache" returned
992 "NBDKIT_CACHE_NONE" or "NBDKIT_CACHE_EMULATE"; in turn, the filter
993 should not call "next->cache" if "next->can_cache" returned
994 "NBDKIT_CACHE_NONE".
995 The parameter "flags" exists in case of future NBD protocol extensions;
997 at this time, it will be 0 on input, and the filter should not pass any
998 flags to "next->cache".
999 If there is an error, ".cache" should call "nbdkit_error" with an error
1001 message and return -1 with "err" set to the positive errno value to
1002 return to the client.
1003
1005 If there is an error in the filter itself, the filter should call
1006 "nbdkit_error" to report an error message. If the callback is involved
1007 in serving data, the explicit "err" parameter determines the error code
1008 that will be sent to the client; other callbacks should return the
1009 appropriate error indication, eg. "NULL" or -1.
1010 "nbdkit_error" has the following prototype and works like printf(3):
1012 void nbdkit_error (const char *fs, ...);
1014 void nbdkit_verror (const char *fs, va_list args);
1015 For convenience, "nbdkit_error" preserves the value of "errno", and
1017 also supports the glibc extension of a single %m in a format string
1018 expanding to strerror(errno), even on platforms that don't support that
1019 natively.
1020
1022 Run the server with -f and -v options so it doesn't fork and you can
1023 see debugging information:
1024 nbdkit -fv --filter=./myfilter.so plugin [key=value [key=value [...]]]
1026 To print debugging information from within the filter, call
1028 "nbdkit_debug", which has the following prototype and works like
1029 printf(3):
1030 void nbdkit_debug (const char *fs, ...);
1032 void nbdkit_vdebug (const char *fs, va_list args);
1033 For convenience, "nbdkit_debug" preserves the value of "errno", and
1035 also supports the glibc extension of a single %m in a format string
1036 expanding to strerror(errno), even on platforms that don't support that
1037 natively. Note that "nbdkit_debug" only prints things when the server
1038 is in verbose mode (-v option).
1039 Debug Flags
1041 Debug Flags in filters work exactly the same way as plugins. See
1042 "Debug Flags" in nbdkit-plugin(3).
1043
1045 The filter is a "*.so" file and possibly a manual page. You can of
1046 course install the filter "*.so" file wherever you want, and users will
1047 be able to use it by running:
1048 nbdkit --filter=/path/to/filter.so plugin [args]
1050 However if the shared library has a name of the form
1052 "nbdkit-name-filter.so" and if the library is installed in the
1053 $filterdir directory, then users can be run it by only typing:
1054 nbdkit --filter=name plugin [args]
1056 The location of the $filterdir directory is set when nbdkit is compiled
1058 and can be found by doing:
1059 nbdkit --dump-config
1061 If using the pkg-config/pkgconf system then you can also find the
1063 filter directory at compile time by doing:
1064 pkg-config nbdkit --variable=filterdir
1066
1068 nbdkit provides a pkg-config/pkgconf file called "nbdkit.pc" which
1069 should be installed on the correct path when the nbdkit development
1070 environment is installed. You can use this in autoconf configure.ac
1071 scripts to test for the development environment:
1072 PKG_CHECK_MODULES([NBDKIT], [nbdkit >= 1.2.3])
1074 The above will fail unless nbdkit ≥ 1.2.3 and the header file is
1076 installed, and will set "NBDKIT_CFLAGS" and "NBDKIT_LIBS" appropriately
1077 for compiling filters.
1078 You can also run pkg-config/pkgconf directly, for example:
1080 if ! pkg-config nbdkit --exists; then
1082 echo "you must install the nbdkit development environment"
1083 exit 1
1084 fi
1085 You can also substitute the filterdir variable by doing:
1087 PKG_CHECK_VAR([NBDKIT_FILTERDIR], [nbdkit], [filterdir])
1089 which defines "$(NBDKIT_FILTERDIR)" in automake-generated Makefiles.
1091
1093 Instead of using C, it is possible to write filters in C++. However
1094 for inclusion in upstream nbdkit we would generally prefer filters
1095 written in C.
1096 Filters in C++ work almost exactly like those in C, but the way you
1098 define the "nbdkit_filter" struct is slightly different:
1099 namespace {
1101 nbdkit_filter create_filter() {
1102 nbdkit_filter filter = nbdkit_filter ();
1103 filter.name = "myfilter";
1104 filter.config = myfilter_config;
1105 return filter;
1106 }
1107 }
1108 static struct nbdkit_filter filter = create_filter ();
1109 NBDKIT_REGISTER_FILTER(filter)
1110
1112 nbdkit(1), nbdkit-plugin(3).
1113 Standard filters provided by nbdkit:
1115 nbdkit-blocksize-filter(1), nbdkit-blocksize-policy-filter(1),
1117 nbdkit-cache-filter(1), nbdkit-cacheextents-filter(1),
1118 nbdkit-checkwrite-filter(1), nbdkit-cow-filter(1),
1119 nbdkit-ddrescue-filter(1), nbdkit-delay-filter(1),
1120 nbdkit-error-filter(1), nbdkit-exitlast-filter(1),
1121 nbdkit-exitwhen-filter(1), nbdkit-exportname-filter(1),
1122 nbdkit-ext2-filter(1), nbdkit-extentlist-filter(1),
1123 nbdkit-evil-filter(1), nbdkit-fua-filter(1), nbdkit-gzip-filter(1),
1124 nbdkit-ip-filter(1), nbdkit-limit-filter(1), nbdkit-log-filter(1),
1125 nbdkit-luks-filter(1), nbdkit-multi-conn-filter(1),
1126 nbdkit-nocache-filter(1), nbdkit-noextents-filter(1),
1127 nbdkit-nofilter-filter(1), nbdkit-noparallel-filter(1),
1128 nbdkit-nozero-filter(1), nbdkit-offset-filter(1),
1129 nbdkit-partition-filter(1), nbdkit-pause-filter(1),
1130 nbdkit-protect-filter(1), nbdkit-qcow2dec-filter(1),
1131 nbdkit-rate-filter(1), nbdkit-readahead-filter(1),
1132 nbdkit-retry-filter(1), nbdkit-retry-request-filter(1),
1133 nbdkit-scan-filter(1), nbdkit-stats-filter(1), nbdkit-swab-filter(1),
1134 nbdkit-tar-filter(1), nbdkit-tls-fallback-filter(1),
1135 nbdkit-truncate-filter(1), nbdkit-xz-filter(1) .
1136
1138 Eric Blake
1139 Richard W.M. Jones
1141
1143 Copyright Red Hat
1144
1146 Redistribution and use in source and binary forms, with or without
1147 modification, are permitted provided that the following conditions are
1148 met:
1149 • Redistributions of source code must retain the above copyright
1151 notice, this list of conditions and the following disclaimer.
1152 • Redistributions in binary form must reproduce the above copyright
1154 notice, this list of conditions and the following disclaimer in the
1155 documentation and/or other materials provided with the
1156 distribution.
1157 • Neither the name of Red Hat nor the names of its contributors may
1159 be used to endorse or promote products derived from this software
1160 without specific prior written permission.
1161 THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND ANY
1163 EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
1164 IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
1165 PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR CONTRIBUTORS BE
1166 LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
1167 CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
1168 SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
1169 BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
1170 WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
1171 OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
1172 ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
1173nbdkit-1.36.2 2023-11-26 nbdkit-filter(3)