1nbdkit-multi-conn-filter(1) NBDKIT nbdkit-multi-conn-filter(1)
2
6 nbdkit-multi-conn-filter - enable, emulate or disable multi-conn
7
9 nbdkit --filter=multi-conn plugin
10 [multi-conn-mode=MODE] [multi-conn-track-dirty=LEVEL]
11 [multi-conn-exportname=BOOL]
12 [plugin-args...]
13
15 "NBD_FLAG_CAN_MULTI_CONN" ("multi-conn") is an NBD protocol feature
16 that permits multiple clients to connect to the same export
17 simultaneously, guaranteeing that flush operations are consistent
18 across connections. Specifically a sequence of getting a write
19 response, sending and waiting for a flush response, then sending a read
20 request will behave the same whether all three commands shared a single
21 connection or were split among three connections. When an NBD client
22 and server are able to negotiate this feature it can provide
23 significant performance benefits. Conversely if the feature is not
24 advertised, clients must presume that separate connections can cache
25 writes independently (so even after waiting for a flush on one
26 connection, a read on another connection may see stale data from a
27 cache). The NBD standard advises clients not to multiplex commands
28 across connections if the server does not support multi-conn.
29 nbdkit(1) plugins must normally opt in to multi-conn, after carefully
31 ensuring the implementation meets the consistency requirements. This
32 filter can emulate flush-consistent semantics across multiple
33 connections for plugins that do not advertise this feature.
34 This filter also has additional modes useful for evaluating performance
36 and correctness of client and plugin multi-conn behaviors.
37 This filter assumes that multiple connections to a plugin will
39 eventually share data, other than any caching effects. It is not
40 suitable for use with a plugin that produces completely independent
41 data per connection from the same export name. An example of a plugin
42 that must not be used with this filter is nbdkit-tmpdisk-plugin(1).
43 Additional control over the behavior of client flush commands is
45 possible by combining this filter with nbdkit-fua-filter(1). Note that
46 nbdkit-cache-filter(1) is also able to provide multi-connection flush
47 consistency, but at the expense of an extra layer of caching not needed
48 with this filter.
49
51 multi-conn-mode=auto
52 This is the default mode. The behaviour of auto is as follows:
53 • If the selected thread model is "SERIALIZE_CONNECTIONS", then
55 this filter behaves the same as disable mode.
56 • If the plugin advertises multi-conn, then this filter behaves
58 the same as plugin mode.
59 • Otherwise, this filter behaves the same as emulate mode.
61 In other words, this mode advertises multi-conn to the client
63 exactly when the plugin supports or can be made to support multiple
64 simultaneous connections.
65 multi-conn-mode=emulate
67 When emulate mode is chosen, then this filter tracks all parallel
68 connections. When a client issues a flush command over any one
69 connection (including an implied flush by a write command with the
70 FUA (force unit access) flag set), the filter then replicates that
71 flush across each connection to the plugin. The number of plugin
72 calls made by the filter can be tuned by adjusting multi-conn-
73 track-dirty.
74 This mode assumes that flushing each connection is enough to clear
76 any per-connection cached data, in order to give each connection a
77 consistent view of the image; therefore, this mode advertises
78 multi-conn to the client.
79 Note that in this mode, a client will be unable to connect if the
81 plugin lacks support for flush, as there would be no way to emulate
82 cross-connection flush consistency.
83 multi-conn-mode=disable
85 When disable mode is chosen, this filter disables advertisement of
86 multi-conn to the client, even if the plugin supports it, and does
87 not replicate flush commands across connections. This is useful
88 for testing whether a client with multiple connections properly
89 sends multiple flushes in order to overcome per-connection caching.
90 multi-conn-mode=plugin
92 When plugin mode is chosen, the filter does not change whether
93 multi-conn is advertised by the plugin, and does not replicate
94 flush commands across connections; but still honors multi-conn-
95 track-dirty for minimizing the number of flush commands passed on
96 to the plugin.
97 multi-conn-mode=unsafe
99 When unsafe mode is chosen, this filter blindly advertises multi-
100 conn to the client even if the plugin lacks support. This is
101 dangerous, and risks data corruption if the client makes
102 assumptions about flush consistency that were not actually met.
103 multi-conn-track-dirty=fast
105 When dirty tracking is set to fast, the filter tracks whether any
106 connection has caused the image to be dirty (any write, zero, or
107 trim commands since the last flush, regardless of connection); if
108 all connections are clean, a client flush command is ignored rather
109 than sent on to the plugin. In this mode, a flush action on one
110 connection marks all other connections as clean, regardless of
111 whether the filter actually advertised multi-conn, which can result
112 in less activity when a client sends multiple flushes rather than
113 taking advantage of multi-conn semantics. This is safe with
114 multi-conn-mode=emulate, but potentially unsafe with
115 multi-conn-mode=plugin when the plugin did not advertise multi-
116 conn, as it does not track whether a read may have cached stale
117 data prior to a flush.
118 multi-conn-track-dirty=connection
120 This is the default setting for multi-conn-track-dirty.
121 The filter tracks whether a given connection is dirty (any write,
123 zero, or trim commands since the last flush on the given
124 connection, and any read since the last flush on any other
125 connection); if the connection is clean, a flush command to that
126 connection (whether directly from the client, or replicated by
127 multi-conn-mode=emulate is ignored rather than sent on to the
128 plugin. This mode may result in more flush calls than
129 multi-conn-track-dirty=fast, but in turn is safe to use with
130 multi-conn-mode=plugin.
131 multi-conn-track-dirty=off
133 When dirty tracking is set to off, all flush commands from the
134 client are passed on to the plugin, regardless of whether the flush
135 would be needed for cross-connection consistency. Note that when
136 combined with multi-conn-mode=emulate, a client which disregards
137 multi-conn by flushing on each connection itself results in a
138 quadratic number of flush operations on the plugin.
139 multi-conn-exportname=false
141 The exportname switch defaults to false for safety, and causes the
142 filter to flush across all active connections regardless of the
143 export name in use by that connection when doing emulation.
144 However, when a plugin supports distinct data according to export
145 name, this behavior will penalize the performance of clients
146 visiting an unrelated export by spending time on replicated flush
147 operations not actually relevant to that export.
148 multi-conn-exportname=true
150 Setting the exportname switch to true causes the filter to only
151 synchronize flushes to connections visiting the same export name.
152 This avoids penalizing clients visiting an unrelated export name
153 (such as nbdkit-file-plugin(1) in dir= mode), but is unsafe when
154 used with a plugin that serves shared content across all
155 connections regardless of the export name requested by the client,
156 if that plugin is not already multi-conn consistent (such as
157 nbdkit-vddk-plugin(1)).
158
160 Provide consistent cross-connection flush semantics on top of a plugin
161 that lacks it natively:
162 nbdkit --filter=multi-conn vddk /absolute/path/to/file.vmdk
164 Minimize the number of expensive flush operations performed when
166 utilizing a plugin that has multi-conn consistency from a client that
167 blindly flushes across every connection:
168 nbdkit --filter=multi-conn file multi-conn-mode=plugin \
170 multi-conn-track-dirty=fast disk.img
171
173 $filterdir/nbdkit-multi-conn-filter.so
174 The filter.
175 Use "nbdkit --dump-config" to find the location of $filterdir.
177
179 "nbdkit-multi-conn-filter" first appeared in nbdkit 1.26.
180
182 nbdkit(1), nbdkit-file-plugin(1), nbdkit-vddk-plugin(1),
183 nbdkit-filter(3), nbdkit-cache-filter(1), nbdkit-fua-filter(1),
184 nbdkit-nocache-filter(1), nbdkit-noextents-filter(1),
185 nbdkit-noparallel-filter(1), nbdkit-nozero-filter(1),
186 https://github.com/NetworkBlockDevice/nbd/blob/master/doc/proto.md
187
189 Eric Blake
190
192 Copyright (C) 2018-2021 Red Hat Inc.
193
195 Redistribution and use in source and binary forms, with or without
196 modification, are permitted provided that the following conditions are
197 met:
198 • Redistributions of source code must retain the above copyright
200 notice, this list of conditions and the following disclaimer.
201 • Redistributions in binary form must reproduce the above copyright
203 notice, this list of conditions and the following disclaimer in the
204 documentation and/or other materials provided with the
205 distribution.
206 • Neither the name of Red Hat nor the names of its contributors may
208 be used to endorse or promote products derived from this software
209 without specific prior written permission.
210 THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND ANY
212 EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
213 IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
214 PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR CONTRIBUTORS BE
215 LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
216 CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
217 SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
218 BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
219 WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
220 OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
221 ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
222nbdkit-1.32.5 2023-01-03 nbdkit-multi-conn-filter(1)