1LVMCACHE(7)                                                        LVMCACHE(7)
2
3
4

NAME

6       lvmcache — LVM caching
7
8

DESCRIPTION

10       lvm(8)  includes  two  kinds of caching that can be used to improve the
11       performance of a Logical Volume  (LV).  Typically,  a  smaller,  faster
12       device is used to improve i/o performance of a larger, slower LV. To do
13       this, a separate LV is created from the faster  device,  and  then  the
14       original LV is converted to start using the fast LV.
15
16       The two kinds of caching are:
17
18
19       · A  read  and  write hot-spot cache, using the dm-cache kernel module.
20         This cache is slow moving, and adjusts the cache content over time so
21         that  the  most  used  parts of the LV are kept on the faster device.
22         Both reads and writes use the cache. LVM refers to this using the  LV
23         type cache.
24
25
26       · A streaming write cache, using the dm-writecache kernel module.  This
27         cache is intended to be used with SSD or PMEM devices to speed up all
28         writes  to  an  LV.  Reads do not use this cache.  LVM refers to this
29         using the LV type writecache.
30
31

USAGE

33       Both kinds of caching use similar lvm commands:
34
35       1. Identify main LV that needs caching
36
37       A main LV exists on slower devices.
38
39         $ lvcreate -n main -L Size vg /dev/slow
40
41       2. Identify fast LV to use as the cache
42
43       A fast LV exists on faster devices.  This LV will be used to  hold  the
44       cache.
45
46         $ lvcreate -n fast -L Size vg /dev/fast
47
48         $ lvs vg -o+devices
49         LV   VG  Attr       LSize   Devices
50         fast vg -wi-------  xx.00m /dev/fast(0)
51         main vg -wi------- yyy.00m /dev/slow(0)
52
53       3. Start caching the main LV
54
55       To  start caching the main LV using the fast LV, convert the main LV to
56       the desired caching type, and specify the fast LV to use:
57
58       using dm-cache:
59
60         $ lvconvert --type cache --cachepool fast vg/main
61
62       or dm-writecache:
63
64         $ lvconvert --type writecache --cachepool fast vg/main
65
66       4. Display LVs
67
68       Once the fast LV has been attached to the main LV, lvm reports the main
69       LV  type  as  either  cache  or  writecache depending on the type used.
70       While attached, the fast LV is hidden, and only displayed when  lvs  is
71       given  -a.  The _corig or _wcorig LV represents the original LV without
72       the cache.
73
74       using dm-cache:
75
76         $ lvs -a -o name,vgname,lvattr,origin,segtype,devices vg
77         LV           VG Attr       Origin       Type   Devices
78         [fast]       vg Cwi-aoC---                     linear /dev/fast(xx)
79         main         vg Cwi-a-C--- [main_corig] cache  main_corig(0)
80         [main_corig] vg owi-aoC---                     linear /dev/slow(0)
81
82       or dm-writecache:
83
84         $ lvs -a -o name,vgname,lvattr,origin,segtype,devices vg
85         LV            VG Attr       Origin        Type       Devices
86         [fast]        vg -wi-ao----               linear     /dev/fast(xx)
87         main          vg Cwi-a----- [main_wcorig] writecache main_wcorig(0)
88         [main_wcorig] vg -wi-ao----               linear     /dev/slow(0)
89
90       5. Use the main LV
91
92       Use the LV until the cache is no longer wanted, or needs to be changed.
93
94       6. Stop caching
95
96       To stop caching the main LV, separate the fast LV  from  the  main  LV.
97       This  changes  the  type  of the main LV back to what it was before the
98       cache was attached.
99
100          $ lvconvert --splitcache vg/main
101
102
103

OPTIONS

105
106
107
108   dm-writecache block size
109
110
111       The dm-writecache block size can be 4096 bytes (the  default),  or  512
112       bytes.   The  default  4096  has  better performance and should be used
113       except when 512 is  necessary  for  compatibility.   The  dm-writecache
114       block  size  is  specified  with  --writecacheblocksize 4096b|512b when
115       caching is started.
116
117       When a file system like xfs already exists on  the  main  LV  prior  to
118       caching,  and  the  file  system is using a block size of 512, then the
119       writecache block size should be set to  512.   (The  file  system  will
120       likely  fail  to mount if writecache block size of 4096 is used in this
121       case.)
122
123       Check the xfs sector size while the fs is mounted:
124
125       $ xfs_info /dev/vg/main
126       Look for sectsz=512 or sectsz=4096
127
128       The writecache block size should be chosen  to  match  the  xfs  sectsz
129       value.
130
131       It  is  also possible to specify a sector size of 4096 to mkfs.xfs when
132       creating the file system.  In this case the writecache  block  size  of
133       4096 can be used.
134
135
136   dm-writecache settings
137
138
139       Tunable  parameters  can  be  passed to the dm-writecache kernel module
140       using the --cachesettings option when caching is started, e.g.
141
142       $ lvconvert --type writecache --cachepool fast \
143            --cachesettings 'high_watermark=N writeback_jobs=N' vg/main
144
145       Tunable options are:
146
147
148       · high_watermark = <count>
149
150         Start writeback when the number of used blocks reach this watermark
151
152
153       · low_watermark = <count>
154
155         Stop writeback when the number of used blocks drops below this water‐
156         mark
157
158
159       · writeback_jobs = <count>
160
161         Limit the number of blocks that are in flight during writeback.  Set‐
162         ting this value reduces writeback  throughput,  but  it  may  improve
163         latency of read requests.
164
165
166       · autocommit_blocks = <count>
167
168         When the application writes this amount of blocks without issuing the
169         FLUSH request, the blocks are automatically commited.
170
171
172       · autocommit_time = <milliseconds>
173
174         The data is automatically commited if this time passes and  no  FLUSH
175         request is received.
176
177
178       · fua = 0|1
179
180         Use the FUA flag when writing data from persistent memory back to the
181         underlying device.  Applicable only to persistent memory.
182
183
184       · nofua = 0|1
185
186         Don't use the FUA flag when writing back  data  and  send  the  FLUSH
187         request afterwards.  Some underlying devices perform better with fua,
188         some with nofua.  Testing is necessary to determine which.   Applica‐
189         ble only to persistent memory.
190
191
192
193   dm-cache with separate data and metadata LVs
194
195
196       When using dm-cache, the cache metadata and cache data can be stored on
197       separate LVs.  To do this, a "cache-pool LV" is  created,  which  is  a
198       special  LV that references two sub LVs, one for data and one for meta‐
199       data.
200
201       To create a cache-pool LV from two separate LVs:
202
203       $ lvcreate -n fastpool -L DataSize vg /dev/fast1
204       $ lvcreate -n fastpoolmeta -L MetadataSize vg /dev/fast2
205       $ lvconvert --type cache-pool --poolmetadata fastpoolmeta vg/fastpool
206
207       Then use the cache-pool LV to start caching the main LV:
208
209       $ lvconvert --type cache --cachepool fastpool vg/main
210
211       A variation of the same procedure automatically  creates  a  cache-pool
212       when  caching  is  started.   To  do  this,  use  a  standard LV as the
213       --cachepool (this will hold cache data), and use another standard LV as
214       the  --poolmetadata (this will hold cache metadata).  LVM will create a
215       cache-pool LV from the two specified LVs, and  use  the  cache-pool  to
216       start caching the main LV.
217
218       $ lvcreate -n fastpool -L DataSize vg /dev/fast1
219       $ lvcreate -n fastpoolmeta -L MetadataSize vg /dev/fast2
220       $ lvconvert --type cache --cachepool fastpool \
221            --poolmetadata fastpoolmeta vg/main
222
223
224   dm-cache cache modes
225
226
227       The  default  dm-cache  cache  mode  is  "writethrough".   Writethrough
228       ensures that any data written will be stored both in the cache  and  on
229       the  origin LV.  The loss of a device associated with the cache in this
230       case would not mean the loss of any data.
231
232       A second cache mode is  "writeback".   Writeback  delays  writing  data
233       blocks  from  the cache back to the origin LV.  This mode will increase
234       performance, but the loss of a cache device can result in lost data.
235
236       With the --cachemode option, the cache mode can be set when caching  is
237       started, or changed on an LV that is already cached.  The current cache
238       mode can be displayed with the cache_mode reporting option:
239
240       lvs -o+cache_mode VG/LV
241
242       lvm.conf(5) allocation/cache_mode
243       defines the default cache mode.
244
245       $ lvconvert --type cache --cachepool fast \
246            --cachemode writethrough vg/main
247
248
249   dm-cache chunk size
250
251
252       The size of data blocks managed by dm-cache can be specified  with  the
253       --chunksize  option  when caching is started.  The default unit is KiB.
254       The value must be a multiple of 32KiB between 32KiB and 1GiB.
255
256       Using a chunk size that is too large can result in wasteful use of  the
257       cache, in which small reads and writes cause large sections of an LV to
258       be stored in the cache.  However, choosing a chunk  size  that  is  too
259       small  can result in more overhead trying to manage the numerous chunks
260       that become mapped into the cache.  Overhead can include both excessive
261       CPU time searching for chunks, and excessive memory tracking chunks.
262
263       Command to display the chunk size:
264       lvs -o+chunksize VG/LV
265
266       lvm.conf(5) cache_pool_chunk_size
267       controls the default chunk size.
268
269       The default value is shown by:
270       lvmconfig --type default allocation/cache_pool_chunk_size
271
272
273
274   dm-cache cache policy
275
276
277       The dm-cache subsystem has additional per-LV parameters: the cache pol‐
278       icy to use, and possibly  tunable  parameters  for  the  cache  policy.
279       Three  policies  are  currently available: "smq" is the default policy,
280       "mq" is an older implementation, and "cleaner" is  used  to  force  the
281       cache to write back (flush) all cached writes to the origin LV.
282
283       The  older "mq" policy has a number of tunable parameters. The defaults
284       are chosen to be suitable for the majority of systems, but  in  special
285       circumstances, changing the settings can improve performance.
286
287       With  the  --cachepolicy  and --cachesettings options, the cache policy
288       and settings can be set when caching  is  started,  or  changed  on  an
289       existing  cached  LV  (both options can be used together).  The current
290       cache policy and settings can be displayed with  the  cache_policy  and
291       cache_settings reporting options:
292
293       lvs -o+cache_policy,cache_settings VG/LV
294
295       Change the cache policy and settings of an existing LV.
296
297       $ lvchange --cachepolicy mq --cachesettings \
298            'migration_threshold=2048 random_threshold=4' vg/main
299
300       lvm.conf(5) allocation/cache_policy
301       defines the default cache policy.
302
303       lvm.conf(5) allocation/cache_settings
304       defines the default cache settings.
305
306
307   dm-cache spare metadata LV
308
309
310       See  lvmthin(7) for a description of the "pool metadata spare" LV.  The
311       same concept is used for cache pools.
312
313
314   dm-cache metadata formats
315
316
317       There are two disk formats for dm-cache metadata.  The metadata  format
318       can  be  specified  with --cachemetadataformat when caching is started,
319       and cannot be changed.  Format 2 has better  performance;  it  is  more
320       compact,  and stores dirty bits in a separate btree, which improves the
321       speed of shutting down the cache.  With  auto,  lvm  selects  the  best
322       option provided by the current dm-cache kernel module.
323
324
325   mirrored cache device
326
327
328       The  fast LV holding the cache can be created as a raid1 mirror so that
329       it can tolerate a device failure.  (When using dm-cache  with  separate
330       data and metadata LVs, each of the sub-LVs can use raid1.)
331
332       $ lvcreate -n main -L Size vg /dev/slow
333       $ lvcreate --type raid1 -m 1 -n fast -L Size vg /dev/fast1 /dev/fast2
334       $ lvconvert --type cache --cachepool fast vg/main
335
336

SEE ALSO

338       lvm.conf(5), lvchange(8), lvcreate(8), lvdisplay(8), lvextend(8), lvre‐
339       move(8), lvrename(8),  lvresize(8),  lvs(8),  vgchange(8),  vgmerge(8),
340       vgreduce(8), vgsplit(8)
341
342
343
344Red Hat, Inc        LVM TOOLS 2.03.02(2)-RHEL8 (2019-01-04)        LVMCACHE(7)
Impressum