1LVMVDO(7) LVMVDO(7)
2
6 lvmvdo — EXPERIMENTAL LVM Virtual Data Optimizer support
7
10 VDO (which includes kvdo and vdo) is software that provides inline
11 block-level deduplication, compression, and thin provisioning capabili‐
12 ties for primary storage.
13 Deduplication is a technique for reducing the consumption of storage
15 resources by eliminating multiple copies of duplicate blocks. Compres‐
16 sion takes the individual unique blocks and shrinks them with coding
17 algorithms; these reduced blocks are then efficiently packed together
18 into physical blocks. Thin provisioning manages the mapping from LBAs
19 presented by VDO to where the data has actually been stored, and also
20 eliminates any blocks of all zeroes.
21 With deduplication, instead of writing the same data more than once
23 each duplicate block is detected and recorded as a reference to the
24 original block. VDO maintains a mapping from logical block addresses
25 (used by the storage layer above VDO) to physical block addresses (used
26 by the storage layer under VDO). After deduplication, multiple logical
27 block addresses may be mapped to the same physical block address; these
28 are called shared blocks and are reference-counted by the software.
29 With VDO's compression, multiple blocks (or shared blocks) are com‐
31 pressed with the fast LZ4 algorithm, and binned together where possible
32 so that multiple compressed blocks fit within a 4 KB block on the
33 underlying storage. Mapping from LBA is to a physical block address and
34 index within it for the desired compressed data. All compressed blocks
35 are individually reference counted for correctness.
36 Block sharing and block compression are invisible to applications using
38 the storage, which read and write blocks as they would if VDO were not
39 present. When a shared block is overwritten, a new physical block is
40 allocated for storing the new block data to ensure that other logical
41 block addresses that are mapped to the shared physical block are not
42 modified.
43 For usage of VDO with lvm(8) standard VDO userspace tools vdoformat(8)
45 and currently non-standard kernel VDO module "kvdo" needs to be
46 installed on the system.
47 The "kvdo" module implements fine-grained storage virtualization, thin
49 provisioning, block sharing, and compression; the "uds" module provides
50 memory-efficient duplicate identification. The userspace tools include
51 vdostats(8) for extracting statistics from those volumes.
52
56 VDODataLV
57 VDO data LV
58 large hidden LV with suffix _vdata created in a VG.
59 used by VDO target to store all data and metadata blocks.
60 VDOPoolLV
63 VDO pool LV
64 maintains virtual for LV(s) stored in attached VDO data LV and
65 it has same size.
66 contains VDOLV(s) (currently supports only a single VDOLV).
67 VDOLV
70 VDO LV
71 created from VDOPoolLV
72 appears blank after creation
73
76 The primary methods for using VDO with lvm2:
77 1. Create VDOPoolLV with VDOLV
80 Create an VDOPoolLV that will holds VDO data together with virtual size
81 VDOLV, that user can use. When the virtual size is not specified, then
82 such LV is created with maximum size that always fits into data volume
83 even if there cannot happen any deduplication and compression (i.e. it
84 can hold uncompressible content of /dev/urandom). When the name of
85 VDOPoolLV is not specified, it tales name from sequence of vpool0,
86 vpool1 ...
87 Note: As the performance of TRIM/Discard operation is slow for large
89 volumes of VDO type, please try to avoid sending discard requests
90 unless necessary as it may take considerable amount of time to finish
91 discard operation.
92 lvcreate --type vdo -n VDOLV -L DataSize -V LargeVirtualSize VG/VDOPoolLV
94 lvcreate --vdo -L DataSize VG
95 Example
97 # lvcreate --type vdo -n vdo0 -L 10G -V 100G vg/vdopool0
98 # mkfs.ext4 -E nodiscard /dev/vg/vdo0
99 2. Create VDOPoolLV and convert existing LV into VDODataLV
102 Convert an already created/existing LV into a volume that can hold VDO
103 data and metadata (a volume reference by VDOPoolLV). User will be
104 prompted to confirm such conversion as it is IRREVERSIBLY DESTROYING
105 content of such volume, as it's being immediately formatted by vdofor‐
106 mat(8) as VDO pool data volume. User can specify virtual size of asso‐
107 ciated VDOLV with this VDOPoolLV. When the virtual size is not speci‐
108 fied, it will set to the maximum size that can keep 100% uncompressible
109 data there.
110 lvconvert --type vdo-pool -n VDOLV -V VirtualSize VG/VDOPoolLV
112 lvconvert --vdopool VG/VDOPoolLV
113 Example
115 # lvconvert --type vdo-pool -n vdo0 -V10G vg/existinglv
116 3. Change default setting used for creating VDOPoolLV
119 VDO allows to set large variety of option. Lots of these setting can be
120 specified by lvm.conf or profile settings. User can prepare number of
121 different profiles and just specify profile file name. Check output of
122 lvmconfig --type full for detailed description of all individual vdo
123 settings.
124 Example
126 # cat <<EOF > vdo.profile
127 allocation {
128 vdo_use_compression=1
129 vdo_use_deduplication=1
130 vdo_use_metadata_hints=1
131 vdo_minimum_io_size=4096
132 vdo_block_map_cache_size_mb=128
133 vdo_block_map_period=16380
134 vdo_check_point_frequency=0
135 vdo_use_sparse_index=0
136 vdo_index_memory_size_mb=256
137 vdo_slab_size_mb=2048
138 vdo_ack_threads=1
139 vdo_bio_threads=1
140 vdo_bio_rotation=64
141 vdo_cpu_threads=2
142 vdo_hash_zone_threads=1
143 vdo_logical_threads=1
144 vdo_physical_threads=1
145 vdo_write_policy="auto"
146 vdo_max_discard=1
147 }
148 EOF
149 # lvcreate --vdo -L10G --metadataprofile vdo.profile vg/vdopool0
151 # lvcreate --vdo -L10G --config 'allocation/vdo_cpu_threads=4' vg/vdopool1
152 4. Change compression and deduplication of VDOPoolLV
155 Disable or enable compression and deduplication for VDO pool LV (the
156 volume that maintains all VDO LV(s) associated with it).
157 lvchange --compression [y|n] --deduplication [y|n] VG/VDOPoolLV
159 Example
161 # lvchange --compression n vg/vdpool0
162 # lvchange --deduplication y vg/vdpool1
163 4. Checking usage of VDOPoolLV
166 To quickly check how much data of VDOPoolLV are already consumed use
167 lvs(8). Field Data% will report how much data occupies content of vir‐
168 tual data for VDOLV and how much space is already consumed with all the
169 data and metadata blocks in VDOPoolLV. For a detailed description use
170 vdostats(8) command.
171 Note: vdostats(8) currently understands only /dev/mapper device names.
173 Example
175 # lvcreate --type vdo -L10G -V20G -n vdo0 vg/vdopool0
176 # mkfs.ext4 -E nodiscard /dev/vg/vdo0
177 # lvs -a vg
178 LV VG Attr LSize Pool Origin Data%
180 vdo0 vg vwi-a-v--- 20.00g vdopool0 0.01
181 vdopool0 vg dwi-ao---- 10.00g 30.16
182 [vdopool0_vdata] vg Dwi-ao---- 10.00g
183 # vdostats --all /dev/mapper/vg-vdopool0
185 /dev/mapper/vg-vdopool0 :
186 version : 30
187 release version : 133524
188 data blocks used : 79
189 ...
190 4. Extending VDOPoolLV size
193 Adding more space to hold VDO data and metadata can be made via exten‐
194 sion of VDODataLV with commands lvresize(8), lvextend(8).
195 Note: Size of VDOPoolLV cannot be reduced.
197 lvextend -L+AddingSize VG/VDOPoolLV
199 Example
201 # lvextend -L+50G vg/vdopool0
202 # lvresize -L300G vg/vdopool1
203 4. Extending or reducing VDOLV size
206 VDO LV can be extended or reduced as standard LV with commands lvre‐
207 size(8), lvextend(8), lvreduce(8).
208 Note: Reduction needs to process TRIM for reduced disk area to unmap
210 used data blocks from VDOPoolLV and it may take a long time.
211 lvextend -L+AddingSize VG/VDOLV
213 lvreduce -L-ReducingSize VG/VDOLV
214 Example
216 # lvextend -L+50G vg/vdo0
217 # lvreduce -L-50G vg/vdo1
218 # lvresize -L200G vg/vdo2
219 5. Component activation of VDODataLV
222 VDODataLV can be activated separately as component LV for examination
223 purposes. It activates data LV in read-only mode and cannot be modi‐
224 fied. If the VDODataLV is active as component, any upper LV using this
225 volume CANNOT be activated. User has to deactivate VDODataLV first to
226 continue to use VDOPoolLV.
227 Example
229 # lvchange -ay vg/vpool0_vdata
230 # lvchange -an vg/vpool0_vdata
231
235 1. Stacking VDO
236 User can convert/stack VDO with existing volumes.
237 2. VDO on top of raid
240 Using Raid type LV for VDO Data LV.
241 Example
243 # lvcreate --type raid1 -L 5G -n vpool vg
244 # lvconvert --type vdo-pool -V 10G vg/vpool
245 3. Caching VDODataLV, VDOPoolLV
248 Cache VDO Data LV (accepts also VDOPoolLV.
249 Example
251 # lvcreate -L 5G -V 10G -n vdo1 vg/vpool
252 # lvcreate --type cache-pool -L 1G -n cpool vg
253 # lvconvert --cache --cachepool vg/cpool vg/vpool
254 # lvconvert --uncache vg/vpool
255 3. Caching VDOLV
258 Cache VDO LV.
259 Example
261 # lvcreate -L 5G -V 10G -n vdo1 vg/vpool
262 # lvcreate --type cache-pool -L 1G -n cpool vg
263 # lvconvert --cache --cachepool vg/cpool vg/vdo1
264 # lvconvert --uncache vg/vdo1
265
271 lvm(8), lvm.conf(5), lvmconfig(8), lvcreate(8), lvconvert(8),
272 lvchange(8), lvextend(8), lvreduce(8), lvresize(8), lvremove(8),
273 lvs(8), vdo(8), vdoformat(8), vdostats(8), mkfs(8)
274Red Hat, Inc LVM TOOLS 2.03.09(2) (2020-03-26) LVMVDO(7)