1LVMRAID(7) LVMRAID(7)
2
6 lvmraid — LVM RAID
7
9 lvm(8) RAID is a way to create a Logical Volume (LV) that uses multiple
10 physical devices to improve performance or tolerate device failures.
11 In LVM, the physical devices are Physical Volumes (PVs) in a single
12 Volume Group (VG).
13 How LV data blocks are placed onto PVs is determined by the RAID level.
15 RAID levels are commonly referred to as 'raid' followed by a number,
16 e.g. raid1, raid5 or raid6. Selecting a RAID level involves making
17 tradeoffs among: physical device requirements, fault tolerance, and
18 performance. A description of the RAID levels can be found at
19 www.snia.org/sites/default/files/SNIA_DDF_Technical_Position_v2.0.pdf
20 LVM RAID uses both Device Mapper (DM) and Multiple Device (MD) drivers
22 from the Linux kernel. DM is used to create and manage visible LVM de‐
23 vices, and MD is used to place data on physical devices.
24 LVM creates hidden LVs (dm devices) layered between the visible LV and
26 physical devices. LVs in the middle layers are called sub LVs. For
27 LVM raid, a sub LV pair to store data and metadata (raid superblock and
28 write intent bitmap) is created per raid image/leg (see lvs command ex‐
29 amples below).
30
32 To create a RAID LV, use lvcreate and specify an LV type. The LV type
33 corresponds to a RAID level. The basic RAID levels that can be used
34 are: raid0, raid1, raid4, raid5, raid6, raid10.
35 lvcreate --type RaidLevel [OPTIONS] --name Name --size Size VG [PVs]
37 To display the LV type of an existing LV, run:
39 lvs -o name,segtype LV
41 (The LV type is also referred to as "segment type" or "segtype".)
43 LVs can be created with the following types:
45 raid0
47 Also called striping, raid0 spreads LV data across multiple devices in
48 units of stripe size. This is used to increase performance. LV data
49 will be lost if any of the devices fail.
50 lvcreate --type raid0 [--stripes Number --stripesize Size] VG [PVs]
52 --stripes Number
54 specifies the Number of devices to spread the LV across.
55 --stripesize Size
57 specifies the Size of each stripe in kilobytes. This is the
58 amount of data that is written to one device before moving to
59 the next.
60 PVs specifies the devices to use. If not specified, lvm will choose
62 Number devices, one for each stripe based on the number of PVs avail‐
63 able or supplied.
64 raid1
66 Also called mirroring, raid1 uses multiple devices to duplicate LV
67 data. The LV data remains available if all but one of the devices
68 fail. The minimum number of devices (i.e. sub LV pairs) required is 2.
69 lvcreate --type raid1 [--mirrors Number] VG [PVs]
71 --mirrors Number
73 specifies the Number of mirror images in addition to the origi‐
74 nal LV image, e.g. --mirrors 1 means there are two images of the
75 data, the original and one mirror image.
76 PVs specifies the devices to use. If not specified, lvm will choose
78 Number devices, one for each image.
79 raid4
81 raid4 is a form of striping that uses an extra, first device dedicated
82 to storing parity blocks. The LV data remains available if one device
83 fails. The parity is used to recalculate data that is lost from a sin‐
84 gle device. The minimum number of devices required is 3.
85 lvcreate --type raid4 [--stripes Number --stripesize Size] VG [PVs]
87 --stripes Number
89 specifies the Number of devices to use for LV data. This does
90 not include the extra device lvm adds for storing parity blocks.
91 A raid4 LV with Number stripes requires Number+1 devices. Num‐
92 ber must be 2 or more.
93 --stripesize Size
95 specifies the Size of each stripe in kilobytes. This is the
96 amount of data that is written to one device before moving to
97 the next.
98 PVs specifies the devices to use. If not specified, lvm will choose
100 Number+1 separate devices.
101 raid4 is called non-rotating parity because the parity blocks are al‐
103 ways stored on the same device.
104 raid5
106 raid5 is a form of striping that uses an extra device for storing par‐
107 ity blocks. LV data and parity blocks are stored on each device, typi‐
108 cally in a rotating pattern for performance reasons. The LV data re‐
109 mains available if one device fails. The parity is used to recalculate
110 data that is lost from a single device. The minimum number of devices
111 required is 3 (unless converting from 2 legged raid1 to reshape to more
112 stripes; see reshaping).
113 lvcreate --type raid5 [--stripes Number --stripesize Size] VG [PVs]
115 --stripes Number
117 specifies the Number of devices to use for LV data. This does
118 not include the extra device lvm adds for storing parity blocks.
119 A raid5 LV with Number stripes requires Number+1 devices. Num‐
120 ber must be 2 or more.
121 --stripesize Size
123 specifies the Size of each stripe in kilobytes. This is the
124 amount of data that is written to one device before moving to
125 the next.
126 PVs specifies the devices to use. If not specified, lvm will choose
128 Number+1 separate devices.
129 raid5 is called rotating parity because the parity blocks are placed on
131 different devices in a round-robin sequence. There are variations of
132 raid5 with different algorithms for placing the parity blocks. The de‐
133 fault variant is raid5_ls (raid5 left symmetric, which is a rotating
134 parity 0 with data restart.) See RAID5 VARIANTS below.
135 raid6
137 raid6 is a form of striping like raid5, but uses two extra devices for
138 parity blocks. LV data and parity blocks are stored on each device,
139 typically in a rotating pattern for performance reasons. The LV data
140 remains available if up to two devices fail. The parity is used to re‐
141 calculate data that is lost from one or two devices. The minimum num‐
142 ber of devices required is 5.
143 lvcreate --type raid6 [--stripes Number --stripesize Size] VG [PVs]
145 --stripes Number
147 specifies the Number of devices to use for LV data. This does
148 not include the extra two devices lvm adds for storing parity
149 blocks. A raid6 LV with Number stripes requires Number+2 de‐
150 vices. Number must be 3 or more.
151 --stripesize Size
153 specifies the Size of each stripe in kilobytes. This is the
154 amount of data that is written to one device before moving to
155 the next.
156 PVs specifies the devices to use. If not specified, lvm will choose
158 Number+2 separate devices.
159 Like raid5, there are variations of raid6 with different algorithms for
161 placing the parity blocks. The default variant is raid6_zr (raid6 zero
162 restart, aka left symmetric, which is a rotating parity 0 with data
163 restart.) See RAID6 VARIANTS below.
164 raid10
166 raid10 is a combination of raid1 and raid0, striping data across mir‐
167 rored devices. LV data remains available if one or more devices re‐
168 mains in each mirror set. The minimum number of devices required is 4.
169 lvcreate --type raid10
171 [--mirrors NumberMirrors]
172 [--stripes NumberStripes --stripesize Size]
173 VG [PVs]
174 --mirrors NumberMirrors
176 specifies the number of mirror images within each stripe. e.g.
177 --mirrors 1 means there are two images of the data, the original
178 and one mirror image.
179 --stripes NumberStripes
181 specifies the total number of devices to use in all raid1 images
182 (not the number of raid1 devices to spread the LV across, even
183 though that is the effective result). The number of devices in
184 each raid1 mirror will be NumberStripes/(NumberMirrors+1), e.g.
185 mirrors 1 and stripes 4 will stripe data across two raid1 mir‐
186 rors, where each mirror is devices.
187 --stripesize Size
189 specifies the Size of each stripe in kilobytes. This is the
190 amount of data that is written to one device before moving to
191 the next.
192 PVs specifies the devices to use. If not specified, lvm will choose
194 the necessary devices. Devices are used to create mirrors in the order
195 listed, e.g. for mirrors 1, stripes 2, listing PV1 PV2 PV3 PV4 results
196 in mirrors PV1/PV2 and PV3/PV4.
197 RAID10 is not mirroring on top of stripes, which would be RAID01, which
199 is less tolerant of device failures.
200 Configuration Options
202 There are a number of options in the LVM configuration file that affect
203 the behavior of RAID LVs. The tunable options are listed below. A de‐
204 tailed description of each can be found in the LVM configuration file
205 itself.
206 mirror_segtype_default
207 raid10_segtype_default
208 raid_region_size
209 raid_fault_policy
210 activation_mode
211 Monitoring
213 When a RAID LV is activated the dmeventd(8) process is started to moni‐
214 tor the health of the LV. Various events detected in the kernel can
215 cause a notification to be sent from device-mapper to the monitoring
216 process, including device failures and synchronization completion (e.g.
217 for initialization or scrubbing).
218 The LVM configuration file contains options that affect how the moni‐
220 toring process will respond to failure events (e.g. raid_fault_policy).
221 It is possible to turn on and off monitoring with lvchange, but it is
222 not recommended to turn this off unless you have a thorough knowledge
223 of the consequences.
224 Synchronization
226 Synchronization is the process that makes all the devices in a RAID LV
227 consistent with each other.
228 In a RAID1 LV, all mirror images should have the same data. When a new
230 mirror image is added, or a mirror image is missing data, then images
231 need to be synchronized. Data blocks are copied from an existing image
232 to a new or outdated image to make them match.
233 In a RAID 4/5/6 LV, parity blocks and data blocks should match based on
235 the parity calculation. When the devices in a RAID LV change, the data
236 and parity blocks can become inconsistent and need to be synchronized.
237 Correct blocks are read, parity is calculated, and recalculated blocks
238 are written.
239 The RAID implementation keeps track of which parts of a RAID LV are
241 synchronized. When a RAID LV is first created and activated the first
242 synchronization is called initialization. A pointer stored in the raid
243 metadata keeps track of the initialization process thus allowing it to
244 be restarted after a deactivation of the RaidLV or a crash. Any writes
245 to the RaidLV dirties the respective region of the write intent bitmap
246 which allow for fast recovery of the regions after a crash. Without
247 this, the entire LV would need to be synchronized every time it was ac‐
248 tivated.
249 Automatic synchronization happens when a RAID LV is activated, but it
251 is usually partial because the bitmaps reduce the areas that are
252 checked. A full sync becomes necessary when devices in the RAID LV are
253 replaced.
254 The synchronization status of a RAID LV is reported by the following
256 command, where "Cpy%Sync" = "100%" means sync is complete:
257 lvs -a -o name,sync_percent
259 Scrubbing
261 Scrubbing is a full scan of the RAID LV requested by a user. Scrubbing
262 can find problems that are missed by partial synchronization.
263 Scrubbing assumes that RAID metadata and bitmaps may be inaccurate, so
265 it verifies all RAID metadata, LV data, and parity blocks. Scrubbing
266 can find inconsistencies caused by hardware errors or degradation.
267 These kinds of problems may be undetected by automatic synchronization
268 which excludes areas outside of the RAID write-intent bitmap.
269 The command to scrub a RAID LV can operate in two different modes:
271 lvchange --syncaction check|repair LV
273 check Check mode is read-only and only detects inconsistent areas in
275 the RAID LV, it does not correct them.
276 repair Repair mode checks and writes corrected blocks to synchronize
278 any inconsistent areas.
279 Scrubbing can consume a lot of bandwidth and slow down application I/O
281 on the RAID LV. To control the I/O rate used for scrubbing, use:
282 --maxrecoveryrate Size[k|UNIT]
284 Sets the maximum recovery rate for a RAID LV. Size is specified
285 as an amount per second for each device in the array. If no
286 suffix is given, then KiB/sec/device is used. Setting the re‐
287 covery rate to 0 means it will be unbounded.
288 --minrecoveryrate Size[k|UNIT]
290 Sets the minimum recovery rate for a RAID LV. Size is specified
291 as an amount per second for each device in the array. If no
292 suffix is given, then KiB/sec/device is used. Setting the re‐
293 covery rate to 0 means it will be unbounded.
294 To display the current scrubbing in progress on an LV, including the
296 syncaction mode and percent complete, run:
297 lvs -a -o name,raid_sync_action,sync_percent
299 After scrubbing is complete, to display the number of inconsistent
301 blocks found, run:
302 lvs -o name,raid_mismatch_count
304 Also, if mismatches were found, the lvs attr field will display the
306 letter "m" (mismatch) in the 9th position, e.g.
307 # lvs -o name,vgname,segtype,attr vg/lv
309 LV VG Type Attr
310 lv vg raid1 Rwi-a-r-m-
311 Scrubbing Limitations
313 The check mode can only report the number of inconsistent blocks, it
314 cannot report which blocks are inconsistent. This makes it impossible
315 to know which device has errors, or if the errors affect file system
316 data, metadata or nothing at all.
317 The repair mode can make the RAID LV data consistent, but it does not
319 know which data is correct. The result may be consistent but incorrect
320 data. When two different blocks of data must be made consistent, it
321 chooses the block from the device that would be used during RAID ini‐
322 tialization. However, if the PV holding corrupt data is known,
323 lvchange --rebuild can be used in place of scrubbing to reconstruct the
324 data on the bad device.
325 Future developments might include:
327 Allowing a user to choose the correct version of data during repair.
329 Using a majority of devices to determine the correct version of data to
331 use in a 3-way RAID1 or RAID6 LV.
332 Using a checksumming device to pin-point when and where an error oc‐
334 curs, allowing it to be rewritten.
335 SubLVs
337 An LV is often a combination of other hidden LVs called SubLVs. The
338 SubLVs either use physical devices, or are built from other SubLVs
339 themselves. SubLVs hold LV data blocks, RAID parity blocks, and RAID
340 metadata. SubLVs are generally hidden, so the lvs -a option is re‐
341 quired to display them:
342 lvs -a -o name,segtype,devices
344 SubLV names begin with the visible LV name, and have an automatic suf‐
346 fix indicating its role:
347 • SubLVs holding LV data or parity blocks have the suffix _rim‐
349 age_#.
350 These SubLVs are sometimes referred to as DataLVs.
351 • SubLVs holding RAID metadata have the suffix _rmeta_#. RAID
353 metadata includes superblock information, RAID type, bitmap, and
354 device health information.
355 These SubLVs are sometimes referred to as MetaLVs.
356 SubLVs are an internal implementation detail of LVM. The way they are
358 used, constructed and named may change.
359 The following examples show the SubLV arrangement for each of the basic
361 RAID LV types, using the fewest number of devices allowed for each.
362 Examples
364 raid0
366 Each rimage SubLV holds a portion of LV data. No parity is used. No
367 RAID metadata is used.
368 # lvcreate --type raid0 --stripes 2 --name lvr0 ...
370 # lvs -a -o name,segtype,devices
372 lvr0 raid0 lvr0_rimage_0(0),lvr0_rimage_1(0)
373 [lvr0_rimage_0] linear /dev/sda(...)
374 [lvr0_rimage_1] linear /dev/sdb(...)
375 raid1
377 Each rimage SubLV holds a complete copy of LV data. No parity is used.
378 Each rmeta SubLV holds RAID metadata.
379 # lvcreate --type raid1 --mirrors 1 --name lvr1 ...
381 # lvs -a -o name,segtype,devices
383 lvr1 raid1 lvr1_rimage_0(0),lvr1_rimage_1(0)
384 [lvr1_rimage_0] linear /dev/sda(...)
385 [lvr1_rimage_1] linear /dev/sdb(...)
386 [lvr1_rmeta_0] linear /dev/sda(...)
387 [lvr1_rmeta_1] linear /dev/sdb(...)
388 raid4
390 At least three rimage SubLVs each hold a portion of LV data and one
391 rimage SubLV holds parity. Each rmeta SubLV holds RAID metadata.
392 # lvcreate --type raid4 --stripes 2 --name lvr4 ...
394 # lvs -a -o name,segtype,devices
396 lvr4 raid4 lvr4_rimage_0(0),\
397 lvr4_rimage_1(0),\
398 lvr4_rimage_2(0)
399 [lvr4_rimage_0] linear /dev/sda(...)
400 [lvr4_rimage_1] linear /dev/sdb(...)
401 [lvr4_rimage_2] linear /dev/sdc(...)
402 [lvr4_rmeta_0] linear /dev/sda(...)
403 [lvr4_rmeta_1] linear /dev/sdb(...)
404 [lvr4_rmeta_2] linear /dev/sdc(...)
405 raid5
407 At least three rimage SubLVs each typically hold a portion of LV data
408 and parity (see section on raid5) Each rmeta SubLV holds RAID metadata.
409 # lvcreate --type raid5 --stripes 2 --name lvr5 ...
411 # lvs -a -o name,segtype,devices
413 lvr5 raid5 lvr5_rimage_0(0),\
414 lvr5_rimage_1(0),\
415 lvr5_rimage_2(0)
416 [lvr5_rimage_0] linear /dev/sda(...)
417 [lvr5_rimage_1] linear /dev/sdb(...)
418 [lvr5_rimage_2] linear /dev/sdc(...)
419 [lvr5_rmeta_0] linear /dev/sda(...)
420 [lvr5_rmeta_1] linear /dev/sdb(...)
421 [lvr5_rmeta_2] linear /dev/sdc(...)
422 raid6
424 At least five rimage SubLVs each typically hold a portion of LV data
425 and parity. (see section on raid6) Each rmeta SubLV holds RAID meta‐
426 data.
427 # lvcreate --type raid6 --stripes 3 --name lvr6
429 # lvs -a -o name,segtype,devices
431 lvr6 raid6 lvr6_rimage_0(0),\
432 lvr6_rimage_1(0),\
433 lvr6_rimage_2(0),\
434 lvr6_rimage_3(0),\
435 lvr6_rimage_4(0),\
436 lvr6_rimage_5(0)
437 [lvr6_rimage_0] linear /dev/sda(...)
438 [lvr6_rimage_1] linear /dev/sdb(...)
439 [lvr6_rimage_2] linear /dev/sdc(...)
440 [lvr6_rimage_3] linear /dev/sdd(...)
441 [lvr6_rimage_4] linear /dev/sde(...)
442 [lvr6_rimage_5] linear /dev/sdf(...)
443 [lvr6_rmeta_0] linear /dev/sda(...)
444 [lvr6_rmeta_1] linear /dev/sdb(...)
445 [lvr6_rmeta_2] linear /dev/sdc(...)
446 [lvr6_rmeta_3] linear /dev/sdd(...)
447 [lvr6_rmeta_4] linear /dev/sde(...)
448 [lvr6_rmeta_5] linear /dev/sdf(...)
449 raid10
451 At least four rimage SubLVs each hold a portion of LV data. No parity
452 is used. Each rmeta SubLV holds RAID metadata.
453 # lvcreate --type raid10 --stripes 2 --mirrors 1 --name lvr10
455 # lvs -a -o name,segtype,devices
457 lvr10 raid10 lvr10_rimage_0(0),\
458 lvr10_rimage_1(0),\
459 lvr10_rimage_2(0),\
460 lvr10_rimage_3(0)
461 [lvr10_rimage_0] linear /dev/sda(...)
462 [lvr10_rimage_1] linear /dev/sdb(...)
463 [lvr10_rimage_2] linear /dev/sdc(...)
464 [lvr10_rimage_3] linear /dev/sdd(...)
465 [lvr10_rmeta_0] linear /dev/sda(...)
466 [lvr10_rmeta_1] linear /dev/sdb(...)
467 [lvr10_rmeta_2] linear /dev/sdc(...)
468 [lvr10_rmeta_3] linear /dev/sdd(...)
469
471 Physical devices in a RAID LV can fail or be lost for multiple reasons.
472 A device could be disconnected, permanently failed, or temporarily dis‐
473 connected. The purpose of RAID LVs (levels 1 and higher) is to con‐
474 tinue operating in a degraded mode, without losing LV data, even after
475 a device fails. The number of devices that can fail without the loss
476 of LV data depends on the RAID level:
477 • RAID0 (striped) LVs cannot tolerate losing any devices. LV data
478 will be lost if any devices fail.
479 • RAID1 LVs can tolerate losing all but one device without LV data
480 loss.
481 • RAID4 and RAID5 LVs can tolerate losing one device without LV
482 data loss.
483 • RAID6 LVs can tolerate losing two devices without LV data loss.
484 • RAID10 is variable, and depends on which devices are lost. It
485 stripes across multiple mirror groups with raid1 layout thus it
486 can tolerate losing all but one device in each of these groups
487 without LV data loss.
488 If a RAID LV is missing devices, or has other device-related problems,
490 lvs reports this in the health_status (and attr) fields:
491 lvs -o name,lv_health_status
493 partial
495 Devices are missing from the LV. This is also indicated by the
496 letter "p" (partial) in the 9th position of the lvs attr field.
497 refresh needed
499 A device was temporarily missing but has returned. The LV needs
500 to be refreshed to use the device again (which will usually re‐
501 quire partial synchronization). This is also indicated by the
502 letter "r" (refresh needed) in the 9th position of the lvs attr
503 field. See Refreshing an LV. This could also indicate a prob‐
504 lem with the device, in which case it should be be replaced, see
505 Replacing Devices.
506 mismatches exist
508 See Scrubbing.
509 Most commands will also print a warning if a device is missing, e.g.
511 WARNING: Device for PV uItL3Z-wBME-DQy0-... not found or rejected ...
512 This warning will go away if the device returns or is removed from the
514 VG (see vgreduce --removemissing).
515 Activating an LV with missing devices
517 A RAID LV that is missing devices may be activated or not, depending on
518 the "activation mode" used in lvchange:
519 lvchange -ay --activationmode complete|degraded|partial LV
521 complete
523 The LV is only activated if all devices are present.
524 degraded
526 The LV is activated with missing devices if the RAID level can
527 tolerate the number of missing devices without LV data loss.
528 partial
530 The LV is always activated, even if portions of the LV data are
531 missing because of the missing device(s). This should only be
532 used to perform extreme recovery or repair operations.
533 Default activation mode when not specified by the command:
535 lvm.conf(5) activation/activation_mode
536 The default value is printed by:
538 # lvmconfig --type default activation/activation_mode
539 Replacing Devices
541 Devices in a RAID LV can be replaced by other devices in the VG. When
542 replacing devices that are no longer visible on the system, use lvcon‐
543 vert --repair. When replacing devices that are still visible, use lv‐
544 convert --replace. The repair command will attempt to restore the same
545 number of data LVs that were previously in the LV. The replace option
546 can be repeated to replace multiple PVs. Replacement devices can be
547 optionally listed with either option.
548 lvconvert --repair LV [NewPVs]
550 lvconvert --replace OldPV LV [NewPV]
552 lvconvert --replace OldPV1 --replace OldPV2 LV [NewPVs]
554 New devices require synchronization with existing devices.
556 See Synchronization.
557 Refreshing an LV
559 Refreshing a RAID LV clears any transient device failures (device was
560 temporarily disconnected) and returns the LV to its fully redundant
561 mode. Restoring a device will usually require at least partial syn‐
562 chronization (see Synchronization). Failure to clear a transient fail‐
563 ure results in the RAID LV operating in degraded mode until it is reac‐
564 tivated. Use the lvchange command to refresh an LV:
565 lvchange --refresh LV
567 # lvs -o name,vgname,segtype,attr,size vg
569 LV VG Type Attr LSize
570 lv vg raid1 Rwi-a-r-r- 100.00g
571 # lvchange --refresh vg/lv
573 # lvs -o name,vgname,segtype,attr,size vg
575 LV VG Type Attr LSize
576 lv vg raid1 Rwi-a-r--- 100.00g
577 Automatic repair
579 If a device in a RAID LV fails, device-mapper in the kernel notifies
580 the dmeventd(8) monitoring process (see Monitoring). dmeventd can be
581 configured to automatically respond using:
582 lvm.conf(5) activation/raid_fault_policy
583 Possible settings are:
585 warn A warning is added to the system log indicating that a device
587 has failed in the RAID LV. It is left to the user to repair the
588 LV, e.g. replace failed devices.
589 allocate
591 dmeventd automatically attempts to repair the LV using spare de‐
592 vices in the VG. Note that even a transient failure is treated
593 as a permanent failure under this setting. A new device is al‐
594 located and full synchronization is started.
595 The specific command run by dmeventd(8) to warn or repair is:
597 lvconvert --repair --use-policies LV
598 Corrupted Data
600 Data on a device can be corrupted due to hardware errors without the
601 device ever being disconnected or there being any fault in the soft‐
602 ware. This should be rare, and can be detected (see Scrubbing).
603 Rebuild specific PVs
605 If specific PVs in a RAID LV are known to have corrupt data, the data
606 on those PVs can be reconstructed with:
607 lvchange --rebuild PV LV
609 The rebuild option can be repeated with different PVs to replace the
611 data on multiple PVs.
612
614 The device mapper integrity target can be used in combination with RAID
615 levels 1,4,5,6,10 to detect and correct data corruption in RAID images.
616 A dm-integrity layer is placed above each RAID image, and an extra sub
617 LV is created to hold integrity metadata (data checksums) for each RAID
618 image. When data is read from an image, integrity checksums are used
619 to detect corruption. If detected, dm-raid reads the data from another
620 (good) image to return to the caller. dm-raid will also automatically
621 write the good data back to the image with bad data to correct the cor‐
622 ruption.
623 When creating a RAID LV with integrity, or adding integrity, space is
625 required for integrity metadata. Every 500MB of LV data requires an
626 additional 4MB to be allocated for integrity metadata, for each RAID
627 image.
628 Create a RAID LV with integrity:
630 lvcreate --type raidN --raidintegrity y
631 Add integrity to an existing RAID LV:
633 lvconvert --raidintegrity y LV
634 Remove integrity from a RAID LV:
636 lvconvert --raidintegrity n LV
637 Integrity options
639 --raidintegritymode journal|bitmap
640 Use a journal (default) or bitmap for keeping integrity check‐
641 sums consistent in case of a crash. The bitmap areas are recal‐
642 culated after a crash, so corruption in those areas would not be
643 detected. A journal does not have this problem. The journal
644 mode doubles writes to storage, but can improve performance for
645 scattered writes packed into a single journal write. bitmap
646 mode can in theory achieve full write throughput of the device,
647 but would not benefit from the potential scattered write opti‐
648 mization.
649 --raidintegrityblocksize 512|1024|2048|4096
651 The block size to use for dm-integrity on raid images. The in‐
652 tegrity block size should usually match the device logical block
653 size, or the file system sector/block sizes. It may be less
654 than the file system sector/block size, but not less than the
655 device logical block size. Possible values: 512, 1024, 2048,
656 4096.
657 Integrity initialization
659 When integrity is added to an LV, the kernel needs to initialize the
660 integrity metadata (checksums) for all blocks in the LV. The data cor‐
661 ruption checking performed by dm-integrity will only operate on areas
662 of the LV that are already initialized. The progress of integrity ini‐
663 tialization is reported by the "syncpercent" LV reporting field (and
664 under the Cpy%Sync lvs column.)
665 Integrity limitations
667 To work around some limitations, it is possible to remove integrity
668 from the LV, make the change, then add integrity again. (Integrity
669 metadata would need to initialized when added again.)
670 LVM must be able to allocate the integrity metadata sub LV on a single
672 PV that is already in use by the associated RAID image. This can poten‐
673 tially cause a problem during lvextend if the original PV holding the
674 image and integrity metadata is full. To work around this limitation,
675 remove integrity, extend the LV, and add integrity again.
676 Additional RAID images can be added to raid1 LVs, but not to other raid
678 levels.
679 A raid1 LV with integrity cannot be converted to linear (remove integ‐
681 rity to do this.)
682 RAID LVs with integrity cannot yet be used as sub LVs with other LV
684 types.
685 The following are not yet permitted on RAID LVs with integrity: lvre‐
687 duce, pvmove, lvconvert --splitmirrors, lvchange --syncaction, lvchange
688 --rebuild.
689
691 A RAID1 LV can be tuned so that certain devices are avoided for reading
692 while all devices are still written to.
693 lvchange --[raid]writemostly PV[:y|n|t] LV
695 The specified device will be marked as "write mostly", which means that
697 reading from this device will be avoided, and other devices will be
698 preferred for reading (unless no other devices are available.) This
699 minimizes the I/O to the specified device.
700 If the PV name has no suffix, the write mostly attribute is set. If
702 the PV name has the suffix :n, the write mostly attribute is cleared,
703 and the suffix :t toggles the current setting.
704 The write mostly option can be repeated on the command line to change
706 multiple devices at once.
707 To report the current write mostly setting, the lvs attr field will
709 show the letter "w" in the 9th position when write mostly is set:
710 lvs -a -o name,attr
712 When a device is marked write mostly, the maximum number of outstanding
714 writes to that device can be configured. Once the maximum is reached,
715 further writes become synchronous. When synchronous, a write to the LV
716 will not complete until writes to all the mirror images are complete.
717 lvchange --[raid]writebehind Number LV
719 To report the current write behind setting, run:
721 lvs -o name,raid_write_behind
723 When write behind is not configured, or set to 0, all LV writes are
725 synchronous.
726
728 RAID takeover is converting a RAID LV from one RAID level to another,
729 e.g. raid5 to raid6. Changing the RAID level is usually done to in‐
730 crease or decrease resilience to device failures or to restripe LVs.
731 This is done using lvconvert and specifying the new RAID level as the
732 LV type:
733 lvconvert --type RaidLevel LV [PVs]
735 The most common and recommended RAID takeover conversions are:
737 linear to raid1
739 Linear is a single image of LV data, and converting it to raid1
740 adds a mirror image which is a direct copy of the original lin‐
741 ear image.
742 striped/raid0 to raid4/5/6
744 Adding parity devices to a striped volume results in raid4/5/6.
745 Unnatural conversions that are not recommended include converting be‐
747 tween striped and non-striped types. This is because file systems of‐
748 ten optimize I/O patterns based on device striping values. If those
749 values change, it can decrease performance.
750 Converting to a higher RAID level requires allocating new SubLVs to
752 hold RAID metadata, and new SubLVs to hold parity blocks for LV data.
753 Converting to a lower RAID level removes the SubLVs that are no longer
754 needed.
755 Conversion often requires full synchronization of the RAID LV (see Syn‐
757 chronization). Converting to RAID1 requires copying all LV data blocks
758 to N new images on new devices. Converting to a parity RAID level re‐
759 quires reading all LV data blocks, calculating parity, and writing the
760 new parity blocks. Synchronization can take a long time depending on
761 the throughpout of the devices used and the size of the RaidLV. It can
762 degrade performance. Rate controls also apply to conversion; see --min‐
763 recoveryrate and --maxrecoveryrate.
764 Warning: though it is possible to create striped LVs with up to 128
766 stripes, a maximum of 64 stripes can be converted to raid0, 63 to
767 raid4/5 and 62 to raid6 because of the added parity SubLVs. A striped
768 LV with a maximum of 32 stripes can be converted to raid10.
769 The following takeover conversions are currently possible:
771 • between striped and raid0.
772 • between linear and raid1.
773 • between mirror and raid1.
774 • between raid1 with two images and raid4/5.
775 • between striped/raid0 and raid4.
776 • between striped/raid0 and raid5.
777 • between striped/raid0 and raid6.
778 • between raid4 and raid5.
779 • between raid4/raid5 and raid6.
780 • between striped/raid0 and raid10.
781 • between striped and raid4.
782 Indirect conversions
784 Converting from one raid level to another may require multiple steps,
785 converting first to intermediate raid levels.
786 linear to raid6
788 To convert an LV from linear to raid6:
790 1. convert to raid1 with two images
791 2. convert to raid5 (internally raid5_ls) with two images
792 3. convert to raid5 with three or more stripes (reshape)
793 4. convert to raid6 (internally raid6_ls_6)
794 5. convert to raid6 (internally raid6_zr, reshape)
795 The commands to perform the steps above are:
797 1. lvconvert --type raid1 --mirrors 1 LV
798 2. lvconvert --type raid5 LV
799 3. lvconvert --stripes 3 LV
800 4. lvconvert --type raid6 LV
801 5. lvconvert --type raid6 LV
802 The final conversion from raid6_ls_6 to raid6_zr is done to avoid the
804 potential write/recovery performance reduction in raid6_ls_6 because of
805 the dedicated parity device. raid6_zr rotates data and parity blocks
806 to avoid this.
807 linear to striped
809 To convert an LV from linear to striped:
811 1. convert to raid1 with two images
812 2. convert to raid5_n
813 3. convert to raid5_n with five 128k stripes (reshape)
814 4. convert raid5_n to striped
815 The commands to perform the steps above are:
817 1. lvconvert --type raid1 --mirrors 1 LV
818 2. lvconvert --type raid5_n LV
819 3. lvconvert --stripes 5 --stripesize 128k LV
820 4. lvconvert --type striped LV
821 The raid5_n type in step 2 is used because it has dedicated parity Sub‐
823 LVs at the end, and can be converted to striped directly. The stripe
824 size is increased in step 3 to add extra space for the conversion
825 process. This step grows the LV size by a factor of five. After con‐
826 version, this extra space can be reduced (or used to grow the file sys‐
827 tem using the LV).
828 Reversing these steps will convert a striped LV to linear.
830 raid6 to striped
832 To convert an LV from raid6_nr to striped:
834 1. convert to raid6_n_6
835 2. convert to striped
836 The commands to perform the steps above are:
838 1. lvconvert --type raid6_n_6 LV
839 2. lvconvert --type striped LV
840 Examples
842 Converting an LV from linear to raid1.
844 # lvs -a -o name,segtype,size vg
846 LV Type LSize
847 lv linear 300.00g
848 # lvconvert --type raid1 --mirrors 1 vg/lv
850 # lvs -a -o name,segtype,size vg
852 LV Type LSize
853 lv raid1 300.00g
854 [lv_rimage_0] linear 300.00g
855 [lv_rimage_1] linear 300.00g
856 [lv_rmeta_0] linear 3.00m
857 [lv_rmeta_1] linear 3.00m
858 Converting an LV from mirror to raid1.
860 # lvs -a -o name,segtype,size vg
862 LV Type LSize
863 lv mirror 100.00g
864 [lv_mimage_0] linear 100.00g
865 [lv_mimage_1] linear 100.00g
866 [lv_mlog] linear 3.00m
867 # lvconvert --type raid1 vg/lv
869 # lvs -a -o name,segtype,size vg
871 LV Type LSize
872 lv raid1 100.00g
873 [lv_rimage_0] linear 100.00g
874 [lv_rimage_1] linear 100.00g
875 [lv_rmeta_0] linear 3.00m
876 [lv_rmeta_1] linear 3.00m
877 Converting an LV from linear to raid1 (with 3 images).
879 # lvconvert --type raid1 --mirrors 2 vg/lv
881 Converting an LV from striped (with 4 stripes) to raid6_n_6.
883 # lvcreate --stripes 4 -L64M -n lv vg
885 # lvconvert --type raid6 vg/lv
887 # lvs -a -o lv_name,segtype,sync_percent,data_copies
889 LV Type Cpy%Sync #Cpy
890 lv raid6_n_6 100.00 3
891 [lv_rimage_0] linear
892 [lv_rimage_1] linear
893 [lv_rimage_2] linear
894 [lv_rimage_3] linear
895 [lv_rimage_4] linear
896 [lv_rimage_5] linear
897 [lv_rmeta_0] linear
898 [lv_rmeta_1] linear
899 [lv_rmeta_2] linear
900 [lv_rmeta_3] linear
901 [lv_rmeta_4] linear
902 [lv_rmeta_5] linear
903 This convert begins by allocating MetaLVs (rmeta_#) for each of the ex‐
905 isting stripe devices. It then creates 2 additional MetaLV/DataLV
906 pairs (rmeta_#/rimage_#) for dedicated raid6 parity.
907 If rotating data/parity is required, such as with raid6_nr, it must be
909 done by reshaping (see below).
910
912 RAID reshaping is changing attributes of a RAID LV while keeping the
913 same RAID level. This includes changing RAID layout, stripe size, or
914 number of stripes.
915 When changing the RAID layout or stripe size, no new SubLVs (MetaLVs or
917 DataLVs) need to be allocated, but DataLVs are extended by a small
918 amount (typically 1 extent). The extra space allows blocks in a stripe
919 to be updated safely, and not be corrupted in case of a crash. If a
920 crash occurs, reshaping can just be restarted.
921 (If blocks in a stripe were updated in place, a crash could leave them
923 partially updated and corrupted. Instead, an existing stripe is qui‐
924 esced, read, changed in layout, and the new stripe written to free
925 space. Once that is done, the new stripe is unquiesced and used.)
926 Examples
928 (Command output shown in examples may change.)
929 Converting raid6_n_6 to raid6_nr with rotating data/parity.
931 This conversion naturally follows a previous conversion from
933 striped/raid0 to raid6_n_6 (shown above). It completes the transition
934 to a more traditional RAID6.
935 # lvs -o lv_name,segtype,sync_percent,data_copies
937 LV Type Cpy%Sync #Cpy
938 lv raid6_n_6 100.00 3
939 [lv_rimage_0] linear
940 [lv_rimage_1] linear
941 [lv_rimage_2] linear
942 [lv_rimage_3] linear
943 [lv_rimage_4] linear
944 [lv_rimage_5] linear
945 [lv_rmeta_0] linear
946 [lv_rmeta_1] linear
947 [lv_rmeta_2] linear
948 [lv_rmeta_3] linear
949 [lv_rmeta_4] linear
950 [lv_rmeta_5] linear
951 # lvconvert --type raid6_nr vg/lv
953 # lvs -a -o lv_name,segtype,sync_percent,data_copies
955 LV Type Cpy%Sync #Cpy
956 lv raid6_nr 100.00 3
957 [lv_rimage_0] linear
958 [lv_rimage_0] linear
959 [lv_rimage_1] linear
960 [lv_rimage_1] linear
961 [lv_rimage_2] linear
962 [lv_rimage_2] linear
963 [lv_rimage_3] linear
964 [lv_rimage_3] linear
965 [lv_rimage_4] linear
966 [lv_rimage_5] linear
967 [lv_rmeta_0] linear
968 [lv_rmeta_1] linear
969 [lv_rmeta_2] linear
970 [lv_rmeta_3] linear
971 [lv_rmeta_4] linear
972 [lv_rmeta_5] linear
973 The DataLVs are larger (additional segment in each) which provides
975 space for out-of-place reshaping. The result is:
976 # lvs -a -o lv_name,segtype,seg_pe_ranges,dataoffset
978 LV Type PE Ranges DOff
979 lv raid6_nr lv_rimage_0:0-32 \
980 lv_rimage_1:0-32 \
981 lv_rimage_2:0-32 \
982 lv_rimage_3:0-32
983 [lv_rimage_0] linear /dev/sda:0-31 2048
984 [lv_rimage_0] linear /dev/sda:33-33
985 [lv_rimage_1] linear /dev/sdaa:0-31 2048
986 [lv_rimage_1] linear /dev/sdaa:33-33
987 [lv_rimage_2] linear /dev/sdab:1-33 2048
988 [lv_rimage_3] linear /dev/sdac:1-33 2048
989 [lv_rmeta_0] linear /dev/sda:32-32
990 [lv_rmeta_1] linear /dev/sdaa:32-32
991 [lv_rmeta_2] linear /dev/sdab:0-0
992 [lv_rmeta_3] linear /dev/sdac:0-0
993 All segments with PE ranges '33-33' provide the out-of-place reshape
995 space. The dataoffset column shows that the data was moved from ini‐
996 tial offset 0 to 2048 sectors on each component DataLV.
997 For performance reasons the raid6_nr RaidLV can be restriped. Convert
999 it from 3-way striped to 5-way-striped.
1000 # lvconvert --stripes 5 vg/lv
1002 Using default stripesize 64.00 KiB.
1003 WARNING: Adding stripes to active logical volume vg/lv will \
1004 grow it from 99 to 165 extents!
1005 Run "lvresize -l99 vg/lv" to shrink it or use the additional \
1006 capacity.
1007 Logical volume vg/lv successfully converted.
1008 # lvs vg/lv
1010 LV VG Attr LSize Cpy%Sync
1011 lv vg rwi-a-r-s- 652.00m 52.94
1012 # lvs -a -o lv_name,attr,segtype,seg_pe_ranges,dataoffset vg
1014 LV Attr Type PE Ranges DOff
1015 lv rwi-a-r--- raid6_nr lv_rimage_0:0-33 \
1016 lv_rimage_1:0-33 \
1017 lv_rimage_2:0-33 ... \
1018 lv_rimage_5:0-33 \
1019 lv_rimage_6:0-33 0
1020 [lv_rimage_0] iwi-aor--- linear /dev/sda:0-32 0
1021 [lv_rimage_0] iwi-aor--- linear /dev/sda:34-34
1022 [lv_rimage_1] iwi-aor--- linear /dev/sdaa:0-32 0
1023 [lv_rimage_1] iwi-aor--- linear /dev/sdaa:34-34
1024 [lv_rimage_2] iwi-aor--- linear /dev/sdab:0-32 0
1025 [lv_rimage_2] iwi-aor--- linear /dev/sdab:34-34
1026 [lv_rimage_3] iwi-aor--- linear /dev/sdac:1-34 0
1027 [lv_rimage_4] iwi-aor--- linear /dev/sdad:1-34 0
1028 [lv_rimage_5] iwi-aor--- linear /dev/sdae:1-34 0
1029 [lv_rimage_6] iwi-aor--- linear /dev/sdaf:1-34 0
1030 [lv_rmeta_0] ewi-aor--- linear /dev/sda:33-33
1031 [lv_rmeta_1] ewi-aor--- linear /dev/sdaa:33-33
1032 [lv_rmeta_2] ewi-aor--- linear /dev/sdab:33-33
1033 [lv_rmeta_3] ewi-aor--- linear /dev/sdac:0-0
1034 [lv_rmeta_4] ewi-aor--- linear /dev/sdad:0-0
1035 [lv_rmeta_5] ewi-aor--- linear /dev/sdae:0-0
1036 [lv_rmeta_6] ewi-aor--- linear /dev/sdaf:0-0
1037 Stripes also can be removed from raid5 and 6. Convert the 5-way
1039 striped raid6_nr LV to 4-way-striped. The force option needs to be
1040 used, because removing stripes (i.e. image SubLVs) from a RaidLV will
1041 shrink its size.
1042 # lvconvert --stripes 4 vg/lv
1044 Using default stripesize 64.00 KiB.
1045 WARNING: Removing stripes from active logical volume vg/lv will \
1046 shrink it from 660.00 MiB to 528.00 MiB!
1047 THIS MAY DESTROY (PARTS OF) YOUR DATA!
1048 If that leaves the logical volume larger than 206 extents due \
1049 to stripe rounding,
1050 you may want to grow the content afterwards (filesystem etc.)
1051 WARNING: to remove freed stripes after the conversion has finished,\
1052 you have to run "lvconvert --stripes 4 vg/lv"
1053 Logical volume vg/lv successfully converted.
1054 # lvs -a -o lv_name,attr,segtype,seg_pe_ranges,dataoffset vg
1056 LV Attr Type PE Ranges DOff
1057 lv rwi-a-r-s- raid6_nr lv_rimage_0:0-33 \
1058 lv_rimage_1:0-33 \
1059 lv_rimage_2:0-33 ... \
1060 lv_rimage_5:0-33 \
1061 lv_rimage_6:0-33 0
1062 [lv_rimage_0] Iwi-aor--- linear /dev/sda:0-32 0
1063 [lv_rimage_0] Iwi-aor--- linear /dev/sda:34-34
1064 [lv_rimage_1] Iwi-aor--- linear /dev/sdaa:0-32 0
1065 [lv_rimage_1] Iwi-aor--- linear /dev/sdaa:34-34
1066 [lv_rimage_2] Iwi-aor--- linear /dev/sdab:0-32 0
1067 [lv_rimage_2] Iwi-aor--- linear /dev/sdab:34-34
1068 [lv_rimage_3] Iwi-aor--- linear /dev/sdac:1-34 0
1069 [lv_rimage_4] Iwi-aor--- linear /dev/sdad:1-34 0
1070 [lv_rimage_5] Iwi-aor--- linear /dev/sdae:1-34 0
1071 [lv_rimage_6] Iwi-aor-R- linear /dev/sdaf:1-34 0
1072 [lv_rmeta_0] ewi-aor--- linear /dev/sda:33-33
1073 [lv_rmeta_1] ewi-aor--- linear /dev/sdaa:33-33
1074 [lv_rmeta_2] ewi-aor--- linear /dev/sdab:33-33
1075 [lv_rmeta_3] ewi-aor--- linear /dev/sdac:0-0
1076 [lv_rmeta_4] ewi-aor--- linear /dev/sdad:0-0
1077 [lv_rmeta_5] ewi-aor--- linear /dev/sdae:0-0
1078 [lv_rmeta_6] ewi-aor-R- linear /dev/sdaf:0-0
1079 The 's' in column 9 of the attribute field shows the RaidLV is still
1081 reshaping. The 'R' in the same column of the attribute field shows the
1082 freed image Sub LVs which will need removing once the reshaping fin‐
1083 ished.
1084 # lvs -o lv_name,attr,segtype,seg_pe_ranges,dataoffset vg
1086 LV Attr Type PE Ranges DOff
1087 lv rwi-a-r-R- raid6_nr lv_rimage_0:0-33 \
1088 lv_rimage_1:0-33 \
1089 lv_rimage_2:0-33 ... \
1090 lv_rimage_5:0-33 \
1091 lv_rimage_6:0-33 8192
1092 Now that the reshape is finished the 'R' attribute on the RaidLV shows
1094 images can be removed.
1095 # lvs -o lv_name,attr,segtype,seg_pe_ranges,dataoffset vg
1097 LV Attr Type PE Ranges DOff
1098 lv rwi-a-r-R- raid6_nr lv_rimage_0:0-33 \
1099 lv_rimage_1:0-33 \
1100 lv_rimage_2:0-33 ... \
1101 lv_rimage_5:0-33 \
1102 lv_rimage_6:0-33 8192
1103 This is achieved by repeating the command ("lvconvert --stripes 4
1105 vg/lv" would be sufficient).
1106 # lvconvert --stripes 4 vg/lv
1108 Using default stripesize 64.00 KiB.
1109 Logical volume vg/lv successfully converted.
1110 # lvs -a -o lv_name,attr,segtype,seg_pe_ranges,dataoffset vg
1112 LV Attr Type PE Ranges DOff
1113 lv rwi-a-r--- raid6_nr lv_rimage_0:0-33 \
1114 lv_rimage_1:0-33 \
1115 lv_rimage_2:0-33 ... \
1116 lv_rimage_5:0-33 8192
1117 [lv_rimage_0] iwi-aor--- linear /dev/sda:0-32 8192
1118 [lv_rimage_0] iwi-aor--- linear /dev/sda:34-34
1119 [lv_rimage_1] iwi-aor--- linear /dev/sdaa:0-32 8192
1120 [lv_rimage_1] iwi-aor--- linear /dev/sdaa:34-34
1121 [lv_rimage_2] iwi-aor--- linear /dev/sdab:0-32 8192
1122 [lv_rimage_2] iwi-aor--- linear /dev/sdab:34-34
1123 [lv_rimage_3] iwi-aor--- linear /dev/sdac:1-34 8192
1124 [lv_rimage_4] iwi-aor--- linear /dev/sdad:1-34 8192
1125 [lv_rimage_5] iwi-aor--- linear /dev/sdae:1-34 8192
1126 [lv_rmeta_0] ewi-aor--- linear /dev/sda:33-33
1127 [lv_rmeta_1] ewi-aor--- linear /dev/sdaa:33-33
1128 [lv_rmeta_2] ewi-aor--- linear /dev/sdab:33-33
1129 [lv_rmeta_3] ewi-aor--- linear /dev/sdac:0-0
1130 [lv_rmeta_4] ewi-aor--- linear /dev/sdad:0-0
1131 [lv_rmeta_5] ewi-aor--- linear /dev/sdae:0-0
1132 # lvs -a -o lv_name,attr,segtype,reshapelen vg
1134 LV Attr Type RSize
1135 lv rwi-a-r--- raid6_nr 24.00m
1136 [lv_rimage_0] iwi-aor--- linear 4.00m
1137 [lv_rimage_0] iwi-aor--- linear
1138 [lv_rimage_1] iwi-aor--- linear 4.00m
1139 [lv_rimage_1] iwi-aor--- linear
1140 [lv_rimage_2] iwi-aor--- linear 4.00m
1141 [lv_rimage_2] iwi-aor--- linear
1142 [lv_rimage_3] iwi-aor--- linear 4.00m
1143 [lv_rimage_4] iwi-aor--- linear 4.00m
1144 [lv_rimage_5] iwi-aor--- linear 4.00m
1145 [lv_rmeta_0] ewi-aor--- linear
1146 [lv_rmeta_1] ewi-aor--- linear
1147 [lv_rmeta_2] ewi-aor--- linear
1148 [lv_rmeta_3] ewi-aor--- linear
1149 [lv_rmeta_4] ewi-aor--- linear
1150 [lv_rmeta_5] ewi-aor--- linear
1151 Future developments might include automatic removal of the freed im‐
1153 ages.
1154 If the reshape space shall be removed any lvconvert command not chang‐
1156 ing the layout can be used:
1157 # lvconvert --stripes 4 vg/lv
1159 Using default stripesize 64.00 KiB.
1160 No change in RAID LV vg/lv layout, freeing reshape space.
1161 Logical volume vg/lv successfully converted.
1162 # lvs -a -o lv_name,attr,segtype,reshapelen vg
1164 LV Attr Type RSize
1165 lv rwi-a-r--- raid6_nr 0
1166 [lv_rimage_0] iwi-aor--- linear 0
1167 [lv_rimage_0] iwi-aor--- linear
1168 [lv_rimage_1] iwi-aor--- linear 0
1169 [lv_rimage_1] iwi-aor--- linear
1170 [lv_rimage_2] iwi-aor--- linear 0
1171 [lv_rimage_2] iwi-aor--- linear
1172 [lv_rimage_3] iwi-aor--- linear 0
1173 [lv_rimage_4] iwi-aor--- linear 0
1174 [lv_rimage_5] iwi-aor--- linear 0
1175 [lv_rmeta_0] ewi-aor--- linear
1176 [lv_rmeta_1] ewi-aor--- linear
1177 [lv_rmeta_2] ewi-aor--- linear
1178 [lv_rmeta_3] ewi-aor--- linear
1179 [lv_rmeta_4] ewi-aor--- linear
1180 [lv_rmeta_5] ewi-aor--- linear
1181 In case the RaidLV should be converted to striped:
1183 # lvconvert --type striped vg/lv
1185 Unable to convert LV vg/lv from raid6_nr to striped.
1186 Converting vg/lv from raid6_nr is directly possible to the \
1187 following layouts:
1188 raid6_nc
1189 raid6_zr
1190 raid6_la_6
1191 raid6_ls_6
1192 raid6_ra_6
1193 raid6_rs_6
1194 raid6_n_6
1195 A direct conversion isn't possible thus the command informed about the
1197 possible ones. raid6_n_6 is suitable to convert to striped so convert
1198 to it first (this is a reshape changing the raid6 layout from raid6_nr
1199 to raid6_n_6).
1200 # lvconvert --type raid6_n_6
1202 Using default stripesize 64.00 KiB.
1203 Converting raid6_nr LV vg/lv to raid6_n_6.
1204 Are you sure you want to convert raid6_nr LV vg/lv? [y/n]: y
1205 Logical volume vg/lv successfully converted.
1206 Wait for the reshape to finish.
1208 # lvconvert --type striped vg/lv
1210 Logical volume vg/lv successfully converted.
1211 # lvs -o lv_name,attr,segtype,seg_pe_ranges,dataoffset vg
1213 LV Attr Type PE Ranges DOff
1214 lv -wi-a----- striped /dev/sda:2-32 \
1215 /dev/sdaa:2-32 \
1216 /dev/sdab:2-32 \
1217 /dev/sdac:3-33
1218 lv -wi-a----- striped /dev/sda:34-35 \
1219 /dev/sdaa:34-35 \
1220 /dev/sdab:34-35 \
1221 /dev/sdac:34-35
1222 From striped we can convert to raid10
1224 # lvconvert --type raid10 vg/lv
1226 Using default stripesize 64.00 KiB.
1227 Logical volume vg/lv successfully converted.
1228 # lvs -o lv_name,attr,segtype,seg_pe_ranges,dataoffset vg
1230 LV Attr Type PE Ranges DOff
1231 lv rwi-a-r--- raid10 lv_rimage_0:0-32 \
1232 lv_rimage_4:0-32 \
1233 lv_rimage_1:0-32 ... \
1234 lv_rimage_3:0-32 \
1235 lv_rimage_7:0-32 0
1236 # lvs -a -o lv_name,attr,segtype,seg_pe_ranges,dataoffset vg
1238 WARNING: Cannot find matching striped segment for vg/lv_rimage_3.
1239 LV Attr Type PE Ranges DOff
1240 lv rwi-a-r--- raid10 lv_rimage_0:0-32 \
1241 lv_rimage_4:0-32 \
1242 lv_rimage_1:0-32 ... \
1243 lv_rimage_3:0-32 \
1244 lv_rimage_7:0-32 0
1245 [lv_rimage_0] iwi-aor--- linear /dev/sda:2-32 0
1246 [lv_rimage_0] iwi-aor--- linear /dev/sda:34-35
1247 [lv_rimage_1] iwi-aor--- linear /dev/sdaa:2-32 0
1248 [lv_rimage_1] iwi-aor--- linear /dev/sdaa:34-35
1249 [lv_rimage_2] iwi-aor--- linear /dev/sdab:2-32 0
1250 [lv_rimage_2] iwi-aor--- linear /dev/sdab:34-35
1251 [lv_rimage_3] iwi-XXr--- linear /dev/sdac:3-35 0
1252 [lv_rimage_4] iwi-aor--- linear /dev/sdad:1-33 0
1253 [lv_rimage_5] iwi-aor--- linear /dev/sdae:1-33 0
1254 [lv_rimage_6] iwi-aor--- linear /dev/sdaf:1-33 0
1255 [lv_rimage_7] iwi-aor--- linear /dev/sdag:1-33 0
1256 [lv_rmeta_0] ewi-aor--- linear /dev/sda:0-0
1257 [lv_rmeta_1] ewi-aor--- linear /dev/sdaa:0-0
1258 [lv_rmeta_2] ewi-aor--- linear /dev/sdab:0-0
1259 [lv_rmeta_3] ewi-aor--- linear /dev/sdac:0-0
1260 [lv_rmeta_4] ewi-aor--- linear /dev/sdad:0-0
1261 [lv_rmeta_5] ewi-aor--- linear /dev/sdae:0-0
1262 [lv_rmeta_6] ewi-aor--- linear /dev/sdaf:0-0
1263 [lv_rmeta_7] ewi-aor--- linear /dev/sdag:0-0
1264 raid10 allows to add stripes but can't remove them.
1266 A more elaborate example to convert from linear to striped with interim
1268 conversions to raid1 then raid5 followed by restripe (4 steps).
1269 We start with the linear LV.
1271 # lvs -a -o name,size,segtype,syncpercent,datastripes,\
1273 stripesize,reshapelenle,devices vg
1274 LV LSize Type Cpy%Sync #DStr Stripe RSize Devices
1275 lv 128.00m linear 1 0 /dev/sda(0)
1276 Then convert it to a 2-way raid1.
1278 # lvconvert --mirrors 1 vg/lv
1280 Logical volume vg/lv successfully converted.
1281 # lvs -a -o name,size,segtype,datastripes,\
1283 stripesize,reshapelenle,devices vg
1284 LV LSize Type #DStr Stripe RSize Devices
1285 lv 128.00m raid1 2 0 lv_rimage_0(0),\
1286 lv_rimage_1(0)
1287 [lv_rimage_0] 128.00m linear 1 0 /dev/sda(0)
1288 [lv_rimage_1] 128.00m linear 1 0 /dev/sdhx(1)
1289 [lv_rmeta_0] 4.00m linear 1 0 /dev/sda(32)
1290 [lv_rmeta_1] 4.00m linear 1 0 /dev/sdhx(0)
1291 Once the raid1 LV is fully synchronized we convert it to raid5_n (only
1293 2-way raid1 LVs can be converted to raid5). We select raid5_n here be‐
1294 cause it has dedicated parity SubLVs at the end and can be converted to
1295 striped directly without any additional conversion.
1296 # lvconvert --type raid5_n vg/lv
1298 Using default stripesize 64.00 KiB.
1299 Logical volume vg/lv successfully converted.
1300 # lvs -a -o name,size,segtype,syncpercent,datastripes,\
1302 stripesize,reshapelenle,devices vg
1303 LV LSize Type #DStr Stripe RSize Devices
1304 lv 128.00m raid5_n 1 64.00k 0 lv_rimage_0(0),\
1305 lv_rimage_1(0)
1306 [lv_rimage_0] 128.00m linear 1 0 0 /dev/sda(0)
1307 [lv_rimage_1] 128.00m linear 1 0 0 /dev/sdhx(1)
1308 [lv_rmeta_0] 4.00m linear 1 0 /dev/sda(32)
1309 [lv_rmeta_1] 4.00m linear 1 0 /dev/sdhx(0)
1310 Now we'll change the number of data stripes from 1 to 5 and request
1312 128K stripe size in one command. This will grow the size of the LV by
1313 a factor of 5 (we add 4 data stripes to the one given). That addi‐
1314 tional space can be used by e.g. growing any contained filesystem or
1315 the LV can be reduced in size after the reshaping conversion has fin‐
1316 ished.
1317 # lvconvert --stripesize 128k --stripes 5 vg/lv
1319 Converting stripesize 64.00 KiB of raid5_n LV vg/lv to 128.00 KiB.
1320 WARNING: Adding stripes to active logical volume vg/lv will grow \
1321 it from 32 to 160 extents!
1322 Run "lvresize -l32 vg/lv" to shrink it or use the additional capacity.
1323 Logical volume vg/lv successfully converted.
1324 # lvs -a -o name,size,segtype,datastripes,\
1326 stripesize,reshapelenle,devices
1327 LV LSize Type #DStr Stripe RSize Devices
1328 lv 640.00m raid5_n 5 128.00k 6 lv_rimage_0(0),\
1329 lv_rimage_1(0),\
1330 lv_rimage_2(0),\
1331 lv_rimage_3(0),\
1332 lv_rimage_4(0),\
1333 lv_rimage_5(0)
1334 [lv_rimage_0] 132.00m linear 1 0 1 /dev/sda(33)
1335 [lv_rimage_0] 132.00m linear 1 0 /dev/sda(0)
1336 [lv_rimage_1] 132.00m linear 1 0 1 /dev/sdhx(33)
1337 [lv_rimage_1] 132.00m linear 1 0 /dev/sdhx(1)
1338 [lv_rimage_2] 132.00m linear 1 0 1 /dev/sdhw(33)
1339 [lv_rimage_2] 132.00m linear 1 0 /dev/sdhw(1)
1340 [lv_rimage_3] 132.00m linear 1 0 1 /dev/sdhv(33)
1341 [lv_rimage_3] 132.00m linear 1 0 /dev/sdhv(1)
1342 [lv_rimage_4] 132.00m linear 1 0 1 /dev/sdhu(33)
1343 [lv_rimage_4] 132.00m linear 1 0 /dev/sdhu(1)
1344 [lv_rimage_5] 132.00m linear 1 0 1 /dev/sdht(33)
1345 [lv_rimage_5] 132.00m linear 1 0 /dev/sdht(1)
1346 [lv_rmeta_0] 4.00m linear 1 0 /dev/sda(32)
1347 [lv_rmeta_1] 4.00m linear 1 0 /dev/sdhx(0)
1348 [lv_rmeta_2] 4.00m linear 1 0 /dev/sdhw(0)
1349 [lv_rmeta_3] 4.00m linear 1 0 /dev/sdhv(0)
1350 [lv_rmeta_4] 4.00m linear 1 0 /dev/sdhu(0)
1351 [lv_rmeta_5] 4.00m linear 1 0 /dev/sdht(0)
1352 Once the conversion has finished we can can convert to striped.
1354 # lvconvert --type striped vg/lv
1356 Logical volume vg/lv successfully converted.
1357 # lvs -a -o name,size,segtype,datastripes,\
1359 stripesize,reshapelenle,devices vg
1360 LV LSize Type #DStr Stripe RSize Devices
1361 lv 640.00m striped 5 128.00k /dev/sda(33),\
1362 /dev/sdhx(33),\
1363 /dev/sdhw(33),\
1364 /dev/sdhv(33),\
1365 /dev/sdhu(33)
1366 lv 640.00m striped 5 128.00k /dev/sda(0),\
1367 /dev/sdhx(1),\
1368 /dev/sdhw(1),\
1369 /dev/sdhv(1),\
1370 /dev/sdhu(1)
1371 Reversing these steps will convert a given striped LV to linear.
1373 Mind the facts that stripes are removed thus the capacity of the RaidLV
1375 will shrink and that changing the RaidLV layout will influence its per‐
1376 formance.
1377 "lvconvert --stripes 1 vg/lv" for converting to 1 stripe will inform
1379 upfront about the reduced size to allow for resizing the content or
1380 growing the RaidLV before actually converting to 1 stripe. The --force
1381 option is needed to allow stripe removing conversions to prevent data
1382 loss.
1383 Of course any interim step can be the intended last one (e.g. striped →
1385 raid1).
1386
1388 raid5_ls
1389 • RAID5 left symmetric
1390 • Rotating parity N with data restart
1391 raid5_la
1393 • RAID5 left asymmetric
1394 • Rotating parity N with data continuation
1395 raid5_rs
1397 • RAID5 right symmetric
1398 • Rotating parity 0 with data restart
1399 raid5_ra
1401 • RAID5 right asymmetric
1402 • Rotating parity 0 with data continuation
1403 raid5_n
1405 • RAID5 parity n
1406 • Dedicated parity device n used for striped/raid0 conversions
1407 • Used for RAID Takeover
1408
1410 raid6
1411 • RAID6 zero restart (aka left symmetric)
1412 • Rotating parity 0 with data restart
1413 • Same as raid6_zr
1414 raid6_zr
1416 • RAID6 zero restart (aka left symmetric)
1417 • Rotating parity 0 with data restart
1418 raid6_nr
1420 • RAID6 N restart (aka right symmetric)
1421 • Rotating parity N with data restart
1422 raid6_nc
1424 • RAID6 N continue
1425 • Rotating parity N with data continuation
1426 raid6_n_6
1428 • RAID6 last parity devices
1429 • Fixed dedicated last devices (P-Syndrome N-1 and Q-Syndrome N)
1430 with striped data used for striped/raid0 conversions
1431 • Used for RAID Takeover
1432 raid6_{ls,rs,la,ra}_6
1434 • RAID6 last parity device
1435 • Dedicated last parity device used for conversions from/to
1436 raid5_{ls,rs,la,ra}
1437 raid6_ls_6
1439 • RAID6 N continue
1440 • Same as raid5_ls for N-1 devices with fixed Q-Syndrome N
1441 • Used for RAID Takeover
1442 raid6_la_6
1444 • RAID6 N continue
1445 • Same as raid5_la for N-1 devices with fixed Q-Syndrome N
1446 • Used forRAID Takeover
1447 raid6_rs_6
1449 • RAID6 N continue
1450 • Same as raid5_rs for N-1 devices with fixed Q-Syndrome N
1451 • Used for RAID Takeover
1452 raid6_ra_6
1454 • RAID6 N continue
1455 • Same as raid5_ra for N-1 devices with fixed Q-Syndrome N
1456 • Used for RAID Takeover
1457
1459 The 2.6.38-rc1 version of the Linux kernel introduced a device-mapper
1460 target to interface with the software RAID (MD) personalities. This
1461 provided device-mapper with RAID 4/5/6 capabilities and a larger devel‐
1462 opment community. Later, support for RAID1, RAID10, and RAID1E (RAID
1463 10 variants) were added. Support for these new kernel RAID targets was
1464 added to LVM version 2.02.87. The capabilities of the LVM raid1 type
1465 have surpassed the old mirror type. raid1 is now recommended instead
1466 of mirror. raid1 became the default for mirroring in LVM version
1467 2.02.100.
1468
1470 lvm(8), lvm.conf(5), lvcreate(8), lvconvert(8), lvchange(8),
1471 lvextend(8), dmeventd(8)
1472Red Hat, Inc LVM TOOLS 2.03.22(2) (2023-08-02) LVMRAID(7)