1LVMRAID(7) LVMRAID(7)
2
3
4
6 lvmraid — LVM RAID
7
8
10 lvm(8) RAID is a way to create a Logical Volume (LV) that uses multiple
11 physical devices to improve performance or tolerate device failures.
12 In LVM, the physical devices are Physical Volumes (PVs) in a single
13 Volume Group (VG).
14
15 How LV data blocks are placed onto PVs is determined by the RAID level.
16 RAID levels are commonly referred to as 'raid' followed by a number,
17 e.g. raid1, raid5 or raid6. Selecting a RAID level involves making
18 tradeoffs among: physical device requirements, fault tolerance, and
19 performance. A description of the RAID levels can be found at
20 www.snia.org/sites/default/files/SNIA_DDF_Technical_Position_v2.0.pdf
21
22 LVM RAID uses both Device Mapper (DM) and Multiple Device (MD) drivers
23 from the Linux kernel. DM is used to create and manage visible LVM
24 devices, and MD is used to place data on physical devices.
25
26 LVM creates hidden LVs (dm devices) layered between the visible LV and
27 physical devices. LVs in the middle layers are called sub LVs. For
28 LVM raid, a sub LV pair to store data and metadata (raid superblock and
29 write intent bitmap) is created per raid image/leg (see lvs command
30 examples below).
31
32
34 To create a RAID LV, use lvcreate and specify an LV type. The LV type
35 corresponds to a RAID level. The basic RAID levels that can be used
36 are: raid0, raid1, raid4, raid5, raid6, raid10.
37
38 lvcreate --type RaidLevel [OPTIONS] --name Name --size Size VG [PVs]
39
40 To display the LV type of an existing LV, run:
41
42 lvs -o name,segtype LV
43
44 (The LV type is also referred to as "segment type" or "segtype".)
45
46 LVs can be created with the following types:
47
48
49 raid0
50
51
52 Also called striping, raid0 spreads LV data across multiple devices in
53 units of stripe size. This is used to increase performance. LV data
54 will be lost if any of the devices fail.
55
56 lvcreate --type raid0 [--stripes Number --stripesize Size] VG [PVs]
57
58
59 --stripes specifies the number of devices to spread the LV across.
60
61
62 --stripesize specifies the size of each stripe in kilobytes. This is
63 the amount of data that is written to one device before moving
64 to the next.
65
66 PVs specifies the devices to use. If not specified, lvm will choose
67 Number devices, one for each stripe based on the number of PVs avail‐
68 able or supplied.
69
70
71 raid1
72
73
74 Also called mirroring, raid1 uses multiple devices to duplicate LV
75 data. The LV data remains available if all but one of the devices
76 fail. The minimum number of devices (i.e. sub LV pairs) required is 2.
77
78 lvcreate --type raid1 [--mirrors Number] VG [PVs]
79
80
81 --mirrors specifies the number of mirror images in addition to the
82 original LV image, e.g. --mirrors 1 means there are two images
83 of the data, the original and one mirror image.
84
85 PVs specifies the devices to use. If not specified, lvm will choose
86 Number devices, one for each image.
87
88
89 raid4
90
91
92 raid4 is a form of striping that uses an extra, first device dedicated
93 to storing parity blocks. The LV data remains available if one device
94 fails. The parity is used to recalculate data that is lost from a sin‐
95 gle device. The minimum number of devices required is 3.
96
97 lvcreate --type raid4 [--stripes Number --stripesize Size] VG [PVs]
98
99
100 --stripes specifies the number of devices to use for LV data. This
101 does not include the extra device lvm adds for storing parity
102 blocks. A raid4 LV with Number stripes requires Number+1
103 devices. Number must be 2 or more.
104
105
106 --stripesize specifies the size of each stripe in kilobytes. This is
107 the amount of data that is written to one device before moving
108 to the next.
109
110 PVs specifies the devices to use. If not specified, lvm will choose
111 Number+1 separate devices.
112
113 raid4 is called non-rotating parity because the parity blocks are
114 always stored on the same device.
115
116
117 raid5
118
119
120 raid5 is a form of striping that uses an extra device for storing par‐
121 ity blocks. LV data and parity blocks are stored on each device, typi‐
122 cally in a rotating pattern for performance reasons. The LV data
123 remains available if one device fails. The parity is used to recalcu‐
124 late data that is lost from a single device. The minimum number of
125 devices required is 3 (unless converting from 2 legged raid1 to reshape
126 to more stripes; see reshaping).
127
128 lvcreate --type raid5 [--stripes Number --stripesize Size] VG [PVs]
129
130
131 --stripes specifies the number of devices to use for LV data. This
132 does not include the extra device lvm adds for storing parity
133 blocks. A raid5 LV with Number stripes requires Number+1
134 devices. Number must be 2 or more.
135
136
137 --stripesize specifies the size of each stripe in kilobytes. This is
138 the amount of data that is written to one device before moving
139 to the next.
140
141 PVs specifies the devices to use. If not specified, lvm will choose
142 Number+1 separate devices.
143
144 raid5 is called rotating parity because the parity blocks are placed on
145 different devices in a round-robin sequence. There are variations of
146 raid5 with different algorithms for placing the parity blocks. The
147 default variant is raid5_ls (raid5 left symmetric, which is a rotating
148 parity 0 with data restart.) See RAID5 variants below.
149
150
151 raid6
152
153
154 raid6 is a form of striping like raid5, but uses two extra devices for
155 parity blocks. LV data and parity blocks are stored on each device,
156 typically in a rotating pattern for perfomramce reasons. The LV data
157 remains available if up to two devices fail. The parity is used to
158 recalculate data that is lost from one or two devices. The minimum
159 number of devices required is 5.
160
161 lvcreate --type raid6 [--stripes Number --stripesize Size] VG [PVs]
162
163
164 --stripes specifies the number of devices to use for LV data. This
165 does not include the extra two devices lvm adds for storing par‐
166 ity blocks. A raid6 LV with Number stripes requires Number+2
167 devices. Number must be 3 or more.
168
169
170 --stripesize specifies the size of each stripe in kilobytes. This is
171 the amount of data that is written to one device before moving
172 to the next.
173
174 PVs specifies the devices to use. If not specified, lvm will choose
175 Number+2 separate devices.
176
177 Like raid5, there are variations of raid6 with different algorithms for
178 placing the parity blocks. The default variant is raid6_zr (raid6 zero
179 restart, aka left symmetric, which is a rotating parity 0 with data
180 restart.) See RAID6 variants below.
181
182
183 raid10
184
185
186 raid10 is a combination of raid1 and raid0, striping data across mir‐
187 rored devices. LV data remains available if one or more devices
188 remains in each mirror set. The minimum number of devices required is
189 4.
190
191 lvcreate --type raid10
192 [--mirrors NumberMirrors]
193 [--stripes NumberStripes --stripesize Size]
194 VG [PVs]
195
196
197 --mirrors specifies the number of mirror images within each stripe.
198 e.g. --mirrors 1 means there are two images of the data, the
199 original and one mirror image.
200
201
202 --stripes specifies the total number of devices to use in all raid1
203 images (not the number of raid1 devices to spread the LV across,
204 even though that is the effective result). The number of
205 devices in each raid1 mirror will be NumberStripes/(NumberMir‐
206 rors+1), e.g. mirrors 1 and stripes 4 will stripe data across
207 two raid1 mirrors, where each mirror is devices.
208
209
210 --stripesize specifies the size of each stripe in kilobytes. This is
211 the amount of data that is written to one device before moving
212 to the next.
213
214 PVs specifies the devices to use. If not specified, lvm will choose
215 the necessary devices. Devices are used to create mirrors in the order
216 listed, e.g. for mirrors 1, stripes 2, listing PV1 PV2 PV3 PV4 results
217 in mirrors PV1/PV2 and PV3/PV4.
218
219 RAID10 is not mirroring on top of stripes, which would be RAID01, which
220 is less tolerant of device failures.
221
222
223
225 Synchronization is the process that makes all the devices in a RAID LV
226 consistent with each other.
227
228 In a RAID1 LV, all mirror images should have the same data. When a new
229 mirror image is added, or a mirror image is missing data, then images
230 need to be synchronized. Data blocks are copied from an existing image
231 to a new or outdated image to make them match.
232
233 In a RAID 4/5/6 LV, parity blocks and data blocks should match based on
234 the parity calculation. When the devices in a RAID LV change, the data
235 and parity blocks can become inconsistent and need to be synchronized.
236 Correct blocks are read, parity is calculated, and recalculated blocks
237 are written.
238
239 The RAID implementation keeps track of which parts of a RAID LV are
240 synchronized. When a RAID LV is first created and activated the first
241 synchronization is called initialization. A pointer stored in the raid
242 metadata keeps track of the initialization process thus allowing it to
243 be restarted after a deactivation of the RaidLV or a crash. Any writes
244 to the RaidLV dirties the respective region of the write intent bitmap
245 which allow for fast recovery of the regions after a crash. Without
246 this, the entire LV would need to be synchronized every time it was
247 activated.
248
249 Automatic synchronization happens when a RAID LV is activated, but it
250 is usually partial because the bitmaps reduce the areas that are
251 checked. A full sync becomes necessary when devices in the RAID LV are
252 replaced.
253
254 The synchronization status of a RAID LV is reported by the following
255 command, where "Cpy%Sync" = "100%" means sync is complete:
256
257 lvs -a -o name,sync_percent
258
259
260
261 Scrubbing
262 Scrubbing is a full scan of the RAID LV requested by a user. Scrubbing
263 can find problems that are missed by partial synchronization.
264
265 Scrubbing assumes that RAID metadata and bitmaps may be inaccurate, so
266 it verifies all RAID metadata, LV data, and parity blocks. Scrubbing
267 can find inconsistencies caused by hardware errors or degradation.
268 These kinds of problems may be undetected by automatic synchronization
269 which excludes areas outside of the RAID write-intent bitmap.
270
271 The command to scrub a RAID LV can operate in two different modes:
272
273 lvchange --syncaction check|repair LV
274
275
276 check Check mode is read-only and only detects inconsistent areas in
277 the RAID LV, it does not correct them.
278
279
280 repair Repair mode checks and writes corrected blocks to synchronize
281 any inconsistent areas.
282
283
284 Scrubbing can consume a lot of bandwidth and slow down application I/O
285 on the RAID LV. To control the I/O rate used for scrubbing, use:
286
287
288 --maxrecoveryrate Size[k|UNIT]
289 Sets the maximum recovery rate for a RAID LV. Size is specified
290 as an amount per second for each device in the array. If no
291 suffix is given, then KiB/sec/device is used. Setting the
292 recovery rate to 0 means it will be unbounded.
293
294
295 --minrecoveryrate Size[k|UNIT]
296 Sets the minimum recovery rate for a RAID LV. Size is specified
297 as an amount per second for each device in the array. If no
298 suffix is given, then KiB/sec/device is used. Setting the
299 recovery rate to 0 means it will be unbounded.
300
301
302 To display the current scrubbing in progress on an LV, including the
303 syncaction mode and percent complete, run:
304
305 lvs -a -o name,raid_sync_action,sync_percent
306
307 After scrubbing is complete, to display the number of inconsistent
308 blocks found, run:
309
310 lvs -o name,raid_mismatch_count
311
312 Also, if mismatches were found, the lvs attr field will display the
313 letter "m" (mismatch) in the 9th position, e.g.
314
315 # lvs -o name,vgname,segtype,attr vg/lv
316 LV VG Type Attr
317 lv vg raid1 Rwi-a-r-m-
318
319
320
321 Scrubbing Limitations
322 The check mode can only report the number of inconsistent blocks, it
323 cannot report which blocks are inconsistent. This makes it impossible
324 to know which device has errors, or if the errors affect file system
325 data, metadata or nothing at all.
326
327 The repair mode can make the RAID LV data consistent, but it does not
328 know which data is correct. The result may be consistent but incorrect
329 data. When two different blocks of data must be made consistent, it
330 chooses the block from the device that would be used during RAID
331 intialization. However, if the PV holding corrupt data is known,
332 lvchange --rebuild can be used in place of scrubbing to reconstruct the
333 data on the bad device.
334
335 Future developments might include:
336
337 Allowing a user to choose the correct version of data during repair.
338
339 Using a majority of devices to determine the correct version of data to
340 use in a 3-way RAID1 or RAID6 LV.
341
342 Using a checksumming device to pin-point when and where an error
343 occurs, allowing it to be rewritten.
344
345
346
348 An LV is often a combination of other hidden LVs called SubLVs. The
349 SubLVs either use physical devices, or are built from other SubLVs
350 themselves. SubLVs hold LV data blocks, RAID parity blocks, and RAID
351 metadata. SubLVs are generally hidden, so the lvs -a option is
352 required to display them:
353
354 lvs -a -o name,segtype,devices
355
356 SubLV names begin with the visible LV name, and have an automatic suf‐
357 fix indicating its role:
358
359
360 · SubLVs holding LV data or parity blocks have the suffix _rimage_#.
361 These SubLVs are sometimes referred to as DataLVs.
362
363
364 · SubLVs holding RAID metadata have the suffix _rmeta_#. RAID meta‐
365 data includes superblock information, RAID type, bitmap, and device
366 health information. These SubLVs are sometimes referred to as Met‐
367 aLVs.
368
369
370 SubLVs are an internal implementation detail of LVM. The way they are
371 used, constructed and named may change.
372
373 The following examples show the SubLV arrangement for each of the basic
374 RAID LV types, using the fewest number of devices allowed for each.
375
376
377 Examples
378 raid0
379 Each rimage SubLV holds a portion of LV data. No parity is used. No
380 RAID metadata is used.
381
382 # lvcreate --type raid0 --stripes 2 --name lvr0 ...
383
384 # lvs -a -o name,segtype,devices
385 lvr0 raid0 lvr0_rimage_0(0),lvr0_rimage_1(0)
386 [lvr0_rimage_0] linear /dev/sda(...)
387 [lvr0_rimage_1] linear /dev/sdb(...)
388
389 raid1
390 Each rimage SubLV holds a complete copy of LV data. No parity is used.
391 Each rmeta SubLV holds RAID metadata.
392
393 # lvcreate --type raid1 --mirrors 1 --name lvr1 ...
394
395 # lvs -a -o name,segtype,devices
396 lvr1 raid1 lvr1_rimage_0(0),lvr1_rimage_1(0)
397 [lvr1_rimage_0] linear /dev/sda(...)
398 [lvr1_rimage_1] linear /dev/sdb(...)
399 [lvr1_rmeta_0] linear /dev/sda(...)
400 [lvr1_rmeta_1] linear /dev/sdb(...)
401
402 raid4
403 At least three rimage SubLVs each hold a portion of LV data and one
404 rimage SubLV holds parity. Each rmeta SubLV holds RAID metadata.
405
406 # lvcreate --type raid4 --stripes 2 --name lvr4 ...
407
408 # lvs -a -o name,segtype,devices
409 lvr4 raid4 lvr4_rimage_0(0),\
410 lvr4_rimage_1(0),\
411 lvr4_rimage_2(0)
412 [lvr4_rimage_0] linear /dev/sda(...)
413 [lvr4_rimage_1] linear /dev/sdb(...)
414 [lvr4_rimage_2] linear /dev/sdc(...)
415 [lvr4_rmeta_0] linear /dev/sda(...)
416 [lvr4_rmeta_1] linear /dev/sdb(...)
417 [lvr4_rmeta_2] linear /dev/sdc(...)
418
419 raid5
420 At least three rimage SubLVs each typcially hold a portion of LV data
421 and parity (see section on raid5) Each rmeta SubLV holds RAID metadata.
422
423 # lvcreate --type raid5 --stripes 2 --name lvr5 ...
424
425 # lvs -a -o name,segtype,devices
426 lvr5 raid5 lvr5_rimage_0(0),\
427 lvr5_rimage_1(0),\
428 lvr5_rimage_2(0)
429 [lvr5_rimage_0] linear /dev/sda(...)
430 [lvr5_rimage_1] linear /dev/sdb(...)
431 [lvr5_rimage_2] linear /dev/sdc(...)
432 [lvr5_rmeta_0] linear /dev/sda(...)
433 [lvr5_rmeta_1] linear /dev/sdb(...)
434 [lvr5_rmeta_2] linear /dev/sdc(...)
435
436 raid6
437 At least five rimage SubLVs each typically hold a portion of LV data
438 and parity. (see section on raid6) Each rmeta SubLV holds RAID meta‐
439 data.
440
441 # lvcreate --type raid6 --stripes 3 --name lvr6
442
443 # lvs -a -o name,segtype,devices
444 lvr6 raid6 lvr6_rimage_0(0),\
445 lvr6_rimage_1(0),\
446 lvr6_rimage_2(0),\
447 lvr6_rimage_3(0),\
448 lvr6_rimage_4(0),\
449 lvr6_rimage_5(0)
450 [lvr6_rimage_0] linear /dev/sda(...)
451 [lvr6_rimage_1] linear /dev/sdb(...)
452 [lvr6_rimage_2] linear /dev/sdc(...)
453 [lvr6_rimage_3] linear /dev/sdd(...)
454 [lvr6_rimage_4] linear /dev/sde(...)
455 [lvr6_rimage_5] linear /dev/sdf(...)
456 [lvr6_rmeta_0] linear /dev/sda(...)
457 [lvr6_rmeta_1] linear /dev/sdb(...)
458 [lvr6_rmeta_2] linear /dev/sdc(...)
459 [lvr6_rmeta_3] linear /dev/sdd(...)
460 [lvr6_rmeta_4] linear /dev/sde(...)
461 [lvr6_rmeta_5] linear /dev/sdf(...)
462
463 raid10
464 At least four rimage SubLVs each hold a portion of LV data. No parity
465 is used. Each rmeta SubLV holds RAID metadata.
466
467 # lvcreate --type raid10 --stripes 2 --mirrors 1 --name lvr10
468
469 # lvs -a -o name,segtype,devices
470 lvr10 raid10 lvr10_rimage_0(0),\
471 lvr10_rimage_1(0),\
472 lvr10_rimage_2(0),\
473 lvr10_rimage_3(0)
474 [lvr10_rimage_0] linear /dev/sda(...)
475 [lvr10_rimage_1] linear /dev/sdb(...)
476 [lvr10_rimage_2] linear /dev/sdc(...)
477 [lvr10_rimage_3] linear /dev/sdd(...)
478 [lvr10_rmeta_0] linear /dev/sda(...)
479 [lvr10_rmeta_1] linear /dev/sdb(...)
480 [lvr10_rmeta_2] linear /dev/sdc(...)
481 [lvr10_rmeta_3] linear /dev/sdd(...)
482
483
484
486 Physical devices in a RAID LV can fail or be lost for multiple reasons.
487 A device could be disconnected, permanently failed, or temporarily dis‐
488 connected. The purpose of RAID LVs (levels 1 and higher) is to con‐
489 tinue operating in a degraded mode, without losing LV data, even after
490 a device fails. The number of devices that can fail without the loss
491 of LV data depends on the RAID level:
492
493
494 · RAID0 (striped) LVs cannot tolerate losing any devices. LV data
495 will be lost if any devices fail.
496
497
498 · RAID1 LVs can tolerate losing all but one device without LV data
499 loss.
500
501
502 · RAID4 and RAID5 LVs can tolerate losing one device without LV data
503 loss.
504
505
506 · RAID6 LVs can tolerate losing two devices without LV data loss.
507
508
509 · RAID10 is variable, and depends on which devices are lost. It
510 stripes across multiple mirror groups with raid1 layout thus it can
511 tolerate losing all but one device in each of these groups without
512 LV data loss.
513
514
515 If a RAID LV is missing devices, or has other device-related problems,
516 lvs reports this in the health_status (and attr) fields:
517
518 lvs -o name,lv_health_status
519
520 partial
521 Devices are missing from the LV. This is also indicated by the letter
522 "p" (partial) in the 9th position of the lvs attr field.
523
524 refresh needed
525 A device was temporarily missing but has returned. The LV needs to be
526 refreshed to use the device again (which will usually require partial
527 synchronization). This is also indicated by the letter "r" (refresh
528 needed) in the 9th position of the lvs attr field. See Refreshing an
529 LV. This could also indicate a problem with the device, in which case
530 it should be be replaced, see Replacing Devices.
531
532 mismatches exist
533 See Scrubbing.
534
535 Most commands will also print a warning if a device is missing, e.g.
536 WARNING: Device for PV uItL3Z-wBME-DQy0-... not found or rejected ...
537
538 This warning will go away if the device returns or is removed from the
539 VG (see vgreduce --removemissing).
540
541
542
543 Activating an LV with missing devices
544 A RAID LV that is missing devices may be activated or not, depending on
545 the "activation mode" used in lvchange:
546
547 lvchange -ay --activationmode complete|degraded|partial LV
548
549 complete
550 The LV is only activated if all devices are present.
551
552 degraded
553 The LV is activated with missing devices if the RAID level can tolerate
554 the number of missing devices without LV data loss.
555
556 partial
557 The LV is always activated, even if portions of the LV data are missing
558 because of the missing device(s). This should only be used to perform
559 extreme recovery or repair operations.
560
561 lvm.conf(5) activation/activation_mode
562 controls the activation mode when not specified by the command.
563
564 The default value is printed by:
565 lvmconfig --type default activation/activation_mode
566
567
568 Replacing Devices
569 Devices in a RAID LV can be replaced by other devices in the VG. When
570 replacing devices that are no longer visible on the system, use lvcon‐
571 vert --repair. When replacing devices that are still visible, use
572 lvconvert --replace. The repair command will attempt to restore the
573 same number of data LVs that were previously in the LV. The replace
574 option can be repeated to replace multiple PVs. Replacement devices
575 can be optionally listed with either option.
576
577 lvconvert --repair LV [NewPVs]
578
579 lvconvert --replace OldPV LV [NewPV]
580
581 lvconvert --replace OldPV1 --replace OldPV2 LV [NewPVs]
582
583 New devices require synchronization with existing devices, see Synchro‐
584 nization.
585
586
587 Refreshing an LV
588 Refreshing a RAID LV clears any transient device failures (device was
589 temporarily disconnected) and returns the LV to its fully redundant
590 mode. Restoring a device will usually require at least partial syn‐
591 chronization (see Synchronization). Failure to clear a transient fail‐
592 ure results in the RAID LV operating in degraded mode until it is reac‐
593 tivated. Use the lvchange command to refresh an LV:
594
595 lvchange --refresh LV
596
597 # lvs -o name,vgname,segtype,attr,size vg
598 LV VG Type Attr LSize
599 lv vg raid1 Rwi-a-r-r- 100.00g
600
601 # lvchange --refresh vg/lv
602
603 # lvs -o name,vgname,segtype,attr,size vg
604 LV VG Type Attr LSize
605 lv vg raid1 Rwi-a-r--- 100.00g
606
607
608 Automatic repair
609 If a device in a RAID LV fails, device-mapper in the kernel notifies
610 the dmeventd(8) monitoring process (see Monitoring). dmeventd can be
611 configured to automatically respond using:
612
613 lvm.conf(5) activation/raid_fault_policy
614
615 Possible settings are:
616
617 warn
618 A warning is added to the system log indicating that a device has
619 failed in the RAID LV. It is left to the user to repair the LV, e.g.
620 replace failed devices.
621
622 allocate
623 dmeventd automatically attempts to repair the LV using spare devices in
624 the VG. Note that even a transient failure is treated as a permanent
625 failure under this setting. A new device is allocated and full syn‐
626 chronization is started.
627
628 The specific command run by dmeventd to warn or repair is:
629 lvconvert --repair --use-policies LV
630
631
632
633 Corrupted Data
634 Data on a device can be corrupted due to hardware errors without the
635 device ever being disconnected or there being any fault in the soft‐
636 ware. This should be rare, and can be detected (see Scrubbing).
637
638
639
640 Rebuild specific PVs
641 If specific PVs in a RAID LV are known to have corrupt data, the data
642 on those PVs can be reconstructed with:
643
644 lvchange --rebuild PV LV
645
646 The rebuild option can be repeated with different PVs to replace the
647 data on multiple PVs.
648
649
650
652 When a RAID LV is activated the dmeventd(8) process is started to moni‐
653 tor the health of the LV. Various events detected in the kernel can
654 cause a notification to be sent from device-mapper to the monitoring
655 process, including device failures and synchronization completion (e.g.
656 for initialization or scrubbing).
657
658 The LVM configuration file contains options that affect how the moni‐
659 toring process will respond to failure events (e.g. raid_fault_policy).
660 It is possible to turn on and off monitoring with lvchange, but it is
661 not recommended to turn this off unless you have a thorough knowledge
662 of the consequences.
663
664
665
667 There are a number of options in the LVM configuration file that affect
668 the behavior of RAID LVs. The tunable options are listed below. A
669 detailed description of each can be found in the LVM configuration file
670 itself.
671 mirror_segtype_default
672 raid10_segtype_default
673 raid_region_size
674 raid_fault_policy
675 activation_mode
676
677
678
680 The device mapper integrity target can be used in combination with RAID
681 levels 1,4,5,6,10 to detect and correct data corruption in RAID images.
682 A dm-integrity layer is placed above each RAID image, and an extra sub
683 LV is created to hold integrity metadata (data checksums) for each RAID
684 image. When data is read from an image, integrity checksums are used
685 to detect corruption. If detected, dm-raid reads the data from another
686 (good) image to return to the caller. dm-raid will also automatically
687 write the good data back to the image with bad data to correct the cor‐
688 ruption.
689
690 When creating a RAID LV with integrity, or adding integrity, space is
691 required for integrity metadata. Every 500MB of LV data requires an
692 additional 4MB to be allocated for integrity metadata, for each RAID
693 image.
694
695 Create a RAID LV with integrity:
696
697 lvcreate --type raidN --raidintegrity y
698
699 Add integrity to an existing RAID LV:
700
701 lvconvert --raidintegrity y LV
702
703 Remove integrity from a RAID LV:
704
705 lvconvert --raidintegrity n LV
706
707
708 Integrity options
709 --raidintegritymode journal|bitmap
710
711 Use a journal (default) or bitmap for keeping integrity checksums con‐
712 sistent in case of a crash. The bitmap areas are recalculated after a
713 crash, so corruption in those areas would not be detected. A journal
714 does not have this problem. The journal mode doubles writes to stor‐
715 age, but can improve performance for scattered writes packed into a
716 single journal write. bitmap mode can in theory achieve full write
717 throughput of the device, but would not benefit from the potential
718 scattered write optimization.
719
720 --raidintegrityblocksize 512|1024|2048|4096
721
722 The block size to use for dm-integrity on raid images. The integrity
723 block size should usually match the device logical block size, or the
724 file system sector/block sizes. It may be less than the file system
725 sector/block size, but not less than the device logical block size.
726 Possible values: 512, 1024, 2048, 4096.
727
728
729 Integrity initialization
730 When integrity is added to an LV, the kernel needs to initialize the
731 integrity metadata (checksums) for all blocks in the LV. The data cor‐
732 ruption checking performed by dm-integrity will only operate on areas
733 of the LV that are already initialized. The progress of integrity ini‐
734 tialization is reported by the "syncpercent" LV reporting field (and
735 under the Cpy%Sync lvs column.)
736
737
738 Integrity limitations
739 To work around some limitations, it is possible to remove integrity
740 from the LV, make the change, then add integrity again. (Integrity
741 metadata would need to initialized when added again.)
742
743 LVM must be able to allocate the integrity metadata sub LV on a single
744 PV that is already in use by the associated RAID image. This can poten‐
745 tially cause a problem during lvextend if the original PV holding the
746 image and integrity metadata is full. To work around this limitation,
747 remove integrity, extend the LV, and add integrity again.
748
749 Additional RAID images can be added to raid1 LVs, but not to other raid
750 levels.
751
752 A raid1 LV with integrity cannot be converted to linear (remove
753 integrity to do this.)
754
755 RAID LVs with integrity cannot yet be used as sub LVs with other LV
756 types.
757
758 The following are not yet permitted on RAID LVs with integrity: lvre‐
759 duce, pvmove, snapshots, splitmirror, raid syncaction commands, raid
760 rebuild.
761
762
764 A RAID1 LV can be tuned so that certain devices are avoided for reading
765 while all devices are still written to.
766
767 lvchange --[raid]writemostly PV[:y|n|t] LV
768
769 The specified device will be marked as "write mostly", which means that
770 reading from this device will be avoided, and other devices will be
771 preferred for reading (unless no other devices are available.) This
772 minimizes the I/O to the specified device.
773
774 If the PV name has no suffix, the write mostly attribute is set. If
775 the PV name has the suffix :n, the write mostly attribute is cleared,
776 and the suffix :t toggles the current setting.
777
778 The write mostly option can be repeated on the command line to change
779 multiple devices at once.
780
781 To report the current write mostly setting, the lvs attr field will
782 show the letter "w" in the 9th position when write mostly is set:
783
784 lvs -a -o name,attr
785
786 When a device is marked write mostly, the maximum number of outstanding
787 writes to that device can be configured. Once the maximum is reached,
788 further writes become synchronous. When synchronous, a write to the LV
789 will not complete until writes to all the mirror images are complete.
790
791 lvchange --[raid]writebehind Number LV
792
793 To report the current write behind setting, run:
794
795 lvs -o name,raid_write_behind
796
797 When write behind is not configured, or set to 0, all LV writes are
798 synchronous.
799
800
801
803 RAID takeover is converting a RAID LV from one RAID level to another,
804 e.g. raid5 to raid6. Changing the RAID level is usually done to
805 increase or decrease resilience to device failures or to restripe LVs.
806 This is done using lvconvert and specifying the new RAID level as the
807 LV type:
808
809 lvconvert --type RaidLevel LV [PVs]
810
811 The most common and recommended RAID takeover conversions are:
812
813
814 linear to raid1
815 Linear is a single image of LV data, and converting it to raid1
816 adds a mirror image which is a direct copy of the original lin‐
817 ear image.
818
819
820 striped/raid0 to raid4/5/6
821 Adding parity devices to a striped volume results in raid4/5/6.
822
823
824 Unnatural conversions that are not recommended include converting
825 between striped and non-striped types. This is because file systems
826 often optimize I/O patterns based on device striping values. If those
827 values change, it can decrease performance.
828
829 Converting to a higher RAID level requires allocating new SubLVs to
830 hold RAID metadata, and new SubLVs to hold parity blocks for LV data.
831 Converting to a lower RAID level removes the SubLVs that are no longer
832 needed.
833
834 Conversion often requires full synchronization of the RAID LV (see Syn‐
835 chronization). Converting to RAID1 requires copying all LV data blocks
836 to N new images on new devices. Converting to a parity RAID level
837 requires reading all LV data blocks, calculating parity, and writing
838 the new parity blocks. Synchronization can take a long time depending
839 on the throughpout of the devices used and the size of the RaidLV. It
840 can degrade performance (rate controls also apply to conversion; see
841 --minrecoveryrate and --maxrecoveryrate.)
842
843 Warning: though it is possible to create striped LVs with up to 128
844 stripes, a maximum of 64 stripes can be converted to raid0, 63 to
845 raid4/5 and 62 to raid6 because of the added parity SubLVs. A striped
846 LV with a maximum of 32 stripes can be converted to raid10.
847
848
849 The following takeover conversions are currently possible:
850
851 · between striped and raid0.
852
853 · between linear and raid1.
854
855 · between mirror and raid1.
856
857 · between raid1 with two images and raid4/5.
858
859 · between striped/raid0 and raid4.
860
861 · between striped/raid0 and raid5.
862
863 · between striped/raid0 and raid6.
864
865 · between raid4 and raid5.
866
867 · between raid4/raid5 and raid6.
868
869 · between striped/raid0 and raid10.
870
871 · between striped and raid4.
872
873
874 Indirect conversions
875 Converting from one raid level to another may require multiple steps,
876 converting first to intermediate raid levels.
877
878 linear to raid6
879
880 To convert an LV from linear to raid6:
881 1. convert to raid1 with two images
882 2. convert to raid5 (internally raid5_ls) with two images
883 3. convert to raid5 with three or more stripes (reshape)
884 4. convert to raid6 (internally raid6_ls_6)
885 5. convert to raid6 (internally raid6_zr, reshape)
886
887 The commands to perform the steps above are:
888 1. lvconvert --type raid1 --mirrors 1 LV
889 2. lvconvert --type raid5 LV
890 3. lvconvert --stripes 3 LV
891 4. lvconvert --type raid6 LV
892 5. lvconvert --type raid6 LV
893
894 The final conversion from raid6_ls_6 to raid6_zr is done to avoid the
895 potential write/recovery performance reduction in raid6_ls_6 because of
896 the dedicated parity device. raid6_zr rotates data and parity blocks
897 to avoid this.
898
899 linear to striped
900
901 To convert an LV from linear to striped:
902 1. convert to raid1 with two images
903 2. convert to raid5_n
904 3. convert to raid5_n with five 128k stripes (reshape)
905 4. convert raid5_n to striped
906
907 The commands to perform the steps above are:
908 1. lvconvert --type raid1 --mirrors 1 LV
909 2. lvconvert --type raid5_n LV
910 3. lvconvert --stripes 5 --stripesize 128k LV
911 4. lvconvert --type striped LV
912
913 The raid5_n type in step 2 is used because it has dedicated parity Sub‐
914 LVs at the end, and can be converted to striped directly. The stripe
915 size is increased in step 3 to add extra space for the conversion
916 process. This step grows the LV size by a factor of five. After con‐
917 version, this extra space can be reduced (or used to grow the file sys‐
918 tem using the LV).
919
920 Reversing these steps will convert a striped LV to linear.
921
922 raid6 to striped
923
924 To convert an LV from raid6_nr to striped:
925 1. convert to raid6_n_6
926 2. convert to striped
927
928 The commands to perform the steps above are:
929 1. lvconvert --type raid6_n_6 LV
930 2. lvconvert --type striped LV
931
932
933
934 Examples
935 Converting an LV from linear to raid1.
936
937 # lvs -a -o name,segtype,size vg
938 LV Type LSize
939 lv linear 300.00g
940
941 # lvconvert --type raid1 --mirrors 1 vg/lv
942
943 # lvs -a -o name,segtype,size vg
944 LV Type LSize
945 lv raid1 300.00g
946 [lv_rimage_0] linear 300.00g
947 [lv_rimage_1] linear 300.00g
948 [lv_rmeta_0] linear 3.00m
949 [lv_rmeta_1] linear 3.00m
950
951 Converting an LV from mirror to raid1.
952
953 # lvs -a -o name,segtype,size vg
954 LV Type LSize
955 lv mirror 100.00g
956 [lv_mimage_0] linear 100.00g
957 [lv_mimage_1] linear 100.00g
958 [lv_mlog] linear 3.00m
959
960 # lvconvert --type raid1 vg/lv
961
962 # lvs -a -o name,segtype,size vg
963 LV Type LSize
964 lv raid1 100.00g
965 [lv_rimage_0] linear 100.00g
966 [lv_rimage_1] linear 100.00g
967 [lv_rmeta_0] linear 3.00m
968 [lv_rmeta_1] linear 3.00m
969
970 Converting an LV from linear to raid1 (with 3 images).
971
972 # lvconvert --type raid1 --mirrors 2 vg/lv
973
974 Converting an LV from striped (with 4 stripes) to raid6_n_6.
975
976 # lvcreate --stripes 4 -L64M -n lv vg
977
978 # lvconvert --type raid6 vg/lv
979
980 # lvs -a -o lv_name,segtype,sync_percent,data_copies
981 LV Type Cpy%Sync #Cpy
982 lv raid6_n_6 100.00 3
983 [lv_rimage_0] linear
984 [lv_rimage_1] linear
985 [lv_rimage_2] linear
986 [lv_rimage_3] linear
987 [lv_rimage_4] linear
988 [lv_rimage_5] linear
989 [lv_rmeta_0] linear
990 [lv_rmeta_1] linear
991 [lv_rmeta_2] linear
992 [lv_rmeta_3] linear
993 [lv_rmeta_4] linear
994 [lv_rmeta_5] linear
995
996 This convert begins by allocating MetaLVs (rmeta_#) for each of the
997 existing stripe devices. It then creates 2 additional MetaLV/DataLV
998 pairs (rmeta_#/rimage_#) for dedicated raid6 parity.
999
1000 If rotating data/parity is required, such as with raid6_nr, it must be
1001 done by reshaping (see below).
1002
1003
1004
1006 RAID reshaping is changing attributes of a RAID LV while keeping the
1007 same RAID level. This includes changing RAID layout, stripe size, or
1008 number of stripes.
1009
1010 When changing the RAID layout or stripe size, no new SubLVs (MetaLVs or
1011 DataLVs) need to be allocated, but DataLVs are extended by a small
1012 amount (typically 1 extent). The extra space allows blocks in a stripe
1013 to be updated safely, and not be corrupted in case of a crash. If a
1014 crash occurs, reshaping can just be restarted.
1015
1016 (If blocks in a stripe were updated in place, a crash could leave them
1017 partially updated and corrupted. Instead, an existing stripe is qui‐
1018 esced, read, changed in layout, and the new stripe written to free
1019 space. Once that is done, the new stripe is unquiesced and used.)
1020
1021
1022 Examples
1023 (Command output shown in examples may change.)
1024
1025 Converting raid6_n_6 to raid6_nr with rotating data/parity.
1026
1027 This conversion naturally follows a previous conversion from
1028 striped/raid0 to raid6_n_6 (shown above). It completes the transition
1029 to a more traditional RAID6.
1030
1031 # lvs -o lv_name,segtype,sync_percent,data_copies
1032 LV Type Cpy%Sync #Cpy
1033 lv raid6_n_6 100.00 3
1034 [lv_rimage_0] linear
1035 [lv_rimage_1] linear
1036 [lv_rimage_2] linear
1037 [lv_rimage_3] linear
1038 [lv_rimage_4] linear
1039 [lv_rimage_5] linear
1040 [lv_rmeta_0] linear
1041 [lv_rmeta_1] linear
1042 [lv_rmeta_2] linear
1043 [lv_rmeta_3] linear
1044 [lv_rmeta_4] linear
1045 [lv_rmeta_5] linear
1046
1047 # lvconvert --type raid6_nr vg/lv
1048
1049 # lvs -a -o lv_name,segtype,sync_percent,data_copies
1050 LV Type Cpy%Sync #Cpy
1051 lv raid6_nr 100.00 3
1052 [lv_rimage_0] linear
1053 [lv_rimage_0] linear
1054 [lv_rimage_1] linear
1055 [lv_rimage_1] linear
1056 [lv_rimage_2] linear
1057 [lv_rimage_2] linear
1058 [lv_rimage_3] linear
1059 [lv_rimage_3] linear
1060 [lv_rimage_4] linear
1061 [lv_rimage_5] linear
1062 [lv_rmeta_0] linear
1063 [lv_rmeta_1] linear
1064 [lv_rmeta_2] linear
1065 [lv_rmeta_3] linear
1066 [lv_rmeta_4] linear
1067 [lv_rmeta_5] linear
1068
1069 The DataLVs are larger (additional segment in each) which provides
1070 space for out-of-place reshaping. The result is:
1071
1072 # lvs -a -o lv_name,segtype,seg_pe_ranges,dataoffset
1073 LV Type PE Ranges DOff
1074 lv raid6_nr lv_rimage_0:0-32 \
1075 lv_rimage_1:0-32 \
1076 lv_rimage_2:0-32 \
1077 lv_rimage_3:0-32
1078 [lv_rimage_0] linear /dev/sda:0-31 2048
1079 [lv_rimage_0] linear /dev/sda:33-33
1080 [lv_rimage_1] linear /dev/sdaa:0-31 2048
1081 [lv_rimage_1] linear /dev/sdaa:33-33
1082 [lv_rimage_2] linear /dev/sdab:1-33 2048
1083 [lv_rimage_3] linear /dev/sdac:1-33 2048
1084 [lv_rmeta_0] linear /dev/sda:32-32
1085 [lv_rmeta_1] linear /dev/sdaa:32-32
1086 [lv_rmeta_2] linear /dev/sdab:0-0
1087 [lv_rmeta_3] linear /dev/sdac:0-0
1088
1089 All segments with PE ranges '33-33' provide the out-of-place reshape
1090 space. The dataoffset column shows that the data was moved from ini‐
1091 tial offset 0 to 2048 sectors on each component DataLV.
1092
1093 For performance reasons the raid6_nr RaidLV can be restriped. Convert
1094 it from 3-way striped to 5-way-striped.
1095
1096 # lvconvert --stripes 5 vg/lv
1097 Using default stripesize 64.00 KiB.
1098 WARNING: Adding stripes to active logical volume vg/lv will \
1099 grow it from 99 to 165 extents!
1100 Run "lvresize -l99 vg/lv" to shrink it or use the additional \
1101 capacity.
1102 Logical volume vg/lv successfully converted.
1103
1104 # lvs vg/lv
1105 LV VG Attr LSize Cpy%Sync
1106 lv vg rwi-a-r-s- 652.00m 52.94
1107
1108 # lvs -a -o lv_name,attr,segtype,seg_pe_ranges,dataoffset vg
1109 LV Attr Type PE Ranges DOff
1110 lv rwi-a-r--- raid6_nr lv_rimage_0:0-33 \
1111 lv_rimage_1:0-33 \
1112 lv_rimage_2:0-33 ... \
1113 lv_rimage_5:0-33 \
1114 lv_rimage_6:0-33 0
1115 [lv_rimage_0] iwi-aor--- linear /dev/sda:0-32 0
1116 [lv_rimage_0] iwi-aor--- linear /dev/sda:34-34
1117 [lv_rimage_1] iwi-aor--- linear /dev/sdaa:0-32 0
1118 [lv_rimage_1] iwi-aor--- linear /dev/sdaa:34-34
1119 [lv_rimage_2] iwi-aor--- linear /dev/sdab:0-32 0
1120 [lv_rimage_2] iwi-aor--- linear /dev/sdab:34-34
1121 [lv_rimage_3] iwi-aor--- linear /dev/sdac:1-34 0
1122 [lv_rimage_4] iwi-aor--- linear /dev/sdad:1-34 0
1123 [lv_rimage_5] iwi-aor--- linear /dev/sdae:1-34 0
1124 [lv_rimage_6] iwi-aor--- linear /dev/sdaf:1-34 0
1125 [lv_rmeta_0] ewi-aor--- linear /dev/sda:33-33
1126 [lv_rmeta_1] ewi-aor--- linear /dev/sdaa:33-33
1127 [lv_rmeta_2] ewi-aor--- linear /dev/sdab:33-33
1128 [lv_rmeta_3] ewi-aor--- linear /dev/sdac:0-0
1129 [lv_rmeta_4] ewi-aor--- linear /dev/sdad:0-0
1130 [lv_rmeta_5] ewi-aor--- linear /dev/sdae:0-0
1131 [lv_rmeta_6] ewi-aor--- linear /dev/sdaf:0-0
1132
1133 Stripes also can be removed from raid5 and 6. Convert the 5-way
1134 striped raid6_nr LV to 4-way-striped. The force option needs to be
1135 used, because removing stripes (i.e. image SubLVs) from a RaidLV will
1136 shrink its size.
1137
1138 # lvconvert --stripes 4 vg/lv
1139 Using default stripesize 64.00 KiB.
1140 WARNING: Removing stripes from active logical volume vg/lv will \
1141 shrink it from 660.00 MiB to 528.00 MiB!
1142 THIS MAY DESTROY (PARTS OF) YOUR DATA!
1143 If that leaves the logical volume larger than 206 extents due \
1144 to stripe rounding,
1145 you may want to grow the content afterwards (filesystem etc.)
1146 WARNING: to remove freed stripes after the conversion has finished,\
1147 you have to run "lvconvert --stripes 4 vg/lv"
1148 Logical volume vg/lv successfully converted.
1149
1150 # lvs -a -o lv_name,attr,segtype,seg_pe_ranges,dataoffset vg
1151 LV Attr Type PE Ranges DOff
1152 lv rwi-a-r-s- raid6_nr lv_rimage_0:0-33 \
1153 lv_rimage_1:0-33 \
1154 lv_rimage_2:0-33 ... \
1155 lv_rimage_5:0-33 \
1156 lv_rimage_6:0-33 0
1157 [lv_rimage_0] Iwi-aor--- linear /dev/sda:0-32 0
1158 [lv_rimage_0] Iwi-aor--- linear /dev/sda:34-34
1159 [lv_rimage_1] Iwi-aor--- linear /dev/sdaa:0-32 0
1160 [lv_rimage_1] Iwi-aor--- linear /dev/sdaa:34-34
1161 [lv_rimage_2] Iwi-aor--- linear /dev/sdab:0-32 0
1162 [lv_rimage_2] Iwi-aor--- linear /dev/sdab:34-34
1163 [lv_rimage_3] Iwi-aor--- linear /dev/sdac:1-34 0
1164 [lv_rimage_4] Iwi-aor--- linear /dev/sdad:1-34 0
1165 [lv_rimage_5] Iwi-aor--- linear /dev/sdae:1-34 0
1166 [lv_rimage_6] Iwi-aor-R- linear /dev/sdaf:1-34 0
1167 [lv_rmeta_0] ewi-aor--- linear /dev/sda:33-33
1168 [lv_rmeta_1] ewi-aor--- linear /dev/sdaa:33-33
1169 [lv_rmeta_2] ewi-aor--- linear /dev/sdab:33-33
1170 [lv_rmeta_3] ewi-aor--- linear /dev/sdac:0-0
1171 [lv_rmeta_4] ewi-aor--- linear /dev/sdad:0-0
1172 [lv_rmeta_5] ewi-aor--- linear /dev/sdae:0-0
1173 [lv_rmeta_6] ewi-aor-R- linear /dev/sdaf:0-0
1174
1175 The 's' in column 9 of the attribute field shows the RaidLV is still
1176 reshaping. The 'R' in the same column of the attribute field shows the
1177 freed image Sub LVs which will need removing once the reshaping fin‐
1178 ished.
1179
1180 # lvs -o lv_name,attr,segtype,seg_pe_ranges,dataoffset vg
1181 LV Attr Type PE Ranges DOff
1182 lv rwi-a-r-R- raid6_nr lv_rimage_0:0-33 \
1183 lv_rimage_1:0-33 \
1184 lv_rimage_2:0-33 ... \
1185 lv_rimage_5:0-33 \
1186 lv_rimage_6:0-33 8192
1187
1188 Now that the reshape is finished the 'R' atribute on the RaidLV shows
1189 images can be removed.
1190
1191 # lvs -o lv_name,attr,segtype,seg_pe_ranges,dataoffset vg
1192 LV Attr Type PE Ranges DOff
1193 lv rwi-a-r-R- raid6_nr lv_rimage_0:0-33 \
1194 lv_rimage_1:0-33 \
1195 lv_rimage_2:0-33 ... \
1196 lv_rimage_5:0-33 \
1197 lv_rimage_6:0-33 8192
1198
1199 This is achieved by repeating the command ("lvconvert --stripes 4
1200 vg/lv" would be sufficient).
1201
1202 # lvconvert --stripes 4 vg/lv
1203 Using default stripesize 64.00 KiB.
1204 Logical volume vg/lv successfully converted.
1205
1206 # lvs -a -o lv_name,attr,segtype,seg_pe_ranges,dataoffset vg
1207 LV Attr Type PE Ranges DOff
1208 lv rwi-a-r--- raid6_nr lv_rimage_0:0-33 \
1209 lv_rimage_1:0-33 \
1210 lv_rimage_2:0-33 ... \
1211 lv_rimage_5:0-33 8192
1212 [lv_rimage_0] iwi-aor--- linear /dev/sda:0-32 8192
1213 [lv_rimage_0] iwi-aor--- linear /dev/sda:34-34
1214 [lv_rimage_1] iwi-aor--- linear /dev/sdaa:0-32 8192
1215 [lv_rimage_1] iwi-aor--- linear /dev/sdaa:34-34
1216 [lv_rimage_2] iwi-aor--- linear /dev/sdab:0-32 8192
1217 [lv_rimage_2] iwi-aor--- linear /dev/sdab:34-34
1218 [lv_rimage_3] iwi-aor--- linear /dev/sdac:1-34 8192
1219 [lv_rimage_4] iwi-aor--- linear /dev/sdad:1-34 8192
1220 [lv_rimage_5] iwi-aor--- linear /dev/sdae:1-34 8192
1221 [lv_rmeta_0] ewi-aor--- linear /dev/sda:33-33
1222 [lv_rmeta_1] ewi-aor--- linear /dev/sdaa:33-33
1223 [lv_rmeta_2] ewi-aor--- linear /dev/sdab:33-33
1224 [lv_rmeta_3] ewi-aor--- linear /dev/sdac:0-0
1225 [lv_rmeta_4] ewi-aor--- linear /dev/sdad:0-0
1226 [lv_rmeta_5] ewi-aor--- linear /dev/sdae:0-0
1227
1228 # lvs -a -o lv_name,attr,segtype,reshapelen vg
1229 LV Attr Type RSize
1230 lv rwi-a-r--- raid6_nr 24.00m
1231 [lv_rimage_0] iwi-aor--- linear 4.00m
1232 [lv_rimage_0] iwi-aor--- linear
1233 [lv_rimage_1] iwi-aor--- linear 4.00m
1234 [lv_rimage_1] iwi-aor--- linear
1235 [lv_rimage_2] iwi-aor--- linear 4.00m
1236 [lv_rimage_2] iwi-aor--- linear
1237 [lv_rimage_3] iwi-aor--- linear 4.00m
1238 [lv_rimage_4] iwi-aor--- linear 4.00m
1239 [lv_rimage_5] iwi-aor--- linear 4.00m
1240 [lv_rmeta_0] ewi-aor--- linear
1241 [lv_rmeta_1] ewi-aor--- linear
1242 [lv_rmeta_2] ewi-aor--- linear
1243 [lv_rmeta_3] ewi-aor--- linear
1244 [lv_rmeta_4] ewi-aor--- linear
1245 [lv_rmeta_5] ewi-aor--- linear
1246
1247 Future developments might include automatic removal of the freed
1248 images.
1249
1250 If the reshape space shall be removed any lvconvert command not chang‐
1251 ing the layout can be used:
1252
1253 # lvconvert --stripes 4 vg/lv
1254 Using default stripesize 64.00 KiB.
1255 No change in RAID LV vg/lv layout, freeing reshape space.
1256 Logical volume vg/lv successfully converted.
1257
1258 # lvs -a -o lv_name,attr,segtype,reshapelen vg
1259 LV Attr Type RSize
1260 lv rwi-a-r--- raid6_nr 0
1261 [lv_rimage_0] iwi-aor--- linear 0
1262 [lv_rimage_0] iwi-aor--- linear
1263 [lv_rimage_1] iwi-aor--- linear 0
1264 [lv_rimage_1] iwi-aor--- linear
1265 [lv_rimage_2] iwi-aor--- linear 0
1266 [lv_rimage_2] iwi-aor--- linear
1267 [lv_rimage_3] iwi-aor--- linear 0
1268 [lv_rimage_4] iwi-aor--- linear 0
1269 [lv_rimage_5] iwi-aor--- linear 0
1270 [lv_rmeta_0] ewi-aor--- linear
1271 [lv_rmeta_1] ewi-aor--- linear
1272 [lv_rmeta_2] ewi-aor--- linear
1273 [lv_rmeta_3] ewi-aor--- linear
1274 [lv_rmeta_4] ewi-aor--- linear
1275 [lv_rmeta_5] ewi-aor--- linear
1276
1277 In case the RaidLV should be converted to striped:
1278
1279 # lvconvert --type striped vg/lv
1280 Unable to convert LV vg/lv from raid6_nr to striped.
1281 Converting vg/lv from raid6_nr is directly possible to the \
1282 following layouts:
1283 raid6_nc
1284 raid6_zr
1285 raid6_la_6
1286 raid6_ls_6
1287 raid6_ra_6
1288 raid6_rs_6
1289 raid6_n_6
1290
1291 A direct conversion isn't possible thus the command informed about the
1292 possible ones. raid6_n_6 is suitable to convert to striped so convert
1293 to it first (this is a reshape changing the raid6 layout from raid6_nr
1294 to raid6_n_6).
1295
1296 # lvconvert --type raid6_n_6
1297 Using default stripesize 64.00 KiB.
1298 Converting raid6_nr LV vg/lv to raid6_n_6.
1299 Are you sure you want to convert raid6_nr LV vg/lv? [y/n]: y
1300 Logical volume vg/lv successfully converted.
1301
1302 Wait for the reshape to finish.
1303
1304 # lvconvert --type striped vg/lv
1305 Logical volume vg/lv successfully converted.
1306
1307 # lvs -o lv_name,attr,segtype,seg_pe_ranges,dataoffset vg
1308 LV Attr Type PE Ranges DOff
1309 lv -wi-a----- striped /dev/sda:2-32 \
1310 /dev/sdaa:2-32 \
1311 /dev/sdab:2-32 \
1312 /dev/sdac:3-33
1313 lv -wi-a----- striped /dev/sda:34-35 \
1314 /dev/sdaa:34-35 \
1315 /dev/sdab:34-35 \
1316 /dev/sdac:34-35
1317
1318 From striped we can convert to raid10
1319
1320 # lvconvert --type raid10 vg/lv
1321 Using default stripesize 64.00 KiB.
1322 Logical volume vg/lv successfully converted.
1323
1324 # lvs -o lv_name,attr,segtype,seg_pe_ranges,dataoffset vg
1325 LV Attr Type PE Ranges DOff
1326 lv rwi-a-r--- raid10 lv_rimage_0:0-32 \
1327 lv_rimage_4:0-32 \
1328 lv_rimage_1:0-32 ... \
1329 lv_rimage_3:0-32 \
1330 lv_rimage_7:0-32 0
1331
1332 # lvs -a -o lv_name,attr,segtype,seg_pe_ranges,dataoffset vg
1333 WARNING: Cannot find matching striped segment for vg/lv_rimage_3.
1334 LV Attr Type PE Ranges DOff
1335 lv rwi-a-r--- raid10 lv_rimage_0:0-32 \
1336 lv_rimage_4:0-32 \
1337 lv_rimage_1:0-32 ... \
1338 lv_rimage_3:0-32 \
1339 lv_rimage_7:0-32 0
1340 [lv_rimage_0] iwi-aor--- linear /dev/sda:2-32 0
1341 [lv_rimage_0] iwi-aor--- linear /dev/sda:34-35
1342 [lv_rimage_1] iwi-aor--- linear /dev/sdaa:2-32 0
1343 [lv_rimage_1] iwi-aor--- linear /dev/sdaa:34-35
1344 [lv_rimage_2] iwi-aor--- linear /dev/sdab:2-32 0
1345 [lv_rimage_2] iwi-aor--- linear /dev/sdab:34-35
1346 [lv_rimage_3] iwi-XXr--- linear /dev/sdac:3-35 0
1347 [lv_rimage_4] iwi-aor--- linear /dev/sdad:1-33 0
1348 [lv_rimage_5] iwi-aor--- linear /dev/sdae:1-33 0
1349 [lv_rimage_6] iwi-aor--- linear /dev/sdaf:1-33 0
1350 [lv_rimage_7] iwi-aor--- linear /dev/sdag:1-33 0
1351 [lv_rmeta_0] ewi-aor--- linear /dev/sda:0-0
1352 [lv_rmeta_1] ewi-aor--- linear /dev/sdaa:0-0
1353 [lv_rmeta_2] ewi-aor--- linear /dev/sdab:0-0
1354 [lv_rmeta_3] ewi-aor--- linear /dev/sdac:0-0
1355 [lv_rmeta_4] ewi-aor--- linear /dev/sdad:0-0
1356 [lv_rmeta_5] ewi-aor--- linear /dev/sdae:0-0
1357 [lv_rmeta_6] ewi-aor--- linear /dev/sdaf:0-0
1358 [lv_rmeta_7] ewi-aor--- linear /dev/sdag:0-0
1359
1360 raid10 allows to add stripes but can't remove them.
1361
1362
1363 A more elaborate example to convert from linear to striped with interim
1364 conversions to raid1 then raid5 followed by restripe (4 steps).
1365
1366 We start with the linear LV.
1367
1368 # lvs -a -o name,size,segtype,syncpercent,datastripes,\
1369 stripesize,reshapelenle,devices vg
1370 LV LSize Type Cpy%Sync #DStr Stripe RSize Devices
1371 lv 128.00m linear 1 0 /dev/sda(0)
1372
1373 Then convert it to a 2-way raid1.
1374
1375 # lvconvert --mirrors 1 vg/lv
1376 Logical volume vg/lv successfully converted.
1377
1378 # lvs -a -o name,size,segtype,datastripes,\
1379 stripesize,reshapelenle,devices vg
1380 LV LSize Type #DStr Stripe RSize Devices
1381 lv 128.00m raid1 2 0 lv_rimage_0(0),\
1382 lv_rimage_1(0)
1383 [lv_rimage_0] 128.00m linear 1 0 /dev/sda(0)
1384 [lv_rimage_1] 128.00m linear 1 0 /dev/sdhx(1)
1385 [lv_rmeta_0] 4.00m linear 1 0 /dev/sda(32)
1386 [lv_rmeta_1] 4.00m linear 1 0 /dev/sdhx(0)
1387
1388 Once the raid1 LV is fully synchronized we convert it to raid5_n (only
1389 2-way raid1 LVs can be converted to raid5). We select raid5_n here
1390 because it has dedicated parity SubLVs at the end and can be converted
1391 to striped directly without any additional conversion.
1392
1393 # lvconvert --type raid5_n vg/lv
1394 Using default stripesize 64.00 KiB.
1395 Logical volume vg/lv successfully converted.
1396
1397 # lvs -a -o name,size,segtype,syncpercent,datastripes,\
1398 stripesize,reshapelenle,devices vg
1399 LV LSize Type #DStr Stripe RSize Devices
1400 lv 128.00m raid5_n 1 64.00k 0 lv_rimage_0(0),\
1401 lv_rimage_1(0)
1402 [lv_rimage_0] 128.00m linear 1 0 0 /dev/sda(0)
1403 [lv_rimage_1] 128.00m linear 1 0 0 /dev/sdhx(1)
1404 [lv_rmeta_0] 4.00m linear 1 0 /dev/sda(32)
1405 [lv_rmeta_1] 4.00m linear 1 0 /dev/sdhx(0)
1406
1407 Now we'll change the number of data stripes from 1 to 5 and request
1408 128K stripe size in one command. This will grow the size of the LV by
1409 a factor of 5 (we add 4 data stripes to the one given). That additonal
1410 space can be used by e.g. growing any contained filesystem or the LV
1411 can be reduced in size after the reshaping conversion has finished.
1412
1413 # lvconvert --stripesize 128k --stripes 5 vg/lv
1414 Converting stripesize 64.00 KiB of raid5_n LV vg/lv to 128.00 KiB.
1415 WARNING: Adding stripes to active logical volume vg/lv will grow \
1416 it from 32 to 160 extents!
1417 Run "lvresize -l32 vg/lv" to shrink it or use the additional capacity.
1418 Logical volume vg/lv successfully converted.
1419
1420 # lvs -a -o name,size,segtype,datastripes,\
1421 stripesize,reshapelenle,devices
1422 LV LSize Type #DStr Stripe RSize Devices
1423 lv 640.00m raid5_n 5 128.00k 6 lv_rimage_0(0),\
1424 lv_rimage_1(0),\
1425 lv_rimage_2(0),\
1426 lv_rimage_3(0),\
1427 lv_rimage_4(0),\
1428 lv_rimage_5(0)
1429 [lv_rimage_0] 132.00m linear 1 0 1 /dev/sda(33)
1430 [lv_rimage_0] 132.00m linear 1 0 /dev/sda(0)
1431 [lv_rimage_1] 132.00m linear 1 0 1 /dev/sdhx(33)
1432 [lv_rimage_1] 132.00m linear 1 0 /dev/sdhx(1)
1433 [lv_rimage_2] 132.00m linear 1 0 1 /dev/sdhw(33)
1434 [lv_rimage_2] 132.00m linear 1 0 /dev/sdhw(1)
1435 [lv_rimage_3] 132.00m linear 1 0 1 /dev/sdhv(33)
1436 [lv_rimage_3] 132.00m linear 1 0 /dev/sdhv(1)
1437 [lv_rimage_4] 132.00m linear 1 0 1 /dev/sdhu(33)
1438 [lv_rimage_4] 132.00m linear 1 0 /dev/sdhu(1)
1439 [lv_rimage_5] 132.00m linear 1 0 1 /dev/sdht(33)
1440 [lv_rimage_5] 132.00m linear 1 0 /dev/sdht(1)
1441 [lv_rmeta_0] 4.00m linear 1 0 /dev/sda(32)
1442 [lv_rmeta_1] 4.00m linear 1 0 /dev/sdhx(0)
1443 [lv_rmeta_2] 4.00m linear 1 0 /dev/sdhw(0)
1444 [lv_rmeta_3] 4.00m linear 1 0 /dev/sdhv(0)
1445 [lv_rmeta_4] 4.00m linear 1 0 /dev/sdhu(0)
1446 [lv_rmeta_5] 4.00m linear 1 0 /dev/sdht(0)
1447
1448 Once the conversion has finished we can can convert to striped.
1449
1450 # lvconvert --type striped vg/lv
1451 Logical volume vg/lv successfully converted.
1452
1453 # lvs -a -o name,size,segtype,datastripes,\
1454 stripesize,reshapelenle,devices vg
1455 LV LSize Type #DStr Stripe RSize Devices
1456 lv 640.00m striped 5 128.00k /dev/sda(33),\
1457 /dev/sdhx(33),\
1458 /dev/sdhw(33),\
1459 /dev/sdhv(33),\
1460 /dev/sdhu(33)
1461 lv 640.00m striped 5 128.00k /dev/sda(0),\
1462 /dev/sdhx(1),\
1463 /dev/sdhw(1),\
1464 /dev/sdhv(1),\
1465 /dev/sdhu(1)
1466
1467 Reversing these steps will convert a given striped LV to linear.
1468
1469 Mind the facts that stripes are removed thus the capacity of the RaidLV
1470 will shrink and that changing the RaidLV layout will influence its per‐
1471 formance.
1472
1473 "lvconvert --stripes 1 vg/lv" for converting to 1 stripe will inform
1474 upfront about the reduced size to allow for resizing the content or
1475 growing the RaidLV before actually converting to 1 stripe. The --force
1476 option is needed to allow stripe removing conversions to prevent data
1477 loss.
1478
1479 Of course any interim step can be the intended last one (e.g. striped->
1480 raid1).
1481
1482
1484 raid5_ls
1485 · RAID5 left symmetric
1486 · Rotating parity N with data restart
1487
1488 raid5_la
1489 · RAID5 left symmetric
1490 · Rotating parity N with data continuation
1491
1492 raid5_rs
1493 · RAID5 right symmetric
1494 · Rotating parity 0 with data restart
1495
1496 raid5_ra
1497 · RAID5 right asymmetric
1498 · Rotating parity 0 with data continuation
1499
1500 raid5_n
1501 · RAID5 parity n
1502 · Dedicated parity device n used for striped/raid0 conversions
1503 · Used for RAID Takeover
1504
1505
1507 raid6
1508 · RAID6 zero restart (aka left symmetric)
1509 · Rotating parity 0 with data restart
1510 · Same as raid6_zr
1511
1512 raid6_zr
1513 · RAID6 zero restart (aka left symmetric)
1514 · Rotating parity 0 with data restart
1515
1516 raid6_nr
1517 · RAID6 N restart (aka right symmetric)
1518 · Rotating parity N with data restart
1519
1520 raid6_nc
1521 · RAID6 N continue
1522 · Rotating parity N with data continuation
1523
1524 raid6_n_6
1525 · RAID6 last parity devices
1526 · Fixed dedicated last devices (P-Syndrome N-1 and Q-Syndrome N)
1527 with striped data used for striped/raid0 conversions
1528 · Used for RAID Takeover
1529
1530 raid6_{ls,rs,la,ra}_6
1531 · RAID6 last parity device
1532 · Dedicated last parity device used for conversions from/to
1533 raid5_{ls,rs,la,ra}
1534
1535 raid6_ls_6
1536 · RAID6 N continue
1537 · Same as raid5_ls for N-1 devices with fixed Q-Syndrome N
1538 · Used for RAID Takeover
1539
1540 raid6_la_6
1541 · RAID6 N continue
1542 · Same as raid5_la for N-1 devices with fixed Q-Syndrome N
1543 · Used forRAID Takeover
1544
1545 raid6_rs_6
1546 · RAID6 N continue
1547 · Same as raid5_rs for N-1 devices with fixed Q-Syndrome N
1548 · Used for RAID Takeover
1549
1550 raid6_ra_6
1551 · RAID6 N continue
1552 · ame as raid5_ra for N-1 devices with fixed Q-Syndrome N
1553 · Used for RAID Takeover
1554
1555
1556
1557
1559 The 2.6.38-rc1 version of the Linux kernel introduced a device-mapper
1560 target to interface with the software RAID (MD) personalities. This
1561 provided device-mapper with RAID 4/5/6 capabilities and a larger devel‐
1562 opment community. Later, support for RAID1, RAID10, and RAID1E (RAID
1563 10 variants) were added. Support for these new kernel RAID targets was
1564 added to LVM version 2.02.87. The capabilities of the LVM raid1 type
1565 have surpassed the old mirror type. raid1 is now recommended instead
1566 of mirror. raid1 became the default for mirroring in LVM version
1567 2.02.100.
1568
1569
1570
1571
1572Red Hat, Inc LVM TOOLS 2.03.10(2) (2020-08-09) LVMRAID(7)