1MULTIPATH.CONF(5) File Formats Manual MULTIPATH.CONF(5)
2
3
4
6 multipath.conf - multipath daemon configuration file.
7
9 /etc/multipath.conf is the configuration file for the multipath daemon.
10 It is used to overwrite the built-in configuration table of multipathd.
11 Any line whose first non-white-space character is a '#' is considered a
12 comment line. Empty lines are ignored.
13
14 Currently used multipathd configuration can be displayed with the mul‐
15 tipath -t or multipathd show config command.
16
18 The configuration file contains entries of the form:
19
20 <section> {
21 <attribute> <value>
22 ...
23 <subsection> {
24 <attribute> <value>
25 ...
26 }
27 }
28
29 Each section contains one or more attributes or subsections. The recog‐
30 nized keywords for attributes or subsections depend on the section in
31 which they occur.
32
33 <attribute> and <value> must be on a single line. <attribute> is one
34 of the keywords listed in this man page. <value> is either a simple
35 word (containing no whitespace and none of the characters '"', '#', and
36 '!') or one string enclosed in double quotes ("..."). Outside a quoted
37 string, text starting with '#', and '!' is regarded as a comment and
38 ignored until the end of the line. Inside a quoted string, '#' and '!'
39 are normal characters, and whitespace is preserved. To represent a
40 double quote character inside a double quoted string, use two consecu‐
41 tive double quotes ('""'). Thus '2.5" SSD' can be written as "2.5""
42 SSD".
43
44 Opening braces ('{') must follow the (sub)section name on the same
45 line. Closing braces ('}') that mark the end of a (sub)section must be
46 the only non-whitespace character on the line. Whitespace is ignored
47 except inside double quotes, thus the indentation shown in the above
48 example is helpful for human readers but not mandatory.
49
50 Note on regular expressions: The multipath.conf syntax allows many at‐
51 tribute values to be specified as POSIX Extended Regular Expressions
52 (see regex(7)). These regular expressions are case sensitive and not
53 anchored, thus the expression "bar" matches "barbie", "rhabarber", and
54 "wunderbar", but not "Barbie". To avoid unwanted substring matches,
55 standard regular expression syntax using the special characters "^" and
56 "$" can be used.
57
58 The following section keywords are recognized:
59
60 defaults This section defines default values for attributes
61 which are used whenever no values are given in the ap‐
62 propriate device or multipath sections.
63
64 blacklist This section defines which devices should be excluded
65 from the multipath topology discovery.
66
67 blacklist_exceptions
68 This section defines which devices should be included
69 in the multipath topology discovery, despite being
70 listed in the blacklist section.
71
72 multipaths This section defines the multipath topologies. They
73 are indexed by a World Wide Identifier(WWID). For de‐
74 tails on the WWID generation see section WWID genera‐
75 tion below. Attributes set in this section take prece‐
76 dence over all others.
77
78 devices This section defines the device-specific settings. De‐
79 vices are identified by vendor, product, and revision.
80
81 overrides This section defines values for attributes that should
82 override the device-specific settings for all devices.
83
85 The defaults section recognizes the following keywords:
86
87 verbosity Default verbosity. Higher values increase the ver‐
88 bosity level. Valid levels are between 0 and 6.
89
90 The default is: 2
91
92 polling_interval Interval between two path checks in seconds. For prop‐
93 erly functioning paths, the interval between checks
94 will gradually increase to max_polling_interval. This
95 value will be overridden by the WatchdogSec setting in
96 the multipathd.service definition if systemd is used.
97
98 The default is: 5
99
100 max_polling_interval
101 Maximal interval between two path checks in seconds.
102
103 The default is: 4 * polling_interval
104
105 reassign_maps Enable reassigning of device-mapper maps. With this
106 option multipathd will remap existing device-mapper
107 maps to always point to multipath device, not the un‐
108 derlying block devices. Possible values are yes and
109 no.
110
111 The default is: no
112
113 multipath_dir This option is deprecated, and will be removed in a
114 future release. Directory where the dynamic shared
115 objects are stored. Defined at compile time, commonly
116 /lib64/multipath/ or /lib/multipath/.
117
118 The default is: <system dependent>
119
120 path_selector The default path selector algorithm to use; they are
121 offered by the kernel multipath target:
122
123 round-robin 0
124 Loop through every path in the path group,
125 sending the same amount of I/O to each.
126 Some aspects of behavior can be controlled
127 with the attributes: rr_min_io,
128 rr_min_io_rq and rr_weight.
129
130 queue-length 0
131 (Since 2.6.31 kernel) Choose the path for
132 the next bunch of I/O based on the amount
133 of outstanding I/O to the path.
134
135 service-time 0
136 (Since 2.6.31 kernel) Choose the path for
137 the next bunch of I/O based on the amount
138 of outstanding I/O to the path and its
139 relative throughput.
140
141 historical-service-time 0
142 (Since 5.8 kernel) Choose the path for the
143 next bunch of I/O based on the estimation
144 of future service time based on the his‐
145 tory of previous I/O submitted to each
146 path.
147
148 io-affinity 0
149 (Since 5.11 kernel) Choose the path for
150 the next bunch of I/O based on a CPU to
151 path mapping the user passes in and what
152 CPU we are executing on.
153
154 The default is: service-time 0
155
156 path_grouping_policy
157 The default path grouping policy to apply to unspeci‐
158 fied multipaths. Possible values are:
159
160 failover One path per priority group.
161
162 multibus All paths in one priority group.
163
164 group_by_serial
165 One priority group per serial number.
166
167 group_by_prio
168 One priority group per priority value.
169 Priorities are determined by callout pro‐
170 grams specified as a global, per-con‐
171 troller or per-multipath option in the
172 configuration file.
173
174 group_by_node_name
175 One priority group per target node name.
176 Target node names are fetched in
177 /sys/class/fc_transport/target*/node_name.
178
179 The default is: failover
180
181 uid_attrs Setting this option activates merging uevents by WWID,
182 which may improve uevent processing effiency. More‐
183 over, it's an alternative method to configure the udev
184 properties to use for determining unique path identi‐
185 fiers (WWIDs).
186
187 The value of this option is a space separated list of
188 records like "type:ATTR", where type is matched
189 against the beginning of the device node name (e.g.
190 sd:ATTR matches sda), and ATTR is the name of the udev
191 property to use for matching devices.
192
193 If this option is configured and matches the device
194 node name of a device, it overrides any other config‐
195 ured methods for determining the WWID for this de‐
196 vice.
197
198 The default is: <unset>. To enable uevent merging, set
199 it e.g. to "sd:ID_SERIAL dasd:ID_UID nvme:ID_WWN".
200
201 uid_attribute The udev attribute providing a unique path identifier
202 (WWID). If uid_attribute is set to the empty string,
203 WWID determination is done using the sysfs method
204 rather then using udev (not recommended in production;
205 see WWID generation below).
206
207 The default is: ID_SERIAL, for SCSI devices
208
209 The default is: ID_UID, for DASD devices
210
211 The default is: ID_WWN, for NVMe devices
212
213 getuid_callout (Superseded by uid_attribute) The default program and
214 args to callout to obtain a unique path identifier.
215 Should be specified with an absolute path.
216
217 The default is: <unset>
218
219 prio The name of the path priority routine. The specified
220 routine should return a numeric value specifying the
221 relative priority of this path. Higher number have a
222 higher priority. "none" is a valid value. Currently
223 the following path priority routines are implemented:
224
225 const Return a constant priority of 1.
226
227 sysfs Use the sysfs attributes access_state and
228 preferred_path to generate the path prior‐
229 ity. This prioritizer accepts the optional
230 prio_arg exclusive_pref_bit.
231
232 emc (Hardware-dependent) Generate the path
233 priority for DGC class arrays as CLARiiON
234 CX/AX and EMC VNX and Unity families.
235
236 alua (Hardware-dependent) Generate the path
237 priority based on the SCSI-3 ALUA set‐
238 tings. This prioritizer accepts the op‐
239 tional prio_arg exclusive_pref_bit.
240
241 ontap (Hardware-dependent) Generate the path
242 priority for NetApp ONTAP class and OEM
243 arrays as IBM NSeries.
244
245 rdac (Hardware-dependent) Generate the path
246 priority for LSI/Engenio/NetApp RDAC class
247 as NetApp SANtricity E/EF Series, and OEM
248 arrays from IBM DELL SGI STK and SUN.
249
250 hp_sw (Hardware-dependent) Generate the path
251 priority for HP/COMPAQ/DEC HSG80 and
252 MSA/HSV arrays with Active/Standby mode
253 exclusively.
254
255 hds (Hardware-dependent) Generate the path
256 priority for Hitachi AMS families of ar‐
257 rays other than AMS 2000.
258
259 random Generate a random priority between 1 and
260 10.
261
262 weightedpath
263 Generate the path priority based on the
264 regular expression and the priority pro‐
265 vided as argument. Requires prio_args key‐
266 word.
267
268 path_latency
269 Generate the path priority based on a la‐
270 tency algorithm. Requires prio_args key‐
271 word.
272
273 ana (Hardware-dependent) Generate the path
274 priority based on the NVMe ANA settings.
275
276 datacore (Hardware-dependent) Generate the path
277 priority for some DataCore storage arrays.
278 Requires prio_args keyword.
279
280 iet (iSCSI only) Generate path priority for
281 iSCSI targets based on IP address. Re‐
282 quires prio_args keyword.
283
284 The default depends on the detect_prio setting: If de‐
285 tect_prio is yes (default), the default priority algo‐
286 rithm is sysfs (except for NetAPP E-Series, where it
287 is alua). If detect_prio is no, the default priority
288 algorithm is const.
289
290 prio_args Arguments to pass to to the prio function. This only
291 applies to certain prioritizers:
292
293 weighted Needs a value of the form "<hbtl|dev‐
294 name|serial|wwn> <regex1> <prio1> <regex2>
295 <prio2> ..."
296
297 hbtl Regex can be of SCSI H:B:T:L for‐
298 mat. For example: 1:0:.:. ,
299 *:0:0:.
300
301 devname Regex can be of device name for‐
302 mat. For example: sda , sd.e
303
304 serial Regex can be of serial number for‐
305 mat. For example: .*J1FR.*324 .
306 The serial can be looked up
307 through sysfs or by running multi‐
308 pathd show paths format "%z". For
309 example: 0395J1FR904324
310
311 wwn Regex can be of the form
312 "host_wwnn:host_wwpn:tar‐
313 get_wwnn:target_wwpn" these values
314 can be looked up through sysfs or
315 by running multipathd show paths
316 format "%N:%R:%n:%r". For example:
317 0x200100e08ba0aea0:0x210100e08ba0aea0:.*:.*
318 , .*:.*:iqn.2009-10.com.red‐
319 hat.msp.lab.ask-06:.*
320
321 path_latency
322 Needs a value of the form "io_num=<20>
323 base_num=<10>"
324
325 io_num The number of read IOs sent to the
326 current path continuously, used to
327 calculate the average path la‐
328 tency. Valid Values: Integer, [2,
329 200].
330
331 base_num
332 The base number value of logarith‐
333 mic scale, used to partition dif‐
334 ferent priority ranks. Valid Val‐
335 ues: Integer, [2, 10]. And Max av‐
336 erage latency value is 100s, min
337 average latency value is 1us. For
338 example: If base_num=10, the paths
339 will be grouped in priority groups
340 with path latency <=1us, (1us,
341 10us], (10us, 100us], (100us,
342 1ms], (1ms, 10ms], (10ms, 100ms],
343 (100ms, 1s], (1s, 10s], (10s,
344 100s], >100s.
345
346 alua If exclusive_pref_bit is set, paths with
347 the preferred path bit set will always be
348 in their own path group.
349
350 sysfs If exclusive_pref_bit is set, paths with
351 the preferred path bit set will always be
352 in their own path group.
353
354 datacore
355
356 preferredsds
357 (Mandatory) The preferred "SDS
358 name".
359
360 timeout (Optional) The timeout for the IN‐
361 QUIRY, in ms.
362
363 iet
364
365 preferredip=...
366 (Mandatory) Th preferred IP ad‐
367 dress, in dotted decimal notation,
368 for iSCSI targets.
369
370 The default is: <unset>
371
372 features Specify any device-mapper features to be used. Syntax
373 is num list where num is the number, between 0 and 8,
374 of features in list. Possible values for the feature
375 list are:
376
377 queue_if_no_path
378 (Deprecated, superseded by no_path_retry)
379 Queue I/O if no path is active. Identical
380 to the no_path_retry with queue value. If
381 both this feature and no_path_retry are
382 set, the latter value takes precedence.
383 See KNOWN ISSUES.
384
385 pg_init_retries <times>
386 (Since kernel 2.6.24) Number of times to
387 retry pg_init, it must be between 1 and
388 50.
389
390 pg_init_delay_msecs <msecs>
391 (Since kernel 2.6.38) Number of msecs be‐
392 fore pg_init retry, it must be between 0
393 and 60000.
394
395 queue_mode <mode>
396 (Since kernel 4.8) Select the the queueing
397 mode per multipath device. <mode> can be
398 bio, rq or mq, which corresponds to bio-
399 based, request-based, and block-multiqueue
400 (blk-mq) request-based, respectively. The
401 default depends on the kernel parameter
402 dm_mod.use_blk_mq. It is mq if the latter
403 is set, and rq otherwise.
404
405 The default is: <unset>
406
407 path_checker The default method used to determine the path's state.
408 The synchronous checkers (all except tur and directio)
409 will cause multipathd to pause most activity, waiting
410 up to checker_timeout seconds for the path to respond.
411 The asynchronous checkers (tur and directio) will not
412 pause multipathd. Instead, multipathd will check for a
413 response once per second, until checker_timeout sec‐
414 onds have elapsed. Possible values are:
415
416 readsector0 (Deprecated) Read the first sector of the
417 device. This checker is being deprecated,
418 please use tur instead.
419
420 tur Issue a TEST UNIT READY command to the de‐
421 vice.
422
423 emc_clariion
424 (Hardware-dependent) Query the DGC/EMC
425 specific EVPD page 0xC0 to determine the
426 path state for CLARiiON CX/AX and EMC VNX
427 and Unity arrays families.
428
429 hp_sw (Hardware-dependent) Check the path state
430 for HP/COMPAQ/DEC HSG80 and MSA/HSV arrays
431 with Active/Standby mode exclusively.
432
433 rdac (Hardware-dependent) Check the path state
434 for LSI/Engenio/NetApp RDAC class as Ne‐
435 tApp SANtricity E/EF Series, and OEM ar‐
436 rays from IBM DELL SGI STK and SUN.
437
438 directio Read the first sector with direct I/O.
439 This checker could cause spurious path
440 failures under high load. Increasing
441 checker_timeout can help with this.
442
443 cciss_tur (Hardware-dependent) Check the path state
444 for HP/COMPAQ Smart Array(CCISS) con‐
445 trollers.
446
447 none Do not check the device, fallback to use
448 the values retrieved from sysfs
449
450 The default is: tur
451
452 alias_prefix The user_friendly_names prefix.
453
454 The default is: mpath
455
456 failback Tell multipathd how to manage path group failback. To
457 select immediate or a value, it's mandatory that the
458 device has support for a working prioritizer.
459
460 immediate Immediately failback to the highest prior‐
461 ity pathgroup that contains active paths.
462
463 manual Do not perform automatic failback.
464
465 followover Used to deal with multiple computers ac‐
466 cessing the same Active/Passive storage
467 devices. Only perform automatic failback
468 when the first path of a pathgroup becomes
469 active. This keeps a cluster node from au‐
470 tomatically failing back when another node
471 requested the failover.
472
473 values > 0 Deferred failback (time to defer in sec‐
474 onds).
475
476 The default is: manual
477
478 rr_min_io Number of I/O requests to route to a path before
479 switching to the next in the same path group. This is
480 only for Block I/O(BIO) based multipath and only apply
481 to round-robin path_selector.
482
483 The default is: 1000
484
485 rr_min_io_rq Number of I/O requests to route to a path before
486 switching to the next in the same path group. This is
487 only for Request based multipath and only apply to
488 round-robin path_selector.
489
490 The default is: 1
491
492 max_fds Specify the maximum number of file descriptors that
493 can be opened by multipath and multipathd. This is
494 equivalent to ulimit -n. A value of max will set this
495 to the system limit from /proc/sys/fs/nr_open. If this
496 is not set, the maximum number of open fds is taken
497 from the calling process. It is usually 1024. To be
498 safe, this should be set to the maximum number of
499 paths plus 32, if that number is greated than 1024.
500
501 The default is: max
502
503 rr_weight If set to priorities the multipath configurator will
504 assign path weights as "path prio * rr_min_io". Possi‐
505 ble values are priorities or uniform . Only apply to
506 round-robin path_selector.
507
508 The default is: uniform
509
510 no_path_retry Specify what to do when all paths are down. Possible
511 values are:
512
513 value > 0 Number of retries until disable I/O queue‐
514 ing.
515
516 fail For immediate failure (no I/O queueing).
517
518 queue For never stop I/O queueing, similar to
519 queue_if_no_path. See KNOWN ISSUES.
520
521 The default is: fail
522
523 queue_without_daemon
524 If set to no , when multipathd stops, queueing will be
525 turned off for all devices. This is useful for de‐
526 vices that set no_path_retry. If a machine is shut
527 down while all paths to a device are down, it is pos‐
528 sible to hang waiting for I/O to return from the de‐
529 vice after multipathd has been stopped. Without multi‐
530 pathd running, access to the paths cannot be restored,
531 and the kernel cannot be told to stop queueing I/O.
532 Setting queue_without_daemon to no , avoids this prob‐
533 lem.
534
535 The default is: no
536
537 checker_timeout Specify the timeout to use for path checkers and pri‐
538 oritizers, in seconds. Only prioritizers that issue
539 scsi commands use checker_timeout. If a path does not
540 respond to the checker command after checker_timeout
541 seconds have elapsed, it is considered down.
542
543 The default is: in /sys/block/<dev>/device/timeout
544
545 allow_usb_devices
546 If set to no , all USB devices will be skipped during
547 path discovery. If you intend to use multipath on USB
548 attached devices, set this to yes.
549
550 The default is: no
551
552 flush_on_last_del
553 If set to yes , multipathd will disable queueing when
554 the last path to a device has been deleted.
555
556 The default is: no
557
558 user_friendly_names
559 If set to yes , using the bindings file /etc/multi‐
560 path/bindings to assign a persistent and unique alias
561 to the multipath, in the form of mpath<n>. If set to
562 no use the WWID as the alias. In either case this be
563 will be overridden by any specific aliases in the mul‐
564 tipaths section.
565
566 The default is: no
567
568 fast_io_fail_tmo Specify the number of seconds the SCSI layer will wait
569 after a problem has been detected on a FC remote port
570 before failing I/O to devices on that remote port.
571 This should be smaller than dev_loss_tmo. Setting this
572 to off will disable the timeout.
573
574 The default is: 5
575
576 dev_loss_tmo Specify the number of seconds the SCSI layer will wait
577 after a problem has been detected on a FC remote port
578 before removing it from the system. This can be set to
579 "infinity" which sets it to the max value of
580 2147483647 seconds, or 68 years. It will be automati‐
581 cally adjusted to the overall retry interval
582 no_path_retry * polling_interval if a number of re‐
583 tries is given with no_path_retry and the overall
584 retry interval is longer than the specified
585 dev_loss_tmo value. The Linux kernel will cap this
586 value to 600 if fast_io_fail_tmo is not set. See KNOWN
587 ISSUES.
588
589 The default is: 600
590
591 eh_deadline Specify the maximum number of seconds the SCSI layer
592 will spend doing error handling when scsi devices
593 fail. After this timeout the scsi layer will perform a
594 full HBA reset. Setting this may be necessary in cases
595 where the rport is never lost, so fast_io_fail_tmo and
596 dev_loss_tmo will never trigger, but (frequently do to
597 load) scsi commands still hang. Note: when the scsi
598 error handler performs the HBA reset, all target paths
599 on that HBA will be affected. eh_deadline should only
600 be set in cases where all targets on the affected HBAs
601 are multipathed.
602
603 The default is: <unset>
604
605 bindings_file This option is deprecated, and will be removed in a
606 future release. The full pathname of the binding file
607 to be used when the user_friendly_names option is set.
608
609 The default is: /etc/multipath/bindings
610
611 wwids_file This option is deprecated, and will be removed in a
612 future release. The full pathname of the WWIDs file,
613 which is used by multipath to keep track of the WWIDs
614 for LUNs it has created multipath devices on in the
615 past.
616
617 The default is: /etc/multipath/wwids
618
619 prkeys_file This option is deprecated, and will be removed in a
620 future release. The full pathname of the prkeys file,
621 which is used by multipathd to keep track of the per‐
622 sistent reservation key used for a specific WWID, when
623 reservation_key is set to file.
624
625 The default is: /etc/multipath/prkeys
626
627 log_checker_err If set to once , multipathd logs the first path
628 checker error at logging level 2. Any later errors are
629 logged at level 3 until the device is restored. If set
630 to always , multipathd always logs the path checker
631 error at logging level 2.
632
633 The default is: always
634
635 reservation_key This is the service action reservation key used by
636 mpathpersist. It must be set for all multipath devices
637 using persistent reservations, and it must be the same
638 as the RESERVATION KEY field of the PERSISTENT RESERVE
639 OUT parameter list which contains an 8-byte value pro‐
640 vided by the application client to the device server
641 to identify the I_T nexus. If the --param-aptpl option
642 is used when registering the key with mpathpersist,
643 :aptpl must be appended to the end of the reservation
644 key.
645
646 Alternatively, this can be set to file, which will
647 store the RESERVATION KEY registered by mpathpersist
648 in the prkeys_file. multipathd will then use this key
649 to register additional paths as they appear. When the
650 registration is removed, the RESERVATION KEY is re‐
651 moved from the prkeys_file. The prkeys file will auto‐
652 matically keep track of whether the key was registered
653 with --param-aptpl.
654
655 The default is: <unset>
656
657 all_tg_pt Set the 'all targets ports' flag when registering keys
658 with mpathpersist. Some arrays automatically set and
659 clear registration keys on all target ports from a
660 host, instead of per target port per host. The
661 ALL_TG_PT flag must be set to successfully use mpath‐
662 persist on these arrays. Setting this option is iden‐
663 tical to calling mpathpersist with --param-alltgpt
664
665 The default is: no
666
667 retain_attached_hw_handler
668 (Obsolete for kernels >= 4.3) If set to yes and the
669 SCSI layer has already attached a hardware_handler to
670 the device, multipath will not force the device to use
671 the hardware_handler specified by mutipath.conf. If
672 the SCSI layer has not attached a hardware handler,
673 multipath will continue to use its configured hardware
674 handler.
675
676 The default is: yes
677
678 Important Note: Linux kernel 4.3 or newer always be‐
679 haves as if "retain_attached_hw_handler yes" was set.
680
681 detect_prio If set to yes , multipath will try to detect if the
682 device supports SCSI-3 ALUA. If so, the device will
683 automatically use the sysfs prioritizer if the re‐
684 quired sysf attributes access_state and preferred_path
685 are supported, or the alua prioritizer if not. If set
686 to no , the prioritizer will be selected as usual.
687
688 The default is: yes
689
690 detect_checker if set to yes , multipath will try to detect if the
691 device supports SCSI-3 ALUA. If so, the device will
692 automatically use the tur checker. If set to no , the
693 checker will be selected as usual.
694
695 The default is: yes
696
697 force_sync If set to yes , multipathd will call the path checkers
698 in sync mode only. This means that only one checker
699 will run at a time. This is useful in the case where
700 many multipathd checkers running in parallel causes
701 significant CPU pressure.
702
703 The default is: no
704
705 strict_timing If set to yes , multipathd will start a new path
706 checker loop after exactly one second, so that each
707 path check will occur at exactly polling_interval sec‐
708 onds. On busy systems path checks might take longer
709 than one second; here the missing ticks will be ac‐
710 counted for on the next round. A warning will be
711 printed if path checks take longer than polling_inter‐
712 val seconds.
713
714 The default is: no
715
716 deferred_remove If set to yes , multipathd will do a deferred remove
717 instead of a regular remove when the last path device
718 has been deleted. This means that if the multipath
719 device is still in use, it will be freed when the last
720 user closes it. If path is added to the multipath de‐
721 vice before the last user closes it, the deferred re‐
722 move will be canceled.
723
724 The default is: no
725
726 partition_delimiter
727 This parameter controls how multipath chooses the
728 names of partition devices of multipath maps if a mul‐
729 tipath map is renamed (e.g. if a map alias is added or
730 changed). If this parameter is set to a string other
731 than "/UNSET/" (even the empty string), multipath in‐
732 serts that string between device name and partition
733 number to construct the partition device name. Other‐
734 wise (i.e. if this parameter is unset or has the value
735 "/UNSET/"), the behavior depends on the map name: if
736 it ends in a digit, a "p" is inserted between name and
737 partition number; otherwise, the partition number is
738 simply appended. Distributions may use a non-null de‐
739 fault value for this option; in this case, the user
740 must set it to "/UNSET/" to obtain the original <un‐
741 set> behavior. Use multipath -T to check the current
742 settings.
743
744 The default is: <unset>
745
746 config_dir This option is deprecated, and will be removed in a
747 future release. If set to anything other than "",
748 multipath will search this directory alphabetically
749 for file ending in ".conf" and it will read configura‐
750 tion information from them, just as if it was in
751 /etc/multipath.conf. config_dir must either be "" or
752 a fully qualified directory name.
753
754 The default is: /etc/multipath/conf.d/
755
756 san_path_err_threshold
757 If set to a value greater than 0, multipathd will
758 watch paths and check how many times a path has been
759 failed due to errors.If the number of failures on a
760 particular path is greater then the
761 san_path_err_threshold, then the path will not rein‐
762 state till san_path_err_recovery_time. These path
763 failures should occur within a san_path_err_for‐
764 get_rate checks, if not we will consider the path is
765 good enough to reinstantate. See "Shaky paths detec‐
766 tion" below.
767
768 The default is: no
769
770 san_path_err_forget_rate
771 If set to a value greater than 0, multipathd will
772 check whether the path failures has exceeded the
773 san_path_err_threshold within this many checks i.e
774 san_path_err_forget_rate . If so we will not rein‐
775 stante the path till san_path_err_recovery_time. See
776 "Shaky paths detection" below.
777
778 The default is: no
779
780 san_path_err_recovery_time
781 If set to a value greater than 0, multipathd will make
782 sure that when path failures has exceeded the
783 san_path_err_threshold within san_path_err_forget_rate
784 then the path will be placed in failed state for
785 san_path_err_recovery_time duration.Once
786 san_path_err_recovery_time has timeout we will rein‐
787 stante the failed path . san_path_err_recovery_time
788 value should be in secs. See "Shaky paths detection"
789 below.
790
791 The default is: no
792
793 marginal_path_double_failed_time
794 One of the four parameters of supporting path check
795 based on accounting IO error such as intermittent er‐
796 ror. When a path failed event occurs twice in mar‐
797 ginal_path_double_failed_time seconds due to an IO er‐
798 ror and all the other three parameters are set, multi‐
799 pathd will fail the path and enqueue this path into a
800 queue of which members are sent a couple of continuous
801 direct reading asynchronous IOs at a fixed sample rate
802 of 10HZ to start IO error accounting process. See
803 "Shaky paths detection" below.
804
805 The default is: no
806
807 marginal_path_err_sample_time
808 One of the four parameters of supporting path check
809 based on accounting IO error such as intermittent er‐
810 ror. If it is set to a value no less than 120, when a
811 path fail event occurs twice in marginal_path_dou‐
812 ble_failed_time second due to an IO error, multipathd
813 will fail the path and enqueue this path into a queue
814 of which members are sent a couple of continuous di‐
815 rect reading asynchronous IOs at a fixed sample rate
816 of 10HZ to start the IO accounting process for the
817 path will last for marginal_path_err_sample_time. If
818 the rate of IO error on a particular path is greater
819 than the marginal_path_err_rate_threshold, then the
820 path will not reinstate for mar‐
821 ginal_path_err_recheck_gap_time seconds unless there
822 is only one active path. After mar‐
823 ginal_path_err_recheck_gap_time expires, the path will
824 be requeueed for rechecking. If checking result is
825 good enough, the path will be reinstated. See "Shaky
826 paths detection" below.
827
828 The default is: no
829
830 marginal_path_err_rate_threshold
831 The error rate threshold as a permillage (1/1000). One
832 of the four parameters of supporting path check based
833 on accounting IO error such as intermittent error. Re‐
834 fer to marginal_path_err_sample_time. If the rate of
835 IO errors on a particular path is greater than this
836 parameter, then the path will not reinstate for mar‐
837 ginal_path_err_recheck_gap_time seconds unless there
838 is only one active path. See "Shaky paths detection"
839 below.
840
841 The default is: no
842
843 marginal_path_err_recheck_gap_time
844 One of the four parameters of supporting path check
845 based on accounting IO error such as intermittent er‐
846 ror. Refer to marginal_path_err_sample_time. If this
847 parameter is set to a positive value, the failed path
848 of which the IO error rate is larger than mar‐
849 ginal_path_err_rate_threshold will be kept in failed
850 state for marginal_path_err_recheck_gap_time seconds.
851 When marginal_path_err_recheck_gap_time seconds ex‐
852 pires, the path will be requeueed for checking. If
853 checking result is good enough, the path will be rein‐
854 stated, or else it will keep failed. See "Shaky paths
855 detection" below.
856
857 The default is: no
858
859 delay_watch_checks
860 This option is deprecated, and mapped to
861 san_path_err_forget_rate. If this is set to a value
862 greater than 0 and no san_path_err options are set,
863 san_path_err_forget_rate will be set to the value of
864 delay_watch_checks and san_path_err_threshold will be
865 set to 1. See the san_path_err_forget_rate and
866 san_path_err_threshold options, and "Shaky paths de‐
867 tection" below for more information.
868
869 The default is: no
870
871 delay_wait_checks
872 This option is deprecated, and mapped to
873 san_path_err_recovery_time. If this is set to a value
874 greater than 0 and no san_path_err options are set,
875 san_path_err_recovery_time will be set to the value of
876 delay_wait_checks times max_polling_interval. This
877 will give approximately the same wait time as de‐
878 lay_wait_checks previously did. Also,
879 san_path_err_threshold will be set to 1. See the
880 san_path_err_recovery_time and san_path_err_threshold
881 options, and "Shaky paths detection" below for more
882 information.
883
884 The default is: no
885
886 marginal_pathgroups
887 If set to no, the delay_*_checks, marginal_path_*, and
888 san_path_err_* options will keep marginal, or "shaky",
889 paths from being reinstated until they have been moni‐
890 tored for some time. This can cause situations where
891 all non-marginal paths are down, and no paths are us‐
892 able until multipathd detects this and reinstates a
893 marginal path. If the multipath device is not config‐
894 ured to queue IO in this case, it can cause IO errors
895 to occur, even though there are marginal paths avail‐
896 able. However, if this option is set to yes, when one
897 of the marginal path detecting methods determines that
898 a path is marginal, it will be reinstated and placed
899 in a seperate pathgroup that will only be used after
900 all the non-marginal pathgroups have been tried first.
901 This prevents the possibility of IO errors occuring
902 while marginal paths are still usable. After the path
903 has been monitored for the configured time, and is de‐
904 clared healthy, it will be returned to its normal
905 pathgroup. See "Shaky paths detection" below for more
906 information.
907
908 The default is: no
909
910 find_multipaths This option controls whether multipath and multipathd
911 try to create multipath maps over non-blacklisted de‐
912 vices they encounter. This matters a) when a device is
913 encountered by multipath -u during udev rule process‐
914 ing (a device is blocked from further processing by
915 higher layers - such as LVM - if and only if it´s con‐
916 sidered a valid multipath device path), and b) when
917 multipathd detects a new device. The following values
918 are possible:
919
920 strict Both multipath and multipathd treat only
921 such devices as multipath devices which have
922 been part of a multipath map previously, and
923 which are therefore listed in the
924 wwids_file. Users can manually set up multi‐
925 path maps using the multipathd add map com‐
926 mand. Once set up manually, the map is re‐
927 membered in the wwids file and will be set
928 up automatically in the future.
929
930 no Multipath behaves like strict. Multipathd
931 behaves like greedy.
932
933 yes Both multipathd and multipath treat a device
934 as multipath device if the conditions for
935 strict are met, or if at least two non-
936 blacklisted paths with the same WWID have
937 been detected.
938
939 greedy Both multipathd and multipath treat every
940 non-blacklisted device as multipath device
941 path.
942
943 smart This differs from find_multipaths yes only
944 in the way it treats new devices for which
945 only one path has been detected yet. When
946 such a device is first encounted in udev
947 rules, it is treated as a multipath device.
948 multipathd waits whether additional paths
949 with the same WWID appears. If that happens,
950 it sets up a multipath map. If it doesn´t
951 happen until a timeout expires, or if set‐
952 ting up the map fails, a new uevent is trig‐
953 gered for the device; at second encounter in
954 the udev rules, the device will be treated
955 as non-multipath and passed on to upper lay‐
956 ers. Note: this may cause delays during de‐
957 vice detection if there are single-path de‐
958 vices which aren´t blacklisted.
959
960 The default is: strict
961
962 find_multipaths_timeout
963 Timeout, in seconds, to wait for additional paths af‐
964 ter detecting the first one, if find_multipaths
965 "smart" (see above) is set. If the value is positive,
966 this timeout is used for all unknown, non-blacklisted
967 devices encountered. If the value is negative (recom‐
968 mended), it's only applied to "known" devices that
969 have an entry in multipath's hardware table, either in
970 the built-in table or in a device section; other ("un‐
971 known") devices will use a timeout of only 1 second to
972 avoid booting delays. The value 0 means "use the
973 built-in default". If find_multipath has a value other
974 than smart, this option has no effect.
975
976 The default is: -10 (10s for known and 1s for unknown
977 hardware)
978
979 uxsock_timeout CLI receive timeout in milliseconds. For larger sys‐
980 tems CLI commands might timeout before the multipathd
981 lock is released and the CLI command can be processed.
982 This will result in errors like "timeout receiving
983 packet" to be returned from CLI commands. In these
984 cases it is recommended to increase the CLI timeout to
985 avoid those issues.
986
987 The default is: 4000
988
989 retrigger_tries Sets the number of times multipathd will try to re‐
990 trigger a uevent to get the WWID.
991
992 The default is: 3
993
994 retrigger_delay Sets the amount of time, in seconds, to wait between
995 retriggers.
996
997 The default is: 10
998
999 missing_uev_wait_timeout
1000 Controls how many seconds multipathd will wait, after
1001 a new multipath device is created, to receive a change
1002 event from udev for the device, before automatically
1003 enabling device reloads. Usually multipathd will delay
1004 reloads on a device until it receives a change uevent
1005 from the initial table load.
1006
1007 The default is: 30
1008
1009 skip_kpartx If set to yes , kpartx will not automatically create
1010 partitions on the device.
1011
1012 The default is: no
1013
1014 disable_changed_wwids
1015 This option is deprecated and ignored. If the WWID of
1016 a path suddenly changes, multipathd handles it as if
1017 it was removed and then added again.
1018
1019 remove_retries This sets how may times multipath will retry removing
1020 a device that is in-use. Between each attempt, multi‐
1021 path will sleep 1 second.
1022
1023 The default is: 0
1024
1025 max_sectors_kb Sets the max_sectors_kb device parameter on all path
1026 devices and the multipath device to the specified
1027 value.
1028
1029 The default is: in /sys/block/<dev>/queue/max_sec‐
1030 tors_kb
1031
1032 ghost_delay Sets the number of seconds that multipath will wait
1033 after creating a device with only ghost paths before
1034 marking it ready for use in systemd. This gives the
1035 active paths time to appear before the multipath runs
1036 the hardware handler to switch the ghost paths to ac‐
1037 tive ones. Setting this to 0 or no makes multipath im‐
1038 mediately mark a device with only ghost paths as
1039 ready.
1040
1041 The default is: no
1042
1043 enable_foreign Enables or disables foreign libraries (see section
1044 FOREIGN MULTIPATH SUPPORT below). The value is a regu‐
1045 lar expression; foreign libraries are loaded if their
1046 name (e.g. "nvme") matches the expression. By default,
1047 no foreign libraries are enabled. Set this to "nvme"
1048 to enable NVMe native multipath support, or ".*" to
1049 enable all foreign libraries.
1050
1051 The default is: "NONE"
1052
1053 recheck_wwid If set to yes, when a failed path is restored, its
1054 wwid is rechecked. If the wwid has changed, the path
1055 is removed from the current multipath device, and re-
1056 added as a new path. Multipathd will also recheck a
1057 path's wwid if it is manually re-added. This option
1058 only works for SCSI devices that are configured to use
1059 the default uid_attribute, ID_SERIAL, or sysfs for
1060 getting their wwid.
1061
1062 The default is: no
1063
1064
1066 The blacklist section is used to exclude specific devices from the mul‐
1067 tipath topology. It is most commonly used to exclude local disks or
1068 non-disk devices (such as LUNs for the storage array controller) from
1069 being handled by multipath-tools.
1070
1071 In the blacklist and blacklist_exceptions sections, starting a quoted
1072 value with an exclamation mark "!" will invert the matching of the rest
1073 of the regular expression. For instance, "!^sd[a-z]" will match all
1074 values that do not start with "sd[a-z]". The exclamation mark can be
1075 escaped "\!" to match a literal ! at the start of a regular expression.
1076 Note: The exclamation mark must be inside quotes, otherwise it will be
1077 treated as starting a comment.
1078
1079 The blacklist_exceptions section is used to revert the actions of the
1080 blacklist section. This allows one to selectively include ("whitelist")
1081 devices which would normally be excluded via the blacklist section. A
1082 common usage is to blacklist "everything" using a catch-all regular ex‐
1083 pression, and create specific blacklist_exceptions entries for those
1084 devices that should be handled by multipath-tools.
1085
1086 The following keywords are recognized in both sections. The defaults
1087 are empty unless explicitly stated.
1088
1089 devnode Regular expression matching the device nodes to be ex‐
1090 cluded/included.
1091
1092 The default blacklist consists of the regular expres‐
1093 sion "!^(sd[a-z]|dasd[a-z]|nvme[0-9])". This causes
1094 all device types other than scsi, dasd, and nvme to be
1095 excluded from multipath handling by default.
1096
1097 wwid Regular expression for the World Wide Identifier of a
1098 device to be excluded/included.
1099
1100 device Subsection for the device description. This subsection
1101 recognizes the vendor and product keywords. Both are
1102 regular expressions. For a full description of these
1103 keywords please see the devices section description.
1104
1105 property Regular expression for an udev property. All devices
1106 that have matching udev properties will be ex‐
1107 cluded/included. The handling of the property keyword
1108 is special, because if a property blacklist_exception
1109 is set, devices must have at least one whitelisted
1110 udev property; otherwise they're treated as black‐
1111 listed, and the message "blacklisted, udev property
1112 missing" is displayed in the logs. For example, set‐
1113 ting the property blacklist_exception to
1114 (SCSI_IDENT_|ID_WWN), will cause well-behaved SCSI de‐
1115 vices and devices that provide a WWN (World Wide Num‐
1116 ber) to be included, and all others to be excluded.
1117 This works to exclude most non-multipathable devices.
1118
1119 Note: The behavior of this option has changed in mul‐
1120 tipath-tools 0.8.2 compared to previous versions.
1121 Blacklisting by missing properties is only applied to
1122 devices which do have the property specified by
1123 uid_attribute (e.g. ID_SERIAL) set. Previously, it was
1124 applied to every device, possibly causing devices to
1125 be blacklisted because of temporary I/O error condi‐
1126 tions.
1127
1128 protocol Regular expression for the protocol of a device to be
1129 excluded/included.
1130
1131 The protocol strings that multipath recognizes are
1132 scsi:fcp, scsi:spi, scsi:ssa, scsi:sbp, scsi:srp,
1133 scsi:iscsi, scsi:sas, scsi:adt, scsi:ata, scsi:unspec,
1134 ccw, cciss, nvme, and undef. The protocol that a path
1135 is using can be viewed by running multipathd show
1136 paths format "%d %P"
1137
1138 For every device, these 5 blacklist criteria are evaluated in the the
1139 order "property, devnode, device, protocol, wwid". If a device turns
1140 out to be blacklisted by any criterion, it's excluded from handling by
1141 multipathd, and the later criteria aren't evaluated any more. For each
1142 criterion, the whitelist takes precedence over the blacklist if a de‐
1143 vice matches both.
1144
1145 Note: Besides the blacklist and whitelist, other configuration options
1146 such as find_multipaths have an impact on whether or not a given device
1147 is handled by multipath-tools.
1148
1150 The multipaths section allows setting attributes of multipath maps. The
1151 attributes that are set via the multipaths section (see list below)
1152 take precedence over all other configuration settings, including those
1153 from the overrides section.
1154
1155 The only recognized attribute for the multipaths section is the multi‐
1156 path subsection. If there are multiple multipath subsections matching a
1157 given WWID, the contents of these sections are merged, and settings
1158 from later entries take precedence.
1159
1160 The multipath subsection recognizes the following attributes:
1161
1162 wwid (Mandatory) World Wide Identifier. Detected multipath
1163 maps are matched agains this attribute. Note that,
1164 unlike the wwid attribute in the blacklist section,
1165 this is not a regular expression or a substring; WWIDs
1166 must match exactly inside the multipaths section.
1167
1168 alias Symbolic name for the multipath map. This takes prece‐
1169 dence over a an entry for the same WWID in the bind‐
1170 ings_file.
1171
1172 The following attributes are optional; if not set the default values
1173 are taken from the overrides, devices, or defaults section:
1174
1175 path_grouping_policy
1176 path_selector
1177 prio
1178 prio_args
1179 failback
1180 rr_weight
1181 no_path_retry
1182 rr_min_io
1183 rr_min_io_rq
1184 flush_on_last_del
1185 features
1186 reservation_key
1187 user_friendly_names
1188 deferred_remove
1189 san_path_err_threshold
1190 san_path_err_forget_rate
1191 san_path_err_recovery_time
1192 marginal_path_err_sample_time
1193 marginal_path_err_rate_threshold
1194 marginal_path_err_recheck_gap_time
1195 marginal_path_double_failed_time
1196 delay_watch_checks
1197 delay_wait_checks
1198 skip_kpartx
1199 max_sectors_kb
1200 ghost_delay
1201
1203 multipath-tools have a built-in device table with reasonable defaults
1204 for more than 100 known multipath-capable storage devices. The devices
1205 section can be used to override these settings. If there are multiple
1206 matches for a given device, the attributes of all matching entries are
1207 applied to it. If an attribute is specified in several matching device
1208 subsections, later entries take precedence. Thus, entries in files un‐
1209 der config_dir (in reverse alphabetical order) have the highest prece‐
1210 dence, followed by entries in multipath.conf; the built-in hardware ta‐
1211 ble has the lowest precedence. Inside a configuration file, later en‐
1212 tries have higher precedence than earlier ones.
1213
1214 The only recognized attribute for the devices section is the device
1215 subsection. Devices detected in the system are matched against the de‐
1216 vice entries using the vendor, product, and revision fields, which are
1217 all POSIX Extended regular expressions (see regex(7)).
1218
1219 The vendor, product, and revision fields that multipath or multipathd
1220 detect for devices in a system depend on the device type. For SCSI de‐
1221 vices, they correspond to the respective fields of the SCSI INQUIRY
1222 page. In general, the command 'multipathd show paths format "%d %s"'
1223 command can be used to see the detected properties for all devices in
1224 the system.
1225
1226 The device subsection recognizes the following attributes:
1227
1228 vendor (Mandatory) Regular expression to match the vendor
1229 name.
1230
1231 product (Mandatory) Regular expression to match the product
1232 name.
1233
1234 revision Regular expression to match the product revision. If
1235 not specified, any revision matches.
1236
1237 product_blacklist
1238 Products with the given vendor matching this string
1239 are blacklisted. This is equivalent to a device entry
1240 in the blacklist section with the vendor attribute set
1241 to this entry's vendor, and the product attribute set
1242 to the value of product_blacklist.
1243
1244 alias_prefix The user_friendly_names prefix to use for this device
1245 type, instead of the default "mpath".
1246
1247 vpd_vendor The vendor specific vpd page information, using the
1248 vpd page abbreviation. The vpd page abbreviation can
1249 be found by running sg_vpd -e. multipathd will use
1250 this information to gather device specific information
1251 that can be displayed with the %g wilcard for the mul‐
1252 tipathd show maps format and multipathd show paths
1253 format commands. Currently only the hp3par vpd page is
1254 supported.
1255
1256 hardware_handler The hardware handler to use for this device type. The
1257 following hardware handler are implemented:
1258
1259 1 emc (Hardware-dependent) Hardware handler for
1260 DGC class arrays as CLARiiON CX/AX and EMC
1261 VNX and Unity families.
1262
1263 1 rdac (Hardware-dependent) Hardware handler for
1264 LSI/Engenio/NetApp RDAC class as NetApp
1265 SANtricity E/EF Series, and OEM arrays
1266 from IBM DELL SGI STK and SUN.
1267
1268 1 hp_sw (Hardware-dependent) Hardware handler for
1269 HP/COMPAQ/DEC HSG80 and MSA/HSV arrays
1270 with Active/Standby mode exclusively.
1271
1272 1 alua (Hardware-dependent) Hardware handler for
1273 SCSI-3 ALUA compatible arrays.
1274
1275 1 ana (Hardware-dependent) Hardware handler for
1276 NVMe ANA compatible arrays.
1277
1278 The default is: <unset>
1279
1280 Important Note: Linux kernels 4.3 and newer automati‐
1281 cally attach a device handler to known devices (which
1282 includes all devices supporting SCSI-3 ALUA) and dis‐
1283 allow changing the handler afterwards. Setting hard‐
1284 ware_handler for such devices on these kernels has no
1285 effect.
1286
1287 The following attributes are optional; if not set the default values
1288 are taken from the defaults section:
1289
1290 path_grouping_policy
1291 uid_attribute
1292 getuid_callout
1293 path_selector
1294 path_checker
1295 prio
1296 prio_args
1297 features
1298 failback
1299 rr_weight
1300 no_path_retry
1301 rr_min_io
1302 rr_min_io_rq
1303 fast_io_fail_tmo
1304 dev_loss_tmo
1305 flush_on_last_del
1306 user_friendly_names
1307 retain_attached_hw_handler
1308 detect_prio
1309 detect_checker
1310 deferred_remove
1311 san_path_err_threshold
1312 san_path_err_forget_rate
1313 san_path_err_recovery_time
1314 marginal_path_err_sample_time
1315 marginal_path_err_rate_threshold
1316 marginal_path_err_recheck_gap_time
1317 marginal_path_double_failed_time
1318 delay_watch_checks
1319 delay_wait_checks
1320 skip_kpartx
1321 max_sectors_kb
1322 ghost_delay
1323 all_tg_pt
1324
1326 The overrides section recognizes the following optional attributes; if
1327 not set the values are taken from the devices or defaults sections:
1328
1329 path_grouping_policy
1330 uid_attribute
1331 getuid_callout
1332 path_selector
1333 path_checker
1334 alias_prefix
1335 features
1336 prio
1337 prio_args
1338 failback
1339 rr_weight
1340 no_path_retry
1341 rr_min_io
1342 rr_min_io_rq
1343 flush_on_last_del
1344 fast_io_fail_tmo
1345 dev_loss_tmo
1346 user_friendly_names
1347 retain_attached_hw_handler
1348 detect_prio
1349 detect_checker
1350 deferred_remove
1351 san_path_err_threshold
1352 san_path_err_forget_rate
1353 san_path_err_recovery_time
1354 marginal_path_err_sample_time
1355 marginal_path_err_rate_threshold
1356 marginal_path_err_recheck_gap_time
1357 marginal_path_double_failed_time
1358 delay_watch_checks
1359 delay_wait_checks
1360 skip_kpartx
1361 max_sectors_kb
1362 ghost_delay
1363 all_tg_pt
1364
1366 Multipath uses a World Wide Identification (WWID) to determine which
1367 paths belong to the same device. Each path presenting the same WWID is
1368 assumed to point to the same device.
1369
1370 The WWID is generated by four methods (in the order of preference):
1371
1372 uid_attrs The WWID is derived from udev attributes by matching
1373 the device node name; cf uid_attrs above.
1374
1375 getuid_callout Use the specified external program; cf getuid_callout
1376 above. Care should be taken when using this method;
1377 the external program needs to be loaded from disk for
1378 execution, which might lead to deadlock situations in
1379 an all-paths-down scenario.
1380
1381 uid_attribute Use the value of the specified udev attribute; cf
1382 uid_attribute above. This method is preferred to ge‐
1383 tuid_callout as multipath does not need to call any
1384 external programs here. However, under certain circum‐
1385 stances udev might not be able to generate the re‐
1386 quested variable.
1387
1388 sysfs Try to determine the WWID from sysfs attributes. For
1389 SCSI devices, this means reading the Vital Product
1390 Data (VPD) page "Device Identification" (0x83).
1391
1392 The default settings (using udev and uid_attribute configured from the
1393 built-in hardware table) should work fine in most scenarios. Users who
1394 want to enable uevent merging must set uid_attrs.
1395
1397 A common problem in SAN setups is the occurence of intermittent errors:
1398 a path is unreachable, then reachable again for a short time, disap‐
1399 pears again, and so forth. This happens typically on unstable intercon‐
1400 nects. It is undesirable to switch pathgroups unnecessarily on such
1401 frequent, unreliable events. multipathd supports three different meth‐
1402 ods for detecting this situation and dealing with it. All methods share
1403 the same basic mode of operation: If a path is found to be "shaky" or
1404 "flipping", and appears to be in healthy status, it is not reinstated
1405 (put back to use) immediately. Instead, it is placed in the "delayed"
1406 state and watched for some time, and only reinstated if the healthy
1407 state appears to be stable. If the marginal_pathgroups option is set,
1408 the path will reinstated immediately, but placed in a special pathgroup
1409 for marginal paths. Marginal pathgroups will not be used until all
1410 other pathgroups have been tried. At the time when the path would nor‐
1411 mally be reinstated, it will be returned to its normal pathgroup. The
1412 logic of determining "shaky" condition, as well as the logic when to
1413 reinstate, differs between the three methods.
1414
1415 "delay_checks" failure tracking
1416 This method is deprecated and mapped to the "san_path_err"
1417 method. See the delay_watch_checks and delay_wait_checks op‐
1418 tions above for more information.
1419
1420
1421 "marginal_path" failure tracking
1422 If a second failure event (good->bad transition) occurs within
1423 marginal_path_double_failed_time seconds after a failure, high-
1424 frequency monitoring is started for the affected path: I/O is
1425 sent at a rate of 10 per second. This is done for mar‐
1426 ginal_path_err_sample_time seconds. During this period, the
1427 path is not reinstated. If the rate of errors remains below
1428 marginal_path_err_rate_threshold during the monitoring period,
1429 the path is reinstated. Otherwise, it is kept in failed state
1430 for marginal_path_err_recheck_gap_time, and after that, it is
1431 monitored again. For this method, time intervals are measured
1432 in seconds.
1433
1434 "san_path_err" failure tracking
1435 multipathd counts path failures for each path. Once the number
1436 of failures exceeds the value given by san_path_err_threshold,
1437 the path is not reinstated for san_path_err_recovery_time sec‐
1438 onds. While counting failures, multipathd "forgets" one past
1439 failure every "san_path_err_forget_rate" ticks; thus if errors
1440 don't occur more often then once in the forget rate interval,
1441 the failure count doesn't increase and the threshold is never
1442 reached. Ticks are the time between path checks by multipathd,
1443 which is variable and controlled by the polling_interval and
1444 max_polling_interval parameters.
1445
1446 This method is deprecated in favor of the "marginal_path" fail‐
1447 ure tracking method, and only offered for backward compatibil‐
1448 ity.
1449
1450 See the documentation of the individual options above for details. It
1451 is strongly discouraged to use more than one of these methods for any
1452 given multipath map, because the two concurrent methods may interact in
1453 unpredictable ways. If the "marginal_path" method is active, the
1454 "san_path_err" parameters are implicitly set to 0.
1455
1457 multipath and multipathd can load "foreign" libraries to add support
1458 for other multipathing technologies besides the Linux device mapper.
1459 Currently this support is limited to printing detected information
1460 about multipath setup. In topology output, the names of foreign maps
1461 are prefixed by the foreign library name in square brackets, as in this
1462 example:
1463
1464 # multipath -ll
1465 uuid.fedcba98-3579-4567-8765-123456789abc [nvme]:nvme4n9 NVMe,Some NVMe controller,FFFFFFFF
1466 size=167772160 features='n/a' hwhandler='ANA' wp=rw
1467 |-+- policy='n/a' prio=50 status=optimized
1468 | `- 4:38:1 nvme4c38n1 0:0 n/a optimized live
1469 `-+- policy='n/a' prio=50 status=optimized
1470 `- 4:39:1 nvme4c39n1 0:0 n/a optimized live
1471
1472 The "nvme" foreign library provides support for NVMe native multi‐
1473 pathing in the kernel. It is part of the standard multipath package.
1474
1476 The usage of queue_if_no_path option can lead to D state processes be‐
1477 ing hung and not killable in situations where all the paths to the LUN
1478 go offline. It is advisable to use the no_path_retry option instead.
1479
1480 The use of queue_if_no_path or no_path_retry might lead to a deadlock
1481 if the dev_loss_tmo setting results in a device being removed while I/O
1482 is still queued. The multipath daemon will update the dev_loss_tmo set‐
1483 ting accordingly to avoid this deadlock. Hence if both values are spec‐
1484 ified the order of precedence is no_path_retry, queue_if_no_path,
1485 dev_loss_tmo.
1486
1488 udev(8), dmsetup(8), multipath(8), multipathd(8).
1489
1491 multipath-tools was developed by Christophe Varoqui, <christophe.varo‐
1492 qui@opensvc.com> and others.
1493
1494
1495
1496Linux 2021-09-08 MULTIPATH.CONF(5)