1MULTIPATH.CONF(5) File Formats Manual MULTIPATH.CONF(5)
2
6 multipath.conf - multipath daemon configuration file.
7
9 /etc/multipath.conf is the configuration file for the multipath daemon.
10 It is used to overwrite the built-in configuration table of multipathd.
11 Any line whose first non-white-space character is a '#' is considered a
12 comment line. Empty lines are ignored.
13 Currently used multipathd configuration can be displayed with the mul‐
15 tipath -t or multipathd show config command.
16
18 The configuration file contains entries of the form:
19 <section> {
21 <attribute> <value>
22 ...
23 <subsection> {
24 <attribute> <value>
25 ...
26 }
27 }
28 Each section contains one or more attributes or subsections. The recog‐
30 nized keywords for attributes or subsections depend on the section in
31 which they occur.
32 <attribute> and <value> must be on a single line. <attribute> is one
34 of the keywords listed in this man page. <value> is either a simple
35 word (containing no whitespace and none of the characters '"', '#', and
36 '!') or one string enclosed in double quotes ("..."). Outside a quoted
37 string, text starting with '#', and '!' is regarded as a comment and
38 ignored until the end of the line. Inside a quoted string, '#' and '!'
39 are normal characters, and whitespace is preserved. To represent a
40 double quote character inside a double quoted string, use two consecu‐
41 tive double quotes ('""'). Thus '2.5" SSD' can be written as "2.5""
42 SSD".
43 Opening braces ('{') must follow the (sub)section name on the same
45 line. Closing braces ('}') that mark the end of a (sub)section must be
46 the only non-whitespace character on the line. Whitespace is ignored
47 except inside double quotes, thus the indentation shown in the above
48 example is helpful for human readers but not mandatory.
49 Note on regular expressions: The multipath.conf syntax allows many
51 attribute values to be specified as POSIX Extended Regular Expressions
52 (see regex(7)). These regular expressions are case sensitive and not
53 anchored, thus the expression "bar" matches "barbie", "rhabarber", and
54 "wunderbar", but not "Barbie". To avoid unwanted substring matches,
55 standard regular expression syntax using the special characters "^" and
56 "$" can be used.
57 The following section keywords are recognized:
59 defaults This section defines default values for attributes
61 which are used whenever no values are given in the
62 appropriate device or multipath sections.
63 blacklist This section defines which devices should be excluded
65 from the multipath topology discovery.
66 blacklist_exceptions
68 This section defines which devices should be included
69 in the multipath topology discovery, despite being
70 listed in the blacklist section.
71 multipaths This section defines the multipath topologies. They
73 are indexed by a World Wide Identifier(WWID). For
74 details on the WWID generation see section WWID gener‐
75 ation below. Attributes set in this section take
76 precedence over all others.
77 devices This section defines the device-specific settings.
79 Devices are identified by vendor, product, and revi‐
80 sion.
81 overrides This section defines values for attributes that should
83 override the device-specific settings for all devices.
84
86 The defaults section recognizes the following keywords:
87 verbosity Default verbosity. Higher values increase the ver‐
89 bosity level. Valid levels are between 0 and 6.
90 The default is: 2
92 polling_interval Interval between two path checks in seconds. For prop‐
94 erly functioning paths, the interval between checks
95 will gradually increase to max_polling_interval. This
96 value will be overridden by the WatchdogSec setting in
97 the multipathd.service definition if systemd is used.
98 The default is: 5
100 max_polling_interval
102 Maximal interval between two path checks in seconds.
103 The default is: 4 * polling_interval
105 reassign_maps Enable reassigning of device-mapper maps. With this
107 option multipathd will remap existing device-mapper
108 maps to always point to multipath device, not the
109 underlying block devices. Possible values are yes and
110 no.
111 The default is: no
113 multipath_dir Directory where the dynamic shared objects are stored.
115 Defined at compile time, commonly /lib64/multipath/ or
116 /lib/multipath/.
117 The default is: <system dependent>
119 path_selector The default path selector algorithm to use; they are
121 offered by the kernel multipath target. There are
122 three selector algorithms:
123 round-robin 0
125 Loop through every path in the path group,
126 sending the same amount of I/O to each.
127 Some aspects of behavior can be controlled
128 with the attributes: rr_min_io,
129 rr_min_io_rq and rr_weight.
130 queue-length 0
132 (Since 2.6.31 kernel) Choose the path for
133 the next bunch of I/O based on the amount
134 of outstanding I/O to the path.
135 service-time 0
137 (Since 2.6.31 kernel) Choose the path for
138 the next bunch of I/O based on the amount
139 of outstanding I/O to the path and its
140 relative throughput.
141 The default is: service-time 0
143 path_grouping_policy
145 The default path grouping policy to apply to unspeci‐
146 fied multipaths. Possible values are:
147 failover One path per priority group.
149 multibus All paths in one priority group.
151 group_by_serial
153 One priority group per serial number.
154 group_by_prio
156 One priority group per priority value.
157 Priorities are determined by callout pro‐
158 grams specified as a global, per-con‐
159 troller or per-multipath option in the
160 configuration file.
161 group_by_node_name
163 One priority group per target node name.
164 Target node names are fetched in
165 /sys/class/fc_transport/target*/node_name.
166 The default is: failover
168 uid_attrs Setting this option activates merging uevents by WWID,
170 which may improve uevent processing effiency. More‐
171 over, it's an alternative method to configure the udev
172 properties to use for determining unique path identi‐
173 fiers (WWIDs).
174 The value of this option is a space separated list of
176 records like "type:ATTR", where type is matched
177 against the beginning of the device node name (e.g.
178 sd:ATTR matches sda), and ATTR is the name of the udev
179 property to use for matching devices.
180 If this option is configured and matches the device
182 node name of a device, it overrides any other config‐
183 ured methods for determining the WWID for this
184 device.
185 The default is: <unset>. To enable uevent merging, set
187 it e.g. to "sd:ID_SERIAL dasd:ID_UID nvme:ID_WWN".
188 uid_attribute The udev attribute providing a unique path identifier
190 (WWID). If uid_attribute is set to the empty string,
191 WWID determination is done using the sysfs method
192 rather then using udev (not recommended in production;
193 see WWID generation below).
194 The default is: ID_SERIAL, for SCSI devices
196 The default is: ID_UID, for DASD devices
198 The default is: ID_WWN, for NVMe devices
200 getuid_callout (Superseded by uid_attribute) The default program and
202 args to callout to obtain a unique path identifier.
203 Should be specified with an absolute path.
204 The default is: <unset>
206 prio The name of the path priority routine. The specified
208 routine should return a numeric value specifying the
209 relative priority of this path. Higher number have a
210 higher priority. "none" is a valid value. Currently
211 the following path priority routines are implemented:
212 const Return a constant priority of 1.
214 sysfs Use the sysfs attributes access_state and
216 preferred_path to generate the path prior‐
217 ity. This prioritizer accepts the optional
218 prio_arg exclusive_pref_bit.
219 emc (Hardware-dependent) Generate the path
221 priority for DGC class arrays as CLARiiON
222 CX/AX and EMC VNX and Unity families.
223 alua (Hardware-dependent) Generate the path
225 priority based on the SCSI-3 ALUA set‐
226 tings. This prioritizer accepts the
227 optional prio_arg exclusive_pref_bit.
228 ontap (Hardware-dependent) Generate the path
230 priority for NetApp ONTAP class and OEM
231 arrays as IBM NSeries.
232 rdac (Hardware-dependent) Generate the path
234 priority for LSI/Engenio/NetApp RDAC class
235 as NetApp SANtricity E/EF Series, and OEM
236 arrays from IBM DELL SGI STK and SUN.
237 hp_sw (Hardware-dependent) Generate the path
239 priority for HP/COMPAQ/DEC HSG80 and
240 MSA/HSV arrays with Active/Standby mode
241 exclusively.
242 hds (Hardware-dependent) Generate the path
244 priority for Hitachi AMS families of
245 arrays other than AMS 2000.
246 random Generate a random priority between 1 and
248 10.
249 weightedpath
251 Generate the path priority based on the
252 regular expression and the priority pro‐
253 vided as argument. Requires prio_args key‐
254 word.
255 path_latency
257 Generate the path priority based on a
258 latency algorithm. Requires prio_args
259 keyword.
260 ana (Hardware-dependent) Generate the path
262 priority based on the NVMe ANA settings.
263 datacore (Hardware-dependent) Generate the path
265 priority for some DataCore storage arrays.
266 Requires prio_args keyword.
267 iet (iSCSI only) Generate path priority for
269 iSCSI targets based on IP address.
270 Requires prio_args keyword.
271 The default depends on the detect_prio setting: If
273 detect_prio is yes (default), the default priority
274 algorithm is sysfs (except for NetAPP E-Series, where
275 it is alua). If detect_prio is no, the default prior‐
276 ity algorithm is const.
277 prio_args Arguments to pass to to the prio function. This only
279 applies to certain prioritizers:
280 weighted Needs a value of the form "<hbtl|dev‐
282 name|serial|wwn> <regex1> <prio1> <regex2>
283 <prio2> ..."
284 hbtl Regex can be of SCSI H:B:T:L for‐
286 mat. For example: 1:0:.:. ,
287 *:0:0:.
288 devname Regex can be of device name for‐
290 mat. For example: sda , sd.e
291 serial Regex can be of serial number for‐
293 mat. For example: .*J1FR.*324 .
294 The serial can be looked up
295 through sysfs or by running multi‐
296 pathd show paths format "%z". For
297 example: 0395J1FR904324
298 wwn Regex can be of the form
300 "host_wwnn:host_wwpn:tar‐
301 get_wwnn:target_wwpn" these values
302 can be looked up through sysfs or
303 by running multipathd show paths
304 format "%N:%R:%n:%r". For example:
305 0x200100e08ba0aea0:0x210100e08ba0aea0:.*:.*
306 , .*:.*:iqn.2009-10.com.red‐
307 hat.msp.lab.ask-06:.*
308 path_latency
310 Needs a value of the form "io_num=<20>
311 base_num=<10>"
312 io_num The number of read IOs sent to the
314 current path continuously, used to
315 calculate the average path
316 latency. Valid Values: Integer,
317 [2, 200].
318 base_num
320 The base number value of logarith‐
321 mic scale, used to partition dif‐
322 ferent priority ranks. Valid Val‐
323 ues: Integer, [2, 10]. And Max
324 average latency value is 100s, min
325 average latency value is 1us. For
326 example: If base_num=10, the paths
327 will be grouped in priority groups
328 with path latency <=1us, (1us,
329 10us], (10us, 100us], (100us,
330 1ms], (1ms, 10ms], (10ms, 100ms],
331 (100ms, 1s], (1s, 10s], (10s,
332 100s], >100s.
333 alua If exclusive_pref_bit is set, paths with
335 the preferred path bit set will always be
336 in their own path group.
337 sysfs If exclusive_pref_bit is set, paths with
339 the preferred path bit set will always be
340 in their own path group.
341 datacore
343 preferredsds
345 (Mandatory) The preferred "SDS
346 name".
347 timeout (Optional) The timeout for the
349 INQUIRY, in ms.
350 iet
352 preferredip=...
354 (Mandatory) Th preferred IP
355 address, in dotted decimal nota‐
356 tion, for iSCSI targets.
357 The default is: <unset>
359 features Specify any device-mapper features to be used. Syntax
361 is num list where num is the number, between 0 and 8,
362 of features in list. Possible values for the feature
363 list are:
364 queue_if_no_path
366 (Deprecated, superseded by no_path_retry)
367 Queue I/O if no path is active. Identical
368 to the no_path_retry with queue value. If
369 both this feature and no_path_retry are
370 set, the latter value takes precedence.
371 See KNOWN ISSUES.
372 pg_init_retries <times>
374 (Since kernel 2.6.24) Number of times to
375 retry pg_init, it must be between 1 and
376 50.
377 pg_init_delay_msecs <msecs>
379 (Since kernel 2.6.38) Number of msecs
380 before pg_init retry, it must be between 0
381 and 60000.
382 queue_mode <mode>
384 (Since kernel 4.8) Select the the queueing
385 mode per multipath device. <mode> can be
386 bio, rq or mq, which corresponds to bio-
387 based, request-based, and block-multiqueue
388 (blk-mq) request-based, respectively. The
389 default depends on the kernel parameter
390 dm_mod.use_blk_mq. It is mq if the latter
391 is set, and rq otherwise.
392 The default is: <unset>
394 path_checker The default method used to determine the paths state.
396 Possible values are:
397 readsector0 (Deprecated) Read the first sector of the
399 device. This checker is being deprecated,
400 please use tur instead.
401 tur Issue a TEST UNIT READY command to the
403 device.
404 emc_clariion
406 (Hardware-dependent) Query the DGC/EMC
407 specific EVPD page 0xC0 to determine the
408 path state for CLARiiON CX/AX and EMC VNX
409 and Unity arrays families.
410 hp_sw (Hardware-dependent) Check the path state
412 for HP/COMPAQ/DEC HSG80 and MSA/HSV arrays
413 with Active/Standby mode exclusively.
414 rdac (Hardware-dependent) Check the path state
416 for LSI/Engenio/NetApp RDAC class as
417 NetApp SANtricity E/EF Series, and OEM
418 arrays from IBM DELL SGI STK and SUN.
419 directio (Deprecated) Read the first sector with
421 direct I/O. This checker is being depre‐
422 cated, it could cause spurious path fail‐
423 ures under high load. Please use tur
424 instead.
425 cciss_tur (Hardware-dependent) Check the path state
427 for HP/COMPAQ Smart Array(CCISS) con‐
428 trollers.
429 none Do not check the device, fallback to use
431 the values retrieved from sysfs
432 The default is: tur
434 alias_prefix The user_friendly_names prefix.
436 The default is: mpath
438 failback Tell multipathd how to manage path group failback. To
440 select immediate or a value, it's mandatory that the
441 device has support for a working prioritizer.
442 immediate Immediately failback to the highest prior‐
444 ity pathgroup that contains active paths.
445 manual Do not perform automatic failback.
447 followover Used to deal with multiple computers
449 accessing the same Active/Passive storage
450 devices. Only perform automatic failback
451 when the first path of a pathgroup becomes
452 active. This keeps a cluster node from
453 automatically failing back when another
454 node requested the failover.
455 values > 0 Deferred failback (time to defer in sec‐
457 onds).
458 The default is: manual
460 rr_min_io Number of I/O requests to route to a path before
462 switching to the next in the same path group. This is
463 only for Block I/O(BIO) based multipath and only apply
464 to round-robin path_selector.
465 The default is: 1000
467 rr_min_io_rq Number of I/O requests to route to a path before
469 switching to the next in the same path group. This is
470 only for Request based multipath and only apply to
471 round-robin path_selector.
472 The default is: 1
474 max_fds Specify the maximum number of file descriptors that
476 can be opened by multipath and multipathd. This is
477 equivalent to ulimit -n. A value of max will set this
478 to the system limit from /proc/sys/fs/nr_open. If this
479 is not set, the maximum number of open fds is taken
480 from the calling process. It is usually 1024. To be
481 safe, this should be set to the maximum number of
482 paths plus 32, if that number is greated than 1024.
483 The default is: max
485 rr_weight If set to priorities the multipath configurator will
487 assign path weights as "path prio * rr_min_io". Possi‐
488 ble values are priorities or uniform . Only apply to
489 round-robin path_selector.
490 The default is: uniform
492 no_path_retry Specify what to do when all paths are down. Possible
494 values are:
495 value > 0 Number of retries until disable I/O queue‐
497 ing.
498 fail For immediate failure (no I/O queueing).
500 queue For never stop I/O queueing, similar to
502 queue_if_no_path. See KNOWN ISSUES.
503 The default is: fail
505 queue_without_daemon
507 If set to no , when multipathd stops, queueing will be
508 turned off for all devices. This is useful for
509 devices that set no_path_retry. If a machine is shut
510 down while all paths to a device are down, it is pos‐
511 sible to hang waiting for I/O to return from the
512 device after multipathd has been stopped. Without mul‐
513 tipathd running, access to the paths cannot be
514 restored, and the kernel cannot be told to stop queue‐
515 ing I/O. Setting queue_without_daemon to no , avoids
516 this problem.
517 The default is: no
519 checker_timeout Specify the timeout to use for path checkers and pri‐
521 oritizers that issue SCSI commands with an explicit
522 timeout, in seconds.
523 The default is: in /sys/block/sd<x>/device/timeout
525 flush_on_last_del
527 If set to yes , multipathd will disable queueing when
528 the last path to a device has been deleted.
529 The default is: no
531 user_friendly_names
533 If set to yes , using the bindings file /etc/multi‐
534 path/bindings to assign a persistent and unique alias
535 to the multipath, in the form of mpath<n>. If set to
536 no use the WWID as the alias. In either case this be
537 will be overridden by any specific aliases in the mul‐
538 tipaths section.
539 The default is: no
541 fast_io_fail_tmo Specify the number of seconds the SCSI layer will wait
543 after a problem has been detected on a FC remote port
544 before failing I/O to devices on that remote port.
545 This should be smaller than dev_loss_tmo. Setting this
546 to off will disable the timeout.
547 The default is: 5
549 dev_loss_tmo Specify the number of seconds the SCSI layer will wait
551 after a problem has been detected on a FC remote port
552 before removing it from the system. This can be set to
553 "infinity" which sets it to the max value of
554 2147483647 seconds, or 68 years. It will be automati‐
555 cally adjusted to the overall retry interval
556 no_path_retry * polling_interval if a number of
557 retries is given with no_path_retry and the overall
558 retry interval is longer than the specified
559 dev_loss_tmo value. The Linux kernel will cap this
560 value to 600 if fast_io_fail_tmo is not set. See KNOWN
561 ISSUES.
562 The default is: 600
564 bindings_file The full pathname of the binding file to be used when
566 the user_friendly_names option is set.
567 The default is: /etc/multipath/bindings
569 wwids_file The full pathname of the WWIDs file, which is used by
571 multipath to keep track of the WWIDs for LUNs it has
572 created multipath devices on in the past.
573 The default is: /etc/multipath/wwids
575 prkeys_file The full pathname of the prkeys file, which is used by
577 multipathd to keep track of the persistent reservation
578 key used for a specific WWID, when reservation_key is
579 set to file.
580 The default is: /etc/multipath/prkeys
582 log_checker_err If set to once , multipathd logs the first path
584 checker error at logging level 2. Any later errors are
585 logged at level 3 until the device is restored. If set
586 to always , multipathd always logs the path checker
587 error at logging level 2.
588 The default is: always
590 reservation_key This is the service action reservation key used by
592 mpathpersist. It must be set for all multipath devices
593 using persistent reservations, and it must be the same
594 as the RESERVATION KEY field of the PERSISTENT RESERVE
595 OUT parameter list which contains an 8-byte value pro‐
596 vided by the application client to the device server
597 to identify the I_T nexus. If the --param-aptpl option
598 is used when registering the key with mpathpersist,
599 :aptpl must be appended to the end of the reservation
600 key.
601 Alternatively, this can be set to file, which will
603 store the RESERVATION KEY registered by mpathpersist
604 in the prkeys_file. multipathd will then use this key
605 to register additional paths as they appear. When the
606 registration is removed, the RESERVATION KEY is
607 removed from the prkeys_file. The prkeys file will
608 automatically keep track of whether the key was regis‐
609 tered with --param-aptpl.
610 The default is: <unset>
612 all_tg_pt Set the 'all targets ports' flag when registering keys
614 with mpathpersist. Some arrays automatically set and
615 clear registration keys on all target ports from a
616 host, instead of per target port per host. The
617 ALL_TG_PT flag must be set to successfully use mpath‐
618 persist on these arrays. Setting this option is iden‐
619 tical to calling mpathpersist with --param-alltgpt
620 The default is: no
622 retain_attached_hw_handler
624 (Obsolete for kernels >= 4.3) If set to yes and the
625 SCSI layer has already attached a hardware_handler to
626 the device, multipath will not force the device to use
627 the hardware_handler specified by mutipath.conf. If
628 the SCSI layer has not attached a hardware handler,
629 multipath will continue to use its configured hardware
630 handler.
631 The default is: yes
633 Important Note: Linux kernel 4.3 or newer always
635 behaves as if "retain_attached_hw_handler yes" was
636 set.
637 detect_prio If set to yes , multipath will try to detect if the
639 device supports SCSI-3 ALUA. If so, the device will
640 automatically use the sysfs prioritizer if the
641 required sysf attributes access_state and pre‐
642 ferred_path are supported, or the alua prioritizer if
643 not. If set to no , the prioritizer will be selected
644 as usual.
645 The default is: yes
647 detect_checker if set to yes , multipath will try to detect if the
649 device supports SCSI-3 ALUA. If so, the device will
650 automatically use the tur checker. If set to no , the
651 checker will be selected as usual.
652 The default is: yes
654 force_sync If set to yes , multipathd will call the path checkers
656 in sync mode only. This means that only one checker
657 will run at a time. This is useful in the case where
658 many multipathd checkers running in parallel causes
659 significant CPU pressure.
660 The default is: no
662 strict_timing If set to yes , multipathd will start a new path
664 checker loop after exactly one second, so that each
665 path check will occur at exactly polling_interval sec‐
666 onds. On busy systems path checks might take longer
667 than one second; here the missing ticks will be
668 accounted for on the next round. A warning will be
669 printed if path checks take longer than polling_inter‐
670 val seconds.
671 The default is: no
673 deferred_remove If set to yes , multipathd will do a deferred remove
675 instead of a regular remove when the last path device
676 has been deleted. This means that if the multipath
677 device is still in use, it will be freed when the last
678 user closes it. If path is added to the multipath
679 device before the last user closes it, the deferred
680 remove will be canceled.
681 The default is: no
683 partition_delimiter
685 This parameter controls how multipath chooses the
686 names of partition devices of multipath maps if a mul‐
687 tipath map is renamed (e.g. if a map alias is added or
688 changed). If this parameter is set to a string other
689 than "/UNSET/" (even the empty string), multipath
690 inserts that string between device name and partition
691 number to construct the partition device name. Other‐
692 wise (i.e. if this parameter is unset or has the value
693 "/UNSET/"), the behavior depends on the map name: if
694 it ends in a digit, a "p" is inserted between name and
695 partition number; otherwise, the partition number is
696 simply appended. Distributions may use a non-null
697 default value for this option; in this case, the user
698 must set it to "/UNSET/" to obtain the original
699 <unset> behavior. Use multipath -T to check the cur‐
700 rent settings.
701 The default is: <unset>
703 config_dir If set to anything other than "", multipath will
705 search this directory alphabetically for file ending
706 in ".conf" and it will read configuration information
707 from them, just as if it was in /etc/multipath.conf.
708 config_dir must either be "" or a fully qualified
709 directory name.
710 The default is: /etc/multipath/conf.d/
712 san_path_err_threshold
714 If set to a value greater than 0, multipathd will
715 watch paths and check how many times a path has been
716 failed due to errors.If the number of failures on a
717 particular path is greater then the
718 san_path_err_threshold, then the path will not rein‐
719 state till san_path_err_recovery_time. These path
720 failures should occur within a san_path_err_for‐
721 get_rate checks, if not we will consider the path is
722 good enough to reinstantate. See "Shaky paths detec‐
723 tion" below.
724 The default is: no
726 san_path_err_forget_rate
728 If set to a value greater than 0, multipathd will
729 check whether the path failures has exceeded the
730 san_path_err_threshold within this many checks i.e
731 san_path_err_forget_rate . If so we will not rein‐
732 stante the path till san_path_err_recovery_time. See
733 "Shaky paths detection" below.
734 The default is: no
736 san_path_err_recovery_time
738 If set to a value greater than 0, multipathd will make
739 sure that when path failures has exceeded the
740 san_path_err_threshold within san_path_err_forget_rate
741 then the path will be placed in failed state for
742 san_path_err_recovery_time duration.Once
743 san_path_err_recovery_time has timeout we will rein‐
744 stante the failed path . san_path_err_recovery_time
745 value should be in secs. See "Shaky paths detection"
746 below.
747 The default is: no
749 marginal_path_double_failed_time
751 One of the four parameters of supporting path check
752 based on accounting IO error such as intermittent
753 error. When a path failed event occurs twice in mar‐
754 ginal_path_double_failed_time seconds due to an IO
755 error and all the other three parameters are set, mul‐
756 tipathd will fail the path and enqueue this path into
757 a queue of which members are sent a couple of continu‐
758 ous direct reading asynchronous IOs at a fixed sample
759 rate of 10HZ to start IO error accounting process. See
760 "Shaky paths detection" below.
761 The default is: no
763 marginal_path_err_sample_time
765 One of the four parameters of supporting path check
766 based on accounting IO error such as intermittent
767 error. If it is set to a value no less than 120, when
768 a path fail event occurs twice in marginal_path_dou‐
769 ble_failed_time second due to an IO error, multipathd
770 will fail the path and enqueue this path into a queue
771 of which members are sent a couple of continuous
772 direct reading asynchronous IOs at a fixed sample rate
773 of 10HZ to start the IO accounting process for the
774 path will last for marginal_path_err_sample_time. If
775 the rate of IO error on a particular path is greater
776 than the marginal_path_err_rate_threshold, then the
777 path will not reinstate for margin‐
778 al_path_err_recheck_gap_time seconds unless there is
779 only one active path. After margin‐
780 al_path_err_recheck_gap_time expires, the path will be
781 requeueed for rechecking. If checking result is good
782 enough, the path will be reinstated. See "Shaky paths
783 detection" below.
784 The default is: no
786 marginal_path_err_rate_threshold
788 The error rate threshold as a permillage (1/1000). One
789 of the four parameters of supporting path check based
790 on accounting IO error such as intermittent error.
791 Refer to marginal_path_err_sample_time. If the rate of
792 IO errors on a particular path is greater than this
793 parameter, then the path will not reinstate for mar‐
794 ginal_path_err_recheck_gap_time seconds unless there
795 is only one active path. See "Shaky paths detection"
796 below.
797 The default is: no
799 marginal_path_err_recheck_gap_time
801 One of the four parameters of supporting path check
802 based on accounting IO error such as intermittent
803 error. Refer to marginal_path_err_sample_time. If this
804 parameter is set to a positive value, the failed path
805 of which the IO error rate is larger than margin‐
806 al_path_err_rate_threshold will be kept in failed
807 state for marginal_path_err_recheck_gap_time seconds.
808 When marginal_path_err_recheck_gap_time seconds
809 expires, the path will be requeueed for checking. If
810 checking result is good enough, the path will be rein‐
811 stated, or else it will keep failed. See "Shaky paths
812 detection" below.
813 The default is: no
815 delay_watch_checks
817 This option is deprecated, and mapped to
818 san_path_err_forget_rate. If this is set to a value
819 greater than 0 and no san_path_err options are set,
820 san_path_err_forget_rate will be set to the value of
821 delay_watch_checks and san_path_err_threshold will be
822 set to 1. See the san_path_err_forget_rate and
823 san_path_err_threshold options, and "Shaky paths
824 detection" below for more information.
825 The default is: no
827 delay_wait_checks
829 This option is deprecated, and mapped to
830 san_path_err_recovery_time. If this is set to a value
831 greater than 0 and no san_path_err options are set,
832 san_path_err_recovery_time will be set to the value of
833 delay_wait_checks times max_polling_interval. This
834 will give approximately the same wait time as
835 delay_wait_checks previously did. Also,
836 san_path_err_threshold will be set to 1. See the
837 san_path_err_recovery_time and san_path_err_threshold
838 options, and "Shaky paths detection" below for more
839 information.
840 The default is: no
842 marginal_pathgroups
844 If set to no, the delay_*_checks, marginal_path_*, and
845 san_path_err_* options will keep marginal, or "shaky",
846 paths from being reinstated until they have been moni‐
847 tored for some time. This can cause situations where
848 all non-marginal paths are down, and no paths are
849 usable until multipathd detects this and reinstates a
850 marginal path. If the multipath device is not config‐
851 ured to queue IO in this case, it can cause IO errors
852 to occur, even though there are marginal paths avail‐
853 able. However, if this option is set to yes, when one
854 of the marginal path detecting methods determines that
855 a path is marginal, it will be reinstated and placed
856 in a seperate pathgroup that will only be used after
857 all the non-marginal pathgroups have been tried first.
858 This prevents the possibility of IO errors occuring
859 while marginal paths are still usable. After the path
860 has been monitored for the configured time, and is
861 declared healthy, it will be returned to its normal
862 pathgroup. See "Shaky paths detection" below for more
863 information.
864 The default is: no
866 find_multipaths This option controls whether multipath and multipathd
868 try to create multipath maps over non-blacklisted
869 devices they encounter. This matters a) when a device
870 is encountered by multipath -u during udev rule pro‐
871 cessing (a device is blocked from further processing
872 by higher layers - such as LVM - if and only if it´s
873 considered a valid multipath device path), and b) when
874 multipathd detects a new device. The following values
875 are possible:
876 strict Both multipath and multipathd treat only
878 such devices as multipath devices which have
879 been part of a multipath map previously, and
880 which are therefore listed in the
881 wwids_file. Users can manually set up multi‐
882 path maps using the multipathd add map com‐
883 mand. Once set up manually, the map is
884 remembered in the wwids file and will be set
885 up automatically in the future.
886 no Multipath behaves like strict. Multipathd
888 behaves like greedy.
889 yes Both multipathd and multipath treat a device
891 as multipath device if the conditions for
892 strict are met, or if at least two non-
893 blacklisted paths with the same WWID have
894 been detected.
895 greedy Both multipathd and multipath treat every
897 non-blacklisted device as multipath device
898 path.
899 smart This differs from find_multipaths yes only
901 in the way it treats new devices for which
902 only one path has been detected yet. When
903 such a device is first encounted in udev
904 rules, it is treated as a multipath device.
905 multipathd waits whether additional paths
906 with the same WWID appears. If that happens,
907 it sets up a multipath map. If it doesn´t
908 happen until a timeout expires, or if set‐
909 ting up the map fails, a new uevent is trig‐
910 gered for the device; at second encounter in
911 the udev rules, the device will be treated
912 as non-multipath and passed on to upper lay‐
913 ers. Note: this may cause delays during
914 device detection if there are single-path
915 devices which aren´t blacklisted.
916 The default is: strict
918 find_multipaths_timeout
920 Timeout, in seconds, to wait for additional paths
921 after detecting the first one, if find_multipaths
922 "smart" (see above) is set. If the value is positive,
923 this timeout is used for all unknown, non-blacklisted
924 devices encountered. If the value is negative (recom‐
925 mended), it's only applied to "known" devices that
926 have an entry in multipath's hardware table, either in
927 the built-in table or in a device section; other
928 ("unknown") devices will use a timeout of only 1 sec‐
929 ond to avoid booting delays. The value 0 means "use
930 the built-in default". If find_multipath has a value
931 other than smart, this option has no effect.
932 The default is: -10 (10s for known and 1s for unknown
934 hardware)
935 uxsock_timeout CLI receive timeout in milliseconds. For larger sys‐
937 tems CLI commands might timeout before the multipathd
938 lock is released and the CLI command can be processed.
939 This will result in errors like "timeout receiving
940 packet" to be returned from CLI commands. In these
941 cases it is recommended to increase the CLI timeout to
942 avoid those issues.
943 The default is: 1000
945 retrigger_tries Sets the number of times multipathd will try to
947 retrigger a uevent to get the WWID.
948 The default is: 3
950 retrigger_delay Sets the amount of time, in seconds, to wait between
952 retriggers.
953 The default is: 10
955 missing_uev_wait_timeout
957 Controls how many seconds multipathd will wait, after
958 a new multipath device is created, to receive a change
959 event from udev for the device, before automatically
960 enabling device reloads. Usually multipathd will delay
961 reloads on a device until it receives a change uevent
962 from the initial table load.
963 The default is: 30
965 skip_kpartx If set to yes , kpartx will not automatically create
967 partitions on the device.
968 The default is: no
970 disable_changed_wwids
972 This option is deprecated and ignored. If the WWID of
973 a path suddenly changes, multipathd handles it as if
974 it was removed and then added again.
975 remove_retries This sets how may times multipath will retry removing
977 a device that is in-use. Between each attempt, multi‐
978 path will sleep 1 second.
979 The default is: 0
981 max_sectors_kb Sets the max_sectors_kb device parameter on all path
983 devices and the multipath device to the specified
984 value.
985 The default is: <device dependent>
987 ghost_delay Sets the number of seconds that multipath will wait
989 after creating a device with only ghost paths before
990 marking it ready for use in systemd. This gives the
991 active paths time to appear before the multipath runs
992 the hardware handler to switch the ghost paths to
993 active ones. Setting this to 0 or on makes multipath
994 immediately mark a device with only ghost paths as
995 ready.
996 The default is: no
998 enable_foreign Enables or disables foreign libraries (see section
1000 FOREIGN MULTIPATH SUPPORT below). The value is a regu‐
1001 lar expression; foreign libraries are loaded if their
1002 name (e.g. "nvme") matches the expression. By default,
1003 all foreign libraries are enabled.
1004 The default is: "" (the empty regular expression)
1006
1009 The blacklist section is used to exclude specific devices from the mul‐
1010 tipath topology. It is most commonly used to exclude local disks or
1011 non-disk devices (such as LUNs for the storage array controller) from
1012 being handled by multipath-tools.
1013 The blacklist_exceptions section is used to revert the actions of the
1015 blacklist section. This allows one to selectively include ("whitelist")
1016 devices which would normally be excluded via the blacklist section. A
1017 common usage is to blacklist "everything" using a catch-all regular
1018 expression, and create specific blacklist_exceptions entries for those
1019 devices that should be handled by multipath-tools.
1020 The following keywords are recognized in both sections. The defaults
1022 are empty unless explicitly stated.
1023 devnode Regular expression matching the device nodes to be
1025 excluded/included.
1026 The default blacklist consists of the regular expres‐
1028 sions "^(ram|zram|raw|loop|fd|md|dm-|sr|scd|st|dcss‐
1029 blk)[0-9]" and "^(td|hd|vd)[a-z]". This causes virtual
1030 devices, non-disk devices, and some other device types
1031 to be excluded from multipath handling by default.
1032 wwid Regular expression for the World Wide Identifier of a
1034 device to be excluded/included.
1035 device Subsection for the device description. This subsection
1037 recognizes the vendor and product keywords. Both are
1038 regular expressions. For a full description of these
1039 keywords please see the devices section description.
1040 property Regular expression for an udev property. All devices
1042 that have matching udev properties will be
1043 excluded/included. The handling of the property key‐
1044 word is special, because if a property black‐
1045 list_exception is set, devices must have at least one
1046 whitelisted udev property; otherwise they're treated
1047 as blacklisted, and the message "blacklisted, udev
1048 property missing" is displayed in the logs. For exam‐
1049 ple, setting the property blacklist_exception to
1050 (SCSI_IDENT_|ID_WWN), will cause well-behaved SCSI
1051 devices and devices that provide a WWN (World Wide
1052 Number) to be included, and all others to be excluded.
1053 This works to exclude most non-multipathable devices.
1054 Note: The behavior of this option has changed in mul‐
1056 tipath-tools 0.8.2 compared to previous versions.
1057 Blacklisting by missing properties is only applied to
1058 devices which do have the property specified by
1059 uid_attribute (e.g. ID_SERIAL) set. Previously, it was
1060 applied to every device, possibly causing devices to
1061 be blacklisted because of temporary I/O error condi‐
1062 tions.
1063 protocol
1065 Regular expression for the protocol of a device
1066 to be excluded/included.
1067 The protocol strings that multipath recognizes
1069 are scsi:fcp, scsi:spi, scsi:ssa, scsi:sbp,
1070 scsi:srp, scsi:iscsi, scsi:sas, scsi:adt,
1071 scsi:ata, scsi:unspec, ccw, cciss, nvme, and
1072 undef. The protocol that a path is using can
1073 be viewed by running multipathd show paths for‐
1074 mat "%d %P"
1075 For every device, these 5 blacklist criteria are eval‐
1077 uated in the the order "property, devnode, device,
1078 protocol, wwid". If a device turns out to be black‐
1079 listed by any criterion, it's excluded from handling
1080 by multipathd, and the later criteria aren't evaluated
1081 any more. For each criterion, the whitelist takes
1082 precedence over the blacklist if a device matches
1083 both.
1084 Note: Besides the blacklist and whitelist, other con‐
1086 figuration options such as find_multipaths have an
1087 impact on whether or not a given device is handled by
1088 multipath-tools.
1089
1091 The multipaths section allows setting attributes of multipath maps. The
1092 attributes that are set via the multipaths section (see list below)
1093 take precedence over all other configuration settings, including those
1094 from the overrides section.
1095 The only recognized attribute for the multipaths section is the multi‐
1097 path subsection. If there are multiple multipath subsections matching a
1098 given WWID, the contents of these sections are merged, and settings
1099 from later entries take precedence.
1100 The multipath subsection recognizes the following attributes:
1102 wwid (Mandatory) World Wide Identifier. Detected multipath
1104 maps are matched agains this attribute. Note that,
1105 unlike the wwid attribute in the blacklist section,
1106 this is not a regular expression or a substring; WWIDs
1107 must match exactly inside the multipaths section.
1108 alias Symbolic name for the multipath map. This takes prece‐
1110 dence over a an entry for the same WWID in the bind‐
1111 ings_file.
1112 The following attributes are optional; if not set the default values
1114 are taken from the overrides, devices, or defaults section:
1115 path_grouping_policy
1117 path_selector
1118 prio
1119 prio_args
1120 failback
1121 rr_weight
1122 no_path_retry
1123 rr_min_io
1124 rr_min_io_rq
1125 flush_on_last_del
1126 features
1127 reservation_key
1128 user_friendly_names
1129 deferred_remove
1130 san_path_err_threshold
1131 san_path_err_forget_rate
1132 san_path_err_recovery_time
1133 marginal_path_err_sample_time
1134 marginal_path_err_rate_threshold
1135 marginal_path_err_recheck_gap_time
1136 marginal_path_double_failed_time
1137 delay_watch_checks
1138 delay_wait_checks
1139 skip_kpartx
1140 max_sectors_kb
1141 ghost_delay
1142
1144 multipath-tools have a built-in device table with reasonable defaults
1145 for more than 100 known multipath-capable storage devices. The devices
1146 section can be used to override these settings. If there are multiple
1147 matches for a given device, the attributes of all matching entries are
1148 applied to it. If an attribute is specified in several matching device
1149 subsections, later entries take precedence. Thus, entries in files
1150 under config_dir (in reverse alphabetical order) have the highest
1151 precedence, followed by entries in multipath.conf; the built-in hard‐
1152 ware table has the lowest precedence. Inside a configuration file,
1153 later entries have higher precedence than earlier ones.
1154 The only recognized attribute for the devices section is the device
1156 subsection. Devices detected in the system are matched against the
1157 device entries using the vendor, product, and revision fields, which
1158 are all POSIX Extended regular expressions (see regex(7)).
1159 The vendor, product, and revision fields that multipath or multipathd
1161 detect for devices in a system depend on the device type. For SCSI
1162 devices, they correspond to the respective fields of the SCSI INQUIRY
1163 page. In general, the command 'multipathd show paths format "%d %s"'
1164 command can be used to see the detected properties for all devices in
1165 the system.
1166 The device subsection recognizes the following attributes:
1168 vendor (Mandatory) Regular expression to match the vendor
1170 name.
1171 product (Mandatory) Regular expression to match the product
1173 name.
1174 revision Regular expression to match the product revision. If
1176 not specified, any revision matches.
1177 product_blacklist
1179 Products with the given vendor matching this string
1180 are blacklisted. This is equivalent to a device entry
1181 in the blacklist section with the vendor attribute set
1182 to this entry's vendor, and the product attribute set
1183 to the value of product_blacklist.
1184 alias_prefix The user_friendly_names prefix to use for this device
1186 type, instead of the default "mpath".
1187 hardware_handler The hardware handler to use for this device type. The
1189 following hardware handler are implemented:
1190 1 emc (Hardware-dependent) Hardware handler for
1192 DGC class arrays as CLARiiON CX/AX and EMC
1193 VNX and Unity families.
1194 1 rdac (Hardware-dependent) Hardware handler for
1196 LSI/Engenio/NetApp RDAC class as NetApp
1197 SANtricity E/EF Series, and OEM arrays
1198 from IBM DELL SGI STK and SUN.
1199 1 hp_sw (Hardware-dependent) Hardware handler for
1201 HP/COMPAQ/DEC HSG80 and MSA/HSV arrays
1202 with Active/Standby mode exclusively.
1203 1 alua (Hardware-dependent) Hardware handler for
1205 SCSI-3 ALUA compatible arrays.
1206 1 ana (Hardware-dependent) Hardware handler for
1208 NVMe ANA compatible arrays.
1209 The default is: <unset>
1211 Important Note: Linux kernels 4.3 and newer automati‐
1213 cally attach a device handler to known devices (which
1214 includes all devices supporting SCSI-3 ALUA) and dis‐
1215 allow changing the handler afterwards. Setting hard‐
1216 ware_handler for such devices on these kernels has no
1217 effect.
1218 The following attributes are optional; if not set the default values
1220 are taken from the defaults section:
1221 path_grouping_policy
1223 uid_attribute
1224 getuid_callout
1225 path_selector
1226 path_checker
1227 prio
1228 prio_args
1229 features
1230 failback
1231 rr_weight
1232 no_path_retry
1233 rr_min_io
1234 rr_min_io_rq
1235 fast_io_fail_tmo
1236 dev_loss_tmo
1237 flush_on_last_del
1238 user_friendly_names
1239 retain_attached_hw_handler
1240 detect_prio
1241 detect_checker
1242 deferred_remove
1243 san_path_err_threshold
1244 san_path_err_forget_rate
1245 san_path_err_recovery_time
1246 marginal_path_err_sample_time
1247 marginal_path_err_rate_threshold
1248 marginal_path_err_recheck_gap_time
1249 marginal_path_double_failed_time
1250 delay_watch_checks
1251 delay_wait_checks
1252 skip_kpartx
1253 max_sectors_kb
1254 ghost_delay
1255 all_tg_pt
1256
1258 The overrides section recognizes the following optional attributes; if
1259 not set the values are taken from the devices or defaults sections:
1260 path_grouping_policy
1262 uid_attribute
1263 getuid_callout
1264 path_selector
1265 path_checker
1266 alias_prefix
1267 features
1268 prio
1269 prio_args
1270 failback
1271 rr_weight
1272 no_path_retry
1273 rr_min_io
1274 rr_min_io_rq
1275 flush_on_last_del
1276 fast_io_fail_tmo
1277 dev_loss_tmo
1278 user_friendly_names
1279 retain_attached_hw_handler
1280 detect_prio
1281 detect_checker
1282 deferred_remove
1283 san_path_err_threshold
1284 san_path_err_forget_rate
1285 san_path_err_recovery_time
1286 marginal_path_err_sample_time
1287 marginal_path_err_rate_threshold
1288 marginal_path_err_recheck_gap_time
1289 marginal_path_double_failed_time
1290 delay_watch_checks
1291 delay_wait_checks
1292 skip_kpartx
1293 max_sectors_kb
1294 ghost_delay
1295 all_tg_pt
1296
1298 Multipath uses a World Wide Identification (WWID) to determine which
1299 paths belong to the same device. Each path presenting the same WWID is
1300 assumed to point to the same device.
1301 The WWID is generated by four methods (in the order of preference):
1303 uid_attrs The WWID is derived from udev attributes by matching
1305 the device node name; cf uid_attrs above.
1306 getuid_callout Use the specified external program; cf getuid_callout
1308 above. Care should be taken when using this method;
1309 the external program needs to be loaded from disk for
1310 execution, which might lead to deadlock situations in
1311 an all-paths-down scenario.
1312 uid_attribute Use the value of the specified udev attribute; cf
1314 uid_attribute above. This method is preferred to
1315 getuid_callout as multipath does not need to call any
1316 external programs here. However, under certain circum‐
1317 stances udev might not be able to generate the
1318 requested variable.
1319 sysfs Try to determine the WWID from sysfs attributes. For
1321 SCSI devices, this means reading the Vital Product
1322 Data (VPD) page "Device Identification" (0x83).
1323 The default settings (using udev and uid_attribute configured from the
1325 built-in hardware table) should work fine in most scenarios. Users who
1326 want to enable uevent merging must set uid_attrs.
1327
1329 A common problem in SAN setups is the occurence of intermittent errors:
1330 a path is unreachable, then reachable again for a short time, disap‐
1331 pears again, and so forth. This happens typically on unstable intercon‐
1332 nects. It is undesirable to switch pathgroups unnecessarily on such
1333 frequent, unreliable events. multipathd supports three different meth‐
1334 ods for detecting this situation and dealing with it. All methods share
1335 the same basic mode of operation: If a path is found to be "shaky" or
1336 "flipping", and appears to be in healthy status, it is not reinstated
1337 (put back to use) immediately. Instead, it is placed in the "delayed"
1338 state and watched for some time, and only reinstated if the healthy
1339 state appears to be stable. If the marginal_pathgroups option is set,
1340 the path will reinstated immediately, but placed in a special pathgroup
1341 for marginal paths. Marginal pathgroups will not be used until all
1342 other pathgroups have been tried. At the time when the path would nor‐
1343 mally be reinstated, it will be returned to its normal pathgroup. The
1344 logic of determining "shaky" condition, as well as the logic when to
1345 reinstate, differs between the three methods.
1346 "delay_checks" failure tracking
1348 This method is deprecated and mapped to the "san_path_err"
1349 method. See the delay_watch_checks and delay_wait_checks
1350 options above for more information.
1351 "marginal_path" failure tracking
1354 If a second failure event (good->bad transition) occurs within
1355 marginal_path_double_failed_time seconds after a failure, high-
1356 frequency monitoring is started for the affected path: I/O is
1357 sent at a rate of 10 per second. This is done for margin‐
1358 al_path_err_sample_time seconds. During this period, the path
1359 is not reinstated. If the rate of errors remains below margin‐
1360 al_path_err_rate_threshold during the monitoring period, the
1361 path is reinstated. Otherwise, it is kept in failed state for
1362 marginal_path_err_recheck_gap_time, and after that, it is moni‐
1363 tored again. For this method, time intervals are measured in
1364 seconds.
1365 "san_path_err" failure tracking
1367 multipathd counts path failures for each path. Once the number
1368 of failures exceeds the value given by san_path_err_threshold,
1369 the path is not reinstated for san_path_err_recovery_time sec‐
1370 onds. While counting failures, multipathd "forgets" one past
1371 failure every "san_path_err_forget_rate" ticks; thus if errors
1372 don't occur more often then once in the forget rate interval,
1373 the failure count doesn't increase and the threshold is never
1374 reached. Ticks are the time between path checks by multipathd,
1375 which is variable and controlled by the polling_interval and
1376 max_polling_interval parameters.
1377 This method is deprecated in favor of the "marginal_path" fail‐
1379 ure tracking method, and only offered for backward compatibil‐
1380 ity.
1381 See the documentation of the individual options above for details. It
1383 is strongly discouraged to use more than one of these methods for any
1384 given multipath map, because the two concurrent methods may interact in
1385 unpredictable ways. If the "marginal_path" method is active, the
1386 "san_path_err" parameters are implicitly set to 0.
1387
1389 multipath and multipathd can load "foreign" libraries to add support
1390 for other multipathing technologies besides the Linux device mapper.
1391 Currently this support is limited to printing detected information
1392 about multipath setup. In topology output, the names of foreign maps
1393 are prefixed by the foreign library name in square brackets, as in this
1394 example:
1395 # multipath -ll
1397 uuid.fedcba98-3579-4567-8765-123456789abc [nvme]:nvme4n9 NVMe,Some NVMe controller,FFFFFFFF
1398 size=167772160 features='n/a' hwhandler='ANA' wp=rw
1399 |-+- policy='n/a' prio=50 status=optimized
1400 | `- 4:38:1 nvme4c38n1 0:0 n/a optimized live
1401 `-+- policy='n/a' prio=50 status=optimized
1402 `- 4:39:1 nvme4c39n1 0:0 n/a optimized live
1403 The "nvme" foreign library provides support for NVMe native multi‐
1405 pathing in the kernel. It is part of the standard multipath package.
1406
1408 The usage of queue_if_no_path option can lead to D state processes
1409 being hung and not killable in situations where all the paths to the
1410 LUN go offline. It is advisable to use the no_path_retry option
1411 instead.
1412 The use of queue_if_no_path or no_path_retry might lead to a deadlock
1414 if the dev_loss_tmo setting results in a device being removed while I/O
1415 is still queued. The multipath daemon will update the dev_loss_tmo set‐
1416 ting accordingly to avoid this deadlock. Hence if both values are spec‐
1417 ified the order of precedence is no_path_retry, queue_if_no_path,
1418 dev_loss_tmo.
1419
1421 udev(8), dmsetup(8), multipath(8), multipathd(8).
1422
1424 multipath-tools was developed by Christophe Varoqui, <christophe.varo‐
1425 qui@opensvc.com> and others.
1426Linux 2018-05-21 MULTIPATH.CONF(5)