1MULTIPATH.CONF(5) File Formats Manual MULTIPATH.CONF(5)
2
6 multipath.conf - multipath daemon configuration file.
7
9 /etc/multipath.conf is the configuration file for the multipath daemon.
10 It is used to overwrite the built-in configuration table of multipathd.
11 Any line whose first non-white-space character is a '#' is considered a
12 comment line. Empty lines are ignored.
13 Currently used multipathd configuration can be displayed with the mul‐
15 tipath -t or multipathd show config command.
16
18 The configuration file contains entries of the form:
19 <section> {
21 <attribute> <value>
22 ...
23 <subsection> {
24 <attribute> <value>
25 ...
26 }
27 }
28 Each section contains one or more attributes or subsections. The recog‐
30 nized keywords for attributes or subsections depend on the section in
31 which they occur.
32 <attribute> and <value> must be on a single line. <attribute> is one
34 of the keywords listed in this man page. <value> is either a simple
35 word (containing no whitespace and none of the characters '"', '#', and
36 '!') or one string enclosed in double quotes ("..."). Outside a quoted
37 string, text starting with '#', and '!' is regarded as a comment and
38 ignored until the end of the line. Inside a quoted string, '#' and '!'
39 are normal characters, and whitespace is preserved. To represent a
40 double quote character inside a double quoted string, use two consecu‐
41 tive double quotes ('""'). Thus '2.5" SSD' can be written as "2.5""
42 SSD".
43 Opening braces ('{') must follow the (sub)section name on the same
45 line. Closing braces ('}') that mark the end of a (sub)section must be
46 the only non-whitespace character on the line. Whitespace is ignored
47 except inside double quotes, thus the indentation shown in the above
48 example is helpful for human readers but not mandatory.
49 Note on regular expressions: The multipath.conf syntax allows many
51 attribute values to be specified as POSIX Extended Regular Expressions
52 (see regex(7)). These regular expressions are case sensitive and not
53 anchored, thus the expression "bar" matches "barbie", "rhabarber", and
54 "wunderbar", but not "Barbie". To avoid unwanted substring matches,
55 standard regular expression syntax using the special characters "^" and
56 "$" can be used.
57 The following section keywords are recognized:
59 defaults This section defines default values for attributes
61 which are used whenever no values are given in the
62 appropriate device or multipath sections.
63 blacklist This section defines which devices should be excluded
65 from the multipath topology discovery.
66 blacklist_exceptions
68 This section defines which devices should be included
69 in the multipath topology discovery, despite being
70 listed in the blacklist section.
71 multipaths This section defines the multipath topologies. They
73 are indexed by a World Wide Identifier(WWID). For
74 details on the WWID generation see section WWID gener‐
75 ation below. Attributes set in this section take
76 precedence over all others.
77 devices This section defines the device-specific settings.
79 Devices are identified by vendor, product, and revi‐
80 sion.
81 overrides This section defines values for attributes that should
83 override the device-specific settings for all devices.
84
86 The defaults section recognizes the following keywords:
87 verbosity Default verbosity. Higher values increase the ver‐
89 bosity level. Valid levels are between 0 and 6.
90 The default is: 2
92 polling_interval Interval between two path checks in seconds. For prop‐
94 erly functioning paths, the interval between checks
95 will gradually increase to max_polling_interval. This
96 value will be overridden by the WatchdogSec setting in
97 the multipathd.service definition if systemd is used.
98 The default is: 5
100 max_polling_interval
102 Maximal interval between two path checks in seconds.
103 The default is: 4 * polling_interval
105 reassign_maps Enable reassigning of device-mapper maps. With this
107 option multipathd will remap existing device-mapper
108 maps to always point to multipath device, not the
109 underlying block devices. Possible values are yes and
110 no.
111 The default is: no
113 multipath_dir Directory where the dynamic shared objects are stored.
115 Defined at compile time, commonly /lib64/multipath/ or
116 /lib/multipath/.
117 The default is: <system dependent>
119 path_selector The default path selector algorithm to use; they are
121 offered by the kernel multipath target. There are
122 three selector algorithms:
123 round-robin 0
125 Loop through every path in the path group,
126 sending the same amount of I/O to each.
127 Some aspects of behavior can be controlled
128 with the attributes: rr_min_io,
129 rr_min_io_rq and rr_weight.
130 queue-length 0
132 (Since 2.6.31 kernel) Choose the path for
133 the next bunch of I/O based on the amount
134 of outstanding I/O to the path.
135 service-time 0
137 (Since 2.6.31 kernel) Choose the path for
138 the next bunch of I/O based on the amount
139 of outstanding I/O to the path and its
140 relative throughput.
141 The default is: service-time 0
143 path_grouping_policy
145 The default path grouping policy to apply to unspeci‐
146 fied multipaths. Possible values are:
147 failover One path per priority group.
149 multibus All paths in one priority group.
151 group_by_serial
153 One priority group per serial number.
154 group_by_prio
156 One priority group per priority value.
157 Priorities are determined by callout pro‐
158 grams specified as a global, per-con‐
159 troller or per-multipath option in the
160 configuration file.
161 group_by_node_name
163 One priority group per target node name.
164 Target node names are fetched in
165 /sys/class/fc_transport/target*/node_name.
166 The default is: failover
168 uid_attrs The udev attribute providing a unique path identifier
170 for corresponding type of path devices. If this field
171 is configured and matched with type of device, it
172 would override any other methods providing for device
173 unique identifier in config file, and it would acti‐
174 vate merging uevents according to the identifier to
175 promote effiecncy in processing uevents. It has no
176 default value, if you want to identify path by udev
177 attribute and to activate merging uevents for SCSI and
178 DASD devices, you can set its value as: uid_attrs
179 "sd:ID_SERIAL dasd:ID_UID".
180 The default is: <unset>
182 uid_attribute The udev attribute providing a unique path identifier.
184 The default is: for SCSI devices ID_SERIAL
186 The default is: for DASD devices ID_UID
188 The default is: for NVME devices ID_WWN
190 getuid_callout (Superseded by uid_attribute) The default program and
192 args to callout to obtain a unique path identifier.
193 Should be specified with an absolute path.
194 The default is: <unset>
196 prio The name of the path priority routine. The specified
198 routine should return a numeric value specifying the
199 relative priority of this path. Higher number have a
200 higher priority. "none" is a valid value. Currently
201 the following path priority routines are implemented:
202 const Return a constant priority of 1.
204 sysfs Use the sysfs attributes access_state and
206 preferred_path to generate the path prior‐
207 ity. This prioritizer accepts the optional
208 prio_arg exclusive_pref_bit.
209 emc (Hardware-dependent) Generate the path
211 priority for DGC class arrays as CLARiiON
212 CX/AX and EMC VNX and Unity families.
213 alua (Hardware-dependent) Generate the path
215 priority based on the SCSI-3 ALUA set‐
216 tings. This prioritizer accepts the
217 optional prio_arg exclusive_pref_bit.
218 ontap (Hardware-dependent) Generate the path
220 priority for NetApp ONTAP class and OEM
221 arrays as IBM NSeries.
222 rdac (Hardware-dependent) Generate the path
224 priority for LSI/Engenio/NetApp RDAC class
225 as NetApp SANtricity E/EF Series, and OEM
226 arrays from IBM DELL SGI STK and SUN.
227 hp_sw (Hardware-dependent) Generate the path
229 priority for HP/COMPAQ/DEC HSG80 and
230 MSA/HSV arrays with Active/Standby mode
231 exclusively.
232 hds (Hardware-dependent) Generate the path
234 priority for Hitachi AMS families of
235 arrays other than AMS 2000.
236 random Generate a random priority between 1 and
238 10.
239 weightedpath
241 Generate the path priority based on the
242 regular expression and the priority pro‐
243 vided as argument. Requires prio_args key‐
244 word.
245 path_latency
247 Generate the path priority based on a
248 latency algorithm. Requires prio_args
249 keyword.
250 ana (Hardware-dependent) Generate the path
252 priority based on the NVMe ANA settings.
253 datacore (Hardware-dependent) Generate the path
255 priority for some DataCore storage arrays.
256 Requires prio_args keyword.
257 iet (iSCSI only) Generate path priority for
259 iSCSI targets based on IP address.
260 Requires prio_args keyword.
261 The default depends on the detect_prio setting: If
263 detect_prio is yes (default), the default priority
264 algorithm is sysfs (except for NetAPP E-Series, where
265 it is alua). If detect_prio is no, the default prior‐
266 ity algorithm is const.
267 prio_args Arguments to pass to to the prio function. This only
269 applies to certain prioritizers:
270 weighted Needs a value of the form "<hbtl|dev‐
272 name|serial|wwn> <regex1> <prio1> <regex2>
273 <prio2> ..."
274 hbtl Regex can be of SCSI H:B:T:L for‐
276 mat. For example: 1:0:.:. ,
277 *:0:0:.
278 devname Regex can be of device name for‐
280 mat. For example: sda , sd.e
281 serial Regex can be of serial number for‐
283 mat. For example: .*J1FR.*324 .
284 The serial can be looked up
285 through sysfs or by running multi‐
286 pathd show paths format "%z". For
287 example: 0395J1FR904324
288 wwn Regex can be of the form
290 "host_wwnn:host_wwpn:tar‐
291 get_wwnn:target_wwpn" these values
292 can be looked up through sysfs or
293 by running multipathd show paths
294 format "%N:%R:%n:%r". For example:
295 0x200100e08ba0aea0:0x210100e08ba0aea0:.*:.*
296 , .*:.*:iqn.2009-10.com.red‐
297 hat.msp.lab.ask-06:.*
298 path_latency
300 Needs a value of the form "io_num=<20>
301 base_num=<10>"
302 io_num The number of read IOs sent to the
304 current path continuously, used to
305 calculate the average path
306 latency. Valid Values: Integer,
307 [2, 200].
308 base_num
310 The base number value of logarith‐
311 mic scale, used to partition dif‐
312 ferent priority ranks. Valid Val‐
313 ues: Integer, [2, 10]. And Max
314 average latency value is 100s, min
315 average latency value is 1us. For
316 example: If base_num=10, the paths
317 will be grouped in priority groups
318 with path latency <=1us, (1us,
319 10us], (10us, 100us], (100us,
320 1ms], (1ms, 10ms], (10ms, 100ms],
321 (100ms, 1s], (1s, 10s], (10s,
322 100s], >100s.
323 alua If exclusive_pref_bit is set, paths with
325 the preferred path bit set will always be
326 in their own path group.
327 sysfs If exclusive_pref_bit is set, paths with
329 the preferred path bit set will always be
330 in their own path group.
331 datacore
333 preferredsds
335 (Mandatory) The preferred "SDS
336 name".
337 timeout (Optional) The timeout for the
339 INQUIRY, in ms.
340 iet
342 preferredip=...
344 (Mandatory) Th preferred IP
345 address, in dotted decimal nota‐
346 tion, for iSCSI targets.
347 The default is: <unset>
349 features Specify any device-mapper features to be used. Syntax
351 is num list where num is the number, between 0 and 8,
352 of features in list. Possible values for the feature
353 list are:
354 queue_if_no_path
356 (Deprecated, superseded by no_path_retry)
357 Queue I/O if no path is active. Identical
358 to the no_path_retry with queue value. If
359 both this feature and no_path_retry are
360 set, the latter value takes precedence.
361 See KNOWN ISSUES.
362 pg_init_retries <times>
364 (Since kernel 2.6.24) Number of times to
365 retry pg_init, it must be between 1 and
366 50.
367 pg_init_delay_msecs <msecs>
369 (Since kernel 2.6.38) Number of msecs
370 before pg_init retry, it must be between 0
371 and 60000.
372 queue_mode <mode>
374 (Since kernel 4.8) Select the the queueing
375 mode per multipath device. <mode> can be
376 bio, rq or mq, which corresponds to bio-
377 based, request-based, and block-multiqueue
378 (blk-mq) request-based, respectively. The
379 default depends on the kernel parameter
380 dm_mod.use_blk_mq. It is mq if the latter
381 is set, and rq otherwise.
382 The default is: <unset>
384 path_checker The default method used to determine the paths state.
386 Possible values are:
387 readsector0 (Deprecated) Read the first sector of the
389 device. This checker is being deprecated,
390 please use tur instead.
391 tur Issue a TEST UNIT READY command to the
393 device.
394 emc_clariion
396 (Hardware-dependent) Query the DGC/EMC
397 specific EVPD page 0xC0 to determine the
398 path state for CLARiiON CX/AX and EMC VNX
399 and Unity arrays families.
400 hp_sw (Hardware-dependent) Check the path state
402 for HP/COMPAQ/DEC HSG80 and MSA/HSV arrays
403 with Active/Standby mode exclusively.
404 rdac (Hardware-dependent) Check the path state
406 for LSI/Engenio/NetApp RDAC class as
407 NetApp SANtricity E/EF Series, and OEM
408 arrays from IBM DELL SGI STK and SUN.
409 directio (Deprecated) Read the first sector with
411 direct I/O. This checker is being depre‐
412 cated, it could cause spurious path fail‐
413 ures under high load. Please use tur
414 instead.
415 cciss_tur (Hardware-dependent) Check the path state
417 for HP/COMPAQ Smart Array(CCISS) con‐
418 trollers.
419 none Do not check the device, fallback to use
421 the values retrieved from sysfs
422 The default is: tur
424 alias_prefix The user_friendly_names prefix.
426 The default is: mpath
428 failback Tell multipathd how to manage path group failback. To
430 select immediate or a value, it's mandatory that the
431 device has support for a working prioritizer.
432 immediate Immediately failback to the highest prior‐
434 ity pathgroup that contains active paths.
435 manual Do not perform automatic failback.
437 followover Used to deal with multiple computers
439 accessing the same Active/Passive storage
440 devices. Only perform automatic failback
441 when the first path of a pathgroup becomes
442 active. This keeps a cluster node from
443 automatically failing back when another
444 node requested the failover.
445 values > 0 Deferred failback (time to defer in sec‐
447 onds).
448 The default is: manual
450 rr_min_io Number of I/O requests to route to a path before
452 switching to the next in the same path group. This is
453 only for Block I/O(BIO) based multipath and only apply
454 to round-robin path_selector.
455 The default is: 1000
457 rr_min_io_rq Number of I/O requests to route to a path before
459 switching to the next in the same path group. This is
460 only for Request based multipath and only apply to
461 round-robin path_selector.
462 The default is: 1
464 max_fds Specify the maximum number of file descriptors that
466 can be opened by multipath and multipathd. This is
467 equivalent to ulimit -n. A value of max will set this
468 to the system limit from /proc/sys/fs/nr_open. If this
469 is not set, the maximum number of open fds is taken
470 from the calling process. It is usually 1024. To be
471 safe, this should be set to the maximum number of
472 paths plus 32, if that number is greated than 1024.
473 The default is: max
475 rr_weight If set to priorities the multipath configurator will
477 assign path weights as "path prio * rr_min_io". Possi‐
478 ble values are priorities or uniform . Only apply to
479 round-robin path_selector.
480 The default is: uniform
482 no_path_retry Specify what to do when all paths are down. Possible
484 values are:
485 value > 0 Number of retries until disable I/O queue‐
487 ing.
488 fail For immediate failure (no I/O queueing).
490 queue For never stop I/O queueing, similar to
492 queue_if_no_path. See KNOWN ISSUES.
493 The default is: fail
495 queue_without_daemon
497 If set to no , when multipathd stops, queueing will be
498 turned off for all devices. This is useful for
499 devices that set no_path_retry. If a machine is shut
500 down while all paths to a device are down, it is pos‐
501 sible to hang waiting for I/O to return from the
502 device after multipathd has been stopped. Without mul‐
503 tipathd running, access to the paths cannot be
504 restored, and the kernel cannot be told to stop queue‐
505 ing I/O. Setting queue_without_daemon to no , avoids
506 this problem.
507 The default is: no
509 checker_timeout Specify the timeout to use for path checkers and pri‐
511 oritizers that issue SCSI commands with an explicit
512 timeout, in seconds.
513 The default is: in /sys/block/sd<x>/device/timeout
515 flush_on_last_del
517 If set to yes , multipathd will disable queueing when
518 the last path to a device has been deleted.
519 The default is: no
521 user_friendly_names
523 If set to yes , using the bindings file /etc/multi‐
524 path/bindings to assign a persistent and unique alias
525 to the multipath, in the form of mpath<n>. If set to
526 no use the WWID as the alias. In either case this be
527 will be overridden by any specific aliases in the mul‐
528 tipaths section.
529 The default is: no
531 fast_io_fail_tmo Specify the number of seconds the SCSI layer will wait
533 after a problem has been detected on a FC remote port
534 before failing I/O to devices on that remote port.
535 This should be smaller than dev_loss_tmo. Setting this
536 to off will disable the timeout.
537 The default is: in 5
539 dev_loss_tmo Specify the number of seconds the SCSI layer will wait
541 after a problem has been detected on a FC remote port
542 before removing it from the system. This can be set to
543 "infinity" which sets it to the max value of
544 2147483647 seconds, or 68 years. It will be automati‐
545 cally adjusted to the overall retry interval
546 no_path_retry * polling_interval if a number of
547 retries is given with no_path_retry and the overall
548 retry interval is longer than the specified
549 dev_loss_tmo value. The Linux kernel will cap this
550 value to 600 if fast_io_fail_tmo is not set. See KNOWN
551 ISSUES.
552 The default is: 600
554 bindings_file The full pathname of the binding file to be used when
556 the user_friendly_names option is set.
557 The default is: /etc/multipath/bindings
559 wwids_file The full pathname of the WWIDs file, which is used by
561 multipath to keep track of the WWIDs for LUNs it has
562 created multipath devices on in the past.
563 The default is: /etc/multipath/wwids
565 prkeys_file The full pathname of the prkeys file, which is used by
567 multipathd to keep track of the persistent reservation
568 key used for a specific WWID, when reservation_key is
569 set to file.
570 The default is /etc/multipath/prkeys
572 log_checker_err If set to once , multipathd logs the first path
574 checker error at logging level 2. Any later errors are
575 logged at level 3 until the device is restored. If set
576 to always , multipathd always logs the path checker
577 error at logging level 2.
578 The default is: always
580 reservation_key This is the service action reservation key used by
582 mpathpersist. It must be set for all multipath devices
583 using persistent reservations, and it must be the same
584 as the RESERVATION KEY field of the PERSISTENT RESERVE
585 OUT parameter list which contains an 8-byte value pro‐
586 vided by the application client to the device server
587 to identify the I_T nexus. If the --param-aptpl option
588 is used when registering the key with mpathpersist,
589 :aptpl must be appended to the end of the reservation
590 key.
591 Alternatively, this can be set to file, which will
593 store the RESERVATION KEY registered by mpathpersist
594 in the prkeys_file. multipathd will then use this key
595 to register additional paths as they appear. When the
596 registration is removed, the RESERVATION KEY is
597 removed from the prkeys_file. The prkeys file will
598 automatically keep track of whether the key was regis‐
599 tered with --param-aptpl.
600 The default is: <unset>
602 all_tg_pt Set the 'all targets ports' flag when registering keys
604 with mpathpersist. Some arrays automatically set and
605 clear registration keys on all target ports from a
606 host, instead of per target port per host. The
607 ALL_TG_PT flag must be set to successfully use mpath‐
608 persist on these arrays. Setting this option is iden‐
609 tical to calling mpathpersist with --param-alltgpt
610 The default is: no
612 retain_attached_hw_handler
614 (Obsolete for kernels >= 4.3) If set to yes and the
615 SCSI layer has already attached a hardware_handler to
616 the device, multipath will not force the device to use
617 the hardware_handler specified by mutipath.conf. If
618 the SCSI layer has not attached a hardware handler,
619 multipath will continue to use its configured hardware
620 handler.
621 The default is: yes
623 Important Note: Linux kernel 4.3 or newer always
625 behaves as if "retain_attached_hw_handler yes" was
626 set.
627 detect_prio If set to yes , multipath will try to detect if the
629 device supports SCSI-3 ALUA. If so, the device will
630 automatically use the sysfs prioritizer if the
631 required sysf attributes access_state and pre‐
632 ferred_path are supported, or the alua prioritizer if
633 not. If set to no , the prioritizer will be selected
634 as usual.
635 The default is: yes
637 detect_checker if set to yes , multipath will try to detect if the
639 device supports SCSI-3 ALUA. If so, the device will
640 automatically use the tur checker. If set to no , the
641 checker will be selected as usual.
642 The default is: yes
644 force_sync If set to yes , multipathd will call the path checkers
646 in sync mode only. This means that only one checker
647 will run at a time. This is useful in the case where
648 many multipathd checkers running in parallel causes
649 significant CPU pressure.
650 The default is: no
652 strict_timing If set to yes , multipathd will start a new path
654 checker loop after exactly one second, so that each
655 path check will occur at exactly polling_interval sec‐
656 onds. On busy systems path checks might take longer
657 than one second; here the missing ticks will be
658 accounted for on the next round. A warning will be
659 printed if path checks take longer than polling_inter‐
660 val seconds.
661 The default is: no
663 deferred_remove If set to yes , multipathd will do a deferred remove
665 instead of a regular remove when the last path device
666 has been deleted. This means that if the multipath
667 device is still in use, it will be freed when the last
668 user closes it. If path is added to the multipath
669 device before the last user closes it, the deferred
670 remove will be canceled.
671 The default is: no
673 partition_delimiter
675 This parameter controls how multipath chooses the
676 names of partition devices of multipath maps if a mul‐
677 tipath map is renamed (e.g. if a map alias is added or
678 changed). If this parameter is set to a string other
679 than "/UNSET/" (even the empty string), multipath
680 inserts that string between device name and partition
681 number to construct the partition device name. Other‐
682 wise (i.e. if this parameter is unset or has the value
683 "/UNSET/"), the behavior depends on the map name: if
684 it ends in a digit, a "p" is inserted between name and
685 partition number; otherwise, the partition number is
686 simply appended. Distributions may use a non-null
687 default value for this option; in this case, the user
688 must set it to "/UNSET/" to obtain the original
689 <unset> behavior. Use multipath -T to check the cur‐
690 rent settings.
691 The default is: <unset>
693 config_dir If set to anything other than "", multipath will
695 search this directory alphabetically for file ending
696 in ".conf" and it will read configuration information
697 from them, just as if it was in /etc/multipath.conf.
698 config_dir must either be "" or a fully qualified
699 directory name.
700 The default is: /etc/multipath/conf.d/
702 san_path_err_threshold
704 If set to a value greater than 0, multipathd will
705 watch paths and check how many times a path has been
706 failed due to errors.If the number of failures on a
707 particular path is greater then the
708 san_path_err_threshold, then the path will not rein‐
709 state till san_path_err_recovery_time. These path
710 failures should occur within a san_path_err_for‐
711 get_rate checks, if not we will consider the path is
712 good enough to reinstantate. See "Shaky paths detec‐
713 tion" below.
714 The default is: no
716 san_path_err_forget_rate
718 If set to a value greater than 0, multipathd will
719 check whether the path failures has exceeded the
720 san_path_err_threshold within this many checks i.e
721 san_path_err_forget_rate . If so we will not rein‐
722 stante the path till san_path_err_recovery_time. See
723 "Shaky paths detection" below.
724 The default is: no
726 san_path_err_recovery_time
728 If set to a value greater than 0, multipathd will make
729 sure that when path failures has exceeded the
730 san_path_err_threshold within san_path_err_forget_rate
731 then the path will be placed in failed state for
732 san_path_err_recovery_time duration.Once
733 san_path_err_recovery_time has timeout we will rein‐
734 stante the failed path . san_path_err_recovery_time
735 value should be in secs. See "Shaky paths detection"
736 below.
737 The default is: no
739 marginal_path_double_failed_time
741 One of the four parameters of supporting path check
742 based on accounting IO error such as intermittent
743 error. When a path failed event occurs twice in mar‐
744 ginal_path_double_failed_time seconds due to an IO
745 error and all the other three parameters are set, mul‐
746 tipathd will fail the path and enqueue this path into
747 a queue of which members are sent a couple of continu‐
748 ous direct reading asynchronous IOs at a fixed sample
749 rate of 10HZ to start IO error accounting process. See
750 "Shaky paths detection" below.
751 The default is: no
753 marginal_path_err_sample_time
755 One of the four parameters of supporting path check
756 based on accounting IO error such as intermittent
757 error. If it is set to a value no less than 120, when
758 a path fail event occurs twice in marginal_path_dou‐
759 ble_failed_time second due to an IO error, multipathd
760 will fail the path and enqueue this path into a queue
761 of which members are sent a couple of continuous
762 direct reading asynchronous IOs at a fixed sample rate
763 of 10HZ to start the IO accounting process for the
764 path will last for marginal_path_err_sample_time. If
765 the rate of IO error on a particular path is greater
766 than the marginal_path_err_rate_threshold, then the
767 path will not reinstate for margin‐
768 al_path_err_recheck_gap_time seconds unless there is
769 only one active path. After margin‐
770 al_path_err_recheck_gap_time expires, the path will be
771 requeueed for rechecking. If checking result is good
772 enough, the path will be reinstated. See "Shaky paths
773 detection" below.
774 The default is: no
776 marginal_path_err_rate_threshold
778 The error rate threshold as a permillage (1/1000). One
779 of the four parameters of supporting path check based
780 on accounting IO error such as intermittent error.
781 Refer to marginal_path_err_sample_time. If the rate of
782 IO errors on a particular path is greater than this
783 parameter, then the path will not reinstate for mar‐
784 ginal_path_err_recheck_gap_time seconds unless there
785 is only one active path. See "Shaky paths detection"
786 below.
787 The default is: no
789 marginal_path_err_recheck_gap_time
791 One of the four parameters of supporting path check
792 based on accounting IO error such as intermittent
793 error. Refer to marginal_path_err_sample_time. If this
794 parameter is set to a positive value, the failed path
795 of which the IO error rate is larger than margin‐
796 al_path_err_rate_threshold will be kept in failed
797 state for marginal_path_err_recheck_gap_time seconds.
798 When marginal_path_err_recheck_gap_time seconds
799 expires, the path will be requeueed for checking. If
800 checking result is good enough, the path will be rein‐
801 stated, or else it will keep failed. See "Shaky paths
802 detection" below.
803 The default is: no
805 delay_watch_checks
807 If set to a value greater than 0, multipathd will
808 watch paths that have recently become valid for this
809 many checks. If they fail again while they are being
810 watched, when they next become valid, they will not be
811 used until they have stayed up for delay_wait_checks
812 checks. See "Shaky paths detection" below.
813 The default is: no
815 delay_wait_checks
817 If set to a value greater than 0, when a device that
818 has recently come back online fails again within
819 delay_watch_checks checks, the next time it comes back
820 online, it will marked and delayed, and not used until
821 it has passed delay_wait_checks checks. See "Shaky
822 paths detection" below.
823 The default is: no
825 find_multipaths This option controls whether multipath and multipathd
827 try to create multipath maps over non-blacklisted
828 devices they encounter. This matters a) when a device
829 is encountered by multipath -u during udev rule pro‐
830 cessing (a device is blocked from further processing
831 by higher layers - such as LVM - if and only if it´s
832 considered a valid multipath device path), and b) when
833 multipathd detects a new device. The following values
834 are possible:
835 strict Both multipath and multipathd treat only
837 such devices as multipath devices which have
838 been part of a multipath map previously, and
839 which are therefore listed in the
840 wwids_file. Users can manually set up multi‐
841 path maps using the multipathd add map com‐
842 mand. Once set up manually, the map is
843 remembered in the wwids file and will be set
844 up automatically in the future.
845 no Multipath behaves like strict. Multipathd
847 behaves like greedy.
848 yes Both multipathd and multipath treat a device
850 as multipath device if the conditions for
851 strict are met, or if at least two non-
852 blacklisted paths with the same WWID have
853 been detected.
854 greedy Both multipathd and multipath treat every
856 non-blacklisted device as multipath device
857 path.
858 smart This differs from find_multipaths yes only
860 in the way it treats new devices for which
861 only one path has been detected yet. When
862 such a device is first encounted in udev
863 rules, it is treated as a multipath device.
864 multipathd waits whether additional paths
865 with the same WWID appears. If that happens,
866 it sets up a multipath map. If it doesn´t
867 happen until a timeout expires, or if set‐
868 ting up the map fails, a new uevent is trig‐
869 gered for the device; at second encounter in
870 the udev rules, the device will be treated
871 as non-multipath and passed on to upper lay‐
872 ers. Note: this may cause delays during
873 device detection if there are single-path
874 devices which aren´t blacklisted.
875 The default is: strict
877 find_multipaths_timeout
879 Timeout, in seconds, to wait for additional paths
880 after detecting the first one, if find_multipaths
881 "smart" (see above) is set. If the value is positive,
882 this timeout is used for all unknown, non-blacklisted
883 devices encountered. If the value is negative (recom‐
884 mended), it's only applied to "known" devices that
885 have an entry in multipath's hardware table, either in
886 the built-in table or in a device section; other
887 ("unknown") devices will use a timeout of only 1 sec‐
888 ond to avoid booting delays. The value 0 means "use
889 the built-in default". If find_multipath has a value
890 other than smart, this option has no effect.
891 The default is: -10 (10s for known and 1s for unknown
893 hardware)
894 uxsock_timeout CLI receive timeout in milliseconds. For larger sys‐
896 tems CLI commands might timeout before the multipathd
897 lock is released and the CLI command can be processed.
898 This will result in errors like "timeout receiving
899 packet" to be returned from CLI commands. In these
900 cases it is recommended to increase the CLI timeout to
901 avoid those issues.
902 The default is: 1000
904 retrigger_tries Sets the number of times multipathd will try to
906 retrigger a uevent to get the WWID.
907 The default is: 3
909 retrigger_delay Sets the amount of time, in seconds, to wait between
911 retriggers.
912 The default is: 10
914 missing_uev_wait_timeout
916 Controls how many seconds multipathd will wait, after
917 a new multipath device is created, to receive a change
918 event from udev for the device, before automatically
919 enabling device reloads. Usually multipathd will delay
920 reloads on a device until it receives a change uevent
921 from the initial table load.
922 The default is: 30
924 skip_kpartx If set to yes , kpartx will not automatically create
926 partitions on the device.
927 The default is: no
929 disable_changed_wwids
931 This option is deprecated and ignored. If the WWID of
932 a path suddenly changes, multipathd handles it as if
933 it was removed and then added again.
934 remove_retries This sets how may times multipath will retry removing
936 a device that is in-use. Between each attempt, multi‐
937 path will sleep 1 second.
938 The default is: 0
940 max_sectors_kb Sets the max_sectors_kb device parameter on all path
942 devices and the multipath device to the specified
943 value.
944 The default is: <device dependent>
946 ghost_delay Sets the number of seconds that multipath will wait
948 after creating a device with only ghost paths before
949 marking it ready for use in systemd. This gives the
950 active paths time to appear before the multipath runs
951 the hardware handler to switch the ghost paths to
952 active ones. Setting this to 0 or on makes multipath
953 immediately mark a device with only ghost paths as
954 ready.
955 The default is no
957
959 The blacklist section is used to exclude specific devices from the mul‐
960 tipath topology. It is most commonly used to exclude local disks or
961 non-disk devices (such as LUNs for the storage array controller) from
962 being handled by multipath-tools.
963 The blacklist_exceptions section is used to revert the actions of the
965 blacklist section. This allows one to selectively include ("whitelist")
966 devices which would normally be excluded via the blacklist section. A
967 common usage is to blacklist "everything" using a catch-all regular
968 expression, and create specific blacklist_exceptions entries for those
969 devices that should be handled by multipath-tools.
970 The following keywords are recognized in both sections. The defaults
972 are empty unless explicitly stated.
973 devnode Regular expression matching the device nodes to be
975 excluded/included.
976 The default blacklist consists of the regular expres‐
978 sions "^(ram|zram|raw|loop|fd|md|dm-|sr|scd|st|dcss‐
979 blk)[0-9]" and "^(td|hd|vd)[a-z]". This causes virtual
980 devices, non-disk devices, and some other device types
981 to be excluded from multipath handling by default.
982 wwid Regular expression for the World Wide Identifier of a
984 device to be excluded/included.
985 device Subsection for the device description. This subsection
987 recognizes the vendor and product keywords. Both are
988 regular expressions. For a full description of these
989 keywords please see the devices section description.
990 property Regular expression for an udev property. All devices
992 that have matching udev properties will be
993 excluded/included. The handling of the property key‐
994 word is special, because if a property black‐
995 list_exception is set, devices must have at least one
996 whitelisted udev property; otherwise they're treated
997 as blacklisted, and the message "blacklisted, udev
998 property missing" is displayed in the logs. For exam‐
999 ple, setting the property blacklist_exception to
1000 (SCSI_IDENT_|ID_WWN), will cause well-behaved SCSI
1001 devices and devices that provide a WWN (World Wide
1002 Number) to be included, and all others to be excluded.
1003 This works to exclude most non-multipathable devices.
1004 protocol Regular expression for the protocol of a device to be
1006 excluded/included.
1007 The protocol strings that multipath recognizes are
1009 scsi:fcp, scsi:spi, scsi:ssa, scsi:sbp, scsi:srp,
1010 scsi:iscsi, scsi:sas, scsi:adt, scsi:ata, scsi:unspec,
1011 ccw, cciss, nvme, and undef. The protocol that a path
1012 is using can be viewed by running multipathd show
1013 paths format "%d %P"
1014 For every device, these 5 blacklist criteria are evaluated in the the
1016 order "property, devnode, device, protocol, wwid". If a device turns
1017 out to be blacklisted by any criterion, it's excluded from handling by
1018 multipathd, and the later criteria aren't evaluated any more. For each
1019 criterion, the whitelist takes precedence over the blacklist if a
1020 device matches both.
1021 Note: Besides the blacklist and whitelist, other configuration options
1023 such as find_multipaths have an impact on whether or not a given device
1024 is handled by multipath-tools.
1025
1027 The multipaths section allows setting attributes of multipath maps. The
1028 attributes that are set via the multipaths section (see list below)
1029 take precedence over all other configuration settings, including those
1030 from the overrides section.
1031 The only recognized attribute for the multipaths section is the multi‐
1033 path subsection. If there are multiple multipath subsections matching a
1034 given WWID, the contents of these sections are merged, and settings
1035 from later entries take precedence.
1036 The multipath subsection recognizes the following attributes:
1038 wwid (Mandatory) World Wide Identifier. Detected multipath
1040 maps are matched agains this attribute. Note that,
1041 unlike the wwid attribute in the blacklist section,
1042 this is not a regular expression or a substring; WWIDs
1043 must match exactly inside the multipaths section.
1044 alias Symbolic name for the multipath map. This takes prece‐
1046 dence over a an entry for the same WWID in the bind‐
1047 ings_file.
1048 The following attributes are optional; if not set the default values
1050 are taken from the overrides, devices, or defaults section:
1051 path_grouping_policy
1053 path_selector
1054 prio
1055 prio_args
1056 failback
1057 rr_weight
1058 no_path_retry
1059 rr_min_io
1060 rr_min_io_rq
1061 flush_on_last_del
1062 features
1063 reservation_key
1064 user_friendly_names
1065 deferred_remove
1066 san_path_err_threshold
1067 san_path_err_forget_rate
1068 san_path_err_recovery_time
1069 marginal_path_err_sample_time
1070 marginal_path_err_rate_threshold
1071 marginal_path_err_recheck_gap_time
1072 marginal_path_double_failed_time
1073 delay_watch_checks
1074 delay_wait_checks
1075 skip_kpartx
1076 max_sectors_kb
1077 ghost_delay
1078
1080 multipath-tools have a built-in device table with reasonable defaults
1081 for more than 100 known multipath-capable storage devices. The devices
1082 section can be used to override these settings. If there are multiple
1083 matches for a given device, the attributes of all matching entries are
1084 applied to it. If an attribute is specified in several matching device
1085 subsections, later entries take precedence. Thus, entries in files
1086 under config_dir (in reverse alphabetical order) have the highest
1087 precedence, followed by entries in multipath.conf; the built-in hard‐
1088 ware table has the lowest precedence. Inside a configuration file,
1089 later entries have higher precedence than earlier ones.
1090 The only recognized attribute for the devices section is the device
1092 subsection. Devices detected in the system are matched against the
1093 device entries using the vendor, product, and revision fields, which
1094 are all POSIX Extended regular expressions (see regex(7)).
1095 The vendor, product, and revision fields that multipath or multipathd
1097 detect for devices in a system depend on the device type. For SCSI
1098 devices, they correspond to the respective fields of the SCSI INQUIRY
1099 page. In general, the command 'multipathd show paths format "%d %s"'
1100 command can be used to see the detected properties for all devices in
1101 the system.
1102 The device subsection recognizes the following attributes:
1104 vendor (Mandatory) Regular expression to match the vendor
1106 name.
1107 product (Mandatory) Regular expression to match the product
1109 name.
1110 revision Regular expression to match the product revision. If
1112 not specified, any revision matches.
1113 product_blacklist
1115 Products with the given vendor matching this string
1116 are blacklisted. This is equivalent to a device entry
1117 in the blacklist section with the vendor attribute set
1118 to this entry's vendor, and the product attribute set
1119 to the value of product_blacklist.
1120 alias_prefix The user_friendly_names prefix to use for this device
1122 type, instead of the default "mpath".
1123 hardware_handler The hardware handler to use for this device type. The
1125 following hardware handler are implemented:
1126 1 emc (Hardware-dependent) Hardware handler for
1128 DGC class arrays as CLARiiON CX/AX and EMC
1129 VNX and Unity families.
1130 1 rdac (Hardware-dependent) Hardware handler for
1132 LSI/Engenio/NetApp RDAC class as NetApp
1133 SANtricity E/EF Series, and OEM arrays
1134 from IBM DELL SGI STK and SUN.
1135 1 hp_sw (Hardware-dependent) Hardware handler for
1137 HP/COMPAQ/DEC HSG80 and MSA/HSV arrays
1138 with Active/Standby mode exclusively.
1139 1 alua (Hardware-dependent) Hardware handler for
1141 SCSI-3 ALUA compatible arrays.
1142 1 ana (Hardware-dependent) Hardware handler for
1144 NVMe ANA compatible arrays.
1145 The default is: <unset>
1147 Important Note: Linux kernels 4.3 and newer automati‐
1149 cally attach a device handler to known devices (which
1150 includes all devices supporting SCSI-3 ALUA) and dis‐
1151 allow changing the handler afterwards. Setting hard‐
1152 ware_handler for such devices on these kernels has no
1153 effect.
1154 The following attributes are optional; if not set the default values
1156 are taken from the defaults section:
1157 path_grouping_policy
1159 uid_attribute
1160 getuid_callout
1161 path_selector
1162 path_checker
1163 prio
1164 prio_args
1165 features
1166 failback
1167 rr_weight
1168 no_path_retry
1169 rr_min_io
1170 rr_min_io_rq
1171 fast_io_fail_tmo
1172 dev_loss_tmo
1173 flush_on_last_del
1174 user_friendly_names
1175 retain_attached_hw_handler
1176 detect_prio
1177 detect_checker
1178 deferred_remove
1179 san_path_err_threshold
1180 san_path_err_forget_rate
1181 san_path_err_recovery_time
1182 marginal_path_err_sample_time
1183 marginal_path_err_rate_threshold
1184 marginal_path_err_recheck_gap_time
1185 marginal_path_double_failed_time
1186 delay_watch_checks
1187 delay_wait_checks
1188 skip_kpartx
1189 max_sectors_kb
1190 ghost_delay
1191 all_tg_pt
1192
1194 The overrides section recognizes the following optional attributes; if
1195 not set the values are taken from the devices or defaults sections:
1196 path_grouping_policy
1198 uid_attribute
1199 getuid_callout
1200 path_selector
1201 path_checker
1202 alias_prefix
1203 features
1204 prio
1205 prio_args
1206 failback
1207 rr_weight
1208 no_path_retry
1209 rr_min_io
1210 rr_min_io_rq
1211 flush_on_last_del
1212 fast_io_fail_tmo
1213 dev_loss_tmo
1214 user_friendly_names
1215 retain_attached_hw_handler
1216 detect_prio
1217 detect_checker
1218 deferred_remove
1219 san_path_err_threshold
1220 san_path_err_forget_rate
1221 san_path_err_recovery_time
1222 marginal_path_err_sample_time
1223 marginal_path_err_rate_threshold
1224 marginal_path_err_recheck_gap_time
1225 marginal_path_double_failed_time
1226 delay_watch_checks
1227 delay_wait_checks
1228 skip_kpartx
1229 max_sectors_kb
1230 ghost_delay
1231 all_tg_pt
1232
1234 Multipath uses a World Wide Identification (WWID) to determine which
1235 paths belong to the same device. Each path presenting the same WWID is
1236 assumed to point to the same device.
1237 The WWID is generated by four methods (in the order of preference):
1239 uid_attrs The WWID is derived from udev attributes by matching
1241 the device node name. See description of uid_attrs in
1242 the defaults section above.
1243 getuid_callout Use the specified external program; cf getuid_callout
1245 above. Care should be taken when using this method;
1246 the external program needs to be loaded from disk for
1247 execution, which might lead to deadlock situations in
1248 an all-paths-down scenario.
1249 uid_attribute Use the value of the specified udev attribute; cf
1251 uid_attribute above. This method is preferred to
1252 getuid_callout as multipath does not need to call any
1253 external programs here. However, under certain circum‐
1254 stances udev might not be able to generate the
1255 requested variable.
1256 vpd_pg83 If none of the getuid_callout or uid_attribute parame‐
1258 ters are present multipath will try to use the sysfs
1259 attribute vpd_pg83 to generate the WWID.
1260
1262 A common problem in SAN setups is the occurence of intermittent errors:
1263 a path is unreachable, then reachable again for a short time, disap‐
1264 pears again, and so forth. This happens typically on unstable intercon‐
1265 nects. It is undesirable to switch pathgroups unnecessarily on such
1266 frequent, unreliable events. multipathd supports three different meth‐
1267 ods for detecting this situation and dealing with it. All methods share
1268 the same basic mode of operation: If a path is found to be "shaky" or
1269 "flipping", and appears to be in healthy status, it is not reinstated
1270 (put back to use) immediately. Instead, it is watched for some time,
1271 and only reinstated if the healthy state appears to be stable. The
1272 logic of determining "shaky" condition, as well as the logic when to
1273 reinstate, differs between the three methods.
1274 "delay_checks" failure tracking
1276 If a path fails again within a delay_watch_checks interval
1277 after a failure, don't reinstate it until it passes a
1278 delay_wait_checks interval in always good status. The inter‐
1279 vals are measured in "ticks", i.e. the time between path checks
1280 by multipathd, which is variable and controlled by the
1281 polling_interval and max_polling_interval parameters.
1282 "marginal_path" failure tracking
1284 If a second failure event (good->bad transition) occurs within
1285 marginal_path_double_failed_time seconds after a failure, high-
1286 frequency monitoring is started for the affected path: I/O is
1287 sent at a rate of 10 per second. This is done for margin‐
1288 al_path_err_sample_time seconds. During this period, the path
1289 is not reinstated. If the rate of errors remains below margin‐
1290 al_path_err_rate_threshold during the monitoring period, the
1291 path is reinstated. Otherwise, it is kept in failed state for
1292 marginal_path_err_recheck_gap_time, and after that, it is moni‐
1293 tored again. For this method, time intervals are measured in
1294 seconds.
1295 "san_path_err" failure tracking
1297 multipathd counts path failures for each path. Once the number
1298 of failures exceeds the value given by san_path_err_threshold,
1299 the path is not reinstated for san_path_err_recovery_time
1300 ticks. While counting failures, multipathd "forgets" one past
1301 failure every "san_path_err_forget_rate" ticks; thus if errors
1302 don't occur more often then once in the forget rate interval,
1303 the failure count doesn't increase and the threshold is never
1304 reached. As for the delay_xy method, intervals are measured in
1305 "ticks".
1306 This method is deprecated in favor of the "marginal_path" fail‐
1308 ure tracking method, and only offered for backward compatibil‐
1309 ity.
1310 See the documentation of the individual options above for details. It
1312 is strongly discouraged to use more than one of these methods for any
1313 given multipath map, because the two concurrent methods may interact in
1314 unpredictable ways. If the "marginal_path" method is active, the
1315 "san_path_err" parameters are implicitly set to 0.
1316
1318 The usage of queue_if_no_path option can lead to D state processes
1319 being hung and not killable in situations where all the paths to the
1320 LUN go offline. It is advisable to use the no_path_retry option
1321 instead.
1322 The use of queue_if_no_path or no_path_retry might lead to a deadlock
1324 if the dev_loss_tmo setting results in a device being removed while I/O
1325 is still queued. The multipath daemon will update the dev_loss_tmo set‐
1326 ting accordingly to avoid this deadlock. Hence if both values are spec‐
1327 ified the order of precedence is no_path_retry, queue_if_no_path,
1328 dev_loss_tmo.
1329
1331 udev(8), dmsetup(8), multipath(8), multipathd(8).
1332
1334 multipath-tools was developed by Christophe Varoqui, <christophe.varo‐
1335 qui@opensvc.com> and others.
1336Linux 2018-05-21 MULTIPATH.CONF(5)