1MULTIPATH.CONF(5) File Formats Manual MULTIPATH.CONF(5)
2
3
4
6 multipath.conf - multipath daemon configuration file.
7
9 /etc/multipath.conf is the configuration file for the multipath daemon.
10 It is used to overwrite the built-in configuration table of multipathd.
11 Any line whose first non-white-space character is a '#' is considered a
12 comment line. Empty lines are ignored.
13
14 Currently used multipathd configuration can be displayed with the mul‐
15 tipath -t or multipathd show config command.
16
18 The configuration file contains entries of the form:
19
20 <section> {
21 <attribute> <value>
22 ...
23 <subsection> {
24 <attribute> <value>
25 ...
26 }
27 }
28
29 Each section contains one or more attributes or subsections. The recog‐
30 nized keywords for attributes or subsections depend on the section in
31 which they occur.
32
33 <attribute> and <value> must be on a single line. <attribute> is one
34 of the keywords listed in this man page. <value> is either a simple
35 word (containing no whitespace and none of the characters '"', '#', and
36 '!') or one string enclosed in double quotes ("..."). Outside a quoted
37 string, text starting with '#', and '!' is regarded as a comment and
38 ignored until the end of the line. Inside a quoted string, '#' and '!'
39 are normal characters, and whitespace is preserved. To represent a
40 double quote character inside a double quoted string, use two consecu‐
41 tive double quotes ('""'). Thus '2.5" SSD' can be written as "2.5""
42 SSD".
43
44 Opening braces ('{') must follow the (sub)section name on the same
45 line. Closing braces ('}') that mark the end of a (sub)section must be
46 the only non-whitespace character on the line. Whitespace is ignored
47 except inside double quotes, thus the indentation shown in the above
48 example is helpful for human readers but not mandatory.
49
50 Note on regular expressions: The multipath.conf syntax allows many
51 attribute values to be specified as POSIX Extended Regular Expressions
52 (see regex(7)). These regular expressions are case sensitive and not
53 anchored, thus the expression "bar" matches "barbie", "rhabarber", and
54 "wunderbar", but not "Barbie". To avoid unwanted substring matches,
55 standard regular expression syntax using the special characters "^" and
56 "$" can be used.
57
58 The following section keywords are recognized:
59
60 defaults This section defines default values for attributes
61 which are used whenever no values are given in the
62 appropriate device or multipath sections.
63
64 blacklist This section defines which devices should be excluded
65 from the multipath topology discovery.
66
67 blacklist_exceptions
68 This section defines which devices should be included
69 in the multipath topology discovery, despite being
70 listed in the blacklist section.
71
72 multipaths This section defines the multipath topologies. They
73 are indexed by a World Wide Identifier(WWID). For
74 details on the WWID generation see section WWID gener‐
75 ation below. Attributes set in this section take
76 precedence over all others.
77
78 devices This section defines the device-specific settings.
79 Devices are identified by vendor, product, and revi‐
80 sion.
81
82 overrides This section defines values for attributes that should
83 override the device-specific settings for all devices.
84
86 The defaults section recognizes the following keywords:
87
88 verbosity Default verbosity. Higher values increase the ver‐
89 bosity level. Valid levels are between 0 and 6.
90
91 The default is: 2
92
93 polling_interval Interval between two path checks in seconds. For prop‐
94 erly functioning paths, the interval between checks
95 will gradually increase to max_polling_interval. This
96 value will be overridden by the WatchdogSec setting in
97 the multipathd.service definition if systemd is used.
98
99 The default is: 5
100
101 max_polling_interval
102 Maximal interval between two path checks in seconds.
103
104 The default is: 4 * polling_interval
105
106 reassign_maps Enable reassigning of device-mapper maps. With this
107 option multipathd will remap existing device-mapper
108 maps to always point to multipath device, not the
109 underlying block devices. Possible values are yes and
110 no.
111
112 The default is: no
113
114 multipath_dir Directory where the dynamic shared objects are stored.
115 Defined at compile time, commonly /lib64/multipath/ or
116 /lib/multipath/.
117
118 The default is: <system dependent>
119
120 path_selector The default path selector algorithm to use; they are
121 offered by the kernel multipath target. There are
122 three selector algorithms:
123
124 round-robin 0
125 Loop through every path in the path group,
126 sending the same amount of I/O to each.
127 Some aspects of behavior can be controlled
128 with the attributes: rr_min_io,
129 rr_min_io_rq and rr_weight.
130
131 queue-length 0
132 (Since 2.6.31 kernel) Choose the path for
133 the next bunch of I/O based on the amount
134 of outstanding I/O to the path.
135
136 service-time 0
137 (Since 2.6.31 kernel) Choose the path for
138 the next bunch of I/O based on the amount
139 of outstanding I/O to the path and its
140 relative throughput.
141
142 The default is: service-time 0
143
144 path_grouping_policy
145 The default path grouping policy to apply to unspeci‐
146 fied multipaths. Possible values are:
147
148 failover One path per priority group.
149
150 multibus All paths in one priority group.
151
152 group_by_serial
153 One priority group per serial number.
154
155 group_by_prio
156 One priority group per priority value.
157 Priorities are determined by callout pro‐
158 grams specified as a global, per-con‐
159 troller or per-multipath option in the
160 configuration file.
161
162 group_by_node_name
163 One priority group per target node name.
164 Target node names are fetched in
165 /sys/class/fc_transport/target*/node_name.
166
167 The default is: failover
168
169 uid_attrs The udev attribute providing a unique path identifier
170 for corresponding type of path devices. If this field
171 is configured and matched with type of device, it
172 would override any other methods providing for device
173 unique identifier in config file, and it would acti‐
174 vate merging uevents according to the identifier to
175 promote effiecncy in processing uevents. It has no
176 default value, if you want to identify path by udev
177 attribute and to activate merging uevents for SCSI and
178 DASD devices, you can set its value as: uid_attrs
179 "sd:ID_SERIAL dasd:ID_UID".
180
181 The default is: <unset>
182
183 uid_attribute The udev attribute providing a unique path identifier.
184
185 The default is: for SCSI devices ID_SERIAL
186
187 The default is: for DASD devices ID_UID
188
189 The default is: for NVME devices ID_WWN
190
191 getuid_callout (Superseded by uid_attribute) The default program and
192 args to callout to obtain a unique path identifier.
193 Should be specified with an absolute path.
194
195 The default is: <unset>
196
197 prio The name of the path priority routine. The specified
198 routine should return a numeric value specifying the
199 relative priority of this path. Higher number have a
200 higher priority. "none" is a valid value. Currently
201 the following path priority routines are implemented:
202
203 const Return a constant priority of 1.
204
205 sysfs Use the sysfs attributes access_state and
206 preferred_path to generate the path prior‐
207 ity. This prioritizer accepts the optional
208 prio_arg exclusive_pref_bit.
209
210 emc (Hardware-dependent) Generate the path
211 priority for DGC class arrays as CLARiiON
212 CX/AX and EMC VNX and Unity families.
213
214 alua (Hardware-dependent) Generate the path
215 priority based on the SCSI-3 ALUA set‐
216 tings. This prioritizer accepts the
217 optional prio_arg exclusive_pref_bit.
218
219 ontap (Hardware-dependent) Generate the path
220 priority for NetApp ONTAP class and OEM
221 arrays as IBM NSeries.
222
223 rdac (Hardware-dependent) Generate the path
224 priority for LSI/Engenio/NetApp RDAC class
225 as NetApp SANtricity E/EF Series, and OEM
226 arrays from IBM DELL SGI STK and SUN.
227
228 hp_sw (Hardware-dependent) Generate the path
229 priority for HP/COMPAQ/DEC HSG80 and
230 MSA/HSV arrays with Active/Standby mode
231 exclusively.
232
233 hds (Hardware-dependent) Generate the path
234 priority for Hitachi AMS families of
235 arrays other than AMS 2000.
236
237 random Generate a random priority between 1 and
238 10.
239
240 weightedpath
241 Generate the path priority based on the
242 regular expression and the priority pro‐
243 vided as argument. Requires prio_args key‐
244 word.
245
246 path_latency
247 Generate the path priority based on a
248 latency algorithm. Requires prio_args
249 keyword.
250
251 ana (Hardware-dependent) Generate the path
252 priority based on the NVMe ANA settings.
253
254 datacore (Hardware-dependent) Generate the path
255 priority for some DataCore storage arrays.
256 Requires prio_args keyword.
257
258 iet (iSCSI only) Generate path priority for
259 iSCSI targets based on IP address.
260 Requires prio_args keyword.
261
262 The default depends on the detect_prio setting: If
263 detect_prio is yes (default), the default priority
264 algorithm is sysfs (except for NetAPP E-Series, where
265 it is alua). If detect_prio is no, the default prior‐
266 ity algorithm is const.
267
268 prio_args Arguments to pass to to the prio function. This only
269 applies to certain prioritizers:
270
271 weighted Needs a value of the form "<hbtl|dev‐
272 name|serial|wwn> <regex1> <prio1> <regex2>
273 <prio2> ..."
274
275 hbtl Regex can be of SCSI H:B:T:L for‐
276 mat. For example: 1:0:.:. ,
277 *:0:0:.
278
279 devname Regex can be of device name for‐
280 mat. For example: sda , sd.e
281
282 serial Regex can be of serial number for‐
283 mat. For example: .*J1FR.*324 .
284 The serial can be looked up
285 through sysfs or by running multi‐
286 pathd show paths format "%z". For
287 example: 0395J1FR904324
288
289 wwn Regex can be of the form
290 "host_wwnn:host_wwpn:tar‐
291 get_wwnn:target_wwpn" these values
292 can be looked up through sysfs or
293 by running multipathd show paths
294 format "%N:%R:%n:%r". For example:
295 0x200100e08ba0aea0:0x210100e08ba0aea0:.*:.*
296 , .*:.*:iqn.2009-10.com.red‐
297 hat.msp.lab.ask-06:.*
298
299 path_latency
300 Needs a value of the form "io_num=<20>
301 base_num=<10>"
302
303 io_num The number of read IOs sent to the
304 current path continuously, used to
305 calculate the average path
306 latency. Valid Values: Integer,
307 [2, 200].
308
309 base_num
310 The base number value of logarith‐
311 mic scale, used to partition dif‐
312 ferent priority ranks. Valid Val‐
313 ues: Integer, [2, 10]. And Max
314 average latency value is 100s, min
315 average latency value is 1us. For
316 example: If base_num=10, the paths
317 will be grouped in priority groups
318 with path latency <=1us, (1us,
319 10us], (10us, 100us], (100us,
320 1ms], (1ms, 10ms], (10ms, 100ms],
321 (100ms, 1s], (1s, 10s], (10s,
322 100s], >100s.
323
324 alua If exclusive_pref_bit is set, paths with
325 the preferred path bit set will always be
326 in their own path group.
327
328 sysfs If exclusive_pref_bit is set, paths with
329 the preferred path bit set will always be
330 in their own path group.
331
332 datacore
333
334 preferredsds
335 (Mandatory) The preferred "SDS
336 name".
337
338 timeout (Optional) The timeout for the
339 INQUIRY, in ms.
340
341 iet
342
343 preferredip=...
344 (Mandatory) Th preferred IP
345 address, in dotted decimal nota‐
346 tion, for iSCSI targets.
347
348 The default is: <unset>
349
350 features Specify any device-mapper features to be used. Syntax
351 is num list where num is the number, between 0 and 8,
352 of features in list. Possible values for the feature
353 list are:
354
355 queue_if_no_path
356 (Deprecated, superseded by no_path_retry)
357 Queue I/O if no path is active. Identical
358 to the no_path_retry with queue value. If
359 both this feature and no_path_retry are
360 set, the latter value takes precedence.
361 See KNOWN ISSUES.
362
363 pg_init_retries <times>
364 (Since kernel 2.6.24) Number of times to
365 retry pg_init, it must be between 1 and
366 50.
367
368 pg_init_delay_msecs <msecs>
369 (Since kernel 2.6.38) Number of msecs
370 before pg_init retry, it must be between 0
371 and 60000.
372
373 queue_mode <mode>
374 (Since kernel 4.8) Select the the queueing
375 mode per multipath device. <mode> can be
376 bio, rq or mq, which corresponds to bio-
377 based, request-based, and block-multiqueue
378 (blk-mq) request-based, respectively. The
379 default depends on the kernel parameter
380 dm_mod.use_blk_mq. It is mq if the latter
381 is set, and rq otherwise.
382
383 The default is: <unset>
384
385 path_checker The default method used to determine the paths state.
386 Possible values are:
387
388 readsector0 (Deprecated) Read the first sector of the
389 device. This checker is being deprecated,
390 please use tur instead.
391
392 tur Issue a TEST UNIT READY command to the
393 device.
394
395 emc_clariion
396 (Hardware-dependent) Query the DGC/EMC
397 specific EVPD page 0xC0 to determine the
398 path state for CLARiiON CX/AX and EMC VNX
399 and Unity arrays families.
400
401 hp_sw (Hardware-dependent) Check the path state
402 for HP/COMPAQ/DEC HSG80 and MSA/HSV arrays
403 with Active/Standby mode exclusively.
404
405 rdac (Hardware-dependent) Check the path state
406 for LSI/Engenio/NetApp RDAC class as
407 NetApp SANtricity E/EF Series, and OEM
408 arrays from IBM DELL SGI STK and SUN.
409
410 directio (Deprecated) Read the first sector with
411 direct I/O. This checker is being depre‐
412 cated, it could cause spurious path fail‐
413 ures under high load. Please use tur
414 instead.
415
416 cciss_tur (Hardware-dependent) Check the path state
417 for HP/COMPAQ Smart Array(CCISS) con‐
418 trollers.
419
420 none Do not check the device, fallback to use
421 the values retrieved from sysfs
422
423 The default is: tur
424
425 alias_prefix The user_friendly_names prefix.
426
427 The default is: mpath
428
429 failback Tell multipathd how to manage path group failback. To
430 select immediate or a value, it's mandatory that the
431 device has support for a working prioritizer.
432
433 immediate Immediately failback to the highest prior‐
434 ity pathgroup that contains active paths.
435
436 manual Do not perform automatic failback.
437
438 followover Used to deal with multiple computers
439 accessing the same Active/Passive storage
440 devices. Only perform automatic failback
441 when the first path of a pathgroup becomes
442 active. This keeps a cluster node from
443 automatically failing back when another
444 node requested the failover.
445
446 values > 0 Deferred failback (time to defer in sec‐
447 onds).
448
449 The default is: manual
450
451 rr_min_io Number of I/O requests to route to a path before
452 switching to the next in the same path group. This is
453 only for Block I/O(BIO) based multipath and only apply
454 to round-robin path_selector.
455
456 The default is: 1000
457
458 rr_min_io_rq Number of I/O requests to route to a path before
459 switching to the next in the same path group. This is
460 only for Request based multipath and only apply to
461 round-robin path_selector.
462
463 The default is: 1
464
465 max_fds Specify the maximum number of file descriptors that
466 can be opened by multipath and multipathd. This is
467 equivalent to ulimit -n. A value of max will set this
468 to the system limit from /proc/sys/fs/nr_open. If this
469 is not set, the maximum number of open fds is taken
470 from the calling process. It is usually 1024. To be
471 safe, this should be set to the maximum number of
472 paths plus 32, if that number is greated than 1024.
473
474 The default is: max
475
476 rr_weight If set to priorities the multipath configurator will
477 assign path weights as "path prio * rr_min_io". Possi‐
478 ble values are priorities or uniform . Only apply to
479 round-robin path_selector.
480
481 The default is: uniform
482
483 no_path_retry Specify what to do when all paths are down. Possible
484 values are:
485
486 value > 0 Number of retries until disable I/O queue‐
487 ing.
488
489 fail For immediate failure (no I/O queueing).
490
491 queue For never stop I/O queueing, similar to
492 queue_if_no_path. See KNOWN ISSUES.
493
494 The default is: fail
495
496 queue_without_daemon
497 If set to no , when multipathd stops, queueing will be
498 turned off for all devices. This is useful for
499 devices that set no_path_retry. If a machine is shut
500 down while all paths to a device are down, it is pos‐
501 sible to hang waiting for I/O to return from the
502 device after multipathd has been stopped. Without mul‐
503 tipathd running, access to the paths cannot be
504 restored, and the kernel cannot be told to stop queue‐
505 ing I/O. Setting queue_without_daemon to no , avoids
506 this problem.
507
508 The default is: no
509
510 checker_timeout Specify the timeout to use for path checkers and pri‐
511 oritizers that issue SCSI commands with an explicit
512 timeout, in seconds.
513
514 The default is: in /sys/block/sd<x>/device/timeout
515
516 flush_on_last_del
517 If set to yes , multipathd will disable queueing when
518 the last path to a device has been deleted.
519
520 The default is: no
521
522 user_friendly_names
523 If set to yes , using the bindings file /etc/multi‐
524 path/bindings to assign a persistent and unique alias
525 to the multipath, in the form of mpath<n>. If set to
526 no use the WWID as the alias. In either case this be
527 will be overridden by any specific aliases in the mul‐
528 tipaths section.
529
530 The default is: no
531
532 fast_io_fail_tmo Specify the number of seconds the SCSI layer will wait
533 after a problem has been detected on a FC remote port
534 before failing I/O to devices on that remote port.
535 This should be smaller than dev_loss_tmo. Setting this
536 to off will disable the timeout.
537
538 The default is: in 5
539
540 dev_loss_tmo Specify the number of seconds the SCSI layer will wait
541 after a problem has been detected on a FC remote port
542 before removing it from the system. This can be set to
543 "infinity" which sets it to the max value of
544 2147483647 seconds, or 68 years. It will be automati‐
545 cally adjusted to the overall retry interval
546 no_path_retry * polling_interval if a number of
547 retries is given with no_path_retry and the overall
548 retry interval is longer than the specified
549 dev_loss_tmo value. The Linux kernel will cap this
550 value to 600 if fast_io_fail_tmo is not set. See KNOWN
551 ISSUES.
552
553 The default is: 600
554
555 bindings_file The full pathname of the binding file to be used when
556 the user_friendly_names option is set.
557
558 The default is: /etc/multipath/bindings
559
560 wwids_file The full pathname of the WWIDs file, which is used by
561 multipath to keep track of the WWIDs for LUNs it has
562 created multipath devices on in the past.
563
564 The default is: /etc/multipath/wwids
565
566 prkeys_file The full pathname of the prkeys file, which is used by
567 multipathd to keep track of the persistent reservation
568 key used for a specific WWID, when reservation_key is
569 set to file.
570
571 The default is /etc/multipath/prkeys
572
573 log_checker_err If set to once , multipathd logs the first path
574 checker error at logging level 2. Any later errors are
575 logged at level 3 until the device is restored. If set
576 to always , multipathd always logs the path checker
577 error at logging level 2.
578
579 The default is: always
580
581 reservation_key This is the service action reservation key used by
582 mpathpersist. It must be set for all multipath devices
583 using persistent reservations, and it must be the same
584 as the RESERVATION KEY field of the PERSISTENT RESERVE
585 OUT parameter list which contains an 8-byte value pro‐
586 vided by the application client to the device server
587 to identify the I_T nexus. If the --param-aptpl option
588 is used when registering the key with mpathpersist,
589 :aptpl must be appended to the end of the reservation
590 key.
591
592 Alternatively, this can be set to file, which will
593 store the RESERVATION KEY registered by mpathpersist
594 in the prkeys_file. multipathd will then use this key
595 to register additional paths as they appear. When the
596 registration is removed, the RESERVATION KEY is
597 removed from the prkeys_file. The prkeys file will
598 automatically keep track of whether the key was regis‐
599 tered with --param-aptpl.
600
601 The default is: <unset>
602
603 all_tg_pt Set the 'all targets ports' flag when registering keys
604 with mpathpersist. Some arrays automatically set and
605 clear registration keys on all target ports from a
606 host, instead of per target port per host. The
607 ALL_TG_PT flag must be set to successfully use mpath‐
608 persist on these arrays. Setting this option is iden‐
609 tical to calling mpathpersist with --param-alltgpt
610
611 The default is: no
612
613 retain_attached_hw_handler
614 (Obsolete for kernels >= 4.3) If set to yes and the
615 SCSI layer has already attached a hardware_handler to
616 the device, multipath will not force the device to use
617 the hardware_handler specified by mutipath.conf. If
618 the SCSI layer has not attached a hardware handler,
619 multipath will continue to use its configured hardware
620 handler.
621
622 The default is: yes
623
624 Important Note: Linux kernel 4.3 or newer always
625 behaves as if "retain_attached_hw_handler yes" was
626 set.
627
628 detect_prio If set to yes , multipath will try to detect if the
629 device supports SCSI-3 ALUA. If so, the device will
630 automatically use the sysfs prioritizer if the
631 required sysf attributes access_state and pre‐
632 ferred_path are supported, or the alua prioritizer if
633 not. If set to no , the prioritizer will be selected
634 as usual.
635
636 The default is: yes
637
638 detect_checker if set to yes , multipath will try to detect if the
639 device supports SCSI-3 ALUA. If so, the device will
640 automatically use the tur checker. If set to no , the
641 checker will be selected as usual.
642
643 The default is: yes
644
645 force_sync If set to yes , multipathd will call the path checkers
646 in sync mode only. This means that only one checker
647 will run at a time. This is useful in the case where
648 many multipathd checkers running in parallel causes
649 significant CPU pressure.
650
651 The default is: no
652
653 strict_timing If set to yes , multipathd will start a new path
654 checker loop after exactly one second, so that each
655 path check will occur at exactly polling_interval sec‐
656 onds. On busy systems path checks might take longer
657 than one second; here the missing ticks will be
658 accounted for on the next round. A warning will be
659 printed if path checks take longer than polling_inter‐
660 val seconds.
661
662 The default is: no
663
664 deferred_remove If set to yes , multipathd will do a deferred remove
665 instead of a regular remove when the last path device
666 has been deleted. This means that if the multipath
667 device is still in use, it will be freed when the last
668 user closes it. If path is added to the multipath
669 device before the last user closes it, the deferred
670 remove will be canceled.
671
672 The default is: no
673
674 partition_delimiter
675 This parameter controls how multipath chooses the
676 names of partition devices of multipath maps if a mul‐
677 tipath map is renamed (e.g. if a map alias is added or
678 changed). If this parameter is set to a string other
679 than "/UNSET/" (even the empty string), multipath
680 inserts that string between device name and partition
681 number to construct the partition device name. Other‐
682 wise (i.e. if this parameter is unset or has the value
683 "/UNSET/"), the behavior depends on the map name: if
684 it ends in a digit, a "p" is inserted between name and
685 partition number; otherwise, the partition number is
686 simply appended. Distributions may use a non-null
687 default value for this option; in this case, the user
688 must set it to "/UNSET/" to obtain the original
689 <unset> behavior. Use multipath -T to check the cur‐
690 rent settings.
691
692 The default is: <unset>
693
694 config_dir If set to anything other than "", multipath will
695 search this directory alphabetically for file ending
696 in ".conf" and it will read configuration information
697 from them, just as if it was in /etc/multipath.conf.
698 config_dir must either be "" or a fully qualified
699 directory name.
700
701 The default is: /etc/multipath/conf.d/
702
703 san_path_err_threshold
704 If set to a value greater than 0, multipathd will
705 watch paths and check how many times a path has been
706 failed due to errors.If the number of failures on a
707 particular path is greater then the
708 san_path_err_threshold, then the path will not rein‐
709 state till san_path_err_recovery_time. These path
710 failures should occur within a san_path_err_for‐
711 get_rate checks, if not we will consider the path is
712 good enough to reinstantate. See "Shaky paths detec‐
713 tion" below.
714
715 The default is: no
716
717 san_path_err_forget_rate
718 If set to a value greater than 0, multipathd will
719 check whether the path failures has exceeded the
720 san_path_err_threshold within this many checks i.e
721 san_path_err_forget_rate . If so we will not rein‐
722 stante the path till san_path_err_recovery_time. See
723 "Shaky paths detection" below.
724
725 The default is: no
726
727 san_path_err_recovery_time
728 If set to a value greater than 0, multipathd will make
729 sure that when path failures has exceeded the
730 san_path_err_threshold within san_path_err_forget_rate
731 then the path will be placed in failed state for
732 san_path_err_recovery_time duration.Once
733 san_path_err_recovery_time has timeout we will rein‐
734 stante the failed path . san_path_err_recovery_time
735 value should be in secs. See "Shaky paths detection"
736 below.
737
738 The default is: no
739
740 marginal_path_double_failed_time
741 One of the four parameters of supporting path check
742 based on accounting IO error such as intermittent
743 error. When a path failed event occurs twice in mar‐
744 ginal_path_double_failed_time seconds due to an IO
745 error and all the other three parameters are set, mul‐
746 tipathd will fail the path and enqueue this path into
747 a queue of which members are sent a couple of continu‐
748 ous direct reading asynchronous IOs at a fixed sample
749 rate of 10HZ to start IO error accounting process. See
750 "Shaky paths detection" below.
751
752 The default is: no
753
754 marginal_path_err_sample_time
755 One of the four parameters of supporting path check
756 based on accounting IO error such as intermittent
757 error. If it is set to a value no less than 120, when
758 a path fail event occurs twice in marginal_path_dou‐
759 ble_failed_time second due to an IO error, multipathd
760 will fail the path and enqueue this path into a queue
761 of which members are sent a couple of continuous
762 direct reading asynchronous IOs at a fixed sample rate
763 of 10HZ to start the IO accounting process for the
764 path will last for marginal_path_err_sample_time. If
765 the rate of IO error on a particular path is greater
766 than the marginal_path_err_rate_threshold, then the
767 path will not reinstate for margin‐
768 al_path_err_recheck_gap_time seconds unless there is
769 only one active path. After margin‐
770 al_path_err_recheck_gap_time expires, the path will be
771 requeueed for rechecking. If checking result is good
772 enough, the path will be reinstated. See "Shaky paths
773 detection" below.
774
775 The default is: no
776
777 marginal_path_err_rate_threshold
778 The error rate threshold as a permillage (1/1000). One
779 of the four parameters of supporting path check based
780 on accounting IO error such as intermittent error.
781 Refer to marginal_path_err_sample_time. If the rate of
782 IO errors on a particular path is greater than this
783 parameter, then the path will not reinstate for mar‐
784 ginal_path_err_recheck_gap_time seconds unless there
785 is only one active path. See "Shaky paths detection"
786 below.
787
788 The default is: no
789
790 marginal_path_err_recheck_gap_time
791 One of the four parameters of supporting path check
792 based on accounting IO error such as intermittent
793 error. Refer to marginal_path_err_sample_time. If this
794 parameter is set to a positive value, the failed path
795 of which the IO error rate is larger than margin‐
796 al_path_err_rate_threshold will be kept in failed
797 state for marginal_path_err_recheck_gap_time seconds.
798 When marginal_path_err_recheck_gap_time seconds
799 expires, the path will be requeueed for checking. If
800 checking result is good enough, the path will be rein‐
801 stated, or else it will keep failed. See "Shaky paths
802 detection" below.
803
804 The default is: no
805
806 delay_watch_checks
807 If set to a value greater than 0, multipathd will
808 watch paths that have recently become valid for this
809 many checks. If they fail again while they are being
810 watched, when they next become valid, they will not be
811 used until they have stayed up for delay_wait_checks
812 checks. See "Shaky paths detection" below.
813
814 The default is: no
815
816 delay_wait_checks
817 If set to a value greater than 0, when a device that
818 has recently come back online fails again within
819 delay_watch_checks checks, the next time it comes back
820 online, it will marked and delayed, and not used until
821 it has passed delay_wait_checks checks. See "Shaky
822 paths detection" below.
823
824 The default is: no
825
826 find_multipaths This option controls whether multipath and multipathd
827 try to create multipath maps over non-blacklisted
828 devices they encounter. This matters a) when a device
829 is encountered by multipath -u during udev rule pro‐
830 cessing (a device is blocked from further processing
831 by higher layers - such as LVM - if and only if it´s
832 considered a valid multipath device path), and b) when
833 multipathd detects a new device. The following values
834 are possible:
835
836 strict Both multipath and multipathd treat only
837 such devices as multipath devices which have
838 been part of a multipath map previously, and
839 which are therefore listed in the
840 wwids_file. Users can manually set up multi‐
841 path maps using the multipathd add map com‐
842 mand. Once set up manually, the map is
843 remembered in the wwids file and will be set
844 up automatically in the future.
845
846 no Multipath behaves like strict. Multipathd
847 behaves like greedy.
848
849 yes Both multipathd and multipath treat a device
850 as multipath device if the conditions for
851 strict are met, or if at least two non-
852 blacklisted paths with the same WWID have
853 been detected.
854
855 greedy Both multipathd and multipath treat every
856 non-blacklisted device as multipath device
857 path.
858
859 smart This differs from find_multipaths yes only
860 in the way it treats new devices for which
861 only one path has been detected yet. When
862 such a device is first encounted in udev
863 rules, it is treated as a multipath device.
864 multipathd waits whether additional paths
865 with the same WWID appears. If that happens,
866 it sets up a multipath map. If it doesn´t
867 happen until a timeout expires, or if set‐
868 ting up the map fails, a new uevent is trig‐
869 gered for the device; at second encounter in
870 the udev rules, the device will be treated
871 as non-multipath and passed on to upper lay‐
872 ers. Note: this may cause delays during
873 device detection if there are single-path
874 devices which aren´t blacklisted.
875
876 The default is: strict
877
878 find_multipaths_timeout
879 Timeout, in seconds, to wait for additional paths
880 after detecting the first one, if find_multipaths
881 "smart" (see above) is set. If the value is positive,
882 this timeout is used for all unknown, non-blacklisted
883 devices encountered. If the value is negative (recom‐
884 mended), it's only applied to "known" devices that
885 have an entry in multipath's hardware table, either in
886 the built-in table or in a device section; other
887 ("unknown") devices will use a timeout of only 1 sec‐
888 ond to avoid booting delays. The value 0 means "use
889 the built-in default". If find_multipath has a value
890 other than smart, this option has no effect.
891
892 The default is: -10 (10s for known and 1s for unknown
893 hardware)
894
895 uxsock_timeout CLI receive timeout in milliseconds. For larger sys‐
896 tems CLI commands might timeout before the multipathd
897 lock is released and the CLI command can be processed.
898 This will result in errors like "timeout receiving
899 packet" to be returned from CLI commands. In these
900 cases it is recommended to increase the CLI timeout to
901 avoid those issues.
902
903 The default is: 1000
904
905 retrigger_tries Sets the number of times multipathd will try to
906 retrigger a uevent to get the WWID.
907
908 The default is: 3
909
910 retrigger_delay Sets the amount of time, in seconds, to wait between
911 retriggers.
912
913 The default is: 10
914
915 missing_uev_wait_timeout
916 Controls how many seconds multipathd will wait, after
917 a new multipath device is created, to receive a change
918 event from udev for the device, before automatically
919 enabling device reloads. Usually multipathd will delay
920 reloads on a device until it receives a change uevent
921 from the initial table load.
922
923 The default is: 30
924
925 skip_kpartx If set to yes , kpartx will not automatically create
926 partitions on the device.
927
928 The default is: no
929
930 disable_changed_wwids
931 If set to yes, multipathd will check the path wwid on
932 change events, and if it has changed from the wwid of
933 the multipath device, multipathd will disable access
934 to the path until the wwid changes back.
935
936 The default is: no
937
938 remove_retries This sets how may times multipath will retry removing
939 a device that is in-use. Between each attempt, multi‐
940 path will sleep 1 second.
941
942 The default is: 0
943
944 max_sectors_kb Sets the max_sectors_kb device parameter on all path
945 devices and the multipath device to the specified
946 value.
947
948 The default is: <device dependent>
949
950 ghost_delay Sets the number of seconds that multipath will wait
951 after creating a device with only ghost paths before
952 marking it ready for use in systemd. This gives the
953 active paths time to appear before the multipath runs
954 the hardware handler to switch the ghost paths to
955 active ones. Setting this to 0 or on makes multipath
956 immediately mark a device with only ghost paths as
957 ready.
958
959 The default is no
960
962 The blacklist section is used to exclude specific devices from the mul‐
963 tipath topology. It is most commonly used to exclude local disks or
964 non-disk devices (such as LUNs for the storage array controller) from
965 being handled by multipath-tools.
966
967 The blacklist_exceptions section is used to revert the actions of the
968 blacklist section. This allows one to selectively include ("whitelist")
969 devices which would normally be excluded via the blacklist section. A
970 common usage is to blacklist "everything" using a catch-all regular
971 expression, and create specific blacklist_exceptions entries for those
972 devices that should be handled by multipath-tools.
973
974 The following keywords are recognized in both sections. The defaults
975 are empty unless explicitly stated.
976
977 devnode Regular expression matching the device nodes to be
978 excluded/included.
979
980 The default blacklist consists of the regular expres‐
981 sions "^(ram|zram|raw|loop|fd|md|dm-|sr|scd|st|dcss‐
982 blk)[0-9]" and "^(td|hd|vd)[a-z]". This causes virtual
983 devices, non-disk devices, and some other device types
984 to be excluded from multipath handling by default.
985
986 wwid Regular expression for the World Wide Identifier of a
987 device to be excluded/included.
988
989 device Subsection for the device description. This subsection
990 recognizes the vendor and product keywords. Both are
991 regular expressions. For a full description of these
992 keywords please see the devices section description.
993
994 property Regular expression for an udev property. All devices
995 that have matching udev properties will be
996 excluded/included. The handling of the property key‐
997 word is special, because if a property black‐
998 list_exception is set, devices must have at least one
999 whitelisted udev property; otherwise they're treated
1000 as blacklisted, and the message "blacklisted, udev
1001 property missing" is displayed in the logs. For exam‐
1002 ple, setting the property blacklist_exception to
1003 (SCSI_IDENT_|ID_WWN), will cause well-behaved SCSI
1004 devices and devices that provide a WWN (World Wide
1005 Number) to be included, and all others to be excluded.
1006 This works to exclude most non-multipathable devices.
1007
1008 protocol Regular expression for the protocol of a device to be
1009 excluded/included.
1010
1011 The protocol strings that multipath recognizes are
1012 scsi:fcp, scsi:spi, scsi:ssa, scsi:sbp, scsi:srp,
1013 scsi:iscsi, scsi:sas, scsi:adt, scsi:ata, scsi:unspec,
1014 ccw, cciss, nvme, and undef. The protocol that a path
1015 is using can be viewed by running multipathd show
1016 paths format "%d %P"
1017
1018 For every device, these 5 blacklist criteria are evaluated in the the
1019 order "property, devnode, device, protocol, wwid". If a device turns
1020 out to be blacklisted by any criterion, it's excluded from handling by
1021 multipathd, and the later criteria aren't evaluated any more. For each
1022 criterion, the whitelist takes precedence over the blacklist if a
1023 device matches both.
1024
1025 Note: Besides the blacklist and whitelist, other configuration options
1026 such as find_multipaths have an impact on whether or not a given device
1027 is handled by multipath-tools.
1028
1030 The multipaths section allows setting attributes of multipath maps. The
1031 attributes that are set via the multipaths section (see list below)
1032 take precedence over all other configuration settings, including those
1033 from the overrides section.
1034
1035 The only recognized attribute for the multipaths section is the multi‐
1036 path subsection. If there are multiple multipath subsections matching a
1037 given WWID, the contents of these sections are merged, and settings
1038 from later entries take precedence.
1039
1040 The multipath subsection recognizes the following attributes:
1041
1042 wwid (Mandatory) World Wide Identifier. Detected multipath
1043 maps are matched agains this attribute. Note that,
1044 unlike the wwid attribute in the blacklist section,
1045 this is not a regular expression or a substring; WWIDs
1046 must match exactly inside the multipaths section.
1047
1048 alias Symbolic name for the multipath map. This takes prece‐
1049 dence over a an entry for the same WWID in the bind‐
1050 ings_file.
1051
1052 The following attributes are optional; if not set the default values
1053 are taken from the overrides, devices, or defaults section:
1054
1055 path_grouping_policy
1056 path_selector
1057 prio
1058 prio_args
1059 failback
1060 rr_weight
1061 no_path_retry
1062 rr_min_io
1063 rr_min_io_rq
1064 flush_on_last_del
1065 features
1066 reservation_key
1067 user_friendly_names
1068 deferred_remove
1069 san_path_err_threshold
1070 san_path_err_forget_rate
1071 san_path_err_recovery_time
1072 marginal_path_err_sample_time
1073 marginal_path_err_rate_threshold
1074 marginal_path_err_recheck_gap_time
1075 marginal_path_double_failed_time
1076 delay_watch_checks
1077 delay_wait_checks
1078 skip_kpartx
1079 max_sectors_kb
1080 ghost_delay
1081
1083 multipath-tools have a built-in device table with reasonable defaults
1084 for more than 100 known multipath-capable storage devices. The devices
1085 section can be used to override these settings. If there are multiple
1086 matches for a given device, the attributes of all matching entries are
1087 applied to it. If an attribute is specified in several matching device
1088 subsections, later entries take precedence. Thus, entries in files
1089 under config_dir (in reverse alphabetical order) have the highest
1090 precedence, followed by entries in multipath.conf; the built-in hard‐
1091 ware table has the lowest precedence. Inside a configuration file,
1092 later entries have higher precedence than earlier ones.
1093
1094 The only recognized attribute for the devices section is the device
1095 subsection. Devices detected in the system are matched against the
1096 device entries using the vendor, product, and revision fields, which
1097 are all POSIX Extended regular expressions (see regex(7)).
1098
1099 The vendor, product, and revision fields that multipath or multipathd
1100 detect for devices in a system depend on the device type. For SCSI
1101 devices, they correspond to the respective fields of the SCSI INQUIRY
1102 page. In general, the command 'multipathd show paths format "%d %s"'
1103 command can be used to see the detected properties for all devices in
1104 the system.
1105
1106 The device subsection recognizes the following attributes:
1107
1108 vendor (Mandatory) Regular expression to match the vendor
1109 name.
1110
1111 product (Mandatory) Regular expression to match the product
1112 name.
1113
1114 revision Regular expression to match the product revision. If
1115 not specified, any revision matches.
1116
1117 product_blacklist
1118 Products with the given vendor matching this string
1119 are blacklisted. This is equivalent to a device entry
1120 in the blacklist section with the vendor attribute set
1121 to this entry's vendor, and the product attribute set
1122 to the value of product_blacklist.
1123
1124 alias_prefix The user_friendly_names prefix to use for this device
1125 type, instead of the default "mpath".
1126
1127 hardware_handler The hardware handler to use for this device type. The
1128 following hardware handler are implemented:
1129
1130 1 emc (Hardware-dependent) Hardware handler for
1131 DGC class arrays as CLARiiON CX/AX and EMC
1132 VNX and Unity families.
1133
1134 1 rdac (Hardware-dependent) Hardware handler for
1135 LSI/Engenio/NetApp RDAC class as NetApp
1136 SANtricity E/EF Series, and OEM arrays
1137 from IBM DELL SGI STK and SUN.
1138
1139 1 hp_sw (Hardware-dependent) Hardware handler for
1140 HP/COMPAQ/DEC HSG80 and MSA/HSV arrays
1141 with Active/Standby mode exclusively.
1142
1143 1 alua (Hardware-dependent) Hardware handler for
1144 SCSI-3 ALUA compatible arrays.
1145
1146 1 ana (Hardware-dependent) Hardware handler for
1147 NVMe ANA compatible arrays.
1148
1149 The default is: <unset>
1150
1151 Important Note: Linux kernels 4.3 and newer automati‐
1152 cally attach a device handler to known devices (which
1153 includes all devices supporting SCSI-3 ALUA) and dis‐
1154 allow changing the handler afterwards. Setting hard‐
1155 ware_handler for such devices on these kernels has no
1156 effect.
1157
1158 The following attributes are optional; if not set the default values
1159 are taken from the defaults section:
1160
1161 path_grouping_policy
1162 uid_attribute
1163 path_selector
1164 path_checker
1165 prio
1166 prio_args
1167 features
1168 failback
1169 rr_weight
1170 no_path_retry
1171 rr_min_io
1172 rr_min_io_rq
1173 fast_io_fail_tmo
1174 dev_loss_tmo
1175 flush_on_last_del
1176 retain_attached_hw_handler
1177 detect_prio
1178 detect_checker
1179 deferred_remove
1180 san_path_err_threshold
1181 san_path_err_forget_rate
1182 san_path_err_recovery_time
1183 marginal_path_err_sample_time
1184 marginal_path_err_rate_threshold
1185 marginal_path_err_recheck_gap_time
1186 marginal_path_double_failed_time
1187 delay_watch_checks
1188 delay_wait_checks
1189 skip_kpartx
1190 max_sectors_kb
1191 ghost_delay
1192
1194 The overrides section recognizes the following optional attributes; if
1195 not set the values are taken from the devices or defaults sections:
1196
1197 path_grouping_policy
1198 uid_attribute
1199 getuid_callout
1200 path_selector
1201 path_checker
1202 alias_prefix
1203 features
1204 prio
1205 prio_args
1206 failback
1207 rr_weight
1208 no_path_retry
1209 rr_min_io
1210 rr_min_io_rq
1211 flush_on_last_del
1212 fast_io_fail_tmo
1213 dev_loss_tmo
1214 user_friendly_names
1215 retain_attached_hw_handler
1216 detect_prio
1217 detect_checker
1218 deferred_remove
1219 san_path_err_threshold
1220 san_path_err_forget_rate
1221 san_path_err_recovery_time
1222 marginal_path_err_sample_time
1223 marginal_path_err_rate_threshold
1224 marginal_path_err_recheck_gap_time
1225 marginal_path_double_failed_time
1226 delay_watch_checks
1227 delay_wait_checks
1228 skip_kpartx
1229 ghost_delay
1230
1232 Multipath uses a World Wide Identification (WWID) to determine which
1233 paths belong to the same device. Each path presenting the same WWID is
1234 assumed to point to the same device.
1235
1236 The WWID is generated by four methods (in the order of preference):
1237
1238 uid_attrs The WWID is derived from udev attributes by matching
1239 the device node name. See description of uid_attrs in
1240 the defaults section above.
1241
1242 getuid_callout Use the specified external program; cf getuid_callout
1243 above. Care should be taken when using this method;
1244 the external program needs to be loaded from disk for
1245 execution, which might lead to deadlock situations in
1246 an all-paths-down scenario.
1247
1248 uid_attribute Use the value of the specified udev attribute; cf
1249 uid_attribute above. This method is preferred to
1250 getuid_callout as multipath does not need to call any
1251 external programs here. However, under certain circum‐
1252 stances udev might not be able to generate the
1253 requested variable.
1254
1255 vpd_pg83 If none of the getuid_callout or uid_attribute parame‐
1256 ters are present multipath will try to use the sysfs
1257 attribute vpd_pg83 to generate the WWID.
1258
1260 A common problem in SAN setups is the occurence of intermittent errors:
1261 a path is unreachable, then reachable again for a short time, disap‐
1262 pears again, and so forth. This happens typically on unstable intercon‐
1263 nects. It is undesirable to switch pathgroups unnecessarily on such
1264 frequent, unreliable events. multipathd supports three different meth‐
1265 ods for detecting this situation and dealing with it. All methods share
1266 the same basic mode of operation: If a path is found to be "shaky" or
1267 "flipping", and appears to be in healthy status, it is not reinstated
1268 (put back to use) immediately. Instead, it is watched for some time,
1269 and only reinstated if the healthy state appears to be stable. The
1270 logic of determining "shaky" condition, as well as the logic when to
1271 reinstate, differs between the three methods.
1272
1273 "delay_checks" failure tracking
1274 If a path fails again within a delay_watch_checks interval
1275 after a failure, don't reinstate it until it passes a
1276 delay_wait_checks interval in always good status. The inter‐
1277 vals are measured in "ticks", i.e. the time between path checks
1278 by multipathd, which is variable and controlled by the
1279 polling_interval and max_polling_interval parameters.
1280
1281 "marginal_path" failure tracking
1282 If a second failure event (good->bad transition) occurs within
1283 marginal_path_double_failed_time seconds after a failure, high-
1284 frequency monitoring is started for the affected path: I/O is
1285 sent at a rate of 10 per second. This is done for margin‐
1286 al_path_err_sample_time seconds. During this period, the path
1287 is not reinstated. If the rate of errors remains below margin‐
1288 al_path_err_rate_threshold during the monitoring period, the
1289 path is reinstated. Otherwise, it is kept in failed state for
1290 marginal_path_err_recheck_gap_time, and after that, it is moni‐
1291 tored again. For this method, time intervals are measured in
1292 seconds.
1293
1294 "san_path_err" failure tracking
1295 multipathd counts path failures for each path. Once the number
1296 of failures exceeds the value given by san_path_err_threshold,
1297 the path is not reinstated for san_path_err_recovery_time
1298 ticks. While counting failures, multipathd "forgets" one past
1299 failure every "san_path_err_forget_rate" ticks; thus if errors
1300 don't occur more often then once in the forget rate interval,
1301 the failure count doesn't increase and the threshold is never
1302 reached. As for the delay_xy method, intervals are measured in
1303 "ticks".
1304
1305 This method is deprecated in favor of the "marginal_path" fail‐
1306 ure tracking method, and only offered for backward compatibil‐
1307 ity.
1308
1309 See the documentation of the individual options above for details. It
1310 is strongly discouraged to use more than one of these methods for any
1311 given multipath map, because the two concurrent methods may interact in
1312 unpredictable ways. If the "marginal_path" method is active, the
1313 "san_path_err" parameters are implicitly set to 0.
1314
1316 The usage of queue_if_no_path option can lead to D state processes
1317 being hung and not killable in situations where all the paths to the
1318 LUN go offline. It is advisable to use the no_path_retry option
1319 instead.
1320
1321 The use of queue_if_no_path or no_path_retry might lead to a deadlock
1322 if the dev_loss_tmo setting results in a device being removed while I/O
1323 is still queued. The multipath daemon will update the dev_loss_tmo set‐
1324 ting accordingly to avoid this deadlock. Hence if both values are spec‐
1325 ified the order of precedence is no_path_retry, queue_if_no_path,
1326 dev_loss_tmo.
1327
1329 udev(8), dmsetup(8), multipath(8), multipathd(8).
1330
1332 multipath-tools was developed by Christophe Varoqui, <christophe.varo‐
1333 qui@opensvc.com> and others.
1334
1335
1336
1337Linux 2018-05-21 MULTIPATH.CONF(5)