1DRBD.CONF(5) Configuration Files DRBD.CONF(5)
2
6 drbd.conf - Configuration file for DRBD's devices
7
9 The file /etc/drbd.conf is read by drbdadm.
10 The file format was designed as to allow to have a verbatim copy of the
12 file on both nodes of the cluster. It is highly recommended to do so in
13 order to keep your configuration manageable. The file /etc/drbd.conf
14 should be the same on both nodes of the cluster. Changes to
15 /etc/drbd.conf do not apply immediately.
16 In this example, there is a single DRBD resource (called r0) which uses
18 protocol C for the connection between its devices. The device which
19 runs on host alice uses /dev/drbd1 as devices for its application, and
20 /dev/sda7 as low-level storage for the data. The IP addresses are used
21 to specify the networking interfaces to be used. An eventually running
22 resync process should use about 10MByte/second of IO bandwidth.
23 There may be multiple resource sections in a single drbd.conf file. For
25 more examples, please have a look at the DRBD User's Guide[1].
26
28 The file consists of sections and parameters. A section begins with a
29 keyword, sometimes an additional name, and an opening brace (“{”). A
30 section ends with a closing brace (“}”. The braces enclose the
31 parameters.
32 section [name] { parameter value; [...] }
34 A parameter starts with the identifier of the parameter followed by
36 whitespace. Every subsequent character is considered as part of the
37 parameter's value. A special case are Boolean parameters which consist
38 only of the identifier. Parameters are terminated by a semicolon (“;”).
39 Some parameter values have default units which might be overruled by K,
41 M or G. These units are defined in the usual way (K = 2^10 = 1024, M =
42 1024 K, G = 1024 M).
43 Comments may be placed into the configuration file and must begin with
45 a hash sign (“#”). Subsequent characters are ignored until the end of
46 the line.
47 Sections
49 skip
50 Comments out chunks of text, even spanning more than one line.
52 Characters between the keyword skip and the opening brace (“{”) are
53 ignored. Everything enclosed by the braces is skipped. This comes
54 in handy, if you just want to comment out some 'resource [name]
55 {...}' section: just precede it with '“skip”'.
56 global
58 Configures some global parameters. Currently only minor-count,
60 dialog-refresh, disable-ip-verification and usage-count are allowed
61 here. You may only have one global section, preferably as the first
62 section.
63 common
65 All resources inherit the options set in this section. The common
67 section might have a startup, a syncer, a handlers, a net and a
68 disk section.
69 resource name
71 Configures a DRBD resource. Each resource section needs to have two
73 (or more) on host sections and may have a startup, a syncer, a
74 handlers, a net and a disk section. Required parameter in this
75 section: protocol.
76 on host-name
78 Carries the necessary configuration parameters for a DRBD device of
80 the enclosing resource. host-name is mandatory and must match the
81 Linux host name (uname -n) of one of the nodes. You may list more
82 than one host name here, in case you want to use the same
83 parameters on several hosts (you'd have to move the IP around
84 usually). Or you may list more than two such sections.
85 resource r1 {
87 protocol C;
88 device minor 1;
89 meta-disk internal;
90 on alice bob {
92 address 10.2.2.100:7801;
93 disk /dev/mapper/some-san;
94 }
95 on charlie {
96 address 10.2.2.101:7801;
97 disk /dev/mapper/other-san;
98 }
99 on daisy {
100 address 10.2.2.103:7801;
101 disk /dev/mapper/other-san-as-seen-from-daisy;
102 }
103 }
104 See also the floating section keyword. Required parameters in this
107 section: device, disk, address, meta-disk, flexible-meta-disk.
108 stacked-on-top-of resource
110 For a stacked DRBD setup (3 or 4 nodes), a stacked-on-top-of is
112 used instead of an on section. Required parameters in this section:
113 device and address.
114 floating AF addr:port
116 Carries the necessary configuration parameters for a DRBD device of
118 the enclosing resource. This section is very similar to the on
119 section. The difference to the on section is that the matching of
120 the host sections to machines is done by the IP-address instead of
121 the node name. Required parameters in this section: device, disk,
122 meta-disk, flexible-meta-disk, all of which may be inherited from
123 the resource section, in which case you may shorten this section
124 down to just the address identifier.
125 resource r2 {
127 protocol C;
128 device minor 2;
129 disk /dev/sda7;
130 meta-disk internal;
131 # short form, device, disk and meta-disk inherited
133 floating 10.1.1.31:7802;
134 # longer form, only device inherited
136 floating 10.1.1.32:7802 {
137 disk /dev/sdb;
138 meta-disk /dev/sdc8;
139 }
140 }
141 disk
145 This section is used to fine tune DRBD's properties in respect to
147 the low level storage. Please refer to drbdsetup(8) for detailed
148 description of the parameters. Optional parameters: on-io-error,
149 size, fencing, use-bmbv, no-disk-barrier, no-disk-flushes,
150 no-disk-drain, no-md-flushes, max-bio-bvecs, disk-timeout.
151 net
153 This section is used to fine tune DRBD's properties. Please refer
155 to drbdsetup(8) for a detailed description of this section's
156 parameters. Optional parameters: sndbuf-size, rcvbuf-size, timeout,
157 connect-int, ping-int, ping-timeout, max-buffers, max-epoch-size,
158 ko-count, allow-two-primaries, cram-hmac-alg, shared-secret,
159 after-sb-0pri, after-sb-1pri, after-sb-2pri, data-integrity-alg,
160 no-tcp-cork, on-congestion, congestion-fill, congestion-extents
161 startup
163 This section is used to fine tune DRBD's properties. Please refer
165 to drbdsetup(8) for a detailed description of this section's
166 parameters. Optional parameters: wfc-timeout, degr-wfc-timeout,
167 outdated-wfc-timeout, wait-after-sb, stacked-timeouts and
168 become-primary-on.
169 syncer
171 This section is used to fine tune the synchronization daemon for
173 the device. Please refer to drbdsetup(8) for a detailed description
174 of this section's parameters. Optional parameters: rate, after,
175 al-extents, use-rle, cpu-mask, verify-alg, csums-alg, c-plan-ahead,
176 c-fill-target, c-delay-target, c-max-rate, c-min-rate and
177 on-no-data-accessible.
178 handlers
180 In this section you can define handlers (executables) that are
182 started by the DRBD system in response to certain events. Optional
183 parameters: pri-on-incon-degr, pri-lost-after-sb, pri-lost,
184 fence-peer (formerly oudate-peer), local-io-error,
185 initial-split-brain, split-brain, before-resync-target,
186 after-resync-target.
187 The interface is done via environment variables:
189 DRBD_RESOURCE
191 is the name of the resource
192 DRBD_MINOR
194 is the minor number of the DRBD device, in decimal.
195 DRBD_CONF
197 is the path to the primary configuration file; if you split
198 your configuration into multiple files (e.g. in
199 /etc/drbd.conf.d/), this will not be helpful.
200 DRBD_PEER_AF, DRBD_PEER_ADDRESS, DRBD_PEERS
202 are the address family (e.g. ipv6), the peer's address and
203 hostnames.
204 DRBD_PEER (note the singular form) is deprecated, and superseeded
207 by DRBD_PEERS.
208 Please note that not all of these might be set for all handlers,
210 and that some values might not be useable for a floating
211 definition.
212 Parameters
214 minor-count count
215 count may be a number from 1 to 255.
216 Use minor-count if you want to define massively more resources
218 later without reloading the DRBD kernel module. Per default the
219 module loads with 11 more resources than you have currently in your
220 config but at least 32.
221 dialog-refresh time
223 time may be 0 or a positive number.
224 The user dialog redraws the second count every time seconds (or
226 does no redraws if time is 0). The default value is 1.
227 disable-ip-verification
229 Use disable-ip-verification if, for some obscure reasons, drbdadm
230 can/might not use ip or ifconfig to do a sanity check for the IP
231 address. You can disable the IP verification with this option.
232 usage-count val
234 Please participate in DRBD's online usage counter[2]. The most
235 convenient way to do so is to set this option to yes. Valid options
236 are: yes, no and ask.
237 protocol prot-id
239 On the TCP/IP link the specified protocol is used. Valid protocol
240 specifiers are A, B, and C.
241 Protocol A: write IO is reported as completed, if it has reached
243 local disk and local TCP send buffer.
244 Protocol B: write IO is reported as completed, if it has reached
246 local disk and remote buffer cache.
247 Protocol C: write IO is reported as completed, if it has reached
249 both local and remote disk.
250 device name minor nr
252 The name of the block device node of the resource being described.
254 You must use this device with your application (file system) and
255 you must not use the low level block device which is specified with
256 the disk parameter.
257 One can ether omit the name or minor and the minor number. If you
259 omit the name a default of /dev/drbdminor will be used.
260 Udev will create additional symlinks in /dev/drbd/by-res and
262 /dev/drbd/by-disk.
263 disk name
265 DRBD uses this block device to actually store and retrieve the
267 data. Never access such a device while DRBD is running on top of
268 it. This also holds true for dumpe2fs(8) and similar commands.
269 address AF addr:port
271 A resource needs one IP address per device, which is used to wait
273 for incoming connections from the partner device respectively to
274 reach the partner device. AF must be one of ipv4, ipv6, ssocks or
275 sdp (for compatibility reasons sci is an alias for ssocks). It may
276 be omited for IPv4 addresses. The actual IPv6 address that follows
277 the ipv6 keyword must be placed inside brackets: ipv6
278 [fd01:2345:6789:abcd::1]:7800.
279 Each DRBD resource needs a TCP port which is used to connect to the
281 node's partner device. Two different DRBD resources may not use the
282 same addr:port combination on the same node.
283 meta-disk internal, flexible-meta-disk internal, meta-disk device
285 [index], flexible-meta-disk device
286 Internal means that the last part of the backing device is used to
288 store the meta-data. You must not use [index] with internal. Note:
289 Regardless of whether you use the meta-disk or the
290 flexible-meta-disk keyword, it will always be of the size needed
291 for the remaining storage size.
292 You can use a single block device to store meta-data of multiple
294 DRBD devices. E.g. use meta-disk /dev/sde6[0]; and meta-disk
295 /dev/sde6[1]; for two different resources. In this case the
296 meta-disk would need to be at least 256 MB in size.
297 With the flexible-meta-disk keyword you specify a block device as
299 meta-data storage. You usually use this with LVM, which allows you
300 to have many variable sized block devices. The required size of the
301 meta-disk block device is 36kB + Backing-Storage-size / 32k. Round
302 this number to the next 4kb boundary up and you have the exact
303 size. Rule of the thumb: 32kByte per 1GByte of storage, round up to
304 the next MB.
305 on-io-error handler
307 handler is taken, if the lower level device reports io-errors to
308 the upper layers.
309 handler may be pass_on, call-local-io-error or detach.
311 pass_on: The node downgrades the disk status to inconsistent, marks
313 the erroneous block as inconsistent in the bitmap and retries the
314 IO on the remote node.
315 call-local-io-error: Call the handler script local-io-error.
317 detach: The node drops its low level device, and continues in
319 diskless mode.
320 fencing fencing_policy
322 By fencing we understand preventive measures to avoid situations
324 where both nodes are primary and disconnected (AKA split brain).
325 Valid fencing policies are:
327 dont-care
329 This is the default policy. No fencing actions are taken.
330 resource-only
332 If a node becomes a disconnected primary, it tries to fence the
333 peer's disk. This is done by calling the fence-peer handler.
334 The handler is supposed to reach the other node over
335 alternative communication paths and call 'drbdadm outdate res'
336 there.
337 resource-and-stonith
339 If a node becomes a disconnected primary, it freezes all its IO
340 operations and calls its fence-peer handler. The fence-peer
341 handler is supposed to reach the peer over alternative
342 communication paths and call 'drbdadm outdate res' there. In
343 case it cannot reach the peer it should stonith the peer. IO is
344 resumed as soon as the situation is resolved. In case your
345 handler fails, you can resume IO with the resume-io command.
346 use-bmbv
348 In case the backing storage's driver has a merge_bvec_fn()
349 function, DRBD has to pretend that it can only process IO requests
350 in units not larger than 4KiB. (At the time of writing the only
351 known drivers which have such a function are: md (software raid
352 driver), dm (device mapper - LVM) and DRBD itself).
353 To get the best performance out of DRBD on top of software RAID (or
355 any other driver with a merge_bvec_fn() function) you might enable
356 this function, if you know for sure that the merge_bvec_fn()
357 function will deliver the same results on all nodes of your
358 cluster. I.e. the physical disks of the software RAID are of
359 exactly the same type. Use this option only if you know what you
360 are doing.
361 no-disk-barrier, no-disk-flushes, no-disk-drain
363 DRBD has four implementations to express write-after-write
364 dependencies to its backing storage device. DRBD will use the first
365 method that is supported by the backing storage device and that is
366 not disabled by the user.
367 When selecting the method you should not only base your decision on
369 the measurable performance. In case your backing storage device has
370 a volatile write cache (plain disks, RAID of plain disks) you
371 should use one of the first two. In case your backing storage
372 device has battery-backed write cache you may go with option 3.
373 Option 4 (disable everything, use "none") is dangerous on most IO
374 stacks, may result in write-reordering, and if so, can
375 theoretically be the reason for data corruption, or disturb the
376 DRBD protocol, causing spurious disconnect/reconnect cycles. Do
377 not use no-disk-drain.
378 Unfortunately device mapper (LVM) might not support barriers.
380 The letter after "wo:" in /proc/drbd indicates with method is
382 currently in use for a device: b, f, d, n. The implementations are:
383 barrier
385 The first requires that the driver of the backing storage
386 device support barriers (called 'tagged command queuing' in
387 SCSI and 'native command queuing' in SATA speak). The use of
388 this method can be disabled by the no-disk-barrier option.
389 Note: Since Linux-2.6.36 (or RHEL's 2.6.32) this method is
390 disabled.
391 flush
393 The second requires that the backing device support disk
394 flushes (called 'force unit access' in the drive vendors
395 speak). The use of this method can be disabled using the
396 no-disk-flushes option.
397 drain
399 The third method is simply to let write requests drain before
400 write requests of a new reordering domain are issued. This was
401 the only implementation before 8.0.9.
402 none
404 The fourth method is to not express write-after-write
405 dependencies to the backing store at all, by also specifying
406 no-disk-drain. This is dangerous on most IO stacks, may result
407 in write-reordering, and if so, can theoretically be the reason
408 for data corruption, or disturb the DRBD protocol, causing
409 spurious disconnect/reconnect cycles. Do not use
410 no-disk-drain.
411 no-md-flushes
413 Disables the use of disk flushes and barrier BIOs when accessing
414 the meta data device. See the notes on no-disk-flushes.
415 max-bio-bvecs
417 In some special circumstances the device mapper stack manages to
418 pass BIOs to DRBD that violate the constraints that are set forth
419 by DRBD's merge_bvec() function and which have more than one bvec.
420 A known example is: phys-disk -> DRBD -> LVM -> Xen -> misaligned
421 partition (63) -> DomU FS. Then you might see "bio would need to,
422 but cannot, be split:" in the Dom0's kernel log.
423 The best workaround is to proper align the partition within the VM
425 (E.g. start it at sector 1024). This costs 480 KiB of storage.
426 Unfortunately the default of most Linux partitioning tools is to
427 start the first partition at an odd number (63). Therefore most
428 distribution's install helpers for virtual linux machines will end
429 up with misaligned partitions. The second best workaround is to
430 limit DRBD's max bvecs per BIO (= max-bio-bvecs) to 1, but that
431 might cost performance.
432 The default value of max-bio-bvecs is 0, which means that there is
434 no user imposed limitation.
435 disk-timeout
437 If the driver of the lower_device does not finish an IO request
438 within disk_timeout, DRBD considers the disk as failed. If DRBD is
439 connected to a remote host, it will reissue local pending IO
440 requests to the peer, and ship all new IO requests to the peer
441 only. The disk state advances to diskless, as soon as the backing
442 block device has finished all IO requests.
443 The default value of is 0, which means that no timeout is enforced.
445 The default unit is 100ms. This option is available since 8.3.12.
446 sndbuf-size size
448 size is the size of the TCP socket send buffer. The default value
449 is 0, i.e. autotune. You can specify smaller or larger values.
450 Larger values are appropriate for reasonable write throughput with
451 protocol A over high latency networks. Values below 32K do not make
452 sense. Since 8.0.13 resp. 8.2.7, setting the size value to 0 means
453 that the kernel should autotune this.
454 rcvbuf-size size
456 size is the size of the TCP socket receive buffer. The default
457 value is 0, i.e. autotune. You can specify smaller or larger
458 values. Usually this should be left at its default. Setting the
459 size value to 0 means that the kernel should autotune this.
460 timeout time
462 If the partner node fails to send an expected response packet
464 within time tenths of a second, the partner node is considered dead
465 and therefore the TCP/IP connection is abandoned. This must be
466 lower than connect-int and ping-int. The default value is 60 = 6
467 seconds, the unit 0.1 seconds.
468 connect-int time
470 In case it is not possible to connect to the remote DRBD device
472 immediately, DRBD keeps on trying to connect. With this option you
473 can set the time between two retries. The default value is 10
474 seconds, the unit is 1 second.
475 ping-int time
477 If the TCP/IP connection linking a DRBD device pair is idle for
479 more than time seconds, DRBD will generate a keep-alive packet to
480 check if its partner is still alive. The default is 10 seconds, the
481 unit is 1 second.
482 ping-timeout time
484 The time the peer has time to answer to a keep-alive packet. In
486 case the peer's reply is not received within this time period, it
487 is considered as dead. The default value is 500ms, the default unit
488 are tenths of a second.
489 max-buffers number
491 Limits the memory usage per DRBD minor device on the receiving
493 side, or for internal buffers during resync or online-verify. Unit
494 is PAGE_SIZE, which is 4 KiB on most systems. The minimum possible
495 setting is hard coded to 32 (=128 KiB). These buffers are used to
496 hold data blocks while they are written to/read from disk. To avoid
497 possible distributed deadlocks on congestion, this setting is used
498 as a throttle threshold rather than a hard limit. Once more than
499 max-buffers pages are in use, further allocation from this pool is
500 throttled. You want to increase max-buffers if you cannot saturate
501 the IO backend on the receiving side.
502 ko-count number
504 In case the secondary node fails to complete a single write request
506 for count times the timeout, it is expelled from the cluster. (I.e.
507 the primary node goes into StandAlone mode.) To disable this
508 feature, you should explicitly set it to 0; defaults may change
509 between versions.
510 max-epoch-size number
512 The highest number of data blocks between two write barriers. If
514 you set this smaller than 10, you might decrease your performance.
515 allow-two-primaries
517 With this option set you may assign the primary role to both nodes.
519 You only should use this option if you use a shared storage file
520 system on top of DRBD. At the time of writing the only ones are:
521 OCFS2 and GFS. If you use this option with any other file system,
522 you are going to crash your nodes and to corrupt your data!
523 unplug-watermark number
525 This setting has no effect with recent kernels that use explicit
526 on-stack plugging (upstream Linux kernel 2.6.39, distributions may
527 have backported).
528 When the number of pending write requests on the standby
530 (secondary) node exceeds the unplug-watermark, we trigger the
531 request processing of our backing storage device. Some storage
532 controllers deliver better performance with small values, others
533 deliver best performance when the value is set to the same value as
534 max-buffers, yet others don't feel much effect at all. Minimum 16,
535 default 128, maximum 131072.
536 cram-hmac-alg
538 You need to specify the HMAC algorithm to enable peer
540 authentication at all. You are strongly encouraged to use peer
541 authentication. The HMAC algorithm will be used for the challenge
542 response authentication of the peer. You may specify any digest
543 algorithm that is named in /proc/crypto.
544 shared-secret
546 The shared secret used in peer authentication. May be up to 64
548 characters. Note that peer authentication is disabled as long as no
549 cram-hmac-alg (see above) is specified.
550 after-sb-0pri policy
552 possible policies are:
553 disconnect
555 No automatic resynchronization, simply disconnect.
556 discard-younger-primary
558 Auto sync from the node that was primary before the split-brain
559 situation happened.
560 discard-older-primary
562 Auto sync from the node that became primary as second during
563 the split-brain situation.
564 discard-zero-changes
566 In case one node did not write anything since the split brain
567 became evident, sync from the node that wrote something to the
568 node that did not write anything. In case none wrote anything
569 this policy uses a random decision to perform a "resync" of 0
570 blocks. In case both have written something this policy
571 disconnects the nodes.
572 discard-least-changes
574 Auto sync from the node that touched more blocks during the
575 split brain situation.
576 discard-node-NODENAME
578 Auto sync to the named node.
579 after-sb-1pri policy
581 possible policies are:
582 disconnect
584 No automatic resynchronization, simply disconnect.
585 consensus
587 Discard the version of the secondary if the outcome of the
588 after-sb-0pri algorithm would also destroy the current
589 secondary's data. Otherwise disconnect.
590 violently-as0p
592 Always take the decision of the after-sb-0pri algorithm, even
593 if that causes an erratic change of the primary's view of the
594 data. This is only useful if you use a one-node FS (i.e. not
595 OCFS2 or GFS) with the allow-two-primaries flag, AND if you
596 really know what you are doing. This is DANGEROUS and MAY CRASH
597 YOUR MACHINE if you have an FS mounted on the primary node.
598 discard-secondary
600 Discard the secondary's version.
601 call-pri-lost-after-sb
603 Always honor the outcome of the after-sb-0pri algorithm. In
604 case it decides the current secondary has the right data, it
605 calls the "pri-lost-after-sb" handler on the current primary.
606 after-sb-2pri policy
608 possible policies are:
609 disconnect
611 No automatic resynchronization, simply disconnect.
612 violently-as0p
614 Always take the decision of the after-sb-0pri algorithm, even
615 if that causes an erratic change of the primary's view of the
616 data. This is only useful if you use a one-node FS (i.e. not
617 OCFS2 or GFS) with the allow-two-primaries flag, AND if you
618 really know what you are doing. This is DANGEROUS and MAY CRASH
619 YOUR MACHINE if you have an FS mounted on the primary node.
620 call-pri-lost-after-sb
622 Call the "pri-lost-after-sb" helper program on one of the
623 machines. This program is expected to reboot the machine, i.e.
624 make it secondary.
625 always-asbp
627 Normally the automatic after-split-brain policies are only used if
628 current states of the UUIDs do not indicate the presence of a third
629 node.
630 With this option you request that the automatic after-split-brain
632 policies are used as long as the data sets of the nodes are somehow
633 related. This might cause a full sync, if the UUIDs indicate the
634 presence of a third node. (Or double faults led to strange UUID
635 sets.)
636 rr-conflict policy
638 This option helps to solve the cases when the outcome of the resync
639 decision is incompatible with the current role assignment in the
640 cluster.
641 disconnect
643 No automatic resynchronization, simply disconnect.
644 violently
646 Sync to the primary node is allowed, violating the assumption
647 that data on a block device are stable for one of the nodes.
648 Dangerous, do not use.
649 call-pri-lost
651 Call the "pri-lost" helper program on one of the machines. This
652 program is expected to reboot the machine, i.e. make it
653 secondary.
654 data-integrity-alg alg
656 DRBD can ensure the data integrity of the user's data on the
657 network by comparing hash values. Normally this is ensured by the
658 16 bit checksums in the headers of TCP/IP packets.
659 This option can be set to any of the kernel's data digest
661 algorithms. In a typical kernel configuration you should have at
662 least one of md5, sha1, and crc32c available. By default this is
663 not enabled.
664 See also the notes on data integrity.
666 no-tcp-cork
668 DRBD usually uses the TCP socket option TCP_CORK to hint to the
669 network stack when it can expect more data, and when it should
670 flush out what it has in its send queue. It turned out that there
671 is at least one network stack that performs worse when one uses
672 this hinting method. Therefore we introducted this option, which
673 disables the setting and clearing of the TCP_CORK socket option by
674 DRBD.
675 on-congestion congestion_policy, congestion-fill fill_threshold,
677 congestion-extents active_extents_threshold
678 By default DRBD blocks when the available TCP send queue becomes
679 full. That means it will slow down the application that generates
680 the write requests that cause DRBD to send more data down that TCP
681 connection.
682 When DRBD is deployed with DRBD-proxy it might be more desirable
684 that DRBD goes into AHEAD/BEHIND mode shortly before the send queue
685 becomes full. In AHEAD/BEHIND mode DRBD does no longer replicate
686 data, but still keeps the connection open.
687 The advantage of the AHEAD/BEHIND mode is that the application is
689 not slowed down, even if DRBD-proxy's buffer is not sufficient to
690 buffer all write requests. The downside is that the peer node falls
691 behind, and that a resync will be necessary to bring it back into
692 sync. During that resync the peer node will have an inconsistent
693 disk.
694 Available congestion_policys are block and pull-ahead. The default
696 is block. Fill_threshold might be in the range of 0 to 10GiBytes.
697 The default is 0 which disables the check.
698 Active_extents_threshold has the same limits as al-extents.
699 The AHEAD/BEHIND mode and its settings are available since DRBD
701 8.3.10.
702 wfc-timeout time
704 Wait for connection timeout.
705 The init script drbd(8) blocks the boot process until the DRBD
707 resources are connected. When the cluster manager starts later, it
708 does not see a resource with internal split-brain. In case you want
709 to limit the wait time, do it here. Default is 0, which means
710 unlimited. The unit is seconds.
711 degr-wfc-timeout time
713 Wait for connection timeout, if this node was a degraded cluster.
715 In case a degraded cluster (= cluster with only one node left) is
716 rebooted, this timeout value is used instead of wfc-timeout,
717 because the peer is less likely to show up in time, if it had been
718 dead before. Value 0 means unlimited.
719 outdated-wfc-timeout time
721 Wait for connection timeout, if the peer was outdated. In case a
723 degraded cluster (= cluster with only one node left) with an
724 outdated peer disk is rebooted, this timeout value is used instead
725 of wfc-timeout, because the peer is not allowed to become primary
726 in the meantime. Value 0 means unlimited.
727 wait-after-sb
729 By setting this option you can make the init script to continue to
730 wait even if the device pair had a split brain situation and
731 therefore refuses to connect.
732 become-primary-on node-name
734 Sets on which node the device should be promoted to primary role by
735 the init script. The node-name might either be a host name or the
736 keyword both. When this option is not set the devices stay in
737 secondary role on both nodes. Usually one delegates the role
738 assignment to a cluster manager (e.g. heartbeat).
739 stacked-timeouts
741 Usually wfc-timeout and degr-wfc-timeout are ignored for stacked
742 devices, instead twice the amount of connect-int is used for the
743 connection timeouts. With the stacked-timeouts keyword you disable
744 this, and force DRBD to mind the wfc-timeout and degr-wfc-timeout
745 statements. Only do that if the peer of the stacked resource is
746 usually not available or will usually not become primary. By using
747 this option incorrectly, you run the risk of causing unexpected
748 split brain.
749 rate rate
751 To ensure a smooth operation of the application on top of DRBD, it
753 is possible to limit the bandwidth which may be used by background
754 synchronizations. The default is 250 KB/sec, the default unit is
755 KB/sec. Optional suffixes K, M, G are allowed.
756 use-rle
758 During resync-handshake, the dirty-bitmaps of the nodes are
760 exchanged and merged (using bit-or), so the nodes will have the
761 same understanding of which blocks are dirty. On large devices, the
762 fine grained dirty-bitmap can become large as well, and the bitmap
763 exchange can take quite some time on low-bandwidth links.
764 Because the bitmap typically contains compact areas where all bits
766 are unset (clean) or set (dirty), a simple run-length encoding
767 scheme can considerably reduce the network traffic necessary for
768 the bitmap exchange.
769 For backward compatibilty reasons, and because on fast links this
771 possibly does not improve transfer time but consumes cpu cycles,
772 this defaults to off.
773 after res-name
775 By default, resynchronization of all devices would run in parallel.
777 By defining a sync-after dependency, the resynchronization of this
778 resource will start only if the resource res-name is already in
779 connected state (i.e., has finished its resynchronization).
780 al-extents extents
782 DRBD automatically performs hot area detection. With this parameter
784 you control how big the hot area (= active set) can get. Each
785 extent marks 4M of the backing storage (= low-level device). In
786 case a primary node leaves the cluster unexpectedly, the areas
787 covered by the active set must be resynced upon rejoining of the
788 failed node. The data structure is stored in the meta-data area,
789 therefore each change of the active set is a write operation to the
790 meta-data device. A higher number of extents gives longer resync
791 times but less updates to the meta-data. The default number of
792 extents is 127. (Minimum: 7, Maximum: 3843)
793 verify-alg hash-alg
795 During online verification (as initiated by the verify
796 sub-command), rather than doing a bit-wise comparison, DRBD applies
797 a hash function to the contents of every block being verified, and
798 compares that hash with the peer. This option defines the hash
799 algorithm being used for that purpose. It can be set to any of the
800 kernel's data digest algorithms. In a typical kernel configuration
801 you should have at least one of md5, sha1, and crc32c available. By
802 default this is not enabled; you must set this option explicitly in
803 order to be able to use on-line device verification.
804 See also the notes on data integrity.
806 csums-alg hash-alg
808 A resync process sends all marked data blocks from the source to
809 the destination node, as long as no csums-alg is given. When one is
810 specified the resync process exchanges hash values of all marked
811 blocks first, and sends only those data blocks that have different
812 hash values.
813 This setting is useful for DRBD setups with low bandwidth links.
815 During the restart of a crashed primary node, all blocks covered by
816 the activity log are marked for resync. But a large part of those
817 will actually be still in sync, therefore using csums-alg will
818 lower the required bandwidth in exchange for CPU cycles.
819 c-plan-ahead plan_time, c-fill-target fill_target, c-delay-target
821 delay_target, c-max-rate max_rate
822 The dynamic resync speed controller gets enabled with setting
823 plan_time to a positive value. It aims to fill the buffers along
824 the data path with either a constant amount of data fill_target, or
825 aims to have a constant delay time of delay_target along the path.
826 The controller has an upper bound of max_rate.
827 By plan_time the agility of the controller is configured. Higher
829 values yield for slower/lower responses of the controller to
830 deviation from the target value. It should be at least 5 times RTT.
831 For regular data paths a fill_target in the area of 4k to 100k is
832 appropriate. For a setup that contains drbd-proxy it is advisable
833 to use delay_target instead. Only when fill_target is set to 0 the
834 controller will use delay_target. 5 times RTT is a reasonable
835 starting value. Max_rate should be set to the bandwidth available
836 between the DRBD-hosts and the machines hosting DRBD-proxy, or to
837 the available disk-bandwidth.
838 The default value of plan_time is 0, the default unit is 0.1
840 seconds. Fill_target has 0 and sectors as default unit.
841 Delay_target has 1 (100ms) and 0.1 as default unit. Max_rate has
842 10240 (100MiB/s) and KiB/s as default unit.
843 The dynamic resync speed controller and its settings are available
845 since DRBD 8.3.9.
846 c-min-rate min_rate
848 A node that is primary and sync-source has to schedule application
849 IO requests and resync IO requests. The min_rate tells DRBD use
850 only up to min_rate for resync IO and to dedicate all other
851 available IO bandwidth to application requests.
852 Note: The value 0 has a special meaning. It disables the limitation
854 of resync IO completely, which might slow down application IO
855 considerably. Set it to a value of 1, if you prefer that resync IO
856 never slows down application IO.
857 Note: Although the name might suggest that it is a lower bound for
859 the dynamic resync speed controller, it is not. If the DRBD-proxy
860 buffer is full, the dynamic resync speed controller is free to
861 lower the resync speed down to 0, completely independent of the
862 c-min-rate setting.
863 Min_rate has 4096 (4MiB/s) and KiB/s as default unit.
865 on-no-data-accessible ond-policy
867 This setting controls what happens to IO requests on a degraded,
868 disk less node (I.e. no data store is reachable). The available
869 policies are io-error and suspend-io.
870 If ond-policy is set to suspend-io you can either resume IO by
872 attaching/connecting the last lost data storage, or by the drbdadm
873 resume-io res command. The latter will result in IO errors of
874 course.
875 The default is io-error. This setting is available since DRBD
877 8.3.9.
878 cpu-mask cpu-mask
880 Sets the cpu-affinity-mask for DRBD's kernel threads of this
882 device. The default value of cpu-mask is 0, which means that DRBD's
883 kernel threads should be spread over all CPUs of the machine. This
884 value must be given in hexadecimal notation. If it is too big it
885 will be truncated.
886 pri-on-incon-degr cmd
888 This handler is called if the node is primary, degraded and if the
890 local copy of the data is inconsistent.
891 pri-lost-after-sb cmd
893 The node is currently primary, but lost the after-split-brain auto
895 recovery procedure. As as consequence, it should be abandoned.
896 pri-lost cmd
898 The node is currently primary, but DRBD's algorithm thinks that it
900 should become sync target. As a consequence it should give up its
901 primary role.
902 fence-peer cmd
904 The handler is part of the fencing mechanism. This handler is
906 called in case the node needs to fence the peer's disk. It should
907 use other communication paths than DRBD's network link.
908 local-io-error cmd
910 DRBD got an IO error from the local IO subsystem.
912 initial-split-brain cmd
914 DRBD has connected and detected a split brain situation. This
916 handler can alert someone in all cases of split brain, not just
917 those that go unresolved.
918 split-brain cmd
920 DRBD detected a split brain situation but remains unresolved.
922 Manual recovery is necessary. This handler should alert someone on
923 duty.
924 before-resync-target cmd
926 DRBD calls this handler just before a resync begins on the node
928 that becomes resync target. It might be used to take a snapshot of
929 the backing block device.
930 after-resync-target cmd
932 DRBD calls this handler just after a resync operation finished on
934 the node whose disk just became consistent after being inconsistent
935 for the duration of the resync. It might be used to remove a
936 snapshot of the backing device that was created by the
937 before-resync-target handler.
938 Other Keywords
940 include file-pattern
941 Include all files matching the wildcard pattern file-pattern. The
943 include statement is only allowed on the top level, i.e. it is not
944 allowed inside any section.
945
947 There are two independent methods in DRBD to ensure the integrity of
948 the mirrored data. The online-verify mechanism and the
949 data-integrity-alg of the network section.
950 Both mechanisms might deliver false positives if the user of DRBD
952 modifies the data which gets written to disk while the transfer goes
953 on. This may happen for swap, or for certain append while global sync,
954 or truncate/rewrite workloads, and not necessarily poses a problem for
955 the integrity of the data. Usually when the initiator of the data
956 transfer does this, it already knows that that data block will not be
957 part of an on disk data structure, or will be resubmitted with correct
958 data soon enough.
959 The data-integrity-alg causes the receiving side to log an error about
961 "Digest integrity check FAILED: Ns +x\n", where N is the sector offset,
962 and x is the size of the requst in bytes. It will then disconnect, and
963 reconnect, thus causing a quick resync. If the sending side at the same
964 time detected a modification, it warns about "Digest mismatch, buffer
965 modified by upper layers during write: Ns +x\n", which shows that this
966 was a false positive. The sending side may detect these buffer
967 modifications immediately after the unmodified data has been copied to
968 the tcp buffers, in which case the receiving side won't notice it.
969 The most recent (2007) example of systematic corruption was an issue
971 with the TCP offloading engine and the driver of a certain type of GBit
972 NIC. The actual corruption happened on the DMA transfer from core
973 memory to the card. Since the TCP checksum gets calculated on the card,
974 this type of corruption stays undetected as long as you do not use
975 either the online verify or the data-integrity-alg.
976 We suggest to use the data-integrity-alg only during a pre-production
978 phase due to its CPU costs. Further we suggest to do online verify runs
979 regularly e.g. once a month during a low load period.
980
982 This document was revised for version 8.3.2 of the DRBD distribution.
983
985 Written by Philipp Reisner <philipp.reisner@linbit.com> and Lars
986 Ellenberg <lars.ellenberg@linbit.com>.
987
989 Report bugs to <drbd-user@lists.linbit.com>.
990
992 Copyright 2001-2008 LINBIT Information Technologies, Philipp Reisner,
993 Lars Ellenberg. This is free software; see the source for copying
994 conditions. There is NO warranty; not even for MERCHANTABILITY or
995 FITNESS FOR A PARTICULAR PURPOSE.
996
998 drbd(8), drbddisk(8), drbdsetup(8), drbdadm(8), DRBD User's Guide[1],
999 DRBD web site[3]
1000
1002 1. DRBD User's Guide
1003 http://www.drbd.org/users-guide/
1004 2. DRBD's online usage counter
1006 http://usage.drbd.org
1007 3. DRBD web site
1009 http://www.drbd.org/
1010DRBD 8.3.2 5 Dec 2008 DRBD.CONF(5)