1BOOTHD(8)                                                            BOOTHD(8)
2
3
4

NAME

6       boothd - The Booth Cluster Ticket Manager.
7

SYNOPSIS

9       boothd daemon [-SD] [-c config] [-l lockfile]
10
11       booth list [-s site] [-c config]
12
13       booth grant [-s site] [-c config] [-FCw] ticket
14
15       booth revoke [-s site] [-c config] [-w] ticket
16
17       booth peers [-s site] [-c config]
18
19       booth status [-D] [-c config]
20

DESCRIPTION

22       Booth manages tickets which authorizes one of the cluster sites located
23       in geographically dispersed distances to run certain resources. It is
24       designed to be extend Pacemaker to support geographically distributed
25       clustering.
26
27       It is based on the RAFT protocol, see eg.
28       https://ramcloud.stanford.edu/wiki/download/attachments/11370504/raft.pdf
29       for details.
30

SHORT EXAMPLES

32           # boothd daemon -D
33
34           # booth list
35
36           # booth grant ticket-nfs
37
38           # booth revoke ticket-nfs
39

OPTIONS

41       -c configfile
42           Configuration to use.
43
44           Can be a full path to a configuration file, or a short name; in the
45           latter case, the directory /etc/booth and suffix .conf are added.
46           Per default booth is used, which results in the path
47           /etc/booth/booth.conf.
48
49           The configuration name also determines the name of the PID file -
50           for the defaults, /var/run/booth/booth.pid.
51
52       -s
53           Site address or name.
54
55               The special value 'other' can be used to specify the other
56               site. Obviously, in that case, the booth configuration must
57               have exactly two sites defined.
58
59       -F
60           immediate grant: Don’t wait for unreachable sites to relinquish the
61           ticket. See the Booth ticket management section below for more
62           details. For manual tickets this option allows one to grant a
63           ticket which is currently granted. See the Manual tickets section
64           below for more details.
65
66               This option may be DANGEROUS. It makes booth grant the ticket
67               even though it cannot ascertain that unreachable sites don't
68               hold the same ticket. It is up to the user to make sure that
69               unreachable sites don't have this ticket as granted.
70
71       -w
72           wait for the request outcome: The client waits for the final
73           outcome of grant or revoke request.
74
75       -C
76           wait for ticket commit to CIB: The client waits for the ticket
77           commit to CIB (only for grant requests). If one or more sites are
78           unreachable, this takes the ticket expire time (plus, if defined,
79           the acquire-after time).
80
81       -h, --help
82           Give a short usage output.
83
84       --version
85           Report version information.
86
87       -S
88           systemd mode: don’t fork. Disables daemonizing, the process will
89           remain in the foreground.
90
91       -D
92           Increases the debug output level.
93
94       -l lockfile
95           Use another lock file. By default, the lock file name is inferred
96           from the configuration file name. Normally not needed.
97

COMMANDS

99       Whether the binary is called as boothd or booth doesn’t matter; the
100       first argument determines the mode of operation.
101
102       daemon
103           Tells boothd to serve a site. The locally configured interfaces are
104           searched for an IP address that is defined in the configuration.
105           booth then runs in either /arbitrator/ or /site/ mode.
106
107       client
108           Booth clients can list the ticket information (see also crm_ticket
109           -L), and revoke or grant tickets to a site.
110
111           The grant and, under certain circumstances, revoke operations may
112           take a while to return a definite operation’s outcome. The client
113           will wait up to the network timeout value (by default 5 seconds)
114           for the result. Unless the -w option was set, in which case the
115           client waits indefinitely.
116
117           In this mode the configuration file is searched for an IP address
118           that is locally reachable, ie. matches a configured subnet. This
119           allows one to run the client commands on another node in the same
120           cluster, as long as the config file and the service IP is locally
121           reachable.
122
123           For instance, if the booth service IP is 192.168.55.200, and the
124           local node has 192.168.55.15 configured on one of its network
125           interfaces, it knows which site it belongs to.
126
127           Use -s to direct client to connect to a different site.
128
129       status
130           boothd looks for the (locked) PID file and the UDP socket, prints
131           some output to stdout (for use in shell scripts) and returns an
132           OCF-compatible return code. With -D, a human-readable message is
133           printed to STDERR as well.
134
135       peers
136           List the other boothd servers we know about.
137
138           In addition to the type, name (IP address), and the last time the
139           server was heard from, network statistics are also printed. The
140           statistics are split into two rows, the first one consists of
141           counters for the sent packets and the second one for the received
142           packets. The first counter is the total number of packets and
143           descriptions of the other counters follows:
144
145       resends
146           Packets which had to be resent because the recipient didn’t
147           acknowledge a message. This usually means that either the message
148           or the acknowledgement got lost. The number of resends usually
149           reflect the network reliability.
150
151       error
152           Packets which either couldn’t be sent, got truncated, or were badly
153           formed. Should be zero.
154
155       invalid
156           These packets contain either invalid or non-existing ticket name or
157           refer to a non-existing ticket leader. Should be zero.
158
159       authfail
160           Packets which couldn’t be authenticated. Should be zero.
161

CONFIGURATION FILE

163       The configuration file must be identical on all sites and arbitrators.
164
165       A minimal file may look like this:
166
167           site="192.168.201.100"
168           site="192.168.202.100"
169           arbitrator="192.168.203.100"
170           ticket="ticket-db8"
171
172       Comments start with a hash-sign ('#'). Whitespace at the start and end
173       of the line, and around the '=', are ignored.
174
175       The following key/value pairs are defined:
176
177       port
178           The UDP/TCP port to use. Default is 9929.
179
180       transport
181           The transport protocol to use for Raft exchanges. Currently only
182           UDP is supported.
183
184           Clients use TCP to communicate with a daemon; Booth will always
185           bind and listen to both UDP and TCP ports.
186
187       authfile
188           File containing the authentication key. The key can be either
189           binary or text. If the latter, then both leading and trailing white
190           space, including new lines, is ignored. This key is a shared secret
191           and used to authenticate both clients and servers. The key must be
192           between 8 and 64 characters long and be readable only by the file
193           owner.
194
195       maxtimeskew
196           As protection against replay attacks, packets contain generation
197           timestamps. Such a timestamp is not allowed to be too old. Just how
198           old can be specified with this parameter. The value is in seconds
199           and the default is 600 (10 minutes). If clocks vary more than this
200           default between sites and nodes (which is definitely something you
201           should fix) then set this parameter to a higher value. The time
202           skew test is performed only in concert with authentication.
203
204       site
205           Defines a site Raft member with the given IP. Sites can acquire
206           tickets. The sites' IP should be managed by the cluster.
207
208       arbitrator
209           Defines an arbitrator Raft member with the given IP. Arbitrators
210           help reach consensus in elections and cannot hold tickets.
211
212       Booth needs at least three members for normal operation. Odd number of
213       members provides more redundancy.
214
215       site-user, site-group, arbitrator-user, arbitrator-group
216           These define the credentials boothd will be running with.
217
218           On a (Pacemaker) site the booth process will have to call
219           crm_ticket, so the default is to use hacluster:'haclient'; for an
220           arbitrator this user and group might not exists, so there we
221           default to nobody:'nobody'.
222
223       ticket
224           Registers a ticket. Multiple tickets can be handled by single Booth
225           instance.
226
227           Use the special ticket name defaults to modify the defaults. The
228           defaults stanza must precede all the other ticket specifications.
229
230       All times are in seconds.
231
232       expire
233           The lease time for a ticket. After that time the ticket can be
234           acquired by another site if the ticket holder is not reachable.
235
236           The default is 600.
237
238       acquire-after
239           Once a ticket is lost, wait this time in addition before acquiring
240           the ticket.
241
242           This is to allow for the site that lost the ticket to relinquish
243           the resources, by either stopping them or fencing a node.
244
245           A typical delay might be 60 seconds, but ultimately it depends on
246           the protected resources and the fencing configuration.
247
248           The default is 0.
249
250       renewal-freq
251           Set the ticket renewal frequency period.
252
253           If the network reliability is often reduced over prolonged periods,
254           it is advisable to try to renew more often.
255
256           Before every renewal, if defined, the command or commands specified
257           in before-acquire-handler is run. In that case the renewal-freq
258           parameter is effectively also the local cluster monitoring
259           interval.
260
261       timeout
262           After that time booth will re-send packets if there was an
263           insufficient number of replies. This should be long enough to allow
264           packets to reach other members.
265
266           The default is 5.
267
268       retries
269           Defines how many times to retry sending packets before giving up
270           waiting for acks from other members.
271
272           Default is 10. Values lower than 3 are illegal.
273
274           Ticket renewals should allow for this number of retries. Hence, the
275           total retry time must be shorter than the renewal time (either half
276           the expire time or renewal-freq):
277
278               timeout*(retries+1) < renewal
279
280       weights
281           A comma-separated list of integers that define the weight of
282           individual Raft members, in the same order as the site and
283           arbitrator lines.
284
285           Default is 0 for all; this means that the order in the
286           configuration file defines priority for conflicting requests.
287
288       before-acquire-handler
289           If set, this parameter specifies either a file containing a program
290           to be run or a directory where a number of programs can reside.
291           They are invoked before boothd tries to acquire or renew a ticket.
292           If any of them exits with a code other than 0, boothd relinquishes
293           the ticket.
294
295           Thus it is possible to ensure whether the services and its
296           dependencies protected by the ticket are in good shape at this
297           site. For instance, if a service in the dependency-chain has a
298           failcount of INFINITY on all available nodes, the service will be
299           unable to run. In that case, it is of no use to claim the ticket.
300
301           One or more arguments may follow the program or directory location.
302           Typically, there is at least the name of one of the resources which
303           depend on this ticket.
304
305           See below for details about booth specific environment variables.
306           The distributed service-runnable script is an example which may be
307           used to test whether a pacemaker resource can be started.
308
309       attr-prereq
310           Sites can have GEO attributes managed with the geostore(8) program.
311           Attributes are within ticket’s scope and may be tested by boothd
312           for additional control of ticket failover (automatic) or ticket
313           acquire (manual).
314
315           Attributes are typically used to convey extra information about
316           resources, for instance database replication status. The attributes
317           are commonly updated by resource agents.
318
319           Attribute values are referenced in expressions and may be tested
320           for equality with the eq binary operator or inequality with the ne
321           operator. The usage is as follows:
322
323               attr-prereq = <grant_type> <name> <op> <value>
324
325               <grant_type>: "auto" | "manual"
326               <name>:       attribute name
327               <op>:         "eq" | "ne"
328               <value>:      attribute value
329
330           The two grant types are auto for ticket failover and manual for
331           grants using the booth client. Only in case the expression
332           evaluates to true can the ticket be granted.
333
334           It is not clear whether the manual grant type has any practical use
335           because, obviously, this operation is anyway controlled by a human.
336
337           Note that there can be no guarantee on whether an attribute value
338           is up to date, i.e. if it actually reflects the current state.
339
340       mode
341           Specifies if the ticket is manual or automatic.
342
343           By default all tickets are automatic (that is, they are fully
344           controlled by Raft algorithm). Assign the strings "manual" or
345           "MANUAL" to define the ticket as manually controlled.
346
347       debug
348           Specifies the debug output level. Alternative to command line
349           argument. Effective only for daemon mode of operation.
350
351       One example of a booth configuration file:
352
353           transport = udp
354           port = 9930
355
356           # D-85774
357           site="192.168.201.100"
358           # D-90409
359           site="::ffff:192.168.202.100"
360           # A-1120
361           arbitrator="192.168.203.100"
362
363           ticket="ticket-db8"
364               expire        = 600
365               acquire-after = 60
366               timeout       = 10
367               retries       = 5
368               renewal-freq  = 60
369               before-acquire-handler = /usr/share/booth/service-runnable db8
370               attr-prereq = auto repl_state eq ACTIVE
371

BOOTH TICKET MANAGEMENT

373       The booth cluster guarantees that every ticket is owned by only one
374       site at the time.
375
376       Tickets must be initially granted with the booth client grant command.
377       Once it gets granted, the ticket is managed by the booth cluster.
378       Hence, only granted tickets are managed by booth.
379
380       If the ticket gets lost, i.e. that the other members of the booth
381       cluster do not hear from the ticket owner in a sufficiently long time,
382       one of the remaining sites will acquire the ticket. This is what is
383       called ticket failover.
384
385       If the remaining members cannot form a majority, then the ticket cannot
386       fail over.
387
388       A ticket may be revoked at any time with the booth client revoke
389       command. For revoke to succeed, the site holding the ticket must be
390       reachable.
391
392       Once the ticket is administratively revoked, it is not managed by the
393       booth cluster anymore. For the booth cluster to start managing the
394       ticket again, it must be again granted to a site.
395
396       The grant operation, in case not all sites are reachable, may get
397       delayed for the ticket expire time (and, if defined, the acquire-after
398       time). The reason is that the other booth members may not know if the
399       ticket is currently granted at the unreachable site.
400
401       This delay may be disabled with the -F option. In that case, it is up
402       to the administrator to make sure that the unreachable site is not
403       holding the ticket.
404
405       When the ticket is managed by booth, it is dangerous to modify it
406       manually using either crm_ticket command or crm site ticket. Neither of
407       these tools is aware of booth and, consequently, booth itself may not
408       be aware of any ticket status changes. A notable exception is setting
409       the ticket to standby which is typically done before a planned
410       failover.
411

NOTES

413       Tickets are not meant to be moved around quickly, the default expire
414       time is 600 seconds (10 minutes).
415
416       booth works with both IPv4 and IPv6 addresses.
417
418       booth renews a ticket before it expires, to account for possible
419       transmission delays. The renewal time, unless explicitly set, is set to
420       half the expire time.
421

HANDLERS

423       Currently, there’s only one external handler defined (see the
424       before-acquire-handler configuration item above).
425
426       The following environment variables are exported to the handler:
427
428       *BOOTH_TICKET
429           The ticket name, as given in the configuration file. (See ticket
430           item above.)
431
432       *BOOTH_LOCAL
433           The local site name, as defined in site.
434
435       *BOOTH_CONF_PATH
436           The path to the active configuration file.
437
438       *BOOTH_CONF_NAME
439           The configuration name, as used by the -c commandline argument.
440
441       *BOOTH_TICKET_EXPIRES
442           When the ticket expires (in seconds since 1.1.1970), or 0.
443
444       The handler is invoked with positional arguments specified after it.
445

FILES

447       /etc/booth/booth.conf
448           The default configuration file name. See also the -c argument.
449
450       /etc/booth/authkey
451           There is no default, but this is a typical location for the shared
452           secret (authentication key).
453
454       /var/run/booth/
455           Directory that holds PID/lock files. See also the status command.
456

RAFT IMPLEMENTATION

458       In essence, every ticket corresponds to a separate Raft cluster.
459
460       A ticket is granted to the Raft Leader which then owns (or keeps) the
461       ticket.
462

ARBITRATOR MANAGEMENT

464       The booth daemon for an arbitrator which typically doesn’t run the
465       cluster stack, may be started through systemd or with
466       /etc/init.d/booth-arbitrator, depending on which init system the
467       platform supports.
468
469       The SysV init script starts a booth arbitrator for every configuration
470       file found in /etc/booth.
471
472       Platforms running systemd can enable and start every configuration
473       separately using systemctl:
474
475           # systemctl enable booth@<configurationname>
476           # systemctl start  booth@<configurationname>
477
478       systemctl requires the configuration name, even for the default name
479       booth.
480
481           MANUAL TICKETS
482
483       Manual tickets allow users to create and manage tickets which are
484       subsequently handled by booth without using the Raft algorithm.
485       Granting and revoking manual tickets is fully controlled by the
486       administrator. It is possible to define a number of manual and normal
487       tickets in one GEO cluster.
488
489       Automatic ticket management provided by Raft algorithm isn’t applied to
490       manually controlled tickets. In particular, there is no elections,
491       automatic failover procedures, and term expiration.
492
493       However, booth controls if a ticket is currently being granted to any
494       site and warns the user approprietly.
495
496       Tickets which were manually granted to a site, will remain there until
497       they are manually revoked. Even if a site becomes offline, the ticket
498       will not be moved to another site. This behavior allows administrators
499       to make sure that some services will remain in a particular site and
500       will not be moved to another site, possibly located in a different
501       geographical location.
502
503       Also, configuring only manual tickets in a GEO cluster, allows one to
504       have just two sites in a cluster, without a need of having an
505       arbitrator. This is possible because there is no automatic elections
506       and no voting performed for manual tickets.
507
508       Manual tickets are defined in a configuration files by adding a mode
509       ticket parameter and setting it to manual or MANUAL:
510
511           ticket="manual-ticket"
512               [...]
513               mode = manual
514               [...]
515
516       Manual tickets can be granted and revoked by using normal grant and
517       revoke commands, with the usual flags and parameters. The only
518       difference is that specyfiyng -F flag during grant command, forced a
519       site to become a leader of the specified ticket, even if the ticket is
520       granted to another site.
521

EXIT STATUS

523       0
524           Success. For the status command: Daemon running.
525
526       1 (PCMK_OCF_UNKNOWN_ERROR)
527           General error code.
528
529       7 (PCMK_OCF_NOT_RUNNING)
530           No daemon process for that configuration active.
531

BUGS

533       Booth is tested regularly. See the README-testing file for more
534       information.
535
536       Please report any bugs either at GitHub:
537       https://github.com/ClusterLabs/booth/issues
538
539       Or, if you prefer bugzilla, at openSUSE bugzilla (component "High
540       Availability"):
541       https://bugzilla.opensuse.org/enter_bug.cgi?product=openSUSE%20Factory
542

AUTHOR

544       boothd was originally written (mostly) by Jiaju Zhang.
545
546       In 2013 and 2014 Philipp Marek took over maintainership.
547
548       Since April 2014 it has been mainly developed by Dejan Muhamedagic.
549
550       Many people contributed (see the AUTHORS file).
551

RESOURCES

553       GitHub: https://github.com/ClusterLabs/booth
554
555       Documentation:
556       http://doc.opensuse.org/products/draft/SLE-HA/SLE-ha-guide_sd_draft/cha.ha.geo.html
557

COPYING

559       Copyright © 2011 Jiaju Zhang jjzhang@suse.de
560
561       Copyright © 2013-2014 Philipp Marek philipp.marek@linbit.com
562
563       Copyright © 2014 Dejan Muhamedagic dmuhamedagic@suse.com
564
565       Free use of this software is granted under the terms of the GNU General
566       Public License (GPL) as of version 2 (see COPYING file) or later.
567
568
569
570                                  2022-05-19                         BOOTHD(8)
Impressum