1OPENAIS_OVERVIEW(8) Openais Programmer's Manual OPENAIS_OVERVIEW(8)
2
6 The openais project is a project to implement a production quality
7 "Revised BSD" licensed implementation of the most recent SA Forum's
8 Application Interface Specification. The Application Interface Speci‐
9 fication is a software API and policies which are used to develop
10 applications that maintain service during faults. The API consists of
11 Availability Management Framework (AMF) which provides application
12 failover, Cluster Membership (CLM), Checkpointing (CKPT), Eventing
13 (EVT), Messaging (MSG), and Distributed Locking (DLOCK).
14 Currently Messaging is unimplemented.
16 Faults occur for various reasons:
18 * Application Faults
20 * Middleware Faults
22 * Operating System Faults
24 * Hardware Faults
26 The major focus of high availability in the past has been to mask hard‐
28 ware faults. Faults in other components of the system have gone
29 unsolved until AIS. AIS can mask many types of faults in applications,
30 middleware, operating systems, or even hardware by providing a simple
31 framework for allowing developers to create redundant applications.
32 These redundant applications can be distributed over multiple nodes
33 such that if any one node faults, another node can recover.
34 Application programmers develop applications to periodically record
36 their state using the checkpointing service. When an active application
37 fails, a standby application recovers the state of the application.
38 This technique, called stateful application failover, provides the fun‐
39 damental difference between openais and other systems that have come
40 before it. With stateful application failover, the end-application
41 user doesn't have to reload the application or redial a telephone. The
42 full state is recorded, so the end-application user sees no interrup‐
43 tion in service.
44 Because programmers can now distribute applications across multiple
46 processes or nodes, a mechanism must exist for them to communicate.
47 This mechanism is provided by two services. The event service provides
48 a publish/subscribe model for events. The messaging service provides
49 end to end messaging. Finally a mechanism to synchronize access is pro‐
50 vided by the distributed lock service.
51 The openais project also provides a group messaging toolkit called EVS.
53 The EVS service implements a messaging model known as Extended Virtual
54 Synchrony. This model allows one sender to transmit to many receivers.
55 Certain guarantees are provided for message and membership delivery
56 which make virtual synchrony ideal for developing distributed applica‐
57 tions.
58
61 The openais executive must be configured. In the directory conf in the
62 source distribution are several files that must be copied to the
63 /etc/ais directory. If openais is packaged by a distro, this may be
64 complete.
65 The directory contains the file openais.conf. Please read the ope‐
67 nais.conf(5) man page for details on the configuration options. The
68 openais project will work out of the box with the default configuration
69 options, although the administrator may desire different options.
70 An user and group of the name "ais" must be added to the system. If
72 openais is packaged from a distro, this step should already be com‐
73 pleted. This can be achieved by executing:
74 [root@slickdeal root]# adduser ais -g ais
76 The openais executive uses cryptographic techniques to ensure authen‐
78 ticity and privacy of the messages. In order for openais to be secure
79 and operate, a private key must be generated and shared to all proces‐
80 sors.
81 First generate the key on one of the nodes:
83 unix# ais-keygen openais authentication key generator.
85 Gathering 1024 bits for key from /dev/random.
87 Writing openais key to /etc/ais/authkey.
89 After this operation, a private key will be in the file
91 /etc/ais/authkey. This private key must be copied to every processor
92 in the cluster. If the private key isn't the same for every node,
93 those nodes with nonmatching private keys will not be able to join the
94 same configuration.
95 Copy the key to some security transportable storage or use ssh to
97 transmit the key from node to node. Then install the key with the com‐
98 mand:
99 unix#: install -D --group=0 --owner=0 --mode=0400
101 /path_to_authkey/authkey /etc/ais/authkey
102 If a message "Invalid digest" appears from the openais executive, the
104 keys are not consistent between processors.
105 Finally run the openais executive. If openais is packaged from a dis‐
107 tro, it may be set to start on system start. It may also be turned off
108 by default in which case the init script for openais must be enabled.
109 After running aisexec, a list of all processors IP addresses running
111 the ais executive and configured on the same multicast address will
112 appear. If they don't appear, there may be a problem with multicast in
113 the distro or hardware. If this happens, participation in the openais
114 mailing list may help solve the problem. The email address is ope‐
115 nais@lists.osdl.org.
116
119 The openais AIS libraries have header files which must be included in
120 the developer's application. Once the header file is included, the
121 developer can reference the AIS interfaces.
122 The openais project recommends to distros to place include files in
124 /usr/include/openais. The following include lines must be added to the
125 application to use each of the following services:
126 #include <openais/saClm.h> For the Cluster Membership B.01.01 service.
128 #include <openais/saCkpt.h> For the Checkpointing B.01.01 service.
130 #include <openais/saEvt.h> For the Eventing B.01.01 service.
132 #include <openais/ais_amf.h> For the AMF A.01.01 service.
134 The openais project recommends to distros to place library files in
136 /usr/lib. The following link lines must be added to the LDFLAGS sec‐
137 tion of the makefile.
138 -lsaClm For the Cluster Membership B.01.01 service
140 -lsaCkpt For the Checkpointing B.01.01 service
142 -lsaEvt For the Eventing B.01.01 service
144 -lsaAmf For the AMF A.01.01 service
146 -lais Specify this to get access to all AIS libraries without specify‐
148 ing each library individually.
149
152 The openais project supports both IPv4 and IPv6 network addresses. The
153 entire cluster must use either IPv4 or IPv6 for the cluster communica‐
154 tion mechanism. In order to use IPv6, IPv6 addresses must be specified
155 in the bindnetaddr and mcastaddr fields in the configuration file. The
156 nodeid field must also be set.
157 An example of this is: nodeid: 2 bindnetaddr: fec0::1:a800:4ff:fe00:20
159 mcastaddr: ff05::1
160 To configure a host for IPv6, use the ifconfig program to add inter‐
162 faces: box20: ifconfig eth0 add fec0::1:a800:4ff:fe00:20/64 box30:
163 ifconfig eth0 add fec0::1:a800:4ff:fe00:30/64
164 If the /64 is not specified, a route for the IPv6 network will not be
166 configured which will cause significant problems. Make sure a route is
167 available for IPv6 traffic.
168
171 The AIS libraries are a thin IPC interface to the openais executive.
172 The openais executive provides services for the SA Forum AIS libraries
173 as well as the EVS and CPG libraries.
174 The openais executive uses the Totem extended virtual synchrony proto‐
176 col. The advantage to the end user is excellent performance character‐
177 istics and a proven protocol with excellent reliability. This protocol
178 connects the processors in a configuration together so they may commu‐
179 nicate.
180
183 The openais executive process uses four environment variables during
184 startup. If these environment variables are not set, defaults will be
185 used.
186 OPENAIS_MAIN_CONFIG_FILE
189 This specifies the fully qualified path to the openais configu‐
190 ration file.
191 The default is /etc/ais/openais.conf.
193 OPENAIS_AMF_CONFIG_FILE
196 This specifies the fully qualified path to the openais Avail‐
197 ability Management Framework configuration file.
198 The default is /etc/ais/amf.conf.
200 OPENAIS_DEFAULT_CONFIG_IFACE
203 This specifies the LCRSO that is used to parse the configuration
204 file. This allows other configuration file parsers to be imple‐
205 mented within the system.
206 The default is to use the default openais configuration file
208 parser which parses the format specified in openais.conf (5).
209 OPENAIS_TOTEM_AUTHKEY_FILE
212 This specifies the fully qualified path to the shared key used
213 to authenticate and encrypt data used within the Totem protocol.
214 The default is /etc/ais/authkey.
216
219 The openais executive optionally encrypts all messages sent over the
220 network using the SOBER-128 stream cipher. The openais executive uses
221 HMAC and SHA1 to authenticate all messages. The openais executive
222 library uses SOBER-128 as a pseudo random number generator. The EVS
223 library feeds the PRNG using the /dev/random Linux device.
224 If membership messages can be captured by intruders, it is possible to
226 execute a denial of service attack on the cluster. In this scenario,
227 the cluster is likely already compromised and a DOS attack is the least
228 of the administration's worries.
229 The security in openais does not offer perfect forward secrecy because
231 the keys are reused. It may be possible for an intruder by capturing
232 packets in an automated fashion to determine the shared key. No such
233 automated attack has been published as of yet. In this scenario, the
234 cluster is likely already compromised to allow the long-term capture of
235 transmitted data.
236 For security reasons, the openais executive binary aisexec should NEVER
238 be setuid or setgid in the filesystem.
239
242 The openais libraries are now nearly compliant with every aspect of the
243 SA Forum's AIS specification. The AMF service, however, is not compli‐
244 ant with the B.01.01 specification. The remaining services pass most
245 of the tests of the saftest suite against the B.01.01 specification.
246
249 The messaging service is partially implemented and not suitable for
250 deployment. The distributed locking service is buggy and not suitable
251 for deployment. The Availability Management Framework is under devel‐
252 opment and not suitable for deployment..
253
256 openais.conf(5), evs_overview(8)
257openais Man Page 2006-05-10 OPENAIS_OVERVIEW(8)