1COROIPC_OVERVIEW(8)Corosync Cluster Engine Programmer's ManuaClOROIPC_OVERVIEW(8)
2
6 coroipc_overview - Overview of coroipc libraries
7
9 The coroipcs and coroipcc libraries provide a generically reusable very
10 high performance shared memory IPC sytem for client and service appli‐
11 cations. It supports many features including:
12 65536 user services and 65536 command ids per service.
14 Shared memory implementation for very high performance.
16 A synchronous request/response channel and asynchronous response chan‐
18 nel per ipc connection.
19 User defined private data per IPC connection.
21 Ability to call a function per service on ipc connection and disconnec‐
23 tion.
24 Authenticated IPC connection with ability for developer to define which
26 UIDs and GIDs are valid at connection time.
27 Fully abstracted poll system so that any poll library may be used.
29 User defined selector for determining the proper function to call per
31 service and id.
32
35 There are two shared libraries available for developing IPC client
36 applications.
37 The first library is coroipcs.so which is responsible for the server
39 implementation. This library should be linked with the server and then
40 initialized with coroipcs_init(3).
41 Once the library is initialized, it will provide service to coroipcc.so
43 library users.
44 The second library is coroipcc.so which is responsible for the client
46 implementation. This library should be linked with the client and
47 requires no initialization. This library provides communication func‐
48 tions for sending and receiving synchronous requests, and also reading
49 asynchronous message requests from the server.
50
53 To use the coroipcs library, the developer creates a
54 coroipcs_init_state structure and populates it with function names.
55 The functions do various operations described in coroipcs_init(3) man
56 page. Not all operations must be specified. If some are missing, the
57 corosync ipcs library will automatically populate the structure with
58 internal versions which provide basic functionality.
59
62 Every ipc connection is represented by a void * pointer which uniquely
63 identifies the data set for the IPC connection. Each IPC connection
64 also contains user defined private data. To obtain this private data
65 pointer, the function coroipcs_private_data_get(3) function can be
66 called.
67 There are two channels for communication. The primary channel is the
69 synchronous request/response channel. Another channel is available for
70 out of band asynchronous responses from the server.
71 To send a response on the syncronous channel,
73 coroipcs_response_send(3) or coroipcs_response_iov_send(3) should be
74 used. To send a message on the asynchronous channel, coroipcs_dis‐
75 patch_send(3) or coroipc_dispatch_iov_send(3) should be used.
76
79 There are many different poll systems in use in applications. They are
80 usually intricately tied up in the implementation of the application
81 and each provide different APIs and functionality. To manage this, the
82 coroipcs library provides callbacks in coroipcs_init(3) which should be
83 called when a new connection should be added for accept system calls or
84 to dispatch messages.
85 These callbacks add the relevant fd to the application's poll system.
87 When the application poll system triggers the callback registered by
88 the user defined poll adding functions, they then call either
89 coroipc_handler_accept(3) or coroipc_handler_dispatch(3).
90
93 No initialization is required in the coroipcc library.
94
97 An IPC connection is made to the server with coroipcc_service_con‐
98 nect(3). This function connects to the server and requests channels be
99 created for communication. To disconnect, the client either exits or
100 executes the function coroipcc_service_disconnect(3).
101
104 There are two functions for sending a request and receiving a response.
105 The first function coroipcc_msg_send_reply_receive(3) sends an iovector
106 request and receives a response. This function copies the response
107 into the response buffer. the second function
108 coroipcc_msg_end_reply_receive_in_buf(3) does not copy the response
109 buffer and allows for zero-copy reading of the response when the life‐
110 time of the response buffer is known.
111
114 The coroipcc_dispatch_recv(3) function receives an out-of-band asyn‐
115 cronous message. Unlike the synchronous communication channel, the
116 asynchronous messages are queued and can provide very high out-of-band
117 performance.
118 To determine when to call coroipcc_dispatch_recv(3) the
120 corosync_fd_get(3) call is used to obtain a file descriptor used in the
121 poll(2) or select(2) system calls.
122 Finally the current dispatch flow control state can be obtained with
124 coroipcc_flow_control_get(3).
125
128 The ipc system is tuned for very high performance while also being com‐
129 letely abstracted from the underlying poll mechanism and any inter‐
130 nalisms required by the server. The ipc system achieves such high per‐
131 formance by using shared memory as oppossed to slower techniques such
132 as UNIX_PF sockets.
133 We intend to do further development to allow syncronous requests to
135 return messages in an asyncronous way to avoid blocking involved in the
136 syncronous request/response model used today for higher throughput in
137 some use cases.
138
141 The ipc system uses default operating system security mechanics to
142 ensure ipc connections are validated. A callback used with
143 coroipcs_init(3) is called for every new ipc connection with the param‐
144 eters of UID and GID. The callback then determines if the UID and GID
145 are authenticated for communication. More about this topic can be
146 viewed in the coroipcs_init(3) man page.
147
150 coroipcs_ipc_init(3), coroipcs_ipc_exit(3), coroipcs_pri‐
151 vate_data_get(3), coroipcs_respone_send(3),
152 coroipcs_response_iov_send(3), coroipcs_dispatch_send(3), coroipcs_dis‐
153 patch_iov_send(3), coroipcs_refcount_inc(3), coroipcs_refcount_dec(3),
154 coroipcs_handler_accept(3), coroipcs_handler_dispatch(3),
155 cooripcc_service_connect(3), coroipcc_service_disconnect(3),
157 coroipcc_msg_send_reply_receive(3),
158 coroipcc_msg_send_reply_receive_in_buf(3), coroipcc_dispatch_recv(3),
159 coroipcc_fd_get(3), coroipcc_dispatch_flow_control_get(3)
160corosync Man Page 2009-03-21 COROIPC_OVERVIEW(8)