1fi_cm(3) Libfabric v1.6.1 fi_cm(3)
2
6 fi_cm - Connection management operations
7 fi_connect / fi_listen / fi_accept / fi_reject / fi_shutdown : Manage
9 endpoint connection state.
10 fi_setname / fi_getname / fi_getpeer : Set local, or return local or
12 peer endpoint address.
13 fi_join / fi_close / fi_mc_addr : Join, leave, or retrieve a multicast
15 address.
16
18 #include <rdma/fi_cm.h>
19 int fi_connect(struct fid_ep *ep, const void *addr,
21 const void *param, size_t paramlen);
22 int fi_listen(struct fid_pep *pep);
24 int fi_accept(struct fid_ep *ep, const void *param, size_t paramlen);
26 int fi_reject(struct fid_pep *pep, fid_t handle,
28 const void *param, size_t paramlen);
29 int fi_shutdown(struct fid_ep *ep, uint64_t flags);
31 int fi_setname(fid_t fid, void *addr, size_t addrlen);
33 int fi_getname(fid_t fid, void *addr, size_t *addrlen);
35 int fi_getpeer(struct fid_ep *ep, void *addr, size_t *addrlen);
37 int fi_join(struct fid_ep *ep, const void *addr, uint64_t flags,
39 struct fid_mc **mc, void *context);
40 int fi_close(struct fid *mc);
42 fi_addr_t fi_mc_addr(struct fid_mc *mc);
44
46 ep / pep : Fabric endpoint on which to change connection state.
47 fid Active or passive endpoint to get/set address.
49 addr : Buffer to address. On a set call, the endpoint will be assigned
51 the specified address. On a get, the local address will be copied into
52 the buffer, up to the space provided. For connect, this parameter
53 indicates the peer address to connect to. The address must be in the
54 same format as that specified using fi_info: addr_format when the end‐
55 point was created.
56 addrlen : On input, specifies size of addr buffer. On output, stores
58 number of bytes written to addr buffer.
59 param : User-specified data exchanged as part of the connection
61 exchange.
62 paramlen : Size of param buffer.
64 info : Fabric information associated with a connection request.
66 mc : Multicast group associated with an endpoint.
68 flags : Additional flags for controlling connection operation.
70 context : User context associated with the request.
72
74 Connection management functions are used to connect an connection-ori‐
75 ented endpoint to a peer endpoint.
76 fi_listen
78 The fi_listen call indicates that the specified endpoint should be
79 transitioned into a passive connection state, allowing it to accept
80 incoming connection requests. Connection requests against a listening
81 endpoint are reported asynchronously to the user through a bound CM
82 event queue using the FI_CONNREQ event type. The number of outstanding
83 connection requests that can be queued at an endpoint is limited by the
84 listening endpoint's backlog parameter. The backlog is initialized
85 based on administrative configuration values, but may be adjusted
86 through the fi_control call.
87 fi_connect
89 The fi_connect call initiates a connection request on a connection-ori‐
90 ented endpoint to the destination address. fi_connect may only be
91 called on an endpoint once in its lifetime.
92 fi_accept / fi_reject
94 The fi_accept and fi_reject calls are used on the passive (listening)
95 side of a connection to accept or reject a connection request, respec‐
96 tively. To accept a connection, the listening application first waits
97 for a connection request event (FI_CONNREQ). After receiving such an
98 event, the application allocates a new endpoint to accept the connec‐
99 tion. This endpoint must be allocated using an fi_info structure ref‐
100 erencing the handle from this FI_CONNREQ event. fi_accept is then
101 invoked with the newly allocated endpoint. If the listening applica‐
102 tion wishes to reject a connection request, it calls fi_reject with the
103 listening endpoint and a reference to the connection request.
104 A successfully accepted connection request will result in the active
106 (connecting) endpoint seeing an FI_CONNECTED event on its associated
107 event queue. A rejected or failed connection request will generate an
108 error event. The error entry will provide additional details describ‐
109 ing the reason for the failed attempt.
110 An FI_CONNECTED event will also be generated on the passive side for
112 the accepting endpoint once the connection has been properly estab‐
113 lished. The fid of the FI_CONNECTED event will be that of the endpoint
114 passed to fi_accept as opposed to the listening passive endpoint. Out‐
115 bound data transfers cannot be initiated on a connection-oriented end‐
116 point until an FI_CONNECTED event has been generated. However, receive
117 buffers may be associated with an endpoint anytime.
118 fi_shutdown
120 The fi_shutdown call is used to gracefully disconnect an endpoint from
121 its peer. The flags parameter is reserved and must be 0.
122 Outstanding operations posted to the endpoint when fi_shutdown is
124 called will be canceled or discarded. Notification of canceled opera‐
125 tions will be reported by the provider to the corresponding completion
126 queue(s). Discarded operations will silently be dropped, with no com‐
127 pletions generated. The choice of canceling, versus discarding opera‐
128 tions, is provider dependent. However, all canceled completions will
129 be written before fi_shutdown returns.
130 When called, fi_shutdown does not affect completions already written to
132 a completion queue. Any queued completions associated with asynchro‐
133 nous operations posted to the endpoint may still be retrieved from the
134 corresponding completion queue(s) after an endpoint has been shutdown.
135 An FI_SHUTDOWN event will be generated for an endpoint when the remote
137 peer issues a disconnect using fi_shutdown or abruptly closes the end‐
138 point. Note that in the abrupt close case, an FI_SHUTDOWN event will
139 only be generated if the peer system is reachable and a service or ker‐
140 nel agent on the peer system is able to notify the local endpoint that
141 the connection has been aborted.
142 fi_close
144 Fi_close is used to disassociate an endpoint from a multicast group and
145 close all resources associated with the group. Fi_close must be called
146 on all multicast groups that an endpoint joins.
147 fi_setname
149 The fi_setname call may be used to modify or assign the address of the
150 local endpoint. It is conceptually similar to the socket bind opera‐
151 tion. An endpoint may be assigned an address on its creation, through
152 the fi_info structure. The fi_setname call allows an endpoint to be
153 created without being associated with a specific service (e.g., port
154 number) and/or node (e.g., network) address, with the addressing
155 assigned dynamically. The format of the specified addressing data must
156 match that specified through the fi_info structure when the endpoint
157 was created.
158 If no service address is specified and a service address has not yet
160 been assigned to the endpoint, then the provider will allocate a ser‐
161 vice address and assign it to the endpoint. If a node or service
162 address is specified, then, upon successful completion of fi_setname,
163 the endpoint will be assigned the given addressing. If an address can‐
164 not be assigned, or the endpoint address cannot be modified, an appro‐
165 priate fabric error number is returned.
166 fi_getname / fi_getpeer
168 The fi_getname and fi_getpeer calls may be used to retrieve the local
169 or peer endpoint address, respectively. On input, the addrlen parame‐
170 ter should indicate the size of the addr buffer. If the actual address
171 is larger than what can fit into the buffer, it will be truncated and
172 -FI_ETOOSMALL will be returned. On output, addrlen is set to the size
173 of the buffer needed to store the address, which may be larger than the
174 input value.
175 fi_getname is not guaranteed to return a valid source address until
177 after the specified endpoint has been enabled or has had an address
178 assigned. An endpoint may be enabled explicitly through fi_enable, or
179 implicitly, such as through fi_connect or fi_listen. An address may be
180 assigned using fi_setname. fi_getpeer is not guaranteed to return a
181 valid peer address until an endpoint has been completely connected --
182 an FI_CONNECTED event has been generated.
183 fi_join
185 This call attaches an endpoint to a multicast group. By default, the
186 endpoint will join the group based on the data transfer capabilities of
187 the endpoint. For example, if the endpoint has been configured to both
188 send and receive data, then the endpoint will be able to initiate and
189 receive transfers to and from the multicast group. The fi_join flags
190 may be used to restrict access to the multicast group, subject to end‐
191 point capability limitations.
192 Multicast join operations complete asynchronously. An endpoint must be
194 bound to an event queue prior to calling fi_join. The result of the
195 join operation will be reported to the EQ as an FI_JOIN_COMPLETE event.
196 Applications cannot issue multicast transfers until receiving notifica‐
197 tion that the join operation has completed. Note that an endpoint may
198 begin receiving messages from the multicast group as soon as the join
199 completes, which can occur prior to the FI_JOIN_COMPLETE event being
200 generated.
201 Applications must call fi_close on the multicast group to disconnect
203 the endpoint from the group. After a join operation has completed, the
204 fi_mc_addr call may be used to retrieve the address associated with the
205 multicast group.
206 fi_mc_addr
208 Returns the fi_addr_t address associated with a multicast group. This
209 address must be used when transmitting data to a multicast group and
210 paired with the FI_MULTICAST operation flag.
211
213 Except in functions noted below, flags are reserved and must be 0.
214 FI_SEND : Applies to fi_join. This flag indicates that the endpoint
216 should join the multicast group as a send only member. The endpoint
217 must be configured for transmit operations to use this flag, or an
218 error will occur.
219 FI_RECV : Applies to fi_join. This flag indicates that the endpoint
221 should join the multicast group with receive permissions only. The
222 endpoint must be configured for receive operations to use this flag, or
223 an error will occur.
224
226 Returns 0 on success. On error, a negative value corresponding to fab‐
227 ric errno is returned. Fabric errno values are defined in
228 rdma/fi_errno.h.
229
232 For connection-oriented endpoints, the buffer referenced by param will
233 be sent as part of the connection request or response, subject to the
234 constraints of the underlying connection protocol. Applications may
235 use fi_getopt with the FI_OPT_CM_DATA_SIZE endpoint option to determine
236 the size of application data that may be exchanged as part of a connec‐
237 tion request or response. The fi_connect, fi_accept, and fi_reject
238 calls will silently truncate any application data which cannot fit into
239 underlying protocol messages. User data exchanged as part of the con‐
240 nection process is available as part of the fi_eq_cm_entry structure,
241 for FI_CONNREQ and FI_CONNECTED events, or as additional err_data to
242 fi_eq_err_entry, in the case of a rejected connection.
243
245 fi_getinfo(3), fi_endpoint(3), fi_domain(3), fi_eq(3)
246
248 OpenFabrics.
249Libfabric Programmer's Manual 2017-06-06 fi_cm(3)