1fi_cm(3)                       Libfabric v1.12.1                      fi_cm(3)
2
3
4

NAME

6       fi_cm - Connection management operations
7
8       fi_connect / fi_listen / fi_accept / fi_reject / fi_shutdown
9              Manage endpoint connection state.
10
11       fi_setname / fi_getname / fi_getpeer
12              Set local, or return local or peer endpoint address.
13
14       fi_join / fi_close / fi_mc_addr
15              Join, leave, or retrieve a multicast address.
16

SYNOPSIS

18              #include <rdma/fi_cm.h>
19
20              int fi_connect(struct fid_ep *ep, const void *addr,
21                  const void *param, size_t paramlen);
22
23              int fi_listen(struct fid_pep *pep);
24
25              int fi_accept(struct fid_ep *ep, const void *param, size_t paramlen);
26
27              int fi_reject(struct fid_pep *pep, fid_t handle,
28                  const void *param, size_t paramlen);
29
30              int fi_shutdown(struct fid_ep *ep, uint64_t flags);
31
32              int fi_setname(fid_t fid, void *addr, size_t addrlen);
33
34              int fi_getname(fid_t fid, void *addr, size_t *addrlen);
35
36              int fi_getpeer(struct fid_ep *ep, void *addr, size_t *addrlen);
37
38              int fi_join(struct fid_ep *ep, const void *addr, uint64_t flags,
39                  struct fid_mc **mc, void *context);
40
41              int fi_close(struct fid *mc);
42
43              fi_addr_t fi_mc_addr(struct fid_mc *mc);
44

ARGUMENTS

46       ep / pep
47              Fabric endpoint on which to change connection state.
48
49       fid Active or passive endpoint to get/set address.
50
51       addr   Buffer to address.  On a set call, the endpoint will be assigned
52              the specified address.  On a get,  the  local  address  will  be
53              copied  into the buffer, up to the space provided.  For connect,
54              this parameter indicates the peer address to  connect  to.   The
55              address  must  be  in  the  same  format as that specified using
56              fi_info: addr_format when the endpoint was created.
57
58       addrlen
59              On input, specifies size of addr buffer.  On output, stores num‐
60              ber of bytes written to addr buffer.
61
62       param  User-specified  data  exchanged  as  part  of the connection ex‐
63              change.
64
65       paramlen
66              Size of param buffer.
67
68       info   Fabric information associated with a connection request.
69
70       mc     Multicast group associated with an endpoint.
71
72       flags  Additional flags for controlling connection operation.
73
74       context
75              User context associated with the request.
76

DESCRIPTION

78       Connection management functions are used to connect an  connection-ori‐
79       ented endpoint to a peer endpoint.
80
81   fi_listen
82       The  fi_listen  call  indicates  that  the specified endpoint should be
83       transitioned into a passive connection state, allowing it to accept in‐
84       coming  connection  requests.   Connection requests against a listening
85       endpoint are reported asynchronously to the user  through  a  bound  CM
86       event queue using the FI_CONNREQ event type.  The number of outstanding
87       connection requests that can be queued at an endpoint is limited by the
88       listening  endpoint's  backlog  parameter.   The backlog is initialized
89       based on administrative  configuration  values,  but  may  be  adjusted
90       through the fi_control call.
91
92   fi_connect
93       The fi_connect call initiates a connection request on a connection-ori‐
94       ented endpoint to the destination  address.   fi_connect  may  only  be
95       called on an endpoint once in its lifetime.
96
97   fi_accept / fi_reject
98       The  fi_accept  and fi_reject calls are used on the passive (listening)
99       side of a connection to accept or reject a connection request,  respec‐
100       tively.   To accept a connection, the listening application first waits
101       for a connection request event (FI_CONNREQ).  After receiving  such  an
102       event,  the  application allocates a new endpoint to accept the connec‐
103       tion.  This endpoint must be allocated using an fi_info structure  ref‐
104       erencing  the handle from this FI_CONNREQ event.  fi_accept is then in‐
105       voked with the newly allocated endpoint.  If the listening  application
106       wishes to reject a connection request, it calls fi_reject with the lis‐
107       tening endpoint and a reference to the connection request.
108
109       A successfully accepted connection request will result  in  the  active
110       (connecting)  endpoint  seeing  an FI_CONNECTED event on its associated
111       event queue.  A rejected or failed connection request will generate  an
112       error  event.  The error entry will provide additional details describ‐
113       ing the reason for the failed attempt.
114
115       An FI_CONNECTED event will also be generated on the  passive  side  for
116       the  accepting  endpoint  once  the connection has been properly estab‐
117       lished.  The fid of the FI_CONNECTED event will be that of the endpoint
118       passed to fi_accept as opposed to the listening passive endpoint.  Out‐
119       bound data transfers cannot be initiated on a connection-oriented  end‐
120       point until an FI_CONNECTED event has been generated.  However, receive
121       buffers may be associated with an endpoint anytime.
122
123   fi_shutdown
124       The fi_shutdown call is used to gracefully disconnect an endpoint  from
125       its peer.  The flags parameter is reserved and must be 0.
126
127       Outstanding  operations  posted  to  the  endpoint  when fi_shutdown is
128       called will be canceled or discarded.  Notification of canceled  opera‐
129       tions  will be reported by the provider to the corresponding completion
130       queue(s).  Discarded operations will silently be dropped, with no  com‐
131       pletions  generated.  The choice of canceling, versus discarding opera‐
132       tions, is provider dependent.  However, all canceled  completions  will
133       be written before fi_shutdown returns.
134
135       When called, fi_shutdown does not affect completions already written to
136       a completion queue.  Any queued completions associated  with  asynchro‐
137       nous  operations posted to the endpoint may still be retrieved from the
138       corresponding completion queue(s) after an endpoint has been shutdown.
139
140       An FI_SHUTDOWN event will be generated for an endpoint when the  remote
141       peer  issues a disconnect using fi_shutdown or abruptly closes the end‐
142       point.  Note that in the abrupt close case, an FI_SHUTDOWN  event  will
143       only be generated if the peer system is reachable and a service or ker‐
144       nel agent on the peer system is able to notify the local endpoint  that
145       the connection has been aborted.
146
147   fi_close
148       Fi_close is used to disassociate an endpoint from a multicast group and
149       close all resources associated with the group.  Fi_close must be called
150       on all multicast groups that an endpoint joins.
151
152   fi_setname
153       The  fi_setname call may be used to modify or assign the address of the
154       local endpoint.  It is conceptually similar to the socket  bind  opera‐
155       tion.   An endpoint may be assigned an address on its creation, through
156       the fi_info structure.  The fi_setname call allows an  endpoint  to  be
157       created  without  being  associated with a specific service (e.g., port
158       number) and/or node (e.g., network) address, with  the  addressing  as‐
159       signed  dynamically.   The format of the specified addressing data must
160       match that specified through the fi_info structure  when  the  endpoint
161       was created.
162
163       If  no  service  address is specified and a service address has not yet
164       been assigned to the endpoint, then the provider will allocate  a  ser‐
165       vice  address  and assign it to the endpoint.  If a node or service ad‐
166       dress is specified, then, upon successful completion of fi_setname, the
167       endpoint  will  be assigned the given addressing.  If an address cannot
168       be assigned, or the endpoint address cannot be modified, an appropriate
169       fabric error number is returned.
170
171   fi_getname / fi_getpeer
172       The  fi_getname  and fi_getpeer calls may be used to retrieve the local
173       or peer endpoint address, respectively.  On input, the addrlen  parame‐
174       ter should indicate the size of the addr buffer.  If the actual address
175       is larger than what can fit into the buffer, it will be  truncated  and
176       -FI_ETOOSMALL  will be returned.  On output, addrlen is set to the size
177       of the buffer needed to store the address, which may be larger than the
178       input value.
179
180       fi_getname is not guaranteed to return a valid source address until af‐
181       ter the specified endpoint has been enabled or has had an  address  as‐
182       signed.   An  endpoint  may be enabled explicitly through fi_enable, or
183       implicitly, such as through fi_connect or fi_listen.  An address may be
184       assigned  using  fi_setname.   fi_getpeer is not guaranteed to return a
185       valid peer address until an endpoint has been completely  connected  --
186       an FI_CONNECTED event has been generated.
187
188   fi_join
189       This  call  attaches an endpoint to a multicast group.  By default, the
190       endpoint will join the group based on the data transfer capabilities of
191       the endpoint.  For example, if the endpoint has been configured to both
192       send and receive data, then the endpoint will be able to  initiate  and
193       receive  transfers  to and from the multicast group.  The fi_join flags
194       may be used to restrict access to the multicast group, subject to  end‐
195       point capability limitations.
196
197       Multicast join operations complete asynchronously.  An endpoint must be
198       bound to an event queue prior to calling fi_join.  The  result  of  the
199       join operation will be reported to the EQ as an FI_JOIN_COMPLETE event.
200       Applications cannot issue multicast transfers until receiving notifica‐
201       tion  that the join operation has completed.  Note that an endpoint may
202       begin receiving messages from the multicast group as soon as  the  join
203       completes,  which  can  occur prior to the FI_JOIN_COMPLETE event being
204       generated.
205
206       Applications must call fi_close on the multicast  group  to  disconnect
207       the endpoint from the group.  After a join operation has completed, the
208       fi_mc_addr call may be used to retrieve the address associated with the
209       multicast group.
210
211   fi_mc_addr
212       Returns  the fi_addr_t address associated with a multicast group.  This
213       address must be used when transmitting data to a  multicast  group  and
214       paired with the FI_MULTICAST operation flag.
215

FLAGS

217       Except in functions noted below, flags are reserved and must be 0.
218
219       FI_SEND
220              Applies  to  fi_join.   This  flag  indicates  that the endpoint
221              should join the multicast group as a send only member.  The end‐
222              point  must  be  configured  for transmit operations to use this
223              flag, or an error will occur.
224
225       FI_RECV
226              Applies to fi_join.   This  flag  indicates  that  the  endpoint
227              should  join  the multicast group with receive permissions only.
228              The endpoint must be configured for receive  operations  to  use
229              this flag, or an error will occur.
230

RETURN VALUE

232       Returns 0 on success.  On error, a negative value corresponding to fab‐
233       ric errno is returned.  Fabric errno values are defined in  rdma/fi_er‐
234       rno.h.
235

ERRORS

NOTES

238       For  connection-oriented endpoints, the buffer referenced by param will
239       be sent as part of the connection request or response, subject  to  the
240       constraints  of  the  underlying connection protocol.  Applications may
241       use fi_getopt with the FI_OPT_CM_DATA_SIZE endpoint option to determine
242       the size of application data that may be exchanged as part of a connec‐
243       tion request or response.  The  fi_connect,  fi_accept,  and  fi_reject
244       calls will silently truncate any application data which cannot fit into
245       underlying protocol messages.  User data exchanged as part of the  con‐
246       nection  process  is available as part of the fi_eq_cm_entry structure,
247       for FI_CONNREQ and FI_CONNECTED events, or as  additional  err_data  to
248       fi_eq_err_entry, in the case of a rejected connection.
249

SEE ALSO

251       fi_getinfo(3), fi_endpoint(3), fi_domain(3), fi_eq(3)
252

AUTHORS

254       OpenFabrics.
255
256
257
258Libfabric Programmer's Manual     2018-10-05                          fi_cm(3)
Impressum