rdma_cm(7)

1RDMA_CM(7)               Librdmacm Programmer's Manual              RDMA_CM(7)
2
3
4

NAME

6       rdma_cm - RDMA communication manager.
7

SYNOPSIS

9       #include <rdma/rdma_cma.h>
10

DESCRIPTION

12       Used to establish communication over RDMA transports.
13

NOTES

15       The  RDMA  CM  is  a communication manager used to setup reliable, con‐
16       nected and unreliable datagram data transfers.   It  provides  an  RDMA
17       transport  neutral  interface for establishing connections.  The API is
18       based on sockets, but adapted for queue pair (QP) based semantics: com‐
19       munication  must be over a specific RDMA device, and data transfers are
20       message based.
21
22       The RDMA CM only provides the communication management (connection set‐
23       up  /  teardown)  portion of an RDMA API.  It works in conjunction with
24       the verbs API  defined  by  the  libibverbs  library.   The  libibverbs
25       library provides the interfaces needed to send and receive data.
26

CLIENT OPERATION

28       This section provides a general overview of the basic operation for the
29       active, or client, side of communication.  A  general  connection  flow
30       would be:
31
32       rdma_create_event_channel
33              create channel to receive events
34
35       rdma_create_id
36              allocate an rdma_cm_id, this is conceptually similar to a socket
37
38       rdma_resolve_addr
39              obtain a local RDMA device to reach the remote address
40
41       rdma_get_cm_event
42              wait for RDMA_CM_EVENT_ADDR_RESOLVED event
43
44       rdma_ack_cm_event
45              ack event
46
47       rdma_create_qp
48              allocate a QP for the communication
49
50       rdma_resolve_route
51              determine the route to the remote address
52
53       rdma_get_cm_event
54              wait for RDMA_CM_EVENT_ROUTE_RESOLVED event
55
56       rdma_ack_cm_event
57              ack event
58
59       rdma_connect
60              connect to the remote server
61
62       rdma_get_cm_event
63              wait for RDMA_CM_EVENT_ESTABLISHED event
64
65       rdma_ack_cm_event
66              ack event
67
68       Perform data transfers over connection
69
70       rdma_disconnect
71              tear-down connection
72
73       rdma_get_cm_event
74              wait for RDMA_CM_EVENT_DISCONNECTED event
75
76       rdma_ack_cm_event
77              ack event
78
79       rdma_destroy_qp
80              destroy the QP
81
82       rdma_destroy_id
83              release the rdma_cm_id
84
85       rdma_destroy_event_channel
86              release the event channel
87
88       An  almost  identical process is used to setup unreliable datagram (UD)
89       communication between nodes.  No actual connection  is  formed  between
90       QPs however, so disconnection is not needed.
91
92       Although  this  example  shows  the  client  initiating the disconnect,
93       either side of a connection may initiate the disconnect.
94

SERVER OPERATION

96       This section provides a general overview of the basic operation for the
97       passive,  or  server, side of communication.  A general connection flow
98       would be:
99
100       rdma_create_event_channel
101              create channel to receive events
102
103       rdma_create_id
104              allocate an rdma_cm_id, this is conceptually similar to a socket
105
106       rdma_bind_addr
107              set the local port number to listen on
108
109       rdma_listen
110              begin listening for connection requests
111
112       rdma_get_cm_event
113              wait  for  RDMA_CM_EVENT_CONNECT_REQUEST  event   with   a   new
114              rdma_cm_id
115
116       rdma_create_qp
117              allocate a QP for the communication on the new rdma_cm_id
118
119       rdma_accept
120              accept the connection request
121
122       rdma_ack_cm_event
123              ack event
124
125       rdma_get_cm_event
126              wait for RDMA_CM_EVENT_ESTABLISHED event
127
128       rdma_ack_cm_event
129              ack event
130
131       Perform data transfers over connection
132
133       rdma_get_cm_event
134              wait for RDMA_CM_EVENT_DISCONNECTED event
135
136       rdma_ack_cm_event
137              ack event
138
139       rdma_disconnect
140              tear-down connection
141
142       rdma_destroy_qp
143              destroy the QP
144
145       rdma_destroy_id
146              release the connected rdma_cm_id
147
148       rdma_destroy_id
149              release the listening rdma_cm_id
150
151       rdma_destroy_event_channel
152              release the event channel
153

RETURN CODES

155       =  0   success
156
157       = -1   error - see errno for more details
158
159       Librdmacm functions return 0 to indicate success, and a -1 return value
160       to indicate failure.  If a function operates asynchronously,  a  return
161       value  of  0  means  that  the operation was successfully started.  The
162       operation could still complete in error; users should check the  status
163       of  the related event.  If the return value is -1, then errno will con‐
164       tain additional information regarding the reason for the failure.
165
166       Prior versions of the library would return -errno and not set errno for
167       some  cases  related  to  ENOMEM, ENODEV, ENODATA, EINVAL, and EADDRNO‐
168       TAVAIL codes. Applications that want to check these codes and have com‐
169       patability  with  prior library versions must manually set errno to the
170       negative of the return code if it is < -1.
171