fi_rstream(7)

1fi_rstream(7)                  Libfabric v1.17.0                 fi_rstream(7)
2
3
4

NAME

6       fi_rstream
7

OVERVIEW

9       The  rstream provider supports stream messaging over message based RMA.
10       It maps stream to message over a core  RMA-based  OFI  provider.   Only
11       Endpoints  and  EQs  are  needed for connection start-up and messaging.
12       Unlike other OFI providers, rstream does not support CQs or memory reg‐
13       istration  of  any kind.  In order to asynchronously wait for a comple‐
14       tion (cm/msg), one can use fi_control on the endpoint/eq to get  an  fd
15       to  use  in  a  poll  call.   For messaging completions, use FI_PEEK on
16       send/recv after poll to see what type of transaction has transpired.
17

SUPPORTED FEATURES

19       The rstream provider currently supports FI_MSG capabilities.
20
21       Endpoint types
22              The provider supports only endpoint type FI_EP_SOCK_STREAM.
23
24       Endpoint capabilities : The following data transfer interface  is  sup‐
25       ported: fi_msg.
26
27       Modes  The  provider does not require the use of any mode bits but sup‐
28              ports core providers that require FI_CONTEXT and FI_RX_CQ_DATA.
29
30       Progress
31              The rstream provider only supports FI_PROGRESS_MANUAL.
32
33       Threading Model
34              The provider supports FI_THREAD_SAFE
35
36       Verbs-iWarp
37              The provider has added features to enable iWarp.   To  use  this
38              feature,  the  ep protocol iWarp must be requested in an fi_get‐
39              info call.
40

LIMITATIONS

42       The rstream provider is experimental and lacks  performance  validation
43       and  extensive  testing.  The iWarp protocol may need extra initializa‐
44       tion work to re-enable.  Currently the rstream provider is used  to  by
45       the  rsockets-OFI  library  as  a  ULP and hooks into the core provider
46       verbs.  It is not interoperable with the previous  rsockets(v1)  proto‐
47       col.   There  are  default  settings  that  limit  the  message  stream
48       (provider memory region size and CQ size).  These can  be  modified  by
49       fi_setopt.
50

SETTINGS

52       The rstream provider settings can be modified via fi_setopt on the end‐
53       point (FI_OPT_ENDPOINT) along with the following parameters:
54
55       FI_OPT_SEND_BUF_SIZE
56              Size of the send buffer.  Default is 32KB.
57
58       FI_OPT_RECV_BUF_SIZE
59              Size of the recv buffer.  Default is 32KB.
60
61       FI_OPT_TX_SIZE
62              Size of the send queue.  Default is 384.
63
64       FI_OPT_RX_SIZE
65              Size of the recv queue.  Default is 384.
66

OFI EXTENSIONS

68       The rstream provider has extended the current OFI API set in  order  to
69       enable a user implementation of Poll.  Specifically sendmsg(FI_PEEK) is
70       supported which replicates the behavior of  the  recvmsg(FI_PEEK)  fea‐
71       ture.
72

AUTHORS

77       OpenFabrics.
78
79
80
81Libfabric Programmer’s Manual     2022-12-11                     fi_rstream(7)