1fi_rstream(7) Libfabric v1.12.1 fi_rstream(7)
2
3
4
6 fi_rstream
7
9 The rstream provider supports stream messaging over message based RMA.
10 It maps stream to message over a core RMA-based OFI provider. Only
11 Endpoints and EQs are needed for connection start-up and messaging.
12 Unlike other OFI providers, rstream does not support CQs or memory reg‐
13 istration of any kind. In order to asynchronously wait for a comple‐
14 tion (cm/msg), one can use fi_control on the endpoint/eq to get an fd
15 to use in a poll call. For messaging completions, use FI_PEEK on
16 send/recv after poll to see what type of transaction has transpired.
17
19 The rstream provider currently supports FI_MSG capabilities.
20
21 Endpoint types
22 The provider supports only endpoint type FI_EP_SOCK_STREAM.
23
24 Endpoint capabilities : The following data transfer interface is sup‐
25 ported: fi_msg.
26
27 Modes The provider does not require the use of any mode bits but sup‐
28 ports core providers that require FI_CONTEXT and FI_RX_CQ_DATA.
29
30 Progress
31 The rstream provider only supports FI_PROGRESS_MANUAL.
32
33 Threading Model
34 The provider supports FI_THREAD_SAFE
35
36 Verbs-iWarp
37 The provider has added features to enable iWarp. To use this
38 feature, the ep protocol iWarp must be requested in an fi_get‐
39 info call.
40
42 The rstream provider is experimental and lacks performance validation
43 and extensive testing. The iWarp protocol may need extra initializa‐
44 tion work to re-enable. Currently the rstream provider is used to by
45 the rsockets-OFI library as a ULP and hooks into the core provider
46 verbs. It is not interoperable with the previous rsockets(v1) proto‐
47 col. There are default settings that limit the message stream
48 (provider memory region size and CQ size). These can be modified by
49 fi_setopt.
50
52 The rstream provider settings can be modified via fi_setopt on the end‐
53 point (FI_OPT_ENDPOINT) along with the following parameters:
54
55 FI_OPT_SEND_BUF_SIZE
56 Size of the send buffer. Default is 32KB.
57
58 FI_OPT_RECV_BUF_SIZE
59 Size of the recv buffer. Default is 32KB.
60
61 FI_OPT_TX_SIZE
62 Size of the send queue. Default is 384.
63
64 FI_OPT_RX_SIZE
65 Size of the recv queue. Default is 384.
66
68 The rstream provider has extended the current OFI API set in order to
69 enable a user implementation of Poll. Specifically sendmsg(FI_PEEK) is
70 supported which replicates the behavior of the recvmsg(FI_PEEK) fea‐
71 ture.
72
74 fabric(7), fi_provider(7), fi_getinfo(3)
75
77 OpenFabrics.
78
79
80
81Libfabric Programmer's Manual 2020-04-14 fi_rstream(7)