fi_mrail(7)

1fi_mrail(7)                    Libfabric v1.15.1                   fi_mrail(7)
2
3
4

NAME

6       fi_mrail - The Multi-Rail Utility Provider
7

OVERVIEW

9       The  mrail provider (ofi_mrail) is an utility provider that layers over
10       an underlying provider to enable the  use  of  multiple  network  ports
11       (rails).  This increases the total available bandwidth of an underlying
12       provider.  The current status of mrail provider is experimental  -  not
13       all libfabric features are supported and performance is not guaranteed.
14

REQUIREMENTS

16   Requirements for underlying provider
17       mrail  provider requires the underlying provider to support the follow‐
18       ing capabilities / modes:
19
20       • Buffered receive (FI_BUFFERED_RECV)
21
22       • FI_SOURCE
23
24       • FI_AV_TABLE
25
26   Requirements for applications
27       Applications need to: * Support FI_MR_RAW MR mode bit to  make  use  of
28       FI_RMA  capability.   * Set FI_OFI_MRAIL_ADDR env variable (see RUNTIME
29       PARAMETERS section below).
30

SUPPORTED FEATURES

32       Endpoint types
33              The provider supports only FI_EP_RDM.
34
35       Endpoint capabilities
36              The following data  transfer  interface  is  supported:  FI_MSG,
37              FI_TAGGED, FI_RMA.
38
39       # LIMITATIONS
40              Limitations  of  the  underlying provider may show up as that of
41              mrail provider.  mrail provider doesn’t  allow  pass-through  of
42              any mode bits to the underlying provider.
43
44   Unsupported features
45       The  following are the major libfabric features that are not supported.
46       Any other feature not listed in “Supported features” can be assumed  as
47       unsupported.
48
49       • FI_ATOMIC
50
51       • Scalable endpoints
52
53       • Shared contexts
54
55       • FABRIC_DIRECT
56
57       • Multicast
58
59       • Triggered operations
60

FUNCTIONALITY OVERVIEW

62       For  messages (FI_MSG, FI_TAGGED), the provider uses different policies
63       to send messages over one or more rails  based  on  message  size  (See
64       FI_OFI_MRIAL_CONFIG  in  the  RUNTIME PARAMETERS section).  Ordering is
65       guaranteed through the use of sequence numbers.
66
67       For RMA, the data is striped equally across all rails.
68

RUNTIME PARAMETERS

70       The ofi_mrail provider checks for the following environment variables.
71
72       FI_OFI_MRAIL_ADDR
73              Comma delimited list of individual rail addresses.  Each address
74              can  be an address in FI_ADDR_STR format, a host name, an IP ad‐
75              dress, or a netdev interface name.
76
77       FI_OFI_MRAIL_ADDR_STRC
78              Deprecated.  Replaced by FI_OFI_MRAIL_ADDR.
79
80       FI_OFI_MRAIL_CONFIG
81              Comma separated list of <max_size>:<policy> pairs, sorted in as‐
82              cending order of <max_size>.  Each pair indicated the rail shar‐
83              ing policy to be used for messages up to the size <max_size> and
84              not covered by all previous pairs.  The value of <policy> can be
85              fixed (a fixed rail is used), round-robin (one rail per message,
86              selected  in  round-robin fashion), or striping (striping across
87              all    the    rails).     The    default    configuration     is
88              16384:fixed,ULONG_MAX:striping.   The value ULONG_MAX can be in‐
89              put as -1.
90

AUTHORS

95       OpenFabrics.
96
97
98
99Libfabric Programmer’s Manual     2021-03-22                       fi_mrail(7)