fi_mrail(7)

1fi_mrail(7)                    Libfabric v1.12.1                   fi_mrail(7)
2
3
4

NAME

6       fi_mrail - The Multi-Rail Utility Provider
7

OVERVIEW

9       The  mrail provider (ofi_mrail) is an utility provider that layers over
10       an underlying provider to enable the  use  of  multiple  network  ports
11       (rails).  This increases the total available bandwidth of an underlying
12       provider.  The current status of mrail provider is experimental  -  not
13       all libfabric features are supported and performance is not guaranteed.
14

REQUIREMENTS

16   Requirements for underlying provider
17       mrail  provider requires the underlying provider to support the follow‐
18       ing capabilities / modes:
19
20       • Buffered receive (FI_BUFFERED_RECV)
21
22       • FI_SOURCE
23
24       • FI_AV_TABLE
25
26   Requirements for applications
27       Applications need to: * Support FI_MR_RAW MR mode bit to  make  use  of
28       FI_RMA  capability.   * Set FI_OFI_MRAIL_ADDR env variable (see RUNTIME
29       PARAMETERS section below).
30

SUPPORTED FEATURES

32       Endpoint types
33              The provider supports only FI_EP_RDM.
34
35       Endpoint capabilities
36              The following data  transfer  interface  is  supported:  FI_MSG,
37              FI_TAGGED, FI_RMA.
38
39       # LIMITATIONS
40              Limitations  of  the  underlying provider may show up as that of
41              mrail provider.
42       mrail provider doesn't allow pass-through of any mode bits to  the  un‐
43       derlying provider.
44
45   Unsupported features
46       The  following are the major libfabric features that are not supported.
47       Any other feature not listed in "Supported features" can be assumed  as
48       unsupported.
49
50       • FI_ATOMIC
51
52       • Scalable endpoints
53
54       • Shared contexts
55
56       • FABRIC_DIRECT
57
58       • Multicast
59
60       • Triggered operations
61

FUNCTIONALITY OVERVIEW

63       For  messages (FI_MSG, FI_TAGGED), the provider uses different policies
64       to send messages over one or more rails  based  on  message  size  (See
65       FI_OFI_MRIAL_CONFIG  in  the  RUNTIME PARAMETERS section).  Ordering is
66       guaranteed through the use of sequence numbers.
67
68       For RMA, the data is striped equally across all rails.
69

RUNTIME PARAMETERS

71       The ofi_mrail provider checks for the following environment variables.
72
73       FI_OFI_MRAIL_ADDR
74              Comma delimited list of individual rail addresses.  Each address
75              can  be an address in FI_ADDR_STR format, a host name, an IP ad‐
76              dress, or a netdev interface name.
77
78       FI_OFI_MRAIL_ADDR_STRC
79              Deprecated.  Replaced by FI_OFI_MRAIL_ADDR.
80
81       FI_OFI_MRAIL_CONFIG
82              Comma separated list of <max_size>:<policy> pairs, sorted in as‐
83              cending order of <max_size>.  Each pair indicated the rail shar‐
84              ing policy to be used for messages up to the size <max_size> and
85              not covered by all previous pairs.  The value of <policy> can be
86              fixed (a fixed rail is used), round-robin (one rail per message,
87              selected  in  round-robin fashion), or striping (striping across
88              all    the    rails).     The    default    configuration     is
89              16384:fixed,ULONG_MAX:striping.   The value ULONG_MAX can be in‐
90              put as -1.
91

AUTHORS

96       OpenFabrics.
97
98
99
100Libfabric Programmer's Manual     2020-04-14                       fi_mrail(7)