1fi_provider(7) Libfabric v1.6.1 fi_provider(7)
2
6 fi_provider - Fabric Interface Providers
7
9 Conceptually, a fabric provider may be viewed as a local hardware NIC
10 driver, though a provider is not limited by this definition. The first
11 component of libfabric is a general purpose framework that is capable
12 of handling different types of fabric hardware. All fabric hardware
13 devices and their software drivers are required to support this frame‐
14 work. Devices and the drivers that plug into the libfabric framework
15 are referred to as fabric providers, or simply providers.
16 This distribution of libfabric contains the following providers
18 (although more may be available via run-time plug-ins):
19 Core providers
21 GNI : A provider for the Aries interconnect in Cray XC(TM) systems uti‐
22 lizing the user-space Generic Networking Interface. See fi_gni(7) for
23 more information.
24 PSM : High-speed InfiniBand networking from Intel. See fi_psm(7) for
26 more information.
27 Sockets : A general purpose provider that can be used on any network
29 that supports TCP/UDP sockets. This provider is not intended to pro‐
30 vide performance improvements over regular TCP/UDP sockets, but rather
31 to allow developers to write, test, and debug application code even on
32 platforms that do not have high-speed networking. See fi_sockets(7)
33 for more information.
34 usNIC : Ultra low latency Ethernet networking over Cisco userspace VIC
36 adapters. See fi_usnic(7) for more information.
37 Verbs : This provider uses the Linux Verbs API for network transport.
39 Application performance is, obviously expected to be similar to that of
40 the native Linux Verbs API. Analogous to the Sockets provider, the
41 Verbs provider is intended to enable developers to write, test, and
42 debug application code on platforms that only have Linux Verbs-based
43 networking. See fi_verbs(7) for more information.
44 Blue Gene/Q : See fi_bgq(7) for more information.
46 Utility providers
48 RxM : The RxM provider (ofi_rxm) is an utility provider that supports
49 RDM endpoints emulated over MSG endpoints of a core provider. See
50 fi_rxm(7) for more information.
51
53 Core providers implement the libfabric interfaces directly over
54 low-level hardware and software interfaces. They are designed to sup‐
55 port a specific class of hardware, and may be limited to supporting a
56 single NIC. Core providers often only support libfabric features and
57 interfaces that map efficiently to their underlying hardware.
58 Utility providers are distinct from core providers in that they are not
60 associated with specific classes of devices. They instead work with
61 core providers to expand their features, and interact with core
62 providers through libfabric interfaces internally. Utility providers
63 are often used to support a specific endpoint type over a simpler end‐
64 point type. For example, the RXD provider implements reliability over
65 unreliable datagram endpoints. The utility providers will not layer
66 over the sockets provider unless it is explicitly requested.
67 Utility providers show up as a component in the core provider's compo‐
69 nent list. See fi_fabric(3). Utility providers are enabled automati‐
70 cally for core providers that do not support the feature set requested
71 by an application.
72
74 Libfabric provides a general framework for supporting multiple types of
75 fabric objects and their related interfaces. Fabric providers have a
76 large amount of flexibility in selecting which components they are able
77 and willing to support, based on specific hardware constraints.
78 Provider developers should refer to docs/provider for information on
79 functionality supplied by the framework to assist in provider implemen‐
80 tation. To assist in the development of applications, libfabric speci‐
81 fies the following requirements that must be met by any fabric
82 provider, if requested by an application.
83 Note that the instantiation of a specific fabric object is subject to
85 application configuration parameters and need not meet these require‐
86 ments.
87 · A fabric provider must support at least one endpoint type.
89 · All endpoints must support the message queue data transfer interface
91 (fi_ops_msg).
92 · An endpoint that advertises support for a specific endpoint capabil‐
94 ity must support the corresponding data transfer interface.
95 · FI_ATOMIC - fi_ops_atomic
97 · FI_RMA - fi_ops_rma
99 · FI_TAGGED - fi_ops_tagged
101 · Endpoints must support all transmit and receive operations for any
103 data transfer interface that they support.
104 · Exception: If an operation is only usable for an operation that the
106 provider does not support, and support for that operation is conveyed
107 using some other mechanism, the operation may return
108 · FI_ENOSYS. For example, if the provider does not support injected
110 data, it can set the attribute inject_size = 0, and fail all
111 fi_inject operations.
112 · The framework supplies wrappers around the 'msg' operations that can
114 be used. For example, the framework implements the sendv() msg oper‐
115 ation by calling sendmsg(). Providers may reference the general
116 operation, and supply on the sendmsg() implementation.
117 · Providers must set all operations to an implementation. Function
119 pointers may not be left NULL or uninitialized. The framework sup‐
120 plies empty functions that return -FI_ENOSYS which can be used for
121 this purpose.
122 · Endpoints must support the CM interface as follows:
124 · FI_EP_MSG endpoints must support all CM operations.
126 · FI_EP_DGRAM endpoints must support CM getname and setname.
128 · FI_EP_RDM endpoints must support CM getname and setname.
130 · Providers that support connectionless endpoints must support all AV
132 operations (fi_ops_av).
133 · Providers that support memory registration, must support all MR oper‐
135 ations (fi_ops_mr).
136 · Providers should support both completion queues and counters.
138 · If FI_RMA_EVENT is not supported, counter support is limited to local
140 events only.
141 · Completion queues must support the FI_CQ_FORMAT_CONTEXT and
143 FI_CQ_FORMAT_MSG.
144 · Providers that support FI_REMOTE_CQ_DATA shall support FI_CQ_FOR‐
146 MAT_DATA.
147 · Providers that support FI_TAGGED shall support FI_CQ_FORMAT_TAGGED.
149 · A provider is expected to be forward compatible, and must be able to
151 be compiled against expanded fi_xxx_ops structures that define new
152 functions added after the provider was written. Any unknown func‐
153 tions must be set to NULL.
154 · Providers shall document in their man page which features they sup‐
156 port, and any missing requirements.
157 Future versions of libfabric will automatically enable a more complete
159 set of features for providers that focus their implementation on a nar‐
160 row subset of libfabric capabilities.
161
163 Logging is performed using the FI_ERR, FI_LOG, and FI_DEBUG macros.
164 DEFINITIONS
166 #define FI_ERR(prov_name, subsystem, ...)
167 #define FI_LOG(prov_name, prov, level, subsystem, ...)
169 #define FI_DEBUG(prov_name, subsystem, ...)
171 ARGUMENTS
173 prov_name : String representing the provider name.
174 prov : Provider context structure.
176 level : Log level associated with log statement.
178 subsystem : Subsystem being logged from.
180 DESCRIPTION
182 FI_ERR : Always logged.
183 FI_LOG : Logged if the intended provider, log level, and subsystem
185 parameters match the user supplied values.
186 FI_DEBUG : Logged if configured with the --enable-debug flag.
188
190 fi_gni(7), fi_psm(7), fi_sockets(7), fi_usnic(7), fi_verbs(7),
191 fi_bgq(7),
192
194 OpenFabrics.
195Libfabric Programmer's Manual 2017-12-01 fi_provider(7)