1fi_provider(7) Libfabric v1.18.1 fi_provider(7)
2 |<- [core provider] <-peer API-> [offload provider]
6 ``` All providers plug into libfabric using an exported provider API.
8 libfabric supports both internal providers, which ship with the library
9 for user convenience, as well as external providers. External provider
10 libraries must be in the library search path, end with the suffix
11 “-fi”, and export the function fi_prov_ini().
12 Once registered with the libfabric core, a provider will be reported to
14 applications as part of the discovery process. Hooking and utility
15 providers will intercept libfabric calls from the application to per‐
16 form some task before calling through to the next provider. If there’s
17 no need to intercept a specific API call, the application will call di‐
18 rectly to the core provider. Where possible provider to provider com‐
19 munication is done using the libfabric APIs itself, including the use
20 of provider specific extensions to reduce call overhead.
21 libfabric defines a set of APIs that specifically target providers that
23 may be used as peers. These APIs are oddly enough called peer APIs.
24 Peer APIs are technically part of the external libfabric API, but are
25 not designed for direct use by applications and are not considered sta‐
26 ble for API backwards compatibility.
27
29 Core providers are stand-alone providers that usually target a specific
30 class of networking devices. That is, a specific NIC, class of network
31 hardware, or lower-level software interface. The core providers are
32 usually what most application developers are concerned with. Core
33 providers may only support libfabric features and interfaces that map
34 efficiently to the underlying hardware or network protocols.
35 The following core providers are built into libfabric by default, as‐
37 suming all build pre-requisites are met. That is, necessary libraries
38 are installed, operating system support is available, etc. This list
39 is not exhaustive.
40 EFA A provider for the Amazon EC2 Elastic Fabric Adapter (EFA)
42 (https://aws.amazon.com/hpc/efa/), a custom-built OS bypass
43 hardware interface for inter-instance communication on EC2. See
44 fi_efa(7) for more information.
45 GNI A provider for the Aries interconnect in Cray XC(TM) systems
47 utilizing the user-space Generic Networking Interface. See
48 fi_gni(7) for more information.
49 OPX Supports Omni-Path networking from Cornelis Networks. See
51 fi_opx(7) for more information.
52 PSM2 Older provider for Omni-Path networks. See fi_psm2(7) for more
54 information.
55 PSM3 Provider for Ethernet networking from Intel. See fi_psm3(7) for
57 more information.
58 SHM A provider for intra-node communication using shared memory.
60 See fi_shm(7) for more information.
61 TCP A provider which runs over the TCP/IP protocol and is available
63 on multiple operating systems. This provider enables develop of
64 libfabric applications on most platforms. See fi_tcp(7) for
65 more information
66 UDP A provider which runs over the UDP/IP protocol and is available
68 on multiple operating systems. This provider enables develop of
69 libfabric applications on most platforms. See fi_udp(7) for
70 more information
71 Verbs This provider targets RDMA NICs for both Linux and Windows plat‐
73 forms. See fi_verbs(7) for more information.
74
76 Utility providers are named with a starting prefix of “ofi_”. Utility
77 providers are distinct from core providers in that they are not associ‐
78 ated with specific classes of devices. They instead work with core
79 providers to expand their features and interact with core providers
80 through libfabric interfaces internally. Utility providers are used to
81 support a specific endpoint type over a simpler endpoint type.
82 Utility providers show up as part of the return’s provider’s name. See
84 fi_fabric(3). Utility providers are enabled automatically for core
85 providers that do not support the feature set requested by an applica‐
86 tion.
87 RxM Implements RDM endpoint semantics over MSG endpoints. See
89 fi_rxm(7) for more information.
90 RxD Implements RDM endpoint semantis over DGRAM endpoints. See
92 fi_rxd(7) for more information.
93
95 Hooking providers are mostly used for debugging purposes. Since hook‐
96 ing providers are built and included in release versions of libfabric,
97 they are always available and have no impact on performance unless en‐
98 abled. Hooking providers can layer over all other providers and inter‐
99 cept, or hook, their calls in order to perform some dedicated task,
100 such as gathering performance data on call paths or providing debug
101 output.
102 See fi_hook(7) for more information.
104
106 Offload providers start with the naming prefix “off_”. An offload
107 provider is meant to be paired with other core and/or utility
108 providers. An offload provider is intended to accelerate specific
109 types of communication, generally by taking advantage of network ser‐
110 vices that have been offloaded into hardware, though actual hardware
111 offload support is not a requirement.
112
114 fabric(7) fi_provider(3)
115
117 OpenFabrics.
118Libfabric Programmer’s Manual 2023-01-02 fi_provider(7)