1fi_shm(7) Libfabric v1.6.1 fi_shm(7)
2
6 fi_shm - The SHM Fabric Provider
7
9 The SHM provider is a complete provider that can be used on Linux sys‐
10 tems supporting shared memory and process_vm_readv/process_vm_writev
11 calls. The provider is intended to provide high-performance communica‐
12 tion between processes on the same system.
13
15 This release contains an initial implementation of the SHM provider
16 that offers the following support:
17 Endpoint types : The provider supports only endpoint type FI_EP_RDM.
19 Endpoint capabilities : Endpoints cna support any combinations of the
21 following data transfer capabilities: FI_MSG, FI_TAGGED, FI_RMA, amd
22 FI_ATOMICS. These capabilities can be further defined by FI_SEND,
23 FI_RECV, FI_READ, FI_WRITE, FI_REMOTE_READ, and FI_REMOTE_WRITE to
24 limit the direction of operations.
25 Modes : The provider does not require the use of any mode bits.
27 Progress : The SHM provider supports FI_PROGRESS_MANUAL. Receive side
29 data buffers are not modified outside of completion processing rou‐
30 tines. The provider processes messages using three different methods,
31 based on the size of the message. For messages smaller than 4096
32 bytes, tx completions are generated immediately after the send. For
33 larger messages, tx completions are not generated until the receiving
34 side has processed the message.
35 Address Format : The SHM provider uses the address format FI_ADDR_STR,
37 which follows the general format pattern "[prefix]://[addr]". The
38 application can provide addresses through the node or hints parameter.
39 As long as the address is in a valid FI_ADDR_STR format (contains
40 "://"), the address will be used as is. If the application input is
41 incorrectly formatted or no input was provided, the SHM provider will
42 resolve it according to the following SHM provider standards:
43 (flags & FI_SOURCE) ? src_addr : dest_addr = - if (node && service) :
45 "fi_ns://node:service" - if (service) : "fi_ns://service" - if (node &&
46 !service) : "fi_shm://node" - if (!node && !service) : "fi_shm://PID"
47 !(flags & FI_SOURCE) - src_addr = "fi_shm://PID"
49 In other words, if the application provides a source and/or destination
51 address in an acceptable FI_ADDR_STR format (contains "://"), the call
52 to util_getinfo will successfully fill in src_addr and dest_addr with
53 the provided input. If the input is not in an ADDR_STR format, the
54 shared memory provider will then create a proper FI_ADDR_STR address
55 with either the "fi_ns://" (node/service format) or "fi_shm://" (shm
56 format) prefixes signaling whether the addr is a "unique" address and
57 does or does not need an extra endpoint name identifier appended in
58 order to make it unique. For the shared memory provider, we assume
59 that the service (with or without a node) is enough to make it unique,
60 but a node alone is not sufficient. If only a node is provided, the
61 "fi_shm://" prefix is used to signify that it is not a unique address.
62 If no node or service are provided (and in the case of setting the src
63 address without FI_SOURCE and no hints), the process ID will be used as
64 a default address. On endpoint creation, if the src_addr has the
65 "fi_shm://" prefix, the provider will append ":[dom_idx]:[ep_idx]" as a
66 unique endpoint name (essentially, in place of a service). In the case
67 of the "fi_ns://" prefix (or any other prefix if one was provided by
68 the application), no supplemental information is required to make it
69 unique and it will remain with only the application-defined address.
70 Note that the actual endpoint name will not include the FI_ADDR_STR
71 "*://" prefix since it cannot be included in any shared memory region
72 names. The provider will strip off the prefix before setting the end‐
73 point name. As a result, the addresses "fi_prefix1://my_node:my_ser‐
74 vice" and "fi_prefix2://my_node:my_service" would result in endpoints
75 and regions of the same name. The application can also override the
76 endpoint name after creating an endpoint using setname() without any
77 address format restrictions.
78 Msg flags The provider currently only supports the FI_REMOTE_CQ_DATA
80 msg flag.
81 MR registration mode The provider implements FI_MR_VIRT_ADDR memory
83 mode.
84 Atomic operations The provider supports all combinations of datatype
86 and operations as long as the message is less than 4096 bytes (or 2048
87 for compare operations).
88
90 The SHM provider has hard-coded maximums for supported queue sizes and
91 data transfers. These values are reflected in the related fabric
92 attribute structures
93 EPs must be bound to both RX and TX CQs.
95 No support for selective completions or multi-recv.
97 No support for counters.
99
101 No runtime parameters are currently defined.
102
104 fabric(7), fi_provider(7), fi_getinfo(3)
105
107 OpenFabrics.
108Libfabric Programmer's Manual 2018-02-13 fi_shm(7)