1fi_shm(7)                      Libfabric v1.14.0                     fi_shm(7)
2
3
4

NAME

6       fi_shm - The SHM Fabric Provider
7

OVERVIEW

9       The  SHM provider is a complete provider that can be used on Linux sys‐
10       tems supporting shared  memory  and  process_vm_readv/process_vm_writev
11       calls.  The provider is intended to provide high-performance communica‐
12       tion between processes on the same system.
13

SUPPORTED FEATURES

15       This release contains an initial implementation  of  the  SHM  provider
16       that offers the following support:
17
18       Endpoint types
19              The provider supports only endpoint type FI_EP_RDM.
20
21       Endpoint capabilities
22              Endpoints  cna  support  any  combinations of the following data
23              transfer capabilities: FI_MSG, FI_TAGGED, FI_RMA,  amd  FI_ATOM‐
24              ICS.   These  capabilities  can  be  further defined by FI_SEND,
25              FI_RECV, FI_READ, FI_WRITE, FI_REMOTE_READ, and  FI_REMOTE_WRITE
26              to limit the direction of operations.
27
28       Modes  The provider does not require the use of any mode bits.
29
30       Progress
31              The SHM provider supports FI_PROGRESS_MANUAL.  Receive side data
32              buffers are not modified outside of completion  processing  rou‐
33              tines.   The  provider  processes messages using three different
34              methods, based on the size of the message.  For messages smaller
35              than  4096 bytes, tx completions are generated immediately after
36              the send.  For larger messages, tx completions are not generated
37              until the receiving side has processed the message.
38
39       Address Format
40              The SHM provider uses the address format FI_ADDR_STR, which fol‐
41              lows the general format pattern “[prefix]://[addr]”.  The appli‐
42              cation  can  provide addresses through the node or hints parame‐
43              ter.  As long as the address is in a  valid  FI_ADDR_STR  format
44              (contains “://”), the address will be used as is.  If the appli‐
45              cation input is incorrectly formatted or no input was  provided,
46              the  SHM provider will resolve it according to the following SHM
47              provider standards:
48
49       (flags & FI_SOURCE) ?  src_addr : dest_addr = - if (node && service)  :
50       “fi_ns://node:service” - if (service) : “fi_ns://service” - if (node &&
51       !service) : “fi_shm://node” - if (!node && !service) : “fi_shm://PID”
52
53       !(flags & FI_SOURCE) - src_addr = “fi_shm://PID”
54
55       In other words, if the application provides a source and/or destination
56       address  in an acceptable FI_ADDR_STR format (contains “://”), the call
57       to util_getinfo will successfully fill in src_addr and  dest_addr  with
58       the  provided  input.   If  the input is not in an ADDR_STR format, the
59       shared memory provider will then create a  proper  FI_ADDR_STR  address
60       with  either  the  “fi_ns://” (node/service format) or “fi_shm://” (shm
61       format) prefixes signaling whether the addr is a “unique”  address  and
62       does or does not need an extra endpoint name identifier appended in or‐
63       der to make it unique.  For the shared memory provider, we assume  that
64       the service (with or without a node) is enough to make it unique, but a
65       node alone is  not  sufficient.   If  only  a  node  is  provided,  the
66       “fi_shm://”  prefix is used to signify that it is not a unique address.
67       If no node or service are provided (and in the case of setting the  src
68       address without FI_SOURCE and no hints), the process ID will be used as
69       a default address.  On endpoint  creation,  if  the  src_addr  has  the
70       “fi_shm://”  prefix,  the  provider  will append “:[uid]:[ep_idx]” as a
71       unique endpoint name (essentially, in place of a service).  In the case
72       of  the  “fi_ns://”  prefix (or any other prefix if one was provided by
73       the application), no supplemental information is required  to  make  it
74       unique  and  it  will remain with only the application-defined address.
75       Note that the actual endpoint name will  not  include  the  FI_ADDR_STR
76       "*://"  prefix  since it cannot be included in any shared memory region
77       names.  The provider will strip off the prefix before setting the  end‐
78       point  name.   As a result, the addresses “fi_prefix1://my_node:my_ser‐
79       vice” and “fi_prefix2://my_node:my_service” would result  in  endpoints
80       and  regions  of  the same name.  The application can also override the
81       endpoint name after creating an endpoint using  setname()  without  any
82       address format restrictions.
83
84       Msg  flags  The  provider currently only supports the FI_REMOTE_CQ_DATA
85       msg flag.
86
87       MR registration mode The  provider  implements  FI_MR_VIRT_ADDR  memory
88       mode.
89
90       Atomic  operations  The  provider supports all combinations of datatype
91       and operations as long as the message is less than 4096 bytes (or  2048
92       for compare operations).
93

LIMITATIONS

95       The  SHM provider has hard-coded maximums for supported queue sizes and
96       data transfers.  These values are reflected in the related  fabric  at‐
97       tribute structures
98
99       EPs must be bound to both RX and TX CQs.
100
101       No support for counters.
102

RUNTIME PARAMETERS

104       The shm provider checks for the following environment variables:
105
106       FI_SHM_SAR_THRESHOLD
107              Maximum message size to use segmentation protocol before switch‐
108              ing to mmap (only valid when CMA is  not  available).   Default:
109              SIZE_MAX (18446744073709551615)
110
111       FI_SHM_TX_SIZE
112              Maximum number of outstanding tx operations.  Default 1024
113
114       FI_SHM_RX_SIZE
115              Maximum number of outstanding rx operations.  Default 1024
116
117       FI_SHM_DISABLE_CMA
118              Manually disables CMA.  Default false
119

SEE ALSO

121       fabric(7), fi_provider(7), fi_getinfo(3)
122

AUTHORS

124       OpenFabrics.
125
126
127
128Libfabric Programmer’s Manual     2021-04-20                         fi_shm(7)
Impressum