1fi_getinfo(3)                  Libfabric v1.12.1                 fi_getinfo(3)
2
3
4

NAME

6       fi_getinfo, fi_freeinfo - Obtain / free fabric interface information
7
8       fi_allocinfo, fi_dupinfo - Allocate / duplicate an fi_info structure
9

SYNOPSIS

11              #include <rdma/fabric.h>
12
13              int fi_getinfo(int version, const char *node, const char *service,
14                      uint64_t flags, const struct fi_info *hints, struct fi_info **info);
15
16              void fi_freeinfo(struct fi_info *info);
17
18              struct fi_info *fi_allocinfo(void);
19
20              struct fi_info *fi_dupinfo(const struct fi_info *info);
21

ARGUMENTS

23       version
24              Interface version requested by application.
25
26       node   Optional, name or fabric address to resolve.
27
28       service
29              Optional, service name or port number of address.
30
31       flags  Operation flags for the fi_getinfo call.
32
33       hints  Reference  to  an  fi_info structure that specifies criteria for
34              selecting the returned fabric information.
35
36       info   A pointer to a linked list of fi_info structures containing  re‐
37              sponse information.
38

DESCRIPTION

40       fi_getinfo  returns  information  about  available  fabric services for
41       reaching specified node or service,  subject  to  any  provided  hints.
42       Callers  may  specify NULL for node, service, and hints in order to re‐
43       trieve information about what providers are available and their optimal
44       usage  models.   If  no  matching fabric information is available, info
45       will be set to NULL and the call will return -FI_ENODATA.
46
47       Based on the input hints, node, and service parameters, a list of  fab‐
48       ric  domains  and  endpoints  will be returned.  Each fi_info structure
49       will describe an endpoint that meets the application's specified commu‐
50       nication  criteria.   Each  endpoint  will be associated with a domain.
51       Applications can restrict the number of returned endpoints by including
52       additional criteria in their search hints.  Relaxing or eliminating in‐
53       put hints will increase the number  and  type  of  endpoints  that  are
54       available.   Providers  that  return  multiple  endpoints  to  a single
55       fi_getinfo call should return the endpoints that are highest performing
56       first.   Providers may indicate that an endpoint and domain can support
57       additional capabilities than those requested by the user only  if  such
58       support will not adversely affect application performance or security.
59
60       The version parameter is used by the application to request the desired
61       version of the interfaces.  The version determines the  format  of  all
62       data  structures  used  by  any of the fabric interfaces.  Applications
63       should use the FI_VERSION(major, minor) macro to indicate the  version,
64       with  hard-coded  integer  values  for the major and minor values.  The
65       FI_MAJOR_VERSION and FI_MINOR_VERSION enum values defined  in  fabric.h
66       specify  the  latest  version of the installed library.  However, it is
67       recommended that the integer values  for  FI_MAJOR_VERSION  and  FI_MI‐
68       NOR_VERSION be used, rather than referencing the enum types in order to
69       ensure compatibility with future versions of the  library.   This  pro‐
70       tects  against  the application being built from source against a newer
71       version of the library that introduces new fields to  data  structures,
72       which would not be initialized by the application.
73
74       Node,  service,  or  hints  may be provided, with any combination being
75       supported.  If node is provided, fi_getinfo will attempt to resolve the
76       fabric  address  to  the  given node.  If node is not given, fi_getinfo
77       will attempt to resolve the fabric addressing information based on  the
78       provided  hints.   Node  is  commonly used to provide a network address
79       (such as an IP address) or hostname.   Service  is  usually  associated
80       with a transport address (such as a TCP port number).  Node and service
81       parameters may be mapped by providers to native fabric addresses.   Ap‐
82       plications  may also pass in an FI_ADDR_STR formatted address (see for‐
83       mat details below) as the node parameter.  In such cases,  the  service
84       parameter must be NULL.
85
86       The  hints  parameter,  if provided, may be used to limit the resulting
87       output as indicated below.  As a general rule,  specifying  a  non-zero
88       value  for  input  hints indicates that a provider must support the re‐
89       quested value or fail the operation with -FI_ENODATA.  With the  excep‐
90       tion  of  mode  bits, hints that are set to zero are treated as a wild‐
91       card.  A zeroed hint value results in providers either returning a  de‐
92       fault  value or a value that works best for their implementation.  Mode
93       bits that are set to zero indicate the application does not support any
94       modes.
95
96       The caller must call fi_freeinfo to release fi_info structures returned
97       by this call.
98
99       The fi_allocinfo call will allocate and zero an fi_info  structure  and
100       all  related  substructures.   The  fi_dupinfo  will duplicate a single
101       fi_info structure and all the substructures within it.
102

FI_INFO

104              struct fi_info {
105                  struct fi_info        *next;
106                  uint64_t              caps;
107                  uint64_t              mode;
108                  uint32_t              addr_format;
109                  size_t                src_addrlen;
110                  size_t                dest_addrlen;
111                  void                  *src_addr;
112                  void                  *dest_addr;
113                  fid_t                 handle;
114                  struct fi_tx_attr     *tx_attr;
115                  struct fi_rx_attr     *rx_attr;
116                  struct fi_ep_attr     *ep_attr;
117                  struct fi_domain_attr *domain_attr;
118                  struct fi_fabric_attr *fabric_attr;
119                  struct fid_nic        *nic;
120              };
121
122       next   Pointer to the next fi_info structure in the list.  Will be NULL
123              if no more structures exist.
124
125       caps - fabric interface capabilities
126              If  specified,  indicates the desired capabilities of the fabric
127              interfaces.  Supported capabilities are listed in the  Capabili‐
128              ties section below.
129
130       mode   Operational  modes  supported  by the application.  See the Mode
131              section below.
132
133       addr_format - address format
134              If specified, indicates the format of  addresses  referenced  by
135              the  fabric  interfaces  and data structures.  Supported formats
136              are listed in the Addressing formats section below.
137
138       src_addrlen - source address length
139              Indicates the length of the source address.  This value must  be
140              >  0  if  src_addr  is  non-NULL.  This field will be ignored in
141              hints if FI_SOURCE flag is set, or src_addr is NULL.
142
143       dest_addrlen - destination address length
144              Indicates the length of the  destination  address.   This  value
145              must  be  >  0 if dest_addr is non-NULL.  This field will be ig‐
146              nored in hints unless the node and service parameters  are  NULL
147              or FI_SOURCE flag is set, or if dst_addr is NULL.
148
149       src_addr - source address
150              If  specified, indicates the source address.  This field will be
151              ignored in hints if FI_SOURCE flag is set.  On output a provider
152              shall  return  an address that corresponds to the indicated fab‐
153              ric, domain, node, and/or service fields.  The format of the ad‐
154              dress is indicated by the returned addr_format field.  Note that
155              any returned address is only used when opening a local endpoint.
156              The address is not guaranteed to be usable by a peer process.
157
158       dest_addr - destination address
159              If  specified,  indicates  the  destination address.  This field
160              will be ignored in hints unless the node and service  parameters
161              are  NULL  or FI_SOURCE flag is set.  If FI_SOURCE is not speci‐
162              fied, on output a provider shall return an  address  the  corre‐
163              sponds  to the indicated node and/or service fields, relative to
164              the fabric and domain.  Note that any returned address  is  only
165              usable locally.
166
167       handle - provider context handle
168              The  use  of this field is operation specific.  If hints->handle
169              is set to struct fid_pep, the hints->handle will  be  copied  to
170              info->handle   on  output  from  fi_getinfo.   Other  values  of
171              hints->handle will be handled in  a  provider  specific  manner.
172              The  fi_info::handle  field  is  also  used by fi_endpoint() and
173              fi_reject() calls when processing connection requests or to  in‐
174              herit  another  endpoint's  attributes.   See  fi_eq(3),  fi_re‐
175              ject(3), and fi_endpoint(3).  The info->handle field will be ig‐
176              nored by fi_dupinfo and fi_freeinfo.
177
178       tx_attr - transmit context attributes
179              Optionally  supplied transmit context attributes.  Transmit con‐
180              text attributes may be specified and returned as part of fi_get‐
181              info.   When  provided  as  hints,  requested  values  of struct
182              fi_tx_ctx_attr should be set.  On output,  the  actual  transmit
183              context  attributes that can be provided will be returned.  Out‐
184              put values will be greater than or equal to the requested  input
185              values.
186
187       rx_attr - receive context attributes
188              Optionally supplied receive context attributes.  Receive context
189              attributes may be specified and returned as part of  fi_getinfo.
190              When provided as hints, requested values of struct fi_rx_ctx_at‐
191              tr should be set.  On output, the  actual  receive  context  at‐
192              tributes  that  can be provided will be returned.  Output values
193              will be greater than or or equal to the requested input values.
194
195       ep_attr - endpoint attributes
196              Optionally supplied endpoint  attributes.   Endpoint  attributes
197              may  be specified and returned as part of fi_getinfo.  When pro‐
198              vided as hints, requested values of struct fi_ep_attr should  be
199              set.  On output, the actual endpoint attributes that can be pro‐
200              vided will be returned.  Output values will be greater  than  or
201              equal  to  requested  input  values.  See fi_endpoint(3) for de‐
202              tails.
203
204       domain_attr - domain attributes
205              Optionally supplied domain attributes.  Domain attributes may be
206              specified  and returned as part of fi_getinfo.  When provided as
207              hints, requested values of struct fi_domain_attr should be  set.
208              On  output,  the  actual  domain attributes that can be provided
209              will be returned.  Output values will be greater than  or  equal
210              to requested input values.  See fi_domain(3) for details.
211
212       fabric_attr - fabric attributes
213              Optionally supplied fabric attributes.  Fabric attributes may be
214              specified and returned as part of fi_getinfo.  When provided  as
215              hints,  requested values of struct fi_fabric_attr should be set.
216              On output, the actual fabric attributes  that  can  be  provided
217              will be returned.  See fi_fabric(3) for details.
218
219       nic - network interface details
220              Optional  attributes related to the hardware NIC associated with
221              the specified fabric, domain, and endpoint data.  This field  is
222              only  valid for providers where the corresponding attributes are
223              closely associated with a hardware NIC.  See fi_nic(3)  for  de‐
224              tails.
225

CAPABILITIES

227       Interface  capabilities  are obtained by OR-ing the following flags to‐
228       gether.  If capabilities in the hint parameter are set to 0, the under‐
229       lying provider will return the set of capabilities which are supported.
230       Otherwise, providers will return data matching the specified set of ca‐
231       pabilities.  Providers may indicate support for additional capabilities
232       beyond those requested when the use of expanded capabilities  will  not
233       adversely affect performance or expose the application to communication
234       beyond that which was requested.  Applications may use this feature  to
235       request a minimal set of requirements, then check the returned capabil‐
236       ities to enable additional optimizations.
237
238       FI_ATOMIC
239              Specifies that the endpoint supports some set of  atomic  opera‐
240              tions.   Endpoints supporting this capability support operations
241              defined by struct fi_ops_atomic.  In the absence of any relevant
242              flags, FI_ATOMIC implies the ability to initiate and be the tar‐
243              get of remote atomic reads and writes.  Applications can use the
244              FI_READ,  FI_WRITE, FI_REMOTE_READ, and FI_REMOTE_WRITE flags to
245              restrict the types of atomic operations  supported  by  an  end‐
246              point.
247
248       FI_COLLECTIVE
249              Requests support for collective operations.  Endpoints that sup‐
250              port this capability support the collective  operations  defined
251              in fi_collective(3).
252
253       FI_DIRECTED_RECV
254              Requests  that the communication endpoint use the source address
255              of an incoming message when matching it with a  receive  buffer.
256              If  this  capability is not set, then the src_addr parameter for
257              msg and tagged receive operations is ignored.
258
259       FI_FENCE
260              Indicates that the endpoint support the FI_FENCE  flag  on  data
261              transfer  operations.  Support requires tracking that all previ‐
262              ous transmit requests to a specified  remote  endpoint  complete
263              prior to initiating the fenced operation.  Fenced operations are
264              often used to enforce ordering between operations that  are  not
265              otherwise guaranteed by the underlying provider or protocol.
266
267       FI_HMEM
268              Specifies that the endpoint should support transfers to and from
269              device memory.
270
271       FI_LOCAL_COMM
272              Indicates that the endpoint support  host  local  communication.
273              This  flag may be used in conjunction with FI_REMOTE_COMM to in‐
274              dicate that local and remote  communication  are  required.   If
275              neither  FI_LOCAL_COMM or FI_REMOTE_COMM are specified, then the
276              provider will indicate support for the configuration that  mini‐
277              mally affects performance.  Providers that set FI_LOCAL_COMM but
278              not FI_REMOTE_COMM, for example a shared  memory  provider,  may
279              only be used to communication between processes on the same sys‐
280              tem.
281
282       FI_MSG Specifies that an endpoint should support sending and  receiving
283              messages  or  datagrams.  Message capabilities imply support for
284              send and/or receive queues.  Endpoints supporting this capabili‐
285              ty support operations defined by struct fi_ops_msg.
286
287       The caps may be used to specify or restrict the type of messaging oper‐
288       ations that are supported.  In  the  absence  of  any  relevant  flags,
289       FI_MSG  implies the ability to send and receive messages.  Applications
290       can use the FI_SEND and  FI_RECV  flags  to  optimize  an  endpoint  as
291       send-only or receive-only.
292
293       FI_MULTICAST
294              Indicates  that  the  endpoint support multicast data transfers.
295              This capability must be paired with  FI_MSG.   Applications  can
296              use  FI_SEND  and  FI_RECV to optimize multicast as send-only or
297              receive-only.
298
299       FI_MULTI_RECV
300              Specifies that the endpoint must support the FI_MULTI_RECV  flag
301              when posting receive buffers.
302
303       FI_NAMED_RX_CTX
304              Requests  that endpoints which support multiple receive contexts
305              allow an initiator to target (or name) a specific  receive  con‐
306              text as part of a data transfer operation.
307
308       FI_READ
309              Indicates that the user requires an endpoint capable of initiat‐
310              ing reads against remote memory  regions.   This  flag  requires
311              that FI_RMA and/or FI_ATOMIC be set.
312
313       FI_RECV
314              Indicates  that the user requires an endpoint capable of receiv‐
315              ing message data transfers.  Message transfers include base mes‐
316              sage operations as well as tagged message functionality.
317
318       FI_REMOTE_COMM
319              Indicates that the endpoint support communication with endpoints
320              located at remote nodes (across the fabric).  See  FI_LOCAL_COMM
321              for  additional  details.  Providers that set FI_REMOTE_COMM but
322              not FI_LOCAL_COMM, for example NICs that lack loopback  support,
323              cannot be used to communicate with processes on the same system.
324
325       FI_REMOTE_READ
326              Indicates  that the user requires an endpoint capable of receiv‐
327              ing read memory operations from remote endpoints.  This flag re‐
328              quires that FI_RMA and/or FI_ATOMIC be set.
329
330       FI_REMOTE_WRITE
331              Indicates  that the user requires an endpoint capable of receiv‐
332              ing write memory operations from remote  endpoints.   This  flag
333              requires that FI_RMA and/or FI_ATOMIC be set.
334
335       FI_RMA Specifies  that  the  endpoint should support RMA read and write
336              operations.  Endpoints supporting this capability support opera‐
337              tions defined by struct fi_ops_rma.  In the absence of any rele‐
338              vant flags, FI_RMA implies the ability to initiate  and  be  the
339              target  of remote memory reads and writes.  Applications can use
340              the FI_READ, FI_WRITE, FI_REMOTE_READ, and FI_REMOTE_WRITE flags
341              to  restrict  the  types  of RMA operations supported by an end‐
342              point.
343
344       FI_RMA_EVENT
345              Requests that an endpoint support the generation  of  completion
346              events  when it is the target of an RMA and/or atomic operation.
347              This flag requires that FI_REMOTE_READ and/or FI_REMOTE_WRITE be
348              enabled on the endpoint.
349
350       FI_RMA_PMEM
351              Indicates  that  the  provider  is 'persistent memory aware' and
352              supports RMA operations to and from persistent memory.   Persis‐
353              tent  memory aware providers must support registration of memory
354              that is backed by non- volatile memory,  RMA  transfers  to/from
355              persistent memory, and enhanced completion semantics.  This flag
356              requires that FI_RMA be set.  This capability is experimental.
357
358       FI_SEND
359              Indicates that the user requires an endpoint capable of  sending
360              message  data transfers.  Message transfers include base message
361              operations as well as tagged message functionality.
362
363       FI_SHARED_AV
364              Requests or indicates support for address vectors which  may  be
365              shared among multiple processes.
366
367       FI_SOURCE
368              Requests that the endpoint return source addressing data as part
369              of its completion data.  This capability only applies to connec‐
370              tionless endpoints.  Note that returning source address informa‐
371              tion may require that the provider perform  address  translation
372              and/or  look-up based on data available in the underlying proto‐
373              col in order to provide the requested data, which may  adversely
374              affect  performance.   The performance impact may be greater for
375              address vectors of type FI_AV_TABLE.
376
377       FI_SOURCE_ERR
378              Must be paired with FI_SOURCE.  When  specified,  this  requests
379              that  raw  source addressing data be returned as part of comple‐
380              tion data for any address that has not been  inserted  into  the
381              local  address  vector.   Use of this capability may require the
382              provider to validate incoming source address  data  against  ad‐
383              dresses  stored in the local address vector, which may adversely
384              affect performance.
385
386       FI_TAGGED
387              Specifies that the endpoint should handle tagged message  trans‐
388              fers.   Tagged  message transfers associate a user-specified key
389              or tag with each message that is used for matching  purposes  at
390              the  remote  side.  Endpoints supporting this capability support
391              operations defined by struct fi_ops_tagged.  In the  absence  of
392              any  relevant  flags,  FI_TAGGED implies the ability to send and
393              receive tagged messages.  Applications can use the  FI_SEND  and
394              FI_RECV  flags  to  optimize  an  endpoint  as  send-only or re‐
395              ceive-only.
396
397       FI_TRIGGER
398              Indicates that the endpoint should support triggered operations.
399              Endpoints  support  this capability must meet the usage model as
400              described by fi_trigger.3.
401
402       FI_VARIABLE_MSG
403              Requests that the provider must notify a receiver when  a  vari‐
404              able  length message is ready to be received prior to attempting
405              to place the data.  Such notification will include the  size  of
406              the message and any associated message tag (for FI_TAGGED).  See
407              'Variable Length Messages' in fi_msg.3 for full details.   Vari‐
408              able  length  messages  are any messages larger than an endpoint
409              configurable  size.   This  flag  requires  that  FI_MSG  and/or
410              FI_TAGGED be set.
411
412       FI_WRITE
413              Indicates that the user requires an endpoint capable of initiat‐
414              ing writes against remote memory regions.   This  flag  requires
415              that FI_RMA and/or FI_ATOMIC be set.
416
417       Capabilities  may  be  grouped  into three general categories: primary,
418       secondary, and primary modifiers.  Primary capabilities must explicitly
419       be  requested by an application, and a provider must enable support for
420       only those primary capabilities which were selected.  Primary modifiers
421       are used to limit a primary capability, such as restricting an endpoint
422       to being send-only.  If no modifiers are specified  for  an  applicable
423       capability,  all relevant modifiers are assumed.  See above definitions
424       for details.
425
426       Secondary capabilities may optionally be requested by  an  application.
427       If  requested,  a  provider  must  support  the  capability or fail the
428       fi_getinfo request (FI_ENODATA).   A  provider  may  optionally  report
429       non-selected  secondary  capabilities  if doing so would not compromise
430       performance or security.
431
432       Primary capabilities: FI_MSG, FI_RMA, FI_TAGGED,  FI_ATOMIC,  FI_MULTI‐
433       CAST,   FI_NAMED_RX_CTX,  FI_DIRECTED_RECV,  FI_VARIABLE_MSG,  FI_HMEM,
434       FI_COLLECTIVE
435
436       Primary modifiers: FI_READ, FI_WRITE, FI_RECV, FI_SEND, FI_REMOTE_READ,
437       FI_REMOTE_WRITE
438
439       Secondary   capabilities:   FI_MULTI_RECV,   FI_SOURCE,   FI_RMA_EVENT,
440       FI_SHARED_AV,  FI_TRIGGER,  FI_FENCE,  FI_LOCAL_COMM,   FI_REMOTE_COMM,
441       FI_SOURCE_ERR, FI_RMA_PMEM.
442

MODE

444       The  operational  mode bits are used to convey requirements that an ap‐
445       plication must adhere to when using the fabric interfaces.  Modes spec‐
446       ify  optimal ways of accessing the reported endpoint or domain.  Appli‐
447       cations that are designed to support a specific mode of  operation  may
448       see improved performance when that mode is desired by the provider.  It
449       is recommended that providers support  applications  that  disable  any
450       provider preferred modes.
451
452       On  input  to fi_getinfo, applications set the mode bits that they sup‐
453       port.  On output, providers will clear mode bits that are not necessary
454       to achieve high-performance.  Mode bits that remain set indicate appli‐
455       cation requirements for using the fabric interfaces created  using  the
456       returned  fi_info.  The set of modes are listed below.  If a NULL hints
457       structure is provided, then the provider's supported set of modes  will
458       be returned in the info structure(s).
459
460       FI_ASYNC_IOV
461              Applications  can  reference  multiple data buffers as part of a
462              single operation through the use of IO vectors (SGEs).  Typical‐
463              ly, the contents of an IO vector are copied by the provider into
464              an internal buffer area, or directly to the underlying hardware.
465              However,  when  a large number of IOV entries are supported, IOV
466              buffering may have a negative impact on performance  and  memory
467              consumption.   The FI_ASYNC_IOV mode indicates that the applica‐
468              tion must provide the buffering needed for the IO vectors.  When
469              set,  an application must not modify an IO vector of length > 1,
470              including any related memory descriptor array, until the associ‐
471              ated operation has completed.
472
473       FI_BUFFERED_RECV
474              The  buffered  receive mode bit indicates that the provider owns
475              the data buffer(s) that are accessed by the networking layer for
476              received  messages.   Typically,  this implies that data must be
477              copied from the provider buffer  into  the  application  buffer.
478              Applications that can handle message processing from network al‐
479              located data buffers can set this mode bit to avoid copies.  For
480              full  details  on application requirements to support this mode,
481              see the 'Buffered Receives' section in fi_msg(3).  This mode bit
482              applies to FI_MSG and FI_TAGGED receive operations.
483
484       FI_CONTEXT
485              Specifies  that  the  provider  requires  that  applications use
486              struct fi_context as their per operation context  parameter  for
487              operations  that  generated  full  completions.   This structure
488              should be treated as opaque to the application.  For performance
489              reasons,  this  structure must be allocated by the user, but may
490              be used by the fabric provider to track the operation.  Typical‐
491              ly,  users  embed  struct  fi_context  within  their own context
492              structure.  The struct fi_context must remain  valid  until  the
493              corresponding  operation  completes or is successfully canceled.
494              As such, fi_context should NOT be allocated on the stack.  Doing
495              so  is  likely to result in stack corruption that will be diffi‐
496              cult to debug.  Users should not update or interpret the  fields
497              in  this structure, or reuse it until the original operation has
498              completed.  If an operation does not generate a completion (i.e.
499              the endpoint was configured with FI_SELECTIVE_COMPLETION and the
500              operation was not initiated with the  FI_COMPLETION  flag)  then
501              the  context  parameter  is ignored by the fabric provider.  The
502              structure is specified in rdma/fabric.h.
503
504       FI_CONTEXT2
505              This bit is similar to FI_CONTEXT, but  doubles  the  provider's
506              requirement on the size of the per context structure.  When set,
507              this specifies that the provider requires that applications  use
508              struct  fi_context2  as  their  per operation context parameter.
509              Or, optionally, an application  can  provide  an  array  of  two
510              fi_context structures (e.g.  struct fi_context[2]) instead.  The
511              requirements for using struct fi_context2 are identical  as  de‐
512              fined for FI_CONTEXT above.
513
514       FI_LOCAL_MR
515              The  provider  is  optimized around having applications register
516              memory for locally accessed data buffers.  Data buffers used  in
517              send and receive operations and as the source buffer for RMA and
518              atomic operations must be registered by the application for  ac‐
519              cess  domains opened with this capability.  This flag is defined
520              for compatibility and is ignored if the application  version  is
521              1.5  or  later  and  the domain mr_mode is set to anything other
522              than FI_MR_BASIC or FI_MR_SCALABLE.  See  the  domain  attribute
523              mr_mode fi_domain(3) and fi_mr(3).
524
525       FI_MSG_PREFIX
526              Message  prefix  mode indicates that an application will provide
527              buffer space in front of all message send  and  receive  buffers
528              for  use  by  the  provider.   Typically, the provider uses this
529              space to implement a protocol, with the protocol  headers  being
530              written  into the prefix area.  The contents of the prefix space
531              should be treated as opaque.  The use of FI_MSG_PREFIX  may  im‐
532              prove application performance over certain providers by reducing
533              the number of IO vectors referenced by underlying  hardware  and
534              eliminating provider buffer allocation.
535
536       FI_MSG_PREFIX  only  applies  to send and receive operations, including
537       tagged sends and receives.  RMA and atomics do not require the applica‐
538       tion  to  provide prefix buffers.  Prefix buffer space must be provided
539       with all sends and receives, regardless of the size of the transfer  or
540       other transfer options.  The ownership of prefix buffers is treated the
541       same as the corresponding message buffers, but the size of  the  prefix
542       buffer is not counted toward any message limits, including inject.
543
544       Applications  that  support prefix mode must supply buffer space before
545       their own message data.  The size of space that  must  be  provided  is
546       specified by the msg_prefix_size endpoint attribute.  Providers are re‐
547       quired to define a msg_prefix_size that is a multiple of 8 bytes.   Ad‐
548       ditionally, applications may receive provider generated packets that do
549       not contain application data.  Such received messages will  indicate  a
550       transfer size of that is equal to or smaller than msg_prefix_size.
551
552       The  buffer pointer given to all send and receive operations must point
553       to the start of the prefix region of the buffer (as opposed to the pay‐
554       load).  For scatter-gather send/recv operations, the prefix buffer must
555       be a contiguous region, though it may or may not be  directly  adjacent
556       to the payload portion of the buffer.
557
558       FI_NOTIFY_FLAGS_ONLY
559              This  bit indicates that general completion flags may not be set
560              by the provider, and are not  needed  by  the  application.   If
561              specified,  completion flags which simply report the type of op‐
562              eration that completed (e.g.  send or receive) may not  be  set.
563              However, completion flags that are used for remote notifications
564              will still be set when applicable.  See fi_cq(3) for details  on
565              which completion flags are valid when this mode bit is enabled.
566
567       FI_RESTRICTED_COMP
568              This  bit indicates that the application will only share comple‐
569              tion queues and counters among endpoints, transmit contexts, and
570              receive contexts that have the same set of capability flags.
571
572       FI_RX_CQ_DATA
573              This  mode  bit  only  applies to data transfers that set FI_RE‐
574              MOTE_CQ_DATA.  When set, a data transfer that carries remote  CQ
575              data  will consume a receive buffer at the target.  This is true
576              even for operations that would normally not consume  posted  re‐
577              ceive buffers, such as RMA write operations.
578

ADDRESSING FORMATS

580       Multiple fabric interfaces take as input either a source or destination
581       address parameter.  This includes struct fi_info (src_addr and dest_ad‐
582       dr),  CM  calls  (getname,  getpeer,  connect, join, and leave), and AV
583       calls (insert, lookup, and straddr).  The fi_info addr_format field in‐
584       dicates the expected address format for these operations.
585
586       A provider may support one or more of the following addressing formats.
587       In some cases, a selected addressing format may need to  be  translated
588       or mapped into an address which is native to the fabric.  See fi_av(3).
589
590       FI_ADDR_BGQ
591              Address  is  an  IBM  proprietary format that is used with their
592              Blue Gene Q systems.
593
594       FI_ADDR_EFA
595              Address is an Amazon Elastic Fabric  Adapter  (EFA)  proprietary
596              format.
597
598       FI_ADDR_GNI
599              Address is a Cray proprietary format that is used with their GNI
600              protocol.
601
602       FI_ADDR_PSMX
603              Address is an Intel proprietary format used with  their  Perfor‐
604              mance Scaled Messaging protocol.
605
606       FI_ADDR_PSMX2
607              Address  is  an Intel proprietary format used with their Perfor‐
608              mance Scaled Messaging protocol version 2.
609
610       FI_ADDR_PSMX3
611              Address is an Intel proprietary format used with  their  Perfor‐
612              mance Scaled Messaging protocol version 3.
613
614       FI_ADDR_STR
615              Address is a formatted character string.  The length and content
616              of the string is address and/or provider specific, but in gener‐
617              al follows a URI model:
618
619              address_format[://[node][:[service][/[field3]...][?[key=value][&k2=v2]...]]]
620
621       Examples:      -     fi_sockaddr://10.31.6.12:7471     -     fi_sockad‐
622       dr_in6://[fe80::6:12]:7471 - fi_sockaddr://10.31.6.12:7471?qos=3
623
624       Since the string formatted address does not contain any provider infor‐
625       mation, the prov_name field of the fabric attribute structure should be
626       used to filter by provider if necessary.
627
628       FI_FORMAT_UNSPEC
629              FI_FORMAT_UNSPEC indicates that a provider specific address for‐
630              mat should be selected.  Provider specific addresses may be pro‐
631              tocol specific or a  vendor  proprietary  format.   Applications
632              that  select  FI_FORMAT_UNSPEC  should  be prepared to treat re‐
633              turned addressing data as opaque.  FI_FORMAT_UNSPEC targets apps
634              which make use of an out of band address exchange.  Applications
635              which use FI_FORMAT_UNSPEC may  use  fi_getname()  to  obtain  a
636              provider specific address assigned to an allocated endpoint.
637
638       FI_SOCKADDR
639              Address is of type sockaddr.  The specific socket address format
640              will be determined at  run  time  by  interfaces  examining  the
641              sa_family field.
642
643       FI_SOCKADDR_IB
644              Address is of type sockaddr_ib (defined in Linux kernel source)
645
646       FI_SOCKADDR_IN
647              Address is of type sockaddr_in (IPv4).
648
649       FI_SOCKADDR_IN6
650              Address is of type sockaddr_in6 (IPv6).
651
652       FI_ADDR_PSMX
653              Address  is  an Intel proprietary format that is used with their
654              PSMX (extended performance scaled messaging) protocol.
655

FLAGS

657       The operation of the fi_getinfo call may be controlled through the  use
658       of input flags.  Valid flags include the following.
659
660       FI_NUMERICHOST
661              Indicates  that the node parameter is a numeric string represen‐
662              tation of a fabric address, such as a dotted decimal IP address.
663              Use  of this flag will suppress any lengthy name resolution pro‐
664              tocol.
665
666       FI_PROV_ATTR_ONLY
667              Indicates that the caller is only querying  for  what  providers
668              are  potentially  available.   All providers will return exactly
669              one fi_info struct, regardless of whether that provider  is  us‐
670              able  on  the  current  platform  or  not.  The returned fi_info
671              struct will contain default values for all members, with the ex‐
672              ception  of  fabric_attr.   The fabric_attr member will have the
673              prov_name and prov_version values filled in.
674
675       FI_SOURCE
676              Indicates that the node and service parameters specify the local
677              source address to associate with an endpoint.  If specified, ei‐
678              ther the node and/or service parameter must be  non-NULL.   This
679              flag is often used with passive endpoints.
680

RETURN VALUE

682       fi_getinfo()  returns  0  on success.  On error, fi_getinfo() returns a
683       negative value corresponding to fabric errno.  Fabric errno values  are
684       defined in rdma/fi_errno.h.
685
686       fi_allocinfo() returns a pointer to a new fi_info structure on success,
687       or NULL on error.  fi_dupinfo() duplicates a single  fi_info  structure
688       and  all  the  substructures  within it, returning a pointer to the new
689       fi_info structure on success, or NULL on  error.   Both  calls  require
690       that the returned fi_info structure be freed via fi_freeinfo().
691

ERRORS

693       FI_EBADFLAGS
694              The  specified  endpoint or domain capability or operation flags
695              are invalid.
696
697       FI_ENODATA
698              Indicates that no providers could be found which support the re‐
699              quested fabric information.
700
701       FI_ENOMEM
702              Indicates that there was insufficient memory to complete the op‐
703              eration.
704

NOTES

706       If hints are provided, the operation will be controlled by  the  values
707       that  are supplied in the various fields (see section on fi_info).  Ap‐
708       plications that require specific communication interfaces, domains, ca‐
709       pabilities  or  other  requirements,  can  specify them using fields in
710       hints.  Libfabric returns a linked list in info that points to  a  list
711       of matching interfaces.  info is set to NULL if there are no communica‐
712       tion interfaces or none match the input hints.
713
714       If node is provided, fi_getinfo will attempt to resolve the fabric  ad‐
715       dress  to the given node.  If node is not provided, fi_getinfo will at‐
716       tempt to resolve the fabric addressing information based on the provid‐
717       ed  hints.   The caller must call fi_freeinfo to release fi_info struc‐
718       tures returned by fi_getinfo.
719
720       If neither node, service or hints are provided, then fi_getinfo  simply
721       returns the list all available communication interfaces.
722
723       Multiple  threads  may  call fi_getinfo simultaneously, without any re‐
724       quirement for serialization.
725

SEE ALSO

727       fi_open(3), fi_endpoint(3), fi_domain(3), fi_nic(3)
728

AUTHORS

730       OpenFabrics.
731
732
733
734Libfabric Programmer's Manual     2021-02-10                     fi_getinfo(3)
Impressum