1fi_getinfo(3) Libfabric v1.15.1 fi_getinfo(3)
2
6 fi_getinfo, fi_freeinfo - Obtain / free fabric interface information
7 fi_allocinfo, fi_dupinfo - Allocate / duplicate an fi_info structure
9
11 #include <rdma/fabric.h>
12 int fi_getinfo(int version, const char *node, const char *service,
14 uint64_t flags, const struct fi_info *hints, struct fi_info **info);
15 void fi_freeinfo(struct fi_info *info);
17 struct fi_info *fi_allocinfo(void);
19 struct fi_info *fi_dupinfo(const struct fi_info *info);
21
23 version
24 Interface version requested by application.
25 node Optional, name or fabric address to resolve.
27 service
29 Optional, service name or port number of address.
30 flags Operation flags for the fi_getinfo call.
32 hints Reference to an fi_info structure that specifies criteria for
34 selecting the returned fabric information.
35 info A pointer to a linked list of fi_info structures containing re‐
37 sponse information.
38
40 fi_getinfo returns information about available fabric services for
41 reaching specified node or service, subject to any provided hints.
42 Callers may specify NULL for node, service, and hints in order to re‐
43 trieve information about what providers are available and their optimal
44 usage models. If no matching fabric information is available, info
45 will be set to NULL and the call will return -FI_ENODATA.
46 Based on the input hints, node, and service parameters, a list of fab‐
48 ric domains and endpoints will be returned. Each fi_info structure
49 will describe an endpoint that meets the application’s specified commu‐
50 nication criteria. Each endpoint will be associated with a domain.
51 Applications can restrict the number of returned endpoints by including
52 additional criteria in their search hints. Relaxing or eliminating in‐
53 put hints will increase the number and type of endpoints that are
54 available. Providers that return multiple endpoints to a single
55 fi_getinfo call should return the endpoints that are highest performing
56 first. Providers may indicate that an endpoint and domain can support
57 additional capabilities than those requested by the user only if such
58 support will not adversely affect application performance or security.
59 The version parameter is used by the application to request the desired
61 version of the interfaces. The version determines the format of all
62 data structures used by any of the fabric interfaces. Applications
63 should use the FI_VERSION(major, minor) macro to indicate the version,
64 with hard-coded integer values for the major and minor values. The
65 FI_MAJOR_VERSION and FI_MINOR_VERSION enum values defined in fabric.h
66 specify the latest version of the installed library. However, it is
67 recommended that the integer values for FI_MAJOR_VERSION and FI_MI‐
68 NOR_VERSION be used, rather than referencing the enum types in order to
69 ensure compatibility with future versions of the library. This pro‐
70 tects against the application being built from source against a newer
71 version of the library that introduces new fields to data structures,
72 which would not be initialized by the application.
73 Node, service, or hints may be provided, with any combination being
75 supported. If node is provided, fi_getinfo will attempt to resolve the
76 fabric address to the given node. If node is not given, fi_getinfo
77 will attempt to resolve the fabric addressing information based on the
78 provided hints. Node is commonly used to provide a network address
79 (such as an IP address) or hostname. Service is usually associated
80 with a transport address (such as a TCP port number). Node and service
81 parameters may be mapped by providers to native fabric addresses. Ap‐
82 plications may also pass in an FI_ADDR_STR formatted address (see for‐
83 mat details below) as the node parameter. In such cases, the service
84 parameter must be NULL.
85 The hints parameter, if provided, may be used to limit the resulting
87 output as indicated below. As a general rule, specifying a non-zero
88 value for input hints indicates that a provider must support the re‐
89 quested value or fail the operation with -FI_ENODATA. With the excep‐
90 tion of mode bits, hints that are set to zero are treated as a wild‐
91 card. A zeroed hint value results in providers either returning a de‐
92 fault value or a value that works best for their implementation. Mode
93 bits that are set to zero indicate the application does not support any
94 modes.
95 The caller must call fi_freeinfo to release fi_info structures returned
97 by this call.
98 The fi_allocinfo call will allocate and zero an fi_info structure and
100 all related substructures. The fi_dupinfo will duplicate a single
101 fi_info structure and all the substructures within it.
102
104 struct fi_info {
105 struct fi_info *next;
106 uint64_t caps;
107 uint64_t mode;
108 uint32_t addr_format;
109 size_t src_addrlen;
110 size_t dest_addrlen;
111 void *src_addr;
112 void *dest_addr;
113 fid_t handle;
114 struct fi_tx_attr *tx_attr;
115 struct fi_rx_attr *rx_attr;
116 struct fi_ep_attr *ep_attr;
117 struct fi_domain_attr *domain_attr;
118 struct fi_fabric_attr *fabric_attr;
119 struct fid_nic *nic;
120 };
121 next Pointer to the next fi_info structure in the list. Will be NULL
123 if no more structures exist.
124 caps - fabric interface capabilities
126 If specified, indicates the desired capabilities of the fabric
127 interfaces. Supported capabilities are listed in the Capabili‐
128 ties section below.
129 mode Operational modes supported by the application. See the Mode
131 section below.
132 addr_format - address format
134 If specified, indicates the format of addresses referenced by
135 the fabric interfaces and data structures. Supported formats
136 are listed in the Addressing formats section below.
137 src_addrlen - source address length
139 Indicates the length of the source address. This value must be
140 > 0 if src_addr is non-NULL. This field will be ignored in
141 hints if FI_SOURCE flag is set, or src_addr is NULL.
142 dest_addrlen - destination address length
144 Indicates the length of the destination address. This value
145 must be > 0 if dest_addr is non-NULL. This field will be ig‐
146 nored in hints unless the node and service parameters are NULL
147 or FI_SOURCE flag is set, or if dst_addr is NULL.
148 src_addr - source address
150 If specified, indicates the source address. This field will be
151 ignored in hints if FI_SOURCE flag is set. On output a provider
152 shall return an address that corresponds to the indicated fab‐
153 ric, domain, node, and/or service fields. The format of the ad‐
154 dress is indicated by the returned addr_format field. Note that
155 any returned address is only used when opening a local endpoint.
156 The address is not guaranteed to be usable by a peer process.
157 dest_addr - destination address
159 If specified, indicates the destination address. This field
160 will be ignored in hints unless the node and service parameters
161 are NULL or FI_SOURCE flag is set. If FI_SOURCE is not speci‐
162 fied, on output a provider shall return an address the corre‐
163 sponds to the indicated node and/or service fields, relative to
164 the fabric and domain. Note that any returned address is only
165 usable locally.
166 handle - provider context handle
168 The use of this field is operation specific. If hints->handle
169 is set to struct fid_pep, the hints->handle will be copied to
170 info->handle on output from fi_getinfo. Other values of
171 hints->handle will be handled in a provider specific manner.
172 The fi_info::handle field is also used by fi_endpoint() and
173 fi_reject() calls when processing connection requests or to in‐
174 herit another endpoint’s attributes. See fi_eq(3), fi_re‐
175 ject(3), and fi_endpoint(3). The info->handle field will be ig‐
176 nored by fi_dupinfo and fi_freeinfo.
177 tx_attr - transmit context attributes
179 Optionally supplied transmit context attributes. Transmit con‐
180 text attributes may be specified and returned as part of fi_get‐
181 info. When provided as hints, requested values of struct
182 fi_tx_ctx_attr should be set. On output, the actual transmit
183 context attributes that can be provided will be returned. Out‐
184 put values will be greater than or equal to the requested input
185 values.
186 rx_attr - receive context attributes
188 Optionally supplied receive context attributes. Receive context
189 attributes may be specified and returned as part of fi_getinfo.
190 When provided as hints, requested values of struct fi_rx_ctx_at‐
191 tr should be set. On output, the actual receive context at‐
192 tributes that can be provided will be returned. Output values
193 will be greater than or or equal to the requested input values.
194 ep_attr - endpoint attributes
196 Optionally supplied endpoint attributes. Endpoint attributes
197 may be specified and returned as part of fi_getinfo. When pro‐
198 vided as hints, requested values of struct fi_ep_attr should be
199 set. On output, the actual endpoint attributes that can be pro‐
200 vided will be returned. Output values will be greater than or
201 equal to requested input values. See fi_endpoint(3) for de‐
202 tails.
203 domain_attr - domain attributes
205 Optionally supplied domain attributes. Domain attributes may be
206 specified and returned as part of fi_getinfo. When provided as
207 hints, requested values of struct fi_domain_attr should be set.
208 On output, the actual domain attributes that can be provided
209 will be returned. Output values will be greater than or equal
210 to requested input values. See fi_domain(3) for details.
211 fabric_attr - fabric attributes
213 Optionally supplied fabric attributes. Fabric attributes may be
214 specified and returned as part of fi_getinfo. When provided as
215 hints, requested values of struct fi_fabric_attr should be set.
216 On output, the actual fabric attributes that can be provided
217 will be returned. See fi_fabric(3) for details.
218 nic - network interface details
220 Optional attributes related to the hardware NIC associated with
221 the specified fabric, domain, and endpoint data. This field is
222 only valid for providers where the corresponding attributes are
223 closely associated with a hardware NIC. See fi_nic(3) for de‐
224 tails.
225
227 Interface capabilities are obtained by OR-ing the following flags to‐
228 gether. If capabilities in the hint parameter are set to 0, the under‐
229 lying provider will return the set of capabilities which are supported.
230 Otherwise, providers will return data matching the specified set of ca‐
231 pabilities. Providers may indicate support for additional capabilities
232 beyond those requested when the use of expanded capabilities will not
233 adversely affect performance or expose the application to communication
234 beyond that which was requested. Applications may use this feature to
235 request a minimal set of requirements, then check the returned capabil‐
236 ities to enable additional optimizations.
237 FI_ATOMIC
239 Specifies that the endpoint supports some set of atomic opera‐
240 tions. Endpoints supporting this capability support operations
241 defined by struct fi_ops_atomic. In the absence of any relevant
242 flags, FI_ATOMIC implies the ability to initiate and be the tar‐
243 get of remote atomic reads and writes. Applications can use the
244 FI_READ, FI_WRITE, FI_REMOTE_READ, and FI_REMOTE_WRITE flags to
245 restrict the types of atomic operations supported by an end‐
246 point.
247 FI_COLLECTIVE
249 Requests support for collective operations. Endpoints that sup‐
250 port this capability support the collective operations defined
251 in fi_collective(3).
252 FI_DIRECTED_RECV
254 Requests that the communication endpoint use the source address
255 of an incoming message when matching it with a receive buffer.
256 If this capability is not set, then the src_addr parameter for
257 msg and tagged receive operations is ignored.
258 FI_FENCE
260 Indicates that the endpoint support the FI_FENCE flag on data
261 transfer operations. Support requires tracking that all previ‐
262 ous transmit requests to a specified remote endpoint complete
263 prior to initiating the fenced operation. Fenced operations are
264 often used to enforce ordering between operations that are not
265 otherwise guaranteed by the underlying provider or protocol.
266 FI_HMEM
268 Specifies that the endpoint should support transfers to and from
269 device memory.
270 FI_LOCAL_COMM
272 Indicates that the endpoint support host local communication.
273 This flag may be used in conjunction with FI_REMOTE_COMM to in‐
274 dicate that local and remote communication are required. If
275 neither FI_LOCAL_COMM or FI_REMOTE_COMM are specified, then the
276 provider will indicate support for the configuration that mini‐
277 mally affects performance. Providers that set FI_LOCAL_COMM but
278 not FI_REMOTE_COMM, for example a shared memory provider, may
279 only be used to communication between processes on the same sys‐
280 tem.
281 FI_MSG Specifies that an endpoint should support sending and receiving
283 messages or datagrams. Message capabilities imply support for
284 send and/or receive queues. Endpoints supporting this capabili‐
285 ty support operations defined by struct fi_ops_msg.
286 The caps may be used to specify or restrict the type of messaging oper‐
288 ations that are supported. In the absence of any relevant flags,
289 FI_MSG implies the ability to send and receive messages. Applications
290 can use the FI_SEND and FI_RECV flags to optimize an endpoint as
291 send-only or receive-only.
292 FI_MULTICAST
294 Indicates that the endpoint support multicast data transfers.
295 This capability must be paired with FI_MSG. Applications can
296 use FI_SEND and FI_RECV to optimize multicast as send-only or
297 receive-only.
298 FI_MULTI_RECV
300 Specifies that the endpoint must support the FI_MULTI_RECV flag
301 when posting receive buffers.
302 FI_NAMED_RX_CTX
304 Requests that endpoints which support multiple receive contexts
305 allow an initiator to target (or name) a specific receive con‐
306 text as part of a data transfer operation.
307 FI_READ
309 Indicates that the user requires an endpoint capable of initiat‐
310 ing reads against remote memory regions. This flag requires
311 that FI_RMA and/or FI_ATOMIC be set.
312 FI_RECV
314 Indicates that the user requires an endpoint capable of receiv‐
315 ing message data transfers. Message transfers include base mes‐
316 sage operations as well as tagged message functionality.
317 FI_REMOTE_COMM
319 Indicates that the endpoint support communication with endpoints
320 located at remote nodes (across the fabric). See FI_LOCAL_COMM
321 for additional details. Providers that set FI_REMOTE_COMM but
322 not FI_LOCAL_COMM, for example NICs that lack loopback support,
323 cannot be used to communicate with processes on the same system.
324 FI_REMOTE_READ
326 Indicates that the user requires an endpoint capable of receiv‐
327 ing read memory operations from remote endpoints. This flag re‐
328 quires that FI_RMA and/or FI_ATOMIC be set.
329 FI_REMOTE_WRITE
331 Indicates that the user requires an endpoint capable of receiv‐
332 ing write memory operations from remote endpoints. This flag
333 requires that FI_RMA and/or FI_ATOMIC be set.
334 FI_RMA Specifies that the endpoint should support RMA read and write
336 operations. Endpoints supporting this capability support opera‐
337 tions defined by struct fi_ops_rma. In the absence of any rele‐
338 vant flags, FI_RMA implies the ability to initiate and be the
339 target of remote memory reads and writes. Applications can use
340 the FI_READ, FI_WRITE, FI_REMOTE_READ, and FI_REMOTE_WRITE flags
341 to restrict the types of RMA operations supported by an end‐
342 point.
343 FI_RMA_EVENT
345 Requests that an endpoint support the generation of completion
346 events when it is the target of an RMA and/or atomic operation.
347 This flag requires that FI_REMOTE_READ and/or FI_REMOTE_WRITE be
348 enabled on the endpoint.
349 FI_RMA_PMEM
351 Indicates that the provider is `persistent memory aware' and
352 supports RMA operations to and from persistent memory. Persis‐
353 tent memory aware providers must support registration of memory
354 that is backed by non- volatile memory, RMA transfers to/from
355 persistent memory, and enhanced completion semantics. This flag
356 requires that FI_RMA be set. This capability is experimental.
357 FI_SEND
359 Indicates that the user requires an endpoint capable of sending
360 message data transfers. Message transfers include base message
361 operations as well as tagged message functionality.
362 FI_SHARED_AV
364 Requests or indicates support for address vectors which may be
365 shared among multiple processes.
366 FI_SOURCE
368 Requests that the endpoint return source addressing data as part
369 of its completion data. This capability only applies to connec‐
370 tionless endpoints. Note that returning source address informa‐
371 tion may require that the provider perform address translation
372 and/or look-up based on data available in the underlying proto‐
373 col in order to provide the requested data, which may adversely
374 affect performance. The performance impact may be greater for
375 address vectors of type FI_AV_TABLE.
376 FI_SOURCE_ERR
378 Must be paired with FI_SOURCE. When specified, this requests
379 that raw source addressing data be returned as part of comple‐
380 tion data for any address that has not been inserted into the
381 local address vector. Use of this capability may require the
382 provider to validate incoming source address data against ad‐
383 dresses stored in the local address vector, which may adversely
384 affect performance.
385 FI_TAGGED
387 Specifies that the endpoint should handle tagged message trans‐
388 fers. Tagged message transfers associate a user-specified key
389 or tag with each message that is used for matching purposes at
390 the remote side. Endpoints supporting this capability support
391 operations defined by struct fi_ops_tagged. In the absence of
392 any relevant flags, FI_TAGGED implies the ability to send and
393 receive tagged messages. Applications can use the FI_SEND and
394 FI_RECV flags to optimize an endpoint as send-only or re‐
395 ceive-only.
396 FI_TRIGGER
398 Indicates that the endpoint should support triggered operations.
399 Endpoints support this capability must meet the usage model as
400 described by fi_trigger(3).
401 FI_VARIABLE_MSG
403 Requests that the provider must notify a receiver when a vari‐
404 able length message is ready to be received prior to attempting
405 to place the data. Such notification will include the size of
406 the message and any associated message tag (for FI_TAGGED). See
407 `Variable Length Messages' in fi_msg.3 for full details. Vari‐
408 able length messages are any messages larger than an endpoint
409 configurable size. This flag requires that FI_MSG and/or
410 FI_TAGGED be set.
411 FI_WRITE
413 Indicates that the user requires an endpoint capable of initiat‐
414 ing writes against remote memory regions. This flag requires
415 that FI_RMA and/or FI_ATOMIC be set.
416 FI_XPU Specifies that the endpoint should support transfers that may be
418 initiated from heterogenous computation devices, such as GPUs.
419 This flag requires that FI_TRIGGER be set. For additional de‐
420 tails on XPU triggers see fi_trigger(3).
421 Capabilities may be grouped into three general categories: primary,
423 secondary, and primary modifiers. Primary capabilities must explicitly
424 be requested by an application, and a provider must enable support for
425 only those primary capabilities which were selected. Primary modifiers
426 are used to limit a primary capability, such as restricting an endpoint
427 to being send-only. If no modifiers are specified for an applicable
428 capability, all relevant modifiers are assumed. See above definitions
429 for details.
430 Secondary capabilities may optionally be requested by an application.
432 If requested, a provider must support the capability or fail the
433 fi_getinfo request (FI_ENODATA). A provider may optionally report
434 non-selected secondary capabilities if doing so would not compromise
435 performance or security.
436 Primary capabilities: FI_MSG, FI_RMA, FI_TAGGED, FI_ATOMIC, FI_MULTI‐
438 CAST, FI_NAMED_RX_CTX, FI_DIRECTED_RECV, FI_VARIABLE_MSG, FI_HMEM,
439 FI_COLLECTIVE, FI_XPU
440 Primary modifiers: FI_READ, FI_WRITE, FI_RECV, FI_SEND, FI_REMOTE_READ,
442 FI_REMOTE_WRITE
443 Secondary capabilities: FI_MULTI_RECV, FI_SOURCE, FI_RMA_EVENT,
445 FI_SHARED_AV, FI_TRIGGER, FI_FENCE, FI_LOCAL_COMM, FI_REMOTE_COMM,
446 FI_SOURCE_ERR, FI_RMA_PMEM.
447
449 The operational mode bits are used to convey requirements that an ap‐
450 plication must adhere to when using the fabric interfaces. Modes spec‐
451 ify optimal ways of accessing the reported endpoint or domain. Appli‐
452 cations that are designed to support a specific mode of operation may
453 see improved performance when that mode is desired by the provider. It
454 is recommended that providers support applications that disable any
455 provider preferred modes.
456 On input to fi_getinfo, applications set the mode bits that they sup‐
458 port. On output, providers will clear mode bits that are not necessary
459 to achieve high-performance. Mode bits that remain set indicate appli‐
460 cation requirements for using the fabric interfaces created using the
461 returned fi_info. The set of modes are listed below. If a NULL hints
462 structure is provided, then the provider’s supported set of modes will
463 be returned in the info structure(s).
464 FI_ASYNC_IOV
466 Applications can reference multiple data buffers as part of a
467 single operation through the use of IO vectors (SGEs). Typical‐
468 ly, the contents of an IO vector are copied by the provider into
469 an internal buffer area, or directly to the underlying hardware.
470 However, when a large number of IOV entries are supported, IOV
471 buffering may have a negative impact on performance and memory
472 consumption. The FI_ASYNC_IOV mode indicates that the applica‐
473 tion must provide the buffering needed for the IO vectors. When
474 set, an application must not modify an IO vector of length > 1,
475 including any related memory descriptor array, until the associ‐
476 ated operation has completed.
477 FI_BUFFERED_RECV
479 The buffered receive mode bit indicates that the provider owns
480 the data buffer(s) that are accessed by the networking layer for
481 received messages. Typically, this implies that data must be
482 copied from the provider buffer into the application buffer.
483 Applications that can handle message processing from network al‐
484 located data buffers can set this mode bit to avoid copies. For
485 full details on application requirements to support this mode,
486 see the `Buffered Receives' section in fi_msg(3). This mode bit
487 applies to FI_MSG and FI_TAGGED receive operations.
488 FI_CONTEXT
490 Specifies that the provider requires that applications use
491 struct fi_context as their per operation context parameter for
492 operations that generated full completions. This structure
493 should be treated as opaque to the application. For performance
494 reasons, this structure must be allocated by the user, but may
495 be used by the fabric provider to track the operation. Typical‐
496 ly, users embed struct fi_context within their own context
497 structure. The struct fi_context must remain valid until the
498 corresponding operation completes or is successfully canceled.
499 As such, fi_context should NOT be allocated on the stack. Doing
500 so is likely to result in stack corruption that will be diffi‐
501 cult to debug. Users should not update or interpret the fields
502 in this structure, or reuse it until the original operation has
503 completed. If an operation does not generate a completion
504 (i.e. the endpoint was configured with FI_SELECTIVE_COMPLETION
505 and the operation was not initiated with the FI_COMPLETION flag)
506 then the context parameter is ignored by the fabric provider.
507 The structure is specified in rdma/fabric.h.
508 FI_CONTEXT2
510 This bit is similar to FI_CONTEXT, but doubles the provider’s
511 requirement on the size of the per context structure. When set,
512 this specifies that the provider requires that applications use
513 struct fi_context2 as their per operation context parameter.
514 Or, optionally, an application can provide an array of two
515 fi_context structures (e.g. struct fi_context[2]) instead. The
516 requirements for using struct fi_context2 are identical as de‐
517 fined for FI_CONTEXT above.
518 FI_LOCAL_MR
520 The provider is optimized around having applications register
521 memory for locally accessed data buffers. Data buffers used in
522 send and receive operations and as the source buffer for RMA and
523 atomic operations must be registered by the application for ac‐
524 cess domains opened with this capability. This flag is defined
525 for compatibility and is ignored if the application version is
526 1.5 or later and the domain mr_mode is set to anything other
527 than FI_MR_BASIC or FI_MR_SCALABLE. See the domain attribute
528 mr_mode fi_domain(3) and fi_mr(3).
529 FI_MSG_PREFIX
531 Message prefix mode indicates that an application will provide
532 buffer space in front of all message send and receive buffers
533 for use by the provider. Typically, the provider uses this
534 space to implement a protocol, with the protocol headers being
535 written into the prefix area. The contents of the prefix space
536 should be treated as opaque. The use of FI_MSG_PREFIX may im‐
537 prove application performance over certain providers by reducing
538 the number of IO vectors referenced by underlying hardware and
539 eliminating provider buffer allocation.
540 FI_MSG_PREFIX only applies to send and receive operations, including
542 tagged sends and receives. RMA and atomics do not require the applica‐
543 tion to provide prefix buffers. Prefix buffer space must be provided
544 with all sends and receives, regardless of the size of the transfer or
545 other transfer options. The ownership of prefix buffers is treated the
546 same as the corresponding message buffers, but the size of the prefix
547 buffer is not counted toward any message limits, including inject.
548 Applications that support prefix mode must supply buffer space before
550 their own message data. The size of space that must be provided is
551 specified by the msg_prefix_size endpoint attribute. Providers are re‐
552 quired to define a msg_prefix_size that is a multiple of 8 bytes. Ad‐
553 ditionally, applications may receive provider generated packets that do
554 not contain application data. Such received messages will indicate a
555 transfer size of that is equal to or smaller than msg_prefix_size.
556 The buffer pointer given to all send and receive operations must point
558 to the start of the prefix region of the buffer (as opposed to the pay‐
559 load). For scatter-gather send/recv operations, the prefix buffer must
560 be a contiguous region, though it may or may not be directly adjacent
561 to the payload portion of the buffer.
562 FI_NOTIFY_FLAGS_ONLY
564 This bit indicates that general completion flags may not be set
565 by the provider, and are not needed by the application. If
566 specified, completion flags which simply report the type of op‐
567 eration that completed (e.g. send or receive) may not be set.
568 However, completion flags that are used for remote notifications
569 will still be set when applicable. See fi_cq(3) for details on
570 which completion flags are valid when this mode bit is enabled.
571 FI_RESTRICTED_COMP
573 This bit indicates that the application will only share comple‐
574 tion queues and counters among endpoints, transmit contexts, and
575 receive contexts that have the same set of capability flags.
576 FI_RX_CQ_DATA
578 This mode bit only applies to data transfers that set FI_RE‐
579 MOTE_CQ_DATA. When set, a data transfer that carries remote CQ
580 data will consume a receive buffer at the target. This is true
581 even for operations that would normally not consume posted re‐
582 ceive buffers, such as RMA write operations.
583
585 Multiple fabric interfaces take as input either a source or destination
586 address parameter. This includes struct fi_info (src_addr and dest_ad‐
587 dr), CM calls (getname, getpeer, connect, join, and leave), and AV
588 calls (insert, lookup, and straddr). The fi_info addr_format field in‐
589 dicates the expected address format for these operations.
590 A provider may support one or more of the following addressing formats.
592 In some cases, a selected addressing format may need to be translated
593 or mapped into an address which is native to the fabric. See fi_av(3).
594 FI_ADDR_BGQ
596 Address is an IBM proprietary format that is used with their
597 Blue Gene Q systems.
598 FI_ADDR_EFA
600 Address is an Amazon Elastic Fabric Adapter (EFA) proprietary
601 format.
602 FI_ADDR_GNI
604 Address is a Cray proprietary format that is used with their GNI
605 protocol.
606 FI_ADDR_PSMX
608 Address is an Intel proprietary format used with their Perfor‐
609 mance Scaled Messaging protocol.
610 FI_ADDR_PSMX2
612 Address is an Intel proprietary format used with their Perfor‐
613 mance Scaled Messaging protocol version 2.
614 FI_ADDR_PSMX3
616 Address is an Intel proprietary format used with their Perfor‐
617 mance Scaled Messaging protocol version 3.
618 FI_ADDR_STR
620 Address is a formatted character string. The length and content
621 of the string is address and/or provider specific, but in gener‐
622 al follows a URI model:
623 address_format[://[node][:[service][/[field3]...][?[key=value][&k2=v2]...]]]
625 Examples: - fi_sockaddr://10.31.6.12:7471 - fi_sockad‐
627 dr_in6://[fe80::6:12]:7471 - fi_sockaddr://10.31.6.12:7471?qos=3
628 Since the string formatted address does not contain any provider infor‐
630 mation, the prov_name field of the fabric attribute structure should be
631 used to filter by provider if necessary.
632 FI_FORMAT_UNSPEC
634 FI_FORMAT_UNSPEC indicates that a provider specific address for‐
635 mat should be selected. Provider specific addresses may be pro‐
636 tocol specific or a vendor proprietary format. Applications
637 that select FI_FORMAT_UNSPEC should be prepared to treat re‐
638 turned addressing data as opaque. FI_FORMAT_UNSPEC targets apps
639 which make use of an out of band address exchange. Applications
640 which use FI_FORMAT_UNSPEC may use fi_getname() to obtain a
641 provider specific address assigned to an allocated endpoint.
642 FI_SOCKADDR
644 Address is of type sockaddr. The specific socket address format
645 will be determined at run time by interfaces examining the
646 sa_family field.
647 FI_SOCKADDR_IB
649 Address is of type sockaddr_ib (defined in Linux kernel source)
650 FI_SOCKADDR_IN
652 Address is of type sockaddr_in (IPv4).
653 FI_SOCKADDR_IN6
655 Address is of type sockaddr_in6 (IPv6).
656 FI_ADDR_PSMX
658 Address is an Intel proprietary format that is used with their
659 PSMX (extended performance scaled messaging) protocol.
660
662 The operation of the fi_getinfo call may be controlled through the use
663 of input flags. Valid flags include the following.
664 FI_NUMERICHOST
666 Indicates that the node parameter is a numeric string represen‐
667 tation of a fabric address, such as a dotted decimal IP address.
668 Use of this flag will suppress any lengthy name resolution pro‐
669 tocol.
670 FI_PROV_ATTR_ONLY
672 Indicates that the caller is only querying for what providers
673 are potentially available. All providers will return exactly
674 one fi_info struct, regardless of whether that provider is us‐
675 able on the current platform or not. The returned fi_info
676 struct will contain default values for all members, with the ex‐
677 ception of fabric_attr. The fabric_attr member will have the
678 prov_name and prov_version values filled in.
679 FI_SOURCE
681 Indicates that the node and service parameters specify the local
682 source address to associate with an endpoint. If specified, ei‐
683 ther the node and/or service parameter must be non-NULL. This
684 flag is often used with passive endpoints.
685
687 fi_getinfo() returns 0 on success. On error, fi_getinfo() returns a
688 negative value corresponding to fabric errno. Fabric errno values are
689 defined in rdma/fi_errno.h.
690 fi_allocinfo() returns a pointer to a new fi_info structure on success,
692 or NULL on error. fi_dupinfo() duplicates a single fi_info structure
693 and all the substructures within it, returning a pointer to the new
694 fi_info structure on success, or NULL on error. Both calls require
695 that the returned fi_info structure be freed via fi_freeinfo().
696
698 FI_EBADFLAGS
699 The specified endpoint or domain capability or operation flags
700 are invalid.
701 FI_ENODATA
703 Indicates that no providers could be found which support the re‐
704 quested fabric information.
705 FI_ENOMEM
707 Indicates that there was insufficient memory to complete the op‐
708 eration.
709
711 If hints are provided, the operation will be controlled by the values
712 that are supplied in the various fields (see section on fi_info). Ap‐
713 plications that require specific communication interfaces, domains, ca‐
714 pabilities or other requirements, can specify them using fields in
715 hints. Libfabric returns a linked list in info that points to a list
716 of matching interfaces. info is set to NULL if there are no communica‐
717 tion interfaces or none match the input hints.
718 If node is provided, fi_getinfo will attempt to resolve the fabric ad‐
720 dress to the given node. If node is not provided, fi_getinfo will at‐
721 tempt to resolve the fabric addressing information based on the provid‐
722 ed hints. The caller must call fi_freeinfo to release fi_info struc‐
723 tures returned by fi_getinfo.
724 If neither node, service or hints are provided, then fi_getinfo simply
726 returns the list all available communication interfaces.
727 Multiple threads may call fi_getinfo simultaneously, without any re‐
729 quirement for serialization.
730
732 fi_open(3), fi_endpoint(3), fi_domain(3), fi_nic(3) fi_trigger(3)
733
735 OpenFabrics.
736Libfabric Programmer’s Manual 2021-11-20 fi_getinfo(3)