1fi_getinfo(3) Libfabric v1.18.1 fi_getinfo(3)
2
6 fi_getinfo, fi_freeinfo - Obtain / free fabric interface information
7 fi_allocinfo, fi_dupinfo - Allocate / duplicate an fi_info structure
9
11 #include <rdma/fabric.h>
12 int fi_getinfo(int version, const char *node, const char *service,
14 uint64_t flags, const struct fi_info *hints, struct fi_info **info);
15 void fi_freeinfo(struct fi_info *info);
17 struct fi_info *fi_allocinfo(void);
19 struct fi_info *fi_dupinfo(const struct fi_info *info);
21
23 version
24 Interface version requested by application.
25 node Optional, name or fabric address to resolve.
27 service
29 Optional, service name or port number of address.
30 flags Operation flags for the fi_getinfo call.
32 hints Reference to an fi_info structure that specifies criteria for
34 selecting the returned fabric information.
35 info A pointer to a linked list of fi_info structures containing re‐
37 sponse information.
38
40 The fi_getinfo() call is used to discover what communication features
41 are available in the system, as well as how they might best be used by
42 an application. The call is loosely modeled on getaddrinfo(). fi_get‐
43 info() permits an application to exchange information between an appli‐
44 cation and the libfabric providers regarding its required set of commu‐
45 nication. It provides the ability to access complex network details,
46 balanced between being expressive but also simple to use.
47 fi_getinfo returns information about available fabric services for
49 reaching a specified node or service, subject to any provided hints.
50 Callers may specify NULL for node, service, and hints in order to re‐
51 trieve information about what providers are available and their optimal
52 usage models. If no matching fabric information is available, info
53 will be set to NULL and the call will return -FI_ENODATA.
54 Based on the input hints, node, and service parameters, a list of fab‐
56 ric domains and endpoints will be returned. Each fi_info structure
57 will describe an endpoint that meets the application’s specified commu‐
58 nication criteria. Each endpoint will be associated with a domain.
59 Applications can restrict the number of returned endpoints by including
60 additional criteria in their search hints. Relaxing or eliminating in‐
61 put hints will increase the number and type of endpoints that are
62 available. Providers that return multiple endpoints to a single
63 fi_getinfo call should return the endpoints that are highest performing
64 first. Providers may indicate that an endpoint and domain can support
65 additional capabilities than those requested by the user only if such
66 support will not adversely affect application performance or security.
67 The version parameter is used by the application to request the desired
69 version of the interfaces. The version determines the format of all
70 data structures used by any of the fabric interfaces. Applications
71 should use the FI_VERSION(major, minor) macro to indicate the version,
72 with hard-coded integer values for the major and minor values. The
73 FI_MAJOR_VERSION and FI_MINOR_VERSION enum values defined in fabric.h
74 specify the latest version of the installed library. However, it is
75 recommended that the integer values for FI_MAJOR_VERSION and FI_MI‐
76 NOR_VERSION be used, rather than referencing the enum types in order to
77 ensure compatibility with future versions of the library. This pro‐
78 tects against the application being built from source against a newer
79 version of the library that introduces new fields to data structures,
80 which would not be initialized by the application.
81 Node, service, or hints may be provided, with any combination being
83 supported. If node is provided, fi_getinfo will attempt to resolve the
84 fabric address to the given node. If node is not given, fi_getinfo
85 will attempt to resolve the fabric addressing information based on the
86 provided hints. Node is commonly used to provide a network address
87 (such as an IP address) or hostname. Service is usually associated
88 with a transport address (such as a TCP port number). Node and service
89 parameters may be mapped by providers to native fabric addresses. Ap‐
90 plications may also pass in an FI_ADDR_STR formatted address (see for‐
91 mat details below) as the node parameter. In such cases, the service
92 parameter must be NULL.
93 The hints parameter, if provided, may be used to limit the resulting
95 output as indicated below. As a general rule, specifying a non-zero
96 value for input hints indicates that a provider must support the re‐
97 quested value or fail the operation with -FI_ENODATA. With the excep‐
98 tion of mode bits, hints that are set to zero are treated as a wild‐
99 card. A zeroed hint value results in providers either returning a de‐
100 fault value or a value that works best for their implementation. Mode
101 bits that are set to zero indicate the application does not support any
102 modes.
103 The caller must call fi_freeinfo to release fi_info structures returned
105 by this call.
106 The fi_allocinfo call will allocate and zero an fi_info structure and
108 all related substructures. The fi_dupinfo will duplicate a single
109 fi_info structure and all the substructures within it.
110
112 struct fi_info {
113 struct fi_info *next;
114 uint64_t caps;
115 uint64_t mode;
116 uint32_t addr_format;
117 size_t src_addrlen;
118 size_t dest_addrlen;
119 void *src_addr;
120 void *dest_addr;
121 fid_t handle;
122 struct fi_tx_attr *tx_attr;
123 struct fi_rx_attr *rx_attr;
124 struct fi_ep_attr *ep_attr;
125 struct fi_domain_attr *domain_attr;
126 struct fi_fabric_attr *fabric_attr;
127 struct fid_nic *nic;
128 };
129 next Pointer to the next fi_info structure in the list. Will be NULL
131 if no more structures exist.
132 caps - fabric interface capabilities
134 If specified, indicates the desired capabilities of the fabric
135 interfaces. Supported capabilities are listed in the Capabili‐
136 ties section below.
137 mode Operational modes supported by the application. See the Mode
139 section below.
140 addr_format - address format
142 If specified, indicates the format of addresses referenced by
143 the fabric interfaces and data structures. Supported formats
144 are listed in the Addressing formats section below.
145 src_addrlen - source address length
147 Indicates the length of the source address. This value must be
148 > 0 if src_addr is non-NULL. This field will be ignored in
149 hints if FI_SOURCE flag is set, or src_addr is NULL.
150 dest_addrlen - destination address length
152 Indicates the length of the destination address. This value
153 must be > 0 if dest_addr is non-NULL. This field will be ig‐
154 nored in hints unless the node and service parameters are NULL
155 or FI_SOURCE flag is set, or if dst_addr is NULL.
156 src_addr - source address
158 If specified, indicates the source address. This field will be
159 ignored in hints if FI_SOURCE flag is set. On output a provider
160 shall return an address that corresponds to the indicated fab‐
161 ric, domain, node, and/or service fields. The format of the ad‐
162 dress is indicated by the returned addr_format field. Note that
163 any returned address is only used when opening a local endpoint.
164 The address is not guaranteed to be usable by a peer process.
165 dest_addr - destination address
167 If specified, indicates the destination address. This field
168 will be ignored in hints unless the node and service parameters
169 are NULL or FI_SOURCE flag is set. If FI_SOURCE is not speci‐
170 fied, on output a provider shall return an address the corre‐
171 sponds to the indicated node and/or service fields, relative to
172 the fabric and domain. Note that any returned address is only
173 usable locally.
174 handle - provider context handle
176 The use of this field is operation specific. If hints->handle
177 is set to struct fid_pep, the hints->handle will be copied to
178 info->handle on output from fi_getinfo. Other values of
179 hints->handle will be handled in a provider specific manner.
180 The fi_info::handle field is also used by fi_endpoint() and
181 fi_reject() calls when processing connection requests or to in‐
182 herit another endpoint’s attributes. See fi_eq(3), fi_re‐
183 ject(3), and fi_endpoint(3). The info->handle field will be ig‐
184 nored by fi_dupinfo and fi_freeinfo.
185 tx_attr - transmit context attributes
187 Optionally supplied transmit context attributes. Transmit con‐
188 text attributes may be specified and returned as part of fi_get‐
189 info. When provided as hints, requested values of struct
190 fi_tx_ctx_attr should be set. On output, the actual transmit
191 context attributes that can be provided will be returned. Out‐
192 put values will be greater than or equal to the requested input
193 values.
194 rx_attr - receive context attributes
196 Optionally supplied receive context attributes. Receive context
197 attributes may be specified and returned as part of fi_getinfo.
198 When provided as hints, requested values of struct fi_rx_ctx_at‐
199 tr should be set. On output, the actual receive context at‐
200 tributes that can be provided will be returned. Output values
201 will be greater than or or equal to the requested input values.
202 ep_attr - endpoint attributes
204 Optionally supplied endpoint attributes. Endpoint attributes
205 may be specified and returned as part of fi_getinfo. When pro‐
206 vided as hints, requested values of struct fi_ep_attr should be
207 set. On output, the actual endpoint attributes that can be pro‐
208 vided will be returned. Output values will be greater than or
209 equal to requested input values. See fi_endpoint(3) for de‐
210 tails.
211 domain_attr - domain attributes
213 Optionally supplied domain attributes. Domain attributes may be
214 specified and returned as part of fi_getinfo. When provided as
215 hints, requested values of struct fi_domain_attr should be set.
216 On output, the actual domain attributes that can be provided
217 will be returned. Output values will be greater than or equal
218 to requested input values. See fi_domain(3) for details.
219 fabric_attr - fabric attributes
221 Optionally supplied fabric attributes. Fabric attributes may be
222 specified and returned as part of fi_getinfo. When provided as
223 hints, requested values of struct fi_fabric_attr should be set.
224 On output, the actual fabric attributes that can be provided
225 will be returned. See fi_fabric(3) for details.
226 nic - network interface details
228 Optional attributes related to the hardware NIC associated with
229 the specified fabric, domain, and endpoint data. This field is
230 only valid for providers where the corresponding attributes are
231 closely associated with a hardware NIC. See fi_nic(3) for de‐
232 tails.
233
235 Interface capabilities are obtained by OR-ing the following flags to‐
236 gether. If capabilities in the hint parameter are set to 0, the under‐
237 lying provider will return the set of capabilities which are supported.
238 Otherwise, providers will return data matching the specified set of ca‐
239 pabilities. Providers may indicate support for additional capabilities
240 beyond those requested when the use of expanded capabilities will not
241 adversely affect performance or expose the application to communication
242 beyond that which was requested. Applications may use this feature to
243 request a minimal set of requirements, then check the returned capabil‐
244 ities to enable additional optimizations.
245 FI_ATOMIC
247 Specifies that the endpoint supports some set of atomic opera‐
248 tions. Endpoints supporting this capability support operations
249 defined by struct fi_ops_atomic. In the absence of any relevant
250 flags, FI_ATOMIC implies the ability to initiate and be the tar‐
251 get of remote atomic reads and writes. Applications can use the
252 FI_READ, FI_WRITE, FI_REMOTE_READ, and FI_REMOTE_WRITE flags to
253 restrict the types of atomic operations supported by an end‐
254 point.
255 FI_AV_USER_ID
257 Requests that the provider support the association of a user
258 specified identifier with each address vector (AV) address. Us‐
259 er identifiers are returned with completion data in place of the
260 AV address. See [fi_av(3)] (fi_av.3.html) for more details.
261 FI_COLLECTIVE
263 Requests support for collective operations. Endpoints that sup‐
264 port this capability support the collective operations defined
265 in fi_collective(3).
266 FI_DIRECTED_RECV
268 Requests that the communication endpoint use the source address
269 of an incoming message when matching it with a receive buffer.
270 If this capability is not set, then the src_addr parameter for
271 msg and tagged receive operations is ignored.
272 FI_FENCE
274 Indicates that the endpoint support the FI_FENCE flag on data
275 transfer operations. Support requires tracking that all previ‐
276 ous transmit requests to a specified remote endpoint complete
277 prior to initiating the fenced operation. Fenced operations are
278 often used to enforce ordering between operations that are not
279 otherwise guaranteed by the underlying provider or protocol.
280 FI_HMEM
282 Specifies that the endpoint should support transfers to and from
283 device memory.
284 FI_LOCAL_COMM
286 Indicates that the endpoint support host local communication.
287 This flag may be used in conjunction with FI_REMOTE_COMM to in‐
288 dicate that local and remote communication are required. If
289 neither FI_LOCAL_COMM or FI_REMOTE_COMM are specified, then the
290 provider will indicate support for the configuration that mini‐
291 mally affects performance. Providers that set FI_LOCAL_COMM but
292 not FI_REMOTE_COMM, for example a shared memory provider, may
293 only be used to communication between processes on the same sys‐
294 tem.
295 FI_MSG Specifies that an endpoint should support sending and receiving
297 messages or datagrams. Message capabilities imply support for
298 send and/or receive queues. Endpoints supporting this capabili‐
299 ty support operations defined by struct fi_ops_msg.
300 The caps may be used to specify or restrict the type of messaging oper‐
302 ations that are supported. In the absence of any relevant flags,
303 FI_MSG implies the ability to send and receive messages. Applications
304 can use the FI_SEND and FI_RECV flags to optimize an endpoint as send-
305 only or receive-only.
306 FI_MULTICAST
308 Indicates that the endpoint support multicast data transfers.
309 This capability must be paired with FI_MSG. Applications can
310 use FI_SEND and FI_RECV to optimize multicast as send-only or
311 receive-only.
312 FI_MULTI_RECV
314 Specifies that the endpoint must support the FI_MULTI_RECV flag
315 when posting receive buffers.
316 FI_NAMED_RX_CTX
318 Requests that endpoints which support multiple receive contexts
319 allow an initiator to target (or name) a specific receive con‐
320 text as part of a data transfer operation.
321 FI_READ
323 Indicates that the user requires an endpoint capable of initiat‐
324 ing reads against remote memory regions. This flag requires
325 that FI_RMA and/or FI_ATOMIC be set.
326 FI_RECV
328 Indicates that the user requires an endpoint capable of receiv‐
329 ing message data transfers. Message transfers include base mes‐
330 sage operations as well as tagged message functionality.
331 FI_REMOTE_COMM
333 Indicates that the endpoint support communication with endpoints
334 located at remote nodes (across the fabric). See FI_LOCAL_COMM
335 for additional details. Providers that set FI_REMOTE_COMM but
336 not FI_LOCAL_COMM, for example NICs that lack loopback support,
337 cannot be used to communicate with processes on the same system.
338 FI_REMOTE_READ
340 Indicates that the user requires an endpoint capable of receiv‐
341 ing read memory operations from remote endpoints. This flag re‐
342 quires that FI_RMA and/or FI_ATOMIC be set.
343 FI_REMOTE_WRITE
345 Indicates that the user requires an endpoint capable of receiv‐
346 ing write memory operations from remote endpoints. This flag
347 requires that FI_RMA and/or FI_ATOMIC be set.
348 FI_RMA Specifies that the endpoint should support RMA read and write
350 operations. Endpoints supporting this capability support opera‐
351 tions defined by struct fi_ops_rma. In the absence of any rele‐
352 vant flags, FI_RMA implies the ability to initiate and be the
353 target of remote memory reads and writes. Applications can use
354 the FI_READ, FI_WRITE, FI_REMOTE_READ, and FI_REMOTE_WRITE flags
355 to restrict the types of RMA operations supported by an end‐
356 point.
357 FI_RMA_EVENT
359 Requests that an endpoint support the generation of completion
360 events when it is the target of an RMA and/or atomic operation.
361 This flag requires that FI_REMOTE_READ and/or FI_REMOTE_WRITE be
362 enabled on the endpoint.
363 FI_RMA_PMEM
365 Indicates that the provider is `persistent memory aware' and
366 supports RMA operations to and from persistent memory. Persis‐
367 tent memory aware providers must support registration of memory
368 that is backed by non- volatile memory, RMA transfers to/from
369 persistent memory, and enhanced completion semantics. This flag
370 requires that FI_RMA be set. This capability is experimental.
371 FI_SEND
373 Indicates that the user requires an endpoint capable of sending
374 message data transfers. Message transfers include base message
375 operations as well as tagged message functionality.
376 FI_SHARED_AV
378 Requests or indicates support for address vectors which may be
379 shared among multiple processes.
380 FI_SOURCE
382 Requests that the endpoint return source addressing data as part
383 of its completion data. This capability only applies to connec‐
384 tionless endpoints. Note that returning source address informa‐
385 tion may require that the provider perform address translation
386 and/or look-up based on data available in the underlying proto‐
387 col in order to provide the requested data, which may adversely
388 affect performance. The performance impact may be greater for
389 address vectors of type FI_AV_TABLE.
390 FI_SOURCE_ERR
392 Must be paired with FI_SOURCE. When specified, this requests
393 that raw source addressing data be returned as part of comple‐
394 tion data for any address that has not been inserted into the
395 local address vector. Use of this capability may require the
396 provider to validate incoming source address data against ad‐
397 dresses stored in the local address vector, which may adversely
398 affect performance.
399 FI_TAGGED
401 Specifies that the endpoint should handle tagged message trans‐
402 fers. Tagged message transfers associate a user-specified key
403 or tag with each message that is used for matching purposes at
404 the remote side. Endpoints supporting this capability support
405 operations defined by struct fi_ops_tagged. In the absence of
406 any relevant flags, FI_TAGGED implies the ability to send and
407 receive tagged messages. Applications can use the FI_SEND and
408 FI_RECV flags to optimize an endpoint as send-only or receive-
409 only.
410 FI_TRIGGER
412 Indicates that the endpoint should support triggered operations.
413 Endpoints support this capability must meet the usage model as
414 described by fi_trigger(3).
415 FI_VARIABLE_MSG
417 Requests that the provider must notify a receiver when a vari‐
418 able length message is ready to be received prior to attempting
419 to place the data. Such notification will include the size of
420 the message and any associated message tag (for FI_TAGGED). See
421 `Variable Length Messages' in fi_msg.3 for full details. Vari‐
422 able length messages are any messages larger than an endpoint
423 configurable size. This flag requires that FI_MSG and/or
424 FI_TAGGED be set.
425 FI_WRITE
427 Indicates that the user requires an endpoint capable of initiat‐
428 ing writes against remote memory regions. This flag requires
429 that FI_RMA and/or FI_ATOMIC be set.
430 FI_XPU Specifies that the endpoint should support transfers that may be
432 initiated from heterogenous computation devices, such as GPUs.
433 This flag requires that FI_TRIGGER be set. For additional de‐
434 tails on XPU triggers see fi_trigger(3).
435 Capabilities may be grouped into three general categories: primary,
437 secondary, and primary modifiers. Primary capabilities must explicitly
438 be requested by an application, and a provider must enable support for
439 only those primary capabilities which were selected. Primary modifiers
440 are used to limit a primary capability, such as restricting an endpoint
441 to being send-only. If no modifiers are specified for an applicable
442 capability, all relevant modifiers are assumed. See above definitions
443 for details.
444 Secondary capabilities may optionally be requested by an application.
446 If requested, a provider must support the capability or fail the
447 fi_getinfo request (FI_ENODATA). A provider may optionally report non-
448 selected secondary capabilities if doing so would not compromise per‐
449 formance or security.
450 Primary capabilities: FI_MSG, FI_RMA, FI_TAGGED, FI_ATOMIC, FI_MULTI‐
452 CAST, FI_NAMED_RX_CTX, FI_DIRECTED_RECV, FI_VARIABLE_MSG, FI_HMEM,
453 FI_COLLECTIVE, FI_XPU
454 Primary modifiers: FI_READ, FI_WRITE, FI_RECV, FI_SEND, FI_REMOTE_READ,
456 FI_REMOTE_WRITE
457 Secondary capabilities: FI_MULTI_RECV, FI_SOURCE, FI_RMA_EVENT,
459 FI_SHARED_AV, FI_TRIGGER, FI_FENCE, FI_LOCAL_COMM, FI_REMOTE_COMM,
460 FI_SOURCE_ERR, FI_RMA_PMEM.
461
463 The operational mode bits are used to convey requirements that an ap‐
464 plication must adhere to when using the fabric interfaces. Modes spec‐
465 ify optimal ways of accessing the reported endpoint or domain. Appli‐
466 cations that are designed to support a specific mode of operation may
467 see improved performance when that mode is desired by the provider. It
468 is recommended that providers support applications that disable any
469 provider preferred modes.
470 On input to fi_getinfo, applications set the mode bits that they sup‐
472 port. On output, providers will clear mode bits that are not necessary
473 to achieve high-performance. Mode bits that remain set indicate appli‐
474 cation requirements for using the fabric interfaces created using the
475 returned fi_info. The set of modes are listed below. If a NULL hints
476 structure is provided, then the provider’s supported set of modes will
477 be returned in the info structure(s).
478 FI_ASYNC_IOV
480 Applications can reference multiple data buffers as part of a
481 single operation through the use of IO vectors (SGEs). Typical‐
482 ly, the contents of an IO vector are copied by the provider into
483 an internal buffer area, or directly to the underlying hardware.
484 However, when a large number of IOV entries are supported, IOV
485 buffering may have a negative impact on performance and memory
486 consumption. The FI_ASYNC_IOV mode indicates that the applica‐
487 tion must provide the buffering needed for the IO vectors. When
488 set, an application must not modify an IO vector of length > 1,
489 including any related memory descriptor array, until the associ‐
490 ated operation has completed.
491 FI_BUFFERED_RECV
493 The buffered receive mode bit indicates that the provider owns
494 the data buffer(s) that are accessed by the networking layer for
495 received messages. Typically, this implies that data must be
496 copied from the provider buffer into the application buffer.
497 Applications that can handle message processing from network al‐
498 located data buffers can set this mode bit to avoid copies. For
499 full details on application requirements to support this mode,
500 see the `Buffered Receives' section in fi_msg(3). This mode bit
501 applies to FI_MSG and FI_TAGGED receive operations.
502 FI_CONTEXT
504 Specifies that the provider requires that applications use
505 struct fi_context as their per operation context parameter for
506 operations that generated full completions. This structure
507 should be treated as opaque to the application. For performance
508 reasons, this structure must be allocated by the user, but may
509 be used by the fabric provider to track the operation. Typical‐
510 ly, users embed struct fi_context within their own context
511 structure. The struct fi_context must remain valid until the
512 corresponding operation completes or is successfully canceled.
513 As such, fi_context should NOT be allocated on the stack. Doing
514 so is likely to result in stack corruption that will be diffi‐
515 cult to debug. Users should not update or interpret the fields
516 in this structure, or reuse it until the original operation has
517 completed. If an operation does not generate a completion
518 (i.e. the endpoint was configured with FI_SELECTIVE_COMPLETION
519 and the operation was not initiated with the FI_COMPLETION flag)
520 then the context parameter is ignored by the fabric provider.
521 The structure is specified in rdma/fabric.h.
522 FI_CONTEXT2
524 This bit is similar to FI_CONTEXT, but doubles the provider’s
525 requirement on the size of the per context structure. When set,
526 this specifies that the provider requires that applications use
527 struct fi_context2 as their per operation context parameter.
528 Or, optionally, an application can provide an array of two
529 fi_context structures (e.g. struct fi_context[2]) instead. The
530 requirements for using struct fi_context2 are identical as de‐
531 fined for FI_CONTEXT above.
532 FI_LOCAL_MR
534 The provider is optimized around having applications register
535 memory for locally accessed data buffers. Data buffers used in
536 send and receive operations and as the source buffer for RMA and
537 atomic operations must be registered by the application for ac‐
538 cess domains opened with this capability. This flag is defined
539 for compatibility and is ignored if the application version is
540 1.5 or later and the domain mr_mode is set to anything other
541 than FI_MR_BASIC or FI_MR_SCALABLE. See the domain attribute
542 mr_mode fi_domain(3) and fi_mr(3).
543 FI_MSG_PREFIX
545 Message prefix mode indicates that an application will provide
546 buffer space in front of all message send and receive buffers
547 for use by the provider. Typically, the provider uses this
548 space to implement a protocol, with the protocol headers being
549 written into the prefix area. The contents of the prefix space
550 should be treated as opaque. The use of FI_MSG_PREFIX may im‐
551 prove application performance over certain providers by reducing
552 the number of IO vectors referenced by underlying hardware and
553 eliminating provider buffer allocation.
554 FI_MSG_PREFIX only applies to send and receive operations, including
556 tagged sends and receives. RMA and atomics do not require the applica‐
557 tion to provide prefix buffers. Prefix buffer space must be provided
558 with all sends and receives, regardless of the size of the transfer or
559 other transfer options. The ownership of prefix buffers is treated the
560 same as the corresponding message buffers, but the size of the prefix
561 buffer is not counted toward any message limits, including inject.
562 Applications that support prefix mode must supply buffer space before
564 their own message data. The size of space that must be provided is
565 specified by the msg_prefix_size endpoint attribute. Providers are re‐
566 quired to define a msg_prefix_size that is a multiple of 8 bytes. Ad‐
567 ditionally, applications may receive provider generated packets that do
568 not contain application data. Such received messages will indicate a
569 transfer size of that is equal to or smaller than msg_prefix_size.
570 The buffer pointer given to all send and receive operations must point
572 to the start of the prefix region of the buffer (as opposed to the pay‐
573 load). For scatter-gather send/recv operations, the prefix buffer must
574 be a contiguous region, though it may or may not be directly adjacent
575 to the payload portion of the buffer.
576 FI_NOTIFY_FLAGS_ONLY
578 This bit indicates that general completion flags may not be set
579 by the provider, and are not needed by the application. If
580 specified, completion flags which simply report the type of op‐
581 eration that completed (e.g. send or receive) may not be set.
582 However, completion flags that are used for remote notifications
583 will still be set when applicable. See fi_cq(3) for details on
584 which completion flags are valid when this mode bit is enabled.
585 FI_RESTRICTED_COMP
587 This bit indicates that the application will only share comple‐
588 tion queues and counters among endpoints, transmit contexts, and
589 receive contexts that have the same set of capability flags.
590 FI_RX_CQ_DATA
592 This mode bit only applies to data transfers that set FI_RE‐
593 MOTE_CQ_DATA. When set, a data transfer that carries remote CQ
594 data will consume a receive buffer at the target. This is true
595 even for operations that would normally not consume posted re‐
596 ceive buffers, such as RMA write operations.
597
599 Multiple fabric interfaces take as input either a source or destination
600 address parameter. This includes struct fi_info (src_addr and dest_ad‐
601 dr), CM calls (getname, getpeer, connect, join, and leave), and AV
602 calls (insert, lookup, and straddr). The fi_info addr_format field in‐
603 dicates the expected address format for these operations.
604 A provider may support one or more of the following addressing formats.
606 In some cases, a selected addressing format may need to be translated
607 or mapped into an address which is native to the fabric. See fi_av(3).
608 FI_ADDR_BGQ
610 Address is an IBM proprietary format that is used with their
611 Blue Gene Q systems.
612 FI_ADDR_EFA
614 Address is an Amazon Elastic Fabric Adapter (EFA) proprietary
615 format.
616 FI_ADDR_GNI
618 Address is a Cray proprietary format that is used with their GNI
619 protocol.
620 FI_ADDR_PSMX
622 Address is an Intel proprietary format used with their Perfor‐
623 mance Scaled Messaging protocol.
624 FI_ADDR_PSMX2
626 Address is an Intel proprietary format used with their Perfor‐
627 mance Scaled Messaging protocol version 2.
628 FI_ADDR_PSMX3
630 Address is an Intel proprietary format used with their Perfor‐
631 mance Scaled Messaging protocol version 3.
632 FI_ADDR_STR
634 Address is a formatted character string. The length and content
635 of the string is address and/or provider specific, but in gener‐
636 al follows a URI model:
637 address_format[://[node][:[service][/[field3]...][?[key=value][&k2=v2]...]]]
639 Examples: - fi_sockaddr://10.31.6.12:7471 - fi_sockad‐
641 dr_in6://[fe80::6:12]:7471 - fi_sockaddr://10.31.6.12:7471?qos=3
642 Since the string formatted address does not contain any provider infor‐
644 mation, the prov_name field of the fabric attribute structure should be
645 used to filter by provider if necessary.
646 FI_FORMAT_UNSPEC
648 FI_FORMAT_UNSPEC indicates that a provider specific address for‐
649 mat should be selected. Provider specific addresses may be pro‐
650 tocol specific or a vendor proprietary format. Applications
651 that select FI_FORMAT_UNSPEC should be prepared to treat re‐
652 turned addressing data as opaque. FI_FORMAT_UNSPEC targets apps
653 which make use of an out of band address exchange. Applications
654 which use FI_FORMAT_UNSPEC may use fi_getname() to obtain a
655 provider specific address assigned to an allocated endpoint.
656 FI_SOCKADDR
658 Address is of type sockaddr. The specific socket address format
659 will be determined at run time by interfaces examining the
660 sa_family field.
661 FI_SOCKADDR_IB
663 Address is of type sockaddr_ib (defined in Linux kernel source)
664 FI_SOCKADDR_IN
666 Address is of type sockaddr_in (IPv4).
667 FI_SOCKADDR_IN6
669 Address is of type sockaddr_in6 (IPv6).
670 FI_ADDR_PSMX
672 Address is an Intel proprietary format that is used with their
673 PSMX (extended performance scaled messaging) protocol.
674
676 The operation of the fi_getinfo call may be controlled through the use
677 of input flags. Valid flags include the following.
678 FI_NUMERICHOST
680 Indicates that the node parameter is a numeric string represen‐
681 tation of a fabric address, such as a dotted decimal IP address.
682 Use of this flag will suppress any lengthy name resolution pro‐
683 tocol.
684 FI_PROV_ATTR_ONLY
686 Indicates that the caller is only querying for what providers
687 are potentially available. All providers will return exactly
688 one fi_info struct, regardless of whether that provider is us‐
689 able on the current platform or not. The returned fi_info
690 struct will contain default values for all members, with the ex‐
691 ception of fabric_attr. The fabric_attr member will have the
692 prov_name and prov_version values filled in.
693 FI_SOURCE
695 Indicates that the node and service parameters specify the local
696 source address to associate with an endpoint. If specified, ei‐
697 ther the node and/or service parameter must be non-NULL. This
698 flag is often used with passive endpoints.
699
701 fi_getinfo() returns 0 on success. On error, fi_getinfo() returns a
702 negative value corresponding to fabric errno. Fabric errno values are
703 defined in rdma/fi_errno.h.
704 fi_allocinfo() returns a pointer to a new fi_info structure on success,
706 or NULL on error. fi_dupinfo() duplicates a single fi_info structure
707 and all the substructures within it, returning a pointer to the new
708 fi_info structure on success, or NULL on error. Both calls require
709 that the returned fi_info structure be freed via fi_freeinfo().
710
712 FI_EBADFLAGS
713 The specified endpoint or domain capability or operation flags
714 are invalid.
715 FI_ENODATA
717 Indicates that no providers could be found which support the re‐
718 quested fabric information.
719 FI_ENOMEM
721 Indicates that there was insufficient memory to complete the op‐
722 eration.
723 F_ENOSYS
725 Indicates that requested version is newer than the library being
726 used.
727
729 If hints are provided, the operation will be controlled by the values
730 that are supplied in the various fields (see section on fi_info). Ap‐
731 plications that require specific communication interfaces, domains, ca‐
732 pabilities or other requirements, can specify them using fields in
733 hints. Libfabric returns a linked list in info that points to a list
734 of matching interfaces. info is set to NULL if there are no communica‐
735 tion interfaces or none match the input hints.
736 If node is provided, fi_getinfo will attempt to resolve the fabric ad‐
738 dress to the given node. If node is not provided, fi_getinfo will at‐
739 tempt to resolve the fabric addressing information based on the provid‐
740 ed hints. The caller must call fi_freeinfo to release fi_info struc‐
741 tures returned by fi_getinfo.
742 If neither node, service or hints are provided, then fi_getinfo simply
744 returns the list all available communication interfaces.
745 Multiple threads may call fi_getinfo simultaneously, without any re‐
747 quirement for serialization.
748
750 fi_open(3), fi_endpoint(3), fi_domain(3), fi_nic(3) fi_trigger(3)
751
753 OpenFabrics.
754Libfabric Programmer’s Manual 2023-03-22 fi_getinfo(3)