1fi_av_set(3)                   Libfabric v1.17.0                  fi_av_set(3)
2
3
4

NAME

6       fi_av_set - Address vector set operations
7
8       fi_av_set / fi_close
9              Open or close an address vector set
10
11       fi_av_set_union
12              Perform a set union operation on two AV sets
13
14       fi_av_set_intersect
15              Perform a set intersect operation on two AV sets
16
17       fi_av_set_diff
18              Perform a set difference operation on two AV sets
19
20       fi_av_set_insert
21              Add an address to an AV set
22
23       fi_av_set_remove
24              Remove an address from an AV set
25
26       fi_av_set_addr
27              Obtain a collective address for current addresses in an AV set
28

SYNOPSIS

30              #include <rdma/fi_collective.h>
31
32              int fi_av_set(struct fid_av *av, struct fi_av_set_attr *attr,
33                    struct fid_av_set **set, void * context);
34
35              int fi_av_set_union(struct fid_av_set *dst, const struct fid_av_set *src);
36
37              int fi_av_set_intersect(struct fid_av_set *dst, const struct fid_av_set *src);
38
39              int fi_av_set_diff(struct fid_av_set *dst, const struct fid_av_set *src);
40
41              int fi_av_set_insert(struct fid_av_set *set, fi_addr_t addr);
42
43              int fi_av_set_remove(struct fid_av_set *set, fi_addr_t addr);
44
45              int fi_av_set_addr(struct fid_av_set *set, fi_addr_t *coll_addr);
46
47              int fi_close(struct fid *av_set);
48

ARGUMENTS

50       av     Address vector
51
52       set    Address vector set
53
54       dst    Address vector set updated by set operation
55
56       src    Address vector set providing input to a set operation
57
58       attr   Address vector set attributes
59
60       context
61              User specified context associated with the address vector set
62
63       flags  Additional flags to apply to the operation.
64
65       addr   Destination address to insert to remove from AV set.
66
67       coll_addr
68              Address identifying collective group.
69

DESCRIPTION

71       An address vector set (AV set) represents an ordered subset of address‐
72       es of an address vector.  AV sets are used to identify the participants
73       in  a collective operation.  Endpoints use the fi_join_collective() op‐
74       eration to associate itself with an AV set.  The join collective opera‐
75       tion provides an fi_addr that is used when communicating with a collec‐
76       tive group.
77
78       The creation and manipulation of an AV set is a  local  operation.   No
79       fabric  traffic  is exchanged between peers.  As a result, each peer is
80       responsible for creating matching AV sets as part of  their  collective
81       membership  definition.   See fi_collective(3) for a discussion of mem‐
82       bership models.
83
84   fi_av_set
85       The fi_av_set call creates a new AV set.  The initial properties of the
86       AV set are specified through the struct fi_av_set_attr parameter.  This
87       structure is defined below, and allows including a subset of  addresses
88       in  the  AV  set as part of AV set creation.  Addresses may be added or
89       removed from an AV set using the AV set interfaces defined below.
90
91   fi_av_set_attr
92              struct fi_av_set_attr {
93                  size_t count;
94                  fi_addr_t start_addr;
95                  fi_addr_t end_addr;
96                  uint64_t stride;
97                  size_t comm_key_size;
98                  uint8_t *comm_key;
99                  uint64_t flags;
100              };
101
102       count  Indicates the expected the number of members that will be a part
103              of  the AV set.  The provider uses this to optimize resource al‐
104              locations.  If count is 0, the provider will select a size based
105              on  available  system  configuration  data or underlying limita‐
106              tions.
107
108       start_addr / end_addr
109              The starting and ending addresses, inclusive, to include as part
110              of  the  AV  set.  The use of start and end address require that
111              the associated AV have been created as type FI_AV_TABLE.   Valid
112              addresses  in  the  AV which fall within the specified range and
113              which meet other requirements (such as stride) will be added  as
114              initial members to the AV set.  The start_addr and end_addr must
115              be set to FI_ADDR_NOTAVAIL if creating an empty AV set, a commu‐
116              nication key is being provided, or the AV is of type FI_AV_MAP.
117
118       The  number  of  addresses between start_addr and end_addr must be less
119       than or equal to the specified count value.
120
121       stride The number of entries between successive addresses  included  in
122              the  AV  set.   The  AV  set  will  include  all  addresses from
123              start_addr + stride x i, for increasing,  non-negative,  integer
124              values  of  i, up to end_addr.  A stride of 1 indicates that all
125              addresses between start_addr and end_addr should be added to the
126              AV  set.   Stride  should  be set to 0 unless the start_addr and
127              end_addr fields are valid.
128
129       comm_key_size
130              The length of the communication key in bytes.  This field should
131              be 0 if a communication key is not available.
132
133       comm_key
134              If  supported  by  the  fabric, this represents a key associated
135              with the AV set.  The communication key is used by  applications
136              that directly manage collective membership through a fabric man‐
137              agement agent or resource manager.  The key is  used  to  convey
138              that results of the membership setup to the underlying provider.
139              The use and format of a communication  key  is  fabric  provider
140              specific.
141
142       flags  Flags may be used to configure the AV set, including restricting
143              which collective operations the AV set needs  to  support.   See
144              the flags section for a list of flags that may be specified when
145              creating the AV set.
146
147   fi_av_set_union
148       The AV set union call adds all addresses in the source AV set that  are
149       not  in the destination AV set to the destination AV set.  Where order‐
150       ing matters, the newly inserted addresses are placed at the end of  the
151       AV set.
152
153   fi_av_set_intersect
154       The  AV set intersect call remove all addresses from the destination AV
155       set that are not also members of the source AV set.  The order  of  the
156       addresses in the destination AV set is unchanged.
157
158   fi_av_set_diff
159       The  AV set difference call removes all address from the destination AV
160       set that are also members of the source AV set.  The order of  the  ad‐
161       dresses in the destination AV set is unchanged.
162
163   fi_av_set_insert
164       The  AV set insert call appends the specified address to the end of the
165       AV set.
166
167   fi_av_set_remove
168       The AV set remove call removes the specified address from the given  AV
169       set.  The order of the remaining addresses in the AV set is unchanged.
170
171   fi_av_set_addr
172       Returns  an  address  that  may be used to communicate with all current
173       members of an AV set.  This is a local operation only that does not in‐
174       volve network communication.  The returned address may be used as input
175       into fi_join_collective.  Note that attempting to use the  address  re‐
176       turned  from  fi_av_set_addr  (e.g. passing  it  to fi_join_collective)
177       while simultaneously modifying the addresses stored in an  AV  set  re‐
178       sults in undefined behavior.
179
180   fi_close
181       Closes  an  AV  set and releases all resources associated with it.  Any
182       operations active at the time an AV set is closed will be aborted, with
183       the result of the collective undefined.
184

FLAGS

186       The following flags may be specified as part of AV set creation.
187
188       FI_UNIVERSE
189              When  set,  then  the  AV set will be created containing all ad‐
190              dresses stored in the corresponding AV.
191
192       FI_BARRIER_SET
193              If set, the AV set will be configured to support barrier  opera‐
194              tions.
195
196       FI_BROADCAST_SET
197              If set, the AV set will be configured to support broadcast oper‐
198              ations.
199
200       FI_ALLTOALL_SET
201              If set, the AV set will be configured to support all to all  op‐
202              erations.
203
204       FI_ALLREDUCE_SET
205              If  set, the AV set will be configured to support all reduce op‐
206              erations.
207
208       FI_ALLGATHER_SET
209              If set, the AV set will be configured to support all gather  op‐
210              erations.
211
212       FI_REDUCE_SCATTER_SET
213              If  set, the AV set will be configured to support reduce scatter
214              operations.
215
216       FI_REDUCE_SET
217              If set, the AV set will be configured to support  reduce  opera‐
218              tions.
219
220       FI_SCATTER_SET
221              If  set, the AV set will be configured to support scatter opera‐
222              tions.
223
224       FI_GATHER_SET
225              If set, the AV set will be configured to support  gather  opera‐
226              tions.
227

NOTES

229       Developers who are familiar with MPI will find that AV sets are similar
230       to MPI groups, and may act as a direct mapping in some,  but  not  all,
231       situations.
232
233       By  default  an AV set will be created to support all collective opera‐
234       tions supported by the underlying provider  (see  fi_query_collective).
235       Users may reduce resource requirements by specifying only those collec‐
236       tion operations needed by the AV set through the use of creation flags:
237       FI_BARRIER_SET, FI_BROADCAST_SET, etc.  If no such flags are specified,
238       the AV set will be configured to support any that are supported.  It is
239       an error for a user to request an unsupported collective.
240

RETURN VALUES

242       Returns 0 on success.  On error, a negative value corresponding to fab‐
243       ric errno is returned.  Fabric errno values are defined in  rdma/fi_er‐
244       rno.h.
245

SEE ALSO

247       fi_av(3), fi_collective(3)
248

AUTHORS

250       OpenFabrics.
251
252
253
254Libfabric Programmer’s Manual     2022-12-11                      fi_av_set(3)
Impressum