1fi_av_set(3)                 Libfabric v1.12.0rc1                 fi_av_set(3)
2
3
4

NAME

6       fi_av_set - Address vector set operations
7
8       fi_av_set / fi_close
9              Open or close an address vector set
10
11       fi_av_set_union
12              Perform a set union operation on two AV sets
13
14       fi_av_set_intersect
15              Perform a set intersect operation on two AV sets
16
17       fi_av_set_diff
18              Perform a set difference operation on two AV sets
19
20       fi_av_set_insert
21              Add an address to an AV set
22
23       fi_av_set_remove
24              Remove an address from an AV set
25
26       fi_av_set_addr
27              Obtain a collective address for current addresses in an AV set
28

SYNOPSIS

30              #include <rdma/fi_collective.h>
31
32              int fi_av_set(struct fid_av *av, struct fi_av_set_attr *attr,
33                    struct fid_av_set **set, void * context);
34
35              int fi_av_set_union(struct fid_av_set *dst, const struct fid_av_set *src);
36
37              int fi_av_set_intersect(struct fid_av_set *dst, const struct fid_av_set *src);
38
39              int fi_av_set_diff(struct fid_av_set *dst, const struct fid_av_set *src);
40
41              int fi_av_set_insert(struct fid_av_set *set, fi_addr_t addr);
42
43              int fi_av_set_remove(struct fid_av_set *set, fi_addr_t addr);
44
45              int fi_av_set_addr(struct fid_av_set *set, fi_addr_t *coll_addr);
46
47              int fi_close(struct fid *av_set);
48

ARGUMENTS

50       av     Address vector
51
52       set    Address vector set
53
54       dst    Address vector set updated by set operation
55
56       src    Address vector set providing input to a set operation
57
58       attr   Address vector set attributes
59
60       context
61              User specified context associated with the address vector set
62
63       flags  Additional flags to apply to the operation.
64
65       addr   Destination address to insert to remove from AV set.
66
67       coll_addr
68              Address identifying collective group.
69

DESCRIPTION

71       An address vector set (AV set) represents an ordered subset of address‐
72       es of an address vector.  AV sets are used to identify the participants
73       in  a collective operation.  Endpoints use the fi_join_collective() op‐
74       eration to associate itself with an AV set.  The join collective opera‐
75       tion provides an fi_addr that is used when communicating with a collec‐
76       tive group.
77
78       The creation and manipulation of an AV set is a  local  operation.   No
79       fabric  traffic  is exchanged between peers.  As a result, each peer is
80       responsible for creating matching AV sets as part of  their  collective
81       membership  definition.   See fi_collective(3) for a discussion of mem‐
82       bership models.
83
84   fi_av_set
85       The fi_av_set call creates a new AV set.  The initial properties of the
86       AV set are specified through the struct fi_av_set_attr parameter.  This
87       structure is defined below, and allows including a subset of  addresses
88       in  the  AV  set as part of AV set creation.  Addresses may be added or
89       removed from an AV set using the AV set interfaces defined below.
90
91   fi_av_set_attr
92              struct fi_av_set_attr {
93                  size_t count;
94                  fi_addr_t start_addr;
95                  fi_addr_t end_addr;
96                  uint64_t stride;
97                  size_t comm_key_size;
98                  uint8_t *comm_key;
99                  uint64_t flags;
100              };
101
102       count  Indicates the expected the number of members that will be a part
103              of  the AV set.  The provider uses this to optimize resource al‐
104              locations.  If count is 0, the provider will select a size based
105              on  available  system  configuration  data or underlying limita‐
106              tions.
107
108       start_addr / end_addr
109              The starting and ending addresses, inclusive, to include as part
110              of  the  AV  set.  The use of start and end address require that
111              the associated AV have been created as type FI_AV_TABLE.   Valid
112              addresses  in  the  AV which fall within the specified range and
113              which meet other requirements (such as stride) will be added  as
114              initial members to the AV set.  The start_addr and end_addr must
115              be set to FI_ADDR_NOTAVAIL if creating an empty AV set, a commu‐
116              nication key is being provided, or the AV is of type FI_AV_MAP.
117
118       The  number  of  addresses between start_addr and end_addr must be less
119       than or equal to the specified count value.
120
121       stride The number of entries between successive addresses  included  in
122              the  AV  set.   The  AV  set  will  include  all  addresses from
123              start_addr + stride x i, for increasing,  non-negative,  integer
124              values  of  i, up to end_addr.  A stride of 1 indicates that all
125              addresses between start_addr and end_addr should be added to the
126              AV  set.   Stride  should  be set to 0 unless the start_addr and
127              end_addr fields are valid.
128
129       comm_key_size
130              The length of the communication key in bytes.  This field should
131              be 0 if a communication key is not available.
132
133       comm_key
134              If  supported  by  the  fabric, this represents a key associated
135              with the AV set.  The communication key is used by  applications
136              that directly manage collective membership through a fabric man‐
137              agement agent or resource manager.  The key is  used  to  convey
138              that results of the membership setup to the underlying provider.
139              The use and format of a communication  key  is  fabric  provider
140              specific.
141
142       flags  If  the flag FI_UNIVERSE is set, then the AV set will be created
143              containing all addresses stored in the AV.
144
145   fi_av_set_union
146       The AV set union call adds all addresses in the source AV set that  are
147       not  in the destination AV set to the destination AV set.  Where order‐
148       ing matters, the newly inserted addresses are placed at the end of  the
149       AV set.
150
151   fi_av_set_intersect
152       The  AV set intersect call remove all addresses from the destination AV
153       set that are not also members of the source AV set.  The order  of  the
154       addresses in the destination AV set is unchanged.
155
156   fi_av_set_diff
157       The  AV set difference call removes all address from the destination AV
158       set that are also members of the source AV set.  The order of  the  ad‐
159       dresses in the destination AV set is unchanged.
160
161   fi_av_set_insert
162       The  AV set insert call appends the specified address to the end of the
163       AV set.
164
165   fi_av_set_remove
166       The AV set remove call removes the specified address from the given  AV
167       set.  The order of the remaining addresses in the AV set is unchanged.
168
169   fi_av_set_addr
170       Returns  an  address  that  may be used to communicate with all current
171       members of an AV set.  This is a local operation only that does not in‐
172       volve network communication.  The returned address may be used as input
173       into fi_join_collective.  Note that attempting to use the  address  re‐
174       turned  from  fi_av_set_addr  (e.g.   passing it to fi_join_collective)
175       while simultaneously modifying the addresses stored in an  AV  set  re‐
176       sults in undefined behavior.
177
178   fi_close
179       Closes  an  AV  set and releases all resources associated with it.  Any
180       operations active at the time an AV set is closed will be aborted, with
181       the result of the collective undefined.
182

NOTES

184       Developers who are familiar with MPI will find that AV sets are similar
185       to MPI groups, and may act as a direct mapping in some,  but  not  all,
186       situations.
187

RETURN VALUES

189       Returns 0 on success.  On error, a negative value corresponding to fab‐
190       ric errno is returned.  Fabric errno values are defined in  rdma/fi_er‐
191       rno.h.
192

SEE ALSO

194       fi_av(3), fi_collective(3)
195

AUTHORS

197       OpenFabrics.
198
199
200
201Libfabric Programmer's Manual     2020-03-20                      fi_av_set(3)
Impressum