1fi_cntr(3) Libfabric v1.6.1 fi_cntr(3)
2
6 fi_cntr - Completion and event counter operations
7 fi_cntr_open / fi_close : Allocate/free a counter
9 fi_cntr_read : Read the current value of a counter
11 fi_cntr_readerr : Reads the number of operations which have completed
13 in error.
14 fi_cntr_add : Increment a counter by a specified value
16 fi_cntr_set : Set a counter to a specified value
18 fi_cntr_wait : Wait for a counter to be greater or equal to a threshold
20 value
21
23 #include <rdma/fi_domain.h>
24 int fi_cntr_open(struct fid_domain *domain, struct fi_cntr_attr *attr,
26 struct fid_cntr **cntr, void *context);
27 int fi_close(struct fid *cntr);
29 uint64_t fi_cntr_read(struct fid_cntr *cntr);
31 uint64_t fi_cntr_readerr(struct fid_cntr *cntr);
33 int fi_cntr_add(struct fid_cntr *cntr, uint64_t value);
35 int fi_cntr_adderr(struct fid_cntr *cntr, uint64_t value);
37 int fi_cntr_set(struct fid_cntr *cntr, uint64_t value);
39 int fi_cntr_seterr(struct fid_cntr *cntr, uint64_t value);
41 int fi_cntr_wait(struct fid_cntr *cntr, uint64_t threshold,
43 int timeout);
44
46 domain : Fabric domain
47 cntr : Fabric counter
49 attr : Counter attributes
51 context : User specified context associated with the counter
53 value : Value to increment or set counter
55 threshold : Value to compare counter against
57 timeout : Time in milliseconds to wait. A negative value indicates
59 infinite timeout.
60
62 Counters record the number of requested operations that have completed.
63 Counters can provide a light-weight completion mechanism by suppressing
64 the generation of a full completion event. They are useful for appli‐
65 cations that only need to know the number of requests that have com‐
66 pleted, and not details about each request. For example, counters may
67 be useful for implementing credit based flow control or tracking the
68 number of remote processes which have responded to a request.
69 Counters typically only count successful completions. However, if an
71 operation completes in error, it may increment an associated error
72 value. That is, a counter actually stores two distinct values, with
73 error completions updating an error specific value.
74 fi_cntr_open
76 fi_cntr_open allocates a new fabric counter. The properties and behav‐
77 ior of the counter are defined by struct fi_cntr_attr.
78 struct fi_cntr_attr {
80 enum fi_cntr_events events; /* type of events to count */
81 enum fi_wait_obj wait_obj; /* requested wait object */
82 struct fid_wait *wait_set; /* optional wait set */
83 uint64_t flags; /* operation flags */
84 };
85 events : A counter captures different types of events. The specific
87 type which is to counted are one of the following:
88 · FI_CNTR_EVENTS_COMP : The counter increments for every successful
90 completion that occurs on an associated bound endpoint. The type of
91 completions -- sends and/or receives -- which are counted may be
92 restricted using control flags when binding the counter and the end‐
93 point. Counters increment on all successful completions, separately
94 from whether the operation generates an entry in an event queue.
95 wait_obj : Counters may be associated with a specific wait object.
97 Wait objects allow applications to block until the wait object is sig‐
98 naled, indicating that a counter has reached a specific threshold.
99 Users may use fi_control to retrieve the underlying wait object associ‐
100 ated with a counter, in order to use it in other system calls. The
101 following values may be used to specify the type of wait object associ‐
102 ated with a counter: FI_WAIT_NONE, FI_WAIT_UNSPEC, FI_WAIT_SET,
103 FI_WAIT_FD, and FI_WAIT_MUTEX_COND. The default is FI_WAIT_NONE.
104 · FI_WAIT_NONE : Used to indicate that the user will not block (wait)
106 for events on the counter.
107 · FI_WAIT_UNSPEC : Specifies that the user will only wait on the
109 counter using fabric interface calls, such as fi_cntr_wait. In this
110 case, the underlying provider may select the most appropriate or
111 highest performing wait object available, including custom wait mech‐
112 anisms. Applications that select FI_WAIT_UNSPEC are not guaranteed
113 to retrieve the underlying wait object.
114 · FI_WAIT_SET : Indicates that the event counter should use a wait set
116 object to wait for events. If specified, the wait_set field must
117 reference an existing wait set object.
118 · FI_WAIT_FD : Indicates that the counter should use a file descriptor
120 as its wait mechanism. A file descriptor wait object must be usable
121 in select, poll, and epoll routines. However, a provider may signal
122 an FD wait object by marking it as readable, writable, or with an
123 error.
124 · FI_WAIT_MUTEX_COND : Specifies that the counter should use a pthread
126 mutex and cond variable as a wait object.
127 wait_set : If wait_obj is FI_WAIT_SET, this field references a wait
129 object to which the event counter should attach. When an event is
130 added to the event counter, the corresponding wait set will be signaled
131 if all necessary conditions are met. The use of a wait_set enables an
132 optimized method of waiting for events across multiple event counters.
133 This field is ignored if wait_obj is not FI_WAIT_SET.
134 flags : Flags are reserved for future use, and must be set to 0.
136 fi_close
138 The fi_close call releases all resources associated with a counter.
139 When closing the counter, there must be no opened endpoints, transmit
140 contexts, receive contexts or memory regions associated with the
141 counter. If resources are still associated with the counter when
142 attempting to close, the call will return -FI_EBUSY.
143 fi_cntr_control
145 The fi_cntr_control call is used to access provider or implementation
146 specific details of the counter. Access to the counter should be seri‐
147 alized across all calls when fi_cntr_control is invoked, as it may re‐
148 direct the implementation of counter operations. The following control
149 commands are usable with a counter:
150 FI_GETOPSFLAG (uint64_t *) : Returns the current default operational
152 flags associated with the counter.
153 FI_SETOPSFLAG (uint64_t *) : Modifies the current default operational
155 flags associated with the counter.
156 FI_GETWAIT (void **) : This command allows the user to retrieve the
158 low-level wait object associated with the counter. The format of the
159 wait-object is specified during counter creation, through the counter
160 attributes. See fi_eq.3 for addition details using control with
161 FI_GETWAIT.
162 fi_cntr_read
164 The fi_cntr_read call returns the current value of the counter.
165 fi_cntr_readerr
167 The read error call returns the number of operations that completed in
168 error and were unable to update the counter.
169 fi_cntr_add
171 This adds the user-specified value to the counter.
172 fi_cntr_adderr
174 This adds the user-specified value to the error value of the counter.
175 fi_cntr_set
177 This sets the counter to the specified value.
178 fi_cntr_seterr
180 This sets the error value of the counter to the specified value.
181 fi_cntr_wait
183 This call may be used to wait until the counter reaches the specified
184 threshold, or until an error or timeout occurs. Upon successful return
185 from this call, the counter will be greater than or equal to the input
186 threshold value.
187 If an operation associated with the counter encounters an error, it
189 will increment the error value associated with the counter. Any change
190 in a counter's error value will unblock any thread inside fi_cntr_wait.
191 If the call returns due to timeout, -FI_ETIMEDOUT will be returned.
193 The error value associated with the counter remains unchanged.
194 It is invalid for applications to call this function if the counter has
196 been configured with a wait object of FI_WAIT_NONE or FI_WAIT_SET.
197
199 Returns 0 on success. On error, a negative value corresponding to fab‐
200 ric errno is returned.
201 fi_cntr_read / fi_cntr_readerr : Returns the current value of the
203 counter.
204 Fabric errno values are defined in rdma/fi_errno.h.
206
208 In order to support a variety of counter implementations, updates made
209 to counter values (e.g. fi_cntr_set or fi_cntr_add) may not be immedi‐
210 ately visible to counter read operations (i.e. fi_cntr_read or
211 fi_cntr_readerr). A small, but undefined, delay may occur between the
212 counter changing and the reported value being updated. However, a
213 final updated value will eventually be reflected in the read counter
214 value, with the order of the updates maintained.
215
217 fi_getinfo(3), fi_endpoint(3), fi_domain(3), fi_eq(3), fi_poll(3)
218
220 OpenFabrics.
221Libfabric Programmer's Manual 2017-09-14 fi_cntr(3)