1PMFETCHGROUP(3) Library Functions Manual PMFETCHGROUP(3)
2
6 pmCreateFetchGroup, pmExtendFetchGroup_item, pmExtendFetchGroup_indom,
7 pmExtendFetchGroup_event, pmExtendFetchGroup_timestamp, pmFetchGroup,
8 pmGetFetchGroupContext, pmClearFetchGroup, pmDestroyFetchGroup - sim‐
9 plified performance metrics value fetch and conversion
10
12 #include <pcp/pmapi.h>
13 int pmCreateFetchGroup(pmFG *ptr, int type, const char *name);
15 int pmExtendFetchGroup_item(pmFG pmfg, const char *metric, const char
16 *instance, const char *scale, pmAtomValue *out_value, int
17 out_type, int out_sts);
18 int pmExtendFetchGroup_indom(pmFG pmfg, const char *metric, const char
19 *scale, int out_inst_codes[], char *out_inst_names[],
20 pmAtomValue out_values[], int out_type, int out_stss[],
21 unsigned int out_maxnum, unsigned int *out_num, int *out_sts);
22 int pmExtendFetchGroup_event(pmFG pmfg, const char *metric, const char
23 *instance, const char *field, const char *scale, struct
24 timespec out_times[], pmAtomValue out_values[], int out_type,
25 int out_stss[], unsigned int out_maxnum, unsigned int *out_num,
26 int *out_sts);
27 int pmExtendFetchGroup_timestamp(pmFG pmfg, struct timeval *out_value);
28 int pmGetFetchGroupContext(pmFG pmfg);
29 int pmFetchGroup(pmFG pmfg);
30 int pmClearFetchGroup(pmFG pmfg);
31 int pmDestroyFetchGroup(pmFG pmfg);
32 cc ... -lpcp
34
36 The fetchgroup functions implement a registration-based mechanism to
37 fetch groups of performance metrics, including automation for general
38 unit, rate and type conversions as well as convenient instance and val‐
39 ue encodings. They constitute a powerful and compact alternative to
40 the classic Performance Metrics Application Programming Interface
41 (PMAPI(3)) sequence of separate lookup, check, fetch, iterate, extract
42 and convert functions.
43 The general idea consists of two stages. In the setup stage, the ap‐
45 plication identifies metrics of interest by name and with desired con‐
46 versions, and register a unique pmAtomValue output location where the
47 fetchgroup system is to later deposit the result. It is also possible
48 to identify a metric with an instance domain, and register a unique
49 vector of pmAtomValue objects for them. In the operation stage, one
50 simple pmFetchGroup function fetches, decodes, converts, and stores all
51 metrics to their destinations, where the application can read them di‐
52 rectly. This function may be called repeatedly, and each time new
53 pmAtomValue values will be stored in the same destinations. Rate con‐
54 versions between consecutive samples may be requested.
55 Each fetchgroup is associated with a private PMAPI context, so it can
57 manipulate instance profiles and other such state without disrupting
58 other contexts. The instance profile is manipulated to optimize fetch‐
59 es of individual items, even if some are derived metrics. This private
60 PMAPI context belongs to the fetchgroup, is used for all of its inter‐
61 nal operations, and will be destroyed.
62 Multiple fetchgroups may be used concurrently, independently. An
64 opaque type pmFG is used to identify a fetchgroup, which is passed to
65 all related function calls.
66 Creating a fetchgroup
68 int pmCreateFetchGroup(pmFG *ptr, int type, const char *name);
69 This function creates a new fetchgroup, associated with a new PMAPI
71 context. The type and name parameters are relayed to pmNewContext(3)
72 for creation of the context. The fetchgroup identifier is returned up‐
73 on success through the ptr pointer. This object is later used as a pa‐
74 rameter to all other fetchgroup functions. The private PMAPI context
75 may be accessed with pmGetFetchGroupContext, if required.
76 The normal function return code is zero, and ptr is set. This function
78 may fail in case of pmNewContext or memory allocation errors. Those
79 are indicated with a negative return code and a cleared ptr value.
80 Getting the private PMAPI context
82 int pmGetFetchGroupContext(pmFG pmfg);
83 This function returns the private PMAPI context used by the given
85 fetchgroup. It may be safely used to adjust some configuration parame‐
86 ters of the context, such as via pmSetMode(3), before fetchgroup exten‐
87 sion and fetching begins.
88 However, mutation of this context by PMAPI functions after this time
90 may disrupt fetchgroup functionality. For example, a pmSetMode call
91 could invalidate one rate-conversion time-step.
92 The normal function return code is the context number.
94 Extending a fetchgroup with a metric instance of interest
96 int pmExtendFetchGroup_item(pmFG pmfg, const char *metric, const char
97 *instance, const char *scale, pmAtomValue *out_value, int
98 out_type, int *out_sts);
99 This function registers interest in a single metric and optional in‐
101 stance. The metric name is given in the mandatory metric parameter,
102 which is checked immediately via pmLookupName(3) and other calls. If
103 and only if the metric has an instance domain, the specific instance of
104 interest may be named by the instance parameter, which is checked imme‐
105 diately via pmNameInDom(3); otherwise pass NULL. If the fetchgroup
106 context is a set of archives, it is possible that the metric / instance
107 pair is not yet defined at the current time origin. Therefore, this
108 function may attempt to seek to the end of the current set of archives
109 temporarily to retry the metric / instance lookup.
110 The optional scale parameter specifies desired unit/scale/rate conver‐
112 sions for the metric value. It can take the following values:
113 NULL
115 No unit/scale conversion. If metric has PM_SEM_COUNTER semantics,
116 perform rate conversion.
117 rate
119 Perform rate conversion regardless of semantics, and no unit/scale
120 conversion.
121 instant
123 Perform no rate conversion regardless of semantics, and no
124 unit/scale conversion.
125 EXPRESSION
127 Perform unit/scale/rate conversion as specified by the EXPRESSION,
128 which is parsed by pmParseUnitsStr(3). This may be useful to as‐
129 sert a canonical scaling for the resulting metric value, indepen‐
130 dent of PCP version or configuration. Dimensionality must match
131 the metric, except if rate conversion is requested, in which case
132 the time dimension must be one smaller than the metric's time di‐
133 mension. Note that the type of rate conversion performed here
134 matches the rate(x) function in derived metric expressions, in that
135 it is calculated as the naive difference between previous and cur‐
136 rent values of a metric, divided by elapsed time. For example, if
137 a counter wraps around, or a non-counter value decreases, a nega‐
138 tive output rate may be computed.
139 The optional but usual out_value parameter specifies the pmAtomValue
141 where the converted result should later be stored. If the value is
142 NULL, fetching and conversions will be attempted, and possible errors
143 reported, but the result tossed away. The mandatory out_type parameter
144 specifes the PM_TYPE_* requested for the output value. It need not
145 match the metric's native type, as the fetchgroup facility is capable
146 of casting between all supported types (including to and from strings).
147 Any errors subsequently encountered during fetching, unit/scale/rate
149 conversion, or casting, will result in the assignment of a sentinel
150 value to the output pmAtomValue (see the ``UNUSUAL SITUATIONS'' section
151 below). In addition, if the optional out_sts parameter is specified,
152 an appropriate PMAPI error code will be stored there.
153 As a review, only the pmfg, metric, and out_type parameters are manda‐
155 tory. Others may be NULL to indicate applicaton disinterest.
156 The normal function return code is zero. This function may fail in
158 case of various lookup, type- and conversion- checking errors. Those
159 are indicated with a negative return code.
160 Extending a fetchgroup with a metric instance domain of interest
162 int pmExtendFetchGroup_indom(pmFG pmfg, const char* metric, const char
163 *scale, int out_inst_codes[], char *out_inst_names[],
164 pmAtomValue out_values[], int out_type, int out_stss[],
165 unsigned int out_maxnum, unsigned int *out_num, int *out_sts);
166 This function generalizes the pmExtendFetchGroup_item function by reg‐
168 istering interest in a whole instance domain. Therefore, the function
169 registers preallocated vectors for output variables (instead of a sin‐
170 gleton). Instances will be stored in sorted order in elements of those
171 vectors. The concepts are otherwise the same.
172 The metric name is specified by the mandatory metric parameter. Note
174 that it may refer to a metric without an instance domain, in which case
175 the single output value will appear as one unnamed instance.
176 The optional scale parameter specifies desired unit/scale/rate conver‐
178 sions for the metric value, same as above.
179 The optional out_inst_codes parameter specifies a vector of integers,
181 where the raw instance number of the fetched metrics should later be
182 stored.
183 The optional out_inst_names parameter specifies a vector of strings,
185 where the instance names of the fetched metrics should later be stored.
186 If an instance does not have a corresponding name, a NULL pointer is
187 stored instead. The application must not modify or free(3) strings in
188 that vector.
189 The optional out_values parameter specifies a vector of pmAtomValue ob‐
191 jects where the converted result should later be stored. The mandatory
192 out_type parameter specifies the PM_TYPE_* requested for the all output
193 values, same as above.
194 The optional out_stss parameter specifies a vector of integers where
196 per-instance error codes should be stored.
197 The mandatory out_maxnum parameter specifies the number of elements of
199 the vectors above. In other words, it tells the fetchgroup the maximum
200 number of instances which are expected. The optional out_num parameter
201 specifies an integer where the actual number of instances should later
202 be stored. It will range between 0 and out_maxnum. It is initialized
203 to 0 by this function.
204 Finally, the optional out_sts parameter specifies a single location
206 where an integer status code for the overall fetch for this metric
207 should be stored. Normally, this will be zero. Other than a severe
208 fetch error, one may see a PM_ERR_TOOBIG here if the number of in‐
209 stances actually encountered was larger than out_maxnum.
210 Any errors subsequently encountered during fetching, unit/scale/rate
212 conversion, or casting, will result in the assignment of a sentinel
213 value to the appropriate output pmAtomValue (see the ``UNUSUAL SITUA‐
214 TIONS'' section below). In addition, if the optional out_stss parame‐
215 ter was specified, a PMAPI error code will be stored in the appropriate
216 position.
217 As a review, only the pmfg, metric, out_type, and out_maxnum parameters
219 are mandatory. Others may be NULL to indicate applicaton disinterest.
220 The normal function return code is zero. This function may fail in
222 case of various lookup, type- and conversion- checking errors. Those
223 are indicated with a negative return code.
224 Extending a fetchgroup with an event field
226 int pmExtendFetchGroup_event(pmFG pmfg, const char *metric, const char
227 *instance, const char *field, const char *scale, struct
228 timespec out_times[], pmAtomValue out_values[], int out_type,
229 int out_stss[], unsigned int out_maxnum, unsigned int *out_num,
230 int *out_sts);
231 This function registers interest in all instances of one field of all
233 records of an event metric. Since event metrics may return multiple
234 records per fetch, and each record may have multiple fields of a given
235 field metric type, this function registers preallocated vectors for
236 output variables, similarly to pmExtendFetchGroup_indom. They are
237 filled in temporal/sequential order.
238 The metric name is specified by the mandatory metric parameter. It
240 must be of PM_TYPE_EVENT. If the metric has an instance domain, the
241 instance parameter is mandatory to identify the instance of interest.
242 The field to extract from event records is specified by the mandatory
244 field parameter, which is a metric name of normal scalar type. As is
245 typical for event field metrics, it should not have an instance domain.
246 The optional scale parameter specifies desired unit/scale conversions
247 on this metric value. Rate conversions are not available, because of
248 ambiguity about which previous value to compute rates from.
249 The optional out_times parameter specifies a vector of timespec
251 structs, which will receive a copy of the timestamp of the event record
252 where each particular field was found.
253 The optional out_values parameter specifies a vector of pmAtomValue ob‐
255 jects where the converted result should later be stored. The mandatory
256 out_type parameter specifies the PM_TYPE_* requested for the output
257 values.
258 The optional out_stss parameter specifies a vector of integers where
260 per-field error codes should be stored.
261 The mandatory out_maxnum parameter specifies the number of elements of
263 the vectors above. In other words, it tells the fetchgroup the maximum
264 number of instances which are expected. The optional out_num parameter
265 specifies an integer where the the actual number of instances should
266 later be stored. It will range between zero and out_maxnum. It is
267 initialized to zero by this function.
268 Finally, the optional out_sts parameter specifies a single location
270 where an integer status code for the overall fetch for this metric
271 should be stored. Normally, this will be zero, even if no event field
272 values were found (out_num would then be zero). Other than a severe
273 fetch error, one may see a PM_ERR_TOOBIG here if the number of fields
274 actually encountered was larger than out_maxnum.
275 Any errors subsequently encountered during fetching, unit/scale conver‐
277 sion, or casting, will result in the assignment of a sentinel value to
278 the appropriate output pmAtomValue (see the ``UNUSUAL SITUATIONS'' sec‐
279 tion below). In addition, if the optional out_stss parameter was spec‐
280 ified, a PMAPI error code will be stored in the appropriate position.
281 As a review, only the pmfg, metric, field, out_type, and out_maxnum pa‐
283 rameters are mandatory. Others may be NULL to indicate applicaton dis‐
284 interest.
285 The normal function return code is zero. This function may fail in
287 case of various lookup, type- and conversion- checking errors. Those
288 are indicated with a negative return code.
289 Extending a fetchgroup with the fetch timestamp
291 int pmExtendFetchGroup_timestamp(pmFG pmfg, struct timeval *out_value);
292 This function registers interest in the pmResult timestamp. If the
294 out_value pointer is non-NULL, at every future pmFetchGroup call, the
295 corresponding result timestamp will be copied there.
296 Fetching all metrics in a fetchgroup
298 int pmFetchGroup(pmFG pmfg);
299 This function performs one pmFetch on its private PMAPI context, in‐
301 cluding all the metrics that were registered via prior pmExtendFetch‐
302 Group_* calls. It runs all the data extraction and conversion opera‐
303 tions necessary to populate all the requested output variables.
304 The normal function return code is zero or positive, as per the under‐
306 lying pmFetch function. This function may fail in case of severe fetch
307 errors, which are indicated with a negative return code.
308 In the case of per-metric availability or conversion errors, or severe
310 fetch errors, output variables are reset to sentinel values and indi‐
311 vidual error codes are set. PM_ERR_AGAIN signals rate-conversion fail‐
312 ure due to lack of a previous value.
313 However, temporarily absent metrics with discrete semantics are exempt
315 from some sentinel/error processing: if a pmFetchGroup fails to collect
316 a result for a discrete metric (pmResult pmValueSet.numval==0), then
317 the last seen valid value (if any) is retained. This is intended to
318 ease the processing of sets of archives with a mixture of once- and re‐
319 peatedly-sampled metrics.
320 Clearing a fetchgroup
322 int pmClearFetchGroup(pmFG pmfg);
323 When the current fetch state of a fetchgroup is no longer needed, it
325 may be explicitly reset with this function. It releases any dynamical‐
326 ly stored state but keeps the private PMAPI context intact for subse‐
327 quent use (i.e. no change to the context is made at all and the context
328 remains at the current fetch offset). It frees any pointers such as
329 indom instance names or strings that may have been stored in output
330 variables.
331 Destroying a fetchgroup
333 int pmDestroyFetchGroup(pmFG pmfg);
334 When the fetchgroup is no longer needed, it may be explicitly freed
336 with this function. It releases any dynamically stored state, as well
337 as the private PMAPI context. It clears frees any pointers such as in‐
338 dom instance names or strings that may have been stored in output vari‐
339 ables.
340
342 The following program demonstrates fetchgroup usage. Run it with dif‐
343 ferent $PCP_DISK_UNITS environment variables to see different unit/rate
344 conversion in effect.
345 #include <pcp/pmapi.h>
347 #include <stdio.h>
348 #define pcpassert(sts) \
350 while (sts<0) { fprintf(stderr, "%s\n", pmErrStr(sts)); exit(42); }
351 int main()
353 {
354 pmFG fg;
355 pmAtomValue v, v2;
356 enum { v3_maxnum = 100 };
357 pmAtomValue v3_values[v3_maxnum];
358 char *v3_names[v3_maxnum];
359 int v3_stss[v3_maxnum];
360 unsigned int v3_num;
361 int sts, i;
362 char *diskunits = getenv("PCP_DISK_UNITS");
363 struct timeval t;
364 sts = pmCreateFetchGroup(&fg, PM_CONTEXT_HOST, "local:");
366 pcpassert(sts);
367 sts = pmExtendFetchGroup_item(fg, "kernel.all.load", "1 minute",
368 NULL, &v, PM_TYPE_FLOAT, NULL);
369 pcpassert(sts);
370 sts = pmExtendFetchGroup_item(fg, "kernel.all.idletime", NULL,
371 "hour", &v2, PM_TYPE_DOUBLE, NULL);
372 pcpassert(sts);
373 sts = pmExtendFetchGroup_indom(fg, "disk.dev.total", diskunits,
374 NULL, v3_names,
375 v3_values, PM_TYPE_STRING,
376 v3_stss, v3_maxnum, &v3_num, NULL);
377 pcpassert(sts);
378 sts = pmExtendFetchGroup_timestamp(fg, &t);
379 pcpassert(sts);
380 for (i=0; i < 10; i++) {
382 unsigned int j;
383 char stamp[28];
384 sts = pmFetchGroup(fg);
386 pcpassert(sts);
387 printf("%s", pmCtime(&t.tv_sec, stamp));
388 printf("1-minute load: %f; idletime: %f h\n", v.f, v2.d);
389 for (j=0; j < v3_num; j++) {
390 if (v3_stss[j] == 0)
391 printf("disk %s i/o operations (%s): %s\n",
392 v3_names[j] ? v3_names[j] : "?",
393 diskunits ? diskunits : "-",
394 v3_values[j].cp);
395 }
396 sleep(1);
397 }
398 sts = pmDestroyFetchGroup(fg);
400 pcpassert(sts);
401 return 0;
402 }
403
405 The fetchgroup API supports only the numeric, string and event metric
406 types. Aggregates are rejected during pmExtendFetchGroup_*.
407 Any strings supplied by the fetchgroup API to the application are
409 "owned" by the API. The application should consider them read-only, so
410 it should not modify them nor free them.
411 Error codes are always negative integers, whether returned from fetch‐
413 group functions as return value, or stored in out_sts type variables.
414 Normal result codes are always zero.
415 Because of the unique ways in which extracted data is shared between
417 the application and a fetchgroup, the functions in this API are not
418 protected by the multi-threading mutexes conventional in other parts of
419 PMAPI. Specifically, for any given pmFG, it is not safe to concurrent‐
420 ly call two or more fetchgroup API functions, nor to traverse the reg‐
421 istered output variables while calling one of the functions. Instead,
422 the calling application must ensure that only one thread at a time uses
423 these calls and the registered output variables. On the other hand,
424 concurrency between different pmFG instances is unrestricted, because
425 they share no global data.
426 Any pointers passed to a successful pmFetchGroupExtent_* call must stay
428 valid throughout the lifetime of the fetchgroup, since future pmFetch‐
429 Group calls may write into them.
430
432 The fetchgroup API offers several options for collecting diagnostics.
433 Negative integer error codes may be returned from each function for se‐
434 rious conditions.
435 In addition, each output pmAtomValue may have a corresponding integer
437 variable, where pmFetchGroup can store per-metric per-instance error
438 codes.
439 As an alternative, per-metric per-instance error conditions are also
441 signalled by setting the corresponding pmAtomValue to a sentinel value.
442 If unambiguous and precise error detection is not required, this may be
443 sufficient. The sentinel value is negative one for all integers (in‐
444 cluding unsigned integers - i.e. all bits are set), NaN for floating
445 point types, a NULL pointer for strings, and 0.0s for the timestamp.
446 The fetchgroup API guarantees that once an output pmAtomValue is regis‐
447 tered (during a successful pmExtendFetchGroup_* call), it will be
448 cleared to the sentinel value or to a valid converted metric value,
449 from the time of registration until the pmDestroyFetchGroup call.
450
452 PMAPI(3), pmLookupName(3), pmFetch(3), pmParseUnitsStr(3), pmUseCon‐
453 text(3), pmRegisterDerived(3) and pmExtractValue(3).
454Performance Co-Pilot PCP PMFETCHGROUP(3)