1PMFETCHGROUP(3) Library Functions Manual PMFETCHGROUP(3)
2
6 pmCreateFetchGroup, pmExtendFetchGroup_item, pmExtendFetchGroup_indom,
7 pmExtendFetchGroup_event, pmExtendFetchGroup_timestamp, pmExtendFetch‐
8 Group_timespec, pmExtendFetchGroup_timeval, pmFetchGroup, pmGetFetch‐
9 GroupContext, pmClearFetchGroup, pmDestroyFetchGroup - simplified per‐
10 formance metrics value fetch and conversion
11
13 #include <pcp/pmapi.h>
14 int pmCreateFetchGroup(pmFG *ptr, int type, const char *name);
16 int pmExtendFetchGroup_item(pmFG pmfg, const char *metric, const char
17 *instance, const char *scale, pmAtomValue *out_value, int
18 out_type, int out_sts);
19 int pmExtendFetchGroup_indom(pmFG pmfg, const char *metric, const char
20 *scale, int out_inst_codes[], char *out_inst_names[],
21 pmAtomValue out_values[], int out_type, int out_stss[],
22 unsigned int out_maxnum, unsigned int *out_num, int *out_sts);
23 int pmExtendFetchGroup_event(pmFG pmfg, const char *metric, const char
24 *instance, const char *field, const char *scale, struct
25 timespec out_times[], pmAtomValue out_values[], int out_type,
26 int out_stss[], unsigned int out_maxnum, unsigned int *out_num,
27 int *out_sts);
28 int pmExtendFetchGroup_timestamp(pmFG pmfg, struct timeval *out_value);
29 int pmExtendFetchGroup_timespec(pmFG pmfg, struct timespec *out_value);
30 int pmExtendFetchGroup_timeval(pmFG pmfg, struct timeval *out_value);
31 int pmGetFetchGroupContext(pmFG pmfg);
32 int pmFetchGroup(pmFG pmfg);
33 int pmClearFetchGroup(pmFG pmfg);
34 int pmDestroyFetchGroup(pmFG pmfg);
35 cc ... -lpcp
37
39 The fetchgroup functions implement a registration-based mechanism to
40 fetch groups of performance metrics, including automation for general
41 unit, rate and type conversions as well as convenient instance and val‐
42 ue encodings. They constitute a powerful and compact alternative to
43 the classic Performance Metrics Application Programming Interface
44 (PMAPI(3)) sequence of separate lookup, check, fetch, iterate, extract
45 and convert functions.
46 The general idea consists of two stages. In the setup stage, the ap‐
48 plication identifies metrics of interest by name and with desired con‐
49 versions, and register a unique pmAtomValue output location where the
50 fetchgroup system is to later deposit the result. It is also possible
51 to identify a metric with an instance domain, and register a unique
52 vector of pmAtomValue objects for them. In the operation stage, one
53 simple pmFetchGroup function fetches, decodes, converts, and stores all
54 metrics to their destinations, where the application can read them di‐
55 rectly. This function may be called repeatedly, and each time new
56 pmAtomValue values will be stored in the same destinations. Rate con‐
57 versions between consecutive samples may be requested.
58 Each fetchgroup is associated with a private PMAPI context, so it can
60 manipulate instance profiles and other such state without disrupting
61 other contexts. The instance profile is manipulated to optimize fetch‐
62 es of individual items, even if some are derived metrics. This private
63 PMAPI context belongs to the fetchgroup, is used for all of its inter‐
64 nal operations, and will be destroyed.
65 Multiple fetchgroups may be used concurrently, independently. An
67 opaque type pmFG is used to identify a fetchgroup, which is passed to
68 all related function calls.
69 Creating a fetchgroup
71 int pmCreateFetchGroup(pmFG *ptr, int type, const char *name);
72 This function creates a new fetchgroup, associated with a new PMAPI
74 context. The type and name parameters are relayed to pmNewContext(3)
75 for creation of the context. The fetchgroup identifier is returned up‐
76 on success through the ptr pointer. This object is later used as a pa‐
77 rameter to all other fetchgroup functions. The private PMAPI context
78 may be accessed with pmGetFetchGroupContext, if required.
79 The normal function return code is zero, and ptr is set. This function
81 may fail in case of pmNewContext or memory allocation errors. Those
82 are indicated with a negative return code and a cleared ptr value.
83 Getting the private PMAPI context
85 int pmGetFetchGroupContext(pmFG pmfg);
86 This function returns the private PMAPI context used by the given
88 fetchgroup. It may be safely used to adjust some configuration parame‐
89 ters of the context, such as via pmSetMode(3), before fetchgroup exten‐
90 sion and fetching begins.
91 However, mutation of this context by PMAPI functions after this time
93 may disrupt fetchgroup functionality. For example, a pmSetMode call
94 could invalidate one rate-conversion time-step.
95 The normal function return code is the context number.
97 Extending a fetchgroup with a metric instance of interest
99 int pmExtendFetchGroup_item(pmFG pmfg, const char *metric, const char
100 *instance, const char *scale, pmAtomValue *out_value, int
101 out_type, int *out_sts);
102 This function registers interest in a single metric and optional in‐
104 stance. The metric name is given in the mandatory metric parameter,
105 which is checked immediately via pmLookupName(3) and other calls. If
106 and only if the metric has an instance domain, the specific instance of
107 interest may be named by the instance parameter, which is checked imme‐
108 diately via pmNameInDom(3); otherwise pass NULL. If the fetchgroup
109 context is a set of archives, it is possible that the metric / instance
110 pair is not yet defined at the current time origin. Therefore, this
111 function may attempt to seek to the end of the current set of archives
112 temporarily to retry the metric / instance lookup.
113 The optional scale parameter specifies desired unit/scale/rate conver‐
115 sions for the metric value. It can take the following values:
116 NULL
118 No unit/scale conversion. If metric has PM_SEM_COUNTER semantics,
119 perform rate conversion.
120 rate
122 Perform rate conversion regardless of semantics, and no unit/scale
123 conversion.
124 instant
126 Perform no rate conversion regardless of semantics, and no
127 unit/scale conversion.
128 EXPRESSION
130 Perform unit/scale/rate conversion as specified by the EXPRESSION,
131 which is parsed by pmParseUnitsStr(3). This may be useful to as‐
132 sert a canonical scaling for the resulting metric value, indepen‐
133 dent of PCP version or configuration. Dimensionality must match
134 the metric, except if rate conversion is requested, in which case
135 the time dimension must be one smaller than the metric's time di‐
136 mension. Note that the type of rate conversion performed here
137 matches the rate(x) function in derived metric expressions, in that
138 it is calculated as the naive difference between previous and cur‐
139 rent values of a metric, divided by elapsed time. For example, if
140 a counter wraps around, or a non-counter value decreases, a nega‐
141 tive output rate may be computed.
142 The optional but usual out_value parameter specifies the pmAtomValue
144 where the converted result should later be stored. If the value is
145 NULL, fetching and conversions will be attempted, and possible errors
146 reported, but the result tossed away. The mandatory out_type parameter
147 specifes the PM_TYPE_* requested for the output value. It need not
148 match the metric's native type, as the fetchgroup facility is capable
149 of casting between all supported types (including to and from strings).
150 Any errors subsequently encountered during fetching, unit/scale/rate
152 conversion, or casting, will result in the assignment of a sentinel
153 value to the output pmAtomValue (see the ``UNUSUAL SITUATIONS'' section
154 below). In addition, if the optional out_sts parameter is specified,
155 an appropriate PMAPI error code will be stored there.
156 As a review, only the pmfg, metric, and out_type parameters are manda‐
158 tory. Others may be NULL to indicate applicaton disinterest.
159 The normal function return code is zero. This function may fail in
161 case of various lookup, type- and conversion- checking errors. Those
162 are indicated with a negative return code.
163 Extending a fetchgroup with a metric instance domain of interest
165 int pmExtendFetchGroup_indom(pmFG pmfg, const char* metric, const char
166 *scale, int out_inst_codes[], char *out_inst_names[],
167 pmAtomValue out_values[], int out_type, int out_stss[],
168 unsigned int out_maxnum, unsigned int *out_num, int *out_sts);
169 This function generalizes the pmExtendFetchGroup_item function by reg‐
171 istering interest in a whole instance domain. Therefore, the function
172 registers preallocated vectors for output variables (instead of a sin‐
173 gleton). Instances will be stored in sorted order in elements of those
174 vectors. The concepts are otherwise the same.
175 The metric name is specified by the mandatory metric parameter. Note
177 that it may refer to a metric without an instance domain, in which case
178 the single output value will appear as one unnamed instance.
179 The optional scale parameter specifies desired unit/scale/rate conver‐
181 sions for the metric value, same as above.
182 The optional out_inst_codes parameter specifies a vector of integers,
184 where the raw instance number of the fetched metrics should later be
185 stored.
186 The optional out_inst_names parameter specifies a vector of strings,
188 where the instance names of the fetched metrics should later be stored.
189 If an instance does not have a corresponding name, a NULL pointer is
190 stored instead. The application must not modify or free(3) strings in
191 that vector.
192 The optional out_values parameter specifies a vector of pmAtomValue ob‐
194 jects where the converted result should later be stored. The mandatory
195 out_type parameter specifies the PM_TYPE_* requested for the all output
196 values, same as above.
197 The optional out_stss parameter specifies a vector of integers where
199 per-instance error codes should be stored.
200 The mandatory out_maxnum parameter specifies the number of elements of
202 the vectors above. In other words, it tells the fetchgroup the maximum
203 number of instances which are expected. The optional out_num parameter
204 specifies an integer where the actual number of instances should later
205 be stored. It will range between 0 and out_maxnum. It is initialized
206 to 0 by this function.
207 Finally, the optional out_sts parameter specifies a single location
209 where an integer status code for the overall fetch for this metric
210 should be stored. Normally, this will be zero. Other than a severe
211 fetch error, one may see a PM_ERR_TOOBIG here if the number of in‐
212 stances actually encountered was larger than out_maxnum.
213 Any errors subsequently encountered during fetching, unit/scale/rate
215 conversion, or casting, will result in the assignment of a sentinel
216 value to the appropriate output pmAtomValue (see the ``UNUSUAL SITUA‐
217 TIONS'' section below). In addition, if the optional out_stss parame‐
218 ter was specified, a PMAPI error code will be stored in the appropriate
219 position.
220 As a review, only the pmfg, metric, out_type, and out_maxnum parameters
222 are mandatory. Others may be NULL to indicate applicaton disinterest.
223 The normal function return code is zero. This function may fail in
225 case of various lookup, type- and conversion- checking errors. Those
226 are indicated with a negative return code.
227 Extending a fetchgroup with an event field
229 int pmExtendFetchGroup_event(pmFG pmfg, const char *metric, const char
230 *instance, const char *field, const char *scale, struct
231 timespec out_times[], pmAtomValue out_values[], int out_type,
232 int out_stss[], unsigned int out_maxnum, unsigned int *out_num,
233 int *out_sts);
234 This function registers interest in all instances of one field of all
236 records of an event metric. Since event metrics may return multiple
237 records per fetch, and each record may have multiple fields of a given
238 field metric type, this function registers preallocated vectors for
239 output variables, similarly to pmExtendFetchGroup_indom. They are
240 filled in temporal/sequential order.
241 The metric name is specified by the mandatory metric parameter. It
243 must be of PM_TYPE_EVENT. If the metric has an instance domain, the
244 instance parameter is mandatory to identify the instance of interest.
245 The field to extract from event records is specified by the mandatory
247 field parameter, which is a metric name of normal scalar type. As is
248 typical for event field metrics, it should not have an instance domain.
249 The optional scale parameter specifies desired unit/scale conversions
250 on this metric value. Rate conversions are not available, because of
251 ambiguity about which previous value to compute rates from.
252 The optional out_times parameter specifies a vector of timespec
254 structs, which will receive a copy of the timestamp of the event record
255 where each particular field was found.
256 The optional out_values parameter specifies a vector of pmAtomValue ob‐
258 jects where the converted result should later be stored. The mandatory
259 out_type parameter specifies the PM_TYPE_* requested for the output
260 values.
261 The optional out_stss parameter specifies a vector of integers where
263 per-field error codes should be stored.
264 The mandatory out_maxnum parameter specifies the number of elements of
266 the vectors above. In other words, it tells the fetchgroup the maximum
267 number of instances which are expected. The optional out_num parameter
268 specifies an integer where the the actual number of instances should
269 later be stored. It will range between zero and out_maxnum. It is
270 initialized to zero by this function.
271 Finally, the optional out_sts parameter specifies a single location
273 where an integer status code for the overall fetch for this metric
274 should be stored. Normally, this will be zero, even if no event field
275 values were found (out_num would then be zero). Other than a severe
276 fetch error, one may see a PM_ERR_TOOBIG here if the number of fields
277 actually encountered was larger than out_maxnum.
278 Any errors subsequently encountered during fetching, unit/scale conver‐
280 sion, or casting, will result in the assignment of a sentinel value to
281 the appropriate output pmAtomValue (see the ``UNUSUAL SITUATIONS'' sec‐
282 tion below). In addition, if the optional out_stss parameter was spec‐
283 ified, a PMAPI error code will be stored in the appropriate position.
284 As a review, only the pmfg, metric, field, out_type, and out_maxnum pa‐
286 rameters are mandatory. Others may be NULL to indicate applicaton dis‐
287 interest.
288 The normal function return code is zero. This function may fail in
290 case of various lookup, type- and conversion- checking errors. Those
291 are indicated with a negative return code.
292 Extending a fetchgroup with the fetch timestamp
294 int pmExtendFetchGroup_timestamp(pmFG pmfg, struct timeval *out_value);
295 int pmExtendFetchGroup_timespec(pmFG pmfg, struct timespec *out_value);
296 int pmExtendFetchGroup_timeval(pmFG pmfg, struct timeval *out_value);
297 These functions register interest in the pmHighResResult timestamp. If
299 the out_value pointer is non-NULL, at every future pmFetchGroup call,
300 the corresponding result timestamp will be copied there.
301 Fetching all metrics in a fetchgroup
303 int pmFetchGroup(pmFG pmfg);
304 This function performs one pmFetch on its private PMAPI context, in‐
306 cluding all the metrics that were registered via prior pmExtendFetch‐
307 Group_* calls. It runs all the data extraction and conversion opera‐
308 tions necessary to populate all the requested output variables.
309 The normal function return code is zero or positive, as per the under‐
311 lying pmFetch function. This function may fail in case of severe fetch
312 errors, which are indicated with a negative return code.
313 In the case of per-metric availability or conversion errors, or severe
315 fetch errors, output variables are reset to sentinel values and indi‐
316 vidual error codes are set. PM_ERR_AGAIN signals rate-conversion fail‐
317 ure due to lack of a previous value.
318 However, temporarily absent metrics with discrete semantics are exempt
320 from some sentinel/error processing: if a pmFetchGroup fails to collect
321 a result for a discrete metric (pmHighResResult pmValueSet.numval==0),
322 then the last seen valid value (if any) is retained. This is intended
323 to ease the processing of sets of archives with a mixture of once- and
324 repeatedly-sampled metrics.
325 Clearing a fetchgroup
327 int pmClearFetchGroup(pmFG pmfg);
328 When the current fetch state of a fetchgroup is no longer needed, it
330 may be explicitly reset with this function. It releases any dynamical‐
331 ly stored state but keeps the private PMAPI context intact for subse‐
332 quent use (i.e. no change to the context is made at all and the context
333 remains at the current fetch offset). It frees any pointers such as
334 indom instance names or strings that may have been stored in output
335 variables.
336 Destroying a fetchgroup
338 int pmDestroyFetchGroup(pmFG pmfg);
339 When the fetchgroup is no longer needed, it may be explicitly freed
341 with this function. It releases any dynamically stored state, as well
342 as the private PMAPI context. It clears frees any pointers such as in‐
343 dom instance names or strings that may have been stored in output vari‐
344 ables.
345
347 The following program demonstrates fetchgroup usage. Run it with dif‐
348 ferent $PCP_DISK_UNITS environment variables to see different unit/rate
349 conversion in effect.
350 #include <pcp/pmapi.h>
352 #include <stdio.h>
353 #define pcpassert(sts) \
355 while (sts<0) { fprintf(stderr, "%s\n", pmErrStr(sts)); exit(42); }
356 int main()
358 {
359 pmFG fg;
360 pmAtomValue v, v2;
361 enum { v3_maxnum = 100 };
362 pmAtomValue v3_values[v3_maxnum];
363 char *v3_names[v3_maxnum];
364 int v3_stss[v3_maxnum];
365 unsigned int v3_num;
366 int sts, i;
367 char *diskunits = getenv("PCP_DISK_UNITS");
368 struct timeval t;
369 sts = pmCreateFetchGroup(&fg, PM_CONTEXT_HOST, "local:");
371 pcpassert(sts);
372 sts = pmExtendFetchGroup_item(fg, "kernel.all.load", "1 minute",
373 NULL, &v, PM_TYPE_FLOAT, NULL);
374 pcpassert(sts);
375 sts = pmExtendFetchGroup_item(fg, "kernel.all.idletime", NULL,
376 "hour", &v2, PM_TYPE_DOUBLE, NULL);
377 pcpassert(sts);
378 sts = pmExtendFetchGroup_indom(fg, "disk.dev.total", diskunits,
379 NULL, v3_names,
380 v3_values, PM_TYPE_STRING,
381 v3_stss, v3_maxnum, &v3_num, NULL);
382 pcpassert(sts);
383 sts = pmExtendFetchGroup_timestamp(fg, &t);
384 pcpassert(sts);
385 for (i=0; i < 10; i++) {
387 unsigned int j;
388 char stamp[28];
389 sts = pmFetchGroup(fg);
391 pcpassert(sts);
392 printf("%s", pmCtime(&t.tv_sec, stamp));
393 printf("1-minute load: %f; idletime: %f h\n", v.f, v2.d);
394 for (j=0; j < v3_num; j++) {
395 if (v3_stss[j] == 0)
396 printf("disk %s i/o operations (%s): %s\n",
397 v3_names[j] ? v3_names[j] : "?",
398 diskunits ? diskunits : "-",
399 v3_values[j].cp);
400 }
401 sleep(1);
402 }
403 sts = pmDestroyFetchGroup(fg);
405 pcpassert(sts);
406 return 0;
407 }
408
410 The fetchgroup API supports only the numeric, string and event metric
411 types. Aggregates are rejected during pmExtendFetchGroup_*.
412 Any strings supplied by the fetchgroup API to the application are
414 "owned" by the API. The application should consider them read-only, so
415 it should not modify them nor free them.
416 Error codes are always negative integers, whether returned from fetch‐
418 group functions as return value, or stored in out_sts type variables.
419 Normal result codes are always zero.
420 Because of the unique ways in which extracted data is shared between
422 the application and a fetchgroup, the functions in this API are not
423 protected by the multi-threading mutexes conventional in other parts of
424 PMAPI. Specifically, for any given pmFG, it is not safe to concurrent‐
425 ly call two or more fetchgroup API functions, nor to traverse the reg‐
426 istered output variables while calling one of the functions. Instead,
427 the calling application must ensure that only one thread at a time uses
428 these calls and the registered output variables. On the other hand,
429 concurrency between different pmFG instances is unrestricted, because
430 they share no global data.
431 Any pointers passed to a successful pmFetchGroupExtent_* call must stay
433 valid throughout the lifetime of the fetchgroup, since future pmFetch‐
434 Group calls may write into them.
435
437 The fetchgroup API offers several options for collecting diagnostics.
438 Negative integer error codes may be returned from each function for se‐
439 rious conditions.
440 In addition, each output pmAtomValue may have a corresponding integer
442 variable, where pmFetchGroup can store per-metric per-instance error
443 codes.
444 As an alternative, per-metric per-instance error conditions are also
446 signalled by setting the corresponding pmAtomValue to a sentinel value.
447 If unambiguous and precise error detection is not required, this may be
448 sufficient. The sentinel value is negative one for all integers (in‐
449 cluding unsigned integers - i.e. all bits are set), NaN for floating
450 point types, a NULL pointer for strings, and 0.0s for the timestamp.
451 The fetchgroup API guarantees that once an output pmAtomValue is regis‐
452 tered (during a successful pmExtendFetchGroup_* call), it will be
453 cleared to the sentinel value or to a valid converted metric value,
454 from the time of registration until the pmDestroyFetchGroup call.
455
457 PMAPI(3), pmLookupName(3), pmFetchHighRes(3), pmParseUnitsStr(3),
458 pmUseContext(3), pmRegisterDerived(3) and pmExtractValue(3).
459Performance Co-Pilot PCP PMFETCHGROUP(3)