1PMFETCHGROUP(3)            Library Functions Manual            PMFETCHGROUP(3)
2
3
4

NAME

6       pmCreateFetchGroup,  pmExtendFetchGroup_item, pmExtendFetchGroup_indom,
7       pmExtendFetchGroup_event,  pmExtendFetchGroup_timestamp,  pmFetchGroup,
8       pmGetFetchGroupContext,  pmClearFetchGroup,  pmDestroyFetchGroup - sim‐
9       plified performance metrics value fetch and conversion
10

C SYNOPSIS

12       #include <pcp/pmapi.h>
13
14       int pmCreateFetchGroup(pmFG *ptr, int type, const char *name);
15       int pmExtendFetchGroup_item(pmFG pmfg, const char *metric, const char
16               *instance, const char *scale, pmAtomValue *out_value, int
17               out_type, int out_sts);
18       int pmExtendFetchGroup_indom(pmFG pmfg, const char *metric, const char
19               *scale, int out_inst_codes[], char *out_inst_names[],
20               pmAtomValue out_values[], int out_type, int out_stss[],
21               unsigned int out_maxnum, unsigned int *out_num, int *out_sts);
22       int pmExtendFetchGroup_event(pmFG pmfg, const char *metric, const char
23               *instance, const char *field, const char *scale, struct
24               timespec out_times[], pmAtomValue out_values[], int out_type,
25               int out_stss[], unsigned int out_maxnum, unsigned int *out_num,
26               int *out_sts);
27       int pmExtendFetchGroup_timestamp(pmFG pmfg, struct timeval *out_value);
28       int pmGetFetchGroupContext(pmFG pmfg);
29       int pmFetchGroup(pmFG pmfg);
30       int pmClearFetchGroup(pmFG pmfg);
31       int pmDestroyFetchGroup(pmFG pmfg);
32
33       cc ... -lpcp
34

DESCRIPTION

36       The  fetchgroup  functions  implement a registration-based mechanism to
37       fetch groups of performance metrics, including automation  for  general
38       unit, rate and type conversions as well as convenient instance and val‐
39       ue encodings.  They constitute a powerful and  compact  alternative  to
40       the  classic  Performance  Metrics  Application  Programming  Interface
41       (PMAPI(3)) sequence of separate lookup, check, fetch, iterate,  extract
42       and convert functions.
43
44       The  general  idea consists of two stages.  In the setup stage, the ap‐
45       plication identifies metrics of interest by name and with desired  con‐
46       versions,  and  register a unique pmAtomValue output location where the
47       fetchgroup system is to later deposit the result.  It is also  possible
48       to  identify  a  metric  with an instance domain, and register a unique
49       vector of pmAtomValue objects for them.  In the  operation  stage,  one
50       simple pmFetchGroup function fetches, decodes, converts, and stores all
51       metrics to their destinations, where the application can read them  di‐
52       rectly.   This  function  may  be  called repeatedly, and each time new
53       pmAtomValue values will be stored in the same destinations.  Rate  con‐
54       versions between consecutive samples may be requested.
55
56       Each  fetchgroup  is associated with a private PMAPI context, so it can
57       manipulate instance profiles and other such  state  without  disrupting
58       other contexts.  The instance profile is manipulated to optimize fetch‐
59       es of individual items, even if some are derived metrics.  This private
60       PMAPI  context belongs to the fetchgroup, is used for all of its inter‐
61       nal operations, and will be destroyed.
62
63       Multiple fetchgroups  may  be  used  concurrently,  independently.   An
64       opaque  type  pmFG is used to identify a fetchgroup, which is passed to
65       all related function calls.
66
67   Creating a fetchgroup
68       int pmCreateFetchGroup(pmFG *ptr, int type, const char *name);
69
70       This function creates a new fetchgroup, associated  with  a  new  PMAPI
71       context.   The  type and name parameters are relayed to pmNewContext(3)
72       for creation of the context.  The fetchgroup identifier is returned up‐
73       on success through the ptr pointer.  This object is later used as a pa‐
74       rameter to all other fetchgroup functions.  The private  PMAPI  context
75       may be accessed with pmGetFetchGroupContext, if required.
76
77       The normal function return code is zero, and ptr is set.  This function
78       may fail in case of pmNewContext or memory  allocation  errors.   Those
79       are indicated with a negative return code and a cleared ptr value.
80
81   Getting the private PMAPI context
82       int pmGetFetchGroupContext(pmFG pmfg);
83
84       This  function  returns  the  private  PMAPI  context used by the given
85       fetchgroup.  It may be safely used to adjust some configuration parame‐
86       ters of the context, such as via pmSetMode(3), before fetchgroup exten‐
87       sion and fetching begins.
88
89       However, mutation of this context by PMAPI functions  after  this  time
90       may  disrupt  fetchgroup  functionality.  For example, a pmSetMode call
91       could invalidate one rate-conversion time-step.
92
93       The normal function return code is the context number.
94
95   Extending a fetchgroup with a metric instance of interest
96       int pmExtendFetchGroup_item(pmFG pmfg, const char *metric, const char
97               *instance, const char *scale, pmAtomValue *out_value, int
98               out_type, int *out_sts);
99
100       This function registers interest in a single metric  and  optional  in‐
101       stance.   The  metric  name is given in the mandatory metric parameter,
102       which is checked immediately via pmLookupName(3) and other  calls.   If
103       and only if the metric has an instance domain, the specific instance of
104       interest may be named by the instance parameter, which is checked imme‐
105       diately  via  pmNameInDom(3);  otherwise  pass NULL.  If the fetchgroup
106       context is a set of archives, it is possible that the metric / instance
107       pair  is  not  yet defined at the current time origin.  Therefore, this
108       function may attempt to seek to the end of the current set of  archives
109       temporarily to retry the metric / instance lookup.
110
111       The  optional scale parameter specifies desired unit/scale/rate conver‐
112       sions for the metric value.  It can take the following values:
113
114       NULL
115           No unit/scale conversion.  If metric has PM_SEM_COUNTER  semantics,
116           perform rate conversion.
117
118       rate
119           Perform  rate conversion regardless of semantics, and no unit/scale
120           conversion.
121
122       instant
123           Perform  no  rate  conversion  regardless  of  semantics,  and   no
124           unit/scale conversion.
125
126       EXPRESSION
127           Perform  unit/scale/rate conversion as specified by the EXPRESSION,
128           which is parsed by pmParseUnitsStr(3).  This may be useful  to  as‐
129           sert  a  canonical scaling for the resulting metric value, indepen‐
130           dent of PCP version or configuration.   Dimensionality  must  match
131           the  metric,  except if rate conversion is requested, in which case
132           the time dimension must be one smaller than the metric's  time  di‐
133           mension.   Note  that  the  type  of rate conversion performed here
134           matches the rate(x) function in derived metric expressions, in that
135           it  is calculated as the naive difference between previous and cur‐
136           rent values of a metric, divided by elapsed time.  For example,  if
137           a  counter  wraps around, or a non-counter value decreases, a nega‐
138           tive output rate may be computed.
139
140       The optional but usual out_value parameter  specifies  the  pmAtomValue
141       where  the  converted  result  should later be stored.  If the value is
142       NULL, fetching and conversions will be attempted, and  possible  errors
143       reported, but the result tossed away.  The mandatory out_type parameter
144       specifes the PM_TYPE_* requested for the output  value.   It  need  not
145       match  the  metric's native type, as the fetchgroup facility is capable
146       of casting between all supported types (including to and from strings).
147
148       Any errors subsequently encountered  during  fetching,  unit/scale/rate
149       conversion,  or  casting,  will  result in the assignment of a sentinel
150       value to the output pmAtomValue (see the ``UNUSUAL SITUATIONS'' section
151       below).   In  addition, if the optional out_sts parameter is specified,
152       an appropriate PMAPI error code will be stored there.
153
154       As a review, only the pmfg, metric, and out_type parameters are  manda‐
155       tory.  Others may be NULL to indicate applicaton disinterest.
156
157       The  normal  function  return  code is zero.  This function may fail in
158       case of various lookup, type- and conversion- checking  errors.   Those
159       are indicated with a negative return code.
160
161   Extending a fetchgroup with a metric instance domain of interest
162       int pmExtendFetchGroup_indom(pmFG pmfg, const char* metric, const char
163               *scale, int out_inst_codes[], char *out_inst_names[],
164               pmAtomValue out_values[], int out_type, int out_stss[],
165               unsigned int out_maxnum, unsigned int *out_num, int *out_sts);
166
167       This function generalizes the pmExtendFetchGroup_item function by  reg‐
168       istering  interest in a whole instance domain.  Therefore, the function
169       registers preallocated vectors for output variables (instead of a  sin‐
170       gleton).  Instances will be stored in sorted order in elements of those
171       vectors.  The concepts are otherwise the same.
172
173       The metric name is specified by the mandatory metric  parameter.   Note
174       that it may refer to a metric without an instance domain, in which case
175       the single output value will appear as one unnamed instance.
176
177       The optional scale parameter specifies desired unit/scale/rate  conver‐
178       sions for the metric value, same as above.
179
180       The  optional  out_inst_codes parameter specifies a vector of integers,
181       where the raw instance number of the fetched metrics  should  later  be
182       stored.
183
184       The  optional  out_inst_names  parameter specifies a vector of strings,
185       where the instance names of the fetched metrics should later be stored.
186       If  an  instance  does not have a corresponding name, a NULL pointer is
187       stored instead.  The application must not modify or free(3) strings  in
188       that vector.
189
190       The optional out_values parameter specifies a vector of pmAtomValue ob‐
191       jects where the converted result should later be stored.  The mandatory
192       out_type parameter specifies the PM_TYPE_* requested for the all output
193       values, same as above.
194
195       The optional out_stss parameter specifies a vector  of  integers  where
196       per-instance error codes should be stored.
197
198       The  mandatory out_maxnum parameter specifies the number of elements of
199       the vectors above.  In other words, it tells the fetchgroup the maximum
200       number of instances which are expected.  The optional out_num parameter
201       specifies an integer where the actual number of instances should  later
202       be  stored.  It will range between 0 and out_maxnum.  It is initialized
203       to 0 by this function.
204
205       Finally, the optional out_sts parameter  specifies  a  single  location
206       where  an  integer  status  code  for the overall fetch for this metric
207       should be stored.  Normally, this will be zero.  Other  than  a  severe
208       fetch  error,  one  may  see  a PM_ERR_TOOBIG here if the number of in‐
209       stances actually encountered was larger than out_maxnum.
210
211       Any errors subsequently encountered  during  fetching,  unit/scale/rate
212       conversion,  or  casting,  will  result in the assignment of a sentinel
213       value to the appropriate output pmAtomValue (see the  ``UNUSUAL  SITUA‐
214       TIONS''  section below).  In addition, if the optional out_stss parame‐
215       ter was specified, a PMAPI error code will be stored in the appropriate
216       position.
217
218       As a review, only the pmfg, metric, out_type, and out_maxnum parameters
219       are mandatory.  Others may be NULL to indicate applicaton disinterest.
220
221       The normal function return code is zero.  This  function  may  fail  in
222       case  of  various lookup, type- and conversion- checking errors.  Those
223       are indicated with a negative return code.
224
225   Extending a fetchgroup with an event field
226       int pmExtendFetchGroup_event(pmFG pmfg, const char *metric, const char
227               *instance, const char *field, const char *scale, struct
228               timespec out_times[], pmAtomValue out_values[], int out_type,
229               int out_stss[], unsigned int out_maxnum, unsigned int *out_num,
230               int *out_sts);
231
232       This function registers interest in all instances of one field  of  all
233       records  of  an  event metric.  Since event metrics may return multiple
234       records per fetch, and each record may have multiple fields of a  given
235       field  metric  type,  this  function registers preallocated vectors for
236       output variables,  similarly  to  pmExtendFetchGroup_indom.   They  are
237       filled in temporal/sequential order.
238
239       The  metric  name  is  specified by the mandatory metric parameter.  It
240       must be of PM_TYPE_EVENT.  If the metric has an  instance  domain,  the
241       instance parameter is mandatory to identify the instance of interest.
242
243       The  field  to extract from event records is specified by the mandatory
244       field parameter, which is a metric name of normal scalar type.   As  is
245       typical for event field metrics, it should not have an instance domain.
246       The optional scale parameter specifies desired  unit/scale  conversions
247       on  this  metric value.  Rate conversions are not available, because of
248       ambiguity about which previous value to compute rates from.
249
250       The  optional  out_times  parameter  specifies  a  vector  of  timespec
251       structs, which will receive a copy of the timestamp of the event record
252       where each particular field was found.
253
254       The optional out_values parameter specifies a vector of pmAtomValue ob‐
255       jects where the converted result should later be stored.  The mandatory
256       out_type parameter specifies the PM_TYPE_*  requested  for  the  output
257       values.
258
259       The  optional  out_stss  parameter specifies a vector of integers where
260       per-field error codes should be stored.
261
262       The mandatory out_maxnum parameter specifies the number of elements  of
263       the vectors above.  In other words, it tells the fetchgroup the maximum
264       number of instances which are expected.  The optional out_num parameter
265       specifies  an  integer  where the the actual number of instances should
266       later be stored.  It will range between zero  and  out_maxnum.   It  is
267       initialized to zero by this function.
268
269       Finally,  the  optional  out_sts  parameter specifies a single location
270       where an integer status code for the  overall  fetch  for  this  metric
271       should  be stored.  Normally, this will be zero, even if no event field
272       values were found (out_num would then be zero).  Other  than  a  severe
273       fetch  error,  one may see a PM_ERR_TOOBIG here if the number of fields
274       actually encountered was larger than out_maxnum.
275
276       Any errors subsequently encountered during fetching, unit/scale conver‐
277       sion,  or casting, will result in the assignment of a sentinel value to
278       the appropriate output pmAtomValue (see the ``UNUSUAL SITUATIONS'' sec‐
279       tion below).  In addition, if the optional out_stss parameter was spec‐
280       ified, a PMAPI error code will be stored in the appropriate position.
281
282       As a review, only the pmfg, metric, field, out_type, and out_maxnum pa‐
283       rameters are mandatory.  Others may be NULL to indicate applicaton dis‐
284       interest.
285
286       The normal function return code is zero.  This  function  may  fail  in
287       case  of  various lookup, type- and conversion- checking errors.  Those
288       are indicated with a negative return code.
289
290   Extending a fetchgroup with the fetch timestamp
291       int pmExtendFetchGroup_timestamp(pmFG pmfg, struct timeval *out_value);
292
293       This function registers interest in the  pmResult  timestamp.   If  the
294       out_value  pointer  is non-NULL, at every future pmFetchGroup call, the
295       corresponding result timestamp will be copied there.
296
297   Fetching all metrics in a fetchgroup
298       int pmFetchGroup(pmFG pmfg);
299
300       This function performs one pmFetch on its private  PMAPI  context,  in‐
301       cluding  all  the metrics that were registered via prior pmExtendFetch‐
302       Group_* calls.  It runs all the data extraction and  conversion  opera‐
303       tions necessary to populate all the requested output variables.
304
305       The  normal function return code is zero or positive, as per the under‐
306       lying pmFetch function.  This function may fail in case of severe fetch
307       errors, which are indicated with a negative return code.
308
309       In  the case of per-metric availability or conversion errors, or severe
310       fetch errors, output variables are reset to sentinel values  and  indi‐
311       vidual error codes are set.  PM_ERR_AGAIN signals rate-conversion fail‐
312       ure due to lack of a previous value.
313
314       However, temporarily absent metrics with discrete semantics are  exempt
315       from some sentinel/error processing: if a pmFetchGroup fails to collect
316       a result for a discrete metric  (pmResult  pmValueSet.numval==0),  then
317       the  last  seen  valid value (if any) is retained.  This is intended to
318       ease the processing of sets of archives with a mixture of once- and re‐
319       peatedly-sampled metrics.
320
321   Clearing a fetchgroup
322       int pmClearFetchGroup(pmFG pmfg);
323
324       When  the  current  fetch state of a fetchgroup is no longer needed, it
325       may be explicitly reset with this function.  It releases any dynamical‐
326       ly  stored  state but keeps the private PMAPI context intact for subse‐
327       quent use (i.e. no change to the context is made at all and the context
328       remains  at  the  current fetch offset).  It frees any pointers such as
329       indom instance names or strings that may have  been  stored  in  output
330       variables.
331
332   Destroying a fetchgroup
333       int pmDestroyFetchGroup(pmFG pmfg);
334
335       When  the  fetchgroup  is  no longer needed, it may be explicitly freed
336       with this function.  It releases any dynamically stored state, as  well
337       as the private PMAPI context.  It clears frees any pointers such as in‐
338       dom instance names or strings that may have been stored in output vari‐
339       ables.
340

EXAMPLE

342       The  following program demonstrates fetchgroup usage.  Run it with dif‐
343       ferent $PCP_DISK_UNITS environment variables to see different unit/rate
344       conversion in effect.
345
346       #include <pcp/pmapi.h>
347       #include <stdio.h>
348
349       #define pcpassert(sts) \
350           while (sts<0) { fprintf(stderr, "%s\n", pmErrStr(sts)); exit(42); }
351
352       int main()
353       {
354           pmFG fg;
355           pmAtomValue v, v2;
356           enum { v3_maxnum = 100 };
357           pmAtomValue v3_values[v3_maxnum];
358           char *v3_names[v3_maxnum];
359           int v3_stss[v3_maxnum];
360           unsigned int v3_num;
361           int sts, i;
362           char *diskunits = getenv("PCP_DISK_UNITS");
363           struct timeval t;
364
365           sts = pmCreateFetchGroup(&fg, PM_CONTEXT_HOST, "local:");
366           pcpassert(sts);
367           sts = pmExtendFetchGroup_item(fg, "kernel.all.load", "1 minute",
368                                         NULL, &v, PM_TYPE_FLOAT, NULL);
369           pcpassert(sts);
370           sts = pmExtendFetchGroup_item(fg, "kernel.all.idletime", NULL,
371                                         "hour", &v2, PM_TYPE_DOUBLE, NULL);
372           pcpassert(sts);
373           sts = pmExtendFetchGroup_indom(fg, "disk.dev.total", diskunits,
374                                          NULL, v3_names,
375                                          v3_values, PM_TYPE_STRING,
376                                          v3_stss, v3_maxnum, &v3_num, NULL);
377           pcpassert(sts);
378           sts = pmExtendFetchGroup_timestamp(fg, &t);
379           pcpassert(sts);
380
381           for (i=0; i < 10; i++) {
382               unsigned int j;
383               char stamp[28];
384
385               sts = pmFetchGroup(fg);
386               pcpassert(sts);
387               printf("%s", pmCtime(&t.tv_sec, stamp));
388               printf("1-minute load: %f; idletime: %f h\n", v.f, v2.d);
389               for (j=0; j < v3_num; j++) {
390                   if (v3_stss[j] == 0)
391                        printf("disk %s i/o operations (%s): %s\n",
392                               v3_names[j] ? v3_names[j] : "?",
393                               diskunits ? diskunits : "-",
394                               v3_values[j].cp);
395               }
396               sleep(1);
397           }
398
399           sts = pmDestroyFetchGroup(fg);
400           pcpassert(sts);
401           return 0;
402       }
403

UNUSUAL SITUATIONS

405       The  fetchgroup  API supports only the numeric, string and event metric
406       types.  Aggregates are rejected during pmExtendFetchGroup_*.
407
408       Any strings supplied by the  fetchgroup  API  to  the  application  are
409       "owned" by the API.  The application should consider them read-only, so
410       it should not modify them nor free them.
411
412       Error codes are always negative integers, whether returned from  fetch‐
413       group  functions  as return value, or stored in out_sts type variables.
414       Normal result codes are always zero.
415
416       Because of the unique ways in which extracted data  is  shared  between
417       the  application  and  a  fetchgroup, the functions in this API are not
418       protected by the multi-threading mutexes conventional in other parts of
419       PMAPI.  Specifically, for any given pmFG, it is not safe to concurrent‐
420       ly call two or more fetchgroup API functions, nor to traverse the  reg‐
421       istered  output variables while calling one of the functions.  Instead,
422       the calling application must ensure that only one thread at a time uses
423       these  calls  and  the registered output variables.  On the other hand,
424       concurrency between different pmFG instances is  unrestricted,  because
425       they share no global data.
426
427       Any pointers passed to a successful pmFetchGroupExtent_* call must stay
428       valid throughout the lifetime of the fetchgroup, since future  pmFetch‐
429       Group calls may write into them.
430

DIAGNOSTICS

432       The  fetchgroup  API offers several options for collecting diagnostics.
433       Negative integer error codes may be returned from each function for se‐
434       rious conditions.
435
436       In  addition,  each output pmAtomValue may have a corresponding integer
437       variable, where pmFetchGroup can store  per-metric  per-instance  error
438       codes.
439
440       As  an  alternative,  per-metric per-instance error conditions are also
441       signalled by setting the corresponding pmAtomValue to a sentinel value.
442       If unambiguous and precise error detection is not required, this may be
443       sufficient.  The sentinel value is negative one for all  integers  (in‐
444       cluding  unsigned  integers  - i.e. all bits are set), NaN for floating
445       point types, a NULL pointer for strings, and 0.0s  for  the  timestamp.
446       The fetchgroup API guarantees that once an output pmAtomValue is regis‐
447       tered (during a  successful  pmExtendFetchGroup_*  call),  it  will  be
448       cleared  to  the  sentinel  value or to a valid converted metric value,
449       from the time of registration until the pmDestroyFetchGroup call.
450

SEE ALSO

452       PMAPI(3), pmLookupName(3),  pmFetch(3),  pmParseUnitsStr(3),  pmUseCon‐
453       text(3), pmRegisterDerived(3) and pmExtractValue(3).
454
455
456
457Performance Co-Pilot                  PCP                      PMFETCHGROUP(3)
Impressum