1PMREGISTERDERIVED(3) Library Functions Manual PMREGISTERDERIVED(3)
2
6 pmRegisterDerived, pmRegisterDerivedMetric - register a derived metric
7 name and definition
8
10 #include <pcp/pmapi.h>
11 char *pmRegisterDerived(char *name, char *expr);
13 int pmRegisterDerivedMetric(char *name, char *expr, char **errmsg);
14 cc ... -lpcp
16
18 Derived metrics provide a way of extending the Performance Metrics Name
19 Space (PMNS) with new metrics defined at the PCP client-side using
20 expressions over the existing performance metrics.
21 Typical uses would be to aggregate a number of similar metrics to pro‐
23 vide a higher-level summary metric or to support the ``delta V over
24 delta V'' class of metrics that are not possible in the base data
25 semantics of PCP. An example of the latter class would be the average
26 I/O size, defined as
27 delta(disk.dev.total_bytes) / delta(disk.dev.total)
28 where both of the disk.dev metrics are counters, and what is required
29 is to to sample both metrics, compute the difference between the cur‐
30 rent and previous values and then calculate the ratio of these differ‐
31 ences.
32 The arguments to pmRegisterDerived are the name of the new derived met‐
34 ric and expr is an expression defining how the values of name should be
35 computed.
36 pmRegisterDerivedMetric is the exact functional equivalent to pmRegis‐
38 terDerived except that it provides a simplified model of error han‐
39 dling, where a formatted message is returned via the errmsg parameter.
40 Syntactic checking is performed at the time pmRegisterDerived is
42 called, but semantic checking is deferred until each new PMAPI context
43 is created with pmNewContext(3) or re-established with pmReconnectCon‐
44 text(3), at which time the PMNS and metadata is available to allow
45 semantic checking and the metadata of the derived metrics to be deter‐
46 mined. This means pmRegisterDerived does not apply retrospectively to
47 any open PMAPI contexts, nor to any PMAPI contexts already open at the
48 time pmRegisterDerived is called, so the normal use would be to make
49 all calls to pmRegisterDerived (possibly via pmLoadDerivedConfig(3)) or
50 pmRegisterDerivedMetric and then call pmNewContext(3).
51 name should follow the syntactic rules for the names of performance
53 metrics, namely one or more components separated with a dot (``.''),
54 and each component must begin with an alphabetic followed by zero or
55 more characters drawn from the alphabetics, numerics and underscore
56 (``_''). For more details, refer to PCPIntro(1) and pmns(5).
57 name must be unique across all derived metrics and should not match the
59 name of any regular metric in the PMNS. It is acceptable for name to
60 share some part of its prefix with an existing subtree of the PMNS,
61 e.g. the average I/O size metric above could be named disk.dev.avgsz
62 which would place it amongst the other disk.dev metrics in the PMNS.
63 Alternatively, derived metrics could populate their own subtree of the
64 PMNS, e.g. the average I/O size metric above could be named my.sum‐
65 mary.disk.avgsz.
66 The expression expr follows these syntactic rules:
68 * Terminal elements are either names of existing metrics or numeric
70 constants. Recursive definitions are not allowed, so only the names
71 of regular metrics (not other derived metrics) may be used. Numeric
72 constants are either integers constrained to the precision of 32-bit
73 unsigned integers or double precision floating point numbers.
74 * The usual binary arithmetic operators are supported, namely addition
76 (``+''), subtraction (``-''), multiplication (``*'') and division
77 (``/'') with the normal precedence rules where multiplication and
78 division have higher precedence than addition and subtraction, so
79 a+b*c is evaluated as a+(b*c).
80 * Unary negation may be used, e.g. -3*some.metric.
82 * C-style relational operators are supported, namely ``<'', ``<='',
84 ``=='', ``>='', ``>'' and ``!=''. Relational expresssions return a
85 value as a 32-bit unsigned number being 0 for false and 1 for true.
86 The expected operator precedence rules apply, so arithmetic operators
87 have higher precedence than relational operators, and a-b>c+d is
88 evaluated as (a-b)>(c+d). All the relational operators have equal
89 precedence, so the (slightly odd) expression involving consecutive
90 relational operators a>b!=c is evaluated as (a>b)!=c.
91 * C-style boolean operators are supported, namely and (``&&'') and or
93 (``||''). Boolean expresssions return a value as a 32-bit unsigned
94 number being 0 for false and 1 for true. The expected operator
95 precedence rules apply, so relational operators have higher prece‐
96 dence than boolean operators, and a>b*c&&d<=e+f is evaluated as
97 (a>(b*c))&&(d<=(e+f)). Both the boolean operators have equal prece‐
98 dence, so the expression involving consecutive boolean operators
99 a>=b||b>c&&d!=e||f>g is evaluated as
100 (((a>=b)||(b>c))&&(d!=e))||(f>g).
101 * Additionally, the ``!'' operator may be used to negate a boolean or
103 relational expression, returning a value as a 32-bit unsigned number
104 being 0 for false and 1 for true. The expected operator precedence
105 rules apply, so boolean (and relational) operators have higher prece‐
106 dence than boolean negation, and !a>b||c<d is evaluated as
107 !((a>b)||(c<d)), while !a<b+c is evaluated as !(a<(b+c)).
108 * C-style ternary conditional expressions are supported. In general
110 terms the expression check ? foo : bar is evaluated as foo (the
111 ``true'' operand) if check (the ``guard'') is true, else the expres‐
112 sion evaluates to bar (the ``false'' operand). Some special semantic
113 rules apply to the ``guard'' expression and the other two operand
114 expressions:
115 (a) Each expression may involve a singular value or a set of values
116 (when the expression involves one or more metrics with an
117 instance domain).
118 (b) All expressions with a set of values must be defined over the
119 same instance domain.
120 (c) Both operand expressions must have the same metadata, so the same
121 metric type, semantics and units (dimension and scale).
122 (d) The ``guard'' expression must have an aritmetic or relational or
123 boolean value, so that it can be evaluated as 0 for false, else
124 true.
125 (e) If the ``guard'' expression has a singular value and one or more
126 of the other operand expressions involves an instance domain, the
127 ``guard'' applies to all instances.
128 (f) If the ``guard'' expression has a set of values and one or more
129 of the other operand expressions involves an instance domain, the
130 ``guard'' is evaluated once for each instance (there must be one
131 instance domain as per rule (b) above).
132 (g) If one of the operand expressions has a singular value and the
133 other has a set of values, and the singular value is selected
134 based on the evaluation of the ``guard'', then the result is a
135 set of values (all the same) with instance enumeration being
136 taken from the other operand expression. For example in the
137 expression: foo ? scalar : set, if foo is true, then the result
138 is a set of values (all having the same value, scalar) over the
139 instance domain of set.
140 * Numeric constants can also be specified using the mkconst() construc‐
142 tor which takes a number of arguments: the first is a numeric con‐
143 stant (either integer or floating point), then follow one or more
144 parameters of the form tag=value or tag= where the allowed values of
145 tag and value are as follows:
146 ┌──────────┬───────────────────────────────────────────────┐
148 │ tag │ value │
149 ├──────────┼───────────────────────────────────────────────┤
150 │type │ one of the numeric metric types from │
151 │ │ <pcp/pmapi.h>, stripped of the PM_TYPE_ pre‐ │
152 │ │ fix, so 32, U32, 64, U64, FLOAT or DOUBLE. │
153 ├──────────┼───────────────────────────────────────────────┤
154 │semantics │ one of the semantic types from <pcp/pmapi.h>, │
155 │ │ stripped of the PM_SEM_ prefix, so COUNTER, │
156 │ │ INSTANT or DISCRETE. │
157 ├──────────┼───────────────────────────────────────────────┤
158 │units │ a specification of dimension and scale │
159 │ │ (together forming the units), in the syntax │
160 │ │ accepted by pmParseUnitsStr(3). │
161 └──────────┴───────────────────────────────────────────────┘
162 The value may optionally be enclosed in double quotes, and may appear
163 in any mix of upper and/or lower case. The tag must be in lower case
164 as shown in the table above.
165 This is most useful when the expression semantics require matching
167 type and/or semantics and/or units for operands, e.g.
168 idle = mem.util.free > mkconst(10485760, units=Kbyte)
169 avg_io_size = delta(disk.dev.total) == 0 ? \
170 -mkconst(1.0, semantics=instant, units="kbyte / count") : \
171 delta(disk.dev.total_bytes) / delta(disk.dev.total)
172 * Expressions may be rescaled using the rescale function that takes two
174 arguments. The first is an arithmetic expression to be rescaled, and
175 the second is the desired units after rescaling that is a string
176 value in the syntax accepted by pmParseUnitsStr(3). For example:
177 rescale(network.interface.total.bytes, "Mbytes/hour")
178 The expression and the desired units must both have the same dimen‐
180 sion, e.g Space=1, Time=-1 and Count=0 in the example above.
181 * The following unary functions operate on a single performance metric
183 and return one or more values. For all functions (except count(),
184 defined() and instant()), the type of the operand metric must be
185 arithmetic (integer of various sizes and signedness, float or dou‐
186 ble).
187 ┌───────────┬───────────────────────────────────────────────┐
189 │ Function │ Value │
190 ├───────────┼───────────────────────────────────────────────┤
191 │avg(x) │ A singular instance being the average value │
192 │ │ across all instances for the metric x. │
193 ├───────────┼───────────────────────────────────────────────┤
194 │count(x) │ A singular instance being the count of the │
195 │ │ number of instances for the metric x. As a │
196 │ │ special case, if fetching the metric x │
197 │ │ returns an error, then count(x) will be 0. │
198 ├───────────┼───────────────────────────────────────────────┤
199 │defined(x) │ A boolean value that is true (``1'') if the │
200 │ │ metric x is defined in the PMNS, else false │
201 │ │ (``0''). The function is evaluated when a │
202 │ │ new PMAPI context is created with pmNewCon‐ │
203 │ │ text(3) or re-established with pmReconnect‐ │
204 │ │ Context(3). So any subsequent changes to the │
205 │ │ PMNS after the PMAPI context has been estab‐ │
206 │ │ lished will not change the value of this │
207 │ │ function in the expression evaluation. │
208 ├───────────┼───────────────────────────────────────────────┤
209 │delta(x) │ Returns the difference in values for the met‐ │
210 │ │ ric x between one call to pmFetch(3) and the │
211 │ │ next. There is one value in the result for │
212 │ │ each instance that appears in both the cur‐ │
213 │ │ rent and the previous sample. │
214 ├───────────┼───────────────────────────────────────────────┤
215 │rate(x) │ Returns the difference in values for the met‐ │
216 │ │ ric x between one call to pmFetch(3) and the │
217 │ │ next divided by the elapsed time between the │
218 │ │ calls to pmFetch(3). The semantics of the │
219 │ │ derived metric are based on the semantics of │
220 │ │ the operand (x) with the dimension in the │
221 │ │ time domain decreased by one and scaling if │
222 │ │ required in the time utilization case where │
223 │ │ the operand is in units of time, and the │
224 │ │ derived metric is unitless. This mimics the │
225 │ │ rate conversion applied to counter metrics by │
226 │ │ tools such as pmval(1), pmie(1) and │
227 │ │ pmchart(1). There is one value in the result │
228 │ │ for each instance that appears in both the │
229 │ │ current and the previous sample. │
230 ├───────────┼───────────────────────────────────────────────┤
231 │instant(x) │ Returns the current value of the metric x, │
232 │ │ even it has the semantics of a counter, i.e. │
233 │ │ PM_SEM_COUNTER. The semantics of the derived │
234 │ │ metric are based on the semantics of the op‐ │
235 │ │ erand (x); if x has semantics PM_SEM_COUNTER, │
236 │ │ the semantics of instant(x) is │
237 │ │ PM_SEM_INSTANT, otherwise the semantics of │
238 │ │ the derived metric is the same as the seman‐ │
239 │ │ tics of the metric x. │
240 ├───────────┼───────────────────────────────────────────────┤
241 │max(x) │ A singular instance being the maximum value │
242 │ │ across all instances for the metric x. │
243 ├───────────┼───────────────────────────────────────────────┤
244 │min(x) │ A singular instance being the minimum value │
245 │ │ across all instances for the metric x. │
246 ├───────────┼───────────────────────────────────────────────┤
247 │sum(x) │ A singular instance being the sum of the val‐ │
248 │ │ ues across all instances for the metric x. │
249 └───────────┴───────────────────────────────────────────────┘
250 * Parenthesis may be used for explicit grouping.
251 * Lines beginning with ``#'' are treated as comments and ignored.
253 * White space is ignored.
255
257 There are a number of conversions required to determine the metadata
258 for a derived metric and to ensure the semantics of the expressions are
259 sound.
260 In an arithmetic expression or a relational expression, if the seman‐
262 tics of both operands is not a counter (i.e. PM_SEM_INSTANT or
263 PM_SEM_DISCRETE) then the result will have semantics PM_SEM_INSTANT
264 unless both operands are PM_SEM_DISCRETE in which case the result is
265 also PM_SEM_DISCRETE.
266 For an arithmetic expression, the dimension of each operand must be the
268 same. For a relational expression, the dimension of each operand must
269 be the same, except that numeric constants (with no dimension) are
270 allowed, e.g. in the expression network.interface.in.drops > 0 .
271 To prevent arbitrary and non-sensical combinations some restrictions
273 apply to expressions that combine metrics with counter semantics to
274 produce a result with counter semantics. For an arithmetic expression,
275 if both operands have the semantics of a counter, then only addition or
276 subraction is allowed, or if the left operand is a counter and the
277 right operand is not, then only multiplication or division are allowed,
278 or if the left operand is not a counter and the right operand is a
279 counter, then only multiplication is allowed.
280 Because relational expressions use the current value only and produce a
282 result that is not a counter, either or both operands of a relational
283 expression may be counters.
284 The mapping of the pmUnits of the metadata uses the following rules:
286 * If both operands have a dimension of Count and the scales are not the
288 same, use the larger scale and convert the values of the operand with
289 the smaller scale.
290 * If both operands have a dimension of Time and the scales are not the
292 same, use the larger scale and convert the values of the operand with
293 the smaller scale.
294 * If both operands have a dimension of Space and the scales are not the
296 same, use the larger scale and convert the values of the operand with
297 the smaller scale.
298 * For addition and subtraction all dimensions for each of the operands
300 and result are identical.
301 * For multiplication, the dimensions of the result are the sum of the
303 dimensions of the operands.
304 * For division, the dimensions of the result are the difference of the
306 dimensions of the operands.
307 Scale conversion involves division if the dimension is positive else
309 multiplication if the dimension is negative. If scale conversion is
310 applied to either of the operands, the result is promoted to type
311 PM_TYPE_DOUBLE.
312 Putting all of this together in an example, consider the derived metric
314 defined as follows:
315 x = network.interface.speed - delta(network.interface.in.bytes) /
316 delta(sample.milliseconds)
317 The type, dimension and scale settings would propagate up the expres‐
318 sion tree as follows.
319 ┌────────────────────────┬────────┬───────────────┬─────────────────┐
321 │ Expression │ Type │ Dimension & │ Scale Factor(s) │
322 │ │ │ Scale │ │
323 ├────────────────────────┼────────┼───────────────┼─────────────────┤
324 │sample.milliseconds │ DOUBLE │ millisec │ │
325 │delta(...) │ DOUBLE │ millisec │ │
326 │network...bytes │ U64 │ byte │ │
327 │delta(...) │ U64 │ byte │ │
328 │delta(...) / delta(...) │ DOUBLE │ byte/millisec │ /1048576 and │
329 │ │ │ │ *1000 │
330 │network...speed │ FLOAT │ Mbyte/sec │ │
331 │x │ DOUBLE │ Mbyte/sec │ │
332 └────────────────────────┴────────┴───────────────┴─────────────────┘
333 Because semantic checking cannot be done at the time pmRegisterDerived
334 is called, errors found during semantic checking (when any subsequent
335 calls to pmNewContext(3) or pmReconnectContext(3) succeed) are reported
336 using pmprintf(3). These include:
337 Error: derived metric <name1>: operand: <name2>: <reason>
339 There was a problem calling pmLookupName(3) to identify the op‐
340 erand metric <name2> used in the definition of the derived met‐
341 ric <name1>.
342 Error: derived metric <name1>: operand (<name2> [<pmid2>]): <reason>
344 There was a problem calling pmLookupDesc(3) to identify the op‐
345 erand metric <name2> with PMID <pmid2> used in the definition of
346 the derived metric <name1>.
347 Semantic error: derived metric <name>: <operand> : <operand> Different
349 <metadata> for ternary operands
350 For a ternary expression, the ``true'' operand and the ``false''
351 operand must have exactly the same metadata, so type, semantics,
352 instance domain, and units (dimension and scale).
353 Semantic error: derived metric <name>: <operand> <op> <operand>: Dimen‐
355 sions are not the same
356 Operands must have the same units (dimension and scale) for each
357 of addition, subtraction, the relational operators and the bool‐
358 ean ``and'' or ``or'' operators.
359 Semantic error: derived metric <name>: <operand> <op> <operand>: Ille‐
361 gal operator for counter and non-counter
362 Only multiplication or division are allowed if the left operand
363 has the semantics of a counter and the right operand is not a
364 counter.
365 Semantic error: derived metric <name>: <operand> <op> <operand>: Ille‐
367 gal operator for counters
368 If both operands have the semantics of counter, only addition or
369 subtraction make sense, so multiplication and division are not
370 allowed.
371 Semantic error: derived metric <name>: <operand> <op> <operand>: Ille‐
373 gal operator for non-counter and counter
374 Only multiplication is allowed if the right operand has the
375 semantics of a counter and the left operand is not a counter.
376 Semantic error: derived metric <metric> <expr> RESCALE <units>: Incom‐
378 patible dimensions
379 The parameters <expr> and <units> to the rescale function must
380 have the same dimension along the axes of Time, Space and Count.
381 Semantic error: derived metric <name>: Incorrect time dimension for op‐
383 erand
384 Rate conversion using the rate() function is only possible for
385 operand metrics with a Time dimension of 0 or 1 (see
386 pmLookupDesc(3)). If the operand metric's Time dimension is 0,
387 then the derived metrics has a value "per second" (Time dimen‐
388 sion of -1). If the operand metric's Time dimension is 1, then
389 the derived metrics has a value of time utilization (Time dimen‐
390 sion of 0).
391 Semantic error: derived metric <name>: <function>(<operand>): Non-
393 arithmetic operand for function
394 The unary functions are only defined if the operand has arith‐
395 metic type. Similarly the first argument to the rescale func‐
396 tion must be of arithmetic type.
397 Semantic error: derived metric <name>: <expr> ? ...: Non-arithmetic op‐
399 erand for ternary guard
400 The first expression for a ternary operator must have an arith‐
401 metic type.
402 Semantic error: derived metric <name>: ... - ...: Non-arithmetic oper‐
404 and for unary negation
405 Unary negation only makes sense if the following expression has
406 an arithmetic type.
407 Semantic error: derived metric <name>: <operand> <op> <operand>: Non-
409 arithmetic type for <left-or-right> operand
410 The binary arithmetic operators are only allowed with operands
411 with an arithmetic type (integer of various sizes and signed‐
412 ness, float or double).
413 Semantic error: derived metric <name>: <operand> <op> <operand>: Non-
415 counter and not dimensionless <left-or-right> operand
416 For multiplication or division or any of the relational opera‐
417 tors, if one of the operands has the semantics of a counter and
418 the other has the semantics of a non-counter (instantaneous or
419 discrete) then the non-counter operand must have no units
420 (dimension and scale).
421 Semantic error: derived metric <name>: <expr> ? <expr> : <expr>: Non-
423 scalar ternary guard with scalar expressions
424 If the ``true'' and ``false'' operands of a ternary expression
425 have a scalar value, then the ``guard'' expression must also
426 have a scalar value.
427 Semantic error: derived metric <name>: <expr> <op> <expr>: Operands
429 should have the same instance domain
430 For all of the binary operators (arithmetic and relational), if
431 both operands have non-scalar values, then they must be defined
432 over the same instance domain.
433
435 For the binary arithmetic operators, if either operand must be scaled
436 (e.g. convert bytes to Kbytes) then the result is promoted to
437 PM_TYPE_DOUBLE. Otherwise the type of the result is determined by the
438 types of the operands, as per the following table which is evaluated
439 from top to bottom until a match is found.
440 ┌─────────────────────────┬──────────┬────────────────┐
442 │ Operand Types │ Operator │ Result Type │
443 ├─────────────────────────┼──────────┼────────────────┤
444 │either is PM_TYPE_DOUBLE │ any │ PM_TYPE_DOUBLE │
445 ├─────────────────────────┼──────────┼────────────────┤
446 │any │ division │ PM_TYPE_DOUBLE │
447 ├─────────────────────────┼──────────┼────────────────┤
448 │either is PM_TYPE_FLOAT │ any │ PM_TYPE_FLOAT │
449 ├─────────────────────────┼──────────┼────────────────┤
450 │either is PM_TYPE_U64 │ any │ PM_TYPE_U64 │
451 ├─────────────────────────┼──────────┼────────────────┤
452 │either is PM_TYPE_64 │ any │ PM_TYPE_64 │
453 ├─────────────────────────┼──────────┼────────────────┤
454 │either is PM_TYPE_U32 │ any │ PM_TYPE_U32 │
455 ├─────────────────────────┼──────────┼────────────────┤
456 │otherwise (both are │ any │ PM_TYPE_32 │
457 │PM_TYPE_32) │ │ │
458 └─────────────────────────┴──────────┴────────────────┘
460 Derived metrics are not available when using pmFetchArchive(3) as this
461 routine does not use a target list of PMIDs that could be remapped (as
462 is done for pmFetch(3)).
463 There is no pmUnregisterDerived method, so once registered a derived
465 metric persists for the life of the application.
466
468 On success, pmRegisterDerived returns NULL.
469 If a syntactic error is found at the time of registration, the value
471 returned by pmRegisterDerived is a pointer into expr indicating where
472 the error was found. To identify what the error was, the application
473 should call pmDerivedErrStr(3) to retrieve the corresponding parser
474 error message.
475 pmRegisterDerivedMetric returns 0 and errmsg is undefined if the pars‐
477 ing is successful.
478 If the given expr does not conform to the required syntax pmRegister‐
480 DerivedMetric returns -1 and a dynamically allocated error message
481 string in errmsg. The error message is terminated with a newline and
482 includes both the input name and expr, along with an indicator of the
483 position at which the error was detected. e.g.
484 Error: pmRegisterDerivedMetric("my.disk.rates", ...) syntax
485 error
486 4rat(disk.dev.read)
487 ^
488 The position indicator line may be followed by an additional diagnostic
490 line describing the nature of the error, when available.
491 In the case of an error, the caller is responsible for calling free(3)
493 to release the space allocated for errmsg.
494
496 PCPIntro(1), free(3), PMAPI(3), pmDerivedErrStr(3), pmFetch(3), pmLoad‐
497 DerivedConfig(3), pmNewContext(3), pmprintf(3) and pmReconnectCon‐
498 text(3).
499Performance Co-Pilot PMREGISTERDERIVED(3)