1PMLOGREWRITE(1) General Commands Manual PMLOGREWRITE(1)
2
6 pmlogrewrite - rewrite Performance Co-Pilot archives
7
9 pmlogrewrite [-Cdiqsvw?] [-c config] [-V version] inlog [outlog]
10
12 pmlogrewrite reads a set of Performance Co-Pilot (PCP) archives identi‐
13 fied by inlog and creates a PCP archive in outlog. Under normal usage,
14 the -c option will be used to nominate a configuration file or files
15 that contains specifications (see the REWRITING RULES SYNTAX section
16 below) that describe how the data and metadata from inlog should be
17 transformed to produce outlog.
18 The typical uses for pmlogrewrite would be to accommodate the evolution
20 of Performance Metric Domain Agents (PMDAs) where the names, metadata
21 and semantics of metrics and their associated instance domains may
22 change over time, e.g. promoting the type of a metric from a 32-bit to
23 a 64-bit integer, or renaming a group of metrics. Refer to the EXAM‐
24 PLES section for some additional use cases.
25 pmlogrewrite may also be used to redact sensitive information from PCP
27 archives in situations where the archives need to be shipped to another
28 organization or to meet privacy policies or legislative requirements.
29 See pmlogredact(1) for an example use of pmlogrewrite in this context.
30 pmlogrewrite is most useful where PMDA changes, or errors in the pro‐
32 duction environment, result in archives that cannot be combined with
33 pmlogextract(1). By pre-processing the archives with pmlogrewrite the
34 resulting archives may be able to be merged with pmlogextract(1).
35 The input inlog must be a set of PCP archives created by pmlogger(1),
37 or possibly one of the tools that read and create PCP archives, e.g.
38 pmlogextract(1) and pmlogreduce(1). inlog is a comma-separated list of
39 names, each of which may be the base name of an archive or the name of
40 a directory containing one or more archives.
41 If no -c option is specified, then the default behavior simply creates
43 outlog as a copy of inlog. This is a little more complicated than
44 cat(1), as each PCP archive is made up of several physical files.
45 While pmlogrewrite may be used to repair some data consistency issues
47 in PCP archives, there is also a class of repair tasks that cannot be
48 handled by pmlogrewrite and pmloglabel(1) may be a useful tool in these
49 cases.
50
52 The available command line options are:
53 -c config, --config=config
55 If config is a file or symbolic link, read and parse rewriting
56 rules from there. If config is a directory, then all of the files
57 or symbolic links in that directory (excluding those beginning
58 with a period ``.'') will be used to provide the rewriting rules.
59 Multiple -c options are allowed.
60 -C, --check
62 Parse the rewriting rules and quit. outlog is not created, so
63 this command line argument is optional with -C. When -C is speci‐
64 fied, this also sets -v and -w so that all warnings and verbose
65 messages are displayed as config is parsed.
66 -d, --desperate
68 Desperate mode. Normally if a fatal error occurs, all trace of
69 the partially written PCP archive outlog is removed. With the -d
70 option, the partially created outlog archive is not removed.
71 -i Rather than creating outlog, inlog is rewritten in place when the
73 -i option is used. A new archive is created using temporary file
74 names and then renamed to inlog in such a way that if any errors
75 (not warnings) are encountered, inlog remains unaltered.
76 -q, --quick
78 Quick mode, where if there are no rewriting actions to be per‐
79 formed (none of the global data, instance domains or metrics from
80 inlog will be changed), then pmlogrewrite will exit (with status
81 0, so success) immediately after parsing the configuration file(s)
82 and outlog is not created.
83 -s, --scale
85 When the ``units'' of a metric are changed, if the dimension in
86 terms of space, time and count is unaltered, then the scaling fac‐
87 tor is being changed, e.g. BYTE to KBYTE, or MSEC-1 to USEC-1, or
88 the composite MBYTE.SEC-1 to KBYTE.USEC-1. The motivation may be
89 (a) that the original metadata was wrong but the values in inlog
90 are correct, or (b) the metadata is changing so the values need to
91 change as well. The default pmlogrewrite behaviour matches case
92 (a). If case (b) applies, then use the -s option and the values
93 of all the metrics with a scale factor change in each result will
94 be rescaled. For finer control over value rescaling refer to the
95 RESCALE option for the UNITS clause of the metric rewriting rule
96 described below.
97 -v, --verbose
99 Enable verbose mode.
100 -V version, --version=version
102 Specifies the version of the output PCP archive being produced.
103 Currently versions 2 and 3 of the archive format is supported.
104 The version of inlog must be at least version (so version upgrade
105 is allowed, but version downgrade is not). By default, in the ab‐
106 sence of the -V option, the version of outlog is the same as the
107 version of inlog.
108 -w, --warnings
110 Emit warnings. Normally pmlogrewrite remains silent for any warn‐
111 ing that is not fatal and it is expected that for a particular ar‐
112 chive, some (or indeed, all) of the rewriting specifications may
113 not apply. For example, changes to a PMDA may be captured in a
114 set of rewriting rules, but a single archive may not contain all
115 of the modified metrics nor all of the modified instance domains
116 and/or instances. Because these cases are expected, they do not
117 prevent pmlogrewrite executing, and rules that do not apply to in‐
118 log are silently ignored by default. Similarly, some rewriting
119 rules may involve no change because the metadata in inlog already
120 matches the intent of the rewriting rule to correct data from a
121 previous version of a PMDA. The -w flag forces warnings to be
122 emitted for all of these cases.
123 -?, --help
125 Display usage message and exit.
126 The argument outlog is required in all cases, except when -i is speci‐
128 fied.
129
131 A configuration file contains zero or more rewriting rules as defined
132 below.
133 Keywords and special punctuation characters are shown below in
135 bolditalic font and are case-insensitive, so METRIC, metric and Metric
136 are all equivalent in rewriting rules.
137 The character ``#'' introduces a comment and the remainder of the line
139 is ignored. Otherwise the input is relatively free format with op‐
140 tional white space (spaces, tabs or newlines) between lexical items in
141 the rules.
142 A global rewriting rule has the form:
144 GLOBAL { globalspec ... }
146 where globalspec is zero or more of the following clauses:
148 HOSTNAME -> hostname
150 Modifies the label records in the outlog PCP archive, so that
152 the metrics will appear to have been collected from the host
153 hostname.
154 TIME -> delta
156 Both metric values and the instance domain metadata in a PCP
158 archive carry timestamps. This clause forces all the time‐
159 stamps to be adjusted by delta, where delta is an optional sign
160 ``+'' (the default) or ``-'', an optional number of hours fol‐
161 lowed by a colon ``:'', an optional number of minutes followed
162 by a colon ``:'', a number of seconds, an optional fraction of
163 seconds following a period ``.''. The simplest example would
164 be ``30'' to increase the timestamps by 30 seconds. A more
165 complex example would be ``-23:59:59.999'' to move the time‐
166 stamps backwards by one millisecond less than one day.
167 TIMEZONE -> "timezone"
169 TZ -> "timezone"
170 Modifies the label records in the outlog PCP archive, so that
172 the metrics will appear to have been collected from a host with
173 a local timezone of timezone. timezone must be enclosed in
174 double quotes, and should conform to the valid timezone syntax
175 rules for the local platform, usually a Posix TZ format, e.g.
176 AEST-10. See tzset(3) for more information.
177 TZ is an alias for TIMEZONE.
179 ZONEINFO -> "zoneinfo"
181 Modifies the label records in the outlog PCP archive, so that
183 the metrics will appear to have been collected from a host with
184 a local timezone of zoneinfo. zoneinfo must be enclosed in
185 double quotes, and should conform to the valid zoneinfo time‐
186 zone syntax rules for the local platform, usually a colon fol‐
187 lowed by a pathname below /usr/share/zoneinfo, e.g.
188 :Africa/Timbuktu. See tzset(3) for more information.
189 The zoneinfo clause is only allowed if the output archive ver‐
191 sion is at least 3.
192 FEATURES -> feature-bits
194 Modifies the label records in the outlog PCP archive, so that
196 the metrics will appear to have been collected from system with
197 a pmlogger(1) that supports the ``features'' defined by the in‐
198 teger value feature-bits, which is formed by ``or''ing the de‐
199 sired feature flags as defined in LOGARCHIVE(5). Alterna‐
200 tively, feature-bits can be specified using the ``macro''
201 BITS() that takes a comma separated argument list of integers
202 (in the inclusive range 0 to 31) and sets the corresponding
203 bits. For example
204 features -> bits(31,7,1)
205 The features clause is only allowed if the output archive ver‐
207 sion is at least 3.
208 An indom rewriting rule modifies an instance domain and has the form:
210 INDOM domain.serial { indomspec ... }
212 where domain and serial identify one or more existing instance domains
214 from inlog - typically domain would be an integer in the range 1 to 510
215 and serial would be an integer in the range 0 to 4194304.
216 As a special case serial could be an asterisk ``*'' which means the
218 rule applies to every instance domain with a domain number of domain.
219 If a designated instance domain is not in inlog the rule has no effect.
221 The indomspec is zero or more of the following clauses:
223 INAME "oldname" -> "newname"
225 The instance identified by the external instance name oldname
227 is renamed to newname. Both oldname and newname must be en‐
228 closed in double quotes.
229 As a special case, the new name may be the keyword DELETE (with
231 no double quotes), and then the instance oldname will be ex‐
232 punged from outlog which removes it from the instance domain
233 metadata and removes all values of this instance for all the
234 associated metrics.
235 If the instance names contain any embedded spaces then special
237 care needs to be taken in respect of the PCP instance naming
238 rule that treats the leading non-space part of the instance
239 name as the unique portion of the name for the purposes of
240 matching and ensuring uniqueness within an instance domain, re‐
241 fer to pmdaInstance(3) for a discussion of this issue.
242 As an illustration, consider the hypothetical instance domain
244 for a metric which contains 2 instances with the following
245 names:
246 red
247 eek urk
248 Then some possible INAME clauses might be:
250 "eek" -> "yellow like a flower"
252 Acceptable, oldname "eek" matches the "eek urk" in‐
253 stance.
254 "red" -> "eek"
256 Error, newname "eek" matches the existing "eek urk"
257 instance.
258 "eek urk" -> "red of another hue"
260 Error, newname "red of another hue" matches the ex‐
261 isting "red" instance.
262 INAME REPLACE /pattern/ -> "replacement"
264 Every external instance name in the instance domain is matched
266 against the regular expression pattern and when a match is
267 found, the name is changed based on the parts of the name
268 matched by pattern and the replacement recipe. pattern follows
269 the syntax of a Posix Extended Regular Expression (see
270 regex(7)) and replacement follows the syntax of the s command
271 of sed(1), so & and \1 through \9 may be used to select all or
272 substrings of the instance name that matches pattern.
273 Note that the match-and-replace is done at most once per exter‐
275 nal instance name, so if there are repeated sequences of the
276 name that match pattern only the first one will be matched and
277 replacement applied.
278 pattern is normally enclosed in slashes (/) or double quotes
280 ("). An escape (\) may be used before any of the metacharac‐
281 ters described in regex(3) to specify a literal character, e.g.
282 \( or \[ or \+ ... The enclosing delimiters cannot be escaped,
283 so to embed a literal slash, use double quotes as the delim‐
284 iter, or vice versa.
285 replacement is normally enclosed by double quotes (") or
287 slashes (/). An escape (\) may be used before & or \ to spec‐
288 ify these characters literally. The enclosing delimiters can‐
289 not be escaped, so to embed a double quote, use slashes as the
290 delimiter, or vice versa.
291 If the instance names after replacement contain any embedded
293 spaces then special care needs to be taken in respect of the
294 PCP instance naming rule that treats the leading non-space part
295 of the instance name as the unique portion of the name for the
296 purposes of matching and ensuring uniqueness within an instance
297 domain. Refer to pmdaInstance(3) for a discussion of this is‐
298 sue.
299 Here are some examples:
301 { iname replace /[a-z]*foo[a-z]*/ -> "FOO" }
303 replace any word containing "foo" with "FOO"
304 { iname replace "([0-9]+) /.*/(.*)" -> "\1 \2" }
306 removes a directory path, so the instance name (for
307 one of the proc PMDA's metrics) "2981799
308 /home/kenj/bin/foobar" would become "2981799 foobar",
309 hiding the user's home directory and implicitly the
310 user's login name
311 INAME REDACT
313 Replace every external instance name in the instance domain by
315 the string "inst [redacted]" where inst is the internal in‐
316 stance identifier in ASCII format.
317 INDOM -> newdomain.newserial
319 Modifies the metadata for the instance domain and every metric
321 associated with the instance domain. As a special case, newse‐
322 rial could be an asterisk ``*'' which means use serial from the
323 indom rewriting rule, although this is most useful when serial
324 is also an asterisk. So for example:
325 indom 29.* { indom -> 109.* }
326 will move all instance domains from domain 29 to domain 109.
327 INDOM -> DUPLICATE newdomain.newserial
329 A special case of the previous INDOM clause where the instance
331 domain is a duplicate copy of the domain.serial instance domain
332 from the indom rewriting rule, and then any mapping rules are
333 applied to the copied newdomain.newserial instance domain.
334 This is useful when a PMDA is split and the same instance do‐
335 main needs to be replicated for domain domain and domain newdo‐
336 main. So for example if the metrics foo.one and foo.two are
337 both defined over instance domain 12.34, and foo.two is moved
338 to another PMDA using domain 27, then the following rewriting
339 rules could be used:
340 indom 12.34 { indom -> duplicate 27.34 }
341 metric foo.two { indom -> 27.34 pmid -> 27.*.* }
342 INST oldid -> newid
344 The instance identified by the internal instance identifier ol‐
346 did is renumbered to newid. Both oldid and newid are integers
347 in the range 0 to 231-1.
348 As a special case, newid may be the keyword DELETE and then the
350 instance oldid will be expunged from outlog which removes it
351 from the instance domain metadata and removes all values of
352 this instance for all the associated metrics.
353 A metric rewriting rule has the form:
355 METRIC metricid { metricspec ... }
357 where metricid identifies one or more existing metrics from inlog using
359 either a metric name, or the internal encoding for a metric's PMID as
360 domain.cluster.item. In the latter case, typically domain would be an
361 integer in the range 1 to 510, cluster would be an integer in the range
362 0 to 4095, and item would be an integer in the range 0 to 1023.
363 As special cases item could be an asterisk ``*'' which means the rule
365 applies to every metric with a domain number of domain and a cluster
366 number of cluster, or cluster could be an asterisk which means the rule
367 applies to every metric with a domain number of domain and an item num‐
368 ber of item, or both cluster and item could be asterisks, and rule ap‐
369 plies to every metric with a domain number of domain.
370 If a designated metric is not in inlog the rule has no effect.
372 The metricspec is zero or more of the following clauses:
374 DELETE
377 The metric is completely removed from outlog, both the metadata
379 and all values in results are expunged.
380 INDOM -> newdomain.newserial [ pick ]
383 Modifies the metadata to change the instance domain for this
385 metric. The new instance domain must exist in outlog.
386 The optional pick clause may be used to select one input value,
388 or compute an aggregate value from the instances in an input
389 result, or assign an internal instance identifier to a single
390 output value. If no pick clause is specified, the default be‐
391 haviour is to copy all input values from each input result to
392 an output result, however if the input instance domain is sin‐
393 gular (indom PM_INDOM_NULL) then the one output value must be
394 assigned an internal instance identifier, which is 0 by de‐
395 fault, unless over-ridden by a INST or INAME clause as defined
396 below.
397 The choices for pick are as follows:
399 OUTPUT FIRST
401 choose the value of the first instance from each
402 input result
403 OUTPUT LAST choose the value of the last instance from each in‐
405 put result
406 OUTPUT INST instid
408 choose the value of the instance with internal in‐
409 stance identifier instid from each result; the se‐
410 quence of rewriting rules ensures the OUTPUT pro‐
411 cessing happens before instance identifier renum‐
412 bering from any associated indom rule, so instid
413 should be one of the internal instance identifiers
414 that appears in inlog
415 OUTPUT INAME "name"
417 choose the value of the instance with name for its
418 external instance name from each result; the se‐
419 quence of rewriting rules ensures the OUTPUT pro‐
420 cessing happens before instance renaming from any
421 associated indom rule, so name should be one of the
422 external instance names that appears in inlog
423 OUTPUT MIN choose the smallest value in each result (metric
425 type must be numeric and output instance will be 0
426 for a non-singular instance domain)
427 OUTPUT MAX choose the largest value in each result (metric
429 type must be numeric and output instance will be 0
430 for a non-singular instance domain)
431 OUTPUT SUM choose the sum of all values in each result (metric
433 type must be numeric and output instance will be 0
434 for a non-singular instance domain)
435 OUTPUT AVG choose the average of all values in each result
437 (metric type must be numeric and output instance
438 will be 0 for a non-singular instance domain)
439 If the input instance domain is singular (indom PM_INDOM_NULL)
441 then independent of any pick specifications, there is at most
442 one value in each input result and so FIRST, LAST, MIN, MAX,
443 SUM and AVG are all equivalent and the output instance identi‐
444 fier will be 0.
445 In general it is an error to specify a rewriting action for the
447 same metadata or result values more than once, e.g. more than
448 one INDOM clause for the same instance domain. The one excep‐
449 tion is the possible interaction between the INDOM clauses in
450 the indom and metric rules. For example the metric sample.bin
451 is defined over the instance domain 29.2 in inlog and the fol‐
452 lowing is acceptable (albeit redundant):
453 indom 29.* { indom -> 109.* }
454 metric sample.bin { indom -> 109.2 }
455 However the following is an error, because the instance domain
456 for sample.bin has two conflicting definitions:
457 indom 29.* { indom -> 109.* }
458 metric sample.bin { indom -> 123.2 }
459 INDOM -> NULL[ pick ]
462 The metric (which must have been previously defined over an in‐
464 stance domain) is being modified to be a singular metric. This
465 involves a metadata change and collapsing all results for this
466 metric so that multiple values become one value.
467 The optional pick part of the clause defines how the one value
469 for each result should be calculated and follows the same rules
470 as described for the non-NULL INDOM case above.
471 In the absence of pick, the default is OUTPUT FIRST.
473 NAME -> newname
476 Renames the metric in the PCP archive's metadata that supports
478 the Performance Metrics Name Space (PMNS). newname should not
479 match any existing name in the archive's PMNS and must follow
480 the syntactic rules for valid metric names as outlined in
481 PMNS(5).
482 PMID -> newdomain.newcluster.newitem
485 Modifies the metadata and results to renumber the metric's
487 PMID. As special cases, newcluster could be an asterisk ``*''
488 which means use cluster from the metric rewriting rule and/or
489 item could be an asterisk which means use item from the metric
490 rewriting rule. This is most useful when cluster and/or item
491 is also an asterisk. So for example:
492 metric 30.*.* { pmid -> 123.*.* }
493 will move all metrics from domain 30 to domain 123.
494 SEM -> newsem
497 Change the semantics of the metric. newsem should be the XXX
499 part of the name of one of the PM_SEM_XXX macros defined in
500 <pcp/pmapi.h> or pmLookupDesc(3), e.g. COUNTER for
501 PM_TYPE_COUNTER.
502 No data value rewriting is performed as a result of the SEM
504 clause, so the usefulness is limited to cases where a version
505 of the associated PMDA was exporting incorrect semantics for
506 the metric. pmlogreduce(1) may provide an alternative in cases
507 where re-computation of result values is desired.
508 TYPE -> newtype
511 Change the type of the metric which alters the metadata and may
513 change the encoding of values in results. newtype should be
514 the XXX part of the name of one of the PM_TYPE_XXX macros de‐
515 fined in <pcp/pmapi.h> or pmLookupDesc(3), e.g. FLOAT for
516 PM_TYPE_FLOAT.
517 Type conversion is only supported for cases where the old and
519 new metric type is numeric, so PM_TYPE_STRING, PM_TYPE_AGGRE‐
520 GATE and PM_TYPE_EVENT are not allowed. Even for the numeric
521 cases, some conversions may produce run-time errors, e.g. inte‐
522 ger overflow, or attempting to rewrite a negative value into an
523 unsigned type.
524 TYPE IF oldtype -> newtype
527 The same as the preceding TYPE clause, except the type of the
529 metric is only changed to newtype if the type of the metric in
530 inlog is oldtype.
531 This useful in cases where the type of metricid in inlog may be
533 platform dependent and so more than one type rewriting rule is
534 required.
535 UNITS -> newunits [ RESCALE ]
538 newunits is six values separated by commas. The first 3 values
540 describe the dimension of the metric along the dimensions of
541 space, time and count; these are integer values, usually 0, 1
542 or -1. The remaining 3 values describe the scale of the met‐
543 ric's values in the dimensions of space, time and count. Space
544 scale values should be 0 (if the space dimension is 0), else
545 the XXX part of the name of one of the PM_SPACE_XXX macros,
546 e.g. KBYTE for PM_TYPE_KBYTE. Time scale values should be 0
547 (if the time dimension is 0), else the XXX part of the name of
548 one of the PM_TIME_XXX macros, e.g. SEC for PM_TIME_SEC.
549 Count scale values should be 0 (if the time dimension is 0),
550 else ONE for PM_COUNT_ONE.
551 The PM_SPACE_XXX, PM_TIME_XXX and PM_COUNT_XXX macros are de‐
553 fined in <pcp/pmapi.h> or pmLookupDesc(3).
554 When the scale is changed (but the dimension is unaltered) the
556 optional keyword RESCALE may be used to chose value rescaling
557 as per the -s command line option, but applied to just this
558 metric.
559 VALUE REPLACE /pattern/ -> "replacement"
562 The value for every instance of the metric is matched against
564 the regular expression pattern and when a match is found, the
565 value is changed based on the parts of the value matched by
566 pattern and the replacement recipe. pattern follows the syntax
567 of a Posix Extended Regular Expression (see regex(7)) and re‐
568 placement follows the syntax of the s command of sed(1), so &
569 and \1 through \9 may be used to select all or substrings of
570 the value that matches pattern.
571 Note that the match-and-replace is done at most once per metric
573 value, so if there are repeated sequences of the metric value
574 that match pattern only the first one will be matched and re‐
575 placement applied.
576 pattern is normally enclosed in slashes (/) or double quotes
578 ("). An escape (\) may be used before any of the metacharac‐
579 ters described in regex(3) to specify a literal character, e.g.
580 \( or \[ or \+ ... The enclosing delimiters cannot be escaped,
581 so to embed a literal slash, use double quotes as the delim‐
582 iter, or vice versa.
583 replacement is normally enclosed by double quotes (") or
585 slashes (/). An escape (\) may be used before & or \ to spec‐
586 ify these characters literally. The enclosing delimiters can‐
587 not be escaped, so to embed a double quote, use slashes as the
588 delimiter, or vice versa.
589 The REPLACE keyword is optional.
591 This clause can only be applied to metrics with values of type
593 PM_TYPE_STRING.
594 Here are some examples:
596 { value /.*/ -> "" }
598 remove the value everywhere
599 value "/.*/(.*)" -> "\1" }
601 removes a directory path, so the metric value "mumble
602 /home/kenj/bin/foobar fumble" would become "mumble
603 foobar fumble", hiding the user's home directory and
604 implicitly the user's login name
605 When changing the domain number for a metric or instance domain,
608 the new domain number will usually match an existing PMDA's domain
609 number. If this is not the case, then the new domain number should
610 not be randomly chosen; consult $PCP_VAR_DIR/pmns/stdpmid for do‐
611 main numbers that are already assigned to PMDAs.
612 A text rewriting rule modifies a help text record and has the form:
614 TEXT textid [ texttype ] [ "textcontent" ] { textspec ... }
616 where textid identifies the metric or instance domain with which the
618 text is currently associated, and is either METRIC metricid or INDOM
619 domain.serial.
620 metricid has the same form and meaning as for a METRIC rewriting rule
622 (see above) and domain.serial has the same form and meaning as for an
623 INDOM rewriting rule (see above).
624 The optional texttype identifies the type of text and may be one of
626 ONELINE to select the one line help text, HELP to select the full help
627 text, or ALL or an asterisk ``*'' to select both types of help text.
628 If texttype is not specified, then the default is ONELINE.
629 The optional textcontent further restricts the selected text records to
631 those containing the specified content. Characters such as double
632 quotes may be escaped by preceding them with a backslash ``\''.
633 If a designated help text record is not in inlog the rule has no ef‐
635 fect.
636 The textspec is zero or more of the following clauses:
638 DELETE
640 The selected text is completely removed from outlog.
642 INDOM -> newdomain.newserial
644 Reassociates the text with the specified instance domain. As a
646 special case, newserial could be an asterisk ``*'' which means
647 use serial from the text rewriting rule, although this is most
648 useful when serial is also an asterisk. So for example:
649 text indom 29.* all { indom -> 109.* }
650 will reassociate all text associated with instance domains from
651 domain 29 to domain 109.
652 METRIC -> newdomain.newcluster.newitem
654 Reassociates the text with the specified metric. As special
656 cases, newcluster could be an asterisk ``*'' which means use
657 cluster from the text rewriting rule and/or item could be an
658 asterisk which means use item from the text rewriting rule.
659 This is most useful when cluster and/or item is also an aster‐
660 isk. So for example:
661 text metric 30.*.* all { metric -> 123.*.* }
662 will reassociate all text associated with metrics from domain
663 30 to domain 123.
664 TEXT -> "new-text"
666 Replaces the content of the selected text with new-text.
668 A label rewriting rule modifies a label record and has the form:
670 LABEL labelid [ instance ] [ "label-name" ] [ "label-value" ] { label‐
672 spec ... }
673 where labelid refers to the global context or identifies the metric do‐
675 main, metric cluster, metric item, instance domain, or instance domain
676 instances with which the label is currently associated, and is either
677 CONTEXT or DOMAIN domainid or CLUSTER domainid.clusterid or ITEM metri‐
678 cid or INDOM domain.serial or INSTANCES domain.serial.
679 metricid has the same form and meaning as for a METRIC rewriting rule
681 (see above). clusterid may be an asterisk ``*'' which means the rule
682 applies to every metric with a domain number of domainid in the same
683 way as an asterisk may be used for the cluster within metricid.
684 domain.serial has the same form and meaning as for an INDOM rewriting
686 rule (see above).
687 In the case of an INSTANCES labelid, the name or number of a specific
689 instance may be optionally specified as instance. This name or number
690 number may be omitted or specified as an asterisk ``*'' to indicate
691 that labels for all instances of the specified instance domain are se‐
692 lected. If an instance name is specified, it must be within double
693 quotes. If the instance name contains any embedded spaces then special
694 care needs to be taken in respect of the PCP instance naming rule that
695 treats the leading non-space part of the instance name as the unique
696 portion of the name for the purposes of matching and ensuring unique‐
697 ness within an instance domain, refer to pmdaInstance(3) for a discus‐
698 sion of this issue.
699 In all cases, a "label-name" and/or a "label-value" may be optionally
701 specified in double quotes in order to select labels with the given
702 name and/or given value. These may individually be omitted or speci‐
703 fied as asterisks ``*'' to indicate that labels with all names and/or
704 values are selected.
705 If a designated label record is not in inlog the rule has no effect.
707 The labelspec is zero or more of the following clauses:
709 DELETE
711 The selected labels are completely removed from outlog.
713 NEW "new-label-name" "new-label-value"
715 A new label with the name "new-label-name" and the value "new-
717 label-value" is created and associated with the specified la‐
718 belid and optional instance (in the case of a INSTANCES la‐
719 belid). If "label-name" or "label-value" were specified, then
720 they are ignored with a warning. If instance is not specified
721 for an INSTANCES labelid, then a new label will be created for
722 each instance in the specified instance domain.
723 LABEL -> "new-label-name"
725 The name of the selected label(s) is changed to "new-label-
727 name".
728 VALUE -> "new-label-value"
730 The value of the selected label(s) is changed to "new-label-
732 value".
733 DOMAIN -> newdomain
735 Reassociates the selected label(s) with the specified metric
737 domain. For example:
738 label domain 30 { domain -> 123 }
739 will reassociate all labels associated with domains from domain
740 30 to domain 123.
741 CLUSTER -> newdomain.newcluster
743 Reassociates the selected label(s) with the specified metric
745 cluster. As a special case, newcluster could be an asterisk
746 ``*'' which means use cluster from the label rewriting rule.
747 This is most useful when cluster is also an asterisk. So for
748 example:
749 label cluster 30.* { cluster -> 123.* }
750 will reassociate all labels associated with clusters from do‐
751 main 30 to domain 123.
752 ITEM -> newdomain.newcluster.newitem
754 Reassociates the selected label(s) with the specified metric
756 item. As special cases, newcluster could be an asterisk ``*''
757 which means use cluster from the label rewriting rule and/or
758 item could be an asterisk which means use item from the label
759 rewriting rule. This is most useful when cluster and/or item
760 is also an asterisk. So for example:
761 label item 30.*.* { item -> 123.*.* }
762 will reassociate all labels associated with metrics from domain
763 30 to domain 123.
764 INDOM -> newdomain.newserial
766 Reassociates the selected label(s) with the specified instance
768 domain. As a special case, newserial could be an asterisk
769 ``*'' which means use serial from the label rewriting rule, al‐
770 though this is most useful when serial is also an asterisk. So
771 for example:
772 label indom 29.* { indom -> 109.* }
773 will reassociate all labels associated with instance domains
774 from domain 29 to domain 109.
775 INSTANCES -> newdomain.newserial
777 This is the same as INDOM except that it reassociates the se‐
779 lected label(s) with the instances of the specified instance
780 domain.
781
783 To promote the values of the per-disk IOPS metrics to 64-bit to allow
784 aggregation over a long time period for capacity planning, or because
785 the PMDA has changed to export 64-bit counters and we want to convert
786 old archives so they can be processed alongside new archives.
787 metric disk.dev.read { type -> U64 }
788 metric disk.dev.write { type -> U64 }
789 metric disk.dev.total { type -> U64 }
790 The instances associated with the load average metric kernel.all.load
792 could be renamed and renumbered by the rules below.
793 # for the Linux PMDA, the kernel.all.load metric is defined
794 # over instance domain 60.2
795 indom 60.2 {
796 inst 1 -> 60 iname "1 minute" -> "60 second"
797 inst 5 -> 300 iname "5 minute" -> "300 second"
798 inst 15 -> 900 iname "15 minute" -> "900 second"
799 }
800 If we decide to split the ``proc'' metrics out of the Linux PMDA, this
802 will involve changing the domain number for the PMID of these metrics
803 and the associated instance domains. The rules below would rewrite an
804 old archive to match the changes after the PMDA split.
805 # all Linux proc metrics are in 7 clusters
806 metric 60.8.* { pmid -> 123.*.* }
807 metric 60.9.* { pmid -> 123.*.* }
808 metric 60.13.* { pmid -> 123.*.* }
809 metric 60.24.* { pmid -> 123.*.* }
810 metric 60.31.* { pmid -> 123.*.* }
811 metric 60.32.* { pmid -> 123.*.* }
812 metric 60.51.* { pmid -> 123.*.* }
813 # only one instance domain for Linux proc metrics
814 indom 60.9 { indom -> 123.0 }
815 If the metric foo.count_em was exported as a native ``long'' then it
817 could be a 32-bit integer on some platforms and a 64-bit integer on
818 other platforms. Subsequent investigations show the value is in fact
819 unsigned, so the following rules could be used.
820 metric foo.count_em {
821 type if 32 -> U32
822 type if 64 -> U64
823 }
824
826 All error conditions detected by pmlogrewrite are reported on stderr
827 with textual (if sometimes terse) explanation.
828 Should the input archive be corrupted (this can happen if the pmlogger
830 instance writing the archive suddenly dies), then pmlogrewrite will de‐
831 tect and report the position of the corruption in the file, and any
832 subsequent information from that archive will not be processed.
833 If the input archive contains no archive records then an ``empty ar‐
835 chive'' warning is issued and no processing is performed.
836 If any error is detected, pmlogrewrite will exit with a non-zero sta‐
838 tus.
839
841 For each of the inlog and outlog archives, several physical files are
842 used.
843 archive.meta
845 metadata (metric descriptions, instance domains, etc.) for the ar‐
846 chive
847 archive.0
849 initial volume of metrics values (subsequent volumes have suffixes
850 1, 2, ...).
851 archive.index
853 temporal index to support rapid random access to the other files
854 in the archive.
855
857 Environment variables with the prefix PCP_ are used to parameterize the
858 file and directory names used by PCP. On each installation, the file
859 /etc/pcp.conf contains the local values for these variables. The
860 $PCP_CONF variable may be used to specify an alternative configuration
861 file, as described in pcp.conf(5).
862 For environment variables affecting PCP tools, see pmGetOptions(3).
864
866 PCPIntro(1), pmdumplog(1), pmlogextract(1), pmlogger(1), pmloglabel(1),
867 pmlogredact(1), pmlogreduce(1), PMAPI(3), pmdaInstance(3), pm‐
868 LookupDesc(3), tzset(3), LOGARCHIVE(5), pcp.conf(5), pcp.env(5),
869 PMNS(5) and regex(7).
870Performance Co-Pilot PMLOGREWRITE(1)