1PMLOGREWRITE(1)             General Commands Manual            PMLOGREWRITE(1)
2
3
4

NAME

6       pmlogrewrite - rewrite Performance Co-Pilot archives
7

SYNOPSIS

9       pmlogrewrite [-Cdiqsvw?]  [-c config] [-V version] inlog [outlog]
10

DESCRIPTION

12       pmlogrewrite reads a set of Performance Co-Pilot (PCP) archives identi‐
13       fied by inlog and creates a PCP archive in outlog.  Under normal usage,
14       the  -c  option  will be used to nominate a configuration file or files
15       that contains specifications (see the REWRITING  RULES  SYNTAX  section
16       below)  that  describe  how  the data and metadata from inlog should be
17       transformed to produce outlog.
18
19       The typical uses for pmlogrewrite would be to accommodate the evolution
20       of  Performance  Metric Domain Agents (PMDAs) where the names, metadata
21       and semantics of metrics and  their  associated  instance  domains  may
22       change  over time, e.g. promoting the type of a metric from a 32-bit to
23       a 64-bit integer, or renaming a group of metrics.  Refer to  the  EXAM‐
24       PLES section for some additional use cases.
25
26       pmlogrewrite  may also be used to redact sensitive information from PCP
27       archives in situations where the archives need to be shipped to another
28       organization  or  to meet privacy policies or legislative requirements.
29       See pmlogredact(1) for an example use of pmlogrewrite in this context.
30
31       pmlogrewrite is most useful where PMDA changes, or errors in  the  pro‐
32       duction  environment,  result  in archives that cannot be combined with
33       pmlogextract(1).  By pre-processing the archives with pmlogrewrite  the
34       resulting archives may be able to be merged with pmlogextract(1).
35
36       The  input  inlog must be a set of PCP archives created by pmlogger(1),
37       or possibly one of the tools that read and create  PCP  archives,  e.g.
38       pmlogextract(1) and pmlogreduce(1).  inlog is a comma-separated list of
39       names, each of which may be the base name of an archive or the name  of
40       a directory containing one or more archives.
41
42       If  no -c option is specified, then the default behavior simply creates
43       outlog as a copy of inlog.  This is  a  little  more  complicated  than
44       cat(1), as each PCP archive is made up of several physical files.
45
46       While  pmlogrewrite  may be used to repair some data consistency issues
47       in PCP archives, there is also a class of repair tasks that  cannot  be
48       handled by pmlogrewrite and pmloglabel(1) may be a useful tool in these
49       cases.
50

OPTIONS

52       The available command line options are:
53
54       -c config, --config=config
55            If config is a file or symbolic link,  read  and  parse  rewriting
56            rules from there.  If config is a directory, then all of the files
57            or symbolic links in that  directory  (excluding  those  beginning
58            with  a period ``.'') will be used to provide the rewriting rules.
59            Multiple -c options are allowed.
60
61       -C, --check
62            Parse the rewriting rules and quit.  outlog  is  not  created,  so
63            this command line argument is optional with -C.  When -C is speci‐
64            fied, this also sets -v and -w so that all  warnings  and  verbose
65            messages are displayed as config is parsed.
66
67       -d, --desperate
68            Desperate  mode.   Normally  if a fatal error occurs, all trace of
69            the partially written PCP archive outlog is removed.  With the  -d
70            option, the partially created outlog archive is not removed.
71
72       -i   Rather  than creating outlog, inlog is rewritten in place when the
73            -i option is used.  A new archive is created using temporary  file
74            names  and  then renamed to inlog in such a way that if any errors
75            (not warnings) are encountered, inlog remains unaltered.
76
77       -q, --quick
78            Quick mode, where if there are no rewriting  actions  to  be  per‐
79            formed  (none of the global data, instance domains or metrics from
80            inlog will be changed), then pmlogrewrite will exit  (with  status
81            0, so success) immediately after parsing the configuration file(s)
82            and outlog is not created.
83
84       -s, --scale
85            When the ``units'' of a metric are changed, if  the  dimension  in
86            terms of space, time and count is unaltered, then the scaling fac‐
87            tor is being changed, e.g. BYTE to KBYTE, or MSEC-1 to USEC-1,  or
88            the  composite MBYTE.SEC-1 to KBYTE.USEC-1.  The motivation may be
89            (a) that the original metadata was wrong but the values  in  inlog
90            are correct, or (b) the metadata is changing so the values need to
91            change as well.  The default pmlogrewrite behaviour  matches  case
92            (a).   If  case (b) applies, then use the -s option and the values
93            of all the metrics with a scale factor change in each result  will
94            be  rescaled.  For finer control over value rescaling refer to the
95            RESCALE option for the UNITS clause of the metric  rewriting  rule
96            described below.
97
98       -v, --verbose
99            Enable verbose mode.
100
101       -V version, --version=version
102            Specifies  the  version  of the output PCP archive being produced.
103            Currently versions 2 and 3 of the  archive  format  is  supported.
104            The  version of inlog must be at least version (so version upgrade
105            is allowed, but version downgrade is not).  By default, in the ab‐
106            sence  of  the -V option, the version of outlog is the same as the
107            version of inlog.
108
109       -w, --warnings
110            Emit warnings.  Normally pmlogrewrite remains silent for any warn‐
111            ing that is not fatal and it is expected that for a particular ar‐
112            chive, some (or indeed, all) of the rewriting  specifications  may
113            not  apply.   For  example, changes to a PMDA may be captured in a
114            set of rewriting rules, but a single archive may not  contain  all
115            of  the  modified metrics nor all of the modified instance domains
116            and/or instances.  Because these cases are expected, they  do  not
117            prevent pmlogrewrite executing, and rules that do not apply to in‐
118            log are silently ignored by default.   Similarly,  some  rewriting
119            rules  may involve no change because the metadata in inlog already
120            matches the intent of the rewriting rule to correct  data  from  a
121            previous  version  of  a  PMDA.  The -w flag forces warnings to be
122            emitted for all of these cases.
123
124       -?, --help
125            Display usage message and exit.
126
127       The argument outlog is required in all cases, except when -i is  speci‐
128       fied.
129

REWRITING RULES SYNTAX

131       A  configuration  file contains zero or more rewriting rules as defined
132       below.
133
134       Keywords  and  special  punctuation  characters  are  shown  below   in
135       bolditalic  font and are case-insensitive, so METRIC, metric and Metric
136       are all equivalent in rewriting rules.
137
138       The character ``#'' introduces a comment and the remainder of the  line
139       is  ignored.   Otherwise  the  input is relatively free format with op‐
140       tional white space (spaces, tabs or newlines) between lexical items  in
141       the rules.
142
143       A global rewriting rule has the form:
144
145       GLOBAL { globalspec ...  }
146
147       where globalspec is zero or more of the following clauses:
148
149           HOSTNAME -> hostname
150
151               Modifies  the  label records in the outlog PCP archive, so that
152               the metrics will appear to have been collected  from  the  host
153               hostname.
154
155           TIME -> delta
156
157               Both  metric  values  and the instance domain metadata in a PCP
158               archive carry timestamps.  This clause  forces  all  the  time‐
159               stamps to be adjusted by delta, where delta is an optional sign
160               ``+'' (the default) or ``-'', an optional number of hours  fol‐
161               lowed  by a colon ``:'', an optional number of minutes followed
162               by a colon ``:'', a number of seconds, an optional fraction  of
163               seconds  following  a period ``.''.  The simplest example would
164               be ``30'' to increase the timestamps by  30  seconds.   A  more
165               complex  example  would  be ``-23:59:59.999'' to move the time‐
166               stamps backwards by one millisecond less than one day.
167
168           TIMEZONE -> "timezone"
169           TZ -> "timezone"
170
171               Modifies the label records in the outlog PCP archive,  so  that
172               the metrics will appear to have been collected from a host with
173               a local timezone of timezone.  timezone  must  be  enclosed  in
174               double  quotes, and should conform to the valid timezone syntax
175               rules for the local platform, usually a Posix TZ  format,  e.g.
176               AEST-10.  See tzset(3) for more information.
177
178               TZ is an alias for TIMEZONE.
179
180           ZONEINFO -> "zoneinfo"
181
182               Modifies  the  label records in the outlog PCP archive, so that
183               the metrics will appear to have been collected from a host with
184               a  local  timezone  of  zoneinfo.  zoneinfo must be enclosed in
185               double quotes, and should conform to the valid  zoneinfo  time‐
186               zone  syntax rules for the local platform, usually a colon fol‐
187               lowed   by   a   pathname   below   /usr/share/zoneinfo,   e.g.
188               :Africa/Timbuktu.  See tzset(3) for more information.
189
190               The  zoneinfo clause is only allowed if the output archive ver‐
191               sion is at least 3.
192
193           FEATURES -> feature-bits
194
195               Modifies the label records in the outlog PCP archive,  so  that
196               the metrics will appear to have been collected from system with
197               a pmlogger(1) that supports the ``features'' defined by the in‐
198               teger  value feature-bits, which is formed by ``or''ing the de‐
199               sired feature flags  as  defined  in  LOGARCHIVE(5).   Alterna‐
200               tively,  feature-bits  can  be  specified  using  the ``macro''
201               BITS() that takes a comma separated argument list  of  integers
202               (in  the  inclusive  range  0 to 31) and sets the corresponding
203               bits.  For example
204                   features -> bits(31,7,1)
205
206               The features clause is only allowed if the output archive  ver‐
207               sion is at least 3.
208
209       An indom rewriting rule modifies an instance domain and has the form:
210
211       INDOM domain.serial { indomspec ...  }
212
213       where  domain and serial identify one or more existing instance domains
214       from inlog - typically domain would be an integer in the range 1 to 510
215       and serial would be an integer in the range 0 to 4194304.
216
217       As  a  special  case  serial could be an asterisk ``*'' which means the
218       rule applies to every instance domain with a domain number of domain.
219
220       If a designated instance domain is not in inlog the rule has no effect.
221
222       The indomspec is zero or more of the following clauses:
223
224           INAME "oldname" -> "newname"
225
226               The instance identified by the external instance  name  oldname
227               is  renamed  to  newname.  Both oldname and newname must be en‐
228               closed in double quotes.
229
230               As a special case, the new name may be the keyword DELETE (with
231               no  double  quotes),  and then the instance oldname will be ex‐
232               punged from outlog which removes it from  the  instance  domain
233               metadata  and  removes  all values of this instance for all the
234               associated metrics.
235
236               If the instance names contain any embedded spaces then  special
237               care  needs  to  be taken in respect of the PCP instance naming
238               rule that treats the leading non-space  part  of  the  instance
239               name  as  the  unique  portion  of the name for the purposes of
240               matching and ensuring uniqueness within an instance domain, re‐
241               fer to pmdaInstance(3) for a discussion of this issue.
242
243               As  an  illustration, consider the hypothetical instance domain
244               for a metric which contains  2  instances  with  the  following
245               names:
246                   red
247                   eek urk
248
249               Then some possible INAME clauses might be:
250
251               "eek" -> "yellow like a flower"
252                         Acceptable,  oldname  "eek" matches the "eek urk" in‐
253                         stance.
254
255               "red" -> "eek"
256                         Error, newname "eek" matches the existing  "eek  urk"
257                         instance.
258
259               "eek urk" -> "red of another hue"
260                         Error,  newname  "red of another hue" matches the ex‐
261                         isting "red" instance.
262
263           INAME REPLACE /pattern/ -> "replacement"
264
265               Every external instance name in the instance domain is  matched
266               against  the  regular  expression  pattern  and when a match is
267               found, the name is changed based  on  the  parts  of  the  name
268               matched by pattern and the replacement recipe.  pattern follows
269               the  syntax  of  a  Posix  Extended  Regular  Expression   (see
270               regex(7))  and  replacement follows the syntax of the s command
271               of sed(1), so & and \1 through \9 may be used to select all  or
272               substrings of the instance name that matches pattern.
273
274               Note that the match-and-replace is done at most once per exter‐
275               nal instance name, so if there are repeated  sequences  of  the
276               name  that match pattern only the first one will be matched and
277               replacement applied.
278
279               pattern is normally enclosed in slashes (/)  or  double  quotes
280               (").   An  escape (\) may be used before any of the metacharac‐
281               ters described in regex(3) to specify a literal character, e.g.
282               \( or \[ or \+ ...  The enclosing delimiters cannot be escaped,
283               so to embed a literal slash, use double quotes  as  the  delim‐
284               iter, or vice versa.
285
286               replacement  is  normally  enclosed  by  double  quotes  (") or
287               slashes (/).  An escape (\) may be used before & or \ to  spec‐
288               ify  these characters literally.  The enclosing delimiters can‐
289               not be escaped, so to embed a double quote, use slashes as  the
290               delimiter, or vice versa.
291
292               If  the  instance  names after replacement contain any embedded
293               spaces then special care needs to be taken in  respect  of  the
294               PCP instance naming rule that treats the leading non-space part
295               of the instance name as the unique portion of the name for  the
296               purposes of matching and ensuring uniqueness within an instance
297               domain.  Refer to pmdaInstance(3) for a discussion of this  is‐
298               sue.
299
300               Here are some examples:
301
302               { iname replace /[a-z]*foo[a-z]*/ -> "FOO" }
303                         replace any word containing "foo" with "FOO"
304
305               { iname replace "([0-9]+) /.*/(.*)" -> "\1 \2" }
306                         removes  a  directory path, so the instance name (for
307                         one   of   the   proc   PMDA's   metrics)    "2981799
308                         /home/kenj/bin/foobar" would become "2981799 foobar",
309                         hiding the user's home directory and  implicitly  the
310                         user's login name
311
312           INAME REDACT
313
314               Replace  every external instance name in the instance domain by
315               the string "inst [redacted]" where inst  is  the  internal  in‐
316               stance identifier in ASCII format.
317
318           INDOM -> newdomain.newserial
319
320               Modifies  the metadata for the instance domain and every metric
321               associated with the instance domain.  As a special case, newse‐
322               rial could be an asterisk ``*'' which means use serial from the
323               indom rewriting rule, although this is most useful when  serial
324               is also an asterisk.  So for example:
325                   indom 29.* { indom -> 109.* }
326               will move all instance domains from domain 29 to domain 109.
327
328           INDOM -> DUPLICATE newdomain.newserial
329
330               A  special case of the previous INDOM clause where the instance
331               domain is a duplicate copy of the domain.serial instance domain
332               from  the  indom rewriting rule, and then any mapping rules are
333               applied to  the  copied  newdomain.newserial  instance  domain.
334               This  is  useful when a PMDA is split and the same instance do‐
335               main needs to be replicated for domain domain and domain newdo‐
336               main.   So  for  example if the metrics foo.one and foo.two are
337               both defined over instance domain 12.34, and foo.two  is  moved
338               to  another  PMDA using domain 27, then the following rewriting
339               rules could be used:
340                   indom 12.34 { indom -> duplicate 27.34 }
341                   metric foo.two { indom -> 27.34 pmid -> 27.*.*  }
342
343           INST oldid -> newid
344
345               The instance identified by the internal instance identifier ol‐
346               did  is renumbered to newid.  Both oldid and newid are integers
347               in the range 0 to 231-1.
348
349               As a special case, newid may be the keyword DELETE and then the
350               instance  oldid  will  be expunged from outlog which removes it
351               from the instance domain metadata and  removes  all  values  of
352               this instance for all the associated metrics.
353
354       A metric rewriting rule has the form:
355
356       METRIC metricid { metricspec ...  }
357
358       where metricid identifies one or more existing metrics from inlog using
359       either a metric name, or the internal encoding for a metric's  PMID  as
360       domain.cluster.item.   In the latter case, typically domain would be an
361       integer in the range 1 to 510, cluster would be an integer in the range
362       0 to 4095, and item would be an integer in the range 0 to 1023.
363
364       As  special  cases item could be an asterisk ``*'' which means the rule
365       applies to every metric with a domain number of domain  and  a  cluster
366       number of cluster, or cluster could be an asterisk which means the rule
367       applies to every metric with a domain number of domain and an item num‐
368       ber  of item, or both cluster and item could be asterisks, and rule ap‐
369       plies to every metric with a domain number of domain.
370
371       If a designated metric is not in inlog the rule has no effect.
372
373       The metricspec is zero or more of the following clauses:
374
375
376           DELETE
377
378               The metric is completely removed from outlog, both the metadata
379               and all values in results are expunged.
380
381
382           INDOM -> newdomain.newserial [ pick ]
383
384               Modifies  the  metadata  to change the instance domain for this
385               metric.  The new instance domain must exist in outlog.
386
387               The optional pick clause may be used to select one input value,
388               or  compute  an  aggregate value from the instances in an input
389               result, or assign an internal instance identifier to  a  single
390               output  value.  If no pick clause is specified, the default be‐
391               haviour is to copy all input values from each input  result  to
392               an  output result, however if the input instance domain is sin‐
393               gular (indom PM_INDOM_NULL) then the one output value  must  be
394               assigned  an  internal  instance  identifier, which is 0 by de‐
395               fault, unless over-ridden by a INST or INAME clause as  defined
396               below.
397
398               The choices for pick are as follows:
399
400               OUTPUT FIRST
401                           choose  the  value  of the first instance from each
402                           input result
403
404               OUTPUT LAST choose the value of the last instance from each in‐
405                           put result
406
407               OUTPUT INST instid
408                           choose  the value of the instance with internal in‐
409                           stance identifier instid from each result; the  se‐
410                           quence  of  rewriting rules ensures the OUTPUT pro‐
411                           cessing happens before instance  identifier  renum‐
412                           bering  from  any  associated indom rule, so instid
413                           should be one of the internal instance  identifiers
414                           that appears in inlog
415
416               OUTPUT INAME "name"
417                           choose  the value of the instance with name for its
418                           external instance name from each  result;  the  se‐
419                           quence  of  rewriting rules ensures the OUTPUT pro‐
420                           cessing happens before instance renaming  from  any
421                           associated indom rule, so name should be one of the
422                           external instance names that appears in inlog
423
424               OUTPUT MIN  choose the smallest value in  each  result  (metric
425                           type  must be numeric and output instance will be 0
426                           for a non-singular instance domain)
427
428               OUTPUT MAX  choose the largest value  in  each  result  (metric
429                           type  must be numeric and output instance will be 0
430                           for a non-singular instance domain)
431
432               OUTPUT SUM  choose the sum of all values in each result (metric
433                           type  must be numeric and output instance will be 0
434                           for a non-singular instance domain)
435
436               OUTPUT AVG  choose the average of all  values  in  each  result
437                           (metric  type  must  be numeric and output instance
438                           will be 0 for a non-singular instance domain)
439
440               If the input instance domain is singular (indom  PM_INDOM_NULL)
441               then  independent  of any pick specifications, there is at most
442               one value in each input result and so FIRST,  LAST,  MIN,  MAX,
443               SUM  and AVG are all equivalent and the output instance identi‐
444               fier will be 0.
445
446               In general it is an error to specify a rewriting action for the
447               same  metadata  or result values more than once, e.g. more than
448               one INDOM clause for the same instance domain.  The one  excep‐
449               tion  is  the possible interaction between the INDOM clauses in
450               the indom and metric rules.  For example the metric  sample.bin
451               is  defined over the instance domain 29.2 in inlog and the fol‐
452               lowing is acceptable (albeit redundant):
453                   indom 29.* { indom -> 109.* }
454                   metric sample.bin { indom -> 109.2 }
455               However the following is an error, because the instance  domain
456               for sample.bin has two conflicting definitions:
457                   indom 29.* { indom -> 109.* }
458                   metric sample.bin { indom -> 123.2 }
459
460
461           INDOM -> NULL[ pick ]
462
463               The metric (which must have been previously defined over an in‐
464               stance domain) is being modified to be a singular metric.  This
465               involves  a metadata change and collapsing all results for this
466               metric so that multiple values become one value.
467
468               The optional pick part of the clause defines how the one  value
469               for each result should be calculated and follows the same rules
470               as described for the non-NULL INDOM case above.
471
472               In the absence of pick, the default is OUTPUT FIRST.
473
474
475           NAME -> newname
476
477               Renames the metric in the PCP archive's metadata that  supports
478               the  Performance Metrics Name Space (PMNS).  newname should not
479               match any existing name in the archive's PMNS and  must  follow
480               the  syntactic  rules  for  valid  metric  names as outlined in
481               PMNS(5).
482
483
484           PMID -> newdomain.newcluster.newitem
485
486               Modifies the metadata and  results  to  renumber  the  metric's
487               PMID.   As special cases, newcluster could be an asterisk ``*''
488               which means use cluster from the metric rewriting  rule  and/or
489               item  could be an asterisk which means use item from the metric
490               rewriting rule.  This is most useful when cluster  and/or  item
491               is also an asterisk.  So for example:
492                   metric 30.*.* { pmid -> 123.*.* }
493               will move all metrics from domain 30 to domain 123.
494
495
496           SEM -> newsem
497
498               Change  the  semantics of the metric.  newsem should be the XXX
499               part of the name of one of the  PM_SEM_XXX  macros  defined  in
500               <pcp/pmapi.h>    or    pmLookupDesc(3),   e.g.    COUNTER   for
501               PM_TYPE_COUNTER.
502
503               No data value rewriting is performed as a  result  of  the  SEM
504               clause,  so  the usefulness is limited to cases where a version
505               of the associated PMDA was exporting  incorrect  semantics  for
506               the metric.  pmlogreduce(1) may provide an alternative in cases
507               where re-computation of result values is desired.
508
509
510           TYPE -> newtype
511
512               Change the type of the metric which alters the metadata and may
513               change  the  encoding  of values in results.  newtype should be
514               the XXX part of the name of one of the PM_TYPE_XXX  macros  de‐
515               fined  in  <pcp/pmapi.h>  or  pmLookupDesc(3),  e.g.  FLOAT for
516               PM_TYPE_FLOAT.
517
518               Type conversion is only supported for cases where the  old  and
519               new  metric  type is numeric, so PM_TYPE_STRING, PM_TYPE_AGGRE‐
520               GATE and PM_TYPE_EVENT are not allowed.  Even for  the  numeric
521               cases, some conversions may produce run-time errors, e.g. inte‐
522               ger overflow, or attempting to rewrite a negative value into an
523               unsigned type.
524
525
526           TYPE IF oldtype -> newtype
527
528               The  same  as the preceding TYPE clause, except the type of the
529               metric is only changed to newtype if the type of the metric  in
530               inlog is oldtype.
531
532               This useful in cases where the type of metricid in inlog may be
533               platform dependent and so more than one type rewriting rule  is
534               required.
535
536
537           UNITS -> newunits [ RESCALE ]
538
539               newunits is six values separated by commas.  The first 3 values
540               describe the dimension of the metric along  the  dimensions  of
541               space,  time  and count; these are integer values, usually 0, 1
542               or -1.  The remaining 3 values describe the scale of  the  met‐
543               ric's values in the dimensions of space, time and count.  Space
544               scale values should be 0 (if the space dimension  is  0),  else
545               the  XXX  part  of  the name of one of the PM_SPACE_XXX macros,
546               e.g.  KBYTE for PM_TYPE_KBYTE.  Time scale values should  be  0
547               (if  the time dimension is 0), else the XXX part of the name of
548               one of the  PM_TIME_XXX  macros,  e.g.   SEC  for  PM_TIME_SEC.
549               Count  scale  values  should be 0 (if the time dimension is 0),
550               else ONE for PM_COUNT_ONE.
551
552               The PM_SPACE_XXX, PM_TIME_XXX and PM_COUNT_XXX macros  are  de‐
553               fined in <pcp/pmapi.h> or pmLookupDesc(3).
554
555               When  the scale is changed (but the dimension is unaltered) the
556               optional keyword RESCALE may be used to chose  value  rescaling
557               as  per  the  -s  command line option, but applied to just this
558               metric.
559
560
561           VALUE REPLACE /pattern/ -> "replacement"
562
563               The value for every instance of the metric is  matched  against
564               the  regular  expression pattern and when a match is found, the
565               value is changed based on the parts of  the  value  matched  by
566               pattern and the replacement recipe.  pattern follows the syntax
567               of a Posix Extended Regular Expression (see regex(7))  and  re‐
568               placement  follows  the syntax of the s command of sed(1), so &
569               and \1 through \9 may be used to select all  or  substrings  of
570               the value that matches pattern.
571
572               Note that the match-and-replace is done at most once per metric
573               value, so if there are repeated sequences of the  metric  value
574               that  match  pattern only the first one will be matched and re‐
575               placement applied.
576
577               pattern is normally enclosed in slashes (/)  or  double  quotes
578               (").   An  escape (\) may be used before any of the metacharac‐
579               ters described in regex(3) to specify a literal character, e.g.
580               \( or \[ or \+ ...  The enclosing delimiters cannot be escaped,
581               so to embed a literal slash, use double quotes  as  the  delim‐
582               iter, or vice versa.
583
584               replacement  is  normally  enclosed  by  double  quotes  (") or
585               slashes (/).  An escape (\) may be used before & or \ to  spec‐
586               ify  these characters literally.  The enclosing delimiters can‐
587               not be escaped, so to embed a double quote, use slashes as  the
588               delimiter, or vice versa.
589
590               The REPLACE keyword is optional.
591
592               This  clause can only be applied to metrics with values of type
593               PM_TYPE_STRING.
594
595               Here are some examples:
596
597               { value /.*/ -> "" }
598                         remove the value everywhere
599
600               value "/.*/(.*)" -> "\1" }
601                         removes a directory path, so the metric value "mumble
602                         /home/kenj/bin/foobar  fumble"  would  become "mumble
603                         foobar fumble", hiding the user's home directory  and
604                         implicitly the user's login name
605
606
607           When  changing  the  domain number for a metric or instance domain,
608           the new domain number will usually match an existing PMDA's  domain
609           number.  If this is not the case, then the new domain number should
610           not be randomly chosen; consult $PCP_VAR_DIR/pmns/stdpmid  for  do‐
611           main numbers that are already assigned to PMDAs.
612
613       A text rewriting rule modifies a help text record and has the form:
614
615       TEXT textid [ texttype ] [ "textcontent" ] { textspec ...  }
616
617       where  textid  identifies  the metric or instance domain with which the
618       text is currently associated, and is either METRIC  metricid  or  INDOM
619       domain.serial.
620
621       metricid  has  the same form and meaning as for a METRIC rewriting rule
622       (see above) and domain.serial has the same form and meaning as  for  an
623       INDOM rewriting rule (see above).
624
625       The  optional  texttype  identifies  the type of text and may be one of
626       ONELINE to select the one line help text, HELP to select the full  help
627       text,  or  ALL  or an asterisk ``*'' to select both types of help text.
628       If texttype is not specified, then the default is ONELINE.
629
630       The optional textcontent further restricts the selected text records to
631       those  containing  the  specified  content.   Characters such as double
632       quotes may be escaped by preceding them with a backslash ``\''.
633
634       If a designated help text record is not in inlog the rule  has  no  ef‐
635       fect.
636
637       The textspec is zero or more of the following clauses:
638
639           DELETE
640
641               The selected text is completely removed from outlog.
642
643           INDOM -> newdomain.newserial
644
645               Reassociates the text with the specified instance domain.  As a
646               special case, newserial could be an asterisk ``*'' which  means
647               use  serial from the text rewriting rule, although this is most
648               useful when serial is also an asterisk.  So for example:
649                   text indom 29.* all { indom -> 109.* }
650               will reassociate all text associated with instance domains from
651               domain 29 to domain 109.
652
653           METRIC -> newdomain.newcluster.newitem
654
655               Reassociates  the  text  with the specified metric.  As special
656               cases, newcluster could be an asterisk ``*''  which  means  use
657               cluster  from  the  text rewriting rule and/or item could be an
658               asterisk which means use item from  the  text  rewriting  rule.
659               This  is most useful when cluster and/or item is also an aster‐
660               isk.  So for example:
661                   text metric 30.*.* all { metric -> 123.*.* }
662               will reassociate all text associated with metrics  from  domain
663               30 to domain 123.
664
665           TEXT -> "new-text"
666
667               Replaces the content of the selected text with new-text.
668
669       A label rewriting rule modifies a label record and has the form:
670
671       LABEL  labelid [ instance ] [ "label-name" ] [ "label-value" ] { label‐
672       spec ...  }
673
674       where labelid refers to the global context or identifies the metric do‐
675       main,  metric cluster, metric item, instance domain, or instance domain
676       instances with which the label is currently associated, and  is  either
677       CONTEXT or DOMAIN domainid or CLUSTER domainid.clusterid or ITEM metri‐
678       cid or INDOM domain.serial or INSTANCES domain.serial.
679
680       metricid has the same form and meaning as for a METRIC  rewriting  rule
681       (see  above).   clusterid may be an asterisk ``*'' which means the rule
682       applies to every metric with a domain number of domainid  in  the  same
683       way as an asterisk may be used for the cluster within metricid.
684
685       domain.serial  has  the same form and meaning as for an INDOM rewriting
686       rule (see above).
687
688       In the case of an INSTANCES labelid, the name or number of  a  specific
689       instance  may be optionally specified as instance.  This name or number
690       number may be omitted or specified as an  asterisk  ``*''  to  indicate
691       that  labels for all instances of the specified instance domain are se‐
692       lected.  If an instance name is specified, it  must  be  within  double
693       quotes.  If the instance name contains any embedded spaces then special
694       care needs to be taken in respect of the PCP instance naming rule  that
695       treats  the  leading  non-space part of the instance name as the unique
696       portion of the name for the purposes of matching and  ensuring  unique‐
697       ness  within an instance domain, refer to pmdaInstance(3) for a discus‐
698       sion of this issue.
699
700       In all cases, a "label-name" and/or a "label-value" may  be  optionally
701       specified  in  double  quotes  in order to select labels with the given
702       name and/or given value.  These may individually be omitted  or  speci‐
703       fied  as  asterisks ``*'' to indicate that labels with all names and/or
704       values are selected.
705
706       If a designated label record is not in inlog the rule has no effect.
707
708       The labelspec is zero or more of the following clauses:
709
710           DELETE
711
712               The selected labels are completely removed from outlog.
713
714           NEW "new-label-name" "new-label-value"
715
716               A new label with the name "new-label-name" and the value  "new-
717               label-value"  is  created and associated with the specified la‐
718               belid and optional instance (in the case  of  a  INSTANCES  la‐
719               belid).   If "label-name" or "label-value" were specified, then
720               they are ignored with a warning.  If instance is not  specified
721               for  an INSTANCES labelid, then a new label will be created for
722               each instance in the specified instance domain.
723
724           LABEL -> "new-label-name"
725
726               The name of the selected label(s)  is  changed  to  "new-label-
727               name".
728
729           VALUE -> "new-label-value"
730
731               The  value  of  the selected label(s) is changed to "new-label-
732               value".
733
734           DOMAIN -> newdomain
735
736               Reassociates the selected label(s) with  the  specified  metric
737               domain.  For example:
738                   label domain 30 { domain -> 123 }
739               will reassociate all labels associated with domains from domain
740               30 to domain 123.
741
742           CLUSTER -> newdomain.newcluster
743
744               Reassociates the selected label(s) with  the  specified  metric
745               cluster.   As  a  special case, newcluster could be an asterisk
746               ``*'' which means use cluster from the  label  rewriting  rule.
747               This  is  most useful when cluster is also an asterisk.  So for
748               example:
749                   label cluster 30.* { cluster -> 123.* }
750               will reassociate all labels associated with clusters  from  do‐
751               main 30 to domain 123.
752
753           ITEM -> newdomain.newcluster.newitem
754
755               Reassociates  the  selected  label(s) with the specified metric
756               item.  As special cases, newcluster could be an asterisk  ``*''
757               which  means  use  cluster from the label rewriting rule and/or
758               item could be an asterisk which means use item from  the  label
759               rewriting  rule.   This is most useful when cluster and/or item
760               is also an asterisk.  So for example:
761                   label item 30.*.* { item -> 123.*.* }
762               will reassociate all labels associated with metrics from domain
763               30 to domain 123.
764
765           INDOM -> newdomain.newserial
766
767               Reassociates  the selected label(s) with the specified instance
768               domain.  As a special case,  newserial  could  be  an  asterisk
769               ``*'' which means use serial from the label rewriting rule, al‐
770               though this is most useful when serial is also an asterisk.  So
771               for example:
772                   label indom 29.* { indom -> 109.* }
773               will  reassociate  all  labels associated with instance domains
774               from domain 29 to domain 109.
775
776           INSTANCES -> newdomain.newserial
777
778               This is the same as INDOM except that it reassociates  the  se‐
779               lected  label(s)  with  the instances of the specified instance
780               domain.
781

EXAMPLES

783       To promote the values of the per-disk IOPS metrics to 64-bit  to  allow
784       aggregation  over  a long time period for capacity planning, or because
785       the PMDA has changed to export 64-bit counters and we want  to  convert
786       old archives so they can be processed alongside new archives.
787           metric disk.dev.read { type -> U64 }
788           metric disk.dev.write { type -> U64 }
789           metric disk.dev.total { type -> U64 }
790
791       The  instances  associated with the load average metric kernel.all.load
792       could be renamed and renumbered by the rules below.
793           # for the Linux PMDA, the kernel.all.load metric is defined
794           # over instance domain 60.2
795           indom 60.2 {
796               inst 1 -> 60 iname "1 minute" -> "60 second"
797               inst 5 -> 300 iname "5 minute" -> "300 second"
798               inst 15 -> 900 iname "15 minute" -> "900 second"
799           }
800
801       If we decide to split the ``proc'' metrics out of the Linux PMDA,  this
802       will  involve  changing the domain number for the PMID of these metrics
803       and the associated instance domains.  The rules below would rewrite  an
804       old archive to match the changes after the PMDA split.
805           # all Linux proc metrics are in 7 clusters
806           metric 60.8.* { pmid -> 123.*.* }
807           metric 60.9.* { pmid -> 123.*.* }
808           metric 60.13.* { pmid -> 123.*.* }
809           metric 60.24.* { pmid -> 123.*.* }
810           metric 60.31.* { pmid -> 123.*.* }
811           metric 60.32.* { pmid -> 123.*.* }
812           metric 60.51.* { pmid -> 123.*.* }
813           # only one instance domain for Linux proc metrics
814           indom 60.9 { indom -> 123.0 }
815
816       If  the  metric  foo.count_em was exported as a native ``long'' then it
817       could be a 32-bit integer on some platforms and  a  64-bit  integer  on
818       other  platforms.   Subsequent investigations show the value is in fact
819       unsigned, so the following rules could be used.
820           metric foo.count_em {
821                type if 32 -> U32
822                type if 64 -> U64
823           }
824

DIAGNOSTICS

826       All error conditions detected by pmlogrewrite are  reported  on  stderr
827       with textual (if sometimes terse) explanation.
828
829       Should  the input archive be corrupted (this can happen if the pmlogger
830       instance writing the archive suddenly dies), then pmlogrewrite will de‐
831       tect  and  report  the  position of the corruption in the file, and any
832       subsequent information from that archive will not be processed.
833
834       If the input archive contains no archive records then  an  ``empty  ar‐
835       chive'' warning is issued and no processing is performed.
836
837       If  any  error is detected, pmlogrewrite will exit with a non-zero sta‐
838       tus.
839

FILES

841       For each of the inlog and outlog archives, several physical  files  are
842       used.
843
844       archive.meta
845            metadata (metric descriptions, instance domains, etc.) for the ar‐
846            chive
847
848       archive.0
849            initial volume of metrics values (subsequent volumes have suffixes
850            1, 2, ...).
851
852       archive.index
853            temporal  index  to support rapid random access to the other files
854            in the archive.
855

PCP ENVIRONMENT

857       Environment variables with the prefix PCP_ are used to parameterize the
858       file  and  directory names used by PCP.  On each installation, the file
859       /etc/pcp.conf contains the  local  values  for  these  variables.   The
860       $PCP_CONF  variable may be used to specify an alternative configuration
861       file, as described in pcp.conf(5).
862
863       For environment variables affecting PCP tools, see pmGetOptions(3).
864

SEE ALSO

866       PCPIntro(1), pmdumplog(1), pmlogextract(1), pmlogger(1), pmloglabel(1),
867       pmlogredact(1),    pmlogreduce(1),   PMAPI(3),   pmdaInstance(3),   pm‐
868       LookupDesc(3),  tzset(3),   LOGARCHIVE(5),   pcp.conf(5),   pcp.env(5),
869       PMNS(5) and regex(7).
870
871
872
873Performance Co-Pilot                                           PMLOGREWRITE(1)
Impressum