1PMLOGREDUCE(1)              General Commands Manual             PMLOGREDUCE(1)
2
3
4

NAME

6       pmlogreduce - temporal reduction of Performance Co-Pilot archives
7

SYNOPSIS

9       $PCP_BINADM_DIR/pmlogreduce [-z] [-A align] [-S starttime] [-s samples]
10       [-T endtime] [-t interval] [-v volsamples] [-Z timezone] input output
11

DESCRIPTION

13       pmlogreduce reads one set of Performance Co-Pilot (PCP) archives  iden‐
14       tified by input and creates a temporally reduced PCP archive in output.
15       input is a comma-separated list of names, each of which may be the base
16       name  of  an  archive or the name of a directory containing one or more
17       archives.  The data reduction involves statistical and temporal  reduc‐
18       tion  of  samples  with  an  output sampling interval defined by the -t
19       option in the output archive (independent of the sampling intervals  in
20       the  input  archives),  and is further controlled by other command line
21       arguments.
22
23       For some metrics, temporal data reduction is not going to  be  helpful,
24       so for metrics with types PM_TYPE_AGGREGATE or PM_TYPE_EVENT, a warning
25       is issued if these metrics are found in input and they will be  skipped
26       and not appear in the output archive.
27

OPTIONS

29       The command line options for pmlogreduce are as follows:
30
31       -A align
32              Specify  a  ``natural''  alignment  of  the output sample times;
33              refer to PCPIntro(1).
34
35       -S starttime
36              Define the start of  a  time  window  to  restrict  the  samples
37              retrieved from the input archives; refer to PCPIntro(1).
38
39       -s samples
40              The argument samples defines the number of samples to be written
41              to output.  If samples is 0 or -s is not specified,  pmlogreduce
42              will sample until the end of the set of PCP archives, or the end
43              of the time window as specified by -T,  whichever  comes  first.
44              The -s option will override the -T option if it occurs sooner.
45
46       -T endtime
47              Define  the termination of a time window to restrict the samples
48              retrieved from the input archives; refer to PCPIntro(1).
49
50       -v volsamples
51              The output archive is potentially a multi-volume data  set,  and
52              the  -v  option  causes  pmlogreduce to start a new volume after
53              volsamples log records have been written to the output archive.
54
55              Independent of any -v option, each volume of an archive is  lim‐
56              ited  to  no more than 2^31 bytes, so pmlogreduce will automati‐
57              cally create a new volume for the archive before this  limit  is
58              reached.
59
60       -t interval
61              Consecutive  samples  in  the  output archive will appear with a
62              time delta defined by interval; refer to PCPIntro(1).  Note  the
63              default value is 600 (seconds, i.e. 10 minutes).
64
65       -Z timezone
66              Use  timezone when displaying the date and time, or interpreting
67              the -S and -T options.  Timezone is in the format of  the  envi‐
68              ronment variable TZ as described in environ(7).
69
70       -z     Use  the local timezone of the host from the input archives when
71              displaying the date and time, or  interpreting  the  -S  and  -T
72              options.   The  default  is to initially use the timezone of the
73              local host.
74

DATA REDUCTION

76       The statistical and temporal reduction follows the following rules:
77
78       1.  Consecutive records from input are read without interpolation,  and
79           at most one output record is written for each interval, summarizing
80           the performance data over that period.
81
82       2.  If the semantics of a metric indicates it is instantaneous or  dis‐
83           crete  then  output value is computed as the arithmetic mean of the
84           observations (if any) over each interval.
85
86       3.  If the semantics of a metric indicates it is  a  counter  then  the
87           following transformations are applied:
88           a)  Metrics with 32-bit precision are promoted to 64-bit precision.
89           b)  Any  counter  wrap (overflow) is noted, and appropriate adjust‐
90               ment made in the value of the metric over each interval.   This
91               will  be correct in the case of a single counter wrap, but will
92               silently underestimate in the case where more than one  counter
93               wrap  occurs  between consecutive observations in the input ar‐
94               chives, and silently overestimate in the case where  a  counter
95               reset  occurs between consecutive observations in the input ar‐
96               chives; unfortunately these situations cannot be detected,  but
97               are believed to be rare events for the sort of production moni‐
98               toring environments where pmlogreduce  is  most  likely  to  be
99               deployed.
100
101       4.  Any  changes  in instance domains, and indeed all metadata, is pre‐
102           served.
103
104       5.  Any ``mark'' records in the input archives (as created by  pmlogex‐
105           tract(1)) will be preserved in the output archive, so periods where
106           no data is available are maintained, and  data  interpolation  will
107           not  occur  across  these periods when the output archive is subse‐
108           quently processed with PCP applications.
109

FILES

111       For each of the input and output archives, several physical  files  are
112       used.
113       archive.meta
114                 metadata  (metric  descriptions,  instance domains, etc.) for
115                 the archive log
116       archive.0 initial volume of metrics  values  (subsequent  volumes  have
117                 suffixes  1,  2,  ...)  - for input these files may have been
118                 previously compressed with bzip2(1) or gzip(1) and  thus  may
119                 have an additional .bz2 or .gz suffix.
120       archive.index
121                 temporal  index  to  support rapid random access to the other
122                 files in the archive log.
123

PCP ENVIRONMENT

125       Environment variables with the prefix PCP_ are used to parameterize the
126       file  and  directory names used by PCP.  On each installation, the file
127       /etc/pcp.conf contains the  local  values  for  these  variables.   The
128       $PCP_CONF  variable may be used to specify an alternative configuration
129       file, as described in pcp.conf(5).
130

SEE ALSO

132       PCPIntro(1),  pmdumplog(1),  pmlc(1),   pmlogextract(1),   pmlogger(1),
133       pcp.conf(5) and pcp.env(5).
134

DIAGNOSTICS

136       All  error  conditions  detected  by pmlogreduce are reported on stderr
137       with textual (if sometimes terse) explanation.
138
139       Should the input archives be corrupted (this can happen if the pmlogger
140       instance  writing  the  archive  suddenly  dies), then pmlogreduce will
141       detect and report the position of the corruption in the file,  and  any
142       subsequent information from the input archives will not be processed.
143
144       If any error is detected, pmlogreduce will exit with a non-zero status.
145

CAVEATS

147       The  preamble  metrics  (pmcd.pmlogger.archive, pmcd.pmlogger.host, and
148       pmcd.pmlogger.port), which are automatically recorded  by  pmlogger  at
149       the  start  of the archive, may not be present in the archive output by
150       pmlogreduce.  These metrics are only  relevant  while  the  archive  is
151       being created, and have no significance once recording has finished.
152
153
154
155Performance Co-Pilot                  PCP                       PMLOGREDUCE(1)
Impressum