1PMLOGREDUCE(1) General Commands Manual PMLOGREDUCE(1)
2
6 pmlogreduce - temporal reduction of Performance Co-Pilot archives
7
9 $PCP_BINADM_DIR/pmlogreduce [-z?] [-A align] [-s samples] [-S start‐
10 time] [-t interval] [-T endtime] [-v volsamples] [-Z timezone] input
11 output
12
14 pmlogreduce reads one set of Performance Co-Pilot (PCP) archives iden‐
15 tified by input and creates a temporally reduced PCP archive in output.
16 input is a comma-separated list of names, each of which may be the base
17 name of an archive or the name of a directory containing one or more
18 archives. The data reduction involves statistical and temporal reduc‐
19 tion of samples with an output sampling interval defined by the -t
20 option in the output archive (independent of the sampling intervals in
21 the input archives), and is further controlled by other command line
22 arguments.
23 For some metrics, temporal data reduction is not going to be helpful,
25 so for metrics with types PM_TYPE_AGGREGATE or PM_TYPE_EVENT, a warning
26 is issued if these metrics are found in input and they will be skipped
27 and not appear in the output archive.
28
30 The available command line options are:
31 -A align, --align=align
33 Specify a ``natural'' alignment of the output sample times; refer
34 to PCPIntro(1).
35 -s samples, --samples=samples
37 The argument samples defines the number of samples to be written
38 to output. If samples is 0 or -s is not specified, pmlogreduce
39 will sample until the end of the set of PCP archives, or the end
40 of the time window as specified by -T, whichever comes first. The
41 -s option will override the -T option if it occurs sooner.
42 -S starttime, --start=starttime
44 Define the start of a time window to restrict the samples
45 retrieved from the input archives; refer to PCPIntro(1).
46 -t interval, --interval=interval
48 Consecutive samples in the output archive will appear with a time
49 delta defined by interval; refer to PCPIntro(1). Note the default
50 value is 600 (seconds, i.e. 10 minutes).
51 -T endtime, --finish=endtime
53 Define the termination of a time window to restrict the samples
54 retrieved from the input archives; refer to PCPIntro(1).
55 -v volsamples
57 The output archive is potentially a multi-volume data set, and the
58 -v option causes pmlogreduce to start a new volume after volsam‐
59 ples log records have been written to the output archive.
60 Independent of any -v option, each volume of an archive is lim‐
62 ited to no more than 2^31 bytes, so pmlogreduce will automati‐
63 cally create a new volume for the archive before this limit is
64 reached.
65 -z, --hostzone
67 Use the local timezone of the host from the input archives when
68 displaying the date and time, or interpreting the -S and -T
69 options. The default is to initially use the timezone of the
70 local host.
71 -Z timezone, --timezone=timezone
73 Use timezone when displaying the date and time, or interpreting
74 the -S and -T options. Timezone is in the format of the environ‐
75 ment variable TZ as described in environ(7).
76 -?, --help
78 Display usage message and exit.
79
81 The statistical and temporal reduction follows the following rules:
82 1. Consecutive records from input are read without interpolation, and
84 at most one output record is written for each interval, summarizing
85 the performance data over that period.
86 2. If the semantics of a metric indicates it is instantaneous or dis‐
88 crete then output value is computed as the arithmetic mean of the
89 observations (if any) over each interval.
90 3. If the semantics of a metric indicates it is a counter then the
92 following transformations are applied:
93 a) Metrics with 32-bit precision are promoted to 64-bit precision.
94 b) Any counter wrap (overflow) is noted, and appropriate adjust‐
95 ment made in the value of the metric over each interval. This
96 will be correct in the case of a single counter wrap, but will
97 silently underestimate in the case where more than one counter
98 wrap occurs between consecutive observations in the input ar‐
99 chives, and silently overestimate in the case where a counter
100 reset occurs between consecutive observations in the input ar‐
101 chives; unfortunately these situations cannot be detected, but
102 are believed to be rare events for the sort of production moni‐
103 toring environments where pmlogreduce is most likely to be
104 deployed.
105 4. Any changes in instance domains, and indeed all metadata, is pre‐
107 served.
108 5. Any ``mark'' records in the input archives (as created by pmlogex‐
110 tract(1)) will be preserved in the output archive, so periods where
111 no data is available are maintained, and data interpolation will
112 not occur across these periods when the output archive is subse‐
113 quently processed with PCP applications.
114
116 The preamble metrics (pmcd.pmlogger.archive, pmcd.pmlogger.host, and
117 pmcd.pmlogger.port), which are automatically recorded by pmlogger at
118 the start of the archive, may not be present in the archive output by
119 pmlogreduce. These metrics are only relevant while the archive is
120 being created, and have no significance once recording has finished.
121
123 All error conditions detected by pmlogreduce are reported on stderr
124 with textual (if sometimes terse) explanation.
125 Should the input archives be corrupted (this can happen if the pmlogger
127 instance writing the archive suddenly dies), then pmlogreduce will
128 detect and report the position of the corruption in the file, and any
129 subsequent information from the input archives will not be processed.
130 If any error is detected, pmlogreduce will exit with a non-zero status.
132
134 For each of the input and output archives, several physical files are
135 used.
136 archive.meta
138 metadata (metric descriptions, instance domains, etc.) for the ar‐
139 chive log
140 archive.0
142 initial volume of metrics values (subsequent volumes have suffixes
143 1, 2, ...) - for input these files may have been previously com‐
144 pressed with bzip2(1) or gzip(1) and thus may have an additional
145 .bz2 or .gz suffix.
146 archive.index
148 temporal index to support rapid random access to the other files
149 in the archive log.
150
152 Environment variables with the prefix PCP_ are used to parameterize the
153 file and directory names used by PCP. On each installation, the file
154 /etc/pcp.conf contains the local values for these variables. The
155 $PCP_CONF variable may be used to specify an alternative configuration
156 file, as described in pcp.conf(5).
157
159 PCPIntro(1), pmdumplog(1), pmlc(1), pmlogextract(1), pmlogger(1),
160 pcp.conf(5) and pcp.env(5).
161Performance Co-Pilot PCP PMLOGREDUCE(1)