1PMLOGREDUCE(1) General Commands Manual PMLOGREDUCE(1)
2
6 pmlogreduce - temporal reduction of Performance Co-Pilot archives
7
9 pmlogreduce [-z?] [-A align] [-s samples] [-S starttime] [-t interval]
10 [-T endtime] [-v volsamples] [-Z timezone] input output
11
13 pmlogreduce reads one set of Performance Co-Pilot (PCP) archives iden‐
14 tified by input and creates a temporally reduced PCP archive in output.
15 input is a comma-separated list of names, each of which may be the base
16 name of an archive or the name of a directory containing one or more
17 archives. The data reduction involves statistical and temporal reduc‐
18 tion of samples with an output sampling interval defined by the -t op‐
19 tion in the output archive (independent of the sampling intervals in
20 the input archives), and is further controlled by other command line
21 arguments.
22 For some metrics, temporal data reduction is not going to be helpful,
24 so for metrics with types PM_TYPE_AGGREGATE or PM_TYPE_EVENT, a warning
25 is issued if these metrics are found in input and they will be skipped
26 and not appear in the output archive.
27
29 The available command line options are:
30 -A align, --align=align
32 Specify a ``natural'' alignment of the output sample times; refer
33 to PCPIntro(1).
34 -s samples, --samples=samples
36 The argument samples defines the number of samples to be written
37 to output. If samples is 0 or -s is not specified, pmlogreduce
38 will sample until the end of the set of PCP archives, or the end
39 of the time window as specified by -T, whichever comes first. The
40 -s option will override the -T option if it occurs sooner.
41 -S starttime, --start=starttime
43 Define the start of a time window to restrict the samples re‐
44 trieved from the input archives; refer to PCPIntro(1).
45 -t interval, --interval=interval
47 Consecutive samples in the output archive will appear with a time
48 delta defined by interval; refer to PCPIntro(1). Note the default
49 value is 600 (seconds, i.e. 10 minutes).
50 -T endtime, --finish=endtime
52 Define the termination of a time window to restrict the samples
53 retrieved from the input archives; refer to PCPIntro(1).
54 -v volsamples
56 The output archive is potentially a multi-volume data set, and the
57 -v option causes pmlogreduce to start a new volume after volsam‐
58 ples records have been written to the output archive.
59 Independent of any -v option, each volume of an archive is lim‐
61 ited to no more than 2^31 bytes, so pmlogreduce will automati‐
62 cally create a new volume for the archive before this limit is
63 reached.
64 -z, --hostzone
66 Use the local timezone of the host from the input archives when
67 displaying the date and time, or interpreting the -S and -T op‐
68 tions. The default is to initially use the timezone of the local
69 host.
70 -Z timezone, --timezone=timezone
72 Use timezone when displaying the date and time, or interpreting
73 the -S and -T options. Timezone is in the format of the environ‐
74 ment variable TZ as described in environ(7).
75 -?, --help
77 Display usage message and exit.
78
80 The statistical and temporal reduction follows the following rules:
81 1. Consecutive records from input are read without interpolation, and
83 at most one output record is written for each interval, summarizing
84 the performance data over that period.
85 2. If the semantics of a metric indicates it is instantaneous or dis‐
87 crete then output value is computed as the arithmetic mean of the
88 observations (if any) over each interval.
89 3. If the semantics of a metric indicates it is a counter then the
91 following transformations are applied:
92 a) Metrics with 32-bit precision are promoted to 64-bit precision.
93 b) Any counter wrap (overflow) is noted, and appropriate adjust‐
94 ment made in the value of the metric over each interval. This
95 will be correct in the case of a single counter wrap, but will
96 silently underestimate in the case where more than one counter
97 wrap occurs between consecutive observations in the input ar‐
98 chives, and silently overestimate in the case where a counter
99 reset occurs between consecutive observations in the input ar‐
100 chives; unfortunately these situations cannot be detected, but
101 are believed to be rare events for the sort of production moni‐
102 toring environments where pmlogreduce is most likely to be de‐
103 ployed.
104 4. Any changes in instance domains, and indeed all metadata, is pre‐
106 served.
107 5. Any ``mark'' records in the input archives (as created by pmlogex‐
109 tract(1)) will be preserved in the output archive, so periods where
110 no data is available are maintained, and data interpolation will
111 not occur across these periods when the output archive is subse‐
112 quently processed with PCP applications.
113
115 The preamble metrics (pmcd.pmlogger.archive, pmcd.pmlogger.host, and
116 pmcd.pmlogger.port), which are automatically recorded by pmlogger at
117 the start of the archive, may not be present in the archive output by
118 pmlogreduce. These metrics are only relevant while the archive is be‐
119 ing created, and have no significance once recording has finished.
120
122 All error conditions detected by pmlogreduce are reported on stderr
123 with textual (if sometimes terse) explanation.
124 Should the input archives be corrupted (this can happen if the pmlogger
126 instance writing the archive suddenly dies), then pmlogreduce will de‐
127 tect and report the position of the corruption in the file, and any
128 subsequent information from the input archives will not be processed.
129 If any error is detected, pmlogreduce will exit with a non-zero status.
131
133 For each of the input and output archives, several physical files are
134 used.
135 archive.meta
137 metadata (metric descriptions, instance domains, etc.) for the ar‐
138 chive
139 archive.0
141 initial volume of metrics values (subsequent volumes have suffixes
142 1, 2, ...) - for input these files may have been previously com‐
143 pressed with bzip2(1) or gzip(1) and thus may have an additional
144 .bz2 or .gz suffix.
145 archive.index
147 temporal index to support rapid random access to the other files
148 in the archive.
149
151 Environment variables with the prefix PCP_ are used to parameterize the
152 file and directory names used by PCP. On each installation, the file
153 /etc/pcp.conf contains the local values for these variables. The
154 $PCP_CONF variable may be used to specify an alternative configuration
155 file, as described in pcp.conf(5).
156
158 PCPIntro(1), pmdumplog(1), pmlc(1), pmlogextract(1), pmlogger(1),
159 pcp.conf(5) and pcp.env(5).
160Performance Co-Pilot PCP PMLOGREDUCE(1)