1PMREP(1) General Commands Manual PMREP(1)
2
6 pmrep - performance metrics reporter
7
9 pmrep [-123CdgGHIjkLnprRuUvVxz?] [-8|-9 limit] [-a archive] [--ar‐
10 chive-folio folio] [-A align] [-b|-B space-scale] [-c config] [--con‐
11 tainer container] [--daemonize] [-e derived] [-E lines] [-f format] [-F
12 outfile] [-h host] [-i instances] [-J rank] [-K spec] [-l delimiter]
13 [-N predicate] [-o output] [-O origin] [-P|-0 precision] [-q|-Q count-
14 scale] [-s samples] [-S starttime] [-t interval] [-T endtime] [-w|-W
15 width] [-X label] [-y|-Y time-scale] [-Z timezone] metricspec [...]
16
18 pmrep is a customizable performance metrics reporting tool. Any avail‐
19 able performance metric, live or archived, system and/or application,
20 can be selected for reporting using one of the output alternatives
21 listed below together with applicable formatting options.
22 pmrep collects the selected metric values through the facilities of the
24 Performance Co-Pilot (PCP), see PCPIntro(1). The metrics to be
25 reported are specified on the command line, in a configuration file, or
26 both. Metrics can be automatically converted and scaled using the PCP
27 facilities, either by default or by per-metric scaling specifications.
28 In addition to the existing metrics, derived metrics can be defined
29 using the arithmetic expressions described in pmRegisterDerived(3).
30 Unless directed to another host by the -h option, pmrep will contact
32 the Performance Metrics Collector Daemon (PMCD, see pmcd(1)) on the
33 local host.
34 The -a option causes pmrep to use the specified set of archive logs
36 rather than connecting to a PMCD. The -a and -h options are mutually
37 exclusive.
38 The -L option causes pmrep to use a local context to collect metrics
40 from DSO PMDAs (Performance Metrics Domain Agents, ``plugins'') on the
41 local host without PMCD. Only some metrics are available in this mode.
42 The -a, -h, and -L options are mutually exclusive.
43 The metrics of interest are named in the metricspec argument(s). If a
45 metricspec specifies a non-leaf node in the Performance Metrics Name
46 Space (PMNS), then pmrep will recursively descend the PMNS and report
47 on all leaf nodes (i.e., metrics) for that metricspec. (Use for exam‐
48 ple pminfo(1) to list all the leaf nodes and their descriptions.)
49 A metricspec has three different forms. First, on the command line it
51 can start with a colon (``:'') to indicate a metricset to be read from
52 a pmrep configuration file (see pmrep.conf(5)) which can then consist
53 of any number of metricspecs. Second, a metricspec starting with non-
54 colon specifies a PMNS node as described above, optionally followed by
55 metric formatting definitions. This so-called compact form of a met‐
56 ricspec is defined as follows:
57 metric[,label[,instances[,unit/scale[,type[,width[,precision[,limit]]]]]]]
59 A valid PMNS node (metric) is mandatory. It can be followed by a text
61 label used by supporting output targets (currently: stdout, see below).
62 The optional instances definition restricts csv and stdout reporting to
63 the specified instances (so non-matching instances will be filtered).
64 An optional unit/scale is applicable for dimension-compatible, non-
65 string metrics. (See below for supported unit/scale specifications.)
66 By default, cumulative counter metrics are converted to rates, an
67 optional type can be set to raw to disable this rate conversion. For
68 supporting output targets (currently: stdout) a numeric width can be
69 used to set the width of the output column for this metric. Too-wide
70 numeric values for output will not be printed (apart from trailing dec‐
71 imals, numeric values will never be silently truncated). Too-wide
72 strings will be truncated. Then, a metric-specific precision can be
73 provided for numeric non-integer output values. Lastly, a metric-spe‐
74 cific limit can be set for filtering numeric values per limit.
75 As a special case with metrics that are counters with time units
77 (nanoseconds to hours), the unit/scale can be used to change the
78 default reporting (for example, milliseconds / second) to normalize to
79 the range zero to one by setting this to sec (see also -y and -Y).
80 The following metricspec requests the metric kernel.all.sysfork to be
82 reported under the text label forks, converting to the default rate
83 count/s in an 8 wide column. Although the definitions in this compact
84 form are optional, they must always be provided in the order specified
85 above.
86 kernel.all.sysfork,forks,,,,8
88 The third form of a metricspec, verbose form, is described and valid
90 only in pmrep.conf(5).
91 Derived metrics are specified like PMNS leaf node metrics.
93 Options via environment values (see pmGetOptions(3)) override the cor‐
95 responding built-in default values (if any). Configuration file
96 options override the corresponding environment variables (if any).
97 Command line options override the corresponding configuration file
98 options (if any).
99
101 The available command line options are:
102 -1, --dynamic-header
104 Print a new dynamically adjusted header every time changes in
105 availability of metric and instance values occur. By default a
106 static header that never changes is printed once. See also -E.
107 -2, --overall-rank
109 Perform overall ranking of instances in archive. By default rank‐
110 ing (see -J) and reporting happens on each interval. With this
111 option all instances and values are ranked before a summary is
112 reported. See pmlogsummary(1) for further archive summary report‐
113 ing alternatives, including averages and peak times for values.
114 -3, --overall-rank-alt
116 Like -2 but print results in pmrep metricspec format.
117 -8 limit, --limit-filter=limit
119 Limit results to instances with values above/below limit. A posi‐
120 tive integer will include instances with values at or above the
121 limit in reporting. A negative integer will include instances
122 with values at or below the limit in reporting. A value of zero
123 performs no limit filtering. This option will not override possi‐
124 ble per-metric specifications. See also -J and -N.
125 -9 limit, --limit-filter-force=limit
127 Like -8 but this option will override per-metric specifications.
128 -a archive, --archive=archive
130 Performance metric values are retrieved from the set of Perfor‐
131 mance Co-Pilot (PCP) archive log files identified by the argument
132 archive, which is a comma-separated list of names, each of which
133 may be the base name of an archive or the name of a directory con‐
134 taining one or more archives. See also -u.
135 --archive-folio
137 Read metric source archives from the PCP archive folio created by
138 tools like pmchart(1) or, less often, manually with mkaf(1).
139 -A align, --align=align
141 Force the initial sample to be aligned on the boundary of a natu‐
142 ral time unit align. Refer to PCPIntro(1) for a complete descrip‐
143 tion of the syntax for align.
144 -b scale, --space-scale=scale
146 Unit/scale for space (byte) metrics, possible values include
147 bytes, Kbytes, KB, Mbytes, MB, and so forth. This option will not
148 override possible per-metric specifications. See also pmParseU‐
149 nitsStr(3).
150 -B scale, --space-scale-force=scale
152 Like -b but this option will override per-metric specifications.
153 -c config, --config=config
155 Specify the config file to use. The default is the first found
156 of: ./pmrep.conf, $HOME/.pmrep.conf, $HOME/pcp/pmrep.conf, and
157 $PCP_SYSCONF_DIR/pmrep/pmrep.conf. See pmrep.conf(5).
158 --container
160 Fetch performance metrics from the specified container, either
161 local or remote (see -h).
162 -C, --check
164 Exit before reporting any values, but after parsing the configura‐
165 tion and metrics and printing possible headers.
166 -d, --delay
168 When replaying from an archive, this option requests that the pre‐
169 vailing real-time delay be applied between samples (see -t) to
170 effect a pause, rather than the default behaviour of replaying at
171 full speed.
172 --daemonize
174 Daemonize on startup.
175 -e derived, --derived=derived
177 Specify derived performance metrics. If derived starts with a
178 slash (``/'') or with a dot (``.'') it will be interpreted as a
179 derived metrics configuration file, otherwise it will be inter‐
180 preted as comma- or semicolon-separated derived metric expres‐
181 sions. For details see pmLoadDerivedConfig(3) and pmRegister‐
182 Derived(3).
183 -E lines, --repeat-header=lines
185 Repeat the header every lines of output. See also -1.
186 -f format, --timestamp-format=format
188 Use the format string for formatting the timestamp. The format
189 will be used with Python's datetime.strftime method which is
190 mostly the same as that described in strftime(3). An empty format
191 string (i.e., "") will remove the timestamps from the output.
192 Defaults to %H:%M:%S when using the stdout output target.
193 Defaults to %Y-%m-%d %H:%M:%S when using the csv output target.
194 -F outfile, --output-file=outfile
196 Specify the output file outfile. See -o.
197 -g, --separate-header
199 Output the column number and complete metric information, one-per-
200 line, before printing the metric values.
201 -G, --no-globals
203 Do not include global metrics in reporting (see pmrep.conf(5)).
204 -h host, --host=host
206 Fetch performance metrics from pmcd(1) on host, rather than from
207 the default localhost.
208 -H, --no-header
210 Do not print any headers.
211 -i instances, --instances=instances
213 Report only the listed instances from current instances (if
214 present, see also -j). By default all current instances are
215 reported, except when writing an archive (see -o) when all
216 instances, present and future, are reported. This is a global
217 option that is used for all metrics unless a metric-specific
218 instance definition is provided as part of a metricspec. By
219 default single-valued ``flat'' metrics without multiple instances
220 are still reported as usual, use -v to change this.
221 The list may consist of one or more comma-separated instances.
223 The instance name may be quoted with single (') or double (")
224 quotes for those cases where the instance name contains commas or
225 whitespace. Note that on the command line when specifying more
226 than one instance, all the names must be quoted.
227 Multiple -i options are allowed as an alternative way of specify‐
229 ing more than one instance of interest. Regular expressions can
230 also be used.
231 As an example, the following would report the same instances:
233 $ pmrep -i "'1 minute','5 minute'" kernel.all.load
235 $ pmrep -i '"1 minute","5 minute"' kernel.all.load
236 $ pmrep -i "'1 minute'" -i "'5 minute'" kernel.all.load
237 $ pmrep kernel.all.load,,"'1 minute','5 minute'"
238 $ pmrep kernel.all.load,,'"1 minute","5 minute"'
239 However, this would report only the 1-minute instance:
242 $ pmrep -i '"1 minute","5 minute"' kernel.all.load,,"1 minute"
244 But this would report all instances (due to per-metric override):
247 $ pmrep -i '"1 minute","5 minute"' 'kernel.all.load,,.*'
249 -I, --ignore-incompat
252 Ignore incompatible metrics. By default incompatible metrics
253 (that is, their type is unsupported or they cannot be scaled as
254 requested) will cause pmrep to terminate with an error message.
255 With this option all incompatible metrics are silently omitted
256 from reporting. This may be especially useful when requesting
257 non-leaf nodes of the PMNS tree for reporting.
258 -j, --live-filter
260 Perform instance live filtering. This allows capturing all fil‐
261 tered instances even if processes are restarted at some point
262 (unlike without live filtering). Doing live filtering over a huge
263 amount of instances naturally comes with some overhead so a bit of
264 user caution is advised. See also -1.
265 -J rank, --rank=rank
267 Limit results to highest/lowest rank instances of set-valued met‐
268 rics. A positive integer will include highest valued instances in
269 reporting. A negative integer will include lowest valued
270 instances in reporting. A value of zero performs no ranking. See
271 also -2 and -8.
272 -k, --extended-csv
274 Write extended CSV output, similar to sadf(1).
275 -K spec, --spec-local=spec
277 When fetching metrics from a local context (see -L), the -K option
278 may be used to control the DSO PMDAs that should be made accessi‐
279 ble. The spec argument conforms to the syntax described in
280 pmSpecLocalPMDA(3). More than one -K option may be used.
281 -l delimiter, --delimiter=delimiter
283 Specify the delimiter that separates each column of csv or stdout
284 output. The default for stdout is two spaces (`` '') and comma
285 (``,'') for csv. In case of CSV output or stdout output with non-
286 whitespace delimiter, any instances of the delimiter in string
287 values will be replaced by the underscore (``_'') character.
288 -L, --local-PMDA
290 Use a local context to collect metrics from DSO PMDAs on the local
291 host without PMCD. See also -K.
292 -n, --invert-filter
294 Perform ranking before live filtering. By default instance live
295 filter filtering (when requested, see -j) happens before instance
296 ranking (when requested, see -J). With this option the logic is
297 inverted and ranking happens before live filtering.
298 -N predicate, --predicate=predicate
300 Specify a comma-separated list of predicate filter reference met‐
301 rics. By default ranking (see -J) happens for each metric indi‐
302 vidually. With predicate filter reference metrics, ranking is
303 done only for the specified metrics. When reporting, the rest of
304 the metrics sharing the same instance domain (see PCPIntro(1)) as
305 the predicates will include only the highest/lowest ranking
306 instances of the corresponding predicates.
307 So for example, when the using proc.memory.rss (resident size of
309 process) as the predicate and including proc.io.total_bytes and
310 mem.util.used as metrics to be reported, only the processes using
311 most/least memory (as per -J) will be included when reporting
312 total bytes written by processes. Since mem.util.used is a sin‐
313 gle-valued metric (thus not sharing the same instance domain as
314 the process-related metrics), it will be reported as usual.
315 -o output, --output=output
317 Use output target for reporting. The default target is stdout.
318 The available target alternatives are:
319 archive
321 Record metrics into a PCP archive which can later be replayed
322 with PCP tools, including pmrep itself. See LOGARCHIVE(5) and
323 PCPIntro(1) for details about PCP archive files. Requires -F.
324 csv
326 Print metrics in CSV format (subject to formatting options).
327 stdout
329 Print metrics to stdout (format subject to formatting options).
330 -O origin, --origin=origin
332 When reporting archived metrics, start reporting at origin within
333 the time window (see -S and -T). Refer to PCPIntro(1) for a com‐
334 plete description of the syntax for origin.
335 -p, --timestamps
337 Print timestamps. By default no timestamps are printed.
338 -P precision, --precision=precision
340 Use precision for numeric non-integer output values. If the value
341 is too wide for its column width, precision is reduced one by one
342 until the value fits, or not printed at all if it does not. The
343 default is to use 3 decimal places (when applicable). This option
344 will not override possible per-metric specifications.
345 -0 precision, --precision-force=precision
347 Like -P but this option will override per-metric specifications.
348 -q scale, --count-scale=scale
350 Unit/scale for count metrics, possible values include count x
351 10^-1, count, count x 10, count x 10^2, and so forth from 10^-8 to
352 10^7. (These values are currently space-sensitive.) This option
353 will not override possible per-metric specifications. See also
354 pmParseUnitsStr(3).
355 -Q scale, --count-scale-force=scale
357 Like -q but this option will override per-metric specifications.
358 -r, --raw
360 Output raw metric values, do not convert cumulative counters to
361 rates. When writing archives, raw values are always used. This
362 option will override possible per-metric specifications.
363 -R, --raw-prefer
365 Like -r but this option will not override per-metric specifica‐
366 tions.
367 -s samples, --samples=samples
369 The argument samples defines the number of samples to be retrieved
370 and reported. If samples is 0 or -s is not specified, pmrep will
371 sample and report continuously (in real time mode) or until the
372 end of the set of PCP archives (in archive mode). See also -T.
373 -S starttime, --start=starttime
375 When reporting archived metrics, the report will be restricted to
376 those records logged at or after starttime. Refer to PCPIntro(1)
377 for a complete description of the syntax for starttime.
378 -t interval, --interval=interval
380 The default update interval may be set to something other than the
381 default 1 second. The interval argument follows the syntax
382 described in PCPIntro(1), and in the simplest form may be an
383 unsigned integer (the implied units in this case are seconds).
384 See also the -T and -u options.
385 -T endtime, --finish=endtime
387 When reporting archived metrics, the report will be restricted to
388 those records logged before or at endtime. Refer to PCPIntro(1)
389 for a complete description of the syntax for endtime.
390 When used to define the runtime before pmrep will exit, if no sam‐
392 ples is given (see -s) then the number of reported samples depends
393 on interval (see -t). If samples is given then interval will be
394 adjusted to allow reporting of samples during runtime. In case
395 all of -T, -s, and -t are given, endtime determines the actual
396 time pmrep will run.
397 -u, --no-interpol
399 When reporting archived metrics, by default values are reported
400 according to the selected sample interval (-t option), not accord‐
401 ing to the actual record interval in an archive. To this effect
402 PCP interpolates the values to be reported based on the records in
403 the archive. With the -u option uninterpolated reporting is
404 enabled, every recorded value for the selected metrics is reported
405 and the requested sample interval (-t) is ignored.
406 So for example, if a PCP archive contains recorded values for
408 every 10 seconds and the requested sample interval is 1 hour, by
409 default pmrep will use an interpolation scheme to compute the val‐
410 ues of the requested metrics from the values recorded in the prox‐
411 imity of these requested metrics and values for every 1 hour are
412 reported. With -u every record every 10 seconds are reported as
413 such (the reported values are still subject to rate conversion,
414 use -r or -R to disable).
415 -U, --no-unit-info
417 Omit unit information from headers.
418 -v, --omit-flat
420 Omit single-valued ``flat'' metrics from reporting, only consider
421 set-valued metrics (i.e., metrics with multiple values) for
422 reporting. See -i and -I.
423 -V, --version
425 Display version number and exit.
426 -w width, --width=width
428 Set the stdout output column width. Strings will be truncated to
429 this width. The default width is the shortest that can fit the
430 metric text label, the forced minimum is 3. This option will not
431 override possible per-metric specifications.
432 -W width, --width-force=width
434 Like -w but this option will override per-metric specifications.
435 -x, --extended-header
437 Print extended header.
438 -X label, --colxrow=label
440 Swap columns and rows in stdout output, reporting one instance per
441 line, using label as the text label for instance column (set to an
442 empty string "" to enable swapping without a specific text label).
443 This is convenient to allow easily using grep(1) to filter results
444 or to more closely mimic other utilities. See also -i.
445 -y scale, --time-scale=scale
447 Unit/scale for time metrics, possible values include nanosec, ns,
448 microsec, us, millisec, ms, and so forth up to hour, hr. This
449 option will not override possible per-metric specifications. See
450 also pmParseUnitsStr(3).
451 -Y scale, --time-scale-force=scale
453 Like -y but this option will override per-metric specifications.
454 -z, --hostzone
456 Use the local timezone of the host that is the source of the per‐
457 formance metrics, as identified by either the -h or the -a
458 options. The default is to use the timezone of the local host.
459 -Z timezone, --timezone=timezone
461 Use timezone for the date and time. Timezone is in the format of
462 the environment variable TZ as described in environ(7). Note that
463 when including a timezone string in output, ISO 8601 -style UTC
464 offsets are used (so something like -Z EST+5 will become UTC-5).
465 -?, --help
467 Display usage message and exit.
468
470 The following examples use the standard PCP facilities for collecting
471 the metric values, no external utilities are needed. The referenced
472 colon-starting metricsets are part of the system pmrep.conf file.
473 Display network interface metrics on the local host:
475 $ pmrep network.interface.total.bytes
476 Display all outgoing network metrics for the wlan0 interface:
478 $ pmrep -i wlan0 -v network.interface.out
479 Display timestamped vmstat(8) like information using megabytes instead
481 of kilobytes and also include the number of inodes used (tab completes
482 available metrics and after a colon metricsets with bash and zsh):
483 $ pmrep -p -B MB :vmstat vfs.inodes.count
484 Display per-device disk reads and writes from the host server1 using
486 two seconds interval and sadf(1) like CSV output format:
487 $ pmrep -h server1 -t 2s -o csv -k disk.dev.read disk.dev.write
488 Display processes using at least 100MB of memory using dynamic headers:
490 $ pmrep -b MB --limit-filter 100 --dynamic-header proc.memory.rss
491 Display the predefined set of metrics from the default pmrep.conf(5)
493 containing information about I/O issued by current firefox process(es):
494 $ pmrep -i '.*firefox.*' :proc-io
495 Display sar -w and sar -W like information at the same time from the
497 PCP archive ./20150921.09.13 showing values recorded between 3 - 5 PM:
498 $ pmrep -a ./20150921.09.13 -S @15:00 -T @17:00 :sar-w :sar-W
499 Record most relevant CPU, memory, and I/O related information about
501 every Java process on the system, present and future, to an archive ./a
502 on one minute interval at every full minute in a background process:
503 $ pmrep --daemonize -A 1m -t 1m -i '.*java.*' -j -o archive -F ./a \
504 :proc-info :proc-cpu :proc-mem :proc-io
505 Record all 389 Directory Server, XFS file system, and CPU/memory/disk
507 metrics every five seconds for five minutes to a PCP archive ./a:
508 $ pmrep -t 5s -T 5m -o archive -F ./a ds389 xfs kernel.all.cpu mem disk
509 Record process memory and I/O information for those processes which are
511 the three most memory-consuming processes:
512 $ pmrep -o archive -F ./a -J 3 -N proc.memory.rss proc.memory proc.io
513
515 pmrep.conf
516 pmrep configuration file (see -c)
517 $PCP_SYSCONF_DIR/pmrep/pmrep.conf
519 system provided pmrep configuration file
520
522 Environment variables with the prefix PCP_ are used to parameterize the
523 file and directory names used by PCP. On each installation, the file
524 /etc/pcp.conf contains the local values for these variables. The
525 $PCP_CONF variable may be used to specify an alternative configuration
526 file, as described in pcp.conf(5).
527 For environment variables affecting PCP tools, see pmGetOptions(3).
529
531 mkaf(1), PCPIntro(1), pcp(1), pcp-atop(1), pcp2elasticsearch(1),
532 pcp2graphite(1), pcp2influxdb(1), pcp2json(1), pcp2xlsx(1), pcp2xml(1),
533 pcp2zabbix(1), pmcd(1), pmchart(1), pmcollectl(1), pmdiff(1),
534 pmdumplog(1), pmdumptext(1), pminfo(1), pmiostat(1), pmlogextract(1),
535 pmlogsummary(1), pmprobe(1), pmstat(1), pmval(1), sadf(1), sar(1),
536 pmGetOptions(3), pmSpecLocalPMDA(3), pmLoadDerivedConfig(3), pmParseU‐
537 nitsStr(3), pmRegisterDerived(3), strftime(3), LOGARCHIVE(5),
538 pcp.conf(5), pmns(5), pmrep.conf(5), environ(7) and vmstat(8).
539Performance Co-Pilot PCP PMREP(1)