1PMREP(1) General Commands Manual PMREP(1)
2
6 pmrep - performance metrics reporter
7
9 pmrep [-1235CdgGHIjkLnprRuUvVxz?] [-4 action] [-8|-9 limit] [-a ar‐
10 chive] [--archive-folio folio] [-A align] [-b|-B space-scale] [-c con‐
11 fig] [--container container] [--daemonize] [-e derived] [-E lines] [-f
12 format] [-F outfile] [-h host] [-i instances] [-J rank] [-K spec] [-l
13 delimiter] [-N predicate] [-o output] [-O origin] [-P|-0 precision]
14 [-q|-Q count-scale] [-s samples] [-S starttime] [-t interval] [-T end‐
15 time] [-w|-W width] [-X label] [-y|-Y time-scale] [-Z timezone] metric‐
16 spec [...]
17
19 pmrep is a customizable performance metrics reporting tool. Any avail‐
20 able performance metric, live or archived, system and/or application,
21 can be selected for reporting using one of the output alternatives
22 listed below together with applicable formatting options.
23 pmrep collects the selected metric values through the facilities of the
25 Performance Co-Pilot (PCP), see PCPIntro(1). The metrics to be
26 reported are specified on the command line, in a configuration file, or
27 both. Metrics can be automatically converted and scaled using the PCP
28 facilities, either by default or by per-metric scaling specifications.
29 In addition to the existing metrics, derived metrics can be defined
30 using the arithmetic expressions described in pmRegisterDerived(3).
31 Unless directed to another host by the -h option, pmrep will contact
33 the Performance Metrics Collector Daemon (PMCD, see pmcd(1)) on the
34 local host.
35 The -a option causes pmrep to use the specified set of archive logs
37 rather than connecting to a PMCD. The -a and -h options are mutually
38 exclusive.
39 The -L option causes pmrep to use a local context to collect metrics
41 from DSO PMDAs (Performance Metrics Domain Agents, ``plugins'') on the
42 local host without PMCD. Only some metrics are available in this mode.
43 The -a, -h, and -L options are mutually exclusive.
44 The metrics of interest are named in the metricspec argument(s). If a
46 metricspec specifies a non-leaf node in the Performance Metrics Name
47 Space (PMNS), then pmrep will recursively descend the PMNS and report
48 on all leaf nodes (i.e., metrics) for that metricspec. (Use for exam‐
49 ple pminfo(1) to list all the leaf nodes and their descriptions.)
50 A metricspec has three different forms. First, on the command line it
52 can start with a colon (``:'') to indicate a metricset to be read from
53 a pmrep configuration file (see pmrep.conf(5)) which can then consist
54 of any number of metricspecs. Second, a metricspec starting with non-
55 colon specifies a PMNS node as described above, optionally followed by
56 metric formatting definitions. This so-called compact form of a met‐
57 ricspec is defined as follows:
58 metric[,label[,instances[,unit/scale[,type[,width[,precision[,limit]]]]]]]
60 A valid PMNS node (metric) is mandatory. It can be followed by a text
62 label used by supporting output targets (currently: stdout, see below).
63 The optional instances definition restricts csv and stdout reporting to
64 the specified instances (so non-matching instances will be filtered).
65 An optional unit/scale is applicable for dimension-compatible, non-
66 string metrics. (See below for supported unit/scale specifications.)
67 By default, cumulative counter metrics are converted to rates, an
68 optional type can be set to raw to disable this rate conversion. For
69 supporting output targets (currently: stdout) a numeric width can be
70 used to set the width of the output column for this metric. Too-wide
71 numeric values for output will not be printed (apart from trailing dec‐
72 imals, numeric values will never be silently truncated). Too-wide
73 strings will be truncated. Then, a metric-specific precision can be
74 provided for numeric non-integer output values. Lastly, a metric-spe‐
75 cific limit can be set for filtering numeric values per limit.
76 As a special case with metrics that are counters with time units
78 (nanoseconds to hours), the unit/scale can be used to change the
79 default reporting (for example, milliseconds / second) to normalize to
80 the range zero to one by setting this to sec (see also -y and -Y).
81 The following metricspec requests the metric kernel.all.sysfork to be
83 reported under the text label forks, converting to the default rate
84 count/s in an 8 wide column. Although the definitions in this compact
85 form are optional, they must always be provided in the order specified
86 above.
87 kernel.all.sysfork,forks,,,,8
89 The third form of a metricspec, verbose form, is described and valid
91 only in pmrep.conf(5).
92 Derived metrics are specified like PMNS leaf node metrics.
94 Options via environment values (see pmGetOptions(3)) override the cor‐
96 responding built-in default values (if any). Configuration file
97 options override the corresponding environment variables (if any).
98 Command line options override the corresponding configuration file
99 options (if any).
100
102 The available command line options are:
103 -0 precision, --precision-force=precision
105 Like -P but this option will override per-metric specifications.
106 -1, --dynamic-header
108 Print a new dynamically adjusted header every time changes in
109 availability of metric and instance values occur. By default a
110 static header that never changes is printed once. See also -E and
111 -4.
112 -2, --overall-rank
114 Perform overall ranking of instances in archive. By default rank‐
115 ing (see -J) and reporting happens on each interval. With this
116 option all instances and values are ranked before a summary is
117 reported. See pmlogsummary(1) for further archive summary report‐
118 ing alternatives, including averages and peak times for values.
119 -3, --overall-rank-alt
121 Like -2 but print results in pmrep metricspec format.
122 -4 action, --names-change=action
124 Specify which action to take on receiving a metric names change
125 event during sampling. These events occur when a PMDA discovers
126 new metrics sometime after starting up, and informs running client
127 tools like pmrep. Valid values for action are update (refresh
128 metrics being sampled), ignore (do nothing - the default behav‐
129 iour) and abort (exit the program if such an event happens).
130 update implies --dynamic-header.
131 -5, --ignore-unknown
133 Silently ignore any metric name that cannot be resolved. At least
134 one metric must be found for the tool to start.
135 -8 limit, --limit-filter=limit
137 Limit results to instances with values above/below limit. A posi‐
138 tive integer will include instances with values at or above the
139 limit in reporting. A negative integer will include instances
140 with values at or below the limit in reporting. A value of zero
141 performs no limit filtering. This option will not override possi‐
142 ble per-metric specifications. See also -J and -N.
143 -9 limit, --limit-filter-force=limit
145 Like -8 but this option will override per-metric specifications.
146 -a archive, --archive=archive
148 Performance metric values are retrieved from the set of Perfor‐
149 mance Co-Pilot (PCP) archive log files identified by the argument
150 archive, which is a comma-separated list of names, each of which
151 may be the base name of an archive or the name of a directory con‐
152 taining one or more archives. See also -u.
153 --archive-folio=folio
155 Read metric source archives from the PCP archive folio created by
156 tools like pmchart(1) or, less often, manually with mkaf(1).
157 -A align, --align=align
159 Force the initial sample to be aligned on the boundary of a natu‐
160 ral time unit align. Refer to PCPIntro(1) for a complete descrip‐
161 tion of the syntax for align.
162 -b scale, --space-scale=scale
164 Unit/scale for space (byte) metrics, possible values include
165 bytes, Kbytes, KB, Mbytes, MB, and so forth. This option will not
166 override possible per-metric specifications. See also pmParseU‐
167 nitsStr(3).
168 -B scale, --space-scale-force=scale
170 Like -b but this option will override per-metric specifications.
171 -c config, --config=config
173 Specify the config file to use. The default is the first found
174 of: ./pmrep.conf, $HOME/.pmrep.conf, $HOME/pcp/pmrep.conf, and
175 $PCP_SYSCONF_DIR/pmrep/pmrep.conf. See pmrep.conf(5).
176 --container=container
178 Fetch performance metrics from the specified container, either
179 local or remote (see -h).
180 -C, --check
182 Exit before reporting any values, but after parsing the configura‐
183 tion and metrics and printing possible headers.
184 -d, --delay
186 When replaying from an archive, this option requests that the pre‐
187 vailing real-time delay be applied between samples (see -t) to
188 effect a pause, rather than the default behaviour of replaying at
189 full speed.
190 --daemonize
192 Daemonize on startup.
193 -e derived, --derived=derived
195 Specify derived performance metrics. If derived starts with a
196 slash (``/'') or with a dot (``.'') it will be interpreted as a
197 derived metrics configuration file, otherwise it will be inter‐
198 preted as comma- or semicolon-separated derived metric expres‐
199 sions. For details see pmLoadDerivedConfig(3) and pmRegister‐
200 Derived(3).
201 -E lines, --repeat-header=lines
203 Repeat the header every lines of output. See also -1.
204 -f format, --timestamp-format=format
206 Use the format string for formatting the timestamp. The format
207 will be used with Python's datetime.strftime method which is
208 mostly the same as that described in strftime(3). An empty format
209 string (i.e., "") will remove the timestamps from the output.
210 Defaults to %H:%M:%S when using the stdout output target.
211 Defaults to %Y-%m-%d %H:%M:%S when using the csv output target.
212 -F outfile, --output-file=outfile
214 Specify the output file outfile. See -o.
215 -g, --separate-header
217 Output the column number and complete metric information, one-per-
218 line, before printing the metric values.
219 -G, --no-globals
221 Do not include global metrics in reporting (see pmrep.conf(5)).
222 -h host, --host=host
224 Fetch performance metrics from pmcd(1) on host, rather than from
225 the default localhost.
226 -H, --no-header
228 Do not print any headers.
229 -i instances, --instances=instances
231 Report only the listed instances from current instances (if
232 present, see also -j). By default all current instances are
233 reported, except when writing an archive (see -o) when all
234 instances, present and future, are reported. This is a global
235 option that is used for all metrics unless a metric-specific
236 instance definition is provided as part of a metricspec. By
237 default single-valued ``flat'' metrics without multiple instances
238 are still reported as usual, use -v to change this.
239 The list may consist of one or more comma-separated instances.
241 The instance name may be quoted with single (') or double (")
242 quotes for those cases where the instance name contains commas or
243 whitespace. Note that on the command line when specifying more
244 than one instance, all the names must be quoted.
245 Multiple -i options are allowed as an alternative way of specify‐
247 ing more than one instance of interest. Regular expressions can
248 also be used.
249 As an example, the following would report the same instances:
251 $ pmrep -i "'1 minute','5 minute'" kernel.all.load
253 $ pmrep -i '"1 minute","5 minute"' kernel.all.load
254 $ pmrep -i "'1 minute'" -i "'5 minute'" kernel.all.load
255 $ pmrep kernel.all.load,,"'1 minute','5 minute'"
256 $ pmrep kernel.all.load,,'"1 minute","5 minute"'
257 However, this would report only the 1-minute instance:
260 $ pmrep -i '"1 minute","5 minute"' kernel.all.load,,"1 minute"
262 But this would report all instances (due to per-metric override):
265 $ pmrep -i '"1 minute","5 minute"' 'kernel.all.load,,.*'
267 -I, --ignore-incompat
270 Ignore incompatible metrics. By default incompatible metrics
271 (that is, their type is unsupported or they cannot be scaled as
272 requested) will cause pmrep to terminate with an error message.
273 With this option all incompatible metrics are silently omitted
274 from reporting. This may be especially useful when requesting
275 non-leaf nodes of the PMNS tree for reporting.
276 -j, --live-filter
278 Perform instance live filtering. This allows capturing all fil‐
279 tered instances even if processes are restarted at some point
280 (unlike without live filtering). Doing live filtering over a huge
281 amount of instances naturally comes with some overhead so a bit of
282 user caution is advised. See also -1.
283 -J rank, --rank=rank
285 Limit results to highest/lowest rank instances of set-valued met‐
286 rics. A positive integer will include highest valued instances in
287 reporting. A negative integer will include lowest valued
288 instances in reporting. A value of zero performs no ranking. See
289 also -2 and -8.
290 -k, --extended-csv
292 Write extended CSV output, similar to sadf(1).
293 -K spec, --spec-local=spec
295 When fetching metrics from a local context (see -L), the -K option
296 may be used to control the DSO PMDAs that should be made accessi‐
297 ble. The spec argument conforms to the syntax described in
298 pmSpecLocalPMDA(3). More than one -K option may be used.
299 -l delimiter, --delimiter=delimiter
301 Specify the delimiter that separates each column of csv or stdout
302 output. The default for stdout is two spaces (`` '') and comma
303 (``,'') for csv. In case of CSV output or stdout output with non-
304 whitespace delimiter, any instances of the delimiter in string
305 values will be replaced by the underscore (``_'') character.
306 -L, --local-PMDA
308 Use a local context to collect metrics from DSO PMDAs on the local
309 host without PMCD. See also -K.
310 -n, --invert-filter
312 Perform ranking before live filtering. By default instance live
313 filter filtering (when requested, see -j) happens before instance
314 ranking (when requested, see -J). With this option the logic is
315 inverted and ranking happens before live filtering.
316 -N predicate, --predicate=predicate
318 Specify a comma-separated list of predicate filter reference met‐
319 rics. By default ranking (see -J) happens for each metric indi‐
320 vidually. With predicate filter reference metrics, ranking is
321 done only for the specified metrics. When reporting, the rest of
322 the metrics sharing the same instance domain (see PCPIntro(1)) as
323 the predicates will include only the highest/lowest ranking
324 instances of the corresponding predicates.
325 So for example, when the using proc.memory.rss (resident size of
327 process) as the predicate and including proc.io.total_bytes and
328 mem.util.used as metrics to be reported, only the processes using
329 most/least memory (as per -J) will be included when reporting
330 total bytes written by processes. Since mem.util.used is a sin‐
331 gle-valued metric (thus not sharing the same instance domain as
332 the process-related metrics), it will be reported as usual.
333 -o output, --output=output
335 Use output target for reporting. The default target is stdout.
336 The available target alternatives are:
337 archive
339 Record metrics into a PCP archive which can later be replayed
340 with PCP tools, including pmrep itself. See LOGARCHIVE(5) and
341 PCPIntro(1) for details about PCP archive files. Requires -F.
342 csv
344 Print metrics in CSV format (subject to formatting options).
345 stdout
347 Print metrics to stdout (format subject to formatting options).
348 -O origin, --origin=origin
350 When reporting archived metrics, start reporting at origin within
351 the time window (see -S and -T). Refer to PCPIntro(1) for a com‐
352 plete description of the syntax for origin.
353 -p, --timestamps
355 Print timestamps. By default no timestamps are printed.
356 -P precision, --precision=precision
358 Use precision for numeric non-integer output values. If the value
359 is too wide for its column width, precision is reduced one by one
360 until the value fits, or not printed at all if it does not. The
361 default is to use 3 decimal places (when applicable). This option
362 will not override possible per-metric specifications.
363 -q scale, --count-scale=scale
365 Unit/scale for count metrics, possible values include count x
366 10^-1, count, count x 10, count x 10^2, and so forth from 10^-8 to
367 10^7. (These values are currently space-sensitive.) This option
368 will not override possible per-metric specifications. See also
369 pmParseUnitsStr(3).
370 -Q scale, --count-scale-force=scale
372 Like -q but this option will override per-metric specifications.
373 -r, --raw
375 Output raw metric values, do not convert cumulative counters to
376 rates. When writing archives, raw values are always used. This
377 option will override possible per-metric specifications.
378 -R, --raw-prefer
380 Like -r but this option will not override per-metric specifica‐
381 tions.
382 -s samples, --samples=samples
384 The argument samples defines the number of samples to be retrieved
385 and reported. If samples is 0 or -s is not specified, pmrep will
386 sample and report continuously (in real time mode) or until the
387 end of the set of PCP archives (in archive mode). See also -T.
388 -S starttime, --start=starttime
390 When reporting archived metrics, the report will be restricted to
391 those records logged at or after starttime. Refer to PCPIntro(1)
392 for a complete description of the syntax for starttime.
393 -t interval, --interval=interval
395 The default update interval may be set to something other than the
396 default 1 second. The interval argument follows the syntax
397 described in PCPIntro(1), and in the simplest form may be an
398 unsigned integer (the implied units in this case are seconds).
399 See also the -T and -u options.
400 -T endtime, --finish=endtime
402 When reporting archived metrics, the report will be restricted to
403 those records logged before or at endtime. Refer to PCPIntro(1)
404 for a complete description of the syntax for endtime.
405 When used to define the runtime before pmrep will exit, if no sam‐
407 ples is given (see -s) then the number of reported samples depends
408 on interval (see -t). If samples is given then interval will be
409 adjusted to allow reporting of samples during runtime. In case
410 all of -T, -s, and -t are given, endtime determines the actual
411 time pmrep will run.
412 -u, --no-interpol
414 When reporting archived metrics, by default values are reported
415 according to the selected sample interval (-t option), not accord‐
416 ing to the actual record interval in an archive. To this effect
417 PCP interpolates the values to be reported based on the records in
418 the archive. With the -u option uninterpolated reporting is
419 enabled, every recorded value for the selected metrics is reported
420 and the requested sample interval (-t) is ignored.
421 So for example, if a PCP archive contains recorded values for
423 every 10 seconds and the requested sample interval is 1 hour, by
424 default pmrep will use an interpolation scheme to compute the val‐
425 ues of the requested metrics from the values recorded in the prox‐
426 imity of these requested metrics and values for every 1 hour are
427 reported. With -u every record every 10 seconds are reported as
428 such (the reported values are still subject to rate conversion,
429 use -r or -R to disable).
430 -U, --no-unit-info
432 Omit unit information from headers.
433 -v, --omit-flat
435 Omit single-valued ``flat'' metrics from reporting, only consider
436 set-valued metrics (i.e., metrics with multiple values) for
437 reporting. See -i and -I.
438 -V, --version
440 Display version number and exit.
441 -w width, --width=width
443 Set the stdout output column width. Strings will be truncated to
444 this width. The default width is the shortest that can fit the
445 metric text label, the forced minimum is 3. This option will not
446 override possible per-metric specifications.
447 -W width, --width-force=width
449 Like -w but this option will override per-metric specifications.
450 -x, --extended-header
452 Print extended header.
453 -X label, --colxrow=label
455 Swap columns and rows in stdout output, reporting one instance per
456 line, using label as the text label for instance column (set to an
457 empty string "" to enable swapping without a specific text label).
458 This is convenient to allow easily using grep(1) to filter results
459 or to more closely mimic other utilities. See also -i.
460 -y scale, --time-scale=scale
462 Unit/scale for time metrics, possible values include nanosec, ns,
463 microsec, us, millisec, ms, and so forth up to hour, hr. This
464 option will not override possible per-metric specifications. See
465 also pmParseUnitsStr(3).
466 -Y scale, --time-scale-force=scale
468 Like -y but this option will override per-metric specifications.
469 -z, --hostzone
471 Use the local timezone of the host that is the source of the per‐
472 formance metrics, as identified by either the -h or the -a
473 options. The default is to use the timezone of the local host.
474 -Z timezone, --timezone=timezone
476 Use timezone for the date and time. Timezone is in the format of
477 the environment variable TZ as described in environ(7). Note that
478 when including a timezone string in output, ISO 8601 -style UTC
479 offsets are used (so something like -Z EST+5 will become UTC-5).
480 -?, --help
482 Display usage message and exit.
483
485 The following examples use the standard PCP facilities for collecting
486 the metric values, no external utilities are needed. The referenced
487 colon-starting metricsets are part of the system pmrep.conf file.
488 Display network interface metrics on the local host:
490 $ pmrep network.interface.total.bytes
491 Display all outgoing network metrics for the wlan0 interface:
493 $ pmrep -i wlan0 -v network.interface.out
494 Display timestamped vmstat(8) like information using megabytes instead
496 of kilobytes and also include the number of inodes used (tab completes
497 available metrics and after a colon metricsets with bash and zsh):
498 $ pmrep -p -B MB :vmstat vfs.inodes.count
499 Display per-device disk reads and writes from the host server1 using
501 two seconds interval and sadf(1) like CSV output format:
502 $ pmrep -h server1 -t 2s -o csv -k disk.dev.read disk.dev.write
503 Display processes using at least 100MB of memory using dynamic headers:
505 $ pmrep -b MB --limit-filter 100 --dynamic-header proc.memory.rss
506 Display the predefined set of metrics from the default pmrep.conf(5)
508 containing information about I/O issued by current firefox process(es):
509 $ pmrep -i '.*firefox.*' :proc-io
510 Display the three most CPU-using processes:
512 $ pmrep -1gUJ 3 proc.hog.cpu
513 Display sar -w and sar -W like information at the same time from the
515 PCP archive ./20150921.09.13 showing values recorded between 3 - 5 PM:
516 $ pmrep -a ./20150921.09.13 -S @15:00 -T @17:00 :sar-w :sar-W
517 Record most relevant CPU, memory, and I/O related information about
519 every Java process on the system, present and future, to an archive ./a
520 on one minute interval at every full minute in a background process:
521 $ pmrep --daemonize -A 1m -t 1m -i '.*java.*' -j -o archive -F ./a \
522 :proc-info :proc-cpu :proc-mem :proc-io
523 Record all 389 Directory Server, XFS file system, and CPU/memory/disk
525 metrics every five seconds for five minutes to a PCP archive ./a:
526 $ pmrep -t 5s -T 5m -o archive -F ./a ds389 xfs kernel.all.cpu mem disk
527 Record process memory and I/O information for those processes which are
529 the three most memory-consuming processes:
530 $ pmrep -o archive -F ./a -J 3 -N proc.memory.rss proc.memory proc.io
531
533 pmrep.conf
534 pmrep configuration file (see -c)
535 $PCP_SYSCONF_DIR/pmrep/pmrep.conf
537 system provided pmrep configuration file
538
540 Environment variables with the prefix PCP_ are used to parameterize the
541 file and directory names used by PCP. On each installation, the file
542 /etc/pcp.conf contains the local values for these variables. The
543 $PCP_CONF variable may be used to specify an alternative configuration
544 file, as described in pcp.conf(5).
545 For environment variables affecting PCP tools, see pmGetOptions(3).
547
549 mkaf(1), PCPIntro(1), pcp(1), pcp-atop(1), pcp2elasticsearch(1),
550 pcp2graphite(1), pcp2influxdb(1), pcp2json(1), pcp2spark(1),
551 pcp2xlsx(1), pcp2xml(1), pcp2zabbix(1), pmcd(1), pmchart(1), pmcol‐
552 lectl(1), pmdiff(1), pmdumplog(1), pmdumptext(1), pminfo(1), pmio‐
553 stat(1), pmlogextract(1), pmlogsummary(1), pmprobe(1), pmstat(1),
554 pmval(1), sadf(1), sar(1), pmGetOptions(3), pmSpecLocalPMDA(3), pmLoad‐
555 DerivedConfig(3), pmParseUnitsStr(3), pmRegisterDerived(3), strf‐
556 time(3), LOGARCHIVE(5), pcp.conf(5), pmns(5), pmrep.conf(5), environ(7)
557 and vmstat(8).
558Performance Co-Pilot PCP PMREP(1)