1PCP-ATOP(1) General Commands Manual PCP-ATOP(1)
2
6 pcp-atop - Advanced System and Process Monitor
7
9 Interactive Usage:
10 pcp [pcp options] atop [-g|-m|-d|-n|-u|-p|-s|-c|-v|-o|-y]
12 [-C|-M|-D|-N|-A] [-afFG1xR] [-L linelen] [-Plabel[,label]...] [interval
13 [samples]]
14 Writing and reading raw logfiles:
16 pcp atop -w rawfile [-a] [-S] [interval [samples]]
18 pcp atop -r [ rawfile ] [-b hh:mm ] [-e hh:mm ]
19 [-g|-m|-d|-n|-u|-p|-s|-c|-v|-o|-y] [-C|-M|-D|-N|-A] [-fFG1xR] [-L line‐
20 len] [-Plabel[,label]...]
21
23 The program pcp-atop is an interactive monitor to view various aspects
24 of load on a system. It shows the occupation of the most critical
25 hardware resources (from a performance point of view) on system level,
26 i.e. cpu, memory, disk and network.
27 It also shows which processes are responsible for the indicated load
28 with respect to cpu and memory load on process level. Disk load is
29 shown per process if "storage accounting" is active in the kernel.
30 Every interval (default: 10 seconds) information is shown about the
32 resource occupation on system level (cpu, memory, disks and network
33 layers), followed by a list of processes which have been active during
34 the last interval (note that all processes that were unchanged during
35 the last interval are not shown, unless the key 'a' has been pressed or
36 unless sorting on memory occupation is done). If the list of active
37 processes does not entirely fit on the screen, only the top of the list
38 is shown (sorted in order of activity).
39 The intervals are repeated till the number of samples (specified as
40 command argument) is reached, or till the key 'q' is pressed in inter‐
41 active mode.
42 When invoked via the pcp(1) command, the PCPIntro(1) options -h/--host,
44 -a/--archive, -O/--origin, -s/--samples, -t/--interval, -Z/--timezone
45 and several other pcp options become indirectly available. The long
46 option form of these is directly available. Additionally, the --hot‐
47 proc option can be used to request the per-process PCP metrics be used
48 instead of the default proc metrics from pmdaproc(1).
49 When pcp-atop is started, it checks whether the standard output channel
51 is connected to a screen, or to a file/pipe. In the first case it pro‐
52 duces screen control codes (via the ncurses library) and behaves inter‐
53 actively; in the second case it produces flat ASCII-output.
54 In interactive mode, the output of pcp-atop scales dynamically to the
56 current dimensions of the screen/window.
57 If the window is resized horizontally, columns will be added or removed
58 automatically. For this purpose, every column has a particular weight.
59 The columns with the highest weights that fit within the current width
60 will be shown.
61 If the window is resized vertically, lines of the process/thread list
62 will be added or removed automatically.
63 Furthermore in interactive mode the output of pcp-atop can be con‐
65 trolled by pressing particular keys. However it is also possible to
66 specify such key as flag on the command line. In that case pcp-atop
67 switches to the indicated mode on beforehand; this mode can be modified
68 again interactively. Specifying such key as flag is especially useful
69 when running pcp-atop with output to a pipe or file (non-interac‐
70 tively). These flags are the same as the keys that can be pressed in
71 interactive mode (see section INTERACTIVE COMMANDS).
72 Additional flags are available to support storage of pcp-atop data in
73 PCP archive format (see section PCP DATA STORAGE).
74
76 For the resource consumption on system level, pcp-atop uses colors to
77 indicate that a critical occupation percentage has been (almost)
78 reached. A critical occupation percentage means that is likely that
79 this load causes a noticeable negative performance influence for appli‐
80 cations using this resource. The critical percentage depends on the
81 type of resource: e.g. the performance influence of a disk with a busy
82 percentage of 80% might be more noticeable for applications/user than a
83 CPU with a busy percentage of 90%.
84 Currently pcp-atop uses the following default values to calculate a
85 weighted percentage per resource:
86 Processor
88 A busy percentage of 90% or higher is considered `critical'.
89 Disk
91 A busy percentage of 70% or higher is considered `critical'.
92 Network
94 A busy percentage of 90% or higher for the load of an interface is
95 considered `critical'.
96 Memory
98 An occupation percentage of 90% is considered `critical'. Notice
99 that this occupation percentage is the accumulated memory consump‐
100 tion of the kernel (including slab) and all processes; the memory
101 for the page cache (`cache' and `buff' in the MEM-line) and the
102 reclaimable part of the slab (`slrec`) is not implied!
103 If the number of pages swapped out (`swout' in the PAG-line) is
104 larger than 10 per second, the memory resource is considered
105 `critical'. A value of at least 1 per second is considered
106 `almost critical'.
107 If the committed virtual memory exceeds the limit (`vmcom' and
108 `vmlim' in the SWP-line), the SWP-line is colored due to overcom‐
109 mitting the system.
110 Swap
112 An occupation percentage of 80% is considered `critical' because
113 swap space might be completely exhausted in the near future; it is
114 not critical from a performance point-of-view.
115 These default values can be modified in the configuration file (see
117 separate man-page of pcp-atoprc).
118 When a resource exceeds its critical occupation percentage, the con‐
120 cerning values in the screen line are colored red by default.
121 When a resource exceeded (default) 80% of its critical percentage (so
122 it is almost critical), the concerning values in the screen line are
123 colored cyan by default. This `almost critical percentage' (one value
124 for all resources) can be modified in the configuration file (see sepa‐
125 rate man-page of pcp-atoprc).
126 The default colors red and cyan can be modified in the configuration
127 file as well (see separate man-page of pcp-atoprc).
128 With the key 'x' (or flag -x), the use of colors can be suppressed.
130
132 GPU statistics can be gathered by pmdanvidia(1) which is a separate
133 data collection daemon process. It gathers cumulative utilization
134 counters of every Nvidia GPU in the system, as well as utilization
135 counters of every process that uses a GPU. When atop notices that the
136 daemon is active, it reads these GPU utilization counters with every
137 interval.
138 Find a description about the utilization counters in the section OUTPUT
140 DESCRIPTION.
141
143 When running pcp-atop interactively (no output redirection), keys can
144 be pressed to control the output. In general, lower case keys can be
145 used to show other information for the active processes and upper case
146 keys can be used to influence the sort order of the active
147 process/thread list.
148 g Show generic output (default).
150 Per process the following fields are shown in case of a window-
152 width of 80 positions: process-id, cpu consumption during the last
153 interval in system and user mode, the virtual and resident memory
154 growth of the process.
155 The subsequent columns depend on the used kernel:
157 When the kernel supports "storage accounting" (>= 2.6.20), the
158 data transfer for read/write on disk, the status and exit code are
159 shown for each process. When the kernel does not support "storage
160 accounting", the username, number of threads in the thread group,
161 the status and exit code are shown.
162 The last columns contain the state, the occupation percentage for
163 the chosen resource (default: cpu) and the process name.
164 When more than 80 positions are available, other information is
166 added.
167 m Show memory related output.
169 Per process the following fields are shown in case of a window-
171 width of 80 positions: process-id, minor and major memory faults,
172 size of virtual shared text, total virtual process size, total
173 resident process size, virtual and resident growth during last
174 interval, memory occupation percentage and process name.
175 When more than 80 positions are available, other information is
177 added.
178 For memory consumption, always all processes are shown (also the
180 processes that were not active during the interval).
181 d Show disk-related output.
183 When "storage accounting" is active in the kernel, the following
185 fields are shown: process-id, amount of data read from disk,
186 amount of data written to disk, amount of data that was written
187 but has been withdrawn again (WCANCL), disk occupation percentage
188 and process name.
189 s Show scheduling characteristics.
191 Per process the following fields are shown in case of a window-
193 width of 80 positions: process-id, number of threads in state
194 'running' (R), number of threads in state 'interruptible sleeping'
195 (S), number of threads in state 'uninterruptible sleeping' (D),
196 scheduling policy (normal timesharing, realtime round-robin, real‐
197 time fifo), nice value, priority, realtime priority, current pro‐
198 cessor, status, exit code, state, the occupation percentage for
199 the chosen resource and the process name.
200 When more than 80 positions are available, other information is
202 added.
203 v Show various process characteristics.
205 Per process the following fields are shown in case of a window-
207 width of 80 positions: process-id, user name and group, start date
208 and time, status (e.g. exit code if the process has finished),
209 state, the occupation percentage for the chosen resource and the
210 process name.
211 When more than 80 positions are available, other information is
213 added.
214 c Show the command line of the process.
216 Per process the following fields are shown: process-id, the occu‐
218 pation percentage for the chosen resource and the command line
219 including arguments.
220 e Show GPU utilization.
222 Per process at least the following fields are shown: process-id,
224 range of GPU numbers on which the process currently runs, GPU busy
225 percentage on all GPUs, memory busy percentage (i.e. read and
226 write accesses on memory) on all GPUs, memory occupation at the
227 moment of the sample, average memory occupation during the sample,
228 and GPU percentage.
229 When the pmdanvidia daemon does not run with root privileges, the
231 GPU busy percentage and the memory busy percentage are not avail‐
232 able on process level. In that case, the GPU percentage on
233 process level reflects the GPU memory occupation instead of the
234 GPU busy percentage (which is preferred).
235 o Show the user-defined line of the process.
237 In the configuration file the keyword ownprocline can be specified
239 with the description of a user-defined output-line.
240 Refer to the man-page of pcp-atoprc for a detailed description.
241 y Show the individual threads within a process (toggle).
243 Single-threaded processes are still shown as one line.
245 For multi-threaded processes, one line represents the process
246 while additional lines show the activity per individual thread (in
247 a different color). Depending on the option 'a' (all or active
248 toggle), all threads are shown or only the threads that were
249 active during the last interval.
250 Whether this key is active or not can be seen in the header line.
251 u Show the process activity accumulated per user.
253 Per user the following fields are shown: number of processes
255 active or terminated during last interval (or in total if combined
256 with command `a'), accumulated cpu consumption during last inter‐
257 val in system and user mode, the current virtual and resident mem‐
258 ory space consumed by active processes (or all processes of the
259 user if combined with command `a').
260 When "storage accounting" is active in the kernel, the accumulated
261 read and write throughput on disk is shown. When the kernel mod‐
262 ule `netatop' has been installed, the number of received and sent
263 network packets are shown.
264 The last columns contain the accumulated occupation percentage for
265 the chosen resource (default: cpu) and the user name.
266 p Show the process activity accumulated per program (i.e. process
268 name).
269 Per program the following fields are shown: number of processes
271 active or terminated during last interval (or in total if combined
272 with command `a'), accumulated cpu consumption during last inter‐
273 val in system and user mode, the current virtual and resident mem‐
274 ory space consumed by active processes (or all processes of the
275 user if combined with command `a').
276 When "storage accounting" is active in the kernel, the accumulated
277 read and write throughput on disk is shown. When the kernel mod‐
278 ule `netatop' has been installed, the number of received and sent
279 network packets are shown.
280 The last columns contain the accumulated occupation percentage for
281 the chosen resource (default: cpu) and the program name.
282 j Show the process activity accumulated per Docker container.
284 Per container the following fields are shown: number of processes
286 active or terminated during last interval (or in total if combined
287 with command `a'), accumulated cpu consumption during last inter‐
288 val in system and user mode, the current virtual and resident mem‐
289 ory space consumed by active processes (or all processes of the
290 user if combined with command `a').
291 When "storage accounting" is active in the kernel, the accumulated
292 read and write throughput on disk is shown. When the kernel mod‐
293 ule `netatop' has been installed, the number of received and sent
294 network packets are shown.
295 The last columns contain the accumulated occupation percentage for
296 the chosen resource (default: cpu) and the Docker container id
297 (CID).
298 C Sort the current list in the order of cpu consumption (default).
300 The one-but-last column changes to ``CPU''.
301 E Sort the current list in the order of GPU utilization (preferred,
303 but only applicable when the pmdanvidia daemon runs under root
304 privileges) or the order of GPU memory occupation). The one-but-
305 last column changes to ``GPU''.
306 M Sort the current list in the order of resident memory consumption.
308 The one-but-last column changes to ``MEM''. In case of sorting on
309 memory, the full process list will be shown (not only the active
310 processes).
311 D Sort the current list in the order of disk accesses issued. The
313 one-but-last column changes to ``DSK''.
314 N Sort the current list in the order of network bandwidth (received
316 and transmitted). The one-but-last column changes to ``NET''.
317 A Sort the current list automatically in the order of the most busy
319 system resource during this interval. The one-but-last column
320 shows either ``ACPU'', ``AMEM'', ``ADSK'' or ``ANET'' (the preced‐
321 ing 'A' indicates automatic sorting-order). The most busy
322 resource is determined by comparing the weighted busy-percentages
323 of the system resources, as described earlier in the section COL‐
324 ORS.
325 This option remains valid until another sorting-order is explic‐
326 itly selected again.
327 A sorting-order for disk is only possible when "storage account‐
328 ing" is active. A sorting-order for network is only possible when
329 the kernel module `netatop' is loaded.
330 Miscellaneous interactive commands:
332 ? Request for help information (also the key 'h' can be pressed).
334 V Request for version information (version number and date).
336 R Gather and calculate the proportional set size of processes (tog‐
338 gle). Gathering of all values that are needed to calculate the
339 PSIZE of a process is a relatively time-consuming task, so this
340 key should only be active when analyzing the resident memory con‐
341 sumption of processes.
342 x Suppress colors to highlight critical resources (toggle).
344 Whether this key is active or not can be seen in the header line.
345 z The pause key can be used to freeze the current situation in order
347 to investigate the output on the screen. While pcp-atop is paused,
348 the keys described above can be pressed to show other information
349 about the current list of processes. Whenever the pause key is
350 pressed again, pcp-atop will continue with a next sample.
351 i Modify the interval timer (default: 10 seconds). If an interval
353 timer of 0 is entered, the interval timer is switched off. In that
354 case a new sample can only be triggered manually by pressing the
355 key 't'.
356 t Trigger a new sample manually. This key can be pressed if the cur‐
358 rent sample should be finished before the timer has exceeded, or
359 if no timer is set at all (interval timer defined as 0). In the
360 latter case pcp-atop can be used as a stopwatch to measure the
361 load being caused by a particular application transaction, without
362 knowing on beforehand how many seconds this transaction will last.
363 When viewing the contents of a raw file, this key can be used to
365 show the next sample from the file.
366 T When viewing the contents of a raw file, this key can be used to
368 show the previous sample from the file.
369 b When viewing the contents of a raw file, this key can be used to
371 branch to a certain timestamp within the file (either forward or
372 backward).
373 r Reset all counters to zero to see the system and process activity
375 since boot again.
376 When viewing the contents of a raw file, this key can be used to
378 rewind to the beginning of the file again.
379 U Specify a search string for specific user names as a regular
381 expression. From now on, only (active) processes will be shown
382 from a user which matches the regular expression. The system sta‐
383 tistics are still system wide. If the Enter-key is pressed with‐
384 out specifying a name, (active) processes of all users will be
385 shown again.
386 Whether this key is active or not can be seen in the header line.
387 I Specify a list with one or more PIDs to be selected. From now on,
389 only processes will be shown with a PID which matches one of the
390 given list. The system statistics are still system wide. If the
391 Enter-key is pressed without specifying a PID, all (active) pro‐
392 cesses will be shown again.
393 Whether this key is active or not can be seen in the header line.
394 P Specify a search string for specific process names as a regular
396 expression. From now on, only processes will be shown with a name
397 which matches the regular expression. The system statistics are
398 still system wide. If the Enter-key is pressed without specifying
399 a name, all (active) processes will be shown again.
400 Whether this key is active or not can be seen in the header line.
401 / Specify a specific command line search string as a regular expres‐
403 sion. From now on, only processes will be shown with a command
404 line which matches the regular expression. The system statistics
405 are still system wide. If the Enter-key is pressed without speci‐
406 fying a string, all (active) processes will be shown again.
407 Whether this key is active or not can be seen in the header line.
408 J Specify a Docker container id of 12 (hexadecimal) characters.
410 From now on, only processes will be shown that run in that spe‐
411 cific Docker container (CID). The system statistics are still
412 system wide. If the Enter-key is pressed without specifying a
413 container id, all (active) processes will be shown again.
414 Whether this key is active or not can be seen in the header line.
415 S Specify search strings for specific logical volume names, specific
417 disk names and specific network interface names. All search
418 strings are interpreted as a regular expressions. From now on,
419 only those system resources are shown that match the concerning
420 regular expression. If the Enter-key is pressed without specify‐
421 ing a search string, all (active) system resources of that type
422 will be shown again.
423 Whether this key is active or not can be seen in the header line.
424 a The `all/active' key can be used to toggle between only show‐
426 ing/accumulating the processes that were active during the last
427 interval (default) or showing/accumulating all processes.
428 Whether this key is active or not can be seen in the header line.
429 G By default, pcp-atop shows/accumulates the processes that are
431 alive and the processes that are exited during the last interval.
432 With this key (toggle), showing/accumulating the processes that
433 are exited can be suppressed.
434 Whether this key is active or not can be seen in the header line.
435 f Show a fixed (maximum) number of header lines for system resources
437 (toggle). By default only the lines are shown about system
438 resources (CPUs, paging, logical volumes, disks, network inter‐
439 faces) that really have been active during the last interval.
440 With this key you can force pcp-atop to show lines of inactive
441 resources as well.
442 Whether this key is active or not can be seen in the header line.
443 F Suppress sorting of system resources (toggle). By default system
445 resources (CPUs, logical volumes, disks, network interfaces) are
446 sorted on utilization.
447 Whether this key is active or not can be seen in the header line.
448 1 Show relevant counters as an average per second (in the format
450 `..../s') instead of as a total during the interval (toggle).
451 Whether this key is active or not can be seen in the header line.
452 l Limit the number of system level lines for the counters per-cpu,
454 the active disks and the network interfaces. By default lines are
455 shown of all CPUs, disks and network interfaces which have been
456 active during the last interval. Limiting these lines can be use‐
457 ful on systems with huge number CPUs, disks or interfaces in order
458 to be able to run pcp-atop on a screen/window with e.g. only 24
459 lines.
460 For all mentioned resources the maximum number of lines can be
461 specified interactively. When using the flag -l the maximum number
462 of per-cpu lines is set to 0, the maximum number of disk lines to
463 5 and the maximum number of interface lines to 3. These values
464 can be modified again in interactive mode.
465 k Send a signal to an active process (a.k.a. kill a process).
467 q Quit the program.
469 PgDn Show the next page of the process/thread list.
471 With the arrow-down key the list can be scrolled downwards with
472 single lines.
473 ^F Show the next page of the process/thread list (forward).
475 With the arrow-down key the list can be scrolled downwards with
476 single lines.
477 PgUp Show the previous page of the process/thread list.
479 With the arrow-up key the list can be scrolled upwards with single
480 lines.
481 ^B Show the previous page of the process/thread list (backward).
483 With the arrow-up key the list can be scrolled upwards with single
484 lines.
485 ^L Redraw the screen.
487
489 In order to store system and process level statistics for long-term
490 analysis (e.g. to check the system load and the active processes run‐
491 ning yesterday between 3:00 and 4:00 PM), pcp-atop can store the system
492 and process level statistics in the PCP archive format, as an archive
493 folio (see mkaf(1)).
494 All information about processes and threads is stored in the archive.
495 The interval (default: 10 seconds) and number of samples (default:
496 infinite) can be passed as last arguments. Instead of the number of
497 samples, the flag -S can be used to indicate that pcp-atop should fin‐
498 ish anyhow before midnight.
499 A PCP archive can be read and visualized again with the flag -r . The
501 argument is a comma-separated list of names, each of which may be the
502 base name of an archive or the name of a directory containing one or
503 more archives. If no argument is specified, the file
504 $PCP_LOG_DIR/pmlogger/HOST/YYYYMMDD is opened for input (where YYYYMMDD
505 are digits representing the current date, and HOST is the hostname of
506 the machine being logged). If a filename is specified in the format
507 YYYYMMDD (representing any valid date), the file $PCP_LOG_DIR/pmlog‐
508 ger/HOST/YYYYMMDD is opened. If a filename with the symbolic name y is
509 specified, yesterday's daily logfile is opened (this can be repeated so
510 'yyyy' indicates the logfile of four days ago).
511 The samples from the file can be viewed interactively by using the key
512 't' to show the next sample, the key 'T' to show the previous sample,
513 the key 'b' to branch to a particular time or the key 'r' to rewind to
514 the begin of the file.
515 When output is redirected to a file or pipe, pcp-atop prints all sam‐
516 ples in plain ASCII. The default line length is 80 characters in that
517 case; with the flag -L followed by an alternate line length, more (or
518 less) columns will be shown.
519 With the flag -b (begin time) and/or -e (end time) followed by a time
520 argument of the form HH:MM, a certain time period within the raw file
521 can be selected.
522
524 The first sample shows the system level activity since boot (the
525 elapsed time in the header shows the time since boot). Note that par‐
526 ticular counters could have reached their maximum value (several times)
527 and started by zero again, so do not rely on these figures.
528 For every sample pcp-atop first shows the lines related to system level
530 activity. If a particular system resource has not been used during the
531 interval, the entire line related to this resource is suppressed. So
532 the number of system level lines may vary for each sample.
533 After that a list is shown of processes which have been active during
534 the last interval. This list is by default sorted on cpu consumption,
535 but this order can be changed by the keys which are previously
536 described.
537 If values have to be shown by pcp-atop which do not fit in the column
539 width, another format is used. If e.g. a cpu-consumption of 233216 mil‐
540 liseconds should be shown in a column width of 4 positions, it is shown
541 as `233s' (in seconds). For large memory figures, another unit is cho‐
542 sen if the value does not fit (Mb instead of Kb, Gb instead of Mb, Tb
543 instead of Gb, ...). For other values, a kind of exponent notation is
544 used (value 123456789 shown in a column of 5 positions gives 123e6).
545
547 The system level information consists of the following output lines:
548 PRC Process and thread level totals.
550 This line contains the total cpu time consumed in system mode
551 (`sys') and in user mode (`user'), the total number of processes
552 present at this moment (`#proc'), the total number of threads
553 present at this moment in state `running' (`#trun'), `sleeping
554 interruptible' (`#tslpi') and `sleeping uninterruptible'
555 (`#tslpu'), the number of zombie processes (`#zombie'), the number
556 of clone system calls (`clones'), and the number of processes that
557 ended during the interval (`#exit') when process accounting is
558 used. Instead of `#exit` the last column may indicate that process
559 accounting could not be activated (`no procacct`).
560 If the screen-width does not allow all of these counters, only a
561 relevant subset is shown.
562 CPU CPU utilization.
564 At least one line is shown for the total occupation of all CPUs
565 together.
566 In case of a multi-processor system, an additional line is shown
567 for every individual processor (with `cpu' in lower case), sorted
568 on activity. Inactive CPUs will not be shown by default. The
569 lines showing the per-cpu occupation contain the cpu number in the
570 field combined with the wait percentage.
571 Every line contains the percentage of cpu time spent in kernel
573 mode by all active processes (`sys'), the percentage of cpu time
574 consumed in user mode (`user') for all active processes (including
575 processes running with a nice value larger than zero), the per‐
576 centage of cpu time spent for interrupt handling (`irq') including
577 softirq, the percentage of unused cpu time while no processes were
578 waiting for disk I/O (`idle'), and the percentage of unused cpu
579 time while at least one process was waiting for disk I/O (`wait').
580 In case of per-cpu occupation, the cpu number and the wait per‐
581 centage (`w') for that cpu. The number of lines showing the per-
582 cpu occupation can be limited.
583 For virtual machines, the steal-percentage (`steal') shows the
585 percentage of cpu time stolen by other virtual machines running on
586 the same hardware.
587 For physical machines hosting one or more virtual machines, the
588 guest-percentage (`guest') shows the percentage of cpu time used
589 by the virtual machines. Notice that this percentage overlaps the
590 user-percentage!
591 When PMC performance monitoring counters are supported by the CPU
593 and the kernel (and pcp-atop runs with root privileges), the num‐
594 ber of instructions per CPU cycle (`ipc') is shown. The first
595 sample always shows the value 'initial', because the counters are
596 just activated at the moment that pcp-atop is started.
597 When the CPU busy percentage is high and the IPC is less than 1.0,
598 it is likely that the CPU is frequently waiting for memory access
599 during instruction execution (larger CPU caches or faster memory
600 might be helpful to improve performance). When the CPU busy per‐
601 centage is high and the IPC is greater than 1.0, it is likely that
602 the CPU is instruction-bound (more/faster cores might be helpful
603 to improve performance).
604 Furthermore, per CPU the effective number of cycles (`cycl') is
605 shown. This value can reach the current CPU frequency if such CPU
606 is 100% busy. When an idle CPU is halted, the number of effective
607 cycles can be (considerably) lower than the current frequency.
608 Notice that the average instructions per cycle and number of
609 cycles is shown in the CPU line for all CPUs.
610 See also: http://www.brendangregg.com/blog/2017-05-09/cpu-utiliza‐
611 tion-is-wrong.html
612 In case of frequency scaling, all previously mentioned CPU per‐
615 centages are relative to the used scaling of the CPU during the
616 interval. If a CPU has been active for e.g. 50% in user mode dur‐
617 ing the interval while the frequency scaling of that CPU was 40%,
618 only 20% of the full capacity of the CPU has been used in user
619 mode.
620 If the screen-width does not allow all of these counters, only a
622 relevant subset is shown.
623 CPL CPU load information.
625 This line contains the load average figures reflecting the number
626 of threads that are available to run on a CPU (i.e. part of the
627 runqueue) or that are waiting for disk I/O. These figures are
628 averaged over 1 (`avg1'), 5 (`avg5') and 15 (`avg15') minutes.
629 Furthermore the number of context switches (`csw'), the number of
630 serviced interrupts (`intr') and the number of available CPUs are
631 shown.
632 If the screen-width does not allow all of these counters, only a
634 relevant subset is shown.
635 GPU GPU utilization (Nvidia).
637 Read the section GPU STATISTICS GATHERING in this document to find
638 the details about the activation of the pmdanvidia daemon.
639 In the first column of every line, the bus-id (last nine charac‐
641 ters) and the GPU number are shown. The subsequent columns show
642 the percentage of time that one or more kernels were executing on
643 the GPU (`gpubusy'), the percentage of time that global (device)
644 memory was being read or written (`membusy'), the occupation per‐
645 centage of memory (`memocc'), the total memory (`total'), the mem‐
646 ory being in use at the moment of the sample (`used'), the average
647 memory being in use during the sample time (`usavg'), the number
648 of processes being active on the GPU at the moment of the sample
649 (`#proc'), and the type of GPU.
650 If the screen-width does not allow all of these counters, only a
652 relevant subset is shown.
653 The number of lines showing the GPUs can be limited.
654 MEM Memory occupation.
656 This line contains the total amount of physical memory (`tot'),
657 the amount of memory which is currently free (`free'), the amount
658 of memory in use as page cache including the total resident shared
659 memory (`cache'), the amount of memory within the page cache that
660 has to be flushed to disk (`dirty'), the amount of memory used for
661 filesystem meta data (`buff'), the amount of memory being used for
662 kernel mallocs (`slab'), the amount of slab memory that is
663 reclaimable (`slrec'), the resident size of shared memory includ‐
664 ing tmpfs (`shmem`), the resident size of shared memory (`shrss`)
665 the amount of shared memory that is currently swapped (`shswp`),
666 the amount of memory that is currently claimed by vmware's balloon
667 driver (`vmbal`), the amount of memory that is claimed for huge
668 pages (`hptot`), and the amount of huge page memory that is really
669 in use (`hpuse`).
670 If the screen-width does not allow all of these counters, only a
672 relevant subset is shown.
673 SWP Swap occupation and overcommit info.
675 This line contains the total amount of swap space on disk (`tot')
676 and the amount of free swap space (`free').
677 Furthermore the committed virtual memory space (`vmcom') and the
678 maximum limit of the committed space (`vmlim', which is by default
679 swap size plus 50% of memory size) is shown. The committed space
680 is the reserved virtual space for all allocations of private mem‐
681 ory space for processes. The kernel only verifies whether the com‐
682 mitted space exceeds the limit if strict overcommit handling is
683 configured (vm.overcommit_memory is 2).
684 PAG Paging frequency.
686 This line contains the number of scanned pages (`scan') due to the
687 fact that free memory drops below a particular threshold and the
688 number times that the kernel tries to reclaim pages due to an
689 urgent need (`stall').
690 Also the number of memory pages the system read from swap space
691 (`swin') and the number of memory pages the system wrote to swap
692 space (`swout') are shown.
693 PSI Pressure Stall Information.
695 This line contains three percentages per category: average pres‐
696 sure percentage over the last 10, 60 and 300 seconds (separated by
697 slashes).
698 The categories are: CPU for 'some' (`cs'), memory for 'some'
699 (`ms'), memory for 'full' (`mf'), I/O for 'some' (`is'), and I/O
700 for 'full' (`if').
701 LVM/MDD/DSK
703 Logical volume/multiple device/disk utilization.
704 Per active unit one line is produced, sorted on unit activity.
705 Such line shows the name (e.g. VolGroup00-lvtmp for a logical vol‐
706 ume or sda for a hard disk), the busy percentage i.e. the portion
707 of time that the unit was busy handling requests (`busy'), the
708 number of read requests issued (`read'), the number of write
709 requests issued (`write'), the number of KiBytes per read
710 (`KiB/r'), the number of KiBytes per write (`KiB/w'), the number
711 of MiBytes per second throughput for reads (`MBr/s'), the number
712 of MiBytes per second throughput for writes (`MBw/s'), the average
713 queue depth (`avq') and the average number of milliseconds needed
714 by a request (`avio') for seek, latency and data transfer.
715 If the screen-width does not allow all of these counters, only a
716 relevant subset is shown.
717 The number of lines showing the units can be limited per class
719 (LVM, MDD or DSK) with the 'l' key or statically (see separate
720 man-page of pcp-atoprc(5)). By specifying the value 0 for a par‐
721 ticular class, no lines will be shown any more for that class.
722 NFM Network Filesystem (NFS) mount at the client side.
724 For each NFS-mounted filesystem, a line is shown that contains the
725 mounted server directory, the name of the server (`srv'), the
726 total number of bytes physically read from the server (`read') and
727 the total number of bytes physically written to the server
728 (`write'). Data transfer is subdivided in the number of bytes
729 read via normal read() system calls (`nread'), the number of bytes
730 written via normal read() system calls (`nwrit'), the number of
731 bytes read via direct I/O (`dread'), the number of bytes written
732 via direct I/O (`dwrit'), the number of bytes read via memory
733 mapped I/O pages (`mread'), and the number of bytes written via
734 memory mapped I/O pages (`mwrit').
735 NFC Network Filesystem (NFS) client side counters.
737 This line contains the number of RPC calls issues by local pro‐
738 cesses (`rpc'), the number of read RPC calls (`read`) and write
739 RPC calls (`rpwrite') issued to the NFS server, the number of RPC
740 calls being retransmitted (`retxmit') and the number of authoriza‐
741 tion refreshes (`autref').
742 NFS Network Filesystem (NFS) server side counters.
744 This line contains the number of RPC calls received from NFS
745 clients (`rpc'), the number of read RPC calls received (`cread`),
746 the number of write RPC calls received (`cwrit'), the number of
747 Megabytes/second returned to read requests by clients (`MBcr/s`),
748 the number of Megabytes/second passed in write requests by clients
749 (`MBcw/s`), the number of network requests handled via TCP
750 (`nettcp'), the number of network requests handled via UDP
751 (`netudp'), the number of reply cache hits (`rchits'), the number
752 of reply cache misses (`rcmiss') and the number of uncached
753 requests (`rcnoca'). Furthermore some error counters indicating
754 the number of requests with a bad format (`badfmt') or a bad
755 authorization (`badaut'), and a counter indicating the number of
756 bad clients (`badcln').
757 NET Network utilization (TCP/IP).
759 One line is shown for activity of the transport layer (TCP and
760 UDP), one line for the IP layer and one line per active interface.
761 For the transport layer, counters are shown concerning the number
762 of received TCP segments including those received in error
763 (`tcpi'), the number of transmitted TCP segments excluding those
764 containing only retransmitted octets (`tcpo'), the number of UDP
765 datagrams received (`udpi'), the number of UDP datagrams transmit‐
766 ted (`udpo'), the number of active TCP opens (`tcpao'), the number
767 of passive TCP opens (`tcppo'), the number of TCP output retrans‐
768 missions (`tcprs'), the number of TCP input errors (`tcpie'), the
769 number of TCP output resets (`tcpor'), the number of UDP no ports
770 (`udpnp'), and the number of UDP input errors (`udpie').
771 If the screen-width does not allow all of these counters, only a
772 relevant subset is shown.
773 These counters are related to IPv4 and IPv6 combined.
774 For the IP layer, counters are shown concerning the number of IP
776 datagrams received from interfaces, including those received in
777 error (`ipi'), the number of IP datagrams that local higher-layer
778 protocols offered for transmission (`ipo'), the number of received
779 IP datagrams which were forwarded to other interfaces (`ipfrw'),
780 the number of IP datagrams which were delivered to local higher-
781 layer protocols (`deliv'), the number of received ICMP datagrams
782 (`icmpi'), and the number of transmitted ICMP datagrams (`icmpo').
783 If the screen-width does not allow all of these counters, only a
784 relevant subset is shown.
785 These counters are related to IPv4 and IPv6 combined.
786 For every active network interface one line is shown, sorted on
788 the interface activity. Such line shows the name of the interface
789 and its busy percentage in the first column. The busy percentage
790 for half duplex is determined by comparing the interface speed
791 with the number of bits transmitted and received per second; for
792 full duplex the interface speed is compared with the highest of
793 either the transmitted or the received bits. When the interface
794 speed can not be determined (e.g. for the loopback interface),
795 `---' is shown instead of the percentage.
796 Furthermore the number of received packets (`pcki'), the number of
797 transmitted packets (`pcko'), the line speed of the interface
798 (`sp'), the effective amount of bits received per second (`si'),
799 the effective amount of bits transmitted per second (`so'), the
800 number of collisions (`coll'), the number of received multicast
801 packets (`mlti'), the number of errors while receiving a packet
802 (`erri'), the number of errors while transmitting a packet
803 (`erro'), the number of received packets dropped (`drpi'), and the
804 number of transmitted packets dropped (`drpo').
805 If the screen-width does not allow all of these counters, only a
806 relevant subset is shown.
807 The number of lines showing the network interfaces can be limited.
808 IFB Infiniband utilization.
810 For every active Infiniband port one line is shown, sorted on
811 activity. Such line shows the name of the port and its busy per‐
812 centage in the first column. The busy percentage is determined by
813 taking the highest of either the transmitted or the received bits
814 during the interval, multiplying that value by the number of lanes
815 and comparing it against the maximum port speed.
816 Furthermore the number of received packets divided by the number
817 of lanes (`pcki'), the number of transmitted packets divided by
818 the number of lanes (`pcko'), the maximum line speed (`sp'), the
819 effective amount of bits received per second (`si'), the effective
820 amount of bits transmitted per second (`so'), and the number of
821 lanes (`lanes').
822 If the screen-width does not allow all of these counters, only a
823 relevant subset is shown.
824 The number of lines showing the Infiniband ports can be limited.
825
827 Following the system level information, the processes are shown from
828 which the resource utilization has changed during the last interval.
829 These processes might have used cpu time or issued disk or network
830 requests. However a process is also shown if part of it has been paged
831 out due to lack of memory (while the process itself was in sleep
832 state).
833 Per process the following fields may be shown (in alphabetical order),
835 depending on the current output mode as described in the section INTER‐
836 ACTIVE COMMANDS and depending on the current width of your window:
837 AVGRSZ The average size of one read-action on disk.
839 AVGWSZ The average size of one write-action on disk.
841 CID Container ID (Docker) of 12 hexadecimal digits, referring to
843 the container in which the process/thread is running. If a
844 process has been started and finished during the last inter‐
845 val, a `?' is shown because the container ID is not part of
846 the standard process accounting record.
847 CMD The name of the process. This name can be surrounded by
849 "less/greater than" signs (`<name>') which means that the
850 process has finished during the last interval.
851 Behind the abbreviation `CMD' in the header line, the current
852 page number and the total number of pages of the
853 process/thread list are shown.
854 COMMAND-LINE
856 The full command line of the process (including arguments). If
857 the length of the command line exceeds the length of the
858 screen line, the arrow keys -> and <- can be used for horizon‐
859 tal scroll.
860 Behind the verb `COMMAND-LINE' in the header line, the current
861 page number and the total number of pages of the
862 process/thread list are shown.
863 CPU The occupation percentage of this process related to the
865 available capacity for this resource on system level.
866 CPUNR The identification of the CPU the (main) thread is running on
868 or has recently been running on.
869 CTID Container ID (OpenVZ). If a process has been started and fin‐
871 ished during the last interval, a `?' is shown because the
872 container ID is not part of the standard process accounting
873 record.
874 DSK The occupation percentage of this process related to the total
876 load that is produced by all processes (i.e. total disk
877 accesses by all processes during the last interval).
878 This information is shown when per process "storage account‐
879 ing" is active in the kernel.
880 EGID Effective group-id under which this process executes.
882 ENDATE Date that the process has been finished. If the process is
884 still running, this field shows `active'.
885 ENTIME Time that the process has been finished. If the process is
887 still running, this field shows `active'.
888 ENVID Virtual environment identified (OpenVZ only).
890 EUID Effective user-id under which this process executes.
892 EXC The exit code of a terminated process (second position of col‐
894 umn `ST' is E) or the fatal signal number (second position of
895 column `ST' is S or C).
896 FSGID Filesystem group-id under which this process executes.
898 FSUID Filesystem user-id under which this process executes.
900 GPU When the pmdanvidia daemon does not run with root privileges,
902 the GPU percentage reflects the GPU memory occupation percent‐
903 age (memory of all GPUs is 100%).
904 When the pmdanvidia daemon runs with root privileges, the GPU
905 percentage reflects the GPU busy percentage.
906 GPUBUSY Busy percentage on all GPUs (one GPU is 100%).
908 When the pmdanvidia daemon does not run with root privileges,
909 this value is not available.
910 GPUNUMS Comma-separated list of GPUs used by the process during the
912 interval. When the comma-separated list exceeds the width of
913 the column, a hexadecimal value is shown.
914 MAJFLT The number of page faults issued by this process that have
916 been solved by creating/loading the requested memory page.
917 MEM The occupation percentage of this process related to the
919 available capacity for this resource on system level.
920 MEMAVG Average memory occupation during the interval on all used
922 GPUs.
923 MEMBUSY Busy percentage of memory on all GPUs (one GPU is 100%), i.e.
925 the time needed for read and write accesses on memory.
926 When the pmdanvidia daemon does not run with root privileges,
927 this value is not available.
928 MEMNOW Memory occupation at the moment of the sample on all used
930 GPUs.
931 MINFLT The number of page faults issued by this process that have
933 been solved by reclaiming the requested memory page from the
934 free list of pages.
935 NET The occupation percentage of this process related to the total
937 load that is produced by all processes (i.e. consumed network
938 bandwidth of all processes during the last interval).
939 This information will only be shown when kernel module
940 `netatop' is loaded.
941 NICE The more or less static priority that can be given to a
943 process on a scale from -20 (high priority) to +19 (low prior‐
944 ity).
945 NPROCS The number of active and terminated processes accumulated for
947 this user or program.
948 PID Process-id.
950 POLI The policies 'norm' (normal, which is SCHED_OTHER), 'btch'
952 (batch) and 'idle' refer to timesharing processes. The poli‐
953 cies 'fifo' (SCHED_FIFO) and 'rr' (round robin, which is
954 SCHED_RR) refer to realtime processes.
955 PPID Parent process-id.
957 PRI The process' priority ranges from 0 (highest priority) to 139
959 (lowest priority). Priority 0 to 99 are used for realtime pro‐
960 cesses (fixed priority independent of their behavior) and pri‐
961 ority 100 to 139 for timesharing processes (variable priority
962 depending on their recent CPU consumption and the nice value).
963 PSIZE The proportional memory size of this process (or user).
965 Every process shares resident memory with other processes.
966 E.g. when a particular program is started several times, the
967 code pages (text) are only loaded once in memory and shared by
968 all incarnations. Also the code of shared libraries is shared
969 by all processes using that shared library, as well as shared
970 memory and memory-mapped files. For the PSIZE calculation of
971 a process, the resident memory of a process that is shared
972 with other processes is divided by the number of sharers.
973 This means, that every process is accounted for a proportional
974 part of that memory. Accumulating the PSIZE values of all pro‐
975 cesses in the system gives a reliable impression of the total
976 resident memory consumed by all processes.
977 Since gathering of all values that are needed to calculate the
978 PSIZE is a relatively time-consuming task, the 'R' key (or
979 '-R' flag) should be active. Gathering these values also
980 requires superuser privileges (otherwise '?K' is shown in the
981 output).
982 RDDSK When the kernel maintains standard io statistics (>= 2.6.20):
984 The read data transfer issued physically on disk (so reading
985 from the disk cache is not accounted for).
986 Unfortunately, the kernel aggregates the data tranfer of a
987 process to the data transfer of its parent process when termi‐
988 nating, so you might see transfers for (parent) processes like
989 cron, bash or init, that are not really issued by them.
990 RGID The real group-id under which the process executes.
992 RGROW The amount of resident memory that the process has grown dur‐
994 ing the last interval. A resident growth can be caused by
995 touching memory pages which were not physically created/loaded
996 before (load-on-demand). Note that a resident growth can also
997 be negative e.g. when part of the process is paged out due to
998 lack of memory or when the process frees dynamically allocated
999 memory. For a process which started during the last interval,
1000 the resident growth reflects the total resident size of the
1001 process at that moment.
1002 RSIZE The total resident memory usage consumed by this process (or
1004 user). Notice that the RSIZE of a process includes all resi‐
1005 dent memory used by that process, even if certain memory parts
1006 are shared with other processes (see also the explanation of
1007 PSIZE).
1008 RTPR Realtime priority according the POSIX standard. Value can be
1010 0 for a timesharing process (policy 'norm', 'btch' or 'idle')
1011 or ranges from 1 (lowest) till 99 (highest) for a realtime
1012 process (policy 'rr' or 'fifo').
1013 RUID The real user-id under which the process executes.
1015 S The current state of the (main) thread: `R' for running (cur‐
1017 rently processing or in the runqueue), `S' for sleeping inter‐
1018 ruptible (wait for an event to occur), `D' for sleeping non-
1019 interruptible, `Z' for zombie (waiting to be synchronized with
1020 its parent process), `T' for stopped (suspended or traced),
1021 `W' for swapping, and `E' (exit) for processes which have fin‐
1022 ished during the last interval.
1023 SGID The saved group-id of the process.
1025 ST The status of a process.
1027 The first position indicates if the process has been started
1028 during the last interval (the value N means 'new process').
1029 The second position indicates if the process has been finished
1031 during the last interval.
1032 The value E means 'exit' on the process' own initiative; the
1033 exit code is displayed in the column `EXC'.
1034 The value S means that the process has been terminated unvol‐
1035 untarily by a signal; the signal number is displayed in the in
1036 the column `EXC'.
1037 The value C means that the process has been terminated unvol‐
1038 untarily by a signal, producing a core dump in its current
1039 directory; the signal number is displayed in the column `EXC'.
1040 STDATE The start date of the process.
1042 STTIME The start time of the process.
1044 SUID The saved user-id of the process.
1046 SWAPSZ The swap space consumed by this process (or user).
1048 SYSCPU CPU time consumption of this process in system mode (kernel
1050 mode), usually due to system call handling.
1051 THR Total number of threads within this process. All related
1053 threads are contained in a thread group, represented by pcp-
1054 atop as one line or as a separate line when the 'y' key (or -y
1055 flag) is active.
1056 On Linux 2.4 systems it is hardly possible to determine which
1058 threads (i.e. processes) are related to the same thread group.
1059 Every thread is represented by pcp-atop as a separate line.
1060 TID Thread-id. All threads within a process run with the same PID
1062 but with a different TID. This value is shown for individual
1063 threads in multi-threaded processes (when using the key 'y').
1064 TRUN Number of threads within this process that are in the state
1066 'running' (R).
1067 TSLPI Number of threads within this process that are in the state
1069 'interruptible sleeping' (S).
1070 TSLPU Number of threads within this process that are in the state
1072 'uninterruptible sleeping' (D).
1073 USRCPU CPU time consumption of this process in user mode, due to pro‐
1075 cessing the own program text.
1076 VDATA The virtual memory size of the private data used by this
1078 process (including heap and shared library data).
1079 VGROW The amount of virtual memory that the process has grown during
1081 the last interval. A virtual growth can be caused by e.g.
1082 issueing a malloc() or attaching a shared memory segment. Note
1083 that a virtual growth can also be negative by e.g. issueing a
1084 free() or detaching a shared memory segment. For a process
1085 which started during the last interval, the virtual growth
1086 reflects the total virtual size of the process at that moment.
1087 VPID Virtual process-id (within an OpenVZ container). If a process
1089 has been started and finished during the last interval, a `?'
1090 is shown because the virtual process-id is not part of the
1091 standard process accounting record.
1092 VSIZE The total virtual memory usage consumed by this process (or
1094 user).
1095 VSLIBS The virtual memory size of the (shared) text of all shared
1097 libraries used by this process.
1098 VSTACK The virtual memory size of the (private) stack used by this
1100 process
1101 VSTEXT The virtual memory size of the (shared) text of the executable
1103 program.
1104 WRDSK When the kernel maintains standard io statistics (>= 2.6.20):
1106 The write data transfer issued physically on disk (so writing
1107 to the disk cache is not accounted for). This counter is
1108 maintained for the application process that writes its data to
1109 the cache (assuming that this data is physically transferred
1110 to disk later on). Notice that disk I/O needed for swapping is
1111 not taken into account.
1112 Unfortunately, the kernel aggregates the data tranfer of a
1113 process to the data transfer of its parent process when termi‐
1114 nating, so you might see transfers for (parent) processes like
1115 cron, bash or init, that are not really issued by them.
1116 WCANCL When the kernel maintains standard io statistics (>= 2.6.20):
1118 The write data transfer previously accounted for this process
1119 or another process that has been cancelled. Suppose that a
1120 process writes new data to a file and that data is removed
1121 again before the cache buffers have been flushed to disk.
1122 Then the original process shows the written data as WRDSK,
1123 while the process that removes/truncates the file shows the
1124 unflushed removed data as WCANCL.
1125
1127 With the flag -P followed by a list of one or more labels (comma-sepa‐
1128 rated), parseable output is produced for each sample. The labels that
1129 can be specified for system-level statistics correspond to the labels
1130 (first verb of each line) that can be found in the interactive output:
1131 "CPU", "cpu", "CPL", "GPU", "MEM", "SWP", "PAG", "PSI", "LVM", "MDD",
1132 "DSK", "NFM", "NFC", "NFS", "NET" and "IFB".
1133 For process-level statistics special labels are introduced: "PRG" (gen‐
1134 eral), "PRC" (cpu), "PRE" (GPU), "PRM" (memory), "PRD" (disk, only if
1135 "storage accounting" is active).
1136 With the label "ALL", all system and process level statistics are
1137 shown.
1138 For every interval all requested lines are shown whereafter pcp-atop
1140 shows a line just containing the label "SEP" as a separator before the
1141 lines for the next sample are generated.
1142 When a sample contains the values since boot, pcp-atop shows a line
1143 just containing the label "RESET" before the lines for this sample are
1144 generated.
1145 The first part of each output-line consists of the following six
1147 fields: label (the name of the label), host (the name of this machine),
1148 epoch (the time of this interval as number of seconds since 1-1-1970),
1149 date (date of this interval in format YYYY/MM/DD), time (time of this
1150 interval in format HH:MM:SS), and interval (number of seconds elapsed
1151 for this interval).
1152 The subsequent fields of each output-line depend on the label:
1154 CPU Subsequent fields: total number of clock-ticks per second for
1156 this machine, number of processors, consumption for all CPUs
1157 in system mode (clock-ticks), consumption for all CPUs in user
1158 mode (clock-ticks), consumption for all CPUs in user mode for
1159 niced processes (clock-ticks), consumption for all CPUs in
1160 idle mode (clock-ticks), consumption for all CPUs in wait mode
1161 (clock-ticks), consumption for all CPUs in irq mode (clock-
1162 ticks), consumption for all CPUs in softirq mode (clock-
1163 ticks), consumption for all CPUs in steal mode (clock-ticks),
1164 consumption for all CPUs in guest mode (clock-ticks) overlap‐
1165 ping user mode, frequency of all CPUs and frequency percentage
1166 of all CPUs.
1167 cpu Subsequent fields: total number of clock-ticks per second for
1169 this machine, processor-number, consumption for this CPU in
1170 system mode (clock-ticks), consumption for this CPU in user
1171 mode (clock-ticks), consumption for this CPU in user mode for
1172 niced processes (clock-ticks), consumption for this CPU in
1173 idle mode (clock-ticks), consumption for this CPU in wait mode
1174 (clock-ticks), consumption for this CPU in irq mode (clock-
1175 ticks), consumption for this CPU in softirq mode (clock-
1176 ticks), consumption for this CPU in steal mode (clock-ticks),
1177 consumption for this CPU in guest mode (clock-ticks) overlap‐
1178 ping user mode, frequency of all CPUs, frequency percentage of
1179 all CPUs, instructions executed by all CPUs and cycles for all
1180 CPUs.
1181 CPL Subsequent fields: number of processors, load average for last
1183 minute, load average for last five minutes, load average for
1184 last fifteen minutes, number of context-switches, and number
1185 of device interrupts.
1186 GPU Subsequent fields: GPU number, bus-id string, type of GPU
1188 string, GPU busy percentage during last second (-1 if not
1189 available), memory busy percentage during last second (-1 if
1190 not available), total memory size (KiB), used memory (KiB) at
1191 this moment, number of samples taken during interval, cumula‐
1192 tive GPU busy percentage during the interval (to be divided by
1193 the number of samples for the average busy percentage, -1 if
1194 not available), cumulative memory busy percentage during the
1195 interval (to be divided by the number of samples for the aver‐
1196 age busy percentage, -1 if not available), and cumulative mem‐
1197 ory occupation during the interval (to be divided by the num‐
1198 ber of samples for the average occupation).
1199 MEM Subsequent fields: page size for this machine (in bytes), size
1201 of physical memory (pages), size of free memory (pages), size
1202 of page cache (pages), size of buffer cache (pages), size of
1203 slab (pages), dirty pages in cache (pages), reclaimable part
1204 of slab (pages), total size of vmware's balloon pages (pages),
1205 total size of shared memory (pages), size of resident shared
1206 memory (pages), size of swapped shared memory (pages), huge
1207 page size (in bytes), total size of huge pages (huge pages),
1208 and size of free huge pages (huge pages).
1209 SWP Subsequent fields: page size for this machine (in bytes), size
1211 of swap (pages), size of free swap (pages), 0 (future use),
1212 size of committed space (pages), and limit for committed space
1213 (pages).
1214 PAG Subsequent fields: page size for this machine (in bytes), num‐
1216 ber of page scans, number of allocstalls, 0 (future use), num‐
1217 ber of swapins, and number of swapouts.
1218 PSI Subsequent fields: PSI statistics present on this system (n or
1220 y), CPU some avg10, CPU some avg60, CPU some avg300, CPU some
1221 accumulated microseconds during interval, memory some avg10,
1222 memory some avg60, memory some avg300, memory some accumulated
1223 microseconds during interval, memory full avg10, memory full
1224 avg60, memory full avg300, memory full accumulated microsec‐
1225 onds during interval, I/O some avg10, I/O some avg60, I/O some
1226 avg300, I/O some accumulated microseconds during interval, I/O
1227 full avg10, I/O full avg60, I/O full avg300, and I/O full
1228 accumulated microseconds during interval.
1229 LVM/MDD/DSK
1231 For every logical volume/multiple device/hard disk one line is
1232 shown.
1233 Subsequent fields: name, number of milliseconds spent for I/O,
1234 number of reads issued, number of sectors transferred for
1235 reads, number of writes issued, and number of sectors trans‐
1236 ferred for write.
1237 NFM Subsequent fields: mounted NFS filesystem, total number of
1239 bytes read, total number of bytes written, number of bytes
1240 read by normal system calls, number of bytes written by normal
1241 system calls, number of bytes read by direct I/O, number of
1242 bytes written by direct I/O, number of pages read by memory-
1243 mapped I/O, and number of pages written by memory-mapped I/O.
1244 NFC Subsequent fields: number of transmitted RPCs, number of
1246 transmitted read RPCs, number of transmitted write RPCs, num‐
1247 ber of RPC retransmissions, and number of authorization
1248 refreshes.
1249 NFS Subsequent fields: number of handled RPCs, number of received
1251 read RPCs, number of received write RPCs, number of bytes read
1252 by clients, number of bytes written by clients, number of RPCs
1253 with bad format, number of RPCs with bad authorization, number
1254 of RPCs from bad client, total number of handled network
1255 requests, number of handled network requests via TCP, number
1256 of handled network requests via UDP, number of handled TCP
1257 connections, number of hits on reply cache, number of misses
1258 on reply cache, and number of uncached requests.
1259 NET First one line is produced for the upper layers of the TCP/IP
1261 stack.
1262 Subsequent fields: the verb "upper", number of packets
1263 received by TCP, number of packets transmitted by TCP, number
1264 of packets received by UDP, number of packets transmitted by
1265 UDP, number of packets received by IP, number of packets
1266 transmitted by IP, number of packets delivered to higher lay‐
1267 ers by IP, and number of packets forwarded by IP.
1268 Next one line is shown for every interface.
1270 Subsequent fields: name of the interface, number of packets
1271 received by the interface, number of bytes received by the
1272 interface, number of packets transmitted by the interface,
1273 number of bytes transmitted by the interface, interface speed,
1274 and duplex mode (0=half, 1=full).
1275 IFB Subsequent fields: name of the InfiniBand interface, port num‐
1277 ber, number of lanes, maximum rate (Mbps), number of bytes
1278 received, number of bytes transmitted, number of packets
1279 received, and number of packets transmitted.
1280 PRG For every process one line is shown.
1282 Subsequent fields: PID (unique ID of task), name (between
1283 brackets), state, real uid, real gid, TGID (group number of
1284 related tasks/threads), total number of threads, exit code (in
1285 case of fatal signal: signal number + 256), start time
1286 (epoch), full command line (between brackets), PPID, number of
1287 threads in state 'running' (R), number of threads in state
1288 'interruptible sleeping' (S), number of threads in state
1289 'uninterruptible sleeping' (D), effective uid, effective gid,
1290 saved uid, saved gid, filesystem uid, filesystem gid, elapsed
1291 time (hertz), is_process (y/n), OpenVZ virtual pid (VPID),
1292 OpenVZ container id (CTID) and Docker container id (CID).
1293 PRC For every process one line is shown.
1295 Subsequent fields: PID, name (between brackets), state, total
1296 number of clock-ticks per second for this machine, CPU-con‐
1297 sumption in user mode (clockticks), CPU-consumption in system
1298 mode (clockticks), nice value, priority, realtime priority,
1299 scheduling policy, current CPU, sleep average, TGID (group
1300 number of related tasks/threads) and is_process (y/n).
1301 PRE For every process one line is shown.
1303 Subsequent fields: PID, name (between brackets), process
1304 state, GPU state (A for active, E for exited, N for no GPU
1305 user), number of GPUs used by this process, bitlist reflecting
1306 used GPUs, GPU busy percentage during interval, memory busy
1307 percentage during interval, memory occupation (KiB) at this
1308 moment cumulative memory occupation (KiB) during interval, and
1309 number of samples taken during interval.
1310 PRM For every process one line is shown.
1312 Subsequent fields: PID, name (between brackets), state, page
1313 size for this machine (in bytes), virtual memory size
1314 (Kbytes), resident memory size (Kbytes), shared text memory
1315 size (Kbytes), virtual memory growth (Kbytes), resident memory
1316 growth (Kbytes), number of minor page faults, number of major
1317 page faults, virtual library exec size (Kbytes), virtual data
1318 size (Kbytes), virtual stack size (Kbytes), swap space used
1319 (Kbytes), TGID (group number of related tasks/threads),
1320 is_process (y/n) and proportional set size (Kbytes) if in 'R'
1321 option is specified.
1322 PRD For every process one line is shown.
1324 Subsequent fields: PID, name (between brackets), state, obso‐
1325 leted kernel patch installed ('n'), standard io statistics
1326 used ('y' or 'n'), number of reads on disk, cumulative number
1327 of sectors read, number of writes on disk, cumulative number
1328 of sectors written, cancelled number of written sectors, TGID
1329 (group number of related tasks/threads) and is_process (y/n).
1330 If the standard I/O statistics (>= 2.6.20) are not used, the
1331 disk I/O counters per process are not relevant. The counters
1332 'number of reads on disk' and 'number of writes on disk' are
1333 obsoleted anyhow.
1334 PRN For every process one line is shown.
1336 Subsequent fields: PID, name (between brackets), state, kernel
1337 module 'netatop' loaded ('y' or 'n'), number of TCP-packets
1338 transmitted, cumulative size of TCP-packets transmitted, num‐
1339 ber of TCP-packets received, cumulative size of TCP-packets
1340 received, number of UDP-packets transmitted, cumulative size
1341 of UDP-packets transmitted, number of UDP-packets received,
1342 cumulative size of UDP-packets transmitted, number of raw
1343 packets transmitted (obsolete, always 0), number of raw pack‐
1344 ets received (obsolete, always 0), TGID (group number of
1345 related tasks/threads) and is_process (y/n).
1346
1348 By sending the SIGUSR1 signal to pcp-atop a new sample will be forced,
1349 even if the current timer interval has not exceeded yet. The behavior
1350 is similar to pressing the `t` key in an interactive session.
1351 By sending the SIGUSR2 signal to pcp-atop a final sample will be forced
1353 after which pcp-atop will terminate.
1354
1356 To monitor the current system load interactively with an interval of 5
1357 seconds:
1358 pcp atop 5
1360 To monitor the system load and write it to a file (in plain ASCII) with
1362 an interval of one minute during half an hour with active processes
1363 sorted on memory consumption:
1364 pcp atop -M 60 30 > /log/pcp-atop.mem
1366 Store information about the system and process activity in a PCP ar‐
1368 chive folio with an interval of ten minutes during an hour:
1369 pcp atop -w /tmp/pcp-atop 600 6
1371 View the contents of this file interactively:
1373 pcp atop -r /tmp/pcp-atop
1375 View the processor and disk utilization of this file in parseable for‐
1377 mat:
1378 pcp atop -PCPU,DSK -r /tmp/pcp-atop.raw
1380 View the contents of today's standard logfile interactively:
1382 pcp atop -r
1384 View the contents of the standard logfile of the day before yesterday
1386 interactively:
1387 pcp atop -r yy
1389 View the contents of the standard logfile of 2014, June 7 from 02:00 PM
1391 onwards interactively:
1392 pcp atop -r 20140607 -b 14:00
1394
1396 /etc/atoprc
1397 Configuration file containing system-wide default values. See
1398 related man-page.
1399 ~/.atoprc
1401 Configuration file containing personal default values. See
1402 related man-page.
1403
1405 pcp-atop is based on the source code of the atop(1) command from
1406 https://atoptool.nl, maintained by Gerlof Langeveld
1407 (gerlof.langeveld@atoptool.nl), and aims to be command line and output
1408 compatible with it as much as possible. Some features of that atop
1409 command are not available in pcp-atop.
1410 Some features of pcp-atop (such as reporting on the Apache HTTP daemon,
1412 Infiniband, NFS client mounts, hardware event counts and GPU statis‐
1413 tics) are only activated if the corresonding PCP metrics are available.
1414 Refer to the documentation for pmdaapache(1), pmdainfiniband(1), pmdan‐
1415 fsclient(1), pmdanvidia(1) and pmdaperfevent(1) for further details on
1416 activating these metrics.
1417
1419 pcp(1), pcp-atopsar(1), pmdaapache(1), pmdainfiniband(1), pmdanf‐
1420 sclient(1), pmdanvidia(1), pmdaproc(1), mkaf(1), pmlogger(1), pmlog‐
1421 ger_daily(1), PCPIntro(1) and pcp-atoprc(5).
1422Performance Co-Pilot PCP PCP-ATOP(1)