1PCP-ATOP(1) General Commands Manual PCP-ATOP(1)
2
6 pcp-atop - Advanced System and Process Monitor
7
9 Interactive Usage:
10 pcp [pcp options] atop [-aABcCdDfFgGHmMnNopRsuvxyY1] [-L linelen]
12 [-Plabel[,label]... [-Z]] [interval [samples]]
13 Writing and reading PCP archive folios:
15 pcp atop -w folio [-a] [-S] [interval [samples]]
17 pcp atop -r folio [-AcCdDfFgGmMnNopRsuvxy1] [-b [yy-mm-dd] hh:mm] [-e
18 yy-mm-dd] hh:mm] [-L linelen] [-Plabel[,label]... [-Z]] [interval [sam‐
19 ples]]
20
22 The program pcp-atop is an interactive monitor to view various aspects
23 of load on a system. Every interval seconds (default: 10 seconds) in‐
24 formation is gathered about the resource occupation on system level of
25 the most critical hardware resources (from a performance point of
26 view), i.e. CPUs, memory, disks and network interfaces. Besides, infor‐
27 mation is gathered about the processes (or threads) that are responsi‐
28 ble for the utilization of the CPUs, memory and disks. Network load
29 per process is shown only when the PCP pmdabcc(1) metrics have been in‐
30 stalled and configured.
31
33 When running pcp-atop you can choose to view the system load in bar
34 graph mode or in text mode. In bar graph mode the resource utilization
35 of CPUs, memory, disks and network interfaces is shown via (character-
36 based) bar graphs, but only on system level. When you want to view
37 more detailed information on system level or when you want to view the
38 resource consumption on process or thread level, you can switch to text
39 mode by pressing the 'B' key. Alternatively, you can use the 'B' key
40 (again) to switch from text mode to bar graph mode.
41 By default, pcp-atop starts in text mode unless the -B flag is used or
42 unless 'B' has been configured as a default flag in the .atoprc file
43 (for further information about default flags, refer to the pcp-ato‐
44 prc(5) man page).
45 In bar graph mode the terminal will be subdivided into four character-
47 based windows, i.e. one window for each hardware resource:
48 Processors
50 The first bar shows the average busy percentage of all CPUs with
51 the bar label 'Avg' (might be abbreviated to 'Av' or even just
52 'A'). The subsequent bars show the busy percentages of single
53 CPUs.
54 When there is not enough horizontal space to show all CPUs, only
55 the most busy CPUs per sample will be shown after the width of
56 each bar has been reduced to a minimum.
57 By default, the categories of CPU consumption are shown by differ‐
59 ent colors in the bars, marked with a character 'S' (system mode),
60 'U' (user mode), 'I' (interrupt handling), 's' (steal) and 'G'
61 (guest, i.e. consumed by virtual machines).
62 The top of the bar might consist of an unmarked color representing
63 a 'neutral' category. Suppose that the scale unit is 5% per line
64 and the total busy percentage is 54% consisting of two categories
65 of 27%. The two categories will be rounded to 25% (5 lines of 5%
66 each) but the total busy percentage will be rounded to 55% (11
67 lines of 5%). Then the top line will represent a 'neutral' cate‐
68 gory.
69 By pressing the 'H' key or by starting pcp-atop with the -H flag,
70 no categories are shown.
71 A red line is drawn in the bar graph as critical threshold. By
73 default this value is 90% and can be modified by the 'cpucritperc'
74 option in the configuration file (see separate pcp-atoprc(5) man
75 page). When this value is set to zero, no threshold line will be
76 drawn.
77 Memory and swap space
79 Memory is presented as a column in which the specific categories
80 of memory consumption are shown. These categories are (code, data
81 and stack of) processes/kernel, slab caches (i.e. dynamically al‐
82 located kernel memory), shared memory, tmpfs, page cache and free
83 memory.
84 Swap space (if present) is also presented as a column in which the
85 categories processes/tmpfs, shared memory and free space are
86 shown.
87 At the right side memory-related event counters are shown.
89 The bottom three counters are colored green when there is no mem‐
90 ory pressure. When considerable activity is noticed such counter
91 might be colored orange and with high activity red.
92 When memory pressure starts, usually memory page scanning will be
93 activated first. When pressure increases, memory pages of pro‐
94 cesses might be swapped out to swap space (if present).
95 The 'oomkills' counter (Out Of Memory killing) is most serious: it
96 reflects the number of processes that are killed due to lack of
97 memory (and swap). Therefore this counter shows the absolute num‐
98 ber (not per second) of processes being killed during the last in‐
99 terval and will immediately be colored red when it is 1 or more.
100 Besides, after pcp-atop has noticed OOM killing the 'oomkills'
101 counter remains orange for the next 15 minutes, just in case that
102 you have missed the OOM killing event itself.
103 When there is enough vertical space in the memory window, event
104 counters are shown about the number of memory pages being swapped
105 in, the number of memory pages paged out to block devices and the
106 number of memory pages paged in from block devices.
107 Memory and swap space consumption will preferably be shown in a
109 character-based window that vertically uses the entire screen for
110 optimal granularity. However, when there are a lot of disks and/or
111 network interfaces the memory and swap space consumption will be
112 shown in a character-based window that only uses the upper half of
113 the screen.
114 Disks
116 For each disk the busy percentage is shown as a bar.
117 When there is not enough horizontal space to show all disks, only
118 the most busy disks per sample will be shown.
119 By default, categories of disk consumption are shown by different
121 colors in the bars, marked with a character 'R' (read) and 'W'
122 (write).
123 The top of the bar might consist of an unmarked color representing
124 a 'neutral' category. Suppose that the scale unit is 5% per line
125 and the total busy percentage is 54% consisting of two categories
126 of 27%. The two categories will be rounded to 25% (5 lines of 5%
127 each) but the total busy percentage will be rounded to 55% (11
128 lines of 5%). Then the top line will represent a 'neutral' cate‐
129 gory.
130 By pressing the 'H' key or by starting pcp-atop with the -H flag,
131 no categories are shown.
132 A red line is drawn in the bar graph as critical threshold. By
134 default this value is 90% and can be modified by the 'dskcritperc'
135 option in the configuration file (see separate atoprc man page).
136 When this value is set to zero, no threshold line will be drawn.
137 Interfaces
139 For each non-virtual network interface a double bar graph is shown
140 with a dedicated scale that reflects the traffic rate. One of the
141 bars shows the transmit rate ('TX') and the other bar the receive
142 rate ('RX'). The traffic scale of each network interface remains
143 at its highest level. All interface scales can be reset during
144 the measurement by pressing the 'L' key.
145 Most often the real speed (maximum bandwidth) of network inter‐
147 faces is not known, e.g. in case of the network interfaces of vir‐
148 tual machines. Therefore it is not possible to show the interface
149 utilization as a percentage. However, when the real speed of an
150 interface is known it will be shown underneath the concerning bar
151 graph.
152 When there is not enough horizontal space to show all network in‐
154 terfaces, only the most busy interfaces per sample will be shown.
155 Usually the bar graphs will not be sorted on busy percentage when there
157 is enough horizontal space. However, after switching from text mode to
158 bar graph mode the bar graphs might have been sorted because this was
159 needed for the presentation in text mode. The next interval in bar
160 graph mode shows the bars unsorted again unless the window width is un‐
161 sufficient for all bars.
162 The remaining part of this manual page mainly describes the information
164 shown in text mode. When certain descriptions also apply to bar graph
165 mode it will be mentioned explicitly.
166
169 The initial screen in text mode shows if pcp-atop runs with restricted
170 view (unprivileged user) or unrestricted view (privileged user). In
171 case of restricted view pcp-atop does not have the privileges (root
172 identity or necessary capabilities) to retrieve all counter values on
173 system level and on process level. does not have the privileges (no
174 root identity nor the necessary capabilities) to retrieve all counter
175 values on system level and on process level.
176 With every interval information is shown about the resource occupation
178 on system level (CPU, memory, disks and network layers), followed by a
179 list of processes which have been active during the last interval. No‐
180 tice that all processes that were unchanged during the last interval re
181 not shown, unless the key 'a' has been pressed or unless sorting on
182 memory occupation is done (then inactive processes are relevant as
183 well). If the list of active processes does not entirely fit on the
184 screen, only the top of the list is shown (sorted in order of activ‐
185 ity).
186 The intervals are repeated till the number of samples (specified as
187 command argument) is reached, or till the key 'q' is pressed in inter‐
188 active mode.
189 When invoked via the pcp(1) command, the PCPIntro(1) options
191 -A/--align, -a/--archive, -h/--host, -O/--origin, -S/--start,
192 -s/--samples, -T/--finish, -t/--interval, -v/--version, -z/--hostzone
193 and -z/--timezone become indirectly available. Additionally, the
194 --hotproc option can be used to request the per-process PCP metrics be
195 used instead of the default proc metrics from pmdaproc(1).
196 When pcp-atop is started, it checks whether the standard output channel
198 is connected to a screen, or to a file/pipe. In the first case it pro‐
199 duces screen control codes (via the ncurses library) and behaves inter‐
200 actively; in the second case it produces flat text output.
201 In interactive mode, the output of pcp-atop scales dynamically to the
203 current dimensions of the screen/window.
204 If the window is resized horizontally, columns will be added or removed
205 automatically. For this purpose, every column has a particular weight.
206 The columns with the highest weights that fit within the current width
207 will be shown.
208 If the window is resized vertically, lines of the process/thread list
209 will be added or removed automatically.
210 In interactive mode the output of pcp-atop can be controlled by press‐
212 ing particular keys. However it is also possible to specify such key
213 as flag on the command line. In that case pcp-atop switches to the in‐
214 dicated mode on beforehand. This mode can be modified again interac‐
215 tively. Specifying such key as flag is especially useful when running
216 pcp-atop with output to a pipe or file (non-interactively). These
217 flags are the same as the keys that can be pressed in interactive mode
218 (see section INTERACTIVE COMMANDS).
219 Additional flags are available to support storage of pcp-atop data in
220 PCP archive format (see section PCP DATA STORAGE).
221
223 For the resource consumption on system level, pcp-atop uses colors in
224 text mode to indicate that a critical occupation percentage has been
225 (almost) reached. A critical occupation percentage means that is like‐
226 ly that this load causes a noticeable negative performance influence
227 for applications using this resource. The critical percentage depends
228 on the type of resource: e.g. the performance influence of a disk with
229 a busy percentage of 80% might be more noticeable for applica‐
230 tions/users than a CPU with a busy percentage of 90%.
231 Currently pcp-atop uses the following default values to calculate a
232 weighted percentage per resource:
233 Processor
235 A busy percentage of 90% or higher is considered 'critical' (also
236 in bar graph mode).
237 Disk
239 A busy percentage of 90% or higher is considered 'critical'.
240 Network
242 A busy percentage of 90% or higher for the load of an interface is
243 considered 'critical'.
244 Memory
246 An occupation percentage of 90% is considered 'critical'. Notice
247 that this occupation percentage is the accumulated memory consump‐
248 tion of the kernel (including slab) and all processes. The memory
249 for the page cache ('cache' and 'buff' in the MEM-line) and the
250 reclaimable part of the slab ('slrec') is not implied!
251 If the number of pages swapped out ('swout' in the PAG-line) is
252 larger than 10 per second, the memory resource is considered
253 'critical'. A value of at least 1 per second is considered 'al‐
254 most critical'.
255 If the committed virtual memory exceeds the limit ('vmcom' and
256 'vmlim' in the SWP-line), the SWP-line is colored due to overcom‐
257 mitting the system.
258 Swap
260 An occupation percentage of 80% is considered 'critical' because
261 swap space might be completely exhausted in the near future. It
262 is not critical from a performance point-of-view.
263 These default values can be modified in the configuration file (see
265 separate pcp-atoprc(5) man page).
266 When a resource exceeds its critical occupation percentage, the con‐
268 cerning values in the screen line are colored red by default.
269 When a resource exceeds (by default) 80% of its critical percentage (so
270 it is almost critical), the concerning values in the screen line are
271 colored cyan by default. This 'almost critical percentage' (one value
272 for all resources) can be modified in the configuration file (see sepa‐
273 rate pcp-atoprc(5) man page).
274 The default colors red and cyan can be modified in the configuration
275 file as well (see separate man-page of pcp-atoprc(5)).
276 With the key 'x' (or flag -x), the use of colors can be suppressed in
278 text mode. The use of colors is however mandatory in case of bar graph
279 mode.
280
282 GPU statistics can be gathered by pmdanvidia(1) which is a separate da‐
283 ta collection daemon process. It gathers cumulative utilization coun‐
284 ters of every Nvidia GPU in the system, as well as utilization counters
285 of every process that uses a GPU. When pcp-atop notices that the dae‐
286 mon is active, it reads these GPU utilization counters with every in‐
287 terval.
288 Find a description about the utilization counters in the section OUTPUT
290 DESCRIPTION.
291
293 When running pcp-atop interactively (no output redirection), keys can
294 be pressed to control the output. In general, lower case keys can be
295 used to show other information for the active processes while certain
296 upper case keys can be used to influence the sort order of the active
297 process/thread list. Some of these keys can also be used to switch from
298 bar graph mode to particular detailed process information in text mode.
299 g Show generic output (default).
301 Per process the following fields are shown in case of a window-
303 width of 80 positions: process-id, CPU consumption during the last
304 interval in system and user mode, the virtual and resident memory
305 growth of the process.
306 The data transfer per process for read/write on disk can only be
307 shown when pcp-atop accesses metrics with root privileges.
308 The last columns contain the state, the occupation percentage for
309 the chosen resource (default: CPU) and the process name.
310 When more than 80 positions are available, other information is
312 added.
313 m Show memory related output.
315 Per process the following fields are shown in case of a window
317 width of 80 positions: process-id, minor and major memory faults,
318 size of virtual shared text, total virtual process size, total
319 resident process size, virtual and resident growth during last in‐
320 terval, memory occupation percentage and process name.
321 When more than 80 positions are available, other information is
323 added.
324 For memory consumption, always all processes are shown (also the
326 processes that were not active during the interval).
327 d Show disk-related output.
329 When pcp-atop runs with root privileges, the following fields are
331 shown: process-id, amount of data read from disk, amount of data
332 written to disk, amount of data that was written but has been
333 withdrawn again (WCANCL), disk occupation percentage and process
334 name.
335 s Show scheduling characteristics.
337 Per process the following fields are shown in case of a window
339 width of 80 positions: process-id, number of threads in state
340 'running' (R), number of threads in state 'interruptible sleeping'
341 (S), number of threads in state 'uninterruptible sleeping' (D),
342 scheduling policy (normal timesharing, realtime round-robin, real‐
343 time fifo), nice value, priority, realtime priority, current pro‐
344 cessor, status, exit code, state, the occupation percentage for
345 the chosen resource and the process name.
346 When more than 80 positions are available, other information is
348 added.
349 v Show various process characteristics.
351 Per process the following fields are shown in case of a window
353 width of 80 positions: process-id, user name and group, start date
354 and time, status (e.g. exit code if the process has finished),
355 state, the occupation percentage for the chosen resource and the
356 process name.
357 When more than 80 positions are available, other information is
359 added.
360 c Show the command line of the process.
362 Per process the following fields are shown: process-id, the occu‐
364 pation percentage for the chosen resource and the command line in‐
365 cluding arguments.
366 X Show cgroup v2 information.
368 Per process the following fields are shown: process-id, the com‐
370 mand name, and the cgroup path name (horizontally scrollable).
371 e Show GPU utilization.
373 Per process at least the following fields are shown: process-id,
375 range of GPU numbers on which the process currently runs, GPU busy
376 percentage on all GPUs, memory busy percentage (i.e. read and
377 write accesses on memory) on all GPUs, memory occupation at the
378 moment of the sample, average memory occupation during the sample,
379 and GPU percentage.
380 When the pmdanvidia daemon does not run with root privileges, the
382 GPU busy percentage and the memory busy percentage are not avail‐
383 able on process level. In that case, the GPU percentage on
384 process level reflects the GPU memory occupation instead of the
385 GPU busy percentage (which is preferred).
386 o Show the user-defined line of the process.
388 In the configuration file the keyword ownprocline can be specified
390 with the description of a user-defined output-line.
391 Refer to the man-page of pcp-atoprc(5) for a detailed description.
392 y Show the individual threads within a process (toggle).
394 Single-threaded processes are still shown as one line.
396 For multi-threaded processes, one line represents the process
397 while additional lines show the activity per individual thread (in
398 a different color). Depending on the option 'a' (all or active
399 toggle), all threads are shown or only the threads that were ac‐
400 tive during the last interval. Depending on the option 'Y' (sort
401 threads), the threads per process will be sorted on the chosen
402 sort criterium or not.
403 Whether this key is active or not can be seen in the header line.
404 Y Sort the threads per process when combined with option 'y' (tog‐
406 gle).
407 u Show the process activity accumulated per user.
409 Per user the following fields are shown: number of processes ac‐
411 tive or terminated during last interval (or in total if combined
412 with command 'a'), accumulated CPU consumption during last inter‐
413 val in system and user mode, the current virtual and resident mem‐
414 ory space consumed by active processes (or all processes of the
415 user if combined with command 'a').
416 When pcp-atop access metrics with root privileges, the accumulated
417 read and write throughput on disk is shown. When the pmdabcc(1)
418 module 'netproc' has been installed, the number of receive and
419 send network calls are shown.
420 The last columns contain the accumulated occupation percentage for
421 the chosen resource (default: CPU) and the user name.
422 p Show the process activity accumulated per program (i.e. process
424 name).
425 Per program the following fields are shown: number of processes
427 active or terminated during last interval (or in total if combined
428 with command 'a'), accumulated CPU consumption during last inter‐
429 val in system and user mode, the current virtual and resident mem‐
430 ory space consumed by active processes (or all processes of the
431 user if combined with command 'a').
432 When pcp-atop access metrics with root privileges, the accumulated
433 read and write throughput on disk is shown. When the pmdabcc(1)
434 module 'netproc' has been installed, the number of receive and
435 send network calls are shown.
436 The last columns contain the accumulated occupation percentage for
437 the chosen resource (default: CPU) and the program name.
438 j Show the process activity accumulated per Docker container.
440 Per container the following fields are shown: number of processes
442 active or terminated during last interval (or in total if combined
443 with command 'a'), accumulated CPU consumption during last inter‐
444 val in system and user mode, the current virtual and resident mem‐
445 ory space consumed by active processes (or all processes of the
446 user if combined with command 'a').
447 When pcp-atop access metrics with root privileges, the accumulated
448 read and write throughput on disk is shown. When the pmdabcc(1)
449 module 'netproc' has been installed, the number of receive and
450 send network calls are shown.
451 The last columns contain the accumulated occupation percentage for
452 the chosen resource (default: CPU) and the Docker container id
453 (CID).
454 C Sort the current list in the order of CPU consumption (default).
456 The one-but-last column changes to 'CPU'.
457 E Sort the current list in the order of GPU utilization (preferred,
459 but only applicable when the pmdanvidia daemon runs under root
460 privileges) or the order of GPU memory occupation). The one-but-
461 last column changes to 'GPU'.
462 M Sort the current list in the order of resident memory consumption.
464 The one-but-last column changes to 'MEM'. In case of sorting on
465 memory, the full process list will be shown (not only the active
466 processes).
467 D Sort the current list in the order of disk accesses issued. The
469 one-but-last column changes to 'DSK'.
470 N Sort the current list in the order of network bandwidth (received
472 and transmitted). The one-but-last column changes to 'NET'.
473 A Sort the current list automatically in the order of the most busy
475 system resource during this interval. The one-but-last column
476 shows either 'ACPU', 'AMEM', 'ADSK' or 'ANET' (the preceding 'A'
477 indicates automatic sorting-order). The most busy resource is de‐
478 termined by comparing the weighted busy-percentages of the system
479 resources, as described earlier in the section COLORS.
480 This option remains valid until another sorting-order is explicit‐
481 ly selected again.
482 A sorting order for disk is only possible when pcp-atop runs with
483 root privileges. A sorting-order for network is only possible
484 when the pmdabcc(1) module 'netproc' has been installed.
485 Miscellaneous interactive commands:
487 ? Request for help information (also the key 'h' can be pressed).
489 V Request for version information (version number and date).
491 R Gather and calculate the proportional set size of processes (tog‐
493 gle). Gathering of all values that are needed to calculate the
494 PSIZE of a process is a very time-consuming task, so this key
495 should only be active when analyzing the resident memory consump‐
496 tion of processes.
497 W Get the WCHAN per thread (toggle). Gathering of the WCHAN string
499 per thread is a relatively time-consuming task, so this key should
500 only be made active when analyzing the reason for threads to be in
501 sleep state.
502 x Suppress colors to highlight critical resources (toggle).
504 Whether this key is active or not can be seen in the header line.
505 z The pause key can be used to freeze the current situation in order
507 to investigate the output on the screen. While pcp-atop is
508 paused, the keys described above can be pressed to show other in‐
509 formation about the current list of processes. Whenever the pause
510 key is pressed again, pcp-atop will continue with the next sample.
511 The pause key can be used in text mode and bar graph mode.
512 i Modify the interval timer (default: 10 seconds). If an interval
514 timer of 0 is entered, the interval timer is switched off. In
515 that case a new sample can only be triggered manually by pressing
516 the key 't'.
517 The interval can be modified in text mode and bar graph mode.
518 t Trigger a new sample manually. This key can be pressed if the
520 current sample should be finished before the timer has exceeded,
521 or if no timer is set at all (interval timer defined as 0). In
522 the latter case pcp-atop can be used as a stopwatch to measure the
523 load being caused by a particular application transaction, without
524 knowing on beforehand how many seconds this transaction will last.
525 This key can be used in text mode and bar graph mode.
526 When viewing the contents of an archive folio, this key can be
528 used to show the next sample from the folio.
529 T When viewing the contents of an archive folio, this key can be
531 used to show the previous sample from the folio.
532 This key can be used in text mode and bar graph mode.
533 b When viewing the contents of an archive folio, this key can be
535 used to move to a certain timestamp within the file (either for‐
536 ward or backward).
537 This key can be used in text mode and bar graph mode.
538 r Reset all counters to zero to see the system and process activity
540 since boot again.
541 This key can be used in text mode and bar graph mode.
542 When viewing the contents of an archive, this key can be used to
544 rewind to the beginning of the file again.
545 U Specify a search string for specific user names as a regular ex‐
547 pression. From now on, only (active) processes will be shown from
548 a user which matches the regular expression. The system statis‐
549 tics are still system wide. If the Enter-key is pressed without
550 specifying a name, (active) processes of all users will be shown
551 again.
552 Whether this key is active or not can be seen in the header line.
553 I Specify a list with one or more PIDs to be selected. From now on,
555 only processes will be shown with a PID which matches one of the
556 given list. The system statistics are still system wide. If the
557 Enter-key is pressed without specifying a PID, all (active) pro‐
558 cesses will be shown again.
559 Whether this key is active or not can be seen in the header line.
560 P Specify a search string for specific process names as a regular
562 expression. From now on, only processes will be shown with a name
563 which matches the regular expression. The system statistics are
564 still system wide. If the Enter-key is pressed without specifying
565 a name, all (active) processes will be shown again.
566 Whether this key is active or not can be seen in the header line.
567 / Specify a specific command line search string as a regular expres‐
569 sion. From now on, only processes will be shown with a command
570 line which matches the regular expression. The system statistics
571 are still system wide. If the Enter-key is pressed without speci‐
572 fying a string, all (active) processes will be shown again.
573 Whether this key is active or not can be seen in the header line.
574 J Specify a Docker container id of 12 (hexadecimal) characters.
576 From now on, only processes will be shown that run in that specif‐
577 ic Docker container (CID). The system statistics are still system
578 wide. If the Enter-key is pressed without specifying a container
579 id, all (active) processes will be shown again.
580 Whether this key is active or not can be seen in the header line.
581 Q Specify a comma-separated list of process state characters. From
583 now on, only processes will be shown that are in those specific
584 process states. Accepted states are: R (running), S (sleeping), D
585 (disk sleep), T (stopped), t (tracing stop), X (dead), Z (zombie)
586 and P (parked). The system statistics are still system wide. If
587 the Enter-key is pressed without specifying a state, all (active)
588 processes will be shown again.
589 Whether this key is active or not can be seen in the header line.
590 S Specify search strings for specific logical volume names, specific
592 disk names and specific network interface names. All search
593 strings are interpreted as a regular expressions. From now on,
594 only those system resources are shown that match the concerning
595 regular expression. If the Enter-key is pressed without specify‐
596 ing a search string, all (active) system resources of that type
597 will be shown again.
598 Whether this key is active or not can be seen in the header line.
599 a The 'all/active' key can be used to toggle between only show‐
601 ing/accumulating the processes that were active during the last
602 interval (default) or showing/accumulating all processes.
603 Whether this key is active or not can be seen in the header line.
604 G By default, pcp-atop shows/accumulates the processes that are
606 alive and the processes that are exited during the last interval.
607 With this key (toggle), showing/accumulating the processes that
608 are exited can be suppressed.
609 Whether this key is active or not can be seen in the header line.
610 f Show a fixed (maximum) number of header lines for system resources
612 (toggle). By default only the lines are shown about system re‐
613 sources (CPUs, paging, logical volumes, disks, network interfaces)
614 that really have been active during the last interval. With this
615 key you can force pcp-atop to show lines of inactive resources as
616 well.
617 Whether this key is active or not can be seen in the header line.
618 F Suppress sorting of system resources (toggle). By default system
620 resources (CPUs, logical volumes, disks, network interfaces) are
621 sorted on utilization.
622 Whether this key is active or not can be seen in the header line.
623 1 Show relevant counters as an average per second (in the format
625 '..../s') instead of as a total during the interval (toggle).
626 Whether this key is active or not can be seen in the header line.
627 l Limit the number of system level lines for the counters per-cpu,
629 the active disks and the network interfaces. By default lines are
630 shown of all CPUs, disks and network interfaces which have been
631 active during the last interval. Limiting these lines can be use‐
632 ful on systems with huge number CPUs, disks or interfaces in order
633 to be able to run pcp-atop on a screen/window with e.g. only 24
634 lines.
635 For all mentioned resources the maximum number of lines can be
636 specified interactively. When using the flag -l the maximum number
637 of per-cpu lines is set to 0, the maximum number of disk lines to
638 5 and the maximum number of interface lines to 3. These values
639 can be modified again in interactive mode.
640 k Send a signal to an active process (a.k.a. kill a process).
642 q Quit the program.
644 This key can be used in text mode and bar graph mode.
645 PgDn Show the next page of the process/thread list.
647 With the arrow-down key the list can be scrolled downwards with
648 single lines.
649 ^F Show the next page of the process/thread list (forward).
651 With the arrow-down key the list can be scrolled downwards with
652 single lines.
653 PgUp Show the previous page of the process/thread list.
655 With the arrow-up key the list can be scrolled upwards with single
656 lines.
657 ^B Show the previous page of the process/thread list (backward).
659 With the arrow-up key the list can be scrolled upwards with single
660 lines.
661 ^L Redraw the screen.
663
665 In order to store system and process level statistics for long-term
666 analysis (e.g. to check the system load and the active processes run‐
667 ning yesterday between 3:00 and 4:00 PM), pcp-atop can store the system
668 and process level statistics in the PCP archive format, as an archive
669 folio (see mkaf(1)).
670 All information about processes and threads is stored in the archive.
671 The interval (default: 10 seconds) and number of samples (default: in‐
672 finite) can be passed as last arguments. Instead of the number of sam‐
673 ples, the flag -S can be used to indicate that pcp-atop should finish
674 anyhow before midnight.
675 A PCP archive can be read and visualized again with the -r option. The
677 argument is a comma-separated list of names, each of which may be the
678 base name of an archive or the name of a directory containing one or
679 more archives. If no argument is specified, the file $PCP_LOG_DIR/pm‐
680 logger/HOST/YYYYMMDD is opened for input (where YYYYMMDD are digits
681 representing the current date, and HOST is the hostname of the machine
682 being logged). If a filename is specified in the format YYYYMMDD (rep‐
683 resenting any valid date), the file $PCP_LOG_DIR/pmlogger/HOST/YYYYMMDD
684 is opened. If a filename with the symbolic name y is specified, yes‐
685 terday's daily logfile is opened (this can be repeated so 'yyyy' indi‐
686 cates the logfile of four days ago).
687 The samples from the file can be viewed interactively by using the key
688 't' to show the next sample, the key 'T' to show the previous sample,
689 the key 'b' to branch to a particular time or the key 'r' to rewind to
690 the beginning of the file. These keys can be used in text mode as well
691 as in bar graph mode.
692 When output is redirected to a file or pipe, pcp-atop prints all sam‐
693 ples in plain ASCII. The default line length is 80 characters in that
694 case. With the flag -L followed by an alternate line length, more (or
695 less) columns will be shown.
696 With the flag -b (begin time) and/or -e (end time) followed by a time
697 argument of the form [YY-MM-DD] HH:MM, a certain time period within the
698 archive can be selected.
699
701 The first sample shows the system level activity since boot (the
702 elapsed time in the header shows the time since boot).
703 In text mode, pcp-atop first shows the lines related to system level
705 activity for every sample. If a particular system resource has not
706 been used during the interval, the entire line related to this resource
707 is suppressed. So the number of system level lines may vary for each
708 sample.
709 After that a list is shown of processes which have been active during
710 the last interval. This list is by default sorted on CPU consumption,
711 but this order can be changed by the keys which are previously de‐
712 scribed.
713 If values have to be shown by pcp-atop which do not fit in the column
715 width, another format is used. If e.g. a CPU consumption of 233216 mil‐
716 liseconds should be shown in a column width of 4 positions, it is shown
717 as '233s' (in seconds). For large memory figures, another unit is cho‐
718 sen if the value does not fit (Mb instead of Kb, Gb instead of Mb, Tb
719 instead of Gb, etcetera). For other values, a kind of exponent nota‐
720 tion is used (value 123456789 shown in a column of 5 positions gives
721 123e6).
722
724 The system level information in text mode consists of the following
725 output lines:
726 PRC Process and thread level totals.
728 This line contains the total CPU time consumed in system mode
729 ('sys') and in user mode ('user'), the total number of processes
730 present at this moment ('#proc'), the total number of threads
731 present at this moment in state 'running' ('#trun'), ´sleeping in‐
732 terruptible' ('#tslpi') and 'sleeping uninterruptible' ('#tslpu'),
733 the number of zombie processes ('#zombie'), the number of clone
734 system calls ('clones'), and the number of processes that ended
735 during the interval ('#exit') when process accounting is used. In‐
736 stead of '#exit' the last column may indicate that process ac‐
737 counting could not be activated ('no procacct').
738 If the screen width does not allow all of these counters, only a
739 relevant subset is shown.
740 CPU CPU utilization.
742 At least one line is shown for the total occupation of all CPUs
743 together.
744 In case of a multi-processor system, an additional line is shown
745 for every individual processor (with 'cpu' in lower case), sorted
746 on activity. Inactive CPUs will not be shown by default. The
747 lines showing the per-cpu occupation contain the CPU number in the
748 field combined with the wait percentage.
749 Every line contains the percentage of CPU time spent in kernel
751 mode by all active processes ('sys'), the percentage of cpu time
752 consumed in user mode ('user') for all active processes (including
753 processes running with a nice value larger than zero), the per‐
754 centage of CPU time spent for interrupt handling ('irq') including
755 softirq, the percentage of unused CPU time while no processes were
756 waiting for disk I/O ('idle'), and the percentage of unused CPU
757 time while at least one process was waiting for disk I/O ('wait').
758 In case of per-cpu occupation, the CPU number and the wait per‐
759 centage ('w') for that CPU. The number of lines showing the per-
760 cpu occupation can be limited.
761 For virtual machines, the steal-percentage ('steal') shows the
763 percentage of CPU time stolen by other virtual machines running on
764 the same hardware.
765 For physical machines hosting one or more virtual machines, the
766 guest-percentage ('guest') shows the percentage of CPU time used
767 by the virtual machines. Notice that this percentage overlaps the
768 user-percentage!
769 When PMC performance monitoring counters are supported by the CPU
771 and the kernel (and pmdaperfevent(1) runs with root privileges),
772 the number of instructions per CPU cycle ('ipc') is shown. The
773 first sample always shows the value 'initial', because the coun‐
774 ters are just activated at the moment that pcp-atop is started.
775 When the CPU busy percentage is high and the IPC is less than 1.0,
776 it is likely that the CPU is frequently waiting for memory access
777 during instruction execution (larger CPU caches or faster memory
778 might be helpful to improve performance). When the CPU busy per‐
779 centage is high and the IPC is greater than 1.0, it is likely that
780 the CPU is instruction-bound (more/faster cores might be helpful
781 to improve performance).
782 Furthermore, per CPU the effective number of cycles ('cycl') is
783 shown. This value can reach the current CPU frequency if such CPU
784 is 100% busy. When an idle CPU is halted, the number of effective
785 cycles can be (considerably) lower than the current frequency.
786 Notice that the average instructions per cycle and number of cy‐
787 cles is shown in the CPU line for all CPUs.
788 See also: http://www.brendangregg.com/blog/2017-05-09/cpu-utiliza‐
789 tion-is-wrong.html
790 In case of frequency scaling, all previously mentioned CPU per‐
792 centages are relative to the used scaling of the CPU during the
793 interval. If a CPU has been active for e.g. 50% in user mode dur‐
794 ing the interval while the frequency scaling of that CPU was 40%,
795 only 20% of the full capacity of the CPU has been used in user
796 mode.
797 In case that the kernel module 'cpufreq_stats' is active (after
798 issuing 'modprobe cpufreq_stats'), the average frequency ('avgf')
799 and the average scaling percentage ('avgscal') is shown. Otherwise
800 the current frequency ('curf') and the current scaling percentage
801 ('curscal') is shown at the moment that the sample is taken. No‐
802 tice that average values for frequency and scaling are shown in
803 the CPU line for every CPU.
804 Frequency scaling statistics are only gathered for systems with
805 maximum 8 CPUs, since gathering of these values per CPU is very
806 time consuming.
807 If the screen-width does not allow all of these counters, only a
809 relevant subset is shown.
810 CPL CPU load information.
812 This line contains the load average figures reflecting the number
813 of threads that are available to run on a CPU (i.e. part of the
814 runqueue) or that are waiting for disk I/O. These figures are av‐
815 eraged over 1 ('avg1'), 5 ('avg5') and 15 ('avg15') minutes.
816 Furthermore the number of context switches ('csw'), the number of
817 serviced interrupts ('intr') and the number of available CPUs are
818 shown.
819 If the screen-width does not allow all of these counters, only a
821 relevant subset is shown.
822 GPU GPU utilization (Nvidia).
824 Read the section GPU STATISTICS GATHERING in this document to find
825 the details about the activation of the pmdanvidia daemon.
826 In the first column of every line, the bus-id (last nine charac‐
828 ters) and the GPU number are shown. The subsequent columns show
829 the percentage of time that one or more kernels were executing on
830 the GPU ('gpubusy'), the percentage of time that global (device)
831 memory was being read or written ('membusy'), the occupation per‐
832 centage of memory ('memocc'), the total memory ('total'), the mem‐
833 ory being in use at the moment of the sample ('used'), the average
834 memory being in use during the sample time ('usavg'), the number
835 of processes being active on the GPU at the moment of the sample
836 ('#proc'), and the type of GPU.
837 If the screen-width does not allow all of these counters, only a
839 relevant subset is shown.
840 The number of lines showing the GPUs can be limited.
841 MEM Memory occupation (two lines).
843 These lines contain the total amount of physical memory ('tot'),
844 the amount of memory which is currently free ('free'), the amount
845 of memory in use as page cache including the total resident shared
846 memory ('cache'), the amount of memory within the page cache that
847 has to be flushed to disk ('dirty'), the amount of memory used for
848 filesystem meta data ('buff'), the amount of memory being used for
849 kernel mallocs ('slab'), the amount of slab memory that is re‐
850 claimable ('slrec'), the resident size of shared memory including
851 tmpfs ('shmem'), the resident size of shared memory ('shrss') the
852 amount of shared memory that is currently swapped ('shswp'), the
853 amount of memory that is currently used for page tables ('pgtab'),
854 the number of NUMA nodes in this system ('numnode'), the amount of
855 memory that is currently claimed by vmware's balloon driver ('vm‐
856 bal'), the amount of memory that is currently claimed by the ARC
857 (cache) of ZFSonlinux ('zfarc'), the amount of memory that is
858 claimed for huge pages ('hptot'), the amount of huge page memory
859 that is really in use ('hpuse'), the amount of memory that is used
860 for TCP sockets ('tcps'), and the amount of memory that is used
861 for UDP sockets ('udps').
862 If the screen-width does not allow all of these counters, only a
864 relevant subset is shown.
865 SWP Swap occupation and overcommit info.
867 This line contains the total amount of swap space on disk ('tot')
868 and the amount of free swap space ('free'), the size of the swap
869 cache ('swcac'), the total size of compressed storage in zswap
870 ('zpool'), the total size of the compressed pages stored in zswap
871 ('zstor'), the total size of the memory used for KSM ('ksuse',
872 i.e. shared), and the total size of the memory saved (deduped) by
873 KSM ('kssav', i.e. sharing).
874 Furthermore the committed virtual memory space ('vmcom') and the
875 maximum limit of the committed space ('vmlim', which is by default
876 swap size plus 50% of memory size) is shown. The committed space
877 is the reserved virtual space for all allocations of private memo‐
878 ry space for processes. The kernel only verifies whether the com‐
879 mitted space exceeds the limit if strict overcommit handling is
880 configured (vm.overcommit_memory is 2).
881 LLC Last-Level Cache of CPU info.
883 This line contains the total memory bandwidth of LLC ('tot'), the
884 bandwidth of the local NUMA node ('loc'), and the percentage of
885 LLC in use ('LLCXX YY%').
886 Note that this feature depends on the 'resctrl' pseudo filesystem.
888 Be sure that the kernel is built with the relevant config and take
889 care that the pseudo-filesystem is mounted:
890 mount -t resctrl resctrl -o mba_MBps /sys/fs/resctrl (on Intel)
892 mount -t resctrl resctrl -o cdp /sys/fs/resctrl (on AMD)
893 NUM Memory utilization per NUMA node (not shown for single NUMA node).
895 This line shows the total amount of physical memory of this node
896 ('tot'), the amount of free memory ('free'), the amount of memory
897 for cached file data ('file'), modified cached file data
898 ('dirty'), recently used memory ('activ'), less recently used mem‐
899 ory ('inact'), memory being used for kernel mallocs ('slab'), the
900 amount of slab memory that is reclaimable ('slrec'), shared memory
901 including tmpfs ('shmem'), total huge pages ('hptot') and the
902 fragmentation percentage ('frag').
903 NUC CPU utilization per NUMA node (not shown for single NUMA node).
905 This line shows the utilization percentages of all CPUs related to
906 this NUMA node, categorized for system mode ('sys'), user mode
907 ('user'), user mode for niced processes ('niced'), idle mode
908 ('idle'), wait mode ('w' preceded by the node number), irq mode
909 ('irq'), softirq mode ('sirq'), steal mode ('steal'), and guest
910 mode ('guest') overlapping user mode.
911 PAG Paging frequency.
913 This line contains the number of scanned pages ('scan') due to the
914 fact that free memory drops below a particular threshold, the num‐
915 ber of times that the kernel tries to reclaim pages due to an ur‐
916 gent need ('stall'), the number of process stalls to run memory
917 compaction to allocate huge pages ('compact'), the number of NUMA
918 pages migrated ('numamig'), and the total number of memory pages
919 migrated successfully e.g. between NUMA nodes or for compaction
920 ('migrate') are shown.
921 Also the number of memory pages the system read from block devices
922 ('pgin'), the number of memory pages the system wrote to block de‐
923 vices ('pgout'), the number of memory pages the system read from
924 swap space ('swin'), the number of memory pages the system wrote
925 to swap space ('swout'), and the number of out-of-memory kills
926 ('oomkill').
927 PSI Pressure Stall Information.
929 This line contains percentages about resource pressure related to
930 CPU, memory and I/O. Certain percentages refer to 'some' meaning
931 that some processes/threads were delayed due to resource overload.
932 Other percentages refer to 'full' meaning a loss of overall
933 throughput due to resource overload.
934 The values 'cpusome', 'memsome', 'memfull', 'iosome' and 'iofull'
935 show the pressure percentage during the entire interval.
936 The values 'cs' (cpu some), 'ms' (memory some), 'mf' (memory
937 full), ´is' (I/O some) and 'if' (I/O full) each show three per‐
938 centages separated by slashes: pressure percentage over the last
939 10, 60 and 300 seconds.
940 LVM/MDD/DSK
942 Logical volume/multiple device/disk utilization.
943 Per active unit one line is produced, sorted on unit activity.
944 Such line shows the name (e.g. VolGroup00-lvtmp for a logical vol‐
945 ume or sda for a hard disk), the percentage of elapsed time during
946 which I/O requests were issued to the device ('busy') (note that
947 for devices serving requests in parallel, such as RAID arrays, SSD
948 and NVMe, this number does not reflect their performance limits),
949 the number of read requests issued ('read'), the number of write
950 requests issued ('write'), the number of discard requests issued
951 ('discrd') if supported by kernel version, the number of KiBytes
952 per read ('KiB/r'), the number of KiBytes per write ('KiB/w'), the
953 number of KiBytes per discard ('KiB/d') if supported by kernel
954 version, the number of MiBytes per second throughput for reads
955 ('MBr/s'), the number of MiBytes per second throughput for writes
956 ('MBw/s'), requests issued to the device driver but not completed
957 ('inflt'), the average queue depth ('avq') and the average number
958 of milliseconds needed by a request ('avio') for seek, latency and
959 data transfer.
960 If the screen-width does not allow all of these counters, only a
961 relevant subset is shown.
962 The number of lines showing the units can be limited per class
964 (LVM, MDD or DSK) with the 'l' key or statically (see separate
965 man-page of pcp-atoprc(5)). By specifying the value 0 for a par‐
966 ticular class, no lines will be shown any more for that class.
967 NFM Network Filesystem (NFS) mount at the client side.
969 For each NFS-mounted filesystem, a line is shown that contains the
970 mounted server directory, the name of the server ('srv'), the to‐
971 tal number of bytes physically read from the server ('read') and
972 the total number of bytes physically written to the server
973 ('write'). Data transfer is subdivided in the number of bytes
974 read via normal read() system calls ('nread'), the number of bytes
975 written via normal read() system calls ('nwrit'), the number of
976 bytes read via direct I/O ('dread'), the number of bytes written
977 via direct I/O ('dwrit'), the number of bytes read via memory
978 mapped I/O pages ('mread'), and the number of bytes written via
979 memory mapped I/O pages ('mwrit').
980 NFC Network Filesystem (NFS) client side counters.
982 This line contains the number of RPC calls issues by local pro‐
983 cesses ('rpc'), the number of read RPC calls ('read') and write
984 RPC calls ('rpwrite') issued to the NFS server, the number of RPC
985 calls being retransmitted ('retxmit') and the number of authoriza‐
986 tion refreshes ('autref').
987 NFS Network Filesystem (NFS) server side counters.
989 This line contains the number of RPC calls received from NFS
990 clients ('rpc'), the number of read RPC calls received ('cread'),
991 the number of write RPC calls received ('cwrit'), the number of
992 Megabytes/second returned to read requests by clients ('MBcr/s'),
993 the number of Megabytes/second passed in write requests by clients
994 ('MBcw/s'), the number of network requests handled via TCP
995 ('nettcp'), the number of network requests handled via UDP ('ne‐
996 tudp'), the number of reply cache hits ('rchits'), the number of
997 reply cache misses ('rcmiss') and the number of uncached requests
998 ('rcnoca'). Furthermore some error counters indicating the number
999 of requests with a bad format ('badfmt') or a bad authorization
1000 ('badaut'), and a counter indicating the number of bad clients
1001 ('badcln').
1002 NET Network utilization (TCP/IP).
1004 One line is shown for activity of the transport layer (TCP and
1005 UDP), one line for the IP layer and one line per active interface.
1006 For the transport layer, counters are shown concerning the number
1007 of received TCP segments including those received in error
1008 ('tcpi'), the number of transmitted TCP segments excluding those
1009 containing only retransmitted octets ('tcpo'), the number of UDP
1010 datagrams received ('udpi'), the number of UDP datagrams transmit‐
1011 ted ('udpo'), the number of active TCP opens ('tcpao'), the number
1012 of passive TCP opens ('tcppo'), the number of TCP output retrans‐
1013 missions ('tcprs'), the number of TCP input errors ('tcpie'), the
1014 number of TCP output resets ('tcpor'), the number of UDP no ports
1015 ('udpnp'), and the number of UDP input errors ('udpie').
1016 If the screen-width does not allow all of these counters, only a
1017 relevant subset is shown.
1018 These counters are related to IPv4 and IPv6 combined.
1019 For the IP layer, counters are shown concerning the number of IP
1021 datagrams received from interfaces, including those received in
1022 error ('ipi'), the number of IP datagrams that local higher-layer
1023 protocols offered for transmission ('ipo'), the number of received
1024 IP datagrams which were forwarded to other interfaces ('ipfrw'),
1025 the number of IP datagrams which were delivered to local higher-
1026 layer protocols ('deliv'), the number of received ICMP datagrams
1027 ('icmpi'), and the number of transmitted ICMP datagrams ('icmpo').
1028 If the screen-width does not allow all of these counters, only a
1029 relevant subset is shown.
1030 These counters are related to IPv4 and IPv6 combined.
1031 For every active network interface one line is shown, sorted on
1033 the interface activity. Such line shows the name of the interface
1034 and its busy percentage in the first column. The busy percentage
1035 for half duplex is determined by comparing the interface speed
1036 with the number of bits transmitted and received per second; for
1037 full duplex the interface speed is compared with the highest of
1038 either the transmitted or the received bits. When the interface
1039 speed can not be determined (e.g. for the loopback interface),
1040 '---' is shown instead of the percentage.
1041 Furthermore the number of received packets ('pcki'), the number of
1042 transmitted packets ('pcko'), the line speed of the interface
1043 ('sp'), the effective amount of bits received per second ('si'),
1044 the effective amount of bits transmitted per second ('so'), the
1045 number of collisions ('coll'), the number of received multicast
1046 packets ('mlti'), the number of errors while receiving a packet
1047 ('erri'), the number of errors while transmitting a packet ('er‐
1048 ro'), the number of received packets dropped ('drpi'), and the
1049 number of transmitted packets dropped ('drpo').
1050 If the screen-width does not allow all of these counters, only a
1051 relevant subset is shown.
1052 The number of lines showing the network interfaces can be limited.
1053 IFB Infiniband utilization.
1055 For every active Infiniband port one line is shown, sorted on ac‐
1056 tivity. Such line shows the name of the port and its busy per‐
1057 centage in the first column. The busy percentage is determined by
1058 taking the highest of either the transmitted or the received bits
1059 during the interval, multiplying that value by the number of lanes
1060 and comparing it against the maximum port speed.
1061 Furthermore the number of received packets divided by the number
1062 of lanes ('pcki'), the number of transmitted packets divided by
1063 the number of lanes ('pcko'), the maximum line speed ('sp'), the
1064 effective amount of bits received per second ('si'), the effective
1065 amount of bits transmitted per second ('so'), and the number of
1066 lanes ('lanes').
1067 If the screen-width does not allow all of these counters, only a
1068 relevant subset is shown.
1069 The number of lines showing the Infiniband ports can be limited.
1070
1072 Following the system level information, a list of processes is shown in
1073 text mode from which the resource utilization has changed during the
1074 last interval. These processes might have used CPU time or issued disk
1075 or network requests. However a process is also shown if part of it has
1076 been paged out due to lack of memory (while the process itself was in
1077 sleep state).
1078 Per process the following fields may be shown (in alphabetical order),
1080 depending on the current output mode as described in the section INTER‐
1081 ACTIVE COMMANDS and depending on the current width of your window:
1082 AVGRSZ The average size of one read-action on disk.
1084 AVGWSZ The average size of one write-action on disk.
1086 BANDWI Total bandwidth for received TCP and UDP packets consumed by
1088 this process (bits-per-second). This value can be compared
1089 with the value 'si' on interface level (used bandwidth per in‐
1090 terface).
1091 This information will only be shown when the pmdabcc(1) module
1092 'netproc' has been installed.
1093 BANDWO Total bandwidth for sent TCP and UDP packets consumed by this
1095 process (bits-per-second). This value can be compared with
1096 the value 'so' on interface level (used bandwidth per inter‐
1097 face).
1098 This information will only be shown when the pmdabcc(1) module
1099 'netproc' has been installed.
1100 BDELAY Aggregated block I/O delay, i.e. time waiting for disk I/O.
1102 CGROUP Path name of the cgroup (version 2) to which this process be‐
1104 longs. This path name is relative to the cgroup root directo‐
1105 ry, which is usually '/sys/fs/cgroup'.
1106 CID Container ID of 12 hexadecimal digits, referring to the con‐
1108 tainer in which the process/thread is running. If a process
1109 has been started and finished during the last interval, a '?'
1110 is shown because the container ID is not part of the standard
1111 process accounting record.
1112 CMD The name of the process. This name can be surrounded by
1114 "less/greater than" signs ('<name>') which means that the
1115 process has finished during the last interval.
1116 Behind the abbreviation 'CMD' in the header line, the current
1117 page number and the total number of pages of the
1118 process/thread list are shown.
1119 COMMAND-LINE
1121 The full command line of the process (including arguments). If
1122 the length of the command line exceeds the length of the
1123 screen line, the arrow keys -> and <- can be used for horizon‐
1124 tal scroll.
1125 Behind the verb 'COMMAND-LINE' in the header line, the current
1126 page number and the total number of pages of the
1127 process/thread list are shown.
1128 CPU The occupation percentage of this process related to the
1130 available capacity for this resource on system level.
1131 CPUNR The identification of the CPU the (main) thread is running on
1133 or has recently been running on.
1134 CTID Container ID (OpenVZ). If a process has been started and fin‐
1136 ished during the last interval, a '?' is shown because the
1137 container ID is not part of the standard process accounting
1138 record.
1139 DSK The occupation percentage of this process related to the total
1141 load that is produced by all processes (i.e. total disk ac‐
1142 cesses by all processes during the last interval).
1143 This information is shown when per process "storage account‐
1144 ing" is active in the kernel.
1145 EGID Effective group-id under which this process executes.
1147 ENDATE Date that the process has been finished. If the process is
1149 still running, this field shows 'active'.
1150 ENTIME Time that the process has been finished. If the process is
1152 still running, this field shows 'active'.
1153 ENVID Virtual environment identified (OpenVZ only).
1155 EUID Effective user-id under which this process executes.
1157 EXC The exit code of a terminated process (second position of col‐
1159 umn 'ST' is E) or the fatal signal number (second position of
1160 column 'ST' is S or C).
1161 FSGID Filesystem group-id under which this process executes.
1163 FSUID Filesystem user-id under which this process executes.
1165 GPU When the pmdanvidia daemon does not run with root privileges,
1167 the GPU percentage reflects the GPU memory occupation percent‐
1168 age (memory of all GPUs is 100%).
1169 When the pmdanvidia daemon runs with root privileges, the GPU
1170 percentage reflects the GPU busy percentage.
1171 GPUBUSY Busy percentage on all GPUs (one GPU is 100%).
1173 When the pmdanvidia daemon does not run with root privileges,
1174 this value is not available.
1175 GPUNUMS Comma-separated list of GPUs used by the process during the
1177 interval. When the comma-separated list exceeds the width of
1178 the column, a hexadecimal value is shown.
1179 LOCKSZ The virtual amount of memory being locked (i.e. non-swappable)
1181 by this process (or user).
1182 MAJFLT The number of page faults issued by this process that have
1184 been solved by creating/loading the requested memory page.
1185 MEM The occupation percentage of this process related to the
1187 available capacity for this resource on system level.
1188 MEMAVG Average memory occupation during the interval on all used
1190 GPUs.
1191 MEMBUSY Busy percentage of memory on all GPUs (one GPU is 100%), i.e.
1193 the time needed for read and write accesses on memory.
1194 When the pmdanvidia daemon does not run with root privileges,
1195 this value is not available.
1196 MEMNOW Memory occupation at the moment of the sample on all used
1198 GPUs.
1199 MINFLT The number of page faults issued by this process that have
1201 been solved by reclaiming the requested memory page from the
1202 free list of pages.
1203 NET The occupation percentage of this process related to the total
1205 load that is produced by all processes (i.e. consumed network
1206 bandwidth of all processes during the last interval).
1207 This information will only be shown when the pmdabcc(1) module
1208 'netproc' has been installed.
1209 NICE The more or less static priority that can be given to a
1211 process on a scale from -20 (high priority) to +19 (low prior‐
1212 ity).
1213 NIVCSW Number of times the process/thread was context-switched invol‐
1215 untarily, in case that the time slice expired.
1216 NPROCS The number of active and terminated processes accumulated for
1218 this user or program.
1219 NVCSW Number of times that the process/thread was context-switched
1221 voluntarily in case of a blocking system call, e.g. to wait
1222 for an I/O operation to complete.
1223 PID Process-id.
1225 POLI The policies 'norm' (normal, which is SCHED_OTHER), 'btch'
1227 (batch) and 'idle' refer to timesharing processes. The poli‐
1228 cies 'fifo' (SCHED_FIFO) and 'rr' (round robin, which is
1229 SCHED_RR) refer to realtime processes.
1230 PPID Parent process-id.
1232 PRI The process' priority ranges from 0 (highest priority) to 139
1234 (lowest priority). Priority 0 to 99 are used for realtime
1235 processes (fixed priority independent of their behavior) and
1236 priority 100 to 139 for timesharing processes (variable prior‐
1237 ity depending on their recent CPU consumption and the nice
1238 value).
1239 PSIZE The proportional memory size of this process (or user).
1241 Every process shares resident memory with other processes.
1242 E.g. when a particular program is started several times, the
1243 code pages (text) are only loaded once in memory and shared by
1244 all incarnations. Also the code of shared libraries is shared
1245 by all processes using that shared library, as well as shared
1246 memory and memory-mapped files. For the PSIZE calculation of
1247 a process, the resident memory of a process that is shared
1248 with other processes is divided by the number of sharers.
1249 This means, that every process is accounted for a proportional
1250 part of that memory. Accumulating the PSIZE values of all
1251 processes in the system gives a reliable impression of the to‐
1252 tal resident memory consumed by all processes.
1253 Since gathering of all values that are needed to calculate the
1254 PSIZE is a very time-consuming task, the 'R' key (or '-R'
1255 flag) should be active. Gathering these values also requires
1256 superuser privileges (otherwise '?K' is shown in the output).
1257 RDDSK The read data transfer issued physically on disk (so reading
1259 from the disk cache is not accounted for).
1260 Unfortunately, the kernel aggregates the data transfer of a
1261 process to the data transfer of its parent process when termi‐
1262 nating, so you might see transfers for (parent) processes like
1263 cron, bash or init, that are not really issued by them.
1264 RDELAY Runqueue delay, i.e. time spent waiting on a runqueue.
1266 RGID The real group-id under which the process executes.
1268 RGROW The amount of resident memory that the process has grown dur‐
1270 ing the last interval. A resident growth can be caused by
1271 touching memory pages which were not physically created/loaded
1272 before (load-on-demand). Note that a resident growth can also
1273 be negative e.g. when part of the process is paged out due to
1274 lack of memory or when the process frees dynamically allocated
1275 memory. For a process which started during the last interval,
1276 the resident growth reflects the total resident size of the
1277 process at that moment.
1278 RSIZE The total resident memory usage consumed by this process (or
1280 user). Notice that the RSIZE of a process includes all resi‐
1281 dent memory used by that process, even if certain memory parts
1282 are shared with other processes (see also the explanation of
1283 PSIZE).
1284 RTPR Realtime priority according the POSIX standard. Value can be
1286 0 for a timesharing process (policy 'norm', 'btch' or 'idle')
1287 or ranges from 1 (lowest) till 99 (highest) for a realtime
1288 process (policy 'rr' or 'fifo').
1289 RUID The real user-id under which the process executes.
1291 S The current state of the (main) thread: 'R' for running (cur‐
1293 rently processing or in the runqueue), 'S' for sleeping inter‐
1294 ruptible (wait for an event to occur), 'D' for sleeping non-
1295 interruptible, 'Z' for zombie (waiting to be synchronized with
1296 its parent process), 'T' for stopped (suspended or traced),
1297 'W' for swapping, and 'E' (exit) for processes which have fin‐
1298 ished during the last interval.
1299 SGID The saved group-id of the process.
1301 ST The status of a process.
1303 The first position indicates if the process has been started
1304 during the last interval (the value N means 'new process').
1305 The second position indicates if the process has been finished
1307 during the last interval.
1308 The value E means 'exit' on the process' own initiative; the
1309 exit code is displayed in the column 'EXC'.
1310 The value S means that the process has been terminated unvol‐
1311 untarily by a signal; the signal number is displayed in the in
1312 the column 'EXC'.
1313 The value C means that the process has been terminated unvol‐
1314 untarily by a signal, producing a core dump in its current di‐
1315 rectory; the signal number is displayed in the column 'EXC'.
1316 STDATE The start date of the process.
1318 STTIME The start time of the process.
1320 SUID The saved user-id of the process.
1322 SWAPSZ The swap space consumed by this process (or user).
1324 SYSCPU CPU time consumption of this process in system mode (kernel
1326 mode), usually due to system call handling.
1327 TCPRASZ The average size of a received TCP buffer in bytes. This in‐
1329 formation will only be shown when the BCC PMDA is active and
1330 the 'netproc' module is enabled.
1331 TCPRCV The number of tcp_recvmsg()/tcp_cleanup_rbuf() calls from this
1333 process. This information will only be shown when the BCC PM‐
1334 DA is active and the 'netproc' module is enabled.
1335 TCPSASZ The average size of a TCP buffer requested to be transmitted
1337 in bytes. This information will only be shown when the BCC
1338 PMDA is active and the 'netproc' module is enabled.
1339 TCPSND The number of tcp_sendmsg() calls from this process. This in‐
1341 formation will only be shown when the BCC PMDA is active and
1342 the 'netproc' module is enabled.
1343 THR Total number of threads within this process. All related
1345 threads are contained in a thread group, represented by pcp-
1346 atop as one line or as a separate line when the 'y' key (or -y
1347 flag) is active.
1348 TID Thread-id. All threads within a process run with the same PID
1350 but with a different TID. This value is shown for individual
1351 threads in multi-threaded processes (when using the key 'y').
1352 TRUN Number of threads within this process that are in the state
1354 'running' (R).
1355 TSLPI Number of threads within this process that are in the state
1357 'interruptible sleeping' (S).
1358 TSLPU Number of threads within this process that are in the state
1360 'uninterruptible sleeping' (D).
1361 UDPRASZ The average size of a received UDP buffer in bytes. This in‐
1363 formation will only be shown when the BCC PMDA is active and
1364 the 'netproc' module is enabled.
1365 UDPRCV The number of udp_recvmsg()/skb_consume_udp() calls from this
1367 process. This information will only be shown when the BCC PM‐
1368 DA is active and the 'netproc' module is enabled.
1369 UDPSASZ The average size of a UDP buffer requested to be transmitted
1371 in bytes. This information will only be shown when the BCC
1372 PMDA is active and the 'netproc' module is enabled.
1373 UDPSND The number of udp_sendmsg() calls from this process. This in‐
1375 formation will only be shown when the BCC PMDA is active and
1376 the 'netproc' module is enabled.
1377 USRCPU CPU time consumption of this process in user mode, due to pro‐
1379 cessing the own program text.
1380 VDATA The virtual memory size of the private data used by this
1382 process (including heap and shared library data).
1383 VGROW The amount of virtual memory that the process has grown during
1385 the last interval. A virtual growth can be caused by e.g. is‐
1386 suing a malloc() or attaching a shared memory segment. Note
1387 that a virtual growth can also be negative by e.g. issuing a
1388 free() or detaching a shared memory segment. For a process
1389 which started during the last interval, the virtual growth re‐
1390 flects the total virtual size of the process at that moment.
1391 VPID Virtual process-id (within an OpenVZ container). If a process
1393 has been started and finished during the last interval, a '?'
1394 is shown because the virtual process-id is not part of the
1395 standard process accounting record.
1396 VSIZE The total virtual memory usage consumed by this process (or
1398 user).
1399 VSLIBS The virtual memory size of the (shared) text of all shared li‐
1401 braries used by this process.
1402 VSTACK The virtual memory size of the (private) stack used by this
1404 process
1405 VSTEXT The virtual memory size of the (shared) text of the executable
1407 program.
1408 WCHAN Wait channel of thread in sleep state, i.e. the name of the
1410 kernel function in which the thread has been put asleep.
1411 Since determining the name string of the kernel function is a
1412 relatively time-consuming task, the 'W' key (or '-W' flag)
1413 should be active.
1414 WRDSK The write data transfer issued physically on disk (so writing
1416 to the disk cache is not accounted for). This counter is
1417 maintained for the application process that writes its data to
1418 the cache (assuming that this data is physically transferred
1419 to disk later on). Notice that disk I/O needed for swapping
1420 is not taken into account.
1421 Unfortunately, the kernel aggregates the data transfer of a
1422 process to the data transfer of its parent process when termi‐
1423 nating, so you might see transfers for (parent) processes like
1424 cron, bash or init, that are not really issued by them.
1425 WCANCL The write data transfer previously accounted for this process
1427 or another process that has been cancelled. Suppose that a
1428 process writes new data to a file and that data is removed
1429 again before the cache buffers have been flushed to disk.
1430 Then the original process shows the written data as WRDSK,
1431 while the process that removes/truncates the file shows the
1432 unflushed removed data as WCANCL.
1433
1435 With the flag -P followed by a list of one or more labels (comma-sepa‐
1436 rated), parsable output is produced for each sample. The labels that
1437 can be specified for system-level statistics correspond to the labels
1438 (first verb of each line) that can be found in the interactive output:
1439 "CPU", "cpu", "CPL", "GPU", "MEM", "SWP", "PAG", "PSI", "LVM", "MDD",
1440 "DSK", "NFM", "NFC", "NFS", "NET", "IFB", "LLC", "NUM" and "NUC".
1441 For process-level statistics special labels are introduced: "PRG" (gen‐
1442 eral), "PRC" (CPU), "PRE" (GPU), "PRM" (memory), "PRD" (disk, only if
1443 "storage accounting" is active).
1444 With the label "ALL", all system and process level statistics are
1445 shown.
1446 The command and command line in the parsable output might contain spa‐
1448 ces and are therefore by default surrounded by parenthesis. However,
1449 since a space is often used as separator between the fields by parsing
1450 tools, with the additional flag -Z it is possible to exchange the spa‐
1451 ces in the command (line) by underscores and omit the parenthesis.
1452 For every interval all requested lines are shown whereafter pcp-atop
1454 shows a line just containing the label "SEP" as a separator before the
1455 lines for the next sample are generated.
1456 When a sample contains the values since boot, pcp-atop shows a line
1457 just containing the label "RESET" before the lines for this sample are
1458 generated.
1459 The first part of each output-line consists of the following six
1461 fields: label (the name of the label), host (the name of this machine),
1462 epoch (the time of this interval as number of seconds since 1-1-1970),
1463 date (date of this interval in format YYYY/MM/DD), time (time of this
1464 interval in format HH:MM:SS), and interval (number of seconds elapsed
1465 for this interval).
1466 The subsequent fields of each output-line depend on the label:
1468 CPU Subsequent fields: total number of clock-ticks per second for
1470 this machine, number of processors, consumption for all CPUs
1471 in system mode (clock-ticks), consumption for all CPUs in user
1472 mode (clock-ticks), consumption for all CPUs in user mode for
1473 niced processes (clock-ticks), consumption for all CPUs in
1474 idle mode (clock-ticks), consumption for all CPUs in wait mode
1475 (clock-ticks), consumption for all CPUs in irq mode (clock-
1476 ticks), consumption for all CPUs in softirq mode (clock-
1477 ticks), consumption for all CPUs in steal mode (clock-ticks),
1478 consumption for all CPUs in guest mode (clock-ticks) overlap‐
1479 ping user mode, frequency of all CPUs and frequency percentage
1480 of all CPUs.
1481 cpu Subsequent fields: total number of clock-ticks per second for
1483 this machine, processor-number, consumption for this CPU in
1484 system mode (clock-ticks), consumption for this CPU in user
1485 mode (clock-ticks), consumption for this CPU in user mode for
1486 niced processes (clock-ticks), consumption for this CPU in
1487 idle mode (clock-ticks), consumption for this CPU in wait mode
1488 (clock-ticks), consumption for this CPU in irq mode (clock-
1489 ticks), consumption for this CPU in softirq mode (clock-
1490 ticks), consumption for this CPU in steal mode (clock-ticks),
1491 consumption for this CPU in guest mode (clock-ticks) overlap‐
1492 ping user mode, frequency of all CPUs, frequency percentage of
1493 all CPUs, instructions executed by all CPUs and cycles for all
1494 CPUs.
1495 CPL Subsequent fields: number of processors, load average for last
1497 minute, load average for last five minutes, load average for
1498 last fifteen minutes, number of context-switches, and number
1499 of device interrupts.
1500 GPU Subsequent fields: GPU number, bus-id string, type of GPU
1502 string, GPU busy percentage during last second (-1 if not
1503 available), memory busy percentage during last second (-1 if
1504 not available), total memory size (KiB), used memory (KiB) at
1505 this moment, number of samples taken during interval, cumula‐
1506 tive GPU busy percentage during the interval (to be divided by
1507 the number of samples for the average busy percentage, -1 if
1508 not available), cumulative memory busy percentage during the
1509 interval (to be divided by the number of samples for the aver‐
1510 age busy percentage, -1 if not available), and cumulative mem‐
1511 ory occupation during the interval (to be divided by the num‐
1512 ber of samples for the average occupation).
1513 MEM Subsequent fields: page size for this machine (in bytes), size
1515 of physical memory (pages), size of free memory (pages), size
1516 of page cache (pages), size of buffer cache (pages), size of
1517 slab (pages), dirty pages in cache (pages), reclaimable part
1518 of slab (pages), total size of vmware's balloon pages (pages),
1519 total size of shared memory (pages), size of resident shared
1520 memory (pages), size of swapped shared memory (pages), huge
1521 page size (in bytes), total size of huge pages (huge pages),
1522 size of free huge pages (huge pages), size of ARC (cache) of
1523 ZFSonlinux (pages), size of sharing pages for KSM (pages),
1524 size of shared pages for KSM (pages), size of memory used for
1525 TCP sockets (pages), size of memory used for UDP sockets
1526 (pages), and size of pagetables (pages).
1527 SWP Subsequent fields: page size for this machine (in bytes), size
1529 of swap (pages), size of free swap (pages), size of swap cache
1530 (pages), size of committed space (pages), limit for committed
1531 space (pages), size of the swap cache (pages), size of com‐
1532 pressed pages stored in zswap (pages), and total size of com‐
1533 pressed pool in zswap (pages).
1534 LLC Subsequent fields: LLC id, percentage of LLC in use, total
1536 memory bandwidth of this LLC (in bytes), and memory bandwidth
1537 on local NUMA node of this LLC (in bytes).
1538 PAG Subsequent fields: page size for this machine (in bytes), num‐
1540 ber of page scans, number of allocstalls, 0 (future use), num‐
1541 ber of swapins, number of swapouts, number of oomkills (-1
1542 when counter not present), number of process stalls to run
1543 memory compaction, number of pages successfully migrated in
1544 total, number of NUMA pages migrated, number of pages read
1545 from block devices, and number of pages written to block de‐
1546 vices.
1547 PSI Subsequent fields: PSI statistics present on this system (n or
1549 y), CPU some avg10, CPU some avg60, CPU some avg300, CPU some
1550 accumulated microseconds during interval, memory some avg10,
1551 memory some avg60, memory some avg300, memory some accumulated
1552 microseconds during interval, memory full avg10, memory full
1553 avg60, memory full avg300, memory full accumulated microsec‐
1554 onds during interval, I/O some avg10, I/O some avg60, I/O some
1555 avg300, I/O some accumulated microseconds during interval, I/O
1556 full avg10, I/O full avg60, I/O full avg300, and I/O full ac‐
1557 cumulated microseconds during interval.
1558 LVM/MDD/DSK
1560 For every logical volume/multiple device/hard disk one line is
1561 shown.
1562 Subsequent fields: name, number of milliseconds spent for I/O,
1563 number of reads issued, number of sectors transferred for
1564 reads, number of writes issued, number of sectors transferred
1565 for write, number of discards issued (-1 if not supported),
1566 number of sectors transferred for discards, number of requests
1567 currently in flight (not yet completed), and the average queue
1568 depth while the disk was busy.
1569 NFM Subsequent fields: mounted NFS filesystem, total number of
1571 bytes read, total number of bytes written, number of bytes
1572 read by normal system calls, number of bytes written by normal
1573 system calls, number of bytes read by direct I/O, number of
1574 bytes written by direct I/O, number of pages read by memory-
1575 mapped I/O, and number of pages written by memory-mapped I/O.
1576 NFC Subsequent fields: number of transmitted RPCs, number of
1578 transmitted read RPCs, number of transmitted write RPCs, num‐
1579 ber of RPC retransmissions, and number of authorization re‐
1580 freshes.
1581 NFS Subsequent fields: number of handled RPCs, number of received
1583 read RPCs, number of received write RPCs, number of bytes read
1584 by clients, number of bytes written by clients, number of RPCs
1585 with bad format, number of RPCs with bad authorization, number
1586 of RPCs from bad client, total number of handled network re‐
1587 quests, number of handled network requests via TCP, number of
1588 handled network requests via UDP, number of handled TCP con‐
1589 nections, number of hits on reply cache, number of misses on
1590 reply cache, and number of uncached requests.
1591 NET First, one line is produced for the upper layers of the TCP/IP
1593 stack.
1594 Subsequent fields: the verb "upper", number of packets re‐
1595 ceived by TCP, number of packets transmitted by TCP, number of
1596 packets received by UDP, number of packets transmitted by UDP,
1597 number of packets received by IP, number of packets transmit‐
1598 ted by IP, number of packets delivered to higher layers by IP,
1599 number of packets forwarded by IP, number of input errors
1600 (UDP), number of noport errors (UDP), number of active opens
1601 (TCP), number of passive opens (TCP), number of passive opens
1602 (TCP), number of established connections at this moment (TCP),
1603 number of retransmitted segments (TCP), number of input errors
1604 (TCP), and number of output resets (TCP).
1605 Next, one line is shown for every interface.
1607 Subsequent fields: name of the interface, number of packets
1608 received by the interface, number of bytes received by the in‐
1609 terface, number of packets transmitted by the interface, num‐
1610 ber of bytes transmitted by the interface, interface speed,
1611 and duplex mode (0=half, 1=full).
1612 IFB Subsequent fields: name of the InfiniBand interface, port num‐
1614 ber, number of lanes, maximum rate (Mbps), number of bytes re‐
1615 ceived, number of bytes transmitted, number of packets re‐
1616 ceived, and number of packets transmitted.
1617 NUM Subsequent fields: NUMA node number, page size for this ma‐
1619 chine (in bytes), the fragmentation percentage of this node,
1620 size of physical memory (pages), size of free memory (pages),
1621 recently (active) used memory (pages), less recently (inac‐
1622 tive) used memory (pages), size of cached file data (pages),
1623 dirty pages in cache (pages), slab memory being used for ker‐
1624 nel mallocs (pages), slab memory that is reclaimable (pages),
1625 shared memory including tmpfs (pages), and total huge pages
1626 (pages).
1627 NUC Subsequent fields: NUMA node number, number of processors for
1629 this node, consumption for node CPUs in system mode (clock-
1630 ticks), consumption for node CPUs in user mode (clock-ticks),
1631 consumption for node CPUs in user mode for niced processes
1632 (clock-ticks), consumption for node CPUs in idle mode (clock-
1633 ticks), consumption for node CPUs in wait mode (clock-ticks),
1634 consumption for node CPUs in irq mode (clock-ticks), consump‐
1635 tion for node CPUs in softirq mode (clock-ticks), consumption
1636 for node CPUs in steal mode (clock-ticks), and consumption for
1637 node CPUs in guest mode (clock-ticks) overlapping user mode.
1638 PRG For every process one line is shown.
1640 Subsequent fields: PID (unique ID of task), name (between
1641 parenthesis or underscores for spaces), state, real uid, real
1642 gid, TGID (group number of related tasks/threads), total num‐
1643 ber of threads, exit code (in case of fatal signal: signal
1644 number + 256), start time (epoch), full command line (between
1645 parenthesis or underscores for spaces), PPID, number of
1646 threads in state 'running' (R), number of threads in state
1647 'interruptible sleeping' (S), number of threads in state 'un‐
1648 interruptible sleeping' (D), effective uid, effective gid,
1649 saved uid, saved gid, filesystem uid, filesystem gid, elapsed
1650 time of terminated process (hertz), is_process (y/n), OpenVZ
1651 virtual pid (VPID), OpenVZ container id (CTID), Container id
1652 (CID), indication if the task is newly started during this in‐
1653 terval ('N'), cgroup v2 path name (between parenthesis or un‐
1654 derscores for spaces), and end time (epoch. or 0 if still ac‐
1655 tive).
1656 PRC For every process one line is shown.
1658 Subsequent fields: PID, name (between parenthesis or under‐
1659 scores for spaces), state, total number of clock-ticks per
1660 second for this machine, CPU-consumption in user mode (clock‐
1661 ticks), CPU-consumption in system mode (clockticks), nice val‐
1662 ue, priority, realtime priority, scheduling policy, current
1663 CPU, sleep average, TGID (group number of related
1664 tasks/threads), is_process (y/n), runqueue delay in nanosec‐
1665 onds for this thread or for all threads (in case of process),
1666 wait channel of this thread (between parenthesis or under‐
1667 scores for spaces), block I/O delay (clockticks), number of
1668 voluntary context switches, and number of involunatary context
1669 switches.
1670 PRE For every process one line is shown.
1672 Subsequent fields: PID, name (between parenthesis or under‐
1673 scores for spaces), process state, GPU state (A for active, E
1674 for exited, N for no GPU user), number of GPUs used by this
1675 process, bitlist reflecting used GPUs, GPU busy percentage
1676 during interval, memory busy percentage during interval, memo‐
1677 ry occupation (KiB) at this moment cumulative memory occupa‐
1678 tion (KiB) during interval, and number of samples taken during
1679 interval.
1680 PRM For every process one line is shown.
1682 Subsequent fields: PID, name (between parenthesis or under‐
1683 scores for spaces), state, page size for this machine (in
1684 bytes), virtual memory size (Kbytes), resident memory size
1685 (Kbytes), shared text memory size (Kbytes), virtual memory
1686 growth (Kbytes), resident memory growth (Kbytes), number of
1687 minor page faults, number of major page faults, virtual li‐
1688 brary exec size (Kbytes), virtual data size (Kbytes), virtual
1689 stack size (Kbytes), swap space used (Kbytes), TGID (group
1690 number of related tasks/threads), is_process (y/n), propor‐
1691 tional set size (Kbytes) if in 'R' option is specified and
1692 virtually locked memory space (Kbytes).
1693 PRD For every process one line is shown.
1695 Subsequent fields: PID, name (between parenthesis or under‐
1696 scores for spaces), state, obsoleted kernel patch installed
1697 ('n'), standard io statistics used ('y' or 'n'), number of
1698 reads on disk, cumulative number of sectors read, number of
1699 writes on disk, cumulative number of sectors written, can‐
1700 celled number of written sectors, TGID (group number of relat‐
1701 ed tasks/threads), obsoleted value ('n'), and is_process
1702 (y/n).
1703 PRN For every process one line is shown.
1705 Subsequent fields: PID, name (between parenthesis or under‐
1706 scores for spaces), state, pmdabcc(1) module 'netproc' loaded
1707 ('y' or 'n'), number of tcp_sendmsg() calls, cumulative size
1708 of TCP buffers requested to be transmitted, number of
1709 tcp_recvmsg()/tcp_cleanup_rbuf() calls, cumulative size of TCP
1710 buffers received, number of udp_sendmsg() calls, cumulative
1711 size of UDP buffers requested to be transmitted, number of
1712 udp_recvmsg()/skb_consume_udp() calls, cumulative size of UDP
1713 buffers transmitted, number of raw packets transmitted (obso‐
1714 lete, always 0), number of raw packets received (obsolete, al‐
1715 ways 0), TGID (group number of related tasks/threads) and
1716 is_process (y/n).
1717
1719 By sending the SIGUSR1 signal to pcp-atop a new sample will be forced,
1720 even if the current timer interval has not exceeded yet. The behavior
1721 is similar to pressing the 't' key in an interactive session.
1722 By sending the SIGUSR2 signal to pcp-atop a final sample will be forced
1724 after which pcp-atop will terminate.
1725
1727 To monitor the current system load interactively with an interval of
1728 (default) 10 seconds:
1729 pcp atop
1731 To monitor the system load as bar graphs with an interval of 5 seconds:
1733 pcp atop -B 5
1735 Store information about the system and process activity in a PCP ar‐
1737 chive folio with an interval of ten minutes during an hour:
1738 pcp atop -w /tmp/pcp-atop 600 6
1740 View the contents of this file interactively:
1742 pcp atop -r /tmp/pcp-atop
1744 View the processor and disk utilization of this file in parsable for‐
1746 mat:
1747 pcp atop -PCPU,DSK -r /tmp/pcp-atop.folio
1749 View the contents of today's standard logfile interactively:
1751 pcp atop -r
1753 View the contents of the standard logfile of the day before yesterday
1755 interactively:
1756 pcp atop -r yy
1758 View the contents of the standard logfile of 2023, June 7 from 02:00 PM
1760 onwards interactively:
1761 pcp atop -r 20230607 -b 14:00
1763 To monitor the system load and write it to a file (in plain ASCII) with
1765 an interval of one minute during half an hour with active processes
1766 sorted on memory consumption:
1767 pcp atop -M 60 30 > /log/pcp-atop.mem
1769
1771 pcp-atop is based on the source code of the atop(1) command from
1772 https://atoptool.nl, maintained by Gerlof Langeveld
1773 (gerlof.langeveld@atoptool.nl), and aims to be command line and output
1774 compatible with it as much as possible.
1775 Some features of pcp-atop (such as reporting on the Apache HTTP daemon,
1777 Infiniband, NFS client mounts, hardware event counts, GPU statistics
1778 and per-process TCP and UDP statistics) are only activated if the cor‐
1779 resonding PCP metrics are available. Refer to the documentation for pm‐
1780 daapache(1), pmdainfiniband(1), pmdanfsclient(1), pmdanvidia(1), pm‐
1781 daperfevent(1) and pmdabcc(1) for further details on activating these
1782 metrics.
1783 The semantics of the per-process network statistics deviate slightly
1785 from the atop(1) tool: instead of the number of TCP/UDP packets
1786 sent/received (which may be inaccurate due to TCP segmentation off‐
1787 load), pcp-atop shows the number of tcp_sendmsg()/udp_sendmsg()/etc.
1788 kernel calls per process.
1789
1791 /etc/atoprc
1792 Configuration file containing system-wide default values. For
1793 further information about the default values, refer to the pcp-
1794 atoprc(5) man page).
1795 ~/.atoprc
1797 Configuration file containing personal default values. For fur‐
1798 ther information about the default values, refer to the pcp-ato‐
1799 prc(5) man page).
1800
1802 Environment variables with the prefix PCP_ are used to parameterize the
1803 file and directory names used by PCP. On each installation, the file
1804 /etc/pcp.conf contains the local values for these variables. The
1805 $PCP_CONF variable may be used to specify an alternative configuration
1806 file, as described in pcp.conf(5).
1807 For environment variables affecting PCP tools, see pmGetOptions(3).
1809
1811 PCPIntro(1), pcp(1), pcp-atopsar(1), pmdaapache(1), pmdabcc(1), pmdain‐
1812 finiband(1), pmdanfsclient(1), pmdanvidia(1), pmdaproc(1), mkaf(1), pm‐
1813 logger(1), pmlogger_daily(1) and pcp-atoprc(5).
1814Performance Co-Pilot PCP PCP-ATOP(1)