1ATOP(1) General Commands Manual ATOP(1)
2
6 atop - Advanced System & Process Monitor
7
9 Interactive Usage:
10 atop [-g|-m|-d|-n|-u|-p|-s|-c|-v|-o|-y] [-C|-M|-D|-N|-A] [-afFG1xR] [-L
12 linelen] [-Plabel[,label]...] [ interval [ samples ]]
13 Writing and reading raw logfiles:
15 atop -w rawfile [-a] [-S] [ interval [ samples ]]
17 atop -r [ rawfile ] [-b hh:mm ] [-e hh:mm ]
18 [-g|-m|-d|-n|-u|-p|-s|-c|-v|-o|-y] [-C|-M|-D|-N|-A] [-fFG1xR] [-L line‐
19 len] [-Plabel[,label]...]
20
22 The program atop is an interactive monitor to view the load on a Linux
23 system. It shows the occupation of the most critical hardware
24 resources (from a performance point of view) on system level, i.e. cpu,
25 memory, disk and network.
26 It also shows which processes are responsible for the indicated load
27 with respect to cpu and memory load on process level. Disk load is
28 shown per process if "storage accounting" is active in the kernel.
29 Network load is shown per process if the kernel module `netatop' has
30 been installed.
31 Every interval (default: 10 seconds) information is shown about the
33 resource occupation on system level (cpu, memory, disks and network
34 layers), followed by a list of processes which have been active during
35 the last interval (note that all processes that were unchanged during
36 the last interval are not shown, unless the key 'a' has been pressed or
37 unless sorting on memory occupation is done). If the list of active
38 processes does not entirely fit on the screen, only the top of the list
39 is shown (sorted in order of activity).
40 The intervals are repeated till the number of samples (specified as
41 command argument) is reached, or till the key 'q' is pressed in inter‐
42 active mode.
43 When atop is started, it checks whether the standard output channel is
45 connected to a screen, or to a file/pipe. In the first case it produces
46 screen control codes (via the ncurses library) and behaves interac‐
47 tively; in the second case it produces flat ASCII-output.
48 In interactive mode, the output of atop scales dynamically to the cur‐
50 rent dimensions of the screen/window.
51 If the window is resized horizontally, columns will be added or removed
52 automatically. For this purpose, every column has a particular weight.
53 The columns with the highest weights that fit within the current width
54 will be shown.
55 If the window is resized vertically, lines of the process/thread list
56 will be added or removed automatically.
57 Furthermore in interactive mode the output of atop can be controlled by
59 pressing particular keys. However it is also possible to specify such
60 key as flag on the command line. In that case atop switches to the
61 indicated mode on beforehand; this mode can be modified again interac‐
62 tively. Specifying such key as flag is especially useful when running
63 atop with output to a pipe or file (non-interactively). These flags
64 are the same as the keys that can be pressed in interactive mode (see
65 section INTERACTIVE COMMANDS).
66 Additional flags are available to support storage of atop-data in raw
67 format (see section RAW DATA STORAGE).
68
70 With every interval, atop reads the kernel administration to obtain
71 information about all running processes. However, it is likely that
72 during the interval also processes have terminated. These processes
73 might have consumed system resources during this interval as well
74 before they terminated. Therefor, atop tries to read the process
75 accounting records that contain the accounting information of termi‐
76 nated processes and report these processes too. Only when the process
77 accounting mechanism in the kernel is activated, the kernel writes such
78 process accounting record to a file for every process that terminates.
79 There are various ways for atop to get access to the process accounting
81 records (tried in this order):
82 1. When the environment variable ATOPACCT is set, it specifies the
84 name of the process accounting file. In that case, process
85 accounting for this file should have been activated on beforehand.
86 Before opening this file for reading, atop drops its root privi‐
87 leges (if any).
88 When this environment variable is present but its contents is
89 empty, process accounting will not be used at all.
90 2. This is the preferred way of handling process accounting records!
92 When the atopacctd daemon is active, it has activated the process
93 accounting mechanism in the kernel and transfers to original
94 accounting records to shadow files. In that case, atop drops its
95 root privileges and opens the current shadow file for reading.
96 This way is preferred, because the atopacctd daemon maintains full
97 control of the sizes of the original process accounting file (writ‐
98 ten by the kernel) and the shadow files (read by the atop pro‐
99 cesses). For further information, refer to the atopacctd man page.
100 3. When the atopacctd daemon is not active, atop verifies if the
102 process accounting mechanism has been switched on via the separate
103 psacct package. In that case, the file /var/account/pacct is in use
104 as process accounting file and atop opens this file for reading.
105 4. As a last possibility, atop itself tries to activate the process
107 accounting mechanism (requires root privileges) using the file
108 /var/cache/atop.d/atop.acct (to be written by the kernel, to be
109 read by atop itself). Process accounting remains active as long as
110 at least one atop process is alive. Whenever the last atop process
111 stops (either by pressing `q' or by `kill -15'), it deactivates the
112 process accounting mechanism again. Therefor you should never ter‐
113 minate atop by `kill -9', because then it has no chance to stop
114 process accounting. As a result, the accounting file may consume a
115 lot of disk space after a while.
116 To avoid that the process accounting file consumes too much disk
117 space, atop verifies at the end of every sample if the size of the
118 process accounting file exceeds 200 MiB and if this atop process is
119 the only one that is currently using the file. In that case the
120 file is truncated to a size of zero.
121 Notice that root-privileges are required to switch on/off process
123 accounting in the kernel. You can start atop as a root user or
124 specify setuid-root privileges to the executable file. In the lat‐
125 ter case, atop switches on process accounting and drops the root-
126 privileges again.
127 If atop does not run with root-privileges, it does not show infor‐
128 mation about finished processes. It indicates this situation with
129 the message message `no procacct` in the top-right corner (instead
130 of the counter that shows the number of exited processes).
131 When during one interval a lot of processes have finished, atop might
133 grow tremendously in memory when reading all process accounting records
134 at the end of the interval. To avoid such excessive growth, atop will
135 never read more than 50 MiB with process information from the process
136 accounting file per interval (approx. 70000 finished processes). In
137 interactive mode a warning is given whenever processes have been
138 skipped for this reason.
139
141 For the resource consumption on system level, atop uses colors to indi‐
142 cate that a critical occupation percentage has been (almost) reached.
143 A critical occupation percentage means that is likely that this load
144 causes a noticeable negative performance influence for applications
145 using this resource. The critical percentage depends on the type of
146 resource: e.g. the performance influence of a disk with a busy percent‐
147 age of 80% might be more noticeable for applications/user than a CPU
148 with a busy percentage of 90%.
149 Currently atop uses the following default values to calculate a
150 weighted percentage per resource:
151 Processor
153 A busy percentage of 90% or higher is considered `critical'.
154 Disk
156 A busy percentage of 70% or higher is considered `critical'.
157 Network
159 A busy percentage of 90% or higher for the load of an interface is
160 considered `critical'.
161 Memory
163 An occupation percentage of 90% is considered `critical'. Notice
164 that this occupation percentage is the accumulated memory consump‐
165 tion of the kernel (including slab) and all processes; the memory
166 for the page cache (`cache' and `buff' in the MEM-line) and the
167 reclaimable part of the slab (`slrec`) is not implied!
168 If the number of pages swapped out (`swout' in the PAG-line) is
169 larger than 10 per second, the memory resource is considered
170 `critical'. A value of at least 1 per second is considered
171 `almost critical'.
172 If the committed virtual memory exceeds the limit (`vmcom' and
173 `vmlim' in the SWP-line), the SWP-line is colored due to overcom‐
174 mitting the system.
175 Swap
177 An occupation percentage of 80% is considered `critical' because
178 swap space might be completely exhausted in the near future; it is
179 not critical from a performance point-of-view.
180 These default values can be modified in the configuration file (see
182 separate man-page of atoprc).
183 When a resource exceeds its critical occupation percentage, the con‐
185 cerning values in the screen line are colored red by default.
186 When a resource exceeded (default) 80% of its critical percentage (so
187 it is almost critical), the concerning values in the screen line are
188 colored cyan by default. This `almost critical percentage' (one value
189 for all resources) can be modified in the configuration file (see sepa‐
190 rate man-page of atoprc).
191 The default colors red and cyan can be modified in the configuration
192 file as well (see separate man-page of atoprc).
193 With the key 'x' (or flag -x), the use of colors can be suppressed.
195
197 Per-process and per-thread network activity can be measured by the
198 netatop kernel module. You can download this kernel module from the
199 website (mentioned at the end of this manual page) and install it on
200 your system if the kernel version is 2.6.24 or newer.
201 When atop gathers counters for a new interval, it verifies if the
202 netatop module is currently active. If so, atop obtains the relevant
203 network counters from this module and shows the number of sent and
204 received packets per process/thread in the generic screen. Besides,
205 detailed counters can be requested by pressing the `n' key.
206 When the netatopd daemon is running as well, atop also reads the net‐
207 work counters of exited processes that are logged by this daemon (com‐
208 parable with process accounting).
209 More information about the optional netatop kernel module and the
211 netatopd daemon can be found in the concerning man-pages and on the
212 website mentioned at the end of this manual page.
213
215 GPU statistics can be gathered by atopgpud which is a separate data
216 collection daemon process. It gathers cumulative utilization counters
217 of every Nvidia GPU in the system, as well as utilization counters of
218 every process that uses a GPU. When atop notices that the daemon is
219 active, it reads these GPU utilization counters with every interval.
220 The atopgpud daemon is written in Python, so a Python interpreter
222 should be installed on the target system. The Python code of the daemon
223 is compatible with Python version 2 and version 3. For the gathering
224 of the statistics, the pynvml module is used by the daemon. Be sure
225 that this module is installed on the target system before activating
226 the daemon, by running the command as root pip (the command pip might
227 be exchanged by pip3 in case of Python3):
228 pip install nvidia-ml-py
230 The atopgpud daemon is installed by default as part of the atop pack‐
232 age, but it is not automatically enabled. The daemon can be enabled
233 and started now by running the following commands (as root):
234 systemctl enable atopgpu
236 systemctl start atopgpu
237 Find a description about the utilization counters in the section OUTPUT
239 DESCRIPTION.
240
242 When running atop interactively (no output redirection), keys can be
243 pressed to control the output. In general, lower case keys can be used
244 to show other information for the active processes and upper case keys
245 can be used to influence the sort order of the active process/thread
246 list.
247 g Show generic output (default).
249 Per process the following fields are shown in case of a window-
251 width of 80 positions: process-id, cpu consumption during the last
252 interval in system and user mode, the virtual and resident memory
253 growth of the process.
254 The subsequent columns depend on the used kernel:
256 When the kernel supports "storage accounting" (>= 2.6.20), the
257 data transfer for read/write on disk, the status and exit code are
258 shown for each process. When the kernel does not support "storage
259 accounting", the username, number of threads in the thread group,
260 the status and exit code are shown.
261 When the kernel module 'netatop' is loaded, the data transfer for
262 send/receive of network packets is shown for each process.
263 The last columns contain the state, the occupation percentage for
264 the chosen resource (default: cpu) and the process name.
265 When more than 80 positions are available, other information is
267 added.
268 m Show memory related output.
270 Per process the following fields are shown in case of a window-
272 width of 80 positions: process-id, minor and major memory faults,
273 size of virtual shared text, total virtual process size, total
274 resident process size, virtual and resident growth during last
275 interval, memory occupation percentage and process name.
276 When more than 80 positions are available, other information is
278 added.
279 For memory consumption, always all processes are shown (also the
281 processes that were not active during the interval).
282 d Show disk-related output.
284 When "storage accounting" is active in the kernel, the following
286 fields are shown: process-id, amount of data read from disk,
287 amount of data written to disk, amount of data that was written
288 but has been withdrawn again (WCANCL), disk occupation percentage
289 and process name.
290 n Show network related output.
292 Per process the following fields are shown in case of a window-
294 width of 80 positions: process-id, thread-id, total bandwidth for
295 received packets, total bandwidth for sent packets, number of
296 received TCP packets with the average size per packet (in bytes),
297 number of sent TCP packets with the average size per packet (in
298 bytes), number of received UDP packets with the average size per
299 packet (in bytes), number of sent UDP packets with the average
300 size per packet (in bytes), the network occupation percentage and
301 process name.
302 This information can only be shown when kernel module `netatop' is
303 installed.
304 When more than 80 positions are available, other information is
306 added.
307 s Show scheduling characteristics.
309 Per process the following fields are shown in case of a window-
311 width of 80 positions: process-id, number of threads in state
312 'running' (R), number of threads in state 'interruptible sleeping'
313 (S), number of threads in state 'uninterruptible sleeping' (D),
314 scheduling policy (normal timesharing, realtime round-robin, real‐
315 time fifo), nice value, priority, realtime priority, current pro‐
316 cessor, status, exit code, state, the occupation percentage for
317 the chosen resource and the process name.
318 When more than 80 positions are available, other information is
320 added.
321 v Show various process characteristics.
323 Per process the following fields are shown in case of a window-
325 width of 80 positions: process-id, user name and group, start date
326 and time, status (e.g. exit code if the process has finished),
327 state, the occupation percentage for the chosen resource and the
328 process name.
329 When more than 80 positions are available, other information is
331 added.
332 c Show the command line of the process.
334 Per process the following fields are shown: process-id, the occu‐
336 pation percentage for the chosen resource and the command line
337 including arguments.
338 e Show GPU utilization.
340 Per process at least the following fields are shown: process-id,
342 range of GPU numbers on which the process currently runs, GPU busy
343 percentage on all GPUs, memory busy percentage (i.e. read and
344 write accesses on memory) on all GPUs, memory occupation at the
345 moment of the sample, average memory occupation during the sample,
346 and GPU percentage.
347 When the atopgpud daemon does not run with root privileges, the
349 GPU busy percentage and the memory busy percentage are not avail‐
350 able on process level. In that case, the GPU percentage on
351 process level reflects the GPU memory occupation instead of the
352 GPU busy percentage (which is preferred).
353 o Show the user-defined line of the process.
355 In the configuration file the keyword ownprocline can be specified
357 with the description of a user-defined output-line.
358 Refer to the man-page of atoprc for a detailed description.
359 y Show the individual threads within a process (toggle).
361 Single-threaded processes are still shown as one line.
363 For multi-threaded processes, one line represents the process
364 while additional lines show the activity per individual thread (in
365 a different color). Depending on the option 'a' (all or active
366 toggle), all threads are shown or only the threads that were
367 active during the last interval.
368 Whether this key is active or not can be seen in the header line.
369 u Show the process activity accumulated per user.
371 Per user the following fields are shown: number of processes
373 active or terminated during last interval (or in total if combined
374 with command `a'), accumulated cpu consumption during last inter‐
375 val in system and user mode, the current virtual and resident mem‐
376 ory space consumed by active processes (or all processes of the
377 user if combined with command `a').
378 When "storage accounting" is active in the kernel, the accumulated
379 read and write throughput on disk is shown. When the kernel mod‐
380 ule `netatop' has been installed, the number of received and sent
381 network packets are shown.
382 The last columns contain the accumulated occupation percentage for
383 the chosen resource (default: cpu) and the user name.
384 p Show the process activity accumulated per program (i.e. process
386 name).
387 Per program the following fields are shown: number of processes
389 active or terminated during last interval (or in total if combined
390 with command `a'), accumulated cpu consumption during last inter‐
391 val in system and user mode, the current virtual and resident mem‐
392 ory space consumed by active processes (or all processes of the
393 user if combined with command `a').
394 When "storage accounting" is active in the kernel, the accumulated
395 read and write throughput on disk is shown. When the kernel mod‐
396 ule `netatop' has been installed, the number of received and sent
397 network packets are shown.
398 The last columns contain the accumulated occupation percentage for
399 the chosen resource (default: cpu) and the program name.
400 j Show the process activity accumulated per Docker container.
402 Per container the following fields are shown: number of processes
404 active or terminated during last interval (or in total if combined
405 with command `a'), accumulated cpu consumption during last inter‐
406 val in system and user mode, the current virtual and resident mem‐
407 ory space consumed by active processes (or all processes of the
408 user if combined with command `a').
409 When "storage accounting" is active in the kernel, the accumulated
410 read and write throughput on disk is shown. When the kernel mod‐
411 ule `netatop' has been installed, the number of received and sent
412 network packets are shown.
413 The last columns contain the accumulated occupation percentage for
414 the chosen resource (default: cpu) and the Docker container id
415 (CID).
416 C Sort the current list in the order of cpu consumption (default).
418 The one-but-last column changes to ``CPU''.
419 E Sort the current list in the order of GPU utilization (preferred,
421 but only applicable when the atopgpud daemon runs under root priv‐
422 ileges) or the order of GPU memory occupation). The one-but-last
423 column changes to ``GPU''.
424 M Sort the current list in the order of resident memory consumption.
426 The one-but-last column changes to ``MEM''. In case of sorting on
427 memory, the full process list will be shown (not only the active
428 processes).
429 D Sort the current list in the order of disk accesses issued. The
431 one-but-last column changes to ``DSK''.
432 N Sort the current list in the order of network bandwidth (received
434 and transmitted). The one-but-last column changes to ``NET''.
435 A Sort the current list automatically in the order of the most busy
437 system resource during this interval. The one-but-last column
438 shows either ``ACPU'', ``AMEM'', ``ADSK'' or ``ANET'' (the preced‐
439 ing 'A' indicates automatic sorting-order). The most busy
440 resource is determined by comparing the weighted busy-percentages
441 of the system resources, as described earlier in the section COL‐
442 ORS.
443 This option remains valid until another sorting-order is explic‐
444 itly selected again.
445 A sorting-order for disk is only possible when "storage account‐
446 ing" is active. A sorting-order for network is only possible when
447 the kernel module `netatop' is loaded.
448 Miscellaneous interactive commands:
450 ? Request for help information (also the key 'h' can be pressed).
452 V Request for version information (version number and date).
454 R Gather and calculate the proportional set size of processes (tog‐
456 gle). Gathering of all values that are needed to calculate the
457 PSIZE of a process is a relatively time-consuming task, so this
458 key should only be active when analyzing the resident memory con‐
459 sumption of processes.
460 x Suppress colors to highlight critical resources (toggle).
462 Whether this key is active or not can be seen in the header line.
463 z The pause key can be used to freeze the current situation in order
465 to investigate the output on the screen. While atop is paused, the
466 keys described above can be pressed to show other information
467 about the current list of processes. Whenever the pause key is
468 pressed again, atop will continue with a next sample.
469 i Modify the interval timer (default: 10 seconds). If an interval
471 timer of 0 is entered, the interval timer is switched off. In that
472 case a new sample can only be triggered manually by pressing the
473 key 't'.
474 t Trigger a new sample manually. This key can be pressed if the cur‐
476 rent sample should be finished before the timer has exceeded, or
477 if no timer is set at all (interval timer defined as 0). In the
478 latter case atop can be used as a stopwatch to measure the load
479 being caused by a particular application transaction, without
480 knowing on beforehand how many seconds this transaction will last.
481 When viewing the contents of a raw file, this key can be used to
483 show the next sample from the file.
484 T When viewing the contents of a raw file, this key can be used to
486 show the previous sample from the file (except when reading raw
487 data from a named pipe).
488 b When viewing the contents of a raw file, this key can be used to
490 branch to a certain timestamp within the file either forward or
491 backward (except when reading raw data from a named pipe).
492 r Reset all counters to zero to see the system and process activity
494 since boot again.
495 When viewing the contents of a raw file, this key can be used to
497 rewind to the beginning of the file again (except when reading raw
498 data from a named pipe).
499 U Specify a search string for specific user names as a regular
501 expression. From now on, only (active) processes will be shown
502 from a user which matches the regular expression. The system sta‐
503 tistics are still system wide. If the Enter-key is pressed with‐
504 out specifying a name, (active) processes of all users will be
505 shown again.
506 Whether this key is active or not can be seen in the header line.
507 I Specify a list with one or more PIDs to be selected. From now on,
509 only processes will be shown with a PID which matches one of the
510 given list. The system statistics are still system wide. If the
511 Enter-key is pressed without specifying a PID, all (active) pro‐
512 cesses will be shown again.
513 Whether this key is active or not can be seen in the header line.
514 P Specify a search string for specific process names as a regular
516 expression. From now on, only processes will be shown with a name
517 which matches the regular expression. The system statistics are
518 still system wide. If the Enter-key is pressed without specifying
519 a name, all (active) processes will be shown again.
520 Whether this key is active or not can be seen in the header line.
521 / Specify a specific command line search string as a regular expres‐
523 sion. From now on, only processes will be shown with a command
524 line which matches the regular expression. The system statistics
525 are still system wide. If the Enter-key is pressed without speci‐
526 fying a string, all (active) processes will be shown again.
527 Whether this key is active or not can be seen in the header line.
528 J Specify a Docker container id of 12 (hexadecimal) characters.
530 From now on, only processes will be shown that run in that spe‐
531 cific Docker container (CID). The system statistics are still
532 system wide. If the Enter-key is pressed without specifying a
533 container id, all (active) processes will be shown again.
534 Whether this key is active or not can be seen in the header line.
535 S Specify search strings for specific logical volume names, specific
537 disk names and specific network interface names. All search
538 strings are interpreted as a regular expressions. From now on,
539 only those system resources are shown that match the concerning
540 regular expression. If the Enter-key is pressed without specify‐
541 ing a search string, all (active) system resources of that type
542 will be shown again.
543 Whether this key is active or not can be seen in the header line.
544 a The `all/active' key can be used to toggle between only show‐
546 ing/accumulating the processes that were active during the last
547 interval (default) or showing/accumulating all processes.
548 Whether this key is active or not can be seen in the header line.
549 G By default, atop shows/accumulates the processes that are alive
551 and the processes that are exited during the last interval. With
552 this key (toggle), showing/accumulating the processes that are
553 exited can be suppressed.
554 Whether this key is active or not can be seen in the header line.
555 f Show a fixed (maximum) number of header lines for system resources
557 (toggle). By default only the lines are shown about system
558 resources (CPUs, paging, logical volumes, disks, network inter‐
559 faces) that really have been active during the last interval.
560 With this key you can force atop to show lines of inactive
561 resources as well.
562 Whether this key is active or not can be seen in the header line.
563 F Suppress sorting of system resources (toggle). By default system
565 resources (CPUs, logical volumes, disks, network interfaces) are
566 sorted on utilization.
567 Whether this key is active or not can be seen in the header line.
568 1 Show relevant counters as an average per second (in the format
570 `..../s') instead of as a total during the interval (toggle).
571 Whether this key is active or not can be seen in the header line.
572 l Limit the number of system level lines for the counters per-cpu,
574 the active disks and the network interfaces. By default lines are
575 shown of all CPUs, disks and network interfaces which have been
576 active during the last interval. Limiting these lines can be use‐
577 ful on systems with huge number CPUs, disks or interfaces in order
578 to be able to run atop on a screen/window with e.g. only 24 lines.
579 For all mentioned resources the maximum number of lines can be
580 specified interactively. When using the flag -l the maximum number
581 of per-cpu lines is set to 0, the maximum number of disk lines to
582 5 and the maximum number of interface lines to 3. These values
583 can be modified again in interactive mode.
584 k Send a signal to an active process (a.k.a. kill a process).
586 q Quit the program.
588 PgDn Show the next page of the process/thread list.
590 With the arrow-down key the list can be scrolled downwards with
591 single lines.
592 ^F Show the next page of the process/thread list (forward).
594 With the arrow-down key the list can be scrolled downwards with
595 single lines.
596 PgUp Show the previous page of the process/thread list.
598 With the arrow-up key the list can be scrolled upwards with single
599 lines.
600 ^B Show the previous page of the process/thread list (backward).
602 With the arrow-up key the list can be scrolled upwards with single
603 lines.
604 ^L Redraw the screen.
606
608 In order to store system and process level statistics for long-term
609 analysis (e.g. to check the system load and the active processes run‐
610 ning yesterday between 3:00 and 4:00 PM), atop can store the system and
611 process level statistics in compressed binary format in a raw file with
612 the flag -w followed by the filename. If this file already exists and
613 is recognized as a raw data file, atop will append new samples to the
614 file (starting with a sample which reflects the activity since boot);
615 if the file does not exist, it will be created.
616 All information about processes and threads is stored in the raw file.
617 The interval (default: 10 seconds) and number of samples (default:
618 infinite) can be passed as last arguments. Instead of the number of
619 samples, the flag -S can be used to indicate that atop should finish
620 anyhow before midnight.
621 A raw file can be read and visualized again with the flag -r followed
623 by the filename. If no filename is specified, the file
624 /var/log/atop/atop_YYYYMMDD is opened for input (where YYYYMMDD are
625 digits representing the current date). If a filename is specified in
626 the format YYYYMMDD (representing any valid date), the file
627 /var/log/atop/atop_YYYYMMDD is opened. If a filename with the symbolic
628 name y is specified, yesterday's daily logfile is opened (this can be
629 repeated so 'yyyy' indicates the logfile of four days ago).
630 The samples from the file can be viewed interactively by using the key
631 't' to show the next sample, the key 'T' to show the previous sample,
632 the key 'b' to branch to a particular time or the key 'r' to rewind to
633 the begin of the file.
634 When output is redirected to a file or pipe, atop prints all samples in
635 plain ASCII. The default line length is 80 characters in that case;
636 with the flag -L followed by an alternate line length, more (or less)
637 columns will be shown.
638 With the flag -b (begin time) and/or -e (end time) followed by a time
639 argument of the form HH:MM, a certain time period within the raw file
640 can be selected.
641 Every day at midnight atop is restarted to write compressed binary data
643 to the file /var/log/atop/atop_YYYYMMDD with an interval of 10 minutes
644 by default. The -R flag is passed by default to gather information
645 about the proportional set size of every process.
646 Furthermore all raw files are removed that are older than 28 days (by
647 default).
648 The mentioned default values can be overruled in the file
649 /etc/default/atop that might contain other values for LOGOPTS (by
650 default the -R flag), LOGINTERVAL (in seconds, by default 600), LOGGEN‐
651 ERATIONS (in days, by default 28), and LOGPATH (directory in which log‐
652 files are stored).
653 Unfortunately, it is not always possible to keep the format of the raw
655 files compatible in newer versions of atop especially when lots of new
656 counters have to be maintained. Therefore, the program atopconvert is
657 installed to convert a raw file created by an older version of atop to
658 a raw file that can be read by a newer version of atop (see the man
659 page of atopconvert for more details).
660
663 The first sample shows the system level activity since boot (the
664 elapsed time in the header shows the time since boot). Note that par‐
665 ticular counters could have reached their maximum value (several times)
666 and started by zero again, so do not rely on these figures.
667 For every sample atop first shows the lines related to system level
669 activity. If a particular system resource has not been used during the
670 interval, the entire line related to this resource is suppressed. So
671 the number of system level lines may vary for each sample.
672 After that a list is shown of processes which have been active during
673 the last interval. This list is by default sorted on cpu consumption,
674 but this order can be changed by the keys which are previously
675 described.
676 If values have to be shown by atop which do not fit in the column
678 width, another format is used. If e.g. a cpu-consumption of 233216 mil‐
679 liseconds should be shown in a column width of 4 positions, it is shown
680 as `233s' (in seconds). For large memory figures, another unit is cho‐
681 sen if the value does not fit (Mb instead of Kb, Gb instead of Mb, Tb
682 instead of Gb, ...). For other values, a kind of exponent notation is
683 used (value 123456789 shown in a column of 5 positions gives 123e6).
684
686 The system level information consists of the following output lines:
687 PRC Process and thread level totals.
689 This line contains the total cpu time consumed in system mode
690 (`sys') and in user mode (`user'), the total number of processes
691 present at this moment (`#proc'), the total number of threads
692 present at this moment in state `running' (`#trun'), `sleeping
693 interruptible' (`#tslpi') and `sleeping uninterruptible'
694 (`#tslpu'), the number of zombie processes (`#zombie'), the number
695 of clone system calls (`clones'), and the number of processes that
696 ended during the interval (`#exit') when process accounting is
697 used. Instead of `#exit` the last column may indicate that process
698 accounting could not be activated (`no procacct`).
699 If the screen-width does not allow all of these counters, only a
700 relevant subset is shown.
701 CPU CPU utilization.
703 At least one line is shown for the total occupation of all CPUs
704 together.
705 In case of a multi-processor system, an additional line is shown
706 for every individual processor (with `cpu' in lower case), sorted
707 on activity. Inactive CPUs will not be shown by default. The
708 lines showing the per-cpu occupation contain the cpu number in the
709 field combined with the wait percentage.
710 Every line contains the percentage of cpu time spent in kernel
712 mode by all active processes (`sys'), the percentage of cpu time
713 consumed in user mode (`user') for all active processes (including
714 processes running with a nice value larger than zero), the per‐
715 centage of cpu time spent for interrupt handling (`irq') including
716 softirq, the percentage of unused cpu time while no processes were
717 waiting for disk I/O (`idle'), and the percentage of unused cpu
718 time while at least one process was waiting for disk I/O (`wait').
719 In case of per-cpu occupation, the cpu number and the wait per‐
720 centage (`w') for that cpu. The number of lines showing the per-
721 cpu occupation can be limited.
722 For virtual machines, the steal-percentage (`steal') shows the
724 percentage of cpu time stolen by other virtual machines running on
725 the same hardware.
726 For physical machines hosting one or more virtual machines, the
727 guest-percentage (`guest') shows the percentage of cpu time used
728 by the virtual machines. Notice that this percentage overlaps the
729 user percentage!
730 When PMC performance monitoring counters are supported by the CPU
732 and the kernel (and atop runs with root privileges), the number of
733 instructions per CPU cycle (`ipc') is shown. The first sample
734 always shows the value 'initial', because the counters are just
735 activated at the moment that atop is started.
736 When the CPU busy percentage is high and the IPC is less than 1.0,
737 it is likely that the CPU is frequently waiting for memory access
738 during instruction execution (larger CPU caches or faster memory
739 might be helpful to improve performance). When the CPU busy per‐
740 centage is high and the IPC is greater than 1.0, it is likely that
741 the CPU is instruction-bound (more/faster cores might be helpful
742 to improve performance).
743 Furthermore, per CPU the effective number of cycles (`cycl') is
744 shown. This value can reach the current CPU frequency if such CPU
745 is 100% busy. When an idle CPU is halted, the number of effective
746 cycles can be (considerably) lower than the current frequency.
747 Notice that the average instructions per cycle and number of
748 cycles is shown in the CPU line for all CPUs.
749 Beware that reading the cycle counter in virtual machines (guests)
750 might introduce performance delays. Therefore this metric is by
751 default disabled in virtual machines. However, with the keyword
752 'perfevents' in the atoprc file this metric can be explicitly set
753 to 'enable' or 'disable' (see separate man-page of atoprc).
754 See also: http://www.brendangregg.com/blog/2017-05-09/cpu-utiliza‐
755 tion-is-wrong.html
756 In case of frequency scaling, all previously mentioned CPU per‐
759 centages are relative to the used scaling of the CPU during the
760 interval. If a CPU has been active for e.g. 50% in user mode dur‐
761 ing the interval while the frequency scaling of that CPU was 40%,
762 only 20% of the full capacity of the CPU has been used in user
763 mode.
764 In case that the kernel module `cpufreq_stats' is active (after
765 issueing `modprobe cpufreq_stats'), the average frequency (`avgf')
766 and the average scaling percentage (`avgscal') is shown. Otherwise
767 the current frequency (`curf') and the current scaling percentage
768 (`curscal') is shown at the moment that the sample is taken.
769 Notice that average values for frequency and scaling are shown in
770 the CPU line for every CPU.
771 Frequency scaling statistics are only gathered for systems with
772 maximum 8 CPUs, since gathering of these values per CPU is very
773 time consuming.
774 If the screen-width does not allow all of these counters, only a
776 relevant subset is shown.
777 CPL CPU load information.
779 This line contains the load average figures reflecting the number
780 of threads that are available to run on a CPU (i.e. part of the
781 runqueue) or that are waiting for disk I/O. These figures are
782 averaged over 1 (`avg1'), 5 (`avg5') and 15 (`avg15') minutes.
783 Furthermore the number of context switches (`csw'), the number of
784 serviced interrupts (`intr') and the number of available CPUs are
785 shown.
786 If the screen-width does not allow all of these counters, only a
788 relevant subset is shown.
789 GPU GPU utilization (Nvidia).
791 Read the section GPU STATISTICS GATHERING in this document to find
792 the details about the activation of the atopgpud daemon.
793 In the first column of every line, the bus-id (last nine charac‐
795 ters) and the GPU number are shown. The subsequent columns show
796 the percentage of time that one or more kernels were executing on
797 the GPU (`gpubusy'), the percentage of time that global (device)
798 memory was being read or written (`membusy'), the occupation per‐
799 centage of memory (`memocc'), the total memory (`total'), the mem‐
800 ory being in use at the moment of the sample (`used'), the average
801 memory being in use during the sample time (`usavg'), the number
802 of processes being active on the GPU at the moment of the sample
803 (`#proc'), and the type of GPU.
804 If the screen-width does not allow all of these counters, only a
806 relevant subset is shown.
807 The number of lines showing the GPUs can be limited.
808 MEM Memory occupation.
810 This line contains the total amount of physical memory (`tot'),
811 the amount of memory which is currently free (`free'), the amount
812 of memory in use as page cache including the total resident shared
813 memory (`cache'), the amount of memory within the page cache that
814 has to be flushed to disk (`dirty'), the amount of memory used for
815 filesystem meta data (`buff'), the amount of memory being used for
816 kernel mallocs (`slab'), the amount of slab memory that is
817 reclaimable (`slrec'), the resident size of shared memory includ‐
818 ing tmpfs (`shmem`), the resident size of shared memory (`shrss`)
819 the amount of shared memory that is currently swapped (`shswp`),
820 the amount of memory that is currently claimed by vmware's balloon
821 driver (`vmbal`), the amount of memory that is claimed for huge
822 pages (`hptot`), and the amount of huge page memory that is really
823 in use (`hpuse`).
824 If the screen-width does not allow all of these counters, only a
826 relevant subset is shown.
827 SWP Swap occupation and overcommit info.
829 This line contains the total amount of swap space on disk (`tot')
830 and the amount of free swap space (`free').
831 Furthermore the committed virtual memory space (`vmcom') and the
832 maximum limit of the committed space (`vmlim', which is by default
833 swap size plus 50% of memory size) is shown. The committed space
834 is the reserved virtual space for all allocations of private mem‐
835 ory space for processes. The kernel only verifies whether the com‐
836 mitted space exceeds the limit if strict overcommit handling is
837 configured (vm.overcommit_memory is 2).
838 PAG Paging frequency.
840 This line contains the number of scanned pages (`scan') due to the
841 fact that free memory drops below a particular threshold and the
842 number times that the kernel tries to reclaim pages due to an
843 urgent need (`stall').
844 Also the number of memory pages the system read from swap space
845 (`swin') and the number of memory pages the system wrote to swap
846 space (`swout') are shown.
847 PSI Pressure Stall Information.
849 This line contains three percentages per category: average pres‐
850 sure percentage over the last 10, 60 and 300 seconds (separated by
851 slashes).
852 The categories are: CPU for 'some' (`cs'), memory for 'some'
853 (`ms'), memory for 'full' (`mf'), I/O for 'some' (`is'), and I/O
854 for 'full' (`if').
855 LVM/MDD/DSK
857 Logical volume/multiple device/disk utilization.
858 Per active unit one line is produced, sorted on unit activity.
859 Such line shows the name (e.g. VolGroup00-lvtmp for a logical vol‐
860 ume or sda for a hard disk), the busy percentage i.e. the portion
861 of time that the unit was busy handling requests (`busy'), the
862 number of read requests issued (`read'), the number of write
863 requests issued (`write'), the number of KiBytes per read
864 (`KiB/r'), the number of KiBytes per write (`KiB/w'), the number
865 of MiBytes per second throughput for reads (`MBr/s'), the number
866 of MiBytes per second throughput for writes (`MBw/s'), the average
867 queue depth (`avq') and the average number of milliseconds needed
868 by a request (`avio') for seek, latency and data transfer.
869 If the screen-width does not allow all of these counters, only a
870 relevant subset is shown.
871 The number of lines showing the units can be limited per class
873 (LVM, MDD or DSK) with the 'l' key or statically (see separate
874 man-page of atoprc). By specifying the value 0 for a particular
875 class, no lines will be shown any more for that class.
876 NFM Network Filesystem (NFS) mount at the client side.
878 For each NFS-mounted filesystem, a line is shown that contains the
879 mounted server directory, the name of the server (`srv'), the
880 total number of bytes physically read from the server (`read') and
881 the total number of bytes physically written to the server
882 (`write'). Data transfer is subdivided in the number of bytes
883 read via normal read() system calls (`nread'), the number of bytes
884 written via normal read() system calls (`nwrit'), the number of
885 bytes read via direct I/O (`dread'), the number of bytes written
886 via direct I/O (`dwrit'), the number of bytes read via memory
887 mapped I/O pages (`mread'), and the number of bytes written via
888 memory mapped I/O pages (`mwrit').
889 NFC Network Filesystem (NFS) client side counters.
891 This line contains the number of RPC calls issues by local pro‐
892 cesses (`rpc'), the number of read RPC calls (`read`) and write
893 RPC calls (`rpwrite') issued to the NFS server, the number of RPC
894 calls being retransmitted (`retxmit') and the number of authoriza‐
895 tion refreshes (`autref').
896 NFS Network Filesystem (NFS) server side counters.
898 This line contains the number of RPC calls received from NFS
899 clients (`rpc'), the number of read RPC calls received (`cread`),
900 the number of write RPC calls received (`cwrit'), the number of
901 Megabytes/second returned to read requests by clients (`MBcr/s`),
902 the number of Megabytes/second passed in write requests by clients
903 (`MBcw/s`), the number of network requests handled via TCP
904 (`nettcp'), the number of network requests handled via UDP
905 (`netudp'), the number of reply cache hits (`rchits'), the number
906 of reply cache misses (`rcmiss') and the number of uncached
907 requests (`rcnoca'). Furthermore some error counters indicating
908 the number of requests with a bad format (`badfmt') or a bad
909 authorization (`badaut'), and a counter indicating the number of
910 bad clients (`badcln').
911 NET Network utilization (TCP/IP).
913 One line is shown for activity of the transport layer (TCP and
914 UDP), one line for the IP layer and one line per active interface.
915 For the transport layer, counters are shown concerning the number
916 of received TCP segments including those received in error
917 (`tcpi'), the number of transmitted TCP segments excluding those
918 containing only retransmitted octets (`tcpo'), the number of UDP
919 datagrams received (`udpi'), the number of UDP datagrams transmit‐
920 ted (`udpo'), the number of active TCP opens (`tcpao'), the number
921 of passive TCP opens (`tcppo'), the number of TCP output retrans‐
922 missions (`tcprs'), the number of TCP input errors (`tcpie'), the
923 number of TCP output resets (`tcpor'), the number of UDP no ports
924 (`udpnp'), and the number of UDP input errors (`udpie').
925 If the screen-width does not allow all of these counters, only a
926 relevant subset is shown.
927 These counters are related to IPv4 and IPv6 combined.
928 For the IP layer, counters are shown concerning the number of IP
930 datagrams received from interfaces, including those received in
931 error (`ipi'), the number of IP datagrams that local higher-layer
932 protocols offered for transmission (`ipo'), the number of received
933 IP datagrams which were forwarded to other interfaces (`ipfrw'),
934 the number of IP datagrams which were delivered to local higher-
935 layer protocols (`deliv'), the number of received ICMP datagrams
936 (`icmpi'), and the number of transmitted ICMP datagrams (`icmpo').
937 If the screen-width does not allow all of these counters, only a
938 relevant subset is shown.
939 These counters are related to IPv4 and IPv6 combined.
940 For every active network interface one line is shown, sorted on
942 the interface activity. Such line shows the name of the interface
943 and its busy percentage in the first column. The busy percentage
944 for half duplex is determined by comparing the interface speed
945 with the number of bits transmitted and received per second; for
946 full duplex the interface speed is compared with the highest of
947 either the transmitted or the received bits. When the interface
948 speed can not be determined (e.g. for the loopback interface),
949 `---' is shown instead of the percentage.
950 Furthermore the number of received packets (`pcki'), the number of
951 transmitted packets (`pcko'), the line speed of the interface
952 (`sp'), the effective amount of bits received per second (`si'),
953 the effective amount of bits transmitted per second (`so'), the
954 number of collisions (`coll'), the number of received multicast
955 packets (`mlti'), the number of errors while receiving a packet
956 (`erri'), the number of errors while transmitting a packet
957 (`erro'), the number of received packets dropped (`drpi'), and the
958 number of transmitted packets dropped (`drpo').
959 If the screen-width does not allow all of these counters, only a
960 relevant subset is shown.
961 The number of lines showing the network interfaces can be limited.
962 IFB Infiniband utilization.
964 For every active Infiniband port one line is shown, sorted on
965 activity. Such line shows the name of the port and its busy per‐
966 centage in the first column. The busy percentage is determined by
967 taking the highest of either the transmitted or the received bits
968 during the interval, multiplying that value by the number of lanes
969 and comparing it against the maximum port speed.
970 Furthermore the number of received packets divided by the number
971 of lanes (`pcki'), the number of transmitted packets divided by
972 the number of lanes (`pcko'), the maximum line speed (`sp'), the
973 effective amount of bits received per second (`si'), the effective
974 amount of bits transmitted per second (`so'), and the number of
975 lanes (`lanes').
976 If the screen-width does not allow all of these counters, only a
977 relevant subset is shown.
978 The number of lines showing the Infiniband ports can be limited.
979
981 Following the system level information, the processes are shown from
982 which the resource utilization has changed during the last interval.
983 These processes might have used cpu time or issued disk or network
984 requests. However a process is also shown if part of it has been paged
985 out due to lack of memory (while the process itself was in sleep
986 state).
987 Per process the following fields may be shown (in alphabetical order),
989 depending on the current output mode as described in the section INTER‐
990 ACTIVE COMMANDS and depending on the current width of your window:
991 AVGRSZ The average size of one read-action on disk.
993 AVGWSZ The average size of one write-action on disk.
995 BANDWI Total bandwidth for received TCP and UDP packets consumed by
997 this process (bits-per-second). This value can be compared
998 with the value `si' on interface level (used bandwidth per
999 interface).
1000 This information will only be shown when the kernel module
1001 `netatop' is loaded.
1002 BANDWO Total bandwidth for sent TCP and UDP packets consumed by this
1004 process (bits-per-second). This value can be compared with
1005 the value `so' on interface level (used bandwidth per inter‐
1006 face).
1007 This information will only be shown when the kernel module
1008 `netatop' is loaded.
1009 CID Container ID (Docker) of 12 hexadecimal digits, referring to
1011 the container in which the process/thread is running. If a
1012 process has been started and finished during the last inter‐
1013 val, a `?' is shown because the container ID is not part of
1014 the standard process accounting record.
1015 CMD The name of the process. This name can be surrounded by
1017 "less/greater than" signs (`<name>') which means that the
1018 process has finished during the last interval.
1019 Behind the abbreviation `CMD' in the header line, the current
1020 page number and the total number of pages of the
1021 process/thread list are shown.
1022 COMMAND-LINE
1024 The full command line of the process (including arguments). If
1025 the length of the command line exceeds the length of the
1026 screen line, the arrow keys -> and <- can be used for horizon‐
1027 tal scroll.
1028 Behind the verb `COMMAND-LINE' in the header line, the current
1029 page number and the total number of pages of the
1030 process/thread list are shown.
1031 CPU The occupation percentage of this process related to the
1033 available capacity for this resource on system level.
1034 CPUNR The identification of the CPU the (main) thread is running on
1036 or has recently been running on.
1037 CTID Container ID (OpenVZ). If a process has been started and fin‐
1039 ished during the last interval, a `?' is shown because the
1040 container ID is not part of the standard process accounting
1041 record.
1042 DSK The occupation percentage of this process related to the total
1044 load that is produced by all processes (i.e. total disk
1045 accesses by all processes during the last interval).
1046 This information is shown when per process "storage account‐
1047 ing" is active in the kernel.
1048 EGID Effective group-id under which this process executes.
1050 ENDATE Date that the process has been finished. If the process is
1052 still running, this field shows `active'.
1053 ENTIME Time that the process has been finished. If the process is
1055 still running, this field shows `active'.
1056 ENVID Virtual environment identified (OpenVZ only).
1058 EUID Effective user-id under which this process executes.
1060 EXC The exit code of a terminated process (second position of col‐
1062 umn `ST' is E) or the fatal signal number (second position of
1063 column `ST' is S or C).
1064 FSGID Filesystem group-id under which this process executes.
1066 FSUID Filesystem user-id under which this process executes.
1068 GPU When the atopgpud daemon does not run with root privileges,
1070 the GPU percentage reflects the GPU memory occupation percent‐
1071 age (memory of all GPUs is 100%).
1072 When the atopgpud daemon runs with root privileges, the GPU
1073 percentage reflects the GPU busy percentage.
1074 GPUBUSY Busy percentage on all GPUs (one GPU is 100%).
1076 When the atopgpud daemon does not run with root privileges,
1077 this value is not available.
1078 GPUNUMS Comma-separated list of GPUs used by the process during the
1080 interval. When the comma-separated list exceeds the width of
1081 the column, a hexadecimal value is shown.
1082 MAJFLT The number of page faults issued by this process that have
1084 been solved by creating/loading the requested memory page.
1085 MEM The occupation percentage of this process related to the
1087 available capacity for this resource on system level.
1088 MEMAVG Average memory occupation during the interval on all used
1090 GPUs.
1091 MEMBUSY Busy percentage of memory on all GPUs (one GPU is 100%), i.e.
1093 the time needed for read and write accesses on memory.
1094 When the atopgpud daemon does not run with root privileges,
1095 this value is not available.
1096 MEMNOW Memory occupation at the moment of the sample on all used
1098 GPUs.
1099 MINFLT The number of page faults issued by this process that have
1101 been solved by reclaiming the requested memory page from the
1102 free list of pages.
1103 NET The occupation percentage of this process related to the total
1105 load that is produced by all processes (i.e. consumed network
1106 bandwidth of all processes during the last interval).
1107 This information will only be shown when kernel module
1108 `netatop' is loaded.
1109 NICE The more or less static priority that can be given to a
1111 process on a scale from -20 (high priority) to +19 (low prior‐
1112 ity).
1113 NPROCS The number of active and terminated processes accumulated for
1115 this user or program.
1116 PID Process-id. If a process has been started and finished during
1118 the last interval, a `?' is shown because the process-id is
1119 not part of the standard process accounting record.
1120 POLI The policies 'norm' (normal, which is SCHED_OTHER), 'btch'
1122 (batch) and 'idle' refer to timesharing processes. The poli‐
1123 cies 'fifo' (SCHED_FIFO) and 'rr' (round robin, which is
1124 SCHED_RR) refer to realtime processes.
1125 PPID Parent process-id. If a process has been started and finished
1127 during the last interval, value 0 is shown because the parent
1128 process-id is not part of the standard process accounting
1129 record.
1130 PRI The process' priority ranges from 0 (highest priority) to 139
1132 (lowest priority). Priority 0 to 99 are used for realtime pro‐
1133 cesses (fixed priority independent of their behavior) and pri‐
1134 ority 100 to 139 for timesharing processes (variable priority
1135 depending on their recent CPU consumption and the nice value).
1136 PSIZE The proportional memory size of this process (or user).
1138 Every process shares resident memory with other processes.
1139 E.g. when a particular program is started several times, the
1140 code pages (text) are only loaded once in memory and shared by
1141 all incarnations. Also the code of shared libraries is shared
1142 by all processes using that shared library, as well as shared
1143 memory and memory-mapped files. For the PSIZE calculation of
1144 a process, the resident memory of a process that is shared
1145 with other processes is divided by the number of sharers.
1146 This means, that every process is accounted for a proportional
1147 part of that memory. Accumulating the PSIZE values of all pro‐
1148 cesses in the system gives a reliable impression of the total
1149 resident memory consumed by all processes.
1150 Since gathering of all values that are needed to calculate the
1151 PSIZE is a relatively time-consuming task, the 'R' key (or
1152 '-R' flag) should be active. Gathering these values also
1153 requires superuser privileges (otherwise '?K' is shown in the
1154 output).
1155 If a process has finished during the last interval, no value
1156 is shown since the proportional memory size is not part of the
1157 standard process accounting record.
1158 RDDSK When the kernel maintains standard io statistics (>= 2.6.20):
1160 The read data transfer issued physically on disk (so reading
1161 from the disk cache is not accounted for).
1162 Unfortunately, the kernel aggregates the data tranfer of a
1163 process to the data transfer of its parent process when termi‐
1164 nating, so you might see transfers for (parent) processes like
1165 cron, bash or init, that are not really issued by them.
1166 RGID The real group-id under which the process executes.
1168 RGROW The amount of resident memory that the process has grown dur‐
1170 ing the last interval. A resident growth can be caused by
1171 touching memory pages which were not physically created/loaded
1172 before (load-on-demand). Note that a resident growth can also
1173 be negative e.g. when part of the process is paged out due to
1174 lack of memory or when the process frees dynamically allocated
1175 memory. For a process which started during the last interval,
1176 the resident growth reflects the total resident size of the
1177 process at that moment.
1178 If a process has finished during the last interval, no value
1179 is shown since resident memory occupation is not part of the
1180 standard process accounting record.
1181 RNET The number of TCP- and UDP packets received by this process.
1183 This information will only be shown when kernel module
1184 `netatop' is installed.
1185 If a process has finished during the last interval, no value
1186 is shown since network counters are not part of the standard
1187 process accounting record.
1188 RSIZE The total resident memory usage consumed by this process (or
1190 user). Notice that the RSIZE of a process includes all resi‐
1191 dent memory used by that process, even if certain memory parts
1192 are shared with other processes (see also the explanation of
1193 PSIZE).
1194 If a process has finished during the last interval, no value
1195 is shown since resident memory occupation is not part of the
1196 standard process accounting record.
1197 RTPR Realtime priority according the POSIX standard. Value can be
1199 0 for a timesharing process (policy 'norm', 'btch' or 'idle')
1200 or ranges from 1 (lowest) till 99 (highest) for a realtime
1201 process (policy 'rr' or 'fifo').
1202 RUID The real user-id under which the process executes.
1204 S The current state of the (main) thread: `R' for running (cur‐
1206 rently processing or in the runqueue), `S' for sleeping inter‐
1207 ruptible (wait for an event to occur), `D' for sleeping non-
1208 interruptible, `Z' for zombie (waiting to be synchronized with
1209 its parent process), `T' for stopped (suspended or traced),
1210 `W' for swapping, and `E' (exit) for processes which have fin‐
1211 ished during the last interval.
1212 SGID The saved group-id of the process.
1214 SNET The number of TCP and UDP packets transmitted by this process.
1216 This information will only be shown when the kernel module
1217 `netatop' is loaded.
1218 ST The status of a process.
1220 The first position indicates if the process has been started
1221 during the last interval (the value N means 'new process').
1222 The second position indicates if the process has been finished
1224 during the last interval.
1225 The value E means 'exit' on the process' own initiative; the
1226 exit code is displayed in the column `EXC'.
1227 The value S means that the process has been terminated unvol‐
1228 untarily by a signal; the signal number is displayed in the in
1229 the column `EXC'.
1230 The value C means that the process has been terminated unvol‐
1231 untarily by a signal, producing a core dump in its current
1232 directory; the signal number is displayed in the column `EXC'.
1233 STDATE The start date of the process.
1235 STTIME The start time of the process.
1237 SUID The saved user-id of the process.
1239 SWAPSZ The swap space consumed by this process (or user).
1241 SYSCPU CPU time consumption of this process in system mode (kernel
1243 mode), usually due to system call handling.
1244 TCPRASZ The average size of a received TCP buffer in bytes. This
1246 information will only be shown when the kernel module
1247 `netatop' is loaded.
1248 TCPRCV The number of TCP packets received for this process. This
1250 information will only be shown when the kernel module
1251 `netatop' is loaded.
1252 TCPSASZ The average size of a transmitted TCP buffer in bytes. This
1254 information will only be shown when the kernel module
1255 `netatop' is loaded.
1256 TCPSND The number of TCP packets transmitted for this process. This
1258 information will only be shown when the kernel module
1259 `netatop' is loaded.
1260 THR Total number of threads within this process. All related
1262 threads are contained in a thread group, represented by atop
1263 as one line or as a separate line when the 'y' key (or -y
1264 flag) is active.
1265 On Linux 2.4 systems it is hardly possible to determine which
1267 threads (i.e. processes) are related to the same thread group.
1268 Every thread is represented by atop as a separate line.
1269 TID Thread-id. All threads within a process run with the same PID
1271 but with a different TID. This value is shown for individual
1272 threads in multi-threaded processes (when using the key 'y').
1273 TRUN Number of threads within this process that are in the state
1275 'running' (R).
1276 TSLPI Number of threads within this process that are in the state
1278 'interruptible sleeping' (S).
1279 TSLPU Number of threads within this process that are in the state
1281 'uninterruptible sleeping' (D).
1282 UDPRASZ The average size of a received UDP packet in bytes. This
1284 information will only be shown when the kernel module
1285 `netatop' is loaded.
1286 UDPRCV The number of UDP packets received by this process. This
1288 information will only be shown when the kernel module
1289 `netatop' is loaded.
1290 UDPSASZ The average size of a transmitted UDP packets in bytes. This
1292 information will only be shown when the kernel module
1293 `netatop' is loaded.
1294 UDPSND The number of UDP packets transmitted by this process. This
1296 information will only be shown when the kernel module
1297 `netatop' is loaded.
1298 USRCPU CPU time consumption of this process in user mode, due to pro‐
1300 cessing the own program text.
1301 VDATA The virtual memory size of the private data used by this
1303 process (including heap and shared library data).
1304 VGROW The amount of virtual memory that the process has grown during
1306 the last interval. A virtual growth can be caused by e.g.
1307 issueing a malloc() or attaching a shared memory segment. Note
1308 that a virtual growth can also be negative by e.g. issueing a
1309 free() or detaching a shared memory segment. For a process
1310 which started during the last interval, the virtual growth
1311 reflects the total virtual size of the process at that moment.
1312 If a process has finished during the last interval, no value
1313 is shown since virtual memory occupation is not part of the
1314 standard process accounting record.
1315 VPID Virtual process-id (within an OpenVZ container). If a process
1317 has been started and finished during the last interval, a `?'
1318 is shown because the virtual process-id is not part of the
1319 standard process accounting record.
1320 VSIZE The total virtual memory usage consumed by this process (or
1322 user).
1323 If a process has finished during the last interval, no value
1324 is shown since virtual memory occupation is not part of the
1325 standard process accounting record.
1326 VSLIBS The virtual memory size of the (shared) text of all shared
1328 libraries used by this process.
1329 VSTACK The virtual memory size of the (private) stack used by this
1331 process
1332 VSTEXT The virtual memory size of the (shared) text of the executable
1334 program.
1335 WRDSK When the kernel maintains standard io statistics (>= 2.6.20):
1337 The write data transfer issued physically on disk (so writing
1338 to the disk cache is not accounted for). This counter is
1339 maintained for the application process that writes its data to
1340 the cache (assuming that this data is physically transferred
1341 to disk later on). Notice that disk I/O needed for swapping is
1342 not taken into account.
1343 Unfortunately, the kernel aggregates the data tranfer of a
1344 process to the data transfer of its parent process when termi‐
1345 nating, so you might see transfers for (parent) processes like
1346 cron, bash or init, that are not really issued by them.
1347 WCANCL When the kernel maintains standard io statistics (>= 2.6.20):
1349 The write data transfer previously accounted for this process
1350 or another process that has been cancelled. Suppose that a
1351 process writes new data to a file and that data is removed
1352 again before the cache buffers have been flushed to disk.
1353 Then the original process shows the written data as WRDSK,
1354 while the process that removes/truncates the file shows the
1355 unflushed removed data as WCANCL.
1356
1358 With the flag -P followed by a list of one or more labels (comma-sepa‐
1359 rated), parseable output is produced for each sample. The labels that
1360 can be specified for system-level statistics correspond to the labels
1361 (first verb of each line) that can be found in the interactive output:
1362 "CPU", "cpu", "CPL", "GPU", "MEM", "SWP", "PAG", "PSI", "LVM", "MDD",
1363 "DSK", "NFM", "NFC", "NFS", "NET" and "IFB".
1364 For process-level statistics special labels are introduced: "PRG" (gen‐
1365 eral), "PRC" (cpu), "PRE" (GPU), "PRM" (memory), "PRD" (disk, only if
1366 "storage accounting" is active) and "PRN" (network, only if the kernel
1367 module 'netatop' has been installed).
1368 With the label "ALL", all system and process level statistics are
1369 shown.
1370 For every interval all requested lines are shown whereafter atop shows
1372 a line just containing the label "SEP" as a separator before the lines
1373 for the next sample are generated.
1374 When a sample contains the values since boot, atop shows a line just
1375 containing the label "RESET" before the lines for this sample are gen‐
1376 erated.
1377 The first part of each output-line consists of the following six
1379 fields: label (the name of the label), host (the name of this machine),
1380 epoch (the time of this interval as number of seconds since 1-1-1970),
1381 date (date of this interval in format YYYY/MM/DD), time (time of this
1382 interval in format HH:MM:SS), and interval (number of seconds elapsed
1383 for this interval).
1384 The subsequent fields of each output-line depend on the label:
1386 CPU Subsequent fields: total number of clock-ticks per second for
1388 this machine, number of processors, consumption for all CPUs
1389 in system mode (clock-ticks), consumption for all CPUs in user
1390 mode (clock-ticks), consumption for all CPUs in user mode for
1391 niced processes (clock-ticks), consumption for all CPUs in
1392 idle mode (clock-ticks), consumption for all CPUs in wait mode
1393 (clock-ticks), consumption for all CPUs in irq mode (clock-
1394 ticks), consumption for all CPUs in softirq mode (clock-
1395 ticks), consumption for all CPUs in steal mode (clock-ticks),
1396 consumption for all CPUs in guest mode (clock-ticks) overlap‐
1397 ping user mode, frequency of all CPUs, frequency percentage of
1398 all CPUs, instructions executed by all CPUs and cycles for all
1399 CPUs.
1400 cpu Subsequent fields: total number of clock-ticks per second for
1402 this machine, processor-number, consumption for this CPU in
1403 system mode (clock-ticks), consumption for this CPU in user
1404 mode (clock-ticks), consumption for this CPU in user mode for
1405 niced processes (clock-ticks), consumption for this CPU in
1406 idle mode (clock-ticks), consumption for this CPU in wait mode
1407 (clock-ticks), consumption for this CPU in irq mode (clock-
1408 ticks), consumption for this CPU in softirq mode (clock-
1409 ticks), consumption for this CPU in steal mode (clock-ticks),
1410 consumption for this CPU in guest mode (clock-ticks) overlap‐
1411 ping user mode, frequency of this CPU, frequency percentage of
1412 this CPU, instructions executed by this CPU and cycles for
1413 this CPU.
1414 CPL Subsequent fields: number of processors, load average for last
1416 minute, load average for last five minutes, load average for
1417 last fifteen minutes, number of context-switches, and number
1418 of device interrupts.
1419 GPU Subsequent fields: GPU number, bus-id string, type of GPU
1421 string, GPU busy percentage during last second (-1 if not
1422 available), memory busy percentage during last second (-1 if
1423 not available), total memory size (KiB), used memory (KiB) at
1424 this moment, number of samples taken during interval, cumula‐
1425 tive GPU busy percentage during the interval (to be divided by
1426 the number of samples for the average busy percentage, -1 if
1427 not available), cumulative memory busy percentage during the
1428 interval (to be divided by the number of samples for the aver‐
1429 age busy percentage, -1 if not available), and cumulative mem‐
1430 ory occupation during the interval (to be divided by the num‐
1431 ber of samples for the average occupation).
1432 MEM Subsequent fields: page size for this machine (in bytes), size
1434 of physical memory (pages), size of free memory (pages), size
1435 of page cache (pages), size of buffer cache (pages), size of
1436 slab (pages), dirty pages in cache (pages), reclaimable part
1437 of slab (pages), total size of vmware's balloon pages (pages),
1438 total size of shared memory (pages), size of resident shared
1439 memory (pages), size of swapped shared memory (pages), huge
1440 page size (in bytes), total size of huge pages (huge pages),
1441 and size of free huge pages (huge pages).
1442 SWP Subsequent fields: page size for this machine (in bytes), size
1444 of swap (pages), size of free swap (pages), 0 (future use),
1445 size of committed space (pages), and limit for committed space
1446 (pages).
1447 PAG Subsequent fields: page size for this machine (in bytes), num‐
1449 ber of page scans, number of allocstalls, 0 (future use), num‐
1450 ber of swapins, and number of swapouts.
1451 PSI Subsequent fields: PSI statistics present on this system (n or
1453 y), CPU some avg10, CPU some avg60, CPU some avg300, CPU some
1454 accumulated microseconds during interval, memory some avg10,
1455 memory some avg60, memory some avg300, memory some accumulated
1456 microseconds during interval, memory full avg10, memory full
1457 avg60, memory full avg300, memory full accumulated microsec‐
1458 onds during interval, I/O some avg10, I/O some avg60, I/O some
1459 avg300, I/O some accumulated microseconds during interval, I/O
1460 full avg10, I/O full avg60, I/O full avg300, and I/O full
1461 accumulated microseconds during interval.
1462 LVM/MDD/DSK
1464 For every logical volume/multiple device/hard disk one line is
1465 shown.
1466 Subsequent fields: name, number of milliseconds spent for I/O,
1467 number of reads issued, number of sectors transferred for
1468 reads, number of writes issued, and number of sectors trans‐
1469 ferred for write.
1470 NFM Subsequent fields: mounted NFS filesystem, total number of
1472 bytes read, total number of bytes written, number of bytes
1473 read by normal system calls, number of bytes written by normal
1474 system calls, number of bytes read by direct I/O, number of
1475 bytes written by direct I/O, number of pages read by memory-
1476 mapped I/O, and number of pages written by memory-mapped I/O.
1477 NFC Subsequent fields: number of transmitted RPCs, number of
1479 transmitted read RPCs, number of transmitted write RPCs, num‐
1480 ber of RPC retransmissions, and number of authorization
1481 refreshes.
1482 NFS Subsequent fields: number of handled RPCs, number of received
1484 read RPCs, number of received write RPCs, number of bytes read
1485 by clients, number of bytes written by clients, number of RPCs
1486 with bad format, number of RPCs with bad authorization, number
1487 of RPCs from bad client, total number of handled network
1488 requests, number of handled network requests via TCP, number
1489 of handled network requests via UDP, number of handled TCP
1490 connections, number of hits on reply cache, number of misses
1491 on reply cache, and number of uncached requests.
1492 NET First one line is produced for the upper layers of the TCP/IP
1494 stack.
1495 Subsequent fields: the verb "upper", number of packets
1496 received by TCP, number of packets transmitted by TCP, number
1497 of packets received by UDP, number of packets transmitted by
1498 UDP, number of packets received by IP, number of packets
1499 transmitted by IP, number of packets delivered to higher lay‐
1500 ers by IP, and number of packets forwarded by IP.
1501 Next one line is shown for every interface.
1503 Subsequent fields: name of the interface, number of packets
1504 received by the interface, number of bytes received by the
1505 interface, number of packets transmitted by the interface,
1506 number of bytes transmitted by the interface, interface speed,
1507 and duplex mode (0=half, 1=full).
1508 IFB Subsequent fields: name of the InfiniBand interface, port num‐
1510 ber, number of lanes, maximum rate (Mbps), number of bytes
1511 received, number of bytes transmitted, number of packets
1512 received, and number of packets transmitted.
1513 PRG For every process one line is shown.
1515 Subsequent fields: PID (unique ID of task), name (between
1516 brackets), state, real uid, real gid, TGID (group number of
1517 related tasks/threads), total number of threads, exit code (in
1518 case of fatal signal: signal number + 256), start time
1519 (epoch), full command line (between brackets), PPID, number of
1520 threads in state 'running' (R), number of threads in state
1521 'interruptible sleeping' (S), number of threads in state
1522 'uninterruptible sleeping' (D), effective uid, effective gid,
1523 saved uid, saved gid, filesystem uid, filesystem gid, elapsed
1524 time (hertz), is_process (y/n), OpenVZ virtual pid (VPID),
1525 OpenVZ container id (CTID) and Docker container id (CID).
1526 PRC For every process one line is shown.
1528 Subsequent fields: PID, name (between brackets), state, total
1529 number of clock-ticks per second for this machine, CPU-con‐
1530 sumption in user mode (clockticks), CPU-consumption in system
1531 mode (clockticks), nice value, priority, realtime priority,
1532 scheduling policy, current CPU, sleep average, TGID (group
1533 number of related tasks/threads) and is_process (y/n).
1534 PRE For every process one line is shown.
1536 Subsequent fields: PID, name (between brackets), process
1537 state, GPU state (A for active, E for exited, N for no GPU
1538 user), number of GPUs used by this process, bitlist reflecting
1539 used GPUs, GPU busy percentage during interval, memory busy
1540 percentage during interval, memory occupation (KiB) at this
1541 moment cumulative memory occupation (KiB) during interval, and
1542 number of samples taken during interval.
1543 PRM For every process one line is shown.
1545 Subsequent fields: PID, name (between brackets), state, page
1546 size for this machine (in bytes), virtual memory size
1547 (Kbytes), resident memory size (Kbytes), shared text memory
1548 size (Kbytes), virtual memory growth (Kbytes), resident memory
1549 growth (Kbytes), number of minor page faults, number of major
1550 page faults, virtual library exec size (Kbytes), virtual data
1551 size (Kbytes), virtual stack size (Kbytes), swap space used
1552 (Kbytes), TGID (group number of related tasks/threads),
1553 is_process (y/n) and proportional set size (Kbytes) if in 'R'
1554 option is specified.
1555 PRD For every process one line is shown.
1557 Subsequent fields: PID, name (between brackets), state, obso‐
1558 leted kernel patch installed ('n'), standard io statistics
1559 used ('y' or 'n'), number of reads on disk, cumulative number
1560 of sectors read, number of writes on disk, cumulative number
1561 of sectors written, cancelled number of written sectors, TGID
1562 (group number of related tasks/threads) and is_process (y/n).
1563 If the standard I/O statistics (>= 2.6.20) are not used, the
1564 disk I/O counters per process are not relevant. The counters
1565 'number of reads on disk' and 'number of writes on disk' are
1566 obsoleted anyhow.
1567 PRN For every process one line is shown.
1569 Subsequent fields: PID, name (between brackets), state, kernel
1570 module 'netatop' loaded ('y' or 'n'), number of TCP-packets
1571 transmitted, cumulative size of TCP-packets transmitted, num‐
1572 ber of TCP-packets received, cumulative size of TCP-packets
1573 received, number of UDP-packets transmitted, cumulative size
1574 of UDP-packets transmitted, number of UDP-packets received,
1575 cumulative size of UDP-packets transmitted, number of raw
1576 packets transmitted (obsolete, always 0), number of raw pack‐
1577 ets received (obsolete, always 0), TGID (group number of
1578 related tasks/threads) and is_process (y/n).
1579 If the kernel module is not active, the network I/O counters
1580 per process are not relevant.
1581
1583 By sending the SIGUSR1 signal to atop a new sample will be forced, even
1584 if the current timer interval has not exceeded yet. The behavior is
1585 similar to pressing the `t` key in an interactive session.
1586 By sending the SIGUSR2 signal to atop a final sample will be forced
1588 after which atop will terminate.
1589
1591 To monitor the current system load interactively with an interval of 5
1592 seconds:
1593 atop 5
1595 To monitor the system load and write it to a file (in plain ASCII) with
1597 an interval of one minute during half an hour with active processes
1598 sorted on memory consumption:
1599 atop -M 60 30 > /log/atop.mem
1601 Store information about the system and process activity in binary com‐
1603 pressed form to a file with an interval of ten minutes during an hour:
1604 atop -w /tmp/atop.raw 600 6
1606 View the contents of this file interactively:
1608 atop -r /tmp/atop.raw
1610 View the processor and disk utilization of this file in parseable for‐
1612 mat:
1613 atop -PCPU,DSK -r /tmp/atop.raw
1615 View the contents of today's standard logfile interactively:
1617 atop -r
1619 View the contents of the standard logfile of the day before yesterday
1621 interactively:
1622 atop -r yy
1624 View the contents of the standard logfile of 2014, June 7 from 02:00 PM
1626 onwards interactively:
1627 atop -r 20140607 -b 14:00
1629
1631 /var/run/pacct_shadow.d/
1632 Directory containing the process accounting shadow files that are
1633 used by atop when the atopacctd daemon is active.
1634 /var/cache/atop.d/atop.acct
1636 File in which the kernel writes the accounting records when atop
1637 itself has activated the process accounting mechanism.
1638 /etc/atoprc
1640 Configuration file containing system-wide default values. See
1641 related man-page.
1642 ~/.atoprc
1644 Configuration file containing personal default values. See
1645 related man-page.
1646 /etc/default/atop
1648 Configuration file to overrule the settings of atop that runs in
1649 the background to create the daily logfile. This file is created
1650 when atop is installed. The default settings are:
1651 LOGOPTS="-R"
1653 LOGINTERVAL=600
1654 LOGGENERATIONS=28
1655 /var/log/atop/atop_YYYYMMDD
1657 Raw file, where YYYYMMDD are digits representing the current date.
1658 This name is used by atop running in the background as default
1659 name for the output file, and by atop as default name for the
1660 input file when using the -r flag.
1661 All binary system and process level data in this file has been
1662 stored in compressed format.
1663 /var/run/netatop.log
1665 File that contains the netpertask structs containing the network
1666 counters of exited processes. These structs are written by the
1667 netatopd daemon and read by atop after reading the standard
1668 process accounting records.
1669
1671 atopsar(1), atopconvert(1), atoprc(5), atopacctd(8), netatop(4),
1672 netatopd(8), atopgpud(8), logrotate(8)
1673 https://www.atoptool.nl
1674
1676 Gerlof Langeveld (gerlof.langeveld@atoptool.nl)
1677 JC van Winkel
1678Linux November 2019 ATOP(1)