1ATOP(1) General Commands Manual ATOP(1)
2
6 atop - Advanced System & Process Monitor
7
9 Interactive Usage:
10 atop [-g|-m|-d|-n|-u|-p|-s|-c|-v|-o|-y] [-C|-M|-D|-N|-A] [-afFG1xR] [-L
12 linelen] [-Plabel[,label]...] [ interval [ samples ]]
13 Writing and reading raw logfiles:
15 atop -w rawfile [-a] [-S] [ interval [ samples ]]
17 atop -r [ rawfile ] [-b hh:mm ] [-e hh:mm ]
18 [-g|-m|-d|-n|-u|-p|-s|-c|-v|-o|-y] [-C|-M|-D|-N|-A] [-fFG1xR] [-L line‐
19 len] [-Plabel[,label]...]
20
22 The program atop is an interactive monitor to view the load on a Linux
23 system. It shows the occupation of the most critical hardware
24 resources (from a performance point of view) on system level, i.e. cpu,
25 memory, disk and network.
26 It also shows which processes are responsible for the indicated load
27 with respect to cpu and memory load on process level. Disk load is
28 shown per process if "storage accounting" is active in the kernel.
29 Network load is shown per process if the kernel module `netatop' has
30 been installed.
31 Every interval (default: 10 seconds) information is shown about the
33 resource occupation on system level (cpu, memory, disks and network
34 layers), followed by a list of processes which have been active during
35 the last interval (note that all processes that were unchanged during
36 the last interval are not shown, unless the key 'a' has been pressed or
37 unless sorting on memory occupation is done). If the list of active
38 processes does not entirely fit on the screen, only the top of the list
39 is shown (sorted in order of activity).
40 The intervals are repeated till the number of samples (specified as
41 command argument) is reached, or till the key 'q' is pressed in inter‐
42 active mode.
43 When atop is started, it checks whether the standard output channel is
45 connected to a screen, or to a file/pipe. In the first case it produces
46 screen control codes (via the ncurses library) and behaves interac‐
47 tively; in the second case it produces flat ASCII-output.
48 In interactive mode, the output of atop scales dynamically to the cur‐
50 rent dimensions of the screen/window.
51 If the window is resized horizontally, columns will be added or removed
52 automatically. For this purpose, every column has a particular weight.
53 The columns with the highest weights that fit within the current width
54 will be shown.
55 If the window is resized vertically, lines of the process/thread list
56 will be added or removed automatically.
57 Furthermore in interactive mode the output of atop can be controlled by
59 pressing particular keys. However it is also possible to specify such
60 key as flag on the command line. In that case atop switches to the
61 indicated mode on beforehand; this mode can be modified again interac‐
62 tively. Specifying such key as flag is especially useful when running
63 atop with output to a pipe or file (non-interactively). These flags
64 are the same as the keys that can be pressed in interactive mode (see
65 section INTERACTIVE COMMANDS).
66 Additional flags are available to support storage of atop-data in raw
67 format (see section RAW DATA STORAGE).
68
70 With every interval, atop reads the kernel administration to obtain
71 information about all running processes. However, it is likely that
72 during the interval also processes have terminated. These processes
73 might have consumed system resources during this interval as well
74 before they terminated. Therefor, atop tries to read the process
75 accounting records that contain the accounting information of termi‐
76 nated processes and report these processes too. Only when the process
77 accounting mechanism in the kernel is activated, the kernel writes such
78 process accounting record to a file for every process that terminates.
79 There are various ways for atop to get access to the process accounting
81 records (tried in this order):
82 1. When the environment variable ATOPACCT is set, it specifies the
84 name of the process accounting file. In that case, process
85 accounting for this file should have been activated on beforehand.
86 Before opening this file for reading, atop drops its root privi‐
87 leges (if any).
88 When this environment variable is present but its contents is
89 empty, process accounting will not be used at all.
90 2. This is the preferred way of handling process accounting records!
92 When the atopacctd daemon is active, it has activated the process
93 accounting mechanism in the kernel and transfers to original
94 accounting records to shadow files. In that case, atop drops its
95 root privileges and opens the current shadow file for reading.
96 This way is preferred, because the atopacctd daemon maintains full
97 control of the sizes of the original process accounting file (writ‐
98 ten by the kernel) and the shadow files (read by the atop pro‐
99 cesses). For further information, refer to the atopacctd man page.
100 3. When the atopacctd daemon is not active, atop verifies if the
102 process accounting mechanism has been switched on via the separate
103 psacct package. In that case, the file /var/account/pacct is in use
104 as process accounting file and atop opens this file for reading.
105 4. As a last possibility, atop itself tries to activate the process
107 accounting mechanism (requires root privileges) using the file
108 /var/cache/atop.d/atop.acct (to be written by the kernel, to be
109 read by atop itself). Process accounting remains active as long as
110 at least one atop process is alive. Whenever the last atop process
111 stops (either by pressing `q' or by `kill -15'), it deactivates the
112 process accounting mechanism again. Therefor you should never ter‐
113 minate atop by `kill -9', because then it has no chance to stop
114 process accounting. As a result, the accounting file may consume a
115 lot of disk space after a while.
116 To avoid that the process accounting file consumes too much disk
117 space, atop verifies at the end of every sample if the size of the
118 process accounting file exceeds 200 MiB and if this atop process is
119 the only one that is currently using the file. In that case the
120 file is truncated to a size of zero.
121 Notice that root-privileges are required to switch on/off process
123 accounting in the kernel. You can start atop as a root user or
124 specify setuid-root privileges to the executable file. In the lat‐
125 ter case, atop switches on process accounting and drops the root-
126 privileges again.
127 If atop does not run with root-privileges, it does not show infor‐
128 mation about finished processes. It indicates this situation with
129 the message message `no procacct` in the top-right corner (instead
130 of the counter that shows the number of exited processes).
131 When during one interval a lot of processes have finished, atop might
133 grow tremendously in memory when reading all process accounting records
134 at the end of the interval. To avoid such excessive growth, atop will
135 never read more than 50 MiB with process information from the process
136 accounting file per interval (approx. 70000 finished processes). In
137 interactive mode a warning is given whenever processes have been
138 skipped for this reason.
139
141 For the resource consumption on system level, atop uses colors to indi‐
142 cate that a critical occupation percentage has been (almost) reached.
143 A critical occupation percentage means that is likely that this load
144 causes a noticeable negative performance influence for applications
145 using this resource. The critical percentage depends on the type of
146 resource: e.g. the performance influence of a disk with a busy percent‐
147 age of 80% might be more noticeable for applications/user than a CPU
148 with a busy percentage of 90%.
149 Currently atop uses the following default values to calculate a
150 weighted percentage per resource:
151 Processor
153 A busy percentage of 90% or higher is considered `critical'.
154 Disk
156 A busy percentage of 70% or higher is considered `critical'.
157 Network
159 A busy percentage of 90% or higher for the load of an interface is
160 considered `critical'.
161 Memory
163 An occupation percentage of 90% is considered `critical'. Notice
164 that this occupation percentage is the accumulated memory consump‐
165 tion of the kernel (including slab) and all processes; the memory
166 for the page cache (`cache' and `buff' in the MEM-line) and the
167 reclaimable part of the slab (`slrec`) is not implied!
168 If the number of pages swapped out (`swout' in the PAG-line) is
169 larger than 10 per second, the memory resource is considered
170 `critical'. A value of at least 1 per second is considered
171 `almost critical'.
172 If the committed virtual memory exceeds the limit (`vmcom' and
173 `vmlim' in the SWP-line), the SWP-line is colored due to overcom‐
174 mitting the system.
175 Swap
177 An occupation percentage of 80% is considered `critical' because
178 swap space might be completely exhausted in the near future; it is
179 not critical from a performance point-of-view.
180 These default values can be modified in the configuration file (see
182 separate man-page of atoprc).
183 When a resource exceeds its critical occupation percentage, the con‐
185 cerning values in the screen line are colored red by default.
186 When a resource exceeded (default) 80% of its critical percentage (so
187 it is almost critical), the concerning values in the screen line are
188 colored cyan by default. This `almost critical percentage' (one value
189 for all resources) can be modified in the configuration file (see sepa‐
190 rate man-page of atoprc).
191 The default colors red and cyan can be modified in the configuration
192 file as well (see separate man-page of atoprc).
193 With the key 'x' (or flag -x), the use of colors can be suppressed.
195
197 Per-process and per-thread network activity can be measured by the
198 netatop kernel module. You can download this kernel module from the
199 website (mentioned at the end of this manual page) and install it on
200 your system if the kernel version is 2.6.24 or newer.
201 When atop gathers counters for a new interval, it verifies if the
202 netatop module is currently active. If so, atop obtains the relevant
203 network counters from this module and shows the number of sent and
204 received packets per process/thread in the generic screen. Besides,
205 detailed counters can be requested by pressing the `n' key.
206 When the netatopd daemon is running as well, atop also reads the net‐
207 work counters of exited processes that are logged by this daemon (com‐
208 parable with process accounting).
209 More information about the optional netatop kernel module and the
211 netatopd daemon can be found in the concerning man-pages and on the
212 website mentioned at the end of this manual page.
213
215 GPU statistics can be gathered by atopgpud which is a separate data
216 collection daemon process. It gathers cumulative utilization counters
217 of every Nvidia GPU in the system, as well as utilization counters of
218 every process that uses a GPU. When atop notices that the daemon is
219 active, it reads these GPU utilization counters with every interval.
220 The atopgpud daemon is written in Python, so a Python interpreter
222 should be installed on the target system. The Python code of the daemon
223 is compatible with Python version 2 and version 3. For the gathering
224 of the statistics, the pynvml module is used by the daemon. Be sure
225 that this module is installed on the target system before activating
226 the daemon, by running the command as root pip (the command pip might
227 be exchanged by pip3 in case of Python3):
228 pip install nvidia-ml-py
230 The atopgpud daemon is installed by default as part of the atop pack‐
232 age, but it is not automatically enabled. The daemon can be enabled
233 and started now by running the following commands (as root):
234 systemctl enable atopgpu
236 systemctl start atopgpu
237 Find a description about the utilization counters in the section OUTPUT
239 DESCRIPTION.
240
242 When running atop interactively (no output redirection), keys can be
243 pressed to control the output. In general, lower case keys can be used
244 to show other information for the active processes and upper case keys
245 can be used to influence the sort order of the active process/thread
246 list.
247 g Show generic output (default).
249 Per process the following fields are shown in case of a window-
251 width of 80 positions: process-id, cpu consumption during the last
252 interval in system and user mode, the virtual and resident memory
253 growth of the process.
254 The subsequent columns depend on the used kernel:
256 When the kernel supports "storage accounting" (>= 2.6.20), the
257 data transfer for read/write on disk, the status and exit code are
258 shown for each process. When the kernel does not support "storage
259 accounting", the username, number of threads in the thread group,
260 the status and exit code are shown.
261 When the kernel module 'netatop' is loaded, the data transfer for
262 send/receive of network packets is shown for each process.
263 The last columns contain the state, the occupation percentage for
264 the chosen resource (default: cpu) and the process name.
265 When more than 80 positions are available, other information is
267 added.
268 m Show memory related output.
270 Per process the following fields are shown in case of a window-
272 width of 80 positions: process-id, minor and major memory faults,
273 size of virtual shared text, total virtual process size, total
274 resident process size, virtual and resident growth during last
275 interval, memory occupation percentage and process name.
276 When more than 80 positions are available, other information is
278 added.
279 For memory consumption, always all processes are shown (also the
281 processes that were not active during the interval).
282 d Show disk-related output.
284 When "storage accounting" is active in the kernel, the following
286 fields are shown: process-id, amount of data read from disk,
287 amount of data written to disk, amount of data that was written
288 but has been withdrawn again (WCANCL), disk occupation percentage
289 and process name.
290 n Show network related output.
292 Per process the following fields are shown in case of a window-
294 width of 80 positions: process-id, thread-id, total bandwidth for
295 received packets, total bandwidth for sent packets, number of
296 received TCP packets with the average size per packet (in bytes),
297 number of sent TCP packets with the average size per packet (in
298 bytes), number of received UDP packets with the average size per
299 packet (in bytes), number of sent UDP packets with the average
300 size per packet (in bytes), the network occupation percentage and
301 process name.
302 This information can only be shown when kernel module `netatop' is
303 installed.
304 When more than 80 positions are available, other information is
306 added.
307 s Show scheduling characteristics.
309 Per process the following fields are shown in case of a window-
311 width of 80 positions: process-id, number of threads in state
312 'running' (R), number of threads in state 'interruptible sleeping'
313 (S), number of threads in state 'uninterruptible sleeping' (D),
314 scheduling policy (normal timesharing, realtime round-robin, real‐
315 time fifo), nice value, priority, realtime priority, current pro‐
316 cessor, status, exit code, state, the occupation percentage for
317 the chosen resource and the process name.
318 When more than 80 positions are available, other information is
320 added.
321 v Show various process characteristics.
323 Per process the following fields are shown in case of a window-
325 width of 80 positions: process-id, user name and group, start date
326 and time, status (e.g. exit code if the process has finished),
327 state, the occupation percentage for the chosen resource and the
328 process name.
329 When more than 80 positions are available, other information is
331 added.
332 c Show the command line of the process.
334 Per process the following fields are shown: process-id, the occu‐
336 pation percentage for the chosen resource and the command line
337 including arguments.
338 e Show GPU utilization.
340 Per process at least the following fields are shown: process-id,
342 range of GPU numbers on which the process currently runs, GPU busy
343 percentage on all GPUs, memory busy percentage (i.e. read and
344 write accesses on memory) on all GPUs, memory occupation at the
345 moment of the sample, average memory occupation during the sample,
346 and GPU percentage.
347 When the atopgpud daemon does not run with root privileges, the
349 GPU busy percentage and the memory busy percentage are not avail‐
350 able on process level. In that case, the GPU percentage on
351 process level reflects the GPU memory occupation instead of the
352 GPU busy percentage (which is preferred).
353 o Show the user-defined line of the process.
355 In the configuration file the keyword ownprocline can be specified
357 with the description of a user-defined output-line.
358 Refer to the man-page of atoprc for a detailed description.
359 y Show the individual threads within a process (toggle).
361 Single-threaded processes are still shown as one line.
363 For multi-threaded processes, one line represents the process
364 while additional lines show the activity per individual thread (in
365 a different color). Depending on the option 'a' (all or active
366 toggle), all threads are shown or only the threads that were
367 active during the last interval.
368 Whether this key is active or not can be seen in the header line.
369 u Show the process activity accumulated per user.
371 Per user the following fields are shown: number of processes
373 active or terminated during last interval (or in total if combined
374 with command `a'), accumulated cpu consumption during last inter‐
375 val in system and user mode, the current virtual and resident mem‐
376 ory space consumed by active processes (or all processes of the
377 user if combined with command `a').
378 When "storage accounting" is active in the kernel, the accumulated
379 read and write throughput on disk is shown. When the kernel mod‐
380 ule `netatop' has been installed, the number of received and sent
381 network packets are shown.
382 The last columns contain the accumulated occupation percentage for
383 the chosen resource (default: cpu) and the user name.
384 p Show the process activity accumulated per program (i.e. process
386 name).
387 Per program the following fields are shown: number of processes
389 active or terminated during last interval (or in total if combined
390 with command `a'), accumulated cpu consumption during last inter‐
391 val in system and user mode, the current virtual and resident mem‐
392 ory space consumed by active processes (or all processes of the
393 user if combined with command `a').
394 When "storage accounting" is active in the kernel, the accumulated
395 read and write throughput on disk is shown. When the kernel mod‐
396 ule `netatop' has been installed, the number of received and sent
397 network packets are shown.
398 The last columns contain the accumulated occupation percentage for
399 the chosen resource (default: cpu) and the program name.
400 j Show the process activity accumulated per Docker container.
402 Per container the following fields are shown: number of processes
404 active or terminated during last interval (or in total if combined
405 with command `a'), accumulated cpu consumption during last inter‐
406 val in system and user mode, the current virtual and resident mem‐
407 ory space consumed by active processes (or all processes of the
408 user if combined with command `a').
409 When "storage accounting" is active in the kernel, the accumulated
410 read and write throughput on disk is shown. When the kernel mod‐
411 ule `netatop' has been installed, the number of received and sent
412 network packets are shown.
413 The last columns contain the accumulated occupation percentage for
414 the chosen resource (default: cpu) and the Docker container id
415 (CID).
416 C Sort the current list in the order of cpu consumption (default).
418 The one-but-last column changes to ``CPU''.
419 E Sort the current list in the order of GPU utilization (preferred,
421 but only applicable when the atopgpud daemon runs under root priv‐
422 ileges) or the order of GPU memory occupation). The one-but-last
423 column changes to ``GPU''.
424 M Sort the current list in the order of resident memory consumption.
426 The one-but-last column changes to ``MEM''. In case of sorting on
427 memory, the full process list will be shown (not only the active
428 processes).
429 D Sort the current list in the order of disk accesses issued. The
431 one-but-last column changes to ``DSK''.
432 N Sort the current list in the order of network bandwidth (received
434 and transmitted). The one-but-last column changes to ``NET''.
435 A Sort the current list automatically in the order of the most busy
437 system resource during this interval. The one-but-last column
438 shows either ``ACPU'', ``AMEM'', ``ADSK'' or ``ANET'' (the preced‐
439 ing 'A' indicates automatic sorting-order). The most busy
440 resource is determined by comparing the weighted busy-percentages
441 of the system resources, as described earlier in the section COL‐
442 ORS.
443 This option remains valid until another sorting-order is explic‐
444 itly selected again.
445 A sorting-order for disk is only possible when "storage account‐
446 ing" is active. A sorting-order for network is only possible when
447 the kernel module `netatop' is loaded.
448 Miscellaneous interactive commands:
450 ? Request for help information (also the key 'h' can be pressed).
452 V Request for version information (version number and date).
454 R Gather and calculate the proportional set size of processes (tog‐
456 gle). Gathering of all values that are needed to calculate the
457 PSIZE of a process is a relatively time-consuming task, so this
458 key should only be active when analyzing the resident memory con‐
459 sumption of processes.
460 x Suppress colors to highlight critical resources (toggle).
462 Whether this key is active or not can be seen in the header line.
463 z The pause key can be used to freeze the current situation in order
465 to investigate the output on the screen. While atop is paused, the
466 keys described above can be pressed to show other information
467 about the current list of processes. Whenever the pause key is
468 pressed again, atop will continue with a next sample.
469 i Modify the interval timer (default: 10 seconds). If an interval
471 timer of 0 is entered, the interval timer is switched off. In that
472 case a new sample can only be triggered manually by pressing the
473 key 't'.
474 t Trigger a new sample manually. This key can be pressed if the cur‐
476 rent sample should be finished before the timer has exceeded, or
477 if no timer is set at all (interval timer defined as 0). In the
478 latter case atop can be used as a stopwatch to measure the load
479 being caused by a particular application transaction, without
480 knowing on beforehand how many seconds this transaction will last.
481 When viewing the contents of a raw file, this key can be used to
483 show the next sample from the file.
484 T When viewing the contents of a raw file, this key can be used to
486 show the previous sample from the file.
487 b When viewing the contents of a raw file, this key can be used to
489 branch to a certain timestamp within the file (either forward or
490 backward).
491 r Reset all counters to zero to see the system and process activity
493 since boot again.
494 When viewing the contents of a raw file, this key can be used to
496 rewind to the beginning of the file again.
497 U Specify a search string for specific user names as a regular
499 expression. From now on, only (active) processes will be shown
500 from a user which matches the regular expression. The system sta‐
501 tistics are still system wide. If the Enter-key is pressed with‐
502 out specifying a name, (active) processes of all users will be
503 shown again.
504 Whether this key is active or not can be seen in the header line.
505 I Specify a list with one or more PIDs to be selected. From now on,
507 only processes will be shown with a PID which matches one of the
508 given list. The system statistics are still system wide. If the
509 Enter-key is pressed without specifying a PID, all (active) pro‐
510 cesses will be shown again.
511 Whether this key is active or not can be seen in the header line.
512 P Specify a search string for specific process names as a regular
514 expression. From now on, only processes will be shown with a name
515 which matches the regular expression. The system statistics are
516 still system wide. If the Enter-key is pressed without specifying
517 a name, all (active) processes will be shown again.
518 Whether this key is active or not can be seen in the header line.
519 / Specify a specific command line search string as a regular expres‐
521 sion. From now on, only processes will be shown with a command
522 line which matches the regular expression. The system statistics
523 are still system wide. If the Enter-key is pressed without speci‐
524 fying a string, all (active) processes will be shown again.
525 Whether this key is active or not can be seen in the header line.
526 J Specify a Docker container id of 12 (hexadecimal) characters.
528 From now on, only processes will be shown that run in that spe‐
529 cific Docker container (CID). The system statistics are still
530 system wide. If the Enter-key is pressed without specifying a
531 container id, all (active) processes will be shown again.
532 Whether this key is active or not can be seen in the header line.
533 S Specify search strings for specific logical volume names, specific
535 disk names and specific network interface names. All search
536 strings are interpreted as a regular expressions. From now on,
537 only those system resources are shown that match the concerning
538 regular expression. If the Enter-key is pressed without specify‐
539 ing a search string, all (active) system resources of that type
540 will be shown again.
541 Whether this key is active or not can be seen in the header line.
542 a The `all/active' key can be used to toggle between only show‐
544 ing/accumulating the processes that were active during the last
545 interval (default) or showing/accumulating all processes.
546 Whether this key is active or not can be seen in the header line.
547 G By default, atop shows/accumulates the processes that are alive
549 and the processes that are exited during the last interval. With
550 this key (toggle), showing/accumulating the processes that are
551 exited can be suppressed.
552 Whether this key is active or not can be seen in the header line.
553 f Show a fixed (maximum) number of header lines for system resources
555 (toggle). By default only the lines are shown about system
556 resources (CPUs, paging, logical volumes, disks, network inter‐
557 faces) that really have been active during the last interval.
558 With this key you can force atop to show lines of inactive
559 resources as well.
560 Whether this key is active or not can be seen in the header line.
561 F Suppress sorting of system resources (toggle). By default system
563 resources (CPUs, logical volumes, disks, network interfaces) are
564 sorted on utilization.
565 Whether this key is active or not can be seen in the header line.
566 1 Show relevant counters as an average per second (in the format
568 `..../s') instead of as a total during the interval (toggle).
569 Whether this key is active or not can be seen in the header line.
570 l Limit the number of system level lines for the counters per-cpu,
572 the active disks and the network interfaces. By default lines are
573 shown of all CPUs, disks and network interfaces which have been
574 active during the last interval. Limiting these lines can be use‐
575 ful on systems with huge number CPUs, disks or interfaces in order
576 to be able to run atop on a screen/window with e.g. only 24 lines.
577 For all mentioned resources the maximum number of lines can be
578 specified interactively. When using the flag -l the maximum number
579 of per-cpu lines is set to 0, the maximum number of disk lines to
580 5 and the maximum number of interface lines to 3. These values
581 can be modified again in interactive mode.
582 k Send a signal to an active process (a.k.a. kill a process).
584 q Quit the program.
586 PgDn Show the next page of the process/thread list.
588 With the arrow-down key the list can be scrolled downwards with
589 single lines.
590 ^F Show the next page of the process/thread list (forward).
592 With the arrow-down key the list can be scrolled downwards with
593 single lines.
594 PgUp Show the previous page of the process/thread list.
596 With the arrow-up key the list can be scrolled upwards with single
597 lines.
598 ^B Show the previous page of the process/thread list (backward).
600 With the arrow-up key the list can be scrolled upwards with single
601 lines.
602 ^L Redraw the screen.
604
606 In order to store system and process level statistics for long-term
607 analysis (e.g. to check the system load and the active processes run‐
608 ning yesterday between 3:00 and 4:00 PM), atop can store the system and
609 process level statistics in compressed binary format in a raw file with
610 the flag -w followed by the filename. If this file already exists and
611 is recognized as a raw data file, atop will append new samples to the
612 file (starting with a sample which reflects the activity since boot);
613 if the file does not exist, it will be created.
614 All information about processes and threads is stored in the raw file.
615 The interval (default: 10 seconds) and number of samples (default:
616 infinite) can be passed as last arguments. Instead of the number of
617 samples, the flag -S can be used to indicate that atop should finish
618 anyhow before midnight.
619 A raw file can be read and visualized again with the flag -r followed
621 by the filename. If no filename is specified, the file
622 /var/log/atop/atop_YYYYMMDD is opened for input (where YYYYMMDD are
623 digits representing the current date). If a filename is specified in
624 the format YYYYMMDD (representing any valid date), the file
625 /var/log/atop/atop_YYYYMMDD is opened. If a filename with the symbolic
626 name y is specified, yesterday's daily logfile is opened (this can be
627 repeated so 'yyyy' indicates the logfile of four days ago).
628 The samples from the file can be viewed interactively by using the key
629 't' to show the next sample, the key 'T' to show the previous sample,
630 the key 'b' to branch to a particular time or the key 'r' to rewind to
631 the begin of the file.
632 When output is redirected to a file or pipe, atop prints all samples in
633 plain ASCII. The default line length is 80 characters in that case;
634 with the flag -L followed by an alternate line length, more (or less)
635 columns will be shown.
636 With the flag -b (begin time) and/or -e (end time) followed by a time
637 argument of the form HH:MM, a certain time period within the raw file
638 can be selected.
639 When atop is installed, the script atop.daily is stored in the
641 /usr/share/atop directory. This scripts takes care that atop is acti‐
642 vated every day at midnight to write compressed binary data to the file
643 /var/log/atop/atop_YYYYMMDD with an interval of 10 minutes by default.
644 The -R flag is passed by default to gather information about the pro‐
645 portional set size of every process.
646 Furthermore the script removes all raw files which are by default older
647 than 28 days.
648 The mentioned default values can be overruled by creating the file
649 /etc/default/atop that might contain other values for LOGOPTS (by
650 default the -R flag), LOGINTERVAL (in seconds, by default 600), and
651 LOGGENERATIONS (in days, by default 28).
652 The atop.daily script is activated via the cron daemon using the file
654 /etc/cron.d/atop with the contents
655 0 0 * * * root /usr/share/atop/atop.daily
656 When the package psacct is installed, the process accounting is auto‐
658 matically restarted via the logrotate mechanism. The file /etc/logro‐
659 tate.d/psaccs_atop takes care that atop is finished just before the
660 rotation of the process accounting file and the file /etc/logro‐
661 tate.d/psaccu_atop takes care that atop is restarted again after the
662 rotation. When the package psacct is not installed, these logrotate-
663 files have no effect.
664 Unfortunately, it is not always possible to keep the format of the raw
666 files compatible in newer versions of atop especially when lots of new
667 counters have to be maintained. Therefore, the program atopconvert is
668 installed to convert a raw file created by an older version of atop to
669 a raw file that can be read by a newer version of atop (see the man
670 page of atopconvert for more details).
671
674 The first sample shows the system level activity since boot (the
675 elapsed time in the header shows the time since boot). Note that par‐
676 ticular counters could have reached their maximum value (several times)
677 and started by zero again, so do not rely on these figures.
678 For every sample atop first shows the lines related to system level
680 activity. If a particular system resource has not been used during the
681 interval, the entire line related to this resource is suppressed. So
682 the number of system level lines may vary for each sample.
683 After that a list is shown of processes which have been active during
684 the last interval. This list is by default sorted on cpu consumption,
685 but this order can be changed by the keys which are previously
686 described.
687 If values have to be shown by atop which do not fit in the column
689 width, another format is used. If e.g. a cpu-consumption of 233216 mil‐
690 liseconds should be shown in a column width of 4 positions, it is shown
691 as `233s' (in seconds). For large memory figures, another unit is cho‐
692 sen if the value does not fit (Mb instead of Kb, Gb instead of Mb, Tb
693 instead of Gb, ...). For other values, a kind of exponent notation is
694 used (value 123456789 shown in a column of 5 positions gives 123e6).
695
697 The system level information consists of the following output lines:
698 PRC Process and thread level totals.
700 This line contains the total cpu time consumed in system mode
701 (`sys') and in user mode (`user'), the total number of processes
702 present at this moment (`#proc'), the total number of threads
703 present at this moment in state `running' (`#trun'), `sleeping
704 interruptible' (`#tslpi') and `sleeping uninterruptible'
705 (`#tslpu'), the number of zombie processes (`#zombie'), the number
706 of clone system calls (`clones'), and the number of processes that
707 ended during the interval (`#exit') when process accounting is
708 used. Instead of `#exit` the last column may indicate that process
709 accounting could not be activated (`no procacct`).
710 If the screen-width does not allow all of these counters, only a
711 relevant subset is shown.
712 CPU CPU utilization.
714 At least one line is shown for the total occupation of all CPUs
715 together.
716 In case of a multi-processor system, an additional line is shown
717 for every individual processor (with `cpu' in lower case), sorted
718 on activity. Inactive CPUs will not be shown by default. The
719 lines showing the per-cpu occupation contain the cpu number in the
720 field combined with the wait percentage.
721 Every line contains the percentage of cpu time spent in kernel
723 mode by all active processes (`sys'), the percentage of cpu time
724 consumed in user mode (`user') for all active processes (including
725 processes running with a nice value larger than zero), the per‐
726 centage of cpu time spent for interrupt handling (`irq') including
727 softirq, the percentage of unused cpu time while no processes were
728 waiting for disk I/O (`idle'), and the percentage of unused cpu
729 time while at least one process was waiting for disk I/O (`wait').
730 In case of per-cpu occupation, the cpu number and the wait per‐
731 centage (`w') for that cpu. The number of lines showing the per-
732 cpu occupation can be limited.
733 For virtual machines, the steal-percentage (`steal') shows the
735 percentage of cpu time stolen by other virtual machines running on
736 the same hardware.
737 For physical machines hosting one or more virtual machines, the
738 guest-percentage (`guest') shows the percentage of cpu time used
739 by the virtual machines. Notice that this percentage overlaps the
740 user percentage!
741 When PMC performance monitoring counters are supported by the CPU
743 and the kernel (and atop runs with root privileges), the number of
744 instructions per CPU cycle (`ipc') is shown. The first sample
745 always shows the value 'initial', because the counters are just
746 activated at the moment that atop is started.
747 When the CPU busy percentage is high and the IPC is less than 1.0,
748 it is likely that the CPU is frequently waiting for memory access
749 during instruction execution (larger CPU caches or faster memory
750 might be helpful to improve performance). When the CPU busy per‐
751 centage is high and the IPC is greater than 1.0, it is likely that
752 the CPU is instruction-bound (more/faster cores might be helpful
753 to improve performance).
754 Furthermore, per CPU the effective number of cycles (`cycl') is
755 shown. This value can reach the current CPU frequency if such CPU
756 is 100% busy. When an idle CPU is halted, the number of effective
757 cycles can be (considerably) lower than the current frequency.
758 Notice that the average instructions per cycle and number of
759 cycles is shown in the CPU line for all CPUs.
760 See also: http://www.brendangregg.com/blog/2017-05-09/cpu-utiliza‐
761 tion-is-wrong.html
762 In case of frequency scaling, all previously mentioned CPU per‐
765 centages are relative to the used scaling of the CPU during the
766 interval. If a CPU has been active for e.g. 50% in user mode dur‐
767 ing the interval while the frequency scaling of that CPU was 40%,
768 only 20% of the full capacity of the CPU has been used in user
769 mode.
770 In case that the kernel module `cpufreq_stats' is active (after
771 issueing `modprobe cpufreq_stats'), the average frequency (`avgf')
772 and the average scaling percentage (`avgscal') is shown. Otherwise
773 the current frequency (`curf') and the current scaling percentage
774 (`curscal') is shown at the moment that the sample is taken.
775 Notice that average values for frequency and scaling are shown in
776 the CPU line for every CPU.
777 Frequency scaling statistics are only gathered for systems with
778 maximum 8 CPUs, since gathering of these values per CPU is very
779 time consuming.
780 If the screen-width does not allow all of these counters, only a
782 relevant subset is shown.
783 CPL CPU load information.
785 This line contains the load average figures reflecting the number
786 of threads that are available to run on a CPU (i.e. part of the
787 runqueue) or that are waiting for disk I/O. These figures are
788 averaged over 1 (`avg1'), 5 (`avg5') and 15 (`avg15') minutes.
789 Furthermore the number of context switches (`csw'), the number of
790 serviced interrupts (`intr') and the number of available CPUs are
791 shown.
792 If the screen-width does not allow all of these counters, only a
794 relevant subset is shown.
795 GPU GPU utilization (Nvidia).
797 Read the section GPU STATISTICS GATHERING in this document to find
798 the details about the activation of the atopgpud daemon.
799 In the first column of every line, the bus-id (last nine charac‐
801 ters) and the GPU number are shown. The subsequent columns show
802 the percentage of time that one or more kernels were executing on
803 the GPU (`gpubusy'), the percentage of time that global (device)
804 memory was being read or written (`membusy'), the occupation per‐
805 centage of memory (`memocc'), the total memory (`total'), the mem‐
806 ory being in use at the moment of the sample (`used'), the average
807 memory being in use during the sample time (`usavg'), the number
808 of processes being active on the GPU at the moment of the sample
809 (`#proc'), and the type of GPU.
810 If the screen-width does not allow all of these counters, only a
812 relevant subset is shown.
813 The number of lines showing the GPUs can be limited.
814 MEM Memory occupation.
816 This line contains the total amount of physical memory (`tot'),
817 the amount of memory which is currently free (`free'), the amount
818 of memory in use as page cache including the total resident shared
819 memory (`cache'), the amount of memory within the page cache that
820 has to be flushed to disk (`dirty'), the amount of memory used for
821 filesystem meta data (`buff'), the amount of memory being used for
822 kernel mallocs (`slab'), the amount of slab memory that is
823 reclaimable (`slrec'), the resident size of shared memory includ‐
824 ing tmpfs (`shmem`), the resident size of shared memory (`shrss`)
825 the amount of shared memory that is currently swapped (`shswp`),
826 the amount of memory that is currently claimed by vmware's balloon
827 driver (`vmbal`), the amount of memory that is claimed for huge
828 pages (`hptot`), and the amount of huge page memory that is really
829 in use (`hpuse`).
830 If the screen-width does not allow all of these counters, only a
832 relevant subset is shown.
833 SWP Swap occupation and overcommit info.
835 This line contains the total amount of swap space on disk (`tot')
836 and the amount of free swap space (`free').
837 Furthermore the committed virtual memory space (`vmcom') and the
838 maximum limit of the committed space (`vmlim', which is by default
839 swap size plus 50% of memory size) is shown. The committed space
840 is the reserved virtual space for all allocations of private mem‐
841 ory space for processes. The kernel only verifies whether the com‐
842 mitted space exceeds the limit if strict overcommit handling is
843 configured (vm.overcommit_memory is 2).
844 PAG Paging frequency.
846 This line contains the number of scanned pages (`scan') due to the
847 fact that free memory drops below a particular threshold and the
848 number times that the kernel tries to reclaim pages due to an
849 urgent need (`stall').
850 Also the number of memory pages the system read from swap space
851 (`swin') and the number of memory pages the system wrote to swap
852 space (`swout') are shown.
853 PSI Pressure Stall Information.
855 This line contains three percentages per category: average pres‐
856 sure percentage over the last 10, 60 and 300 seconds (separated by
857 slashes).
858 The categories are: CPU for 'some' (`cs'), memory for 'some'
859 (`ms'), memory for 'full' (`mf'), I/O for 'some' (`is'), and I/O
860 for 'full' (`if').
861 LVM/MDD/DSK
863 Logical volume/multiple device/disk utilization.
864 Per active unit one line is produced, sorted on unit activity.
865 Such line shows the name (e.g. VolGroup00-lvtmp for a logical vol‐
866 ume or sda for a hard disk), the busy percentage i.e. the portion
867 of time that the unit was busy handling requests (`busy'), the
868 number of read requests issued (`read'), the number of write
869 requests issued (`write'), the number of KiBytes per read
870 (`KiB/r'), the number of KiBytes per write (`KiB/w'), the number
871 of MiBytes per second throughput for reads (`MBr/s'), the number
872 of MiBytes per second throughput for writes (`MBw/s'), the average
873 queue depth (`avq') and the average number of milliseconds needed
874 by a request (`avio') for seek, latency and data transfer.
875 If the screen-width does not allow all of these counters, only a
876 relevant subset is shown.
877 The number of lines showing the units can be limited per class
879 (LVM, MDD or DSK) with the 'l' key or statically (see separate
880 man-page of atoprc). By specifying the value 0 for a particular
881 class, no lines will be shown any more for that class.
882 NFM Network Filesystem (NFS) mount at the client side.
884 For each NFS-mounted filesystem, a line is shown that contains the
885 mounted server directory, the name of the server (`srv'), the
886 total number of bytes physically read from the server (`read') and
887 the total number of bytes physically written to the server
888 (`write'). Data transfer is subdivided in the number of bytes
889 read via normal read() system calls (`nread'), the number of bytes
890 written via normal read() system calls (`nwrit'), the number of
891 bytes read via direct I/O (`dread'), the number of bytes written
892 via direct I/O (`dwrit'), the number of bytes read via memory
893 mapped I/O pages (`mread'), and the number of bytes written via
894 memory mapped I/O pages (`mwrit').
895 NFC Network Filesystem (NFS) client side counters.
897 This line contains the number of RPC calls issues by local pro‐
898 cesses (`rpc'), the number of read RPC calls (`read`) and write
899 RPC calls (`rpwrite') issued to the NFS server, the number of RPC
900 calls being retransmitted (`retxmit') and the number of authoriza‐
901 tion refreshes (`autref').
902 NFS Network Filesystem (NFS) server side counters.
904 This line contains the number of RPC calls received from NFS
905 clients (`rpc'), the number of read RPC calls received (`cread`),
906 the number of write RPC calls received (`cwrit'), the number of
907 Megabytes/second returned to read requests by clients (`MBcr/s`),
908 the number of Megabytes/second passed in write requests by clients
909 (`MBcw/s`), the number of network requests handled via TCP
910 (`nettcp'), the number of network requests handled via UDP
911 (`netudp'), the number of reply cache hits (`rchits'), the number
912 of reply cache misses (`rcmiss') and the number of uncached
913 requests (`rcnoca'). Furthermore some error counters indicating
914 the number of requests with a bad format (`badfmt') or a bad
915 authorization (`badaut'), and a counter indicating the number of
916 bad clients (`badcln').
917 NET Network utilization (TCP/IP).
919 One line is shown for activity of the transport layer (TCP and
920 UDP), one line for the IP layer and one line per active interface.
921 For the transport layer, counters are shown concerning the number
922 of received TCP segments including those received in error
923 (`tcpi'), the number of transmitted TCP segments excluding those
924 containing only retransmitted octets (`tcpo'), the number of UDP
925 datagrams received (`udpi'), the number of UDP datagrams transmit‐
926 ted (`udpo'), the number of active TCP opens (`tcpao'), the number
927 of passive TCP opens (`tcppo'), the number of TCP output retrans‐
928 missions (`tcprs'), the number of TCP input errors (`tcpie'), the
929 number of TCP output resets (`tcpor'), the number of UDP no ports
930 (`udpnp'), and the number of UDP input errors (`udpie').
931 If the screen-width does not allow all of these counters, only a
932 relevant subset is shown.
933 These counters are related to IPv4 and IPv6 combined.
934 For the IP layer, counters are shown concerning the number of IP
936 datagrams received from interfaces, including those received in
937 error (`ipi'), the number of IP datagrams that local higher-layer
938 protocols offered for transmission (`ipo'), the number of received
939 IP datagrams which were forwarded to other interfaces (`ipfrw'),
940 the number of IP datagrams which were delivered to local higher-
941 layer protocols (`deliv'), the number of received ICMP datagrams
942 (`icmpi'), and the number of transmitted ICMP datagrams (`icmpo').
943 If the screen-width does not allow all of these counters, only a
944 relevant subset is shown.
945 These counters are related to IPv4 and IPv6 combined.
946 For every active network interface one line is shown, sorted on
948 the interface activity. Such line shows the name of the interface
949 and its busy percentage in the first column. The busy percentage
950 for half duplex is determined by comparing the interface speed
951 with the number of bits transmitted and received per second; for
952 full duplex the interface speed is compared with the highest of
953 either the transmitted or the received bits. When the interface
954 speed can not be determined (e.g. for the loopback interface),
955 `---' is shown instead of the percentage.
956 Furthermore the number of received packets (`pcki'), the number of
957 transmitted packets (`pcko'), the line speed of the interface
958 (`sp'), the effective amount of bits received per second (`si'),
959 the effective amount of bits transmitted per second (`so'), the
960 number of collisions (`coll'), the number of received multicast
961 packets (`mlti'), the number of errors while receiving a packet
962 (`erri'), the number of errors while transmitting a packet
963 (`erro'), the number of received packets dropped (`drpi'), and the
964 number of transmitted packets dropped (`drpo').
965 If the screen-width does not allow all of these counters, only a
966 relevant subset is shown.
967 The number of lines showing the network interfaces can be limited.
968 IFB Infiniband utilization.
970 For every active Infiniband port one line is shown, sorted on
971 activity. Such line shows the name of the port and its busy per‐
972 centage in the first column. The busy percentage is determined by
973 taking the highest of either the transmitted or the received bits
974 during the interval, multiplying that value by the number of lanes
975 and comparing it against the maximum port speed.
976 Furthermore the number of received packets divided by the number
977 of lanes (`pcki'), the number of transmitted packets divided by
978 the number of lanes (`pcko'), the maximum line speed (`sp'), the
979 effective amount of bits received per second (`si'), the effective
980 amount of bits transmitted per second (`so'), and the number of
981 lanes (`lanes').
982 If the screen-width does not allow all of these counters, only a
983 relevant subset is shown.
984 The number of lines showing the Infiniband ports can be limited.
985
987 Following the system level information, the processes are shown from
988 which the resource utilization has changed during the last interval.
989 These processes might have used cpu time or issued disk or network
990 requests. However a process is also shown if part of it has been paged
991 out due to lack of memory (while the process itself was in sleep
992 state).
993 Per process the following fields may be shown (in alphabetical order),
995 depending on the current output mode as described in the section INTER‐
996 ACTIVE COMMANDS and depending on the current width of your window:
997 AVGRSZ The average size of one read-action on disk.
999 AVGWSZ The average size of one write-action on disk.
1001 BANDWI Total bandwidth for received TCP and UDP packets consumed by
1003 this process (bits-per-second). This value can be compared
1004 with the value `si' on interface level (used bandwidth per
1005 interface).
1006 This information will only be shown when the kernel module
1007 `netatop' is loaded.
1008 BANDWO Total bandwidth for sent TCP and UDP packets consumed by this
1010 process (bits-per-second). This value can be compared with
1011 the value `so' on interface level (used bandwidth per inter‐
1012 face).
1013 This information will only be shown when the kernel module
1014 `netatop' is loaded.
1015 CID Container ID (Docker) of 12 hexadecimal digits, referring to
1017 the container in which the process/thread is running. If a
1018 process has been started and finished during the last inter‐
1019 val, a `?' is shown because the container ID is not part of
1020 the standard process accounting record.
1021 CMD The name of the process. This name can be surrounded by
1023 "less/greater than" signs (`<name>') which means that the
1024 process has finished during the last interval.
1025 Behind the abbreviation `CMD' in the header line, the current
1026 page number and the total number of pages of the
1027 process/thread list are shown.
1028 COMMAND-LINE
1030 The full command line of the process (including arguments). If
1031 the length of the command line exceeds the length of the
1032 screen line, the arrow keys -> and <- can be used for horizon‐
1033 tal scroll.
1034 Behind the verb `COMMAND-LINE' in the header line, the current
1035 page number and the total number of pages of the
1036 process/thread list are shown.
1037 CPU The occupation percentage of this process related to the
1039 available capacity for this resource on system level.
1040 CPUNR The identification of the CPU the (main) thread is running on
1042 or has recently been running on.
1043 CTID Container ID (OpenVZ). If a process has been started and fin‐
1045 ished during the last interval, a `?' is shown because the
1046 container ID is not part of the standard process accounting
1047 record.
1048 DSK The occupation percentage of this process related to the total
1050 load that is produced by all processes (i.e. total disk
1051 accesses by all processes during the last interval).
1052 This information is shown when per process "storage account‐
1053 ing" is active in the kernel.
1054 EGID Effective group-id under which this process executes.
1056 ENDATE Date that the process has been finished. If the process is
1058 still running, this field shows `active'.
1059 ENTIME Time that the process has been finished. If the process is
1061 still running, this field shows `active'.
1062 ENVID Virtual environment identified (OpenVZ only).
1064 EUID Effective user-id under which this process executes.
1066 EXC The exit code of a terminated process (second position of col‐
1068 umn `ST' is E) or the fatal signal number (second position of
1069 column `ST' is S or C).
1070 FSGID Filesystem group-id under which this process executes.
1072 FSUID Filesystem user-id under which this process executes.
1074 GPU When the atopgpud daemon does not run with root privileges,
1076 the GPU percentage reflects the GPU memory occupation percent‐
1077 age (memory of all GPUs is 100%).
1078 When the atopgpud daemon runs with root privileges, the GPU
1079 percentage reflects the GPU busy percentage.
1080 GPUBUSY Busy percentage on all GPUs (one GPU is 100%).
1082 When the atopgpud daemon does not run with root privileges,
1083 this value is not available.
1084 GPUNUMS Comma-separated list of GPUs used by the process during the
1086 interval. When the comma-separated list exceeds the width of
1087 the column, a hexadecimal value is shown.
1088 MAJFLT The number of page faults issued by this process that have
1090 been solved by creating/loading the requested memory page.
1091 MEM The occupation percentage of this process related to the
1093 available capacity for this resource on system level.
1094 MEMAVG Average memory occupation during the interval on all used
1096 GPUs.
1097 MEMBUSY Busy percentage of memory on all GPUs (one GPU is 100%), i.e.
1099 the time needed for read and write accesses on memory.
1100 When the atopgpud daemon does not run with root privileges,
1101 this value is not available.
1102 MEMNOW Memory occupation at the moment of the sample on all used
1104 GPUs.
1105 MINFLT The number of page faults issued by this process that have
1107 been solved by reclaiming the requested memory page from the
1108 free list of pages.
1109 NET The occupation percentage of this process related to the total
1111 load that is produced by all processes (i.e. consumed network
1112 bandwidth of all processes during the last interval).
1113 This information will only be shown when kernel module
1114 `netatop' is loaded.
1115 NICE The more or less static priority that can be given to a
1117 process on a scale from -20 (high priority) to +19 (low prior‐
1118 ity).
1119 NPROCS The number of active and terminated processes accumulated for
1121 this user or program.
1122 PID Process-id. If a process has been started and finished during
1124 the last interval, a `?' is shown because the process-id is
1125 not part of the standard process accounting record.
1126 POLI The policies 'norm' (normal, which is SCHED_OTHER), 'btch'
1128 (batch) and 'idle' refer to timesharing processes. The poli‐
1129 cies 'fifo' (SCHED_FIFO) and 'rr' (round robin, which is
1130 SCHED_RR) refer to realtime processes.
1131 PPID Parent process-id. If a process has been started and finished
1133 during the last interval, value 0 is shown because the parent
1134 process-id is not part of the standard process accounting
1135 record.
1136 PRI The process' priority ranges from 0 (highest priority) to 139
1138 (lowest priority). Priority 0 to 99 are used for realtime pro‐
1139 cesses (fixed priority independent of their behavior) and pri‐
1140 ority 100 to 139 for timesharing processes (variable priority
1141 depending on their recent CPU consumption and the nice value).
1142 PSIZE The proportional memory size of this process (or user).
1144 Every process shares resident memory with other processes.
1145 E.g. when a particular program is started several times, the
1146 code pages (text) are only loaded once in memory and shared by
1147 all incarnations. Also the code of shared libraries is shared
1148 by all processes using that shared library, as well as shared
1149 memory and memory-mapped files. For the PSIZE calculation of
1150 a process, the resident memory of a process that is shared
1151 with other processes is divided by the number of sharers.
1152 This means, that every process is accounted for a proportional
1153 part of that memory. Accumulating the PSIZE values of all pro‐
1154 cesses in the system gives a reliable impression of the total
1155 resident memory consumed by all processes.
1156 Since gathering of all values that are needed to calculate the
1157 PSIZE is a relatively time-consuming task, the 'R' key (or
1158 '-R' flag) should be active. Gathering these values also
1159 requires superuser privileges (otherwise '?K' is shown in the
1160 output).
1161 If a process has finished during the last interval, no value
1162 is shown since the proportional memory size is not part of the
1163 standard process accounting record.
1164 RDDSK When the kernel maintains standard io statistics (>= 2.6.20):
1166 The read data transfer issued physically on disk (so reading
1167 from the disk cache is not accounted for).
1168 Unfortunately, the kernel aggregates the data tranfer of a
1169 process to the data transfer of its parent process when termi‐
1170 nating, so you might see transfers for (parent) processes like
1171 cron, bash or init, that are not really issued by them.
1172 RGID The real group-id under which the process executes.
1174 RGROW The amount of resident memory that the process has grown dur‐
1176 ing the last interval. A resident growth can be caused by
1177 touching memory pages which were not physically created/loaded
1178 before (load-on-demand). Note that a resident growth can also
1179 be negative e.g. when part of the process is paged out due to
1180 lack of memory or when the process frees dynamically allocated
1181 memory. For a process which started during the last interval,
1182 the resident growth reflects the total resident size of the
1183 process at that moment.
1184 If a process has finished during the last interval, no value
1185 is shown since resident memory occupation is not part of the
1186 standard process accounting record.
1187 RNET The number of TCP- and UDP packets received by this process.
1189 This information will only be shown when kernel module
1190 `netatop' is installed.
1191 If a process has finished during the last interval, no value
1192 is shown since network counters are not part of the standard
1193 process accounting record.
1194 RSIZE The total resident memory usage consumed by this process (or
1196 user). Notice that the RSIZE of a process includes all resi‐
1197 dent memory used by that process, even if certain memory parts
1198 are shared with other processes (see also the explanation of
1199 PSIZE).
1200 If a process has finished during the last interval, no value
1201 is shown since resident memory occupation is not part of the
1202 standard process accounting record.
1203 RTPR Realtime priority according the POSIX standard. Value can be
1205 0 for a timesharing process (policy 'norm', 'btch' or 'idle')
1206 or ranges from 1 (lowest) till 99 (highest) for a realtime
1207 process (policy 'rr' or 'fifo').
1208 RUID The real user-id under which the process executes.
1210 S The current state of the (main) thread: `R' for running (cur‐
1212 rently processing or in the runqueue), `S' for sleeping inter‐
1213 ruptible (wait for an event to occur), `D' for sleeping non-
1214 interruptible, `Z' for zombie (waiting to be synchronized with
1215 its parent process), `T' for stopped (suspended or traced),
1216 `W' for swapping, and `E' (exit) for processes which have fin‐
1217 ished during the last interval.
1218 SGID The saved group-id of the process.
1220 SNET The number of TCP and UDP packets transmitted by this process.
1222 This information will only be shown when the kernel module
1223 `netatop' is loaded.
1224 ST The status of a process.
1226 The first position indicates if the process has been started
1227 during the last interval (the value N means 'new process').
1228 The second position indicates if the process has been finished
1230 during the last interval.
1231 The value E means 'exit' on the process' own initiative; the
1232 exit code is displayed in the column `EXC'.
1233 The value S means that the process has been terminated unvol‐
1234 untarily by a signal; the signal number is displayed in the in
1235 the column `EXC'.
1236 The value C means that the process has been terminated unvol‐
1237 untarily by a signal, producing a core dump in its current
1238 directory; the signal number is displayed in the column `EXC'.
1239 STDATE The start date of the process.
1241 STTIME The start time of the process.
1243 SUID The saved user-id of the process.
1245 SWAPSZ The swap space consumed by this process (or user).
1247 SYSCPU CPU time consumption of this process in system mode (kernel
1249 mode), usually due to system call handling.
1250 TCPRASZ The average size of a received TCP buffer in bytes. This
1252 information will only be shown when the kernel module
1253 `netatop' is loaded.
1254 TCPRCV The number of TCP packets received for this process. This
1256 information will only be shown when the kernel module
1257 `netatop' is loaded.
1258 TCPSASZ The average size of a transmitted TCP buffer in bytes. This
1260 information will only be shown when the kernel module
1261 `netatop' is loaded.
1262 TCPSND The number of TCP packets transmitted for this process. This
1264 information will only be shown when the kernel module
1265 `netatop' is loaded.
1266 THR Total number of threads within this process. All related
1268 threads are contained in a thread group, represented by atop
1269 as one line or as a separate line when the 'y' key (or -y
1270 flag) is active.
1271 On Linux 2.4 systems it is hardly possible to determine which
1273 threads (i.e. processes) are related to the same thread group.
1274 Every thread is represented by atop as a separate line.
1275 TID Thread-id. All threads within a process run with the same PID
1277 but with a different TID. This value is shown for individual
1278 threads in multi-threaded processes (when using the key 'y').
1279 TRUN Number of threads within this process that are in the state
1281 'running' (R).
1282 TSLPI Number of threads within this process that are in the state
1284 'interruptible sleeping' (S).
1285 TSLPU Number of threads within this process that are in the state
1287 'uninterruptible sleeping' (D).
1288 UDPRASZ The average size of a received UDP packet in bytes. This
1290 information will only be shown when the kernel module
1291 `netatop' is loaded.
1292 UDPRCV The number of UDP packets received by this process. This
1294 information will only be shown when the kernel module
1295 `netatop' is loaded.
1296 UDPSASZ The average size of a transmitted UDP packets in bytes. This
1298 information will only be shown when the kernel module
1299 `netatop' is loaded.
1300 UDPSND The number of UDP packets transmitted by this process. This
1302 information will only be shown when the kernel module
1303 `netatop' is loaded.
1304 USRCPU CPU time consumption of this process in user mode, due to pro‐
1306 cessing the own program text.
1307 VDATA The virtual memory size of the private data used by this
1309 process (including heap and shared library data).
1310 VGROW The amount of virtual memory that the process has grown during
1312 the last interval. A virtual growth can be caused by e.g.
1313 issueing a malloc() or attaching a shared memory segment. Note
1314 that a virtual growth can also be negative by e.g. issueing a
1315 free() or detaching a shared memory segment. For a process
1316 which started during the last interval, the virtual growth
1317 reflects the total virtual size of the process at that moment.
1318 If a process has finished during the last interval, no value
1319 is shown since virtual memory occupation is not part of the
1320 standard process accounting record.
1321 VPID Virtual process-id (within an OpenVZ container). If a process
1323 has been started and finished during the last interval, a `?'
1324 is shown because the virtual process-id is not part of the
1325 standard process accounting record.
1326 VSIZE The total virtual memory usage consumed by this process (or
1328 user).
1329 If a process has finished during the last interval, no value
1330 is shown since virtual memory occupation is not part of the
1331 standard process accounting record.
1332 VSLIBS The virtual memory size of the (shared) text of all shared
1334 libraries used by this process.
1335 VSTACK The virtual memory size of the (private) stack used by this
1337 process
1338 VSTEXT The virtual memory size of the (shared) text of the executable
1340 program.
1341 WRDSK When the kernel maintains standard io statistics (>= 2.6.20):
1343 The write data transfer issued physically on disk (so writing
1344 to the disk cache is not accounted for). This counter is
1345 maintained for the application process that writes its data to
1346 the cache (assuming that this data is physically transferred
1347 to disk later on). Notice that disk I/O needed for swapping is
1348 not taken into account.
1349 Unfortunately, the kernel aggregates the data tranfer of a
1350 process to the data transfer of its parent process when termi‐
1351 nating, so you might see transfers for (parent) processes like
1352 cron, bash or init, that are not really issued by them.
1353 WCANCL When the kernel maintains standard io statistics (>= 2.6.20):
1355 The write data transfer previously accounted for this process
1356 or another process that has been cancelled. Suppose that a
1357 process writes new data to a file and that data is removed
1358 again before the cache buffers have been flushed to disk.
1359 Then the original process shows the written data as WRDSK,
1360 while the process that removes/truncates the file shows the
1361 unflushed removed data as WCANCL.
1362
1364 With the flag -P followed by a list of one or more labels (comma-sepa‐
1365 rated), parseable output is produced for each sample. The labels that
1366 can be specified for system-level statistics correspond to the labels
1367 (first verb of each line) that can be found in the interactive output:
1368 "CPU", "cpu", "CPL", "GPU", "MEM", "SWP", "PAG", "PSI", "LVM", "MDD",
1369 "DSK", "NFM", "NFC", "NFS", "NET" and "IFB".
1370 For process-level statistics special labels are introduced: "PRG" (gen‐
1371 eral), "PRC" (cpu), "PRE" (GPU), "PRM" (memory), "PRD" (disk, only if
1372 "storage accounting" is active) and "PRN" (network, only if the kernel
1373 module 'netatop' has been installed).
1374 With the label "ALL", all system and process level statistics are
1375 shown.
1376 For every interval all requested lines are shown whereafter atop shows
1378 a line just containing the label "SEP" as a separator before the lines
1379 for the next sample are generated.
1380 When a sample contains the values since boot, atop shows a line just
1381 containing the label "RESET" before the lines for this sample are gen‐
1382 erated.
1383 The first part of each output-line consists of the following six
1385 fields: label (the name of the label), host (the name of this machine),
1386 epoch (the time of this interval as number of seconds since 1-1-1970),
1387 date (date of this interval in format YYYY/MM/DD), time (time of this
1388 interval in format HH:MM:SS), and interval (number of seconds elapsed
1389 for this interval).
1390 The subsequent fields of each output-line depend on the label:
1392 CPU Subsequent fields: total number of clock-ticks per second for
1394 this machine, number of processors, consumption for all CPUs
1395 in system mode (clock-ticks), consumption for all CPUs in user
1396 mode (clock-ticks), consumption for all CPUs in user mode for
1397 niced processes (clock-ticks), consumption for all CPUs in
1398 idle mode (clock-ticks), consumption for all CPUs in wait mode
1399 (clock-ticks), consumption for all CPUs in irq mode (clock-
1400 ticks), consumption for all CPUs in softirq mode (clock-
1401 ticks), consumption for all CPUs in steal mode (clock-ticks),
1402 consumption for all CPUs in guest mode (clock-ticks) overlap‐
1403 ping user mode, frequency of all CPUs, frequency percentage of
1404 all CPUs, instructions executed by all CPUs and cycles for all
1405 CPUs.
1406 cpu Subsequent fields: total number of clock-ticks per second for
1408 this machine, processor-number, consumption for this CPU in
1409 system mode (clock-ticks), consumption for this CPU in user
1410 mode (clock-ticks), consumption for this CPU in user mode for
1411 niced processes (clock-ticks), consumption for this CPU in
1412 idle mode (clock-ticks), consumption for this CPU in wait mode
1413 (clock-ticks), consumption for this CPU in irq mode (clock-
1414 ticks), consumption for this CPU in softirq mode (clock-
1415 ticks), consumption for this CPU in steal mode (clock-ticks),
1416 consumption for this CPU in guest mode (clock-ticks) overlap‐
1417 ping user mode, frequency of this CPU, frequency percentage of
1418 this CPU, instructions executed by this CPU and cycles for
1419 this CPU.
1420 CPL Subsequent fields: number of processors, load average for last
1422 minute, load average for last five minutes, load average for
1423 last fifteen minutes, number of context-switches, and number
1424 of device interrupts.
1425 GPU Subsequent fields: GPU number, bus-id string, type of GPU
1427 string, GPU busy percentage during last second (-1 if not
1428 available), memory busy percentage during last second (-1 if
1429 not available), total memory size (KiB), used memory (KiB) at
1430 this moment, number of samples taken during interval, cumula‐
1431 tive GPU busy percentage during the interval (to be divided by
1432 the number of samples for the average busy percentage, -1 if
1433 not available), cumulative memory busy percentage during the
1434 interval (to be divided by the number of samples for the aver‐
1435 age busy percentage, -1 if not available), and cumulative mem‐
1436 ory occupation during the interval (to be divided by the num‐
1437 ber of samples for the average occupation).
1438 MEM Subsequent fields: page size for this machine (in bytes), size
1440 of physical memory (pages), size of free memory (pages), size
1441 of page cache (pages), size of buffer cache (pages), size of
1442 slab (pages), dirty pages in cache (pages), reclaimable part
1443 of slab (pages), total size of vmware's balloon pages (pages),
1444 total size of shared memory (pages), size of resident shared
1445 memory (pages), size of swapped shared memory (pages), huge
1446 page size (in bytes), total size of huge pages (huge pages),
1447 and size of free huge pages (huge pages).
1448 SWP Subsequent fields: page size for this machine (in bytes), size
1450 of swap (pages), size of free swap (pages), 0 (future use),
1451 size of committed space (pages), and limit for committed space
1452 (pages).
1453 PAG Subsequent fields: page size for this machine (in bytes), num‐
1455 ber of page scans, number of allocstalls, 0 (future use), num‐
1456 ber of swapins, and number of swapouts.
1457 PSI Subsequent fields: PSI statistics present on this system (n or
1459 y), CPU some avg10, CPU some avg60, CPU some avg300, CPU some
1460 accumulated microseconds during interval, memory some avg10,
1461 memory some avg60, memory some avg300, memory some accumulated
1462 microseconds during interval, memory full avg10, memory full
1463 avg60, memory full avg300, memory full accumulated microsec‐
1464 onds during interval, I/O some avg10, I/O some avg60, I/O some
1465 avg300, I/O some accumulated microseconds during interval, I/O
1466 full avg10, I/O full avg60, I/O full avg300, and I/O full
1467 accumulated microseconds during interval.
1468 LVM/MDD/DSK
1470 For every logical volume/multiple device/hard disk one line is
1471 shown.
1472 Subsequent fields: name, number of milliseconds spent for I/O,
1473 number of reads issued, number of sectors transferred for
1474 reads, number of writes issued, and number of sectors trans‐
1475 ferred for write.
1476 NFM Subsequent fields: mounted NFS filesystem, total number of
1478 bytes read, total number of bytes written, number of bytes
1479 read by normal system calls, number of bytes written by normal
1480 system calls, number of bytes read by direct I/O, number of
1481 bytes written by direct I/O, number of pages read by memory-
1482 mapped I/O, and number of pages written by memory-mapped I/O.
1483 NFC Subsequent fields: number of transmitted RPCs, number of
1485 transmitted read RPCs, number of transmitted write RPCs, num‐
1486 ber of RPC retransmissions, and number of authorization
1487 refreshes.
1488 NFS Subsequent fields: number of handled RPCs, number of received
1490 read RPCs, number of received write RPCs, number of bytes read
1491 by clients, number of bytes written by clients, number of RPCs
1492 with bad format, number of RPCs with bad authorization, number
1493 of RPCs from bad client, total number of handled network
1494 requests, number of handled network requests via TCP, number
1495 of handled network requests via UDP, number of handled TCP
1496 connections, number of hits on reply cache, number of misses
1497 on reply cache, and number of uncached requests.
1498 NET First one line is produced for the upper layers of the TCP/IP
1500 stack.
1501 Subsequent fields: the verb "upper", number of packets
1502 received by TCP, number of packets transmitted by TCP, number
1503 of packets received by UDP, number of packets transmitted by
1504 UDP, number of packets received by IP, number of packets
1505 transmitted by IP, number of packets delivered to higher lay‐
1506 ers by IP, and number of packets forwarded by IP.
1507 Next one line is shown for every interface.
1509 Subsequent fields: name of the interface, number of packets
1510 received by the interface, number of bytes received by the
1511 interface, number of packets transmitted by the interface,
1512 number of bytes transmitted by the interface, interface speed,
1513 and duplex mode (0=half, 1=full).
1514 IFB Subsequent fields: name of the InfiniBand interface, port num‐
1516 ber, number of lanes, maximum rate (Mbps), number of bytes
1517 received, number of bytes transmitted, number of packets
1518 received, and number of packets transmitted.
1519 PRG For every process one line is shown.
1521 Subsequent fields: PID (unique ID of task), name (between
1522 brackets), state, real uid, real gid, TGID (group number of
1523 related tasks/threads), total number of threads, exit code (in
1524 case of fatal signal: signal number + 256), start time
1525 (epoch), full command line (between brackets), PPID, number of
1526 threads in state 'running' (R), number of threads in state
1527 'interruptible sleeping' (S), number of threads in state
1528 'uninterruptible sleeping' (D), effective uid, effective gid,
1529 saved uid, saved gid, filesystem uid, filesystem gid, elapsed
1530 time (hertz), is_process (y/n), OpenVZ virtual pid (VPID),
1531 OpenVZ container id (CTID) and Docker container id (CID).
1532 PRC For every process one line is shown.
1534 Subsequent fields: PID, name (between brackets), state, total
1535 number of clock-ticks per second for this machine, CPU-con‐
1536 sumption in user mode (clockticks), CPU-consumption in system
1537 mode (clockticks), nice value, priority, realtime priority,
1538 scheduling policy, current CPU, sleep average, TGID (group
1539 number of related tasks/threads) and is_process (y/n).
1540 PRE For every process one line is shown.
1542 Subsequent fields: PID, name (between brackets), process
1543 state, GPU state (A for active, E for exited, N for no GPU
1544 user), number of GPUs used by this process, bitlist reflecting
1545 used GPUs, GPU busy percentage during interval, memory busy
1546 percentage during interval, memory occupation (KiB) at this
1547 moment cumulative memory occupation (KiB) during interval, and
1548 number of samples taken during interval.
1549 PRM For every process one line is shown.
1551 Subsequent fields: PID, name (between brackets), state, page
1552 size for this machine (in bytes), virtual memory size
1553 (Kbytes), resident memory size (Kbytes), shared text memory
1554 size (Kbytes), virtual memory growth (Kbytes), resident memory
1555 growth (Kbytes), number of minor page faults, number of major
1556 page faults, virtual library exec size (Kbytes), virtual data
1557 size (Kbytes), virtual stack size (Kbytes), swap space used
1558 (Kbytes), TGID (group number of related tasks/threads),
1559 is_process (y/n) and proportional set size (Kbytes) if in 'R'
1560 option is specified.
1561 PRD For every process one line is shown.
1563 Subsequent fields: PID, name (between brackets), state, obso‐
1564 leted kernel patch installed ('n'), standard io statistics
1565 used ('y' or 'n'), number of reads on disk, cumulative number
1566 of sectors read, number of writes on disk, cumulative number
1567 of sectors written, cancelled number of written sectors, TGID
1568 (group number of related tasks/threads) and is_process (y/n).
1569 If the standard I/O statistics (>= 2.6.20) are not used, the
1570 disk I/O counters per process are not relevant. The counters
1571 'number of reads on disk' and 'number of writes on disk' are
1572 obsoleted anyhow.
1573 PRN For every process one line is shown.
1575 Subsequent fields: PID, name (between brackets), state, kernel
1576 module 'netatop' loaded ('y' or 'n'), number of TCP-packets
1577 transmitted, cumulative size of TCP-packets transmitted, num‐
1578 ber of TCP-packets received, cumulative size of TCP-packets
1579 received, number of UDP-packets transmitted, cumulative size
1580 of UDP-packets transmitted, number of UDP-packets received,
1581 cumulative size of UDP-packets transmitted, number of raw
1582 packets transmitted (obsolete, always 0), number of raw pack‐
1583 ets received (obsolete, always 0), TGID (group number of
1584 related tasks/threads) and is_process (y/n).
1585 If the kernel module is not active, the network I/O counters
1586 per process are not relevant.
1587
1589 By sending the SIGUSR1 signal to atop a new sample will be forced, even
1590 if the current timer interval has not exceeded yet. The behavior is
1591 similar to pressing the `t` key in an interactive session.
1592 By sending the SIGUSR2 signal to atop a final sample will be forced
1594 after which atop will terminate.
1595
1597 To monitor the current system load interactively with an interval of 5
1598 seconds:
1599 atop 5
1601 To monitor the system load and write it to a file (in plain ASCII) with
1603 an interval of one minute during half an hour with active processes
1604 sorted on memory consumption:
1605 atop -M 60 30 > /log/atop.mem
1607 Store information about the system and process activity in binary com‐
1609 pressed form to a file with an interval of ten minutes during an hour:
1610 atop -w /tmp/atop.raw 600 6
1612 View the contents of this file interactively:
1614 atop -r /tmp/atop.raw
1616 View the processor and disk utilization of this file in parseable for‐
1618 mat:
1619 atop -PCPU,DSK -r /tmp/atop.raw
1621 View the contents of today's standard logfile interactively:
1623 atop -r
1625 View the contents of the standard logfile of the day before yesterday
1627 interactively:
1628 atop -r yy
1630 View the contents of the standard logfile of 2014, June 7 from 02:00 PM
1632 onwards interactively:
1633 atop -r 20140607 -b 14:00
1635
1637 /var/run/pacct_shadow.d/
1638 Directory containing the process accounting shadow files that are
1639 used by atop when the atopacctd daemon is active.
1640 /var/cache/atop.d/atop.acct
1642 File in which the kernel writes the accounting records when atop
1643 itself has activated the process accounting mechanism.
1644 /etc/atoprc
1646 Configuration file containing system-wide default values. See
1647 related man-page.
1648 ~/.atoprc
1650 Configuration file containing personal default values. See
1651 related man-page.
1652 /etc/default/atop
1654 Configuration file to overrule the settings of atop that runs in
1655 the background to create the daily logfile. This file is not cre‐
1656 ated or overwritten when atop is installed, so it has to be cre‐
1657 ated manually to override the default settings. The default set‐
1658 tings are:
1659 LOGOPTS="-R"
1661 LOGINTERVAL=600
1662 LOGGENERATIONS=28
1663 /var/log/atop/atop_YYYYMMDD
1665 Raw file, where YYYYMMDD are digits representing the current date.
1666 This name is used by the script atop.daily as default name for the
1667 output file, and by atop as default name for the input file when
1668 using the -r flag.
1669 All binary system and process level data in this file has been
1670 stored in compressed format.
1671 /var/run/netatop.log
1673 File that contains the netpertask structs containing the network
1674 counters of exited processes. These structs are written by the
1675 netatopd daemon and read by atop after reading the standard
1676 process accounting records.
1677
1679 atopsar(1), atopconvert(1), atoprc(5), atopacctd(8), netatop(4),
1680 netatopd(8), atopgpud(8), logrotate(8)
1681 https://www.atoptool.nl
1682
1684 Gerlof Langeveld (gerlof.langeveld@atoptool.nl)
1685 JC van Winkel
1686Linux January 2019 ATOP(1)