1ATOP(1)                     General Commands Manual                    ATOP(1)
2
3
4

NAME

6       atop - Advanced System & Process Monitor
7

SYNOPSIS

9       Interactive Usage:
10
11       atop  [-g|-m|-d|-n|-u|-p|-s|-c|-v|-o|-y|-Y] [-C|-M|-D|-N|-A] [-afFG1xR]
12       [-L linelen] [-Plabel[,label]... [-Z]] [ interval [ samples ]]
13
14       Writing and reading raw logfiles:
15
16       atop -w rawfile [-a] [-S] [ interval [ samples ]]
17       atop  -r  [  rawfile  ]  [-b  [YYYYMMDD]hhmm  ]  [-e  [YYYYMMDD]hhmm  ]
18       [-g|-m|-d|-n|-u|-p|-s|-c|-v|-o|-y|-Y]  [-C|-M|-D|-N|-A]  [-fFG1xR]  [-L
19       linelen] [-Plabel[,label]... [-Z]]
20

DESCRIPTION

22       The program atop is an interactive monitor to view the load on a  Linux
23       system.   It  shows  the  occupation  of the most critical hardware re‐
24       sources (from a performance point of view) on system level,  i.e.  cpu,
25       memory, disk and network.
26       It  also  shows  which processes are responsible for the indicated load
27       with respect to cpu and memory load on process  level.   Disk  load  is
28       shown  per  process  if  "storage  accounting" is active in the kernel.
29       Network load is shown per process if the kernel  module  `netatop'  has
30       been installed.
31
32       The  initial  screen  shows if atop runs with restricted view (unprivi‐
33       leged) or unrestricted view (privileged).  In case of  restricted  view
34       atop does not have the privileges (root identity or necessary capabili‐
35       ties) to retrieve all counter values on system  level  and  on  process
36       level.
37
38       Every interval (default: 10 seconds) information is shown about the re‐
39       source occupation on system level (cpu, memory, disks and network  lay‐
40       ers), followed by a list of processes which have been active during the
41       last interval (note that all processes that were unchanged  during  the
42       last interval are not shown, unless the key 'a' has been pressed or un‐
43       less sorting on memory occupation is done).  If the list of active pro‐
44       cesses does not entirely fit on the screen, only the top of the list is
45       shown (sorted in order of activity).
46       The intervals are repeated till the number  of  samples  (specified  as
47       command  argument) is reached, or till the key 'q' is pressed in inter‐
48       active mode.
49
50       When atop is started, it checks whether the standard output channel  is
51       connected to a screen, or to a file/pipe. In the first case it produces
52       screen control codes (via the ncurses  library)  and  behaves  interac‐
53       tively; in the second case it produces flat ASCII-output.
54
55       In  interactive mode, the output of atop scales dynamically to the cur‐
56       rent dimensions of the screen/window.
57       If the window is resized horizontally, columns will be added or removed
58       automatically.  For this purpose, every column has a particular weight.
59       The columns with the highest weights that fit within the current  width
60       will be shown.
61       If  the  window is resized vertically, lines of the process/thread list
62       will be added or removed automatically.
63
64       Furthermore in interactive mode the output of atop can be controlled by
65       pressing  particular keys.  However it is also possible to specify such
66       key as flag on the command line. In that case atop switches to the  in‐
67       dicated  mode  on  beforehand; this mode can be modified again interac‐
68       tively. Specifying such key as flag is especially useful  when  running
69       atop  with  output  to a pipe or file (non-interactively).  These flags
70       are the same as the keys that can be pressed in interactive  mode  (see
71       section INTERACTIVE COMMANDS).
72       Additional  flags  are available to support storage of atop-data in raw
73       format (see section RAW DATA STORAGE).
74

PROCESS ACCOUNTING

76       With every interval, atop reads the kernel administration to obtain in‐
77       formation about all running processes.  However, it is likely that dur‐
78       ing the interval also processes have terminated.  These processes might
79       have consumed system resources during this interval as well before they
80       terminated.  Therefor,  atop  tries  to  read  the  process  accounting
81       records that contain the accounting information of terminated processes
82       and report these processes too.  Only when the process accounting mech‐
83       anism  in  the  kernel is activated, the kernel writes such process ac‐
84       counting record to a file for every process that terminates.
85
86       There are various ways for atop to get access to the process accounting
87       records (tried in this order):
88
89       1.  When  the  environment  variable  ATOPACCT is set, it specifies the
90           name of the process accounting file.  In  that  case,  process  ac‐
91           counting  for  this  file should have been activated on beforehand.
92           Before opening this file for reading, atop drops  its  root  privi‐
93           leges (if any).
94           When  this  environment  variable  is  present  but its contents is
95           empty, process accounting will not be used at all.
96
97       2.  This is the preferred way of handling process accounting records!
98           When the atopacctd daemon is active, it has activated  the  process
99           accounting  mechanism  in  the kernel and transfers to original ac‐
100           counting records to shadow files.  In that  case,  atop  drops  its
101           root privileges and opens the current shadow file for reading.
102           This  way is preferred, because the atopacctd daemon maintains full
103           control of the size of the original process accounting file written
104           by the kernel and the shadow files read by the atop process(es).
105
106           The  atopacct  service will be activated before the atop service to
107           enable atop to detect that process accounting  is  managed  by  the
108           atopacctd  daemon.  As a forking service, atopacctd takes care that
109           all directories and files are initialized before the parent process
110           dies. The child process continues as the daemon process.
111
112           For further information, refer to the atopacctd man page.
113
114       3.  When  the  atopacctd  daemon  is  not  active, atop verifies if the
115           process accounting mechanism has been switched on via the  separate
116           psacct  or acct package (the package name depends on the Linux dis‐
117           tro). In that case,  one  of  the  files  /var/log/pacct,  /var/ac‐
118           count/pacct or /var/log/account/pacct is in use as process account‐
119           ing file and atop opens this file for reading.
120
121       4.  As a last possibility, atop itself tries to  activate  the  process
122           accounting  mechanism  (requires  root  privileges)  using the file
123           /var/cache/atop.d/atop.acct (to be written by  the  kernel,  to  be
124           read  by atop itself). Process accounting remains active as long as
125           at least one atop process is alive.  Whenever the last atop process
126           stops (either by pressing `q' or by `kill -15'), it deactivates the
127           process accounting mechanism again. Therefor you should never  ter‐
128           minate  atop  by  `kill  -9', because then it has no chance to stop
129           process accounting.  As a result, the accounting file may consume a
130           lot of disk space after a while.
131           To  avoid  that  the process accounting file consumes too much disk
132           space, atop verifies at the end of every sample if the size of  the
133           process accounting file exceeds 200 MiB and if this atop process is
134           the only one that is currently using the file.  In  that  case  the
135           file is truncated to a size of zero.
136
137           Notice  that  root-privileges are required to switch on/off process
138           accounting in the kernel. You can start atop  as  a  root  user  or
139           specify setuid-root privileges to the executable file.  In the lat‐
140           ter case, atop switches on process accounting and drops  the  root-
141           privileges again.
142           If  atop does not run with root-privileges, it does not show infor‐
143           mation about finished processes.  It indicates this situation  with
144           the  message message `no procacct` in the top-right corner (instead
145           of the counter that shows the number of exited processes).
146
147       When during one interval a lot of processes have finished,  atop  might
148       grow tremendously in memory when reading all process accounting records
149       at the end of the interval. To avoid such excessive growth,  atop  will
150       never  read  more than 50 MiB with process information from the process
151       accounting file per interval (approx. 70000  finished  processes).   In
152       interactive  mode  a  warning  is  given  whenever  processes have been
153       skipped for this reason.
154

COLORS

156       For the resource consumption on system level, atop uses colors to indi‐
157       cate  that  a critical occupation percentage has been (almost) reached.
158       A critical occupation percentage means that is likely  that  this  load
159       causes a noticeable negative performance influence for applications us‐
160       ing this resource. The critical percentage depends on the type  of  re‐
161       source: e.g. the performance influence of a disk with a busy percentage
162       of 80% might be more noticeable for applications/user than a CPU with a
163       busy percentage of 90%.
164       Currently  atop  uses  the  following  default  values  to  calculate a
165       weighted percentage per resource:
166
167        Processor
168            A busy percentage of 90% or higher is considered `critical'.
169
170        Disk
171            A busy percentage of 70% or higher is considered `critical'.
172
173        Network
174            A busy percentage of 90% or higher for the load of an interface is
175            considered `critical'.
176
177        Memory
178            An  occupation percentage of 90% is considered `critical'.  Notice
179            that this occupation percentage is the accumulated memory consump‐
180            tion  of the kernel (including slab) and all processes; the memory
181            for the page cache (`cache' and `buff' in the  MEM-line)  and  the
182            reclaimable part of the slab (`slrec`) is not implied!
183            If  the  number  of pages swapped out (`swout' in the PAG-line) is
184            larger than 10 per  second,  the  memory  resource  is  considered
185            `critical'.   A  value of at least 1 per second is considered `al‐
186            most critical'.
187            If the committed virtual memory exceeds  the  limit  (`vmcom'  and
188            `vmlim'  in the SWP-line), the SWP-line is colored due to overcom‐
189            mitting the system.
190
191        Swap
192            An occupation percentage of 80% is considered  `critical'  because
193            swap space might be completely exhausted in the near future; it is
194            not critical from a performance point-of-view.
195
196       These default values can be modified in  the  configuration  file  (see
197       separate man-page of atoprc).
198
199       When  a  resource  exceeds its critical occupation percentage, the con‐
200       cerning values in the screen line are colored red by default.
201       When a resource exceeded (default) 80% of its critical  percentage  (so
202       it  is  almost  critical), the concerning values in the screen line are
203       colored cyan by default. This `almost critical percentage'  (one  value
204       for all resources) can be modified in the configuration file (see sepa‐
205       rate man-page of atoprc).
206       The default colors red and cyan can be modified  in  the  configuration
207       file as well (see separate man-page of atoprc).
208
209       With the key 'x' (or flag -x), the use of colors can be suppressed.
210

NETATOP MODULE

212       Per-process  and per-thread network activity can be measured by the ne‐
213       tatop kernel module. You can download this kernel module from the  web‐
214       site  (mentioned at the end of this manual page) and install it on your
215       system if the kernel version is 2.6.24 or newer.
216       When atop gathers counters for a new interval, it verifies if  the  ne‐
217       tatop module is currently active. If so, atop obtains the relevant net‐
218       work counters from this module and shows the number  of  sent  and  re‐
219       ceived  packets  per process/thread in the generic screen. Besides, de‐
220       tailed counters can be requested by pressing the `n' key.
221       When the netatopd daemon is running as well, atop also reads  the  net‐
222       work  counters of exited processes that are logged by this daemon (com‐
223       parable with process accounting).
224
225       More information about the optional netatop kernel module and  the  ne‐
226       tatopd  daemon can be found in the concerning man-pages and on the web‐
227       site mentioned at the end of this manual page.
228

GPU STATISTICS GATHERING

230       GPU statistics can be gathered by atopgpud which  is  a  separate  data
231       collection  daemon process.  It gathers cumulative utilization counters
232       of every Nvidia GPU in the system, as well as utilization  counters  of
233       every  process  that  uses a GPU.  When atop notices that the daemon is
234       active, it reads these GPU utilization counters with every interval.
235
236       The atopgpud daemon is written  in  Python,  so  a  Python  interpreter
237       should be installed on the target system. The Python code of the daemon
238       is compatible with Python version 2 and version 3.  For  the  gathering
239       of  the  statistics,  the  pynvml module is used by the daemon. Be sure
240       that this module is installed on the target  system  before  activating
241       the  daemon,  by running the command as root pip (the command pip might
242       be exchanged by pip3 in case of Python3):
243
244         pip install nvidia-ml-py
245
246       The atopgpud daemon is installed by default as part of the  atop  pack‐
247       age,  but  it  is not automatically enabled.  The daemon can be enabled
248       and started now by running the following commands (as root):
249
250         systemctl enable atopgpu
251         systemctl start atopgpu
252
253       Find a description about the utilization counters in the section OUTPUT
254       DESCRIPTION.
255

INTERACTIVE COMMANDS

257       When  running  atop  interactively (no output redirection), keys can be
258       pressed to control the output. In general, lower case keys can be  used
259       to  show other information for the active processes and upper case keys
260       can be used to influence the sort order of  the  active  process/thread
261       list.
262
263       g    Show generic output (default).
264
265            Per  process  the  following fields are shown in case of a window-
266            width of 80 positions: process-id, cpu consumption during the last
267            interval  in system and user mode, the virtual and resident memory
268            growth of the process.
269
270            The subsequent columns depend on the used kernel:
271            When the kernel supports "storage  accounting"  (>=  2.6.20),  the
272            data transfer for read/write on disk, the status and exit code are
273            shown for each process.  When the kernel does not support "storage
274            accounting",  the username, number of threads in the thread group,
275            the status and exit code are shown.
276            When the kernel module 'netatop' is loaded, the data transfer  for
277            send/receive of network packets is shown for each process.
278            The  last columns contain the state, the occupation percentage for
279            the chosen resource (default: cpu) and the process name.
280
281            When more than 80 positions are available,  other  information  is
282            added.
283
284       m    Show memory related output.
285
286            Per  process  the  following fields are shown in case of a window-
287            width of 80 positions: process-id, minor and major memory  faults,
288            size  of  virtual  shared  text, total virtual process size, total
289            resident process size, virtual and resident growth during last in‐
290            terval, memory occupation percentage and process name.
291
292            When  more  than  80 positions are available, other information is
293            added.
294
295            For memory consumption, always all processes are shown  (also  the
296            processes that were not active during the interval).
297
298       d    Show disk-related output.
299
300            When  "storage  accounting" is active in the kernel, the following
301            fields are shown: process-id,  amount  of  data  read  from  disk,
302            amount  of  data  written to disk, amount of data that was written
303            but has been withdrawn again (WCANCL), disk occupation  percentage
304            and process name.
305
306       n    Show network related output.
307
308            Per  process  the  following fields are shown in case of a window-
309            width of 80 positions: process-id, thread-id, total bandwidth  for
310            received  packets, total bandwidth for sent packets, number of re‐
311            ceived TCP packets with the average size per  packet  (in  bytes),
312            number  of  sent  TCP packets with the average size per packet (in
313            bytes), number of received UDP packets with the average  size  per
314            packet  (in  bytes),  number  of sent UDP packets with the average
315            size per packet (in bytes), the network occupation percentage  and
316            process name.
317            This information can only be shown when kernel module `netatop' is
318            installed.
319
320            When more than 80 positions are available,  other  information  is
321            added.
322
323       s    Show scheduling characteristics.
324
325            Per  process  the  following fields are shown in case of a window-
326            width of 80 positions: process-id,  number  of  threads  in  state
327            'running' (R), number of threads in state 'interruptible sleeping'
328            (S), number of threads in state  'uninterruptible  sleeping'  (D),
329            scheduling policy (normal timesharing, realtime round-robin, real‐
330            time fifo), nice value, priority, realtime priority, current  pro‐
331            cessor,  status,  exit  code, state, the occupation percentage for
332            the chosen resource and the process name.
333
334            When more than 80 positions are available,  other  information  is
335            added.
336
337       v    Show various process characteristics.
338
339            Per  process  the  following fields are shown in case of a window-
340            width of 80 positions: process-id, user name and group, start date
341            and  time,  status  (e.g.  exit code if the process has finished),
342            state, the occupation percentage for the chosen resource  and  the
343            process name.
344
345            When  more  than  80 positions are available, other information is
346            added.
347
348       c    Show the command line of the process.
349
350            Per process the following fields are shown: process-id, the  occu‐
351            pation percentage for the chosen resource and the command line in‐
352            cluding arguments.
353
354       e    Show GPU utilization.
355
356            Per process at least the following fields are  shown:  process-id,
357            range of GPU numbers on which the process currently runs, GPU busy
358            percentage on all GPUs, memory  busy  percentage  (i.e.  read  and
359            write  accesses  on  memory) on all GPUs, memory occupation at the
360            moment of the sample, average memory occupation during the sample,
361            and GPU percentage.
362
363            When  the  atopgpud  daemon does not run with root privileges, the
364            GPU busy percentage and the memory busy percentage are not  avail‐
365            able  on  process  level.   In  that  case,  the GPU percentage on
366            process level reflects the GPU memory occupation  instead  of  the
367            GPU busy percentage (which is preferred).
368
369       o    Show the user-defined line of the process.
370
371            In the configuration file the keyword ownprocline can be specified
372            with the description of a user-defined output-line.
373            Refer to the man-page of atoprc for a detailed description.
374
375       y    Show the individual threads within a process (toggle).
376
377            Single-threaded processes are still shown as one line.
378            For multi-threaded processes,  one  line  represents  the  process
379            while additional lines show the activity per individual thread (in
380            a different color). Depending on the option  'a'  (all  or  active
381            toggle),  all  threads are shown or only the threads that were ac‐
382            tive during the last interval.  Depending on the option 'Y'  (sort
383            threads),  the  threads  per  process will be sorted on the chosen
384            sort criterium or not.
385            Whether this key is active or not can be seen in the header line.
386
387       Y    Sort the threads per process when combined with option  'y'  (tog‐
388            gle).
389
390       u    Show the process activity accumulated per user.
391
392            Per  user  the following fields are shown: number of processes ac‐
393            tive or terminated during last interval (or in total  if  combined
394            with  command `a'), accumulated cpu consumption during last inter‐
395            val in system and user mode, the current virtual and resident mem‐
396            ory  space  consumed  by active processes (or all processes of the
397            user if combined with command `a').
398            When "storage accounting" is active in the kernel, the accumulated
399            read  and write throughput on disk is shown.  When the kernel mod‐
400            ule `netatop' has been installed, the number of received and  sent
401            network packets are shown.
402            The last columns contain the accumulated occupation percentage for
403            the chosen resource (default: cpu) and the user name.
404
405       p    Show the process activity accumulated per  program  (i.e.  process
406            name).
407
408            Per  program  the  following fields are shown: number of processes
409            active or terminated during last interval (or in total if combined
410            with  command `a'), accumulated cpu consumption during last inter‐
411            val in system and user mode, the current virtual and resident mem‐
412            ory  space  consumed  by active processes (or all processes of the
413            user if combined with command `a').
414            When "storage accounting" is active in the kernel, the accumulated
415            read  and write throughput on disk is shown.  When the kernel mod‐
416            ule `netatop' has been installed, the number of received and  sent
417            network packets are shown.
418            The last columns contain the accumulated occupation percentage for
419            the chosen resource (default: cpu) and the program name.
420
421       j    Show the process activity accumulated per Docker container.
422
423            Per container the following fields are shown: number of  processes
424            active or terminated during last interval (or in total if combined
425            with command `a'), accumulated cpu consumption during last  inter‐
426            val in system and user mode, the current virtual and resident mem‐
427            ory space consumed by active processes (or all  processes  of  the
428            user if combined with command `a').
429            When "storage accounting" is active in the kernel, the accumulated
430            read and write throughput on disk is shown.  When the kernel  mod‐
431            ule  `netatop' has been installed, the number of received and sent
432            network packets are shown.
433            The last columns contain the accumulated occupation percentage for
434            the  chosen  resource  (default:  cpu) and the Docker container id
435            (CID).
436
437       C    Sort the current list in the order of cpu  consumption  (default).
438            The one-but-last column changes to ``CPU''.
439
440       E    Sort  the current list in the order of GPU utilization (preferred,
441            but only applicable when the atopgpud daemon runs under root priv‐
442            ileges)  or the order of GPU memory occupation).  The one-but-last
443            column changes to ``GPU''.
444
445       M    Sort the current list in the order of resident memory consumption.
446            The  one-but-last column changes to ``MEM''. In case of sorting on
447            memory, the full process list will be shown (not only  the  active
448            processes).
449
450       D    Sort  the  current list in the order of disk accesses issued.  The
451            one-but-last column changes to ``DSK''.
452
453       N    Sort the current list in the order of network bandwidth  (received
454            and transmitted).  The one-but-last column changes to ``NET''.
455
456       A    Sort  the current list automatically in the order of the most busy
457            system resource during this  interval.   The  one-but-last  column
458            shows either ``ACPU'', ``AMEM'', ``ADSK'' or ``ANET'' (the preced‐
459            ing 'A' indicates automatic sorting-order).   The  most  busy  re‐
460            source is determined by comparing the weighted busy-percentages of
461            the system resources, as described earlier in the section COLORS.
462            This option remains valid until another sorting-order  is  explic‐
463            itly selected again.
464            A  sorting-order  for disk is only possible when "storage account‐
465            ing" is active.  A sorting-order for network is only possible when
466            the kernel module `netatop' is loaded.
467
468       Miscellaneous interactive commands:
469
470       ?    Request for help information (also the key 'h' can be pressed).
471
472       V    Request for version information (version number and date).
473
474       R    Gather  and calculate the proportional set size of processes (tog‐
475            gle).  Gathering of all values that are needed  to  calculate  the
476            PSIZE  of  a  process  is  a very time-consuming task, so this key
477            should only be active when analyzing the resident memory  consump‐
478            tion of processes.
479
480       W    Get  the WCHAN per thread (toggle).  Gathering of the WCHAN string
481            per thread is a relatively time-consuming task, so this key should
482            only be made active when analyzing the reason for threads to be in
483            sleep state.
484
485       x    Suppress colors to highlight critical resources (toggle).
486            Whether this key is active or not can be seen in the header line.
487
488       z    The pause key can be used to freeze the current situation in order
489            to investigate the output on the screen. While atop is paused, the
490            keys described above can be  pressed  to  show  other  information
491            about  the  current  list of processes.  Whenever the pause key is
492            pressed again, atop will continue with a next sample.
493
494       i    Modify the interval timer (default: 10 seconds).  If  an  interval
495            timer of 0 is entered, the interval timer is switched off. In that
496            case a new sample can only be triggered manually by  pressing  the
497            key 't'.
498
499       t    Trigger a new sample manually. This key can be pressed if the cur‐
500            rent sample should be finished before the timer has  exceeded,  or
501            if  no  timer  is set at all (interval timer defined as 0). In the
502            latter case atop can be used as a stopwatch to  measure  the  load
503            being  caused  by  a  particular  application transaction, without
504            knowing on beforehand how many seconds this transaction will last.
505
506            When viewing the contents of a raw file this key can  be  used  to
507            show the next sample from the file. This key can also be used when
508            viewing raw data via a pipe.
509
510       T    When viewing the contents of a raw file this key can  be  used  to
511            show  the  previous sample from the file, however not when reading
512            raw data from a pipe.
513
514       b    When viewing the contents of a raw file, this key can be  used  to
515            branch  to  a  certain timestamp within the file either forward or
516            backward.  When viewing raw data from a pipe only forward branches
517            are possible.
518
519       r    Reset  all counters to zero to see the system and process activity
520            since boot again.
521
522            When viewing the contents of a raw file, this key can be  used  to
523            rewind to the beginning of the file again (except when reading raw
524            data from a pipe).
525
526       U    Specify a search string for specific user names as a  regular  ex‐
527            pression.  From now on, only (active) processes will be shown from
528            a user which matches the regular expression.  The  system  statis‐
529            tics  are  still system wide.  If the Enter-key is pressed without
530            specifying a name, (active) processes of all users will  be  shown
531            again.
532            Whether this key is active or not can be seen in the header line.
533
534       I    Specify a list with one or more PIDs to be selected.  From now on,
535            only processes will be shown with a PID which matches one  of  the
536            given  list.  The system statistics are still system wide.  If the
537            Enter-key is pressed without specifying a PID, all  (active)  pro‐
538            cesses will be shown again.
539            Whether this key is active or not can be seen in the header line.
540
541       P    Specify  a  search  string for specific process names as a regular
542            expression.  From now on, only processes will be shown with a name
543            which  matches  the regular expression.  The system statistics are
544            still system wide.  If the Enter-key is pressed without specifying
545            a name, all (active) processes will be shown again.
546            Whether this key is active or not can be seen in the header line.
547
548       /    Specify a specific command line search string as a regular expres‐
549            sion.  From now on, only processes will be shown  with  a  command
550            line  which matches the regular expression.  The system statistics
551            are still system wide.  If the Enter-key is pressed without speci‐
552            fying a string, all (active) processes will be shown again.
553            Whether this key is active or not can be seen in the header line.
554
555       J    Specify  a  Docker  container  id  of 12 (hexadecimal) characters.
556            From now on, only processes will be shown that run  in  that  spe‐
557            cific  Docker  container  (CID).   The system statistics are still
558            system wide.  If the Enter-key is  pressed  without  specifying  a
559            container id, all (active) processes will be shown again.
560            Whether this key is active or not can be seen in the header line.
561
562       Q    Specify a comma-separated list of process/thread state characters.
563            From now on, only processes/threads will  be  shown  that  are  in
564            those  specific  states.   Accepted  states  are:  R  (running), S
565            (sleeping), D (disk sleep), I  (idle),  T  (stopped),  t  (tracing
566            stop), X (dead), Z (zombie) and P (parked).  The system statistics
567            are still system wide.  If the Enter-key is pressed without speci‐
568            fying a state, all (active) processes/threads will be shown again.
569            Whether this key is active or not can be seen in the header line.
570
571       S    Specify search strings for specific logical volume names, specific
572            disk names  and  specific  network  interface  names.  All  search
573            strings  are  interpreted  as a regular expressions.  From now on,
574            only those system resources are shown that  match  the  concerning
575            regular  expression.  If the Enter-key is pressed without specify‐
576            ing a search string, all (active) system resources  of  that  type
577            will be shown again.
578            Whether this key is active or not can be seen in the header line.
579
580       a    The  `all/active'  key  can  be  used to toggle between only show‐
581            ing/accumulating the processes that were active  during  the  last
582            interval (default) or showing/accumulating all processes.
583            Whether this key is active or not can be seen in the header line.
584
585       G    By  default,  atop  shows/accumulates the processes that are alive
586            and the processes that are exited during the last  interval.  With
587            this key (toggle), showing/accumulating the processes that are ex‐
588            ited can be suppressed.
589            Whether this key is active or not can be seen in the header line.
590
591       f    Show a fixed (maximum) number of header lines for system resources
592            (toggle).   By  default  only the lines are shown about system re‐
593            sources (CPUs, paging, logical volumes, disks, network interfaces)
594            that  really have been active during the last interval.  With this
595            key you can force atop to show  lines  of  inactive  resources  as
596            well.
597            Whether this key is active or not can be seen in the header line.
598
599       F    Suppress  sorting of system resources (toggle).  By default system
600            resources (CPUs, logical volumes, disks, network  interfaces)  are
601            sorted on utilization.
602            Whether this key is active or not can be seen in the header line.
603
604       1    Show  relevant  counters  as  an average per second (in the format
605            `..../s') instead of as a total during the interval (toggle).
606            Whether this key is active or not can be seen in the header line.
607
608       l    Limit the number of system level lines for the  counters  per-cpu,
609            the active disks and the network interfaces.  By default lines are
610            shown of all CPUs, disks and network interfaces  which  have  been
611            active during the last interval.  Limiting these lines can be use‐
612            ful on systems with huge number CPUs, disks or interfaces in order
613            to be able to run atop on a screen/window with e.g. only 24 lines.
614            For  all  mentioned  resources  the maximum number of lines can be
615            specified interactively. When using the flag -l the maximum number
616            of  per-cpu lines is set to 0, the maximum number of disk lines to
617            5 and the maximum number of interface lines to  3.   These  values
618            can be modified again in interactive mode.
619
620       k    Send a signal to an active process (a.k.a. kill a process).
621
622       q    Quit the program.
623
624       PgDn Show the next page of the process/thread list.
625            With  the  arrow-down  key the list can be scrolled downwards with
626            single lines.
627
628       ^F   Show the next page of the process/thread list (forward).
629            With the arrow-down key the list can be  scrolled  downwards  with
630            single lines.
631
632       PgUp Show the previous page of the process/thread list.
633            With the arrow-up key the list can be scrolled upwards with single
634            lines.
635
636       ^B   Show the previous page of the process/thread list (backward).
637            With the arrow-up key the list can be scrolled upwards with single
638            lines.
639
640       ^L   Redraw the screen.
641

RAW DATA STORAGE

643       In  order  to  store  system and process level statistics for long-term
644       analysis (e.g. to check the system load and the active  processes  run‐
645       ning yesterday between 3:00 and 4:00 PM), atop can store the system and
646       process level statistics in compressed binary format in a raw file with
647       the  flag -w followed by the filename.  If this file already exists and
648       is recognized as a raw data file, atop will append new samples  to  the
649       file  (starting  with a sample which reflects the activity since boot);
650       if the file does not exist, it will be created.
651       All information about processes and threads is stored in the raw file.
652       The interval (default: 10 seconds) and number of samples (default:  in‐
653       finite)  can be passed as last arguments. Instead of the number of sam‐
654       ples, the flag -S can be used to indicate that atop should finish  any‐
655       how before midnight.
656
657       A  raw  file can be read and visualized again with the flag -r followed
658       by  the   filename.   If   no   filename   is   specified,   the   file
659       /var/log/atop/atop_YYYYMMDD  is  opened  for  input (where YYYYMMDD are
660       digits representing the current date).  If a filename is  specified  in
661       the   format   YYYYMMDD   (representing   any  valid  date),  the  file
662       /var/log/atop/atop_YYYYMMDD is opened.  If a filename with the symbolic
663       name  y  is specified, yesterday's daily logfile is opened (this can be
664       repeated so 'yyyy' indicates the logfile of four  days  ago).   If  the
665       filename - is used, stdin will be read.
666       The  samples from the file can be viewed interactively by using the key
667       't' to show the next sample, the key 'T' to show the  previous  sample,
668       the  key 'b' to branch to a particular time or the key 'r' to rewind to
669       the begin of the file.
670       When output is redirected to a file or pipe, atop prints all samples in
671       plain  ASCII.  The  default  line length is 80 characters in that case;
672       with the flag -L followed by an alternate line length, more  (or  less)
673       columns will be shown.
674       With  the  flag -b (begin time) and/or -e (end time) followed by a time
675       argument of the form [YYYYMMDD]hhmm, a certain time period  within  the
676       raw file can be selected.
677
678       Every day at midnight atop is restarted to write compressed binary data
679       to the file /var/log/atop/atop_YYYYMMDD with an interval of 10  minutes
680       by default.
681       Furthermore  all  raw files are removed that are older than 28 days (by
682       default).
683       The mentioned default values can be  overruled  in  the  file  /etc/de‐
684       fault/atop  that  might  contain  other  values for LOGOPTS (by default
685       without any flag), LOGINTERVAL (in seconds, by default 600), LOGGENERA‐
686       TIONS  (in  days,  by default 28), and LOGPATH (directory in which log‐
687       files are stored).
688
689       Unfortunately, it is not always possible to keep the format of the  raw
690       files  compatible in newer versions of atop especially when lots of new
691       counters have to be maintained.  Therefore, the program atopconvert  is
692       installed  to convert a raw file created by an older version of atop to
693       a raw file that can be read by a newer version of  atop  (see  the  man
694       page of atopconvert for more details).
695
696

OUTPUT DESCRIPTION

698       The  first  sample  shows  the  system  level  activity since boot (the
699       elapsed time in the header shows the time since boot).  Note that  par‐
700       ticular counters could have reached their maximum value (several times)
701       and started by zero again, so do not rely on these figures.
702
703       For every sample atop first shows the lines related to system level ac‐
704       tivity.  If  a  particular system resource has not been used during the
705       interval, the entire line related to this resource  is  suppressed.  So
706       the number of system level lines may vary for each sample.
707       After  that  a list is shown of processes which have been active during
708       the last interval. This list is by default sorted on  cpu  consumption,
709       but  this  order  can  be  changed by the keys which are previously de‐
710       scribed.
711
712       If values have to be shown by atop which  do  not  fit  in  the  column
713       width, another format is used. If e.g. a cpu-consumption of 233216 mil‐
714       liseconds should be shown in a column width of 4 positions, it is shown
715       as `233s' (in seconds).  For large memory figures, another unit is cho‐
716       sen if the value does not fit (Mb instead of Kb, Gb instead of  Mb,  Tb
717       instead  of Gb, ...).  For other values, a kind of exponent notation is
718       used (value 123456789 shown in a column of 5 positions gives 123e6).
719

OUTPUT DESCRIPTION - SYSTEM LEVEL

721       The system level information consists of the following output lines:
722
723       PRC  Process and thread level totals.
724            This line contains the total cpu  time  consumed  in  system  mode
725            (`sys')  and  in user mode (`user'), the total number of processes
726            present at this moment (`#proc'),  the  total  number  of  threads
727            present at this moment in state `running' (`#trun'), `sleeping in‐
728            terruptible' (`#tslpi') and `sleeping uninterruptible' (`#tslpu'),
729            the  number  of  zombie processes (`#zombie'), the number of clone
730            system calls (`clones'), and the number of  processes  that  ended
731            during the interval (`#exit') when process accounting is used. In‐
732            stead of `#exit` the last column may  indicate  that  process  ac‐
733            counting could not be activated (`no procacct`).
734            If  the  screen-width does not allow all of these counters, only a
735            relevant subset is shown.
736
737       CPU  CPU utilization.
738            At least one line is shown for the total occupation  of  all  CPUs
739            together.
740            In  case  of a multi-processor system, an additional line is shown
741            for every individual processor (with `cpu' in lower case),  sorted
742            on  activity.  Inactive  CPUs  will  not be shown by default.  The
743            lines showing the per-cpu occupation contain the cpu number in the
744            field combined with the wait percentage.
745
746            Every  line  contains  the  percentage of cpu time spent in kernel
747            mode by all active processes (`sys'), the percentage of  cpu  time
748            consumed in user mode (`user') for all active processes (including
749            processes running with a nice value larger than  zero),  the  per‐
750            centage of cpu time spent for interrupt handling (`irq') including
751            softirq, the percentage of unused cpu time while no processes were
752            waiting  for  disk  I/O (`idle'), and the percentage of unused cpu
753            time while at least one process was waiting for disk I/O (`wait').
754            In case of per-cpu occupation, the cpu number and  the  wait  per‐
755            centage  (`w') for that cpu.  The number of lines showing the per-
756            cpu occupation can be limited.
757
758            For virtual machines, the  steal-percentage  (`steal')  shows  the
759            percentage of cpu time stolen by other virtual machines running on
760            the same hardware.
761            For physical machines hosting one or more  virtual  machines,  the
762            guest-percentage  (`guest')  shows the percentage of cpu time used
763            by the virtual machines. Notice that this percentage overlaps  the
764            user percentage!
765
766            When  PMC performance monitoring counters are supported by the CPU
767            and the kernel (and atop runs with root privileges), the number of
768            instructions per CPU cycle (`ipc') is shown.  The first sample al‐
769            ways shows the value 'initial', because the counters are just  ac‐
770            tivated at the moment that atop is started.
771            When the CPU busy percentage is high and the IPC is less than 1.0,
772            it is likely that the CPU is frequently waiting for memory  access
773            during  instruction  execution (larger CPU caches or faster memory
774            might be helpful to improve performance).  When the CPU busy  per‐
775            centage is high and the IPC is greater than 1.0, it is likely that
776            the CPU is instruction-bound (more/faster cores might  be  helpful
777            to improve performance).
778            Furthermore,  per  CPU  the effective number of cycles (`cycl') is
779            shown.  This value can reach the current CPU frequency if such CPU
780            is 100% busy.  When an idle CPU is halted, the number of effective
781            cycles can be (considerably) lower than the current frequency.
782            Notice that the average instructions per cycle and number  of  cy‐
783            cles is shown in the CPU line for all CPUs.
784            Beware that reading the cycle counter in virtual machines (guests)
785            might introduce performance delays. Therefore this  metric  is  by
786            default  disabled  in  virtual machines. However, with the keyword
787            'perfevents' in the atoprc file this metric can be explicitly  set
788            to 'enable' or 'disable' (see separate man-page of atoprc).
789            See also: http://www.brendangregg.com/blog/2017-05-09/cpu-utiliza
790            tion-is-wrong.html
791
792
793            In case of frequency scaling, all previously  mentioned  CPU  per‐
794            centages  are  relative  to the used scaling of the CPU during the
795            interval.  If a CPU has been active for e.g. 50% in user mode dur‐
796            ing  the interval while the frequency scaling of that CPU was 40%,
797            only 20% of the full capacity of the CPU has  been  used  in  user
798            mode.
799            In  case  that  the kernel module `cpufreq_stats' is active (after
800            issueing `modprobe cpufreq_stats'), the average frequency (`avgf')
801            and the average scaling percentage (`avgscal') is shown. Otherwise
802            the current frequency (`curf') and the current scaling  percentage
803            (`curscal')  is shown at the moment that the sample is taken.  No‐
804            tice that average values for frequency and scaling  are  shown  in
805            the CPU line for every CPU.
806            Frequency  scaling  statistics  are only gathered for systems with
807            maximum 8 CPUs, since gathering of these values per  CPU  is  very
808            time consuming.
809
810            If  the  screen-width does not allow all of these counters, only a
811            relevant subset is shown.
812
813       CPL  CPU load information.
814            This line contains the load average figures reflecting the  number
815            of  threads  that  are available to run on a CPU (i.e. part of the
816            runqueue) or that are waiting for disk I/O. These figures are  av‐
817            eraged over 1 (`avg1'), 5 (`avg5') and 15 (`avg15') minutes.
818            Furthermore  the number of context switches (`csw'), the number of
819            serviced interrupts (`intr') and the number of available CPUs  are
820            shown.
821
822            If  the  screen-width does not allow all of these counters, only a
823            relevant subset is shown.
824
825       GPU  GPU utilization (Nvidia).
826            Read the section GPU STATISTICS GATHERING in this document to find
827            the details about the activation of the atopgpud daemon.
828
829            In  the  first column of every line, the bus-id (last nine charac‐
830            ters) and the GPU number are shown.  The subsequent  columns  show
831            the  percentage of time that one or more kernels were executing on
832            the GPU (`gpubusy'), the percentage of time that  global  (device)
833            memory  was being read or written (`membusy'), the occupation per‐
834            centage of memory (`memocc'), the total memory (`total'), the mem‐
835            ory being in use at the moment of the sample (`used'), the average
836            memory being in use during the sample time (`usavg'),  the  number
837            of  processes  being active on the GPU at the moment of the sample
838            (`#proc'), and the type of GPU.
839
840            If the screen-width does not allow all of these counters,  only  a
841            relevant subset is shown.
842            The number of lines showing the GPUs can be limited.
843
844       MEM  Memory occupation.
845            This  line  contains  the total amount of physical memory (`tot'),
846            the amount of memory which is currently free (`free'), the  amount
847            of memory in use as page cache including the total resident shared
848            memory (`cache'), the amount of memory within the page cache  that
849            has to be flushed to disk (`dirty'), the amount of memory used for
850            filesystem meta data (`buff'), the amount of memory being used for
851            kernel  mallocs  (`slab'),  the  amount of slab memory that is re‐
852            claimable (`slrec'), the resident size of shared memory  including
853            tmpfs  (`shmem`), the resident size of shared memory (`shrss`) the
854            amount of shared memory that is currently swapped  (`shswp`),  the
855            amount  of  memory  that  is currently claimed by vmware's balloon
856            driver (`vmbal`), the amount of memory that is  currently  claimed
857            by  the  ARC (cache) of ZFSonlinux (`zfarc`), the amount of memory
858            that is claimed for huge pages (`hptot`), the amount of huge  page
859            memory  that  is  really  in use (`hpuse`), and the number of NUMA
860            nodes in this system (`numnode').
861
862            If the screen-width does not allow all of these counters,  only  a
863            relevant subset is shown.
864
865       SWP  Swap occupation and overcommit info.
866            This line contains the total amount of swap space on disk (`tot'),
867            the amount of free swap space (`free'), the size of the swap cache
868            (`swcac'),   the   total  size  of  compressed  storage  in  zswap
869            (`zpool`), the total size of the compressed pages stored in  zswap
870            (`zstor'),  the  total  size  of the memory used for KSM (`ksuse`,
871            i.e. shared), and the total size of the memory saved (deduped)  by
872            KSM (`kssav`, i.e. sharing).
873            Furthermore  the  committed virtual memory space (`vmcom') and the
874            maximum limit of the committed space (`vmlim', which is by default
875            swap  size plus 50% of memory size) is shown.  The committed space
876            is the reserved virtual space for all allocations of private  mem‐
877            ory space for processes. The kernel only verifies whether the com‐
878            mitted space exceeds the limit if strict  overcommit  handling  is
879            configured (vm.overcommit_memory is 2).
880
881       NUM  Memory utilization per NUMA node.
882            This  line  shows the total amount of physical memory of this node
883            (`tot'), the amount of free memory (`free'), the amount of  memory
884            for   cached   file  data  (`file'),  modified  cached  file  data
885            (`dirty'), recently used memory (`activ'), less recently used mem‐
886            ory  (`inact'), memory being used for kernel mallocs (`slab'), the
887            amount of slab memory that is reclaimable (`slrec'), shared memory
888            including  tmpfs  (`shmem'),  total  huge  pages (`hptot') and the
889            fragmentation percentage (`frag').
890
891       NUC  CPU utilization per NUMA node (not shown in case of a single  NUMA
892            node).
893            This line shows the utilization percentages of all CPUs related to
894            this NUMA node, categorized for  system mode  (`sys'),  user  mode
895            (`user'),  user  mode  for  niced  processes  (`niced'), idle mode
896            (`idle'), wait mode (`w' preceded by the node  number),  irq  mode
897            (`irq'),  softirq  mode  (`sirq'), steal mode (`steal'), and guest
898            mode (`guest') overlapping user mode.
899
900       PAG  Paging frequency.
901            This line contains the number of scanned pages (`scan') due to the
902            fact that free memory drops below a particular threshold, the num‐
903            ber times that the kernel tries to reclaim pages due to an  urgent
904            need  (`stall'),  the  number of process stalls to run memory com‐
905            paction to allocate huge pages (`compact'),  the  number  of  NUMA
906            pages  migrated  (`numamig'), and the total number of memory pages
907            migrated succesfully e.g. between NUMA  nodes  or  for  compaction
908            (`migrate') are shown.
909            Also  the  number  of memory pages the system read from swap space
910            (`swin'), the number of memory pages  the  system  wrote  to  swap
911            space   (`swout'),   and   the   number   of  out-of-memory  kills
912            (`oomkill').
913
914       PSI  Pressure Stall Information.
915            This line contains percentages about resource pressure related  to
916            CPU,  memory  and I/O. Certain percentages refer to 'some' meaning
917            that some processes/threads were delayed due to resource overload.
918            Other  percentages  refer  to  'full'  meaning  a  loss of overall
919            throughput due to resource overload.
920            The values `cpusome', `memsome', `memfull', `iosome' and  `iofull'
921            show the pressure percentage during the entire interval.
922            The  values  `cs'  (cpu  some),  `ms'  (memory some), `mf' (memory
923            full), `is' (I/O some) and `if' (I/O full) each  show  three  per‐
924            centages  separated  by slashes: pressure percentage over the last
925            10, 60 and 300 seconds.
926
927       LVM/MDD/DSK
928            Logical volume/multiple device/disk utilization.
929            Per active unit one line is produced,  sorted  on  unit  activity.
930            Such line shows the name (e.g. VolGroup00-lvtmp for a logical vol‐
931            ume or sda for a hard disk), the busy percentage i.e. the  portion
932            of  time  that  the  unit was busy handling requests (`busy'), the
933            number of read requests issued (`read'), the number of  write  re‐
934            quests  issued  (`write'),  the  number of discard requests issued
935            (`discrd') if supported by kernel version, the number  of  KiBytes
936            per read (`KiB/r'), the number of KiBytes per write (`KiB/w'), the
937            number of KiBytes per discard (`KiB/d')  if  supported  by  kernel
938            version,  the  number  of  MiBytes per second throughput for reads
939            (`MBr/s'), the number of MiBytes per second throughput for  writes
940            (`MBw/s'),  the average queue depth (`avq') and the average number
941            of milliseconds needed by a request (`avio') for seek, latency and
942            data transfer.
943            If  the  screen-width does not allow all of these counters, only a
944            relevant subset is shown.
945
946            The number of lines showing the units can  be  limited  per  class
947            (LVM,  MDD  or  DSK)  with the 'l' key or statically (see separate
948            man-page of atoprc).  By specifying the value 0 for  a  particular
949            class, no lines will be shown any more for that class.
950
951       NFM  Network Filesystem (NFS) mount at the client side.
952            For each NFS-mounted filesystem, a line is shown that contains the
953            mounted server directory, the name of the server (`srv'), the  to‐
954            tal  number  of bytes physically read from the server (`read') and
955            the total  number  of  bytes  physically  written  to  the  server
956            (`write').   Data  transfer  is  subdivided in the number of bytes
957            read via normal read() system calls (`nread'), the number of bytes
958            written  via  normal  read() system calls (`nwrit'), the number of
959            bytes read via direct I/O (`dread'), the number of  bytes  written
960            via  direct  I/O  (`dwrit'),  the  number of bytes read via memory
961            mapped I/O pages (`mread'), and the number of  bytes  written  via
962            memory mapped I/O pages (`mwrit').
963
964       NFC  Network Filesystem (NFS) client side counters.
965            This  line  contains  the number of RPC calls issues by local pro‐
966            cesses (`rpc'), the number of read RPC calls  (`read`)  and  write
967            RPC  calls (`rpwrite') issued to the NFS server, the number of RPC
968            calls being retransmitted (`retxmit') and the number of authoriza‐
969            tion refreshes (`autref').
970
971       NFS  Network Filesystem (NFS) server side counters.
972            This  line  contains  the  number  of  RPC calls received from NFS
973            clients (`rpc'), the number of read RPC calls received  (`cread`),
974            the  number  of  write RPC calls received (`cwrit'), the number of
975            Megabytes/second returned to read requests by clients  (`MBcr/s`),
976            the number of Megabytes/second passed in write requests by clients
977            (`MBcw/s`),  the  number  of  network  requests  handled  via  TCP
978            (`nettcp'),  the  number of network requests handled via UDP (`ne‐
979            tudp'), the number of reply cache hits (`rchits'), the  number  of
980            reply  cache misses (`rcmiss') and the number of uncached requests
981            (`rcnoca').  Furthermore some error counters indicating the number
982            of  requests  with  a bad format (`badfmt') or a bad authorization
983            (`badaut'), and a counter indicating the  number  of  bad  clients
984            (`badcln').
985
986       NET  Network utilization (TCP/IP).
987            One  line  is  shown  for activity of the transport layer (TCP and
988            UDP), one line for the IP layer and one line per active interface.
989            For the transport layer, counters are shown concerning the  number
990            of  received  TCP  segments  including  those  received  in  error
991            (`tcpi'), the number of transmitted TCP segments  excluding  those
992            containing  only  retransmitted octets (`tcpo'), the number of UDP
993            datagrams received (`udpi'), the number of UDP datagrams transmit‐
994            ted (`udpo'), the number of active TCP opens (`tcpao'), the number
995            of passive TCP opens (`tcppo'), the number of TCP output  retrans‐
996            missions  (`tcprs'), the number of TCP input errors (`tcpie'), the
997            number of TCP output resets (`tcpor'), the number of UDP no  ports
998            (`udpnp'), and the number of UDP input errors (`udpie').
999            If  the  screen-width does not allow all of these counters, only a
1000            relevant subset is shown.
1001            These counters are related to IPv4 and IPv6 combined.
1002
1003            For the IP layer, counters are shown concerning the number  of  IP
1004            datagrams  received  from  interfaces, including those received in
1005            error (`ipi'), the number of IP datagrams that local  higher-layer
1006            protocols offered for transmission (`ipo'), the number of received
1007            IP datagrams which were forwarded to other  interfaces  (`ipfrw'),
1008            the  number  of IP datagrams which were delivered to local higher-
1009            layer protocols (`deliv'), the number of received  ICMP  datagrams
1010            (`icmpi'), and the number of transmitted ICMP datagrams (`icmpo').
1011            If  the  screen-width does not allow all of these counters, only a
1012            relevant subset is shown.
1013            These counters are related to IPv4 and IPv6 combined.
1014
1015            For every active network interface one line is  shown,  sorted  on
1016            the interface activity.  Such line shows the name of the interface
1017            and its busy percentage in the first column.  The busy  percentage
1018            for  half  duplex  is  determined by comparing the interface speed
1019            with the number of bits transmitted and received per  second;  for
1020            full  duplex  the  interface speed is compared with the highest of
1021            either the transmitted or the received bits.  When  the  interface
1022            speed  can  not  be  determined (e.g. for the loopback interface),
1023            `---' is shown instead of the percentage.
1024            Furthermore the number of received packets (`pcki'), the number of
1025            transmitted  packets  (`pcko'),  the  line  speed of the interface
1026            (`sp'), the effective amount of bits received per  second  (`si'),
1027            the  effective  amount  of bits transmitted per second (`so'), the
1028            number of collisions (`coll'), the number  of  received  multicast
1029            packets  (`mlti'),  the  number of errors while receiving a packet
1030            (`erri'),  the  number  of  errors  while  transmitting  a  packet
1031            (`erro'), the number of received packets dropped (`drpi'), and the
1032            number of transmitted packets dropped (`drpo').
1033            If the screen-width does not allow all of these counters,  only  a
1034            relevant subset is shown.
1035            The number of lines showing the network interfaces can be limited.
1036
1037       IFB  Infiniband utilization.
1038            For  every active Infiniband port one line is shown, sorted on ac‐
1039            tivity.  Such line shows the name of the port and  its  busy  per‐
1040            centage in the first column.  The busy percentage is determined by
1041            taking the highest of either the transmitted or the received  bits
1042            during the interval, multiplying that value by the number of lanes
1043            and comparing it against the maximum port speed.
1044            Furthermore the number of received packets divided by  the  number
1045            of  lanes  (`pcki'),  the number of transmitted packets divided by
1046            the number of lanes (`pcko'), the maximum line speed  (`sp'),  the
1047            effective amount of bits received per second (`si'), the effective
1048            amount of bits transmitted per second (`so'), and  the  number  of
1049            lanes (`lanes').
1050            If  the  screen-width does not allow all of these counters, only a
1051            relevant subset is shown.
1052            The number of lines showing the Infiniband ports can be limited.
1053

OUTPUT DESCRIPTION - PROCESS LEVEL

1055       Following the system level information, the processes  are  shown  from
1056       which  the  resource  utilization has changed during the last interval.
1057       These processes might have used cpu time or issued disk or network  re‐
1058       quests.  However  a  process is also shown if part of it has been paged
1059       out due to lack of memory  (while  the  process  itself  was  in  sleep
1060       state).
1061
1062       Per  process the following fields may be shown (in alphabetical order),
1063       depending on the current output mode as described in the section INTER‐
1064       ACTIVE COMMANDS and depending on the current width of your window:
1065
1066       AVGRSZ   The average size of one read-action on disk.
1067
1068       AVGWSZ   The average size of one write-action on disk.
1069
1070       BANDWI   Total  bandwidth  for received TCP and UDP packets consumed by
1071                this process (bits-per-second).  This value  can  be  compared
1072                with the value `si' on interface level (used bandwidth per in‐
1073                terface).
1074                This information will only be shown  when  the  kernel  module
1075                `netatop' is loaded.
1076
1077       BANDWO   Total  bandwidth for sent TCP and UDP packets consumed by this
1078                process (bits-per-second).  This value can  be  compared  with
1079                the  value  `so' on interface level (used bandwidth per inter‐
1080                face).
1081                This information will only be shown  when  the  kernel  module
1082                `netatop' is loaded.
1083
1084       CID      Container  ID  (Docker) of 12 hexadecimal digits, referring to
1085                the container in which the process/thread is  running.   If  a
1086                process  has  been started and finished during the last inter‐
1087                val, a `?' is shown because the container ID is  not  part  of
1088                the standard process accounting record.
1089
1090       CMD      The  name  of  the  process.   This  name can be surrounded by
1091                "less/greater than" signs  (`<name>')  which  means  that  the
1092                process has finished during the last interval.
1093                Behind  the abbreviation `CMD' in the header line, the current
1094                page  number  and  the  total   number   of   pages   of   the
1095                process/thread list are shown.
1096
1097       COMMAND-LINE
1098                The full command line of the process (including arguments). If
1099                the length of the command  line  exceeds  the  length  of  the
1100                screen line, the arrow keys -> and <- can be used for horizon‐
1101                tal scroll.
1102                Behind the verb `COMMAND-LINE' in the header line, the current
1103                page   number   and   the   total   number  of  pages  of  the
1104                process/thread list are shown.
1105
1106       CPU      The occupation percentage  of  this  process  related  to  the
1107                available capacity for this resource on system level.
1108
1109       CPUNR    The  identification of the CPU the (main) thread is running on
1110                or has recently been running on.
1111
1112       CTID     Container ID (OpenVZ).  If a process has been started and fin‐
1113                ished  during  the  last  interval, a `?' is shown because the
1114                container ID is not part of the  standard  process  accounting
1115                record.
1116
1117       DSK      The occupation percentage of this process related to the total
1118                load that is produced by all processes (i.e.  total  disk  ac‐
1119                cesses by all processes during the last interval).
1120                This  information  is shown when per process "storage account‐
1121                ing" is active in the kernel.
1122
1123       EGID     Effective group-id under which this process executes.
1124
1125       ENDATE   Date that the process has been finished.  If  the  process  is
1126                still running, this field shows `active'.
1127
1128       ENTIME   Time  that  the  process  has been finished. If the process is
1129                still running, this field shows `active'.
1130
1131       ENVID    Virtual environment identified (OpenVZ only).
1132
1133       EUID     Effective user-id under which this process executes.
1134
1135       EXC      The exit code of a terminated process (second position of col‐
1136                umn  `ST' is E) or the fatal signal number (second position of
1137                column `ST' is S or C).
1138
1139       FSGID    Filesystem group-id under which this process executes.
1140
1141       FSUID    Filesystem user-id under which this process executes.
1142
1143       GPU      When the atopgpud daemon does not run  with  root  privileges,
1144                the GPU percentage reflects the GPU memory occupation percent‐
1145                age (memory of all GPUs is 100%).
1146                When the atopgpud daemon runs with root  privileges,  the  GPU
1147                percentage reflects the GPU busy percentage.
1148
1149       GPUBUSY  Busy percentage on all GPUs (one GPU is 100%).
1150                When  the  atopgpud  daemon does not run with root privileges,
1151                this value is not available.
1152
1153       GPUNUMS  Comma-separated list of GPUs used by the  process  during  the
1154                interval.  When  the comma-separated list exceeds the width of
1155                the column, a hexadecimal value is shown.
1156
1157       LOCKSZ   The virtual amount of memory being locked (i.e. non-swappable)
1158                by this process (or user).
1159
1160       MAJFLT   The  number  of  page  faults issued by this process that have
1161                been solved by creating/loading the requested memory page.
1162
1163       MEM      The occupation percentage  of  this  process  related  to  the
1164                available capacity for this resource on system level.
1165
1166       MEMAVG   Average  memory  occupation  during  the  interval on all used
1167                GPUs.
1168
1169       MEMBUSY  Busy percentage of memory on all GPUs (one GPU is 100%),  i.e.
1170                the time needed for read and write accesses on memory.
1171                When  the  atopgpud  daemon does not run with root privileges,
1172                this value is not available.
1173
1174       MEMNOW   Memory occupation at the moment of  the  sample  on  all  used
1175                GPUs.
1176
1177       MINFLT   The  number  of  page  faults issued by this process that have
1178                been solved by reclaiming the requested memory page  from  the
1179                free list of pages.
1180
1181       NET      The occupation percentage of this process related to the total
1182                load that is produced by all processes (i.e. consumed  network
1183                bandwidth of all processes during the last interval).
1184                This  information  will  only be shown when kernel module `ne‐
1185                tatop' is loaded.
1186
1187       NICE     The more or less static  priority  that  can  be  given  to  a
1188                process on a scale from -20 (high priority) to +19 (low prior‐
1189                ity).
1190
1191       NPROCS   The number of active and terminated processes accumulated  for
1192                this user or program.
1193
1194       PID      Process-id.  If a process has been started and finished during
1195                the last interval, a `?' is shown because  the  process-id  is
1196                not part of the standard process accounting record.
1197
1198       POLI     The  policies  'norm'  (normal,  which is SCHED_OTHER), 'btch'
1199                (batch) and 'idle' refer to timesharing processes.  The  poli‐
1200                cies  'fifo'  (SCHED_FIFO)  and  'rr'  (round  robin, which is
1201                SCHED_RR) refer to realtime processes.
1202
1203       PPID     Parent process-id.  If a process has been started and finished
1204                during  the last interval, value 0 is shown because the parent
1205                process-id is not part  of  the  standard  process  accounting
1206                record.
1207
1208       PRI      The  process' priority ranges from 0 (highest priority) to 139
1209                (lowest priority). Priority 0 to 99 are used for realtime pro‐
1210                cesses (fixed priority independent of their behavior) and pri‐
1211                ority 100 to 139 for timesharing processes (variable  priority
1212                depending on their recent CPU consumption and the nice value).
1213
1214       PSIZE    The proportional memory size of this process (or user).
1215                Every  process  shares  resident  memory with other processes.
1216                E.g. when a particular program is started several  times,  the
1217                code pages (text) are only loaded once in memory and shared by
1218                all incarnations. Also the code of shared libraries is  shared
1219                by  all processes using that shared library, as well as shared
1220                memory and memory-mapped files.  For the PSIZE calculation  of
1221                a  process,  the  resident  memory of a process that is shared
1222                with other processes is divided  by  the  number  of  sharers.
1223                This means, that every process is accounted for a proportional
1224                part of that memory. Accumulating the PSIZE values of all pro‐
1225                cesses  in the system gives a reliable impression of the total
1226                resident memory consumed by all processes.
1227                Since gathering of all values that are needed to calculate the
1228                PSIZE  is  a  very  time-consuming  task, the 'R' key (or '-R'
1229                flag) should be active. Gathering these values  also  requires
1230                superuser privileges (otherwise '?K' is shown in the output).
1231                If  a  process has finished during the last interval, no value
1232                is shown since the proportional memory size is not part of the
1233                standard process accounting record.
1234
1235       RDDSK    When the kernel maintains standard io statistics (>= 2.6.20):
1236                The  read  data transfer issued physically on disk (so reading
1237                from the disk cache is not accounted for).
1238                Unfortunately, the kernel aggregates the  data  tranfer  of  a
1239                process to the data transfer of its parent process when termi‐
1240                nating, so you might see transfers for (parent) processes like
1241                cron, bash or init, that are not really issued by them.
1242
1243       RDELAY   Runqueue delay, i.e. time spent waiting on a runqueue.
1244
1245       RGID     The real group-id under which the process executes.
1246
1247       RGROW    The  amount of resident memory that the process has grown dur‐
1248                ing the last interval. A resident  growth  can  be  caused  by
1249                touching memory pages which were not physically created/loaded
1250                before (load-on-demand).  Note that a resident growth can also
1251                be  negative e.g. when part of the process is paged out due to
1252                lack of memory or when the process frees dynamically allocated
1253                memory.  For a process which started during the last interval,
1254                the resident growth reflects the total resident  size  of  the
1255                process at that moment.
1256                If  a  process has finished during the last interval, no value
1257                is shown since resident memory occupation is not part  of  the
1258                standard process accounting record.
1259
1260       RNET     The  number  of TCP- and UDP packets received by this process.
1261                This information will only be shown when  kernel  module  `ne‐
1262                tatop' is installed.
1263                If  a  process has finished during the last interval, no value
1264                is shown since network counters are not part of  the  standard
1265                process accounting record.
1266
1267       RSIZE    The  total  resident memory usage consumed by this process (or
1268                user).  Notice that the RSIZE of a process includes all  resi‐
1269                dent memory used by that process, even if certain memory parts
1270                are shared with other processes (see also the  explanation  of
1271                PSIZE).
1272                If  a  process has finished during the last interval, no value
1273                is shown since resident memory occupation is not part  of  the
1274                standard process accounting record.
1275
1276       RTPR     Realtime  priority according the POSIX standard.  Value can be
1277                0 for a timesharing process (policy 'norm', 'btch' or  'idle')
1278                or  ranges  from  1  (lowest) till 99 (highest) for a realtime
1279                process (policy 'rr' or 'fifo').
1280
1281       RUID     The real user-id under which the process executes.
1282
1283       S        The current state of the (main) thread: `R' for running  (cur‐
1284                rently processing or in the runqueue), `S' for sleeping inter‐
1285                ruptible (wait for an event to occur), `D' for  sleeping  non-
1286                interruptible, `Z' for zombie (waiting to be synchronized with
1287                its parent process), `T' for stopped  (suspended  or  traced),
1288                `W' for swapping, and `E' (exit) for processes which have fin‐
1289                ished during the last interval.
1290
1291       SGID     The saved group-id of the process.
1292
1293       SNET     The number of TCP and UDP packets transmitted by this process.
1294                This  information  will  only  be shown when the kernel module
1295                `netatop' is loaded.
1296
1297       ST       The status of a process.
1298                The first position indicates if the process has  been  started
1299                during the last interval (the value N means 'new process').
1300
1301                The second position indicates if the process has been finished
1302                during the last interval.
1303                The value E means 'exit' on the process' own  initiative;  the
1304                exit code is displayed in the column `EXC'.
1305                The  value S means that the process has been terminated unvol‐
1306                untarily by a signal; the signal number is displayed in the in
1307                the column `EXC'.
1308                The  value C means that the process has been terminated unvol‐
1309                untarily by a signal, producing a core dump in its current di‐
1310                rectory; the signal number is displayed in the column `EXC'.
1311
1312       STDATE   The start date of the process.
1313
1314       STTIME   The start time of the process.
1315
1316       SUID     The saved user-id of the process.
1317
1318       SWAPSZ   The swap space consumed by this process (or user).
1319
1320       SYSCPU   CPU  time  consumption  of this process in system mode (kernel
1321                mode), usually due to system call handling.
1322
1323       TCPRASZ  The average size of a received TCP buffer in bytes.  This  in‐
1324                formation  will only be shown when the kernel module `netatop'
1325                is loaded.
1326
1327       TCPRCV   The number of TCP packets received for this process.  This in‐
1328                formation  will only be shown when the kernel module `netatop'
1329                is loaded.
1330
1331       TCPSASZ  The average size of a transmitted TCP buffer in  bytes.   This
1332                information  will  only  be  shown when the kernel module `ne‐
1333                tatop' is loaded.
1334
1335       TCPSND   The number of TCP packets transmitted for this process.   This
1336                information  will  only  be  shown when the kernel module `ne‐
1337                tatop' is loaded.
1338
1339       THR      Total number of threads  within  this  process.   All  related
1340                threads  are  contained in a thread group, represented by atop
1341                as one line or as a separate line when  the  'y'  key  (or  -y
1342                flag) is active.
1343
1344                On  Linux 2.4 systems it is hardly possible to determine which
1345                threads (i.e. processes) are related to the same thread group.
1346                Every thread is represented by atop as a separate line.
1347
1348       TID      Thread-id.  All threads within a process run with the same PID
1349                but with a different TID. This value is shown  for  individual
1350                threads in multi-threaded processes (when using the key 'y').
1351
1352       TRUN     Number  of  threads  within this process that are in the state
1353                'running' (R).
1354
1355       TSLPI    Number of threads within this process that are  in  the  state
1356                'interruptible sleeping' (S).
1357
1358       TSLPU    Number  of  threads  within this process that are in the state
1359                'uninterruptible sleeping' (D).
1360
1361       UDPRASZ  The average size of a received UDP packet in bytes.  This  in‐
1362                formation  will only be shown when the kernel module `netatop'
1363                is loaded.
1364
1365       UDPRCV   The number of UDP packets received by this process.  This  in‐
1366                formation  will only be shown when the kernel module `netatop'
1367                is loaded.
1368
1369       UDPSASZ  The average size of a transmitted UDP packets in bytes.   This
1370                information  will  only  be  shown when the kernel module `ne‐
1371                tatop' is loaded.
1372
1373       UDPSND   The number of UDP packets transmitted by this  process.   This
1374                information  will  only  be  shown when the kernel module `ne‐
1375                tatop' is loaded.
1376
1377       USRCPU   CPU time consumption of this process in user mode, due to pro‐
1378                cessing the own program text.
1379
1380       VDATA    The  virtual  memory  size  of  the  private data used by this
1381                process (including heap and shared library data).
1382
1383       VGROW    The amount of virtual memory that the process has grown during
1384                the  last interval. A virtual growth can be caused by e.g. is‐
1385                sueing a malloc() or attaching a shared memory  segment.  Note
1386                that  a virtual growth can also be negative by e.g. issueing a
1387                free() or detaching a shared memory segment.   For  a  process
1388                which started during the last interval, the virtual growth re‐
1389                flects the total virtual size of the process at that moment.
1390                If a process has finished during the last interval,  no  value
1391                is  shown  since  virtual memory occupation is not part of the
1392                standard process accounting record.
1393
1394       VPID     Virtual process-id (within an OpenVZ container).  If a process
1395                has  been started and finished during the last interval, a `?'
1396                is shown because the virtual process-id is  not  part  of  the
1397                standard process accounting record.
1398
1399       VSIZE    The  total  virtual  memory usage consumed by this process (or
1400                user).
1401                If a process has finished during the last interval,  no  value
1402                is  shown  since  virtual memory occupation is not part of the
1403                standard process accounting record.
1404
1405       VSLIBS   The virtual memory size of the (shared) text of all shared li‐
1406                braries used by this process.
1407
1408       VSTACK   The  virtual  memory  size of the (private) stack used by this
1409                process
1410
1411       VSTEXT   The virtual memory size of the (shared) text of the executable
1412                program.
1413
1414       WCHAN    Wait  channel  of  thread in sleep state, i.e. the name of the
1415                kernel function in which the thread has been put asleep.
1416                Since determining the name string of the kernel function is  a
1417                relatively  time-consuming  task,  the  'W' key (or '-W' flag)
1418                should be active.
1419
1420       WRDSK    When the kernel maintains standard io statistics (>= 2.6.20):
1421                The write data transfer issued physically on disk (so  writing
1422                to  the  disk  cache  is  not accounted for).  This counter is
1423                maintained for the application process that writes its data to
1424                the  cache  (assuming that this data is physically transferred
1425                to disk later on). Notice that disk I/O needed for swapping is
1426                not taken into account.
1427                Unfortunately,  the  kernel  aggregates  the data tranfer of a
1428                process to the data transfer of its parent process when termi‐
1429                nating, so you might see transfers for (parent) processes like
1430                cron, bash or init, that are not really issued by them.
1431
1432       WCANCL   When the kernel maintains standard io statistics (>= 2.6.20):
1433                The write data transfer previously accounted for this  process
1434                or  another  process  that has been cancelled.  Suppose that a
1435                process writes new data to a file and  that  data  is  removed
1436                again  before  the  cache  buffers  have been flushed to disk.
1437                Then the original process shows the  written  data  as  WRDSK,
1438                while  the  process  that removes/truncates the file shows the
1439                unflushed removed data as WCANCL.
1440

PARSEABLE OUTPUT

1442       With the flag -P followed by a list of one or more labels  (comma-sepa‐
1443       rated),  parseable output is produced for each sample.  The labels that
1444       can be specified for system-level statistics correspond to  the  labels
1445       (first  verb of each line) that can be found in the interactive output:
1446       "CPU", "cpu", "CPL", "GPU", "MEM", "SWP", "PAG", "PSI",  "LVM",  "MDD",
1447       "DSK", "NFM", "NFC", "NFS", "NET", "IFB", "NUM" and "NUC".
1448       For  process-level statistics special labels are available: "PRG" (gen‐
1449       eral), "PRC" (cpu), "PRE" (GPU), "PRM" (memory), "PRD" (disk,  only  if
1450       "storage  accounting" is active) and "PRN" (network, only if the kernel
1451       module 'netatop' has been installed).
1452       With the label "ALL", all  system  and  process  level  statistics  are
1453       shown.
1454
1455       The command and command line in the parseable output might contain spa‐
1456       ces and are therefore by default surrounded  by  parenthesis.  However,
1457       since  a space is often used as separator between the fields by parsing
1458       tools, with the additional flag -Z it is possible to exchange the  spa‐
1459       ces in the command (line) by underscores and omit the parenthesis.
1460
1461       For  every interval all requested lines are shown whereafter atop shows
1462       a line just containing the label "SEP" as a separator before the  lines
1463       for the next sample are generated.
1464       When  a  sample  contains the values since boot, atop shows a line just
1465       containing the label "RESET" before the lines for this sample are  gen‐
1466       erated.
1467
1468       The  first  part  of  each  output-line  consists  of the following six
1469       fields: label (the name of the label), host (the name of this machine),
1470       epoch  (the time of this interval as number of seconds since 1-1-1970),
1471       date (date of this interval in format YYYY/MM/DD), time (time  of  this
1472       interval  in  format HH:MM:SS), and interval (number of seconds elapsed
1473       for this interval).
1474
1475       The subsequent fields of each output-line depend on the label:
1476
1477       CPU      Subsequent fields: total number of clock-ticks per second  for
1478                this  machine,  number of processors, consumption for all CPUs
1479                in system mode (clock-ticks), consumption for all CPUs in user
1480                mode  (clock-ticks), consumption for all CPUs in user mode for
1481                niced processes (clock-ticks), consumption  for  all  CPUs  in
1482                idle mode (clock-ticks), consumption for all CPUs in wait mode
1483                (clock-ticks), consumption for all CPUs in  irq  mode  (clock-
1484                ticks),  consumption  for  all  CPUs  in  softirq mode (clock-
1485                ticks), consumption for all CPUs in steal mode  (clock-ticks),
1486                consumption  for all CPUs in guest mode (clock-ticks) overlap‐
1487                ping user mode, frequency of all CPUs, frequency percentage of
1488                all CPUs, instructions executed by all CPUs and cycles for all
1489                CPUs.
1490
1491       cpu      Subsequent fields: total number of clock-ticks per second  for
1492                this  machine,  processor-number,  consumption for this CPU in
1493                system mode (clock-ticks), consumption for this  CPU  in  user
1494                mode  (clock-ticks), consumption for this CPU in user mode for
1495                niced processes (clock-ticks), consumption  for  this  CPU  in
1496                idle mode (clock-ticks), consumption for this CPU in wait mode
1497                (clock-ticks), consumption for this CPU in  irq  mode  (clock-
1498                ticks),  consumption  for  this  CPU  in  softirq mode (clock-
1499                ticks), consumption for this CPU in steal mode  (clock-ticks),
1500                consumption  for this CPU in guest mode (clock-ticks) overlap‐
1501                ping user mode, frequency of this CPU, frequency percentage of
1502                this  CPU,  instructions  executed  by this CPU and cycles for
1503                this CPU.
1504
1505       CPL      Subsequent fields: number of processors, load average for last
1506                minute,  load  average for last five minutes, load average for
1507                last fifteen minutes, number of context-switches,  and  number
1508                of device interrupts.
1509
1510       GPU      Subsequent  fields:  GPU  number,  bus-id  string, type of GPU
1511                string, GPU busy percentage during  last  second  (-1  if  not
1512                available),  memory  busy percentage during last second (-1 if
1513                not available), total memory size (KiB), used memory (KiB)  at
1514                this  moment, number of samples taken during interval, cumula‐
1515                tive GPU busy percentage during the interval (to be divided by
1516                the  number  of samples for the average busy percentage, -1 if
1517                not available), cumulative memory busy percentage  during  the
1518                interval (to be divided by the number of samples for the aver‐
1519                age busy percentage, -1 if not available), and cumulative mem‐
1520                ory  occupation during the interval (to be divided by the num‐
1521                ber of samples for the average occupation).
1522
1523       MEM      Subsequent fields: page size for this machine (in bytes), size
1524                of  physical memory (pages), size of free memory (pages), size
1525                of page cache (pages), size of buffer cache (pages),  size  of
1526                slab  (pages),  dirty pages in cache (pages), reclaimable part
1527                of slab (pages), total size of vmware's balloon pages (pages),
1528                total  size  of shared memory (pages), size of resident shared
1529                memory (pages), size of swapped shared  memory  (pages),  huge
1530                page  size  (in bytes), total size of huge pages (huge pages),
1531                size of free huge pages (huge pages), size of ARC  (cache)  of
1532                ZFSonlinux (pages), size of sharing pages for KSM (pages), and
1533                size of shared pages for KSM (pages).
1534
1535       SWP      Subsequent fields: page size for this machine (in bytes), size
1536                of swap (pages), size of free swap (pages), size of swap cache
1537                (pages), size of committed space (pages), limit for  committed
1538                space  (pages),  size  of the swap cache (pages), size of com‐
1539                pressed pages stored in zswap (pages), and total size of  com‐
1540                pressed pool in zswap (pages).
1541
1542       PAG      Subsequent fields: page size for this machine (in bytes), num‐
1543                ber of page scans, number of allocstalls, 0 (future use), num‐
1544                ber  of  swapins,  number  of swapouts, number of oomkills (-1
1545                when counter not present), number of  process  stalls  to  run
1546                memory  compaction,  number  of pages successfully migrated in
1547                total, and number of NUMA pages migrated.
1548
1549       PSI      Subsequent fields: PSI statistics present on this system (n or
1550                y),  CPU some avg10, CPU some avg60, CPU some avg300, CPU some
1551                accumulated microseconds during interval, memory  some  avg10,
1552                memory some avg60, memory some avg300, memory some accumulated
1553                microseconds during interval, memory full avg10,  memory  full
1554                avg60,  memory  full avg300, memory full accumulated microsec‐
1555                onds during interval, I/O some avg10, I/O some avg60, I/O some
1556                avg300, I/O some accumulated microseconds during interval, I/O
1557                full avg10, I/O full avg60, I/O full avg300, and I/O full  ac‐
1558                cumulated microseconds during interval.
1559
1560       LVM/MDD/DSK
1561                For every logical volume/multiple device/hard disk one line is
1562                shown.
1563                Subsequent fields: name, number of milliseconds spent for I/O,
1564                number  of  reads  issued,  number  of sectors transferred for
1565                reads, number of writes issued, number of sectors  transferred
1566                for  write,  number  of discards issued (-1 if not supported),
1567                and number of sectors transferred for discards.
1568
1569       NFM      Subsequent fields: mounted NFS  filesystem,  total  number  of
1570                bytes  read,  total  number  of bytes written, number of bytes
1571                read by normal system calls, number of bytes written by normal
1572                system  calls,  number  of bytes read by direct I/O, number of
1573                bytes written by direct I/O, number of pages read  by  memory-
1574                mapped I/O, and number of pages written by memory-mapped I/O.
1575
1576       NFC      Subsequent  fields:  number  of  transmitted  RPCs,  number of
1577                transmitted read RPCs, number of transmitted write RPCs,  num‐
1578                ber  of  RPC  retransmissions, and number of authorization re‐
1579                freshes.
1580
1581       NFS      Subsequent fields: number of handled RPCs, number of  received
1582                read RPCs, number of received write RPCs, number of bytes read
1583                by clients, number of bytes written by clients, number of RPCs
1584                with bad format, number of RPCs with bad authorization, number
1585                of RPCs from bad client, total number of handled  network  re‐
1586                quests,  number of handled network requests via TCP, number of
1587                handled network requests via UDP, number of handled  TCP  con‐
1588                nections,  number  of hits on reply cache, number of misses on
1589                reply cache, and number of uncached requests.
1590
1591       NET      First, one line is produced for the upper layers of the TCP/IP
1592                stack.
1593                Subsequent  fields:  the  verb  "upper", number of packets re‐
1594                ceived by TCP, number of packets transmitted by TCP, number of
1595                packets received by UDP, number of packets transmitted by UDP,
1596                number of packets received by IP, number of packets  transmit‐
1597                ted by IP, number of packets delivered to higher layers by IP,
1598                number of packets forwarded by  IP,  number  of  input  errors
1599                (UDP),  number  of noport errors (UDP), number of active opens
1600                (TCP), number of passive opens (TCP), number of passive  opens
1601                (TCP), number of established connections at this moment (TCP),
1602                number of retransmitted segments (TCP), number of input errors
1603                (TCP), and number of output resets (TCP).
1604
1605                Next, one line is shown for every interface.
1606                Subsequent  fields:  name  of the interface, number of packets
1607                received by the interface, number of bytes received by the in‐
1608                terface,  number of packets transmitted by the interface, num‐
1609                ber of bytes transmitted by the  interface,  interface  speed,
1610                and duplex mode (0=half, 1=full).
1611
1612       IFB      Subsequent fields: name of the InfiniBand interface, port num‐
1613                ber, number of lanes, maximum rate (Mbps), number of bytes re‐
1614                ceived,  number  of  bytes  transmitted, number of packets re‐
1615                ceived, and number of packets transmitted.
1616
1617       NUM      Subsequent fields: NUMA node number, page size  for  this  ma‐
1618                chine  (in  bytes), the fragmentation percentage of this node,
1619                size of physical memory (pages), size of free memory  (pages),
1620                recently  (active)  used  memory (pages), less recently (inac‐
1621                tive) used memory (pages), size of cached file  data  (pages),
1622                dirty  pages in cache (pages), slab memory being used for ker‐
1623                nel mallocs (pages), slab memory that is reclaimable  (pages),
1624                shared  memory  including  tmpfs (pages), and total huge pages
1625                (pages).
1626
1627       NUC      Subsequent fields: NUMA node number, number of processors  for
1628                this  node,  consumption  for node CPUs in system mode (clock-
1629                ticks), consumption for node CPUs in user mode  (clock-ticks),
1630                consumption  for  node  CPUs  in user mode for niced processes
1631                (clock-ticks), consumption for node CPUs in idle mode  (clock-
1632                ticks),  consumption for node CPUs in wait mode (clock-ticks),
1633                consumption for node CPUs in irq mode (clock-ticks),  consump‐
1634                tion  for node CPUs in softirq mode (clock-ticks), consumption
1635                for node CPUs in steal mode (clock-ticks), and consumption for
1636                node CPUs in guest mode (clock-ticks) overlapping user mode.
1637
1638       PRG      For every process one line is shown.
1639                Subsequent  fields:  PID  (unique  ID  of task), name (between
1640                parenthesis or underscores for spaces), state, real uid,  real
1641                gid,  TGID (group number of related tasks/threads), total num‐
1642                ber of threads, exit code (in case  of  fatal  signal:  signal
1643                number  + 256), start time (epoch), full command line (between
1644                parenthesis  or  underscores  for  spaces),  PPID,  number  of
1645                threads  in  state  'running'  (R), number of threads in state
1646                'interruptible sleeping' (S), number of threads in state  'un‐
1647                interruptible  sleeping'  (D),  effective  uid, effective gid,
1648                saved uid, saved gid, filesystem uid, filesystem gid,  elapsed
1649                time  (hertz),  is_process  (y/n), OpenVZ  virtual pid (VPID),
1650                OpenVZ container id (CTID), Docker container id (CID), and in‐
1651                dication  if  the  task  is newly started during this interval
1652                ('N').
1653
1654       PRC      For every process one line is shown.
1655                Subsequent fields: PID, name (between  parenthesis  or  under‐
1656                scores  for  spaces),  state,  total number of clock-ticks per
1657                second for this machine, CPU-consumption in user mode  (clock‐
1658                ticks),  CPU-consumption  in  system  mode  (clockticks), nice
1659                value, priority, realtime priority, scheduling policy, current
1660                CPU,   sleep   average,   TGID   (group   number   of  related
1661                tasks/threads), is_process (y/n), runqueue delay  in  nanosec‐
1662                onds  for this thread or for all threads (in case of process),
1663                and wait channel of this thread (between parenthesis or under‐
1664                scores for spaces).
1665
1666       PRE      For every process one line is shown.
1667                Subsequent  fields:  PID,  name (between parenthesis or under‐
1668                scores for spaces), process state, GPU state (A for active,  E
1669                for  exited,  N  for no GPU user), number of GPUs used by this
1670                process, bitlist reflecting used  GPUs,  GPU  busy  percentage
1671                during  interval, memory busy percentage during interval, mem‐
1672                ory occupation (KiB) at this moment cumulative memory  occupa‐
1673                tion (KiB) during interval, and number of samples taken during
1674                interval.
1675
1676       PRM      For every process one line is shown.
1677                Subsequent fields: PID, name (between  parenthesis  or  under‐
1678                scores  for  spaces),  state,  page  size for this machine (in
1679                bytes), virtual memory size  (Kbytes),  resident  memory  size
1680                (Kbytes),  shared  text  memory  size (Kbytes), virtual memory
1681                growth (Kbytes), resident memory growth  (Kbytes),  number  of
1682                minor  page  faults,  number of major page faults, virtual li‐
1683                brary exec size (Kbytes), virtual data size (Kbytes),  virtual
1684                stack  size  (Kbytes),  swap  space used (Kbytes), TGID (group
1685                number of related tasks/threads),  is_process  (y/n),  propor‐
1686                tional  set  size  (Kbytes)  if in 'R' option is specified and
1687                virtually locked memory space (Kbytes).
1688
1689       PRD      For every process one line is shown.
1690                Subsequent fields: PID, name (between  parenthesis  or  under‐
1691                scores  for  spaces),  state, obsoleted kernel patch installed
1692                ('n'), standard io statistics used ('y'  or  'n'),  number  of
1693                reads  on  disk,  cumulative number of sectors read, number of
1694                writes on disk, cumulative number  of  sectors  written,  can‐
1695                celled  number  of  written sectors, TGID (group number of re‐
1696                lated tasks/threads), obsoleted value  ('n'),  and  is_process
1697                (y/n).
1698                If  the  standard I/O statistics (>= 2.6.20) are not used, the
1699                disk I/O counters per process are not relevant.  The  counters
1700                'number  of  reads on disk' and 'number of writes on disk' are
1701                obsoleted anyhow.
1702
1703       PRN      For every process one line is shown.
1704                Subsequent fields: PID, name (between  parenthesis  or  under‐
1705                scores for spaces), state, kernel module 'netatop' loaded ('y'
1706                or 'n'), number of TCP-packets transmitted, cumulative size of
1707                TCP-packets transmitted, number of TCP-packets received, cumu‐
1708                lative size of TCP-packets  received,  number  of  UDP-packets
1709                transmitted,  cumulative size of UDP-packets transmitted, num‐
1710                ber of UDP-packets received, cumulative  size  of  UDP-packets
1711                transmitted,  number of raw packets transmitted (obsolete, al‐
1712                ways 0), number of raw packets received (obsolete, always  0),
1713                TGID  (group  number  of related tasks/threads) and is_process
1714                (y/n).
1715                If the kernel module is not active, the network  I/O  counters
1716                per process are not relevant.
1717

SIGNALS

1719       By sending the SIGUSR1 signal to atop a new sample will be forced, even
1720       if the current timer interval has not exceeded  yet.  The  behavior  is
1721       similar to pressing the `t` key in an interactive session.
1722
1723       By sending the SIGUSR2 signal to atop a final sample will be forced af‐
1724       ter which atop will terminate.
1725

EXAMPLES

1727       To monitor the current system load interactively with an interval of  5
1728       seconds:
1729
1730         atop 5
1731
1732       To monitor the system load and write it to a file (in plain ASCII) with
1733       an interval of one minute during half an  hour  with  active  processes
1734       sorted on memory consumption:
1735
1736         atop -M 60 30 > /log/atop.mem
1737
1738       Store  information about the system and process activity in binary com‐
1739       pressed form to a file with an interval of ten minutes during an hour:
1740
1741         atop -w /tmp/atop.raw 600 6
1742
1743       View the contents of this file interactively:
1744
1745         atop -r /tmp/atop.raw
1746
1747       View the processor and disk utilization of this file in parseable  for‐
1748       mat:
1749
1750         atop -PCPU,DSK -r /tmp/atop.raw
1751
1752       View the contents of today's standard logfile interactively:
1753
1754         atop -r
1755
1756       View  the  contents of the standard logfile of the day before yesterday
1757       interactively:
1758
1759         atop -r yy
1760
1761       View the contents of the standard logfile of 2014, June 7 from 02:00 PM
1762       onwards interactively:
1763
1764         atop -r 20140607 -b 1400
1765
1766       Concatenate  all  raw  log  files of January 2020 and generate parsable
1767       output about the CPU utilization:
1768
1769         atopcat /var/log/atop/atop_202001?? | atop -r - -PCPU
1770

FILES

1772       /var/run/pacct_shadow.d/
1773            Directory containing the process accounting shadow files that  are
1774            used by atop when the atopacctd daemon is active.
1775
1776       /var/cache/atop.d/atop.acct
1777            File  in  which the kernel writes the accounting records when atop
1778            itself has activated the process accounting mechanism.
1779
1780       /etc/atoprc
1781            Configuration file containing system-wide default values.  See re‐
1782            lated man-page.
1783
1784       ~/.atoprc
1785            Configuration  file  containing  personal default values.  See re‐
1786            lated man-page.
1787
1788       /etc/default/atop
1789            Configuration file to overrule the settings of atop that  runs  in
1790            the  background to create the daily logfile.  This file is created
1791            when atop is installed.  The default settings are:
1792
1793       LOGOPTS=""
1794               LOGINTERVAL=600
1795               LOGGENERATIONS=28
1796
1797       /var/log/atop/atop_YYYYMMDD
1798            Raw file, where YYYYMMDD are digits representing the current date.
1799            This  name  is  used  by atop running in the background as default
1800            name for the output file, and by atop as default name for the  in‐
1801            put file when using the -r flag.
1802            All  binary  system  and  process level data in this file has been
1803            stored in compressed format.
1804
1805       /var/run/netatop.log
1806            File that contains the netpertask structs containing  the  network
1807            counters of exited processes. These structs are written by the ne‐
1808            tatopd daemon and read by atop after reading the standard  process
1809            accounting records.
1810

SEE ALSO

1812       atopsar(1),  atopconvert(1),  atopcat(1),  atoprc(5), atopacctd(8), ne‐
1813       tatop(4), netatopd(8), atopgpud(8), logrotate(8)
1814       https://www.atoptool.nl
1815

AUTHOR

1817       Gerlof Langeveld (gerlof.langeveld@atoptool.nl)
1818       JC van Winkel
1819
1820
1821
1822Linux                            December 2021                         ATOP(1)
Impressum