bcc-runqlat(8)

1runqlat(8)                  System Manager's Manual                 runqlat(8)
2
3
4

NAME

6       runqlat - Run queue (scheduler) latency as a histogram.
7

SYNOPSIS

9       runqlat [-h] [-T] [-m] [-P] [--pidnss] [-L] [-p PID] [interval] [count]
10

DESCRIPTION

12       This measures the time a task spends waiting on a run queue (or equiva‐
13       lent scheduler data structure) for a turn on-CPU, and shows  this  time
14       as  a histogram. This time should be small, but a task may need to wait
15       its turn due to CPU load. The higher the CPU load, the  longer  a  task
16       will generally need to wait its turn.
17
18       This tool measures two types of run queue latency:
19
20       1.  The  time  from a task being enqueued on a run queue to its context
21       switch and execution. This traces ttwu_do_wakeup(),  wake_up_new_task()
22       ->  finish_task_switch()  with either raw tracepoints (if supported) or
23       kprobes and instruments the run queue latency after a voluntary context
24       switch.
25
26       2. The time from when a task was involuntary context switched and still
27       in the runnable state, to when it next executed. This  is  instrumented
28       from finish_task_switch() alone.
29
30       This  tool uses in-kernel eBPF maps for storing timestamps and the his‐
31       togram, for efficiency. Despite this, the overhead  of  this  tool  may
32       become significant for some workloads: see the OVERHEAD section.
33
34       This  works by tracing various kernel scheduler functions using dynamic
35       tracing, and will need updating to match any  changes  to  these  func‐
36       tions.
37
38       Since this uses BPF, only the root user can use this tool.
39

REQUIREMENTS

41       CONFIG_BPF and bcc.
42

OPTIONS

44       -h     Print usage message.
45
46       -T     Include timestamps on output.
47
48       -m     Output histogram in milliseconds.
49
50       -P     Print a histogram for each PID.
51
52       --pidnss
53              Print  a  histogram for each PID namespace (short for PID names‐
54              paces). For container analysis.
55
56       -L     Print a histogram for each thread ID.
57
58       -p PID Only show this PID (filtered in kernel for efficiency).
59
60       interval
61              Output interval, in seconds.
62
63       count  Number of outputs.
64

EXAMPLES

66       Summarize run queue latency as a histogram:
67              # runqlat
68
69       Print 1 second summaries, 10 times:
70              # runqlat 1 10
71
72       Print 1 second summaries, using milliseconds  as  units  for  the  his‐
73       togram, and include timestamps on output:
74              # runqlat -mT 1
75
76       Trace PID 186 only, 1 second summaries:
77              # runqlat -P 185 1
78

FIELDS

80       usecs  Microsecond range
81
82       msecs  Millisecond range
83
84       count  How many times a task event fell into this range
85
86       distribution
87              An ASCII bar chart to visualize the distribution (count column)
88

OVERHEAD

90       This  traces scheduler functions, which can become very frequent. While
91       eBPF has very low overhead, and this tool uses in-kernel maps for effi‐
92       ciency,  the  frequency  of  scheduler events for some workloads may be
93       high enough that the overhead of this tool becomes significant. Measure
94       in a lab environment to quantify the overhead before use.
95

SOURCE

97       This is from bcc.
98
99              https://github.com/iovisor/bcc
100
101       Also  look  in  the bcc distribution for a companion _examples.txt file
102       containing example usage, output, and commentary for this tool.
103

OS

105       Linux
106

STABILITY

108       Unstable - in development.
109

AUTHOR

111       Brendan Gregg
112