bcc-funclatency(8)

1funclatency(8)              System Manager's Manual             funclatency(8)
2
3
4

NAME

6       funclatency - Time functions and print latency as a histogram.
7

SYNOPSIS

9       funclatency  [-h]  [-p  PID] [-i INTERVAL] [-d DURATION] [-T] [-u] [-m]
10       [-F] [-r] [-v] pattern
11

DESCRIPTION

13       This tool traces function calls and times their duration (latency), and
14       shows  the  latency  distribution  as a histogram. The time is measured
15       from when the function is called to when it returns, and  is  inclusive
16       of both on-CPU time and time spent blocked.
17
18       This  tool uses in-kernel eBPF maps for storing timestamps and the his‐
19       togram, for efficiency.
20
21       Currently nested or recursive functions are not supported properly, and
22       timestamps  will  be overwritten, creating dubious output. Try to match
23       single functions, or groups of functions that run  at  the  same  stack
24       layer, and don't ultimately call each other.
25
26       WARNING:  This uses dynamic tracing of (what can be many) functions, an
27       activity that has had issues on some kernel versions (risk of panics or
28       freezes). Test, and know what you are doing, before use.
29
30       Since this uses BPF, only the root user can use this tool.
31

REQUIREMENTS

33       CONFIG_BPF and bcc.
34

OPTIONS

36       pattern  Function  name  or search pattern. Supports "*" wildcards. See
37       EXAMPLES.  You can also use -r for regular expressions.
38
39       -h     Print usage message.
40
41       -p PID Trace this process ID only.
42
43       -i INTERVAL
44              Print output every interval seconds.
45
46       -d DURATION
47              Total duration of trace, in seconds.
48
49       -l LEVEL
50              Set the level of nested or recursive functions.
51
52       -T     Include timestamps on output.
53
54       -u     Output histogram in microseconds.
55
56       -m     Output histogram in milliseconds.
57
58       -F     Print a separate histogram per function matched.
59
60       -r     Use regular expressions for the search pattern.
61
62       -v     Print the BPF program (for debugging purposes).
63

EXAMPLES

65       Time the do_sys_open() kernel function, and print the distribution as a
66       histogram:
67              # funclatency do_sys_open
68
69       Time the read() function in libc across all processes on the system:
70              # funclatency c:read
71
72       Time vfs_read(), and print the histogram in units of microseconds:
73              # funclatency -u vfs_read
74
75       Time do_nanosleep(), and print the histogram in units of milliseconds:
76              # funclatency -m do_nanosleep
77
78       Time  libc  open(),  and  print output every 2 seconds, for duration 10
79       seconds:
80              # funclatency -i 2 -d 10 c:read
81
82       Time vfs_read(), and print output every 5 seconds, with timestamps:
83              # funclatency -mTi 5 vfs_read
84
85       Time vfs_read() for process ID 181 only:
86              # funclatency -p 181 vfs_read:
87
88       Time both vfs_fstat() and vfs_fstatat() calls, by use of a wildcard:
89              # funclatency 'vfs_fstat*'
90
91       Time both vfs_fstat* calls, and print a separate histogram for each:
92              # funclatency -F 'vfs_fstat*'
93

FIELDS

95       necs   Nanosecond range
96
97       usecs  Microsecond range
98
99       msecs  Millisecond range
100
101       count  How many calls fell into this range
102
103       distribution
104              An ASCII bar chart to visualize the distribution (count column)
105

OVERHEAD

107       This traces kernel functions and maintains in-kernel timestamps  and  a
108       histogram,  which  are  asynchronously copied to user-space. While this
109       method is very efficient, the rate of kernel functions can also be very
110       high  (>1M/sec),  at which point the overhead is expected to be measur‐
111       able. Measure in a test environment  and  understand  overheads  before
112       use. You can also use funccount to measure the rate of kernel functions
113       over a short duration, to set some expectations before use.
114

SOURCE

116       This is from bcc.
117
118              https://github.com/iovisor/bcc
119
120       Also look in the bcc distribution for a  companion  _examples.txt  file
121       containing example usage, output, and commentary for this tool.
122

OS

124       Linux
125

STABILITY

127       Unstable - in development.
128

AUTHOR

130       Brendan Gregg, Sasha Goldshtein
131