bcc-funclatency(8)

1funclatency(8)              System Manager's Manual             funclatency(8)
2
3
4

NAME

6       funclatency - Time functions and print latency as a histogram.
7

SYNOPSIS

9       funclatency  [-h]  [-p  PID] [-i INTERVAL] [-d DURATION] [-T] [-u] [-m]
10       [-F] [-r] [-v] pattern
11

DESCRIPTION

13       This tool traces function calls and times their duration (latency), and
14       shows  the  latency  distribution  as a histogram. The time is measured
15       from when the function is called to when it returns, and  is  inclusive
16       of both on-CPU time and time spent blocked.
17
18       This  tool uses in-kernel eBPF maps for storing timestamps and the his‐
19       togram, for efficiency.
20
21       Currently nested or recursive functions are not supported properly, and
22       timestamps  will  be overwritten, creating dubious output. Try to match
23       single functions, or groups of functions that run  at  the  same  stack
24       layer, and don't ultimately call each other.
25
26       WARNING:  This uses dynamic tracing of (what can be many) functions, an
27       activity that has had issues on some kernel versions (risk of panics or
28       freezes). Test, and know what you are doing, before use.
29
30       Since this uses BPF, only the root user can use this tool.
31

REQUIREMENTS

33       CONFIG_BPF and bcc.
34

OPTIONS

36       pattern  Function  name  or search pattern. Supports "*" wildcards. See
37       EXAMPLES.  You can also use -r for regular expressions.  -h Print usage
38       message.
39
40       -p PID Trace this process ID only.
41
42       -i INTERVAL
43              Print output every interval seconds.
44
45       -d DURATION
46              Total duration of trace, in seconds.
47
48       -l LEVEL
49              Set the level of nested or recursive functions.
50
51       -T     Include timestamps on output.
52
53       -u     Output histogram in microseconds.
54
55       -m     Output histogram in milliseconds.
56
57       -F     Print a separate histogram per function matched.
58
59       -r     Use regular expressions for the search pattern.
60
61       -v     Print the BPF program (for debugging purposes).
62

EXAMPLES

64       Time the do_sys_open() kernel function, and print the distribution as a
65       histogram:
66              # funclatency do_sys_open
67
68       Time the read() function in libc across all processes on the system:
69              # funclatency c:read
70
71       Time vfs_read(), and print the histogram in units of microseconds:
72              # funclatency -u vfs_read
73
74       Time do_nanosleep(), and print the histogram in units of milliseconds:
75              # funclatency -m do_nanosleep
76
77       Time libc open(), and print output every 2  seconds,  for  duration  10
78       seconds:
79              # funclatency -i 2 -d 10 c:read
80
81       Time vfs_read(), and print output every 5 seconds, with timestamps:
82              # funclatency -mTi 5 vfs_read
83
84       Time vfs_read() for process ID 181 only:
85              # funclatency -p 181 vfs_read:
86
87       Time both vfs_fstat() and vfs_fstatat() calls, by use of a wildcard:
88              # funclatency 'vfs_fstat*'
89
90       Time both vfs_fstat* calls, and print a separate histogram for each:
91              # funclatency -F 'vfs_fstat*'
92

FIELDS

94       necs   Nanosecond range
95
96       usecs  Microsecond range
97
98       msecs  Millisecond range
99
100       count  How many calls fell into this range
101
102       distribution
103              An ASCII bar chart to visualize the distribution (count column)
104

OVERHEAD

106       This  traces  kernel functions and maintains in-kernel timestamps and a
107       histogram, which are asynchronously copied to  user-space.  While  this
108       method is very efficient, the rate of kernel functions can also be very
109       high (>1M/sec), at which point the overhead is expected to  be  measur‐
110       able.  Measure  in  a  test environment and understand overheads before
111       use. You can also use funccount to measure the rate of kernel functions
112       over a short duration, to set some expectations before use.
113

SOURCE

115       This is from bcc.
116
117              https://github.com/iovisor/bcc
118
119       Also  look  in  the bcc distribution for a companion _examples.txt file
120       containing example usage, output, and commentary for this tool.
121

OS

123       Linux
124

STABILITY

126       Unstable - in development.
127

AUTHOR

129       Brendan Gregg, Sasha Goldshtein
130