bcc-funcslower(8)

1funcslower(8)               System Manager's Manual              funcslower(8)
2
3
4

NAME

6       funcslower - Trace slow kernel or user function calls.
7

SYNOPSIS

9       funcslower  [-hf]  [-p PID] [-U | -K] [-m MIN_MS] [-u MIN_US] [-a ARGU‐
10       MENTS] [-T] [-t] [-v] function [function ...]
11

DESCRIPTION

13       This script traces a kernel or user function's entry and return points,
14       and prints a message when the function's latency exceeded the specified
15       threshold.  Multiple functions are supported, and you  can  mix  kernel
16       functions with user functions in different libraries.
17
18       WARNING: See the OVERHEAD section.
19
20       By  default,  a  minimum  millisecond threshold of 1 is used. Recursive
21       functions are not supported: only the inner-most  recursive  invocation
22       will be traced.
23
24       Since this uses BPF, only the root user can use this tool.
25

REQUIREMENTS

27       CONFIG_BPF and bcc.
28

OPTIONS

30       -p PID Trace this PID only.
31
32       -m MIN_NS
33              Minimum duration to trace, in milliseconds. Default is 1 ms.
34
35       -u MIN_US
36              Minimum duration to trace, in microseconds.
37
38       -a ARGUMENTS
39              Print the function's arguments, up to 6.
40
41       -T     Print a HH:MM:SS timestamp with each entry.
42
43       -t     Print  a seconds timestamp with each entry, at microsecond reso‐
44              lution.
45
46       -f     Print output in folded stack format.
47
48       -U     Show stacks from user space only (no kernel space stacks).
49
50       -K     Show stacks from kernel space only (no user space stacks).
51
52       -v     Print the resulting BPF program, for debugging purposes.
53
54       function
55              The function to trace -- multiple functions are supported. If  a
56              plain function name is provided, the function is assumed to be a
57              kernel function. For user functions, provide  the  library  name
58              and the function name, e.g. bash:readline or c:malloc.
59

EXAMPLES

61       Trace vfs_write calls slower than 1ms:
62              # funcslower vfs_write
63
64       Trace open() calls in libc slower than 10us:
65              # funcslower -u 10 c:open
66
67       Trace both malloc() and free() slower than 10us, in pid 135 only:
68              # funcslower -p 135 -u 10 c:malloc c:free
69
70       Trace the write syscall and print its first 4 arguments:
71              # funcslower -a 4 SyS_write
72
73       Trace opens from libc and print the user and kernel stack frames:
74              # funcslower -UK c:open
75

FIELDS

77       TIME   Time  of  the  event  as  a human-readable HH:MM:SS format, or a
78              timestamp in seconds  at  microsecond-accuracy  from  the  first
79              event seen.
80
81       COMM   Process name.
82
83       PID    Process ID.
84
85       LAT    Latency  of  the  operation  in either microseconds (us) or mil‐
86              liseconds (ms).
87
88       RVAL   The return value from the function. Often useful for  diagnosing
89              a relationship between slow and failed function calls.
90
91       FUNC   The function name, followed by its arguments if requested.
92

OVERHEAD

94       Depending  on the function(s) being traced, overhead can become severe.
95       For example, tracing a common function like malloc() can  slow  down  a
96       C/C++  program  by  a factor of 2 or more. On the other hand, tracing a
97       low-frequency event like the SyS_setreuid() function will probably  not
98       be as prohibitive, and in fact negligible for functions that are called
99       up to 100-1000 times per second.
100
101       You should first use the funclatency and argdist tools  for  investiga‐
102       tion, because they summarize data in-kernel and have a much lower over‐
103       head than this tool. To get a general idea of the  number  of  times  a
104       particular  function  is  called  (and  estimate the overhead), use the
105       funccount tool, e.g.:
106
107       # funccount c:open
108

SOURCE

110       This is from bcc.
111
112              https://github.com/iovisor/bcc
113
114       Also look in the bcc distribution for a  companion  _examples.txt  file
115       containing example usage, output, and commentary for this tool.
116

OS

118       Linux
119

STABILITY

121       Unstable - in development.
122

AUTHOR

124       Sasha Goldshtein
125