1argdist(8) System Manager's Manual argdist(8)
2
6 argdist - Trace a function and display a histogram or frequency count
7 of its parameter values. Uses Linux eBPF/bcc.
8
10 argdist [-h] [-p PID] [-z STRING_SIZE] [-i INTERVAL] [-d DURATION] [-n
11 COUNT] [-v] [-T TOP] [-H specifier] [-C specifier] [-I header]
12
14 argdist attaches to function entry and exit points, collects specified
15 parameter values, and stores them in a histogram or a frequency collec‐
16 tion that counts the number of times a parameter value occurred. It can
17 also filter parameter values and instrument multiple entry points at
18 once.
19 Since this uses BPF, only the root user can use this tool.
21
23 CONFIG_BPF and bcc.
24
26 -h Print usage message.
27 -p PID Trace only functions in the process PID.
29 -z STRING_SIZE
31 When collecting string arguments (of type char*), collect up to
32 STRING_SIZE characters. Longer strings will be truncated.
33 -i INTERVAL
35 Print the collected data every INTERVAL seconds. The default is
36 1 second.
37 -d DURATION
39 Total duration of trace in seconds.
40 -n NUMBER
42 Print the collected data COUNT times and then exit.
43 -v Display the generated BPF program, for debugging purposes.
45 -T TOP When collecting frequency counts, display only the top TOP
47 entries.
48 -H specifiers, -C specifiers
50 One or more probe specifications that instruct argdist which
51 functions to probe, which parameters to collect, how to aggre‐
52 gate them, and whether to perform any filtering. See SPECIFIER
53 SYNTAX below.
54 -I header
56 One or more header files that should be included in the BPF pro‐
57 gram. This enables the use of structure definitions, enumera‐
58 tions, and constants that are available in these headers. You
59 should provide the same path you would include in the BPF pro‐
60 gram, e.g. 'linux/blkdev.h' or 'linux/time.h'. Note: in many
61 cases, argdist will deduce the necessary header files automati‐
62 cally.
63
65 The general specifier syntax is as follows:
66 {p,r,t,u}:{[library],category}:function(signa‐
68 ture)[:type[,type...]:expr[,expr...][:filter]][#label]
69 {p,r,t,u}
71 Probe type - "p" for function entry, "r" for function return,
72 "t" for kernel tracepoint, "u" for USDT probe; -H for histogram
73 collection, -C for frequency count. Indicates where to place
74 the probe and whether the probe should collect frequency count
75 information, or aggregate the collected values into a histogram.
76 Counting probes will collect the number of times every parameter
77 value was observed, whereas histogram probes will collect the
78 parameter values into a histogram. Only integral types can be
79 used with histogram probes; there is no such limitation for
80 counting probes.
81 [library]
83 Library containing the probe. Specify the full path to the .so
84 or executable file where the function to probe resides. Alterna‐
85 tively, you can specify just the lib name: for example, "c"
86 refers to libc. If no library name is specified, the kernel is
87 assumed.
88 category
90 The category of the kernel tracepoint. For example: net, sched,
91 block.
92 function(signature)
94 The function to probe, and its signature. The function name
95 must match exactly for the probe to be placed. The signature, on
96 the other hand, is only required if you plan to collect parame‐
97 ter values based on that signature. For example, if you only
98 want to collect the first parameter, you don't have to specify
99 the rest of the parameters in the signature. When capturing
100 kernel tracepoints, this should be the name of the event, e.g.
101 net_dev_start_xmit. The signature for kernel tracepoints should
102 be empty. When capturing USDT probes, this should be the name of
103 the probe, e.g. reloc_complete. The signature for USDT probes
104 should be empty.
105 [type[,type...]]
107 The type(s) of the expression(s) to capture. This is the type
108 of the keys in the histogram or raw event collection that are
109 collected by the probes.
110 [expr[,expr...]]
112 The expression(s) to capture. These are the values that are
113 assigned to the histogram or raw event collection. You may use
114 the parameters directly, or valid C expressions that involve the
115 parameters, such as "size % 10". Tracepoints may access a spe‐
116 cial structure called "args" that is formatted according to the
117 tracepoint format (which you can obtain using tplist). For
118 example, the block:block_rq_complete tracepoint can access
119 args->nr_sector. USDT probes may access the arguments defined
120 by the tracing program in the special arg1, arg2, ... variables.
121 To obtain their types, use the tplist tool. Return probes can
122 use the argument values received by the function when it was
123 entered, through the $entry(paramname) special variable. Return
124 probes can also access the function's return value in $retval,
125 and the function's execution time in nanoseconds in $latency.
126 Note that adding the $latency or $entry(paramname) variables to
127 the expression will introduce an additional probe at the func‐
128 tion's entry to collect this data, and therefore introduce addi‐
129 tional overhead.
130 [filter]
132 The filter applied to the captured data. Only parameter values
133 that pass the filter will be collected. This is any valid C
134 expression that refers to the parameter values, such as "fd == 1
135 && length > 16". The $entry, $retval, and $latency variables
136 can be used here as well, in return probes. The filter expres‐
137 sion may also use the STRCMP pseudo-function to compare a prede‐
138 fined string to a string argument. For example: STR‐
139 CMP("test.txt", file). The order of arguments is important: the
140 first argument MUST be a quoted literal string, and the second
141 argument can be a runtime string.
142 [label]
144 The label that will be displayed when printing the probed val‐
145 ues. By default, this is the probe specifier.
146
148 Print a histogram of allocation sizes passed to kmalloc:
149 # argdist -H 'p::__kmalloc(u64 size):u64:size'
150 Print a count of how many times process 1005 called malloc with an
152 allocation size of 16 bytes:
153 # argdist -p 1005 -C 'p:c:malloc(size_t
154 size):size_t:size:size==16'
155 Snoop on all strings returned by gets():
157 # argdist -C 'r:c:gets():char*:$retval'
158 Print a histogram of read sizes that were longer than 1ms:
160 # argdist -H 'r::__vfs_read(void *file, void *buf, size_t
161 count):size_t:$entry(count):$latency > 1000000'
162 Print frequency counts of how many times writes were issued to a par‐
164 ticular file descriptor number, in process 1005:
165 # argdist -p 1005 -C 'p:c:write(int fd):int:fd'
166 Print a histogram of error codes returned by read() in process 1005:
168 # argdist -p 1005 -H 'r:c:read()'
169 Print a histogram of buffer sizes passed to write() across all pro‐
171 cesses, where the file descriptor was 1 (STDOUT):
172 # argdist -H 'p:c:write(int fd, const void *buf, size_t
173 count):size_t:count:fd==1'
174 Count fork() calls in libc across all processes, grouped by pid:
176 # argdist -C 'p:c:fork():int:$PID;fork per process'
177 Print histogram of number of sectors in completing block I/O requests:
179 # argdist -H 't:block:block_rq_complete():u32:nr_sector'
180 Aggregate interrupts by interrupt request (IRQ):
182 # argdist -C 't:irq:irq_handler_entry():int:irq'
183 Print the functions used as thread entry points and how common they
185 are:
186 # argdist -C 'u:pthread:pthread_start():u64:arg2' -p 1337
187 Print histograms of sleep() and nanosleep() parameter values:
189 # argdist -H 'p:c:sleep(u32 seconds):u32:seconds' -H
190 'p:c:nanosleep(struct timespec *req):long:req->tv_nsec'
191 Spy on writes to STDOUT performed by process 2780, up to a string size
193 of 120 characters:
194 # argdist -p 2780 -z 120 -C 'p:c:write(int fd, char* buf, size_t
195 len):char*:buf:fd==1'
196 Group files being read from and the read sizes from __vfs_read:
198 # argdist -C 'p::__vfs_read(struct file *file, void *buf, size_t
199 count):char*,size_t:file->f_path.den‐
200 try->d_iname,count:file->f_path.dentry->d_iname[0]!=0'
201
203 This is from bcc.
204 https://github.com/iovisor/bcc
206 Also look in the bcc distribution for a companion _examples.txt file
208 containing example usage, output, and commentary for this tool.
209
211 Linux
212
214 Unstable - in development.
215
217 Sasha Goldshtein
218USER COMMANDS 2016-02-11 argdist(8)