1argdist(8) System Manager's Manual argdist(8)
2
6 argdist - Trace a function and display a histogram or frequency count
7 of its parameter values. Uses Linux eBPF/bcc.
8
10 argdist [-h] [-p PID] [-z STRING_SIZE] [-i INTERVAL] [-d DURATION] [-n
11 COUNT] [-v] [-T TOP] [-H specifier] [-C specifier] [-I header] [-t TID]
12
14 argdist attaches to function entry and exit points, collects specified
15 parameter values, and stores them in a histogram or a frequency collec‐
16 tion that counts the number of times a parameter value occurred. It can
17 also filter parameter values and instrument multiple entry points at
18 once.
19 Since this uses BPF, only the root user can use this tool.
21
23 CONFIG_BPF and bcc.
24
26 -h Print usage message.
27 -p PID Trace only functions in the process PID.
29 -t TID Trace only functions in the thread TID.
31 -z STRING_SIZE
33 When collecting string arguments (of type char*), collect up to
34 STRING_SIZE characters. Longer strings will be truncated.
35 -i INTERVAL
37 Print the collected data every INTERVAL seconds. The default is
38 1 second.
39 -d DURATION
41 Total duration of trace in seconds.
42 -n NUMBER
44 Print the collected data COUNT times and then exit.
45 -v Display the generated BPF program, for debugging purposes.
47 -T TOP When collecting frequency counts, display only the top TOP en‐
49 tries.
50 -H specifiers, -C specifiers
52 One or more probe specifications that instruct argdist which
53 functions to probe, which parameters to collect, how to aggre‐
54 gate them, and whether to perform any filtering. See SPECIFIER
55 SYNTAX below.
56 -I header
58 One or more header files that should be included in the BPF pro‐
59 gram. This enables the use of structure definitions, enumera‐
60 tions, and constants that are available in these headers. You
61 should provide the same path you would include in the BPF pro‐
62 gram, e.g. 'linux/blkdev.h' or 'linux/time.h'. Note: in many
63 cases, argdist will deduce the necessary header files automati‐
64 cally.
65
67 The general specifier syntax is as follows:
68 {p,r,t,u}:{[library],category}:function(signa‐
70 ture)[:type[,type...]:expr[,expr...][:filter]][#label]
71 {p,r,t,u}
73 Probe type - "p" for function entry, "r" for function return,
74 "t" for kernel tracepoint, "u" for USDT probe; -H for histogram
75 collection, -C for frequency count. Indicates where to place
76 the probe and whether the probe should collect frequency count
77 information, or aggregate the collected values into a histogram.
78 Counting probes will collect the number of times every parameter
79 value was observed, whereas histogram probes will collect the
80 parameter values into a histogram. Only integral types can be
81 used with histogram probes; there is no such limitation for
82 counting probes.
83 [library]
85 Library containing the probe. Specify the full path to the .so
86 or executable file where the function to probe resides. Alterna‐
87 tively, you can specify just the lib name: for example, "c"
88 refers to libc. If no library name is specified, the kernel is
89 assumed.
90 category
92 The category of the kernel tracepoint. For example: net, sched,
93 block.
94 function(signature)
96 The function to probe, and its signature. The function name
97 must match exactly for the probe to be placed. The signature, on
98 the other hand, is only required if you plan to collect parame‐
99 ter values based on that signature. For example, if you only
100 want to collect the first parameter, you don't have to specify
101 the rest of the parameters in the signature. When capturing
102 kernel tracepoints, this should be the name of the event, e.g.
103 net_dev_start_xmit. The signature for kernel tracepoints should
104 be empty. When capturing USDT probes, this should be the name of
105 the probe, e.g. reloc_complete. The signature for USDT probes
106 should be empty.
107 [type[,type...]]
109 The type(s) of the expression(s) to capture. This is the type
110 of the keys in the histogram or raw event collection that are
111 collected by the probes.
112 [expr[,expr...]]
114 The expression(s) to capture. These are the values that are as‐
115 signed to the histogram or raw event collection. You may use
116 the parameters directly, or valid C expressions that involve the
117 parameters, such as "size % 10". Tracepoints may access a spe‐
118 cial structure called "args" that is formatted according to the
119 tracepoint format (which you can obtain using tplist). For ex‐
120 ample, the block:block_rq_complete tracepoint can access
121 args->nr_sector. USDT probes may access the arguments defined
122 by the tracing program in the special arg1, arg2, ... variables.
123 To obtain their types, use the tplist tool. Return probes can
124 use the argument values received by the function when it was en‐
125 tered, through the $entry(paramname) special variable. Return
126 probes can also access the function's return value in $retval,
127 and the function's execution time in nanoseconds in $latency.
128 Note that adding the $latency or $entry(paramname) variables to
129 the expression will introduce an additional probe at the func‐
130 tion's entry to collect this data, and therefore introduce addi‐
131 tional overhead.
132 [filter]
134 The filter applied to the captured data. Only parameter values
135 that pass the filter will be collected. This is any valid C ex‐
136 pression that refers to the parameter values, such as "fd == 1
137 && length > 16". The $entry, $retval, and $latency variables
138 can be used here as well, in return probes. The filter expres‐
139 sion may also use the STRCMP pseudo-function to compare a prede‐
140 fined string to a string argument. For example: STR‐
141 CMP("test.txt", file). The order of arguments is important: the
142 first argument MUST be a quoted literal string, and the second
143 argument can be a runtime string.
144 [label]
146 The label that will be displayed when printing the probed val‐
147 ues. By default, this is the probe specifier.
148
150 Print a histogram of allocation sizes passed to kmalloc:
151 # argdist -H 'p::__kmalloc(u64 size):u64:size'
152 Print a count of how many times process 1005 called malloc with an al‐
154 location size of 16 bytes:
155 # argdist -p 1005 -C 'p:c:malloc(size_t
156 size):size_t:size:size==16'
157 Snoop on all strings returned by gets():
159 # argdist -C 'r:c:gets():char*:$retval'
160 Print a histogram of read sizes that were longer than 1ms:
162 # argdist -H 'r::__vfs_read(void *file, void *buf, size_t
163 count):size_t:$entry(count):$latency > 1000000'
164 Print frequency counts of how many times writes were issued to a par‐
166 ticular file descriptor number, in process 1005:
167 # argdist -p 1005 -C 'p:c:write(int fd):int:fd'
168 Print a histogram of error codes returned by read() in process 1005:
170 # argdist -p 1005 -H 'r:c:read()'
171 Print a histogram of buffer sizes passed to write() across all pro‐
173 cesses, where the file descriptor was 1 (STDOUT):
174 # argdist -H 'p:c:write(int fd, const void *buf, size_t
175 count):size_t:count:fd==1'
176 Count fork() calls in libc across all processes, grouped by pid:
178 # argdist -C 'p:c:fork():int:$PID;fork per process'
179 Print histogram of number of sectors in completing block I/O requests:
181 # argdist -H 't:block:block_rq_complete():u32:nr_sector'
182 Aggregate interrupts by interrupt request (IRQ):
184 # argdist -C 't:irq:irq_handler_entry():int:irq'
185 Print the functions used as thread entry points and how common they
187 are:
188 # argdist -C 'u:pthread:pthread_start():u64:arg2' -p 1337
189 Print histograms of sleep() and nanosleep() parameter values:
191 # argdist -H 'p:c:sleep(u32 seconds):u32:seconds' -H
192 'p:c:nanosleep(struct timespec *req):long:req->tv_nsec'
193 Spy on writes to STDOUT performed by process 2780, up to a string size
195 of 120 characters:
196 # argdist -p 2780 -z 120 -C 'p:c:write(int fd, char* buf, size_t
197 len):char*:buf:fd==1'
198 Group files being read from and the read sizes from __vfs_read:
200 # argdist -C 'p::__vfs_read(struct file *file, void *buf, size_t
201 count):char*,size_t:file->f_path.dentry->d_in‐
202 ame,count:file->f_path.dentry->d_iname[0]!=0'
203
205 This is from bcc.
206 https://github.com/iovisor/bcc
208 Also look in the bcc distribution for a companion _examples.txt file
210 containing example usage, output, and commentary for this tool.
211
213 Linux
214
216 Unstable - in development.
217
219 Sasha Goldshtein
220USER COMMANDS 2016-02-11 argdist(8)