1trace(8)                    System Manager's Manual                   trace(8)
2
3
4

NAME

6       trace  -  Trace  a  function  and  print its arguments or return value,
7       optionally evaluating a filter. Uses Linux eBPF/bcc.
8

SYNOPSIS

10       trace [-h] [-b BUFFER_PAGES] [-p PID] [-L TID]  [-v]  [-Z  STRING_SIZE]
11       [-S] [-s SYM_FILE_LIST]
12                [-M MAX_EVENTS] [-t] [-T] [-C] [-K] [-U] [-a] [-I header]
13                probe [probe ...]
14

DESCRIPTION

16       trace  probes  functions  you  specify and displays trace messages if a
17       particular condition is met. You can control the message format to dis‐
18       play function arguments and return values.
19
20       Since this uses BPF, only the root user can use this tool.
21

REQUIREMENTS

23       CONFIG_BPF and bcc.
24

OPTIONS

26       -h     Print usage message.
27
28       -p PID Trace only functions in the process PID.
29
30       -L TID Trace only functions in the thread TID.
31
32       -v     Display the generated BPF program, for debugging purposes.
33
34       -z STRING_SIZE
35              When  collecting string arguments (of type char*), collect up to
36              STRING_SIZE characters. Longer strings will be truncated.
37
38       -s SYM_FILE_LIST
39              When collecting stack trace in build id  format,  use  the  coma
40              separated list for symbol resolution.
41
42       -S     If  set,  trace  messages  from trace's own process. By default,
43              this is off to avoid tracing storms -- for example, if you trace
44              the write system call, and consider that trace is writing to the
45              standard output.
46
47       -M MAX_EVENTS
48              Print up to MAX_EVENTS trace messages and then exit.
49
50       -t     Print times relative to the beginning of the trace (offsets), in
51              seconds.
52
53       -T     Print the time column.
54
55       -C     Print CPU id.
56
57       -B     Treat argument of STRCMP helper as a binary value
58
59       -K     Print the kernel stack for each event.
60
61       -U     Print  the  user stack for each event.  -a Print virtual address
62              in kernel and user stacks.
63
64       -I header
65              Additional header files to include in the BPF program.  This  is
66              needed  if  your  filter  or print expressions use types or data
67              structures that are not available in the standard  headers.  For
68              example: 'linux/mm.h'
69
70       probe [probe ...]
71              One  or more probes that attach to functions, filter conditions,
72              and print information. See PROBE SYNTAX below.
73

PROBE SYNTAX

75       The general probe syntax is as follows:
76
77       [{p,r}]:[library]:function[(signature)]     [(predicate)]      ["format
78       string"[, arguments]]
79
80       {t:category:event,u:library:probe}   [(predicate)]  ["format  string"[,
81       arguments]]
82
83       {[{p,r}],t,u}
84              Probe type - "p" for function entry, "r"  for  function  return,
85              "t" for kernel tracepoint, "u" for USDT probe. The default probe
86              type is "p".
87
88       [library]
89              Library containing the probe.  Specify the full path to the  .so
90              or executable file where the function to probe resides. Alterna‐
91              tively, you can specify just the  lib  name:  for  example,  "c"
92              refers  to  libc. If no library name is specified, the kernel is
93              assumed. Also, you can specify an  executable  name  (without  a
94              full path) if it is in the PATH.  For example, "bash".
95
96       category
97              The tracepoint category. For example, "sched" or "irq".
98
99       function
100              The function to probe.
101
102       signature
103              The  optional  signature of the function to probe. This can make
104              it easier to access the function's arguments, instead  of  using
105              the  "arg1",  "arg2"  etc.   argument  specifiers.  For example,
106              "(struct timespec *ts)" in the signature position lets  you  use
107              "ts" in the filter or print expressions.
108
109       event  The tracepoint event. For example, "block_rq_complete".
110
111       probe  The USDT probe name. For example, "pthread_create".
112
113       [(predicate)]
114              The  filter  applied  to  the  captured data. Only if the filter
115              evaluates as true, the trace message will be printed. The filter
116              can  use any valid C expression that refers to the argument val‐
117              ues: arg1, arg2, etc., or to the return value retval in a return
118              probe.  If  necessary,  use C cast operators to coerce the argu‐
119              ments to the desired type. For example, if arg1 is of type  int,
120              use  the  expression  ((int)arg1  < 0) to trace only invocations
121              where arg1 is negative.  Note that only arg1-arg6 are supported,
122              and only if the function is using the standard x86_64 convention
123              where the first six arguments are in the RDI, RSI, RDX, RCX, R8,
124              R9 registers. If no predicate is specified, all function invoca‐
125              tions are traced.
126
127              The predicate expression may also use the STRCMP pseudo-function
128              to  compare  a predefined string to a string argument. For exam‐
129              ple: STRCMP("test", arg1).  The order of arguments is important:
130              the first argument MUST be a quoted literal string, and the sec‐
131              ond argument can be a runtime string, most  typically  an  argu‐
132              ment.
133
134       ["format string"[, arguments]]
135              A  printf-style  format  string  that will be used for the trace
136              message. You can use the following format  specifiers:  %s,  %d,
137              %u,  %lld,  %llu, %hd, %hu, %c, %x, %llx -- with the same seman‐
138              tics as printf's. Make sure to pass the exact  number  of  argu‐
139              ments as there are placeholders in the format string. The format
140              specifier replacements may be any C expressions, and  may  refer
141              to  the  same  special keywords as in the predicate (arg1, arg2,
142              etc.).
143
144              In addition to the above format specifiers, you can also use  %K
145              and %U when the expression is an address that potentially points
146              to executable code (i.e., a symbol). trace will resolve %K spec‐
147              ifiers  to  a kernel symbol, such as vfs__read, and will resolve
148              %U specifiers to a user-space symbol in that  process,  such  as
149              sprintf.
150
151              In  tracepoints,  both the predicate and the arguments may refer
152              to the tracepoint format structure, which is stored in the  spe‐
153              cial  "args"  variable. For example, the block:block_rq_complete
154              tracepoint can print or filter by args->nr_sector.  To  discover
155              the format of your tracepoint, use the tplist tool.
156
157              In  USDT  probes,  the  arg1,  ...,  argN variables refer to the
158              probe's arguments.  To determine which arguments your probe has,
159              use the tplist tool.
160
161              The  predicate  expression and the format specifier replacements
162              for printing may also use the following special keywords:  $pid,
163              $tgid  to refer to the current process' pid and tgid; $uid, $gid
164              to refer to the current user's uid and gid; $cpu to refer to the
165              current processor number.
166

EXAMPLES

168       Trace all invocations of the open system call with the name of the file
169       being opened:
170              # trace '::do_sys_open "%s", arg2'
171
172       Trace all invocations of the read system call where the number of bytes
173       requested is greater than 20,000:
174              # trace '::sys_read (arg3 > 20000) "read %d bytes", arg3'
175
176       Trace all malloc calls and print the size of the requested allocation:
177              # trace ':c:malloc "size = %d", arg1'
178
179       Trace  returns  from the readline function in bash and print the return
180       value as a string:
181              # trace 'r:bash:readline "%s", retval'
182
183       Trace the block:block_rq_complete tracepoint and print  the  number  of
184       sectors completed:
185              #  trace  't:block:block_rq_complete "%d sectors", args->nr_sec‐
186              tor'
187
188       Trace the pthread_create USDT probe from the pthread library and  print
189       the address of the thread's start function:
190              # trace 'u:pthread:pthread_create "start addr = %llx", arg3'
191
192       Trace  the  nanosleep  system  call  and  print  the  sleep duration in
193       nanoseconds:
194              # trace 'p::SyS_nanosleep(struct timespec *ts) sleep for %lld ns
195              , ts->tv_nsec'
196
197       Trace  the inet_pton system call using build id mechanism and print the
198       stack
199              #     trace     -s     /lib/x86_64-linux-gnu/libc.so.6,/bin/ping
200              'p:c:inet_pton' -U
201

SOURCE

203       This is from bcc.
204
205              https://github.com/iovisor/bcc
206
207       Also  look  in  the bcc distribution for a companion _examples.txt file
208       containing example usage, output, and commentary for this tool.
209

OS

211       Linux
212

STABILITY

214       Unstable - in development.
215

AUTHOR

217       Sasha Goldshtein
218
219
220
221USER COMMANDS                     2016-02-18                          trace(8)
Impressum