bcc-zfsslower(8)

1zfsslower(8)                System Manager's Manual               zfsslower(8)
2
3
4

NAME

6       zfsslower - Trace slow zfs file operations, with per-event details.
7

SYNOPSIS

9       zfsslower [-h] [-j] [-p PID] [min_ms]
10

DESCRIPTION

12       This  tool traces common ZFS file operations: reads, writes, opens, and
13       syncs. It measures the time  spent  in  these  operations,  and  prints
14       details for each that exceeded a threshold.
15
16       WARNING: See the OVERHEAD section.
17
18       By default, a minimum millisecond threshold of 10 is used. If a thresh‐
19       old of 0 is used, all events are printed (warning: verbose).
20
21       This uses kernel dynamic  tracing  of  the  ZPL  interface  (ZFS  POSIX
22       Layer), and will need updates to match any changes to this interface.
23
24       This is intended to work with the ZFS on Linux project:
25              http://zfsonlinux.org
26
27       Since this uses BPF, only the root user can use this tool.
28

REQUIREMENTS

30       CONFIG_BPF and bcc.
31

OPTIONS

33       -p PID Trace this PID only.
34
35       min_ms Minimum  I/O  latency  (duration)  to  trace,  in  milliseconds.
36              Default is 10 ms.
37

EXAMPLES

39       Trace synchronous file reads and writes slower than 10 ms:
40              # zfsslower
41
42       Trace slower than 1 ms:
43              # zfsslower 1
44
45       Trace slower than 1 ms, and output just the fields in  parsable  format
46       (csv):
47              # zfsslower -j 1
48
49       Trace all file reads and writes (warning: the output will be verbose):
50              # zfsslower 0
51
52       Trace slower than 1 ms, for PID 181 only:
53              # zfsslower -p 181 1
54

FIELDS

56       TIME(s)
57              Time of I/O completion since the first I/O seen, in seconds.
58
59       COMM   Process name.
60
61       PID    Process ID.
62
63       T      Type of operation. R == read, W == write, O == open, S == fsync.
64
65       OFF_KB File offset for the I/O, in Kbytes.
66
67       BYTES  Size of I/O, in bytes.
68
69       LAT(ms)
70              Latency  (duration)  of I/O, measured from when it was issued by
71              VFS to the filesystem, to when it completed. This time is inclu‐
72              sive  of  block  device I/O, file system CPU cycles, file system
73              locks, run queue latency, etc. It's a more accurate  measure  of
74              the latency suffered by applications performing file system I/O,
75              than to measure this down at the block device interface.
76
77       FILENAME
78              A cached kernel file name (comes from dentry->d_name.name).
79
80       ENDTIME_us
81              Completion timestamp, microseconds (-j only).
82
83       OFFSET_b
84              File offset, bytes (-j only).
85
86       LATENCY_us
87              Latency (duration) of the I/O, in microseconds (-j only).
88

OVERHEAD

90       This adds low-overhead instrumentation to these ZFS operations, includ‐
91       ing  reads and writes from the file system cache. Such reads and writes
92       can be very frequent (depending on the workload; eg, 1M/sec), at  which
93       point  the overhead of this tool (even if it prints no "slower" events)
94       can begin to become significant. Measure and quantify  before  use.  If
95       this  continues  to  be  a  problem,  consider switching to a tool that
96       prints in-kernel summaries only.
97
98       Note that the overhead of this tool should be less than  fileslower(8),
99       as  this  tool  targets zfs functions only, and not all file read/write
100       paths (which can include socket I/O).
101

SOURCE

103       This is from bcc.
104
105              https://github.com/iovisor/bcc
106
107       Also look in the bcc distribution for a  companion  _examples.txt  file
108       containing example usage, output, and commentary for this tool.
109

OS

111       Linux
112

STABILITY

114       Unstable - in development.
115

AUTHOR

117       Brendan Gregg
118