1deadlock(8)                 System Manager's Manual                deadlock(8)
2
3
4

NAME

6       deadlock  -  Find potential deadlocks (lock order inversions) in a run‐
7       ning program.
8

SYNOPSIS

10       deadlock [-h] [--binary BINARY] [--dump-graph  DUMP_GRAPH]  [--verbose]
11       [--lock-symbols LOCK_SYMBOLS] [--unlock-symbols UNLOCK_SYMBOLS] pid
12

DESCRIPTION

14       deadlock  finds  potential  deadlocks in a running process. The program
15       attaches uprobes on `pthread_mutex_lock` and `pthread_mutex_unlock`  by
16       default  to  build  a  mutex  wait directed graph, and then looks for a
17       cycle in this graph.  This graph has the following properties:
18
19       - Nodes in the graph represent mutexes.
20
21       - Edge (A, B) exists if there exists some thread T  where  lock(A)  was
22       called and lock(B) was called before unlock(A) was called.
23
24       If  there is a cycle in this graph, this indicates that there is a lock
25       order inversion (potential deadlock). If the program finds a lock order
26       inversion,  the  program will dump the cycle of mutexes, dump the stack
27       traces where each mutex was acquired, and then exit.
28
29       This program can only find potential deadlocks  that  occur  while  the
30       program  is tracing the process. It cannot find deadlocks that may have
31       occurred before the program was attached to the process.
32
33       This tool does not work for shared mutexes or recursive mutexes.
34
35       Since this uses BPF, only the root user can use this tool.
36

REQUIREMENTS

38       CONFIG_BPF and bcc
39

OPTIONS

41       -h, --help
42              show this help message and exit
43
44       --binary BINARY
45              If set, trace the mutexes from the  binary  at  this  path.  For
46              statically-linked  binaries, this argument is not required.  For
47              dynamically-linked  binaries,  this  argument  is  required  and
48              should  be  the path of the pthread library the binary is using.
49              Example: /lib/x86_64-linux-gnu/libpthread.so.0
50
51       --dump-graph DUMP_GRAPH
52              If set, this will dump the mutex graph to the specified file.
53
54       --verbose
55              Print statistics about the mutex wait graph.
56
57       --lock-symbols LOCK_SYMBOLS
58              Comma-separated list  of  lock  symbols  to  trace.  Default  is
59              pthread_mutex_lock.   These  symbols  cannot  be  inlined in the
60              binary.
61
62       --unlock-symbols UNLOCK_SYMBOLS
63              Comma-separated list of unlock  symbols  to  trace.  Default  is
64              pthread_mutex_unlock.  These  symbols  cannot  be inlined in the
65              binary.
66
67       pid    Pid to trace
68

EXAMPLES

70       Find potential deadlocks in PID  181.  The  --binary  argument  is  not
71       needed for statically-linked binaries.
72              # deadlock 181
73
74       Find  potential deadlocks in PID 181. If the process was created from a
75       dynamically-linked executable, the --binary argument  is  required  and
76       must be the path of the pthread library:
77              # deadlock 181 --binary /lib/x86_64-linux-gnu/libpthread.so.0
78
79       Find  potential deadlocks in PID 181. If the process was created from a
80       statically-linked executable,  optionally  pass  the  location  of  the
81       binary.  On  older kernels without https://lkml.org/lkml/2017/1/13/585,
82       binaries that contain `:` in the path cannot be attached with  uprobes.
83       As a workaround, we can create a symlink to the binary, and provide the
84       symlink name instead with the `--binary` option:
85              # deadlock 181 --binary /usr/local/bin/lockinversion
86
87       Find potential deadlocks in PID 181 and dump the mutex wait graph to  a
88       file:
89              # deadlock 181 --dump-graph graph.json
90
91       Find  potential deadlocks in PID 181 and print mutex wait graph statis‐
92       tics:
93              # deadlock 181 --verbose
94
95       Find potential deadlocks in PID 181 with custom mutexes:
96              #   deadlock    181    --lock-symbols    custom_mutex1_lock,cus‐
97              tom_mutex2_lock    --unlock_symbols    custom_mutex1_unlock,cus‐
98              tom_mutex2_unlock
99

OUTPUT

101       This program does not output any fields. Rather, it will  keep  running
102       until  it  finds  a potential deadlock, or the user hits Ctrl-C. If the
103       program finds a potential deadlock, it will output the stack traces and
104       lock order inversion in the following format and exit:
105
106       Potential Deadlock Detected!
107
108       Cycle in lock order graph: Mutex M0 => Mutex M1 => Mutex M0
109
110       Mutex M1 acquired here while holding Mutex M0 in Thread T:
111              [stack trace]
112
113       Mutex M0 previously acquired by the same Thread T here:
114              [stack trace]
115
116       Mutex M0 acquired here while holding Mutex M1 in Thread S:
117              [stack trace]
118
119       Mutex M1 previously acquired by the same Thread S here:
120              [stack trace]
121
122       Thread T created by Thread R here:
123              [stack trace]
124
125       Thread S created by Thread Q here:
126              [stack trace]
127

OVERHEAD

129       This  traces  all  mutex lock and unlock events and all thread creation
130       events on the traced process. The overhead of this can be high  if  the
131       process  has  many  threads  and mutexes. You should only run this on a
132       process where the slowdown is acceptable.
133

SOURCE

135       This is from bcc.
136
137              https://github.com/iovisor/bcc
138
139       Also look in the bcc distribution for a  companion  _examples.txt  file
140       containing example usage, output, and commentary for this tool.
141

OS

143       Linux
144

STABILITY

146       Unstable - in development.
147

AUTHOR

149       Kenny Yu
150
151
152
153USER COMMANDS                     2017-02-01                       deadlock(8)