1PERF-C2C(1) perf Manual PERF-C2C(1)
2
6 perf-c2c - Shared Data C2C/HITM Analyzer.
7
9 perf c2c record [<options>] <command>
10 perf c2c record [<options>] -- [<record command options>] <command>
11 perf c2c report [<options>]
12
14 C2C stands for Cache To Cache.
15 The perf c2c tool provides means for Shared Data C2C/HITM analysis. It
17 allows you to track down the cacheline contentions.
18 On x86, the tool is based on load latency and precise store facility
20 events provided by Intel CPUs. On PowerPC, the tool uses random
21 instruction sampling with thresholding feature.
22 These events provide: - memory address of the access - type of the
24 access (load and store details) - latency (in cycles) of the load
25 access
26 The c2c tool provide means to record this data and report back access
28 details for cachelines with highest contention - highest number of HITM
29 accesses.
30 The basic workflow with this tool follows the standard record/report
32 phase. User uses the record command to record events data and report
33 command to display it.
34
36 -e, --event=
37 Select the PMU event. Use perf c2c record -e list to list available
38 events.
39 -v, --verbose
41 Be more verbose (show counter open errors, etc).
42 -l, --ldlat
44 Configure mem-loads latency. (x86 only)
45 -k, --all-kernel
47 Configure all used events to run in kernel space.
48 -u, --all-user
50 Configure all used events to run in user space.
51
53 -k, --vmlinux=<file>
54 vmlinux pathname
55 -v, --verbose
57 Be more verbose (show counter open errors, etc).
58 -i, --input
60 Specify the input file to process.
61 -N, --node-info
63 Show extra node info in report (see NODE INFO section)
64 -c, --coalesce
66 Specify sorting fields for single cacheline display. Following
67 fields are available: tid,pid,iaddr,dso (see COALESCE)
68 -g, --call-graph
70 Setup callchains parameters. Please refer to perf-report man page
71 for details.
72 --stdio
74 Force the stdio output (see STDIO OUTPUT)
75 --stats
77 Display only statistic tables and force stdio mode.
78 --full-symbols
80 Display full length of symbols.
81 --no-source
83 Do not display Source:Line column.
84 --show-all
86 Show all captured HITM lines, with no regard to HITM % 0.0005
87 limit.
88 -f, --force
90 Don’t do ownership validation.
91 -d, --display
93 Switch to HITM type (rmt, lcl) to display and sort on. Total HITMs
94 as default.
95 --stitch-lbr
97 Show callgraph with stitched LBRs, which may have more complete
98 callgraph. The perf.data file must have been obtained using perf
99 c2c record --call-graph lbr. Disabled by default. In common cases
100 with call stack overflows, it can recreate better call stacks than
101 the default lbr call stack output. But this approach is not full
102 proof. There can be cases where it creates incorrect call stacks
103 from incorrect matches. The known limitations include exception
104 handing such as setjmp/longjmp will have calls/returns not match.
105
107 The perf c2c record command setup options related to HITM cacheline
108 analysis and calls standard perf record command.
109 Following perf record options are configured by default: (check perf
111 record man page for details)
112 -W,-d,--phys-data,--sample-cpu
114 Unless specified otherwise with -e option, following events are
116 monitored by default on x86:
117 cpu/mem-loads,ldlat=30/P
119 cpu/mem-stores/P
120 and following on PowerPC:
122 cpu/mem-loads/
124 cpu/mem-stores/
125 User can pass any perf record option behind -- mark, like (to enable
127 callchains and system wide monitoring):
128 $ perf c2c record -- -g -a
130 Please check RECORD OPTIONS section for specific c2c record options.
132
134 The perf c2c report command displays shared data analysis. It comes in
135 two display modes: stdio and tui (default).
136 The report command workflow is following: - sort all the data based on
138 the cacheline address - store access details for each cacheline - sort
139 all cachelines based on user settings - display data
140 In general perf report output consist of 2 basic views: 1) most
142 expensive cachelines list 2) offsets details for each cacheline
143 For each cacheline in the 1) list we display following data: (Both
145 stdio and TUI modes follow the same fields output)
146 Index
148 - zero based index to identify the cacheline
149 Cacheline
151 - cacheline address (hex number)
152 Rmt/Lcl Hitm
154 - cacheline percentage of all Remote/Local HITM accesses
155 LLC Load Hitm - Total, LclHitm, RmtHitm
157 - count of Total/Local/Remote load HITMs
158 Total records
160 - sum of all cachelines accesses
161 Total loads
163 - sum of all load accesses
164 Total stores
166 - sum of all store accesses
167 Store Reference - L1Hit, L1Miss
169 L1Hit - store accesses that hit L1
170 L1Miss - store accesses that missed L1
171 Core Load Hit - FB, L1, L2
173 - count of load hits in FB (Fill Buffer), L1 and L2 cache
174 LLC Load Hit - LlcHit, LclHitm
176 - count of LLC load accesses, includes LLC hits and LLC HITMs
177 RMT Load Hit - RmtHit, RmtHitm
179 - count of remote load accesses, includes remote hits and remote HITMs
180 Load Dram - Lcl, Rmt
182 - count of local and remote DRAM accesses
183 For each offset in the 2) list we display following data:
185 HITM - Rmt, Lcl
187 - % of Remote/Local HITM accesses for given offset within cacheline
188 Store Refs - L1 Hit, L1 Miss
190 - % of store accesses that hit/missed L1 for given offset within cacheline
191 Data address - Offset
193 - offset address
194 Pid
196 - pid of the process responsible for the accesses
197 Tid
199 - tid of the process responsible for the accesses
200 Code address
202 - code address responsible for the accesses
203 cycles - rmt hitm, lcl hitm, load
205 - sum of cycles for given accesses - Remote/Local HITM and generic load
206 cpu cnt
208 - number of cpus that participated on the access
209 Symbol
211 - code symbol related to the 'Code address' value
212 Shared Object
214 - shared object name related to the 'Code address' value
215 Source:Line
217 - source information related to the 'Code address' value
218 Node
220 - nodes participating on the access (see NODE INFO section)
221
223 The Node field displays nodes that accesses given cacheline offset. Its
224 output comes in 3 flavors: - node IDs separated by , - node IDs with
225 stats for each ID, in following format: Node{cpus %hitms %stores} -
226 node IDs with list of affected CPUs in following format: Node{cpu list}
227 User can switch between above flavors with -N option or use n key to
229 interactively switch in TUI mode.
230
232 User can specify how to sort offsets for cacheline.
233 Following fields are available and governs the final output fields set
235 for cacheline offsets output:
236 tid - coalesced by process TIDs
238 pid - coalesced by process PIDs
239 iaddr - coalesced by code address, following fields are displayed:
240 Code address, Code symbol, Shared Object, Source line
241 dso - coalesced by shared object
242 By default the coalescing is setup with pid,iaddr.
244
246 The stdio output displays data on standard output.
247 Following tables are displayed: Trace Event Information - overall
249 statistics of memory accesses
250 Global Shared Cache Line Event Information
252 - overall statistics on shared cachelines
253 Shared Data Cache Line Table
255 - list of most expensive cachelines
256 Shared Cache Line Distribution Pareto
258 - list of all accessed offsets for each cacheline
259
261 The TUI output provides interactive interface to navigate through
262 cachelines list and to display offset details.
263 For details please refer to the help window by pressing ? key.
265
267 Although Don Zickus, Dick Fowles and Joe Mario worked together to get
268 this implemented, we got lots of early help from Arnaldo Carvalho de
269 Melo, Stephane Eranian, Jiri Olsa and Andi Kleen.
270
272 Check Joe’s blog on c2c tool for detailed use case explanation:
273 https://joemario.github.io/blog/2016/09/01/c2c-blog/
274
276 perf-record(1), perf-mem(1)
277perf 11/22/2021 PERF-C2C(1)