1PQOS(8) System Manager's Manual PQOS(8)
2
6 pqos, pqos-msr, pqos-os - Intel(R) Resource Director Technology/AMD
7 PQoS monitoring and control tool
8
10 pqos [OPTIONS]...
11
13 Intel(R) Resource Director Technology/AMD PQoS is designed to monitor
14 and manage cpu resources and improve performance of applications and
15 virtual machines.
16 Intel(R) Resource Director Technology/AMD PQoS includes monitoring and
18 control technologies. Monitoring technologies include CMT (Cache Moni‐
19 toring Technology), which monitors occupancy of last level cache, and
20 MBM (Memory Bandwidth Monitoring). Control technologies include CAT
21 (Cache Allocation Technology), CDP (Code Data Prioritization) and MBA
22 (Memory Bandwidth Allocation).
23 pqos supports CMT and MBM on a per core or hardware thread basis. MBM
25 supports two types of events reporting local and remote memory band‐
26 width.
27 pqos-msr and pqos-os are simple pqos wrapper scripts that automatically
28 select the MSR or OS/Kernel library interface to program the technolo‐
29 gies.
30 Please see the -I option below for more information.
31 For hardware information please refer to the README located on:
33 https://github.com/intel/intel-cmt-cat/blob/master/README
34
36 pqos options are as follow:
37 -h, --help
39 show help
40 -v, --verbose
42 verbose mode
43 -V, --super-verbose
45 super-verbose mode
46 -l FILE, --log-file=FILE
48 log messages into selected log FILE
49 -s, --show
51 show the current allocation and monitoring configuration
52 -d, --display
54 display supported Intel(R) Resource Director Technology/AMD PQoS
55 capabilities
56 -D, --display-verbose
58 display supported Intel(R) Resource Director Technology/AMD PQoS
59 capabilities in verbose mode
60 -f FILE, --config-file=FILE
62 load commands from selected configuration FILE
63 -e CLASSDEF, --alloc-class=CLASSDEF
65 define the allocation classes on all CPU sockets. CLASSDEF for‐
66 mat is "TYPE:ID=DEFINITION;...".
67 define classes for selected CPU resources. CLASSDEF format is
68 "TYPE[@RESOURCE_ID]:ID=DEFINITION;...".
69 For CAT, TYPE is "llc" for the last level cache (aka l3) and
70 "l2" for level 2 cache, ID is a CLOS number and DEFINITION is a
71 bitmask.
72 For MBA, TYPE is "mba", ID is a CLOS number and DEFINITION is a
73 value between 1 and 100 representing the percentage of available
74 memory bandwidth.
75 For MBA CTRL, TYPE is "mba_max", ID is a CLOS number and DEFINI‐
76 TION is a value representing the requested memory bandwidth
77 specified in MBps.
78 RESOURCE_ID is a unique number that can represent a socket or
79 l2/l3 cache identifier. The RESOURCE_ID for each logical CPU can
80 be found using "pqos -s"
81 Note: When L2/L3 CDP is on, ID can be postfixed with 'D' for
82 data or 'C' for code.
83 Note: L2/L3 CDP is available on selected CPUs only.
84 Note: MBA CTRL is supported only by the OS interface and re‐
85 quires Linux and kernel version 4.18 or newer.
86 Some examples:
87 "-e llc:0=0xffff;llc:1=0x00ff"
88 "-e llc@0-1:2=0xff00;l2:2=0x3f;l2@2:1=0xf"
89 "-e llc:0d=0xfff;llc:0c=0xfff00"
90 "-e l2:0d=0xf;l2:0c=0xc"
91 "-e mba:1=30;mba@1:3=80"
92 "-e mba_max:1=6000;mba_max@1:3=10000"
93 Note:
94 "-e l2:2=0x3f" means that COS2 for all L2 cache clusters
95 is changed to 0x3f.
96 "-e l2@2:1=0xf" means that COS1 for L2 cache cluster 2 is
97 changed to 0xf.
98 "-e mba:1=30" means that COS1, on all sockets, can uti‐
99 lize up to 30% of available memory bandwidth.
100 "-e mba_max:1=6000" means that COS1, on all sockets, can
101 utilize up to 6000 MBps of memory bandwidth.
102 -a CLASS2ID, --alloc-assoc=CLASS2ID
104 associate allocation classes with cores or processes. CLASS2ID
105 format is "TYPE:ID=CORE_LIST;..." or "TYPE:ID=TASK_LIST;...".
106 For COS association required for CAT or MBA, TYPE is "cos",
107 "llc", "core" (for COS-core association) or "pid" (for COS-
108 process association) and ID is a class number. CORE_LIST is
109 comma or dash separated list of cores. TASK_LIST is comma or
110 dash separated list of process/task ID's.
111 For example:
112 "-a cos:0=0,2,4,6-10;cos:1=1;" associates cores 0, 2, 4,
113 6, 7, 8, 9, 10 with CAT class 0 and core 1 with class 1.
114 "-a llc:0=0,2,4,6-10;llc:1=1;" associates cores 0, 2, 4,
115 6, 7, 8, 9, 10 with CAT class 0 and core 1 with class 1.
116 "-a core:0=0,2,4,6-10;core:1=1;" associates cores 0, 2,
117 4, 6, 7, 8, 9, 10 with CAT class 0 and core 1 with class
118 1.
119 "-I -a pid:0=3543,7643,4556;pid:1=7644;" associates
120 process ID 3543, 7643, 4556 with CAT class 0 and process
121 ID 7644 with class 1.
122 Notes:
123 "llc" TYPE label is considered deprecated, please use
124 "cos" or "core" instead.
125 The -I option must be used for PID association.
126 -R [CONFIG[,CONFIG]], --alloc-reset[=CONFIG[,CONFIG]]
128 reset allocation setting (L3 CAT, L2 CAT, MBA) and reconfigure
129 L3 CDP. CONFIG is one of the following options:
130 l3cdp-on sets L3 CDP on
131 l3cdp-off sets L3 CDP off
132 l3cdp-any keeps current L3 CDP setting (default)
133 l2cdp-on sets L2 CDP on
134 l2cdp-off sets L2 CDP off
135 l2cdp-any keeps current L2 CDP setting (default)
136 mbaCtrl-on sets MBA CTRL on
137 mbaCtrl-off sets MBA CTRL off
138 mbaCtrl-any keeps current MBA CTRL setting (default)
139 -m EVTCORES, --mon-core=EVTCORES
141 select the cores and events for monitoring, EVTCORES format is
142 "EVENT:CORE_LIST". Valid EVENT settings are:
143 - "llc" for CMT (LLC occupancy)
144 - "mbr" for MBR (remote memory bandwidth)
145 - "mbl" for MBL (local memory bandwidth)
146 - "mbt" for MBT (total memory bandwidth)
147 - "all" or "" for all detected event types (except MBT)
148 CORE_LIST is comma or dash separated list of cores.
149 Example "-m all:0,2,4-10;llc:1,3;mbr:11-12".
150 Core statistics can be grouped by enclosing the core list in
151 square brackets.
152 Example "-m llc:[0-3];all:[4,5,6];mbr:[0-3],7,8".
153 -p [EVTPIDS], --mon-pid[=EVTPIDS]
155 select top 10 most active (CPU utilizing) process ids to monitor
156 or select the process ids and events to monitor, EVTPIDS format
157 is "EVENT:PID_LIST".
158 See -m option for valid EVENT settings. PID_LIST is comma sepa‐
159 rated list of process ids.
160 Examples:
161 "-p llc:22,25673"
162 "-p all:892,4588-4592"
163 Process's IDs can be grouped by enclosing them in square brack‐
165 ets.
166 Examples:
167 "-p llc:[22,25673]"
168 "-p all:892,[4588-4592]"
169 Note:
171 Requires Linux and kernel versions 4.10 and newer.
172 The -I option must be used for PID monitoring.
173 It is not possible to track both processes and cores at
174 the same time.
175 --mon-uncore[=EVTUNCORE]
178 select sockets and uncore events for monitoring, EVTUNCORE for‐
179 mat is 'EVENT:SOCKET_LIST. Socket's IDs can be grouped by en‐
180 closing them in square brackets.
181 Examples:
182 "--mon-uncore"
183 "--mon-uncore=all:0"
184 Note: It is not possible to track both sockets and cores at the
186 same time.
187 -T, --mon-top
189 enable top like monitoring output sorted by highest LLC occu‐
190 pancy
191 -o FILE, --mon-file FILE
193 select output FILE to store monitored data in, the default is
194 'stdout'
195 -u TYPE, --mon-file-type=TYPE
197 select the output format TYPE for monitored data. Supported TYPE
198 settings are: "text" (default), "xml" and "csv".
199 -i INTERVAL, --mon-interval=INTERVAL
201 define monitoring sampling INTERVAL in 100ms units, 1=100ms, de‐
202 fault 10=10x100ms=1s
203 -t SECONDS, --mon-time=SECONDS
205 define monitoring time in seconds, use 'inf' or 'infinite' for
206 infinite monitoring. Use CTRL+C to stop monitoring at any time.
207 -r, --mon-reset
209 reset monitoring and use all RMID's and cores in the system
210 --disable-mon-ipc
212 Disable IPC monitoring
213 --disable-mon-llc_miss
215 Disable LLC misses monitoring
216 -H, --profile-list
218 list supported allocation profiles
219 -c PROFILE, --profile-set=PROFILE
221 select a PROFILE from predefined allocation classes, use -H to
222 list available profiles
223 -I, --iface-os
225 set the library interface to use the kernel implementation. If
226 not set the default implementation is to program the MSR's di‐
227 rectly.
228 --iface=INTERFACE
230 set the library interface to automatically detected one
231 ('auto'), MSR ('msr') or kernel interface ('os'). INTERFACE can
232 be set to either 'auto' (default), 'msr' or 'os'. If automatic
233 detection is selected ('auto'), it:
234 1) Takes RDT_IFACE environment variable into account if this
235 variable is set
236 2) Selects OS interface if the kernel interface is supported
237 3) Selects MSR interface otherwise
238
240 CMT, MBM and CAT are configured using Model Specific Registers (MSRs).
241 The pqos software executes in user space, and access to the MSRs is ob‐
242 tained through a standard Linux* interface. The msr file interface is
243 protected and requires root privileges. The msr driver might not be
244 auto-loaded and on some modular kernels the driver may need to be
245 loaded manually:
246 For Linux:
248 sudo modprobe msr
249 For FreeBSD:
251 sudo kldload cpuctl
252 Interface enforcement:
254 If you require system wide interface enforcement you can do so by set‐
255 ting the "RDT_IFACE" environment variable.
256
258 msr(4)
259
261 pqos was written by Tomasz Kantecki <tomasz.kantecki@intel.com>, Marcel
262 Cornu <marcel.d.cornu@intel.com>, Aaron Hetherington <aaron.hethering‐
263 ton@intel.com>
264 This is free software; see the source for copying conditions. There is
266 NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR
267 PURPOSE.
268 Apr 19, 2022 PQOS(8)