1HWLOC(7)                             hwloc                            HWLOC(7)
2
3
4

NAME

6       hwloc - General information about hwloc ("hardware locality").
7

DESCRIPTION

9       hwloc  provides command line tools and a C API to obtain the hierarchi‐
10       cal map of key computing elements, such as: NUMA memory  nodes,  shared
11       caches,  processor  packages, processor cores, and processor "threads".
12       hwloc also gathers various attributes such as cache and memory informa‐
13       tion,  and  is portable across a variety of different operating systems
14       and platforms.
15
16   Definitions
17       hwloc has some specific definitions for terms that are used in this man
18       page and other hwloc documentation.
19
20       hwloc CPU set:
21            A  set  of  processors included in an hwloc object, expressed as a
22            bitmask indexed by the physical numbers of the CPUs (as  announced
23            by  the OS).  The hwloc definition of "CPU set" does not carry any
24            of the same connotations  as  Linux's  "CPU  set"  (e.g.,  process
25            affinity, cgroup, etc.).
26
27       hwloc node set:
28            A  set  of  NUMA memory nodes near an hwloc object, expressed as a
29            bitmask indexed by the physical numbers  of  the  NUMA  nodes  (as
30            announced by the OS).
31
32       Linux CPU set:
33            See http://www.mjmwired.net/kernel/Documentation/cpusets.txt for a
34            discussion of Linux CPU sets.  A super-short-ignoring-many-details
35            description (taken from that page) is:
36
37             "Cpusets provide a mechanism for assigning a set of CPUs and Mem‐
38            ory Nodes to a set of tasks."
39
40       Linux Cgroup:
41            See http://www.mjmwired.net/kernel/Documentation/cgroups.txt for a
42            discussion  of Linux control groups.  A super-short-ignoring-many-
43            details description (taken from that page) is:
44
45             "Control Groups provide a mechanism for  aggregating/partitioning
46            sets  of  tasks,  and all their future children, into hierarchical
47            groups with specialized behaviour."
48
49       To be clear, hwloc supports all of the above concepts.   It  is  simply
50       worth noting that they are different things.
51
52   Location Specification
53       Locations  refer to specific regions within a topology.  Before reading
54       the rest of this man page, it may be useful to  read  lstopo(1)  and/or
55       run  lstopo  on your machine to see the reported topology tree.  Seeing
56       and understanding a topology tree will definitely help in understanding
57       the concepts that are discussed below.
58
59       Locations can be specified in multiple ways:
60
61       Tuples:   Tuples of hwloc "objects" and associated indexes can be spec‐
62                 ified in the  form  object:index.   hwloc  objects  represent
63                 types  of  mapped  items  (e.g.,  packages, cores, etc.) in a
64                 topology tree; indexes are non-negative integers that specify
65                 a  unique  physical object in a topology tree.  Both concepts
66                 are described in detail, below.
67
68                 Indexes may also be specified as ranges.  x-y enumerates from
69                 index x to y.  x:y enumerates y objects starting from index x
70                 (wrapping around the end of the index range if  needed).   x-
71                 enumerates  all objects starting from index x.  all, odd, and
72                 even are also supported for  listing  all  objects,  or  only
73                 those with odd or even indexes.
74
75                 Chaining  multiple  tuples  together in the more general form
76                 object1:index[.object2:index2[...]]  is  permissable.   While
77                 the first tuple's object may appear anywhere in the topology,
78                 the Nth tuple's object must have a shallower  topology  depth
79                 than  the  (N+1)th  tuple's  object.  Put simply: as you move
80                 right in a tuple chain, objects must go deeper in the  topol‐
81                 ogy tree.  When using logical indexes (which is the default),
82                 indexes specified in chained tuples are relative to the scope
83                 of the parent object.  For example, "package:0.core:1" refers
84                 to the second core in the first package.
85
86                 When using OS/physical indexes, the first object matching the
87                 given index is used.
88
89                 PCI  and  OS devices may also be designed using their identi‐
90                 fier.  For example, "pci=02:03.1" is the PCI device with  bus
91                 ID "02:03.1".  "os=eth0" is the network interface whose soft‐
92                 ware name is "eth0".  PCI devices may also be filtered  based
93                 on   their   vendor   and/or   device   IDs,   for   instance
94                 "pci[15b3:]:2" for the third Mellanox PCI device  (vendor  ID
95                 0x15b3).  OS devices may also be filtered based on their sub‐
96                 type, for instance "os[gpu]:all" for all GPU OS devices.
97
98       Hex:      For tools that manipulate object as sets (e.g. hwloc-calc and
99                 hwloc-bind),  locations  can also be specified as hexidecimal
100                 bitmasks prefixed with "0x".  Commas must be used to separate
101                 the    hex    digits    into    blocks    of   8,   such   as
102                 "0xffc0140,0x00020110".  Leading zeros in each block  do  not
103                 need  to  be  specified.  For example, "0xffc0140,0x20110" is
104                 equivalent to the prior example, and "0x0000000f" is  exactly
105                 equivalent  to  "0xf".   Intermediate blocks of 8 digits that
106                 are all zeoro can be left empty; "0xff0,,0x13" is  equivalent
107                 to "0xff0,0x00000000,0x13".  If the location is prefixed with
108                 the special string "0xf...f", then all unspecified  bits  are
109                 set (as if the set were infinite). For example, "0xf...f,0x1"
110                 sets both the first bit and all bits starting with the  33rd.
111                 The  string  "0xf...f"  --  with no other specified values --
112                 sets all bits.
113
114       "all" and "root" are special locations consisting in the root object in
115       tree. It contains the entire current topology.
116
117       Some  tools  directly  operate  on  these  objects (e.g. hwloc-info and
118       hwloc-annotate).  They do not  support  hexadecimal  locations  because
119       each  location may correspond to multiple objects.  For instance, there
120       can be exactly one L3 cache per package and NUMA node, which means it's
121       the  same  location.   If  multiple locations are given on the command-
122       line, these tools will operation on each location individually and con‐
123       secutively.
124
125       Some other tools internally manipulate objects as sets (e.g. hwloc-calc
126       and hwloc-bind).  They translate each input location into a hexidecimal
127       location.   When I/O or Misc objects are used, they are translated into
128       the set of processors (or NUMA nodes)  that  are  close  to  the  given
129       object  (because  I/O or Misc objects do not contain processors or NUMA
130       nodes).
131
132       If multiple locations are specified on the command-line  (delimited  by
133       whitespace),  they  are  combined  (the overall location is wider).  If
134       prefixed with "~", the given location will be cleared instead of  added
135       to  the  current  list  of  locations.  If prefixed with "x", the given
136       location will be and'ed instead of added to the current list.  If  pre‐
137       fixed with "^", the given location will be xor'ed.
138
139       More complex operations may be performed by using hwloc-calc to compute
140       intermediate values.
141
142   hwloc Objects
143       Objects in tuples can be any of  the  following  strings  (listed  from
144       "biggest" to "smallest"):
145
146       machine   A set of processors and memory.
147
148       numanode  A NUMA node; a set of processors around memory which the pro‐
149                 cessors can directly access.   If  hbm  is  used  instead  of
150                 numanode in locations, command-line tools only consider high-
151                 bandwidth memory nodes such as Intel Xeon Phi MCDRAM.
152
153       package   Typically a physical package or chip, that goes into a  pack‐
154                 age, it is a grouping of one or more processors.
155
156       l1cache ... l5cache
157                 A data (or unified) cache.
158
159       l1icache ... l3icache
160                 An instruction cache.
161
162       core      A  single,  physical  processing unit which may still contain
163                 multiple logical processors, such as hardware threads.
164
165       pu        Short for processor unit (not process!).  The smallest physi‐
166                 cal execution unit that hwloc recognizes.  For example, there
167                 may be multiple PUs on a core (e.g., hardware threads).
168
169       osdev, pcidev, bridge, and misc may also be  used  to  specify  special
170       devices  although  some  of  them have dedicated identification ways as
171       explained in Location Specification.
172
173       Finally, note that an object can be denoted by its numeric  "depth"  in
174       the topology graph.
175
176   hwloc Indexes
177       Indexes  are  integer  values that uniquely specify a given object of a
178       specific type.  Indexes can be expressed either as  logical  values  or
179       physical  values.   Most  hwloc  utilities  accept  logical  indexes by
180       default.  Passing --physical switches  to  physical/OS  indexes.   Both
181       logical and physical indexes are described on this man page.
182
183       Logical indexes are relative to the object order in the output from the
184       lstopo command.  They always start with 0 and increment by 1  for  each
185       successive object.
186
187       Physical  indexes are how the operating system refers to objects.  Note
188       that while physical indexes are non-negative integer values, the  hard‐
189       ware  and/or  operating  system may choose arbitrary values -- they may
190       not start with 0, and successive objects may not have consecutive  val‐
191       ues.
192
193       For example, if the first few lines of lstopo -p output are the follow‐
194       ing:
195
196         Machine (47GB)
197           NUMANode P#0 (24GB) + Package P#0 + L3 (12MB)
198             L2 (256KB) + L1 (32KB) + Core P#0 + PU P#0
199             L2 (256KB) + L1 (32KB) + Core P#1 + PU P#0
200             L2 (256KB) + L1 (32KB) + Core P#2 + PU P#0
201             L2 (256KB) + L1 (32KB) + Core P#8 + PU P#0
202             L2 (256KB) + L1 (32KB) + Core P#9 + PU P#0
203             L2 (256KB) + L1 (32KB) + Core P#10 + PU P#0
204           NUMANode P#1 (24GB) + Package P#1 + L3 (12MB)
205             L2 (256KB) + L1 (32KB) + Core P#0 + PU P#0
206             L2 (256KB) + L1 (32KB) + Core P#1 + PU P#0
207             L2 (256KB) + L1 (32KB) + Core P#2 + PU P#0
208             L2 (256KB) + L1 (32KB) + Core P#8 + PU P#0
209             L2 (256KB) + L1 (32KB) + Core P#9 + PU P#0
210             L2 (256KB) + L1 (32KB) + Core P#10 + PU P#0
211
212       In this example, the first core on the second package is logically num‐
213       ber  6  (i.e.,  logically the 7th core, starting from 0).  Its physical
214       index is 0, but note that another core also has a physical index of  0.
215       Hence,  physical indexes may only be relevant within the scope of their
216       parent (or set of ancestors).  In this example,  to  uniquely  identify
217       logical  core  6 with physical indexes, you must specify (at a minimum)
218       both a package and a core: package 1, core 0.
219
220       Index values, regardless of whether they are logical or  physical,  can
221       be expressed in several different forms (where X, Y, and N are positive
222       integers):
223
224       X         The object with index value X.
225
226       X-Y       All the objects with index values >= X and <= Y.
227
228       X-        All the objects with index values >= X.
229
230       X:N       N objects starting with index X, possibly wrapping around the
231                 end of the level.
232
233       all       A special index value indicating all valid index values.
234
235       odd       A special index value indicating all valid odd index values.
236
237       even      A special index value indicating all valid even index values.
238
239       REMEMBER:  hwloc's  command line tools accept logical indexes for loca‐
240       tion values by default.  Use --physical and --logical  to  switch  from
241       one mode to another.
242

SEE ALSO

244       hwloc's  command  line  tool  documentation:  lstopo(1), hwloc-bind(1),
245       hwloc-calc(1), hwloc-distrib(1), hwloc-ps(1).
246
247       hwloc has many C API functions, each of which have their own man  page.
248       Some  top-level man pages are also provided, grouping similar functions
249       together.   A  few  good  places  to  start  might  include:   hwlocal‐
250       ity_objects(3),  hwlocality_types(3),  hwlocality_creation(3), hwlocal‐
251       ity_cpuset(3), hwlocality_information(3), and hwlocality_binding(3).
252
253       For a listing of all available hwloc man pages, look  at  all  "hwloc*"
254       files in the man1 and man3 directories.
255
256
257
2582.2.0                            Mar 30, 2020                         HWLOC(7)
Impressum