1HWLOC(7) hwloc HWLOC(7)
2
6 hwloc - General information about hwloc ("hardware locality").
7
9 hwloc provides command line tools and a C API to obtain the hierarchi‐
10 cal map of key computing elements, such as: NUMA memory nodes, shared
11 caches, processor sockets, processor cores, and processor "threads".
12 hwloc also gathers various attributes such as cache and memory informa‐
13 tion, and is portable across a variety of different operating systems
14 and platforms.
15 Definitions
17 Hwloc has some specific definitions for terms that are used in this man
18 page and other hwloc documentation.
19 Hwloc CPU set:
21 A set of processors included in an hwloc object, expressed as a
22 bitmask indexed by the physical numbers of the CPUs (as announced
23 by the OS). The hwloc definition of "CPU set" does not carry any
24 the same connotations as Linux's "CPU set" (e.g., process affin‐
25 ity, etc.).
26 Linux CPU set:
28 See http://www.mjmwired.net/kernel/Documentation/cpusets.txt for a
29 discussion of Linux CPU sets. A super-short-ignoring-many-details
30 description (taken from that page) is:
31 "Cpusets provide a mechanism for assigning a set of CPUs and Mem‐
34 ory Nodes to a set of tasks."
35 Linux Cgroup:
37 See http://www.mjmwired.net/kernel/Documentation/cgroups.txt for a
38 discussion of Linux control groups. A super-short-ignoring-many-
39 details description (taken from that page) is:
40 "Control Groups provide a mechanism for aggregating/partitioning
43 sets of tasks, and all their future children, into hierarchical
44 groups with specialized behaviour."
45 To be clear, hwloc supports all of the above concepts. It is simply
47 worth noting that they are 3 different things.
48 Location Specification
50 Locations refer to specific regions within a topology. Before reading
51 the rest of this man page, it may be useful to read lstopo(1) and/or
52 run lstopo on your machine to see the reported topology tree. Seeing
53 and understanding a topology tree will definitely help in understanding
54 the concepts that are discussed below.
55 Locations can be specified in multiple ways:
57 Tuples: Tuples of hwloc "objects" and associated indexes can be spec‐
59 ified in the form object:index. Hwloc objects represent
60 types of mapped items (e.g., sockets, cores, etc.) in a
61 topology tree; indexes are non-negative integers that specify
62 a unique physical object in a topology tree. Both concepts
63 are described in detail, below.
64 Chaining multiple tuples together in the more general form
66 object1:index[.object2:index2[...]] is permissable. While
67 the first tuple's object may appear anywhere in the topology,
68 the Nth tuple's object must have a shallower topology depth
69 than the (N+1)th tuple's object. Put simply: as you move
70 right in a tuple chain, objects must go deeper in the topol‐
71 ogy tree. When using logical indexes (which is the default),
72 indexes specified in chained tuples are relative to the scope
73 of the parent object. For example, "socket:0.core:1" refers
74 to the second core in the first socket. When using OS/physi‐
75 cal indexes, the first object matching the given index is
76 used.
77 Hex: Locations can also be specified as hexidecimal bitmasks pre‐
79 fixed with "0x". Commas must be used to separate the hex
80 digits into blocks of 8, such as "0xffc0140,0x00020110".
81 Leading zeros in each block do not need to be specified. For
82 example, "0xffc0140,0x20110" is equivalent to the prior exam‐
83 ple, and "0x0000000f" is exactly equivalent to "0xf". Inter‐
84 mediate blocks of 8 digits that are all zeoro can be left
85 empty; "0xff0,,0x13" is equivalent to
86 "0xff0,0x00000000,0x13". If the location is prefixed with
87 the special string "0xf...f", then all unspecified bits are
88 set (as if the set were infinite). For example, "0xf...f,0x1"
89 sets both the first bit and all bits starting with the 33rd.
90 The string "0xf...f" -- with no other specified values --
91 sets all bits.
92 I/O devices:
94 Locations may also be a PCI or OS object. The corresponding
95 value is the set of CPUs that are close to the physical
96 device. For example, "pci=02:03.1" is equivalent to the set
97 of processors that are close to the hostbridge above PCI
98 device with bus ID "02:03.1". "os=eth0" is equivalent to all
99 processors close to the network interface whose software name
100 is "eth0".
101 Multiple locations can be specified on the hwloc-bind command line
103 (delimited by whitespace); the first token of the execution command is
104 assumed to either follow "--" (if specified) or the first token that is
105 unrecognized as a location.
106 By default, if multiple locations are specified, they are added, mean‐
108 ing that the binding will be wider in the sense that the process may
109 run on more objects.
110 If prefixed with "~", the given location will be cleared instead of
112 added to the current list of locations. If prefixed with "x", the
113 given location will be and'ed instead of added to the current list. If
114 prefixed with "^", the given location will be xor'ed.
115 "all" and "root" are a special location consisting in the entire cur‐
117 rent topology. More complex operations may be performed by using
118 hwloc-calc to compute intermediate values.
119 Hwloc Objects
121 Objects can be any of the following strings (listed from "biggest" to
122 "smallest"):
123 machine A set of processors and memory.
125 node A NUMA node; a set of processors around memory which the pro‐
127 cessors can directly access.
128 socket Typically a physical package or chip, it is a grouping of one
130 or more processors.
131 core A single, physical processing unit which may still contain
133 multiple logical processors, such as hardware threads.
134 pu Short for processor unit (not process!). The smallest physi‐
136 cal execution unit that hwloc recognizes. For example, there
137 may be multiple PUs on a core (e.g., hardware threads).
138 The additional system type can be used when several machines form an
140 overall single system image (SSI), such as Kerrighed.
141 Finally, note that an object can be denoted by its numeric "depth" in
143 the topology graph.
144 Hwloc Indexes
146 Indexes are integer values that uniquely specify a given object of a
147 specific type. Indexes can be expressed either as logical values or
148 physical values. Most hwloc utilities accept logical indexes by
149 default. Passing --physical switches to physical/OS indexes. Both
150 logical and physical indexes are described on this man page.
151 Logical indexes are relative to the object order in the output from the
153 lstopo command. They always start with 0 and increment by 1 for each
154 successive object.
155 Physical indexes are how the operating system refers to objects. Note
157 that while physical indexes are non-negative integer values, the hard‐
158 ware and/or operating system may choose arbitrary values -- they may
159 not start with 0, and successive objects may not have consecutive val‐
160 ues.
161 For example, if the first few lines of lstopo -p output are the follow‐
163 ing:
164 Machine (47GB)
166 NUMANode P#0 (24GB) + Socket P#0 + L3 (12MB)
167 L2 (256KB) + L1 (32KB) + Core P#0 + PU P#0
168 L2 (256KB) + L1 (32KB) + Core P#1 + PU P#0
169 L2 (256KB) + L1 (32KB) + Core P#2 + PU P#0
170 L2 (256KB) + L1 (32KB) + Core P#8 + PU P#0
171 L2 (256KB) + L1 (32KB) + Core P#9 + PU P#0
172 L2 (256KB) + L1 (32KB) + Core P#10 + PU P#0
173 NUMANode P#1 (24GB) + Socket P#1 + L3 (12MB)
174 L2 (256KB) + L1 (32KB) + Core P#0 + PU P#0
175 L2 (256KB) + L1 (32KB) + Core P#1 + PU P#0
176 L2 (256KB) + L1 (32KB) + Core P#2 + PU P#0
177 L2 (256KB) + L1 (32KB) + Core P#8 + PU P#0
178 L2 (256KB) + L1 (32KB) + Core P#9 + PU P#0
179 L2 (256KB) + L1 (32KB) + Core P#10 + PU P#0
180 In this example, the first core on the second socket is logically num‐
182 ber 6 (i.e., logically the 7th core, starting from 0). Its physical
183 index is 0, but note that another core also has a physical index of 0.
184 Hence, physical indexes may only be relevant within the scope of their
185 parent (or set of ancestors). In this example, to uniquely identify
186 logical core 6 with physical indexes, you must specify (at a minimum)
187 both a socket and a core: socket 1, core 0.
188 Index values, regardless of whether they are logical or physical, can
190 be expressed in several different forms (where X, Y, and N are positive
191 integers):
192 X The object with index value X.
194 X-Y All the objects with index values >= X and <= Y.
196 X- All the objects with index values >= X.
198 X:N N objects starting with index X, possibly wrapping around the
200 end of the level.
201 all A special index value indicating all valid index values.
203 odd A special index value indicating all valid odd index values.
205 even A special index value indicating all valid even index values.
207 REMEMBER: hwloc's command line tools accept logical indexes for loca‐
209 tion values by default. Use --physical and --logical to switch from
210 one mode to another.
211
213 Hwloc's command line tool documentation: lstopo(1), hwloc-bind(1),
214 hwloc-calc(1), hwloc-distrib(1), hwloc-ps(1).
215 Hwloc has many C API functions, each of which have their own man page.
217 Some top-level man pages are also provided, grouping similar functions
218 together. A few good places to start might include: hwlocal‐
219 ity_objects(3), hwlocality_types(3), hwlocality_creation(3), hwlocal‐
220 ity_cpuset(3), hwlocality_information(3), and hwlocality_binding(3).
221 For a listing of all available hwloc man pages, look at all "hwloc*"
223 files in the man1 and man3 directories.
2241.5 Jul 30, 2012 HWLOC(7)