1HWLOC(7) hwloc HWLOC(7)
2
6 hwloc - General information about hwloc ("hardware locality").
7
9 hwloc provides command line tools and a C API to obtain the hierarchi‐
10 cal map of key computing elements, such as: NUMA memory nodes, shared
11 caches, processor sockets, processor cores, and processor "threads".
12 hwloc also gathers various attributes such as cache and memory informa‐
13 tion, and is portable across a variety of different operating systems
14 and platforms.
15 Definitions
17 Hwloc has some specific definitions for terms that are used in this man
18 page and other hwloc documentation.
19 Hwloc CPU set:
21 A set of processors included in an hwloc object, expressed as a
22 bitmask indexed by the physical numbers of the CPUs (as announced
23 by the OS). The hwloc definition of "CPU set" does not carry any
24 the same connotations as Linux's "CPU set" (e.g., process affin‐
25 ity, etc.).
26 Linux CPU set:
28 See http://www.mjmwired.net/kernel/Documentation/cpusets.txt for a
29 discussion of Linux CPU sets. A super-short-ignoring-many-details
30 description (taken from that page) is:
31 "Cpusets provide a mechanism for assigning a set of CPUs and Mem‐
34 ory Nodes to a set of tasks."
35 Linux Cgroup:
37 See http://www.mjmwired.net/kernel/Documentation/cgroups.txt for a
38 discussion of Linux control groups. A super-short-ignoring-many-
39 details description (taken from that page) is:
40 "Control Groups provide a mechanism for aggregating/partitioning
43 sets of tasks, and all their future children, into hierarchical
44 groups with specialized behaviour."
45 To be clear, hwloc supports all of the above concepts. It is simply
47 worth noting that they are 3 different things.
48 Location Specification
50 Locations refer to specific regions within a topology. Before reading
51 the rest of this man page, it may be useful to read lstopo(1) and/or
52 run lstopo on your machine to see the reported topology tree. Seeing
53 and understanding a topology tree will definitely help in understanding
54 the concepts that are discussed below.
55 Locations can be specified in multiple ways:
57 Tuples: Tuples of hwloc "objects" and associated indexes can be spec‐
59 ified in the form object:index. Hwloc objects represent
60 types of mapped items (e.g., sockets, cores, etc.) in a
61 topology tree; indexes are non-negative integers that specify
62 a unique physical object in a topology tree. Both concepts
63 are described in detail, below.
64 Chaining multiple tuples together in the more general form
66 object1:index[.object2:index2[...]] is permissable. While
67 the first tuple's object may appear anywhere in the topology,
68 the Nth tuple's object must have a shallower topology depth
69 than the (N+1)th tuple's object. Put simply: as you move
70 right in a tuple chain, objects must go deeper in the topol‐
71 ogy tree. When using logical indexes (which is the default),
72 indexes specified in chained tuples are relative to the scope
73 of the parent object. For example, "socket:0.core:1" refers
74 to the second core in the first socket. When using OS/physi‐
75 cal indexes, the first object matching the given index is
76 used.
77 Hex: Locations can also be specified as hexidecimal bitmasks pre‐
79 fixed with "0x". Commas must be used to separate the hex
80 digits into blocks of 8, such as "0xffc0140,0x00020110".
81 Leading zeros in each block do not need to be specified. For
82 example, "0xffc0140,0x20110" is equivalent to the prior exam‐
83 ple, and "0x0000000f" is exactly equivalent to "0xf". Inter‐
84 mediate blocks of 8 digits that are all zeoro can be left
85 empty; "0xff0,,0x13" is equivalent to
86 "0xff0,0x00000000,0x13". If the location is prefixed with
87 the special string "0xf...f", then all unspecified bits are
88 set (as if the set were infinite). For example, "0xf...f,0x1"
89 sets both the first bit and all bits starting with the 33rd.
90 The string "0xf...f" -- with no other specified values --
91 sets all bits.
92 Multiple locations can be specified on the hwloc-bind command line
94 (delimited by whitespace); the first token of the execution command is
95 assumed to either follow "--" (if specified) or the first token that is
96 unrecognized as a location.
97 By default, if multiple locations are specified, they are added, mean‐
99 ing that the binding will be wider in the sense that the process may
100 run on more objects.
101 If prefixed with "~", the given location will be cleared instead of
103 added to the current list of locations. If prefixed with "x", the
104 given location will be and'ed instead of added to the current list. If
105 prefixed with "^", the given location will be xor'ed.
106 "all" and "root" are a special location consisting in the entire cur‐
108 rent topology. More complex operations may be performed by using
109 hwloc-calc to compute intermediate values.
110 Hwloc Objects
112 Objects can be any of the following strings (listed from "biggest" to
113 "smallest"):
114 machine A set of processors and memory.
116 node A NUMA node; a set of processors around memory which the pro‐
118 cessors can directly access.
119 socket Typically a physical package or chip, it is a grouping of one
121 or more processors.
122 core A single, physical processing unit which may still contain
124 multiple logical processors, such as hardware threads.
125 pu Short for processor unit (not process!). The smallest physi‐
127 cal execution unit that hwloc recognizes. For example, there
128 may be multiple PUs on a core (e.g., hardware threads).
129 The additional system type can be used when several machines form an
131 overall single system image (SSI), such as Kerrighed.
132 Finally, note that an object can be denoted by its numeric "depth" in
134 the topology graph.
135 Hwloc Indexes
137 Indexes are integer values that uniquely specify a given object of a
138 specific type. Indexes can be expressed either as logical values or
139 physical values. Most hwloc utilities accept logical indexes by
140 default. Passing --physical switches to physical/OS indexes. Both
141 logical and physical indexes are described on this man page.
142 Logical indexes are relative to the object order in the output from the
144 lstopo command. They always start with 0 and increment by 1 for each
145 successive object.
146 Physical indexes are how the operating system refers to objects. Note
148 that while physical indexes are non-negative integer values, the hard‐
149 ware and/or operating system may choose arbitrary values -- they may
150 not start with 0, and successive objects may not have consecutive val‐
151 ues.
152 For example, if the first few lines of lstopo -p output are the follow‐
154 ing:
155 Machine (47GB)
157 NUMANode P#0 (24GB) + Socket P#0 + L3 (12MB)
158 L2 (256KB) + L1 (32KB) + Core P#0 + PU P#0
159 L2 (256KB) + L1 (32KB) + Core P#1 + PU P#0
160 L2 (256KB) + L1 (32KB) + Core P#2 + PU P#0
161 L2 (256KB) + L1 (32KB) + Core P#8 + PU P#0
162 L2 (256KB) + L1 (32KB) + Core P#9 + PU P#0
163 L2 (256KB) + L1 (32KB) + Core P#10 + PU P#0
164 NUMANode P#1 (24GB) + Socket P#1 + L3 (12MB)
165 L2 (256KB) + L1 (32KB) + Core P#0 + PU P#0
166 L2 (256KB) + L1 (32KB) + Core P#1 + PU P#0
167 L2 (256KB) + L1 (32KB) + Core P#2 + PU P#0
168 L2 (256KB) + L1 (32KB) + Core P#8 + PU P#0
169 L2 (256KB) + L1 (32KB) + Core P#9 + PU P#0
170 L2 (256KB) + L1 (32KB) + Core P#10 + PU P#0
171 In this example, the first core on the second socket is logically num‐
173 ber 6 (i.e., logically the 7th core, starting from 0). Its physical
174 index is 0, but note that another core also has a physical index of 0.
175 Hence, physical indexes may only be relevant within the scope of their
176 parent (or set of ancestors). In this example, to uniquely identify
177 logical core 6 with physical indexes, you must specify (at a minimum)
178 both a socket and a core: socket 1, core 0.
179 Index values, regardless of whether they are logical or physical, can
181 be expressed in several different forms (where X, Y, and N are positive
182 integers):
183 X The object with index value X.
185 X-Y All the objects with index values >= X and <= Y.
187 X- All the objects with index values >= X.
189 X:N N objects starting with index X, possibly wrapping around the
191 end of the level.
192 all A special index value indicating all valid index values.
194 odd A special index value indicating all valid odd index values.
196 even A special index value indicating all valid even index values.
198 REMEMBER: hwloc's command line tools accept logical indexes for loca‐
200 tion values by default. Use --physical and --logical to switch from
201 one mode to another.
202
204 Hwloc's command line tool documentation: lstopo(1), hwloc-bind(1),
205 hwloc-calc(1), hwloc-distrib(1), hwloc-ps(1).
206 Hwloc has many C API functions, each of which have their own man page.
208 Some top-level man pages are also provided, grouping similar functions
209 together. A few good places to start might include: hwlocal‐
210 ity_objects(3), hwlocality_types(3), hwlocality_creation(3), hwlocal‐
211 ity_cpuset(3), hwlocality_information(3), and hwlocality_binding(3).
212 For a listing of all available hwloc man pages, look at all "hwloc*"
214 files in the man1 and man3 directories.
2151.2 Apr 14, 2011 HWLOC(7)