1OSDMAPTOOL(8) Ceph OSDMAPTOOL(8)
2
6 osdmaptool - ceph osd cluster map manipulation tool
7
9 osdmaptool mapfilename [--print] [--createsimple numosd
10 [--pgbits bitsperosd ] ] [--clobber]
11 osdmaptool mapfilename [--import-crush crushmap]
12 osdmaptool mapfilename [--export-crush crushmap]
13 osdmaptool mapfilename [--upmap file] [--upmap-max max-optimizations]
14 [--upmap-deviation max-deviation] [--upmap-pool poolname]
15 [--save] [--upmap-active]
16 osdmaptool mapfilename [--upmap-cleanup] [--upmap file]
17
20 osdmaptool is a utility that lets you create, view, and manipulate OSD
21 cluster maps from the Ceph distributed storage system. Notably, it lets
22 you extract the embedded CRUSH map or import a new CRUSH map. It can
23 also simulate the upmap balancer mode so you can get a sense of what is
24 needed to balance your PGs.
25
27 --print
28 will simply make the tool print a plaintext dump of the map, af‐
29 ter any modifications are made.
30 --dump <format>
32 displays the map in plain text when <format> is 'plain', 'json'
33 if specified format is not supported. This is an alternative to
34 the print option.
35 --clobber
37 will allow osdmaptool to overwrite mapfilename if changes are
38 made.
39 --import-crush mapfile
41 will load the CRUSH map from mapfile and embed it in the OSD
42 map.
43 --export-crush mapfile
45 will extract the CRUSH map from the OSD map and write it to map‐
46 file.
47 --createsimple numosd [--pg-bits bitsperosd] [--pgp-bits bits]
49 will create a relatively generic OSD map with the numosd de‐
50 vices. If --pg-bits is specified, the initial placement group
51 counts will be set with bitsperosd bits per OSD. That is, the
52 pg_num map attribute will be set to numosd shifted by bitsper‐
53 osd. If --pgp-bits is specified, then the pgp_num map attribute
54 will be set to numosd shifted by bits.
55 --create-from-conf
57 creates an osd map with default configurations.
58 --test-map-pgs [--pool poolid] [--range-first <first> --range-last
60 <last>]
61 will print out the mappings from placement groups to OSDs. If
62 range is specified, then it iterates from first to last in the
63 directory specified by argument to osdmaptool. Eg: osdmaptool
64 --test-map-pgs --range-first 0 --range-last 2 osdmap_dir. This
65 will iterate through the files named 0,1,2 in osdmap_dir.
66 --test-map-pgs-dump [--pool poolid] [--range-first <first> --range-last
68 <last>]
69 will print out the summary of all placement groups and the map‐
70 pings from them to the mapped OSDs. If range is specified, then
71 it iterates from first to last in the directory specified by ar‐
72 gument to osdmaptool. Eg: osdmaptool --test-map-pgs-dump
73 --range-first 0 --range-last 2 osdmap_dir. This will iterate
74 through the files named 0,1,2 in osdmap_dir.
75 --test-map-pgs-dump-all [--pool poolid] [--range-first <first>
77 --range-last <last>]
78 will print out the summary of all placement groups and the map‐
79 pings from them to all the OSDs. If range is specified, then it
80 iterates from first to last in the directory specified by argu‐
81 ment to osdmaptool. Eg: osdmaptool --test-map-pgs-dump-all
82 --range-first 0 --range-last 2 osdmap_dir. This will iterate
83 through the files named 0,1,2 in osdmap_dir.
84 --test-random
86 does a random mapping of placement groups to the OSDs.
87 --test-map-pg <pgid>
89 map a particular placement group(specified by pgid) to the OSDs.
90 --test-map-object <objectname> [--pool <poolid>]
92 map a particular placement group(specified by objectname) to the
93 OSDs.
94 --test-crush [--range-first <first> --range-last <last>]
96 map placement groups to acting OSDs. If range is specified,
97 then it iterates from first to last in the directory specified
98 by argument to osdmaptool. Eg: osdmaptool --test-crush
99 --range-first 0 --range-last 2 osdmap_dir. This will iterate
100 through the files named 0,1,2 in osdmap_dir.
101 --mark-up-in
103 mark osds up and in (but do not persist).
104 --mark-out
106 mark an osd as out (but do not persist)
107 --mark-up <osdid>
109 mark an osd as up (but do not persist)
110 --mark-in <osdid>
112 mark an osd as in (but do not persist)
113 --tree Displays a hierarchical tree of the map.
115 --clear-temp
117 clears pg_temp and primary_temp variables.
118 --clean-temps
120 clean pg_temps.
121 --health
123 dump health checks
124 --with-default-pool
126 include default pool when creating map
127 --upmap-cleanup <file>
129 clean up pg_upmap[_items] entries, writing commands to <file>
130 [default: - for stdout]
131 --upmap <file>
133 calculate pg upmap entries to balance pg layout writing commands
134 to <file> [default: - for stdout]
135 --upmap-max <max-optimizations>
137 set max upmap entries to calculate [default: 10]
138 --upmap-deviation <max-deviation>
140 max deviation from target [default: 5]
141 --upmap-pool <poolname>
143 restrict upmap balancing to 1 pool or the option can be repeated
144 for multiple pools
145 --upmap-active
147 Act like an active balancer, keep applying changes until bal‐
148 anced
149 --adjust-crush-weight <osdid:weight>[,<osdid:weight>,<...>]
151 Change CRUSH weight of <osdid>
152 --save write modified osdmap with upmap or crush-adjust changes
154
156 To create a simple map with 16 devices:
157 osdmaptool --createsimple 16 osdmap --clobber
159 To view the result:
161 osdmaptool --print osdmap
163 To view the mappings of placement groups for pool 1:
165 osdmaptool osdmap --test-map-pgs-dump --pool 1
167 pool 1 pg_num 8
169 1.0 [0,2,1] 0
170 1.1 [2,0,1] 2
171 1.2 [0,1,2] 0
172 1.3 [2,0,1] 2
173 1.4 [0,2,1] 0
174 1.5 [0,2,1] 0
175 1.6 [0,1,2] 0
176 1.7 [1,0,2] 1
177 #osd count first primary c wt wt
178 osd.0 8 5 5 1 1
179 osd.1 8 1 1 1 1
180 osd.2 8 2 2 1 1
181 in 3
182 avg 8 stddev 0 (0x) (expected 2.3094 0.288675x))
183 min osd.0 8
184 max osd.0 8
185 size 0 0
186 size 1 0
187 size 2 0
188 size 3 8
189 In which,
191 1. pool 1 has 8 placement groups. And two tables follow:
193 2. A table for placement groups. Each row presents a placement
195 group. With columns of:
196 • placement group id,
198 • acting set, and
200 • primary OSD.
202 3. A table for all OSDs. Each row presents an OSD. With columns
204 of:
205 • count of placement groups being mapped to this OSD,
207 • count of placement groups where this OSD is the first one
209 in their acting sets,
210 • count of placement groups where this OSD is the primary of
212 them,
213 • the CRUSH weight of this OSD, and
215 • the weight of this OSD.
217 4. Looking at the number of placement groups held by 3 OSDs. We
219 have
220 • avarge, stddev, stddev/average, expected stddev, expected
222 stddev / average
223 • min and max
225 5. The number of placement groups mapping to n OSDs. In this
227 case, all 8 placement groups are mapping to 3 different OSDs.
228 In a less-balanced cluster, we could have following output for the sta‐
230 tistics of placement group distribution, whose standard deviation is
231 1.41421:
232 #osd count first primary c wt wt
234 osd.0 8 5 5 1 1
235 osd.1 8 1 1 1 1
236 osd.2 8 2 2 1 1
237 #osd count first primary c wt wt
239 osd.0 33 9 9 0.0145874 1
240 osd.1 34 14 14 0.0145874 1
241 osd.2 31 7 7 0.0145874 1
242 osd.3 31 13 13 0.0145874 1
243 osd.4 30 14 14 0.0145874 1
244 osd.5 33 7 7 0.0145874 1
245 in 6
246 avg 32 stddev 1.41421 (0.0441942x) (expected 5.16398 0.161374x))
247 min osd.4 30
248 max osd.1 34
249 size 00
250 size 10
251 size 20
252 size 364
253 To simulate the active balancer in upmap mode:
255 osdmaptool --upmap upmaps.out --upmap-active --upmap-deviation 6 --upmap-max 11 osdmap
257 osdmaptool: osdmap file 'osdmap'
259 writing upmap command output to: upmaps.out
260 checking for upmap cleanups
261 upmap, max-count 11, max deviation 6
262 pools movies photos metadata data
263 prepared 11/11 changes
264 Time elapsed 0.00310404 secs
265 pools movies photos metadata data
266 prepared 11/11 changes
267 Time elapsed 0.00283402 secs
268 pools data metadata movies photos
269 prepared 11/11 changes
270 Time elapsed 0.003122 secs
271 pools photos metadata data movies
272 prepared 11/11 changes
273 Time elapsed 0.00324372 secs
274 pools movies metadata data photos
275 prepared 1/11 changes
276 Time elapsed 0.00222609 secs
277 pools data movies photos metadata
278 prepared 0/11 changes
279 Time elapsed 0.00209916 secs
280 Unable to find further optimization, or distribution is already perfect
281 osd.0 pgs 41
282 osd.1 pgs 42
283 osd.2 pgs 42
284 osd.3 pgs 41
285 osd.4 pgs 46
286 osd.5 pgs 39
287 osd.6 pgs 39
288 osd.7 pgs 43
289 osd.8 pgs 41
290 osd.9 pgs 46
291 osd.10 pgs 46
292 osd.11 pgs 46
293 osd.12 pgs 46
294 osd.13 pgs 41
295 osd.14 pgs 40
296 osd.15 pgs 40
297 osd.16 pgs 39
298 osd.17 pgs 46
299 osd.18 pgs 46
300 osd.19 pgs 39
301 osd.20 pgs 42
302 Total time elapsed 0.0167765 secs, 5 rounds
303
305 osdmaptool is part of Ceph, a massively scalable, open-source, distrib‐
306 uted storage system. Please refer to the Ceph documentation at
307 https://docs.ceph.com for more information.
308
310 ceph(8), crushtool(8),
311
313 2010-2023, Inktank Storage, Inc. and contributors. Licensed under Cre‐
314 ative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0)
315dev Nov 02, 2023 OSDMAPTOOL(8)