1OSDMAPTOOL(8) Ceph OSDMAPTOOL(8)
2
6 osdmaptool - ceph osd cluster map manipulation tool
7
9 osdmaptool mapfilename [--print] [--createsimple numosd
10 [--pgbits bitsperosd ] ] [--clobber]
11 osdmaptool mapfilename [--import-crush crushmap]
12 osdmaptool mapfilename [--export-crush crushmap]
13 osdmaptool mapfilename [--upmap file] [--upmap-max max-optimizations]
14 [--upmap-deviation max-deviation] [--upmap-pool poolname]
15 [--upmap-save file] [--upmap-save newosdmap] [--upmap-active]
16 osdmaptool mapfilename [--upmap-cleanup] [--upmap-save newosdmap]
17
20 osdmaptool is a utility that lets you create, view, and manipulate OSD
21 cluster maps from the Ceph distributed storage system. Notably, it lets
22 you extract the embedded CRUSH map or import a new CRUSH map. It can
23 also simulate the upmap balancer mode so you can get a sense of what is
24 needed to balance your PGs.
25
27 --print
28 will simply make the tool print a plaintext dump of the map,
29 after any modifications are made.
30 --dump <format>
32 displays the map in plain text when <format> is 'plain', 'json'
33 if specified format is not supported. This is an alternative to
34 the print option.
35 --clobber
37 will allow osdmaptool to overwrite mapfilename if changes are
38 made.
39 --import-crush mapfile
41 will load the CRUSH map from mapfile and embed it in the OSD
42 map.
43 --export-crush mapfile
45 will extract the CRUSH map from the OSD map and write it to map‐
46 file.
47 --createsimple numosd [--pg-bits bitsperosd] [--pgp-bits bits]
49 will create a relatively generic OSD map with the numosd
50 devices. If --pg-bits is specified, the initial placement group
51 counts will be set with bitsperosd bits per OSD. That is, the
52 pg_num map attribute will be set to numosd shifted by bitsper‐
53 osd. If --pgp-bits is specified, then the pgp_num map attribute
54 will be set to numosd shifted by bits.
55 --create-from-conf
57 creates an osd map with default configurations.
58 --test-map-pgs [--pool poolid] [--range-first <first> --range-last
60 <last>]
61 will print out the mappings from placement groups to OSDs. If
62 range is specified, then it iterates from first to last in the
63 directory specified by argument to osdmaptool. Eg: osdmaptool
64 --test-map-pgs --range-first 0 --range-last 2 osdmap_dir. This
65 will iterate through the files named 0,1,2 in osdmap_dir.
66 --test-map-pgs-dump [--pool poolid] [--range-first <first> --range-last
68 <last>]
69 will print out the summary of all placement groups and the map‐
70 pings from them to the mapped OSDs. If range is specified, then
71 it iterates from first to last in the directory specified by
72 argument to osdmaptool. Eg: osdmaptool --test-map-pgs-dump
73 --range-first 0 --range-last 2 osdmap_dir. This will iterate
74 through the files named 0,1,2 in osdmap_dir.
75 --test-map-pgs-dump-all [--pool poolid] [--range-first <first>
77 --range-last <last>]
78 will print out the summary of all placement groups and the map‐
79 pings from them to all the OSDs. If range is specified, then it
80 iterates from first to last in the directory specified by argu‐
81 ment to osdmaptool. Eg: osdmaptool --test-map-pgs-dump-all
82 --range-first 0 --range-last 2 osdmap_dir. This will iterate
83 through the files named 0,1,2 in osdmap_dir.
84 --test-random
86 does a random mapping of placement groups to the OSDs.
87 --test-map-pg <pgid>
89 map a particular placement group(specified by pgid) to the OSDs.
90 --test-map-object <objectname> [--pool <poolid>]
92 map a particular placement group(specified by objectname) to the
93 OSDs.
94 --test-crush [--range-first <first> --range-last <last>]
96 map placement groups to acting OSDs. If range is specified,
97 then it iterates from first to last in the directory specified
98 by argument to osdmaptool. Eg: osdmaptool --test-crush
99 --range-first 0 --range-last 2 osdmap_dir. This will iterate
100 through the files named 0,1,2 in osdmap_dir.
101 --mark-up-in
103 mark osds up and in (but do not persist).
104 --mark-out
106 mark an osd as out (but do not persist)
107 --tree Displays a hierarchical tree of the map.
109 --clear-temp
111 clears pg_temp and primary_temp variables.
112 --clean-temps
114 clean pg_temps.
115 --health
117 dump health checks
118 --with-default-pool
120 include default pool when creating map
121 --upmap-cleanup <file>
123 clean up pg_upmap[_items] entries, writing commands to <file>
124 [default: - for stdout]
125 --upmap <file>
127 calculate pg upmap entries to balance pg layout writing commands
128 to <file> [default: - for stdout]
129 --upmap-max <max-optimizations>
131 set max upmap entries to calculate [default: 10]
132 --upmap-deviation <max-deviation>
134 max deviation from target [default: 5]
135 --upmap-pool <poolname>
137 restrict upmap balancing to 1 pool or the option can be repeated
138 for multiple pools
139 --upmap-save
141 write modified OSDMap with upmap changes
142 --upmap-active
144 Act like an active balancer, keep applying changes until bal‐
145 anced
146
148 To create a simple map with 16 devices:
149 osdmaptool --createsimple 16 osdmap --clobber
151 To view the result:
153 osdmaptool --print osdmap
155 To view the mappings of placement groups for pool 1:
157 osdmaptool osdmap --test-map-pgs-dump --pool 1
159 pool 0 pg_num 8
161 1.0 [0,2,1] 0
162 1.1 [2,0,1] 2
163 1.2 [0,1,2] 0
164 1.3 [2,0,1] 2
165 1.4 [0,2,1] 0
166 1.5 [0,2,1] 0
167 1.6 [0,1,2] 0
168 1.7 [1,0,2] 1
169 #osd count first primary c wt wt
170 osd.0 8 5 5 1 1
171 osd.1 8 1 1 1 1
172 osd.2 8 2 2 1 1
173 in 3
174 avg 8 stddev 0 (0x) (expected 2.3094 0.288675x))
175 min osd.0 8
176 max osd.0 8
177 size 0 0
178 size 1 0
179 size 2 0
180 size 3 8
181 In which,
183 1. pool 1 has 8 placement groups. And two tables follow:
185 2. A table for placement groups. Each row presents a placement
187 group. With columns of:
188 · placement group id,
190 · acting set, and
192 · primary OSD.
194 3. A table for all OSDs. Each row presents an OSD. With columns
196 of:
197 · count of placement groups being mapped to this OSD,
199 · count of placement groups where this OSD is the first one
201 in their acting sets,
202 · count of placement groups where this OSD is the primary of
204 them,
205 · the CRUSH weight of this OSD, and
207 · the weight of this OSD.
209 4. Looking at the number of placement groups held by 3 OSDs. We
211 have
212 · avarge, stddev, stddev/average, expected stddev, expected
214 stddev / average
215 · min and max
217 5. The number of placement groups mapping to n OSDs. In this
219 case, all 8 placement groups are mapping to 3 different OSDs.
220 In a less-balanced cluster, we could have following output for the sta‐
222 tistics of placement group distribution, whose standard deviation is
223 1.41421:
224 #osd count first primary c wt wt
226 osd.0 8 5 5 1 1
227 osd.1 8 1 1 1 1
228 osd.2 8 2 2 1 1
229 #osd count first primary c wt wt
231 osd.0 33 9 9 0.0145874 1
232 osd.1 34 14 14 0.0145874 1
233 osd.2 31 7 7 0.0145874 1
234 osd.3 31 13 13 0.0145874 1
235 osd.4 30 14 14 0.0145874 1
236 osd.5 33 7 7 0.0145874 1
237 in 6
238 avg 32 stddev 1.41421 (0.0441942x) (expected 5.16398 0.161374x))
239 min osd.4 30
240 max osd.1 34
241 size 00
242 size 10
243 size 20
244 size 364
245 To simulate the active balancer in upmap mode::
247 osdmaptool --upmap upmaps.out --upmap-active --upmap-deviation 6 --upmap-max 11 osdmap
249 osdmaptool: osdmap file 'osdmap'
251 writing upmap command output to: upmaps.out
252 checking for upmap cleanups
253 upmap, max-count 11, max deviation 6
254 pools movies photos metadata data
255 prepared 11/11 changes
256 Time elapsed 0.00310404 secs
257 pools movies photos metadata data
258 prepared 11/11 changes
259 Time elapsed 0.00283402 secs
260 pools data metadata movies photos
261 prepared 11/11 changes
262 Time elapsed 0.003122 secs
263 pools photos metadata data movies
264 prepared 11/11 changes
265 Time elapsed 0.00324372 secs
266 pools movies metadata data photos
267 prepared 1/11 changes
268 Time elapsed 0.00222609 secs
269 pools data movies photos metadata
270 prepared 0/11 changes
271 Time elapsed 0.00209916 secs
272 Unable to find further optimization, or distribution is already perfect
273 osd.0 pgs 41
274 osd.1 pgs 42
275 osd.2 pgs 42
276 osd.3 pgs 41
277 osd.4 pgs 46
278 osd.5 pgs 39
279 osd.6 pgs 39
280 osd.7 pgs 43
281 osd.8 pgs 41
282 osd.9 pgs 46
283 osd.10 pgs 46
284 osd.11 pgs 46
285 osd.12 pgs 46
286 osd.13 pgs 41
287 osd.14 pgs 40
288 osd.15 pgs 40
289 osd.16 pgs 39
290 osd.17 pgs 46
291 osd.18 pgs 46
292 osd.19 pgs 39
293 osd.20 pgs 42
294 Total time elapsed 0.0167765 secs, 5 rounds
295
297 osdmaptool is part of Ceph, a massively scalable, open-source, distrib‐
298 uted storage system. Please refer to the Ceph documentation at
299 http://ceph.com/docs for more information.
300
302 ceph(8), crushtool(8),
303
305 2010-2021, Inktank Storage, Inc. and contributors. Licensed under Cre‐
306 ative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0)
307dev Mar 18, 2021 OSDMAPTOOL(8)