1OSDMAPTOOL(8) Ceph OSDMAPTOOL(8)
2
6 osdmaptool - ceph osd cluster map manipulation tool
7
9 osdmaptool mapfilename [--print] [--createsimple numosd
10 [--pgbits bitsperosd ] ] [--clobber]
11 osdmaptool mapfilename [--import-crush crushmap]
12 osdmaptool mapfilename [--export-crush crushmap]
13 osdmaptool mapfilename [--upmap file] [--upmap-max max-optimizations]
14 [--upmap-deviation max-deviation] [--upmap-pool poolname]
15 [--upmap-save file] [--upmap-save newosdmap] [--upmap-active]
16 osdmaptool mapfilename [--upmap-cleanup] [--upmap-save newosdmap]
17
20 osdmaptool is a utility that lets you create, view, and manipulate OSD
21 cluster maps from the Ceph distributed storage system. Notably, it lets
22 you extract the embedded CRUSH map or import a new CRUSH map. It can
23 also simulate the upmap balancer mode so you can get a sense of what is
24 needed to balance your PGs.
25
27 --print
28 will simply make the tool print a plaintext dump of the map,
29 after any modifications are made.
30 --dump <format>
32 displays the map in plain text when <format> is 'plain', 'json'
33 if specified format is not supported. This is an alternative to
34 the print option.
35 --clobber
37 will allow osdmaptool to overwrite mapfilename if changes are
38 made.
39 --import-crush mapfile
41 will load the CRUSH map from mapfile and embed it in the OSD
42 map.
43 --export-crush mapfile
45 will extract the CRUSH map from the OSD map and write it to map‐
46 file.
47 --createsimple numosd [--pg-bits bitsperosd] [--pgp-bits bits]
49 will create a relatively generic OSD map with the numosd
50 devices. If --pg-bits is specified, the initial placement group
51 counts will be set with bitsperosd bits per OSD. That is, the
52 pg_num map attribute will be set to numosd shifted by bitsper‐
53 osd. If --pgp-bits is specified, then the pgp_num map attribute
54 will be set to numosd shifted by bits.
55 --create-from-conf
57 creates an osd map with default configurations.
58 --test-map-pgs [--pool poolid] [--range-first <first> --range-last
60 <last>]
61 will print out the mappings from placement groups to OSDs. If
62 range is specified, then it iterates from first to last in the
63 directory specified by argument to osdmaptool. Eg: osdmaptool
64 --test-map-pgs --range-first 0 --range-last 2 osdmap_dir. This
65 will iterate through the files named 0,1,2 in osdmap_dir.
66 --test-map-pgs-dump [--pool poolid] [--range-first <first> --range-last
68 <last>]
69 will print out the summary of all placement groups and the map‐
70 pings from them to the mapped OSDs. If range is specified, then
71 it iterates from first to last in the directory specified by
72 argument to osdmaptool. Eg: osdmaptool --test-map-pgs-dump
73 --range-first 0 --range-last 2 osdmap_dir. This will iterate
74 through the files named 0,1,2 in osdmap_dir.
75 --test-map-pgs-dump-all [--pool poolid] [--range-first <first>
77 --range-last <last>]
78 will print out the summary of all placement groups and the map‐
79 pings from them to all the OSDs. If range is specified, then it
80 iterates from first to last in the directory specified by argu‐
81 ment to osdmaptool. Eg: osdmaptool --test-map-pgs-dump-all
82 --range-first 0 --range-last 2 osdmap_dir. This will iterate
83 through the files named 0,1,2 in osdmap_dir.
84 --test-random
86 does a random mapping of placement groups to the OSDs.
87 --test-map-pg <pgid>
89 map a particular placement group(specified by pgid) to the OSDs.
90 --test-map-object <objectname> [--pool <poolid>]
92 map a particular placement group(specified by objectname) to the
93 OSDs.
94 --test-crush [--range-first <first> --range-last <last>]
96 map placement groups to acting OSDs. If range is specified,
97 then it iterates from first to last in the directory specified
98 by argument to osdmaptool. Eg: osdmaptool --test-crush
99 --range-first 0 --range-last 2 osdmap_dir. This will iterate
100 through the files named 0,1,2 in osdmap_dir.
101 --mark-up-in
103 mark osds up and in (but do not persist).
104 --mark-out
106 mark an osd as out (but do not persist)
107 --tree Displays a hierarchical tree of the map.
109 --clear-temp
111 clears pg_temp and primary_temp variables.
112 --health
114 dump health checks
115 --with-default-pool
117 include default pool when creating map
118 --upmap-cleanup <file>
120 clean up pg_upmap[_items] entries, writing commands to <file>
121 [default: - for stdout]
122 --upmap <file>
124 calculate pg upmap entries to balance pg layout writing commands
125 to <file> [default: - for stdout]
126 --upmap-max <max-optimizations>
128 set max upmap entries to calculate [default: 10]
129 --upmap-deviation <max-deviation>
131 max deviation from target [default: 5]
132 --upmap-pool <poolname>
134 restrict upmap balancing to 1 pool or the option can be repeated
135 for multiple pools
136 --upmap-save
138 write modified OSDMap with upmap changes
139 --upmap-active
141 Act like an active balancer, keep applying changes until bal‐
142 anced
143
145 To create a simple map with 16 devices:
146 osdmaptool --createsimple 16 osdmap --clobber
148 To view the result:
150 osdmaptool --print osdmap
152 To view the mappings of placement groups for pool 1:
154 osdmaptool osdmap --test-map-pgs-dump --pool 1
156 pool 0 pg_num 8
158 1.0 [0,2,1] 0
159 1.1 [2,0,1] 2
160 1.2 [0,1,2] 0
161 1.3 [2,0,1] 2
162 1.4 [0,2,1] 0
163 1.5 [0,2,1] 0
164 1.6 [0,1,2] 0
165 1.7 [1,0,2] 1
166 #osd count first primary c wt wt
167 osd.0 8 5 5 1 1
168 osd.1 8 1 1 1 1
169 osd.2 8 2 2 1 1
170 in 3
171 avg 8 stddev 0 (0x) (expected 2.3094 0.288675x))
172 min osd.0 8
173 max osd.0 8
174 size 0 0
175 size 1 0
176 size 2 0
177 size 3 8
178 In which,
180 1. pool 1 has 8 placement groups. And two tables follow:
182 2. A table for placement groups. Each row presents a placement
184 group. With columns of:
185 · placement group id,
187 · acting set, and
189 · primary OSD.
191 3. A table for all OSDs. Each row presents an OSD. With columns
193 of:
194 · count of placement groups being mapped to this OSD,
196 · count of placement groups where this OSD is the first one
198 in their acting sets,
199 · count of placement groups where this OSD is the primary of
201 them,
202 · the CRUSH weight of this OSD, and
204 · the weight of this OSD.
206 4. Looking at the number of placement groups held by 3 OSDs. We
208 have
209 · avarge, stddev, stddev/average, expected stddev, expected
211 stddev / average
212 · min and max
214 5. The number of placement groups mapping to n OSDs. In this
216 case, all 8 placement groups are mapping to 3 different OSDs.
217 In a less-balanced cluster, we could have following output for the sta‐
219 tistics of placement group distribution, whose standard deviation is
220 1.41421:
221 #osd count first primary c wt wt
223 osd.0 8 5 5 1 1
224 osd.1 8 1 1 1 1
225 osd.2 8 2 2 1 1
226 #osd count first primary c wt wt
228 osd.0 33 9 9 0.0145874 1
229 osd.1 34 14 14 0.0145874 1
230 osd.2 31 7 7 0.0145874 1
231 osd.3 31 13 13 0.0145874 1
232 osd.4 30 14 14 0.0145874 1
233 osd.5 33 7 7 0.0145874 1
234 in 6
235 avg 32 stddev 1.41421 (0.0441942x) (expected 5.16398 0.161374x))
236 min osd.4 30
237 max osd.1 34
238 size 00
239 size 10
240 size 20
241 size 364
242 To simulate the active balancer in upmap mode::
244 osdmaptool --upmap upmaps.out --upmap-active --upmap-deviation 6 --upmap-max 11 osdmap
246 osdmaptool: osdmap file 'osdmap'
248 writing upmap command output to: upmaps.out
249 checking for upmap cleanups
250 upmap, max-count 11, max deviation 6
251 pools movies photos metadata data
252 prepared 11/11 changes
253 Time elapsed 0.00310404 secs
254 pools movies photos metadata data
255 prepared 11/11 changes
256 Time elapsed 0.00283402 secs
257 pools data metadata movies photos
258 prepared 11/11 changes
259 Time elapsed 0.003122 secs
260 pools photos metadata data movies
261 prepared 11/11 changes
262 Time elapsed 0.00324372 secs
263 pools movies metadata data photos
264 prepared 1/11 changes
265 Time elapsed 0.00222609 secs
266 pools data movies photos metadata
267 prepared 0/11 changes
268 Time elapsed 0.00209916 secs
269 Unable to find further optimization, or distribution is already perfect
270 osd.0 pgs 41
271 osd.1 pgs 42
272 osd.2 pgs 42
273 osd.3 pgs 41
274 osd.4 pgs 46
275 osd.5 pgs 39
276 osd.6 pgs 39
277 osd.7 pgs 43
278 osd.8 pgs 41
279 osd.9 pgs 46
280 osd.10 pgs 46
281 osd.11 pgs 46
282 osd.12 pgs 46
283 osd.13 pgs 41
284 osd.14 pgs 40
285 osd.15 pgs 40
286 osd.16 pgs 39
287 osd.17 pgs 46
288 osd.18 pgs 46
289 osd.19 pgs 39
290 osd.20 pgs 42
291 Total time elapsed 0.0167765 secs, 5 rounds
292
294 osdmaptool is part of Ceph, a massively scalable, open-source, distrib‐
295 uted storage system. Please refer to the Ceph documentation at
296 http://ceph.com/docs for more information.
297
299 ceph(8), crushtool(8),
300
302 2010-2020, Inktank Storage, Inc. and contributors. Licensed under Cre‐
303 ative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0)
304dev Apr 21, 2020 OSDMAPTOOL(8)