1OSDMAPTOOL(8)                        Ceph                        OSDMAPTOOL(8)
2
3
4

NAME

6       osdmaptool - ceph osd cluster map manipulation tool
7

SYNOPSIS

9       osdmaptool mapfilename [--print] [--createsimple numosd
10       [--pgbits bitsperosd ] ] [--clobber]
11       osdmaptool mapfilename [--import-crush crushmap]
12       osdmaptool mapfilename [--export-crush crushmap]
13       osdmaptool mapfilename [--upmap file] [--upmap-max max-optimizations]
14       [--upmap-deviation max-deviation] [--upmap-pool poolname]
15       [--save] [--upmap-active]
16       osdmaptool mapfilename [--upmap-cleanup] [--upmap file]
17
18

DESCRIPTION

20       osdmaptool  is a utility that lets you create, view, and manipulate OSD
21       cluster maps from the Ceph distributed storage system. Notably, it lets
22       you  extract  the embedded CRUSH map or import a new CRUSH map.  It can
23       also simulate the upmap balancer mode so you can get a sense of what is
24       needed to balance your PGs.
25

OPTIONS

27       --print
28              will simply make the tool print a plaintext dump of the map, af‐
29              ter any modifications are made.
30
31       --dump <format>
32              displays the map in plain text when <format> is 'plain',  'json'
33              if  specified format is not supported. This is an alternative to
34              the print option.
35
36       --clobber
37              will allow osdmaptool to overwrite mapfilename  if  changes  are
38              made.
39
40       --import-crush mapfile
41              will  load  the  CRUSH  map from mapfile and embed it in the OSD
42              map.
43
44       --export-crush mapfile
45              will extract the CRUSH map from the OSD map and write it to map‐
46              file.
47
48       --createsimple numosd [--pg-bits bitsperosd] [--pgp-bits bits]
49              will  create  a  relatively  generic OSD map with the numosd de‐
50              vices.  If --pg-bits is specified, the initial  placement  group
51              counts  will  be  set with bitsperosd bits per OSD. That is, the
52              pg_num map attribute will be set to numosd shifted  by  bitsper‐
53              osd.  If --pgp-bits is specified, then the pgp_num map attribute
54              will be set to numosd shifted by bits.
55
56       --create-from-conf
57              creates an osd map with default configurations.
58
59       --test-map-pgs  [--pool  poolid]  [--range-first  <first>  --range-last
60       <last>]
61              will  print  out the mappings from placement groups to OSDs.  If
62              range is specified, then it iterates from first to last  in  the
63              directory  specified  by argument to osdmaptool.  Eg: osdmaptool
64              --test-map-pgs --range-first 0 --range-last 2 osdmap_dir.   This
65              will iterate through the files named 0,1,2 in osdmap_dir.
66
67       --test-map-pgs-dump [--pool poolid] [--range-first <first> --range-last
68       <last>]
69              will print out the summary of all placement groups and the  map‐
70              pings from them to the mapped OSDs.  If range is specified, then
71              it iterates from first to last in the directory specified by ar‐
72              gument   to   osdmaptool.   Eg:  osdmaptool  --test-map-pgs-dump
73              --range-first 0 --range-last 2 osdmap_dir.   This  will  iterate
74              through the files named 0,1,2 in osdmap_dir.
75
76       --test-map-pgs-dump-all    [--pool   poolid]   [--range-first   <first>
77       --range-last <last>]
78              will print out the summary of all placement groups and the  map‐
79              pings from them to all the OSDs.  If range is specified, then it
80              iterates from first to last in the directory specified by  argu‐
81              ment  to  osdmaptool.   Eg:  osdmaptool  --test-map-pgs-dump-all
82              --range-first 0 --range-last 2 osdmap_dir.   This  will  iterate
83              through the files named 0,1,2 in osdmap_dir.
84
85       --test-random
86              does a random mapping of placement groups to the OSDs.
87
88       --test-map-pg <pgid>
89              map a particular placement group(specified by pgid) to the OSDs.
90
91       --test-map-object <objectname> [--pool <poolid>]
92              map a particular placement group(specified by objectname) to the
93              OSDs.
94
95       --test-crush [--range-first <first> --range-last <last>]
96              map placement groups to acting OSDs.   If  range  is  specified,
97              then  it  iterates from first to last in the directory specified
98              by  argument  to  osdmaptool.    Eg:   osdmaptool   --test-crush
99              --range-first  0  --range-last  2 osdmap_dir.  This will iterate
100              through the files named 0,1,2 in osdmap_dir.
101
102       --mark-up-in
103              mark osds up and in (but do not persist).
104
105       --mark-out
106              mark an osd as out (but do not persist)
107
108       --mark-up <osdid>
109              mark an osd as up (but do not persist)
110
111       --mark-in <osdid>
112              mark an osd as in (but do not persist)
113
114       --tree Displays a hierarchical tree of the map.
115
116       --clear-temp
117              clears pg_temp and primary_temp variables.
118
119       --clean-temps
120              clean pg_temps.
121
122       --health
123              dump health checks
124
125       --with-default-pool
126              include default pool when creating map
127
128       --upmap-cleanup <file>
129              clean up pg_upmap[_items] entries, writing  commands  to  <file>
130              [default: - for stdout]
131
132       --upmap <file>
133              calculate pg upmap entries to balance pg layout writing commands
134              to <file> [default: - for stdout]
135
136       --upmap-max <max-optimizations>
137              set max upmap entries to calculate [default: 10]
138
139       --upmap-deviation <max-deviation>
140              max deviation from target [default: 5]
141
142       --upmap-pool <poolname>
143              restrict upmap balancing to 1 pool or the option can be repeated
144              for multiple pools
145
146       --upmap-active
147              Act  like  an  active balancer, keep applying changes until bal‐
148              anced
149
150       --adjust-crush-weight <osdid:weight>[,<osdid:weight>,<...>]
151              Change CRUSH weight of <osdid>
152
153       --save write modified osdmap with upmap or crush-adjust changes
154

EXAMPLE

156       To create a simple map with 16 devices:
157
158          osdmaptool --createsimple 16 osdmap --clobber
159
160       To view the result:
161
162          osdmaptool --print osdmap
163
164       To view the mappings of placement groups for pool 1:
165
166          osdmaptool osdmap --test-map-pgs-dump --pool 1
167
168          pool 1 pg_num 8
169          1.0     [0,2,1] 0
170          1.1     [2,0,1] 2
171          1.2     [0,1,2] 0
172          1.3     [2,0,1] 2
173          1.4     [0,2,1] 0
174          1.5     [0,2,1] 0
175          1.6     [0,1,2] 0
176          1.7     [1,0,2] 1
177          #osd    count   first   primary c wt    wt
178          osd.0   8       5       5       1       1
179          osd.1   8       1       1       1       1
180          osd.2   8       2       2       1       1
181           in 3
182           avg 8 stddev 0 (0x) (expected 2.3094 0.288675x))
183           min osd.0 8
184           max osd.0 8
185          size 0  0
186          size 1  0
187          size 2  0
188          size 3  8
189
190       In which,
191
192              1. pool 1 has 8 placement groups. And two tables follow:
193
194              2. A table for placement groups. Each row presents  a  placement
195                 group. With columns of:
196
197                 • placement group id,
198
199                 • acting set, and
200
201                 • primary OSD.
202
203              3. A  table for all OSDs. Each row presents an OSD. With columns
204                 of:
205
206                 • count of placement groups being mapped to this OSD,
207
208                 • count of placement groups where this OSD is the  first  one
209                   in their acting sets,
210
211                 • count  of placement groups where this OSD is the primary of
212                   them,
213
214                 • the CRUSH weight of this OSD, and
215
216                 • the weight of this OSD.
217
218              4. Looking at the number of placement groups held by 3 OSDs.  We
219                 have
220
221                 • avarge,  stddev,  stddev/average, expected stddev, expected
222                   stddev / average
223
224                 • min and max
225
226              5. The number of placement groups mapping to  n  OSDs.  In  this
227                 case, all 8 placement groups are mapping to 3 different OSDs.
228
229       In a less-balanced cluster, we could have following output for the sta‐
230       tistics of placement group distribution, whose  standard  deviation  is
231       1.41421:
232
233          #osd    count   first   primary c wt    wt
234          osd.0   8       5       5       1       1
235          osd.1   8       1       1       1       1
236          osd.2   8       2       2       1       1
237
238          #osd    count   first    primary c wt    wt
239          osd.0   33      9        9       0.0145874     1
240          osd.1   34      14       14      0.0145874     1
241          osd.2   31      7        7       0.0145874     1
242          osd.3   31      13       13      0.0145874     1
243          osd.4   30      14       14      0.0145874     1
244          osd.5   33      7        7       0.0145874     1
245           in 6
246           avg 32 stddev 1.41421 (0.0441942x) (expected 5.16398 0.161374x))
247           min osd.4 30
248           max osd.1 34
249          size 00
250          size 10
251          size 20
252          size 364
253
254       To simulate the active balancer in upmap mode:
255
256               osdmaptool --upmap upmaps.out --upmap-active --upmap-deviation 6 --upmap-max 11 osdmap
257
258          osdmaptool: osdmap file 'osdmap'
259          writing upmap command output to: upmaps.out
260          checking for upmap cleanups
261          upmap, max-count 11, max deviation 6
262          pools movies photos metadata data
263          prepared 11/11 changes
264          Time elapsed 0.00310404 secs
265          pools movies photos metadata data
266          prepared 11/11 changes
267          Time elapsed 0.00283402 secs
268          pools data metadata movies photos
269          prepared 11/11 changes
270          Time elapsed 0.003122 secs
271          pools photos metadata data movies
272          prepared 11/11 changes
273          Time elapsed 0.00324372 secs
274          pools movies metadata data photos
275          prepared 1/11 changes
276          Time elapsed 0.00222609 secs
277          pools data movies photos metadata
278          prepared 0/11 changes
279          Time elapsed 0.00209916 secs
280          Unable to find further optimization, or distribution is already perfect
281          osd.0 pgs 41
282          osd.1 pgs 42
283          osd.2 pgs 42
284          osd.3 pgs 41
285          osd.4 pgs 46
286          osd.5 pgs 39
287          osd.6 pgs 39
288          osd.7 pgs 43
289          osd.8 pgs 41
290          osd.9 pgs 46
291          osd.10 pgs 46
292          osd.11 pgs 46
293          osd.12 pgs 46
294          osd.13 pgs 41
295          osd.14 pgs 40
296          osd.15 pgs 40
297          osd.16 pgs 39
298          osd.17 pgs 46
299          osd.18 pgs 46
300          osd.19 pgs 39
301          osd.20 pgs 42
302          Total time elapsed 0.0167765 secs, 5 rounds
303

AVAILABILITY

305       osdmaptool is part of Ceph, a massively scalable, open-source, distrib‐
306       uted storage  system.   Please  refer  to  the  Ceph  documentation  at
307       http://ceph.com/docs for more information.
308

SEE ALSO

310       ceph(8), crushtool(8),
311
313       2010-2022,  Inktank Storage, Inc. and contributors. Licensed under Cre‐
314       ative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0)
315
316
317
318
319dev                              Jun 22, 2022                    OSDMAPTOOL(8)
Impressum