1CEPH-VOLUME(8)                       Ceph                       CEPH-VOLUME(8)
2
3
4

NAME

6       ceph-volume - Ceph OSD deployment and inspection tool
7

SYNOPSIS

9       ceph-volume    [-h]   [--cluster   CLUSTER]   [--log-level   LOG_LEVEL]
10       [--log-path LOG_PATH]
11
12       ceph-volume inventory
13
14       ceph-volume lvm [ trigger | create | activate | prepare | zap | list  |
15       batch | new-wal | new-db | migrate ]
16
17       ceph-volume simple [ trigger | scan | activate ]
18

DESCRIPTION

20       ceph-volume  is  a  single  purpose command line tool to deploy logical
21       volumes as OSDs, trying to maintain a similar  API  to  ceph-disk  when
22       preparing, activating, and creating OSDs.
23
24       It  deviates  from  ceph-disk by not interacting or relying on the udev
25       rules that come installed for Ceph. These rules allow automatic  detec‐
26       tion of previously setup devices that are in turn fed into ceph-disk to
27       activate them.
28

COMMANDS

30   inventory
31       This subcommand provides information about a host's physical  disc  in‐
32       ventory and reports metadata about these discs. Among this metadata one
33       can find disc specific data items  (like  model,  size,  rotational  or
34       solid  state)  as  well  as data items specific to ceph using a device,
35       such as if it is available for use with ceph or if logical volumes  are
36       present.
37
38       Examples:
39
40          ceph-volume inventory
41          ceph-volume inventory /dev/sda
42          ceph-volume inventory --format json-pretty
43
44       Optional arguments:
45
46       -h, --help
47              show the help message and exit
48
49       --format
50              report  format,  valid  values  are  plain  (default),  json and
51              json-pretty
52
53   lvm
54       By making use of LVM tags, the lvm sub-command is  able  to  store  and
55       later  re-discover  and query devices associated with OSDs so that they
56       can later activated.
57
58       Subcommands:
59
60   batch
61       Creates OSDs from a list of devices using a filestore or bluestore (de‐
62       fault)  setup.  It  will create all necessary volume groups and logical
63       volumes required to have a working OSD.
64
65       Example usage with three devices:
66
67          ceph-volume lvm batch --bluestore /dev/sda /dev/sdb /dev/sdc
68
69       Optional arguments:
70
71       -h, --help
72              show the help message and exit
73
74       --bluestore
75              Use the bluestore objectstore (default)
76
77       --filestore
78              Use the filestore objectstore
79
80       --yes  Skip the report and prompt to continue provisioning
81
82       --prepare
83              Only prepare OSDs, do not activate
84
85       --dmcrypt
86              Enable encryption for the underlying OSD devices
87
88       --crush-device-class
89              Define a CRUSH device class to assign the OSD to
90
91       --no-systemd
92              Do not enable or create any systemd units
93
94       --osds-per-device
95              Provision more than 1 (the default) OSD per device
96
97       --report
98              Report what the potential outcome would be for the current input
99              (requires devices to be passed in)
100
101       --format
102              Output  format when reporting (used along with --report), can be
103              one of 'pretty' (default) or 'json'
104
105       --block-db-size
106              Set (or override) the "bluestore_block_db_size" value, in bytes
107
108       --journal-size
109              Override the "osd_journal_size" value, in megabytes
110
111       Required positional arguments:
112
113       <DEVICE>
114              Full path to a raw  device,  like  /dev/sda.  Multiple  <DEVICE>
115              paths can be passed in.
116
117   activate
118       Enables  a  systemd  unit  that  persists the OSD ID and its UUID (also
119       called fsid in Ceph CLI tools), so that at boot time it can  understand
120       what OSD is enabled and needs to be mounted.
121
122       Usage:
123
124          ceph-volume lvm activate --bluestore <osd id> <osd fsid>
125
126       Optional Arguments:
127
128       -h, --help
129              show the help message and exit
130
131       --auto-detect-objectstore
132              Automatically detect the objectstore by inspecting the OSD
133
134       --bluestore
135              bluestore objectstore (default)
136
137       --filestore
138              filestore objectstore
139
140       --all  Activate all OSDs found in the system
141
142       --no-systemd
143              Skip  creating  and  enabling  systemd units and starting of OSD
144              services
145
146       Multiple OSDs can be activated at once by using the (idempotent)  --all
147       flag:
148
149          ceph-volume lvm activate --all
150
151   prepare
152       Prepares  a  logical  volume  to  be used as an OSD and journal using a
153       filestore or bluestore (default) setup. It will not  create  or  modify
154       the logical volumes except for adding extra metadata.
155
156       Usage:
157
158          ceph-volume lvm prepare --filestore --data <data lv> --journal <journal device>
159
160       Optional arguments:
161
162       -h, --help
163              show the help message and exit
164
165       --journal JOURNAL
166              logical  group  name, path to a logical volume, or path to a de‐
167              vice
168
169       --bluestore
170              Use the bluestore objectstore (default)
171
172       --block.wal
173              Path to a bluestore block.wal logical volume or partition
174
175       --block.db
176              Path to a bluestore block.db logical volume or partition
177
178       --filestore
179              Use the filestore objectstore
180
181       --dmcrypt
182              Enable encryption for the underlying OSD devices
183
184       --osd-id OSD_ID
185              Reuse an existing OSD id
186
187       --osd-fsid OSD_FSID
188              Reuse an existing OSD fsid
189
190       --crush-device-class
191              Define a CRUSH device class to assign the OSD to
192
193       Required arguments:
194
195       --data A logical group name or a path to a logical volume
196
197       For encrypting an OSD, the --dmcrypt flag must be added when  preparing
198       (also supported in the create sub-command).
199
200   create
201       Wraps  the  two-step  process  to  provision a new osd (calling prepare
202       first and then activate) into a single one. The reason to  prefer  pre‐
203       pare  and then activate is to gradually introduce new OSDs into a clus‐
204       ter, and avoiding large amounts of data being rebalanced.
205
206       The single-call process unifies exactly what prepare and  activate  do,
207       with  the  convenience of doing it all at once. Flags and general usage
208       are equivalent to those of the prepare and activate subcommand.
209
210   trigger
211       This subcommand is not meant to be used directly, and  it  is  used  by
212       systemd so that it proxies input to ceph-volume lvm activate by parsing
213       the input from systemd, detecting the UUID and ID  associated  with  an
214       OSD.
215
216       Usage:
217
218          ceph-volume lvm trigger <SYSTEMD-DATA>
219
220       The systemd "data" is expected to be in the format of:
221
222          <OSD ID>-<OSD UUID>
223
224       The  lvs associated with the OSD need to have been prepared previously,
225       so that all needed tags and metadata exist.
226
227       Positional arguments:
228
229       <SYSTEMD_DATA>
230              Data from a systemd unit containing ID and UUID of the OSD.
231
232   list
233       List devices or logical volumes associated with Ceph. An association is
234       determined if a device has information relating to an OSD. This is ver‐
235       ified by querying LVM's metadata and correlating it with devices.
236
237       The lvs associated with the OSD need to have been  prepared  previously
238       by ceph-volume so that all needed tags and metadata exist.
239
240       Usage:
241
242          ceph-volume lvm list
243
244       List a particular device, reporting all metadata about it:
245
246          ceph-volume lvm list /dev/sda1
247
248       List  a  logical  volume,  along  with all its metadata (vg is a volume
249       group, and lv the logical volume name):
250
251          ceph-volume lvm list {vg/lv}
252
253       Positional arguments:
254
255       <DEVICE>
256              Either in the form of vg/lv for logical  volumes,  /path/to/sda1
257              or /path/to/sda for regular devices.
258
259   zap
260       Zaps  the given logical volume or partition. If given a path to a logi‐
261       cal volume it must be in the format of vg/lv. Any file systems  present
262       on  the  given  lv  or  partition  will be removed and all data will be
263       purged.
264
265       However, the lv or partition will be kept intact.
266
267       Usage, for logical volumes:
268
269          ceph-volume lvm zap {vg/lv}
270
271       Usage, for logical partitions:
272
273          ceph-volume lvm zap /dev/sdc1
274
275       For full removal of the device use the --destroy flag (allowed for  all
276       device types):
277
278          ceph-volume lvm zap --destroy /dev/sdc1
279
280       Multiple devices can be removed by specifying the OSD ID and/or the OSD
281       FSID:
282
283          ceph-volume lvm zap --destroy --osd-id 1
284          ceph-volume lvm zap --destroy --osd-id 1 --osd-fsid C9605912-8395-4D76-AFC0-7DFDAC315D59
285
286       Positional arguments:
287
288       <DEVICE>
289              Either in the form of vg/lv for logical  volumes,  /path/to/sda1
290              or /path/to/sda for regular devices.
291
292   new-wal
293       Attaches  the given logical volume to OSD as a WAL. Logical volume name
294       format is vg/lv. Fails if OSD has already got attached WAL.
295
296       Usage:
297
298          ceph-volume lvm new-wal --osd-id OSD_ID --osd-fsid OSD_FSID --target <target lv>
299
300       Optional arguments:
301
302       -h, --help
303              show the help message and exit
304
305       --no-systemd
306              Skip checking OSD systemd unit
307
308       Required arguments:
309
310       --target
311              logical volume name to attach as WAL
312
313   new-db
314       Attaches the given logical volume to OSD as a DB. Logical  volume  name
315       format is vg/lv. Fails if OSD has already got attached DB.
316
317       Usage:
318
319          ceph-volume lvm new-db --osd-id OSD_ID --osd-fsid OSD_FSID --target <target lv>
320
321       Optional arguments:
322
323       -h, --help
324              show the help message and exit
325
326       --no-systemd
327              Skip checking OSD systemd unit
328
329       Required arguments:
330
331       --target
332              logical volume name to attach as DB
333
334   migrate
335       Moves  BlueFS data from source volume(s) to the target one, source vol‐
336       umes (except the main, i.e. data or block one) are removed on  success.
337       LVM volumes are permitted for Target only, both already attached or new
338       one. In the latter case it is attached to the OSD replacing one of  the
339       source  devices.  Following  replacement  rules  apply (in the order of
340       precedence, stop on the first match):
341
342          • if source list has DB volume - target device replaces it.
343
344          • if source list has WAL volume - target device replace it.
345
346          • if source list has slow volume only - operation is not  permitted,
347            requires explicit allocation via new-db/new-wal command.
348
349       Usage:
350
351          ceph-volume lvm migrate --osd-id OSD_ID --osd-fsid OSD_FSID --target <target lv> --from {data|db|wal} [{data|db|wal} ...]
352
353       Optional arguments:
354
355       -h, --help
356              show the help message and exit
357
358       --no-systemd
359              Skip checking OSD systemd unit
360
361       Required arguments:
362
363       --from list of source device type names
364
365       --target
366              logical volume to move data to
367
368   simple
369       Scan  legacy OSD directories or data devices that may have been created
370       by ceph-disk, or manually.
371
372       Subcommands:
373
374   activate
375       Enables a systemd unit that persists the OSD  ID  and  its  UUID  (also
376       called  fsid in Ceph CLI tools), so that at boot time it can understand
377       what OSD is enabled and needs to be mounted, while reading  information
378       that  was  previously  created  and persisted at /etc/ceph/osd/ in JSON
379       format.
380
381       Usage:
382
383          ceph-volume simple activate --bluestore <osd id> <osd fsid>
384
385       Optional Arguments:
386
387       -h, --help
388              show the help message and exit
389
390       --bluestore
391              bluestore objectstore (default)
392
393       --filestore
394              filestore objectstore
395
396       NOTE:
397          It requires a matching JSON file with the following format:
398
399              /etc/ceph/osd/<osd id>-<osd fsid>.json
400
401   scan
402       Scan a running OSD or data device for an  OSD  for  metadata  that  can
403       later be used to activate and manage the OSD with ceph-volume. The scan
404       method will create a JSON file with the required information plus  any‐
405       thing found in the OSD directory as well.
406
407       Optionally, the JSON blob can be sent to stdout for further inspection.
408
409       Usage on all running OSDs:
410
411          ceph-volume simple scan
412
413       Usage on data devices:
414
415          ceph-volume simple scan <data device>
416
417       Running OSD directories:
418
419          ceph-volume simple scan <path to osd dir>
420
421       Optional arguments:
422
423       -h, --help
424              show the help message and exit
425
426       --stdout
427              Send the JSON blob to stdout
428
429       --force
430              If the JSON file exists at destination, overwrite it
431
432       Optional Positional arguments:
433
434       <DATA DEVICE or OSD DIR>
435              Actual data partition or a path to the running OSD
436
437   trigger
438       This  subcommand  is  not  meant to be used directly, and it is used by
439       systemd so that it proxies input  to  ceph-volume  simple  activate  by
440       parsing  the  input  from systemd, detecting the UUID and ID associated
441       with an OSD.
442
443       Usage:
444
445          ceph-volume simple trigger <SYSTEMD-DATA>
446
447       The systemd "data" is expected to be in the format of:
448
449          <OSD ID>-<OSD UUID>
450
451       The JSON file associated with the OSD need to have been persisted  pre‐
452       viously  by  a  scan  (or manually), so that all needed metadata can be
453       used.
454
455       Positional arguments:
456
457       <SYSTEMD_DATA>
458              Data from a systemd unit containing ID and UUID of the OSD.
459

AVAILABILITY

461       ceph-volume is part of Ceph, a massively  scalable,  open-source,  dis‐
462       tributed   storage   system.  Please  refer  to  the  documentation  at
463       http://docs.ceph.com/ for more information.
464

SEE ALSO

466       ceph-osd(8),
467
469       2010-2023, Inktank Storage, Inc. and contributors. Licensed under  Cre‐
470       ative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0)
471
472
473
474
475dev                              Nov 02, 2023                   CEPH-VOLUME(8)
Impressum