1CEPH-VOLUME(8) Ceph CEPH-VOLUME(8)
2
6 ceph-volume - Ceph OSD deployment and inspection tool
7
9 ceph-volume [-h] [--cluster CLUSTER] [--log-level LOG_LEVEL]
10 [--log-path LOG_PATH]
11 ceph-volume inventory
13 ceph-volume lvm [ trigger | create | activate | prepare | zap | list |
15 batch | new-wal | new-db | migrate ]
16 ceph-volume simple [ trigger | scan | activate ]
18
20 ceph-volume is a single purpose command line tool to deploy logical
21 volumes as OSDs, trying to maintain a similar API to ceph-disk when
22 preparing, activating, and creating OSDs.
23 It deviates from ceph-disk by not interacting or relying on the udev
25 rules that come installed for Ceph. These rules allow automatic detec‐
26 tion of previously setup devices that are in turn fed into ceph-disk to
27 activate them.
28
30 inventory
31 This subcommand provides information about a host's physical disc in‐
32 ventory and reports metadata about these discs. Among this metadata one
33 can find disc specific data items (like model, size, rotational or
34 solid state) as well as data items specific to ceph using a device,
35 such as if it is available for use with ceph or if logical volumes are
36 present.
37 Examples:
39 ceph-volume inventory
41 ceph-volume inventory /dev/sda
42 ceph-volume inventory --format json-pretty
43 Optional arguments:
45 -h, --help
47 show the help message and exit
48 --format
50 report format, valid values are plain (default), json and
51 json-pretty
52 lvm
54 By making use of LVM tags, the lvm sub-command is able to store and
55 later re-discover and query devices associated with OSDs so that they
56 can later activated.
57 Subcommands:
59 batch
61 Creates OSDs from a list of devices using a bluestore (default) setup.
62 It will create all necessary volume groups and logical volumes required
63 to have a working OSD.
64 Example usage with three devices:
66 ceph-volume lvm batch --bluestore /dev/sda /dev/sdb /dev/sdc
68 Optional arguments:
70 -h, --help
72 show the help message and exit
73 --bluestore
75 Use the bluestore objectstore (default)
76 --yes Skip the report and prompt to continue provisioning
78 --prepare
80 Only prepare OSDs, do not activate
81 --dmcrypt
83 Enable encryption for the underlying OSD devices
84 --crush-device-class
86 Define a CRUSH device class to assign the OSD to
87 --no-systemd
89 Do not enable or create any systemd units
90 --osds-per-device
92 Provision more than 1 (the default) OSD per device
93 --report
95 Report what the potential outcome would be for the current input
96 (requires devices to be passed in)
97 --format
99 Output format when reporting (used along with --report), can be
100 one of 'pretty' (default) or 'json'
101 --block-db-size
103 Set (or override) the "bluestore_block_db_size" value, in bytes
104 --journal-size
106 Override the "osd_journal_size" value, in megabytes
107 Required positional arguments:
109 <DEVICE>
111 Full path to a raw device, like /dev/sda. Multiple <DEVICE>
112 paths can be passed in.
113 activate
115 Enables a systemd unit that persists the OSD ID and its UUID (also
116 called fsid in Ceph CLI tools), so that at boot time it can understand
117 what OSD is enabled and needs to be mounted.
118 Usage:
120 ceph-volume lvm activate --bluestore <osd id> <osd fsid>
122 Optional Arguments:
124 -h, --help
126 show the help message and exit
127 --auto-detect-objectstore
129 Automatically detect the objectstore by inspecting the OSD
130 --bluestore
132 bluestore objectstore (default)
133 --all Activate all OSDs found in the system
135 --no-systemd
137 Skip creating and enabling systemd units and starting of OSD
138 services
139 Multiple OSDs can be activated at once by using the (idempotent) --all
141 flag:
142 ceph-volume lvm activate --all
144 prepare
146 Prepares a logical volume to be used as an OSD and journal using a
147 bluestore (default) setup. It will not create or modify the logical
148 volumes except for adding extra metadata.
149 Usage:
151 ceph-volume lvm prepare --bluestore --data <data lv> --journal <journal device>
153 Optional arguments:
155 -h, --help
157 show the help message and exit
158 --journal JOURNAL
160 logical group name, path to a logical volume, or path to a de‐
161 vice
162 --bluestore
164 Use the bluestore objectstore (default)
165 --block.wal
167 Path to a bluestore block.wal logical volume or partition
168 --block.db
170 Path to a bluestore block.db logical volume or partition
171 --dmcrypt
173 Enable encryption for the underlying OSD devices
174 --osd-id OSD_ID
176 Reuse an existing OSD id
177 --osd-fsid OSD_FSID
179 Reuse an existing OSD fsid
180 --crush-device-class
182 Define a CRUSH device class to assign the OSD to
183 Required arguments:
185 --data A logical group name or a path to a logical volume
187 For encrypting an OSD, the --dmcrypt flag must be added when preparing
189 (also supported in the create sub-command).
190 create
192 Wraps the two-step process to provision a new osd (calling prepare
193 first and then activate) into a single one. The reason to prefer pre‐
194 pare and then activate is to gradually introduce new OSDs into a clus‐
195 ter, and avoiding large amounts of data being rebalanced.
196 The single-call process unifies exactly what prepare and activate do,
198 with the convenience of doing it all at once. Flags and general usage
199 are equivalent to those of the prepare and activate subcommand.
200 trigger
202 This subcommand is not meant to be used directly, and it is used by
203 systemd so that it proxies input to ceph-volume lvm activate by parsing
204 the input from systemd, detecting the UUID and ID associated with an
205 OSD.
206 Usage:
208 ceph-volume lvm trigger <SYSTEMD-DATA>
210 The systemd "data" is expected to be in the format of:
212 <OSD ID>-<OSD UUID>
214 The lvs associated with the OSD need to have been prepared previously,
216 so that all needed tags and metadata exist.
217 Positional arguments:
219 <SYSTEMD_DATA>
221 Data from a systemd unit containing ID and UUID of the OSD.
222 list
224 List devices or logical volumes associated with Ceph. An association is
225 determined if a device has information relating to an OSD. This is ver‐
226 ified by querying LVM's metadata and correlating it with devices.
227 The lvs associated with the OSD need to have been prepared previously
229 by ceph-volume so that all needed tags and metadata exist.
230 Usage:
232 ceph-volume lvm list
234 List a particular device, reporting all metadata about it:
236 ceph-volume lvm list /dev/sda1
238 List a logical volume, along with all its metadata (vg is a volume
240 group, and lv the logical volume name):
241 ceph-volume lvm list {vg/lv}
243 Positional arguments:
245 <DEVICE>
247 Either in the form of vg/lv for logical volumes, /path/to/sda1
248 or /path/to/sda for regular devices.
249 zap
251 Zaps the given logical volume or partition. If given a path to a logi‐
252 cal volume it must be in the format of vg/lv. Any file systems present
253 on the given lv or partition will be removed and all data will be
254 purged.
255 However, the lv or partition will be kept intact.
257 Usage, for logical volumes:
259 ceph-volume lvm zap {vg/lv}
261 Usage, for logical partitions:
263 ceph-volume lvm zap /dev/sdc1
265 For full removal of the device use the --destroy flag (allowed for all
267 device types):
268 ceph-volume lvm zap --destroy /dev/sdc1
270 Multiple devices can be removed by specifying the OSD ID and/or the OSD
272 FSID:
273 ceph-volume lvm zap --destroy --osd-id 1
275 ceph-volume lvm zap --destroy --osd-id 1 --osd-fsid C9605912-8395-4D76-AFC0-7DFDAC315D59
276 Positional arguments:
278 <DEVICE>
280 Either in the form of vg/lv for logical volumes, /path/to/sda1
281 or /path/to/sda for regular devices.
282 new-wal
284 Attaches the given logical volume to OSD as a WAL. Logical volume name
285 format is vg/lv. Fails if OSD has already got attached WAL.
286 Usage:
288 ceph-volume lvm new-wal --osd-id OSD_ID --osd-fsid OSD_FSID --target <target lv>
290 Optional arguments:
292 -h, --help
294 show the help message and exit
295 --no-systemd
297 Skip checking OSD systemd unit
298 Required arguments:
300 --target
302 logical volume name to attach as WAL
303 new-db
305 Attaches the given logical volume to OSD as a DB. Logical volume name
306 format is vg/lv. Fails if OSD has already got attached DB.
307 Usage:
309 ceph-volume lvm new-db --osd-id OSD_ID --osd-fsid OSD_FSID --target <target lv>
311 Optional arguments:
313 -h, --help
315 show the help message and exit
316 --no-systemd
318 Skip checking OSD systemd unit
319 Required arguments:
321 --target
323 logical volume name to attach as DB
324 migrate
326 Moves BlueFS data from source volume(s) to the target one, source vol‐
327 umes (except the main, i.e. data or block one) are removed on success.
328 LVM volumes are permitted for Target only, both already attached or new
329 one. In the latter case it is attached to the OSD replacing one of the
330 source devices. Following replacement rules apply (in the order of
331 precedence, stop on the first match):
332 • if source list has DB volume - target device replaces it.
334 • if source list has WAL volume - target device replace it.
336 • if source list has slow volume only - operation is not permitted,
338 requires explicit allocation via new-db/new-wal command.
339 Usage:
341 ceph-volume lvm migrate --osd-id OSD_ID --osd-fsid OSD_FSID --target <target lv> --from {data|db|wal} [{data|db|wal} ...]
343 Optional arguments:
345 -h, --help
347 show the help message and exit
348 --no-systemd
350 Skip checking OSD systemd unit
351 Required arguments:
353 --from list of source device type names
355 --target
357 logical volume to move data to
358 simple
360 Scan legacy OSD directories or data devices that may have been created
361 by ceph-disk, or manually.
362 Subcommands:
364 activate
366 Enables a systemd unit that persists the OSD ID and its UUID (also
367 called fsid in Ceph CLI tools), so that at boot time it can understand
368 what OSD is enabled and needs to be mounted, while reading information
369 that was previously created and persisted at /etc/ceph/osd/ in JSON
370 format.
371 Usage:
373 ceph-volume simple activate --bluestore <osd id> <osd fsid>
375 Optional Arguments:
377 -h, --help
379 show the help message and exit
380 --bluestore
382 bluestore objectstore (default)
383 NOTE:
385 It requires a matching JSON file with the following format:
386 /etc/ceph/osd/<osd id>-<osd fsid>.json
388 scan
390 Scan a running OSD or data device for an OSD for metadata that can
391 later be used to activate and manage the OSD with ceph-volume. The scan
392 method will create a JSON file with the required information plus any‐
393 thing found in the OSD directory as well.
394 Optionally, the JSON blob can be sent to stdout for further inspection.
396 Usage on all running OSDs:
398 ceph-volume simple scan
400 Usage on data devices:
402 ceph-volume simple scan <data device>
404 Running OSD directories:
406 ceph-volume simple scan <path to osd dir>
408 Optional arguments:
410 -h, --help
412 show the help message and exit
413 --stdout
415 Send the JSON blob to stdout
416 --force
418 If the JSON file exists at destination, overwrite it
419 Optional Positional arguments:
421 <DATA DEVICE or OSD DIR>
423 Actual data partition or a path to the running OSD
424 trigger
426 This subcommand is not meant to be used directly, and it is used by
427 systemd so that it proxies input to ceph-volume simple activate by
428 parsing the input from systemd, detecting the UUID and ID associated
429 with an OSD.
430 Usage:
432 ceph-volume simple trigger <SYSTEMD-DATA>
434 The systemd "data" is expected to be in the format of:
436 <OSD ID>-<OSD UUID>
438 The JSON file associated with the OSD need to have been persisted pre‐
440 viously by a scan (or manually), so that all needed metadata can be
441 used.
442 Positional arguments:
444 <SYSTEMD_DATA>
446 Data from a systemd unit containing ID and UUID of the OSD.
447
449 ceph-volume is part of Ceph, a massively scalable, open-source, dis‐
450 tributed storage system. Please refer to the documentation at
451 http://docs.ceph.com/ for more information.
452
454 ceph-osd(8),
455
457 2010-2023, Inktank Storage, Inc. and contributors. Licensed under Cre‐
458 ative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0)
459dev Nov 15, 2023 CEPH-VOLUME(8)