1CEPH-VOLUME(8) Ceph CEPH-VOLUME(8)
2
6 ceph-volume - Ceph OSD deployment and inspection tool
7
9 ceph-volume [-h] [--cluster CLUSTER] [--log-level LOG_LEVEL]
10 [--log-path LOG_PATH]
11 ceph-volume inventory
13 ceph-volume lvm [ trigger | create | activate | prepare | zap | list |
15 batch | new-wal | new-db | migrate ]
16 ceph-volume simple [ trigger | scan | activate ]
18
20 ceph-volume is a single purpose command line tool to deploy logical
21 volumes as OSDs, trying to maintain a similar API to ceph-disk when
22 preparing, activating, and creating OSDs.
23 It deviates from ceph-disk by not interacting or relying on the udev
25 rules that come installed for Ceph. These rules allow automatic detec‐
26 tion of previously setup devices that are in turn fed into ceph-disk to
27 activate them.
28
30 inventory
31 This subcommand provides information about a host's physical disc in‐
32 ventory and reports metadata about these discs. Among this metadata one
33 can find disc specific data items (like model, size, rotational or
34 solid state) as well as data items specific to ceph using a device,
35 such as if it is available for use with ceph or if logical volumes are
36 present.
37 Examples:
39 ceph-volume inventory
41 ceph-volume inventory /dev/sda
42 ceph-volume inventory --format json-pretty
43 Optional arguments:
45 -h, --help
47 show the help message and exit
48 --format
50 report format, valid values are plain (default), json and
51 json-pretty
52 lvm
54 By making use of LVM tags, the lvm sub-command is able to store and
55 later re-discover and query devices associated with OSDs so that they
56 can later activated.
57 Subcommands:
59 batch
61 Creates OSDs from a list of devices using a filestore or bluestore (de‐
62 fault) setup. It will create all necessary volume groups and logical
63 volumes required to have a working OSD.
64 Example usage with three devices:
66 ceph-volume lvm batch --bluestore /dev/sda /dev/sdb /dev/sdc
68 Optional arguments:
70 -h, --help
72 show the help message and exit
73 --bluestore
75 Use the bluestore objectstore (default)
76 --filestore
78 Use the filestore objectstore
79 --yes Skip the report and prompt to continue provisioning
81 --prepare
83 Only prepare OSDs, do not activate
84 --dmcrypt
86 Enable encryption for the underlying OSD devices
87 --crush-device-class
89 Define a CRUSH device class to assign the OSD to
90 --no-systemd
92 Do not enable or create any systemd units
93 --osds-per-device
95 Provision more than 1 (the default) OSD per device
96 --report
98 Report what the potential outcome would be for the current input
99 (requires devices to be passed in)
100 --format
102 Output format when reporting (used along with --report), can be
103 one of 'pretty' (default) or 'json'
104 --block-db-size
106 Set (or override) the "bluestore_block_db_size" value, in bytes
107 --journal-size
109 Override the "osd_journal_size" value, in megabytes
110 Required positional arguments:
112 <DEVICE>
114 Full path to a raw device, like /dev/sda. Multiple <DEVICE>
115 paths can be passed in.
116 activate
118 Enables a systemd unit that persists the OSD ID and its UUID (also
119 called fsid in Ceph CLI tools), so that at boot time it can understand
120 what OSD is enabled and needs to be mounted.
121 Usage:
123 ceph-volume lvm activate --bluestore <osd id> <osd fsid>
125 Optional Arguments:
127 -h, --help
129 show the help message and exit
130 --auto-detect-objectstore
132 Automatically detect the objectstore by inspecting the OSD
133 --bluestore
135 bluestore objectstore (default)
136 --filestore
138 filestore objectstore
139 --all Activate all OSDs found in the system
141 --no-systemd
143 Skip creating and enabling systemd units and starting of OSD
144 services
145 Multiple OSDs can be activated at once by using the (idempotent) --all
147 flag:
148 ceph-volume lvm activate --all
150 prepare
152 Prepares a logical volume to be used as an OSD and journal using a
153 filestore or bluestore (default) setup. It will not create or modify
154 the logical volumes except for adding extra metadata.
155 Usage:
157 ceph-volume lvm prepare --filestore --data <data lv> --journal <journal device>
159 Optional arguments:
161 -h, --help
163 show the help message and exit
164 --journal JOURNAL
166 logical group name, path to a logical volume, or path to a de‐
167 vice
168 --bluestore
170 Use the bluestore objectstore (default)
171 --block.wal
173 Path to a bluestore block.wal logical volume or partition
174 --block.db
176 Path to a bluestore block.db logical volume or partition
177 --filestore
179 Use the filestore objectstore
180 --dmcrypt
182 Enable encryption for the underlying OSD devices
183 --osd-id OSD_ID
185 Reuse an existing OSD id
186 --osd-fsid OSD_FSID
188 Reuse an existing OSD fsid
189 --crush-device-class
191 Define a CRUSH device class to assign the OSD to
192 Required arguments:
194 --data A logical group name or a path to a logical volume
196 For encrypting an OSD, the --dmcrypt flag must be added when preparing
198 (also supported in the create sub-command).
199 create
201 Wraps the two-step process to provision a new osd (calling prepare
202 first and then activate) into a single one. The reason to prefer pre‐
203 pare and then activate is to gradually introduce new OSDs into a clus‐
204 ter, and avoiding large amounts of data being rebalanced.
205 The single-call process unifies exactly what prepare and activate do,
207 with the convenience of doing it all at once. Flags and general usage
208 are equivalent to those of the prepare and activate subcommand.
209 trigger
211 This subcommand is not meant to be used directly, and it is used by
212 systemd so that it proxies input to ceph-volume lvm activate by parsing
213 the input from systemd, detecting the UUID and ID associated with an
214 OSD.
215 Usage:
217 ceph-volume lvm trigger <SYSTEMD-DATA>
219 The systemd "data" is expected to be in the format of:
221 <OSD ID>-<OSD UUID>
223 The lvs associated with the OSD need to have been prepared previously,
225 so that all needed tags and metadata exist.
226 Positional arguments:
228 <SYSTEMD_DATA>
230 Data from a systemd unit containing ID and UUID of the OSD.
231 list
233 List devices or logical volumes associated with Ceph. An association is
234 determined if a device has information relating to an OSD. This is ver‐
235 ified by querying LVM's metadata and correlating it with devices.
236 The lvs associated with the OSD need to have been prepared previously
238 by ceph-volume so that all needed tags and metadata exist.
239 Usage:
241 ceph-volume lvm list
243 List a particular device, reporting all metadata about it:
245 ceph-volume lvm list /dev/sda1
247 List a logical volume, along with all its metadata (vg is a volume
249 group, and lv the logical volume name):
250 ceph-volume lvm list {vg/lv}
252 Positional arguments:
254 <DEVICE>
256 Either in the form of vg/lv for logical volumes, /path/to/sda1
257 or /path/to/sda for regular devices.
258 zap
260 Zaps the given logical volume or partition. If given a path to a logi‐
261 cal volume it must be in the format of vg/lv. Any file systems present
262 on the given lv or partition will be removed and all data will be
263 purged.
264 However, the lv or partition will be kept intact.
266 Usage, for logical volumes:
268 ceph-volume lvm zap {vg/lv}
270 Usage, for logical partitions:
272 ceph-volume lvm zap /dev/sdc1
274 For full removal of the device use the --destroy flag (allowed for all
276 device types):
277 ceph-volume lvm zap --destroy /dev/sdc1
279 Multiple devices can be removed by specifying the OSD ID and/or the OSD
281 FSID:
282 ceph-volume lvm zap --destroy --osd-id 1
284 ceph-volume lvm zap --destroy --osd-id 1 --osd-fsid C9605912-8395-4D76-AFC0-7DFDAC315D59
285 Positional arguments:
287 <DEVICE>
289 Either in the form of vg/lv for logical volumes, /path/to/sda1
290 or /path/to/sda for regular devices.
291 new-wal
293 Attaches the given logical volume to OSD as a WAL. Logical volume name
294 format is vg/lv. Fails if OSD has already got attached WAL.
295 Usage:
297 ceph-volume lvm new-wal --osd-id OSD_ID --osd-fsid OSD_FSID --target <target lv>
299 Optional arguments:
301 -h, --help
303 show the help message and exit
304 --no-systemd
306 Skip checking OSD systemd unit
307 Required arguments:
309 --target
311 logical volume name to attach as WAL
312 new-db
314 Attaches the given logical volume to OSD as a DB. Logical volume name
315 format is vg/lv. Fails if OSD has already got attached DB.
316 Usage:
318 ceph-volume lvm new-db --osd-id OSD_ID --osd-fsid OSD_FSID --target <target lv>
320 Optional arguments:
322 -h, --help
324 show the help message and exit
325 --no-systemd
327 Skip checking OSD systemd unit
328 Required arguments:
330 --target
332 logical volume name to attach as DB
333 migrate
335 Moves BlueFS data from source volume(s) to the target one, source vol‐
336 umes (except the main, i.e. data or block one) are removed on success.
337 LVM volumes are permitted for Target only, both already attached or new
338 one. In the latter case it is attached to the OSD replacing one of the
339 source devices. Following replacement rules apply (in the order of
340 precedence, stop on the first match):
341 • if source list has DB volume - target device replaces it.
343 • if source list has WAL volume - target device replace it.
345 • if source list has slow volume only - operation is not permitted,
347 requires explicit allocation via new-db/new-wal command.
348 Usage:
350 ceph-volume lvm migrate --osd-id OSD_ID --osd-fsid OSD_FSID --target <target lv> --from {data|db|wal} [{data|db|wal} ...]
352 Optional arguments:
354 -h, --help
356 show the help message and exit
357 --no-systemd
359 Skip checking OSD systemd unit
360 Required arguments:
362 --from list of source device type names
364 --target
366 logical volume to move data to
367 simple
369 Scan legacy OSD directories or data devices that may have been created
370 by ceph-disk, or manually.
371 Subcommands:
373 activate
375 Enables a systemd unit that persists the OSD ID and its UUID (also
376 called fsid in Ceph CLI tools), so that at boot time it can understand
377 what OSD is enabled and needs to be mounted, while reading information
378 that was previously created and persisted at /etc/ceph/osd/ in JSON
379 format.
380 Usage:
382 ceph-volume simple activate --bluestore <osd id> <osd fsid>
384 Optional Arguments:
386 -h, --help
388 show the help message and exit
389 --bluestore
391 bluestore objectstore (default)
392 --filestore
394 filestore objectstore
395 NOTE:
397 It requires a matching JSON file with the following format:
398 /etc/ceph/osd/<osd id>-<osd fsid>.json
400 scan
402 Scan a running OSD or data device for an OSD for metadata that can
403 later be used to activate and manage the OSD with ceph-volume. The scan
404 method will create a JSON file with the required information plus any‐
405 thing found in the OSD directory as well.
406 Optionally, the JSON blob can be sent to stdout for further inspection.
408 Usage on all running OSDs:
410 ceph-volume simple scan
412 Usage on data devices:
414 ceph-volume simple scan <data device>
416 Running OSD directories:
418 ceph-volume simple scan <path to osd dir>
420 Optional arguments:
422 -h, --help
424 show the help message and exit
425 --stdout
427 Send the JSON blob to stdout
428 --force
430 If the JSON file exists at destination, overwrite it
431 Optional Positional arguments:
433 <DATA DEVICE or OSD DIR>
435 Actual data partition or a path to the running OSD
436 trigger
438 This subcommand is not meant to be used directly, and it is used by
439 systemd so that it proxies input to ceph-volume simple activate by
440 parsing the input from systemd, detecting the UUID and ID associated
441 with an OSD.
442 Usage:
444 ceph-volume simple trigger <SYSTEMD-DATA>
446 The systemd "data" is expected to be in the format of:
448 <OSD ID>-<OSD UUID>
450 The JSON file associated with the OSD need to have been persisted pre‐
452 viously by a scan (or manually), so that all needed metadata can be
453 used.
454 Positional arguments:
456 <SYSTEMD_DATA>
458 Data from a systemd unit containing ID and UUID of the OSD.
459
461 ceph-volume is part of Ceph, a massively scalable, open-source, dis‐
462 tributed storage system. Please refer to the documentation at
463 http://docs.ceph.com/ for more information.
464
466 ceph-osd(8),
467
469 2010-2023, Inktank Storage, Inc. and contributors. Licensed under Cre‐
470 ative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0)
471dev Nov 02, 2023 CEPH-VOLUME(8)