ceph-volume(8)

1CEPH-VOLUME(8)                       Ceph                       CEPH-VOLUME(8)
2
3
4

NAME

6       ceph-volume - Ceph OSD deployment and inspection tool
7

SYNOPSIS

9       ceph-volume [-h] [--cluster CLUSTER] [--log-level LOG_LEVEL]
10         [--log-path LOG_PATH]
11
12       ceph-volume inventory
13
14       ceph-volume lvm [ trigger | create | activate | prepare
15       zap | list | batch | new-wal | new-db | migrate ]
16
17       ceph-volume simple [ trigger | scan | activate ]
18
19

DESCRIPTION

21       ceph-volume  is  a  single  purpose command line tool to deploy logical
22       volumes as OSDs, trying to maintain a similar  API  to  ceph-disk  when
23       preparing, activating, and creating OSDs.
24
25       It  deviates  from  ceph-disk by not interacting or relying on the udev
26       rules that come installed for Ceph. These rules allow automatic  detec‐
27       tion of previously setup devices that are in turn fed into ceph-disk to
28       activate them.
29

COMMANDS

31   inventory
32       This subcommand provides information about a host's physical  disc  in‐
33       ventory and reports metadata about these discs. Among this metadata one
34       can find disc specific data items  (like  model,  size,  rotational  or
35       solid  state)  as  well  as data items specific to ceph using a device,
36       such as if it is available for use with ceph or if logical volumes  are
37       present.
38
39       Examples:
40
41          ceph-volume inventory
42          ceph-volume inventory /dev/sda
43          ceph-volume inventory --format json-pretty
44
45       Optional arguments:
46
47       • [-h, --help]          show the help message and exit
48
49       •
50
51         [--format] report format, valid values are plain (default),
52                json and json-pretty
53
54   lvm
55       By  making  use  of  LVM tags, the lvm sub-command is able to store and
56       later re-discover and query devices associated with OSDs so  that  they
57       can later activated.
58
59       Subcommands:
60
61       batch  Creates  OSDs  from a list of devices using a filestore or blue‐
62       store (default) setup. It will create all necessary volume  groups  and
63       logical volumes required to have a working OSD.
64
65       Example usage with three devices:
66
67          ceph-volume lvm batch --bluestore /dev/sda /dev/sdb /dev/sdc
68
69       Optional arguments:
70
71       • [-h, --help]          show the help message and exit
72
73       • [--bluestore]         Use the bluestore objectstore (default)
74
75       • [--filestore]         Use the filestore objectstore
76
77       • [--yes]                Skip  the report and prompt to continue provi‐
78         sioning
79
80       • [--prepare]           Only prepare OSDs, do not activate
81
82       • [--dmcrypt]           Enable encryption for the  underlying  OSD  de‐
83         vices
84
85       • [--crush-device-class]  Define a CRUSH device class to assign the OSD
86         to
87
88       • [--no-systemd]         Do not enable or create any systemd units
89
90       • [--osds-per-device]   Provision more than 1 (the default) OSD per de‐
91         vice
92
93       • [--report]         Report what the potential outcome would be for the
94         current input (requires devices to be passed in)
95
96       • [--format]         Output format  when  reporting  (used  along  with
97         --report), can be one of 'pretty' (default) or 'json'
98
99       • [--block-db-size]     Set (or override) the "bluestore_block_db_size"
100         value, in bytes
101
102       • [--journal-size]       Override  the  "osd_journal_size"  value,   in
103         megabytes
104
105       Required positional arguments:
106
107       •
108
109         <DEVICE> Full path to a raw device, like /dev/sda. Multiple
110                <DEVICE> paths can be passed in.
111
112       activate  Enables  a systemd unit that persists the OSD ID and its UUID
113       (also called fsid in Ceph CLI tools), so that at boot time it  can  un‐
114       derstand what OSD is enabled and needs to be mounted.
115
116       Usage:
117
118          ceph-volume lvm activate --bluestore <osd id> <osd fsid>
119
120       Optional Arguments:
121
122       • [-h, --help]  show the help message and exit
123
124       • [--auto-detect-objectstore]  Automatically  detect the objectstore by
125         inspecting the OSD
126
127       • [--bluestore] bluestore objectstore (default)
128
129       • [--filestore] filestore objectstore
130
131       • [--all] Activate all OSDs found in the system
132
133       • [--no-systemd] Skip creating and enabling systemd units and  starting
134         of OSD services
135
136       Multiple  OSDs can be activated at once by using the (idempotent) --all
137       flag:
138
139          ceph-volume lvm activate --all
140
141       prepare Prepares a logical volume to be used as an OSD and journal  us‐
142       ing  a  filestore  or  bluestore (default) setup. It will not create or
143       modify the logical volumes except for adding extra metadata.
144
145       Usage:
146
147          ceph-volume lvm prepare --filestore --data <data lv> --journal <journal device>
148
149       Optional arguments:
150
151       • [-h, --help]          show the help message and exit
152
153       • [--journal JOURNAL]   A logical group name, path to a logical volume,
154         or path to a device
155
156       • [--bluestore]         Use the bluestore objectstore (default)
157
158       • [--block.wal]         Path to a bluestore block.wal logical volume or
159         partition
160
161       • [--block.db]          Path to a bluestore block.db logical volume  or
162         partition
163
164       • [--filestore]         Use the filestore objectstore
165
166       • [--dmcrypt]            Enable  encryption  for the underlying OSD de‐
167         vices
168
169       • [--osd-id OSD_ID]     Reuse an existing OSD id
170
171       • [--osd-fsid OSD_FSID] Reuse an existing OSD fsid
172
173       • [--crush-device-class] Define a CRUSH device class to assign the  OSD
174         to
175
176       Required arguments:
177
178       •
179
180         --data A logical group name or a path to a logical volume
181
182       For  encrypting an OSD, the --dmcrypt flag must be added when preparing
183       (also supported in the create sub-command).
184
185       create Wraps the two-step process to provision a new osd (calling  pre‐
186       pare  first  and then activate) into a single one. The reason to prefer
187       prepare and then activate is to gradually introduce  new  OSDs  into  a
188       cluster, and avoiding large amounts of data being rebalanced.
189
190       The  single-call  process unifies exactly what prepare and activate do,
191       with the convenience of doing it all at once. Flags and  general  usage
192       are equivalent to those of the prepare and activate subcommand.
193
194       trigger  This  subcommand  is  not meant to be used directly, and it is
195       used by systemd so that it proxies input to ceph-volume lvm activate by
196       parsing  the  input  from systemd, detecting the UUID and ID associated
197       with an OSD.
198
199       Usage:
200
201          ceph-volume lvm trigger <SYSTEMD-DATA>
202
203       The systemd "data" is expected to be in the format of:
204
205          <OSD ID>-<OSD UUID>
206
207       The lvs associated with the OSD need to have been prepared  previously,
208       so that all needed tags and metadata exist.
209
210       Positional arguments:
211
212       • <SYSTEMD_DATA>   Data  from  a systemd unit containing ID and UUID of
213         the OSD.
214
215       list List devices or logical volumes associated with Ceph. An  associa‐
216       tion is determined if a device has information relating to an OSD. This
217       is verified by querying LVM's metadata and correlating it with devices.
218
219       The lvs associated with the OSD need to have been  prepared  previously
220       by ceph-volume so that all needed tags and metadata exist.
221
222       Usage:
223
224          ceph-volume lvm list
225
226       List a particular device, reporting all metadata about it:
227
228          ceph-volume lvm list /dev/sda1
229
230       List  a  logical  volume,  along  with all its metadata (vg is a volume
231       group, and lv the logical volume name):
232
233          ceph-volume lvm list {vg/lv}
234
235       Positional arguments:
236
237       • <DEVICE>   Either  in  the  form  of  vg/lv  for   logical   volumes,
238         /path/to/sda1 or /path/to/sda for regular devices.
239
240       zap  Zaps  the  given logical volume or partition. If given a path to a
241       logical volume it must be in the format  of  vg/lv.  Any  file  systems
242       present  on the given lv or partition will be removed and all data will
243       be purged.
244
245       However, the lv or partition will be kept intact.
246
247       Usage, for logical volumes:
248
249          ceph-volume lvm zap {vg/lv}
250
251       Usage, for logical partitions:
252
253          ceph-volume lvm zap /dev/sdc1
254
255       For full removal of the device use the --destroy flag (allowed for  all
256       device types):
257
258          ceph-volume lvm zap --destroy /dev/sdc1
259
260       Multiple devices can be removed by specifying the OSD ID and/or the OSD
261       FSID:
262
263          ceph-volume lvm zap --destroy --osd-id 1
264          ceph-volume lvm zap --destroy --osd-id 1 --osd-fsid C9605912-8395-4D76-AFC0-7DFDAC315D59
265
266       Positional arguments:
267
268       • <DEVICE>   Either  in  the  form  of  vg/lv  for   logical   volumes,
269         /path/to/sda1 or /path/to/sda for regular devices.
270
271   new-wal
272       Attaches  the given logical volume to OSD as a WAL. Logical volume name
273       format is vg/lv. Fails if OSD has already got attached WAL.
274
275       Usage:
276
277          ceph-volume lvm new-wal --osd-id OSD_ID --osd-fsid OSD_FSID --target <target lv>
278
279       Optional arguments:
280
281       -h, --help
282              show the help message and exit
283
284       --no-systemd
285              Skip checking OSD systemd unit
286
287       Required arguments:
288
289       --target
290              logical volume name to attach as WAL
291
292   new-db
293       Attaches the given logical volume to OSD as a DB. Logical  volume  name
294       format is vg/lv. Fails if OSD has already got attached DB.
295
296       Usage:
297
298          ceph-volume lvm new-db --osd-id OSD_ID --osd-fsid OSD_FSID --target <target lv>
299
300       Optional arguments:
301
302       -h, --help
303              show the help message and exit
304
305       --no-systemd
306              Skip checking OSD systemd unit
307
308       Required arguments:
309
310       --target
311              logical volume name to attach as DB
312
313   migrate
314       Moves  BlueFS data from source volume(s) to the target one, source vol‐
315       umes (except the main, i.e. data or block one) are removed on  success.
316       LVM volumes are permitted for Target only, both already attached or new
317       one. In the latter case it is attached to the OSD replacing one of  the
318       source  devices.  Following  replacement  rules  apply (in the order of
319       precedence, stop on the first match):
320
321          • if source list has DB volume - target device replaces it.
322
323          • if source list has WAL volume - target device replace it.
324
325          • if source list has slow volume only - operation is not  permitted,
326            requires explicit allocation via new-db/new-wal command.
327
328       Usage:
329
330          ceph-volume lvm migrate --osd-id OSD_ID --osd-fsid OSD_FSID --target <target lv> --from {data|db|wal} [{data|db|wal} ...]
331
332       Optional arguments:
333
334       -h, --help
335              show the help message and exit
336
337       --no-systemd
338              Skip checking OSD systemd unit
339
340       Required arguments:
341
342       --from list of source device type names
343
344       --target
345              logical volume to move data to
346
347   simple
348       Scan  legacy OSD directories or data devices that may have been created
349       by ceph-disk, or manually.
350
351       Subcommands:
352
353       activate Enables a systemd unit that persists the OSD ID and  its  UUID
354       (also  called  fsid in Ceph CLI tools), so that at boot time it can un‐
355       derstand what OSD is enabled and needs to be mounted, while reading in‐
356       formation  that  was previously created and persisted at /etc/ceph/osd/
357       in JSON format.
358
359       Usage:
360
361          ceph-volume simple activate --bluestore <osd id> <osd fsid>
362
363       Optional Arguments:
364
365       • [-h, --help]  show the help message and exit
366
367       • [--bluestore] bluestore objectstore (default)
368
369       • [--filestore] filestore objectstore
370
371       Note: It requires a matching JSON file with the following format:
372
373          /etc/ceph/osd/<osd id>-<osd fsid>.json
374
375       scan Scan a running OSD or data device for an OSD for metadata that can
376       later be used to activate and manage the OSD with ceph-volume. The scan
377       method will create a JSON file with the required information plus  any‐
378       thing found in the OSD directory as well.
379
380       Optionally, the JSON blob can be sent to stdout for further inspection.
381
382       Usage on all running OSDs:
383
384          ceph-volume simple scan
385
386       Usage on data devices:
387
388          ceph-volume simple scan <data device>
389
390       Running OSD directories:
391
392          ceph-volume simple scan <path to osd dir>
393
394       Optional arguments:
395
396       • [-h, --help]          show the help message and exit
397
398       • [--stdout]            Send the JSON blob to stdout
399
400       • [--force]              If  the JSON file exists at destination, over‐
401         write it
402
403       Optional Positional arguments:
404
405       • <DATA DEVICE or OSD DIR>  Actual data partition or a path to the run‐
406         ning OSD
407
408       trigger  This  subcommand  is  not meant to be used directly, and it is
409       used by systemd so that it proxies input to ceph-volume simple activate
410       by parsing the input from systemd, detecting the UUID and ID associated
411       with an OSD.
412
413       Usage:
414
415          ceph-volume simple trigger <SYSTEMD-DATA>
416
417       The systemd "data" is expected to be in the format of:
418
419          <OSD ID>-<OSD UUID>
420
421       The JSON file associated with the OSD need to have been persisted  pre‐
422       viously  by  a  scan  (or manually), so that all needed metadata can be
423       used.
424
425       Positional arguments:
426
427       • <SYSTEMD_DATA>  Data from a systemd unit containing ID  and  UUID  of
428         the OSD.
429

AVAILABILITY

431       ceph-volume  is  part  of Ceph, a massively scalable, open-source, dis‐
432       tributed  storage  system.  Please  refer  to  the   documentation   at
433       http://docs.ceph.com/ for more information.
434

COPYRIGHT

439       2010-2021,  Inktank Storage, Inc. and contributors. Licensed under Cre‐
440       ative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0)
441
442
443
444
445dev                              Sep 28, 2021                   CEPH-VOLUME(8)

NAME

SYNOPSIS

DESCRIPTION

COMMANDS

AVAILABILITY

SEE ALSO

COPYRIGHT