1RBD(8) Ceph RBD(8)
2
3
4
6 rbd - manage rados block device (RBD) images
7
9 rbd [ -c ceph.conf ] [ -m monaddr ] [--cluster cluster-name]
10 [ -p | --pool pool ] [ command ... ]
11
12
14 rbd is a utility for manipulating rados block device (RBD) images, used
15 by the Linux rbd driver and the rbd storage driver for QEMU/KVM. RBD
16 images are simple block devices that are striped over objects and
17 stored in a RADOS object store. The size of the objects the image is
18 striped over must be a power of two.
19
21 -c ceph.conf, --conf ceph.conf
22 Use ceph.conf configuration file instead of the default
23 /etc/ceph/ceph.conf to determine monitor addresses during
24 startup.
25
26 -m monaddress[:port]
27 Connect to specified monitor (instead of looking through
28 ceph.conf).
29
30 --cluster cluster-name
31 Use different cluster name as compared to default cluster name
32 ceph.
33
34 -p pool-name, --pool pool-name
35 Interact with the given pool. Required by most commands.
36
37 --namespace namespace-name
38 Use a pre-defined image namespace within a pool
39
40 --no-progress
41 Do not output progress information (goes to standard error by
42 default for some commands).
43
45 --image-format format-id
46 Specifies which object layout to use. The default is 2.
47
48 • format 1 - (deprecated) Use the original format for a new rbd
49 image. This format is understood by all versions of librbd and
50 the kernel rbd module, but does not support newer features
51 like cloning.
52
53 • format 2 - Use the second rbd format, which is supported by
54 librbd since the Bobtail release and the kernel rbd module
55 since kernel 3.10 (except for "fancy" striping, which is sup‐
56 ported since kernel 4.17). This adds support for cloning and
57 is more easily extensible to allow more features in the fu‐
58 ture.
59
60 -s size-in-M/G/T, --size size-in-M/G/T
61 Specifies the size of the new rbd image or the new size of the
62 existing rbd image in M/G/T. If no suffix is given, unit M is
63 assumed.
64
65 --object-size size-in-B/K/M
66 Specifies the object size in B/K/M. Object size will be rounded
67 up the nearest power of two; if no suffix is given, unit B is
68 assumed. The default object size is 4M, smallest is 4K and max‐
69 imum is 32M.
70
71 The default value can be changed with the configuration option
72 rbd_default_order, which takes a power of two (default object
73 size is 2 ^ rbd_default_order).
74
75 --stripe-unit size-in-B/K/M
76 Specifies the stripe unit size in B/K/M. If no suffix is given,
77 unit B is assumed. See striping section (below) for more de‐
78 tails.
79
80 --stripe-count num
81 Specifies the number of objects to stripe over before looping
82 back to the first object. See striping section (below) for more
83 details.
84
85 --snap snap
86 Specifies the snapshot name for the specific operation.
87
88 --id username
89 Specifies the username (without the client. prefix) to use with
90 the map command.
91
92 --keyring filename
93 Specifies a keyring file containing a secret for the specified
94 user to use with the map command. If not specified, the default
95 keyring locations will be searched.
96
97 --keyfile filename
98 Specifies a file containing the secret key of --id user to use
99 with the map command. This option is overridden by --keyring if
100 the latter is also specified.
101
102 --shared lock-tag
103 Option for lock add that allows multiple clients to lock the
104 same image if they use the same tag. The tag is an arbitrary
105 string. This is useful for situations where an image must be
106 open from more than one client at once, like during live migra‐
107 tion of a virtual machine, or for use underneath a clustered
108 file system.
109
110 --format format
111 Specifies output formatting (default: plain, json, xml)
112
113 --pretty-format
114 Make json or xml formatted output more human-readable.
115
116 -o krbd-options, --options krbd-options
117 Specifies which options to use when mapping or unmapping an im‐
118 age via the rbd kernel driver. krbd-options is a comma-sepa‐
119 rated list of options (similar to mount(8) mount options). See
120 kernel rbd (krbd) options section below for more details.
121
122 --read-only
123 Map the image read-only. Equivalent to -o ro.
124
125 --image-feature feature-name
126 Specifies which RBD format 2 feature should be enabled when cre‐
127 ating an image. Multiple features can be enabled by repeating
128 this option multiple times. The following features are sup‐
129 ported:
130
131 • layering: layering support
132
133 • striping: striping v2 support
134
135 • exclusive-lock: exclusive locking support
136
137 • object-map: object map support (requires exclusive-lock)
138
139 • fast-diff: fast diff calculations (requires object-map)
140
141 • deep-flatten: snapshot flatten support
142
143 • journaling: journaled IO support (requires exclusive-lock)
144
145 • data-pool: erasure coded pool support
146
147 --image-shared
148 Specifies that the image will be used concurrently by multiple
149 clients. This will disable features that are dependent upon ex‐
150 clusive ownership of the image.
151
152 --whole-object
153 Specifies that the diff should be limited to the extents of a
154 full object instead of showing intra-object deltas. When the ob‐
155 ject map feature is enabled on an image, limiting the diff to
156 the object extents will dramatically improve performance since
157 the differences can be computed by examining the in-memory ob‐
158 ject map instead of querying RADOS for each object within the
159 image.
160
161 --limit
162 Specifies the limit for the number of snapshots permitted.
163
165 bench --io-type <read | write | readwrite | rw> [--io-size
166 size-in-B/K/M/G/T] [--io-threads num-ios-in-flight] [--io-total
167 size-in-B/K/M/G/T] [--io-pattern seq | rand] [--rw-mix-read read pro‐
168 portion in readwrite] image-spec
169 Generate a series of IOs to the image and measure the IO
170 throughput and latency. If no suffix is given, unit B is as‐
171 sumed for both --io-size and --io-total. Defaults are:
172 --io-size 4096, --io-threads 16, --io-total 1G, --io-pattern
173 seq, --rw-mix-read 50.
174
175 children snap-spec
176 List the clones of the image at the given snapshot. This checks
177 every pool, and outputs the resulting poolname/imagename.
178
179 This requires image format 2.
180
181 clone [--object-size size-in-B/K/M] [--stripe-unit size-in-B/K/M
182 --stripe-count num] [--image-feature feature-name] [--image-shared]
183 parent-snap-spec child-image-spec
184 Will create a clone (copy-on-write child) of the parent snap‐
185 shot. Object size will be identical to that of the parent image
186 unless specified. Size will be the same as the parent snapshot.
187 The --stripe-unit and --stripe-count arguments are optional, but
188 must be used together.
189
190 The parent snapshot must be protected (see rbd snap protect).
191 This requires image format 2.
192
193 config global get config-entity key
194 Get a global-level configuration override.
195
196 config global list [--format plain | json | xml] [--pretty-format] con‐
197 fig-entity
198 List global-level configuration overrides.
199
200 config global set config-entity key value
201 Set a global-level configuration override.
202
203 config global remove config-entity key
204 Remove a global-level configuration override.
205
206 config image get image-spec key
207 Get an image-level configuration override.
208
209 config image list [--format plain | json | xml] [--pretty-format] im‐
210 age-spec
211 List image-level configuration overrides.
212
213 config image set image-spec key value
214 Set an image-level configuration override.
215
216 config image remove image-spec key
217 Remove an image-level configuration override.
218
219 config pool get pool-name key
220 Get a pool-level configuration override.
221
222 config pool list [--format plain | json | xml] [--pretty-format]
223 pool-name
224 List pool-level configuration overrides.
225
226 config pool set pool-name key value
227 Set a pool-level configuration override.
228
229 config pool remove pool-name key
230 Remove a pool-level configuration override.
231
232 cp (src-image-spec | src-snap-spec) dest-image-spec
233 Copy the content of a src-image into the newly created dest-im‐
234 age. dest-image will have the same size, object size, and image
235 format as src-image. Note: snapshots are not copied, use deep
236 cp command to include snapshots.
237
238 create (-s | --size size-in-M/G/T) [--image-format format-id] [--ob‐
239 ject-size size-in-B/K/M] [--stripe-unit size-in-B/K/M --stripe-count
240 num] [--thick-provision] [--no-progress] [--image-feature fea‐
241 ture-name]... [--image-shared] image-spec
242 Will create a new rbd image. You must also specify the size via
243 --size. The --stripe-unit and --stripe-count arguments are op‐
244 tional, but must be used together. If the --thick-provision is
245 enabled, it will fully allocate storage for the image at cre‐
246 ation time. It will take a long time to do. Note: thick provi‐
247 sioning requires zeroing the contents of the entire image.
248
249 deep cp (src-image-spec | src-snap-spec) dest-image-spec
250 Deep copy the content of a src-image into the newly created
251 dest-image. Dest-image will have the same size, object size,
252 image format, and snapshots as src-image.
253
254 device list [-t | --device-type device-type] [--format plain | json |
255 xml] --pretty-format
256 Show the rbd images that are mapped via the rbd kernel module
257 (default) or other supported device.
258
259 device map [-t | --device-type device-type] [--cookie device-cookie]
260 [--show-cookie] [--snap-id snap-id] [--read-only] [--exclusive] [-o |
261 --options device-options] image-spec | snap-spec
262 Map the specified image to a block device via the rbd kernel
263 module (default) or other supported device (nbd on Linux or
264 ggate on FreeBSD).
265
266 The --options argument is a comma separated list of device type
267 specific options (opt1,opt2=val,...).
268
269 device unmap [-t | --device-type device-type] [-o | --options de‐
270 vice-options] [--snap-id snap-id] image-spec | snap-spec | device-path
271 Unmap the block device that was mapped via the rbd kernel module
272 (default) or other supported device.
273
274 The --options argument is a comma separated list of device type
275 specific options (opt1,opt2=val,...).
276
277 device attach [-t | --device-type device-type] --device device-path
278 [--cookie device-cookie] [--show-cookie] [--snap-id snap-id]
279 [--read-only] [--exclusive] [--force] [-o | --options device-options]
280 image-spec | snap-spec
281 Attach the specified image to the specified block device (cur‐
282 rently only nbd on Linux). This operation is unsafe and should
283 not be normally used. In particular, specifying the wrong image
284 or the wrong block device may lead to data corruption as no val‐
285 idation is performed by nbd kernel driver.
286
287 The --options argument is a comma separated list of device type
288 specific options (opt1,opt2=val,...).
289
290 device detach [-t | --device-type device-type] [-o | --options de‐
291 vice-options] [--snap-id snap-id] image-spec | snap-spec | device-path
292 Detach the block device that was mapped or attached (currently
293 only nbd on Linux). This operation is unsafe and should not be
294 normally used.
295
296 The --options argument is a comma separated list of device type
297 specific options (opt1,opt2=val,...).
298
299 diff [--from-snap snap-name] [--whole-object] image-spec | snap-spec
300 Dump a list of byte extents in the image that have changed since
301 the specified start snapshot, or since the image was created.
302 Each output line includes the starting offset (in bytes), the
303 length of the region (in bytes), and either 'zero' or 'data' to
304 indicate whether the region is known to be zeros or may contain
305 other data.
306
307 du [-p | --pool pool-name] [image-spec | snap-spec] [--merge-snapshots]
308 Will calculate the provisioned and actual disk usage of all im‐
309 ages and associated snapshots within the specified pool. It can
310 also be used against individual images and snapshots.
311
312 If the RBD fast-diff feature is not enabled on images, this op‐
313 eration will require querying the OSDs for every potential ob‐
314 ject within the image.
315
316 The --merge-snapshots will merge snapshots used space into their
317 parent images.
318
319 encryption format image-spec format passphrase-file [--cipher-alg alg]
320 Formats image to an encrypted format. All data previously writ‐
321 ten to the image will become unreadable. Supported formats:
322 luks1, luks2. Supported cipher algorithms: aes-128, aes-256
323 (default).
324
325 export [--export-format format (1 or 2)] (image-spec | snap-spec)
326 [dest-path]
327 Export image to dest path (use - for stdout). The --export-for‐
328 mat accepts '1' or '2' currently. Format 2 allow us to export
329 not only the content of image, but also the snapshots and other
330 properties, such as image_order, features.
331
332 export-diff [--from-snap snap-name] [--whole-object] (image-spec |
333 snap-spec) dest-path
334 Export an incremental diff for an image to dest path (use - for
335 stdout). If an initial snapshot is specified, only changes
336 since that snapshot are included; otherwise, any regions of the
337 image that contain data are included. The end snapshot is spec‐
338 ified using the standard --snap option or @snap syntax (see be‐
339 low). The image diff format includes metadata about image size
340 changes, and the start and end snapshots. It efficiently repre‐
341 sents discarded or 'zero' regions of the image.
342
343 feature disable image-spec feature-name...
344 Disable the specified feature on the specified image. Multiple
345 features can be specified.
346
347 feature enable image-spec feature-name...
348 Enable the specified feature on the specified image. Multiple
349 features can be specified.
350
351 flatten [--encryption-format encryption-format --encryp‐
352 tion-passphrase-file passphrase-file]... image-spec
353 If image is a clone, copy all shared blocks from the parent
354 snapshot and make the child independent of the parent, severing
355 the link between parent snap and child. The parent snapshot can
356 be unprotected and deleted if it has no further dependent
357 clones.
358
359 This requires image format 2.
360
361 group create group-spec
362 Create a group.
363
364 group image add group-spec image-spec
365 Add an image to a group.
366
367 group image list group-spec
368 List images in a group.
369
370 group image remove group-spec image-spec
371 Remove an image from a group.
372
373 group ls [-p | --pool pool-name]
374 List rbd groups.
375
376 group rename src-group-spec dest-group-spec
377 Rename a group. Note: rename across pools is not supported.
378
379 group rm group-spec
380 Delete a group.
381
382 group snap create group-snap-spec
383 Make a snapshot of a group.
384
385 group snap list group-spec
386 List snapshots of a group.
387
388 group snap rm group-snap-spec
389 Remove a snapshot from a group.
390
391 group snap rename group-snap-spec snap-name
392 Rename group's snapshot.
393
394 group snap rollback group-snap-spec
395 Rollback group to snapshot.
396
397 image-meta get image-spec key
398 Get metadata value with the key.
399
400 image-meta list image-spec
401 Show metadata held on the image. The first column is the key and
402 the second column is the value.
403
404 image-meta remove image-spec key
405 Remove metadata key with the value.
406
407 image-meta set image-spec key value
408 Set metadata key with the value. They will displayed in im‐
409 age-meta list.
410
411 import [--export-format format (1 or 2)] [--image-format format-id]
412 [--object-size size-in-B/K/M] [--stripe-unit size-in-B/K/M
413 --stripe-count num] [--image-feature feature-name]... [--image-shared]
414 src-path [image-spec]
415 Create a new image and imports its data from path (use - for
416 stdin). The import operation will try to create sparse rbd im‐
417 ages if possible. For import from stdin, the sparsification
418 unit is the data block size of the destination image (object
419 size).
420
421 The --stripe-unit and --stripe-count arguments are optional, but
422 must be used together.
423
424 The --export-format accepts '1' or '2' currently. Format 2 allow
425 us to import not only the content of image, but also the snap‐
426 shots and other properties, such as image_order, features.
427
428 import-diff src-path image-spec
429 Import an incremental diff of an image and applies it to the
430 current image. If the diff was generated relative to a start
431 snapshot, we verify that snapshot already exists before continu‐
432 ing. If there was an end snapshot we verify it does not already
433 exist before applying the changes, and create the snapshot when
434 we are done.
435
436 info image-spec | snap-spec
437 Will dump information (such as size and object size) about a
438 specific rbd image. If image is a clone, information about its
439 parent is also displayed. If a snapshot is specified, whether
440 it is protected is shown as well.
441
442 journal client disconnect journal-spec
443 Flag image journal client as disconnected.
444
445 journal export [--verbose] [--no-error] src-journal-spec path-name
446 Export image journal to path (use - for stdout). It can be make
447 a backup of the image journal especially before attempting dan‐
448 gerous operations.
449
450 Note that this command may not always work if the journal is
451 badly corrupted.
452
453 journal import [--verbose] [--no-error] path-name dest-journal-spec
454 Import image journal from path (use - for stdin).
455
456 journal info journal-spec
457 Show information about image journal.
458
459 journal inspect [--verbose] journal-spec
460 Inspect and report image journal for structural errors.
461
462 journal reset journal-spec
463 Reset image journal.
464
465 journal status journal-spec
466 Show status of image journal.
467
468 lock add [--shared lock-tag] image-spec lock-id
469 Lock an image. The lock-id is an arbitrary name for the user's
470 convenience. By default, this is an exclusive lock, meaning it
471 will fail if the image is already locked. The --shared option
472 changes this behavior. Note that locking does not affect any op‐
473 eration other than adding a lock. It does not protect an image
474 from being deleted.
475
476 lock ls image-spec
477 Show locks held on the image. The first column is the locker to
478 use with the lock remove command.
479
480 lock rm image-spec lock-id locker
481 Release a lock on an image. The lock id and locker are as output
482 by lock ls.
483
484 ls [-l | --long] [pool-name]
485 Will list all rbd images listed in the rbd_directory object.
486 With -l, also show snapshots, and use longer-format output in‐
487 cluding size, parent (if clone), format, etc.
488
489 merge-diff first-diff-path second-diff-path merged-diff-path
490 Merge two continuous incremental diffs of an image into one sin‐
491 gle diff. The first diff's end snapshot must be equal with the
492 second diff's start snapshot. The first diff could be - for
493 stdin, and merged diff could be - for stdout, which enables mul‐
494 tiple diff files to be merged using something like 'rbd
495 merge-diff first second - | rbd merge-diff - third result'. Note
496 this command currently only support the source incremental diff
497 with stripe_count == 1
498
499 migration abort image-spec
500 Cancel image migration. This step may be run after successful or
501 failed migration prepare or migration execute steps and returns
502 the image to its initial (before migration) state. All modifica‐
503 tions to the destination image are lost.
504
505 migration commit image-spec
506 Commit image migration. This step is run after a successful mi‐
507 gration prepare and migration execute steps and removes the
508 source image data.
509
510 migration execute image-spec
511 Execute image migration. This step is run after a successful mi‐
512 gration prepare step and copies image data to the destination.
513
514 migration prepare [--order order] [--object-size object-size] [--im‐
515 age-feature image-feature] [--image-shared] [--stripe-unit stripe-unit]
516 [--stripe-count stripe-count] [--data-pool data-pool] [--import-only]
517 [--source-spec json] [--source-spec-path path] src-image-spec [dest-im‐
518 age-spec]
519 Prepare image migration. This is the first step when migrating
520 an image, i.e. changing the image location, format or other pa‐
521 rameters that can't be changed dynamically. The destination can
522 match the source, and in this case dest-image-spec can be omit‐
523 ted. After this step the source image is set as a parent of the
524 destination image, and the image is accessible in copy-on-write
525 mode by its destination spec.
526
527 An image can also be migrated from a read-only import source by
528 adding the --import-only optional and providing a JSON-encoded
529 --source-spec or a path to a JSON-encoded source-spec file using
530 the --source-spec-path optionals.
531
532 mirror image demote image-spec
533 Demote a primary image to non-primary for RBD mirroring.
534
535 mirror image disable [--force] image-spec
536 Disable RBD mirroring for an image. If the mirroring is config‐
537 ured in image mode for the image's pool, then it can be explic‐
538 itly disabled mirroring for each image within the pool.
539
540 mirror image enable image-spec mode
541 Enable RBD mirroring for an image. If the mirroring is config‐
542 ured in image mode for the image's pool, then it can be explic‐
543 itly enabled mirroring for each image within the pool.
544
545 The mirror image mode can either be journal (default) or snap‐
546 shot. The journal mode requires the RBD journaling feature.
547
548 mirror image promote [--force] image-spec
549 Promote a non-primary image to primary for RBD mirroring.
550
551 mirror image resync image-spec
552 Force resync to primary image for RBD mirroring.
553
554 mirror image status image-spec
555 Show RBD mirroring status for an image.
556
557 mirror pool demote [pool-name]
558 Demote all primary images within a pool to non-primary. Every
559 mirroring enabled image will demoted in the pool.
560
561 mirror pool disable [pool-name]
562 Disable RBD mirroring by default within a pool. When mirroring
563 is disabled on a pool in this way, mirroring will also be dis‐
564 abled on any images (within the pool) for which mirroring was
565 enabled explicitly.
566
567 mirror pool enable [pool-name] mode
568 Enable RBD mirroring by default within a pool. The mirroring
569 mode can either be pool or image. If configured in pool mode,
570 all images in the pool with the journaling feature enabled are
571 mirrored. If configured in image mode, mirroring needs to be
572 explicitly enabled (by mirror image enable command) on each im‐
573 age.
574
575 mirror pool info [pool-name]
576 Show information about the pool mirroring configuration. It in‐
577 cludes mirroring mode, peer UUID, remote cluster name, and re‐
578 mote client name.
579
580 mirror pool peer add [pool-name] remote-cluster-spec
581 Add a mirroring peer to a pool. remote-cluster-spec is [remote
582 client name@]remote cluster name.
583
584 The default for remote client name is "client.admin".
585
586 This requires mirroring mode is enabled.
587
588 mirror pool peer remove [pool-name] uuid
589 Remove a mirroring peer from a pool. The peer uuid is available
590 from mirror pool info command.
591
592 mirror pool peer set [pool-name] uuid key value
593 Update mirroring peer settings. The key can be either client or
594 cluster, and the value is corresponding to remote client name or
595 remote cluster name.
596
597 mirror pool promote [--force] [pool-name]
598 Promote all non-primary images within a pool to primary. Every
599 mirroring enabled image will promoted in the pool.
600
601 mirror pool status [--verbose] [pool-name]
602 Show status for all mirrored images in the pool. With --ver‐
603 bose, also show additionally output status details for every
604 mirroring image in the pool.
605
606 mirror snapshot schedule add [-p | --pool pool] [--namespace namespace]
607 [--image image] interval [start-time]
608 Add mirror snapshot schedule.
609
610 mirror snapshot schedule list [-R | --recursive] [--format format]
611 [--pretty-format] [-p | --pool pool] [--namespace namespace] [--image
612 image]
613 List mirror snapshot schedule.
614
615 mirror snapshot schedule remove [-p | --pool pool] [--namespace name‐
616 space] [--image image] interval [start-time]
617 Remove mirror snapshot schedule.
618
619 mirror snapshot schedule status [-p | --pool pool] [--format format]
620 [--pretty-format] [--namespace namespace] [--image image]
621 Show mirror snapshot schedule status.
622
623 mv src-image-spec dest-image-spec
624 Rename an image. Note: rename across pools is not supported.
625
626 namespace create pool-name/namespace-name
627 Create a new image namespace within the pool.
628
629 namespace list pool-name
630 List image namespaces defined within the pool.
631
632 namespace remove pool-name/namespace-name
633 Remove an empty image namespace from the pool.
634
635 object-map check image-spec | snap-spec
636 Verify the object map is correct.
637
638 object-map rebuild image-spec | snap-spec
639 Rebuild an invalid object map for the specified image. An image
640 snapshot can be specified to rebuild an invalid object map for a
641 snapshot.
642
643 pool init [pool-name] [--force]
644 Initialize pool for use by RBD. Newly created pools must ini‐
645 tialized prior to use.
646
647 resize (-s | --size size-in-M/G/T) [--allow-shrink] [--encryption-for‐
648 mat encryption-format --encryption-passphrase-file passphrase-file]...
649 image-spec
650 Resize rbd image. The size parameter also needs to be specified.
651 The --allow-shrink option lets the size be reduced.
652
653 rm image-spec
654 Delete an rbd image (including all data blocks). If the image
655 has snapshots, this fails and nothing is deleted.
656
657 snap create snap-spec
658 Create a new snapshot. Requires the snapshot name parameter
659 specified.
660
661 snap limit clear image-spec
662 Remove any previously set limit on the number of snapshots al‐
663 lowed on an image.
664
665 snap limit set [--limit] limit image-spec
666 Set a limit for the number of snapshots allowed on an image.
667
668 snap ls image-spec
669 Dump the list of snapshots inside a specific image.
670
671 snap protect snap-spec
672 Protect a snapshot from deletion, so that clones can be made of
673 it (see rbd clone). Snapshots must be protected before clones
674 are made; protection implies that there exist dependent cloned
675 children that refer to this snapshot. rbd clone will fail on a
676 nonprotected snapshot.
677
678 This requires image format 2.
679
680 snap purge image-spec
681 Remove all unprotected snapshots from an image.
682
683 snap rename src-snap-spec dest-snap-spec
684 Rename a snapshot. Note: rename across pools and images is not
685 supported.
686
687 snap rm [--force] snap-spec
688 Remove the specified snapshot.
689
690 snap rollback snap-spec
691 Rollback image content to snapshot. This will iterate through
692 the entire blocks array and update the data head content to the
693 snapshotted version.
694
695 snap unprotect snap-spec
696 Unprotect a snapshot from deletion (undo snap protect). If
697 cloned children remain, snap unprotect fails. (Note that clones
698 may exist in different pools than the parent snapshot.)
699
700 This requires image format 2.
701
702 sparsify [--sparse-size sparse-size] image-spec
703 Reclaim space for zeroed image extents. The default sparse size
704 is 4096 bytes and can be changed via --sparse-size option with
705 the following restrictions: it should be power of two, not less
706 than 4096, and not larger than image object size.
707
708 status image-spec
709 Show the status of the image, including which clients have it
710 open.
711
712 trash ls [pool-name]
713 List all entries from trash.
714
715 trash mv image-spec
716 Move an image to the trash. Images, even ones actively in-use by
717 clones, can be moved to the trash and deleted at a later time.
718
719 trash purge [pool-name]
720 Remove all expired images from trash.
721
722 trash restore image-id
723 Restore an image from trash.
724
725 trash rm image-id
726 Delete an image from trash. If image deferment time has not ex‐
727 pired you can not removed it unless use force. But an actively
728 in-use by clones or has snapshots can not be removed.
729
730 trash purge schedule add [-p | --pool pool] [--namespace namespace] in‐
731 terval [start-time]
732 Add trash purge schedule.
733
734 trash purge schedule list [-R | --recursive] [--format format]
735 [--pretty-format] [-p | --pool pool] [--namespace namespace]
736 List trash purge schedule.
737
738 trash purge schedule remove [-p | --pool pool] [--namespace namespace]
739 interval [start-time]
740 Remove trash purge schedule.
741
742 trash purge schedule status [-p | --pool pool] [--format format]
743 [--pretty-format] [--namespace namespace]
744 Show trash purge schedule status.
745
746 watch image-spec
747 Watch events on image.
748
750 image-spec is [pool-name/[namespace-name/]]image-name
751 snap-spec is [pool-name/[namespace-name/]]image-name@snap-name
752 group-spec is [pool-name/[namespace-name/]]group-name
753 group-snap-spec is [pool-name/[namespace-name/]]group-name@snap-name
754 journal-spec is [pool-name/[namespace-name/]]journal-name
755
756
757 The default for pool-name is "rbd" and namespace-name is "". If an im‐
758 age name contains a slash character ('/'), pool-name is required.
759
760 The journal-name is image-id.
761
762 You may specify each name individually, using --pool, --namespace,
763 --image, and --snap options, but this is discouraged in favor of the
764 above spec syntax.
765
767 RBD images are striped over many objects, which are then stored by the
768 Ceph distributed object store (RADOS). As a result, read and write re‐
769 quests for the image are distributed across many nodes in the cluster,
770 generally preventing any single node from becoming a bottleneck when
771 individual images get large or busy.
772
773 The striping is controlled by three parameters:
774
775 object-size
776 The size of objects we stripe over is a power of two. It will be
777 rounded up the nearest power of two. The default object size is
778 4 MB, smallest is 4K and maximum is 32M.
779
780 stripe_unit
781 Each [stripe_unit] contiguous bytes are stored adjacently in the
782 same object, before we move on to the next object.
783
784 stripe_count
785 After we write [stripe_unit] bytes to [stripe_count] objects, we
786 loop back to the initial object and write another stripe, until
787 the object reaches its maximum size. At that point, we move on
788 to the next [stripe_count] objects.
789
790 By default, [stripe_unit] is the same as the object size and
791 [stripe_count] is 1. Specifying a different [stripe_unit] and/or
792 [stripe_count] is often referred to as using "fancy" striping and re‐
793 quires format 2.
794
796 Most of these options are useful mainly for debugging and benchmarking.
797 The default values are set in the kernel and may therefore depend on
798 the version of the running kernel.
799
800 Per client instance rbd device map options:
801
802 • fsid=aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee - FSID that should be as‐
803 sumed by the client.
804
805 • ip=a.b.c.d[:p] - IP and, optionally, port the client should use.
806
807 • share - Enable sharing of client instances with other mappings (de‐
808 fault).
809
810 • noshare - Disable sharing of client instances with other mappings.
811
812 • crc - Enable CRC32C checksumming for msgr1 on-the-wire protocol (de‐
813 fault). For msgr2.1 protocol this option is ignored: full checksum‐
814 ming is always on in 'crc' mode and always off in 'secure' mode.
815
816 • nocrc - Disable CRC32C checksumming for msgr1 on-the-wire protocol.
817 Note that only payload checksumming is disabled, header checksumming
818 is always on. For msgr2.1 protocol this option is ignored.
819
820 • cephx_require_signatures - Require msgr1 message signing feature
821 (since 3.19, default). This option is deprecated and will be removed
822 in the future as the feature has been supported since the Bobtail re‐
823 lease.
824
825 • nocephx_require_signatures - Don't require msgr1 message signing fea‐
826 ture (since 3.19). This option is deprecated and will be removed in
827 the future.
828
829 • tcp_nodelay - Disable Nagle's algorithm on client sockets (since 4.0,
830 default).
831
832 • notcp_nodelay - Enable Nagle's algorithm on client sockets (since
833 4.0).
834
835 • cephx_sign_messages - Enable message signing for msgr1 on-the-wire
836 protocol (since 4.4, default). For msgr2.1 protocol this option is
837 ignored: message signing is built into 'secure' mode and not offered
838 in 'crc' mode.
839
840 • nocephx_sign_messages - Disable message signing for msgr1 on-the-wire
841 protocol (since 4.4). For msgr2.1 protocol this option is ignored.
842
843 • mount_timeout=x - A timeout on various steps in rbd device map and
844 rbd device unmap sequences (default is 60 seconds). In particular,
845 since 4.2 this can be used to ensure that rbd device unmap eventually
846 times out when there is no network connection to a cluster.
847
848 • osdkeepalive=x - OSD keepalive timeout (default is 5 seconds).
849
850 • osd_idle_ttl=x - OSD idle TTL (default is 60 seconds).
851
852 Per mapping (block device) rbd device map options:
853
854 • rw - Map the image read-write (default). Overridden by --read-only.
855
856 • ro - Map the image read-only. Equivalent to --read-only.
857
858 • queue_depth=x - queue depth (since 4.2, default is 128 requests).
859
860 • lock_on_read - Acquire exclusive lock on reads, in addition to writes
861 and discards (since 4.9).
862
863 • exclusive - Disable automatic exclusive lock transitions (since
864 4.12). Equivalent to --exclusive.
865
866 • lock_timeout=x - A timeout on waiting for the acquisition of exclu‐
867 sive lock (since 4.17, default is 0 seconds, meaning no timeout).
868
869 • notrim - Turn off discard and write zeroes offload support to avoid
870 deprovisioning a fully provisioned image (since 4.17). When enabled,
871 discard requests will fail with -EOPNOTSUPP, write zeroes requests
872 will fall back to manually zeroing.
873
874 • abort_on_full - Fail write requests with -ENOSPC when the cluster is
875 full or the data pool reaches its quota (since 5.0). The default be‐
876 haviour is to block until the full condition is cleared.
877
878 • alloc_size - Minimum allocation unit of the underlying OSD object
879 store backend (since 5.1, default is 64K bytes). This is used to
880 round off and drop discards that are too small. For bluestore, the
881 recommended setting is bluestore_min_alloc_size (currently set to 4K
882 for all types of drives, previously used to be set to 64K for hard
883 disk drives and 16K for solid-state drives). For filestore with
884 filestore_punch_hole = false, the recommended setting is image object
885 size (typically 4M).
886
887 • crush_location=x - Specify the location of the client in terms of
888 CRUSH hierarchy (since 5.8). This is a set of key-value pairs sepa‐
889 rated from each other by '|', with keys separated from values by ':'.
890 Note that '|' may need to be quoted or escaped to avoid it being in‐
891 terpreted as a pipe by the shell. The key is the bucket type name
892 (e.g. rack, datacenter or region with default bucket types) and the
893 value is the bucket name. For example, to indicate that the client
894 is local to rack "myrack", data center "mydc" and region "myregion":
895
896 crush_location=rack:myrack|datacenter:mydc|region:myregion
897
898 Each key-value pair stands on its own: "myrack" doesn't need to re‐
899 side in "mydc", which in turn doesn't need to reside in "myregion".
900 The location is not a path to the root of the hierarchy but rather a
901 set of nodes that are matched independently, owning to the fact that
902 bucket names are unique within a CRUSH map. "Multipath" locations
903 are supported, so it is possible to indicate locality for multiple
904 parallel hierarchies:
905
906 crush_location=rack:myrack1|rack:myrack2|datacenter:mydc
907
908 • read_from_replica=no - Disable replica reads, always pick the primary
909 OSD (since 5.8, default).
910
911 • read_from_replica=balance - When issued a read on a replicated pool,
912 pick a random OSD for serving it (since 5.8).
913
914 This mode is safe for general use only since Octopus (i.e. after
915 "ceph osd require-osd-release octopus"). Otherwise it should be lim‐
916 ited to read-only workloads such as images mapped read-only every‐
917 where or snapshots.
918
919 • read_from_replica=localize - When issued a read on a replicated pool,
920 pick the most local OSD for serving it (since 5.8). The locality
921 metric is calculated against the location of the client given with
922 crush_location; a match with the lowest-valued bucket type wins. For
923 example, with default bucket types, an OSD in a matching rack is
924 closer than an OSD in a matching data center, which in turn is closer
925 than an OSD in a matching region.
926
927 This mode is safe for general use only since Octopus (i.e. after
928 "ceph osd require-osd-release octopus"). Otherwise it should be lim‐
929 ited to read-only workloads such as images mapped read-only every‐
930 where or snapshots.
931
932 • compression_hint=none - Don't set compression hints (since 5.8, de‐
933 fault).
934
935 • compression_hint=compressible - Hint to the underlying OSD object
936 store backend that the data is compressible, enabling compression in
937 passive mode (since 5.8).
938
939 • compression_hint=incompressible - Hint to the underlying OSD object
940 store backend that the data is incompressible, disabling compression
941 in aggressive mode (since 5.8).
942
943 • ms_mode=legacy - Use msgr1 on-the-wire protocol (since 5.11, de‐
944 fault).
945
946 • ms_mode=crc - Use msgr2.1 on-the-wire protocol, select 'crc' mode,
947 also referred to as plain mode (since 5.11). If the daemon denies
948 'crc' mode, fail the connection.
949
950 • ms_mode=secure - Use msgr2.1 on-the-wire protocol, select 'secure'
951 mode (since 5.11). 'secure' mode provides full in-transit encryption
952 ensuring both confidentiality and authenticity. If the daemon denies
953 'secure' mode, fail the connection.
954
955 • ms_mode=prefer-crc - Use msgr2.1 on-the-wire protocol, select 'crc'
956 mode (since 5.11). If the daemon denies 'crc' mode in favor of 'se‐
957 cure' mode, agree to 'secure' mode.
958
959 • ms_mode=prefer-secure - Use msgr2.1 on-the-wire protocol, select 'se‐
960 cure' mode (since 5.11). If the daemon denies 'secure' mode in favor
961 of 'crc' mode, agree to 'crc' mode.
962
963 • rxbounce - Use a bounce buffer when receiving data (since 5.17). The
964 default behaviour is to read directly into the destination buffer. A
965 bounce buffer is needed if the destination buffer isn't guaranteed to
966 be stable (i.e. remain unchanged while it is being read to). In par‐
967 ticular this is the case for Windows where a system-wide "dummy"
968 (throwaway) page may be mapped into the destination buffer in order
969 to generate a single large I/O. Otherwise, "libceph: ... bad
970 crc/signature" or "libceph: ... integrity error, bad crc" errors and
971 associated performance degradation are expected.
972
973 • udev - Wait for udev device manager to finish executing all matching
974 "add" rules and release the device before exiting (default). This
975 option is not passed to the kernel.
976
977 • noudev - Don't wait for udev device manager. When enabled, the de‐
978 vice may not be fully usable immediately on exit.
979
980 rbd device unmap options:
981
982 • force - Force the unmapping of a block device that is open (since
983 4.9). The driver will wait for running requests to complete and then
984 unmap; requests sent to the driver after initiating the unmap will be
985 failed.
986
987 • udev - Wait for udev device manager to finish executing all matching
988 "remove" rules and clean up after the device before exiting (de‐
989 fault). This option is not passed to the kernel.
990
991 • noudev - Don't wait for udev device manager.
992
994 To create a new rbd image that is 100 GB:
995
996 rbd create mypool/myimage --size 102400
997
998 To use a non-default object size (8 MB):
999
1000 rbd create mypool/myimage --size 102400 --object-size 8M
1001
1002 To delete an rbd image (be careful!):
1003
1004 rbd rm mypool/myimage
1005
1006 To create a new snapshot:
1007
1008 rbd snap create mypool/myimage@mysnap
1009
1010 To create a copy-on-write clone of a protected snapshot:
1011
1012 rbd clone mypool/myimage@mysnap otherpool/cloneimage
1013
1014 To see which clones of a snapshot exist:
1015
1016 rbd children mypool/myimage@mysnap
1017
1018 To delete a snapshot:
1019
1020 rbd snap rm mypool/myimage@mysnap
1021
1022 To map an image via the kernel with cephx enabled:
1023
1024 rbd device map mypool/myimage --id admin --keyfile secretfile
1025
1026 To map an image via the kernel with different cluster name other than
1027 default ceph:
1028
1029 rbd device map mypool/myimage --cluster cluster-name
1030
1031 To unmap an image:
1032
1033 rbd device unmap /dev/rbd0
1034
1035 To create an image and a clone from it:
1036
1037 rbd import --image-format 2 image mypool/parent
1038 rbd snap create mypool/parent@snap
1039 rbd snap protect mypool/parent@snap
1040 rbd clone mypool/parent@snap otherpool/child
1041
1042 To create an image with a smaller stripe_unit (to better distribute
1043 small writes in some workloads):
1044
1045 rbd create mypool/myimage --size 102400 --stripe-unit 65536B --stripe-count 16
1046
1047 To change an image from one image format to another, export it and then
1048 import it as the desired image format:
1049
1050 rbd export mypool/myimage@snap /tmp/img
1051 rbd import --image-format 2 /tmp/img mypool/myimage2
1052
1053 To lock an image for exclusive use:
1054
1055 rbd lock add mypool/myimage mylockid
1056
1057 To release a lock:
1058
1059 rbd lock remove mypool/myimage mylockid client.2485
1060
1061 To list images from trash:
1062
1063 rbd trash ls mypool
1064
1065 To defer delete an image (use --expires-at to set expiration time, de‐
1066 fault is now):
1067
1068 rbd trash mv mypool/myimage --expires-at "tomorrow"
1069
1070 To delete an image from trash (be careful!):
1071
1072 rbd trash rm mypool/myimage-id
1073
1074 To force delete an image from trash (be careful!):
1075
1076 rbd trash rm mypool/myimage-id --force
1077
1078 To restore an image from trash:
1079
1080 rbd trash restore mypool/myimage-id
1081
1082 To restore an image from trash and rename it:
1083
1084 rbd trash restore mypool/myimage-id --image mynewimage
1085
1087 rbd is part of Ceph, a massively scalable, open-source, distributed
1088 storage system. Please refer to the Ceph documentation at
1089 https://docs.ceph.com for more information.
1090
1092 ceph(8), rados(8)
1093
1095 2010-2023, Inktank Storage, Inc. and contributors. Licensed under Cre‐
1096 ative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0)
1097
1098
1099
1100
1101dev Nov 15, 2023 RBD(8)