1RBD(8) Ceph RBD(8)
2
6 rbd - manage rados block device (RBD) images
7
9 rbd [ -c ceph.conf ] [ -m monaddr ] [--cluster cluster-name]
10 [ -p | --pool pool ] [ command ... ]
11
14 rbd is a utility for manipulating rados block device (RBD) images, used
15 by the Linux rbd driver and the rbd storage driver for QEMU/KVM. RBD
16 images are simple block devices that are striped over objects and
17 stored in a RADOS object store. The size of the objects the image is
18 striped over must be a power of two.
19
21 -c ceph.conf, --conf ceph.conf
22 Use ceph.conf configuration file instead of the default
23 /etc/ceph/ceph.conf to determine monitor addresses during
24 startup.
25 -m monaddress[:port]
27 Connect to specified monitor (instead of looking through
28 ceph.conf).
29 --cluster cluster-name
31 Use different cluster name as compared to default cluster name
32 ceph.
33 -p pool-name, --pool pool-name
35 Interact with the given pool. Required by most commands.
36 --namespace namespace-name
38 Use a pre-defined image namespace within a pool
39 --no-progress
41 Do not output progress information (goes to standard error by
42 default for some commands).
43
45 --image-format format-id
46 Specifies which object layout to use. The default is 2.
47 • format 1 - (deprecated) Use the original format for a new rbd
49 image. This format is understood by all versions of librbd and
50 the kernel rbd module, but does not support newer features
51 like cloning.
52 • format 2 - Use the second rbd format, which is supported by
54 librbd since the Bobtail release and the kernel rbd module
55 since kernel 3.10 (except for "fancy" striping, which is sup‐
56 ported since kernel 4.17). This adds support for cloning and
57 is more easily extensible to allow more features in the fu‐
58 ture.
59 -s size-in-M/G/T, --size size-in-M/G/T
61 Specifies the size of the new rbd image or the new size of the
62 existing rbd image in M/G/T. If no suffix is given, unit M is
63 assumed.
64 --object-size size-in-B/K/M
66 Specifies the object size in B/K/M. Object size will be rounded
67 up the nearest power of two; if no suffix is given, unit B is
68 assumed. The default object size is 4M, smallest is 4K and max‐
69 imum is 32M.
70 --stripe-unit size-in-B/K/M
72 Specifies the stripe unit size in B/K/M. If no suffix is given,
73 unit B is assumed. See striping section (below) for more de‐
74 tails.
75 --stripe-count num
77 Specifies the number of objects to stripe over before looping
78 back to the first object. See striping section (below) for more
79 details.
80 --snap snap
82 Specifies the snapshot name for the specific operation.
83 --id username
85 Specifies the username (without the client. prefix) to use with
86 the map command.
87 --keyring filename
89 Specifies a keyring file containing a secret for the specified
90 user to use with the map command. If not specified, the default
91 keyring locations will be searched.
92 --keyfile filename
94 Specifies a file containing the secret key of --id user to use
95 with the map command. This option is overridden by --keyring if
96 the latter is also specified.
97 --shared lock-tag
99 Option for lock add that allows multiple clients to lock the
100 same image if they use the same tag. The tag is an arbitrary
101 string. This is useful for situations where an image must be
102 open from more than one client at once, like during live migra‐
103 tion of a virtual machine, or for use underneath a clustered
104 file system.
105 --format format
107 Specifies output formatting (default: plain, json, xml)
108 --pretty-format
110 Make json or xml formatted output more human-readable.
111 -o krbd-options, --options krbd-options
113 Specifies which options to use when mapping or unmapping an im‐
114 age via the rbd kernel driver. krbd-options is a comma-sepa‐
115 rated list of options (similar to mount(8) mount options). See
116 kernel rbd (krbd) options section below for more details.
117 --read-only
119 Map the image read-only. Equivalent to -o ro.
120 --image-feature feature-name
122 Specifies which RBD format 2 feature should be enabled when cre‐
123 ating an image. Multiple features can be enabled by repeating
124 this option multiple times. The following features are sup‐
125 ported:
126 • layering: layering support
128 • striping: striping v2 support
130 • exclusive-lock: exclusive locking support
132 • object-map: object map support (requires exclusive-lock)
134 • fast-diff: fast diff calculations (requires object-map)
136 • deep-flatten: snapshot flatten support
138 • journaling: journaled IO support (requires exclusive-lock)
140 • data-pool: erasure coded pool support
142 --image-shared
144 Specifies that the image will be used concurrently by multiple
145 clients. This will disable features that are dependent upon ex‐
146 clusive ownership of the image.
147 --whole-object
149 Specifies that the diff should be limited to the extents of a
150 full object instead of showing intra-object deltas. When the ob‐
151 ject map feature is enabled on an image, limiting the diff to
152 the object extents will dramatically improve performance since
153 the differences can be computed by examining the in-memory ob‐
154 ject map instead of querying RADOS for each object within the
155 image.
156 --limit
158 Specifies the limit for the number of snapshots permitted.
159
161 bench --io-type <read | write | readwrite | rw> [--io-size
162 size-in-B/K/M/G/T] [--io-threads num-ios-in-flight] [--io-total
163 size-in-B/K/M/G/T] [--io-pattern seq | rand] [--rw-mix-read read pro‐
164 portion in readwrite] image-spec
165 Generate a series of IOs to the image and measure the IO
166 throughput and latency. If no suffix is given, unit B is as‐
167 sumed for both --io-size and --io-total. Defaults are:
168 --io-size 4096, --io-threads 16, --io-total 1G, --io-pattern
169 seq, --rw-mix-read 50.
170 children snap-spec
172 List the clones of the image at the given snapshot. This checks
173 every pool, and outputs the resulting poolname/imagename.
174 This requires image format 2.
176 clone [--object-size size-in-B/K/M] [--stripe-unit size-in-B/K/M
178 --stripe-count num] [--image-feature feature-name] [--image-shared]
179 parent-snap-spec child-image-spec
180 Will create a clone (copy-on-write child) of the parent snap‐
181 shot. Object size will be identical to that of the parent image
182 unless specified. Size will be the same as the parent snapshot.
183 The --stripe-unit and --stripe-count arguments are optional, but
184 must be used together.
185 The parent snapshot must be protected (see rbd snap protect).
187 This requires image format 2.
188 config global get config-entity key
190 Get a global-level configuration override.
191 config global list [--format plain | json | xml] [--pretty-format] con‐
193 fig-entity
194 List global-level configuration overrides.
195 config global set config-entity key value
197 Set a global-level configuration override.
198 config global remove config-entity key
200 Remove a global-level configuration override.
201 config image get image-spec key
203 Get an image-level configuration override.
204 config image list [--format plain | json | xml] [--pretty-format] im‐
206 age-spec
207 List image-level configuration overrides.
208 config image set image-spec key value
210 Set an image-level configuration override.
211 config image remove image-spec key
213 Remove an image-level configuration override.
214 config pool get pool-name key
216 Get a pool-level configuration override.
217 config pool list [--format plain | json | xml] [--pretty-format]
219 pool-name
220 List pool-level configuration overrides.
221 config pool set pool-name key value
223 Set a pool-level configuration override.
224 config pool remove pool-name key
226 Remove a pool-level configuration override.
227 cp (src-image-spec | src-snap-spec) dest-image-spec
229 Copy the content of a src-image into the newly created dest-im‐
230 age. dest-image will have the same size, object size, and image
231 format as src-image.
232 create (-s | --size size-in-M/G/T) [--image-format format-id] [--ob‐
234 ject-size size-in-B/K/M] [--stripe-unit size-in-B/K/M --stripe-count
235 num] [--thick-provision] [--no-progress] [--image-feature fea‐
236 ture-name]... [--image-shared] image-spec
237 Will create a new rbd image. You must also specify the size via
238 --size. The --stripe-unit and --stripe-count arguments are op‐
239 tional, but must be used together. If the --thick-provision is
240 enabled, it will fully allocate storage for the image at cre‐
241 ation time. It will take a long time to do. Note: thick provi‐
242 sioning requires zeroing the contents of the entire image.
243 deep cp (src-image-spec | src-snap-spec) dest-image-spec
245 Deep copy the content of a src-image into the newly created
246 dest-image. Dest-image will have the same size, object size,
247 image format, and snapshots as src-image.
248 device list [-t | --device-type device-type] [--format plain | json |
250 xml] --pretty-format
251 Show the rbd images that are mapped via the rbd kernel module
252 (default) or other supported device.
253 device map [-t | --device-type device-type] [--read-only] [--exclusive]
255 [-o | --options device-options] image-spec | snap-spec
256 Map the specified image to a block device via the rbd kernel
257 module (default) or other supported device (nbd on Linux or
258 ggate on FreeBSD).
259 The --options argument is a comma separated list of device type
261 specific options (opt1,opt2=val,...).
262 device unmap [-t | --device-type device-type] [-o | --options de‐
264 vice-options] image-spec | snap-spec | device-path
265 Unmap the block device that was mapped via the rbd kernel module
266 (default) or other supported device.
267 The --options argument is a comma separated list of device type
269 specific options (opt1,opt2=val,...).
270 diff [--from-snap snap-name] [--whole-object] image-spec | snap-spec
272 Dump a list of byte extents in the image that have changed since
273 the specified start snapshot, or since the image was created.
274 Each output line includes the starting offset (in bytes), the
275 length of the region (in bytes), and either 'zero' or 'data' to
276 indicate whether the region is known to be zeros or may contain
277 other data.
278 du [-p | --pool pool-name] [image-spec | snap-spec] [--merge-snapshots]
280 Will calculate the provisioned and actual disk usage of all im‐
281 ages and associated snapshots within the specified pool. It can
282 also be used against individual images and snapshots.
283 If the RBD fast-diff feature is not enabled on images, this op‐
285 eration will require querying the OSDs for every potential ob‐
286 ject within the image.
287 The --merge-snapshots will merge snapshots used space into their
289 parent images.
290 encryption format image-spec format passphrase-file [--cipher-alg alg]
292 Formats image to an encrypted format. All data previously writ‐
293 ten to the image will become unreadable. A cloned image cannot
294 be formatted, although encrypted images can be cloned. Sup‐
295 ported formats: luks1, luks2. Supported cipher algorithms:
296 aes-128, aes-256 (default).
297 export [--export-format format (1 or 2)] (image-spec | snap-spec)
299 [dest-path]
300 Export image to dest path (use - for stdout). The --export-for‐
301 mat accepts '1' or '2' currently. Format 2 allow us to export
302 not only the content of image, but also the snapshots and other
303 properties, such as image_order, features.
304 export-diff [--from-snap snap-name] [--whole-object] (image-spec |
306 snap-spec) dest-path
307 Export an incremental diff for an image to dest path (use - for
308 stdout). If an initial snapshot is specified, only changes
309 since that snapshot are included; otherwise, any regions of the
310 image that contain data are included. The end snapshot is spec‐
311 ified using the standard --snap option or @snap syntax (see be‐
312 low). The image diff format includes metadata about image size
313 changes, and the start and end snapshots. It efficiently repre‐
314 sents discarded or 'zero' regions of the image.
315 feature disable image-spec feature-name...
317 Disable the specified feature on the specified image. Multiple
318 features can be specified.
319 feature enable image-spec feature-name...
321 Enable the specified feature on the specified image. Multiple
322 features can be specified.
323 flatten image-spec
325 If image is a clone, copy all shared blocks from the parent
326 snapshot and make the child independent of the parent, severing
327 the link between parent snap and child. The parent snapshot can
328 be unprotected and deleted if it has no further dependent
329 clones.
330 This requires image format 2.
332 group create group-spec
334 Create a group.
335 group image add group-spec image-spec
337 Add an image to a group.
338 group image list group-spec
340 List images in a group.
341 group image remove group-spec image-spec
343 Remove an image from a group.
344 group ls [-p | --pool pool-name]
346 List rbd groups.
347 group rename src-group-spec dest-group-spec
349 Rename a group. Note: rename across pools is not supported.
350 group rm group-spec
352 Delete a group.
353 group snap create group-snap-spec
355 Make a snapshot of a group.
356 group snap list group-spec
358 List snapshots of a group.
359 group snap rm group-snap-spec
361 Remove a snapshot from a group.
362 group snap rename group-snap-spec snap-name
364 Rename group's snapshot.
365 group snap rollback group-snap-spec
367 Rollback group to snapshot.
368 image-meta get image-spec key
370 Get metadata value with the key.
371 image-meta list image-spec
373 Show metadata held on the image. The first column is the key and
374 the second column is the value.
375 image-meta remove image-spec key
377 Remove metadata key with the value.
378 image-meta set image-spec key value
380 Set metadata key with the value. They will displayed in im‐
381 age-meta list.
382 import [--export-format format (1 or 2)] [--image-format format-id]
384 [--object-size size-in-B/K/M] [--stripe-unit size-in-B/K/M
385 --stripe-count num] [--image-feature feature-name]... [--image-shared]
386 src-path [image-spec]
387 Create a new image and imports its data from path (use - for
388 stdin). The import operation will try to create sparse rbd im‐
389 ages if possible. For import from stdin, the sparsification
390 unit is the data block size of the destination image (object
391 size).
392 The --stripe-unit and --stripe-count arguments are optional, but
394 must be used together.
395 The --export-format accepts '1' or '2' currently. Format 2 allow
397 us to import not only the content of image, but also the snap‐
398 shots and other properties, such as image_order, features.
399 import-diff src-path image-spec
401 Import an incremental diff of an image and applies it to the
402 current image. If the diff was generated relative to a start
403 snapshot, we verify that snapshot already exists before continu‐
404 ing. If there was an end snapshot we verify it does not already
405 exist before applying the changes, and create the snapshot when
406 we are done.
407 info image-spec | snap-spec
409 Will dump information (such as size and object size) about a
410 specific rbd image. If image is a clone, information about its
411 parent is also displayed. If a snapshot is specified, whether
412 it is protected is shown as well.
413 journal client disconnect journal-spec
415 Flag image journal client as disconnected.
416 journal export [--verbose] [--no-error] src-journal-spec path-name
418 Export image journal to path (use - for stdout). It can be make
419 a backup of the image journal especially before attempting dan‐
420 gerous operations.
421 Note that this command may not always work if the journal is
423 badly corrupted.
424 journal import [--verbose] [--no-error] path-name dest-journal-spec
426 Import image journal from path (use - for stdin).
427 journal info journal-spec
429 Show information about image journal.
430 journal inspect [--verbose] journal-spec
432 Inspect and report image journal for structural errors.
433 journal reset journal-spec
435 Reset image journal.
436 journal status journal-spec
438 Show status of image journal.
439 lock add [--shared lock-tag] image-spec lock-id
441 Lock an image. The lock-id is an arbitrary name for the user's
442 convenience. By default, this is an exclusive lock, meaning it
443 will fail if the image is already locked. The --shared option
444 changes this behavior. Note that locking does not affect any op‐
445 eration other than adding a lock. It does not protect an image
446 from being deleted.
447 lock ls image-spec
449 Show locks held on the image. The first column is the locker to
450 use with the lock remove command.
451 lock rm image-spec lock-id locker
453 Release a lock on an image. The lock id and locker are as output
454 by lock ls.
455 ls [-l | --long] [pool-name]
457 Will list all rbd images listed in the rbd_directory object.
458 With -l, also show snapshots, and use longer-format output in‐
459 cluding size, parent (if clone), format, etc.
460 merge-diff first-diff-path second-diff-path merged-diff-path
462 Merge two continuous incremental diffs of an image into one sin‐
463 gle diff. The first diff's end snapshot must be equal with the
464 second diff's start snapshot. The first diff could be - for
465 stdin, and merged diff could be - for stdout, which enables mul‐
466 tiple diff files to be merged using something like 'rbd
467 merge-diff first second - | rbd merge-diff - third result'. Note
468 this command currently only support the source incremental diff
469 with stripe_count == 1
470 migration abort image-spec
472 Cancel image migration. This step may be run after successful or
473 failed migration prepare or migration execute steps and returns
474 the image to its initial (before migration) state. All modifica‐
475 tions to the destination image are lost.
476 migration commit image-spec
478 Commit image migration. This step is run after a successful mi‐
479 gration prepare and migration execute steps and removes the
480 source image data.
481 migration execute image-spec
483 Execute image migration. This step is run after a successful mi‐
484 gration prepare step and copies image data to the destination.
485 migration prepare [--order order] [--object-size object-size] [--im‐
487 age-feature image-feature] [--image-shared] [--stripe-unit stripe-unit]
488 [--stripe-count stripe-count] [--data-pool data-pool] [--import-only]
489 [--source-spec json] [--source-spec-path path] src-image-spec [dest-im‐
490 age-spec]
491 Prepare image migration. This is the first step when migrating
492 an image, i.e. changing the image location, format or other pa‐
493 rameters that can't be changed dynamically. The destination can
494 match the source, and in this case dest-image-spec can be omit‐
495 ted. After this step the source image is set as a parent of the
496 destination image, and the image is accessible in copy-on-write
497 mode by its destination spec.
498 An image can also be migrated from a read-only import source by
500 adding the --import-only optional and providing a JSON-encoded
501 --source-spec or a path to a JSON-encoded source-spec file using
502 the --source-spec-path optionals.
503 mirror image demote image-spec
505 Demote a primary image to non-primary for RBD mirroring.
506 mirror image disable [--force] image-spec
508 Disable RBD mirroring for an image. If the mirroring is config‐
509 ured in image mode for the image's pool, then it can be explic‐
510 itly disabled mirroring for each image within the pool.
511 mirror image enable image-spec mode
513 Enable RBD mirroring for an image. If the mirroring is config‐
514 ured in image mode for the image's pool, then it can be explic‐
515 itly enabled mirroring for each image within the pool.
516 The mirror image mode can either be journal (default) or snap‐
518 shot. The journal mode requires the RBD journaling feature.
519 mirror image promote [--force] image-spec
521 Promote a non-primary image to primary for RBD mirroring.
522 mirror image resync image-spec
524 Force resync to primary image for RBD mirroring.
525 mirror image status image-spec
527 Show RBD mirroring status for an image.
528 mirror pool demote [pool-name]
530 Demote all primary images within a pool to non-primary. Every
531 mirroring enabled image will demoted in the pool.
532 mirror pool disable [pool-name]
534 Disable RBD mirroring by default within a pool. When mirroring
535 is disabled on a pool in this way, mirroring will also be dis‐
536 abled on any images (within the pool) for which mirroring was
537 enabled explicitly.
538 mirror pool enable [pool-name] mode
540 Enable RBD mirroring by default within a pool. The mirroring
541 mode can either be pool or image. If configured in pool mode,
542 all images in the pool with the journaling feature enabled are
543 mirrored. If configured in image mode, mirroring needs to be
544 explicitly enabled (by mirror image enable command) on each im‐
545 age.
546 mirror pool info [pool-name]
548 Show information about the pool mirroring configuration. It in‐
549 cludes mirroring mode, peer UUID, remote cluster name, and re‐
550 mote client name.
551 mirror pool peer add [pool-name] remote-cluster-spec
553 Add a mirroring peer to a pool. remote-cluster-spec is [remote
554 client name@]remote cluster name.
555 The default for remote client name is "client.admin".
557 This requires mirroring mode is enabled.
559 mirror pool peer remove [pool-name] uuid
561 Remove a mirroring peer from a pool. The peer uuid is available
562 from mirror pool info command.
563 mirror pool peer set [pool-name] uuid key value
565 Update mirroring peer settings. The key can be either client or
566 cluster, and the value is corresponding to remote client name or
567 remote cluster name.
568 mirror pool promote [--force] [pool-name]
570 Promote all non-primary images within a pool to primary. Every
571 mirroring enabled image will promoted in the pool.
572 mirror pool status [--verbose] [pool-name]
574 Show status for all mirrored images in the pool. With --ver‐
575 bose, also show additionally output status details for every
576 mirroring image in the pool.
577 mirror snapshot schedule add [-p | --pool pool] [--namespace namespace]
579 [--image image] interval [start-time]
580 Add mirror snapshot schedule.
581 mirror snapshot schedule list [-R | --recursive] [--format format]
583 [--pretty-format] [-p | --pool pool] [--namespace namespace] [--image
584 image]
585 List mirror snapshot schedule.
586 mirror snapshot schedule remove [-p | --pool pool] [--namespace name‐
588 space] [--image image] interval [start-time]
589 Remove mirror snapshot schedule.
590 mirror snapshot schedule status [-p | --pool pool] [--format format]
592 [--pretty-format] [--namespace namespace] [--image image]
593 Show mirror snapshot schedule status.
594 mv src-image-spec dest-image-spec
596 Rename an image. Note: rename across pools is not supported.
597 namespace create pool-name/namespace-name
599 Create a new image namespace within the pool.
600 namespace list pool-name
602 List image namespaces defined within the pool.
603 namespace remove pool-name/namespace-name
605 Remove an empty image namespace from the pool.
606 object-map check image-spec | snap-spec
608 Verify the object map is correct.
609 object-map rebuild image-spec | snap-spec
611 Rebuild an invalid object map for the specified image. An image
612 snapshot can be specified to rebuild an invalid object map for a
613 snapshot.
614 pool init [pool-name] [--force]
616 Initialize pool for use by RBD. Newly created pools must ini‐
617 tialized prior to use.
618 resize (-s | --size size-in-M/G/T) [--allow-shrink] image-spec
620 Resize rbd image. The size parameter also needs to be specified.
621 The --allow-shrink option lets the size be reduced.
622 rm image-spec
624 Delete an rbd image (including all data blocks). If the image
625 has snapshots, this fails and nothing is deleted.
626 snap create snap-spec
628 Create a new snapshot. Requires the snapshot name parameter
629 specified.
630 snap limit clear image-spec
632 Remove any previously set limit on the number of snapshots al‐
633 lowed on an image.
634 snap limit set [--limit] limit image-spec
636 Set a limit for the number of snapshots allowed on an image.
637 snap ls image-spec
639 Dump the list of snapshots inside a specific image.
640 snap protect snap-spec
642 Protect a snapshot from deletion, so that clones can be made of
643 it (see rbd clone). Snapshots must be protected before clones
644 are made; protection implies that there exist dependent cloned
645 children that refer to this snapshot. rbd clone will fail on a
646 nonprotected snapshot.
647 This requires image format 2.
649 snap purge image-spec
651 Remove all unprotected snapshots from an image.
652 snap rename src-snap-spec dest-snap-spec
654 Rename a snapshot. Note: rename across pools and images is not
655 supported.
656 snap rm [--force] snap-spec
658 Remove the specified snapshot.
659 snap rollback snap-spec
661 Rollback image content to snapshot. This will iterate through
662 the entire blocks array and update the data head content to the
663 snapshotted version.
664 snap unprotect snap-spec
666 Unprotect a snapshot from deletion (undo snap protect). If
667 cloned children remain, snap unprotect fails. (Note that clones
668 may exist in different pools than the parent snapshot.)
669 This requires image format 2.
671 sparsify [--sparse-size sparse-size] image-spec
673 Reclaim space for zeroed image extents. The default sparse size
674 is 4096 bytes and can be changed via --sparse-size option with
675 the following restrictions: it should be power of two, not less
676 than 4096, and not larger than image object size.
677 status image-spec
679 Show the status of the image, including which clients have it
680 open.
681 trash ls [pool-name]
683 List all entries from trash.
684 trash mv image-spec
686 Move an image to the trash. Images, even ones actively in-use by
687 clones, can be moved to the trash and deleted at a later time.
688 trash purge [pool-name]
690 Remove all expired images from trash.
691 trash restore image-id
693 Restore an image from trash.
694 trash rm image-id
696 Delete an image from trash. If image deferment time has not ex‐
697 pired you can not removed it unless use force. But an actively
698 in-use by clones or has snapshots can not be removed.
699 trash purge schedule add [-p | --pool pool] [--namespace namespace] in‐
701 terval [start-time]
702 Add trash purge schedule.
703 trash purge schedule list [-R | --recursive] [--format format]
705 [--pretty-format] [-p | --pool pool] [--namespace namespace]
706 List trash purge schedule.
707 trash purge schedule remove [-p | --pool pool] [--namespace namespace]
709 interval [start-time]
710 Remove trash purge schedule.
711 trash purge schedule status [-p | --pool pool] [--format format]
713 [--pretty-format] [--namespace namespace]
714 Show trash purge schedule status.
715 watch image-spec
717 Watch events on image.
718
720 image-spec is [pool-name/[namespace-name/]]image-name
721 snap-spec is [pool-name/[namespace-name/]]image-name@snap-name
722 group-spec is [pool-name/[namespace-name/]]group-name
723 group-snap-spec is [pool-name/[namespace-name/]]group-name@snap-name
724 journal-spec is [pool-name/[namespace-name/]]journal-name
725 The default for pool-name is "rbd" and namespace-name is "". If an im‐
728 age name contains a slash character ('/'), pool-name is required.
729 The journal-name is image-id.
731 You may specify each name individually, using --pool, --namespace,
733 --image, and --snap options, but this is discouraged in favor of the
734 above spec syntax.
735
737 RBD images are striped over many objects, which are then stored by the
738 Ceph distributed object store (RADOS). As a result, read and write re‐
739 quests for the image are distributed across many nodes in the cluster,
740 generally preventing any single node from becoming a bottleneck when
741 individual images get large or busy.
742 The striping is controlled by three parameters:
744 object-size
746 The size of objects we stripe over is a power of two. It will be
747 rounded up the nearest power of two. The default object size is
748 4 MB, smallest is 4K and maximum is 32M.
749 stripe_unit
751 Each [stripe_unit] contiguous bytes are stored adjacently in the
752 same object, before we move on to the next object.
753 stripe_count
755 After we write [stripe_unit] bytes to [stripe_count] objects, we
756 loop back to the initial object and write another stripe, until
757 the object reaches its maximum size. At that point, we move on
758 to the next [stripe_count] objects.
759 By default, [stripe_unit] is the same as the object size and
761 [stripe_count] is 1. Specifying a different [stripe_unit] and/or
762 [stripe_count] is often referred to as using "fancy" striping and re‐
763 quires format 2.
764
766 Most of these options are useful mainly for debugging and benchmarking.
767 The default values are set in the kernel and may therefore depend on
768 the version of the running kernel.
769 Per client instance rbd device map options:
771 • fsid=aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee - FSID that should be as‐
773 sumed by the client.
774 • ip=a.b.c.d[:p] - IP and, optionally, port the client should use.
776 • share - Enable sharing of client instances with other mappings (de‐
778 fault).
779 • noshare - Disable sharing of client instances with other mappings.
781 • crc - Enable CRC32C checksumming for msgr1 on-the-wire protocol (de‐
783 fault). For msgr2.1 protocol this option is ignored: full checksum‐
784 ming is always on in 'crc' mode and always off in 'secure' mode.
785 • nocrc - Disable CRC32C checksumming for msgr1 on-the-wire protocol.
787 Note that only payload checksumming is disabled, header checksumming
788 is always on. For msgr2.1 protocol this option is ignored.
789 • cephx_require_signatures - Require msgr1 message signing feature
791 (since 3.19, default). This option is deprecated and will be removed
792 in the future as the feature has been supported since the Bobtail re‐
793 lease.
794 • nocephx_require_signatures - Don't require msgr1 message signing fea‐
796 ture (since 3.19). This option is deprecated and will be removed in
797 the future.
798 • tcp_nodelay - Disable Nagle's algorithm on client sockets (since 4.0,
800 default).
801 • notcp_nodelay - Enable Nagle's algorithm on client sockets (since
803 4.0).
804 • cephx_sign_messages - Enable message signing for msgr1 on-the-wire
806 protocol (since 4.4, default). For msgr2.1 protocol this option is
807 ignored: message signing is built into 'secure' mode and not offered
808 in 'crc' mode.
809 • nocephx_sign_messages - Disable message signing for msgr1 on-the-wire
811 protocol (since 4.4). For msgr2.1 protocol this option is ignored.
812 • mount_timeout=x - A timeout on various steps in rbd device map and
814 rbd device unmap sequences (default is 60 seconds). In particular,
815 since 4.2 this can be used to ensure that rbd device unmap eventually
816 times out when there is no network connection to a cluster.
817 • osdkeepalive=x - OSD keepalive timeout (default is 5 seconds).
819 • osd_idle_ttl=x - OSD idle TTL (default is 60 seconds).
821 Per mapping (block device) rbd device map options:
823 • rw - Map the image read-write (default). Overridden by --read-only.
825 • ro - Map the image read-only. Equivalent to --read-only.
827 • queue_depth=x - queue depth (since 4.2, default is 128 requests).
829 • lock_on_read - Acquire exclusive lock on reads, in addition to writes
831 and discards (since 4.9).
832 • exclusive - Disable automatic exclusive lock transitions (since
834 4.12). Equivalent to --exclusive.
835 • lock_timeout=x - A timeout on waiting for the acquisition of exclu‐
837 sive lock (since 4.17, default is 0 seconds, meaning no timeout).
838 • notrim - Turn off discard and write zeroes offload support to avoid
840 deprovisioning a fully provisioned image (since 4.17). When enabled,
841 discard requests will fail with -EOPNOTSUPP, write zeroes requests
842 will fall back to manually zeroing.
843 • abort_on_full - Fail write requests with -ENOSPC when the cluster is
845 full or the data pool reaches its quota (since 5.0). The default be‐
846 haviour is to block until the full condition is cleared.
847 • alloc_size - Minimum allocation unit of the underlying OSD object
849 store backend (since 5.1, default is 64K bytes). This is used to
850 round off and drop discards that are too small. For bluestore, the
851 recommended setting is bluestore_min_alloc_size (typically 64K for
852 hard disk drives and 16K for solid-state drives). For filestore with
853 filestore_punch_hole = false, the recommended setting is image object
854 size (typically 4M).
855 • crush_location=x - Specify the location of the client in terms of
857 CRUSH hierarchy (since 5.8). This is a set of key-value pairs sepa‐
858 rated from each other by '|', with keys separated from values by ':'.
859 Note that '|' may need to be quoted or escaped to avoid it being in‐
860 terpreted as a pipe by the shell. The key is the bucket type name
861 (e.g. rack, datacenter or region with default bucket types) and the
862 value is the bucket name. For example, to indicate that the client
863 is local to rack "myrack", data center "mydc" and region "myregion":
864 crush_location=rack:myrack|datacenter:mydc|region:myregion
866 Each key-value pair stands on its own: "myrack" doesn't need to re‐
868 side in "mydc", which in turn doesn't need to reside in "myregion".
869 The location is not a path to the root of the hierarchy but rather a
870 set of nodes that are matched independently, owning to the fact that
871 bucket names are unique within a CRUSH map. "Multipath" locations
872 are supported, so it is possible to indicate locality for multiple
873 parallel hierarchies:
874 crush_location=rack:myrack1|rack:myrack2|datacenter:mydc
876 • read_from_replica=no - Disable replica reads, always pick the primary
878 OSD (since 5.8, default).
879 • read_from_replica=balance - When issued a read on a replicated pool,
881 pick a random OSD for serving it (since 5.8).
882 This mode is safe for general use only since Octopus (i.e. after
884 "ceph osd require-osd-release octopus"). Otherwise it should be lim‐
885 ited to read-only workloads such as images mapped read-only every‐
886 where or snapshots.
887 • read_from_replica=localize - When issued a read on a replicated pool,
889 pick the most local OSD for serving it (since 5.8). The locality
890 metric is calculated against the location of the client given with
891 crush_location; a match with the lowest-valued bucket type wins. For
892 example, with default bucket types, an OSD in a matching rack is
893 closer than an OSD in a matching data center, which in turn is closer
894 than an OSD in a matching region.
895 This mode is safe for general use only since Octopus (i.e. after
897 "ceph osd require-osd-release octopus"). Otherwise it should be lim‐
898 ited to read-only workloads such as images mapped read-only every‐
899 where or snapshots.
900 • compression_hint=none - Don't set compression hints (since 5.8, de‐
902 fault).
903 • compression_hint=compressible - Hint to the underlying OSD object
905 store backend that the data is compressible, enabling compression in
906 passive mode (since 5.8).
907 • compression_hint=incompressible - Hint to the underlying OSD object
909 store backend that the data is incompressible, disabling compression
910 in aggressive mode (since 5.8).
911 • ms_mode=legacy - Use msgr1 on-the-wire protocol (since 5.11, de‐
913 fault).
914 • ms_mode=crc - Use msgr2.1 on-the-wire protocol, select 'crc' mode,
916 also referred to as plain mode (since 5.11). If the daemon denies
917 'crc' mode, fail the connection.
918 • ms_mode=secure - Use msgr2.1 on-the-wire protocol, select 'secure'
920 mode (since 5.11). 'secure' mode provides full in-transit encryption
921 ensuring both confidentiality and authenticity. If the daemon denies
922 'secure' mode, fail the connection.
923 • ms_mode=prefer-crc - Use msgr2.1 on-the-wire protocol, select 'crc'
925 mode (since 5.11). If the daemon denies 'crc' mode in favor of 'se‐
926 cure' mode, agree to 'secure' mode.
927 • ms_mode=prefer-secure - Use msgr2.1 on-the-wire protocol, select 'se‐
929 cure' mode (since 5.11). If the daemon denies 'secure' mode in favor
930 of 'crc' mode, agree to 'crc' mode.
931 • udev - Wait for udev device manager to finish executing all matching
933 "add" rules and release the device before exiting (default). This
934 option is not passed to the kernel.
935 • noudev - Don't wait for udev device manager. When enabled, the de‐
937 vice may not be fully usable immediately on exit.
938 rbd device unmap options:
940 • force - Force the unmapping of a block device that is open (since
942 4.9). The driver will wait for running requests to complete and then
943 unmap; requests sent to the driver after initiating the unmap will be
944 failed.
945 • udev - Wait for udev device manager to finish executing all matching
947 "remove" rules and clean up after the device before exiting (de‐
948 fault). This option is not passed to the kernel.
949 • noudev - Don't wait for udev device manager.
951
953 To create a new rbd image that is 100 GB:
954 rbd create mypool/myimage --size 102400
956 To use a non-default object size (8 MB):
958 rbd create mypool/myimage --size 102400 --object-size 8M
960 To delete an rbd image (be careful!):
962 rbd rm mypool/myimage
964 To create a new snapshot:
966 rbd snap create mypool/myimage@mysnap
968 To create a copy-on-write clone of a protected snapshot:
970 rbd clone mypool/myimage@mysnap otherpool/cloneimage
972 To see which clones of a snapshot exist:
974 rbd children mypool/myimage@mysnap
976 To delete a snapshot:
978 rbd snap rm mypool/myimage@mysnap
980 To map an image via the kernel with cephx enabled:
982 rbd device map mypool/myimage --id admin --keyfile secretfile
984 To map an image via the kernel with different cluster name other than
986 default ceph:
987 rbd device map mypool/myimage --cluster cluster-name
989 To unmap an image:
991 rbd device unmap /dev/rbd0
993 To create an image and a clone from it:
995 rbd import --image-format 2 image mypool/parent
997 rbd snap create mypool/parent@snap
998 rbd snap protect mypool/parent@snap
999 rbd clone mypool/parent@snap otherpool/child
1000 To create an image with a smaller stripe_unit (to better distribute
1002 small writes in some workloads):
1003 rbd create mypool/myimage --size 102400 --stripe-unit 65536B --stripe-count 16
1005 To change an image from one image format to another, export it and then
1007 import it as the desired image format:
1008 rbd export mypool/myimage@snap /tmp/img
1010 rbd import --image-format 2 /tmp/img mypool/myimage2
1011 To lock an image for exclusive use:
1013 rbd lock add mypool/myimage mylockid
1015 To release a lock:
1017 rbd lock remove mypool/myimage mylockid client.2485
1019 To list images from trash:
1021 rbd trash ls mypool
1023 To defer delete an image (use --expires-at to set expiration time, de‐
1025 fault is now):
1026 rbd trash mv mypool/myimage --expires-at "tomorrow"
1028 To delete an image from trash (be careful!):
1030 rbd trash rm mypool/myimage-id
1032 To force delete an image from trash (be careful!):
1034 rbd trash rm mypool/myimage-id --force
1036 To restore an image from trash:
1038 rbd trash restore mypool/myimage-id
1040 To restore an image from trash and rename it:
1042 rbd trash restore mypool/myimage-id --image mynewimage
1044
1046 rbd is part of Ceph, a massively scalable, open-source, distributed
1047 storage system. Please refer to the Ceph documentation at
1048 http://ceph.com/docs for more information.
1049
1051 ceph(8), rados(8)
1052
1054 2010-2021, Inktank Storage, Inc. and contributors. Licensed under Cre‐
1055 ative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0)
1056dev May 13, 2021 RBD(8)