1RBD(8) Ceph RBD(8)
2
6 rbd - manage rados block device (RBD) images
7
9 rbd [ -c ceph.conf ] [ -m monaddr ] [--cluster cluster-name]
10 [ -p | --pool pool ] [ command ... ]
11
14 rbd is a utility for manipulating rados block device (RBD) images, used
15 by the Linux rbd driver and the rbd storage driver for QEMU/KVM. RBD
16 images are simple block devices that are striped over objects and
17 stored in a RADOS object store. The size of the objects the image is
18 striped over must be a power of two.
19
21 -c ceph.conf, --conf ceph.conf
22 Use ceph.conf configuration file instead of the default
23 /etc/ceph/ceph.conf to determine monitor addresses during
24 startup.
25 -m monaddress[:port]
27 Connect to specified monitor (instead of looking through
28 ceph.conf).
29 --cluster cluster-name
31 Use different cluster name as compared to default cluster name
32 ceph.
33 -p pool-name, --pool pool-name
35 Interact with the given pool. Required by most commands.
36 --namespace namespace-name
38 Use a pre-defined image namespace within a pool
39 --no-progress
41 Do not output progress information (goes to standard error by
42 default for some commands).
43
45 --image-format format-id
46 Specifies which object layout to use. The default is 2.
47 · format 1 - (deprecated) Use the original format for a new rbd
49 image. This format is understood by all versions of librbd and
50 the kernel rbd module, but does not support newer features
51 like cloning.
52 · format 2 - Use the second rbd format, which is supported by
54 librbd since the Bobtail release and the kernel rbd module
55 since kernel 3.10 (except for "fancy" striping, which is sup‐
56 ported since kernel 4.17). This adds support for cloning and
57 is more easily extensible to allow more features in the
58 future.
59 -s size-in-M/G/T, --size size-in-M/G/T
61 Specifies the size of the new rbd image or the new size of the
62 existing rbd image in M/G/T. If no suffix is given, unit M is
63 assumed.
64 --object-size size-in-B/K/M
66 Specifies the object size in B/K/M. Object size will be rounded
67 up the nearest power of two; if no suffix is given, unit B is
68 assumed. The default object size is 4M, smallest is 4K and max‐
69 imum is 32M.
70 --stripe-unit size-in-B/K/M
72 Specifies the stripe unit size in B/K/M. If no suffix is given,
73 unit B is assumed. See striping section (below) for more
74 details.
75 --stripe-count num
77 Specifies the number of objects to stripe over before looping
78 back to the first object. See striping section (below) for more
79 details.
80 --snap snap
82 Specifies the snapshot name for the specific operation.
83 --id username
85 Specifies the username (without the client. prefix) to use with
86 the map command.
87 --keyring filename
89 Specifies a keyring file containing a secret for the specified
90 user to use with the map command. If not specified, the default
91 keyring locations will be searched.
92 --keyfile filename
94 Specifies a file containing the secret key of --id user to use
95 with the map command. This option is overridden by --keyring if
96 the latter is also specified.
97 --shared lock-tag
99 Option for lock add that allows multiple clients to lock the
100 same image if they use the same tag. The tag is an arbitrary
101 string. This is useful for situations where an image must be
102 open from more than one client at once, like during live migra‐
103 tion of a virtual machine, or for use underneath a clustered
104 file system.
105 --format format
107 Specifies output formatting (default: plain, json, xml)
108 --pretty-format
110 Make json or xml formatted output more human-readable.
111 -o krbd-options, --options krbd-options
113 Specifies which options to use when mapping or unmapping an
114 image via the rbd kernel driver. krbd-options is a comma-sepa‐
115 rated list of options (similar to mount(8) mount options). See
116 kernel rbd (krbd) options section below for more details.
117 --read-only
119 Map the image read-only. Equivalent to -o ro.
120 --image-feature feature-name
122 Specifies which RBD format 2 feature should be enabled when cre‐
123 ating an image. Multiple features can be enabled by repeating
124 this option multiple times. The following features are sup‐
125 ported:
126 · layering: layering support
128 · striping: striping v2 support
130 · exclusive-lock: exclusive locking support
132 · object-map: object map support (requires exclusive-lock)
134 · fast-diff: fast diff calculations (requires object-map)
136 · deep-flatten: snapshot flatten support
138 · journaling: journaled IO support (requires exclusive-lock)
140 · data-pool: erasure coded pool support
142 --image-shared
144 Specifies that the image will be used concurrently by multiple
145 clients. This will disable features that are dependent upon
146 exclusive ownership of the image.
147 --whole-object
149 Specifies that the diff should be limited to the extents of a
150 full object instead of showing intra-object deltas. When the
151 object map feature is enabled on an image, limiting the diff to
152 the object extents will dramatically improve performance since
153 the differences can be computed by examining the in-memory
154 object map instead of querying RADOS for each object within the
155 image.
156 --limit
158 Specifies the limit for the number of snapshots permitted.
159
161 bench --io-type <read | write | readwrite | rw> [--io-size
162 size-in-B/K/M/G/T] [--io-threads num-ios-in-flight] [--io-total
163 size-in-B/K/M/G/T] [--io-pattern seq | rand] [--rw-mix-read read pro‐
164 portion in readwrite] image-spec
165 Generate a series of IOs to the image and measure the IO
166 throughput and latency. If no suffix is given, unit B is
167 assumed for both --io-size and --io-total. Defaults are:
168 --io-size 4096, --io-threads 16, --io-total 1G, --io-pattern
169 seq, --rw-mix-read 50.
170 children snap-spec
172 List the clones of the image at the given snapshot. This checks
173 every pool, and outputs the resulting poolname/imagename.
174 This requires image format 2.
176 clone [--object-size size-in-B/K/M] [--stripe-unit size-in-B/K/M
178 --stripe-count num] [--image-feature feature-name] [--image-shared]
179 parent-snap-spec child-image-spec
180 Will create a clone (copy-on-write child) of the parent snap‐
181 shot. Object size will be identical to that of the parent image
182 unless specified. Size will be the same as the parent snapshot.
183 The --stripe-unit and --stripe-count arguments are optional, but
184 must be used together.
185 The parent snapshot must be protected (see rbd snap protect).
187 This requires image format 2.
188 config global get config-entity key
190 Get a global-level configuration override.
191 config global list [--format plain | json | xml] [--pretty-format] con‐
193 fig-entity
194 List global-level configuration overrides.
195 config global set config-entity key value
197 Set a global-level configuration override.
198 config global remove config-entity key
200 Remove a global-level configuration override.
201 config image get image-spec key
203 Get an image-level configuration override.
204 config image list [--format plain | json | xml] [--pretty-format]
206 image-spec
207 List image-level configuration overrides.
208 config image set image-spec key value
210 Set an image-level configuration override.
211 config image remove image-spec key
213 Remove an image-level configuration override.
214 config pool get pool-name key
216 Get a pool-level configuration override.
217 config pool list [--format plain | json | xml] [--pretty-format]
219 pool-name
220 List pool-level configuration overrides.
221 config pool set pool-name key value
223 Set a pool-level configuration override.
224 config pool remove pool-name key
226 Remove a pool-level configuration override.
227 cp (src-image-spec | src-snap-spec) dest-image-spec
229 Copy the content of a src-image into the newly created
230 dest-image. dest-image will have the same size, object size,
231 and image format as src-image.
232 create (-s | --size size-in-M/G/T) [--image-format format-id]
234 [--object-size size-in-B/K/M] [--stripe-unit size-in-B/K/M
235 --stripe-count num] [--thick-provision] [--no-progress] [--image-fea‐
236 ture feature-name]... [--image-shared] image-spec
237 Will create a new rbd image. You must also specify the size via
238 --size. The --stripe-unit and --stripe-count arguments are
239 optional, but must be used together. If the --thick-provision
240 is enabled, it will fully allocate storage for the image at cre‐
241 ation time. It will take a long time to do. Note: thick provi‐
242 sioning requires zeroing the contents of the entire image.
243 deep cp (src-image-spec | src-snap-spec) dest-image-spec
245 Deep copy the content of a src-image into the newly created
246 dest-image. Dest-image will have the same size, object size,
247 image format, and snapshots as src-image.
248 device list [-t | --device-type device-type] [--format plain | json |
250 xml] --pretty-format
251 Show the rbd images that are mapped via the rbd kernel module
252 (default) or other supported device.
253 device map [-t | --device-type device-type] [--read-only] [--exclusive]
255 [-o | --options device-options] image-spec | snap-spec
256 Map the specified image to a block device via the rbd kernel
257 module (default) or other supported device (nbd on Linux or
258 ggate on FreeBSD).
259 The --options argument is a comma separated list of device type
261 specific options (opt1,opt2=val,...).
262 device unmap [-t | --device-type device-type] [-o | --options
264 device-options] image-spec | snap-spec | device-path
265 Unmap the block device that was mapped via the rbd kernel module
266 (default) or other supported device.
267 The --options argument is a comma separated list of device type
269 specific options (opt1,opt2=val,...).
270 diff [--from-snap snap-name] [--whole-object] image-spec | snap-spec
272 Dump a list of byte extents in the image that have changed since
273 the specified start snapshot, or since the image was created.
274 Each output line includes the starting offset (in bytes), the
275 length of the region (in bytes), and either 'zero' or 'data' to
276 indicate whether the region is known to be zeros or may contain
277 other data.
278 du [-p | --pool pool-name] [image-spec | snap-spec] [--merge-snapshots]
280 Will calculate the provisioned and actual disk usage of all
281 images and associated snapshots within the specified pool. It
282 can also be used against individual images and snapshots.
283 If the RBD fast-diff feature is not enabled on images, this
285 operation will require querying the OSDs for every potential
286 object within the image.
287 The --merge-snapshots will merge snapshots used space into their
289 parent images.
290 export [--export-format format (1 or 2)] (image-spec | snap-spec)
292 [dest-path]
293 Export image to dest path (use - for stdout). The --export-for‐
294 mat accepts '1' or '2' currently. Format 2 allow us to export
295 not only the content of image, but also the snapshots and other
296 properties, such as image_order, features.
297 export-diff [--from-snap snap-name] [--whole-object] (image-spec |
299 snap-spec) dest-path
300 Export an incremental diff for an image to dest path (use - for
301 stdout). If an initial snapshot is specified, only changes
302 since that snapshot are included; otherwise, any regions of the
303 image that contain data are included. The end snapshot is spec‐
304 ified using the standard --snap option or @snap syntax (see
305 below). The image diff format includes metadata about image
306 size changes, and the start and end snapshots. It efficiently
307 represents discarded or 'zero' regions of the image.
308 feature disable image-spec feature-name...
310 Disable the specified feature on the specified image. Multiple
311 features can be specified.
312 feature enable image-spec feature-name...
314 Enable the specified feature on the specified image. Multiple
315 features can be specified.
316 flatten image-spec
318 If image is a clone, copy all shared blocks from the parent
319 snapshot and make the child independent of the parent, severing
320 the link between parent snap and child. The parent snapshot can
321 be unprotected and deleted if it has no further dependent
322 clones.
323 This requires image format 2.
325 group create group-spec
327 Create a group.
328 group image add group-spec image-spec
330 Add an image to a group.
331 group image list group-spec
333 List images in a group.
334 group image remove group-spec image-spec
336 Remove an image from a group.
337 group ls [-p | --pool pool-name]
339 List rbd groups.
340 group rename src-group-spec dest-group-spec
342 Rename a group. Note: rename across pools is not supported.
343 group rm group-spec
345 Delete a group.
346 group snap create group-snap-spec
348 Make a snapshot of a group.
349 group snap list group-spec
351 List snapshots of a group.
352 group snap rm group-snap-spec
354 Remove a snapshot from a group.
355 group snap rename group-snap-spec snap-name
357 Rename group's snapshot.
358 group snap rollback group-snap-spec
360 Rollback group to snapshot.
361 image-meta get image-spec key
363 Get metadata value with the key.
364 image-meta list image-spec
366 Show metadata held on the image. The first column is the key and
367 the second column is the value.
368 image-meta remove image-spec key
370 Remove metadata key with the value.
371 image-meta set image-spec key value
373 Set metadata key with the value. They will displayed in
374 image-meta list.
375 import [--export-format format (1 or 2)] [--image-format format-id]
377 [--object-size size-in-B/K/M] [--stripe-unit size-in-B/K/M
378 --stripe-count num] [--image-feature feature-name]... [--image-shared]
379 src-path [image-spec]
380 Create a new image and imports its data from path (use - for
381 stdin). The import operation will try to create sparse rbd
382 images if possible. For import from stdin, the sparsification
383 unit is the data block size of the destination image (object
384 size).
385 The --stripe-unit and --stripe-count arguments are optional, but
387 must be used together.
388 The --export-format accepts '1' or '2' currently. Format 2 allow
390 us to import not only the content of image, but also the snap‐
391 shots and other properties, such as image_order, features.
392 import-diff src-path image-spec
394 Import an incremental diff of an image and applies it to the
395 current image. If the diff was generated relative to a start
396 snapshot, we verify that snapshot already exists before continu‐
397 ing. If there was an end snapshot we verify it does not already
398 exist before applying the changes, and create the snapshot when
399 we are done.
400 info image-spec | snap-spec
402 Will dump information (such as size and object size) about a
403 specific rbd image. If image is a clone, information about its
404 parent is also displayed. If a snapshot is specified, whether
405 it is protected is shown as well.
406 journal client disconnect journal-spec
408 Flag image journal client as disconnected.
409 journal export [--verbose] [--no-error] src-journal-spec path-name
411 Export image journal to path (use - for stdout). It can be make
412 a backup of the image journal especially before attempting dan‐
413 gerous operations.
414 Note that this command may not always work if the journal is
416 badly corrupted.
417 journal import [--verbose] [--no-error] path-name dest-journal-spec
419 Import image journal from path (use - for stdin).
420 journal info journal-spec
422 Show information about image journal.
423 journal inspect [--verbose] journal-spec
425 Inspect and report image journal for structural errors.
426 journal reset journal-spec
428 Reset image journal.
429 journal status journal-spec
431 Show status of image journal.
432 lock add [--shared lock-tag] image-spec lock-id
434 Lock an image. The lock-id is an arbitrary name for the user's
435 convenience. By default, this is an exclusive lock, meaning it
436 will fail if the image is already locked. The --shared option
437 changes this behavior. Note that locking does not affect any
438 operation other than adding a lock. It does not protect an image
439 from being deleted.
440 lock ls image-spec
442 Show locks held on the image. The first column is the locker to
443 use with the lock remove command.
444 lock rm image-spec lock-id locker
446 Release a lock on an image. The lock id and locker are as output
447 by lock ls.
448 ls [-l | --long] [pool-name]
450 Will list all rbd images listed in the rbd_directory object.
451 With -l, also show snapshots, and use longer-format output
452 including size, parent (if clone), format, etc.
453 merge-diff first-diff-path second-diff-path merged-diff-path
455 Merge two continuous incremental diffs of an image into one sin‐
456 gle diff. The first diff's end snapshot must be equal with the
457 second diff's start snapshot. The first diff could be - for
458 stdin, and merged diff could be - for stdout, which enables mul‐
459 tiple diff files to be merged using something like 'rbd
460 merge-diff first second - | rbd merge-diff - third result'. Note
461 this command currently only support the source incremental diff
462 with stripe_count == 1
463 migration abort image-spec
465 Cancel image migration. This step may be run after successful or
466 failed migration prepare or migration execute steps and returns
467 the image to its initial (before migration) state. All modifica‐
468 tions to the destination image are lost.
469 migration commit image-spec
471 Commit image migration. This step is run after a successful
472 migration prepare and migration execute steps and removes the
473 source image data.
474 migration execute image-spec
476 Execute image migration. This step is run after a successful
477 migration prepare step and copies image data to the destination.
478 migration prepare [--order order] [--object-size object-size]
480 [--image-feature image-feature] [--image-shared] [--stripe-unit
481 stripe-unit] [--stripe-count stripe-count] [--data-pool data-pool]
482 src-image-spec [dest-image-spec]
483 Prepare image migration. This is the first step when migrating
484 an image, i.e. changing the image location, format or other
485 parameters that can't be changed dynamically. The destination
486 can match the source, and in this case dest-image-spec can be
487 omitted. After this step the source image is set as a parent of
488 the destination image, and the image is accessible in
489 copy-on-write mode by its destination spec.
490 mirror image demote image-spec
492 Demote a primary image to non-primary for RBD mirroring.
493 mirror image disable [--force] image-spec
495 Disable RBD mirroring for an image. If the mirroring is config‐
496 ured in image mode for the image's pool, then it can be explic‐
497 itly disabled mirroring for each image within the pool.
498 mirror image enable image-spec mode
500 Enable RBD mirroring for an image. If the mirroring is config‐
501 ured in image mode for the image's pool, then it can be explic‐
502 itly enabled mirroring for each image within the pool.
503 The mirror image mode can either be journal (default) or snap‐
505 shot. The journal mode requires the RBD journaling feature.
506 mirror image promote [--force] image-spec
508 Promote a non-primary image to primary for RBD mirroring.
509 mirror image resync image-spec
511 Force resync to primary image for RBD mirroring.
512 mirror image status image-spec
514 Show RBD mirroring status for an image.
515 mirror pool demote [pool-name]
517 Demote all primary images within a pool to non-primary. Every
518 mirroring enabled image will demoted in the pool.
519 mirror pool disable [pool-name]
521 Disable RBD mirroring by default within a pool. When mirroring
522 is disabled on a pool in this way, mirroring will also be dis‐
523 abled on any images (within the pool) for which mirroring was
524 enabled explicitly.
525 mirror pool enable [pool-name] mode
527 Enable RBD mirroring by default within a pool. The mirroring
528 mode can either be pool or image. If configured in pool mode,
529 all images in the pool with the journaling feature enabled are
530 mirrored. If configured in image mode, mirroring needs to be
531 explicitly enabled (by mirror image enable command) on each
532 image.
533 mirror pool info [pool-name]
535 Show information about the pool mirroring configuration. It
536 includes mirroring mode, peer UUID, remote cluster name, and
537 remote client name.
538 mirror pool peer add [pool-name] remote-cluster-spec
540 Add a mirroring peer to a pool. remote-cluster-spec is [remote
541 client name@]remote cluster name.
542 The default for remote client name is "client.admin".
544 This requires mirroring mode is enabled.
546 mirror pool peer remove [pool-name] uuid
548 Remove a mirroring peer from a pool. The peer uuid is available
549 from mirror pool info command.
550 mirror pool peer set [pool-name] uuid key value
552 Update mirroring peer settings. The key can be either client or
553 cluster, and the value is corresponding to remote client name or
554 remote cluster name.
555 mirror pool promote [--force] [pool-name]
557 Promote all non-primary images within a pool to primary. Every
558 mirroring enabled image will promoted in the pool.
559 mirror pool status [--verbose] [pool-name]
561 Show status for all mirrored images in the pool. With --ver‐
562 bose, also show additionally output status details for every
563 mirroring image in the pool.
564 mirror snapshot schedule add [-p | --pool pool] [--namespace namespace]
566 [--image image] interval [start-time]
567 Add mirror snapshot schedule.
568 mirror snapshot schedule list [-R | --recursive] [--format format]
570 [--pretty-format] [-p | --pool pool] [--namespace namespace] [--image
571 image]
572 List mirror snapshot schedule.
573 mirror snapshot schedule remove [-p | --pool pool] [--namespace names‐
575 pace] [--image image] interval [start-time]
576 Remove mirror snapshot schedule.
577 mirror snapshot schedule status [-p | --pool pool] [--format format]
579 [--pretty-format] [--namespace namespace] [--image image]
580 Show mirror snapshot schedule status.
581 mv src-image-spec dest-image-spec
583 Rename an image. Note: rename across pools is not supported.
584 namespace create pool-name/namespace-name
586 Create a new image namespace within the pool.
587 namespace list pool-name
589 List image namespaces defined within the pool.
590 namespace remove pool-name/namespace-name
592 Remove an empty image namespace from the pool.
593 object-map check image-spec | snap-spec
595 Verify the object map is correct.
596 object-map rebuild image-spec | snap-spec
598 Rebuild an invalid object map for the specified image. An image
599 snapshot can be specified to rebuild an invalid object map for a
600 snapshot.
601 pool init [pool-name] [--force]
603 Initialize pool for use by RBD. Newly created pools must ini‐
604 tialized prior to use.
605 resize (-s | --size size-in-M/G/T) [--allow-shrink] image-spec
607 Resize rbd image. The size parameter also needs to be specified.
608 The --allow-shrink option lets the size be reduced.
609 rm image-spec
611 Delete an rbd image (including all data blocks). If the image
612 has snapshots, this fails and nothing is deleted.
613 snap create snap-spec
615 Create a new snapshot. Requires the snapshot name parameter
616 specified.
617 snap limit clear image-spec
619 Remove any previously set limit on the number of snapshots
620 allowed on an image.
621 snap limit set [--limit] limit image-spec
623 Set a limit for the number of snapshots allowed on an image.
624 snap ls image-spec
626 Dump the list of snapshots inside a specific image.
627 snap protect snap-spec
629 Protect a snapshot from deletion, so that clones can be made of
630 it (see rbd clone). Snapshots must be protected before clones
631 are made; protection implies that there exist dependent cloned
632 children that refer to this snapshot. rbd clone will fail on a
633 nonprotected snapshot.
634 This requires image format 2.
636 snap purge image-spec
638 Remove all unprotected snapshots from an image.
639 snap rename src-snap-spec dest-snap-spec
641 Rename a snapshot. Note: rename across pools and images is not
642 supported.
643 snap rm [--force] snap-spec
645 Remove the specified snapshot.
646 snap rollback snap-spec
648 Rollback image content to snapshot. This will iterate through
649 the entire blocks array and update the data head content to the
650 snapshotted version.
651 snap unprotect snap-spec
653 Unprotect a snapshot from deletion (undo snap protect). If
654 cloned children remain, snap unprotect fails. (Note that clones
655 may exist in different pools than the parent snapshot.)
656 This requires image format 2.
658 sparsify [--sparse-size sparse-size] image-spec
660 Reclaim space for zeroed image extents. The default sparse size
661 is 4096 bytes and can be changed via --sparse-size option with
662 the following restrictions: it should be power of two, not less
663 than 4096, and not larger than image object size.
664 status image-spec
666 Show the status of the image, including which clients have it
667 open.
668 trash ls [pool-name]
670 List all entries from trash.
671 trash mv image-spec
673 Move an image to the trash. Images, even ones actively in-use by
674 clones, can be moved to the trash and deleted at a later time.
675 trash purge [pool-name]
677 Remove all expired images from trash.
678 trash restore image-id
680 Restore an image from trash.
681 trash rm image-id
683 Delete an image from trash. If image deferment time has not
684 expired you can not removed it unless use force. But an actively
685 in-use by clones or has snapshots can not be removed.
686 trash purge schedule add [-p | --pool pool] [--namespace namespace]
688 interval [start-time]
689 Add trash purge schedule.
690 trash purge schedule list [-R | --recursive] [--format format]
692 [--pretty-format] [-p | --pool pool] [--namespace namespace]
693 List trash purge schedule.
694 trash purge schedule remove [-p | --pool pool] [--namespace namespace]
696 interval [start-time]
697 Remove trash purge schedule.
698 trash purge schedule status [-p | --pool pool] [--format format]
700 [--pretty-format] [--namespace namespace]
701 Show trash purge schedule status.
702 watch image-spec
704 Watch events on image.
705
707 image-spec is [pool-name/[namespace-name/]]image-name
708 snap-spec is [pool-name/[namespace-name/]]image-name@snap-name
709 group-spec is [pool-name/[namespace-name/]]group-name
710 group-snap-spec is [pool-name/[namespace-name/]]group-name@snap-name
711 journal-spec is [pool-name/[namespace-name/]]journal-name
712 The default for pool-name is "rbd" and namespace-name is "". If an
715 image name contains a slash character ('/'), pool-name is required.
716 The journal-name is image-id.
718 You may specify each name individually, using --pool, --namespace,
720 --image, and --snap options, but this is discouraged in favor of the
721 above spec syntax.
722
724 RBD images are striped over many objects, which are then stored by the
725 Ceph distributed object store (RADOS). As a result, read and write
726 requests for the image are distributed across many nodes in the clus‐
727 ter, generally preventing any single node from becoming a bottleneck
728 when individual images get large or busy.
729 The striping is controlled by three parameters:
731 object-size
733 The size of objects we stripe over is a power of two. It will be
734 rounded up the nearest power of two. The default object size is
735 4 MB, smallest is 4K and maximum is 32M.
736 stripe_unit
738 Each [stripe_unit] contiguous bytes are stored adjacently in the
739 same object, before we move on to the next object.
740 stripe_count
742 After we write [stripe_unit] bytes to [stripe_count] objects, we
743 loop back to the initial object and write another stripe, until
744 the object reaches its maximum size. At that point, we move on
745 to the next [stripe_count] objects.
746 By default, [stripe_unit] is the same as the object size and
748 [stripe_count] is 1. Specifying a different [stripe_unit] and/or
749 [stripe_count] is often referred to as using "fancy" striping and
750 requires format 2.
751
753 Most of these options are useful mainly for debugging and benchmarking.
754 The default values are set in the kernel and may therefore depend on
755 the version of the running kernel.
756 Per client instance rbd device map options:
758 · fsid=aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee - FSID that should be
760 assumed by the client.
761 · ip=a.b.c.d[:p] - IP and, optionally, port the client should use.
763 · share - Enable sharing of client instances with other mappings
765 (default).
766 · noshare - Disable sharing of client instances with other mappings.
768 · crc - Enable CRC32C checksumming for msgr1 on-the-wire protocol
770 (default). For msgr2.1 protocol this option is ignored: full check‐
771 summing is always on in 'crc' mode and always off in 'secure' mode.
772 · nocrc - Disable CRC32C checksumming for msgr1 on-the-wire protocol.
774 Note that only payload checksumming is disabled, header checksumming
775 is always on. For msgr2.1 protocol this option is ignored.
776 · cephx_require_signatures - Require msgr1 message signing feature
778 (since 3.19, default). This option is deprecated and will be removed
779 in the future as the feature has been supported since the Bobtail
780 release.
781 · nocephx_require_signatures - Don't require msgr1 message signing fea‐
783 ture (since 3.19). This option is deprecated and will be removed in
784 the future.
785 · tcp_nodelay - Disable Nagle's algorithm on client sockets (since 4.0,
787 default).
788 · notcp_nodelay - Enable Nagle's algorithm on client sockets (since
790 4.0).
791 · cephx_sign_messages - Enable message signing for msgr1 on-the-wire
793 protocol (since 4.4, default). For msgr2.1 protocol this option is
794 ignored: message signing is built into 'secure' mode and not offered
795 in 'crc' mode.
796 · nocephx_sign_messages - Disable message signing for msgr1 on-the-wire
798 protocol (since 4.4). For msgr2.1 protocol this option is ignored.
799 · mount_timeout=x - A timeout on various steps in rbd device map and
801 rbd device unmap sequences (default is 60 seconds). In particular,
802 since 4.2 this can be used to ensure that rbd device unmap eventually
803 times out when there is no network connection to a cluster.
804 · osdkeepalive=x - OSD keepalive timeout (default is 5 seconds).
806 · osd_idle_ttl=x - OSD idle TTL (default is 60 seconds).
808 Per mapping (block device) rbd device map options:
810 · rw - Map the image read-write (default). Overridden by --read-only.
812 · ro - Map the image read-only. Equivalent to --read-only.
814 · queue_depth=x - queue depth (since 4.2, default is 128 requests).
816 · lock_on_read - Acquire exclusive lock on reads, in addition to writes
818 and discards (since 4.9).
819 · exclusive - Disable automatic exclusive lock transitions (since
821 4.12). Equivalent to --exclusive.
822 · lock_timeout=x - A timeout on waiting for the acquisition of exclu‐
824 sive lock (since 4.17, default is 0 seconds, meaning no timeout).
825 · notrim - Turn off discard and write zeroes offload support to avoid
827 deprovisioning a fully provisioned image (since 4.17). When enabled,
828 discard requests will fail with -EOPNOTSUPP, write zeroes requests
829 will fall back to manually zeroing.
830 · abort_on_full - Fail write requests with -ENOSPC when the cluster is
832 full or the data pool reaches its quota (since 5.0). The default be‐
833 haviour is to block until the full condition is cleared.
834 · alloc_size - Minimum allocation unit of the underlying OSD object
836 store backend (since 5.1, default is 64K bytes). This is used to
837 round off and drop discards that are too small. For bluestore, the
838 recommended setting is bluestore_min_alloc_size (typically 64K for
839 hard disk drives and 16K for solid-state drives). For filestore with
840 filestore_punch_hole = false, the recommended setting is image object
841 size (typically 4M).
842 · crush_location=x - Specify the location of the client in terms of
844 CRUSH hierarchy (since 5.8). This is a set of key-value pairs sepa‐
845 rated from each other by '|', with keys separated from values by ':'.
846 Note that '|' may need to be quoted or escaped to avoid it being
847 interpreted as a pipe by the shell. The key is the bucket type name
848 (e.g. rack, datacenter or region with default bucket types) and the
849 value is the bucket name. For example, to indicate that the client
850 is local to rack "myrack", data center "mydc" and region "myregion":
851 crush_location=rack:myrack|datacenter:mydc|region:myregion
853 Each key-value pair stands on its own: "myrack" doesn't need to
855 reside in "mydc", which in turn doesn't need to reside in "myregion".
856 The location is not a path to the root of the hierarchy but rather a
857 set of nodes that are matched independently, owning to the fact that
858 bucket names are unique within a CRUSH map. "Multipath" locations
859 are supported, so it is possible to indicate locality for multiple
860 parallel hierarchies:
861 crush_location=rack:myrack1|rack:myrack2|datacenter:mydc
863 · read_from_replica=no - Disable replica reads, always pick the primary
865 OSD (since 5.8, default).
866 · read_from_replica=balance - When issued a read on a replicated pool,
868 pick a random OSD for serving it (since 5.8).
869 This mode is safe for general use only since Octopus (i.e. after
871 "ceph osd require-osd-release octopus"). Otherwise it should be lim‐
872 ited to read-only workloads such as images mapped read-only every‐
873 where or snapshots.
874 · read_from_replica=localize - When issued a read on a replicated pool,
876 pick the most local OSD for serving it (since 5.8). The locality
877 metric is calculated against the location of the client given with
878 crush_location; a match with the lowest-valued bucket type wins. For
879 example, with default bucket types, an OSD in a matching rack is
880 closer than an OSD in a matching data center, which in turn is closer
881 than an OSD in a matching region.
882 This mode is safe for general use only since Octopus (i.e. after
884 "ceph osd require-osd-release octopus"). Otherwise it should be lim‐
885 ited to read-only workloads such as images mapped read-only every‐
886 where or snapshots.
887 · compression_hint=none - Don't set compression hints (since 5.8,
889 default).
890 · compression_hint=compressible - Hint to the underlying OSD object
892 store backend that the data is compressible, enabling compression in
893 passive mode (since 5.8).
894 · compression_hint=incompressible - Hint to the underlying OSD object
896 store backend that the data is incompressible, disabling compression
897 in aggressive mode (since 5.8).
898 · ms_mode=legacy - Use msgr1 on-the-wire protocol (since 5.11,
900 default).
901 · ms_mode=crc - Use msgr2.1 on-the-wire protocol, select 'crc' mode,
903 also referred to as plain mode (since 5.11). If the daemon denies
904 'crc' mode, fail the connection.
905 · ms_mode=secure - Use msgr2.1 on-the-wire protocol, select 'secure'
907 mode (since 5.11). 'secure' mode provides full in-transit encryption
908 ensuring both confidentiality and authenticity. If the daemon denies
909 'secure' mode, fail the connection.
910 · ms_mode=prefer-crc - Use msgr2.1 on-the-wire protocol, select 'crc'
912 mode (since 5.11). If the daemon denies 'crc' mode in favor of
913 'secure' mode, agree to 'secure' mode.
914 · ms_mode=prefer-secure - Use msgr2.1 on-the-wire protocol, select
916 'secure' mode (since 5.11). If the daemon denies 'secure' mode in
917 favor of 'crc' mode, agree to 'crc' mode.
918 · udev - Wait for udev device manager to finish executing all matching
920 "add" rules and release the device before exiting (default). This
921 option is not passed to the kernel.
922 · noudev - Don't wait for udev device manager. When enabled, the
924 device may not be fully usable immediately on exit.
925 rbd device unmap options:
927 · force - Force the unmapping of a block device that is open (since
929 4.9). The driver will wait for running requests to complete and then
930 unmap; requests sent to the driver after initiating the unmap will be
931 failed.
932 · udev - Wait for udev device manager to finish executing all matching
934 "remove" rules and clean up after the device before exiting
935 (default). This option is not passed to the kernel.
936 · noudev - Don't wait for udev device manager.
938
940 To create a new rbd image that is 100 GB:
941 rbd create mypool/myimage --size 102400
943 To use a non-default object size (8 MB):
945 rbd create mypool/myimage --size 102400 --object-size 8M
947 To delete an rbd image (be careful!):
949 rbd rm mypool/myimage
951 To create a new snapshot:
953 rbd snap create mypool/myimage@mysnap
955 To create a copy-on-write clone of a protected snapshot:
957 rbd clone mypool/myimage@mysnap otherpool/cloneimage
959 To see which clones of a snapshot exist:
961 rbd children mypool/myimage@mysnap
963 To delete a snapshot:
965 rbd snap rm mypool/myimage@mysnap
967 To map an image via the kernel with cephx enabled:
969 rbd device map mypool/myimage --id admin --keyfile secretfile
971 To map an image via the kernel with different cluster name other than
973 default ceph:
974 rbd device map mypool/myimage --cluster cluster-name
976 To unmap an image:
978 rbd device unmap /dev/rbd0
980 To create an image and a clone from it:
982 rbd import --image-format 2 image mypool/parent
984 rbd snap create mypool/parent@snap
985 rbd snap protect mypool/parent@snap
986 rbd clone mypool/parent@snap otherpool/child
987 To create an image with a smaller stripe_unit (to better distribute
989 small writes in some workloads):
990 rbd create mypool/myimage --size 102400 --stripe-unit 65536B --stripe-count 16
992 To change an image from one image format to another, export it and then
994 import it as the desired image format:
995 rbd export mypool/myimage@snap /tmp/img
997 rbd import --image-format 2 /tmp/img mypool/myimage2
998 To lock an image for exclusive use:
1000 rbd lock add mypool/myimage mylockid
1002 To release a lock:
1004 rbd lock remove mypool/myimage mylockid client.2485
1006 To list images from trash:
1008 rbd trash ls mypool
1010 To defer delete an image (use --expires-at to set expiration time,
1012 default is now):
1013 rbd trash mv mypool/myimage --expires-at "tomorrow"
1015 To delete an image from trash (be careful!):
1017 rbd trash rm mypool/myimage-id
1019 To force delete an image from trash (be careful!):
1021 rbd trash rm mypool/myimage-id --force
1023 To restore an image from trash:
1025 rbd trash restore mypool/myimage-id
1027 To restore an image from trash and rename it:
1029 rbd trash restore mypool/myimage-id --image mynewimage
1031
1033 rbd is part of Ceph, a massively scalable, open-source, distributed
1034 storage system. Please refer to the Ceph documentation at
1035 http://ceph.com/docs for more information.
1036
1038 ceph(8), rados(8)
1039
1041 2010-2021, Inktank Storage, Inc. and contributors. Licensed under Cre‐
1042 ative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0)
1043dev Mar 18, 2021 RBD(8)