1RBD(8) Ceph RBD(8)
2
3
4
6 rbd - manage rados block device (RBD) images
7
9 rbd [ -c ceph.conf ] [ -m monaddr ] [--cluster cluster-name]
10 [ -p | --pool pool ] [ command ... ]
11
12
14 rbd is a utility for manipulating rados block device (RBD) images, used
15 by the Linux rbd driver and the rbd storage driver for QEMU/KVM. RBD
16 images are simple block devices that are striped over objects and
17 stored in a RADOS object store. The size of the objects the image is
18 striped over must be a power of two.
19
21 -c ceph.conf, --conf ceph.conf
22 Use ceph.conf configuration file instead of the default
23 /etc/ceph/ceph.conf to determine monitor addresses during
24 startup.
25
26 -m monaddress[:port]
27 Connect to specified monitor (instead of looking through
28 ceph.conf).
29
30 --cluster cluster-name
31 Use different cluster name as compared to default cluster name
32 ceph.
33
34 -p pool-name, --pool pool-name
35 Interact with the given pool. Required by most commands.
36
37 --namespace namespace-name
38 Use a pre-defined image namespace within a pool
39
40 --no-progress
41 Do not output progress information (goes to standard error by
42 default for some commands).
43
45 --image-format format-id
46 Specifies which object layout to use. The default is 2.
47
48 • format 1 - (deprecated) Use the original format for a new rbd
49 image. This format is understood by all versions of librbd and
50 the kernel rbd module, but does not support newer features
51 like cloning.
52
53 • format 2 - Use the second rbd format, which is supported by
54 librbd since the Bobtail release and the kernel rbd module
55 since kernel 3.10 (except for "fancy" striping, which is sup‐
56 ported since kernel 4.17). This adds support for cloning and
57 is more easily extensible to allow more features in the fu‐
58 ture.
59
60 -s size-in-M/G/T, --size size-in-M/G/T
61 Specifies the size of the new rbd image or the new size of the
62 existing rbd image in M/G/T. If no suffix is given, unit M is
63 assumed.
64
65 --object-size size-in-B/K/M
66 Specifies the object size in B/K/M. Object size will be rounded
67 up the nearest power of two; if no suffix is given, unit B is
68 assumed. The default object size is 4M, smallest is 4K and max‐
69 imum is 32M.
70
71 The default value can be changed with the configuration option
72 rbd_default_order, which takes a power of two (default object
73 size is 2 ^ rbd_default_order).
74
75 --stripe-unit size-in-B/K/M
76 Specifies the stripe unit size in B/K/M. If no suffix is given,
77 unit B is assumed. See striping section (below) for more de‐
78 tails.
79
80 --stripe-count num
81 Specifies the number of objects to stripe over before looping
82 back to the first object. See striping section (below) for more
83 details.
84
85 --snap snap
86 Specifies the snapshot name for the specific operation.
87
88 --id username
89 Specifies the username (without the client. prefix) to use with
90 the map command.
91
92 --keyring filename
93 Specifies a keyring file containing a secret for the specified
94 user to use with the map command. If not specified, the default
95 keyring locations will be searched.
96
97 --keyfile filename
98 Specifies a file containing the secret key of --id user to use
99 with the map command. This option is overridden by --keyring if
100 the latter is also specified.
101
102 --shared lock-tag
103 Option for lock add that allows multiple clients to lock the
104 same image if they use the same tag. The tag is an arbitrary
105 string. This is useful for situations where an image must be
106 open from more than one client at once, like during live migra‐
107 tion of a virtual machine, or for use underneath a clustered
108 file system.
109
110 --format format
111 Specifies output formatting (default: plain, json, xml)
112
113 --pretty-format
114 Make json or xml formatted output more human-readable.
115
116 -o krbd-options, --options krbd-options
117 Specifies which options to use when mapping or unmapping an im‐
118 age via the rbd kernel driver. krbd-options is a comma-sepa‐
119 rated list of options (similar to mount(8) mount options). See
120 kernel rbd (krbd) options section below for more details.
121
122 --read-only
123 Map the image read-only. Equivalent to -o ro.
124
125 --image-feature feature-name
126 Specifies which RBD format 2 feature should be enabled when cre‐
127 ating an image. Multiple features can be enabled by repeating
128 this option multiple times. The following features are sup‐
129 ported:
130
131 • layering: layering support
132
133 • striping: striping v2 support
134
135 • exclusive-lock: exclusive locking support
136
137 • object-map: object map support (requires exclusive-lock)
138
139 • fast-diff: fast diff calculations (requires object-map)
140
141 • deep-flatten: snapshot flatten support
142
143 • journaling: journaled IO support (requires exclusive-lock)
144
145 • data-pool: erasure coded pool support
146
147 --image-shared
148 Specifies that the image will be used concurrently by multiple
149 clients. This will disable features that are dependent upon ex‐
150 clusive ownership of the image.
151
152 --whole-object
153 Specifies that the diff should be limited to the extents of a
154 full object instead of showing intra-object deltas. When the ob‐
155 ject map feature is enabled on an image, limiting the diff to
156 the object extents will dramatically improve performance since
157 the differences can be computed by examining the in-memory ob‐
158 ject map instead of querying RADOS for each object within the
159 image.
160
161 --limit
162 Specifies the limit for the number of snapshots permitted.
163
165 bench --io-type <read | write | readwrite | rw> [--io-size
166 size-in-B/K/M/G/T] [--io-threads num-ios-in-flight] [--io-total
167 size-in-B/K/M/G/T] [--io-pattern seq | rand] [--rw-mix-read read pro‐
168 portion in readwrite] image-spec
169 Generate a series of IOs to the image and measure the IO
170 throughput and latency. If no suffix is given, unit B is as‐
171 sumed for both --io-size and --io-total. Defaults are:
172 --io-size 4096, --io-threads 16, --io-total 1G, --io-pattern
173 seq, --rw-mix-read 50.
174
175 children snap-spec
176 List the clones of the image at the given snapshot. This checks
177 every pool, and outputs the resulting poolname/imagename.
178
179 This requires image format 2.
180
181 clone [--object-size size-in-B/K/M] [--stripe-unit size-in-B/K/M
182 --stripe-count num] [--image-feature feature-name] [--image-shared]
183 parent-snap-spec child-image-spec
184 Will create a clone (copy-on-write child) of the parent snap‐
185 shot. Object size will be identical to that of the parent image
186 unless specified. Size will be the same as the parent snapshot.
187 The --stripe-unit and --stripe-count arguments are optional, but
188 must be used together.
189
190 The parent snapshot must be protected (see rbd snap protect).
191 This requires image format 2.
192
193 config global get config-entity key
194 Get a global-level configuration override.
195
196 config global list [--format plain | json | xml] [--pretty-format] con‐
197 fig-entity
198 List global-level configuration overrides.
199
200 config global set config-entity key value
201 Set a global-level configuration override.
202
203 config global remove config-entity key
204 Remove a global-level configuration override.
205
206 config image get image-spec key
207 Get an image-level configuration override.
208
209 config image list [--format plain | json | xml] [--pretty-format] im‐
210 age-spec
211 List image-level configuration overrides.
212
213 config image set image-spec key value
214 Set an image-level configuration override.
215
216 config image remove image-spec key
217 Remove an image-level configuration override.
218
219 config pool get pool-name key
220 Get a pool-level configuration override.
221
222 config pool list [--format plain | json | xml] [--pretty-format]
223 pool-name
224 List pool-level configuration overrides.
225
226 config pool set pool-name key value
227 Set a pool-level configuration override.
228
229 config pool remove pool-name key
230 Remove a pool-level configuration override.
231
232 cp (src-image-spec | src-snap-spec) dest-image-spec
233 Copy the content of a src-image into the newly created dest-im‐
234 age. dest-image will have the same size, object size, and image
235 format as src-image. Note: snapshots are not copied, use deep
236 cp command to include snapshots.
237
238 create (-s | --size size-in-M/G/T) [--image-format format-id] [--ob‐
239 ject-size size-in-B/K/M] [--stripe-unit size-in-B/K/M --stripe-count
240 num] [--thick-provision] [--no-progress] [--image-feature fea‐
241 ture-name]... [--image-shared] image-spec
242 Will create a new rbd image. You must also specify the size via
243 --size. The --stripe-unit and --stripe-count arguments are op‐
244 tional, but must be used together. If the --thick-provision is
245 enabled, it will fully allocate storage for the image at cre‐
246 ation time. It will take a long time to do. Note: thick provi‐
247 sioning requires zeroing the contents of the entire image.
248
249 deep cp (src-image-spec | src-snap-spec) dest-image-spec
250 Deep copy the content of a src-image into the newly created
251 dest-image. Dest-image will have the same size, object size,
252 image format, and snapshots as src-image.
253
254 device list [-t | --device-type device-type] [--format plain | json |
255 xml] --pretty-format
256 Show the rbd images that are mapped via the rbd kernel module
257 (default) or other supported device.
258
259 device map [-t | --device-type device-type] [--cookie device-cookie]
260 [--show-cookie] [--read-only] [--exclusive] [-o | --options device-op‐
261 tions] image-spec | snap-spec
262 Map the specified image to a block device via the rbd kernel
263 module (default) or other supported device (nbd on Linux or
264 ggate on FreeBSD).
265
266 The --options argument is a comma separated list of device type
267 specific options (opt1,opt2=val,...).
268
269 device unmap [-t | --device-type device-type] [-o | --options de‐
270 vice-options] image-spec | snap-spec | device-path
271 Unmap the block device that was mapped via the rbd kernel module
272 (default) or other supported device.
273
274 The --options argument is a comma separated list of device type
275 specific options (opt1,opt2=val,...).
276
277 device attach [-t | --device-type device-type] --device device-path
278 [--cookie device-cookie] [--show-cookie] [--read-only] [--exclusive]
279 [--force] [-o | --options device-options] image-spec | snap-spec
280 Attach the specified image to the specified block device (cur‐
281 rently only nbd on Linux). This operation is unsafe and should
282 not be normally used. In particular, specifying the wrong image
283 or the wrong block device may lead to data corruption as no val‐
284 idation is performed by nbd kernel driver.
285
286 The --options argument is a comma separated list of device type
287 specific options (opt1,opt2=val,...).
288
289 device detach [-t | --device-type device-type] [-o | --options de‐
290 vice-options] image-spec | snap-spec | device-path
291 Detach the block device that was mapped or attached (currently
292 only nbd on Linux). This operation is unsafe and should not be
293 normally used.
294
295 The --options argument is a comma separated list of device type
296 specific options (opt1,opt2=val,...).
297
298 diff [--from-snap snap-name] [--whole-object] image-spec | snap-spec
299 Dump a list of byte extents in the image that have changed since
300 the specified start snapshot, or since the image was created.
301 Each output line includes the starting offset (in bytes), the
302 length of the region (in bytes), and either 'zero' or 'data' to
303 indicate whether the region is known to be zeros or may contain
304 other data.
305
306 du [-p | --pool pool-name] [image-spec | snap-spec] [--merge-snapshots]
307 Will calculate the provisioned and actual disk usage of all im‐
308 ages and associated snapshots within the specified pool. It can
309 also be used against individual images and snapshots.
310
311 If the RBD fast-diff feature is not enabled on images, this op‐
312 eration will require querying the OSDs for every potential ob‐
313 ject within the image.
314
315 The --merge-snapshots will merge snapshots used space into their
316 parent images.
317
318 encryption format image-spec format passphrase-file [--cipher-alg alg]
319 Formats image to an encrypted format. All data previously writ‐
320 ten to the image will become unreadable. A cloned image cannot
321 be formatted, although encrypted images can be cloned. Sup‐
322 ported formats: luks1, luks2. Supported cipher algorithms:
323 aes-128, aes-256 (default).
324
325 export [--export-format format (1 or 2)] (image-spec | snap-spec)
326 [dest-path]
327 Export image to dest path (use - for stdout). The --export-for‐
328 mat accepts '1' or '2' currently. Format 2 allow us to export
329 not only the content of image, but also the snapshots and other
330 properties, such as image_order, features.
331
332 export-diff [--from-snap snap-name] [--whole-object] (image-spec |
333 snap-spec) dest-path
334 Export an incremental diff for an image to dest path (use - for
335 stdout). If an initial snapshot is specified, only changes
336 since that snapshot are included; otherwise, any regions of the
337 image that contain data are included. The end snapshot is spec‐
338 ified using the standard --snap option or @snap syntax (see be‐
339 low). The image diff format includes metadata about image size
340 changes, and the start and end snapshots. It efficiently repre‐
341 sents discarded or 'zero' regions of the image.
342
343 feature disable image-spec feature-name...
344 Disable the specified feature on the specified image. Multiple
345 features can be specified.
346
347 feature enable image-spec feature-name...
348 Enable the specified feature on the specified image. Multiple
349 features can be specified.
350
351 flatten image-spec
352 If image is a clone, copy all shared blocks from the parent
353 snapshot and make the child independent of the parent, severing
354 the link between parent snap and child. The parent snapshot can
355 be unprotected and deleted if it has no further dependent
356 clones.
357
358 This requires image format 2.
359
360 group create group-spec
361 Create a group.
362
363 group image add group-spec image-spec
364 Add an image to a group.
365
366 group image list group-spec
367 List images in a group.
368
369 group image remove group-spec image-spec
370 Remove an image from a group.
371
372 group ls [-p | --pool pool-name]
373 List rbd groups.
374
375 group rename src-group-spec dest-group-spec
376 Rename a group. Note: rename across pools is not supported.
377
378 group rm group-spec
379 Delete a group.
380
381 group snap create group-snap-spec
382 Make a snapshot of a group.
383
384 group snap list group-spec
385 List snapshots of a group.
386
387 group snap rm group-snap-spec
388 Remove a snapshot from a group.
389
390 group snap rename group-snap-spec snap-name
391 Rename group's snapshot.
392
393 group snap rollback group-snap-spec
394 Rollback group to snapshot.
395
396 image-meta get image-spec key
397 Get metadata value with the key.
398
399 image-meta list image-spec
400 Show metadata held on the image. The first column is the key and
401 the second column is the value.
402
403 image-meta remove image-spec key
404 Remove metadata key with the value.
405
406 image-meta set image-spec key value
407 Set metadata key with the value. They will displayed in im‐
408 age-meta list.
409
410 import [--export-format format (1 or 2)] [--image-format format-id]
411 [--object-size size-in-B/K/M] [--stripe-unit size-in-B/K/M
412 --stripe-count num] [--image-feature feature-name]... [--image-shared]
413 src-path [image-spec]
414 Create a new image and imports its data from path (use - for
415 stdin). The import operation will try to create sparse rbd im‐
416 ages if possible. For import from stdin, the sparsification
417 unit is the data block size of the destination image (object
418 size).
419
420 The --stripe-unit and --stripe-count arguments are optional, but
421 must be used together.
422
423 The --export-format accepts '1' or '2' currently. Format 2 allow
424 us to import not only the content of image, but also the snap‐
425 shots and other properties, such as image_order, features.
426
427 import-diff src-path image-spec
428 Import an incremental diff of an image and applies it to the
429 current image. If the diff was generated relative to a start
430 snapshot, we verify that snapshot already exists before continu‐
431 ing. If there was an end snapshot we verify it does not already
432 exist before applying the changes, and create the snapshot when
433 we are done.
434
435 info image-spec | snap-spec
436 Will dump information (such as size and object size) about a
437 specific rbd image. If image is a clone, information about its
438 parent is also displayed. If a snapshot is specified, whether
439 it is protected is shown as well.
440
441 journal client disconnect journal-spec
442 Flag image journal client as disconnected.
443
444 journal export [--verbose] [--no-error] src-journal-spec path-name
445 Export image journal to path (use - for stdout). It can be make
446 a backup of the image journal especially before attempting dan‐
447 gerous operations.
448
449 Note that this command may not always work if the journal is
450 badly corrupted.
451
452 journal import [--verbose] [--no-error] path-name dest-journal-spec
453 Import image journal from path (use - for stdin).
454
455 journal info journal-spec
456 Show information about image journal.
457
458 journal inspect [--verbose] journal-spec
459 Inspect and report image journal for structural errors.
460
461 journal reset journal-spec
462 Reset image journal.
463
464 journal status journal-spec
465 Show status of image journal.
466
467 lock add [--shared lock-tag] image-spec lock-id
468 Lock an image. The lock-id is an arbitrary name for the user's
469 convenience. By default, this is an exclusive lock, meaning it
470 will fail if the image is already locked. The --shared option
471 changes this behavior. Note that locking does not affect any op‐
472 eration other than adding a lock. It does not protect an image
473 from being deleted.
474
475 lock ls image-spec
476 Show locks held on the image. The first column is the locker to
477 use with the lock remove command.
478
479 lock rm image-spec lock-id locker
480 Release a lock on an image. The lock id and locker are as output
481 by lock ls.
482
483 ls [-l | --long] [pool-name]
484 Will list all rbd images listed in the rbd_directory object.
485 With -l, also show snapshots, and use longer-format output in‐
486 cluding size, parent (if clone), format, etc.
487
488 merge-diff first-diff-path second-diff-path merged-diff-path
489 Merge two continuous incremental diffs of an image into one sin‐
490 gle diff. The first diff's end snapshot must be equal with the
491 second diff's start snapshot. The first diff could be - for
492 stdin, and merged diff could be - for stdout, which enables mul‐
493 tiple diff files to be merged using something like 'rbd
494 merge-diff first second - | rbd merge-diff - third result'. Note
495 this command currently only support the source incremental diff
496 with stripe_count == 1
497
498 migration abort image-spec
499 Cancel image migration. This step may be run after successful or
500 failed migration prepare or migration execute steps and returns
501 the image to its initial (before migration) state. All modifica‐
502 tions to the destination image are lost.
503
504 migration commit image-spec
505 Commit image migration. This step is run after a successful mi‐
506 gration prepare and migration execute steps and removes the
507 source image data.
508
509 migration execute image-spec
510 Execute image migration. This step is run after a successful mi‐
511 gration prepare step and copies image data to the destination.
512
513 migration prepare [--order order] [--object-size object-size] [--im‐
514 age-feature image-feature] [--image-shared] [--stripe-unit stripe-unit]
515 [--stripe-count stripe-count] [--data-pool data-pool] [--import-only]
516 [--source-spec json] [--source-spec-path path] src-image-spec [dest-im‐
517 age-spec]
518 Prepare image migration. This is the first step when migrating
519 an image, i.e. changing the image location, format or other pa‐
520 rameters that can't be changed dynamically. The destination can
521 match the source, and in this case dest-image-spec can be omit‐
522 ted. After this step the source image is set as a parent of the
523 destination image, and the image is accessible in copy-on-write
524 mode by its destination spec.
525
526 An image can also be migrated from a read-only import source by
527 adding the --import-only optional and providing a JSON-encoded
528 --source-spec or a path to a JSON-encoded source-spec file using
529 the --source-spec-path optionals.
530
531 mirror image demote image-spec
532 Demote a primary image to non-primary for RBD mirroring.
533
534 mirror image disable [--force] image-spec
535 Disable RBD mirroring for an image. If the mirroring is config‐
536 ured in image mode for the image's pool, then it can be explic‐
537 itly disabled mirroring for each image within the pool.
538
539 mirror image enable image-spec mode
540 Enable RBD mirroring for an image. If the mirroring is config‐
541 ured in image mode for the image's pool, then it can be explic‐
542 itly enabled mirroring for each image within the pool.
543
544 The mirror image mode can either be journal (default) or snap‐
545 shot. The journal mode requires the RBD journaling feature.
546
547 mirror image promote [--force] image-spec
548 Promote a non-primary image to primary for RBD mirroring.
549
550 mirror image resync image-spec
551 Force resync to primary image for RBD mirroring.
552
553 mirror image status image-spec
554 Show RBD mirroring status for an image.
555
556 mirror pool demote [pool-name]
557 Demote all primary images within a pool to non-primary. Every
558 mirroring enabled image will demoted in the pool.
559
560 mirror pool disable [pool-name]
561 Disable RBD mirroring by default within a pool. When mirroring
562 is disabled on a pool in this way, mirroring will also be dis‐
563 abled on any images (within the pool) for which mirroring was
564 enabled explicitly.
565
566 mirror pool enable [pool-name] mode
567 Enable RBD mirroring by default within a pool. The mirroring
568 mode can either be pool or image. If configured in pool mode,
569 all images in the pool with the journaling feature enabled are
570 mirrored. If configured in image mode, mirroring needs to be
571 explicitly enabled (by mirror image enable command) on each im‐
572 age.
573
574 mirror pool info [pool-name]
575 Show information about the pool mirroring configuration. It in‐
576 cludes mirroring mode, peer UUID, remote cluster name, and re‐
577 mote client name.
578
579 mirror pool peer add [pool-name] remote-cluster-spec
580 Add a mirroring peer to a pool. remote-cluster-spec is [remote
581 client name@]remote cluster name.
582
583 The default for remote client name is "client.admin".
584
585 This requires mirroring mode is enabled.
586
587 mirror pool peer remove [pool-name] uuid
588 Remove a mirroring peer from a pool. The peer uuid is available
589 from mirror pool info command.
590
591 mirror pool peer set [pool-name] uuid key value
592 Update mirroring peer settings. The key can be either client or
593 cluster, and the value is corresponding to remote client name or
594 remote cluster name.
595
596 mirror pool promote [--force] [pool-name]
597 Promote all non-primary images within a pool to primary. Every
598 mirroring enabled image will promoted in the pool.
599
600 mirror pool status [--verbose] [pool-name]
601 Show status for all mirrored images in the pool. With --ver‐
602 bose, also show additionally output status details for every
603 mirroring image in the pool.
604
605 mirror snapshot schedule add [-p | --pool pool] [--namespace namespace]
606 [--image image] interval [start-time]
607 Add mirror snapshot schedule.
608
609 mirror snapshot schedule list [-R | --recursive] [--format format]
610 [--pretty-format] [-p | --pool pool] [--namespace namespace] [--image
611 image]
612 List mirror snapshot schedule.
613
614 mirror snapshot schedule remove [-p | --pool pool] [--namespace name‐
615 space] [--image image] interval [start-time]
616 Remove mirror snapshot schedule.
617
618 mirror snapshot schedule status [-p | --pool pool] [--format format]
619 [--pretty-format] [--namespace namespace] [--image image]
620 Show mirror snapshot schedule status.
621
622 mv src-image-spec dest-image-spec
623 Rename an image. Note: rename across pools is not supported.
624
625 namespace create pool-name/namespace-name
626 Create a new image namespace within the pool.
627
628 namespace list pool-name
629 List image namespaces defined within the pool.
630
631 namespace remove pool-name/namespace-name
632 Remove an empty image namespace from the pool.
633
634 object-map check image-spec | snap-spec
635 Verify the object map is correct.
636
637 object-map rebuild image-spec | snap-spec
638 Rebuild an invalid object map for the specified image. An image
639 snapshot can be specified to rebuild an invalid object map for a
640 snapshot.
641
642 pool init [pool-name] [--force]
643 Initialize pool for use by RBD. Newly created pools must ini‐
644 tialized prior to use.
645
646 resize (-s | --size size-in-M/G/T) [--allow-shrink] image-spec
647 Resize rbd image. The size parameter also needs to be specified.
648 The --allow-shrink option lets the size be reduced.
649
650 rm image-spec
651 Delete an rbd image (including all data blocks). If the image
652 has snapshots, this fails and nothing is deleted.
653
654 snap create snap-spec
655 Create a new snapshot. Requires the snapshot name parameter
656 specified.
657
658 snap limit clear image-spec
659 Remove any previously set limit on the number of snapshots al‐
660 lowed on an image.
661
662 snap limit set [--limit] limit image-spec
663 Set a limit for the number of snapshots allowed on an image.
664
665 snap ls image-spec
666 Dump the list of snapshots inside a specific image.
667
668 snap protect snap-spec
669 Protect a snapshot from deletion, so that clones can be made of
670 it (see rbd clone). Snapshots must be protected before clones
671 are made; protection implies that there exist dependent cloned
672 children that refer to this snapshot. rbd clone will fail on a
673 nonprotected snapshot.
674
675 This requires image format 2.
676
677 snap purge image-spec
678 Remove all unprotected snapshots from an image.
679
680 snap rename src-snap-spec dest-snap-spec
681 Rename a snapshot. Note: rename across pools and images is not
682 supported.
683
684 snap rm [--force] snap-spec
685 Remove the specified snapshot.
686
687 snap rollback snap-spec
688 Rollback image content to snapshot. This will iterate through
689 the entire blocks array and update the data head content to the
690 snapshotted version.
691
692 snap unprotect snap-spec
693 Unprotect a snapshot from deletion (undo snap protect). If
694 cloned children remain, snap unprotect fails. (Note that clones
695 may exist in different pools than the parent snapshot.)
696
697 This requires image format 2.
698
699 sparsify [--sparse-size sparse-size] image-spec
700 Reclaim space for zeroed image extents. The default sparse size
701 is 4096 bytes and can be changed via --sparse-size option with
702 the following restrictions: it should be power of two, not less
703 than 4096, and not larger than image object size.
704
705 status image-spec
706 Show the status of the image, including which clients have it
707 open.
708
709 trash ls [pool-name]
710 List all entries from trash.
711
712 trash mv image-spec
713 Move an image to the trash. Images, even ones actively in-use by
714 clones, can be moved to the trash and deleted at a later time.
715
716 trash purge [pool-name]
717 Remove all expired images from trash.
718
719 trash restore image-id
720 Restore an image from trash.
721
722 trash rm image-id
723 Delete an image from trash. If image deferment time has not ex‐
724 pired you can not removed it unless use force. But an actively
725 in-use by clones or has snapshots can not be removed.
726
727 trash purge schedule add [-p | --pool pool] [--namespace namespace] in‐
728 terval [start-time]
729 Add trash purge schedule.
730
731 trash purge schedule list [-R | --recursive] [--format format]
732 [--pretty-format] [-p | --pool pool] [--namespace namespace]
733 List trash purge schedule.
734
735 trash purge schedule remove [-p | --pool pool] [--namespace namespace]
736 interval [start-time]
737 Remove trash purge schedule.
738
739 trash purge schedule status [-p | --pool pool] [--format format]
740 [--pretty-format] [--namespace namespace]
741 Show trash purge schedule status.
742
743 watch image-spec
744 Watch events on image.
745
747 image-spec is [pool-name/[namespace-name/]]image-name
748 snap-spec is [pool-name/[namespace-name/]]image-name@snap-name
749 group-spec is [pool-name/[namespace-name/]]group-name
750 group-snap-spec is [pool-name/[namespace-name/]]group-name@snap-name
751 journal-spec is [pool-name/[namespace-name/]]journal-name
752
753
754 The default for pool-name is "rbd" and namespace-name is "". If an im‐
755 age name contains a slash character ('/'), pool-name is required.
756
757 The journal-name is image-id.
758
759 You may specify each name individually, using --pool, --namespace,
760 --image, and --snap options, but this is discouraged in favor of the
761 above spec syntax.
762
764 RBD images are striped over many objects, which are then stored by the
765 Ceph distributed object store (RADOS). As a result, read and write re‐
766 quests for the image are distributed across many nodes in the cluster,
767 generally preventing any single node from becoming a bottleneck when
768 individual images get large or busy.
769
770 The striping is controlled by three parameters:
771
772 object-size
773 The size of objects we stripe over is a power of two. It will be
774 rounded up the nearest power of two. The default object size is
775 4 MB, smallest is 4K and maximum is 32M.
776
777 stripe_unit
778 Each [stripe_unit] contiguous bytes are stored adjacently in the
779 same object, before we move on to the next object.
780
781 stripe_count
782 After we write [stripe_unit] bytes to [stripe_count] objects, we
783 loop back to the initial object and write another stripe, until
784 the object reaches its maximum size. At that point, we move on
785 to the next [stripe_count] objects.
786
787 By default, [stripe_unit] is the same as the object size and
788 [stripe_count] is 1. Specifying a different [stripe_unit] and/or
789 [stripe_count] is often referred to as using "fancy" striping and re‐
790 quires format 2.
791
793 Most of these options are useful mainly for debugging and benchmarking.
794 The default values are set in the kernel and may therefore depend on
795 the version of the running kernel.
796
797 Per client instance rbd device map options:
798
799 • fsid=aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee - FSID that should be as‐
800 sumed by the client.
801
802 • ip=a.b.c.d[:p] - IP and, optionally, port the client should use.
803
804 • share - Enable sharing of client instances with other mappings (de‐
805 fault).
806
807 • noshare - Disable sharing of client instances with other mappings.
808
809 • crc - Enable CRC32C checksumming for msgr1 on-the-wire protocol (de‐
810 fault). For msgr2.1 protocol this option is ignored: full checksum‐
811 ming is always on in 'crc' mode and always off in 'secure' mode.
812
813 • nocrc - Disable CRC32C checksumming for msgr1 on-the-wire protocol.
814 Note that only payload checksumming is disabled, header checksumming
815 is always on. For msgr2.1 protocol this option is ignored.
816
817 • cephx_require_signatures - Require msgr1 message signing feature
818 (since 3.19, default). This option is deprecated and will be removed
819 in the future as the feature has been supported since the Bobtail re‐
820 lease.
821
822 • nocephx_require_signatures - Don't require msgr1 message signing fea‐
823 ture (since 3.19). This option is deprecated and will be removed in
824 the future.
825
826 • tcp_nodelay - Disable Nagle's algorithm on client sockets (since 4.0,
827 default).
828
829 • notcp_nodelay - Enable Nagle's algorithm on client sockets (since
830 4.0).
831
832 • cephx_sign_messages - Enable message signing for msgr1 on-the-wire
833 protocol (since 4.4, default). For msgr2.1 protocol this option is
834 ignored: message signing is built into 'secure' mode and not offered
835 in 'crc' mode.
836
837 • nocephx_sign_messages - Disable message signing for msgr1 on-the-wire
838 protocol (since 4.4). For msgr2.1 protocol this option is ignored.
839
840 • mount_timeout=x - A timeout on various steps in rbd device map and
841 rbd device unmap sequences (default is 60 seconds). In particular,
842 since 4.2 this can be used to ensure that rbd device unmap eventually
843 times out when there is no network connection to a cluster.
844
845 • osdkeepalive=x - OSD keepalive timeout (default is 5 seconds).
846
847 • osd_idle_ttl=x - OSD idle TTL (default is 60 seconds).
848
849 Per mapping (block device) rbd device map options:
850
851 • rw - Map the image read-write (default). Overridden by --read-only.
852
853 • ro - Map the image read-only. Equivalent to --read-only.
854
855 • queue_depth=x - queue depth (since 4.2, default is 128 requests).
856
857 • lock_on_read - Acquire exclusive lock on reads, in addition to writes
858 and discards (since 4.9).
859
860 • exclusive - Disable automatic exclusive lock transitions (since
861 4.12). Equivalent to --exclusive.
862
863 • lock_timeout=x - A timeout on waiting for the acquisition of exclu‐
864 sive lock (since 4.17, default is 0 seconds, meaning no timeout).
865
866 • notrim - Turn off discard and write zeroes offload support to avoid
867 deprovisioning a fully provisioned image (since 4.17). When enabled,
868 discard requests will fail with -EOPNOTSUPP, write zeroes requests
869 will fall back to manually zeroing.
870
871 • abort_on_full - Fail write requests with -ENOSPC when the cluster is
872 full or the data pool reaches its quota (since 5.0). The default be‐
873 haviour is to block until the full condition is cleared.
874
875 • alloc_size - Minimum allocation unit of the underlying OSD object
876 store backend (since 5.1, default is 64K bytes). This is used to
877 round off and drop discards that are too small. For bluestore, the
878 recommended setting is bluestore_min_alloc_size (currently set to 4K
879 for all types of drives, previously used to be set to 64K for hard
880 disk drives and 16K for solid-state drives). For filestore with
881 filestore_punch_hole = false, the recommended setting is image object
882 size (typically 4M).
883
884 • crush_location=x - Specify the location of the client in terms of
885 CRUSH hierarchy (since 5.8). This is a set of key-value pairs sepa‐
886 rated from each other by '|', with keys separated from values by ':'.
887 Note that '|' may need to be quoted or escaped to avoid it being in‐
888 terpreted as a pipe by the shell. The key is the bucket type name
889 (e.g. rack, datacenter or region with default bucket types) and the
890 value is the bucket name. For example, to indicate that the client
891 is local to rack "myrack", data center "mydc" and region "myregion":
892
893 crush_location=rack:myrack|datacenter:mydc|region:myregion
894
895 Each key-value pair stands on its own: "myrack" doesn't need to re‐
896 side in "mydc", which in turn doesn't need to reside in "myregion".
897 The location is not a path to the root of the hierarchy but rather a
898 set of nodes that are matched independently, owning to the fact that
899 bucket names are unique within a CRUSH map. "Multipath" locations
900 are supported, so it is possible to indicate locality for multiple
901 parallel hierarchies:
902
903 crush_location=rack:myrack1|rack:myrack2|datacenter:mydc
904
905 • read_from_replica=no - Disable replica reads, always pick the primary
906 OSD (since 5.8, default).
907
908 • read_from_replica=balance - When issued a read on a replicated pool,
909 pick a random OSD for serving it (since 5.8).
910
911 This mode is safe for general use only since Octopus (i.e. after
912 "ceph osd require-osd-release octopus"). Otherwise it should be lim‐
913 ited to read-only workloads such as images mapped read-only every‐
914 where or snapshots.
915
916 • read_from_replica=localize - When issued a read on a replicated pool,
917 pick the most local OSD for serving it (since 5.8). The locality
918 metric is calculated against the location of the client given with
919 crush_location; a match with the lowest-valued bucket type wins. For
920 example, with default bucket types, an OSD in a matching rack is
921 closer than an OSD in a matching data center, which in turn is closer
922 than an OSD in a matching region.
923
924 This mode is safe for general use only since Octopus (i.e. after
925 "ceph osd require-osd-release octopus"). Otherwise it should be lim‐
926 ited to read-only workloads such as images mapped read-only every‐
927 where or snapshots.
928
929 • compression_hint=none - Don't set compression hints (since 5.8, de‐
930 fault).
931
932 • compression_hint=compressible - Hint to the underlying OSD object
933 store backend that the data is compressible, enabling compression in
934 passive mode (since 5.8).
935
936 • compression_hint=incompressible - Hint to the underlying OSD object
937 store backend that the data is incompressible, disabling compression
938 in aggressive mode (since 5.8).
939
940 • ms_mode=legacy - Use msgr1 on-the-wire protocol (since 5.11, de‐
941 fault).
942
943 • ms_mode=crc - Use msgr2.1 on-the-wire protocol, select 'crc' mode,
944 also referred to as plain mode (since 5.11). If the daemon denies
945 'crc' mode, fail the connection.
946
947 • ms_mode=secure - Use msgr2.1 on-the-wire protocol, select 'secure'
948 mode (since 5.11). 'secure' mode provides full in-transit encryption
949 ensuring both confidentiality and authenticity. If the daemon denies
950 'secure' mode, fail the connection.
951
952 • ms_mode=prefer-crc - Use msgr2.1 on-the-wire protocol, select 'crc'
953 mode (since 5.11). If the daemon denies 'crc' mode in favor of 'se‐
954 cure' mode, agree to 'secure' mode.
955
956 • ms_mode=prefer-secure - Use msgr2.1 on-the-wire protocol, select 'se‐
957 cure' mode (since 5.11). If the daemon denies 'secure' mode in favor
958 of 'crc' mode, agree to 'crc' mode.
959
960 • rxbounce - Use a bounce buffer when receiving data (since 5.17). The
961 default behaviour is to read directly into the destination buffer. A
962 bounce buffer is needed if the destination buffer isn't guaranteed to
963 be stable (i.e. remain unchanged while it is being read to). In par‐
964 ticular this is the case for Windows where a system-wide "dummy"
965 (throwaway) page may be mapped into the destination buffer in order
966 to generate a single large I/O. Otherwise, "libceph: ... bad
967 crc/signature" or "libceph: ... integrity error, bad crc" errors and
968 associated performance degradation are expected.
969
970 • udev - Wait for udev device manager to finish executing all matching
971 "add" rules and release the device before exiting (default). This
972 option is not passed to the kernel.
973
974 • noudev - Don't wait for udev device manager. When enabled, the de‐
975 vice may not be fully usable immediately on exit.
976
977 rbd device unmap options:
978
979 • force - Force the unmapping of a block device that is open (since
980 4.9). The driver will wait for running requests to complete and then
981 unmap; requests sent to the driver after initiating the unmap will be
982 failed.
983
984 • udev - Wait for udev device manager to finish executing all matching
985 "remove" rules and clean up after the device before exiting (de‐
986 fault). This option is not passed to the kernel.
987
988 • noudev - Don't wait for udev device manager.
989
991 To create a new rbd image that is 100 GB:
992
993 rbd create mypool/myimage --size 102400
994
995 To use a non-default object size (8 MB):
996
997 rbd create mypool/myimage --size 102400 --object-size 8M
998
999 To delete an rbd image (be careful!):
1000
1001 rbd rm mypool/myimage
1002
1003 To create a new snapshot:
1004
1005 rbd snap create mypool/myimage@mysnap
1006
1007 To create a copy-on-write clone of a protected snapshot:
1008
1009 rbd clone mypool/myimage@mysnap otherpool/cloneimage
1010
1011 To see which clones of a snapshot exist:
1012
1013 rbd children mypool/myimage@mysnap
1014
1015 To delete a snapshot:
1016
1017 rbd snap rm mypool/myimage@mysnap
1018
1019 To map an image via the kernel with cephx enabled:
1020
1021 rbd device map mypool/myimage --id admin --keyfile secretfile
1022
1023 To map an image via the kernel with different cluster name other than
1024 default ceph:
1025
1026 rbd device map mypool/myimage --cluster cluster-name
1027
1028 To unmap an image:
1029
1030 rbd device unmap /dev/rbd0
1031
1032 To create an image and a clone from it:
1033
1034 rbd import --image-format 2 image mypool/parent
1035 rbd snap create mypool/parent@snap
1036 rbd snap protect mypool/parent@snap
1037 rbd clone mypool/parent@snap otherpool/child
1038
1039 To create an image with a smaller stripe_unit (to better distribute
1040 small writes in some workloads):
1041
1042 rbd create mypool/myimage --size 102400 --stripe-unit 65536B --stripe-count 16
1043
1044 To change an image from one image format to another, export it and then
1045 import it as the desired image format:
1046
1047 rbd export mypool/myimage@snap /tmp/img
1048 rbd import --image-format 2 /tmp/img mypool/myimage2
1049
1050 To lock an image for exclusive use:
1051
1052 rbd lock add mypool/myimage mylockid
1053
1054 To release a lock:
1055
1056 rbd lock remove mypool/myimage mylockid client.2485
1057
1058 To list images from trash:
1059
1060 rbd trash ls mypool
1061
1062 To defer delete an image (use --expires-at to set expiration time, de‐
1063 fault is now):
1064
1065 rbd trash mv mypool/myimage --expires-at "tomorrow"
1066
1067 To delete an image from trash (be careful!):
1068
1069 rbd trash rm mypool/myimage-id
1070
1071 To force delete an image from trash (be careful!):
1072
1073 rbd trash rm mypool/myimage-id --force
1074
1075 To restore an image from trash:
1076
1077 rbd trash restore mypool/myimage-id
1078
1079 To restore an image from trash and rename it:
1080
1081 rbd trash restore mypool/myimage-id --image mynewimage
1082
1084 rbd is part of Ceph, a massively scalable, open-source, distributed
1085 storage system. Please refer to the Ceph documentation at
1086 https://docs.ceph.com for more information.
1087
1089 ceph(8), rados(8)
1090
1092 2010-2022, Inktank Storage, Inc. and contributors. Licensed under Cre‐
1093 ative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0)
1094
1095
1096
1097
1098dev Oct 18, 2022 RBD(8)