1RBD(8)                               Ceph                               RBD(8)
2
3
4

NAME

6       rbd - manage rados block device (RBD) images
7

SYNOPSIS

9       rbd [ -c ceph.conf ] [ -m monaddr ] [--cluster cluster-name]
10       [ -p | --pool pool ] [ command ... ]
11
12

DESCRIPTION

14       rbd is a utility for manipulating rados block device (RBD) images, used
15       by the Linux rbd driver and the rbd storage driver for  QEMU/KVM.   RBD
16       images  are  simple  block  devices  that  are striped over objects and
17       stored in a RADOS object store. The size of the objects  the  image  is
18       striped over must be a power of two.
19

OPTIONS

21       -c ceph.conf, --conf ceph.conf
22              Use   ceph.conf   configuration  file  instead  of  the  default
23              /etc/ceph/ceph.conf  to  determine  monitor   addresses   during
24              startup.
25
26       -m monaddress[:port]
27              Connect   to  specified  monitor  (instead  of  looking  through
28              ceph.conf).
29
30       --cluster cluster-name
31              Use different cluster name as compared to default  cluster  name
32              ceph.
33
34       -p pool-name, --pool pool-name
35              Interact with the given pool. Required by most commands.
36
37       --namespace namespace-name
38              Use a pre-defined image namespace within a pool
39
40       --no-progress
41              Do  not  output  progress information (goes to standard error by
42              default for some commands).
43

PARAMETERS

45       --image-format format-id
46              Specifies which object layout to use. The default is 2.
47
48              • format 1 - (deprecated) Use the original format for a new  rbd
49                image. This format is understood by all versions of librbd and
50                the kernel rbd module, but does  not  support  newer  features
51                like cloning.
52
53              • format  2  -  Use the second rbd format, which is supported by
54                librbd since the Bobtail release and  the  kernel  rbd  module
55                since  kernel 3.10 (except for "fancy" striping, which is sup‐
56                ported since kernel 4.17). This adds support for  cloning  and
57                is  more  easily  extensible to allow more features in the fu‐
58                ture.
59
60       -s size-in-M/G/T, --size size-in-M/G/T
61              Specifies the size of the new rbd image or the new size  of  the
62              existing  rbd  image in M/G/T.  If no suffix is given, unit M is
63              assumed.
64
65       --object-size size-in-B/K/M
66              Specifies the object size in B/K/M.  Object size will be rounded
67              up  the  nearest  power of two; if no suffix is given, unit B is
68              assumed.  The default object size is 4M, smallest is 4K and max‐
69              imum is 32M.
70
71       --stripe-unit size-in-B/K/M
72              Specifies the stripe unit size in B/K/M.  If no suffix is given,
73              unit B is assumed.  See striping section (below)  for  more  de‐
74              tails.
75
76       --stripe-count num
77              Specifies  the  number  of objects to stripe over before looping
78              back to the first object.  See striping section (below) for more
79              details.
80
81       --snap snap
82              Specifies the snapshot name for the specific operation.
83
84       --id username
85              Specifies  the username (without the client. prefix) to use with
86              the map command.
87
88       --keyring filename
89              Specifies a keyring file containing a secret for  the  specified
90              user to use with the map command.  If not specified, the default
91              keyring locations will be searched.
92
93       --keyfile filename
94              Specifies a file containing the secret key of --id user  to  use
95              with the map command.  This option is overridden by --keyring if
96              the latter is also specified.
97
98       --shared lock-tag
99              Option for lock add that allows multiple  clients  to  lock  the
100              same  image  if  they  use the same tag. The tag is an arbitrary
101              string. This is useful for situations where  an  image  must  be
102              open  from more than one client at once, like during live migra‐
103              tion of a virtual machine, or for  use  underneath  a  clustered
104              file system.
105
106       --format format
107              Specifies output formatting (default: plain, json, xml)
108
109       --pretty-format
110              Make json or xml formatted output more human-readable.
111
112       -o krbd-options, --options krbd-options
113              Specifies  which options to use when mapping or unmapping an im‐
114              age via the rbd kernel driver.  krbd-options  is  a  comma-sepa‐
115              rated  list of options (similar to mount(8) mount options).  See
116              kernel rbd (krbd) options section below for more details.
117
118       --read-only
119              Map the image read-only.  Equivalent to -o ro.
120
121       --image-feature feature-name
122              Specifies which RBD format 2 feature should be enabled when cre‐
123              ating  an  image.  Multiple features can be enabled by repeating
124              this option multiple times.  The  following  features  are  sup‐
125              ported:
126
127              • layering: layering support
128
129              • striping: striping v2 support
130
131              • exclusive-lock: exclusive locking support
132
133              • object-map: object map support (requires exclusive-lock)
134
135              • fast-diff: fast diff calculations (requires object-map)
136
137              • deep-flatten: snapshot flatten support
138
139              • journaling: journaled IO support (requires exclusive-lock)
140
141              • data-pool: erasure coded pool support
142
143       --image-shared
144              Specifies  that  the image will be used concurrently by multiple
145              clients.  This will disable features that are dependent upon ex‐
146              clusive ownership of the image.
147
148       --whole-object
149              Specifies  that  the  diff should be limited to the extents of a
150              full object instead of showing intra-object deltas. When the ob‐
151              ject  map  feature  is enabled on an image, limiting the diff to
152              the object extents will dramatically improve  performance  since
153              the  differences  can be computed by examining the in-memory ob‐
154              ject map instead of querying RADOS for each  object  within  the
155              image.
156
157       --limit
158              Specifies the limit for the number of snapshots permitted.
159

COMMANDS

161       bench   --io-type   <read   |   write  |  readwrite  |  rw>  [--io-size
162       size-in-B/K/M/G/T]   [--io-threads    num-ios-in-flight]    [--io-total
163       size-in-B/K/M/G/T]  [--io-pattern  seq | rand] [--rw-mix-read read pro‐
164       portion in readwrite] image-spec
165              Generate a series of  IOs  to  the  image  and  measure  the  IO
166              throughput  and  latency.   If no suffix is given, unit B is as‐
167              sumed  for  both  --io-size  and  --io-total.    Defaults   are:
168              --io-size  4096,  --io-threads  16,  --io-total 1G, --io-pattern
169              seq, --rw-mix-read 50.
170
171       children snap-spec
172              List the clones of the image at the given snapshot. This  checks
173              every pool, and outputs the resulting poolname/imagename.
174
175              This requires image format 2.
176
177       clone   [--object-size   size-in-B/K/M]   [--stripe-unit  size-in-B/K/M
178       --stripe-count  num]  [--image-feature  feature-name]  [--image-shared]
179       parent-snap-spec child-image-spec
180              Will  create  a  clone (copy-on-write child) of the parent snap‐
181              shot.  Object size will be identical to that of the parent image
182              unless  specified. Size will be the same as the parent snapshot.
183              The --stripe-unit and --stripe-count arguments are optional, but
184              must be used together.
185
186              The  parent  snapshot  must be protected (see rbd snap protect).
187              This requires image format 2.
188
189       config global get config-entity key
190              Get a global-level configuration override.
191
192       config global list [--format plain | json | xml] [--pretty-format] con‐
193       fig-entity
194              List global-level configuration overrides.
195
196       config global set config-entity key value
197              Set a global-level configuration override.
198
199       config global remove config-entity key
200              Remove a global-level configuration override.
201
202       config image get image-spec key
203              Get an image-level configuration override.
204
205       config  image  list [--format plain | json | xml] [--pretty-format] im‐
206       age-spec
207              List image-level configuration overrides.
208
209       config image set image-spec key value
210              Set an image-level configuration override.
211
212       config image remove image-spec key
213              Remove an image-level configuration override.
214
215       config pool get pool-name key
216              Get a pool-level configuration override.
217
218       config pool list  [--format  plain  |  json  |  xml]  [--pretty-format]
219       pool-name
220              List pool-level configuration overrides.
221
222       config pool set pool-name key value
223              Set a pool-level configuration override.
224
225       config pool remove pool-name key
226              Remove a pool-level configuration override.
227
228       cp (src-image-spec | src-snap-spec) dest-image-spec
229              Copy  the content of a src-image into the newly created dest-im‐
230              age.  dest-image will have the same size, object size, and image
231              format as src-image.
232
233       create  (-s  |  --size size-in-M/G/T) [--image-format format-id] [--ob‐
234       ject-size size-in-B/K/M]  [--stripe-unit  size-in-B/K/M  --stripe-count
235       num]    [--thick-provision]   [--no-progress]   [--image-feature   fea‐
236       ture-name]... [--image-shared] image-spec
237              Will create a new rbd image. You must also specify the size  via
238              --size.   The --stripe-unit and --stripe-count arguments are op‐
239              tional, but must be used together.  If the --thick-provision  is
240              enabled,  it  will  fully allocate storage for the image at cre‐
241              ation time. It will take a long time to do.  Note: thick  provi‐
242              sioning requires zeroing the contents of the entire image.
243
244       deep cp (src-image-spec | src-snap-spec) dest-image-spec
245              Deep  copy  the  content  of  a src-image into the newly created
246              dest-image.  Dest-image will have the same  size,  object  size,
247              image format, and snapshots as src-image.
248
249       device  list  [-t | --device-type device-type] [--format plain | json |
250       xml] --pretty-format
251              Show the rbd images that are mapped via the  rbd  kernel  module
252              (default) or other supported device.
253
254       device map [-t | --device-type device-type] [--read-only] [--exclusive]
255       [-o | --options device-options] image-spec | snap-spec
256              Map the specified image to a block device  via  the  rbd  kernel
257              module  (default)  or  other  supported  device (nbd on Linux or
258              ggate on FreeBSD).
259
260              The --options argument is a comma separated list of device  type
261              specific options (opt1,opt2=val,...).
262
263       device  unmap  [-t  |  --device-type  device-type]  [-o | --options de‐
264       vice-options] image-spec | snap-spec | device-path
265              Unmap the block device that was mapped via the rbd kernel module
266              (default) or other supported device.
267
268              The  --options argument is a comma separated list of device type
269              specific options (opt1,opt2=val,...).
270
271       diff [--from-snap snap-name] [--whole-object] image-spec | snap-spec
272              Dump a list of byte extents in the image that have changed since
273              the  specified  start  snapshot, or since the image was created.
274              Each output line includes the starting offset  (in  bytes),  the
275              length  of the region (in bytes), and either 'zero' or 'data' to
276              indicate whether the region is known to be zeros or may  contain
277              other data.
278
279       du [-p | --pool pool-name] [image-spec | snap-spec] [--merge-snapshots]
280              Will  calculate the provisioned and actual disk usage of all im‐
281              ages and associated snapshots within the specified pool.  It can
282              also be used against individual images and snapshots.
283
284              If  the RBD fast-diff feature is not enabled on images, this op‐
285              eration will require querying the OSDs for every  potential  ob‐
286              ject within the image.
287
288              The --merge-snapshots will merge snapshots used space into their
289              parent images.
290
291       encryption format image-spec format passphrase-file [--cipher-alg alg]
292              Formats image to an encrypted format.  All data previously writ‐
293              ten  to the image will become unreadable.  A cloned image cannot
294              be formatted, although encrypted images  can  be  cloned.   Sup‐
295              ported  formats:  luks1,  luks2.   Supported  cipher algorithms:
296              aes-128, aes-256 (default).
297
298       export [--export-format format  (1  or  2)]  (image-spec  |  snap-spec)
299       [dest-path]
300              Export image to dest path (use - for stdout).  The --export-for‐
301              mat accepts '1' or '2' currently. Format 2 allow  us  to  export
302              not  only the content of image, but also the snapshots and other
303              properties, such as image_order, features.
304
305       export-diff  [--from-snap  snap-name]  [--whole-object]  (image-spec  |
306       snap-spec) dest-path
307              Export  an incremental diff for an image to dest path (use - for
308              stdout).  If an initial  snapshot  is  specified,  only  changes
309              since  that snapshot are included; otherwise, any regions of the
310              image that contain data are included.  The end snapshot is spec‐
311              ified  using the standard --snap option or @snap syntax (see be‐
312              low).  The image diff format includes metadata about image  size
313              changes, and the start and end snapshots.  It efficiently repre‐
314              sents discarded or 'zero' regions of the image.
315
316       feature disable image-spec feature-name...
317              Disable the specified feature on the specified  image.  Multiple
318              features can be specified.
319
320       feature enable image-spec feature-name...
321              Enable  the  specified  feature on the specified image. Multiple
322              features can be specified.
323
324       flatten image-spec
325              If image is a clone, copy all  shared  blocks  from  the  parent
326              snapshot  and make the child independent of the parent, severing
327              the link between parent snap and child.  The parent snapshot can
328              be  unprotected  and  deleted  if  it  has  no further dependent
329              clones.
330
331              This requires image format 2.
332
333       group create group-spec
334              Create a group.
335
336       group image add group-spec image-spec
337              Add an image to a group.
338
339       group image list group-spec
340              List images in a group.
341
342       group image remove group-spec image-spec
343              Remove an image from a group.
344
345       group ls [-p | --pool pool-name]
346              List rbd groups.
347
348       group rename src-group-spec dest-group-spec
349              Rename a group.  Note: rename across pools is not supported.
350
351       group rm group-spec
352              Delete a group.
353
354       group snap create group-snap-spec
355              Make a snapshot of a group.
356
357       group snap list group-spec
358              List snapshots of a group.
359
360       group snap rm group-snap-spec
361              Remove a snapshot from a group.
362
363       group snap rename group-snap-spec snap-name
364              Rename group's snapshot.
365
366       group snap rollback group-snap-spec
367              Rollback group to snapshot.
368
369       image-meta get image-spec key
370              Get metadata value with the key.
371
372       image-meta list image-spec
373              Show metadata held on the image. The first column is the key and
374              the second column is the value.
375
376       image-meta remove image-spec key
377              Remove metadata key with the value.
378
379       image-meta set image-spec key value
380              Set  metadata  key  with  the  value. They will displayed in im‐
381              age-meta list.
382
383       import [--export-format format (1  or  2)]  [--image-format  format-id]
384       [--object-size      size-in-B/K/M]     [--stripe-unit     size-in-B/K/M
385       --stripe-count num] [--image-feature feature-name]...  [--image-shared]
386       src-path [image-spec]
387              Create  a  new  image  and imports its data from path (use - for
388              stdin).  The import operation will try to create sparse rbd  im‐
389              ages  if  possible.   For  import from stdin, the sparsification
390              unit is the data block size of  the  destination  image  (object
391              size).
392
393              The --stripe-unit and --stripe-count arguments are optional, but
394              must be used together.
395
396              The --export-format accepts '1' or '2' currently. Format 2 allow
397              us  to  import not only the content of image, but also the snap‐
398              shots and other properties, such as image_order, features.
399
400       import-diff src-path image-spec
401              Import an incremental diff of an image and  applies  it  to  the
402              current  image.   If  the diff was generated relative to a start
403              snapshot, we verify that snapshot already exists before continu‐
404              ing.  If there was an end snapshot we verify it does not already
405              exist before applying the changes, and create the snapshot  when
406              we are done.
407
408       info image-spec | snap-spec
409              Will  dump  information  (such  as size and object size) about a
410              specific rbd image.  If image is a clone, information about  its
411              parent  is  also displayed.  If a snapshot is specified, whether
412              it is protected is shown as well.
413
414       journal client disconnect journal-spec
415              Flag image journal client as disconnected.
416
417       journal export [--verbose] [--no-error] src-journal-spec path-name
418              Export image journal to path (use - for stdout). It can be  make
419              a  backup of the image journal especially before attempting dan‐
420              gerous operations.
421
422              Note that this command may not always work  if  the  journal  is
423              badly corrupted.
424
425       journal import [--verbose] [--no-error] path-name dest-journal-spec
426              Import image journal from path (use - for stdin).
427
428       journal info journal-spec
429              Show information about image journal.
430
431       journal inspect [--verbose] journal-spec
432              Inspect and report image journal for structural errors.
433
434       journal reset journal-spec
435              Reset image journal.
436
437       journal status journal-spec
438              Show status of image journal.
439
440       lock add [--shared lock-tag] image-spec lock-id
441              Lock  an  image. The lock-id is an arbitrary name for the user's
442              convenience. By default, this is an exclusive lock,  meaning  it
443              will  fail  if  the image is already locked. The --shared option
444              changes this behavior. Note that locking does not affect any op‐
445              eration  other  than adding a lock. It does not protect an image
446              from being deleted.
447
448       lock ls image-spec
449              Show locks held on the image. The first column is the locker  to
450              use with the lock remove command.
451
452       lock rm image-spec lock-id locker
453              Release a lock on an image. The lock id and locker are as output
454              by lock ls.
455
456       ls [-l | --long] [pool-name]
457              Will list all rbd images listed  in  the  rbd_directory  object.
458              With  -l,  also show snapshots, and use longer-format output in‐
459              cluding size, parent (if clone), format, etc.
460
461       merge-diff first-diff-path second-diff-path merged-diff-path
462              Merge two continuous incremental diffs of an image into one sin‐
463              gle  diff.  The first diff's end snapshot must be equal with the
464              second diff's start snapshot.  The first diff  could  be  -  for
465              stdin, and merged diff could be - for stdout, which enables mul‐
466              tiple  diff  files  to  be  merged  using  something  like  'rbd
467              merge-diff first second - | rbd merge-diff - third result'. Note
468              this command currently only support the source incremental  diff
469              with stripe_count == 1
470
471       migration abort image-spec
472              Cancel image migration. This step may be run after successful or
473              failed migration prepare or migration execute steps and  returns
474              the image to its initial (before migration) state. All modifica‐
475              tions to the destination image are lost.
476
477       migration commit image-spec
478              Commit image migration. This step is run after a successful  mi‐
479              gration  prepare  and  migration  execute  steps and removes the
480              source image data.
481
482       migration execute image-spec
483              Execute image migration. This step is run after a successful mi‐
484              gration prepare step and copies image data to the destination.
485
486       migration  prepare  [--order  order] [--object-size object-size] [--im‐
487       age-feature image-feature] [--image-shared] [--stripe-unit stripe-unit]
488       [--stripe-count  stripe-count]  [--data-pool data-pool] [--import-only]
489       [--source-spec json] [--source-spec-path path] src-image-spec [dest-im‐
490       age-spec]
491              Prepare  image  migration. This is the first step when migrating
492              an image, i.e. changing the image location, format or other  pa‐
493              rameters  that can't be changed dynamically. The destination can
494              match the source, and in this case dest-image-spec can be  omit‐
495              ted.  After this step the source image is set as a parent of the
496              destination image, and the image is accessible in  copy-on-write
497              mode by its destination spec.
498
499              An  image can also be migrated from a read-only import source by
500              adding the --import-only optional and providing  a  JSON-encoded
501              --source-spec or a path to a JSON-encoded source-spec file using
502              the --source-spec-path optionals.
503
504       mirror image demote image-spec
505              Demote a primary image to non-primary for RBD mirroring.
506
507       mirror image disable [--force] image-spec
508              Disable RBD mirroring for an image. If the mirroring is  config‐
509              ured  in image mode for the image's pool, then it can be explic‐
510              itly disabled mirroring for each image within the pool.
511
512       mirror image enable image-spec mode
513              Enable RBD mirroring for an image. If the mirroring  is  config‐
514              ured  in image mode for the image's pool, then it can be explic‐
515              itly enabled mirroring for each image within the pool.
516
517              The mirror image mode can either be journal (default)  or  snap‐
518              shot. The journal mode requires the RBD journaling feature.
519
520       mirror image promote [--force] image-spec
521              Promote a non-primary image to primary for RBD mirroring.
522
523       mirror image resync image-spec
524              Force resync to primary image for RBD mirroring.
525
526       mirror image status image-spec
527              Show RBD mirroring status for an image.
528
529       mirror pool demote [pool-name]
530              Demote  all  primary images within a pool to non-primary.  Every
531              mirroring enabled image will demoted in the pool.
532
533       mirror pool disable [pool-name]
534              Disable RBD mirroring by default within a pool.  When  mirroring
535              is  disabled  on a pool in this way, mirroring will also be dis‐
536              abled on any images (within the pool) for  which  mirroring  was
537              enabled explicitly.
538
539       mirror pool enable [pool-name] mode
540              Enable  RBD  mirroring  by default within a pool.  The mirroring
541              mode can either be pool or image.  If configured in  pool  mode,
542              all  images  in the pool with the journaling feature enabled are
543              mirrored.  If configured in image mode, mirroring  needs  to  be
544              explicitly  enabled (by mirror image enable command) on each im‐
545              age.
546
547       mirror pool info [pool-name]
548              Show information about the pool mirroring configuration.  It in‐
549              cludes  mirroring  mode, peer UUID, remote cluster name, and re‐
550              mote client name.
551
552       mirror pool peer add [pool-name] remote-cluster-spec
553              Add a mirroring peer to a pool.  remote-cluster-spec is  [remote
554              client name@]remote cluster name.
555
556              The default for remote client name is "client.admin".
557
558              This requires mirroring mode is enabled.
559
560       mirror pool peer remove [pool-name] uuid
561              Remove  a mirroring peer from a pool. The peer uuid is available
562              from mirror pool info command.
563
564       mirror pool peer set [pool-name] uuid key value
565              Update mirroring peer settings.  The key can be either client or
566              cluster, and the value is corresponding to remote client name or
567              remote cluster name.
568
569       mirror pool promote [--force] [pool-name]
570              Promote all non-primary images within a pool to primary.   Every
571              mirroring enabled image will promoted in the pool.
572
573       mirror pool status [--verbose] [pool-name]
574              Show  status  for  all mirrored images in the pool.  With --ver‐
575              bose, also show additionally output  status  details  for  every
576              mirroring image in the pool.
577
578       mirror snapshot schedule add [-p | --pool pool] [--namespace namespace]
579       [--image image] interval [start-time]
580              Add mirror snapshot schedule.
581
582       mirror snapshot schedule list  [-R  |  --recursive]  [--format  format]
583       [--pretty-format]  [-p  | --pool pool] [--namespace namespace] [--image
584       image]
585              List mirror snapshot schedule.
586
587       mirror snapshot schedule remove [-p | --pool pool]  [--namespace  name‐
588       space] [--image image] interval [start-time]
589              Remove mirror snapshot schedule.
590
591       mirror  snapshot  schedule  status [-p | --pool pool] [--format format]
592       [--pretty-format] [--namespace namespace] [--image image]
593              Show mirror snapshot schedule status.
594
595       mv src-image-spec dest-image-spec
596              Rename an image.  Note: rename across pools is not supported.
597
598       namespace create pool-name/namespace-name
599              Create a new image namespace within the pool.
600
601       namespace list pool-name
602              List image namespaces defined within the pool.
603
604       namespace remove pool-name/namespace-name
605              Remove an empty image namespace from the pool.
606
607       object-map check image-spec | snap-spec
608              Verify the object map is correct.
609
610       object-map rebuild image-spec | snap-spec
611              Rebuild an invalid object map for the specified image. An  image
612              snapshot can be specified to rebuild an invalid object map for a
613              snapshot.
614
615       pool init [pool-name] [--force]
616              Initialize pool for use by RBD. Newly created  pools  must  ini‐
617              tialized prior to use.
618
619       resize (-s | --size size-in-M/G/T) [--allow-shrink] image-spec
620              Resize rbd image. The size parameter also needs to be specified.
621              The --allow-shrink option lets the size be reduced.
622
623       rm image-spec
624              Delete an rbd image (including all data blocks).  If  the  image
625              has snapshots, this fails and nothing is deleted.
626
627       snap create snap-spec
628              Create  a  new  snapshot.  Requires  the snapshot name parameter
629              specified.
630
631       snap limit clear image-spec
632              Remove any previously set limit on the number of  snapshots  al‐
633              lowed on an image.
634
635       snap limit set [--limit] limit image-spec
636              Set a limit for the number of snapshots allowed on an image.
637
638       snap ls image-spec
639              Dump the list of snapshots inside a specific image.
640
641       snap protect snap-spec
642              Protect  a snapshot from deletion, so that clones can be made of
643              it (see rbd clone).  Snapshots must be protected  before  clones
644              are  made;  protection implies that there exist dependent cloned
645              children that refer to this snapshot.  rbd clone will fail on  a
646              nonprotected snapshot.
647
648              This requires image format 2.
649
650       snap purge image-spec
651              Remove all unprotected snapshots from an image.
652
653       snap rename src-snap-spec dest-snap-spec
654              Rename  a  snapshot. Note: rename across pools and images is not
655              supported.
656
657       snap rm [--force] snap-spec
658              Remove the specified snapshot.
659
660       snap rollback snap-spec
661              Rollback image content to snapshot. This  will  iterate  through
662              the  entire blocks array and update the data head content to the
663              snapshotted version.
664
665       snap unprotect snap-spec
666              Unprotect a snapshot from  deletion  (undo  snap  protect).   If
667              cloned children remain, snap unprotect fails.  (Note that clones
668              may exist in different pools than the parent snapshot.)
669
670              This requires image format 2.
671
672       sparsify [--sparse-size sparse-size] image-spec
673              Reclaim space for zeroed image extents. The default sparse  size
674              is  4096  bytes and can be changed via --sparse-size option with
675              the following restrictions: it should be power of two, not  less
676              than 4096, and not larger than image object size.
677
678       status image-spec
679              Show  the  status  of the image, including which clients have it
680              open.
681
682       trash ls [pool-name]
683              List all entries from trash.
684
685       trash mv image-spec
686              Move an image to the trash. Images, even ones actively in-use by
687              clones, can be moved to the trash and deleted at a later time.
688
689       trash purge [pool-name]
690              Remove all expired images from trash.
691
692       trash restore image-id
693              Restore an image from trash.
694
695       trash rm image-id
696              Delete  an image from trash. If image deferment time has not ex‐
697              pired you can not removed it unless use force. But  an  actively
698              in-use by clones or has snapshots can not be removed.
699
700       trash purge schedule add [-p | --pool pool] [--namespace namespace] in‐
701       terval [start-time]
702              Add trash purge schedule.
703
704       trash  purge  schedule  list  [-R  |  --recursive]  [--format   format]
705       [--pretty-format] [-p | --pool pool] [--namespace namespace]
706              List trash purge schedule.
707
708       trash  purge schedule remove [-p | --pool pool] [--namespace namespace]
709       interval [start-time]
710              Remove trash purge schedule.
711
712       trash purge schedule  status  [-p  |  --pool  pool]  [--format  format]
713       [--pretty-format] [--namespace namespace]
714              Show trash purge schedule status.
715
716       watch image-spec
717              Watch events on image.
718

IMAGE, SNAP, GROUP AND JOURNAL SPECS

720       image-spec      is [pool-name/[namespace-name/]]image-name
721       snap-spec       is [pool-name/[namespace-name/]]image-name@snap-name
722       group-spec      is [pool-name/[namespace-name/]]group-name
723       group-snap-spec is [pool-name/[namespace-name/]]group-name@snap-name
724       journal-spec    is [pool-name/[namespace-name/]]journal-name
725
726
727       The  default for pool-name is "rbd" and namespace-name is "". If an im‐
728       age name contains a slash character ('/'), pool-name is required.
729
730       The journal-name is image-id.
731
732       You may specify each  name  individually,  using  --pool,  --namespace,
733       --image,  and  --snap  options, but this is discouraged in favor of the
734       above spec syntax.
735

STRIPING

737       RBD images are striped over many objects, which are then stored by  the
738       Ceph distributed object store (RADOS).  As a result, read and write re‐
739       quests for the image are distributed across many nodes in the  cluster,
740       generally  preventing  any  single node from becoming a bottleneck when
741       individual images get large or busy.
742
743       The striping is controlled by three parameters:
744
745       object-size
746              The size of objects we stripe over is a power of two. It will be
747              rounded up the nearest power of two.  The default object size is
748              4 MB, smallest is 4K and maximum is 32M.
749
750       stripe_unit
751              Each [stripe_unit] contiguous bytes are stored adjacently in the
752              same object, before we move on to the next object.
753
754       stripe_count
755              After we write [stripe_unit] bytes to [stripe_count] objects, we
756              loop back to the initial object and write another stripe,  until
757              the  object reaches its maximum size.  At that point, we move on
758              to the next [stripe_count] objects.
759
760       By  default,  [stripe_unit]  is  the  same  as  the  object  size   and
761       [stripe_count]  is  1.   Specifying  a  different  [stripe_unit] and/or
762       [stripe_count] is often referred to as using "fancy" striping  and  re‐
763       quires format 2.
764

KERNEL RBD (KRBD) OPTIONS

766       Most of these options are useful mainly for debugging and benchmarking.
767       The default values are set in the kernel and may  therefore  depend  on
768       the version of the running kernel.
769
770       Per client instance rbd device map options:
771
772       • fsid=aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee  -  FSID that should be as‐
773         sumed by the client.
774
775       • ip=a.b.c.d[:p] - IP and, optionally, port the client should use.
776
777       • share - Enable sharing of client instances with other  mappings  (de‐
778         fault).
779
780       • noshare - Disable sharing of client instances with other mappings.
781
782       • crc  - Enable CRC32C checksumming for msgr1 on-the-wire protocol (de‐
783         fault).  For msgr2.1 protocol this option is ignored: full  checksum‐
784         ming is always on in 'crc' mode and always off in 'secure' mode.
785
786       • nocrc  -  Disable CRC32C checksumming for msgr1 on-the-wire protocol.
787         Note that only payload checksumming is disabled, header  checksumming
788         is always on.  For msgr2.1 protocol this option is ignored.
789
790       • cephx_require_signatures  -  Require  msgr1  message  signing feature
791         (since 3.19, default).  This option is deprecated and will be removed
792         in the future as the feature has been supported since the Bobtail re‐
793         lease.
794
795       • nocephx_require_signatures - Don't require msgr1 message signing fea‐
796         ture  (since 3.19).  This option is deprecated and will be removed in
797         the future.
798
799       • tcp_nodelay - Disable Nagle's algorithm on client sockets (since 4.0,
800         default).
801
802       • notcp_nodelay  -  Enable  Nagle's  algorithm on client sockets (since
803         4.0).
804
805       • cephx_sign_messages - Enable message signing  for  msgr1  on-the-wire
806         protocol  (since  4.4, default).  For msgr2.1 protocol this option is
807         ignored: message signing is built into 'secure' mode and not  offered
808         in 'crc' mode.
809
810       • nocephx_sign_messages - Disable message signing for msgr1 on-the-wire
811         protocol (since 4.4).  For msgr2.1 protocol this option is ignored.
812
813       • mount_timeout=x - A timeout on various steps in rbd  device  map  and
814         rbd  device  unmap sequences (default is 60 seconds).  In particular,
815         since 4.2 this can be used to ensure that rbd device unmap eventually
816         times out when there is no network connection to a cluster.
817
818       • osdkeepalive=x - OSD keepalive timeout (default is 5 seconds).
819
820       • osd_idle_ttl=x - OSD idle TTL (default is 60 seconds).
821
822       Per mapping (block device) rbd device map options:
823
824       • rw - Map the image read-write (default).  Overridden by --read-only.
825
826       • ro - Map the image read-only.  Equivalent to --read-only.
827
828       • queue_depth=x - queue depth (since 4.2, default is 128 requests).
829
830       • lock_on_read - Acquire exclusive lock on reads, in addition to writes
831         and discards (since 4.9).
832
833       • exclusive -  Disable  automatic  exclusive  lock  transitions  (since
834         4.12).  Equivalent to --exclusive.
835
836       • lock_timeout=x  -  A timeout on waiting for the acquisition of exclu‐
837         sive lock (since 4.17, default is 0 seconds, meaning no timeout).
838
839       • notrim - Turn off discard and write zeroes offload support  to  avoid
840         deprovisioning  a fully provisioned image (since 4.17). When enabled,
841         discard requests will fail with -EOPNOTSUPP,  write  zeroes  requests
842         will fall back to manually zeroing.
843
844       • abort_on_full  - Fail write requests with -ENOSPC when the cluster is
845         full or the data pool reaches its quota (since 5.0).  The default be‐
846         haviour is to block until the full condition is cleared.
847
848       • alloc_size  -  Minimum  allocation  unit of the underlying OSD object
849         store backend (since 5.1, default is 64K bytes).   This  is  used  to
850         round  off  and drop discards that are too small.  For bluestore, the
851         recommended setting is bluestore_min_alloc_size  (typically  64K  for
852         hard disk drives and 16K for solid-state drives).  For filestore with
853         filestore_punch_hole = false, the recommended setting is image object
854         size (typically 4M).
855
856       • crush_location=x  -  Specify  the  location of the client in terms of
857         CRUSH hierarchy (since 5.8).  This is a set of key-value pairs  sepa‐
858         rated from each other by '|', with keys separated from values by ':'.
859         Note that '|' may need to be quoted or escaped to avoid it being  in‐
860         terpreted  as  a  pipe by the shell.  The key is the bucket type name
861         (e.g. rack, datacenter or region with default bucket types)  and  the
862         value  is  the bucket name.  For example, to indicate that the client
863         is local to rack "myrack", data center "mydc" and region "myregion":
864
865            crush_location=rack:myrack|datacenter:mydc|region:myregion
866
867         Each key-value pair stands on its own: "myrack" doesn't need  to  re‐
868         side  in  "mydc", which in turn doesn't need to reside in "myregion".
869         The location is not a path to the root of the hierarchy but rather  a
870         set  of nodes that are matched independently, owning to the fact that
871         bucket names are unique within a CRUSH  map.   "Multipath"  locations
872         are  supported,  so  it is possible to indicate locality for multiple
873         parallel hierarchies:
874
875            crush_location=rack:myrack1|rack:myrack2|datacenter:mydc
876
877       • read_from_replica=no - Disable replica reads, always pick the primary
878         OSD (since 5.8, default).
879
880       • read_from_replica=balance  - When issued a read on a replicated pool,
881         pick a random OSD for serving it (since 5.8).
882
883         This mode is safe for general use  only  since  Octopus  (i.e.  after
884         "ceph osd require-osd-release octopus").  Otherwise it should be lim‐
885         ited to read-only workloads such as images  mapped  read-only  every‐
886         where or snapshots.
887
888       • read_from_replica=localize - When issued a read on a replicated pool,
889         pick the most local OSD for serving it  (since  5.8).   The  locality
890         metric  is  calculated  against the location of the client given with
891         crush_location; a match with the lowest-valued bucket type wins.  For
892         example,  with  default  bucket  types,  an OSD in a matching rack is
893         closer than an OSD in a matching data center, which in turn is closer
894         than an OSD in a matching region.
895
896         This  mode  is  safe  for  general use only since Octopus (i.e. after
897         "ceph osd require-osd-release octopus").  Otherwise it should be lim‐
898         ited  to  read-only  workloads such as images mapped read-only every‐
899         where or snapshots.
900
901       • compression_hint=none - Don't set compression hints (since  5.8,  de‐
902         fault).
903
904       • compression_hint=compressible  -  Hint  to  the underlying OSD object
905         store backend that the data is compressible, enabling compression  in
906         passive mode (since 5.8).
907
908       • compression_hint=incompressible  -  Hint to the underlying OSD object
909         store backend that the data is incompressible, disabling  compression
910         in aggressive mode (since 5.8).
911
912       • ms_mode=legacy  -  Use  msgr1  on-the-wire  protocol (since 5.11, de‐
913         fault).
914
915       • ms_mode=crc - Use msgr2.1 on-the-wire protocol,  select  'crc'  mode,
916         also  referred  to  as plain mode (since 5.11).  If the daemon denies
917         'crc' mode, fail the connection.
918
919       • ms_mode=secure - Use msgr2.1 on-the-wire  protocol,  select  'secure'
920         mode (since 5.11).  'secure' mode provides full in-transit encryption
921         ensuring both confidentiality and authenticity.  If the daemon denies
922         'secure' mode, fail the connection.
923
924       • ms_mode=prefer-crc  -  Use msgr2.1 on-the-wire protocol, select 'crc'
925         mode (since 5.11).  If the daemon denies 'crc' mode in favor of  'se‐
926         cure' mode, agree to 'secure' mode.
927
928       • ms_mode=prefer-secure - Use msgr2.1 on-the-wire protocol, select 'se‐
929         cure' mode (since 5.11).  If the daemon denies 'secure' mode in favor
930         of 'crc' mode, agree to 'crc' mode.
931
932       • udev  - Wait for udev device manager to finish executing all matching
933         "add" rules and release the device before  exiting  (default).   This
934         option is not passed to the kernel.
935
936       • noudev  -  Don't wait for udev device manager.  When enabled, the de‐
937         vice may not be fully usable immediately on exit.
938
939       rbd device unmap options:
940
941       • force - Force the unmapping of a block device  that  is  open  (since
942         4.9).  The driver will wait for running requests to complete and then
943         unmap; requests sent to the driver after initiating the unmap will be
944         failed.
945
946       • udev  - Wait for udev device manager to finish executing all matching
947         "remove" rules and clean up after  the  device  before  exiting  (de‐
948         fault).  This option is not passed to the kernel.
949
950       • noudev - Don't wait for udev device manager.
951

EXAMPLES

953       To create a new rbd image that is 100 GB:
954
955          rbd create mypool/myimage --size 102400
956
957       To use a non-default object size (8 MB):
958
959          rbd create mypool/myimage --size 102400 --object-size 8M
960
961       To delete an rbd image (be careful!):
962
963          rbd rm mypool/myimage
964
965       To create a new snapshot:
966
967          rbd snap create mypool/myimage@mysnap
968
969       To create a copy-on-write clone of a protected snapshot:
970
971          rbd clone mypool/myimage@mysnap otherpool/cloneimage
972
973       To see which clones of a snapshot exist:
974
975          rbd children mypool/myimage@mysnap
976
977       To delete a snapshot:
978
979          rbd snap rm mypool/myimage@mysnap
980
981       To map an image via the kernel with cephx enabled:
982
983          rbd device map mypool/myimage --id admin --keyfile secretfile
984
985       To  map  an image via the kernel with different cluster name other than
986       default ceph:
987
988          rbd device map mypool/myimage --cluster cluster-name
989
990       To unmap an image:
991
992          rbd device unmap /dev/rbd0
993
994       To create an image and a clone from it:
995
996          rbd import --image-format 2 image mypool/parent
997          rbd snap create mypool/parent@snap
998          rbd snap protect mypool/parent@snap
999          rbd clone mypool/parent@snap otherpool/child
1000
1001       To create an image with a smaller  stripe_unit  (to  better  distribute
1002       small writes in some workloads):
1003
1004          rbd create mypool/myimage --size 102400 --stripe-unit 65536B --stripe-count 16
1005
1006       To change an image from one image format to another, export it and then
1007       import it as the desired image format:
1008
1009          rbd export mypool/myimage@snap /tmp/img
1010          rbd import --image-format 2 /tmp/img mypool/myimage2
1011
1012       To lock an image for exclusive use:
1013
1014          rbd lock add mypool/myimage mylockid
1015
1016       To release a lock:
1017
1018          rbd lock remove mypool/myimage mylockid client.2485
1019
1020       To list images from trash:
1021
1022          rbd trash ls mypool
1023
1024       To defer delete an image (use --expires-at to set expiration time,  de‐
1025       fault is now):
1026
1027          rbd trash mv mypool/myimage --expires-at "tomorrow"
1028
1029       To delete an image from trash (be careful!):
1030
1031          rbd trash rm mypool/myimage-id
1032
1033       To force delete an image from trash (be careful!):
1034
1035          rbd trash rm mypool/myimage-id  --force
1036
1037       To restore an image from trash:
1038
1039          rbd trash restore mypool/myimage-id
1040
1041       To restore an image from trash and rename it:
1042
1043          rbd trash restore mypool/myimage-id --image mynewimage
1044

AVAILABILITY

1046       rbd  is  part  of  Ceph, a massively scalable, open-source, distributed
1047       storage  system.  Please   refer   to   the   Ceph   documentation   at
1048       http://ceph.com/docs for more information.
1049

SEE ALSO

1051       ceph(8), rados(8)
1052
1054       2010-2021,  Inktank Storage, Inc. and contributors. Licensed under Cre‐
1055       ative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0)
1056
1057
1058
1059
1060dev                              Sep 28, 2021                           RBD(8)
Impressum