1RBD(8)                               Ceph                               RBD(8)
2
3
4

NAME

6       rbd - manage rados block device (RBD) images
7

SYNOPSIS

9       rbd [ -c ceph.conf ] [ -m monaddr ] [--cluster cluster name]
10       [ -p | --pool pool ] [ command ... ]
11
12

DESCRIPTION

14       rbd is a utility for manipulating rados block device (RBD) images, used
15       by the Linux rbd driver and the rbd storage driver for  Qemu/KVM.   RBD
16       images  are  simple  block  devices  that  are striped over objects and
17       stored in a RADOS object store. The size of the objects  the  image  is
18       striped over must be a power of two.
19

OPTIONS

21       -c ceph.conf, --conf ceph.conf
22              Use   ceph.conf   configuration  file  instead  of  the  default
23              /etc/ceph/ceph.conf  to  determine  monitor   addresses   during
24              startup.
25
26       -m monaddress[:port]
27              Connect   to  specified  monitor  (instead  of  looking  through
28              ceph.conf).
29
30       --cluster cluster name
31              Use different cluster name as compared to default  cluster  name
32              ceph.
33
34       -p pool-name, --pool pool-name
35              Interact with the given pool. Required by most commands.
36
37       --no-progress
38              Do  not  output  progress information (goes to standard error by
39              default for some commands).
40

PARAMETERS

42       --image-format format-id
43              Specifies which object layout to use. The default is 2.
44
45              · format 1 - (deprecated) Use the original format for a new  rbd
46                image. This format is understood by all versions of librbd and
47                the kernel rbd module, but does  not  support  newer  features
48                like cloning.
49
50              · format  2  -  Use the second rbd format, which is supported by
51                librbd and kernel since version 3.11  (except  for  striping).
52                This adds support for cloning and is more easily extensible to
53                allow more features in the future.
54
55       -s size-in-M/G/T, --size size-in-M/G/T
56              Specifies the size of the new rbd image or the new size  of  the
57              existing  rbd  image in M/G/T.  If no suffix is given, unit M is
58              assumed.
59
60       --object-size size-in-B/K/M
61              Specifies the object size in B/K/M.  Object size will be rounded
62              up  the  nearest  power of two; if no suffix is given, unit B is
63              assumed.  The default object size is 4M, smallest is 4K and max‐
64              imum is 32M.
65
66       --stripe-unit size-in-B/K/M
67              Specifies the stripe unit size in B/K/M.  If no suffix is given,
68              unit B is  assumed.   See  striping  section  (below)  for  more
69              details.
70
71       --stripe-count num
72              Specifies  the  number  of objects to stripe over before looping
73              back to the first object.  See striping section (below) for more
74              details.
75
76       --snap snap
77              Specifies the snapshot name for the specific operation.
78
79       --id username
80              Specifies  the username (without the client. prefix) to use with
81              the map command.
82
83       --keyring filename
84              Specifies a keyring file containing a secret for  the  specified
85              user to use with the map command.  If not specified, the default
86              keyring locations will be searched.
87
88       --keyfile filename
89              Specifies a file containing the secret key of --id user  to  use
90              with the map command.  This option is overridden by --keyring if
91              the latter is also specified.
92
93       --shared lock-tag
94              Option for lock add that allows multiple  clients  to  lock  the
95              same  image  if  they  use the same tag. The tag is an arbitrary
96              string. This is useful for situations where  an  image  must  be
97              open  from more than one client at once, like during live migra‐
98              tion of a virtual machine, or for  use  underneath  a  clustered
99              filesystem.
100
101       --format format
102              Specifies output formatting (default: plain, json, xml)
103
104       --pretty-format
105              Make json or xml formatted output more human-readable.
106
107       -o krbd-options, --options krbd-options
108              Specifies  which  options  to  use  when mapping or unmapping an
109              image via the rbd kernel driver.  krbd-options is a  comma-sepa‐
110              rated  list of options (similar to mount(8) mount options).  See
111              kernel rbd (krbd) options section below for more details.
112
113       --read-only
114              Map the image read-only.  Equivalent to -o ro.
115
116       --image-feature feature-name
117              Specifies which RBD format 2 feature should be enabled when cre‐
118              ating  an  image.  Multiple features can be enabled by repeating
119              this option multiple times.  The  following  features  are  sup‐
120              ported:
121
122              · layering: layering support
123
124              · striping: striping v2 support
125
126              · exclusive-lock: exclusive locking support
127
128              · object-map: object map support (requires exclusive-lock)
129
130              · fast-diff: fast diff calculations (requires object-map)
131
132              · deep-flatten: snapshot flatten support
133
134              · journaling: journaled IO support (requires exclusive-lock)
135
136       --image-shared
137              Specifies  that  the image will be used concurrently by multiple
138              clients.  This will disable features  that  are  dependent  upon
139              exclusive ownership of the image.
140
141       --whole-object
142              Specifies  that  the  diff should be limited to the extents of a
143              full object instead of showing  intra-object  deltas.  When  the
144              object  map feature is enabled on an image, limiting the diff to
145              the object extents will dramatically improve  performance  since
146              the  differences  can  be  computed  by  examining the in-memory
147              object map instead of querying RADOS for each object within  the
148              image.
149

COMMANDS

151       ls [-l | --long] [pool-name]
152              Will  list  all  rbd  images listed in the rbd_directory object.
153              With -l, also  show  snapshots,  and  use  longer-format  output
154              including size, parent (if clone), format, etc.
155
156       du [-p | --pool pool-name] [image-spec | snap-spec]
157              Will  calculate  the  provisioned  and  actual disk usage of all
158              images and associated snapshots within the specified  pool.   It
159              can also be used against individual images and snapshots.
160
161              If the RBD fast-diff feature isn't enabled on images, this oper‐
162              ation will require querying the OSDs for every potential  object
163              within the image.
164
165       info image-spec | snap-spec
166              Will  dump  information  (such  as size and object size) about a
167              specific rbd image.  If image is a clone, information about  its
168              parent  is  also displayed.  If a snapshot is specified, whether
169              it is protected is shown as well.
170
171       create  (-s  |   --size   size-in-M/G/T)   [--image-format   format-id]
172       [--object-size      size-in-B/K/M]     [--stripe-unit     size-in-B/K/M
173       --stripe-count num] [--image-feature feature-name]...  [--image-shared]
174       image-spec
175              Will  create a new rbd image. You must also specify the size via
176              --size.  The  --stripe-unit  and  --stripe-count  arguments  are
177              optional, but must be used together.
178
179       clone   [--object-size   size-in-B/K/M]   [--stripe-unit  size-in-B/K/M
180       --stripe-count  num]  [--image-feature  feature-name]  [--image-shared]
181       parent-snap-spec child-image-spec
182              Will  create  a  clone (copy-on-write child) of the parent snap‐
183              shot.  Object size will be identical to that of the parent image
184              unless  specified. Size will be the same as the parent snapshot.
185              The --stripe-unit and --stripe-count arguments are optional, but
186              must be used together.
187
188              The  parent  snapshot  must be protected (see rbd snap protect).
189              This requires image format 2.
190
191       flatten image-spec
192              If image is a clone, copy all  shared  blocks  from  the  parent
193              snapshot  and make the child independent of the parent, severing
194              the link between parent snap and child.  The parent snapshot can
195              be  unprotected  and  deleted  if  it  has  no further dependent
196              clones.
197
198              This requires image format 2.
199
200       children snap-spec
201              List the clones of the image at the given snapshot. This  checks
202              every pool, and outputs the resulting poolname/imagename.
203
204              This requires image format 2.
205
206       resize (-s | --size size-in-M/G/T) [--allow-shrink] image-spec
207              Resizes  rbd  image.  The size parameter also needs to be speci‐
208              fied.  The --allow-shrink option lets the size be reduced.
209
210       rm image-spec
211              Deletes an rbd image (including all data blocks). If  the  image
212              has snapshots, this fails and nothing is deleted.
213
214       export (image-spec | snap-spec) [dest-path]
215              Exports image to dest path (use - for stdout).
216
217       import   [--image-format   format-id]   [--object-size   size-in-B/K/M]
218       [--stripe-unit size-in-B/K/M --stripe-count num] [--image-feature  fea‐
219       ture-name]... [--image-shared] src-path [image-spec]
220              Creates  a  new  image and imports its data from path (use - for
221              stdin).  The import operation will  try  to  create  sparse  rbd
222              images  if  possible.  For import from stdin, the sparsification
223              unit is the data block size of  the  destination  image  (object
224              size).
225
226              The --stripe-unit and --stripe-count arguments are optional, but
227              must be used together.
228
229       export-diff  [--from-snap  snap-name]  [--whole-object]  (image-spec  |
230       snap-spec) dest-path
231              Exports an incremental diff for an image to dest path (use - for
232              stdout).  If an initial  snapshot  is  specified,  only  changes
233              since  that snapshot are included; otherwise, any regions of the
234              image that contain data are included.  The end snapshot is spec‐
235              ified  using  the  standard  --snap  option or @snap syntax (see
236              below).  The image diff format  includes  metadata  about  image
237              size  changes,  and the start and end snapshots.  It efficiently
238              represents discarded or 'zero' regions of the image.
239
240       merge-diff first-diff-path second-diff-path merged-diff-path
241              Merge two continuous incremental diffs of an image into one sin‐
242              gle  diff.  The first diff's end snapshot must be equal with the
243              second diff's start snapshot.  The first diff  could  be  -  for
244              stdin, and merged diff could be - for stdout, which enables mul‐
245              tiple  diff  files  to  be  merged  using  something  like  'rbd
246              merge-diff first second - | rbd merge-diff - third result'. Note
247              this command currently only support the source incremental  diff
248              with stripe_count == 1
249
250       import-diff src-path image-spec
251              Imports  an  incremental  diff of an image and applies it to the
252              current image.  If the diff was generated relative  to  a  start
253              snapshot, we verify that snapshot already exists before continu‐
254              ing.  If there was an end snapshot we verify it does not already
255              exist  before applying the changes, and create the snapshot when
256              we are done.
257
258       diff [--from-snap snap-name] [--whole-object] image-spec | snap-spec
259              Dump a list of byte extents in the image that have changed since
260              the  specified  start  snapshot, or since the image was created.
261              Each output line includes the starting offset  (in  bytes),  the
262              length  of the region (in bytes), and either 'zero' or 'data' to
263              indicate whether the region is known to be zeros or may  contain
264              other data.
265
266       cp (src-image-spec | src-snap-spec) dest-image-spec
267              Copies  the  content  of  a  src-image  into  the  newly created
268              dest-image.  dest-image will have the same  size,  object  size,
269              and image format as src-image.
270
271       mv src-image-spec dest-image-spec
272              Renames an image.  Note: rename across pools is not supported.
273
274       image-meta list image-spec
275              Show metadata held on the image. The first column is the key and
276              the second column is the value.
277
278       image-meta get image-spec key
279              Get metadata value with the key.
280
281       image-meta set image-spec key value
282              Set  metadata  key  with  the  value.  They  will  displayed  in
283              image-meta list.
284
285       image-meta remove image-spec key
286              Remove metadata key with the value.
287
288       object-map rebuild image-spec | snap-spec
289              Rebuilds an invalid object map for the specified image. An image
290              snapshot can be specified to rebuild an invalid object map for a
291              snapshot.
292
293       snap ls image-spec
294              Dumps the list of snapshots inside a specific image.
295
296       snap create snap-spec
297              Creates  a  new  snapshot.  Requires the snapshot name parameter
298              specified.
299
300       snap rollback snap-spec
301              Rollback image content to snapshot. This  will  iterate  through
302              the  entire blocks array and update the data head content to the
303              snapshotted version.
304
305       snap rm snap-spec
306              Removes the specified snapshot.
307
308       snap purge image-spec
309              Removes all snapshots from an image.
310
311       snap protect snap-spec
312              Protect a snapshot from deletion, so that clones can be made  of
313              it  (see  rbd clone).  Snapshots must be protected before clones
314              are made; protection implies that there exist  dependent  cloned
315              children  that refer to this snapshot.  rbd clone will fail on a
316              nonprotected snapshot.
317
318              This requires image format 2.
319
320       snap unprotect snap-spec
321              Unprotect a snapshot from  deletion  (undo  snap  protect).   If
322              cloned children remain, snap unprotect fails.  (Note that clones
323              may exist in different pools than the parent snapshot.)
324
325              This requires image format 2.
326
327       map [-o | --options krbd-options ] [--read-only] image-spec | snap-spec
328              Maps the specified image to a block device via  the  rbd  kernel
329              module.
330
331       unmap   [-o  |  --options  krbd-options  ]  image-spec  |  snap-spec  |
332       device-path
333              Unmaps the block device that was mapped via the rbd kernel  mod‐
334              ule.
335
336       showmapped
337              Show the rbd images that are mapped via the rbd kernel module.
338
339       nbd map [--device device-path] [--read-only] image-spec | snap-spec
340              Maps the specified image to a block device via the rbd-nbd tool.
341
342       nbd unmap device-path
343              Unmaps the block device that was mapped via the rbd-nbd tool.
344
345       nbd list
346              Show the list of used nbd devices via the rbd-nbd tool.
347
348       status image-spec
349              Show  the  status  of the image, including which clients have it
350              open.
351
352       feature disable image-spec feature-name...
353              Disables the specified feature on the specified image.  Multiple
354              features can be specified.
355
356       feature enable image-spec feature-name...
357              Enables  the  specified feature on the specified image. Multiple
358              features can be specified.
359
360       lock list image-spec
361              Show locks held on the image. The first column is the locker  to
362              use with the lock remove command.
363
364       lock add [--shared lock-tag] image-spec lock-id
365              Lock  an  image. The lock-id is an arbitrary name for the user's
366              convenience. By default, this is an exclusive lock,  meaning  it
367              will  fail  if  the image is already locked. The --shared option
368              changes this behavior. Note that locking  does  not  affect  any
369              operation other than adding a lock. It does not protect an image
370              from being deleted.
371
372       lock remove image-spec lock-id locker
373              Release a lock on an image. The lock id and locker are as output
374              by lock ls.
375
376       bench-write       [--io-size      size-in-B/K/M/G/T]      [--io-threads
377       num-ios-in-flight] [--io-total size-in-B/K/M/G/T] [--io-pattern  seq  |
378       rand] image-spec
379              Generate  a  series of writes to the image and measure the write
380              throughput and latency.  If  no  suffix  is  given,  unit  B  is
381              assumed  for  both  --io-size  and  --io-total.   Defaults  are:
382              --io-size 4096, --io-threads  16,  --io-total  1G,  --io-pattern
383              seq.
384

IMAGE AND SNAP SPECS

386       image-spec is [pool-name]/image-name
387       snap-spec  is [pool-name]/image-name@snap-name
388
389
390       The  default for pool-name is "rbd".  If an image name contains a slash
391       character ('/'), pool-name is required.
392
393       You may specify each  name  individually,  using  --pool,  --image  and
394       --snap options, but this is discouraged in favor of the above spec syn‐
395       tax.
396

STRIPING

398       RBD images are striped over many objects, which are then stored by  the
399       Ceph  distributed  object  store  (RADOS).  As a result, read and write
400       requests for the image are distributed across many nodes in  the  clus‐
401       ter,  generally  preventing  any single node from becoming a bottleneck
402       when individual images get large or busy.
403
404       The striping is controlled by three parameters:
405
406       -size  The size of objects we stripe over is a power of two. It will be
407              rounded up the nearest power of two.  The default object size is
408              4 MB, smallest is 4K and maximum is 32M.
409
410       stripe_unit
411              Each [stripe_unit] contiguous bytes are stored adjacently in the
412              same object, before we move on to the next object.
413
414       stripe_count
415              After we write [stripe_unit] bytes to [stripe_count] objects, we
416              loop back to the initial object and write another stripe,  until
417              the  object reaches its maximum size.  At that point, we move on
418              to the next [stripe_count] objects.
419
420       By  default,  [stripe_unit]  is  the  same  as  the  object  size   and
421       [stripe_count]  is  1.   Specifying  a different [stripe_unit] requires
422       that the STRIPINGV2 feature be supported (added in Ceph v0.53) and for‐
423       mat 2 images be used.
424

KERNEL RBD (KRBD) OPTIONS

426       Most of these options are useful mainly for debugging and benchmarking.
427       The default values are set in the kernel and may  therefore  depend  on
428       the version of the running kernel.
429
430       Per client instance rbd map options:
431
432       · fsid=aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee   -  FSID  that  should  be
433         assumed by the client.
434
435       · ip=a.b.c.d[:p] - IP and, optionally, port the client should use.
436
437       · share - Enable  sharing  of  client  instances  with  other  mappings
438         (default).
439
440       · noshare - Disable sharing of client instances with other mappings.
441
442       · crc - Enable CRC32C checksumming for data writes (default).
443
444       · nocrc - Disable CRC32C checksumming for data writes.
445
446       · cephx_require_signatures - Require cephx message signing (since 3.19,
447         default).
448
449       · nocephx_require_signatures -  Don't  require  cephx  message  signing
450         (since 3.19).
451
452       · tcp_nodelay - Disable Nagle's algorithm on client sockets (since 4.0,
453         default).
454
455       · notcp_nodelay - Enable Nagle's algorithm  on  client  sockets  (since
456         4.0).
457
458       · cephx_sign_messages - Enable message signing (since 4.4, default).
459
460       · nocephx_sign_messages - Disable message signing (since 4.4).
461
462       · mount_timeout=x - A timeout on various steps in rbd map and rbd unmap
463         sequences (default is 60 seconds).  In particular, since 4.2 this can
464         be  used  to ensure that rbd unmap eventually times out when there is
465         no network connection to a cluster.
466
467       · osdkeepalive=x - OSD keepalive timeout (default is 5 seconds).
468
469       · osd_idle_ttl=x - OSD idle TTL (default is 60 seconds).
470
471       Per mapping (block device) rbd map options:
472
473       · rw - Map the image read-write (default).
474
475       · ro - Map the image read-only.  Equivalent to --read-only.
476
477       · queue_depth=x - queue depth (since 4.2, default is 128 requests).
478
479       · lock_on_read - Acquire exclusive lock on reads, in addition to writes
480         and discards (since 4.9).
481
482       rbd unmap options:
483
484       · force  -  Force  the  unmapping of a block device that is open (since
485         4.9).  The driver will wait for running requests to complete and then
486         unmap; requests sent to the driver after initiating the unmap will be
487         failed.
488

EXAMPLES

490       To create a new rbd image that is 100 GB:
491
492          rbd create mypool/myimage --size 102400
493
494       To use a non-default object size (8 MB):
495
496          rbd create mypool/myimage --size 102400 --object-size 8M
497
498       To delete an rbd image (be careful!):
499
500          rbd rm mypool/myimage
501
502       To create a new snapshot:
503
504          rbd snap create mypool/myimage@mysnap
505
506       To create a copy-on-write clone of a protected snapshot:
507
508          rbd clone mypool/myimage@mysnap otherpool/cloneimage
509
510       To see which clones of a snapshot exist:
511
512          rbd children mypool/myimage@mysnap
513
514       To delete a snapshot:
515
516          rbd snap rm mypool/myimage@mysnap
517
518       To map an image via the kernel with cephx enabled:
519
520          rbd map mypool/myimage --id admin --keyfile secretfile
521
522       To map an image via the kernel with different cluster name  other  than
523       default ceph.
524          rbd map mypool/myimage --cluster cluster name
525
526       To unmap an image:
527
528          rbd unmap /dev/rbd0
529
530       To create an image and a clone from it:
531
532          rbd import --image-format 2 image mypool/parent
533          rbd snap create mypool/parent@snap
534          rbd snap protect mypool/parent@snap
535          rbd clone mypool/parent@snap otherpool/child
536
537       To  create  an  image  with a smaller stripe_unit (to better distribute
538       small writes in some workloads):
539
540          rbd create mypool/myimage --size 102400 --stripe-unit 65536B --stripe-count 16
541
542       To change an image from one image format to another, export it and then
543       import it as the desired image format:
544
545          rbd export mypool/myimage@snap /tmp/img
546          rbd import --image-format 2 /tmp/img mypool/myimage2
547
548       To lock an image for exclusive use:
549
550          rbd lock add mypool/myimage mylockid
551
552       To release a lock:
553
554          rbd lock remove mypool/myimage mylockid client.2485
555

AVAILABILITY

557       rbd  is  part  of  Ceph, a massively scalable, open-source, distributed
558       storage  system.  Please   refer   to   the   Ceph   documentation   at
559       http://ceph.com/docs for more information.
560

SEE ALSO

562       ceph(8), rados(8)
563
565       2010-2014,  Inktank Storage, Inc. and contributors. Licensed under Cre‐
566       ative Commons BY-SA
567
568
569
570
571dev                            October 30, 2018                         RBD(8)
Impressum