1RBD(8) Ceph RBD(8)
2
6 rbd - manage rados block device (RBD) images
7
9 rbd [ -c ceph.conf ] [ -m monaddr ] [--cluster cluster-name]
10 [ -p | --pool pool ] [ command ... ]
11
14 rbd is a utility for manipulating rados block device (RBD) images, used
15 by the Linux rbd driver and the rbd storage driver for QEMU/KVM. RBD
16 images are simple block devices that are striped over objects and
17 stored in a RADOS object store. The size of the objects the image is
18 striped over must be a power of two.
19
21 -c ceph.conf, --conf ceph.conf
22 Use ceph.conf configuration file instead of the default
23 /etc/ceph/ceph.conf to determine monitor addresses during
24 startup.
25 -m monaddress[:port]
27 Connect to specified monitor (instead of looking through
28 ceph.conf).
29 --cluster cluster-name
31 Use different cluster name as compared to default cluster name
32 ceph.
33 -p pool-name, --pool pool-name
35 Interact with the given pool. Required by most commands.
36 --no-progress
38 Do not output progress information (goes to standard error by
39 default for some commands).
40
42 --image-format format-id
43 Specifies which object layout to use. The default is 2.
44 · format 1 - (deprecated) Use the original format for a new rbd
46 image. This format is understood by all versions of librbd and
47 the kernel rbd module, but does not support newer features
48 like cloning.
49 · format 2 - Use the second rbd format, which is supported by
51 librbd and kernel since version 3.11 (except for striping).
52 This adds support for cloning and is more easily extensible to
53 allow more features in the future.
54 -s size-in-M/G/T, --size size-in-M/G/T
56 Specifies the size of the new rbd image or the new size of the
57 existing rbd image in M/G/T. If no suffix is given, unit M is
58 assumed.
59 --object-size size-in-B/K/M
61 Specifies the object size in B/K/M. Object size will be rounded
62 up the nearest power of two; if no suffix is given, unit B is
63 assumed. The default object size is 4M, smallest is 4K and max‐
64 imum is 32M.
65 --stripe-unit size-in-B/K/M
67 Specifies the stripe unit size in B/K/M. If no suffix is given,
68 unit B is assumed. See striping section (below) for more
69 details.
70 --stripe-count num
72 Specifies the number of objects to stripe over before looping
73 back to the first object. See striping section (below) for more
74 details.
75 --snap snap
77 Specifies the snapshot name for the specific operation.
78 --id username
80 Specifies the username (without the client. prefix) to use with
81 the map command.
82 --keyring filename
84 Specifies a keyring file containing a secret for the specified
85 user to use with the map command. If not specified, the default
86 keyring locations will be searched.
87 --keyfile filename
89 Specifies a file containing the secret key of --id user to use
90 with the map command. This option is overridden by --keyring if
91 the latter is also specified.
92 --shared lock-tag
94 Option for lock add that allows multiple clients to lock the
95 same image if they use the same tag. The tag is an arbitrary
96 string. This is useful for situations where an image must be
97 open from more than one client at once, like during live migra‐
98 tion of a virtual machine, or for use underneath a clustered
99 filesystem.
100 --format format
102 Specifies output formatting (default: plain, json, xml)
103 --pretty-format
105 Make json or xml formatted output more human-readable.
106 -o krbd-options, --options krbd-options
108 Specifies which options to use when mapping or unmapping an
109 image via the rbd kernel driver. krbd-options is a comma-sepa‐
110 rated list of options (similar to mount(8) mount options). See
111 kernel rbd (krbd) options section below for more details.
112 --read-only
114 Map the image read-only. Equivalent to -o ro.
115 --image-feature feature-name
117 Specifies which RBD format 2 feature should be enabled when cre‐
118 ating an image. Multiple features can be enabled by repeating
119 this option multiple times. The following features are sup‐
120 ported:
121 · layering: layering support
123 · striping: striping v2 support
125 · exclusive-lock: exclusive locking support
127 · object-map: object map support (requires exclusive-lock)
129 · fast-diff: fast diff calculations (requires object-map)
131 · deep-flatten: snapshot flatten support
133 · journaling: journaled IO support (requires exclusive-lock)
135 --image-shared
137 Specifies that the image will be used concurrently by multiple
138 clients. This will disable features that are dependent upon
139 exclusive ownership of the image.
140 --whole-object
142 Specifies that the diff should be limited to the extents of a
143 full object instead of showing intra-object deltas. When the
144 object map feature is enabled on an image, limiting the diff to
145 the object extents will dramatically improve performance since
146 the differences can be computed by examining the in-memory
147 object map instead of querying RADOS for each object within the
148 image.
149 --limit
151 Specifies the limit for the number of snapshots permitted.
152
154 ls [-l | --long] [pool-name]
155 Will list all rbd images listed in the rbd_directory object.
156 With -l, also show snapshots, and use longer-format output
157 including size, parent (if clone), format, etc.
158 du [-p | --pool pool-name] [image-spec | snap-spec]
160 Will calculate the provisioned and actual disk usage of all
161 images and associated snapshots within the specified pool. It
162 can also be used against individual images and snapshots.
163 If the RBD fast-diff feature is not enabled on images, this
165 operation will require querying the OSDs for every potential
166 object within the image.
167 info image-spec | snap-spec
169 Will dump information (such as size and object size) about a
170 specific rbd image. If image is a clone, information about its
171 parent is also displayed. If a snapshot is specified, whether
172 it is protected is shown as well.
173 create (-s | --size size-in-M/G/T) [--image-format format-id]
175 [--object-size size-in-B/K/M] [--stripe-unit size-in-B/K/M
176 --stripe-count num] [--image-feature feature-name]... [--image-shared]
177 image-spec
178 Will create a new rbd image. You must also specify the size via
179 --size. The --stripe-unit and --stripe-count arguments are
180 optional, but must be used together.
181 clone [--object-size size-in-B/K/M] [--stripe-unit size-in-B/K/M
183 --stripe-count num] [--image-feature feature-name] [--image-shared]
184 parent-snap-spec child-image-spec
185 Will create a clone (copy-on-write child) of the parent snap‐
186 shot. Object size will be identical to that of the parent image
187 unless specified. Size will be the same as the parent snapshot.
188 The --stripe-unit and --stripe-count arguments are optional, but
189 must be used together.
190 The parent snapshot must be protected (see rbd snap protect).
192 This requires image format 2.
193 flatten image-spec
195 If image is a clone, copy all shared blocks from the parent
196 snapshot and make the child independent of the parent, severing
197 the link between parent snap and child. The parent snapshot can
198 be unprotected and deleted if it has no further dependent
199 clones.
200 This requires image format 2.
202 children snap-spec
204 List the clones of the image at the given snapshot. This checks
205 every pool, and outputs the resulting poolname/imagename.
206 This requires image format 2.
208 resize (-s | --size size-in-M/G/T) [--allow-shrink] image-spec
210 Resize rbd image. The size parameter also needs to be specified.
211 The --allow-shrink option lets the size be reduced.
212 rm image-spec
214 Delete an rbd image (including all data blocks). If the image
215 has snapshots, this fails and nothing is deleted.
216 export [--export-format format (1 or 2)] (image-spec | snap-spec)
218 [dest-path]
219 Export image to dest path (use - for stdout). The --export-for‐
220 mat accepts '1' or '2' currently. Format 2 allow us to export
221 not only the content of image, but also the snapshots and other
222 properties, such as image_order, features.
223 import [--export-format format (1 or 2)] [--image-format format-id]
225 [--object-size size-in-B/K/M] [--stripe-unit size-in-B/K/M
226 --stripe-count num] [--image-feature feature-name]... [--image-shared]
227 src-path [image-spec]
228 Create a new image and imports its data from path (use - for
229 stdin). The import operation will try to create sparse rbd
230 images if possible. For import from stdin, the sparsification
231 unit is the data block size of the destination image (object
232 size).
233 The --stripe-unit and --stripe-count arguments are optional, but
235 must be used together.
236 The --export-format accepts '1' or '2' currently. Format 2 allow
238 us to import not only the content of image, but also the snap‐
239 shots and other properties, such as image_order, features.
240 export-diff [--from-snap snap-name] [--whole-object] (image-spec |
242 snap-spec) dest-path
243 Export an incremental diff for an image to dest path (use - for
244 stdout). If an initial snapshot is specified, only changes
245 since that snapshot are included; otherwise, any regions of the
246 image that contain data are included. The end snapshot is spec‐
247 ified using the standard --snap option or @snap syntax (see
248 below). The image diff format includes metadata about image
249 size changes, and the start and end snapshots. It efficiently
250 represents discarded or 'zero' regions of the image.
251 merge-diff first-diff-path second-diff-path merged-diff-path
253 Merge two continuous incremental diffs of an image into one sin‐
254 gle diff. The first diff's end snapshot must be equal with the
255 second diff's start snapshot. The first diff could be - for
256 stdin, and merged diff could be - for stdout, which enables mul‐
257 tiple diff files to be merged using something like 'rbd
258 merge-diff first second - | rbd merge-diff - third result'. Note
259 this command currently only support the source incremental diff
260 with stripe_count == 1
261 import-diff src-path image-spec
263 Import an incremental diff of an image and applies it to the
264 current image. If the diff was generated relative to a start
265 snapshot, we verify that snapshot already exists before continu‐
266 ing. If there was an end snapshot we verify it does not already
267 exist before applying the changes, and create the snapshot when
268 we are done.
269 diff [--from-snap snap-name] [--whole-object] image-spec | snap-spec
271 Dump a list of byte extents in the image that have changed since
272 the specified start snapshot, or since the image was created.
273 Each output line includes the starting offset (in bytes), the
274 length of the region (in bytes), and either 'zero' or 'data' to
275 indicate whether the region is known to be zeros or may contain
276 other data.
277 cp (src-image-spec | src-snap-spec) dest-image-spec
279 Copy the content of a src-image into the newly created
280 dest-image. dest-image will have the same size, object size,
281 and image format as src-image.
282 mv src-image-spec dest-image-spec
284 Rename an image. Note: rename across pools is not supported.
285 image-meta list image-spec
287 Show metadata held on the image. The first column is the key and
288 the second column is the value.
289 image-meta get image-spec key
291 Get metadata value with the key.
292 image-meta set image-spec key value
294 Set metadata key with the value. They will displayed in
295 image-meta list.
296 image-meta remove image-spec key
298 Remove metadata key with the value.
299 object-map rebuild image-spec | snap-spec
301 Rebuild an invalid object map for the specified image. An image
302 snapshot can be specified to rebuild an invalid object map for a
303 snapshot.
304 snap ls image-spec
306 Dump the list of snapshots inside a specific image.
307 snap create snap-spec
309 Create a new snapshot. Requires the snapshot name parameter
310 specified.
311 snap rollback snap-spec
313 Rollback image content to snapshot. This will iterate through
314 the entire blocks array and update the data head content to the
315 snapshotted version.
316 snap rm [--force] snap-spec
318 Remove the specified snapshot.
319 snap purge image-spec
321 Remove all snapshots from an image.
322 snap protect snap-spec
324 Protect a snapshot from deletion, so that clones can be made of
325 it (see rbd clone). Snapshots must be protected before clones
326 are made; protection implies that there exist dependent cloned
327 children that refer to this snapshot. rbd clone will fail on a
328 nonprotected snapshot.
329 This requires image format 2.
331 snap unprotect snap-spec
333 Unprotect a snapshot from deletion (undo snap protect). If
334 cloned children remain, snap unprotect fails. (Note that clones
335 may exist in different pools than the parent snapshot.)
336 This requires image format 2.
338 snap limit set [--limit] limit image-spec
340 Set a limit for the number of snapshots allowed on an image.
341 snap limit clear image-spec
343 Remove any previously set limit on the number of snapshots
344 allowed on an image.
345 map [-o | --options krbd-options ] [--read-only] image-spec | snap-spec
347 Map the specified image to a block device via the rbd kernel
348 module.
349 unmap [-o | --options krbd-options ] image-spec | snap-spec |
351 device-path
352 Unmap the block device that was mapped via the rbd kernel mod‐
353 ule.
354 showmapped
356 Show the rbd images that are mapped via the rbd kernel module.
357 nbd map [--device device-path] [--read-only] image-spec | snap-spec
359 Map the specified image to a block device via the rbd-nbd tool.
360 nbd unmap device-path
362 Unmap the block device that was mapped via the rbd-nbd tool.
363 nbd list
365 Show the list of used nbd devices via the rbd-nbd tool.
366 status image-spec
368 Show the status of the image, including which clients have it
369 open.
370 feature disable image-spec feature-name...
372 Disable the specified feature on the specified image. Multiple
373 features can be specified.
374 feature enable image-spec feature-name...
376 Enable the specified feature on the specified image. Multiple
377 features can be specified.
378 lock list image-spec
380 Show locks held on the image. The first column is the locker to
381 use with the lock remove command.
382 lock add [--shared lock-tag] image-spec lock-id
384 Lock an image. The lock-id is an arbitrary name for the user's
385 convenience. By default, this is an exclusive lock, meaning it
386 will fail if the image is already locked. The --shared option
387 changes this behavior. Note that locking does not affect any
388 operation other than adding a lock. It does not protect an image
389 from being deleted.
390 lock remove image-spec lock-id locker
392 Release a lock on an image. The lock id and locker are as output
393 by lock ls.
394 bench --io-type <read | write> [--io-size size-in-B/K/M/G/T]
396 [--io-threads num-ios-in-flight] [--io-total size-in-B/K/M/G/T]
397 [--io-pattern seq | rand] image-spec
398 Generate a series of IOs to the image and measure the IO
399 throughput and latency. If no suffix is given, unit B is
400 assumed for both --io-size and --io-total. Defaults are:
401 --io-size 4096, --io-threads 16, --io-total 1G, --io-pattern
402 seq.
403 trash ls [pool-name]
405 List all entries from trash.
406 trash mv image-spec
408 Move an image to the trash. Images, even ones actively in-use by
409 clones, can be moved to the trash and deleted at a later time.
410 trash rm image-id
412 Delete an image from trash. If image deferment time has not
413 expired you can not removed it unless use force. But an actively
414 in-use by clones or has snapshots can not be removed.
415 trash restore image-id
417 Restore an image from trash.
418
420 image-spec is [pool-name]/image-name
421 snap-spec is [pool-name]/image-name@snap-name
422 The default for pool-name is "rbd". If an image name contains a slash
425 character ('/'), pool-name is required.
426 You may specify each name individually, using --pool, --image and
428 --snap options, but this is discouraged in favor of the above spec syn‐
429 tax.
430
432 RBD images are striped over many objects, which are then stored by the
433 Ceph distributed object store (RADOS). As a result, read and write
434 requests for the image are distributed across many nodes in the clus‐
435 ter, generally preventing any single node from becoming a bottleneck
436 when individual images get large or busy.
437 The striping is controlled by three parameters:
439 object-size
441 The size of objects we stripe over is a power of two. It will be
442 rounded up the nearest power of two. The default object size is
443 4 MB, smallest is 4K and maximum is 32M.
444 stripe_unit
446 Each [stripe_unit] contiguous bytes are stored adjacently in the
447 same object, before we move on to the next object.
448 stripe_count
450 After we write [stripe_unit] bytes to [stripe_count] objects, we
451 loop back to the initial object and write another stripe, until
452 the object reaches its maximum size. At that point, we move on
453 to the next [stripe_count] objects.
454 By default, [stripe_unit] is the same as the object size and
456 [stripe_count] is 1. Specifying a different [stripe_unit] requires
457 that the STRIPINGV2 feature be supported (added in Ceph v0.53) and for‐
458 mat 2 images be used.
459
461 Most of these options are useful mainly for debugging and benchmarking.
462 The default values are set in the kernel and may therefore depend on
463 the version of the running kernel.
464 Per client instance rbd map options:
466 · fsid=aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee - FSID that should be
468 assumed by the client.
469 · ip=a.b.c.d[:p] - IP and, optionally, port the client should use.
471 · share - Enable sharing of client instances with other mappings
473 (default).
474 · noshare - Disable sharing of client instances with other mappings.
476 · crc - Enable CRC32C checksumming for data writes (default).
478 · nocrc - Disable CRC32C checksumming for data writes.
480 · cephx_require_signatures - Require cephx message signing (since 3.19,
482 default).
483 · nocephx_require_signatures - Don't require cephx message signing
485 (since 3.19).
486 · tcp_nodelay - Disable Nagle's algorithm on client sockets (since 4.0,
488 default).
489 · notcp_nodelay - Enable Nagle's algorithm on client sockets (since
491 4.0).
492 · cephx_sign_messages - Enable message signing (since 4.4, default).
494 · nocephx_sign_messages - Disable message signing (since 4.4).
496 · mount_timeout=x - A timeout on various steps in rbd map and rbd unmap
498 sequences (default is 60 seconds). In particular, since 4.2 this can
499 be used to ensure that rbd unmap eventually times out when there is
500 no network connection to a cluster.
501 · osdkeepalive=x - OSD keepalive timeout (default is 5 seconds).
503 · osd_idle_ttl=x - OSD idle TTL (default is 60 seconds).
505 Per mapping (block device) rbd map options:
507 · rw - Map the image read-write (default).
509 · ro - Map the image read-only. Equivalent to --read-only.
511 · queue_depth=x - queue depth (since 4.2, default is 128 requests).
513 · lock_on_read - Acquire exclusive lock on reads, in addition to writes
515 and discards (since 4.9).
516 · exclusive - Disable automatic exclusive lock transitions (since
518 4.12).
519 rbd unmap options:
521 · force - Force the unmapping of a block device that is open (since
523 4.9). The driver will wait for running requests to complete and then
524 unmap; requests sent to the driver after initiating the unmap will be
525 failed.
526
528 To create a new rbd image that is 100 GB:
529 rbd create mypool/myimage --size 102400
531 To use a non-default object size (8 MB):
533 rbd create mypool/myimage --size 102400 --object-size 8M
535 To delete an rbd image (be careful!):
537 rbd rm mypool/myimage
539 To create a new snapshot:
541 rbd snap create mypool/myimage@mysnap
543 To create a copy-on-write clone of a protected snapshot:
545 rbd clone mypool/myimage@mysnap otherpool/cloneimage
547 To see which clones of a snapshot exist:
549 rbd children mypool/myimage@mysnap
551 To delete a snapshot:
553 rbd snap rm mypool/myimage@mysnap
555 To map an image via the kernel with cephx enabled:
557 rbd map mypool/myimage --id admin --keyfile secretfile
559 To map an image via the kernel with different cluster name other than
561 default ceph:
562 rbd map mypool/myimage --cluster cluster-name
564 To unmap an image:
566 rbd unmap /dev/rbd0
568 To create an image and a clone from it:
570 rbd import --image-format 2 image mypool/parent
572 rbd snap create mypool/parent@snap
573 rbd snap protect mypool/parent@snap
574 rbd clone mypool/parent@snap otherpool/child
575 To create an image with a smaller stripe_unit (to better distribute
577 small writes in some workloads):
578 rbd create mypool/myimage --size 102400 --stripe-unit 65536B --stripe-count 16
580 To change an image from one image format to another, export it and then
582 import it as the desired image format:
583 rbd export mypool/myimage@snap /tmp/img
585 rbd import --image-format 2 /tmp/img mypool/myimage2
586 To lock an image for exclusive use:
588 rbd lock add mypool/myimage mylockid
590 To release a lock:
592 rbd lock remove mypool/myimage mylockid client.2485
594 To list images from trash:
596 rbd trash ls mypool
598 To defer delete an image (use --delay to set delay-time, default is 0):
600 rbd trash mv mypool/myimage
602 To delete an image from trash (be careful!):
604 rbd trash rm mypool/myimage-id
606 To force delete an image from trash (be careful!):
608 rbd trash rm mypool/myimage-id --force
610 To restore an image from trash:
612 rbd trash restore mypool/myimage-id
614 To restore an image from trash and rename it:
616 rbd trash restore mypool/myimage-id --image mynewimage
618
620 rbd is part of Ceph, a massively scalable, open-source, distributed
621 storage system. Please refer to the Ceph documentation at
622 http://ceph.com/docs for more information.
623
625 ceph(8), rados(8)
626
628 2010-2014, Inktank Storage, Inc. and contributors. Licensed under Cre‐
629 ative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0)
630dev Apr 14, 2019 RBD(8)