1RBD(8) Ceph RBD(8)
2
3
4
6 rbd - manage rados block device (RBD) images
7
9 rbd [ -c ceph.conf ] [ -m monaddr ] [--cluster cluster name]
10 [ -p | --pool pool ] [ command ... ]
11
12
14 rbd is a utility for manipulating rados block device (RBD) images, used
15 by the Linux rbd driver and the rbd storage driver for Qemu/KVM. RBD
16 images are simple block devices that are striped over objects and
17 stored in a RADOS object store. The size of the objects the image is
18 striped over must be a power of two.
19
21 -c ceph.conf, --conf ceph.conf
22 Use ceph.conf configuration file instead of the default
23 /etc/ceph/ceph.conf to determine monitor addresses during
24 startup.
25
26 -m monaddress[:port]
27 Connect to specified monitor (instead of looking through
28 ceph.conf).
29
30 --cluster cluster name
31 Use different cluster name as compared to default cluster name
32 ceph.
33
34 -p pool-name, --pool pool-name
35 Interact with the given pool. Required by most commands.
36
37 --no-progress
38 Do not output progress information (goes to standard error by
39 default for some commands).
40
42 --image-format format-id
43 Specifies which object layout to use. The default is 2.
44
45 · format 1 - (deprecated) Use the original format for a new rbd
46 image. This format is understood by all versions of librbd and
47 the kernel rbd module, but does not support newer features
48 like cloning.
49
50 · format 2 - Use the second rbd format, which is supported by
51 librbd and kernel since version 3.11 (except for striping).
52 This adds support for cloning and is more easily extensible to
53 allow more features in the future.
54
55 -s size-in-M/G/T, --size size-in-M/G/T
56 Specifies the size of the new rbd image or the new size of the
57 existing rbd image in M/G/T. If no suffix is given, unit M is
58 assumed.
59
60 --object-size size-in-B/K/M
61 Specifies the object size in B/K/M. Object size will be rounded
62 up the nearest power of two; if no suffix is given, unit B is
63 assumed. The default object size is 4M, smallest is 4K and max‐
64 imum is 32M.
65
66 --stripe-unit size-in-B/K/M
67 Specifies the stripe unit size in B/K/M. If no suffix is given,
68 unit B is assumed. See striping section (below) for more
69 details.
70
71 --stripe-count num
72 Specifies the number of objects to stripe over before looping
73 back to the first object. See striping section (below) for more
74 details.
75
76 --snap snap
77 Specifies the snapshot name for the specific operation.
78
79 --id username
80 Specifies the username (without the client. prefix) to use with
81 the map command.
82
83 --keyring filename
84 Specifies a keyring file containing a secret for the specified
85 user to use with the map command. If not specified, the default
86 keyring locations will be searched.
87
88 --keyfile filename
89 Specifies a file containing the secret key of --id user to use
90 with the map command. This option is overridden by --keyring if
91 the latter is also specified.
92
93 --shared lock-tag
94 Option for lock add that allows multiple clients to lock the
95 same image if they use the same tag. The tag is an arbitrary
96 string. This is useful for situations where an image must be
97 open from more than one client at once, like during live migra‐
98 tion of a virtual machine, or for use underneath a clustered
99 filesystem.
100
101 --format format
102 Specifies output formatting (default: plain, json, xml)
103
104 --pretty-format
105 Make json or xml formatted output more human-readable.
106
107 -o krbd-options, --options krbd-options
108 Specifies which options to use when mapping or unmapping an
109 image via the rbd kernel driver. krbd-options is a comma-sepa‐
110 rated list of options (similar to mount(8) mount options). See
111 kernel rbd (krbd) options section below for more details.
112
113 --read-only
114 Map the image read-only. Equivalent to -o ro.
115
116 --image-feature feature-name
117 Specifies which RBD format 2 feature should be enabled when cre‐
118 ating an image. Multiple features can be enabled by repeating
119 this option multiple times. The following features are sup‐
120 ported:
121
122 · layering: layering support
123
124 · striping: striping v2 support
125
126 · exclusive-lock: exclusive locking support
127
128 · object-map: object map support (requires exclusive-lock)
129
130 · fast-diff: fast diff calculations (requires object-map)
131
132 · deep-flatten: snapshot flatten support
133
134 · journaling: journaled IO support (requires exclusive-lock)
135
136 --image-shared
137 Specifies that the image will be used concurrently by multiple
138 clients. This will disable features that are dependent upon
139 exclusive ownership of the image.
140
141 --whole-object
142 Specifies that the diff should be limited to the extents of a
143 full object instead of showing intra-object deltas. When the
144 object map feature is enabled on an image, limiting the diff to
145 the object extents will dramatically improve performance since
146 the differences can be computed by examining the in-memory
147 object map instead of querying RADOS for each object within the
148 image.
149
151 ls [-l | --long] [pool-name]
152 Will list all rbd images listed in the rbd_directory object.
153 With -l, also show snapshots, and use longer-format output
154 including size, parent (if clone), format, etc.
155
156 du [-p | --pool pool-name] [image-spec | snap-spec]
157 Will calculate the provisioned and actual disk usage of all
158 images and associated snapshots within the specified pool. It
159 can also be used against individual images and snapshots.
160
161 If the RBD fast-diff feature isn't enabled on images, this oper‐
162 ation will require querying the OSDs for every potential object
163 within the image.
164
165 info image-spec | snap-spec
166 Will dump information (such as size and object size) about a
167 specific rbd image. If image is a clone, information about its
168 parent is also displayed. If a snapshot is specified, whether
169 it is protected is shown as well.
170
171 create (-s | --size size-in-M/G/T) [--image-format format-id]
172 [--object-size size-in-B/K/M] [--stripe-unit size-in-B/K/M
173 --stripe-count num] [--image-feature feature-name]... [--image-shared]
174 image-spec
175 Will create a new rbd image. You must also specify the size via
176 --size. The --stripe-unit and --stripe-count arguments are
177 optional, but must be used together.
178
179 clone [--object-size size-in-B/K/M] [--stripe-unit size-in-B/K/M
180 --stripe-count num] [--image-feature feature-name] [--image-shared]
181 parent-snap-spec child-image-spec
182 Will create a clone (copy-on-write child) of the parent snap‐
183 shot. Object size will be identical to that of the parent image
184 unless specified. Size will be the same as the parent snapshot.
185 The --stripe-unit and --stripe-count arguments are optional, but
186 must be used together.
187
188 The parent snapshot must be protected (see rbd snap protect).
189 This requires image format 2.
190
191 flatten image-spec
192 If image is a clone, copy all shared blocks from the parent
193 snapshot and make the child independent of the parent, severing
194 the link between parent snap and child. The parent snapshot can
195 be unprotected and deleted if it has no further dependent
196 clones.
197
198 This requires image format 2.
199
200 children snap-spec
201 List the clones of the image at the given snapshot. This checks
202 every pool, and outputs the resulting poolname/imagename.
203
204 This requires image format 2.
205
206 resize (-s | --size size-in-M/G/T) [--allow-shrink] image-spec
207 Resizes rbd image. The size parameter also needs to be speci‐
208 fied. The --allow-shrink option lets the size be reduced.
209
210 rm image-spec
211 Deletes an rbd image (including all data blocks). If the image
212 has snapshots, this fails and nothing is deleted.
213
214 export (image-spec | snap-spec) [dest-path]
215 Exports image to dest path (use - for stdout).
216
217 import [--image-format format-id] [--object-size size-in-B/K/M]
218 [--stripe-unit size-in-B/K/M --stripe-count num] [--image-feature fea‐
219 ture-name]... [--image-shared] src-path [image-spec]
220 Creates a new image and imports its data from path (use - for
221 stdin). The import operation will try to create sparse rbd
222 images if possible. For import from stdin, the sparsification
223 unit is the data block size of the destination image (object
224 size).
225
226 The --stripe-unit and --stripe-count arguments are optional, but
227 must be used together.
228
229 export-diff [--from-snap snap-name] [--whole-object] (image-spec |
230 snap-spec) dest-path
231 Exports an incremental diff for an image to dest path (use - for
232 stdout). If an initial snapshot is specified, only changes
233 since that snapshot are included; otherwise, any regions of the
234 image that contain data are included. The end snapshot is spec‐
235 ified using the standard --snap option or @snap syntax (see
236 below). The image diff format includes metadata about image
237 size changes, and the start and end snapshots. It efficiently
238 represents discarded or 'zero' regions of the image.
239
240 merge-diff first-diff-path second-diff-path merged-diff-path
241 Merge two continuous incremental diffs of an image into one sin‐
242 gle diff. The first diff's end snapshot must be equal with the
243 second diff's start snapshot. The first diff could be - for
244 stdin, and merged diff could be - for stdout, which enables mul‐
245 tiple diff files to be merged using something like 'rbd
246 merge-diff first second - | rbd merge-diff - third result'. Note
247 this command currently only support the source incremental diff
248 with stripe_count == 1
249
250 import-diff src-path image-spec
251 Imports an incremental diff of an image and applies it to the
252 current image. If the diff was generated relative to a start
253 snapshot, we verify that snapshot already exists before continu‐
254 ing. If there was an end snapshot we verify it does not already
255 exist before applying the changes, and create the snapshot when
256 we are done.
257
258 diff [--from-snap snap-name] [--whole-object] image-spec | snap-spec
259 Dump a list of byte extents in the image that have changed since
260 the specified start snapshot, or since the image was created.
261 Each output line includes the starting offset (in bytes), the
262 length of the region (in bytes), and either 'zero' or 'data' to
263 indicate whether the region is known to be zeros or may contain
264 other data.
265
266 cp (src-image-spec | src-snap-spec) dest-image-spec
267 Copies the content of a src-image into the newly created
268 dest-image. dest-image will have the same size, object size,
269 and image format as src-image.
270
271 mv src-image-spec dest-image-spec
272 Renames an image. Note: rename across pools is not supported.
273
274 image-meta list image-spec
275 Show metadata held on the image. The first column is the key and
276 the second column is the value.
277
278 image-meta get image-spec key
279 Get metadata value with the key.
280
281 image-meta set image-spec key value
282 Set metadata key with the value. They will displayed in
283 image-meta list.
284
285 image-meta remove image-spec key
286 Remove metadata key with the value.
287
288 object-map rebuild image-spec | snap-spec
289 Rebuilds an invalid object map for the specified image. An image
290 snapshot can be specified to rebuild an invalid object map for a
291 snapshot.
292
293 snap ls image-spec
294 Dumps the list of snapshots inside a specific image.
295
296 snap create snap-spec
297 Creates a new snapshot. Requires the snapshot name parameter
298 specified.
299
300 snap rollback snap-spec
301 Rollback image content to snapshot. This will iterate through
302 the entire blocks array and update the data head content to the
303 snapshotted version.
304
305 snap rm snap-spec
306 Removes the specified snapshot.
307
308 snap purge image-spec
309 Removes all snapshots from an image.
310
311 snap protect snap-spec
312 Protect a snapshot from deletion, so that clones can be made of
313 it (see rbd clone). Snapshots must be protected before clones
314 are made; protection implies that there exist dependent cloned
315 children that refer to this snapshot. rbd clone will fail on a
316 nonprotected snapshot.
317
318 This requires image format 2.
319
320 snap unprotect snap-spec
321 Unprotect a snapshot from deletion (undo snap protect). If
322 cloned children remain, snap unprotect fails. (Note that clones
323 may exist in different pools than the parent snapshot.)
324
325 This requires image format 2.
326
327 map [-o | --options krbd-options ] [--read-only] image-spec | snap-spec
328 Maps the specified image to a block device via the rbd kernel
329 module.
330
331 unmap [-o | --options krbd-options ] image-spec | snap-spec |
332 device-path
333 Unmaps the block device that was mapped via the rbd kernel mod‐
334 ule.
335
336 showmapped
337 Show the rbd images that are mapped via the rbd kernel module.
338
339 nbd map [--device device-path] [--read-only] image-spec | snap-spec
340 Maps the specified image to a block device via the rbd-nbd tool.
341
342 nbd unmap device-path
343 Unmaps the block device that was mapped via the rbd-nbd tool.
344
345 nbd list
346 Show the list of used nbd devices via the rbd-nbd tool.
347
348 status image-spec
349 Show the status of the image, including which clients have it
350 open.
351
352 feature disable image-spec feature-name...
353 Disables the specified feature on the specified image. Multiple
354 features can be specified.
355
356 feature enable image-spec feature-name...
357 Enables the specified feature on the specified image. Multiple
358 features can be specified.
359
360 lock list image-spec
361 Show locks held on the image. The first column is the locker to
362 use with the lock remove command.
363
364 lock add [--shared lock-tag] image-spec lock-id
365 Lock an image. The lock-id is an arbitrary name for the user's
366 convenience. By default, this is an exclusive lock, meaning it
367 will fail if the image is already locked. The --shared option
368 changes this behavior. Note that locking does not affect any
369 operation other than adding a lock. It does not protect an image
370 from being deleted.
371
372 lock remove image-spec lock-id locker
373 Release a lock on an image. The lock id and locker are as output
374 by lock ls.
375
376 bench-write [--io-size size-in-B/K/M/G/T] [--io-threads
377 num-ios-in-flight] [--io-total size-in-B/K/M/G/T] [--io-pattern seq |
378 rand] image-spec
379 Generate a series of writes to the image and measure the write
380 throughput and latency. If no suffix is given, unit B is
381 assumed for both --io-size and --io-total. Defaults are:
382 --io-size 4096, --io-threads 16, --io-total 1G, --io-pattern
383 seq.
384
386 image-spec is [pool-name]/image-name
387 snap-spec is [pool-name]/image-name@snap-name
388
389
390 The default for pool-name is "rbd". If an image name contains a slash
391 character ('/'), pool-name is required.
392
393 You may specify each name individually, using --pool, --image and
394 --snap options, but this is discouraged in favor of the above spec syn‐
395 tax.
396
398 RBD images are striped over many objects, which are then stored by the
399 Ceph distributed object store (RADOS). As a result, read and write
400 requests for the image are distributed across many nodes in the clus‐
401 ter, generally preventing any single node from becoming a bottleneck
402 when individual images get large or busy.
403
404 The striping is controlled by three parameters:
405
406 -size The size of objects we stripe over is a power of two. It will be
407 rounded up the nearest power of two. The default object size is
408 4 MB, smallest is 4K and maximum is 32M.
409
410 stripe_unit
411 Each [stripe_unit] contiguous bytes are stored adjacently in the
412 same object, before we move on to the next object.
413
414 stripe_count
415 After we write [stripe_unit] bytes to [stripe_count] objects, we
416 loop back to the initial object and write another stripe, until
417 the object reaches its maximum size. At that point, we move on
418 to the next [stripe_count] objects.
419
420 By default, [stripe_unit] is the same as the object size and
421 [stripe_count] is 1. Specifying a different [stripe_unit] requires
422 that the STRIPINGV2 feature be supported (added in Ceph v0.53) and for‐
423 mat 2 images be used.
424
426 Most of these options are useful mainly for debugging and benchmarking.
427 The default values are set in the kernel and may therefore depend on
428 the version of the running kernel.
429
430 Per client instance rbd map options:
431
432 · fsid=aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee - FSID that should be
433 assumed by the client.
434
435 · ip=a.b.c.d[:p] - IP and, optionally, port the client should use.
436
437 · share - Enable sharing of client instances with other mappings
438 (default).
439
440 · noshare - Disable sharing of client instances with other mappings.
441
442 · crc - Enable CRC32C checksumming for data writes (default).
443
444 · nocrc - Disable CRC32C checksumming for data writes.
445
446 · cephx_require_signatures - Require cephx message signing (since 3.19,
447 default).
448
449 · nocephx_require_signatures - Don't require cephx message signing
450 (since 3.19).
451
452 · tcp_nodelay - Disable Nagle's algorithm on client sockets (since 4.0,
453 default).
454
455 · notcp_nodelay - Enable Nagle's algorithm on client sockets (since
456 4.0).
457
458 · cephx_sign_messages - Enable message signing (since 4.4, default).
459
460 · nocephx_sign_messages - Disable message signing (since 4.4).
461
462 · mount_timeout=x - A timeout on various steps in rbd map and rbd unmap
463 sequences (default is 60 seconds). In particular, since 4.2 this can
464 be used to ensure that rbd unmap eventually times out when there is
465 no network connection to a cluster.
466
467 · osdkeepalive=x - OSD keepalive timeout (default is 5 seconds).
468
469 · osd_idle_ttl=x - OSD idle TTL (default is 60 seconds).
470
471 Per mapping (block device) rbd map options:
472
473 · rw - Map the image read-write (default).
474
475 · ro - Map the image read-only. Equivalent to --read-only.
476
477 · queue_depth=x - queue depth (since 4.2, default is 128 requests).
478
479 · lock_on_read - Acquire exclusive lock on reads, in addition to writes
480 and discards (since 4.9).
481
482 rbd unmap options:
483
484 · force - Force the unmapping of a block device that is open (since
485 4.9). The driver will wait for running requests to complete and then
486 unmap; requests sent to the driver after initiating the unmap will be
487 failed.
488
490 To create a new rbd image that is 100 GB:
491
492 rbd create mypool/myimage --size 102400
493
494 To use a non-default object size (8 MB):
495
496 rbd create mypool/myimage --size 102400 --object-size 8M
497
498 To delete an rbd image (be careful!):
499
500 rbd rm mypool/myimage
501
502 To create a new snapshot:
503
504 rbd snap create mypool/myimage@mysnap
505
506 To create a copy-on-write clone of a protected snapshot:
507
508 rbd clone mypool/myimage@mysnap otherpool/cloneimage
509
510 To see which clones of a snapshot exist:
511
512 rbd children mypool/myimage@mysnap
513
514 To delete a snapshot:
515
516 rbd snap rm mypool/myimage@mysnap
517
518 To map an image via the kernel with cephx enabled:
519
520 rbd map mypool/myimage --id admin --keyfile secretfile
521
522 To map an image via the kernel with different cluster name other than
523 default ceph.
524 rbd map mypool/myimage --cluster cluster name
525
526 To unmap an image:
527
528 rbd unmap /dev/rbd0
529
530 To create an image and a clone from it:
531
532 rbd import --image-format 2 image mypool/parent
533 rbd snap create mypool/parent@snap
534 rbd snap protect mypool/parent@snap
535 rbd clone mypool/parent@snap otherpool/child
536
537 To create an image with a smaller stripe_unit (to better distribute
538 small writes in some workloads):
539
540 rbd create mypool/myimage --size 102400 --stripe-unit 65536B --stripe-count 16
541
542 To change an image from one image format to another, export it and then
543 import it as the desired image format:
544
545 rbd export mypool/myimage@snap /tmp/img
546 rbd import --image-format 2 /tmp/img mypool/myimage2
547
548 To lock an image for exclusive use:
549
550 rbd lock add mypool/myimage mylockid
551
552 To release a lock:
553
554 rbd lock remove mypool/myimage mylockid client.2485
555
557 rbd is part of Ceph, a massively scalable, open-source, distributed
558 storage system. Please refer to the Ceph documentation at
559 http://ceph.com/docs for more information.
560
562 ceph(8), rados(8)
563
565 2010-2014, Inktank Storage, Inc. and contributors. Licensed under Cre‐
566 ative Commons BY-SA
567
568
569
570
571dev October 30, 2018 RBD(8)