1BTRBK.CONF(5) Btrbk Manual BTRBK.CONF(5)
2
6 btrbk.conf - btrbk configuration file
7
9 /etc/btrbk.conf
10 /etc/btrbk/btrbk.conf
11
13 The btrbk configuration file specifies which btrfs subvolumes on the
14 filesystem are to be processed, what target subvolumes should be used
15 to create the backups, and where the snapshots should be generated. The
16 retention policy, as well as most other options can be defined either
17 globally or within a section.
18 The options specified always apply to the last section encountered,
20 superseding the values set in upper-level sections. This means that
21 global options must be set before any sections are defined.
22 Blank lines are ignored. A hash character (#) starts a comment
24 extending until end of line.
25
27 volume <volume-directory>|<url> (optional)
28 Absolute path pointing to a btrfs file system containing the source
29 subvolume(s) to be backed up. Usually the mount point of a btrfs
30 filesystem mounted with the subvolid=5 option.
31 subvolume <subvolume-name>
33 Subvolume to be backed up, relative to the <volume-directory> of
34 the volume section, or absolute if the volume section is omitted.
35 Accepts wildcard character "*".
36 Note that if this subvolume is btrfs root (id=5), it needs to have
38 a valid UUID, which is not the case for file systems created with
39 btrfs-progs < 4.16.
40 target [send-receive|raw] <target-directory>|<url>
42 Target directory where the backup subvolumes are to be created. The
43 optional target type defaults to “send-receive”, see TARGET TYPES
44 below for details.
45 Multiple target sections are allowed, in any context: a target
47 defined in volume or global context will be used for all underlying
48 subvolume sections (hint: run "btrbk list" or "btrbk config print"
49 to see the resulting configuration).
50 If a <url> is specified, btrbk actions (shell commands) are executed
52 remotely via ssh, using the SSH Options described below. Accepted
53 formats are:
54 ssh://<hostname>[:<port>]/<directory>
56 <hostname>:<directory>
57 Where <hostname> is either a host name, an IPv4 address in
59 dotted-decimal form, or an IP literal encapsulated within square
60 brackets (e.g. "[2001:db8::7]").
61 If you are connecting to virtual machines, consider configuring several
63 volume sections for a <hostname>, with distinct <port> numbers for each
64 machine.
65
67 The options described here can be specified in global context as well
68 as volume, subvolume and target sections, unless stated otherwise.
69 Basic Options
71 timestamp_format short|long|long-iso
72 Timestamp format used as postfix for new snapshot subvolume names.
73 Defaults to “long”.
74 short
76 YYYYMMDD[_N] (e.g. "20150825", "20150825_1")
77 long
79 YYYYMMDD<T>hhmm[_N] (e.g. "20150825T1531")
80 long-iso
82 YYYYMMDD<T>hhmmss±hhmm[_N] (e.g.
83 "20150825T153123+0200")
84 Note that a postfix "_N" is appended to the timestamp if a snapshot
86 or backup already exists with the timestamp of current date/time.
87 Use “long-iso” if you want to make sure that btrbk never creates
89 ambiguous time stamps (which can happen if multiple snapshots are
90 created during a daylight saving time clock change).
91 Note that using “long-iso” has implications on the scheduling, see
93 Reference Time below.
94 snapshot_dir <directory>
96 Directory in which the btrfs snapshots are created, relative to
97 <volume-directory> of the volume section, or absolute if the volume
98 section is omitted. Note that btrbk does not automatically create
99 this directory, and the snapshot creation will fail if it is not
100 present.
101 snapshot_name <basename>
103 Base name of the created snapshot (and backup). This option is only
104 valid in the subvolume section. Defaults to <subvolume-name>.
105 snapshot_create always|onchange|ondemand|no
107 If set to “always”, snapshots are always created. If set to
108 “onchange”, snapshots are only created if the last snapshot is not
109 up-to-date, i.e. the source subvolume has changed (more precisely:
110 the btrfs generation has been increased) since the last snapshot
111 was created. If set to “ondemand”, snapshots are only created if at
112 least one target subvolume is reachable (useful if you are tight on
113 disk space and you only need btrbk for backups to an external disk
114 which is not always connected). If set to “no”, the snapshots are
115 never created (useful if another instance of btrbk is taking care
116 of snapshot creation). Defaults to “always”.
117 incremental yes|no|strict
119 If set, incremental backups are created. If set to “strict”,
120 non-incremental (initial) backups are never created, and
121 incremental backups are restricted to related parents (by
122 parent-uuid relationship). Defaults to “yes”.
123 Note that even if the parent-uuid chain is broken, snapshots and
125 backups can still share data (which is especially true for backups
126 created with incremental option enabled), and are perfectly
127 suitable as parents for incremental send-receive operations. But as
128 btrbk can not be certain about this, such operations are disallowed
129 in "incremental strict" mode.
130 noauto yes|no
132 If set, the context is skipped by all btrbk actions unless
133 explicitly enabled by a matching btrbk <filter> command line
134 argument (e.g. "btrbk run myfilter").
135 Grouping Options
137 group <group-name> [<group-name>]...
138 Add the current section (volume, subvolume or target) to
139 user-defined groups, which can be used as filter for most btrbk
140 commands (see btrbk(1) FILTER STATEMENTS). This option can be set
141 multiple times within the same context.
142 Retention Policy Options
144 preserve_day_of_week monday|tuesday|...|sunday
145 Defines on what day a snapshot/backup is considered to be a
146 "weekly" backup. Weekly, monthly and yearly backups are preserved
147 on this day of week (see RETENTION POLICY below). Defaults to
148 “sunday”.
149 preserve_hour_of_day [0..23]
151 Defines after what time (in full hours since midnight) a
152 snapshot/backup is considered to be a "daily" backup. Daily,
153 weekly, monthly and yearly backups are preserved on this hour (see
154 RETENTION POLICY below). Ignored on snapshots or backups without
155 time information (timestamp_format short). Defaults to “0”.
156 snapshot_preserve no|<retention_policy>
158 Set retention policy for snapshots (see RETENTION POLICY below). If
159 set to “no”, preserve snapshots according to snapshot_preserve_min
160 only. Defaults to “no”.
161 Note that snapshot_preserve has no effect if snapshot_preserve_min
163 is set to “all” (the default).
164 snapshot_preserve_min all|latest|<number>{h,d,w,m,y}
166 Preserve all snapshots for a minimum amount of hours (h), days (d),
167 weeks (w), months (m) or years (y), regardless of how many there
168 are. If set to “all”, preserve all snapshots forever. If set to
169 “latest”, preserve latest snapshot. Defaults to “all”.
170 target_preserve no|<retention_policy>
172 Set retention policy for backups (see RETENTION POLICY below). If
173 set to “no”, preserve backups according to target_preserve_min
174 only. Defaults to “no”.
175 Note that target_preserve has no effect if target_preserve_min is
177 set to “all” (the default).
178 target_preserve_min all|latest|no|<number>{h,d,w,m,y}
180 Preserve all backups for a minimum amount of hours (h), days (d),
181 weeks (w), months (m) or years (y), regardless of how many there
182 are. If set to “all”, preserve all backups forever. If set to
183 “latest”, always preserve the latest backup (useful in conjunction
184 with "target_preserve no", if you want to keep the latest backup
185 only). If set to “no”, only the backups following the
186 target_preserve policy are created. Defaults to “all”.
187 archive_preserve no|<retention_policy>
189 Set retention policy for archives ("btrbk archive" command), with
190 same semantics as target_preserve.
191 archive_preserve_min all|latest|no|<number>{h,d,w,m,y}
193 Set retention policy for archives ("btrbk archive" command), with
194 same semantics as target_preserve_min.
195 archive_exclude <pattern>
197 Exclude subvolumes matching <pattern> from archiving. The pattern
198 accepts wildcard character "*", and is matched against the end of
199 the pathname.
200 SSH Options
202 ssh_identity <file>|no
203 Absolute path to a ssh identity file (private key). If not set, the
204 ssh default is used (see ssh(1), "-i identity_file"). Note that if
205 the identity key is password protected and no authentication agent
206 is used, btrbk will prompt for user input on every connection
207 attempt.
208 ssh_user <username>|no
210 Remote username for ssh. Defaults to “root”. Make sure the remote
211 user is able to run "btrfs" with root privileges (see option
212 backend for details). If set to “no”, the ssh default is used.
213 ssh_compression yes|no
215 Enables or disables the compression of ssh connections. Defaults to
216 “no”. Note that if stream_compress is enabled, ssh compression will
217 always be disabled for send/receive operations.
218 ssh_cipher_spec default|<cipher_spec>
220 Selects the cipher specification for encrypting the session
221 (comma-separated list of ciphers in order of preference). See the
222 "-c cipher_spec" option in ssh(1) for more information. Defaults to
223 “default” (the ciphers specified in ssh_config(5)).
224 Data Stream Options
226 stream_compress <compress_command>|no
227 Compress the btrfs send stream before transferring it from/to
228 remote locations. Defaults to “no”. If enabled, make sure that
229 <compress_command> is available on the source and target hosts.
230 Supported <compress_command>: gzip, pigz, bzip2, pbzip2, bzip3, xz,
231 lzo, lz4, zstd.
232 stream_compress_level default|<number>
234 Compression level for the specified <compress_command>. Refer to
235 the related man-page for details (usually [1..9], where 1 means
236 fastest compression). Defaults to “default” (the default
237 compression level of <compress_command>).
238 stream_compress_long default|<number>
240 Enable long distance matching for the specified <compress_command>.
241 Refer to the related man-page for details. Only supported for
242 "zstd".
243 stream_compress_threads default|<number>
245 Number of threads to use for <compress_command>. Only supported for
246 "pigz", "pbzip2", "bzip3", "zstd" and recent versions of "xz".
247 stream_compress_adapt default|<number>
249 Enable adaptive compression for <compress_command>. Only supported
250 for "zstd" (version >= 1.3.6).
251 stream_buffer <size>|no
253 Add a buffer to the btrfs send stream (locally, on uncompressed
254 data), with a maximum size of <size>. This can give a speed
255 improvement (measured up to 20%) on both local or remote
256 operations, but also increases system load. A suffix of "k", "m",
257 "g", or "%" can be added to <size> to denote kilobytes (*1024),
258 megabytes, gigabytes, or a percentage of total physical memory.
259 Defaults to “no”.
260 If enabled, make sure that the "mbuffer" command (at least version
262 20180505) is available on the host running btrbk. As of
263 btrbk-0.29.0, mbuffer(1) is used for both rate_limit and
264 stream_buffer options:
265 mbuffer [-m <stream_buffer>] [-r <rate_limit>]
267 Note that mbuffer(1) always reads defaults from "/etc/mbuffer.rc"
269 and "~/.mbuffer.rc".
270 Leave this option disabled if your main concern is a stable backup
272 process: while recent versions of mbuffer have proven reliable, it
273 is often desirable to keep things simple rather than adding an
274 additional, multi-threaded process to the command pipe.
275 stream_buffer_remote <size>|no
277 Add a buffer on remote hosts (either source or target). Defaults to
278 “no”.
279 Enable this if you prefer buffering on the remote side, or even on
281 both sides: reasons for this depend on available memory, disk and
282 cpu performance (btrfs send/receive, compression), as well as
283 networking constraints.
284 rate_limit <rate>|no
286 Limit the read rate of the btrfs send stream to <rate> bytes per
287 second (locally, on uncompressed send stream). A suffix of "k",
288 "m", "g", or "t" can be added to denote kilobytes (*1024),
289 megabytes, and so on. Defaults to “no”. Note that rate_limit
290 implicitly adds a stream buffer (see stream_buffer option above).
291 rate_limit_remote <rate>|no
293 Add rate limit on remote hosts (either source or target). Defaults
294 to “no”. Note that it usually does not make much sense to enable
295 both rate_limit and rate_limit_remote.
296 System Options
298 transaction_log <file>|no
299 If set, all transactions (snapshot create, subvolume send-receive,
300 subvolume delete) as well as abort messages are logged to <file>,
301 in a space-separated table format: "localtime type status
302 target_url source_url parent_url message".
303 transaction_syslog <facility>|no
305 If set, all transactions (as described in transaction_log above)
306 are logged to syslog. The program name used in the messages is
307 "btrbk". Accepted parameters for <facility>: user, mail, daemon,
308 auth, lpr, news, cron, authpriv, local0..local7.
309 lockfile <file>|no
311 Create lockfile <file> on startup; checks lockfile before running
312 any btrfs commands (using perl "flock"), and exits if the lock is
313 held by another btrbk instance. Ignored on dryrun (-n, --dry-run).
314 See also --lockfile command-line option.
315 backend <backend>
317 Backend filesystem utilities to be used for btrfs specific
318 operations. Available backends:
319 btrfs-progs
321 Default backend, btrfs commands are called as specified in
322 btrfs(8) (e.g. "btrfs subvolume show").
323 btrfs-progs-btrbk
325 btrfs commands are separated by a dash instead of a whitespace
326 (e.g. "btrfs-subvolume-show" instead of "btrfs subvolume
327 show"). Useful for setting suid or file capabilities (setcap)
328 on specific btrfs commands, as implemented in
329 https://github.com/digint/btrfs-progs-btrbk.
330 btrfs-progs-sudo
332 btrfs commands are prefixed with "sudo -n" (e.g. "sudo -n btrfs
333 subvolume show" instead of "btrfs subvolume show"). Make sure
334 to have appropriate (root) permissions for the "btrfs" command
335 groups as well as the "readlink" and "test" commands in
336 /etc/sudoers.
337 btrfs-progs-doas
339 Similar to btrfs-progs-sudo, using prefix "doas -n".
340 If you want to set this option for local or remote hosts only, set
342 backend_local or backend_remote (e.g. "backend_remote
343 btrfs-progs-btrbk").
344 If you want to set this option for regular (non-root) user only,
346 set backend_local_user.
347 compat <compat-option>...
349 Enable compatibility options. Available compat-option:
350 busybox
352 Use busybox compatible commands, at the expense of slight
353 overhead while reading filesystem information.
354 ignore_receive_errors *experimental*
356 Tell btrfs-receive(8) to not terminate on errors by setting
357 "--max-errors=0" option. Print warnings instead.
358 A known use case for this are target hosts lacking xattr
360 support (e.g. some Synology NAS), while the send-stream
361 contains "lsetxattr" commands. Another case is targets failing
362 to set otime, complaining with "ERROR: attribute 12 requested
363 but not present".
364 Note that there is no guarantee that backups created with this
366 option enabled can be restored at all.
367 If you want to set this option for local or remote hosts only, set
369 compat_local or compat_remote (e.g. "compat_remote busybox").
370 cache_dir <directory>
372 If set, cache extent maps for the "btrbk extents" command.
373 Btrfs Specific Options
375 incremental_prefs <list-spec>[:<amount>]...
376 Specify the preferences to determine the best common (correlated)
377 parent and clone sources for incremental backups, by choosing from
378 predefined candidate lists.
379 The list-spec defines from what candidate list the next
381 parent/clone-src should be appended to a result list; amount
382 defines how many (e.g. "sro:1 sro:1" is identical to "sro:2"), or
383 all if omitted. Any candidate which is already in the results is
384 dropped.
385 The resulting list of subvolumes is then used as parameters for the
387 btrfs-send(8) command: the first for "-p <parent>", all others for
388 "-c <clone-src>".
389 Available list-spec (candidate lists = filtered subsets of
391 correlated subvolumes):
392 sro,srn
394 All from snapshot_dir matching snapshot_name, with parent_uuid
395 relationship, sorted by btrbk timestamp (o=older n=newer).
396 sao,san
398 All from snapshot_dir matching snapshot_name, sorted by btrbk
399 timestamp (o=older n=newer).
400 aro,arn
402 All from incremental_resolve, with parent_uuid relationship,
403 sorted by cgen (o=older n=newer).
404 Defaults to "sro:1 srn:1 sao:1 san:1 aro:1 arn:1". Note that for
406 most operations the default resolves a single parent, as there
407 usually are no newer snapshots, and all "sro:1 sao:1 aro:1" resolve
408 to the same snapshot.
409 Example: "defaults,sao,san,aro,arn" takes the defaults, and adds
411 clone sources for all (!) known candidates on the filesystem.
412 incremental_clones yes|no
414 If enabled, btrbk adds "-c <clone-src>" to the btrfs-send(8)
415 command for all correlated subvolumes resolved by
416 incremental_prefs. If disabled, only "-p <parent>" is used.
417 Defaults to “yes”.
418 incremental_resolve mountpoint|directory
420 Specifies where to search for the best common parent for
421 incremental backups. If set to “mountpoint”, use parents in the
422 filesystem tree below the mount point of the snapshot and target
423 directory. If set to “directory”, use parents strictly below
424 snapshot/target directories. Set this to “directory” if you get
425 access problems (when not running btrbk as root). Defaults to
426 “mountpoint”.
427 btrfs_commit_delete yes|no
429 If set, wait for the transaction commit at the end of each snapshot
430 or backup deletion (sets --commit-each option for "btrfs subvolume
431 delete"). Defaults to “no”.
432 send_protocol <number>|no *experimental*
434 Use btrfs send protocol version N. If enabled on target, btrbk adds
435 "--proto <number>" to the btrfs-send(8) command. Defaults to “no”
436 (btrfs default).
437 send_compressed_data yes|no *experimental*
439 Send data that is compressed on the filesystem directly without
440 decompressing it. This requires protocol version 2 or higher
441 (btrfs-progs >= 5.19), and implies "send_protocol 2". If enabled on
442 target, btrbk adds "--compressed-data" to the btrfs-send(8)
443 command. Defaults to “no” (btrfs default).
444 snapshot_qgroup_destroy yes|no *experimental*
446 target_qgroup_destroy yes|no *experimental*
449 archive_qgroup_destroy yes|no *experimental*
452 Whenever a subvolume is deleted, also destroy corresponding default
453 qgroup "0/<subvol-id>". Only useful if you have enabled btrfs quota
454 support. See also:
455 https://bugzilla.kernel.org/show_bug.cgi?id=91751
456 Informative Options
458 warn_unknown_targets yes|no
459 If set, prints a warning if btrbk encounters a target subvolume at
460 a unknown location (i.e. not following btrbk naming scheme, or
461 outside the target directory). Defaults to “no”.
462
464 Retention policies are defined individually for snapshots, backups and
465 archives (summarized as "backups" in the following text), using a
466 combination of:
467 *_preserve_min all|latest|no|<number>{h,d,w,m,y}
469 Amount of time (duration) in which all backups are preserved.
470 *_preserve no|<retention_policy>
472 Schedule (single points in time) for which individual backups are
473 preserved.
474 Note that if "preserve_min" is set to “all” (the default), any setting
476 of "preserve" obviously has no effect.
477 The format for <retention_policy> is:
479 [<hourly>h] [<daily>d] [<weekly>w] [<monthly>m] [<yearly>y]
481 hourly
483 Defines how many hours back hourly backups should be preserved. The
484 first backup of an hour is considered an hourly backup.
485 daily
487 Defines how many days back daily backups should be preserved. The
488 first backup of a day (starting at preserve_hour_of_day) is
489 considered a daily backup.
490 weekly
492 Defines how many weeks back weekly backups should be preserved. The
493 first daily backup created at preserve_day_of_week (or the first
494 backup in this week if none was made on the exact day) is
495 considered as a weekly backup.
496 monthly
498 Defines how many months back monthly backups should be preserved.
499 Every first weekly backup in a month is considered a monthly
500 backup.
501 yearly
503 Defines for how many years back yearly backups should be preserved.
504 Every first monthly backup in a year is considered a yearly backup.
505 Use an asterisk for “all” (e.g. "target_preserve 60d *m" states:
507 "preserve daily backups for 60 days back, and all monthly backups").
508 Hint: Run btrbk with the -S, --print-schedule option to get a
510 comprehensive output of the scheduler results.
511 Reference Time
513 The local time on the host running btrbk defines the reference time for
514 all date/time calculations, especially for "beginning of a day", and as
515 a consequence for the first daily, weekly, monthly or yearly backups.
516 The local time on remote hosts (ssh source/target) is never used.
517 Unless "timestamp_format long-iso" is set, daily backups are preserved
519 at "preserve_hour_of_day" (defaults to midnight) of the respective time
520 zone (and not for "00:00 UTC", which would be "14:00" in Honolulu).
521 This becomes relevant for setups with multiple btrbk instances, e.g.
522 many snapshot-only instances (spread around the world), and a
523 fetch-only instance on the backup server.
524 Caveat:
526 • If "timestamp_format long-iso" is set, each btrbk instance on has a
528 different interpretation of "first in day". Make sure to run btrbk
529 with the same time zone on every host, e.g. by setting the TZ
530 environment variable (see tzset(3)).
531
533 send-receive
534 Backup to a btrfs filesystem, using "btrfs send/receive". This is
535 the recommended (standard) target type. The <target-directory> must
536 be an absolute path and point to a subvolume or directory within a
537 btrfs file system. See btrfs-send(8), btrfs-receive(8).
538 raw *experimental*
540 Backup to a raw (filesystem independent) file from the output of
541 btrfs-send(8), with optional compression and encryption.
542 Note that the target preserve mechanism is currently disabled for
544 incremental raw backups (btrbk does not delete any incremental raw
545 files)!
546 Raw backups consist of two files: the main data file containing the
548 btrfs send stream, and a sidecar file ".info" containing metadata:
549 <snapshot-name>.<timestamp>[_N].btrfs[.gz|.bz2|...][.gpg]
551 <snapshot-name>.<timestamp>[_N].btrfs[.gz|.bz2|...][.gpg].info
552 For incremental backups ("incremental yes"), please note that:
554 • As soon as a single incremental backup file is lost or
556 corrupted, all later incremental backups become invalid, as
557 there is no common parent for the subsequent incremental images
558 anymore. This might be a good compromise for a vacation backup
559 plan, but for the long term make sure that a non-incremental
560 backup is triggered from time to time.
561 • There is currently no support for rotation of incremental
563 backups: if incremental is set, a full backup must be triggered
564 manually from time to time in order to be able to delete old
565 backups.
566 Additional options for raw targets:
568 raw_target_compress <compress_command>|no
570 Compression algorithm to use for raw backup target. Supported
571 <compress_command>: gzip, pigz, bzip2, pbzip2, bzip3, xz, lzo,
572 lz4, zstd.
573 raw_target_compress_level default|<number>
575 Compression level for the specified <compress_command>.
576 raw_target_compress_long default|<number>
578 Enable long distance matching for the specified
579 <compress_command>.
580 raw_target_compress_threads default|<number>
582 Number of threads to use for <compress_command>.
583 raw_target_split <size>|no
585 Split the raw backup file into pieces of size <size>.
586 raw_target_block_size <number>
588 Block size to use for writing the raw backup file. Defaults to
589 “128K”.
590 raw_target_encrypt gpg|openssl_enc|no
592 If enabled, encrypt the target raw file using gpg or
593 openssl_enc.
594 Additional options for "raw_target_encrypt gpg":
596 gpg_keyring <file>
598 Keyring to use for gpg, e.g. "/etc/btrbk/gpg/pubring.kbx".
599 gpg_recipient <name>...
601 Encrypt for user id <name> (email address).
602 Additional options for "raw_target_encrypt openssl_enc" (very
604 experimental):
605 openssl_ciphername <name>
607 Defaults to “aes-256-cbc”.
608 openssl_iv_size <size-in-bytes>|no
610 Depends on selected cipher.
611 openssl_keyfile <file>|no
613 Point to a key file in hex (absolute path). Example key file
614 creation (256bit key):
615 # dd if=/dev/urandom bs=1 count=32 \
617 | od -x -A n \
618 | tr -d "[:space:]" > /path/to/keyfile
619 kdf_backend <file>|no
621 KDF backend to be executed, e.g.
622 "/usr/share/btrbk/scripts/kdf_pbkdf2.py".
623 kdf_keysize <size-in-bytes>
625 Defaults to “32”.
626 kdf_keygen once|each
628 Defaults to “once”.
629
631 Please refer to the btrbk project page https://digint.ch/btrbk/ for
632 further details.
633
635 btrbk(1)
636
638 Axel Burri axel@tty0.ch
639Btrbk 0.32.6 2023-03-25 BTRBK.CONF(5)