1BTRBK.CONF(5) Btrbk Manual BTRBK.CONF(5)
2
6 btrbk.conf - btrbk configuration file
7
9 /etc/btrbk.conf
10 /etc/btrbk/btrbk.conf
11
13 The btrbk configuration file specifies which btrfs subvolumes on the
14 filesystem are to be processed, what target subvolumes should be used
15 to create the backups, and where the snapshots should be generated. The
16 retention policy, as well as most other options can be defined either
17 globally or within a section.
18 The options specified always apply to the last section encountered,
20 superseding the values set in upper-level sections. This means that
21 global options must be set before any sections are defined.
22 Blank lines are ignored. A hash character (#) starts a comment
24 extending until end of line.
25 Whitespace or unicode characters are not allowed for file names.
27 Allowed characters are:
28 [0-9] [a-z] [A-Z] and "._+-@"
30 This is for sanity/safety/security reasons, we apologize for the
32 inconvenience.
33
35 volume <volume-directory>|<url>
36 Directory of a btrfs volume containing the source subvolume(s) to
37 be backed up. <volume-directory> must be an absolute path and point
38 to a btrfs volume (or subvolume). Usually the mount point of a
39 btrfs filesystem mounted with the subvolid=5 option.
40 subvolume <subvolume-name>
42 Subvolume to be backed up, relative to the <volume-directory>
43 specified in the volume section. Multiple subvolume sections are
44 allowed within volume sections. Accepts wildcard character "*".
45 If set to ".", the subvolume at <volume-directory> is used as
47 backup source, and the snapshots will be created within the source
48 subvolume itself (see snapshot_dir option below), which is not
49 recommended. Note that if this subvolume is btrfs root (id=5), it
50 needs to have a valid UUID, which is not the case for file systems
51 created with btrfs-progs < 4.16.
52 target [send-receive|raw] <target-directory>|<url>
54 Target directory where the backup subvolumes are to be created. The
55 optional target type defaults to “send-receive”, see TARGET TYPES
56 below for details.
57 Multiple target sections are allowed, in any context: a target
59 defined in volume or global context will be used for all underlying
60 subvolume sections (hint: run "btrbk list" or "btrbk config print"
61 to see the resulting configuration).
62 If a <url> is specified, btrbk actions (shell commands) are executed
64 remotely via ssh, using the SSH Options described below. Accepted
65 formats are:
66 ssh://<hostname>[:<port>]/<directory>
68 <hostname>:<directory>
69 If you are connecting to virtual machines, consider configuring several
71 volume sections for a <hostname>, with distinct <port> numbers for each
72 machine.
73
75 The options described here can be specified in global context as well
76 as volume, subvolume and target sections, unless stated otherwise.
77 Basic Options
79 timestamp_format short|long|long-iso
80 Timestamp format used as postfix for new snapshot subvolume names.
81 Defaults to “short”.
82 short
84 YYYYMMDD[_N] (e.g. "20150825", "20150825_1")
85 long
87 YYYYMMDD<T>hhmm[_N] (e.g. "20150825T1531")
88 long-iso
90 YYYYMMDD<T>hhmmss±hhmm[_N] (e.g.
91 "20150825T153123+0200")
92 Note that a postfix "_N" is appended to the timestamp if a snapshot
94 or backup already exists with the timestamp of current date/time.
95 Use “long-iso” if you want to make sure that btrbk never creates
97 ambiguous time stamps (which can happen if multiple snapshots are
98 created during a daylight saving time clock change).
99 Note that using “long-iso” has implications on the scheduling, see
101 RETENTION POLICY (caveats) below.
102 snapshot_dir <directory>
104 Directory in which the btrfs snapshots are created, relative to
105 <volume-directory> of the volume section. Note that btrbk does not
106 autmatically create this directory, and the snapshot creation will
107 fail if it is not present.
108 snapshot_name <basename>
110 Base name of the created snapshot (and backup). This option is only
111 valid in the subvolume section. Defaults to <subvolume-name>.
112 snapshot_create always|onchange|ondemand|no
114 If set to “always”, snapshots are always created. If set to
115 “onchange”, snapshots are only created if the source subvolume has
116 changed since the last snapshot (more precisely: if the btrfs
117 generation has been increased since the last snapshot). If set to
118 “ondemand”, snapshots are only created if at least one target
119 subvolume is reachable (useful if you are tight on disk space and
120 you only need btrbk for backups to an external disk which is not
121 always connected). If set to “no”, the snapshots are never created
122 (useful if another instance of btrbk is taking care of snapshot
123 creation). Defaults to “always”.
124 incremental yes|no|strict
126 If set, incremental backups are created. If set to “strict”,
127 non-incremental (initial) backups are never created, and
128 incremental backups are restricted to related parents (by
129 parent-uuid relationship). Defaults to “yes”.
130 Note that even if the parent-uuid chain is broken, snapshots and
132 backups can still share data (which is especially true for backups
133 created with incremental option enabled), and are perfectly
134 suitable as parents for incremental send-receive operations. But as
135 btrbk can not be certain about this, such operations are disallowed
136 in "incremental strict" mode.
137 noauto yes|no
139 If set, the context is skipped by all btrbk actions unless
140 explicitely enabled by a matching btrbk <filter> command line
141 argument (e.g. "btrbk run myfilter").
142 Grouping Options
144 group <group-name> [<group-name>]...
145 Add the current section (volume, subvolume or target) to
146 user-defined groups, which can be used as filter for most btrbk
147 commands. This option can be set multiple times within the same
148 context.
149 Retention Policy Options
151 preserve_day_of_week monday|tuesday|...|sunday
152 Defines on what day a snapshot/backup is considered to be a
153 "weekly" backup. Weekly, monthly and yearly backups are preserved
154 on this day of week (see RETENTION POLICY below). Defaults to
155 “sunday”.
156 preserve_hour_of_day [0..23]
158 Defines after what time (in full hours since midnight) a
159 snapshot/backup is considered to be a "daily" backup. Daily,
160 weekly, monthly and yearly backups are preserved on this hour (see
161 RETENTION POLICY below). If you set this option, make sure to also
162 set timestamp_format to “long” or “long-iso” (backups and snapshots
163 having no time information will ignore this option). Defaults to
164 “0”.
165 snapshot_preserve no|<retention_policy>
167 Set retention policy for snapshots (see RETENTION POLICY below). If
168 set to “no”, preserve snapshots according to snapshot_preserve_min
169 only. Defaults to “no”.
170 snapshot_preserve_min all|latest|<number>{h,d,w,m,y}
172 Preserve all snapshots for a minimum amount of hours (h), days (d),
173 weeks (w), months (m) or years (y), regardless of how many there
174 are. If set to “all”, preserve all snapshots forever. If set to
175 “latest”, preserve latest snapshot. Defaults to “all”.
176 target_preserve no|<retention_policy>
178 Set retention policy for backups (see RETENTION POLICY below). If
179 set to “no”, preserve backups according to target_preserve_min
180 only. Defaults to “no”.
181 target_preserve_min all|latest|no|<number>{h,d,w,m,y}
183 Preserve all backups for a minimum amount of hours (h), days (d),
184 weeks (w), months (m) or years (y), regardless of how many there
185 are. If set to “all”, preserve all backups forever. If set to
186 “latest”, always preserve the latest backup (useful in conjunction
187 with "target_preserve no", if you want to keep the latest backup
188 only). If set to “no”, only the backups following the
189 target_preserve policy are created. Defaults to “all”.
190 archive_preserve no|<retention_policy>
192 Set retention policy for archives ("btrbk archive" command), with
193 same semantics as target_preserve.
194 archive_preserve_min all|latest|no|<number>{h,d,w,m,y}
196 Set retention policy for archives ("btrbk archive" command), with
197 same semantics as target_preserve_min.
198 archive_exclude <pattern>
200 Exclude subvolumes matching <pattern> from archiving. The pattern
201 accepts wildcard character "*", and is matched against the end of
202 the pathname.
203 SSH Options
205 ssh_identity <file>
206 Absolute path to a ssh identity file (private key). Note that if
207 the private key is password protected, btrbk will prompt for user
208 input, which is usually not desired.
209 ssh_user <username>
211 Remote username for ssh. Defaults to “root”. Make sure the remote
212 user is able to run "btrfs" with root privileges (see option
213 backend for details).
214 ssh_compression yes|no
216 Enables or disables the compression of ssh connections. Defaults to
217 “no”.
218 ssh_cipher_spec <cipher_spec>
220 Selects the cipher specification for encrypting the session
221 (comma-separated list of ciphers in order of preference). See the
222 "-c cipher_spec" option in ssh(1) for more information. Defaults to
223 “default” (the ciphers specified in ssh_config).
224 Previous versions btrbk allowed you to set a ssh_port option, this has
226 been dropped in favor of the ssh://hostname:port notation in the volume
227 and target sections. If you want to set a global port for all SSH
228 connections to remote hosts, set the “Port” option in ssh_config(5).
229 Data Stream Options
231 stream_compress <compress_command>|no
232 Compress the btrfs send stream before transferring it from/to
233 remote locations. Defaults to “no”. If enabled, make sure that
234 <compress_command> is available on the source and target hosts.
235 Supported <compress_command>: gzip, pigz, bzip2, pbzip2, xz, lzo,
236 lz4.
237 stream_compress_level default|<number>
239 Compression level for the specified <compress_command>. Refer to
240 the related man-page for details (usually [1..9], where 1 means
241 fastest compression). Defaults to “default” (the default
242 compression level of <compress_command>).
243 stream_compress_threads default|<number>
245 Number of threads to use for <compress_command>. Only supported for
246 "pigz", "pbzip2" and recent versions of "xz".
247 stream_buffer <size>|no
249 Add a buffer to the btrfs send stream (in front of "btrfs
250 receive"), with a maximum size of <size>. This can give a speed
251 improvement (measured up to 20%) on both local or remote
252 operations, but also increases system load. A suffix of "k", "m",
253 "g", or "%" can be added to <size> to denote kilobytes (*1024),
254 megabytes, gigabytes, or a percentage of total physical memory.
255 Defaults to “no”.
256 If enabled, make sure that the "mbuffer" command is available on
258 the target host. Note that versions of mbuffer < 20180505 suffered
259 from nasty bugs (infinite loops, hangs); if your main concern is a
260 stable backup process, leave this option disabled.
261 rate_limit <rate>|no
263 Limit the transfer to a maximum of <rate> bytes per second. A
264 suffix of "k", "m", "g", or "t" can be added to denote kilobytes
265 (*1024), megabytes, and so on. Defaults to “no”. If enabled for
266 remote sources, make sure that the "pv" command is available on the
267 source host.
268 System Options
270 transaction_log <file>|no
271 If set, all transactions (snapshot create, subvolume send-receive,
272 subvolume delete) as well as abort messages are logged to <file>,
273 in a space-separated table format: "localtime type status
274 target_url source_url parent_url message".
275 transaction_syslog <facility>|no
277 If set, all transactions (as described in transaction_log above)
278 are logged to syslog. The program name used in the messages is
279 "btrbk". Accepted parameters for <facility>: user, mail, daemon,
280 auth, lpr, news, cron, authpriv, local0..local7.
281 lockfile <file>|no
283 Create lockfile <file> on startup; checks lockfile before running
284 any btrfs commands (using perl "flock"), and exits if the lock is
285 held by another btrbk instance. Ignored on dryrun (-n, --dry-run).
286 See also --lockfile command-line option.
287 backend btrfs-progs|btrfs-progs-btrbk|btrfs-progs-sudo
289 Backend filesystem utilities to be used for btrfs specific
290 operations. Defaults to “btrfs-progs”.
291 btrfs-progs
293 Default backend, btrfs commands are called as specified in
294 btrfs(8) (e.g. "btrfs subvolume show").
295 btrfs-progs-btrbk
297 btrfs commands are separated by a dash instead of a whitespace
298 (e.g. "btrfs-subvolume-show" instead of "btrfs subvolume
299 show"). Useful for setting suid or file capabilities (setcap)
300 on specific btrfs commands, as implemented in
301 https://github.com/digint/btrfs-progs-btrbk.
302 btrfs-progs-sudo
304 btrfs commands are prefixed with "sudo -n" (e.g. "sudo -n btrfs
305 subvolume show" instead of "btrfs subvolume show"). Make sure
306 to have appropriate (root) permissions for the "btrfs" command
307 groups and the "readlink" command in /etc/sudoers.
308 For convenience, it is also possible to set backend_local or
310 backend_remote options, which will override the backend only for
311 local or remote sources/targets (e.g. "backend_remote
312 btrfs-progs-btrbk").
313 Btrfs Specific Options
315 btrfs_commit_delete after|each|no
316 If set, make sure the deletion of snapshot and backup subvolumes
317 are committed to disk when btrbk terminates. Defaults to “no”.
318 incremental_clones <number>
320 Maximum number of clone sources allowed for incremental send. If
321 set, btrbk adds "-c <clone-src>" to the btrfs-send(8) command for
322 all present snapshot/backup pairs (correlated subvolumes matching
323 matching received_uuid, printed by "btrbk stats"). Set this to a
324 high number if you want to make sure that no common data is missed
325 on incremental backups, in expense of btrfs-send performance.
326 Defaults to 0.
327 incremental_resolve mountpoint|directory
329 Specifies where to search for the best common parent for
330 incremental backups. If set to “mountpoint”, use parents in the
331 filesystem tree below mount points of source
332 "<volume-directory>/<snapshot-dir>" and target
333 "<target-directory>". If set to “directory”, use parents strictly
334 below source/target directories. Set this to “directory” if you get
335 access problems (when not running btrbk as root). Defaults to
336 “mountpoint”.
337 snapshot_qgroup_destroy yes|no *experimental*
339 target_qgroup_destroy yes|no *experimental*
342 archive_qgroup_destroy yes|no *experimental*
345 Whenever a subvolume is deleted, also destroy corresponding default
346 qgroup "0/<subvol-id>". Only useful if you have enabled btrfs quota
347 support. See also:
348 https://bugzilla.kernel.org/show_bug.cgi?id=91751
349
351 btrbk uses separate retention policies for snapshots and backups, which
352 are defined by the snapshot_preserve_min, snapshot_preserve,
353 target_preserve_min, target_preserve, preserve_day_of_week and
354 preserve_hour_of_day configuration options.
355 Within this section, any statement about "backups" is always valid for
357 backups as well as snapshots, referring to target_preserve or
358 snapshot_preserve respectively.
359 The format for <retention_policy> is:
361 [<hourly>h] [<daily>d] [<weekly>w] [<monthly>m] [<yearly>y]
363 With the following semantics:
365 hourly
367 Defines how many hours back hourly backups should be preserved. The
368 first backup of an hour is considered an hourly backup. Note that
369 if you use <hourly> scheduling, make sure to also set
370 timestamp_format to “long” or “long-iso”, or the scheduler will
371 interpret the time as "00:00" (midnight).
372 daily
374 Defines how many days back daily backups should be preserved. The
375 first backup of a day (starting at preserve_hour_of_day) is
376 considered a daily backup.
377 weekly
379 Defines how many weeks back weekly backups should be preserved. The
380 first daily backup created at preserve_day_of_week (or the first
381 backup in this week if none was made on the exact day) is
382 considered as a weekly backup.
383 monthly
385 Defines how many months back monthly backups should be preserved.
386 Every first weekly backup in a month is considered a monthly
387 backup.
388 yearly
390 Defines for how many years back yearly backups should be preserved.
391 Every first monthly backup in a year is considered a yearly backup.
392 Use an asterisk for “all” (e.g. "target_preserve 60d *m" states:
394 "preserve daily backups for 60 days back, and all monthly backups").
395 The reference time (which defines the beginning of a day, week, month
397 or year) for all date/time calculations is the local time of the host
398 running btrbk.
399 Hint: Run btrbk with the -S, --print-schedule option to get a
401 comprehensive output of the scheduler results.
402 Caveats:
404 • If you run a setup with several btrbk instances (e.g. one
406 snapshot-only instance per remote client, and a separate fetch-only
407 instance on the backup server), it makes perfectly sense to run
408 btrbk with different local time on the clients, in order to make
409 sure the backups from all the remote hosts are preserved for
410 "midnight", and not at "00:00 UTC" (which would be "14:00" in
411 Honolulu). If you want this behaviour, do NOT use "timestamp_format
412 long-iso".
413 • If "timestamp_format long-iso" is set, running btrbk from different
415 time zones leads to different interpretation of "first in day,
416 week, month, or year". Make sure to run btrbk with the same time
417 zone on every host, e.g. by setting the TZ environment variable
418 (see tzset(3)).
419
421 send-receive
422 Backup to a btrfs filesystem, using "btrfs send/receive". This is
423 the recommended (standard) target type. The <target-directory> must
424 be an absolute path and point to a btrfs volume (or subvolume), or
425 to a directory within a subvolume. See btrfs-send(8),
426 btrfs-receive(8).
427 raw *experimental*
429 Backup to a raw (filesystem independent) file from the output of
430 btrfs-send(8), with optional compression and encryption.
431 Note that the target preserve mechanism is currently disabled for
433 incremental raw backups (btrbk does not delete any incremental raw
434 files)!
435 Raw backups consist of two files: the main data file containing the
437 btrfs send stream, and a sidecar file ".info" containing metadata:
438 <snapshot-name>.<timestamp>[_N].btrfs[.gz|.bz2|.xz][.gpg]
440 <snapshot-name>.<timestamp>[_N].btrfs[.gz|.bz2|.xz][.gpg].info
441 For incremental backups ("incremental yes"), please note that:
443 • As soon as a single incremental backup file is lost or
445 corrupted, all later incremental backups become invalid, as
446 there is no common parent for the subsequent incremental images
447 anymore. This might be a good compromise for a vacation backup
448 plan, but for the long term make sure that a non-incremental
449 backup is triggered from time to time.
450 • There is currently no support for rotation of incremental
452 backups: if incremental is set, a full backup must be triggered
453 manually from time to time in order to be able to delete old
454 backups.
455 Additional options for raw targets:
457 raw_target_compress <compress_command>|no
459 Compression algorithm to use for raw backup target. Supported
460 <compress_command>: gzip, pigz, bzip2, pbzip2, xz, lzo, lz4.
461 raw_target_compress_level default|<number>
463 Compression level for the specified <compress_command>.
464 raw_target_compress_threads default|<number>
466 Number of threads to use for <compress_command>.
467 raw_target_split <size>|no
469 Split the raw backup file into pieces of size <size>.
470 raw_target_block_size <number>
472 Block size to use for writing the raw backup file. Defaults to
473 “128K”.
474 raw_target_encrypt gpg|openssl_enc|no
476 If enabled, encrypt the target raw file using gpg or
477 openssl_enc.
478 Additional options for "raw_target_encrypt gpg":
480 gpg_keyring <file>
482 Keyring to use for gpg, e.g. "/etc/btrbk/gpg/pubring.kbx".
483 gpg_recipient <name>
485 Encrypt for user id <name> (email address).
486 Additional options for "raw_target_encrypt openssl_enc" (very
488 experimental):
489 openssl_ciphername <name>
491 Defaults to “aes-256-cbc”.
492 openssl_iv_size <size-in-bytes>|no
494 Depends on selected cipher.
495 openssl_keyfile <file>|no
497 Point to a key file in hex (absolute path). Example key file
498 creation (256bit key):
499 # dd if=/dev/urandom bs=1 count=32 \
501 | od -x -A n \
502 | tr -d "[:space:]" > /path/to/keyfile
503 kdf_backend <file>|no
505 KDF backend to be executed, e.g.
506 "/usr/share/btrbk/scripts/kdf_pbkdf2.py".
507 kdf_keysize <size-in-bytes>
509 Defaults to “32”.
510 kdf_keygen once|each
512 Defaults to “once”.
513
515 Please refer to the btrbk project page https://digint.ch/btrbk/ for
516 further details.
517
519 btrbk(1)
520
522 Axel Burri axel@tty0.ch
523Btrbk 0.28.3 2019-07-28 BTRBK.CONF(5)