1BTRBK.CONF(5)                    Btrbk Manual                    BTRBK.CONF(5)
2
3
4

NAME

6       btrbk.conf - btrbk configuration file
7

SYNOPSIS

9       /etc/btrbk.conf
10       /etc/btrbk/btrbk.conf
11

DESCRIPTION

13       The btrbk configuration file specifies which btrfs subvolumes on the
14       filesystem are to be processed, what target subvolumes should be used
15       to create the backups, and where the snapshots should be generated. The
16       retention policy, as well as most other options can be defined either
17       globally or within a section.
18
19       The options specified always apply to the last section encountered,
20       superseding the values set in upper-level sections. This means that
21       global options must be set before any sections are defined.
22
23       Blank lines are ignored. A hash character (#) starts a comment
24       extending until end of line.
25
26       Whitespace or unicode characters are not allowed for file names.
27       Allowed characters are:
28
29           [0-9] [a-z] [A-Z] and "._+-@"
30
31       This is for sanity/safety/security reasons, we apologize for the
32       inconvenience.
33

SECTIONS

35       volume <volume-directory>|<url>
36           Directory of a btrfs volume containing the source subvolume(s) to
37           be backed up. <volume-directory> must be an absolute path and point
38           to a btrfs volume (or subvolume). Usually the mount point of a
39           btrfs filesystem mounted with the subvolid=5 option.
40
41       subvolume <subvolume-name>
42           Subvolume to be backed up, relative to the <volume-directory>
43           specified in the volume section. Multiple subvolume sections are
44           allowed within volume sections. Accepts wildcard character "*".
45
46           If set to ".", the subvolume at <volume-directory> is used as
47           backup source, and the snapshots will be created within the source
48           subvolume itself (see snapshot_dir option below), which is not
49           recommended. Note that if this subvolume is btrfs root (id=5), it
50           needs to have a valid UUID, which is not the case for file systems
51           created with btrfs-progs < 4.16.
52
53       target [send-receive|raw] <target-directory>|<url>
54           Target directory where the backup subvolumes are to be created. The
55           optional target type defaults to “send-receive”, see TARGET TYPES
56           below for details.
57
58           Multiple target sections are allowed, in any context: a target
59           defined in volume or global context will be used for all underlying
60           subvolume sections (hint: run "btrbk list" or "btrbk config print"
61           to see the resulting configuration).
62
63       If a <url> is specified, btrbk actions (shell commands) are executed
64       remotely via ssh, using the SSH Options described below. Accepted
65       formats are:
66
67           ssh://<hostname>[:<port>]/<directory>
68           <hostname>:<directory>
69
70       If you are connecting to virtual machines, consider configuring several
71       volume sections for a <hostname>, with distinct <port> numbers for each
72       machine.
73

OPTIONS

75       The options described here can be specified in global context as well
76       as volume, subvolume and target sections, unless stated otherwise.
77
78   Basic Options
79       timestamp_format short|long|long-iso
80           Timestamp format used as postfix for new snapshot subvolume names.
81           Defaults to “short”.
82
83           short
84               YYYYMMDD[_N]  (e.g. "20150825", "20150825_1")
85
86           long
87               YYYYMMDD<T>hhmm[_N]  (e.g. "20150825T1531")
88
89           long-iso
90               YYYYMMDD<T>hhmmss&plusmn;hhmm[_N]  (e.g.
91               "20150825T153123+0200")
92
93           Note that a postfix "_N" is appended to the timestamp if a snapshot
94           or backup already exists with the timestamp of current date/time.
95
96           Use “long-iso” if you want to make sure that btrbk never creates
97           ambiguous time stamps (which can happen if multiple snapshots are
98           created during a daylight saving time clock change).
99
100           Note that using “long-iso” has implications on the scheduling, see
101           RETENTION POLICY (caveats) below.
102
103       snapshot_dir <directory>
104           Directory in which the btrfs snapshots are created, relative to
105           <volume-directory> of the volume section. Note that btrbk does not
106           autmatically create this directory, and the snapshot creation will
107           fail if it is not present.
108
109       snapshot_name <basename>
110           Base name of the created snapshot (and backup). This option is only
111           valid in the subvolume section. Defaults to <subvolume-name>.
112
113       snapshot_create always|onchange|ondemand|no
114           If set to “always”, snapshots are always created. If set to
115           “onchange”, snapshots are only created if the source subvolume has
116           changed since the last snapshot (more precisely: if the btrfs
117           generation has been increased since the last snapshot). If set to
118           “ondemand”, snapshots are only created if at least one target
119           subvolume is reachable (useful if you are tight on disk space and
120           you only need btrbk for backups to an external disk which is not
121           always connected). If set to “no”, the snapshots are never created
122           (useful if another instance of btrbk is taking care of snapshot
123           creation). Defaults to “always”.
124
125       incremental yes|no|strict
126           If set, incremental backups are created. If set to “strict”,
127           non-incremental (initial) backups are never created, and
128           incremental backups are restricted to related parents (by
129           parent-uuid relationship). Defaults to “yes”.
130
131           Note that even if the parent-uuid chain is broken, snapshots and
132           backups can still share data (which is especially true for backups
133           created with incremental option enabled), and are perfectly
134           suitable as parents for incremental send-receive operations. But as
135           btrbk can not be certain about this, such operations are disallowed
136           in "incremental strict" mode.
137
138       noauto yes|no
139           If set, the context is skipped by all btrbk actions unless
140           explicitely enabled by a matching btrbk <filter> command line
141           argument (e.g. "btrbk run myfilter").
142
143   Grouping Options
144       group <group-name> [<group-name>]...
145           Add the current section (volume, subvolume or target) to
146           user-defined groups, which can be used as filter for most btrbk
147           commands. This option can be set multiple times within the same
148           context.
149
150   Retention Policy Options
151       preserve_day_of_week monday|tuesday|...|sunday
152           Defines on what day a snapshot/backup is considered to be a
153           "weekly" backup. Weekly, monthly and yearly backups are preserved
154           on this day of week (see RETENTION POLICY below). Defaults to
155           “sunday”.
156
157       preserve_hour_of_day [0..23]
158           Defines after what time (in full hours since midnight) a
159           snapshot/backup is considered to be a "daily" backup. Daily,
160           weekly, monthly and yearly backups are preserved on this hour (see
161           RETENTION POLICY below). If you set this option, make sure to also
162           set timestamp_format to “long” or “long-iso” (backups and snapshots
163           having no time information will ignore this option). Defaults to
164           “0”.
165
166       snapshot_preserve no|<retention_policy>
167           Set retention policy for snapshots (see RETENTION POLICY below). If
168           set to “no”, preserve snapshots according to snapshot_preserve_min
169           only. Defaults to “no”.
170
171       snapshot_preserve_min all|latest|<number>{h,d,w,m,y}
172           Preserve all snapshots for a minimum amount of hours (h), days (d),
173           weeks (w), months (m) or years (y), regardless of how many there
174           are. If set to “all”, preserve all snapshots forever. If set to
175           “latest”, preserve latest snapshot. Defaults to “all”.
176
177       target_preserve no|<retention_policy>
178           Set retention policy for backups (see RETENTION POLICY below). If
179           set to “no”, preserve backups according to target_preserve_min
180           only. Defaults to “no”.
181
182       target_preserve_min  all|latest|no|<number>{h,d,w,m,y}
183           Preserve all backups for a minimum amount of hours (h), days (d),
184           weeks (w), months (m) or years (y), regardless of how many there
185           are. If set to “all”, preserve all backups forever. If set to
186           “latest”, always preserve the latest backup (useful in conjunction
187           with "target_preserve no", if you want to keep the latest backup
188           only). If set to “no”, only the backups following the
189           target_preserve policy are created. Defaults to “all”.
190
191       archive_preserve no|<retention_policy>
192           Set retention policy for archives ("btrbk archive" command), with
193           same semantics as target_preserve.
194
195       archive_preserve_min all|latest|no|<number>{h,d,w,m,y}
196           Set retention policy for archives ("btrbk archive" command), with
197           same semantics as target_preserve_min.
198
199       archive_exclude <pattern>
200           Exclude subvolumes matching <pattern> from archiving. The pattern
201           accepts wildcard character "*", and is matched against the end of
202           the pathname.
203
204   SSH Options
205       ssh_identity <file>
206           Absolute path to a ssh identity file (private key). Note that if
207           the private key is password protected, btrbk will prompt for user
208           input, which is usually not desired.
209
210       ssh_user <username>
211           Remote username for ssh. Defaults to “root”. Make sure the remote
212           user is able to run "btrfs" with root privileges (see option
213           backend for details).
214
215       ssh_compression yes|no
216           Enables or disables the compression of ssh connections. Defaults to
217           “no”.
218
219       ssh_cipher_spec <cipher_spec>
220           Selects the cipher specification for encrypting the session
221           (comma-separated list of ciphers in order of preference). See the
222           "-c cipher_spec" option in ssh(1) for more information. Defaults to
223           “default” (the ciphers specified in ssh_config).
224
225       Previous versions btrbk allowed you to set a ssh_port option, this has
226       been dropped in favor of the ssh://hostname:port notation in the volume
227       and target sections. If you want to set a global port for all SSH
228       connections to remote hosts, set the “Port” option in ssh_config(5).
229
230   Data Stream Options
231       stream_compress <compress_command>|no
232           Compress the btrfs send stream before transferring it from/to
233           remote locations. Defaults to “no”. If enabled, make sure that
234           <compress_command> is available on the source and target hosts.
235           Supported <compress_command>: gzip, pigz, bzip2, pbzip2, xz, lzo,
236           lz4.
237
238       stream_compress_level default|<number>
239           Compression level for the specified <compress_command>. Refer to
240           the related man-page for details (usually [1..9], where 1 means
241           fastest compression). Defaults to “default” (the default
242           compression level of <compress_command>).
243
244       stream_compress_threads default|<number>
245           Number of threads to use for <compress_command>. Only supported for
246           "pigz", "pbzip2" and recent versions of "xz".
247
248       stream_buffer <size>|no
249           Add a buffer to the btrfs send stream (in front of "btrfs
250           receive"), with a maximum size of <size>. This can give a speed
251           improvement (measured up to 20%) on both local or remote
252           operations, but also increases system load. A suffix of "k", "m",
253           "g", or "%" can be added to <size> to denote kilobytes (*1024),
254           megabytes, gigabytes, or a percentage of total physical memory.
255           Defaults to “no”.
256
257           If enabled, make sure that the "mbuffer" command is available on
258           the target host. Note that versions of mbuffer < 20180505 suffered
259           from nasty bugs (infinite loops, hangs); if your main concern is a
260           stable backup process, leave this option disabled.
261
262       rate_limit <rate>|no
263           Limit the transfer to a maximum of <rate> bytes per second. A
264           suffix of "k", "m", "g", or "t" can be added to denote kilobytes
265           (*1024), megabytes, and so on. Defaults to “no”. If enabled for
266           remote sources, make sure that the "pv" command is available on the
267           source host.
268
269   System Options
270       transaction_log <file>|no
271           If set, all transactions (snapshot create, subvolume send-receive,
272           subvolume delete) as well as abort messages are logged to <file>,
273           in a space-separated table format: "localtime type status
274           target_url source_url parent_url message".
275
276       transaction_syslog  <facility>|no
277           If set, all transactions (as described in transaction_log above)
278           are logged to syslog. The program name used in the messages is
279           "btrbk". Accepted parameters for <facility>: user, mail, daemon,
280           auth, lpr, news, cron, authpriv, local0..local7.
281
282       lockfile <file>|no
283           Create lockfile <file> on startup; checks lockfile before running
284           any btrfs commands (using perl "flock"), and exits if the lock is
285           held by another btrbk instance. Ignored on dryrun (-n, --dry-run).
286           See also --lockfile command-line option.
287
288       backend btrfs-progs|btrfs-progs-btrbk|btrfs-progs-sudo
289           Backend filesystem utilities to be used for btrfs specific
290           operations. Defaults to “btrfs-progs”.
291
292           btrfs-progs
293               Default backend, btrfs commands are called as specified in
294               btrfs(8) (e.g. "btrfs subvolume show").
295
296           btrfs-progs-btrbk
297               btrfs commands are separated by a dash instead of a whitespace
298               (e.g. "btrfs-subvolume-show" instead of "btrfs subvolume
299               show"). Useful for setting suid or file capabilities (setcap)
300               on specific btrfs commands, as implemented in
301               https://github.com/digint/btrfs-progs-btrbk.
302
303           btrfs-progs-sudo
304               btrfs commands are prefixed with "sudo -n" (e.g. "sudo -n btrfs
305               subvolume show" instead of "btrfs subvolume show"). Make sure
306               to have appropriate (root) permissions for the "btrfs" command
307               groups and the "readlink" command in /etc/sudoers.
308
309           For convenience, it is also possible to set backend_local or
310           backend_remote options, which will override the backend only for
311           local or remote sources/targets (e.g. "backend_remote
312           btrfs-progs-btrbk").
313
314   Btrfs Specific Options
315       btrfs_commit_delete after|each|no
316           If set, make sure the deletion of snapshot and backup subvolumes
317           are committed to disk when btrbk terminates. Defaults to “no”.
318
319       incremental_clones <number>
320           Maximum number of clone sources allowed for incremental send. If
321           set, btrbk adds "-c <clone-src>" to the btrfs-send(8) command for
322           all present snapshot/backup pairs (correlated subvolumes matching
323           matching received_uuid, printed by "btrbk stats"). Set this to a
324           high number if you want to make sure that no common data is missed
325           on incremental backups, in expense of btrfs-send performance.
326           Defaults to 0.
327
328       incremental_resolve mountpoint|directory
329           Specifies where to search for the best common parent for
330           incremental backups. If set to “mountpoint”, use parents in the
331           filesystem tree below mount points of source
332           "<volume-directory>/<snapshot-dir>" and target
333           "<target-directory>". If set to “directory”, use parents strictly
334           below source/target directories. Set this to “directory” if you get
335           access problems (when not running btrbk as root). Defaults to
336           “mountpoint”.
337
338       snapshot_qgroup_destroy yes|no  *experimental*
339
340
341       target_qgroup_destroy yes|no  *experimental*
342
343
344       archive_qgroup_destroy yes|no  *experimental*
345           Whenever a subvolume is deleted, also destroy corresponding default
346           qgroup "0/<subvol-id>". Only useful if you have enabled btrfs quota
347           support. See also:
348           https://bugzilla.kernel.org/show_bug.cgi?id=91751
349

RETENTION POLICY

351       btrbk uses separate retention policies for snapshots and backups, which
352       are defined by the snapshot_preserve_min, snapshot_preserve,
353       target_preserve_min, target_preserve, preserve_day_of_week and
354       preserve_hour_of_day configuration options.
355
356       Within this section, any statement about "backups" is always valid for
357       backups as well as snapshots, referring to target_preserve or
358       snapshot_preserve respectively.
359
360       The format for <retention_policy> is:
361
362           [<hourly>h] [<daily>d] [<weekly>w] [<monthly>m] [<yearly>y]
363
364       With the following semantics:
365
366       hourly
367           Defines how many hours back hourly backups should be preserved. The
368           first backup of an hour is considered an hourly backup. Note that
369           if you use <hourly> scheduling, make sure to also set
370           timestamp_format to “long” or “long-iso”, or the scheduler will
371           interpret the time as "00:00" (midnight).
372
373       daily
374           Defines how many days back daily backups should be preserved. The
375           first backup of a day (starting at preserve_hour_of_day) is
376           considered a daily backup.
377
378       weekly
379           Defines how many weeks back weekly backups should be preserved. The
380           first daily backup created at preserve_day_of_week (or the first
381           backup in this week if none was made on the exact day) is
382           considered as a weekly backup.
383
384       monthly
385           Defines how many months back monthly backups should be preserved.
386           Every first weekly backup in a month is considered a monthly
387           backup.
388
389       yearly
390           Defines for how many years back yearly backups should be preserved.
391           Every first monthly backup in a year is considered a yearly backup.
392
393       Use an asterisk for “all” (e.g. "target_preserve 60d *m" states:
394       "preserve daily backups for 60 days back, and all monthly backups").
395
396       The reference time (which defines the beginning of a day, week, month
397       or year) for all date/time calculations is the local time of the host
398       running btrbk.
399
400       Hint: Run btrbk with the -S, --print-schedule option to get a
401       comprehensive output of the scheduler results.
402
403       Caveats:
404
405       ·   If you run a setup with several btrbk instances (e.g. one
406           snapshot-only instance per remote client, and a separate fetch-only
407           instance on the backup server), it makes perfectly sense to run
408           btrbk with different local time on the clients, in order to make
409           sure the backups from all the remote hosts are preserved for
410           "midnight", and not at "00:00 UTC" (which would be "14:00" in
411           Honolulu). If you want this behaviour, do NOT use "timestamp_format
412           long-iso".
413
414       ·   If "timestamp_format long-iso" is set, running btrbk from different
415           time zones leads to different interpretation of "first in day,
416           week, month, or year". Make sure to run btrbk with the same time
417           zone on every host, e.g. by setting the TZ environment variable
418           (see tzset(3)).
419

TARGET TYPES

421       send-receive
422           Backup to a btrfs filesystem, using "btrfs send/receive". This is
423           the recommended (standard) target type. The <target-directory> must
424           be an absolute path and point to a btrfs volume (or subvolume), or
425           to a directory within a subvolume. See btrfs-send(8),
426           btrfs-receive(8).
427
428       raw  *experimental*
429           Backup to a raw (filesystem independent) file from the output of
430           btrfs-send(8), with optional compression and encryption.
431
432           Note that the target preserve mechanism is currently disabled for
433           incremental raw backups (btrbk does not delete any incremental raw
434           files)!
435
436           Raw backups consist of two files: the main data file containing the
437           btrfs send stream, and a sidecar file ".info" containing metadata:
438
439               <snapshot-name>.<timestamp>[_N].btrfs[.gz|.bz2|.xz][.gpg]
440               <snapshot-name>.<timestamp>[_N].btrfs[.gz|.bz2|.xz][.gpg].info
441
442           For incremental backups ("incremental yes"), please note that:
443
444           ·   As soon as a single incremental backup file is lost or
445               corrupted, all later incremental backups become invalid, as
446               there is no common parent for the subsequent incremental images
447               anymore. This might be a good compromise for a vacation backup
448               plan, but for the long term make sure that a non-incremental
449               backup is triggered from time to time.
450
451           ·   There is currently no support for rotation of incremental
452               backups: if incremental is set, a full backup must be triggered
453               manually from time to time in order to be able to delete old
454               backups.
455
456           Additional options for raw targets:
457
458           raw_target_compress <compress_command>|no
459               Compression algorithm to use for raw backup target. Supported
460               <compress_command>: gzip, pigz, bzip2, pbzip2, xz, lzo, lz4.
461
462           raw_target_compress_level default|<number>
463               Compression level for the specified <compress_command>.
464
465           raw_target_compress_threads default|<number>
466               Number of threads to use for <compress_command>.
467
468           raw_target_split <size>|no
469               Split the raw backup file into pieces of size <size>.
470
471           raw_target_block_size <number>
472               Block size to use for writing the raw backup file. Defaults to
473               “128K”.
474
475           raw_target_encrypt gpg|openssl_enc|no
476               If enabled, encrypt the target raw file using gpg or
477               openssl_enc.
478
479           Additional options for "raw_target_encrypt gpg":
480
481           gpg_keyring <file>
482               Keyring to use for gpg, e.g. "/etc/btrbk/gpg/pubring.kbx".
483
484           gpg_recipient <name>
485               Encrypt for user id <name> (email address).
486
487           Additional options for "raw_target_encrypt openssl_enc" (very
488           experimental):
489
490           openssl_ciphername <name>
491               Defaults to “aes-256-cbc”.
492
493           openssl_iv_size <size-in-bytes>|no
494               Depends on selected cipher.
495
496           openssl_keyfile <file>|no
497               Point to a key file in hex (absolute path). Example key file
498               creation (256bit key):
499
500                     # dd if=/dev/urandom bs=1 count=32 \
501                       | od -x -A n \
502                       | tr -d "[:space:]" > /path/to/keyfile
503
504           kdf_backend <file>|no
505               KDF backend to be executed, e.g.
506               "/usr/share/btrbk/scripts/kdf_pbkdf2.py".
507
508           kdf_keysize <size-in-bytes>
509               Defaults to “32”.
510
511           kdf_keygen once|each
512               Defaults to “once”.
513

AVAILABILITY

515       Please refer to the btrbk project page https://digint.ch/btrbk/ for
516       further details.
517

SEE ALSO

519       btrbk(1)
520

AUTHOR

522       Axel Burri <axel@tty0.ch>
523
524
525
526Btrbk 0.28.3                      2019-07-28                     BTRBK.CONF(5)
Impressum