1BORG(1) Borg - Deduplicating Archiver BORG(1)
2
6 borg - BorgBackup is a deduplicating backup program with optional com‐
7 pression and authenticated encryption.
8
10 Borg consists of a number of commands. Each command accepts a number of
11 arguments and options and interprets various environment variables.
12 The following sections will describe each command in detail.
13 Commands, options, parameters, paths and such are set in fixed-width.
15 Option values are underlined. Borg has few options accepting a fixed
16 set of values (e.g. --encryption of borg init). Experimental features
17 are marked with red stripes on the sides, like this paragraph.
18 Experimental features are not stable, which means that they may be
20 changed in incompatible ways or even removed entirely without prior no‐
21 tice in following releases.
22 Positional Arguments and Options: Order matters
24 Borg only supports taking options (-s and --progress in the example) to
25 the left or right of all positional arguments (repo::archive and path
26 in the example), but not in between them:
27 borg create -s --progress repo::archive path # good and preferred
29 borg create repo::archive path -s --progress # also works
30 borg create -s repo::archive path --progress # works, but ugly
31 borg create repo::archive -s --progress path # BAD
32 This is due to a problem in the argparse module:
34 https://bugs.python.org/issue15112
35 Repository URLs
37 Local filesystem (or locally mounted network filesystem):
38 /path/to/repo - filesystem path to repo directory, absolute path
40 path/to/repo - filesystem path to repo directory, relative path
42 Also, stuff like ~/path/to/repo or ~other/path/to/repo works (this is
44 expanded by your shell).
45 Note: you may also prepend a file:// to a filesystem path to get URL
47 style.
48 Remote repositories accessed via ssh user@host:
50 user@host:/path/to/repo - remote repo, absolute path
52 ssh://user@host:port/path/to/repo - same, alternative syntax, port can
54 be given
55 Remote repositories with relative paths can be given using this syntax:
57 user@host:path/to/repo - path relative to current directory
59 user@host:~/path/to/repo - path relative to user's home directory
61 user@host:~other/path/to/repo - path relative to other's home directory
63 Note: giving user@host:/./path/to/repo or user@host:/~/path/to/repo or
65 user@host:/~other/path/to/repo is also supported, but not required
66 here.
67 Remote repositories with relative paths, alternative syntax with port:
69 ssh://user@host:port/./path/to/repo - path relative to current direc‐
71 tory
72 ssh://user@host:port/~/path/to/repo - path relative to user's home di‐
74 rectory
75 ssh://user@host:port/~other/path/to/repo - path relative to other's
77 home directory
78 If you frequently need the same repo URL, it is a good idea to set the
80 BORG_REPO environment variable to set a default for the repo URL:
81 export BORG_REPO='ssh://user@host:port/path/to/repo'
83 Then just leave away the repo URL if only a repo URL is needed and you
85 want to use the default - it will be read from BORG_REPO then.
86 Use :: syntax to give the repo URL when syntax requires giving a posi‐
88 tional argument for the repo (e.g. borg mount :: /mnt).
89 Repository / Archive Locations
91 Many commands want either a repository (just give the repo URL, see
92 above) or an archive location, which is a repo URL followed by ::ar‐
93 chive_name.
94 Archive names must not contain the / (slash) character. For simplicity,
96 maybe also avoid blanks or other characters that have special meaning
97 on the shell or in a filesystem (borg mount will use the archive name
98 as directory name).
99 If you have set BORG_REPO (see above) and an archive location is
101 needed, use ::archive_name - the repo URL part is then read from
102 BORG_REPO.
103 Logging
105 Borg writes all log output to stderr by default. But please note that
106 something showing up on stderr does not indicate an error condition
107 just because it is on stderr. Please check the log levels of the mes‐
108 sages and the return code of borg for determining error, warning or
109 success conditions.
110 If you want to capture the log output to a file, just redirect it:
112 borg create repo::archive myfiles 2>> logfile
114 Custom logging configurations can be implemented via BORG_LOGGING_CONF.
116 The log level of the builtin logging configuration defaults to WARNING.
118 This is because we want Borg to be mostly silent and only output warn‐
119 ings, errors and critical messages, unless output has been requested by
120 supplying an option that implies output (e.g. --list or --progress).
121 Log levels: DEBUG < INFO < WARNING < ERROR < CRITICAL
123 Use --debug to set DEBUG log level - to get debug, info, warning, error
125 and critical level output.
126 Use --info (or -v or --verbose) to set INFO log level - to get info,
128 warning, error and critical level output.
129 Use --warning (default) to set WARNING log level - to get warning, er‐
131 ror and critical level output.
132 Use --error to set ERROR log level - to get error and critical level
134 output.
135 Use --critical to set CRITICAL log level - to get critical level out‐
137 put.
138 While you can set misc. log levels, do not expect that every command
140 will give different output on different log levels - it's just a possi‐
141 bility.
142 WARNING:
144 Options --critical and --error are provided for completeness, their
145 usage is not recommended as you might miss important information.
146 Return codes
148 Borg can exit with the following return codes (rc):
149 ┌────────────┬────────────────────────────┐
151 │Return code │ Meaning │
152 ├────────────┼────────────────────────────┤
153 │0 │ success (logged as INFO) │
154 ├────────────┼────────────────────────────┤
155 │1 │ warning (operation reached │
156 │ │ its normal end, but there │
157 │ │ were warnings -- you │
158 │ │ should check the log, │
159 │ │ logged as WARNING) │
160 ├────────────┼────────────────────────────┤
161 │2 │ error (like a fatal error, │
162 │ │ a local or remote excep‐ │
163 │ │ tion, the operation did │
164 │ │ not reach its normal end, │
165 │ │ logged as ERROR) │
166 ├────────────┼────────────────────────────┤
167 │128+N │ killed by signal N (e.g. │
168 │ │ 137 == kill -9) │
169 └────────────┴────────────────────────────┘
170 If you use --show-rc, the return code is also logged at the indicated
172 level as the last log entry.
173 Environment Variables
175 Borg uses some environment variables for automation:
176 General:
178 BORG_REPO
180 When set, use the value to give the default repository
181 location. If a command needs an archive parameter, you
182 can abbreviate as ::archive. If a command needs a reposi‐
183 tory parameter, you can either leave it away or abbrevi‐
184 ate as ::, if a positional parameter is required.
185 BORG_PASSPHRASE
187 When set, use the value to answer the passphrase question
188 for encrypted repositories. It is used when a passphrase
189 is needed to access an encrypted repo as well as when a
190 new passphrase should be initially set when initializing
191 an encrypted repo. See also BORG_NEW_PASSPHRASE.
192 BORG_PASSCOMMAND
194 When set, use the standard output of the command (trail‐
195 ing newlines are stripped) to answer the passphrase ques‐
196 tion for encrypted repositories. It is used when a
197 passphrase is needed to access an encrypted repo as well
198 as when a new passphrase should be initially set when
199 initializing an encrypted repo. Note that the command is
200 executed without a shell. So variables, like $HOME will
201 work, but ~ won't. If BORG_PASSPHRASE is also set, it
202 takes precedence. See also BORG_NEW_PASSPHRASE.
203 BORG_PASSPHRASE_FD
205 When set, specifies a file descriptor to read a
206 passphrase from. Programs starting borg may choose to
207 open an anonymous pipe and use it to pass a passphrase.
208 This is safer than passing via BORG_PASSPHRASE, because
209 on some systems (e.g. Linux) environment can be examined
210 by other processes. If BORG_PASSPHRASE or BORG_PASSCOM‐
211 MAND are also set, they take precedence.
212 BORG_NEW_PASSPHRASE
214 When set, use the value to answer the passphrase question
215 when a new passphrase is asked for. This variable is
216 checked first. If it is not set, BORG_PASSPHRASE and
217 BORG_PASSCOMMAND will also be checked. Main usecase for
218 this is to fully automate borg change-passphrase.
219 BORG_DISPLAY_PASSPHRASE
221 When set, use the value to answer the "display the
222 passphrase for verification" question when defining a new
223 passphrase for encrypted repositories.
224 BORG_HOST_ID
226 Borg usually computes a host id from the FQDN plus the
227 results of uuid.getnode() (which usually returns a unique
228 id based on the MAC address of the network interface. Ex‐
229 cept if that MAC happens to be all-zero - in that case it
230 returns a random value, which is not what we want (be‐
231 cause it kills automatic stale lock removal). So, if you
232 have a all-zero MAC address or other reasons to better
233 externally control the host id, just set this environment
234 variable to a unique value. If all your FQDNs are unique,
235 you can just use the FQDN. If not, use fqdn@uniqueid.
236 BORG_LOGGING_CONF
238 When set, use the given filename as INI-style logging
239 configuration. A basic example conf can be found at
240 docs/misc/logging.conf.
241 BORG_RSH
243 When set, use this command instead of ssh. This can be
244 used to specify ssh options, such as a custom identity
245 file ssh -i /path/to/private/key. See man ssh for other
246 options. Using the --rsh CMD commandline option overrides
247 the environment variable.
248 BORG_REMOTE_PATH
250 When set, use the given path as borg executable on the
251 remote (defaults to "borg" if unset). Using --re‐
252 mote-path PATH commandline option overrides the environ‐
253 ment variable.
254 BORG_FILES_CACHE_SUFFIX
256 When set to a value at least one character long, in‐
257 structs borg to use a specifically named (based on the
258 suffix) alternative files cache. This can be used to
259 avoid loading and saving cache entries for backup sources
260 other than the current sources.
261 BORG_FILES_CACHE_TTL
263 When set to a numeric value, this determines the maximum
264 "time to live" for the files cache entries (default: 20).
265 The files cache is used to quickly determine whether a
266 file is unchanged. The FAQ explains this more detailed
267 in: It always chunks all my files, even unchanged ones!
268 BORG_SHOW_SYSINFO
270 When set to no (default: yes), system information (like
271 OS, Python version, ...) in exceptions is not shown.
272 Please only use for good reasons as it makes issues
273 harder to analyze.
274 BORG_FUSE_IMPL
276 Choose the lowlevel FUSE implementation borg shall use
277 for borg mount. This is a comma-separated list of imple‐
278 mentation names, they are tried in the given order, e.g.:
279 • pyfuse3,llfuse: default, first try to load pyfuse3,
281 then try to load llfuse.
282 • llfuse,pyfuse3: first try to load llfuse, then try to
284 load pyfuse3.
285 • pyfuse3: only try to load pyfuse3
287 • llfuse: only try to load llfuse
289 • none: do not try to load an implementation
291 BORG_SELFTEST
293 This can be used to influence borg's builtin self-tests.
294 The default is to execute the tests at the beginning of
295 each borg command invocation.
296 BORG_SELFTEST=disabled can be used to switch off the
298 tests and rather save some time. Disabling is not recom‐
299 mended for normal borg users, but large scale borg stor‐
300 age providers can use this to optimize production servers
301 after at least doing a one-time test borg (with selftests
302 not disabled) when installing or upgrading machines / OS
303 / borg.
304 BORG_WORKAROUNDS
306 A list of comma separated strings that trigger work‐
307 arounds in borg, e.g. to work around bugs in other soft‐
308 ware.
309 Currently known strings are:
311 basesyncfile
313 Use the more simple BaseSyncFile code to avoid is‐
314 sues with sync_file_range. You might need this to
315 run borg on WSL (Windows Subsystem for Linux) or
316 in systemd.nspawn containers on some architectures
317 (e.g. ARM). Using this does not affect data
318 safety, but might result in a more bursty write to
319 disk behaviour (not continuously streaming to
320 disk).
321 retry_erofs
323 Retry opening a file without O_NOATIME if opening
324 a file with O_NOATIME caused EROFS. You will need
325 this to make archives from volume shadow copies in
326 WSL1 (Windows Subsystem for Linux 1).
327 Some automatic "answerers" (if set, they automatically answer confirma‐
329 tion questions):
330 BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK=no (or =yes)
332 For "Warning: Attempting to access a previously unknown
333 unencrypted repository"
334 BORG_RELOCATED_REPO_ACCESS_IS_OK=no (or =yes)
336 For "Warning: The repository at location ... was previ‐
337 ously located at ..."
338 BORG_CHECK_I_KNOW_WHAT_I_AM_DOING=NO (or =YES)
340 For "This is a potentially dangerous function..." (check
341 --repair)
342 BORG_DELETE_I_KNOW_WHAT_I_AM_DOING=NO (or =YES)
344 For "You requested to completely DELETE the repository
345 including all archives it contains:"
346 Note: answers are case sensitive. setting an invalid answer
348 value might either give the default answer or ask you interac‐
349 tively, depending on whether retries are allowed (they by de‐
350 fault are allowed). So please test your scripts interactively
351 before making them a non-interactive script.
352 Directories and files:
354 BORG_BASE_DIR
356 Defaults to $HOME or ~$USER or ~ (in that order). If you
357 want to move all borg-specific folders to a custom path
358 at once, all you need to do is to modify BORG_BASE_DIR:
359 the other paths for cache, config etc. will adapt accord‐
360 ingly (assuming you didn't set them to a different custom
361 value).
362 BORG_CACHE_DIR
364 Defaults to $BORG_BASE_DIR/.cache/borg. If BORG_BASE_DIR
365 is not explicitly set while XDG env var XDG_CACHE_HOME is
366 set, then $XDG_CACHE_HOME/borg is being used instead.
367 This directory contains the local cache and might need a
368 lot of space for dealing with big repositories. Make sure
369 you're aware of the associated security aspects of the
370 cache location: Do I need to take security precautions
371 regarding the cache?
372 BORG_CONFIG_DIR
374 Defaults to $BORG_BASE_DIR/.config/borg. If BORG_BASE_DIR
375 is not explicitly set while XDG env var XDG_CONFIG_HOME
376 is set, then $XDG_CONFIG_HOME/borg is being used instead.
377 This directory contains all borg configuration directo‐
378 ries, see the FAQ for a security advisory about the data
379 in this directory: How important is the $HOME/.con‐
380 fig/borg directory?
381 BORG_SECURITY_DIR
383 Defaults to $BORG_CONFIG_DIR/security. This directory
384 contains information borg uses to track its usage of
385 NONCES ("numbers used once" - usually in encryption con‐
386 text) and other security relevant data.
387 BORG_KEYS_DIR
389 Defaults to $BORG_CONFIG_DIR/keys. This directory con‐
390 tains keys for encrypted repositories.
391 BORG_KEY_FILE
393 When set, use the given filename as repository key file.
394 TMPDIR This is where temporary files are stored (might need a
396 lot of temporary space for some operations), see tempfile
397 for details.
398 Building:
400 BORG_OPENSSL_PREFIX
402 Adds given OpenSSL header file directory to the default
403 locations (setup.py).
404 BORG_LIBLZ4_PREFIX
406 Adds given prefix directory to the default locations. If
407 a 'include/lz4.h' is found Borg will be linked against
408 the system liblz4 instead of a bundled implementation.
409 (setup.py)
410 BORG_LIBB2_PREFIX
412 Adds given prefix directory to the default locations. If
413 a 'include/blake2.h' is found Borg will be linked against
414 the system libb2 instead of a bundled implementation.
415 (setup.py)
416 BORG_LIBZSTD_PREFIX
418 Adds given prefix directory to the default locations. If
419 a 'include/zstd.h' is found Borg will be linked against
420 the system libzstd instead of a bundled implementation.
421 (setup.py)
422 Please note:
424 • Be very careful when using the "yes" sayers, the warnings with prompt
426 exist for your / your data's security/safety.
427 • Also be very careful when putting your passphrase into a script, make
429 sure it has appropriate file permissions (e.g. mode 600, root:root).
430 File systems
432 We strongly recommend against using Borg (or any other database-like
433 software) on non-journaling file systems like FAT, since it is not pos‐
434 sible to assume any consistency in case of power failures (or a sudden
435 disconnect of an external drive or similar failures).
436 While Borg uses a data store that is resilient against these failures
438 when used on journaling file systems, it is not possible to guarantee
439 this with some hardware -- independent of the software used. We don't
440 know a list of affected hardware.
441 If you are suspicious whether your Borg repository is still consistent
443 and readable after one of the failures mentioned above occurred, run
444 borg check --verify-data to make sure it is consistent.
445 Requirements for Borg repository file systems
447 • Long file names
449 • At least three directory levels with short names
451 • Typically, file sizes up to a few hundred MB. Large repositories may
453 require large files (>2 GB).
454 • Up to 1000 files per directory (10000 for repositories initialized
456 with Borg 1.0)
457 • rename(2) / MoveFile(Ex) should work as specified, i.e. on the same
459 file system it should be a move (not a copy) operation, and in case
460 of a directory it should fail if the destination exists and is not an
461 empty directory, since this is used for locking.
462 • Hardlinks are needed for borg upgrade (if --inplace option is not
464 used). Also hardlinks are used for more safe and secure file updat‐
465 ing (e.g. of the repo config file), but the code tries to work also
466 if hardlinks are not supported.
467 Units
469 To display quantities, Borg takes care of respecting the usual conven‐
470 tions of scale. Disk sizes are displayed in decimal, using powers of
471 ten (so kB means 1000 bytes). For memory usage, binary prefixes are
472 used, and are indicated using the IEC binary prefixes, using powers of
473 two (so KiB means 1024 bytes).
474 Date and Time
476 We format date and time conforming to ISO-8601, that is: YYYY-MM-DD and
477 HH:MM:SS (24h clock).
478 For more information about that, see: https://xkcd.com/1179/
480 Unless otherwise noted, we display local date and time. Internally, we
482 store and process date and time as UTC.
483 Resource Usage
485 Borg might use a lot of resources depending on the size of the data set
486 it is dealing with.
487 If one uses Borg in a client/server way (with a ssh: repository), the
489 resource usage occurs in part on the client and in another part on the
490 server.
491 If one uses Borg as a single process (with a filesystem repo), all the
493 resource usage occurs in that one process, so just add up client +
494 server to get the approximate resource usage.
495 CPU client:
497 • borg create: does chunking, hashing, compression, crypto (high
499 CPU usage)
500 • chunks cache sync: quite heavy on CPU, doing lots of hashtable
502 operations.
503 • borg extract: crypto, decompression (medium to high CPU usage)
505 • borg check: similar to extract, but depends on options given.
507 • borg prune / borg delete archive: low to medium CPU usage
509 • borg delete repo: done on the server
511 It won't go beyond 100% of 1 core as the code is currently sin‐
513 gle-threaded. Especially higher zlib and lzma compression lev‐
514 els use significant amounts of CPU cycles. Crypto might be cheap
515 on the CPU (if hardware accelerated) or expensive (if not).
516 CPU server:
518 It usually doesn't need much CPU, it just deals with the
519 key/value store (repository) and uses the repository index for
520 that.
521 borg check: the repository check computes the checksums of all
523 chunks (medium CPU usage) borg delete repo: low CPU usage
524 CPU (only for client/server operation):
526 When using borg in a client/server way with a ssh:-type repo,
527 the ssh processes used for the transport layer will need some
528 CPU on the client and on the server due to the crypto they are
529 doing - esp. if you are pumping big amounts of data.
530 Memory (RAM) client:
532 The chunks index and the files index are read into memory for
533 performance reasons. Might need big amounts of memory (see be‐
534 low). Compression, esp. lzma compression with high levels might
535 need substantial amounts of memory.
536 Memory (RAM) server:
538 The server process will load the repository index into memory.
539 Might need considerable amounts of memory, but less than on the
540 client (see below).
541 Chunks index (client only):
543 Proportional to the amount of data chunks in your repo. Lots of
544 chunks in your repo imply a big chunks index. It is possible to
545 tweak the chunker params (see create options).
546 Files index (client only):
548 Proportional to the amount of files in your last backups. Can be
549 switched off (see create options), but next backup might be much
550 slower if you do. The speed benefit of using the files cache is
551 proportional to file size.
552 Repository index (server only):
554 Proportional to the amount of data chunks in your repo. Lots of
555 chunks in your repo imply a big repository index. It is possi‐
556 ble to tweak the chunker params (see create options) to influ‐
557 ence the amount of chunks being created.
558 Temporary files (client):
560 Reading data and metadata from a FUSE mounted repository will
561 consume up to the size of all deduplicated, small chunks in the
562 repository. Big chunks won't be locally cached.
563 Temporary files (server):
565 A non-trivial amount of data will be stored on the remote temp
566 directory for each client that connects to it. For some remotes,
567 this can fill the default temporary directory at /tmp. This can
568 be remediated by ensuring the $TMPDIR, $TEMP, or $TMP environ‐
569 ment variable is properly set for the sshd process. For some
570 OSes, this can be done just by setting the correct value in the
571 .bashrc (or equivalent login config file for other shells), how‐
572 ever in other cases it may be necessary to first enable Permi‐
573 tUserEnvironment yes in your sshd_config file, then add environ‐
574 ment="TMPDIR=/my/big/tmpdir" at the start of the public key to
575 be used in the authorized_hosts file.
576 Cache files (client only):
578 Contains the chunks index and files index (plus a collection of
579 single- archive chunk indexes which might need huge amounts of
580 disk space, depending on archive count and size - see FAQ about
581 how to reduce).
582 Network (only for client/server operation):
584 If your repository is remote, all deduplicated (and optionally
585 compressed/ encrypted) data of course has to go over the connec‐
586 tion (ssh:// repo url). If you use a locally mounted network
587 filesystem, additionally some copy operations used for transac‐
588 tion support also go over the connection. If you backup multiple
589 sources to one target repository, additional traffic happens for
590 cache resynchronization.
591 Support for file metadata
593 Besides regular file and directory structures, Borg can preserve
594 • symlinks (stored as symlink, the symlink is not followed)
596 • special files:
598 • character and block device files (restored via mknod)
600 • FIFOs ("named pipes")
602 • special file contents can be backed up in --read-special mode. By
604 default the metadata to create them with mknod(2), mkfifo(2) etc.
605 is stored.
606 • hardlinked regular files, devices, FIFOs (considering all items in
608 the same archive)
609 • timestamps in nanosecond precision: mtime, atime, ctime
611 • other timestamps: birthtime (on platforms supporting it)
613 • permissions:
615 • IDs of owning user and owning group
617 • names of owning user and owning group (if the IDs can be resolved)
619 • Unix Mode/Permissions (u/g/o permissions, suid, sgid, sticky)
621 On some platforms additional features are supported:
623 ┌─────────────────┬──────────┬───────────┬───────────┐
625 │Platform │ ACLs [5] │ xattr [6] │ Flags [7] │
626 ├─────────────────┼──────────┼───────────┼───────────┤
627 │Linux │ Yes │ Yes │ Yes [1] │
628 ├─────────────────┼──────────┼───────────┼───────────┤
629 │Mac OS X │ Yes │ Yes │ Yes (all) │
630 ├─────────────────┼──────────┼───────────┼───────────┤
631 │FreeBSD │ Yes │ Yes │ Yes (all) │
632 ├─────────────────┼──────────┼───────────┼───────────┤
633 │OpenBSD │ n/a │ n/a │ Yes (all) │
634 ├─────────────────┼──────────┼───────────┼───────────┤
635 │NetBSD │ n/a │ No [2] │ Yes (all) │
636 ├─────────────────┼──────────┼───────────┼───────────┤
637 │Solaris and de‐ │ No [3] │ No [3] │ n/a │
638 │rivatives │ │ │ │
639 ├─────────────────┼──────────┼───────────┼───────────┤
640 │Windows (cygwin) │ No [4] │ No │ No │
641 └─────────────────┴──────────┴───────────┴───────────┘
642 Other Unix-like operating systems may work as well, but have not been
644 tested at all.
645 Note that most of the platform-dependent features also depend on the
647 file system. For example, ntfs-3g on Linux isn't able to convey NTFS
648 ACLs.
649 [1] Only "nodump", "immutable", "compressed" and "append" are sup‐
651 ported. Feature request #618 for more flags.
652 [2] Feature request #1332
654 [3] Feature request #1337
656 [4] Cygwin tries to map NTFS ACLs to permissions with varying degrees
658 of success.
659 [5] The native access control list mechanism of the OS. This normally
661 limits access to non-native ACLs. For example, NTFS ACLs aren't
662 completely accessible on Linux with ntfs-3g.
663 [6] extended attributes; key-value pairs attached to a file, mainly
665 used by the OS. This includes resource forks on Mac OS X.
666 [7] aka BSD flags. The Linux set of flags [1] is portable across plat‐
668 forms. The BSDs define additional flags.
669 In case you are interested in more details (like formulas), please
671 see Internals. For details on the available JSON output, refer to
672 All about JSON: How to develop frontends.
673 Common options
675 All Borg commands share these options:
676 -h, --help
678 show this help message and exit
679 --critical
681 work on log level CRITICAL
682 --error
684 work on log level ERROR
685 --warning
687 work on log level WARNING (default)
688 --info, -v, --verbose
690 work on log level INFO
691 --debug
693 enable debug output, work on log level DEBUG
694 --debug-topic TOPIC
696 enable TOPIC debugging (can be specified multiple times). The
697 logger path is borg.debug.<TOPIC> if TOPIC is not fully quali‐
698 fied.
699 -p, --progress
701 show progress information
702 --iec format using IEC units (1KiB = 1024B)
704 --log-json
706 Output one JSON object per log line instead of formatted text.
707 --lock-wait SECONDS
709 wait at most SECONDS for acquiring a repository/cache lock (de‐
710 fault: 1).
711 --bypass-lock
713 Bypass locking mechanism
714 --show-version
716 show/log the borg version
717 --show-rc
719 show/log the return code (rc)
720 --umask M
722 set umask to M (local only, default: 0077)
723 --remote-path PATH
725 use PATH as borg executable on the remote (default: "borg")
726 --remote-ratelimit RATE
728 deprecated, use --upload-ratelimit instead
729 --upload-ratelimit RATE
731 set network upload rate limit in kiByte/s (default: 0=unlimited)
732 --remote-buffer UPLOAD_BUFFER
734 deprecated, use --upload-buffer instead
735 --upload-buffer UPLOAD_BUFFER
737 set network upload buffer size in MiB. (default: 0=no buffer)
738 --consider-part-files
740 treat part files like normal files (e.g. to list/extract them)
741 --debug-profile FILE
743 Write execution profile in Borg format into FILE. For local use
744 a Python-compatible file can be generated by suffixing FILE with
745 ".pyprof".
746 --rsh RSH
748 Use this command to connect to the 'borg serve' process (de‐
749 fault: 'ssh')
750 Option --bypass-lock allows you to access the repository while bypass‐
752 ing borg's locking mechanism. This is necessary if your repository is
753 on a read-only storage where you don't have write permissions or capa‐
754 bilities and therefore cannot create a lock. Examples are repositories
755 stored on a Bluray disc or a read-only network storage. Avoid this op‐
756 tion if you are able to use locks as that is the safer way; see the
757 warning below.
758 WARNING:
760 If you do use --bypass-lock, you are responsible to ensure that no
761 other borg instances have write access to the repository. Otherwise,
762 you might experience errors and read broken data if changes to that
763 repository are being made at the same time.
764 Examples
766 # Create an archive and log: borg version, files list, return code
767 $ borg create --show-version --list --show-rc /path/to/repo::my-files files
768
770 borg [common options] init [options] [REPOSITORY]
771 Description
773 This command initializes an empty repository. A repository is a
774 filesystem directory containing the deduplicated data from zero or more
775 archives.
776 Encryption mode TLDR
778 The encryption mode can only be configured when creating a new reposi‐
779 tory - you can neither configure it on a per-archive basis nor change
780 the encryption mode of an existing repository.
781 Use repokey:
783 borg init --encryption repokey /path/to/repo
785 Or repokey-blake2 depending on which is faster on your client machines
787 (see below):
788 borg init --encryption repokey-blake2 /path/to/repo
790 Borg will:
792 1. Ask you to come up with a passphrase.
794 2. Create a borg key (which contains 3 random secrets. See Key files).
796 3. Encrypt the key with your passphrase.
798 4. Store the encrypted borg key inside the repository directory (in the
800 repo config). This is why it is essential to use a secure
801 passphrase.
802 5. Encrypt and sign your backups to prevent anyone from reading or
804 forging them unless they have the key and know the passphrase. Make
805 sure to keep a backup of your key outside the repository - do not
806 lock yourself out by "leaving your keys inside your car" (see borg
807 key export). For remote backups the encryption is done locally -
808 the remote machine never sees your passphrase, your unencrypted key
809 or your unencrypted files. Chunking and id generation are also
810 based on your key to improve your privacy.
811 6. Use the key when extracting files to decrypt them and to verify that
813 the contents of the backups have not been accidentally or mali‐
814 ciously altered.
815 Picking a passphrase
817 Make sure you use a good passphrase. Not too short, not too simple. The
818 real encryption / decryption key is encrypted with / locked by your
819 passphrase. If an attacker gets your key, he can't unlock and use it
820 without knowing the passphrase.
821 Be careful with special or non-ascii characters in your passphrase:
823 • Borg processes the passphrase as unicode (and encodes it as utf-8),
825 so it does not have problems dealing with even the strangest charac‐
826 ters.
827 • BUT: that does not necessarily apply to your OS / VM / keyboard con‐
829 figuration.
830 So better use a long passphrase made from simple ascii chars than one
832 that includes non-ascii stuff or characters that are hard/impossible to
833 enter on a different keyboard layout.
834 You can change your passphrase for existing repos at any time, it won't
836 affect the encryption/decryption key or other secrets.
837 More encryption modes
839 Only use --encryption none if you are OK with anyone who has access to
840 your repository being able to read your backups and tamper with their
841 contents without you noticing.
842 If you want "passphrase and having-the-key" security, use --encryption
844 keyfile. The key will be stored in your home directory (in ~/.con‐
845 fig/borg/keys).
846 If you do not want to encrypt the contents of your backups, but still
848 want to detect malicious tampering use --encryption authenticated.
849 If BLAKE2b is faster than SHA-256 on your hardware, use --encryption
851 authenticated-blake2, --encryption repokey-blake2 or --encryption key‐
852 file-blake2. Note: for remote backups the hashing is done on your local
853 machine.
854 ┌─────────┬──────────────────┬──────────────────┬──────────────────┐
856 │Hash/MAC │ Not encrypted no │ Not encrypted, │ Encrypted (AEAD │
857 │ │ auth │ but authenti‐ │ w/ AES) and au‐ │
858 │ │ │ cated │ thenticated │
859 ├─────────┼──────────────────┼──────────────────┼──────────────────┤
860 │SHA-256 │ none │ authenticated │ repokey keyfile │
861 ├─────────┼──────────────────┼──────────────────┼──────────────────┤
862 │BLAKE2b │ n/a │ authenti‐ │ repokey-blake2 │
863 │ │ │ cated-blake2 │ keyfile-blake2 │
864 └─────────┴──────────────────┴──────────────────┴──────────────────┘
865 Modes marked like this in the above table are new in Borg 1.1 and are
867 not backwards-compatible with Borg 1.0.x.
868 On modern Intel/AMD CPUs (except very cheap ones), AES is usually hard‐
870 ware-accelerated. BLAKE2b is faster than SHA256 on Intel/AMD 64-bit
871 CPUs (except AMD Ryzen and future CPUs with SHA extensions), which
872 makes authenticated-blake2 faster than none and authenticated.
873 On modern ARM CPUs, NEON provides hardware acceleration for SHA256 mak‐
875 ing it faster than BLAKE2b-256 there. NEON accelerates AES as well.
876 Hardware acceleration is always used automatically when available.
878 repokey and keyfile use AES-CTR-256 for encryption and HMAC-SHA256 for
880 authentication in an encrypt-then-MAC (EtM) construction. The chunk ID
881 hash is HMAC-SHA256 as well (with a separate key). These modes are
882 compatible with Borg 1.0.x.
883 repokey-blake2 and keyfile-blake2 are also authenticated encryption
885 modes, but use BLAKE2b-256 instead of HMAC-SHA256 for authentication.
886 The chunk ID hash is a keyed BLAKE2b-256 hash. These modes are new and
887 not compatible with Borg 1.0.x.
888 authenticated mode uses no encryption, but authenticates repository
890 contents through the same HMAC-SHA256 hash as the repokey and keyfile
891 modes (it uses it as the chunk ID hash). The key is stored like re‐
892 pokey. This mode is new and not compatible with Borg 1.0.x.
893 authenticated-blake2 is like authenticated, but uses the keyed
895 BLAKE2b-256 hash from the other blake2 modes. This mode is new and not
896 compatible with Borg 1.0.x.
897 none mode uses no encryption and no authentication. It uses SHA256 as
899 chunk ID hash. This mode is not recommended, you should rather consider
900 using an authenticated or authenticated/encrypted mode. This mode has
901 possible denial-of-service issues when running borg create on contents
902 controlled by an attacker. Use it only for new repositories where no
903 encryption is wanted and when compatibility with 1.0.x is important. If
904 compatibility with 1.0.x is not important, use authenticated-blake2 or
905 authenticated instead. This mode is compatible with Borg 1.0.x.
906 Examples
908 # Local repository, repokey encryption, BLAKE2b (often faster, since Borg 1.1)
909 $ borg init --encryption=repokey-blake2 /path/to/repo
910 # Local repository (no encryption)
912 $ borg init --encryption=none /path/to/repo
913 # Remote repository (accesses a remote borg via ssh)
915 # repokey: stores the (encrypted) key into <REPO_DIR>/config
916 $ borg init --encryption=repokey-blake2 user@hostname:backup
917 # Remote repository (accesses a remote borg via ssh)
919 # keyfile: stores the (encrypted) key into ~/.config/borg/keys/
920 $ borg init --encryption=keyfile user@hostname:backup
921
923 borg [common options] create [options] ARCHIVE [PATH...]
924 Description
926 This command creates a backup archive containing all files found while
927 recursively traversing all paths specified. Paths are added to the ar‐
928 chive as they are given, that means if relative paths are desired, the
929 command has to be run from the correct directory.
930 When giving '-' as path, borg will read data from standard input and
932 create a file 'stdin' in the created archive from that data. In some
933 cases it's more appropriate to use --content-from-command, however. See
934 section Reading from stdin below for details.
935 The archive will consume almost no disk space for files or parts of
937 files that have already been stored in other archives.
938 The archive name needs to be unique. It must not end in '.checkpoint'
940 or '.checkpoint.N' (with N being a number), because these names are
941 used for checkpoints and treated in special ways.
942 In the archive name, you may use the following placeholders: {now},
944 {utcnow}, {fqdn}, {hostname}, {user} and some others.
945 Backup speed is increased by not reprocessing files that are already
947 part of existing archives and weren't modified. The detection of unmod‐
948 ified files is done by comparing multiple file metadata values with
949 previous values kept in the files cache.
950 This comparison can operate in different modes as given by
952 --files-cache:
953 • ctime,size,inode (default)
955 • mtime,size,inode (default behaviour of borg versions older than
957 1.1.0rc4)
958 • ctime,size (ignore the inode number)
960 • mtime,size (ignore the inode number)
962 • rechunk,ctime (all files are considered modified - rechunk, cache
964 ctime)
965 • rechunk,mtime (all files are considered modified - rechunk, cache
967 mtime)
968 • disabled (disable the files cache, all files considered modified -
970 rechunk)
971 inode number: better safety, but often unstable on network filesystems
973 Normally, detecting file modifications will take inode information into
975 consideration to improve the reliability of file change detection.
976 This is problematic for files located on sshfs and similar network file
977 systems which do not provide stable inode numbers, such files will al‐
978 ways be considered modified. You can use modes without inode in this
979 case to improve performance, but reliability of change detection might
980 be reduced.
981 ctime vs. mtime: safety vs. speed
983 • ctime is a rather safe way to detect changes to a file (metadata and
985 contents) as it can not be set from userspace. But, a metadata-only
986 change will already update the ctime, so there might be some unneces‐
987 sary chunking/hashing even without content changes. Some filesystems
988 do not support ctime (change time). E.g. doing a chown or chmod to a
989 file will change its ctime.
990 • mtime usually works and only updates if file contents were changed.
992 But mtime can be arbitrarily set from userspace, e.g. to set mtime
993 back to the same value it had before a content change happened. This
994 can be used maliciously as well as well-meant, but in both cases
995 mtime based cache modes can be problematic.
996 The mount points of filesystems or filesystem snapshots should be the
998 same for every creation of a new archive to ensure fast operation. This
999 is because the file cache that is used to determine changed files
1000 quickly uses absolute filenames. If this is not possible, consider
1001 creating a bind mount to a stable location.
1002 The --progress option shows (from left to right) Original, Compressed
1004 and Deduplicated (O, C and D, respectively), then the Number of files
1005 (N) processed so far, followed by the currently processed path.
1006 When using --stats, you will get some statistics about how much data
1008 was added - the "This Archive" deduplicated size there is most inter‐
1009 esting as that is how much your repository will grow. Please note that
1010 the "All archives" stats refer to the state after creation. Also, the
1011 --stats and --dry-run options are mutually exclusive because the data
1012 is not actually compressed and deduplicated during a dry run.
1013 For more help on include/exclude patterns, see the borg help patterns
1015 command output.
1016 For more help on placeholders, see the borg help placeholders command
1018 output.
1019 The --exclude patterns are not like tar. In tar --exclude .bundler/gems
1021 will exclude foo/.bundler/gems. In borg it will not, you need to use
1022 --exclude '*/.bundler/gems' to get the same effect.
1023 In addition to using --exclude patterns, it is possible to use --ex‐
1025 clude-if-present to specify the name of a filesystem object (e.g. a
1026 file or folder name) which, when contained within another folder, will
1027 prevent the containing folder from being backed up. By default, the
1028 containing folder and all of its contents will be omitted from the
1029 backup. If, however, you wish to only include the objects specified by
1030 --exclude-if-present in your backup, and not include any other contents
1031 of the containing folder, this can be enabled through using the
1032 --keep-exclude-tags option.
1033 The -x or --one-file-system option excludes directories, that are
1035 mountpoints (and everything in them). It detects mountpoints by com‐
1036 paring the device number from the output of stat() of the directory and
1037 its parent directory. Specifically, it excludes directories for which
1038 stat() reports a device number different from the device number of
1039 their parent. Be aware that in Linux (and possibly elsewhere) there are
1040 directories with device number different from their parent, which the
1041 kernel does not consider a mountpoint and also the other way around.
1042 Examples are bind mounts (possibly same device number, but always a
1043 mountpoint) and ALL subvolumes of a btrfs (different device number from
1044 parent but not necessarily a mountpoint). Therefore when using
1045 --one-file-system, one should make doubly sure that the backup works as
1046 intended especially when using btrfs. This is even more important, if
1047 the btrfs layout was created by someone else, e.g. a distribution in‐
1048 staller.
1049 Item flags
1051 --list outputs a list of all files, directories and other file system
1052 items it considered (no matter whether they had content changes or
1053 not). For each item, it prefixes a single-letter flag that indicates
1054 type and/or status of the item.
1055 If you are interested only in a subset of that output, you can give
1057 e.g. --filter=AME and it will only show regular files with A, M or E
1058 status (see below).
1059 A uppercase character represents the status of a regular file relative
1061 to the "files" cache (not relative to the repo -- this is an issue if
1062 the files cache is not used). Metadata is stored in any case and for
1063 'A' and 'M' also new data chunks are stored. For 'U' all data chunks
1064 refer to already existing chunks.
1065 • 'A' = regular file, added (see also I am seeing 'A' (added) status
1067 for an unchanged file!? in the FAQ)
1068 • 'M' = regular file, modified
1070 • 'U' = regular file, unchanged
1072 • 'C' = regular file, it changed while we backed it up
1074 • 'E' = regular file, an error happened while accessing/reading this
1076 file
1077 A lowercase character means a file type other than a regular file, borg
1079 usually just stores their metadata:
1080 • 'd' = directory
1082 • 'b' = block device
1084 • 'c' = char device
1086 • 'h' = regular file, hardlink (to already seen inodes)
1088 • 's' = symlink
1090 • 'f' = fifo
1092 Other flags used include:
1094 • 'i' = backup data was read from standard input (stdin)
1096 • '-' = dry run, item was not backed up
1098 • 'x' = excluded, item was not backed up
1100 • '?' = missing status code (if you see this, please file a bug re‐
1102 port!)
1103 Reading from stdin
1105 There are two methods to read from stdin. Either specify - as path and
1106 pipe directly to borg:
1107 backup-vm --id myvm --stdout | borg create REPO::ARCHIVE -
1109 Or use --content-from-command to have Borg manage the execution of the
1111 command and piping. If you do so, the first PATH argument is inter‐
1112 preted as command to execute and any further arguments are treated as
1113 arguments to the command:
1114 borg create --content-from-command REPO::ARCHIVE -- backup-vm --id myvm --stdout
1116 -- is used to ensure --id and --stdout are not considered arguments to
1118 borg but rather backup-vm.
1119 The difference between the two approaches is that piping to borg cre‐
1121 ates an archive even if the command piping to borg exits with a fail‐
1122 ure. In this case, one can end up with truncated output being backed
1123 up. Using --content-from-command, in contrast, borg is guaranteed to
1124 fail without creating an archive should the command fail. The command
1125 is considered failed when it returned a non-zero exit code.
1126 Reading from stdin yields just a stream of data without file metadata
1128 associated with it, and the files cache is not needed at all. So it is
1129 safe to disable it via --files-cache disabled and speed up backup cre‐
1130 ation a bit.
1131 By default, the content read from stdin is stored in a file called
1133 'stdin'. Use --stdin-name to change the name.
1134 Examples
1136 # Backup ~/Documents into an archive named "my-documents"
1137 $ borg create /path/to/repo::my-documents ~/Documents
1138 # same, but list all files as we process them
1140 $ borg create --list /path/to/repo::my-documents ~/Documents
1141 # Backup ~/Documents and ~/src but exclude pyc files
1143 $ borg create /path/to/repo::my-files \
1144 ~/Documents \
1145 ~/src \
1146 --exclude '*.pyc'
1147 # Backup home directories excluding image thumbnails (i.e. only
1149 # /home/<one directory>/.thumbnails is excluded, not /home/*/*/.thumbnails etc.)
1150 $ borg create /path/to/repo::my-files /home \
1151 --exclude 'sh:home/*/.thumbnails'
1152 # Backup the root filesystem into an archive named "root-YYYY-MM-DD"
1154 # use zlib compression (good, but slow) - default is lz4 (fast, low compression ratio)
1155 $ borg create -C zlib,6 --one-file-system /path/to/repo::root-{now:%Y-%m-%d} /
1156 # Backup onto a remote host ("push" style) via ssh to port 2222,
1158 # logging in as user "borg" and storing into /path/to/repo
1159 $ borg create ssh://borg@backup.example.org:2222/path/to/repo::{fqdn}-root-{now} /
1160 # Backup a remote host locally ("pull" style) using sshfs
1162 $ mkdir sshfs-mount
1163 $ sshfs root@example.com:/ sshfs-mount
1164 $ cd sshfs-mount
1165 $ borg create /path/to/repo::example.com-root-{now:%Y-%m-%d} .
1166 $ cd ..
1167 $ fusermount -u sshfs-mount
1168 # Make a big effort in fine granular deduplication (big chunk management
1170 # overhead, needs a lot of RAM and disk space, see formula in internals
1171 # docs - same parameters as borg < 1.0 or attic):
1172 $ borg create --chunker-params buzhash,10,23,16,4095 /path/to/repo::small /smallstuff
1173 # Backup a raw device (must not be active/in use/mounted at that time)
1175 $ borg create --read-special --chunker-params fixed,4194304 /path/to/repo::my-sdx /dev/sdX
1176 # Backup a sparse disk image (must not be active/in use/mounted at that time)
1178 $ borg create --sparse --chunker-params fixed,4194304 /path/to/repo::my-disk my-disk.raw
1179 # No compression (none)
1181 $ borg create --compression none /path/to/repo::arch ~
1182 # Super fast, low compression (lz4, default)
1184 $ borg create /path/to/repo::arch ~
1185 # Less fast, higher compression (zlib, N = 0..9)
1187 $ borg create --compression zlib,N /path/to/repo::arch ~
1188 # Even slower, even higher compression (lzma, N = 0..9)
1190 $ borg create --compression lzma,N /path/to/repo::arch ~
1191 # Only compress compressible data with lzma,N (N = 0..9)
1193 $ borg create --compression auto,lzma,N /path/to/repo::arch ~
1194 # Use short hostname, user name and current time in archive name
1196 $ borg create /path/to/repo::{hostname}-{user}-{now} ~
1197 # Similar, use the same datetime format that is default as of borg 1.1
1198 $ borg create /path/to/repo::{hostname}-{user}-{now:%Y-%m-%dT%H:%M:%S} ~
1199 # As above, but add nanoseconds
1200 $ borg create /path/to/repo::{hostname}-{user}-{now:%Y-%m-%dT%H:%M:%S.%f} ~
1201 # Backing up relative paths by moving into the correct directory first
1203 $ cd /home/user/Documents
1204 # The root directory of the archive will be "projectA"
1205 $ borg create /path/to/repo::daily-projectA-{now:%Y-%m-%d} projectA
1206 # Use external command to determine files to archive
1208 # Use --paths-from-stdin with find to only backup files less than 1MB in size
1209 $ find ~ -size -1000k | borg create --paths-from-stdin /path/to/repo::small-files-only
1210 # Use --paths-from-command with find to only backup files from a given user
1211 $ borg create --paths-from-command /path/to/repo::joes-files -- find /srv/samba/shared -user joe
1212 # Use --paths-from-stdin with --paths-delimiter (for example, for filenames with newlines in them)
1213 $ find ~ -size -1000k -print0 | borg create \
1214 --paths-from-stdin \
1215 --paths-delimiter "\0" \
1216 /path/to/repo::smallfiles-handle-newline
1217
1219 borg [common options] extract [options] ARCHIVE [PATH...]
1220 Description
1222 This command extracts the contents of an archive. By default the entire
1223 archive is extracted but a subset of files and directories can be se‐
1224 lected by passing a list of PATHs as arguments. The file selection can
1225 further be restricted by using the --exclude option.
1226 For more help on include/exclude patterns, see the borg help patterns
1228 command output.
1229 By using --dry-run, you can do all extraction steps except actually
1231 writing the output data: reading metadata and data chunks from the
1232 repo, checking the hash/hmac, decrypting, decompressing.
1233 --progress can be slower than no progress display, since it makes one
1235 additional pass over the archive metadata.
1236 NOTE:
1238 Currently, extract always writes into the current working directory
1239 ("."), so make sure you cd to the right place before calling borg
1240 extract.
1241 When parent directories are not extracted (because of using file/di‐
1243 rectory selection or any other reason), borg can not restore parent
1244 directories' metadata, e.g. owner, group, permission, etc.
1245 Examples
1247 # Extract entire archive
1248 $ borg extract /path/to/repo::my-files
1249 # Extract entire archive and list files while processing
1251 $ borg extract --list /path/to/repo::my-files
1252 # Verify whether an archive could be successfully extracted, but do not write files to disk
1254 $ borg extract --dry-run /path/to/repo::my-files
1255 # Extract the "src" directory
1257 $ borg extract /path/to/repo::my-files home/USERNAME/src
1258 # Extract the "src" directory but exclude object files
1260 $ borg extract /path/to/repo::my-files home/USERNAME/src --exclude '*.o'
1261 # Restore a raw device (must not be active/in use/mounted at that time)
1263 $ borg extract --stdout /path/to/repo::my-sdx | dd of=/dev/sdx bs=10M
1264
1266 borg [common options] check [options] [REPOSITORY_OR_ARCHIVE]
1267 Description
1269 The check command verifies the consistency of a repository and the cor‐
1270 responding archives.
1271 check --repair is a potentially dangerous function and might lead to
1273 data loss (for kinds of corruption it is not capable of dealing with).
1274 BE VERY CAREFUL!
1275 Pursuant to the previous warning it is also highly recommended to test
1277 the reliability of the hardware running this software with stress test‐
1278 ing software such as memory testers. Unreliable hardware can also lead
1279 to data loss especially when this command is run in repair mode.
1280 First, the underlying repository data files are checked:
1282 • For all segments, the segment magic header is checked.
1284 • For all objects stored in the segments, all metadata (e.g. CRC and
1286 size) and all data is read. The read data is checked by size and CRC.
1287 Bit rot and other types of accidental damage can be detected this
1288 way.
1289 • In repair mode, if an integrity error is detected in a segment, try
1291 to recover as many objects from the segment as possible.
1292 • In repair mode, make sure that the index is consistent with the data
1294 stored in the segments.
1295 • If checking a remote repo via ssh:, the repo check is executed on the
1297 server without causing significant network traffic.
1298 • The repository check can be skipped using the --archives-only option.
1300 • A repository check can be time consuming. Partial checks are possible
1302 with the --max-duration option.
1303 Second, the consistency and correctness of the archive metadata is ver‐
1305 ified:
1306 • Is the repo manifest present? If not, it is rebuilt from archive
1308 metadata chunks (this requires reading and decrypting of all metadata
1309 and data).
1310 • Check if archive metadata chunk is present; if not, remove archive
1312 from manifest.
1313 • For all files (items) in the archive, for all chunks referenced by
1315 these files, check if chunk is present. In repair mode, if a chunk is
1316 not present, replace it with a same-size replacement chunk of zeroes.
1317 If a previously lost chunk reappears (e.g. via a later backup), in
1318 repair mode the all-zero replacement chunk will be replaced by the
1319 correct chunk. This requires reading of archive and file metadata,
1320 but not data.
1321 • In repair mode, when all the archives were checked, orphaned chunks
1323 are deleted from the repo. One cause of orphaned chunks are input
1324 file related errors (like read errors) in the archive creation
1325 process.
1326 • In verify-data mode, a complete cryptographic verification of the ar‐
1328 chive data integrity is performed. This conflicts with --reposi‐
1329 tory-only as this mode only makes sense if the archive checks are en‐
1330 abled. The full details of this mode are documented below.
1331 • If checking a remote repo via ssh:, the archive check is executed on
1333 the client machine because it requires decryption, and this is always
1334 done client-side as key access is needed.
1335 • The archive checks can be time consuming; they can be skipped using
1337 the --repository-only option.
1338 The --max-duration option can be used to split a long-running reposi‐
1340 tory check into multiple partial checks. After the given number of sec‐
1341 onds the check is interrupted. The next partial check will continue
1342 where the previous one stopped, until the complete repository has been
1343 checked. Example: Assuming a complete check took 7 hours, then running
1344 a daily check with --max-duration=3600 (1 hour) resulted in one com‐
1345 pleted check per week.
1346 Attention: A partial --repository-only check can only do way less
1348 checking than a full --repository-only check: only the non-crypto‐
1349 graphic checksum checks on segment file entries are done, while a full
1350 --repository-only check would also do a repo index check. A partial
1351 check cannot be combined with the --repair option. Partial checks may
1352 therefore be useful only with very large repositories where a full
1353 check would take too long. Doing a full repository check aborts a par‐
1354 tial check; the next partial check will restart from the beginning.
1355 The --verify-data option will perform a full integrity verification (as
1357 opposed to checking the CRC32 of the segment) of data, which means
1358 reading the data from the repository, decrypting and decompressing it.
1359 This is a cryptographic verification, which will detect (accidental)
1360 corruption. For encrypted repositories it is tamper-resistant as well,
1361 unless the attacker has access to the keys. It is also very slow.
1362
1364 borg [common options] rename [options] ARCHIVE NEWNAME
1365 Description
1367 This command renames an archive in the repository.
1368 This results in a different archive ID.
1370 Examples
1372 $ borg create /path/to/repo::archivename ~
1373 $ borg list /path/to/repo
1374 archivename Mon, 2016-02-15 19:50:19
1375 $ borg rename /path/to/repo::archivename newname
1377 $ borg list /path/to/repo
1378 newname Mon, 2016-02-15 19:50:19
1379
1381 borg [common options] list [options] [REPOSITORY_OR_ARCHIVE] [PATH...]
1382 Description
1384 This command lists the contents of a repository or an archive.
1385 For more help on include/exclude patterns, see the borg help patterns
1387 command output.
1388 The FORMAT specifier syntax
1390 The --format option uses python's format string syntax.
1391 Examples:
1393 $ borg list --format '{archive}{NL}' /path/to/repo
1395 ArchiveFoo
1396 ArchiveBar
1397 ...
1398 # {VAR:NUMBER} - pad to NUMBER columns.
1400 # Strings are left-aligned, numbers are right-aligned.
1401 # Note: time columns except ``isomtime``, ``isoctime`` and ``isoatime`` cannot be padded.
1402 $ borg list --format '{archive:36} {time} [{id}]{NL}' /path/to/repo
1403 ArchiveFoo Thu, 2021-12-09 10:22:28 [0b8e9a312bef3f2f6e2d0fc110c196827786c15eba0188738e81697a7fa3b274]
1404 $ borg list --format '{mode} {user:6} {group:6} {size:8} {mtime} {path}{extra}{NL}' /path/to/repo::ArchiveFoo
1405 -rw-rw-r-- user user 1024 Thu, 2021-12-09 10:22:17 file-foo
1406 ...
1407 # {VAR:<NUMBER} - pad to NUMBER columns left-aligned.
1409 # {VAR:>NUMBER} - pad to NUMBER columns right-aligned.
1410 $ borg list --format '{mode} {user:>6} {group:>6} {size:<8} {mtime} {path}{extra}{NL}' /path/to/repo::ArchiveFoo
1411 -rw-rw-r-- user user 1024 Thu, 2021-12-09 10:22:17 file-foo
1412 ...
1413 The following keys are always available:
1415 • NEWLINE: OS dependent line separator
1417 • NL: alias of NEWLINE
1419 • NUL: NUL character for creating print0 / xargs -0 like output, see
1421 barchive and bpath keys below
1422 • SPACE
1424 • TAB
1426 • CR
1428 • LF
1430 Keys available only when listing archives in a repository:
1432 • archive: archive name interpreted as text (might be missing non-text
1434 characters, see barchive)
1435 • name: alias of "archive"
1437 • barchive: verbatim archive name, can contain any character except NUL
1439 • comment: archive comment interpreted as text (might be missing
1441 non-text characters, see bcomment)
1442 • bcomment: verbatim archive comment, can contain any character except
1444 NUL
1445 • id: internal ID of the archive
1447 • start: time (start) of creation of the archive
1449 • time: alias of "start"
1451 • end: time (end) of creation of the archive
1453 • command_line: command line which was used to create the archive
1455 • hostname: hostname of host on which this archive was created
1457 • username: username of user who created this archive
1459 Keys available only when listing files in an archive:
1461 • type
1463 • mode
1465 • uid
1467 • gid
1469 • user
1471 • group
1473 • path: path interpreted as text (might be missing non-text characters,
1475 see bpath)
1476 • bpath: verbatim POSIX path, can contain any character except NUL
1478 • source: link target for links (identical to linktarget)
1480 • linktarget
1482 • flags
1484 • size
1486 • csize: compressed size
1488 • dsize: deduplicated size
1490 • dcsize: deduplicated compressed size
1492 • num_chunks: number of chunks in this file
1494 • unique_chunks: number of unique chunks in this file
1496 • mtime
1498 • ctime
1500 • atime
1502 • isomtime
1504 • isoctime
1506 • isoatime
1508 • blake2b
1510 • blake2s
1512 • md5
1514 • sha1
1516 • sha224
1518 • sha256
1520 • sha384
1522 • sha3_224
1524 • sha3_256
1526 • sha3_384
1528 • sha3_512
1530 • sha512
1532 • xxh64: XXH64 checksum of this file (note: this is NOT a cryptographic
1534 hash!)
1535 • archiveid
1537 • archivename
1539 • extra: prepends {source} with " -> " for soft links and " link to "
1541 for hard links
1542 • health: either "healthy" (file ok) or "broken" (if file has all-zero
1544 replacement chunks)
1545 Examples
1547 $ borg list /path/to/repo
1548 Monday Mon, 2016-02-15 19:15:11
1549 repo Mon, 2016-02-15 19:26:54
1550 root-2016-02-15 Mon, 2016-02-15 19:36:29
1551 newname Mon, 2016-02-15 19:50:19
1552 ...
1553 $ borg list /path/to/repo::root-2016-02-15
1555 drwxr-xr-x root root 0 Mon, 2016-02-15 17:44:27 .
1556 drwxrwxr-x root root 0 Mon, 2016-02-15 19:04:49 bin
1557 -rwxr-xr-x root root 1029624 Thu, 2014-11-13 00:08:51 bin/bash
1558 lrwxrwxrwx root root 0 Fri, 2015-03-27 20:24:26 bin/bzcmp -> bzdiff
1559 -rwxr-xr-x root root 2140 Fri, 2015-03-27 20:24:22 bin/bzdiff
1560 ...
1561 $ borg list /path/to/repo::root-2016-02-15 --pattern "- bin/ba*"
1563 drwxr-xr-x root root 0 Mon, 2016-02-15 17:44:27 .
1564 drwxrwxr-x root root 0 Mon, 2016-02-15 19:04:49 bin
1565 lrwxrwxrwx root root 0 Fri, 2015-03-27 20:24:26 bin/bzcmp -> bzdiff
1566 -rwxr-xr-x root root 2140 Fri, 2015-03-27 20:24:22 bin/bzdiff
1567 ...
1568 $ borg list /path/to/repo::archiveA --format="{mode} {user:6} {group:6} {size:8d} {isomtime} {path}{extra}{NEWLINE}"
1570 drwxrwxr-x user user 0 Sun, 2015-02-01 11:00:00 .
1571 drwxrwxr-x user user 0 Sun, 2015-02-01 11:00:00 code
1572 drwxrwxr-x user user 0 Sun, 2015-02-01 11:00:00 code/myproject
1573 -rw-rw-r-- user user 1416192 Sun, 2015-02-01 11:00:00 code/myproject/file.ext
1574 -rw-rw-r-- user user 1416192 Sun, 2015-02-01 11:00:00 code/myproject/file.text
1575 ...
1576 $ borg list /path/to/repo/::archiveA --pattern 're:\.ext$'
1578 -rw-rw-r-- user user 1416192 Sun, 2015-02-01 11:00:00 code/myproject/file.ext
1579 ...
1580 $ borg list /path/to/repo/::archiveA --pattern 're:.ext$'
1582 -rw-rw-r-- user user 1416192 Sun, 2015-02-01 11:00:00 code/myproject/file.ext
1583 -rw-rw-r-- user user 1416192 Sun, 2015-02-01 11:00:00 code/myproject/file.text
1584 ...
1585
1587 borg [common options] diff [options] REPO::ARCHIVE1 ARCHIVE2 [PATH...]
1588 Description
1590 This command finds differences (file contents, user/group/mode) between
1591 archives.
1592 A repository location and an archive name must be specified for
1594 REPO::ARCHIVE1. ARCHIVE2 is just another archive name in same reposi‐
1595 tory (no repository location allowed).
1596 For archives created with Borg 1.1 or newer diff automatically detects
1598 whether the archives are created with the same chunker params. If so,
1599 only chunk IDs are compared, which is very fast.
1600 For archives prior to Borg 1.1 chunk contents are compared by default.
1602 If you did not create the archives with different chunker params, pass
1603 --same-chunker-params. Note that the chunker params changed from Borg
1604 0.xx to 1.0.
1605 For more help on include/exclude patterns, see the borg help patterns
1607 command output.
1608 Examples
1610 $ borg init -e=none testrepo
1611 $ mkdir testdir
1612 $ cd testdir
1613 $ echo asdf > file1
1614 $ dd if=/dev/urandom bs=1M count=4 > file2
1615 $ touch file3
1616 $ borg create ../testrepo::archive1 .
1617 $ chmod a+x file1
1619 $ echo "something" >> file2
1620 $ borg create ../testrepo::archive2 .
1621 $ echo "testing 123" >> file1
1623 $ rm file3
1624 $ touch file4
1625 $ borg create ../testrepo::archive3 .
1626 $ cd ..
1628 $ borg diff testrepo::archive1 archive2
1629 [-rw-r--r-- -> -rwxr-xr-x] file1
1630 +135 B -252 B file2
1631 $ borg diff testrepo::archive2 archive3
1633 +17 B -5 B file1
1634 added 0 B file4
1635 removed 0 B file3
1636 $ borg diff testrepo::archive1 archive3
1638 +17 B -5 B [-rw-r--r-- -> -rwxr-xr-x] file1
1639 +135 B -252 B file2
1640 added 0 B file4
1641 removed 0 B file3
1642 $ borg diff --json-lines testrepo::archive1 archive3
1644 {"path": "file1", "changes": [{"type": "modified", "added": 17, "removed": 5}, {"type": "mode", "old_mode": "-rw-r--r--", "new_mode": "-rwxr-xr-x"}]}
1645 {"path": "file2", "changes": [{"type": "modified", "added": 135, "removed": 252}]}
1646 {"path": "file4", "changes": [{"type": "added", "size": 0}]}
1647 {"path": "file3", "changes": [{"type": "removed", "size": 0}]}
1648
1650 borg [common options] delete [options] [REPOSITORY_OR_ARCHIVE] [ARCHIVE...]
1651 Description
1653 This command deletes an archive from the repository or the complete
1654 repository.
1655 Important: When deleting archives, repository disk space is not freed
1657 until you run borg compact.
1658 When you delete a complete repository, the security info and local
1660 cache for it (if any) are also deleted. Alternatively, you can delete
1661 just the local cache with the --cache-only option, or keep the security
1662 info with the --keep-security-info option.
1663 When in doubt, use --dry-run --list to see what would be deleted.
1665 When using --stats, you will get some statistics about how much data
1667 was deleted - the "Deleted data" deduplicated size there is most inter‐
1668 esting as that is how much your repository will shrink. Please note
1669 that the "All archives" stats refer to the state after deletion.
1670 You can delete multiple archives by specifying a shell pattern to match
1672 multiple archives using the --glob-archives GLOB option (for more info
1673 on these patterns, see borg help patterns).
1674 To avoid accidentally deleting archives, especially when using glob
1676 patterns, it might be helpful to use the --dry-run to test out the com‐
1677 mand without actually making any changes to the repository.
1678 Examples
1680 # delete a single backup archive:
1681 $ borg delete /path/to/repo::Monday
1682 # actually free disk space:
1683 $ borg compact /path/to/repo
1684 # delete all archives whose names begin with the machine's hostname followed by "-"
1686 $ borg delete --glob-archives '{hostname}-*' /path/to/repo
1687 # delete all archives whose names contain "-2012-"
1689 $ borg delete --glob-archives '*-2012-*' /path/to/repo
1690 # see what would be deleted if delete was run without --dry-run
1692 $ borg delete --list --dry-run -a '*-May-*' /path/to/repo
1693 # delete the whole repository and the related local cache:
1695 $ borg delete /path/to/repo
1696 You requested to completely DELETE the repository *including* all archives it contains:
1697 repo Mon, 2016-02-15 19:26:54
1698 root-2016-02-15 Mon, 2016-02-15 19:36:29
1699 newname Mon, 2016-02-15 19:50:19
1700 Type 'YES' if you understand this and want to continue: YES
1701
1703 borg [common options] prune [options] [REPOSITORY]
1704 Description
1706 The prune command prunes a repository by deleting all archives not
1707 matching any of the specified retention options.
1708 Important: Repository disk space is not freed until you run borg com‐
1710 pact.
1711 This command is normally used by automated backup scripts wanting to
1713 keep a certain number of historic backups. This retention policy is
1714 commonly referred to as GFS (Grandfather-father-son) backup rotation
1715 scheme.
1716 Also, prune automatically removes checkpoint archives (incomplete ar‐
1718 chives left behind by interrupted backup runs) except if the checkpoint
1719 is the latest archive (and thus still needed). Checkpoint archives are
1720 not considered when comparing archive counts against the retention lim‐
1721 its (--keep-X).
1722 If a prefix is set with -P, then only archives that start with the pre‐
1724 fix are considered for deletion and only those archives count towards
1725 the totals specified by the rules. Otherwise, all archives in the
1726 repository are candidates for deletion! There is no automatic distinc‐
1727 tion between archives representing different contents. These need to be
1728 distinguished by specifying matching prefixes.
1729 If you have multiple sequences of archives with different data sets
1731 (e.g. from different machines) in one shared repository, use one prune
1732 call per data set that matches only the respective archives using the
1733 -P option.
1734 The --keep-within option takes an argument of the form "<int><char>",
1736 where char is "H", "d", "w", "m", "y". For example, --keep-within 2d
1737 means to keep all archives that were created within the past 48 hours.
1738 "1m" is taken to mean "31d". The archives kept with this option do not
1739 count towards the totals specified by any other options.
1740 A good procedure is to thin out more and more the older your backups
1742 get. As an example, --keep-daily 7 means to keep the latest backup on
1743 each day, up to 7 most recent days with backups (days without backups
1744 do not count). The rules are applied from secondly to yearly, and
1745 backups selected by previous rules do not count towards those of later
1746 rules. The time that each backup starts is used for pruning purposes.
1747 Dates and times are interpreted in the local timezone, and weeks go
1748 from Monday to Sunday. Specifying a negative number of archives to keep
1749 means that there is no limit. As of borg 1.2.0, borg will retain the
1750 oldest archive if any of the secondly, minutely, hourly, daily, weekly,
1751 monthly, or yearly rules was not otherwise able to meet its retention
1752 target. This enables the first chronological archive to continue aging
1753 until it is replaced by a newer archive that meets the retention crite‐
1754 ria.
1755 The --keep-last N option is doing the same as --keep-secondly N (and it
1757 will keep the last N archives under the assumption that you do not cre‐
1758 ate more than one backup archive in the same second).
1759 When using --stats, you will get some statistics about how much data
1761 was deleted - the "Deleted data" deduplicated size there is most inter‐
1762 esting as that is how much your repository will shrink. Please note
1763 that the "All archives" stats refer to the state after pruning.
1764 Examples
1766 Be careful, prune is a potentially dangerous command, it will remove
1767 backup archives.
1768 The default of prune is to apply to all archives in the repository un‐
1770 less you restrict its operation to a subset of the archives using
1771 --glob-archives. When using --glob-archives, be careful to choose a
1772 good matching pattern - e.g. do not use "foo*" if you do not also want
1773 to match "foobar".
1774 It is strongly recommended to always run prune -v --list --dry-run ...
1776 first so you will see what it would do without it actually doing any‐
1777 thing.
1778 # Keep 7 end of day and 4 additional end of week archives.
1780 # Do a dry-run without actually deleting anything.
1781 $ borg prune -v --list --dry-run --keep-daily=7 --keep-weekly=4 /path/to/repo
1782 # Same as above but only apply to archive names starting with the hostname
1784 # of the machine followed by a "-" character:
1785 $ borg prune -v --list --keep-daily=7 --keep-weekly=4 --glob-archives='{hostname}-*' /path/to/repo
1786 # actually free disk space:
1787 $ borg compact /path/to/repo
1788 # Keep 7 end of day, 4 additional end of week archives,
1790 # and an end of month archive for every month:
1791 $ borg prune -v --list --keep-daily=7 --keep-weekly=4 --keep-monthly=-1 /path/to/repo
1792 # Keep all backups in the last 10 days, 4 additional end of week archives,
1794 # and an end of month archive for every month:
1795 $ borg prune -v --list --keep-within=10d --keep-weekly=4 --keep-monthly=-1 /path/to/repo
1796 There is also a visualized prune example in docs/misc/prune-exam‐
1798 ple.txt:
1799 borg prune visualized
1801 =====================
1802 Assume it is 2016-01-01, today's backup has not yet been made, you have
1804 created at least one backup on each day in 2015 except on 2015-12-19 (no
1805 backup made on that day), and you started backing up with borg on
1806 2015-01-01.
1807 This is what borg prune --keep-daily 14 --keep-monthly 6 --keep-yearly 1
1809 would keep.
1810 Backups kept by the --keep-daily rule are marked by a "d" to the right,
1812 backups kept by the --keep-monthly rule are marked by a "m" to the right,
1813 and backups kept by the --keep-yearly rule are marked by a "y" to the
1814 right.
1815 Calendar view
1817 -------------
1818 2015
1820 January February March
1821 Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa Su
1822 1y 2 3 4 1 1
1823 5 6 7 8 9 10 11 2 3 4 5 6 7 8 2 3 4 5 6 7 8
1824 12 13 14 15 16 17 18 9 10 11 12 13 14 15 9 10 11 12 13 14 15
1825 19 20 21 22 23 24 25 16 17 18 19 20 21 22 16 17 18 19 20 21 22
1826 26 27 28 29 30 31 23 24 25 26 27 28 23 24 25 26 27 28 29
1827 30 31
1828 April May June
1830 Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa Su
1831 1 2 3 4 5 1 2 3 1 2 3 4 5 6 7
1832 6 7 8 9 10 11 12 4 5 6 7 8 9 10 8 9 10 11 12 13 14
1833 13 14 15 16 17 18 19 11 12 13 14 15 16 17 15 16 17 18 19 20 21
1834 20 21 22 23 24 25 26 18 19 20 21 22 23 24 22 23 24 25 26 27 28
1835 27 28 29 30 25 26 27 28 29 30 31 29 30m
1836 July August September
1839 Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa Su
1840 1 2 3 4 5 1 2 1 2 3 4 5 6
1841 6 7 8 9 10 11 12 3 4 5 6 7 8 9 7 8 9 10 11 12 13
1842 13 14 15 16 17 18 19 10 11 12 13 14 15 16 14 15 16 17 18 19 20
1843 20 21 22 23 24 25 26 17 18 19 20 21 22 23 21 22 23 24 25 26 27
1844 27 28 29 30 31m 24 25 26 27 28 29 30 28 29 30m
1845 31m
1846 October November December
1848 Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa Su
1849 1 2 3 4 1 1 2 3 4 5 6
1850 5 6 7 8 9 10 11 2 3 4 5 6 7 8 7 8 9 10 11 12 13
1851 12 13 14 15 16 17 18 9 10 11 12 13 14 15 14 15 16 17d18d19 20d
1852 19 20 21 22 23 24 25 16 17 18 19 20 21 22 21d22d23d24d25d26d27d
1853 26 27 28 29 30 31m 23 24 25 26 27 28 29 28d29d30d31d
1854 30m
1855 List view
1857 ---------
1858 --keep-daily 14 --keep-monthly 6 --keep-yearly 1
1860 ----------------------------------------------------------------
1861 1. 2015-12-31 (2015-12-31 kept (2015-12-31 kept
1862 2. 2015-12-30 by daily rule) by daily rule)
1863 3. 2015-12-29 1. 2015-11-30 1. 2015-01-01 (oldest)
1864 4. 2015-12-28 2. 2015-10-31
1865 5. 2015-12-27 3. 2015-09-30
1866 6. 2015-12-26 4. 2015-08-31
1867 7. 2015-12-25 5. 2015-07-31
1868 8. 2015-12-24 6. 2015-06-30
1869 9. 2015-12-23
1870 10. 2015-12-22
1871 11. 2015-12-21
1872 12. 2015-12-20
1873 (no backup made on 2015-12-19)
1874 13. 2015-12-18
1875 14. 2015-12-17
1876 Notes
1879 -----
1880 2015-12-31 is kept due to the --keep-daily 14 rule (because it is applied
1882 first), not due to the --keep-monthly or --keep-yearly rule.
1883 The --keep-yearly 1 rule does not consider the December 31st backup because it
1885 has already been kept due to the daily rule. There are no backups available
1886 from previous years, so the --keep-yearly target of 1 backup is not satisfied.
1887 Because of this, the 2015-01-01 archive (the oldest archive available) is kept.
1888 The --keep-monthly 6 rule keeps Nov, Oct, Sep, Aug, Jul and Jun. December is
1890 not considered for this rule, because that backup was already kept because of
1891 the daily rule.
1892 2015-12-17 is kept to satisfy the --keep-daily 14 rule - because no backup was
1894 made on 2015-12-19. If a backup had been made on that day, it would not keep
1895 the one from 2015-12-17.
1896 We did not include weekly, hourly, minutely or secondly rules to keep this
1898 example simple. They all work in basically the same way.
1899 The weekly rule is easy to understand roughly, but hard to understand in all
1901 details. If interested, read "ISO 8601:2000 standard week-based year".
1902
1905 borg [common options] compact [options] [REPOSITORY]
1906 Description
1908 This command frees repository space by compacting segments.
1909 Use this regularly to avoid running out of space - you do not need to
1911 use this after each borg command though. It is especially useful after
1912 deleting archives, because only compaction will really free repository
1913 space.
1914 borg compact does not need a key, so it is possible to invoke it from
1916 the client or also from the server.
1917 Depending on the amount of segments that need compaction, it may take a
1919 while, so consider using the --progress option.
1920 A segment is compacted if the amount of saved space is above the per‐
1922 centage value given by the --threshold option. If omitted, a threshold
1923 of 10% is used. When using --verbose, borg will output an estimate of
1924 the freed space.
1925 After upgrading borg (server) to 1.2+, you can use borg compact
1927 --cleanup-commits to clean up the numerous 17byte commit-only segments
1928 that borg 1.1 did not clean up due to a bug. It is enough to do that
1929 once per repository. After cleaning up the commits, borg will also do a
1930 normal compaction.
1931 See Separate compaction in Additional Notes for more details.
1933 Examples
1935 # compact segments and free repo disk space
1936 $ borg compact /path/to/repo
1937 # same as above plus clean up 17byte commit-only segments
1939 $ borg compact --cleanup-commits /path/to/repo
1940
1942 borg [common options] info [options] [REPOSITORY_OR_ARCHIVE]
1943 Description
1945 This command displays detailed information about the specified archive
1946 or repository.
1947 Please note that the deduplicated sizes of the individual archives do
1949 not add up to the deduplicated size of the repository ("all archives"),
1950 because the two are meaning different things:
1951 This archive / deduplicated size = amount of data stored ONLY for this
1953 archive = unique chunks of this archive. All archives / deduplicated
1954 size = amount of data stored in the repo = all chunks in the reposi‐
1955 tory.
1956 Borg archives can only contain a limited amount of file metadata. The
1958 size of an archive relative to this limit depends on a number of fac‐
1959 tors, mainly the number of files, the lengths of paths and other meta‐
1960 data stored for files. This is shown as utilization of maximum sup‐
1961 ported archive size.
1962 Examples
1964 $ borg info /path/to/repo::2017-06-29T11:00-srv
1965 Archive name: 2017-06-29T11:00-srv
1966 Archive fingerprint: b2f1beac2bd553b34e06358afa45a3c1689320d39163890c5bbbd49125f00fe5
1967 Comment:
1968 Hostname: myhostname
1969 Username: root
1970 Time (start): Thu, 2017-06-29 11:03:07
1971 Time (end): Thu, 2017-06-29 11:03:13
1972 Duration: 5.66 seconds
1973 Number of files: 17037
1974 Command line: /usr/sbin/borg create /path/to/repo::2017-06-29T11:00-srv /srv
1975 Utilization of max. archive size: 0%
1976 ------------------------------------------------------------------------------
1977 Original size Compressed size Deduplicated size
1978 This archive: 12.53 GB 12.49 GB 1.62 kB
1979 All archives: 121.82 TB 112.41 TB 215.42 GB
1980 Unique chunks Total chunks
1982 Chunk index: 1015213 626934122
1983 $ borg info /path/to/repo --last 1
1985 Archive name: 2017-06-29T11:00-srv
1986 Archive fingerprint: b2f1beac2bd553b34e06358afa45a3c1689320d39163890c5bbbd49125f00fe5
1987 Comment:
1988 Hostname: myhostname
1989 Username: root
1990 Time (start): Thu, 2017-06-29 11:03:07
1991 Time (end): Thu, 2017-06-29 11:03:13
1992 Duration: 5.66 seconds
1993 Number of files: 17037
1994 Command line: /usr/sbin/borg create /path/to/repo::2017-06-29T11:00-srv /srv
1995 Utilization of max. archive size: 0%
1996 ------------------------------------------------------------------------------
1997 Original size Compressed size Deduplicated size
1998 This archive: 12.53 GB 12.49 GB 1.62 kB
1999 All archives: 121.82 TB 112.41 TB 215.42 GB
2000 Unique chunks Total chunks
2002 Chunk index: 1015213 626934122
2003 $ borg info /path/to/repo
2005 Repository ID: d857ce5788c51272c61535062e89eac4e8ef5a884ffbe976e0af9d8765dedfa5
2006 Location: /path/to/repo
2007 Encrypted: Yes (repokey)
2008 Cache: /root/.cache/borg/d857ce5788c51272c61535062e89eac4e8ef5a884ffbe976e0af9d8765dedfa5
2009 Security dir: /root/.config/borg/security/d857ce5788c51272c61535062e89eac4e8ef5a884ffbe976e0af9d8765dedfa5
2010 ------------------------------------------------------------------------------
2011 Original size Compressed size Deduplicated size
2012 All archives: 121.82 TB 112.41 TB 215.42 GB
2013 Unique chunks Total chunks
2015 Chunk index: 1015213 626934122
2016
2018 borg [common options] mount [options] REPOSITORY_OR_ARCHIVE MOUNTPOINT [PATH...]
2019 Description
2021 This command mounts an archive as a FUSE filesystem. This can be useful
2022 for browsing an archive or restoring individual files. Unless the
2023 --foreground option is given the command will run in the background un‐
2024 til the filesystem is umounted.
2025 The command borgfs provides a wrapper for borg mount. This can also be
2027 used in fstab entries: /path/to/repo /mnt/point fuse.borgfs de‐
2028 faults,noauto 0 0
2029 To allow a regular user to use fstab entries, add the user option:
2031 /path/to/repo /mnt/point fuse.borgfs defaults,noauto,user 0 0
2032 For FUSE configuration and mount options, see the mount.fuse(8) manual
2034 page.
2035 Borg's default behavior is to use the archived user and group names of
2037 each file and map them to the system's respective user and group ids.
2038 Alternatively, using numeric-ids will instead use the archived user and
2039 group ids without any mapping.
2040 The uid and gid mount options (implemented by Borg) can be used to
2042 override the user and group ids of all files (i.e., borg mount -o
2043 uid=1000,gid=1000).
2044 The man page references user_id and group_id mount options (implemented
2046 by fuse) which specify the user and group id of the mount owner (aka,
2047 the user who does the mounting). It is set automatically by libfuse (or
2048 the filesystem if libfuse is not used). However, you should not specify
2049 these manually. Unlike the uid and gid mount options which affect all
2050 files, user_id and group_id affect the user and group id of the mounted
2051 (base) directory.
2052 Additional mount options supported by borg:
2054 • versions: when used with a repository mount, this gives a merged,
2056 versioned view of the files in the archives. EXPERIMENTAL, layout may
2057 change in future.
2058 • allow_damaged_files: by default damaged files (where missing chunks
2060 were replaced with runs of zeros by borg check --repair) are not
2061 readable and return EIO (I/O error). Set this option to read such
2062 files.
2063 • ignore_permissions: for security reasons the "default_permissions"
2065 mount option is internally enforced by borg. "ignore_permissions" can
2066 be given to not enforce "default_permissions".
2067 The BORG_MOUNT_DATA_CACHE_ENTRIES environment variable is meant for ad‐
2069 vanced users to tweak the performance. It sets the number of cached
2070 data chunks; additional memory usage can be up to ~8 MiB times this
2071 number. The default is the number of CPU cores.
2072 When the daemonized process receives a signal or crashes, it does not
2074 unmount. Unmounting in these cases could cause an active rsync or sim‐
2075 ilar process to unintentionally delete data.
2076 When running in the foreground ^C/SIGINT unmounts cleanly, but other
2078 signals or crashes do not.
2079
2081 borg [common options] umount [options] MOUNTPOINT
2082 Description
2084 This command un-mounts a FUSE filesystem that was mounted with borg
2085 mount.
2086 This is a convenience wrapper that just calls the platform-specific
2088 shell command - usually this is either umount or fusermount -u.
2089 Examples
2091 # Mounting the repository shows all archives.
2092 # Archives are loaded lazily, expect some delay when navigating to an archive
2093 # for the first time.
2094 $ borg mount /path/to/repo /tmp/mymountpoint
2095 $ ls /tmp/mymountpoint
2096 root-2016-02-14 root-2016-02-15
2097 $ borg umount /tmp/mymountpoint
2098 # Mounting a specific archive is possible as well.
2100 $ borg mount /path/to/repo::root-2016-02-15 /tmp/mymountpoint
2101 $ ls /tmp/mymountpoint
2102 bin boot etc home lib lib64 lost+found media mnt opt
2103 root sbin srv tmp usr var
2104 $ borg umount /tmp/mymountpoint
2105 # The "versions view" merges all archives in the repository
2107 # and provides a versioned view on files.
2108 $ borg mount -o versions /path/to/repo /tmp/mymountpoint
2109 $ ls -l /tmp/mymountpoint/home/user/doc.txt/
2110 total 24
2111 -rw-rw-r-- 1 user group 12357 Aug 26 21:19 doc.cda00bc9.txt
2112 -rw-rw-r-- 1 user group 12204 Aug 26 21:04 doc.fa760f28.txt
2113 $ borg umount /tmp/mymountpoint
2114 # Archive filters are supported.
2116 # These are especially handy for the "versions view",
2117 # which does not support lazy processing of archives.
2118 $ borg mount -o versions --glob-archives '*-my-home' --last 10 /path/to/repo /tmp/mymountpoint
2119 # Exclusion options are supported.
2121 # These can speed up mounting and lower memory needs significantly.
2122 $ borg mount /path/to/repo /tmp/mymountpoint only/that/path
2123 $ borg mount --exclude '...' /path/to/repo /tmp/mymountpoint
2124 borgfs
2126 $ echo '/mnt/backup /tmp/myrepo fuse.borgfs defaults,noauto 0 0' >> /etc/fstab
2127 $ echo '/mnt/backup::root-2016-02-15 /tmp/myarchive fuse.borgfs defaults,noauto 0 0' >> /etc/fstab
2128 $ mount /tmp/myrepo
2129 $ mount /tmp/myarchive
2130 $ ls /tmp/myrepo
2131 root-2016-02-01 root-2016-02-2015
2132 $ ls /tmp/myarchive
2133 bin boot etc home lib lib64 lost+found media mnt opt root sbin srv tmp usr var
2134 NOTE:
2136 borgfs will be automatically provided if you used a distribution
2137 package, pip or setup.py to install Borg. Users of the standalone
2138 binary will have to manually create a symlink (see Standalone Bi‐
2139 nary).
2140
2142 borg [common options] key change-passphrase [options] [REPOSITORY]
2143 Description
2145 The key files used for repository encryption are optionally passphrase
2146 protected. This command can be used to change this passphrase.
2147 Please note that this command only changes the passphrase, but not any
2149 secret protected by it (like e.g. encryption/MAC keys or chunker seed).
2150 Thus, changing the passphrase after passphrase and borg key got compro‐
2151 mised does not protect future (nor past) backups to the same reposi‐
2152 tory.
2153 Examples
2155 # Create a key file protected repository
2156 $ borg init --encryption=keyfile -v /path/to/repo
2157 Initializing repository at "/path/to/repo"
2158 Enter new passphrase:
2159 Enter same passphrase again:
2160 Remember your passphrase. Your data will be inaccessible without it.
2161 Key in "/root/.config/borg/keys/mnt_backup" created.
2162 Keep this key safe. Your data will be inaccessible without it.
2163 Synchronizing chunks cache...
2164 Archives: 0, w/ cached Idx: 0, w/ outdated Idx: 0, w/o cached Idx: 0.
2165 Done.
2166 # Change key file passphrase
2168 $ borg key change-passphrase -v /path/to/repo
2169 Enter passphrase for key /root/.config/borg/keys/mnt_backup:
2170 Enter new passphrase:
2171 Enter same passphrase again:
2172 Remember your passphrase. Your data will be inaccessible without it.
2173 Key updated
2174 # Import a previously-exported key into the specified
2176 # key file (creating or overwriting the output key)
2177 # (keyfile repositories only)
2178 $ BORG_KEY_FILE=/path/to/output-key borg key import /path/to/repo /path/to/exported
2179 Fully automated using environment variables:
2181 $ BORG_NEW_PASSPHRASE=old borg init -e=repokey repo
2183 # now "old" is the current passphrase.
2184 $ BORG_PASSPHRASE=old BORG_NEW_PASSPHRASE=new borg key change-passphrase repo
2185 # now "new" is the current passphrase.
2186
2188 borg [common options] key export [options] [REPOSITORY] [PATH]
2189 Description
2191 If repository encryption is used, the repository is inaccessible with‐
2192 out the key. This command allows one to backup this essential key.
2193 Note that the backup produced does not include the passphrase itself
2194 (i.e. the exported key stays encrypted). In order to regain access to a
2195 repository, one needs both the exported key and the original
2196 passphrase.
2197 There are three backup formats. The normal backup format is suitable
2199 for digital storage as a file. The --paper backup format is optimized
2200 for printing and typing in while importing, with per line checks to re‐
2201 duce problems with manual input. The --qr-html creates a printable HTML
2202 template with a QR code and a copy of the --paper-formatted key.
2203 For repositories using keyfile encryption the key is saved locally on
2205 the system that is capable of doing backups. To guard against loss of
2206 this key, the key needs to be backed up independently of the main data
2207 backup.
2208 For repositories using the repokey encryption the key is saved in the
2210 repository in the config file. A backup is thus not strictly needed,
2211 but guards against the repository becoming inaccessible if the file is
2212 damaged for some reason.
2213 Examples:
2215 borg key export /path/to/repo > encrypted-key-backup
2217 borg key export --paper /path/to/repo > encrypted-key-backup.txt
2218 borg key export --qr-html /path/to/repo > encrypted-key-backup.html
2219 # Or pass the output file as an argument instead of redirecting stdout:
2220 borg key export /path/to/repo encrypted-key-backup
2221 borg key export --paper /path/to/repo encrypted-key-backup.txt
2222 borg key export --qr-html /path/to/repo encrypted-key-backup.html
2223
2225 borg [common options] key import [options] [REPOSITORY] [PATH]
2226 Description
2228 This command restores a key previously backed up with the export com‐
2229 mand.
2230 If the --paper option is given, the import will be an interactive
2232 process in which each line is checked for plausibility before proceed‐
2233 ing to the next line. For this format PATH must not be given.
2234 For repositories using keyfile encryption, the key file which borg key
2236 import writes to depends on several factors. If the BORG_KEY_FILE envi‐
2237 ronment variable is set and non-empty, borg key import creates or over‐
2238 writes that file named by $BORG_KEY_FILE. Otherwise, borg key import
2239 searches in the $BORG_KEYS_DIR directory for a key file associated with
2240 the repository. If a key file is found in $BORG_KEYS_DIR, borg key im‐
2241 port overwrites it; otherwise, borg key import creates a new key file
2242 in $BORG_KEYS_DIR.
2243
2245 borg [common options] upgrade [options] [REPOSITORY]
2246 Description
2248 Upgrade an existing, local Borg repository.
2249 When you do not need borg upgrade
2251 Not every change requires that you run borg upgrade.
2252 You do not need to run it when:
2254 • moving your repository to a different place
2256 • upgrading to another point release (like 1.0.x to 1.0.y), except when
2258 noted otherwise in the changelog
2259 • upgrading from 1.0.x to 1.1.x, except when noted otherwise in the
2261 changelog
2262 Borg 1.x.y upgrades
2264 Use borg upgrade --tam REPO to require manifest authentication intro‐
2265 duced with Borg 1.0.9 to address security issues. This means that modi‐
2266 fying the repository after doing this with a version prior to 1.0.9
2267 will raise a validation error, so only perform this upgrade after up‐
2268 dating all clients using the repository to 1.0.9 or newer.
2269 This upgrade should be done on each client for safety reasons.
2271 If a repository is accidentally modified with a pre-1.0.9 client after
2273 this upgrade, use borg upgrade --tam --force REPO to remedy it.
2274 If you routinely do this you might not want to enable this upgrade
2276 (which will leave you exposed to the security issue). You can reverse
2277 the upgrade by issuing borg upgrade --disable-tam REPO.
2278 See
2280 https://borgbackup.readthedocs.io/en/stable/changes.html#pre-1-0-9-manifest-spoofing-vulnerability
2281 for details.
2282 Attic and Borg 0.xx to Borg 1.x
2284 This currently supports converting an Attic repository to Borg and also
2285 helps with converting Borg 0.xx to 1.0.
2286 Currently, only LOCAL repositories can be upgraded (issue #465).
2288 Please note that borg create (since 1.0.0) uses bigger chunks by de‐
2290 fault than old borg or attic did, so the new chunks won't deduplicate
2291 with the old chunks in the upgraded repository. See --chunker-params
2292 option of borg create and borg recreate.
2293 borg upgrade will change the magic strings in the repository's segments
2295 to match the new Borg magic strings. The keyfiles found in $AT‐
2296 TIC_KEYS_DIR or ~/.attic/keys/ will also be converted and copied to
2297 $BORG_KEYS_DIR or ~/.config/borg/keys.
2298 The cache files are converted, from $ATTIC_CACHE_DIR or ~/.cache/attic
2300 to $BORG_CACHE_DIR or ~/.cache/borg, but the cache layout between Borg
2301 and Attic changed, so it is possible the first backup after the conver‐
2302 sion takes longer than expected due to the cache resync.
2303 Upgrade should be able to resume if interrupted, although it will still
2305 iterate over all segments. If you want to start from scratch, use borg
2306 delete over the copied repository to make sure the cache files are also
2307 removed:
2308 borg delete borg
2310 Unless --inplace is specified, the upgrade process first creates a
2312 backup copy of the repository, in REPOSITORY.before-upgrade-DATETIME,
2313 using hardlinks. This requires that the repository and its parent di‐
2314 rectory reside on same filesystem so the hardlink copy can work. This
2315 takes longer than in place upgrades, but is much safer and gives
2316 progress information (as opposed to cp -al). Once you are satisfied
2317 with the conversion, you can safely destroy the backup copy.
2318 WARNING: Running the upgrade in place will make the current copy unus‐
2320 able with older version, with no way of going back to previous ver‐
2321 sions. This can PERMANENTLY DAMAGE YOUR REPOSITORY! Attic CAN NOT READ
2322 BORG REPOSITORIES, as the magic strings have changed. You have been
2323 warned.
2324 Examples
2326 # Upgrade the borg repository to the most recent version.
2327 $ borg upgrade -v /path/to/repo
2328 making a hardlink copy in /path/to/repo.before-upgrade-2016-02-15-20:51:55
2329 opening attic repository with borg and converting
2330 no key file found for repository
2331 converting repo index /path/to/repo/index.0
2332 converting 1 segments...
2333 converting borg 0.xx to borg current
2334 no key file found for repository
2335 Upgrading a passphrase encrypted attic repo
2337 attic offered a "passphrase" encryption mode, but this was removed in
2338 borg 1.0 and replaced by the "repokey" mode (which stores the
2339 passphrase-protected encryption key into the repository config).
2340 Thus, to upgrade a "passphrase" attic repo to a "repokey" borg repo, 2
2342 steps are needed, in this order:
2343 • borg upgrade repo
2345 • borg key migrate-to-repokey repo
2347
2349 borg [common options] recreate [options] [REPOSITORY_OR_ARCHIVE] [PATH...]
2350 Description
2352 Recreate the contents of existing archives.
2353 recreate is a potentially dangerous function and might lead to data
2355 loss (if used wrongly). BE VERY CAREFUL!
2356 Important: Repository disk space is not freed until you run borg com‐
2358 pact.
2359 --exclude, --exclude-from, --exclude-if-present, --keep-exclude-tags
2361 and PATH have the exact same semantics as in "borg create", but they
2362 only check for files in the archives and not in the local file system.
2363 If PATHs are specified, the resulting archives will only contain files
2364 from these PATHs.
2365 Note that all paths in an archive are relative, therefore absolute pat‐
2367 terns/paths will not match (--exclude, --exclude-from, PATHs).
2368 --recompress allows one to change the compression of existing data in
2370 archives. Due to how Borg stores compressed size information this
2371 might display incorrect information for archives that were not recre‐
2372 ated at the same time. There is no risk of data loss by this.
2373 --chunker-params will re-chunk all files in the archive, this can be
2375 used to have upgraded Borg 0.xx or Attic archives deduplicate with Borg
2376 1.x archives.
2377 USE WITH CAUTION. Depending on the PATHs and patterns given, recreate
2379 can be used to permanently delete files from archives. When in doubt,
2380 use --dry-run --verbose --list to see how patterns/PATHS are inter‐
2381 preted. See Item flags in borg create for details.
2382 The archive being recreated is only removed after the operation com‐
2384 pletes. The archive that is built during the operation exists at the
2385 same time at "<ARCHIVE>.recreate". The new archive will have a differ‐
2386 ent archive ID.
2387 With --target the original archive is not replaced, instead a new ar‐
2389 chive is created.
2390 When rechunking (or recompressing), space usage can be substantial -
2392 expect at least the entire deduplicated size of the archives using the
2393 previous chunker (or compression) params.
2394 If you recently ran borg check --repair and it had to fix lost chunks
2396 with all-zero replacement chunks, please first run another backup for
2397 the same data and re-run borg check --repair afterwards to heal any ar‐
2398 chives that had lost chunks which are still generated from the input
2399 data.
2400 Important: running borg recreate to re-chunk will remove the
2402 chunks_healthy metadata of all items with replacement chunks, so heal‐
2403 ing will not be possible any more after re-chunking (it is also un‐
2404 likely it would ever work: due to the change of chunking parameters,
2405 the missing chunk likely will never be seen again even if you still
2406 have the data that produced it).
2407 Examples
2409 # Make old (Attic / Borg 0.xx) archives deduplicate with Borg 1.x archives.
2410 # Archives created with Borg 1.1+ and the default chunker params are skipped
2411 # (archive ID stays the same).
2412 $ borg recreate /mnt/backup --chunker-params default --progress
2413 # Create a backup with little but fast compression
2415 $ borg create /mnt/backup::archive /some/files --compression lz4
2416 # Then compress it - this might take longer, but the backup has already completed,
2417 # so no inconsistencies from a long-running backup job.
2418 $ borg recreate /mnt/backup::archive --recompress --compression zlib,9
2419 # Remove unwanted files from all archives in a repository.
2421 # Note the relative path for the --exclude option - archives only contain relative paths.
2422 $ borg recreate /mnt/backup --exclude home/icke/Pictures/drunk_photos
2423 # Change archive comment
2425 $ borg create --comment "This is a comment" /mnt/backup::archivename ~
2426 $ borg info /mnt/backup::archivename
2427 Name: archivename
2428 Fingerprint: ...
2429 Comment: This is a comment
2430 ...
2431 $ borg recreate --comment "This is a better comment" /mnt/backup::archivename
2432 $ borg info /mnt/backup::archivename
2433 Name: archivename
2434 Fingerprint: ...
2435 Comment: This is a better comment
2436 ...
2437
2439 borg [common options] import-tar [options] ARCHIVE TARFILE
2440 Description
2442 This command creates a backup archive from a tarball.
2443 When giving '-' as path, Borg will read a tar stream from standard in‐
2445 put.
2446 By default (--tar-filter=auto) Borg will detect whether the file is
2448 compressed based on its file extension and pipe the file through an ap‐
2449 propriate filter:
2450 • .tar.gz or .tgz: gzip -d
2452 • .tar.bz2 or .tbz: bzip2 -d
2454 • .tar.xz or .txz: xz -d
2456 • .tar.zstd or .tar.zst: zstd -d
2458 • .tar.lz4: lz4 -d
2460 Alternatively, a --tar-filter program may be explicitly specified. It
2462 should read compressed data from stdin and output an uncompressed tar
2463 stream on stdout.
2464 Most documentation of borg create applies. Note that this command does
2466 not support excluding files.
2467 import-tar is a lossy conversion: BSD flags, ACLs, extended attributes
2469 (xattrs), atime and ctime are not exported. Timestamp resolution is
2470 limited to whole seconds, not the nanosecond resolution otherwise sup‐
2471 ported by Borg.
2472 A --sparse option (as found in borg create) is not supported.
2474 import-tar reads POSIX.1-1988 (ustar), POSIX.1-2001 (pax), GNU tar,
2476 UNIX V7 tar and SunOS tar with extended attributes.
2477
2479 borg [common options] export-tar [options] ARCHIVE FILE [PATH...]
2480 Description
2482 This command creates a tarball from an archive.
2483 When giving '-' as the output FILE, Borg will write a tar stream to
2485 standard output.
2486 By default (--tar-filter=auto) Borg will detect whether the FILE should
2488 be compressed based on its file extension and pipe the tarball through
2489 an appropriate filter before writing it to FILE:
2490 • .tar.gz or .tgz: gzip
2492 • .tar.bz2 or .tbz: bzip2
2494 • .tar.xz or .txz: xz
2496 • .tar.zstd or .tar.zst: zstd
2498 • .tar.lz4: lz4
2500 Alternatively, a --tar-filter program may be explicitly specified. It
2502 should read the uncompressed tar stream from stdin and write a com‐
2503 pressed/filtered tar stream to stdout.
2504 The generated tarball uses the GNU tar format.
2506 export-tar is a lossy conversion: BSD flags, ACLs, extended attributes
2508 (xattrs), atime and ctime are not exported. Timestamp resolution is
2509 limited to whole seconds, not the nanosecond resolution otherwise sup‐
2510 ported by Borg.
2511 A --sparse option (as found in borg extract) is not supported.
2513 By default the entire archive is extracted but a subset of files and
2515 directories can be selected by passing a list of PATHs as arguments.
2516 The file selection can further be restricted by using the --exclude op‐
2517 tion.
2518 For more help on include/exclude patterns, see the borg help patterns
2520 command output.
2521 --progress can be slower than no progress display, since it makes one
2523 additional pass over the archive metadata.
2524 Examples
2526 # export as uncompressed tar
2527 $ borg export-tar /path/to/repo::Monday Monday.tar
2528 # exclude some types, compress using gzip
2530 $ borg export-tar /path/to/repo::Monday Monday.tar.gz --exclude '*.so'
2531 # use higher compression level with gzip
2533 $ borg export-tar --tar-filter="gzip -9" testrepo::linux Monday.tar.gz
2534 # export a tar, but instead of storing it on disk,
2536 # upload it to a remote site using curl.
2537 $ borg export-tar /path/to/repo::Monday - | curl --data-binary @- https://somewhere/to/POST
2538 # remote extraction via "tarpipe"
2540 $ borg export-tar /path/to/repo::Monday - | ssh somewhere "cd extracted; tar x"
2541
2543 borg [common options] serve [options]
2544 Description
2546 This command starts a repository server process. This command is usu‐
2547 ally not used manually.
2548 Examples
2550 borg serve has special support for ssh forced commands (see autho‐
2551 rized_keys example below): if the environment variable SSH_ORIGI‐
2552 NAL_COMMAND is set it will ignore some options given on the command
2553 line and use the values from the variable instead. This only applies to
2554 a carefully controlled allowlist of safe options. This list currently
2555 contains:
2556 • Options that control the log level and debug topics printed such as
2558 --verbose, --info, --debug, --debug-topic, etc.
2559 • --lock-wait to allow the client to control how long to wait before
2561 giving up and aborting the operation when another process is holding
2562 a lock.
2563 Environment variables (such as BORG_XXX) contained in the original com‐
2565 mand sent by the client are not interpreted, but ignored. If BORG_XXX
2566 environment variables should be set on the borg serve side, then these
2567 must be set in system-specific locations like /etc/environment or in
2568 the forced command itself (example below).
2569 # Allow an SSH keypair to only run borg, and only have access to /path/to/repo.
2571 # Use key options to disable unneeded and potentially dangerous SSH functionality.
2572 # This will help to secure an automated remote backup system.
2573 $ cat ~/.ssh/authorized_keys
2574 command="borg serve --restrict-to-path /path/to/repo",restrict ssh-rsa AAAAB3[...]
2575 # Set a BORG_XXX environment variable on the "borg serve" side
2577 $ cat ~/.ssh/authorized_keys
2578 command="export BORG_XXX=value; borg serve [...]",restrict ssh-rsa [...]
2579 NOTE:
2581 The examples above use the restrict directive. This does automati‐
2582 cally block potential dangerous ssh features, even when they are
2583 added in a future update. Thus, this option should be preferred.
2584 If you're using openssh-server < 7.2, however, you have to explic‐
2586 itly specify the ssh features to restrict and cannot simply use the
2587 restrict option as it has been introduced in v7.2. We recommend to
2588 use no-port-forwarding,no-X11-forwarding,no-pty,no-agent-forward‐
2589 ing,no-user-rc in this case.
2590 Details about sshd usage: sshd(8)
2592 SSH Configuration
2594 borg serve's pipes (stdin/stdout/stderr) are connected to the sshd
2595 process on the server side. In the event that the SSH connection be‐
2596 tween borg serve and the client is disconnected or stuck abnormally
2597 (for example, due to a network outage), it can take a long time for
2598 sshd to notice the client is disconnected. In the meantime, sshd con‐
2599 tinues running, and as a result so does the borg serve process holding
2600 the lock on the repository. This can cause subsequent borg operations
2601 on the remote repository to fail with the error: Failed to create/ac‐
2602 quire the lock.
2603 In order to avoid this, it is recommended to perform the following ad‐
2605 ditional SSH configuration:
2606 Either in the client side's ~/.ssh/config file, or in the client's
2608 /etc/ssh/ssh_config file:
2609 Host backupserver
2611 ServerAliveInterval 10
2612 ServerAliveCountMax 30
2613 Replacing backupserver with the hostname, FQDN or IP address of the
2615 borg server.
2616 This will cause the client to send a keepalive to the server every 10
2618 seconds. If 30 consecutive keepalives are sent without a response (a
2619 time of 300 seconds), the ssh client process will be terminated, caus‐
2620 ing the borg process to terminate gracefully.
2621 On the server side's sshd configuration file (typically
2623 /etc/ssh/sshd_config):
2624 ClientAliveInterval 10
2626 ClientAliveCountMax 30
2627 This will cause the server to send a keep alive to the client every 10
2629 seconds. If 30 consecutive keepalives are sent without a response (a
2630 time of 300 seconds), the server's sshd process will be terminated,
2631 causing the borg serve process to terminate gracefully and release the
2632 lock on the repository.
2633 If you then run borg commands with --lock-wait 600, this gives suffi‐
2635 cient time for the borg serve processes to terminate after the SSH con‐
2636 nection is torn down after the 300 second wait for the keepalives to
2637 fail.
2638 You may, of course, modify the timeout values demonstrated above to
2640 values that suit your environment and use case.
2641
2643 borg [common options] config [options] [REPOSITORY] [NAME] [VALUE]
2644 Description
2646 This command gets and sets options in a local repository or cache con‐
2647 fig file. For security reasons, this command only works on local
2648 repositories.
2649 To delete a config value entirely, use --delete. To list the values of
2651 the configuration file or the default values, use --list. To get and
2652 existing key, pass only the key name. To set a key, pass both the key
2653 name and the new value. Keys can be specified in the format "sec‐
2654 tion.name" or simply "name"; the section will default to "repository"
2655 and "cache" for the repo and cache configs, respectively.
2656 By default, borg config manipulates the repository config file. Using
2658 --cache edits the repository cache's config file instead.
2659 NOTE:
2661 The repository & cache config files are some of the only directly
2662 manipulable parts of a repository that aren't versioned or backed
2663 up, so be careful when making changes!
2664 Examples
2666 # find cache directory
2667 $ cd ~/.cache/borg/$(borg config /path/to/repo id)
2668 # reserve some space
2670 $ borg config /path/to/repo additional_free_space 2G
2671 # make a repo append-only
2673 $ borg config /path/to/repo append_only 1
2674
2676 borg [common options] with-lock [options] REPOSITORY COMMAND [ARGS...]
2677 Description
2679 This command runs a user-specified command while the repository lock is
2680 held.
2681 It will first try to acquire the lock (make sure that no other opera‐
2683 tion is running in the repo), then execute the given command as a sub‐
2684 process and wait for its termination, release the lock and return the
2685 user command's return code as borg's return code.
2686 NOTE:
2688 If you copy a repository with the lock held, the lock will be
2689 present in the copy. Thus, before using borg on the copy from a dif‐
2690 ferent host, you need to use "borg break-lock" on the copied reposi‐
2691 tory, because Borg is cautious and does not automatically remove
2692 stale locks made by a different host.
2693
2695 borg [common options] break-lock [options] [REPOSITORY]
2696 Description
2698 This command breaks the repository and cache locks. Please use care‐
2699 fully and only while no borg process (on any machine) is trying to ac‐
2700 cess the Cache or the Repository.
2701
2703 borg [common options] benchmark crud [options] REPOSITORY PATH
2704 Description
2706 This command benchmarks borg CRUD (create, read, update, delete) opera‐
2707 tions.
2708 It creates input data below the given PATH and backups this data into
2710 the given REPO. The REPO must already exist (it could be a fresh empty
2711 repo or an existing repo, the command will create / read / update /
2712 delete some archives named borg-benchmark-crud* there.
2713 Make sure you have free space there, you'll need about 1GB each (+
2715 overhead).
2716 If your repository is encrypted and borg needs a passphrase to unlock
2718 the key, use:
2719 BORG_PASSPHRASE=mysecret borg benchmark crud REPO PATH
2721 Measurements are done with different input file sizes and counts. The
2723 file contents are very artificial (either all zero or all random), thus
2724 the measurement results do not necessarily reflect performance with
2725 real data. Also, due to the kind of content used, no compression is
2726 used in these benchmarks.
2727 C- == borg create (1st archive creation, no compression, do not use
2729 files cache)
2730 C-Z- == all-zero files. full dedup, this is primarily measuring
2731 reader/chunker/hasher. C-R- == random files. no dedup, measur‐
2732 ing throughput through all processing stages.
2733 R- == borg extract (extract archive, dry-run, do everything, but do not
2735 write files to disk)
2736 R-Z- == all zero files. Measuring heavily duplicated files.
2737 R-R- == random files. No duplication here, measuring throughput
2738 through all processing stages, except writing to disk.
2739 U- == borg create (2nd archive creation of unchanged input files, mea‐
2741 sure files cache speed)
2742 The throughput value is kind of virtual here, it does not actu‐
2743 ally read the file. U-Z- == needs to check the 2 all-zero
2744 chunks' existence in the repo. U-R- == needs to check existence
2745 of a lot of different chunks in the repo.
2746 D- == borg delete archive (delete last remaining archive, measure dele‐
2748 tion + compaction)
2749 D-Z- == few chunks to delete / few segments to compact/remove.
2750 D-R- == many chunks to delete / many segments to compact/remove.
2751 Please note that there might be quite some variance in these measure‐
2753 ments. Try multiple measurements and having a otherwise idle machine
2754 (and network, if you use it).
2755
2757 borg help patterns
2758 The path/filenames used as input for the pattern matching start from
2759 the currently active recursion root. You usually give the recursion
2760 root(s) when invoking borg and these can be either relative or absolute
2761 paths.
2762 If you give /absolute/ as root, the paths going into the matcher will
2764 look relative like absolute/.../file.ext, because file paths in Borg
2765 archives are always stored normalized and relative. This means that
2766 e.g. borg create /path/to/repo ../some/path will store all files as
2767 some/path/.../file.ext and borg create /path/to/repo /home/user will
2768 store all files as home/user/.../file.ext.
2769 A directory exclusion pattern can end either with or without a slash
2771 ('/'). If it ends with a slash, such as some/path/, the directory will
2772 be included but not its content. If it does not end with a slash, such
2773 as some/path, both the directory and content will be excluded.
2774 File patterns support these styles: fnmatch, shell, regular expres‐
2776 sions, path prefixes and path full-matches. By default, fnmatch is used
2777 for --exclude patterns and shell-style is used for the --pattern op‐
2778 tion. For commands that support patterns in their PATH argument like
2779 (borg list), the default pattern is path prefix.
2780 Starting with Borg 1.2, discovered fs paths are normalised, have lead‐
2782 ing slashes removed and then are matched against your patterns. Note:
2783 You need to review your include / exclude patterns and make sure they
2784 do not expect leading slashes. Borg can only deal with this for some
2785 very simple patterns by removing leading slashes there also.
2786 If followed by a colon (':') the first two characters of a pattern are
2788 used as a style selector. Explicit style selection is necessary when a
2789 non-default style is desired or when the desired pattern starts with
2790 two alphanumeric characters followed by a colon (i.e. aa:something/*).
2791 Fnmatch, selector fm:
2793 This is the default style for --exclude and --exclude-from.
2794 These patterns use a variant of shell pattern syntax, with '*'
2795 matching any number of characters, '?' matching any single char‐
2796 acter, '[...]' matching any single character specified, includ‐
2797 ing ranges, and '[!...]' matching any character not specified.
2798 For the purpose of these patterns, the path separator (backslash
2799 for Windows and '/' on other systems) is not treated specially.
2800 Wrap meta-characters in brackets for a literal match (i.e. [?]
2801 to match the literal character ?). For a path to match a pat‐
2802 tern, the full path must match, or it must match from the start
2803 of the full path to just before a path separator. Except for the
2804 root path, paths will never end in the path separator when
2805 matching is attempted. Thus, if a given pattern ends in a path
2806 separator, a '*' is appended before matching is attempted. A
2807 leading path separator is always removed.
2808 Shell-style patterns, selector sh:
2810 This is the default style for --pattern and --patterns-from.
2811 Like fnmatch patterns these are similar to shell patterns. The
2812 difference is that the pattern may include **/ for matching zero
2813 or more directory levels, * for matching zero or more arbitrary
2814 characters with the exception of any path separator. A leading
2815 path separator is always removed.
2816 Regular expressions, selector re:
2818 Regular expressions similar to those found in Perl are sup‐
2819 ported. Unlike shell patterns regular expressions are not re‐
2820 quired to match the full path and any substring match is suffi‐
2821 cient. It is strongly recommended to anchor patterns to the
2822 start ('^'), to the end ('$') or both. Path separators (back‐
2823 slash for Windows and '/' on other systems) in paths are always
2824 normalized to a forward slash ('/') before applying a pattern.
2825 The regular expression syntax is described in the Python docu‐
2826 mentation for the re module.
2827 Path prefix, selector pp:
2829 This pattern style is useful to match whole sub-directories. The
2830 pattern pp:root/somedir matches root/somedir and everything
2831 therein. A leading path separator is always removed.
2832 Path full-match, selector pf:
2834 This pattern style is (only) useful to match full paths. This
2835 is kind of a pseudo pattern as it can not have any variable or
2836 unspecified parts - the full path must be given.
2837 pf:root/file.ext matches root/file.ext only. A leading path sep‐
2838 arator is always removed.
2839 Implementation note: this is implemented via very time-efficient
2841 O(1) hashtable lookups (this means you can have huge amounts of
2842 such patterns without impacting performance much). Due to that,
2843 this kind of pattern does not respect any context or order. If
2844 you use such a pattern to include a file, it will always be in‐
2845 cluded (if the directory recursion encounters it). Other in‐
2846 clude/exclude patterns that would normally match will be ig‐
2847 nored. Same logic applies for exclude.
2848 NOTE:
2850 re:, sh: and fm: patterns are all implemented on top of the Python
2851 SRE engine. It is very easy to formulate patterns for each of these
2852 types which requires an inordinate amount of time to match paths. If
2853 untrusted users are able to supply patterns, ensure they cannot sup‐
2854 ply re: patterns. Further, ensure that sh: and fm: patterns only
2855 contain a handful of wildcards at most.
2856 Exclusions can be passed via the command line option --exclude. When
2858 used from within a shell, the patterns should be quoted to protect them
2859 from expansion.
2860 The --exclude-from option permits loading exclusion patterns from a
2862 text file with one pattern per line. Lines empty or starting with the
2863 number sign ('#') after removing whitespace on both ends are ignored.
2864 The optional style selector prefix is also supported for patterns
2865 loaded from a file. Due to whitespace removal, paths with whitespace at
2866 the beginning or end can only be excluded using regular expressions.
2867 To test your exclusion patterns without performing an actual backup you
2869 can run borg create --list --dry-run ....
2870 Examples:
2872 # Exclude '/home/user/file.o' but not '/home/user/file.odt':
2874 $ borg create -e '*.o' backup /
2875 # Exclude '/home/user/junk' and '/home/user/subdir/junk' but
2877 # not '/home/user/importantjunk' or '/etc/junk':
2878 $ borg create -e 'home/*/junk' backup /
2879 # Exclude the contents of '/home/user/cache' but not the directory itself:
2881 $ borg create -e home/user/cache/ backup /
2882 # The file '/home/user/cache/important' is *not* backed up:
2884 $ borg create -e home/user/cache/ backup / /home/user/cache/important
2885 # The contents of directories in '/home' are not backed up when their name
2887 # ends in '.tmp'
2888 $ borg create --exclude 're:^home/[^/]+\.tmp/' backup /
2889 # Load exclusions from file
2891 $ cat >exclude.txt <<EOF
2892 # Comment line
2893 home/*/junk
2894 *.tmp
2895 fm:aa:something/*
2896 re:^home/[^/]+\.tmp/
2897 sh:home/*/.thumbnails
2898 # Example with spaces, no need to escape as it is processed by borg
2899 some file with spaces.txt
2900 EOF
2901 $ borg create --exclude-from exclude.txt backup /
2902 A more general and easier to use way to define filename matching pat‐
2904 terns exists with the --pattern and --patterns-from options. Using
2905 these, you may specify the backup roots (starting points) and patterns
2906 for inclusion/exclusion. A root path starts with the prefix R, fol‐
2907 lowed by a path (a plain path, not a file pattern). An include rule
2908 starts with the prefix +, an exclude rule starts with the prefix -, an
2909 exclude-norecurse rule starts with !, all followed by a pattern.
2910 NOTE:
2912 Via --pattern or --patterns-from you can define BOTH inclusion and
2913 exclusion of files using pattern prefixes + and -. With --exclude
2914 and --exclude-from ONLY excludes are defined.
2915 Inclusion patterns are useful to include paths that are contained in an
2917 excluded path. The first matching pattern is used so if an include pat‐
2918 tern matches before an exclude pattern, the file is backed up. If an
2919 exclude-norecurse pattern matches a directory, it won't recurse into it
2920 and won't discover any potential matches for include rules below that
2921 directory.
2922 NOTE:
2924 It's possible that a sub-directory/file is matched while parent di‐
2925 rectories are not. In that case, parent directories are not backed
2926 up thus their user, group, permission, etc. can not be restored.
2927 Note that the default pattern style for --pattern and --patterns-from
2929 is shell style (sh:), so those patterns behave similar to rsync in‐
2930 clude/exclude patterns. The pattern style can be set via the P prefix.
2931 Patterns (--pattern) and excludes (--exclude) from the command line are
2933 considered first (in the order of appearance). Then patterns from
2934 --patterns-from are added. Exclusion patterns from --exclude-from files
2935 are appended last.
2936 Examples:
2938 # backup pics, but not the ones from 2018, except the good ones:
2940 # note: using = is essential to avoid cmdline argument parsing issues.
2941 borg create --pattern=+pics/2018/good --pattern=-pics/2018 repo::arch pics
2942 # use a file with patterns:
2944 borg create --patterns-from patterns.lst repo::arch
2945 The patterns.lst file could look like that:
2947 # "sh:" pattern style is the default, so the following line is not needed:
2949 P sh
2950 R /
2951 # can be rebuild
2952 - home/*/.cache
2953 # they're downloads for a reason
2954 - home/*/Downloads
2955 # susan is a nice person
2956 # include susans home
2957 + home/susan
2958 # also back up this exact file
2959 + pf:home/bobby/specialfile.txt
2960 # don't backup the other home directories
2961 - home/*
2962 # don't even look in /proc
2963 ! proc
2964 You can specify recursion roots either on the command line or in a pat‐
2966 ternfile:
2967 # these two commands do the same thing
2969 borg create --exclude home/bobby/junk repo::arch /home/bobby /home/susan
2970 borg create --patterns-from patternfile.lst repo::arch
2971 The patternfile:
2973 # note that excludes use fm: by default and patternfiles use sh: by default.
2975 # therefore, we need to specify fm: to have the same exact behavior.
2976 P fm
2977 R /home/bobby
2978 R /home/susan
2979 - home/bobby/junk
2981 This allows you to share the same patterns between multiple reposito‐
2983 ries without needing to specify them on the command line.
2984 borg help placeholders
2986 Repository (or Archive) URLs, --prefix, --glob-archives, --comment and
2987 --remote-path values support these placeholders:
2988 {hostname}
2990 The (short) hostname of the machine.
2991 {fqdn} The full name of the machine.
2993 {reverse-fqdn}
2995 The full name of the machine in reverse domain name notation.
2996 {now} The current local date and time, by default in ISO-8601 format.
2998 You can also supply your own format string, e.g.
2999 {now:%Y-%m-%d_%H:%M:%S}
3000 {utcnow}
3002 The current UTC date and time, by default in ISO-8601 format.
3003 You can also supply your own format string, e.g. {utc‐
3004 now:%Y-%m-%d_%H:%M:%S}
3005 {user} The user name (or UID, if no name is available) of the user run‐
3007 ning borg.
3008 {pid} The current process ID.
3010 {borgversion}
3012 The version of borg, e.g.: 1.0.8rc1
3013 {borgmajor}
3015 The version of borg, only the major version, e.g.: 1
3016 {borgminor}
3018 The version of borg, only major and minor version, e.g.: 1.0
3019 {borgpatch}
3021 The version of borg, only major, minor and patch version, e.g.:
3022 1.0.8
3023 If literal curly braces need to be used, double them for escaping:
3025 borg create /path/to/repo::{{literal_text}}
3027 Examples:
3029 borg create /path/to/repo::{hostname}-{user}-{utcnow} ...
3031 borg create /path/to/repo::{hostname}-{now:%Y-%m-%d_%H:%M:%S} ...
3032 borg prune --glob-archives '{hostname}-*' ...
3033 NOTE:
3035 systemd uses a difficult, non-standard syntax for command lines in
3036 unit files (refer to the systemd.unit(5) manual page).
3037 When invoking borg from unit files, pay particular attention to es‐
3039 caping, especially when using the now/utcnow placeholders, since
3040 systemd performs its own %-based variable replacement even in quoted
3041 text. To avoid interference from systemd, double all percent signs
3042 ({hostname}-{now:%Y-%m-%d_%H:%M:%S} becomes {host‐
3043 name}-{now:%%Y-%%m-%%d_%%H:%%M:%%S}).
3044 borg help compression
3046 It is no problem to mix different compression methods in one repo,
3047 deduplication is done on the source data chunks (not on the compressed
3048 or encrypted data).
3049 If some specific chunk was once compressed and stored into the repo,
3051 creating another backup that also uses this chunk will not change the
3052 stored chunk. So if you use different compression specs for the back‐
3053 ups, whichever stores a chunk first determines its compression. See
3054 also borg recreate.
3055 Compression is lz4 by default. If you want something else, you have to
3057 specify what you want.
3058 Valid compression specifiers are:
3060 none Do not compress.
3062 lz4 Use lz4 compression. Very high speed, very low compression. (de‐
3064 fault)
3065 zstd[,L]
3067 Use zstd ("zstandard") compression, a modern wide-range algo‐
3068 rithm. If you do not explicitly give the compression level L
3069 (ranging from 1 to 22), it will use level 3. Archives com‐
3070 pressed with zstd are not compatible with borg < 1.1.4.
3071 zlib[,L]
3073 Use zlib ("gz") compression. Medium speed, medium compression.
3074 If you do not explicitly give the compression level L (ranging
3075 from 0 to 9), it will use level 6. Giving level 0 (means "no
3076 compression", but still has zlib protocol overhead) is usually
3077 pointless, you better use "none" compression.
3078 lzma[,L]
3080 Use lzma ("xz") compression. Low speed, high compression. If
3081 you do not explicitly give the compression level L (ranging from
3082 0 to 9), it will use level 6. Giving levels above 6 is point‐
3083 less and counterproductive because it does not compress better
3084 due to the buffer size used by borg - but it wastes lots of CPU
3085 cycles and RAM.
3086 auto,C[,L]
3088 Use a built-in heuristic to decide per chunk whether to compress
3089 or not. The heuristic tries with lz4 whether the data is com‐
3090 pressible. For incompressible data, it will not use compression
3091 (uses "none"). For compressible data, it uses the given C[,L]
3092 compression - with C[,L] being any valid compression specifier.
3093 obfuscate,SPEC,C[,L]
3095 Use compressed-size obfuscation to make fingerprinting attacks
3096 based on the observable stored chunk size more difficult. Note:
3097 - you must combine this with encryption or it won't make any
3098 sense. - your repo size will be bigger, of course.
3099 The SPEC value will determine how the size obfuscation will
3101 work:
3102 Relative random reciprocal size variation: Size will increase by
3104 a factor, relative to the compressed data size. Smaller factors
3105 are often used, larger factors rarely. 1: factor 0.01 .. 100.0
3106 2: factor 0.1 .. 1000.0 3: factor 1.0 .. 10000.0 4: factor 10.0
3107 .. 100000.0 5: factor 100.0 .. 1000000.0 6: factor 1000.0 ..
3108 10000000.0
3109 Add a randomly sized padding up to the given size: 110: 1kiB ...
3111 120: 1MiB ... 123: 8MiB (max.)
3112 Examples:
3114 borg create --compression lz4 REPO::ARCHIVE data
3116 borg create --compression zstd REPO::ARCHIVE data
3117 borg create --compression zstd,10 REPO::ARCHIVE data
3118 borg create --compression zlib REPO::ARCHIVE data
3119 borg create --compression zlib,1 REPO::ARCHIVE data
3120 borg create --compression auto,lzma,6 REPO::ARCHIVE data
3121 borg create --compression auto,lzma ...
3122 borg create --compression obfuscate,3,none ...
3123 borg create --compression obfuscate,3,auto,zstd,10 ...
3124 borg create --compression obfuscate,2,zstd,6 ...
3125
3127 There is a borg debug command that has some subcommands which are all
3128 not intended for normal use and potentially very dangerous if used in‐
3129 correctly.
3130 For example, borg debug put-obj and borg debug delete-obj will only do
3132 what their name suggests: put objects into repo / delete objects from
3133 repo.
3134 Please note:
3136 • they will not update the chunks cache (chunks index) about the object
3138 • they will not update the manifest (so no automatic chunks index
3140 resync is triggered)
3141 • they will not check whether the object is in use (e.g. before
3143 delete-obj)
3144 • they will not update any metadata which may point to the object
3146 They exist to improve debugging capabilities without direct system ac‐
3148 cess, e.g. in case you ever run into some severe malfunction. Use them
3149 only if you know what you are doing or if a trusted Borg developer
3150 tells you what to do.
3151 Borg has a --debug-topic TOPIC option to enable specific debugging mes‐
3153 sages. Topics are generally not documented.
3154 A --debug-profile FILE option exists which writes a profile of the main
3156 program's execution to a file. The format of these files is not di‐
3157 rectly compatible with the Python profiling tools, since these use the
3158 "marshal" format, which is not intended to be secure (quoting the
3159 Python docs: "Never unmarshal data received from an untrusted or unau‐
3160 thenticated source.").
3161 The borg debug profile-convert command can be used to take a Borg pro‐
3163 file and convert it to a profile file that is compatible with the
3164 Python tools.
3165 Additionally, if the filename specified for --debug-profile ends with
3167 ".pyprof" a Python compatible profile is generated. This is only in‐
3168 tended for local use by developers.
3169
3171 Here are misc. notes about topics that are maybe not covered in enough
3172 detail in the usage section.
3173 --chunker-params
3175 The chunker params influence how input files are cut into pieces
3176 (chunks) which are then considered for deduplication. They also have a
3177 big impact on resource usage (RAM and disk space) as the amount of re‐
3178 sources needed is (also) determined by the total amount of chunks in
3179 the repository (see Indexes / Caches memory usage for details).
3180 --chunker-params=buzhash,10,23,16,4095 results in a fine-grained dedu‐
3182 plication| and creates a big amount of chunks and thus uses a lot of
3183 resources to manage them. This is good for relatively small data vol‐
3184 umes and if the machine has a good amount of free RAM and disk space.
3185 --chunker-params=buzhash,19,23,21,4095 (default) results in a
3187 coarse-grained deduplication and creates a much smaller amount of
3188 chunks and thus uses less resources. This is good for relatively big
3189 data volumes and if the machine has a relatively low amount of free RAM
3190 and disk space.
3191 --chunker-params=fixed,4194304 results in fixed 4MiB sized block dedu‐
3193 plication and is more efficient than the previous example when used for
3194 for block devices (like disks, partitions, LVM LVs) or raw disk image
3195 files.
3196 --chunker-params=fixed,4096,512 results in fixed 4kiB sized blocks, but
3198 the first header block will only be 512B long. This might be useful to
3199 dedup files with 1 header + N fixed size data blocks. Be careful to not
3200 produce a too big amount of chunks (like using small block size for
3201 huge files).
3202 If you already have made some archives in a repository and you then
3204 change chunker params, this of course impacts deduplication as the
3205 chunks will be cut differently.
3206 In the worst case (all files are big and were touched in between back‐
3208 ups), this will store all content into the repository again.
3209 Usually, it is not that bad though:
3211 • usually most files are not touched, so it will just re-use the old
3213 chunks it already has in the repo
3214 • files smaller than the (both old and new) minimum chunksize result in
3216 only one chunk anyway, so the resulting chunks are same and dedupli‐
3217 cation will apply
3218 If you switch chunker params to save resources for an existing repo
3220 that already has some backup archives, you will see an increasing ef‐
3221 fect over time, when more and more files have been touched and stored
3222 again using the bigger chunksize and all references to the smaller
3223 older chunks have been removed (by deleting / pruning archives).
3224 If you want to see an immediate big effect on resource usage, you bet‐
3226 ter start a new repository when changing chunker params.
3227 For more details, see Chunks.
3229 --noatime / --noctime
3231 You can use these borg create options to not store the respective time‐
3232 stamp into the archive, in case you do not really need it.
3233 Besides saving a little space for the not archived timestamp, it might
3235 also affect metadata stream deduplication: if only this timestamp
3236 changes between backups and is stored into the metadata stream, the
3237 metadata stream chunks won't deduplicate just because of that.
3238 --nobsdflags / --noflags
3240 You can use this to not query and store (or not extract and set) flags
3241 - in case you don't need them or if they are broken somehow for your
3242 fs.
3243 On Linux, dealing with the flags needs some additional syscalls. Espe‐
3245 cially when dealing with lots of small files, this causes a noticeable
3246 overhead, so you can use this option also for speeding up operations.
3247 --umask
3249 borg uses a safe default umask of 077 (that means the files borg cre‐
3250 ates have only permissions for owner, but no permissions for group and
3251 others) - so there should rarely be a need to change the default behav‐
3252 iour.
3253 This option only affects the process to which it is given. Thus, when
3255 you run borg in client/server mode and you want to change the behaviour
3256 on the server side, you need to use borg serve --umask=XXX ... as a ssh
3257 forced command in authorized_keys. The --umask value given on the
3258 client side is not transferred to the server side.
3259 Also, if you choose to use the --umask option, always be consistent and
3261 use the same umask value so you do not create a mixup of permissions in
3262 a borg repository or with other files borg creates.
3263 --read-special
3265 The --read-special option is special - you do not want to use it for
3266 normal full-filesystem backups, but rather after carefully picking some
3267 targets for it.
3268 The option --read-special triggers special treatment for block and char
3270 device files as well as FIFOs. Instead of storing them as such a device
3271 (or FIFO), they will get opened, their content will be read and in the
3272 backup archive they will show up like a regular file.
3273 Symlinks will also get special treatment if (and only if) they point to
3275 such a special file: instead of storing them as a symlink, the target
3276 special file will get processed as described above.
3277 One intended use case of this is backing up the contents of one or mul‐
3279 tiple block devices, like e.g. LVM snapshots or inactive LVs or disk
3280 partitions.
3281 You need to be careful about what you include when using --read-spe‐
3283 cial, e.g. if you include /dev/zero, your backup will never terminate.
3284 Restoring such files' content is currently only supported one at a time
3286 via --stdout option (and you have to redirect stdout to where ever it
3287 shall go, maybe directly into an existing device file of your choice or
3288 indirectly via dd).
3289 To some extent, mounting a backup archive with the backups of special
3291 files via borg mount and then loop-mounting the image files from inside
3292 the mount point will work. If you plan to access a lot of data in
3293 there, it likely will scale and perform better if you do not work via
3294 the FUSE mount.
3295 Example
3297 Imagine you have made some snapshots of logical volumes (LVs) you want
3298 to backup.
3299 NOTE:
3301 For some scenarios, this is a good method to get "crash-like" con‐
3302 sistency (I call it crash-like because it is the same as you would
3303 get if you just hit the reset button or your machine would abruptly
3304 and completely crash). This is better than no consistency at all
3305 and a good method for some use cases, but likely not good enough if
3306 you have databases running.
3307 Then you create a backup archive of all these snapshots. The backup
3309 process will see a "frozen" state of the logical volumes, while the
3310 processes working in the original volumes continue changing the data
3311 stored there.
3312 You also add the output of lvdisplay to your backup, so you can see the
3314 LV sizes in case you ever need to recreate and restore them.
3315 After the backup has completed, you remove the snapshots again.
3317 $ # create snapshots here
3319 $ lvdisplay > lvdisplay.txt
3320 $ borg create --read-special /path/to/repo::arch lvdisplay.txt /dev/vg0/*-snapshot
3321 $ # remove snapshots here
3322 Now, let's see how to restore some LVs from such a backup.
3324 $ borg extract /path/to/repo::arch lvdisplay.txt
3326 $ # create empty LVs with correct sizes here (look into lvdisplay.txt).
3327 $ # we assume that you created an empty root and home LV and overwrite it now:
3328 $ borg extract --stdout /path/to/repo::arch dev/vg0/root-snapshot > /dev/vg0/root
3329 $ borg extract --stdout /path/to/repo::arch dev/vg0/home-snapshot > /dev/vg0/home
3330 Separate compaction
3332 Borg does not auto-compact the segment files in the repository at com‐
3333 mit time (at the end of each repository-writing command) any more.
3334 This is new since borg 1.2.0 and requires borg >= 1.2.0 on client and
3336 server.
3337 This causes a similar behaviour of the repository as if it was in ap‐
3339 pend-only mode (see below) most of the time (until borg compact is in‐
3340 voked or an old client triggers auto-compaction).
3341 This has some notable consequences:
3343 • repository space is not freed immediately when deleting / pruning ar‐
3345 chives
3346 • commands finish quicker
3348 • repository is more robust and might be easier to recover after dam‐
3350 ages (as it contains data in a more sequential manner, historic mani‐
3351 fests, multiple commits - until you run borg compact)
3352 • user can choose when to run compaction (it should be done regularly,
3354 but not necessarily after each single borg command)
3355 • user can choose from where to invoke borg compact to do the com‐
3357 paction (from client or from server, it does not need a key)
3358 • less repo sync data traffic in case you create a copy of your reposi‐
3360 tory by using a sync tool (like rsync, rclone, ...)
3361 You can manually run compaction by invoking the borg compact command.
3363 Append-only mode (forbid compaction)
3365 A repository can be made "append-only", which means that Borg will
3366 never overwrite or delete committed data (append-only refers to the
3367 segment files, but borg will also reject to delete the repository com‐
3368 pletely).
3369 If borg compact command is used on a repo in append-only mode, there
3371 will be no warning or error, but no compaction will happen.
3372 append-only is useful for scenarios where a backup client machine back‐
3374 ups remotely to a backup server using borg serve, since a hacked client
3375 machine cannot delete backups on the server permanently.
3376 To activate append-only mode, set append_only to 1 in the repository
3378 config:
3379 borg config /path/to/repo append_only 1
3381 Note that you can go back-and-forth between normal and append-only op‐
3383 eration with borg config; it's not a "one way trip."
3384 In append-only mode Borg will create a transaction log in the transac‐
3386 tions file, where each line is a transaction and a UTC timestamp.
3387 In addition, borg serve can act as if a repository is in append-only
3389 mode with its option --append-only. This can be very useful for
3390 fine-tuning access control in .ssh/authorized_keys:
3391 command="borg serve --append-only ..." ssh-rsa <key used for not-always-trustable backup clients>
3393 command="borg serve ..." ssh-rsa <key used for backup management>
3394 Running borg init via a borg serve --append-only server will not create
3396 an append-only repository. Running borg init --append-only creates an
3397 append-only repository regardless of server settings.
3398 Example
3400 Suppose an attacker remotely deleted all backups, but your repository
3401 was in append-only mode. A transaction log in this situation might look
3402 like this:
3403 transaction 1, UTC time 2016-03-31T15:53:27.383532
3405 transaction 5, UTC time 2016-03-31T15:53:52.588922
3406 transaction 11, UTC time 2016-03-31T15:54:23.887256
3407 transaction 12, UTC time 2016-03-31T15:55:54.022540
3408 transaction 13, UTC time 2016-03-31T15:55:55.472564
3409 From your security logs you conclude the attacker gained access at
3411 15:54:00 and all the backups where deleted or replaced by compromised
3412 backups. From the log you know that transactions 11 and later are com‐
3413 promised. Note that the transaction ID is the name of the last file in
3414 the transaction. For example, transaction 11 spans files 6 to 11.
3415 In a real attack you'll likely want to keep the compromised repository
3417 intact to analyze what the attacker tried to achieve. It's also a good
3418 idea to make this copy just in case something goes wrong during the re‐
3419 covery. Since recovery is done by deleting some files, a hard link copy
3420 (cp -al) is sufficient.
3421 The first step to reset the repository to transaction 5, the last un‐
3423 compromised transaction, is to remove the hints.N, index.N and integ‐
3424 rity.N files in the repository (these files are always expendable). In
3425 this example N is 13.
3426 Then remove or move all segment files from the segment directories in
3428 data/ starting with file 6:
3429 rm data/**/{6..13}
3431 That's all to do in the repository.
3433 If you want to access this rollbacked repository from a client that al‐
3435 ready has a cache for this repository, the cache will reflect a newer
3436 repository state than what you actually have in the repository now, af‐
3437 ter the rollback.
3438 Thus, you need to clear the cache:
3440 borg delete --cache-only repo
3442 The cache will get rebuilt automatically. Depending on repo size and
3444 archive count, it may take a while.
3445 You also will need to remove ~/.config/borg/security/REPOID/mani‐
3447 fest-timestamp.
3448 Drawbacks
3450 As data is only appended, and nothing removed, commands like prune or
3451 delete won't free disk space, they merely tag data as deleted in a new
3452 transaction.
3453 Be aware that as soon as you write to the repo in non-append-only mode
3455 (e.g. prune, delete or create archives from an admin machine), it will
3456 remove the deleted objects permanently (including the ones that were
3457 already marked as deleted, but not removed, in append-only mode). Auto‐
3458 mated edits to the repository (such as a cron job running borg prune)
3459 will render append-only mode moot if data is deleted.
3460 Even if an archive appears to be available, it is possible an attacker
3462 could delete just a few chunks from an archive and silently corrupt its
3463 data. While in append-only mode, this is reversible, but borg check
3464 should be run before a writing/pruning operation on an append-only
3465 repository to catch accidental or malicious corruption:
3466 # run without append-only mode
3468 borg check --verify-data repo && borg compact repo
3469 Aside from checking repository & archive integrity you may want to also
3471 manually check backups to ensure their content seems correct.
3472 Further considerations
3474 Append-only mode is not respected by tools other than Borg. rm still
3475 works on the repository. Make sure that backup client machines only get
3476 to access the repository via borg serve.
3477 Ensure that no remote access is possible if the repository is temporar‐
3479 ily set to normal mode for e.g. regular pruning.
3480 Further protections can be implemented, but are outside of Borg's
3482 scope. For example, file system snapshots or wrapping borg serve to set
3483 special permissions or ACLs on new data files.
3484 SSH batch mode
3486 When running Borg using an automated script, ssh might still ask for a
3487 password, even if there is an SSH key for the target server. Use this
3488 to make scripts more robust:
3489 export BORG_RSH='ssh -oBatchMode=yes'
3491
3493 The Borg Collective (see AUTHORS file)
3494
3496 2010-2022 Jonas Borgström, 2015-2022 The Borg Collective (see AUTHORS
3497 file)
34981.2.3 2022-12-27 BORG(1)