1DUPLICITY(1) User Manuals DUPLICITY(1)
2
6 duplicity - Encrypted backup using rsync algorithm
7
10 duplicity [options] source_directory target_url
11 duplicity [options] source_url target_directory
13 duplicity full [options] source_directory target_url
15 duplicity incremental [options] source_directory target_url
17 duplicity restore [options] source_url target_directory
19 duplicity verify [options] source_url target_directory
21 duplicity collection-status [options] target_url
23 duplicity list-current-files [options] target_url
25 duplicity cleanup [options] [--force] target_url
27 duplicity remove-older-than time [options] [--force] target_url
29 duplicity remove-all-but-n-full count [options] [--force] target_url
31 duplicity remove-all-inc-of-but-n-full count [options] [--force] tar‐
33 get_url
34
37 Duplicity incrementally backs up files and directory by encrypting tar-
38 format volumes with GnuPG and uploading them to a remote (or local)
39 file server. Currently local, ftp, ssh/scp, rsync, WebDAV, WebDAVs,
40 HSi and Amazon S3 backends are available. Because duplicity uses
41 librsync, the incremental archives are space efficient and only record
42 the parts of files that have changed since the last backup. Currently
43 duplicity supports deleted files, full Unix permissions, directories,
44 symbolic links, fifos, etc., but not hard links.
45 If you are backing up the root directory /, remember to --exclude
47 /proc, or else duplicity will probably crash on the weird stuff in
48 there.
49
52 Here is an example of a backup, using scp to back up /home/me to
53 some_dir on the other.host machine:
54 duplicity /home/me scp://uid@other.host/some_dir
56 If the above is run repeatedly, the first will be a full backup, and
58 subsequent ones will be incremental. To force a full backup, use the
59 full action:
60 duplicity full /home/me scp://uid@other.host/some_dir
62 Now suppose we accidentally delete /home/me and want to restore it the
64 way it was at the time of last backup:
65 duplicity scp://uid@other.host/some_dir /home/me
67 Duplicity enters restore mode because the URL comes before the local
69 directory. If we wanted to restore just the file "Mail/article" in
70 /home/me as it was three days ago into /home/me/restored_file:
71 duplicity -t 3D --file-to-restore Mail/article
73 scp://uid@other.host/some_dir /home/me/restored_file
74 The following command compares the files we backed up, so see what has
76 changed since then:
77 duplicity verify scp://uid@other.host/some_dir /home/me
79 Finally, duplicity recognizes several include/exclude options. For
81 instance, the following will backup the root directory, but exclude
82 /mnt, /tmp, and /proc:
83 duplicity --exclude /mnt --exclude /tmp --exclude /proc /
85 file:///usr/local/backup
86 Note that in this case the destination is the local directory
88 /usr/local/backup. The following will backup only the /home and /etc
89 directories under root:
90 duplicity --include /home --include /etc --exclude '**' /
92 file:///usr/local/backup
93 Duplicity can also access a repository via ftp. If a user name is
95 given, the environment variable FTP_PASSWORD is read to determine the
96 password:
97 FTP_PASSWORD=mypassword duplicity /local/dir
99 ftp://user@other.host/some_dir
100
103 cleanup
104 Delete the extraneous duplicity files on the given backend.
105 Non-duplicity files, or files in complete data sets will not be
106 deleted. This should only be necessary after a duplicity ses‐
107 sion fails or is aborted prematurely. Note that --force will be
108 needed to delete the files rather than just list them.
109 collection-status
112 Summarize the status of the backup repository by printing the
113 chains and sets found, and the number of volumes in each.
114 full Indicate full backup. If this is set, perform full backup even
117 if signatures are available.
118 incr If this is requested an incremental backup will be performed.
121 Duplicity will abort if old signatures cannot be found. The
122 default is to switch to full backup under these conditions.
123 list-current-files
126 Lists the files currently backed up in the archive. The infor‐
127 mation will be extracted from the signature files, not the ar‐
128 chive data itself. Thus the whole archive does not have to be
129 downloaded, but on the other hand if the archive has been
130 deleted or corrupted, this command may not detect it.
131 remove-older-than time
134 Delete all backup sets older than the given time. Old backup
135 sets will not be deleted if backup sets newer than time depend
136 on them. See the TIME FORMATS section for more information.
137 Note, this action cannot be combined with backup or other
138 actions, such as cleanup. Note also that --force will be needed
139 to delete the files rather than just list them.
140 remove-all-but-n-full count
143 Delete all backups sets that are older than the count:th last
144 full backup (in other words, keep the last count full backups
145 and associated incremental sets). count must be larger than
146 zero. A value of 1 means that only the single most recent backup
147 chain will be kept. Note that --force will be needed to delete
148 the files rather than just list them.
149 remove-all-inc-of-but-n-full count
152 Delete incremental sets of all backups sets that are older than
153 the count:th last full backup (in other words, keep only old
154 full backups and not their increments). count must be larger
155 than zero. A value of 1 means that only the single most recent
156 backup chain will be kept intact. Note that --force will be
157 needed to delete the files rather than just list them.
158 verify Enter verify mode instead of restore. If the --file-to-restore
161 option is given, restrict verify to that file or directory.
162 duplicity will exit with a non-zero error level if any files are
163 different. On verbosity level 4 or higher, log a message for
164 each file that has changed.
165
168 --allow-source-mismatch
169 Do not abort on attempts to use the same archive dir or remote
170 backend to back up different directories. duplicity will tell
171 you if you need this switch.
172 --archive-dir path
175 The archive directory. NOTE: This option changed in 0.6.0. The
176 archive directory is now necessary in order to manage persis‐
177 tence for current and future enhancements. As such, this option
178 is now used only to change the location of the archive direc‐
179 tory. The archive directory should not be deleted, or duplicity
180 will have to recreate it from the remote repository (which may
181 require decrypting the backup contents).
182 When backing up or restoring, this option specifies that the
184 local archive directory is to be created in path. If the ar‐
185 chive directory is not specified, the default will be to create
186 the archive directory in ~/.cache/duplicity/.
187 The archive directory can be shared between backups to multiple
189 targets, because a subdirectory of the archive dir is used for
190 individual backups (see --name ).
191 The combination of archive directory and backup name must be
193 unique in order to separate the data of different backups.
194 The interaction between the --archive-dir and the --name options
196 allows for four possible combinations for the location of the
197 archive dir:
198 1. neither specified (default)
201 ~/.cache/duplicity/hash-of-url
202 2. --archive-dir=/arch, no --name
204 /arch/hash-of-url
205 3. no --archive-dir, --name=foo
207 ~/.cache/duplicity/foo
208 4. --archive-dir=/arch, --name=foo
210 /arch/foo
211 --asynchronous-upload
214 (EXPERIMENTAL) Perform file uploads asynchronously in the back‐
215 ground, with respect to volume creation. This means that duplic‐
216 ity can upload a volume while, at the same time, preparing the
217 next volume for upload. The intended end-result is a faster
218 backup, because the local CPU and your bandwidth can be more
219 consistently utilized. Use of this option implies additional
220 need for disk space in the temporary storage location; rather
221 than needing to store only one volume at a time, enough storage
222 space is required to store two volumes.
223 --dry-run
226 Calculate what would be done, but do not perform any backend
227 actions
228 --encrypt-key key
231 When backing up, encrypt to the given public key, instead of
232 using symmetric (traditional) encryption. Can be specified mul‐
233 tiple times.
234 --encrypt-secret-keyring filename
237 This option can only be used with --encrypt-key, and changes the
238 path to the secret keyring for the encrypt key to filename This
239 keyring is not used when creating a backup. If not specified,
240 the default secret keyring is used which is usually located at
241 .gnupg/secring.gpg
242 --exclude shell_pattern
245 Exclude the file or files matched by shell_pattern. If a direc‐
246 tory is matched, then files under that directory will also be
247 matched. See the FILE SELECTION section for more information.
248 --exclude-device-files
251 Exclude all device files. This can be useful for security/per‐
252 missions reasons or if rdiff-backup is not handling device files
253 correctly.
254 --exclude-filelist filename
257 Excludes the files listed in filename. See the FILE SELECTION
258 section for more information.
259 --exclude-filelist-stdin
262 Like --exclude-filelist, but the list of files will be read from
263 standard input. See the FILE SELECTION section for more infor‐
264 mation.
265 --exclude-globbing-filelist filename
268 Like --exclude-filelist but each line of the filelist will be
269 interpreted according to the same rules as --include and
270 --exclude.
271 --exclude-if-present filename
274 Exclude directories if filename is present. This option needs to
275 come before any other include or exclude options.
276 --exclude-other-filesystems
279 Exclude files on file systems (identified by device number)
280 other than the file system the root of the source directory is
281 on.
282 --exclude-regexp regexp
285 Exclude files matching the given regexp. Unlike the --exclude
286 option, this option does not match files in a directory it
287 matches. See the FILE SELECTION section for more information.
288 --extra-clean
291 When cleaning up, be more aggressive about saving space. For
292 example, this may delete signature files for old backup chains.
293 See the cleanup argument for more information.
294 --file-to-restore path
297 This option may be given in restore mode, causing only path to
298 be restored instead of the entire contents of the backup ar‐
299 chive. path should be given relative to the root of the direc‐
300 tory backed up.
301 --full-if-older-than time
304 Perform a full backup if an incremental backup is requested, but
305 the latest full backup in the collection is older than the given
306 time. See the TIME FORMATS section for more information.
307 --force
310 Proceed even if data loss might result. Duplicity will let the
311 user know when this option is required.
312 --ftp-passive
315 Use passive (PASV) data connections. The default is to use pas‐
316 sive, but to fallback to regular if the passive connection fails
317 or times out.
318 --ftp-regular
321 Use regular (PORT) data connections.
322 --gio Use the GIO backend and interpret any URLs as GIO would.
325 --ignore-errors
328 Try to ignore certain errors if they happen. This option is only
329 intended to allow the restoration of a backup in the face of
330 certain problems that would otherwise cause the backup to fail.
331 It is not ever recommended to use this option unless you have a
332 situation where you are trying to restore from backup and it is
333 failing because of an issue which you want duplicity to ignore.
334 Even then, depending on the issue, this option may not have an
335 effect.
336 Please note that while ignored errors will be logged, there will
338 be no summary at the end of the operation to tell you what was
339 ignored, if anything. If this is used for emergency restoration
340 of data, it is recommended that you run the backup in such a way
341 that you can revisit the backup log (look for lines containing
342 the string IGNORED_ERROR).
343 If you ever have to use this option for reasons that are not
345 understood or understood but not your own responsibility, please
346 contact duplicity maintainers. The need to use this option under
347 production circumstances would normally be considered a bug.
348 --imap-mailbox option
351 Allows you to specify a different mailbox. The default is
352 "INBOX". Other languages may require a different mailbox than
353 the default.
354 --gpg-options options
357 Allows you to pass options to gpg encryption. The options list
358 should be of the form "opt1=parm1 opt2=parm2" where the string
359 is quoted and the only spaces allowed are between options.
360 --include shell_pattern
363 Similar to --exclude but include matched files instead. Unlike
364 --exclude, this option will also match parent directories of
365 matched files (although not necessarily their contents). See
366 the FILE SELECTION section for more information.
367 --include-filelist filename
370 Like --exclude-filelist, but include the listed files instead.
371 See the FILE SELECTION section for more information.
372 --include-filelist-stdin
375 Like --include-filelist, but read the list of included files
376 from standard input.
377 --include-globbing-filelist filename
380 Like --include-filelist but each line of the filelist will be
381 interpreted according to the same rules as --include and
382 --exclude.
383 --include-regexp regexp
386 Include files matching the regular expression regexp. Only
387 files explicitly matched by regexp will be included by this
388 option. See the FILE SELECTION section for more information.
389 --log-fd number
392 Write specially-formatted versions of output messages to the
393 specified file descriptor. The format used is designed to be
394 easily consumable by other programs.
395 --log-file filename
398 Write specially-formatted versions of output messages to the
399 specified file. The format used is designed to be easily con‐
400 sumable by other programs.
401 --name symbolicname
404 Set the symbolic name of the backup being operated on. The
405 intent is to use a separate name for each logically distinct
406 backup. For example, someone may use "home_daily_s3" for the
407 daily backup of a home directory to Amazon S3. The structure of
408 the name is up to the user, it is only important that the names
409 be distinct. The symbolic name is currently only used to affect
410 the expansion of --archive-dir , but may be used for additional
411 features in the future. Users running more than one distinct
412 backup are encouraged to use this option.
413 If not specified, the default value is a hash of the backend
415 URL.
416 --no-encryption
419 Do not use GnuPG to encrypt files on remote system. Instead
420 just write gzipped volumes.
421 --no-print-statistics
424 By default duplicity will print statistics about the current
425 session after a successful backup. This switch disables that
426 behavior.
427 --null-separator
430 Use nulls (\0) instead of newlines (\n) as line separators,
431 which may help when dealing with filenames containing newlines.
432 This affects the expected format of the files specified by the
433 --{include|exclude}-filelist[-stdin] switches as well as the
434 format of the directory statistics file.
435 --num-retries number
438 Number of retries to make on errors before giving up.
439 --old-filenames
442 Use the old filename format (incompatible with Windows/Samba)
443 rather than the new filename format.
444 --rename orig new
447 Treats the path orig in the backup as if it were the path new.
448 Can be passed multiple times. An example:
449 duplicity restore --rename Documents/metal Music/metal
451 scp://uid@other.host/some_dir /home/me
452 --s3-european-buckets
454 When using the Amazon S3 backend, create buckets in Europe
455 instead of the default (requires --s3-use-new-style ). Also see
456 the EUROPEAN S3 BUCKETS section.
457 --s3-unencrypted-connection
460 Don't use SSL for connections to S3.
461 This may be much faster, at some cost to confidentiality.
463 With this option, anyone who can observe traffic between your
465 computer and S3 will be able to tell: that you are using Duplic‐
466 ity, the name of the bucket, your AWS Access Key ID, the incre‐
467 ment dates and the amount of data in each increment.
468 This option affects only the connection, not the GPG encryption
470 of the backup increment files. Unless that is disabled, an
471 observer will not be able to see the file names or contents.
472 --s3-use-new-style
475 When operating on Amazon S3 buckets, use new-style subdomain
476 bucket addressing. This is now the preferred method to access
477 Amazon S3, but is not backwards compatible if your bucket name
478 contains upper-case characters or other characters that are not
479 valid in a hostname.
480 --scp-command command
482 This option only matters when using the ssh/scp backend. The
483 command will be used instead of scp to send or receive files.
484 The default command is "scp". To list and delete existing files,
485 the sftp command is used. See --ssh-options and --sftp-command.
486 --sftp-command command
489 This option only matters when using the ssh/scp backend. The
490 command will be used instead of sftp for listing and deleting
491 files. The default is "sftp". File transfers are done using the
492 sftp command. See --ssh-options, --use-scp, and --scp-command.
493 --sign-key key
496 This option can be used when backing up or restoring. When
497 backing up, all backup files will be signed with keyid key.
498 When restoring, duplicity will signal an error if any remote
499 file is not signed with the given keyid. key should be an 8
500 character hex string, like AA0E73D2.
501 --ssh-askpass
504 Tells the ssh/scp backend to use FTP_PASSWORD from the environ‐
505 ment, or, if that is not present, to prompt the user for the
506 remote system password.
507 --ssh-options options
510 Allows you to pass options to the ssh/scp/sftp backend. The
511 options list should be of the form "opt1=parm1 opt2=parm2" where
512 the option string is quoted and the only spaces allowed are
513 between options. The option string will be passed verbatim to
514 both scp and sftp, whose command line syntax differs slightly:
515 options passed with --ssh-options should therefore be given in
516 the long option format described in ssh_config(5), like in this
517 example:
518 duplicity --ssh-options="-oProtocol=2 -oIdentity‐
520 File=/my/backup/id" /home/me scp://uid@other.host/some_dir
521 --short-filenames
523 If this option is specified, the names of the files duplicity
524 writes will be shorter (about 30 chars) but less understandable.
525 This may be useful when backing up to MacOS or another OS or FS
526 that doesn't support long filenames.
527 --tempdir directory
530 Use this existing directory for duplicity temporary files
531 instead of the system default, which is usually the /tmp direc‐
532 tory. This option supersedes any environment variable.
533 -ttime, --time time, --restore-time time
536 Specify the time from which to restore or list files.
537 --time-separator char
540 Use char as the time separator in filenames instead of colon
541 (":").
542 --timeout seconds
545 Use seconds as the socket timeout value if duplicity begins to
546 timeout during network operations. The default is 30 seconds.
547 --use-agent
550 If this option is specified, then --use-agent is passed to the
551 GnuPG encryption process and it will turn off any passphrase
552 interaction with the user with respect to --encrypt-key or
553 --sign-key.
554 --use-scp
557 If this option is specified, then the ssh backend will use scp
558 rather than sftp for the get and put backend operations. The
559 default is to use sftp for all operations. With this option,
560 duplicity will use sftp for list and delete operations, and scp
561 for put and get operations
562 -vverb, --verbosity verb
565 Specify verbosity level (0 is total silent, 4 is the default,
566 and 9 is noisiest). Verbosity may also be one of: character
567 ewnid, or word error, warning, notice, info, debug. The default
568 is 4 (Notice). The options -v4, -vn, and -vnotice are function‐
569 ally equivalent, as are the mixed/upper-case versions, -vN,
570 -vNotice, and -vNOTICE.
571 --version
574 Print duplicity's version and quit.
575 --volsize number
578 Change the volume size to number Mb. Default is 25Mb.
579
582 TMPDIR, TEMP, TMP
583 In decreasing order of importance, specifies the directory to
584 use for temporary files (inherited from Python's tempfile mod‐
585 ule).
586 FTP_PASSWORD
588 Supported by most backends which are password capable. More
589 secure than setting it in the backend url (which might be read‐
590 able in the operating systems process listing to other users on
591 the same machine).
592 PASSPHRASE
594 This passphrase is passed to GnuPG. If this is not set, the user
595 will be prompted for the passphrase.
596 SIGN_PASSPHRASE
598 The passphrase to be used for --sign-key , if SIGN_PASSPHRASE is
599 not set but PASSPHRASE is set, the latter will be used. Other‐
600 wise, if no passphrase is available, the user will be prompted
601 for it.
602
605 Duplicity tries to maintain a standard URL format as much as possible.
606 The generic format for a URL is:
607 scheme://user[:password]@host[:port]/[/]path
609 It is not recommended to expose the password on the command line since
611 it could be revealed to anyone with permissions to do process listings,
612 however, it is permitted. Consider setting the environment variable
613 FTP_PASSWORD instead, which is supported by most, but not all backends.
614 Regardless of its name it can be used with other backends.
615 In protocols that support it, the path may be preceded by a single
617 slash, '/path', to represent a relative path to the target home direc‐
618 tory, or preceded by a double slash, '//path', to represent an absolute
619 filesystem path.
620 Formats of each of the URL schemes follow:
622 cf+http://container_name
624 file:///some_dir
626 ftp://user[:password]@other.host[:port]/some_dir
628 hsi://user[:password]@other.host/some_dir
630 imap://user[:password]@host.com[/from_address_prefix]
632 imaps://user[:password]@host.com[/from_address_prefix]
634 using rsync daemon
636 rsync://user[:password]@host.com[:port]::[/]module/some_dir
637 using rsync over ssh (only key auth)
639 rsync://user@host.com[:port]/relative_path
640 rsync://user@host.com[:port]//absolute_path
641 s3://host/bucket_name[/prefix]
643 s3+http://bucket_name[/prefix]
645 Ubuntu One
647 u1://host/volume_path
648 u1+http://volume_path
649 ssh protocols
651 scp://.. or sftp://.. are synonymous for
652 ssh://user[:password]@other.host[:port]/[/]some_dir
653 see also A NOTE ON SSH/SCP PROTOCOLS and --use-scp
654 tahoe://alias/directory
656 webdav://user[:password]@other.host/some_dir
658 webdavs://user[:password]@other.host/some_dir
660
664 duplicity uses time strings in two places. Firstly, many of the files
665 duplicity creates will have the time in their filenames in the w3 date‐
666 time format as described in a w3 note at http://www.w3.org/TR/NOTE-
667 datetime. Basically they look like "2001-07-15T04:09:38-07:00", which
668 means what it looks like. The "-07:00" section means the time zone is
669 7 hours behind UTC.
670 Secondly, the -t, --time, and --restore-time options take a time
672 string, which can be given in any of several formats:
673 1. the string "now" (refers to the current time)
675 2. a sequences of digits, like "123456890" (indicating the time in
677 seconds after the epoch)
678 3. A string like "2002-01-25T07:00:00+02:00" in datetime format
680 4. An interval, which is a number followed by one of the characters
682 s, m, h, D, W, M, or Y (indicating seconds, minutes, hours,
683 days, weeks, months, or years respectively), or a series of such
684 pairs. In this case the string refers to the time that preceded
685 the current time by the length of the interval. For instance,
686 "1h78m" indicates the time that was one hour and 78 minutes ago.
687 The calendar here is unsophisticated: a month is always 30 days,
688 a year is always 365 days, and a day is always 86400 seconds.
689 5. A date format of the form YYYY/MM/DD, YYYY-MM-DD, MM/DD/YYYY, or
691 MM-DD-YYYY, which indicates midnight on the day in question,
692 relative to the current time zone settings. For instance,
693 "2002/3/5", "03-05-2002", and "2002-3-05" all mean March 5th,
694 2002.
695
698 duplicity accepts the same file selection options rdiff-backup does,
699 including --exclude, --exclude-filelist-stdin, etc.
700 When duplicity is run, it searches through the given source directory
702 and backs up all the files specified by the file selection system. The
703 file selection system comprises a number of file selection conditions,
704 which are set using one of the following command line options:
705 --exclude, --exclude-device-files, --exclude-filelist, --exclude-
706 filelist-stdin, --exclude-globbing-filelist, --exclude-regexp,
707 --include, --include-filelist, --include-filelist-stdin, --include-
708 globbing-filelist, and --include-regexp. Each file selection condition
709 either matches or doesn't match a given file. A given file is excluded
710 by the file selection system exactly when the first matching file
711 selection condition specifies that the file be excluded; otherwise the
712 file is included.
713 For instance,
715 duplicity --include /usr --exclude /usr /usr
717 scp://user@host/backup
718 is exactly the same as
720 duplicity /usr scp://user@host/backup
722 because the include and exclude directives match exactly the same
724 files, and the --include comes first, giving it precedence. Similarly,
725 duplicity --include /usr/local/bin --exclude /usr/local /usr
727 scp://user@host/backup
728 would backup the /usr/local/bin directory (and its contents), but not
730 /usr/local/doc.
731 The include, exclude, include-globbing-filelist, and exclude-globbing-
733 filelist options accept extended shell globbing patterns. These pat‐
734 terns can contain the special patterns *, **, ?, and [...]. As in a
735 normal shell, * can be expanded to any string of characters not con‐
736 taining "/", ? expands to any character except "/", and [...] expands
737 to a single character of those characters specified (ranges are accept‐
738 able). The new special pattern, **, expands to any string of charac‐
739 ters whether or not it contains "/". Furthermore, if the pattern
740 starts with "ignorecase:" (case insensitive), then this prefix will be
741 removed and any character in the string can be replaced with an upper-
742 or lowercase version of itself.
743 Remember that you may need to quote these characters when typing them
745 into a shell, so the shell does not interpret the globbing patterns
746 before duplicity sees them.
747 The --exclude pattern option matches a file iff:
749 1. pattern can be expanded into the file's filename, or
752 2. the file is inside a directory matched by the option.
755 Conversely, --include pattern matches a file iff:
757 1. pattern can be expanded into the file's filename,
760 2. the file is inside a directory matched by the option, or
763 3. the file is a directory which contains a file matched by the
766 option.
767 For example,
769 --exclude /usr/local
771 matches /usr/local, /usr/local/lib, and /usr/local/lib/netscape. It is
773 the same as --exclude /usr/local --exclude '/usr/local/**'.
774 --include /usr/local
776 specifies that /usr, /usr/local, /usr/local/lib, and
778 /usr/local/lib/netscape (but not /usr/doc) all be backed up. Thus you
779 don't have to worry about including parent directories to make sure
780 that included subdirectories have somewhere to go. Finally,
781 --include ignorecase:'/usr/[a-z0-9]foo/*/**.py'
783 would match a file like /usR/5fOO/hello/there/world.py. If it did
785 match anything, it would also match /usr. If there is no existing file
786 that the given pattern can be expanded into, the option will not match
787 /usr.
788 The --include-filelist, --exclude-filelist, --include-filelist-stdin,
790 and --exclude-filelist-stdin options also introduce file selection con‐
791 ditions. They direct duplicity to read in a file, each line of which
792 is a file specification, and to include or exclude the matching files.
793 Lines are separated by newlines or nulls, depending on whether the
794 --null-separator switch was given. Each line in a filelist is inter‐
795 preted similarly to the way extended shell patterns are, with a few
796 exceptions:
797 1. Globbing patterns like *, **, ?, and [...] are not expanded.
800 2. Include patterns do not match files in a directory that is
803 included. So /usr/local in an include file will not match
804 /usr/local/doc.
805 3. Lines starting with "+ " are interpreted as include directives,
808 even if found in a filelist referenced by --exclude-filelist.
809 Similarly, lines starting with "- " exclude files even if they
810 are found within an include filelist.
811 For example, if file "list.txt" contains the lines:
813 /usr/local
815 - /usr/local/doc
816 /usr/local/bin
817 + /var
818 - /var
819 then "--include-filelist list.txt" would include /usr, /usr/local, and
821 /usr/local/bin. It would exclude /usr/local/doc,
822 /usr/local/doc/python, etc. It neither excludes nor includes
823 /usr/local/man, leaving the fate of this directory to the next specifi‐
824 cation condition. Finally, it is undefined what happens with /var. A
825 single file list should not contain conflicting file specifications.
826 The --include-globbing-filelist and --exclude-globbing-filelist options
828 also specify filelists, but each line in the filelist will be inter‐
829 preted as a globbing pattern the way --include and --exclude options
830 are interpreted (although "+ " and "- " prefixing is still allowed).
831 For instance, if the file "globbing-list.txt" contains the lines:
832 dir/foo
834 + dir/bar
835 - **
836 Then "--include-globbing-filelist globbing-list.txt" would be exactly
838 the same as specifying "--include dir/foo --include dir/bar --exclude
839 **" on the command line.
840 Finally, the --include-regexp and --exclude-regexp allow files to be
842 included and excluded if their filenames match a python regular expres‐
843 sion. Regular expression syntax is too complicated to explain here,
844 but is covered in Python's library reference. Unlike the --include and
845 --exclude options, the regular expression options don't match files
846 containing or contained in matched files. So for instance
847 --include '[0-9]{7}(?!foo)'
849 matches any files whose full pathnames contain 7 consecutive digits
851 which aren't followed by 'foo'. However, it wouldn't match /home even
852 if /home/ben/1234567 existed.
853
856 This section describes duplicity's basic operation and the format of
857 its data files. It should not necessary to read this section to use
858 duplicity.
859 The files used by duplicity to store backup data are tarfiles in GNU
861 tar format. They can be produced independently by rdiffdir(1). For
862 incremental backups, new files are saved normally in the tarfile. But
863 when a file changes, instead of storing a complete copy of the file,
864 only a diff is stored, as generated by rdiff(1). If a file is deleted,
865 a 0 length file is stored in the tar. It is possible to restore a
866 duplicity archive "manually" by using tar and then cp, rdiff, and rm as
867 necessary. These duplicity archives have the extension difftar.
868 Both full and incremental backup sets have the same format. In effect,
870 a full backup set is an incremental one generated from an empty signa‐
871 ture (see below). The files in full backup sets will start with
872 duplicity-full while the incremental sets start with duplicity-inc.
873 When restoring, duplicity applies patches in order, so deleting, for
874 instance, a full backup set may make related incremental backup sets
875 unusable.
876 In order to determine which files have been deleted, and to calculate
878 diffs for changed files, duplicity needs to process information about
879 previous sessions. It stores this information in the form of tarfiles
880 where each entry's data contains the signature (as produced by rdiff)
881 of the file instead of the file's contents. These signature sets have
882 the extension sigtar.
883 Signature files are not required to restore a backup set, but without
885 an up-to-date signature, duplicity cannot append an incremental backup
886 to an existing archive.
887 To save bandwidth, duplicity generates full signature sets and incre‐
889 mental signature sets. A full signature set is generated for each full
890 backup, and an incremental one for each incremental backup. These
891 start with duplicity-full-signatures and duplicity-new-signatures
892 respectively. These signatures will be stored both locally and
893 remotely. The remote signatures will be encrypted if encryption is
894 enabled. The local signatures will not be encrypted and stored in the
895 archive dir (see --archive-dir ).
896
899 Amazon S3 provides the ability to choose the location of a bucket upon
900 its creation. The purpose is to enable the user to choose a location
901 which is better located network topologically relative to the user,
902 because it may allow for faster data transfers.
903 duplicity will create a new bucket the first time a bucket access is
905 attempted. At this point, the bucket will be created in Europe if
906 --s3-european-buckets was given. For reasons having to do with how the
907 Amazon S3 service works, this also requires the use of the --s3-use-
908 new-style option. This option turns on subdomain based bucket address‐
909 ing in S3. The details are beyond the scope of this man page, but it is
910 important to know that your bucket must not contain upper case letters
911 or any other characters that are not valid parts of a hostname. Conse‐
912 quently, for reasons of backwards compatibility, use of subdomain based
913 bucket addressing is not enabled by default.
914 Note that you will need to use --s3-use-new-style for all operations on
916 European buckets; not just upon initial creation.
917 You only need to use --s3-european-buckets upon initial creation, but
919 you may may use it at all times for consistency.
920 Further note that when creating a new European bucket, it can take a
922 while before the bucket is fully accessible. At the time of this writ‐
923 ing it is unclear to what extent this is an expected feature of Amazon
924 S3, but in practice you may experience timeouts, socket errors or HTTP
925 errors when trying to upload files to your newly created bucket. Give
926 it a few minutes and the bucket should function normally.
927
930 Connecting to Ubuntu One requires that you be running duplicity inside
931 of an X session so that you can be prompted for your credentials if
932 necessary by the Ubuntu One session daemon.
933 See https://one.ubuntu.com/ for more information about Ubuntu One.
935
937 An IMAP account can be used as a target for the upload. The userid may
938 be specified and the password will be requested.
939 The from_address_prefix may be specified (and probably should be). The
941 text will be used as the "From" address in the IMAP server. Then on a
942 restore (or list) command the from_address_prefix will distinguish
943 between different backups.
944
946 Duplicity specifies two protocol names for the same protocol. This is
947 a known and user-confusing issue. Both use the same protocol suite,
948 namely ssh through its' utility routines scp and sftp. Older versions
949 of duplicity used scp for get and put operations and sftp for list and
950 delete operations. The current version uses sftp for all four sup‐
951 ported operations, unless the --use-scp option is used to revert to old
952 behavior. The change was made to all-sftp in order to allow the remote
953 system to chroot the backup, thus providing better security.
954
957 Hard links currently unsupported (they will be treated as non-linked
958 regular files).
959 Bad signatures will be treated as empty instead of logging appropriate
961 error message.
962 If symmetric encryption is used and the signing key is passphrase-pro‐
964 tected, the encryption passphrase must equal the passphrase of the
965 signing key. This limitation can be circumvented by using gpg-agent for
966 storing the passphrase of the signing key and the PASSPHRASE environ‐
967 ment variable for the encryption key or by enabling asymmetric encryp‐
968 tion using the --encrypt-key option.
969
972 Original Author - Ben Escoto <bescoto@stanford.edu>
973 Current Maintainer - Kenneth Loafman <kenneth@loafman.com>
975
978 rdiffdir(1), python(1), rdiff(1), rdiff-backup(1).
979Version 0.6.14 June 18, 2011 DUPLICITY(1)