1DIRVISH.CONF(5) File Formats Manual DIRVISH.CONF(5)
2
6 dirvish.conf - dirvish configuration file.
7
9 The dirvish.conf file provides configuration information and default
10 values for dirvish.
11 The file format is fairly simple. Each option requires either a sin‐
13 gle-value or a list of values and unless otherwise indicated must be
14 specified according to its expected type. Single value options are
15 specified by lines of the form option: value. Options expecting list
16 must be specified in a multi-line format as shown here where the lines
17 specifying values are indented by any kind of whitespace even if only
18 one value is being specified.
19 option:
21 value1
22 value2
23 .
24 .
25 .
26 valueN
27 Each value must be provided on its own line. Any leading and trailing
28 whitespace is discarded. Options whose names with an initial capital
29 (ex: Foo) are discarded by dirvish itself but may be used by support
30 utilities. Blank lines are ignored.
31 While this simplistic format may allow for configuration errors it
33 allows arbitrary options to be declared that custom support scripts
34 could use.
35 A # introduces a comment to the end of the line.
37 On startup the dirvish utilities will first load a master dirvish.conf
39 file. /etc/dirvish.conf will be tried first but if not present
40 /etc/dirvish/master.conf will be tried.
41 During installation dirvish may have been configured expect the system-
43 wide configuration files in some location other than /etc/dirvish.
44 Multiple configuration files will be loaded by the --vault and --branch
46 command-line options as well as the config: and client: configuration
47 parameters. To prevent looping each configuration file can only be
48 loaded once.
49
52 Like the command line each option may be specified any number of times.
53 Those options that expect lists will accumulate all of their arguments
54 and for single value options each specification will override the ones
55 before.
56 Boolean values need to specified as 1 or 0 or may be specified using
58 SET or UNSET. Some Boolean values are set by default and must be
59 explicitly unset if unwanted.
60 Each option is marked here with one of (B) for Boolean, (S) single
62 value, (L) list or (0) other.
63 SET option option ... (O)
66 UNSET option option ... (O)
68 Set or unset one or more boolean options.
69 NOTE: The SET and UNSET directives do not use colons <:>.
71 RESET option (O)
73 Reset a list option so that it contains no values.
74 This may be used to start specification of the option.
76 NOTE: The RESET directive does not use a colon <:>.
78 bank: (L)
80 Specify paths to directories containing vaults.
81 A bank is a directory containing one or more vaults. The system
83 supports multiple banks so that filesystem mount-points can be
84 managed more effectively.
85 When a vault is specified the banks will be searched in list
87 order until the vault is found. This way vaults can be moved
88 between banks or added without having to update a master index.
89 Multiple bank: values will accumulate.
91 branch: branch_name (S)
93 Specify a branch to use.
94 A branch is a sequence of images.
96 This also specifies a default value for reference:.
98 Setting this in a config file may cause the command line option
100 to be overridden. Use branch-default: instead.
101 branch-default: branch_name (S)
103 Specify a default branch to use.
104 client: [username@]client_name (S)
106 specify a client to back up.
107 Setting this to the same value as hostname will cause dirvish to
109 do a local copy and stay off the network. This automatically
110 invokes whole-file.
111 The first time this parameter is set /etc/dirvish/client_name or
113 /etc/dirvish/client_name.conf will be loaded.
114 checksum: (B)
116 Force the checksum comparison of file contents even when the
117 inode fails to indicate a change has occurred.
118 Default value: 0
120 config: filename (S)
122 Load configuration file.
123 Similar to #include, filename or filename.conf will be loaded
125 immediately.
126 If filename is a relative path it will be looked for in the
128 vault and then the system-wide configuration directories.
129 devices: (B)
132 If this is unset device special files will be excluded from
133 backups.
134 This may need to be unset when doing backups of where the client
136 OS uses device numbers or types unsupported by the server OSs or
137 where the presence of device nodes in the vault present a secu‐
138 rity issue.
139 exclude: (L)
142 Specify a filename patterns to exclude.
143 Patterns are based on shell glob with some enhancements.
145 See rsync(1) for more details.
147 Multiple exclude: values will accumulate.
149 file-exclude: filename (S)
151 Load a set of patterns from a file.
152 If filename is a relative path it will be looked for in the
154 vault and then the system-wide configuration directories.
155 expire: expire_date (S)
157 Specify a time for the image to expire.
158 This does not actually expire anything. What it does do is add
160 an Expire: option to the image summary file with the absolute
161 time appended so that dirvish-expire can automate old image
162 removal.
163 Setting this in a config file may cause the command line option
165 to be overridden. Use expire-rule: and expire-default: instead.
166 See Time::ParseDate(3pm) for more details.
168 expire-default: expire_date (S)
170 Specify a default expiration time.
171 This value will only be used if expire is not set and
173 expire-rule doesn't have a match.
174 expire-rule: (L)
176 specify rules for expiration.
177 Rules are specified similar to crontab or in Time::Periodformat.
179 See EXPIRE RULES for more details.
181 Multiple expire-rule: values will accumulate.
183 image: image_name (S)
185 Specify a name for the image.
186 image_name is passed through POSIX::strftime
188 Setting this in a config file may cause the command line option
190 to be overridden. Use image-default: instead.
191 See strftime(3) for more details.
193 image-default: image_name (S)
195 Set the default image_name.
196 This value will only be used if image: is not set.
198 image-perm: octal_mode (S)
200 Set the permissions for the image.
201 While the image is being created the image directory permissions
203 will be 0700. After completion it will be changed to octal_mode
204 or 0755.
205 See chmod(1) and umask(2) for more details.
207 image-time: parsedate_expression (S)
209 Time to use when creating the image name.
210 If an absolute time without a date is provided it will be forced
212 into the past.
213 If this isn't set the current time will be used.
215 See Time::ParseDate(3pm) for more details.
217 image-temp: dirname (S)
219 Temporary directory name to use for new image. This allows you
220 to have images created with the same directory name each run so
221 that automatic processes can access them.
222 The next time an image is made on the branch this option will
224 cause the directory to be renamed to its official name.
225 index: none|text|gzip|bzip2 (S)
227 Create an index file listing all files in the image.
228 The index file will be created using find -ls so the list will
230 be in the same format as ls-dils with paths converted to reflect
231 the source location.
232 If index is set to bzip2 or gzip or a path to one the index file
234 will be compressed accordingly.
235 This index will be used by dirvish-locate to locate versions of
237 files. See dirvish-locate(8) for more details.
238 init: (B)
240 Create an initial image.
241 Turning this on will prevent backups from being incremental.
243 log: text|gzip|bzip2 (S)
245 Specify format for the image log file.
246 If log is set to bzip2 or gzip or a path to one the log file
248 will be compressed accordingly.
249 meta-perm: octal-mode (S)
251 Set the permissions for the image meta-data files.
252 If this value is set the permissions of the meta-data files in
254 the image will be changed after the image is created. Otherwise
255 the active umask will prevail.
256 SECURITY NOTE: The log, index, and error files contain lists of
258 files. It may be possible that filenames themselves may be or
259 contain confidential information so uncontrolled access may con‐
260 stitute a security weakness.
261 See chmod(1) and umask(2) for more details.
263 numeric-ids: (B)
265 Use numeric uid/gid values instead of looking up user/group
266 names for setting permissions.
267 See rsync(1) for more details.
269 Default value: 1
271 password-file: filepath (S)
273 Specify file containing password for connection to an rsync dae‐
274 mon on backup client.
275 This is not useful for remote shell passwords.
277 See --password-file in rsync(1) for more details.
279 permissions: (B)
281 Preserve file permissions. If this is unset permissions will
282 not be checked or preserved.
283 With rsync version 2.5.6 not preserving permissions will break
285 the linking. Only unset this if you are running a later version
286 of rsync.
287 See rsync(1) for more details.
289 Default value: 1
291 pre-server: shell_command (S)
293 pre-client: shell_command (S)
295 post-client: shell_command (S)
297 post-server: shell_command (S)
299 Execute shell_command on client or server before or after making
300 backup.
301 The client commands are run on the client system using the
303 remote shell command (see the rsh: parameter).
304 The order of execution is pre-server, pre-client, rsync,
306 post-client, post-server. The shell_command will be passed
307 through strftime(3) to allow date strings to be expanded.
308 Each pre or post shell_commands will be run with these environ‐
310 ment variables DIRVISH_SERVER, DIRVISH_CLIENT, DIRVISH_SRC,
311 DIRVISH_DEST and DIRVISH_IMAGE set. The current directory will
312 be DIRVISH_SRC on the client and DIRVISH_DEST on the server. If
313 there are any exclude patterns defined the pre-server shell com‐
314 mand will also have the exclude file's path in DIRVISH_EXCLUDE
315 so it may read or modify the exlude list.
316 STDOUT from each shell_command will be written to the image log
318 file.
319 The exit status of each script will be checked. Non-zero values
321 will be recognised as failure and logged. Failure of the
322 pre-server command will halt all further action. Failure of the
323 pre-client command will prevent the rsync from running and the
324 post-server command, if any, will be run.
325 Post shell_commands will also have DIRVISH_STATUS set to suc‐
327 cess, warning, error, or fatal error.
328 This is useful for multiple things. The client shell_commands
330 can be used to stop and start services so their files can be
331 backed up safely. You might use post-server: to schedule repli‐
332 cation or a tape backup of the new image. Use your imagination.
333 reference: branch_name|image_name (S)
335 Specify an existing image or a branch from which to create the
336 new image.
337 If a branch_name is specified, the last existing image from its
339 history file will be used. A branch will take precedence over
340 an image of the same name.
341 If this isn't specified the branch name will be used as a
343 default value.
344 rsh: command (S)
346 Remote shell utility.
347 This can be used to specify the location of ssh or rsh and/or to
349 provide addition options for said utility such as -p port for
350 ssh to use an alternate port number.
351 If not specified ssh will be used.
353 This remote shell command will be used not only as the default
355 rsync transport but also for any pre-client and post-client com‐
356 mands.
357 rsync: command (S)
359 Path to rsync executable on the server.
360 rsync-client: command (S)
362 Path to rsync executable on the client.
363 rsync-option: (L)
365 Specify additional options for the rsync command.
366 Only one option per list item is supported.
368 This allows you to use rsync features that are not directly sup‐
370 ported by dirvish. Where dirvish does support an rsync feature
371 it is probably better to use the the dirvish supplied mechanism
372 for setting it.
373 Multiple rsync-options: values will accumulate.
375 sparse: (B)
377 Try to handle sparse files efficiently so they take up less
378 space in the vault.
379 NOTE: Some filesystem types may have problems seeking over null
381 regions.
382 speed-limit: Mbps (S)
384 Specify a maximum transfer rate.
385 This allows you to limit the network bandwidth consumed. The
387 value is specified in approximate Mega-bits per second which
388 correlates to network transport specifications. An adaptive
389 algorithm is used so the actual bandwidth usage may exceed Mbps
390 occasionally.
391 See --bwlimit in rsync(1) for more details.
393 stats: (B)
395 Have rsync report transfer statistics.
396 See rsync(1) for more details.
398 Default value: 1
400 summary: short|long (S)
402 Specify summary format.
403 A short summary will only include final used values. A long
405 summary will include all configuration values.
406 With long format you custom options in the configuration files
408 will appear in the summary.
409 The default is short.
411 tree: path [alias] (S)
413 Specify a directory path on the client to backup.
414 If path is prefixed with a colon the transfer will be done from
416 an rsync daemon on the client otherwise the transfer will be
417 done through a remote shell process.
418 The optional alias specifies the path that should appear in the
420 index so dirvish-locate will report paths consistant with com‐
421 mon usage. This can help reduce confusion when dealing with
422 users unfamiliar with the physical topology of their network
423 provided files.
424 no-run: (B)
426 Don't actually do anything.
427 Process all configuration files, options and tests then produce
429 a summary/configuration file on standard output and exit.
430 I can't think why you would do this in a configuration file but
432 if you want to shoot yourself in the foot, be my guest.
433 vault: vault (S)
435 Specify the vault to store the image in.
436 Although multiple vaults may share a filesystem a given vault
438 cannot span filesystems. For filesystem purposes the vault is
439 the level of atomicity.
440 This will seldom be specified in a configuration file.
442 whole-file: (B)
444 Transfer whole files instead of just the parts that have
445 changed.
446 This may be slightly faster for files that have more changed
448 than left the same such as compressed or encrypted files. In
449 most cases this will be slower when transferring over the net‐
450 work but will use less CPU resources. This will be faster if
451 the transfers are not over the network or when the network is
452 faster than the destination disk subsystem.
453 xdev: (B)
455 Do not cross mount-points when traversing the tree on the
456 client.
457 zxfer: (B)
459 Enable compression on data-transfer.
460
462 Dirvish: path (S)
463 Location of dirvish executable.
464 If not set defaults to dirvish.
466 Runall: (L)
468 Specify branches to be scheduled for automated backups. Each
469 value is specified in the form
470 vault:branch [image_time]
471 If image_time is set here it will be used.
473 This option can only be set in the master configuration file and
475 multiple values will accumulate.
476
478 Expire rules is a list of rules used to determine an expiration time
479 for an image.
480 The last rule that matches will apply so list order is significant.
482 This allows rules to be set in client, vault and branch configuration
483 files to override rules set in the master configuration file without
484 having to use RESET. In most cases it is better to use a
485 expire-default: value than to define a rule that matches all possible
486 times.
487 Each rule has an pattern expression against which the current time is
489 compared followed by a date specifier in Time::ParseDate format. See
490 Time::ParseDate(3pm) for more details.
491 A matching rule with an empty/missing date specifier or specifying
493 never will result in no expiration.
494 The time pattern expression may be in either crontab or in Time::Period
496 format. See crontab(5) and Time::Period(3pm) for more details.
497 The crontab formated patterns are converted to Time::Period format so
499 the limitations and extensions for the specification of option values
500 of Time::Period apply to the crontab format as well. Most notable is
501 that the days of the week are numbered 1-7 for sun-sat so 0 is not a
502 valid wday but sat
503 is.
504 Here are two equivalent examples of an expire-rule list.
506 expire-default: +5 weeks
508 expire-rule:
509 #MIN HR DOM MON DOW EXPIRE
511 * * * * 1 +3 months
512 * * 1-7 * su +1 year
513 * * 1-7 1,4,7,10 1 never
514 * 10-20 * * * +10 days
515 or:
516 wd { sun } +3 months
517 wd { sun } md { 1-7 } +1 year
518 wd { 1 } md { 1-7 } mo { 1,4,7,10 } never
519 hr { 10-20 } +10 days
520 This describes is an aggressive retention schedule. If the nightly
522 backup is made dated the 1st Sunday of each quarter it is is kept for‐
523 ever, the 1st Sunday of any other month is kept for 1 year, all other
524 Sunday's are kept for 3 months, the remaining nightlies are kept for 5
525 weeks. In addition, if the backup is made between 10AM and 8PM it will
526 expire after 10 days. This would be appropriate for someone with a
527 huge backup server who is so paranoid he makes two backups per day.
528 The other possibility for the hour spec would be for ad-hoc special
529 backups to have a default that differs from the normal dailies.
530 It should be noted that all expiration rules will do is to cause
532 dirvish to put an Expire: option in the summary file. The
533 dirvish-expire utility will have to be run to actually delete any
534 expired images.
535
538 /etc/dirvish/master.conf
539 alternate master configuration file.
540 /etc/dirvish.conf
542 master configuration file.
543 /etc/dirvish/client[.conf]
545 client configuration file.
546 bank/vault/dirvish/default[.conf]
548 default vault configuration file.
549 bank/vault/dirvish/branch[.conf]
551 branch configuration file.
552 bank/vault/dirvish/branch.hist
554 branch history file.
555 bank/vault/image/summary
557 image creation summary.
558 bank/vault/image/log
560 image creation log.
561 bank/vault/image/tree
563 actual image of source directory tree.
564 bank/vault/image/rsync_error
566 Error output from rsync if errors or warnings were detected.
567
570 dirvish(8)
571 dirvish-expire(8)
572 dirvish-runall(8)
573 dirvish-locate(8)
574 ssh(1),
575 rsync(1)
576 Time::ParseDate(3pm)
577 strftime(3)
578
580 Dirvish was created by J.W. Schultz of Pegasystems Technologies.
581
583 Rsync version 2.5.6 has a bug so that unsetting the perms option will
584 not disable testing for permissions. Disabling perms will break image
585 linking.
586 Options set in configuration files will override command line options
588 that have been set before the file is read. This behaviour while con‐
589 sistent may confuse users. For this reason the more frequently used
590 command line options have options paired with a default option so the
591 order of specification will be more forgiving. It is recommended that
592 where such default options exist in configuration files they should be
593 preferred over the primary option.
594 It is possible to specify almost any command line option as a option.
596 Some of them just don't make sense to use here.
597 DIRVISH.CONF(5)