1DIRVISH.CONF(5)               File Formats Manual              DIRVISH.CONF(5)
2
3
4

NAME

6       dirvish.conf - dirvish configuration file.
7

DESCRIPTION

9       The  dirvish.conf  file  provides configuration information and default
10       values for dirvish.
11
12       The file format is fairly simple.  Each option requires either  a  sin‐
13       gle-value  or  a  list of values and unless otherwise indicated must be
14       specified according to its expected type.   Single  value  options  are
15       specified  by  lines of the form option: value.  Options expecting list
16       must be specified in a multi-line format as shown here where the  lines
17       specifying  values  are indented by any kind of whitespace even if only
18       one value is being specified.
19
20            option:
21                 value1
22                 value2
23                 .
24                 .
25                 .
26                 valueN
27        Each value must be provided on its own line.  Any leading and trailing
28       whitespace  is  discarded.  Options whose names with an initial capital
29       (ex: Foo) are discarded by dirvish itself but may be  used  by  support
30       utilities.  Blank lines are ignored.
31
32       While  this  simplistic  format  may  allow for configuration errors it
33       allows arbitrary options to be declared  that  custom  support  scripts
34       could use.
35
36       A # introduces a comment to the end of the line.
37
38       On  startup the dirvish utilities will first load a master dirvish.conf
39       file.  /etc/dirvish.conf  will  be  tried  first  but  if  not  present
40       /etc/dirvish/master.conf will be tried.
41
42       During installation dirvish may have been configured expect the system-
43       wide configuration files in some location other than /etc/dirvish.
44
45       Multiple configuration files will be loaded by the --vault and --branch
46       command-line  options  as well as the config: and client: configuration
47       parameters.  To prevent looping each configuration  file  can  only  be
48       loaded once.
49
50

DIRVISH OPTIONS

52       Like the command line each option may be specified any number of times.
53       Those options that expect lists will accumulate all of their  arguments
54       and  for single value options each specification will override the ones
55       before.
56
57       Boolean values need to specified as 1 or 0 or may  be  specified  using
58       SET  or  UNSET.   Some  Boolean  values  are set by default and must be
59       explicitly unset if unwanted.
60
61       Each option is marked here with one of  (B)  for  Boolean,  (S)  single
62       value, (L) list or (0) other.
63
64
65       SET option option ... (O)
66
67       UNSET option option ... (O)
68              Set or unset one or more boolean options.
69
70              NOTE: The SET and UNSET directives do not use colons <:>.
71
72       RESET option (O)
73              Reset a list option so that it contains no values.
74
75              This may be used to start specification of the option.
76
77              NOTE: The RESET directive does not use a colon <:>.
78
79       bank: (L)
80              Specify paths to directories containing vaults.
81
82              A bank is a directory containing one or more vaults.  The system
83              supports multiple banks so that filesystem mount-points  can  be
84              managed more effectively.
85
86              When  a  vault  is  specified the banks will be searched in list
87              order until the vault is found.  This way vaults  can  be  moved
88              between banks or added without having to update a master index.
89
90              Multiple bank: values will accumulate.
91
92       branch: branch_name (S)
93              Specify a branch to use.
94
95              A branch is a sequence of images.
96
97              This also specifies a default value for reference:.
98
99              Setting  this in a config file may cause the command line option
100              to be overridden.  Use branch-default: instead.
101
102       branch-default: branch_name (S)
103              Specify a default branch to use.
104
105       client: [username@]client_name (S)
106              specify a client to back up.
107
108              Setting this to the same value as hostname will cause dirvish to
109              do  a  local  copy and stay off the network.  This automatically
110              invokes whole-file.
111
112              The first time this parameter is set /etc/dirvish/client_name or
113              /etc/dirvish/client_name.conf will be loaded.
114
115       checksum: (B)
116              Force  the  checksum  comparison  of file contents even when the
117              inode fails to indicate a change has occurred.
118
119              Default value: 0
120
121       config: filename (S)
122              Load configuration file.
123
124              Similar to #include, filename or filename.conf  will  be  loaded
125              immediately.
126
127              If  filename  is  a  relative  path it will be looked for in the
128              vault and then the system-wide configuration directories.
129
130
131       devices: (B)
132              If this is unset device special  files  will  be  excluded  from
133              backups.
134
135              This may need to be unset when doing backups of where the client
136              OS uses device numbers or types unsupported by the server OSs or
137              where  the presence of device nodes in the vault present a secu‐
138              rity issue.
139
140
141       exclude: (L)
142              Specify a filename patterns to exclude.
143
144              Patterns are based on shell glob with some enhancements.
145
146              See rsync(1) for more details.
147
148              Multiple exclude: values will accumulate.
149
150       file-exclude: filename (S)
151              Load a set of patterns from a file.
152
153              If filename is a relative path it will  be  looked  for  in  the
154              vault and then the system-wide configuration directories.
155
156       expire: expire_date (S)
157              Specify a time for the image to expire.
158
159              This  does not actually expire anything.  What it does do is add
160              an Expire: option to the image summary file  with  the  absolute
161              time  appended  so  that  dirvish-expire  can automate old image
162              removal.
163
164              Setting this in a config file may cause the command line  option
165              to be overridden.  Use expire-rule: and expire-default: instead.
166
167              See Time::ParseDate(3pm) for more details.
168
169       expire-default: expire_date (S)
170              Specify a default expiration time.
171
172              This  value  will  only  be  used  if  expire  is  not  set  and
173              expire-rule doesn't have a match.
174
175       expire-rule: (L)
176              specify rules for expiration.
177
178              Rules are specified similar to crontab or in Time::Periodformat.
179
180              See EXPIRE RULES for more details.
181
182              Multiple expire-rule: values will accumulate.
183
184       image: image_name (S)
185              Specify a name for the image.
186
187              image_name is passed through POSIX::strftime
188
189              Setting this in a config file may cause the command line  option
190              to be overridden.  Use image-default: instead.
191
192              See strftime(3) for more details.
193
194       image-default: image_name (S)
195              Set the default image_name.
196
197              This value will only be used if image: is not set.
198
199       image-perm: octal_mode (S)
200              Set the permissions for the image.
201
202              While the image is being created the image directory permissions
203              will be 0700.  After completion it will be changed to octal_mode
204              or 0755.
205
206              See chmod(1) and umask(2) for more details.
207
208       image-time: parsedate_expression (S)
209              Time to use when creating the image name.
210
211              If an absolute time without a date is provided it will be forced
212              into the past.
213
214              If this isn't set the current time will be used.
215
216              See Time::ParseDate(3pm) for more details.
217
218       image-temp: dirname (S)
219              Temporary directory name to use for new image.  This allows  you
220              to  have images created with the same directory name each run so
221              that automatic processes can access them.
222
223              The next time an image is made on the branch  this  option  will
224              cause the directory to be renamed to its official name.
225
226       index: none|text|gzip|bzip2 (S)
227              Create an index file listing all files in the image.
228
229              The  index  file will be created using find -ls so the list will
230              be in the same format as ls-dils with paths converted to reflect
231              the source location.
232
233              If index is set to bzip2 or gzip or a path to one the index file
234              will be compressed accordingly.
235
236              This index will be used by dirvish-locate to locate versions  of
237              files.  See dirvish-locate(8) for more details.
238
239       init: (B)
240              Create an initial image.
241
242              Turning this on will prevent backups from being incremental.
243
244       log: text|gzip|bzip2 (S)
245              Specify format for the image log file.
246
247              If  log  is  set  to bzip2 or gzip or a path to one the log file
248              will be compressed accordingly.
249
250       meta-perm: octal-mode (S)
251              Set the permissions for the image meta-data files.
252
253              If this value is set the permissions of the meta-data  files  in
254              the image will be changed after the image is created.  Otherwise
255              the active umask will prevail.
256
257              SECURITY NOTE: The log, index, and error files contain lists  of
258              files.   It  may be possible that filenames themselves may be or
259              contain confidential information so uncontrolled access may con‐
260              stitute a security weakness.
261
262              See chmod(1) and umask(2) for more details.
263
264       numeric-ids: (B)
265              Use  numeric  uid/gid  values  instead  of looking up user/group
266              names for setting permissions.
267
268              See rsync(1) for more details.
269
270              Default value: 1
271
272       password-file: filepath (S)
273              Specify file containing password for connection to an rsync dae‐
274              mon on backup client.
275
276              This is not useful for remote shell passwords.
277
278              See --password-file in rsync(1) for more details.
279
280       permissions: (B)
281              Preserve  file  permissions.   If this is unset permissions will
282              not be checked or preserved.
283
284              With rsync version 2.5.6 not preserving permissions  will  break
285              the linking.  Only unset this if you are running a later version
286              of rsync.
287
288              See rsync(1) for more details.
289
290              Default value: 1
291
292       pre-server: shell_command (S)
293
294       pre-client: shell_command (S)
295
296       post-client: shell_command (S)
297
298       post-server: shell_command (S)
299              Execute shell_command on client or server before or after making
300              backup.
301
302              The  client  commands  are  run  on  the client system using the
303              remote shell command (see the rsh: parameter).
304
305              The  order  of  execution  is  pre-server,  pre-client,   rsync,
306              post-client,  post-server.   The  shell_command  will  be passed
307              through strftime(3) to allow date strings to be expanded.
308
309              Each pre or post shell_commands will be run with these  environ‐
310              ment   variables  DIRVISH_SERVER,  DIRVISH_CLIENT,  DIRVISH_SRC,
311              DIRVISH_DEST and DIRVISH_IMAGE set.  The current directory  will
312              be DIRVISH_SRC on the client and DIRVISH_DEST on the server.  If
313              there are any exclude patterns defined the pre-server shell com‐
314              mand  will  also have the exclude file's path in DIRVISH_EXCLUDE
315              so it may read or modify the exlude list.
316
317              STDOUT from each shell_command will be written to the image  log
318              file.
319
320              The exit status of each script will be checked.  Non-zero values
321              will be recognised  as  failure  and  logged.   Failure  of  the
322              pre-server command will halt all further action.  Failure of the
323              pre-client command will prevent the rsync from running  and  the
324              post-server command, if any, will be run.
325
326              Post  shell_commands  will  also have DIRVISH_STATUS set to suc‐
327              cess, warning, error, or fatal error.
328
329              This is useful for multiple things.  The  client  shell_commands
330              can  be  used  to  stop and start services so their files can be
331              backed up safely.  You might use post-server: to schedule repli‐
332              cation or a tape backup of the new image.  Use your imagination.
333
334       reference: branch_name|image_name (S)
335              Specify  an  existing image or a branch from which to create the
336              new image.
337
338              If a branch_name is specified, the last existing image from  its
339              history  file  will be used.  A branch will take precedence over
340              an image of the same name.
341
342              If this isn't specified the  branch  name  will  be  used  as  a
343              default value.
344
345       rsh: command (S)
346              Remote shell utility.
347
348              This can be used to specify the location of ssh or rsh and/or to
349              provide addition options for said utility such as  -p  port  for
350              ssh to use an alternate port number.
351
352              If not specified ssh will be used.
353
354              This  remote  shell command will be used not only as the default
355              rsync transport but also for any pre-client and post-client com‐
356              mands.
357
358       rsync: command (S)
359              Path to rsync executable on the server.
360
361       rsync-client: command (S)
362              Path to rsync executable on the client.
363
364       rsync-option: (L)
365              Specify additional options for the rsync command.
366
367              Only one option per list item is supported.
368
369              This allows you to use rsync features that are not directly sup‐
370              ported by dirvish.  Where dirvish does support an rsync  feature
371              it  is probably better to use the the dirvish supplied mechanism
372              for setting it.
373
374              Multiple rsync-options: values will accumulate.
375
376       sparse: (B)
377              Try to handle sparse files efficiently  so  they  take  up  less
378              space in the vault.
379
380              NOTE:  Some filesystem types may have problems seeking over null
381              regions.
382
383       speed-limit: Mbps (S)
384              Specify a maximum transfer rate.
385
386              This allows you to limit the network  bandwidth  consumed.   The
387              value  is  specified  in  approximate Mega-bits per second which
388              correlates to network  transport  specifications.   An  adaptive
389              algorithm  is used so the actual bandwidth usage may exceed Mbps
390              occasionally.
391
392              See --bwlimit in rsync(1) for more details.
393
394       stats: (B)
395              Have rsync report transfer statistics.
396
397              See rsync(1) for more details.
398
399              Default value: 1
400
401       summary: short|long (S)
402              Specify summary format.
403
404              A short summary will only include final  used  values.   A  long
405              summary will include all configuration values.
406
407              With  long  format you custom options in the configuration files
408              will appear in the summary.
409
410              The default is short.
411
412       tree: path [alias] (S)
413              Specify a directory path on the client to backup.
414
415              If path is prefixed with a colon the transfer will be done  from
416              an  rsync  daemon  on  the client otherwise the transfer will be
417              done through a remote shell process.
418
419              The optional alias specifies the path that should appear in  the
420              index  so dirvish-locate will report paths consistant with  com‐
421              mon usage.  This can help reduce  confusion  when  dealing  with
422              users  unfamiliar  with  the  physical topology of their network
423              provided files.
424
425       no-run: (B)
426              Don't actually do anything.
427
428              Process all configuration files, options and tests then  produce
429              a summary/configuration file on standard output and exit.
430
431              I  can't think why you would do this in a configuration file but
432              if you want to shoot yourself in the foot, be my guest.
433
434       vault: vault (S)
435              Specify the vault to store the image in.
436
437              Although multiple vaults may share a filesystem  a  given  vault
438              cannot  span  filesystems.  For filesystem purposes the vault is
439              the level of atomicity.
440
441              This will seldom be specified in a configuration file.
442
443       whole-file: (B)
444              Transfer whole  files  instead  of  just  the  parts  that  have
445              changed.
446
447              This  may  be  slightly  faster for files that have more changed
448              than left the same such as compressed or  encrypted  files.   In
449              most  cases  this will be slower when transferring over the net‐
450              work but will use less CPU resources.  This will  be  faster  if
451              the  transfers  are  not over the network or when the network is
452              faster than the destination disk subsystem.
453
454       xdev: (B)
455              Do not cross  mount-points  when  traversing  the  tree  on  the
456              client.
457
458       zxfer: (B)
459              Enable compression on data-transfer.
460

SCHEDULING OPTIONS

462       Dirvish: path (S)
463              Location of dirvish executable.
464
465              If not set defaults to dirvish.
466
467       Runall: (L)
468              Specify  branches  to  be scheduled for automated backups.  Each
469              value is specified in the form
470                   vault:branch [image_time]
471
472              If image_time is set here it will be used.
473
474              This option can only be set in the master configuration file and
475              multiple values will accumulate.
476

EXPIRE RULES

478       Expire  rules  is  a list of rules used to determine an expiration time
479       for an image.
480
481       The last rule that matches will apply so  list  order  is  significant.
482       This  allows  rules to be set in client, vault and branch configuration
483       files to override rules set in the master  configuration  file  without
484       having   to   use  RESET.   In  most  cases  it  is  better  to  use  a
485       expire-default: value than to define a rule that matches  all  possible
486       times.
487
488       Each  rule  has an pattern expression against which the current time is
489       compared followed by a date specifier in Time::ParseDate  format.   See
490       Time::ParseDate(3pm) for more details.
491
492       A  matching  rule  with  an  empty/missing date specifier or specifying
493       never will result in no expiration.
494
495       The time pattern expression may be in either crontab or in Time::Period
496       format.  See crontab(5) and Time::Period(3pm) for more details.
497
498       The  crontab  formated patterns are converted to Time::Period format so
499       the limitations and extensions for the specification of  option  values
500       of  Time::Period  apply to the crontab format as well.  Most notable is
501       that the days of the week are numbered 1-7 for sun-sat so 0  is  not  a
502       valid wday but sat
503        is.
504
505       Here are two equivalent examples of an expire-rule list.
506
507            expire-default: +5 weeks
508            expire-rule:
509
510            #MIN  HR    DOM   MON         DOW   EXPIRE
511            *     *     *     *           1     +3 months
512            *     *     1-7   *           su    +1 year
513            *     *     1-7   1,4,7,10    1     never
514            *     10-20 *     *           *     +10 days
515       or:
516            wd { sun }                          +3 months
517            wd { sun } md { 1-7 }               +1 year
518            wd { 1 } md { 1-7 } mo { 1,4,7,10 } never
519            hr { 10-20 }                        +10 days
520
521       This  describes  is  an  aggressive retention schedule.  If the nightly
522       backup is made dated the 1st Sunday of each quarter it is is kept  for‐
523       ever,  the  1st Sunday of any other month is kept for 1 year, all other
524       Sunday's are kept for 3 months, the remaining nightlies are kept for  5
525       weeks.  In addition, if the backup is made between 10AM and 8PM it will
526       expire after 10 days.  This would be appropriate  for  someone  with  a
527       huge  backup  server  who  is so paranoid he makes two backups per day.
528       The other possibility for the hour spec would  be  for  ad-hoc  special
529       backups to have a default that differs from the normal dailies.
530
531       It  should  be  noted  that  all  expiration  rules will do is to cause
532       dirvish  to  put  an  Expire:  option  in  the   summary   file.    The
533       dirvish-expire  utility  will  have  to  be  run to actually delete any
534       expired images.
535
536

FILES

538       /etc/dirvish/master.conf
539              alternate master configuration file.
540
541       /etc/dirvish.conf
542              master configuration file.
543
544       /etc/dirvish/client[.conf]
545              client configuration file.
546
547       bank/vault/dirvish/default[.conf]
548              default vault configuration file.
549
550       bank/vault/dirvish/branch[.conf]
551              branch configuration file.
552
553       bank/vault/dirvish/branch.hist
554              branch history file.
555
556       bank/vault/image/summary
557              image creation summary.
558
559       bank/vault/image/log
560              image creation log.
561
562       bank/vault/image/tree
563              actual image of source directory tree.
564
565       bank/vault/image/rsync_error
566              Error output from rsync if errors or warnings were detected.
567
568

SEE ALSO

570       dirvish(8)
571       dirvish-expire(8)
572       dirvish-runall(8)
573       dirvish-locate(8)
574       ssh(1),
575       rsync(1)
576       Time::ParseDate(3pm)
577       strftime(3)
578

AUTHOR

580       Dirvish was created by J.W. Schultz of Pegasystems Technologies.
581

BUGS

583       Rsync version 2.5.6 has a bug so that unsetting the perms  option  will
584       not  disable testing for permissions.  Disabling perms will break image
585       linking.
586
587       Options set in configuration files will override command  line  options
588       that  have been set before the file is read.  This behaviour while con‐
589       sistent may confuse users.  For this reason the  more  frequently  used
590       command  line  options have options paired with a default option so the
591       order of specification will be more forgiving.  It is recommended  that
592       where  such default options exist in configuration files they should be
593       preferred over the primary option.
594
595       It is possible to specify almost any command line option as  a  option.
596       Some of them just don't make sense to use here.
597
598
599
600                                                               DIRVISH.CONF(5)
Impressum