1HOLLAND(1) Holland HOLLAND(1)
2
6 holland - Holland Documentation
7
9 Introduction to Holland
10 Holland is an Open Source backup framework originally developed by
11 Rackspace and written in Python. It's goal is to help facilitate back‐
12 ing up databases with greater configurability, consistency, and ease.
13 Holland currently focuses on MySQL, however future development will in‐
14 clude other database platforms and even non-database related applica‐
15 tions. Because of it's plugin structure, Holland can be used to backup
16 anything you want by whatever means you want.
17 Python2 support has been removed as of holland 1.2.0
19 Dependencies
21 The core Holland framework has the following dependencies (available on
22 any remotely modern Linux distribution):
23 • Python >= 3.6
25 • pkg_resources
27 • python-setuptools
29 • ConfigParser
31 MySQL based plugins additional require the MySQLdb python connector:
33 • pyMySQL
35 Note that other plugins may have additional dependency requirements.
37 Holland Command-Line Reference
39 Here are the commands available from the 'holland' command-line tool:
40 help (h)
42 Usage: holland <command> --help(-h)
43 Provides basic information about the provided command. If no command is
45 provided, it displays global help instead.
46 backup (bk)
48 Usage: holland backup [backup-set1, backup-set2, ..., backup-setN]
49 Runs the backup operation. If no backup-sets are specified, all active
51 backup-sets (those defined in the 'backupsets' variable in hol‐
52 land.conf) are backed up.
53 One or more backup-sets can be specified directly, in which case only
55 those backup-sets are backed up.
56 Additional Command Line Arguments:
58 --dry-run (-n): Can be used here to simulate, but not actually run, a
60 backup. This should be used when troubleshooting a particular error be‐
61 fore trying to run a real backup.
62 --no-lock (-f): Normally, only one instance of Holland can run at any
64 given time using lock-files. Using this flag causes the lock-files to
65 be ignored. This has some very clear use-cases but otherwise be mindful
66 of using this setting as it can cause backups to fail in some cases.
67 --abort-immediately: abort on the first backup-set that fails (assuming
69 multiple backupsets were specified)
70 Examples:
72 # holland bk --dry-run weekly: Attempts a dry-run of the weekly
74 backup-set.
75 # holland bk --no-lock --abort-immediately: Attempts a backup of all
77 the default backup-sets ignoring locks and aborting immediately if one
78 of the backup-sets fails.
79 list-backups (lb)
81 Usage: holland list-backups
82 Provides extended information about available backups.
84 list-plugins (lp)
86 Usage: holland list-plugins
87 Lists all the available (installed) plugins available to Holland.
89 mk-config (mc)
91 Usage: holland mk-config <provider>
92 Generates a template backup-set for a particular provider (such as
94 mysqldump). By default, the output is sent to standard out but can be
95 copied to a file, either by using the --file, --edit, or -name options
96 (see below).
97 Additional Command Line Arguments:
99 --edit: Load the file into the system text-editor for further modifica‐
101 tions.
102 --file=FILE (-f): Write the output directly to provided file.
104 --name=NAME: Creates a backup-set usable in Holland, which basically
106 means that a file is created of the provided name under the backup-set
107 directory.
108 --provider: Indicates that the default provider configuration should be
110 outputted instead. This is really only used when creating a provider
111 config specifically - it should not be used for backup-sets.
112 Examples:
114 # holland mk-config mysql-lvm > mysql-lvm.conf: Output the default con‐
116 figuration for MySQL-LVM backups and write the contents out to
117 mysql-lvm.conf in the current working directory.
118 # holland mc mysqldump --name=Bob --edit: Create a backup-set using the
120 mysqldump provider named Bob and allow interactive editing of the
121 backup-set before saving the file.
122 purge (pg)
124 Usage: holland purge <backup-set>/<backup-id>
125 Purges old backups by specifying the backup-set name and set-id.
127 For example: # holland purge mybackups/20090502_155438: Purge one of
129 the backups taken on May 2nd, 2009 from the mybackups backup-set.
130 Usage and Implementation Overview
132 Because Holland is very pluggable, it may first seem a bit confusing
133 when it comes to configuring Holland to do something useful. Out of the
134 box, Holland is designed to backup MySQL databases using the mysqldump
135 provider. This is the simplest setup, and may be sufficient for most
136 people. However, others may wish to have more fine-grained control over
137 their backups and/or use another method other than mysqldump.
138 For instance, one can configure a backup set to backup certain data‐
140 bases using mysqldump, others using the mysql-lvm plugins etc. All this
141 is done by a mix of providers and backup-sets.
142 Backup-Sets
144 Each backup-set implements exactly one provider and will inherit the
145 default values of that provider. These values can be overridden to ad‐
146 just the behavior of the backup set. This includes defining what data‐
147 bases or tables to include (or exclude) in the backup, the type of com‐
148 pression used (if any), what locking method to use, among other things.
149 Providers
151 Providers essentially provide a backup service for use in a backup set.
152 • mysqldump
154 Uses the mysqldump utility to backup databases.
155 • MySQL + LVM
157 Backup MySQL databases using LVM snapshots which allows for near
158 lockless or fully lockless (when transactional engines are used)
159 backups.
160 • mysqldump + LVM
162 This plugin creates an LVM snapshot, starts a mysql instance using
163 the snapshot as it's datadir, and then use the mysqldump utility
164 to backup the databases.
165 • XtraBackup
167 New in version 1.0.8.
168 Backup MySQL databases using Percona's XtraBackup tool. This pro‐
171 vides a near lockless backup when using the InnoDB storage engine.
172 • pgdump
174 Backup PostgreSQL databases using the pgdump utility.
175 • pg_basebackup
177 New in version 1.2.8.
178 Backup PostgreSQL databases using the pg_basebackup utility.
181 • mariabackup
183 New in version 1.1.0.
184 Backup MySQL databases using MariaDB's mariabackup tool.
187 • mongodump
189 New in version 1.1.0.
190 This plugin performs logical backups of a MongoDB by using the
193 mongodump utility.
194 • Example
196 This is used solely as a template for designing providers. It oth‐
197 erwise does nothing.
198 As Holland is a framework, it can actually backup most anything as long
200 as there is a provider for it. This includes things that have nothing
201 to do with databases. The idea is to present an easy to use and clear
202 method of backing up and restoring backups no matter the source.
203 Holland Config Files
205 By default, Holland's configuration files reside in /etc/holland. The
206 main configuration file is holland.conf, however there are a number of
207 other configuration files for configuring default settings for
208 providers and for configuring backup sets.
209 Each configuration file has one ore more sections, defined by square
211 brackets Underneath each section, one or more configuration option can
212 be specified. These options are in a standard "option = value" format.
213 Comments are prefixed by the # sign.
214 Note that many settings have default values and, as a result, can ei‐
216 ther be commented out or omitted entirely.
217 holland.conf - main config
219 The main configuration file (usually /etc/holland/holland.conf) defines
220 both global settings as well as the active backup sets. It is divided
221 into two sections [holland] and [logging].
222 [holland]
224 plugin-dirs
225 Defines where the plugins can be found. This can be a comma-sep‐
226 arated list but usually does not need to be modified.
227 backup_directory
229 Top-level directory where backups are held.
230 backupsets
232 A comma-separated list of all the backup sets Holland should
233 backup. Each backup set is defined in /etc/holland/back‐
234 upsets/<name>.conf by default.
235 umask Sets the umask of the resulting backup files.
237 path Defines a path for holland and its spawned processes
239 [logging]
241 filename
242 The log file itself.
243 level Sets the verbosity of Holland's logging process. Available op‐
245 tions are debug, info, warning, error, and critical
246 format Define the format of the log message. This options should be in
248 the format defined by the 'logging' python library.
249 New in version 1.1.0.
251 Provider Configs
254 These files control the global settings / defaults for the providers
255 used by the backup-sets. Many of these global settings can be overrid‐
256 den if defined in a backup-set. Note that each provider's configuration
257 file should begin with [provider-name].
258 mysqldump Provider Configuration [mysqldump]
260 Backs up a MySQL database using the mysqldump tool.
261 [mysqldump]
263 mysql-binpath = /path/to/mysql/bin
264 Defines the location of the MySQL binary utilities. If not provided,
265 Holland will use whatever is in the path.
266 lock-method = flush-lock | lock-tables | single-transaction | auto-de‐
268 tect | none
269 Defines which lock method to use. By default, auto-detect will be
270 used.
271 • flush-lock
273 flush-lock will place a global lock on all tables involved in
274 the backup regardless of whether or not they are in the
275 backup-set. If file-per-database is enabled, then flush-lock
276 will lock all tables for every database being backed up. In
277 other words, this option may not make much sense when using
278 file-per-database.
279 • lock-tables
281 lock-tables will lock all tables involved in the backup. If
282 file-per-database is enabled, then lock-tables will only lock
283 all the tables associated with that database.
284 • single-transaction
286 Forces the use of --single-transaction which enabled
287 semi-transparent backups of transactional tables. Forcing this
288 can cause inconsistencies with non-transactional tables, how‐
289 ever. While non-transactional tables will still lock, they
290 will only lock when they are actually being backed up. Use
291 this setting with extreme caution when backing non-transac‐
292 tional tables.
293 • auto-detect
295 Let Holland decide which option to use by checking to see if a
296 database or backup-set only contains transactional tables. If
297 so, --single-transaction will be used. Otherwise, --lock-ta‐
298 bles will be used.
299 • none
301 Does absolutely no explicit locking when backing up the data‐
302 bases or backup-set. This should only be used when backing up
303 a slave and only after the slave has been turned off (ie, this
304 can be used with the stop-slave option).
305 exclude-invalid-views = yes | no (default: no)
307 Whether to automate exclusion of invalid views that would otherwise
308 cause mysqldump to fail. This adds additional overhead so this op‐
309 tion is not enabled by default.
310 When enabled, thos option will scan the INFORMATION_SCHEMA.VIEWS ta‐
312 ble and execute SHOW FIELDS against each view. If a view is detects
313 as invalid, an ignore-table option will be added to exclude the ta‐
314 ble. Additionally, the plugin will attempt to save the view defin‐
315 ion to 'invalid_views.sql' in the backupset's backup directory.
316 New in version 1.0.8.
318 dump-routines = yes | no (default: yes)
321 Whether or not to backup routines in the backup set directly. Rou‐
322 tines are stored in the 'mysql' database, but it can sometimes be
323 convenient to include them in a backup-set directly.
324 Changed in version 1.0.8: This option now enabled by default.
326 dump-events = yes | no
329 Whether or not to dump events explicitly. Like routines, events are
330 stored in the 'mysql' database. Nonetheless, it can sometimes be
331 convenient to include them in the backup-set directly.
332 Note: This feature requires MySQL 5.1 or later. The mysqldump plugin
334 will automatically disable events if the version of mysqldump is too
335 old.
336 Changed in version 1.0.8: This option is now enabled by default
338 stop-slave = yes | no
341 Stops the SQL_THREAD during the backup. This means that writes from
342 the master will continue to spool but will not be replayed. This
343 helps avoid lock wait timeouts among things while still allowing
344 data to be spooled from the master.
345 Note that previous versions of Holland prior to 1.0.6 simply ran a
347 STOP SLAVE instead, which suspends both replication threads.
348 bin-log-position = yes | no
350 Record the binary log name and position at the time of the backup.
351 The information provied by this option is collected just before
352 locking the database.
353 Note that if both 'stop-slave' and 'bin-log-position' are enabled,
355 Holland will grab the master binary log name and position at the
356 time of the backup which can be useful in using the backup to create
357 slaves or for point in time recovery using the master's binary log.
358 This information is found within the 'backup.conf' file located in
359 the backup-set destination directory (/var/spool/hol‐
360 land/<backup-set>/<backup> by default). For example:
361 [mysql:replication]
363 slave_master_log_pos = 4512
364 slave_master_log_file = 260792-mmm-agent1-bin-log.000001
365 flush-logs = yes | no
367 Whether or not to run FLUSH LOGS in MySQL with the backup. When
368 FLUSH LOGS is actually executed depends on which if database filter‐
369 ing is being used and whether or not file-per-database is enabled.
370 Generally speaking, it does not make sense to use flush-logs with
371 file-per-database since the binary logs will not be consistent with
372 the backup.
373 file-per-database = yes | no
375 Whether or not to split up each database into its own file. Note
376 that it can be more consistent an efficient to backup all databases
377 into one file, however this means that restore a single database can
378 be difficult if multiple databases are defined in the backup set.
379 arg-per-database = JSON object
381 If file-per-database is enable this argument is ued to specify
382 mysqldump arguments per database. It takes a JSON object with the
383 database names as keys. Example: {"employee1": "--no-data"} Adds
384 the '--no-data' argument to the mysqldump command when backing up
385 the 'employee1' database
386 New in version 1.0.9.
388 additional-options = <mysqldump argument>[, <mysqldump argument>]
391 Can optionally specify additional options directly to mysqldump if
392 there is no native Holland option available. This option accepts a
393 comma delimited list of arguments to pass on the commandline.
394 extra-defaults = yes | no (default: no)
396 This option controls whether mysqldump will only read options as set
397 by holland or if additional options from global config files are
398 read. By default, the plugin only uses optons as set in the back‐
399 upset config and includes authentication credentials only from the
400 [client] section in ~/.my.cnf.
401 estimate-method = plugin | const:<size> (default: plugin)
403 This option will skip some of the heavyweight queries necessary to
404 calculate the size of tables to be backed up. If a constant size is
405 specified, then only table names are evaluated and only if table
406 filtering is being used. Additionally, engines will be looked up via
407 SHOW CREATE TABLE if lock-method = auto-detect, in order for the
408 plugin to determine if tables are using a transactional storage en‐
409 gine. With 'plugin', the default behavior of reading both size in‐
410 formation and table names from the information schema is used, which
411 may be slow particularly for a large number of tables.
412 Database and Table filtering
414 Database and Table filtering
415 databases = <glob>
416 exclude-databases = <glob>
418 tables = <glob>
420 exclude-tables = <glob>
422 The above options accepts GLOBs in comma-separated lists. Multiple fil‐
424 tering options can be specified. When filtering on tables, be sure to
425 include both the database and table name.
426 Be careful with quotes. Normally these are not needed, but when quotes
428 are necessary, be sure to only quote each filtering statement, as op‐
429 posed to putting quotes around all statements.
430 Below are a few examples of how these can be applied:
432 Default (backup everything):
434 databases = *
436 tables = *
437 Using database inclusion and exclusions:
439 databases = drupal*, smf_forum,
441 exclude-databases = drupal5
442 Including Tables:
444 tables = phpBB.sucks, drupal6.node*, smf_forum.*
446 Excluding Tables:
448 exclude-tables = mydb.uselesstable1, x_cart.*, *.sessions
450 [compression]
452 Specify various compression settings, such as compression utility, com‐
453 pression level, etc.
454 method = gzip| gzip-rsyncable | pigz | bzip2 | pbzip2 | lzop | lzma |
456 gpg | zstd
457 Define which compression method to use. Note that some methods may
458 not be available by default on every system and may need to be com‐
459 piled or installed.
460 For gpg compression, a key should already exist(gpg --gen-key) and
462 default-recipient must be configured in ~/.gnupg/gpg.conf.
463 inline = yes | no
465 Whether or not to pipe the output of the backup command into the
466 compression utility. Enabling this is recommended since it usually
467 only marginally impacts performance, particularly when using a lower
468 compression level.
469 level = 0-9
471 Specify the compression ratio. The lower the number, the lower the
472 compression ratio, but the faster the backup will take. Generally,
473 setting the lever to 1 or 2 results in favorable compression of tex‐
474 tual data and is noticeably faster than the higher levels. Setting
475 the level to 0 effectively disables compression.
476 bin-path = <full path to utility>
478 This only needs to be defined if the compression utility is not in
479 the usual places or not in the system path.
480 options = <string>
482 Add commandline options to the configuration compression command.
483 options = "-Q4"
485 split = yes | no
487 Defautls to no. If set the backup will be piped through the split
488 command. This may be useful for user's with large databases, as
489 some backup systems perform better with many smaller files instead
490 of 1 large one. This defaults to 1GB file size, so this option isn't
491 helpful if your dumps are smaller than that.
492 For python2.6, this option will be disabled if the subprocess32 mod‐
494 ule isn't avaiable.
495 New in version 1.1.13.
497 MySQL connection info [mysql:client]
500 These are optional and, if left undefined, Holland will try to login
501 using the standard .my.cnf conventions.
502 user = <user>
504 The user to connect to MySQL as.
506 password = <password>
508 The password for the MySQL user
510 socket = <socket>
512 The socket file to connect to MySQL with.
514 host = <host>
516 This would be used for connecting to MySQL remotely.
518 port = <port>
520 Used if MySQL is running on a port other than 3306.
522 MySQL LVM Provider Configuration [mysql-lvm]
524 Creates an LVM snapshot of a running MySQL instance and performs a bi‐
525 nary-based backup with minimal locking. MySQL must be running on an LVM
526 volume with reserved space for snapshots. It is highly recommended that
527 this volume be separate from the one storing the resulting backups.
528 [mysql-lvm]
530 snapshot-size = <size-in-MB>
531 The size of the snapshot itself. By default it is 20% of the size of
532 the MySQL LVM mount or the remaining free-space in the Volume Group
533 (if there is less than 20% available) up to 15GB. If snapshot-size
534 is defined, the number represents the size of the snapshot in
535 megabytes.
536 snapshot-name = <name>
538 The name of the snapshot, the default being the name of the MySQL
539 LVM volume + "_snapshot" (ie Storage-MySQL_snapshot)
540 snapshot-mountpoint = <path>
542 Where to mount the snapshot. By default a randomly generated direc‐
543 tory under /tmp is used.
544 innodb-recovery = yes | no (default: no)
546 Whether or not to run an InnoDB recovery operation. This avoids
547 needing to do so during a restore, though will make the backup
548 process itself take longer.
549 force-innodb-backup = yes | no (default: no)
551 Whether to attempt a backup even if the mysql-lvm plugin thinks it
552 cannot obtain a good backup. This can occur when innodb data files
553 are outside of the mysql datadir or exist on entirely separate logi‐
554 cal volumes.
555 lock-tables = yes | no (default: yes)
557 Whether or not to run a FLUSH TABLES WITH READ LOCK to grab various
558 bits of information (such as the binary log name and position). Dis‐
559 abling this requires that binary logging is disabled and InnoDB is
560 being used exclusively. Otherwise, it is possible that the backup
561 could contain crashed tables.
562 extra-flush-tables = yes | no (default: yes)
564 Whether or not to run a FLUSH TABLES before running the full FLUSH
565 TABLES WITH READ LOCK. Should make the FLUSH TABLES WITH READ LOCK
566 operation a bit faster.
567 archive-method = tar | dir (default: tar)
569 Create a tar file of the datadir, or just copy it.
570 [tar]
572 exclude = pattern[, pattern...]
573 Patterns to exclude from archive. These should be relative paths and
575 are almost always relative to the mysql data directory. For instance
576 to exclude binary logs in the data directory from the backup you might
577 specify: exclude = ./bin-log.*, mysql.sock
578 pre-args = <string>
580 Additional arguments to append to the tar commandline before the backup
582 path is specified. This should be the full string as you might specify
583 on the commandline. Shell globbing is not supported.
584 For instance you might add the /etc/my.cnf to the tar archive via:
586 pre-args = -C /etc ./my.cnf
587 post-args = <string>
589 Additional arguments to append to the tar commandline after the backup
591 path is specified. This should be a string exactly as you might spec‐
592 ify on the commandline. Shell globbing is not evaluated.
593 [compression]
595 Specify various compression settings, such as compression utility, com‐
596 pression level, etc.
597 method = gzip| gzip-rsyncable | pigz | bzip2 | pbzip2 | lzop | lzma |
599 gpg | zstd
600 Define which compression method to use. Note that some methods may
601 not be available by default on every system and may need to be com‐
602 piled or installed.
603 For gpg compression, a key should already exist(gpg --gen-key) and
605 default-recipient must be configured in ~/.gnupg/gpg.conf.
606 inline = yes | no
608 Whether or not to pipe the output of the backup command into the
609 compression utility. Enabling this is recommended since it usually
610 only marginally impacts performance, particularly when using a lower
611 compression level.
612 level = 0-9
614 Specify the compression ratio. The lower the number, the lower the
615 compression ratio, but the faster the backup will take. Generally,
616 setting the lever to 1 or 2 results in favorable compression of tex‐
617 tual data and is noticeably faster than the higher levels. Setting
618 the level to 0 effectively disables compression.
619 bin-path = <full path to utility>
621 This only needs to be defined if the compression utility is not in
622 the usual places or not in the system path.
623 options = <string>
625 Add commandline options to the configuration compression command.
626 options = "-Q4"
628 split = yes | no
630 Defautls to no. If set the backup will be piped through the split
631 command. This may be useful for user's with large databases, as
632 some backup systems perform better with many smaller files instead
633 of 1 large one. This defaults to 1GB file size, so this option isn't
634 helpful if your dumps are smaller than that.
635 For python2.6, this option will be disabled if the subprocess32 mod‐
637 ule isn't avaiable.
638 New in version 1.1.13.
640 MySQL connection info [mysql:client]
643 These are optional and, if left undefined, Holland will try to login
644 using the standard .my.cnf conventions.
645 user = <user>
647 The user to connect to MySQL as.
649 password = <password>
651 The password for the MySQL user
653 socket = <socket>
655 The socket file to connect to MySQL with.
657 host = <host>
659 This would be used for connecting to MySQL remotely.
661 port = <port>
663 Used if MySQL is running on a port other than 3306.
665 mysqldump LVM Provider Configuration [mysqldump-lvm]
667 Backs up one or more MySQL databases by creating an LVM snapshot and
668 then starting a instance of MySQL on top of it to then perform a mysql‐
669 dump. This effectively produces a non-blocking logical backup.
670 [mysql-lvm]
672 snapshot-size = <size-in-MB>
673 The size of the snapshot itself. By default it is 20% of the size of
674 the MySQL LVM mount or the remaining free-space in the Volume Group
675 (if there is less than 20% available) up to 15GB. If snapshot-size
676 is defined, the number represents the size of the snapshot in
677 megabytes.
678 snapshot-name = <name>
680 The name of the snapshot, the default being the name of the MySQL
681 LVM volume + "_snapshot" (ie Storage-MySQL_snapshot)
682 snapshot-mountpoint = <path>
684 Where to mount the snapshot. By default a randomly generated direc‐
685 tory under /tmp is used.
686 innodb-recovery = yes | no (default: no)
688 Whether or not to run an InnoDB recovery operation. This avoids
689 needing to do so during a restore, though will make the backup
690 process itself take longer.
691 lock-tables = yes | no (default: yes)
693 Whether or not to run a FLUSH TABLES WITH READ LOCK to grab various
694 bits of information (such as the binary log name and position). Dis‐
695 abling this requires that binary logging is disabled and InnoDB is
696 being used exclusively. Otherwise, it is possible that the backup
697 could contain crashed tables.
698 extra-flush-tables = yes | no (default: yes)
700 Whether or not to run a FLUSH TABLES before running the full FLUSH
701 TABLES WITH READ LOCK. Should make the FLUSH TABLES WITH READ LOCK
702 operation a bit faster.
703 [mysqld]
705 mysqld-exe = <path>[, <path>...] (default: mysqld in PATH,
706 /usr/libexec/mysqld)
707 This provides a list of locations where the mysqld process to use
708 might be found. This is searched in order of entries in this list.
709 user = <name>
711 The --user parameter to use with mysqld.
712 innodb-buffer-pool-size = <size> (default: 128M)
714 How large to size the innodb-buffer-pool-size.
715 tmpdir = <path> (default: system tempdir)
717 Path to the --tmpdir that mysqld should use.
718 log-error = <path> (default: tempdir/holland_lvm.log)
720 Define path for mysqld's error log. The default location get cleaned
721 up by Holland after the backup is complete. This settings allows the
722 user to define the log file in another location and can be useful
723 for debugging issue with the MySQL instance running on the snapshot.
724 New in version 1.0.9.
726 [mysqldump]
729 mysqldump-lvm supports almost all of the options from the mysqldump
730 plugin. --master-data is not supported, as the mysqld process will not
731 read binary logs, so this plugin will automatically disable bin-log-po‐
732 sition, if set.
733 Binary log information from SOHW MASTER STATUS and SHOW SLAVE STATUS is
735 recorded in the ${backup_directory}/backup.conf file under the
736 [mysql:replication] section.
737 [compression]
739 Specify various compression settings, such as compression utility, com‐
740 pression level, etc.
741 method = gzip| gzip-rsyncable | pigz | bzip2 | pbzip2 | lzop | lzma |
743 gpg | zstd
744 Define which compression method to use. Note that some methods may
745 not be available by default on every system and may need to be com‐
746 piled or installed.
747 For gpg compression, a key should already exist(gpg --gen-key) and
749 default-recipient must be configured in ~/.gnupg/gpg.conf.
750 inline = yes | no
752 Whether or not to pipe the output of the backup command into the
753 compression utility. Enabling this is recommended since it usually
754 only marginally impacts performance, particularly when using a lower
755 compression level.
756 level = 0-9
758 Specify the compression ratio. The lower the number, the lower the
759 compression ratio, but the faster the backup will take. Generally,
760 setting the lever to 1 or 2 results in favorable compression of tex‐
761 tual data and is noticeably faster than the higher levels. Setting
762 the level to 0 effectively disables compression.
763 bin-path = <full path to utility>
765 This only needs to be defined if the compression utility is not in
766 the usual places or not in the system path.
767 options = <string>
769 Add commandline options to the configuration compression command.
770 options = "-Q4"
772 split = yes | no
774 Defautls to no. If set the backup will be piped through the split
775 command. This may be useful for user's with large databases, as
776 some backup systems perform better with many smaller files instead
777 of 1 large one. This defaults to 1GB file size, so this option isn't
778 helpful if your dumps are smaller than that.
779 For python2.6, this option will be disabled if the subprocess32 mod‐
781 ule isn't avaiable.
782 New in version 1.1.13.
784 MySQL connection info [mysql:client]
787 These are optional and, if left undefined, Holland will try to login
788 using the standard .my.cnf conventions.
789 user = <user>
791 The user to connect to MySQL as.
793 password = <password>
795 The password for the MySQL user
797 socket = <socket>
799 The socket file to connect to MySQL with.
801 host = <host>
803 This would be used for connecting to MySQL remotely.
805 port = <port>
807 Used if MySQL is running on a port other than 3306.
809 Xtrabackup Provider Configuration [xtrabackup]
811 Backs up a MySQL instance using Percona's Xtrabackup tool.
812 [xtrabackup]
814 global-defaults = <path> (default: /etc/my.cnf)
815 The MySQL configuration file for xtrabackup to parse. This is !in‐
816 clude'd into the my.cnf the xtrabackup plugin generates
817 innobackupex = <name> (default: innobackupex-1.5.1)
819 The path to the innobackupex script to run. If this is a relative
820 path this will be found in holland's environment PATH as configured
821 in /etc/holland/holland.conf.
822 Changed in version 1.1.12: For xtrabackup version 8.0 or greater, this
824 option is ignored
825 ibbackup = <name>
828 The path to the ibbackup command to use. By default, no --ibbackup
829 option is pass to the innobackupex script. Usually innobackupex
830 will detect this by itself and this should not need to be set.
831 stream = tar|xbstream|yes|no (default: tar)
833 Whether to generate a streaming backup.
834 Changed in version 1.0.8: 'tar' and 'xbstream' are now valid options.
836 The old stream = yes is now equivalent to stream = tar and stream = no
837 disables streaming entirely and will result in a normal directory copy
838 with xtrabackup
839 Changed in version 1.1.12: For xtrabackup version 8.0 or
842 greater,'xstream' will be used unless this value is set to 'no'
843 apply-logs = yes | no (default: yes)
846 Whether to run innobackupex --apply-logs at the end of the backup.
847 This is only supported when performing a non-streaming, non-com‐
848 pressed backup. In this case, even if apply-logs = yes (the de‐
849 fault), the prepare stage will be skipped. Even with an uncom‐
850 pressed, non-streaming backup you may want to disable apply-logs if
851 you wish to use incremental backups.
852 New in version 1.0.8.
854 slave-info = yes | no (default: yes)
857 Whether to enable the --slave-info innobackupex option
858 safe-slave-backup = yes | no (default: yes)
860 Whether to enable the --safe-slave-backup innobackupex option.
861 no-lock = yes | no (default: no)
863 Whether to enable the --no-lock innobackupex option
864 tmpdir = <path> (default: ${backup_directory})
866 The path for the innobackupex --tmpdir option. By default this will
867 use the current holland backup directory to workaround the following
868 bug: https://bugs.launchpad.net/percona-xtrabackup/+bug/1007446
869 New in version 1.0.8.
871 additional-options = <option>[, <option>...]
874 A list of additional options to pass to innobackupex. This is a
875 comma separated list of options.
876 pre-command = <command-string>
878 A command to run prior to running this xtrabackup run. This can be
879 used, for instance, to generate a mysqldump schema dump prior to
880 running xtrabackup. instances of ${backup_directory} and ${back‐
881 updir} will be replaced with the current holland backup directory
882 where the xtrabackup data will be stored.
883 strict = yes | no (default: yes)
885 If strict is set to no, add --strict=OFF option to the xtrabackup
886 command. The strict option in xtrabackup prevents errors and warning
887 for invalid command line options.
888 New in version 1.2.10.
889 [compression]
892 Specify various compression settings, such as compression utility, com‐
893 pression level, etc.
894 method = gzip| gzip-rsyncable | pigz | bzip2 | pbzip2 | lzop | lzma |
896 gpg | zstd
897 Define which compression method to use. Note that some methods may
898 not be available by default on every system and may need to be com‐
899 piled or installed.
900 For gpg compression, a key should already exist(gpg --gen-key) and
902 default-recipient must be configured in ~/.gnupg/gpg.conf.
903 inline = yes | no
905 Whether or not to pipe the output of the backup command into the
906 compression utility. Enabling this is recommended since it usually
907 only marginally impacts performance, particularly when using a lower
908 compression level.
909 level = 0-9
911 Specify the compression ratio. The lower the number, the lower the
912 compression ratio, but the faster the backup will take. Generally,
913 setting the lever to 1 or 2 results in favorable compression of tex‐
914 tual data and is noticeably faster than the higher levels. Setting
915 the level to 0 effectively disables compression.
916 bin-path = <full path to utility>
918 This only needs to be defined if the compression utility is not in
919 the usual places or not in the system path.
920 options = <string>
922 Add commandline options to the configuration compression command.
923 options = "-Q4"
925 split = yes | no
927 Defautls to no. If set the backup will be piped through the split
928 command. This may be useful for user's with large databases, as
929 some backup systems perform better with many smaller files instead
930 of 1 large one. This defaults to 1GB file size, so this option isn't
931 helpful if your dumps are smaller than that.
932 For python2.6, this option will be disabled if the subprocess32 mod‐
934 ule isn't avaiable.
935 New in version 1.1.13.
937 MySQL connection info [mysql:client]
940 These are optional and, if left undefined, Holland will try to login
941 using the standard .my.cnf conventions.
942 user = <user>
944 The user to connect to MySQL as.
946 password = <password>
948 The password for the MySQL user
950 socket = <socket>
952 The socket file to connect to MySQL with.
954 host = <host>
956 This would be used for connecting to MySQL remotely.
958 port = <port>
960 Used if MySQL is running on a port other than 3306.
962 pgdump Provider Configuration [pgdump]
964 Backs up a PostgreSQL instance using the pgdump utility.
965 [pgdump]
967 format = custom | tar | plain (default: custom)
968 Defines the --format option for pg_dump. This defaults to --for‐
969 mat=custom. The custom format is required for pg_restore to do par‐
970 tial restore as well as enabling parallel restores. If set to cus‐
971 tom, the --compress option will be passed to pgdump
972 additional-options = <command-string>
974 Pass additional options to the pg_dump command
975 Only the 'level' option will be used if 'format=custom'
977 [compression]
979 Specify various compression settings, such as compression utility, com‐
980 pression level, etc.
981 method = gzip| gzip-rsyncable | pigz | bzip2 | pbzip2 | lzop | lzma |
983 gpg | zstd
984 Define which compression method to use. Note that some methods may
985 not be available by default on every system and may need to be com‐
986 piled or installed.
987 For gpg compression, a key should already exist(gpg --gen-key) and
989 default-recipient must be configured in ~/.gnupg/gpg.conf.
990 inline = yes | no
992 Whether or not to pipe the output of the backup command into the
993 compression utility. Enabling this is recommended since it usually
994 only marginally impacts performance, particularly when using a lower
995 compression level.
996 level = 0-9
998 Specify the compression ratio. The lower the number, the lower the
999 compression ratio, but the faster the backup will take. Generally,
1000 setting the lever to 1 or 2 results in favorable compression of tex‐
1001 tual data and is noticeably faster than the higher levels. Setting
1002 the level to 0 effectively disables compression.
1003 bin-path = <full path to utility>
1005 This only needs to be defined if the compression utility is not in
1006 the usual places or not in the system path.
1007 options = <string>
1009 Add commandline options to the configuration compression command.
1010 options = "-Q4"
1012 split = yes | no
1014 Defautls to no. If set the backup will be piped through the split
1015 command. This may be useful for user's with large databases, as
1016 some backup systems perform better with many smaller files instead
1017 of 1 large one. This defaults to 1GB file size, so this option isn't
1018 helpful if your dumps are smaller than that.
1019 For python2.6, this option will be disabled if the subprocess32 mod‐
1021 ule isn't avaiable.
1022 New in version 1.1.13.
1024 [pgauth]
1027 username = <name>
1028 Username for pg_dump to authenticate with
1029 password = <string>
1031 Password for pg_dump to authenticate with
1032 hostname = <string>
1034 Hostname for pg_dump to connect with
1035 port = <integer>
1037 TCP port for pg_dump to connect on
1038 pg_basebackup Provider Configuration [pg_basebackup]
1040 Backs up a PostgreSQL instance using the pg_basebackup utility.
1041 [pg_basebackup]
1043 format = tar | plain (default: tar)
1044 Defines the --format option for pg_basebackup.
1045 wal-method = none | fetch | plain (default: fetch)
1047 Defines the --wal-method used for pg_basebackup.
1048 checkpoint = none | fast | spread (default: fast)
1050 Defines the --checkpoint used for pg_basebackup.
1051 additional-options = <command-string> (default: none)
1053 Pass additional options to the pg_basebackup command. This will ex‐
1054 pand backup_directory/backupdir variables.
1055 [compression]
1057 Specify various compression settings, such as compression utility, com‐
1058 pression level, etc.
1059 method = gzip| gzip-rsyncable | pigz | bzip2 | pbzip2 | lzop | lzma |
1061 gpg | zstd
1062 Define which compression method to use. Note that some methods may
1063 not be available by default on every system and may need to be com‐
1064 piled or installed.
1065 For gpg compression, a key should already exist(gpg --gen-key) and
1067 default-recipient must be configured in ~/.gnupg/gpg.conf.
1068 inline = yes | no
1070 Whether or not to pipe the output of the backup command into the
1071 compression utility. Enabling this is recommended since it usually
1072 only marginally impacts performance, particularly when using a lower
1073 compression level.
1074 level = 0-9
1076 Specify the compression ratio. The lower the number, the lower the
1077 compression ratio, but the faster the backup will take. Generally,
1078 setting the lever to 1 or 2 results in favorable compression of tex‐
1079 tual data and is noticeably faster than the higher levels. Setting
1080 the level to 0 effectively disables compression.
1081 bin-path = <full path to utility>
1083 This only needs to be defined if the compression utility is not in
1084 the usual places or not in the system path.
1085 options = <string>
1087 Add commandline options to the configuration compression command.
1088 options = "-Q4"
1090 split = yes | no
1092 Defautls to no. If set the backup will be piped through the split
1093 command. This may be useful for user's with large databases, as
1094 some backup systems perform better with many smaller files instead
1095 of 1 large one. This defaults to 1GB file size, so this option isn't
1096 helpful if your dumps are smaller than that.
1097 For python2.6, this option will be disabled if the subprocess32 mod‐
1099 ule isn't avaiable.
1100 New in version 1.1.13.
1102 [pgauth]
1105 username = <name>
1106 Username for pg_basebackup to authenticate with
1107 password = <string>
1109 Password for pg_basebackup to authenticate with
1110 hostname = <string>
1112 Hostname for pg_basebackup to connect with
1113 port = <integer>
1115 TCP port for pg_basebackup to connect on
1116 mongodump Provider Configuration [mongodump]
1118 This plugin performs logical backups of a MongoDB by using the mongo‐
1119 dump utility.
1120 [mongodump]
1122 host = <string>
1123 Hostname for mongodump to connect with
1124 username = <name>
1126 Username for mongodump to authenticate with
1127 password = <string>
1129 Password for mongodump to authenticate with
1130 port = <integer>
1132 TCP port for mongodump to connect on
1133 uri = <string>
1135 Use a connection string instead of a host, username, and pass‐
1136 word
1137 New in version 1.1.14.
1139 [compression]
1142 Specify various compression settings, such as compression utility, com‐
1143 pression level, etc.
1144 method = gzip| gzip-rsyncable | pigz | bzip2 | pbzip2 | lzop | lzma |
1146 gpg | zstd
1147 Define which compression method to use. Note that some methods may
1148 not be available by default on every system and may need to be com‐
1149 piled or installed.
1150 For gpg compression, a key should already exist(gpg --gen-key) and
1152 default-recipient must be configured in ~/.gnupg/gpg.conf.
1153 inline = yes | no
1155 Whether or not to pipe the output of the backup command into the
1156 compression utility. Enabling this is recommended since it usually
1157 only marginally impacts performance, particularly when using a lower
1158 compression level.
1159 level = 0-9
1161 Specify the compression ratio. The lower the number, the lower the
1162 compression ratio, but the faster the backup will take. Generally,
1163 setting the lever to 1 or 2 results in favorable compression of tex‐
1164 tual data and is noticeably faster than the higher levels. Setting
1165 the level to 0 effectively disables compression.
1166 bin-path = <full path to utility>
1168 This only needs to be defined if the compression utility is not in
1169 the usual places or not in the system path.
1170 options = <string>
1172 Add commandline options to the configuration compression command.
1173 options = "-Q4"
1175 split = yes | no
1177 Defautls to no. If set the backup will be piped through the split
1178 command. This may be useful for user's with large databases, as
1179 some backup systems perform better with many smaller files instead
1180 of 1 large one. This defaults to 1GB file size, so this option isn't
1181 helpful if your dumps are smaller than that.
1182 For python2.6, this option will be disabled if the subprocess32 mod‐
1184 ule isn't avaiable.
1185 New in version 1.1.13.
1187 Mariabackup Provider Configuration [mariabackup]
1190 Backs up a MySQL instance using mariabackup tool.
1191 [mariabackup]
1193 global-defaults = <path> (default: /etc/my.cnf)
1194 The MySQL configuration file for mariabackup to parse. This is !in‐
1195 clude'd into the my.cnf the mariabackup plugin generates
1196 innobackupex = <name> (default: innobackupex-1.5.1)
1198 The path to the innobackupex script to run. If this is a relative
1199 path this will be found in holland's environment PATH as configured
1200 in /etc/holland/holland.conf.
1201 ibbackup = <name>
1203 The path to the ibbackup command to use. By default, no --ibbackup
1204 option is pass to the innobackupex script. Usually innobackupex
1205 will detect this by itself and this should not need to be set.
1206 stream = mbstream(default: tar)
1208 Placeholder
1209 apply-logs = yes | no (default: yes)
1211 Whether to run innobackupex --apply-logs at the end of the backup.
1212 This is only supported when performing a non-streaming, non-com‐
1213 pressed backup. In this case, even if apply-logs = yes (the de‐
1214 fault), the prepare stage will be skipped. Even with an uncom‐
1215 pressed, non-streaming backup you may want to disable apply-logs if
1216 you wish to use incremental backups.
1217 slave-info = yes | no (default: yes)
1219 Whether to enable the --slave-info innobackupex option
1220 safe-slave-backup = yes | no (default: yes)
1222 Whether to enable the --safe-slave-backup innobackupex option.
1223 no-lock = yes | no (default: no)
1225 Whether to enable the --no-lock innobackupex option
1226 tmpdir = <path> (default: ${backup_directory})
1228 The path for the innobackupex --tmpdir option.
1229 additional-options = <option>[, <option>...]
1231 A list of additional options to pass to innobackupex. This is a
1232 comma separated list of options.
1233 pre-command = <command-string>
1235 A command to run prior to running this mariabackup run. This can be
1236 used, for instance, to generate a mysqldump schema dump prior to
1237 running mariabackup. instances of ${backup_directory} and ${back‐
1238 updir} will be replaced with the current holland backup directory
1239 where the mariabackup data will be stored.
1240 [compression]
1242 Specify various compression settings, such as compression utility, com‐
1243 pression level, etc.
1244 method = gzip| gzip-rsyncable | pigz | bzip2 | pbzip2 | lzop | lzma |
1246 gpg | zstd
1247 Define which compression method to use. Note that some methods may
1248 not be available by default on every system and may need to be com‐
1249 piled or installed.
1250 For gpg compression, a key should already exist(gpg --gen-key) and
1252 default-recipient must be configured in ~/.gnupg/gpg.conf.
1253 inline = yes | no
1255 Whether or not to pipe the output of the backup command into the
1256 compression utility. Enabling this is recommended since it usually
1257 only marginally impacts performance, particularly when using a lower
1258 compression level.
1259 level = 0-9
1261 Specify the compression ratio. The lower the number, the lower the
1262 compression ratio, but the faster the backup will take. Generally,
1263 setting the lever to 1 or 2 results in favorable compression of tex‐
1264 tual data and is noticeably faster than the higher levels. Setting
1265 the level to 0 effectively disables compression.
1266 bin-path = <full path to utility>
1268 This only needs to be defined if the compression utility is not in
1269 the usual places or not in the system path.
1270 options = <string>
1272 Add commandline options to the configuration compression command.
1273 options = "-Q4"
1275 split = yes | no
1277 Defautls to no. If set the backup will be piped through the split
1278 command. This may be useful for user's with large databases, as
1279 some backup systems perform better with many smaller files instead
1280 of 1 large one. This defaults to 1GB file size, so this option isn't
1281 helpful if your dumps are smaller than that.
1282 For python2.6, this option will be disabled if the subprocess32 mod‐
1284 ule isn't avaiable.
1285 New in version 1.1.13.
1287 MySQL connection info [mysql:client]
1290 These are optional and, if left undefined, Holland will try to login
1291 using the standard .my.cnf conventions.
1292 user = <user>
1294 The user to connect to MySQL as.
1296 password = <password>
1298 The password for the MySQL user
1300 socket = <socket>
1302 The socket file to connect to MySQL with.
1304 host = <host>
1306 This would be used for connecting to MySQL remotely.
1308 port = <port>
1310 Used if MySQL is running on a port other than 3306.
1312 Backup-Set Configs
1314 Backup-Set configuration files largely inherit the configuration op‐
1315 tions of the specified provider. To define a provider for the backup
1316 set, you must put the following at the top of the backup set configura‐
1317 tion file:
1318 [holland:backup]
1320 plugin = <plugin>
1321 backups-to-keep = #
1322 estimated-size-factor = #
1323 historic-size = <yes|no>
1324 historic-size-factor = #
1325 historic-estimated-size-factor = #
1326 Configuration Options
1328 plugin = <plugin>
1329 This is the name of the provider that will be used for the
1330 backup-set. This is required in order for the backup-set to func‐
1331 tion.
1332 backups-to-keep = #
1334 Specifies the number of backups to keep for a backup-set. Defaults
1335 to retaining 1 backup.
1336 estimated-size-factor = #
1338 Specifies the scale factor when Holland decides if there is enough
1339 free space to perform a backup. The default is 1.0 and this number
1340 is multiplied against what each individual plugin reports its esti‐
1341 mated backup size when Holland is verifying sufficient free space
1342 for the backupset.
1343 As of Holland 1.1.1, the 'historic' backup size will be used. If
1345 Holland is unable to determine that size, it will default back to
1346 this.
1347 auto-purge-failures = yes | no
1349 Specifies whether to keep a failed backup or to automatically remove
1350 the backup directory. By default this is on with the intention that
1351 whatever process is calling holland will retry when a backup fails.
1352 This behavior can be disabled by setting auto-purge-failures = no
1353 when partial backups might be useful or when troubleshooting a
1354 backup failure.
1355 purge-policy = manual | before-backup | after-backup
1357 Specifies when to run the purge routine on a backupset. By default
1358 this is run after a new successful backup completes. Up to back‐
1359 ups-to-keep backups will be retained including the most recent.
1360 purge-policy = before-backup will run the purge routine just before
1362 a new backup starts. This will retain up to backups-to-keep backups
1363 before the new backup is even started allowing purging all previous
1364 backups if backups-to-keep is set to 0. This behavior is useful if
1365 some other process is retaining backups off-server and disk space is
1366 at a premium.
1367 purge-policy = manual will never run the purge routine automati‐
1369 cally. Either holland purge must be run externally or an explicit
1370 removal of desired backup directories can be done at some later
1371 time.
1372 before-backup-command = string
1374 Run a shell command before a backup starts. This allows a command
1375 to perform some action before the backup starts such as setting up
1376 an iptables rule (taking a mysql slave out of a load balancer) or
1377 aborting the backup based on some external condition.
1378 The backup will fail if this command exits with a non-zero status.
1380 New in version 1.0.7.
1382 after-backup-command = string
1385 Run a shell command after a backup completes. This allows a command
1386 to perform some action when a backup completes successfully such as
1387 sending out a success notification.
1388 The backup will fail if this command exits with a non-zero status.
1390 New in version 1.0.7.
1392 failed-backup-command = string
1395 Run a shell command if a backup starts. This allows some command to
1396 perform some action when a backup fails such as sending out a fail‐
1397 ure notification.
1398 The backup will fail if this command exits with a non-zero status.
1400 New in version 1.0.7.
1402 historic-size = yes | no
1405 Defaults to yes. Check for the 'backup.conf' file in the 'newest'
1406 spooled folder for the running "backupset". If the configuration
1407 file exists and contains 'estimated-size' and 'on-disk-size', these
1408 values will be used to decide if holland has enough free space for a
1409 backupset. This means Holland will use values from the previous
1410 backup to estimate the size of the next backup.
1411 New in version 1.1.1.
1413 historic-size-factor = #
1416 Defaults to '1.5'. If the estimated size of the database has changed
1417 by more than this multiple, the 'estimated-size-factor' value will
1418 be used to determine if there is sufficient free space for the back‐
1419 upset.
1420 New in version 1.1.1.
1422 historic-estimated-size-factor = #
1425 Defaults to '1.1'. Specifies the scale factor when Holland decides
1426 if there is enough free space to perform a backup. Holland will
1427 throw an error if the system has less free space than the last
1428 backup size multiplied by this value
1429 New in version 1.1.1.
1431 create-symlinks = yes | no
1434 Defaults to yes. Enable the creation of "newest" and "oldest" sym‐
1435 links in the backupset directory. Disabling the symlink creation
1436 process and be helpful on filesystems that don't support symlinks.
1437 New in version 1.2.7.
1439 $ ls -l /var/spool/holland/default/
1442 total 0
1443 drwxrwx---. 3 root root 58 Aug 18 16:18 20210818_161856
1444 lrwxrwxrwx. 1 root root 42 Aug 18 16:18 newest -> /var/spool/holland/default/20210818_161856
1445 lrwxrwxrwx. 1 root root 42 Aug 18 16:18 oldest -> /var/spool/holland/default/20210818_161856
1446 For all hook commands, Holland will perform simple text substitution on
1448 the three parameters:
1449 • hook - name of the hook being called (one of: before-backup-com‐
1451 mand, after-backup-command, failed-backup-command)
1452 • backupdir - path to the current backup directory (e.g.
1454 /var/spool/holland/mysqldump/YYYYmmdd_HHMMSS)
1455 • backupset - name of the backupset being run (e.g. 'mysql-lvm')
1457 For example:
1459 [holland:backup]
1461 plugin = mysqldump
1462 before-backup-command = /usr/local/bin/my-custom-script --hook ${hook} --backupset ${backupset} --backupdir ${backupdir}
1463 after-backup-command = echo ${backupset} completed successfully. Files are in ${backupdir}
1464 [mysqldump]
1466 ...
1467 Backup-Set files are defined in the "backupsets" directory which is, by
1469 default, /etc/holland/backupsets. The name of the backup-set is defined
1470 by its configuration filename and can really be most anything. That
1471 means backup-sets can be organized in any arbitrary way, although
1472 backup set files must end in .conf. The file extension is not part of
1473 the name of the backup-set.
1474 As noted above, in order for a backup-set to be active, it must be
1476 listed in the backupsets variable.
1477 Backups are placed under the directory defined in the backup_directory
1479 section of the main configuration file. Each backup resides under a di‐
1480 rectory corresponding to the backup-set name followed by a date-encoded
1481 directory.
1482
1484 Holland Core Team
1485
1487 2022
14881.2.10 Jul 21, 2022 HOLLAND(1)