1HOLLAND(1) Holland HOLLAND(1)
2
6 holland - Holland Documentation
7
9 Introduction to Holland
10 Holland is an Open Source backup framework originally developed by
11 Rackspace and written in Python. It's goal is to help facilitate back‐
12 ing up databases with greater configurability, consistency, and ease.
13 Holland currently focuses on MySQL, however future development will in‐
14 clude other database platforms and even non-database related applica‐
15 tions. Because of it's plugin structure, Holland can be used to backup
16 anything you want by whatever means you want.
17 Python2 support has been removed as of holland 1.2.0
19 Dependencies
21 The core Holland framework has the following dependencies (available on
22 any remotely modern Linux distribution):
23 • Python >= 3.6
25 • pkg_resources
27 • python-setuptools
29 • ConfigParser
31 MySQL based plugins additional require the MySQLdb python connector:
33 • pyMySQL
35 Note that other plugins may have additional dependency requirements.
37 Holland Command-Line Reference
39 Here are the commands available from the 'holland' command-line tool:
40 help (h)
42 Usage: holland <command> --help(-h)
43 Provides basic information about the provided command. If no command is
45 provided, it displays global help instead.
46 backup (bk)
48 Usage: holland backup [backup-set1, backup-set2, ..., backup-setN]
49 Runs the backup operation. If no backup-sets are specified, all active
51 backup-sets (those defined in the 'backupsets' variable in hol‐
52 land.conf) are backed up.
53 One or more backup-sets can be specified directly, in which case only
55 those backup-sets are backed up.
56 Additional Command Line Arguments:
58 --dry-run (-n): Can be used here to simulate, but not actually run, a
60 backup. This should be used when troubleshooting a particular error be‐
61 fore trying to run a real backup.
62 --no-lock (-f): Normally, only one instance of Holland can run at any
64 given time using lock-files. Using this flag causes the lock-files to
65 be ignored. This has some very clear use-cases but otherwise be mindful
66 of using this setting as it can cause backups to fail in some cases.
67 --abort-immediately: abort on the first backup-set that fails (assuming
69 multiple backupsets were specified)
70 Examples:
72 # holland bk --dry-run weekly: Attempts a dry-run of the weekly
74 backup-set.
75 # holland bk --no-lock --abort-immediately: Attempts a backup of all
77 the default backup-sets ignoring locks and aborting immediately if one
78 of the backup-sets fails.
79 list-backups (lb)
81 Usage: holland list-backups
82 Provides extended information about available backups.
84 list-plugins (lp)
86 Usage: holland list-plugins
87 Lists all the available (installed) plugins available to Holland.
89 mk-config (mc)
91 Usage: holland mk-config <provider>
92 Generates a template backup-set for a particular provider (such as
94 mysqldump). By default, the output is sent to standard out but can be
95 copied to a file, either by using the --file, --edit, or -name options
96 (see below).
97 Additional Command Line Arguments:
99 --edit: Load the file into the system text-editor for further modifica‐
101 tions.
102 --file=FILE (-f): Write the output directly to provided file.
104 --name=NAME: Creates a backup-set usable in Holland, which basically
106 means that a file is created of the provided name under the backup-set
107 directory.
108 --provider: Indicates that the default provider configuration should be
110 outputted instead. This is really only used when creating a provider
111 config specifically - it should not be used for backup-sets.
112 Examples:
114 # holland mk-config mysql-lvm > mysql-lvm.conf: Output the default con‐
116 figuration for MySQL-LVM backups and write the contents out to
117 mysql-lvm.conf in the current working directory.
118 # holland mc mysqldump --name=Bob --edit: Create a backup-set using the
120 mysqldump provider named Bob and allow interactive editing of the
121 backup-set before saving the file.
122 purge (pg)
124 Usage: holland purge <backup-set>/<backup-id>
125 Purges old backups by specifying the backup-set name and set-id.
127 For example: # holland purge mybackups/20090502_155438: Purge one of
129 the backups taken on May 2nd, 2009 from the mybackups backup-set.
130 Usage and Implementation Overview
132 Because Holland is very pluggable, it may first seem a bit confusing
133 when it comes to configuring Holland to do something useful. Out of the
134 box, Holland is designed to backup MySQL databases using the mysqldump
135 provider. This is the simplest setup, and may be sufficient for most
136 people. However, others may wish to have more fine-grained control over
137 their backups and/or use another method other than mysqldump.
138 For instance, one can configure a backup set to backup certain data‐
140 bases using mysqldump, others using the mysql-lvm plugins etc. All this
141 is done by a mix of providers and backup-sets.
142 Backup-Sets
144 Each backup-set implements exactly one provider and will inherit the
145 default values of that provider. These values can be overridden to ad‐
146 just the behavior of the backup set. This includes defining what data‐
147 bases or tables to include (or exclude) in the backup, the type of com‐
148 pression used (if any), what locking method to use, among other things.
149 Providers
151 Providers essentially provide a backup service for use in a backup set.
152 • mysqldump
154 Uses the mysqldump utility to backup databases.
155 • MySQL + LVM
157 Backup MySQL databases using LVM snapshots which allows for near
158 lockless or fully lockless (when transactional engines are used)
159 backups.
160 • mysqldump + LVM
162 This plugin creates an LVM snapshot, starts a mysql instance using
163 the snapshot as it's datadir, and then use the mysqldump utility
164 to backup the databases.
165 • XtraBackup
167 New in version 1.0.8.
168 Backup MySQL databases using Percona's XtraBackup tool. This pro‐
171 vides a near lockless backup when using the InnoDB storage engine.
172 • pgdump
174 Backup PostgreSQL databases using the pgdump utility.
175 • pg_basebackup
177 New in version 1.2.8.
178 Backup PostgreSQL databases using the pg_basebackup utility.
181 • mariabackup
183 New in version 1.1.0.
184 Backup MySQL databases using MariaDB's mariabackup tool.
187 • mongodump
189 New in version 1.1.0.
190 This plugin performs logical backups of a MongoDB by using the
193 mongodump utility.
194 • Example
196 This is used solely as a template for designing providers. It oth‐
197 erwise does nothing.
198 As Holland is a framework, it can actually backup most anything as long
200 as there is a provider for it. This includes things that have nothing
201 to do with databases. The idea is to present an easy to use and clear
202 method of backing up and restoring backups no matter the source.
203 Holland Config Files
205 By default, Holland's configuration files reside in /etc/holland. The
206 main configuration file is holland.conf, however there are a number of
207 other configuration files for configuring default settings for
208 providers and for configuring backup sets.
209 Each configuration file has one ore more sections, defined by square
211 brackets Underneath each section, one or more configuration option can
212 be specified. These options are in a standard "option = value" format.
213 Comments are prefixed by the # sign.
214 Note that many settings have default values and, as a result, can ei‐
216 ther be commented out or omitted entirely.
217 holland.conf - main config
219 The main configuration file (usually /etc/holland/holland.conf) defines
220 both global settings as well as the active backup sets. It is divided
221 into two sections [holland] and [logging].
222 [holland]
224 plugin-dirs
225 Defines where the plugins can be found. This can be a comma-sep‐
226 arated list but usually does not need to be modified.
227 backup_directory
229 Top-level directory where backups are held.
230 backupsets
232 A comma-separated list of all the backup sets Holland should
233 backup. Each backup set is defined in /etc/holland/back‐
234 upsets/<name>.conf by default.
235 umask Sets the umask of the resulting backup files.
237 path Defines a path for holland and its spawned processes
239 [logging]
241 filename
242 The log file itself.
243 level Sets the verbosity of Holland's logging process. Available op‐
245 tions are debug, info, warning, error, and critical
246 format Define the format of the log message. This options should be in
248 the format defined by the 'logging' python library.
249 New in version 1.1.0.
251 Provider Configs
254 These files control the global settings / defaults for the providers
255 used by the backup-sets. Many of these global settings can be overrid‐
256 den if defined in a backup-set. Note that each provider's configuration
257 file should begin with [provider-name].
258 mysqldump Provider Configuration [mysqldump]
260 Backs up a MySQL database using the mysqldump tool.
261 [mysqldump]
263 mysql-binpath = /path/to/mysql/bin
264 Defines the location of the MySQL binary utilities. If not provided,
265 Holland will use whatever is in the path.
266 lock-method = flush-lock | lock-tables | single-transaction | auto-de‐
268 tect | none
269 Defines which lock method to use. By default, auto-detect will be
270 used.
271 • flush-lock
273 flush-lock will place a global lock on all tables involved in
274 the backup regardless of whether or not they are in the
275 backup-set. If file-per-database is enabled, then flush-lock
276 will lock all tables for every database being backed up. In
277 other words, this option may not make much sense when using
278 file-per-database.
279 • lock-tables
281 lock-tables will lock all tables involved in the backup. If
282 file-per-database is enabled, then lock-tables will only lock
283 all the tables associated with that database.
284 • single-transaction
286 Forces the use of --single-transaction which enabled
287 semi-transparent backups of transactional tables. Forcing this
288 can cause inconsistencies with non-transactional tables, how‐
289 ever. While non-transactional tables will still lock, they
290 will only lock when they are actually being backed up. Use
291 this setting with extreme caution when backing non-transac‐
292 tional tables.
293 • auto-detect
295 Let Holland decide which option to use by checking to see if a
296 database or backup-set only contains transactional tables. If
297 so, --single-transaction will be used. Otherwise, --lock-ta‐
298 bles will be used.
299 • none
301 Does absolutely no explicit locking when backing up the data‐
302 bases or backup-set. This should only be used when backing up
303 a slave and only after the slave has been turned off (ie, this
304 can be used with the stop-slave option).
305 exclude-invalid-views = yes | no (default: no)
307 Whether to automate exclusion of invalid views that would otherwise
308 cause mysqldump to fail. This adds additional overhead so this op‐
309 tion is not enabled by default.
310 When enabled, thos option will scan the INFORMATION_SCHEMA.VIEWS ta‐
312 ble and execute SHOW FIELDS against each view. If a view is detects
313 as invalid, an ignore-table option will be added to exclude the ta‐
314 ble. Additionally, the plugin will attempt to save the view defin‐
315 ion to 'invalid_views.sql' in the backupset's backup directory.
316 New in version 1.0.8.
318 dump-routines = yes | no (default: yes)
321 Whether or not to backup routines in the backup set directly. Rou‐
322 tines are stored in the 'mysql' database, but it can sometimes be
323 convenient to include them in a backup-set directly.
324 Changed in version 1.0.8: This option now enabled by default.
326 dump-events = yes | no
329 Whether or not to dump events explicitly. Like routines, events are
330 stored in the 'mysql' database. Nonetheless, it can sometimes be
331 convenient to include them in the backup-set directly.
332 Note: This feature requires MySQL 5.1 or later. The mysqldump plugin
334 will automatically disable events if the version of mysqldump is too
335 old.
336 Changed in version 1.0.8: This option is now enabled by default
338 stop-slave = yes | no
341 Stops the SQL_THREAD during the backup. This means that writes from
342 the master will continue to spool but will not be replayed. This
343 helps avoid lock wait timeouts among things while still allowing
344 data to be spooled from the master.
345 Note that previous versions of Holland prior to 1.0.6 simply ran a
347 STOP SLAVE instead, which suspends both replication threads.
348 bin-log-position = yes | no
350 Record the binary log name and position at the time of the backup.
351 The information provied by this option is collected just before
352 locking the database.
353 Note that if both 'stop-slave' and 'bin-log-position' are enabled,
355 Holland will grab the master binary log name and position at the
356 time of the backup which can be useful in using the backup to create
357 slaves or for point in time recovery using the master's binary log.
358 This information is found within the 'backup.conf' file located in
359 the backup-set destination directory (/var/spool/hol‐
360 land/<backup-set>/<backup> by default). For example:
361 [mysql:replication]
363 slave_master_log_pos = 4512
364 slave_master_log_file = 260792-mmm-agent1-bin-log.000001
365 flush-logs = yes | no
367 Whether or not to run FLUSH LOGS in MySQL with the backup. When
368 FLUSH LOGS is actually executed depends on which if database filter‐
369 ing is being used and whether or not file-per-database is enabled.
370 Generally speaking, it does not make sense to use flush-logs with
371 file-per-database since the binary logs will not be consistent with
372 the backup.
373 file-per-database = yes | no
375 Whether or not to split up each database into its own file. Note
376 that it can be more consistent an efficient to backup all databases
377 into one file, however this means that restore a single database can
378 be difficult if multiple databases are defined in the backup set.
379 arg-per-database = JSON object
381 If file-per-database is enable this argument is ued to specify
382 mysqldump arguments per database. It takes a JSON object with the
383 database names as keys. Example: {"employee1": "--no-data"} Adds
384 the '--no-data' argument to the mysqldump command when backing up
385 the 'employee1' database
386 New in version 1.0.9.
388 additional-options = <mysqldump argument>[, <mysqldump argument>]
391 Can optionally specify additional options directly to mysqldump if
392 there is no native Holland option available. This option accepts a
393 comma delimited list of arguments to pass on the commandline.
394 extra-defaults = yes | no (default: no)
396 This option controls whether mysqldump will only read options as set
397 by holland or if additional options from global config files are
398 read. By default, the plugin only uses optons as set in the back‐
399 upset config and includes authentication credentials only from the
400 [client] section in ~/.my.cnf.
401 estimate-method = plugin | const:<size> (default: plugin)
403 This option will skip some of the heavyweight queries necessary to
404 calculate the size of tables to be backed up. If a constant size is
405 specified, then only table names are evaluated and only if table
406 filtering is being used. Additionally, engines will be looked up via
407 SHOW CREATE TABLE if lock-method = auto-detect, in order for the
408 plugin to determine if tables are using a transactional storage en‐
409 gine. With 'plugin', the default behavior of reading both size in‐
410 formation and table names from the information schema is used, which
411 may be slow particularly for a large number of tables.
412 Database and Table filtering
414 Database and Table filtering
415 databases = <glob>
416 exclude-databases = <glob>
418 tables = <glob>
420 exclude-tables = <glob>
422 The above options accepts GLOBs in comma-separated lists. Multiple fil‐
424 tering options can be specified. When filtering on tables, be sure to
425 include both the database and table name.
426 Be careful with quotes. Normally these are not needed, but when quotes
428 are necessary, be sure to only quote each filtering statement, as op‐
429 posed to putting quotes around all statements.
430 Below are a few examples of how these can be applied:
432 Default (backup everything):
434 databases = *
436 tables = *
437 Using database inclusion and exclusions:
439 databases = drupal*, smf_forum,
441 exclude-databases = drupal5
442 Including Tables:
444 tables = phpBB.sucks, drupal6.node*, smf_forum.*
446 Excluding Tables:
448 exclude-tables = mydb.uselesstable1, x_cart.*, *.sessions
450 [compression]
452 Specify various compression settings, such as compression utility, com‐
453 pression level, etc.
454 method = gzip| gzip-rsyncable | pigz | bzip2 | pbzip2 | lzop | lzma |
456 gpg | zstd
457 Define which compression method to use. Note that some methods may
458 not be available by default on every system and may need to be com‐
459 piled or installed.
460 For gpg compression, a key should already exist(gpg --gen-key) and
462 default-recipient must be configured in ~/.gnupg/gpg.conf.
463 inline = yes | no
465 Whether or not to pipe the output of the backup command into the
466 compression utility. Enabling this is recommended since it usually
467 only marginally impacts performance, particularly when using a lower
468 compression level.
469 level = 0-9
471 Specify the compression ratio. The lower the number, the lower the
472 compression ratio, but the faster the backup will take. Generally,
473 setting the lever to 1 or 2 results in favorable compression of tex‐
474 tual data and is noticeably faster than the higher levels. Setting
475 the level to 0 effectively disables compression.
476 bin-path = <full path to utility>
478 This only needs to be defined if the compression utility is not in
479 the usual places or not in the system path.
480 options = <string>
482 Add commandline options to the configuration compression command.
483 options = "-Q4"
485 split = yes | no
487 Defautls to no. If set the backup will be piped through the split
488 command. This may be useful for user's with large databases, as
489 some backup systems perform better with many smaller files instead
490 of 1 large one. This defaults to 1GB file size, so this option isn't
491 helpful if your dumps are smaller than that.
492 For python2.6, this option will be disabled if the subprocess32 mod‐
494 ule isn't avaiable.
495 New in version 1.1.13.
497 MySQL connection info [mysql:client]
500 These are optional and, if left undefined, Holland will try to login
501 using the standard .my.cnf conventions.
502 user = <user>
504 The user to connect to MySQL as.
506 password = <password>
508 The password for the MySQL user
510 socket = <socket>
512 The socket file to connect to MySQL with.
514 host = <host>
516 This would be used for connecting to MySQL remotely.
518 port = <port>
520 Used if MySQL is running on a port other than 3306.
522 MySQL LVM Provider Configuration [mysql-lvm]
524 Creates an LVM snapshot of a running MySQL instance and performs a bi‐
525 nary-based backup with minimal locking. MySQL must be running on an LVM
526 volume with reserved space for snapshots. It is highly recommended that
527 this volume be separate from the one storing the resulting backups.
528 [mysql-lvm]
530 snapshot-size = <size-in-MB>
531 The size of the snapshot itself. By default it is 20% of the size of
532 the MySQL LVM mount or the remaining free-space in the Volume Group
533 (if there is less than 20% available) up to 15GB. If snapshot-size
534 is defined, the number represents the size of the snapshot in
535 megabytes.
536 snapshot-name = <name>
538 The name of the snapshot, the default being the name of the MySQL
539 LVM volume + "_snapshot" (ie Storage-MySQL_snapshot)
540 snapshot-mountpoint = <path>
542 Where to mount the snapshot. By default a randomly generated direc‐
543 tory under /tmp is used.
544 innodb-recovery = yes | no (default: no)
546 Whether or not to run an InnoDB recovery operation. This avoids
547 needing to do so during a restore, though will make the backup
548 process itself take longer.
549 force-innodb-backup = yes | no (default: no)
551 Whether to attempt a backup even if the mysql-lvm plugin thinks it
552 cannot obtain a good backup. This can occur when innodb data files
553 are outside of the mysql datadir or exist on entirely separate logi‐
554 cal volumes.
555 lock-tables = yes | no (default: yes)
557 Whether or not to run a FLUSH TABLES WITH READ LOCK to grab various
558 bits of information (such as the binary log name and position). Dis‐
559 abling this requires that binary logging is disabled and InnoDB is
560 being used exclusively. Otherwise, it is possible that the backup
561 could contain crashed tables.
562 extra-flush-tables = yes | no (default: yes)
564 Whether or not to run a FLUSH TABLES before running the full FLUSH
565 TABLES WITH READ LOCK. Should make the FLUSH TABLES WITH READ LOCK
566 operation a bit faster.
567 archive-method = tar | dir (default: tar)
569 Create a tar file of the datadir, or just copy it.
570 [tar]
572 exclude = pattern[, pattern...]
573 Patterns to exclude from archive. These should be relative paths and
575 are almost always relative to the mysql data directory. For instance
576 to exclude binary logs in the data directory from the backup you might
577 specify: exclude = ./bin-log.*, mysql.sock
578 pre-args = <string>
580 Additional arguments to append to the tar commandline before the backup
582 path is specified. This should be the full string as you might specify
583 on the commandline. Shell globbing is not supported.
584 For instance you might add the /etc/my.cnf to the tar archive via:
586 pre-args = -C /etc ./my.cnf
587 post-args = <string>
589 Additional arguments to append to the tar commandline after the backup
591 path is specified. This should be a string exactly as you might spec‐
592 ify on the commandline. Shell globbing is not evaluated.
593 [compression]
595 Specify various compression settings, such as compression utility, com‐
596 pression level, etc.
597 method = gzip| gzip-rsyncable | pigz | bzip2 | pbzip2 | lzop | lzma |
599 gpg | zstd
600 Define which compression method to use. Note that some methods may
601 not be available by default on every system and may need to be com‐
602 piled or installed.
603 For gpg compression, a key should already exist(gpg --gen-key) and
605 default-recipient must be configured in ~/.gnupg/gpg.conf.
606 inline = yes | no
608 Whether or not to pipe the output of the backup command into the
609 compression utility. Enabling this is recommended since it usually
610 only marginally impacts performance, particularly when using a lower
611 compression level.
612 level = 0-9
614 Specify the compression ratio. The lower the number, the lower the
615 compression ratio, but the faster the backup will take. Generally,
616 setting the lever to 1 or 2 results in favorable compression of tex‐
617 tual data and is noticeably faster than the higher levels. Setting
618 the level to 0 effectively disables compression.
619 bin-path = <full path to utility>
621 This only needs to be defined if the compression utility is not in
622 the usual places or not in the system path.
623 options = <string>
625 Add commandline options to the configuration compression command.
626 options = "-Q4"
628 split = yes | no
630 Defautls to no. If set the backup will be piped through the split
631 command. This may be useful for user's with large databases, as
632 some backup systems perform better with many smaller files instead
633 of 1 large one. This defaults to 1GB file size, so this option isn't
634 helpful if your dumps are smaller than that.
635 For python2.6, this option will be disabled if the subprocess32 mod‐
637 ule isn't avaiable.
638 New in version 1.1.13.
640 MySQL connection info [mysql:client]
643 These are optional and, if left undefined, Holland will try to login
644 using the standard .my.cnf conventions.
645 user = <user>
647 The user to connect to MySQL as.
649 password = <password>
651 The password for the MySQL user
653 socket = <socket>
655 The socket file to connect to MySQL with.
657 host = <host>
659 This would be used for connecting to MySQL remotely.
661 port = <port>
663 Used if MySQL is running on a port other than 3306.
665 mysqldump LVM Provider Configuration [mysqldump-lvm]
667 Backs up one or more MySQL databases by creating an LVM snapshot and
668 then starting a instance of MySQL on top of it to then perform a mysql‐
669 dump. This effectively produces a non-blocking logical backup.
670 [mysql-lvm]
672 snapshot-size = <size-in-MB>
673 The size of the snapshot itself. By default it is 20% of the size of
674 the MySQL LVM mount or the remaining free-space in the Volume Group
675 (if there is less than 20% available) up to 15GB. If snapshot-size
676 is defined, the number represents the size of the snapshot in
677 megabytes.
678 snapshot-name = <name>
680 The name of the snapshot, the default being the name of the MySQL
681 LVM volume + "_snapshot" (ie Storage-MySQL_snapshot)
682 snapshot-mountpoint = <path>
684 Where to mount the snapshot. By default a randomly generated direc‐
685 tory under /tmp is used.
686 innodb-recovery = yes | no (default: no)
688 Whether or not to run an InnoDB recovery operation. This avoids
689 needing to do so during a restore, though will make the backup
690 process itself take longer.
691 lock-tables = yes | no (default: yes)
693 Whether or not to run a FLUSH TABLES WITH READ LOCK to grab various
694 bits of information (such as the binary log name and position). Dis‐
695 abling this requires that binary logging is disabled and InnoDB is
696 being used exclusively. Otherwise, it is possible that the backup
697 could contain crashed tables.
698 extra-flush-tables = yes | no (default: yes)
700 Whether or not to run a FLUSH TABLES before running the full FLUSH
701 TABLES WITH READ LOCK. Should make the FLUSH TABLES WITH READ LOCK
702 operation a bit faster.
703 [mysqld]
705 mysqld-exe = <path>[, <path>...] (default: mysqld in PATH,
706 /usr/libexec/mysqld)
707 This provides a list of locations where the mysqld process to use
708 might be found. This is searched in order of entries in this list.
709 user = <name>
711 The --user parameter to use with mysqld.
712 innodb-buffer-pool-size = <size> (default: 128M)
714 How large to size the innodb-buffer-pool-size.
715 tmpdir = <path> (default: system tempdir)
717 Path to the --tmpdir that mysqld should use.
718 log-error = <path> (default: tempdir/holland_lvm.log)
720 Define path for mysqld's error log. The default location get cleaned
721 up by Holland after the backup is complete. This settings allows the
722 user to define the log file in another location and can be useful
723 for debugging issue with the MySQL instance running on the snapshot.
724 New in version 1.0.9.
726 [mysqldump]
729 mysqldump-lvm supports almost all of the options from the mysqldump
730 plugin. --master-data is not supported, as the mysqld process will not
731 read binary logs, so this plugin will automatically disable bin-log-po‐
732 sition, if set.
733 Binary log information from SOHW MASTER STATUS and SHOW SLAVE STATUS is
735 recorded in the ${backup_directory}/backup.conf file under the
736 [mysql:replication] section.
737 [compression]
739 Specify various compression settings, such as compression utility, com‐
740 pression level, etc.
741 method = gzip| gzip-rsyncable | pigz | bzip2 | pbzip2 | lzop | lzma |
743 gpg | zstd
744 Define which compression method to use. Note that some methods may
745 not be available by default on every system and may need to be com‐
746 piled or installed.
747 For gpg compression, a key should already exist(gpg --gen-key) and
749 default-recipient must be configured in ~/.gnupg/gpg.conf.
750 inline = yes | no
752 Whether or not to pipe the output of the backup command into the
753 compression utility. Enabling this is recommended since it usually
754 only marginally impacts performance, particularly when using a lower
755 compression level.
756 level = 0-9
758 Specify the compression ratio. The lower the number, the lower the
759 compression ratio, but the faster the backup will take. Generally,
760 setting the lever to 1 or 2 results in favorable compression of tex‐
761 tual data and is noticeably faster than the higher levels. Setting
762 the level to 0 effectively disables compression.
763 bin-path = <full path to utility>
765 This only needs to be defined if the compression utility is not in
766 the usual places or not in the system path.
767 options = <string>
769 Add commandline options to the configuration compression command.
770 options = "-Q4"
772 split = yes | no
774 Defautls to no. If set the backup will be piped through the split
775 command. This may be useful for user's with large databases, as
776 some backup systems perform better with many smaller files instead
777 of 1 large one. This defaults to 1GB file size, so this option isn't
778 helpful if your dumps are smaller than that.
779 For python2.6, this option will be disabled if the subprocess32 mod‐
781 ule isn't avaiable.
782 New in version 1.1.13.
784 MySQL connection info [mysql:client]
787 These are optional and, if left undefined, Holland will try to login
788 using the standard .my.cnf conventions.
789 user = <user>
791 The user to connect to MySQL as.
793 password = <password>
795 The password for the MySQL user
797 socket = <socket>
799 The socket file to connect to MySQL with.
801 host = <host>
803 This would be used for connecting to MySQL remotely.
805 port = <port>
807 Used if MySQL is running on a port other than 3306.
809 Xtrabackup Provider Configuration [xtrabackup]
811 Backs up a MySQL instance using Percona's Xtrabackup tool.
812 [xtrabackup]
814 global-defaults = <path> (default: /etc/my.cnf)
815 The MySQL configuration file for xtrabackup to parse. This is !in‐
816 clude'd into the my.cnf the xtrabackup plugin generates
817 innobackupex = <name> (default: innobackupex-1.5.1)
819 The path to the innobackupex script to run. If this is a relative
820 path this will be found in holland's environment PATH as configured
821 in /etc/holland/holland.conf.
822 Changed in version 1.1.12: For xtrabackup version 8.0 or greater, this
824 option is ignored
825 ibbackup = <name>
828 The path to the ibbackup command to use. By default, no --ibbackup
829 option is pass to the innobackupex script. Usually innobackupex
830 will detect this by itself and this should not need to be set.
831 stream = tar|xbstream|yes|no (default: tar)
833 Whether to generate a streaming backup.
834 Changed in version 1.0.8: 'tar' and 'xbstream' are now valid options.
836 The old stream = yes is now equivalent to stream = tar and stream = no
837 disables streaming entirely and will result in a normal directory copy
838 with xtrabackup
839 Changed in version 1.1.12: For xtrabackup version 8.0 or
842 greater,'xstream' will be used unless this value is set to 'no'
843 apply-logs = yes | no (default: yes)
846 Whether to run innobackupex --apply-logs at the end of the backup.
847 This is only supported when performing a non-streaming, non-com‐
848 pressed backup. In this case, even if apply-logs = yes (the de‐
849 fault), the prepare stage will be skipped. Even with an uncom‐
850 pressed, non-streaming backup you may want to disable apply-logs if
851 you wish to use incremental backups.
852 New in version 1.0.8.
854 slave-info = yes | no (default: yes)
857 Whether to enable the --slave-info innobackupex option
858 safe-slave-backup = yes | no (default: yes)
860 Whether to enable the --safe-slave-backup innobackupex option.
861 no-lock = yes | no (default: no)
863 Whether to enable the --no-lock innobackupex option
864 tmpdir = <path> (default: ${backup_directory})
866 The path for the innobackupex --tmpdir option. By default this will
867 use the current holland backup directory to workaround the following
868 bug: https://bugs.launchpad.net/percona-xtrabackup/+bug/1007446
869 New in version 1.0.8.
871 additional-options = <option>[, <option>...]
874 A list of additional options to pass to innobackupex. This is a
875 comma separated list of options.
876 pre-command = <command-string>
878 A command to run prior to running this xtrabackup run. This can be
879 used, for instance, to generate a mysqldump schema dump prior to
880 running xtrabackup. instances of ${backup_directory} and ${back‐
881 updir} will be replaced with the current holland backup directory
882 where the xtrabackup data will be stored.
883 [compression]
885 Specify various compression settings, such as compression utility, com‐
886 pression level, etc.
887 method = gzip| gzip-rsyncable | pigz | bzip2 | pbzip2 | lzop | lzma |
889 gpg | zstd
890 Define which compression method to use. Note that some methods may
891 not be available by default on every system and may need to be com‐
892 piled or installed.
893 For gpg compression, a key should already exist(gpg --gen-key) and
895 default-recipient must be configured in ~/.gnupg/gpg.conf.
896 inline = yes | no
898 Whether or not to pipe the output of the backup command into the
899 compression utility. Enabling this is recommended since it usually
900 only marginally impacts performance, particularly when using a lower
901 compression level.
902 level = 0-9
904 Specify the compression ratio. The lower the number, the lower the
905 compression ratio, but the faster the backup will take. Generally,
906 setting the lever to 1 or 2 results in favorable compression of tex‐
907 tual data and is noticeably faster than the higher levels. Setting
908 the level to 0 effectively disables compression.
909 bin-path = <full path to utility>
911 This only needs to be defined if the compression utility is not in
912 the usual places or not in the system path.
913 options = <string>
915 Add commandline options to the configuration compression command.
916 options = "-Q4"
918 split = yes | no
920 Defautls to no. If set the backup will be piped through the split
921 command. This may be useful for user's with large databases, as
922 some backup systems perform better with many smaller files instead
923 of 1 large one. This defaults to 1GB file size, so this option isn't
924 helpful if your dumps are smaller than that.
925 For python2.6, this option will be disabled if the subprocess32 mod‐
927 ule isn't avaiable.
928 New in version 1.1.13.
930 MySQL connection info [mysql:client]
933 These are optional and, if left undefined, Holland will try to login
934 using the standard .my.cnf conventions.
935 user = <user>
937 The user to connect to MySQL as.
939 password = <password>
941 The password for the MySQL user
943 socket = <socket>
945 The socket file to connect to MySQL with.
947 host = <host>
949 This would be used for connecting to MySQL remotely.
951 port = <port>
953 Used if MySQL is running on a port other than 3306.
955 pgdump Provider Configuration [pgdump]
957 Backs up a PostgreSQL instance using the pgdump utility.
958 [pgdump]
960 format = custom | tar | plain (default: custom)
961 Defines the --format option for pg_dump. This defaults to --for‐
962 mat=custom. The custom format is required for pg_restore to do par‐
963 tial restore as well as enabling parallel restores. If set to cus‐
964 tom, the --compress option will be passed to pgdump
965 additional-options = <command-string>
967 Pass additional options to the pg_dump command
968 Only the 'level' option will be used if 'format=custom'
970 [compression]
972 Specify various compression settings, such as compression utility, com‐
973 pression level, etc.
974 method = gzip| gzip-rsyncable | pigz | bzip2 | pbzip2 | lzop | lzma |
976 gpg | zstd
977 Define which compression method to use. Note that some methods may
978 not be available by default on every system and may need to be com‐
979 piled or installed.
980 For gpg compression, a key should already exist(gpg --gen-key) and
982 default-recipient must be configured in ~/.gnupg/gpg.conf.
983 inline = yes | no
985 Whether or not to pipe the output of the backup command into the
986 compression utility. Enabling this is recommended since it usually
987 only marginally impacts performance, particularly when using a lower
988 compression level.
989 level = 0-9
991 Specify the compression ratio. The lower the number, the lower the
992 compression ratio, but the faster the backup will take. Generally,
993 setting the lever to 1 or 2 results in favorable compression of tex‐
994 tual data and is noticeably faster than the higher levels. Setting
995 the level to 0 effectively disables compression.
996 bin-path = <full path to utility>
998 This only needs to be defined if the compression utility is not in
999 the usual places or not in the system path.
1000 options = <string>
1002 Add commandline options to the configuration compression command.
1003 options = "-Q4"
1005 split = yes | no
1007 Defautls to no. If set the backup will be piped through the split
1008 command. This may be useful for user's with large databases, as
1009 some backup systems perform better with many smaller files instead
1010 of 1 large one. This defaults to 1GB file size, so this option isn't
1011 helpful if your dumps are smaller than that.
1012 For python2.6, this option will be disabled if the subprocess32 mod‐
1014 ule isn't avaiable.
1015 New in version 1.1.13.
1017 [pgauth]
1020 username = <name>
1021 Username for pg_dump to authenticate with
1022 password = <string>
1024 Password for pg_dump to authenticate with
1025 hostname = <string>
1027 Hostname for pg_dump to connect with
1028 port = <integer>
1030 TCP port for pg_dump to connect on
1031 pg_basebackup Provider Configuration [pg_basebackup]
1033 Backs up a PostgreSQL instance using the pg_basebackup utility.
1034 [pg_basebackup]
1036 format = tar | plain (default: tar)
1037 Defines the --format option for pg_basebackup.
1038 wal-method = none | fetch | plain (default: fetch)
1040 Defines the --wal-method used for pg_basebackup.
1041 checkpoint = none | fast | spread (default: fast)
1043 Defines the --checkpoint used for pg_basebackup.
1044 additional-options = <command-string> (default: none)
1046 Pass additional options to the pg_basebackup command
1047 [compression]
1049 Specify various compression settings, such as compression utility, com‐
1050 pression level, etc.
1051 method = gzip| gzip-rsyncable | pigz | bzip2 | pbzip2 | lzop | lzma |
1053 gpg | zstd
1054 Define which compression method to use. Note that some methods may
1055 not be available by default on every system and may need to be com‐
1056 piled or installed.
1057 For gpg compression, a key should already exist(gpg --gen-key) and
1059 default-recipient must be configured in ~/.gnupg/gpg.conf.
1060 inline = yes | no
1062 Whether or not to pipe the output of the backup command into the
1063 compression utility. Enabling this is recommended since it usually
1064 only marginally impacts performance, particularly when using a lower
1065 compression level.
1066 level = 0-9
1068 Specify the compression ratio. The lower the number, the lower the
1069 compression ratio, but the faster the backup will take. Generally,
1070 setting the lever to 1 or 2 results in favorable compression of tex‐
1071 tual data and is noticeably faster than the higher levels. Setting
1072 the level to 0 effectively disables compression.
1073 bin-path = <full path to utility>
1075 This only needs to be defined if the compression utility is not in
1076 the usual places or not in the system path.
1077 options = <string>
1079 Add commandline options to the configuration compression command.
1080 options = "-Q4"
1082 split = yes | no
1084 Defautls to no. If set the backup will be piped through the split
1085 command. This may be useful for user's with large databases, as
1086 some backup systems perform better with many smaller files instead
1087 of 1 large one. This defaults to 1GB file size, so this option isn't
1088 helpful if your dumps are smaller than that.
1089 For python2.6, this option will be disabled if the subprocess32 mod‐
1091 ule isn't avaiable.
1092 New in version 1.1.13.
1094 [pgauth]
1097 username = <name>
1098 Username for pg_basebackup to authenticate with
1099 password = <string>
1101 Password for pg_basebackup to authenticate with
1102 hostname = <string>
1104 Hostname for pg_basebackup to connect with
1105 port = <integer>
1107 TCP port for pg_basebackup to connect on
1108 mongodump Provider Configuration [mongodump]
1110 This plugin performs logical backups of a MongoDB by using the mongo‐
1111 dump utility.
1112 [mongodump]
1114 host = <string>
1115 Hostname for mongodump to connect with
1116 username = <name>
1118 Username for mongodump to authenticate with
1119 password = <string>
1121 Password for mongodump to authenticate with
1122 port = <integer>
1124 TCP port for mongodump to connect on
1125 uri = <string>
1127 Use a connection string instead of a host, username, and pass‐
1128 word
1129 New in version 1.1.14.
1131 [compression]
1134 Specify various compression settings, such as compression utility, com‐
1135 pression level, etc.
1136 method = gzip| gzip-rsyncable | pigz | bzip2 | pbzip2 | lzop | lzma |
1138 gpg | zstd
1139 Define which compression method to use. Note that some methods may
1140 not be available by default on every system and may need to be com‐
1141 piled or installed.
1142 For gpg compression, a key should already exist(gpg --gen-key) and
1144 default-recipient must be configured in ~/.gnupg/gpg.conf.
1145 inline = yes | no
1147 Whether or not to pipe the output of the backup command into the
1148 compression utility. Enabling this is recommended since it usually
1149 only marginally impacts performance, particularly when using a lower
1150 compression level.
1151 level = 0-9
1153 Specify the compression ratio. The lower the number, the lower the
1154 compression ratio, but the faster the backup will take. Generally,
1155 setting the lever to 1 or 2 results in favorable compression of tex‐
1156 tual data and is noticeably faster than the higher levels. Setting
1157 the level to 0 effectively disables compression.
1158 bin-path = <full path to utility>
1160 This only needs to be defined if the compression utility is not in
1161 the usual places or not in the system path.
1162 options = <string>
1164 Add commandline options to the configuration compression command.
1165 options = "-Q4"
1167 split = yes | no
1169 Defautls to no. If set the backup will be piped through the split
1170 command. This may be useful for user's with large databases, as
1171 some backup systems perform better with many smaller files instead
1172 of 1 large one. This defaults to 1GB file size, so this option isn't
1173 helpful if your dumps are smaller than that.
1174 For python2.6, this option will be disabled if the subprocess32 mod‐
1176 ule isn't avaiable.
1177 New in version 1.1.13.
1179 Mariabackup Provider Configuration [mariabackup]
1182 Backs up a MySQL instance using mariabackup tool.
1183 [mariabackup]
1185 global-defaults = <path> (default: /etc/my.cnf)
1186 The MySQL configuration file for mariabackup to parse. This is !in‐
1187 clude'd into the my.cnf the mariabackup plugin generates
1188 innobackupex = <name> (default: innobackupex-1.5.1)
1190 The path to the innobackupex script to run. If this is a relative
1191 path this will be found in holland's environment PATH as configured
1192 in /etc/holland/holland.conf.
1193 ibbackup = <name>
1195 The path to the ibbackup command to use. By default, no --ibbackup
1196 option is pass to the innobackupex script. Usually innobackupex
1197 will detect this by itself and this should not need to be set.
1198 stream = mbstream(default: tar)
1200 Placeholder
1201 apply-logs = yes | no (default: yes)
1203 Whether to run innobackupex --apply-logs at the end of the backup.
1204 This is only supported when performing a non-streaming, non-com‐
1205 pressed backup. In this case, even if apply-logs = yes (the de‐
1206 fault), the prepare stage will be skipped. Even with an uncom‐
1207 pressed, non-streaming backup you may want to disable apply-logs if
1208 you wish to use incremental backups.
1209 slave-info = yes | no (default: yes)
1211 Whether to enable the --slave-info innobackupex option
1212 safe-slave-backup = yes | no (default: yes)
1214 Whether to enable the --safe-slave-backup innobackupex option.
1215 no-lock = yes | no (default: no)
1217 Whether to enable the --no-lock innobackupex option
1218 tmpdir = <path> (default: ${backup_directory})
1220 The path for the innobackupex --tmpdir option.
1221 additional-options = <option>[, <option>...]
1223 A list of additional options to pass to innobackupex. This is a
1224 comma separated list of options.
1225 pre-command = <command-string>
1227 A command to run prior to running this mariabackup run. This can be
1228 used, for instance, to generate a mysqldump schema dump prior to
1229 running mariabackup. instances of ${backup_directory} and ${back‐
1230 updir} will be replaced with the current holland backup directory
1231 where the mariabackup data will be stored.
1232 [compression]
1234 Specify various compression settings, such as compression utility, com‐
1235 pression level, etc.
1236 method = gzip| gzip-rsyncable | pigz | bzip2 | pbzip2 | lzop | lzma |
1238 gpg | zstd
1239 Define which compression method to use. Note that some methods may
1240 not be available by default on every system and may need to be com‐
1241 piled or installed.
1242 For gpg compression, a key should already exist(gpg --gen-key) and
1244 default-recipient must be configured in ~/.gnupg/gpg.conf.
1245 inline = yes | no
1247 Whether or not to pipe the output of the backup command into the
1248 compression utility. Enabling this is recommended since it usually
1249 only marginally impacts performance, particularly when using a lower
1250 compression level.
1251 level = 0-9
1253 Specify the compression ratio. The lower the number, the lower the
1254 compression ratio, but the faster the backup will take. Generally,
1255 setting the lever to 1 or 2 results in favorable compression of tex‐
1256 tual data and is noticeably faster than the higher levels. Setting
1257 the level to 0 effectively disables compression.
1258 bin-path = <full path to utility>
1260 This only needs to be defined if the compression utility is not in
1261 the usual places or not in the system path.
1262 options = <string>
1264 Add commandline options to the configuration compression command.
1265 options = "-Q4"
1267 split = yes | no
1269 Defautls to no. If set the backup will be piped through the split
1270 command. This may be useful for user's with large databases, as
1271 some backup systems perform better with many smaller files instead
1272 of 1 large one. This defaults to 1GB file size, so this option isn't
1273 helpful if your dumps are smaller than that.
1274 For python2.6, this option will be disabled if the subprocess32 mod‐
1276 ule isn't avaiable.
1277 New in version 1.1.13.
1279 MySQL connection info [mysql:client]
1282 These are optional and, if left undefined, Holland will try to login
1283 using the standard .my.cnf conventions.
1284 user = <user>
1286 The user to connect to MySQL as.
1288 password = <password>
1290 The password for the MySQL user
1292 socket = <socket>
1294 The socket file to connect to MySQL with.
1296 host = <host>
1298 This would be used for connecting to MySQL remotely.
1300 port = <port>
1302 Used if MySQL is running on a port other than 3306.
1304 Backup-Set Configs
1306 Backup-Set configuration files largely inherit the configuration op‐
1307 tions of the specified provider. To define a provider for the backup
1308 set, you must put the following at the top of the backup set configura‐
1309 tion file:
1310 [holland:backup]
1312 plugin = <plugin>
1313 backups-to-keep = #
1314 estimated-size-factor = #
1315 historic-size = <yes|no>
1316 historic-size-factor = #
1317 historic-estimated-size-factor = #
1318 Configuration Options
1320 plugin = <plugin>
1321 This is the name of the provider that will be used for the
1322 backup-set. This is required in order for the backup-set to func‐
1323 tion.
1324 backups-to-keep = #
1326 Specifies the number of backups to keep for a backup-set. Defaults
1327 to retaining 1 backup.
1328 estimated-size-factor = #
1330 Specifies the scale factor when Holland decides if there is enough
1331 free space to perform a backup. The default is 1.0 and this number
1332 is multiplied against what each individual plugin reports its esti‐
1333 mated backup size when Holland is verifying sufficient free space
1334 for the backupset.
1335 As of Holland 1.1.1, the 'historic' backup size will be used. If
1337 Holland is unable to determine that size, it will default back to
1338 this.
1339 auto-purge-failures = yes | no
1341 Specifies whether to keep a failed backup or to automatically remove
1342 the backup directory. By default this is on with the intention that
1343 whatever process is calling holland will retry when a backup fails.
1344 This behavior can be disabled by setting auto-purge-failures = no
1345 when partial backups might be useful or when troubleshooting a
1346 backup failure.
1347 purge-policy = manual | before-backup | after-backup
1349 Specifies when to run the purge routine on a backupset. By default
1350 this is run after a new successful backup completes. Up to back‐
1351 ups-to-keep backups will be retained including the most recent.
1352 purge-policy = before-backup will run the purge routine just before
1354 a new backup starts. This will retain up to backups-to-keep backups
1355 before the new backup is even started allowing purging all previous
1356 backups if backups-to-keep is set to 0. This behavior is useful if
1357 some other process is retaining backups off-server and disk space is
1358 at a premium.
1359 purge-policy = manual will never run the purge routine automati‐
1361 cally. Either holland purge must be run externally or an explicit
1362 removal of desired backup directories can be done at some later
1363 time.
1364 before-backup-command = string
1366 Run a shell command before a backup starts. This allows a command
1367 to perform some action before the backup starts such as setting up
1368 an iptables rule (taking a mysql slave out of a load balancer) or
1369 aborting the backup based on some external condition.
1370 The backup will fail if this command exits with a non-zero status.
1372 New in version 1.0.7.
1374 after-backup-command = string
1377 Run a shell command after a backup completes. This allows a command
1378 to perform some action when a backup completes successfully such as
1379 sending out a success notification.
1380 The backup will fail if this command exits with a non-zero status.
1382 New in version 1.0.7.
1384 failed-backup-command = string
1387 Run a shell command if a backup starts. This allows some command to
1388 perform some action when a backup fails such as sending out a fail‐
1389 ure notification.
1390 The backup will fail if this command exits with a non-zero status.
1392 New in version 1.0.7.
1394 historic-size = yes | no
1397 Defaults to yes. Check for the 'backup.conf' file in the 'newest'
1398 spooled folder for the running "backupset". If the configuration
1399 file exists and contains 'estimated-size' and 'on-disk-size', these
1400 values will be used to decide if holland has enough free space for a
1401 backupset. This means Holland will use values from the previous
1402 backup to estimate the size of the next backup.
1403 New in version 1.1.1.
1405 historic-size-factor = #
1408 Defaults to '1.5'. If the estimated size of the database has changed
1409 by more than this multiple, the 'estimated-size-factor' value will
1410 be used to determine if there is sufficient free space for the back‐
1411 upset.
1412 New in version 1.1.1.
1414 historic-estimated-size-factor = #
1417 Defaults to '1.1'. Specifies the scale factor when Holland decides
1418 if there is enough free space to perform a backup. Holland will
1419 throw an error if the system has less free space than the last
1420 backup size multiplied by this value
1421 New in version 1.1.1.
1423 create-symlinks = yes | no
1426 Defaults to yes. Enable the creation of "newest" and "oldest" sym‐
1427 links in the backupset directory. Disabling the symlink creation
1428 process and be helpful on filesystems that don't support symlinks.
1429 New in version 1.2.7.
1431 $ ls -l /var/spool/holland/default/
1434 total 0
1435 drwxrwx---. 3 root root 58 Aug 18 16:18 20210818_161856
1436 lrwxrwxrwx. 1 root root 42 Aug 18 16:18 newest -> /var/spool/holland/default/20210818_161856
1437 lrwxrwxrwx. 1 root root 42 Aug 18 16:18 oldest -> /var/spool/holland/default/20210818_161856
1438 For all hook commands, Holland will perform simple text substitution on
1440 the three parameters:
1441 • hook - name of the hook being called (one of: before-backup-com‐
1443 mand, after-backup-command, failed-backup-command)
1444 • backupdir - path to the current backup directory (e.g.
1446 /var/spool/holland/mysqldump/YYYYmmdd_HHMMSS)
1447 • backupset - name of the backupset being run (e.g. 'mysql-lvm')
1449 For example:
1451 [holland:backup]
1453 plugin = mysqldump
1454 before-backup-command = /usr/local/bin/my-custom-script --hook ${hook} --backupset ${backupset} --backupdir ${backupdir}
1455 after-backup-command = echo ${backupset} completed successfully. Files are in ${backupdir}
1456 [mysqldump]
1458 ...
1459 Backup-Set files are defined in the "backupsets" directory which is, by
1461 default, /etc/holland/backupsets. The name of the backup-set is defined
1462 by its configuration filename and can really be most anything. That
1463 means backup-sets can be organized in any arbitrary way, although
1464 backup set files must end in .conf. The file extension is not part of
1465 the name of the backup-set.
1466 As noted above, in order for a backup-set to be active, it must be
1468 listed in the backupsets variable.
1469 Backups are placed under the directory defined in the backup_directory
1471 section of the main configuration file. Each backup resides under a di‐
1472 rectory corresponding to the backup-set name followed by a date-encoded
1473 directory.
1474
1476 Holland Core Team
1477
1479 2021
14801.2.8 Nov 17, 2021 HOLLAND(1)