1HOLLAND(1) Holland HOLLAND(1)
2
6 holland - Holland Documentation
7
9 Introduction to Holland
10 Holland is an Open Source backup framework originally developed by
11 Rackspace and written in Python. It's goal is to help facilitate back‐
12 ing up databases with greater configurability, consistency, and ease.
13 Holland currently focuses on MySQL, however future development will
14 include other database platforms and even non-database related applica‐
15 tions. Because of it's plugin structure, Holland can be used to backup
16 anything you want by whatever means you want.
17 Dependencies
19 The core Holland framework has the following dependencies (available on
20 any remotely modern Linux distribution):
21 · Python >= 2.6
23 · pkg_resources
25 · python-setuptools
27 · future
29 · six
31 · ConfigParser
33 · ArgParser
35 MySQL based plugins additional require the MySQLdb python connector:
37 · Python 2: MySQLdb(mysql-python)
39 · Python 3: MySQLdb(mysqlclient-python)
41 Note that other plugins may have additional dependency requirements.
43 Holland Command-Line Reference
45 Here are the commands available from the 'holland' command-line tool:
46 help (h)
48 Usage: holland <command> --help(-h)
49 Provides basic information about the provided command. If no command is
51 provided, it displays global help instead.
52 backup (bk)
54 Usage: holland backup [backup-set1, backup-set2, ..., backup-setN]
55 Runs the backup operation. If no backup-sets are specified, all active
57 backup-sets (those defined in the 'backupsets' variable in hol‐
58 land.conf) are backed up.
59 One or more backup-sets can be specified directly, in which case only
61 those backup-sets are backed up.
62 Additional Command Line Arguments:
64 --dry-run (-n): Can be used here to simulate, but not actually run, a
66 backup. This should be used when troubleshooting a particular error
67 before trying to run a real backup.
68 --no-lock (-f): Normally, only one instance of Holland can run at any
70 given time using lock-files. Using this flag causes the lock-files to
71 be ignored. This has some very clear use-cases but otherwise be mindful
72 of using this setting as it can cause backups to fail in some cases.
73 --abort-immediately: abort on the first backup-set that fails (assuming
75 multiple backupsets were specified)
76 Examples:
78 # holland bk --dry-run weekly: Attempts a dry-run of the weekly
80 backup-set.
81 # holland bk --no-lock --abort-immediately: Attempts a backup of all
83 the default backup-sets ignoring locks and aborting immediately if one
84 of the backup-sets fails.
85 list-backups (lb)
87 Usage: holland list-backups
88 Provides extended information about available backups.
90 list-plugins (lp)
92 Usage: holland list-plugins
93 Lists all the available (installed) plugins available to Holland.
95 mk-config (mc)
97 Usage: holland mk-config <provider>
98 Generates a template backup-set for a particular provider (such as
100 mysqldump). By default, the output is sent to standard out but can be
101 copied to a file, either by using the --file, --edit, or -name options
102 (see below).
103 Additional Command Line Arguments:
105 --edit: Load the file into the system text-editor for further modifica‐
107 tions.
108 --file=FILE (-f): Write the output directly to provided file.
110 --name=NAME: Creates a backup-set usable in Holland, which basically
112 means that a file is created of the provided name under the backup-set
113 directory.
114 --provider: Indicates that the default provider configuration should be
116 outputted instead. This is really only used when creating a provider
117 config specifically - it should not be used for backup-sets.
118 Examples:
120 # holland mk-config mysql-lvm > mysql-lvm.conf: Output the default con‐
122 figuration for MySQL-LVM backups and write the contents out to
123 mysql-lvm.conf in the current working directory.
124 # holland mc mysqldump --name=Bob --edit: Create a backup-set using the
126 mysqldump provider named Bob and allow interactive editing of the
127 backup-set before saving the file.
128 purge (pg)
130 Usage: holland purge <backup-set>/<backup-id>
131 Purges old backups by specifying the backup-set name and set-id.
133 For example: # holland purge mybackups/20090502_155438: Purge one of
135 the backups taken on May 2nd, 2009 from the mybackups backup-set.
136 Usage and Implementation Overview
138 Because Holland is very pluggable, it may first seem a bit confusing
139 when it comes to configuring Holland to do something useful. Out of the
140 box, Holland is designed to backup MySQL databases using the mysqldump
141 provider. This is the simplest setup, and may be sufficient for most
142 people. However, others may wish to have more fine-grained control over
143 their backups and/or use another method other than mysqldump.
144 For instance, one can configure a backup set to backup certain data‐
146 bases using mysqldump, others using the mysql-lvm plugins etc. All this
147 is done by a mix of providers and backup-sets.
148 Backup-Sets
150 Each backup-set implements exactly one provider and will inherit the
151 default values of that provider. These values can be overridden to
152 adjust the behavior of the backup set. This includes defining what
153 databases or tables to include (or exclude) in the backup, the type of
154 compression used (if any), what locking method to use, among other
155 things.
156 Providers
158 Providers essentially provide a backup service for use in a backup set.
159 · mysqldump
161 Uses the mysqldump utility to backup databases.
162 · MySQL + LVM
164 Backup MySQL databases using LVM snapshots which allows for near
165 lockless or fully lockless (when transactional engines are used)
166 backups.
167 · mysqldump + LVM
169 This plugin creates an LVM snapshot, starts a mysql instance using
170 the snapshot as it's datadir, and then use the mysqldump utility
171 to backup the databases.
172 · XtraBackup
174 New in version 1.0.8.
175 Backup MySQL databases using Percona's XtraBackup tool. This pro‐
178 vides a near lockless backup when using the InnoDB storage engine.
179 · pgdump
181 Backup PostgreSQL databases using the pgdump utility.
182 · mariabackup
184 New in version 1.1.0.
185 Backup MySQL databases using MariaDB's mariabackup tool.
188 · mongodump
190 New in version 1.1.0.
191 This plugin performs logical backups of a MongoDB by using the
194 mongodump utility.
195 · Example
197 This is used solely as a template for designing providers. It oth‐
198 erwise does nothing.
199 As Holland is a framework, it can actually backup most anything as long
201 as there is a provider for it. This includes things that have nothing
202 to do with databases. The idea is to present an easy to use and clear
203 method of backing up and restoring backups no matter the source.
204 Holland Config Files
206 By default, Holland's configuration files reside in /etc/holland. The
207 main configuration file is holland.conf, however there are a number of
208 other configuration files for configuring default settings for
209 providers and for configuring backup sets.
210 Each configuration file has one ore more sections, defined by square
212 brackets Underneath each section, one or more configuration option can
213 be specified. These options are in a standard "option = value" format.
214 Comments are prefixed by the # sign.
215 Note that many settings have default values and, as a result, can
217 either be commented out or omitted entirely.
218 holland.conf - main config
220 The main configuration file (usually /etc/holland/holland.conf) defines
221 both global settings as well as the active backup sets. It is divided
222 into two sections [holland] and [logging].
223 [holland]
225 plugin-dirs
226 Defines where the plugins can be found. This can be a comma-sep‐
227 arated list but usually does not need to be modified.
228 backup_directory
230 Top-level directory where backups are held.
231 backupsets
233 A comma-separated list of all the backup sets Holland should
234 backup. Each backup set is defined in /etc/holland/back‐
235 upsets/<name>.conf by default.
236 umask Sets the umask of the resulting backup files.
238 path Defines a path for holland and its spawned processes
240 [logging]
242 filename
243 The log file itself.
244 level Sets the verbosity of Holland's logging process. Available
246 options are debug, info, warning, error, and critical
247 format Define the format of the log message. This options should be in
249 the format defined by the 'logging' python library.
250 New in version 1.1.0.
252 Provider Configs
255 These files control the global settings / defaults for the providers
256 used by the backup-sets. Many of these global settings can be overrid‐
257 den if defined in a backup-set. Note that each provider's configuration
258 file should begin with [provider-name].
259 mysqldump Provider Configuration [mysqldump]
261 Backs up a MySQL database using the mysqldump tool.
262 [mysqldump]
264 mysql-binpath = /path/to/mysql/bin
265 Defines the location of the MySQL binary utilities. If not provided,
266 Holland will use whatever is in the path.
267 lock-method = flush-lock | lock-tables | single-transaction |
269 auto-detect | none
270 Defines which lock method to use. By default, auto-detect will be
271 used.
272 · flush-lock
274 flush-lock will place a global lock on all tables involved in
275 the backup regardless of whether or not they are in the
276 backup-set. If file-per-database is enabled, then flush-lock
277 will lock all tables for every database being backed up. In
278 other words, this option may not make much sense when using
279 file-per-database.
280 · lock-tables
282 lock-tables will lock all tables involved in the backup. If
283 file-per-database is enabled, then lock-tables will only lock
284 all the tables associated with that database.
285 · single-transaction
287 Forces the use of --single-transaction which enabled
288 semi-transparent backups of transactional tables. Forcing this
289 can cause inconsistencies with non-transactional tables, how‐
290 ever. While non-transactional tables will still lock, they
291 will only lock when they are actually being backed up. Use
292 this setting with extreme caution when backing non-transac‐
293 tional tables.
294 · auto-detect
296 Let Holland decide which option to use by checking to see if a
297 database or backup-set only contains transactional tables. If
298 so, --single-transaction will be used. Otherwise,
299 --lock-tables will be used.
300 · none
302 Does absolutely no explicit locking when backing up the data‐
303 bases or backup-set. This should only be used when backing up
304 a slave and only after the slave has been turned off (ie, this
305 can be used with the stop-slave option).
306 exclude-invalid-views = yes | no (default: no)
308 Whether to automate exclusion of invalid views that would otherwise
309 cause mysqldump to fail. This adds additional overhead so this
310 option is not enabled by default.
311 When enabled, thos option will scan the INFORMATION_SCHEMA.VIEWS ta‐
313 ble and execute SHOW FIELDS against each view. If a view is detects
314 as invalid, an ignore-table option will be added to exclude the ta‐
315 ble. Additionally, the plugin will attempt to save the view defin‐
316 ion to 'invalid_views.sql' in the backupset's backup directory.
317 New in version 1.0.8.
319 dump-routines = yes | no (default: yes)
322 Whether or not to backup routines in the backup set directly. Rou‐
323 tines are stored in the 'mysql' database, but it can sometimes be
324 convenient to include them in a backup-set directly.
325 Changed in version 1.0.8: This option now enabled by default.
327 dump-events = yes | no
330 Whether or not to dump events explicitly. Like routines, events are
331 stored in the 'mysql' database. Nonetheless, it can sometimes be
332 convenient to include them in the backup-set directly.
333 Note: This feature requires MySQL 5.1 or later. The mysqldump plugin
335 will automatically disable events if the version of mysqldump is too
336 old.
337 Changed in version 1.0.8: This option is now enabled by default
339 stop-slave = yes | no
342 Stops the SQL_THREAD during the backup. This means that writes from
343 the master will continue to spool but will not be replayed. This
344 helps avoid lock wait timeouts among things while still allowing
345 data to be spooled from the master.
346 Note that previous versions of Holland prior to 1.0.6 simply ran a
348 STOP SLAVE instead, which suspends both replication threads.
349 bin-log-position = yes | no
351 Record the binary log name and position at the time of the backup.
352 The information provied by this option is collected just before
353 locking the database.
354 Note that if both 'stop-slave' and 'bin-log-position' are enabled,
356 Holland will grab the master binary log name and position at the
357 time of the backup which can be useful in using the backup to create
358 slaves or for point in time recovery using the master's binary log.
359 This information is found within the 'backup.conf' file located in
360 the backup-set destination directory (/var/spool/hol‐
361 land/<backup-set>/<backup> by default). For example:
362 [mysql:replication]
364 slave_master_log_pos = 4512
365 slave_master_log_file = 260792-mmm-agent1-bin-log.000001
366 flush-logs = yes | no
368 Whether or not to run FLUSH LOGS in MySQL with the backup. When
369 FLUSH LOGS is actually executed depends on which if database filter‐
370 ing is being used and whether or not file-per-database is enabled.
371 Generally speaking, it does not make sense to use flush-logs with
372 file-per-database since the binary logs will not be consistent with
373 the backup.
374 file-per-database = yes | no
376 Whether or not to split up each database into its own file. Note
377 that it can be more consistent an efficient to backup all databases
378 into one file, however this means that restore a single database can
379 be difficult if multiple databases are defined in the backup set.
380 arg-per-database = JSON object
382 If file-per-database is enable this argument is ued to specify
383 mysqldump arguments per database. It takes a JSON object with the
384 database names as keys. Example: {"employee1": "--no-data"} Adds
385 the '--no-data' argument to the mysqldump command when backing up
386 the 'employee1' database
387 New in version 1.0.9.
389 additional-options = <mysqldump argument>[, <mysqldump argument>]
392 Can optionally specify additional options directly to mysqldump if
393 there is no native Holland option available. This option accepts a
394 comma delimited list of arguments to pass on the commandline.
395 extra-defaults = yes | no (default: no)
397 This option controls whether mysqldump will only read options as set
398 by holland or if additional options from global config files are
399 read. By default, the plugin only uses optons as set in the back‐
400 upset config and includes authentication credentials only from the
401 [client] section in ~/.my.cnf.
402 estimate-method = plugin | const:<size> (default: plugin)
404 This option will skip some of the heavyweight queries necessary to
405 calculate the size of tables to be backed up. If a constant size is
406 specified, then only table names are evaluated and only if table
407 filtering is being used. Additionally, engines will be looked up via
408 SHOW CREATE TABLE if lock-method = auto-detect, in order for the
409 plugin to determine if tables are using a transactional storage
410 engine. With 'plugin', the default behavior of reading both size
411 information and table names from the information schema is used,
412 which may be slow particularly for a large number of tables.
413 Database and Table filtering
415 Database and Table filtering
416 databases = <glob>
417 exclude-databases = <glob>
419 tables = <glob>
421 exclude-tables = <glob>
423 The above options accepts GLOBs in comma-separated lists. Multiple fil‐
425 tering options can be specified. When filtering on tables, be sure to
426 include both the database and table name.
427 Be careful with quotes. Normally these are not needed, but when quotes
429 are necessary, be sure to only quote each filtering statement, as
430 opposed to putting quotes around all statements.
431 Below are a few examples of how these can be applied:
433 Default (backup everything):
435 databases = *
437 tables = *
438 Using database inclusion and exclusions:
440 databases = drupal*, smf_forum,
442 exclude-databases = drupal5
443 Including Tables:
445 tables = phpBB.sucks, drupal6.node*, smf_forum.*
447 Excluding Tables:
449 exclude-tables = mydb.uselesstable1, x_cart.*, *.sessions
451 [compression]
453 Specify various compression settings, such as compression utility, com‐
454 pression level, etc.
455 method = gzip| gzip-rsyncable | pigz | bzip2 | pbzip2 | lzop | lzma |
457 gpg | zstd
458 Define which compression method to use. Note that some methods may
459 not be available by default on every system and may need to be com‐
460 piled or installed.
461 For gpg compression, a key should already exist(gpg --gen-key) and
463 default-recipient must be configured in ~/.gnupg/gpg.conf.
464 inline = yes | no
466 Whether or not to pipe the output of the backup command into the
467 compression utility. Enabling this is recommended since it usually
468 only marginally impacts performance, particularly when using a lower
469 compression level.
470 level = 0-9
472 Specify the compression ratio. The lower the number, the lower the
473 compression ratio, but the faster the backup will take. Generally,
474 setting the lever to 1 or 2 results in favorable compression of tex‐
475 tual data and is noticeably faster than the higher levels. Setting
476 the level to 0 effectively disables compression.
477 bin-path = <full path to utility>
479 This only needs to be defined if the compression utility is not in
480 the usual places or not in the system path.
481 split = yes | no
483 Defautls to no. If set the backup will be piped through the split
484 command. This may be useful for user's with large databases, as
485 some backup systems perform better with many smaller files instead
486 of 1 large one. This defaults to 1GB file size, so this option isn't
487 helpful if your dumps are smaller than that.
488 New in version 1.1.13.
490 MySQL connection info [mysql:client]
493 These are optional and, if left undefined, Holland will try to login
494 using the standard .my.cnf conventions.
495 user = <user>
497 The user to connect to MySQL as.
499 password = <password>
501 The password for the MySQL user
503 socket = <socket>
505 The socket file to connect to MySQL with.
507 host = <host>
509 This would be used for connecting to MySQL remotely.
511 port = <port>
513 Used if MySQL is running on a port other than 3306.
515 MySQL LVM Provider Configuration [mysql-lvm]
517 Creates an LVM snapshot of a running MySQL instance and performs a
518 binary-based backup with minimal locking. MySQL must be running on an
519 LVM volume with reserved space for snapshots. It is highly recommended
520 that this volume be separate from the one storing the resulting back‐
521 ups.
522 [mysql-lvm]
524 snapshot-size = <size-in-MB>
525 The size of the snapshot itself. By default it is 20% of the size of
526 the MySQL LVM mount or the remaining free-space in the Volume Group
527 (if there is less than 20% available) up to 15GB. If snapshot-size
528 is defined, the number represents the size of the snapshot in
529 megabytes.
530 snapshot-name = <name>
532 The name of the snapshot, the default being the name of the MySQL
533 LVM volume + "_snapshot" (ie Storage-MySQL_snapshot)
534 snapshot-mountpoint = <path>
536 Where to mount the snapshot. By default a randomly generated direc‐
537 tory under /tmp is used.
538 innodb-recovery = yes | no (default: no)
540 Whether or not to run an InnoDB recovery operation. This avoids
541 needing to do so during a restore, though will make the backup
542 process itself take longer.
543 force-innodb-backup = yes | no (default: no)
545 Whether to attempt a backup even if the mysql-lvm plugin thinks it
546 cannot obtain a good backup. This can occur when innodb data files
547 are outside of the mysql datadir or exist on entirely separate logi‐
548 cal volumes.
549 lock-tables = yes | no (default: yes)
551 Whether or not to run a FLUSH TABLES WITH READ LOCK to grab various
552 bits of information (such as the binary log name and position). Dis‐
553 abling this requires that binary logging is disabled and InnoDB is
554 being used exclusively. Otherwise, it is possible that the backup
555 could contain crashed tables.
556 extra-flush-tables = yes | no (default: yes)
558 Whether or not to run a FLUSH TABLES before running the full FLUSH
559 TABLES WITH READ LOCK. Should make the FLUSH TABLES WITH READ LOCK
560 operation a bit faster.
561 archive-method = tar | dir (default: tar)
563 Create a tar file of the datadir, or just copy it.
564 [tar]
566 exclude = pattern[, pattern...]
567 Patterns to exclude from archive. These should be relative paths and
569 are almost always relative to the mysql data directory. For instance
570 to exclude binary logs in the data directory from the backup you might
571 specify: exclude = ./bin-log.*, mysql.sock
572 pre-args = <string>
574 Additional arguments to append to the tar commandline before the backup
576 path is specified. This should be the full string as you might specify
577 on the commandline. Shell globbing is not supported.
578 For instance you might add the /etc/my.cnf to the tar archive via:
580 pre-args = -C /etc ./my.cnf
581 post-args = <string>
583 Additional arguments to append to the tar commandline after the backup
585 path is specified. This should be a string exactly as you might spec‐
586 ify on the commandline. Shell globbing is not evaluated.
587 [compression]
589 Specify various compression settings, such as compression utility, com‐
590 pression level, etc.
591 method = gzip| gzip-rsyncable | pigz | bzip2 | pbzip2 | lzop | lzma |
593 gpg | zstd
594 Define which compression method to use. Note that some methods may
595 not be available by default on every system and may need to be com‐
596 piled or installed.
597 For gpg compression, a key should already exist(gpg --gen-key) and
599 default-recipient must be configured in ~/.gnupg/gpg.conf.
600 inline = yes | no
602 Whether or not to pipe the output of the backup command into the
603 compression utility. Enabling this is recommended since it usually
604 only marginally impacts performance, particularly when using a lower
605 compression level.
606 level = 0-9
608 Specify the compression ratio. The lower the number, the lower the
609 compression ratio, but the faster the backup will take. Generally,
610 setting the lever to 1 or 2 results in favorable compression of tex‐
611 tual data and is noticeably faster than the higher levels. Setting
612 the level to 0 effectively disables compression.
613 bin-path = <full path to utility>
615 This only needs to be defined if the compression utility is not in
616 the usual places or not in the system path.
617 split = yes | no
619 Defautls to no. If set the backup will be piped through the split
620 command. This may be useful for user's with large databases, as
621 some backup systems perform better with many smaller files instead
622 of 1 large one. This defaults to 1GB file size, so this option isn't
623 helpful if your dumps are smaller than that.
624 New in version 1.1.13.
626 MySQL connection info [mysql:client]
629 These are optional and, if left undefined, Holland will try to login
630 using the standard .my.cnf conventions.
631 user = <user>
633 The user to connect to MySQL as.
635 password = <password>
637 The password for the MySQL user
639 socket = <socket>
641 The socket file to connect to MySQL with.
643 host = <host>
645 This would be used for connecting to MySQL remotely.
647 port = <port>
649 Used if MySQL is running on a port other than 3306.
651 mysqldump LVM Provider Configuration [mysqldump-lvm]
653 Backs up one or more MySQL databases by creating an LVM snapshot and
654 then starting a instance of MySQL on top of it to then perform a mysql‐
655 dump. This effectively produces a non-blocking logical backup.
656 [mysql-lvm]
658 snapshot-size = <size-in-MB>
659 The size of the snapshot itself. By default it is 20% of the size of
660 the MySQL LVM mount or the remaining free-space in the Volume Group
661 (if there is less than 20% available) up to 15GB. If snapshot-size
662 is defined, the number represents the size of the snapshot in
663 megabytes.
664 snapshot-name = <name>
666 The name of the snapshot, the default being the name of the MySQL
667 LVM volume + "_snapshot" (ie Storage-MySQL_snapshot)
668 snapshot-mountpoint = <path>
670 Where to mount the snapshot. By default a randomly generated direc‐
671 tory under /tmp is used.
672 innodb-recovery = yes | no (default: no)
674 Whether or not to run an InnoDB recovery operation. This avoids
675 needing to do so during a restore, though will make the backup
676 process itself take longer.
677 lock-tables = yes | no (default: yes)
679 Whether or not to run a FLUSH TABLES WITH READ LOCK to grab various
680 bits of information (such as the binary log name and position). Dis‐
681 abling this requires that binary logging is disabled and InnoDB is
682 being used exclusively. Otherwise, it is possible that the backup
683 could contain crashed tables.
684 extra-flush-tables = yes | no (default: yes)
686 Whether or not to run a FLUSH TABLES before running the full FLUSH
687 TABLES WITH READ LOCK. Should make the FLUSH TABLES WITH READ LOCK
688 operation a bit faster.
689 [mysqld]
691 mysqld-exe = <path>[, <path>...] (default: mysqld in PATH,
692 /usr/libexec/mysqld)
693 This provides a list of locations where the mysqld process to use
694 might be found. This is searched in order of entries in this list.
695 user = <name>
697 The --user parameter to use with mysqld.
698 innodb-buffer-pool-size = <size> (default: 128M)
700 How large to size the innodb-buffer-pool-size.
701 tmpdir = <path> (default: system tempdir)
703 Path to the --tmpdir that mysqld should use.
704 log-error = <path> (default: tempdir/holland_lvm.log)
706 Define path for mysqld's error log. The default location get cleaned
707 up by Holland after the backup is complete. This settings allows the
708 user to define the log file in another location and can be useful
709 for debugging issue with the MySQL instance running on the snapshot.
710 New in version 1.0.9.
712 [mysqldump]
715 mysqldump-lvm supports almost all of the options from the mysqldump
716 plugin. --master-data is not supported, as the mysqld process will not
717 read binary logs, so this plugin will automatically disable
718 bin-log-position, if set.
719 Binary log information from SOHW MASTER STATUS and SHOW SLAVE STATUS is
721 recorded in the ${backup_directory}/backup.conf file under the
722 [mysql:replication] section.
723 [compression]
725 Specify various compression settings, such as compression utility, com‐
726 pression level, etc.
727 method = gzip| gzip-rsyncable | pigz | bzip2 | pbzip2 | lzop | lzma |
729 gpg | zstd
730 Define which compression method to use. Note that some methods may
731 not be available by default on every system and may need to be com‐
732 piled or installed.
733 For gpg compression, a key should already exist(gpg --gen-key) and
735 default-recipient must be configured in ~/.gnupg/gpg.conf.
736 inline = yes | no
738 Whether or not to pipe the output of the backup command into the
739 compression utility. Enabling this is recommended since it usually
740 only marginally impacts performance, particularly when using a lower
741 compression level.
742 level = 0-9
744 Specify the compression ratio. The lower the number, the lower the
745 compression ratio, but the faster the backup will take. Generally,
746 setting the lever to 1 or 2 results in favorable compression of tex‐
747 tual data and is noticeably faster than the higher levels. Setting
748 the level to 0 effectively disables compression.
749 bin-path = <full path to utility>
751 This only needs to be defined if the compression utility is not in
752 the usual places or not in the system path.
753 split = yes | no
755 Defautls to no. If set the backup will be piped through the split
756 command. This may be useful for user's with large databases, as
757 some backup systems perform better with many smaller files instead
758 of 1 large one. This defaults to 1GB file size, so this option isn't
759 helpful if your dumps are smaller than that.
760 New in version 1.1.13.
762 MySQL connection info [mysql:client]
765 These are optional and, if left undefined, Holland will try to login
766 using the standard .my.cnf conventions.
767 user = <user>
769 The user to connect to MySQL as.
771 password = <password>
773 The password for the MySQL user
775 socket = <socket>
777 The socket file to connect to MySQL with.
779 host = <host>
781 This would be used for connecting to MySQL remotely.
783 port = <port>
785 Used if MySQL is running on a port other than 3306.
787 Xtrabackup Provider Configuration [xtrabackup]
789 Backs up a MySQL instance using Percona's Xtrabackup tool.
790 [xtrabackup]
792 global-defaults = <path> (default: /etc/my.cnf)
793 The MySQL configuration file for xtrabackup to parse. This is
794 !include'd into the my.cnf the xtrabackup plugin generates
795 innobackupex = <name> (default: innobackupex-1.5.1)
797 The path to the innobackupex script to run. If this is a relative
798 path this will be found in holland's environment PATH as configured
799 in /etc/holland/holland.conf.
800 Changed in version 1.1.12: For xtrabackup version 8.0 or greater, this
802 option is ignored
803 ibbackup = <name>
806 The path to the ibbackup command to use. By default, no --ibbackup
807 option is pass to the innobackupex script. Usually innobackupex
808 will detect this by itself and this should not need to be set.
809 stream = tar|xbstream|yes|no (default: tar)
811 Whether to generate a streaming backup.
812 Changed in version 1.0.8: 'tar' and 'xbstream' are now valid options.
814 The old stream = yes is now equivalent to stream = tar and stream = no
815 disables streaming entirely and will result in a normal directory copy
816 with xtrabackup
817 Changed in version 1.1.12: For xtrabackup version 8.0 or
820 greater,'xstream' will be used unless this value is set to 'no'
821 apply-logs = yes | no (default: yes)
824 Whether to run innobackupex --apply-logs at the end of the backup.
825 This is only supported when performing a non-streaming, non-com‐
826 pressed backup. In this case, even if apply-logs = yes (the
827 default), the prepare stage will be skipped. Even with an uncom‐
828 pressed, non-streaming backup you may want to disable apply-logs if
829 you wish to use incremental backups.
830 New in version 1.0.8.
832 slave-info = yes | no (default: yes)
835 Whether to enable the --slave-info innobackupex option
836 safe-slave-backup = yes | no (default: yes)
838 Whether to enable the --safe-slave-backup innobackupex option.
839 no-lock = yes | no (default: no)
841 Whether to enable the --no-lock innobackupex option
842 tmpdir = <path> (default: ${backup_directory})
844 The path for the innobackupex --tmpdir option. By default this will
845 use the current holland backup directory to workaround the following
846 bug: https://bugs.launchpad.net/percona-xtrabackup/+bug/1007446
847 New in version 1.0.8.
849 additional-options = <option>[, <option>...]
852 A list of additional options to pass to innobackupex. This is a
853 comma separated list of options.
854 pre-command = <command-string>
856 A command to run prior to running this xtrabackup run. This can be
857 used, for instance, to generate a mysqldump schema dump prior to
858 running xtrabackup. instances of ${backup_directory} will be
859 replaced with the current holland backup directory where the xtra‐
860 backup data will be stored.
861 [compression]
863 Specify various compression settings, such as compression utility, com‐
864 pression level, etc.
865 method = gzip| gzip-rsyncable | pigz | bzip2 | pbzip2 | lzop | lzma |
867 gpg | zstd
868 Define which compression method to use. Note that some methods may
869 not be available by default on every system and may need to be com‐
870 piled or installed.
871 For gpg compression, a key should already exist(gpg --gen-key) and
873 default-recipient must be configured in ~/.gnupg/gpg.conf.
874 inline = yes | no
876 Whether or not to pipe the output of the backup command into the
877 compression utility. Enabling this is recommended since it usually
878 only marginally impacts performance, particularly when using a lower
879 compression level.
880 level = 0-9
882 Specify the compression ratio. The lower the number, the lower the
883 compression ratio, but the faster the backup will take. Generally,
884 setting the lever to 1 or 2 results in favorable compression of tex‐
885 tual data and is noticeably faster than the higher levels. Setting
886 the level to 0 effectively disables compression.
887 bin-path = <full path to utility>
889 This only needs to be defined if the compression utility is not in
890 the usual places or not in the system path.
891 split = yes | no
893 Defautls to no. If set the backup will be piped through the split
894 command. This may be useful for user's with large databases, as
895 some backup systems perform better with many smaller files instead
896 of 1 large one. This defaults to 1GB file size, so this option isn't
897 helpful if your dumps are smaller than that.
898 New in version 1.1.13.
900 MySQL connection info [mysql:client]
903 These are optional and, if left undefined, Holland will try to login
904 using the standard .my.cnf conventions.
905 user = <user>
907 The user to connect to MySQL as.
909 password = <password>
911 The password for the MySQL user
913 socket = <socket>
915 The socket file to connect to MySQL with.
917 host = <host>
919 This would be used for connecting to MySQL remotely.
921 port = <port>
923 Used if MySQL is running on a port other than 3306.
925 pgdump Provider Configuration [pgdump]
927 Backs up a PostgreSQL instance using the pgdump utility.
928 [pgdump]
930 format = custom | tar | plain (default: custom)
931 Defines the --format option for pg_dump. This defaults to --for‐
932 mat=custom. The custom format is required for pg_restore to do par‐
933 tial restore as well as enabling parallel restores. If set to cus‐
934 tom, the --compress option will be passed to pgdump
935 additional-options = <command-string>
937 Pass additional options to the pg_dump command
938 Only the 'level' option will be used if 'format=custom'
940 [compression]
942 Specify various compression settings, such as compression utility, com‐
943 pression level, etc.
944 method = gzip| gzip-rsyncable | pigz | bzip2 | pbzip2 | lzop | lzma |
946 gpg | zstd
947 Define which compression method to use. Note that some methods may
948 not be available by default on every system and may need to be com‐
949 piled or installed.
950 For gpg compression, a key should already exist(gpg --gen-key) and
952 default-recipient must be configured in ~/.gnupg/gpg.conf.
953 inline = yes | no
955 Whether or not to pipe the output of the backup command into the
956 compression utility. Enabling this is recommended since it usually
957 only marginally impacts performance, particularly when using a lower
958 compression level.
959 level = 0-9
961 Specify the compression ratio. The lower the number, the lower the
962 compression ratio, but the faster the backup will take. Generally,
963 setting the lever to 1 or 2 results in favorable compression of tex‐
964 tual data and is noticeably faster than the higher levels. Setting
965 the level to 0 effectively disables compression.
966 bin-path = <full path to utility>
968 This only needs to be defined if the compression utility is not in
969 the usual places or not in the system path.
970 split = yes | no
972 Defautls to no. If set the backup will be piped through the split
973 command. This may be useful for user's with large databases, as
974 some backup systems perform better with many smaller files instead
975 of 1 large one. This defaults to 1GB file size, so this option isn't
976 helpful if your dumps are smaller than that.
977 New in version 1.1.13.
979 [pgauth]
982 username = <name>
983 Username for pg_dump to authenticate with
984 password = <string>
986 Password for pg_dump to authenticate with
987 hostname = <string>
989 Hostname for pg_dump to connect with
990 port = <integer>
992 TCP port for pg_dump to connect on
993 mongodump Provider Configuration [mongodump]
995 This plugin performs logical backups of a MongoDB by using the mongo‐
996 dump utility.
997 [mongodump]
999 host = <string>
1000 Hostname for mongodump to connect with
1001 username = <name>
1003 Username for mongodump to authenticate with
1004 password = <string>
1006 Password for mongodump to authenticate with
1007 port = <integer>
1009 TCP port for mongodump to connect on
1010 [compression]
1012 Specify various compression settings, such as compression utility, com‐
1013 pression level, etc.
1014 method = gzip| gzip-rsyncable | pigz | bzip2 | pbzip2 | lzop | lzma |
1016 gpg | zstd
1017 Define which compression method to use. Note that some methods may
1018 not be available by default on every system and may need to be com‐
1019 piled or installed.
1020 For gpg compression, a key should already exist(gpg --gen-key) and
1022 default-recipient must be configured in ~/.gnupg/gpg.conf.
1023 inline = yes | no
1025 Whether or not to pipe the output of the backup command into the
1026 compression utility. Enabling this is recommended since it usually
1027 only marginally impacts performance, particularly when using a lower
1028 compression level.
1029 level = 0-9
1031 Specify the compression ratio. The lower the number, the lower the
1032 compression ratio, but the faster the backup will take. Generally,
1033 setting the lever to 1 or 2 results in favorable compression of tex‐
1034 tual data and is noticeably faster than the higher levels. Setting
1035 the level to 0 effectively disables compression.
1036 bin-path = <full path to utility>
1038 This only needs to be defined if the compression utility is not in
1039 the usual places or not in the system path.
1040 split = yes | no
1042 Defautls to no. If set the backup will be piped through the split
1043 command. This may be useful for user's with large databases, as
1044 some backup systems perform better with many smaller files instead
1045 of 1 large one. This defaults to 1GB file size, so this option isn't
1046 helpful if your dumps are smaller than that.
1047 New in version 1.1.13.
1049 Mariabackup Provider Configuration [mariabackup]
1052 Backs up a MySQL instance using mariabackup tool.
1053 [mariabackup]
1055 global-defaults = <path> (default: /etc/my.cnf)
1056 The MySQL configuration file for mariabackup to parse. This is
1057 !include'd into the my.cnf the mariabackup plugin generates
1058 innobackupex = <name> (default: innobackupex-1.5.1)
1060 The path to the innobackupex script to run. If this is a relative
1061 path this will be found in holland's environment PATH as configured
1062 in /etc/holland/holland.conf.
1063 ibbackup = <name>
1065 The path to the ibbackup command to use. By default, no --ibbackup
1066 option is pass to the innobackupex script. Usually innobackupex
1067 will detect this by itself and this should not need to be set.
1068 stream = mbstream(default: tar)
1070 Placeholder
1071 apply-logs = yes | no (default: yes)
1073 Whether to run innobackupex --apply-logs at the end of the backup.
1074 This is only supported when performing a non-streaming, non-com‐
1075 pressed backup. In this case, even if apply-logs = yes (the
1076 default), the prepare stage will be skipped. Even with an uncom‐
1077 pressed, non-streaming backup you may want to disable apply-logs if
1078 you wish to use incremental backups.
1079 slave-info = yes | no (default: yes)
1081 Whether to enable the --slave-info innobackupex option
1082 safe-slave-backup = yes | no (default: yes)
1084 Whether to enable the --safe-slave-backup innobackupex option.
1085 no-lock = yes | no (default: no)
1087 Whether to enable the --no-lock innobackupex option
1088 tmpdir = <path> (default: ${backup_directory})
1090 The path for the innobackupex --tmpdir option.
1091 additional-options = <option>[, <option>...]
1093 A list of additional options to pass to innobackupex. This is a
1094 comma separated list of options.
1095 pre-command = <command-string>
1097 A command to run prior to running this mariabackup run. This can be
1098 used, for instance, to generate a mysqldump schema dump prior to
1099 running mariabackup. instances of ${backup_directory} will be
1100 replaced with the current holland backup directory where the mari‐
1101 abackup data will be stored.
1102 [compression]
1104 Specify various compression settings, such as compression utility, com‐
1105 pression level, etc.
1106 method = gzip| gzip-rsyncable | pigz | bzip2 | pbzip2 | lzop | lzma |
1108 gpg | zstd
1109 Define which compression method to use. Note that some methods may
1110 not be available by default on every system and may need to be com‐
1111 piled or installed.
1112 For gpg compression, a key should already exist(gpg --gen-key) and
1114 default-recipient must be configured in ~/.gnupg/gpg.conf.
1115 inline = yes | no
1117 Whether or not to pipe the output of the backup command into the
1118 compression utility. Enabling this is recommended since it usually
1119 only marginally impacts performance, particularly when using a lower
1120 compression level.
1121 level = 0-9
1123 Specify the compression ratio. The lower the number, the lower the
1124 compression ratio, but the faster the backup will take. Generally,
1125 setting the lever to 1 or 2 results in favorable compression of tex‐
1126 tual data and is noticeably faster than the higher levels. Setting
1127 the level to 0 effectively disables compression.
1128 bin-path = <full path to utility>
1130 This only needs to be defined if the compression utility is not in
1131 the usual places or not in the system path.
1132 split = yes | no
1134 Defautls to no. If set the backup will be piped through the split
1135 command. This may be useful for user's with large databases, as
1136 some backup systems perform better with many smaller files instead
1137 of 1 large one. This defaults to 1GB file size, so this option isn't
1138 helpful if your dumps are smaller than that.
1139 New in version 1.1.13.
1141 MySQL connection info [mysql:client]
1144 These are optional and, if left undefined, Holland will try to login
1145 using the standard .my.cnf conventions.
1146 user = <user>
1148 The user to connect to MySQL as.
1150 password = <password>
1152 The password for the MySQL user
1154 socket = <socket>
1156 The socket file to connect to MySQL with.
1158 host = <host>
1160 This would be used for connecting to MySQL remotely.
1162 port = <port>
1164 Used if MySQL is running on a port other than 3306.
1166 Backup-Set Configs
1168 Backup-Set configuration files largely inherit the configuration
1169 options of the specified provider. To define a provider for the backup
1170 set, you must put the following at the top of the backup set configura‐
1171 tion file:
1172 [holland:backup]
1174 plugin = <plugin>
1175 backups-to-keep = #
1176 estimated-size-factor = #
1177 historic-size = <yes|no>
1178 historic-size-factor = #
1179 historic-estimated-size-factor = #
1180 Configuration Options
1182 plugin = <plugin>
1183 This is the name of the provider that will be used for the
1184 backup-set. This is required in order for the backup-set to func‐
1185 tion.
1186 backups-to-keep = #
1188 Specifies the number of backups to keep for a backup-set. Defaults
1189 to retaining 1 backup.
1190 estimated-size-factor = #
1192 Specifies the scale factor when Holland decides if there is enough
1193 free space to perform a backup. The default is 1.0 and this number
1194 is multiplied against what each individual plugin reports its esti‐
1195 mated backup size when Holland is verifying sufficient free space
1196 for the backupset.
1197 As of Holland 1.1.1, the 'historic' backup size will be used. If
1199 Holland is unable to determine that size, it will default back to
1200 this.
1201 auto-purge-failures = yes | no
1203 Specifies whether to keep a failed backup or to automatically remove
1204 the backup directory. By default this is on with the intention that
1205 whatever process is calling holland will retry when a backup fails.
1206 This behavior can be disabled by setting auto-purge-failures = no
1207 when partial backups might be useful or when troubleshooting a
1208 backup failure.
1209 purge-policy = manual | before-backup | after-backup
1211 Specifies when to run the purge routine on a backupset. By default
1212 this is run after a new successful backup completes. Up to back‐
1213 ups-to-keep backups will be retained including the most recent.
1214 purge-policy = before-backup will run the purge routine just before
1216 a new backup starts. This will retain up to backups-to-keep backups
1217 before the new backup is even started allowing purging all previous
1218 backups if backups-to-keep is set to 0. This behavior is useful if
1219 some other process is retaining backups off-server and disk space is
1220 at a premium.
1221 purge-policy = manual will never run the purge routine automati‐
1223 cally. Either holland purge must be run externally or an explicit
1224 removal of desired backup directories can be done at some later
1225 time.
1226 before-backup-command = string
1228 Run a shell command before a backup starts. This allows a command
1229 to perform some action before the backup starts such as setting up
1230 an iptables rule (taking a mysql slave out of a load balancer) or
1231 aborting the backup based on some external condition.
1232 The backup will fail if this command exits with a non-zero status.
1234 New in version 1.0.7.
1236 after-backup-command = string
1239 Run a shell command after a backup completes. This allows a command
1240 to perform some action when a backup completes successfully such as
1241 sending out a success notification.
1242 The backup will fail if this command exits with a non-zero status.
1244 New in version 1.0.7.
1246 failed-backup-command = string
1249 Run a shell command if a backup starts. This allows some command to
1250 perform some action when a backup fails such as sending out a fail‐
1251 ure notification.
1252 The backup will fail if this command exits with a non-zero status.
1254 New in version 1.0.7.
1256 historic-size = yes | no
1259 Defaults to yes. Check for the 'backup.conf' file in the 'newest'
1260 spooled folder for the running "backupset". If the configuration
1261 file exists and contains 'estimated-size' and 'on-disk-size', these
1262 values will be used to decide if holland has enough free space for a
1263 backupset. This means Holland will use values from the previous
1264 backup to estimate the size of the next backup.
1265 New in version 1.1.1.
1267 historic-size-factor = #
1270 Defaults to '1.5'. If the estimated size of the database has changed
1271 by more than this multiple, the 'estimated-size-factor' value will
1272 be used to determine if there is sufficient free space for the back‐
1273 upset.
1274 New in version 1.1.1.
1276 historic-estimated-size-factor = #
1279 Defaults to '1.1'. Specifies the scale factor when Holland decides
1280 if there is enough free space to perform a backup. Holland will
1281 throw an error if the system has less free space than the last
1282 backup size multiplied by this value
1283 New in version 1.1.1.
1285 For all hook commands, Holland will perform simple text substitution on
1288 the three parameters:
1289 · hook - name of the hook being called (one of: before-backup-com‐
1291 mand, after-backup-command, failed-backup-command)
1292 · backupdir - path to the current backup directory (e.g.
1294 /var/spool/holland/mysqldump/YYYYmmdd_HHMMSS)
1295 · backupset - name of the backupset being run (e.g. 'mysql-lvm')
1297 For example:
1299 [holland:backup]
1301 plugin = mysqldump
1302 before-backup-command = /usr/local/bin/my-custom-script --hook ${hook} --backupset ${backupset} --backupdir ${backupdir}
1303 after-backup-command = echo ${backupset} completed successfully. Files are in ${backupdir}
1304 [mysqldump]
1306 ...
1307 Backup-Set files are defined in the "backupsets" directory which is, by
1309 default, /etc/holland/backupsets. The name of the backup-set is defined
1310 by its configuration filename and can really be most anything. That
1311 means backup-sets can be organized in any arbitrary way, although
1312 backup set files must end in .conf. The file extension is not part of
1313 the name of the backup-set.
1314 As noted above, in order for a backup-set to be active, it must be
1316 listed in the backupsets variable.
1317 Backups are placed under the directory defined in the backup_directory
1319 section of the main configuration file. Each backup resides under a
1320 directory corresponding to the backup-set name followed by a
1321 date-encoded directory.
1322
1324 Holland Core Team
1325
1327 2018, Holland Core Team
13281.1.13 Mar 12, 2019 HOLLAND(1)