1DEBMIRROR(1)          User Contributed Perl Documentation         DEBMIRROR(1)
2
3
4

NAME

6       debmirror - Debian partial mirror script, with ftp, http or rsync and
7       package pool support
8

SYNOPSIS

10       debmirror [options] mirrordir
11

DESCRIPTION

13       This program downloads and maintains a partial local Debian mirror. It
14       can mirror any combination of architectures, distributions, and
15       sections. Files are transferred by ftp, and package pools are fully
16       supported. It also does locking and updates trace files.
17
18       The partial mirror created by this program is not suitable to be used
19       as a public Debian mirror. If that is your aim, you should instead
20       follow the instructions at <http://www.debian.org/mirrors/ftpmirror>.
21
22       This program mirrors in three steps.
23
24       1. download Packages and Sources files
25           First it downloads all Packages and Sources files for the subset of
26           Debian it was instructed to get.
27
28       2. download everything else
29           The Packages and Sources files are scanned, to build up a list of
30           all the files they refer to. A few other miscellaneous files are
31           added to the list.  Then the program makes sure that each file in
32           the list is present on the local mirror and is up-to-date, using
33           file size (and optionally checksum) checks.  Any necessary files
34           are downloaded.
35
36       3. clean up unknown files
37           Any files and directories on the local mirror that are not in the
38           list are removed.
39

OPTIONS

41       mirrordir
42           This required (unless defined in a configuration file) parameter
43           specifies where the local mirror directory is. If the directory
44           does not exist, it will be created. Be careful; telling this
45           program that your home directory is the mirrordir is guaranteed to
46           replace your home directory with a Debian mirror!
47
48       -p, --progress
49           Displays progress bars as files are downloaded.
50
51       -v, --verbose
52           Displays progress between file downloads.
53
54       --debug
55           Enables verbose debug output, including ftp protocol dump.
56
57       --dry-run
58           Simulate a mirror run. This will still download the meta files to
59           the ./.temp working directory, but won't replace the old meta
60           files, won't download debs and source files and only simulates
61           cleanup.
62
63       --help
64           Display a usage summary.
65
66       --version
67           Print version number and exit.
68
69       -h, --host=remotehost
70           Specify the remote host to mirror from. Defaults to ftp.debian.org,
71           you are strongly encouraged to find a closer mirror.
72
73       -r, --root=directory
74           Specifies the directory on the remote host that is the root of the
75           Debian archive. Defaults to debian, which will work for most
76           mirrors. The root directory has a dists subdirectory.
77
78       --method=method
79           Specify the method to download files. Currently, supported methods
80           are ftp, http, https, and rsync. The file method is experimentally
81           supported.
82
83       --passive
84           Download in passive mode when using ftp.
85
86       -u, --user=remoteusername
87           Specify the remote user name to use to log into the remote host.
88           Defaults to "anonymous".
89
90       --passwd=remoteuserpassword
91           Specify the remote user password to use to log into the remote ftp
92           host.  It is used with --user and defaults to "anonymous@".
93
94       --proxy=http://user:pass@url:port/
95           Specifies the http proxy (like Squid) to use for http or ftp
96           methods.
97
98       -d, --dist=foo[,bar,..]
99           Specify the distribution (etch, lenny, squeeze, sid) of Debian to
100           mirror. This switch may be used multiple times, and multiple
101           distributions may be specified at once, separated by commas.
102
103           You may also use the stable, testing, unstable, names.
104
105       --omit-suite-symlinks
106           With this option set, debmirror will not create the symlink from
107           suite to codename.  This is needed for example when mirroring
108           archived Debian releases as they will all have either "stable" or
109           "oldstable" as suite in their Release files.
110
111       -s, --section=foo[,bar,..]
112           Specify the section of Debian to mirror. Defaults to
113           "main,contrib,non-free,main/debian-installer".
114
115       -a, --arch=foo[,bar,..]
116           Specify the architectures to mirror. The default is --arch=i386.
117           Specifying --arch=none will mirror no archs.
118
119       --rsync-extra=foo[,bar,..]
120           Allows you to also mirror files from a number of directories that
121           are not part of the package archive itself.
122
123           Debmirror will always use rsync for the transfer of these files,
124           irrespective of what transfer method is specified in the --method
125           option.  This will therefore not work if your remote mirror does
126           not support rsync, or if the mirror needs a different --root option
127           for rsync than for the main transfer method specified with
128           --method.
129
130           Note that excluding individual files in the directories is not
131           supported.
132
133           The following values are supported.
134
135           doc
136             Download all files and subdirectories in doc directory, and all
137             README files in the root directory of the archive.
138
139           indices
140             Download all files and subdirectories in indices directory. Note
141             that this directory can contain some rather large files; don't
142             include this type unless you know you need these files.
143
144           tools
145             Download all files and subdirectories in tools directory.
146
147           trace
148             Download the remote mirror's trace files for the archive
149             (project/trace/*).  This is enabled by default.
150
151           none
152             This can be used to disable getting extra files with rsync.
153
154           If specified, the update of trace files will be done at the
155           beginning of the mirror run; the other types are done near the end.
156
157           This switch may be used multiple times, and multiple values may be
158           specified at once, separated by commas; unknown values are ignored.
159
160       --di-dist=dists | foo[,bar,..]
161           Mirror current Debian Installer images for the specified dists.
162           See further the section "Mirroring Debian Installer images" below.
163
164       --di-arch=arches | foo[,bar,..]
165           Mirror current Debian Installer images for the specified
166           architectures.  See further the section "Mirroring Debian Installer
167           images" below.
168
169       --source
170           Include source in the mirror (default).
171
172       --nosource
173           Do not include source.
174
175       --i18n
176           Additionally download Translation-<lang>.* files, which contain
177           translations of package descriptions. Selection of specific
178           translations is possible using the --include and --exclude options.
179           The default is to download only the English file.
180
181       --getcontents
182           Additionally download Contents.<arch>.gz files. Note that these
183           files can be relatively big and can change frequently, especially
184           for the testing and unstable suites. Use of the available diff
185           files is strongly recommended (see the --diff option).
186
187       --checksums
188           Use checksums to determine if files on the local mirror that are
189           the correct size actually have the correct content. Not enabled by
190           default, because it is too paranoid, and too slow.
191
192           When the state cache is used, debmirror will only check checksums
193           during runs where the cache has expired or been invalidated, so it
194           is worth considering to use these two options together.
195
196       --ignore-missing-release
197           Don't fail if the Release file is missing.
198
199       --check-gpg, --no-check-gpg
200           Controls whether gpg signatures from the Release.gpg file should be
201           checked. The default is to check signatures.
202
203       --keyring=file
204           Use file as an additional gpg-format keyring.  May be given
205           multiple times.
206
207           Note that these will be used in addition to
208           $GNUPGHOME/trustedkeys.gpg.  The latter can be removed from the set
209           of keyrings by setting $GNUPGHOME to something non-existent when
210           using this option.
211
212           On a typical Debian system, the Debian archive keyring can be used
213           directly with this option:
214
215               debmirror --keyring /usr/share/keyrings/debian-archive-keyring.gpg ...
216
217       --ignore-release-gpg
218           Don't fail if the Release.gpg file is missing. If the file does
219           exist, it is mirrored and verified, but any errors are ignored.
220
221       --ignore=regex
222           Never delete any files whose filenames match the regex. May be used
223           multiple times.
224
225       --exclude=regex
226           Never download any files whose filenames match the regex. May be
227           used multiple times.
228
229       --include=regex
230           Don't exclude any files whose filenames match the regex. May be
231           used multiple times.
232
233       --exclude-deb-section=regex
234           Never download any files whose Debian Section (games, doc, oldlibs,
235           science, ...) match the regex. May be used multiple times.
236
237       --limit-priority=regex
238           Limit download to files whose Debian Priority (required, extra,
239           optional, ...) match the regex. May be used multiple times.
240
241       --exclude-field=fieldname=regex
242           Never download any binary packages where the contents of fieldname
243           match the regex. May be used multiple times. If this option is used
244           and the mirror includes source packages, only those source packages
245           corresponding to included binary packages will be downloaded.
246
247       --include-field=fieldname=regex
248           Don't exclude any binary packages where the contents of fieldname
249           match the regex. May be used multiple times. If this option is used
250           and the mirror includes source packages, only those source packages
251           corresponding to included binary packages will be downloaded.
252
253       -t, --timeout=seconds
254           Specifies the timeout to use for network operations (either FTP or
255           rsync).  Set this to a higher value if you experience failed
256           downloads. Defaults to 300 seconds.
257
258       --max-batch=number
259           Download at most max-batch number of files (and ignore rest).
260
261       --rsync-batch=number
262           Download at most number of files with each rsync call and then
263           loop.
264
265       --rsync-options=options
266           Specify alternative rsync options to be used. Default options are
267           "-aIL --partial". Care must be taken when specifying alternative
268           options not to disrupt operations, it's best to only add to those
269           options.
270
271           The most likely option to add is "--bwlimit=x" to avoid saturating
272           the bandwidth of your link.
273
274       --postcleanup
275           Clean up the local mirror but only after mirroring is complete and
276           only if there was no error.
277
278           This is the default, because it ensures that the mirror is
279           consistent at all times.
280
281       --precleanup
282           Clean up the local mirror before starting mirroring.
283
284           This option may be useful if you have limited disk space, but it
285           will result in an inconsistent mirror when debmirror is running.
286
287           The deprecated --cleanup option also enables this mode.
288
289       --nocleanup
290           Do not clean up the local mirror.
291
292       --skippackages
293           Don't re-download Packages and Sources files.  Useful if you know
294           they are up-to-date.
295
296       --diff=use|mirror|none
297           If --diff=use is specified and the Release file contains entries
298           for diff files, then debmirror will attempt to use them to update
299           Packages, Sources, and Contents files (which can significantly
300           reduce the download size for meta files), but will not include them
301           in the mirror. This is the default behavior and avoids having time
302           consuming diff files for a fast local mirror.
303
304           Specifying --diff=mirror does the same as use, but will also
305           include the downloaded diff files in the local mirror. Specify
306           --diff=none to completely ignore diff files.
307
308           Note that if rsync is used as method to download files and the
309           archive being mirrored has "rsyncable" gzipped meta files, then
310           using --diff=none may be the most efficient way to download them.
311           See the gzip(1) man page for information about its rsyncable
312           option.
313
314       --gzip-options=options
315           Specify alternative options to be used when calling gzip(1) to
316           compress meta files after applying diffs. The default options are
317           "-9 -n --rsyncable" which corresponds with the options used to gzip
318           meta files for the main Debian archive.
319
320           These options may need to be modified if the checksum of the file
321           as gzipped by debmirror does not match the checksum listed in the
322           Release file (which will result in the gzipped file being
323           downloaded unnecessarily after diffs were successfully applied).
324
325       --slow-cpu
326           By default debmirror saves some bandwidth by performing cpu-
327           intensive tasks, such as compressing files to generate .gz and .xz
328           files. Use this mode if the computer's CPU is slow, and it makes
329           more sense to use more bandwidth and less CPU.
330
331           This option implies --diff=none.
332
333       --state-cache-days=number
334           Save the state of the mirror in a cache file between runs. The
335           cache will expire after the specified number of days, at which time
336           a full check and cleanup of the mirror will be done. While the
337           cache is valid, debmirror will trust that the mirror is consistent
338           with this cache.
339
340           The cache is only used for files that have a unique name, i.e.
341           binary packages and source files. If a mirror update fails for any
342           reason, the cache will be invalidated and the next run will include
343           a full check.
344
345           Main advantage of using the state cache is that it avoids a large
346           amount of disk access while checking which files need to be
347           fetched. It may also reduce the time required for mirror updates.
348
349       --ignore-small-errors
350           Normally debmirror will report an error if any deb files or sources
351           fail to download and refuse to update the meta data to an
352           inconsistent mirror. Normally this is a good things as it indicates
353           something went wrong during download and should be retried. But
354           sometimes the upstream mirror actually is broken. Specifying
355           --ignore-small-errors causes debmirror to ignore missing or broken
356           deb and source files but still be pedantic about checking meta
357           files.
358
359       --allow-dist-rename
360           The directory name for a dist should be equal to its Codename and
361           not to a Suite. If the local mirror currently has directories named
362           after Suites, debmirror can rename them automatically.  An existing
363           symlink from codename to suite will be removed, but debmirror will
364           automatically create a new symlink suite -> codename (immediately
365           after moving meta files in place). This conversion should only be
366           needed once.
367
368       --disable-ssl-verification
369           When https is used, debmirror checks that the SSL certificate is
370           value.
371
372           If the server has a self-signed certificate, the check can be
373           disabled with this option.
374
375       --debmarshal
376           On each pull, keep the repository meta data from dists/* in a
377           numbered subdirectory, and maintain a symlink latest to the most
378           recent pull.  This is similar to Debmarshal in tracking mode, see
379           debmarshal.debian.net for examples and use.  debmirror cleanup is
380           disabled when this flag is specified.  Separate pool and snapshot
381           cleanup utilities are available at
382           http://code.google.com/p/debmarshal/source/browse/#svn/trunk/repository2
383
384       --config-file=file
385           Specify a configuration file. This option may be repeated to read
386           multiple configuration files. By default debmirror reads
387           /etc/debmirror.conf and ~/.debmirror.conf (see section FILES).
388
389   Experimental options
390       --retry-rsync-packages=number
391           While downloading Packages and related files via rsync, try up to
392           this many times if rsync fails to connect. Defaults to 1, to try
393           only once. (A typical nondefault value is 10. To try an unlimited
394           number of times, use -1 or 0.)
395

USING DEBMIRROR

397   Using regular expressions in options
398       Various options accept regular expressions that can be used to tune
399       what is included in the mirror. They can be any regular expression
400       valid in perl, which also means that extended syntax is standard. Make
401       sure to anchor regular expressions appropriately: this is not done by
402       debmirror.
403
404       The --include and --exclude options can be combined. This combination
405       for example will, if the --i18n option is used, exclude all Translation
406       files, except for the ones for Portuguese (pt) and Brazillian (pt_BR):
407
408         --exclude='/Translation-.*' --include='/Translation-pt.*'
409
410   Mirroring Debian Installer images
411       Debmirror will only mirror the "current" images that are on the remote
412       mirror. At least one of the options --di-dist or --di-arch must be
413       passed to enable mirroring of the images.
414
415       The special values "dists" and "arches" can be used to tell debmirror
416       to use the same dists and architectures for D-I images as for the
417       archive, but it is also possible to specify different values. If either
418       option is not set, it will default to the same values as for the
419       archive.
420
421       If you wish to create custom CD images using for example debian-cd, you
422       will probably also want add the option "--rsync-extra=doc,tools".
423
424       Limitations
425
426       There are no progress updates displayed for D-I images.
427
428   Archive size
429       The tables in the file /usr/share/doc/debmirror/mirror_size give an
430       indication of the space needed to mirror the Debian archive. They are
431       particularly useful if you wish to set up a partial mirror.  Only the
432       size of source and binary packages is included. You should allow for
433       around 1-4 GB of meta data (in ./dists/<dist>) per suite (depending in
434       your settings). Plus whatever space is needed for extra directories
435       (e.g. tools, doc) you wish to mirror.
436
437       The tables also show how much additional space is required if you add a
438       release on top of its predecessor. Note that the additional space
439       needed for testing and (to a lesser extent) unstable varies during the
440       development cycle of a release. The additional space needed for testing
441       is zero immediately after a stable release and grows from that time
442       onwards.
443
444       Note Debmirror keeps an extra copy of all meta data. This is necessary
445       to guarantee that the local mirror stays consistent while debmirror is
446       running.
447

EXAMPLES

449       Simply make a mirror in /srv/mirror/debian, using all defaults (or the
450       settings defined in debmirror.conf):
451
452         debmirror /srv/mirror/debian
453
454       Make a mirror of i386 and sparc binaries, main only, and include both
455       unstable and testing versions of Debian; download from
456       'ftp.kernel.org':
457
458         debmirror -a i386,sparc -d sid -d etch -s main --nosource \
459                   -h ftp.nl.debian.org --progress $HOME/mirror/debian
460
461       Make a mirror using rsync (rsync server is 'ftp.debian.org::debian'),
462       excluding the section 'debug' and the package 'foo-doc':
463
464         debmirror -e rsync $HOME/mirror/debian --exclude='/foo-doc_' \
465                   --exclude-deb-section='^debug$'
466

FILES

468         /etc/debmirror.conf
469         ~/.debmirror.conf
470
471           Debmirror will look for the presence of these files and load them
472           in the indicated order if they exist.
473           See the example in /usr/share/doc/debmirror/examples for syntax.
474
475         ~/.gnupg/trustedkeys.gpg
476
477           When gpg checking is enabled,
478           debmirror uses gpgv to verify Release and Release.gpg using the
479           default keying ~/.gnupg/trustedkeys.gpg. This can be changed by
480           exporting GNUPGHOME resulting in $GNUPGHOME/trustedkeys.gpg being
481           used.  (Note that keyring files can also be specified directly
482           with debmirror's --keyring option -- see above).
483
484           To add the right key to this keyring you can import it from the
485           debian keyring (in case of the debian archive) using:
486
487             gpg --keyring /usr/share/keyrings/debian-archive-keyring.gpg --export \
488                 | gpg --no-default-keyring --keyring trustedkeys.gpg --import
489
490           or download the key from a keyserver:
491
492             gpg --no-default-keyring --keyring trustedkeys.gpg \
493                 --keyserver keyring.debian.org --recv-keys <key ID>
494
495           The <key ID> can be found in the gpgv error message in debmirror:
496           gpgv: Signature made Tue Jan 23 09:07:53 2007 CET using DSA key ID 2D230C5F
497
499       This program is copyright 2000-2001, 2010-2014 by Joey Hess
500       <joeyh@debian.org>, under the terms of the GNU GPL (either version 2 of
501       the licence or, at your option, any later version), copyright 2001-2002
502       by Joerg Wendland <joergland@debian.org>, copyright 2003-2007 by Goswin
503       von Brederlow <goswin-v-b@web.de>, copyright 2009-2010 by Frans Pop
504       <fjp@debian.org>, copyright 2015 by Thaddeus H. Black <thb@debian.org>,
505       and copyright 2016 by Colin Watson <cjwatson@debian.org>.
506
507       The author disclaims any responsibility for any mangling of your
508       system, unexpected bandwidth usage bills, meltdown of the Debian mirror
509       network, etc, that this script may cause. See NO WARRANTY section of
510       GPL.
511

AUTHOR

513        Author:
514          Joey Hess <joeyh@debian.org>
515
516        Previous maintainers:
517          Joerg Wendland <joergland@debian.org>
518          Goswin von Brederlow <goswin-v-b@web.de>
519          Frans Pop <fjp@debian.org>
520          Joey Hess <joeyh@debian.org>
521          Thaddeus H. Black <thb@debian.org>
522
523        Current maintainer:
524          Colin Watson <cjwatson@debian.org>
525

MOTTO

527       Waste bandwidth -- put a partial mirror on your laptop today!
528
529
530
531perl v5.30.1                      2020-01-28                      DEBMIRROR(1)
Impressum