1DEBMIRROR(1) User Contributed Perl Documentation DEBMIRROR(1)
2
6 debmirror - Debian partial mirror script, with ftp, http, hftp or rsync
7 and package pool support
8
10 debmirror [options] <mirrordir>
11
13 This program downloads and maintains a partial local Debian mirror. It
14 can mirror any combination of architectures, distributions, and
15 sections. Files are transferred by ftp, and package pools are fully
16 supported. It also does locking and updates trace files.
17 To support package pools, this program mirrors in three steps.
19 1. download Packages and Sources files
21 First it downloads all Packages and Sources files for the subset of
22 Debian it was instructed to get.
23 2. clean up unknown files
25 Any files and directories on the local mirror that are not in the
26 list are removed.
27 3. download everything else
29 The Packages and Sources files are scanned, to build up a list of
30 all the files they refer to. A few other miscellaneous files are
31 added to the list. Then the program makes sure that each file in
32 the list is present on the local mirror and is up-to-date, using
33 file size (and optionally md5sum) checks. Any necessary files are
34 downloaded.
35
37 <mirrordir>
38 This required (unless defined in a configuration file) parameter
39 specifies where the local mirror directory is. If the directory
40 does not exist, it will be created. Be careful; telling this
41 program that your home directory is the mirrordir is guaranteed to
42 replace your home directory with a Debian mirror!
43 --progress -p
45 Displays progress bars as files are downloaded.
46 --verbose -v
48 Displays progress between file downloads.
49 --debug
51 Enables verbose debug output, including ftp protocol dump.
52 --dry-run
54 Simulate a mirror run. This will still download the meta files to
55 the ./.temp working directory, but won't replace the old meta
56 files, won't download debs and source files and only simulates
57 cleanup.
58 --help
60 Display a usage summary.
61 --host=remotehost -h
63 Specify the remote host to mirror from. Defaults to
64 'ftp.debian.org', you are strongly encouraged to find a closer
65 mirror.
66 --root=directory -r directory
68 Specifies the directory on the remote host that is the root of the
69 Debian archive. Defaults to "debian", which will work for most
70 mirrors. The root directory has a ./dists subdirectory.
71 --method=ftp|hftp|http|rsync -e
73 Specify the method to download files. Currently, supported methods
74 are ftp, hftp (ftp over http proxy), http or rsync.
75 Note: starting with version 1.1 it is no longer needed to add a ':'
77 prefix for the root directory.
78 --passive
80 Download in passive mode.
81 --user=remoteusername -u
83 Specify the remote user name to use to log to the remote host.
84 Helpful when dealing with brain damaged proxy servers. Defaults to
85 anonymous.
86 --passwd=remoteuserpassword
88 Specify the remote user password to use to log into the remote ftp
89 host. It is used with --user and defaults to anonymous@.
90 --proxy=http://user:pass@url:port/
92 Specifies the http proxy (like Squid) to use for http and hftp
93 method.
94 --dist=foo[,bar,..] -d foo
96 Specify the distribution (etch, lenny, squeeze, sid) of Debian to
97 mirror. This switch may be used multiple times, and multiple
98 distributions may be specified at once, separated by commas. Using
99 the links (stable, testing, unstable) does not have the expected
100 results but you may add those links manually. Defaults to mirroring
101 sid.
102 --omit-suite-symlinks
104 With this option set, debmirror will not create the
105 'suite -> codename' symlink. This is needed for example when
106 mirroring archived Debian releases as they will all have either
107 'stable' or 'oldstable' as suite in their Release files.
108 --section=foo[,bar,..] -s foo
110 Specify the section of Debian to mirror. Defaults to
111 main,contrib,non-free,main/debian-installer.
112 --arch=foo[,bar,..] -a foo
114 Specify the architectures to mirror. The default is --arch=i386.
115 Specifying --arch=none will mirror no archs.
116 --adddir directory
118 Also download Packages and Sources files from the specified
119 directory on the remote host (the directory is relative to the root
120 of the Debian archive). This feature is now obsolete and may be
121 removed in a future release.
122 --rsync-extra=foo[,bar,..]
124 Allows to also mirror files from a number of directories that are
125 not part of the package archive itself. Debmirror will always use
126 rsync for the transfer of these files, irrespective of what
127 transfer method is specified in the --method option.
128 This option can therefore not be used if your remote mirror does
130 not support rsync, or if the mirror needs a different --root option
131 for rsync than for the main transfer method specified with
132 --method. Excluding individual files in the directories is not
133 supported.
134 The following values are supported.
136 doc
138 Download all files and subdirectories in ./doc directory, and all
139 README files in the root directory of the archive.
140 indices
142 Download all files and subdirectories in ./indices directory.
143 Note that this directory can contain some rather large files;
144 don't include this type unless you know you need these files.
145 tools
147 Download all files and subdirectories in ./tools directory.
148 trace
150 Download the remote mirror's trace files for the archive
151 (./project/trace/*).
152 If specified, the update of trace files will be done at the
154 beginning of the mirror run; the other types are done near the end.
155 This switch may be used multiple times, and multiple values may be
157 specified at once, separated by comma's; unknown values are
158 ignored.
159 --di-dist=dists | foo[,bar,..]
161 Mirror "current" Debian Installer images for the specified dists.
162 See further the section "Mirroring Debian Installer images" below.
163 --di-arch=arches | foo[,bar,..]
165 Mirror "current" Debian Installer images for the specified
166 architectures. See further the section "Mirroring Debian Installer
167 images" below.
168 --source
170 Include source in the mirror (default).
171 --nosource
173 Do not include source.
174 --i18n
176 Additionally download Translation-<lang>.bz2 files, which contain
177 translations of package descriptions. Selection of specific
178 translations is possible using the --include and --exclude options.
179 --getcontents
181 Additionally download Contents.<arch>.gz files. Note that these
182 files can be relatively big and can change frequently, especially
183 for the testing and unstable suites. Use of the available diff
184 files is strongly recommended (see the --diff option).
185 --md5sums -m
187 Use md5sums to determine if files on the local mirror that are the
188 correct size actually have the correct content. Not enabled by
189 default, because it is too paranoid, and too slow.
190 When the state cache is used, debmirror will only check md5sums
192 during runs where the cache has expired or been invalidated, so it
193 is worth considering to use these two options together.
194 --ignore-missing-release
196 Don't fail if the Release file is missing.
197 --ignore-release-gpg
199 Don't fail if the Release.gpg file is missing. If the file does
200 exist, it is mirrored and verified, but any errors are ignored.
201 --ignore=regex
203 Never delete any files whose filenames match the regex. May be used
204 multiple times.
205 --exclude=regex
207 Never download any files whose filenames match the regex. May be
208 used multiple times.
209 --include=regex
211 Don't exclude any files whose filenames match the regex. May be
212 used multiple times.
213 --exclude-deb-section=regex
215 Never download any files whose Debian Section (games, doc, oldlibs,
216 science, ...) match the regex. May be used multiple times.
217 --limit-priority=regex
219 Limit download to files whose Debian Priority (required, extra,
220 optional, ...) match the regex. May be used multiple times.
221 --timeout=seconds -t
223 Specifies the timeout to use for network operations (either FTP or
224 rsync). Set this to a higher value if you experience failed
225 downloads. Defaults to 300 seconds.
226 --max-batch=number
228 Download at most max-batch number of files (and ignore rest).
229 --rsync-batch=number
231 Download at most number of files with each rsync call and then
232 loop.
233 --rsync-options=options
235 Specify alternative rsync options to be used. Default options are
236 "-aIL --partial". Care must be taken when specifying alternative
237 options not to disrupt operations, it's best to only add to those
238 options.
239 The most likely option to add is "--bwlimit=x" to avoid saturating
241 the bandwidth of your link.
242 --postcleanup
244 Clean up the local mirror but only after mirroring is complete and
245 only if there was no error. This is the default.
246 --cleanup
248 Do clean up any unknown files and directories on the local mirror
249 (see step 2 above).
250 --nocleanup
252 Do not clean up the local mirror after mirroring is complete.
253 --skippackages
255 Don't re-download Packages and Sources files. Useful if you know
256 they are up-to-date.
257 --diff=use|mirror|none
259 If --diff=use is specified and the Release file contains entries
260 for diff files, then debmirror will attempt to use them to update
261 Packages, Sources and Contents files (which can significantly
262 reduce the download size for meta files), but will not include them
263 in the mirror. This is the default behavior and avoids having time
264 consuming diff files for a fast local mirror.
265 Specifying --diff=mirror does the same as 'use', but will also
267 include the downloaded diff files in the local mirror. Specify
268 --diff=none to completely ignore diff files.
269 Note that if rsync is used as method to download files and the
271 archive being mirrored has "rsyncable" gzipped meta files, then
272 using --diff=none may be the most efficient way to download them.
273 See the gzip(1) man page for information about its rsyncable
274 option.
275 --gzip-options=options
277 Specify alternative options to be used when calling gzip to
278 compress meta files after applying diffs. The default options are
279 "-9 -n --rsyncable" which corresponds with the options used to gzip
280 meta files for the main Debian archive.
281 These options may need to be modified if the md5sum of the file as
283 gzipped by debmirror does not match the md5sum listed in the
284 Release file (which will result in the gzipped file being
285 downloaded unnecessarily after diffs were successfully applied).
286 --state-cache-days=number
288 Save the state of the mirror in a cache file between runs. The
289 cache will expire after the specified number of days, at which time
290 a full check and cleanup of the mirror will be done. While the
291 cache is valid, debmirror will trust that the mirror is consistent
292 with this cache.
293 The cache is only used for files that have a unique name, i.e.
295 binary packages and source files. If a mirror update fails for any
296 reason, the cache will be invalidated and the next run will include
297 a full check.
298 Main advantage of using the state cache is that it avoids a large
300 amount of disk access while checking which files need to be
301 fetched. It may also reduce the time required for mirror updates.
302 --ignore-small-errors
304 Normally debmirror will report an error if any deb files or sources
305 fail to download and refuse to update the meta data to an
306 inconsistent mirror. Normally this is a good things as it indicates
307 something went wrong during download and should be retried. But
308 sometimes the upstream mirror actually is broken. Specifying
309 --ignore-small-errors causes debmirror to ignore missing or broken
310 deb and source files but still be pedantic about checking meta
311 files.
312 --allow-dist-rename
314 The directory name for a dist should be equal to its Codename and
315 not to a Suite. If the local mirror currently has directories named
316 after Suites, debmirror can rename them automatically. An existing
317 symlink codename -> suite will be removed, but debmirror will
318 automatically create a new symlink suite -> codename (immediately
319 after moving meta files in place). This conversion should only be
320 needed once.
321
323 Using regular expressions in options
324 Various options accept regular expressions that can be used to tune
325 what is included in the mirror. They can be any regular expression
326 valid in perl, which also means that extended syntax is standard. Make
327 sure to anchor regular expressions appropriately: this is not done by
328 debmirror.
329 The --include and --exclude options can be combined. This combination
331 for example will, if the --i18n option is used, exclude all Translation
332 files, except for the ones for Portuguese (pt) and Brazillian (pt_BR):
333 --exclude='/Translation-.*\.bz2$' --include='/Translation-pt.*\.bz2$'
335 Mirroring Debian Installer images
337 Debmirror will only mirror the "current" images that are on the remote
338 mirror. At least one of the options --di-dist or --di-arch must be
339 passed to enable mirroring of the images.
340 The special values "dists" resp. "arches" can be used to tell debmirror
342 to use the same dists and architectures for D-I images as for the
343 archive, but it is also possible to specify different values. If either
344 option is not set, it will default to the same values as for the
345 archive.
346 If you wish to create custom CD images using for example debian-cd, you
348 will probably also want add the option "--rsync-extra=doc,tools".
349 Limitations
351 There are no progress updates displayed for D-I images.
353 Archive size
355 The tables in the file /usr/share/doc/debmirror/archive_size give an
356 indication of the space needed to mirror the Debian archive. They are
357 particularly useful if you wish to set up a partial mirror. Only the
358 size of source and binary packages is included. You should allow for
359 around 1-4 GB of meta data (in ./dists/<dist>) per suite (depending in
360 your settings). Plus whatever space is needed for extra directories
361 (e.g. tools, doc) you wish to mirror.
362 The tables also show how much additional space is required if you add a
364 release on top of its predecessor. Note that the additional space
365 needed for testing and (to a lesser extend) unstable varies during the
366 development cycle of a release. The additional space needed for testing
367 is zero immediately after a stable release and grows from that time
368 onwards.
369 Note Debmirror keeps an extra copy of all meta data. This is necessary
371 to guarantee that the local mirror stays consistent while debmirror is
372 running.
373
375 Simply make a mirror in /srv/mirror/debian, using all defaults (or the
376 settings defined in debmirror.conf):
377 debmirror /srv/mirror/debian
379 Make a mirror of i386 and sparc binaries, main only, and include both
381 unstable and testing versions of Debian; download from
382 'ftp.kernel.org':
383 debmirror -a i386,sparc -d sid -d etch -s main --nosource \
385 -h ftp.nl.debian.org --progress $HOME/mirror/debian
386 Make a mirror using rsync (rsync server is 'ftp.debian.org::debian'),
388 excluding the section 'debug' and the package 'foo-doc':
389 debmirror -e rsync $HOME/mirror/debian --exclude='/foo-doc_' \
391 --exclude-deb-section='^debug$'
392
394 /etc/debmirror.conf
395 ~/.debmirror.conf
396 Debmirror will look for the presence of these files and load them
398 in the indicated order if they exist.
399 See the example in /usr/share/doc/debmirror/examples for syntax.
400 ~/.gnupg/trustedkeys.gpg
402 Debmirror uses gpgv to verify Release and Release.gpg using the
404 default keying ~/.gnupg/trustedkeys.gpg. This can be changed by
405 exporting GNUPGHOME resulting in $GNUPGHOME/trustedkeys.gpg being
406 used.
407 To add the right key to this keyring you can import it from the
409 debian keyring (in case of the debian archive) using:
410 gpg --keyring /usr/share/keyrings/debian-archive-keyring.gpg --export \
412 | gpg --no-default-keyring --keyring trustedkeys.gpg --import
413 or download the key from a keyserver:
415 gpg --no-default-keyring --keyring trustedkeys.gpg \
417 --keyserver keyring.debian.org --recv-keys <key ID>
418 The <key ID> can be found in the gpgv error message in debmirror:
420 gpgv: Signature made Tue Jan 23 09:07:53 2007 CET using DSA key ID 2D230C5F
421
423 This program is copyright 2001 by Joey Hess <joeyh@debian.org>, under
424 the terms of the GNU GPL (either version 2 of the licence or, at your
425 option, any later version), copyright 2001-2002 by Joerg Wendland
426 <joergland@debian.org>, copyright 2003-2007 by Goswin von Brederlow
427 <goswin-v-b@web.de> and copyright 2009 by Frans Pop <fjp@debian.org>.
428 The author disclaims any responsibility for any mangling of your
430 system, unexpected bandwidth usage bills, meltdown of the Debian mirror
431 network, etc, that this script may cause. See NO WARRANTY section of
432 GPL.
433
435 Current maintainer:
436 Frans Pop <fjp@debian.org>
437 Previous authors:
439 Joey Hess <joeyh@debian.org> (original author)
440 Joerg Wendland <joergland@debian.org>
441 Goswin von Brederlow <goswin-v-b@web.de>
442
444 Waste bandwith -- put a partial mirror on your laptop today!
445perl v5.12.1 2010-03-25 DEBMIRROR(1)