1MIRMON(1) User Contributed Perl Documentation MIRMON(1)
2
6 mirmon - monitor the state of mirrors
7
9 mirmon [-v] [-q] [-t timeout] [-c conf] [-get all⎪update⎪url url]
10
12 -v Be verbose ; mirmon normally only reports errors and changes in the
13 mirror list.
14 -q Be quiet.
16 -t timeout
18 Set the timeout ; the default is 300.
19 -get all ⎪ update ⎪ url <url>
21 With all, probe all sites. With update, probe a selection of the
22 sites ; see option "max_poll" below. With url, probe only the
23 given url, which must appear in the mirror-list.
24 -c name
26 Use config file name. The default list is
27 ./mirmon.conf $HOME/.mirmon.conf /etc/mirmon.conf
29
31 The program is intended to be run by cron every hour.
32 42 * * * * perl /path/to/mirmon -get update
34 It quietly probes a subset of the sites in a given list, writes the
36 results in the 'state' file and generates a web page with the results.
37 The subset contains the sites that are new, bad and/or not probed for a
38 specified time.
39 When no 'get' option is specified, the program just generates a new web
41 page from the last known state.
42 The program checks the mirrors by running a (user specified) program on
44 a pipe. A (user specified) number of probes is run in parallel using
45 nonblocking IO. When something can be read from the pipe, it switches
46 the pipe to blocking IO and reads one line from the pipe. Then it
47 flushes and closes the pipe. No attempt is made to kill the probe.
48 The probe should return something that looks like
50 1043625600 ...
52 that is, a line of text starting with a timestamp. The exit status of
54 the probe is ignored.
55
57 location
58 A config file can be specified with the -c option. If -c is not used,
60 the program looks for a config file in
61 * ./mirmon.conf
63 * $HOME/.mirmon.conf
64 * /etc/mirmon.conf
65 syntax
67 A config file looks like this :
69 +--------------------------------------------------
71 ⎪# lines that start with '#' are comment
72 ⎪# blank lines are ignored too
73 ⎪# tabs are replaced by a space
74 ⎪
75 ⎪# the config entries are 'key' and 'value' pairs
76 ⎪# a 'key' begins in column 1
77 ⎪# the 'value' is the rest of the line
78 ⎪somekey A_val B_val ...
79 ⎪otherkey X_val Y_val ...
80 ⎪
81 ⎪# indented lines are glued
82 ⎪# the next three lines mean 'somekey part1 part2 part3'
83 ⎪somekey part1
84 ⎪ part2
85 ⎪ part3
86 ⎪
87 ⎪# lines starting with a '+' are concatenated
88 ⎪# the next three lines mean 'somekey part1part2part3'
89 ⎪somekey part1
90 ⎪+ part2
91 ⎪+ part3
92 ⎪
93 ⎪# lines starting with a '.' are glued too
94 ⎪# don't use a '.' on a line by itself
95 ⎪# 'somekey' gets the value "part1\n part2\n part3"
96 ⎪somekey part1
97 ⎪. part2
98 ⎪. part3
99 +--------------------------------------------------
100 required entries
102 project_name name
104 Specify a short plaintext name for the project.
105 project_name Apache
107 project_name CTAN
108 project_url url
110 Specify an url pointing to the 'home' of the project.
111 project_url http://www.apache.org/
113 mirror_list file-name
115 Specify the file containing the mirrors to probe.
116 mirror_list /path/to/mirror-list
118 If your mirror list is generated by a program, use
120 mirror_list /path/to/program arg1 ... ⎪
122 Two formats are supported :
124 * plain : lines like
126 us http://www.tux.org/ [email] ...
127 nl http://apache.cs.uu.nl/dist/ [email] ...
128 nl rsync://archive.cs.uu.nl/apache-dist/ [email] ...
129 * apache : lines like those in the apache mirrors.list
131 ftp us ftp://ftp.tux.org/pub/net/apache/dist/ user@tux.org ...
132 http nl http://apache.cs.uu.nl/dist/ user@cs.uu.nl ...
133 Note that in style 'plain' the third item is reserved for an
135 optional email address : the site's contact address.
136 Specify the required format with option "list_style" (see below).
138 The default style is 'plain'.
139 web_page file-name
141 Specify where the html report page is written.
142 icons directory-name
144 Specify the directory where the icons can be found, relative to the
145 web_page, or relative to the DOCUMENTROOT of the web server.
146 If/when the web_page lives in directory ".../mirmon/" and the icons
148 live in directory ".../mirmon/icons/", specify
149 icons icons
151 If/when the icons live in "/path/to/DOCUMENTROOT/icons/mirmon/",
153 specify
154 icons /icons/mirmon
156 probe program + arguments
158 Specify the program+args to probe the mirrors. Example:
159 probe /usr/bin/wget -q -O - -T %TIMEOUT% -t 1 %URL%TIME.txt
161 Before the program is started, %TIMEOUT% and %URL% are substituted
163 with the proper timeout and url values.
164 Here it is assumed that each hour the root server writes a time‐
166 stamp in /path/to/archive/TIME.txt, for instance with a crontab
167 entry like
168 42 * * * * perl -e 'print time, "\n"' > /path/to/archive/TIME.txt
170 Mirmon reads one line of output from the probe and interprets the
172 first word on that line as a timestamp ; for example :
173 1043625600
175 1043625600 Mon Jan 27 00:00:00 2003
176 1043625600 www.apache.org Mon Jan 27 00:00:00 2003
177 Mirmon is distributed with a program "probe" that handles ftp, http
179 and rsync urls.
180 state file-name
182 Specify where the file containing the state is written.
183 The program reads this file on startup and writes the file when
185 mirrors are probed (-get is specified).
186 countries file-name
188 Specify the file containing the country codes; The file should con‐
189 tain lines like
190 us - United States
192 nl - Netherlands
193 The mirmon package contains a recent ISO list.
195 Fake domains like Backup, Master are allowed, and are listed first
197 in the report ; lowercase-first fake domains (like backup) are
198 listed last.
199 optional entries
201 max_probes number
203 Optionally specify the number of parallel probes (default 25).
204 timeout seconds
206 Optionally specify the timeout for the probes (default 300).
207 After the last probe is started, the program waits for <timeout> +
209 10 seconds, cleans up and exits.
210 project_logo logo
212 Optionally specify (the SRC of the IMG of) a logo to be placed top
213 right on the page.
214 project_logo /icons/apache.gif
216 project_logo http://www.apache.org/icons/...
217 htm_head html
219 Optionally specify some HTML to be placed before </HEAD>.
220 htm_head
222 <link REL=StyleSheet HREF="/style.css" TYPE="text/css">
223 htm_top html
225 Optionally specify some HTML to be placed near the top of the page.
226 htm_top testing 1, 2, 3
228 htm_foot html
230 Optionally specify HTML to be placed near the bottom of the page.
231 htm_foot
233 <HR>
234 <A HREF="..."><IMG SRC="..." BORDER=0></A>
235 <HR>
236 put_histo top⎪bottom⎪nowhere
238 Optionally specify where the age histogram must be placed. The
239 default is 'top'.
240 min_poll time-spec
242 For 'min_poll' see next item. A time-spec is a number followed by a
243 unit 's' (seconds), or 'm' (minutes), or 'h' (hours), or 'd'
244 (days). For example '3d' (three days) or '36h' (36 hours).
245 max_poll time-spec
247 Optionally specify the maximum probe interval. When the program is
248 called with option '-get update', all sites are probed which are :
249 * new
251 the site appears in the list, but there is no known state
252 * bad
254 the last probe of the site was unsuccessful
255 * old
257 the last probe was more than 'max_poll' ago.
258 Sites are not probed if the last probe was less than 'min_poll'
260 ago. So, if you specify
261 min_poll 4h
263 max_poll 12h
264 the 'reachable' sites are probed twice daily and the 'unreachable'
266 sites are probed at most six times a day.
267 The default 'min_poll' is '1h' (1 hour). The default 'max_poll' is
269 '4h' (4 hours).
270 min_sync time-spec
272 Optionally specify how often the mirrors are required to make an
273 update.
274 The default 'min_sync' is '1d' (1 day).
276 max_sync time-spec
278 Optionally specify the maximum allowable sync interval.
279 Sites exceeding the limit will be considered 'old'. The default
281 'max_sync' is '2d' (2 days).
282 always_get region ...
284 Optionally specify a list of regions that must be probed always.
285 always_get Master Tier1
287 This is intended for fake regions like Master etc.
289 no_randomize
291 Mirmon tries to balance the probe load over the hourly mirmon runs.
292 If the current run has a below average number of mirrors to probe,
293 mirmon probes a few extra, randomly chosen mirrors, picked from the
294 runs that have the highest load.
295 If you don't want this behaviour, use no_randomize.
297 no_add_slash
299 If the url part of a line in the mirror_list doesn't end in a slash
300 ('/'), mirmon adds a slash and issues a warning unless it is in
301 quiet mode.
302 If you don't want this behaviour, use no_add_slash.
304 list_style plain⎪apache
306 Optionally specify the format ('plain' or 'apache') of the mir‐
307 ror-list.
308 See the description of 'mirror_list' above. The default list_style
310 is 'plain'.
311 site_url site url
313 Optionally specify a substitute url for a site.
314 When access to a site is restricted (in Australia, for instance),
316 another (sometimes secret) url can be used to probe the site. The
317 <site> of an url is the part between '://' and the first '/'.
318 env key value
320 Optionally specify an environment variable.
321 include file-name
323 Optionally specify a file to include.
324 The specified file is processed 'in situ'. After the specified file
326 is read and processed, config processing is resumed in the file
327 where the "include" was encountered. The include depth is unlim‐
328 ited. However, it is a fatal error to include a file twice under
329 the same name.
330 show
332 When the config processor encounters the 'show' command, it dumps
333 the content of the current config to standout, if option "-v" is
334 specified. This is intented for debugging.
335 exit
337 When the config processor encounters the 'exit' command, it termi‐
338 nates the program. This is intented for debugging.
339
341 The state file consists of lines; one line per site. Each line con‐
342 sists of white space separated fields. The seven fields are :
343 * field 1 : url
345 The url as given in the mirror list.
346 * field 2 : age
348 The mirror's timestamp found by the last successful probe, or
349 'undef' if no probe was ever successful.
350 * field 3 : status last probe
352 The status of the last probe, or 'undef' if the mirror was never
353 probed.
354 * field 4 : time last successful probe
356 The timestamp of the last successful probe or 'undef' if the mirror
357 was never successfully probed.
358 * field 5 : probe history
360 The probe history is a list of 's' (for success) and 'f' (for fail‐
361 ure) characters indicating the result of the probe. New results are
362 appended whenever the mirror is probed.
363 * field 6 : state history
365 The state history consists of a timestamp, a '-' char, and a list
366 of chars indicating a past status: 's' (fresh), 'b' (oldish), 'f'
367 (old), 'z' (bad) or 'x' (skip). The timestamp indicates when the
368 state history was last updated. The current status of the mirror
369 is determined by the mirror's age and a few configuration parame‐
370 ters (min_sync, max_sync, max_poll). The state history is updated
371 when the mirror is probed. If the last update of the history was
372 less than 24 hours ago, the last status is replaced by the current
373 status. If the last update of the history was more than 24 hours
374 ago, the current status is appended to the history. One or more
375 'skip's is inserted, if the timestamp is two or more days old (when
376 mirmon hasn't run for more than two days).
377 * field 7 : last probe
379 The timestamp of the last probe, or 'undef' if the mirror was never
380 probed.
381
383 general
384 * Note: The (empty) state file must exist before mirmon runs.
386 * The mirmon repository is here :
387 https://svn.science.uu.nl/repos/project.mirmon/trunk/
388 * The mirmon tarball is here :
390 http://www.staff.science.uu.nl/~penni101/mirmon/mirmon.tar.gz
391 installation suggestions
393 To install and configure mirmon, take the following steps :
395 * First, make the webdir :
397 cd DOCUMENTROOT
398 mkdir mirmon
399 For DOCUMENTROOT, substitute the full pathname of the document root
401 of your webserver.
402 * Check out the mirmon repository :
404 cd /usr/local/src
405 svn checkout REPO mirmon
406 where
408 REPO = https://svn.science.uu.nl/repos/project.mirmon/trunk/
410 or download the package and unpack it.
412 * Chdir to directory mirmon :
414 cd mirmon
415 * Create the (empty) state file :
417 touch state.txt
418 * Install the icons in the webdir :
420 mkdir DOCUMENTROOT/mirmon/icons
421 cp icons/* DOCUMENTROOT/mirmon/icons
422 * Create a mirror list "mirror_list" ;
424 Use your favorite editor, or genererate the list from an existing
425 database.
426 nl http://archive.cs.uu.nl/your-project/ contact@cs.uu.nl
428 uk http://mirrors.this.org/your-project/ mirrors@this.org
429 us http://mirrors.that.org/your-project/ mirrors@that.org
430 The email addresses are optional.
432 * Create a mirmon config file "mirmon.conf" with your favorite editor.
434 # lines must start in the first column ; no leading white space
435 project_name ....
436 project_url ....
437 mirror_list mirror_list
438 state state.txt
439 countries countries.list
440 web_page DOCUMENTROOT/mirmon/index.html
441 icons /mirmon/icons
442 probe /usr/bin/wget -q -O - -T %TIMEOUT% -t 1 %URL%TIME.txt
443 This assumes the project's timestamp is in file "TIME.txt".
445 * If you have rsync urls, change the probe line to :
447 probe perl /usr/local/src/mirmon/probe -t %TIMEOUT% %URL%TIME.txt
448 * Run mirmon :
450 perl mirmon -v -get all
451 The mirmon report should now be in 'DOCUMENTROOT/mirmon/index.html'
453 http://www.your.project.org/mirmon/
455 * If/when, at a later date, you want to upgrade mirmon :
457 cd /usr/local/src/mirmon
458 svn status -u
459 svn up
460
462 mirmon.pm(3)
463
466 (c) 2003-2016 Henk P. Penning
467 Faculty of Science, Utrecht University
468 http://www.staff.science.uu.nl/~penni101/ -- penning@uu.nl
469 mirmon-2.11 - Sat Jul 23 09:12:31 2016 ; henkp
470perl v5.8.8 2016-07-23 MIRMON(1)