1MIMEDEFANG-MULTIPLEXOR(8) System Manager's Manual MIMEDEFANG-MULTIPLEXOR(8)
2
6 mimedefang-multiplexor - Process pool controller for mail filters.
7
9 mimedefang-multiplexor [options]
10
13 mimedefang-multiplexor manages a pool of Perl processes for scanning e-
14 mail. It is designed to work in conjunction with mimedefang(8) and
15 mimedefang.pl(8).
16 mimedefang-multiplexor opens a UNIX-domain socket and listens for
18 requests for work from mimedefang. As requests come in, mimedefang-
19 multiplexor creates Perl processes as needed to scan mail. The Perl
20 processes are not killed when scanning is completed, but continue to
21 run in a loop. Perl processes are re-used for subsequent e-mail mes‐
22 sages. This eliminates the large overhead of starting a new Perl
23 process for each incoming message.
24 To avoid memory leaks, the Perl processes are killed after they have
26 handled some number of scans.
27
30 -U user
31 Runs the multiplexor as user rather than root. This option is
32 mandatory, and must match the -U option supplied to mimedefang.
33 -m minSlaves
36 The minimum number of Perl processes to keep running at all
37 times. The default is zero.
38 -x maxSlaves
41 The maximum number of Perl processes to run simultaneously. If
42 a request comes in and all processes are busy, a temporary fail‐
43 ure is signalled to the SMTP peer. The default is 2.
44 -r maxRequests
47 The maximum number of requests a given process handles before it
48 is killed and a replacement started. The default is 500.
49 -i idleTime
52 The idle time in seconds after which to kill of excess Perl pro‐
53 cesses. That is, if the process is idle for longer than this
54 time, and there are more than minSlaves running, the process is
55 killed. Note that this is implemented as a timer which ticks
56 every idleTime seconds; therefore, processes may be idle for up
57 to twice this time before they are killed. The default for
58 idleTime is 300 seconds.
59 -b busyTime
62 The longest a Perl process is allowed to spend scanning an e-
63 mail before it is declared hung up and killed. The default is
64 120 seconds.
65 -Z This option specifies that the multiplexor should accept and
68 process "status updates" from busy slaves. Note that this con‐
69 sumes one extra file descriptor per slave, plus a small amount
70 of CPU time per status update.
71 -c cmdTime
74 The timeout for communication between mimedefang-multiplexor and
75 mimedefang, or between mimedefang-multiplexor and a Perl scan‐
76 ning process. The default is 10 seconds. This timeout should
77 be kept quite short.
78 -w waitTime
81 When mimedefang-multiplexor starts the initial slaves, or needs
82 to bring the number of running slaves up to the number defined
83 by the -m option, it does not start all the slaves at once,
84 because this could overload your server. Instead, it starts one
85 slave every waitTime seconds. The default value for waitTime is
86 3.
87 -W waitTime
90 If you use this option, mimedefang-multiplexor will never acti‐
91 vate a slave until waitTime seconds have elapsed since the last
92 slave activation. This could result in mail being tempfailed if
93 slave activations do not keep pace with incoming mail. However,
94 it may be preferable to tempfail mail rather than allow the load
95 on your server to spike up too quickly. The default value for
96 this option is 0, meaning that mimedefang-multiplexor will start
97 slaves as quickly as necessary to keep up with incoming mail.
98 -z spooldir
101 Set the spool directory to spooldir. If this option is omitted,
102 the spool directory defaults to /var/spool/MIMEDefang.
103 -s pathName
106 The UNIX-domain socket on which mimedefang-multiplexor listens
107 for requests. This should be specified as an absolute pathname.
108 If this option is not supplied, it defaults to mimedefang-multi‐
109 plexor.sock under the spool directory.
110 -a socket
113 A socket for listening for requests. This is similar to the -s
114 socket, except that a restricted set of requests are processed.
115 On this socket, the multiplexor will only process requests ask‐
116 ing for status; it will not accept any commands to do scanning
117 or that would consume a slave. See the SOCKET SPECIFICATION
118 section for the format of socket.
119 -p fileName
122 Causes mimedefang-multiplexor to write its process-ID (after
123 becoming a daemon) to the specified file.
124 -f filter_path
127 Normally, mimedefang-multiplexor executes a Perl filter script
128 called mimedefang.pl to scan the e-mail. However, you can have
129 it execute any program you like by specifying the full path to
130 the program with the -f option. This program must obey the pro‐
131 tocol documented in mimedefang-protocol(7); see that manual page
132 for details.
133 Note that the -f option does not specify the "filter" to use
135 with mimedefang.pl; instead, it specifies the program for
136 mimedefang-multiplexor to execute. You almost certainly should
137 not use this option unless you wish to replace mimedefang.pl
138 with your own program.
139 -F rules_path
142 Specifies the path to the filter rules. By default,
143 /etc/mail/mimedefang-filter is used. If you use the -F option,
144 its value is passed to the underlying Perl filter program using
145 -f.
146 -l Log certain events, including the output of the Perl slaves'
149 standard-error, using syslog. Normally, the multiplexor does
150 not log much information.
151 -d Write debugging information about event-handling code in
154 /var/log/mimedefang-event-debug.log. This is only of use to
155 people debugging mimedefang-multiplexor.
156 -R kbytes
159 Limits the resident-set size of the slave filter processes to
160 kbytes kilobytes. This limit is not supported on all operating
161 systems; it is known to work on Linux.
162 -M kbytes
165 Limits the total memory space of slave filter processes to
166 kbytes kilobytes. This limit is supported on all operating sys‐
167 tems which support the setrlimit(2) system call. This should
168 include most modern UNIX systems.
169 We recommend that you monitor your slave filter processes and
171 get a feel for how much memory they use. You should then limit
172 the memory to two or three times the worst-case that you have
173 observed. This can help mitigate denial-of-service attacks
174 which use complicated MIME messages to force mimedefang.pl to
175 consume lots of memory.
176 -h Print usage information and exit.
179 -t filename
182 Log statistical information to filename. See the section STA‐
183 TISTICS for more information.
184 -T Log statistical information using syslog(2). You may use any -t
187 and -T together, in which case statistical information is logged
188 in a file and using syslog.
189 -u Flush the statistics file after every write. Normally, mimede‐
192 fang-multiplexor does not flush the file; this is the best
193 choice for minimizing disk I/O on a busy mail server. However,
194 if you wish to watch statistics entries in real-time, you should
195 enable flushing.
196 -D Do not fork into the background and become a daemon. Instead,
199 stay in the foreground. Useful mainly for debugging or if you
200 have a supervisory process managing mimedefang-multiplexor.
201 -q queue_size
204 Normally, if all slaves are busy and mimedefang-multiplexor
205 receives another request, it fails it with the error "No free
206 slaves." However, if you use the -q option, then up to
207 queue_size requests will be queued. As soon as a slave becomes
208 free, the queued requests will be handed off in FIFO order. If
209 the queue is full and another request comes in, then the request
210 is failed with "No free slaves".
211 -Q queue_timeout
214 Queued requests should not stay on the queue indefinitely. If a
215 queued request cannot be processed within queue_timeout (default
216 30) seconds of being placed on the queue, it is failed with a
217 "Queued request timed out" message. See the section "QUEUEING
218 REQUESTS" for more discussion.
219 -O sock
222 Listen on a notification socket for connections from listeners.
223 mimedefang-multiplexor can inform external programs of state
224 changes by sending messages over a notification socket. The
225 external programs connect to this socket and then listen for
226 notifications. See the section SOCKET SPECIFICATION for the
227 format of sock.
228 See the mimedefang-notify(7) man page for details of the notifi‐
230 cation protocol.
231 -N map_sock
234 Listen on a map socket for Sendmail SOCKETMAP connections. As
235 of Sendmail 8.13, you can define a Sendmail map type that talks
236 to a daemon over a socket. mimedefang-multiplexor implements
237 that protocol; consult the mimedefang-filter(5) man page for
238 detils (see the SOCKET MAPS section).
239 See the section SOCKET SPECIFICATION for the format of map_sock.
241 -I backlog
246 When mimedefang-multiplexor creates a listening socket, it cal‐
247 culates the "backlog" argument to listen(2) based on the maximum
248 number of slaves. However, you can explicitly set this backlog
249 with the -I option. Setting the backlog to a high value (around
250 30-50) may help on a very busy server. If you see mail log mes‐
251 sages saying "MXCommand: socket: Connection refused" during busy
252 periods, then that's an indication you need a higher listen
253 backlog.
254 -L interval
257 Log the slave status every interval seconds. This logs a line
258 using syslog; the line looks like this:
259 Slave status: Stopped=s Idle=i Busy=b Killed=k Queued=q Msgs=m Activations=a
262 Here, "Stopped" is the number of non-running slaves, "Idle" is
264 the number of idle slaves, "Busy" is the number of busy slaves,
265 "Killed" is the number of killed slaves yet to be reaped,
266 "Queued" is the number of queued requests, "Msgs" is the total
267 number of messages processed since the multiplexor began run‐
268 ning, and "Activations" is the number of times a Perl process
269 has been started since the multiplexor began running.
270 If you supply an interval of 0 (which is the default), no peri‐
272 odic status logging is performed. If you supply an interval of
273 less than 5 seconds, it is silently reset to 5 seconds.
274 -S facility
277 Specifies the syslog facility for log messages. The default is
278 mail. See openlog(3) for a list of valid facilities. You can
279 use either the short name ("mail") or long name ("LOG_MAIL") for
280 the facility name.
281 -E Specifies that the multiplexor should create an embedded Perl
284 interpreter. This can improve performance dramatically. But
285 see the section "EMBEDDING PERL" for more information.
286 -X n Specifies that the multiplexor should initiate a "tick" request
289 every n seconds. This causes your filter_tick function (if
290 defined) to be called. Note that you have no control over which
291 slave executes filter_tick. If all slaves are busy when a tick
292 occurs, that tick request is skipped and a warning message is
293 logged.
294 -P n Specifies that the multiplexor should run n tick requests in
297 parallel. Each tick is run as often as specified with the -X
298 argument. (If you omit the -P option, then the multiplexor
299 behaves as if -P 1 had been specified.)
300 If you run parallel ticks, each tick is assigned an integer
302 identifying its "type". The type ranges from 0 to n-1. While
303 there may be as many as n tick requests running at a time, only
304 one tick of each type will be active at any time.
305 -Y label
308 Sets the tag used in the multiplexor's syslog messages to label
309 instead of mimedefang-multiplexor.
310 -y n Limits the maximum number of concurrent recipok checks to n on a
313 per-domain basis. The value of n can range from 0 (in which
314 case no limit is applied) to maxSlaves, where maxSlaves is the
315 argument to the -x option. If n is outside that range, it is
316 ignored (and no limit is applied.)
317 The recipok command ultimately invokes the filter_recipient
320 function in your filter. If you are doing recipient verifica‐
321 tion against servers that may be slow or unreliable, you can use
322 the -y option to limit the number of concurrent recipient veri‐
323 fications per domain. That way, if one domain's server becomes
324 very slow, it won't consume all available slaves for recipient
325 verification. Instead, its RCPT commands will be tempfailed and
326 there will be slaves available to handle RCPT commands for other
327 domains.
328
331 The -a, -N and -O options take a socket as an argument. This socket
332 can be specified as:
333 /path/to/socket
336 A UNIX-domain socket
337 inet:portnum
340 A TCP socket bound to port portnum, but which accepts connec‐
341 tions only from the IPv4 loopback address (127.0.0.1).
342 inet_any:portnum
345 A TCP socket bound to port portnum which will accept connections
346 from any address. Use inet_any with caution!
347 inet6:portnum
350 A TCP socket bound to port portnum listening on the IPv6 loop‐
351 back address.
352 inet6_any:portnum
355 A TCP socket bound to port portnum listening on the IPv6 wild‐
356 card address.
357
360 Normally, if all slaves are busy, any additional requests are failed
361 immediately. However, the -q and -Q options allow you to queue
362 requests for a short amount of time. This facility is intended to
363 gracefully handle a temporary overload; most of the time, your queue
364 should be empty.
365 Because mimedefang checks the number of free slaves when a connection
367 is opened and fails the connection if there are no free slaves, the
368 intent of the queue is to allow SMTP transactions that are already
369 underway to continue if there is a slight overload. Any new connec‐
370 tions will be failed if all slaves are busy, but existing connections
371 are allowed to continue. Queuing requests may improve throughput on
372 extremely busy servers.
373 Note that if you supply the -q option to mimedefang, then even new con‐
375 nections are allowed to queue. This may improve throughput by keeping
376 the slave utilization higher.
377 The -R option to mimedefang can be used to reserve a specified number
379 of slaves for connections from the loopback address. Using the -R
380 option has the side-effect of permitting new connections from the loop‐
381 back address to queue.
382
385 Normally, when mimedefang-multiplexor activates a slave, it forks and
386 execs mimedefang.pl. However, if the multiplexor was compiled with
387 embedded Perl support, and you supply the -E command-line option, the
388 multiplexor works like this:
389 1 It creates an embedded Perl interpreter, and sources mimede‐
392 fang.pl with a special command-line argument telling it to read
393 the filter, but not to enter the main loop.
394 2 Each time a slave is activated, the multiplexor calls fork() and
397 runs the mimedefang.pl main loop. This invokes filter_initial‐
398 ize and then runs the main loop.
399 On some platforms (for example, Red Hat Linux 7.3 with Perl 5.6.1), it
402 is not safe to destroy and recreate a Perl interpreter without causing
403 a memory leak. On those platforms, if you attempt to reread the filter
404 file (by sending the multiplexor a HUP signal or reread command), the
405 filter will not be re-read, and a message will be logged to syslog. On
406 those platforms, you must kill and restart MIMEDefang if you change the
407 filter file.
408 On most platforms, however, a filter reread is accomplished by destroy‐
411 ing and re-creating the embedded interpreter, re-sourcing mimedefang.pl
412 and killing slaves as soon as they are idle.
413 WARNING: If you use the embedded Perl interpreter, the Perl variable
416 "$$" will not be updated with the process ID of each slave. Instead,
417 it will reflect the process ID of the master multiplexor. This is a
418 limitation of embedded Perl; we have no plans to fix it.
419
422 With the -t option, mimedefang-multiplexor logs certain events to a
423 file. This file can be post-processed to gather statistics about the
424 multiplexor. You can use it to tune the number of slaves you run,
425 adjust timeouts, and so on.
426 Each line of the file looks like this:
429 YYYY/MM/DD:HH:MM:SS timestamp event key=val key=val...
431 Here, YYYY/MM/DD:HH:MM:SS is the local time of day. Timestamp is the
433 number of seconds since January 1, 1970. Event is the name of an
434 event. The valid events are:
435 StartSlave
438 A slave process has been started.
439 KillSlave
442 A slave process has been killed.
443 ReapSlave
446 A dead slave process has been reaped. It is possible to have a
447 ReapSlave event without a previous KillSlave event if the slave
448 process terminated abnormally.
449 StartFilter
452 A slave process has begun filtering an e-mail message.
453 EndFilter
456 A slave process has finished filtering an e-mail message.
457 The possible keys in the key=value pairs are:
460 slave=n
463 The slave involved in the event. Every slave is identified by a
464 small integer.
465 nslaves=n
468 The total number of running slaves immediately after the event
469 happened.
470 nbusy=n
473 The number of busy slaves (slaves which are processing an e-mail
474 message) immediately after the event happened.
475 reason="string"
478 The reason for a StartSlave or KillSlave event. (Present only
479 for these events.)
480 numRequests=n
483 The number of e-mails processed by the slave. Present only for
484 an EndFilter event.
485 If you send the mimedefang-multiplexor process a SIGHUP signal (kill -1
488 pid), it closes and reopens the statistics file. This is useful during
489 log file rotation.
490 If you send the mimedefang-multiplexor process a SIGINT signal (kill
493 -INT pid), it terminates all active-but-idle slaves. Also, any active-
494 and-busy slaves terminate as soon as they finish filtering the current
495 message. This is useful to force a reread of the filter rules file
496 without stopping and restarting Sendmail.
497 If you send the mimedefang-multiplexor process a SIGTERM signal (kill
500 pid), it terminates all slaves and exits immediately.
501
504 mimedefang-mulitplexor was written by David F. Skoll <dfs@roaringpen‐
505 guin.com>. The mimedefang home page is http://www.mimedefang.org/.
506
509 mimedefang.pl(8), mimedefang-filter(5), mimedefang(8), mimedefang-pro‐
510 tocol(7)
5114th Berkeley Distribution 8 February 2005 MIMEDEFANG-MULTIPLEXOR(8)