1MIMEDEFANG-MULTIPLEXOR(8)   System Manager's Manual  MIMEDEFANG-MULTIPLEXOR(8)
2
3
4

NAME

6       mimedefang-multiplexor - Process pool controller for mail filters.
7

SYNOPSIS

9       mimedefang-multiplexor [options]
10
11

DESCRIPTION

13       mimedefang-multiplexor manages a pool of Perl processes for scanning e-
14       mail.  It is designed to work in  conjunction  with  mimedefang(8)  and
15       mimedefang.pl(8).
16
17       mimedefang-multiplexor  opens  a  UNIX-domain  socket  and  listens for
18       requests for work from mimedefang.  As requests  come  in,  mimedefang-
19       multiplexor  creates  Perl  processes as needed to scan mail.  The Perl
20       processes are not killed when scanning is completed,  but  continue  to
21       run  in  a loop.  Perl processes are re-used for subsequent e-mail mes‐
22       sages.  This eliminates the large  overhead  of  starting  a  new  Perl
23       process for each incoming message.
24
25       To  avoid  memory  leaks, the Perl processes are killed after they have
26       handled some number of scans.
27
28

OPTIONS

30       -U user
31              Runs the multiplexor as user rather than root.  This  option  is
32              mandatory, and must match the -U option supplied to mimedefang.
33
34
35       -m minSlaves
36              The  minimum  number  of  Perl  processes to keep running at all
37              times.  The default is zero.
38
39
40       -x maxSlaves
41              The maximum number of Perl processes to run simultaneously.   If
42              a request comes in and all processes are busy, a temporary fail‐
43              ure is signalled to the SMTP peer.  The default is 2.
44
45
46       -r maxRequests
47              The maximum number of requests a given process handles before it
48              is killed and a replacement started.  The default is 500.
49
50
51       -i idleTime
52              The idle time in seconds after which to kill of excess Perl pro‐
53              cesses.  That is, if the process is idle for  longer  than  this
54              time,  and there are more than minSlaves running, the process is
55              killed.  Note that this is implemented as a  timer  which  ticks
56              every  idleTime seconds; therefore, processes may be idle for up
57              to twice this time before they  are  killed.   The  default  for
58              idleTime is 300 seconds.
59
60
61       -b busyTime
62              The  longest  a  Perl process is allowed to spend scanning an e-
63              mail before it is declared hung up and killed.  The  default  is
64              120 seconds.
65
66
67       -Z     This  option  specifies  that  the multiplexor should accept and
68              process "status updates" from busy slaves.  Note that this  con‐
69              sumes  one  extra file descriptor per slave, plus a small amount
70              of CPU time per status update.
71
72
73       -c cmdTime
74              The timeout for communication between mimedefang-multiplexor and
75              mimedefang,  or  between mimedefang-multiplexor and a Perl scan‐
76              ning process.  The default is 10 seconds.  This  timeout  should
77              be kept quite short.
78
79
80       -w waitTime
81              When  mimedefang-multiplexor starts the initial slaves, or needs
82              to bring the number of running slaves up to the  number  defined
83              by  the  -m  option,  it  does not start all the slaves at once,
84              because this could overload your server.  Instead, it starts one
85              slave every waitTime seconds.  The default value for waitTime is
86              3.
87
88
89       -W waitTime
90              If you use this option, mimedefang-multiplexor will never  acti‐
91              vate  a slave until waitTime seconds have elapsed since the last
92              slave activation.  This could result in mail being tempfailed if
93              slave activations do not keep pace with incoming mail.  However,
94              it may be preferable to tempfail mail rather than allow the load
95              on  your  server to spike up too quickly.  The default value for
96              this option is 0, meaning that mimedefang-multiplexor will start
97              slaves as quickly as necessary to keep up with incoming mail.
98
99
100       -z spooldir
101              Set the spool directory to spooldir.  If this option is omitted,
102              the spool directory defaults to /var/spool/MIMEDefang.
103
104
105       -s pathName
106              The UNIX-domain socket on which  mimedefang-multiplexor  listens
107              for requests.  This should be specified as an absolute pathname.
108              If this option is not supplied, it defaults to mimedefang-multi‐
109              plexor.sock under the spool directory.
110
111
112       -a socket
113              A  socket for listening for requests.  This is similar to the -s
114              socket, except that a restricted set of requests are  processed.
115              On  this socket, the multiplexor will only process requests ask‐
116              ing for status; it will not accept any commands to  do  scanning
117              or  that  would  consume  a slave.  See the SOCKET SPECIFICATION
118              section for the format of socket.
119
120
121       -p fileName
122              Causes mimedefang-multiplexor to  write  its  process-ID  (after
123              becoming a daemon) to the specified file.
124
125
126       -f filter_path
127              Normally,  mimedefang-multiplexor  executes a Perl filter script
128              called mimedefang.pl to scan the e-mail.  However, you can  have
129              it  execute  any program you like by specifying the full path to
130              the program with the -f option.  This program must obey the pro‐
131              tocol documented in mimedefang-protocol(7); see that manual page
132              for details.
133
134              Note that the -f option does not specify  the  "filter"  to  use
135              with  mimedefang.pl;  instead,  it  specifies  the  program  for
136              mimedefang-multiplexor to execute.  You almost certainly  should
137              not  use  this  option  unless you wish to replace mimedefang.pl
138              with your own program.
139
140
141       -F rules_path
142              Specifies  the  path  to  the   filter   rules.    By   default,
143              /etc/mail/mimedefang-filter  is used.  If you use the -F option,
144              its value is passed to the underlying Perl filter program  using
145              -f.
146
147
148       -l     Log  certain  events,  including  the output of the Perl slaves'
149              standard-error, using syslog.  Normally,  the  multiplexor  does
150              not log much information.
151
152
153       -d     Write   debugging   information  about  event-handling  code  in
154              /var/log/mimedefang-event-debug.log.  This is  only  of  use  to
155              people debugging mimedefang-multiplexor.
156
157
158       -R kbytes
159              Limits  the  resident-set  size of the slave filter processes to
160              kbytes kilobytes.  This limit is not supported on all  operating
161              systems; it is known to work on Linux.
162
163
164       -M kbytes
165              Limits  the  total  memory  space  of  slave filter processes to
166              kbytes kilobytes.  This limit is supported on all operating sys‐
167              tems  which  support  the setrlimit(2) system call.  This should
168              include most modern UNIX systems.
169
170              We recommend that you monitor your slave  filter  processes  and
171              get  a feel for how much memory they use.  You should then limit
172              the memory to two or three times the worst-case  that  you  have
173              observed.   This  can  help  mitigate  denial-of-service attacks
174              which use complicated MIME messages to  force  mimedefang.pl  to
175              consume lots of memory.
176
177
178       -h     Print usage information and exit.
179
180
181       -t filename
182              Log  statistical  information to filename.  See the section STA‐
183              TISTICS for more information.
184
185
186       -T     Log statistical information using syslog(2).  You may use any -t
187              and -T together, in which case statistical information is logged
188              in a file and using syslog.
189
190
191       -u     Flush the statistics file after every write.  Normally,  mimede‐
192              fang-multiplexor  does  not  flush  the  file;  this is the best
193              choice for minimizing disk I/O on a busy mail server.   However,
194              if you wish to watch statistics entries in real-time, you should
195              enable flushing.
196
197
198       -D     Do not fork into the background and become a  daemon.   Instead,
199              stay  in  the foreground.  Useful mainly for debugging or if you
200              have a supervisory process managing mimedefang-multiplexor.
201
202
203       -q queue_size
204              Normally, if all  slaves  are  busy  and  mimedefang-multiplexor
205              receives  another  request,  it fails it with the error "No free
206              slaves."  However,  if  you  use  the  -q  option,  then  up  to
207              queue_size  requests will be queued.  As soon as a slave becomes
208              free, the queued requests will be handed off in FIFO order.   If
209              the queue is full and another request comes in, then the request
210              is failed with "No free slaves".
211
212
213       -Q queue_timeout
214              Queued requests should not stay on the queue indefinitely.  If a
215              queued request cannot be processed within queue_timeout (default
216              30) seconds of being placed on the queue, it is  failed  with  a
217              "Queued  request  timed out" message.  See the section "QUEUEING
218              REQUESTS" for more discussion.
219
220
221       -O sock
222              Listen on a notification socket for connections from  listeners.
223              mimedefang-multiplexor  can  inform  external  programs of state
224              changes by sending messages over  a  notification  socket.   The
225              external  programs  connect  to  this socket and then listen for
226              notifications.  See the section  SOCKET  SPECIFICATION  for  the
227              format of sock.
228
229              See the mimedefang-notify(7) man page for details of the notifi‐
230              cation protocol.
231
232
233       -N map_sock
234              Listen on a map socket for Sendmail SOCKETMAP  connections.   As
235              of  Sendmail 8.13, you can define a Sendmail map type that talks
236              to a daemon over a  socket.   mimedefang-multiplexor  implements
237              that  protocol;  consult  the  mimedefang-filter(5) man page for
238              detils (see the SOCKET MAPS section).
239
240              See the section SOCKET SPECIFICATION for the format of map_sock.
241
242
243
244
245       -I backlog
246              When mimedefang-multiplexor creates a listening socket, it  cal‐
247              culates the "backlog" argument to listen(2) based on the maximum
248              number of slaves.  However, you can explicitly set this  backlog
249              with the -I option.  Setting the backlog to a high value (around
250              30-50) may help on a very busy server.  If you see mail log mes‐
251              sages saying "MXCommand: socket: Connection refused" during busy
252              periods, then that's an indication  you  need  a  higher  listen
253              backlog.
254
255
256       -L interval
257              Log  the  slave status every interval seconds.  This logs a line
258              using syslog; the line looks like this:
259
260
261       Slave status: Stopped=s Idle=i Busy=b Killed=k Queued=q Msgs=m Activations=a
262
263              Here, "Stopped" is the number of non-running slaves,  "Idle"  is
264              the  number of idle slaves, "Busy" is the number of busy slaves,
265              "Killed" is the number  of  killed  slaves  yet  to  be  reaped,
266              "Queued"  is  the number of queued requests, "Msgs" is the total
267              number of messages processed since the  multiplexor  began  run‐
268              ning,  and  "Activations"  is the number of times a Perl process
269              has been started since the multiplexor began running.
270
271              If you supply an interval of 0 (which is the default), no  peri‐
272              odic  status logging is performed.  If you supply an interval of
273              less than 5 seconds, it is silently reset to 5 seconds.
274
275
276       -S facility
277              Specifies the syslog facility for log messages.  The default  is
278              mail.   See  openlog(3) for a list of valid facilities.  You can
279              use either the short name ("mail") or long name ("LOG_MAIL") for
280              the facility name.
281
282
283       -E     Specifies  that  the  multiplexor should create an embedded Perl
284              interpreter.  This can improve  performance  dramatically.   But
285              see the section "EMBEDDING PERL" for more information.
286
287
288       -X n   Specifies  that the multiplexor should initiate a "tick" request
289              every n seconds.  This  causes  your  filter_tick  function  (if
290              defined) to be called.  Note that you have no control over which
291              slave executes filter_tick.  If all slaves are busy when a  tick
292              occurs,  that  tick  request is skipped and a warning message is
293              logged.
294
295
296       -P n   Specifies that the multiplexor should run  n  tick  requests  in
297              parallel.   Each  tick  is run as often as specified with the -X
298              argument.  (If you omit the  -P  option,  then  the  multiplexor
299              behaves as if -P 1 had been specified.)
300
301              If  you  run  parallel  ticks,  each tick is assigned an integer
302              identifying its "type".  The type ranges from 0 to  n-1.   While
303              there  may be as many as n tick requests running at a time, only
304              one tick of each type will be active at any time.
305
306
307       -Y label
308              Sets the tag used in the multiplexor's syslog messages to  label
309              instead of mimedefang-multiplexor.
310
311
312       -y n   Limits the maximum number of concurrent recipok checks to n on a
313              per-domain basis.  The value of n can range  from  0  (in  which
314              case  no  limit is applied) to maxSlaves, where maxSlaves is the
315              argument to the -x option.  If n is outside that  range,  it  is
316              ignored (and no limit is applied.)
317
318
319              The  recipok  command  ultimately  invokes  the filter_recipient
320              function in your filter.  If you are doing  recipient  verifica‐
321              tion against servers that may be slow or unreliable, you can use
322              the -y option to limit the number of concurrent recipient  veri‐
323              fications  per domain.  That way, if one domain's server becomes
324              very slow, it won't consume all available slaves  for  recipient
325              verification.  Instead, its RCPT commands will be tempfailed and
326              there will be slaves available to handle RCPT commands for other
327              domains.
328
329

SOCKET SPECIFICATION

331       The  -a,  -N  and -O options take a socket as an argument.  This socket
332       can be specified as:
333
334
335       /path/to/socket
336              A UNIX-domain socket
337
338
339       inet:portnum
340              A TCP socket bound to port portnum, but  which  accepts  connec‐
341              tions only from the IPv4 loopback address (127.0.0.1).
342
343
344       inet_any:portnum
345              A TCP socket bound to port portnum which will accept connections
346              from any address.  Use inet_any with caution!
347
348
349       inet6:portnum
350              A TCP socket bound to port portnum listening on the  IPv6  loop‐
351              back address.
352
353
354       inet6_any:portnum
355              A  TCP  socket bound to port portnum listening on the IPv6 wild‐
356              card address.
357
358

QUEUEING REQUESTS

360       Normally, if all slaves are busy, any additional  requests  are  failed
361       immediately.   However,  the  -q  and  -Q  options  allow  you to queue
362       requests for a short amount of time.   This  facility  is  intended  to
363       gracefully  handle  a  temporary overload; most of the time, your queue
364       should be empty.
365
366       Because mimedefang checks the number of free slaves when  a  connection
367       is  opened  and  fails  the connection if there are no free slaves, the
368       intent of the queue is to allow  SMTP  transactions  that  are  already
369       underway  to  continue  if there is a slight overload.  Any new connec‐
370       tions will be failed if all slaves are busy, but  existing  connections
371       are  allowed  to  continue.  Queuing requests may improve throughput on
372       extremely busy servers.
373
374       Note that if you supply the -q option to mimedefang, then even new con‐
375       nections  are allowed to queue.  This may improve throughput by keeping
376       the slave utilization higher.
377
378       The -R option to mimedefang can be used to reserve a  specified  number
379       of  slaves  for  connections  from  the loopback address.  Using the -R
380       option has the side-effect of permitting new connections from the loop‐
381       back address to queue.
382
383

EMBEDDING PERL

385       Normally,  when  mimedefang-multiplexor activates a slave, it forks and
386       execs mimedefang.pl.  However, if the  multiplexor  was  compiled  with
387       embedded  Perl  support, and you supply the -E command-line option, the
388       multiplexor works like this:
389
390
391       1      It creates an embedded Perl  interpreter,  and  sources  mimede‐
392              fang.pl  with a special command-line argument telling it to read
393              the filter, but not to enter the main loop.
394
395
396       2      Each time a slave is activated, the multiplexor calls fork() and
397              runs  the mimedefang.pl main loop.  This invokes filter_initial‐
398              ize and then runs the main loop.
399
400
401       On some platforms (for example, Red Hat Linux 7.3 with Perl 5.6.1),  it
402       is  not safe to destroy and recreate a Perl interpreter without causing
403       a memory leak.  On those platforms, if you attempt to reread the filter
404       file  (by  sending the multiplexor a HUP signal or reread command), the
405       filter will not be re-read, and a message will be logged to syslog.  On
406       those platforms, you must kill and restart MIMEDefang if you change the
407       filter file.
408
409
410       On most platforms, however, a filter reread is accomplished by destroy‐
411       ing and re-creating the embedded interpreter, re-sourcing mimedefang.pl
412       and killing slaves as soon as they are idle.
413
414
415       WARNING:  If you use the embedded Perl interpreter, the  Perl  variable
416       "$$"  will  not be updated with the process ID of each slave.  Instead,
417       it will reflect the process ID of the master multiplexor.   This  is  a
418       limitation of embedded Perl; we have no plans to fix it.
419
420

STATISTICS

422       With  the  -t  option,  mimedefang-multiplexor logs certain events to a
423       file.  This file can be post-processed to gather statistics  about  the
424       multiplexor.   You  can  use  it  to tune the number of slaves you run,
425       adjust timeouts, and so on.
426
427
428       Each line of the file looks like this:
429
430            YYYY/MM/DD:HH:MM:SS timestamp event key=val key=val...
431
432       Here, YYYY/MM/DD:HH:MM:SS is the local time of day.  Timestamp  is  the
433       number  of  seconds  since  January  1,  1970.  Event is the name of an
434       event.  The valid events are:
435
436
437       StartSlave
438              A slave process has been started.
439
440
441       KillSlave
442              A slave process has been killed.
443
444
445       ReapSlave
446              A dead slave process has been reaped.  It is possible to have  a
447              ReapSlave  event without a previous KillSlave event if the slave
448              process terminated abnormally.
449
450
451       StartFilter
452              A slave process has begun filtering an e-mail message.
453
454
455       EndFilter
456              A slave process has finished filtering an e-mail message.
457
458
459       The possible keys in the key=value pairs are:
460
461
462       slave=n
463              The slave involved in the event.  Every slave is identified by a
464              small integer.
465
466
467       nslaves=n
468              The  total  number of running slaves immediately after the event
469              happened.
470
471
472       nbusy=n
473              The number of busy slaves (slaves which are processing an e-mail
474              message) immediately after the event happened.
475
476
477       reason="string"
478              The  reason  for a StartSlave or KillSlave event.  (Present only
479              for these events.)
480
481
482       numRequests=n
483              The number of e-mails processed by the slave.  Present only  for
484              an EndFilter event.
485
486
487       If you send the mimedefang-multiplexor process a SIGHUP signal (kill -1
488       pid), it closes and reopens the statistics file.  This is useful during
489       log file rotation.
490
491
492       If  you  send  the mimedefang-multiplexor process a SIGINT signal (kill
493       -INT pid), it terminates all active-but-idle slaves.  Also, any active-
494       and-busy  slaves terminate as soon as they finish filtering the current
495       message.  This is useful to force a reread of  the  filter  rules  file
496       without stopping and restarting Sendmail.
497
498
499       If  you  send the mimedefang-multiplexor process a SIGTERM signal (kill
500       pid), it terminates all slaves and exits immediately.
501
502

AUTHOR

504       mimedefang-mulitplexor was written by David F.  Skoll  <dfs@roaringpen‐
505       guin.com>.  The mimedefang home page is http://www.mimedefang.org/.
506
507

SEE ALSO

509       mimedefang.pl(8),  mimedefang-filter(5), mimedefang(8), mimedefang-pro‐
510       tocol(7)
511
512
513
514
515
5164th Berkeley Distribution       8 February 2005      MIMEDEFANG-MULTIPLEXOR(8)
Impressum