head     1.2;
access   ;
symbols  ;
locks    ; strict;
comment  @# @;


1.2
date     90.05.22.10.03.02;  author vixie;  state Exp;
branches ;
next     1.1;

1.1
date     90.05.22.09.53.18;  author vixie;  state Exp;
branches ;
next     ;


desc
@@


1.2
log
@vixie
@
text
@$Header:$

This package is the decwrl multiqueue sendmail system. It is used to prevent
decwrl's mailer from collapsing in a puddle when things start to get tough.
decwrl moves about 15,000 messages per day, which means that if the network
gets clogged or blocked for a day, there are going to be about 7500 messages
in the queue (half of the traffic is inbound, which won't arrive when the
network is down). You try running sendmail with 7500 messages in mqueue
someday and see just how far you get.

The basic principle here is to move the mail around behind sendmail's back,
but to obey sendmail's locking protocol so we won't trip over each other.

Here is the internal documentation that we use at decwrl:

---------------------------------------------------------------------------

How the decwrl mail queueing system works.	Brian Reid, August 1989
			(last update:		Paul Vixie, May 1990)

Sendmail leaves undelivered messages in /usr/spool/mqueue.  The running
sendmail listener /usr/lib/sendmail -bd is not invoked with the -q option,
so it does not attempt delivery out of mqueue.

At intervals, a "queue mover" script, requeue.sh, is run to examine mqueue
and to move out of it any message deemed too old. There is a series of
queues, each one holding mail too old to be left in the previous queue and
too young to be moved on.

There are at this writing 8 queues, and the requeue code looks like this:

PN=/usr/local/adm/sendmail/requeue
$PN -d  2 -f /usr/spool/mqueue -t /usr/spool/mq1	
$PN -d  4 -f /usr/spool/mq1    -t /usr/spool/mq2	
$PN -d  8 -f /usr/spool/mq2    -t /usr/spool/mq3    
$PN -d 16 -f /usr/spool/mq3    -t /usr/spool/mq4
$PN -d 32 -f /usr/spool/mq4    -t /usr/spool/mq5 
$PN -d 64 -f /usr/spool/mq5    -t /usr/spool/mq6	
$PN -d 96 -f /usr/spool/mq6    -t /usr/spool/mq7 

In other words, anything older than 2 hours is moved to mq1, anything older
than 4 hours is moved to mq2, anything older than 8 hours is moved to mq3,
etc.

From time to time the sendmail queue processing is interrupted with various
lock files set. The requeue program times out the locks after 3 hours and
breaks them. The number "3 hours" is compiled into the requeue binary.
Requeue also does a garbage collection (delete old unreferenced files
in the "-f" directory); this behavior can be turned off by adding the "-n"
option to requeue.

The queue processing is done by the wrl.smqd daemons, which are started
from rc.local.  (wrl.smqd is usually placed in /etc for this reason.)
wrl.smqd's code is as follows:

	#! /bin/sh
	find /tmp -name 'wrl.smqd.*' -mtime +1 -exec /bin/rm {} \;
	while :
	do
		nice -10 /usr/lib/sendmail -q -v -oY $@@ > /tmp/wrl.smqd.$$ 2>&1
		sleep 60
	done
	exit

That is, it starts by removing old versions of its lock files, then loops
forever on (run the queue; sleep a minute).  wrl.smqd's startup is usually
part of the "sendmail" startup in rc.local, such as:

[ -f /usr/lib/sendmail ] && {
        find /mail/mq* -name '[tl]f*' -print | xargs rm         >/dev/console
        /usr/lib/sendmail -bz
        /usr/lib/sendmail -bd & echo -n ' sendmail'             >/dev/console
        for x in /mail/mq*
        do
                /etc/wrl.smqd -oQ$x & sleep 2
                /etc/wrl.smqd -oQ$x & sleep 2
                /etc/wrl.smqd -oQ$x & sleep 2
        done
}

This example starts three queue daemons per queue, which is right for decwrl
but almost certainly wrong for anywhere else.

The "countmail" program (usually installed in /usr/local/bin/countmail) can
be used to count the number of messages in each of the queues. They are
usually too big to look at with mailq, but if you want, you can say
	mailq -oQ/usr/spool/mq5
to look into that queue.
@


1.1
log
@original/reid
@
text
@d1 2
d19 1
d21 3
a23 4
Sendmail leaves undelivered messages in /usr/spool/mqueue.
The running sendmail listener /usr/lib/sendmail -bd is 
not invoked with the -q option, so it does not attempt delivery out
of mqueue.
d40 1
d52 3
a54 4
The queue processing is done by the sendmailq.* programs, each of which is
a symbolic link to sendmailq.proto, which learns the name of the queue
directory from the name of the symlink. For example, sendmailq.mq4 uses mq4
as a spooling directory.
d56 8
a63 8
The unspooler sendmailq.proto looks into the file .forks in the spooling
directory to find out how many times to fork. If that file is missing, it
runs one stream. If the file has contents, then it forks one additional
unspooling stream for each word in the .forks file. For example, if you say
	echo A B C > /usr/spool/mq5/.forks
then it will run sendmailq, sendmailqA, sendmailqB, and sendmailqC. The
letters A B C are used only as suffixes to the log file names, so they have
to be unique and be something reasonable in a file name.
d65 3
a67 5
The individual log files (more detail than syslog) are kept in (e.g.)
/usr/spool/mq4/slowq.log.nnX, where "nn" is the hour that the unspooler was
started and "X" is the suffix letter. This makes it messy to start an
unspooler run more than once during each wall-clock hour, because they will
overwrite each other's log files.
d69 18
a86 3
The program /usr/local/bin/countmail can be used to count the number of
messages in each of the queues. They are usually too big to look at with
mailq, but if you want, you can say
@
