[PREVIOUS CHAPTER] [______TOC_______]

2 Digest Delivery / Matome Okuri (Japanese term)

    2.1	Mechanism Of Digest Delivery

    Digest delivery process works as follows:

       for (;;) {	# infinite loop
    	wait for the digest delivery time

    	check $ACTIVE_LIST and $MSEND_RC to determine
    	"whom to send which range of articles" and "now"?
    	If msend.pl has members to be sent a bundle of articles,
    	msend.pl sends it to them.
       }

    This periodic sending mechanism is digest delivery. This mechanism
    clearly requires some help from OS. Cron on some systems (OS) kicks
    digest delivery program periodically. Also you need to set up the
    setting in your OS. We use "cron", standard service of UNIX.

    Each user controls its digest delivery parameters, "when" and "how
    articles are packed". FML provides commands for the control, "digest"
    and "matome" (Japanese term), described below.

    2.2	Digest Delivery Server (msend.pl)

    SYNOPSIS
       msend.pl    [ options ] $DIR [ options ]  [ $LIBDIR [ options ] ]

       * fml.pl, msend.pl and fmlserv.pl have the same argument syntax.

       -d		debug mode
       -bt		test mode

       -bd		daemon mode (become daemon)
       -sOPT $OPT	$OPT = 1;
       -uOPT $OPT	$OPT = 0;
       -lfile	$LOAD_LIBRARY = "file";
       -q		quite mode
    		if not quiet mode, msend.pl runs with showing
    		a lot of information.

    2.3	Example of crontab

    "makefml newml <ML>" command generates examples which include crontab
    example. Let $DIR be the ML's home directory. You need to set up cron
    kicks msend.pl periodically since system(OS) does not work for you in
    default. To set up cron, do

       (shell prompt)% crontab $DIR/crontab

    An example of crontab (4.4BSD) is

    0 * * * * /usr/libexec/fml/msend.pl /var/spool/ml/Elena /usr/libexec/fml

    where msend.pl runs at 0 minute in each hour.

    2.4	Control file of msend.pl $MSEND_RC(/var/log/msendrc)

    You need to define $MSEND_RC (defined as $DIR/var/log/msendrc in
    default). If it does not exist, msend.pl warns it and exits.  You
    should know you can run msend.pl. This policy comes from against the
    possibility that msend.pl may send a big bundle of articles if the
    configuration has an error.

    2.5	Command To Control Each User Digest Delivery Configuration

    Each user can control each digest delivery configuration by a command.
    e.g.
    	digest 3u
    	matome 3u
    which sets up digest delivery is sent once 3 hours with UNIX FROM
    format(PLAIN-TEXT). Please see help file for available options.

    	matome 3u

    2.6	Subject of Digest

    	$MSEND_SUBJECT_TEMPLATE

    is a template for Subject: of digest mails. It is defined as

        Digest _ARTICLE_RANGE_ _PART_ _ML_FN_

    in default. Each variable (_\S+_ style) is converted to in the
    following way

    	   _ARTICLE_RANGE_  <=>    Article 768
    	   _DOC_MODE_       <=>    [last:10 tar + gzip]
    	   _PART_           <=>    (1/4)
    	   _ML_FN_          <=>    $ML_FN (here is "(Elena ML)")

    These variables control Subject:. So you can customize Subject: for
    you. In default the subject: after this conversion will be like

       Digest Article 768 (1/1) (Elena ML)

    2.7	Available Hooks In Digest Delivery

    $MSEND_START_HOOK is evaluated after initialization but before the
    main routine.

    $MSEND_HEADER_HOOK is evaluated in the main routine and just before
    Subject: substitution described above.

    2.8	Digest Delivery Default Configuration

    The default file format is gzip-ed file format (for highly compression
    efficiency). You can change the default by the variable

    	$MSEND_MODE_DEFAULT

    where a set of available parameters is the same as ones for
    $MGET_MODE_DEFAULT (xref: ../internals 5.2).

    * A few shortcuts

    (After config.ph CFVersion 3)

    	$MSEND_MODE_DEFAULT = "rfc1153"    (default RFC1153)
    	$MSEND_MODE_DEFAULT = "rfc934"	   (default RFC934)

    Example: change the default to RFC934 style (e.g. You can split the
    format mail up by MH burst).

    	$MSEND_MODE_DEFAULT = "rfc934";

    	------- Forwarded Message

    	From: uja
    	Subject: daa

    	Books by Takemoto Izumi are wonderful.

    	------- Forwarded Message

    	From: uja
    	Subject: daa

    	Aoichan Panic is good, but Seifukumono is also good :D

    	--fukachan@ Kerochan is good

    	------- Forwarded Message

    To change the default to RFC1153 style.

    	$MSEND_MODE_DEFAULT = "rfc1153";

    FML does not check the sequence number since each user is each user
    parameters, time, file format ... and the time parameter can change.
    We cannot control them. Hence the sequence number is not maintained as
    unique.
    To hack RFC1153, see librfc1153.pl.

    2.9	"digest" command parameters

    FML checks modulus between the time and parameters.  "digest" command
    fundamental syntax is

    digest N 		If N is 0-23, a digest is delivered once N hours.
    			If N is 13-36, a digest is delivered once a day
    			at N hour.

    After N, file option may be added. If without a file format option,
    the file format is "gzip", which is gzip-ped file containing articles.

    	u	PLAIN-TEXT(UNIX FROM)
    	uf	PLAIN-TEXT(UNIX FROM)
    	i	lha + ish
    	ish	lha + ish
    	mp	MIME/Multipart	PLAIN-TEXT
    	d	RFC1153 format 	PLAIN-TEXT
    	b	RFC934 format 	PLAIN-TEXT(mh-burst)
    	rfc934
    	ish	lha + ish
    	lhaish
    	i
    	li
    	lu	lha + uuencode
    	lhauu
    	unpack	PLAIN-TEXT(UNIX FROM)

    digest 0		real time delivery (stop digest)

       matome 1		once in an hour  (file format is gzipd)
       matome 2		once in two hours (file format is gzipd)
       matome 2u		once in two hours (file format is unix from)
       matome 2mp		once in two hours (file format is MIME/Multipart)
       matome 17		once in a day (send at 17:00)
       matome 29		once in a day (send at 05:00; 29 = 24 + 5 )

    2.10	Notify that "no digest is delivered today" to members

    	$MSEND_NOTIFICATION = 1;

    FML sends "no digest today for no article" to members of the ML.

    	$MSEND_NOTIFICATION_SUBJECT (default "Notification $ML_FN")

    is Subject: of the notification mail.

    2.11	To add or not add X-ML-Info:

    	$MSEND_NOT_USE_X_ML_INFO = 1;

    disables X-ML-Info: in the header of digest mails.

    2.12	How msend.pl works

    Let actives (delivery list) be like this.

    	-------- actives --------
    	katri
    	pollyanna	m=6u
    	-------- actives --------

    	* cron kicks msend.pl once in an hour.

    	* checks delivery list (here actives) and find
    	  "pollyanna" hopes FML sends a pack of articles once in 6 hours
    	   with the file format "u"(UNIX FROM style plain text).

    	* checks whether the present hour % 6 is 0 or not.
    	  If the modulus is 0, msend.pl should sent articles to "pollyanna".
    	  If not, skip this user entry.

    	* logs the largest number of articles in $MSEND_RC for the next time.
    	  The range of sending articles is
    	  from the number for "pollyanna" in $MSEND_RC
    	  to the current sequence number ($SEQUENCE_FILE).

    Conclusion: FML does not send stored articles somewhere but generates
    mail with a bundle of articles for a user and sends it to him/her each
    time. It enables the customization for each user. The degree of
    freedom is the same as that of "mget" command.

    2.13	Modify when msend.pl should send articles?

    Please hack "sub MSendP" (msend-predicate) in msend.pl.

    Example: current code(sending if return value is 1)

        # CHECK: Modulus
        if (($ID - $Request{$who}) >= 0) {
    	return 1 if (0 == ($Hour % $When{$who}));
    	return 1 if ($When{$who} > 12 && ($Hour + 24 == $When{$who}));
        }

        return 0; # not send

    where
    	$ID		sequence number ($SEQUENCE_FILE)
    	$Hour		the present time (hour)
    	$who		user mail address
    	$When{$who}	user's parameter (e.g. 3, 6, 12, 17 ...)

    2.14	Digest only mailing list

    Please see contrib/msend-anytime patches.

    Apply this patch and run "msend.pl -a -A other-options".

    	$MSEND_MODE_DEFAULT

    $MSEND_MODE_DEFAULT is the default value (e.g. 3mp) of digest delivery
    parameter.

    2.15	About bin/cron.pl

    (^^)

    2.16	Digest Once In A Week

    See bin/weekend-msend.sh for an example.

    2.17	Cron (FML Vixie cron compatible program, bin/cron.pl)
    2.17	Cron (FML Vixie cron compatible program, bin/cron.pl)

    	man cron
    	man crontab

    The present OS may have a cron for users. Please check it in the
    first.  e.g. man cron, man crontab If only root can set up OS's cron
    so a general user cannot control cron, you can use FML cron (clone of
    4.4BSD cron).

    FML cron sleeps and runs once a minute.  If the crontab entry has a
    job now, cron run it.  It is same as UNIX cron daemon.

    You can run cron manually. fml.pl can run cron.pl automatically. If
    fml.pl automatically runs cron.pl, it runs each cron.pl for each
    ML. It is of no use. It is enough that one cron.pl for one user.

    fml.pl runs only one cron.pl, not plural for one ML. fml.pl checks
    $CRON_PIDFILE ($DIR/var/run/cron.pid) to ensure only one cron.pl
    running.

    cron.pl may be down. OS does not run cron.pl automatically, so
    digest delivery does not run until you are aware of it.
    For it, you can set up fml.pl runs cron.pl when mail comes in.

    In default cron.pl itself exits, and new cron.pl runs to avoid strange
    behavior when a process runs long long time.
    However -n option enables infinite running.

    SYNOPSIS
    	cron.pl [-a] [-d] [-b43] [-fcrontab] [-bd] $DIR [$LIBDIR]

        -d                debug mode
        -a                run eternally(default: 180sec. = 60sec. * 3times)
        -n                working all times without RESTART
        -mtimes           run from now to (60 * times) sec. after
        -fcrontab-file    alternative crontab
        -h                show this help and exit
        -bd		      daemon (Become Daemon)
        -bOSTYPE          -b43(OSTYPE = 43): 4.3BSD Like. not mailed to you

    -b43 option set

    	$CRON_NOTIFY    = 0; (default 1)

    In default, cron.pl notify the cron job log to the owner of the
    process by mail (like 4.4BSD cron).

    Also in default, cron.pl does not detach the current tty, that is, not
    become a daemon since becoming daemon does not work well on an OS.

    	-onotty

    cron.pl becomes daemon by itself. In default

    	$NOT_USE_TIOCNOTTY = 1;		# no ioctl

    2.18	Vixie Cron

    4.4BSD has Vixie Cron, so you can use it for yourself. Please use it
    on 4.4BSD. Also firstly you should check your OS's manuals.

    cron (8) - daemon to execute scheduled commands (Vixie Cron)
    crontab (1) - maintain crontab files for individual users (V3)
    crontab (5) - tables for driving cron

    An old OS has no such cron. However an administrator installed Vixie
    cron. It is of value to trek your system:-)

    cron.pl is a clone for such an old OS.

    2.19	How To Install cron.pl

    The system in the bootstrap time does not run cron.pl automatically.
    We would like to run cron.pl anyway. What should we do?

    To run cron.pl, the arguments are

    	% cron.pl $DIR(config.ph location) $LIBDIR(lib*.pl location)

    The problem is how to run cron!.  For example, you can set up the
    following ways:

       1. 	set up your ~/.login which run cron.pl. When you log in the
    	system, you run cron.pl. Cron.pl always checks other cron.pl
    	runs or not, so	only one cron.pl runs.

       2. runs cron.pl manually :)

       3. fml.pl runs cron.pl when mail comes in.

    config.ph example for case 3 is

    	$CRON_PIDFILE = "$DIR/var/run/cron.pid";
    	$CRONTAB      = "$DIR/etc/crontab";

    	$FML_EXIT_PROG = "$LIBDIR/bin/cron.pl $DIR $LIBDIR -a";

    cron.pid ($DIR/var/run/cron.pid) corresponds to /var/run/cron.pid but
    exists for each ML under FML.

    When you run "makefml newml", FML sets up each crontab for each ML.
    It is an example for Vixie cron.  You can use it on 4.4BSD and under
    Vixie cron.

    [Example]
    	% makefml newml elena
    	  ...
    	% cd /var/spool/ml/elena
    	% crontab crontab