sendmailanalyzer/doc/sendmailanalyzer.3

.\" Automatically generated by Pod::Man 2.25 (Pod::Simple 3.16)
.\"
.\" Standard preamble:
.\" ========================================================================
.de Sp \" Vertical space (when we can't use .PP)
.if t .sp .5v
.if n .sp
..
.de Vb \" Begin verbatim text
.ft CW
.nf
.ne \\$1
..
.de Ve \" End verbatim text
.ft R
.fi
..
.\" Set up some character translations and predefined strings.  \*(-- will
.\" give an unbreakable dash, \*(PI will give pi, \*(L" will give a left
.\" double quote, and \*(R" will give a right double quote.  \*(C+ will
.\" give a nicer C++.  Capital omega is used to do unbreakable dashes and
.\" therefore won't be available.  \*(C` and \*(C' expand to `' in nroff,
.\" nothing in troff, for use with C<>.
.tr \(*W-
.ds C+ C\v'-.1v'\h'-1p'\s-2+\h'-1p'+\s0\v'.1v'\h'-1p'
.ie n \{\
.    ds -- \(*W-
.    ds PI pi
.    if (\n(.H=4u)&(1m=24u) .ds -- \(*W\h'-12u'\(*W\h'-12u'-\" diablo 10 pitch
.    if (\n(.H=4u)&(1m=20u) .ds -- \(*W\h'-12u'\(*W\h'-8u'-\"  diablo 12 pitch
.    ds L" ""
.    ds R" ""
.    ds C` ""
.    ds C' ""
'br\}
.el\{\
.    ds -- \|\(em\|
.    ds PI \(*p
.    ds L" ``
.    ds R" ''
'br\}
.\"
.\" Escape single quotes in literal strings from groff's Unicode transform.
.ie \n(.g .ds Aq \(aq
.el       .ds Aq '
.\"
.\" If the F register is turned on, we'll generate index entries on stderr for
.\" titles (.TH), headers (.SH), subsections (.SS), items (.Ip), and index
.\" entries marked with X<> in POD.  Of course, you'll have to process the
.\" output yourself in some meaningful fashion.
.ie \nF \{\
.    de IX
.    tm Index:\\$1\t\\n%\t"\\$2"
..
.    nr % 0
.    rr F
.\}
.el \{\
.    de IX
..
.\}
.\"
.\" Accent mark definitions (@(#)ms.acc 1.5 88/02/08 SMI; from UCB 4.2).
.\" Fear.  Run.  Save yourself.  No user-serviceable parts.
.    \" fudge factors for nroff and troff
.if n \{\
.    ds #H 0
.    ds #V .8m
.    ds #F .3m
.    ds #[ \f1
.    ds #] \fP
.\}
.if t \{\
.    ds #H ((1u-(\\\\n(.fu%2u))*.13m)
.    ds #V .6m
.    ds #F 0
.    ds #[ \&
.    ds #] \&
.\}
.    \" simple accents for nroff and troff
.if n \{\
.    ds ' \&
.    ds ` \&
.    ds ^ \&
.    ds , \&
.    ds ~ ~
.    ds /
.\}
.if t \{\
.    ds ' \\k:\h'-(\\n(.wu*8/10-\*(#H)'\'\h"|\\n:u"
.    ds ` \\k:\h'-(\\n(.wu*8/10-\*(#H)'\`\h'|\\n:u'
.    ds ^ \\k:\h'-(\\n(.wu*10/11-\*(#H)'^\h'|\\n:u'
.    ds , \\k:\h'-(\\n(.wu*8/10)',\h'|\\n:u'
.    ds ~ \\k:\h'-(\\n(.wu-\*(#H-.1m)'~\h'|\\n:u'
.    ds / \\k:\h'-(\\n(.wu*8/10-\*(#H)'\z\(sl\h'|\\n:u'
.\}
.    \" troff and (daisy-wheel) nroff accents
.ds : \\k:\h'-(\\n(.wu*8/10-\*(#H+.1m+\*(#F)'\v'-\*(#V'\z.\h'.2m+\*(#F'.\h'|\\n:u'\v'\*(#V'
.ds 8 \h'\*(#H'\(*b\h'-\*(#H'
.ds o \\k:\h'-(\\n(.wu+\w'\(de'u-\*(#H)/2u'\v'-.3n'\*(#[\z\(de\v'.3n'\h'|\\n:u'\*(#]
.ds d- \h'\*(#H'\(pd\h'-\w'~'u'\v'-.25m'\f2\(hy\fP\v'.25m'\h'-\*(#H'
.ds D- D\\k:\h'-\w'D'u'\v'-.11m'\z\(hy\v'.11m'\h'|\\n:u'
.ds th \*(#[\v'.3m'\s+1I\s-1\v'-.3m'\h'-(\w'I'u*2/3)'\s-1o\s+1\*(#]
.ds Th \*(#[\s+2I\s-2\h'-\w'I'u*3/5'\v'-.3m'o\v'.3m'\*(#]
.ds ae a\h'-(\w'a'u*4/10)'e
.ds Ae A\h'-(\w'A'u*4/10)'E
.    \" corrections for vroff
.if v .ds ~ \\k:\h'-(\\n(.wu*9/10-\*(#H)'\s-2\u~\d\s+2\h'|\\n:u'
.if v .ds ^ \\k:\h'-(\\n(.wu*10/11-\*(#H)'\v'-.4m'^\v'.4m'\h'|\\n:u'
.    \" for low resolution devices (crt and lpr)
.if \n(.H>23 .if \n(.V>19 \
\{\
.    ds : e
.    ds 8 ss
.    ds o a
.    ds d- d\h'-1'\(ga
.    ds D- D\h'-1'\(hy
.    ds th \o'bp'
.    ds Th \o'LP'
.    ds ae ae
.    ds Ae AE
.\}
.rm #[ #] #H #V #F C
.\" ========================================================================
.\"
.IX Title "SENDMAILANALYZER 1"
.TH SENDMAILANALYZER 1 "2013-03-02" "perl v5.14.2" "User Contributed Perl Documentation"
.\" For nroff, turn off justification.  Always turn off hyphenation; it makes
.\" way too many mistakes in technical documents.
.if n .ad l
.nh
.SH "NAME"
SendmailAnalyzer \- Sendmail/Postfix log analyzer
.SH "DESCRIPTION"
.IX Header "DESCRIPTION"
SendmailAnalyzer as is name suggest is a Sendmail log analyzer. It process
maillog files and generate dynamic statistics in \s-1HTML\s0 and graphical output.
The reports are generated in real time so that it let you know at any moment
what is going on your mail servers. It use time (hour, day, month and year
views) and cross-linked navigation for easy use.
.PP
SendmailAnalyzer is easy to install and highly configurable to match the dozen
of Sendmail possible configurations. It also support report for all the major
milter or sendmail filters like SpamAssassin, MailScanner, Clamav, etc.
.PP
Collected data are stored in flat files that are automatically archived or
delete to keep disk space. All reports before the current day are cached to
save system resources and are displayed in the 1 second into your browser.
.PP
SendmailAnalyzer can be run on a home dedicated mail server, on multiple
enterprise mail servers and on \s-1ISP\s0 mail servers for free. His low resources
usage allow SendmailAnalyzer appliance embedding. Since version 8.0 the caching
mechanism has a very low memory footprint as well as the reports views.
.PP
This is the most advanced and complete statistics tool dedicated to
the great Sendmail \s-1MTA\s0. It's goal is not to support any kind of \s-1MTA\s0 or
other log format but only being a full featured tool for Sendmail users and
administrators. If you're searching something more general take a look at
SawMill, it's not so bad :\-)
.SH "POSTFIX SUPPORT"
.IX Header "POSTFIX SUPPORT"
SendmailAnalyzer is a statistical dedicated tool for Sendmail and it is very
good in this task. As many people ask me to have such free tool for the Postfix
\&\s-1MTA\s0, Since release v7.0 SendmailAnalyzer now also support the Postfix mail.log
statistics report.
.PP
Postfix is now fully supported, if you have any issue or unsupported feature
please let me know. Note that as I don't use postfix I may ask you for log
file to reproduce some issues or develop features.
.SH "FEATURES"
.IX Header "FEATURES"
It reports all you ever wanted to know about email trafic on your network.
.SS "Global Statistics"
.IX Subsection "Global Statistics"
All the following reports also show statistics per bytes and average
of bytes per message.
.IP "\(bu" 4
Number of inbound messages.
.IP "\(bu" 4
Number of outbound messages.
.IP "\(bu" 4
Number of inbound spams.
.IP "\(bu" 4
Number of outbound spams.
.IP "\(bu" 4
Number of inbound virus.
.IP "\(bu" 4
Number of outbound virus.
.IP "\(bu" 4
Sendmail rejection rules flow.
.IP "\(bu" 4
Syserr flow (Sendmail error messages).
.IP "\(bu" 4
Sendmail \s-1DSN\s0 Flow (Delivery Status Notification)
.IP "\(bu" 4
The global \s-1MTA\s0 status allocation per messages, bytes and percentage.
.IP "\(bu" 4
Distributed messages coming from Internet.
.IP "\(bu" 4
Distributed messages sent internaly.
.IP "\(bu" 4
Distributed messages sent to Internet.
.IP "\(bu" 4
Distributed messages coming from and going to Internet.
.IP "\(bu" 4
Sendmail \s-1SMTP\s0 Auth statistics by type (server or client), mechanismi and user.
.IP "\(bu" 4
Postgrey usage statistics.
.PP
If you deliver marqued spam / virus to recipients, SendmailAnalyzer
will report the delivery flow for:
.IP "\(bu" 4
Spam / virus coming from Internet.
.IP "\(bu" 4
Spam / virus sent internaly.
.IP "\(bu" 4
Spam / virus sent to Internet.
.IP "\(bu" 4
Spam / virus coming from and going to Internet.
.PP
Note: In the report you will see 'local' inbound or outbound message, that
mean a mail coming from (sender relay) or going to (recipient relay) the mail
server.  This is not the same that internal, which mean coming from or sent to
your internal network / private domain.
.SS "Top Statistics"
.IX Subsection "Top Statistics"
Once you have define in the configuration file the Top Max statistics to show
(25 by default), the Max Recipient for a message (25 by default), the message
Size Max (5Mb by default) you will see the top statistics of:
.IP "\(bu" 4
Top sender domain, top sender relay, top sender address.
.IP "\(bu" 4
Top recipient domain, top recipient relay, top recipients address.
.IP "\(bu" 4
Top spams rules, top spammers domain, top spammers relays, top spammers address,
top spam recipients address.
.IP "\(bu" 4
Top virus name, top virus sender, top virus sender relay, top virus recipient
address, top infected filename.
.IP "\(bu" 4
Top \s-1DSN\s0 status, top \s-1DSN\s0 sender, top \s-1DSN\s0 Relay, top \s-1DSN\s0 recipient address.
.IP "\(bu" 4
Top rejection rules, top rejected domain, top rejected relay, top rejected
sender, top rejection status.
.IP "\(bu" 4
Top Sendmail Error messages.
.IP "\(bu" 4
Top max number of recipient for one message.
.IP "\(bu" 4
Top max size message with number of recipients and sender address.
.IP "\(bu" 4
Top Sendmail \s-1SMTP\s0 Auth mechanisms, relays and users (server or client).
.IP "\(bu" 4
Top Postgrey status, relay, senders and recipients.
.PP
Note: on daily view you can click on each of the reported element to see the
detailled informations. For example if you follow link on a sender relay you
will see all messages detailled information coming from that relay.
This kind of navigation is only available for the days of the current month
to keep disk space, memory usage and privacy.
.SS "\s-1ISP\s0 like feature"
.IX Subsection "ISP like feature"
Begining at version 4.0 of SendmailAnalyzer some features could be related
to an \s-1ISP\s0 like environment and allow statistics on very huge \s-1SMTP\s0 flow:
.IP "\(bu" 4
Support centralized maillog for multiple Sendmail serveurs througth rsyslog.
.IP "\(bu" 4
Support per domain reports with user access control.
.IP "\(bu" 4
Support per user reports. Each user can see is own statistics. (Removed in v5.0 until now)
.IP "\(bu" 4
Low memory usage, small disk space utilization and really speed with daily caching.
.IP "\(bu" 4
Support parsing of compressed maillog file.
.SS "Milter / Filter supported"
.IX Subsection "Milter / Filter supported"
SendmailAnalyzer support some of the most used milter and filter for
spam and virus filtering. If you don't find yours drop me a line and
it will be included.
.IP "\(bu" 4
MimeDefang Spam and Virus reports
.IP "\(bu" 4
Amavis Spam / Virus detection
.IP "\(bu" 4
Clamav virus detection
.IP "\(bu" 4
Jchkmail Spam / Virus report
.IP "\(bu" 4
MailScanner Spam / virus detection
.IP "\(bu" 4
SpamAssassin Spam detection (spamd output)
.IP "\(bu" 4
Sendmail \s-1DNSLB\s0 report (check_relay)
.IP "\(bu" 4
Sendmail \s-1DSN\s0 (Delivery Status Notification)
.IP "\(bu" 4
DNSLB-Milter Spam detection
.PP
If your one is not listed here and you can send me some relevant
maillog lines I can add his support in a day.
.SS "New features"
.IX Subsection "New features"
If you need new features and support for new/other milters or filters,
let me know. This help a lot to develop a better/useful tool. This
piece of software is widely use at my work (espacially for \s-1IT\s0 report)
but this reflect only a part of the Sendmail usage.
.SS "Internationalization"
.IX Subsection "Internationalization"
SendmailAnalyzer can be translated to any language with your contribution.
At this time supported language are: French, English, Spanish, Bulgarian, German.
If you want to add your own language, it's really simple, take a look in
the cgi\-bin/lang/ directory and send me the translation file.
.SH "REQUIREMENT"
.IX Header "REQUIREMENT"
SendmailAnalyzer can work in any platform where Sendmail and Perl could run.
What you need is a modern Perl distribution 5.8.x or more is good but older
version should also work.
.PP
You need the following Perl modules. If they are not yet include in your
\&\s-1OS\s0 distribution you can always find them at http://search.cpan.org/
.PP
.Vb 6
\&        MIME::Base64;
\&        MIME::QuotedPrint;
\&        GD
\&        GD::Graph
\&        GD::TextUtil
\&        GD::Graph::bars3d;
.Ve
.PP
The graph output are generated using the libgd and libpng. You can get them
at the following places:
.PP
.Vb 2
\&        http://www.libgd.org/
\&        http://www.libpng.org/
.Ve
.SH "INSTALLATION"
.IX Header "INSTALLATION"
.SS "Generic install"
.IX Subsection "Generic install"
Here is the generic installation steps, but if you want you can create and
install your own distribution package, see \*(L"Package install\*(R" bellow.
.PP
1) Unpack the distribution tarball in the desired location as follow:
.PP
.Vb 4
\&        tar xzf sendmailanalyzer\-x.x.tar.gz
\&        cd sendmailanalyzer\-x.x/
\&        perl Makefile.PL
\&        make && make install
.Ve
.PP
2) Follow the instruction given at the end of install. With this default
install everything will be installed under /usr/local/sendmailanalyzer.
.PP
3) Edit sendmailanalyzer.conf file to customize your SendmailAnalyzer reports.
See the configuration file and \s-1CONFIGURATION\s0 section bellow for usage.
.SS "Post install"
.IX Subsection "Post install"
1. Start SendmailAnalyzer daemon with:
.PP
.Vb 1
\&        /usr/local/sendmailanalyzer/sendmailanalyzer \-f
.Ve
.PP
or use one of the starters script provided in the start_scripts/ directory.
.PP
2. Modify your httpd.conf to allow access to \s-1CGI\s0 scripts like follow:
.PP
.Vb 11
\&        Alias /sareport /usr/local/sendmailanalyzer/www
\&        <Directory /usr/local/sendmailanalyzer/www>
\&                Options ExecCGI
\&                AddHandler cgi\-script .cgi
\&                DirectoryIndex sa_report.cgi
\&                Order deny,allow
\&                Deny from all
\&                Allow from 127.0.0.1
\&                Allow from ::1
\&                # Allow from .example.com
\&        </Directory>
.Ve
.PP
3. If necessary, give additional host access to SendmailAnalyzer in httpd.conf. Restart and ensure that httpd is running.
.PP
4. Browse to http://mta.host.dom/sareport/ to ensure that things are working properly.
.PP
5. Setup a cronjob to run sa_cache and restart SendmailAnalyzer daemon after maillog logrotate as follow:
.PP
.Vb 4
\&        # SendmailAnalyzer log reporting daily cache
\&        0 1 * * * /usr/local/sendmailanalyzer/sa_cache > /dev/null 2>&1
\&        # On huge MTA you may want to have five minutes caching
\&        #*/5 * * * * /usr/local/sendmailanalyzer/sa_cache \-a > /dev/null 2>&1
.Ve
.PP
6. Add an entry in /etc/logrotate.d/syslog to restart SendmailAnalyzer when maillog is rotated or create a cron job.
.SS "Log rotate case"
.IX Subsection "Log rotate case"
If you use real time mode and you have logrotate installed on you maillog
file you must restart SendmailAnalyzer each time logrotate is used.
To install it edit the /etc/logrotate.d/syslog file and add a line
in the postrotate part, for example:
.PP
.Vb 8
\&        /var/log/cron /var/log/debug /var/log/maillog /var/log/messages /var/log/secure /var/log/spooler /var/log/syslog {
\&            sharedscripts
\&            postrotate
\&                /bin/kill \-HUP \`cat /var/run/syslogd.pid 2>/dev/null\` 2>/dev/null || true
\&                /bin/kill \-HUP \`cat /var/run/sendmailanalyzer.pid\`  2>/dev/null || true
\&                # or /etc/init.d/sendmailanalyzer restart >/dev/null 2>&1 || true
\&            endscript
\&        }
.Ve
.PP
If you are using rsyslog the file is named /etc/logrotate.d/rsyslog, things must be written differently, but not so much.
.SS "Package install"
.IX Subsection "Package install"
In the packaging/ directory you will find all scripts and file to generate
binary \s-1RPM\s0, Slackware and Debian package. See \s-1README\s0 in this directory.
.SS "Custom install"
.IX Subsection "Custom install"
You can create your fully customized SendmailAnalyzer installation by using
the Makefile.PL Perl script. Here is a sample:
.PP
.Vb 9
\&        perl Makefile.PL \e
\&                LOGFILE=/var/log/maillog \e
\&                BINDIR=/usr/bin \e
\&                CONFDIR=/etc \e
\&                PIDDIR=/var/run \e
\&                BASEDIR=/var/lib/sendmailanalyzer \e
\&                HTMLDIR=/var/www/sendmailanalyzer \e
\&                MANDIR=/usr/man/man3 \e
\&                DOCDIR=/usr/share/doc/sendmailanalyzer
.Ve
.PP
If you want to build a distro package, there are two other options that you may
use. The \s-1QUIET\s0 option is to tell to Makefile.PL to not show the default post
install \s-1README\s0. The \s-1DESTDIR\s0 is to create and install all files in a package
build base directory. For example for Fedora \s-1RPM\s0, thing may look like that:
.PP
.Vb 10
\&        # Make Perl and SendmailAnalyzer distrib files
\&        %{_\|_perl} Makefile.PL \e
\&            INSTALLDIRS=vendor \e
\&            QUIET=1 \e
\&            LOGFILE=/var/log/maillog \e
\&            BINDIR=%{_bindir} \e
\&            CONFDIR=%{_sysconfdir} \e
\&            PIDDIR=%{rundir} \e
\&            BASEDIR=%{_localstatedir}/lib/%{uname} \e
\&            HTMLDIR=%{webdir} \e
\&            MANDIR=%{_mandir}/man3 \e
\&            DOCDIR=%{_docdir}/%{uname}\-%{version} \e
\&            DESTDIR=%{buildroot} < /dev/null
.Ve
.PP
See spec file in packaging/RPM for full \s-1RPM\s0 build script.
.SH "USAGE"
.IX Header "USAGE"
There's two way to use SendmailAnalyzer. If you don't need real time 
you can run it each night so that maillog will be parsed and reports
generated once a day. Note that if you have a huge \s-1MTA\s0 load this not
a good solution.
.PP
The other way is to run it in daemon mode, in this way it can parse huge
maillog (million line per day) preserving system resources.
.PP
To know all possible command line arguments, run 'sendmailanalyzer \-\-help'
.PP
Important: if you experience high memory usage with SendmailAnalyzer
use the \-w (\-\-write\-delay) command line option to reduce the time where in
memory data are flushed to disk. Default is 60 secondes, this is good in most
configuration but in huge servers you may set it as low as 5 secondes.
You must test it to find a compromise between speed and memory usage.
.SS "Standalone"
.IX Subsection "Standalone"
To run SendmailAnalyzer in standalone mode you have to setup a cron
entry each night as follow assuming log and configuration files in
default place (/var/log/maillog and /usr/local/sendmailanalyzer/sendmailanalyzer.conf):
.PP
.Vb 1
\&        /usr/local/sendmailanalyzer/sendmailanalyzer \-i \-b \-f
.Ve
.PP
This will run the program in interactive mode (\-i), parse full maillog seeking
after the last run ending position (\-f) and exiting at end of maillog parsing
(\-b).
.SS "Daemon mode"
.IX Subsection "Daemon mode"
To run SendmailAnalyzer as a daemon, use the start/stop/restart script given
with the distribution (in start_script/ directory). See the \s-1README\s0 file in
that directory for more explanation about how to install.
.PP
It will start as 'sendmailanalyzer \-f' that tell him to start in daemon mode
(default), parse full maillog seeking after the last run ending position (\-f)
and to open a pipe to a tail command on /var/log/maillog. It will never end
until you kill it or restart it.
.PP
To restart sendmailanalyzer use the \s-1SIGHUP\s0 signal as follow :
.PP
.Vb 3
\&        /bin/kill \-HUP \`cat /var/run/sendmailanalyzer.pid\`
\&or
\&        /bin/kill \-HUP sendmailanalyzer
.Ve
.PP
This will force sendmailanalyzer to reread his configuration file and reopen
a pipe to the tail command on you mail log file. The originals command line
arguments that you've given ar startup^will be preserved.
.PP
Important: If you have syslog rotate enable (I hope so :\-) you will have to
restart SendmailAnalyzer after each log rotation to always tail the good file
descriptor.
.PP
Edit /etc/logrotate.d/syslog and add the following after syslog restart:
.PP
.Vb 1
\&        /bin/kill \-HUP \`cat /var/run/sendmailanalyzer.pid\`  2>/dev/null || true
.Ve
.PP
or
	/etc/rc.d/rc.sendmailanalyzer restart /dev/null 2>&1 || true
.PP
or on Debian/Redhat like system
.PP
.Vb 1
\&        /etc/init.d/sendmailanalyzer restart /dev/null 2>&1 || true
.Ve
.PP
this must be in the postrotate section.
.SS "Stopping SendmailAnalyzer"
.IX Subsection "Stopping SendmailAnalyzer"
Just kill it with \s-1SIGTERM\s0 signal it will flush current collected object
to disk and free open files.
.PP
.Vb 3
\&        kill \-TERM \`cat /var/run/sendmailanalyzer.pid\`\*(Aq
\&or
\&        kill \-TERM sendmailanalyzer
.Ve
.PP
use the starter script. This will kill the current sendmailanalyzer
process and the pipe to the tail command.
.SS "Caching"
.IX Subsection "Caching"
SendmailAnalyzer collect maillog entries to write datas to flat files,
when you run the \s-1CGI\s0 script sa_report.cgi it had to read each data files
for the given period to compute statistics and output \s-1HTML\s0 reports.
This can be enought for day views but when you jump to month view it
cost a lot in \s-1CPU\s0 and memory usage unless you have a home \s-1MTA\s0.
.PP
To speed up things and free system resources you have to run the script
sa_cache each night by cron to create cache files. After that viewing
a month or year view take less than a second.
.PP
The script sa_cache must be run by cron as follow:
.PP
.Vb 1
\&        /usr/local/sendmailanalyzer/sa_cache >/dev/null 2>&1
.Ve
.PP
If you have set per domain report sa_cache will create cache files for each
domains. These cache files are name cache.pm for the \s-1MTA\s0 global statistics
and cache.pmYOURDOMAIM.DOM for each domain report. To lowered the memory
footprint of the sa_cache program, since version 8.0 it start computing cache
file per hours.
.PP
Since version 4.0 sa_report.cgi will warm you to avoid out of memory when
your entering a month view without caching.
.SS "Huge \s-1MTA\s0 activity"
.IX Subsection "Huge MTA activity"
On \s-1MTA\s0 server with very huge activity you can experience out of memory or
wait a very long time before seeing anything in day view. In this case you
must run by cron job the perl script sa_cache with the \-a option to build
cache files for the current day. Statistics will not be shown in realtime but
only at the time of the last sa_cache run. You can run it each five minute for
example as follow:
.PP
.Vb 1
\&    */5 * * * * /usr/local/sendmailanalyzer/sa_cache \-a
.Ve
.PP
or
.PP
.Vb 1
\&    */5 * * * * /usr/local/sendmailanalyzer/sa_cache \-\-actual\-day\-only
.Ve
.PP
It will only parse data stored in the current day so five minutes interval
may be enough for most case.
.SS "Database"
.IX Subsection "Database"
SendmailAnalyzer store data into flat file database. Data are store in
a time hierarchical directory structure ending at daily level. This structure
is composed as follow : 'mailhost'/year/month/day/
In each day repository you can find the following data files.
.PP
.Vb 10
\&        senders.dat: senders informations.
\&        recipient.dat: recipients informations.
\&        spam.dat: spams informations.
\&        virus.dat: viruses informations.
\&        rejected.dat: rejected mail informations.
\&        dsn.dat: Delivery Status Notification report
\&        syserr.dat: SYSERR MTA informations.
\&        other.dat: other message grabbed into the log file.
\&        auth.dat: SMTP auth message grabbed into the log file.
\&        miltername.dat: message related to a milter, antivir or antispam.
.Ve
.PP
The format of each file is explain in the SendmailAnalyzer code source.
.SS "Archiving"
.IX Subsection "Archiving"
When sa_cache is run and following the value of the \s-1FREE_SPACE\s0 configuration
option it will try to archive data older than the current month. If \s-1FREE_SPACE\s0
is set to 'delete' sa_cache will simply remove the data file from disk. If you
set it to 'archive', sa_cache will build a gzipped tarball for all daily data
file into the corresponding month directory and the remove date file from disk.
.PP
If you set it to 'none', data file are kept.
.PP
If you're primary concerned in disk space saving set it to 'delete'. If you
want to preserve data for a year or more you can safely set it to 'archive'.
For your information one of my server has 100,000 inbound message a day and
a year of 'archive' storage take around 1Gb and a 'delete' storage around
250Mb.
.PP
One advantage of the 'archive' method is that you can replay the cached stats
(for example after an upgrade to fix a sa_cache bug :\-). In this case, you
just have to delete any cache file and extract all tarbal as follow :
.PP
.Vb 3
\&        find /path/to/SendmailReport/ \-name "cache.pm*" | xargs \-i rm \-f {}
\&        find /path/to/SendmailReport/ \-name "history.tar.gz" | xargs \-i \e
\&                tar xzf {} \-\-directory /
.Ve
.PP
and then rerun sa_cache again.
.PP
Important: running sa_cache in one pass on en entire year could cost a lot
of resources and takes very long time. In this case add a second argument to
the command line giving the year/month to proceed, for example:
.PP
.Vb 1
\&        sa_cache \-s \*(Aqmailhost\*(Aq \-d "2008/06"
.Ve
.PP
repeat this command for each month.
.SH "CONFIGURATION"
.IX Header "CONFIGURATION"
The default path to configuration file is /etc/sendmailanalyzer.conf If you want
to change this path, please edit cgi\-bin/sa_report.cgi, sa_cache to match you're
need. For sendmailanalyzer use the \-\-config|\-c command line argument.
.PP
The configuration file consist in a text file with a configuration option
in upper case and a value or list of value separated by a tab character.
.PP
Here are the definition of all this configuration directives.
.SS "System commands options"
.IX Subsection "System commands options"
.IP "\s-1TAIL_PROG\s0" 4
.IX Item "TAIL_PROG"
Path to the system tail command. Can be overwritten with \-\-tail or \-t in
sendmailanalyzer args. Default is /usr/bin/tail.
.IP "\s-1TAIL_ARGS\s0" 4
.IX Item "TAIL_ARGS"
Command line argument passed to the tail system command. Can be overwritten
with \-\-args or \-a in sendmailanalyzer args. Default is \-n 0 \-f.
.IP "\s-1ZCAT_PROG\s0" 4
.IX Item "ZCAT_PROG"
Path to zcat system command used to parse compressed log file. Can be
overwritten with \-\-zcat or \-z in sendmailanalyzer args. Default is
/usr/bin/zcat.
.IP "\s-1FREE_SPACE\s0" 4
.IX Item "FREE_SPACE"
Select the freeing space method for data files older than the current month.
The value can be:
.Sp
.Vb 3
\&        \- delete: definitively remove all data files.
\&        \- archive: make a gzipped tarball of data files before deleting them.
\&        \- none: don\*(Aqt do anything. Need lot of space disk.
.Ve
.Sp
Default is archive.
.SS "Input/output options"
.IX Subsection "Input/output options"
.IP "\s-1LOG_FILE\s0" 4
.IX Item "LOG_FILE"
Path to the maillog file to analyse. Can be overwritten with \-\-log or \-l
in sendmailanalyzer args. Default is /var/log/maillog. If the extension
is .gz SendmailAnalyzer will automatically use zcat to parse the compressed
log. For Postix you may use /var/log/mail.log instead.
.IP "\s-1OUT_DIR\s0" 4
.IX Item "OUT_DIR"
Output directory for data storage. Can be overwritten with \-\-output or \-o
in sendmailanalyzer args. The directory must exist, being writable by
the user running sendmailanalyzer and sa_cache. It must be readable
by the http user for \s-1CGI\s0 script sa_report.cgi.
Default is /var/www/sendmailanalyzer
.IP "\s-1DEBUG\s0" 4
.IX Item "DEBUG"
Turn on/off debug/verbose output mode. Can be overwritten with \-\-debug or \-d
in sendmailanalyzer args. Default is 0, disable.
.IP "\s-1DELAY\s0" 4
.IX Item "DELAY"
Delay in second to flush collected data to disk. Can be overwritten with
\&\-\-write\-delay or \-w in sendmailanalyzer args. Default is 60 seconds.
During this time data are kept in memory to limit disk I/O and gain speed.
If you experience an out of memory on huge mail server adjust this value
to something smaller depending of your hardware configuration.
.SS "Reporting/display options"
.IX Subsection "Reporting/display options"
.IP "\s-1ERROR_CODE\s0" 4
.IX Item "ERROR_CODE"
Path to \s-1SMTP\s0 error code file (relative to \s-1CGI\s0 directory) where sa_report.cgi
is running. Default: lang/ERROR_CODE.
.IP "\s-1LANG\s0" 4
.IX Item "LANG"
Path to the translation file (relative to \s-1CGI\s0 directory) where sa_report.cgi
is running. Default: lang/en_US.
.IP "\s-1HTML_CHARSET\s0" 4
.IX Item "HTML_CHARSET"
Used to define the \s-1HTML\s0 charset to use. Default is iso\-8859\-1, but with cyrillics
character you have to use utf\-8 instead.
.IP "\s-1TTFONT\s0" 4
.IX Item "TTFONT"
True type font to use in graphs. When using Bulgarian langage for example,
GD::Graph is unable to draw Cyrillics font without a true type font. You can
also use it with other langage, this must be the full path to the ttf file.
For example:
.Sp
.Vb 1
\&        TTFONT  /usr/share/fonts/truetype/msttcorefonts/arial.ttf
.Ve
.Sp
to use Arial.
.IP "\s-1URL_LOGO\s0" 4
.IX Item "URL_LOGO"
Url to the barorng image. Default: /salogo.gif
.IP "\s-1TOP\s0" 4
.IX Item "TOP"
Number of object displayed in the top statistics. Default is 25.
.IP "\s-1TOP_MBOX\s0" 4
.IX Item "TOP_MBOX"
Number of object displayed in the top email addresses statistics.
Default is 25.
.IP "\s-1MAX_RCPT\s0" 4
.IX Item "MAX_RCPT"
Max number of recipients per message where senders will be reported.
Default 25 recipients max.
.IP "\s-1MAX_SIZE\s0	10000000" 4
.IX Item "MAX_SIZE	10000000"
Max size in bytes per message where senders will be reported.
Default is 10000000.
.IP "\s-1MAX_LINE\s0" 4
.IX Item "MAX_LINE"
Max lines to show in detail view. Default is 100.
.IP "\s-1SIZE_UNIT\s0" 4
.IX Item "SIZE_UNIT"
Size Unit to use, default is Bytes. Other values are KBytes and MBytes.
.IP "\s-1DOMAIN_REPORT\s0" 4
.IX Item "DOMAIN_REPORT"
Compute statistics and cache for a list of domain and display a link in the
front page for a per domain access. See \s-1DOMAIN_USER\s0 if you want to grant
special access on these pages. You can have multiple \s-1DOMAIN_REPORT\s0 lines.
If you are running rsyslog with multiple host use \s-1DOMAIN_HOST_REPORT\s0 instead.
Example:
.Sp
.Vb 1
\&        DOMAIN_REPORT   domain1.com,domain2.com
.Ve
.IP "\s-1DOMAIN_HOST_REPORT\s0" 4
.IX Item "DOMAIN_HOST_REPORT"
Compute statistics and cache for the given host followed by a list of domain
and display a link in the front page for a per domain access under each host.
You can have multiple \s-1DOMAIN_HOST_REPORT\s0 lines. See \s-1DOMAIN_USER\s0 if you want
to grant special access on these pages. For example:
.Sp
.Vb 2
\&        DOMAIN_HOST_REPORT      host1   domain1.com,domain2.com
\&        DOMAIN_HOST_REPORT      host2   domain2.com,domain3.com
.Ve
.IP "\s-1ANONYMIZE\s0" 4
.IX Item "ANONYMIZE"
This option allow the anonymization of the output, i\-e it remove any
sender/recipient personal information from the report.
.IP "\s-1REPLACE_HOST\s0" 4
.IX Item "REPLACE_HOST"
This option replace some hostname in all relay information for anonymization
You must used one \s-1REPLACE_HOST\s0 line per replacement.
.Sp
.Vb 1
\&        REPLACE_HOST    internal.relay.dom      external.relay.dom
.Ve
.IP "\s-1SPAM_VIEW\s0" 4
.IX Item "SPAM_VIEW"
Enable/Disable menu links to Spam views. Default show it: 1
.IP "\s-1VIRUS_VIEW\s0" 4
.IX Item "VIRUS_VIEW"
Enable/Disable menu links to Virus views. Default show it: 1
.IP "\s-1DSN_VIEW\s0" 4
.IX Item "DSN_VIEW"
Enable/Disable menu links to Notification views. Default show it: 1
.IP "\s-1POSTGREY_VIEW\s0" 4
.IX Item "POSTGREY_VIEW"
Enable/Disable menu links to Postgrey usage views. Default show it: 1
.IP "\s-1SHOW_DIRECTION\s0" 4
.IX Item "SHOW_DIRECTION"
Enable/Disable messaging/spam/virus/dsn direction statistics. Default is show.
On some mailhost this could show wrong information if the direction could
not be easily determined. So you can remove these views by setting it to 0.
.IP "\s-1SPAM_TOOLS\s0" 4
.IX Item "SPAM_TOOLS"
List of antispam name separated by a comma used for Spam details view. You may
want to custom this list to just show menu link on available reports. Default
list is:
.Sp
.Vb 1
\&        spamdmilter,jchkmail,dnsbl,spamassassin,amavis,mimedefang,dnsblmilter,spamd
.Ve
.Sp
Feel free to remove those you're not using to not see link to empty report in
the menu.
.SS "Maillog parsing options"
.IX Subsection "Maillog parsing options"
.IP "\s-1FULL\s0" 4
.IX Item "FULL"
Parse maillog from begining before running tail program. Can be overwritten
with \-\-full or \-f in sendmailanalyzer args. Default is 0, jump at the end
of log. Most of the time you may want to enable this to jump at the last parsed
line during the previous run.
.IP "\s-1BREAK\s0" 4
.IX Item "BREAK"
Do not run tail program and exit after a full parsing of the log file.
Can be overwritten with \-\-break or \-b in sendmailanalyzer args. Default
is 0, go ahead with tail.
.IP "\s-1MTA_NAME\s0" 4
.IX Item "MTA_NAME"
Syslog name of the \s-1MTA\s0. Syslog write it to maillog with the pid as
\&... sendmail[1234] ... This is required to only parse relevant lines.
Can be overwritten with \-\-sendmail or \-s in sendmailanalyzer args.
Default is sendmail, some distro come with sm-mta instead. Some other
have multiple names (ex: sm-mta, sendmail and sm-msp-queue) in this
case you can set the value of this directive to a pipe separated list
of values, for example: sm\-mta|sendmail|sm\-msp\-queue.
.Sp
Default: sm\-mta|sendmail|postfix
.IP "\s-1MAILSCAN_NAME\s0" 4
.IX Item "MAILSCAN_NAME"
Syslog name of MailScanner. Syslog write it to maillog with the pid as
\&... MailScanner[1234] ... This is required to only parse relevant lines
Can be overwritten with \-\-mailscanner or \-m in sendmailanalyzer args.
Default is MailScanner.
.IP "\s-1AMAVIS_NAME\s0" 4
.IX Item "AMAVIS_NAME"
Syslog name of Amavis. Syslog write it to maillog with the pid as
\&... amavis[1234] ... This is required to only parse relevant lines.
Default is amavis.
.IP "\s-1MD_NAME\s0" 4
.IX Item "MD_NAME"
Syslog name of MimeDefang. Syslog write it to maillog with the pid as
\&... mimedefang.pl[1234] ... This is required to only parse relevant lines
based on parsing mimedefang log generated by method \fImd_graphdefang_log()\fR
Default is mimedefang.pl.
.IP "\s-1CLAMD_NAME\s0" 4
.IX Item "CLAMD_NAME"
Syslog name of Clamd. When using Mailscanner with clamd if you want virus
report you must configure clamd to log with syslog and use \s-1LOG_MAIL\s0. Default
value is 'clamd' (... clamd[1234] ...)
Can be overwritten with \-\-clamd or \-n
.IP "\s-1POSTGREY_NAME\s0" 4
.IX Item "POSTGREY_NAME"
Syslog name of Postgrey. Syslog write it to maillog with the pid as follow:
\&... postgrey[1234] ... This is required to only parse relevant logged lines
Can be overwritten with \-\-postgrey or \-g. Default is postgrey
.IP "\s-1LOCAL_DOMAIN\s0" 4
.IX Item "LOCAL_DOMAIN"
Coma separated list of internal domain to be used when SendmailAnalyzer
is running on a mail host which received message from any side. \s-1SA\s0 can't
know what message are internal or external in this case, so the only way
to know if a mail come from Internet or Lan/Wan is to check the domain
part of the relay sender address. You can have multiple \s-1LOCAL_DOMAIN\s0 lines
for better reading.
.Sp
For example:
.Sp
.Vb 3
\&        LOCAL_DOMAIN    domain1.com,domain2.com,...
\&        LOCAL_DOMAIN    domain3.com
\&        LOCAL_DOMAIN    domain4.com
.Ve
.IP "\s-1LOCAL_HOST_DOMAIN\s0" 4
.IX Item "LOCAL_HOST_DOMAIN"
Same as above but with host distinction for use with rsyslog.
You can have multiple \s-1LOCAL_HOST_DOMAIN\s0 lines, ie: one per host.
.Sp
For example:
.Sp
.Vb 2
\&        LOCAL_HOST_DOMAIN   sysloghost1        domain1.com,domain2.com
\&        LOCAL_HOST_DOMAIN   sysloghost2        domain3.com,domain4.com
.Ve
.IP "\s-1MAIL_HUB\s0" 4
.IX Item "MAIL_HUB"
\&\s-1FQDN\s0 coma separated list of internal mail hubs, aka: where email are
redirected if the host is a gateway. For example: mailhost.mydom.dom
This directive is very important to help SendmailAnalyzer to find the
direction of incoming and outgoing message.
.IP "\s-1MAIL_GW\s0" 4
.IX Item "MAIL_GW"
\&\s-1FQDN\s0 coma separated list of \s-1MTA\s0 gateways where external mail comes from.
This directive is very important to help SendmailAnalyzer to find the
direction of incoming and outgoing message.
.IP "\s-1DEFAULT_DOMAIN\s0" 4
.IX Item "DEFAULT_DOMAIN"
Default domain or hostname to add to an email address if there's just the
username. When the host is a delivery system it is possible that the user
email address do not have the domain part (ex: \f(CW@domain\fR.com). By default
SendmailAnalyzer will add the '@localhost' domain but you may want to change
this domain, so use this directive
.IP "\s-1SPAM_DETAIL\s0" 4
.IX Item "SPAM_DETAIL"
This directive allow report for Spam details. Enable by default. This allow
you to see complete detail of your favorite antispam as well as score, cache
hit and autolearn if your antispam report it. To disable set it to 0, you
will save disk space.
.IP "\s-1SMTP_AUTH\s0" 4
.IX Item "SMTP_AUTH"
This directive allow report for \s-1SMTP\s0 authentication. Enable by default. This
allow you to see per authent type (server or client) user and relay statistics.
If you not use \s-1SMTP\s0 Auth set it to 0 to disable this feature. These stats are
not available in per domain views.
.IP "\s-1MERGING_HOST\s0" 4
.IX Item "MERGING_HOST"
Use this directive to combined multiple mailhost report on a single report.
This allow you to aggregate multiple mailhost that syslogs to a remote server
throught rsyslog to have only one SendmailAnalyzer report. The value must only
use alphanumeric character as it is used to create subdirectory.
.IP "\s-1SKIP_RCPT_RELAY\s0" 4
.IX Item "SKIP_RCPT_RELAY"
Use this to set the recipient relay used for local delivery if your message
appears twice in details view and in messaging, sender and recipient counter.
This is especially right when with postfix configuraed to have local delivery
via dovecot service. Default: dovecot, that mean that recipient log lines with
relay=dovecot will instruct sendmailanalyzer to skip those messages.
.SS "Domain / user views options"
.IX Subsection "Domain / user views options"
.IP "\s-1LOW_LIMIT\s0, \s-1MEDIUM_LIMIT\s0, \s-1HIGH_LIMIT\s0 (\s-1NO\s0 \s-1MORE\s0 \s-1USED\s0)" 4
.IX Item "LOW_LIMIT, MEDIUM_LIMIT, HIGH_LIMIT (NO MORE USED)"
User messaging data limit in megabytes to show/warn the level of mail activity.
\&\s-1LOW_LIMIT\s0 (3 by default), mail activity under this limit is shown as green.
\&\s-1MEDIUM_LIMIT\s0 (5 by default), mail activity under this limit is shown as orange.
\&\s-1HIGH_LIMIT\s0 (10 by default), mail activity under this limit is shown as red.
above the hight limit the user is warn for abuse. Set all to 0 if you want to
disable this feature.
.IP "\s-1ADMIN\s0" 4
.IX Item "ADMIN"
List of admins username separated by coma that must have full access to all
report. The username is checked again the http \s-1REMOTE_USER\s0 environment variable.
Default is every one can access, in this case you may want to add a .htaccess.
.IP "\s-1DOMAIN_USER\s0" 4
.IX Item "DOMAIN_USER"
List of per user domain access control. The first field is the username and
the second field (separated by tabulation) is a coma separated list of domain
name to be allowed to this user. You could add as many lines of \s-1DOMAIN_USER\s0
as you want in the configuration file.
.SH "ACCESS CONTROL"
.IX Header "ACCESS CONTROL"
Access control is based on the \s-1REMOTE_USER\s0 environment variable stored by the
httpd server during an htaccess Authentication. If this variable is not set,
there is full access for every one.
.SH "AUTHOR"
.IX Header "AUTHOR"
Gilles Darold <gilles \f(CW@nospam\fR@ darold.net>
.SH "COPYRIGHT"
.IX Header "COPYRIGHT"
Copyright (c) 2002\-2013 Gilles Darold \- All rights reserved.
.PP
.Vb 4
\&        This program is free software: you can redistribute it and/or modify
\&        it under the terms of the GNU General Public License as published by
\&        the Free Software Foundation, either version 3 of the License, or
\&        any later version.
\&
\&        This program is distributed in the hope that it will be useful,
\&        but WITHOUT ANY WARRANTY; without even the implied warranty of
\&        MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
\&        GNU General Public License for more details.
\&
\&        You should have received a copy of the GNU General Public License
\&        along with this program.  If not, see < http://www.gnu.org/licenses/ >.
.Ve
.SH "BUGS"
.IX Header "BUGS"
Your volontee to help construct a better software by submitting bug report or
feature request as well as code contribution are welcome.
.SH "ACKNOWLEDGEMENT"
.IX Header "ACKNOWLEDGEMENT"
Thank to Sendmail.org for the kind permission to use the \*(L"Bat\*(R" logo.