Web Hosting Talk : Web Hosting Main Forums : Hosting Security and Technology : Hosting Security and Technology Tutorials : exim + cpanel + dspam howto

[brettcave]

I found a handful of howto's for dspam, but none of them catered for exim with virtual users. After hunting around, I eventually have it working on a cPanel server, with user authentication for mail users.

My setup:
dspam version 3.6.8, using mysql driver.
exim version 4.
mysql 4.1
CentOS (2.6.9-023stab033.9-enterprise)
cPanel / WHM - latest RELEASE version.

Download the source, configure and compile:

# cd /usr/local/src
# wget http://dspam.nuclearelephant.com/sou...m-3.6.8.tar.gz
# tar -zxf dspam-3.6.8.tar.gz
# cd dspam-3.6.8

Configure, replacing user/groups with your web-server user (web / apache / nobody), and use your mysql-include / library paths (will need mysql-devel on rh based systems).

# ./configure --prefix=/opt/dspam-3.6.8 --with-local-delivery-agent=/usr/sbin/exim --with-storage-driver=mysql_drv --with-userdir=/var/spool/mail/dspam --with-userdir-owner=nobody --with-userdir-group=nobody --with-dspam-mode=none --with-dspam-owner=nobody --with-dspam-group=nobody --enable-whitelist --enable-spam-delivery --enable-alternative-bayesian --disable-dependency-tracking --enable-virtual-users --with-mysql-includes=/usr/include/mysql --with-mysql-libraries=/usr/lib/mysql/ --with-dspam-home=/opt/dspam-3.6.8/var/dspam
# make && make install

Set up mysql

# mysqladmin -p create dspamdb
# mysql -p
>grant all privileges on dspamdb.* to dspamuser@localhost identified by dspampass;
>flush privileges;
>exit;

Create tables:

mysql -p dspamdb < /usr/local/src/dspam-3.6.8/src/tools.mysql_drv/mysql_objects_speed.sql
mysql -p dspamdb < /usr/local/src/dspam-3.6.8/src/tools.mysql_drv/virtual_users.sql

Link dspam in opt for easy versioning:

ln -s dspam-3.6.8 /opt/dspam

Copy the web interface files to a web directory:

# cp webui/cgi-bin /opt/dspam -r
# cp webui/htdocs /opt/dspam/

This next step is required for pop3 authentication.
Install perl module Apache::AuthPOP3 - which does apache pop3 authorisation:

perl -MCPAN -e shell
install Apache::AuthPOP3

Next, apache will need mod_perl installed - WHM -> Apache Update will allow you to enable the perl module (I am running it alongside php with no issues).

Then in /usr/local/apache/conf/httpd.conf:

ScriptAlias /dspam/ /opt/dspam/cgi-bin/
   Alias /dspam_files/ /opt/dspam/htdocs/
   <Directory /opt/dspam/cgi-bin>
         Options None
         AllowOverride AuthConfig
         Order allow,deny
         Allow from all
   </Directory>

Create .htaccess in /opt/dspam/cgi-bin as follows:

AuthName "Dspam"
 AuthType Basic
 PerlAuthenHandler Apache::AuthPOP3
 PerlSetVar        MailHost localhost
 Require valid-user
 #PerlSetVar        UserMap pop3user1=>realname1,pop3user2=>realname2
 #Require user      pop3user1 pop3user2 pop3user3 pop3user4

there are 2 commented parameters you can set when using POP3 auth - sure its pretty self-explanatory.

Set up admin user (the admin_user must be able to authenticate as a pop user):

#echo "admin_user" >> /opt/dspam/cgi-bin/admins

Create a queuesize script for web user - so dspam can determine how many messages in the queue.

vi /usr/local/bin/eximqsize
#!/bin/sh

/usr/bin/find /var/spool/exim/input/ -type f | wc -l | cut -d" " -f1-
##EOF

# chmod 4755 /usr/local/bin/eximqsize
# chown nobody /usr/local/bin/eximqsize

Configure web ui, edit /opt/dspam/cgi-bin/configure.pl:

$CONFIG{'MAIL_QUEUE'}   = "/usr/local/bin/eximqsize";
$CONFIG{'WEB_ROOT'}     = "/dspam_files";
$CONFIG{'LOCAL_DOMAIN'} = "FQDN"; #your servers fully qualified domain name - e.g. host.yourdomain.com

Next, set the default preferences for the system (you need /opt/dspam/bin in your path if you copy and paste this...):

     dspam_admin ch pref default trainingMode TEFT
     dspam_admin ch pref default spamAction quarantine
     dspam_admin ch pref default spamSubject "[SPAM]"
     dspam_admin ch pref default enableWhitelist on
     dspam_admin ch pref default showFactors off

Permissions:
I would suggest reading the README over dspam to get a full understanding of the permissions required for running of dspam. My permissions were:

# chown nobody:mail /opt/dspam/var/dspam -R
# chown nobody:mail /opt/dspam/etc/ -R

Edit dspam.conf (in /opt/dspam/etc/. I have only listed the parameters I changed here...):

TrustedDeliveryAgent "/usr/sbin/exim -oMr spam-scanned"

Trust: root
Trust: mail
Trust: nobody / httpd   #choose 1 - what ever your webserver runs as - `ps axu | grep httpd` to find out

#Use the same details as you did for the "grant all privileges on...." statement in mysql.
MySQLServer             /var/lib/mysql/mysql.sock
MySQLPort
MySQLUser               dspamuser
MySQLPass               dspampass
MySQLDb                  dspamdb
MySQLCompress          true

MySQLVirtualTable          dspam_virtual_uids
MySQLVirtualUIDField       uid
MySQLVirtualUsernameField  username

Almost there....
Confirm that mysql is configure to listen on a socket in /etc/my.cnf (or whereever your config file is):

# cat /etc/my.cnf
[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock

Now the final step - exim configuration. This is the part that took the longest, hopefully it works for you. Just as I read in the howto's I used for this, please please please dont just copy and paste - you stand a good chance of breaking your mail server if you make changes without understanding. Be warned.

My config file is /etc/exim.conf. This should be edited using the WHM -> Exim Configuration Editor -> Advanced.

#Routers - Add these in the box before virtual_user delivery / user delivery router).
dspam_router:
   no_verify
   #uncomment the next line to disable dspam for virtual users.
   # check_local_user        
   condition = "${if and { \
                             {!def:h_X-Spam-Flag:} \
                             {!def:h_X-FILTER-DSPAM:} \
                             {!eq {$sender_address_domain}{$domain}} \
                             {!eq {$received_protocol}{local}} \
                             {!eq {$received_protocol}{spam-scanned}} \ 
                       } }"
   headers_add = "X-FILTER-DSPAM: by $primary_hostname on $tod_full"
   driver = accept
   transport = dspam_spamcheck

## The next 2 routers allow you to forward spam / non-spam to dspam for training (e.g. spam-yourmail@yourdomain.net).
# spam-username
dspam_addspam_router:
   driver = accept
   local_part_prefix = spam-
   transport = dspam_addspam

# nospam-username
dspam_falsepositive_router:
   driver = accept
   local_part_prefix = notspam-
   transport = dspam_falsepositive


##Transports - can be added anywhere:
#this adds the spam-scanned protocol header, so when it is passed back to exim after being processed by dspam, it doesnt get stuck in a loop.
dspam_spamcheck:
  driver = pipe
  command = "/usr/sbin/exim -oMr spam-scanned -bS"
  transport_filter = "/opt/dspam/bin/dspam --stdout --deliver=innocent,spam --user $local_part@$domain"
  use_bsmtp = true
  home_directory = "/tmp"
  current_directory = "/tmp"
  user = nobody
  group = mail
  log_output = true
  return_fail_output = true
  return_path_add = false
  message_prefix =
  message_suffix =

dspam_addspam:
  driver = pipe
  command = "/opt/dspam/bin/dspam --user $local_part@$domain --class=spam --source=error"
  home_directory = "/tmp"
  current_directory = "/tmp"
  user = nobody
  group = mail
  log_output = true
  return_fail_output = true
  return_path_add = false
  message_prefix =
  message_suffix =

dspam_falsepositive:
  driver = pipe
  command = "/opt/dspam/bin/dspam --user $local_part@$domain --class=innocent --source=error"
  home_directory = "/tmp"
  current_directory = "/tmp"
  user = nobody
  group = mail
  log_output = true
  return_fail_output = true
  return_path_add = false
  message_prefix =
  message_suffix =

If you have set up authentication correctly as well, then you should be able to open http://yourhost/dspam/dspam.cgi[/url] and log in - if you add your login details to the "admins" file, you can configure defaults, etc. It also allows ALL users (with 1 user being an email account) to log in, using www.yourclientsdomain.com/dspam/dspam.cgi

This will not work with suexec enabled!! This is because dspam needs specific permissions, and it is expecting user nobody to access it. If suexec is enabled, you will need to use the default host, and NOT virtual hosts (and even this may not work - testing still required).

Watch exim_mainlog after this - you should pick up what transports and routers are being used.

Dspam can really hammer a system - mysql, cpu and memory usage will go up a bit, especially on busy production servers. Monitor your servers performance.

Other settings: add /opt/dspam/man to MANPATH in /etc/man.config or move dspam man directory to an existing man directory.


[ADDED]
This dspam.cgi hack will do a lookup in the cpanel config file to find the domain for any username without a domain, and append it on match (or leave just the username part if nothing is found). This requires unsecuring your system a bit - your http user will need to be able to read /etc/trueuserdomains (either chmod 644 or chown nobody):

#add this just after $CURRENT_USER is set.
if ($CURRENT_USER !~ /\@.+\./) {
        open(TUD, "</etc/trueuserdomains");
        while(<TUD>) {
                my ($domain, $user) = split(/:/,$_);
                chomp($user);
                $user =~ s/^\s*//g;
                if ($user eq $CURRENT_USER) {
                        $CURRENT_USER = $CURRENT_USER . "\@$domain";
                }
        }
        close(TUD);
}

That should do it

dspam will allow all messages through by default, and will require some training. With this config, users can train using email commands - all they need to do is forward any spam that hits their mailbox to spam-emailaddress@domain.com (their own email address with spam- prepended). Unfortunately this does not allow handling of false positives if you are using a "quarantine" policy instead of subject. the web interface comes in handy for this.

I am busy testing a combination of dspam with assp, which seems to be working well - I especially like the greylisting feature of assp and ProtectionBox... Will add to this howto when testing is finish.

[ResellerPlanet]

Thank you for the HowTo. I bookmarked it and will read it later

Btw, should have posted it in the tutorials subforum though

[brettcave]

Thanks for that. Any forum moderators reading this? pls move to tutorials.

[writespeak]

Originally Posted by brettcave

Any forum moderators reading this? pls move to tutorials.

Moved.

Lois

[brettcave]

ta

Brett

[brettcave]

Picked up a possible problem with regards to the router accepting mail and filling up the queue.

Still working on the router, but this is what I have updated it to.

dspam_router:
   no_verify
   # check_local_user
   condition = "${if and { \
                                     #{ or { \
                                            {eq {${perl{check_deliver}{$domain}{$local_part}}}{yes}} \
                                            #{eq {${lookup{$local_part@$domain}lsearch{/etc/valiases/$domain}{yes}{no}}}{yes}} \
                                     #} } \
                                     {!def:h_X-Spam-Flag:} \
                                     {!def:h_X-FILTER-DSPAM:} \
                                     {!eq {$sender_address_domain}{$domain}} \
                                     {!eq {$received_protocol}{local}} \
                                     {!eq {$received_protocol}{spam-scanned}} \
                       } }"
   headers_add = "X-FILTER-DSPAM: by $primary_hostname on $tod_full"
   driver = accept
   transport = dspam_spamcheck

I have commented out a check that i was writing to validate whether there was a virtual alias for the recipient, though from watching the logs, the dspam router is being called for virtual users and aliases.