pop-before-smtp in C

Written by Kevin L. Ellis

Download current release (version 1.0)


            So you want to allow your users to send e-mail through smtp but you don't want to become a spam haven?  What do you do?  Setup a pop-before-smtp daemon of course!  Now only users who can authenticate through pop can send e-mail through your mail server.  This is what I wanted to do, and since I'm using Postfix I looked on their website for a solution.  I found one, the pop-before-smtp perl script.  It looked easy enough, at least the instructions for setting it up were.  But alas it didn't work out that way.  The first problem I ran into was my perl was using a different version of Berkeley DB then what Postfix was using.  I finally solved that problem by downloading the Berkeley DB library so that I could compile up a version of Postfix that would "talk" the same DB format as Perl was doing.  This annoyed me because now any time I wanted to upgrade Postfix I'd have to remember that I needed to set it up to use some other version of the DB library.  But that still didn't solve my problems.  They were both speaking the same DB format, but for some reason the pop-before-smtp perl script wasn't putting the pop ip addresses into the database.  I isolated the problem down to the "findNetblock($ipaddr)" portion of the script.  For some reason it thought any ip address that came in was part of the mynetworks config for Postfix - but this wasn't the case.  Commenting out that line solved the problem, but once again it was a hack.  If I ever upgraded the pop-before-smtp perl script I'd have to go back in and fix it up again.  Not something I was wanting to do.  After looking at the code and seeing how simple it was I decided I'd just roll my own in C since I really don't speak perl.  Thus was born pop-before-smtp in C!

            In writing this program I tried to keep close to the Unix belief of re-using already existing programs.  So to monitor the log file I use tail and pipe the output to my program.  Simple, don't have to write any code for tracking the log file!  Creating the database was the easiest part.  Afterall, Postfix has a perfectly good program to create it, postmap .  And by using the postmap program which is compiled when you build postfix, you never have a problem of Postfix and the pop-before-smtp daemon speaking different database formats.  It would have been nice if the perl script did this to begin with.
 

Setting up pop-before-smtp in C
 
            First download and modify the file pop-before-smtp.c by changing these five defines at the beginning
 

#define GRACE                    (60 * 30)
#define NUM_ENTRIES              250
#define POP_BEFORE_SMTP_DIR      "/etc/postfix"
#define POP_LOG                  "/var/log/maillog"

/* define which pop3 deamon you are using */
/* VMPOP3D  QPOPPER IPOP3D TEAPOP VDPOP3D */
/* COURIER TPOP3D CYRUS */
#define VMPOP3D

            GRACE is the number of seconds that an ip address will stay authenticated so that the user can send e-mail.  Right now it's setup to give the user a grace period of 30 minutes.  Adjust this to your liking.  NUM_ENTRIES is the maximum number of authenticated ip addresses that pop-before-smtp in C will keep track of.   POP_BEFORE_SMTP_DIR should point to the directory where you want to store the text version of the ip addresses that postmap will convert into database form.  Since Postfix stores it's config files in /etc/postfix, this is the default setting.  POP_LOG should point to the log file that your pop3 daemon writes to.  Since I'm running Slackware and using vm-pop3d, the file is called maillog in /var/log - adjust this for your particular system and pop3 daemon.

            The last define tells the program which matching function to use.  Currently there are three matching functions.  As people submit more matching functions I'll add them to the code so all you have to do is define which pop3 daemon you are using.  The code comes defaulted to use vm-pop3d.  Change the #define to the pop3 daemon you are using.

            Now comes the tricky part, the function match might need to be modified for your particular pop3 daemon if your pop3 daemon isn't currently supported.  Each pop3 daemon outputs a different format in the log file and pop-before-smtp in C needs to know how to read this format to find the ip address of the user that just authenticated.  I use vm-pop3d and have modified it so that it will output the users ip address when there is a successful authentication.  vm-pop3d normally outputs this to the log:

Mar 15 14:04:32 www vm-pop3d[8647]: connect from kevin@10.0.0.2
Mar 15 14:04:32 www vm-pop3d[8647]: Connect from 10.0.0.2
Mar 15 14:04:32 www vm-pop3d[8647]: User 'kevin' of 'bluelavalamp.net' logged in

            I modified it to output this:

Mar 15 14:04:32 www vm-pop3d[8647]: connect from kevin@10.0.0.2
Mar 15 14:04:32 www vm-pop3d[8647]: Connect from 10.0.0.2
Mar 15 14:04:32 www vm-pop3d[8647]: User 'kevin' of 'bluelavalamp.net' logged in from 10.0.0.2

            I'm not sure why the developers didn't output the ip address on the last line.  So, you might have to code up a match function for the pop3 daemon your using.  If you aren't sure how to do this send me a sample of your log when a user authenticates by pop3 and I can code one up for you, it'll be pretty easy.  Also, if you modify the match function to work with your pop3 daemon, let me know, I'll roll it into the code so that others can use it!

            Once you've done all that go ahead and run ./compile   Now you'll have the daemon compiled up and you can start testing it.  Start it up in debug mode first, so execute './pop-before-smtp -d '.  The daemon will output info but not actually create any database for that information.  What you should do is monitor the log file your pop3 daemon writes to (use 'tail -f') and see what pop-before-smtp in C writes out.  As users authenticate and get written to the log file pop-before-smtp will write out info telling you what ip addresses would be in the database.  For example, when you first start it you might get:

pop-before-smtp in C   version 0.3
Performing debug mode
Starting command in 3 seconds: tail -f /var/log/messages
Postmap command: postmap -r /etc/postfix/pop-before-smtp
Starting command in 3 seconds: tail -f /var/log/maillog
cur_time: 1027023217   oldest_entry_time: 1027025017
Alarm set for 1800 seconds
x= 0  10.0.0.2 OK
head=1 tail=0 1016221164 from 10.0.0.2

cur_time: 1027023217   oldest_entry_time: 1027025017
Alarm set for 1800 seconds

x= 0  10.0.0.2 OK
x= 1  10.5.0.2 OK
head=2 tail=0 1016221164 from 10.5.0.2

            The pop-before-smtp daemon found two ip addresses to add to the database.   After running for a while pop-before-smtp might print out:

purging: 1016220866  10.5.0.2
purging: 1016220866  10.0.0.3
purging: 1016220872  10.0.0.10
purging: 1016220873  10.0.0.16
purging: 1016220874  10.0.0.4
x= 5  10.5.0.12 OK
x= 6  10.5.0.10 OK
x= 7  10.0.0.2 OK
head=8 tail=5 1016221109 from 10.0.0.2

            Right now there are three addresses that would be in the database and allowed to smtp, there were five ip addresses that have expired (past their GRACE period) and have been removed.

            If it looks like everything is working correctly you can startup the daemon in the background.  Place a copy where ever you want (/usr/local/bin might be good) and make the appropriate changes to rc.d so that it'll start up each time the computer reboots. 


Features

            pop-before-smtp in C will check the list of authenticated POP3 users for old entries every time a new user authenticates.  In addition, the program also uses the alarm function to wake up and purge the oldest entries in the list.  This prevents IP addresses from staying in the system longer then the grace period.

            pop-before-smtp in C also supports dynamic restarting after log rotation.  If you are running a log rotator on your system, just send a SIGHUP signal to pop-before-smtp in C after the log rotator is run.  The program will kill off the currently executing tail process and fork off a new one for the new log.  The signal can be sent with killall, like so: killall -HUP pop-before-smtp

Using pop-before-smtp in C with Postfix

            This is the easy part.  The default-UCE-Restrictions for Postfix are:

smtpd_recipient_restrictions =
   permit_mynetworks,
   check_relay_domains

            Just change that to:

smtpd_recipient_restrictions =
   permit_mynetworks,
   check_client_access hash:/etc/postfix/pop-before-smtp,
   check_relay_domains
 


Using pop-before-smtp in C with vm-pop3d  

            In order to use pop-before-smtp in C with vm-pop3d there are a couple of changes that need to be made to vm-pop3d.  As stated above vm-pop3d needs to output the ip address in the log file.  The two files that need to modified are user.c and vm-pop3d.c.  To make it easy for people I've made up a "patch" file that has the necessary changes.  Just download vm-pop3d.patch.zip and unzip the file in the source directory for vm-pop3d (NOTE: these changes are for version 1.1.6 of vm-pop3d).  This will expand two files:

user.c.changes
vm-pop3d.c.changes


            If you want to see what changes I made you can run diff:

diff user.c user.c.changes
diff vm-pop3d.c vm-pop3d.c.changes


             Now just mv the .changes files to the .c file names:

mv user.c.changes user.c

mv vm-pop3d.c.changes vm-pop3d.c


             recompile, install, and you should be all set!



Using pop-before-smtp in C with qpopper

            Special thanks is given to Anton Krall for submitting this information on how to use the program with qpopper.  Anton can be e-mailed at akrall@team.inter.net or netlord@axtel.net.

            Anton writes:

        For modifying qpopper for this, we need first to modify the file pop_init.c in order to make qpopper have some kind of string we can check using the match function:

        Look for this string in pop_init.c:

#ifdef    LOG_LOGIN
     p->pLog_login = "(v%0) POP login by user \"%1\" at (%2) %3";
#endif /* LOG_LOGIN */

  
        And modify adding some () like this:

#ifdef    LOG_LOGIN
     p->pLog_login = "(v%0) POP login by user \"%1\" at ((%2)) %3";
#endif /* LOG_LOGIN */

  
        This is so that we can then later tell pop-before-smtp to look for the ending )) above.

         Then, recompile qpopper with the following parameter: --enable-log-login

         This is so that qpopper will create this kind of entries in  /var/log/maillog:

Apr  1 08:43:21 team qpopper[21802]: (v4.0.3) POP login by user "akrall"  at ((10.0.1.1)) 10.0.1.1
Apr  1 08:43:24 team qpopper[21802]: Stats: akrall 0 0 1 2127 10.0.1.1  10.0.1.1

         Then we need to modify pop-before-smtp to look for these changes:

inline char* match(char *ptr)
{
    char *str_ptr;

    if (strstr(ptr, "qpopper") != NULL ) {
        str_ptr = strstr(ptr, "))");

        if (str_ptr != NULL) {
            str_ptr += strlen(")) ");

            return str_ptr;
        }
    }
    return NULL;
}

        Don't forget to let pop-before-smtp know where your qpopper log is:

#define POP_LOG "/var/log/maillog"


    

            The last part about modifying the match function has already been taken care off.  All you need to do is change the fifth #define to QPOPPER and it'll use that match function.



Using pop-before-smtp in C with ipop3d

            This one's easy, just set the last
#define to IPOP3D and you'll be all set.  




Using pop-before-smtp in C with teapop

            Special thanks is given to Skip Watson for submitting the match function for teapop.  Just change the fifth define to:

#define TEAPOP




Using pop-before-smtp in C with vdpop3d

            Special thanks is given to Blake Watters for submitting the match function for vdpop3d.  Just change the fifth define to:

#define VDPOP3D


Using pop-before-smtp in C with Courier

            Special thanks is given to Lang, Andreas Grundler, and Scott Bronson for submitting a log file entry (Lang) and code for the different formats of Courier's log entries (Andreas & Scott).   The Courier match function should work with both imapd and pop3d, with and without ssl support.  It turned out that Courier has at least two different log formats:

Jul 18 15:25:58 [imapd] Connection, ip=[::ffff:64.130.203.13]
Jul 18 15:25:58 [imapd] LOGIN, user=lang, ip=[::ffff:64.130.203.13]
Jul 18 15:26:19 [imapd] DISCONNECTED, user=lang, ip=[::ffff:64.130.203.13], headers=0, body=1011

Jul 24 14:52:24 divino imapd-ssl: Connection, ip=[68.6.112.98]
Jul 24 14:52:24 divino imapd-ssl: LOGIN, user=bronson, ip=[68.6.112.98]

            The current match code is a combination of those submitted to match both log formats.  If you are using Courier and the function doesn't work with your log entries please e-mail me any changes you make to the match function or a copy of the log entries so that the function can be adjusted to work with all the log formats.  To use the Courier match function just change the fifth define to:

#define COURIER_IMAP


Using pop-before-smtp in C with tpop3d

            Special thanks is given to Alessandro Gatti for submitting the match function for tpop3d.  Just change the fifth define to:

#define TPOP3D



Using pop-before-smtp in C with cyrus

            Special thanks is given to Laurens van Alphen for submitting the match function for cyrus.  Just change the fifth define to:

#define CYRUS





Optimizations

            Normally when a user authenticates the daemon would write out a text file containing all the ip addresses that are currently authenticated and then run
postmap to create the database for postfix to use to determine if a user is allowed to send e-mail.  This meant that every time a user authenticated via pop3 the system performed two harddrive writes and one process creation.  If you had a system with lots of users who check their e-mail regularly then this could occur quite often.  

            Starting with version 0.4 pop-before-smtp in C has some optimizations that you can turn on that will cut down this number considerably.  By uncommenting the
#define OPTIMIZE then the optimized code will be compiled in.  The optimized code works like so - a user signs in via pop3, the array of authorized ip addresses is scanned from the newest ip address to the oldest ip address (the number of entries scanned is determined by the #define SCAN_DEPTH setting).  If the ip address is found then the daemon does nothing, no new file is written and postmap is not run.  Also when an ip address is removed from the array because it's grace period has expired, the list is scanned again from the newest to the oldest.  If there's an entry in the array newer then the daemon doesn't have to create the file or run postmap .  Depending upon the number of users of the system and how frequently they authenticate by pop3 this can cut down on the number of harddrive accesses considerably.




            If you're using pop-before-smtp in C or have changed some code to use a certain pop3 daemon I'd like to know!  Send me an e-mail and let me know what you think.  If you have a
match function for your pop3 daemon you'll get credit for your code.