I'm looking to replace our current use of an external smtp provider with an in-house installation. It should handle the following situations:
Certain addresses should be forwarded to gmail addresses; we'd like to continue to use gmail as our primary email interface.
Other addresses should be available as pop/imap mailboxes.
Other addresses will be handled programmatically: they'll kick off various tasks, get logged, etc. These addresses should either kick off a process to the handle the email addresses, or should store in an easily processable format.
It should also be fairly easily configurable with domain keys, spf, and anything else it takes to allow email deliverability.
I've used sendmail in the distant past. It seems postfix and exim are the recommended options these days. My main question is: what's the best choice and setup for the forwarding of the addresses to gmail, and for the programmatic access? Is procmail still the way to go or are there better options these days?
We're using linux/ubuntu servers.
Postfix is solid, well-supported and easy to configure. All the stuff you describe is fairly routine.
for the programmatic stuff, you have to be aware of what UID the programs/scripts will be run as.
For postfix, for example, scripts run from /etc/aliases (or any other root-owned aliases file) will be run as the 'nobody' user.
postfix has the ability to have multiple aliases files and (here's the good part) programs run from an aliases file will be run as the user who owns that file (except for root - as mentioned before, aliases files owned by root are run as nobody).
if you ever need to run a script from mail as root, then the easiest way (if you have sudo installed) is to configure sudo to allow 'nobody' to run it as root without a password and run it from a wrapper script that runs it with sudo.
NOTE: whenever triggering scripts from mail, you have to remember that the data (i.e. the mail) is from an untrusted source (the internet)....so it is extremely unsafe to use the data from either the body or the header as arguments to commands executed by the script. this is basically the same issue as trusting user-supplied data in a CGI script - just don't do it. e.g. consider what a badly written script could do if the user-supplied data was something like " ... innocuousdatahere ...; rm -rf /". you might think it would be easy to check for obvious things like semi-colons and strip them out, but there's a lot of non-obvious things you'd need to check for as well and you'd just be setting yourself up for a failure since you're unlikely to think of or catch them all. far safer to eliminate the security risk entirely by just not trusting user supplied data ever.
this is important no matter what UID the script will run as but is especially important when/if it is being run as root.
Finally, to answer one of your direct questions, procmail is still a useful tool for what you want to do. as is maildrop and other similar programs. think of procmail etc as specialised tools for parsing body and headers to make decisions about what to do, so you don't have to have all that parsing code in your scripts. write your script to be just do its job, and use procmail to decide when the script should be run. for example, you could create a new user called, say, 'mailjobs' which has its own home dir and .procmailrc file. then create aliases that forward mail to that user. mailjobs' .procmailrc would then check the X-Original-To header to decide which script(s) to run.
Both exim and postfix are more than capable of doing what you need, best advice is to pick the one that's easiest for you to use and/or most familiar with.
As for dealing with forwards to gmail and into scripts best way to do this is with a global aliases file (usually kept in /etc/aliases or /etc/mail/aliases).