Home / server / Questions / 411988
Accepted
java_pill
Asked: 2012-07-28 13:44:53 +0800 CST

Postfix, read/process email content

  • 772

Here's a use case for my application. All incoming emails to different email addresses have to be processed by reading the content of email messages, along with the information such as: from email address, subject line and date/time sent. This information later has to be HTTP POSTed to a web service API or stored in a DB.

We are thinking about using PostFix as our MTA software. Using PostFix, is it possible to implement such a process to read emails? If yes please explain how? Or are there any programs that are readily available to read emails and store in a DB from PostFix?

Thanks!

email

2 Answers

  1. Best Answer

    No, it is not possible with Postfix itself. As you said it is a MTA where T stands for transport. It is not a mail processing agent. But you can configure different MDAs (mail delivery agents).

    You can do this with procmail for example or any other program that accepts mail via STDIN. These programs can be included into Postfix with the pipe configuration options. See the manual for details and instructions.

    • 3

  2. Yes postfix email server has the notion of a content_filter

    cat /etc/postfix/master.cf

    to which I added

    filter  unix  -  n  n  -  -  pipe
    flags=Rq  user=ehookfilter  argv=/usr/local/bin/content_filter.sh
    -f ${sender}  --  ${recipient}

    So each incoming email received by postfix server will execute my script ( my app is b2b so low traffic this can be converted to a daemon if need be )

    /usr/local/bin/content_filter.sh

    which is loosely based on http://www.postfix.org/FILTER_README.html

    above shell content_filter.sh in turn executes my below referenced golang code smtp_filter which does the heavy lifting of parsing email headers + body ... then I have my go code iterate across all attachments ... it persists by making REST calls into our in house workflow

    /usr/local/bin/smtp_filter ${curr_output_file_prefix} < in.$$ >> $curr_amalgamated_file

    to gain appreciation of what my 900 lines of golang does here are the libraries it pulls in

    import (
    "bufio"
    "bytes"
    "fmt"
    "io"
    "io/ioutil"
    "log"
    "net/mail"
    "os"
    "strconv"
    "strings"
    "time"

    "context"
    "crypto/tls"
    "crypto/x509"
    "github.com/google/go-github/github"
    "golang.org/x/oauth2"
    "net/http"
    "regexp"

    "github.com/jaytaylor/html2text"
    "mime"
    "mime/multipart"
    "mime/quotedprintable"
    "encoding/json"
    "encoding/base64"
)

    needless to say this is not cookie cutter yet I had no prior expertise on SMTP ... I am thinking of posting a full writeup of this solution if demands warrants ... a big benefit of writing code to read email is this shunts to ground incoming email malware vectors as we no longer had to read email ;-)

    • 0