We have a Rails app used for support & help tickets.
It sends customers a confirmation email when they submit a request. It also sends an email when we reply to their request. It does not receive mail.
Within the last week, 25%+ of our customers stopped getting their replies. They think we aren't replying to their ticket (when we actually are).
I did a test with my @yahoo.com account and found this in the mail.log file:
Jul 9 16:11:45 bighelp postfix/smtp[9051]: BF673324365: host b.mx.mail.yahoo.com[66.196.97.250] said: 451 Message temporarily deferred - [140] (in reply to end of DATA command)
Jul 9 16:11:45 bighelp postfix/smtp[9051]: BF673324365: to=<[email protected]>, relay=d.mx.mail.yahoo.com[68.142.202.247]:25, delay=0.73, delays=0.02/0.02/0.64/0.05, dsn=4.0.$
Other non-yahoo addresses are experiencing this as well.
The Rails app is running on Ubuntu and we are sending via:
ActionMailer::Base.delivery_method = :sendmail
I made sure the server isn't an open relay.
What else can I do to ensure most of our emails get through?
This message is often to do with greylisting (particuarly with yahoo who seem to do it to everyone at some points). Basically the mail server temporarily defers your first mail, a legitimate email server will see this message, wait a period and then try again. Servers that are sending spam are often set to just send out bulk loads of mail and forget about, it so will never try again, and so the spam is dropped.
To ensure you can get through the greylisting ensure your mail server is set to try again after a deferral, and give it a reasonable time windows, 10-20 mins is usually enough.
Getting tempfailed by some other mail server isn't entirely unusual; you should expect this to happen from time to time.
I imagine that most email providers (disclaimer: I only work for ONE) will tempfail things more often for "bad reputation" senders. We do this to selectively prioritise resources for clean(er) mail. If your server has managed to get its IP address a bad reputation, that probably means that messages are being classified as spam either sent by your application, or something else from the same IP address.
You should DEFINITELY monitor your mail queues. You should possibly put some auditing on individual deliveries so you can track them.
If you see large numbers of messages queued for delivery, then that indicates a fault of some kind, either on the recipient's mail server, or that they're blacklisting / de-prioritising your mail in some way.
Exactly how you do these things are application-specific.
In this particular case you should contact Yahoo's support, assuming you think you're definitely not sending them spam (nor is anyone else from the same IP).
You seem to have two issues here:
WRT #1, I would do the following checks:
WRT #2, I would suggest including a transparent gif, or a specially marked logo in your email which will 'phone home', back to your server. Yes, this means you need to send HTML email, and yes, some customers will block emails from retrieving images by default, however, you'll quickly see your normal response rates, and be able to detect if that has dropped. If you have high value customers, and you can see they may not have received your response, you can proactively call them.
That looks like a temporary failure. postfix should retry it again after some interval. What's the configuration for postfix look like? Does the message end up in postfix's mail queue? Do you have postfix configured to periodically check for queued messages?
You should also make sure that your mail server is identifying itself correctly (as mail.yourapp.com or whatever) and that a PTR record exists linking that IP to that name.
Furthermore, you can add in an SPF record allowing that IP/server to send mail for your domain, so at least it also gets an SPF:Pass.
+1 for recommending SPF record. It is also a good idea to set-up a bounce mail handler.