I use MantisBT 1.2.6 on an Amazon Linux server. It has its own email-sending class layered on top of the PHP mail() function. Actually, it has other options to use smtp or sendmail directly, but I use PHP mail. Then PHP mail in turn uses sendmail.
I configured sendmail to use a smart host, which was working fine as I could write a little PHP program to send a message successfully through its mail() function.
But mail sent by MantisBT never arrived and did not bounce.
/var/log/maillog showed the MantiBT messages being sent to the smarthost successfully
(relay=my-smarthost-hostname) with "stat=Sent (ok nnnn qp nnnn)"
If I changed the /etc/mail/authinfo file to use the wrong password, maillog showed "stat=Service unavailable" for that relay.
So it seemed the problem had to be something subtle in the MantiBT code. I used xdebug to find the call it made to mail(), and get the values of its parameters, and extracted them to a separate php file to play with. It passed a number of headers in addition to "From:" in the fourth parameter, and I suspected that they might have been separated by just \n instead of \r\n as the RFC (2)821 requires, which caused the problem. But I added code to change \n to \r\n in the header string and that made no difference.
In the end, what caused the problem was the presence of a (correct) "Date:" header. When I removed that header from the 4th parameter to mail(), the mail got delivered immediately. So I edited the MantisBT source not to generate that header, and all was fine (I also edited the MantisBT source not to put a space between the "-f" option and the sender address).
So my question is just whether the presence of a "Date:" header in the 4th parameter to mail() is known to cause a problem sending through a smarthost. With a follow-on of wouldn't you expect either a bounce or a message in maillog if the smarthost didn't like seeing the Date header?
PS I actually called Network Solutions technical support about it when I saw the sent ok message in maillog, but before I isolated it to the Date header. Naturally that yielded nothing useful, just instructions for how to configure your email client for POP or IMAP :-)
Apparently I've been hoist by my own methodology's petard.
It now looks like the reason that the date header caused the message to be accepted by the smart host without being delivered is that I didn't change the timestamp in the header with each test. So the smart host saw multiple message with identical message IDs and timestamps coming in, and just didn't deliver the duplicates.
In the process of getting things working, I had seen errors where the -f option was being taken as a recipient address (perhaps because of the space between the "-f" and the sender address), and other issues. So it's possible that when I got that stuff fixed, a message got through okay. But then attempts to confirm that success got stuck in the duplicate message trap. In real life, of course, the timestamp will never be a duplicate along with the message ID, that was just an artifact of capturing the parameters being passed to mail() into a standalone static testcase.
At any rate, I think I'm all set now, and I'll restore the Date header to the MantisBT code.
Not having played much with sending mail before, this was a learning experience for me. And perhaps this will be of use to someone else, to know that if you want to test outgoing mail, you can't use canned headers with invariant MessageID and Date headers!