Home / server / Questions / 984321
In Process
cnst
Asked: 2019-09-16 12:08:41 +0800 CST

Forwarding to Gmail doesn't work for emails from Microsoft.com due to DMARC, but works for PayPal.com

  • 772

I've noticed that I'm not getting certain emails in my Gmail and Yandex.Mail that are forwarded via UNIX systems (without SRS — not too sure if Sender Rewriting Scheme is still the best practice? Because with DMARC I think it'll also have to apply to the actual From: header within the email itself.) from DMARC-enabled senders.

I can't quite figure out what's going on — emails that always go through include PayPal.com, whereas Microsoft.com and some others get rejected (only getting delivered locally to systems that don't implement DMARC on the receiving side).

% echo _dmarc.{microsoft.com,paypal.com} | xargs -n1 dig -t txt | fgrep v=D
_dmarc.microsoft.com.   3302    IN      TXT     "v=DMARC1\; p=reject\; pct=100\; rua=mailto:[email protected]\; ruf=mailto:[email protected]\; fo=1"
_dmarc.paypal.com.      3311    IN      TXT     "v=DMARC1\; p=reject\; rua=mailto:[email protected]\; ruf=mailto:[email protected]"
%

When both domains have the same reject policy — and Google even specifically mentions that PayPal does have a definitive reject policy — I'm not exactly sure if there's something wrong in my own setup, or if it's the sending party that's to blame. Any ideas?

Is it just because of SPF's fail vs. softfail, or is there more to it?

% echo {microsoft.com,paypal.com} | xargs -n1 dig -t txt | fgrep v= | sed 's#[^[:space:]]*:[^[:space:]]*#:#g'
microsoft.com.          3332    IN      TXT     "v=spf1 : : : : : : : : : : -all"
paypal.com.             300     IN      TXT     "v=spf1 : : : : : : ~all"
%

Here's what Gmail reports for PayPal emails that do get delivered through forwarding:

ARC-Authentication-Results: i=1; mx.google.com;
       dkim=pass [email protected] header.s=pp-epsilon1 header.b=K96c6GUZ;
       spf=fail (google.com: domain of [email protected] does not designate 2001:470:7240:: as permitted sender) [email protected];
       dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=paypal.com
Return-Path: <[email protected]>
spf

3 Answers

  1. The DMARC policy is protecting the domain used in the From: header: to pass the DMARC check this must align either with DKIM or SPF. For SPF, this requires a matching envelope sender (i.e. Return-Path i.e. address used in SMTP MAIL FROM command) with a passed SPF test.

    With DMARC, forwarding the mail (without the final server being aware of it) will cause problems either way:

    • If you don't change the envelope sender, you won't pass the SPF check, and
    • if you change it, it won't align with the From: address anymore.

    The example message from PayPal manages to pass the DMARC test, because it has a valid DKIM signature that is also in align with the From header. Because the error for the other domains was a standard DMARC rejection, we can assume the messages are either missing a valid DKIM signature or it's not aligned with the From header.

    The only way around would be trusting that the forwarding server has already checked for SPF, DKIM & DMARC and blindly accepting every message coming from that server. That's how it's done in a standard primary/secondary MX configuration for the messages coming in through the secondary server – and how it should be done in any forwarding scenario accepted on both sides.

    Unfortunately, based on Gmail Help Community's discussion on "Can I please turn off DMARC", Gmail doesn't allow adding exceptions for DMARC tests. Conclusion: do not forward to Gmail.

    • 1

  2. It seems to me that the -all "hard fail" for Microsoft SPF could be the cause. If the receiving system has a mechanism for enforcing BOTH SPF and DMARC, then the SPF "hard fail" would cause this message to fail, even though DMARC would pass it due to the DKIM pass. Remember, SPF is it's own thing and was around before DMARC. One of the problems with using it, though, was that there was no standard for receiving systems in how to interpret the "soft fail" and "hard fail." So, receivers were on their own to decide what, if anything, to do with both. Microsoft may still have SPF working for inbound mails and may drop messages that evaluate to a "hard fail." Paypal uses a "Soft fail" which may be interpreted as not really important enough to block with the SPF mechanism. Both may be evaluated by DMARC, but the SPF check may kill the messages before DMARC has a chance to pass them though.

    • 1

  3. There are 2 policies actually at play here. One is the hard fail policy for the Microsoft SPF and since you're not changing the envelope.from address, Gmail might just drop it based on SPF.

    To tackle this, it would be wise to rewrite the envelope.from a.k.a return-path. However, this breaks the alignment that is required for DMARC to pass based on SPF. So, that leaves you DKIM to get you that PASS.

    In forwarding scenarios, this situation is sometimes referred to as "DKIM Survival", since DMARC compliance is based on how well the DKIM signature survives the forwarding. Surviving forwarders depends largely on which header fields the forwarder changes, or if it strips the original signature (and optionally replaces it with its own).

    In your case, the paypal message DKIM signature survives the forwarding. Elsewhere in the headers for that email you can find information on the DKIM signing itself and specifically the fields that have been signed. Depending on which type of email you receive from Microsoft (Marketing, Survey etc.), these are likely the header fields that are signed: h=From:To:Subject:Date:MIME-Version:Reply-To:List-ID:Message-ID:Content-Type; It is important to understand which fields your system touches while forwarding messages. Here's a reference to the RFC for DKIM that recommends which headers to actually sign: https://www.rfc-editor.org/rfc/rfc4871#section-5.5

    And then there is ARC (for Authenticated Received Chain). It is still in Draft stage, but typically large mailbox hosters such as Google implement these new guidance rather quickly. DMARC Analyzer has a good description of what it does: https://www.dmarcanalyzer.com/arc-is-here/

    I hope this helps you.

    • 0