FINAL EDIT : I was completely wrong about DKIM it seems, the signing domain does not have to be the same as the sender domain, thus the whole premise for my question is flawed. A lot of thanks to Paul for pointing out my mistake!

Original Question below:

I have tried reading up on both SPF and DKIM, but I do not understand the point of using both at the same time, at least in the context of fighting spam (forged sender addresses, which can result in my email server/domain becoming blacklisted). As far as I can understand DKIM alone should do the job the SPF is supposed to do.

My understanding so far is the following:

1. When sending an email the sender can claim anything they want (e.g. fake sender address)
2. DKIM allows to detect a fake sender email address, because you can verify the DKIM signature against the public key in the DNS TXT record.
3. SPF allows you to verify that the email was sent from a mail server that is authorized to send emails for a given sender address.

The thing that I do not understand is this: Unless the DKIM private key has become compromised in some way, the DKIM verification alone should be sufficient to verify that the email was sent from an authorized email server, because otherwise the email server would not have the private key to sign the email.

I have seen the answer to a very similar question here: https://serverfault.com/a/780248/154775, in it the author claims that DKIM cannot prevent replay attacks. I will concede that point, but I find that to be very much a corner case, the by far biggest issue in my opinion is spam with fake sender addresses - DKIM should prevent that easily on its own.

Is there a scenario beyond replay attacks where SPF provides additional protection compared to only DKIM?

EDIT : I have marked the core of my question in bold to clarify what exactly I want an answer to.

I operate a small mail server for my private emails, some friends who have websites and two NGOs. In total my server sends between 60 and 400 messages a day. Now a lot of these emails are personal mails, between two or more people who know each other. Occasionally (usually once or twice a week) there will be a mailing that goes out to "members" of one NGO, informing them what's new etc.

Now I have already moved off the "mass mailings" (about 100 recipients, all personally known and manually subscribed through a paper form) to mailgun.org.

I still get (and increasingly so), rejected messages. Especially big email providers like Gmail, Yahoo or Microsoft (hotmail, live.com, ...) just decide to reject with a 550 or send personal messages to the Spam folder of the recipients. Sometimes this happens:

• gmail user sends email to user on my system
• user on my system replies
• the reply is being rejected or sent to spam

Things I have done:

• set up DKIM (per-domain signing of all outgoing email)
• set up SPF, domains usually have ~all, some -all
• I have a correct PTR for my mail server IP
• obviously no open relay, users can only send from their own email address after authentication
• I have DMARC policies for most of the domains
• I rate limit outgoing messages, for some mail servers down to 1 per minute
• mail test services report "perfect" scores (all pass) for all of the above
• I regularily check my IP for blacklisting using http://www.dnsbl.info - it's always all green

Now the paradox comes here: for most of the big mail providers, there is a way to register to monitor rejection rates and IP reputation:

• https://postmaster.google.com
• https://postmaster.live.com/snds
• and I believe Yahoo has something similar

but I do not classify as bulk sender, because of the low volume. So I did register to monitor my reputation and rejection rates, but because I do not send bulk email, there are no reports.

Is there anything else I can do to improve mail delivery rates? Or should I give in and stop trying to operate my own mail server?

In case it is relevant: I use postfix and have very strict rules about incoming mail (i.e. no unknown domains/host names or invalid SPF records, I use spamassassin etc.)

Update

Here is an example, sent from me to my in-laws and it arrived in their SPAM folder: http://pastebin.com/BC6YgjpQ (I replaced the sending address domain with example.com and the receivers address with [email protected])

Since the question came up: Connections to Gmail are Untrusted TLS connection established to gmail-smtp-in.l.google.com[2a00:1450:400c:c0b::1b]:25: TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits) encrypted.

This is a Canonical Question about Fighting Spam.
Also related:

• How to stop people from using my domain to send spam?
• What are SPF records, and how do I configure them?

There are so many techniques and so much to know about fighting SPAM. What widely used techniques and technologies are available to Administrator, Domain Owners, and End Users to help keep the junk out of our inboxes?

We're looking for an answer that covers different tech from various angles. The accepted answer should include a variety of technologies (eg SPF/SenderID, DomainKeys/DKIM, Graylisting, DNS RBLs, Reputation Services, Filtering Software [SpamAssassin, etc]); best practices (eg mail on Port 25 should never be allowed to relay, Port 587 should be used; etc), terminology (eg, Open Relay, Backscatter, MSA/MTA/MUA, Spam/Ham), and possibly other techniques.

I'm trying to enter a 4028 bit DKIM key into DNS and it seems that I'm exceeding both the UDP 512 byte limit and also the maximum record size for a TXT record.

How does someone properly create a large key (with implied larger encoded size) and import it into DNS?

I got DKIM setup on my mail server (postfix and ubuntu) so it signs outgoing emails. I used these instructions: https://help.ubuntu.com/community/Postfix/DKIM

However, I need it to sign emails from any domain (in the From address) and not just my own. I'm building an email newsletter service and clients will be sending their own email through the server.

First I set "Domain *" in /etc/dkim-filter.conf. This got it to include the DKIM headers in all outgoing emails, no matter what the domain.

However, the verification check fails on gmail because it is checking the domain in the from address, and not my domain (and dns record). Does anyone know how to do this?