Home / server / Questions / 1095900
Accepted
Rohan
Asked: 2022-03-11 20:54:45 +0800 CST

How to set up DNS settings with a domain in the middle for DKIM and SPF?

  • 772

I am working on a tool that helps the users send emails. I plan on using an MTA (Mail Transfer Agent) in the back end like AWS-SES or Sendgrid etc. In order for the emails to arrive successfully in the recipeints inboxes, the users will have to set up DKIM/SPF by configuring the DNS settings of their respective domains.

Now if I take SES for example, I know they have an API that allows me to add an "Identity" and fetch back all the necessary DNS records for it using the API. I am sure Sendgrid and other MTAs have similar APIs that allow to add identities and give back the DNS records for the user to apply.

I show the returned DKIM DNS settings to the user, and they add it to their DNS provider, and after that when they send emails, the recipeints get it correctly (without any "via amazonses.com" stuff in the headers)

Now for the sake of example - let's assume that the tool I am building is hosted on chillybilly.xyz and one of the users that uses my tool, they have a domain called frankthetank.xyz which they want to use to send emails via my platform.

When the user tries to verify his domain via my platform, I will hit that API mentioned above in AWS SES - and show something like this to the user:

enter image description here

After which they can add these CNAMES and TXT records for successful DKIM/SPF and can start sending emails. But if you look closely, they can see I am using SES because of the values of those CNAMES and TXT records. And that is something I want to avoid, instead I want to have those called something like 7nuk24xywyawocu6ctqjxmjasiaiq3vq.dkim.chillybilly.xyz which would show my branding, but in the background it will still point to the correct SES one.

Now I am aware, that such a thing is possible because when I signed up for ConvertKit, they showed me something like so:

enter image description here

Those two values in there, as you can see, are pointing towards converkit.com BUT when I run them through a DNS lookup:

https://dnschecker.org/all-dns-records-of-domain.php?query=spf.dm-5mk8zo6m.sg7.convertkit.com.&rtype=ALL&dns=google

https://dnschecker.org/all-dns-records-of-domain.php?query=dkim.dm-5mk8zo6m.sg7.convertkit.com.&rtype=ALL&dns=google

enter image description here enter image description here enter image description here

I can see that in the background it points to MX and TXT records that belong to Sendgrid. How can I achieve this? (I believe the same principles will apply for SES or any other MTA as well too)

EDIT: I tried a few things - And I set the CNAME, MX and TXT in chillybilly.xyz (my project's domain) and I pointed two CNAMES to it from frankthetank.xyz called spf.frankthetank.xyz and dkim.frankthetankxyz

https://dnschecker.org/all-dns-records-of-domain.php?query=spf.frankthetank.xyz&rtype=ALL&dns=google

enter image description here enter image description here

https://dnschecker.org/all-dns-records-of-domain.php?query=dkim.frankthetank.xyz&rtype=ALL&dns=google

enter image description here

As you can see, I was able to achieve very similar results to what ConvertKit is doing with Sendgrid. But it is not getting verified this way. :(

The only difference I see when I check those DNS lookups (links above) is that the CNAMEs also show up in the lookup for me, but not in the case of convertkit. So I think I am close to a solution, but not sure what I am missing, any ideas? :)

domain-name-system spf dkim amazon-web-services sendgrid

3 Answers

  1. First SPF: if you replace eg. sendgrid's outgoing SPF record, which is included in your SPF-record, with your own branding you have at the same time taken on the job of following all the dynamic things sendgrid might do to their outgoing mailserver's names and addresses. You don't want to do that, so if you are sending via sendgrid you need to include sendgrid's SPF. So since you use sendgrid, there's no way you can get rid of that "include: sendgrid.net" in your SPF-record without inflicting pain on yourself.

    DKIM is similar, the amazonses example is creating CNAMEs to records that exist in Amazon's DNS. If you want to replace it with your own (which you can, no problem) you have to put those records into your own DNS. Which is no problem, however, you have to somehow transfer the signing key to whoever signs those emails, and that might be a problem. With the Amazon solution they create the key and give you the link to the public key they put in their DNS that fits the private key they sign your email with.

    However, if you want to remove "branding" from Amazon's or sendgrid's mailservers the only way is to buy them out, because that is done by reverse DNS, and they will NEVER point their outgoing servers to names in your DNS namespace.

    • 3

  2. So you could just do CNAME records that point to the original source record or just put the correct SPF like the convertkit example but under your domain.

    or have a middleware that goes through the domain verification process in sendgrid/AWS, creating the DNS records on your domain, which you then tell your customer to CNAME to the new DNS records you created, and then completes the verification once the customer clicks a button to say they have done this

    this won't stop email headers or following the DNS records to find the original provider but for an average person this is likely fine.

    However I want to caveat this - layers of obfuscation actually hurts at the DNS layer.

    For example SPF has an explicit hard coded 10 lookups limit in the RFC - Sendgrid themselves write about it - https://docs.sendgrid.com/ui/account-and-settings/spf-limitations

    • Original RFC. You are at high risk of hitting this issue if you do any kind of obfuscation for SPF

    additional DNS lookups will increase response times for any DNS based lookup in the mail delivery flow and/or your application - this could cause much pain at many levels especially as MTA's are processing in real time, any delay has huge potential knock on impacts.

    To put some maths into this, imagine a single email takes 20ms to be processed and sent via an MTA with DNS - this is taking the best case example assuming this logic holds true. You can send Approx 180k emails per hour. If we need to double this by adding at least 1 extra dns lookup this halves to 90k emails per hour. If you are reliant on volume you would need to now throw twice the amount of hardware to maintain same throughput.

    • 1
  3. Best Answer

    SG has an option called automatic_security in the Authenticate a Domain API. What this option does I think in the back end is it lets SG manage/rotate DKIM keys and other records using their DNS records. Consequently when you make this option true, they return CNAME records instead of MX and TXT records. And the MX/TXT records are managed behind the scenes by SG.

    enter image description here

    The convinient part about CNAME records is that other CNAME records can easily point towards them. So I pointed chillybilly.xyz -> SG CNAME records that I got back from SG, and I pointed frankthetank.xyz records -> chillybilly.xyz CNAME records and thus creating a sort of a middleware in between.

    Please not that chillybilly.xyz is my app's domain and frankthetank.xyz is the client's domain. That is the assumption we are operting on while solving thie problem.

    Now of course as one of the answers pointed out that this will increase the DNS lookup because of another layer in between, we still need to evaluate if doing this actually makes sense for us or if it will increase email delivery times when scaling up eventually. But the solution does work. I think its an SG only implementation tho I think. I wasn't able to make it work with AWS SES. I have a call with them soon - If I know any better, will edit this answer :)

    • 1