postfix multiple smtpd_sender_login_maps

After much head-scratching, I think I found the answer. Postfix defines a "match" when searching sender_login_maps as any lookup that succeeds, but the result of that lookup is not considered.

tl; dr: If you intend to use a union of the results from all mappings in smtpd_sender_login_maps, instead of just taking the first succeeding lookup result, the mappings must be combined somehow (e.g. via SQL UNION or unionmap).

At its core, it's a surprisingly simple problem caused by Postfix's short-circuiting behavior when doing mapping lookups. Buckle up for the ride, this is a long answer...

Background

From the Postfix documentation on sender login maps:

smtpd_sender_login_maps (default: empty) ...

Specify zero or more "type:name" lookup tables, separated by whitespace or comma.
Tables will be searched in the specified order until a match is found.

http://www.postfix.org/postconf.5.html#smtpd_sender_login_maps

The purpose of using smtpd_sender_login_maps is to validate whether the currently authenticated SMTP user is allowed to send from a given FROM address in their message. It's a powerful feature to restrict the addresses a client can send from based on who the authenticated SMTP user is.

smtpd_sender_login_maps has many use cases, e.g.:

• Prevent a user from logging in and sending a message as someone else
  > Bob can send as [email protected], but can't send as [email protected]

• Allow multiple users to send from a single shared address neither of them own
  > Bob and Alice can both send as [email protected]

• Allow a single user to send from multiple addresses they own
  > Alice can send as both [email protected] and [email protected]

• Allow an admin to send messages from many addresses (even owned by others)
  > Tracy, the IT admin can send as anyone .*@example.com

• And more... (remember, lookups can be via TCP socket, SQL query, regex, etc.)

Example Scenario

We're going to investigate this use-case in particular:

• Allow an admin to send messages from many addresses (even owned by others)
  > Tracy, the IT admin can send as anyone .*@example.com

Imagine a setup where we have Postfix and MySQL running on a mailserver. There are 3 users stored in the DB who can authenticate via SMTP with postfix:

• [email protected]: can only send from [email protected],[email protected]
• [email protected]: can only send from [email protected],[email protected]
• [email protected]: should be able to send from any .*@example.com address

Regular users can only send from their main address and their aliases, but we want to allow the admin to be able to masquerade as any sender without restrictions. In real life, this type of setup is useful whenever a user needs to be able to send from many addresses .*@example.com using only one SMTP login (e.g. if the addresses are dynamically-generated or belong to other users).

The normal approach is to use smtpd_sender_login_maps to achieve this setup.

Example Config

/etc/postfix/main.cf:

...

smtpd_sender_login_maps = 
    mysql:/etc/postfix/sender_logins.cf,
    pcre:/etc/postfix/sender_overrides.cf

...

MySQL is checked first, then the pcre is checked only if the mysql lookup returned 0 results.

---

/etc/postfix/sender_logins.cf (mysql):

hosts = 127.0.0.1
user = postfix
password = yourDatabasePasswordHere
dbname = mail
query = SELECT email FROM users WHERE email='%s'

This mapping checks the DB and returns the normal SMTP user for a given FROM addr, e.g.:
[email protected] -> [email protected]
[email protected] -> [email protected]

---

/etc/postfix/sender_overrides.cf (pcre):

/.*@example.com/     [email protected]

This mapping matches all @example.com FROM addrs and returns the admin SMTP user, e.g.:
.*@example.com -> [email protected]

The problem

The behavior the user expects:

1. Postfix looks up the FROM addr in the first sender_login_maps db
2. it finds a matching entry for the FROM addr
3. the returned SMTP user != logged in user, so we try the next mapping db
4. Postfix looks up the FROM addr in the next sender_login_maps db
5. it finds a matching entry for the FROM addr
6. the SMTP user == logged in user
7. Postfix sends the message succesfully

What actually happens:

1. Postfix looks up the FROM addr in the first sender_login_maps db
2. it finds a matching entry for the FROM addr, so the lookup process stops
3. the returned SMTP user != logged in user
4. Postfix rejects the message

Explanation

The issue is that Postfix doesn't check both mappings and merge the results, instead it stops the lookup process the moment it encounters the first matching lookup returning any SMTP user.

If the returned SMTP user doesn't match the currently authed user, it won't proceed to lookup the address in the next database, it'll just immediately DENY.

The same lookup conflict can happen with any two mappings that share keys, e.g. two mysql databases mysql:...,mysql:..., not necessarily just a pcre with /.*/ or /.*@example.com/. Any exact match or wildcard match like @example.com [email protected] in the first mapping will take precedence and prevent the second mapping from being queried at all.

Scenario 1: [email protected] tries to send from [email protected]

✅ This works fine, the email is sent because the address matches the sender as expected.

1. [email protected] logs in via SMTP to send a msg from [email protected]
2. [email protected] lookup in smtpd_sender_login_maps returns [email protected]
3. Sending succeeds, [email protected] == SMTP authed user [email protected]

Scenario 2: [email protected] tries to send from [email protected]

❌ This does not work, the email is rejected because the SMTP authed user [email protected] doesn't match the first lookup result [email protected].

1. [email protected] logs in via SMTP to send a msg from [email protected]
2. [email protected] lookup in smtpd_sender_login_maps returns [email protected]
3. Sending fails, [email protected] != SMTP authed user [email protected]
postfix/smtpd[16645]: NOQUEUE: reject: RCPT from webmail.mailserver[192.168.1.5]: 553 5.7.1 <[email protected]>: Sender address rejected: not owned by user [email protected]; from=<[email protected]> to=<[email protected]> proto=ESMTP helo=<mail.example.com>

What if we flip the order of the mappings?

smtpd_sender_login_maps = 
    pcre:...,        # moving the pcre mapping above mysql makes it worse
    mysql:...

Flipping the order of the mappings so that the pcre file is checked before mysql won't fix the problem. It even makes it worse because the catchall .*@example.com will overshadow all the real users in MySQL and prevent any user other than [email protected] from sending email.

When it looks up [email protected] in the pcre file, it returns [email protected] as the only allowed user and fails before it ever checks the mysql database.

The solutions

A. Make the smtpd_sender_login_maps disjoint

If the mappings don't contain any overlap in keys, then the order doesn't matter, and any lookup that doesn't match the first db will proceed to check the subsequent ones as expected.

Mappings earlier in the list cannot have any "catchall" or wildcard keys or they will match everything and overshadow results from later mappings.

B. Join multiple lookup results in SQL with UNION

If you're using the mysql table type for your smtpd_sender_login_maps mappings, then you can control the SQL query run when doing an address lookup and you may be able to join multiple mappings at the SQL level.

Assuming all your mappings are accessible in the same MySQL database, you can concatenante the results of multiple address lookups at the SQL level using a UNION statement like so:

main.cf:

smtpd_sender_login_maps = mysql:/etc/postfix/sender_logins.cf

sender_logins.cf:

hosts = 127.0.0.1
user = postfix
password = yourDatabasePasswordHere
dbname = mail
query = SELECT email
            FROM users
            WHERE email='%s'
        UNION SELECT destination
            FROM aliases
            WHERE source='%s'
        UNION SELECT email
            FROM users
            WHERE wildcard_sending=1

In this example we would set [email protected] to have wildcard_sending=1 in SQL, and then it would be returned with every lookup result along with the normal user and alias matches, e.g.

[email protected]    -> [email protected],[email protected]
[email protected]            -> [email protected],[email protected]

C. Use unionmap to combine multiple mappings

If you're using Postfix 3.0 or above, you might be able to try using the new unionmap feature, which performs a lookup to all the mappings at once and concatenates the results together.

smtpd_sender_login_maps = unionmap:{
        mysql:/etc/postfix/sender_logins.cf,
        pcre:/etc/postfix/sender_overrides.cf }

With this setup, the mysql results will be concatenated with the pcre lookup results, e.g.

[email protected]    -> [email protected],[email protected]
[email protected]            -> [email protected],[email protected]

See http://www.postfix.org/DATABASE_README.html#types unionmap for more info.

Sources

Word "match" is thus to be taken here in the narrowest sense: just a matching address, not a matching (address, login name) pair. Moreover, when a matching address is found, the lookup chain is immediately halted with a binary reject/accept decision.

• https://www.mail-archive.com/[email protected]/msg00677.html
  (I highly recommend reading this entire mailing list thread for more context)
• http://www.postfix.org/postconf.5.html#smtpd_sender_login_maps
• http://www.postfix.org/SMTPD_ACCESS_README.html#lists
• http://www.postfix.org/DATABASE_README.html#types
• http://www.postfix.org/access.5.html
• https://unix.stackexchange.com/questions/294300/postfix-prevent-users-from-changing-the-real-e-mail-address
• https://groups.google.com/forum/#!topic/mailing.postfix.users/X7Wj_nSEyKI
• https://workaround.org/ispmail/jessie/relaying-smtp-authentication