SNMP through NAT when server has two IP addresses

We have an SNMP client talking to an SNMP agent through NAT. That works most of the time. We send an SNMP request e.g. from 1.2.3.4:12345 to 10.0.0.1:161 and get a reply from 10.0.0.1:161 to 1.2.3.4:12345.

However, some agents have multiple IP addresses, and for these when for such a server an SNMP request from 1.2.3.4:12345 to 10.0.0.1:161 we may get a reply from a totally different IP address, e.g. from 10.0.1.1:161 to 1.2.3.4:161. I'm presuming this is because SNMP uses UDP and therefore doesn't have a connect, and the agent's default gateway uses 10.0.1.1:161.

This works fine on a "real" directly-connected client without NAT, but when NAT is involved the client's response is not sent through NAT to the client.

We're using standard iptables NAT.

• Is there any way to get iptables to only look at port numbers for UDP NAT so it'll send these replies from other IP addresses through?
• Are there better NAT implementations for our purposes that we haven't tried?

Changing the IP address we ask is unfortunately not an option...