I am using LVS (ipvsadm) in NAT mode to load balance UDP traffic for a number of "realservers". I am using one-packet-scheduling so that traffic originating from a single source port on the client is distributed to different realservers.
What I see, though, is that the UDP datagrams, that originate on the realservers and are sent back to the client have their source ip / port set to the one of the realserver, which confuses the client, because it expects to receive replies with source ip / port matching the ones, that he sent the original datagram to.
This is very strange indeed, because LVS is supposed to "hide" the realservers behind the virtual ip / port.
It seems, that if I turn off one-packet-scheduling, the source ip / port of outgoing datagrams are rewritten correctly by LVS.
Has anyone encountered this? If so, what is the way around this?
If you still are interested:
I believe one-packet scheduling does not expect a response from the packet it has just scheduled, and so does not store connection information. Then when the response comes back, LVS fails to find the connection for it and so does not know what source IP/port to use.
This seems to be by design. Search through here for "OPS is implemented for setups where there is no reply for the original packet" and you'll see some explanation.
http://www.austintek.com/LVS/LVS-HOWTO/HOWTO/LVS-HOWTO.UDP.html
The solution is to set a persistence timeout for UDP connections (you can set it low if you expect quick responses), thus explicitly setting a connection time. Since you can do this there's no point for OPS to remember connections.