I made a C program on Linux (Ubuntu 18.04) that forwards traffic to another server via IPIP. When the IPIP endpoint sends replies back, the program also sends back the data to the client after stripping the outer IP header. The application and IPIP tunnel endpoint on the machine traffic is being forwarded to sits inside of a network namespace with the IPIP tunnel being the default route. This all works fine. However, I'd like to implement something that would make it so the application sends traffic back to the client directly by spoofing the source IP address to the forwarding machine IP via the POSTROUTING
chain in IPTables. This would result in less load on the forwarding machine since the application wouldn't need to send traffic back to the clients through the IPIP tunnel and I believe it'd make it more consistent.
To do this, I tried setting up a veth pair where one veth is in the default network namespace while the peer veth is in the custom network namespace with the application. I also setup a bridge on the main network namespace, assigned an IP address to it, and made an SNAT rule in the POSTROUTING
chain to source traffic out as a specific IP. I then bridged the veth interface on the main network namespace. The application still binds to the IPIP tunnel, but when it needs to send outbound traffic, it sends the traffic through the default route. Therefore, I set the default route to the veth interface inside the network namespace.
All of this mostly is working. Traffic from the application is sent back to the clients directly and the client responds back to the forwarding server. The traffic is then sent back to the server traffic is being forwarded to via IPIP. I confirmed all of this via tcpdump
. However, the application I'm running is not processing the replies. I believe the reason for this is the application is expecting the responses to come back on the veth
interface (default route) since that's what it used to send the replies back to the clients. Unfortunately, the application is closed-source, so I can't inspect the code. I might however, create a test C program that uses the default route to send outbound connections and do further troubleshooting.
I was just wondering if anybody had suggestions for this. I've read some things about marking the packet via IPTables. I'm not entirely sure if that mark will stick after the client replies and it's sent to the forwarding server.
I believe there is certain traffic that is expected on the IPIP tunnel and then other traffic that is expected on the default route. In my case, the clients connected to the application is sending traffic to the IPIP tunnel and the IPIP tunnel is expecting it. However, the application also makes outbound connections to a backbone and I believe this is the type of traffic expected on the default interface inside of the network namespace.
In the case that all traffic is being expected on the default route, is it possible to forward the traffic from the forwarding server to this specific interface along with removing the IPIP protocol (stripping the outer IP header, etc)? I'd assume this would need to happen after the traffic hits the IPIP tunnel.
With that said, the reason I want to use IPIP is because I can perform NAT on the destination machine (traffic gets forwarded to the IPIP tunnel using the inner IP header's destination address). Therefore, I would prefer not getting rid of the IPIP tunnel.
Any help is highly appreciated and if you need additional information, please let me know!
I just wanted to provide an update. I resolved this issue around a week or so after posting this question by creating a TC BPF program that automatically sends outbound IPIP packets back to the client directly. For those interested, feel free to check it out here.
Thanks!