Home / server / Questions / 12285
Accepted
Joe Holloway
Asked: 2009-05-27 08:26:45 +0800 CST

When IP aliasing how does the OS determine which IP address will be used as source for outbound TCP/IP connections?

  • 772

I have a server running Ubuntu Server with four IP addresses aliased on a single NIC. 

eth0       192.168.1.100
eth0:0     192.168.1.101
eth0:1     192.168.1.102
eth0:2     192.168.1.103

(Using 192.168.x.x for sake of example, assume these are NAT-ed to a range of public IP addresses)

One of our clients publishes their inventory via FTP, so we log in nightly to download a large file from their server. Their firewall expects our (passive) FTP connection to be made from 192.168.1.100.

Given that my server logically has four IP addresses on a single adapter, how does the operating system determine which IP address is used as source for outbound TCP/IP connections?

Let's say I ssh into my server on 192.168.1.101 and run FTP interactively. Will the outbound TCP/IP connection use 192.168.1.101 because the OS knows that's the interface over which my shell is connected?

What if the FTP task is run non-interactively via a cron job where there is no shell?

As you can probably tell, this has me quite confused, so I hope my questions have at least made sense.

Edit

To clarify why I'm asking -- I haven't made any changes to the routing table and it actually lists 'eth0' as the IFace for the 0.0.0.0 routes. However, all indications are that it is actually using eth0:0 as the source. 

Destination    Gateway         Genmask         Flags Metric Ref    Use Iface
192.168.1.0    0.0.0.0         255.255.255.0   U     0      0        0 eth0
0.0.0.0        192.168.1.1     0.0.0.0         UG    100    0        0 eth0

I can fiddle with the routing table or have our client change their firewall rules to get the behavior I need, but I'm trying to gain a little insight into how this works to know if there's a bug in the OS or just my naive understanding of how all the pieces fit together.

Thanks

linux ftp ssh tcpip ip-aliasing

6 Answers

  1. Best Answer

    By default, on Linux, if an interface has multiple addresses that are on different subnets, traffic destined for the respective subnets will have the proper source IP. That is, if eth0 has two addresses 192.168.1.1/24 and 10.1.1.1/8, then traffic to anything on the 10.0.0.0 subnet will have source 10.1.1.1, and traffic to anything on the 192.168.1.0 subnet will have source 192.168.1.1. You can also assign source addresses explicitly in this case by using the "src 1.2.3.4" option to "ip route".

    In your case, though, all your addresses are on the same subnet, so the "primary" one (as revealed by "ip addr list dev eth0") is used as the source IP for traffic exiting on that interface. I think it's possible to control the source IPs in this case just using "ip route", but I've found it easier to use iptables to rewrite the source addresses for traffic of interest.

    If you want to force a specific source address to be used for specific destinations, you can do it with a SNAT rule:

    iptables -t nat -I POSTROUTING -o eth0 -d dest-IP-or-net/mask -s primary-IP-of-eth0 -j SNAT --to-source desired-source-IP

    So if your "primary" eth0 IP is 192.168.100.1, but you want traffic to 1.2.3.4 to have a source of 192.168.100.2, then do this:

    iptables -t nat -I POSTROUTING -o eth0 -d 1.2.3.4/0 -s 192.168.100.1 -j SNAT --to-source 192.168.100.2

    Note that the "-s 192.168.100.1" is important: it prevents forwarded traffic's source addresses being rewritten by this rule.

    If you are going to implement complex network configurations on Linux, you should read the Linux Advanced Routing and Traffic Control documentation, http://lartc.org

    • 12

  2. It uses whatever the default gateway is in the routing table, unless there is a specific route telling it to use another: route -n

    EDIT: I read your question too quick it seems...

    Since you are using passive mode and the client will always be initiating the connection, I think src ip field in the IP header will always appear as whatever IP the client connected to. If it were active mode the server was initiating the connection, I think it would always be the 'Primary' IP. If your addresses are in the same subnet, Linux will make the first address you added 'Primary' and the others secondary.

    I am not entirely sure though, I would run tcpdump -n and see what it sees as the src IP.

    EDIT2: Okay, I wrote the above from the standpoint that you were running the server, so since you are the client and initiating the connection, I think it will always appear to come from the Primary IP address, but again, try it and see with tcpdump.

    • 5

  3. I see in your example that all the ips are too close not to be in the same network

    are you sure that you are actually multihoming and not simply having 4 IP aliases?

    if the latter is the case then you can set the source ip on a route with something similar to this

    /sbin/ip route show 192.168.222.0/24 dev eth0 proto kernel scope link src 192.168.222.178 169.254.0.0/16 dev eth0 scope link default via 192.168.222.1 dev eth0

    sudo /sbin/ip route replace default via 192.168.222.1 src 192.168.222.178

    /sbin/ip route show
    192.168.222.0/24 dev eth0 proto kernel scope link src 192.168.222.178 169.254.0.0/16 dev eth0 scope link default via 192.168.222.1 dev eth0 src 192.168.222.178

    see man interfaces on how to make it persistent between reboots

    • 5

  4. Unless your FTP job has a way of specifying the interface to use for connections, I believe it defaults to the first physical interface on the relevant subnet (eth0 in this case). If you had a server with two NICs on different subnets, it'd figure out which interface to use based on the routing table.

    As there is only a single physical interface on the system (eth0) and four virtuals/aliases (eth0:0 through eth0:2) on the same subnet, outbound traffic will use the eth0 IP address as the source unless the application is smart enough to declare an outbound interface.

    • 4

  5. You could see which device and src ip address will be used by ip route get command like below:

    $ /sbin/ip route get 1.1.1.1
1.1.1.1 via 2.2.2.2 dev eth0  src 2.2.2.2 
    cache  mtu 1500 advmss 1460 hoplimit 64

    I haven't tried this in aliased environment, but hope this helps.

    • 4

  6. When establishing an outbound connection, your server will look in its routing table to determine which of your four interfaces to use; your TCP connections will have a source IP of your exit interface.

    netstat -rn

    Will give you the output of your routing table; look for any specific entries matching the client IP you are trying to connect to. If none exist, then you will be using a default route (0.0.0.0, mask 0.0.0.0). If you have multiple default routes, the one with the lowest cost will be the one used.

    • 1