I am using Debian 6 - 64-bit and this is what my /etc/network/interfaces file looks like
auto eth0
iface eth0 inet static
address PrimaryIP
gateway X.X.X.X
netmask 255.255.255.0
auto eth0:1
iface eth0:1 inet static
address SecondaryIP
gateway X.X.X.X
netmask 255.255.255.0
auto lo
iface lo inet loopback
I need to switch between the Primary and Secondary IPs for most of my processes/scripts so quite a lot of times I swap PrimaryIP with SecondaryIP (and vice versa) in the above file and restart the interfaces to change the primary IP system is working on.
A problem I'm facing with this is that although the System's Primary IP is set in the above file and works perfectly after networking restart, the session/ssh IP still remains the secondary one. So where the system uses the PrimaryIP, anything I execute in the SSH session is still executed as the SecondaryIP.
Note: I have also tried logging in with both the IPs but the session still gets locked to the secondary IP although any system commands such as cronjobs are executed as primary.
You need to bind the query to the appropriate interface. Eg. for curl:
curl --interface eth0:1 .....
If the command you run doesn't allow you to bind to an IP/interface then you are SOOL ...
Or use SOCKS or something to force the server to use a specific IP .
After days of working around, I finally found the answer to my own question.
The correct way of setting up a secondary interface would be to not include the "gateway" option.