Ubuntu failover from Ethernet-to-ADSL-modem to USB 3G dongle

I have done this any times for customers and i have not found a developed system to make this so i have ever roll my own, the steps that Manwe has given you are more or less what i do when i need them but i will paste here any crude bash scripts i am using (when i have time i want to make this much better in python).

Basically i check if i have internet or not and if i am using the wan backup and make the changes needed

#!/bin/bash
PATH="/bin:/sbin:/usr/bin:/usr/sbin"
primary_gw="192.168.1.1" #for example.
check_one="8.8.8.8"
check_two="8.8.4.4"

#first we check internet connection.
if `ping -c 1 -W 1 $check_one |grep -E '(unreachable|100\%\ packet\ loss)' &> /dev/null` &&\
   `ping -c 1 -W 1 $check_two |grep -E '(unreachable|100\%\ packet\ loss)' &>/dev/null`
  then #if we don't have internet
    if [ -e /tmp/wan_backup ]
      #if we are using backup right now we try to change to primary connection
      then ./script_change_to_primary.sh && rm /tmp/wan_backup
      #else we change to wan backup.
      else ./script_change_to_backup.sh && touch /tmp/wan_backup
    fi
fi

#if we are using wan backup right now we check if primary connection works.
if [ -e /tmp/wan_backup ]
  then
    if `ip route add $check_one via $primary_gw; ip route add $check_two via $primary_gw;\
        sleep 2; ping -c 1 -W 1 $check_one | grep -E '(unreachable|100\%\ packet\ loss)' &> /dev/null &&\
        ping -c 1 -W 1 $check_two | grep -E '(unreachable|100\%\ packet\ loss)' &> /dev/null`
      then #don't works we clean the routes and stay using backup
        ip route del $check_one via $primary_gw
        ip route del $check_two via $primary_gw
      else #it works so we change active connection 
        ip route del $check_one via $primary_gw
        ip route del $check_two via $primary_gw
        ./script_change_to_primary.sh
        rm /tmp/wan_backup
    fi
fi

Given that you only want your server to use 3g if adsl goes down i would only use iptables snat or masquerade only in the adsl iface and i would block access to squid in ./script_change_to_secondary.sh, your files could be:

script_change_to_secondary.sh

#!/bin/bash
pon 3gIsp #this one it is going to change the default route of server anyway
#drop squid connections, you could disable here the boot snat or masquerade for adsl
#but given your adsl is not active i don't see the need anyway
iptables -I INPUT -s LAN_IP_RANGE -d SERVER_IP -p tcp --dport 3128 -j DROP

script_change_to_primary.sh

#!/bin/bash
poff 3gIsp
iptables -D INPUT -s LAN_IP_RANGE -d SERVER_IP -p tcp --dport 3128 -j DROP
/etc/init.d/openvpn restart

You should have too in /etc/ppp/ip-up.d/ a bash script with "/etc/init.d/openvpn restart", this way every time you connect to a ppp provider your openvpn will restart automatically.

Well like i have said it is a bit ugly and crude but it works :) if you find a integrated clean solution for this make me know please :), one good thing of roll-your-own it is that you have a complete control of the system, this is an oversimplification of what i do in any customers that have two or three connections alive at the same time and do a load balancing and QoS all integrated with scripts that detect connections problems and changes the routes and the QoS.

If you prefer an integrated solution to a roll-your-own you can use a distribution like zentyal, it supports what you want to use but it is a complete distribution tailored to create a SmallBusiness server, i usually prefer to configure my servers at my own but this is a good distribution that can be managed via web.

We have couple of smaller edimax 3g routers that have 1 wlan, 1 lan and usb for 3G stick. Lan port can be configured to be part of the lan or as primary wan port port (3g acting as failover). I'm not affiliated with edimax and we use them on the road or on conventions for their size (and one of the ones we have is battery powered).

Having said that I would go with a setup in the ubuntu server.

Answers:

1. Easy if you are experienced server admin. Otherwise if you are willing to spend the time to learn and understand what you are doing it's doable for a poweruser.
2. I don't know about available packages, but

change to 3g - script example:

pon YourIsp-name
route del default gw "your_adls_defaultGW"
route add -host "ping_or_other_test_host_for_checking_adsl_route" gw  "your_adls_defaultGW"
"do_other_stuff_like_restart_openvpn_maybe"

change to adsl - script example:

poff YourIsp-name
route add default gw "your_adls_defaultGW"
route del -host "ping_or_other_test_host_for_checking_adsl_route" gw  "your_adls_defaultGW"
"do_other_stuff_like_restart_openvpn_maybe"

Using dhcp with adsl will screw this simple example and you have to either modify /etc/resolv.conf with every change or use open dns servers like google 8.8.8.8 (or your own resolver). Now.. monitoring is a bit more difficult, but putting something like this to cron (don't, this is a naive exmaple), will give you failover to 3g and back.

ping -q -c 2 "ping_or_other_test_host_for_checking_adsl_route"  && TARGETUP=1 || TARGETUP=0 ;  echo $TARGETUP ; if [ "$TARGETUP" == 1 ] ; then ( rm /tmp/.adsl_down ; [ -e /tmp/.adsl_up ] || ( PATH_TO_YOUR_ADSL_UP_SCRIPT && touch /tmp/.adsl_up ) ); else  ( rm /tmp/.adsl_up ; [ -e /tmp/.adsl_down ] || ( PATH_TO_YOUR_3G_UP_SCRIPT && touch /tmp/.adsl_down )) ; fi

Three) See Edimax 3g routers ( zeroshell linuxfw-distribution looks like zeroshell has failover in it, so get a cheap pc for a router and use zeroshell. I haven't tested it though )

Four) Test, persist and when failover is needed for the first time, it's going to fail (because of something stupid...).

I'm not sure that you need squid for this setup unless to speed up 3g connection with local cache of most common pages you visit..

Have you considered any hardware based solutions as i can see most posters using software solutions?

Not sure of how much you want to spend but a DrayTek Vigor 2830n would accommodate your needs. There may be cheaper alternatives, but this is the one I've used at two different sites and has functioned exceptionally well.

There are 3 WAN connectivity on this device, the Standard ADSL line, Ethernet connection and 3G dongle. You can set this up just using the ADSL and 3G dongle and using the router itself. The 3G dongle does not need to be on 'load-balance mode' but pure WAN-backup mode only.

Having re-read the original post, I'd like to add that you can even restrict VPN traffic only over the 3G dongle to minimise bandwidth cost on your 3G dongle.

1) Not too difficult - run a task and check for the status of your DSL - when it failes, start up the 3G-Interface and fail over the default route. Maybe you might need to restart your VPN as well.

2 & 3) Never checked for a prebuild package myself, as my favourite firewall does include that feature by default. Check www.astaro.com (the software-appliance is free for home-use or you can use the default 30day trial for testing)

4) Never had any sigificant problems with WAN failovers (other than the backup-line beeing also down)

tsg

I think you could use keepalived to ping a remote internet address (Google mayhaps) every .. 30 seconds with some timeout. If the ping times out configure keepalived to shutdown eth0 and bring up the 3G interface then restart your VPN.

You should consider delays etc. when configuring keepalived so that it does not switch interfaces when not needed. Some testing/experiments will be needed. I cant think of a way to revert to the ADSL line when its back alive though.

If you have a router between the ADSL modem and your server, or the modem has a external address that you can ping you could probably use that as criteria for reverting the connection.

KeepaliveD site

Hope that can get you started!

You could use shorewall (ubuntu package available), and a very useful script named gwping to create a roll-your-own solution.