We have a Head-Office/Branch-Office WAN like this,
Server LAN <-> Cisco PIX 515e <-VPN tunnel-> Cisco ASA 5505 <-> Client LAN 1
<-VPN tunnel-> Cisco ASA 5505 <-> Client LAN 2
<-VPN tunnel-> Cisco ASA 5505 <-> Client LAN 3
...
<-VPN tunnel-> Cisco ASA 5505 <-> Client LAN 66
Problem:
5% of these VPN tunnels degrade over time.
Symptoms:
- Clients respond to PING, but not to RPC or RDP.
- On the ASA, VPN tunnels goes from 1 x IKE, 2 x IPSec down to 1 x IKE, 1 x IPSec.
- A restart of the ASA resolves the problem temporarily.
This PIX has been unreliable, and will probably be replaced with a more modern bit of gear. Although usually under 10%, the CPU on the PIX periodically hits 80-90% with traffic spikes, but I can't say I've been able to correlate dropped tunnels with these loads.
I have a few specific questions, but am grateful for any and all insights.
Can I monitor (via SNMP) the total IPSec tunnels on the PIX? This should always be (at least?) twice the number of branch offices, and (at least?) twice the total IKE - if it drops then I probably have a problem.
Is there an event I can alarm on in the PIX's own logging, when one of these tunnels is dropped? Maybe,
snmp-server enable traps ipsec start stop
Is there anything I can do to keep this tunnel alive, until the PIX can be replaced? I was thinking of scriptable keep-alive traffic, PING doesn't seem to cut it. I am also looking at idle time-out values, maybe re-keying intervals, any other ideas?
PIX515E# show run isakmp
crypto isakmp identity address
crypto isakmp enable outside
crypto isakmp policy 10
authentication pre-share
encryption 3des
hash sha
group 2
lifetime 86400
crypto isakmp nat-traversal 20
PIX515E# show run ipsec
crypto ipsec transform-set ESP-3DES-SHA esp-3des esp-sha-hmac
PIX515E# show version
Cisco PIX Security Appliance Software Version 7.2(4)
Device Manager Version 5.2(4)
1) You absolutely can monitor the number of IPSec tunnels, but we’ve found that not to be a truly reliable way of determining if connectivity is working. It’s always best to send and receive traffic via the tunnel to confirm connectivity (e.g. ping monitor).
2) Same as #1 – it can be done, but may not give you usable information. Tunnels will start and stop in the normal course of operation depending on timeout intervals.
3) While it’s not supposed to be necessary, we have seen improvement with tunnel connectivity in some situations by running a ping at frequent intervals (3-5 minutes). Hard to say whether that would help in this situation without in-depth analysis.
Generally speaking, issues like this occur frequently due to VPN config mismatches between the head end and remote end VPN peers. Differing ACLs are often a problem.
Do the tunnel ever come back up by itself or do you manually intervene to get the tunnel back up?
What is the lifetime set on the ASA?
And do you have keepalives enabled/disabled on both devices?
I've seen this issue before between a Cisco 6500 running IOS and an ASA where the IOS is happy to run without an SA (if it expires for whatever reason) where a ASA is not and the tunnel dies for a random period of time until it renegotiates and the tunnel comes back up until the SA expires again.
I am seeing the same thing myself. I just setup a PIX515 running 8.04 IPSEC to my ASA5510 8.2. It works great and then the tunnel just dumps everyone. During this time, the internet keeps going just fine. So, it's just the tunnel that is having problems.
I am having the same problem, but with a ASA-5505 and Juniper SRX220h (colo) I spent over 12 hrs with JTAC and its nothing on their end (so they say). So I called Cisco TAC, no support warranty is out. So I been searching all morning. This thread is the closest I found to my problem.
My solution I set both devices to 86400 seconds, and also disabled Keep-alives.
I will let you know the result ASAP.