Home / server / Questions / 151609
Accepted
The_cobra666
Asked: 2010-06-17 00:53:32 +0800 CST

Mac and L2TP VPN no problems, xp, vista and 7 no go :s

  • 772

I've got some weird problem and I'm out off options. The situation:

When connecting from my mac to the VPN server (Windows Server 2003 R2) with L2TP PSK, everything works like it should.

However, when I connect from a Windows PC, nothing happens. it spits out error 809 and sometimes 789. Now I know that my ports are OK, since the mac can connect without any problems.

It's the same for: XP, Vista SP2 and 7. None can connect. If I connect to the VPN server directly (to the internal IP instead of WAN from the router), it connect's without a problem. Connecting using PPTP works... now if only L2TP would work thank you very much Windows!

I have checked the counters on my linux router with iptables -L -nv and they do not raise when connecting. Not on ACCEPT and not on DROP. Only when connecting from the mac.

I've found the guide from Microsoft to enable: AssumeUDPEncapsulationContextOnSendRule in the registery. I have set it to "2", on the server and client. Still no go. After that registery key it started giving me error 789 instead of 809. The IPSEC services are running on the client and server.

Is there anyone that ppleease can help me with this! I've been working on this for 2 days and I'm out of options.

Thanks!

//edit: see solution below. This is ONLY for windows clients trying to connect to a VPN server behind a NAT router!

vpn windows-server-2003 windows-7 l2tp

1 Answers

  1. Best Answer

    Apparently, it was the "AssumeUDPEncapsulationContextOnSendRule" that was needed. I manually restarted the IPsec services and then it worked. Very strange!

    The steps needed:

    1. Log on to the Windows Vista client computer as a user who is a member of the Administrators group.
    2. Click Start, point to All Programs, click Accessories, click Run, type regedit, and then click OK. If the User Account Control dialog box is displayed on the screen and prompts you to elevate your administrator token, click Continue.
    3. Locate and then click the following registry subkey: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Servic es\PolicyAgent Note You can also apply the AssumeUDPEncapsulationContextOnSendRule DWORD value to a Microsoft Windows XP Service Pack 2 (SP2)-based VPN client computer. To do this, locate and then click the following registry subkey: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Servic es\IPSec
    4. On the Edit menu, point to New, and then click DWORD (32-bit) Value.
    5. Type AssumeUDPEncapsulationContextOnSendRule, and then press ENTER.
    6. Right-click AssumeUDPEncapsulationContextOnSendRule, and then click Modify.

    7. In the Value Data box, type one of the following values: • 0 A value of 0 (zero) configures Windows so that it cannot establish security associations with servers that are located behind NAT devices. This is the default value. • 1 A value of 1 configures Windows so that it can establish security associations with servers that are located behind NAT devices. • 2 A value of 2 configures Windows so that it can establish security associations when both the server and the Windows Vista-based or Microsoft Windows Server Code Name "Longhorn"-based VPN client computer are behind NAT devices.

    8. Click OK, and then exit Registry Editor.

    9. Restart the computer

    With thanks to: http://www.errorforum.com/microsoft-windows-vista-error/6499-configure-l2tp-ipsec-server-behind-nat-t-device-vista.html

    • 2