Home / server / Questions / 103251
Accepted
Sunny
Asked: 2010-01-16 17:00:11 +0800 CST

openvpn dual install (client and server) routing problem

  • 772

I have the following install:

PROD net (10.88.88.0/24)

OFFICE net (192.168.2.0/24)

on PROD I have openvpn server (vpnprod), so OFFICE can connect.

on OFFICE I have openvpn machine (vpnoffice) which runs both openvpn server to allow external users, as well as client to connect to PROD.

Both vpnprod and vpnoffice are running linux.

All works OK, i.e. from OFFICE (any machine), I can make connection to PROD (any) - with some restrictions.

I have all routing set properly.

Also, I can connect clients to OFFICE, and they can access OFFICE machines - no problem.

What fails is if a client (remote, connected to OFFICE) tries to access some PROD machine. It times out.

I have run tcpdump on both tun interfaces on vpnoffice, and it shows the packets sent by the connected client. I would guess that this means that the routing there is OK.

But on tun0 on vpnprod I do not see these packets at all - they do not get there at all.

So, to recap:

officemachine -> vpnoffice -> vpnprod -> prodmachine - WORKS
remote -> vpnoffice -> officemachine - WORKS
remote -> vpnoffice -> vpnprod -> prodmachine - FAILS!!!

My knowledge of tcpdump or similar tools is not very good. Any idea how to approach this problem and how to investigate it?

What else I need to check?

I have checked the firewall rules (IPTABLES), and each and every rule, which would drop any request writes in the log. But I do not see any entries for this particular requests made from a remote client trough vpnoffice to vpmprod.

As requested by @Andrew McGregor (I have put in brackets some explanations):

ip addr (vpnprod):

> ip addr
1: lo: <LOOPBACK,UP> mtu 16436 qdisc noqueue
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2(DMZ): eth2: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 00:60:cf:20:bb:54 brd ff:ff:ff:ff:ff:ff
    inet 10.88.8.1/24 brd 10.88.8.255 scope global eth2
    inet6 fe80::260:cfff:fe20:bb54/64 scope link
       valid_lft forever preferred_lft forever
3(internal): eth1: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 00:10:18:02:30:c4 brd ff:ff:ff:ff:ff:ff
    inet 10.88.88.1/24 brd 10.88.88.255 scope global eth1
    inet6 fe80::210:18ff:fe02:30c4/64 scope link
       valid_lft forever preferred_lft forever
4: eth0: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 00:02:b3:25:94:7d brd ff:ff:ff:ff:ff:ff
    inet MY_EXT_NET/28 brd EXT_IP_BCAST scope global eth0
    inet EXT_IP_1/28 brd EXT_IP_BCAST scope global secondary eth0:FWB1
    inet EXT_IP_2/28 brd EXT_IP_BCAST scope global secondary eth0:FWB2
    inet EXT_IP_3/28 brd EXT_IP_BCAST scope global secondary eth0:FWB3
    inet EXT_IP_4/28 brd EXT_IP_BCAST scope global secondary eth0:FWB4
    inet6 MY::PROD:EXT:NET:XXXX/64 scope link
       valid_lft forever preferred_lft forever
5(OTHER_INT_NET): eth3: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 00:b0:d0:b0:bd:94 brd ff:ff:ff:ff:ff:ff
    inet 172.19.2.193/27 brd 172.19.2.223 scope global eth3
    inet 172.19.2.194/27 brd 172.19.2.223 scope global secondary eth3:FWB5
    inet 172.19.2.195/27 brd 172.19.2.223 scope global secondary eth3:FWB6
    inet6 fe80::2b0:d0ff:feb0:bd94/64 scope link
       valid_lft forever preferred_lft forever
6: sit0: <NOARP> mtu 1480 qdisc noop
    link/sit 0.0.0.0 brd 0.0.0.0
7(PRODVPN server): tun0: <POINTOPOINT,MULTICAST,NOARP,UP> mtu 1500 qdisc pfifo_fast qlen 100
    link/[65534]
    inet 10.136.136.1 peer 10.136.136.2/32 scope global tun0

So, tun0 is the vpn server on prod, its bound to EXT_IP_1 ip address.

ip route prodvpn:

10.136.136.2 dev tun0  proto kernel  scope link  src 10.136.136.1
EXT_NET/28 dev eth0  proto kernel  scope link  src EXT_NET_IP0
172.19.2.192/27 dev eth3  proto kernel  scope link  src 172.19.2.193
10.136.135.0/24 via 10.136.136.2 dev tun0
192.168.2.0/24 (office_int) via 10.136.136.2 dev tun0
192.168.1.0/24 (office_DMZ) via 10.136.136.2 dev tun0
10.39.3.0/24 via 172.19.2.222 dev eth3
10.88.88.0/24 dev eth1  proto kernel  scope link  src 10.88.88.1
10.39.12.0/24 via 172.19.2.222 dev eth3
10.88.8.0/24 dev eth2  proto kernel  scope link  src 10.88.8.1
10.136.136.0/24 via 10.136.136.2 dev tun0
10.176.0.0/16 (OTHER_INT_NET) via 172.19.2.222 dev eth3
169.254.0.0/16 dev eth0  scope link
127.0.0.0/8 dev lo  scope link
default via EXT_IP_ISP_GATEWAY dev eth0

ip addr officevpn:

1: lo: <LOOPBACK,UP,10000> mtu 16436 qdisc noqueue
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
2: eth0(officeDMZ): <BROADCAST,MULTICAST,UP,10000> mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 00:d0:b7:84:ab:a2 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.1/24 brd 192.168.1.255 scope global eth0
3: eth1(office external IPs): <BROADCAST,MULTICAST,UP,10000> mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 00:06:5b:39:c4:21 brd ff:ff:ff:ff:ff:ff
    inet OFF_EXT_IP1/29 brd 216.17.90.95 scope global eth1
    inet OFF_EXT_IP2/29 brd 216.17.90.95 scope global secondary eth1:FWB1
    inet OFF_EXT_IP3/29 brd 216.17.90.95 scope global secondary eth1:FWB2
    inet OFF_EXT_IP4/29 brd 216.17.90.95 scope global secondary eth1:FWB3
4: eth2(office_internal): <BROADCAST,MULTICAST,UP,10000> mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 00:50:da:d7:14:77 brd ff:ff:ff:ff:ff:ff
    inet 192.168.2.254/24 brd 192.168.2.255 scope global eth2
11: tun1(officevpn_server): <POINTOPOINT,MULTICAST,NOARP,UP,10000> mtu 1500 qdisc pfifo_fast qlen 100
    link/[65534]
    inet 10.136.135.1 peer 10.136.135.2/32 scope global tun1
12: tun0(officevpn-client-to-prod): <POINTOPOINT,MULTICAST,NOARP,UP,10000> mtu 1500 qdisc pfifo_fast qlen 100
    link/[65534]
    inet 10.136.136.6 peer 10.136.136.5/32 scope global tun0

tun1 is bound to OFF_EXT_IP1 to serve connecting clients.

ip route officevpn:

10.136.135.2 dev tun1  proto kernel  scope link  src 10.136.135.1
10.136.136.5 dev tun0  proto kernel  scope link  src 10.136.136.6
10.136.136.1 via 10.136.136.5 dev tun0
EXT_IP_NET/29 dev eth1  proto kernel  scope link  src EXT_IP1
172.19.2.192/27 via 10.136.136.5 dev tun0
10.135.137.0/24 via 10.136.135.2 dev tun1
10.136.135.0/24 via 10.136.135.2 dev tun1
192.168.2.0/24 dev eth2  proto kernel  scope link  src 192.168.2.254
10.39.3.0/24 via 10.136.136.5 dev tun0
192.168.1.0/24 dev eth0  proto kernel  scope link  src 192.168.1.1
10.88.88.0/24 via 10.136.136.5 dev tun0
10.39.12.0/24 via 10.136.136.5 dev tun0
10.88.8.0/24 via 10.136.136.5 dev tun0
10.176.0.0/16 via 10.136.136.5 dev tun0
169.254.0.0/16 dev eth0  scope link
127.0.0.0/8 dev lo  scope link
default via EXT_IP0(ISP GW) dev eth1

To sum up: on prod env I have 4 external IP addresses, bound to one interface. I have 1 internal networks (bound to 2 interfaces), and a DMZ (fourth interface):

  • PROD_INT_ZONE1 - 10.88.88.x
  • PROD_INT_ZONE2 - 172.x.x.x (and 10.176.x.x behind it - it has its router)
  • PROD_DMZ - 10.88.8.x
  • EXTERNAP_IPs - EXT_IPxx
  • openvpnserver (bound to EXT_IP) - tun0

on office:

  • OFFICE_INT_ZONE - 192.168.2.x
  • OFFICE_DMZ - 192.168.1.x
  • OFFICE_EXT_IPs - OFF_EXT_IPx
  • openvpn server - tun1
  • openvpn client (to connect to prod) - tun0
firewall routing openvpn tcpdump

1 Answers

  1. Best Answer

    My guess would be that remote clients don't know about PROD network. Push the route through the OFFICE server. Something like push "route 10.88.88.0 255.255.255.0" will should do it.

    • 0