Home / server / Questions / 1024847
In Process
sqlbie
Asked: 2020-07-11 22:11:06 +0800 CST

Forwarded Packets are received by namespace's veth0 but not received by application

  • 772

I use libtins (It uses Pcap) to capture link layer packets and forward to a network namespace where the actual application runs on.

Client(Browser) -> Server -> Pcap -> Pcap Send -> br0 (Bridge) -> Namespace -> Application

Now, I see that the packets are forwarded and visible on Tcpdump snapshot but I don't see they are received by the application it-self, no signs of outgoing packets either.

It appear that the loopback interface in namespace haven't gotten packets either so looks like Kernel doesn't route the packets to loopback for some reason.

I believe the application by default listen to loopback interface, and based on my assumption it's where localhost runs on.

I've tried to bind the application into the veth0 network interface's IP in namespace but it didn't work either.

I am able ping any website inside network namespace.

Regarding Checksum, TCP Dump shows the packets has correct checksum

Here how I set up my network namespace,

sysctl -w net.ipv4.ip_forward=1 &&
sysctl -w net.ipv6.conf.all.forwarding=1 &&
ip netns add namespace1 &&
ip link add veth0 type veth peer name veth1 &&
ip link set veth0 netns namespace1 &&
ip netns exec namespace1 ip addr add 192.168.1.11/24 dev veth0 &&
ip link add name br0 type bridge &&
ip link set br0 up &&
ip link set veth1 up &&
ip netns exec namespace1 ip link set veth0 up &&
ip netns exec namespace1 ip link set lo up &&
ip link set veth1 master br0 &&
ip addr add 192.168.1.10/24 brd + dev br0 &&
ip -all netns exec ip route add default via 192.168.1.10 &&
iptables -t nat -A POSTROUTING -s 192.168.1.0/24 -j MASQUERADE

Here an example of a packet captured by TCP Dump on network namespace,

16:50:11.742116 IP (tos 0x2a,ECT(0), ttl 115, id 8487, offset 0, flags [DF], proto TCP (6), length 52)
    MYHOMEIP.51202 > SERVERIP.https: Flags [SEW], cksum 0x59a4 (correct), seq 332112346, win 64240, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0

Here is output of ss -lntp on namespace,

State           Recv-Q          Send-Q                     Local Address:Port                     Peer Address:Port          Process
LISTEN          0               511                                    *:443                                 *:*              users:(("node",pid=30798,fd=20))
LISTEN          0               511                                    *:7000                                *:*              users:(("node",pid=30798,fd=19))

Looking at the output, I see Send Q is filled with 511 and I'm not sure what that means.

The answer below doesn't add anything new and it didn't fix my problem.

I would love to award a bounty but can't find the button.

Apparently, A similar question has been asked on this site years ago but no one succeeded providing working solution.

tcpdump tcpip link-layer packet-capture packet

1 Answers

  1. Talk to veth

    Your exact commands work on my system.

    Here's what I've checked:

    man nc
ip netns exec namespace1 nc -l 192.168.1.11 8123

    Now on the remote client:

    ip ro add 192.168.1.11/32 via SERVERIP
echo MyTestData | nc -v 192.168.1.11 8123

    Talk to loopback

    You are saying something about loopback though. Obviously won't work for 127/8, so I've prepared routing:

    ip netns exec namespace1 ip a add 192.168.2.2 dev lo
ip ro add 192.168.2.2/32 via 192.168.1.11

    Now on the remote client:

    ip ro add 192.168.2.2/32 via SERVERIP
echo MyTestData | nc -v  192.168.2.2 8123

    That being said, one more caveat. You seem to be coming from netadmin background. The server is not a router and so using loopback for anything else than a 127/8 "traffic to itself" is a very uncommon practice on application servers. Clients are expected to talk to the eth/veth.

    But I cannot tell all the clients to add a route

    Looking at your previous question (now deleted) about how kernel is supposed to handle routing in case client talks to "public" IP I'd like to remind about a thing called DNAT. (It's something else that MASQUERADE. You often use DNAT together with MASQUERADE.)

    So instead of ip ro add at a client you can do DNAT without touching the client. I don't want to come off offensive, I'm stating the obvious, just in case.

    • 1