Francis Booth's questions

I have a PFSense firewall sitting as a gateway for a group of VM's sitting on top of a Xen hypervisor (all machines except for the host are virtual). I have PFSense acting as a waypoint for me to be able to route traffic out to the Internet as my hosting provider does a thing where if you want devices to connect at all you have to have a failover IP assigned to a virtual MAC for it to go out to their network so I figured I'd just port forward on this one public address for when I need to. Sounds good right?

PFSense also has an IPv6 exit (just for the sake of dual-stacking and if I only want a host to have an IPv6 address for individual reachability)

From PFSense itself I can ping,curl,dig...etc on its public IPv4 address however the same cannot be done for clients on the LAN network. For whatever reason that traffic is lost and not sent out. I've checked the packet capture on the public WAN interface and I do not see the packets going out to the Internet. I've checked the firewall to see if it was being filtered but no. I'm using the default rules from install.

All VM's are using e1000e as their virtual ethernet adapter and from what I can tell it SHOULD be working. IPv6 connectivity works just fine and I can confirm its not a LAN issue as I can connect normally using the LAN IP addresses to each box inside that.

Topo:

[ISP] -> (eno3) [Xen Host] (br0) -> (xn0) [PFSense] (xn1) -> (br1) -> [client01]

br0 - Bridged adapter for eno3 on the host. Things that are connected here have a public IPv4 address and a virtual MAC set which allow the ISP to route traffic back to the individual IP.

br1 - Bridged adapter for virtual machines + PFSense's LAN interface.

All bridges are located on the Xen host. EDIT: Forgot to mention. I can use ICMP from all guests just fine (ex: ping 8.8.8.8) and those work just fine.

I have a small topology of virtualized machines (one PFSense Firewall and a Windows 7 box for testing connectivity).

The host has two bridge interfaces configured

br0 connects the host WAN (eno3) and PFSense WAN (xn0). This provides the internet connectivity for both hosts. I can successfully connect to both from the outside.

br1 connects the PFSense LAN (xn1) to the virtual adapters of the other guests (Windows 7 for this case).

My problem is that for whatever reason I cannot get hosts inside br1 to ping each other. If I add an address to br1 from the host I am able to send and receive pings to both guest's IP addresses and they can ping the host address. In checking Wireshark I can see the two hosts broadcasting ARP to find the other machine.

but does not ever turn into a successful ping. I've tried disabling the firewall on the Windows 7 client but it still does not work.

So I have a dedicated server from OVH. With that I got a /64 range of IPv6 addresses and 1 public IPv4 address. I purchased a second IPv4 address as a fallback.

The host is running the Xen hypervisor and I already set up the bridge

bridge name bridge id       STP enabled interfaces
br0     8000.0cc47ac4292c   no      eno3
                                    vif4.0
                                    vif4.0-emu
virbr0  8000.525400a2390a   yes     virbr0-nic

I was able to get the VM connected with both the IPv4 and IPv6 address (it will only be using IPv6 though) however as a test I changed the VM's MAC address from the one listed for my fallback and boom, no more IPv6 connectivity but once I set the MAC address back it was working right as rain.

Host ipv6 routing table

2607:5300:61:45b::/64 dev br0  proto kernel  metric 256  pref medium
2607:5300:61:400::/56 dev eno3  proto kernel  metric 202  mtu 1500 pref medium
fe80::/64 dev eno3  proto kernel  metric 256  pref medium
fe80::/64 dev eno4  proto kernel  metric 256  pref medium
fe80::/64 dev br0  proto kernel  metric 256  pref medium
fe80::/64 dev vif4.0-emu  proto kernel  metric 256  pref medium
fe80::/64 dev vif4.0  proto kernel  metric 256  pref medium
ff00::/8 dev eno3  metric 256  pref medium
ff00::/8 dev eno4  metric 256  pref medium
ff00::/8 dev br0  metric 256  pref medium
ff00::/8 dev vif4.0-emu  metric 256  pref medium
ff00::/8 dev vif4.0  metric 256  pref medium
default via fe80::205:73ff:fea0:0 dev br0  metric 1024  pref medium

Guest ipv6 routing table

2607:5300:61:45b::/64 dev eth0  proto kernel  metric 256  pref medium
2607:5300:61:400::/56 dev eth0  proto kernel  metric 256  expires 2591946sec pref medium
fe80::/64 dev eth0  proto kernel  metric 256  pref medium
default via fe80::205:73ff:fea0:0 dev eth0  proto ra  metric 1024  expires 1746sec hoplimit 64 pref medium

Ideally what I'm looking to do is be able to create VM's that will have their own publicly routable IPv6 address without having to have a separate fallback IPv4 address for each one.

The host system is running Gentoo btw.