I appear to have some fundamental misunderstanding of how VLANs work on Linux, and I'm hoping the good people here can educate me.
Cast: One Cisco 3560, one VLAN, and one Linux box [1].
Cisco --------------- Linux
ge0/1 eth0
The Cisco has a Vlan 37 interface, with IP address 10.40.37.252/24. I want to place 10.40.37.1/24 on the Linux box.
When the Cisco de-encapsulates vlan 37, everything works fine [2]:
# Cisco
interface Vlan37
ip address 10.40.37.252/24
interface GigabitEthernet 0/1
switchport mode access
switchport access vlan 37
# Linux
ip link set eth0 up
ip addr add 10.40.37.1/24 dev eth0
$ ping 10.40.37.252 && echo It works
However, when I set the port to trunking and assign vlan 37 on the Linux side, it stops working:
# Cisco
interface GigabitEthernet 0/1
switchport trunk encapsulation dot1q
switchport mode trunk
! [3] [4] [7]
# Linux
vconfig add eth0 37
ip link set eth0.37 up
ifconfig eth0 0.0.0.0 up # ensure no address
ip addr add 10.40.37.1/24 dev eth0.37
$ ping 10.40.37.252 || echo Why does this not work
What am I missing here?
Edit: Solutions:
Shane's question about the mac address table led me to a solution: Use "ip addr" to set different unique L2 (MAC) addresses on each of the VLAN sub-interfaces, and it suddenly works.
Another possible solution that I didn't try (because my hardware is too old) is using "ethtool" to disable VLAN offloading by the NIC itself, and forcing the kernel to deal with the tags.
Thank you Shane!
Edit: More info as per comments:
The overall goal is to have three vlans (public, private, oam&p) terminating on three individual IP addresses on the linux box, with different applications binding to the local addresses. I can expand further if necessary, but I'm trying to keep the problem description and discussion simple, since before I can have three vlans working, I kind of need one to be working. :)
Antoine --> ifup versus ifconfig makes no difference.
Pepoluan --> I'm assuming this is what you were looking for. Note the lack of references by phy drivers is apparently normal. [5]
$ lsmod | grep 802
8021q 25545 1 cxgb3
Handyman -->
$ ifconfig eth0
eth0 Link encap: Ethernet HWaddr 00:17:08:92:87:22
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 [...]
TX packets:31932 errors:0 dropped:0 overruns:0 carrier:0
$ ifconfig eth0.37
eth0.37 Link encap: Ethernet HWaddr 00:17:08:92:87:22
UP BROADCAST RUNNING MULTICAST MUT:1500 Metric:1
RX packets: 0 [...]
TX packets:32024 errors:90 dropped:0 overruns:0 carrier:0
$ cat /proc/net/vlan/config
VLAN Dev Name | VLAN ID
Name-Type: VLAN_NAME_TYPE_RAW_PLUS_VID_NO_PAD
eth0.37 | 37 | eth0
Chuck --> wireshark and/or tcpdump do not show the tags, but this is apparently a normal limitation on Linux, due to the processing order of vlan handling and pcap in the kernel [6]. Also, the untagged VLAN is set to 1 [7].
[1] I've tried this with both CentOS 5.5 and Ubuntu 11.04, and both have the same issue.
[2] Note the configs are not a cut&paste, so any typos here are simply my bad memory.
[3] "nonegotiate" on or off has no effect on the problem.
[4] Vlan 37 is shown as active & non-pruned on the link, so "allowed" is not the problem.
[5] serverfault: Enabling 8021q on a nic
[6] http://wiki.wireshark.org/CaptureSetup/VLAN#Linux
[7] The native (untagged) VLAN is 1. Manually setting it with "switchport trunk native vlan 1" has no effect.
Do you want the host to have access to just vlan 37 or do you want the host to have access to multiple vlans?
This IOS configuration means set the native (that's untagged) vlan to 37.
On the linux side, the vconfig command creates an interface alias for traffic tagged as vlan 37.
Do you see the problem? The switch is sending your host untagged traffic and the host is looking for/generating tagged traffic.
You either need just use eth0 in the native vlan 37 or change the switch conf so that it's passing tagged traffic, eg.
On fairly old IOS devices you man need to set the truck encapsulation to 8021q as they will default to ISL.
Now I am not an expert with the Linux side of this, by going by my Switching knowledge, do you have the eth0 interface on the Linux machine configured for dot1q trunking? I dont know if there is a provision for multiple vlans to operate within a Linux machine, but I assume that you have a single interface which is configured to work as part of Vlan37, thus essentially making your Linux machine's NIC to be an access port. An access port cannot communicate directly with a trunk port, it wont be able to create or understand the trunking encapsulation.
From what I understand you want the Linux machine to work within Vlan 37. Just revert the ge0/1 port to be an access port under Vlan37 and routinely assign the linux machine any IP under the subnet of Vlan37. You really do not need trunking which is used only to convey multiple Vlan information over a single link.
I think the issue is with your switch config. After setting the port to trunk mode with 802.1q tags you need to configure the switch to send vlan 37 as tagged traffic and you may also need to set the port to use another vlan for non-tagged traffic. When I set this up I also had to set which vlan were allowed/denied on that port. My IOS is a bit rusty but I think this is what you are looking for.
You should also be able to confirm your switch config using wireshark on eth0 since it will show you the VLAN tags on the packets. The Cisco LLDP packets may also give you a clue to what port ge0/1 is doing.