Background: I'm a sysadmin at a relatively large university (about 15,000 users in residential space/ResNet). While my group is responsible for administering ResNet space (authentication, DHCP, etc.) the actual network devices are owned and run by another department. Most dorm buildings are on their own router and, by policy, VLANs cannot cross routers.
We're experiencing some strange DHCP problems, most likely related to viruses that run DHCP servers. Our DHCP failover plan is based on our DHCP servers not being authoritative, so when a box with a DHCP server (virus) pops up on a network, everyone starts taking its offers instead of the ones from the actual DHCP box.
Troubleshooting is greatly complicated by the fact that, since we don't have any devices in the subnet or on the VLAN, our only method of identifying the infected user is to physically drive out to the site, hook up to a port, and run a traffic capture (then associate a MAC with a switch port and turn off the port).
So, my idea is to make a few traffic sniffers/probes to install on problem networks (with one interface in the affected network and one in a known good subnet, to communicate back to us).
I've got two theories on how to do this:
- Setup the device with vtund, and (as needed) tunnel all traffic from the interface in the affected network back to one of our boxes.
- Install tcpdump and SShd on the device, write a little script that phones home with the device's known good IP, and then SSH into the device and do a packet capture as needed.
Any alternate theories? Has anyone had to handle a problem like this (specifically capturing traffic on a network that you can get a route to but can't actually hop on). Unfortunately, this needs to be a zero-budget solution (though I have a few Soekris boards with 3 interfaces each lying around).
Thanks, Jason
Can you not just install a machine on the network with a static IP? You said you were in control of DHCP, so just reserve a specific IP for that machine. It doesn't need to get the address from the DHCP server as long as no one else takes it. Of course, you still have the possibility of the bogus DHCP server handing out that address.
You could also log all DHCP offer and ack packets that come from systems other than your official DHCP servers. If you have login access to the switch (you say you don't own them, but you do say that you can "turn off the port"), you could even automate the disabling of the port.
Thanks to everyone for the input. I ended up installing Debian on a Soekris box. It gets DHCP for eth0 via a known good subnet, then phones home with its' IP. The idea is to plug the other two interfaces into bad subnets, and then SSH into the box which has tcpdump, dhclient, nmap, etc. on it.
If anyone finds this in the archives, I've setup a small project page at http://lunchbox.jasonantman.com/ with information on how I built it, as well as filesystem tarballs and a 1GB CF card image ready to use on a Soekris net4801 (or possibly net4501).