I have a Win2k8 guest running on ESXi 4.1. The host has 12 cores and the problem happens even if the guest is the only VM on the host. We have 4 cores dedicated to the guest.
We noticed that network starts chocking when the CPU load goes up. After some testing we noticed that when running a simple CPU hogging tool set up to run 3 threads at 100% the regular CPU load goes to 75% (like it should) and the "kernel times" graph in task manager goes up to 25%.
My intuition tells me that the network problem and kernel times problem are the same. This is confirmed by another similar VM we created on the same host which doesn't have either of the problems.
VMWare tools are installed and the NIC is e1000.
What else can we do to troubleshoot this?
You want to use the most advanced network drivers you can for every guest. Windows server 208 r2 supports the paravirtualization drivers. Paravirtulized drivers present the least overhead to both guest and host. If you are forcing the host to completely emulate a piece of hardware, you are forcing CPU utilization. Switch to vmxnet3 and see if that reduces the workload and increases performance. You also mention that you are looking at the guest for performance issue. This is a lie. you should always reference the vmware performacnce counters instead of the guest performance counters as the system is virtualized. as an example you might see the cPU at 100% but in vmware you will realize that it's 100% of 500mhz that the machine is currently using.
If you haven't already destroyed the guest, you might try some basic OS troubleshooting (e.g. chkdsk, drivers, etc.) to see if it can be repaired. I recently ran into an issue with a guest instance reporting two different CPU/core states.
The guest was assigned 4 cores, but was only showing 3 on the task manager while their was 4 loaded up in the device manager. A chkdsk found errors. When the errors were corrected, the CPU/cores reported correctly throughout and all prior idiosyncrasies with that guest disappeared.