Performance-wise, how much does the L3 cache size on Intel Xeon i7 processors matter for virtualization functions?
I'm picking out a Xeon 1366 Nehalem/Westmere CPU for a server I'm spec'ing to be a low end virtualization host for about 4-5 VM's.
There seems to be a price divide between 4MB L3 Cache and 8MB L3 Cache. How much performance would I get out of stepping up to 8MB vs 4MB?
You will always benefit from having more cache.
However, the question is a matter of price-benefit ratio. Since you're doing low-end virtualisation, with only 4-5 VMs, I would say that you will benefit more from having more RAM. You did not mention how much RAM you were spec-ing.
If the difference in price can get you more RAM, I would recommend that you get yourself more RAM instead.
I think the answer depends a lot on whether you think you'll be overcommitting your CPUs.
If you think that you'll regularly have more than one active vCPU per physical CPU, then more cache might be beneficial.
If, on the other hand, you expect only one or two of your VMs to be active at a time, or you expect to have as many processor cores as virtual processors, then, as sybreon said, you're much better off putting your money into RAM. Once you have enough RAM, put it into disk I/O bandwidth. Then worry about network bandwidth. Then worry about processors.