CPU's on Hyper-V host system is just idling, even though VM's are at full throttle

Bjørn, you're worried about something that isn't an issue. In your case, you have a hypervisor and four virtual machines. One of those virtual machines (the management OS) came with the hypervisor. You then installed three virtual machines.

Ask yourself whether you'd like to see the management OS using CPU time. Ideally, you want it to leave as much CPU time as it possibly can to the three VMs that you set up.

This is exactly what Hyper-V does, assuming that you have installed the "VM Integration Components" in all your VMs. These components make I/O much more efficient by installing drivers that are meant for VMs. Then, unless your VMs are particularly I/O-heavy, your management OS will just sit there waiting for you to interact with it, attempting to use as few resources as possible. This is a good thing.

If you want to see the actual CPU metrics from the hypervisor, not the management OS, use Performance Monitor in the management OS and look for the Hyper-V counters. These will tell you about physical CPU usage.

Doug Luxem (who seems to know a lot about this) answered this well on another thread today. So I'll just steal his text:

First, you have to remember that in Hyper-V that the "host" is called a parent partition and it really just like a virtualized guest with special permissions and roles. Just like any other child/guest, when you open up Task Manager, you can not see the CPU usage of the other children on the server.

Ben Armstrong has a good explanation of this here: http://blogs.msdn.com/virtual_pc_guy/archive/2008/02/28/hyper-v-virtual-machine-cpu-usage-and-task-manager.aspx

Lastly, look at RAM usage in your VMs. Look at disk queue length. These will tell you if you need more RAM. It sounds, though, like your VMs just need more total CPU than the host has to give.

I cannot remember 100% how this worked, but look into the following

I believe the Hyper-V OS (host OS if you will) is just a shell and doesn't display these counters properly, not in perfmon anyway. It could also be that it displays only what the host OS itself is using, and not the VMs.

It's been a while since I worked with Hyper-V but I think this was the baseline of what you're seeing.

Good luck

Ok, as someone running a lot of VM's ;)

• Memory is not the issue, pretty obviously. Ou have to watch IO on the host side (i.e. all totaly) AND on the VM side, but if you manage to allocate 100% cpu within a VM - it pretty obviously does not trach the discs and not use the allocated CPU slots. Not saying more memory is not usefull, but like a physical computer - a VM going into virtual memory heavily would not be able to fully utilize all allocated CPU cycles.

• You say the CPU is at 100%, but the host does barely bulge. How many CPU cores does the HOST have? Iff that is a 4 core, hyper-threading system (exposing 8 cores - sorry, I am totally a AMD guy), a VM using 2 cores at 100% would ONLY show up as - 25% to start on the host. That is already assuming you use the proper way to measur CPU utilization to start with (prpoer Hyper-V counters, or the Hyper-V manager for a short idea). A ONE CORE server on a 8 CORE host CAN - physically - only ever use up 12.5% of the available CPU power to start with. If that makes the VM sluggish from within (which could be - depends on VM internal CPU Prioritization) then - this is like a normal computer being overloaded, but the host barely sees it.

As such, 3 VM's and a 8 core CPU make zero sense to use 1 virtual CPU per host. You basically leave most of your processor power unavailable for the virtual machines. If you go dual core on every server - assuming the server and processes gain from it - this will give you better uitlization. THat said, you could possibl go higher - depending how load is distributed (does SQ LServer do a lot while the other servers are busy?).

Just as example - one of my machines is a 32 gigabyte AMD Opteron system (soon to be 64 gbyte). I rarely get the CPU fully loaded (with only 4 cores) - simply because albeit I run more than a dozen servers on it, most are not busy at the same time, or light load systems (multiple DNS for example). I have another server where TFS (Ms equivalent to TeamCity) Runs, and if I have some action there - things get nasty. Builds, unit tests, scheduled virtual machine lab tests, database unit tests - all use up tons of CPU power. Depending on how busy that is this CAN requier more CPU power. How buy is mostly a "how many users" thing - 2-3 developers rarely keep the server busy most of the time.

So, even if you allocate multiple cores for for example TeamCity - will you do tings like multiple compiles at the same time, or not? If there is one developer checking in, he simply will never check in in two projects at the same time ;) But give it a try ;)

Ok, what I have done so far:

Memory allocated to each server: 2300, 2300 and 2000 megs. This leaves just a little bit free memory on the host and each of the VM's.

4 virtual processors on each server. At any one time, its mostly two servers working, the SQL server either together with the build server, or with the staging server. So that means that probably most of the time 8 virtual processors will be busy at one time, with the last 4 not doing that much.

Virtual Machine Reserve set to 0 on all VM's. Virtual Machine Limit set to 100% on all VM's.

In a desperate attempt to get the CPU's busy I have set the VM's worker processes to Realtime Priority on the Host.

The CPU readings are still very low on the host, but now they are not so intense on the VM's, and they seem more responsive.

This is a short and good post about Hyper-V and CPU usage. I'll leave the explaining to the article.

Furthermore, the best practice for getting optimal performance out of VMs is to make sure you have a 1:1 mapping of logical to physical cores. So if you have 3 VMs and 8 cores, you could do 2 logical cpu's on all VMs, leaving two for the hypervisor/host.

It is generally a bad idea to increase the logical number above the physical number.

Another way to speed up performance is to set up static VHDs or pass through disks.

8 GB of RAM should be more than enough for three of most VMs, but it all depends on the applications running and how much they'd like to work optimally. Check documentation on each.

Hope this helps.