Is virtual machine slower than the underlying physical machine?

The typical experience for a general purpose server workload on a bare metal\Type 1 Hypervisor is around 1-5% of CPU overhead and 5-10% Memory overhead, with some additional overhead that varies depending on overall IO load. That is pretty much consistent in my experience for modern Guest OS's running under VMware ESX\ESXi, Microsoft Hyper-V and Xen where the underlying hardware has been appropriately designed. For 64 bit Server operating systems running on hardware that supports the most current cpu hardware virtualization extensions I would expect all Type 1 hypervisors to be heading for that 1% overhead number. KVM's maturity isn't quite up to Xen (or VMware) at this point but I see no reason to think that it would be noticeably worse than them for the example you describe.

For specific use cases though the overall\aggregate "performance" of a virtual environment can exceed bare metal \ discrete servers. Here's an example of a discussion on how a VMware Clustered implentation can be faster\better\cheaper than a bare metal Oracle RAC. VMware's memory management techniques (especially transparent page sharing) can eliminate the memory overhead almost entirely if you have enough VM's that are similar enough. The important thing in all these cases is that the performance\efficiency benefits that virtualization can deliver will only be realised if you are consolidating multiple VM's onto hosts, your example (1 VM on the host) will always be slower than bare metal to some degree.

While this is all useful the real issues in terms of Server virtualization tend to be centered around management, high availability techniques and scalability. A 2-5% CPU performance margin is not as important as being able to scale efficiently to 20, 40 or however many VM's you need on each host. You can deal with the performance hit by selecting a slightly faster CPU as your baseline, or by adding more nodes in your clusters but if the host can't scale out the number of VM's it can run, or the environment is hard to manage or unreliable then its worthless from a server virtualization perspective.

"Performance" has many aspects. The n00bs measure the boot time of an OS, and say e.g. Windows 2012 is sooooooo great because it boots in 12 sec on real HD, maybe 1 sec on SSD.
But this sort of measure not very useful: performance is equal to OS boot time, but the OS boots once a month so optimizing that doesn't make much sense.

Because it's my daily business I might point out the 4 following parts which made up the "performance"

1. CPU load
   This should be comparable, meaning a task taking 1000 ms on bare metal will execute in 1000 ms process time and probably 1050 ms of clock time in an idle VM environment on the same hardware (some details later). Google the MSDN for processtime and queryperformancecounter and yu can do a thing that can show how much the VM eats up yur CPU time.

2. SQL performance
   SQL performance highly relies on IO to the datastore where the SQL data is stored. I've seen 300% difference between 1'st gen ISCSI which you may find on Buffalo home NAS, then ISCSI with DCE and a real old school FC environment, on all levels. The FC still wins nowadays, because the FC latency is the lowesst archievable which lead to a "copy" of the FC protocol for TCP/IP datacenter enhancements. Here IOps and latency is vital but also IO bandwidth from the server process to the media - depends if the app tends to No-SQL or to Datawarehousing or is in the middle of that like ERP sytems ... Sage KHK for small enterprises, SAP for the huge ones. Both have a CEO view on enterprise financial statistics and when the CEO hits the button he effectively grants vacations for some days when the IO subsystem of the database has weaknesses.

3. Filesystem Access
   Some applications, like video streaming relies on a guaranteed minimum bandwidth, others rely on max IO throughput like just openeing large files in a hex editor, loading a video project into yur favorite movie making prog. Not a typical situation on a vm.... the IOps may also be important to developers. Developers often make use of VMs because developing environment s are very sensitive and so the temptation to do that in a VM is high. Compiling a large project often means reading tons of small files, do the compiler stuff and build an EXE and the accompaining components.

4. Network latency to the client
   Here the usability of WYSIWIG progs like word 2010, Openoffice Writer, LaTEX, GSView and others highly rely on the speed - how fast a mouse action gets from the client to the server. Especially in CAD apps this is important.... but also not a LAN issue, it'S remote access over WAN where this is important.

But - and I speak from the perspective of years of consulting - there are users having the admin password (and they're often employees of a BIG company with a BIG budget and a BIG pocketbook) complaining this and that, but it must be clarified which performance compoent is important to them and which is important from the perspective of the application they use.
It's most likely not notepad, but a highly sophisticated application for engineering this and that, which was also very expenssive and should be moved on the VMware, HyperV or Xenapp and it doesn't perform as expected.

But they don't have in mind that it might run on 1.5 GHz Xeons on blades not made for pure CPU performance, they're built for an average, let's say "optimized for $ per CPU cycle" or "CPU cycles per Watt".

And when we talk about tradeoffs and economisations - that mostly leads to overcommitments. Overcommitments lead to lack of ressources where CPU can be handled pretty well, but lack of memory leads to paging, lack of IO in the core routers leads to increased answer times on everything , and transactional overload on any kind of storage might stop every useful app from responding too quickly. Here monitoring is required, but many software vendors are not able to provide such informations....on the other hand a host with ressources of 3 physical servers can most likely handle 8 virtual machines of the same layout like the physical ones...

The CPU tradeoffs on idle systems often leads to systems performing 50% slower than physical systems, on the other hand nobody is able to install the "real world" os and the "real world" app the customer's IT guys want to move into the VM box. And it takes days (maybe weeks but for sure 42 meetings) to make clear that VM technology can offer flexibility by trading pure CPU speed. This is just built into the CPUs on these blade systems hosting nowadays larger VM environments. Also the memory won't be comparable, also some tradeoffs apply. DDR3 1600 CL10 will have higher memory bandwidth than DDR2 800 ECC LLR - and everyone knows that Intel CPUs profit from this in a different way than AMD cpus. But they're rarely used on productive environments, more in whiteboxes or in datacaenters hosted in 3rd world countries who offer datacenter service for 10% of the price a datacenter in your own homeland may bill yu. Thanks to Citrx a datacenter can be everywhere if it's less than 150 ms of latency between the end user and the datacenter.

And the home users perspective....

Last but not least some people wanna throw away Win7 or XP and trade it for a Linux, and then the gaming question comes up because actually only few games are available for Linux and Windows. Gaming relies higly on 3D acceleration. VMWare 6.5 Workstation and the connected free player can handle DirectX 9, meaning a Doom3 in a VM can run on the host graphic card in full screen. Games are mostly 32 bit apps, so they won't eat up more than 3 GB and mostly not more than 3 CPUs (seen on Crysis). Newer VM players and WS can handle higher DirectX versions and probably OpenGL as well... I gamed UT and UT2004 on VMware 6.5, the host had a ATI Radeon 2600 mobile and a T5440 CPU. It was stable at 1280x800 and playable even on network games....

I would point out that virtualisation can exceed physical performance in certain situations. Since the network layer is not limited to gigabit speed (even though the hardware emulation is of a specific lan card), VM's on the same server can communicate between each other at speeds beyond that of multiple phyiscal servers with average network equipment.

Yes. But that is not the question. The difference is normally neglegible (1% to 5%).

You're trying to compare an operating system, software, and data installed on a certain physical hardware to that same operating system, software, and data installed by itself inside a hypervisor on the same original hardware. This comparison is just not valid, because almost no one does this (at least at first). Of course that would likely be slower. Thankfully, it completely misses the most common point of why you virtualize servers at all.

A better example here is to look at two (or more!) older servers in your data center. Look for servers that are performing reasonably well, but are old now and coming up on their refresh cycle. These servers already perform well on older hardware, and so thanks to Moore's law anything new you get is gonna be way over-spec'd.

So what do you do? It's simple. Rather than buying two new servers you buy just one, and then migrate both of your old servers to the same physical new device. When preparing to purchase your new server, you plan so that you have enough capacity to not only handle the load from both older servers but also any load from the hypervisor (and maybe a little extra, so that you can still get a performance boost and can allow for growth).

In summary: virtual machines provide "good enough" performance for most situations, and help you make better use of your servers to avoid "wasted" computing power.

Now let's stretch this a little further. Since these are old servers, perhaps you were looking at a couple simple $1500 pizza box servers to replace them. Chances are, even one of these pizza boxes could still easily handle the load from both hypothetical older machines... but let's say you decide to spend $7500 or more on some real hardware instead. Now you have a device that can easily handle as many as a dozen of your existing servers (depending on how you handle storage and networking), with an initial cost of only 5. You also have the benefits of only managing one physical server, decoupling your software from your hardware (ie: hardware refresh is now less likely to need a new windows license or cause downtime), you save a ton on power, and your hypervisor can give you better information on performance than you've had in the past. Get two of these and depending on how big you are maybe your entire data center is down to just two machines, or perhaps you want to use the second server as a hot standby to tell a better high-availability story.

My point here is that it's not just about performance. I would never take a perfectly good production server and virtualize it alone to equivalent hardware just because. It's more about cost savings and other benefits you can gain from consolidation, such as high-availability. Realizing these benefits means you're moving servers to different hardware, and that in turn means you need to take the time to size that hardware appropriately, including accounting for the hypervisor penalty. Yes, you might need slightly more computing power in total than if each of those machines were on their own physical device (hint: you actually probably need much less total computing power), but it's gonna be a whole lot cheaper, more energy efficient, and easier to maintaint to run one physical server than it is to run many.

I have been doing some test comparisons of the same software running the same test (.NET-based web application with high volumes of web traffic and considerable SQL Server access). Here's what I've seen:

• The physical machine is better at instantiating classes (which translates to allocating memory at the system level) - this makes sense to me because physical machines do this through memory management hardware and VMs do this through software (with partial hardware assist) (On the VM, the app spent a significant amount of time in its constructors (where the memory is allocated (and nothing else is done), on the physical machine, the constructors weren't even included in the top 1000)
• When you are in the middle of a method, the two are about equivalent - this is probably how most of the benchmarks are constructed that show the two "being the same"
• When you access a network controller, the physical beats out the VM a little - again, the physical doesn't have very much sitting in between the .NET process and the hardware. VM adds adds other "stuff" that each transaction needs to travel through.
• Really, the same thing applied to disk access (the SQL Server was on another machine) - the difference is very small, but when you add them all up, it is noticeable. This could have been caused by the slower network access or by a slower disk access.

I can easily see how someone could build benchmarks that prove they are 1% different or the same or where VMs are faster. Don't include anything where your process takes advantage of the benefits of the local hardware support where the VM needs to simulate it in software.

I have just upgraded to an SSD (OCZ Vertex 2) and I run my XP VM development environment on it, I am a software developer. One thing I have noticed is that when I launch a program (one big enough to take time to load), one core of the virtual CPU pegs out. This happens when loading IE also. Since the CPU pegs out, I assume the bottleneck is the CPU and not the SSD. But it seems odd, I have a feeling that if the same thing were done on a physical machine that it would load faster and my feeling is that there is some extra processing overhead VMWare is doing that is consuming CPU on disk access.

One example, I use Delphi and on a physical machine with a regular HDD it can take 20 seconds to start from a cold boot. In the VM running off an SSD, it loads in 19 seconds from a cold start. Not much difference, I bet if the SSD were on the physical machine it would load faster. Howevere I did not check the CPU usage on the physical machine, its possible the CPU were the bottleneck there as well.

But the feel of the VM is that disk access taxes the VM.

Obviously a virtual machine is slower than the physical machine. But when you're in this scenario you have to evaluate what is optimal to cover your needs. If you need only one system and you need it to be fast, then install it directly to the hardware. In the other side, if you need flexibility, scalability (and all other virtualization benefits :P) deploy a VM. It will be slower, but IMHO in some cases it's justified and the performance is not significantly slow.

It seems Microsoft has done some benchmark testing using BizTalk server, and SQL Server in different configurations in this regard. See link below:

http://msdn.microsoft.com/en-us/library/cc768537(v=BTS.10).aspx

Ideally Virtual PC performance is at:

CPU: 96-97% of host

Network: 70-90% of host

Disk: 40-70% of host