Does CPU power management affect server performance?

I'm not sure about servers, but the current thinking in embedded devices is not to bother with steps between low-power and flat-out because the extra time involved will eat your power savings, so basically they run low power until they get any real amount of cpu load at which point they flip over to fastest-possible so they can finish the job and get back to idling at low power.

I have always turned off any type of power management on servers. I am curious to what others have experienced, but I always assumed that if the server is under-clocking, there will always be some delay to 'step up' the CPU to 100%, and in a data-center setting any delay like this is unacceptable.

The data you provided seems to support this assumption. So, I have not done any specific testing but it would seem that you should not use any power-saving technology within Windows or the BIOS. I even turn off the 'shut off Network' and PCI card settings to be ultra conservative.

How Much Power will this actually Save you:
If you do decide that this feature might put the stability of your servers at risk (I have on experience with this), then you might look elsewhere for the energy savings.

I would try to find out just how much energy this might save for the amount of servers you have (Although perhaps you already did this). Since the graph you posted in your answer is percentages, for your company, the savings might actually be very little actual power. If you don't have many servers, it might not actually be that much, and getting motion activated lights or something like that in your office might save more energy (even though that is not as marketable).

I remember reading a few years back about one of the major American car companies (forget which) having pressure to change the emissions of the exhaust on their cars. Instead, the company showed that if it capped some of its factories, that would be much cheaper for them as well as resulting in far more emissions savings.

Don't Forget Disks:
Also, you might want to check that these power savings feature don't spin down the disk(s) if they are not used. Maybe for a little while all the SQL query results would be in RAM, the disk would be used and go to sleep (Not sure if it works like that though)? If this can happen, there would be a big performance penalty while everything spins up again.

you will give up some performance with CPU power management enabled, unless your server is always under so much load that the power management has effectively turned itself off. This result surprised me.

Preface: I'm making some leaps/generalizations about Intel Xeons and their power saving performace with SpeedStep. In reading about the Intel Xeon "Yorkfield" 45nm CPUs, Enhanced Intel SpeedStep Technology (EIST) and Enhanced Halt State (C1E) seem to be the real culprit of the situation. I would agree with your statment in believing that turning on such power management features would aid the conservation of energy but when the CPUs needed the energy under load that the system would return to a normal voltage clock speed settings. It appears that EIST and C1E have some side effects that aren't intuitively implied when using either/or option in the BIOS. After crawling through numerous overclocking websites, it appears that these two settings in the BIOS cause quite a bit of frustration.

From http://www.overclock.net/intel-cpus/376099-speedstep-guide-why-does-my-processor.html:

C1E (Enhanced Halt State): C1E is the simpler of the two components. It can be enabled or disabled in the BIOS, and performs independently of the operating system. C1E has two configurations - idle, and load. When CPU usage is relatively low, this feature lowers your processor's multiplier to its lowest setting (usually 6x) and slightly lowers its vCore. During a CPU-intensive application, it will raise the multipler to its maximum value, and will provide a small boost in vCore to compensate. In our example, C1E will make your processor run at either 6x or 9x the FSB.

EIST (Enhanced Intel SpeedStep Technology): This is a very robust feature and has a wide variety of power-saving capabilities. Like its simpler cousin, EIST can affect both your CPU's voltage and it's multiplier - however, it has many more levels of configuration. Instead of a simple "slow or fast" setting, SpeedStep can utilize all of the available multipliers. In our example case, EIST will allow your processor to run with a multiplier of 6, 7, 8, or 9, and chooses which one to use based on how much demand your CPU is under. EIST is controlled by Windows, and utilizes the different "power schemes" you may have seen in your control panel.

While adjusting your performance settings for "high performance" is probably the best setting for a database server, I'm fairly certain either EIST and/or C1E caused the CPUs to under perform even though they should have gone back to normal settings when the load increased substantially. The big caveat to me appears to be "what is a substantial load?" According to the overclockers.net site they claim that EIST uses those "power schemes" settings for how to manipulate your CPU settings. But there's no indication of percentage of load or for how long to know when to turn the CPUs back to normal voltage.

Again, I'm by no means an expert on the subject matter for Intel CPUs but I would wager that adjusting these two settings might get you the power savings you want and the performance you should get, but sticking with the "maximum performance" setting is just as effective without the need to reboot.

The fast answer is: Of course power saving will affect performance.

The longer answer is no fun. Basically, try a setting, test performance, and decide what you can live with.

Applications and systems are so very complicated that there is no cut and dry answer here, other than "yes, reaction time and other system speeds will be affected." If it is that much slower than the hard drive, or the network -- well, you get the idea. Test in reality.

I always try to VM as many servers as I can but where I have to 'bare-metal' a server it's usually as I need or want totally consistent performance. So for these business-critical machines I NEVER switch on anything power-saving related whatsoever for exactly the reasons you're experiencing.

***bang-goes-my-green-credentials*

A few things:

1. Check in the BIOS to make sure that power management is under OS control. It could be possible that it's set to be managed by the firmware, and therefore using dumb, unoptimal processor power management.

2. Check to see if there are any power management-related hotfixes that you might be missing. There were quite a few notable ones back in the day when Vista/Server 2008 came out.

3. Check the detailed configuration for Balanced. It's possible that another power saving feature is causing the reduced performance. In theory, the performance hit from EIST should be negligible, though then again, an SQL database has a unique footprint, and it's conceivable that it gets disproportionately affected by processor power management.

Some information from Microsoft (Word Doc format, unfortunately)

Improve Energy Efficiency and Manage Power Consumption with Windows Server 2008 R2

Windows Server 2008 is more energy efficient overall than its predecessor, Windows Server 2003. By default, Windows Server 2008 runs the “Balanced” power savings plan, which aims to keep performance high while saving power whenever possible. This means that Windows Server 2008 uses less power than does a baseline installation of Windows Server 2003. Because the “Balanced” mode maximizes out-of-the-box (OOB) power efficiency, Microsoft highly recommends leaving the default “Balanced” settings selected in most cases.

Windows Server 2008 includes two additional default modes, “Power Saver” and “High Performance,” which have different power and performance goals and may be appropriate in some situations. The “High Performance” mode may be appropriate for servers that run at very high utilization and need to provide maximum performance, regardless of power cost. The “Power Saver” mode can be used for little-utilized servers that have more performance capability than they really need; using “Power Saver” in this situation may provide incremental power savings.

These particular hardware-level CPU power saving features are the same under any OS of course, it's just a question of whether or not you turn them on.

The power savings graph of no CPU power management, versus CPU power management:

We're clear that (and this graph shows that) at high utilization levels, CPU power management is automatically turned off. What I'm not clear on, however, is whether at low utilization levels there is impact to overall server performance, e.g. turnaround time on simple-ish SQL Server queries.

You should never, ever resort to using the Windows settings or the Bios Speedstep which comes on Intel processors and there's also an AMD equivalent. These can cause issues, and I've seen such issues where with Speedstep the CPU clock would keep bouncing up and down erratically even though the CPU resource usage was consistent.

If you want to be greener and save power, use low power processors, designated with the L character before the model name, such as L54XX series and L55XX series from Intel.

EDIT: I'm sorry if I gave the impression that this feature will always fail, I've just been burned by it, and in a mission critical system I can't have this sort of stuff happen, so I just try to stay away from it.

When you're talking about performance on a server, there are a few different ways of looking at it. There's the apparent response time (similar to network latency) and the throughput (similar to network bandwidth).

Some versions of Windows Server ship with Balanced Power settings enabled by default. As Jeff pointed out. Windows 2008 R2 is one of them. Very few CPU's these days are single core so this explanation applies to almost every Windows server you will run into with the exception of single-core VM's. (more on those later).

When the Balanced power plan is active, the CPU attempts to throttle back how much power it's using. The way it does this is by disabling half of the CPU cores in a process known as "parking". Only half of the CPU's will be available at a time so it uses less power during times of low traffic. This isn't a problem in and of itself.

What IS a problem is the fact that when CPU's are unparked, you've doubled the available CPU cycles available to the system and suddenly unbalanced the load on the system, taking it from (for example) 70% utilization to 35% utilization. The system looks at that and after the burst of traffic is processed, it thinks "Hey, I should dial this back a bit to conserve power". And so it does.

Here's the bad part. In order to prevent an uneven distribution of heat & power within the CPU cores, it has a tendency to park the CPU's that haven't been parked recently. And in order for that to function properly, the CPU needs to flush everything from the CPU registers (L1, L2 & L3 cache) to some other location (most likely main memory).

As a hypothetical example, let's say you have an 8 core CPU with C1-C8.

• Active: C1, C3, C5, C7
• Parked: C2, C4, C6, C8

When this happens, all of them become active for some period of time, and then the system will park them as follows:

• Active: C2, C4, C6, C8
• Parked: C1, C3, C5, C7

But in doing so, there's a good amount of overhead associated with flushing all of the data from the L1-L3 cache to make this happen so that weird errors don't happen to programs that were flushed from the CPU pipeline.

There's likely an official name for it, but I like to explain it as CPU thrashing. Basically the processors are spending more time doing busy work moving data around internally than they are fielding work requests.

If you have any kind of application that needs low latency for its requests, you need to disable the Balanced Power settings. If you're not sure if this is a problem, do the following:

1. Open up the "Task Manager"
2. Click the "Performance" tab.
3. Click "Open Resource Monitor"
4. Select the "CPU" tab
5. Look at the right-side of the window at the various CPU's.

If you see any of them getting parked, you'll notice that half of them are parked at any given time, they'll all fire up, and then the other half get parked. It alternates back and forth. Thus, the system CPU's are thrashing.

Virtual Machines: This problem is even worse when you're running a virtual machine because there's the additional overhead of the hypervisor. Generally speaking, in order for a VM to run, the hardware needs to have a slot in time available for each of the cores at each timeslice.

If you have a 16 core piece of hardware, you can run VM's using more than 16 total cores but for each timeslice, only up to 16 virtual CPU's will be eligible for that time slice and the hypervisor must fit all of the cores for a VM into that timeslice. It can't be spread out over multiple timeslices. (A timeslice is essentially a set of X CPU cycles. It might be 1000 or it might be 100k cycles)

Ex: 16 core hardware with 8 VM's. 6 have 4 virtual CPU's(4C) and 2 have 8 virtual CPU's(8C).

Timeslice 1: 4x4C Timeslice 2: 2x8C Timeslice 3: 2x4C + 1x8C Timeslice 4: 1x8C + 2x4C

What the hypervisor cannot do is split half of the allotment for a timeslice to the first 4 CPU's of an 8 vCPU VM and then on the next timeslice, give the rest to the other 4 vCPU's of that VM. It's all or nothing within a timeslice.

If you're using Microsoft's Hyper-V, the power control settings could be enabled in the host OS, meaning it will propagate down to the client systems, thus impacting them as well.

Once you see how this works, it's easy to see how using Balanced Power Control settings causes performance problems and sluggish servers. One of the underlying issues is that the incoming request needs to wait for the CPU parking/unparking process to complete before the server is going to be able to respond to the incoming request, whether that's a database query, a web server request or anything else.

Sometimes, the system will park or unpark CPU's in the middle of a request. In these cases, the request will start into the CPU pipeline, get dumped out of it, and then a different CPU core will pick up the process from there. If it's a hefty enough request, this might happen several times throughout the course of the request, changing what should have been a 5 second database query to a 15 second database query.

The biggest thing you're going to see from using Balanced Power is that the systems are going to feel slower to respond to just about every request you make.