Home / ubuntu / Questions / 1164934
In Process
amanusk
Asked: 2019-08-12 01:17:28 +0800 CST

How to turn off Nvidia GPU on a headless server?

  • 772

I am running a headless server with an Nvidia GPU. Even when the GPU is not doing any work, it is consuming about 25 Watts of power:

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 430.26       Driver Version: 430.26       CUDA Version: 10.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 950     Off  | 00000000:01:00.0 Off |                  N/A |
|  0%   61C    P0    26W / 110W |      0MiB /  2001MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

Is there a way to completely turn off power delivery to the GPU when it is not in use?

I tried sudo prime-select intel Which does cause nvidia-smi to stop working, but a power meter connect to the wall shows exactly the same power consumption with either intel or nvidia selected.

Completely removing the GPU reduces the power consumption by about 30 Watts as expected.

The main purpose is to save power and costs during idle operations, with an option to spin up the GPU when it is needed (remotely via ssh).

power-management server nvidia gpu headless

3 Answers

  1. It can be observed that the GPU is using Performance mode P0, where the power consumption of the card is very high. In Idle, it should ideally throttle back to P8 mode, where it will consume less power.

    I follow the following steps to change the behaviour. Assuming that you are using Linux:

    You would need sudo privileges to the machine. First, enable the persistence mode:

    sudo nvidia-smi -pm 1

    Then, reset the clocks for the GPU:

    sudo nvidia-smi -rgc

    Note: To know more about these commands, please type

    nvidia-smi --help
    • 0

  2. These OPTIMUS combo solutions are made with the Windows operating system in mind.

    From previous observation, NVIDIA cards seem to shut down completely on Windows OSs. My assumption is that BIOS handles calls from a Windows OS differently and it should.

    I do not have one of these right now to test this method but it's worth a try nonetheless.

    The method is to first disable ACPI OSI built-in OS vendor strings then report Windows 7 as the current OS. Sort of lying to the BIOS... Linux kernels do this all the time by default, but this is a more controlled way of doing it.

    To do that, first edit /etc/default/grub like so:

    sudo nano /etc/default/grub

    Then, add the options acpi_osi=! and acpi_osi=\"Windows 2009\" to GRUB_CMDLINE_LINUX_DEFAULT like so:

    GRUB_CMDLINE_LINUX_DEFAULT="quiet acpi_osi=! acpi_osi=\"Windows 2009\""

    Then, save and close the file by pressing Ctrl+x then prss y then press Enter.

    After that, update GRUB like so:

    sudo update-grub

    Finally, reboot your system and test if disabling the NVIDIA card is done differently now.

    This method, however, might have higher success levels with the open source driver nouveau and VGA Switcheroo than with the proprietary NVIDIA drivers.

    For further reading, please see kernel parameters.

    • 0

  3. You could try with one of the PowerMizerModes available, to see if that brings your system into state P8

    $ nvidia-settings -a "[gpu:0]/GPUPowerMizerMode=<your no.>"

    Note that PowerMizerModes and P-states available vary across systems. Use

    $ nvidia-settings -q GpuPowerMizerMode

    to check this.

    Or you can also try forcing some parameters

    1. The power limit via nvidia-smi -i 0 -pl 10 (not sure it would work).
    2. GPUGraphicsClockOffset, GPUMemoryTransferRateOffset, via nvidia-settings.
    3. Via xorg.conf, ref.

    You may need to reboot to test options.

    Related:

    1. nvidia - high power consumption on idle GTX 1080 cards
    2. https://gist.github.com/DavidMetcalfe/fabbd2ddcb6ba4927269f6f0db31a92f
    3. https://forums.developer.nvidia.com/t/gtx-1070-driver-4-15-25-performance-mode-p0-does-not-automatically-downshift-to-idle-p8-after/69238
    4. https://github.com/devonkinghorn/linux-nvidia-dynamic-power-management-setup
    • 0