Reduce GPU Power Use When Running AI Models

Running large AI models can push your GPU to its limits, drawing maximum power and generating significant heat. This guide shows you practical ways to cut that power draw—lowering your electricity bill, reducing fan noise, and extending the lifespan of your hardware—without necessarily sacrificing too much performance. You’ll learn how to use software settings, system utilities, and a few hardware tweaks to achieve a more efficient setup.

What You’ll Need

A computer with an NVIDIA or AMD GPU (the steps differ slightly by brand)
Administrator access to your operating system
For NVIDIA: the latest NVIDIA driver and optionally NVIDIA System Management Interface (nvidia-smi) (usually included with the driver)
For AMD: the latest AMD Adrenalin driver (or Radeon Software)
Optional: a monitoring tool like MSI Afterburner or HWiNFO to track power draw and temperatures

Step-by-Step

Start with a power limit cap. The single most effective method is to limit the maximum power your GPU can draw. On NVIDIA, open a terminal (Command Prompt or PowerShell) and type nvidia-smi -pl 70 to cap power at 70% of the default maximum. Replace 70 with any value between 50 and 100. On AMD, open Radeon Software, go to Performance > Tuning, enable Manual Tuning, and set the Power Limit slider to a lower percentage (e.g., 70%). This immediately reduces power draw and heat, with a modest impact on performance.
Undervolt the GPU. Lowering the voltage supplied to the GPU core reduces power consumption quadratically (since power ∝ voltage²). Use MSI Afterburner (works with both brands) to open the voltage/frequency curve editor (Ctrl+F). Select a voltage point near the maximum, then flatten the curve so that the GPU runs at a lower voltage for the same clock speed. Start with a conservative offset (e.g., -50 mV) and test stability with your AI workload. This is more advanced but yields significant savings.
Lower the clock speed. Reducing the core clock and memory clock directly cuts power draw. In MSI Afterburner, drag the Core Clock slider down by 100–200 MHz and the Memory Clock by 200–400 MHz. Alternatively, use nvidia-smi -lgc 1500 to lock the core clock to 1500 MHz on NVIDIA. For AMD, use Radeon Software’s tuning sliders. Monitor performance—if your AI model still runs at acceptable speed, you can keep the lower clocks.
Enable power-saving modes in the driver. In NVIDIA Control Panel, go to Manage 3D Settings > Power Management Mode and select “Optimal Power” or “Prefer Maximum Performance” only if needed. For AI workloads, “Optimal Power” lets the GPU downclock when idle. On AMD, enable “Power Saving” in Radeon Software > Preferences > Graphics.
Reduce batch size or model size. If you’re running inference (not training), using a smaller batch size reduces the GPU’s utilization spikes. For training, consider using mixed precision (FP16 or BF16) which halves the memory bandwidth and power per operation. Many frameworks (PyTorch, TensorFlow) support this with a single flag.
Use a GPU that matches the task. If you have multiple GPUs, run your AI model on the most efficient one for the job. For instance, a laptop GPU might use less power than a desktop GPU for small models. Also, consider using the integrated GPU (iGPU) for light tasks and the discrete GPU only when needed.
Set a frame rate limit (for real-time inference). If you’re running real-time AI (e.g., object detection on video), capping the FPS to what you actually need (say 30 FPS) prevents the GPU from rendering at unnecessary high speeds. Tools like RivaTuner Statistics Server (RTSS) can cap FPS globally.
Update drivers. Newer drivers often include power optimizations. Check for updates regularly from NVIDIA or AMD. For example, NVIDIA’s “Studio Driver” is optimized for AI workloads and may be more power-efficient than the Game Ready driver.
Keep the GPU cool. A hot GPU consumes more power due to leakage current. Ensure your case has good airflow, clean dust from fans, and consider repasting the thermal compound if it’s old. Lower temperatures also allow the GPU to maintain lower voltages for the same clock.
Consider using a cloud GPU for heavy tasks. If you’re running large models frequently, offloading to a cloud service (like AWS, Google Cloud, or Lambda Labs) can be more power-efficient overall, since you’re not paying for electricity at home and the cloud provider uses optimized hardware. This is more of a cost-saving tip, but it also reduces your local power draw to zero.

Common Mistakes

Setting the power limit too low. Dropping below 50% can cause instability or crashes, especially under load. Start at 70% and work down gradually.
Undervolting too aggressively. A too-low voltage can cause crashes or corrupted output. Always test with a short run of your actual workload after each step.
Ignoring memory clock. Memory power is a significant part of total draw. Lowering it helps, but too much can cause errors.
Forgetting to reset settings. If you change hardware or drivers, your custom settings may not apply. Check after updates.

Tips

Use a monitoring tool to see the actual power draw in watts. This helps you measure the effect of each change.
For NVIDIA, nvidia-smi dmon shows real-time power, temperature, and utilization in the terminal.
If you’re using PyTorch, set torch.backends.cudnn.benchmark = False to reduce dynamic memory allocation, which can cause power spikes.
Consider using a power meter (like a Kill A Watt) to measure the whole system’s power draw, not just the GPU.

Troubleshooting

System crashes or freezes. This usually means your undervolt or clock reduction is too aggressive. Revert to the last stable settings and adjust in smaller steps.
Performance drops too much. If your AI model runs too slowly, try increasing the power limit or clock speed slightly. You can also try using mixed precision to regain speed without raising power.
nvidia-smi -pl fails with “Unable to set Power Management Limit”. This can happen if your GPU model doesn’t support power limits, or if you’re using a laptop with hybrid graphics. Try using MSI Afterburner instead, or check if the GPU is in a multi-GPU setup that requires specific settings.
Undervolt curve resets after reboot. MSI Afterburner has a “Start with Windows” option and a “Apply overclocking at system startup” checkbox. Make sure both are enabled.
GPU still runs hot. If power is capped but temperature remains high, check your case airflow and fan curves. Set a custom fan curve in MSI Afterburner to ramp up fans at lower temperatures.

Frequently Asked Questions

Will reducing power limit damage my GPU?

No, it’s safe. The GPU is designed to run within a range, and lowering the power limit only reduces the maximum allowed draw. It won’t harm the hardware, but it may reduce performance.

How much power can I save?

Typically, capping at 70% can save 20-30% of GPU power draw with a 5-10% performance loss. Undervolting can save an additional 10-20% depending on your chip’s silicon quality.

Does undervolting void my warranty?

No, undervolting is a software-level change and does not physically alter the card. Most manufacturers allow it, but check your warranty terms to be sure.

Can I use these methods for gaming too?

Yes, the same principles apply. You can cap power and undervolt for gaming to reduce heat and noise, though you may see a slight FPS drop.

Why does my GPU still draw power when idle?

Even idle, the GPU needs power for memory and output. You can reduce idle power by enabling ASPM (Active State Power Management) in BIOS and setting the Windows power plan to “Power Saver”.

Is it better to use a smaller model or a lower power limit?

Reducing the model size (e.g., using a quantized version) often gives a better performance-per-watt ratio than just capping power. If you can, do both.

General steps for current mainstream versions; menus can differ by device and version. See the disclaimer or report an error.

More in this bed

The whole Tech How-To bed →