NVIDIA GPU monitoring tools (nvidia-smi, nvtop, or desktop widgets) can crash or freeze on Linux, often due to driver conflicts, permission issues, or misconfigured monitoring daemons. This guide walks you through diagnosing and fixing the most common causes, from checking driver health to adjusting kernel parameters, so you can get stable, real-time GPU stats again.
What You’ll Need
- A Linux system with an NVIDIA GPU and the proprietary driver installed (or the open-source nouveau driver, if you’re using it).
- Root or sudo access to run diagnostic commands and edit system files.
- Access to a terminal emulator (e.g., GNOME Terminal, Konsole, or xterm).
- An internet connection (optional, for downloading driver updates or tools).
Step-by-Step
- Check the GPU driver is loaded and healthy. Run
lsmod | grep nvidiato see if the kernel modules are loaded. If nothing appears, the driver may not be installed correctly. Then runnvidia-smifrom the terminal—if it prints GPU info without crashing, the driver itself is fine; the issue is likely with the monitoring tool you’re using. - Verify driver version and compatibility. Run
nvidia-smi --query-gpu=driver_version --format=csvto see the installed driver version. Check that it’s recent enough for your GPU model and Linux kernel. If you’re on a very new kernel, the driver might be outdated—update it via your distribution’s package manager (e.g.,sudo apt update && sudo apt upgradeon Debian/Ubuntu) or by downloading from NVIDIA’s website. - Check for X11 or Wayland session issues. Some monitoring tools (like nvidia-settings) require an X server. If you’re on Wayland, try running the tool with
X11backend or switch to a different tool. For nvidia-smi, it should work in both, but if it crashes, try running it withsudo nvidia-smito rule out permission problems. - Fix permission issues for /dev/nvidia* devices. Monitoring tools need read access to GPU device files. Check permissions with
ls -l /dev/nvidia*. If they’re not readable by your user, add yourself to thevideogroup:sudo usermod -aG video $USER, then log out and back in. Alternatively, create a udev rule to set permissions automatically. - Test with a different monitoring tool. If nvidia-smi works but a GUI tool like nvtop or a desktop widget crashes, the problem is likely in that tool. Try a lightweight alternative like
nvtop(install via your package manager) orgpustat(Python-based). If those work, the original tool may have a bug or conflict with your desktop environment. - Check for conflicting monitoring daemons. Tools like
nvidia-smican be called by multiple processes (e.g., Prometheus exporters, desktop applets). If two processes try to access the GPU simultaneously, they can crash. Useps aux | grep nvidiato see what’s running. Stop any unnecessary daemons and see if the crash persists. - Review kernel logs for GPU errors. Run
dmesg | grep -i nvidiato look for errors like “NVRM: failed to initialize” or “Xid” errors. These indicate hardware or driver problems. If you see Xid errors, note the code (e.g., Xid 79) and search for it—it may point to overheating, power issues, or a faulty GPU. - Adjust kernel parameters for stability. Some GPUs need specific kernel parameters to avoid crashes. For example, adding
nvidia-drm.modeset=1can help with Wayland. Edit/etc/default/grub, add the parameter toGRUB_CMDLINE_LINUX_DEFAULT, then runsudo update-gruband reboot. Be cautious—only add parameters you’ve researched. - Reinstall the NVIDIA driver cleanly. If all else fails, remove the driver completely and reinstall. On Ubuntu, use
sudo apt purge nvidia-*thensudo ubuntu-drivers autoinstall. For other distros, follow their specific instructions. After reinstalling, reboot and test nvidia-smi again. - Consider using the open-source nouveau driver as a test. If the proprietary driver keeps crashing, you can temporarily switch to nouveau (remove the proprietary driver, reboot). If monitoring tools work with nouveau, the issue is with NVIDIA’s driver. Note that nouveau has lower performance but is more stable for some older GPUs.
Common Mistakes
- Ignoring kernel updates. After a kernel update, the NVIDIA driver may not be recompiled automatically, leading to crashes. Always reinstall or rebuild the driver after a kernel upgrade.
- Using a 32-bit monitoring tool on a 64-bit system. Some older tools are 32-bit only and will crash. Install the 64-bit version or use a modern alternative.
- Running monitoring tools as root unnecessarily. While sudo can bypass permission issues, it can also mask configuration problems. Use sudo only for testing, but fix the underlying permissions for regular use.
- Overlooking GPU overheating. If the GPU overheats, monitoring tools may crash as the driver resets the device. Check temperatures with
nvidia-smi --query-gpu=temperature.gpu --format=csvand ensure proper cooling. - Assuming the tool is at fault. Often the crash is a symptom of a deeper driver issue. Always check dmesg before blaming the monitoring tool.
Tips
- Use the latest stable driver. NVIDIA regularly releases drivers with bug fixes. Check the NVIDIA driver download page for your GPU model.
- Keep your system updated. Ensure your Linux distribution’s kernel and graphics stack are up to date, as they often include fixes for NVIDIA compatibility.
- Create a systemd service for monitoring. If you need persistent monitoring, create a systemd service that starts nvidia-smi or nvtop at boot, so crashes are less likely to go unnoticed.
- Use a script to restart the tool automatically. For a quick fix, wrap your monitoring command in a loop that restarts it if it exits unexpectedly:
while true; do nvtop; sleep 1; done. - Check for BIOS settings. Some motherboards have a “Above 4G Decoding” option that must be enabled for NVIDIA GPUs to work correctly with monitoring tools. Enable it in BIOS if you’re having random crashes.
Troubleshooting
If the steps above don’t resolve the issue, try these additional diagnostics:
- Test with a live USB. Boot from a live Linux USB (e.g., Ubuntu Live) and run nvidia-smi. If it works there, the issue is specific to your installed system. If it crashes there too, it’s likely a hardware problem.
- Check for hardware issues. Run a GPU stress test like
glmark2orfurmark(if available) to see if the GPU crashes under load. If it does, the GPU may be faulty or overheating. - Inspect Xorg logs. If the crash happens when using a GUI tool, check
/var/log/Xorg.0.logfor NVIDIA-related errors. Look for lines containing “EE” (errors) or “WW” (warnings). - Try a different display manager. If you’re using GDM, try switching to LightDM or SDDM temporarily to see if the crash is related to the display manager.
- Disable GPU overclocking. If you’ve overclocked your GPU, revert to stock clocks. Overclocks can cause instability that manifests as monitoring tool crashes.
Frequently Asked Questions
Why does nvidia-smi crash with a segmentation fault?
A segfault often indicates a driver mismatch or corrupted installation. Reinstall the driver, and if the problem persists, check for kernel module conflicts with lsmod | grep nvidia and remove any stale modules.
Can I use nvidia-smi without sudo?
Yes, if your user has read access to /dev/nvidia* devices. Add yourself to the video group or create a udev rule to set permissions. After that, nvidia-smi should work without sudo.
Why does nvtop crash on Wayland?
nvtop may have issues with Wayland’s compositor. Try running it with nvtop --display :0 or switch to a different terminal emulator. Alternatively, use a Wayland-native tool like nvtop with the --wayland flag if supported.
What is an Xid error and how do I fix it?
Xid errors are NVIDIA’s way of reporting GPU hardware or driver errors. The code (e.g., Xid 79) indicates the type. Common fixes include updating the driver, checking power connections, and ensuring proper cooling. Search the code online for specific guidance.
My monitoring tool works after reboot but crashes later. Why?
This often points to a resource leak or a conflict with another process. Check system logs (journalctl -xe) at the time of crash. Also, monitor memory usage of the tool—it may be leaking memory over time.
Should I use the proprietary NVIDIA driver or the open-source nouveau?
For most modern GPUs, the proprietary driver offers better performance and features, including proper monitoring support. Use nouveau only if the proprietary driver is unstable on your hardware, but expect limited monitoring capabilities.
Related
General steps for current mainstream versions; menus can differ by device and version. See the disclaimer or report an error.
More in this bed
- Fix Windows Could Not Prepare the Computer to Boot
- Windows 11 Security Settings: Best Config Guide
- Set a Google Play Budget & Stop Accidental Purchases
- Destiny 2 Error Code Anteater: Fixes & Causes
- View Android Photos on Chromebook: Step-by-Step
- Install Custom Recovery on Samsung Galaxy Trend