sodcalculator

10 Tech How-To

Check GPU Usage & Temp on Remote Server

Guide 10-43239 · Updated · Step by step

To check GPU usage and temperature on a remote server, you connect via SSH and run a command-line tool like nvidia-smi (for NVIDIA GPUs) or rocm-smi (for AMD GPUs), which displays current utilization, memory usage, and temperature in one screen. For continuous monitoring, you can use watch -n 1 nvidia-smi to refresh every second, or install a monitoring agent like Prometheus with the NVIDIA DCGM exporter for historical data and alerts.

Why You Need to Monitor Remote GPU Metrics

When you rent a cloud GPU instance or manage a cluster of servers, the GPU is often the most expensive component. High temperature can throttle performance, reducing training speed by 20-50%. High usage without corresponding throughput might indicate a bottleneck elsewhere (CPU, disk, or network). Monitoring these metrics helps you:

  • Detect thermal throttling before it slows your job.
  • Identify if your code actually uses the GPU or accidentally runs on CPU.
  • Plan scaling: if usage is consistently near 100%, you may need a larger instance.
  • Troubleshoot crashes or instability that correlate with overheating.

How GPU Monitoring Works on a Remote Server

Your local computer cannot directly see the GPU on a remote server—the GPU is a physical device attached to that server. You must execute a program on the remote machine that reads the GPU’s sensors and reports them back. SSH (Secure Shell) is the standard way to run such commands. The GPU driver exposes a management API; tools like nvidia-smi query that API and format the output for the terminal.

For NVIDIA GPUs, nvidia-smi is included with the driver. For AMD GPUs, rocm-smi comes with the ROCm stack. Intel GPUs (like Arc) have xpu-smi or intel_gpu_top. These tools read the GPU’s internal sensors (temperature diodes, power meters, utilization counters) and present them in a table.

Step-by-Step: Checking GPU Usage and Temperature via SSH

  1. Connect to the server: Open a terminal (Linux/macOS) or PowerShell (Windows) and run:
    ssh username@server_ip
  2. Verify a GPU is present: Run nvidia-smi -L (NVIDIA) or rocm-smi --showproductname (AMD). If you get an error, the GPU driver may not be installed or the instance may not have a GPU.
  3. View current stats: Run nvidia-smi (or rocm-smi). The output shows a table with GPU index, name, temperature (in °C), utilization (%), memory usage, and power draw.
  4. Monitor continuously: Run watch -n 2 nvidia-smi to refresh every 2 seconds. Press Ctrl+C to stop.
  5. Get a one-line summary (for scripts): Use nvidia-smi --query-gpu=utilization.gpu,temperature.gpu --format=csv to get just the numbers, useful for logging.

For AMD, the equivalent continuous command is watch -n 2 rocm-smi.

Choosing the Right Tool for Your Server

The best tool depends on your GPU vendor and how much detail you need.

  • NVIDIA: nvidia-smi is the default. For per-process usage, add --query-compute-apps=pid,used_memory.
  • AMD: rocm-smi shows temperature, utilization, and power. For per-process, use rocm-smi --showpids.
  • Intel: xpu-smi (newer) or intel_gpu_top (older).
  • Cross-vendor (cloud): Cloud providers (AWS, GCP, Azure) offer web dashboards that show GPU metrics without SSH, but they often lag by a minute.
  • Historical data: Install Prometheus with the NVIDIA DCGM exporter to collect metrics into a time-series database and create Grafana dashboards.

Common Problems and How to Fix Them

“nvidia-smi: command not found” – The driver is not installed or not in PATH. Install the NVIDIA driver (on Ubuntu: sudo apt install nvidia-driver-545 or the version your OS recommends) and reboot. On cloud instances, choose a GPU-enabled image.

Temperature shows 0°C or N/A – This usually means the driver is not fully loaded or the GPU is in a low-power state. Try sudo nvidia-smi -pm 1 (persistence mode) to keep the driver active.

Utilization shows 0% while a job is running – Your code may not be using CUDA/ROCm. Check that your deep learning framework sees the GPU (e.g., in Python: import torch; print(torch.cuda.is_available())).

SSH connection is slow or drops – Use ssh -o ServerAliveInterval=60 to keep the connection alive. For long-running monitoring, use tmux or screen so the command continues after you disconnect.

Comparing Local vs. Remote GPU Monitoring

On a local machine, you can open Task Manager (Windows) or Activity Monitor (macOS) to see GPU stats graphically. On a remote server, you lack that GUI, so you rely on command-line tools. The underlying data is the same, but remote monitoring often requires you to parse text output or set up a web-based dashboard. Some tools like gpustat (Python) colorize the output and show multiple GPUs at once, making it easier to read over SSH.

Safety and Accuracy Considerations

GPU temperatures above 85°C are generally safe but may cause throttling; above 100°C can damage the card. Always ensure adequate cooling, especially in dense server racks. When monitoring remotely, be aware that the temperature reading is taken at the GPU die, not the ambient air—so it may be hotter than the room temperature. Do not rely solely on a single snapshot; use continuous monitoring to catch spikes during heavy workloads.

Frequently Asked Questions

Q: Can I check GPU usage without SSH?
A: Yes, if your cloud provider offers a web console (e.g., AWS CloudWatch, GCP Monitoring), you can view GPU metrics there. Some providers also have agent-based tools that send metrics to a dashboard, but they require initial setup.

Q: What is the normal GPU temperature range?
A: Idle GPUs typically sit at 30-50°C. Under load, 65-85°C is normal. If it exceeds 90°C, check cooling and airflow. Throttling usually begins around 84°C for NVIDIA cards.

Q: How do I monitor multiple GPUs at once?
A: nvidia-smi shows all GPUs by default. For a cleaner view, use gpustat (install with pip install gpustat) which displays a colorized list of all GPUs with usage and temperature.

Q: Can I monitor GPU usage on a Windows remote server?
A: Yes. Use PowerShell with nvidia-smi (if installed) or use Windows Admin Center. Alternatively, run wmic path win32_videocontroller get name to identify the GPU, but for detailed usage you need vendor tools.

Q: What does GPU utilization percentage actually mean?
A: It is the percentage of time in the last sampling interval that the GPU had at least one kernel executing. It does not mean 100% of the compute units are busy—just that the GPU was not idle. For finer detail, look at memory usage and power draw.

Q: How do I log GPU metrics to a file for later analysis?
A: Use nvidia-smi --query-gpu=utilization.gpu,temperature.gpu --format=csv -l 5 > gpu_log.csv to append a new line every 5 seconds. Stop with Ctrl+C. For AMD, use rocm-smi --showuse --showtemp --csv with a loop.

Related

General steps for current mainstream versions; menus can differ by device and version. See the disclaimer or report an error.

More in this bed

The whole Tech How-To bed →