sodcalculator

10 Tech How-To

Troubleshoot Slow Linux Server: Step-by-Step Fixes

Guide 10-35171 · Updated · Step by step

To troubleshoot a slow Linux server, start by checking system load, memory, disk I/O, and CPU usage with commands like top, free -h, iostat, and df -h. Then identify and stop the specific process or service causing the bottleneck, and consider recent changes like new deployments or configuration edits. This guide walks through the most common causes and concrete fixes, from runaway processes to misconfigured swap.

First: Identify the Bottleneck

Before changing anything, determine which resource is exhausted. Run uptime to see the load average—if it’s higher than your CPU core count, processes are waiting. Use top or htop to sort by CPU and memory usage; press P for CPU, M for memory. Check disk space with df -h and inode usage with df -i; a full disk or exhausted inodes can cause mysterious slowdowns. For disk I/O, iostat -x 1 shows utilization and wait times, and iotop reveals which processes are reading/writing the most. Network issues can also cause perceived slowness—use ss -s for socket statistics and iftop to see bandwidth usage.

Common Causes and Their Fixes

Here are the most frequent culprits and how to address them:

  • Runaway processes: A single process consuming 100% CPU can stall everything. Use top to find it, then kill it with kill -9 PID or restart the service. Investigate why it spun out of control—check logs in /var/log/syslog or journalctl -u service-name.
  • Memory pressure and swapping: When RAM is full, the kernel swaps to disk, which is drastically slower. Check free -h; if the swap line shows significant usage, you need to free memory. Kill memory-hungry processes, or add more RAM if this is recurring. Adjust vm.swappiness (e.g., sysctl vm.swappiness=10) to reduce swap tendency, but this is a temporary workaround.
  • Disk I/O saturation: Even with free RAM, heavy disk reads/writes can bottleneck. iostat -x 1 shows %util near 100% on a disk. Common causes: log rotation, database queries, or backups. Move logs to a faster disk, optimize queries, or schedule backups during off-peak hours.
  • Full disk or inodes: A 100% full disk can cause processes to hang when they try to write. Delete old logs, clear package caches (apt clean or yum clean all), and remove old kernels. Check inodes with df -i; a huge number of small files (like email spools) can exhaust them even if space is free.
  • Network issues: High latency or packet loss can make services feel slow. Use ping and mtr to test connectivity, and ss -s to check for connection timeouts. Misconfigured firewalls or DNS can also cause delays—test with dig and curl -v.

How to Use top, htop, and ps Effectively

The top command is your first stop. Sort by CPU with P, memory with M, and time with T. Press c to see full command lines—this often reveals the culprit (e.g., a PHP script running wild). htop is more user-friendly with color-coded bars and easier process management. For a one-time snapshot, ps aux --sort=-%cpu | head -20 gives the top CPU consumers. To see threads within a process, top -H -p PID. Remember that top shows instantaneous usage; a process that spikes briefly may not appear in a single snapshot. Use top -b -n 1 for batch mode, or atop for historical data.

Checking Logs for Errors and Warnings

Logs often contain the root cause. Start with journalctl -xe to see recent system errors. For application-specific issues, check /var/log/nginx/error.log, /var/log/mysql/error.log, or the equivalent for your services. Look for repeated errors, out-of-memory (OOM) killer messages, or disk I/O errors. Use grep -i error /var/log/syslog | tail -50 to filter. If logs are rotated, you may need to check archived files. Also, enable persistent journaling if not already: mkdir -p /var/log/journal && systemctl restart systemd-journald.

When to Reboot vs. Investigate Further

Rebooting clears temporary issues like memory leaks and stuck processes, but it’s a band-aid. If the problem recurs, you need to find the root cause. Before rebooting, capture a snapshot: top -b -n 1 > /tmp/top-before-reboot.txt and free -h. After reboot, monitor uptime and logs to see if the issue returns. A reboot is appropriate when a kernel update requires it, or when the server is completely unresponsive and you have no other option. For recurring issues, consider setting up monitoring with tools like monit or nagios to alert you before a slowdown becomes critical.

Preventive Measures and Best Practices

Once you’ve resolved the immediate issue, take steps to prevent future slowdowns:

  • Set up log rotation with logrotate to keep logs from filling the disk.
  • Configure swap space appropriately—typically 1-2x RAM for servers, but avoid relying on it.
  • Use a process supervisor like systemd to automatically restart crashed services.
  • Implement monitoring with netdata or grafana to catch trends before they become problems.
  • Schedule regular maintenance: update packages, clean old kernels, and review cron jobs for resource-heavy tasks.
  • Consider tuning kernel parameters like vm.swappiness and fs.file-max for your workload.

Frequently Asked Questions

Why is my server slow even when load average is low?

Low load doesn’t rule out disk I/O or network bottlenecks. A process might be blocked on disk reads, or the network interface could be saturated. Check iostat for disk utilization and ss -s for network stats. Also, memory pressure can cause swapping even if load is low—verify with free -h.

What does ‘load average’ actually mean?

Load average is the number of processes in the run queue (running or waiting for CPU) averaged over 1, 5, and 15 minutes. A rule of thumb: if the load is higher than the number of CPU cores, processes are waiting. But it’s not a perfect metric—it doesn’t distinguish between CPU, I/O, or memory waits.

How do I check if a specific service is the problem?

Use systemctl status service-name to see if it’s active and any errors. Then check its logs with journalctl -u service-name. You can also run strace -p PID to see system calls the process is making—this can reveal if it’s stuck on I/O or network operations.

Is it safe to kill a process to speed up the server?

It’s generally safe to kill a user process that you own, but for system services, use systemctl restart service-name instead. Killing a critical service (like sshd or mysql) can cause downtime. Always investigate what the process is before killing it—use ps aux | grep PID to see the command line.

Can too many cron jobs slow down a server?

Yes, especially if they run simultaneously at the top of the hour. Use crontab -l to list jobs and stagger their times. Also, check if any job is taking longer than expected—you can add logging to cron jobs to track their duration.

What should I do if the server is completely unresponsive?

If you can’t SSH in, try connecting via the hosting provider’s console (like KVM or IPMI). If that fails, you may need to force reboot from the provider’s panel. Once back, check journalctl -b -1 to see logs from the previous boot—this often reveals what caused the crash.

Related

General steps for current mainstream versions; menus can differ by device and version. See the disclaimer or report an error.

More in this bed

The whole Tech How-To bed →