sodcalculator

10 Tech How-To

How to Secure a Newly Deployed Linux Server

Guide 10-9939 · Updated · Step by step

This guide walks you through the essential steps to harden a freshly deployed Linux server, reducing its attack surface and protecting it from common threats. You’ll learn how to configure user access, firewall rules, SSH security, automatic updates, and basic intrusion detection—everything you need to make your server safe to put on the internet.

What You’ll Need

  • A Linux server (Ubuntu, Debian, CentOS, or similar) with root or sudo access
  • An SSH client (like OpenSSH on Linux/macOS or PuTTY on Windows)
  • Your server’s IP address and root password (or key if already set up)
  • A text editor on the server (nano or vim)

Step-by-Step

  1. Log in and update the system. SSH into your server as root (or a sudo user) and run the package manager update and upgrade commands. For Debian/Ubuntu: apt update && apt upgrade -y. For CentOS/RHEL: yum update -y or dnf update -y. This installs the latest security patches.
  2. Create a new user with sudo privileges. Using root, create a non-root user: adduser yourusername. Then add them to the sudo group: usermod -aG sudo yourusername (on CentOS, use wheel instead of sudo). This prevents you from logging in as root directly.
  3. Set up SSH key authentication. From your local machine, generate a key pair if you don’t have one: ssh-keygen -t ed25519. Then copy the public key to the server: ssh-copy-id yourusername@server_ip. Test logging in with the key before disabling password auth.
  4. Disable SSH password and root login. Edit the SSH config file: sudo nano /etc/ssh/sshd_config. Set PermitRootLogin no, PasswordAuthentication no, and PubkeyAuthentication yes. Also, consider changing the default SSH port (e.g., Port 2222) to reduce automated attacks. Save, then restart SSH: sudo systemctl restart sshd (or ssh on Ubuntu).
  5. Configure a firewall. Enable UFW (Ubuntu) or firewalld (CentOS). For UFW: sudo ufw allow OpenSSH, then sudo ufw enable. For firewalld: sudo systemctl start firewalld and sudo systemctl enable firewalld, then allow SSH: sudo firewall-cmd --permanent --add-service=ssh and reload. Only open ports you actually need (like 80/443 for web).
  6. Set up automatic security updates. On Ubuntu/Debian, install unattended-upgrades and configure it: sudo apt install unattended-upgrades, then sudo dpkg-reconfigure unattended-upgrades and choose Yes. On CentOS, install dnf-automatic: sudo dnf install dnf-automatic, then enable it: sudo systemctl enable --now dnf-automatic.timer.
  7. Install and configure fail2ban. This tool blocks IPs after repeated failed login attempts. Install with sudo apt install fail2ban or sudo yum install fail2ban. Then copy the default config: sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local. In jail.local, set enabled = true for the sshd jail. Restart fail2ban: sudo systemctl restart fail2ban.
  8. Harden system settings (optional but recommended). Edit /etc/sysctl.conf to add network hardening parameters like net.ipv4.conf.all.rp_filter=1 (reverse path filtering) and net.ipv4.tcp_syncookies=1 (SYN flood protection). Apply with sudo sysctl -p.
  9. Install a basic intrusion detection tool. For example, install aide (Advanced Intrusion Detection Environment) to monitor file integrity. sudo apt install aide or sudo yum install aide. Initialize the database: sudo aideinit, then move the database to the active location. Schedule daily scans with cron.
  10. Disable unused services. List running services with systemctl list-units --type=service --state=running. Disable any that you don’t need, like cups or avahi-daemon, using sudo systemctl disable --now servicename.
  11. Set up log monitoring. Configure log rotation and consider sending logs to a remote server or using a tool like logwatch to email you daily summaries. Install logwatch: sudo apt install logwatch and set up a cron job to email reports.
  12. Reboot and verify. Reboot the server: sudo reboot. After it comes back, try SSH with your key and your new user. Check that the firewall is active and that fail2ban is running. Verify that root login is disabled by attempting ssh root@server_ip—it should be refused.

Common Mistakes

  • Locking yourself out by disabling password auth before testing key login. Always test key-based SSH in a separate session first.
  • Changing the SSH port and forgetting to update the firewall rule—this will block your connection.
  • Forgetting to update the system after deployment, leaving known vulnerabilities open.
  • Using a weak password for the new sudo user. Use a long passphrase or, better, rely on SSH keys.
  • Opening too many ports in the firewall. Only expose what’s necessary.
  • Not enabling automatic updates, so critical patches are missed.

Tips

  • Use a passphrase on your SSH private key for extra security.
  • Consider using a non-standard SSH port, but weigh the convenience vs. security trade-off.
  • Regularly review /var/log/auth.log (or secure on CentOS) for suspicious login attempts.
  • Use a configuration management tool like Ansible to replicate these settings across multiple servers.
  • Keep a backup of your SSH config and key files in a safe place.
  • Set up a time-based one-time password (TOTP) for SSH as an extra layer of protection.

Troubleshooting

You’re locked out of SSH. If you disabled password auth and your key doesn’t work, you’ll need to access the server via a web-based console (like Vultr, AWS, or DigitalOcean) or physically. Log in as root, edit /etc/ssh/sshd_config to re-enable password auth, restart SSH, and fix your key setup.

Firewall blocked your connection. If you enabled UFW and forgot to allow SSH, you won’t be able to connect. Use the provider’s console to disable the firewall (sudo ufw disable) or add the rule via console access.

SSH port change didn’t work. Make sure you’re connecting to the new port (e.g., ssh -p 2222 user@server) and that the firewall allows it. Check with sudo ufw status or sudo firewall-cmd --list-all.

Automatic updates not running. Check the status of the service: systemctl status unattended-upgrades or systemctl status dnf-automatic.timer. Look for errors in the logs and ensure the service is enabled.

Fail2ban not banning IPs. Check the jail status: sudo fail2ban-client status sshd. Ensure the jail is enabled and the log path is correct. Sometimes you need to set backend = systemd in the jail config.

Frequently Asked Questions

Q: Is it safe to use a password instead of SSH keys?
A: Passwords are weaker and more vulnerable to brute-force attacks. SSH keys are strongly recommended, but if you must use a password, make it very long and complex and enable fail2ban.

Q: Should I change the default SSH port?
A: Changing from port 22 to a random high port reduces automated attacks, but it’s not a substitute for strong authentication. It’s a good extra layer, but don’t rely on it alone.

Q: How often should I update my server?
A: At least weekly, but automatic security updates should run daily. Always apply critical patches as soon as possible.

Q: What is the difference between UFW and firewalld?
A: UFW is the default on Ubuntu and is simpler to use. firewalld is the default on CentOS/RHEL and uses zones. Both are fine; choose the one your distribution supports.

Q: Do I need a separate user, or can I just use root?
A: Using root directly is risky because any mistake can be catastrophic, and automated attacks target root. Always use a non-root user with sudo privileges.

Q: How do I know if my server has been compromised?
A: Look for unusual logins in /var/log/auth.log, unexpected processes, or files. Tools like aide can alert you to file changes. If you suspect a breach, disconnect from the network and investigate.

Related

General steps for current mainstream versions; menus can differ by device and version. See the disclaimer or report an error.

More in this bed

The whole Tech How-To bed →