10 Steps to Securing SSH on a Linux Server | NOC.org

Why SSH Security Matters

SSH (Secure Shell) is the primary remote access protocol for Linux servers. Every internet-facing server with SSH enabled is a target — automated scanners constantly probe port 22, attempting brute force attacks against common usernames and weak passwords. A compromised SSH connection gives an attacker the same level of access as the legitimate administrator, making SSH hardening one of the most critical steps in any Linux security checklist.

The default SSH configuration on most Linux distributions prioritizes ease of use over security. Root login is often permitted, password authentication is enabled, and the service listens on the well-known port 22. Each of these defaults represents an attack surface that should be reduced. The following ten steps address the most impactful SSH hardening measures, ordered from the simplest changes to more advanced configurations.

Step 1: Change the Default SSH Port

Changing the SSH port from 22 to a non-standard port number does not provide strong security on its own — a determined attacker will find the port through a port scan. However, it dramatically reduces noise from automated scanners and opportunistic bots that only target port 22. This reduces log clutter and the volume of brute force attempts your server must handle.

Edit /etc/ssh/sshd_config and change the port directive:

# /etc/ssh/sshd_config
Port 2222

Choose a port above 1024 and below 65535 that is not used by another service. After changing the port, update your firewall rules to allow the new port before restarting SSH — otherwise you will lock yourself out:

# UFW example
sudo ufw allow 2222/tcp
sudo ufw deny 22/tcp

# Then restart SSH
sudo systemctl restart sshd

Always test the new port in a separate terminal session before closing your existing connection.

Step 2: Disable Root Login

Allowing direct root login over SSH is one of the highest-risk configurations possible. Every attacker knows the root username exists, so they only need to guess one thing: the password. Disabling root login forces attackers to guess both a valid username and a password, and it creates an audit trail of which user account was used to gain access before escalating to root via sudo.

In /etc/ssh/sshd_config:

PermitRootLogin no

Before making this change, ensure you have a regular user account with sudo privileges. Test that you can log in as that user and run sudo commands before disabling root login.

Step 3: Use SSH Key-Based Authentication

SSH keys are cryptographically stronger than any password a human can memorize. A 4096-bit RSA key or an Ed25519 key provides authentication strength that is effectively impossible to brute force. Key-based authentication also eliminates the risk of credential stuffing and phishing attacks against SSH.

Generate a key pair on your local machine:

# Ed25519 (recommended — smaller, faster, more secure)
ssh-keygen -t ed25519 -C "admin@example.com"

# RSA 4096-bit (if Ed25519 is not supported)
ssh-keygen -t rsa -b 4096 -C "admin@example.com"

Copy the public key to the server:

ssh-copy-id -i ~/.ssh/id_ed25519.pub user@server -p 2222

Verify you can log in with the key before proceeding to disable password authentication.

Step 4: Disable Password Authentication

Once SSH key authentication is confirmed working, disable password authentication entirely. This eliminates brute force attacks against passwords because the server will not accept them at all.

# /etc/ssh/sshd_config
PasswordAuthentication no
ChallengeResponseAuthentication no
UsePAM no

Restart the SSH service after making changes:

sudo systemctl restart sshd

Warning: If you lose your SSH private key and password authentication is disabled, you will be locked out of the server. Keep secure backups of your private keys, and if your hosting provider offers console access, ensure you know how to use it as a recovery mechanism.

Step 5: Restrict Access with AllowUsers and AllowGroups

Even with key-based authentication, you should limit which user accounts are permitted to log in via SSH. The AllowUsers and AllowGroups directives create an explicit whitelist:

# Allow specific users
AllowUsers deployer admin

# Or allow a group
AllowGroups sshusers

Any user not listed (or not in the specified group) will be denied SSH access regardless of whether they have valid credentials. This is particularly important on shared servers where multiple accounts exist but only a few should have remote access.

To use AllowGroups, create the group and add authorized users:

sudo groupadd sshusers
sudo usermod -aG sshusers deployer
sudo usermod -aG sshusers admin

Step 6: Install and Configure Fail2ban

Fail2ban monitors log files for failed authentication attempts and automatically bans offending IP addresses using firewall rules. It is an essential layer of defense even when password authentication is disabled, because it also catches SSH protocol-level probes and other suspicious activity.

# Install on Ubuntu/Debian
sudo apt update && sudo apt install fail2ban -y

# Install on CentOS/RHEL
sudo yum install epel-release -y && sudo yum install fail2ban -y

Create a local configuration file (never edit the default jail.conf directly, as it is overwritten on updates):

# /etc/fail2ban/jail.local
[sshd]
enabled  = true
port     = 2222
filter   = sshd
logpath  = /var/log/auth.log
maxretry = 3
bantime  = 3600
findtime = 600

This configuration bans an IP for one hour (3600 seconds) after 3 failed attempts within 10 minutes (600 seconds). Enable and start fail2ban:

sudo systemctl enable fail2ban
sudo systemctl start fail2ban

# Check banned IPs
sudo fail2ban-client status sshd

Step 7: Enable Two-Factor Authentication (2FA) for SSH

Adding a second factor to SSH authentication provides defense in depth. Even if an attacker somehow obtains a user's private key, they still cannot authenticate without the time-based one-time password (TOTP). This is especially valuable for SSH IP authentication in high-security environments.

Install Google Authenticator PAM module:

sudo apt install libpam-google-authenticator -y

Run the setup as the user who will use 2FA:

google-authenticator

Follow the prompts to configure TOTP, then edit PAM and SSH configuration:

# /etc/pam.d/sshd — add at the end:
auth required pam_google_authenticator.so

# /etc/ssh/sshd_config
ChallengeResponseAuthentication yes
AuthenticationMethods publickey,keyboard-interactive

This requires both a valid SSH key and a TOTP code for each login. Restart SSH to apply.

Step 8: Enforce SSH Protocol 2 Only

SSH Protocol 1 has known cryptographic weaknesses that allow session hijacking and other attacks. All modern SSH implementations default to Protocol 2, but explicitly enforcing it prevents any fallback to the older protocol:

# /etc/ssh/sshd_config
Protocol 2

On modern versions of OpenSSH (7.4+), Protocol 1 support has been completely removed, so this directive is unnecessary. However, on older systems or hardened compliance environments, explicitly setting it is a best practice.

Step 9: Configure Idle Timeout

Idle SSH sessions that remain open indefinitely represent a security risk. If an administrator leaves a session open and walks away from their workstation, anyone with physical access can use that session. Configure automatic disconnection of idle sessions:

# /etc/ssh/sshd_config
ClientAliveInterval 300
ClientAliveCountMax 2

This sends a keepalive message every 300 seconds (5 minutes). If two consecutive keepalive messages receive no response, the client is disconnected. The total idle timeout is ClientAliveInterval × ClientAliveCountMax = 600 seconds (10 minutes).

Step 10: Enable Logging and Monitoring

Comprehensive logging is essential for detecting and investigating unauthorized access attempts. SSH logs to /var/log/auth.log on Debian/Ubuntu and /var/log/secure on CentOS/RHEL. Increase the logging verbosity:

# /etc/ssh/sshd_config
LogLevel VERBOSE

Key events to monitor:

• Failed authentication attempts: High volumes indicate brute force or credential stuffing attacks.
• Successful logins from unexpected IPs: May indicate compromised credentials.
• Login times outside normal hours: Unusual activity patterns can reveal unauthorized access.
• New SSH key additions: An attacker who gains access may add their own key to ~/.ssh/authorized_keys for persistent access.

Forward SSH logs to a centralized logging system (syslog, ELK stack, or a SIEM) so they cannot be tampered with if the server is compromised.

Complete Hardened sshd_config Example

The following is a consolidated example incorporating all ten steps:

# /etc/ssh/sshd_config — hardened configuration
Port 2222
Protocol 2
PermitRootLogin no
PasswordAuthentication no
ChallengeResponseAuthentication yes
UsePAM yes
AuthenticationMethods publickey,keyboard-interactive
PubkeyAuthentication yes
AllowGroups sshusers
MaxAuthTries 3
ClientAliveInterval 300
ClientAliveCountMax 2
X11Forwarding no
AllowTcpForwarding no
AllowAgentForwarding no
LogLevel VERBOSE
Banner /etc/ssh/banner

Additional hardening options shown above include disabling X11 forwarding, TCP forwarding, and agent forwarding — all of which reduce attack surface if not needed.

Secure Your Server Infrastructure

SSH hardening is a fundamental part of any Linux security checklist, but it is one layer of a complete defense strategy. Combine these SSH controls with network-level protections like a properly configured firewall, regular security patching, and application-layer defenses from a web application firewall to protect your infrastructure from the full spectrum of threats. Explore NOC.org's plans to add professional security monitoring to your servers.