Mastering Secure SSH Access in Ubuntu: A Step-by-Step Guide
A single misstep in SSH configuration—leaving password authentication open, missing a permissions setting—can turn remote access into the weakest link in your Ubuntu deployment. For production workloads, neglecting SSH hardening is an invitation for persistent attacks. Here’s how experienced Linux engineers build robust, resilient, and auditable SSH setups on Ubuntu (tested with 22.04 LTS, notes on older versions where needed).
1. Installing and Verifying OpenSSH Server
Most cloud Ubuntu images (AWS, GCP, Azure) enable openssh-server by default. For on-prem or minimal installs, verify all components are present:
sudo apt update
sudo apt install --assume-yes openssh-server
Check service state:
sudo systemctl status ssh
Output should include:
Active: active (running)
If not, correct with:
sudo systemctl start ssh
and guarantee auto-start:
sudo systemctl enable ssh
Side note: /var/log/auth.log is your friend for SSH troubleshooting—review immediately if connections fail.
2. Basic SSH Connection: Local to Remote
No mystery here—SSH defaults to port 22. Connect from a secure workstation using:
ssh engineer@10.160.0.10
If you see:
Permission denied (publickey,password).
either keys aren’t set up, the username is wrong, or the remote host is not running sshd.
Windows 10+ ships with OpenSSH in PowerShell; older Windows uses PuTTY. Mac and all Unix variants use the built-in ssh client.
3. SSH Key Authentication: Eliminating Password Relics
Passwords are a brute-force liability. SSH keys (ideally ed25519 for modern OpenSSH, RSA discouraged unless legacy) radically reduce attack vectors.
Generate locally:
ssh-keygen -t ed25519 -C "admin@corp.example"
Use a distinct passphrase. Files land in ~/.ssh/; public key is .pub.
Distribute public key:
ssh-copy-id engineer@10.160.0.10
(If ssh-copy-id unavailable, append .pub to ~/.ssh/authorized_keys manually—permissions matter, see below.)
Test:
ssh engineer@10.160.0.10
Prompt for "Enter passphrase" means the key is working.
Gotcha:
On Ubuntu, ~/.ssh/authorized_keys must be chmod 600, and ~/.ssh/ must be 700, else authentication will silently fail:
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys
4. Lock Down PasswordAuthentication
Once keys confirmed, immediately disable passwords to prevent fallback:
sudo nano /etc/ssh/sshd_config
Set or ensure:
PasswordAuthentication no
ChallengeResponseAuthentication no
For environments where PAM modules are required (e.g. LDAP, 2FA), set UsePAM yes. For standard key-only systems, you can use:
UsePAM no
Restart to apply:
sudo systemctl restart ssh
If you lose your private key at this point, you lose admin access unless you have console or rescue options. Plan for this (e.g., out-of-band access, or alternate keys).
5. Changing SSH Port (Obfuscation Layer, Not True Security)
Port 22 is the default; bots scan it by the million. Shifting up (say, port 4422) reduces noise in logs—does not prevent a targeted attack.
Config:
sudo nano /etc/ssh/sshd_config
Set:
Port 4422
Update firewalls (UFW example):
sudo ufw allow 4422/tcp
Restart:
sudo systemctl restart ssh
Connect:
ssh -p 4422 engineer@10.160.0.10
Known issue:
SELinux or AppArmor users may require additional policy edits—failure to do so silently blocks traffic, logs reveal cause.
6. Automated Intrusion Response: Fail2ban
Even with keys, attackers will attempt automated brute force. Fail2ban inspects logs and blocks repeat offenders.
Install:
sudo apt install --assume-yes fail2ban
sudo systemctl enable --now fail2ban
Default jail (/etc/fail2ban/jail.conf) covers sshd; customize if you need non-default ports:
[sshd]
port = 4422
maxretry = 3
bantime = 12h
Service reload:
sudo systemctl reload fail2ban
Check bans:
sudo fail2ban-client status sshd
Fail2ban is not foolproof—see logs for evasion patterns.
7. Advanced Server Hardening
A few additional changes (high value, low cost):
-
Limit which users can SSH:
/etc/ssh/sshd_config:AllowUsers engineer deploysvcOnly named users can connect.
-
Disable root login:
PermitRootLogin noNote: Some legacy cloud images default to
PermitRootLogin prohibit-password—double-check. -
Client idle timeouts:
ClientAliveInterval 300 ClientAliveCountMax 0This disconnects idle sessions after 5 minutes.
-
Preferred ciphers:
Modern OpenSSH versions use strong ciphers by default, but you may enforce:Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.comEd25519 keys + chacha20 cipher = fast and secure.
-
Banner (Compliance):
Banner /etc/issue.netDisplay a legal warning at login (regulatory requirement in some orgs).
Practical Example: Engineering SSH at Scale
Ansible playbook segment to provision keys for multiple users:
- name: Install admin keys
authorized_key:
user: "{{ item.username }}"
key: "{{ lookup('file', item.keyfile) }}"
loop:
- { username: 'engineer', keyfile: 'keys/engineer.pub' }
- { username: 'auditor', keyfile: 'keys/auditor.pub' }
Automate this for fleet consistency. Manual key addition is not sustainable at scale.
Non-Obvious Tip
Use ssh-agent correctly. For engineers juggling multiple servers, ssh-agent caches keys securely in memory. Avoid passphrase re-entry:
eval "$(ssh-agent -s)"
ssh-add ~/.ssh/id_ed25519
But—never leave agents running on shared systems.
The Bottom Line
SSH is often treated as “set and forget”—but every layer, from key management to service configuration and log analysis, determines whether you’re operating a resilient environment or one easily compromised. Commit to key-based access, minimize attack surface, monitor aggressively, and automate where possible. There are always more advanced controls (MFA via pam_u2f, FIDO2 tokens), but the fundamentals above will raise your baseline beyond the typical target.
If something breaks, check /var/log/auth.log first. And, always keep a rescue out-of-band channel.
Side note: If you’re scripting SSH actions (with Ansible, Jenkins, or GitHub Actions), SSH certificate authorities (sshd 7.2+) and agent forwarding are worth exploring—hidden time-savers for large teams.