SSH on Linux: Field Notes from Real-World Ops

SSH is the definitive tool for secure remote management of Unix systems. Whether automating deployments, digging through logs over a flaky VPN, or troubleshooting at 3 AM, SSH provides the backbone for Linux administration.

The Problem: Remote Access with Accountability

You’re responsible for a fleet of cloud VMs, or perhaps just one on-premises box behind an office firewall. Work happens from coffee shops, airports, or a CI/CD runner at HQ. How do you ensure access is secure, uninterrupted, and audit-friendly?

SSH: Secure Shell Protocol Overview

SSH (Secure Shell, RFC 4251) offers encrypted terminal and file transfer sessions over TCP—almost always on port 22/TCP. Its authentication (public key or password), flexible tunneling, and composition with scripts make it an essential sysadmin tool.

Prerequisites

• Remote host running OpenSSH ≥7.6 (modern Linux distributions: Ubuntu 20.04+, RHEL 8+).
• Local machine with OpenSSH client (ssh in PATH). macOS and most Linux systems include this by default.
• Network path to target IP/hostname (consider firewall/NAT).
• User credentials (username, usually non-root; key or password).
• (Recommended) SSH key pair (ed25519 or RSA ≥4096 bits).

Verify SSH Client & OpenSSH Version

On your local system:

ssh -V

Sample output:

OpenSSH_8.9p1 Ubuntu-3ubuntu0.5, OpenSSL 3.0.2 15 Mar 2022

Older versions may lack modern ciphers (e.g., chacha20-poly1305). Consider upgrading if your distro ships pre-2018 OpenSSH.

Basic Usage: Connect by Username and Host

ssh -l alice 192.0.2.42

or equivalently,

ssh alice@192.0.2.42

Note: On first connect, expect this prompt:

The authenticity of host '192.0.2.42 (192.0.2.42)' can't be established.
ECDSA key fingerprint is SHA256:...
Are you sure you want to continue connecting (yes/no/[fingerprint])?

Always verify the fingerprint via a trusted out-of-band channel. Otherwise, MITM risk.

Troubleshooting: SSH Daemon and Firewall

If connection fails:

• Confirm SSH daemon status on the remote host:

  sudo systemctl status ssh
  

  Output should include Active: active (running).
• Check firewall (UFW, iptables, AWS security groups). SSH default is 22/TCP. Non-standard ports require explicit -p parameter.

Example error:

ssh: connect to host 192.0.2.42 port 22: Connection refused

Diagnose for blocked ports or misconfigured sshd.

SSH Key Pairs: Passwordless and Auditable

Generate a modern key pair:

ssh-keygen -t ed25519 -C "alice@example.com"

RSA is still common, but ed25519 gives compact keys with robust security.

To authorize your public key, push it to the server:

ssh-copy-id alice@192.0.2.42

Or, manually append ~/.ssh/id_ed25519.pub to the server’s ~/.ssh/authorized_keys.

Gotcha: File permissions (chmod 700 ~/.ssh; chmod 600 ~/.ssh/authorized_keys) are critical, or SSH will silently reject keys.

Practical Example: Run a Remote Command Non-Interactively

ssh alice@192.0.2.42 'journalctl -u nginx --since "1 hour ago"'

Useful for automation—exit status is reflected in local shell.

Advanced: Custom Port, Config File, and Reusable Aliases

If sshd listens on 2222:

ssh -p 2222 alice@192.0.2.42

For recurring access to multiple systems, maintain an SSH config:

# ~/.ssh/config
Host web-prod
    HostName 203.0.113.17
    User deploysvc
    Port 2222
    ForwardAgent yes

Now connect via ssh web-prod.
This also supports custom identities, per-host ProxyCommand, and ControlMaster multiplexing.

Debugging & Logs

For verbose output:

ssh -vvv alice@192.0.2.42

Log file on server: /var/log/auth.log (Debian/Ubuntu) or /var/log/secure (RHEL/CentOS).

Real-World Tip: SSH Agent Forwarding (With Caution)

Agent forwarding (ForwardAgent yes) passes your local keys to the remote host, allowing hop-to-hop authentication.
Critical: Use only on trusted hosts; compromise exposes your agent to remote access.

Security Notes & Hardening Steps

• Disable password logins (PasswordAuthentication no) in /etc/ssh/sshd_config once keys are in place.
• Change default port only if you enjoy security through obscurity. Firewalling or port-knocking is more robust.
• Use AllowUsers or AllowGroups to restrict login access.

Known Issue: Host Key Rotation

If your server is rebuilt, you’ll get a warning:

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

Investigate before proceeding; blindly removing lines from ~/.ssh/known_hosts can mask real attacks.

Ascii Sketch: Typical SSH Workflow

[Local Laptop] ---[internet]---> [Remote Server:22]
            |                      |
         ~/.ssh/                 /etc/ssh/
         config                 sshd_config

Summary

SSH remains the workhorse of Linux remote administration. Secure key management, habitually checking fingerprints, and thoughtful client configuration can prevent headaches (and breaches) down the line.

Not every edge case here—Kerberos, GSSAPI, jump hosts with ProxyJump—was covered. Sometimes you’ll have to dig into the man pages, or sift through -vvv output at midnight. That’s part of the trade.

Further reading: man ssh, sshd_config(5), Mozilla’s SSH guidelines