SSH Access to AWS EC2: Practical Steps and Gotchas

Direct terminal access to an EC2 instance is foundational in AWS-based operations: configuration, incident response, even ad-hoc patching. Here’s a concise walkthrough, focused on Linux-based hosts, with hard-earned details for smoother sessions.

Scenario: Engineer needs to debug application logs on a production EC2 running Amazon Linux 2

No access? Downtime risk rises. The key question—can you reliably SSH in, every time, from a developer workstation with proper security? Let's ensure you can.

Required Prerequisites

• Existing AWS account, permissions sufficient to launch and view EC2
• EC2 instance in running state (Amazon Linux 2, tested with ami-0c02fb55956c7d316)
• Key pair .pem file—must match the instance.
• The instance's public IPv4 address or public DNS.
• SSH client: tested with OpenSSH_8.4p1 on Linux/macOS, or equivalent on Windows via WSL/PuTTY.
• Security group permits inbound TCP/22 from your IP.

1. Verify Instance Launch and Security Group

• In AWS Console > EC2 > Instances: confirm state is running.
• Inbound rule for SSH (port 22) is required. CIDR range like 203.0.113.5/32 is recommended over 0.0.0.0/0—broaden only for staging or initial testing, never for production.

Sample security group configuration:

Type     Protocol  Port Range  Source
SSH      TCP       22          203.0.113.5/32

2. Secure Your Private Key

OpenSSH refuses to use overly-permissive keys:

chmod 400 ~/keys/prod-server.pem

Anecdotally, permission errors result in:

WARNING: UNPROTECTED PRIVATE KEY FILE!
Permissions 0644 for 'prod-server.pem' are too open.

Ignore this, and access is denied.

3. Locate the Instance's Connect Details

• In EC2 console, under “Description”:
  • Note “Public IPv4 address” or “Public IPv4 DNS”.
    • Example: ec2-3-235-162-82.compute-1.amazonaws.com
    • Or bare IP: 3.235.162.82

4. Establish SSH Session

On Linux/macOS (or WSL):

ssh -i ~/keys/prod-server.pem ec2-user@ec2-3-235-162-82.compute-1.amazonaws.com

Default Usernames by AMI

Note: Using the wrong username returns Permission denied (publickey)—classic pitfall when working with mixed OS images.

On initial connection:

The authenticity of host 'ec2-3-235-162-82.compute-1.amazonaws.com (3.235.162.82)' can't be established.
ECDSA key fingerprint is SHA256:...
Are you sure you want to continue connecting (yes/no/[fingerprint])?

Inspect and accept with yes.

5. Special Case: SSH from Windows

• PuTTY: Convert .pem to .ppk via PuTTYgen. Set username in "Connection > Data".
• WSL/PowerShell (with OpenSSH): Use standard command line above.
• Old-school cmd/powershell: Use WSL or migrate to native OpenSSH for less friction.

6. Common Failure Modes

• Security group does not permit source IP: Check with curl ifconfig.me on your workstation, then verify the matching allowed source CIDR.
• Wrong key or permissions:
  • Wrong key yields Permission denied (publickey).
  • Public key mismatch (re-created key pair after instance launch): must redeploy or update authorized_keys, there's no workaround via AWS Console.
• Instance rebooted, new IP: Elastic IP solves this for persistent endpoints.

To programmatically verify details:

aws ec2 describe-instances --instance-ids i-0abc123456def7890 \
  --query "Reservations[0].Instances[0].[PublicDnsName, PublicIpAddress]" \
  --output table

Non-Obvious Tips

• Multi-factor the Bastion: For any production workload, consider a jump host (bastion) with MFA and IP whitelisting rather than opening SSH to every instance.
• EC2 Instance Connect (browser-based): Available for Amazon Linux 2, but limited. SSH remains essential for automation/CI/CD.
• Automated key rotation: No built-in AWS mechanism for rotating .pem files. Manage this via configuration management or scripts.
• Cloud Shell/Session Manager: For audit/compliance, sometimes preferable, but brings own trade-offs (latency, session persistence).

Recap—Core Checklist

1. Key file must match instance, permissions set (chmod 400).
2. Correct username per AMI.
3. Public IP/DNS verified post-launch or post-reboot.
4. Port 22 open to your source IP.
5. Capture initial host fingerprint for future security audits.

Break any of these, expect connection failures.

No one enjoys being locked out in the middle of an incident. Keep your .pem secure, automate where possible, and always double-check security groups after AWS changes. Consider alternate connection strategies for modern environments, but don’t ignore the basics—SSH remains the foundation.