Mastering Secure and Efficient SSH Access to EC2 Instances: More Than Just a Connection

Default SSH access on EC2 often looks like this:

ssh -i ~/Downloads/my-key.pem ec2-user@ec2-34-201-xx-xx.compute-1.amazonaws.com

But operational environments grow. Attackers scan for open port 22 within minutes. Key reuse multiplies blast radius. Access often ends up broad, untracked, or slow.

Below are hardened, scalable, and performance-tuned strategies that have kept production environments secure and manageable.

Harden: Key Management and Port Hygiene

Unique keys, every time. Generate a dedicated key pair per user, per role, or ideally both. Do not reuse keys; rotate regularly. Example with RSA 4096 (avoid DSA/ECDSA for EC2, those sometimes trip agent compatibility):

ssh-keygen -t rsa -b 4096 -C "alice.infra@yourcorp.com" -f ~/.ssh/ec2_alice_rsa

Assign the public key when launching the EC2 instance. If adding later, use ec2-user’s ~/.ssh/authorized_keys, or rely on SSM and Instance Connect for automation.

Disable password login immediately. On Amazon Linux 2 (OpenSSH_8.0p1 at this writing), edit /etc/ssh/sshd_config:

PasswordAuthentication no
PermitRootLogin no

Restart with sudo systemctl restart sshd.

Lock down security groups. Only your management subnet or VPN gateway should ever have inbound TCP/22. Example:

Type	Protocol	Port	Source	Description
SSH	TCP	22	10.1.4.0/24	Office VPN
SSH	TCP	22	198.51.100.7/32	Admin home IP

Any 0.0.0.0/0 is a risk—eliminate for anything other than quick test labs.

Note: Changing default port to something high (Port 40222) doesn't stop targeted scans, only basic ones. Still, it can help with noisy logs.

Keyless, Auditable Access: AWS Systems Manager Session Manager

With SSM Session Manager, port 22 never opens. SSM Agent (2.3.1644.0+ on Amazon Linux 2 AMIs) provides interactive access over HTTPS fully tracked in CloudTrail.

aws ssm start-session --target i-012345678

Prerequisites:

Attach AmazonSSMManagedInstanceCore policy to the instance profile.
Outbound HTTPS allowed (security group and NACL).
SSM agent running. Confirm with:

sudo systemctl status amazon-ssm-agent

Gotcha: Session Manager only supports interactive shell. Some binaries (e.g., SCP) require port 22.

Known issue: Older AMI versions pre-2021 may have out-of-date SSM agents. Upgrade or install manually using the repo RPM.

Multi-Tier: Bastion, SSH ProxyJump, and Agent Forwarding

Architecture:

local
  |
  | (SSH over internet)
  v
bastion (public subnet, port 22 open to trusted IPs)
  |
  | (SSH over private VPC subnet)
  v
private-ec2 (no public IP)

SSH config:

Host bastion
  HostName 54.210.xx.xx
  User ec2-user
  IdentityFile ~/.ssh/ec2_bastion.pem

Host web-db-*
  User ec2-user
  ProxyJump bastion
  IdentityFile ~/.ssh/ec2_private.pem
  ForwardAgent yes

Now, jump with:

ssh web-db-app1

No manual port forwarding. No private keys copied to bastion.

Practical tip: Use ssh-add -l to confirm your agent contains the keys you expect:

ssh-add ~/.ssh/ec2_private.pem

Security caveat: Agent forwarding exposes your agent socket on the bastion. Only forward to trusted hosts.

Automated Key Delivery: EC2 Instance Connect

Ephemeral SSH keys, delivered via IAM policy, drop compliance hassle. One less persistence vector if a contractor leaves or IAM rotates. Supported AMIs: Amazon Linux 2 (from 2019.03+), Ubuntu 20.04+.

aws ec2-instance-connect send-ssh-public-key \
  --instance-id i-11223344556677abc \
  --availability-zone us-east-1c \
  --instance-os-user ec2-user \
  --ssh-public-key file://~/.ssh/alice_tempkey.pub

Valid for 60 seconds per default.

Note: Not (yet) compatible with all base AMIs or custom users.

Performance Tweaks: ControlMaster, IPv4 Bias, and More

SSH connection overhead compounds at scale or within automation. Eliminate it:

Host *
  ControlMaster auto
  ControlPath ~/.ssh/sockets/%r@%h-%p
  ControlPersist 5m
  ServerAliveInterval 30
  AddressFamily inet
  HashKnownHosts yes

Multiplexing: ControlMaster and ControlPath reuse TCP for repeated logins—essential for CI automation.
IPv4 only: Forces SSH to skip slow IPv6 DNS queries.
ServerAliveInterval: Sidestep idle timeout drops—important behind NAT or in flaky Wi-Fi.

Side note: Older OpenSSH (<7.4) handling of ControlPersist differs; check ssh -V.

MFA for SSH: Add as Needed

Combine Google Authenticator or Duo PAM with SSH on high-assurance nodes. Large orgs often enforce this via configuration management, but introduces friction for scripting or fast logins.

/etc/pam.d/sshd addition for TOTP, after module install:

auth required pam_google_authenticator.so

Changing PAM configs can fat-finger your access. Have an open root session before testing, or you risk cut-off.

Reference Matrix

Tactic	Impact	Tradeoff or Side Note
Unique per-user SSH keys	Limits breach scope, tracks access	Key sprawl, must rotate/invalidate departures
Lock password and root login	Kills basic brute attempts	Must test other auth paths
Tight security groups	Narrows attack surface	Can lock self out on IP change
SSM Session Manager	Completely keyless, full audit trail	No SCP/rsync, dependency on SSM agent
ProxyJump + agent forwarding	Fast, no-key movement for multi-tier	Agent can be hijacked at weak bastions
EC2 Instance Connect	Temp key injection, IAM managed	Not universal to all AMIs/users—verify first
SSH config multiplexing and tuning	Faster, more resilient sessions	Older SSH may not support all flags
SSH MFA	Thwarts credential theft	Hinders automation, support/OTP setup reqd

Real-world gotcha: On large teams, people forget to remove keys from old instances. Use automation (Ansible, SSM, or custom Lambda) to sweep for stale keys weekly.

SSH into EC2 isn’t just plumbing; it’s your weakest link or your first bastion. Enforce strong controls, automate key hygiene, and centralize auditing where possible. Prefer keyless solutions once the surface area grows.

If you want practical snippets for Ansible-managed key rotation, or live SSM session audit queries, connect with me. Otherwise—inspect your own SSH surfaces. Gaps hide where change is fastest.

Ssh To Ec2