Mastering Secure SSH Access to Azure VMs: Beyond the Basics

Opening SSH (port 22) to an Azure VM might get you quick access, but for production environments the story stops there. Attack surface, auditing, and operational safety are at stake. Below: practical, field-tested approaches for securing and automating SSH access workflows in Azure.

SSH Key Pairs: Baseline, Not Optional

Disabling password login is foundational. For any Linux image, use only SSH public keys—do not enable passwords under any circumstance in production. Example with OpenSSH (v8.9p1):

ssh-keygen -t ed25519 -C "ops@example.com"
# Produces ~/.ssh/id_ed25519(.pub)

Provision the VM directly with your public key:

az vm create \
  --resource-group sec-rg \
  --name prod-vm01 \
  --image Canonical:0001-com-ubuntu-server-focal:20_04-lts:latest \
  --admin-username cloudadmin \
  --ssh-key-values ~/.ssh/id_ed25519.pub

If you see Authentication refused, inspect /var/log/auth.log and verify permissions on ~/.ssh/authorized_keys (should be 600).

Network Attack Surface: Shut It Down

Opening port 22 to 0.0.0.0/0 is unacceptable—not even temporarily. Lock it to a static office subnet or enforce VPN-only access. For critical environments, front all SSH with Azure Bastion or a hardened jump host.

Example: Restrict SSH in NSG to a single source IP

MY_IP=$(curl -s https://ifconfig.me)/32
az network nsg rule create \
    --resource-group sec-rg \
    --nsg-name prodVMNSG \
    --name allowOpsSSH \
    --protocol Tcp \
    --direction Inbound \
    --priority 100 \
    --source-address-prefixes $MY_IP \
    --destination-address-prefixes '*' \
    --destination-port-ranges 22 \
    --access Allow

Note: Azure Portal sometimes caches old NSG config; always validate with az network nsg rule list.

Azure AD-Integrated SSH: The Modern Standard

Azure AD authentication brings real RBAC (role-based access control) to VM logins. No key juggling, no rogue admin accounts. Works with Linux VMs running Ubuntu 18.04+, provided the AAD extension is installed.

Enable AAD login:

1. Assign Virtual Machine Administrator Login (or VM User Login) to your group or user. Scope narrowly.
2. Install the extension:

az vm extension set \
  --publisher Microsoft.Azure.ActiveDirectory \
  --name AADLoginForLinux \
  --resource-group sec-rg \
  --vm-name prod-vm01

Authenticate natively with Azure CLI v2.44+:

az ssh vm --name prod-vm01 --resource-group sec-rg

Known issue: az ssh can hang if the VM clock drifts; ensure NTP is correctly configured.

Just-in-Time (JIT) Access: Dynamic Exposure for Ops

Azure Security Center’s JIT VM access allows on-demand port opening, sharply reducing dwell time for exposed ports. Ops teams request access; ports auto-close after a configurable window.

Typical flow:

• Define allowed source IP and time window.
• JIT opens port 22 only during approved window.
• All activity is audited.

az security jit-policy update -n 'default' -g sec-rg \
  --vm-name prod-vm01 --ports '[{"number":22,"protocol":"*"}]'

Requests and approvals: via Azure Portal or REST API. Side effect: ramp-up times can frustrate CI/CD if not pre-approved.

OS Hardening: Tighten the Last Mile

SSH config on the VM itself warrants attention. Defaults are rarely sufficient.

• Disallow root login:
  In /etc/ssh/sshd_config:

  PermitRootLogin no
  PasswordAuthentication no
  

• Restrict user logins:

  AllowUsers cloudadmin opsuser
  

• Change listen port:

  Port 2022
  

  Be cautious—Azure NSG rules must match OS port.

• Enable intrusion prevention:
  fail2ban or similar—configure with:

  sudo apt install fail2ban
  sudo systemctl enable fail2ban
  

After edits:

sudo systemctl reload sshd

Side note: Changing ports or usernames can break automation. Keep documentation up to date and test pipelines after every alteration.

ProxyJump & Agent Forwarding: Secure Multi-Hop and Automation

Architectures often place production hosts behind jumpboxes/bastion VMs. Avoid key proliferation. Use ~/.ssh/config to chain connections securely:

Host bastion
    HostName bastion.centralus.cloudapp.azure.com
    User cloudadmin
    IdentityFile ~/.ssh/id_ed25519

Host prod-vm01
    HostName 10.20.3.14
    User cloudadmin
    ProxyJump bastion

Now, a simple:

ssh prod-vm01

performs a double-hop securely, with no keys copied to bastions. Add ForwardAgent yes only if you trust the intermediary.

Practical tip: Use -J flag in scripts for ephemeral jumps:

ssh -J cloudadmin@bastion.centralus.cloudapp.azure.com cloudadmin@10.20.3.14

Quick Reference: Decision Table

Practical Gaps and Non-Obvious Considerations

• Bastion hosts add latency; test interactive commands for lag.
• AAD login is unsupported on all distros—test on staging before rollout.
• JIT struggles with multi-user workflows; not ideal for ephemeral CI instances.
• If cloud engineer turnover is high, prioritize short-lived credentials (AAD) over static keys.
• Use diagnostic settings (az monitor diagnostic-settings) to pipe successful/failed SSH attempts to Log Analytics.

Azure’s flexibility allows for layering—SSH keys, network controls, identity via AAD, dynamic JIT—each compensating for gaps in the others. Understand the operational trade-offs for your environment. Automation beats all: codify access patterns in infra-as-code (Bicep, ARM) for reproducibility.

Got a legacy VM only accessible via password due to an application dependency? Quarantine it, limit NSG scope, and schedule re-platforming. There’s always one outlier.

No cloud environment is ever perfectly locked down, but every layer bought is risk removed. Expect evolving threat models; audit and iterate accordingly.