Jenkins on Amazon Linux EC2: Deploying a Scalable Automation Server

Provisioning Jenkins on AWS EC2 provides a flexible, cloud-native base for high-throughput CI/CD. Unlike hosted Jenkins alternatives or managed CI tools, running Jenkins as a dedicated system on EC2 allows complete OS-level customization, plugin control, and resize-on-demand. This guide walks through the deployment on Amazon Linux 2—adjust to other distributions as needed.

AWS EC2 Reminders

• Provision via AWS Console or IaC (e.g., Terraform, CloudFormation).
• For sustained workloads, avoid t2.micro or t3.micro—t3.medium or larger reduces throttling risk.
• Security: restrict SSH (port 22) and UI (port 8080) to trusted sources. Broadly opening 8080 to 0.0.0.0/0 leads to bot attacks within minutes.

Instance Launch & Network

1. AMI: Select Amazon Linux 2 (Kernel 5.x)
   • Kernel version impacts Java compatibility and performance.
2. Instance type: t3.medium (2 vCPU, 4GB RAM) is baseline for small teams. Production systems demand more memory and storage, especially if running Maven/Gradle builds.
3. Root volume: Start with 16GB EBS GP3. For artifact-heavy jobs, plan for rapid expansion (100GB+ not uncommon in active orgs).
4. Security Group:

   Port	Protocol	Source	Purpose
   22	TCP	Admin IP(s)	SSH
   8080	TCP	Office/WFH IP(s)	Jenkins UI

SSH Access

Obtain the public IP or DNS from the EC2 console. Connect:

ssh -i /path/to/key.pem ec2-user@EC2_PUBLIC_DNS

• Replace /path/to/key.pem with your private key.
• If you see Permission denied (publickey), double-check key permissions (chmod 400 ...).

System Preparation & Java Install

Amazon Linux 2 ships with yum. Jenkins LTS (as of 2.414.x) is stable on OpenJDK 11.

sudo yum update -y
sudo amazon-linux-extras install java-openjdk11 -y
java -version
# Expect: openjdk version "11.0.22" 2024-04-16 LTS

• Side note: Using Java 17 can break some old plugins. Default to 11 unless your plugin set is vetted.

Jenkins Repo & Installation

1. Add Jenkins repo & key:

   sudo wget -O /etc/yum.repos.d/jenkins.repo \
     https://pkg.jenkins.io/redhat-stable/jenkins.repo
   sudo rpm --import https://pkg.jenkins.io/redhat-stable/jenkins.io.key
   

2. Install Jenkins:

   sudo yum install jenkins -y
   

   • Version installed: Check with rpm -q jenkins. (2.414.3 is typical at this writing.)

Start, Enable, and Verify Jenkins

sudo systemctl start jenkins
sudo systemctl enable jenkins
sudo systemctl status jenkins

• If systemctl status reports Active: active (running), server startup is healthy.
• If you encounter Java version errors:
  Jenkins requires Java versions [11, 17] but found: 1.8.0_x
  — verify the default Java with alternatives --config java.

Firewall & Security Group

• AWS-level: Inbound rule for port 8080 in Security Group is mandatory.
• Instance-level: Amazon Linux 2 disables firewalld by default. If you’ve hardened the server, check with sudo firewall-cmd --list-all.

Initial Configuration via Web UI

1. Load http://<EC2_PUBLIC_DNS>:8080/ in your browser.

2. When prompted for the unlock key:

   sudo cat /var/lib/jenkins/secrets/initialAdminPassword
   

3. You may see:

   3105b69baba54e5ac78d4987e895188d
   

4. Proceed with plugin installation. Selecting “Suggested plugins” covers most common use cases (e.g., Git, Pipeline, SSH Agent, Blue Ocean).

• Gotcha: Plugin install failures often relate to outbound traffic restrictions (NAT gateway required for private VMs).

Admin User & Basics

• Set a complex admin user/password, not the default.
• By default, Jenkins listens on all network interfaces—expose port 8080 only as necessary.

Hardening and Production-readiness

• TLS: Do not expose raw HTTP in production. Integrate nginx with certbot for HTTPS (sample below).
• Backups: Periodically snapshot /var/lib/jenkins. Data loss is almost always catastrophic.
• Jenkins User: Default process runs as jenkins (check with ps aux | grep jenkins). Confirm directory permissions if mounting EFS or additional storage.

Example nginx Reverse Proxy Config

server {
    listen 80;
    server_name jenkins.example.com;

    location / {
        proxy_pass         http://localhost:8080;
        proxy_set_header   Host $host;
        proxy_set_header   X-Real-IP $remote_addr;
        proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

• Test with curl -I http://jenkins.example.com for a 200 response.

Non-Obvious Tips

• Jenkins builds can saturate EBS IO quickly. If jobs hang or fail unexpectedly, monitor with iostat or CloudWatch.
• EC2 hibernation isn’t compatible with running Jenkins. Always use stop/start or scheduled scaling.
• Need disposable agents? Use the Kubernetes plugin, but prepare for credential and pod template management headaches.
• Jenkins logs: /var/log/jenkins/jenkins.log sometimes gives more info than the web UI “System Log” panel.

Troubleshooting: Common Failure Modes

Routine Operations

• Upgrade Jenkins monthly; LTS plugins are not always backward compatible.
• Keep the installed Java version synchronized with plugin requirements.
• Consider disabling anonymous read access entirely in Configure Global Security.

Deployed Jenkins on EC2 delivers robust, upgradable CI/CD with all the quirks (and benefits) of self-management. The trade-off: increased admin time vs. total pipeline flexibility. When constraints or integrations get complicated, you’ll appreciate the control. If you need full managed simplicity, alternatives like AWS CodeBuild exist—but with less OS/plugin bandwidth.

For more advanced automation, look at setting up ephemeral build agents (Kubernetes, Docker), S3 artifact archiving, and IAM integrations for secure cloud deployments.

Production Jenkins engineering is never just copy-paste—understand your security and scaling needs before running that first job.