Upgrading AlmaLinux 8 to 9: Real-World Engineering Approach

AlmaLinux 8 systems reach the end of full support earlier than you expect—ongoing security updates and modern kernel support become critical for production infrastructures. The move to AlmaLinux 9 brings an updated glibc (2.34), OpenSSL 3, improved SELinux policies, and a major package refresh (GCC 11, Python 3.9+). Notably, compatibility with RHEL 9 means long-term stability and a predictable patch cadence.

A non-disruptive in-place upgrade is achievable using ELevate (built on Leapp), but beware: poorly handled third-party repositories or custom kernel modules will break the upgrade path. Here’s how to execute the migration with minimal downtime and maximal control.

Prerequisites & Pre-Flight Checklist

Proven backup: Block-level snapshot (LVM/disk/PVC) or offline rsync/rsnapshot copy. Test your restores.
Root or sudo privileges.
Stable connectivity and at least 2 GB free disk space for package operations/logging.
Inventory all third-party or epel packages; mismatched versions may cause blockers.

Do not attempt this on production before running trial upgrades elsewhere. No tool can fully anticipate all corner-case failures.

1. System Update: Cleaning Up AlmaLinux 8

First, bring all packages current to avoid dependency issues during the upgrade.

sudo dnf clean all
sudo dnf --enablerepo=baseos,appstream,extras -y update
sudo reboot

Gotcha: Stale kernel modules (e.g., legacy Nvidia/ELRepo DKMS drivers) often complicate upgrades. Remove or update them before proceeding.

2. ELevate: Installation & Context

ELevate coordinates the in-place transition between RHEL-compatible distros. Under the hood, it uses Leapp’s workflows but applies AlmaLinux-specific data and policies. Download directly from the official AlmaLinux repo—never trust random mirrors.

sudo dnf install -y https://repo.almalinux.org/elevate/elevate-release-latest.el8.noarch.rpm
sudo dnf install -y leapp-upgrade leapp-data-almalinux

Check the available version:

rpm -q leapp-upgrade

You want at least leapp-upgrade-0.16.0 for full AlmaLinux 9 compatibility. Older versions have edge-case bugs with NetworkManager configs.

3. Pre-Upgrade Checks: Finding Edge Cases

Run the automated dry run:

sudo leapp preupgrade

This produces /var/log/leapp/leapp-report.txt. Common failures:

Unsupported repositories detected: epel, remi
Blocked by unsigned kernel module: kmod-wireguard
Incompatible package: python36-*

Use sudo dnf config-manager --disable <repo> to temporarily shut off unsupported repositories. Remove obsolete packages carefully; sometimes a single Python dependency can block everything.

Tip: If custom firewalld policies are in use, export them. Leapp occasionally fails to reimport highly customized rulesets.

4. Running the Upgrade Process

After resolving blockers in leapp-report.txt, proceed:

sudo leapp upgrade

This downloads and stages all upgrade data. Audit logs at /var/log/leapp/leapp-upgrade.log—especially if anything appears to “hang”. Expect at least 10–30 minutes of package downloads and transaction checks on a typical cloud VM.

Once prompted, reboot:

sudo reboot

Note: ELevate/Leapp adds a dedicated boot entry (AlmaLinux 9 Upgrade Initramfs). If you see a failed boot or kernel panic here, select the previous kernel to recover, then review logs.

5. Verifying AlmaLinux 9

Upon successful boot, confirm the OS baseline:

cat /etc/os-release

Expect output:

NAME="AlmaLinux"
VERSION="9.3 (Shamrock Pampas Cat)"
ID="almalinux"

Also confirm kernel version. You should see 5.14.x (typical for AlmaLinux 9).
Check that systemd --version reports >= 250.

Practical example: applications compiled against OpenSSL 1.1 may now fail due to OpenSSL 3. Audit any binaries or Python environments with legacy dependencies.

6. Post-Upgrade Cleanup

Synchronize package versions and purge leftovers:

sudo dnf distro-sync -y
sudo dnf clean all

List and restart core services:

sudo systemctl --failed
sudo systemctl restart httpd mariadb

Partial failures are common with custom or third-party systemd units. Review and remediate.

Troubleshooting & Watchpoints

Boot loops? The likely culprit is a missing or incompatible driver/kernel module.
TCP/IP or firewall failures? Custom nftables rules may not convert cleanly.
Application missing libraries? Use dnf provides to locate missing .so files.

Log analysis is essential:

less /var/log/leapp/leapp-report.txt
less /var/log/leapp/leapp-upgrade.log
journalctl -b -1

Side Notes & Non-Obvious Lessons

ELevate will not migrate custom SELinux policies verbatim—manually verify all custom modules, especially if operating in enforcing mode.
Systems running Docker with overlay2 storage may require a manual systemctl restart docker and occasionally even a rebuild of containers post-upgrade due to the storage driver’s kernel dependencies.

Summary: Direct upgrades from AlmaLinux 8 to 9 are reliable for most standard packages and infrastructure stacks, so long as pre-upgrade reports are strictly followed and all third-party packages are accounted for. The process is scripted but sometimes imperfect; be prepared for manual interventions—especially where one-off customisations exist.

Alternative: Some engineers still prefer fresh installs then data migration, especially for very high-uptime or complex systems. That’s a trade-off; weigh the risks.

For environments using Infrastructure-as-Code (Terraform/Ansible), capture post-upgrade diffs to maintain idempotency.

For hard-won stability, pay close attention to customizations and edge-case dependencies. No in-place upgrade is ever “simple”—plan accordingly.

Almalinux 8 To 9