Seamless Migration: CentOS 7 to AlmaLinux 8 with Zero Downtime

CentOS 7’s EOL (June 30, 2024) means no more security fixes or upstream patches. Not optional for production workloads—get off now, or accept operational risk. AlmaLinux 8 presents the least pain for RHEL-based environments: binary-compatible, independently governed, and presently stable through 2029.

Below: a tested method to migrate business-critical services from CentOS 7 to AlmaLinux 8, aiming for zero downtime. No unreliable in-place upgrades; parallel migration with real data and traffic shifting. Example stack: Apache + MySQL.

Migration Approach & Constraints

Zero downtime in practice means users shouldn’t notice the move. Absolute zero is rare—session stickiness, slow replication, or DNS caching can create brief gaps. Nonetheless, the steps below keep interruption below user-threshold for most web+database deployments.

Key constraints and gotchas:

• No in-place upgrades: You cannot just dnf upgrade from CentOS 7 to AlmaLinux 8.
• RPM package incompatibility: EL7 uses yum 3, Python 2.7; EL8 jumps to DNF, Python 3, and newer library ABI versions.
• Custom or third-party repositories must be checked for EL8 compatibility.
• Data sync needs attention—not just files, but live databases.

Pre-Migration Checklist

• Offload full system backups (rsnapshot, Veeam, or tarballs from a rescue environment—avoid live copy disasters).
• Collect system state

  rpm -qa > /root/installed_rpm_list.txt
  crontab -l > /root/crontab_backup.txt
  cp -a /etc /root/etc_backup
  

• Document customizations: Anything that’s not default matters—SELinux rules, systemd overrides, homegrown scripts in /usr/local/bin.
• Verify app compatibility: EL8 deprecates several PHP and Python versions; Ruby, Java stacks likely require new runtime deployments.
• Notify stakeholders and schedule maintenance: To reduce exposure if reverting is required.

1. Provision AlmaLinux 8 – Parallel Host

Spin up AlmaLinux 8.9 on hardware, KVM, or a cloud instance. Mirror source spec: CPUs, disk, network. Match user UID/GID values (especially for NFS or shared storage).
Example, cloud CLI:

openstack server create --image almalinux-8.9 --flavor vcpu-4-ram-8GB ...

Install necessary services—note: defaults are not always enabled.

dnf groupinstall "Web Server" "SQL Database Server"
dnf install rsync mariadb-server

2. Data Synchronization: Files & Databases

Static Data: Use repeated rsyncs, then a final delta during cutover.

rsync -aAX --delete /var/www/ root@almalinux:/var/www/

Tip: Use --dry-run and compare logs for missing owner/group corrections.

Live Database:

• For MySQL 5.7/8.0 (assume source is 5.7, target is 8.0—test your workload, some queries break).
• Establish replication (binlog-based).
• Example:
  • On CentOS 7 (master):

    GRANT REPLICATION SLAVE ON *.* TO 'repl'@'almalinux_ip' IDENTIFIED BY 'secret';
    FLUSH TABLES WITH READ LOCK;
    SHOW MASTER STATUS;  -- Record file/position
    

  • Dump init snapshot:

    mysqldump --all-databases --single-transaction --master-data=2 | gzip > dump.sql.gz
    

  • On AlmaLinux 8 (slave):
    • Import dump.
    • Set up replication credentials with correct log file/position.
    • Start slave, verify with SHOW SLAVE STATUS\G.

Application Caches/Datastores:

• Stateless (Redis, Memcached): warm on new host, failover via VIP or DNS only at switchover.
• Stateful (Elasticsearch): plan for reindex or cluster migration, not covered here.

3. Configuration & Package Alignment

• Match RPMs: EL7 → EL8 package names sometimes change or split. Use dnf provides to map equivalents.
• Reapply config. Do not blindly overwrite /etc; compare and merge.
  Example:

  diff -ruN /etc/httpd/conf/httpd.conf /root/etc_backup/httpd/conf/httpd.conf
  

• Gotcha: Many EL8 services are managed differently (systemd unit differences, SELinux defaults stricter).
• Test systemd units:

  systemctl status httpd mariadb
  tail -f /var/log/httpd/error_log
  

4. Testing—Shadow and Staging

• Deploy staging network to isolate traffic.

• For web: point internal DNS or use /etc/hosts overrides for canary testing.

• For DB: run consistency checks.

  mysqlcheck -u root -p --all-databases --auto-repair
  

• Validate: no major error logs, latency stable, auth integrations work.

• Non-obvious tip: For low TTL external DNS isn't always honored—Chrome hard-caches DNS for up to a minute, some ISPs ignore TTLs.

5. Cutover—Traffic Switch

With Load Balancer:

• Add AlmaLinux backend to pool, set weighting to 0 for soak test.
• Gradually raise weighting, observe metrics (nginx logs, 5xx rates, DB replication lag).
• Remove CentOS node only after sustained period with AlmaLinux error-free.

Without LB (just DNS):

• Lower DNS TTL at least 48h in advance to 30–60 seconds, or some clients won’t see the change.
• Cut over A/AAAA records at low-traffic window.
• Monitor old server for lingering connections.

After Move:

• Migrate any residual files or DB changes.
• Monitor dmesg, application logs; expect a few silent tunable mismatches (kernel parameters, ulimits).
• Keep CentOS 7 host powered on for rollback until after at least a maintenance window.

Practical Example: Apache + MySQL

Summary table:

Gotcha: After switching to AlmaLinux, SELinux can block Apache PHP file writes—common with old CMS stacks. Use audit2allow to identify and patch type errors.

Final Recommendations

• Parallel host migration is safer and typically faster, especially in virtualized or cloud environments.
• Never trust a single rsync or dump—always verify data on target prior to cutover.
• Some edge-case applications (kernel modules, vendor daemons) may not be portable; test those early in any migration planning.

If you require “true zero” downtime (no logged-in session interruption), review options for live migration at the hypervisor or container orchestration level. For almost all web+database cases, the approach above balances risk and effort.

Questions about a workload or app not addressed here? Post details below—real-world migrations surface edge cases that generic guides miss.