CentOS to Rocky Linux: Seamless Migration Without Downtime

CentOS 7’s EOL (June 2024) has made production migrations unavoidable for anyone still depending on the legacy RHEL clone. Rocky Linux, spearheaded by the original CentOS founder, fills the operational gap—but moving critical workloads isn’t a copy-paste affair, especially with zero downtime constraints.

Immediate Priority: Snapshots, Backups, and State Inventory

Backup strategy determines the achievable recovery point objective (RPO). Redundancy can’t be an afterthought.

• System snapshots (LVM, ZFS, VM image, or cloud snapshots)
• Filesystem-level backup (rsync -Aax or your tool of choice)
• Logical database dumps—mysqldump with consistent flags, or pg_dump for PostgreSQL
• Custom config archive: /etc, plus any application-specific directories
• Package state inventory:

  rpm -qa --qf '%{NAME}-%{VERSION}-%{RELEASE}.%{ARCH}\n' > /root/installed_pkgs_centos.txt
  

Oddly, many migrations skip thorough package mapping. Don’t—if you care about parity.

Parallel Environment Staging: The Only Sensible Route

In-place conversions (convert2rocky or migration scripts) may work for prototypes, but cannot guarantee operational consistency across complex setups—SSL chains, custom modules, kernel drivers, SELinux modes, etc. Parallel deployment solves most risk:

• Spin up equivalent Rocky Linux VMs or physical servers.

• Match hardware profiles where possible (nics, storage, CPU arch). Differences can surface subtle bugs.

• Install base Rocky Linux 9 (or 8.8 if older kernel required for compatibility).

• Add third-party repos as needed (epel-release, internal corp repos).

• Bulk-install packages from saved inventory:

  xargs -a /root/installed_pkgs_centos.txt sudo dnf install -y
  

  Note: Missing or renamed packages? Major jumps (CentOS 7 to Rocky 9) may need manual resolution and lab testing.

• Restore app configs, but don’t just overwrite /etc. Diff and merge changes.

• Recreate system users/services as defined (especially custom UIDs/GIDs).

Data Sync: Maintain Consistency in Real Time

For purely static content, a straightforward rsync can suffice:

rsync -avz --delete --numeric-ids /srv/data/ rockyadmin@rocky-host:/srv/data/

For live production databases or job queues, approaches differ:

• MySQL/MariaDB: Set up replication, promote Rocky as slave; test failover.
• PostgreSQL: Streaming replication, logical replication for zero-downtime cutover.
• Queues/Caches: Use built-in replication (e.g., Redis replica, RabbitMQ clustering).

Note: File-level sync alone does not guarantee application consistency—always pause writes before final sync if not using live replication.

Functional Validation—Assume Nothing

Humans skip tests when cutover pressure mounts. This is when breakage slips through.

Checklist:

• All daemons running and submitting healthy logs? (systemctl, /var/log/, journald)
• Network listeners: check with ss -ltnp, compare to legacy server.
• Authentication and PAM modules working for both system and application users?
• SELinux context mismatches (CentOS 7 → Rocky 9 especially)
• Run app-specific acceptance tests, smoke test ingress/egress, probe endpoints

Example (troubleshooting httpd):

httpd[5021]: AH00558: httpd: Could not reliably determine the server's fully qualified domain name

Document or pre-create any missing settings (e.g., /etc/hosts, ServerName).

Cutover—Minimize and Control Downtime

Perfect cutover is rare, but minimizing user impact is feasible:

1. Announce a short maintenance window.
2. Place source system in read-only or maintenance mode.
3. Final delta rsync:

   rsync -az --delete /data/ rocky@rocky-host:/data/
   

4. point DNS or load balancer (LB) targets to the new Rocky Linux host. For HA, update cluster membership.
5. Restart critical services:

   systemctl daemon-reload
   systemctl restart nginx mariadb redis
   

6. Monitor logs and behavior for 15–30 min before announcing fully live state.

Gotcha: Post-migration, some clients (esp. Java apps, persistent DB connections) may need to reconnect. Log connection drops and inform stakeholders in advance.

Practical Tips & Unexpected Issues

Version differences: RHEL9-based Rocky may have PHP 8.x, OpenSSL 3.x, or kernel 5.** series; legacy applications can and do break on ABI incompatibility.

Disable auto updates pre-cutover:

sudo systemctl disable --now dnf-automatic.timer

Re-enable only after verifying system stability.

Rollback/Abort plan: Test “failback” on a staging copy. Keep old system live in isolation until confident in Rocky.

SELinux: On Rocky 9, many SELinux policies are stricter. Verify mode and audit with ausearch if permissions fail.

Community support: The Rocky Linux Mattermost, forums, and GitHub issues are responsive—but have relevant logs ready before engaging.

Appendix: Real World Command Snippets

Migrate service enablement:

systemctl list-unit-files --state=enabled  | awk '{print $1}' | grep .service | xargs -n1 systemctl enable

Caveat: Systemd units may be renamed; audit before auto-enabling.

Not Perfect? Neither Is Real Life

Even with best-laid plans, you’ll run into subtle bugs, minor misconfigurations, or package drift between clones. Rocky generally tracks RHEL closely, but hardware drivers and edge repos won’t always align. Don’t delete the old node until Rocky’s been in production for at least 48–72 hours with full traffic and monitoring enabled.

Success is not just zero downtime at cutover, but no lingering user impact in the weeks that follow.

Questions or migration horror stories? The Rocky Linux community probably has seen it before.