Recovering a CentOS VPS from an rsync backup

I have been helping a friend recover his 123-Reg CentOS 6 based VPS from an rsync files backup. My friend has a 123-Reg VPS with a number of small websites running on it, last weekend 123-Reg completely wrecked his VPS by “upgrading” the underlying host. The upgrade led to massive file-system corruption and because it’s “not a managed product” 123-Reg washed their hands of helping repair their mistake. The only “support” that 123Reg offer is button to trigger a fresh rebuild with the latest CentOS installed.

Fortunately I had been working to migrate some websites and so had taken a full rsync backup just before this upgrade took place. The problem with backups is often recovery this recovery process has now been tested and works. The biggest challenge was working out why the restore fails as 123-Reg don’t provide console access which makes figuring out recovery failures tricky.

First un-mount all the bind mounted bits that make up the chrooted named stack

mount | grep bind | awk '{print $1}' | xargs umount

Next copy fstab someplace safe, in this case we aren’t rebuilding /boot so stash a copy there. We do this because the fresh install has different UUIDs embedded in the filesystems so we need to make sure they are correct otherwise the VPS will fail to boot and need rebuilding again.

cp /etc/fstab /boot

Now we have a safe copy of fstab we need to copy the files over, it’s a good idea to copy over /etc/passwd and /etc/group first so that file ownerships work correctly. The copy needs to run as root in order to read all the files.

sudo -E rsync -avzPHS \
--one-file-system --delete-after \
/path/to/backups root@VPS:/

Once this copy has completed, we need to replace fstab with the one we copied before.

cp /boot/fstab /etc/fstab

Next we need to make sure we can boot the VPS, we only get one go at this because we have no console access so we rebuild all of the boot process. First reinstall the current kernel, this is not necessarily the running kernel but the one installed on the backup image.

rpm -ivh --force <kernel rpm>

Next reinstall grub

grub-install /dev/vda

Finally check the UUID for / matches in grub.conf, initramfs and fstab