Restore from backup

Restore MSR data

If your MSR has a majority of unhealthy replicas, the one way to restore it to a working state is by restoring from an existing backup.

To restore MSR, you need to:

  1. Stop any MSR containers that might be running

  2. Restore the images from a backup

  3. Restore MSR metadata from a backup

  4. Re-fetch the vulnerability database

You need to restore MSR on the same MKE cluster where you’ve created the backup. If you restore on a different MKE cluster, all MSR resources will be owned by users that don’t exist, so you’ll not be able to manage the resources, even though they’re stored in the MSR data store.

When restoring, you need to use the same version of the mirantis/dtr image that you’ve used when creating the update. Other versions are not guaranteed to work.

Remove MSR containers

Start by removing any MSR container that is still running:

docker run -it --rm \
  mirantis/dtr:2.8.13 destroy \
  --ucp-insecure-tls

Restore images

If you had MSR configured to store images on the local filesystem, you can extract your backup:

sudo tar -xf {{ image_backup_file }} -C /var/lib/docker/volumes

If you’re using a different storage backend, follow the best practices recommended for that system.

Restore MSR metadata

You can restore the MSR metadata with the mirantis/dtr restore command. This performs a fresh installation of MSR, and reconfigures it with the configuration created during a backup.

Load your MKE client bundle, and run the following command, replacing the placeholders for the real values:

read -sp 'ucp password: ' UCP_PASSWORD;

This prompts you for the MKE password. Next, run the following to restore MSR from your backup. You can learn more about the supported flags in mirantis/dtr restore.

docker run -i --rm \
  --env UCP_PASSWORD=$UCP_PASSWORD \
  mirantis/dtr:2.8.13 restore \
  --ucp-url <mke-url> \
  --ucp-insecure-tls \
  --ucp-username <mke-username> \
  --ucp-node <hostname> \
  --replica-id <replica-id> \
  --dtr-external-url <msr-external-url> < {{ metadata_backup_file }}

Where:

  • <mke-url> is the url you use to access MKE

  • <mke-username> is the username of a MKE administrator

  • <hostname> is the hostname of the node where you’ve restored the images

  • <replica-id> the id of the replica you backed up

  • <msr-external-url>the url that clients use to access MSR

DTR 2.5 and below

If you’re using NFS as a storage backend, also include --nfs-storage-url as part of your restore command, otherwise MSR is restored but starts using a local volume to persist your Docker images.

DTR 2.6.0-2.6.3 (with experimental online garbage collection)

Warning

When running 2.6.0 to 2.6.3 (with experimental online garbage collection), there is an issue with reconfiguring and restoring MSR with --nfs-storage-url, which leads to erased tags. Make sure to back up your MSR metadata before you proceed. To work around the --nfs-storage-url flag issue, manually create a storage volume on each MSR node. To restore MSR from an existing backup, use mirantis/dtr restore with --dtr-storage-volume and the new volume.

Re-fetch the vulnerability database

If you’re scanning images, you now need to download the vulnerability database.

After you successfully restore MSR, you can join new replicas the same way you would after a fresh installation.

Where to go next