Restore from backup

Restore MSR data

If your MSR has a majority of unhealthy replicas, you can restore it to a working state by restoring from an existing backup.

To restore MSR, you must:

  1. Stop and remove any MSR containers that might be running.

  2. Restore the images from a backup.

  3. Restore MSR metadata from a backup.

  4. Re-fetch the vulnerability database.

Important

  • You must restore MSR on the same MKE cluster upon which you created the backup. If you restore on a different MKE cluster, the MSR resources will be owned by non-existant users, and thus you will not be able to manage the resources despite their being stored in the MSR data store.

  • When restoring, you must use the same version of the mirantis/dtr image that you used in creating the backup.

Remove MSR containers

Start by removing any MSR container that is still running. Run the following command with the client bundle:

docker run -it --rm \
  mirantis/dtr:2.9.16 destroy \
  --ucp-insecure-tls

Note

If the client bundle is not activated, the command can be run from an MSR node.

Restore images

If you had MSR configured to store images on the local filesystem, you can extract your backup:

sudo tar -xf {{ image_backup_file }} -C /var/lib/docker/volumes

If you’re using a different storage backend, follow the best practices recommended for that system.

Restore MSR metadata

You can restore the MSR metadata with the mirantis/dtr restore command. This performs a fresh installation of MSR, and reconfigures it with the configuration created during a backup.

Load your MKE client bundle, and run the following command, replacing the placeholders for the real values:

read -sp 'ucp password: ' UCP_PASSWORD;

This prompts you for the MKE password. Next, run the following to restore MSR from your backup. You can learn more about the supported flags in mirantis/dtr restore.

docker run -i --rm \
  --env UCP_PASSWORD=$UCP_PASSWORD \
  mirantis/dtr:2.9.16 restore \
  --ucp-url <mke-url> \
  --ucp-insecure-tls \
  --ucp-username <mke-username> \
  --ucp-node <hostname> \
  --replica-id <replica-id> \
  --dtr-external-url <msr-external-url> < {{ metadata_backup_file }}

Where:

  • <mke-url> is the url you use to access MKE

  • <mke-username> is the username of a MKE administrator

  • <hostname> is the hostname of the node where you’ve restored the images

  • <replica-id> the id of the replica you backed up

  • <msr-external-url>the url that clients use to access MSR

DTR 2.5 and below

If you’re using NFS as a storage backend, also include --nfs-storage-url as part of your restore command, otherwise MSR is restored but starts using a local volume to persist your Docker images.

DTR 2.6.0-2.6.3 (with experimental online garbage collection)

Warning

When running 2.6.0 to 2.6.3 (with experimental online garbage collection), there is an issue with reconfiguring and restoring MSR with --nfs-storage-url, which leads to erased tags. Make sure to back up your MSR metadata before you proceed. To work around the --nfs-storage-url flag issue, manually create a storage volume on each MSR node. To restore MSR from an existing backup, use mirantis/dtr restore with --dtr-storage-volume and the new volume.

Re-fetch the vulnerability database

If you’re scanning images, you now need to download the vulnerability database.

After you successfully restore MSR, you can join new replicas the same way you would after a fresh installation.

Where to go next