If your DTR has a majority of unhealthy replicas, the one way to restore it to a working state is by restoring from an existing backup.
To restore DTR, you need to:
You need to restore DTR on the same UCP cluster where you’ve created the backup. If you restore on a different UCP cluster, all DTR resources will be owned by users that don’t exist, so you’ll not be able to manage the resources, even though they’re stored in the DTR data store.
When restoring, you need to use the same version of the docker/dtr
image that you’ve used when creating the update. Other versions are not
guaranteed to work.
Start by removing any DTR container that is still running:
docker run -it --rm \
docker/dtr:2.7.5 destroy \
--ucp-insecure-tls
If you had DTR configured to store images on the local filesystem, you can extract your backup:
sudo tar -xf {{ image_backup_file }} -C /var/lib/docker/volumes
If you’re using a different storage backend, follow the best practices recommended for that system.
You can restore the DTR metadata with the docker/dtr restore
command. This performs a fresh installation of DTR, and reconfigures it
with the configuration created during a backup.
Load your UCP client bundle, and run the following command, replacing the placeholders for the real values:
read -sp 'ucp password: ' UCP_PASSWORD;
This prompts you for the UCP password. Next, run the following to restore DTR from your backup. You can learn more about the supported flags in docker/dtr restore.
docker run -i --rm \
--env UCP_PASSWORD=$UCP_PASSWORD \
docker/dtr:2.7.5 restore \
--ucp-url <ucp-url> \
--ucp-insecure-tls \
--ucp-username <ucp-username> \
--ucp-node <hostname> \
--replica-id <replica-id> \
--dtr-external-url <dtr-external-url> < {{ metadata_backup_file }}
Where:
<ucp-url> is the url you use to access UCP<ucp-username> is the username of a UCP administrator<hostname> is the hostname of the node where you’ve restored the
images<replica-id> the id of the replica you backed up<dtr-external-url>the url that clients use to access DTRIf you’re using NFS as a storage backend, also include
--nfs-storage-url as part of your restore command, otherwise DTR is
restored but starts using a local volume to persist your Docker images.
Warning
When running DTR 2.5 (with experimental online garbage collection)
and 2.6.0 to 2.6.3, there is an issue with reconfiguring and
restoring DTR with
`–nfs-storage-url`` <dtr-rn-2-6>` which
leads to erased tags. Make sure to back up your DTR
metadata
before you proceed. To work around the --nfs-storage-urlflag
issue, manually create a storage volume on each DTR node. To restore
DTR from an existing backup, use
docker/dtr restore with --dtr-storage-volume and the new
volume.
See Restore to a Local NFS Volume for Docker’s recommended recovery strategy.
If you’re scanning images, you now need to download the vulnerability database.
After you successfully restore DTR, you can join new replicas the same way you would after a fresh installation. Learn more.