Database Access Configuration

This guide assumes you are working on a dedicated migration workstation, a machine with access to both the source and destination environments, used for managing the migration.

Configure PostgreSQL access

To enable access to the MSR 4.x PostgreSQL instance:

1. Update any required inbound firewall rules to allow PostgreSQL traffic.

   Note

   Before running kubectl commands, source the client bundle by exporting the kubeconfig file that provides access to the target MSR 4 registry.

2. Retrieve the MSR 4 PostgreSQL credentials for the migration process:

   1. Username:

      kubectl get secret \
        msr.msr-postgres.credentials.postgresql.acid.zalan.do \
        -o jsonpath="{.data.username}" | base64 --decode; echo
      

   2. Password:

      kubectl get secret \
        msr.msr-postgres.credentials.postgresql.acid.zalan.do \
        -o jsonpath="{.data.password}" | base64 --decode; echo
      

   Note

   Connectivity will be validated in the later step.

3. Ensure that socat is installed on PosgreSQL worker nodes.

4. Identify the PostgreSQL leader Pod:

   kubectl exec msr-postgres-0 -- patronictl list | grep -i leader
   

5. Forward the port to expose PostgreSQL locally:

   kubectl port-forward pod/<LEADER-POD-NAME> 5432:5432
   

   Replace <LEADER-POD-NAME> with the actual Pod name returned in the previous command.

Local database access

Before running the migration tool, you must first copy and run both the MKE authorization store and the MSR database store locally.

To do so, complete the following steps on your local migration workstation:

1. Verify that a container runtime is installed, such as Docker Desktop, Mirantis Container Runtime (MCR), or Docker CE.

2. Verify that RethinkDB is installed.

3. Save the manage_source_registry_db.sh script to your local machine. This script copies the Enzi and MSR databases and starts local instances.

   Click for the script

   #!/bin/bash
   set -euo pipefail
   
   SCRIPT_VERSION="1.0.1"
   
   # Default ports
   ENZI_RETHINKDB_PORT=28015
   ENZI_CLUSTER_PORT=29015
   MSR_RETHINKDB_PORT=28016
   MSR_CLUSTER_PORT=29016
   
   SCRIPT_NAME=$(basename "$0")
   
   check_client_bundle_sourced() {
     if [[ -z "${DOCKER_HOST:-}" ]] || [[ -z "${DOCKER_TLS_VERIFY:-}" ]] || [[ -z "${DOCKER_CERT_PATH:-}" ]]; then
       echo
       echo "WARNING: Docker client environment variables not detected."
       echo "It is recommended to source the MKE admin client bundle (e.g., 'source env.sh')"
       echo "to ensure access to the source registry cluster."
       echo
     fi
   }
   
   show_help() {
     echo
     echo "Overview:"
     echo "  Use this script to copy and expose the source registry databases (MKE auth"
     echo "  store and MSR DB store) to the MSR 4 migration tool."
     echo
     echo "Prerequisites:"
     echo "  All prerequisites apply to the system where this script is executed."
     echo "  - Docker (or MCR) installed and running (see https://docs.docker.com/get-docker)."
     echo "  - RethinkDB installed (see https://rethinkdb.com/docs/install)."
     echo "  - MKE admin client bundle applied to access the source registry cluster (see"
     echo "    https://docs.mirantis.com/mke/3.8/ops/access-cluster/client-bundle/download-client-bundle.html)."
     echo
     echo "Usage:"
     echo "  $SCRIPT_NAME [options]"
     echo
     echo "Options:"
     echo "  -c, --copy                Copy both eNZi and MSR databases (requires Docker)"
     echo "      --copy-enzidb         Copy only the eNZi DB (requires Docker)"
     echo "      --copy-msrdb          Copy only the MSR DB (requires Docker)"
     echo "  -e, --start-enzidb        Start eNZi DB (requires RethinkDB)"
     echo "  -m, --start-msrdb         Start MSR DB (requires RethinkDB)"
     echo "      --enzi-driver PORT    Override eNZi driver port (default: 28015)"
     echo "      --enzi-cluster PORT   Override eNZi cluster port (default: 29015)"
     echo "      --msr-driver PORT     Override MSR driver port (default: 28016)"
     echo "      --msr-cluster PORT    Override MSR cluster port (default: 29016)"
     echo "  -v, --version             Show script version"
     echo "  -h, --help                Show this help message"
     echo
     echo "Notes:"
     echo "  The --start-enzidb and --start-msrdb options run RethinkDB in the foreground (i.e. blocking)."
     echo "  The script will not return until the database process exits."
     echo "  Do not use both options in the same invocation (use a separate terminal for each)."
     echo
     echo "Examples:"
     echo "  $ # Copy and start the MKE auth store (eNZi) DB"
     echo "  $ ./$SCRIPT_NAME --copy-enzidb --start-enzidb"
     echo
     echo "  $ # Copy and start the MSR DB"
     echo "  $ ./$SCRIPT_NAME --copy-msrdb --start-msrdb"
     echo
     exit 0
   }
   
   error_missing_binary() {
     echo "Error: Required binary '$1' is not installed or not in PATH." >&2
     exit 1
   }
   
   check_docker() {
     if ! command -v docker >/dev/null 2>&1; then
       error_missing_binary "docker"
     fi
   }
   
   check_rethinkdb() {
     if ! command -v rethinkdb >/dev/null 2>&1; then
       error_missing_binary "rethinkdb"
     fi
   }
   
   copyEnziDb() {
     check_docker
     mkdir -p db_data
     echo "Copying eNZi DB from Swarm leader..."
   
     # Step 1: Get Swarm leader hostname
     local LEADER_HOSTNAME
     LEADER_HOSTNAME=$(docker node ls --format '{{.Hostname}}\t{{.ManagerStatus}}' | awk '$2 == "Leader" {print $1}')
   
     if [ -z "$LEADER_HOSTNAME" ]; then
       echo "ERROR: Could not identify Swarm leader node." >&2
       exit 1
     fi
   
     echo "Swarm leader is: $LEADER_HOSTNAME"
   
     # Step 2: Find matching container
     local CONTAINER
     CONTAINER=$(docker ps -a --format '{{.Names}}' | grep "$LEADER_HOSTNAME/ucp-auth-store")
   
     if [ -z "$CONTAINER" ]; then
       echo "ERROR: Could not find ucp-auth-store container on leader node ($LEADER_HOSTNAME)." >&2
       exit 1
     fi
   
     echo "Using container: $CONTAINER"
   
     # Step 3: Perform the copy with retries
     local RETRIES=3
     local SUCCESS=false
     for i in $(seq 1 $RETRIES); do
       if docker cp "$CONTAINER:/var/data" db_data/enzi; then
         SUCCESS=true
         break
       fi
       echo "Retry $i failed. Retrying in 3 seconds..."
       sleep 3
     done
   
     if ! $SUCCESS; then
       echo "ERROR: Failed to copy eNZi DB after $RETRIES attempts." >&2
       exit 1
     fi
   }
   
   copyMsrDb() {
     check_docker
     mkdir -p db_data
     echo "Copying MSR DB..."
   
     REPLICA_ID=$(docker container ls --format '{{.Names}}' -f name=dtr-rethink | awk -F'-' '{print $NF}' | sort | head -n1)
     if [[ -z "$REPLICA_ID" ]]; then
       echo "Error: Could not determine DTR replica ID." >&2
       exit 1
     fi
   
     local RETRIES=3
     local SUCCESS=false
     for i in $(seq 1 $RETRIES); do
       if docker cp dtr-rethinkdb-"$REPLICA_ID":/data db_data/msr; then
         SUCCESS=true
         break
       fi
       echo "Retry $i failed. Retrying in 3 seconds..."
       sleep 3
     done
   
     if ! $SUCCESS; then
       echo "ERROR: Failed to copy MSR DB after $RETRIES attempts." >&2
       exit 1
     fi
   }
   
   startEnziDb() {
     check_rethinkdb
     echo "Starting eNZi DB on driver port $ENZI_RETHINKDB_PORT and cluster port $ENZI_CLUSTER_PORT..."
     rethinkdb --bind all --no-update-check --no-http-admin \
       --directory ./db_data/enzi/rethinkdb \
       --driver-port "$ENZI_RETHINKDB_PORT" \
       --cluster-port "$ENZI_CLUSTER_PORT"
   }
   
   startMsrDb() {
     check_rethinkdb
     echo "Starting MSR DB on driver port $MSR_RETHINKDB_PORT and cluster port $MSR_CLUSTER_PORT..."
     rethinkdb --bind all --no-update-check --no-http-admin \
       --directory ./db_data/msr/rethink \
       --driver-port "$MSR_RETHINKDB_PORT" \
       --cluster-port "$MSR_CLUSTER_PORT"
   }
   
   # Flags
   COPY_DB=false
   COPY_ENZI=false
   COPY_MSR=false
   START_ENZI=false
   START_MSR=false
   
   # Parse arguments
   TEMP=$(getopt -o cemhv --long copy,copy-enzidb,copy-msrdb,start-enzidb,start-msrdb,help,version,enzi-driver:,enzi-cluster:,msr-driver:,msr-cluster: -n "$SCRIPT_NAME" -- "$@")
   
   if [ $? != 0 ]; then show_help; fi
   eval set -- "$TEMP"
   
   while true; do
     case "$1" in
       -c|--copy) COPY_DB=true; shift ;;
       --copy-enzidb) COPY_ENZI=true; shift ;;
       --copy-msrdb) COPY_MSR=true; shift ;;
       -e|--start-enzidb) START_ENZI=true; shift ;;
       -m|--start-msrdb) START_MSR=true; shift ;;
       --enzi-driver) ENZI_RETHINKDB_PORT="$2"; shift 2 ;;
       --enzi-cluster) ENZI_CLUSTER_PORT="$2"; shift 2 ;;
       --msr-driver) MSR_RETHINKDB_PORT="$2"; shift 2 ;;
       --msr-cluster) MSR_CLUSTER_PORT="$2"; shift 2 ;;
       -v|--version)
         echo "$SCRIPT_NAME version $SCRIPT_VERSION"
         exit 0
         ;;
       -h|--help) show_help ;;
       --) shift; break ;;
       *) echo "Unexpected option: $1"; show_help ;;
     esac
   done
   
   # Show help if no actionable options were passed
   if ! $COPY_DB && ! $COPY_ENZI && ! $COPY_MSR && ! $START_ENZI && ! $START_MSR; then
     show_help
   fi
   
   # Prevent simultaneous start (both are blocking)
   if $START_ENZI && $START_MSR; then
     echo
     echo "ERROR: Cannot start both eNZi and MSR DBs in the same script run."
     echo "These are blocking processes. Please run them in separate terminal sessions."
     echo
     exit 1
   fi
   
   # Prevent mismatched copy/start combinations unless using --copy
   if ! $COPY_DB; then
     if { $COPY_ENZI && $START_MSR; } || { $COPY_MSR && $START_ENZI; }; then
       echo
       echo "ERROR: Cannot mix eNZi and MSR operations in a single invocation."
       echo "For example, do not use --copy-msrdb with --start-enzidb."
       echo "Use consistent options for the same registry component."
       echo
       exit 1
     fi
   fi
   
   # Warn if copying without client bundle
   if $COPY_DB || $COPY_ENZI || $COPY_MSR; then
     check_client_bundle_sourced
   fi
   
   # Perform copy
   if $COPY_DB || $COPY_ENZI; then
     copyEnziDb
   fi
   
   if $COPY_DB || $COPY_MSR; then
     copyMsrDb
   fi
   
   # Start DBs
   if $START_ENZI; then
     startEnziDb
   fi
   
   if $START_MSR; then
     startMsrDb
   fi
   

4. Start the required local databases:

   Note

   You need to source a client bundle that has access to the source registry to use the copy commands.

   MSR 2.9

   Important

   Both commands must be executed, and the processes must remain active throughout the migration. Select one of the following options to ensure they stay running:

   • Open each command in a separate terminal window or tab.

   • Run each command in the background by appending &.

   1. Enzi database access

      To copy and start a local Enzi database instance, run:

      ./manage_source_registry_db.sh --copy-enzidb --start-enzidb
      

   2. MSR RethinkDB access

      To copy and start a local MSR RethinkDB instance, run:

      ./manage_source_registry_db.sh --copy-msrdb --start-msrdb
      

   MSR 3.1

   1. In the directory where you saved the manage_source_registry_db.sh script, create the required local directories:

      mkdir -p ./db_data/enzi/rethinkdb
      mkdir -p ./db_data/msr/rethink
      

   2. Identify the name of the MSR 3.1 RethinkDB cluster Pod:

      kubectl get pod -l \
        app.kubernetes.io/component=cluster,app.kubernetes.io/name=rethinkdb,statefulset.kubernetes.io/pod-name \
        -o name | grep -- '-0$'
      

   3. Copy the RethinkDB data from the Pod to the local directories:

      kubectl cp pod/<MSR3-RETHINKDB-CLUSTER-POD-NAME>:/data/db ./db_data/enzi/rethinkdb/
      kubectl cp pod/<MSR3-RETHINKDB-CLUSTER-POD-NAME>:/data/db ./db_data/msr/rethink/
      

      Important

      You must run both copy commands to retrieve the Enzi and MSR databases.

   4. Start the Enzi database:

      ./manage_source_registry_db.sh --start-enzidb
      

   5. Start the MSR RethinkDB:

      ./manage_source_registry_db.sh --start-msrdb