Install an HA MSR deployment

Note

The instruction herein is a supplement to the MSR installation procedure detailed at Installation Guide.

Kubernetes deployments

High availability (HA) MSR deployments require a Kubernetes environment that have:

  • At least two different nodes on which to run an MSR deployment

  • An additional node on which to replicate the RethinkDB cluster, to ensure fault tolerance

To install an HA MSR deployment using the MSR Operator:

  1. Modify your cr-sample-manifest.yaml file to includes the values contained in the following YAML example:

    spec:
      podAntiAffinityPreset: hard
      rethinkdb:
        cluster:
          replicaCount: 3
        proxy:
          replicaCount: 2
      enzi:
        api:
          replicaCount: 2
        worker:
          replicaCount: 2
      nginx:
        replicaCount: 2
      garant:
        replicaCount: 2
      api:
        replicaCount: 2
      jobrunner:
        deployments:
          default:
            replicaCount: 2
      notarySigner:
        replicaCount: 2
      notaryServer:
        replicaCount: 2
      registry:
        replicaCount: 2
    

    Note

    You can edit the replica counts in the custom resource manifest, but be aware that rethinkdb.cluster.replicaCount must always be an odd number. Refer to the RethinkDB scaling chart for details.

  2. Invoke the following command to run the webhook health check and apply the changes to the custom resource:

    kubectl wait --for=condition=ready pod -l \
    app.kubernetes.io/name="msr-operator" && kubectl apply -f cr-sample-manifest.yaml
    
  3. Verify completion of the reconciliation process for the custom resource:

    kubectl get msrs.msr.mirantis.com
    kubectl get rethinkdbs.rethinkdb.com
    

    To troubleshoot the reconciliation process, run the following commands:

    kubectl describe msrs.msr.mirantis.com
    kubectl describe rethinkdbs.rethinkdb.com
    

    Review the MSR Operator Pod logs for more detailed results:

    kubectl logs <msr-operator-pod-name>
    
  4. Optional. Another way to troubleshoot the reconciliation process is to monitor the cluster scaling in the RethinkDB admin console.

To install an HA MSR deployment using a Helm chart:

  1. Create an ha.yaml file with the following content:

    The ha.yaml file sample
    global:
      podAntiAffinityPreset: hard
      rethinkdb:
        cluster:
          replicaCount: 3
        proxy:
          replicaCount: 2
      enzi:
        api:
          replicaCount: 2
        worker:
          replicaCount: 2
      nginx:
        replicaCount: 2
      garant:
        replicaCount: 2
      api:
        replicaCount: 2
      jobrunner:
        deployments:
          default:
            replicaCount: 2
      notarySigner:
        replicaCount: 2
      notaryServer:
        replicaCount: 2
      registry:
        replicaCount: 2
    

    Note

    You can edit the replica counts in the ha.yaml file. However, you must make sure that rethinkdb.cluster.replicaCount is always an odd number. Refer to the RethinkDB scaling chart for details.

  2. Use Helm to apply the YAML file to a new installation:

    helm install msrofficial/msr -f ha.yaml
    

Swarm deployments

You must have at least three worker nodes to run a robust and fault-tolerant high availability (HA) MSR deployment.

Note

The procedure that follows is supplementary to the MSR installation procedure. Refer to Install MSR online for the comprehensive installation instructions.

  1. SSH into a manager node.

  2. Obtain a list of non-manager nodes along with their node IDs:

    docker node ls --format "{{ .ID }}" --filter "role=worker"
    
  3. Install an HA deployment, specifying the node IDs of the workers on which MSR will run:

    docker run \
      --rm \
      --it \
      -v /var/run/docker.sock:/var/run/docker.sock \
      -v <path-to-values.yml>:/config/values.yml \
      registry.mirantis.com/msr/msr-installer:<msr-version> \
      install \
      --node <node-id> \
      --node <node-id> \
      --node <node-id> \
      --https_port 443 \
      --http_port 80
    

    Important

    You must install MSR onto an odd number of worker nodes, the reason for which is that RethinkDB uses a raft consensus algorithm to ensure data consistency and fault tolerance.

  4. Review the status of the deployed services:

    docker stack services msr