Back up TF databases

Tungsten Fabric (TF) uses Cassandra and ZooKeeper to store its data. Cassandra is a fault-tolerant and horizontally scalable database that provides persistent storage of configuration and analytics data. ZooKeeper is used by TF for allocation of unique object identifiers and transactions implementation.

Starting from MOSK 22.1, you can perform the automatic TF data backup using the tf-dbbackup-job cron job. For the earlier MOSK versions, you should perform the manual backup using the db_json_exim.py script, which creates a dump of Cassandra and ZooKeeper databases.

To prevent data loss, Mirantis recommends that you simultaneously back up the ZooKeeper database dedicated to configuration services and the Cassandra database.

Caution

The backup of database must be consistent across all systems because the state of the Tungsten Fabric databases is associated with other system databases, such as OpenStack databases.

Automatically back up TF data in the JSON format

Available since MOSK 22.1

To automatically back up TF databases in the JSON format, you can use the tf-dbbackup-job cron job. By default, it is disabled. To back up the TF databases, enable tf-dbBackup in the TF operator CR:

spec:
  controllers:
     tf-dbBackup:
       enabled: true

By default, the tf-dbbackup-job job is scheduled for weekly execution, allocating PVC of 5 Gi size for storing backups and keeping 5 previous backups. To configure the backup parameters according to the needs of your cluster, use the following structure:

spec:
  controllers:
     tf-dbBackup:
       enabled: true
       dataCapacity: 30Gi
       schedule: "0 0 13 * 5"
       storedBackups: 10

To temporarily disable tf-dbbackup-job, suspend the job:

spec:
  controllers:
     tf-dbBackup:
       enabled: true
       suspend: true

To delete the tf-dbbackup-job job:

  1. Disable tf-dbBackup in the TF operator CR:

    spec:
      controllers:
         tf-dbBackup:
                enabled: false
    
  2. Delete the tfdbbackup Kubernetes object:

    kubectl -n tf delete tfdbbackups.tf-dbbackup.tf.mirantis.com tf-dbbackup
    

Manually back up TF data in the JSON format

  1. Disable the Neutron server that is used by OpenStack to communicate with the Tungsten Fabric API:

    Note

    The database changes associated with northbound APIs must be stopped on all systems before performing any backup operations.

    1. Scale the neutron-server deployment to 0 replicas:

      kubectl -n openstack scale deploy neutron-server --replicas 0
      
    2. Verify the number of replicas:

      kubectl -n openstack get deploy neutron-server
      

      Example of a positive system response:

      NAME             READY   UP-TO-DATE   AVAILABLE   AGE
      neutron-server   0/0     0            0           6d22h
      
  2. Join the Tungsten Fabric API that is part of the config DaemonSet:

    1. Obtain the tf-config pod:

      kubectl -n tf get pod -l tungstenfabric=config
      

      Example of a system response:

      NAME              READY   STATUS    RESTARTS   AGE
      tf-config-6ppvc   5/5     Running   0         5d4h
      tf-config-rgqqq   5/5     Running   0         5d4h
      tf-config-sb4kk   5/5     Running   0         5d4h
      
    2. Join the Bash shell of one of the api container from the previous step:

      kubectl -n tf exec -it tf-config-<hash> -c api -- bash
      

      Example of a system response:

      (config-api[hostname])[<USER>@<HOSTNAME> /]$
      
  3. Inside the api container, change the directory to the Python config management packages:

    cd /usr/lib/python2.7/site-packages/cfgm_common
    
  4. Back up data using db_json_exim in JSON format:

    python db_json_exim.py --export-to /tmp/db-dump.json
    
  5. Verify the created dump:

    cat /tmp/db-dump.json | python -m json.tool | less
    
  6. Copy the backup from the container:

    kubectl -n tf cp tf-config-<hash>:/tmp/db-dump.json <DESTINATION-PATH-FOR-BACKUP>
    
  7. On the same config node, copy the contrail-api.conf file from the container to the host:

    kubectl -n tf cp tf-config-<hash>:/etc/contrail/contrail-api.conf <DESTINATION-PATH-FOR-CONF>
    
  8. Enable the Neutron server:

    1. Scale the neutron-server deployment back to the desired number of replicas. Default is 3.

      kubectl -n openstack scale deploy neutron-server --replicas <DESIRED-NUM-REPLICAS>
      
    2. Verify the number of replicas:

      kubectl -n openstack get deploy neutron-server
      

      Example of a system response:

      NAME             READY   UP-TO-DATE   AVAILABLE   AGE
      neutron-server   3/3     3            3           6d23h
      

See also

Restore TF data