Back up TF databases

This section describes how to back up the TF data manually.

For the details on how to perform the automatic TF data backup using the tf-dbbackup-job cron job, refer to the Periodic Tungsten Fabric database backups section in Reference Architecture.

Warning

Use the tf-dbbackup-job cron job only if your cluster is not affected by issue 53831. The issue causes some container images silently truncate any Cassandra column family that has more than 5000 rows during export, without any error or warning, which can drop real user data. Until the cluster is updated to a release that includes the fix, use the manual backup procedure below, which includes a hot fix that prevents the truncation.

To verify whether your container image is affected:

Inside the api container of the tf-config pod, change the directory to the Python config management packages:
- Since MOSK 25.1:
```
cd /usr/lib/python3.6/site-packages/cfgm_common
```
- Before MOSK 25.1:
```
cd /usr/lib/python2.7/site-packages/cfgm_common
```
Verify whether db_json_exim.py contains the affected code:
```
grep -n "current_rows" db_json_exim.py
```

If the output contains the line range = session.execute(cql).current_rows, the container image is affected.

Manually back up TF data in the JSON format

Disable the Neutron server that is used by OpenStack to communicate with the Tungsten Fabric API:

Note

The database changes associated with northbound APIs must be stopped on all systems before performing any backup operations.
1. Scale the neutron-server deployment to 0 replicas:
```
kubectl -n openstack scale deploy neutron-server --replicas 0
```
2. Verify the number of replicas:
```
kubectl -n openstack get deploy neutron-server
```
  Example of a positive system response:
```
NAME             READY   UP-TO-DATE   AVAILABLE   AGE
neutron-server   0/0     0            0           6d22h
```

Join the Tungsten Fabric API that is part of the config DaemonSet:

Obtain the tf-config pod:

kubectl -n tf get pod -l tungstenfabric=config

Example of a system response:

NAME              READY   STATUS    RESTARTS   AGE
tf-config-6ppvc   5/5     Running   0         5d4h
tf-config-rgqqq   5/5     Running   0         5d4h
tf-config-sb4kk   5/5     Running   0         5d4h

Join the Bash shell of one of the api container from the previous step:

kubectl -n tf exec -it tf-config-<hash> -c api -- bash

Example of a system response:

(config-api[hostname])[<USER>@<HOSTNAME> /]$

Inside the api container, change the directory to the Python config management packages:
Since MOSK 25.1
cd /usr/lib/python3.6/site-packages/cfgm_common
Before MOSK 25.1
cd /usr/lib/python2.7/site-packages/cfgm_common
If applicable, apply the hot fix for the known issue 53831 to db_json_exim.py:

Warning

Skipping this step on the affected clusters, silently truncates any Cassandra column family that has more than 5000 rows during export, causing real data to be missing from the resulting dump.
1. Verify whether the container image is affected:
```
grep -n "current_rows" db_json_exim.py
```
  Example of a system response that indicates the container image is affected:
```
range = session.execute(cql).current_rows
```
2. If the container image is affected, apply the patch:
```
sed -i "s/range = session.execute(cql).current_rows/range = session.execute(cql)/" db_json_exim.py
```
  Note
  
  This patch only changes the file inside the running container and does not persist across container or pod restarts. Repeat this step before every manual backup until the cluster is updated to a release that includes the fix for the known issue 53831.
  
  Otherwise, if the container image is not affected, skip this step.

Back up data using db_json_exim in JSON format:

python db_json_exim.py --export-to /tmp/db-dump.json

Verify the created dump:

cat /tmp/db-dump.json | python -m json.tool | less

Copy the backup from the container:

kubectl -n tf cp tf-config-<hash>:/tmp/db-dump.json <DESTINATION-PATH-FOR-BACKUP>

On the same config node, copy the contrail-api.conf file from the container to the host:

kubectl -n tf cp tf-config-<hash>:/etc/contrail/contrail-api.conf <DESTINATION-PATH-FOR-CONF>

Enable the Neutron server:

Scale the neutron-server deployment back to the desired number of replicas. Default is 3.

kubectl -n openstack scale deploy neutron-server --replicas <DESIRED-NUM-REPLICAS>

Verify the number of replicas:

kubectl -n openstack get deploy neutron-server

Example of a system response:

NAME             READY   UP-TO-DATE   AVAILABLE   AGE
neutron-server   3/3     3            3           6d23h