Newer documentation is now live.You are currently reading an older version.

Periodic Tungsten Fabric database backups

MOSK enables you to perform the automatic TF data backup in the JSON format using the tf-dbbackup-job cron job. By default, it is disabled.

Warning

Use the tf-dbbackup-job cron job only if your cluster is not affected by issue 53831. The issue causes some container images silently truncate any Cassandra column family that has more than 5000 rows during export, without any error or warning, which can drop real user data. Until the cluster is updated to a release that includes the fix, use the manual backup procedure in Manually back up TF data in the JSON format instead, which includes a hot fix that prevents the truncation.

To verify whether your container image is affected:

  1. Inside the api container of the tf-config pod, change the directory to the Python config management packages:

    • Since MOSK 25.1:

      cd /usr/lib/python3.6/site-packages/cfgm_common
      
    • Before MOSK 25.1:

      cd /usr/lib/python2.7/site-packages/cfgm_common
      
  2. Verify whether db_json_exim.py contains the affected code:

    grep -n "current_rows" db_json_exim.py
    

If the output contains the line range = session.execute(cql).current_rows, the container image is affected.

To back up the TF databases, enable tf-dbBackup in the TF Operator custom resource:

spec:
  features:
     dbBackup:
        enabled: true
spec:
  controllers:
    tf-dbBackup:
      enabled: true

By default, the tf-dbbackup-job job is scheduled for weekly execution, allocating PVC of 5 Gi size for storing backups and keeping 5 previous backups. To configure the backup parameters according to the needs of your cluster, use the following structure:

spec:
  features:
     dbBackup:
        enabled: true
        dataCapacity: 30Gi
        schedule: "0 0 13 * 5"
        storedBackups: 10
spec:
  controllers:
    tf-dbBackup:
      enabled: true
      dataCapacity: 30Gi
      schedule: "0 0 13 * 5"
      storedBackups: 10

To temporarily disable tf-dbbackup-job, suspend the job:

spec:
  features:
     dbBackup:
        enabled: true
        suspend: true
spec:
  controllers:
    tf-dbBackup:
      enabled: true
      suspend: true

To delete the tf-dbbackup-job job, disable tf-dbBackup in the TF Operator custom resource:

spec:
  features:
     dbBackup:
        enabled: false
spec:
  controllers:
    tf-dbBackup:
      enabled: false