Kubernetes underlay backup

To ensure the safety of your cloud data, Mirantis provides several backup custom resources for the Kubernetes underlay of management and MOSK clusters. Mirantis highly recommends backing up your clusters regularly as well as configuring reliable remote storage and encryption for your backups.

Storage for backup data

By default, MOSK backup-controller stores the cluster data locally on one of the cluster manager nodes.

To store the backup data on remote storage, a cloud operator can use either NFS or Amazon Simple Storage Service (S3).

For NFS, configure it to allow mounting the share onto the cluster manager nodes. For S3, use the RCloneCredential custom resource (CR):

spec:
  remoteConfig:
    access_key_id: <key-id>
    endpoint: <endpoint>
    type: s3
  remoteConfigSecretKey: <secret-key-name>
  remoteName: <remote-name>
  remoteSecret:
    value: <secret-key-value>

Important

Remote storage and encryption of backup data are highly recommended for production environments. For the configuration details, refer to RCloneCredential resource and KeyEncryptionKey resource.

Backup encryption

Security compliance may require storing backups in an encrypted format. MOSK enables encryption of backups using the KeyEncryptionKey CR:

spec:
  secret:
    value: <encryption-key>

KeyEncryptionKey requires remote storage to be configured. For details, refer to KeyEncryptionKey resource.

Default backup settings

By default, MOSK creates a local backup of the Kubernetes underlay before and after updates of management and MOSK clusters using the ClusterBackup CR. You can also use this CR to create manual backups between releases as well as configure remote backup storage and encryption for remote backups. For example, for rclone:

spec:
  parameters:
    keyEncryptionKey: <key-encryption-key-name>
    rclone:
      bucket: <bucket-name>
      credential: <rclone-credential-name>
  targetCluster: <cluster-name>

If the parameters section is not set, MOSK uses the backup section values of the target Cluster object spec, if any. If backup parameters are configured neither through the Cluster nor through the ClusterBackup object, a local backup will be created and stored on one of the manager nodes of the target cluster.

To configure default settings for remote backup storage and encryption during cluster updates, use the backup section in the Cluster CR. For example:

spec:
  ...
  providerSpec:
    value:
      ...
      backup:
       keyEncryptionKey: <key-encryption-key-name>
       rclone:
         bucket: <bucket-name>
         credential: <rclone-credential-name>

For the configuration details, refer to ClusterBackup resource and Create backups of the Kubernetes underlay.

Backup during cluster updates

The Kubernetes underlay backup is performed automatically before and after update of management and MOSK clusters using the ClusterBackup object.

If remote backup storage and encryption are configured, the backup files are encrypted by the KeyEncryptionKey object and then uploaded to remote storage.

If remote backup storage is not configured, a local backup is performed on one of the manager nodes of the target cluster.

The name of the ClusterBackup object for a MOSK cluster before update consists of the following parts:

  • Distribution-specific prefix, which is ucp-backup for an MKE-based cluster and k0s-backup for a k0s-based one

  • Version of the Kubernetes distribution used in the existing Cluster release

  • Existing Cluster release

  • Cluster name

  • First part of the cluster UID

For example:

ucp-backup-3.7.24-mosk-1-20.0.0-test-1234567890

The name of the ClusterBackup object for a management cluster before update, which is triggered by a MOSK cluster update, consists of the following parts:

  • Distribution-specific prefix described above

  • Version of the Kubernetes distribution used in the existing Cluster release of the MOSK cluster

  • Existing Cluster release of the MOSK cluster

  • Name of the management cluster

  • First part of the management cluster UID

  • Namespace and name of the MOSK cluster

  • First part of the MOSK cluster UID

For example:

ucp-backup-3.7.24-mosk-1-20.0.0-mgmt-0123456789-ns-test-1234567890

After update, backup object names differ in prefix, which is either ucp-backup-after-update or k0s-backup-after-update, and the Cluster release that indicates the target release of the update. For example:

ucp-backup-after-update-3.7.28-mosk-1-21.1.0-mgmt-0123456789-ns-test-1234567890

Caution

A management cluster migrated from MKE to k0s keeps the previously created ClusterBackup objects with the ucp-backup prefix. Such backups contain MKE data and cannot be used to restore the cluster after the migration.

The pre-update backup step occurs before the update of manager nodes. The post-update backup step occurs after the update of the last worker update group. Both steps are part of the ClusterUpdatePlan custom resource for the Granularly update MOSK using the ClusterUpdatePlan object procedure.

Periodic backups

By default, periodic backups are turned off. Though, a cloud operator can easily enable this capability by adding the following structure to the ClusterBackupSchedule CR depending on the storage type:

  • For rclone:

spec:
  parameters:
    keyEncryptionKey: <key-encryption-key-name>
    rclone:
      bucket: <bucket-name>
      credential: <rclone-credential-name>
  schedule: “0 * * * *”
  targetCluster: <cluster-name>

  • For NFS:

spec:
  parameters:
    keyEncryptionKey: <key-encryption-key-name>
    nfsUploadUrl: <nfs-upload-url>
  schedule: “0 * * * *”
  targetCluster: <cluster-name>

Backup schedule is described in the standard cron schedule format. For example, set 0 0 * * * for daily backups at 00:00. The schedule is set to the time zone of the management cluster.

For configuration details, refer to ClusterBackupSchedule resource and Create backups of the Kubernetes underlay.