Using vSphere Volumes

The vSphere Storage for Kubernetes driver enables customers to address persistent storage requirements for Kubernetes Pods in vSphere environments. The driver allows you to create a Persistent Volume (PV) on a Virtual Machine File System (VMFS), and use it to manage persistent storage requirements independent of pod and VM lifecycle.

Note

Of the three main storage backends offered by vSphere on Kubernetes (VMFS, vSAN, and NFS), Docker supports VMFS.

You can use the vSphere Cloud Provider to manage storage with Kubernetes in MKE 3.1 and later. This includes support for:

  • Volumes

  • PVs

  • StorageClasses and provisioning volumes

Prerequisites

  • Ensure that vsphere.conf is populated according to the vSphere Cloud Provider Configuration Deployment Guide.

  • The disk.EnableUUID value on the worker VMs must be set to True.

  • When running Docker MKE 3.2, you must specify the “vm-uuid” flag in the vsphere.conf file on the MKE managers.

Configure for Kubernetes

Kubernetes cloud providers provide a method of provisioning cloud resources through Kubernetes via the --cloud-provider option. This is to ensure that the kubelet is aware that it must be initialized by the ucp-kube-controller-manager before any work is scheduled.

docker container run --rm -it --name ucp -e REGISTRY_USERNAME=$REGISTRY_USERNAME -e REGISTRY_PASSWORD=$REGISTRY_PASSWORD \
  -v /var/run/docker.sock:/var/run/docker.sock \
  "dockereng/ucp:3.1.0-tp2" \
  install \
  --host-address <HOST_ADDR> \
  --admin-username admin \
  --admin-password XXXXXXXX \
  --cloud-provider=vsphere \
  --image-version latest:

Create a StorageClass

  1. Create a StorageClass with a user specified disk format.

    apiVersion: storage.k8s.io/v1
    kind: StorageClass
    metadata:
      name: fast
    provisioner: kubernetes.io/vsphere-volume
    parameters:
      diskformat: zeroedthick
    

    For example, diskformat can be thin, zeroedthick, or eagerzeroedthick. The default format is thin.

  2. Create a StorageClass with a disk format on a user-specified datastore.

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: fast
provisioner: kubernetes.io/vsphere-volume
parameters:
    diskformat: zeroedthick
    datastore: VSANDatastore

You can also specify the ``datastore`` in the StorageClass. The volume will
be created on the datastore specified in the StorageClass, which in this
case is ``VSANDatastore``. This field is optional. If the datastore is not
specified, then the volume will be created on the datastore specified in the
vSphere configuration file used to initialize the vSphere Cloud Provider.

Deploy vSphere Volumes

After you create a StorageClass, you can create PVs that deploy volumes attached to hosts and mounted inside pods. A PersistentVolumeClaim (PVC) is a claim for storage resources that are bound to a PV when storage resources are granted.

We recommend that you use the StorageClass and PVC resources as these abstraction layers provide more portability as well as control over the storage layer across environments.

To deploy vSphere volumes:

  1. Create a PVC from the plugin. When you define a PVC to use the StorageClass, a PV is created and bound.

  2. Create a reference to the PVC from the Pod.

  3. Start a Pod using the PVC that you defined.