Use Azure Disk Storage¶

You can provide persistent storage for MKE workloads on Microsoft Azure by using Azure Disk Storage. You can either pre-provision Azure Disk Storage to be consumed by Kubernetes Pods, or you can use the Azure Kubernetes integration to dynamically provision Azure Disks as needed.

This guide assumes that you have already provisioned an MKE environment on Microsoft Azure and that you have provisioned a cluster after meeting all of the prerequisites listed in Install MKE on Azure.

To complete the steps in this topic, you must download and configure the client bundle.

Manually provision Azure Disks¶

You can use existing Azure Disks or manually provision new ones to provide persistent storage for Kubernetes Pods. You can manually provision Azure Disks in the Azure Portal, using ARM Templates, or using the Azure CLI. The following example uses the Azure CLI to manually provision an Azure Disk.

Create an environment variable for myresourcegroup:
```
RG=myresourcegroup
```

Provision an Azure Disk:

az disk create \
--resource-group $RG \
--name k8s_volume_1  \
--size-gb 20 \
--query id \
--output tsv

This command returns the Azure ID of the Azure Disk Object.

Example output:

/subscriptions/<subscriptionID>/resourceGroups/<resourcegroup>/providers/Microsoft.Compute/disks/<diskname>

Make note of the Azure ID of the Azure Disk Object returned by the previous step.

You can now create Kubernetes Objects that refer to this Azure Disk. The following example uses a Kubernetes Pod, though the same Azure Disk syntax can be used for DaemonSets, Deployments, and StatefulSets. In the example, the Azure diskName and diskURI refer to the manually created Azure Disk:

$ cat <<EOF | kubectl create -f -
apiVersion: v1
kind: Pod
metadata:
  name: mypod-azuredisk
spec:
  containers:
  - image: nginx
    name: mypod
    volumeMounts:
      - name: mystorage
        mountPath: /data
  volumes:
      - name: mystorage
        azureDisk:
          kind: Managed
          diskName: k8s_volume_1
          diskURI: /subscriptions/<subscriptionID>/resourceGroups/<resourcegroup>/providers/Microsoft.Compute/disks/<diskname>
EOF

Dynamically provision Azure Disks¶

Kubernetes can dynamically provision Azure Disks using the Azure Kubernetes integration, configured at the time of your MKE installation. For Kubernetes to determine which APIs to use when provisioning storage, you must create Kubernetes StorageClass objects specific to each storage back end.

There are two different Azure Disk types that can be consumed by Kubernetes: Azure Disk Standard Volumes and Azure Disk Premium Volumes.

Depending on your use case, you can deploy one or both of the Azure Disk storage classes.

To define the Azure Disk storage class:

Create the storage class:

cat <<EOF | kubectl create -f -
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: standard
provisioner: kubernetes.io/azure-disk
parameters:
  storageaccounttype: <disk-type>
  kind: Managed
EOF

For storageaccounttype, enter Standard_LRS for the standard storage class Premium_LRS for the premium storage class.

Verify which storage classes have been provisioned:

kubectl get storageclasses

Example output:

NAME       PROVISIONER                AGE
premium    kubernetes.io/azure-disk   1m
standard   kubernetes.io/azure-disk   1m

To create an Azure Disk with a PersistentVolumeClaim:

After you create a storage class, you can use Kubernetes Objects to dynamically provision Azure Disks. This is done using Kubernetes PersistentVolumesClaims.

The following example uses the standard storage class and creates a 5 GiB Azure Disk. Alter these values to fit your use case.

Create a PersistentVolumeClaim:

cat <<EOF | kubectl create -f -
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: azure-disk-pvc
spec:
  storageClassName: standard
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi
EOF

Verify the creation of the PersistentVolumeClaim:

kubectl get persistentvolumeclaim

Example output:

NAME              STATUS    VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
azure-disk-pvc    Bound     pvc-587deeb6-6ad6-11e9-9509-0242ac11000b   5Gi        RWO            standard       1m

Verify the creation of the PersistentVolume:

kubectl get persistentvolume

Expected output:

NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS    CLAIM                     STORAGECLASS   REASON    AGE
pvc-587deeb6-6ad6-11e9-9509-0242ac11000b   5Gi        RWO            Delete           Bound     default/azure-disk-pvc    standard                 3m

Verify the creation of a new Azure Disk in the Azure Portal.

To attach the new Azure Disk to a Kubernetes Pod:

You can now mount the Kubernetes PersistentVolume into a Kubernetes Pod. The disk can be consumed by any Kubernetes object type, including a Deployment, DaemonSet, or StatefulSet. However, the following example simply mounts the PersistentVolume into a standalone Pod.

Attach the new Azure Disk to a Kubernetes pod:

cat <<EOF | kubectl create -f -
kind: Pod
apiVersion: v1
metadata:
  name: mypod-dynamic-azuredisk
spec:
  containers:
    - name: mypod
      image: nginx
      ports:
        - containerPort: 80
          name: "http-server"
      volumeMounts:
        - mountPath: "/usr/share/nginx/html"
          name: storage
  volumes:
    - name: storage
      persistentVolumeClaim:
        claimName: azure-disk-pvc
EOF

Data disk capacity of an Azure Virtual Machine¶

Azure limits the number of data disks that can be attached to each Virtual Machine. Refer to Azure Virtual Machine Sizes for this information. Kubernetes prevents Pods from deploying on Nodes that have reached their maximum Azure Disk Capacity. In such cases, Pods remain stuck in the ContainerCreating status, as demonstrated in the following example:

Review Pods:

kubectl get pods

Example output:

NAME                  READY     STATUS              RESTARTS   AGE
mypod-azure-disk      0/1       ContainerCreating   0          4m

Describe the Pod to display troubleshooting logs, which indicate the node has reached its capacity:

kubectl describe pods mypod-azure-disk

Example output:

Warning  FailedAttachVolume  7s (x11 over 6m)  attachdetach-controller  \
AttachVolume.Attach failed for volume "pvc-6b09dae3-6ad6-11e9-9509-0242ac11000b" : \
Attach volume "kubernetes-dynamic-pvc-6b09dae3-6ad6-11e9-9509-0242ac11000b" to instance \
"/subscriptions/<sub-id>/resourceGroups/<rg>/providers/Microsoft.Compute/virtualMachines/worker-03" \
failed with compute.VirtualMachinesClient#CreateOrUpdate: Failure sending request: \
StatusCode=409 -- Original Error: failed request: autorest/azure: \
Service returned an error. Status=<nil> Code="OperationNotAllowed" \
Message="The maximum number of data disks allowed to be attached to a VM of this size is 4." \
Target="dataDisks"