Use Kubernetes on Windows Server nodes

Observe the following prerequisites prior to using Kubernetes on Windows Server nodes.

  1. Install MKE.

  2. Create a single-node, linux-only cluster.

Note

Running Kubernetes on Windows Server nodes is only supported on MKE 3.3.0 and later. If you want to run Kubernetes on Windows Server nodes on a cluster that is currently running an earlier version of MKE than 3.3.0, you must perform a fresh install of MKE 3.3.0 or later.

Add Windows Server nodes

  1. Log in to the MKE web UI.

  2. In the left-side navigation panel, navigate to Shared Resources > Nodes and click Add Node.

  3. Under NODE TYPE, select Windows. Windows Server nodes can only be workers.

  4. Optional. Specify custom listen and advertise addresses by using the relevant slider.

  5. Copy the command generated at the bottom of the Add Node page, which includes the join-token.

    Example command:

    docker swarm join \
    --token SWMTKN-1-2is7c14ff43tq1g61ubc5egvisgilh6m8qxm6dndjzgov9qjme-4388n8bpyqivzudz4fidqm7ey \
    172.31.2.154:2377
    
  6. Add your Windows Server node to the MKE cluster by running the docker swarm join command copied in the previous step.

Validate your cluster

To validate your cluster using the MKE web UI:

  1. Log in to the MKE web UI.

  2. In the left-side navigation panel, navigate to Nodes. A green circle indicates a healthy node state. All nodes should be green.

  3. Change each node orchestrator to Kubernetes:

    1. Click on the node.

    2. In the upper-right corner, click the slider icon.

    3. In the Role section of the Details tab, select Kubernetes under ORCHESTRATOR TYPE.

    4. Click Save.

    5. Repeat the above steps for each node.


To validate your cluster using the command line:

  1. View the status of all the nodes in your cluster:

    kubectl get nodes
    

    Your nodes should all have a status value of Ready, as in the following example:

    NAME                   STATUS   ROLES    AGE     VERSION
    user-135716-win-0      Ready    <none>   2m16s   v1.17.2
    user-7d985f-ubuntu-0   Ready    master   4m55s   v1.17.2-docker-d-2
    user-135716-win-1      Ready    <none>   1m12s   v1.17.2
    
  2. Change each node orchestrator to Kubernetes:

    docker node update <node name> --label-add com.docker.ucp.orchestrator.kubernetes=true
    
  3. Repeat the last step for each node.

  4. Deploy a workload on your cluster to verify that everything works as expected.

Troubleshoot

If you cannot join your Windows Server node to the cluster, confirm that the correct processes are running on the node.

  1. Verify that the calico-node process is operational:

    PS C:\> Get-Process calico-node
    

    Example output:

    Handles  NPM(K)    PM(K)      WS(K)     CPU(s)     Id  SI ProcessName
    -------  ------    -----      -----     ------     --  -- -----------
        276      17    33284      40948      39.89   8132   0 calico-node
    
  2. Verify that the kubelet process is operational:

    PS C:\> Get-Process kubelet
    

    Example output:

    Handles  NPM(K)    PM(K)      WS(K)     CPU(s)     Id  SI ProcessName
    -------  ------    -----      -----     ------     --  -- -----------
        524      23    47332      73380     828.50   6520   0 kubelet
    
  3. Verify that the kube-proxy process is operational:

    PS C:\> Get-Process kube-proxy
    

    Example output:

    Handles  NPM(K)    PM(K)      WS(K)     CPU(s)     Id  SI ProcessName
    -------  ------    -----      -----     ------     --  -- -----------
        322      19    25464      33488      21.00   7852   0 kube-proxy
    
  4. If any of the process verifications indicate a problem, review the container logs that bootstrap the Kubernetes components on the Windows node:

    docker container logs (docker container ls --filter name=ucp-kubelet-win -q)
    docker container logs (docker container ls --filter name=ucp-kube-proxy -q)
    docker container logs (docker container ls --filter name=ucp-tigera-node-win -q)
    docker container logs (docker container ls --filter name=ucp-tigera-felix-win -q)
    

Deploy a workload on Windows Server

The following procedure deploys a complete web application on IIS servers as Kubernetes Services. The example workload includes an MSSQL database and a load balancer. The procedure includes the following tasks:

  • Namespace creation

  • Pod and deployment scheduling

  • Kubernetes Service provisioning

  • Application workload deployment

  • Pod, Node, and Service configuration

  1. Download and configure the client bundle.

  2. Create the following namespace file:

    demo-namespace.yaml
    apiVersion: v1
    kind: Namespace
    metadata:
      name: demo
    
  3. Create a namespace:

    kubectl create -f demo-namespace.yaml
    
  4. Create the following Windows web server file:

    win-webserver.yaml
    apiVersion: v1
    kind: Service
    metadata:
      name: win-webserver
      labels:
        app: win-webserver
      namespace: demo
    spec:
      ports:
      - port: 80
        targetPort: 80
      selector:
        app: win-webserver
      type: NodePort
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: win-webserver
      labels:
        app: win-webserver
      namespace: demo
    spec:
      replicas: 2
      selector:
        matchLabels:
          app: win-webserver
      template:
        metadata:
          labels:
            app: win-webserver
        spec:
          affinity:
            podAntiAffinity:
              requiredDuringSchedulingIgnoredDuringExecution:
              - labelSelector:
                  matchExpressions:
                  - key: app
                    operator: In
                    values:
                      - win-webserver
                topologyKey: "kubernetes.io/hostname"
          containers:
          - name: windowswebserver
            image: mcr.microsoft.com/windows/servercore:ltsc2019
            command:
            - powershell.exe
            - -command
            - "<#code used from https://gist.github.com/wagnerandrade/5424431#> ; $$listener = New-Object System.Net.HttpListener ; $$listener.Prefixes.Add('http://*:80/') ; $$listener.Start() ; $$callerCounts = @{} ; Write-Host('Listening at http://*:80/') ; while ($$listener.IsListening) { ;$$context = $$listener.GetContext() ;$$requestUrl = $$context.Request.Url ;$$clientIP = $$context.Request.RemoteEndPoint.Address ;$$response = $$context.Response ;Write-Host '' ;Write-Host('> {0}' -f $$requestUrl) ;  ;$$count = 1 ;$$k=$$callerCounts.Get_Item($$clientIP) ;if ($$k -ne $$null) { $$count += $$k } ;$$callerCounts.Set_Item($$clientIP, $$count) ;$$ip=(Get-NetAdapter | Get-NetIpAddress); $$header='<html><body><H1>Windows Container Web Server</H1>' ;$$callerCountsString='' ;$$callerCounts.Keys | % { $$callerCountsString+='<p>IP {0} callerCount {1} ' -f $$ip[1].IPAddress,$$callerCounts.Item($$_) } ;$$footer='</body></html>' ;$$content='{0}{1}{2}' -f $$header,$$callerCountsString,$$footer ;Write-Output $$content ;$$buffer = [System.Text.Encoding]::UTF8.GetBytes($$content) ;$$response.ContentLength64 = $$buffer.Length ;$$response.OutputStream.Write($$buffer, 0, $$buffer.Length) ;$$response.Close() ;$$responseStatus = $$response.StatusCode ;Write-Host('< {0}' -f $$responseStatus)  } ; "
          nodeSelector:
            kubernetes.io/os: windows
    

    Note

    If the Windows nodes in your MKE cluster are Windows Server 2022, edit the image tag in the win-webserver.yaml file from ltsc2019 to ltsc2022.

  5. Create the web service:

    kubectl create -f win-webserver.yaml
    

    Expected output:

    service/win-webserver created
    deployment.apps/win-webserver created
    
  6. Verify creation of the Kubernetes Service:

    kubectl get service --namespace demo
    

    Expected output:

    NAME            TYPE       CLUSTER-IP    EXTERNAL-IP   PORT(S)        AGE
    win-webserver   NodePort   10.96.29.12   <none>        80:35048/TCP   12m
    
  7. Review the pods deployed on your Windows Server worker nodes with inter-pod affinity and anti-affinity.

    Note

    After creating the web service, it may take several minutes for the pods to enter a ready state.

    kubectl get pod --namespace demo
    

    Expected output:

    NAME                            READY   STATUS    RESTARTS   AGE
    win-webserver-8c5678c68-qggzh   1/1     Running   0          6m21s
    win-webserver-8c5678c68-v8p84   1/1     Running   0          6m21s
    
  8. Review the detailed status of pods deployed:

    kubectl describe pod win-webserver-8c5678c68-qggzh --namespace demo
    
  9. From a kubectl client, access the web service using node-to-pod communication across the network:

    kubectl get pods --namespace demo -o wide
    

    Example output:

    NAME                            READY   STATUS    RESTARTS   AGE   IP              NODE              NOMINATED NODE   READINESS GATES
    win-webserver-8c5678c68-qggzh   1/1     Running   0          16m   192.168.77.68   user-135716-win-1 <none>           <none>
    win-webserver-8c5678c68-v8p84   1/1     Running   0          16m   192.168.4.206   user-135716-win-0 <none>           <none>
    
  10. SSH into the master node:

    ssh -o ServerAliveInterval=15 root@<master-node>
    
  11. Use curl to access the web service by way of the CLUSTER-IP listed for the win-webserver service.

    curl 10.96.29.12
    

    Example output:

    <html><body><H1>Windows Container Web Server</H1><p>IP 192.168.77.68 callerCount 1 </body></html>
    
  12. Run the curl command a second time. You can see the second request load-balanced to a different pod:

    curl 10.96.29.12
    

    Example output:

    <html><body><H1>Windows Container Web Server</H1><p>IP 192.168.4.206 callerCount 1 </body></html>
    
  13. From a kubectl client, access the web service using pod-to-pod communication across the network:

    kubectl get service --namespace demo
    

    Expample output:

    NAME            TYPE       CLUSTER-IP    EXTERNAL-IP   PORT(S)        AGE
    win-webserver   NodePort   10.96.29.12   <none>        80:35048/TCP   12m
    
  14. Review the pod status:

    kubectl get pods --namespace demo -o wide
    

    Example output:

    NAME                            READY   STATUS    RESTARTS   AGE   IP              NODE              NOMINATED NODE   READINESS GATES
    win-webserver-8c5678c68-qggzh   1/1     Running   0          16m   192.168.77.68   user-135716-win-1 <none>           <none>
    win-webserver-8c5678c68-v8p84   1/1     Running   0          16m   192.168.4.206   user-135716-win-0 <none>           <none>
    
  15. Exec into the web service:

    kubectl exec -it win-webserver-8c5678c68-qggzh --namespace demo cmd
    

    Example output:

    Microsoft Windows [Version 10.0.17763.1098]
    (c) 2018 Microsoft Corporation. All rights reserved.
    
  16. Use curl to access the web service:

    C:\>curl 10.96.29.12
    

    Example output:

    <html><body><H1>Windows Container Web Server</H1><p>IP 192.168.77.68
    callerCount 1 <p>IP 192.168.77.68 callerCount 1 </body></html>