Use Kubernetes on Windows Server nodes#

Observe the following prerequisites prior to using Kubernetes on Windows Server nodes.

Install MKE.
Create a single-node, linux-only cluster.

Note

Running Kubernetes on Windows Server nodes is only supported on MKE 3.3.0 and later. If you want to run Kubernetes on Windows Server nodes on a cluster that is currently running an earlier version of MKE than 3.3.0, you must perform a fresh install of MKE 3.3.0 or later.

Add Windows Server nodes#

Log in to the MKE web UI.
In the left-side navigation panel, navigate to Shared Resources > Nodes and click Add Node.
Under NODE TYPE, select Windows. Windows Server nodes can only be workers.
Optional. Specify custom listen and advertise addresses by using the relevant slider.

Copy the command generated at the bottom of the Add Node page, which includes the join-token.

Example command:

docker swarm join \
--token SWMTKN-1-2is7c14ff43tq1g61ubc5egvisgilh6m8qxm6dndjzgov9qjme-4388n8bpyqivzudz4fidqm7ey \
172.31.2.154:2377

Add your Windows Server node to the MKE cluster by running the docker swarm join command copied in the previous step.

Validate your cluster#

To validate your cluster using the MKE web UI:

Log in to the MKE web UI.
In the left-side navigation panel, navigate to Nodes. A green circle indicates a healthy node state. All nodes should be green.
Change each node orchestrator to Kubernetes:
1. Click on the node.
2. In the upper-right corner, click the slider icon.
3. In the Role section of the Details tab, select Kubernetes under ORCHESTRATOR TYPE.
4. Click Save.
5. Repeat the above steps for each node.

To validate your cluster using the command line:

View the status of all the nodes in your cluster:

kubectl get nodes

Your nodes should all have a status value of Ready, as in the following example:

NAME                   STATUS   ROLES    AGE     VERSION
user-135716-win-0      Ready    <none>   2m16s   v1.17.2
user-7d985f-ubuntu-0   Ready    master   4m55s   v1.17.2-docker-d-2
user-135716-win-1      Ready    <none>   1m12s   v1.17.2

Change each node orchestrator to Kubernetes:

docker node update <node name> --label-add com.docker.ucp.orchestrator.kubernetes=true

Repeat the last step for each node.
Deploy a workload on your cluster to verify that everything works as expected.

Troubleshoot#

If you cannot join your Windows Server node to the cluster, confirm that the correct processes are running on the node.

Verify that the calico-node process is operational:

PS C:\> Get-Process calico-node

Example output:

Handles  NPM(K)    PM(K)      WS(K)     CPU(s)     Id  SI ProcessName
-------  ------    -----      -----     ------     --  -- -----------
    276      17    33284      40948      39.89   8132   0 calico-node

Verify that the kubelet process is operational:

PS C:\> Get-Process kubelet

Example output:

Handles  NPM(K)    PM(K)      WS(K)     CPU(s)     Id  SI ProcessName
-------  ------    -----      -----     ------     --  -- -----------
    524      23    47332      73380     828.50   6520   0 kubelet

Verify that the kube-proxy process is operational:

PS C:\> Get-Process kube-proxy

Example output:

Handles  NPM(K)    PM(K)      WS(K)     CPU(s)     Id  SI ProcessName
-------  ------    -----      -----     ------     --  -- -----------
    322      19    25464      33488      21.00   7852   0 kube-proxy

If any of the process verifications indicate a problem, review the container logs that bootstrap the Kubernetes components on the Windows node:

docker container logs (docker container ls --filter name=ucp-kubelet-win -q)
docker container logs (docker container ls --filter name=ucp-kube-proxy -q)
docker container logs (docker container ls --filter name=ucp-tigera-node-win -q)
docker container logs (docker container ls --filter name=ucp-tigera-felix-win -q)

Deploy a workload on Windows Server#

The following procedure deploys a complete web application on IIS servers as Kubernetes Services. The example workload includes an MSSQL database and a load balancer. The procedure includes the following tasks:

Namespace creation
Pod and deployment scheduling
Kubernetes Service provisioning
Application workload deployment
Pod, Node, and Service configuration
Download and configure the client bundle.

Create the following namespace file:

demo-namespace.yaml

apiVersion: v1
kind: Namespace
metadata:
  name: demo

Create a namespace:
```
kubectl create -f demo-namespace.yaml
```

Create the following Windows web server file:

win-webserver.yaml

apiVersion: v1
kind: Service
metadata:
  name: win-webserver
  labels:
    app: win-webserver
  namespace: demo
spec:
  ports:
  - port: 80
    targetPort: 80
  selector:
    app: win-webserver
  type: NodePort
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: win-webserver
  labels:
    app: win-webserver
  namespace: demo
spec:
  replicas: 2
  selector:
    matchLabels:
      app: win-webserver
  template:
    metadata:
      labels:
        app: win-webserver
    spec:
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchExpressions:
              - key: app
                operator: In
                values:
                  - win-webserver
            topologyKey: "kubernetes.io/hostname"
      containers:
      - name: windowswebserver
        image: mcr.microsoft.com/windows/servercore:ltsc2019
        command:
        - powershell.exe
        - -command
        - "<#code used from https://gist.github.com/wagnerandrade/5424431#> ; $$listener = New-Object System.Net.HttpListener ; $$listener.Prefixes.Add('http://*:80/') ; $$listener.Start() ; $$callerCounts = @{} ; Write-Host('Listening at http://*:80/') ; while ($$listener.IsListening) { ;$$context = $$listener.GetContext() ;$$requestUrl = $$context.Request.Url ;$$clientIP = $$context.Request.RemoteEndPoint.Address ;$$response = $$context.Response ;Write-Host '' ;Write-Host('> {0}' -f $$requestUrl) ;  ;$$count = 1 ;$$k=$$callerCounts.Get_Item($$clientIP) ;if ($$k -ne $$null) { $$count += $$k } ;$$callerCounts.Set_Item($$clientIP, $$count) ;$$ip=(Get-NetAdapter | Get-NetIpAddress); $$header='<html><body><H1>Windows Container Web Server</H1>' ;$$callerCountsString='' ;$$callerCounts.Keys | % { $$callerCountsString+='<p>IP {0} callerCount {1} ' -f $$ip[1].IPAddress,$$callerCounts.Item($$_) } ;$$footer='</body></html>' ;$$content='{0}{1}{2}' -f $$header,$$callerCountsString,$$footer ;Write-Output $$content ;$$buffer = [System.Text.Encoding]::UTF8.GetBytes($$content) ;$$response.ContentLength64 = $$buffer.Length ;$$response.OutputStream.Write($$buffer, 0, $$buffer.Length) ;$$response.Close() ;$$responseStatus = $$response.StatusCode ;Write-Host('< {0}' -f $$responseStatus)  } ; "
      nodeSelector:
        kubernetes.io/os: windows

Note

If the Windows nodes in your MKE cluster are Windows Server 2022, edit the image tag in the win-webserver.yaml file from ltsc2019 to ltsc2022.

Create the web service:

kubectl create -f win-webserver.yaml

Expected output:

service/win-webserver created
deployment.apps/win-webserver created

Verify creation of the Kubernetes Service:

kubectl get service --namespace demo

Expected output:

NAME            TYPE       CLUSTER-IP    EXTERNAL-IP   PORT(S)        AGE
win-webserver   NodePort   10.96.29.12   <none>        80:35048/TCP   12m

Review the pods deployed on your Windows Server worker nodes with inter-pod affinity and anti-affinity.

Note

After creating the web service, it may take several minutes for the pods to enter a ready state.

kubectl get pod --namespace demo

Expected output:

NAME                            READY   STATUS    RESTARTS   AGE
win-webserver-8c5678c68-qggzh   1/1     Running   0          6m21s
win-webserver-8c5678c68-v8p84   1/1     Running   0          6m21s

Review the detailed status of pods deployed:

kubectl describe pod win-webserver-8c5678c68-qggzh --namespace demo

From a kubectl client, access the web service using node-to-pod communication across the network:

kubectl get pods --namespace demo -o wide

Example output:

NAME                            READY   STATUS    RESTARTS   AGE   IP              NODE              NOMINATED NODE   READINESS GATES
win-webserver-8c5678c68-qggzh   1/1     Running   0          16m   192.168.77.68   user-135716-win-1 <none>           <none>
win-webserver-8c5678c68-v8p84   1/1     Running   0          16m   192.168.4.206   user-135716-win-0 <none>           <none>

SSH into the master node:

ssh -o ServerAliveInterval=15 root@<master-node>

Use curl to access the web service by way of the CLUSTER-IP listed for the win-webserver service.

curl 10.96.29.12

Example output:

<html><body><H1>Windows Container Web Server</H1><p>IP 192.168.77.68 callerCount 1 </body></html>

Run the curl command a second time. You can see the second request load-balanced to a different pod:

curl 10.96.29.12

Example output:

<html><body><H1>Windows Container Web Server</H1><p>IP 192.168.4.206 callerCount 1 </body></html>

From a kubectl client, access the web service using pod-to-pod communication across the network:

kubectl get service --namespace demo

Expample output:

NAME            TYPE       CLUSTER-IP    EXTERNAL-IP   PORT(S)        AGE
win-webserver   NodePort   10.96.29.12   <none>        80:35048/TCP   12m

Review the pod status:

kubectl get pods --namespace demo -o wide

Example output:

NAME                            READY   STATUS    RESTARTS   AGE   IP              NODE              NOMINATED NODE   READINESS GATES
win-webserver-8c5678c68-qggzh   1/1     Running   0          16m   192.168.77.68   user-135716-win-1 <none>           <none>
win-webserver-8c5678c68-v8p84   1/1     Running   0          16m   192.168.4.206   user-135716-win-0 <none>           <none>

Exec into the web service:

kubectl exec -it win-webserver-8c5678c68-qggzh --namespace demo cmd

Example output:

Microsoft Windows [Version 10.0.17763.1098]
(c) 2018 Microsoft Corporation. All rights reserved.

Use curl to access the web service:

C:\>curl 10.96.29.12

Example output:

<html><body><H1>Windows Container Web Server</H1><p>IP 192.168.77.68
callerCount 1 <p>IP 192.168.77.68 callerCount 1 </body></html>