Use Kubernetes on Windows Server nodes

Starting with version 3.3.0, MKE now supports the Kubernetes orchestrator on Windows Server nodes.

Prerequisites

  1. Install MKE

  2. Create a linux-only cluster

Note

The Kubernetes orchestrator on Windows Server nodes is not supported on clusters upgraded from previous versions of MKE. To deploy the Kubernetes orchestrator on Windows Server nodes you will need to deploy a new cluster running MKE 3.3.0 and higher.

Adding Windows Server nodes

Having installed a one-node Linux-only MKE cluster, we are now ready to add additional Windows workers to this cluster using these steps:

  1. Use your browser on your local system to log into the MKE installation above.

  2. Navigate to the nodes list and click on Add Node at the top right of the page.

  3. In the Add Node page select “Windows” as the node type. You will notice that Windows Server nodes are only allowed to have worker roles.

  4. Optionally, you may also select and set custom listen and advertise addresses.

  5. A command line will be generated that includes a join-token. It should look something like:

    docker swarm join ... --token <join-token> ...
    

    Copy this command.

  6. Add your Windows Server node to the MKE cluster by running the swarm-join commandline generated above.

Validating the cluster setup

  1. You can use either the MKE web interface or the command line to view your clusters.

    • To use the web interface, log into MKE, and navigate to the node list view. All nodes should be green.

    • Check the node status using this command on your local system:

      kubectl get nodes
      

    Your nodes should all have a status value of “Ready”, as in the following example.

    NAME                   STATUS   ROLES    AGE     VERSION
    user-135716-win-0      Ready    <none>   2m16s   v1.17.2
    user-7d985f-ubuntu-0   Ready    master   4m55s   v1.17.2-docker-d-2
    user-135716-win-1      Ready    <none>   1m12s   v1.17.2
    
  1. Now that you have confirmed that the nodes are ready, the next step is to change the orchestrator to kubernetes for the pods. You can change the orchestrator using the MKE web interface, either by changing the default orchestrator in the Administrator settings, or by using the web interface to toggle the orchestrator after joining a node. The equivalent CLI command is:

    docker node update <node name> --label-add com.docker.ucp.orchestrator.kubernetes=true
    
  2. Optionally, you can deploy a workload on the cluster to make sure everything is working as expected.

Troubleshooting

If you can’t join your Windows Server node to the cluster, confirm that the correct processes are running on the node.

PS C:\> Get-Process tigera-calico

You should see something like this.

Handles  NPM(K)    PM(K)      WS(K)     CPU(s)     Id  SI ProcessName
-------  ------    -----      -----     ------     --  -- -----------
   276      17    33284      40948      39.89   8132   0 tigera-calico

The next troubleshooting step is to check the kubelet process.

PS C:\> Get-Process kubelet

You should see something like this.

Handles  NPM(K)    PM(K)      WS(K)     CPU(s)     Id  SI ProcessName
-------  ------    -----      -----     ------     --  -- -----------
   524      23    47332      73380     828.50   6520   0 kubelet

After that check the kube-proxy process.

PS C:\> Get-Process kube-proxy

You should see something like this.

Handles  NPM(K)    PM(K)      WS(K)     CPU(s)     Id  SI ProcessName
-------  ------    -----      -----     ------     --  -- -----------
   322      19    25464      33488      21.00   7852   0 kube-proxy

If any of the process checks show that something isn’t working, the next step is to check the logs. For kubelet and kubeproxy, the logs are placed under C:klogs. Tigera/Calico logs are placed under C:TigeraCalicologs.

An example workload on Windows Server

This section captures an example workload that serves to illustrate Kubernetes on Windows Server capabilities. The following procedure deploys a complete web application on IIS servers as Kubernetes Services. The example workload includes an MSSQL database and a loadbalancer.

Specifically, the procedure covers:

  • Namespace creation

  • Pod and Deployment scheduling

  • Kubernetes service provisioning

  • Addition of application workloads

  • Connectivity of Pods, Nodes and Services

  1. Create a Namespace.

    $ kubectl create -f demo-namespace.yaml
    
    # demo-namespace.yaml
    apiVersion: v1
    kind: Namespace
    metadata:
    name: demo
    
  2. Create a web service as a Kubernetes service.

    $ kubectl create -f win-webserver.yaml
    service/win-webserver created
    deployment.apps/win-webserver created
    
    $ kubectl get service --namespace demo
    NAME            TYPE       CLUSTER-IP    EXTERNAL-IP   PORT(S)        AGE
    win-webserver   NodePort   10.96.29.12   <none>        80:35048/TCP   12m
    
    # win-webserver.yaml
    apiVersion: v1
    kind: Service
    metadata:
    name: win-webserver
    namespace: demo
    labels:
       app: win-webserver
    spec:
    ports:
       # the port that this service should serve on
       - port: 80
          targetPort: 80
    selector:
       app: win-webserver
    type: NodePort
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
    labels:
       app: win-webserver
       namespace: demo
    name: win-webserver
    namespace: demo
    spec:
    replicas: 2
    selector:
       matchLabels:
          app: win-webserver
    template:
       metadata:
          labels:
          app: win-webserver
          name: win-webserver
       spec:
          affinity:
          podAntiAffinity:
             requiredDuringSchedulingIgnoredDuringExecution:
                - labelSelector:
                   matchExpressions:
                      - key: app
                      operator: In
                      values:
                         - win-webserver
                topologyKey: "kubernetes.io/hostname"
          containers:
          - name: windowswebserver
             image: mcr.microsoft.com/windows/servercore:ltsc2019
             command:
                - powershell.exe
                - -command
                - "<#code used from https://gist.github.com/wagnerandrade/5424431#> ; $$listener = New-Object System.Net.HttpListener ; $$listener.Prefixes.Add('http://*:80/') ; $$listener.Start() ; $$callerCounts = @{} ; Write-Host('Listening at http://*:80/') ; while ($$listener.IsListening) { ;$$context = $$listener.GetContext() ;$$requestUrl = $$context.Request.Url ;$$clientIP = $$context.Request.RemoteEndPoint.Address ;$$response = $$context.Response ;Write-Host '' ;Write-Host('> {0}' -f $$requestUrl) ;  ;$$count = 1 ;$$k=$$callerCounts.Get_Item($$clientIP) ;if ($$k -ne $$null) { $$count += $$k } ;$$callerCounts.Set_Item($$clientIP, $$count) ;$$ip=(Get-NetAdapter | Get-NetIpAddress); $$header='<html><body><H1>Windows Container Web Server</H1>' ;$$callerCountsString='' ;$$callerCounts.Keys | % { $$callerCountsString+='<p>IP {0} callerCount {1} ' -f $$ip[1].IPAddress,$$callerCounts.Item($$_) } ;$$footer='</body></html>' ;$$content='{0}{1}{2}' -f $$header,$$callerCountsString,$$footer ;Write-Output $$content ;$$buffer = [System.Text.Encoding]::UTF8.GetBytes($$content) ;$$response.ContentLength64 = $$buffer.Length ;$$response.OutputStream.Write($$buffer, 0, $$buffer.Length) ;$$response.Close() ;$$responseStatus = $$response.StatusCode ;Write-Host('< {0}' -f $$responseStatus)  } ; "
          nodeSelector:
          beta.kubernetes.io/os: windows
    
  3. Check the pods deployed on different Windows Server worker nodes with Inter-pod affinity and anti-affinity.

    $ kubectl get pod --namespace demo
    
    NAME                            READY   STATUS    RESTARTS   AGE
    win-webserver-8c5678c68-qggzh   1/1     Running   0          6m21s
    win-webserver-8c5678c68-v8p84   1/1     Running   0          6m21s
    
    # Check the detailed status of pods deployed
    $ kubectl describe pod win-webserver-8c5678c68-qggzh --namespace demo
    
  4. Access the web service by node-to-pod communication across the network.

    From a kubectl client:

    $ kubectl get pods --namespace demo -o wide
    
    NAME                            READY   STATUS    RESTARTS   AGE   IP              NODE              NOMINATED NODE   READINESS GATES
    win-webserver-8c5678c68-qggzh   1/1     Running   0          16m   192.168.77.68   user-135716-win-1 <none>           <none>
    win-webserver-8c5678c68-v8p84   1/1     Running   0          16m   192.168.4.206   user-135716-win-0 <none>           <none>
    
    $ ssh -o ServerAliveInterval=15 root@<master-node>
    
    $ curl 10.96.29.12
    <html><body><H1>Windows Container Web Server</H1><p>IP 192.168.77.68 callerCount 1 </body></html>
    $
    

    Run the curl command a second time. You can see the second request load-balanced to a different pod.

    $ curl 10.96.29.12
    <html><body><H1>Windows Container Web Server</H1><p>IP 192.168.4.206 callerCount 1 </body></html>
    
  5. Access the web service by pod-to-pod communication across the network.

    $ kubectl get service --namespace demo
    
    NAME            TYPE       CLUSTER-IP    EXTERNAL-IP   PORT(S)        AGE
    win-webserver   NodePort   10.96.29.12   <none>        80:35048/TCP   12m
    
    $ kubectl get pods --namespace demo -o wide
    
    NAME                            READY   STATUS    RESTARTS   AGE   IP              NODE              NOMINATED NODE   READINESS GATES
    win-webserver-8c5678c68-qggzh   1/1     Running   0          16m   192.168.77.68   user-135716-win-1 <none>           <none>
    win-webserver-8c5678c68-v8p84   1/1     Running   0          16m   192.168.4.206   user-135716-win-0 <none>           <none>
    
    $ kubectl exec -it win-webserver-8c5678c68-qggzh --namespace demo cmd
    
    Microsoft Windows [Version 10.0.17763.1098]
    (c) 2018 Microsoft Corporation. All rights reserved.
    
    C:\>curl 10.96.29.12
    <html><body><H1>Windows Container Web Server</H1><p>IP 192.168.77.68
    callerCount 1 <p>IP 192.168.77.68 callerCount 1 </body></html>
    

See also

Kubernetes