Deploy a layer 7 routing solution

This topic describes how to route traffic to Swarm services by deploying a layer 7 routing solution into a Swarm-orchestrated cluster. It has the following prerequisites:


Enabling layer 7 routing causes the following to occur:

  1. MKE creates the ucp-interlock overlay network.

  2. MKE deploys the ucp-interlock service and attaches it both to the Docker socket and the overlay network that was created. This allows the Interlock service to use the Docker API, which is why this service needs to run on a manger node.

  3. The ucp-interlock service starts the ucp-interlock-extension service and attaches it to the ucp-interlock network, allowing both services to communicate.

  4. The ucp-interlock-extension generates a configuration for the proxy service to use. By default the proxy service is NGINX, so this service generates a standard NGINX configuration. MKE creates the com.docker.ucp.interlock.conf-1 configuration file and uses it to configure all the internal components of this service.

  5. The ucp-interlock service takes the proxy configuration and uses it to start the ucp-interlock-proxy service.

Note

Layer 7 routing is disabled by default.


To enable layer 7 routing using the MKE web UI:

  1. Log in to the MKE web UI as an administrator.

  2. Navigate to <user-name> > Admin Settings.

  3. Click Ingress.

  4. Toggle the Swarm HTTP ingress slider to the right.

  5. Optional. By default, the routing mesh service listens on port 8080 for HTTP and 8443 for HTTPS. Change these ports if you already have services using them.

The three primary Interlock services include the core service, the extensions, and the proxy. The following is the default MKE configuration, which is created automatically when you enable Interlock as described in this topic.

ListenAddr = ":8080"
DockerURL = "unix:///var/run/docker.sock"
AllowInsecure = false
PollInterval = "3s"

[Extensions]
  [Extensions.default]
    Image = "mirantis/ucp-interlock-extension:3.6.20"
    ServiceName = "ucp-interlock-extension"
    Args = []
    Constraints = ["node.labels.com.docker.ucp.orchestrator.swarm==true", "node.platform.os==linux"]
    ProxyImage = "mirantis/ucp-interlock-proxy:3.6.20"
    ProxyServiceName = "ucp-interlock-proxy"
    ProxyConfigPath = "/etc/nginx/nginx.conf"
    ProxyReplicas = 2
    ProxyStopSignal = "SIGQUIT"
    ProxyStopGracePeriod = "5s"
    ProxyConstraints = ["node.labels.com.docker.ucp.orchestrator.swarm==true", "node.platform.os==linux"]
    PublishMode = "ingress"
    PublishedPort = 8080
    TargetPort = 80
    PublishedSSLPort = 8443
    TargetSSLPort = 443
    [Extensions.default.Labels]
      "com.docker.ucp.InstanceID" = "fewho8k85kyc6iqypvvdh3ntm"
    [Extensions.default.ContainerLabels]
      "com.docker.ucp.InstanceID" = "fewho8k85kyc6iqypvvdh3ntm"
    [Extensions.default.ProxyLabels]
      "com.docker.ucp.InstanceID" = "fewho8k85kyc6iqypvvdh3ntm"
    [Extensions.default.ProxyContainerLabels]
      "com.docker.ucp.InstanceID" = "fewho8k85kyc6iqypvvdh3ntm"
    [Extensions.default.Config]
      Version = ""
      User = "nginx"
      PidPath = "/var/run/proxy.pid"
      MaxConnections = 1024
      ConnectTimeout = 5
      SendTimeout = 600
      ReadTimeout = 600
      IPHash = false
      AdminUser = ""
      AdminPass = ""
      SSLOpts = ""
      SSLDefaultDHParam = 1024
      SSLDefaultDHParamPath = ""
      SSLVerify = "required"
      WorkerProcesses = 1
      RLimitNoFile = 65535
      SSLCiphers = "HIGH:!aNULL:!MD5"
      SSLProtocols = "TLSv1.2"
      AccessLogPath = "/dev/stdout"
      ErrorLogPath = "/dev/stdout"
      MainLogFormat = "'$remote_addr - $remote_user [$time_local] \"$request\" '\n\t\t    '$status $body_bytes_sent \"$http_referer\" '\n\t\t    '\"$http_user_agent\" \"$http_x_forwarded_for\"';"
      TraceLogFormat = "'$remote_addr - $remote_user [$time_local] \"$request\" $status '\n\t\t    '$body_bytes_sent \"$http_referer\" \"$http_user_agent\" '\n\t\t    '\"$http_x_forwarded_for\" $request_id $msec $request_time '\n\t\t    '$upstream_connect_time $upstream_header_time $upstream_response_time';"
      KeepaliveTimeout = "75s"
      ClientMaxBodySize = "32m"
      ClientBodyBufferSize = "8k"
      ClientHeaderBufferSize = "1k"
      LargeClientHeaderBuffers = "4 8k"
      ClientBodyTimeout = "60s"
      UnderscoresInHeaders = false
      HideInfoHeaders = false

Note

The value of LargeClientHeaderBuffers indicates the number of buffers to use to read a large client request header, as well as the size of those buffers.


To enable layer 7 routing from the command line:

Interlock uses a TOML file for the core service configuration. The following example uses Swarm deployment and recovery features by creating a Docker config object.

  1. Create a Docker config object:

    cat << EOF | docker config create service.interlock.conf -
ListenAddr = ":8080"
DockerURL = "unix:///var/run/docker.sock"
PollInterval = "3s"

[Extensions]
  [Extensions.default]
    Image = "mirantis/ucp-interlock-extension:3.6.20"
    Args = ["-D"]
    ProxyImage = "mirantis/ucp-interlock-proxy:3.6.20"
    ProxyArgs = []
    ProxyConfigPath = "/etc/nginx/nginx.conf"
    ProxyReplicas = 1
    ProxyStopGracePeriod = "3s"
    ServiceCluster = ""
    PublishMode = "ingress"
    PublishedPort = 8080
    TargetPort = 80
    PublishedSSLPort = 8443
    TargetSSLPort = 443
    [Extensions.default.Config]
      User = "nginx"
      PidPath = "/var/run/proxy.pid"
      WorkerProcesses = 1
      RlimitNoFile = 65535
      MaxConnections = 2048
EOF
oqkvv1asncf6p2axhx41vylgt

  2. Create a dedicated network for Interlock and the extensions:

    docker network create --driver overlay ucp-interlock

  3. Create the Interlock service:

    docker service create \
--name ucp-interlock \
--mount src=/var/run/docker.sock,dst=/var/run/docker.sock,type=bind \
--network ucp-interlock \
--constraint node.role==manager \
--config src=service.interlock.conf,target=/config.toml \
mirantis/ucp-interlock:3.6.20 -D run -c /config.toml

    Note

    The Interlock core service must have access to a Swarm manager (--constraint node.role==manager), however the extension and proxy services are recommended to run on workers.

  4. Verify that the three services are created, one for the Interlock service, one for the extension service, and one for the proxy service:

    docker service ls
ID                  NAME                     MODE                REPLICAS            IMAGE                                                                PORTS
sjpgq7h621ex        ucp-interlock            replicated          1/1                 mirantis/ucp-interlock:3.6.20
oxjvqc6gxf91        ucp-interlock-extension  replicated          1/1                 mirantis/ucp-interlock-extension:3.6.20
lheajcskcbby        ucp-interlock-proxy      replicated          1/1                 mirantis/ucp-interlock-proxy:3.6.20        *:80->80/tcp *:443->443/tcp