Skip to content

Gateway API#

Ingress versus ingress

The word "Ingress," capitalized, refers to the Kubernetes Ingress resource, whereas "ingress" in lowercase refers to ingress functionality.

Starting with version 4.1.3, MKE 4k uses the Kubernetes Gateway API for management ingress. Envoy Gateway, a controller for Gateway API, provides this function. In line with this, management ingress will no longer use Kubernetes Ingress coupled with the NGINX Ingress Controller; however upgraded clusters will continue to have access to the NGINX Ingress Controller, and Ingress objects that are a part of user workloads will persist.

Note

MKE 4k always uses the Gateway API and Envoy Gateway controller to satisfy its ingress needs, regardless of any configuration choices. You can, however, continue to use Kubernetes Ingress with a supported controller, which can be NGINX Ingress Controller, either installed by you or sourced from MKE 4k.

Ingress support detail by deployment scenario#

The following table offers a high-level summary of the ingress scenarios, by origination.

Ingress factors Fresh installation Upgrade from MKE 3 Upgrade from previous MKE 4k
Envoy Gateway Install Envoy Gateway as a k0rdent addon, and as it is required to service management ingress, you cannot disable it. Install Envoy Gateway, either as BYO (bring your own) or as a k0rdent addon, to service management ingress. Install Envoy Gateway as a k0rdent addon. As it is required to service management ingress, you cannot disable it.
NGINX Ingress Controller Not available with any install configuration. If enabled in the source MKE 3 cluster, NGINX Ingress Controller will be available following the upgrade. By default, NGINX Ingress Controller is not available following the upgrade, though you can configure the upgrade to make it available.
Controllers Customers can install an ingress controller of their choice. Customers can stop using NGINX Ingress Controller, even from ingress-enabled MKE 3 clusters upon upgrade. Customers can continue using the ingress-nginx addon, and thereby the NGINX Ingress Controller, following the upgrade.

Fresh install of MKE 4k 4.1.3+#

Fresh installations of MKE 4k starting with version 4.1.3 use the Gateway API for management ingress by way of the Envoy Gateway controller. Unlike MKE 4k clusters that are the result of an upgrade operation, there is no provision to install NGINX Ingress Controller on fresh MKE 4k clusters.

Upgrade from MKE 3#

Following an upgrade from MKE 3 to MKE 4K, the NGINX Ingress Controller remains available by default. It is removed only if ingress was disabled in the source MKE 3 cluster or if you specify the --remove-ingress-nginx flag during the upgrade.

Envoy Gateway uses the HTTP and HTTPS node ports that were previously assigned to NGINX Ingress Controller. As such, to continue to use NGINX Ingress Controller, you must reconfigure the HTTP and HTTPS node ports it will use. To do this, include the following flags in the upgrade command:

--gateway-http-node-port - <node-port-for-gateway-ingress-controller-HTTP-use-post-upgrade>
--gateway-https-node-port - <node-port-for-gateway-ingress-controller-HTTPS-use-post-upgrade>

If you are not using 443 as your gateway listener port, append the number of your chosen port to the DNS name, and use the resulting value to specify the external address with the --external address flag.

Caution

The HTTP and HTTPS node ports that you configure for NGINX Ingress Controller must be different from the configured HTTP and HTTPS node ports for the MKE 3 cluster, as otherwise the upgrade will fail.

Management ingress on MKE 4k will always use Gateway API and not Kubernetes Ingress. If you choose to continue to use NGINX Ingress Controller, the configuration values under spec.ingressController will not be applicable for management ingress. Also, the values of the HTTP and HTTPS node ports under spec.ingressController will switch over to alternate values that you have specified.

Regarding the Envoy Gateway controller, the upgrade from MKE 3 to MKE 4k presents two options:

Option 1

Continue with a pre-existing Envoy Gateway controller installation that the customer will continue to manage.

To specify this, use both of the command line flags detailed below with the upgrade command:

Flag                                                             Purpose
--use-existing-envoy-gateway Indicates the use of an existing Envoy Gateway controller instance for management ingress.
--existing-envoy-gateway-class <GatewayClass-resource-name> Specifies an existing GatewayClass resource that is serviced by an instance of an existing Envoy Gateway controller installation that is managed by the customer.

As a result, MKE 4k will use the existing GatewayClass resource to create Gateway API CR-s and Envoy Gateway controller CR-s for its own use, all of them in MKE namespace. Starting with MKE 4k 4.1.3, the product supports Envoy Gateway controller versions v1.4.0 and later, with the condition that it must work with the particular version of distroless envoy image that is in use, which is docker.io/envoyproxy/envoy:distroless-v1.36.3.

Option 2

Use MKE 4k to install and manage the envoy-gateway addon. As this is the default behavior, this option does not require the inclusion of any particular flags when you run the upgrade command, and MKE 4k will as a result manage the Envoy Gateway controller.

Notes

  • MKE 4k does not support HTTP routes that use the Envoy Gateway controller for management routes. Such routes may be present in MKE 4k 4.1.3, however they may be removed in a future MKE 4k release.

  • MKE 4k 4.1.3 installs and uses Envoy Gateway controller v1.6.1, along with Gateway API CR-s v1.4.1.

  • MKE 4k 4.1.3 installs Envoy Gateway controller with the immutable controller name gateway.envoyproxy.io/mke-gatewayclass-controller.

  • You can use the GatewayClass mke-gateway-ingress, a cluster scoped resource that is required to service management ingress, to create your own Gateway API and envoy-gateway CR-s.

  • Do not create any Gateway API resources in the MKE namespace.

Upgrade from an earlier version of MKE 4k#

By default, when you upgrade from a previous version of MKE 4k the NGINX Ingress Controller does not persist. To override this behavior, run the upgrade flag with all three of the flags detailed below:

Flag                                                                     Purpose
--keep-ingress-nginx Ensures that the NGINX Ingress Controller is retained following upgrade.
--gateway-http-node-port Sets the HTTP node port for the Gateway API controller.
--gateway-https-node-port Sets the HTTPS node port for the Gateway API controller.
--gateway-listener-port Sets the gateway listener port for the Gateway controller. Ensure that your load balancer is configured to pass requests that reach this port to the port specified by --gateway-https-node-port.

Notes

  • MKE 4k does not support HTTP routes that use the Envoy Gateway controller for management routes.

  • MKE 4k 4.1.3 installs and uses Envoy Gateway controller v1.6.1, along with Gateway API CR-s v1.4.1.

  • MKE 4k 4.1.3 installs Envoy Gateway controller with the immutable controller name gateway.envoyproxy.io/mke-gatewayclass-controller. The GatewayClass[mke-gateway-ingress], which is a cluster scoped resource that is required to service management ingress, can be used by customers to create their own Gateway API and envoy-gateway CR-s.

Configuration#

Envoy Gateway controller#

MKE 4k supports the Envoy Gateway controller configurations that are detailed below:

Configuration                                               Description
spec.envoyGateway.replicaCount Indicates the number of replicas for the Envoy Gateway controller deployment. Default: 2.
spec.envoyGateway.tolerations Sets the tolerations that are used for the Envoy Gateway controller deployment. By default, MKE 4k adds appropriate tolerations for scheduling the Envoy Gateway controller deployment pods on controller nodes. If you specify your own tolerations, however, you must also include any that are require scheduling on the controller nodes.

Management ingress#

MKE 4k supports the management ingress configurations that are detailed below:

Configuration                                                             Description
spec.gatewayMKEIngress.nodePorts.HTTP HTTP node port for management ingress. Unsupported and can be removed at any time.
spec.gatewayMKEIngress.nodePorts.HTTPS HTTPS node port for management ingress. Ensure that this matches the target port of the MKE 4k Dashboard in your load balancer configuration.