MKE configuration management

This section describes configuration specifics of an MKE cluster deployed using Container Cloud.

MKE configuration managed by Container Cloud

Since 2.25.1, Container Cloud does not override changes in MKE configuration except the following list of parameters that are automatically managed by Container Cloud. These parameters are always overridden by the Container Cloud default values if modified direclty using the MKE API. For details on configuration using the MKE API, see MKE configuration managed directly by the MKE API.

However, you can manually configure a few options from this list using the Cluster object of a Container Cloud cluster. They are labeled with the superscript and contain references to the respective configuration procedures in the Comments columns of the tables.

[audit_log_configuration]

MKE parameter name

Default value in Container Cloud

Comments

level

"metadata"

You can enable this option either using MKE API with no Container Cloud overrides or using the Cluster object of a Container Cloud cluster. For details, see Configure Kubernetes auditing and profiling and MKE documentation: MKE audit logging.

If enabled using the Cluster object, use the same object to disable the option. Otherwise, it will be overridden by Container Cloud.

support_bundle_include_audit_logs

false

See comments above.

[auth]

MKE parameter name

Default value in Container Cloud

default_new_user_role

"restrictedcontrol"

backend

"managed"

samlEnabled

false

managedPasswordDisabled

false

[auth.external_identity_provider]

MKE parameter name

Default value in Container Cloud

issuer

"https://<Keycloak-external-address>/auth/realms/iam"

userServiceId

"<userServiceId>"

clientId

"kaas"

wellKnownConfigUrl

"https://<Keycloak-external-address>/auth/realms/iam/.well-known/openid-configuration"

caBundle

"<caCert>"

usernameClaim

""

httpProxy

""

httpsProxy

""

[hardening_configuration]

MKE parameter name

Default value in Container Cloud

hardening_enabled

true

limit_kernel_capabilities

true

pids_limit_int

100000

pids_limit_k8s

100000

pids_limit_swarm

100000

[scheduling_configuration]

MKE parameter name

Default value in Container Cloud

enable_admin_ucp_scheduling

true

default_node_orchestrator

kubernetes

[tracking_configuration]

MKE parameter name

Default value in Container Cloud

cluster_label

"prod"

[cluster_config]

MKE parameter name

Default value in Container Cloud

Comments

calico_ip_auto_method

  • Bare metal: interface=k8s-pods

  • OpenStack, vSphere: ""

calico_mtu

"1440"

For configuration steps, see Set the MTU size for Calico.

calico_vxlan

true

calico_vxlan_mtu

"1440"

calico_vxlan_port

"4792"

cloud_provider

  • Bare metal: ""

  • OpenStack, vSphere: external

  • vSphere before 2.25.1: vsphere

Depends on the selected cloud provider.

controller_port

  • Bare metal, vSphere: 4443

  • OpenStack: 6443

custom_kube_api_server_flags

["--event-ttl=720h"]

Applies only to MKE on the management cluster.

custom_kube_controller_manager_flags

  • ["--leader-elect-lease-duration=120s", "--leader-elect-renew-deadline=60s"]

  • ["--feature-gates=CSIMigrationvSphere=true"] 0

custom_kube_scheduler_flags

["--leader-elect-lease-duration=120s", "--leader-elect-renew-deadline=60s"]

custom_kubelet_flags

  • ["--serialize-image-pulls=false"]

  • ["--feature-gates=CSIMigrationvSphere=true"] 0

etcd_storage_quota

""

For configuration steps, see Increase storage quota for etcd.

exclude_server_identity_headers

true

ipip_mtu

"1440"

kube_api_server_auditing

true

For configuration steps, see Configure Kubernetes auditing and profiling.

kube_api_server_profiling_enabled

false

For configuration steps, see Configure Kubernetes auditing and profiling.

kube_apiserver_port

  • Bare metal, vSphere: 5443

  • OpenStack: 443

kube_protect_kernel_defaults

true

local_volume_collection_mapping

false

manager_kube_reserved_resources

"cpu=1000m,memory=2Gi,ephemeral-storage=4Gi"

metrics_retention_time

"24h"

metrics_scrape_interval

"1m"

nodeport_range

"30000-32768"

pod_cidr

"10.233.64.0/18"

You can override this value in spec::clusterNetwork::pods::cidrBlocks: of the Cluster object.

priv_attributes_allowed_for_service_accounts 1

["hostBindMounts", "hostIPC", "hostNetwork", "hostPID", "kernelCapabilities", "privileged"]

priv_attributes_priv_attributes_service_accounts 1

["kube-system:helm-controller-sa", "kube-system:pod-garbage-collector", "stacklight:stacklight-helm-controller"]service_accounts

profiling_enabled

false

prometheus_memory_limit

"4Gi"

prometheus_memory_request

"2Gi"

secure_overlay

true

service_cluster_ip_range

"10.233.0.0/18"

You can override this value in spec::clusterNetwork::services::cidrBlocks: of the Cluster object.

swarm_port

2376

swarm_strategy

"spread"

unmanaged_cni

false

vxlan_vni

10000

worker_kube_reserved_resources

"cpu=100m,memory=300Mi,ephemeral-storage=500Mi"

0(1,2)

The CSIMigrationvSphere flag applies only to the vSphere provider since 2.25.1.

1(1,2)

For priv_attributes parameters, you can add custom options on top of existing parameters using the MKE API.

Note

All possible values for parameters labeled with the superscript, which you can manually configure using the Cluster object are described in MKE Operations Guide: Configuration options.

MKE configuration managed directly by the MKE API

Since 2.25.1, aside from MKE parameters described in MKE configuration managed by Container Cloud, Container Cloud does not override changes in MKE configuration that are applied directly through the MKE API. For the configuration options and procedure, see MKE documentation:

  • MKE configuration options

  • Configure an existing MKE cluster

    While using this procedure, replace the command to upload the newly edited MKE configuration file with the following one:

    curl --silent --insecure -X PUT -H "X-UCP-Allow-Restricted-API: i-solemnly-swear-i-am-up-to-no-good" -H "accept: application/toml" -H "Authorization: Bearer $AUTHTOKEN" --upload-file 'mke-config.toml' https://$MKE_HOST/api/ucp/config-toml
    

Important

Mirantis cannot guarrantee the expected behavior of the functionality configured using the MKE API as long as customer-specific configuration does not undergo testing within Container Cloud. Therefore, Mirantis recommends that you test custom MKE settings configured through the MKE API on a staging environment before applying them to production.