MKE configuration management¶

This section describes configuration specifics of an MKE cluster deployed using Container Cloud.

MKE configuration managed by Container Cloud¶

Since 2.25.1 (Cluster releases 16.0.1 and 17.0.1), Container Cloud does not override changes in MKE configuration except the following list of parameters that are automatically managed by Container Cloud. These parameters are always overridden by the Container Cloud default values if modified direclty using the MKE API. For details on configuration using the MKE API, see MKE configuration managed directly by the MKE API.

However, you can manually configure a few options from this list using the Cluster object of a Container Cloud cluster. They are labeled with the superscript and contain references to the respective configuration procedures in the Comments columns of the tables.

[audit_log_configuration]¶

MKE parameter name	Default value in Container Cloud	Comments
`level`	`"metadata"` 0 `""` 1	You can configure this option either using MKE API with no Container Cloud overrides or using the `Cluster` object of a Container Cloud cluster. For details, see Configure Kubernetes auditing and profiling and MKE documentation: MKE audit logging. If configured using the `Cluster` object, use the same object to disable the option. Otherwise, it will be overridden by Container Cloud.
`support_bundle_include_audit_logs`	`false`	For configuration procedure, see comments above.

MKE parameter name

Default value in Container Cloud

Comments

level

"metadata" 0

"" 1

You can configure this option either using MKE API with no Container Cloud overrides or using the Cluster object of a Container Cloud cluster. For details, see Configure Kubernetes auditing and profiling and MKE documentation: MKE audit logging.

If configured using the Cluster object, use the same object to disable the option. Otherwise, it will be overridden by Container Cloud.

support_bundle_include_audit_logs

false

For configuration procedure, see comments above.

0: For management clusters since 2.26.0 (Cluster release 16.1.0)
1: For management and managed clusters since 2.24.3 (Cluster releases 15.0.2 and 14.0.2)

[auth]¶

MKE parameter name	Default value in Container Cloud
`default_new_user_role`	`"restrictedcontrol"`
`backend`	`"managed"`
`samlEnabled`	`false`
`managedPasswordDisabled`	`false`

[auth.external_identity_provider]¶

MKE parameter name	Default value in Container Cloud
`issuer`	`"https://<Keycloak-external-address>/auth/realms/iam"`
`userServiceId`	`"<userServiceId>"`
`clientId`	`"kaas"`
`wellKnownConfigUrl`	`"https://<Keycloak-external-address>/auth/realms/iam/.well-known/openid-configuration"`
`caBundle`	`"<caCert>"`
`usernameClaim`	`""`
`httpProxy`	`""`
`httpsProxy`	`""`

[hardening_configuration]¶

MKE parameter name	Default value in Container Cloud
`hardening_enabled`	`true`
`limit_kernel_capabilities`	`true`
`pids_limit_int`	`100000`
`pids_limit_k8s`	`100000`
`pids_limit_swarm`	`100000`

[scheduling_configuration]¶

MKE parameter name	Default value in Container Cloud
`enable_admin_ucp_scheduling`	`true`
`default_node_orchestrator`	`kubernetes`

[tracking_configuration]¶

MKE parameter name	Default value in Container Cloud
`cluster_label`	`"prod"`

[cluster_config]¶

Caution

Since Container Cloud 2.27.3 (Cluster release 16.2.3), support for vSphere-based clusters is suspended. For details, see Deprecation notes.

MKE parameter name	Default value in Container Cloud	Comments
`calico_ip_auto_method`	Bare metal: `interface=k8s-pods` OpenStack, vSphere: `""`
`calico_mtu`	`"1440"`	For configuration steps, see Set the MTU size for Calico.
`calico_vxlan`	`true`
`calico_vxlan_mtu`	`"1440"`
`calico_vxlan_port`	`"4792"`
`cloud_provider`	Bare metal: `""` OpenStack, vSphere: `external` vSphere before 2.25.1: `vsphere`	Depends on the selected cloud provider.
`controller_port`	Bare metal, vSphere: `4443` OpenStack: `6443`
`custom_kube_api_server_flags`	`["--event-ttl=720h"]`	Applies only to MKE on the management cluster.
`custom_kube_controller_manager_flags`	`["--leader-elect-lease-duration=120s", "--leader-elect-renew-deadline=60s"]` `["--feature-gates=CSIMigrationvSphere=true"]` 2
`custom_kube_scheduler_flags`	`["--leader-elect-lease-duration=120s", "--leader-elect-renew-deadline=60s"]`
`custom_kubelet_flags`	`["--serialize-image-pulls=false"]` `["--feature-gates=CSIMigrationvSphere=true"]` 2
`etcd_storage_quota`	`""`	For configuration steps, see Increase storage quota for etcd.
`exclude_server_identity_headers`	`true`
`ipip_mtu`	`"1440"`
`kube_api_server_auditing`	`true` 4 `false` 5	For configuration steps, see Configure Kubernetes auditing and profiling.
`kube_api_server_audit_log_maxage` 6	`30`
`kube_api_server_audit_log_maxbackup` 6	`10`
`kube_api_server_audit_log_maxsize` 6	`10`
`kube_api_server_profiling_enabled`	`false`	For configuration steps, see Configure Kubernetes auditing and profiling.
`kube_apiserver_port`	Bare metal, vSphere: `5443` OpenStack: `443`
`kube_protect_kernel_defaults`	`true`
`local_volume_collection_mapping`	`false`
`manager_kube_reserved_resources`	`"cpu=1000m,memory=2Gi,ephemeral-storage=4Gi"`
`metrics_retention_time`	`"24h"`
`metrics_scrape_interval`	`"1m"`
`nodeport_range`	`"30000-32768"`
`pod_cidr`	`"10.233.64.0/18"`	You can override this value in `spec::clusterNetwork::pods::cidrBlocks:` of the `Cluster` object.
`priv_attributes_allowed_for_service_accounts` 3	`["hostBindMounts", "hostIPC", "hostNetwork", "hostPID", "kernelCapabilities", "privileged"]`
`priv_attributes_priv_attributes_service_accounts` 3	`["kube-system:helm-controller-sa", "kube-system:pod-garbage-collector", "stacklight:stacklight-helm-controller"]service_accounts`
`profiling_enabled`	`false`
`prometheus_memory_limit`	`"4Gi"`
`prometheus_memory_request`	`"2Gi"`
`secure_overlay`	`true`
`service_cluster_ip_range`	`"10.233.0.0/18"`	You can override this value in `spec::clusterNetwork::services::cidrBlocks:` of the `Cluster` object.
`swarm_port`	`2376`
`swarm_strategy`	`"spread"`
`unmanaged_cni`	`false`
`vxlan_vni`	`10000`
`worker_kube_reserved_resources`	`"cpu=100m,memory=300Mi,ephemeral-storage=500Mi"`

2(1,2): The CSIMigrationvSphere flag applies only to the vSphere provider since 2.25.1.
3(1,2): For priv_attributes parameters, you can add custom options on top of existing parameters using the MKE API.
4: For management clusters since 2.26.0 (Cluster release 16.1.0).
5: For management and managed clusters since 2.24.3 (Cluster releases 15.0.2 and 14.0.2).
6(1,2,3): For management and managed clusters since 2.27.0 (Cluster releases 17.2.0 and 16.2.0). For configuration steps, see Configure Kubernetes auditing and profiling.

Note

All possible values for parameters labeled with the superscript, which you can manually configure using the Cluster object are described in MKE Operations Guide: Configuration options.

MKE configuration managed directly by the MKE API¶

Since 2.25.1, aside from MKE parameters described in MKE configuration managed by Container Cloud, Container Cloud does not override changes in MKE configuration that are applied directly through the MKE API. For the configuration options and procedure, see MKE documentation:

MKE configuration options

Configure an existing MKE cluster

While using this procedure, replace the command to upload the newly edited MKE configuration file with the following one:

curl --silent --insecure -X PUT -H "X-UCP-Allow-Restricted-API: i-solemnly-swear-i-am-up-to-no-good" -H "accept: application/toml" -H "Authorization: Bearer $AUTHTOKEN" --upload-file 'mke-config.toml' https://$MKE_HOST/api/ucp/config-toml

Important

Mirantis cannot guarrantee the expected behavior of the functionality configured using the MKE API as long as customer-specific configuration does not undergo testing within Container Cloud. Therefore, Mirantis recommends that you test custom MKE settings configured through the MKE API on a staging environment before applying them to production.